Global food production must increase by 50-70% to feed world populations expected to increase from 8 billion to 10 billion by 2050. At the same time, the agriculture industry is facing seismic shifts as farmers and ranchers contend with multiple challenges from a shrinking labor pool to climate change, which affects water resources, growing seasons and migration patterns. Fortunately, advances in automation and robotics technology are enabling more precision in the field — from automated planting to real-time data on yields and soil conditions.

Southwest Research Institute has a long history of supporting the agriculture sector through research and development of technologies such as microencapsulation to control the release of agrochemicals and conducting chemical analyses of soil. More recently, SwRI’s Intelligent Systems Division has invested in advancing autonomous systems that are increasingly being adapted for farming, vineyard management and other agricultural applications across the country.

Building on decades of experience in off-road perception and automation, SwRI is now advancing smart agriculture by adapting technologies originally developed for rugged military operations. Robust sensor fusion, computer vision and machine learning techniques enable autonomous systems to identify crops and obstacles while navigating uneven terrain and operating safely around workers.

These systems support tasks such as weed mitigation, automated harvesting and continuous monitoring of crop health and soil conditions, all aimed at reducing labor demands and improving resource efficiency. By tailoring its perception and autonomy solutions to the unique challenges of farms and ranches, SwRI is helping accelerate the transition to data-driven, resilient agricultural operations.

Machine Vision & Perception

Agricultural producers and farm managers have traditionally relied on manual operations in the fields. However, soil inspection, harvesting and pest or crop management are becoming increasingly difficult due to supply shortages, labor restrictions and changing environmental conditions. Smart agriculture technologies that use machine vision and perception are pivotal to overcoming these challenges. The new technology supports advanced operations in row crops, foraging and other production areas. SwRI offers deep expertise using a full array of visual technology deployed using aerial drones, ground vehicles and remote sensing.

DETAIL

A windrow is a long, narrow row of hay or grain left in a field to dry, or a similar pile of material like snow, leaves or compost formed by wind, machinery or plows. Farmers usually rake hay in the same direction it was mowed or in a counter-clockwise loop around the field to help heavy stems dry and facilitate baling.

In the forage industry, timing and precision can make the difference between producing premium-grade feed or experiencing reduced yields. Before hay can be rolled into round bales, it must be cut in long, uniform windrows and air-dried in the sun. After cutting and raking, tractor operators pull baling equipment, maintaining precise alignment along windrows for quality harvesting. Once bales are formed, they must be quickly located, collected and cleared from the field before the weather changes, a challenge for large acreage or hilly terrain. Smart perception systems and machine vision can help equipment “see” windrows and bales and automatically guide tractors and balers along the optimal path. Identifying bales and mapping their locations for efficient collection reduces waste, operator fatigue and time spent in the field.

A man outfitting agricultural tractors with machine vision.

Outfitting agricultural tractors with machine vision allows detailed mapping of the environment, capturing row geometry, obstacles and crop locations to support advanced autonomous planning.

A visualization of highlighted rows in a photo of drying hay for bailing operations

SwRI developed intelligent systems to visualize rows of drying hay and automate baling operations.

DETAIL

Farm implements are equipment typically attached to a tractor, such as plows, mowers, and harrows.

In May 2025, SwRI earned Patent No. 12,306,318 for its Synchronized Windrow Intelligent Path Estimator (SWIPE) used for hay baling equipment. This system combines advanced sensing with embedded intelligence to actively predict and lock onto the optimal path along a windrow. As it moves, the navigation-assisted technology not only tracks vehicle position and speed, but also continuously builds a rich profile of the windrow, capturing its size, density and direction and enabling more consistent bale formation with less operator effort.

Autonomously clearing a field of round bales introduced a new set of perception challenges, requiring detection of each bale’s position and orientation with less than 5% error. Uneven and rutted terrain, variations in bale size and shape and fluctuating lighting over the course of a day made robust detection challenging. SwRI worked with a client to address these issues and integrated a tailored perception solution into their custom autonomous system to reliably locate, approach and collect bales across the field. Integrating advanced perception with intelligent machine learning allows the system to map each bale for collection while accurately assessing its orientation. This solution allowed consistent bale pickup, efficient routing and faster field clearing under real-world operating conditions. The technology helped a client win the award for Best Field Robot in the 2021 International Forum for Agricultural Robotics Future Farming category.

Image
A graphic showing a dual-camera visualization system that's observing plants in three dimensions.

SwRI’s dual-camera visualization system observes plants in three dimensions, providing a richer dataset identifying weeds while assessing the health of the crop.

Row farming grows plants in straight, evenly spaced lines wide enough to let machinery or tools pass between them for tilling, weeding and harvesting. Weeds emerging alongside high-value crops and competing for the same limited resources present a challenge in these fields. Uniformity and quality are crucial for commercial crops, so even modest weed intrusion can reduce yields, increase costs and complicate harvest operations. Producers rely on a mix of mechanical tools, herbicides and lasers or scorching methods to manage weeds, often under tight timing windows, because interventions are most effective early in the growth cycle. At the same time, labor shortages, chemical regulations and requirements for consistent, documented practices are driving the industry toward more automated, data-driven solutions. Within this context, intelligent vision-based systems that can detect, classify and localize plant material in real time offer a path to more precise, efficient and scalable weed management in modern crop row production. These challenges create a clear need for intelligent perception technology that can “see” the crop row in detail, distinguish crops from weeds in real time and guide implements to act precisely where intervention is needed.

Two photos of advanced dual-view-machine vision technology that can see the crop row in detail.

SwRI’s advanced dual-view machine vision technology can “see” the crop row in detail, distinguish crops from weeds in real time and guide implements to act precisely where intervention is needed.

SwRI developed an intelligent perception system for crop row automation that uses a dual-camera configuration and advanced machine vision to support weed management in specialty crops. The system employs two synchronized cameras, a top-down view and an angled-side view, mounted on a client’s weeding implement for each row. Together, these views feed neural networks and perception algorithms to simultaneously identify crop and weed plants and extract rich geometric features from the canopy. Unlike single-view approaches, this dual-view system provides a more complete understanding of plant structure and context within the row, enabling both precise weed targeting and deeper insights into crop condition.

DETAIL

Spatial analysis examines the locations, attributes and relationships of features in geographic data to uncover patterns and make informed decisions.

SwRI’s solution observes crops in three dimensions, analyzing features such as plant height, spacing and apparent vigor, while still reliably detecting weeds. By combining these perspectives, the perception system can inform not only about where to remove weeds, but also about crop uniformity, potential stress and overall quality indicators. This richer data opens the door to more sophisticated strategies, such as adjusting weeding intensity based on crop vigor or flagging areas of the field that may require additional attention. The perception system is designed to integrate directly with client implements that apply multiple weed mitigation strategies.

By combining advanced machine vision, dual-view sensing, and adaptable integration with multiple weed mitigation strategies, the SwRI-developed perception system helps growers automate critical tasks in transplant crops. Ultimately, this technology supports more efficient, sustainable crop production by turning visual information into actionable guidance for modern, automated weeding implements.

Navigation, Geolocation & Mapping

For specialty farms such as orchards and vineyards, automation must operate in tight, unforgiving spaces where mistakes are costly. Equipment moves through narrow rows bordered by delicate vines, trellises or tree trunks, while the terrain may disguise hazards like gopher holes, ruts and uneven areas. Robust navigation is crucial for operating in the row, avoiding contact that can damage plants or infrastructure, and maintaining safe, repeatable paths. Detailed mapping of the environment — capturing row geometry, obstacles and crop locations — supports advanced autonomous planning, allowing systems to optimize routes and adjust to changing field conditions. Together, reliable navigation and high-fidelity geolocation and mapping form the foundation for practical, scalable automation to produce high-value specialty crops.

Advanced autonomous planning imaging on grape vineyards.

High-value specialty crops such as grapes require systems to operate in tight spaces where mistakes are costly. Advanced autonomous planning allows systems to optimize routes and adjust to changing field conditions.

Spatial analysis is vital for enabling intelligent tractors and field equipment to autonomously navigate specialty crops. SwRI developed the Intelligent Perception System for Agricultural Automation (IPSAA) to produce rich maps and gather data, providing information on daily and weekly growth. The system also supports integrated safety to detect pedestrians in the field while maintaining system guidance.

IPSAA fuses advanced sensor data from stereo cameras, lidar and GPS systems with data processing and environment modeling to provide off-road agricultural vehicles with a precise understanding of their surroundings and location. The system processes sensor data into depth, terrain and object information, then leverages GPS and visual odometry to fuse these input data into a robust localization solution to build feature-rich maps.

DETAIL

Lidar, which stands for light detection and ranging, is a remote sensing method that uses pulsed laser light to create precise, 3D maps of objects and the environment.

IPSAA supports navigation, obstacle awareness and data collection tasks crucial for automating operations for specialty crops, where tight rows, delicate plants and complex layouts demand accurate perception and positioning. By globally registering data over multiple passes, IPSAA can capture detailed canopy structures, trellis locations and row geometry, enabling repeatable mapping across the growing season. Comparing these time‑sequenced maps can support canopy management, yield estimation and evaluation of crop health trends. In addition, the high‑fidelity environment representation improves precision navigation and obstacle avoidance within the rows, providing a foundation for targeted interventions and more efficient, data‑driven specialty crop operations.

Two images side by side showing a photo of a vineyard in the first one, and the intelligent perception system scan of the vineyard in the second one.

SwRI developed the Intelligent Perception System for Agricultural Automation for vineyard operations, but the system can be applied to any farming application requiring navigation, obstacle awareness and data collection.

Internet of Things

Connecting farm equipment to networks, machine learning algorithms and distributed sensors is rarely straightforward, especially in fields with limited connectivity and harsh environments. Agricultural equipment operates in dusty, wet and vibration-heavy conditions, often far from reliable cellular or Wi‑Fi coverage. SwRI is positioned to address these challenges by applying AI‑powered Internet of Things (IoT) architectures that link edge computing, onboard sensors and cloud services into a cohesive system tailored for agricultural operations.

Conceptually, a tractor system could process sensor data using embedded machine learning models and then publish summarized maps and health indicators to a farm network when connectivity is available. These data streams can include multilayered environmental models — for example, canopy structure, trellis locations, and soil and terrain features — stored and indexed over time to support crop management. In parallel, pedestrian and obstacle detection classifiers run locally to enhance safety, while low‑cost row guidance algorithms provide steering cues and positioning without requiring continuous high-bandwidth connections.

DETAIL

Edge computing brings computation and storage closer to the data source instead of relying on distant data centers.

Sensor nodes on implements can also report status, usage patterns and anomalies back to centralized dashboards, enabling predictive maintenance and fine-tuning weed management strategies. By carefully partitioning intelligence between the edge and the cloud and using IoT messaging to synchronize perception, control and analytics, SwRI’s solutions give growers actionable insights and automation capabilities even in connectivity-constrained, real-world specialty crop environments.

Combustion Group Lead Dr. Chris Thomas discusses energy projects, including a MW-scale direct-fired oxy-fuel combustion system incorporating supercritical carbon dioxide as the working fluid.

SwRI is collaborating with UT San Antonio to cultivate the next generation of specialists in smart agriculture and related research. Here, Combustion Group Lead Dr. Chris Thomas discusses energy projects, including a MW-scale direct-fired oxy-fuel combustion system incorporating supercritical carbon dioxide as the working fluid.

A photo of a tractor in a field.

To remove farmers from hazardous situations, SwRI added a motion sensor to this tractor to help automate pesticide applications.

The Future of Farming

Smart agriculture is rapidly evolving toward fleets of interconnected, autonomous systems that can sense, study, plan and act covering operations from soil preparation and planting to harvest and post-processing. Emerging technologies will combine advanced perception, robotics, edge AI and cloud analytics to deliver detailed insights on crop health, soil moisture and input efficiency, while enabling precise interventions such as targeted weeding, variable-rate spraying and automated harvesting.

With extensive expertise in off-road autonomy, sensor fusion and artificial intelligence, SwRI is positioned to drive this next wave of innovation across all aspects of agriculture. Building on current work in intelligent perception for row crops, specialty crops and the forage industry, SwRI can expand into projects for autonomy, intelligent implements and aerial drone applications, including automated fruit and vegetable harvesting, precision thinning and pruning, and canopy management using 4D mapping. SwRI’s experience in industrial IoT and edge/cloud architectures will underpin scalable data platforms that integrate perception data with agronomic models, farm management software and other external services.

A key part of advancing smart agriculture will be cultivating the next generation of experts and accelerating technology transfer from research to field deployment. SwRI is collaborating with The University of Texas at San Antonio (UT San Antonio) through a joint program designed to train students in robotics, AI, sensing and agricultural engineering for real-world applications. Under this collaboration, SwRI engineers and scientists mentor UT San Antonio students working on applied projects on smart agricultural solutions, creating a talent pipeline skilled in both cutting‑edge automation and practical agricultural constraints so that innovations remain grounded in growers’ needs.

Dr. Edmond DuPont, a principal engineer in SwRI’s Intelligent Systems Division, sitting in a tractor.

ABOUT THE AUTHOR

Dr. Edmond DuPont, a principal engineer in SwRI’s Intelligent Systems Division, has extensive experience in automated vehicles, particularly for off-road applications. He is applying that expertise to the growing field of intelligent agriculture.