Investigation of Visual Sensor Technologies and Requirements for Flight Control, 09-R9772

Printer Friendly Version

Principal Investigators
George T. McWilliams
Kevin J. Alley
Chris I. Mentzer
Kendric L. Taylor

Inclusive Dates:  12/17/07 – 04/17/08

Background - Current state-of-the-art Unmanned Aerial Vehicles (UAVs) use "traditional" air vehicle sensors for flight control such as a Global Positioning System (GPS) sensor, inertial sensors, magnetometers, pressure sensors and air flow sensors. Next generation UAVs have the potential to provide advanced capabilities such as flying in urban canyons, avoiding obstacles, detecting and recognizing potential targets, flying in a leader-follower formation, automated visual searches, localization using geo-registration, and even flying inside buildings or other structures. These capabilities will require additional types of data that could be provided by adding sensors like laser range finders, ultrasonic and acoustic sensors, and radar systems, which could increase the size and cost of the entire system and would not be feasible in a small UAV. However, it is possible that the type of data needed for these advanced capabilities, as well as flight control, could be provided using visual sensors in lieu of a multitude of sensors.

Approach - The program objective is to investigate the feasibility of using visual sensor technology for estimating pitch, roll, and groundspeed for future use in advanced flight controls. This feasibility investigation includes an analysis of the estimation errors and comparison to GPS/IMU measurements. It also includes requirements and limitations to the algorithms providing the estimations. The pitch, roll, and groundspeed estimates are generated in a simulated environment using a model of a UAV and an open-source graphical simulation. By using a graphical environment, real image processing algorithms are implemented and used to estimate these vehicle states.

Accomplishments - SwRI engineers have set up a simulation environment for vision-based control of a UAV. This environment includes a model of a UAV, including the flight dynamics, flight controller, and sensors. The environment also includes an open-source graphical simulation, which is able to simulate vision sensors and feed the images back to the sensor model through a frame grabber. Image processing algorithms have been written to measure the roll and pitch angles of the aircraft by detecting the angle of the horizon in front and to the side of the aircraft. The groundspeed of the aircraft is also measured by calculating how fast an image of the ground is moving underneath the aircraft. The pitch and roll algorithm is limited by the flatness of the horizon, while the groundspeed algorithm is limited by the detail in the ground images. These algorithms have yielded results that are close to the actual states of the aircraft when the documented limitations have not been exceeded. While the results are reasonable, an estimation algorithm like a Kalman filter would also be necessary to make use of the algorithms developed in this project as an approach to flight control.

Figure 1. The model of the UAV flies through an urban canyon in the simulation.
Figure 2. The graphical flight simulator displays views that simulate visual sensors pointing in various directions. The middle sensor is displayed after the horizon detection algorithm has been performed.

2008 Program Home