2014 IR&D Annual Report

Vision-based Localization, 10-R8452

Principal Investigator
Kristopher C. Kozak

Inclusive Dates: 02/21/14 – 06/21/14

Background — For many applications that require localization, a Global Positioning System (GPS) receiver serves as an adequate, low-cost solution. However, for some applications, such as automated driving, the limitations and vulnerabilities of GPS make it unreliable as the sole source of localization. For instance, GPS accuracy can fluctuate widely and requires an unobstructed line-of-sight to multiple satellites. In addition, its signals are subject to interference, multi-path, jamming and spoofing. In spite of this, there are essentially no practical alternatives to GPS that are readily available. An alternative/supplement to GPS, which utilizes a downward facing camera, was developed at SwRI under a previous internal research project (10-R8248, 2011-2013). Development of that technology, also known as Ranger, was successfully implemented and demonstrated on a military HMMWV platform. To better address the localization needs of commercial clients, this project was carried out with the goal of porting the Ranger hardware and software to a passenger vehicle.

Figure 1: Features of opportunity in ground images are matched by appearance subject to geometric constraints to identify overlapping frames and determine vehicle location.
Figure 1. Features of opportunity in ground images are matched by appearance subject to geometric constraints to identify overlapping frames and determine vehicle location.

Approach — The limited ground clearance and electrical power available on typical passenger vehicles made it necessary to change key aspects of the original design. Instead of a large commercial off-the-shelf camera enclosure, a small custom enclosure was designed and fabricated. Instead of a high-powered, rack-mount server, a compact, desktop PC was used. Additionally, a smaller camera and a more flexible illumination system were incorporated into the system. Ranger system capabilities were also split into separate mapping and localization systems, which allowed for compromises on size, cost, field-of-view, etc., for the localization systems, which may be significantly constrained with respect to mounting in a passenger vehicle. Significant improvements to the software were similarly made to accommodate the hardware changes. Among these improvements were changes to image matching methods, and map optimization methods that allowed the system to operate at frame rates of about 10 fps.

Figure 2: Solid model of custom camera enclosure.
Figure 2: Solid model of custom camera enclosure.

Accomplishments — During this project, not only was the Ranger system successfully ported to passenger vehicles, but also its overall performance and capabilities were greatly improved. The size, power and cost of components for the second-generation prototype were significantly reduced. The camera and lighting hardware was simplified and reduced in size. The image-processing pipeline was streamlined and modified to use new image feature detectors and descriptors, which improved the feature matching performance (both increasing the matching speed and increasing the percentage of inliers per matching frame). The image pre-processing steps were updated to include a dynamic contrast-stretching approach rather than a static normalization map, which significantly reduced over-exposure and harsh shadowing issues with sunlight in the image frames. The updated Ranger system was successfully demonstrated in a variety of real-world conditions including on public roads and freeways (at speeds of up to 70 mph), in locations where GPS was poor or unavailable, and in wet road/rainy conditions.

Benefiting government, industry and the public through innovative science and technology
Southwest Research Institute® (SwRI®), headquartered in San Antonio, Texas, is a multidisciplinary, independent, nonprofit, applied engineering and physical sciences research and development organization with 9 technical divisions.