Quick Look

Automated Monitoring of Waterways and Evaluation of Imagery Anomaly Detection Technologies, 10-9415

Printer Friendly Version

Principal Investigators
Michael J. Magee
Michael P. Rigney

Inclusive Dates: 07/31/03 - 10/03/03

Background - With increased awareness of the need to monitor international borders, there has been substantial interest in the development of technologies that can be used to assist with detecting activities both on and in the vicinity of waterways along these borders. In this regard, agencies responsible for monitoring activities along international borders that the United States shares with its neighbors face substantial challenges. First, since these borders are thousands of miles in length, it is difficult to fund and allocate enough personnel to provide effective coverage on a continuous basis. Within this context, multiple imaging devices (e.g., television cameras) can provide visual coverage that could be monitored by crews of border enforcement personnel. However, the ability of a human observer to detect activities present on any single camera channel diminishes as the number of cameras being monitored increases. Operator visual fatigue from extended periods of monitoring is detrimental to vigilance and incident detection. Furthermore, low illumination levels can adversely affect the ability to detect objects in nighttime scenes unless special sensing technology is used.

Approach - In order to address the above issues, this project investigated and evaluated sensor technologies and methods for the automated detection of anomalous nighttime events on waterways and in wooded areas as might be encountered along international borders. The sensing technologies investigated and evaluated included an electro-optical (EO) visible light camera with an image intensifier, an EO camera with infrared illuminator, and an infrared (IR) camera. The algorithmic methodologies investigated included the adaptive background mixture model (ABMM), mean-sigma, and median-sigma.

The primary emphasis of the project was to investigate, develop, and implement image processing methodologies that are capable of detecting conditions that vary substantially from those that normally occur. In general, these methodologies consisted of a four-step process:

  • An intensity characteristic model of the monitored scene is learned during a training phase.
  • During the anomaly detection phase, objects are segmented based on pixel characteristics that vary substantially from those embodied in the intensity characteristic model.
  • Segmented objects that match certain morphological, topological, and/or geometric constraints are flagged as being candidates for further (temporal) processing.
  • Segmented objects with unanticipated temporal persistence or geometric characteristics are identified as being worthy of visual inspection by a human operator.

Accomplishments - The sensing technologies and image processing algorithms applied during this investigation successfully demonstrated the utility of using automated image analysis software for detecting anomalous events on waterways and in foliage and wooded areas at night. With respect to the sensing technologies evaluated, it was generally the case that infrared imaging was deemed to be the preferred sensing technology for both scenarios investigated. This conclusion was based primarily on two considerations. First, it was generally the case that the background characteristics were more stable over time with the foreground objects (in motion) having greater statistical variance. Second, since the objects of interest were sources of energy that could be sensed even when partially occluded by foliage, no active illumination was required.

The results of the investigation are illustrated by Figures 1 and 2, in which a watercraft and an individual partially occluded by foliage are detected. The results of processing sequences of images are illustrated by viewing the waterway video and the foliage video, which show the results of tracking watercraft and an individual moving through foliage. It should be noted that in the case of watercraft detection, the wake generated by the watercraft is detected as secondary evidence of its presence.

Figure 1. Watercraft Detected in Infrared Image

Figure 2. Partially Occluded Individual in Wooded Area Detected in Infrared Image

2003 Program Home