2011 IR&D Annual Report

3D Imaging for Behavior Classification, 10-R8221

Principal Investigator
 Chris Lewis

Inclusive Dates:  05/01/11 – 09/01/11

Background — The automated interpretation of the movement of bodies is a broad field with applications in robotics, surveillance, traffic management, healthcare and homeland security, among others. This project leverages the recent advancement in low-cost three-dimensional imaging technologies, which generate high-resolution, colored point clouds to develop a behavior recognition capability.

The first objective of the project was to track a body in motion using the 3D colored data. The second objective was to recognize the subject's behavior using its motion history. Researchers showcased SwRI's ability for customized tracking and behavior recognition using a healthcare application that monitors disabled children being treated for severe self-abuse behavioral problems.

Approach — The approach for tracking the body state assumes there is an underlying kinematic model for the subject that constrains the relative motion of the 3D linkages. At each instant in time, the kinematic model is fit to the observed 3D data and the resulting measurement is smoothed using a state estimation filter. This provides a time history of the body's motion to the behavior classifier. The behavior classifier operates with two layers of classification. First, basic actions are detected, and then the subject's behavior is ascertained from histograms of basic actions. A pitcher's basic actions might include the positions of his pitching and glove hand, a back step, a weight shift, head nods, sideways glances, the length of his stride and the height of his kick. SwRI's hypothesis is that these basic actions are predictors of the pitch thrown.

Figure 1. Colorized clouds of data from Microsoft's Kinect sensor are processed by a ROS node to first remove the background data (left). Then, a node detects and characterizes salient features (middle). Correspondence of features in subsequent frames helps estimate the subjects pose.
Figure 1. Colorized clouds of data from Microsoft's Kinect sensor are processed by a ROS node to first remove the background data (left). Then, a node detects and characterizes salient features (middle). Correspondence of features in subsequent frames helps estimate the subject's pose (right).

Accomplishments —

  • Evaluated the kinematic results returned directly from the Microsoft™ Kinect.

  • Developed software for subtracting the background.

  • Developed software for automatically detecting and tracking features in the foreground data.

  • Developed architecture for a multi-layered behavior classifier from body motion statistics.

 

Benefiting government, industry and the public through innovative science and technology
Southwest Research Institute® (SwRI®), headquartered in San Antonio, Texas, is a multidisciplinary, independent, nonprofit, applied engineering and physical sciences research and development organization with 11 technical divisions.
03/28/13