2013 IR&D Annual Report

Robotic Part Handling for Unstructured Industrial Applications, 10-R8301

Principal Investigator
Clay Flannigan

Inclusive Dates:  04/01/12 – 03/01/13

Photo: a demonstration cell that was capable of sorting multiple object classes from a cluttered pile.
SwRI has developed multiple perception and planning approaches to address manipulation in cluttered, dynamic, or unstructured environments. The figure shows a demonstration cell that was capable of sorting multiple object classes from a cluttered pile.

Background — Recent developments in 3D sensor technologies, adaptive robotic grippers and advanced perception and planning algorithms are rapidly advancing the use of robots in complex and dynamic environments. Traditional industrial application of robots requires that parts be precisely located using dedicated fixtures so that there is little uncertainty in the location of the workpiece and condition of the workspace. More advanced industrial robotic systems use 2D or 3D vision sensing to handle minor variations in part locations. However, there are large classes of problems where high part variability or dynamic environments prohibit rigid fixturing methods and confound commercially available vision solutions. An example application is sorting residential recyclables, where there is an almost infinite variability of parts to be manipulated. Aerospace manufacturing is another market that is challenged by high-mix, low-volume processes.

Approach — SwRI is the founder of an open-source software framework for industrial robotics called ROS-Industrial that builds upon the work of the huge community of robotic researchers using the Robot Operating System (ROS). The current research is extending the ROS-Industrial program through investigation of perception, motion planning and grasp planning methods to address unstructured manipulation. Specifically, PrimeSense structured light sensors are used for colorized, 3D-range image acquisition. Object recognition and pose estimation algorithms have been developed that use the color 3D data to identify objects in cluttered environments. Both vacuum and adaptive finger grippers were employed for grasping. Motion planning objectives included generating efficient and collision-free motion even in dynamic or complex scenes.

Accomplishments — For relatively simple geometric objects, bin picking was successfully demonstrated, including cases where multiple classes of objects were randomly sorted. For more complex objects types, typical of those in manufacturing operations, multiple algorithms were developed and tested for object segmentation and pose estimation. Depending on the optical characteristics of the object and the environment, the algorithms used shape cues, color cues, image texture cues or combinations of the three. For highly cluttered scenes, heuristics were developed to singulate parts so that recognition algorithms could be more effectively applied. The results show that certain classes of problems are tractable for industrial application, especially those where parts have salient optical or geometric features. Each application poses unique challenges that must be considered for perception and grasping solutions.

Benefiting government, industry and the public through innovative science and technology
Southwest Research Institute® (SwRI®), headquartered in San Antonio, Texas, is a multidisciplinary, independent, nonprofit, applied engineering and physical sciences research and development organization with 10 technical divisions.