Real-Time Applications of 3D Object Detection and Tracking

Citation

Ma, Jeremy Chee-Ming (2010) Real-Time Applications of 3D Object Detection and Tracking. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/4N1K-GK74. https://resolver.caltech.edu/CaltechTHESIS:01152010-143831008

Abstract

Robot perception is a fundamental aspect of any autonomous system. It gives the robot the capacity to make sense of vast amounts of data and gain an understanding of the world around it. An active problem in the area of robot perception is real-time detection and pose estimation of 3D objects. This thesis presents an approach to 3D object detection and tracking utilizing a stereo-camera sensor. Geometric object models are learned in short order time via a training phase and real-time detection and tracking is made possible by performing sparse stereo calculations on the chosen features within an adaptive region of interest of the camera image. The experimental results obtained by using this method will show the effectiveness of the approach as compared against ground truth measures in real-time. Using that framework as a basis, extensions to two other problems in robot sensing are then considered: (1) sensor-planning for model identification, and (2) sensor-planning for object-search. In the former, a novel algorithm for determining the next-best-view for a mobile sensor to identify an unknown 3D object from among a database of known models is presented and tested across two experiments involving real robotic systems. An information theoretic approach is taken to quantify the utility of each potential sensing action and the validity of the algorithm is discussed. In the latter area, a novel approach is presented that allows an autonomous mobile robot to search for a 3D object using an onboard stereo camera sensor mounted on a pan-tilt head. Search efficiency is realized by the combination of a coarse-scale global search coupled with a fine-scale local search, guided by a grid-based probability map. Obstacle avoidance during the search is naturally integrated into the method with additional experimental results on a mobile robot presented to illustrate and validate the proposed search strategy. All presented experiments were carried out in real-time processing with modest computation done by a single laptop computer.

Thesis Files