A Caltech Library Service

Real-Time Applications of 3D Object Detection and Tracking


Ma, Jeremy Chee-Ming (2010) Real-Time Applications of 3D Object Detection and Tracking. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/4N1K-GK74.


Robot perception is a fundamental aspect of any autonomous system. It gives the robot the capacity to make sense of vast amounts of data and gain an understanding of the world around it. An active problem in the area of robot perception is real-time detection and pose estimation of 3D objects. This thesis presents an approach to 3D object detection and tracking utilizing a stereo-camera sensor. Geometric object models are learned in short order time via a training phase and real-time detection and tracking is made possible by performing sparse stereo calculations on the chosen features within an adaptive region of interest of the camera image. The experimental results obtained by using this method will show the effectiveness of the approach as compared against ground truth measures in real-time. Using that framework as a basis, extensions to two other problems in robot sensing are then considered: (1) sensor-planning for model identification, and (2) sensor-planning for object-search. In the former, a novel algorithm for determining the next-best-view for a mobile sensor to identify an unknown 3D object from among a database of known models is presented and tested across two experiments involving real robotic systems. An information theoretic approach is taken to quantify the utility of each potential sensing action and the validity of the algorithm is discussed. In the latter area, a novel approach is presented that allows an autonomous mobile robot to search for a 3D object using an onboard stereo camera sensor mounted on a pan-tilt head. Search efficiency is realized by the combination of a coarse-scale global search coupled with a fine-scale local search, guided by a grid-based probability map. Obstacle avoidance during the search is naturally integrated into the method with additional experimental results on a mobile robot presented to illustrate and validate the proposed search strategy. All presented experiments were carried out in real-time processing with modest computation done by a single laptop computer.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:3D object detection; stereo vision; object recognition; sensor planning; object search; robotics
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Mechanical Engineering
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Burdick, Joel Wakeman
Thesis Committee:
  • Burdick, Joel Wakeman (chair)
  • Murray, Richard M.
  • Beck, James L.
  • Perona, Pietro
Defense Date:19 October 2009
Non-Caltech Author (AT)
Record Number:CaltechTHESIS:01152010-143831008
Persistent URL:
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:5522
Deposited By: Jeremy Ma
Deposited On:05 Feb 2010 17:18
Last Modified:08 Nov 2019 18:08

Thesis Files

PDF (Complete Thesis (single sided)) - Final Version
See Usage Policy.

PDF (Complete Thesis (double sided)) - Final Version
See Usage Policy.


Repository Staff Only: item control page