A Caltech Library Service

Automated Visual Tracking for Behavioral Analysis of Biological Model Organisms


Fontaine, Ebraheem Ihsan (2008) Automated Visual Tracking for Behavioral Analysis of Biological Model Organisms. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/TSQ7-SN68.


Capturing the detailed motion and behavior of biological organisms plays an important role in a wide variety of research disciplines. Many studies in biomechanics, neuroethology, and developmental biology rely on analysis of video sequences to understand the underlying behavior. However, the efficient and rapid quantification of these complex behavioral traits imposes a major bottleneck on the elucidation of many interesting scientific questions. The goal of this thesis is to develop a suite of model-based visual tracking algorithms that will apply across a variety of model organisms used in biology. These automated tracking algorithms operate in a high-throughput, high-resolution manner needed for a productive synthesis with modern genetic approaches. To this end, I demonstrate automated estimation of the detailed body posture of nematodes, zebrafish, and fruit flies from calibrated video.

The current algorithm utilizes a generative geometric model to capture the organism's shape and appearance. To accurately predict the organism's motion between video frames, I incorporate a motion model that matches tracked motion patterns to patterns in a training set. This technique is invariant with respect to the organism's velocity and can easily incorporate training data from completely different motion patterns. The prediction of the motion model is refined using measurements from the image. In addition to high-contrast feature points, I introduce a region, segmentation model based on level sets that are formally integrated into the observation framework of an Iterated Kalman Filter (IKF). The prior knowledge provided by the geometric and motion models improves tracking accuracy in the presence of partial occlusions and misleading visual cues.

The method is used to track the position and shape of multiple nematodes during mating behavior, zebrafish of different ages during escape response, and fruit flies during take off maneuvers. These applications demonstrate the modular design of this model-based visual tracking system, where the user can specify which components are appropriate to a given experiment. In contrast to other approaches, which are customized to a particular organism or experimental setup, my approach provides a foundation that requires little re-engineering whenever the experimental parameters are changed.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:estimation; geometric modeling; Kalman filtering; model-based tracking
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Mechanical Engineering
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Burdick, Joel Wakeman
Thesis Committee:
  • Burdick, Joel Wakeman (chair)
  • Barr, Alan H.
  • Perona, Pietro
  • Murray, Richard M.
Defense Date:16 April 2008
Record Number:CaltechETD:etd-05272008-161801
Persistent URL:
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:2172
Deposited By: Imported from ETD-db
Deposited On:02 Jun 2008
Last Modified:17 Jan 2020 21:08

Thesis Files

PDF (thesis) - Final Version
See Usage Policy.


Repository Staff Only: item control page