Automated Visual Tracking for Behavioral Analysis of Biological Model Organisms

Citation

Fontaine, Ebraheem Ihsan (2008) Automated Visual Tracking for Behavioral Analysis of Biological Model Organisms. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/TSQ7-SN68. https://resolver.caltech.edu/CaltechETD:etd-05272008-161801

Abstract

Capturing the detailed motion and behavior of biological organisms plays an important role in a wide variety of research disciplines. Many studies in biomechanics, neuroethology, and developmental biology rely on analysis of video sequences to understand the underlying behavior. However, the efficient and rapid quantification of these complex behavioral traits imposes a major bottleneck on the elucidation of many interesting scientific questions. The goal of this thesis is to develop a suite of model-based visual tracking algorithms that will apply across a variety of model organisms used in biology. These automated tracking algorithms operate in a high-throughput, high-resolution manner needed for a productive synthesis with modern genetic approaches. To this end, I demonstrate automated estimation of the detailed body posture of nematodes, zebrafish, and fruit flies from calibrated video.

The current algorithm utilizes a generative geometric model to capture the organism's shape and appearance. To accurately predict the organism's motion between video frames, I incorporate a motion model that matches tracked motion patterns to patterns in a training set. This technique is invariant with respect to the organism's velocity and can easily incorporate training data from completely different motion patterns. The prediction of the motion model is refined using measurements from the image. In addition to high-contrast feature points, I introduce a region, segmentation model based on level sets that are formally integrated into the observation framework of an Iterated Kalman Filter (IKF). The prior knowledge provided by the geometric and motion models improves tracking accuracy in the presence of partial occlusions and misleading visual cues.

The method is used to track the position and shape of multiple nematodes during mating behavior, zebrafish of different ages during escape response, and fruit flies during take off maneuvers. These applications demonstrate the modular design of this model-based visual tracking system, where the user can specify which components are appropriate to a given experiment. In contrast to other approaches, which are customized to a particular organism or experimental setup, my approach provides a foundation that requires little re-engineering whenever the experimental parameters are changed.

Thesis Files