A Caltech Library Service

Computational Modeling and Psvchophysics in Low- and Mid-Level Vision


Hou, Xiaodi (2014) Computational Modeling and Psvchophysics in Low- and Mid-Level Vision. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/1H1Q-J703.


This thesis addresses a series of topics related to the question of how people find the foreground objects from complex scenes. With both computer vision modeling, as well as psychophysical analyses, we explore the computational principles for low- and mid-level vision.

We first explore the computational methods of generating saliency maps from images and image sequences. We propose an extremely fast algorithm called Image Signature that detects the locations in the image that attract human eye gazes. With a series of experimental validations based on human behavioral data collected from various psychophysical experiments, we conclude that the Image Signature and its spatial-temporal extension, the Phase Discrepancy, are among the most accurate algorithms for saliency detection under various conditions.

In the second part, we bridge the gap between fixation prediction and salient object segmentation with two efforts. First, we propose a new dataset that contains both fixation and object segmentation information. By simultaneously presenting the two types of human data in the same dataset, we are able to analyze their intrinsic connection, as well as understanding the drawbacks of today’s “standard” but inappropriately labeled salient object segmentation dataset. Second, we also propose an algorithm of salient object segmentation. Based on our novel discoveries on the connections of fixation data and salient object segmentation data, our model significantly outperforms all existing models on all 3 datasets with large margins.

In the third part of the thesis, we discuss topics around the human factors of boundary analysis. Closely related to salient object segmentation, boundary analysis focuses on delimiting the local contours of an object. We identify the potential pitfalls of algorithm evaluation for the problem of boundary detection. Our analysis indicates that today’s popular boundary detection datasets contain significant level of noise, which may severely influence the benchmarking results. To give further insights on the labeling process, we propose a model to characterize the principles of the human factors during the labeling process.

The analyses reported in this thesis offer new perspectives to a series of interrelating issues in low- and mid-level vision. It gives warning signs to some of today’s “standard” procedures, while proposing new directions to encourage future research.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:saliency, fixations, salient object segmentation, boundary detection, dataset analysis, human factors
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computation and Neural Systems
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Koch, Christof
Thesis Committee:
  • Perona, Pietro (chair)
  • Koch, Christof
  • Yuille, Alan L.
  • Shimojo, Shinsuke
Defense Date:7 May 2014
Record Number:CaltechTHESIS:05142014-044033712
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for ch. 2 adapted for ch. 3 adapted for ch. 4 adapted for ch. 5
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:8233
Deposited By: Xiaodi Hou
Deposited On:28 May 2014 23:47
Last Modified:30 Aug 2022 22:45

Thesis Files

PDF - Final Version
See Usage Policy.

PDF (Preprint of article adapted for ch. 3) - Supplemental Material
See Usage Policy.

PDF (Preprint of article adapted for ch. 5) - Supplemental Material
See Usage Policy.


Repository Staff Only: item control page