A Caltech Library Service

From Restoring Human Vision to Enhancing Computer Vision


Liu, Yang (2020) From Restoring Human Vision to Enhancing Computer Vision. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/sq58-z682.


The central theme of this work is enabling vision, which includes two subtopics: restoring vision for blind humans, and enhancing computer vision models in visual recognition. Chapter 1 first provides a gentle introduction to relevant high level principles of human visual computations and summarizes two fundamental questions that vision answers: "what" and "where." Chapters 2, 3, and 4 contain three published projects that are anchored by those two fundamental questions.

Chapter 2 introduces a cognitive assistant to restore visual function for blind humans by focusing on an interface powered by audio augmented reality. The assistant communicates the "what" and "where" aspects of visual scenes by a combination of natural language and spatialized sound. We experimentally demonstrated that the assistant enables many aspects of visual functions for naive blind users.

Chapters 3 and 4 develop data augmentation methods to address the data inefficiency problem in neural network based computer visual recognition models. In Chapter 3, a 3D-simulation based data augmentation method is developed for improving the generalization of visual classification models for rare classes. In Chapter 4, a fast and efficient data augmentation method is developed for the newly formulated panoptic segmentation task. The method improves performance of state-of-the-art panoptic segmentation models and generalizes across dataset domains, sizes, model architectures, and backbones.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Vision, computer vision, blind
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computation and Neural Systems
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Meister, Markus
Thesis Committee:
  • Perona, Pietro (chair)
  • Siapas, Athanassios G.
  • Yue, Yisong
  • Meister, Markus
Defense Date:2 June 2020
Non-Caltech Author Email:youngleoel (AT)
Record Number:CaltechTHESIS:06092020-120629159
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for Chapter 2. related to Chapter 2. adapted for Chapter 3. adapted for Chapter 4.
Liu, Yang0000-0002-8155-9134
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:13808
Deposited By: Yang Liu
Deposited On:09 Jun 2020 21:17
Last Modified:17 Jun 2020 19:34

Thesis Files

PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page