A Caltech Library Service

Representation of the Semantic Structures: from Discovery to Applications


Ryou, Serim (2022) Representation of the Semantic Structures: from Discovery to Applications. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/rvnc-dp57.


The world surrounding us is full of structured entities. Scenes can be structured as the sum of objects arranged in space, objects can be decomposed into parts, and even small molecules are composed of atoms. As humans can organize and structure many concepts into smaller components, structural representation has become a powerful tool for various applications. Computer vision utilizes the part-based representation for classical object detection and categorization tasks, and computational neuroscientists use the structural representation to achieve an interpretable and low-dimensional encoding for behavior analysis. Furthermore, structural encoding of the molecules allows the application of machine learning models to optimize experimental reaction conditions in organic chemistry.

To perform the high-level tasks described above, accurate detection of the structural component should be accomplished in advance. In this dissertation, we first propose methods to improve the pose estimation algorithm, where the task is to localize the semantic parts of the target instance from a 2D image. As the collection of a large number of human annotations is a prerequisite for the task to be successful, we aim to design a model that automatically discovers the structure information from the visual inputs without supervision. Lastly, we demonstrate the efficacy of the structural representation by applying it to various scientific applications such as behavior analysis and organic chemistry.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Computer Vision; Machine Learning
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Electrical Engineering
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Perona, Pietro
Thesis Committee:
  • Abu-Mostafa, Yaser S. (chair)
  • Perona, Pietro
  • Kostina, Victoria
  • Bouman, Katherine L.
Defense Date:18 October 2021
Record Number:CaltechTHESIS:12092021-125955775
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for chapter 2 adapted for chapter 3 adapted for chapter 4 adapted for chapter 5 adapted for chapter 6
Ryou, Serim0000-0003-1344-1158
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:14444
Deposited By: Se Rim Ryou
Deposited On:16 Dec 2021 22:25
Last Modified:08 Nov 2023 00:44

Thesis Files

[img] PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page