A Caltech Library Service

Visual and Spatial Representation Learning with Applications in Ecology


Cole, Elijah Henry John (2023) Visual and Spatial Representation Learning with Applications in Ecology. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/xp2k-es46.


Machine learning has the potential to empower scientists, physicians, and other human experts working to solve problems of societal importance. To realize this goal, we need algorithms that can distill useful knowledge from real-world data. However, most machine learning research focuses on benchmarks that seldom reflect real-world challenges, such as learning from limited, noisy, or weak supervision. This thesis develops new benchmarks, algorithms, and problem settings that link fundamental machine learning research to impactful applications in ecology. In Part I, we provide context and motivation for our work. How and why should machine learning researchers work with domain experts on real-world problems? What is the appeal of ecology specifically Part II focuses on visual representation learning with an emphasis on label efficiency. We discuss the strengths and limitations of self-supervised learning, the relationship between concept specificity and representation learning, and multi-label learning with minimal labeled data. Part III covers our work in the emerging field on spatial representation learning. In particular, we consider the problem of modeling the spatial distribution of plant and animal species. We review this important ecological problem from a machine learning perspective before showing how deep learning can transform the way these models are applied (using spatial models to assist image classifiers) and developed (jointly learning spatial distributions and representations). Finally, Part IV concludes and highlights opportunities for future work.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:computer vision; machine learning; deep learning; conservation technology
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computing and Mathematical Sciences
Awards:Amori Doctoral Prize in CMS, 2023.
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Perona, Pietro
Thesis Committee:
  • Perona, Pietro
  • Yue, Yisong (chair)
  • Bouman, Katherine L.
  • Belongie, Serge J.
Defense Date:10 May 2023
Funding AgencyGrant Number
National Science FoundationDGE1745301
Resnick Sustainability InstituteUNSPECIFIED
U.S. Fish and Wildlife ServiceF22AP01490-00
Nissan CorporationUNSPECIFIED
Record Number:CaltechTHESIS:06072023-210232983
Persistent URL:
Related URLs:
URLURL TypeDescription for Chapter 2 for Chapter 3 for Chapter 4 for Chapter 5 for Chapter 6 for Chapter 7 for Chapter 8
Cole, Elijah Henry John0000000166230966
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:16093
Deposited By: Elijah Cole
Deposited On:09 Jun 2023 14:58
Last Modified:08 Nov 2023 00:44

Thesis Files

[img] PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page