Visual and Spatial Representation Learning with Applications in Ecology

Citation

Cole, Elijah Henry John (2023) Visual and Spatial Representation Learning with Applications in Ecology. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/xp2k-es46. https://resolver.caltech.edu/CaltechTHESIS:06072023-210232983

Abstract

Machine learning has the potential to empower scientists, physicians, and other human experts working to solve problems of societal importance. To realize this goal, we need algorithms that can distill useful knowledge from real-world data. However, most machine learning research focuses on benchmarks that seldom reflect real-world challenges, such as learning from limited, noisy, or weak supervision. This thesis develops new benchmarks, algorithms, and problem settings that link fundamental machine learning research to impactful applications in ecology. In Part I, we provide context and motivation for our work. How and why should machine learning researchers work with domain experts on real-world problems? What is the appeal of ecology specifically Part II focuses on visual representation learning with an emphasis on label efficiency. We discuss the strengths and limitations of self-supervised learning, the relationship between concept specificity and representation learning, and multi-label learning with minimal labeled data. Part III covers our work in the emerging field on spatial representation learning. In particular, we consider the problem of modeling the spatial distribution of plant and animal species. We review this important ecological problem from a machine learning perspective before showing how deep learning can transform the way these models are applied (using spatial models to assist image classifiers) and developed (jointly learning spatial distributions and representations). Finally, Part IV concludes and highlights opportunities for future work.

Thesis Files