Perception-Driven Autonomy and Learning Control for Ground Vehicles

Citation

Lupu, Elena Sorina (2025) Perception-Driven Autonomy and Learning Control for Ground Vehicles. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/79tk-eg16. https://resolver.caltech.edu/CaltechTHESIS:06092025-020707222

Abstract

Autonomous robots are widely recognized as highly valuable and are expected to become increasingly prevalent. They will play a critical role across a wide range of terrestrial applications in complex, unstructured environments, as well as in space, supporting infrastructure and exploration on various bodies throughout the solar system and beyond. Looking ahead, autonomous robots will play a crucial role in the search for extraterrestrial life by enabling exploration of remote and extreme environments beyond Earth. As robots need to approach more complex tasks, the ability to rapidly perceive, understand, make real-time decisions, and operate at speed requires advances in perception-driven controls, improved predictability, and robustness to disturbances. To enable these capabilities, the first part of this thesis proposes an innovative approach to enhancing ground vehicle mobility by integrating a vision-based control algorithm that adapts to changes in real-time. Our approach improves the vehicle's ability to assess and respond to complex terrains in real-time by leveraging visual information through visual foundation models and meta-learning. Our controller has provable guarantees of exponential stability and was validated on board two ground vehicles. Next, an extension of the previously mentioned method applied to detecting objects in space using a visual foundation model is presented. Our method was successfully demonstrated in space in early 2025 aboard the EdgeNode Lite spacecraft. Efficient operation comes from the synergy of suitable autonomy and control with a suitable robot body. Following this consideration, the second part of the thesis presents the design and control of multi-degrees of freedom robots designed for mobility in complex environments. It presents a nonlinear tracking controller with adaptation to improve the walking performance of walking-flying robots. This is illustrated by our implementation on Leonardo, the first robot to combine walking with flying to create a new type of locomotion, which we showcase in complex acrobatic movements such as slacklining and skateboarding. In a second case study, we aim to further understand and improve biped walking by introducing a bipedal robot designed to be lightweight, easily manufactured, and easily repaired, serving as a platform for testing learning-based controllers. We introduce and demonstrate the performance of two controllers: a model-based and a learning-based control. This work highlights the importance of tightly integrated perception, control, and electromechanical design in achieving robust autonomy: on Earth, in orbit, and beyond.

Thesis Files