Reliable Autonomy Under Uncertainty: From Learning-Based to Non-Rational Control

Citation

Kargin, Taylan (2025) Reliable Autonomy Under Uncertainty: From Learning-Based to Non-Rational Control. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/838c-yz69. https://resolver.caltech.edu/CaltechTHESIS:06092025-233932460

Abstract

Autonomous systems are profoundly reshaping our societies, industries, and daily lives, delivering unprecedented levels of efficiency, innovation, and adaptability. From self-driving vehicles navigating dense urban traffic and coordinated swarms of search-and-rescue robots operating in hazardous environments, to next-generation intelligent power grids and high-precision industrial automation, these systems are increasingly deployed in safety-critical and high-stakes settings where they are routinely entrusted with split‑second decisions that carry profound economic and lethal consequences. In such contexts, the imperative for reliability, safety, and robustness is paramount: a single unanticipated failure within a power distribution network can trigger extensive blackouts, and a momentary lapse in decision-making or perception by an autonomous vehicle can endanger lives.

Despite their remarkable capabilities, securing such reliability guarantees faces formidable and multifaceted challenges. The environments in which these systems operate are characterized by unprecedented complexity, vast scale, and pervasive uncertainty as they frequently interact with numerous external entities such as humans or other autonomous agents whose behaviors may be volatile, adversarial, or fundamentally unknown. Explicitly and exhaustively modeling this complexity a priori is practically infeasible, compelling systems to infer, adapt, and respond to the novel environments by learning from data. Although contemporary machine‑learning models afford expressive representations, their assurances are limited by the scope and fidelity of their training data. Consequently, such models remain vulnerable to distribution shifts, rare events, or unmodeled edge cases, which can precipitate catastrophic failure.

Further complicating matters, real-world applications frequently impose stringent resource constraints, including limited computation, memory, communication, and power. These constraints demand principled trade-offs between competing performance objectives and operational constraints such as safety, stability, robustness, and efficiency, especially in high-stakes and uncertainty-laden settings. This dissertation addresses these challenges by contributing fundamental theoretical results and practical computational tools towards provably reliable, resource‑efficient, and scalable autonomy.

Operating safely in dynamic and a priori unknown environments poses a fundamental challenge for autonomous systems: balancing exploration, i.e., the pursuit of long-term optimality by probing uncertain policy landscape at the risk of degraded safety, against exploitation, i.e., leveraging current knowledge to ensure short-term performance and stability at the expense of settling for a suboptimal policy. In Part 1, we study online reinforcement learning approaches for unknown linear dynamical systems to address this challenge. We present computationally efficient algorithms for online learning and control in both state-feedback and measurement-feedback settings that operate safely without any prior knowledge of the system. We rigorously establish their feasibility through finite-time guarantees on performance, computational complexity, and stability, matching the fundamental theoretical bounds.

Statistical models underlie every layer of an autonomous system, serving as representations of complex data-generating phenomena. Typically constructed from empirical data through a blend of explicit modeling, machine learning, and simulation, these models are vulnerable to distribution shift, i.e., discrepancies between design and deployment conditions, which can jeopardize both performance and safety. In Part 2, we investigate distributionally robust optimization (DRO) methods for control, prediction, communication, and unsupervised learning to guard against model misspecification and distribution shifts. DRO blends average-case optimality with worst‑case guarantees: by maximizing expected performance against the least‑favorable statistical model consistent with the available data, it strikes a balanced trade-off between robustness and performance informed by data.

Autonomous control systems must often balance several performance goals, such as cost efficiency, robustness, risk tolerance, and stability, while meeting practical constraints such as suitability for real‑time implementation and scalability. Because these design problems are inherently infinite‑dimensional, only a handful of special cases admit exact, tractable solutions (e.g., Linear‑Quadratic‑Gaussian, ℋ_∞‑optimal, or regret‑optimal control) while widely studied formulations like mixed ℋ₂/ℋ_∞ control remain unresolved. In Part 3, we present non‑rational control, a unified framework that makes many such problems both solvable and implementable. The key is an optimize‑then‑approximate strategy that delivers provably near‑optimal, stabilizing, finite‑order (rational) controllers even when the true optimum resides in an infinite‑dimensional (non‑rational) policy space.

Thesis Files