A Caltech Library Service

Do Robots Dream of Random Trees? Monte Carlo Tree Search for Dynamical, Partially Observable, and Multi-Agent Systems


Rivière, Benjamin Pierre (2024) Do Robots Dream of Random Trees? Monte Carlo Tree Search for Dynamical, Partially Observable, and Multi-Agent Systems. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/dbwa-we50.


Autonomous robots are poised to transform various aspects of society, spanning transportation, labor, and scientific space exploration. A critical component to enable their capabilities is the algorithm that interprets sensor data to generate intelligent planned behavior. Although reinforcement learning methods that train parameterized policies offline from data have shown recent success, they are inherently limited when robots inevitably encounter situations outside their training domain. In contrast, optimal control techniques, which compute trajectories in real-time using numerical optimization, typically yield only locally optimal solutions.

This research endeavors to bridge the gap by developing algorithms that compute trajectories in real-time while converging towards globally optimal solutions. Building upon the Monte Carlo Tree Search (MCTS) framework—a stochastic tree search method that simulates future trajectories while balancing exploration and exploitation—the research focus is twofold: (i) constructing an efficient discrete representation of continuous systems in a decision trees, and (ii) searching on the resulting tree while balancing exploration and exploitation to achieve global optimality.

The study spans theoretical analysis, algorithmic design, and hardware demonstrations across dynamical, partially observable, and multi-agent systems. By addressing these critical questions, this research aims to advance the field of autonomous robotics, enabling the deployment of intelligent robots in complex and diverse environments.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Robotics; Planning; Dynamical Systems; Machine Learning
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Aeronautics
Awards:William F. Ballhaus Prize, 2024.
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Chung, Soon-Jo
Thesis Committee:
  • Yue, Yisong (chair)
  • Chung, Soon-Jo
  • Hadaegh, Fred
  • Pellegrino, Sergio
Defense Date:29 May 2024
Non-Caltech Author Email:benjamin.p.riviere (AT)
Funding AgencyGrant Number
Aerospace CorporationUNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)UNSPECIFIED
United Technologies CorporationUNSPECIFIED
Record Number:CaltechTHESIS:06032024-152357240
Persistent URL:
Related URLs:
URLURL TypeDescription for Chapter 3 for Chapter 4 for Chapter 5 for Appendix C
Rivière, Benjamin Pierre0000-0002-0597-5400
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:16482
Deposited By: Benjamin Riviere
Deposited On:06 Jun 2024 22:14
Last Modified:17 Jun 2024 18:38

Thesis Files

[img] PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page