A Caltech Library Service

Formal Methods for Control Synthesis in Partially Observed Environments: Application to Autonomous Robotic Manipulation


Sharan, Rangoli (2014) Formal Methods for Control Synthesis in Partially Observed Environments: Application to Autonomous Robotic Manipulation. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/RQKC-N871.


Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.

This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.

This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.

The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.

The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:control synthesis, formal methods, POMDP, linear temporal logic, LTL, robotic manipulation
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Control and Dynamical Systems
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Burdick, Joel Wakeman
Thesis Committee:
  • Burdick, Joel Wakeman (chair)
  • Murray, Richard M.
  • Beck, James L.
  • Hudson, Nicolas H.
Defense Date:2 April 2014
Record Number:CaltechTHESIS:05292014-063852576
Persistent URL:
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:8420
Deposited By: Rangoli Sharan
Deposited On:29 May 2014 23:37
Last Modified:04 Oct 2019 00:05

Thesis Files

PDF (Full Thesis) - Final Version
See Usage Policy.


Repository Staff Only: item control page