A Caltech Library Service

Computational Methods for Bayesian Inference in Complex Systems


Catanach, Thomas Anthony (2017) Computational Methods for Bayesian Inference in Complex Systems. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/Z9RX9948.


Bayesian methods are critical for the complete understanding of complex systems. In this approach, we capture all of our uncertainty about a system’s properties using a probability distribution and update this understanding as new information becomes available. By taking the Bayesian perspective, we are able to effectively incorporate our prior knowledge about a model and to rigorously assess the plausibility of candidate models based upon observed data from the system. We can then make probabilistic predictions that incorporate uncertainties, which allows for better decision making and design. However, while these Bayesian methods are critical, they are often computationally intensive, thus necessitating the development of new approaches and algorithms.

In this work, we discuss two approaches to Markov Chain Monte Carlo (MCMC). For many statistical inference and system identification problems, the development of MCMC made the Bayesian approach possible. However, as the size and complexity of inference problems has dramatically increased, improved MCMC methods are required. First, we present Second-Order Langevin MCMC (SOL-MC), a stochastic dynamical system-based MCMC algorithm that uses the damped second-order Langevin stochastic differential equation (SDE) to sample a desired posterior distribution. Since this method is based on an underlying dynamical system, we can utilize existing work in the theory for dynamical systems to develop, implement, and optimize the sampler's performance. Second, we present advances and theoretical results for Sequential Tempered MCMC (ST-MCMC) algorithms. Sequential Tempered MCMC is a family of parallelizable algorithms, based upon Transitional MCMC and Sequential Monte Carlo, that gradually transform a population of samples from the prior to the posterior through a series of intermediate distributions. Since the method is population-based, it can easily be parallelized. In this work, we derive theoretical results to help tune parameters within the algorithm. We also introduce a new sampling algorithm for ST-MCMC called the Rank-One Modified Metropolis Algorithm (ROMMA). This algorithm improves sampling efficiency for inference problems where the prior distribution constrains the posterior. In particular, this is shown to be relevant for problems in geophysics.

We also discuss the application of Bayesian methods to state estimation, disturbance detection, and system identification problems in complex systems. We introduce a Bayesian perspective on learning models and properties of physical systems based upon a layered architecture that can learn quickly and flexibly. We then apply this architecture to detecting and characterizing changes in physical systems with applications to power systems and biology. In power systems, we develop a new formulation of the Extended Kalman Filter for estimating dynamic states described by differential algebraic equations. This filter is then used as the basis for sub-second fault detection and classification. In synthetic biology, we use a Bayesian approach to detect and identify unknown chemical inputs in a biosensor system implemented in a cell population. This approach uses the tools of Bayesian model selection.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Bayesian Inference; Markov Chain Monte Carlo; Sequential Monte Carlo; Dynamic State Estimation; Fault Detection; Input Detection
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Applied And Computational Mathematics
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Beck, James L.
Thesis Committee:
  • Stuart, Andrew M. (chair)
  • Simons, Mark
  • Low, Steven H.
  • Beck, James L.
Defense Date:16 May 2017
Funding AgencyGrant Number
Department of Energy Computational Science Graduate Fellowship (DOE-CSGF)DE-FG02-97ER25308
United States Geological SurveyUNSPECIFIED
Caltech Department of Computing & Mathematical SciencesUNSPECIFIED
Record Number:CaltechTHESIS:06042017-011739366
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for ch. 2 DOIPaper adapted for ch. 5 adapted for ch. 4
Catanach, Thomas Anthony0000-0002-4321-3159
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:10263
Deposited By: Thomas Catanach
Deposited On:05 Jun 2017 23:21
Last Modified:04 Oct 2019 00:16

Thesis Files

PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page