A Caltech Library Service

Understanding and Improving Reliability of Inference Dynamics in Deep Neural Networks


Huang, Yujia (2024) Understanding and Improving Reliability of Inference Dynamics in Deep Neural Networks. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/77cc-6z86.


Reliability is a crucial aspect for the successful deployment of deep learning systems across various domains. In generative modeling, it is essential to create content that adheres to specific rules. In the field of control, ensuring that robots operate safely without falling or entering hazardous areas is paramount. Similarly, in visual perception, the robustness of perception results against perturbations are vital.

In this thesis, we explore the reliability of inference dynamics in deep neural networks such as ResNet, neural Ordinary Differential Equations (ODEs), and diffusion models. We begin by examining the inference dynamics in standard networks with a discrete sequence of hidden layers, applying self-consistency and local Lipschitz bounds to enhance robustness against input perturbations. Our exploration then extends to neural ODEs, where the neural network specifies a vector field that continuously transforms the state. We employ forward invariance to achieve robustness, marking the first instance of training neural ODE policies with non-vacuous certified guarantees. The focus shifts next to diffusion models and their inference processes, particularly in adhering to symbolic constraints. For this, we introduce a novel sampling algorithm inspired by stochastic control principles. This algorithm not only guides these models in generating rule-specific content but also sets a new benchmark in symbolic music generation. Our work offers a cohesive understanding of inference dynamics in various deep learning architectures and propose new algorithms to significantly improve their reliability.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Neural Networks; Reliability; Robustness; neural ODE; Forward Invariance; Diffusion Models; Rule Guidance; Stochastic Control
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Electrical Engineering
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Yue, Yisong
Thesis Committee:
  • Bouman, Katherine L. (chair)
  • Wierman, Adam C.
  • Gkioxari, Georgia
  • Shi, Yuanyuan
  • Yue, Yisong
Defense Date:8 January 2024
Non-Caltech Author Email:yujiahuang16 (AT)
Funding AgencyGrant Number
Information Science and Technology (IST) initiative at CaltechUNSPECIFIED
Record Number:CaltechTHESIS:03052024-025023714
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for Chapter 2 adapted for Chapter 3 adapted for Chapter 4 adapted for Chapter 5
Huang, Yujia0000-0001-7667-8342
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:16314
Deposited By: Yujia Huang
Deposited On:13 Mar 2024 20:15
Last Modified:20 Mar 2024 16:38

Thesis Files

[img] PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page