A Caltech Library Service

Learning-Augmented Control and Decision-Making: Theory and Applications in Smart Grids


Li, Tongxin (2023) Learning-Augmented Control and Decision-Making: Theory and Applications in Smart Grids. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/cdf6-0w78.


Achieving carbon neutrality by 2050 does not only lead to the increasing penetration of renewable energy, but also an explosive growth of smart meter data. Recently, augmenting classical methods in real-world cyber-physical systems such as smart grids with black-box AI tools, forecasts, and ML algorithms has attracted a lot of growing interest. Integrating AI techniques into smart grids, on the one hand, provides a new approach to handle the uncertainties caused by renewable resources and human behaviors, but on the other hand, creates practical issues such as reliability, stability, privacy, and scalability, etc. to the AI-integrated algorithms.

This dissertation focuses on solving problems raised in designing learning-augmented control and decision-making algorithms.

The results presented in this dissertation are three-fold. We first study a problem in linear quadratic control, where imperfect/untrusted AI predictions of system perturbations are available. We show that it is possible to design a learning-augmented algorithm with performance guarantees that is aggressive if the predictions are accurate and conservative if they are imperfect. Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We next study the problem of equipping a black-box control policy with model-based advice for nonlinear control on a single trajectory. We first show a general negative result that a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing. We then propose an adaptive λ-confident policy, with a coefficient λ indicating the confidence in a black-box policy, and prove its stability. With bounded nonlinearity, in addition, we show that the adaptive λ-confident policy achieves a bounded competitive ratio when a black-box policy is near-optimal. Finally, we propose an online learning approach to implement the adaptive λ-confident policy and verify its efficacy in case studies about the Cart-Pole problem and a real-world electric vehicle (EV) charging problem with data bias due to COVID-19.

Aggregators have emerged as crucial tools for the coordination of distributed, controllable loads. To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, known as the aggregate flexibility to a system operator. However, most existing aggregate flexibility measures often are slow-timescale estimations and much less attention has been paid to real-time coordination between an aggregator and an operator. In the second part of this dissertation, we consider solving an online decision-making problem in a closed-loop system and present a design of real-time aggregate flexibility feedback, termed the maximum entropy feedback (MEF). In addition to deriving analytic properties of the MEF, combining learning and control, we show that it can be approximated using reinforcement learning and used as a penalty term in a novel control algorithm--the penalized predictive control (PPC) that enables efficient communication, fast computation, and lower costs. We illustrate the efficacy of the PPC using a dataset from an adaptive electric vehicle charging network and show that PPC outperforms classical MPC. We show that under certain regularity assumptions, the PPC is optimal. We illustrate the efficacy of the PPC using a dataset from an adaptive electric vehicle charging network and show that PPC outperforms classical model predictive control (MPC). In a theoretical perspective, a two-controller problem is formulated. A central controller chooses an action from a feasible set that is determined by time-varying and coupling constraints, which depend on all past actions and states. The central controller's goal is to minimize the cumulative cost; however, the controller has access to neither the feasible set nor the dynamics directly, which are determined by a remote local controller. Instead, the central controller receives only an aggregate summary of the feasibility information from the local controller, which does not know the system costs. We show that it is possible for an online algorithm using feasibility information to nearly match the dynamic regret of an online algorithm using perfect information whenever the feasible sets satisfy some criterion, which is satisfied by inventory and tracking constraints.

The third part of this dissertation consists of examples of learning, inference, and data analysis methods for power system identification and electric charging. We present a power system identification problem with noisy nodal measurements and efficient algorithms, based on fundamental trade-offs between the number of measurements, the complexity of the graph class, and the probability of error. Next, we specifically consider prediction and unsupervised learning tasks in EV charging. We provide basic data analysis results of a public dataset released by Caltech and develop a novel iterative clustering method for classifying time series of EV charging rates.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Control; decision-making; machine learning; artificial intelligence; EV charging; smart grid; power systems
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computing and Mathematical Sciences
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Low, Steven H. (advisor)
  • Wierman, Adam C. (co-advisor)
Group:Caltech Rigorous Systems Research Group (RSRG), Netlab
Thesis Committee:
  • Yue, Yisong (chair)
  • Low, Steven H.
  • Wierman, Adam C.
  • Mazumdar, Eric V.
Defense Date:22 June 2022
Funding AgencyGrant Number
NSFECCS 1931662
Resnick Sustainability Institute2021 Impact Grants
Record Number:CaltechThesis:07202022-040725024
Persistent URL:
Related URLs:
URLURL TypeDescription adapted for Chapter II adapted for Chapter IV adapted for Chapter IV adapted for Chapter V adapted for Chapter VI adapted for Chapter VII adapted for Chapter VII
Li, Tongxin0000-0002-9806-8964
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:14980
Deposited By: Tongxin Li
Deposited On:25 Jul 2022 23:14
Last Modified:01 Aug 2022 17:47

Thesis Files

[img] PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page