Distributionally Robust LQG with Kullback-Leibler Ambiguity Sets

Marta Fochesato, Lucia Falconi, Mattia Zorzi, Augusto Ferrante, John Lygeros · 2025

Read Distributionally Robust LQG with Kullback-Leibler Ambiguity Sets by Marta Fochesato, Lucia Falconi, Mattia Zorzi, Augusto Ferrante, John Lygeros (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08370.txt.

Introduction

Marta Fochesato and co-authors harden in 2025 a classic control method against wrong assumptions about noise. The method steers systems that are seen only partly and noisily, and it is famously fragile when the noise model is off. Real plants rarely match the neat noise shapes the theory assumes.

Their robust version guards each time step with a set of allowed noise distributions, bounded by closeness to the assumed one. The controller then plans for the worst case inside that set. The team proves the best policy stays linear, as in the classic method, which keeps it usable. They give an iterative scheme that provably settles. Robust steering of this kind suits drones and grids whose noise is never quite as modeled.

Catalog record

CREATORMarta Fochesato, Lucia Falconi, Mattia Zorzi, Augusto Ferrante, John Lygeros
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08370

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distributionally Robust LQG with Kullback-Leibler Ambiguity Sets Marta Fochesato, Lucia Falconi, Mattia Zorzi, Augusto Ferrante, John Lygeros Abstract The Linear Quadratic Gaussian (LQG) controller is known to be inherently fragile to model misspecifications common in real-world situations. We consider discrete-time partially observable stochastic linear systems and provide a robustification of the standard LQG against distributional uncertainties on the process and measurement noise. Our distributionally robust formulation specifies the admissible perturbations by defining a relative entropy based ambiguity set individually for each time step along a finite-horizon trajectory, and minimizes the worst-case cost across all admissible distributions. We prove that the optimal control policy is still linear, as in standard LQG, and derive a computational scheme grounded on iterative best response that provably converges to the set of saddle points. Finally, we consider the case of endogenous uncertainty captured via decision-dependent ambiguity sets and we propose an approximation scheme based on dynamic programming. I Introduction The Linear Quadratic Regulator (LQG) is a cornerstone of control theory, and has been applied across several domains ranging from engineering, to economics, to computer science. It involves controlling a linear system subject to additive disturbances via the design of a control policy that minimizes a quadratic cost function. For linear systems, quadratic costs and additive Gaussian noise, it is well-known that the resulting optimal policy is linear in the observations. However, in practice, the system dynamics are often only approximately known, and the noise is not necessarily Gaussian. This leads to a mismatch between the nominal model and the true system, which can adversely affect performances [ 1 ] . We consider a generalization of the finite-horizon LQG framework for discrete-time linear stochastic systems subject to model mismatch. We interpret the problem as a zero-sum game between the controller and a distribution chosen from an ambiguity set that may change at each time step. Each ambiguity set is represented as a ball defined in the Kullback-Leibler (KL) divergence centered at a nominal Gaussian distribution. The goal of the control designer is then to synthesize a control policy that minimizes the worst-case expected cost. We refer to this problem as the Distributionally Robust LQG (DR-LQG) problem with KL ambiguity sets. For the DR-LGQ problem, we show that, even under distributional ambiguity, the optimal control policy is linear in the observations, as in the standard LQG. Following [ 2 , 3 ] we re-parametrize the control policy in terms of purified observations and propose a novel "sandwich" argument to show that the DR-LQG is optimally solved by a linear policy and that the worst-case distribution is still Gaussian. We show that the resulting game admits a Nash equilibrium and provide a numerical scheme based on regularized best response that converges linearly to the set of saddle points of the DR-LQG. The best responses admits closed-form expression, making the algorithm computationally attractive. We further consider the case of endogenous uncertainty captured via decision-dependent ambiguity sets, well suited to capture perturbations in the system matrices. We derive an approximated closed-form recursion based on dynamic programming, and use it within a coordinate gradient descent scheme to solve the problem with linear convergence rate guarantees. Finally, we provide numerical simulations to illustrate the effectiveness of our approaches. Literature review Traditionally, the optimal control of systems subject to uncertainty has been studied through the LQR/LQG theory by synthesizing policies that either minimize a quadratic functional in the state and control action pair, or the \mathcal{H}_{2} norm of the system’s transfer function [ 4 ] . Motivated by the fragility of the LQG against model mismatch [ 1 ] , several robustification approaches were suggested in the literature. For example, if the perturbation is modeled as adversarial with known finite energy bound, the \mathcal{H}_{\infty} synthesis approach can be used to minimize the infinity norm of the transfer function mapping disturbances to a measurable performance metric [ 5 ] . While stability is guaranteed under any allowable process noise signal, the \mathcal{H}_{\infty} controller might result in conservative behaviors as it plans for the worst uncertainty. This motivated the development of combined \mathcal{H}_{2}/\mathcal{H}_{\infty} strategies, trying to alleviate part of the conservatism. A different approach is represented by risk-sensitive control that incorporates risk sensitivity into the control design task by minimizing the entropic risk measure of the cost, rather than its expected value [ 6 , 7 ] . Distributional robust control (DRC) is an alternative paradigm that is gaining momentum [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ] . The DRC problem seeks a control policy that minimizes the expected cost under the worst-case noise distribution in an ambiguity set, i.e., a set of probability distributions among which we can reasonably expect to find the true noise distribution. DRC robustifies against model misspecifications in the space of probabilities. Different types of ambiguity sets have been proposed in the literature based on moment constraints [ 8 ] , total variation distance [ 16 ] , Wasserstein distance [ 11 ] (or, more broadly, optimal transport distances), and \phi- divergences [ 12 ] . In [ 2 ] a DR-LQG problem similar to the one considered here is solved by considering ambiguity sets defined on the basis of the Wasserstein distance. Compared to [ 2 ] , (i) we motivate the choice of the KL distance by providing interesting connections with system identification and risk measure theory, (ii) we prove optimality of the linear policies for the DR-LQG problem with KL ambiguity sets, and (iii) we provide a novel best response algorithm that exploits the availability of closed-form expressions for the best responses, unlike the gradient-based method suggested in [ 2 ] that relies on automatic differentiation. The dynamic-programming based recursion for decision-dependent ambiguity sets, is related to the approaches in [ 13 ] and [ 9 ] . These works address a regularized relaxed problem with a fixed penalty value in the objective function; hence, there are no guarantees that the worst-case distribution, against which the controller is hedging, belongs to the specified ambiguity set. On the contrary, our formulation addresses the exact constrained formulation. Additionally, in [ 13 ] and [ 9 ] distributed uncertainty modeling is not considered. Finally, in [ 13 ] and [ 9 ] the problem is solved by neglecting a crucial non-linear dependence in the optimality equations; we tackle this problem by linearizing this dependence (instead of disregarding it) with obvious advantages in terms of performances, as demonstrated in the numerical section. Outline In Section II we present the necessary background, while in Section III we describe the problem statement. In Section IV the theoretical properties of the DR-LQG are analyzed and in Section V a computational framework is presented. Section VI considers the case with endogenous ambiguity sets, while in Section VII simulation results are reported. Finally, we draw our conclusions in Section VIII. We relegate all proofs to the appendix. II Preliminaries Notation \mathbb{R}^{n},\mathbb{R}^{n}_{+},\mathbb{R}^{n}_{++} denote the set of reals, nonnegative reals, and positive reals, respectively. \mathbb{S}^{n}_{+} (resp. \mathbb{S}^{n}_{++} ) denotes the cone of n\times n symmetric positive semi-definite (resp. positive definite) matrices. For any A,B\in\mathbb{S}_{+}^{n} , the relation A\succeq B ( A\succ B ) means that A-B\in\mathbb{S}^{n}_{+} ( A-B\in\mathbb{S}^{n}_{++}

Read the complete text