Markov matrix perturbations to optimize dynamical and entropy functionals
Manuel Santos Gutiérrez, Niccolò Zagli, Giulia Carigi · 2025
Read Markov matrix perturbations to optimize dynamical and entropy functionals by Manuel Santos Gutiérrez, Niccolò Zagli, Giulia Carigi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-14040.txt.
Introduction
Manuel Santos Gutiérrez, Niccolò Zagli, and Giulia Carigi study how to push a random process toward a desired response in this two thousand twenty-five paper. Many systems, from fluids to markets, are modeled as Markov chains that hop between states. A key question is which outside push yields the largest change in a chosen measure. The authors connect perturbation theory for Markov matrices with linear response theory from physics.
From that link they derive fast linear algorithms. The targets include entropy, a divergence measure, and entropy production. They test the methods on chain models of flows, both settled and driven out of equilibrium. The tools could help steer systems with minimal force. Applied mathematicians and statistical physicists are the audience.
Catalog record
| CREATOR | Manuel Santos Gutiérrez, Niccolò Zagli, Giulia Carigi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-14040
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Markov matrix perturbations to optimize dynamical and entropy functionals Manuel Santos Gutiérrez, Niccolò Zagli, Giulia Carigi Abstract An important problem in applied dynamical systems is to compute the external forcing that provokes the largest response of a desired observable quantity. For this, we investigate the perturbation theory of Markov matrices in connection with linear response theory in statistical physics. We use perturbative expansions to derive linear algorithms to optimize physically relevant quantities such as: entropy, Kullback-Liebler-divergence and entropy production of Markov matrices and their related probability vectors. These optimisation algorithms are applied to Markov chain representations of discrete and continuous flows in and out of equilibrium. We consider Markov matrix representations originating from Ulam-type approximations of transfer operators and a reduced order model of a turbulent flow based on unstable periodic orbits theory. We also propose a numerical protocol to recast matrix perturbations into vector field perturbations. The results allow to physically interpret the obtained optimizing perturbations without knowledge of the underlying equations, in a data-driven way. 1 Introduction The perturbation theory of Markov matrices has extensively been developed over the past years, drawing essential contributions from diverse scientific disciplines like probability [ 41 ] , population dynamics [ 8 ] and statistical physics [ 38 ] , to name a few. The ubiquity of Markov matrices, especially in applications, can surely be associated with their simplicity in linking time-evolving systems with algebraic concepts. In fact such matrices represent the dynamics of coarse-grained dynamical systems, composed of finitely many states, and studying their linear algebraic structure provides information about their evolution and long-term properties. The dependence of these algebraic features with respect to the different model parameters is crucial to understand the sensitivity and stability of the system, and identify which states are more robust to perturbations. A fundamental application of Markov matrices is in the numerical analysis and approximation of ergodic properties of dynamical systems [ 60 , 14 ] . Generally, a deterministic system obeying an ordinary differential equation (ODE) is equivalent to a linear operator acting on observable function spaces, as originally realized by B. O. Koopman [ 25 ] . Discretizing such an operator— or its dual conterpart, namely, the Liouville operator [ 27 ] — yields Markov matrix representations, whose spectral properties approximate those of the full operator. A similar procedure is carried out when dealing with stochastic differential equations (SDEs). Here the operator to approximate is the Fokker-Planck operator which describes the evolution of probability density functions on the phase space [ 45 ] . When an external forcing is applied to a time-evolving system its statistical properties are susceptible to changing. Linking forcing, response and the system’s unforced variability is an ongoing endeavor in statistical physics [ 37 ] . In this regard, linear response theory provides the fundamental fluctuation dissipation theorems applicable— with corresponding variations— in equilibrium, near-equilibrium [ 26 ] and non-equilibirum systems [ 19 , 46 , 20 , 7 ] . Generally, these theorems relate the response of the forced system to perturbations with its natural fluctuations in the shape of correlation functions. However, to establish if a system exhibits linear response or not is far from being a simple feat and much effort is still put in studying different systems, both theoretically and numerically. Recently, Markov matrix discretizations of dynamical systems have been used for the computation of response operators in continuous systems out of thermodynamic equilibrium [ 31 , 48 , 32 ] . Novel results established that suitable discretizations of the Koopman operator lead to an efficient and interpretable application of the fluctuation dissipation theorem to investigate response properties of non-equilibrium systems [ 63 , 33 ] . A natural inverse question would then be, how to perturb the system to get a desired response of the system. There has been recent developments in this direction [ 1 , 15 , 18 , 50 , 16 ] , starting from [ 2 ] where finite-state Markov chains are considered. The authors provide a stategy to select an optimal perturbation so as to maximize the linear response of the equilibrium distribution of the system, the expectation of a specified observable, and the rate of convergence of the system to the equilibrium distribution. In this work, we aim to investigate other functionals related to the probabilistic and thermodynamic structure of dynamical systems, such as entropy, Kullback-Liebler (KL) divergence, and entropy production. These quantities encode key features of the dynamics—uncertainty, divergence from a reference process, and irreversibility—and optimizing them provides insights into the underlying mechanisms that govern non-equilibrium behavior. A first motivation for addressing this problem comes from climate science, where one would be interested in determining the forcing to the Earth system that would create more information with respect to the present climatology [ 36 ] . Secondly, a seemingly unrelated motivation stems from Markov Chain Monte Carlo [ 43 , Sec. 4.8] , where one would seek to sample a prescribed probability measure by suitably modifiying a Markov matrix so that its invariant measure remains unchanged, yet the rate of convergence to stationarity is maximized. Both of these problems, among other shown in the main text, are here framed in the dynamical systems perspective and are linked to Markov matrix linear algebra. Linking perturbations at the Markov matrix level with perturbations at the vector field domain is a problem that also motivates the present work. While the path from continuous dynamics to matrices is granted by Ulam’s method [ 60 , 14 ] , the reverse process is much less discussed in the context of response to forcings. Divising optimal perturbations for statistical quantities via algebraic methods is effective, but their physical interpretation would be eased by tracing back the shape of the vector field that yields such a perturbation. Consequently, the effects of optimizing statistical quantities crucially depend on the nature of the system under scrutiny. For such reason, in this paper, low-dimensional systems are investigated to cover a range of dynamical processes: discrete maps— Lanford map—, equilibrium and non-equilibrium stochastic dynamics— double well potential dynamics without and with applied rotation— and chaotic flows— the Lorenz 63 system [ 30 ] —. On top of this, attempting to escalate the methods to higher dimensions, coarse grained representations of turbulent flows are analyzed via transition rates between unstable periodic orbits [ 62 , 34 ] . The paper is organized as follows. In Sect. 2 we review the perturbation theory of Markov matrices and conceptually link it with the theory of linear response. The main theoretical results of this work are reported in Sect. 3 , where each of the three subsections within address the problems of optimizing the three entropy functionals mentioned earlier in this paragraph. In Sect. 4 we briefly review how to discretize à la Ulam the Liouville (Fokker-Plank) operator of an ODE (SDE) to numerically approximate ergodic features of the system using Markov matrices. We also explain a simple algorithm of how to numerically reconstruct from perturbations of the Markov Chain an associated effective drift perturbations at the SDE level. Five systems are numerically analyzed in Sect. 5 , all of them being of different nature. We show how the optimisation problems grant