A Distributed Bilevel Framework for the Macroscopic Optimization of Multi-Agent Systems
Riccardo Brumali, Guido Carnevale, Sonia Martínez, Giuseppe Notarstefano · 2026
Read A Distributed Bilevel Framework for the Macroscopic Optimization of Multi-Agent Systems by Riccardo Brumali, Guido Carnevale, Sonia Martínez, Giuseppe Notarstefano (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11712.txt.
Introduction
Brumali and three colleagues propose a distributed algorithm tuning macroscopic behavior of multi-agent systems via small actions. They cast the task as bilevel optimization: the upper level sets a large-scale state through a score, shaped below by a compressed summary. The large-scale state uses an exponential-family distribution from small-scale setups. Agents rebuild the large-scale state through distributed estimation, with hypergradient-based updates of small states to improve group action. Convergence to stationary points follows via split timescales, and simulations confirm it.
Catalog record
| CREATOR | Riccardo Brumali, Guido Carnevale, Sonia Martínez, Giuseppe Notarstefano |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49202 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11712
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Distributed Bilevel Framework for the Macroscopic Optimization of Multi-Agent Systems Riccardo Brumali, Guido Carnevale, Sonia Martínez, Giuseppe Notarstefano Abstract In this paper, we propose a novel distributed algorithm to optimize the emergent macroscopic behavior of large-scale multi-agent systems via microscopic actions. We cast this task as a bilevel optimization problem, where the upper level formalizes the desired macroscopic target behavior through a suitable performance criterion, which is shaped in the lower level by leveraging a compressed aggregate representation estimating the macroscopic state. More precisely, the macroscopic state is parametrized by an exponential-family of distributions and constructed from the multi-agent microscopic configuration. The proposed algorithm integrates a distributed estimation mechanism, through which each agent reconstructs the macroscopic state locally, with a hypergradient-based update of the microscopic states aimed at improving the collective macroscopic behavior. We prove convergence to the set of stationary points of the bilevel problem via timescale separation arguments. Numerical simulations validate the effectiveness of the proposed method. I INTRODUCTION Large-scale multi-agent systems are becoming increasingly common in various domains, especially for multi-robot applications [ 9 , 35 ] . For these systems, agents interact with each other in order to achieve personal and collective goals, giving rise to complex emergent behaviors at the macroscopic level. The design of strategies to optimize or impose a desired emergent macroscopic behavior is an important challenge currently studied across several communities [ 10 ] . A common approach to this problem follows the idea where the microscopic dynamics of the agents are approximated by Partial Differential Equations (PDEs) describing the macroscopic behavior of the system. Then, the algorithm design is performed at the PDE level and the resulting strategy is implemented microscopically by means of a suitable discretization [ 11 , 26 , 27 ] . A direct example of this method is presented in [ 8 ] , where a large-scale shepherding problem [ 25 ] is addressed. Other applications of this approach include motion planning [ 28 ] , sensor placement [ 15 ] , and coverage control [ 22 , 36 ] . Tightly connected to this strategy, other works cast the problem as an Optimal Transport problem [ 1 , 21 , 34 , 13 ] . In contrast, in [ 5 , 4 ] a different solution is proposed, as agents directly optimize their microscopic state while simultaneously learning the macroscopic behavior. For large-scale systems, centralized strategies where a single entity controls all the agents are becoming impractical. Consequently, distributed methods using local computation and neighboring communication are becoming increasingly popular [ 16 , 29 , 37 ] . In this setting, a key aspect concerns the estimation of the macroscopic state which is unknown to individual agents. We introduce a distributed framework to solve simultaneously both the estimation and optimization tasks leveraging a bilevel optimization formulation [ 39 ] . In this context, centralized solution strategies for bilevel optimization problems have been proposed in [ 14 , 33 , 20 , 24 ] , while examples of distributed approaches can be found in [ 38 , 12 , 31 ] . The main contribution of this paper is twofold. First, we formalize this problem by introducing a distributed bilevel optimization framework tailored for emergent behaviors described by Probability Density Functions (PDFs) that realize a given microscopic configuration. In our formulation, the lower level estimates a compressed lower-dimensional representation of the macroscopic state from the microscopic configuration, while the upper level enforces a desired macroscopic behavior as a function of the compressed macroscopic state. By modeling this state as the parameters of an exponential-family distribution, we cast the estimation as a regularized maximum likelihood estimation problem. Second, we propose BILevel Distributed hypergradient for MACRoscopic Optimization (BILD-MACRO), a fully distributed method to solve the proposed distributed bilevel formulation. It integrates a distributed estimation mechanism to locally learn the macroscopic state in each agent, together with a hypergradient-based update on the microscopic states to optimize the emergent behavior. In turn, we interconnect the estimation and optimization tasks via a consensus mechanism. This allows agents to locally reconstruct global quantities from the estimation problem that are required for hypergradient computation. We highlight that our approach reduces the communication burden with respect to [ 5 ] . Indeed, exchanges involve only quantities related to a compressed macroscopic representation instead of a full PDF. The paper is organized as follows. In Section II we state the problem setup, in Section III we present the BILD-MACRO algorithm and its convergence properties, while in Section IV we theoretically analyze our method. Then, in Section V we validate our strategy through numerical simulations. Notation The symbol \text{col}(x_{1},\dots,x_{N}) denotes the column stacking of the vectors x_{1},\dots,x_{N} . \text{diag}(M_{1},\dots,M_{N}) denotes the block diagonal matrix with matrices M_{1},\dots,M_{N} on the diagonal. I_{m} denotes the identity matrix in \mathbb{R}^{m\times m} . 1_{N} and 0_{N} denote the vectors of N ones and N zeros, respectively, while \otimes is the Kronecker product. Dimensions are omitted when they are clear from the context. We denote the set of positive real numbers as \mathbb{R}_{+} . Given v\in\mathbb{R}^{n} and M\in\mathbb{R}^{n\times n} , \left\lVert v\right\rVert is the Euclidean norm of v , while \left\lVert M\right\rVert_{F} is the Frobenius norm of M . Given f:\mathbb{R}^{n}\times\mathbb{R}^{m}\to\mathbb{R}^{n} , \nabla_{1}f(x,y)\in\mathbb{R}^{n} and \nabla_{2}f(x,y)\in\mathbb{R}^{m} are the gradients of f with respect to its first and second arguments evaluated at (x,y) , respectively. Then \nabla_{1,2}^{2}f(x,y)\in\mathbb{R}^{n\times m} and \nabla_{2}^{2}f(x,y)\in\mathbb{R}^{m\times m} are the mixed and second-order gradients with respect to the second argument of f , respectively. \mathbb{E}[\cdot] and \text{cov}[\cdot] are the expectation and covariance operators. II Problem Setup and Bilevel Reformulation We consider a large-scale multi-agent system composed of N cooperating agents that want to organize themselves to achieve a desired macroscopic behavior. Agents can access only their local information and can communicate with each other following the edges of a directed graph \mathcal{G}\coloneq\left(\mathcal{I},\mathcal{E},\mathcal{A}\right) . In particular, we consider \mathcal{I}\coloneq\{1,\dots,N\} as the set of nodes, i.e., the agents, \mathcal{E}\subset\mathcal{I}\times\mathcal{I} as the set of edges, and \mathcal{A}\in\mathbb{R}^{N\times N} as the weighted adjacency matrix such that the (i,j) -entry a_{ij}>0 if (i,j)\in\mathcal{E} , otherwise a_{ij}=0 . We define the set of in-neighbors of agent i as \mathcal{N}_{i}\coloneq\{j\in\mathcal{I}\mid(j,i)\in\mathcal{E}\} . The next Assumption formalizes the class of graphs we consider. Assumption 1 . The graph \mathcal{G} is strongly connected and the weighted adjacency matrix \mathcal{A} is doubly stochastic. \square For each agent i , we identify with x_{i}\in\mathcal{C}_{i}\subset\mathbb{R}^{n_{i}} its microscopic state and we denote with x\coloneq\text{col}(x_{1},\dots,x_{N})\in\mathbb{R}^{n} , where n\coloneq\sum_{i=1}^{N}n_{i} , the total microscopic configuration of the system. We formalize this problem as \displaystyle\min_{x\in\mathcal{C}}\int_{\mathcal{C}}J(\rho(c,x))\mathrm{d}c, (1) where \mathcal{C}\coloneq\mathcal{C}_{1}\times\dots\times\mathcal{C}_{N} , J:\mathbb{R}_{+}\to\mathbb{R} is a performance criterion, and \rho:\mathcal{C}\times\mathbb{R}^{n}\to\mathbb{R}_{+} is a function describing the macroscopic behavior of the system. II-A Bilevel Reformulation Being the function \rho(\cdot,x) generally unknown, we introduce p:\mathcal{C}\times\mathbb{R}^{m}\to\mathbb{R}_{+} , such that \displaystyle\rho(c,x)\approx p(c,\theta(x)), where \theta:\mathbb{R}^{n}\to\mathbb{R}^{m} is a lower-dimensional representation of the microscopic configuration x , while p(\cdot,\theta(x)) is a parametrized family of functions describing the macroscopic behavior of the system. For example, \theta(x) can be the result of a model reduction or feature encoding operator, see also Example 1 . We call \theta(x) the macroscopic state, as it is a compressed representation of the macroscopic behavior of the system. Indeed, we consider cases where m<n . Thus, we