Persistently Exciting Online Feedback Optimization Controller with Minimal Perturbations*
Tore Gude, Marta Anna Zagorowska, Lars Struen Imsland · 2025
Read Persistently Exciting Online Feedback Optimization Controller with Minimal Perturbations* by Tore Gude, Marta Anna Zagorowska, Lars Struen Imsland (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19910.txt.
Introduction
Tore Gude, Marta Zagorowska, and Lars Imsland refine feedback optimization in twenty twenty-five. Online feedback optimization steers a process toward its best operating point using measurements. It needs the process's sensitivity, which is often unknown. Random perturbations are the usual way to estimate it, but they push the process off course.
Their controller generates persistently exciting inputs with minimal deviation from the descent direction. The design is a bilevel optimization program. A nonconvex rank constraint is relaxed with linear constraints and penalties. Validation uses a simulated resource allocation problem modeled on oil and gas production. Profit matched the random-perturbation controller while inputs were perturbed less. Readers in process control get gradient estimates bought with smaller disturbances to the plant.
Catalog record
| CREATOR | Tore Gude, Marta Anna Zagorowska, Lars Struen Imsland |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19910
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Persistently Exciting Online Feedback Optimization Controller with Minimal Perturbations* Tore Gude, Marta Anna Zagorowska, Lars Struen Imsland Abstract This paper develops a persistently exciting input generating Online Feedback Optimization (OFO) controller that estimates the sensitivity of a process ensuring minimal deviations from the descent direction while converging. This eliminates the need for random perturbations in feedback loop. The proposed controller is formulated as a bilevel optimization program, where a nonconvex full rank constraint is relaxed using linear constraints and penalization. The validation of the method is performed in a simulated scenario where multiple systems share a limited, costly resource for production optimization, simulating an oil and gas resource allocation problem. The method allows for less input perturbations while accurately estimating gradients, allowing faster convergence when the gradients are unknown. In the case study, the proposed method achieved the same profit compared to an OFO controller with random input perturbations, and 1.4 % 1.4\% higher profit compared to an OFO controller without input perturbations. I Introduction Increasingly complex industrial processes must be optimally operated to be competitive and minimize environmental impacts. The availability of sensor data, computational power, and advances in machine learning motivate utilizing real-time measurements to find the optimal operating point of a process even in the presence of model-plant mismatch and external disturbances. To compensate for model-plant mismatch, the measurements must contain enough information about the system, formalized through persistency of excitation [ 1 ] . In this work, we propose a method for generating persistently exciting inputs while minimally disturbing an Online Feedback Optimization (OFO) controller. Persistently exciting input signals are necessary for accurate model estimation, which is used by the OFO controller. The proposed controller is tested in a simulated scenario in which multiple systems utilize the same shared, limited, resource for production optimization, reflecting resource allocation challenges in the oil and gas industry. An OFO controller optimizes a system in a feedback loop [ 1 ] . Using optimization in feedback loops has recently been investigated and has shown promising results for optimal power system operation [ 2 ] . OFO controllers that generate persistently exciting input have also been investigated in [ 1 ] and [ 3 ] . However, these approaches involve random perturbations and do not consider possible adverse effects of these perturbations. An overview of other methods that combine optimization with feedback control and their relation to OFO is provided in [ 4 ] , including a comparison of real-time optimization methods such as extremum seeking and modifier adaptation. Furthermore, an overview of extremum seeking approaches including treatment of persistent excitations is given in [ 5 ] . Our contribution is a persistently exciting input generating OFO controller with minimal perturbations while converging toward the optimal operating point. This approach eliminates the need for random perturbations, which an operator or production engineer may be reluctant to introduce into a critical feedback loop. For the controller to efficiently find a solution, a nonconvex full rank constraint is relaxed using linear constraints and penalization. While this looses the theoretical guarantee for generating persistently exciting inputs, it renders the problem efficient to solve and demonstrates effectiveness in the case study. The remainder of this paper is organized as follows: In Section II we present preliminary theory of OFO and default gradient estimation. In Section III , we present our persistently exciting input generating OFO controller, with case study results given in Section IV . Finally, the paper is concluded in Section V . II Preliminaries II-A Online Feedback Optimization The recent research in optimization in closed loop is based on three main benefits, 1) optimize systems with inaccurate data and time-varying parameters, 2) obey constraints without an accurate model of the whole system and 3) eliminate the need for exogenous setpoints and reference signals [ 4 ] . OFO is a real-time optimization scheme that solves an optimization problem by iteratively driving a system closer to the optimal operating point. This is done utilizing steady-state measurements in a feedback loop instead of a model of the system. Given a general control system with inputs u\in\mathbb{R}^{n_{u}} , outputs y\in\mathbb{R}^{n_{y}} and a steady-state mapping y=h(u) , OFO solves the following optimization problem, \displaystyle\min_{u,y} \displaystyle\quad\Phi\left(u,y\right) (1a) s.t. \displaystyle u\in\mathcal{U} (1b) \displaystyle y\in\mathcal{Y} (1c) \displaystyle y=h\left(u\right), (1d) where \Phi:\mathbb{R}^{n_{u}}\times\mathbb{R}^{n_{y}}\rightarrow\mathbb{R} is a continuously differentiable objective function, h:\mathbb{R}^{n_{u}}\rightarrow\mathbb{R}^{n_{y}} is a continuously differentiable nonlinear input-output mapping and \mathcal{U} and \mathcal{Y} are the sets of input and output constraints, respectively. To solve the optimization problem ( 1 ), OFO typically actuates the real system in the direction of the negative descent of the objective function until a local minimum of ( 1 ) is found. To account for input and output constraints, in this work OFO uses projected gradient descent, which projects the gradient onto the closest direction in the tangent cone, \mathcal{T}_{u} , w_{t}=\underset{w\in\mathcal{T}_{u}}{\mathrm{arg\,min}}\lVert w+\nabla_{u}\Phi^{\top}\left(u,y\right)\rVert^{2}_{2}, (2) ensuring safe and feasible operation while converging [ 6 ] . Then the solution to ( 2 ), w_{t} , is applied as an input actuation u_{t+1}=u_{t}+\alpha w_{t}, (3) where \alpha is a constant stepsize and \nabla_{u}\Phi^{\top}\left(u,y\right)=\frac{\partial\Phi^{\top}\left(u,y\right)}{\partial u}+\nabla_{u}h(u)^{\top}\frac{\partial\Phi^{\top}\left(u,y\right)}{\partial y} (4) is the gradient of the objective function with respect to u . An analysis of the impact of the step size \alpha in OFO was done by [ 6 ] . When the input-output mapping ( 1d ) is explicitly known, the derivatives in ( 4 ) can be found analytically. To obtain the descent direction ( 4 ) given an unknown input-output mapping y=h(u) , the Jacobian, \nabla_{u}h needs to be estimated [ 1 ] . It can be shown that OFO will converge to an optimal operating point with the estimated \nabla_{u}h and with disturbances affecting the system [ 7 ] . II-B Gradient Estimation For estimation of \nabla_{u}h , we use a similar approach as in [ 1 ] where its column-wise vector representation \hat{h}_{t} , is estimated through a recursive least-squares algorithm, \displaystyle K_{t-1}= \displaystyle\Sigma_{t-1}U^{\top}_{\Delta,{t-1}}\left(\Sigma_{m,t-1}+U_{\Delta,t-1}\Sigma_{t-1}U_{\Delta,t-1}^{\top}\right)^{-1} (5a) \displaystyle\hat{h}_{t}= \displaystyle\hat{h}_{t-1}+K_{t-1}\left(\Delta y_{t-1}-U_{\Delta,t-1}\hat{h}_{t-1}\right) (5b) \displaystyle\Sigma_{t}= \displaystyle(I_{n_{u}n_{y}}-K_{t-1}U_{\Delta,t-1})\Sigma_{t-1}(I_{n_{u}n_{y}}-K_{t-1}U_{\Delta,t-1})^{\top} \displaystyle\quad+K_{t-1}\Sigma_{m,t-1}K_{t-1}^{T}+\Sigma_{p,t-1}, (5c) where \Delta y_{t-1}=y_{t}-y_{t-1} , \Delta u_{t-1}=u_{t}-u_{t-1} , U_{\Delta,t-1}=\Delta u_{t-1}\otimes I_{n_{y}} , where \otimes is the Kronecker product, I_{n_{y}} is the identity matrix of dimension n_{y} , \Sigma_{p,t}=\Sigma_{p1}+\Sigma_{p2}\lVert\Delta u_{t}\rVert_{2}^{2} is the covariance matrix of the Gaussian process noise and \Sigma_{m,t}=\Sigma_{m1}+\Sigma_{m2}\lVert\Delta u_{t}\rVert_{2}^{2}+\Sigma_{m3}\lVert\Delta u_{t}\rVert_{2}^{4} is the covariance matrix of the Gaussian measurement noise. The parts \Sigma_{p2},\Sigma_{m2} and \Sigma_{m3} of the covariance matrices are proportional to \lVert\Delta u_{t}\rVert_{2} , so that larger variations in input are associated with greater uncertainty. The covariance matrices also have parts independent of the input, \Sigma_{p1} and \Sigma_{m1} to account for uncontrolled random changes. In this paper, \Sigma_{m,0} and \Sigma_{m,0} are initialized as identity and \hat{h}_{0} as a vector of ones. III Persistently Exciting OFO Controller To ensure accurate estimation of \nabla_{u}h , the inputs \Delta u need to be persistently exciting, enabling exploration of a range of outputs \Delta y [ 1 ] . III-A Random Perturbations In the OFO formulation from [ 1 ] , the descent direction ( 4 ) is perturbed with Gaussian noise, s_{t}\sim\mathcal{N}\left(0,\sigma\right) , to ensure persistency of excitation with high probability. Combining the OFO controller ( 2 ) from [ 6 ] with the perturbation from [ 1 ] the following formulation is an OFO controller that generates persistently