An Efficient Data-Driven Framework for Linear Quadratic Output Feedback Control Corresponding author: Yuan-Hua Ni, and Yiqin Yang
Jun Xie Yuan-Hua Ni Yiqin Yang Bo Xu · 2025
Read An Efficient Data-Driven Framework for Linear Quadratic Output Feedback Control Corresponding author: Yuan-Hua Ni, and Yiqin Yang by Jun Xie Yuan-Hua Ni Yiqin Yang Bo Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20748.txt.
Introduction
Jun Xie, Yuan-Hua Ni, Yiqin Yang, and Bo Xu give a data driven frame for output feedback control in two thousand twenty-five. Linear quadratic control with hidden states fits real plants best. The system matrix is often unknown in those plants. No general method yet trades optimality against feasibility well.
Their frame swaps unknown states for substitute states from data. Tests of controllability show when the swap can stand in fully. Two model free algorithms then solve for the best controller. They carry theory guarantees on their answers. Multi output cases that break past learning methods work here.
Catalog record
| CREATOR | Jun Xie Yuan-Hua Ni Yiqin Yang Bo Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 118 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-20748
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An Efficient Data-Driven Framework for Linear Quadratic Output Feedback Control Corresponding author: Yuan-Hua Ni, and Yiqin Yang Jun Xie Yuan-Hua Ni Yiqin Yang Bo Xu Abstract Linear quadratic regulator with unmeasurable states and unknown system matrix parameters better aligns with practical scenarios. However, for this problem, balancing the optimality of the resulting controller and the leniency of the algorithm’s feasibility conditions remains a non-trivial challenge, as no well-established general method has yet been developed to address this trade-off. To address this gap, this study first develops a comprehensive theoretical framework for state parameterization that equivalently substitutes for unknown states. By analyzing the controllability of consistent systems satisfied by substitute states, this framework quantifies the capability of substitute state data matrices to parameterize unknown closed-loop systems and output feedback controllers, thereby constructing a modified state parameterization form that meets the complete data parameterization condition of Willems’ Fundamental Lemma. Leveraging this framework, this study proposes efficient model-free off-policy policy iteration and value iteration algorithms with theoretical guarantees to solve for the optimal output feedback controller, while eliminating reliance on the traditional least squares numerical solution paradigm. Compared with existing studies, particularly for multi-output problems where existing model-free reinforcement learning algorithms may fail, the proposed method removes redundant information in substitute states and the additional full row rank condition on regression matrices, thereby ensuring the solution of optimal output feedback controllers equivalent to optimal state feedback controllers for multi-output systems. Furthermore, this study pioneers a comprehensive and highly scalable theoretical analysis of state parameterization from a data-driven viewpoint, and the proposed algorithms exhibit significant advantages in implementation conditions, data demand, unknown handling, and convergence speed. Keywords: direct data-driven control, reinforcement learning, linear-quadratic optimal control, output feedback, state parameterization. 1 Introduction Linear quadratic regulator (LQR) is a foundational optimal control method that enables stable and optimal control of linear systems by optimizing a quadratic performance index under measurable states. It plays a critical role in ensuring precise and efficient process control across key fields, such as robotic control [ 1 ] , aircraft control [ 2 ] , and autonomous driving [ 3 ] . However, in practice, LQR faces two realities that are more aligned with real-world applications yet are inherently challenging: 1) the matrix parameters of linear systems are seldom known a priori; 2) system states often cannot be directly measured due to cost-effectiveness and engineering practicality constraints (for instance, only output data is transmitted in remote network transmission to protect sensitive information [ 4 ] ). These realities underscore the significant practical value of research on model-free output feedback LQR. Moreover, there exist key theoretical hurdles that need to be addressed. First, since system states are unmeasurable, the state-feedback-related Riccati equation that characterizes the optimal LQR controller becomes unsolvable. In such cases, even if controllers are derived via static output feedback (SOF) or dynamic output feedback under the separation principle [ 5 ] , they do not necessarily yield the optimal solution to LQR problem [ 6 , 7 , 8 ] . Furthermore, unknown system matrix parameters add a further layer of difficulty to the theoretical solution. In this combined scenario, independently designing state observers for dynamic output feedback controllers becomes extremely challenging. Notably, while some model-free algorithms adopt policies that redundantly utilize output data and can yield optimal solutions, their convergence conditions for multi-output (MO) systems remain overly strict, even to the point of being unachievable. Some existing data-driven methods have attempted to address this LQR problem with unmeasurable states and unknown system parameters. However, first-principles modeling or system identification is prone to complex operations and high costs; furthermore, the lack of a separation principle between system identification and model-based controller design may result in suboptimal control policies [ 9 ] . Utilizing policy optimization algorithms in reinforcement learning (RL) to solve optimal SOF or dynamic controllers encounters disconnected feasible regions and complex optimization structures, making it difficult to guarantee algorithmic optimality [ 6 , 7 ] . While calculating substitute vectors for unmeasurable states and directly applying them to RL-based policy iteration (PI) or value iteration (VI) algorithms can ensure the optimality, such approaches remain largely infeasible for MO problems, which stems from their neglect of the intrinsic structure among available data and reliance on least squares (LS) numerical methods [ 10 , 8 ] . Within the behavioral systems framework [ 11 ] , characterizing unknown systems and controllers via Willems’ Fundamental Lemma [ 12 ] enables more comprehensive integration of data information to overcome limitations of traditional methods; however, to date, such techniques still struggle to extend to MO scenarios [ 13 ] . To address these gaps, this paper proposes general and efficient direct data-driven algorithms that address the potential failure of the aforementioned RL algorithms in MO problems and extend the applicability of Willems’ Fundamental Lemma to MO settings, while ensuring controller optimality. More specifically, to obtain an efficient substitute form for unmeasurable states, we first develop a comprehensive and highly scalable state parameterization framework along with its corresponding control theory. Within this framework, we revisit two types of equivalence relationships between substitute state vectors (generated from input-output data) and unmeasurable states, and focus on analyzing the capacity of the substitute state data matrices to parameterize unknown closed-loop systems and output feedback controllers. This analysis yields a key conclusion: the data parameterization ability is strongly correlated with the controllability of the consistent systems exhibited by the substitute states. Building on this conclusion, we eliminate redundant information in substitute states and propose a modified state parameterization method that not only ensures complete data parameterization but is also guaranteed to apply to MO problems, thereby enabling the feasibility of Willems’ Fundamental Lemma in MO settings. Second, utilizing the substitute state data matrix generated via the proposed state parameterization, we re-represents the Bellman iteration equation and further proposes off-policy PI and VI algorithms for solving the output feedback LQR problem. These algorithms exhibit higher efficiency in terms of data demand, handling of unknowns, and convergence speed. Moreover, as we leverage the data matrix to fully exploit the representation capability of Willems’ Fundamental Lemma, there is no need to convert the iteration equations into LS problems for solution, which avoids additional data requirements for regression matrices—requirements that are difficult to satisfy in MO settings. 1.1 Related Works (i) Behavioral Systems Theory. Behavioral systems theory [ 11 ] focuses on system trajectories and eliminates the need for a state-space representation, with each input-output trajectory termed a “behavior”. As the core result of this theory, Willems’ Fundamental Lemma demonstrates that for a controllable discrete-time linear time-invariant (DT-LTI) system, any valid trajectory can be expressed as a linear combination of time-shifted persistently exciting (PE) measured trajectories [ 12 ] . This lemma has found extensive applications in autonomous driving [ 14 ] , power grid [ 9 ] and quadrotor control [ 15 ] . Leveraging subspace relationships, the research in [ 13 ] demonstrates that under a specific rank condition, closed-loop systems and controllers can be parameterized via input-state data. For single-output (SO) systems, sufficiently long historical input-output trajectories can directly replace unknown states without altering the rank condition [ 13 ] , though this conclusion cannot be readily extended