CLT-Optimal Parameter Error Bounds for Linear System Identification
Yichen Zhou, Stephen Tu · 2026
Read CLT-Optimal Parameter Error Bounds for Linear System Identification by Yichen Zhou, Stephen Tu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21270.txt.
Introduction
Yichen Zhou and Stephen Tu sharpen error bounds for identifying linear dynamical systems from data by ordinary least squares. They show current bounds overstate parameter error by a factor of the state dimension. This holds in both spectral and Frobenius norm, for classes of instances identified via asymptotic normality. A second-order decomposition, whose lower-order term is a matrix-valued martingale capturing central-limit scaling, yields finite bounds. It matches optimal rates up to constants for stable systems and the many-trajectory setting.
Catalog record
| CREATOR | Yichen Zhou, Stephen Tu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 129767 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21270
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CLT-Optimal Parameter Error Bounds for Linear System Identification Yichen Zhou, Stephen Tu Abstract . There has been remarkable progress over the past decade in establishing finite-sample, non-asymptotic bounds on recovering unknown system parameters from observed system behavior. Surprisingly, however, we show that the current state-of-the-art bounds do not accurately capture the statistical complexity of system identification, even in the most fundamental setting of estimating a discrete-time linear dynamical system (LDS) via ordinary least-squares regression (OLS). Specifically, we utilize asymptotic normality to identify classes of problem instances for which current bounds overstate the squared parameter error, in both spectral and Frobenius norm, by a factor of the state-dimension of the system. Informed by this discrepancy, we then sharpen the OLS parameter error bounds via a novel second-order decomposition of the parameter error, where crucially the lower-order term is a matrix-valued martingale that we show correctly captures the CLT scaling. From our analysis we obtain finite-sample bounds for both (i) stable systems and (ii) the many-trajectories setting that match the instance-specific optimal rates up to constant factors in Frobenius norm, and polylogarithmic state-dimension factors in spectral norm. 1. Introduction System identification—the task of recovering unknown parameters of a dynamical system from observed system behavior—is a central problem in control. While system identification has a long and rich history in the control theory literature, the rise of machine learning approaches to control has led to a recent surge of results, particularly in establishing non-asymptotic, finite-sample guarantees for parameter recovery [ 28 , 26 , 27 , 34 ] . Such bounds are valuable both for characterizing the statistical limits of system identification, and for providing principled guidance on the data requirements needed to learn accurate models in practice. Despite this remarkable progress, we argue that existing state-of-the-art bounds still fail to fully capture the statistical complexity of system identification—surprisingly even in the most basic setting of estimating the dynamics matrix of a discrete-time linear dynamical system (LDS) given one or many trajectories of state observations using ordinary least-squares (OLS) regression, a model known as the \mathrm{VAR}(1) (vector auto-regressive) model which we focus on in this work. At first glance, this statement seems to contradict existing claims of minimax-optimality of OLS for this setting [ 28 , 14 , 15 ] . However, a more careful introspection reveals that minimax-optimal rates are only sharp on a carefully selected set of problem instances; they do not guarantee sharp rates on every problem instance. Concretely, we consider two setups where asymptotic normality can be used to compute the exact instance-specific problem scaling: (i) strictly stable transition dynamics, and (ii) the many trajectories [ 30 ] setting, where the number of independent trajectories outnumber the dimension. We remark that (i) and (ii) are not mutually exclusive. Our analysis reveals many problem instances where the sharpest known OLS bounds overestimate the squared parameter estimation error, in either Frobenius or operator norm, by a factor of the state-dimension. These instances are characterized by a sufficiently fast decay of the eigenvalues of either the noise covariance matrix or the inverse state covariance matrix, so that trace \Tr(M) of said matrices is significantly smaller than d\lVert M\rVert_{\mathsf{op}} , where d is the state-dimension and \lVert M\rVert_{\mathsf{op}} is the operator norm. A practical setting where this naturally arises is when disturbances enter the system in a non-uniform way, such as through an approximately low-rank subspace, or due to different physical scaling across state variables. We then turn to the problem of sharpening the non-asymptotic OLS bounds to match the instance-specific optimal rates from asymptotic normality. We establish sufficient conditions on the trajectory length in the case of stable transition dynamics, and on the number of trajectories in the many trajectories setting, for the squared parameter error of the OLS estimator to match the instance-specific optimal rate up to a constant factor in Frobenius norm, and up to \mathrm{polylog}(d) factors in operator norm. At a high level, the deficiency of existing approaches is rooted in the standard decomposition of the OLS error introduced by [ 28 ] , which splits the OLS error into two terms that are analyzed separately: (i) a self-normalized martingale term [ 1 ] , and (ii) the minimum eigenvalue of the empirical state covariance matrix. We propose a novel second-order decomposition where the lowest order term becomes a simpler matrix-valued martingale that captures the CLT scaling. In Frobenius norm analyzing said martingale is immediate thanks to the inner-product structure, but for the operator norm extracting the CLT scaling is non-trivial and relies on the machinery of non-commutative Burkholder inequalities [ 24 , 16 ] . Returning back to our second-order decomposition, the new higher-order term places a stronger requirement on the empirical state covariance—namely a p -th moment approximate isometry condition—for which we establish by taking inspiration from the Hanson-Wright approach described in [ 15 ] . Our paper is organized as follows. In Section 2 , we formalize the problem setting, discuss in detail what target rates we should expect from asymptotic normality, and illustrate examples where existing bounds are loose. In Section 3 , we state our main non-asymptotic results for both Frobenius and operator norms. Section 4 provides a detailed proof outline, and Section 5 concludes with future directions. All proof details are deferred to Appendices A to C . 1.1. Related Work We focus our literature discussion on both asymptotic and finite-sample analysis of parameter estimation in discrete-time linear dynamical systems with full state observation; see [ 34 , 12 ] for a broader overview of recent non-asymptotic results and perspectives, and [ 20 ] for a classical treatment. The starting point for non-asymptotic error analysis of the OLS estimator is due to [ 28 ] , which controls the operator norm of the parameter error via decomposition into a self-normalized process and the minimum eigenvalue of empirical covariance. In the case where the underlying process is stable, [ 15 ] improves the required sample complexity for (\varepsilon,\delta) -PAC, although as \varepsilon\to 0 the leading order term is the same. We will show that, in the case where the noise matrix is non-isotropic, these results can be loose. The work [ 35 ] controls the excess risk (squared parameter error in a weighted Frobenius norm by the process covariance) of OLS for the stable setting, and matches the CLT variance order-wise; such a bound immediately yields control of the squared Frobenius norm of the parameter error. However as we will also discuss, this conversion introduces un-necessary conservatism in the bound. Most related to this paper is the work [ 30 ] , which initiates a study of multi-trajectory estimation for dependent linear regression (in contrast, all aforementioned papers consider the single trajectory setting). Similar to [ 35 ] , the presented bounds are also for excess risk, and suffer from the same conversion issue. One additional notable digression in [ 30 ] compared to the aforementioned papers is in controlling the expected value of excess risk, instead of with high probability; this requires several key technical tools, which we build off of in this work. On the asymptotic side, there is a well-established literature on the strong consistency and asymptotic normality of single trajectory