Finite Sample Analyses for Continuous-time Linear Systems: System Identification and Online Control
Hongyi ZhouIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected], Jingwei LiIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected] Jingzhao ZhangIIIS, Tsinghua UniversityShanghai Qi zhi Institute [email protected] · 2025
Read Finite Sample Analyses for Continuous-time Linear Systems: System Identification and Online Control by Hongyi ZhouIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected], Jingwei LiIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected] Jingzhao ZhangIIIS, Tsinghua UniversityShanghai Qi zhi Institute [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22741.txt.
Introduction
Hongyi Zhou, Jingwei Li, and Jingzhao Zhang study learning and control of systems that evolve in continuous time, from finite data. Real systems run continuously, but algorithms see only samples. Their first target is system identification: recovering a linear system's parameters from observations. Their algorithm needs no integrated observations over time windows, which suits real-world use. They give the first non-asymptotic error analysis for this setting, plus a lower bound showing the estimator is optimal up to constants.
They then plug the identifier into online control. Faster identification lets the controller drop bad policies sooner. Over a single horizon, their method achieves square-root regret while using only a linear number of observations.
Catalog record
| CREATOR | Hongyi ZhouIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected], Jingwei LiIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected] Jingzhao ZhangIIIS, Tsinghua UniversityShanghai Qi zhi Institute [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 140 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22741
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Finite Sample Analyses for Continuous-time Linear Systems: System Identification and Online Control Hongyi ZhouIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected], Jingwei LiIIIS, Tsinghua UniversityShanghai Qizhi Institute [email protected] Jingzhao ZhangIIIS, Tsinghua UniversityShanghai Qi zhi Institute [email protected] Abstract Real world evolves in continuous time but computations are done from finite samples. Therefore, we study algorithms using finite observations in continuous-time linear dynamical systems. We first study the system identification problem, and propose a first non-asymptotic error analysis with finite observations. Our algorithm identifies system parameters without needing integrated observations over certain time intervals, making it more practical for real-world applications. Further we propose a lower bound result that shows our estimator is provably optimal up to constant factors. Moreover, we apply the above algorithm to online control regret analysis for continuous-time linear system. Our system identification method allows us to explore more efficiently, enabling the swift detection of ineffective policies. We achieve a regret of 𝒪 ( T ) \mathcal{O}(\sqrt{T}) over a single T T -time horizon in a controllable system, requiring only 𝒪 ( T ) \mathcal{O}(T) observations of the system. 1 Introduction Finding optimal control policies requires accurately modelling the system [ 18 ] . However, real-world environments often involve unknown system parameters. In such cases, estimating unknown parameters from exploration becomes essential to identify the unseen dynamics. This process is recognized as system identification, a fundamental tool employed in various research fields, including time-series analysis [ 20 ] , control theory [ 21 ] , robotics [ 16 ] , and reinforcement learning [ 28 ] . The identification of linear systems has long been studied because linear systems, as one of the most fundamental systems in both theoretical frameworks and practical applications, has wide applications ranging from natural physical processes to robotics. Most classical results provide only asymptotic convergence guarantees for parameter estimation [ 3 , 24 , 5 ] . On the other hand, with the rapid increase in data scale, there is a growing concern for statistical efficiency. Consequently, the non-asymptotic convergence of discrete-time linear system identification has emerged as another pivotal topic in this field. Investigations into this matter delve into understanding how estimation confidence is influenced by the sample complexity of trajectories [ 9 ] , or the running time on a single trajectory [ 33 , 29 ] . Furthermore, many of these studies operate under the common assumption of stochastic noise, there has been a parallel exploration into the identification of discrete-time linear dynamical systems with diverse setups. This includes scenarios where perturbations are adversarial [ 15 ] or when only black-box access is available [ 7 ] . In contrast to studies in discrete time system, there have been relatively fewer non-asymptotic results addressing parameter identification for continuous-time systems . Two problems exist for continuous time analysis. First, nonasymptotic analysis in continuous system without noise can be degenerate, as a short time interval can contain infinite pieces of information. Second, if we consider the non-degenerate case when finite noisy observations are available, then the analyses require concentration results that become known only as in [ 33 , 9 , 29 ] . Recently [ 4 ] provides novel analyses for estimating system parameters, which relies on continuous data collection and interaction with the environment. Motivated by progress in these works, our first goal is to answer the question below: Can we design a continuous-time stochastic system identification algorithm that provides nonasymptotic error bounds with only a finite number of samples? We will introduce our system identification algorithms tailored to meet the above requirements. As expected, we discretize time into small intervals, thereby reducing the problem to a discrete system. The interesting part involves ensuring that the discretization remains bijective and that the inversion is unbiased. Our algorithm identifies the continuous system using only a finite number of samples from the discrete system. We further propose a information theoretic lower bound that shows our algorithm is optimal. As an application of our system identification methods, we study an online continuous-time linear control problem as introduced in [ 30 ] . In this context, exploration is essential for estimating unknown parameters, with the goal of identifying a more optimal control policy that narrows the performance gap. The primary challenge involves finding the right balance between exploration and exploitation. Leveraging our identification method for more efficient parameter estimation allows us to effectively manage exploration and exploitation, achieving an expected regret of \mathcal{O}(\sqrt{T}) over a single trajectory with only \mathcal{O}(T) samples in time horizon T . This surpasses the previously best known result of \mathcal{O}(\sqrt{T}\log(T)) , which needs continuous data collection from the system. We summarize our contributions below. 1. When the system can be stabilized by a known controller, we establish an algorithm with \mathcal{O}(T) samples that achieves estimation error \mathcal{O}(\sqrt{1/T}) on a single trajectory with running time T , which is shown in Theorem 2 . We also provide Theorem 4 which shows that the estimation error of our system identification method is optimal up to constant factors. 2. When a stable controller is not available, we can use N independent short trajectories to obtain estimators with error \mathcal{O}(\sqrt{1/N}) , as is shown in Theorem 5 . 3. We apply our system identification method to an online continuous linear control algorithm, which only requires \mathcal{O}(T) samples and achieves \mathcal{O}(\sqrt{T}) regret on a single trajectory with lasting time T (Theorem 6 ), improving upon the best known result \mathcal{O}(\sqrt{T}\log(T)) in [ 30 ] . 2 Related Works Control of both discrete and continuous linear dynamical systems have been extensively studied in various settings, such as linear quadratic optimal control [ 27 ] , H_{2} stochastic control [ 10 ] , H_{\infty} robust control [ 34 , 17 ] and system identification [ 21 , 24 ] . Below we introduce some of the important results on both system identification and optimal control for linear dynamical systems. System Identification Earlier literature focused primarily on the asymptotic convergence of system identification [ 6 , 25 ] . Recently, there has been a resurgence of interest in non-asymptotic system identification for discrete-time systems. [ 9 ] studied the sample complexity of multiple trajectories, with \mathcal{O}(\sqrt{1/N}) estimation error on N independent trajectories. For systems with dynamics x_{t+1}=Ax_{t}+w_{t} (without controllers), [ 33 ] established an analysis for \mathcal{O}(\sqrt{1/T}) estimation error on a single stable trajectory with running time T , while [ 13 ] and [ 29 ] extended to more general discrete-time systems. Non-asymptotic analyses for continuous-time linear system are less studied. Recently, [ 4 ] examined continuous-time linear quadratic control systems with standard brown noise and unknown system dynamics. Our algorithm is specifically designed for finite observations, achieving an error rate that cannot be attained through the direct discretization of integrals as done in [ 4 ] . Regret Analysis of Online Control In online control, if the system’s parameters are known, achieving the optimal control policy in this setup can be straightforward [ 34 , 35 ] . However, when the system parameters are unknown, identifying the system incurs regret. [ 1 ] achieved an \mathcal{O}(\sqrt{T}) regret for discrete-time online linear control, which has been proven optimal in T under that setting in [ 32