Identification with Orthogonal Basis Functions: Convergence Speed, Asymptotic Bias, and Rate-Optimal Pole Selection

Jiayun Li, Yiwen Lu, Yilin Mo, Jie Chen · 2025

Read Identification with Orthogonal Basis Functions: Convergence Speed, Asymptotic Bias, and Rate-Optimal Pole Selection by Jiayun Li, Yiwen Lu, Yilin Mo, Jie Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-21096.txt.

Introduction

Jiayun Li, Jie Chen, and two co-authors put hard numbers on a workhorse system-identification method. Expanding a linear system's dynamics in orthogonal basis functions reduces identification to least squares, but everything hinges on where the basis functions' poles are placed. The team derives an analytic convergence rate for the algorithm and an explicit bound on its asymptotic bias, expressed through both the true system's poles and the chosen model poles.

Their pole-selection algorithm then robustly minimizes the worst-case bias, measured in the H2 sense, and provably reaches the fundamental lower bound: no choice can do better asymptotically. The optimal poles are the classical Tsuji points, and bias falls exponentially as basis functions are added. Numerical experiments confirm the bounds, turning a heuristic tuning knob into a solved design step.

Catalog record

CREATORJiayun Li, Yiwen Lu, Yilin Mo, Jie Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-21096

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Identification with Orthogonal Basis Functions: Convergence Speed, Asymptotic Bias, and Rate-Optimal Pole Selection Jiayun Li, Yiwen Lu, Yilin Mo, Jie Chen Abstract This paper is concerned with performance analysis and pole selection problem in identifying linear time-invariant (LTI) systems using orthogonal basis functions (OBFs), a system identification approach that consists of solving least-squares problems and selecting poles within the OBFs. Specifically, we analyze the convergence properties and asymptotic bias of the OBF algorithm, and propose a pole selection algorithm that robustly minimizes the worst-case identification bias, with the bias measured under the ℋ 2 \mathcal{H}_{2} error criterion. Our results include an analytical expression for the convergence rate and an explicit bound on the asymptotic identification bias, which depends on both the true system poles and the preselected model poles. Furthermore, we demonstrate that the pole selection algorithm is asymptotically optimal, achieving the fundamental lower bound on the identification bias. The algorithm explicitly determines the model poles as the so-called Tsuji points, and the asymptotic identification bias decreases exponentially with the number of basis functions, with the rate of decrease governed by the hyperbolic Chebyshev constant. Numerical experiments validate the derived bounds and demonstrate the effectiveness of the proposed pole selection algorithm. I Introduction System identification has long been a focus in the design of control systems, and more broadly in the modeling of dynamical systems [ 25 ] , statistical time series analysis [ 26 ] , and in system realization theory [ 23 ] , with early inspirations going back to the works of Zadeh [ 51 ] and Kalman [ 18 ] in the 1960s. Traditional identification methods usually address this problem in the asymptotic sense, where the number of data samples tends to infinity. The central issue is whether the identified mathematical model asymptotically approaches the true physical system, depending on a priori information available about the system and a posteriori data acquired experimentally, and, if so, whether a bias arises between the model and the system. Classical identification algorithms, such as the Ho-Kalman algorithm [ 17 ] and the multivariable output-error state space (MOESP) approach [ 48 ] , are known to be asymptotically unbiased given infinite samples under certain conditions [ 25 , 3 , 35 ] . However, these algorithms are typically nonlinear (involving nonlinear operations such as SVD or matrix inversion of the Hankel matrices) with respect to the collected data and may exhibit high sensitivity to noise when only finite-length samples are available [ 7 , 13 ] , resulting in potential high signal-to-noise ratio requirement and thus high sample complexity. This behavior is also illustrated in a numerical example in Fig. 6 . It is intuitively plausible that the availability of a priori information could improve the data efficiency of identification algorithms. In this vein, several methods [ 10 , 50 , 32 ] seek to approximate a system’s transfer function using a linear combination of orthogonal basis functions (OBFs). A least-squares (LS) algorithm is leveraged to identify the coefficients of OBFs, which is linear with respect to the data. In contrast, the set of the OBFs, parametrized by their poles, are preselected based on a priori information, eliminating the need for nonlinear identification algorithms to estimate system poles. Common choices for OBFs include the Laguerre [ 49 ] and the Kautz functions [ 50 ] . Furthermore, Van den Hof et al. [ 47 ] propose the generalized OBF (GOBF) and derive upper bounds on the asymptotic identification bias as sample size approaches infinity [ 47 , 15 , 34 ] . Ninness and Gustafsson [ 31 ] develop a unifying construction of the OBFs, with which they show that the identification result converges in the mean-square sense at a rate of O(N^{-1}) with respect to the sample size N . Additionally, Ninness and Gomez [ 34 ] quantify the asymptotic bias in identification using OBF methods, which can vary significantly depending on the choice of OBFs. Consequently, the selection of poles in the OBFs plays a crucial role in the quality of the identified model, directly influencing the bias of the identification algorithm. To facilitate the selection of OBF poles, one line of work aims to address this difficulty from a robustness perspective, ensuring any system consistent with the given a priori information can be accurately approximated by the OBFs with the selected poles. For example, Oliveira e Silva [ 16 , Chapter 11] introduces a pole selection algorithm for the OBFs by numerically solving a nonlinear minimax optimization problem, based on the pole region of the true system, i.e., a closed set in the complex plane that encompasses all poles of the true system, as specified by a priori information. However, this problem involves a non-convex semi-infinite programming, which may suffer from drastic growth in computational complexity with the number of poles [ 16 , Chapter 11] . Toth et al. [ 42 ] propose a clustering algorithm for selecting OBF poles in the context of linear parameter-varying systems. However, the algorithm requires an ensemble of sampled system poles, a requirement that renders the algorithm’s performance heavily dependent on the accuracy of the sampled poles. There are also less conventional methods that seek to select the OBF poles adaptively, such as particle swarm optimization [ 38 ] and genetic optimization [ 39 ] . Other approaches include solving an empirical Bayes problem [ 6 , 9 , 8 ] and applying a greedy algorithm using frequency-domain data [ 29 , 28 ] . However, these methods often suffer from high computational complexity and may become trapped in local optima. This paper enhances the OBF approach by proposing an optimal pole selection algorithm to robustly minimize the worst-case identification bias among a particular class of systems. To this end, we first analyze, the convergence properties and the asymptotic bias of OBF algorithms (i.e., with prescribed poles and OBF bases). We then solve an asymptotic pole selection problem. We show that the so-called Tsuji points [ 46 ] asymptotically achieve the minimal identification bias possible and hence are asymptotically optimal poles. These points can be determined by solving a maximization problem on the boundary of the pole region, ridding of the need to solve a significantly more complex minimax problem, such as the one proposed by Oliveira e Silva [ 16 , Chapter 11] . We also present an algorithm to compute a set of near-optimal initial points for the maximization problem, thus alleviating the issue of local optimality. The resulting system identification bias asymptotically achieves the fundamental lower bound on the worst-case identification bias. Our main contributions can be summarized as follows: 1. With preselected bases and corresponding poles, we show that the identification error of the OBF methods measured under the \mathcal{H}_{2} norm converges to an asymptotic bias almost surely at a rate of O(N^{-0.5+\epsilon}) , for any \epsilon>0 . We also derive an upper bound on the identification bias, which depends on the discrepancy between the true system poles and the preselected poles in the OBF basis functions. When system poles are known to lie within a specific region \mathcal{D} within the unit disk in the complex plane, we establish a fundamental lower bound on the asymptotic

Read the complete text