FEDERAL UNIVERSITY OF MINAS GERAIS Department of Electronic Engineering Technical Report Bayesian Subspace Identification in the MIMO Case

2025

Read FEDERAL UNIVERSITY OF MINAS GERAIS Department of Electronic Engineering Technical Report Bayesian Subspace Identification in the MIMO Case (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24435.txt.

Introduction

Alexandre Rodrigues Mesquita extends a Bayesian method for identifying systems to a harder case. The earlier method handled simpler input-output setups. This report treats systems with many inputs and many outputs. The Bayesian version of subspace identification adds priors and posteriors over the model.

New priors with the right symmetry are derived for this case. Matching posterior laws follow. The DAISY dataset supplies the numerical tests. Results on that benchmark validate the extension. The report is short and technical. It serves control engineers who fit dynamics to measured data. The Bayesian route promises uncertainty estimates beside the fitted model, a virtue the extension keeps.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-24435

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FEDERAL UNIVERSITY OF MINAS GERAIS Department of Electronic Engineering Technical Report Bayesian Subspace Identification in the MIMO Case Abstract This report investigates the extension of the Bayesian Subspace System Identification method proposed in [ 4 ] to the Multiple-Input Multiple-Output (MIMO) case. We derive new equivariant priors and posterior distributions specifically suited for the MIMO framework. Numerical results utilizing the DAISY dataset are reported to validate the approach. 1 Introduction Subspace system identification relies on estimating subspaces defined by a large number of parameters. This estimation process may benefit significantly from variance reduction techniques like regularization and Bayesian inference. In the last decade, regularization techniques such as nuclear norm minimization [ 12 , 8 , 5 , 1 , 9 ] have been explored in the context of subspace identification methods. Shrinkage estimators were first applied to subspace identification in [ 13 ] . While Bayesian methods were recently introduced in [ 4 ] , they were restricted to single-input single-output (SISO) models. The goal of this contribution is to extend Bayesian subspace identification to the multiple-input multiple-output (MIMO) case. Two key steps in subspace identification are i) some variant of linear regression and ii) the approximation of some data matrix by a low-rank matrix obtained by truncating the singular values of the original matrix. Bayesian subspace identification replaces these two steps by alternating Bayesian regressions in the row space and column space of the observed data matrix. In other words, the empirical posterior distribution is computed by a series of Gibbs sampling steps consisting of alternating regularized least squares problems. The prior distribution is designed to obey some structural properties of the estimated matrices, such as the lower block triangular structure of the noise covariance matrix. In particular, equivariant prior distributions were derived for such matrices. In this contribution, we derive i) an extended equivariant prior distribution that applies to the MIMO case and ii) derive the corresponding posterior distribution suited for the MIMO case. Since the desired extended result was not trivially available from the SISO case, we made use of properties of the block vectorization operation in order to find a computable posterior distribution. 2 The subspace identification framework We assume the system whose model we want to identify obeys the linear state space model in innovation form \displaystyle x_{k+1} \displaystyle=Ax_{k}+Bu_{k}+Ke_{k} (1) \displaystyle y_{k} \displaystyle=Cx_{k}+Du_{k}+e_{k}\kern 5.0pt, (2) where x_{k}\in\mdmathbb{R}^{n_{x}} is the state, y_{k}\in\mdmathbb{R}^{n_{o}} is the measured variable, u_{k}\in\mdmathbb{R}^{n_{i}} is the input variable and e_{k} is the innovations process, assumed to be white Gaussian noise. The model parameters A,B,C,D and K are matrices with the appropriate dimensions. This same model may be expressed equivalently in in the predictor form \displaystyle x_{k+1} \displaystyle=A_{K}x_{k}+B_{K}z_{k} (3) \displaystyle y_{k} \displaystyle=Cx_{k}+Du_{k}+e_{k}\kern 5.0pt, (4) where z_{k}=[u_{k}^{T}~~y_{k}^{T}]^{T} , A_{K}=A-KC , B_{K}=[B-KD~~K] . Following the notation in [ 6 ] , we make use of the extended state space model Y_{f}=\Gamma_{f}X_{k}+H_{f}U_{f}+G_{f}E_{f} (5) and its predictor form Y_{f}=H_{fp}Z_{p}+H_{f}U_{f}+G_{f}E_{f}\kern 5.0pt, (6) where the available data is arranged in block Hankel matrices defined as Y_{f}=\left[\begin{array}[]{cccc}y_{k}&y_{k+1}&\cdots&y_{k+N-1}\\ y_{k+1}&y_{k+2}&\cdots&y_{k+N}\\ \vdots&\vdots&\ddots&\vdots\\ y_{k+f-1}&y_{k+f}&\cdots&y_{k+f+N-2}\\ \end{array}\right] (7) and similarly are defined U_{f} and E_{f} . As usual in the subspace identifiation literature, f and p are used both as labels for future and past variables as well as numbers ( f>n_{x} , p>n_{x} ) that denote the future and past horizons, respectively. Here N is an integer that depends on the size of the available data set. The state sequence is defined as X_{k}=\left[\begin{array}[]{cccc}x_{k}&x_{k+1}&\cdots&x_{k+N-1}\end{array}\right]\kern 5.0pt. (8) The past information is collected in Z_{p}=[U_{p}^{T}~Y_{p}^{T}]^{T} up to the horizon p>n_{x} and arranged as U_{p}=\left[\begin{array}[]{cccc}u_{k-p}&u_{k-p+1}&\cdots&u_{k-p+N-1}\\ u_{k-p+1}&u_{k-p+2}&\cdots&u_{k-p+N}\\ \vdots&\vdots&\ddots&\vdots\\ u_{k-1}&u_{k}&\cdots&u_{k+N-2}\\ \end{array}\right] (9) and similarly for Y_{p} . As a consequence, \Gamma_{f} is the extended observability matrix as defined by \Gamma_{f}=\left[\begin{array}[]{c}C\\ CA\\ \vdots\\ CA^{f-1}\end{array}\right]\kern 5.0pt, (10) H_{f} and G_{f} are block Toeplitz matrices given by H_{f}=\left[\begin{array}[]{cccc}D&0&\cdots&0\\ CB&D&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ CA^{f-2}B&CA^{f-3}B&\cdots&D\end{array}\right]\kern 5.0pt, (11) and G_{f}=\left[\begin{array}[]{cccc}I_{n_{o}}&0&\cdots&0\\ CK&I_{n_{o}}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ CA^{f-2}K&CA^{f-3}K&\cdots&I_{n_{o}}\end{array}\right]\cdot(I_{f}\otimes\Sigma^{1/2})\kern 5.0pt, (12) where I_{f} is the f\times f identity matrix and \Sigma is the innovations covariance. The matrix H_{fp} is reminiscent of the matrix of Markov parameters and is given by H_{fp}=\left[\begin{array}[]{cc}H_{fp}^{(1)}&H_{fp}^{(2)}\end{array}\right] (13) with H_{fp}^{(i)}=\left[\begin{array}[]{cccc}CA_{K}^{p-1}B_{K}^{(i)}&CA_{K}^{p-2}B_{K}^{(i)}&\cdots&CB_{K}^{(i)}\\ CAA_{K}^{p-1}B_{K}^{(i)}&CAA_{K}^{p-2}B_{K}^{(i)}&\cdots&CAB_{K}^{(i)}\\ \vdots&\vdots&\ddots&\vdots\\ CA^{f-1}A_{K}^{p-1}B_{K}^{(i)}&CA^{f-1}A_{K}^{p-2}B_{K}^{(i)}&\cdots&CA^{f-1}B_{K}^{(i)}\\ \end{array}\right]\kern 5.0pt, (14) for i=1,2 and B_{K}^{(1)}=B-KD and B_{K}^{(2)}=K . These matrices can also be decomposed as products H_{fp}^{(i)}=\Gamma_{f}L_{p}^{(i)} of the extended observability \Gamma_{f} and controllability matrices: L_{p}^{(i)}=[\begin{array}[]{ccc}A_{K}^{p-1}B_{K}^{(i)}&A_{K}^{p-2}B_{K}^{(i)}\cdots B_{K}^{(i)}\end{array}]\kern 5.0pt (15) for i=1,2 . Building upon this formulation, numerous estimation methods have been proposed in the literature that exploit the low-dimensional subspace structure of the data as organized in ( 6 ) (see [ 6 , 11 ] for a comprehensive review). A common approach is to solve ( 6 ) via least squares, followed by estimating \Gamma_{f} from a truncated singular value decomposition of H_{fp} . Once \Gamma_{f} is determined, the system matrices A and C can be recovered from the relatioship between \Gamma_{f} and \Gamma_{f+1} . Subsequently, the matrices B and D are obtained either by solving a secondary least squares problem or by utilizing estimates of the controllability matrix L_{p} . Generally, most subspace methods proceed by obtaining an initial estimate \hat{H}_{fp} through least squares. Given two weight matrices W_{1} and W_{2} , a low-dimensional estimate \hat{\hat{H}}_{fp} is then derived by truncating the singular value decomposition of {W_{1}\hat{H}}_{fp}W_{2} to its r largest components: W_{1}\hat{H}_{fp}W_{2}=USV^{T}\approx U_{r}S_{r}V_{r}^{T}=:W_{1}\hat{\hat{H}}_{fp}W_{2}\kern 5.0pt. (16) In this context, the truncation rank r serves as an estimate of the system order n_{x} . Traditional subspace identification methods obtain an initial estimate of H_{fp} from ( 6 ) using the following least squares solution: [\hat{H}_{fp}~\hat{H}_{f}]=Y_{f}\left[\begin{array}[]{c}U_{p}\\ Y_{p}\\ U_{f}\end{array}\right]^{\dagger}\kern 5.0pt. (17) However, it is important to note that this estimate merely approximates the maximum likelihood estimate. This is because the noise term G_{f}E_{f} in ( 6 ) is, at best, only approximately white and the structure of H_{f} as a block lower triangular Toeplitz matrix is not taken into account. 3 Problem Description Since there are many competing methods in the literature to estimate the matrices (A,B,C,D,K) from H_{fp} and since the estimation errors from such methods are upper bounded by a constant times \|H_{fp}-\hat{H}_{fp}\| as demonstrated in [ 10 ] , we concentrate our efforts on the estimators of H_{fp} . With this in mind, and following the statistical theory of point estimation [ 7 ] , we seek estimators that perform well with regards to risk functions of the form \mathsf{R}_{\theta}(\hat{\hat{H}}_{fp})=\text{E}\left[\|\left.W_{1}(H_{fp}-\hat{\hat{H}}_{fp})W_{2}\|_{F}^{2})\right|(A,B,C,D,K)=\theta\right]\kern 5.0pt. (18) The risk is a function of the estimator being used and also a function of the system parameters \theta . The expectation is over all input-output realizations for the system model with parameter \theta . In general, it is not possible to minimize the risk for all \theta . Therefore one often seeks to minimize the expectation of the risk over some distribution of \theta or the risk for the worst case of \theta . Within this context, our goal is to investigate robustified methods to estimate H_{fp} . In particular, we want to design Bayesian estimators that can be efficiently computed with a Gibbs sampling scheme. 4 An Alternating Least Squares Bayesian Approach Bayesian estimators are inherently robust. Under mild conditions [ 7 ] , they are admissible, i.e., there exists no other estimator that improves upon their risk for all the parameters in the parameter space. Under structural conditions involving group invariance of priors and loss functions (see [ 7

Read the complete text