Data-Driven Covariance Steering with Output Feedback Dimitrios Moustroufis, Panagiotis Tsiotras Abstract This paper addresses the problem of output-feedback covariance steering for stochastic, discrete-time, linear, time-invariant systems without knowledge of the system model. We employ a controllable, non-minimal state representation constructed from past inputs and outputs and convert the problem to one in state-feedback form. In this representation, the induced disturbance becomes temporally correlated, which requires explicit propagation of the cross-covariance between the state and disturbance processes. To handle the lack of a system model, we leverage persistently exciting data collected offline and formulate the mean and covariance steering problems using an indirect and a direct approach, respectively. The indirect formulation requires an estimate of the mean dynamics model, while the direct formulation relies on an estimate of the noise realization in the collected data. To this end, we present an estimation method suitable to handle temporally correlated noise, enabling consistent identification of both components. Using a convex relaxation, we convert the covariance steering problem to a semidefinite program that can be solved efficiently. We conduct numerical simulations to evaluate the performance of the developed framework. I Introduction As engineering systems become more complex, sensors become cheaper and more widely available, and the interest for autonomous operation increases, there has been a recent surge in the literature around data-driven simulation and control methods. These methods can be broadly classified as direct and indirect. According to the indirect paradigm, a model of the plant is first estimated from the observed data, and then it is used for control purposes. In contrast, direct methods compute the control law using a system representation directly from data. While “black-box” methods such as deep neural networks have been employed to model highly complex systems, they generally lack theoretical guarantees in terms of stability and robustness. On the contrary, under the restriction of linear, time-invariant systems with no noise, the seminal work by Willems et al. [ 29 ] proved that future trajectories of a system can be represented by a set of persistently exciting input-output data. This result, often referred to as the “Willem’s Fundamental Lemma” (WFL) has been exploited for data-driven simulation [ 20 ] , for the design of state- and output-feedback stabilizing controllers [ 11 , 10 , 13 ] , as well as for the design of data-driven predictive controllers [ 8 ] , [ 5 ] , and data-enabled policy optimization algorithms [ 31 ] , [ 32 ] . These methods have been applied to control quadrotors, power networks, and synchronous motor drives [ 14 , 16 , 6 ] . Willems’ result has been extended to data collected from multiple trajectories [ 28 ] , while robustifying regularizations [ 9 ] have been employed to handle noise and nonlinearities in the data. Finally, the connection between indirect and direct methods based on the WFL has also been investigated [ 12 ] . A control design paradigm that aims to provide guarantees on the control performance under stochastic disturbances by controlling the statistical distribution of the system’s state is that of Covariance Steering (CS). In particular, discrete-time CS deals with the problem of steering the state of a system from an initial distribution to a terminal one within a specified horizon. This problem has been extensively investigated in the literature [ 2 , 15 , 17 , 4 ] , and an efficient method to obtain the optimal solution in terms of a semi-definite program has been proposed [ 18 ] , [ 26 ] . The CS problem for partially observed systems has also been recently addressed in the literature [ 23 ] , [ 19 ] . All previous methods assume knowledge of the system model. Aiming to alleviate this assumption, [ 22 ] formulated the CS problem for noiseless systems in a model-free fashion by exploiting the WFL, while [ 24 ] , [ 25 ] extended these results for the case when process noise is present. In these works, an indirect approach is employed for the mean steering problem, while the covariance steering problem is formulated using a direct approach. A major assumption of the current data-driven covariance steering literature is the availability of full state measurements without any measurement noise. This, however, may not be possible for many practical applications, motivating the present work. The main contribution of this paper is to extend the state-feedback data-driven covariance steering results to the output-feedback case. By exploiting a non-minimal state representation [ 11 ] , we utilize state-feedback ideas to solve the data-driven covariance steering problem. However, in this representation, the disturbances are temporally correlated, which complicates model estimation for the indirect mean controller design and noise estimation for the direct covariance controller design. This work also addresses the discrete-time covariance steering problem under temporally correlated noise, which, to the best of the authors’ knowledge, has not been addressed in the literature. II Problem Formulation II-A Notation To denote a stacked vector composed of the column vectors a,b , we use the operator \mathrm{col}(a,b)\triangleq[a^{\mbox{\tiny\sf T}}\ b^{\mbox{\tiny\sf T}}]^{\mbox{\tiny\sf T}} . Considering the T+1 long vector sequence \{a_{0},\ a_{1},\ \cdots,\ a_{T+1}\} , we denote its Hankel matrix of depth L and length T with \mathcal{H}_{i,L,T}(a) . The j -th column of \mathcal{H}_{i,L,T}(a) is \mathrm{col}(a_{i+j},\ \ldots,\ a_{i+j+L-1}) , j\in[0,\ T-1] . We also denote with \mu^{(\cdot)}_{k} and \Sigma^{(\cdot)}_{k} the mean and covariance of the random variable in the superscript at some time k . For a symmetric positive (semi-)definite matrix A we use the notation A\succ 0 ( \succeq 0 ). II-B Problem Statement Consider the stochastic, discrete-time, linear, time-invariant system \displaystyle x_{k+1} \displaystyle=Ax_{k}+Bu_{k}+w_{k}, (1) \displaystyle y_{k} \displaystyle=Cx_{k}+q_{k}, where x_{k},w_{k}\in\mathbb{R}^{n} , y_{k},q_{k}\in\mathbb{R}^{p} , u_{k}\in\mathbb{R}^{m} and w_{k},q_{k} are i.i.d. Gaussian white noise processes representing the process and measurement noise, respectively. The data-driven output distribution steering problem is posed as steering the output from some initial distribution \mathcal{N}(\mu_{i}^{y},\Sigma_{i}^{y}) to a final distribution \mathcal{N}(\mu_{f}^{y},\Sigma_{f}^{y}) over a specified horizon, N , while minimizing some cost index, without knowledge of the model matrices A,B,C , but rather utilizing only T+1 input-output data pairs from a single previous trajectory \mathcal{D}=\{y^{(d)}_{k},u^{(d)}_{k}\},k\in[0,T] . For the state-feedback case, and for a known system model, the optimal controller that solves the covariance steering problem, along with its formulation as a semi-definite program (SDP), has been established in the literature for discrete-time [ 2 ] , [ 18 ] , [ 3 ] , and for continuous-time systems [ 7 ] . An extension to the data-driven case for discrete-time systems was developed in [ 24 ] . We aim to extend these state-feedback results to the output-feedback case by utilizing a non-minimal state space representation constructed from past inputs and outputs. II-C Non-minimal State Space Representation A non-minimal state space representation of the system is defined as \displaystyle z_{k}\triangleq\mathrm{col}(u_{k-1},\cdots,u_{k-\ell},y_{k-1},\cdots,y_{k-\ell}),\ z_{k}\in\mathbb{R}^{h}, (2) where \ell denotes the observability index of the system, i.e, the smallest integer such that the observability matrix \mathcal{O}=[CA^{\ell-1}\ \cdots\ CA\quad C]^{\mbox{\tiny\sf T}} has rank n and h=\ell(m+p) . In case \ell is not known, an upper bound of \ell can be used, with the largest such upper bound being the order of the system, n . In case n is not known, it can be estimated from input-output measurements [ 11 ] . We also define the vectors of the stacked past process and measurement noise values \tilde{q}_{k}\triangleq\mathrm{col}(q_{k-1},\cdots,q_{k-\ell}) and \tilde{w}_{k}\triangleq\mathrm{col}(w_{k-1},\cdots,w_{k-\ell}) , as well as the matrices \displaystyle\mathcal{C}_{x} \displaystyle\triangleq\begin{bmatrix}B&AB&\cdots&A^{\ell-1}B\end{bmatrix}, \displaystyle\mathcal{C}_{w} \displaystyle\triangleq\begin{bmatrix}I&A&\cdots&A^{\ell-1}\end{bmatrix}, \displaystyle\mathcal{T} \displaystyle\triangleq\begin{bmatrix}0&CB&CAB&\cdots&CA^{\ell-2}B\\ 0&0&CB&\cdots&CA^{\ell-3}B\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ 0&0&0&\cdots&CB\\ 0&0&0&\cdots&0\end{bmatrix}, \displaystyle\mathcal{T}_{w} \displaystyle\triangleq\begin{bmatrix}0&C&CA&\cdots&CA^{\ell-2}\\ 0&0&C&\cdots&CA^{\ell-3}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ 0&0&0&\cdots&C\\ 0&0&0&\cdots&0\end{bmatrix}, \displaystyle\mathcal{S} \displaystyle\triangleq\begin{bmatrix}C(\mathcal{C}_{x}-A^{\ell}\mathcal{O}^{\dagger}\mathcal{T})&CA^{\ell}\mathcal{O}^{\dagger}\end{bmatrix}. We can then express the dynamics of the non-minimal state z_{k} through the system \displaystyle z_{k+1} \displaystyle=A_{z}z_{k}+B_{z}u_{k}+\tilde{\xi}_{k}, (3) \displaystyle y_{k} \displaystyle=Sz_{k}+\tilde{\zeta}_{k}. (4) where \tilde{\xi}_{k}=D_{z}\tilde{\zeta}_{k} , A_{z},B_{z},D_{z} are given as in ( 6 ), with \mathcal{S}_{i} ( i=1,\ldots,2\ell ) the i -th column of \mathcal{S} [ 11 ] . The noise \tilde{\zeta}_{k} is given by \displaystyle\tilde{\zeta}_{k} \displaystyle\triangleq C(\mathcal{C}_{w}-A^{\ell}\mathcal{O}^{\dagger}\mathcal{T}_{w})\tilde{w}_{k}-CA^{\ell}\mathcal{O}^{\dagger}\tilde{q}_{k}+q_{k}, (5) \displaystyle=F_{w}\tilde{w}_{k}+F_{q}\tilde{q}_{k}+q_{k}. A_{z}=\begin{bmatrix}0&0&\cdots&0&0&0&0&\cdots&0&0\\ I_{m}&0&\cdots&0&0&0&0&\cdots&0&0\\ 0&I_{m}&\cdots&0&0&0&0&\cdots&0&0\\ \vdots&\vdots&\ddots&\vdots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&\cdots&I_{m}&0&0&0&\cdots&0&0\\ \hline\cr\\ S_{1}&S_{2}&\cdots&S_{\ell-1}&S_{\ell}&S_{\ell+1}&S_{\ell+2}&\cdots&S_{2\ell-1}&S_{2\ell}\\ 0&0&\cdots&0&0&I_{p}&0&\cdots&0&0\\ 0&0&\cdots&0&0&0&I_{p}&\cdots&0&0\\ \vdots&\vdots&\ddots&\vdots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&\cdots&0&0&0&0&\cdots&I_{p}&0\\ \end{bmatrix},\quad B_{z}=\begin{bmatrix}I_{m}\\ 0\\ 0\\ \vdots\\ 0\\ \hline\cr\\ 0\\ 0\\ 0\\ \vdots\\ 0\end{bmatrix},\quad D_{z}=\begin{bmatrix}0\\ 0\\ 0\\ \vdots\\ 0\\ \hline\cr\\ I_{p}\\ 0\\ 0\\ \vdots\\ 0\end{bmatrix}. (6) Lemma 1 . The samples \tilde{\zeta}_{k} are zero-mean, Gaussian, identically distributed, but not independent. Proof. The zero-mean, Gaussian nature of \tilde{\zeta}_{k} is evident from the fact that \tilde{\zeta}_{k} is a linear combination of zero-mean Gaussian random variables, namely, \tilde{w}_{k},\tilde{q}_{k} , and q_{k} . To show that the samples of the noise process \tilde{\zeta}_{k} are correlated in time, we compute \mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}] assuming, without loss of generality, that m>k . The expectation \mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}] is expanded as \displaystyle\mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}] \displaystyle=\mathbb{E}[F_{w}\tilde{w}_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}}F_{w}^{\mbox{\tiny\sf T}}+F_{w}\tilde{w}_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}}F_{q}^{\mbox{\tiny\sf T}}+F_{w}\tilde{w}_{k}q_{m}^{\mbox{\tiny\sf T}} \displaystyle+F_{q}\tilde{q}_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}}F_{w}^{\mbox{\tiny\sf T}}+F_{q}\tilde{q}_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}}F_{q}^{\mbox{\tiny\sf T}}+F_{q}\tilde{q}_{k}q_{m}^{\mbox{\tiny\sf T}} \displaystyle+q_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}}F_{w}^{\mbox{\tiny\sf T}}+q_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}}F_{q}^{\mbox{\tiny\sf T}}+q_{k}q_{m}^{\mbox{\tiny\sf T}}]. Notice that since w_{k} is independent of q_{m} , it follows that \mathbb{E}[\tilde{w}_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}}]=\mathbb{E}[\tilde{q}_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}}]=0 . Therefore, the expression for \mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}] becomes \displaystyle\mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}]=\mathbb{E}[F_{w}\tilde{w}_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}}F_{w}^{\mbox{\tiny\sf T}}+F_{q}\tilde{q}_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}}F_{q}^{\mbox{\tiny\sf T}}+q_{k}q_{m}^{\mbox{\tiny\sf T}}]. For m\neq k , and since q_{k} is white, it holds that \mathbb{E}[q_{k}q_{m}^{\mbox{\tiny\sf T}}]=0 and, similarly, \mathbb{E}[w_{k}w_{m}^{\mbox{\tiny\sf T}}]=0 . Recalling that \tilde{w}_{k},\tilde{q}_{k} contain the elements of w and q from time k-1 to time k-\ell , one sees that for k-1\geq m-\ell , the matrices \tilde{w}_{k}\tilde{w}_{m}^{\mbox{\tiny\sf T}} and \tilde{q}_{k}\tilde{q}_{m}^{\mbox{\tiny\sf T}} contain non-zero elements of the form w_{i}w_{i}^{\mbox{\tiny\sf T}} and q_{i}q_{i}^{\mbox{\tiny\sf T}} , respectively. Therefore, there exist k,m with k\neq m, such that \mathbb{E}[\tilde{\zeta}_{k}\tilde{\zeta}_{m}^{\mbox{\tiny\sf T}}]\neq 0 . ∎ It has been shown in [ 1 ] that for p\ell>n and for systems without noise, the system ( 3 ) is not controllable. Moreover, the matrix \begin{bmatrix}\mathcal{H}_{0,1,T}(u)\\ \mathcal{H}_{0,1,T}(z)\end{bmatrix} does not have full row rank, thus violating a fundamental condition for the data-driven control theory. Following the method of [ 1 ] , we employ a new, non-minimal state-space representation, \chi_{k}\triangleq Lz_{k}\in\mathbb{R}^{r}, (7) where L\in\mathbb{R}^{r\times h} removes the linearly dependent components of z_{k} . Details on how to compute L from noisy data are presented in Section III . The dynamics of \chi_{k} can be expressed as \displaystyle\chi_{k+1} \displaystyle=\mathcal{A}\chi_{k}+\mathcal{B}u_{k}+\mathcal{D}\zeta_{k}=\mathcal{A}\chi_{k}+\mathcal{B}u_{k}+\xi_{k}, (8) \displaystyle y_{k} \displaystyle=\mathcal{C}\chi_{k}+\zeta_{k}, where \xi_{k}\triangleq\mathcal{D}\zeta_{k} and the matrices \mathcal{A},\mathcal{B},\mathcal{C},\mathcal{D} will be estimated from data in the sequel. The noise \zeta_{k} is temporally correlated, as was the case with \tilde{\zeta}_{k} . We consider controllers of the form u_{k}=K_{k}(\chi_{k}-\mu_{k})+v_{k}+\nu_{k} . As shown in [ 18 ] , this class of controllers includes the optimal one for distribution steering problems. II-D Non-minimal State Distribution Steering Problem In the absence of probabilistic chance constraints, the distribution steering problem for the state \chi_{k} can be separated to the mean and the covariance steering problems [ 21 ] . Defining \mu_{k}\triangleq\mathbb{E}[\chi_{k}] and \Sigma_{k}\triangleq\mathbb{E}[(\chi_{k}-\mu_{k})(\chi_{k}-\mu_{k})^{\mbox{\tiny\sf T}}] , as well as \mu_{k}^{y}\triangleq\mathbb{E}[y_{k}] and \Sigma_{k}^{y}\triangleq\mathbb{E}[(y_{k}-\mu_{k}^{y})(y_{k}-\mu_{k}^{y})^{\mbox{\tiny\sf T}}] , the mean output steering problem is posed as \displaystyle\min_{\mu_{k},v_{k}}J=\sum_{k=0}^{N-1}(\mu_{k}-\chi_{k}^{r})^{\mbox{\tiny\sf T}}Q_{k}(\mu_{k}-\chi^{r}_{k})+v_{k}^{\mbox{\tiny\sf T}}R_{k}v_{k}, (9a) such that, for all k=0,1,\dots,N-1 , \displaystyle\mu_{k+1}=\mathcal{A}\mu_{k}+\mathcal{B}v_{k}, (9b) \displaystyle\mu_{0}^{y}=\mu_{i}^{y},\quad\mu_{N}^{y}=\mu_{f}^{y}, (9c) where v_{k} is a feedforward control input and \chi_{k}^{r} is a reference value for \chi_{k} . While the mean propagation equation extends trivially from the state feedback case to the output feedback case, this is not true for the covariance propagation equation. From the dynamics ( 8 ), one notices that the state \chi_{k} is correlated with with the noise \zeta_{k-1} . However, from Lemma 1 , \zeta_{k-1} is correlated with \zeta_{k} , thus making \chi_{k} correlated with \zeta_{k} as well. This means that the cross-covariance between \zeta_{k} and \chi_{k} will not be zero, and this has to be accounted for. In this paper, we assume that the noise \zeta_{k} can be modeled as \displaystyle\zeta_{k+1} \displaystyle=\Psi\zeta_{k}+\eta_{k}, (10) \displaystyle\xi_{k} \displaystyle=\mathcal{D}\zeta_{k}. (11) where the spectral radius, \rho(\Psi)<1 for stationarity, and \eta_{k} is an i.i.d. Gaussian noise process. The stationarity of the noise process ensures that \Sigma^{\zeta}_{k}\equiv\Sigma^{\zeta} . The propagation equation for \Sigma_{k} is written as \displaystyle\Sigma_{k+1} \displaystyle=\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}\begin{bmatrix}K_{k}\\ I_{r}\end{bmatrix}\Sigma_{k}\begin{bmatrix}K_{k}\\ I_{r}\end{bmatrix}^{\mbox{\tiny\sf T}}\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}^{\mbox{\tiny\sf T}} (12) \displaystyle+\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}\begin{bmatrix}K_{k}\\ I_{r}\end{bmatrix}\Sigma^{\chi\zeta}_{k}\mathcal{D}^{\mbox{\tiny\sf T}}+(\Sigma^{\chi\zeta}_{k}\mathcal{D}^{\mbox{\tiny\sf T}})^{\mbox{\tiny\sf T}}\begin{bmatrix}K_{k}\\ I_{r}\end{bmatrix}^{\mbox{\tiny\sf T}}\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}^{\mbox{\tiny\sf T}} \displaystyle+\mathcal{B}\Sigma^{\nu}_{k}\mathcal{B}^{\mbox{\tiny\sf T}}+\mathcal{D}\Sigma^{\zeta}\mathcal{D}^{\mbox{\tiny\sf T}}, where \Sigma^{\chi\zeta}_{k}\triangleq\mathbb{E}[\chi_{k}\zeta_{k}^{\mbox{\tiny\sf T}}] , is propagated according to the equation \displaystyle\Sigma^{\chi\zeta}_{k+1}=\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}\begin{bmatrix}K_{k}\\ I_{r}\end{bmatrix}\Sigma^{\chi\zeta}_{k}\Psi^{\mbox{\tiny\sf T}}+\mathcal{D}\Sigma^{\zeta}\Psi^{\mbox{\tiny\sf T}}. (13) See the Appendix for the full derivations. The covariance controller consists of a feedback term K_{k}(\chi_{k}-\mu_{k}) and an independent random term \nu_{k}\sim\mathcal{N}(0,\Sigma^{\nu}_{k}) . The output covariance steering problem can now be posed as \displaystyle\min_{\Sigma_{k},\Sigma^{\chi\zeta}_{k},K_{k},\Sigma^{\nu}_{k}}J=\sum_{k=0}^{N-1} \displaystyle\mathrm{tr}(Q\Sigma_{k})+\mathrm{tr}(R(K_{k}\Sigma_{k}K_{k}^{\mbox{\tiny\sf T}}+\Sigma^{\nu}_{k})), (14a) such that, for all k=0,1,\dots,N-1 , \displaystyle~~~~~\eqref{eq:chi_covar_propagation},~\eqref{eq:chi_xi_covar_propagation} (14b) \displaystyle\Sigma_{0}^{y}=\Sigma_{i}^{y},\quad\Sigma_{N}^{y}=\Sigma_{f}^{y}, (14c) \displaystyle\Sigma^{\chi\zeta}_{0}=\Sigma^{\chi\zeta}_{i}, (14d) III Data-Driven System Representation The data-driven output distribution steering problem is addressed using an indirect method to compute the feed-forward mean steering controller and a direct method to parameterize the feedback gains of the covariance steering controller. As shown in [ 24 ] , an estimate of the noise realization is required to solve the direct covariance steering problem. In this section, we present two methods to identify the matrices \mathcal{A},\mathcal{B} in ( 8 ). The identified model is then used to express the mean dynamics and to compute an estimate of the noise realization. III-A Persistency of Excitation and Multi-Output Systems Consider the data matrices constructed from a (T+1) -long trajectory, U_{0,T}\triangleq\mathcal{H}_{0,1,T}(u) , Z_{0,T}\triangleq\mathcal{H}_{0,1,T}(z) , Z_{1,T}\triangleq\mathcal{H}_{1,1,T}(z) , \mathcal{X}_{0,T}\triangleq\mathcal{H}_{0,1,T}(\chi)=LZ_{0,T} , \mathcal{X}_{1,T}\triangleq\mathcal{H}_{1,1,T}(\chi)=LZ_{1,T} , Y_{0,T}\triangleq\mathcal{H}_{0,1,T}(y) , \Xi_{0,T}\triangleq\mathcal{H}_{0,1,T}(\xi) , \Xi_{1,T}\triangleq\mathcal{H}_{1,1,T}(\xi) , \mathcal{Z}_{0,T}\triangleq\mathcal{H}_{0,1,T}(\zeta) . Theorem 1 . (Willem’s Fundamental Lemma [ 29 ] , expressed for single-step predictions). Letting \zeta_{k}\equiv 0 in the discrete-time LTI system ( 3 ), and assuming that the system is controllable and that the sequence U_{0,T} is persistently exciting of order h+1 , then the matrix S\triangleq\mathcal{H}_{0,1,T}(\mathrm{col}(u,z))=\begin{bmatrix}U_{0,T}\\ Z_{0,T}\end{bmatrix} has full row rank, that is, \mathrm{rank}(S)=m+h. (15) It is critical that S has full row rank in order to be used as the regressor matrix in the model estimation methods. However, as shown in [ 1 ] , in the common case where p\ell>n , S does not have full row rank. To alleviate this problem, and borrowing from the methods in [ 1 ] , we have introduced the non-minimal state space representation \chi_{k} in ( 7 ), where the matrix L is computed such that the data matrix \begin{bmatrix}U_{0,T}\\ LZ_{0,T}\end{bmatrix} has full row rank equal to m(\ell+1)+n . To compute L in the presence of noise in the data, we use the singular value decomposition (SVD) of S , which is expressed as \displaystyle S=\begin{bmatrix}U_{1}&U_{2}\end{bmatrix}\begin{bmatrix}\Lambda_{1}&0\\ 0&\Lambda_{2}\end{bmatrix}\begin{bmatrix}V_{1}^{\mbox{\tiny\sf T}}\\ V_{2}^{\mbox{\tiny\sf T}}\end{bmatrix}, where the partitioning of the block matrices is such that \Lambda_{1} contains the \kappa largest singular values. We would like to select \kappa=m\ell+n , but since the order n might be unknown, we select \kappa>m\ell and such that the selected singular values are clearly distinguishable from the smaller singular values, which correspond to noise. The matrix L is then computed as \displaystyle L=\Lambda_{1}^{-1}U_{1}^{\mbox{\tiny\sf T}}, (16) and it essentially selects the row space of the matrix S , since V_{1}^{\mbox{\tiny\sf T}}=LS . Finally, we define the data matrix \displaystyle P\triangleq\begin{bmatrix}U_{0,T}\\ \mathcal{X}_{0,T}\end{bmatrix}=\begin{bmatrix}U_{0,T}\\ LZ_{0,T}\end{bmatrix}. (17) It can be easily shown that the collected data must satisfy the following equations \displaystyle\mathcal{X}_{1,T} \displaystyle=\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}P+\Xi_{0,T}, (18) \displaystyle Y_{0,T} \displaystyle=\mathcal{C}\mathcal{X}_{0,T}+\mathcal{Z}_{0,T}. (19) III-B Total Least Squares Method The standard approach to solve the model estimation problem would be to use the ordinary least squares (LS) method with P in ( 18 ) as the regressors and \mathcal{X}_{1,T} as the observations. However, it is evident that, for our problem, uncertainty exists both in the regressor matrix and in the observations, thus making the Total Least Squares (TLS) [ 27 ] approach more suitable than standard LS. The TLS problem is formulated as \displaystyle\min_{\mathcal{A},\mathcal{B},\Delta\mathcal{X},\Delta P}\left\|\begin{bmatrix}\Delta\mathcal{X}&\Delta P\end{bmatrix}\right\|_{\mathrm{F}}, such that \displaystyle\mathcal{X}_{1,T}+\Delta\mathcal{X}=\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}(P+\Delta P). This problem admits a closed-form solution via the singular value decomposition of the augmented matrix U\Lambda V^{\mbox{\tiny\sf T}}=\begin{bmatrix}P\\ \mathcal{X}_{1,T}\end{bmatrix}^{\mbox{\tiny\sf T}} . By partitioning the matrix V as \begin{bmatrix}V_{11}&V_{12}\\ V_{21}&V_{22}\end{bmatrix} , the estimated model is computed as \displaystyle\begin{bmatrix}\hat{\mathcal{B}}&\hat{\mathcal{A}}\end{bmatrix}=(-V_{21}^{\mbox{\tiny\sf T}}(V_{22}^{\mbox{\tiny\sf T}})^{-1})^{\mbox{\tiny\sf T}}. (21) III-C Instrumental Variables Method As discussed in Section- II-D , \chi_{k} is correlated with \zeta_{k} . In such cases, where the regressors are correlated with the noise, the TLS estimator will not be consistent, necessitating another approach. Here, we apply the Instrumental Variables (IV) method [ 30 ] . Remark 1 . Despite the IV estimator being theoretically consistent, it is less data-efficient than the TLS estimator, and it performs poorly both in terms of mean error and error variance when the available sample size is small. Thus, TLS serves as an alternative method for such cases. In the IV method, a number of parameters, called the “instrumental variables” or “instruments,” which are independent of \zeta_{k} , are used as the independent variables of the problem. To present the IV method, we consider the parameters \beta\triangleq\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix} , the regressors j_{k}\triangleq[u_{k}^{\mbox{\tiny\sf T}}\ \chi_{k}^{\mbox{\tiny\sf T}}] , the regressor matrix J\triangleq P^{\mbox{\tiny\sf T}} , and the observations Y\triangleq\mathcal{X}_{1,T}^{\mbox{\tiny\sf T}} . The dynamics ( 8 ) are a system of equations in which the regressors are the same for each equation and the error terms (components of \zeta_{k} ) are independent among equations. Since the model is unknown, we do not consider any relation between the parameters of different equations (e.g., some element of \mathcal{A} being common in multiple equations). Under these conditions, the parameters of the system can be consistently estimated on a per-equation basis, without requiring joint estimation of all equations. If additional knowledge about the model structure is available, this information can be exploited to construct a more efficient estimator. However, such structural knowledge is not required for consistency of the IV estimator. An attractive choice for the instrument vector for our problem is the control input at some time, k , and the p past control inputs, i.e., g_{k}\triangleq[u_{k}^{\mbox{\tiny\sf T}}\ u_{k-1}^{\mbox{\tiny\sf T}}\ \cdots\ u_{k-p}^{\mbox{\tiny\sf T}}] . We consider the partition j_{k}=\begin{bmatrix}j_{k,1}\ j_{k,2}\end{bmatrix} , where j_{k,1} are the exogenous components of j (i.e., those not correlated with the noise) and j_{k,2} are the endogenous components of j_{k} . The goal is to express j_{k,2} as a linear combination of g_{k} and j_{k,1} . To this end, we employ the two-stage least squares (2SLS) estimator, which is the IV estimator that chooses the linear combinations of g_{k} with the highest correlation with j_{k,2} [ 30 ] . The 2SLS estimator is given by \displaystyle\hat{\beta}=\left[(J^{\mbox{\tiny\sf T}}G(G^{\mbox{\tiny\sf T}}G)^{-1}G^{\mbox{\tiny\sf T}}J)^{-1}(J^{\mbox{\tiny\sf T}}G(G^{\mbox{\tiny\sf T}}G)^{-1}G^{\mbox{\tiny\sf T}}Y)\right]^{\mbox{\tiny\sf T}}, (22) where G\triangleq[g_{0}\ g_{1}\ \cdots\ g_{T-1}] is the matrix of instrument vectors over all samples. In order for the 2SLS estimator to be consistent, the following assumptions are made: Assumption 1 . The instruments are independent of the noise, \mathbb{E}(g_{k}\zeta_{k}^{\mbox{\tiny\sf T}})=0 . Assumption 2 . The instruments are linearly independent, \mathrm{rank}(\mathbb{E}(g_{k}g_{k}^{\mbox{\tiny\sf T}}))=p+1 . Assumption 3 . The instruments are linearly dependent with the regressors to some degree, \mathrm{rank}(\mathbb{E}(g_{k}j_{k}^{\mbox{\tiny\sf T}}))=r+m . A necessary condition for Assumption 3 to hold is that the number of instruments is greater than or equal to the number of regressors, i.e., p+1\geq r+m . Assumptions 1 and 2 pertain to a proper design of the instrument vector, while Assumption 3 is verified numerically by computing the expectation from the sample data. For the selected instrument vector, Assumption 1 is satisfied since the control inputs during the data collection phase are independent of the noise. Moreover, if i.i.d control inputs are applied during the data collection process, which is a common choice to achieve persistency of excitation, then the matrix \mathbb{E}(g_{k}g_{k}^{\mbox{\tiny\sf T}}) will have full rank p+1 . The following theorem establishes the consistency of the 2SLS estimator under the aforementioned assumptions. Theorem 2 . (Theorem 5.1 [ 30 ] ) If Assumptions 1 , 2 , 3 are satisfied, then the 2SLS estimator is consistent. IV Data-Driven Distribution Steering In this section, we present the indirect formulation of the mean steering problem and the direct formulation of the covariance steering problem. IV-A Data-driven Mean Steering The mean steering problem ( 9 ) is trivially expressed in terms of an estimated model, since the cost ( 9a ) remains the same, while the dynamics constraints ( 9b ) are written in terms of \hat{\mathcal{A}},\hat{\mathcal{B}} instead of \mathcal{A},\mathcal{B} . For the initial and terminal output mean constraints ( 9c ), we need to estimate the matrix \mathcal{C} in ( 8 ). An estimate, \hat{\mathcal{C}} , is obtained using TLS / IV with \mathcal{X}_{0,T} as the regressors and Y_{0,T} as the observations. Considering the instrument matrix G as in ( 22 ), and defining the regressor and observation matrices J_{c}\triangleq\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}} and Y_{c}\triangleq Y_{0,T}^{\mbox{\tiny\sf T}} , respectively, the IV estimator for \mathcal{C} is given by \displaystyle\hat{\mathcal{C}}=\left[(J_{c}^{\mbox{\tiny\sf T}}G(G^{\mbox{\tiny\sf T}}G)^{-1}G^{\mbox{\tiny\sf T}}J_{c})^{-1}(J_{c}^{\mbox{\tiny\sf T}}G(G^{\mbox{\tiny\sf T}}G)^{-1}G^{\mbox{\tiny\sf T}}Y_{c})\right]^{\mbox{\tiny\sf T}}, (23) The initial and terminal mean constraints are now posed as \displaystyle\mu^{y}_{i}=\hat{\mathcal{C}}\mu_{0},\quad\mu^{y}_{f}=\hat{\mathcal{C}}\mu_{N}. (24) IV-B Data-driven Covariance Steering Using the data matrix P from ( 17 ), we construct the matrix \displaystyle\Phi=PP^{\mbox{\tiny\sf T}}. (25) For persistently exciting data, Theorem 1 implies that P will have full row rank, and thus, \Phi will have full rank as well. Hence, there exists a solution G_{k}\in\mathbb{R}^{(m+r)\times r} to the following system of linear equations \displaystyle\begin{bmatrix}K_{k}\\ I\end{bmatrix}=\Phi G_{k}. (26) From ( 26 ), we can express the feedback gains K_{k} in ( 12 ) and ( 13 ) in terms of the data matrix \Phi and the decision variables G_{k} . Using ( 18 ) and considering the sub-optimal control law u_{k}=K_{k}(\chi_{k}-\mu_{k})+v_{k} , we derive the following model-independent form of the covariance and cross-covariance propagation equations \displaystyle\Sigma_{k+1} \displaystyle=(\mathcal{X}_{1,T}-\mathcal{D}\mathcal{Z}_{0,T})P^{\mbox{\tiny\sf T}}G_{k}\Sigma_{k}G_{k}^{\mbox{\tiny\sf T}}P(\mathcal{X}_{1,T}-\mathcal{D}\mathcal{Z}_{0,T})^{\mbox{\tiny\sf T}} \displaystyle+(\mathcal{X}_{1,T}-\mathcal{D}\mathcal{Z}_{0,T})P^{\mbox{\tiny\sf T}}G_{k}\Sigma^{\chi\zeta}_{k}\mathcal{D}^{\mbox{\tiny\sf T}} \displaystyle+\mathcal{D}(\Sigma^{\chi\zeta}_{k})^{\mbox{\tiny\sf T}}G_{k}^{\mbox{\tiny\sf T}}P(\mathcal{X}_{1,T}-\mathcal{D}\mathcal{Z}_{0,T})^{\mbox{\tiny\sf T}}+\mathcal{D}\Sigma^{\zeta}\mathcal{D}^{\mbox{\tiny\sf T}}, (27) \displaystyle\Sigma^{\chi\zeta}_{k+1} \displaystyle=(\mathcal{X}_{1,T}-\mathcal{D}\mathcal{Z}_{0,T})P^{\mbox{\tiny\sf T}}G_{k}\Sigma^{\chi\zeta}_{k}\Psi^{\mbox{\tiny\sf T}}+\mathcal{D}\Sigma^{\zeta}\Psi^{\mbox{\tiny\sf T}}. (28) In this data-driven formulation, the matrices \mathcal{A},\mathcal{B} are not parameterized separately, but the whole feedback interconnection is expressed in terms of the data. Thus, we cannot express the term \mathcal{B}\Sigma^{\nu}\mathcal{B}^{\mbox{\tiny\sf T}} in ( 12 ) in a model-independent fashion, which is the reason for using the sub-optimal control law u_{k}=K_{k}(\chi_{k}-\mu_{k})+v_{k} in ( 27 ) instead of u_{k}=K_{k}(\chi_{k}-\mu_{k})+v_{k}+\nu_{k} . Next, the output covariance is expressed in terms of the optimization variables as \displaystyle\Sigma^{y}_{k} \displaystyle=\mathbb{E}[(y_{k}-\mu^{y}_{k})(y_{k}-\mu^{y}_{k})^{\mbox{\tiny\sf T}}] \displaystyle=\mathcal{C}\Sigma_{k}\mathcal{C}^{\mbox{\tiny\sf T}}+\mathcal{C}\Sigma^{\chi\zeta}_{k}+(\Sigma^{\chi\zeta}_{k})^{\mbox{\tiny\sf T}}\mathcal{C}^{\mbox{\tiny\sf T}}+\Sigma^{\zeta}. (29) See the Appendix for the full derivation. Since \mathcal{C} is unknown, we will use \hat{\mathcal{C}} from ( 23 ) to implement ( 29 ). Furthermore, from the first block equation of ( 26 ), the objective function of the optimization problem is expressed in terms of the optimization variables as \displaystyle J=\sum_{k=0}^{N-1} \displaystyle\mathrm{tr}(Q\Sigma_{k})+\mathrm{tr}(R\Theta G_{k}\Sigma_{k}G_{k}^{\mbox{\tiny\sf T}}\Theta^{\mbox{\tiny\sf T}}), (30) where \Theta=\begin{bmatrix}U_{0,T}U_{0,T}^{\mbox{\tiny\sf T}}&U_{0,T}\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}}\end{bmatrix} . To implement equations ( 27 )-( 29 ), the noise realization and covariance \mathcal{Z}_{0,T},\Sigma^{\zeta} , the dynamics and noise model matrices \mathcal{D},\Psi , and the initial cross-covariance \Sigma^{\chi\zeta}_{i} must be estimated. To this end, we first approximate the realizations of \xi and \zeta from equations ( 18 ) and ( 19 ) as the residuals between the observations and the predictions of an estimated model. Then, the covariances of \xi_{k} and \zeta_{k} are computed as the sample covariances of the estimated realizations, as follows \displaystyle\hat{\Xi}_{0,T} \displaystyle=\mathcal{X}_{1,T}-\begin{bmatrix}\hat{\mathcal{B}}&\hat{\mathcal{A}}\end{bmatrix}P,\quad\quad\hat{\Sigma}^{\xi}=\frac{1}{T}\hat{\Xi}_{0,T}\hat{\Xi}_{0,T}^{\mbox{\tiny\sf T}} (31) \displaystyle\hat{\mathcal{Z}}_{0,T} \displaystyle=Y_{0,T}-\hat{\mathcal{C}}\mathcal{X}_{0,T},\quad\quad\hat{\Sigma}^{\zeta}=\frac{1}{T}\hat{\mathcal{Z}}_{0,T}\hat{\mathcal{Z}}_{0,T}^{\mbox{\tiny\sf T}}. (32) Since the noise process is stationary, \hat{\Sigma}^{\xi}_{k}\equiv\hat{\Sigma}^{\xi} and \hat{\Sigma}^{\zeta}_{k}\equiv\hat{\Sigma}^{\zeta} . Additionally, an estimate, \hat{\Psi} , of the noise model ( 10 ) is obtained using TLS with the estimated realizations \hat{\mathcal{Z}}_{0,T-1}\triangleq\mathcal{H}_{0,1,T-1}(\hat{\zeta}) as the regressors and \hat{\mathcal{Z}}_{1,T-1}\triangleq\mathcal{H}_{1,1,T-1}(\hat{\zeta}) as the observations. Furthermore, using ordinary least squares on ( 11 ), we compute an approximation, \hat{\mathcal{D}} , of \mathcal{D} using \hat{\mathcal{Z}}_{0,T} as the regressors and \hat{\Xi}_{0,T} as the observations. Finally, expressing the output equation of ( 8 ) for the estimated model, multiplying from the right by \chi_{k}^{\mbox{\tiny\sf T}} , and taking expectations, an estimate, \hat{\Sigma}^{\chi\zeta}_{i} , of the initial cross-covariance is obtained as \displaystyle\hat{\Sigma}^{\chi\zeta}_{i}=(\mathbb{E}[y_{k}\chi_{k}^{\mbox{\tiny\sf T}}]-\hat{\mathcal{C}}\mathbb{E}[\chi_{k}\chi_{k}^{\mbox{\tiny\sf T}}])^{\mbox{\tiny\sf T}}, (33) where the expectations are computed as \mathbb{E}[y_{k}\chi_{k}^{\mbox{\tiny\sf T}}]=Y_{0,T}\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}}/T and \mathbb{E}[\chi_{k}\chi_{k}^{\mbox{\tiny\sf T}}]=\mathcal{X}_{0,T}\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}}/T . Equations ( 27 ), ( 28 ), and ( 30 ), yield a non-convex optimization problem due to the terms G_{k}\Sigma^{\chi\zeta}_{k} and G_{k}\Sigma_{k}G_{k}^{\mbox{\tiny\sf T}} . To alleviate this issue, we define the new decision variables U_{k}\triangleq G_{k}\Sigma_{k} and W_{k}\triangleq G_{k}\Sigma^{\chi\zeta}_{k} . The term G_{k}\Sigma_{k}G_{k}^{\mbox{\tiny\sf T}} now becomes U_{k}\Sigma^{-1}_{k}U_{k}^{\mbox{\tiny\sf T}} , which is still non-convex. In order to convexify the problem, we introduce the slack variable Y_{k}\succeq U_{k}\Sigma_{k}^{-1}U_{k}^{\mbox{\tiny\sf T}} . Remark 2 . In contrast to [ 26 ] , the relaxation Y_{k}\succeq U_{k}\Sigma_{k}^{-1}U_{k}^{\mbox{\tiny\sf T}} does not result in a lossless convexification for the present problem. This implies that, if the estimated terms in ( 34 ) were exactly equal to the true ones, then the relaxed terminal constraint \Sigma^{y}_{N}\preceq\Sigma^{y}_{f} would be met. However, under the presence of estimation errors, satisfaction of this terminal condition is not guaranteed. We consider the quantification of these errors and the design of robust covariance steering controllers as a valuable direction for further investigation. The relaxed optimization problem is formulated as follows \displaystyle\min_{\Sigma_{k},\Sigma^{\chi\zeta}_{k},U_{k},W_{k},Y_{k}}\sum_{k=0}^{N-1} \displaystyle\mathrm{tr}(Q\Sigma_{k})+\mathrm{tr}(R\Theta Y_{k}\Theta^{\mbox{\tiny\sf T}}), (34a) such that, for all k=0,1,\dots,N-1 , \displaystyle\Sigma_{k+1} \displaystyle=(\mathcal{X}_{1,T}-\hat{\mathcal{D}}\hat{\mathcal{Z}}_{0,T})P^{\mbox{\tiny\sf T}}Y_{k}P(\mathcal{X}_{1,T}-\hat{\mathcal{D}}\hat{\mathcal{Z}}_{0,T})^{\mbox{\tiny\sf T}} \displaystyle+(\mathcal{X}_{1,T}-\hat{\mathcal{D}}\hat{\mathcal{Z}}_{0,T})P^{\mbox{\tiny\sf T}}W_{k}\hat{\mathcal{D}}^{\mbox{\tiny\sf T}} \displaystyle+\hat{\mathcal{D}}W_{k}^{\mbox{\tiny\sf T}}P(\mathcal{X}_{1,T}-\hat{\mathcal{D}}\hat{\mathcal{Z}}_{0,T})^{\mbox{\tiny\sf T}}+\hat{\mathcal{D}}\hat{\Sigma}^{\zeta}\hat{\mathcal{D}}^{\mbox{\tiny\sf T}}, (34b) \displaystyle\Sigma^{\chi\zeta}_{k+1} \displaystyle=(\mathcal{X}_{1,T}-\hat{\mathcal{D}}\hat{\mathcal{Z}}_{0,T})P^{\mbox{\tiny\sf T}}W_{k}\hat{\Psi}^{\mbox{\tiny\sf T}}+\hat{\mathcal{D}}\hat{\Sigma}^{\zeta}\hat{\Psi}^{\mbox{\tiny\sf T}}, (34c) \displaystyle\begin{bmatrix}\Sigma_{k}&U_{k}^{\mbox{\tiny\sf T}}&\Sigma^{\chi\zeta}_{k}\hat{\mathcal{D}}^{\mbox{\tiny\sf T}}\\ U_{k}&Y_{k}&W_{k}\hat{\mathcal{D}}^{\mbox{\tiny\sf T}}\\ \hat{\mathcal{D}}(\Sigma^{\chi\zeta}_{k})^{\mbox{\tiny\sf T}}&\hat{\mathcal{D}}W_{k}^{\mbox{\tiny\sf T}}&\hat{\mathcal{D}}\hat{\Sigma}^{\zeta}\hat{\mathcal{D}}^{\mbox{\tiny\sf T}}\end{bmatrix}\succeq 0, (34d) \displaystyle\Sigma_{k} \displaystyle=\begin{bmatrix}\mathcal{X}_{0,T}U_{0,T}^{\mbox{\tiny\sf T}}&\mathcal{X}_{0,T}\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}}\end{bmatrix}U_{k}, (34e) \displaystyle\Sigma^{\chi\zeta}_{k} \displaystyle=\begin{bmatrix}\mathcal{X}_{0,T}U_{0,T}^{\mbox{\tiny\sf T}}&\mathcal{X}_{0,T}\mathcal{X}_{0,T}^{\mbox{\tiny\sf T}}\end{bmatrix}W_{k}, (34f) \displaystyle\Sigma^{y}_{f} \displaystyle=\hat{\mathcal{C}}\Sigma_{N}\hat{\mathcal{C}}^{\mbox{\tiny\sf T}}+\hat{\mathcal{C}}\Sigma^{\chi\zeta}_{N}+(\Sigma^{\chi\zeta}_{N})^{\mbox{\tiny\sf T}}\hat{\mathcal{C}}^{\mbox{\tiny\sf T}}+\hat{\Sigma}^{\zeta}, (34g) \displaystyle\Sigma^{y}_{i} \displaystyle=\hat{\mathcal{C}}\Sigma_{0}\hat{\mathcal{C}}^{\mbox{\tiny\sf T}}+\hat{\mathcal{C}}\Sigma^{\chi\zeta}_{0}+(\Sigma^{\chi\zeta}_{0})^{\mbox{\tiny\sf T}}\hat{\mathcal{C}}^{\mbox{\tiny\sf T}}+\hat{\Sigma}^{\zeta}, (34h) \displaystyle\Sigma^{\chi\zeta}_{0} \displaystyle=\hat{\Sigma}^{\chi\zeta}_{i}, (34i) where the constraints ( 34e ) and ( 34f ) come from the second block of ( 26 ) after multiplying by \Sigma_{k} and \Sigma_{k}^{\chi\zeta} from the right, respectively. Moreover, the constraint ( 34d ) implies that the covariance matrix of the the vector a_{k}\triangleq[\chi_{k}\ G_{k}\chi_{k}\ \hat{\mathcal{D}}\zeta_{k}]^{\mbox{\tiny\sf T}} must be positive semi-definite. V Numerical Example V-A Model Estimation First, we compare the performance of the aforementioned model estimation methods in terms of their sample efficiency and their performance, predicated on the error metric e=\left\|\begin{bmatrix}\mathcal{B}&\mathcal{A}\end{bmatrix}-\begin{bmatrix}\hat{\mathcal{B}}&\hat{\mathcal{A}}\end{bmatrix}\right\|_{\mathrm{F}} . The statistical properties of e were computed by running the model estimation procedure for 200 different datasets with sample sizes from 50 up to 5,000 samples. In addition to the TLS and IV methods, the ordinary least squares method (LS) was added for comparison, and to demonstrate the superiority of TLS in the present problem. A system with p\ell=n was used because in this case L=I , and thus, we have knowledge of the true matrices \mathcal{A},\mathcal{B} . \displaystyle A=\begin{bmatrix}0.5&1&0&0\\ 0&0.5&0&0\\ 0&0&0.3&1\\ 0&0&0&0.3\end{bmatrix},\quad B=\begin{bmatrix}1&-0.7\\ 0&1\\ 0.5&-0.2\\ 0&0.5\end{bmatrix}, \displaystyle C=\begin{bmatrix}1&0&0&0\\ 0&0&1&0\end{bmatrix},\quad\Sigma^{w}=0.1^{2}I,\quad\Sigma^{q}=0.2^{2}I. The mean and the variance of e as a function of the dataset size are presented in Figure 1 . For very small sample sizes, the IV method exhibits poor performance both in terms of mean error and error variance. However, as the sample size increases, IV outperforms the other methods. Fig. 1: Model estimation error. V-B Density Steering To demonstrate the density steering algorithm, the following MIMO system was used: \displaystyle A=0.55\begin{bmatrix}-0.5&1.4&0.4\\ -0.9&0.3&-1.5\\ 1.1&1.0&-0.4\\ \end{bmatrix},\quad B=\begin{bmatrix}0.1&-0.3\\ -0.1&-0.7\\ 0.7&-1\\ \end{bmatrix}, \displaystyle C=\begin{bmatrix}1&0&0\\ 0&1&0\end{bmatrix},\quad\Sigma^{w}=0.1^{2}I_{3},\quad\Sigma^{q}=0.1^{2}I_{2}, for which \ell=2 and p\ell=4>n=3 . The initial output mean and covariance are \mu_{i}^{y}=[0,\ 0]^{\mbox{\tiny\sf T}} and \Sigma_{i}^{y}=2.5I_{2} . A Lissajous reference trajectory was set for the mean, with the terminal mean target being \mu^{y}_{f}=[-5.878\ -9.511]^{\mbox{\tiny\sf T}} and the terminal output covariance target being \Sigma^{y}_{f}=0.1\Sigma_{i}^{y} . The horizon length is N=15 steps, and the IV method was used for model and noise estimation. The distribution steering framework was run for various dataset sizes, for 200 different datasets of each size. For each dataset, we simulated 1,000 trajectories with different initial conditions and computed the empirical terminal mean and covariance. The mean steering performance is evaluated on the metric of the terminal mean error norm, which is presented in Figure 2 for various dataset sizes. With respect to the covariance steering performance, we present the terminal covariance ellipses in Figure 3 . Finally, using the IV method for mean steering and noise estimation and a dataset of 1,500 samples, the resulting output trajectories and covariance ellipses are presented in Figure 4 , where the blue line represents the optimal solution of the mean steering problem. It is evident that both the mean and the covariance steering performance improve as the dataset size increases, since the IV method is able to provide better estimates. We consider the development of a robust formulation against estimation uncertainty, similar to that presented in [ 24 ] , as a fruitful avenue of future work. Fig. 2: Terminal mean error. Fig. 3: Terminal covariance ellipses. Fig. 4: Output trajectories in phase plane. VI