Weak Closed-loop Solvability for Discrete-time Linear-Quadratic Optimal Control

Yue Sun, Xianping Wu, Xun Li · 2025

Read Weak Closed-loop Solvability for Discrete-time Linear-Quadratic Optimal Control by Yue Sun, Xianping Wu, Xun Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11415.txt.

Introduction

Yue Sun, Xianping Wu, and Xun Li take up a control theory question in a 2025 paper. Their topic is discrete-time linear-quadratic control, a standard way to steer a system step by step at low cost. They compare three senses in which such a problem can be solved: open-loop, closed-loop, and a weaker closed-loop form.

The authors first tie closed-loop solvability to solving a generalized Riccati equation, a key equation in the field. They then show that even when only an open-loop solution exists, a weaker feedback form can still be found by a perturbation method. Their tools rest on solving forward and backward difference equations step by step, and a worked example shows the results in action.

Catalog record

CREATORYue Sun, Xianping Wu, Xun Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11415

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Weak Closed-loop Solvability for Discrete-time Linear-Quadratic Optimal Control Yue Sun, Xianping Wu, Xun Li Abstract In this paper, the open-loop, closed-loop, and weak closed-loop solvability for discrete-time linear-quadratic (LQ) control problem is considered due to the fact that it is always open-loop optimal solvable if the LQ control problem is closed-loop optimal solvable but not vice versa. The contributions are two-fold. On the one hand, the equivalent relationship between the closed-loop optimal solvability and the solution of the generalized Riccati equation is given. On the other hand, when the system is merely open-loop solvable, we have found the equivalent existence form of the optimal solution by perturbation method, which is said to be a weak closed-loop solution. Moreover, it obtains that there is an open-loop optimal control with a linear feedback form of the state. The essential technique is to solve the forward and backward difference equations by iteration. An example sheds light on the theoretical results established. I Introduction Optimal control theory is an essential component of modern control theory, and linear-quadratic (LQ) optimal control is the fundamental problem and the core foundation of optimal control. The research on traditional LQ control problems began in the 1960s, and after Kalman’s pioneering work [ 8 ] and [ 9 ] , LQ control problems have been widely and deeply studied. Deterministic LQ problem, such as the study in [ 10 ] with single input, is concerned with \displaystyle\min\sum^{N-1}_{t=0}(x^{\prime}_{t}Qx_{t}+2u^{\prime}_{t}Sx_{t}+u^{\prime}_{t}Ru_{t})+x^{\prime}_{N}Hx_{N}, \displaystyle\mbox{subject\!\quad\!to}\quad x_{t+1}=Ax_{t}+Bu_{t}, where x_{t}\in\mathbb{R}^{n} is the state with the initial value x_{0} , u_{t}\in\mathbb{R}^{m} is the control input, A and B are constant matrices with compatible dimensions. Under the traditional assumption, the cost functional always has positive semi-definite weighting matrices for the control and the state. For a constant \delta>0 , assume that \displaystyle R\geq\delta I,\quad Q-S^{\prime}R^{-1}S\geq 0. Under these conditions, the LQ problem admits a unique solution if and only if the Riccati equation with the terminal value P_{N}=H has a unique positive definite solution satisfying \displaystyle P_{t} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptQ+A^{\prime}P_{t+1}A-A^{\prime}P_{t+1}B(R+B^{\prime}P_{t+1}B)^{-1}B^{\prime}P_{t+1}A, while the unique open-loop optimal solution to the problem is a linear feedback form of the state. Based on the assumption of positive semi-definite weighting matrices, the deterministic LQ problem with time delay has been considered since the 1970s. Koivo-Lee [ 7 ] , Alekal-Brunovsky-Chyung-Lee [ 1 ] and Artstein [ 2 ] , Delfour [ 3 ] successively focused on the single-delay case and the multiple-delay case. In the positive semi-definite case, no problems exist with the existence of an optimal solution since the optimal cost functional is always positive semi-definite. For the indefinite LQ problem, that is, there is only an assumption on the symmetry of the weighting matrices, referring to Ran-Trentelman [ 13 ] , which studied the infinite-horizon LQ problem with an indefinite quadratic form and presented necessary and sufficient conditions for the existence of an optimal control. As pointed out in Ferrante-Ntogramatzidis [ 4 ] , unlike the positive semi-definite case, the finite-horizon indefinite LQ problem may not admit solutions because the performance index can be arbitrarily negative through suitable control actions. Using the generalized Riccati difference equation: \displaystyle P_{t} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptQ+A^{\prime}P_{t+1}A-A^{\prime}P_{t+1}B(R+B^{\prime}P_{t+1}B)^{\dagger}B^{\prime}P_{t+1}A, with the terminal value P_{N}=H , Ferrante-Ntogramatzidis [ 4 ] presented a necessary and sufficient condition for the existence of an optimal control. In the traditional LQ optimal control with a single input, the open-loop optimal solvability is equivalent to the closed-loop optimal solvability. In the indefinite LQ problem, the closed-loop optimal solvability implies the open-loop optimal solvability, but we have found in the following example that it’s not vice versa. Example 1 . Consider the following system with one-dimension state equation: \displaystyle x_{t+1}=x_{t}+u_{t},\quad t\in\{0,1\}, (1) with the initial value x_{0} . The associated cost functional is given by \displaystyle J(x_{0},u)=x^{2}_{2}-\sum\limits^{1}_{t=0}u^{2}_{t}. (2) Following from the difference Riccati equation ( I ) with the terminal value P_{2}=1 , and by simple calculation, the solution is exactly P_{t}\equiv 1 , t\in\{0,1,2\} . Combining with the maximum principle, a generalized Riccati equation approach specifies the corresponding closed-loop solution as follows \displaystyle u^{*}_{t}=-(R+B^{\prime}P_{t+1}B)^{\dagger}B^{\prime}P_{t+1}Ax_{t}\equiv 0,\quad t\in\{0,1\}, (3) due to R=-1 and 0^{\dagger}=0 . It should be noted that ( 3 ) is no longer an optimal open-loop solution if x_{0}\neq 0 . To be specific, adding u^{*}_{t} in ( 3 ) into ( 1 ), the state x^{*}_{t} is calculated as \displaystyle x^{*}_{t}=x_{0}+\sum\limits^{t-1}_{i=0}u^{*}_{i}=x_{0},\quad t\in\{1,2\}. To this end, the cost functional in ( 2 ) is given by \displaystyle J(x_{0},u^{*})=x^{2}_{0}>0, for any x_{0}\neq 0 . Moreover, let \bar{u}_{t} be the control defined by \displaystyle\bar{u}_{t}=Mx_{t},\quad M=-0.3522. In this way, we have \displaystyle\bar{u}_{0} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptMx_{0}, \displaystyle\bar{u}_{1} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptMx_{1}=M(x_{0}+u_{0})=M(x_{0}+Mx_{0}) \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptM(1+M)x_{0}. Substituting \bar{u}_{0} and \bar{u}_{1} into ( 1 ) yields, \displaystyle x_{2}=x_{0}+\bar{u}_{0}+\bar{u}_{1}. Thus, the cost functional J(x_{0},\bar{u}) is given by \displaystyle J(x_{0},\bar{u}) \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptx^{2}_{2}-\bar{u}^{2}_{0}-\bar{u}^{2}_{1} (4) \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003pt(x_{0}+\bar{u}_{0}+\bar{u}_{1})^{2}-\bar{u}^{2}_{0}-\bar{u}^{2}_{1} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptx^{2}_{0}+2x_{0}\bar{u}_{0}+2x_{0}\bar{u}_{1}+2\bar{u}_{0}\bar{u}_{1} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003pt(2M^{3}+4M^{2}+4M+1)x^{2}_{0}. Since the real root of 2M^{3}+4M^{2}+4M+1=0 is expressed as \displaystyle M \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003pt\sqrt[3]{\frac{13}{108}+\sqrt{\frac{13^{2}}{108^{2}}+\frac{2^{3}}{9^{3}}}}+\sqrt[3]{\frac{13}{108}-\sqrt{\frac{13^{2}}{108^{2}}+\frac{2^{3}}{9^{3}}}} \displaystyle\hskip-8.00003pt-\frac{2}{3}=-0.3522, using \bar{u}_{t}=Mx_{t} and M=-0.3522 for any x_{0}\neq 0 , we get \displaystyle J(x_{0},\bar{u}) \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003pt0\cdot x^{2}_{0}=0<J(x_{0},u^{*}). Based on the discussion above, since the cost functional is nonnegative, \bar{u} is open-loop optimal for the initial value x_{0} , but u^{*} is not. The above example shows that the generalized Riccati equation ( I ) is helpless in the open-loop solvability of certain LQ problems. Does it inspire that when the LQ problem is merely open-loop solvable, is it possible to get a linear feedback form of state for an open-loop optimal control? Motivated by it, combined with the study of open-loop and closed-loop solvability of Itô differential system in [ 14 ] and [ 15 ] , the necessity and sufficient condition for the closed-loop solvability of LQ problem is derived for a system with discrete time, which is the fundamental of the following discussion. Moreover, by using the perturbation approach introduced in [ 16 ] and [ 18 ] , some equivalent conditions for the open-loop solvability of the LQ problem are given. Moreover, a weak closed-loop solvability of the LQ problem is introduced to obtain a linear state presentation of the open-loop optimal control. And it proves that the existence of a weak closed-loop solution is equivalent to the open-loop solvability of the LQ problem, whose outcome is an open-loop optimal control. The outline of this paper is arranged as follows. Problem formulation and some preliminaries related to the LQ control problem are introduced in Section II. The perturbation method investigates the equivalent conditions of open-loop solvability in Section III. The relationship between open-loop and weak closed-loop solvability is proposed in Section IV. A numerical example in Section V sheds light on the results established in this paper. Notations: \mathbb{R}^{n} denotes an n -dimensional Euclidean space. A^{\prime} means the transpose of the matrix A . A symmetric matrix A>0 (reps. \geq 0 ) means that it is positive definite (reps. positive semi-definite). Let \mathbb{N} and \tilde{\mathbb{N}} represent the sets of \{0,1,\cdots,N-1\} and \{0,1,\cdots,m\} with m\leq N-2 , respectively. Denote L^{2}(\mathbb{N};\mathbb{R}^{n})=\{\phi:\mathbb{N}\rightarrow\mathbb{R}^{n}\Big|\sum\limits^{N-1}_{t=0}|\phi_{t}|^{2}<\infty\} . Let M^{{\dagger}} be Moore-Penrose inverse of matrix M satisfying MM^{{\dagger}}M=M , M^{{\dagger}}MM^{{\dagger}}=M^{{\dagger}} , (MM^{{\dagger}})^{\prime}=MM^{{\dagger}} and (M^{{\dagger}}M)^{\prime}=M^{{\dagger}}M . Rang(A)=\{Ax|x\in\mathbb{R}^{n}\} with n being the dimension of x is the range of A . II Problem Formulation Consider the discrete-time system: \displaystyle x_{t+1} \displaystyle\hskip-8.00003pt= \displaystyle\hskip-8.00003ptA_{t}x_{t}+B_{t}u_{t}, (5) where x_{t}\in\mathbb{R}^{n} is the state with the initial value x_{0} , u_{t}\in\mathbb{R}^{m} is

Read the complete text