Seierstad Sufficient Conditions for Stochastic Optimal Control Problems with Infinite Horizon
Anton O. Belyakov, Yuri M. Kabanov, Ivan A. Terekhov, Maxim M. Savinov · 2025
Read Seierstad Sufficient Conditions for Stochastic Optimal Control Problems with Infinite Horizon by Anton O. Belyakov, Yuri M. Kabanov, Ivan A. Terekhov, Maxim M. Savinov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12728.txt.
Introduction
Anton Belyakov and co-authors state conditions in a 2025 note under which a long-run random control plan can be judged the best choice. The setting involves guiding a system forever while chance keeps acting on it. Their tests build on dual variables described by backward equations that arise in the classic maximum principle.
Catalog record
| CREATOR | Anton O. Belyakov, Yuri M. Kabanov, Ivan A. Terekhov, Maxim M. Savinov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-12728
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Seierstad Sufficient Conditions for Stochastic Optimal Control Problems with Infinite Horizon Anton O. Belyakov, Yuri M. Kabanov, Ivan A. Terekhov, Maxim M. Savinov Abstract In this note we consider a problem of stochastic optimal control with the infinite-time horizon. We present analogues of the Seierstad sufficient conditions of overtaking optimality based on the dual variables stochastic described by BSDEs appeared in the Bismut–Pontryagin maximum principle. MSC 93E20 49K45 1 Introduction Infinite-horizon optimal control problems play an important role in the modern economic theory. A typical examples are optimal investment problems and optimal consumptions of households. In both cases stochastic parameters are natural, see, e.g., Romer . Historically, the main tools for stochastic optimal control problems were HJB equations — in a striking contrast with the deterministic optimal control problems, where many results were obtained using the Pontryagin maximum principle. Necessary optimality conditions in the form of the stochastic maximum principle are discussed already for several decades. They appeared via convex duality in the seminal paper by Bismut Bismut1973 , where backward stochastic differential equations (BSDEs) with linear driver were introduced to describe the dynamics of dual variables. Bismut applied successfully his technique to the stochastic linear-quadratic regulator and the Merton consumption-investments problems in Bismut1976a ; Bismut1976 . By this reason we believe that his contribution merits to be called the Bismut–Pontryagin stochastic maximum principle. The relation between BSDEs and the predictable representation theorem for the Wiener functionals was understood in the rarely available paper Kabanov1978 , where stochastic maximum principle in the form of necessary and sufficient condition was established for a model with linear dynamics of the phase variable. Arkin and Saksonov discussed in ArkinSaksonov1978 some important features of the stochastic maximum principle in the non-convex case. Nowadays the mentioned papers have only a historical interest. The modern theory of non-linear BSDEs was developed in much deeper works by Pardoux and Peng, PP1990 ; P1990 . The literature in this field and its applications is still exploding, see, e.g., a more recent book CZ on stochastic contract theory or OksendalSulem on stochastic control of jump diffusions. In this paper we suggest a stochastic analogue of the Seierstad theorem, Seierstad1977 ; Seierstad1978 ; Belyakov2020 , on sufficient conditions for overtaking optimality. As in the Mangasarian conditions, it is assumed that the Hamilton–Pontryagin function is concave in the state and control variables. The obtained sufficient conditions are written using solutions of the BSDEs depending on the running time horizon. We also prove that in the case of a linear Hamilton–Pontryagin function the Seierstad conditions become not only sufficient but also necessary for the overtaking optimality of the control. We provide examples, where the optimality conditions can be checked. In particular, in Example 2 we consider a stochastic version of the Nerlove–Arrow dynamics, see NA ; Marinelli and present a result on the overtaking optimality for the optimal investment problem with quadratic investment adaptation costs and stochastic marginal productivity of capital, cf. Romer . 2 Setting and main result Let (\Omega,{\mathcal{F}},{\bf F}=({\mathcal{F}}_{t})_{t\geq 0},{\bf P}) be a filtered probability space satisfy the usual condition. Assume that {\bf F} is generated by a Wiener process {\bf W}=(W_{t})_{t\geq 0} . Let L^{22}_{T}:=L^{2}(\Omega\times[0,T],{\mathcal{F}}\otimes{\mathcal{B}}[0,T],dPdt) . Fix a non-empty subset U of {\mathbb{R}} . The control process {\bf u}=(u_{t})_{t\geq 0} in our setting is just a U -valued predictable process. We say that the control {\bf u} is admissible if (u_{t})_{t\in[0,T]}\in L^{22}_{T} for each T . We denote {\mathcal{U}} the set of admissible controls. The controlled dynamics corresponding to {\bf u} is given by the scalar process \boldsymbol{x}=\boldsymbol{x}^{\boldsymbol{u}}=(x_{t}) with a fixed initial value x_{0} solving the stochastic differential equation dx_{t}=f(t,u_{t},x_{t})\,dt+\sigma(t,u_{t},x_{t})\,dW_{t}. To guarantee the existence and uniqueness of strong solution we assume several rather standard hypotheses. The drift coefficient f=f(\omega,t,u,x) is continuously differentiable in (u,x) for each (\omega,t) and predictable in (\omega,t) for each (u,x)\in U\times{\mathbb{R}} , that is the function (\omega,t)\mapsto f(\omega,t,u,x) is predictable. The derivative in x is uniformly bounded: |f_{x}(\omega,t,u,x)|\leq{\rm const} . Moreover, the linear growth condition is fulfilled: |f(\omega,t,u,x)|\leq C_{t}(1+|u|+|x|),\qquad\forall\,(\omega,t,u)\in\Omega\times{\mathbb{R}}_{+}\times U\times{\mathbb{R}}, where {\bf C}=(C_{t}) is a finite increasing function The diffusion coefficient \sigma=\sigma(\omega,t,u,x) satisfy the same conditions as the drift coefficient. It follows that {\bf E}\int_{0}^{T}\sigma^{2}(t,u_{t},x_{t})dt<\infty. For a given finite time horizon T , we consider the value functional (to maximize) J_{T}(\boldsymbol{u}):={\bf E}\int_{0}^{T}g(t,u_{t},x_{t})dt, (1) where the process x={\bf x}^{\bf u} . We assume that the function g=g(\omega,t,u,x) is continuously differentiable in (u,x) for each (\omega,t) and predictable in (\omega,t) for each (u,x)\in U\times{\mathbb{R}} and, for simplicity, satisfies the linear growth condition as f and \sigma . Moreover, we assume that \big|g_{x}(\omega,t,u,x)\big|\leq\tilde{c}_{T}(1+|u|+|x|),\qquad\forall\,(\omega,t,u)\in\Omega\times[0,t]\times U\times{\mathbb{R}}, where \tilde{\bf c}=(\tilde{c}_{t}) is a finite increasing function. This condition ensures that for any admissible control {\bf E}\int_{0}^{T}\left|g_{x}(t,u_{t},x_{t})\right|^{2}dt<\infty. (2) These assumptions imply that for any admissible control J_{T}(\boldsymbol{u})<\infty . However, the linear growth assumption of g is too restrictive (it excludes quadratic cost). One can work without restricting controls to those with well-defined value functional. Definition 1 The control \hat{\bf u}\in{\mathcal{U}} is called weakly overtaking optimal, WOO, if for any {\bf u}\in{\mathcal{U}} \liminf_{T\to\infty}\left(J_{T}(\boldsymbol{u})-J_{T}(\hat{\boldsymbol{u}})\right)\leq 0. Definition 2 The control \hat{\bf u}\in{\mathcal{U}} is called overtaking optimal, OO, if for any {\bf u}\in{\mathcal{U}} \limsup_{T\to\infty}\left(J_{T}(\boldsymbol{u})-J_{T}(\hat{\boldsymbol{u}})\right)\leq 0. Let us define the Hamilton–Pontryagin function \mathcal{H}(t,u,x,p,h)=p\,f(t,u,x)+h\,\sigma(t,u,x)+g(t,u,x). (3) We skip, as usual, \omega in this formula. Let \hat{\bf u} be an admissible control which we want to test for optimality and let \hat{\bf x} be the corresponding process. We consider on the interval [0,T] the BSDE \displaystyle dp_{t}=-\mathcal{H}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t},h_{t})dt+h_{t}dW_{t},\quad p_{T}=0. (4) The driver of this equation, \mathcal{H}_{x}(t,\hat{u}_{t}(\omega),\hat{x}_{t}(\omega),p,h):=pf_{x}(t,\hat{u}_{t},\hat{x}_{t})+h\sigma_{x}(t,\hat{u}_{t},\hat{x}_{t})+g_{x}(t,\hat{u}_{t},\hat{x}_{t}), is such that {\bf E}\int_{0}^{T}|\mathcal{H}_{x}(t,\hat{u}_{t},\hat{x}_{t},0,0)|^{2}dt={\bf E}\int_{0}^{T}|g_{x}(t,\hat{u}_{t},\hat{x}_{t})|^{2}dt<\infty in virtue of ( 2 ). Also, \displaystyle|\mathcal{H}_{x}(t,\hat{u}_{t}(\omega),\hat{x}_{t}(\omega),p,h)-\mathcal{H}_{x}(t,\hat{u}_{t}(\omega),\hat{x}_{t}(\omega),\tilde{p},\tilde{h})| \displaystyle\leq|p-\tilde{p}|f_{x}(t,\hat{u}_{t},\hat{x}_{t})+|h-\tilde{h}|\sigma_{x}(t,\hat{u}_{t},\hat{x}_{t}) \displaystyle\leq C(|p-\tilde{p}|+|h-\tilde{h}|). These conditions ensure the existence and uniqueness of solution ({\bf p}^{T},{\bf h}^{T}) of BSDE ( 4 ) in the space X_{T}\times L^{22}_{T} , where X_{T} is the space of adapted continuous processes {\bf p} such that {\bf E}\sup_{s\leq T}|p_{s}|^{2}<\infty , see CZ . Our hypotheses guarantee the existence and uniqueness of solution (\hat{\bf p},\hat{\bf h})=(\hat{\bf p}^{T},\hat{\bf h}^{T}) in the Banach space B_{T}\times L^{22}_{T} , where B_{T} is the space of continuous adapted processes p such that {\bf E}\sup_{t\leq T}|p_{t}|^{2}<\infty . Assumption 2.1 There is a convex set \tilde{U}\supseteq U and T_{1}>0 such that for all t\geq T_{1} the function (u,x)\mapsto\mathcal{H}(t,u,x,p,h) is concave on \tilde{U}\times{\mathbb{R}} . Theorem 2.2 (Optimality: sufficient conditions.) Let Assumption 2.1 be fulfilled. Then the admissible control (\hat{\boldsymbol{u}}) is WOO, if \liminf_{T\to\infty}{\bf E}\int_{0}^{T}{\mathcal{H}}_{u}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\left(u_{t}-\hat{u}_{t}\right)dt\leq 0, (5) and OO, if \limsup_{T\to\infty}{\bf E}\int_{0}^{T}{\mathcal{H}}_{u}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\left(u_{t}-\hat{u}_{t}\right)dt\leq 0, (6) for every admissible control \boldsymbol{u} . Theorem 2.3 (Optimality: necessary and sufficient conditions.) If the function \mathcal{H}(t,u,x,p) is linear in (u,x) on \tilde{U}\times X , then the conditions ( 5 ) and ( 6 ) in Theorem 2.2 are also necessary. 3 Proofs Proof of Theorem 2.2 . Put {\bf y}:={\bf x}-\hat{\bf x} . Since dy_{t}=\left(f(t,u_{t},x_{t})-f(t,\hat{x}_{t},\hat{u}_{t})\right)dt+\left(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\right)dW_{t}, it follows by the Ito product formula \displaystyle d(p_{t}^{T}y_{t}) \displaystyle= \displaystyle p_{t}^{T}\,dy_{t}+y_{t}\,d{p}_{t}^{T}+h^{T}_{t}\big(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)dt \displaystyle= \displaystyle p_{t}^{T}\big(f(t,u_{t},x_{t})-f(t,\hat{u}_{t},\hat{x}_{t})\big)\,dt+h^{T}_{t}(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)dt \displaystyle-{\mathcal{H}}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})y_{t}\,dt+h^{T}_{t}y_{t}\,dW_{t} \displaystyle+p_{t}^{T}\big(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)\,dW_{t}. Using its integral form we have: \displaystyle p_{T}^{T}y_{T} \displaystyle= \displaystyle p_{0}^{T}y_{0}+\int_{0}^{T}p_{t}^{T}\,dy_{t}+y_{t}\,d{p}_{t}^{T}+\int_{0}^{T}h^{T}_{t}\big(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)\,dt \displaystyle= \displaystyle\int_{0}^{T}p_{t}^{T}\left[\big(f(t,u_{t},x_{t})-f(t,\hat{u}_{t},\hat{x}_{t})\big)+h^{T}_{t}(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)\right]dt \displaystyle-\int_{0}^{T}{\mathcal{H}}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\,y_{t}\,dt+\int_{0}^{T}h^{T}_{t}y_{t}\,dW_{t} \displaystyle+\int_{0}^{T}p_{t}^{T}\left(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\right)dW_{t}. The stochastic integrals above have zero expectations due to our assumptions. Since p^{T}_{T}=0 and y_{0}=0 we get that \displaystyle{\bf E}\int_{0}^{T}p_{t}^{T}\left[\big(f(t,u_{t},x_{t})-f(t,\hat{u}_{t},\hat{x}_{t})\big)+h^{T}_{t}(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)\right]dt \displaystyle={\bf E}\int_{0}^{T}{\mathcal{H}}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\,y_{t}\,dt. With this we complete the proof easily. Indeed, \displaystyle\Delta J_{T}:=J_{T}(\hat{\boldsymbol{u}})-J_{T}(\boldsymbol{u})={\bf E}\int\limits_{0}^{T}\big(g(t,\hat{u}_{t},\hat{x}_{t})-g(t,u_{t},x_{t})\big)\,dt \displaystyle={\bf E}\int_{0}^{T}\big({\mathcal{H}}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})-{\mathcal{H}}(t,u_{t},x_{t},p_{t}^{T},h^{T}_{t})\big)\,dt \displaystyle\quad-{\bf E}\int_{0}^{T}p_{t}^{T}\left[\big(f(t,u_{t},x_{t})-f(t,\hat{x}_{t},\hat{u}_{t})\big)+h^{T}_{t}(\sigma(t,u_{t},x_{t})-\sigma(t,\hat{u}_{t},\hat{x}_{t})\big)\right]dt \displaystyle={\bf E}\int_{0}^{T}\big({\mathcal{H}}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})-{\mathcal{H}}(t,u_{t},x_{t},p_{t}^{T},h^{T}_{t})\big)\,dt \displaystyle\quad-{\bf E}\int_{0}^{T}{\mathcal{H}}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\,y_{t}\,dt. In virtue of Assumption 2.1 on concavity \displaystyle{\mathcal{H}}(t,u_{t},x_{t},p_{t}^{T},h^{T}_{t})-{\mathcal{H}}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t}) \displaystyle\leq{\mathcal{H}}_{u}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\big(u_{t}-\hat{u}_{t}\big)+{\mathcal{H}}_{x}(t,\hat{u}_{t},\hat{x}_{t},p_{t}^{T},h^{T}_{t})\big(x_{t}-\hat{x}_{t}\big). (7) Using this inequality we obtain