Primal-dual programs for the constrained optimal impulse control: discounted model
Alexey Piunovskiy, Yi Zhang · 2025
Read Primal-dual programs for the constrained optimal impulse control: discounted model by Alexey Piunovskiy, Yi Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18387.txt.
Introduction
Alexey Piunovskiy and Yi Zhang study constrained impulse control in this 2025 math paper. Impulse control steers a system with sudden jumps rather than smooth pushes. Costs come both from each impulse and from running the system over time, with a discount on future costs.
The authors set up a relaxed version of the problem and write it as convex programs in measures. They then build the matching dual programs and prove the duality results that link them. As a payoff, they give a general method to build optimal mixed strategies for the original problem. A worked example shows the full procedure in action.
Catalog record
| CREATOR | Alexey Piunovskiy, Yi Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 119 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18387
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Primal-dual programs for the constrained optimal impulse control: discounted model Alexey Piunovskiy, Yi Zhang Abstract Abstract . This paper studies constrained optimal impulse control problems of a deterministic system described by a (semi)flow, where the performance measures are the discounted total costs including both the costs incurred with applying impulses as well as running costs. We formulate the relaxed problem and the associated primal convex programs in measures together with the dual programs, and establish the relevant duality results. As an application, we formulate and justify a general procedure for obtaining optimal (J+1) -mixed strategies for the original impulse control problems. This procedure is illustrated with a solved example. Keywords. Impulse control. Markov decision process. Discounted costs. Optimal strategy. Convex program. Linear program. Duality. AMS 2000 subject classification: Primary 49N25; Secondary 90C05, 90C40. 1 Introduction This paper studies optimal impulse control problems of a deterministic system described by a (semi)flow, where the performance measures are the discounted total costs including both the lump sum cost associated with applying impulses as well as running costs. The decision maker aims at minimizing one performance measure, subject to constraints on the other similar performance measures given by different impulse cost functions and running cost rates. We devote this paper to Professor A. Plakhov on the occasion of his 65th birthday. Let us start with comments on this paper in relation to the remarkable contributions made by Professor A. Plakhov in this area, while postponing the general literature review until a later part of this section. In absence of constraints and discounting, together with Professor A. Plakhov, we investigated this problem in [ 19 ] , where the dynamic programming approach was studied. The first step of investigation in that paper was to reformulate the impulse control problem as a Markov decision process (MDP), leading to the so called optimality equation in integral form, and after that, optimality equation in differential form (close to the so called quasi-variational inequalities) was established, and the connection between these two equations was studied in detail. This work admitted natural extensions. One is for more general system dynamics, and this was carried out together with Professor A. Plakhov in [ 8 ] for optimal impulse control problems of piecewise deterministic processes. Another possibility is to consider problems with (functional) constraints. For handling such problems, the convex analytic approach, also called the linear programming method, is quite convenient. This method is based on reformulating the original problem as a static optimization problem in suitably defined occupation measures. This optimization problem can be relaxed to a convex program in measures after one imposes the constraint given by the characteristic equation satisfied by occupation measures. A fundamental issue in this convex analytic approach was to investigate the equivalence of this convex program and the original impulse control problem, known as the absence of relaxation gap. This was the main result in [ 22 , 23 ] , where we acknowledged the initial input from Professor A. Plakhov. We mention that corresponding to the two types of optimality equations for unconstrained problems, one may deal with two types of occupation measures, leading to two convex program formulations, which we may again call of integral form and of differential form. The first one was investigated in [ 22 ] , and the second one was investigated in [ 23 ] . After the absence of relaxation gap is showed, the next natural step is to formulate the dual program of the convex program, and investigate the absence of duality gap. This was done in [ 24 ] , but for unconstrained optimal impulse control problem with total undiscounted costs. The aim of the present paper is to extend and complement the results in [ 24 ] to problems with constraints and discounted costs, for which we further investigate the convex analytic approach by focusing on the duality issues and formulate procedures for finding optimal strategies in specific forms, the so called (J+1) -mixed strategies with J being the number of constraints. Note that, without further conditions, the relaxed problem of the original impulse control problem is in space of infinite measures, even in presence of discounting. This is in contrast with situations encountered in convex analytic approach for (classic) optimal gradual control problems. The primal convex program is in the occupation measures similar to those considered in [ 22 ] . One may formulate its dual program, and the crude but otherwise useful observation is the absence of the duality gap, which follows from the relevant results in [ 16 ] . The dual functional is in the space of non-negative Lagrange multipliers. It is desirable (and is required in our general procedure to solve constrained optimal impulse problems) to compute the value of the dual functional for each Lagrange multiplier by solving an unconstrained impulse control problem with a new cost function modified with the Lagrange multipliers. To guarantee that these two values coincide, we impose suitable conditions on the cost functions, which also imply that the relaxed problem is equivalent to a (so called second) convex program in finite measures. Having said the above, our main contributions lie in the suitable formulation of the dual program of the second convex program, proving the absence of the duality gap, showing that the dual functional of the relaxed problem can be computed by solving an unconstrained problem using the dynamic programming method, and presenting with justifications the general procedure for computing the optimal (J+1) -mixed optimal strategies for the impulse control problems. We also demonstrate the procedure by solving in closed form an example that can be understood in the context of fluid queueing systems. We note that while the existence of optimal mixed strategies over J+1 deterministic stationary strategies, which was not addressed in the aforementioned works, follows from the recent paper [ 25 ] , here we showed that the mixture could be done over deterministic stationary strategies which are uniformly optimal for the unconstrained problem modulated by optimal Lagrange multipliers. Finally, we briefly review the other relevant and recent literature on impulse control problems. In the absence of constraints, the standard method of investigation of optimal impulse control problems is dynamic programming, leading to the so called quasi-variational inequalities [ 3 , 6 , 8 ] , see e.g., [ 3 , 6 , 8 , 15 , 19 , 20 ] . The Pontryagin maximum principle and similar methods were developed in [ 1 , 17 ] . Another approach that is especially convenient in handling problems with constraints is the convex analytic approach. This method is based on reformulating the original problem as linear or convex programs in the space of suitably defined occupation measures. Such measures and linear programs were introduced and investigated in [ 5 , 12 , 22 , 24 ] in different settings. The convex analytic approach can be also developed for classical (gradual) optimal control problems, see [ 11 ] . A little more details about the cited works are as follows. • The state space was a subset \mathbb{R}^{n} and the dynamical system was defined through an ordinary differential equation in [ 1 , 3 , 11 , 12 , 17 ,