Long-Term Average Impulse Control with Mean Field Interactions
K.L. Helmes, R.H. Stockbridge, C. Zhu · 2025
Read Long-Term Average Impulse Control with Mean Field Interactions by K.L. Helmes, R.H. Stockbridge, C. Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11345.txt.
Introduction
Helmes, Stockbridge, and Zhu solve long term impulse control problems where many agents interact through a shared market, in work from twenty twenty-five. Each agent acts at discrete moments and earns a running reward plus income from each action. The price per action depends on the group's supply rate.
Two settings are solved in full. In the competitive setting, an equilibrium strategy is found and described under mild terms. In the cooperative mean field control setting, agents join to maximize shared long term profit. Examples cover a stochastic logistic growth model and a population growth model. Uses span renewable resource care and portfolio management.
Catalog record
| CREATOR | K.L. Helmes, R.H. Stockbridge, C. Zhu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11345
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Long-Term Average Impulse Control with Mean Field Interactions Thanks: This research was supported by the Simons Foundation under grant number 8035009. K.L. Helmes, R.H. Stockbridge, C. Zhu Abstract This paper analyzes and explicitly solves a class of long-term average impulse control problems with a specific mean-field interaction. The underlying process is a general one-dimensional diffusion with appropriate boundary behavior. The model is motivated by applications such as the optimal long-term management of renewable resources and financial portfolio management. Each individual agent seeks to maximize her long-term average reward, which consists of a running reward and income from discrete impulses, where the unit intervention price depends on the market through a stationary supply rate, the specific mean field variable to be considered. In a competitive market setting, we establish the existence of and explicitly characterize an equilibrium strategy within a large class of policies under mild conditions. Additionally, we formulate and solve the mean field control problem, in which agents cooperate with each other, aiming to realize a common maximal long-term average profit. To illustrate the theoretical results, we examine a stochastic logistic growth model and a population growth model in a stochastic environment with impulse control. Keywords: Mean field game; mean field control; impulse control; long-term average reward; equilibrium strategy; renewal theory; stochastic logistic growth models. AMS 2020 subject classifications: 91A16, 91A15, 93E20, 60H30, 60J60, 91G80 1 Introduction This paper considers and explicitly solves a long-term average stochastic impulse control problem with a particular type of mean-field interaction. Our motivation stems from two sources. The first is the applications in natural resource management, specifically in the context of optimal and sustainable harvesting strategies. The second is mathematical in nature. It concerns (a) the important but subtle interplay between two revenue streams, the incomes from a running reward and from impulse decisions, and (b) exploring a direct approach, using renewal theory and the renewal reward theorem, to analyze such impulse control problems with a special mean field interaction that will be described in detail momentarily. This approach differs from the general and well established principle: “Set up the proper HJB/QVI of the model, couple it with the corresponding Fokker-Planck equation, and apply a fixed point argument.” Let us now introduce the problem. In the absence of controls, the dynamics of a one-dimensional state process – which may describe the evolution of some renewable resource – is modeled by a one-dimensional diffusion process on an interval {\mathcal{I}}\subset\mathbb{R} dX_{0}(t)=\mu(X_{0}(t))dt+\sigma(X_{0}(t))dW(t),\ \ X_{0}(0)=x_{0}, (1.1) where x_{0}\in{\mathcal{I}} is an arbitrary but fixed point throughout the paper, W is a one-dimensional standard Brownian motion, and the drift and diffusion are given by the functions \mu and \sigma , respectively. The diffusion process is assumed to have certain boundary behavior; see Condition 2.1 for details. Furthermore, an individual agent wants to specify when and by how much the state of the process should be reduced to achieve economic benefits. Her strategy is modeled by an impulse control R:=\{(\tau_{k},Y_{k}),k=1,2,\dots\} such that for each k\in\mathbb{N} , \tau_{k} is the time of the k th intervention and Y_{k} is the size of the intervention. The resulting controlled process X^{R} satisfies \displaystyle X^{R}(t) \displaystyle=x_{0}+\int_{0}^{t}\mu(X^{R}(s))ds+\int_{0}^{t}\sigma(X^{R}(s))dW(s)-\sum_{k=1}^{\infty}I_{\{\tau_{k}\leq t\}}Y_{k},\quad t\geq 0. (1.2) A fundamental quantity associated with each policy R is the long-term average supply rate of product to the market given by \displaystyle\kappa^{R}: \displaystyle=\limsup_{t\to\infty}\frac{1}{t}\mathbb{E}\Bigg[\sum_{k=1}^{\infty}I_{\{\tau_{k}\leq t\}}Y_{k}\Bigg]=\limsup_{t\to\infty}\frac{1}{t}\mathbb{E}\Bigg[\sum_{k=1}^{\infty}I_{\{\tau_{k}\leq t\}}(X^{R}(\tau_{k}-)-X^{R}(\tau_{k}))\Bigg]. (1.3) Regarding the market structure, we assume that the market’s supply side comprises a continuum of agents, each with the same state dynamics (in the absence of control) and reward structure as the individual agent under consideration. An individual agent’s reward depends not only on her own impulse strategy R , but also crucially on the market’s long-term average supply rate \kappa^{Q} , which results when all other agents adopt policy Q . The supply rate \kappa^{Q} is the key mean-field interaction that determines the market price of the product of interest through a continuous function \varphi . Given a positive fixed cost K for each intervention, a running reward function c , a product supply rate to the market z:=\kappa^{Q} , and a price function \varphi , the reward functional for an individual agent who adopts policy R is her expected long-term average profit: \displaystyle J(R;z): \displaystyle=\liminf_{t\to\infty}t^{-1}\mathbb{E}\bigg[\int_{0}^{t}c(X^{R}(s))ds (1.4) \displaystyle+\sum_{k=1}^{\infty}I_{\{\tau_{k}\leq t\}}(\varphi(z)(X^{R}(\tau_{k}-)-X^{R}(\tau_{k}))-K)\bigg]. The presence of the positive fixed cost K for each intervention implies that \lim_{k\to\infty}\tau_{k}=\infty a.s.; otherwise J(R;z)=-\infty , so R will not be optimal. The expected long-term average revenue has two components consisting of a running reward and the reward obtained from the impulse control. The function c quantifies the running rewards based on the values of the controlled process. In the context of harvesting problems, the function c can represent the utility derived from maintaining desirable state values X^{R}(s) at time s , as well as the state’s contribution to the overall ecosystem’s stability. For example, the function c can be used to model a subsidy or a stream of carbon credits for managing large tracts of forest. The reward from each control action is the net price \varphi(z) times the difference in states at the time of intervention minus the fixed cost K . In general, the impulse cost for production-type problems has a fixed component and a variable cost. For the models considered in this paper, the variable cost is proportional to the size of the intervention and the net price \varphi(z) subsumes this proportional cost. The fixed cost for an intervention in ( 1.4 ) makes the problem one of impulse control. The optimal policy thus involves discrete interventions rather than continuous adjustments, ensuring effective product management while maximizing the overall profit rate. A fundamental assumption on the model is that each price \varphi(z) in the range of supply rates z is large enough so that some active policy yields a better long-term average reward than the “do-nothing” policy that never intervenes. Such prices are called feasible. The collection of feasible prices will be denoted by \mathfrak{P} , for which a functional representation will be given in ( 3.3 ). Due to the presence of the nonnegative running reward rate c , the interplay between c and the production rate \mu is one of the essential and important features of the model and requires careful analysis. Although the case of a negative function c is also relevant in applications such as inventory control and industrial animal husbandry, a negative running or holding cost term in fact simplifies both the analysis and the characterization of optimal controls near the right boundary of the state space. Specifically, a negative c prizes interventions that keep the controlled process away from the right boundary, thereby avoiding challenges associated with boundary behavior of the underlying diffusion. By contrast, a nonnegative running reward function c may encourage the process to approach the right boundary, necessitating a more careful examination of several elementary results. Two problems will be investigated in this paper. First, we consider a competitive market where the agents compete with each other. Our goal is to establish the existence of an equilibrium strategy under the long-term average criterion in a large class of admissible policies. In other words, we wish to determine whether there exists an admissible strategy Q so that for all admissible R in the class