Selection principles for quasi-stationary distributions and reinforcement processes

Michel Benaïm, Tresnia Berah, Pierre Germain, [0.3cm] Institut de Mathématiques, Université de Neuchâtel, Switzerland. E-mail: [email protected] · 2026

Read Selection principles for quasi-stationary distributions and reinforcement processes by Michel Benaïm, Tresnia Berah, Pierre Germain, [0.3cm] Institut de Mathématiques, Université de Neuchâtel, Switzerland. E-mail: [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25857.txt.

Introduction

Michel Benaïm, Tresnia Berah, and Pierre Germain study killed Markov chains, random processes that wander among states until they are absorbed and stop. A quasi-stationary distribution describes how such a chain behaves in the long run when attention is restricted to runs that have survived. The team couples the chain to a reinforced process that, each time it dies, restarts at a spot sampled from its own weighted history.

When the chain can reach every state from every other, the occupation measure settles on the single quasi-stationary distribution. Benaïm and colleagues handle the harder case where that reachability fails and several such distributions exist. Selection then hinges on the weights: if the inverse cumulative weights sum to infinity, the distribution tied to the largest Perron value wins almost surely. When the sum is finite, every candidate keeps a positive chance of being selected.

Catalog record

CREATORMichel Benaïm, Tresnia Berah, Pierre Germain, [0.3cm] Institut de Mathématiques, Université de Neuchâtel, Switzerland. E-mail: [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25857

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Selection principles for quasi-stationary distributions and reinforcement processes Michel Benaïm, Tresnia Berah, Pierre Germain, [0.3cm] Institut de Mathématiques, Université de Neuchâtel, Switzerland. E-mail: [email protected] Abstract Let P P be a sub-Markov matrix on a finite set S S , representing the transition probabilities of a Markov chain on S S absorbed at a cemetery point ∂ ∉ S \partial\notin S . We consider a reinforced process ( X n , μ n ) (X_{n},\mu_{n}) defined as follows: ( X n ) (X_{n}) behaves like a chain with kernel P P until it dies, and when it dies at time n n , it is instantaneously “resurrected” at a point sampled according to its weighted past occupation measure μ n = 1 W n ​ ( w 0 ​ μ 0 + ∑ k = 1 n w k ​ δ X k ) , W n = ∑ k = 0 n w k , \mu_{n}=\frac{1}{W_{n}}\left(w_{0}\mu_{0}+\sum_{k=1}^{n}w_{k}\delta_{X_{k}}\right),\qquad W_{n}=\sum_{k=0}^{n}w_{k}, where the positive weights w k w_{k} satisfy certain technical assumptions, a typical example being given by w k = k q w_{k}=k^{q} , with q ≥ − 1 q\geq-1 . When P P is irreducible, the behaviour of ( μ n ) (\mu_{n}) is well understood [ 1 ] , [ 2 ] : it converges almost surely toward the unique quasi-stationary distribution (QSD) of P P . The purpose of this paper is to investigate the general situation where P P is not irreducible. Under generic assumptions on P P , there are finitely many QSDs. We prove that the asymptotic selection depends on the summability of the inverse cumulative weights 1 / W n 1/W_{n} . If ∑ n ≥ 0 1 W n = ∞ , \sum_{n\geq 0}\frac{1}{W_{n}}=\infty, then ( μ n ) (\mu_{n}) almost surely converges toward the QSD associated with the largest Perron value. If instead ∑ n ≥ 0 1 W n < ∞ , \sum_{n\geq 0}\frac{1}{W_{n}}<\infty, then each QSD is selected with positive probability. In particular, for polynomial weights w 0 = 1 w_{0}=1 and w k = k q w_{k}=k^{q} , k ≥ 1 k\geq 1 , this gives almost sure selection of the QSD with largest Perron value for − 1 ≤ q ≤ 0 -1\leq q\leq 0 , whereas each quasi-stationary distribution is selected with positive probability for q > 0 q>0 . 1 Introduction Let S be a finite state space and let P=(P(x,y))_{x,y\in S} be a sub-Markovian kernel on S , namely \displaystyle P(x,y)\geq 0,\qquad 0<\sum_{y\in S}P(x,y)\leq 1,\qquad x,y\in S. (1) Such a kernel naturally defines a killed Markov chain by adjoining a cemetery state \partial\notin S and considering the Markov kernel \widehat{P} on S\cup\{\partial\} given by \widehat{P}(x,y)=P(x,y),\qquad\widehat{P}(x,\partial)=1-\sum_{y\in S}P(x,y),\qquad x,y\in S, together with \widehat{P}(\partial,\partial)=1,\qquad\widehat{P}(\partial,y)=0,\qquad y\in S. We denote by (\widehat{X}_{n})_{n\geq 0} the Markov chain with kernel \widehat{P} . Its law and expectation are denoted by \widehat{\mathbb{P}}_{x} and \widehat{\mathbb{E}}_{x} when \widehat{X}_{0}=x , and by \widehat{\mathbb{P}}_{\nu} and \widehat{\mathbb{E}}_{\nu} when \widehat{X}_{0}\sim\nu . A central object in the study of killed Markov processes is the notion of quasi-stationary distribution. We denote by \mathcal{P}(S) the set of probability measures on S . A probability measure \nu\in\mathcal{P}(S) is called a quasi-stationary distribution (QSD) if, for every n\geq 0 , \nu(\cdot)=\widehat{\mathbb{P}}_{\nu}\left(\widehat{X}_{n}\in\cdot\,\middle|\,n<\tau_{\partial}\right), where \tau_{\partial}:=\inf\{n\geq 0:\widehat{X}_{n}=\partial\} denotes the absorption time. QSDs play the role of invariant measures for non-conservative dynamics. Under Hypothesis 1 below, absorption occurs almost surely, but the law conditioned on survival may exhibit a non-trivial equilibrium behaviour. The theory of QSDs for irreducible sub-Markovian kernels is by now well understood. In particular, irreducibility implies uniqueness of a QSD. In contrast, the non-irreducible setting is substantially richer and more delicate. Multiple QSDs may coexist, reflecting the possible presence of several communicating classes. In the present work, we focus precisely on this reducible framework. Under an additional structural assumption introduced later, the set of QSDs is finite. A natural and challenging question is then to design stochastic algorithms capable of sampling or selecting these different QSDs. The purpose of this article is to study such a sampling procedure based on reinforced stochastic dynamics. Our approach fits within the general framework of stochastic approximation and self-interacting processes. In order to define the process we wish to study, let us first recall the definition of the resurrected Markov kernel given a fixed measure \mu . For x\in S , let q(x):=1-\sum_{z\in S}P(x,z) denote the killing probability at state x . For a probability measure \mu on S , we define the reinforced transition kernel by K_{\mu}(x,y):=P(x,y)+q(x)\mu(y),\qquad x,y\in S. (2) The reinforced process is constructed recursively as follows. Its definition only depends on the state space S , the given sub-Markovian kernel P , the initial condition (X_{0},\mu_{0}) and a deterministic step-size sequence (\gamma_{n})_{n\geq 1}\in(0,1)^{\mathbb{N}^{*}} . We start from an initial condition (X_{0},\mu_{0}) , possibly random, such that (X_{0},\mu_{0})\in S\times\mathcal{P}(S)\qquad\text{almost surely}. The reinforced process is defined recursively with respect to the filtration \mathcal{F}_{n}=\sigma(\mu_{0},X_{0},\ldots,X_{n}), through the iteration \left\{\begin{aligned} \mathbb{P}(X_{n+1}=y\,|\,\mathcal{F}_{n})&=K_{\mu_{n}}(X_{n},y),\qquad y\in S,\\ \mu_{n+1}&=(1-\gamma_{n+1})\mu_{n}+\gamma_{n+1}\delta_{X_{n+1}}.\end{aligned}\right. (3) Throughout the article, \mathbb{P} and \mathbb{E} denote probability and expectation for the reinforced process (X_{n},\mu_{n}) defined by ( 3 ) and the initial condition (X_{0},\mu_{0}) . The notation \widehat{\mathbb{P}} and \widehat{\mathbb{E}} is reserved for the killed chain with kernel \widehat{P} . In the classical stochastic approximation setting where \gamma_{n}=\frac{1}{n+1} and where \mu_{0}=\delta_{X_{0}} , \mu_{n} coincides with the empirical occupation measure of the trajectory up to time n \mu_{n}=\frac{1}{n+1}\sum_{k=0}^{n}\delta_{X_{k}}. For a general sequence (\gamma_{n}) , setting r_{0}:=1,\quad r_{n}:=\prod_{k=1}^{n}(1-\gamma_{k}),\qquad\text{ and }\qquad w_{0}:=1,\quad w_{k}:=\frac{\gamma_{k}}{r_{k}}, the measure \mu_{n} should instead be interpreted as a weighted occupation measure \mu_{n}=\frac{1}{W_{n}}\left(w_{0}\mu_{0}+\sum_{k=1}^{n}w_{k}\delta_{X_{k}}\right),\qquad\ W_{n}:=\sum_{k=0}^{n}w_{k}. This decomposition will be formalised in the next section. The heuristic interpretation of the reinforced process is the following. Between two killing events, the particle evolves according to the original sub-Markovian dynamics P . Whenever the process is killed, instead of being sent to the cemetery state, it is instantaneously resurrected at a position sampled according to its past occupation measure \mu_{n} (uniformly over the past trajectory in the classical case \gamma_{n}=\frac{1}{n+1} ). Consequently, states that have already been frequently visited are more likely to be selected again after future killing events. The process therefore progressively favours regions of the state space that it has already explored, creating a reinforcement mechanism through its own history. This self-reinforcement phenomenon explains the terminology of reinforced process. Reinforced stochastic approximation algorithms for the simulation of quasi-stationary distributions are now well understood in irreducible settings. In the finite state space case, the reinforced process was introduced in [ 7 ] , and was shown to converge toward the unique QSD. An averaging variant which was shown to improve the convergence rate of the algorithm was introduced in [ 10 ] . Related reinforced mechanisms linked with QSD appeared earlier in the work [ 1 ] . The stochastic approximation approach was later generalised to compact state spaces in [ 6 ] , while related diffusion models in bounded domains were investigated in [ 5 , 15 ] . A common feature of these works is that the underlying deterministic dynamics admits a unique globally attractive equilibrium, corresponding to the unique QSD of the killed process. The reducible setting is substantially more delicate. In this case, several QSDs may coexist and the asymptotic behaviour of the reinforced dynamics becomes

Read the complete text