Model-based Bootstrap of Controlled Markov Chains
Ziwei Su, Imon Banerjee, Diego Klabjan · 2026
Read Model-based Bootstrap of Controlled Markov Chains by Ziwei Su, Imon Banerjee, Diego Klabjan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12410.txt.
Description
Su, Banerjee and Klabjan propose a bootstrap method for transition kernels of finite controlled Markov chains. The setting covers nonstationary or history-dependent control policies. It arises in offline learning where the data policy is unknown. They prove distributional consistency in single long-chain and episodic regimes. The proof uses a new bootstrap law of large numbers for visit counts. It also uses a martingale central limit theorem for increments. The delta method extends this to policy evaluation and optimal policy recovery, giving valid confidence intervals.
Catalog record
| CREATOR | Ziwei Su, Imon Banerjee, Diego Klabjan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 129747 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12410