Model-based Bootstrap of Controlled Markov Chains

Ziwei Su, Imon Banerjee, Diego Klabjan · 2026

Read Model-based Bootstrap of Controlled Markov Chains by Ziwei Su, Imon Banerjee, Diego Klabjan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12410.txt.

Description

Su, Banerjee and Klabjan propose a bootstrap method for transition kernels of finite controlled Markov chains. The setting covers nonstationary or history-dependent control policies. It arises in offline learning where the data policy is unknown. They prove distributional consistency in single long-chain and episodic regimes. The proof uses a new bootstrap law of large numbers for visit counts. It also uses a martingale central limit theorem for increments. The delta method extends this to policy evaluation and optimal policy recovery, giving valid confidence intervals.

Catalog record

CREATORZiwei Su, Imon Banerjee, Diego Klabjan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129747
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.12410

Browse the full library · About the Tower of Babel