Is Pure Exploitation Sufficient in Exogenous MDPs with Linear Function Approximation?

Hao Liang, Jiayu Cheng, Sean R. Sinclair, Yali Du · 2026

Read Is Pure Exploitation Sufficient in Exogenous MDPs with Linear Function Approximation? by Hao Liang, Jiayu Cheng, Sean R. Sinclair, Yali Du (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20694.txt.

Introduction

Hao Liang, Jiayu Cheng, Sean Sinclair, and Yali Du ask a sharp question about decision making under uncertainty. Their 2026 paper studies exogenous Markov decision processes, or Exo-MDPs. In these models, all randomness comes from outside inputs. Those inputs evolve on their own, untouched by the learner's actions. Demand, arrivals, and prices are classic examples.

Such models are common in operations work: inventory control, energy storage, and resource allocation. The authors study them with linear function approximation, a compact way to represent value. Their question is in the title: is pure exploitation enough? That means always taking the action that looks best now, with no probing of new options. The answer shapes how simple a good policy for these systems can be.

Catalog record

CREATORHao Liang, Jiayu Cheng, Sean R. Sinclair, Yali Du
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE153 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20694

Browse the full library · About the Tower of Babel