A Graphical Approach to State Variable Selection in Off-policy Learning

Joakim Blach Andersen, Qingyuan Zhao · 2025

Read A Graphical Approach to State Variable Selection in Off-policy Learning by Joakim Blach Andersen, Qingyuan Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-00854.txt.

Description

Joakim Blach Andersen and Qingyuan Zhao use graphs to decide which state variables matter when learning policies from past data. Their 2025 paper addresses off-policy learning, where data were not collected at random. Two fields study this problem in unlike ways: medicine, with short treatment chains, and game-based learning, with tight closed systems.

Using causal graph theory, the authors give identification rules that cover both settings in one frame. They show how their results relate to hidden assumptions in each field's literature, and they clear up several common misconceptions. A simulation of dynamic pricing in container shipping shows the stakes. When the graph rules are broken, the learned policy falls short of the best one.

Catalog record

CREATORJoakim Blach Andersen, Qingyuan Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE726 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-00854

Browse the full library · About the Tower of Babel