Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol

Pai Liu, Lingfeng Zhao, Shivangi Agarwal, Jinghan Liu, Audrey Huang, Philip Amortila, Nan Jiang · 2025

Read Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol by Pai Liu, Lingfeng Zhao, Shivangi Agarwal, Jinghan Liu, Audrey Huang, Philip Amortila, Nan Jiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08021.txt.

Description

Pai Liu and co-authors tackle a neglected tuning problem in offline reinforcement learning in a 2025 paper. Before judging a policy from logged data, one must choose the evaluation method itself, and those methods bring their own settings or wild variance. The team focus on tuning the evaluators: picking among candidate value functions or candidate world models to best grade a target policy.

They develop new selection methods for both families, with theoretical guarantees. They also design a fresh experimental protocol that generates candidate value functions in a stable, controlled way without optimization. The protocol can test both families on equal footing. On a standard hopping robot task, their tournament style selector showed promising results. The paper treats evaluator choice as a first class problem.

Catalog record

CREATORPai Liu, Lingfeng Zhao, Shivangi Agarwal, Jinghan Liu, Audrey Huang, Philip Amortila, Nan Jiang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-08021

Browse the full library · About the Tower of Babel