CANDOR: Counterfactual ANnotated DOubly Robust Off-Policy Evaluation

Aishwarya Mandyam, Shengpu Tang, Jiayu Yao, Jenna Wiens, Barbara E. Engelhardt · 2024

Read CANDOR: Counterfactual ANnotated DOubly Robust Off-Policy Evaluation by Aishwarya Mandyam, Shengpu Tang, Jiayu Yao, Jenna Wiens, Barbara E. Engelhardt (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08052.txt.

Description

Aishwarya Mandyam and four co-authors improved in 2024 how new decision rules are graded from old data when experts add guesses about untried actions. Such grading guides high-stakes fields like health care, where trying a new policy live is risky. Expert notes on what would have happened widen the data, but the notes are imperfect and can mislead.

Their family of estimators folds the notes into a doubly robust frame that pairs sampling weights with a reward model. Three ways of using the notes are compared, with proofs that the notes help under mild assumptions. Safer grading follows.

Catalog record

CREATORAishwarya Mandyam, Shengpu Tang, Jiayu Yao, Jenna Wiens, Barbara E. Engelhardt
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE133 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-08052

Browse the full library · About the Tower of Babel