On admissibility in post-hoc hypothesis testing

Ben Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald · 2025

Read On admissibility in post-hoc hypothesis testing by Ben Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00770.txt.

Introduction

Ben Chugg, Tyron Lardy, Aaditya Ramdas, and Peter Grünwald built a theory for tests that pick their bar after seeing data. Their paper, from twenty twenty-five, works on post-hoc hypothesis testing. In the classic setup, the bar for significance must be fixed before any test runs. That rule blocks updates when costs shift or evidence looks far stronger than planned.

The team defines a test of quality for such loose rules. An adversary maps the data to a bar, and a good rule must hold up across that set. They sort which rules pass for which sets. Readers get a firm base for judging tests that choose their bar late, with clear bounds on when the choice stays safe.

Catalog record

CREATORBen Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE171 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00770

Browse the full library · About the Tower of Babel