On admissibility in post-hoc hypothesis testing
Ben Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald · 2025
Read On admissibility in post-hoc hypothesis testing by Ben Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00770.txt.
Introduction
Ben Chugg, Tyron Lardy, Aaditya Ramdas, and Peter Grünwald built a theory for tests that pick their bar after seeing data. Their paper, from twenty twenty-five, works on post-hoc hypothesis testing. In the classic setup, the bar for significance must be fixed before any test runs. That rule blocks updates when costs shift or evidence looks far stronger than planned.
The team defines a test of quality for such loose rules. An adversary maps the data to a bar, and a good rule must hold up across that set. They sort which rules pass for which sets. Readers get a firm base for judging tests that choose their bar late, with clear bounds on when the choice stays safe.
Catalog record
| CREATOR | Ben Chugg, Tyron Lardy, Aaditya Ramdas, Peter Grünwald |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 171 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00770