Parametric Scaling Law of Tuning Bias in Conformal Prediction
Hao Zeng, Kangdao Liu, Bingyi Jing, Hongxin Wei · 2025
Read Parametric Scaling Law of Tuning Bias in Conformal Prediction by Hao Zeng, Kangdao Liu, Bingyi Jing, Hongxin Wei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03023.txt.
Description
Hao Zeng and co-authors wrote this 2025 statistics paper on a quiet rule-breaking habit in conformal prediction. That method builds prediction sets with guaranteed coverage, and its math assumes the tuning data and calibration data stay separate. In practice, people often tune on the same data they calibrate with.
The authors measure the harm, a coverage gap they call tuning bias. Empirically, the gap stays tiny for simple tuning in many standard methods. A pattern emerges: the bias grows with the size of the tuning space and shrinks as the calibration set grows. The paper proves that scaling law with an upper bound on the bias. It closes with theory-guided ways to keep the bias small.
Catalog record
| CREATOR | Hao Zeng, Kangdao Liu, Bingyi Jing, Hongxin Wei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-03023