In-sample calibration yields conformal calibration guarantees
Sam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel · 2025
Read In-sample calibration yields conformal calibration guarantees by Sam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03841.txt.
Description
Sam Allen and others show that a forecasting system tuned to be honest on its own training data can be upgraded by a fixed recipe. The upgrade gives it honesty guarantees on future data. Conformal sets already wrap forecasts with coverage pledges, assuming exchangeable data. Fuller systems go further and issue whole outcome spreads. Writing in twenty twenty-five, the team proves that conformal systems quietly rely on in-sample calibrated predictors, and that any in-sample calibrated method can be conformalized to inherit out-of-sample calibration. The logic extends from plain range coverage to the stronger conditional calibration notions used in forecasting theory.
Two new systems follow: one based on binning the data, another on isotonic regression of distributions, a shape-constrained method. Model tests and two case studies test them against current conformal systems. The bridge between calibration and conformal methods eases building forecasters with guarantees.
Catalog record
| CREATOR | Sam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 110 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-03841