In-sample calibration yields conformal calibration guarantees

Sam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel · 2025

Read In-sample calibration yields conformal calibration guarantees by Sam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03841.txt.

Description

Sam Allen and others show that a forecasting system tuned to be honest on its own training data can be upgraded by a fixed recipe. The upgrade gives it honesty guarantees on future data. Conformal sets already wrap forecasts with coverage pledges, assuming exchangeable data. Fuller systems go further and issue whole outcome spreads. Writing in twenty twenty-five, the team proves that conformal systems quietly rely on in-sample calibrated predictors, and that any in-sample calibrated method can be conformalized to inherit out-of-sample calibration. The logic extends from plain range coverage to the stronger conditional calibration notions used in forecasting theory.

Two new systems follow: one based on binning the data, another on isotonic regression of distributions, a shape-constrained method. Model tests and two case studies test them against current conformal systems. The bridge between calibration and conformal methods eases building forecasters with guarantees.

Catalog record

CREATORSam Allen, Georgios Gavrilopoulos, Alexander Henzi, Gian-Reto Kleger, Johanna Ziegel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03841

Browse the full library · About the Tower of Babel