On the Interconnections of Calibration, Quantification, and Classifier Accuracy Prediction under Dataset Shift
Alejandro Moreo · 2025
Read On the Interconnections of Calibration, Quantification, and Classifier Accuracy Prediction under Dataset Shift by Alejandro Moreo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11380.txt.
Introduction
Alejandro Moreo, writing alone in 2025, proves that three seemingly separate problems in machine learning are secretly the same problem. The setting is dataset shift, where test data drifts from training data. The three tasks are calibrating a classifier's scores, estimating the share of positives in new data, and predicting the classifier's accuracy on that new data.
Moreo shows each task can be reduced to the others: an oracle for any one solves all three. That equivalence lets methods cross borders. Adapting a strong calibration method yields a quantification method, and so on. In tests, the borrowed methods often matched or beat the dedicated ones. The paper urges the three research communities to build unified tools.
Catalog record
| CREATOR | Alejandro Moreo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 145 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11380