Class Adaptive Conformal Training

Badr-Eddine Marani, Julio Silva-Rodríguez, Ismail Ben Ayed, Maria Vakalopoulou, Stergios Christodoulidis, Jose Dolz, [1ex] CentraleSupélec, Université Paris-Saclay · 2026

Read Class Adaptive Conformal Training by Badr-Eddine Marani, Julio Silva-Rodríguez, Ismail Ben Ayed, Maria Vakalopoulou, Stergios Christodoulidis, Jose Dolz, [1ex] CentraleSupélec, Université Paris-Saclay (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09522.txt.

Introduction

Badr-Eddine Marani and five co-authors work on making neural network predictions honest about doubt. Their 2026 arXiv paper adapts conformal prediction to each class separately. Deep networks often sound sure even when they are wrong, and their stated probabilities can mislead.

Conformal prediction wraps a model's output in a set with a guaranteed coverage rate. A single global rule, though, can give some classes loose sets and others tight ones. Class-adaptive training aims to even that out while learning. Fairer uncertainty matters most where a wrong, confident answer is costly, as in medical imaging.

Catalog record

CREATORBadr-Eddine Marani, Julio Silva-Rodríguez, Ismail Ben Ayed, Maria Vakalopoulou, Stergios Christodoulidis, Jose Dolz, [1ex] CentraleSupélec, Université Paris-Saclay
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09522

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Class Adaptive Conformal Training Badr-Eddine Marani, Julio Silva-Rodríguez, Ismail Ben Ayed, Maria Vakalopoulou, Stergios Christodoulidis, Jose Dolz, [1ex] CentraleSupélec, Université Paris-Saclay Abstract Deep neural networks have achieved remarkable success across a variety of tasks, yet they often suffer from unreliable probability estimates. As a result, they can be overconfident in their predictions. Conformal Prediction (CP) offers a principled framework for uncertainty quantification, yielding prediction sets with rigorous coverage guarantees. Existing conformal training methods optimize for overall set size, but shaping the prediction sets in a class-conditional manner is not straightforward and typically requires prior knowledge of the data distribution. In this work, we introduce Class Adaptive Conformal Training (CaCT), which formulates conformal training as an augmented Lagrangian optimization problem that adaptively learns to shape prediction sets class-conditionally without making any distributional assumptions. Experiments on multiple benchmark datasets, including standard and long-tailed image recognition as well as text classification, demonstrate that CaCT consistently outperforms prior conformal training methods, producing significantly smaller and more informative prediction sets while maintaining the desired coverage guarantees. 1 Introduction Deploying machine learning based systems in high-stakes, real-world scenarios requires not only strong predictive performance but also reliable measures of uncertainty. Conformal Prediction (CP) has emerged as a principled approach to meet this requirement 42 ; 41 . Given a pre-trained model, a calibration dataset, and a new test example X_{\mathrm{test}}\in\mathcal{X} with unknown label Y_{\mathrm{test}}\in\mathcal{Y} , CP outputs prediction sets \mathcal{C}(X_{\mathrm{test}})\subseteq\mathcal{Y} , which are agnostic to the model and data distribution, and statistically guaranteed to include the correct output with a user-defined mis-coverage level \alpha\in[0,1] : \Pr(Y_{\mathrm{test}}\in\mathcal{C}(X_{\mathrm{test}}))\geq 1-\alpha. (1) This distribution-free guarantee is a significant property, as it ensures the validity of the prediction sets. Yet, while validity is crucial, the practical value of a prediction set depends on its efficiency, which is typically evaluated as the expected size of the predicted sets \mathbb{E}_{X}|\mathcal{C}(X)| . For instance, a trivial set that contains all labels is valid but uninformative . Moreover, CP sets should exhibit adaptivity , producing compact sets for instances deemed easy by the model and larger ones for more uncertain or ambiguous cases. Figure 1 : Existing conformal training methods fail to account for datasets with a large number of classes or long-tailed distributions. We compare CaCT against conformal training methods, and report coverage gap and set size across relevant benchmarks. Lower values for both metrics indicate better conformalization and more informative prediction sets ( i.e . , better efficiency ) across all classes. Among all methods, CaCT exhibits superior performance. Methods for enhancing the efficiency of conformal predictors span from designing effective non-conformity scores 1 ; 31 ; 34 to explicit optimization-based strategies 39 ; 8 ; 11 ; 3 ; 36 . While the former approach enjoys widespread adoption, applying CP solely as a post-training procedure prevents the underlying model from adapting to the construction of confidence sets 39 . Consequently, training conformal models end-to-end has emerged as a particularly promising strategy for achieving both validity and improved efficiency. In particular, ConfTr 39 represents one of the pioneer efforts to integrate model training with conformal prediction end-to-end. By resorting to smooth implementations of recent non-conformity scores, ConfTr introduces a differentiable formulation of CP, allowing the adoption of existing CP methods during training. Compared to standard training, this strategy reduces the average confidence set size of state-of-the-art CP approaches. Nevertheless, although ConfTr 39 advocates for improved efficiency, their adaptivity remains fundamentally constrained by the underlying formulation. To illustrate this limitation, let us examine its main learning objective, which introduces a regularization term to control efficiency by penalizing the size of the prediction set. This penalty is modulated by a single hyperparameter \lambda , applied uniformly across all classes. While simple, this design makes the model’s performance highly sensitive to the choice of \lambda , especially in large-scale or long-tailed settings. In addition, this weight is typically set before training and remains fixed, preventing the model from learning an optimal trade-off between accuracy and conformalization. Such a class-agnostic, rigid formulation is suboptimal, as different classes often exhibit distinct underlying characteristics. This limitation is further exacerbated in long-tailed scenarios, a common setting in real-world applications, where the majority classes often dominate the optimization dynamics, leading to biased conformalization and reduced adaptivity for underrepresented classes ( Fig. 2 ). Figure 2 : Impact of per-class conformal set size and coverage with class frequency ( \alpha=0.1 ) in CIFAR10-LT . As classes become under-represented (toward the right of the x-axis), existing methods deteriorate their conformal performance, i.e . , increase set sizes and decrease class-conditional coverage. In contrast, CaCT is less sensitive to class imbalance, yielding smaller class set sizes ( left ) while maintaining higher class coverage ( right ) for these classes. In light of the above limitations, we propose a novel conformal training strategy for efficient and adaptive conformal prediction. The key contributions can be summarized as: • We introduce Class Adaptive Conformal Training (CaCT), a novel framework that employs class-wise multipliers instead of a single global weight to construct efficient and adaptive prediction sets. This solves the issues highlighted above: (i) scaling to datasets with a large number of classes, which exhibit diverse intrinsic difficulties, and (ii) effectively learning under class imbalance. • The resulting constrained optimization problem is efficiently solved by the proposed modified Augmented Lagrangian algorithm, which provides adaptive and optimal weights for the class-wise penalties. Compared to existing conformal training methods, CaCT shapes class-conditional prediction sets without designing new loss functions or requiring prior knowledge of the data distribution. CaCT naturally handles datasets with many classes ( e.g . , ImageNet) and mitigates class imbalance issues ( e.g . , ImageNet-LT) by learning adaptive class weights. • Comprehensive experiments across multiple benchmarks and distinct tasks, including standard image classification, long-tailed image recognition, and text classification, show that CaCT produces smaller and more informative prediction sets, while maintaining coverage, compared to existing training CP methods. As shown in Fig. 1 , CaCT outperforms state-of-the-art conformal training methods when considering both set efficiency and coverage gap, particularly in more realistic large-scale datasets with a large number of classes or under class imbalance. 2 Related Work Conformal Prediction (CP) provides a principled framework to quantify uncertainty 2 ; 41 . Rather than returning a single label, CP outputs a prediction set that contains the true outcome with a user-specified probability. For instance, in medical diagnosis, a conformal predictor might return a set of possible diseases and guarantee that the correct one is included 90% of the time. This framework has recently attracted significant attention 2 and has been successfully applied across tasks including regression 30 , classification 1 ; 31 ; 37 ; 13 , image segmentation 6 ; 24 , and language modeling 44 . Split CP 27 ; 41 is a widely used method for constructing prediction sets. Its core appeal lies in its simplicity and generality: given a pre-trained model and a calibration set, conformal prediction wraps around the model, outputting prediction sets with guaranteed coverage on a held-out sample under exchangeability, at a fraction of the computational cost, and without making any assumptions about the underlying data distribution. Despite its broad applicability, CP is a post-hoc and model-agnostic procedure, and

Read the complete text