Your Model is Overconfident, and Other Lies We Tell Ourselves

Timothee Mickus Aman Sinha Raúl Vázquez, firstname.lastname@{helsinki.fi,univ-lorraine.fr} · 2025

Read Your Model is Overconfident, and Other Lies We Tell Ourselves by Timothee Mickus Aman Sinha Raúl Vázquez, firstname.lastname@{helsinki.fi,univ-lorraine.fr} (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01235.txt.

Introduction

Timothee Mickus, Aman Sinha, and Raúl Vázquez ask how we should judge whether a test example is truly hard for a language model. Their paper appeared in two thousand twenty-five. Hardness can come from the example itself, when its meaning is unclear. The team compared three ways to measure it. These were disagreement among human raters, how training unfolds, and how sure the model sounds.

They ran twenty-nine models on three data sets. The three measures did line up in part, but the links were neither straight nor steady. A case that looks hard by one measure may look easy by another. The authors argue that splitting these kinds of doubt apart gives a clearer view of data complexity. That view can help researchers judge and improve their models.

Catalog record

CREATORTimothee Mickus Aman Sinha Raúl Vázquez, firstname.lastname@{helsinki.fi,univ-lorraine.fr}
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01235

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Your Model is Overconfident, and Other Lies We Tell Ourselves Timothee Mickus Aman Sinha Raúl Vázquez, firstname.lastname@{helsinki.fi,univ-lorraine.fr} Abstract The difficulty intrinsic to a given example, rooted in its inherent ambiguity, is a key yet often overlooked factor in evaluating neural NLP models. We investigate the interplay and divergence among various metrics for assessing intrinsic difficulty, including annotator dissensus, training dynamics, and model confidence. Through a comprehensive analysis using 29 models on three datasets, we reveal that while correlations exist among these metrics, their relationships are neither linear nor monotonic. By disentangling these dimensions of uncertainty, we aim to refine our understanding of data complexity and its implications for evaluating and improving NLP models. 1 Introduction A central, but often overlooked, concept in natural language processing is the consensus of annotators when labeling a specific datapoint. Annotators often express different perspectives, and prior literature provides strong evidence that this dissensus is a legitimate characteristic of language data, rather than a consequence of noisy annotation processes ( Plank et al., 2014 ; Plank, 2022 ; Uma et al., 2021 ) . Annotator dissensus is often linked to data complexity : if humans do not agree as to whether a pair of sentences contradict each other, then we expect this pair to be hard to label with a neural network. Data complexity can be quantified through training dynamics (e.g., how early a training example is learned), model confidence, or performance variability across models ( Guo et al., 2017 ; Swayamdipta et al., 2020 ; Hendrycks and Dietterich, 2019 ) . This relationship shapes the design of evaluation benchmarks, which increasingly incorporate multiple annotations per datapoint ( Bowman et al., 2015 ; Nie et al., 2020 , e.g.,) . As such, matching human uncertainty with model uncertainty is an explicit desideratum laid out in numerous NLP applications. Common drivers of data uncertainty include annotation noise, semantic ambiguity, and overlapping class boundaries ( Hu et al., 2023 ) . These factors not only lower inter-annotator agreement but also highlight the presence of linguistic phenomena such as semantic indeterminacy — which lead to label ambiguity, as extensively documented across NLP tasks ( Bowman et al., 2015 ) . Hence, prior work has expected annotator dissensus to be a reasonable proxy for data complexity ( Hachey et al., 2005 ; Lalor et al., 2018 , e.g.,) . Our findings reveal non-linear and conflicting relationships between annotator dissensus and model-derived complexity metrics. Through experiments using 29 models on the ChaosNLI and DynaSent datasets ( Nie et al., 2020 ; Potts et al., 2021 ) , we observe that various metric indicators of data complexity often correlate with one another — and yet that the relationship they hold with human-based assessments of linguistic dissensus is far from linear or monotonic. Moreover, indicators of data complexity derived from model behavior tend to conflict depending on whether they account for the correctness of the models’ predictions — for instance, assessments derived from conformal prediction methods do not align with the training dynamics approach of Swayamdipta et al. (2020) , and both fail to adequately capture the true human label variation collected by Nie et al. (2020) . 2 Background Our study falls at the intersection of data complexity and uncertainty in NLP, particularly in relation to annotator disagreement, label variation, and the metrics used to evaluate them ( Jiang and de Marneffe, 2022 ; Uma et al., 2021 ; Lorena et al., 2019 ; Baan et al., 2023 ) . Though closely related, these concepts involve distinct challenges. Data uncertainty or aleatoric uncertainty describes the randomness or noise inherent to the data ( Hu et al., 2023 ) . This type of uncertainty is irreducible and cannot be eliminated through model improvements or tuning ( Kiureghian and Ditlevsen, 2009 ; Kendall and Gal, 2017 ; Hüllermeier and Waegeman, 2021 ) . Sources of data uncertainty include noisy observations, overlapping classes, ground truth errors or inherent randomness. Annotator disagreement is highlighted as a fundamental characteristic of linguistic data , stemming from both annotation noise and the inherent ambiguity of language ( Plank et al., 2014 ; Aroyo and Welty, 2015 ; Pavlick and Kwiatkowski, 2019 ; Fornaciari et al., 2021 ) . It often correlates with label uncertainty , where no single correct label exists. High-disagreement examples contain valuable signals for classifiers ( Basile et al., 2021 ; Palomaki et al., 2018 ) . However, disagreement does not necessarily indicate annotation noise—it may instead reflect genuine linguistic or contextual ambiguity. Data complexity or data difficulty refers to the characteristics of a data sample that make classification inherently difficult . It is related to the structural properties of the data, not to randomness or noise, as is the case with data uncertainty. Several factors contribute to data complexity: proximity to decision boundaries and class overlap, semantic indeterminacy, and task-specific challenges, such as requiring world knowledge ( Plank, 2022 ; Jiang and de Marneffe, 2022 ) . Metrics for evaluating data uncertainty and complexity include model confidence (probability of the predicted class), entropy of predicted probabilities (measuring classification uncertainty), and confidence calibration (aligning confidence with performance). These help assess label uncertainty caused by overlapping class boundaries ( Geng et al., 2024 ; Zhou et al., 2022 ; Xiao and Wang, 2019 , e.g.,) . Training dynamics, such as how quickly a model learns to classify an example or the shape of the loss curve, further reveal the relative difficulty of datapoints ( Swayamdipta et al., 2020 ; Toneva et al., 2019 ; Baldock et al., 2021 ) . These metrics offer tools to analyze the challenges of data complexity, yet they do not provide a nuanced perspective on the interplay between annotator disagreement and model uncertainty. Complexity, disagreement & uncertainty. Data complexity, annotator disagreement, and data uncertainty are intertwined phenomena with similar root causes. For instance, while authors differ in their terminology, ‘overlapping classes’ ( Ho and Basu, 2002 ; Peterson et al., 2019 ; Lorena et al., 2019 ) , ‘absence of a single ground truth’ ( Aroyo and Welty, 2015 ; Baan et al., 2022 ) , or ‘linguistic ambiguity’ due to semantic and social factors ( Plank, 2022 ) all refer to the fact that some datapoints can have different labels — which drives up complexity, disagreement and uncertainty. Hence, these three phenomena have been conflated in the literature. For example, uncertainty is often measured through label distribution entropy, used as a proxy for annotator disagreement ( Zhang et al., 2022 ; Baumler et al., 2023 ) . Similarly, Lalor et al. (2018) has found alignment between human difficulty and model-assigned probability mass, suggesting that both perceive difficulty similarly. This viewpoint is also prevalent in active learning ( Hachey et al., 2005 , e.g.,) and attested less directly in quality estimation ( Jamison and Gurevych, 2015 , e.g.,) . Additionally, anecdotal evidence has been used to support this connection, especially in studies exploring the underlying causes of data complexity ( Swayamdipta et al., 2020 ; Baldock et al., 2021 ; Rajpurkar et al., 2018 ) . This perspective has also been employed as a working hypothesis. For example, Weinshall et al. (2018) assume that knowledge distillation from a model can play

Read the complete text