Confidence Calibration in Large Language Models

Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore · 2026

Read Confidence Calibration in Large Language Models by Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23909.txt.

Description

Michael and three colleagues test how well large language models judge their own certainty across varied tasks. In a preregistered study, the models, like people, proved overconfident on average, with confidence running ahead of accuracy. The gap grew on hard tests yet flipped on easy ones, where models showed marked underconfidence. To measure this pattern across difficulty levels, the team built LifeEval, a new test of model calibration.

Catalog record

CREATORNoam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56467
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23909

Browse the full library · About the Tower of Babel