Confidence Calibration in Large Language Models
Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore · 2026
Read Confidence Calibration in Large Language Models by Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23909.txt.
Description
Michael and three colleagues test how well large language models judge their own certainty across varied tasks. In a preregistered study, the models, like people, proved overconfident on average, with confidence running ahead of accuracy. The gap grew on hard tests yet flipped on easy ones, where models showed marked underconfidence. To measure this pattern across difficulty levels, the team built LifeEval, a new test of model calibration.
Catalog record
| CREATOR | Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56467 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.23909