Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs
Sree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang · 2026
Read Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs by Sree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07806.txt.
Introduction
Sree Bhattacharyya and five co-authors ask language models to judge themselves more richly than a single confidence number. Stated confidence is a known too-optimistic and unsteady sign of whether a model is right. Borrowing appraisal theory from psychology, which splits self-judgment into parts, they elicit six appraisal aspects alongside confidence from the models.
Across twelve models and thirty-eight tasks in eight domains, competence-related aspects, above all effort and ability, matched or beat confidence at predicting failure. Effort estimates were also less too-optimistic and stable across model sizes, while emotional aspects added little. The most telling aspect shifted sensibly with the task: effort for reasoning-heavy work, ability and confidence for retrieval.
Catalog record
| CREATOR | Sree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 248 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07806