Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs

Sree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang · 2026

Read Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs by Sree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07806.txt.

Introduction

Sree Bhattacharyya and five co-authors ask language models to judge themselves more richly than a single confidence number. Stated confidence is a known too-optimistic and unsteady sign of whether a model is right. Borrowing appraisal theory from psychology, which splits self-judgment into parts, they elicit six appraisal aspects alongside confidence from the models.

Across twelve models and thirty-eight tasks in eight domains, competence-related aspects, above all effort and ability, matched or beat confidence at predicting failure. Effort estimates were also less too-optimistic and stable across model sizes, while emotional aspects added little. The most telling aspect shifted sensibly with the task: effort for reasoning-heavy work, ability and confidence for retrieval.

Catalog record

CREATORSree Bhattacharyya, Samarth Khanna, Leona Chen, Lucas Craig, Tharun Dilliraj, James Z. Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE248 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07806

Browse the full library · About the Tower of Babel