Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception

Shiyu Ni Keping Bi Jiafeng Guo Lulu Yu Baolong Bi Xueqi Cheng · 2025

Read Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception by Shiyu Ni Keping Bi Jiafeng Guo Lulu Yu Baolong Bi Xueqi Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11677.txt.

Description

Shiyu Ni and co-authors explore in 2025 whether a language model knows what it does not know, using signals from inside the model itself. Models often answer with confidence when they are wrong. The team asked whether internal states, read before any answer is generated, already betray that uncertainty.

Tests on question sets including Natural Questions, HotpotQA, and MMLU showed strong pre-generation perception, sharpened further after generation. The gaps stayed steady across conditions. A new calibration trick checks whether reworded questions draw consistent confidence, and it lifted the rate at which models spot their own gaps by about five points. The work targets both savings and safety.

Catalog record

CREATORShiyu Ni Keping Bi Jiafeng Guo Lulu Yu Baolong Bi Xueqi Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11677

Browse the full library · About the Tower of Babel