The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models

Seonglae Cho, Zekun Wu, Kleyton Da Costa, Adriano Koshiyama · 2026

Read The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models by Seonglae Cho, Zekun Wu, Kleyton Da Costa, Adriano Koshiyama (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08159.txt.

Description

Language models state falsehoods as fluently as facts, so asking whether a model knows its claim is wrong requires looking inside, not at its confidence. Seonglae Cho and three co-authors map the geometry of correctness representations across eleven models from 124 million to 14 billion parameters, testing causally with steering, erasure, and alignment search.

The structure is strikingly simple: two class centroids in a 2- to 8-dimensional subspace match a trained linear probe. Just 25 labeled examples recover 90 percent of full-data detection results on GPT-2. Steering, meanwhile, moves the hallucination rate by 9.1 points, erasure destroys detection, and causal dimensions number at most five. Interpretability teams gain evidence that correctness lives in a tiny, manipulable subspace.

Catalog record

CREATORSeonglae Cho, Zekun Wu, Kleyton Da Costa, Adriano Koshiyama
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08159

Browse the full library · About the Tower of Babel