How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing

Javier Marín · 2026

Read How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing by Javier Marín (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13259.txt.

Description

Javier Marín studies how decoder-only transformers process correct and incorrect factual continuations. When forced to carry a wrong token, a model's hidden states rotate away from the correct path. Angular separation grows through mid-depth, then late layers settle on an asymmetric outcome. Marín reads this pattern as geometric, and stresses that the evidence is observational, not causal. The pattern held across six models from four architecture families, with one flat exception. Probes and patching tests suggest the effect is spread across the trajectory, not one layer.

Catalog record

CREATORJavier Marín
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36195
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13259

Browse the full library · About the Tower of Babel