Tracing Facts or just Copies? A critical investigation of the Competitions of Mechanisms in Large Language Models
e Campregher [email protected], u Chen [email protected], er Hoffman [email protected], a Heuss [email protected] · 2025
Read Tracing Facts or just Copies? A critical investigation of the Competitions of Mechanisms in Large Language Models by e Campregher [email protected], u Chen [email protected], er Hoffman [email protected], a Heuss [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11809.txt.
Description
Dante Campregher, Yanxu Chen, Sander Hoffman and Maria Heuss revisit how language models choose between learned facts and facts given in context. Three recent studies used mechanistic tools to study that contest, with unlike claims about attention heads. Their reproduction study tests whether head strength predicts how often the model gives the learned fact.
Heads that favour factual output seem to work by general copy suppression, not by blocking only false context. Boosting them can even block correct facts. Head behaviour also varied by domain, and larger models showed more specialised, category-sensitive patterns. The findings urge care when single head roles are claimed across all kinds of text.
Catalog record
| CREATOR | e Campregher [email protected], u Chen [email protected], er Hoffman [email protected], a Heuss [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-11809