How Do LLMs Cite?
Ian van Dort, Maria Heuss · 2026
Read How Do LLMs Cite? by Ian van Dort, Maria Heuss (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28358.txt.
Description
Ian van Dort and Maria Heuss open up a language model to watch it decide whether to attach a citation. Retrieval-augmented systems answer questions from fetched documents and add inline citations for trust, but a citation only helps if it faithfully marks where the answer came from.
Van Dort and Heuss use activation patching, swapping internal activations between runs, on Llama-3.1-8B-Instruct answering factoid questions. They locate an attributional ensemble, a set of attention heads and feed-forward layers that together drive citation decisions. The circuitry leans heavily on shallow shortcuts, such as matching entities between question and passage, rather than deep checks of what supports the claim. Citations that look reassuring, they warn, may rest on much thinner evidence than readers assume.
Catalog record
| CREATOR | Ian van Dort, Maria Heuss |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28358