How Do LLMs Cite?

Ian van Dort, Maria Heuss · 2026

Read How Do LLMs Cite? by Ian van Dort, Maria Heuss (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28358.txt.

Description

Ian van Dort and Maria Heuss open up a language model to watch it decide whether to attach a citation. Retrieval-augmented systems answer questions from fetched documents and add inline citations for trust, but a citation only helps if it faithfully marks where the answer came from.

Van Dort and Heuss use activation patching, swapping internal activations between runs, on Llama-3.1-8B-Instruct answering factoid questions. They locate an attributional ensemble, a set of attention heads and feed-forward layers that together drive citation decisions. The circuitry leans heavily on shallow shortcuts, such as matching entities between question and passage, rather than deep checks of what supports the claim. Citations that look reassuring, they warn, may rest on much thinner evidence than readers assume.

Catalog record

CREATORIan van Dort, Maria Heuss
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28358

Browse the full library · About the Tower of Babel