ReCall : Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles

Munachiso Nwadike, Zangir Iklassov, Toluwani Aremu, Tatsuya Hiraoka, Benjamin Heinzerling, Velibor Bojkovic, Hilal Alqaubeh, Martin Takáč, Kentaro Inui · 2025

Read ReCall : Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles by Munachiso Nwadike, Zangir Iklassov, Toluwani Aremu, Tatsuya Hiraoka, Benjamin Heinzerling, Velibor Bojkovic, Hilal Alqaubeh, Martin Takáč, Kentaro Inui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13491.txt.

Description

Munachiso Nwadike and Zangir Iklassov lead a team on this paper from two thousand twenty-five, whose byline lists a slightly shifted author set. It names a trick, the self-referencing causal cycle, by which language models can beat the so-called reversal curse.

Models learn facts in one token order, so they often fail to give the line that comes before a known line. The authors find loops in the training text, which they call cycle tokens, that link far parts of a text. Those loops let a model reach back to what came before. With a firm probability frame and set tests, they show how the cycles steer what a model can recall. Code and full test notes are shared.

Catalog record

CREATORMunachiso Nwadike, Zangir Iklassov, Toluwani Aremu, Tatsuya Hiraoka, Benjamin Heinzerling, Velibor Bojkovic, Hilal Alqaubeh, Martin Takáč, Kentaro Inui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13491

Browse the full library · About the Tower of Babel