Low-Perplexity LLM-Generated Sequences and Where To Find Them

Arthur Wuhrmann, Anastasiia Kucherenko, Andrei Kucharavy · 2025

Read Low-Perplexity LLM-Generated Sequences and Where To Find Them by Arthur Wuhrmann, Anastasiia Kucherenko, Andrei Kucharavy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01844.txt.

Description

Arthur Wuhrmann, Anastasiia Kucherenko, and Andrei Kucharavy wrote a 2025 paper on what large language models memorize. Their method hunts for stretches of text that a model finds very likely, word for word. Such stretches may be copied from training data, or they may be text the model builds on its own.

The team built a pipeline that pulls out long, likely passages on many topics without the text falling apart into repeats. They then searched the training corpus for matches. Oddly, a large share of these passages could not be found in the corpus at all. For those that did match, they counted how often each source appeared. The results shape debates on privacy and credit, since they show recall is real but patchy.

Catalog record

CREATORArthur Wuhrmann, Anastasiia Kucherenko, Andrei Kucharavy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01844

Browse the full library · About the Tower of Babel