Gap-K%: Measuring Top-1 Prediction Gap for Detecting Pretraining Data

Minseo Kwak, Jaehyung Kim · 2026

Read Gap-K%: Measuring Top-1 Prediction Gap for Detecting Pretraining Data by Minseo Kwak, Jaehyung Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19936.txt.

Description

Minseo Kwak and Jaehyung Kim work on detecting whether a text was used to train a language model. The secrecy around massive pretraining corpora raises privacy and copyright concerns. Detection methods usually rely on token likelihoods: text the model finds unsurprising was probably in its training data. These methods overlook how the text diverges from the model's own top prediction.

Their method, Gap-K%, measures that top-one prediction gap. It also uses local correlation between neighbouring tokens. Combining both signals should sharpen detection. Reliable detection tools matter for authors, publishers, and regulators auditing models. The work adds a new signal to a contested measurement problem.

Catalog record

CREATORMinseo Kwak, Jaehyung Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-19936

Browse the full library · About the Tower of Babel