Language models fail at extended rule following

Tianxiang Dai, Jonathan A. Fan · 2026

Read Language models fail at extended rule following by Tianxiang Dai, Jonathan A. Fan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02028.txt.

Description

Tianxiang Dai and Jonathan Fan test whether language models can follow a simple rule over a long stretch without error. Agents need more than recall and reasoning over long context. They must keep an exact state while applying the same rule again and again. The authors use counting a long string of repeated characters as a clean test of that skill.

Across 126 leading model variants, every model failed to count accurately above a threshold that depends on the model and on syntax. Failures were abrupt and persisted with larger models, more compute, and external tools. Probing suggests models mimic counting with a finite set of internal states and fail when those states run out. The authors argue new architectures are needed.

Catalog record

CREATORTianxiang Dai, Jonathan A. Fan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02028

Browse the full library · About the Tower of Babel