Language models fail at extended rule following
Tianxiang Dai, Jonathan A. Fan · 2026
Read Language models fail at extended rule following by Tianxiang Dai, Jonathan A. Fan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02028.txt.
Description
Tianxiang Dai and Jonathan Fan test whether language models can follow a simple rule over a long stretch without error. Agents need more than recall and reasoning over long context. They must keep an exact state while applying the same rule again and again. The authors use counting a long string of repeated characters as a clean test of that skill.
Across 126 leading model variants, every model failed to count accurately above a threshold that depends on the model and on syntax. Failures were abrupt and persisted with larger models, more compute, and external tools. Probing suggests models mimic counting with a finite set of internal states and fail when those states run out. The authors argue new architectures are needed.
Catalog record
| CREATOR | Tianxiang Dai, Jonathan A. Fan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 109 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.02028