From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks

Ayan Datta, Mounika Marreddy, Alexander Mehler, Zhixue Zhao, Radhika Mamidi, [0.5em] IIIT Hyderabad · 2026

Read From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks by Ayan Datta, Mounika Marreddy, Alexander Mehler, Zhixue Zhao, Radhika Mamidi, [0.5em] IIIT Hyderabad (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00778.txt.

Description

Datta and colleagues use character counting, such as asking how many p's appear in apple, as a controlled probe of language models' symbolic reasoning. They find models often compute the right answer internally yet fail to state it. Late-layer components suppress correct signals in favor of likelier wrong outputs. The study concludes these failures come from structured interference inside the computation, not from missing representations or too little scale.

Catalog record

CREATORAyan Datta, Mounika Marreddy, Alexander Mehler, Zhixue Zhao, Radhika Mamidi, [0.5em] IIIT Hyderabad
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56426
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.00778

Browse the full library · About the Tower of Babel