Darkness Visible: Reading the Exception Handler of a Language Model

Peter Balogh · 2026

Read Darkness Visible: Reading the Exception Handler of a Language Model by Peter Balogh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04756.txt.

Description

Balogh reads the final MLP layer of GPT-2 Small as a legible routing program, with 27 neurons in a three-tier exception handler. Five core neurons reset the vocabulary toward function words, ten differentiators weed out wrong candidates, five specialists spot breaks, and seven consensus neurons each watch one trait. The point where MLP meddling flips from help to harm is sharp, falling between four of seven and five of seven consensus levels. Tests suggest knowledge neurons act as routing gear, not fact stores, amplifying or damping stream signals.

Catalog record

CREATORPeter Balogh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34211
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04756

Browse the full library · About the Tower of Babel