Flash Interpretability: Decoding Specialised Feature Neurons in Large Language Models with the LM-Head

Harry J. Davies · 2025

Read Flash Interpretability: Decoding Specialised Feature Neurons in Large Language Models with the LM-Head by Harry J. Davies (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-02688.txt.

Description

Harry Davies shows that individual neurons in a large language model can be decoded directly into interpretable features, in a paper from twenty twenty-five. Language models have billions of parameters and resist interpretation. Davies routes neuron weights through the model's final projection layer, the LM-head. That turns weights into token probabilities a human can read.

Working with Llama 3.1 8B, the method finds specialized neurons, including a dog neuron and a California neuron. Clamping those neurons changes the probability of the concept in the output, validating the reading. Over three quarters of up-projection neurons kept the same top token between pretrained and instruct models. Clamping the dog neuron made the instruct model always discuss dogs when asked its favorite animal. The full neuron map takes under ten seconds to compute.

Catalog record

CREATORHarry J. Davies
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-02688

Browse the full library · About the Tower of Babel