Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan Anthropic * Equal contribution; † \dagger Corresponding author: [email protected] HTML version · 2026

Read Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet by Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan Anthropic * Equal contribution; † \dagger Corresponding author: [email protected] HTML version (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29358.txt.

Description

Adly Templeton and colleagues show that sparse autoencoders can extract clear features from Claude 3 Sonnet, a production language model. The authors trained them with up to thirty four million features on the model's middle hidden stream, using scaling laws to guide choice. The features are many language and many mode, working on images despite text only training, respond to cases and talk, and can steer the model. Features cover names, concepts like sarcasm or errors, and harms, like lies, power seeking, praise, and bias, which shape outputs when tweaked.

Catalog record

CREATORAdly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan Anthropic * Equal contribution; † \dagger Corresponding author: [email protected] HTML version
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135628
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29358

Browse the full library · About the Tower of Babel