Towards Spectroscopy: Susceptibility Clusters in Language Models

Andrew GordonTimaeus [email protected], Garrett BakerTimaeus [email protected] George WangTimaeus [email protected] William SnellTimaeus [email protected] Stan van WingerdenTimaeus [email protected] Daniel MurfetTimaeus [email protected] · 2026

Read Towards Spectroscopy: Susceptibility Clusters in Language Models by Andrew GordonTimaeus [email protected], Garrett BakerTimaeus [email protected] George WangTimaeus [email protected] William SnellTimaeus [email protected] Stan van WingerdenTimaeus [email protected] Daniel MurfetTimaeus [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12703.txt.

Introduction

Andrew Gordon and five co authors at Timaeus borrow a method from physics to study language models. Spectroscopy infers internal structure by measuring how a system responds to perturbations. They apply that principle to neural networks. Their perturbation upweights one token in a given context, shifting the data distribution slightly.

The response is measured through susceptibilities, covariances between component level observables and the perturbation. These are computed over a localized Gibbs posterior, in the language of the work. A susceptibility map could show which parts of a model care about which tokens. That would give interpretability a tool with roots in statistical physics. The team frame this as a step toward a spectroscopy of models.

Catalog record

CREATORAndrew GordonTimaeus [email protected], Garrett BakerTimaeus [email protected] George WangTimaeus [email protected] William SnellTimaeus [email protected] Stan van WingerdenTimaeus [email protected] Daniel MurfetTimaeus [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE164 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12703

Browse the full library · About the Tower of Babel