Weight Patching: Toward Source-Level Mechanistic Localization in LLMs

Chenghao Sun, Chengsheng Zhang, Guanzheng Qin, Rui Dai, and Xinmei Tian · 2026

Read Weight Patching: Toward Source-Level Mechanistic Localization in LLMs by Chenghao Sun, Chengsheng Zhang, Guanzheng Qin, Rui Dai, and Xinmei Tian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13694.txt.

Description

Sun and four colleagues localize language-model behavior to the weights that truly source it, not just activations that pass signals along. Their Weight Patching method swaps selected module weights from a behavior-specialized model into a base model of the same architecture under a fixed input. A vector-anchor interface judges whether a task-relevant control state has formed. The analysis finds layers of sources: shallow candidate carriers first, then aggregation and routing modules, then downstream execution circuits. Its scores also guide better model merging.

Catalog record

CREATORChenghao Sun, Chengsheng Zhang, Guanzheng Qin, Rui Dai, and Xinmei Tian
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE171557
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13694

Browse the full library · About the Tower of Babel