CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs

Florent Draye, Abir Harrasse, Vedant Palit, Tung-Yu Wu, Jiarui Liu, Punya Syon Pandey, Roderick Wu, Terry Jingchen Zhang, Zhijing Jin, Bernhard Schölkopf, [0.3em] Max Planck Institute for Intelligent Systems, Tübingen, Germany · 2026

Read CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs by Florent Draye, Abir Harrasse, Vedant Palit, Tung-Yu Wu, Jiarui Liu, Punya Syon Pandey, Roderick Wu, Terry Jingchen Zhang, Zhijing Jin, Bernhard Schölkopf, [0.3em] Max Planck Institute for Intelligent Systems, Tübingen, Germany (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21014.txt.

Description

Draye and co-authors release CLT-Forge, a library for training and analyzing Cross-Layer Transcoders. Transcoders explain model computation through sparse features and their links, drawn as attribution graphs. Cross-Layer Transcoders share features across layers, which makes the graphs more compact, but they are hard to train at scale. The library adds distributed training with model sharding and compressed activation caching. It also bundles an automated feature explanation pipeline, attribution graph computation, and a flexible viewer. The code is public.

Catalog record

CREATORFlorent Draye, Abir Harrasse, Vedant Palit, Tung-Yu Wu, Jiarui Liu, Punya Syon Pandey, Roderick Wu, Terry Jingchen Zhang, Zhijing Jin, Bernhard Schölkopf, [0.3em] Max Planck Institute for Intelligent Systems, Tübingen, Germany
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26656
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21014

Browse the full library · About the Tower of Babel