HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks

Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger · 2025

Read HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks by Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10894.txt.

Description

Jiuding Sun and six co-authors introduced HyperDAS in two thousand twenty-five, a tool to automate the study of how language models work inside. Past methods could find the inner features that carry a concept, but they had to search by brute force for where those features sit. That search is slow and costly.

HyperDAS uses a transformer-based helper network to find the right spots on its own and build the features there. In tests with Llama3-8B it scored best on the RAVEL test for pulling concepts apart. The authors also review design choices meant to keep the tool honest, so it reads the model rather than adding to it.

Catalog record

CREATORJiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10894

Browse the full library · About the Tower of Babel