HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger · 2025
Read HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks by Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10894.txt.
Description
Jiuding Sun and six co-authors introduced HyperDAS in two thousand twenty-five, a tool to automate the study of how language models work inside. Past methods could find the inner features that carry a concept, but they had to search by brute force for where those features sit. That search is slow and costly.
HyperDAS uses a transformer-based helper network to find the right spots on its own and build the features there. In tests with Llama3-8B it scored best on the RAVEL test for pulling concepts apart. The authors also review design choices meant to keep the tool honest, so it reads the model rather than adding to it.
Catalog record
| CREATOR | Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, Atticus Geiger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-10894