End-to-end Learning of Sparse Interventions on Activations to Steer Generation
Pau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗ · 2025
Read End-to-end Learning of Sparse Interventions on Activations to Steer Generation by Pau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10679.txt.
Description
Pau Rodriguez and co-authors present a method to steer what a generative model produces by shifting its inner signals, in a paper from 2025. The head of the record gives the work a second title that names the method LinEAS. Steering can help make outputs safer or let users try style changes.
Older steering maps are tuned one layer at a time and can cause unwanted shifts in new cases. The new method trains across all layers at once with a single loss, and it can pick which neurons to change. It needs only a few unpaired examples to work. Tests on text and image models beat similar methods on reducing toxic output and on adding new concepts.
Catalog record
| CREATOR | Pau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-10679