End-to-end Learning of Sparse Interventions on Activations to Steer Generation

Pau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗ · 2025

Read End-to-end Learning of Sparse Interventions on Activations to Steer Generation by Pau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10679.txt.

Description

Pau Rodriguez and co-authors present a method to steer what a generative model produces by shifting its inner signals, in a paper from 2025. The head of the record gives the work a second title that names the method LinEAS. Steering can help make outputs safer or let users try style changes.

Older steering maps are tuned one layer at a time and can cause unwanted shifts in new cases. The new method trains across all layers at once with a single loss, and it can pick which neurons to change. It needs only a few unpaired examples to work. Tests on text and image models beat similar methods on reducing toxic output and on adding new concepts.

Catalog record

CREATORPau Rodríguez ∗, Michal Klein, Eleonora Gualdoni, Arno Blaas, Luca Zappella, Marco Cuturi, Xavier Suau ∗
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10679

Browse the full library · About the Tower of Babel