Supervised sparse auto-encoders as unconstrained feature models for semantic composition
Ouns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao · 2026
Read Supervised sparse auto-encoders as unconstrained feature models for semantic composition by Ouns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00924.txt.
Description
Ouns El Harzli and colleagues supervise decoder-only sparse auto-encoders to rebuild feature vectors by jointly learning sparse concept vectors and decoder weights. Sparse auto-encoders face non-smooth L1 penalty problems hurting rebuilds and scaling, plus mismatch between learned features and human meanings. They adapt free feature models from neural collapse theory and supervise the task. On Stable Diffusion 3.5, the approach shows compositional generalization, rebuilding images with unseen concept mixes and enabling feature-level edits for meaning-based image editing without prompt changes.
Catalog record
| CREATOR | Ouns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35217 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00924