Supervised sparse auto-encoders as unconstrained feature models for semantic composition

Ouns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao · 2026

Read Supervised sparse auto-encoders as unconstrained feature models for semantic composition by Ouns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00924.txt.

Description

Ouns El Harzli and colleagues supervise decoder-only sparse auto-encoders to rebuild feature vectors by jointly learning sparse concept vectors and decoder weights. Sparse auto-encoders face non-smooth L1 penalty problems hurting rebuilds and scaling, plus mismatch between learned features and human meanings. They adapt free feature models from neural collapse theory and supervise the task. On Stable Diffusion 3.5, the approach shows compositional generalization, rebuilding images with unseen concept mixes and enabling feature-level edits for meaning-based image editing without prompt changes.

Catalog record

CREATOROuns El Harzli, Hugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35217
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00924

Browse the full library · About the Tower of Babel