Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
Michał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland · 2026
Read Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE) by Michał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18629.txt.
Description
Michał Brzozowski and Neo Christopher Chung propose, in a 2026 paper, aligned training. It is a parameter-free redo of sparse autoencoders that lifts feature quality and stability. The method sets a geometric rule: encoder-decoder inner products equal one for every feature, cutting a degeneracy source with no new hyperparameters. The idea comes from alignment scores forming a bimodal spread across modern architectures. Across models, dictionary sizes, and sparsity levels it shows Pareto gains on SAEBench, killing dead units and firming training seeds.
Catalog record
| CREATOR | Michał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30984 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18629