Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)

Michał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland · 2026

Read Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE) by Michał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18629.txt.

Description

Michał Brzozowski and Neo Christopher Chung propose, in a 2026 paper, aligned training. It is a parameter-free redo of sparse autoencoders that lifts feature quality and stability. The method sets a geometric rule: encoder-decoder inner products equal one for every feature, cutting a degeneracy source with no new hyperparameters. The idea comes from alignment scores forming a bimodal spread across modern architectures. Across models, dictionary sizes, and sparsity levels it shows Pareto gains on SAEBench, killing dead units and firming training seeds.

Catalog record

CREATORMichał Brzozowski, Neo Christopher Chung, [0.6em] Samsung AI Center, Warsaw, Poland
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30984
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18629

Browse the full library · About the Tower of Babel