SOSAE: Self-Organizing Sparse Autoencoders
Sarthak Ketanbhai Modi, Zi Pong Lim, Yushi Cao, Yupeng Cheng, Yon Shin Teo, Shang-Wei Lin · 2025
Read SOSAE: Self-Organizing Sparse Autoencoders by Sarthak Ketanbhai Modi, Zi Pong Lim, Yushi Cao, Yupeng Cheng, Yon Shin Teo, Shang-Wei Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04644.txt.
Description
Sarthak Modi and co-authors present an autoencoder that sizes its own inner feature space while it trains, in a 2025 paper. Autoencoders compress data into a short code, and picking the code's length by trial takes much time and computing power. Their design borrows ideas from physics to let the space shrink on its own.
A penalty that weighs both the size and the position of each feature pushes unused parts of the code toward zero. The dead part can then be cut with no loss of information. In tests on varied data sets, the method matched the tuning quality of rival tools while using up to 130 times fewer operations to find the right size.
Catalog record
| CREATOR | Sarthak Ketanbhai Modi, Zi Pong Lim, Yushi Cao, Yupeng Cheng, Yon Shin Teo, Shang-Wei Lin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04644