Bilinear autoencoders find interpretable manifolds

Thomas Dooms, Ward Gauderis, Geraint Wiggins, Jose Oramas · 2026

Read Bilinear autoencoders find interpretable manifolds by Thomas Dooms, Ward Gauderis, Geraint Wiggins, Jose Oramas (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08891.txt.

Description

Thomas Dooms and three coauthors stretch a key tool for reading model guts from flat traits to curved sheets. Sparse autoencoders split model acts into traits folks can grasp. But key ideas can live on curved sheets of more than one size. Flat word lists fit such sheets poorly. Their bilinear form uses square latent terms. Those terms join cleanly in weight space. They allow shape checks that do not rest on one input.

Tests show curved shapes are in fact common. The new latents catch them and cut rebuild error in language models. Unlike priors still find the same input span. A live viewer shows sheet finds on a live model. The tool gives curved traits that stay in reach of math.

Catalog record

CREATORThomas Dooms, Ward Gauderis, Geraint Wiggins, Jose Oramas
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08891

Browse the full library · About the Tower of Babel