Improving Relative Representations with Learned Anchors and Whitened Inner Products
Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship · 2026
Read Improving Relative Representations with Learned Anchors and Whitened Inner Products by Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30596.txt.
Description
Svendsen and colleagues tackle a barrier to modular AI: separately trained models settle into incompatible internal coordinate systems. Relative representations bridge models by mapping coordinates to similarities against shared anchor points. But random anchors and cosine similarity miss the stretched geometry of modern architectures. Learning anchors as robust semantic prototypes and using a geometry-aware similarity that keeps magnitude information and ignores affine shifts, they enable near-lossless transfer and stable zero-shot communication between unlike models. The approach works even for small language models of different sizes.
Catalog record
| CREATOR | Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41161 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30596