Improving Relative Representations with Learned Anchors and Whitened Inner Products

Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship · 2026

Read Improving Relative Representations with Learned Anchors and Whitened Inner Products by Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30596.txt.

Description

Svendsen and colleagues tackle a barrier to modular AI: separately trained models settle into incompatible internal coordinate systems. Relative representations bridge models by mapping coordinates to similarities against shared anchor points. But random anchors and cosine similarity miss the stretched geometry of modern architectures. Learning anchors as robust semantic prototypes and using a geometry-aware similarity that keeps magnitude information and ignores affine shifts, they enable near-lossless transfer and stable zero-shot communication between unlike models. The approach works even for small language models of different sizes.

Catalog record

CREATOROscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, Hiba Nassar, [0.5em] These authors contributed equally to this work, share first authorship
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41161
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30596

Browse the full library · About the Tower of Babel