Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models
Arnas Uselis, Andrea Dittadi, Seong Joon Oh · 2026
Read Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models by Arnas Uselis, Andrea Dittadi, Seong Joon Oh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-24264.txt.
Description
Arnas Uselis and two co-authors argue that compositional generalization in vision embedding models requires linear, orthogonal representations. The 2026 paper defines compositional generalization as recognizing familiar parts in novel combinations.
The authors formalize, for compositional generalization under standard training, three desiderata: divisibility, transferability, and stability. They show necessary geometric constraints follow: per-concept components must combine linearly to form representations. That result grounds the linear representation hypothesis. Tests across modern vision models find factors that are partly linear, low-rank, and near-orthogonal per concept. Dimension bounds link the number of composable concepts to embedding geometry.
Catalog record
| CREATOR | Arnas Uselis, Andrea Dittadi, Seong Joon Oh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 168 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-24264