Positional Encodings Anchor Spatial Structure in Vision Transformers: A Geometric Perspective on Robustness
Mahmoud Mannes · 2026
Read Positional Encodings Anchor Spatial Structure in Vision Transformers: A Geometric Perspective on Robustness by Mahmoud Mannes (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00124.txt.
Description
Mannes asks how place codes shape what a vision model learns inside. Vision models split an image into tiles and add a code for each tile's spot. Past work showed these codes change scores and strength. Their role in the inner map of space was less clear.
Mannes builds a score that tracks how much space form stays in the tile codes. Models trained with no place codes still form some space map. That map rests on image content and breaks when tiles are shuffled. All three code types he tests tie the map to tile index instead. Such models stay firm when content is scrambled. Strength seems to come from a fixed frame, not from one code type.
Catalog record
| CREATOR | Mahmoud Mannes |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00124