Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers

Felipe Parodi, Jordan K. Matelsky, Melanie Segado, [4pt] University of Pennsylvania · 2026

Read Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers by Felipe Parodi, Jordan K. Matelsky, Melanie Segado, [4pt] University of Pennsylvania (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14433.txt.

Description

Parodi, Matelsky, and Segado challenge the claim that registers are indispensable in DINO vision transformers. Zeroing registers in DINOv2 and DINOv3 slashes scores, with classification falling up to 36.6 points, yet three gentler swaps hold steady: mean substitution, noise substitution, and shuffling registers across images. Those swaps stay within about a point of baseline on classification, correspondence, and segmentation, even though cosine checks confirm they truly perturb internal states. Zeroing alone, the team argues, overstates dependence on exact register content, since plausible register-like activity matters more than image values.

Catalog record

CREATORFelipe Parodi, Jordan K. Matelsky, Melanie Segado, [4pt] University of Pennsylvania
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41998
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14433

Browse the full library · About the Tower of Babel