Robust Representation Learning in Masked Autoencoders

Anika Shrivastava, Renu Rameshan, and Samar Agnihotri · 2026

Read Robust Representation Learning in Masked Autoencoders by Anika Shrivastava, Renu Rameshan, and Samar Agnihotri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03531.txt.

Description

Masked autoencoders learn strong image features by rebuilding hidden patches, but why their features classify so well was unclear. Anika Shrivastava and 2 co-authors probe those internal representations in a 2026 paper. What began as a study of downstream accuracy turned into a discovery: the learned features classify well even under blur and occlusions.

A layer-by-layer look shows the model builds a class-aware latent space, with different classes settling into ever more separable subspaces. The model also shows early, lasting global attention, unlike standard vision transformers. Two new sensitivity gauges measure how embeddings hold up under damage. Together the results explain why masked autoencoders classify so robustly.

Catalog record

CREATORAnika Shrivastava, Renu Rameshan, and Samar Agnihotri
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03531

Browse the full library · About the Tower of Babel