Robust Representation Learning in Masked Autoencoders
Anika Shrivastava, Renu Rameshan, and Samar Agnihotri · 2026
Read Robust Representation Learning in Masked Autoencoders by Anika Shrivastava, Renu Rameshan, and Samar Agnihotri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03531.txt.
Description
Masked autoencoders learn strong image features by rebuilding hidden patches, but why their features classify so well was unclear. Anika Shrivastava and 2 co-authors probe those internal representations in a 2026 paper. What began as a study of downstream accuracy turned into a discovery: the learned features classify well even under blur and occlusions.
A layer-by-layer look shows the model builds a class-aware latent space, with different classes settling into ever more separable subspaces. The model also shows early, lasting global attention, unlike standard vision transformers. Two new sensitivity gauges measure how embeddings hold up under damage. Together the results explain why masked autoencoders classify so robustly.
Catalog record
| CREATOR | Anika Shrivastava, Renu Rameshan, and Samar Agnihotri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03531