Human-Like Coarse Object Representations in Vision Models

Andrey Gizdov, Andrea Procopio, Yichen Li, Daniel Harari, Tomer Ullman, [0.3em] Harvard University · 2026

Read Human-Like Coarse Object Representations in Vision Models by Andrey Gizdov, Andrea Procopio, Yichen Li, Daniel Harari, Tomer Ullman, [0.3em] Harvard University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12486.txt.

Description

Humans reason about physics using coarse, volumetric bodies that smooth concavities, trading fine detail for efficient prediction. Andrey Gizdov led this work with six co-authors, asking whether segmentation models, trained for pixel-accurate masks, nonetheless acquire human-like object representations. The structure of human bodies remains largely unknown, so the team built a pipeline and alignment metric to compare model and human representations.

Tests use time-to-collision and change detection tasks with data from 178 and 50 human participants, across architectures including DINOv2, SegFormer, DeepLabV3+, and UPerNet, varying size and training time. Briefly trained models segment too coarsely and align poorly, while fully trained models segment too finely. Each model has an intermediate regime that best matches human bodies, reached earlier by larger models, suggesting these bodies emerge under resource constraints in general-purpose vision models.

Catalog record

CREATORAndrey Gizdov, Andrea Procopio, Yichen Li, Daniel Harari, Tomer Ullman, [0.3em] Harvard University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-12486

Browse the full library · About the Tower of Babel