Computer Vision Models Show Human-Like Sensitivity to Geometric and Topological Concepts
Zekun Wang ([email protected]), Sashank Varma ([email protected]) · 2025
Read Computer Vision Models Show Human-Like Sensitivity to Geometric and Topological Concepts by Zekun Wang ([email protected]), Sashank Varma ([email protected]) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13281.txt.
Introduction
Zekun Wang and Sashank Varma test, in a 2025 paper, whether computer vision models share the human knack for seeing geometric and topological ideas, like holes, symmetry, and inside versus outside. One theory says children are born with core knowledge for such concepts, wired into dedicated circuits. The rival view: the concepts are learned free from everyday seeing.
Vision models, which learn only from images, are a clean test of the second view. Three model families were tried on an odd one out task covering forty three concepts in seven classes. Transformer based models scored highest, beating even young children's accuracy. More telling, they found the same concept classes easy and hard that children do. Language vision models lagged in alignment. If a model with no innate geometry module grows human like sensitivities from pictures alone, the case for learned core knowledge gains real weight.
Catalog record
| CREATOR | Zekun Wang ([email protected]), Sashank Varma ([email protected]) |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13281
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Computer Vision Models Show Human-Like Sensitivity to Geometric and Topological Concepts Zekun Wang ([email protected]), Sashank Varma ([email protected]) Abstract With the rapid improvement of machine learning (ML) models, cognitive scientists are increasingly asking about their alignment with how humans think. Here, we ask this question for computer vision models and human sensitivity to geometric and topological (GT) concepts. Under the core knowledge account, these concepts are innate and supported by dedicated neural circuitry. In this work, we investigate an alternative explanation, that GT concepts are learned “for free” through everyday interaction with the environment. We do so using computer visions models, which are trained on large image datasets. We build on prior studies to investigate the overall performance and human alignment of three classes of models – convolutional neural networks (CNNs), transformer-based models, and vision-language models – on an odd-one-out task testing 43 GT concepts spanning seven classes. Transformer-based models achieve the highest overall accuracy, surpassing that of young children. They also show strong alignment with children’s performance, finding the same classes of concepts easy vs. difficult. By contrast, vision-language models underperform their vision-only counterparts and deviate further from human profiles, indicating that naïve multimodality might compromise abstract geometric sensitivity. These findings support the use of computer vision models to evaluate the sufficiency of the learning account for explaining human sensitivity to GT concepts, while also suggesting that integrating linguistic and visual representations might have unpredicted deleterious consequences. Keywords: geometric reasoning; mathematical cognition; cognitive alignment; computer vision models; vision-language models Introduction Humans learn geometric and topological (GT) concepts through mathematics instruction. Long before they begin formal school, however, they show sensitivity to concepts such as shape, angle, rotation, and translation, leading to the proposal that they are part of core knowledge , i.e., innate and supported by dedicated neural circuitry Spelke & Kinzler (2007) . This may be the consequence of evolutionary processes building in sensitivity to the mathematical and spatial structure of the world, for example to support navigation Gibson (1979) ; Shepard (1994) . Supporting evidence comes from the finding of similar sensitivities in non-human animals Chiandetti & Vallortigara (2007) ; Pasupathy & Connor (1999) ; Vallortigara (2018) . The core knowledge account “explains” the emergence of GT concepts in young children who have not yet entered school and received formal mathematics instruction. A contrasting account is that GT concepts are learned “for free”, through everyday experience in the world. This account has been less studied, perhaps because a strong empirical test would require “ablation” studies where organisms are deprived of a typically rich environment in which to learn, raising ethical questions even for animal subjects. The recent rapid rise of deep learning models provides a new way to test the sufficiency of the learning account. These models are increasingly demonstrating human-level performance in domains ranging from language acquisition Ma, Wang & Chai (2024) ; Ma et al. (2023) ; Chang & Bergen (2022) to theory of mind Ma, Sansom et al. (2024) ; Jung et al. (2024) ; Y. He et al. (2023) to mathematical reasoning Shah et al. (2023) ; Ahn et al. (2024) to concept understanding Jin et al. (2024) ; Vemuri et al. (2024) . In addition to their fidelity at the behavioral level, there is also increasing evidence of their fidelity at the brain level. Researchers have mapped the layers of convolutional neural network (CNN) models to areas of the human visual system, specifically the ventral visual stream K. He et al. (2015) ; Simonyan & Zisserman (2015) ; Krizhevsky et al. (2012) ; Jacob et al. (2021) ; De Cesarei et al. (2021) ; Lindsay (2021) . Portelance (2022) argues that despite differences in the underling learning processes of humans and ML models, we can use these models to derive new hypotheses about human cognition, and conversely we can use cognitiv (neuro)science data to train more human-like models. We adopt a similar view. The current study evaluates the sufficiency of the account that GT concepts are learned through experience in the world. It builds on prior studies that have used CNN models and found promising initial results. It replicates their findings and extends their approach to two new and important classes of computer vision models. The first is vision transformer models, which are notable not just for their different architecture but also for their larger size and ability to be trained on more data. The second class is vision-language models, which learn about both the visual-spatial structure of the environment and also its linguistic structure, and the mapping between the two. These models are of potential cognitive interest given that dual-coding theory Paivio (1991) suggests that processing information through both non-verbal and verbal channels should be advantageous. We compare these three classes of ML models to humans in three research questions: 1. Do CNNs, transformers, and/or vision-language models approach (or even exceed) the sensitivity to GT concepts shown by children and adults? 2. Which models / architectures best align with human performance, i.e., find the same classes of concepts easy vs. difficult. Is their alignment close enough to consider them viable cognitive science models? 3. How does adding language (i.e., symbols) to vision (i.e., space) affect the overall sensitivity and class-by-class alignment of vision-language models? Related Work GT concept understanding in humans Dehaene et al. (2006) conducted a pioneering and comprehensive study of human sensitivity to GT concepts. They developed a purely visual task that was appropriate even for participants with no formal mathematics education. In this task, each stimulus consists of six images. Five images embody a particular GT concept but the sixth does not. The images otherwise vary in their visual features. The participant is simply asked to indicate the “odd one out”. They developed stimuli for 43 individual concepts spanning seven broader classes: Topology, Euclidean Geometry, Geometrical Figures, Symmetrical Figures, Chiral Figures, Metric Properties, and Geometrical Transformations. See Figure 1 for example stimuli for 7 concepts. Dehaene et al. (2006) found that the Mundurucu, an indigenous Amazonian group whose members have no formal schooling, were nevertheless sensitive to 39 of the 43 concepts. Moreover, the Mundurucu adults and children performed comparably. Marupudi & Varma (2023) replicated these findings using a modified 2 alternative forced-choice version of the odd-one-out task, and furthermore showed that cognitive ability in general (i.e., fluid intelligence) and visuospatial ability in particular (i.e., mental rotation) explained only a small portion of the variability in people’s performance. Izard & Spelke (2009) extended the Dehaene et al. (2006) study to people from the US. A distinctive contribution was testing both young children (ages 3–6 years) and adults. The young children also displayed sensitivity to GT concepts, displaying above-chance performance for 27 of the 43 concepts. The finding of sensitivity to GT concepts in samples that have had no formal schooling has been generally interpreted as evidence for the core knowledge account. Interestingly, the three studies found similar variation in the relative difficulty of different GT concepts. For example, participants from all groups – Mundurucu and Western, children and adults – were highly accurate for Euclidean Geometry concepts, but found Geometrical Transformations to be difficult. Capturing this variability is an important goal. Figure 1: Example stimuli from the 7 categories