Vision Language Models Cannot Reason About Physical Transformation

Dezhi Luo, Yijiang Li, Maijunxian Wang, Tianwei Zhao, Bingyang Wang, Siheng Wang, Pinyuan Feng, Pooyan Rahmanzadehgervi, Ziqiao Ma, Hokin Deng · 2026

Read Vision Language Models Cannot Reason About Physical Transformation by Dezhi Luo, Yijiang Li, Maijunxian Wang, Tianwei Zhao, Bingyang Wang, Siheng Wang, Pinyuan Feng, Pooyan Rahmanzadehgervi, Ziqiao Ma, Hokin Deng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-07109.txt.

Description

Luo and colleagues test whether vision-language models truly grasp physical transformation, using ConservationBench. The benchmark spans four conserved properties with paired conserving and non-conserving scenes, yielding 23,040 questions across 112 models. Results show systematic failure near chance level, and gains on conservation tasks come with drops on controls. Neither sharper time resolution, prompting, nor curated sampling helps, and text priors favoring invariance do not rescue visual reasoning.

Catalog record

CREATORDezhi Luo, Yijiang Li, Maijunxian Wang, Tianwei Zhao, Bingyang Wang, Siheng Wang, Pinyuan Feng, Pooyan Rahmanzadehgervi, Ziqiao Ma, Hokin Deng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53119
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.07109

Browse the full library · About the Tower of Babel