InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoning
Gautam Sreekumar, Vishnu Naresh Boddeti · 2025
Read InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoning by Gautam Sreekumar, Vishnu Naresh Boddeti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-12263.txt.
Introduction
Big multimodal models can recite physics laws they saw in training, but can they learn a new law from a few examples on the spot? Gautam Sreekumar and one co-author test exactly that in 2025 with InPhyRe, a benchmark of synthetic collision videos paired with questions about what happens next.
More than thirteen open and commercial models were examined. The verdict is weak across the board: models fumble even familiar laws, reason poorly when the governing law was never in training, and often lean on language cues while ignoring the video. The authors frame the gap as a trust problem for safety-critical uses.
Catalog record
| CREATOR | Gautam Sreekumar, Vishnu Naresh Boddeti |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 111 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-12263
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoning Gautam Sreekumar, Vishnu Naresh Boddeti Abstract Large multimodal models (LMMs) encode universal physical laws observed during training, such as momentum conservation, as parametric knowledge . It allows LMMs to answer physical reasoning queries, such as the outcome of a potential collision event from visual input. However, since parametric knowledge includes only the physical laws seen during training, it is insufficient for reasoning when the inference scenario violates these physical laws. In contrast, humans possess the skill to adapt their physical reasoning to unseen physical environments from a few visual examples. This ability, which we refer to as inductive physical reasoning , is indispensable for LMMs if they are to replace human agents in safety-critical applications. Despite its importance, existing visual benchmarks evaluate only the parametric knowledge in LMMs, and not inductive physical reasoning. To this end, we propose InPhyRe , the first visual question answering benchmark to measure inductive physical reasoning in LMMs. InPhyRe evaluates LMMs on their ability to predict the outcome of collision events in algorithmically generated synthetic collision videos. By inspecting 13 LMMs, InPhyRe informs us that (1) LMMs struggle to apply their limited parametric knowledge about universal physical laws to reasoning, (2) inductive physical reasoning in LMMs is weak when demonstration samples violate universal physical laws, and (3) inductive physical reasoning in LMMs suffers from language bias and largely ignores the visual inputs, questioning the trustworthiness of LMMs regarding visual inputs. 1 Introduction Case Study A large multimodal model (LMM) is used to determine whether a car crash will occur on a snowy road from a video. To ensure the LMM understands that the physical coefficients of the snowy road differ from those of a dry road, a few demonstration videos of snowy roads with and without collisions are provided as context. The LMM predicts that a crash is unlikely. Question : Did the LMM account for the unseen physical coefficients using the demonstration videos, or did it use only its parametric physical knowledge to make its prediction? Large multimodal models (LMMs) are known to encode universal physical laws ( e.g. , momentum conservation) observed during training as parametric knowledge to answer physical reasoning queries ( e.g. , whether a collision occurs or not) from visual input [ 16 , 20 , 49 ] . However, since parametric knowledge includes only the physical laws seen during training, it is insufficient in scenarios that violate these observed physical laws, such as the case study of the snowy road. In contrast, humans would easily adapt their physical knowledge about collisions to snowy road conditions with the help of demonstration videos to predict any collision, if presented with the same case study. This crucial skill, that we refer to as inductive physical reasoning 1 1 1 Inductive physical reasoning is closer to inductive reasoning than general visual reasoning. See Appendix D . , is a hallmark of intelligence that humans develop at a very young age [ 29 , 56 ] . Inductive physical reasoning is an indispensable ability that LMMs must possess in addition to parametric knowledge if they are to be deployed in safety-critical applications such as autonomous driving [ 78 , 75 ] to replace human agents. Figure 1 : ( Left ) A large multimodal model (LMM) is asked to predict the change in vertical velocity of an object colliding with a vertical wall. The model will output “possibility 2” if it uses its parametric knowledge that encodes the universal physical laws (in this case, the momentum conservation principle). However, parametric knowledge would be insufficient if the collision event violated the physical laws encoded in the model. For the model to infer the underlying physical laws, we provide the model with exemplar videos of collisions that violate the momentum conservation principle. The model may now rely on its inductive physical reasoning capabilities to generate “possibility 1”. ( Right ) InPhyRe shows that LMMs struggle with inductive physical reasoning. Despite its vital nature, there are no visual benchmarks that quantitatively evaluate inductive physical reasoning in LMMs. Existing benchmarks [ 9 , 8 , 61 , 17 , 47 ] evaluate the parametric knowledge of LMMs, particularly about universal physical concepts observable in natural videos, such as friction and gravity. However, creating benchmarks to evaluate inductive physical reasoning is challenging. The primary challenge is the inherent impossibility of finding natural videos that violate universal physical laws. It is also prohibitively expensive to manually edit and repurpose existing benchmarks that measure parametric knowledge to evaluate inductive physical reasoning. Thus, algorithmically generated synthetic video datasets become the only viable option. Algorithmic generation of physical events violating universal physical laws still requires manual interventions on environment design, modified physical laws, and object trajectories, limiting the scalability of any such benchmark. We propose InPhyRe ( In ductive Phy sical Re asoning), the first visual question answering benchmark that evaluates how well LMMs can make predictions in scenarios that violate universal physical laws by inferring the underlying physics from demonstration samples. InPhyRe comprises algorithmically generated synthetic videos of collision events. During evaluation, LMMs are given the first frame of a video and asked questions about the outcome of a textually described collision involving the objects in this frame. However, some of these scenarios violate universal physical laws. In those scenarios, LMMs must infer the underlying physics from demonstration videos taken from the same scenarios and use it to make their predictions. InPhyRe quantifies inductive physical reasoning in LMMs as the performance disparity between scenarios that follow the true physical laws and those that do not. Why use impossible scenarios? With InPhyRe , our goal is not to enhance the utility of LMMs in unrealistic scenarios that violate universal physical laws, but rather to enhance their adaptability to scenarios with physical laws unseen during training. However, since we cannot always know the training scenarios observed by LMMs, we build our benchmark using physically impossible scenarios that are less likely to have been observed by LMMs during training. Scope of InPhyRe : Since it is difficult to create a benchmark visualizing the violations of every universal physical law exhaustively, we limit our studies to the laws of mechanics, such as momentum and energy conservation principles. Nonetheless, the outcomes of the collision events in InPhyRe violate the most fundamental laws of mechanics. Therefore, we believe that the conclusions from InPhyRe are likely valid for inductive physical reasoning in LMMs about other branches of physics. Our results show that InPhyRe is a formidable benchmark for current LMMs, despite being a synthetic dataset. Our results indicate that LMMs struggle to infer and utilize unseen physical laws from demonstration samples ( Fig. 1 ). From empirical evidence, we conjecture that LMMs do not understand physical laws as transferrable mathematical models, but rather as a fixed set of rules that all objects obey. Moreover, we show that LMMs primarily derive their inductive physical reasoning capabilities from their language components and ignore the visual inputs in the demonstration samples. This means that the LMM in our case study probably did not account for the snowy conditions. Contributions and Findings ✦ We introduce InPhyRe , the first visual question answering benchmark