GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks

Ryan Spencer 1 Roey Yaari 1 Ritvik Vemavarapu 1 Joyce Yang 1 Steven Ngo 1,2 Utkarsh Sharma 1,3 · 2025

Read GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks by Ryan Spencer 1 Roey Yaari 1 Ritvik Vemavarapu 1 Joyce Yang 1 Steven Ngo 1,2 Utkarsh Sharma 1,3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22207.txt.

Introduction

Ryan Spencer and five co-authors present GamiBench, a benchmark that tests whether multimodal AI models can truly reason about folding paper. Models handle perception and instructions well but struggle to track objects across views and steps. Origami makes an ideal test: a flat crease pattern must be followed mentally through a sequence into a 3D shape.

GamiBench holds 186 regular and 186 impossible crease patterns, each paired with folded shapes shown from six viewpoints. Its three tasks ask models to predict folded forms, judge valid viewpoints, and detect impossible patterns. New metrics track viewpoint consistency and how often models select impossible folds. Even leading systems such as GPT-5 and Gemini-2.5-Pro struggled with single-step spatial understanding in the authors' tests.

Catalog record

CREATORRyan Spencer 1 Roey Yaari 1 Ritvik Vemavarapu 1 Joyce Yang 1 Steven Ngo 1,2 Utkarsh Sharma 1,3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-22207

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks Ryan Spencer 1 Roey Yaari 1 Ritvik Vemavarapu 1 Joyce Yang 1 Steven Ngo 1,2 Utkarsh Sharma 1,3 Abstract Multimodal large language models (MLLMs) are proficient in perception and instruction-following, but they still struggle with spatial reasoning: the ability to mentally track and manipulate objects across multiple views and over time. Spatial reasoning is a key component of human intelligence, but most existing benchmarks focus on static images or final outputs, failing to account for the sequential and viewpoint-dependent nature of this skill. To close this gap, we introduce GamiBench , a benchmark designed to evaluate spatial reasoning and 2D-to-3D planning in MLLMs through origami-inspired folding tasks. GamiBench includes 186 regular and 186 impossible 2D crease patterns paired with their corresponding 3D folded shapes, produced from six distinct viewpoints across three visual question-answering (VQA) tasks: predicting 3D fold configurations, distinguishing valid viewpoints, and detecting impossible patterns. Unlike previous benchmarks that assess only final predictions, GamiBench holistically evaluates the entire reasoning process of the models; measuring cross-view consistency, physical feasibility through impossible-fold detection and interpretation of intermediate folding steps. It further introduces new diagnostic metrics—viewpoint consistency (VC) and impossible fold selection rate (IFSR)—to measure how well models handle folds of varying complexity. By linking geometric evaluation with sequential reasoning, GamiBench enables a comprehensive evaluation of state-of-the-art MLLMs, revealing significant limitations in spatial reasoning capabilities, such as multi-view inconsistency and difficulty detecting physically impossible folds. Our experiments show that even leading models such as GPT-5 and Gemini-2.5-Pro struggle on single-step spatial understanding, while other MLLMs tend to show highly variable or inconsistent answering trends. These contributions establish a standardized framework for evaluating and advancing geometric understanding and spatial reasoning in MLLMs. The dataset and code is available at: https://github.com/stvngo/GamiBench . 1 Introduction Figure 1: 2D normal crease pattern (top-left), right-view 3D final fold state (top-right), top-view alternative 3D final fold state (bottom-left), and 2D impossible crease pattern (bottom-right) of an example butterfly variation ( komatsu_butterfly ). Impossible crease pattern has inverted creases and missing valley/mountain folds. Spatial reasoning is a fundamental component of human intelligence, crucial for interacting with the physical world, understanding relationships between objects, and executing multi-step actions. Tasks such as building furniture or folding origami require mentally simulating spatial transformations and tracking changing object states. As the interactions between artificial intelligence systems and real-world environments evolve, developing models that can reason about space and change has become a core challenge. Recent advances in multimodal large language models (MLLMs) show strong progress in image recognition, VQA, and instruction following ( 4 ) , ( 10 ) ; however, they clearly struggle on high-quality and temporally extended sequential spatial reasoning tasks ( 19 ) , ( 20 ) , ( 21 ) . Existing multimodal benchmarks and tools have advanced evaluation breadth (e.g., perception, captioning, instruction following) ( 5 ) , ( 14 ) , but typically emphasize single-step judgments, static scenes, or end-state accuracy ( 5 ) . Specialized efforts in spatial or procedural reasoning test important skills such as geometric consistency ( 20 ) , ( 19 ) , arrangement from descriptions ( 20 ) , or plan generation ( 21 ) , but often isolate narrow capabilities, fix visual formats, or under-specify multi-view consistency and temporal coherence. Thus, currently there is no framework that holistically evaluates how models plan, update, and validate spatial states across time and across multiple viewpoints of the same 3D structure. In this work, we introduce GamiBench , a novel benchmark and framework for sequential spatial planning and reasoning that uses origami-inspired tasks to map 2D crease patterns to 3D multi-view final states (front, back, top, bottom, left, right). We take inspiration from the common traditional Japanese art of paper folding, Origami, using a single square sheet of paper to create a figure without cutting, gluing, or marking. The assembly of the 3D Origami model consists of dozens of intermediary discrete folds to transform a 2D paper plane into a 3D structure, providing an ideal foundation for testing sequential spatial reasoning abilities. We carefully curate a dataset of 186 regular and 186 impossible 2D square crease pattern configurations and 372 adjacent 3D folds, where each fold is part of a pair of correct folds for a given 2D crease pattern, using existing online tools like Oriedita ( 17 ) and Origami Simulator ( 6 ) . GamiBench encompasses a collection of 744 Visual Question-Answering (VQA) multiple-choice sets spanning across 3 tasks, one of which is conditioned on the final answer, from this dataset. First, we develop a set of tests to assess MLLMs’ basic spatial transformation understanding capabilities, including single-step 2D-to-3D mapping, reinforcement of initial correctness via alternative 3D final states, and assessment of impossible fold configuration acknowledgement. We develop our own metrics, such as Viewpoint Consistency (VC) and Impossible Fold Selection Rate (IFSR), to measure model performance across these 3 tasks. GamiBench provides several distinct advantages compared to existing spatial understanding benchmarks by evaluating (1) multi-perspective consistency across 3D views, (2) feasibility via impossible-fold detection (violations of physical origami axioms), and (3) sequential interpretation of intermediate states induced by textual instructions and visual transitions. Furthermore, we classify folds by complexity to stress-test models’ robustness to geometric density. Leveraging GamiBench , we conduct evaluations on 21 state-of-the-art MLLMs, including proprietary models such as GPT-5 and Gemini-2.5-Flash, as well as leading open-source alternatives such as Llama-4-Maverick, Gemma-3-27B-IT, Cogito-V2-Preview-Llama-109B-MoE, and GLM-4.5V. Our results reveal a substantial gap in MLLM spatial reasoning proficiency. Even the strongest models struggle with basic spatial understanding tasks, at times performing worse than weaker models on the same task. Concretely, our contributions are as follows: • A multi-view, sequential spatial benchmark that evaluates 2D-to-3D reasoning beyond final-state accuracy, with tasks that require coherence across six views and across time. • New evaluation axes—Viewpoint Consistency and Impossible Fold Selection Rate (IFSR)—that diagnose failure modes missed by standard accuracy metrics. • Origami-inspired task suite combining textual instructions with visual states to test whether MLLMs can plan, update, and verify geometric transformations. • Complexity controls (simple vs. complex crease patterns) enabling analysis of scale effects on spatial planning. Together, these contributions to spatial evaluation from static recognition toward procedural, multi-view reasoning provide clearer signals about where current MLLMs fall short and how future architectures or training regimes might improve. 2 Related Work Various comprehensive benchmarks have been introduced to assess various multimodal capabilities. Foundations of Spatial and Embodied Reasoning. Spatial intelligence has long been identified as a fundamental component of human cognition, underlying reasoning about geometry and physical relationships ( 3 ) . Recent embodied MLLM frameworks extend this concept to machine perception and control. OpenVLA ( 11 ) links vision, language, and action to achieve generalist visuomotor manipulation across diverse robots, while ManipLLM ( 13 ) integrates affordance reasoning and pose prediction for object-centric manipulation in real environments. In autonomous driving, DriveMLLM ( 7 ) benchmarks spatial scene understanding under occlusion and dynamic layouts, and DriveMLM ( 22 ) aligns multimodal perception with behavioral-planning states for closed-loop navigation. Follow-up studies reveal that modality imbalance limits generalization from simple to complex visual reasoning tasks ( 18 ) , and that MLLMs

Read the complete text