The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang · 2026

Read The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space by Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09883.txt.

Introduction

Hu and six colleagues expose a shortcut behind strong vision-reasoning scores. Most benchmarks lay tasks out on orthogonal grids, which models can quietly convert into text coordinates and solve by deduction rather than sight. The authors call this the Cartesian Shortcut and ask whether high scores reflect real visual understanding at all. Their answer is a new benchmark, Polaris-Bench.

Polaris-Bench restates 53 visual reasoning tasks in polar coordinates, with paired Cartesian versions as controls, keeping the logic identical. Fourteen leading multimodal models that scored seventy to eighty-three percent on Cartesian layouts fell to thirty-one to thirty-nine percent on polar equivalents. Reasoning gains seen on Cartesian tasks largely vanished too, pointing to a missing skill: topology-invariant visual reasoning.

Catalog record

CREATORXia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09883

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang Abstract As current Multimodal Large Language Models rapidly saturate canonical visual reasoning benchmarks, a key question emerges: do these strong scores genuinely reflect robust visual understanding? We identify a pervasive vulnerability, the Cartesian Shortcut : visual reasoning benchmarks prevalently build on orthogonal grid-based layouts that can be readily discretized into explicit textual coordinates. Models systematically exploit this property, heavily leveraging text-based deductive reasoning to assist visual problem-solving. To systematically dismantle this shortcut, we introduce Polaris-Bench , which re-formulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts as reference, while preserving consistent logical constraints and task semantics—thus fundamentally breaking the orthogonal prior that models exploit. Comprehensive evaluation across 14 14 state-of-the-art MLLMs reveals that frontier models achieving 70 70 – 83 % 83\% on Cartesian layouts collapse to 31 31 – 39 % 39\% on Polar equivalents, with degradation persisting even under complete logical equivalence. Moreover, reasoning gains observed on Cartesian layouts are severely diminished on Polar equivalents. These findings expose a critical deficiency in current MLLMs: the lack of topology-invariant visual reasoning. \uselogo \reportnumber 0001 1 Introduction Recent Multimodal Large Language Models (MLLMs) [ 16 , 27 , 2 , 24 ] achieve remarkable, often human-level performance across canonical visual reasoning benchmarks [ 41 , 6 , 18 , 30 ] . However, as these datasets rapidly saturate, a critical question emerges: have models genuinely acquired visual reasoning abilities, or are they merely exploiting text-based heuristics [ 4 , 38 , 8 , 20 ] ? In this study, we expose a pervasive systemic vulnerability that creates an illusion of profound visual capability: the Cartesian Shortcut . Figure 1: Illustration and empirical evidence of the Cartesian Shortcut. (Top) An equivalent sudoku question in Cartesian and Polar spaces (row to ring, column to angular sector). The model correctly solves the orthogonal layout but fails on its topological equivalent. (Bottom Left) The percentage of intermediate CoT that explicitly invokes textual coordinates. Gemini-3.1-Pro and GPT-5.2 both exhibit high frequencies on Cartesian layouts. (Bottom Middle & Right) Two-stage perception and reasoning analysis on Gemini-3.1-Pro: grid size extraction accuracy estimates visual perception capability ( Middle ), while task accuracy is evaluated on correctly perceived instances ( Right ). High Cartesian accuracy validates the effectiveness of the shortcut, while the sharp decline on Polar layouts demonstrates that it collapses once the orthogonal structure is disrupted. Orthogonal grid-based layouts, which underlie many prominent visual reasoning benchmarks, can be readily discretized into explicit textual coordinates. We find that MLLMs systematically exploit this property: across a synthetic subset of 9 prominent benchmarks (e.g., [ 18 , 19 ] ), frontier models explicitly invoke Cartesian coordinates (e.g., “row 2”, “(x,y)”) in over 56\% of intermediate Chain-of-Thought [ 34 ] reasoning, converting visual layouts into textual representations to assist problem-solving. By substantially offloading reasoning from visual perception to text-based deduction, this shortcut significantly confounds the evaluation of visual reasoning on grid-based testbeds. Crucially, this shortcut collapses once the orthogonal layout is distorted: on a logically identical Sudoku puzzle rendered in Polar coordinates, frontier models that correctly solve the Cartesian version fail on its topological equivalent (Figure 1 ). Drawing on this observation, we introduce Polaris-Bench , comprising 53 procedurally generated visual reasoning tasks in Polar coordinate space, each accompanied by a Cartesian counterpart as controlled reference under consistent logical constraints. The Cartesian-to-Polar topological transformation breaks the orthogonal structure underlying the Cartesian shortcut, disrupting models’ ability to easily discretize visual layouts into textual coordinates. This design provides a controlled diagnostic testbed to re-evaluate visual reasoning under topological transformation, with the consistent logical constraints enabling direct cross-topology evaluation and analysis. We comprehensively evaluate Polaris-Bench across 14 state-of-the-art MLLMs spanning both proprietary and open-source families. The results reveal a striking and consistent pattern that frontier models achieving 70 – 83\% accuracy on Cartesian layouts see their performance collapse to 31 – 39\% on Polar equivalents, with the largest degradation approaching 47 points. Critically, the degradation persists even on tasks where the transformation preserves complete logical equivalence (i.e., identical rules, constraints, and ground truths), confirming that the coordinate transformation alone suffices to disrupt model reasoning. Moreover, while enabling high reasoning mode yields substantial accuracy gains on Cartesian layouts, these gains are severely diminished on Polar equivalents. These findings reveal a critical problem that current MLLMs lack fundamental topology-invariant visual reasoning capability. Our main contributions are as follows: • We identify and empirically validate the Cartesian Shortcut, whereby MLLMs systematically discretize orthogonal grid-based layouts into explicit textual coordinates, offloading visual reasoning onto text-based deduction and confounding existing visual reasoning evaluation. • We introduce Polaris-Bench, which re-formulates 53 visual reasoning tasks in Polar coordinate space to systematically disrupt the Cartesian Shortcut, with paired Cartesian counterparts enabling cross-topology evaluation. • Our comprehensive evaluation across 14 state-of-the-art MLLMs demonstrates that topological transformation induces drastic and universal performance collapse, with degradation persisting across diverse model families, scales, and reasoning configurations. Figure 2: Representative examples for each of the five core taxonomies in Polaris-Bench. 2 Rethinking vision reasoning: the Cartesian Shortcut Current MLLMs excel on vision reasoning benchmarks, where many of these tasks are built upon synthetic, grid-based layouts [ 28 , 25 ] . To investigate how models approach them, we analyzed over 3,800 synthetic-based questions across 9 prominent visual reasoning benchmarks (e.g., [ 18 , 19 ] ). Our Chain-of-Thought (CoT) [ 34 ] analysis reveals a striking pattern: in over 56\% of these examples, frontier models (e.g., Gemini-3-Flash) explicitly discretize the input image into Cartesian coordinates, generating text such as “ starting at (2,3), moving to (3,5) .” By translating these visual layouts into discrete matrices, models heavily offload visual logic onto text-based deductive reasoning, thereby reducing the need for persistent visual grounding. We define this phenomenon as the Cartesian Shortcut . While this behavior demonstrates robust deductive capability, it exposes a critical flaw in current evaluations: models can leverage textual discretization to assist visual reasoning, projecting an inflated estimate of visual capability. To further empirically validate this Cartesian Shortcut, we analyzed model behavior on our paired Cartesian-Polar benchmark. Intermediate CoT from frontier models reveal a deep dependency on textual discretization, explicitly invoking Cartesian coordinates in over 80\% of Cartesian examples (Figure 1 (left)). We further probed this dependency by a two-stage experiment on Gemini-3.1-Pro [ 16 ] . The model first generates a descriptive caption, then solves the task based on the caption. As shown in Figure 1 (middle, right), the model achieves 80.2\% accuracy on visual layout perception, estimated via grid dimension extraction from model-generated captions, and task accuracy on the subset of correctly perceived instances reaches 86.9\% . This empirically validates that the Cartesian Shortcut is not merely a behavioral pattern but a highly effective problem-solving strategy for current MLLMs. This effectiveness raises a fundamental concern: benchmarks built on orthogonal grids may systematically overestimate visual reasoning capabilities. Drawing on the fundamental duality of coordinate systems [ 3 , 42 ] , we introduce Polaris-Bench to dismantle the Cartesian shortcut. Polaris-Bench re-formulates visual reasoning tasks in Polar coordinate space, pairing each instance with a Cartesian counterpart as controlled

Read the complete text