Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence
Yanbing Zhang, Bo Wang, Jianhui Liu, Nan Jiang, Jiaxiu Jiang, Haoze Sun, Yijun Yang, Shenghe Zheng, Lin Song, Haoyang Huang, Nan Duan, Wenbo Li Joy Future Academy · 2026
Read Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence by Yanbing Zhang, Bo Wang, Jianhui Liu, Nan Jiang, Jiaxiu Jiang, Haoze Sun, Yijun Yang, Shenghe Zheng, Lin Song, Haoyang Huang, Nan Duan, Wenbo Li Joy Future Academy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10588.txt.
Introduction
Yanbing Zhang and eleven colleagues give multi-mode models a way to look from a new angle. Current large multi-mode models struggle with spatial reasoning tasks needing view-based grasp, largely because they see only one static view. Their 2026 paper proposes Thinking with Novel Views, or TwNV, an approach adding generative novel-view making into the reasoning loop.
A Reasoner model spots spatial doubt, tells a Painter to build a fresh viewpoint, and checks the scene again with the extra proof. Planned experiments address three questions. Numerical camera-pose specs yield steadier view control than free-form language. Generation sharpness is tightly coupled with later spatial accuracy. Step-by-step multi-turn view tuning further improves performance. Across four spatial subtask categories and four model designs, TwNV always improves accuracy by 1.3 to 3.9 points, with the largest gains on angle-based subtasks.
Catalog record
| CREATOR | Yanbing Zhang, Bo Wang, Jianhui Liu, Nan Jiang, Jiaxiu Jiang, Haoze Sun, Yijun Yang, Shenghe Zheng, Lin Song, Haoyang Huang, Nan Duan, Wenbo Li Joy Future Academy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10588
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence Yanbing Zhang, Bo Wang, Jianhui Liu, Nan Jiang, Jiaxiu Jiang, Haoze Sun, Yijun Yang, Shenghe Zheng, Lin Song, Haoyang Huang, Nan Duan, Wenbo Li Joy Future Academy Abstract Current Large Multimodal Models (LMMs) struggle with spatial reasoning tasks requiring viewpoint-dependent understanding, largely because they are confined to a single, static observation. We propose Thinking with Novel Views (TwNV), a paradigm that integrates generative novel-view synthesis into the reasoning loop: a Reasoner LMM identifies spatial ambiguity, instructs a Painter to synthesize an alternative viewpoint, and re-examines the scene with the additional evidence. Through systematic experiments we address three research questions. (1) Instruction format: numerical camera-pose specifications yield more reliable view control than free-form language. (2) Generation fidelity: synthesized view quality is tightly coupled with downstream spatial accuracy. (3) Inference-time visual scaling: iterative multi-turn view refinement further improves performance, echoing recent scaling trends in language reasoning. Across four spatial subtask categories and four LMM architectures (both closed- and open-source), TwNV consistently improves accuracy by +1.3 to +3.9 pp, with the largest gains on viewpoint-sensitive subtasks. These results establish novel-view generation as a practical lever for advancing spatial intelligence of LMMs. † † footnotetext: Corresponding author. 1 Introduction Current Large Multi-modal Models (LMMs) have traditionally operated under a “Single-Pass Perception” paradigm, where reasoning is strictly conditioned on the static content of the provided input image Liu et al. 2024 ; Achiam et al. 2023 ; Chen et al. 2024b . Recently, several works have attempted to evolve this toward a broader “Thinking with Images” (TwI) framework by enabling models to invoke external tools for image manipulation, such as cropping, zooming, or 2D rotation Yang et al. 2024 ; Yang et al. 2025 . However, our preliminary investigation reveals a critical bottleneck: while these heuristic 2D transformations may improve local feature recognition, they offer negligible or even negative gains in complex spatial reasoning. Concretely, when we equip GPT-5 OpenAI 2025a with a tool-based grounding pipeline that lets the model identify and crop task-relevant regions, overall spatial reasoning accuracy drops by 0.8 percentage points relative to the single-image baseline (69.6% vs. 70.4%), with the deficit widening to 2.0 pp on multi-object relationship questions. This finding suggests that re-framing within the original 2D plane is fundamentally insufficient for spatial cognition: cropping and zooming cannot reveal occluded geometry or disambiguate 3D relational structures that are simply invisible from the input viewpoint. Humans, by contrast, resolve such ambiguities through mental simulation — we do not merely re-orient what we already see, but mentally construct entirely new perspectives to build a more complete spatial workspace. A parallel line of work injects 3D structure into VLMs via depth maps, point clouds, or features from explicit 3D reconstruction Chen et al. 2024a ; Fan et al. 2025 ; Cai et al. 2024 ; Zhang et al. 2026 . However, recovering 3D geometry from a single image is fundamentally ill-posed and scale-ambiguous Zeng et al. 2024 , so the geometry these methods inject is intrinsically noisy in the single-image regime that most spatial-reasoning benchmarks present. They also discard the image priors (textures, semantic context, fine-grained appearance) that pretrained VLMs are trained to read. Generative novel-view synthesis avoids both trade-offs: pretrained image generators have absorbed 3D-aware, world-model-level visual priors during web-scale pretraining Gabeur et al. 2026 and can be repurposed to render the same scene from a different camera pose Liu et al. 2023 , producing a new RGB image that keeps the VLM within its trained input distribution while supplying geometric evidence the original view lacks. Figure 1: Overview of the Thinking with Novel Views paradigm. (Top) Among three strategies for resolving viewpoint ambiguity, only generative novel-view synthesis provides sufficient 3D information while preserving semantics. (Center) Consistent accuracy gains (+1.3 to +3.9 pp) across closed- and open-source VLMs. (Bottom) The system (1) constructs a camera-motion instruction ( RQ 1 ), (2) renders a novel view ( RQ 2 ), and optionally (3) refines iteratively ( RQ 3 ). Details in § 4 . In this paper, we propose a systematic shift from 2D tool-calling to a generative Thinking with Novel Views (TwNV) paradigm. Rather than treating image generation as a creative end-goal, we reconceptualize it as a dynamic, 3D-aware reasoning workspace. As illustrated in Fig. 1 , we introduce a three-stage pipeline: an LMM first acts as the Planner , proactively determining which novel camera pose would best disambiguate the spatial query; a generative model then serves as the Synthesizer , rendering the requested viewpoint by simulating ego-motion such as translation, panning, or tilting; finally, the LMM acts as the Reasoner , jointly interpreting the original and synthesized images to produce an answer. By transforming a static single-image query into an iterative, generation-augmented reasoning process that dynamically explores the 3D scene, we move beyond the perspective constraints of the initial input. As the evidence in Fig. 1 shows, this approach yields consistent accuracy gains (+1.3 to +3.9 percentage points) across four VLMs spanning frontier closed-source systems and open-source models of varying scale, substantially outperforming the tool-based grounding baseline (e.g., 71.7% vs. 69.6% overall on GPT-5, with a striking 5.6 pp advantage on multi-object relationships). Building on this foundation, we conduct an in-depth exploration into the governing principles of the TwNV paradigm, structured around five critical dimensions: Scene Coverage. Performance gains are non-uniform across spatial subtasks: viewpoint-sensitive categories (orientation, multi-object relationships) benefit the most, while size estimation sees slight degradation, likely because novel views alter apparent object scale. Detailed analysis is in § 4 . Model Robustness. We evaluate TwNV across a diverse spectrum of model capacities, from Gemini-3-Flash Gemini Team et al. 2025 and GPT-5 OpenAI 2025a to the open-source Qwen3-VL series (235B and 32B parameters) Bai et al. 2025 . The paradigm delivers consistent improvements at every scale, with a notable “Small-Model Dividend” : the relative gain is more pronounced in smaller models (e.g., +3.9 pp for Qwen3-VL-32B vs. +1.3 pp for GPT-5), suggesting that explicit view synthesis serves as a compensatory mechanism for parameter-constrained models by offloading 3D reasoning to an external visual workspace. Instruction Design (RQ 1). To maximize the efficacy of the TwNV framework, we assess the optimal instruction interface for the Synthesizer. Our results indicate that continuous 3D camera parameters ( e.g. , explicit translation offsets and rotation angles) vastly outperform both free-form natural language descriptions and discrete categorical instructions. We hypothesize that providing explicit geometric priors minimizes the “semantic noise” inherent in linguistic prompts, thereby ensuring the generation of geometrically precise viewpoints that serve as faithful cues for downstream reasoning. Generation Fidelity (RQ 2). We examine how the quality of the Synthesizer influences the final reasoning outcome. By training a specialized pose-aware novel-view editing model to ensure geometric precision, we substantiate the hypothesis that a better Synthesizer indeed yields a better Reasoner . Our analysis reveals a positive correlation between the accuracy of viewpoint synthesis and the success rate of subsequent spatial inference, underscoring that the fidelity of the visual workspace is a decisive factor in the robustness of the entire pipeline. Inference-Time Visual Scaling (RQ 3). Finally, we ask whether visual reasoning can be scaled at inference time, analogous to Chain-of-Thought prompting in the