How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

Qian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi, Perouz Taslakian, Aishwarya Agrawal · 2026

Read How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning by Qian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi, Perouz Taslakian, Aishwarya Agrawal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27310.txt.

Introduction

Qian Yang and colleagues ask how to make image-based reasoning actually count in unified multimodal models. They train with View Dropout, which hides part of one input view from the answer while leaving it open to the thinking image. Comparing top-down, panoramic, and point-matching thinking images, they find only panoramic thinking with View Dropout is both informative and learnable.

Catalog record

CREATORQian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi, Perouz Taslakian, Aishwarya Agrawal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61872
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27310

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning Qian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi, Perouz Taslakian, Aishwarya Agrawal Abstract Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they reason in language and discard the fine-grained geometry the task requires. Thinking with images aims to fix this by generating an intermediate thinking-image , but recent work shows the visual evidence in these traces is largely ignored. We therefore ask how to make visual thinking matter, and what kind of visual thinking works best. We ask these questions for unified multimodal models (UMMs) that natively support interleaved image–text generation. For the how , we propose View Dropout (VDrop) , a training-time intervention that hides parts of one input view from the answer span while leaving it visible to the thinking-image tokens. This incentivizes the model to make use of the thinking-image when answering, rather than answering based on the input views only. With the thinking-image now being used in answer prediction, we ask which kind of visual thinking works best. We frame this as a Learnability–Informativeness (L–I) tradeoff and compare three thinking-image variants: top-down, panoramic, and point-matching renderings. Trained on synthetic scenes and evaluated on five real-world out-of-domain benchmarks, panoramic visual thinking with VDrop is the only configuration that is simultaneously informative and learnable, and achieves the best out-of-domain generalization. 1 Introduction Cross-view spatial reasoning requires inferring scene layout, object placement, and geometry from images taken at different viewpoints. It underlies a range of vision-language model (VLM) applications, from embodied agents navigating a room Wang et al. 2025 ; Han et al. 2025 to video VLMs integrating temporally distant frames Wu et al. 2026 , all of which reduce to the same problem: maintaining a consistent scene representation across viewpoints that share only partial visual content. We study this capability in its basic form: given two partially overlapping views and a question, a VLM must reason across views to answer correctly, the format adopted by recent multi-view benchmarks Yang et al. 2026a ; Jia et al. 2026 ; Wang et al. 2026 . Despite strong single-image performance, the strongest VLMs perform marginally above chance at cross-view spatial reasoning Yang et al. 2026a ; Jia et al. 2026 . We argue this stems from a representational mismatch: cross-view reasoning is inherently visual , yet VLMs reason only through language, verbalizing observations into intermediates that discard the fine-grained geometry the task demands. Humans, in contrast, reason spatially in the visual domain by mentally constructing internal layouts Tversky 2003 ; Levinson 2003 ; Garrod and Anderson 1987 , suggesting that letting models think visually , by using visual intermediates as part of the reasoning chain, is key to closing this gap. Existing think-with-image approaches realize this by generating or invoking intermediate visual representations, such as 3D reconstructions, depth maps, or predicted camera trajectories Yang et al. 2026b ; Zhang et al. 2026b ; Chen et al. 2025b . Yet, recent work questions whether these intermediates do real perceptual work: under controlled interventions, model predictions barely change when the visual content of the intermediate is altered, indicating that visual evidence is largely ignored Liu et al. 2025b . Figure 1: Visual thinking for cross-view spatial reasoning. Given two input views and a cross-view spatial question ( left ), a UMM can generate one of three intermediate thinking-image types ( middle ) before answering: panorama, point matching, or top-down. Right: without View Dropout, the answer pathway takes a shortcut through the input views, leaving the generated thinking-image unused; with View Dropout, part of one input view is masked, forcing the answer to route through the thinking-image and making visual thinking causally load-bearing. Our experiments corroborate that visual evidence is under-used: with standard supervised fine-tuning (SFT), dropping the generated thinking-image at inference barely changes accuracy (Figure 3 , “Visual Thinking w/o VDrop”). SFT teaches the model to generate a plausible thinking-image but not to use it when answering: the thinking-image becomes a decorative by-product of training, present in form but not in function. This under-use motivates our first research question: (1) how to make visual thinking matter during learning. Once the thinking-image is genuinely used, a second question arises: (2) which kind of thinking-image is most effective for cross-view spatial reasoning, among natural candidates such as panoramic views, top-down layouts, and point-matching overlays that explicitly connect the two views. To study these questions, we use unified multimodal models (UMMs), which natively generate the thinking-image, enabling end-to-end learning of visual thinking and controlled comparison across intermediate representations within a single model. To make visual thinking matter during learning, we propose View Dropout (VDrop) (Figure 1 Right), a training-time intervention that masks part of one input view from the answer span, so the only remaining path for that spatial evidence runs through the generated thinking-image. VDrop requires no architectural change and is agnostic to which thinking-image is generated. Across every thinking-image variant we test, it consistently improves cross-view spatial reasoning. With VDrop in place, candidate thinking-image representations become meaningfully comparable, and we ask which works best. The different thinking-image variants trade off along two axes that prior work has not cleanly separated: informativeness (how much spatial structure the thinking-image variant unveils) and learnability (how reliably the UMM can produce that variant). We formalize this as a Learnability–Informativeness ( L–I ) tradeoff: a thinking-image type benefits spatial reasoning only if it is both spatially informative and learnable from data alone, and neither axis is sufficient on its own. We instantiate this study with synthetic scenes from Infinigen Indoors, training each strategy on BAGEL, a representative open-source UMM, and evaluating on one in-domain synthetic benchmark and five real-world out-of-domain benchmarks. Experiments show that visual thinking improves cross-view spatial reasoning both in- and out-of-domain, and that VDrop makes the generated thinking-image causally load-bearing, consistently improving OOD performance. Trained on only 8K synthetic samples, our best configuration achieves a 6.7 -point OOD gain over vanilla BAGEL and surpasses all prior methods we compare against, including methods trained on at least 3\times more data. Once visual thinking is forced to matter, the choice of representation also matters: top-down views, though informative, are not directly learnable by current UMMs, leaving panoramic visual thinking as the only candidate that scores high on both L–I axes and the only one that consistently beats prior methods on OOD generalization. Our contributions are as follows: • Method. We identify under-use as a pervasive failure mode of visual thinking and propose View Dropout , a training-time intervention that requires no architectural change, is agnostic to the thinking-image type, and consistently improves OOD cross-view spatial reasoning across all three thinking-image variants. • Framework. We frame the choice of visual thinking as a Learnability–Informativeness tradeoff , disentangling two axes that prior work has conflated: a representation may fail either because it does not encode sufficient information or because it is difficult to learn. • Empirical analysis. On one in-domain synthetic benchmark and five real-world OOD benchmarks, we show that the thinking-image becomes causally used only after VDrop training, and that panoramic visual thinking is the most informative and learnable representation. With only 8K training samples, it outperforms prior BAGEL-based

Read the complete text