Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model
2026
Read Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07695.txt.
Introduction
A 2026 preprint tackles a weak point of vision-language models: giving exact numbers for three-dimensional scenes. Precise numerical prediction is a bottleneck for 3D scene understanding. Traditional reinforcement learning for this task ranks answers against each other. That relative approach often suffers sparse rewards and unstable gradients. It also fails to use the checkable signals that 3D physical constraints provide.
The authors propose a smooth verifiable reward, presented under the name Smooth Operator. Verifiable rewards pay the model only when an answer can be checked as right. Smoothing that signal aims to keep training stable while staying honest. Better spatial numbers would help robots and assistants judge distance, size, and position. The paper targets scholars training vision-language models for grounded spatial reasoning.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-07695
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model Abstract Vision-Language Models (VLMs) face a critical bottleneck in achieving precise numerical prediction for 3D scene understanding. Traditional reinforcement learning (RL) approaches, primarily based on relative ranking, often suffer from severe reward sparsity and gradient instability, failing to effectively exploit the verifiable signals provided by 3D physical constraints. Notably, in standard GRPO frameworks, relative normalization causes “near-miss” samples (characterized by small but non-zero errors) to suffer from advantage collapse. This leads to a severe data utilization bottleneck where valuable boundary samples are discarded during optimization. To address this, we introduce the Smooth Numerical Reward Activation (SNRA) operator and the Absolute-Preserving GRPO (AP-GRPO) framework. SNRA employs a dynamically parameterized Sigmoid function to transform raw feedback into a dense, continuous reward continuum. Concurrently, AP-GRPO integrates absolute scalar gradients to mitigate the numerical information loss inherent in conventional relative-ranking mechanisms. By leveraging this approach, we constructed Numerical3D-50k, a dataset comprising 50,000 verifiable 3D subtasks. Empirical results indicate that AP-GRPO achieves performance parity with large-scale supervised methods while maintaining higher data efficiency, effectively activating latent 3D reasoning in VLMs without requiring architectural modifications. Siwen Jiao 1,2 , Tianxiong Lv 1†§ , Kangan Qian 3 , Chenxu Zhao 1 , Xiuyuan Zhu 1 , Tianlun Li 1 , Xiaolong Cheng 1 , Jinyu Li 1 , Zhihao Liao 1 , Yang Cai 1 1 Amap, Alibaba Group 2 National University of Singapore 3 Tsinghua University 2 2 footnotetext: Project Leader 4 4 footnotetext: Corresponding Author 1 Introduction Vision-Language Models (VLMs) excel in 2D visual understanding, often rivaling or exceeding human accuracy across diverse benchmarks. However, in genuine 3D scene understanding, these models reveal a critical weakness: they struggle with precise numerical estimation and geometric reasoning, such as depth perception, inter-object distance calculation, and object sizing ( Chen et al. 2024a ; Yang et al. 2024 ; Liu et al. 2025a ; Yang et al. 2025a ; Cheng et al. 2024 ) . This limitation stems from a reliance on 2D visual encoders that lack explicit geometric priors and depend instead on statistical patterns from planar images. Figure 1 : Standard GRPO vs. AP-GRPO(Ours). Standard GRPO (left) assigns binary rewards, yielding zero advantages for near-correct responses and discarding all samples, resulting in no policy update. In contrast, AP-GRPO (right) preserves absolute precision through SNRA smoothing, producing dense rewards and non-zero advantages modulated by the absolute term, thereby enabling effective policy updates and high sample utilization for precise numerical perception. Existing approaches to enhancing 3D capabilities in vision-language models (VLMs) fall into two main categories. The first involves feature augmentation, which incorporates external geometric representations (e.g., depth maps, point clouds, or specialized latent features from encoders like VGGT) as additional inputs or supervision ( Wang et al. 2025 ; Fan et al. 2025 ; Cheng et al. 2024 ; Zhu et al. 2025 ) . While effective, these methods introduce substantial computational overhead, strong reliance on auxiliary 3D modalities or pre-trained geometric models, and potential disruption of the original VLM alignment. The second strategy relies on large-scale, diverse training datasets to implicitly acquire 3D geometric understanding through extensive supervised or self-supervised training ( Chen et al. 2024a ; Jia et al. 2024 ; Daxberger et al. 2025 ; Cheng et al. 2024 ) . However, such approaches demand enormous data volumes for robust generalization and incur prohibitive training costs, limiting their practicality in resource-constrained settings. In this work, we argue that verifiable numerical signals grounded in physical laws offer a promising yet underexplored pathway for bridging the gap between 2D perception and 3D understanding. However, effectively leveraging such signals via reinforcement learning (RL) faces several challenges. First, conventional preference optimization frameworks ( Shao et al. 2024 ) rely primarily on relative comparisons rather than absolute, verifiable scalar feedback, compressing fine-grained numerical information into coarse preference signals. Second, traditional RL often adopts near-binary 0 – 1 rewards, resulting in extremely sparse supervision and unstable gradients. These limitations are particularly evident in group-relative methods such as Group Relative Policy Optimization (GRPO) ( Shao et al. 2024 ) . Under sparse rewards, samples with small but non-zero numerical errors may yield near-zero advantages after group-wise normalization, causing many “near-miss” samples to contribute little to policy updates, especially during early training.To address these issues, we introduce two complementary components: the Smooth Numerical Reward Activation Operator (SNRA) and Absolute-Preserving GRPO (AP-GRPO). SNRA is a dynamically parameterized sigmoid-based reward transformation that maps raw verifiable signals, including unbounded squared errors and discrete graded scores, into a dense reward space within [0,1] . This yields smoother and more informative gradients in early optimization while progressively enforcing stricter physical tolerances as training proceeds.Building on this transformed reward signal, we propose Absolute-Preserving GRPO (AP-GRPO), a modification of standard GRPO tailored for physics-grounded RL. Unlike pure group-relative normalization, which compresses absolute reward scale information, AP-GRPO introduces an auxiliary absolute reward component aligned with task-specific physical tolerances. This component serves as a corrective reference that anchors group-normalized advantages to a physically meaningful scale, alleviating the loss of scalar gradient information while retaining GRPO’s variance reduction benefits. Through the combined effect of SNRA and AP-GRPO, previously underutilized boundary samples with small numerical errors are reactivated, improving data efficiency. To fully leverage this, we construct Numerical3D-50k, a compact dataset of roughly 50k samples derived from complex 3D spatial queries with verifiable numerical (e.g., depth, distance, size) and relational (e.g., direction, spatial ordering) subtasks, and train our models on it. Notably, our method preserves the standard VLM architecture without adding encoders, modalities, or structural modifications. The main contributions of this work are summarized as follows: • Numerical3D-50k Dataset. A compact dataset of approximately 50k samples derived from existing 3D benchmarks, decomposed into verifiable numerical and discrete relational subtasks with deterministic, physics-grounded supervision. • SNRA and AP-GRPO. A dynamic sigmoid-based reward activation operator for densifying sparse physical signals, and a modified GRPO framework that partially preserves absolute reward information through a hybrid advantage formulation while maintaining GRPO’s variance control properties ( Shao et al. 2024 ) . • Empirical Evidence on Numerical Perception. Experiments demonstrate improved data efficiency by enabling effective learning from near-miss samples, achieving competitive performance with substantially larger-scale training and feature-augmentation baselines using only 50k samples, without architectural modifications. 2 Related Works 2.1 Visual Spatial Understanding and Reasoning Despite strong progress of large vision-language models (LVLMs) on many visual tasks ( Liu et al. 2024 ; Bai et al. 2025 ; Chen et al. 2024b ; Wang et al. 2024 ; Chen et al. 2024a ) , multiple benchmarks ( Yang et al. 2025a ; Ray et al. 2024 ; Cheng et al. 2024 ) show that spatial understanding and reasoning remain challenging. Prior work improves spatial capability from two complementary angles, namely spatial understanding (perceiving geometry, depth, and relations) and spatial reasoning (multi-step inference grounded in spatial cues) ( Qian et al. 2025a ; Qian et al. 2025d ; Qian et al. 2025c ; Qian et al. 2025b ; Qian et al. 2024 ) . On the understanding side, Semantic Abstraction ( Ha & Song 2022 ) laid an early foundation for open-world 3D scene understanding with 2D VLMs; subsequent efforts include SpatialVLM ( Chen et al.