G 3 VLA: Geometric inductive bias for Vision-Language-Action Models
Yue Peng Yongzhe Zhao Artur Habuda Khuyen Pham Yanheng Zhu Tran Nguyen Le Fares Abu-Dakka Li Guo New York University Shanghai Technical University of Denmark MBZUAI - Mohamed bin Zayed University of Artificial Intelligence New York University Abu Dhabi Project Leads. · 2026
Read G 3 VLA: Geometric inductive bias for Vision-Language-Action Models by Yue Peng Yongzhe Zhao Artur Habuda Khuyen Pham Yanheng Zhu Tran Nguyen Le Fares Abu-Dakka Li Guo New York University Shanghai Technical University of Denmark MBZUAI - Mohamed bin Zayed University of Artificial Intelligence New York University Abu Dhabi Project Leads. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24472.txt.
Introduction
Peng and colleagues give robot vision-language-action models a sense of the geometry binding their cameras together. Such models inherit knowledge from pretrained vision language backbones. Their visual tokens live in flat image coordinates, not in the calibrated three-dimensional geometry of the robot's cameras. In multi camera rigs the views are coupled by known intrinsic and extrinsic parameters, yet standard models process them as independent pictures.
Their module injects calibrated geometry into a pretrained model's stream of visual tokens. It changes neither its action space nor its imitation objective. Intrinsic conditioned ray embeddings, projective encoding, and two way cross view fusion do the work. Supervision comes from ground truth point maps when available, else from a teacher model's confidence gated predictions. No depth sensors or manual labels are needed. Instantiated on a leading open model, the approach yields consistent gains across the LIBERO suite.
Catalog record
| CREATOR | Yue Peng Yongzhe Zhao Artur Habuda Khuyen Pham Yanheng Zhu Tran Nguyen Le Fares Abu-Dakka Li Guo New York University Shanghai Technical University of Denmark MBZUAI - Mohamed bin Zayed University of Artificial Intelligence New York University Abu Dhabi Project Leads. |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24472
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
G 3 VLA: Geometric inductive bias for Vision-Language-Action Models Yue Peng Yongzhe Zhao Artur Habuda Khuyen Pham Yanheng Zhu Tran Nguyen Le Fares Abu-Dakka Li Guo New York University Shanghai Technical University of Denmark MBZUAI - Mohamed bin Zayed University of Artificial Intelligence New York University Abu Dhabi Project Leads. Abstract Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot’s cameras — a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G 3 VLA , a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated π 3 \pi^{3} X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on π 0 \pi_{0} , G 3 VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on π 0.5 \pi_{0.5} and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at https://sites.google.com/view/g3vla . Figure 1: G 3 VLA overview. (A) Geometric inductive bias is injected into VLA visual tokens via intrinsic-conditioned ray embeddings ( K^{-1} ) and bidirectional cross-view fusion with PRoPE, leaving the pretrained backbone and action objective unchanged. (B) Stage 1 distills dense point maps from \pi^{3} X to pretrain the geometry modules; Stage 2 fine-tunes the full policy under action and distillation losses jointly. Keywords: Vision-Language-Action, Geometric Inductive Bias, Camera Geometry 1 Introduction Generalist robot manipulation requires two capabilities that have historically been difficult to combine: semantic understanding of instructions and objects, and spatial precision in estimating poses, distances, and cross-view geometry. Vision-Language-Action (VLA) models such as RT-2, OpenVLA, \pi_{0} , and GR00T leverage pretrained vision-language backbones to support language-conditioned control across diverse tasks [ 25 , 11 , 2 , 1 ] , but their visual interfaces remain largely 2D-image-token based, forcing calibrated camera geometry to be learned implicitly from action supervision. This limitation is especially consequential in calibrated multi-camera systems, where known intrinsics and extrinsics geometrically couple the views — structure that conventional image-token formulations leave unexploited. Explicit geometric structure has been shown to benefit manipulation: PerAct voxelizes RGB-D observations [ 18 ] , RVT aggregates multi-view renders [ 9 ] , and Act3D operates in 3D feature fields [ 8 ] . These methods achieve strong spatial precision but rely on task-specific architectures that cannot leverage pretrained VLM semantics. Recent bridging efforts such as SpatialVLA [ 17 ] and 3D-VLA [ 24 ] begin to close this gap, but require explicit 3D sensor inputs, large-scale spatial pretraining, or modifications to the action representation. The key question remains: Can calibrated camera geometry serve as a lightweight visual-token pathway for pretrained VLAs, improving spatial generalization without modifying the backbone, action space, or imitation objective? We answer this affirmatively with G 3 VLA , a camera-aware geometric framework that injects calibrated structure directly into the visual-token stream of pretrained VLA policies without modifying the backbone or action formulation. G 3 VLA leverages three geometric modules: intrinsic-conditioned ray embeddings [ 23 ] , which tag each ViT patch token with its back-projected viewing direction from K^{-1} ; projective positional encoding (PRoPE) [ 12 ] , which augments rotary positional embeddings with a camera-calibrated bias encoding cross-view projective relationships; and bidirectional cross-view fusion , which exchanges geometric context across camera streams before tokens reach the action model. Geometric supervision is provided either from ground-truth point maps when depth is available, or from confidence-gated \pi^{3} X [ 21 ] distillation when only camera intrinsics and extrinsics are provided — requiring no depth sensors or manual annotations. Geometry enters the policy exclusively through the visual-token representation, leaving the pretrained backbone and action objective intact. To evaluate whether calibrated geometry provides a transferable inductive bias beyond a single policy, we instantiate G 3 VLA on three architecturally distinct VLA systems: \pi_{0} , \pi_{0.5} , and GR00T 1.5. On \pi_{0} , G 3 VLA yields consistent improvements across LIBERO, RoboCasa24, and RoboTwin2.0, with the largest gains on spatially and object-sensitive tasks. On \pi_{0.5} , it further improves near-saturated LIBERO performance, confirming backbone compatibility. On GR00T 1.5, gains are mixed: its two-tower architecture — where the diffusion policy accesses visual features via cross-attention to a frozen VLM rather than consuming geometry-aware tokens directly — may attenuate the geometric signal. We treat this as suggestive evidence that the benefit of geometric injection depends on how directly geometry-aware tokens participate in action generation, though a more controlled study is needed to isolate the architectural factor. Our contributions are as follows: (1) A geometric gap in VLAs: We identify the mismatch between 2D-grounded visual tokens in pretrained VLAs and the calibrated spatial structure required for precise manipulation. (2) G 3 VLA: A lightweight, backbone-preserving visual-token pathway that injects calibrated camera geometry into pretrained VLA policies via ray embeddings, PRoPE, and cross-view fusion, without modifying the action space or imitation objective. (3) Geometry distillation from \pi^{3} X: A two-stage procedure supervising geometric modules with confidence-gated dense point maps from a visual geometry teacher, requiring no manual 3D annotations. (4) Multi-architecture validation: Consistent gains on \pi_{0} and \pi_{0.5} , together with an architectural analysis on GR00T 1.5 whose mixed results suggest that the benefit of geometric injection depends on how directly geometry-aware tokens reach the action pathway. 2 Related Work Vision-language-action models. VLA models reframe language-conditioned manipulation as sequence modeling over images, text, and actions. RT-1 and RT-2 showed that transformer policies with Internet-scale vision-language pretraining transfer semantic knowledge to robot control [ 3 , 25 ] , and OpenVLA, \pi_{0} , and \pi_{0.5} make this practical via open checkpoints, continuous-action flow matching, and co-trained open-world generalization [ 11 , 2 , 16 ] . These models nonetheless inherit largely 2D visual interfaces, learning camera geometry only indirectly from action supervision. Rather than scaling pretraining or altering the action decoder, we preserve the policy interface and expose calibrated camera structure to the visual tokens that drive the action model. Structured 3D representations for manipulation. Many policies improve spatial precision through explicit 3D scene representations: PerAct uses voxelized RGB-D observations [ 18 ] , RVT aggregates rendered multi-view features [ 9 ] , Act3D lifts 2D features into a 3D feature field [ 8 ] , PolarNet operates on point clouds [ 5 ] , and 3D Diffuser Actor pairs 3D scene representations with action diffusion for stronger cross-viewpoint generalization [ 10 ] . These approaches confirm the value of geometric structure—especially for precise end-effector placement—but modify the policy interface with voxels or point clouds; we instead keep an RGB-based VLA and introduce calibration as a token-level inductive bias rather than an explicit scene representation. Spatially aware VLAs. Other work connects VLA policies more directly to