OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation

Heyu Guo 1 * , Shanmu Wang 2 * , Ruichun Ma 3 † , Shiqi Jiang 3 · 2025

Read OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation by Heyu Guo 1 * , Shanmu Wang 2 * , Ruichun Ma 3 † , Shiqi Jiang 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01210.txt.

Description

Heyu Guo and colleagues give a robot model more senses than a camera. Vision language action models predict robot actions from images and instructions and generalize well, but systems that see only ordinary color images miss physical information that other sensors carry.

OmniVLA folds extra sensors into one representation. The sensors are an infrared camera, a millimeter wave radar, and a microphone array. The representation is a sensor masked image, which overlays physically meaningful masks on the color picture. Keeping the input close to image statistics makes training efficient and gives each sensor type a uniform interface with a light projector. Built on a pretrained backbone, OmniVLA reached an average task success rate of eighty four percent on real world manipulation tasks, beating color only and raw sensor baselines by wide margins.

Catalog record

CREATORHeyu Guo 1 * , Shanmu Wang 2 * , Ruichun Ma 3 † , Shiqi Jiang 3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01210

Browse the full library · About the Tower of Babel