AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim · 2026
Read AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models by Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05868.txt.
Introduction
Heo and colleagues propose zero-shot camera adaptation for viewpoint-robust vision-language-action models, needing no demonstrations, fine-tuning, or architectural changes. Fine-tuned manipulation models prove sensitive to viewpoint changes common in unstructured environments. The key idea adjusts test-time observations to match training camera configuration in real time via feed-forward novel view synthesis. This plug-and-play approach preserves pretrained capabilities for any RGB policy. On LIBERO, the method beats augmentation and 3D-aware baselines, and real-world tests confirm stronger robustness across varying cameras.
Catalog record
| CREATOR | Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35255 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05868
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim Abstract Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. I Introduction Vision-Language-Action models (VLAs) trained on large-scale vision-language-action paired datasets have recently emerged as a promising paradigm for general embodied intelligence [ 4 , 48 , 34 , 19 , 3 , 13 , 2 ] . VLAs leverage the vast scale of internet vision-language data to acquire rich perceptual and semantic priors, positioning VLAs as scalable foundation models for robotic control. Consequently, a growing paradigm is to adopt such pre-trained VLAs and fine-tune them for specific tasks and environments encountered by end users. Instead of manually defining task-specific objectives or explicitly modeling 3D reconstruction, a small number of demo trajectories are sufficient to deploy the robot in real-world settings. However, recent work shows that fine-tuned VLAs often overfit to the specific configurations provided in the training trajectory, instead of deducing the spatial context [ 40 , 22 , 8 , 38 , 9 ] . We mainly focus on deviations of hardware arrangement between the camera and the robot, where minor shifts are unavoidable in everyday unstructured environments, such as homes or offices. Even under minor variations, the performance degrades significantly. For example, we observed that a 3 cm shift in the wrist camera can halve the success rate. The inherent end-to-end nature of VLAs necessitates acquiring demo trajectories in the modified setup and fine-tuning the entire model even after small shifts. To address this sensitivity, prior work trains policies with data collected from diverse viewpoints [ 17 ] or augments datasets using novel view synthesis [ 36 , 6 ] . However, improving viewpoint coverage through augmentation requires substantially more data and training, which is particularly costly for large VLAs. Another line of work improves robustness using geometry-aware or view-invariant representations, such as multi-view features, point clouds, or depth [ 32 , 42 , 1 , 30 , 39 , 41 ] . However, because VLAs build upon Vision-Language Models (VLMs) pre-trained on internet-scale RGB data, incorporating additional modalities often requires architectural modifications and limits the ability to fully leverage pre-trained priors [ 44 , 46 ] . Fig. 1: We present a zero-shot camera adaptation framework for VLAs that operates in real-time under various camera configuration changes from those used during training. At each control step, images from a test-time camera are synthesized into the training camera viewpoint at 30 Hz, while the frozen VLA policy runs at 10 Hz. Without any policy fine-tuning, our framework enables robust task execution across diverse setups—including different extrinsics, intrinsics, and freely moving handheld cameras such as an iPhone, ZED, and RealSense. In this paper, we propose a simple yet effective framework for zero-shot viewpoint adaptation that requires no additional demonstrations, policy fine-tuning, or architectural modifications, as shown in Figure 1 . The key idea is that we can freeze and leverage the latent space of pre-trained VLAs as long as we can virtually access images from the trained viewpoint. While we can easily compensate for minor viewpoint changes with traditional image warping, inaccurate multi-view geometry can critically degrade overall performance. Instead, we draw inspiration from recent progress in novel view synthesis in the computer vision community and generate precise projective warping of 3D geometry using a large view synthesis model [ 14 ] . The model is powerful at robustly handling various changes in pose, focal length, and lighting. Such a model handles only visual input, which is significantly smaller than the conventional VLAs. The viewpoint adaptation runs in real-time and can be plugged into the input of any VLAs to robustly process realistic variations. Our approach naturally relieves the rigid requirements for camera-robot configuration without altering the pre-trained VLAs. Remarkably, even under large viewpoint shifts (up to 15 cm translation and 60° rotation in camera extrinsics), the performance degradation of our method is minimal, showing the best task success rate over baselines on the LIBERO benchmark [ 23 ] . The view synthesis module is fast and delivers consistent performance across various camera configurations in real-world settings. Together with the VLA module, everyday users can easily operate robots even with casual handheld capture. II Related Work II-A Vision-Language-Action Models Vision-Language-Action models (VLAs) unify visual perception, language understanding, and robotic control by co-fine-tuning pre-trained Vision-Language Models (VLMs) on robotic trajectory data [ 4 , 48 , 34 , 19 , 3 , 13 , 2 ] . By leveraging internet-scale vision-language pre-training, VLAs achieve impressive semantic generalization to novel objects, scenes, and natural language instructions without task-specific engineering. However, VLAs exhibit significant fragility to distribution shifts, especially to camera viewpoint variations [ 8 , 38 , 9 ] . As these models operate directly on 2D RGB observations without explicit spatial supervision or camera pose awareness, they tend to overfit to training viewpoints. Recent evaluations on LIBERO-Plus [ 8 ] and VLATest [ 38 ] reveal substantial performance degradation under camera perturbations, with success rates dropping from over 90% to below 30% [ 8 ] . This camera viewpoint sensitivity poses a critical barrier to real-world deployment in unstructured environments where camera placements differ from trained environments. II-B Viewpoint Robust Robotic Policies To address the viewpoint brittleness of visuomotor policies, recent work has explored two complementary directions: representation learning and data augmentation strategies. Within the first direction, several methods enhance robustness through improved spatial representations beyond raw RGB images. Specifically, some works use explicit 3D representations such as point clouds [ 31 , 43 ] , demonstrating substantial gains by converting the representation to the canonical reference frame which is consistent over viewpoints. However, open-source large-scale robotic datasets lack point cloud data [ 29 ] , limiting the scalability of point cloud-based VLAs. To overcome this reliance on depth sensors, GeoAwareVLA leverages implicit geometry-aware features from a pre-trained multi-view vision model [ 1 ] . Parallel to these representational advancements, the second direction explores viewpoint generalization through data augmentation. Recent works in this area use generative models to augment training data across diverse viewpoints [ 36 , 6 ] . However, these methods struggle with drastic viewpoint changes and incur substantial computational costs, especially for large-scale VLAs.