AimBot : A Simple Auxiliary Visual Cue to Enhance Spatial Awareness of Visuomotor Policies

Yinpei Dai † * , Jayjun Lee ‡ * , Yichi Zhang † , Ziqiao Ma † , Jianing Yang †, Amir Zadeh, Chuan Li, Nima Fazeli †‡ ⋆, Joyce Chai † ⋆ † Computer Science and Engineering Department, University of Michigan ‡ Robotics Department, University of Michigan · 2025

Read AimBot : A Simple Auxiliary Visual Cue to Enhance Spatial Awareness of Visuomotor Policies by Yinpei Dai † * , Jayjun Lee ‡ * , Yichi Zhang † , Ziqiao Ma † , Jianing Yang †, Amir Zadeh, Chuan Li, Nima Fazeli †‡ ⋆, Joyce Chai † ⋆ † Computer Science and Engineering Department, University of Michigan ‡ Robotics Department, University of Michigan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08113.txt.

Introduction

Yinpei Dai and co-authors give robot control policies a simple visual hint: drawn-on aiming lines. Their method, AimBot, overlays shooting lines and a scope reticle onto the robot's camera views. The overlays are computed from depth data, camera positions, and the gripper's pose, so they show where the gripper sits relative to objects. The drawing costs under a millisecond and needs no change to the learning model, since the policy sees augmented images.

Across visuomotor policies, in simulation and on real robots, the cues consistently improved performance. Policies struggle to infer spatial relations from raw pixels, and an explicit drawn cue relieves that burden. The trick is deliberately low-tech: no new sensors, no bigger networks. It suggests that some robot failures are perception-presentation problems, fixable by showing the model the geometry it needs in a form it can read.

Catalog record

CREATORYinpei Dai † * , Jayjun Lee ‡ * , Yichi Zhang † , Ziqiao Ma † , Jianing Yang †, Amir Zadeh, Chuan Li, Nima Fazeli †‡ ⋆, Joyce Chai † ⋆ † Computer Science and Engineering Department, University of Michigan ‡ Robotics Department, University of Michigan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08113

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AimBot : A Simple Auxiliary Visual Cue to Enhance Spatial Awareness of Visuomotor Policies Yinpei Dai † * , Jayjun Lee ‡ * , Yichi Zhang † , Ziqiao Ma † , Jianing Yang †, Amir Zadeh, Chuan Li, Nima Fazeli †‡ ⋆, Joyce Chai † ⋆ † Computer Science and Engineering Department, University of Michigan ‡ Robotics Department, University of Michigan Abstract In this paper, we propose AimBot , a lightweight visual augmentation technique that provides explicit spatial cues to improve visuomotor policy learning in robotic manipulation. AimBot overlays shooting lines and scope reticles onto multi-view RGB images, offering auxiliary visual guidance that encodes the end-effector’s state. The overlays are computed from depth images, camera extrinsics, and the current end-effector pose, explicitly conveying spatial relationships between the gripper and objects in the scene. AimBot incurs minimal computational overhead (less than 1 ms) and requires no changes to model architectures, as it simply replaces original RGB images with augmented counterparts. Despite its simplicity, our results show that AimBot consistently improves the performance of various visuomotor policies in both simulation and real-world settings, highlighting the benefits of spatially grounded visual feedback. Codes and videos can be found at https://aimbot-reticle.github.io/ Keywords: Robotic Manipulation, Visuomotor Policy, Imitation Learning Figure 1: Overview of AimBot , a lightweight visual guidance method that adds spatial cues onto RGB images for visuomotor policy learning. Given the robot’s end-effector pose, camera extrinsic and depth image, AimBot computes shooting lines and reticle overlay to highlight the spatial relationship between the gripper and objects of interest. The augmented RGB images are used as input to visuomotor policies for closed-loop control, enhancing task performance with minimal overhead. 1 Introduction Robotic manipulation in unstructured environments demands visuomotor policies that can robustly and accurately predict continuous actions from raw RGB observations. Although recent advances, such as diffusion-based policies [ 1 , 2 ] and vision-language-action (VLA) models [ 3 , 4 , 5 , 6 , 7 ] , have leveraged large-scale datasets to learn complex behaviors, they often lack explicit spatial grounding in visual input. As a result, these policies often exhibit limited spatial awareness of their end-effector (EE) pose and its relationship to surrounding objects [ 8 , 9 , 10 , 11 , 12 , 13 ] . Motivated by the intuitive visual feedback offered by scope reticles in optical sighting systems [ 14 , 15 , 16 ] , we present AimBot , a lightweight and effective visual augmentation technique that enhances spatial awareness in robotic manipulation with visual targeting cues. AimBot overlays auxiliary scope reticles and shooting lines onto multi-view RGB images by leveraging depth information, camera extrinsics, and the robot’s EE pose. These spatial cues project the intended grasp point and the orientation of the EE onto the image plane, offering an interpretable visualization of the alignment between the gripper and objects of interest (see Figure 1 ). Beyond highlighting spatial relationships, these augmentations also intuitively encode the robot’s proprioceptive state, making the EE position and orientation directly accessible in the visual domain. AimBot requires no changes to model architecture and introduces negligible computational overhead (less than 1 ms), while substantially enriching the spatial information available to any visuomotor policy. Through extensive experiments in both simulation and real-world environments, we demonstrate that AimBot consistently improves overall task performance across diverse VLA backbones, with particularly strong gains on challenging, long-horizon tasks that demand effective spatial alignment between the EE and objects. We believe this simple yet effective approach provides a practical and scalable means to enhance learning-based robotic manipulation in 3D environments. 2 Related Works 2.1 Vision-Language-Action Models for Manipulation Recent advances in pre-trained foundation models [ 17 , 18 , 19 , 20 ] have catalyzed the development of vision-language-action (VLA) models, which significantly boost visuomotor policy learning in generalization, scalability, and robustness [ 3 , 4 , 21 ] . OpenVLA [ 4 ] and OpenVLA-OFT [ 22 ] extend the Llama model [ 17 ] to large-scale real-world robot data, achieving strong performance and effective action generation. \pi_{0} [ 3 ] , \pi_{0} -FAST [ 5 ] leverage the Gemma model [ 23 ] and compression techniques to develop generalist policies capable of handling complex manipulation tasks. GR00T [ 24 ] utilize a heterogeneous mixture of real-robot trajectories, human videos, and synthetic datasets for expressive humanoid control. FuSe [ 25 ] moves beyond data scaling by leveraging multiple sensory modalities, demonstrating that natural language can universally ground vision, touch, and sound without requiring extensive multi-modal datasets. GO-1 [ 26 ] improves long-horizon reasoning through a VQ-VAE latent action model trained on web-scale video data. All these work highlight the rapid progress in robotic manipulation, providing a foundation upon which our work can build. 2.2 Visual Augmentation/Guidance for Manipulation With the success of visual prompting in VLMs [ 27 , 28 , 29 , 30 , 31 , 32 ] , recent approaches have explored visual intermediaries to enhance generalization in robotic manipulation. RT-Affordance [ 33 ] predicts affordance plans (i.e., key robot poses) conditioned on the visual input, enabling flexible learning across diverse supervision sources. RT-Trajectory [ 34 ] extends this idea by conditioning policies on coarse trajectory sketches, facilitating generalization to novel scenarios. GENIMA [ 35 ] fine-tunes a diffusion model to overlay joint-action targets onto RGB images, which are then translated into joint positions. TraceVLA [ 9 ] introduces visual trace prompting that encodes spatiotemporal trajectories directly into visual inputs, enabling VLA models to better predict actions. RoboPoint [ 36 ] leverages a VLM to predict keypoint affordances in terms of points on RGB images. HAMSTER [ 37 ] introduces a hierarchical approach that separates high-level task planning from low-level motor control using intermediate 2D path prediction. However, all those methods require online model inference during deployment, significantly limiting their practicality for real-time control. In contrast, our work presents a lightweight visual augmentation technique that overlays interpretable 2.5D spatial cues onto RGB images, offering spatial information without incurring inference costs. Moreover, by embedding EE state information directly into the pixel space, our method provides a novel and effective EE representation to enhance vision-language-action models. 3 Methodology 3.1 Method Overview Visuomotor policies [ 1 , 38 ] aim to directly map RGB camera observations and proprioceptive states to robot actions for sensorimotor control. Our goal is to enhance these policies with auxiliary visual cues, such as shooting lines and crosshair reticles, to improve spatial alignment and task success. The proposed technique, AimBot , is model-agnostic and requires no modifications to underlying policy architectures. It operates by augmenting the original multi-view RGB images with visual guidance, embedding spatial information directly into the pixel space. Fine-tuning is then performed exclusively on these new images, enabling any visuomotor policy or vision-language-action model to leverage the enhanced spatial cues. 3.2 AimBot Visual Guidance Suppose we have the camera extrinsics \mathbb{E}\in\mathbb{R}^{4\times 4} and intrinsics \mathbb{K}\in\mathbb{R}^{3\times 3} , along with the RGB image I and depth image D captured at the current timestep from camera c . Given a 3D point \mathbf{p}_{\text{wld}}\in\mathbb{R}^{3} in the world frame, we can project it into camera frame with pinhole camera model [ 39 ] : \begin{bmatrix}\mathbf{p}_{\text{cam}}\\ 1\end{bmatrix}=\mathbb{E}\cdot\begin{bmatrix}\mathbf{p}_{\text{wld}}\\ 1\end{bmatrix},\quad\text{where}\;\mathbf{p}_{\text{cam}}=(x_{c},y_{c},z_{c})^{\top}. (1) Then, we can continue to

Read the complete text