MonoMPC: Monocular Vision Based Navigation with Learned Collision Model and Risk-Aware Model Predictive Control

Basant Sharma, Prajyot Jadhav, Pranjal Paul, K.Madhava Krishna, Arun Kumar Singh · 2025

Read MonoMPC: Monocular Vision Based Navigation with Learned Collision Model and Risk-Aware Model Predictive Control by Basant Sharma, Prajyot Jadhav, Pranjal Paul, K.Madhava Krishna, Arun Kumar Singh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07387.txt.

Introduction

Sharma and four co-authors built this robot navigation system in twenty twenty-five. It steers with a single color camera and no depth sensor. Depth guessed from images is often too noisy for direct crash checks in cluttered rooms. Their system, MonoMPC, feeds that noisy depth into a learned model instead.

The model predicts how much clearance a robot can expect along a planned path. A planner then picks paths with the lowest estimated crash risk. Training used both safe and unsafe runs, which kept the model's uncertainty honest. In real-world tests the robot crashed less often than strong rivals. It also reached goals more often and moved faster through clutter.

Catalog record

CREATORBasant Sharma, Prajyot Jadhav, Pranjal Paul, K.Madhava Krishna, Arun Kumar Singh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07387

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MonoMPC: Monocular Vision Based Navigation with Learned Collision Model and Risk-Aware Model Predictive Control Basant Sharma, Prajyot Jadhav, Pranjal Paul, K.Madhava Krishna, Arun Kumar Singh Abstract Navigating unknown environments with a single RGB camera is challenging, as the lack of depth information prevents reliable collision-checking. While some methods use estimated depth to build collision maps, we found that depth estimates from vision foundation models are too noisy for zero-shot navigation in cluttered environments. We propose an alternative approach: instead of using noisy estimated depth for direct collision-checking, we use it as a rich context input to a learned collision model. This model predicts the distribution of minimum obstacle clearance that the robot can expect for a given control sequence. At inference, these predictions inform a risk-aware MPC planner that minimizes estimated collision risk. Our joint learning pipeline co-trains the collision model and risk metric using both safe and unsafe trajectories. Crucially, our joint-training ensures optimal variance in our collision model that improves navigation in highly cluttered environments. Consequently, real-world experiments show 9x and 7x improvements in success rates over NoMaD and the ROS stack, respectively. Ablation studies further validate the effectiveness of our design choices. I Introduction Navigation in unknown environments using a monocular RGB camera is well-suited for lightweight robotic platforms like aerial or compact ground robots, where size, weight, and power constraints make LiDARs impractical. Vision-only setups reduce hardware complexity and energy usage, supporting longer missions and broader deployment in cost-sensitive settings. However, monocular navigation remains fundamentally challenging due to the absence of direct depth perception, which hampers reliable collision-checking essential for safe operation in cluttered environments. Recent progress in vision foundation models like DepthAnything [ 2 , 1 ] and ZoeDepth [ 3 ] has enabled depth prediction from a single RGB image, opening new avenues for monocular navigation. Prior work has used these depth estimates to build local 3D or collision maps for planning [ 4 ] . However, our experiments show that such depth predictions are often too noisy or inconsistent for reliable collision-checking, particularly in close-proximity scenarios where small errors can result in collisions (see Fig. ). In zero-shot settings—where the robot must generalize to unseen environments—these models fall short as standalone solutions. In this work, we propose a novel approach to monocular navigation that reinterprets estimated depth not as a proxy for ground truth, but as an informative input to a learned probabilistic collision model. While monocular depth estimates may be noisy, they contain valuable structural cues. Our model takes the RGB image and a candidate control sequence as input and predicts a distribution over worst-case obstacle clearance along that trajectory. These predictions are used at inference time to estimate collision risk, which is minimized via a risk-aware model predictive control (MPC) planner to generate safe and goal-directed actions. We propose a joint learning pipeline that co-trains the collision model and risk metric using both safe and unsafe trajectories. This ensures the model learns how its predictions impact downstream risk, providing valuable feedback that helps regularize variance to enable safe yet non-conservative behavior. We validate our approach through real-world hardware experiments, showing substantial improvements in navigation success over strong baselines like NoMaD [ 5 ] —a diffusion-based end-to-end method—and the ROS navigation stack, which relies on estimated depth-based costmaps (Fig. ). Ablation studies further highlight the impact of our design choices. Fig. 1 : Overview of baseline learning pipeline for our probabilistic collision model that predicts worst-case obstacle clearance along a trajectory. Given an RGB image and control sequence, we extract geometric features from the estimated point cloud using a pre-trained depth estimator and PointNet++. Combined with the initial robot state, these form the observation vector, which an MLP uses to predict the mean and variance of obstacle clearance. The learnable components (yellow blocks) are trained end-to-end using Gaussian negative log-likelihood loss. II Related Works In the following, we restrict our review to works that use only monocular camera for navigation and exclude those that rely on depth sensors or LiDARs. II-A End-to-End Approaches for Visual Navigation A common approach in monocular visual navigation is to train end-to-end models that map RGB images directly to control outputs like velocity commands or waypoints. Many rely on supervised learning via behavioral cloning or imitation learning [ 6 ] , [ 7 ] , [ 8 ] , [ 9 ] , [ 10 ] , [ 11 ] . While simple and appealing, these methods often generalize poorly, are sensitive to perception noise, and lack interpretability. They also omit explicit modeling of geometry, uncertainty, and constraints, making them data-hungry and hard to adapt. Recent methods like ViNT [ 12 ] and NoMaD [ 5 ] improve generalization by using image-conditioned diffusion models to generate trajectories from RGB input. However, they depend on high-quality supervision and do not model collision uncertainty. Another class of approaches focuses on learning spatial representations from monocular input [ 9 , 10 , 4 ] . For example, [ 4 ] constructs a map from estimated depth and then performs planning over it. However, since our problem setting focuses on navigation in unknown environments without explicit map construction, such methods fall outside the scope of this work. Deep reinforcement learning (DRL) has also been explored for visual navigation, particularly in unseen or partially known environments [ 13 ] , [ 14 ] , [ 15 ] . While DRL enables flexible end-to-end learning from experience, it typically requires extensive training in simulation, careful reward shaping, and offers limited safety guarantees. II-B World Model-Based Approaches for Visual Navigation World model-based methods [ 16 ] , [ 17 ] , [ 18 ] learn action-conditioned predictive models to simulate dynamics—predicting the next state given the current state and action—for downstream planning and control, rather than directly learning policies. [ 19 ] adopt a slightly different approach and propose a visual MPC controller that outputs a sequence of velocity commands based on the current image and a trajectory of subgoal images. Authors in [ 20 ] regress depth images and actions to collision probabilities which is then minimized within an MPC framework. Although theoretically, this approach can be extended to work with estimated depth, it is unclear how the associated noise will affect the efficiency of the overall pipeline. Moreover, we believe that predicting obstacle clearance distribution is easier than directly predicting collision probabilities, as the supervision data for the former can be easily obtained. Our design choice also allows us to use sophisticated statistical tools to capture collision risk from the clearance samples in an efficient manner. II-C Improvement Over State-of-the-Art: Unlike end-to-end pipelines, our approach explicitly incorporates safety constraints by learning an action-conditioned collision model, placing us closer to world-model methods. However, unlike [ 16 ] , [ 17 ] , [ 18 ] , [ 19 ] , we model predictive uncertainty and collision risk, and introduce a novel method to learn task-aware uncertainty. Moreover, we also learn optimal parameters of our risk metric to improve the efficacy of our approach. Most monocular navigation methods demonstrate results in simple environments like hallways. In contrast, we show—for the first time—robust and reliable navigation in cluttered settings typically requiring LiDAR or depth sensors.

Read the complete text