Camera-Only Bird’s Eye View Perception: A Neural Approach to LiDAR-Free Environmental Mapping for Autonomous Vehicles

Anupkumar Bochare · 2025

Read Camera-Only Bird’s Eye View Perception: A Neural Approach to LiDAR-Free Environmental Mapping for Autonomous Vehicles by Anupkumar Bochare (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06113.txt.

Introduction

Anupkumar Bochare proposes in 2025 a camera only system that gives self driving cars a bird's eye map of the road, with no laser scanner. Laser gear is costly, so cheaper cars often lack it. His design stretches a known map building method with modern detection and depth tools.

Cameras around the car feed object spotting and depth guessing models, and the views merge into one top down map. On two public driving data sets, the system hit high road marking scores and found most cars. Its place errors stayed near one meter on average. Bochare frames the result as proof that cameras alone can guide budget cars.

Catalog record

CREATORAnupkumar Bochare
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06113

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Camera-Only Bird’s Eye View Perception: A Neural Approach to LiDAR-Free Environmental Mapping for Autonomous Vehicles Anupkumar Bochare Abstract Autonomous vehicle perception systems traditionally rely on expensive LiDAR sensors to create accurate environmental representations. This paper presents a camera-only perception system that generates Bird’s Eye View (BEV) maps by implementing and extending the Lift-Splat-Shoot neural architecture. Our approach integrates YOLOv11 object detection with DepthAnythingV2 monocular depth estimation across multiple camera perspectives to achieve 360-degree environmental awareness. Through extensive evaluation on the OpenLane-V2 and NuScenes datasets, we demonstrate that our approach achieves 85% road segmentation accuracy and 85-90% vehicle detection rates compared to LiDAR ground truth, with average positional errors limited to 1.2 meters. These results suggest that deep learning techniques can extract sufficient spatial understanding from standard cameras to support autonomous navigation at a fraction of the hardware cost. Our method has significant implications for reducing the cost barrier to autonomous vehicle technology while maintaining performance comparable to LiDAR-based systems. I Introduction Autonomous vehicle perception systems traditionally rely on sensor fusion approaches that combine camera data with expensive LiDAR sensors to create accurate environmental representations. While LiDAR provides precise depth information, its high cost ($10,000-$80,000 per unit) creates a significant barrier to widespread autonomous vehicle adoption. This research addresses the fundamental question: Can modern deep learning techniques extract sufficient 3D understanding from standard cameras to enable reliable autonomous navigation without LiDAR? We present a camera-only perception system that generates accurate Bird’s Eye View (BEV) representations by implementing and extending the Lift-Splat-Shoot neural architecture. Our approach integrates state-of-the-art object detection (YOLOv11) with monocular depth estimation (DepthAnythingV2) across multiple camera perspectives to achieve 360-degree environmental awareness. The system transforms 2D camera inputs into a unified BEV representation through three key stages: depth-aware feature extraction, 3D space projection via quaternion-based coordinate transformation, and BEV semantic segmentation. Through extensive evaluation on the OpenLane-V2 and NuScenes datasets, we demonstrate that our camera-only approach achieves 85% road segmentation accuracy and 85-90% vehicle detection rates compared to LiDAR ground truth, with average positional errors limited to 1.2 meters. These results suggest that deep learning techniques can extract sufficient spatial understanding from standard cameras to support autonomous navigation at a fraction of the hardware cost. The main contributions of this work include: 1. A complete camera-only BEV generation pipeline that eliminates the need for expensive LiDAR sensors 2. Novel integration of YOLOv11 object detection with DepthAnythingV2 for accurate object placement in BEV space 3. A custom multi-component loss function (BEVLoss) that evaluates position accuracy, existence detection, and class identification 4. Comprehensive evaluation using two industry-standard autonomous driving datasets 5. Analysis of failure cases and limitations compared to LiDAR-based approaches The remainder of this paper is organized as follows: Section II discusses related work in BEV perception systems. Section III details our methodology, including the neural architecture and implementation details. Section IV describes our experimental setup and evaluation metrics. Section V presents quantitative and qualitative results. Section VI provides discussion and analysis, followed by conclusions and future work in Section VII . II Related Work II-A Bird’s Eye View Representations Bird’s Eye View (BEV) representations have become increasingly important in autonomous driving perception systems due to their ability to represent spatial relationships in a manner that facilitates path planning and navigation [ 1 ] . Traditional approaches to BEV generation rely heavily on LiDAR sensors, which provide direct 3D point cloud data that can be easily projected into the BEV space [ 2 ] . These systems typically achieve high accuracy but come with prohibitive hardware costs. Early camera-only approaches to BEV generation used handcrafted features and geometric transformations, known as Inverse Perspective Mapping (IPM) [ 3 ] . While computationally efficient, these methods struggled with occlusions and varying elevation. More recent approaches leverage deep learning to overcome these limitations. II-B Camera-Only Perception Approaches Recent advancements in deep learning have enabled increasingly sophisticated camera-only perception systems. The pioneering work by Philion and Fidler [ 4 ] introduced the Lift-Splat-Shoot architecture that projects image features into 3D space using learned depth distributions. This approach demonstrated the feasibility of generating BEV representations without explicit depth sensors. Building on this work, several researchers have proposed extensions to improve accuracy and computational efficiency. Chen et al. [ 5 ] developed BEVFormer, which employs a transformer architecture to capture spatial relationships more effectively. Similarly, Tesla’s Autopilot system has demonstrated a production-ready camera-only approach, although detailed technical information remains limited in the public domain [ 6 ] . II-C Monocular and Multi-View Depth Estimation Accurate depth estimation is critical for camera-only BEV systems. Traditional stereo matching techniques have largely been supplanted by learning-based approaches that can estimate depth from monocular images. Eigen et al. [ 7 ] introduced one of the first deep learning approaches to monocular depth estimation, while more recent work by Ranftl et al. [ 8 ] has demonstrated significantly improved performance through advanced network architectures and training techniques. The DepthAnything framework [ 9 ] represents the current state-of-the-art in monocular depth estimation, leveraging self-supervised learning techniques to achieve robust performance across diverse environments. For multi-view scenarios, MVSNet [ 10 ] and its derivatives have shown promising results by incorporating information from multiple camera perspectives. II-D Object Detection for Autonomous Vehicles Object detection forms another critical component of our system. The YOLO (You Only Look Once) family of detectors has shown remarkable progress in real-time object detection. The recent YOLOv11 [ 11 ] achieves state-of-the-art performance while maintaining computational efficiency suited for autonomous driving applications. Integration of object detection with BEV generation remains challenging, with most prior work treating these as separate tasks. Our approach differs by jointly optimizing both components through a unified training procedure, leading to improved performance and efficiency. III Methodology III-A System Overview Our camera-only BEV perception system builds upon the Lift-Splat-Shoot (LSS) architecture while incorporating significant enhancements for improved performance. Fig. 1 illustrates the overall pipeline of our approach, which consists of four main components: (1) multi-camera input processing, (2) feature extraction with depth estimation, (3) 3D projection and feature aggregation, and (4) BEV semantic map generation. Fig. 1: System architecture diagram showing the complete pipeline from multi-camera inputs to BEV output with depth estimation and feature extraction components. The system processes input images from six or seven camera perspectives simultaneously, depending on the dataset configuration. Each image is processed through a shared feature extraction backbone, followed by parallel heads for depth estimation and semantic feature extraction. The extracted features are then lifted to 3D space using the estimated depth distributions and splat onto a bird’s eye view grid. Finally, a lightweight BEV encoder processes the aggregated features to produce semantic segmentation maps and object detection outputs. III-B Multi-Camera Setup We utilize the standard camera configurations provided in the OpenLane-V2 and NuScenes datasets. The OpenLane-V2 dataset employs seven cameras positioned around the vehicle: front, front-left, front-right, rear, rear-left, rear-right, and side-left. The NuScenes dataset uses a six-camera setup, excluding the side-left view. Both configurations provide near-complete 360-degree coverage around the vehicle. To properly account for the spatial relationships between cameras, we utilize the extrinsic camera calibration parameters provided in the datasets. These parameters define the transformation from each camera’s local coordinate system to the vehicle’s coordinate system, allowing

Read the complete text