BEVCon: Advancing Bird’s Eye View Perception with Contrastive Learning

Ziyang Leng, Jiawei Yang, Zhicheng Ren, Bolei Zhou · 2025

Read BEVCon: Advancing Bird’s Eye View Perception with Contrastive Learning by Ziyang Leng, Jiawei Yang, Zhicheng Ren, Bolei Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04702.txt.

Introduction

Ziyang Leng and co-authors built BEVCon in 2025, a learning frame to sharpen top-down views for self-driving cars. A bird's-eye view maps the road around the car in one flat picture, which suits spotting cars, splitting lanes, and guessing paths. Past work tuned the view builders and task heads, but left the codes inside the view largely unshaped.

BEVCon trains those codes by contrast, pulling codes for the same scene part together and pushing unlike parts apart. One part works on the flat view codes, and a second part sharpens the camera backbone that feeds them. The frame is plain but lifts scores across the main driving tasks.

Catalog record

CREATORZiyang Leng, Jiawei Yang, Zhicheng Ren, Bolei Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-04702

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BEVCon: Advancing Bird’s Eye View Perception with Contrastive Learning Ziyang Leng, Jiawei Yang, Zhicheng Ren, Bolei Zhou Abstract We present BEVCon, a simple yet effective contrastive learning framework designed to improve Bird’s Eye View (BEV) perception in autonomous driving. BEV perception offers a top-down-view representation of the surrounding environment, making it crucial for 3D object detection, segmentation, and trajectory prediction tasks. While prior work has primarily focused on enhancing BEV encoders and task-specific heads, we address the underexplored potential of representation learning in BEV models. BEVCon introduces two contrastive learning modules: an instance feature contrast module for refining BEV features and a perspective view contrast module that enhances the image backbone. The dense contrastive learning designed on top of detection losses leads to improved feature representations across both the BEV encoder and the backbone. Extensive experiments on the nuScenes dataset demonstrate that BEVCon achieves consistent performance gains, achieving up to +2.4% mAP improvement over state-of-the-art baselines. Our results highlight the critical role of representation learning in BEV perception and offer a complementary avenue to conventional task-specific optimizations. Code and models are available at https://github.com/matthew-leng/BEVCon . I Introduction In recent years, Bird’s Eye View (BEV) perception has emerged as a crucial component in autonomous driving and robotic systems [ 1 , 2 , 3 , 4 ] . Its ability to aggregate multi-view data and transform the surrounding environment in a unified top-down-view representation makes it highly effective and versatile for tasks like object detection, segmentation, trajectory prediction, and planning [ 5 , 6 ] . A typical BEV perception model architecture comprises an image backbone, a BEV encoder, and task-specific heads [ 7 ] . While many prior efforts have focused on optimizing the design of BEV encoders and task heads to improve performance [ 8 , 9 ] , much less attention has been paid to enhancing BEV perception from a representation learning perspective. We argue that learned representations are central to a model’s performance, and improving them can lead to uniform gains across various BEV architectures, offering broader benefits complementary to task-specific designs. This work tackles this gap by integrating contrastive learning into BEV perception. Contrastive learning, which has shown remarkable success in the natural image domain through self-supervised learning approaches [ 10 , 11 ] , has recently demonstrated substantial potential in supervised settings as well [ 12 , 13 , 14 ] . By leveraging contrastive learning, we seek to enhance the underlying feature representations in BEV perception models, thereby improving their performance on downstream tasks like object detection. This introduces a new perspective of harnessing additional learning objectives beyond conventional detection losses without extra annotations, enriching the learning process. However, incorporating contrastive learning into BEV representation learning presents many challenges. To investigate whether a naive application of contrastive learning could improve BEV performance, we conduct preliminary experiments by both pre-training (without detection losses) and co-training (with detection losses) the image backbones on various datasets and report results in table V . Surprisingly, we observe that conventional contrastive learning methods, which focus on object-centric, image-level contrasts, fail to improve BEV detection performance. On average, pre-training methods result in 1.3% mAP decrease, while co-training only yields 1.4% mAP increase. These demonstrate that neither pre-training the image backbone with contrastive losses nor combining detection losses with image-level contrastive losses results in noticeable gains. We attribute the lack of improvement to two main factors: (1) Sample diversity: Driving [ 15 , 16 , 17 ] and robotics datasets [ 18 , 19 ] tend to exhibit much lower sample diversity compared to large-scale general-purpose datasets like ImageNet [ 20 ] . These datasets typically contain a limited variety of object classes and backgrounds, making it challenging to learn sufficiently discriminative features through contrastive learning. (2) Level of contrasting: Image-level contrast methods focus on entire images rather than specific objects or details. This approach is effective for object-centric datasets like ImageNet, where the object of interest occupies most of the image. However, it becomes less effective in BEV perception tasks, where wide field-of-view images contain multiple objects and intricate details across the scene. The lack of focus on object-level contrasts limits the ability to capture fine-grained features. To overcome these challenges, we introduce BEVCon, a dense contrastive learning framework to enhance BEV representation learning that can be seamlessly integrated into existing approaches. BEVCon consists of two contrast modules: (1) an instance feature contrast module, which enhances BEV feature extraction by performing dense contrastive learning on BEV features lifted from multi-view image data, leveraging annotations to improve localization and discriminativeness for detection tasks; and (2) a perspective view contrast module, which guides image backbone by focusing on region-specific features associated with different objects in the scene, helping the backbone capture more fine-grained, instance-level details. Together, these modules provide an effective strategy for learning better BEV representations. We evaluate the effectiveness of our BEVCon through extensive experiments, instantiating it with three representative families of state-of-the-art BEV perception methods: depth-unprojection-based methods such as BEVDet [ 5 ] , dense query-based methods like BEVformer [ 21 ] , and sparse query-based methods such as Sparse4D [ 22 ] . Our results show that the framework significantly enhances the learned representations of both the BEV encoder and the image backbone for BEV detection. Specifically, on the competitive nuScenes detection benchmark [ 15 ] , it achieves a 2.4 mAP gain over the BEVFormer-tiny model and a 1.3 mAP gain over the Sparse4D T=1 model. Moreover, it consistently generalizes these improvements across multiple BEV methods. Our contributions can be summarized as follows: • We introduce contrastive learning into BEV detection models with two specialized contrastive learning modules, providing additional training objectives that enhance BEV representation learning. • We design a unified framework that integrates these modules with detection tasks, aligning their optimization to enable joint training and improving detection performance. • We conduct extensive experiments to validate the applicability and generalizability of our BEVCon across various BEV approaches, achieving consistent improvements. Our study highlights that well-learned BEV representations lead to universal improvements across multiple BEV architectures and should receive more attention. This offers a complementary path to task-specific design optimizations. II Related Work BEV Perception Visual perception is one of the most important components of autonomous vehicles and robotics, including tasks like 3D object detection, semantic segmentation, and object tracking. Traditional perception algorithms perform 3D object detection independently on multiple cameras and then fuse those outputs in the object tracking component [ 2 ] . Recently, Bird’s-Eye View (BEV) perception has gained rising attention since it constructs a unified top-down-view feature map from perspective-view images for downstream perception tasks. Jonah et al. [ 2 ] proposes a novel algorithm that converts image data from different cameras into a BEV scene representation, by first generating 3D features from 2D image features using a latent depth distribution, then flattening those 3D features to BEV feature maps. Huang et al. [ 5 ] develops a similar framework of BEV feature map generation to 3D object detection tasks in autonomous driving and achieves significant performance gain. Different from the above two bottom-up approaches, Wang et al. [ 23 ] and Li et al. [ 21 ] use top-down approaches to

Read the complete text