RadarGen: Automotive Radar Point Cloud Generation from Cameras
Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany · 2025
Read RadarGen: Automotive Radar Point Cloud Generation from Cameras by Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17897.txt.
Introduction
Tomer Borreda and four coauthors built RadarGen, a tool that invents car radar data from camera views. Radar helps self driving cars see in rain and fog, yet real radar records are scarce for training. Their system paints a top down radar map, tagged with strength and speed clues, then turns it into a point cloud.
Hints about depth, object type, and motion, drawn from ready made vision models, steer the results toward scenes that could be real. On large driving data sets the fake radar matched the spread of real radar well. Tests also showed a smaller gap for sensing models trained on the fake data versus real data.
Catalog record
| CREATOR | Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17897
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RadarGen: Automotive Radar Point Cloud Generation from Cameras Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany Abstract We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery. RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird’s-eye-view form that encodes spatial structure together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Conditioning on images makes the approach broadly compatible, in principle, with existing visual datasets and simulation frameworks, offering a scalable direction for multimodal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step toward unified generative simulation across sensing modalities. Figure 1 : Controllable radar synthesis from vision. (Top) Given multi-view camera images, RadarGen generates realistic radar point clouds that align with real-world radar statistics and can be consumed by downstream perception models. (Bottom) The generation is semantically consistent: modifying the input scene with an off-the-shelf image editing tool (e.g., replacing a distant car with a closer truck) updates the radar response, removing returns from newly occluded regions and reflecting the new object geometry. 1 Introduction Recent advances in neural and generative simulation have made it increasingly practical to synthesize photorealistic data at scale for autonomous driving. By reconstructing real scenes with neural fields or generating entirely new ones using video diffusion models, these systems can produce diverse and controllable environments that closely mimic real sensor observations. This capability enables large scale resimulation of traffic, lighting, and weather conditions without costly rerecording or manual setup NVIDIA et al. 2025 ; Russell et al. 2025 ; Lin et al. 2025 . Despite this rapid progress, most neural simulators remain limited to the visual domain, focusing on the generation of RGB imagery and video. Recent efforts have begun extending these ideas to LiDAR, demonstrating controllable three-dimensional point cloud generation from camera inputs Ran et al. 2024 ; Zyrianov et al. 2022 ; Ren et al. 2025 . Radar, however, remains an open frontier. Although it is already ubiquitous in production vehicles, providing low cost, lightweight, and weather resilient perception, it has received far less attention within the generative modeling community. This imbalance limits the fidelity of current neural simulators, which cannot reproduce radar’s distinctive sensing characteristics, including signal sparsity, radar cross section (RCS), and Doppler. Generating radar data poses unique challenges. Radar measurements exhibit strong stochasticity due to multipath reflections, interference, and material-dependent scattering that vary with scene geometry. Operating at longer wavelengths, radar interacts with surfaces and internal structures beyond what cameras or LiDAR perceive, making it highly complementary yet difficult to model from vision alone. A further challenge lies in the nature of available radar data. In most large scale driving datasets, radar is provided only after proprietary signal processing that converts raw radio frequency waveforms into sparse point clouds with RCS and Doppler values. This processing chain, which includes range Doppler transforms, beamforming, and detection algorithms such as constant false alarm rate (CFAR), is closed and lossy, discarding phase and other fine grained signal information. In practice, storing raw radar signals is extremely memory intensive, so even commercial survey vehicles often record only processed point clouds. As a result, point clouds remain the practical representation of radar data for large scale learning. To address these challenges, we propose RadarGen , a generative framework for synthesizing automotive radar point clouds directly from camera imagery. RadarGen learns a distribution over radar observations conditioned on the visual scene, producing diverse and semantically consistent measurements rather than a single deterministic prediction, reflecting the inherent stochasticity of real radar signals. Conditioning on images allows RadarGen to leverage existing visual data and simulators, providing a scalable and modular way to enrich them with realistic radar signals. A key design choice in RadarGen is to build on SANA Xie et al. 2024 , an efficient image-latent diffusion model proven effective and scalable for image synthesis. To the best of our knowledge, no existing generative model can reliably support scene level point cloud synthesis from multiview images of real driving scenes, making an image diffusion backbone a natural and practical foundation. SANA’s architecture supports conditioning on visual input while efficiently handling the large number of tokens introduced by the multi-channel radar representation and by the additional conditioning cues incorporated later in our framework, allowing RadarGen to remain computationally efficient and deployable in large scale simulation settings. To make radar data compatible with the image diffusion backbone, we express each radar point cloud as an image-like bird’s eye view (BEV) representation. This representation encodes spatial structure together with radar cross section (RCS) and Doppler attributes, enabling all radar channels to share a unified latent space and allowing SANA’s pretrained autoencoder to operate without modification. A lightweight smoothing and recovery step preserves geometric accuracy while ensuring stable latent encoding. Learning radar purely from images is inherently difficult, as it requires reasoning about depth, semantics, and motion. To avoid forcing the denoiser to learn these cues from scratch, RadarGen leverages pretrained foundation models that provide dense visual priors. Depth, semantic, and motion cues extracted from each camera view are projected into bird’s eye view to align with the radar representation and are fused within the diffusion model. We summarize our main contributions below. • We present RadarGen , the first probabilistic diffusion framework to generate realistic automotive radar point clouds including location, RCS, and Doppler from multi-view camera inputs. • We introduce a latent diffusion methodology that trains on a BEV representation of sparse radar attributes, conditioned by BEV-aligned visual depth, semantic, and motion priors from foundation models. • We establish comprehensive metrics for radar point clouds based on geometric fidelity, radar attribute fidelity, and distribution similarity, validating our design through extensive ablations. • We demonstrate that the generated radar data can be interpreted by detectors trained on real data and supports applications such as scene editing. 2 Related Work 2.1 Physics-based radar simulation Physics-based simulators model the emission, propagation and reception of electromagnetic (EM) waves based on physical laws. To rigorously simulate time-domain EM propagation, some works directly solve Maxwell’s equations in the integral ( Capsoni and D’Amico 1998 ) or differential ( Liu et al. 2019 ; Stowell et al. 2008 ; Giannakis et al. 2015 ) form, offering high physical accuracy but are computation-intensive for real-time applications. For practical usage, other works approximate EM wave propagation based on geometric optics and ray-tracing ( Yun and Iskander 2015 ) . This technique has been widely adapted to enhance system-level fidelity ( Topak et al. 2011 ; Dudek et al. 2011 ; Holder et al. 2019 ; He et al. 2022 ) , improve scalability and real-time performance ( Hirsenkorn et al. 2017 ; Thieling et al. 2020 ; Sligar 2020 ) , and support specific modern applications ( Schüßler et al. 2021 ; Gubelli et al. 2013 ; Arnold et al.