Discrete Contrastive Learning for Diffusion Policies in Autonomous Driving
Kalle Kujanpää, Daulet Baimukashev, Farzeen Munir, Shoaib Azam, Tomasz Piotr Kucner, Joni Pajarinen, Ville Kyrki · 2025
Read Discrete Contrastive Learning for Diffusion Policies in Autonomous Driving by Kalle Kujanpää, Daulet Baimukashev, Farzeen Munir, Shoaib Azam, Tomasz Piotr Kucner, Joni Pajarinen, Ville Kyrki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05229.txt.
Introduction
Kalle Kujanpää and co-authors teach a driving simulator the variety of human styles, in a 2025 paper. Testing self-driving cars needs simulated drivers who behave like real people, whose styles vary widely. The team first mines a set of recorded human driving for a dictionary of styles, using contrastive learning to pull the styles apart, then turns them into discrete codes.
A diffusion policy, trained to generate driving behavior given one of these style codes, then plays the simulated drivers. In tests, the generated driving was both safer and more human-like than output from standard machine-learning baselines. The authors say richer, more faithful simulation should help evaluate and improve real self-driving systems.
Catalog record
| CREATOR | Kalle Kujanpää, Daulet Baimukashev, Farzeen Munir, Shoaib Azam, Tomasz Piotr Kucner, Joni Pajarinen, Ville Kyrki |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-05229
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Discrete Contrastive Learning for Diffusion Policies in Autonomous Driving Kalle Kujanpää, Daulet Baimukashev, Farzeen Munir, Shoaib Azam, Tomasz Piotr Kucner, Joni Pajarinen, Ville Kyrki Abstract Learning to perform accurate and rich simulations of human driving behaviors from data for autonomous vehicle testing remains challenging due to human driving styles’ high diversity and variance. We address this challenge by proposing a novel approach that leverages contrastive learning to extract a dictionary of driving styles from pre-existing human driving data. We discretize these styles with quantization, and the styles are used to learn a conditional diffusion policy for simulating human drivers. Our empirical evaluation confirms that the behaviors generated by our approach are both safer and more human-like than those of the machine-learning-based baseline methods. We believe this has the potential to enable higher realism and more effective techniques for evaluating and improving the performance of autonomous vehicles. I Introduction Human behavior modeling is essential in developing intelligent learning systems, particularly for autonomous driving. This approach offers crucial insights into how humans interact, behave, and respond within complex environments [ 1 ] . However, modeling and simulating human behavior to create a data-driven simulation for autonomous driving faces challenges due to the high diversity and variability of human driving behaviors [ 2 , 3 ] . Accurately capturing these behaviors is critical for enhancing simulation realism, which in turn improves sim-to-real transfer and facilitates safer autonomous vehicle development. Imitation learning approaches, particularly Behavior Cloning (BC), offer a viable method to model human behaviors [ 4 ] . In BC, an agent is trained to emulate demonstrator actions using an offline dataset of observation-action pairs, effectively replicating human-like decision-making processes. However, popular BC approaches often simplify distribution modeling for accessible learning, typically using point estimates via Mean Squared Error (MSE) or action space discretization [ 5 , 6 ] . These limitations highlight the need for an improved behavior model that can capture long and short-term dependencies, represent diverse modes of behavior, and accurately replicate learned behaviors. Recent advancements in diffusion models, currently leading in image, video, and audio generation [ 7 , 8 , 9 , 10 , 11 ] , offer a promising approach to modeling the full spectrum of human driving behavior. Prior work has demonstrated how to model action distributions without approximation with diffusion, capturing the complete action distribution [ 12 ] . This could overcome the limitations of traditional behavior modeling techniques in autonomous driving, but it may struggle to fully capture the diversity inherent in human driving due to not explicitly modeling how temporally consistent human driving styles affect driving behavior. Fig. 1 : An overview of our method, DSDP (Discrete Style Diffusion Policy). DSDP takes a history of past driving behavior as observation-action pairs (\mathbf{o},\mathbf{a}) and uses a prior network to sample a driving style \mathbf{c} . The styles are learned via discrete contrastive learning. The denoising network then uses the encoded current observation \mathbf{o} and style \mathbf{c} to generate a driving action \mathbf{a} through diffusion. In this work, we propose DSDP: Discrete Style Diffusion Policy , a novel framework for learning to model human driving behavior from pre-existing datasets. Our approach is predicated on the assumption that human drivers exhibit a variety of distinct driving styles, contributing to the diversity of observed driving behaviors. To capture this diversity, DSDP employs contrastive learning with the InfoNCE loss [ 13 ] and Lookup-Free Quantization (LFQ) [ 14 ] to extract discrete driving styles from unlabeled data. Then, we train a conditional Denoising Diffusion Probabilistic Model (DDPM) [ 15 ] conditioned on both observations and driving styles to generate actions. By integrating these components, DSDP aims to overcome the limitations of traditional behavior cloning techniques in representing diverse human driving behaviors. We evaluate the performance of our proposed method in various environments, demonstrating superior performance in capturing the range and fidelity of human driving behavior compared to the baselines and parametric models. Our results suggest that incorporating explicit driving styles can improve the realism of driving simulations, which is critical for training and validating autonomous vehicle policies. The main contributions of our work are: 1. An unsupervised discrete driving style extraction method using the contrastive InfoNCE loss and LFQ on unlabeled data to model human driving behavior. 2. DSDP, a method that combines the driving style extraction with a conditional DDPM for action generation 3. A comprehensive evaluation on historical driving data with a series of ablation studies II Related Work Human driving behavior is complex, involving interactions between the driver, vehicle, and environment and integrating various psychological aspects, complicating modeling [ 16 , 17 ] . The variability and diversity of drivers further complicate developing Driver Behavior Models (DBMs) [ 17 ] . Traditional DBMs like the Intelligent Driver Model (IDM) [ 18 ] , MOBIL [ 19 ] , and models by Gipps [ 20 , 21 ] , are primarily deterministic. Despite attempts to calibrate them with real-world data [ 22 , 23 , 24 , 25 ] , they often fail to capture the stochastic nature of human driving behavior. Recent research has increasingly turned to machine learning, particularly neural networks, to fit DBMs to large driving datasets [ 26 , 27 , 28 ] . However, even these approaches struggle to capture the inherent stochasticity of driving behavior. While adding noise can introduce randomness, simple approaches like Gaussian noise often fail to capture variability in human driving behavior [ 29 , 30 , 31 , 32 ] . Stochasticity can also be introduced through model parameters, such as by using game theory [ 33 ] or genetic algorithms [ 34 ] . [ 35 ] and [ 36 ] proposed stochastic car-following models that effectively represent the distribution of time progress to align simulation environments more closely with real-world data. However, these studies focus primarily on single-lane road scenarios and do not address the accumulation of errors over time. BC models human behavior by directly learning policies from demonstrating data, minimizing the forward loss relative to the expert distribution [ 37 , 38 ] . However, naïve BC struggles with multimodal distributions often present in real-world datasets from multiple demonstrators [ 6 ] . To overcome this, advanced models like energy-based [ 39 ] and generative adversarial models [ 40 ] have been explored. However, energy-based models suffer from complex sampling, while generative adversarial models face mode collapse [ 41 ] , limiting their suitability for modeling driving behavior. Recently, diffusion models have gained popularity for policy learning, demonstrating success in, among others, robotic control and 3D gaming environments [ 42 , 43 , 44 , 45 , 12 ] . Diffusion models effectively capture multimodal, complex distributions and offer more straightforward sampling compared to energy-based models [ 39 ] . Planning with diffusion models [ 46 , 47 , 48 ] is an alternative to using them to represent the BC policy. In contrast, consistency models offer a faster, one-step sampling alternative to diffusion models with possibly lower generation quality [ 49 , 50 ] . We demonstrate that modeling human driving style explicitly while using diffusion as the policy learning backbone achieves more accurate results than directly using diffusion. Fig. 2