Towards Natural Language Communication for Cooperative Autonomous Driving via Self-Play

Jiaxun Cui 1 , Chen Tang 1 , Jarrett Holtz 2 , Janice Nguyen 3 , Alessandro G. Allievi 2 , Hang Qiu 3 , Peter Stone 1,4 · 2025

Read Towards Natural Language Communication for Cooperative Autonomous Driving via Self-Play by Jiaxun Cui 1 , Chen Tang 1 , Jarrett Holtz 2 , Janice Nguyen 3 , Alessandro G. Allievi 2 , Hang Qiu 3 , Peter Stone 1,4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18334.txt.

Introduction

Jiaxun Cui and co-authors teach self-driving cars to talk to each other in plain language. Cars that share plans can dodge crashes and smooth traffic. Past links used codes no human can read. Their 2025 work builds language-model drivers inside a new gym of traffic scenes. Plain chain-of-thought talkers failed to coordinate. So the team adds CoopReflect, a learning frame with trial, error, and group debriefs.

Agents learn both what to say and what to do from the debriefs. Their messages grow clearer and more useful than baseline chatter. Crash dodges and flow gains follow. Last, the scene lessons are distilled into one policy model. That step keeps the skill while cutting decision delay hard.

Catalog record

CREATORJiaxun Cui 1 , Chen Tang 1 , Jarrett Holtz 2 , Janice Nguyen 3 , Alessandro G. Allievi 2 , Hang Qiu 3 , Peter Stone 1,4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18334

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Natural Language Communication for Cooperative Autonomous Driving via Self-Play Jiaxun Cui 1 , Chen Tang 1 , Jarrett Holtz 2 , Janice Nguyen 3 , Alessandro G. Allievi 2 , Hang Qiu 3 , Peter Stone 1,4 Abstract Past work has demonstrated that autonomous vehicles can drive more safely if they communicate with one another than if they do not. However, their communication has often not been human-understandable. Using natural language as a vehicle-to-vehicle (V2V) communication protocol offers the potential for autonomous vehicles to drive cooperatively not only with each other but also with human drivers. In this work, we propose a suite of traffic tasks in autonomous driving where vehicles in a traffic scenario need to communicate in natural language to facilitate coordination in order to avoid an imminent collision and/or support efficient traffic flow. To this end, this paper introduces a novel method, LLM+Debrief , to learn a message generation and high-level decision-making policy for autonomous vehicles through multi-agent discussion. To evaluate LLM agents for driving, we developed a gym-like simulation environment that contains a range of driving scenarios. Our experimental results demonstrate that LLM+Debrief is more effective at generating meaningful and human-understandable natural language messages to facilitate cooperation and coordination than a zero-shot LLM agent. Our code and demo videos are available at https://talking-vehicles.github.io/ . Keywords: multi-agent communication, LLM agent, autonomous driving 1 Introduction Driving is inherently a multi-agent problem [ 1 ] : each driver makes independent decisions based on their own perceptions while interacting with others on the road. As we transition towards (semi-) autonomous vehicles, centralized control [ 2 ] of all cars may appear efficient, but it is impractical and unlikely to gain widespread adoption. On the other hand, cooperative driving through communication channels is more practical and can still offer significant benefits even when implemented in a limited capacity. Past research has demonstrated the advantages of cooperative perception among autonomous cars [ 3 , 4 , 5 ] . However, these benefits are limited to vehicles that use the same learned environmental representation and communication language, limiting broader participation from those with different representations or language and leaving human drivers reliant solely on their local perceptions without being privy to the collaboration efforts. With the rise of visual-language models that are capable of language-conditioned high-level reasoning and planning in complex traffic situations, using natural language as a universal communication channel for both vehicle-human and vehicle-vehicle coordination holds significant promise. To this end, we have developed a cooperative driving system that leverages natural language for communication and coordination, thus freeing each participant to reason in its own representation. Prior work has explored training driving agents to make and explain driving decisions in natural language [ 6 , 7 ] or to coordinate with human drivers within a vehicle [ 8 ] , leveraging extensive datasets [ 9 , 10 , 11 , 12 ] . However, there is a lack of datasets and prior work featuring inter-vehicle communication in natural language alongside corresponding driving actions to facilitate cooperation. Recent advances in Large Language Models (LLMs) present new opportunities for agents to learn to speak and understand natural language messages in cooperative driving scenarios. In this work, we explore the potential of LLM agents to collaborate using natural language and optimize communication and collaboration strategies over self-play interactions, avoiding the need to collect real-world data. For this purpose, we develop TalkingVehiclesGym , a multi-agent simulation framework that realistically models vehicle-to-vehicle communication in a set of accident-prone scenarios. Building on an LLM agent framework, this paper introduces a multi-agent learning method for LLM agents, LLM+Debrief , which enables LLM agents to engage in centralized discussions post-interaction to refine their cooperation strategies, which are later incorporated into decentralized agent execution. Our experimental results in simulation demonstrate that when LLM agents struggle to collaborate effectively, our proposed learning method helps them learn what to communicate and how to respond to messages through interactions. Finally, we distill the learned behaviors of large models into a compact language model for near-real-time inference. While this work has not gotten to the point of fully human-usable communication — e.g., by enforcing short, real-time messaging — this paper takes a crucial step in that direction by restricting all messages to be in natural language. We demonstrate that natural language enables LLM-powered autonomous vehicles to engage in cooperative perception and negotiation, allowing effective coordination that improves traffic efficiency and safety even without human involvement. 2 Problem Definition In this paper, we focus on the subset of agents that are actively participating in the cooperation. We assume that these cooperative vehicles implicitly aim to help each other, treating all other (referred to as "background") vehicles as uncontrollable elements of the environment. Therefore, we frame the problem of Talking Vehicles as a partially observable stochastic game (POSG), focusing on optimizing the social welfare of a focal population ( \mathcal{F} ) [ 13 ] — defined as the joint reward of all participating agents — as the primary objective. The reward functions associated with each agent’s individual tasks may or may not fully align, necessitating coordination among agents to achieve high joint rewards. Each agent’s observation space is limited to a partial view of the full state, and agents make decisions in a decentralized manner based on their own partial observations and messages received from other agents. In this problem, each agent’s action space comprises two main components: (1) generating messages and (2) controlling the vehicle . In this work, the message generation space is defined over natural language (English). A POSG is defined by the tuple \langle\mathcal{I},\mathcal{S},\{\mathcal{O}_{i}\},\{\mathcal{A}_{i}\},\mathcal{P},\{\mathcal{R}_{i}\}\rangle , where \mathcal{I}=\{1,2,...,N\} refers to the identities of all agents in a scenario; \mathcal{S} is the state space comprehensively describing the environment; \mathcal{O}_{i} is the observation space describing agent i ’s view of the state; \mathcal{A}_{i} is the action space of agent i ; \mathcal{P} is the state transition function \mathcal{S}\times\mathcal{A}_{1}\times\mathcal{A}_{2}\times...\times\mathcal{A}_{N}\rightarrow\mathcal{S} ; \mathcal{R}_{i} is the reward function of agent i . The focal group of agents is denoted by \mathcal{F}\subseteq\mathcal{I} , representing a subset of all agents \mathcal{I} . The goal for each agent i\in\mathcal{F} is to optimize a policy \pi_{i} to maximize the expected cumulative task returns of all the agents in \mathcal{F} , given background agent policies outside the focal group: {\color[rgb]{0.345,0.545,0.902}\max_{\{\pi_{i}\}_{i\in{\mathcal{F}}}}~\mathbb{E}\Big[\sum_{{i\in{\mathcal{F}}}}\sum_{t=0}^{t=\infty}{{{R_{i}}(s_{t},\mathbf{a}_{t})}}\Big|{\{\pi_{{j}}\}}_{{j\notin\mathcal{F},j\in\mathcal{I}}}\Big]} , where s_{t} is the state at time t , and \mathbf{a_{t}}=(a_{1}^{t},a_{2}^{t},...,a_{N}^{t}) is the joint action of all agents at time t . The agent’s policy is structured to output both control and communication commands. Specifically, {\pi_{i}(O_{i},\{M_{j}\}_{j\in{\mathcal{F}}})\rightarrow\mathcal{A}_{i}} maps the observation of agent i and the received messages \{M_{j}\}_{j\in{\mathcal{F}}} to its action space {\mathcal{A}_{i}=\langle\mathcal{M}_{i},\mathcal{C}_{i}\rangle} , where \mathcal{M}_{i} represents the message generation space, which is constrained to natural language, and \mathcal{C}_{i} denotes the vehicle control space with dimensions for throttle, brake, and steering inputs. At time step t , the message M_{i} generated by agent i is broadcast to all the connected agents within a certain communication radius, at the next time step t+1 . This problem presents the following technical challenges: (1) How can agents understand the situation and generate meaningful messages to collaboratively perceive the environment or negotiate in natural language; (2) How can agents comprehend incoming natural language messages and incorporate them

Read the complete text