Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information

Yassine Ibork, Myounggyu Won, Lokesh Das · 2025

Read Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information by Yassine Ibork, Myounggyu Won, Lokesh Das (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14910.txt.

Introduction

Yassine Ibork, Myounggyu Won, and Lokesh Das present Z-Merge, a learning system that coordinates cars merging onto a highway. Merging is risky in mixed traffic where self-driving and human-driven cars share the road. Past methods either manage the lane change or the gap creation, guided only by what nearby sensors can see.

Z-Merge uses vehicle-to-everything messages from a roadside unit to give each zone of the merge area its own shared picture. Several learning agents then coordinate both strategies at once, balancing lane changes against gap making. The setup is framed as teamwork under partial information. Better coordination at ramps could cut both crashes and the slowdowns that ripple down highways.

Catalog record

CREATORYassine Ibork, Myounggyu Won, Lokesh Das
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14910

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information Yassine Ibork, Myounggyu Won, Lokesh Das Abstract Ramp merging is a critical and challenging task for autonomous vehicles (AVs), particularly in mixed traffic environments with human-driven vehicles (HVs). Existing approaches typically rely on either lane-changing or inter-vehicle gap creation strategies based solely on local or neighboring information, often leading to suboptimal performance in terms of safety and traffic efficiency. In this paper, we present a V2X (vehicle-to-everything communication)-assisted Multiagent Reinforcement Learning (MARL) framework for on-ramp merging that effectively coordinates the complex interplay between lane-changing and inter-vehicle gap adaptation strategies by utilizing zone-specific global information available from a roadside unit (RSU). The merging control problem is formulated as a Multiagent Partially Observable Markov Decision Process (MA-POMDP), where agents leverage both local and global observations through V2X communication. To support both discrete and continuous control decisions, we design a hybrid action space and adopt a parameterized deep Q-learning approach. Extensive simulations, integrating the SUMO traffic simulator and the MOSAIC V2X simulator, demonstrate that our framework significantly improves merging success rate, traffic efficiency, and road safety across diverse traffic scenarios. 1 School of Computing, Wichita State University 2 Department of Computer Science, University of Memphis [email protected], [email protected], [email protected] Introduction Ramp merging is a critical maneuver that involves complex interactions between merging vehicles and mainline traffic. According to the U.S. Department of Transportation, approximately 300,000 car accidents occur each year in merging areas, contributing to around 50,000 fatal crashes ( el abidine Kherroubi et al. 2021 ) . Ramp merging is also a major contributor to traffic congestion, often caused by poor coordination between vehicles and unpredictable driving behavior ( Liao et al. 2021 ) . Highway merging accounts for approximately 40–80% of traffic congestion in the U.S., and about 1.7% of traffic fatalities occur during lane change or merging maneuvers ( Nakka et al. 2022 ) . With the rapid advancement of autonomous vehicles (AVs), numerous studies have been conducted to control AV motion in merging areas, aiming to enhance traffic efficiency, safety, and driving comfort. Two primary control strategies are inter-vehicle gap adjustment and lane-change maneuvers. Gap control approaches are primarily designed for mainline vehicles to create sufficient space for smooth merging ( Das and Won 2021b ; Das and Won 2021a ; Yadavalli et al. 2023 ; Zhu et al. 2024 ) . Alternatively, lane-change strategies focus on enabling mainline vehicles to change lanes to accommodate merging vehicles, while longitudinal control is typically used only for collision avoidance ( Xia et al. 2024 ; Liu et al. 2024 ; Cai et al. 2024 ; Chen and Yang 2024 ) . More recent approaches have focused on integrating both inter-vehicle gap adjustment and lane-change strategies to simultaneously cordinate the lane-change and gap creation for efficient merging. ( Wang et al. 2024 ; Hou et al. 2023 ; Chen et al. 2023 ) . However, these existing methods rely solely on local sensor data or limited information from neighboring vehicles via standard V2X communication, missing the opportunity to leverage “global” and “zone-specific” information that could significantly enhance traffic efficiency, safety, and driver comfort. In this paper, we present Z-Merge, a zone-based on-ramp merging control method using multi-agent reinforcement learning (MARL). The roadway segment is strategically divided into three zones: pre-merging, merging, and ramp. Summarized traffic information from each zone, collected via a roadside unit (RSU), is independently integrated into the MARL framework. This enables agents to make more informed and holistic decisions by combining inter-vehicle gap adjustment and lane-change strategies, thereby facilitating safe, efficient, and smooth on-ramp merging. To the best of our knowledge, this is the first MARL-based merging control method that incorporates both local and zone-specific traffic information. More specifically, we formulate the problem of simultaneously controlling the motion of multiple agents in a merging area—through the complex interplay of lane-changing, acceleration/deceleration, and inter-vehicle gap adaptation—using a Multi-Agent Partially Observable Markov Decision Process (MA-POMDP) framework. The state space is carefully designed, informed by transportation literature, to incorporate critical traffic parameters that capture both local information ( i.e., data collected through onboard sensors and vehicle-to-vehicle communication) and global (zone-specific) traffic conditions—factors that significantly influence traffic efficiency, safety, and driving comfort. To enable efficient, smooth, and collision-free merging, we design a hybrid action space that integrates lane-changing, acceleration/deceleration, and inter-vehicle gap adjustment controls. A robust multi-agent decision-making algorithm, based on the Parameterized Deep Q-Network (PDQN), is developed to select optimal control policies. We conduct extensive simulations incorporating V2X communication to train and evaluate the reinforcement learning agents. The results demonstrate that the proposed V2X-assisted MARL framework effectively enhances road safety, traffic efficiency, merging success rate, and queue length under complex merging conditions compared with state-of-the-art method. The contributions of our work are summarized as follows. • We propose the first zone-based on-ramp merging control method using multi-agent reinforcement learning (MARL) incorporating RSU-collected, zone-specific traffic information from pre-merging, merging, and ramp zones. • We carefully design the state space and hybrid action space to support holistic decision-making: the state space integrates both local and global traffic information, while the hybrid action space includes discrete (lane-change) and continuous (acceleration and gap control) actions. • We incorporate the Parameterized Deep Q-Network (PDQN) algorithm to achieve end-to-end vehicle control with both discrete and continuous action parameters. We develop a Double Parameterized Deep Q-Network to mitigate overestimation bias and improve learning stability. • We implement a simulation framework integrating the SUMO traffic simulator ( Lopez et al. 2018 ) with Eclipse MOSAIC ( Schrab et al. 2022 ) to enable realistic V2X communication and conduct extensive simulations under various traffic conditions. Related Work A substantial body of research on on-ramp merging control strategies can be broadly categorized into three types: (1) inter-vehicle gap control-based approaches, (2) lane-change-based approaches, and (3) hybrid approaches that combine both. Inter-vehicle gap control strategies primarily focus on adjusting the spacing between vehicles on the mainline to ensure a safe merging gap for ramp vehicles. Das and Won proposed a deep Q-Network-assisted dynamic ACC method that adaptively adjusts inter-vehicle gaps ( Das and Won 2021b ) . They later developed a safety-aware intelligent ACC using dual reinforcement learning agents to dynamically tune the time-to-collision threshold and desired gaps ( Das and Won 2021a ) . Yadavalli et al. designed a reinforcement learning-based approach for dynamic intra-platoon gap adaptation to support on-ramp merging ( Yadavalli et al. 2023 ) . Zhu et al. introduced a hierarchical gap creation strategy where mainline vehicles decelerate to create space for a platoon formed by ramp vehicles ( Zhu et al. 2024 ) . However, these methods are limited to controlling mainline vehicles within the merging zone, often causing upstream congestion—especially in the pre-merging area. Additionally, since they do not account for lateral maneuvers such as lane changes, these approaches are less effective in dynamically allocating road space in dense or highly interactive traffic scenarios. Another extensively used merging control strategy is the lane-change-based approach, which regulates mandatory lane changes by merging vehicles to enter the mainline and discretionary lane changes by mainline vehicles to create space for merging. Xia et al. proposed a multi-lane on-ramp merging strategy integrating dynamic lane-changing decisions with trajectory optimization ( Xia

Read the complete text