Spatiotemporal Decision Transformer for Multi-Agent Traffic Coordination

Haoran Su, Yandong Sun, Hanxiao Deng · 2026

Read Spatiotemporal Decision Transformer for Multi-Agent Traffic Coordination by Haoran Su, Yandong Sun, Hanxiao Deng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02903.txt.

Introduction

Coordinating traffic lights across a city network is the problem in a 2026 paper by Haoran Su and 2 co-authors. Reinforcement learning can adapt signals, but multi-agent coordination and sample efficiency remain hard. Each intersection must act with its neighbors in mind, not alone.

Their system, MADT, recasts signal control as sequence modeling with a decision transformer. A graph attention part captures links between intersections. A temporal encoder tracks how traffic evolves. Target performance is set through return-to-go conditioning, a way of asking for a desired total reward. Tests on synthetic grids and real traffic followed. Average travel time fell by five to six percent against the strongest baseline.

Catalog record

CREATORHaoran Su, Yandong Sun, Hanxiao Deng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02903

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spatiotemporal Decision Transformer for Multi-Agent Traffic Coordination Haoran Su, Yandong Sun, Hanxiao Deng Abstract Traffic signal control is a critical challenge in urban transportation, requiring coordination among multiple intersections to optimize network-wide traffic flow. While reinforcement learning has shown promise for adaptive signal control, existing methods struggle with multi-agent coordination and sample efficiency. We introduce MADT (Multi-Agent Decision Transformer), a novel approach that reformulates multi-agent traffic signal control as a sequence modeling problem. MADT extends the Decision Transformer paradigm to multi-agent settings by incorporating: (1) a graph attention mechanism for modeling spatial dependencies between intersections, (2) a temporal transformer encoder for capturing traffic dynamics, and (3) return-to-go conditioning for target performance specification. Our approach enables offline learning from historical traffic data, with architecture design that facilitates potential online fine-tuning. Experiments on synthetic grid networks and real-world traffic scenarios demonstrate that MADT achieves state-of-the-art performance, reducing average travel time by 5-6% compared to the strongest baseline while exhibiting superior coordination among adjacent intersections. Keywords Decision Transformer \cdot Multi-Agent Reinforcement Learning \cdot Traffic Signal Control \cdot Offline RL \cdot Graph Attention Networks 1 Introduction Traffic congestion represents one of the most pressing challenges facing modern urban areas, with the 2021 Urban Mobility Report estimating annual costs exceeding $87 billion in the United States alone due to wasted time and fuel ( 15 ) . Beyond economic impacts, traffic congestion contributes significantly to air pollution, greenhouse gas emissions, and reduced quality of life for urban residents. Traffic signal control, which determines when vehicles can proceed through intersections, plays a crucial role in managing urban traffic flow. Effective signal control can reduce travel times, decrease vehicle emissions from idling, and improve overall network throughput. Traditional approaches to traffic signal control, such as fixed-time plans and actuated control based on loop detector readings, fail to adapt to the complex, dynamic patterns of urban traffic. These methods are designed for average conditions and perform poorly during peak hours, special events, or when traffic patterns deviate from historical norms. The emergence of connected vehicle technology and ubiquitous sensing creates unprecedented opportunities for data-driven adaptive signal control that can respond to real-time traffic conditions across entire urban networks. Recent advances in deep reinforcement learning (DRL) have shown promising results for traffic signal control ( 22 ; 20 ; 21 ; 16 ) . These methods learn control policies through trial-and-error interaction with traffic simulators, demonstrating the ability to outperform traditional approaches in complex scenarios. However, scaling DRL to city-wide networks with hundreds of intersections remains challenging due to several fundamental issues. First, the exponential growth of the joint action space makes naive multi-agent formulations intractable. Second, the need for effective coordination between intersections—to enable “green waves” and prevent spillback—requires sophisticated information sharing mechanisms. Third, the sample complexity of online RL methods necessitates millions of simulator interactions, limiting practical deployment. Meanwhile, the paradigm of reinforcement learning via sequence modeling has emerged as a powerful alternative to traditional RL algorithms. The Decision Transformer ( 4 ) demonstrated that offline RL can be effectively cast as a sequence modeling problem, achieving strong performance without dynamic programming or policy gradients. Rather than learning value functions or computing policy gradients, the Decision Transformer simply predicts actions conditioned on desired returns using a GPT-style architecture. This approach offers several compelling advantages: the training stability of supervised learning, the ability to leverage large offline datasets without exploration, natural handling of long-horizon credit assignment through attention mechanisms, and the flexibility to specify performance targets at test time. However, the Decision Transformer was designed for single-agent settings and does not naturally extend to multi-agent coordination. Extending this paradigm to traffic signal control presents unique challenges that have not been addressed in prior work: • Information sharing : How should intersections share information to enable coordination? The road network imposes a natural structure where adjacent intersections must coordinate more closely than distant ones. • Spatial structure : How can we capture the topology of the traffic network? Unlike games where agent ordering may be arbitrary, traffic networks have fixed spatial relationships that determine coordination requirements. • Network-level optimization : How do we condition on network-level performance targets rather than individual agent rewards? Traffic optimization requires balancing local and global objectives. • Scalability : How can we maintain computational efficiency as the number of intersections grows? City-wide deployment requires inference times compatible with real-time control (typically < 1 second). In this paper, we introduce MADT (Multi-Agent Decision Transformer), a novel architecture that addresses these challenges by combining the sequence modeling framework of Decision Transformer with graph-structured attention for spatial coordination. Our approach treats multi-agent traffic signal control as a conditional sequence generation problem: given the current traffic state and a target network-level return, generate coordinated actions for all intersections that achieve the desired performance. Our key contributions are as follows: 1. Graph-structured spatial attention for multi-agent coordination : While prior work like DTLight ( 7 ) applies Decision Transformer to traffic control using independent agents with discounted neighbor information, we introduce explicit graph attention layers that model the road network topology. This enables each intersection to attend to spatial neighbors with learned attention weights, achieving permutation equivariance and ensuring coordination patterns emerge from road connectivity rather than arbitrary agent indexing. 2. Unified architecture combining graph and temporal attention : We present a novel architecture that integrates graph attention networks within the Decision Transformer framework, jointly capturing spatial dependencies (which intersections should coordinate) and temporal dynamics (when to change phases for traffic progression). This unified approach outperforms both graph-based methods like CoLight (which lack temporal sequence modeling) and sequence-based methods like DTLight (which lack explicit graph structure). 3. Return-to-go conditioning for network-level optimization : We design a reward structure and return-to-go conditioning mechanism that enables specifying desired network-level performance at test time. This allows flexible control over optimization objectives and enables the same model to achieve different performance-efficiency tradeoffs. 4. Comprehensive evaluation and analysis : We demonstrate state-of-the-art performance on both synthetic grid networks and real-world traffic scenarios from Atlanta, GA and Boston, MA. Detailed ablation studies quantify the contribution of graph attention (8.4% improvement) and return conditioning (5.2% improvement), validating our architectural choices. The remainder of this paper is organized as follows. Section 2 reviews related work in traffic signal control, sequence modeling for RL, and multi-agent learning. Section 3 provides background on Dec-POMDPs, Decision Transformers, and graph attention networks. Section 4 presents our MADT architecture in detail. Section 5 describes our experimental setup and presents comprehensive results. Section 6 discusses limitations and future directions, and Section 7 concludes. 2 Related Work 2.1 Deep Reinforcement Learning for Traffic Signal Control Deep reinforcement learning has been extensively applied to traffic signal control, evolving from single-intersection methods to network-level coordination. 22 introduced IntelliLight, pioneering DQN-based control for single intersections with comprehensive state representations including queue lengths, waiting times, and traffic movement patterns. For multi-agent settings, 20 proposed PressLight, incorporating the theoretical guarantees of Max Pressure control ( 18 ) into the DRL reward design, achieving provably throughput-optimal coordination under certain conditions. A significant advancement came with CoLight ( 21 ) , the first method to use graph attention networks for traffic signal control. By modeling intersections as nodes in a graph

Read the complete text