MAVEN-T: Multi-Agent enVironment-aware Enhanced Neural Trajectory predictor with Reinforcement Learning Wenchang Duan, Zhenguo Gao, Yaoliang Yu, Yi Shi Abstract Trajectory prediction remains a critical yet challenging component in autonomous driving systems, requiring sophisticated reasoning capabilities while meeting stringent real-time deployment constraints. While knowledge distillation has demonstrated effectiveness in model compression, existing approaches fail to preserve complex decision-making capabilities during distillation, particularly in dynamic multi-agent scenarios. This paper introduces MAVEN-T, a teacher-student framework achieving state-of-the-art trajectory prediction through complementary architectural co-design and progressive distillation. The teacher employs hybrid attention mechanisms for maximum representational capacity, while the student uses efficient architectures optimized for deployment. Knowledge transfer occurs via multi-granular distillation with adaptive curriculum learning that dynamically adjusts complexity based on performance. Critically, the framework incorporates reinforcement learning to break the ”imitation ceiling” of traditional distillation—enabling the student to verify, refine, and optimize teacher knowledge through dynamic environmental interaction, potentially achieving more robust decision-making than the teacher itself. Extensive experiments on NGSIM and highD datasets demonstrate 6.2× parameter compression and 3.7× inference speedup while maintaining state-of-the-art accuracy, establishing a new paradigm for deploying sophisticated reasoning models under resource constraints. I Introduction Autonomous driving has emerged as a representative application of intelligent perception and decision-making systems, with broad potential for improving transportation safety, efficiency, and mobility. However, Autonomous driving systems require sophisticated perception and decision-making capabilities while operating under stringent real-time constraints. Current trajectory prediction models face a fundamental trade-off between computational efficiency and reasoning sophistication, creating barriers to practical deployment in safety-critical applications. Autonomous driving systems require sophisticated perception and decision-making capabilities while operating under stringent real-time constraints. Current trajectory prediction models face a fundamental trade-off between computational efficiency and reasoning sophistication, creating barriers to practical deployment in safety-critical applications. Recent advances have demonstrated impressive performance through sophisticated architectures. Tamba introduced selective state-space models achieving linear computational complexity while maintaining prediction accuracy. However, current approaches still require substantial computational resources exceeding edge deployment constraints, as they assume a single model architecture must simultaneously optimize for both representational capacity and computational efficiency. Existing knowledge distillation methods in autonomous driving fail to preserve complex multimodal reasoning during model compression. Traditional approaches [ 2 , 13 ] focus on output-level knowledge transfer, neglecting critical intermediate decision-making processes, and employ fixed distillation strategies that cannot adapt to varying driving scenario complexity. More broadly, recent studies suggest that static one-shot imitation is often insufficient for complex reasoning, and that reward-guided optimization can further improve decision quality [ 19 , 18 ] . Current approaches suffer from three fundamental gaps: (1) Existing frameworks assume architectural similarity between teacher and student models, preventing exploitation of complementary design principles. (2) Fixed distillation objectives fail to adapt to dynamic driving scenario complexity. (3) Approaches neglect the hierarchical nature of driving decisions, from perceptual features to semantic reasoning. This work proposes Multi-Agent enVironment-aware Enhanced Neural Trajectory predictor MAVEN-T , addressing these limitations through three innovations: Complementary Architectural Design employing different principles for teacher (hybrid attention with Mamba blocks and shift-window attention) and student (GRU-based modeling with squeeze-and-excitation mechanisms). Progressive Adaptive Curriculum dynamically adjusting distillation complexity based on student performance and scenario characteristics. Multi-Granular Knowledge Distillation capturing knowledge transfer across perceptual, contextual, and semantic levels. The main contributions include: (1) A teacher-student framework with complementary architectural designs. (2) Progressive adaptive curriculum learning adjusting distillation complexity dynamically. (3) Multi-granular distillation objectives preserving complete decision-making capabilities. (4) Demonstration of significant computational reductions while maintaining performance on autonomous driving benchmarks. Extensive experiments demonstrate that MAVEN-T achieves substantial computational efficiency gains while preserving sophisticated reasoning capabilities, enabling practical deployment of advanced autonomous driving models in resource-constrained environments. II Related Work II-A Knowledge Distillation in Autonomous Driving Knowledge distillation for autonomous driving has gained attention for deploying sophisticated models in resource-constrained environments. Chen et al. [ 2 ] proposed S2TNet with transformer architectures but lacks efficient knowledge transfer for real-time deployment. Yuan et al. [ 23 ] introduced AgentFormer for multi-agent modeling, yet suffers from computational inefficiency on edge devices. Liu et al. [ 13 ] developed mmTransformer for multimodal prediction but fails to preserve complex reasoning in lightweight models. Huang et al. [ 6 ] highlighted the gap between sophisticated modeling and deployment constraints. Zhou et al. [ 25 ] presented HiVT with hierarchical attention but lacks progressive knowledge transfer strategies. Shi et al. [ 17 ] demonstrated advanced MTR capabilities but does not address efficient model compression without sacrificing decision-making quality. II-B Graph Neural Networks for Vehicle Interaction Modeling Graph neural networks show promise for modeling vehicle interactions but suffer from computational limitations. Liang et al. [ 12 ] introduced LaneGCN for actor-map interactions but is computationally intensive for real-time applications. Li et al. [ 11 ] proposed GRIP using GCNs but lacks sophisticated attention mechanisms for long-range dependencies. Kosaraju et al. [ 8 ] employed Social-BiGAT for trajectory forecasting but struggles with computational efficiency in multi-vehicle scenarios. Salzmann et al. [ 16 ] developed Trajectron++ achieving strong performance but requiring substantial computational resources. Gilles et al. [ 4 ] proposed GOHOME representing HD maps as graphs but lacks efficient knowledge distillation mechanisms. Li et al. [ 10 ] extended GRIP++ with dynamic edge weights but still suffers from scalability issues. II-C Hybrid Attention Mechanisms and Temporal Modeling Attention mechanisms for autonomous driving face computational complexity challenges. Mercat et al. [ 14 ] explored multi-head self-attention but failed to address quadratic complexity issues. Giuliari et al. [ 5 ] proposed encoder-decoder transformers but lacks linear scalability for extended temporal horizons. Nayakanti et al. [ 15 ] introduced Wayformer with multi-axis attention but struggles with computational efficiency. Chen et al. [ 1 ] proposed VNAGT combining variational approaches with graph transformers but lacks progressive distillation mechanisms for efficient deployment. II-D Progressive Learning and Parameter-Efficient Adaptation Curriculum learning applications lack adaptive mechanisms for knowledge distillation. Huang et al. [ 7 ] introduced GameFormer with level-k reasoning but does not provide adaptive curriculum progression based on student model performance. Parameter-efficient adaptation methods remain limited in autonomous driving contexts. Feng et al. [ 3 ] proposed MacFormer with map-aware attention but lacks low-rank adaptation strategies for efficient knowledge transfer. II-E Multimodal Fusion and Attention Transfer Multimodal fusion strategies face attention transfer challenges. Xu et al. [ 22 ] developed Tra2Tra with global attention but lacks multi-scale feature alignment for effective knowledge distillation. Li et al. [ 9 ] proposed RAIN combining reinforcement learning with attention but does not address progressive transfer of attention patterns. Zhou et al. [ 24 ] introduced GA-STT for spatial-temporal transformers but lacks sophisticated attention transfer mechanisms for preserving teacher model reasoning in lightweight architectures. III Methodology III-A Overall Architecture Design This work presents a novel teacher–student knowledge–distillation framework that addresses the computational constraints of real-time autonomous driving while preserving sophisticated reasoning capabilities. The approach leverages complementary architectural designs wherein the teacher model maximises representational capacity through hybrid attention mechanisms, whereas the student model achieves deployment efficiency via parameter-efficient adaptations. Fig. 1: Architecture overview of the proposed teacher–student framework for autonomous-driving policy learning. The teacher network (upper) incorporates surround-aware GATv2 encoding for inter-vehicle relationship modelling, hybrid Mamba–SWA mechanisms for spatiotemporal reasoning, and MoE-Transformer blocks for computational efficiency. The student network (lower) employs lightweight GRU–SE encoding and LoRA-parameterised policy heads for efficient knowledge transfer. Multi-level distillation enables feature alignment across intermediate representations while maintaining policy optimisation through environmental feedback. The proposed architecture, depicted in Fig. 1 , operates on multimodal observation sequences \mathcal{O}=\{o_{1},\dots,o_{T}\} where each timestep t encapsulates ego-vehicle dynamics s_{t}^{\text{ego}}\!\in\!\mathbb{R}^{d_{\text{ego}}} , surrounding-vehicle configurations \mathcal{S}_{t}=\{s_{t}^{i}\}_{i=1}^{N}\!\subset\!\mathbb{R}^{d_{\text{surr}}} , and contextual environmental states c_{t}\in\mathbb{R}^{d_{\text{env}}} . The framework learns policy mappings that optimise long-horizon driving performance while adhering to safety-critical constraints. The high-capacity teacher is defined as f_{\theta_{T}}(\mathcal{O})=\mathcal{D}_{T}^{\text{MoE}}\!\Bigl(\mathcal{E}_{T}^{\text{Hybrid}}\!\bigl(\mathcal{G}_{\text{GATv2}}(\mathcal{S}_{t},\mathcal{E}_{t}),\,\mathcal{O}\bigr)\Bigr), (1) whereas the lightweight student is f_{\theta_{S}}(\mathcal{O})=\pi_{\theta_{S}}\!\Bigl(\mathcal{E}_{S}^{\text{GRU}}\!\bigl(\mathcal{G}_{\text{GRU--SE}}(\mathcal{S}_{t}),\,\mathcal{O}\bigr)\Bigr), (2) with \pi_{\theta_{S}} denoting a policy head implemented by LoRA-adapted MLPs (replacing the original “Student Multimodal Decoder” so that the student directly outputs driving actions). III-B Surround-aware Graph Neural Encoder The surround-aware encoder addresses the fundamental challenge of modeling complex inter-vehicle relationships in dynamic driving environments. This component processes multi-vehicle trajectory data through a graph neural network architecture that captures both spatial proximity and temporal dependencies between vehicles. The encoder operates on context matrices derived from multi-vehicle trajectory sequences, where each vehicle i at timestep t is represented by state vector s_{t}^{i}=[x_{t}^{i},y_{t}^{i},v_{t}^{i},\theta_{t}^{i},a_{t}^{i}] encompassing position, velocity, orientation, and acceleration. These states are aggregated into context matrices \mathbf{C}_{t}\in\mathbb{R}^{N\times d_{context}} that encode spatiotemporal relationships across the vehicle ensemble. The graph construction process establishes dynamic adjacency matrices \mathbf{A}_{t}\in\mathbb{R}^{N\times N} based on spatial proximity and interaction relevance: \mathbf{A}_{t,ij}=\begin{cases}\exp\left(-\frac{d_{ij}^{2}}{2\sigma^{2}}\right)&\text{if }d_{ij}