Dynamic Occlusion–Predictive Neural Network for Robust Roadside Multi-Vehicle Tracking Authors: Shuai Wang, Yafei Wang, Bowen Wang, Chongfeng Wei, Hao Liu Published: 2026 ABSTRACT Highlights What are the main findings? Dynamic Occlusion State Prediction via Transformer: We propose a Transformer-based framework that explicitly forecasts the future occlusion ratio of targets by modeling historical trends. By integrating this prediction as a dynamic weighting factor into the loss function, our model adaptively learns to mitigate the impact of varying occlusion severities, significantly enhancing tracking stability during continuous state changes.Physical-Constraint Reasoning via GNN: We develop a GNN-based module that leverages road occupancy and neighboring vehicle poses to infer the existence and motion patterns of targets within occluded regions. This module constructs a heterogeneous graph to determine scene physics, effectively linking fragmented trajectories by generating “virtual perception” states for invisible targets. What are the implications of the main findings? Implication of Dynamic Occlusion State Prediction: Introducing predicted occlusion states into the optimization objective shifts the paradigm from passive reaction to proactive anticipation. This implies that explicitly modeling the dynamics of visibility is crucial for robust tracking, as it allows the system to maintain trajectory continuity even when targets undergo rapid and severe transitions between visible and occluded states.Implication of Physical-Constraint Reasoning: Inferring target existence through road occupancy and neighbor interactions demonstrates that scene priors can effectively substitute for missing sensory data. This creates a “reasoning-based tracking” capability that overcomes the physical limitations of roadside sensors, ensuring high-precision association and minimizing ID switches even in fully occluded or “blind” spots. AbstractDespite their extended detection ranges and superior precision compared with onboard sensors, roadside perception systems suffer from severe occlusion artifacts in complex traffic, causing significant tracking failures and ID switches. To address this, we propose a novel Dynamic Occlusion–Predictive Neural Network tailored to challenging roadside environments. First, we introduce a Transformer-based Dynamic Occlusion State Predictor to explicitly model the temporal evolution of occlusion. Unlike traditional tracking methods, this module continuously forecasts future occlusion ratios for each target by analyzing historical occlusion patterns. Critically, these predictions are integrated into the tracking framework as dynamic weighting factors in the loss function, enabling the model to adaptively penalize tracking errors based on the predicted occlusion severity and significantly enhancing robustness against dynamic occlusion scenarios. Second, leveraging the predicted occlusion states, we propose a GNN-based Spatial Reasoning Module to address trajectory fragmentation. This module constructs a heterogeneous graph integrating road occupancy information and neighboring vehicle poses to infer the existence and motion patterns of targets within occluded regions. By analyzing scene-level physical constraints, it generates motion predictions for invisible targets and links these inferred states to fragmented trajectories, ensuring temporally continuous tracking even during prolonged visual occlusions. Experiments on the DAIR-V2X and our self-collected roadside dataset show that our framework outperforms state-of-the-art methods in precision and robustness, achieving a 5.1% MOTA gain over the best baseline. This advantage peaks under high occlusion, where preserving ID continuity and minimizing failures validates its efficacy for real-world roadside multi-target tracking. 1. Introduction Roadside perception systems equipped with advanced sensor arrays offer extended detection ranges and superior precision compared with onboard sensors, positioning them as a cornerstone for intelligent transportation systems [1,2,3]. However, this “bird’s-eye” advantage is severely limited in long-range, high-density traffic scenarios, where the unique geometry of roadside views introduces distinct challenges. Unlike onboard perspectives, roadside sensors suffer from extreme perspective distortions at a distance, where vehicles appear as sparse, indistinguishable clusters. This leads to a highly dynamic occlusion environment: targets frequently transition between partial and full occlusion states due to the dense packing of traffic and the specific viewing angle. These rapid fluctuations in visibility, combined with feature sparsity caused by long-range distances, create a noisy observation space where standard tracking filters struggle to maintain continuity, leading to a significant degradation in precision and frequent identity (ID) switches [4,5]. The core difficulty in robust roadside tracking stems from the ill-posed nature of tracking under dynamic and total occlusion. First, the temporal instability of occlusion states, where a target’s visibility ratio changes drastically from frame to frame, makes it difficult for traditional models to distinguish between actual trajectory deviations and observation noise. Without a mechanism to explicitly anticipate these changes, tracking models often fail to adapt their confidence levels dynamically. Second, in scenarios of total occlusion caused by dense traffic, the target’s motion priors (velocity, acceleration) become completely unobservable. Traditional methods, which rely heavily on historical motion states, fail here because they lack the contextual reasoning capability to infer the missing motion patterns from the surrounding environment (e.g., utilizing the “social forces” of neighboring vehicles or the geometric constraints of the road). Consequently, preserving target identity through these periods of invisibility remains a formidable challenge, often resulting in irreversible track fragmentation [6,7]. While concerted efforts have been directed toward mitigating occlusion, existing state-of-the-art methods struggle to generalize in roadside scenarios. Recent Transformer-based trackers [8,9,10] excel at global sequence modeling but often treat occlusion as a passive “missing data” problem, relying on interpolation rather than active reasoning. While effective in onboard settings, they lack a mechanism to explicitly model the dynamic evolution of occlusion states, rendering them vulnerable to the rapid visibility fluctuations typical of roadside views. Conversely, Graph Neural Network (GNN) approaches [11,12,13] model inter-object interactions but typically focus on social grouping or simple collision avoidance. They frequently overlook the physical constraints of the road geometry (e.g., lane occupancy), which are critical for inferring the state of a target when visual features are sparse. Consequently, without explicitly leveraging road topology or predicting future occlusion severity, these methods fail to maintain robust identity consistency during severe and prolonged occlusion events [14,15]. In this paper, to bridge these gaps, we propose a Dynamic Occlusion–Predictive Neural Network tailored for robust roadside multi-vehicle tracking. Unlike reactive paradigms that merely compensate for detection failures post hoc, our framework treats occlusion as a predictable physical state. We introduce a dual-component architecture designed to handle the specific challenges of the roadside perspective: (1) Proactive State Prediction: We employ a Transformer-based Occlusion State Predictor that not only captures long-range temporal dependencies to filter stochastic noise but also explicitly forecasts future occlusion ratios. This prediction is fed back as a dynamic loss weight, allowing the model to adaptively prioritize targets at risk of disappearing. (2) Physical-Constraint Reasoning: We develop a GNN-based Spatial Reasoning Module that goes beyond simple social pooling. By constructing a heterogeneous graph incorporating road occupancy and geometric constraints, the module infers the existence and motion of invisible targets based on the physical plausibility of the scene. This transforms the ill-posed problem of missing priors into a solvable inference task, ensuring precise association even under complete visibility loss. The main contributions of this work are summarized as follows:(1)Dynamic Occlusion State Prediction via Transformer with Feedback-Driven Loss Optimization: We propose a novel Transformer-based framework that explicitly models the temporal evolution of occlusion. Unlike standard tracking methods that treat occlusion as a static label, our approach forecasts the future occlusion ratio by analyzing historical trends across frames. Crucially, we introduce a mechanism where this predicted state is fed back into the tracking framework as a dynamic weighting factor within the loss function. This allows the model to adaptively adjust the penalty for prediction errors based on the predicted severity of occlusion, significantly enhancing the network’s robustness against targets undergoing continuous and dynamic state changes.(2)Scene-Aware Virtual Perception via GNN for Existence Inference and Trajectory Linking: Building upon the predicted occlusion states, we develop a Graph Neural Network (GNN) module designed for reasoning under uncertainty. This module constructs a heterogeneous graph incorporating road occupancy and neighboring vehicle poses to explicitly infer the existence of targets within occluded regions. By analyzing these physical constraints, the model predicts the latent motion patterns of invisible targets and generates “virtual features” to bridge trajectory gaps. This approach effectively transforms the tracking problem from simple feature matching to physics-constrained inference, ensuring continuous trajectory linking and minimizing ID switches even when targets are completely obscured.(3)State-of-the-Art Performance on Benchmark Datasets: We conduct extensive evaluations using the large-scale DAIR-V2X dataset and a self-collected complex urban dataset. The quantitative results demonstrate that our method achieves a MOTA of 92.6% on DAIR-V2X and 80.6% on the self-collected complex dataset, outperforming current state-of-the-art methods. Notably, our approach reduces identity switches (IDSs) by 37% and trajectory fragmentation (FRAG) by 25% compared with the second-best method (AGO-Net), validating the superior robustness of our temporal–spatial fusion architecture in maintaining track continuity. (1)Dynamic Occlusion State Prediction via Transformer with Feedback-Driven Loss Optimization: We propose a novel Transformer-based framework that explicitly models the temporal evolution of occlusion. Unlike standard tracking methods that treat occlusion as a static label, our approach forecasts the future occlusion ratio by analyzing historical trends across frames. Crucially, we introduce a mechanism where this predicted state is fed back into the tracking framework as a dynamic weighting factor within the loss function. This allows the model to adaptively adjust the penalty for prediction errors based on the predicted severity of occlusion, significantly enhancing the network’s robustness against targets undergoing continuous and dynamic state changes. Dynamic Occlusion State Prediction via Transformer with Feedback-Driven Loss Optimization: We propose a novel Transformer-based framework that explicitly models the temporal evolution of occlusion. Unlike standard tracking methods that treat occlusion as a static label, our approach forecasts the future occlusion ratio by analyzing historical trends across frames. Crucially, we introduce a mechanism where this predicted state is fed back into the tracking framework as a dynamic weighting factor within the loss function. This allows the model to adaptively adjust the penalty for prediction errors based on the predicted severity of occlusion, significantly enhancing the network’s robustness against targets undergoing continuous and dynamic state changes. (2)Scene-Aware Virtual Perception via GNN for Existence Inference and Trajectory Linking: Building upon the predicted occlusion states, we develop a Graph Neural Network (GNN) module designed for reasoning under uncertainty. This module constructs a heterogeneous graph incorporating road occupancy and neighboring vehicle poses to explicitly infer the existence of targets within occluded regions. By analyzing these physical constraints, the model predicts the latent motion patterns of invisible targets and generates “virtual features” to bridge trajectory gaps. This approach effectively transforms the tracking problem from simple feature matching to physics-constrained inference, ensuring continuous trajectory linking and minimizing ID switches even when targets are completely obscured. Scene-Aware Virtual Perception via GNN for Existence Inference and Trajectory Linking: Building upon the predicted occlusion states, we develop a Graph Neural Network (GNN) module designed for reasoning under uncertainty. This module constructs a heterogeneous graph incorporating road occupancy and neighboring vehicle poses to explicitly infer the existence of targets within occluded regions. By analyzing these physical constraints, the model predicts the latent motion patterns of invisible targets and generates “virtual features” to bridge trajectory gaps. This approach effectively transforms the tracking problem from simple feature matching to physics-constrained inference, ensuring continuous trajectory linking and minimizing ID switches even when targets are completely obscured. (3)State-of-the-Art Performance on Benchmark Datasets: We conduct extensive evaluations using the large-scale DAIR-V2X dataset and a self-collected complex urban dataset. The quantitative results demonstrate that our method achieves a MOTA of 92.6% on DAIR-V2X and 80.6% on the self-collected complex dataset, outperforming current state-of-the-art methods. Notably, our approach reduces identity switches (IDSs) by 37% and trajectory fragmentation (FRAG) by 25% compared with the second-best method (AGO-Net), validating the superior robustness of our temporal–spatial fusion architecture in maintaining track continuity. State-of-the-Art Performance on Benchmark Datasets: We conduct extensive evaluations using the large-scale DAIR-V2X dataset and a self-collected complex urban dataset. The quantitative results demonstrate that our method achieves a MOTA of 92.6% on DAIR-V2X and 80.6% on the self-collected complex dataset, outperforming current state-of-the-art methods. Notably, our approach reduces identity switches (IDSs) by 37% and trajectory fragmentation (FRAG) by 25% compared with the second-best method (AGO-Net), validating the superior robustness of our temporal–spatial fusion architecture in maintaining track continuity. The remainder of this paper is organized as follows, with the overall framework illustrated in Figure 1. Following the introduction of related work in Section 2, the core methodology is detailed in Section 3 and Section 4. As depicted in the pipeline, the system takes voxelized point clouds and historical trajectories as input. Section 3 presents the Transformer-based global temporal modeling module. This component addresses the temporal instability of occlusion by encoding the historical trajectory of each target, predicting the dynamic evolution of the occlusion state, and constructing a dynamic loss function to suppress stochastic noise caused by abrupt maneuvers, ensuring robust tracking continuity (as shown in the “Dynamic trajectory tracking” panel). Subsequently, Section 4 introduces the GNN-based Interaction Module designed to recover fragmented trajectories. By constructing a heterogeneous graph that incorporates road occupancy and neighboring vehicle poses, this module infers the existence and latent motion patterns of occluded targets. This mechanism effectively links trajectory fragments under severe occlusion (illustrated in the “Fragmented trajectory tracking” panel), transforming ambiguous associations into a structured inference task. Finally, Section 5 validates the proposed framework through extensive experiments, demonstrating significant improvements in tracking precision and ID consistency. Section 6 concludes the paper and discusses future research directions. 2. Related Work on Occlusion-Aware Tracking Methods In this section, we review the evolution of 3D multi-object tracking, establishing the theoretical foundations of this work. We scrutinize the dominant paradigms, specifically detection-driven association and Bayesian filtering, and their limitations in modeling non-linear motions and multi-modal uncertainties. We highlight how the unimodal Gaussian assumption in traditional filters degrades performance in complex scenarios and how recent hybrid architectures bridge probabilistic inference with deep learning. Building upon this analysis, we address the critical bottleneck of occlusion-aware tracking. While existing methods leverage temporal smoothing, prediction, and semantic priors to mitigate fragmentation, they lack a unified mechanism for proactive occlusion reasoning. This gap in predictive state estimation is the motivation for the proposed framework that integrates spatial topologies with temporal dynamics to forecast occlusion states. 2.1. 3D Multi-Object Tracking Paradigms 3D multi-object tracking (MOT) methodologies are generally categorized into detection-driven paradigms and Bayesian state estimation frameworks [16,17]. While detection-driven approaches utilizing 3D CNNs or PointNet-like architectures have achieved significant progress in feature extraction [18], they often treat tracking as a static association problem, neglecting the temporal continuity essential for handling roadside occlusion. Conversely, Bayesian filtering frameworks, such as Kalman filters (KFs) and their non-linear extensions (EKF, UKF), model target dynamics by propagating state means and covariances [19]. However, a critical limitation of these parametric Bayesian methods lies in their strict unimodal Gaussian assumption. In roadside scenarios characterized by severe occlusion and long-range perspective distortion, the target’s state distribution is inherently multi-modal (e.g., a vehicle could be behind an obstacle or simply missed by the sensor). The unimodal assumption fundamentally precludes accurate modeling of these complex uncertainties [20]. Although Sequential Monte Carlo (SMC) methods (Particle Filters) were introduced to approximate arbitrary distributions, they incur exponential computational complexity and suffer from particle degeneracy in high-density traffic [21,22]. Recently, the field has shifted towards integrating deep learning with probabilistic inference. For instance, in 2025, K Chen et al. proposed an uncertainty-based roadside point cloud detection and tracking method to enhance perceptual robustness [23]. While this represents a significant step forward in acknowledging sensor noise, it primarily focuses on spatial uncertainty at the detection level and lacks a mechanism to model the temporal evolution of occlusion, failing to address the specific challenge of rapid visibility changes caused by the unique “bird’s-eye” perspective of roadside sensors. Consequently, the existing methods remain reactive, struggling to maintain trajectory continuity when targets transition abruptly between visible and fully occluded states. 2.2. Occlusion-Aware Tracking Methods Robust tracking in occluded scenarios remains a bottleneck, primarily due to feature sparsity and ambiguous data association in crowded environments [24,25,26]. The existing literature addresses these challenges through three main paradigms, each with inherent limitations in the context of roadside perception. First, temporal modeling approaches leverage Recurrent Neural Networks (RNNs) like LSTMs to capture motion dynamics [27,28,29]. However, their sequential processing nature limits parallelization and prevents them from capturing the global spatial–temporal context required to resolve complex occlusions. Second, semantic-aware methods incorporate priors such as lane geometry or social forces [30,31]. While helpful, these approaches often treat environmental constraints as static regularizers rather than dynamic predictors, while relying on observable cues and struggling to infer latent motion patterns when targets are fully occluded. More recently, communication-aware approaches have emerged to tackle environmental challenges. Notably, in 2025, A Rehman et al. proposed an uncertainty-based roadside point cloud detection and tracking method that utilizes intelligent reflecting surfaces to mitigate signal blockage in Non-Line-of-Sight (NLOS) conditions [32]. While effective for communication connectivity, such physical-layer solutions do not resolve the perceptual ambiguity in computer vision tasks and do not provide the semantic understanding or trajectory prediction capabilities necessary for a roadside perception system to “see through” occlusions via algorithmic inference. Crucially, a significant gap remains in proactive occlusion modeling. The current methods are largely reactive, compensating for missed detections after they occur, and there is a lack of frameworks that can explicitly predict the dynamic evolution of occlusion states (e.g., predicting when a vehicle will emerge from behind a bus) by jointly leveraging temporal history and spatial interactions. To bridge this gap, our work departs from passive observation by fusing a Transformer-based temporal encoder with a GNN-based interaction module, enabling the proactive forecasting of occlusion states and robust trajectory recovery. 3. Dynamic Occlusion State Prediction via Spatiotemporal Transformer The fundamental challenge of 3D MOT under occlusion is bridging noisy observations with continuous physical motion. Traditional methods fail to capture multi-modal scenarios or maintain identity consistency during occlusion. In this section, we introduce a predictive framework that treats occlusion as a physically constrained state transition, integrating a Spatiotemporal Transformer with Occlusion-Aware Attention to model temporal dependencies and a dynamic loss function to enforce motion inertia. In this section, we present the proposed Occlusion-Predictive Spatiotemporal Transformer, whose overall architecture and information flow are illustrated in Figure 2. Aligned with the methodology detailed in this chapter, the framework is built upon three core components: (1) Occlusion State Prediction (Section 3.1), which leverages historical observations to estimate occlusion probabilities and employs a semi-supervised temporal consistency mechanism to hallucinate features for occluded targets; (2) Global Spatiotemporal Attention (Section 3.2), designed to capture long-range dependencies via linear-transformed queries, keys, and values, while integrating an occlusion mask to refine detection-track association in complex scenarios; and (3) Trajectory Consistency (Section 3.3), which enforces robust tracking through space-time smoothing by optimizing trajectory alignment via an MSE-based consistency loss and a KL divergence-based occlusion prediction loss. This synergistic design provides a robust foundation for addressing occlusion challenges, with the specific mechanisms of each module elaborated in the following subsections. 3.1. Problem Formulation and Occlusion State Encoding 3.1.1. Occlusion State Transition We define the occlusion state st∈[0,1] at time t, where st≈0 indicates full visibility and st≈1 indicates full occlusion. To capture the non-linear motion dynamics that linear Kalman filters cannot represent, we employ a Long Short-Term Memory (LSTM) network. Let xt=pt,vt,at be the input vector at time t, comprising kinematic features (position p, velocity v) and appearance features a. The LSTM unit updates its internal hidden state ht and cell state ct based on the previous hidden state ht−text 1 and the current input xt. Let ht−1,xt denote the concatenation of the two vectors. The gate activations are computed as follows:(1)it=σWiht−1,xt+bift=σWfht−1,xt+bfot=σWoht−1,xt+boC˜t=tanhWCht−1,xt+bC Here, it, ft, and ot denote the input, forget, and output gates, respectively. σ is the sigmoid activation function, and ⊙ represents the Hadamard product. The forget gate ft plays a crucial role in enforcing temporal inertia by determining how much of the previous cell state Ct−1 (historical context) is retained. By learning to keep ft≈1 during stable motion, the model resists rapid, noisy fluctuations in the occlusion state. W and b are learnable weight matrices and bias vectors, respectively, while σ denotes the sigmoid function. The cell state Ct and hidden state ht are then updated as follows:(2)Ct=ft⊙Ct−1+it⊙C˜tht=ot⊙tanhCt The predicted occlusion probability s^t is derived from the hidden state ht:(3)s^t=σWs⋅ht+bs 3.1.2. Semi-Supervised Learning with Inertia Regularization A critical challenge is the scarcity of fully annotated occlusion labels in real-world datasets. To address this, we formulate a semi-supervised learning objective. The total loss function Ltotal consists of a supervised classification term Lcls and a novel inertia regularization term Linert:(4)Ltotal =Lcls s^t,stgt+λLinert The inertia loss Linert penalizes the L2-norm of the difference between consecutive predictions:(5)Linert =s^t−s^t−12 Physically, this enforces the principle that motion continuity–occlusion states should evolve smoothly unless interrupted by a drastic event (e.g., a sudden cut-in). The hyperparameter λ balances the tradeoff between responsiveness and stability. 3.1.3. Temporal Feature Hallucination via Visible Feature Bank When an object is fully occluded (st≈1), direct feature extraction fails. To maintain the “identity anchor,” we propose a Temporal Feature Hallucination mechanism. Unlike naive methods that carry over the last known feature (leading to ID drift), we synthesize the feature atsyn by attending to a Visible Feature Bank (VFB). The VFB is implemented as a First-In–First-Out (FIFO) queue, B=ai,li,ti, storing appearance features ai, lane IDs li, and timestamps ti of all visible neighbors. To construct the synthesized feature, we first select candidate neighbors N within the same lane (li=ltarget) and within a spatial threshold dmax:(6)N=i∈B∣pi−ptarget