Occlusion-Aware Diffusion Model for Pedestrian Intention Prediction
Yu Liu, Zhijie Liu, Zedong Yang, You-Fu Li, He Kong · 2025
Read Occlusion-Aware Diffusion Model for Pedestrian Intention Prediction by Yu Liu, Zhijie Liu, Zedong Yang, You-Fu Li, He Kong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00858.txt.
Introduction
Yu Liu and colleagues propose an occlusion aware diffusion model for predicting whether a pedestrian is about to cross the street. Drivers and robots need that forecast, but people are often partly hidden behind cars or other obstacles, and most earlier models ignore the missing information.
Their model reconstructs the hidden motion patterns and uses them to guide the forecast. A diffusion transformer estimates noise features tied to the occluded parts, A reverse process guided by the occlusion mask keeps the reconstruction faithful to what was seen. That limits error build up. On the PIE and JAAD benchmarks, across many occlusion levels, the method proved more robust than current approaches.
Catalog record
| CREATOR | Yu Liu, Zhijie Liu, Zedong Yang, You-Fu Li, He Kong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00858
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Occlusion-Aware Diffusion Model for Pedestrian Intention Prediction Yu Liu, Zhijie Liu, Zedong Yang, You-Fu Li, He Kong Abstract Predicting pedestrian crossing intentions is crucial for the navigation of mobile robots and intelligent vehicles. Although recent deep learning-based models have shown significant success in forecasting intentions, few consider incomplete observation under occlusion scenarios. To tackle this challenge, we propose an Occlusion-Aware Diffusion Model (ODM) that reconstructs occluded motion patterns and leverages them to guide future intention prediction. During the denoising stage, we introduce an occlusion-aware diffusion transformer architecture to estimate noise features associated with occluded patterns, thereby enhancing the model’s ability to capture contextual relationships in occluded semantic scenarios. Furthermore, an occlusion mask-guided reverse process is introduced to effectively utilize observation information, reducing the accumulation of prediction errors and enhancing the accuracy of reconstructed motion features. The performance of the proposed method under various occlusion scenarios is comprehensively evaluated and compared with existing methods on popular benchmarks, namely PIE and JAAD. Extensive experimental results demonstrate that the proposed method achieves more robust performance than existing methods in the literature. I Introduction With the rapid advancement of intelligent sensing and computing technologies, much progress has been made in recent years in developing autonomous vehicles to enhance traffic efficiency and road safety. To prevent collisions, path planning of autonomous vehicles [ 1 , 2 ] is essential, requiring an understanding of interactions between road users and the ability to forecast their potential actions [ 3 , 4 , 5 ] . Fig. 1: The typical scenario of visual occlusion is illustrated here. Solid green lines represent the parts of the observation that are within the field of view and visible, while dashed red lines indicate positional features that are undetectable due to occlusion. Observations with occlusion pose a significant challenge for pedestrian intention prediction. Given the observed traffic environment information and the historical motion data of individuals, pedestrian intention refers specifically to the binary decision of whether a pedestrian intends to cross the road within a short future time. One of the key challenges in intention prediction is the probabilistic and multimodal nature of road users’ behaviors, which are influenced by a variety of factors [ 6 ] . Recently, several methods have been proposed to incorporate multimodal data to estimate pedestrian intention, including, for example, visual context information [ 7 , 8 ] , ego vehicle velocity [ 9 ] , bounding box annotation [ 10 , 11 ] , and pose information [ 12 ] . However, visual occlusions, which frequently occur in practical applications can cause onboard sensors to fail in detecting pedestrians’ current states in complex environments, as illustrated in Figure 1 . The inaccessibility of certain modalities may prevent vehicles from accurately estimating pedestrians’ future motion intentions, posing a risk to their safe operation. Existing intention prediction methods typically rely on complete multimodal observation sequences and do not consider occluded observations. While a few motion prediction studies [ 13 , 14 ] consider incomplete observations, they primarily focus on observable sequences and overlook the impact of missing motion patterns. To the best of our knowledge, learning-based prediction of pedestrian crossing intentions in occlusion scenarios has not been well-explored. From this perspective, in this work, we propose an Occlusion-Aware Diffusion Model for Intention Prediction to address the challenges of intention prediction in occluded scenarios. Our framework integrates an occlusion-masked diffusion transformer with an occlusion mask-guided inference process. By leveraging multimodal inputs, such as bounding boxes and ego-vehicle velocity, the model reconstructs occluded motion features before estimating pedestrian crossing intentions. The occlusion masking strategy is designed to handle incomplete pedestrian motion observations caused by occlusion in traffic environments. Unlike token masking in NLP or conventional diffusion models, our approach directly targets motion features, enabling the diffusion model to recover and learn from occluded patterns for more accurate intention prediction. This method effectively captures both motion and occlusion patterns, allowing the model to reconstruct missing features while predicting future intentions. The contributions of this paper are as follows: (1) We present a diffusion-based framework tailored for pedestrian intention prediction under occlusion scenarios. Unlike most existing studies that implicitly assume full visibility of pedestrian motion cues, our framework explicitly addresses the challenge of incomplete observations caused by occlusions, a problem insufficiently emphasized in prior work. (2) We propose two key technical modules: the occlusion-masked diffusion transformer and the occlusion mask-guided reverse process. The occlusion-masked diffusion transformer that embeds semantic occlusion information into the denoising process, along with an occlusion mask-guided reverse process that leverages partial observations to mitigate error. Together, these designs enable more accurate reconstruction of motion features and intention prediction under occlusion. (3) Extensive experiments have been conducted to thoroughly evaluate the performance of the proposed approach. The experimental results on the PIE and JAAD datasets demonstrate that our method consistently achieves improved performance in terms of intention prediction under occlusion scenarios compared to state-of-the-art approaches [ 15 , 16 ] . The remainder of this article is organized as follows: Section II reviews related methods for intention prediction and diffusion models. Section III outlines the proposed methodology. Section IV describes the experimental setup and presents the evaluation results. Section V concludes the paper. II Related Works II-A Pedestrian Intention Prediction For intention prediction, early methods relied on dynamic physical cues, such as walking trajectories [ 17 ] , trajectory endpoints [ 18 ] , or pedestrian head orientations [ 19 ] , to estimate crossing intentions. However, the limited and often simplistic incorporation of contextual features made these models vulnerable to challenges in complex, dynamic environments. More recently, with the advent of benchmark datasets [ 20 , 21 ] for pedestrian intention prediction, data-driven models have become a dominant approach in this field. For instance, multiple Recurrent Neural Network (RNN) branches have been employed to process different feature sources [ 22 ] . In SFRNN [ 23 ] , a Stacked RNN is used to hierarchically process features, gradually fusing them at each level to capture increasingly complex representations. PCPA [ 15 ] proposed a Convolutional Neural Network (CNN) to extract visual features from scenes and RNNs to process non-image features for predicting crossing intentions. Inspired by the role of human pose in motion prediction tasks, [ 24 ] integrated image-based 2D human pose features into the model to estimate crossing intentions. Similarly, pedestrian pose features were employed in [ 25 ] to predict pedestrian crossing behaviors at intersections. To better capture spatial interactions, Graph Neural Networks (GNNs) [ 26 ] have been introduced to address intention prediction tasks by leveraging the spatiotemporal context of scenes. GNNs model interactions within scenarios using a topological graph structure, where dynamic objects, such as pedestrians and vehicles, are represented as graph nodes, and their interactions are captured as graph edges through adjacency matrices. For instance, ST CrossingPose [ 27 ] proposed predicting pedestrian crossing intentions using spatial-temporal graph convolutional networks applied to skeleton data, effectively learning both spatial and temporal patterns. Additionally, Pedestrian Graph [ 28 ] incorporated two convolutional modules to provide supplementary contextual information to the primary graph convolutional module, thereby enhancing prediction accuracy. Furthermore, the attention mechanism has been effectively utilized to capture semantic relationships within the data. For