Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD
Ruthvik Bokkasam, Shankar Gangisetty, A. H. Abdul Hafez, C. V. Jawahar · 2025
Read Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD by Ruthvik Bokkasam, Shankar Gangisetty, A. H. Abdul Hafez, C. V. Jawahar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22111.txt.
Introduction
Ruthvik Bokkasam and co-authors introduce a dataset for predicting pedestrian behaviour in unstructured Indian traffic, in work from 2025. Self-driving systems must forecast where people will walk, yet most datasets come from orderly traffic with signals and marked crossings. Indian streets add occlusions, mixed vehicles, and unsignalised scenes.
The dataset carries rich annotations, from broad scene labels down to fine details, focused on pedestrians the vehicle must attend to. Leading intention prediction methods lost up to fifteen percent in performance on this data compared with standard datasets. Trajectory methods fared far worse, with errors rising by an order of magnitude. The team provides full quantitative and qualitative analysis of baseline methods. They offer the dataset as a harder testbed for safety research.
Catalog record
| CREATOR | Ruthvik Bokkasam, Shankar Gangisetty, A. H. Abdul Hafez, C. V. Jawahar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-22111
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD Ruthvik Bokkasam, Shankar Gangisetty, A. H. Abdul Hafez, C. V. Jawahar Abstract With the rapid advancements in autonomous driving, accurately predicting pedestrian behavior has become essential for ensuring safety in complex and unpredictable traffic conditions. The growing interest in this challenge highlights the need for comprehensive datasets that capture unstructured environments, enabling the development of more robust prediction models to enhance pedestrian safety and vehicle navigation. In this paper, we introduce an Indian driving pedestrian dataset designed to address the complexities of modeling pedestrian behavior in unstructured environments, such as illumination changes, occlusion of pedestrians, unsignalized scene types and vehicle-pedestrian interactions. The dataset provides high-level and detailed low-level comprehensive annotations focused on pedestrians requiring the ego-vehicle’s attention. Evaluation of the state-of-the-art intention prediction methods on our dataset shows a significant performance drop of up to 𝟏𝟓 % \mathbf{15\%} , while trajectory prediction methods underperform with an increase of up to 𝟏𝟐𝟎𝟖 \mathbf{1208} MSE, defeating standard pedestrian datasets. Additionally, we present exhaustive quantitative and qualitative analysis of intention and trajectory baselines. We believe that our dataset will open new challenges for the pedestrian behavior research community to build robust models. Project Page: https://cvit.iiit.ac.in/research/projects/cvit-projects/iddped I Introduction Autonomous vehicles and ADAS must smoothly interact with pedestrians, avoiding collisions, ensuring safe crossings, and promoting efficient movement, especially in dense and dynamic environments [ 1 , 2 ] . In traffic environments, well-established infrastructure like signals and crosswalks simplify pedestrian behavior prediction. However, the lack of such infrastructure makes pedestrian behavior far more unpredictable [ 3 ] . Ego-vehicles must be capable of anticipating sudden changes in pedestrian direction, speed, or intent as real-time predictions of pedestrian intentions and trajectories are critical for enhancing road safety and ensuring smooth navigation in these chaotic and unpredictable traffic settings. Recent advances in deep learning have significantly improved the perception and prediction of dynamic object motion in structured settings, with several datasets developed for predicting pedestrian intentions and trajectories. Datasets like Argoverse [ 4 ] , nuScenes [ 5 ] , IDD-3D [ 6 ] and Waymo [ 7 ] focus on dynamic agent interactions but are primarily designed for driving perception and planning, not pedestrian intention and navigation. Datasets such as UCY [ 8 ] , ETH [ 9 ] , and SSD [ 10 ] provide pedestrian trajectories from top-down views, lacking driver-pedestrian interaction insights on the ground. Although vehicle-pedestrian interaction datasets like JAAD [ 11 ] , PIE [ 12 ] , TITAN [ 13 ] , PSI [ 14 ] , Euro-PVI [ 15 ] and TBD [ 16 ] target intention and trajectory prediction, they are tailored to structured environments with signals and crosswalks, and do not account for diverse pedestrian behaviors in unstructured settings. On chaotic urban streets without signals, autonomous vehicles face pedestrian behaviors like jaywalking or sudden halts. Anticipating these actions is crucial for safe navigation and collision avoidance, as conventional traffic rules often don’t apply in such settings. Considering the complexity of pedestrian behavior in unstructured traffic environments, we identify four key factors that impact prediction accuracy: (i) occlusions , (ii) lighting conditions , (iii) signalized type , and (iv) vehicle-pedestrian interactions . These factors add unpredictability, demanding robust models to handle such diverse scenarios. However, most existing datasets overlook these nuances, limiting their effectiveness in modeling pedestrian behavior in these challenging conditions. Figure 2 : Rolling behavior is common in unstructured environments, where pedestrians hesitate, pause, or change direction unpredictably, even after they start to cross. (left) In our dataset, due to rolling behaviour, the pedestrian does not cross and the ego-vehicle moves ahead, while (right) in PIE, the pedestrian crosses since structured settings typically prompt the ego-vehicle to yield giving priority to the pedestrian. Now, let us understand these four key factors that are crucial within unstructured traffic conditions, see scenarios in Fig. and comparison in Table I ): (i) Occlusion : Dense and dynamic traffic often occludes pedestrians, and datasets like TITAN [ 13 ] attempt to bypass this by removing occluded regions, which degrades pedestrian intent and trajectory predictions. (ii) Illumination variation : The variability in lighting conditions can significantly impact pedestrian behaviour predictions. PIE and TITAN lack nightlight data, while a dataset like NightOwls [ 17 ] lacks daylight data, degrading the performance across varying lighting conditions. (iii) Signalized type : The absence of traffic signals and crosswalks especially in unstructured settings lead to unpredictable pedestrian-vehicle interactions. JAAD and PIE are captured in scenes with well-established infrastructure including signals and crosswalks simplifying pedestrian behaviour predictions. (iv) Vehicle-Pedestrian Interactions : Datasets like PIE and JAAD lack interaction information, which reduces their effectiveness in modeling pedestrian behaviors. In contrast, unstructured environments present more frequent and complex interactions [ 18 ] . For example, “rolling behavior” [ 19 ] describes pedestrians constantly adjusting their speed and path to navigate traffic, leading to more frequent interactions. As shown in Fig. 2 , weak traffic rule enforcement in unstructured environments results in minimal vehicle yielding, even at crosswalks. While TITAN [ 13 ] and Euro-PVI [ 15 ] capture vehicle-pedestrian interactions, they lack explicit annotations, limiting their ability to model these behaviors accurately. To bridge these gaps, we introduce the IDD-PeD dataset. The dataset contains over 650K annotated bounding boxes and provides comprehensive annotations for pedestrian intention, location, ego-vehicle motion, scene context, occlusion, and pedestrian-vehicle interactions. By capturing these factors, IDD-PeD supports the development of robust models capable of navigating complex, real-world unstructured environments. Fig. depicts samples from the proposed dataset comprising of the diverse challenges and Table I compares our dataset over the existing datasets addressing these challenges. In summary, we make the following contributions: • We introduce the largest dataset, i.e., IDD-PeD, with more than 650K pedestrian bounding boxes and 19 behavioral attributes, captured in dense and unstructured traffic environments. The dataset addresses key challenges such as Occlusions , Lighting Conditions , Signalized Types , and Vehicle-Pedestrian Interactions (see Fig. and Table I ). • We benchmark eight baseline methods for pedestrian intention prediction (PIP) and four trajectory prediction (PTP) against the IDD-PeD dataset, showing a performance drop of up to 15\% in PIP (see Table II ) and an increase in MSE of up to 1208 for deterministic models and 224 for stochastic models in PTP (see Table III ) compared to previous benchmark datasets. • We present a comprehensive quantitative and qualitative analysis of the key challenges highlighting their substantial impact on model robustness (see Tables IV and V ). Dataset #Pedestrians #Bounding Boxes #Annotated Frames #Behavioral Classes Traffic Density OBD Video Seq. Group Location Occlusion Nightlight Signalized Interaction JAAD [ 11 ] 2.8K 391K 75K 11 Low ✗ ✗ ✓ ✗ ✓ ✓ ✓ ✗ PIE [ 12 ] 1.8K 750K 293K 11 Low ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ TITAN [ 13 ] 8.6K NA 75K 33 High ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ STIP [ 20 ] 3.3K 350K 110K – Medium ✗ ✓ ✗ ✗ ✗ ✗ * ✗ ✗ PSI [ 14 ] 180 NA 26K – Low ✗ ✗ ✗ ✗ ✓ ✗ * ✗ ✗ IDD-PeD (Ours) 5K 686K 205K 19 High ✓ ✓