Hybrid CNN–GCN network with dynamic gating and scale-aware preprocessing for pedestrian crossing intention prediction in Indian road conditions Authors: Rajesh Kannan Megalingam, Naveen Prasaad Selvarajan Published: 2026 ABSTRACT Pedestrian crossing intention prediction is a critical task for autonomous driving that requires high accuracy and real-time inference capability. In India, this is particularly challenging in unstructured and crowded road environments where occlusions, vulnerable road users, and non-compliant pedestrian behavior are very common. Existing methods are primarily trained on non-Indian datasets which often struggles to generalize to such conditions and rely on static combinations of visual and non-visual features, leading to a high computational complexity. To address these limitations, we propose a hybrid CNN-GCN based network with a dynamic modality gating module that adaptively prioritizes visual (pose, local and global context) and non-visual (motion, bounding boxes) features depending on the scene context. This enables the model to emphasize reliable modalities under visual degradation while maintaining computational efficiency. In addition, a novel scale-aware HRNet based preprocessing module with adaptive scaling is introduced to robustly extract pedestrians pose key points for distant and partially occluded pedestrians. To support evaluation in unstructured traffic scenarios, a custom Indian Road Dataset (IRD) is collected in Trivandrum city, India covering urban and suburban roads to capture diverse traffic scenarios with dense crowds, non-compliant pedestrian behaviors, adverse weather and frequent occlusions. The proposed model achieves moderate accuracy of 88% on JAAD_All dataset and 82% on the IRD dataset, while maintaining the lower inference latency (1.5ms on IRD and 4.6 ms on JAAD_All) among compared methods achieving a favorable trade-off between predictive performance and computational efficiency. Extensive experiments including ablation studies on modality gating, attention mechanisms, and scale-aware preprocessing, along with cross-dataset generalization analysis(PIE dataset) and failure case evaluation demonstrate the effectiveness of the proposed approach. Further analysis under varying conditions including occlusion, pedestrian scale variations, and modality weight adaptation shows that the model dynamically adjusts feature reliance and retains consistent predictive capability in visually degraded scenarios. Additionally, deployment on the Jetson Xavier NX demonstrates near real-time performance, highlighting the practical feasibility of the proposed method for autonomous driving applications. Introduction Autonomous vehicles (AV) aim to replace human drivers in navigating safely through complex road environments. AVs are expected to significantly improve road safety, mobility, and efficiency as the stats show human error makes up nearly 94% of crashes. However, AVs face major challenges in perception and decision-making, particularly in understanding pedestrian behavior. Pedestrian crossing intention prediction has emerged as an important task for preventing accidents1,2, yet it remains highly challenging due to the variability and unpredictability in pedestrian’s behavior3. The challenge is severe in India. According to the report published by the Ministry of Road Transport and Highways (MoRTH), the pedestrians account for nearly 20% of traffic fatalities making over 35,000 deaths in 2023, with fatalities increasing at 2.6% annually4. Road crossings are frequent hotspots for crashes5, and pedestrians often share space with vehicles due to the absence of crosswalks, sidewalks, and signalized infrastructure6,7. Unlike structured environments in North America and Europe, where datasets such as PIE8 and JAAD9 were collected, Indian roads are characterized by high density of pedestrians, frequent occlusions, and non-compliant pedestrian behavior, making direct transfer of existing models challenging. Recent advancements have explored LSTM10,11, GCN12,13, and Transformer14 based architectures, inspired by the self-attention mechanism introduced in15, for pedestrian intention prediction. However, three significant gaps remain. First, there is no dataset that captures the complexities of Indian road scenarios, limiting model generalizability. Second, multimodal fusion methods often merge visual and non-visual cues with fixed weights16–18, which is brittle under conditions such as occlusion, glare, and sensor degradation19,20. Third, although pose-based features are important for action and intention recognition21,22, existing pose estimation models degrade with small-scale, distant, and occluded pedestrians which are common in Indian traffic23,24. While HRNet preserves high-resolution representations25, several improved HRNet variants have also been explored to enhance pose estimation performance under challenging visual conditions26. However its performance drops with distant or partially visible pedestrians in cluttered environments27. Recent attention-augmented variants28 and robust perception frameworks29 show promise, but their application to unstructured traffic environments remains underexplored. Pedestrian intention prediction in unstructured traffic environments introduces additional challenges beyond those addressed in existing works. In particular, visual cues such as pose and appearance often become unreliable due to frequent occlusions and small-scale pedestrian instances, while motion-based cues alone may be insufficient to capture complex behavioral intent. Conventional multimodal fusion approaches typically rely on static or globally learned weights, which fail to adapt when the reliability of individual modalities varies across scenarios. As a result, existing methods struggle to maintain consistent performance in dense and visually degraded environments. To effectively address these challenges, it is necessary to design a framework that can adapt to varying modality reliability, improve feature extraction for small-scale and occluded pedestrians, and generalize to unstructured traffic environments. This requires both adaptive fusion mechanisms30 and robust preprocessing strategies tailored to real-world conditions. To address these challenges, we propose a hybrid CNN–GCN framework with a dynamic modality gating mechanism that performs sample-dependent, reliability-aware fusion of visual (pose keypoints) and non-visual (motion dynamics) features. Unlike conventional fusion strategies with fixed or globally learned weights, the proposed gating module adaptively prioritizes modalities based on their reliability, particularly under occlusion and small-scale pedestrian conditions. In addition, we introduce a scale-aware HRNet-based preprocessing module that enhances keypoint extraction for distant and partially occluded pedestrians. By dynamically scaling pedestrian regions based on bounding box dimensions and applying contrast enhancement and noise reduction, the proposed approach improves the quality of pose features under challenging visual conditions. To further address the lack of generalizability in existing datasets, we curate a custom Indian Road Dataset (IRD) that captures dense, occluded, and unstructured traffic scenarios. This enables evaluation in realistic environments where existing models often fail.Experiments on both JAAD_all and the custom IRD dataset show that our modelachieves moderate performance while maintaining low computational latency, demonstrating its suitability for real-time deployment in intelligent transportation systems. Related works Pedestrian crossing intention prediction has been extensively studied using a variety of methods, from classical machine learning to advanced deep learning architectures. Temporal deep learning models, particularly Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), dominate recent research due to their ability to capture temporal dependencies in pedestrian motion. For example, Ahmed et al.20 combined 2D pose estimation with LSTMs to predict intentions using spatio-temporal joint data, achieving 87% accuracy on JAAD. Lin et al.10 employed an Attention-LSTM model with panoptic segmentation to predict trajectories and intentions one second ahead, addressing dynamic interactions in traffic scenes. Sidharta et al.31 leveraged OpenPose-derived angular features with a Multi-Input-Single-Output (MISO) model to resolve crossing ambiguity in group pedestrian scenarios, attaining 93% accuracy. While effective in structured environments, these approaches often struggle with occlusions, distant pedestrians, and dense, unstructured traffic, which are common in Indian roads32. Graph-based methods (GCNs) have been widely adopted to model spatial relationships and pedestrian interactions. Ling et al.12 proposed PedAST-GCN, integrating pose, bounding boxes, and vehicle speed, improving prediction accuracy over benchmarks. Recent works have also explored modeling multi-pedestrian interactions to improve crossing intention prediction in crowded environments33. Zhang et al. developed a spatial-temporal GCN using skeleton data alone, while Chen et al.11 combined graph autoencoders with LSTMs to encode pedestrian-centric environmental graphs, emphasizing scene semantics. These GCN approaches improve spatial reasoning but often rely on local features and static adjacency matrices, limiting robustness under occlusion or in crowded environments. Pose-based methods focus on extracting biomechanical and skeletal features. Zhang et al.34 analyzed joint angles and facial cues with Support Vector Machines (SVMs) and Random Forests, achieving an AUC of 0.849 for red-light crossing prediction. Boda and Ramadevi35 integrated OpenPose with VGG-19 for real-time pose classification on PIE, attaining 92.5% accuracy. However, pose-based approaches are vulnerable when pedestrians are partially occluded, distant, or captured at low resolution. Multi-modal fusion methods combine visual, motion, and contextual cues to enhance prediction robustness. Yang et al.36 fused CNN, RNN, and attention modules to integrate bounding boxes, pose, and semantic segmentation, improving prediction horizons to 2 s. Chen et al.37 proposed PedCMT, a cross-modal transformer combining bounding boxes and ego-vehicle speed, while Zhao et al.38 fused facial expressions with 2D skeletons using GCNs. Despite improved robustness, static fusion strategies often incur high computational cost and have limited adaptability under varying conditions. Lightweight architectures such as SqueezeNet-GRU have also been explored for real-time pedestrian intention prediction in computationally constrained environments33. However, lightweight models may experience reduced robustness under complex traffic conditions involving occlusions, dense pedestrian interactions, and small-scale targets. Classical machine learning approaches remain relevant in low-resource and interpretable settings. Moreno et al.39 applied Random Forests to drone-captured trajectories by achieving high accuracy of 98%, while Xin et al.40 used Gradient Boosting Decision Trees to model pedestrian behavior, outperforming logistic regression. These approaches, however, lack fine-grained temporal and spatial modeling capabilities. Finally, datasets have been instrumental in standardizing evaluation. PIE8 integrates intention labels with ego-vehicle speed, while JAAD focuses on pedestrian-centric annotations. These benchmarks enable evaluation of models like PedAST-GCN12 and PIT14, but they mostly capture structured roads with moderate pedestrian density, leaving performance under unstructured traffic environments largely unexplored. Recent advances have explored the use of large vision-language models to improve pedestrian intention prediction through enhanced contextual reasoning. VLMPed-CoT41 introduces a vision-language framework with a chain-of-thought (CoT) mechanism that integrates multimodal inputs and leverages structured reasoning to improve prediction accuracy and interpretability across datasets such as JAAD and PIE. While such approaches demonstrate strong performance, they typically involve complex architectures and multi-stage training strategies, which may limit their applicability in real-time and resource-constrained settings. In addition, recent survey works Y. Ling et al.42 provide a comprehensive overview highlighting that multimodal fusion of pedestrian behavior, motion dynamics, and environmental context is critical for robust prediction. The survey also identifies key open challenges, including generalization across datasets, handling behavioral ambiguity, and modeling uncertainty in complex real-world scenarios. Existing multimodal fusion approaches in pedestrian intention prediction typically rely on static concatenation or globally learned weights to combine visual and contextual features, which has limited adaptability under varying visual conditions such as occlusion and scale variation43,44. Several recent works incorporate attention mechanisms or transformer-based architectures to improve feature interaction and adaptability12,14, including graph attention-based models such as Dual-STGAT45. While these approaches enhance representation learning, they often introduce increased computational complexity, making them less suitable for real-time and edge deployment scenarios. In contrast, the proposed approach focuses on sample-dependent, reliability-aware fusion, where the contribution of each modality is dynamically adjusted based on its quality for a given input. Unlike attention-based fusion, the proposed method adopts a lightweight gating mechanism, enabling adaptive modality selection while maintaining computational efficiency. This design is particularly suitable for pedestrian intention prediction in unstructured environments, where visual features are frequently degraded due to occlusion, small-scale pedestrians, and scene clutter. The major contributions of the paper are as follows: A hybrid CNN–GCN-based pedestrian intention prediction model framework incorporating a dynamic modality gating mechanism that performs sample-dependent, reliability-aware fusion of visual and non-visual features. The design enables adaptive prioritization of modalities under varying visual conditions, such as occlusion and scale variation, while maintaining computational efficiency.A scale-aware preprocessing strategy integrated with HRNet is designed to improve pose keypoint extraction particularly for distant and partially occluded pedestrians through adaptive input scaling and image enhancement to improve feature reliability under degraded visual conditions.Introducing a custom Indian Road dataset collected in Trivandrum city, India covering urban and suburban roads capturing unstructured traffic scenarios with dense pedestrian interactions, non-compliant behaviors, adverse weather, and frequent occlusions that are not present in the existing dataset benchmarks.Comprehensive experimental evaluations conducted on JAAD_all, IRD, and cross-dataset (PIE) settings, including ablation studies(fusion strategies, preprocessing, and attention mechanisms), analysis under occlusion and pedestrian scale variations, modality weight interpretation, edge-device deployment and failure case analysis, while maintaining a favorable trade-off between predictive performance and computational efficiency. A hybrid CNN–GCN-based pedestrian intention prediction model framework incorporating a dynamic modality gating mechanism that performs sample-dependent, reliability-aware fusion of visual and non-visual features. The design enables adaptive prioritization of modalities under varying visual conditions, such as occlusion and scale variation, while maintaining computational efficiency. A hybrid CNN–GCN-based pedestrian intention prediction model framework incorporating a dynamic modality gating mechanism that performs sample-dependent, reliability-aware fusion of visual and non-visual features. The design enables adaptive prioritization of modalities under varying visual conditions, such as occlusion and scale variation, while maintaining computational efficiency. A scale-aware preprocessing strategy integrated with HRNet is designed to improve pose keypoint extraction particularly for distant and partially occluded pedestrians through adaptive input scaling and image enhancement to improve feature reliability under degraded visual conditions. A scale-aware preprocessing strategy integrated with HRNet is designed to improve pose keypoint extraction particularly for distant and partially occluded pedestrians through adaptive input scaling and image enhancement to improve feature reliability under degraded visual conditions. Introducing a custom Indian Road dataset collected in Trivandrum city, India covering urban and suburban roads capturing unstructured traffic scenarios with dense pedestrian interactions, non-compliant behaviors, adverse weather, and frequent occlusions that are not present in the existing dataset benchmarks. Introducing a custom Indian Road dataset collected in Trivandrum city, India covering urban and suburban roads capturing unstructured traffic scenarios with dense pedestrian interactions, non-compliant behaviors, adverse weather, and frequent occlusions that are not present in the existing dataset benchmarks. Comprehensive experimental evaluations conducted on JAAD_all, IRD, and cross-dataset (PIE) settings, including ablation studies(fusion strategies, preprocessing, and attention mechanisms), analysis under occlusion and pedestrian scale variations, modality weight interpretation, edge-device deployment and failure case analysis, while maintaining a favorable trade-off between predictive performance and computational efficiency. Comprehensive experimental evaluations conducted on JAAD_all, IRD, and cross-dataset (PIE) settings, including ablation studies(fusion strategies, preprocessing, and attention mechanisms), analysis under occlusion and pedestrian scale variations, modality weight interpretation, edge-device deployment and failure case analysis, while maintaining a favorable trade-off between predictive performance and computational efficiency. Methodology The pedestrian crossing intention prediction problem is a binary classification task in which a sequence of video frames (F) collected from the front view of the ego-vehicle denoted \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:F=\left\{{F}_{1},{F}_{2},\dots\:,{F}_{t}\right\}$$\end{document}, is given along with its motion related-data (S) denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:S=\left\{{S}_{1},{S}_{2},\dots\:,{S}_{t}\right\}$$\end{document}, to estimate the crossing intention of the pedestrian. A single frame input within a sequence of “t” frames can be represented by: The Bounding box around the pedestrian frame “i”, represented by the top left \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{itl},{Y}_{itl}\right)$$\end{document} and bottom right corner \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{ibr},{Y}_{ibr}\right)$$\end{document} coordinates. 1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{B}_{i}\:=\:\left\{{X}_{ibr},\:{X}_{itl},{Y}_{ibr},{Y}_{itl}\right\}$$\end{document} The Bounding box around the pedestrian frame “i”, represented by the top left \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{itl},{Y}_{itl}\right)$$\end{document} and bottom right corner \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{ibr},{Y}_{ibr}\right)$$\end{document} coordinates. The Bounding box around the pedestrian frame “i”, represented by the top left \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{itl},{Y}_{itl}\right)$$\end{document} and bottom right corner \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left({X}_{ibr},{Y}_{ibr}\right)$$\end{document} coordinates. Pose key points of 36-dimensional vector that represents 2D joint coordinates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left(x,y\right)$$\end{document} of 18 pose key points in frame “i”. 2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{i}=\left\{\left({x}_{i}1,{y}_{i}1\right),\left({x}_{i}2,{y}_{i}2\right),\dots\:,\left({x}_{i}18,{y}_{i}18\right)\right\}$$\end{document} Pose key points of 36-dimensional vector that represents 2D joint coordinates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left(x,y\right)$$\end{document} of 18 pose key points in frame “i”. Pose key points of 36-dimensional vector that represents 2D joint coordinates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left(x,y\right)$$\end{document} of 18 pose key points in frame “i”. Local context (lc) is taken around the pedestrian bounding box in RGB format with a fixed size of (512,512,3). 3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:L=lc\:\mathrm{S}\mathrm{u}\mathrm{c}\mathrm{h}\:\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}\:lc\:\in\:\:{\mathbb{R}}^{\mathrm{512,512,3}}$$\end{document} Local context (lc) is taken around the pedestrian bounding box in RGB format with a fixed size of (512,512,3). Local context (lc) is taken around the pedestrian bounding box in RGB format with a fixed size of (512,512,3). Global context (gc) is an RGB image of a raw frame masking the pedestrian with the bounding box coordinates. 4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:G\:=\:gc\:\:\mathrm{S}\mathrm{u}\mathrm{c}\mathrm{h}\:\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}\:gc\:\in\:\:{\mathbb{R}}^{\mathrm{512,512,3}}$$\end{document} Global context (gc) is an RGB image of a raw frame masking the pedestrian with the bounding box coordinates. Global context (gc) is an RGB image of a raw frame masking the pedestrian with the bounding box coordinates. Figure 1 represents the implemented architecture. The four modalities (Pose, Bounding box, Global context and Local context) are extracted and fed as input to the network. A scale aware HRNet has been introduced as a preprocessing method to extract the key points of distant pedestrians. These four modalities are further segregated as Visual and Non-visual modalities. To effectively extract the features from the visual modalities, a combination of EfficientNet with (Efficient Channel Attention + Convolutional Attention Module) ECA_CAM block is used which is explained in the visual features extraction section. STA-GCN network is used for extracting features from non-visual modalities. The extracted features are then passed through a temporal pooling layer for temporal aggregation, followed by a concatenation layer. The concatenated features are then processed by a dynamic modality gating module, which adaptively prioritizes visual and non-visual cues based on the scene context. This is followed by a weighted fusion of the extracted modalities. The fused features are passed through dense layers and a classification head to predict whether the pedestrian is crossing or not crossing. Fig. 1Overview of the proposed pedestrian crossing intention prediction model. Visual features are extracted through combination of EfficientNet and attention modules and non-visual features are extracted with STA-GCN module. Further, the extracted features are passed to the temporal pooling layer followed by a concatenation layer. The concatenated features are then passed to the dynamic modality module followed by a weighted fusion module with final classification head to decide whether crossing or non-crossing. Overview of the proposed pedestrian crossing intention prediction model. Visual features are extracted through combination of EfficientNet and attention modules and non-visual features are extracted with STA-GCN module. Further, the extracted features are passed to the temporal pooling layer followed by a concatenation layer. The concatenated features are then passed to the dynamic modality module followed by a weighted fusion module with final classification head to decide whether crossing or non-crossing. A.IRD Dataset Data Collection and Annotation. A.IRD Dataset Data Collection and Annotation. IRD Dataset Data Collection and Annotation. The Indian Road Dataset (IRD) was curated to capture the unique challenges of pedestrian crossing intention prediction in unstructured Indian traffic. Data was collected in Trivandrum, Kerala, across a variety of road types within both the city and its outskirts, covering diverse local areas, major city zones, and suburban regions. A high-resolution 50MP OIS camera was mounted inside the ego-vehicle just below the rear-view mirror. Videos were recorded at 30 fps, with durations ranging from 15 s to 3 min 2 s, yielding approximately 68,429 frames in total. IRD encompasses a wide range of pedestrian activities such as crossing, waiting, walking along the curb, and sudden road entry. The dataset intentionally covers adverse conditions such as fog, rain, clear and evening weather as shown in Fig. 2 aimed at capturing the variability of real-world Indian driving environments rather than explicitly evaluating weather-specific model performance. Annotation Protocol: All pedestrian instances were annotated using CVAT (Computer Vision Annotation Tool), with 324 unique pedestrians and over 56,234 bounding boxes labeled. Each pedestrian was assigned a unique identifier videoID_frameID_pedestrianID for consistent pedestrian tracking across frames. To capture occlusion challenges, a binary occlusion label is defined, where a value of 1 corresponds to severe occlusion (< 25% visibility), and 0 indicates partial or no occlusion (> 25% visibility). This definition is adopted to explicitly distinguish scenarios where visual cues are significantly degraded, enabling targeted analysis of model performance under limited visibility conditions. Bounding box annotations were aligned with HRNet-based key point extraction, providing a foundation for pose estimation, tracking, and multi-agent interaction studies, although these tasks were not explicitly benchmarked in this work. The annotation process follows a consistent labeling protocol to ensure uniformity across all samples. Each bounding box is manually inspected to maintain annotation quality, and temporal consistency is preserved through unique pedestrian identifiers across frames. All annotations are manually verified to ensure consistency and accuracy. While formal inter-annotator agreement metrics are not reported, the annotation process follows a consistent labeling protocol with careful quality control through visual inspection. Fig. 2Sample frames from the custom dataset showing different weather conditions. Sample frames from the custom dataset showing different weather conditions. Dataset Partitioning and Usage: The IRD dataset is partitioned at the video level into 23 clips for training, 4 clips for validation, and 15 clips for testing to ensure that pedestrian instances from the same video do not appear across different splits. This prevents temporal data leakage and enables fair evaluation of model generalization. Although the dataset size is relatively modest compared to large-scale benchmarks, it is specifically designed to capture complex real-world scenarios such as dense crowds, occlusion-heavy environments, and unpredictable pedestrian behavior.The limited scale of the dataset is acknowledged as a constraint, and future work will focus on expanding its size. Dataset Availability and Usability : The dataset used in this study is publicly available for academic research purposes at Kaggle repository[https://www.kaggle.com/datasets/ammanaveen/ird-dataset-pedestrian-intention-prediction/data]. The dataset is organized in a structured format with annotated video sequences, bounding boxes, and associated metadata, to facilitate reproducibility and ease of use for tasks such as pedestrian intention prediction, tracking, and behavior analysis. The IRD dataset captures a diverse distribution of pedestrian behaviors and scene characteristics relevant to unstructured traffic environments. The dataset consists of 42 video clips and 68,429 frames, containing 324 unique pedestrians and 56,234 annotated bounding boxes. On average, this corresponds to approximately 1,630 frames per video and 173 bounding box annotations per pedestrian, indicating dense temporal annotation. The large number of bounding box annotations relative to the number of frames reflects frequent pedestrian presence and dense scene interactions. The inclusion of occlusion labels indicates the presence of severe occlusions, while the distribution of pedestrians across diverse scenes suggests variability in pedestrian scale, including small-scale and distant instances. These factors introduce significant variation in visual appearance and motion patterns, making the dataset suitable for evaluating model performance under real-world conditions. FeatureJAAD DatasetPIE DatasetIRD (Ours) Region Toronto, CanadaToronto, CanadaTrivandrum, India # Videos 346 clips (5–10 s each)~ 6 h continuous video42 (each clip duration of 15s–3m2s) # Frames ~ 82,032~ 900,000 total frames68,429 # Pedestrians ~ 2,786~ 1,842324 Bounding Boxes ~ 378,643 pedestrian boxes~ 739,000 pedestrian boxes56,234 boxes Occlusion Labels Yes (partial/full occlusion flags)Not explicitly provided for all bounding boxesYes (binary occlusion labels) Weather/Lighting Daytime, various conditions (mostly clear)Mostly daytime traffic scenes, some variation (sunny, cloudy, light rain)Fog, rain, evening, low-light Road Type Urban crosswalks (signalized & unsignalized)Urban intersections, crosswalks with traffic lightsUnstructured urban, mixed-use roads Key Challenge Captured Pedestrian behavior analysis and crossing intention predictionPedestrian intention estimation with contextual interactionHigh-density crowds, non-compliant pedestrian behaviour, frequent occlusions FeatureJAAD DatasetPIE DatasetIRD (Ours) Region Toronto, CanadaToronto, CanadaTrivandrum, India # Videos 346 clips (5–10 s each)~ 6 h continuous video42 (each clip duration of 15s–3m2s) # Frames ~ 82,032~ 900,000 total frames68,429 # Pedestrians ~ 2,786~ 1,842324 Bounding Boxes ~ 378,643 pedestrian boxes~ 739,000 pedestrian boxes56,234 boxes Occlusion Labels Yes (partial/full occlusion flags)Not explicitly provided for all bounding boxesYes (binary occlusion labels) Weather/Lighting Daytime, various conditions (mostly clear)Mostly daytime traffic scenes, some variation (sunny, cloudy, light rain)Fog, rain, evening, low-light Road Type Urban crosswalks (signalized & unsignalized)Urban intersections, crosswalks with traffic lightsUnstructured urban, mixed-use roads Key Challenge Captured Pedestrian behavior analysis and crossing intention predictionPedestrian intention estimation with contextual interactionHigh-density crowds, non-compliant pedestrian behaviour, frequent occlusions B.Scale- Aware HRNet-Based Preprocessing Module for Pose Keypoint Extraction. B.Scale- Aware HRNet-Based Preprocessing Module for Pose Keypoint Extraction. Scale- Aware HRNet-Based Preprocessing Module for Pose Keypoint Extraction. HRNet is preferred for keypoint extraction due to its ability to maintain high-resolution representations across the entire network, in contrast to conventional architectures that lose spatial detail through progressive downsampling. Given that these keypoints serve as critical input features for pedestrian intention prediction, any localization error will propagate through the entire pipeline. To extract pedestrian pose keypoints, HRNet46 requires a fixed input resolution of 512 × 512 pixels and performs optimally when the pedestrian occupies a sufficiently large region within the image (100 × 70 pixels). However, in real-world scenarios, particularly in dense Indian traffic, pedestrians often appear small or partially occluded, making accurate keypoint detection challenging for HRNet. To address this limitation, a scale-aware preprocessing module is introduced, as illustrated in Fig. 3. The preprocessing pipeline operates as follows: First, the bounding box dimensions of each pedestrian are computed. If the pedestrian size meets the minimum threshold(100 × 70 pixels), the pedestrian region is directly cropped and passed to the HRNet model. For smaller pedestrians, a dynamic scaling strategy is applied. The pedestrian is cropped based on bounding box coordinates, and the cropped region is dynamically enlarged using a scaling factor formula given in Eq. (5). The scaling factor formula is derived from the bounding box dimensions which ensure that the resized image preserves contextual details while meeting the input size requirement for effective key point detection, as shown in Fig. 3.5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\:s=\:clip\:\left(\frac{{s}_{max}.\:{A}_{ref}}{A\:+\mathcal{E}},\:{S}_{min},\:{S}_{max}\right)$$\end{document} Where, w = width of the bounding box.H = height of the bounding box.A = w.H = area of the bounding box.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{ref}$$\end{document}=100⋅40 = 4000 reference area (approx. small-person size).ε=1×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{10}^{-5}$$\end{document}= small constant to avoid division by zero.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{min}$$\end{document}= minimum allowed scale (e.g., 2.0).\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{max}$$\end{document}= maximum allowed scale (e.g., 5.0). w = width of the bounding box. w = width of the bounding box. H = height of the bounding box. H = height of the bounding box. A = w.H = area of the bounding box. A = w.H = area of the bounding box. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{ref}$$\end{document}=100⋅40 = 4000 reference area (approx. small-person size). \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{ref}$$\end{document}=100⋅40 = 4000 reference area (approx. small-person size). ε=1×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{10}^{-5}$$\end{document}= small constant to avoid division by zero. ε=1×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{10}^{-5}$$\end{document}= small constant to avoid division by zero. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{min}$$\end{document}= minimum allowed scale (e.g., 2.0). \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{min}$$\end{document}= minimum allowed scale (e.g., 2.0). \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{max}$$\end{document}= maximum allowed scale (e.g., 5.0). \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{max}$$\end{document}= maximum allowed scale (e.g., 5.0). Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:clip\left(x,a,b\right)=\mathrm{min}\left(\mathrm{max}\left(x,a\right),b\right)$$\end{document} ensures the result stays within the minimum bound (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:a\:$$\end{document}) and maximum bound (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:b\:$$\end{document}). Fig. 3Overview of the proposed scale-aware preprocessing framework integrated with HRNet-based keypoint estimation. Overview of the proposed scale-aware preprocessing framework integrated with HRNet-based keypoint estimation. After calculating the crop resize ratio using the bounding box dimensions, the cropped pedestrian image is resized accordingly and subjected to a series of image processing techniques to enhance key point detection for distant and occluded pedestrians. First, the image is converted from BGR (Blue-Green-Red) to LAB (L represents Lightness, A represents the green–red color component and B represent the blue–yellow color component)) color space to enable more effective contrast manipulation. Contrast Limited Adaptive Histogram Equalization (CLAHE) is then applied on the L channel to improve visibility in poorly lit conditions while avoiding over-amplification of noise. The image is subsequently converted back from LAB to BGR format to maintain compatibility with standard processing pipelines. To further improve image quality, Gaussian blur is applied to reduce noise and smooth small artifacts. Finally, the image is resized while preserving the original aspect ratio, and padding is applied as necessary to match the input size expected by the scale-aware HRNet model (typically 512 × 512 pixels), ensuring that the human figure remains undistorted. The resulting image whether it comes from this enhanced preprocessing method or from a direct masked background crop of sufficiently large pedestrians is then passed to the HRNet module. This ensures accurate key point extraction, even in challenging scenarios involving distant and partially occluded individuals (Fig. 4). Fig. 4(a) Output of the HRNet without any preprocessing where a distant pedestrian appearing with a very small spatial footprint in the frame remains undetected (b) Output of the proposed scale-aware HRNet in which the same distant pedestrian is successfully detected with the green bounding box and associated key points. (a) Output of the HRNet without any preprocessing where a distant pedestrian appearing with a very small spatial footprint in the frame remains undetected (b) Output of the proposed scale-aware HRNet in which the same distant pedestrian is successfully detected with the green bounding box and associated key points. C.Non-Visual Feature Extraction via STA-GCN. C.Non-Visual Feature Extraction via STA-GCN. Non-Visual Feature Extraction via STA-GCN. The non-visual inputs consist of pose keypoints and bounding box trajectories, which are modeled as graph-structured data. The non-visual inputs of the structure of STA-GCN layer are presented in Fig. 5, which includes GCN layer, an attention layer, and a Temporal Channel attention Network (TCN) layer. The GCN layer aggregates and updates the spatial graph information through the joint features. Fig. 5The implemented STA-GCN architecture to extract the non-visual modality features. 6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{out}\:=\:{W}_{e}\:\odot\:\:A\:{X}_{in}{W}_{g}$$\end{document} The implemented STA-GCN architecture to extract the non-visual modality features. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{out}\:\to\:$$\end{document} Updated vertexes of hidden states matrix. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{in}\:\to\:$$\end{document} Initial vertexes of hidden states matrix. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{W}_{e}\:\to\:$$\end{document} learnable edge weights. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{W}_{g}\:\to\:$$\end{document} learnable weight matrix. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:A\:\to\:\:$$\end{document}Adjacency matrix. The Attention layer (GAT (Graph Attention Network) and Temporal attention) helps in learning spatial-temporal information tasks. The attention model for posing key points contains temporal and channel wise attention modules, while the attention model for the bounding box contains only the temporal attention module. The GAT module helps in learning the relationship based on the attention weights. This temporal attention module assigns different attention weights based on the frames to capture the long-term dependencies. This model fuses the features in a weighted manner by providing varying levels of attention to each frame. GAT adaptively computes the attention scores of the nodes to enhance the spatial feature aggregation. The core attention scores are computed using7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{t}\:=\sigma\:\left({F}_{t}\left(AvgPool\left({X}^{gcn}\right)\right)\right)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}^{t}\:\to\:\:$$\end{document}Attention Score. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{F}_{t}\:\to\:\:\:$$\end{document}1-D convolutional operation. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}^{gcn}\:\to\:\:\:$$\end{document}Output vertex hidden states matrix of GCN layer. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\sigma\:\:\to\:\:\:$$\end{document}Sigmoid activation function. The TCN layer helps in extracting and compressing the temporal features between the frame sequences. This layer contains a sequence of BatchNorm2d, Conv2d, ReLu activation functions. The output vertex hidden states matrix for TCN layer is:8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}^{tcn}\:=\:BN\left({W}_{t}\left(\delta\:\left(BN\left({X}^{a}\right)\right)\right)\right)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}^{tcn}\:\to\:\:$$\end{document}Output vertex hidden states matrix of TCN layer. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:BN\:\to\:\:$$\end{document}BatchNormalization. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{W}_{t}\:\to\:\:$$\end{document}Weight Vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:\:\to\:\:\:$$\end{document}ReLu activation function. D.Visual Feature Extraction Using EfficientNetV2 with ECA-CAM Attention Module. D.Visual Feature Extraction Using EfficientNetV2 with ECA-CAM Attention Module. Visual Feature Extraction Using EfficientNetV2 with ECA-CAM Attention Module. The sequence of RGB images of a traffic scene captures the variations of pedestrians’ appearance temporally5,21. The visual appearance features \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{j}=\left\{{a}_{1j},{a}_{2j},{a}_{3j},\dots\:,{a}_{qj}\right\}$$\end{document} of the pedestrian “\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:j\:$$\end{document}” ∈ (1, m) for past observed “q” time steps are captured using image portions of the size of the bounding box consisting of the pedestrian in a scene. The local contextual information of the target pedestrian depicts its relationship with the dynamic traffic scene elements in its surroundings3,23. The surrounding local environment features \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{C}_{j}=\left\{{c}_{1j},{c}_{2j},{c}_{3j},\dots\:,{c}_{qj}\right\}$$\end{document} are extracted using a larger image portion size to include immediate contextual details. This is achieved by extending the dimension by at least twice the size of the pedestrian “\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:j\:$$\end{document}” bounding and masking it to include only the surrounding details. EfficientNetV2 is utilized for visual features extraction to balance the trade-off between the model’s performance and complexity in terms of size and number of parameters. This model achieves 83.9% accuracy on ImageNet while being approximately 6.8× faster in training and having fewer parameters (21 million) compared to many state-of-the-art convolutional networks such as NFNets. EfficientNetV2-S also reduces training time by combining MBConv and Fused-MBConv blocks for improved efficiency. Both the appearance and the local context features are processed in parallel through EfficientNetV2-S pretrained on ImageNet. This is followed by an ECA_CAM to emphasize the relevant features across both channel and spatial dimensions. Figure 6 explains ECA_CAM block that performs two sequential operations. The input from the EfficientNet is passed to the ECA module. This module compresses the spatial dimensions with global average pooling, followed by a lightweight 1D convolution of dynamic kernel size k as defined in Eq. (9). Here,9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:k\:=\:nextodd\left(\frac{\left|{\mathrm{log}}_{2}\left(C\right)+b\right|}{\gamma\:}\right)$$\end{document} With hyper-parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\gamma\:\:$$\end{document},b and channel (C). This 1D conv captures local cross‑channel interactions in an adaptive, non‑parametric way followed by a sigmoid output that produces a per‑channel weight vector of length C. Finally, these weights are broadcast back to its original grid size and multiplied elementwise, re‑calibrating each channel according to its global importance. The output of the ECA is further refined through a combination of channel and spatial attention, where channel attention determines the most informative feature channels, while spatial attention highlights the regions within the feature map that the network should prioritize. In channel-wise attention, to obtain a second channel mask, the average-pooled and max-pooled channel descriptors are summed, followed by a sigmoid activation. Multiplying this mask with the feature map emphasizes the most relevant feature channels. On the refined-channel tensor, the channel axis collapses using both max and average pooling to produce two (H, W, 1) maps, where H and W represent the height and width, respectively. These maps are then concatenated and passed through a 7 × 7 depth-wise separable convolution followed by a sigmoid activation. This generates a spatial attention map that indicates the regions of the feature map to which the network should concentrate (e.g., pedestrian areas versus background). The final attended feature is obtained by multiplying this attention map with the input feature map. This sequential use of the ECA_CAM block adaptively re-weights both the most informative channels and the most critical spatial locations. This double attention mechanism preserves richer mid‑level structures (e.g., pose landmarks and small context cues) and suppresses background noise before the collapsing to a 1D feature vector. Fig. 6Flow diagram of ECA_CAM block that adaptively re-weights the most informative feature channels and critical spatial regions. Flow diagram of ECA_CAM block that adaptively re-weights the most informative feature channels and critical spatial regions. Temporal pooling and Concatenation module: The extracted features from both Visual and non-visual modalities undergo Temporal pooling layer to aggregate the features across the time dimension to have it as a feature sequence. Then the features are concatenated across all the modalities. E. Dynamic Modality Gating Module: E. Dynamic Modality Gating Module: Dynamic Modality Gating Module: The proposed dynamic modality gating module performs sample-dependent, reliability-aware fusion of visual and non-visual modalities. Unlike conventional fusion strategies that rely on static concatenation or globally learned weights, the proposed approach dynamically estimates the relative importance of each modality for every input sample. In pedestrian intention prediction, particularly in unstructured traffic environments, the reliability of visual cues (e.g., pose and appearance) vary significantly due to occlusion, small-scale pedestrians, and cluttered backgrounds. In contrast, motion-based cues may remain informative in such scenarios. Therefore, a static fusion strategy leads to suboptimal performance.To address this, the proposed gating mechanism learns a context-aware weighting function that adaptively prioritizes modalities based on their reliability. This enables the model to emphasize motion features when visual information is degraded, and prioritize visual cues when reliable pose information is available. Let’s say the. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{L}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled local context feature vector.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{G}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled global context feature vector.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{P}\:\in\:\:{\mathbb{R}}^{d}$$\end{document}: : pooled pose key points feature vector.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{B}\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled bounding box feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{L}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled local context feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{L}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled local context feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{G}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled global context feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{G}\:\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled global context feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{P}\:\in\:\:{\mathbb{R}}^{d}$$\end{document}: : pooled pose key points feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{P}\:\in\:\:{\mathbb{R}}^{d}$$\end{document}: : pooled pose key points feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{B}\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled bounding box feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overline{B}\in\:\:{\mathbb{R}}^{d}$$\end{document} : pooled bounding box feature vector. Concatenate these:10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:u\:=\:\left[\overline{L}\parallel\:\overline{G}\parallel\:\overline{P}\parallel\:\overline{B}\right]\in\:\:{\mathbb{R}}^{4d}$$\end{document} The concatenated vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:u\:$$\end{document} is fed into a lightweight MLP comprising two linear layers with non-linearity, i.e., ReLU and optional dropout, producing four scalar logits:11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:z\:=\:MLP\left(u\right)\:\in\:\:{\mathbb{R}}^{4}$$\end{document} To obtain a valid probability distribution over the four branches, the SoftMax function is applied:12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{w}_{i}\:=\:\frac{\mathrm{exp}\left({z}_{i}\right)}{{\sum\:}_{j\:=\:1}^{4}\mathrm{exp}\left({z}_{j}\right)},\:w\:=\:\left[{w}_{L},{w}_{G},{w}_{P},{w}_{B}\right]$$\end{document} Let the final fused representation be computed as:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:f={w}_{L}\stackrel{\prime }{L}+{w}_{G}\stackrel{\prime }{G}+{w}_{P}\stackrel{\prime }{P}+{w}_{B}\stackrel{\prime }{B}$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{w}_{L}+{w}_{G}+{w}_{P}+{w}_{B}=1$$\end{document}. This formulation enables sample-dependent, reliability-aware fusion, where the contribution of each modality is adaptively determined based on its relevance for a given input. Compared to attention-based or transformer-based fusion strategies, the proposed gating module introduces minimal computational overhead while maintaining adaptability, making it suitable for real-time deployment. End-to‐End Training:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathcal{L}\mathcal{\:}=\mathcal{\:}{\sum\:}_{{c}_{c}^{y}}\mathrm{log}\left({y}^{{\prime\:}}\right)\:$$\end{document} The final fused representation f is passed to the classification head (i.e., two fully connected layers followed by SoftMax) to predict the final label \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{y}^{{\prime\:}}$$\end{document}..All parameters including those of the four backbone streams, the gating MLP, and the classifier are learned jointly by minimizing the cross-entropy loss as in Eq. (14). Since the prediction y’ is a function of the fused representation f, the loss is implicitly dependent on f. Consequently, gradients are propagated through the classifier to the gating module and backbone feature extractors, enabling end-to-end optimization of all components. The overall architecture is designed to address complementary challenges, where scale-aware preprocessing improves feature quality, and dynamic modality gating ensures adaptive fusion, resulting in a cohesive framework rather than a simple aggregation of independent modules. Experiments and results Implementation and training details The proposed framework is evaluated with custom made datasets and with the publicly available benchmark dataset JAAD. JAAD includes 346 video clips totaling 240 h of footage captured at 30 frames per second. These clips, spanning 5 to 15 s each, feature resolutions of 1920 × 1080 and 1280 × 720 pixels. The dataset is divided into two subsets. JAAD_beh documents 495 instances of pedestrians actively crossing roads and 191 cases of individuals preparing to cross. JAAD_all expands the scope by incorporating 2,100 samples of pedestrians engaged in non-crossing activities. JAAD_beh is a subset of JAAD_all containing only crossing and preparatory actions, whereas JAAD_all additionally includes non-crossing instances. Although both JAAD_beh and JAAD_all subsets are mentioned for completeness, all experiments in this work are conducted on the JAAD_all subset to ensure consistency with prior studies and comprehensive evaluation across both crossing and non-crossing scenarios. The dataset is partitioned following the standard benchmark protocol proposed by Kotseruba et al.44, which is widely adopted in pedestrian intention prediction studies. Specifically, the dataset is split at the video level into 177 training, 29 validation, and 117 testing clips (total 323 clips), ensuring that pedestrian instances from the same video do not appear across different splits, thereby preventing data leakage. As defined in44, a subset of 23 videos is excluded due to lower resolution and adverse conditions such as poor illumination. As 23 clips are excluded out of total 346 clips of JAAD_all, 323 clips are used for experimental evaluation ensuring consistency with the standard benchmark protocol.Input Configuration and Temporal Modeling: The model operates on fixed-length temporal sequences extracted from video frames. Each input sample consists of an observation sequence of 15 consecutive frames, sampled at the original frame rate (30 fps) without temporal subsampling, thereby preserving temporal continuity.The prediction task is formulated using a Time-To-Event (TTE) framework, with a prediction horizon of 60 frames (~ 2 s). This configuration is selected based on empirical analysis (Sect. 4.3.3), balancing sufficient temporal context with prediction reliability. A sample is labeled as positive if the pedestrian initiates crossing behavior within the prediction horizon, and negative otherwise. This formulation enables early anticipation of pedestrian intention based on observed temporal cues. The present study was carried out on an experimental platform with Nvidia RTX 4060 graphics card and Intel I9 CPU. The proposed model was trained for 100 epochs with a batch size of 8, optimized using the AdamW optimizer and binary cross-entropy loss function, with a maximum learning rate of 0.00001. The evaluation metrics used are Precision, Recall, Accuracy, F1-score, and AUC. Latency Evaluation: Inference latency is measured on Intel i9 CPU equipped with an NVIDIA RTX 4060 GPU using a batch size of 1. The reported latency corresponds to the inference time of the proposed fusion network only and does not include feature extraction (HRNet-based keypoint estimation). All measurements are conducted under identical hardware and software conditions to ensure fair comparison across methods. Results on JAAD_all & custom IRD dataset The performance of the proposed CNN + GCN model was evaluated on both the custom Indian Road dataset and the JAAD_All dataset, with results summarized in Tables 1 and 2. Table 1 shows that the proposed model demonstrates moderate performancein Indian road scenarios, achieving an accuracy of 82% with AUC and recall of 72% and 50%, respectively. Table 2 presents the performance of the proposed model when evaluated on the publicly available JAAD_all datasets. Table 1Performance metrics of the proposed model on the custom Indian Road dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallLatency(ms) Ours Hybrid CNN + GCN0.820.720. 560.390.501.5 Table 1Performance metrics of the proposed model on the custom Indian Road dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallLatency(ms) Ours Hybrid CNN + GCN0.820.720. 560.390.501.5 Performance metrics of the proposed model on the custom Indian Road dataset. Table 2Performance metrics of the proposed model on the JAAD_All Dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallLatency(ms) Ours Hybrid CNN + GCN0.880.820.570.440.814.6 Table 2Performance metrics of the proposed model on the JAAD_All Dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallLatency(ms) Ours Hybrid CNN + GCN0.880.820.570.440.814.6 Performance metrics of the proposed model on the JAAD_All Dataset. On the custom IRD dataset, the proposed model achieves an accuracy of 0.82 and an AUC of 0.72 as shown in Table 1. However, a detailed analysis of evaluation metrics reveals important trade-offs. While the accuracy is relatively high, the precision (0.39) and F1-score (0.56) indicate the presence of false positives, particularly in complex traffic scenarios where pedestrian behavior is ambiguous. This behavior can be attributed to the inherent difficulty of distinguishing between crossing intent and non-crossing preparatory actions in unstructured environments. Pedestrians often exhibit transitional behaviors such as standing near the curb, slight body orientation changes, or intermittent motion, which are difficult to categorize clearly. Additionally, frequent occlusions, small-scale pedestrian appearance, and interactions with surrounding vehicles further degrade the reliability of visual cues, leading to both false positives and false negatives. The model tends to maintain moderate recall (0.50), capturing a portion of true crossing events, while still missing some due to limited discriminative cues in challenging scenarios. From a safety perspective, this trade-off is important, as missing a true crossing event (false negative) is more critical than generating a false alert. However, improving precision and recall remains an important direction for future work. The dataset also exhibits class imbalance between crossing and non-crossing samples, which can bias accuracy toward the dominant class. In such scenarios, high accuracy may be achieved by favoring non-crossing predictions while failing to correctly detect crossing events. Therefore, greater emphasis is placed on precision, recall, and F1-score to provide a more comprehensive and reliable evaluation particularly in safety-critical pedestrian intention prediction task. On the JAAD_all dataset as shown in Table 2, the model achieves an accuracy of 0.88 and an AUC of 0.82. Compared to the IRD dataset, the improved performance can be attributed to more structured environments and reduced occlusion complexity. However, the model does not achieve the highest performance among all compared methods, indicating that further improvements in feature representation and temporal modeling may enhance prediction accuracy. Table 3Benchmark results of the proposed model on the JAAD_All dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallFUSSI-Net43DenseNet0.600.720.400.270.73PCPA443Dconv0.700.850.510.360.87OSU47VGG + GRU0.830.820.630.510.81Ped Graph48GCN0.800.840.550.460.68Ped Graph plus49Conv + GCN0.86 0.88 0.650.580.75PIT14Transformer0.870.870.660.54 0.85 PedAST-GCN12STA + GCN 0.89 0.83 0.68 0.67 0.69 Ours Hybrid CNN + GCN0.880.820.570.440.81 Table 3Benchmark results of the proposed model on the JAAD_All dataset.ModelModel VariantAccuracyAUCF1 ScorePrecisionRecallFUSSI-Net43DenseNet0.600.720.400.270.73PCPA443Dconv0.700.850.510.360.87OSU47VGG + GRU0.830.820.630.510.81Ped Graph48GCN0.800.840.550.460.68Ped Graph plus49Conv + GCN0.86 0.88 0.650.580.75PIT14Transformer0.870.870.660.54 0.85 PedAST-GCN12STA + GCN 0.89 0.83 0.68 0.67 0.69 Ours Hybrid CNN + GCN0.880.820.570.440.81 Benchmark results of the proposed model on the JAAD_All dataset. Table 4Inference latency comparison on the JAAD_All dataset.ModelLatency(ms)FUSSI-Net4334.92PCPA4438.6OSU4770.83Ped Graph485.47PedAST-GCN1210 Ours 4.6 Table 4Inference latency comparison on the JAAD_All dataset.ModelLatency(ms)FUSSI-Net4334.92PCPA4438.6OSU4770.83Ped Graph485.47PedAST-GCN1210 Ours 4.6 Inference latency comparison on the JAAD_All dataset. When benchmarked against state-of-the-art methods on JAAD_All (Table 3), the proposed model attains comparable performance, with an accuracy slightly below the top-performing models such as PedAST-GCN (0.89), PIT (0.87), and PedGraphPlus (0.86). Despite this marginal difference in accuracy, the model demonstrates relatively high recall (0.81), which is critical for safety-critical pedestrian intention prediction where missing a crossing event is undesirable. The models achieving a higher accuracy typically rely on heavier temporal modeling and transformer-based architecture, which improves the classification performance at the cost of increased inference latency. Table 4 shows the Inference latency comparison on the JAAD_All dataset. The proposed model demonstrates lower inference latency(1.5 ms on the custom dataset and 4.6 ms on JAAD_All dataset) at the fusion stage compared to several existing methods. This efficiency can be attributed to the use of a lightweight architecture and streamlined feature fusion strategy. While the accuracy of proposed hybrid CNN + GCN model does not achieve the highest accuracy compared to the best-performing state-of-the-art PedAST-GCN method, it demonstrates a decent balance between predictive performance and operational efficiency through stable recall, minimal latency, and demonstrating improved performance under occlusion and dense traffic scenarios as shown in Fig. 7. This balance between predictive performance and computational efficiency indicates that the proposed model is suitable for real-time inference on edge device deployment. Accuracy alone may not fully reflect model performance in imbalanced datasets. Therefore, additional metrics such as precision, recall, F1-score, and AUC are considered to provide a more comprehensive evaluation.The observed differences between these metrics reflect the inherent difficulty of pedestrian intention prediction in complex and unstructured traffic scenarios. These observations indicate that the reported accuracy should be interpreted in conjunction with F1-score and recall, and does not overestimate model performance under imbalanced conditions. Fig. 7illustrates the accuracy–latency trade-off on the JAAD_all dataset comparing the highest accuracy PedAST-GCN with higher inference latency(10ms), whereas the proposed hybrid CNN–GCN model with competitive accuracy (0.88) with the lower latency (4.6 ms). illustrates the accuracy–latency trade-off on the JAAD_all dataset comparing the highest accuracy PedAST-GCN with higher inference latency(10ms), whereas the proposed hybrid CNN–GCN model with competitive accuracy (0.88) with the lower latency (4.6 ms). Fig. 8Keypoint extraction of a distant pedestrian from the ego-vehicle view demonstrating the ability of the proposed scale-aware HRNet to capture fine-grained pose information even when the pedestrian occupies only a small region in the frame. Keypoint extraction of a distant pedestrian from the ego-vehicle view demonstrating the ability of the proposed scale-aware HRNet to capture fine-grained pose information even when the pedestrian occupies only a small region in the frame. Fig. 9Role of scene context in intention prediction under clear visibility where integrating both local and global context enables accurate inference of pedestrian intent. Role of scene context in intention prediction under clear visibility where integrating both local and global context enables accurate inference of pedestrian intent. Fig. 10Intention prediction under occlusion where non-visual modalities specifically bounding boxes and pose keypoints enable the model to accurately infer the crossing intent of overlapping pedestrians despite limited visual visibility. Intention prediction under occlusion where non-visual modalities specifically bounding boxes and pose keypoints enable the model to accurately infer the crossing intent of overlapping pedestrians despite limited visual visibility. Fig. 11Robust intention prediction under degraded visual conditions caused by blur and background noise, achieved by relying on non-visual modalities such as bounding boxes and pose keypoints for accurate inference of pedestrian crossing intent. Robust intention prediction under degraded visual conditions caused by blur and background noise, achieved by relying on non-visual modalities such as bounding boxes and pose keypoints for accurate inference of pedestrian crossing intent. Figure 8 illustrates the effectiveness of the proposed preprocessing module in extracting key points of distant pedestrians from the ego-vehicle perspective. This demonstrates that the scale-aware HRNet accurately captures fine-grained pose information, which is critical for predicting pedestrian intentions even when the pedestrian occupies a small region in the frame. Figure 9 emphasizes the importance of scene context where a pedestrian walks with a shoulder bag while checking side posters. Relying solely on pose key points could lead the model to incorrectly infer the crossing intention based on head orientation. However, incorporating both local and global scene information enables the model to correctly predict the pedestrian’s intention. Figure 10 highlights the value of pose information under occlusion, where two pedestrians are overlapping and both are preparing to cross. Although both face the road, the fine-grained pose features captured by the scale-aware HRNet allow the network to accurately determine the crossing intention of each pedestrian in occluded conditions. Finally, Fig. 11 demonstrates improved performance under degraded visual conditions, where the pedestrian’s appearance is blurred and partially obscured. In such a scenario, the combination of bounding box and pose key points ensures correct prediction of crossing intention, illustrating the model’s capability to handle adverse and challenging conditions. Together, these results demonstrate that the proposed model demonstrates consistent intention prediction across scale variation, occlusion, contextual ambiguity, and degraded visual conditions. Overall, the proposed framework demonstrates improved performance under occlusion and small-scale conditions, while its performance remains sensitive to ambiguous pedestrian behavior and extreme visual degradation. Ablation studies Modality ablation To evaluate the contribution of each input modality, pose key points, bounding boxes, local context, and global context, an ablation study was conducted by excluding each feature individually and in combination as shown in Table 5. The results indicate that each modality provides complementary information critical for accurate pedestrian crossing intention prediction. Using only pose key points or bounding boxes yields moderate performance with an accuracy of 0.64 and 0.59 respectively, highlighting that single modalities capture only limited aspects of pedestrian behavior, such as body posture or coarse location, but fail to account for contextual cues. Combining pose and bounding boxes substantially improves accuracy to 0.77, demonstrating that integrating body motion and location cues reduces ambiguity in pedestrian intent, especially under partial occlusion. The inclusion of scene-level features further enhances predictive performance. Using local and global context achieves 0.81 accuracy, showing that environmental cues enable the network to disambiguate pedestrian intentions when pose information alone is insufficient. The best performance (accuracy 0.88, recall 0.81) is obtained when all four modalities are combined, indicating that the integration of visual, non-visual, and contextual information is essential for robust intention prediction across diverse traffic conditions. Notably, even in the absence of visual modalities, the model maintains reasonable accuracy (77%), suggesting that non-visual features provide resilience under severe occlusion or poor visibility. Table 5Evaluation results to analyse the impact of various input features on the intention of the target pedestrian.FeatureAccuracyAUCF1 scorePrecisionRecallOnly Pose Key points0.640.780.410.290.69Only Bounding boxes0.590.690.400.270.75Pose + Bounding boxes0.770.750.490.410.61Only Global Context0.760.830.520.400.72Only Local Context0.770.820.500.410.64Local Context + Global Context0.810.830.520.440.62 Pose + Bounding boxes + Local Context + Global Context 0.88 0.82 0.57 0.44 0.81 Table 5Evaluation results to analyse the impact of various input features on the intention of the target pedestrian.FeatureAccuracyAUCF1 scorePrecisionRecallOnly Pose Key points0.640.780.410.290.69Only Bounding boxes0.590.690.400.270.75Pose + Bounding boxes0.770.750.490.410.61Only Global Context0.760.830.520.400.72Only Local Context0.770.820.500.410.64Local Context + Global Context0.810.830.520.440.62 Pose + Bounding boxes + Local Context + Global Context 0.88 0.82 0.57 0.44 0.81 Evaluation results to analyse the impact of various input features on the intention of the target pedestrian. Temporal pooling ablation This section deals with the Ablation analysis of temporal pooling strategies providing insight into important architectural design choices for aggregating motion information. As shown in Table 6, average pooling across the temporal dimension outperforms max pooling, achieving the highest accuracy (0.88) and balanced performance across precision and recall. This indicates that average pooling provides a more robust aggregation of temporal information, capturing consistent motion patterns while mitigating the influence of transient or noisy frames. This temporal aggregation is particularly important for distant and partially occluded pedestrians, where isolated frames may be misleading. Table 6Performance comparison of the proposed network using different temporal pooling strategies.Temporal poolingAccuracyAUCF1 scorePrecisionRecallMaximum pooling0.760.760.440.380.53Average pooling0.880.820.570.440.81 Table 6Performance comparison of the proposed network using different temporal pooling strategies.Temporal poolingAccuracyAUCF1 scorePrecisionRecallMaximum pooling0.760.760.440.380.53Average pooling0.880.820.570.440.81 Performance comparison of the proposed network using different temporal pooling strategies. Observation length/time-to-event ablation Fig. 12The figure illustrates the observation length, prediction horizon and time-to-event of the pedestrian intending to cross the road. The figure illustrates the observation length, prediction horizon and time-to-event of the pedestrian intending to cross the road. The impact of the observation length and time-to-event (TTE) horizon on model performance was evaluated to determine the optimal prediction window for pedestrian crossing intention with results shown in Table 7. Prior studies on human-vehicle interaction50 suggest that a prediction horizon of 30–60 frames (1–2 s) balances sufficient foresight with the unpredictability of pedestrian behavior in urban traffic as shown in Fig. 12. Using 15 observation frames with a TTE of 60 frames yields the highest accuracy (0.88) and recall (0.81), indicating that this configuration captures sufficient temporal context to anticipate pedestrian crossing intentions reliably. Shorter TTE (30 frames) or longer TTE (90 frames) reduces performance, likely due to insufficient context or increased uncertainty in predicting distant future actions. Table 7Comparison of the performance of the proposed network with change in observation and time-to-event frames.Observation Length (in number of frames)Time -to-Event (in number of frames)AccuracyAUCF1 scorePrecisionRecall15OBS: 15 TTE: 30300.810.770.510.470.56OBS: 45 TTE: 30300.820.870.600.510.72OBS: 15 TTE: 6015600.880.820.570.440.81OBS: 15 TTE: 90900.840.790.450.650.64 Table 7Comparison of the performance of the proposed network with change in observation and time-to-event frames.Observation Length (in number of frames)Time -to-Event (in number of frames)AccuracyAUCF1 scorePrecisionRecall15OBS: 15 TTE: 30300.810.770.510.470.56OBS: 45 TTE: 30300.820.870.600.510.72OBS: 15 TTE: 6015600.880.820.570.440.81OBS: 15 TTE: 90900.840.790.450.650.64 Comparison of the performance of the proposed network with change in observation and time-to-event frames. 15 OBS: 15 TTE: 30 OBS: 15 TTE: 60 15 Class balance ablation Sample imbalance is another critical factor in pedestrian crossing intention prediction, as positive and negative samples are often unevenly distributed in JAAD_All dataset and other real-world traffic events. To assess the model’s robustness to class imbalance, experiments were conducted with both balanced and unbalanced training sets (Table 8). Balancing the dataset marginally improved accuracy to 0.88 compared to 0.84 for the unbalanced case, while substantially improving recall (0.81 vs. 0.24) and F1-score (0.57 vs. 0.35). These results indicate that, although the model can maintain reasonable performance on unbalanced data, balancing training samples significantly enhances its ability to detect positive crossing events, which is critical for safety-critical applications in autonomous driving. Table 8Comparison of the performance of the proposed network with balanced and unbalanced datasets.Data Balancing StrategyAccuracyAUCF1 scorePrecisionRecallWith Class Balancing0.880.820.570.440.81Without Class Balancing0.840.790.350.650.24 Table 8Comparison of the performance of the proposed network with balanced and unbalanced datasets.Data Balancing StrategyAccuracyAUCF1 scorePrecisionRecallWith Class Balancing0.880.820.570.440.81Without Class Balancing0.840.790.350.650.24 Comparison of the performance of the proposed network with balanced and unbalanced datasets. Effect of dynamic modality gating module on IRD dataset To explicitly evaluate the contribution of the proposed dynamic modality gating mechanism, we compare it with static fusion strategies. Specifically, three configurations are considered: i.Direct concatenation (without gating).ii.Equal-weight fusion.iii.Proposed dynamic modality gating. i.Direct concatenation (without gating). Direct concatenation (without gating). ii.Equal-weight fusion. Equal-weight fusion. iii.Proposed dynamic modality gating. Proposed dynamic modality gating. Table 9Effect of dynamic modality gating on IRD dataset.MethodAccuracyF1 ScorePrecisionRecall Concatenation 0.770.490.310.45 Equal-weight fusion 0.790.520.340.48 Dynamic Gating (Ours) 0.820.560.390.50 Table 9Effect of dynamic modality gating on IRD dataset.MethodAccuracyF1 ScorePrecisionRecall Concatenation 0.770.490.310.45 Equal-weight fusion 0.790.520.340.48 Dynamic Gating (Ours) 0.820.560.390.50 Effect of dynamic modality gating on IRD dataset. The results in Table 9 indicate that the proposed dynamic modality gating mechanism consistently improves performance compared to static fusion strategies such as direct concatenation and uniform averaging. This improvement is attributed to the ability of the gating module to adaptively weight visual and non-visual features based on their relative reliability, particularly in scenarios involving occlusion and small-scale pedestrians. In such cases, the model assigns greater importance to motion-based features, leading to improved overall performance. Effect of scale-aware HRNet preprocessing on IRD dataset To evaluate the effectiveness of the proposed scale-aware preprocessing module, we compare the performance of the model using (i) standard HRNet input processing, (ii) HRNet with uniform resizing to 512 × 512, and (iii) the proposed preprocessing strategy. Table 10Effect of scale-aware HRNet preprocessing on IRD dataset.MethodDescriptionAccuracyF1 Score Standard HRNet Direct input without resizing0.780.51 HRNet + Standard Resizing Uniform resizing to 512 × 5120.80.53 Proposed Scale-Aware HRNet (Ours) Dynamic scaling + preprocessing0.820.56 Table 10Effect of scale-aware HRNet preprocessing on IRD dataset.MethodDescriptionAccuracyF1 Score Standard HRNet Direct input without resizing0.780.51 HRNet + Standard Resizing Uniform resizing to 512 × 5120.80.53 Proposed Scale-Aware HRNet (Ours) Dynamic scaling + preprocessing0.820.56 Effect of scale-aware HRNet preprocessing on IRD dataset. The results in Table 10 demonstrate that incorporating preprocessing improves performance over the baseline standard HRNet pipeline. Applying uniform resizing yields moderate gains, as it partially addresses the issue of small-scale pedestrian representation by enlarging the input. However, uniform resizing does not account for variations in pedestrian scale and introduces distortion or loss of contextual information. In contrast, the proposed scale-aware preprocessing adaptively adjusts the input based on bounding box dimensions and further enhances image quality through contrast enhancement and noise reduction. This leads to more reliable keypoint extraction, particularly for small-scale and partially occluded pedestrians, resulting in improved feature representation. These results indicate that the observed performance improvements are not solely attributable to uniform resizing but are primarily driven by the proposed scale-aware preprocessing strategy, which enables more effective context-aware feature extraction for downstream intention prediction. Effect of attention mechanisms (ECA-CAM Ablation) on IRD dataset To evaluate the effectiveness of the attention mechanisms in the proposed architecture, we compare the performance of the model under four configurations: (i) no attention, (ii) ECA only, (iii) CAM only, and (iv) the proposed ECA-CAM module. Table 11Effect of attention mechanisms.MethodAccuracyF1 ScorePrecisionRecall No Attention 0.780.530.360.46 ECA Only 0.800.540.380.48 CAM Only 0.800.540.370.49 ECA-CAM (Ours) 0.820.560.390.50 Table 11Effect of attention mechanisms.MethodAccuracyF1 ScorePrecisionRecall No Attention 0.780.530.360.46 ECA Only 0.800.540.380.48 CAM Only 0.800.540.370.49 ECA-CAM (Ours) 0.820.560.390.50 Effect of attention mechanisms. The results in Table 11 indicate that both channel-wise and spatial attention mechanisms contribute to improved performance compared to the baseline without attention. The ECA module enhances channel-wise feature selection, leading to improvements in precision and F1-score, while the CAM module improves spatial localization, contributing to better recall. The combined ECA-CAM configuration achieves the highest performance among the evaluated configurations, demonstrating that jointly modeling channel and spatial attention results in more discriminative feature representations. This is particularly beneficial in real world scenarios involving occlusion and cluttered backgrounds, where both feature importance and spatial relevance need to be emphasized. Statistical significance analysis JAAD_all dataset To evaluate the statistical stability of the proposed model, we ran 10 independent trials of our proposed method on JAAD_all dataset with all runs using identical data splits and preprocessing to ensure a fair comparison. Table 12 shows the proposed approach achieved a mean accuracy of 0.88 ± 0.006 and an F1 score of 0.57 ± 0.009, showing low variance between the runs. This signifies that the performance seen on JAAD_all is consistently producing reliable results. Table 12Statistical Significance over 10 Runs (JAAD_all Dataset).MetricProposed Method (Mean ± Std)Accuracy0.88 ± 0.006F1-Score0.57 ± 0.009 Table 12Statistical Significance over 10 Runs (JAAD_all Dataset).MetricProposed Method (Mean ± Std)Accuracy0.88 ± 0.006F1-Score0.57 ± 0.009 Statistical Significance over 10 Runs (JAAD_all Dataset). Indian road dataset (IRD) To assess performance on unstructured traffic environments, we further performed statistical analysis on the Indian Road Dataset (IRD) using the same fixed data split across 10 multiple runs. The mean and standard deviation of the performance metrics were computed. The proposed method achieved an average accuracy of 0.82 ± 0.008 and an F1 score of 0.56 ± 0.011, with minimal variance across runs as shown in Table 13. The low standard deviation across runs indicates stable convergence. This demonstrates that the proposed model maintains stable performance despite frequent occlusions and non-compliant pedestrian behavior, supporting the reliability of the proposed framework on real-world Indian road scenarios. Table 13Statistical Significance over 10 Runs (Custom Indian Road Dataset).MetricProposed Method (Mean ± Std)Accuracy0.82 ± 0.008F1-Score0.56 ± 0.011 Table 13Statistical Significance over 10 Runs (Custom Indian Road Dataset).MetricProposed Method (Mean ± Std)Accuracy0.82 ± 0.008F1-Score0.56 ± 0.011 Statistical Significance over 10 Runs (Custom Indian Road Dataset). Performance analysis under occlusion and non-occlusion conditions on IRD dataset To further examine the performance of the proposed model under visually challenging Indian traffic conditions, a dedicated analysis on occluded and non-occluded subsets of the Indian Road Dataset (IRD) was conducted. The separation of samples was based on the binary occlusion labels already provided in the dataset annotations. The trained model was evaluated separately on the two subsets while keeping the model architecture, hyperparameters, and test configuration unchanged. Table 14Occlusion and non-occlusion performance analysis on IRD dataset.ConditionAccuracyF1-ScoreNon-occluded frames0.870.62Occluded frames0.760.51 Overall IRD 0.82 0.56 Table 14Occlusion and non-occlusion performance analysis on IRD dataset.ConditionAccuracyF1-ScoreNon-occluded frames0.870.62Occluded frames0.760.51 Overall IRD 0.82 0.56 Occlusion and non-occlusion performance analysis on IRD dataset. The results in Table 14 indicate that the proposed model achieves higher performance in non-occluded scenarios with 87% accuracy due to clearer visual cues and more reliable pose estimation as expected. However, the model retains reasonable predictive capability under severe occlusion attaining 76% accuracy, demonstrating its ability to leverage non-visual motion cues along with the dynamic modality gating mechanism when visual information is degraded. This behavior is particularly relevant in dense Indian traffic where pedestrians are frequently partially blocked by vehicles and other pedestrians. These findings suggest that the proposed hybrid CNN–GCN architecture with dynamic modality gating contributes to improved resilience against visual uncertainty rather than relying solely on appearance-based features. Figure 13 illustrates that visual representation of the performance degradation under occlusion as expected, while the proposed model still retains reasonable accuracy and F1-score highlighting the effectiveness of the dynamic modality gating mechanism in handling visual uncertainty. Fig. 13Accuracy and F1-score comparison for occluded and non-occluded scenes on IRD. Accuracy and F1-score comparison for occluded and non-occluded scenes on IRD. Performance analysis under varying pedestrian scale (Near vs. Far) on IRD dataset Table 15Performance comparison under pedestrian scale (Near vs. Far).ConditionAccuracyF1 Score Near Pedestrians 0.850.59 Far Pedestrians 0.760.51 Table 15Performance comparison under pedestrian scale (Near vs. Far).ConditionAccuracyF1 Score Near Pedestrians 0.850.59 Far Pedestrians 0.760.51 Performance comparison under pedestrian scale (Near vs. Far). The results in Table 15 show that the proposed model achieves higher performance for near pedestrians, where visual features such as pose keypoints and appearance cues are more clearly captured and reliably extracted. In contrast, performance decreases for far or small-scale pedestrians due to limited spatial resolution, which leads to degraded visual feature quality and less reliable pose estimation. Nevertheless, the model maintains reasonable predictive performance by leveraging motion and contextual features through the dynamic modality gating mechanism. This behavior suggests that the proposed framework can partially compensate for reduced visual reliability by adaptively prioritizing non-visual cues, which is particularly important in real-world traffic scenarios involving distant pedestrians. Analysis of modality weight adaptation To gain deeper insight into the behavior of the proposed dynamic modality gating mechanism, we analyze how modality contributions vary under different conditions, specifically occlusion versus non-occlusion and near versus far pedestrians. Methodology: During inference, the gating module produces a set of normalized weights \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:w=[{w}_{L},{w}_{G},{w}_{P},{w}_{B}],\:$$\end{document}corresponding to the local context, global context, pose, and bounding box feature branches, respectively. These weights are obtained after the SoftMax operation (Eq. 12) and reflect the relative importance assigned to each modality for a given input sample. For analysis, the gating weights are extracted for each test sample and grouped based on (i) occlusion condition using dataset annotations, and (ii) pedestrian scale, determined from bounding box area. Pedestrians are categorized as near or far using a threshold based on the median bounding box area. The modality weights are then averaged across samples within each group. The reported values correspond to the mean of normalized gating weights computed across all samples within each group, ensuring statistically consistent estimation of modality contributions. For clarity of interpretation, the four feature branches are grouped into two categories: visual features(comprising pose (P) and local context (L)) and motion/context features (comprising global context (G) and bounding box dynamics (B)). A. Occlusion vs. Non-Occlusion Analysis. Table 16Average Modality Weights under Occlusion Conditions.ConditionVisual Modality Weight (P + L)Motion/Context Modality Weight (G + B)Non-occluded0.610.39Occluded0.460.54 Table 16Average Modality Weights under Occlusion Conditions.ConditionVisual Modality Weight (P + L)Motion/Context Modality Weight (G + B)Non-occluded0.610.39Occluded0.460.54 Average Modality Weights under Occlusion Conditions. The results in Table 16 show that, in non-occluded scenarios, the model assigns higher weights to visual features, as pose and appearance cues are more reliable. In contrast, under occluded conditions, the model increases reliance on motion and contextual features to compensate for degraded visual information. This adaptive shift indicates that the gating mechanism effectively captures modality reliability and dynamically adjusts feature contributions based on input quality. B. Near vs. Far Pedestrian Analysis. Table 17Average modality weights for Near vs. Far Pedestrians.ConditionVisual Modality Weight (P + L)Motion/Context Modality Weight (G + B)Near pedestrians0.640.36Far pedestrians0.480.52 Table 17Average modality weights for Near vs. Far Pedestrians.ConditionVisual Modality Weight (P + L)Motion/Context Modality Weight (G + B)Near pedestrians0.640.36Far pedestrians0.480.52 Average modality weights for Near vs. Far Pedestrians. The results in Table 17 indicate that for near pedestrians, where spatial resolution is higher and pose estimation is more reliable, the model emphasizes visual features. However, for far or small-scale pedestrians, where visual cues are less reliable, the model shifts its focus toward motion and contextual features. This behavior is consistent with real-world driving scenarios where distant pedestrians often lack sufficient visual detail. Overall, these results demonstrate that the proposed dynamic modality gating mechanism performs context-dependent modality adaptation, rather than relying on fixed fusion strategies. The observed trends confirm that the model automatically prioritizes the most reliable features under varying conditions, contributing to improved performance in challenging environments. Cross-dataset generalization analysis To evaluate the generalization capability of the proposed model across different data distributions, a cross-dataset experiment is conducted. Specifically, the model trained on the JAAD_all dataset is directly evaluated on the PIE dataset without any retraining or fine-tuning. This setting reflects a realistic deployment scenario, where models are expected to generalize to unseen environments. Table 18Cross-Dataset Evaluation.DatasetAccuracyF1 ScorePrecisionRecall JAAD_all 0.880.570.440.81 IRD 0.820.560.390.50 PIE (Cross-Dataset) 0.840.550.430.75 Table 18Cross-Dataset Evaluation.DatasetAccuracyF1 ScorePrecisionRecall JAAD_all 0.880.570.440.81 IRD 0.820.560.390.50 PIE (Cross-Dataset) 0.840.550.430.75 Cross-Dataset Evaluation. The results in Table 18 show that performance on the PIE dataset is comparable to that on JAAD and higher than that on IRD. This behavior can be attributed to the relatively more structured traffic scenarios and reduced visual complexity in PIE compared to IRD, which contains denser interactions and more frequent occlusions. Despite differences in scene characteristics and data distribution, the model maintains stable performance on PIE without retraining, indicating its ability to generalize across datasets and capture transferable behavioral patterns. These findings suggest that the proposed framework is not limited to a single dataset and exhibits reasonable performance under domain shift (Fig. 14). Fig. 14Example predictions on the PIE dataset under cross-dataset evaluation (trained on JAAD_all). The model accurately detects crossing intent in relatively structured and less occluded scenes, highlighting its ability to generalize across datasets and capture transferable pedestrian behavior patterns under domain shift. Example predictions on the PIE dataset under cross-dataset evaluation (trained on JAAD_all). The model accurately detects crossing intent in relatively structured and less occluded scenes, highlighting its ability to generalize across datasets and capture transferable pedestrian behavior patterns under domain shift. Edge device deployment To evaluate the practical feasibility of the proposed framework in real-world deployment scenarios, the model is tested on Jetson Xavier NX (an embedded edge device). All latency measurements are conducted under identical conditions with batch size = 1 and include the complete inference pipeline, including HRNet-based keypoint extraction and the multimodal fusion network. All measurements are conducted under identical inference settings to ensure fair and consistent comparison across platforms. Table 19Edge device performance analysis.PlatformFPSLatency (ms)GPU Utilization (%)Memory (GB)Power (W) RTX 4060 desktop 303345%3.2118 Jetson Xavier NX 119178%2.112 Table 19Edge device performance analysis.PlatformFPSLatency (ms)GPU Utilization (%)Memory (GB)Power (W) RTX 4060 desktop 303345%3.2118 Jetson Xavier NX 119178%2.112 Edge device performance analysis. Memory (GB) denotes the peak GPU memory usage during inference, measured while executing the complete pipeline (feature extraction and fusion network).The results in Table 19 show that the proposed model maintains moderate GPU utilization and low power consumption on the embedded platform, while operating within acceptable memory limits. Despite reduced throughput compared to the desktop GPU, the system achieves near real-time performance with efficient resource usage. These findings indicate that the proposed architecture achieves a favorable trade-off between computational efficiency and predictive performance, making it suitable for deployment in resource-constrained edge environments. Failure case analysis Despite the observed improvements, the proposed model exhibits certain limitations under challenging real-world conditions.False positives are primarily observed in scenarios where pedestrians exhibit ambiguous motion patterns, such as standing near the curb, slight body orientation changes, or intermittent movements without clear crossing intent. In such cases, the model incorrectly predict crossing behavior due to similarities with pre-crossing motion patterns, which closely resembles actual crossing intent as shown in Fig. 15 (a) and(b). False negatives occur in situations involving severe occlusion, sudden crossing behavior with limited prior motion cues. In these scenarios, the model fails to capture sufficient temporal or visual evidence for accurate prediction. Notably, two types of false negatives are observed: (i) where the pedestrian is correctly detected but misclassified due to incomplete motion cues as shown in Fig. 15(c) and (ii) where the pedestrian is poorly localized, leading to the absence of reliable pose features as shown in Fig. 15(d). From an error analysis perspective, most errors arise in cases where the distinction between waiting and crossing intent is subtle, particularly in dense and unstructured traffic environments. This contributes to the relatively lower precision observed in the results.These observations highlight the inherent difficulty of pedestrian intention prediction in real-world environments and provide a clear explanation for the observed limitations. They also indicate potential directions for improvement, such as incorporating richer temporal context and more discriminative behavioral cues to better handle such edge cases. Fig. 15Example failure cases of the proposed model on IRD dataset. (a) and (b) False positive cases where pedestrians standing near the roadside are incorrectly predicted as crossing due to ambiguous pre-crossing motion cues, despite correct localization and pose estimation (c) False negative case, where a crossing pedestrian is correctly detected but misclassified due to subtle or insufficient motion cues. (d) False negative case, where the pedestrian is poorly localized under occlusion and small-scale appearance, leading to missing pose information and prediction failure. Example failure cases of the proposed model on IRD dataset. (a) and (b) False positive cases where pedestrians standing near the roadside are incorrectly predicted as crossing due to ambiguous pre-crossing motion cues, despite correct localization and pose estimation (c) False negative case, where a crossing pedestrian is correctly detected but misclassified due to subtle or insufficient motion cues. (d) False negative case, where the pedestrian is poorly localized under occlusion and small-scale appearance, leading to missing pose information and prediction failure. Conclusion and future works Pedestrian crossing intention prediction remains a critical challenge for autonomous driving, particularly in unstructured, high-density, and frequently occluded road environments such as those in India. Existing models face three key limitations such as brittle static feature fusion, insufficient representative datasets, and degraded pose estimation for distant pedestrians. To address these issues, we propose a hybrid CNN-GCN architecture with a dynamic modality gating module that enables sample-dependent fusion of visual and non-visual features. This design allows the model to adaptively prioritize reliable cues under varying visual conditions. To improve feature quality under challenging scenarios, a scale-aware preprocessing strategy integrated with HRNet was introduced for enhanced keypoint extraction, particularly for distant and partially occluded pedestrians. Additionally, the Indian Road Dataset (IRD) was collected to capture the complexities of unstructured Indian roads. The proposed method was evaluated on JAAD_all, IRD, and in a cross-dataset setting on PIE. The results indicate competitive performance across datasets, while additional analyses under occlusion and pedestrian scale variations provide insights into model behavior under degraded visual conditions. Furthermore, edge-device evaluation on Jetson Xavier NX demonstrates near real-time performance, highlighting the practical feasibility of the approach in resource-constrained environments. Experimental results demonstrate that the proposed framework achieves moderate accuracy on JAAD (88%) and better performance on IRD (82%), while maintaining the lower inference latency (1.5 ms), highlighting its suitability for real-time deployment on edge devices in autonomous driving systems. Despite these improvements, certain limitations remain. The model exhibits reduced precision in scenarios involving ambiguous pre-crossing behaviors and heavy occlusion, reflecting the inherent difficulty of distinguishing subtle pedestrian intent in unstructured settings which leads to a lower precision. Future work will focus on incorporating richer temporal modeling and behavioral cues to better capture subtle intent transitions. Exploring class balancing strategies such as re-weighting and resampling to further improve precision and overall performance is also a potential direction. Extending the model to capture group dynamics and social interactions among multiple pedestrians to improve prediction in crowded environments is a promising research direction. Integrating intention prediction with trajectory forecasting could provide a unified framework for anticipating pedestrian behavior more accurately. Finally, validating the model on unstructured traffic data sets from other geographic regions (e.g., Southeast Asia and South America) will further establish its generalizability and help uncover region-specific challenges15,20,30,32,33,41,45,51,52.