Radar-vision multimodal fusion for dynamic target trajectory prediction and threat assessment in power transmission corridors Authors: Jun Zhang, Zhiwei Zhang, Xiao Tan, Jin Ling, Qing Jie Deng Published: 2026 ABSTRACT AbstractPower transmission corridors face growing intrusion risks from unmanned aerial vehicles (UAVs), construction machinery, and wind-borne debris. Conventional single-sensor monitoring, however, covers less than 20% of corridor length and falls short of the sub-meter, real-time accuracy needed for rapid threat response. To close this gap, we present a radar-vision multimodal fusion framework that integrates and adapts established sensing and learning techniques into a coherent end-to-end pipeline tailored to the corridor protection domain. The framework makes three principal contributions at the system-integration level. First, a reliability-aware adaptive fusion strategy learns environment-dependent modality weights through sigmoid-gated quality indicators—signal-to-noise ratio, detection confidence, illumination variance, and a weather-condition index—enabling the network to shift reliance toward whichever sensor is more trustworthy rather than averaging degraded inputs. Second, a cross-modal attention module allows radar kinematic queries to interrogate vision semantic keys, uncovering complementary feature correspondences without manual supervision. Third, a category-specific hierarchical threat scorer combines distance, velocity, and trajectory-trend sub-scores with target-class multipliers and maps a continuous composite value onto five severity levels (Negligible, Low, Moderate, High, Critical) through field-calibrated thresholds. We validated the framework on 8,547 trajectory sequences collected over six months from twelve monitoring stations spanning suburban, mountainous, and agricultural terrains. Mean Absolute Error (MAE) of predicted positions reached 1.62 m at a 3-second horizon and 3.21 m at 5 s—a 43% relative reduction compared with the radar-only baseline (2.84 m at 3 s). Threat-level classification achieved 89.6% precision, 87.3% recall, and an 88.4% macro-averaged F1 score across five classes, exceeding early-fusion and late-fusion alternatives by 7.4 and 9.8% points respectively. Deployment at three operational 500 kV / 220 kV sites detected 89 genuine threats with 17.8 s mean warning lead time, 3.2% false-alarm rate, and 99.4% system uptime. Operator workload—measured as person-hours of continuous manual surveillance—fell by 67%, while instrumented detection coverage expanded from 18% to 94% of corridor length.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-026-48978-2.Keywords: Power transmission corridor, Multimodal fusion, Trajectory prediction, Threat assessment, Radar-vision integration, Cross-modal attentionSubject terms: Engineering, Mathematics and computing Introduction Power transmission corridors are lifeline infrastructures for long-distance electrical energy transport. They encompass high-voltage lines, support towers, insulators, and 30–50 m safety buffer zones. These corridors form the backbone of modern grids, spanning hundreds of kilometers at 110 kV-1000 kV. A single 500 kV line serves millions of households; disruptions cascade through interconnected systems. Despite their criticality, corridors face increasing threats from UAVs, construction machinery, and airborne debris1. Threat mechanisms vary but share catastrophic potential. UAVs pose collision risks severing conductors or damaging insulators, causing fires and multi-hour outages affecting 50,000 + customers with $200,000 + repair costs2. Construction machinery creates flashover conditions through close approach—electrical arcing occurs without contact, ionizing air gaps to carbonize insulation. Airborne debris (plastic sheeting, kites, metallicized balloons) causes phase bridging and short-circuit faults. Agricultural harvesting releases materials causing seasonal fault patterns. Each threat type exhibits distinct signatures: controllable drone trajectories, terrain-constrained machinery paths, wind-driven debris drift. All demand rapid detection for timely intervention. Traditional monitoring relies on manual inspection with periodic visual surveys, supplemented by single-sensor systems3. This paradigm has fundamental limitations. Manual inspection provides only weekly/monthly coverage with temporal gaps for undetected intrusions. Single sensors face inherent constraints: radar measures range and velocity but struggles with classification; cameras provide visual detail but degrade under fog, rain, or darkness. Real-time response for fast-approaching targets exceeds manual monitoring capabilities. Accuracy requirements demand sub-meter precision that conventional approaches cannot deliver. These inadequacies motivate advanced multimodal sensing frameworks. Radar and vision technologies have emerged as complementary sensing modalities for power transmission corridor monitoring. Radar systems provide all-weather detection capabilities and precise distance-velocity measurements, effectively tracking targets under adverse environmental conditions4. Vision systems offer rich texture and semantic information, enabling detailed target recognition and behavior analysis5. Recent studies have explored their integration in power system applications, demonstrating improved detection performance over individual sensors6. However, most existing implementations focus on simple data-level fusion or parallel processing, lacking sophisticated feature-level integration mechanisms that fully exploit the complementary characteristics of these modalities7. Dynamic target trajectory prediction has received considerable attention in autonomous systems and intelligent surveillance domains. Model-based methods employing Kalman filtering and particle filtering have been widely applied but struggle with complex nonlinear motion patterns8. Deep learning approaches, particularly recurrent neural networks and attention mechanisms, have shown superior performance in capturing temporal dependencies and long-term motion trends9. In the context of power transmission corridors, several research efforts have investigated trajectory forecasting for aerial vehicles and ground equipment10. Nevertheless, these methods predominantly rely on single-modal data, underutilizing the spatial-temporal information available from multimodal sensors. Threat assessment constitutes a crucial component of intelligent monitoring systems, enabling proactive risk management and decision support. Existing threat evaluation frameworks typically employ rule-based expert systems or fuzzy logic approaches, which depend heavily on manually designed features and predefined thresholds11. Machine learning-based methods have demonstrated improved adaptability but often treat threat assessment as a binary classification problem, providing insufficient granularity for operational decision-making12. The challenge of quantifying threat levels from heterogeneous sensor data while considering multiple risk factors remains inadequately addressed in current literature13. The integration of radar and vision modalities for trajectory prediction and threat assessment in power transmission corridors presents several fundamental challenges. First, the heterogeneity in data characteristics, temporal resolution, and spatial reference frames complicates effective feature alignment and fusion. Second, the dynamic and unpredictable nature of target behaviors in complex environments demands robust prediction models capable of handling multimodal uncertainties. Third, comprehensive threat evaluation requires not only accurate trajectory forecasting but also contextual reasoning about proximity, speed, heading, and target type. These challenges necessitate innovative approaches that transcend conventional multimodal fusion paradigms14. Against this backdrop, we put forward a radar-vision multimodal fusion framework for trajectory prediction and threat assessment in power transmission corridors. None of the individual techniques we employ—attention mechanisms, LSTM-based sequence modeling, sigmoid gating—are new in isolation. Our contribution lies in how these components are integrated, adapted, and validated for the specific demands of corridor protection, where sensor degradation is the norm rather than the exception and where operators require interpretable, graded alerts rather than opaque binary flags. The contributions of this work, each addressing a specific shortcoming identified in prior approaches, are outlined below. First, unlike conventional fusion schemes that assign fixed or heuristic weights to each sensor channel, we integrate a reliability-aware adaptive fusion strategy in which sigmoid-gated quality indicators—radar SNR, detection confidence, illumination variance, and a weather-condition index—continuously modulate modality contributions. This mechanism lets the system shift reliance toward the more dependable sensor under fog, rain, or darkness rather than simply averaging degraded inputs. Second, we adapt a cross-modal attention module to the radar-vision setting, where radar kinematic features serve as queries and vision semantic features serve as keys and values. Compared with early-fusion concatenation or late-fusion score averaging, this configuration discovers complementary correspondences between heterogeneous representations without hand-crafted alignment rules, reducing prediction error by 26.7% relative to early fusion at 5-second horizons. Third, we develop a hierarchical, category-specific threat assessment framework that fuses distance, velocity, and trajectory-trend sub-scores with target-class multipliers and maps the resulting composite value onto five severity levels through field-calibrated thresholds. In contrast to binary threat classifiers reported in prior work12, this multi-level design delivers the graded situational awareness that field operators require for proportionate response—and, importantly, it remains fully interpretable and auditable, a property that power-grid regulators increasingly mandate for automated decision support. Taken together, these three elements form an end-to-end pipeline from raw sensor input to actionable threat classification, validated on 8,547 real-world trajectory sequences and deployed at three operational monitoring sites over six months15. The remainder of this paper is organized as follows. Section II surveys radar target detection, visual recognition, and multimodal fusion techniques, situating our work within the current literature. Section III details the proposed framework: overall five-layer architecture (Sect. 3.1), multimodal feature fusion network with cross-modal attention and adaptive weight learning (Sect. 3.2), and the LSTM-based trajectory predictor coupled with the hierarchical threat assessment algorithm (Sect. 3.3). Section IV describes the experimental setup, dataset composition, ablation studies, and comparative results, followed by field deployment case analyses. Section V discusses the sources of performance gains, practical deployment considerations, operational limitations, and system security and reliability concerns. Section VI concludes the paper with a summary of contributions and concrete future research directions. Background and related technologies Radar target detection and tracking technology Radar systems extract target parameters by transmitting electromagnetic pulses and analyzing backscattered signals. They operate reliably under varying illumination and weather conditions16. Standard processing employs pulse compression, Doppler filtering, and coherent integration for enhanced sensitivity17. CFAR algorithms maintain consistent false alarm rates through adaptive thresholding18,19, while Kalman and particle filters estimate target states from sequential measurements20,21. For power corridor monitoring, radar delivers range, azimuth, elevation, velocity, and cross-section data for robust long-range tracking. Visual target recognition and feature extraction technology Vision systems convert optical signals into digital images preserving color, texture, and geometric details22. Deep learning detectors like YOLO and Faster R-CNN enable real-time recognition23,24, while feature extraction captures texture and motion for tracking25. Vision provides rich semantic context but degrades under poor lighting, rain, and fog, with shorter effective ranges than radar26. Multimodal data fusion methods Multimodal fusion seeks to combine heterogeneous sensor streams so that the resulting representation exceeds what any single modality can deliver alone27. The literature broadly groups fusion strategies into three tiers: data-level fusion, which concatenates raw measurements before any abstraction takes place; feature-level fusion, which merges learned intermediate representations; and decision-level fusion, which aggregates independent inference outputs28. Among these, feature-level approaches offer a pragmatic middle ground—they preserve richer information than decision-level methods while imposing lighter computational overhead than raw-data concatenation—and are therefore well suited to radar-vision integration where radar supplies kinematic state vectors and vision provides semantic classification cues. In recent years, attention-based fusion has attracted considerable interest. Vaswani et al.29 introduced the scaled dot-product attention mechanism, which was originally designed for sequence-to-sequence language modeling but has since been widely adopted for cross-modal feature weighting. The general attention-weighted fusion can be expressed as Eq. (5). Several extensions have emerged that specifically target sensor fusion problems. For instance, Liang et al.30 proposed a deep continuous fusion architecture that projects LiDAR bird-eye-view features into camera feature maps through continuous convolutions, demonstrating that spatially aware cross-modal attention substantially outperforms naive concatenation for 3-D object detection. Similarly, Chen et al.31 developed a multi-view 3-D detection network in which region-based fusion layers encode interactions between LiDAR point clouds and multi-view camera images, establishing an influential paradigm for heterogeneous sensor integration. More recently, Liu et al.32 presented BEVFusion, a unified bird-eye-view representation that fuses camera and LiDAR modalities in a shared spatial coordinate frame, achieving state-of-the-art results on the nuScenes benchmark with minimal latency overhead. where represents the fused feature, denotes the number of modalities, indicates the attention weight for modality , and represents the feature from modality 29. The attention weights are learned through neural networks that assess feature quality and contextual importance. Trajectory prediction has likewise evolved rapidly. Classical model-based estimators—Kalman filters and their extended or unscented variants—remain effective for near-linear dynamics, yet they degrade when targets execute sharp turns, sudden stops, or erratic maneuvers8. Recurrent architectures, especially LSTMs33, capture longer-range temporal dependencies through gating mechanisms, and several studies have demonstrated their advantages for aircraft and vehicle path forecasting9. More recently, Transformer-based predictors have shown competitive results; Giuliari et al.34 applied a Transformer encoder-decoder to pedestrian trajectory prediction and found that self-attention over time steps naturally handles variable-length histories without the vanishing-gradient issues of recurrent cells. Yuan et al.35 introduced AgentFormer, a stochastic Transformer that models the joint distribution of multi-agent futures, capturing social interactions that recurrent baselines struggle to represent. In the specific context of power corridor monitoring, Zhao et al.10 demonstrated UAV tracking control strategies, while Li et al.6 explored visual matching for safety monitoring; however, neither study combined radar kinematic data with vision semantics in a learned fusion framework, leaving a gap that our work directly addresses. Spatial-temporal alignment remains a foundational challenge when integrating sensors that operate in different coordinate frames and at disparate sampling rates. Coordinate transformation projects measurements into a common Cartesian space, while temporal interpolation resamples asynchronous observations onto a coherent time grid. Adaptive fusion weight strategies dynamically adjust modality contributions according to environmental conditions and sensor reliability, as formulated in Eq. (6). where w_i(t) is the time-varying weight for modality i, q_i(t) denotes a quality metric derived from sensor diagnostics, and β controls the sharpness of the softmax distribution (standard softmax temperature formulation). Cao et al.36 showed that adaptive fusion of this kind substantially improves multi-object tracking robustness under partial occlusion and sensor degradation, a finding that motivates our own reliability-aware weighting scheme. Beyond fusion and prediction, threat assessment for critical infrastructure has received growing attention. Rule-based expert systems and fuzzy-logic classifiers11 remain common in operational deployments, yet they rely on hand-crafted features and fixed thresholds that generalize poorly across sites. Machine-learning approaches have improved adaptability, but many treat threat evaluation as a binary classification problem, sacrificing the granularity that operators need for graduated response12,13. Wang et al.37 proposed a probabilistic risk assessment framework for smart grid anomaly detection that integrates heterogeneous sensor streams, yet their work focused on static network faults rather than dynamic moving targets. Our hierarchical, category-specific scoring framework builds on these foundations while introducing target-class multipliers and field-calibrated severity thresholds designed for the moving-target domain. Radar-vision fusion-based trajectory prediction and threat assessment model This section presents our integrated framework addressing three core challenges: fusing heterogeneous radar-vision data, predicting target trajectories, and assessing threat levels. We first describe the overall architecture connecting five processing layers. Then we detail the multimodal fusion network with cross-modal attention. Finally, we explain the LSTM-based trajectory predictor and hierarchical threat assessment algorithm. Each component builds upon the previous, creating an end-to-end system from raw sensor input to actionable threat classification. Overall model framework design Our model processes multimodal sensor data through five hierarchical layers for end-to-end trajectory prediction and threat assessment38. Figure 1 shows the complete pipeline from raw data to threat classification. The five stages handle distinct tasks: data acquisition captures and synchronizes sensor streams; feature extraction transforms measurements into semantic representations; multimodal fusion integrates information through attention mechanisms; trajectory prediction forecasts future positions; threat assessment evaluates risk levels. This separation enables modular optimization while maintaining coherent information flow. Overall architecture of the radar-vision fusion-based trajectory prediction and threat assessment model. Figure 1 depicts the five-layer architecture. From bottom to top, the data acquisition layer synchronizes radar and camera streams at 20 Hz; the feature extraction layer encodes radar kinematics via a three-layer MLP and visual semantics via ResNet-50, each producing 512-dimensional vectors; the fusion layer applies cross-modal attention with sigmoid-gated adaptive weighting; the trajectory prediction layer forecasts future positions through a stacked LSTM; and the threat assessment layer maps composite risk scores onto five severity levels to guide graduated operator responses. Detailed descriptions of each layer follow in Sect. 3.2 and 3.3. Radar systems continuously monitor the surveillance region with update rates of 10–20 Hz, capturing range, azimuth, elevation, and Doppler velocity for detected targets. Vision cameras operate at 25–30 frames per second, providing high-resolution imagery covering overlapping fields of view with radar coverage zones. Spatial-temporal registration modules within this layer perform coordinate transformation and time synchronization to align heterogeneous sensor data into unified reference frames39. The feature extraction layer processes aligned multimodal data through parallel processing channels optimized for each modality. Radar feature extractors compute kinematic parameters, target trajectories, and motion patterns from sequential measurements, while vision feature extractors employ deep convolutional networks to generate semantic embeddings, appearance descriptors, and target classification probabilities. This layer outputs fixed-dimensional feature vectors encapsulating complementary information from both modalities, preparing data for subsequent fusion operations. The multimodal fusion layer integrates radar and vision features through an adaptive attention-based mechanism that dynamically weights modality contributions according to environmental conditions and sensor reliability. Cross-modal correlation modules identify corresponding targets across different sensor streams, resolving potential ambiguities through joint analysis of spatial positions, motion characteristics, and appearance attributes. The fusion layer generates enriched feature representations that leverage radar’s precise kinematic measurements and vision’s rich semantic information40. The trajectory prediction layer employs a temporal encoder-decoder architecture with long short-term memory networks to model target motion dynamics from fused multimodal features. Historical trajectory sequences spanning 3–5 s serve as input, enabling the prediction of future positions over 5–10 s horizons. Attention mechanisms within this layer selectively focus on relevant temporal patterns and multimodal cues, adapting predictions to diverse target behaviors including uniform motion, acceleration, and turning maneuvers. The threat assessment layer evaluates predicted trajectories against multiple risk criteria—proximity to transmission lines, approach velocity, heading angles, and target classification—and maps weighted composite scores onto five severity levels whose thresholds and alert actions are detailed in Sect. 3.3 (Tables 1, 2, 3, 4, 5 and 6)41. Table 1UAV threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.15< 0.15DivergingRoutine monitoring1Low0.15–0.350.15–0.35ParallelEnhanced tracking2Moderate0.35–0.500.35–0.50ApproachingOperator notification3High0.50–0.750.50–0.75Direct approachImmediate warning4Critical> 0.75> 0.75Imminent collisionEmergency response UAV threat level classification standards. Table 2Construction machinery threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.20< 0.25DivergingRoutine monitoring1Low0.20–0.400.25–0.45ParallelEnhanced tracking2Moderate0.40–0.600.45–0.65ApproachingOperator notification3High0.60–0.800.65–0.80Direct approachImmediate warning4Critical> 0.80> 0.80Imminent collisionEmergency response Construction machinery threat level classification standards. Table 3Bird threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.30< 0.30DivergingRoutine monitoring1Low0.30–0.500.30–0.50ParallelEnhanced tracking2Moderate0.50–0.700.50–0.65ApproachingOperator notification3High0.70–0.850.65–0.85Direct approachImmediate warning4Critical> 0.85> 0.85Imminent collisionEmergency response Bird threat level classification standards. Table 4Balloon threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.25< 0.20DivergingRoutine monitoring1Low0.25–0.450.20–0.40ParallelEnhanced tracking2Moderate0.45–0.650.40–0.60ApproachingOperator notification3High0.65–0.800.60–0.75Direct approachImmediate warning4Critical> 0.80> 0.75Imminent collisionEmergency response Balloon threat level classification standards. Table 5Kite threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.22< 0.22DivergingRoutine monitoring1Low0.22–0.420.22–0.42ParallelEnhanced tracking2Moderate0.42–0.620.42–0.60ApproachingOperator notification3High0.62–0.780.60–0.78Direct approachImmediate warning4Critical> 0.78> 0.78Imminent collisionEmergency response Kite threat level classification standards. Table 6Airborne debris threat level classification standards.Threat LevelLevel NameDistance ThreatVelocity ThreatTrajectory TrendAlert Action0Negligible< 0.28< 0.25DivergingRoutine monitoring1Low0.28–0.480.25–0.45ParallelEnhanced tracking2Moderate0.48–0.680.45–0.63ApproachingOperator notification3High0.68–0.830.63–0.80Direct approachImmediate warning4Critical> 0.83> 0.80Imminent collisionEmergency response Airborne debris threat level classification standards. Data flows sequentially through the five layers, with feedback connections from the fusion and prediction stages allowing adaptive refinement of feature extraction parameters. The model ingests multimodal sensor streams and produces three outputs: predicted trajectories with uncertainty bounds, threat-level classifications with confidence scores, and visualization overlays for operator interfaces. Processing operates on sliding temporal windows, and the architecture achieves end-to-end latencies below 200 milliseconds through parallel computation, meeting the real-time requirements of power transmission monitoring. Multimodal feature fusion network The fusion network integrates radar and vision data through specialized pathways exploiting each modality’s strengths42. Figure 2 shows parallel encoding streams transforming raw inputs into aligned representations before cross-modal attention synthesizes unified descriptors. Data flows left to right through four stages: modality encoding, spatial-temporal alignment, attention-based correlation, and adaptive weight fusion. Architecture of the multimodal feature fusion network with cross-modal attention mechanism. Figure 2 illustrates the four-stage fusion pipeline. The radar and vision encoders (left) produce 512-dimensional representations and from raw measurements and image patches, respectively. A spatial-temporal alignment module (center) resamples both streams onto a common 20 Hz grid and transforms radar spherical coordinates into the camera Cartesian frame. Cross-modal attention (right-center) uses radar-derived queries to interrogate vision-derived keys and values, producing an attention-weighted descriptor that augments kinematics with semantic context. Finally, adaptive weight learning (right) applies sigmoid-gated reliability scores to balance modality contributions before PCA compression to 256 dimensions. The mathematical formulations for each stage are detailed in Eqs. (7)–(13). The radar feature encoder processes sequential measurements through a multilayer perceptron architecture with residual connections. Input radar data at time step comprises a feature vector , where , , , , and represent range, azimuth, elevation, velocity, and radar cross section respectively. The encoding operation transforms raw measurements into high-dimensional representations: where , , denote weight matrices, , , represent bias vectors, and indicates the encoded radar feature. Batch normalization layers follow each activation function to stabilize training dynamics. The vision feature encoder adopts a ResNet-50 backbone pretrained on ImageNet, extracting hierarchical visual representations from target image patches43. As shown in Fig. 2, the encoder removes the final classification layer and extracts features from the last residual block, generating a 2048-dimensional feature vector. A subsequent projection layer compresses the representation: where denotes the input image at time , represents the feature extraction function, is the projection matrix, and represents the encoded vision feature. This dimensionality alignment facilitates subsequent fusion operations. The feature alignment module synchronizes temporal and spatial inconsistencies between radar and vision modalities. Temporal alignment employs linear interpolation to resample asynchronous observations onto a common time grid with 20 Hz sampling rate. Spatial alignment transforms radar measurements from spherical coordinates to Cartesian space aligned with camera reference frames: where represents the rotation matrix from radar to camera coordinates, and denotes the translation vector determined through extrinsic calibration. The cross-modal attention fusion mechanism computes correlation weights between the 512-dimensional radar and vision feature vectors at each time step, operating at the modality level rather than over spatial tokens, temporal sequences, or image regions. In concrete terms, the radar encoding h_t^(r) serves as the query source and the vision encoding h_t^(v) serves as the key-value source, so that the module learns which aspects of the visual semantics are most informative for augmenting the kinematic representation at time t. We adopt a single-head attention design following the scaled dot-product formulation of Vaswani et al.29: where , , are learnable projection matrices. The attention weights quantify feature relevance: where d_k = 256 is the key dimension that scales the dot products to prevent gradient saturation in the softmax. The resulting attention-weighted vision feature A_rv V^(v) enriches the radar representation with semantic context44. We chose single-head attention over its multi-head counterpart after empirical comparison: a four-head variant yielded only a 0.3% increase in F1 with 18% greater latency—an unfavorable trade-off given our real-time constraint. The single-head design keeps inference below 2.5 ms for the fusion stage while retaining sufficient capacity to model radar-vision correspondences. Adaptive weight learning dynamically adjusts modality contributions based on input quality indicators. Environmental factors including illumination variance , weather condition index , and signal-to-noise ratio determine reliability scores: where denotes the sigmoid function, represents radar detection confidence, and , are learnable weight vectors. The final fused feature combines modality-specific encodings weighted by learned coefficients45: Dimensionality reduction through principal component analysis retains 95% variance while compressing to 256 dimensions, balancing representational capacity with computational efficiency for downstream trajectory prediction. The resulting compact representation encapsulates complementary spatial, kinematic, and semantic information from both sensing modalities. Most equations in this manuscript build on well-established formulations whose sources are cited inline: scaled dot-product attention29, ResNet-50 feature extraction43, sigmoid gating45, LSTM sequence modeling33, and dropout-based uncertainty estimation46. Equations that represent original contributions of this work include: Eq. (13), which extends the adaptive tracking formulation of Cao et al.36 with explicit reliability normalization for our fusion rule; Eqs. (16) and (17), which define the distance and velocity threat sub-scores with domain-specific safety parameters; and the composite threat score formula with class multiplier γ_class, which draws on multi-dimensional scoring concepts from Chen et al.47. Trajectory prediction and threat assessment algorithm Why use LSTM instead of simpler methods? Target behaviors in power corridors are nonlinear and unpredictable33. Constant-velocity extrapolation and Kalman filtering work for straight-line motion but fail when drones turn sharply, vehicles decelerate, or debris tumbles erratically. Our 256-dimensional fused features contain three semantic categories: spatial position, kinematic trends, and classification probabilities. Each 3-second sequence provides 60 frames at 20 Hz—rich temporal structure that simple rules cannot exploit. LSTM gating mechanisms selectively preserve relevant history while forgetting noise. Ablation studies (Sect. 4.2) confirm this: replacing LSTM with linear regression increases 5-second error by 127%; Kalman filtering degrades accuracy by 89%. The stacked LSTM has two 256-unit layers followed by linear projection to Cartesian coordinates. Given historical features , the network generates predictions over frames (5 s), capturing velocity, acceleration, and direction changes through learned patterns. For example, consider a drone at position meters. The LSTM processes 60-frame history and predicts —slight approach toward lines. Autoregressive decoding continues until , within 25 m of conductors, triggering threat protocols. We estimate uncertainty via Monte Carlo dropout sampling. The threat assessment framework integrates spatial proximity, approach kinematics, and trajectory trends into hierarchical classifications. Figure 3 shows the decision flowchart transforming predictions into threat levels. Threat assessment decision flowchart. The process computes three scores from predicted positions. Distance threat: where is minimum distance to lines and meters; values near 1.0 indicate critical proximity. Velocity threat: for inbound motion, with m/s threshold. Trajectory trend: angular deviation between velocity and line orientation, with direct approaches scoring higher. For the drone example at meters: m yields ; radial velocity 8.6 m/s gives ; 12-degree angular deviation produces . Composite score: , mapping to Level 2 (Moderate), triggering operator notification. Temporal feature extraction within the LSTM layers captures velocity, acceleration, and directional changes through learned representations of sequential patterns. The network implicitly learns motion dynamics without explicit feature engineering, adapting to diverse target behaviors including linear trajectories, curved paths, and sudden maneuvers. Residual connections between LSTM layers facilitate gradient flow during training, enabling effective learning of long-term dependencies. Multi-step prediction generates future trajectory points over a horizon of frames (5 s). The decoder employs an autoregressive strategy where predicted positions at step condition subsequent predictions: where denotes the predicted position at time , and represents the decoder network parameterized by . This recursive formulation accumulates prediction uncertainty over the horizon, which is quantified through variance estimation from dropout sampling during inference46. The threat assessment framework evaluates predicted trajectories against multiple risk dimensions to generate comprehensive threat scores. Distance threat degree quantifies spatial proximity to transmission infrastructure: where represents the minimum predicted distance to transmission lines over the prediction horizon, and meters defines the safety buffer zone. Values approaching unity indicate critical proximity violations. Velocity threat degree assesses approach speed toward protected infrastructure: where denotes the radial velocity component toward transmission lines, and m/s establishes the critical approach speed threshold. Positive values indicate inbound motion requiring heightened monitoring. Trajectory trend threat degree evaluates whether predicted paths exhibit convergent motion toward infrastructure. Angular deviation between velocity vectors and transmission line directions determines trend risk, with small angles indicating direct approach trajectories. Historical trajectory smoothness and acceleration patterns further refine this metric to distinguish deliberate approaches from random motion. A deliberate design boundary separates the learned and rule-based portions of our pipeline. Trajectory prediction is handled by the LSTM, whose parameters are optimized end-to-end from data. Threat classification, by contrast, is threshold-based: the three sub-scores—, , and —are computed from closed-form analytical expressions (Eqs. 16–17 and the angular deviation formula) applied to the LSTM-predicted trajectory, then combined into a composite value through fixed, domain-informed weights (, , ) and a target-class multiplier . This composite value is mapped to one of five severity levels by comparing it against the threshold boundaries specified in Tables 1, 2, 3, 4, 5 and 6. We considered—and ultimately rejected—replacing the threshold stage with a learned classifier, for three reasons grounded in the operational realities of power-grid monitoring. First, regulatory and operational norms in the power sector increasingly demand that automated alert systems produce decisions traceable to interpretable physical quantities; a black-box classifier that outputs a threat label without transparent reasoning would face resistance from both grid operators and safety auditors. Second, high-severity events (Levels 3 and 4) are inherently rare in our dataset—they account for fewer than 6% of all labeled sequences—creating a severe class imbalance that degrades learned classifiers precisely for the categories where misclassification is most dangerous. During preliminary experiments, a three-layer MLP trained to predict threat levels directly from fused features achieved only 71.3% recall on Level 4 threats, compared with 94.2% under our threshold scheme, because the limited number of critical-event training samples was insufficient for reliable generalization. Third, threshold tables can be updated on site—when a new line configuration is commissioned or safety regulations change—without retraining the neural network, a practical necessity given that retraining requires GPU resources and validated data that field stations do not always have available. We acknowledge that a learned threat model could, in principle, capture nonlinear interactions among risk dimensions that fixed weights miss, and we discuss this as a concrete future research direction in Section VI. Different targets require different assessment criteria. UAVs warrant stricter thresholds—operators can deliberately steer them toward infrastructure. Birds and balloons drift passively, rarely sustaining approach trajectories. Construction machinery causes catastrophic damage but moves predictably. Lightweight debris creates entanglement hazards despite low impact. Tables 1, 2, 3, 4, 5 and 6 define category-specific thresholds reflecting these risk profiles. Table 1 applies strictest UAV criteria—Level 3 activates at 0.5 distance threat, accounting for rapid acceleration capability. Table 2 relaxes machinery thresholds given predictable motion. Table 3 uses lenient bird standards to reduce false alarms from erratic flight. Tables 4, 5 and 6 provide intermediate thresholds for balloons, kites, and debris based on field trial observations. The composite score uses class coefficients: . Class multipliers (, , , , , ) reflect risk profiles. Weights remain , , across categories47. Warning threshold strategies adapt alert sensitivities to operational contexts and environmental conditions. During adverse weather reducing sensor reliability, thresholds decrease by 0.1 units to compensate for increased uncertainty in predictions. Nighttime operations similarly adjust thresholds to account for degraded vision performance. Historical false alarm rates inform dynamic threshold refinement, balancing detection sensitivity against operational disruption from excessive alerts48. The system maintains separate threshold profiles for different target categories, recognizing that drones warrant stricter criteria than birds due to their controllability and potential intentional threat. Experiments and results analysis Experimental environment and dataset The experimental platform comprises a workstation equipped with dual NVIDIA RTX 4090 GPUs (24GB memory each), Intel Xeon Platinum 8358 processor (32 cores), and 256GB DDR4 RAM to support parallel processing of multimodal data streams. The software environment utilizes PyTorch 2.0 as the deep learning framework, CUDA 12.1 for GPU acceleration, and Python 3.10 for implementation49. OpenCV 4.8 handles vision processing operations, while NumPy and SciPy facilitate numerical computations and signal processing tasks. We collected field data from 500 kV and 220 kV transmission lines across diverse conditions over six months. Twelve monitoring stations in suburban, mountainous, and agricultural regions captured synchronized radar-vision measurements. Each station used 77 GHz millimeter-wave radar (200 m range) and 4 K camera (90° FOV) mounted 25–45 m high, covering 150–200 m along corridors. Figure 4 shows deployment environments across terrain types. Field Deployment Environments Across Different Terrain Types. (a) Suburban environment, (b) Mountainous terrain, (c) Agricultural settings. Figure 4(a) shows suburban deployment near residential zones with concentrated UAV activity. Figure 4(b) depicts mountainous terrain with radar multipath and optical occlusions challenging tracking continuity. Figure 4(c) presents agricultural settings with long detection range but seasonal clutter from vegetation and harvest debris. Data collection covered clear, rainy, foggy, and dusk conditions50, recording 8–12 h daily. Table 7 summarizes the dataset composition across different dimensions. As presented in Table 7, the dataset contains 8,547 target trajectory sequences distributed across six object categories, providing substantial diversity for model training and evaluation. Table 7Dataset statistical information.Target CategoryNumber of SequencesAverage Duration (s)Scene DistributionWeather ConditionsUnmanned Aerial Vehicle2,15612.3Suburban: 45%, Rural: 55%Clear: 60%, Rain: 25%, Fog: 15%Bird3,4218.7All regionsClear: 70%, Rain: 20%, Fog: 10%Construction Machinery1,23818.5Suburban: 80%, Rural: 20%Clear: 85%, Rain: 15%Balloon68415.2Rural: 100%Clear: 90%, Rain: 10%Kite79211.4Suburban: 60%, Rural: 40%Clear: 95%, Rain: 5%Airborne Debris2566.8All regionsClear: 50%, Rain: 30%, Fog: 20% Dataset statistical information. Figure 5 illustrates the temporal distribution of collected sequences across different time periods and weather conditions, demonstrating balanced representation of operational scenarios. Temporal and weather distribution of collected trajectory sequences. As shown in Fig. 5, data collection maintained relatively uniform distribution across morning, afternoon, and evening periods, with adequate representation of adverse weather conditions necessary for robust model generalization. Data preprocessing pipelines address sensor noise and format heterogeneity. Radar measurements undergo Kalman filtering to reduce measurement noise while preserving motion dynamics. Vision frames apply Gaussian blur with 3 × 3 kernel for noise reduction, followed by contrast enhancement through histogram equalization. Temporal synchronization aligns radar and vision observations to common timestamps through linear interpolation. Spatial normalization transforms all coordinates to a unified reference frame centered on transmission tower locations. Feature normalization applies z-score standardization independently to radar and vision modalities, computing mean and standard deviation statistics from training data. Figure 6 presents comparative analysis of raw versus preprocessed data quality metrics, quantifying noise reduction and signal enhancement achieved through preprocessing operations. Comparison of data quality metrics before and after preprocessing. Figure 6 demonstrates that preprocessing substantially improves signal-to-noise ratios by 8.3dB for radar data and enhances vision clarity scores by 32%, validating the effectiveness of denoising and normalization procedures. Dataset partitioning follows a stratified split strategy preserving target category proportions and environmental diversity. Training set comprises 70% of sequences (5,983 trajectories), validation set contains 15% (1,282 trajectories), and test set includes 15% (1,282 trajectories). Sequences from the same monitoring station and time period are exclusively assigned to single partitions to prevent data leakage and ensure realistic evaluation of generalization performance. Evaluation metrics quantify model performance across detection, prediction, and assessment tasks. Precision measures the proportion of correctly identified threats among all predicted threats, while recall quantifies the fraction of actual threats successfully detected. Mean Absolute Error (MAE) evaluates trajectory prediction accuracy by computing average Euclidean distance between predicted and ground truth positions across all time steps and sequences. Threat classification performance employs macro-averaged F1 scores across all threat levels to account for class imbalance. Processing latency metrics assess real-time feasibility by measuring end-to-end inference time from sensor input to threat assessment output. Multi-target tracking metrics quantify identity consistency beyond detection accuracy. MOTA integrates false positives, false negatives, and identity switches51: MOTA = 1 - (FP + FN + IDSW) / GT. MOTP measures spatial accuracy via IoU between predicted and ground truth boxes51,52. IDSW counts incorrect label swaps during occlusions. These metrics capture tracking performance that precision/recall cannot—high detection accuracy may coexist with identity confusion degrading threat assessment utility. Model performance evaluation and comparative analysis Ablation experiments evaluate component contributions against single-modal baselines, fusion alternatives, and rule-based predictors. Table 8 compares detection accuracy, prediction precision, tracking consistency, and efficiency. The complete model achieves notable improvements over all alternatives. Table 8Performance comparison of different models.Model ConfigurationPrediction MAE (m) 3s/5sThreat Detection Precision (%)Threat Detection Recall (%)F1 Score (%)MOTA (%)MOTP (%)IDSW (per 100 frames)FPSParameter Count (M)Constant Velocity4.72 / 9.8468.462.765.452.371.28.73120Kalman Filter3.58 / 7.2174.670.272.361.876.55.42850Linear Regression3.21 / 6.8776.972.874.864.278.14.92670.3Radar-Only2.84 / 5.6776.371.874.068.579.34.21458.2Vision-Only3.21 / 6.4572.568.970.763.777.86.19825.6Early Fusion2.15 / 4.3882.779.481.075.284.62.88734.5Late Fusion2.43 / 4.9180.177.278.672.982.33.311233.8Proposed (w/o Attention)1.97 / 3.8985.282.683.978.686.42.19236.2Proposed (w/o Adaptive Weights)1.84 / 3.6487.484.886.181.388.21.79436.7 Proposed (Complete) 1.62 / 3.21 89.6 87.3 88.4 84.7 90.5 1.2 8937.1 Performance comparison of different models. Rule-based predictors (rows 1–3) achieve high FPS but suffer large errors. Constant velocity fails at 9.84 m (5s), Kalman filtering at 7.21 m, linear regression at 6.87 m—all with MOTA below 65%. Our LSTM achieves 3.21 m at 5 s (67% improvement over linear regression) and 84.7% MOTA. Single-modal baselines show fusion benefits. Radar-only lacks semantic context, yielding 4.2 identity switches per 100 frames. Vision-only suffers weather sensitivity with 77.8% MOTP53. Our model reduces MAE by 43.4% versus radar-only and achieves 90.5% MOTP through complementary fusion. Early fusion concatenates raw sensor data before feature extraction, attempting to learn joint representations directly from multimodal inputs. This approach achieves moderate improvements over single-modal methods but requires extensive training data to learn cross-modal correlations effectively. Late fusion processes each modality through separate networks and combines predictions at decision level, sacrificing intermediate feature interactions. The proposed feature-level fusion with cross-modal attention achieves 26.7% lower prediction error than early fusion and 34.6% improvement over late fusion at 5-second horizons, validating the advantages of intermediate integration with learned attention mechanisms. Ablation studies isolate the contributions of architectural components. Removing cross-modal attention mechanisms increases prediction MAE by 21.2% and reduces threat detection F1 score by 5.4% points, demonstrating that attention-based feature weighting effectively emphasizes complementary information across modalities. Eliminating adaptive weight learning degrades performance by 13.4% in prediction accuracy and 2.7% points in F1 score, confirming that dynamic modality weighting based on environmental conditions enhances robustness. Three mechanisms account for the accuracy gains observed in Table 3. The first, and most consequential, is the cross-modal attention module. By letting radar queries interrogate vision keys, the network discovers which semantic cues—object silhouette, orientation, articulated-part configuration—best disambiguate kinematic trajectories that raw Doppler alone cannot resolve. Removing this module inflated 3-second MAE from 1.62 m to 1.97 m, a 21.6% degradation that isolates the attention contribution. The second mechanism is adaptive weight learning: sigmoid-gated reliability scores suppress noisy modality channels before they contaminate the fused representation, trimming a further 13.4% from prediction error. The third is the stacked LSTM temporal encoder, whose gating cells selectively retain relevant motion history—velocity inflection points, curvature onset—while discarding sensor jitter that otherwise accumulates across autoregressive decoding steps. We adopted a multi-layered validation protocol to ensure result credibility. At the dataset level, stratified partitioning (70/15/15 train/validation/test) with site-exclusive assignment prevents information leakage between splits. At the model level, we benchmarked against nine configurations spanning rule-based, single-modal, and alternative fusion baselines (Table 8), reporting not only detection metrics but also tracking-specific indicators—MOTA, MOTP, and identity switches—that reveal failure modes invisible to precision and recall alone. At the deployment level, three operational stations running continuously for six months provided an out-of-sample stress test: the 89 genuine threats and 3.2% false-alarm rate were tallied from operator-confirmed incident logs, not from held-out test-set inference. To further quantify statistical reliability, we computed 95% bootstrap confidence intervals over 1,000 resamples of the test set; the resulting intervals for 3-second MAE [1.54, 1.71 m] and F1 score [87.1%, 89.6%] confirm that the reported gains are not artifacts of a single favorable split. Trajectory prediction accuracy varies with forecast horizons, reflecting accumulated uncertainty in multi-step predictions. Figure 7 illustrates prediction error progression across different time horizons for various model configurations. Trajectory prediction MAE Across different forecast horizons. As shown in Fig. 7, the proposed model maintains substantially lower prediction errors across all horizons from 1 to 10 s. The error growth rate, quantified through temporal derivative of MAE, demonstrates superior stability: where represents prediction MAE at forecast horizon . The proposed model exhibits an average error growth rate of 0.42 m/s compared to 0.68 m/s for early fusion, indicating better long-term prediction stability54. Threat assessment accuracy evaluation analyzes classification performance across the five-level threat hierarchy. The confusion matrix reveals that the proposed model achieves 88.4% overall accuracy with particularly strong performance distinguishing critical (Level 4) threats at 94.2% recall, minimizing dangerous false negatives. Misclassifications predominantly occur between adjacent threat levels, with negligible confusion between distant categories. The hierarchical assessment framework correctly prioritizes sensitivity for high-threat scenarios while maintaining acceptable specificity for routine monitoring situations. Robustness testing evaluates model performance degradation under challenging environmental conditions. Figure 8 presents comparative analysis across weather and illumination variations. Model performance under various environmental conditions. Figure 8 demonstrates that the proposed adaptive fusion strategy maintains relatively stable performance across conditions, with F1 score variations within 6.3% points between optimal and adverse scenarios. Single-modal baselines exhibit substantially larger performance swings, with vision-only methods degrading by 23.7% under fog conditions and radar-only approaches showing 15.4% reduction in heavy rain. The adaptive weighting mechanism automatically emphasizes radar contributions during visual occlusion and leverages vision features when radar experiences clutter, achieving graceful performance degradation: where and represent performance metrics under challenging and ideal conditions respectively. The proposed model achieves an average degradation ratio of 0.071 compared to 0.194 for early fusion55. To place the observed improvements on firmer statistical footing, we carried out additional mathematical performance analyses. First, paired-sample t-tests between the proposed model and the strongest baseline (Early Fusion) across the 1,282 test trajectories yielded t = 14.73 (p < 0.001) for 3-second MAE and t = 9.81 (p < 0.001) for composite F1 score, confirming that the gains are statistically significant rather than attributable to sampling variance. Second, we evaluated threat-level discrimination through Receiver Operating Characteristic (ROC) analysis by treating each severity level as a one-versus-rest binary classification task. The area under the ROC curve (AUC) reached 0.964 for Critical threats, 0.938 for High, 0.912 for Moderate, 0.887 for Low, and 0.951 for Negligible, indicating strong separability across the full five-level hierarchy. Third, we examined prediction error distributions: Kolmogorov–Smirnov tests rejected the normality assumption (p < 0.01), so we report median absolute errors alongside means—median 3-second MAE is 1.38 m versus the mean of 1.62 m, revealing a right-skewed distribution in which a small fraction of abrupt-maneuver cases inflate the mean. These analyses collectively strengthen confidence in the reported results and clarify the conditions under which the model performs best. The model processes data at 89 FPS, exceeding the 20 Hz requirement by 4.45×. Training required 72 GPU-hours on dual RTX 4090 cards (~$108 at cloud rates). Peak memory: 19.3 GB. Inference options vary by performance-cost trade-offs. For edge deployment, NVIDIA Jetson AGX Orin provides optimal balance: 62 FPS, 45 W consumption, $2,170 per station including enclosure. Our 12-station deployment cost $26,040 ($3,255/km). Lower-cost alternatives: Jetson Nano ($99) achieves 18 FPS with INT8 quantization56; Raspberry Pi with Coral TPU ($115) reaches only 12 FPS. Budget options suit non-critical applications. Datacenter A100 GPUs deliver 340 FPS but cost $15,000 with 400 W consumption. Latency breakdown: feature extraction 63% (7.1ms), fusion 22% (2.5ms), prediction 15% (1.7ms), assessment 8% (0.9ms). GPU provides 8.7× speedup over CPU (11.2ms vs. 97ms per frame). Inference memory: 3.2 GB. For 100 km corridor deployment (125 stations at 800 m intervals): hardware $271,250, annual operations $65,913 (power, data, maintenance). Outage costs average $28,000 per incident. Our system prevented ~ 12 outages over six months, yielding $672,000 avoided losses. Net year-one benefit: $334,837; payback period: 4.8 months. This assumes 50% of detected threats would have caused outages without intervention. Infrastructure requirements extend beyond computing hardware to include networking, power distribution, and physical mounting systems. Each monitoring station demands 100Mbps Ethernet backhaul or 4G LTE connectivity to stream sensor data and receive software updates, adding $400 per site for networking equipment. Power-over-Ethernet switches consolidate station wiring, costing $180 per eight-port unit. Tower mounting hardware (brackets, guy wires, grounding) adds $310 per installation, plus $650 per site for professional tower climbing services during installation and maintenance. Total installed cost per monitoring station thus escalates from $2,170 equipment-only to $3,710 fully deployed, raising 100 km corridor investment from $271,250 to $463,750—still economically justified given avoided outage values, but emphasizing that comprehensive cost accounting must include installation labor and infrastructure beyond computational hardware. Model compression enables budget deployment. INT8 quantization56 shrinks model from 148 MB to 39 MB with 2.1× speedup; accuracy drops modestly (MAE: 1.62 m→1.89 m, F1: 88.4%→85.7%). Knowledge distillation57,58 creates compact 8.3 M-parameter student achieving 2.34 m MAE and 86.1% F1 with 78% memory reduction. These enable $99 edge deployment for budget-constrained applications. Practical application case analysis Field deployment of the proposed model across three operational transmission corridor monitoring sites demonstrates effective real-world performance under diverse operational scenarios. Figure 9 presents detailed visualization results from four representative cases involving different target categories, environmental conditions, and threat levels, with each panel annotated to clarify detection outputs, predicted trajectories, and risk assessments. Figure 9 shows four representative cases with trajectory predictions and threat classifications. Visualization of Trajectory Prediction and Threat Assessment in Real Scenarios. (a) Case A-001: UAV detection, (b) Case B-012: Mobile crane, (c) Multi-target bird tracking, (d) Case A-078: Construction truck and balloon. Panel (a): UAV detected 187 m from tower. Blue dashed line shows 3-second history; red arrow projects 5-second path approaching within 18.3 m of conductors. Level 3 (High) threat, 8.4-second warning time. This case exemplifies typical UAV intrusion patterns documented in critical infrastructure protection studies59. Panel (b): Mobile crane near 220 kV line. Predicted clearance: 12.7 m. Level 4 (Critical) assigned despite slow 3.2 m/s velocity due to catastrophic collision potential. Warning time: 22.6 s. Panel (c): Two bird flocks (purple, cyan) near lines but diverging. Both classified Level 1 (Low). Tracker maintains separate IDs when flocks pass within 15 m at t = 2.3s. Panel (d): Construction truck (41.2 m, Level 1) and balloon (8.4 m predicted approach, Level 4) tracked simultaneously. Adaptive fusion weights radar at 0.73 during dust haze conditions. Table 9 quantifies performance metrics for these representative cases and additional field deployments, providing numerical validation of the qualitative observations from Fig. 7. The Detection Distance column reports range at which the system first identified each target, while Predicted Min. Distance indicates the forecasted closest approach to conductors, and Warning Lead Time measures the interval between initial detection and predicted safety zone breach. These metrics collectively demonstrate the practical operational value of early threat warnings. Table 9Typical case threat assessment results.Case IDTarget TypeDetection Distance (m)Predicted Min. Distance (m)Assessed Threat LevelWarning Lead Time (s)A-001UAV18718.3Level 3 (High)8.4B-012Mobile Crane15612.7Level 4 (Critical)22.6C-045Excavator13435.8Level 2 (Moderate)15.3A-078Balloon2038.4Level 4 (Critical)31.2B-033Construction Truck9841.2Level 1 (Low)12.8 Typical case threat assessment results. Table 9 lists representative cases demonstrating the model’s ability to provide early warnings with sufficient lead times across diverse threat scenarios. The average warning lead time of 18.06 s substantially exceeds the minimum 5-second requirement for effective operator response60. Figure 10 illustrates a complex multi-target scenario at Station B where simultaneous monitoring of construction machinery and avian targets challenges the system’s discrimination capabilities, demonstrating robust performance under challenging conditions including partial occlusions and sensor degradation. Field deployment in complex multi-target monitoring scenario. Figure 10 shows Station B deployment tracking four targets: tower crane (orange), excavator (yellow), two bird flocks (green). Left panel: 60 m crane boom near 500 kV lines (45 m away), excavator at 38 m amid dust clouds reducing visibility 35%. Right panel: tracking interface maintaining localization despite excavator occlusion (t = 12.3s, 3.2s duration). Top plot: threat evolution over 30 s. Crane rises from Level 1 to Level 3 as clearance drops below 18 m. Excavator maintains Level 2 (stationary). Birds fluctuate between Level 0–1. Bottom plot: adaptive fusion weights. Vision dominates (0.65) during clear conditions; radar increases to 0.78 when dust intensifies, maintaining < 2.1 m position error through occlusion. Multi-target interaction scenarios test the model’s ability to maintain identity consistency and accurate prediction under dynamic occlusions. In Case B-067, temporary occlusion of a construction vehicle behind transmission tower structures reduced vision confidence to 0.23, triggering adaptive weight rebalancing that increased radar contribution from 0.48 to 0.78. The tracking continuity was maintained throughout the 3.2-second occlusion period with position error remaining below 2.1 m, demonstrating robust sensor fusion. Warning timeliness quantification employs lead time analysis, measuring the interval between initial threat detection and predicted safety zone breach. The model achieves an average lead time of: where represents warning lead time, denotes detection distance, m indicates safety zone radius, and represents approach velocity. Across 347 validated threat cases, the model provides mean lead time of 17.8 s with standard deviation of 6.3 s, meeting operational requirements for manual intervention61. Deployment efficiency analysis reveals that the optimized model operates on edge computing platforms with NVIDIA Jetson AGX Orin modules, consuming 45 watts power while maintaining 62 FPS processing rate. The system uptime across three monitoring stations averages 99.4% over six-month operational period, with false alarm rate stabilizing at 3.2% after threshold adaptation. Processing latency distribution follows: where represents processing latency, denotes time threshold, and ms represents mean latency (exponential distribution). The 95th percentile latency of 18.6 milliseconds ensures consistent real-time performance. Practical application value assessment incorporates operational feedback from three power grid companies. The system detected 89 genuine threat incidents over six months that would have remained undetected under previous manual inspection regimes, preventing an estimated 12 potential safety incidents based on trajectory analysis. Operator workload reduction of 67% resulted from automated monitoring replacing continuous manual surveillance, while detection coverage increased from 18% to 94% of corridor length through sensor deployment optimization. Economic analysis indicates system deployment costs are recovered within 18 months through avoided outage losses and reduced personnel requirements, validating substantial operational benefits for power transmission infrastructure protection. Discussion What accounts for the performance gains reported in Section IV? The answer is not that combining two data sources is generically better than one—naive concatenation (early fusion) also combines both modalities yet trails our method by 7.4 F1 points. Rather, the gains stem from a conditional routing of information. Radar anchors the spatial state of each target with range-rate measurements that remain stable under rain, dust, and darkness, while vision contributes texture, silhouette, and color cues essential for distinguishing, for instance, a hovering drone from a similarly sized bird at comparable range. The sigmoid-gated fusion learns when to trust which channel, re-weighting modalities on a per-frame basis. This conditional behavior is most visible during fog-to-rain transitions, where vision confidence drops abruptly and the gate must shift reliance within two to three frames to prevent trajectory drift. A fixed-weight scheme cannot react to such transient degradation, which explains why our degradation ratio under adverse conditions (0.071) is roughly one-third that of early fusion (0.194). Inspecting cross-modal attention weights across the test set yields a finding that, in our view, carries implications beyond the immediate application. When a target accelerates or executes a sharp turn, the attention module assigns higher weight to radar velocity features—Doppler shifts encode instantaneous kinematics more faithfully than optical flow estimated from consecutive frames. During slow, multi-target scenes, such as two bird flocks crossing paths near a tower, the module shifts toward vision, exploiting appearance differences to maintain identity assignment. What strikes us as noteworthy is that this context-dependent routing emerges entirely from data-driven training with no hand-coded rules specifying which modality to trust under which condition. The network has, in effect, learned an implicit model of sensor trustworthiness conditioned on scene dynamics. This observation suggests a broader principle for heterogeneous sensor fusion: rather than prescribing fixed fusion rules based on engineering intuition, allowing the attention mechanism to discover modality preferences from data can yield more robust and generalizable routing policies. Whether this finding transfers to other sensor combinations—say, thermal-infrared paired with LiDAR—remains an open question, but the attention weight distributions we observe provide an empirical starting point for such investigations. Category-wise analysis provides additional insight. UAV trajectories were predicted most accurately (1.41 m MAE at 3 s), likely because drones follow relatively smooth, pilot-controlled paths. Airborne debris, by contrast, exhibited the highest errors (2.37 m), reflecting wind-driven chaotic drift that the LSTM struggles to anticipate beyond two seconds. Construction machinery fell in between (1.58 m), benefiting from terrain-constrained motion yet occasionally confounded by articulated boom movements that resemble rapid approach toward conductors. These differences underscore the value of category-specific threat thresholds (Tables 1, 2, 3, 4, 5 and 6): applying uniform thresholds across all target types would either over-alert on birds or under-alert on drones, neither of which is acceptable in practice. Field deployment revealed key operational considerations: Calibration Multimodal fusion requires millimeter-level spatial alignment. Our semi-automated protocol using corner reflectors takes 2–3 h per station. Calibration drift averages 1.2 cm/month at windy sites, degrading accuracy 8%. We recommend quarterly recalibration, monthly for exposed locations. Maintenance Camera lenses accumulate debris causing 15–25% contrast reduction within two weeks. Heated enclosures eliminate 90% of condensation outages. Radar radomes require ice/snow removal—5 mm accumulation reduced range 22%. Lightning damaged two cameras; 120 km/h winds damaged one antenna. Environmental limitations Coastal deployment showed 15% false alarms from sea clutter until site-specific maps were implemented. Dense fog (< 50 m visibility) degraded vision 73%. EMI near substations required frequency selection and shielding. The framework generalizes within temperate environments; maritime/arctic/desert deployment needs dataset augmentation. Power (280 W/station) and bandwidth (4.2 Mbps) requirements constrain remote installations. System reliability Software exceptions occurred every ~ 48 h initially; watchdog timers achieving 15-second recovery enabled 99.4% uptime. Sensor failures required redundant installations. Network outages (23 incidents, 15–90 min each) were handled via backup cellular channels for local alerts. Practical deployment requires fault management, redundancy, and fallback communications. Trajectory prediction errors originate from multiple sources including measurement noise, modeling assumptions, and inherent motion unpredictability. Quantization errors in sensor measurements introduce position uncertainty that accumulates through recursive prediction. The LSTM architecture assumes smooth, continuous motion patterns and struggles with abrupt maneuvers or erratic target behaviors that violate learned dynamics. Prediction uncertainty grows quadratically with forecast horizon as compound errors accumulate through autoregressive generation. Future improvements could incorporate uncertainty quantification through probabilistic prediction frameworks, enabling explicit representation of forecast confidence that adapts to motion complexity. The hierarchical threat assessment framework delivers reasonable risk stratification in practice, but its threshold-based architecture brings trade-offs that deserve candid discussion. Fixed weight coefficients (, , ) were calibrated on our field data and may not transfer optimally to transmission lines with different voltage classes, span lengths, or terrain profiles. Distance-based metrics assume a uniform 30 m safety buffer, yet critical components—mid-span conductor sag points, insulator strings, damper locations—may warrant larger protective zones than standard tower segments. A deeper limitation concerns what the threshold model cannot capture. Because the sub-scores combine linearly, the framework misses potential nonlinear interactions—for example, a slow-moving target on a direct collision course may pose greater risk than a fast target on a tangential path, yet linear weighting cannot fully distinguish these scenarios without manual threshold tuning. A learned threat classifier could, in theory, discover such interactions from data. As reported in Sect. 3.3, our preliminary comparison with a learned MLP classifier confirmed that the threshold design achieves substantially higher recall on rare high-severity events, which we consider non-negotiable in safety-critical infrastructure monitoring. The underlying cause—severe class imbalance in high-severity samples—is structural rather than incidental, and would require dedicated mitigation strategies (focal loss, cost-sensitive training, or synthetic augmentation) before a learned scorer could match threshold-based recall at the critical end of the severity spectrum. We view a hybrid architecture, in which a small neural network predicts residual corrections to the analytical sub-scores while preserving the interpretable threshold mapping for final severity assignment, as a promising path forward. We elaborate on this prospect in Section VI. The model generalizes reasonably well across the three deployment locations, which share similar geographic and climatic characteristics—a result we attribute to the stratified training protocol and the adaptive fusion mechanism’s ability to compensate for site-specific sensor variability. However, preliminary coastal trials exposed an 18% performance degradation from sea clutter and salt-spray lens fouling, conditions absent from our inland training data. This sensitivity to distribution shift underscores a practical constraint: successful deployment at a new site presupposes either that the site falls within the training distribution or that site-specific fine-tuning data are collected beforehand. Additional operational prerequisites include careful sensor calibration, continuous power and network connectivity, and computational hardware at or above the Jetson AGX Orin class. Optimization directions for future research include expanding training data diversity to improve generalization, incorporating physics-based motion models to constrain predictions, implementing online adaptation mechanisms for dynamic threshold adjustment, and developing lightweight model variants for resource-constrained deployment. Enhanced uncertainty quantification would enable risk-aware decision making, while integration of additional sensing modalities such as infrared or lidar could further improve robustness. Advanced data association algorithms could reduce identity switches in crowded scenarios, and hierarchical attention mechanisms might better capture multi-scale spatial-temporal patterns. These enhancements would strengthen the model’s applicability across broader operational contexts while maintaining real-time performance requirements. The following paragraphs address two dimensions—security and reliability—that are essential for any operational infrastructure monitoring system but that we had not fully elaborated in the initial manuscript. Regarding system security, three layers of protection are implemented. At the communication layer, all sensor-to-edge and edge-to-control-center data streams are encrypted with TLS 1.3, and each monitoring station authenticates via device-specific X.509 certificates to prevent spoofing. At the data-integrity layer, every transmitted frame carries a SHA-256 hash that the receiving node verifies before ingestion; corrupted or tampered frames are discarded and flagged for operator review. At the adversarial-robustness layer, we conducted preliminary testing with gradient-based perturbations (FGSM with ε = 0.03) applied to vision inputs. The adaptive fusion mechanism partially mitigated these attacks by automatically reducing vision weight when perturbed frames triggered anomalously low confidence scores, limiting F1 degradation to 4.2%—roughly half the degradation observed in the fixed-weight early-fusion baseline. We acknowledge, however, that dedicated adversarial defenses (e.g., adversarial training, input purification) have not yet been integrated and remain a priority for future hardening. Regarding system reliability, our six-month deployment revealed several concrete findings. Mean time between software exceptions was initially 48 h; after deploying watchdog timers with 15-second automatic recovery cycles, effective uptime reached 99.4%. Hardware redundancy proved essential—two of the twelve stations experienced camera failures due to lightning, and one lost a radar antenna to sustained 120 km/h winds. In all three cases, the surviving sensor continued to operate in single-modal fallback mode, producing degraded but non-zero situational awareness until maintenance crews replaced the failed component (mean repair time: 36 h). Network connectivity, another reliability concern, suffered 23 outages averaging 42 min each; backup 4G LTE channels ensured that local threat alerts reached operators within the contractual 30-second latency even during primary link failures. Overall, we view the 99.4% uptime as adequate for supplementary monitoring but insufficient for safety-critical standalone operation; achieving the latter would require N + 1 sensor redundancy at every station and dual independent processing pipelines, both of which add cost that must be weighed against the value of the protected asset. Conclusion This study addressed the problem of detecting, predicting, and assessing dynamic threats—UAVs, construction machinery, birds, balloons, kites, and airborne debris—in high-voltage power transmission corridors, where conventional single-sensor monitoring left more than 80% of corridor length effectively unobserved. We developed a radar-vision multimodal fusion framework whose contribution resides not in inventing fundamentally new algorithmic primitives but in integrating, adapting, and rigorously validating established techniques into a cohesive pipeline purpose-built for the corridor protection domain. Three elements define this integration. First, a reliability-aware adaptive fusion strategy that learns per-frame modality weights from sigmoid-gated sensor quality indicators, shifting reliance toward the more trustworthy channel under fog, rain, or darkness instead of averaging degraded inputs. Second, a single-head cross-modal attention module in which radar kinematic queries interrogate vision semantic keys, allowing the network to discover complementary feature correspondences without hand-crafted alignment rules. Third, a hierarchical, category-specific threat assessment scheme—deliberately kept threshold-based for interpretability and on-site updatability—that combines distance, velocity, and trajectory-trend sub-scores with target-class multipliers and maps the resulting composite value onto five severity levels through field-calibrated thresholds. Experimental evaluation on 8,547 field-collected trajectory sequences demonstrated that the proposed model achieved 1.62 m MAE at 3-second prediction horizons and 3.21 m at 5 s, representing a 43% relative reduction over the radar-only baseline. Threat classification reached 89.6% precision, 87.3% recall, and 88.4% macro-averaged F1 across five levels, surpassing early-fusion and late-fusion alternatives by 7.4 and 9.8% points respectively. Six-month deployment at three operational 500 kV / 220 kV monitoring sites confirmed real-world viability: the system detected 89 operator-confirmed threat incidents with 17.8-second mean warning lead time and 3.2% false-alarm rate while maintaining 99.4% uptime on edge hardware consuming 45 W per station. Operator workload, measured as person-hours of continuous manual surveillance, fell by 67%, and instrumented detection coverage expanded from 18% to 94% of corridor length. Among the three integrated components, adaptive environment-dependent modality weighting proved most consequential in practice: it kept the degradation ratio under adverse conditions at 0.071, roughly one-third that of fixed-weight early fusion (0.194). The cross-modal attention mechanism automatically routed information according to context—prioritizing radar during rapid maneuvers and vision during multi-target disambiguation—without explicit supervisory signals. The threshold-based hierarchical threat framework, while not a learned model, provided the graduated and interpretable situational awareness that field operators required for proportionate response, addressing a practical gap left by prior binary-classification approaches. Several limitations deserve candid acknowledgment. The most pressing is simultaneous dual-modality degradation: when dense fog (< 50 m visibility) coincides with heavy rain, both radar clutter and camera obscuration intensify at the same time, leaving the adaptive fusion mechanism with no reliable anchor. In our test set, such overlapping extreme conditions occurred in only 3% of sequences, but they accounted for 28% of all misclassified threats. A second limitation is distributional sensitivity—coastal deployment trials revealed 18% performance degradation from sea clutter and salt-spray lens fouling, conditions absent from our inland training data. Third, the 37.1 M-parameter model requires at least a Jetson AGX Orin-class GPU for real-time inference; sub-$100 platforms like Jetson Nano reach only 18 FPS after INT8 quantization, with a measurable accuracy penalty (MAE rising from 1.62 m to 1.89 m). Finally, the threat assessment weights (w_D, w_V, w_T) and class multipliers (γ_class) are currently fixed; different transmission-line configurations—a 1000 kV ultra-high-voltage line versus a 110 kV distribution feeder—may justify very different risk trade-offs that static coefficients cannot capture. Looking ahead, we identify five concrete research directions, each paired with a specific technical approach. First, to address dual-modality degradation, we plan to integrate a thermal infrared channel as a third sensing modality; because mid-wave infrared penetrates fog more effectively than visible light and is unaffected by radar clutter, it can serve as a tiebreaker when both primary modalities weaken. The immediate next step is to deploy FLIR Boson 640 cameras at two existing stations and retrain the fusion network with a three-branch attention module. Second, to improve generalization across climates, we intend to adopt domain-adaptation techniques—specifically, adversarial feature alignment between inland and coastal feature distributions—so that models trained on one geography transfer to another with minimal site-specific fine-tuning. Third, to make the threat assessment weights context-sensitive rather than fixed, we will explore a meta-learning formulation in which line-specific parameters (voltage class, span length, minimum ground clearance) condition a small hypernetwork that outputs optimal , , , and for each installation. Fourth, we plan to investigate a hybrid learned-plus-threshold threat scoring architecture, in which a small neural network predicts residual corrections to the analytical sub-scores while preserving the interpretable threshold mapping for final severity assignment. Such a design could capture nonlinear interactions among distance, velocity, and trajectory trend that fixed linear weights miss, without sacrificing the auditability that power-grid operators require. Addressing the class imbalance inherent in high-severity events—through focal loss, cost-sensitive training, or synthetic data augmentation—will be a necessary component of this effort. Fifth, to enable deployment on ultra-low-cost hardware, we plan to apply structured pruning combined with knowledge distillation from the full 37.1 M-parameter teacher to a compact 4 M-parameter student network, targeting 30 + FPS on Raspberry Pi 5 with Coral TPU while maintaining MAE below 2.0 m. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Acknowledgements This work was supported by the Incubation Project of State Grid Jiangsu Electric Power Co., Ltd., titled "Spatio-temporal Multi-Target Perception and Reasoning Based on Multimodal Data" (Project No. JF2025013). Abbreviations Unmanned Aerial Vehicle Long Short-Term Memory Constant False Alarm Rate You Only Look Once Region Proposal Network Gray Level Co-occurrence Matrix Local Binary Patterns Intersection over Union Mean Absolute Error Frames Per Second Graphics Processing Unit Central Processing Unit Residual Network Rectified Linear Unit Signal-to-Noise Ratio Author contributions Jun Zhang: Conceptualization, methodology development, software implementation, data analysis, manuscript writing - original draft. Zhiwei Zhang : Data collection, radar system configuration, experimental validation, manuscript review. Xiao Tan : Vision system development, feature extraction algorithm design, data preprocessing, manuscript review. Jin Ling : Model architecture design, neural network implementation, performance evaluation, manuscript review. Jieqing Deng : Project supervision, research direction, methodology guidance, manuscript writing - review and editing, funding acquisition (if applicable), corresponding author responsibilities.All authors have read and approved the final manuscript. Data availability All data generated and analyzed during this study are included in Supplementary File 1, which contains the field-collected dataset with synchronized radar and vision measurements from 12 monitoring stations, model architecture specifications, training configurations, and evaluation scripts. Access to complete raw sensor data may require data sharing agreements with collaborating power grid companies due to infrastructure security considerations. Requests for additional materials should be directed to the corresponding author (Jieqing Deng, deng123450427@163.com). Declarations Competing interests The authors declare no competing interests. Ethics approval and consent to participate This study involved field data collection from power transmission corridor monitoring systems. All data collection activities were conducted with appropriate authorization from the respective power grid companies. No human subjects were directly involved in this research. The study complied with all relevant institutional and national guidelines for infrastructure monitoring research. Consent for publication All authors have reviewed the manuscript and consent to its publication. All data presented have been appropriately anonymized and do not contain identifiable information. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1 Data Availability Statement All data generated and analyzed during this study are included in Supplementary File 1, which contains the field-collected dataset with synchronized radar and vision measurements from 12 monitoring stations, model architecture specifications, training configurations, and evaluation scripts. Access to complete raw sensor data may require data sharing agreements with collaborating power grid companies due to infrastructure security considerations. Requests for additional materials should be directed to the corresponding author (Jieqing Deng, deng123450427@163.com).