T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking Journal: Computer Vision and Image Understanding
Hojat Ardi, Amir Jahanshahi, Ali Diba · 2025
Read T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking Journal: Computer Vision and Image Understanding by Hojat Ardi, Amir Jahanshahi, Ali Diba (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-12913.txt.
Introduction
Hojat Ardi, Amir Jahanshahi, and Ali Diba present T-SiamTPN in a 2025 paper on tracking objects from drones. Aerial tracking is hard because of scale variation, dynamic backgrounds, clutter, and frequent occlusions. Most trackers emphasize spatial cues and overlook how appearance changes over time, limiting robustness during long tracking or occlusion.
Their system extends a Siamese transformer pyramid architecture with explicit temporal modeling. Temporal feature fusion and attention-based interactions strengthen consistency over time and produce richer feature representations. Link-based Siamese trackers are limited by the linear nature of link, which struggles with complex non-linear appearance changes; the new design addresses that weakness. Despite added temporal modules, the tracker preserves computational efficiency while achieving results competitive with leading trackers, the authors report.
Catalog record
| CREATOR | Hojat Ardi, Amir Jahanshahi, Ali Diba |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-12913
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking Journal: Computer Vision and Image Understanding Hojat Ardi, Amir Jahanshahi, Ali Diba Abstract Aerial object tracking remains a challenging task due to scale variations, dynamic backgrounds, clutter, and frequent occlusions. While most existing trackers emphasize spatial cues, they often overlook temporal dependencies, resulting in limited robustness in long-term tracking and under occlusion. Furthermore, correlation-based Siamese trackers are inherently constrained by the linear nature of correlation operations, making them ineffective against complex, non-linear appearance changes. To address these limitations, we introduce T-SiamTPN, a temporal-aware Siamese tracking framework that extends the SiamTPN architecture with explicit temporal modeling. Our approach incorporates temporal feature fusion and attention-based interactions, strengthening temporal consistency and enabling richer feature representations. These enhancements yield significant improvements over the baseline and achieve performance competitive with state-of-the-art trackers. Crucially, despite the added temporal modules, T-SiamTPN preserves computational efficiency. Deployed on the resource-constrained Jetson Nano, the tracker runs in real time at 7.1 FPS, demonstrating its suitability for real-world embedded applications without notable runtime overhead. Experimental results highlight substantial gains: compared to the baseline, T-SiamTPN improves success rate by 13.7% and precision by 14.7%. These findings underscore the importance of temporal modeling in Siamese tracking frameworks and establish T-SiamTPN as a strong and efficient solution for aerial object tracking. Code is available at: https://github.com/to/be/released 1 Introduction Visual object tracking is a fundamental and active research area within the field of computer vision, aiming to estimate and continuously update the position of one or more objects of interest as they move through a video sequence. This task involves not only detecting objects in individual frames but also maintaining their identities over time, even in the presence of challenges such as occlusion, scale variation, illumination changes, fast motion, and background clutter. Due to its broad utility, visual tracking has become a critical component in numerous real-world applications. For instance, in autonomous driving systems, robust tracking is essential for understanding the dynamics of surrounding vehicles and pedestrians. In video surveillance, tracking supports abnormal activity detection and real-time monitoring. Other key application domains include augmented and virtual reality, human-computer interaction, robotics, and military systems, where accurate tracking contributes to situational awareness and decision-making [ 1 ] . Tracking algorithms are generally categorized into two principal types: Single-Object Tracking (SOT) and Multiple-Object Tracking (MOT). In the SOT paradigm, the tracker is given an initial bounding box of the target in the first frame and is required to follow that specific target across subsequent frames, without relying on pre-trained object detectors or labeled training data [ 2 ] . This scenario is often used in real-time applications where immediate tracking is needed after manual initialization. In contrast, MOT is a more complex task that involves detecting and tracking multiple targets simultaneously throughout a video sequence [ 3 ] . It requires solving both the detection problem and the data association problem — ensuring that each detection is correctly assigned to an existing trajectory or initialized as a new one. Figure 1: Comparison of T-SiamTPN with state-of-the-art aerial trackers in terms of precision and speed. The proposed model (red star) achieves a well-balanced trade-off between performance and efficiency, attaining a high average precision of 83.8% and a processing speed of 7.1 FPS. UAV-based aerial tracking has gained significant attention due to its wide field of view and high flexibility, particularly in applications like surveillance and aerial imaging. However, tracking in aerial videos faces numerous challenges, including small target sizes, fast motions, perspective changes, occlusions, cluttered backgrounds, objects resembling the target, and computational limitations in embedded systems. Since UAVs typically rely on lightweight, low-power hardware, they struggle to execute computationally intensive algorithms. This creates a balancing act between tracking performance, processing speed, and energy efficiency, especially in real-time scenarios. To address all these issues, several visual tracking algorithms have been developed, and most are feature-based and deep learning-based [ 4 ] . Traditional feature-based algorithms such as [ 5 , 6 , 7 , 8 , 9 ] are preprogrammed-rule-based and fail to perform in cases with occl, usions, background complexity, or motion with a high speed. Deep learning-based algorithms, i.e., convolutional neural networks (CNN) and Siamese networks [ 10 , 11 , 12 , 13 , 14 , 15 ] , have made a remarkable breakthrough in terms of reliability and performance. One recent tendency has been to use transformer [ 16 ] models for tracking. CNN is well-suited for localized feature extraction and thus for fine detail and small variation in target detection. Transformers with their attention-based architecture capture interdependencies between different parts in an image and are well-suited for dealing with problems like camera angle variation and scale variation. Hybrid networks inherit advantages from both architectures and thus are best suited for embedded systems and real-world tasks like UAV tracking, where performance and speed are crucial with limited resources. One successful model in this domain is the Siamese Transformer Pyramid Network (SiamTPN), which combines Siamese architecture with a transformer pyramid network (TPN). This model enhances feature representation and improves tracking performance compared to earlier approaches. However, it still faces certain limitations: it does not utilize temporal information, which reduces the model’s robustness in occlusion scenarios and sudden target appearance changes, especially in long-term tracking. Its reliance on cross-correlation performs poorly under severe and complex appearance variations and occlusions, and it delivers suboptimal performance in scenarios with cluttered backgrounds and similar objects. Additionally, it struggles with tracking and precisely localizing small objects. In this work, we introduce a novel model called T-SiamTPN , which addresses the limitations of SiamTPN while maintaining computational efficiency and enhancing tracking performance. Large-scale experiments on six aerial tracking benchmarks, as illustrated in Figure 1 , demonstrate that T-SiamTPN achieves a favorable balance between speed and accuracy. Unlike existing temporal models that suffer from significant speed degradation and increased complexity as the number of templates grows, our method introduces a lightweight and intelligent temporal mechanism that effectively captures inter-frame dependencies with minimal computational overhead. The proposed architecture incorporates one static template and multiple dynamic templates, where dynamic templates are updated selectively and only when the tracker exhibits high confidence. This strategy helps prevent noise accumulation and mitigates model drift due to erroneous updates. Our model achieves several key improvements: 1. It leverages temporal information across frames to enhance robustness against occlusion and improve stability in long-term tracking. In addition, a smart template update mechanism mitigates drift caused by abrupt target appearance changes or background noise. 2. It replaces conventional cross-correlation with a multi-scale transformer attention mechanism, enhancing the model’s ability to distinguish targets in cluttered or complex backgrounds. 3. It integrates a channel attention mechanism into the Pooling Attention (PA) block of the base model and introduces a novel Modulated Pooling Attention (MPA) block. These additions improve feature refinement, resulting in better performance and efficiency, particularly when tracking small objects. 2 Related Works Since the advent of deep networks, visual object tracking using deep learning, especially with Siamese networks, has seen remarkable advancements since 2016 [ 10 ] . More recently, transformers have been introduced in the fields of computer vision and object tracking, drawing on their success in natural language processing