High-Frequency Semantics and Geometric Priors for End-to-End Detection Transformers in Challenging UAV Imagery
Hongxing Peng, Lide Chen, Hui Zhu, Yan Chen · 2025
Read High-Frequency Semantics and Geometric Priors for End-to-End Detection Transformers in Challenging UAV Imagery by Hongxing Peng, Lide Chen, Hui Zhu, Yan Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00825.txt.
Introduction
Hongxing Peng and co-authors sharpen drone image detection for small, crowded objects, in work from 2025. Aerial photos pack tiny targets into cluttered scenes, often overlapping. Classic detectors lean on preset anchors and hand rules that bottleneck in such density.
Their end-to-end transformer adds three targeted upgrades. One backbone keeps fine detail alongside strong meaning signals. A lean pyramid merges high-resolution features to rescue small objects. Shape priors guide the model toward plausible object geometry. Together the parts attack the known aerial failure modes directly. The design runs in real time, the authors report. Tests target hard drone imagery benchmarks.
Catalog record
| CREATOR | Hongxing Peng, Lide Chen, Hui Zhu, Yan Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-00825
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High-Frequency Semantics and Geometric Priors for End-to-End Detection Transformers in Challenging UAV Imagery Hongxing Peng, Lide Chen, Hui Zhu, Yan Chen Abstract Unmanned Aerial Vehicle-based Object Detection (UAV-OD) faces substantial challenges, including small target sizes, high-density distributions, and cluttered backgrounds in UAV imagery. Current algorithms often depend on hand-crafted components like anchor boxes, which demand fine-tuning and exhibit limited generalization, and Non-Maximum Suppression (NMS), which is threshold-sensitive and prone to misclassifying dense objects. These generic architectures thus struggle to adapt to aerial imaging characteristics, resulting in performance limitations. Moreover, emerging end-to-end frameworks have yet to effectively mitigate these aerial-specific challenges.To address these issues, we propose HEGS-DETR, a comprehensively enhanced, real-time Detection Transformer framework tailored for UAVs. First, we introduce the High-Frequency Enhanced Semantics Network (HFESNet) as a novel backbone. HFESNet preserves critical high-frequency spatial details to extract robust semantic features, thereby improving discriminative capability for small and occluded targets in complex backgrounds. Second, our Efficient Small Object Pyramid (ESOP) strategy strategically fuses high-resolution feature maps with minimal computational overhead, significantly boosting small object detection. Finally, the proposed Selective Query Recollection (SQR) and Geometry-Aware Positional Encoding (GAPE) modules enhance the detector’s decoder stability and localization accuracy, effectively optimizing bounding boxes and providing explicit spatial priors for dense scenes. Experiments on the VisDrone dataset demonstrate that HEGS-DETR achieves a 5.1% AP 50 and 3.8% AP increase over the baseline, while maintaining real-time speed and reducing parameter count by 4M. [ orcid=0000-0002-1872-8855, style=chinese] [ style=chinese, orcid=0009-0009-3953-1689] [style=chinese] [style=chinese] keywords Unmanned Aerial Vehicle,Small object detection ,Detection Transformer ,Real-time † † corresponding: Corresponding author 1 Introduction Real-time object detection on Unmanned Aerial Vehicles (UAVs) has emerged as a cornerstone technology, underpinning a multitude of critical applications ranging from precision agriculture [ 30 ] and highway surveillance [ 34 ] to urban environmental assessment [ 9 ] . However, the unique vantage point of aerial imagery presents a confluence of complex challenges that push the performance of existing detection models to their limits. Models must contend with high-density scenes populated with numerous small objects and frequent occlusions caused by cluttered backgrounds, all while operating under the stringent constraints of limited on-board computational resources and the demand for real-time processing [ 26 ] . Figure 1: Challenges in UAV Object Detection: dense small objects and cluttered backgrounds Prevailing UAV object detection models [ 32 , 36 , 8 , 19 ] rely on predefined anchor boxes and Non-Maximum Suppression (NMS), which introduce inherent limitations. Anchor box parameters require meticulous tuning for specific datasets and lack generalization capability. Meanwhile, NMS is sensitive to its threshold, potentially leading to the erroneous suppression of highly overlapping or adjacent objects or thEe failure to remove redundant detections. These issues create significant processing bottlenecks, particularly in the dense-object scenarios common in UAV footage. The advent of the Detection Transformer (DETR) [ 2 ] marked a paradigm shift towards truly end-to-end, NMS-free detection, though its substantial computational overhead initially rendered it impractical for real-time applications. A major breakthrough arrived with RT-DETR [ 33 ] , the first end-to-end framework to surpass state-of-the-art YOLO models under real-time conditions. Its advantage is credited to an innovative hybrid encoder design that decouples intra-scale feature interaction from cross-scale feature fusion. Despite RT-DETR’s impressive performance on general-purpose benchmarks such as COCO [ 15 ] , its architecture is not inherently optimized for the distinct challenges of aerial imagery—namely, small target sizes, high object density, and frequent occlusions, as illustrated in fig. 1 . Its feature extractor and decoder were not purpose-built to address these core problems endemic to the aerial domain. Consequently, while RT-DETR provides a powerful baseline, unlocking its full potential for aerial object detection necessitates a targeted redesign of its core components. To address these challenges, we propose a novel framework built upon RT-DETR that integrates several synergistic innovations. First, to remedy the inadequate feature representation of standard networks for complex aerial scenes, we replace the conventional backbone with a High-Frequency Enhanced Semantics Network (HFESNet). This network is meticulously engineered to preserve fine-grained, high frequency details, thereby enriching the robust semantic context of deep features. These details are crucial for distinguishing objects with significant scale variation, overlapping, and occluded objects in higher-level layers. Second, to overcome the irreversible loss of spatial information for diminutive objects caused by aggressive downsampling in standard models, we introduce an Efficient Small Object Pyramid (ESOP). This novel feature pyramid explicitly rescues and integrates high-resolution S2 feature maps into the multi-scale fusion process in a computationally efficient manner. Finally, to bolster the decoder’s localization capabilities in dense scenes, we introduce a two-pronged strategy: a Selective Query Recollection (SQR) training mechanism, which mitigates cascading errors by providing stronger supervision to later decoding stages to address training instability caused by complex and ambiguous backgrounds; and a Geometry-aware Position Encoding (GAPE) strategy, which creates explicit spatial priors to guide the precise localization of tightly clustered targets. Our main contributions are summarized as follows: 1) We propose the High-Frequency Enhanced Semantics Network (HFESNet), a novel backbone that optimally balances the preservation of high-frequency spatial details with the extraction of high-level semantic information, yielding more discriminative features for dense and cluttered aerial scenes. 2) We introduce the Efficient Small Object Pyramid (ESOP), a multi-scale fusion neck that strategically integrates high-resolution features (S2) via a lightweight pre-fusion module. This design significantly boosts small-object detection performance at a modest computational cost. 3) We enhance the decoder’s stability and localization accuracy through two key innovations: the Selective Query Recollection (SQR) training strategy stabilizes bounding box optimization by mitigating error accumulation, while the Geometry-aware Position Encoding (GAPE) module provides the decoder with rich spatial priors, markedly improving localization precision for dense and small objects. 4) Collectively, our proposed enhancements enable our model to significantly outperform the baseline RT-DETR on the challenging VisDrone aerial detection dataset [ 35 ] . While maintaining the real-time performance of RT-DETR, our model achieves a 5.1-point increase in AP 50 and a 3.8-point increase in AP, with a 4M reduction in parameters, thereby validating the efficacy of our targeted architectural redesign. 2 Relation Work 2.1 Real-time Object Detection on UAVs Real-time object detection on Unmanned Aerial Vehicles (UAV-OD) is challenged by the detection of dense small objects and the handling of occlusions, compounded by stringent constraints on computational resources and real-time performance [ 26 ] . The field of real-time object detection has long been dominated by the YOLO family of models [ 32 , 36 , 8 , 19 ] , which established a high-performance benchmark for the trade-off between speed and accuracy. However, YOLO’s inherent dependence on a hand-crafted Non-Maximum Suppression (NMS) post-processing stage induces hyperparameter sensitivity and processing latency, a limitation that becomes a critical bottleneck in the dense-object scenarios characteristic of aerial viewpoints. The introduction of the Detection Transformer (DETR) [ 2 ] signaled a paradigm shift toward truly end-to-end, NMS-free detection, though its substantial computational cost initially precluded its use in real-time applications. RT-DETR [ 33 ] represents a significant breakthrough as the first framework to achieve real-time end-to-end performance, even surpassing state-of-the-art YOLO models. This achievement is attributed to its innovative hybrid encoder, which decouples intra-scale feature interaction from cross-scale feature fusion. Despite RT-DETR’s exceptional performance on general-purpose datasets such