DS-Det: Single-Query Paradigm and Attention Disentangled Learning for Flexible Object Detection Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27
Guiping Cao, Xiangyuan Lan, Wenjian Huang, Jianguo Zhang, Dongmei Jiang, Yaowei Wang · 2025
Read DS-Det: Single-Query Paradigm and Attention Disentangled Learning for Flexible Object Detection Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27 by Guiping Cao, Xiangyuan Lan, Wenjian Huang, Jianguo Zhang, Dongmei Jiang, Yaowei Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19807.txt.
Introduction
Guiping Cao and co-authors present a more flexible object detector for images in 2025. Popular detectors use a fixed set of search slots, called queries. Fixed slots limit how many objects can be found. The team finds two flaws in the fixed design. Two attention steps fight each other, and shared layers blur two kinds of matching.
Their detector, called DS-Det, uses a single query form that can flex in count. One attention step places boxes, and the other removes copies. That split fixes both flaws and speeds the decoder. A new loss leans on box size to stress hard cases like small objects. Tests on two data sets and five backbones show broad gains.
Catalog record
| CREATOR | Guiping Cao, Xiangyuan Lan, Wenjian Huang, Jianguo Zhang, Dongmei Jiang, Yaowei Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-19807
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DS-Det: Single-Query Paradigm and Attention Disentangled Learning for Flexible Object Detection Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland DOI: 10.1145/3746027.3755045 ISBN: 979-8-4007-2035-2/2025/10 CCS: Computing methodologies Computer Vision Guiping Cao, Xiangyuan Lan, Wenjian Huang, Jianguo Zhang, Dongmei Jiang, Yaowei Wang Abstract . Popular transformer detectors have achieved promising performance through query-based learning using attention mechanisms. However, the roles of existing decoder query types ( e.g. , content query and positional query) are still underexplored. These queries are generally predefined with a fixed number ( fixed-query ), which limits their flexibility. We find that the learning of these fixed-query is impaired by Recurrent Opposing inTeractions (ROT) between two attention operations: Self-Attention (query-to-query) and Cross-Attention (query-to-encoder), thereby degrading decoder efficiency. Furthermore, “ query ambiguity ” arises when shared-weight decoder layers are processed with both one-to-one and one-to-many label assignments during training, violating DETR’s one-to-one matching principle. To address these challenges, we propose DS-Det , a more efficient detector capable of detecting a flexible number of objects in images. Specifically, we reformulate and introduce a new unified Single-Query paradigm for decoder modeling, transforming the fixed-query into flexible. Furthermore, we propose a simplified decoder framework through attention disentangled learning: locating boxes with Cross-Attention (one-to-many process), deduplicating predictions with Self-Attention (one-to-one process), addressing “query ambiguity” and “ROT” issues directly, and enhancing decoder efficiency. We further introduce a unified PoCoo loss that leverages box size priors to prioritize query learning on hard samples such as small objects. Extensive experiments across five different backbone models on COCO2017 and WiderPerson datasets demonstrate the general effectiveness and superiority of DS-Det. The source codes are available at https://github.com/Med-Process/DS-Det/ . 1. Introduction In the last few years, transformer detectors like DETR ( Carion et al. 2020 ) and DINO ( Zhang et al. 2022a ) , have simplified the detection pipeline by providing end-to-end detection capabilities through query-based learning using attention mechanisms, demonstrating promising performance compared to classical CNN-based detectors ( Girshick 2015 ; Ren et al. 2015 ; Liu et al. 2016 ; Redmon et al. 2016 ; He et al. 2017 ) . Subsequently, a series of follow-up works have been proposed to boost DETR on the architecture of encoder and decoder ( Zhu et al. 2020 ; Cao et al. 2022 ) , query formulations ( Meng et al. 2021 ; Liu et al. 2022 ; Cao et al. 2024 ) , and training efficacy ( Meng et al. 2021 ; Zhang et al. 2022b ; Zhang et al. 2022a ; Jia et al. 2023 ; Hu et al. 2024 ) . Query Components and Roles. DETR-like models ( Carion et al. 2020 ; Meng et al. 2021 ; Yao et al. 2021 ; Liu et al. 2022 ; Li et al. 2022 ; Zhang et al. 2022a ; Zheng et al. 2023 ; Cao et al. 2024 ; Cao et al. 2025 ) formulate object detection (OD) as a set prediction task using a transformer encoder-decoder architecture. These models detect objects through interactions in the decoder between object queries and encoder features. Each query is responsible for the prediction of the class label and bounding box coordinates of a single object, following a one-to-one matching manner. However, the query components and roles are underexplored ( Liu et al. 2022 ; Zhang et al. 2022a ) . For example, the pioneering DETR ( Carion et al. 2020 ) introduces two types of queries: one initialized as a zero vector and another with static learned embeddings, yet their semantic meanings for detection remain unclear. Although subsequent works ( Meng et al. 2021 ; Liu et al. 2022 ; Li et al. 2022 ; Zhang et al. 2022a ; Hu et al. 2024 ) have further categorized these decoder queries into Content Query (CQ) and Positional Query (PQ), thus providing some explicit semantics, the precise roles and impacts of these queries for object detection remain understudied. This limitation constrains further performance enhancement for transformer detectors. Fixed-Query’s Limitation. Most existing DETR-like models ( Zhu et al. 2020 ; Zhang et al. 2022a ; Liu et al. 2023 ; Hu et al. 2024 ) predefine a fixed-size set of N learnable object queries (including both the PQ and CQ), which can only detect a fixed number of objects, thus limiting their flexibility in dense scenarios. Query Ambiguity. Both CQ and PQ are updated layer-by-layer in the transformer decoder through Self-Attention (SA, query-to-query interactions) and Cross-Attention (CA, query-to-encoder feature interactions) mechanisms. However, existing researches ( Meng et al. 2021 ; Hu et al. 2024 ) demonstrate that these attention operations exert opposing effects: CA gathers multiple queries around the same object, while SA disperses queries from each other. These Recurrent Opposing inTeractions (ROT) in current decoder frameworks ( Carion et al. 2020 ; Li et al. 2022 ; Zhang et al. 2022a ; Hu et al. 2024 ; Cao et al. 2024 ; Huang et al. 2024 ) create “ query ambiguity ”, where queries struggle to both converge and diverge for the same object during the optimization process, thereby degrading decoder efficiency. Furthermore, the mixing of one-to-one matching (DETR’s foundational property for end-to-end detection) and one-to-many matching (the core method for improving training efficiency and convergence) also introduces “ query ambiguity ” when applied to queries with shared decoder weights. Thus, models struggle to predict single or multiple results for the same object, as illustrated in Fig. 1 (c). Figure 1 . The mixing of one-to-one and one-to-many matching with shared weights in the decoder introduces query ambiguity , causing multiple queries to predict the same single object and increasing false positives. DS-Det. To address these challenges, we first comprehensively evaluate the query components in transformer detectors, analyzing their functional roles. Building on this analysis, we reformulate the query composition mechanism by introducing a novel Single-Query paradigm that transforms traditional fixed-query into flexible-query . Furthermore, to address the “ query ambiguity ” problem in decoder architectures, we systematically investigate CA and SA mechanisms and further develop a novel efficient decoder architecture that significantly improves its efficiency. Overall, our key contributions can be summarized as follows: (1) We propose an efficient transformer detector termed DS-Det , which reformulates the query paradigm through Flexible singLe-quEry generaTion (FLET) module, transforming the fixed-query into flexible-query and enhancing the model’s detection flexibility. (2) We design a novel Attention Disentangled Decoder (ADD) architecture that disentangles the attentions of CA and SA and further separates one-to-one and one-to-many matching processes, efficiently addressing the issues of “query ambiguity” and “ROT” and significantly improving decoder efficiency. (3) We introduce a new loss function, which incorPOrates Classification with iOu and bOx size (PoCoo) for re-weighting the classification loss, enhancing query’s learning on hard samples. (4) Extensive experiments on the COCO2017 and WiderPerson datasets demonstrate the general effectiveness of DS-Det across five backbone models. Furthermore, DS-Det shows significant advantages in flexible-query and decoder efficiency, improving the decoder inference speed by +34.8% . 2. Related Works Transformer Detectors. DETR ( Carion et al. 2020 ) , as the pioneer transformer detector, represents a significant breakthrough in OD. By framing OD as