GateMOT: Q-Gated Attention for Dense Object Tracking

Mingjin Lv, Zelin Liu, Feifei Shao, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, Zikai Song · 2026

Read GateMOT: Q-Gated Attention for Dense Object Tracking by Mingjin Lv, Zelin Liu, Feifei Shao, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, Zikai Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26353.txt.

Introduction

Lv and six colleagues built GateMOT, an online tracker for dense scenes where standard attention's all-to-all cost is prohibitive. Its Q-gated attention turns the query into a learnable gate that scales key features element by element, picking relevant signals without global aggregation. Parallel gated heads then feed detection, motion, and re-identification from one shared feature map. On BEE24 it set a leading HOTA mark of 48.4, with strong results on further dense-tracking benchmarks.

Catalog record

CREATORMingjin Lv, Zelin Liu, Feifei Shao, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, Zikai Song
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38780
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.26353

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GateMOT: Q-Gated Attention for Dense Object Tracking Mingjin Lv, Zelin Liu, Feifei Shao, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, Zikai Song Abstract While large models demonstrate the strong representational power of vanilla attention, this core mechanism cannot be directly applied to Dense Object Tracking: its quadratic all-to-all interactions are computationally prohibitive for dense motion estimation on high-resolution features. This mismatch prevents Dense Object Tracking from fully leveraging attention-based modeling in crowded and occlusion-heavy scenes. To address this challenge, we introduce GateMOT , an online tracking framework centered on Q-Gated Attention (Q-Attention) , an efficient and spatially aware attention variant. Our key idea is to repurpose the Query from a similarity-conditioning term into a learnable gating unit. This Gating-Query (Gating-Q) produces a probabilistic gate that modulates Key features in an element-wise manner, enabling explicit relevance selection instead of costly global aggregation. Built on this mechanism, parallel Q-Attention heads transform one shared feature map into task-specific yet consistent representations for detection, motion, and re-identification, yielding a tightly coupled multi-task decoder with linear-complexity gating operations. GateMOT achieves state-of-the-art HOTA of 48.4, MOTA of 67.8, and IDF1 of 64.5 on BEE24, and demonstrates strong performance on additional Dense Object Tracking benchmarks. These results show that Q-Attention is a simple, effective, and transferable building block for attention-based tracking in dense tracking scenarios. 1 Introduction Dense Object Tracking is a fundamental task in computer vision [ 39 , 90 , 40 , 11 , 32 ] , supporting numerous applications [ 79 , 89 , 91 , 41 , 10 , 84 ] such as autonomous driving and robotics [ 71 , 50 , 46 , 26 , 52 ] . Its core challenge is to maintain reliable identities over time while jointly addressing detection, motion estimation, and re-identification. Although attention mechanisms [ 68 , 64 , 61 ] have driven major advances in vision [ 20 , 8 , 55 , 77 ] , their use for dense motion estimation in Dense Object Tracking remains limited [ 65 ] . The quadratic complexity of vanilla attention makes it expensive on high-resolution feature maps with many potential targets, especially in crowded and occlusion-heavy scenes, which has strongly shaped the design of existing trackers [ 69 , 16 , 17 ] . Figure 1 : Comparison between our Q-Attention and vanilla attention. Top : our Q-Attention employs a learned Gating-Q to perform efficient, element-wise gating on Key (K) features. Bottom : vanilla attention relies on two expensive matrix multiplications for global feature aggregation. This computational barrier has shaped most existing Dense Object Tracking frameworks, which can be broadly grouped into three paradigms: (1) tracking-by-detection with lightweight motion predictors [ 4 , 86 , 62 , 60 ] , where hand-crafted models such as the Kalman Filter [ 34 ] keep motion estimation separate from deep visual features and struggle in complex, non-linear dynamics [ 72 , 6 , 63 ] ; (2) query-based association using sparse instance-level queries [ 49 , 81 ] , which apply attention only to a small set of queries and thus avoid dense computation but weaken explicit geometric reasoning for motion [ 5 ] ; and (3) deep motion learning approaches [ 73 ] that infer motion from visual data yet remain decoupled from attention in the motion head to preserve efficiency. Consequently, there is still no mechanism that combines the feature selection strength of attention with the efficiency required for dense motion estimation. To bridge this gap, we rethink attention for Dense Object Tracking by transforming the role of Query. Instead of only conditioning compatibility with Keys, Query is repurposed as a learnable gating unit that generates a dynamic, task-specific control signal for information flow [ 9 , 12 ] . This simple reformulation enables compact and stable dense tracking decoders. Based on this principle, we build GateMOT , an online tracking framework centered on Q-Gated Attention (Q-Attention) . As illustrated in Figure 1 , the Gating-Query (Gating-Q) learns a probabilistic gate that modulates Key features via efficient element-wise gating, replacing costly all-to-all interactions [ 69 , 16 , 17 ] . The gated Key pathway is fused with an unfiltered Value pathway to preserve feature integrity. Because Q-Attention is lightweight enough to be used in every task head, GateMOT adopts a tightly coupled decoder that generates task-specific yet consistent features for detection, motion, and re-identification while keeping computation efficient in dense settings. Our contributions are summarized as follows: • We introduce GateMOT , a tightly coupled online tracking framework that jointly models detection, motion, and re-identification from a shared dense feature map. • We propose Q-Gated Attention , a compact gating-based attention module in GateMOT that transforms the Query into a learnable gate, enabling efficient and task-specific feature selection for dense tracking. • We show that this design yields strong performance in dense tracking scenarios and transfers across backbone families, indicating that Q-Attention is a practical and general building block beyond a single architecture. 2 Related Work We group prior Dense Object Tracking methods into three families: Kalman-based, query-based, and learnable-motion approaches. 2.1 Kalman Filter-Based Methods The Tracking-by-Detection (TBD) paradigm [ 21 , 74 ] underlies many modern Dense Object Tracking systems [ 4 , 72 , 86 , 1 , 6 ] . These methods decouple motion prediction from visual feature extraction for efficiency: state estimation is delegated to a Kalman Filter operating in a low-dimensional state space [ 34 , 83 , 33 ] , while deep networks focus solely on detection. Although computationally attractive, this decoupling forces the tracker to rely on a fixed linear motion model and prevents motion estimation from fully exploiting rich visual cues, which can hurt robustness under complex, non-linear dynamics [ 72 , 6 ] . To address this limitation, GateMOT uses Q-Gated Attention (Q-Attention) to derive motion features directly from dense visual representations, reducing the reliance on fixed linear priors while keeping an efficient online tracking pipeline. 2.2 Query-Based Methods A line of query-based Dense Object Tracking methods, inspired by Transformer-based object detection [ 8 , 95 ] and multimodel reasoning [ 37 , 85 , 38 , 92 , 31 ] , applies attention to a sparse set of instance-level track queries rather than dense feature maps [ 49 , 81 , 66 , 82 , 78 ] . Each query represents an object and attends across frames, thereby avoiding the cost of dense attention [ 23 , 14 ] . However, because attention is primarily used for association, motion is modeled only implicitly and geometric priors are weakened, while the persistent query–object binding can be inflexible for frequent track births and deaths in dynamic scenes [ 5 ] . To address this trade-off, GateMOT applies Q-Attention in dense heads to preserve explicit motion prediction and geometry-aware association, while still controlling computation through linear-complexity gating. 2.3 Learnable Motion-Based Methods Seeking to overcome the limitations of the Kalman Filter, another line of work develops learnable, data-driven motion models that regress inter-frame offsets using dedicated network heads [ 2 , 73 , 51 , 93 , 67 , 54 ] . While more flexible than rigid hand-crafted models, these approaches typically avoid using attention within the motion head to remain tractable,

Read the complete text