DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association
Zenghuang Fu, Xiaofeng Han, Mingda Jia, Jinming Yang, Qi Zeng, Muyang Zhang, Changwei Wang, Weiliang Meng Xiaopeng Zhang · 2025
Read DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association by Zenghuang Fu, Xiaofeng Han, Mingda Jia, Jinming Yang, Qi Zeng, Muyang Zhang, Changwei Wang, Weiliang Meng Xiaopeng Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17860.txt.
Introduction
Zenghuang Fu, with eight co-authors, builds DMTrack to follow many moving objects in video shot from drones. Drone views are hard because targets move in odd ways, hide behind each other, and look alike. Sudden drone motion also breaks plain motion models and swaps identities between tracks.
DMTrack mixes a flexible state-space predictor with a Kalman filter through a small gating unit. It then matches targets by motion trend and by how sure the model is. Tests on drone tracking data back the design. The paper should help readers who need steady tracking from fast, shaky aerial cameras.
Catalog record
| CREATOR | Zenghuang Fu, Xiaofeng Han, Mingda Jia, Jinming Yang, Qi Zeng, Muyang Zhang, Changwei Wang, Weiliang Meng Xiaopeng Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17860
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association Zenghuang Fu, Xiaofeng Han, Mingda Jia, Jinming Yang, Qi Zeng, Muyang Zhang, Changwei Wang, Weiliang Meng Xiaopeng Zhang Abstract Multi-object tracking (MOT) from unmanned aerial vehicles (UAVs) presents unique challenges due to unpredictable object motion, frequent occlusions, and limited appearance cues inherent to aerial viewpoints. These issues are further exacerbated by abrupt UAV movements, leading to unreliable trajectory estimation and identity switches. Conventional motion models, such as Kalman filters or static sequence encoders, often fall short in capturing both linear and non-linear dynamics under such conditions. To tackle these limitations, we propose DMTrack, a deformable motion tracking framework tailored for UAV-based MOT. Our DMTrack introduces three key components: DeformMamba, a deformable state-space predictor that dynamically aggregates historical motion states for adaptive trajectory modeling; MotionGate, a lightweight gating module that fuses Kalman and Mamba predictions based on motion context and uncertainty; and an uncertainty-aware association strategy that enhances identity preservation by aligning motion trends with prediction confidence. Extensive experiments on the VisDrone-MOT and UAVDT benchmarks demonstrate that our DMTrack achieves state-of-the-art performance in identity consistency and tracking accuracy, particularly under high-speed and non-linear motion. Importantly, our method operates without appearance models and maintains competitive efficiency, highlighting its practicality for robust UAV-based tracking. Introduction Figure 1: Under abrupt motion (yellow arrows), Kalman-only matching yields lower matching scores (0.6, 0.7), whereas DMTrack achieves higher confidence (0.8, 0.9) through deformable motion modeling, adaptive fusion, and uncertainty-aware association. Multi-object tracking (MOT) is a fundamental task in computer vision with broad applications in surveillance ( Yang et al. 2023 ; Hassan et al. 2024 ) , aerial inspection ( Isaac-Medina et al. 2021 ; Dong et al. 2025 ) , intelligent transportation ( Gao et al. 2024 ; Li et al. 2024 ) , and autonomous systems ( Xu, Razavi, and Zheng 2023 ; Nagy et al. 2025 ) . In recent years, the increasing use of unmanned aerial vehicles (UAVs) for urban monitoring and emergency response has brought significant attention to UAV-based MOT (UAV-MOT)—a particularly challenging variant of the task ( Jiang et al. 2021 ) . The top-down view, small target sizes, agile platform movement, and frequent changes in perspective make UAV-MOT uniquely difficult, often resulting in complex, unpredictable object dynamics ( Wu et al. 2021 ) . Specifically, tracked objects frequently exhibit high-speed motion, strong non-linearity, repeated occlusion, and disappear-reappear patterns. At the same time, visual cues degrade severely due to low resolution and significant scale variation. These factors collectively undermine the performance of traditional tracking-by-detection frameworks, which depend heavily on high-quality detections and robust appearance-based Re-ID features ( Chu et al. 2023 ) . On the one hand, appearance features become unreliable in UAV scenarios due to resolution loss and perspective distortion. On the other hand, standard motion models such as Kalman filters—based ( Welch, Bishop et al. 1995 ) on linear constant-velocity assumptions—struggle with complex trajectories, often leading to inaccurate motion predictions and identity switches. To mitigate these issues, recent works have explored deep sequence models such as RNNs ( Zhang et al. 2020 ) and neural state-space models to capture long-term motion dependencies ( Smith et al. 2023 ) . Others have proposed learning Kalman-like filters to improve adaptability ( Zhou, Koltun, and Krähenbühl 2020 ; Shuai et al. 2021 ) . However, these approaches are typically constrained by linear priors, static update mechanisms, and a lack of dynamic memory control, making them inadequate for modeling highly non-linear and context-dependent motion patterns in real-world UAV settings. Broadly speaking, existing methods suffer from two critical limitations:(i) They lack the ability to adaptively select motion history based on the current motion context;(ii) They fail to integrate the stability of classical physics-based models (e.g., Kalman filtering) with the modeling capacity of deep neural networks.Figure. 1 clearly illustrates these limitations.Under abrupt motion conditions, traditional Kalman-based matching methods yield lower matching confidence due to rigid linear assumptions. In contrast, our proposed DMTrack successfully resolves this issue, obtaining significantly higher matching scores through deformable modeling and adaptive fusion. To overcome these challenges, we propose DMTrack—a deformable motion modeling framework specifically designed for UAV-based multi-object tracking. Our DMTrack focuses on improving motion prediction accuracy and robust data association in aerial scenes, while completely removing reliance on appearance features. Our framework comprises three novel components: • DeformMamba. A deformable state-space encoder that dynamically predicts temporal offsets to select and interpolate informative motion states from history, enabling flexible modeling of complex, non-linear trajectories. • MotionGate. A lightweight gating mechanism that adaptively fuses motion predictions from both Kalman filtering and DeformMamba, leveraging motion context to balance physical stability and learned adaptability. • Uncertainty-aware matching strategy. A robust data association method that discards appearance cues and instead leverages motion trend alignment, prediction uncertainty, and temporal consistency to enhance identity preservation during occlusions and reappearances. Extensive experiments on two challenging UAV-MOT benchmarks VisDrone-MOT ( Zhu et al. 2021 ) and UAVDT ( Du et al. 2018 ) validate that our DMTrack consistently achieves state-of-the-art tracking accuracy and identity consistency, especially under high-speed, non-linear motion and dense crowd conditions. Importantly, our DMTrack maintains real-time performance and does so without any appearance modeling, making it well-suited for real-world UAV applications with constrained computational resources. Related Works Tracking-by-Detection Frameworks Tracking-by-detection has become the dominant paradigm in multi-object tracking (MOT), where object trajectories are formed by associating detections across frames ( Zhang et al. 2021 ) . Traditional methods often rely on a two-stage pipeline: a detector provides candidate bounding boxes, and an association algorithm links them into trajectories based on motion or appearance cues ( Zhang et al. 2022 ) . Recent advances such as SORT ( Bewley et al. 2016 ) and DeepSORT ( Wojke, Bewley, and Paulus 2017 ) improve association accuracy by integrating Kalman filters and appearance features. However, in UAV scenarios, appearance cues are unreliable due to low resolution, drastic scale changes, and frequent occlusion ( Sandino et al. 2020 ) . This limitation has sparked interest in appearance-free trackers, most notably ByteTrack ( Zhang et al. 2022 ) , which proposes a high-low confidence detection matching strategy solely based on motion predictions from Kalman filtering. Despite its simplicity, ByteTrack achieves strong performance and has inspired several successors including OC-SORT ( Cao et al. 2023 ) and BoT-SORT ( Aharon, Orfaig, and Bobrovsky 2022 ) . Nevertheless, these methods rely heavily on handcrafted motion models (e.g., linear Kalman updates), making them less effective in modeling nonlinear dynamics common in UAV videos, such as sudden acceleration or directional shifts. Our work builds upon this motion-only paradigm, aiming to enhance the trajectory prediction module by incorporating adaptive and learnable motion models. Figure 2: Overview of our DMTrack framework. Our DMTrack first models motion using both Deformable Mamba and Kalman filters, then adaptively fuses their predictions via MotionGate. The final association relies solely on motion cues—IoU, trajectory trend, and uncertainty—to generate robust tracking results without appearance features. Motion Modeling in MOT Accurate motion modeling is vital for MOT, especially in UAV scenarios where objects exhibit fast and non-linear dynamics ( Zheng et al. 2025 ) . Traditional approaches rely on Kalman filters, which assume linear motion and often fail under