Hypergraph-State Collaborative Reasoning for Multi-Object Tracking
Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Xinchao Wang · 2026
Read Hypergraph-State Collaborative Reasoning for Multi-Object Tracking by Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Xinchao Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12665.txt.
Introduction
Zikai Song and four colleagues steady multi-object tracking with joint motion reasoning. Motion reasoning links targets across frames, but noise and occlusion break it. Noisy guesses destabilize tracks, and hidden targets split paths. Their plan lets objects with like motion check and refine each other. A HyperSSM design joins hypergraph math with a state space model. Hyperedges capture spatial motion links, while state steps enforce smooth time flow. Space agreement and time coherence are tuned together. Tests on four benchmarks show leading scores across varied scenes.
Catalog record
| CREATOR | Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Xinchao Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36562 |
| LICENSE | http://creativecommons.org/publicdomain/zero/1.0/ |
| RIGHTS | CC0 |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12665
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hypergraph-State Collaborative Reasoning for Multi-Object Tracking Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Xinchao Wang Abstract Motion reasoning serves as the cornerstone of multi-object tracking (MOT), as it enables consistent association of targets across frames. However, existing motion estimation approaches face two major limitations: (1) instability caused by noisy or probabilistic predictions, and (2) vulnerability under occlusion, where trajectories often fragment once visual cues disappear. To overcome these issues, we propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects. By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded. To realize this concept, we design HyperSSM , an architecture that integrates Hypergraph computation and a State Space Model (SSM) for unified spatial–temporal reasoning. The Hypergraph module captures spatial motion correlations through dynamic hyperedges, while the SSM enforces temporal smoothness via structured state transitions. This synergistic design enables simultaneous optimization of spatial consensus and temporal coherence, resulting in robust and stable motion estimation. Extensive experiments on four mainstream and diverse benchmarks(MOT17, MOT20, DanceTrack, and SportsMOT) covering various motion patterns and scene complexities, demonstrate that our approach achieves state-of-the-art performance across a wide range of tracking scenarios. 0 0 footnotetext: * Corresponding Authors: Xinchao Wang ([email protected]) and Junqing Yu ([email protected]) 0 0 footnotetext: Code is available at https://github.com/SkyeSong38/HyperMOT 1 Introduction Figure 1 : Hypergraph-based collaborative motion estimation . We first construct a motion-aware hypergraph, represented by an incidence matrix, the hyperedge ( e ) selectively associating strongly correlated targets based on motion states ( gathering ), the gathered group is then dispersed back to individual nodes for fine-grained motion refinement ( scattering ), achieving both collaborative inference and precise object-specific estimation. Multi-object tracking (MOT) aims to detect and continuously track multiple targets in dynamic scenes while preserving their identities across consecutive frames 73 ; 72 ; 26 ; 31 ; 53 ; 49 ; 56 ; 40 . Unlike object detection, which focuses solely on identifying objects within a single image 43 ; 46 ; 45 ; 44 , MOT must establish temporal associations between detections over time. This makes accurate estimation of inter-frame motion indispensable. Motion estimation thus lies at the core of MOT, serving as the fundamental mechanism that enables consistent target association and identity maintenance throughout a video sequence 27 ; 70 ; 71 ; 50 . Existing MOT methods can be divided into two distinct categories depending on whether they explicitly estimate motion displacement. (1) Query-based methods 66 ; 69 ; 16 eliminate the need for explicit motion modeling. They represent each target as a learnable query with each query corresponding to a specific object, through iterative network updates, the query directly regresses the object’s position. (2) Motion-based methods explicitly estimate object motion and integrate it with detection results for tracking. Traditional approaches 68 ; 33 often employ Kalman Filter motion estimation . Recent frameworks 25 ; 48 , further leverage learnable motion estimation to capture complex, nonlinear motion dynamics, leading to more robust tracking performance. Our approach follows the motion-based paradigm and aims to tackle two long-standing challenges in motion estimation. (C1) Motion noise and instability . Paradoxically, despite the rapid development of learning-based approaches, the hand-crafted Kalman Filter (KF) and its variants remain among the most stable motion estimators. KF assumes constant-velocity linear motion and performs remarkably well in simple, linear trajectories (such as MOT17 37 ). In contrast, learnable motion estimators can model complex nonlinear dynamics, yet their inherently probabilistic predictions often introduce noise and fluctuations, leading to less consistent performance in linear scenarios. (C2) Robustness under occlusion . When a target becomes heavily occluded, motion estimation tends to fail, resulting in trajectory fragmentation or identity switches. Both KF and learnable motion estimation methods struggle to maintain temporal continuity once visual evidence is lost. To address these challenges, we introduce a collaborative reasoning framework for motion estimation, as shown in Figure 1 . Unlike other approaches that estimate motion independently for each target, our method performs joint estimation across multiple correlated objects. Rather than aggregating all targets indiscriminately, we selectively associate those with highly correlated motion states (in terms of direction and velocity), to preserve individual motion characteristics while enabling cooperative inference. Our approach builds on two key principles to tackle the aforementioned challenges. (1) Joint reasoning inherently stabilizes motion estimation (for C1 ). When the trajectory of one target becomes noisy or unstable, the shared constraints from other motion-correlated targets help suppress instability and restore consistency. (2) Cross-object motion transfer enhances occlusion resilience (for C2 ). When a target becomes temporarily invisible, its motion can be probabilistically inferred from the observable dynamics of other targets exhibiting similar motion patterns. (e.g. estimating the trajectory of an occluded pedestrian based on the motion trends of others moving in the same direction.) We propose HyperSSM to realize collaborative reasoning for motion estimation, HyperSSM is a unified architecture that synergizes Hypergraph computation with State Space Model (SSM) for spatial collaborative reasoning and temporal motion estimation . (a) From the spatial perspective, the Hypergraph module dynamically constructs motion-aware hyperedges that group objects exhibiting correlated motion states. These hyperedges aggregate object-specific motion information into shared latent group states, enabling spatial collaboration that suppresses individual estimation noise through consensus reasoning. (b) From the temporal perspective, the SSM module models motion evolution via a structured autoregressive process, where motion states are propagated through learnable transition matrices to maintain temporal continuity and adapt to dynamic changes. (c) Finally, HyperSSM bridges spatial and temporal reasoning by embedding hypergraph-derived relational constraints directly into the SSM’s state transition rules. This integration enforces consistent optimization of spatial group coherence and temporal trajectory smoothness, resulting in robust and stable motion estimation. Comprehensive experiments on both linear motion datasets (MOT17 37 , MOT20 10 ) and non-linear motion datasets (DanceTrack 54 , SportsMOT 8 ) demonstrate the state-of-the-art performance of our method. In summary, our main contributions are as follows: 1. We propose a collaborative motion estimation framework that performs joint reasoning among correlated objects, effectively suppressing noise fluctuations and enhancing robustness under occlusion through cross-object motion constraint and information sharing. 2. We design HyperSSM, a unified architecture that integrates Hypergraph computation with SSM to achieve spatial–temporal collaborative reasoning. The Hypergraph module dynamically captures motion correlations among objects, while the SSM propagates these relationships through structured temporal transitions, jointly optimizing spatial coherence and temporal smoothness for stable motion estimation. 2 Related Work Object tracking has numerous applications in computer vision 64 ; 12 ; 32 ; 13 ; 62 ; 52 ; 51 . Existing MOT methods can be divided into Query-based methods and Motion-based methods depending on whether they explicitly estimate motion of objects. Query-based methods (such as TrackFormer 36 and the MOTR series 66 ; 69 ; 63 ) do not require explicit motion computation. Derived from DETR 6 ; 74 , they represent each target as a learnable query that regresses object positions, maintaining identity consistency through persistent queries across the sequence. Recently, when enhanced with historical memory and large-scale hybrid modeling 63 ; 42 ; 29 ; 16 , these methods have achieved strong performance in highly