A Hybrid Approach for Visual Multi-Object Tracking

Toan Van Nguyen ID, Rasmus G. K. Christiansen ID, Dirk Kraft ID, Leon Bodenhagen ID · 2025

Read A Hybrid Approach for Visual Multi-Object Tracking by Toan Van Nguyen ID, Rasmus G. K. Christiansen ID, Dirk Kraft ID, Leon Bodenhagen ID (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24410.txt.

Introduction

Toan Van Nguyen leads a team of four authors on a vision task. The task is multi-object tracking, which means following many things in video. Each thing must keep the same label as frames pass, even in crowds. Their method, called GenTrack2, mixes two styles of tracking in one system. One style uses chance and sampling, the other uses firm matching rules.

The sampling part spreads guesses for where each thing moves next. A swarm search pulls those guesses toward the most likely spots. The matching part pairs guesses with fresh sightings by a cost score. That score weighs place, sighting strength, and past track quality. Past speeds give trend seeds that guide both sampling and updates. Tests show the tracker beats strong rivals on video and live streams.

Catalog record

CREATORToan Van Nguyen ID, Rasmus G. K. Christiansen ID, Dirk Kraft ID, Leon Bodenhagen ID
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-24410

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Hybrid Approach for Visual Multi-Object Tracking Toan Van Nguyen ID, Rasmus G. K. Christiansen ID, Dirk Kraft ID, Leon Bodenhagen ID Abstract This paper proposes a visual multi-object tracking method that jointly employs stochastic and deterministic mechanisms to ensure identifier consistency for unknown and time-varying target numbers under nonlinear dynamics. A stochastic particle filter addresses nonlinear dynamics and non-Gaussian noise, with support from particle swarm optimization (PSO) to guide particles toward state distribution modes and mitigate divergence through proposed fitness measures incorporating motion consistency, appearance similarity, and social-interaction cues with neighboring targets. Deterministic association further enforces identifier consistency via a proposed cost matrix incorporating spatial consistency between particles and current detections, detection confidences, and track penalties. Subsequently, a novel scheme is proposed for the smooth updating of target states while preserving their identities, particularly for weak tracks during interactions with other targets and prolonged occlusions. Moreover, velocity regression over past states provides trend-seed velocities, enhancing particle sampling and state updates. The proposed tracker is designed to operate flexibly for both pre-recorded videos and camera live streams, where future frames are unavailable. Experimental results confirm superior performance compared to state-of-the-art trackers. The source-code reference implementations of both the proposed method and compared-trackers are provided on GitHub: https://github.com/SDU-VelKoTek/GenTrack2 . firstpage,color=gray!90,angle=0,scale=0.28, xpos=0in,ypos=-5in]*correspondence: [email protected] 1 Introduction Visual multi-object tracking (MOT) enables a variety of robotic tasks such as object manipulation, human-robot interaction, autonomous navigation in dynamic environments. Despite significant progress, MOT remains challenging due to factors such as object similarity, occlusions, and irregular motion. High similarity between objects can cause identity confusion, while partial or complete occlusions may lead to temporary loss of tracked targets. In addition, irregular or unpredictable object motion further complicates the association of objects across frames. Addressing these challenges is essential for developing robust and accurate MOT systems that can operate effectively in real-world applications. 1.1 Related Works Multi-object tracking methods are commonly categorized into tracking-by-detection and detection-free paradigms. In settings with variable numbers of targets, tracking-by-detection offers distinct advantages, as frame-wise detections enable the automatic initiation and termination of tracks, and mitigate drift accumulation. These characteristics, reinforced by recent advances in object detection, have established tracking-by-detection as the prevailing approach in state-of-the-art MOT methods. Here, the Kalman filter is widely used for updating motion state and estimating positions, commonly paired with tracklet-detection association. In [ 1 ] , a practical MOT approach, known as SORT, enables real-time tracking but relies heavily on detection quality. In [ 2 ] , DeepSORT extends SORT with appearance features to reduce identity switches, while ByteTrack [ 3 ] improves association by handling high and low confidence detections separately. In [ 4 ] , BoT-SORT further refines performance with a refined Kalman filter, camera compensation, and motion and appearance cues. In [ 5 ] , OC-SORT handles occlusion noise by estimating virtual trajectories from observations. In [ 6 ] , SMILEtrack builds on ByteTrack and BoT-SORT, integrating an object detector with a Siamese similarity module to enhance appearance similarity compute and matching. In [ 7 ] , ConfTrack adds low-confidence penalization and cascading to a Kalman filter-based framework to handle noisy detections. Notably, Kalman-based trackers assume linear motion and Gaussian noise, yet real-world MOT often involves non-linear dynamics and non-Gaussian noise. Since the introduction of particle filter-based single-object tracker, numerous innovative approaches have been made to enhance accuracy and robustness. In [ 8 ] , MCMC (Markov Chain Monte Carlo) resampling and kernel-based particle pipelining were proposed to improve computational efficiency. In [ 9 ] , an occlusion-aware particle filter was introduced, integrating color and motion vector features within a patch-based model to manage occlusion effectively. To mitigate degeneracy during importance resampling, particle swarm optimization is employed in [ 10 ] , while the framework further with multi-task correlation filters that jointly exploit interdependencies among features are conducted in [ 11 ] . Chaos theory was incorporated in [ 12 ] through a chaotic particle filter, supported by global motion estimation and color-based refinement. In [ 13 ] , crow search optimization accelerated convergence in the particle filter tracking framework and was paired with an adaptive multi-cue fusion model. In [ 14 ] , a scene-dependent feature fusion is proposed with a partitioned template model, addressing real-world challenges such as shadows and illumination changes. Hybrid quantum particle swarm and adaptive genetic optimization [ 15 ] , as well as a minimax-based sequential Monte Carlo filter [ 16 ] , further improved particle diversity and tracking reliability. Particle filter–based multi-object tracking faces the challenge of a high-dimensional state space that grows with the number of objects. Independent particle filters for single targets [ 17 , 18 ] often fail when similar objects interact, while multi-object particle filters [ 19 , 20 ] usually assume a fixed number of targets. Few studies address particle filter with unknown and time-varying target numbers [ 21 , 22 , 23 , 24 , 25 ] . The approach in [ 21 ] tackled this but scaled poorly as target numbers increased. To improve the tracking performance, a method in [ 22 ] used a modified Metropolis-Hastings algorithm with add-delete and stay-leave operations, but suffered from duplicate tracks and reduced reliability under frequent target changes. These particle-based methods focus on handling variable target counts, paying less attention to factors like object scale or appearance. In contrast, detection-based approaches [ 23 , 24 , 25 ] handle target addition and removal more deterministically and emphasize better observation models, though often neglect optimization of the tracking inference. 1.2 Contributions and Organizations The main contributions of this paper include: (1) a hybrid visual multi-object tracking framework that maintains identifier consistency for unknown and time-varying target numbers under nonlinear dynamics by integrating stochastic and deterministic approaches. A novel target update scheme is introduced, incorporating target interactions in state estimation and particle optimization via PSO with tailored fitness measures. Additionally, a more generalized and systematic cost matrix is proposed for matching tracklets to current detections; (2) a velocity regression that leverages historical states to improve particle sampling and state updates, particularly under occlusions or with weak/noisy detectors; and (3) source-code reference implementations of the proposed method and seven compared-trackers to facilitate re-implementation and comparative evaluation. The remainder of this paper is structured as follows: Section II outlines the methodological background and a visual multi-object tracking framework. Section III evaluates the proposed tracker on human tracking, followed by conclusions in Section IV. 2 Methodology This section begins by formulating the proposed tracking framework, and is then followed by a comprehensive and detailed description of primary processes involved in visual multi-object tracking. 2.1 Tracking Approach Object tracking can be modelled as a nonlinear Bayesian filtering problem, where the posterior state is conditionally estimated from available measurements. In multi-object tracking, targets may enter or leave the scene, requiring a tracker that handles a variable number of targets. Thus, a set of target identifiers is essentially maintained [ 21 ] . The system state is defined as \{K_{t},X_{t}\} , with K_{t} representing the identifiers of k current targets, while X_{t} representing their states. Here, k denotes the time-varying number of targets, assuming independent target entries and leaves. In sampling-based methods, the density P(K_{t},X_{t}|Z_{0:t}) is approximated with S random

Read the complete text