Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models
Shaoxiu Wei, Mingchao Liang, and Florian Meyer · 2025
Read Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models by Shaoxiu Wei, Mingchao Liang, and Florian Meyer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-18124.txt.
Introduction
Shaoxiu Wei, Mingchao Liang, and Florian Meyer blend neural networks into Bayesian multi-object tracking. Their paper from twenty twenty-five weighs two camps. Model-based trackers work anywhere but lean on plain motion and sensor models. Data-driven trackers shine where labeled data abound but travel poorly. Their hybrid keeps the Bayesian frame and upgrades its weak parts with learned nets.
Belief propagation keeps the math tractable in high dimensions. Sequential Monte Carlo steps handle the low-dimensional parts with care. The nets enrich the motion and measurement models where hand forms fall short. Tests on the nuScenes driving set reach top-tier scores. The authors claim the mix keeps model-based range while gaining data-driven sharpness. Uses span driving, ocean study, and air surveillance.
Catalog record
| CREATOR | Shaoxiu Wei, Mingchao Liang, and Florian Meyer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-18124
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models Shaoxiu Wei, Mingchao Liang, and Florian Meyer Abstract Multiobject tracking (MOT) is an important task in applications including autonomous driving, ocean sciences, and aerospace surveillance. Traditional MOT methods are model-based and combine sequential Bayesian estimation with data association and an object birth model. More recent methods are fully data-driven and rely on the training of neural networks. Both approaches offer distinct advantages in specific settings. In particular, model-based methods are generally applicable across a wide range of scenarios, whereas data-driven MOT achieves superior performance in scenarios where abundant labeled data for training is available. A natural thought is whether a general framework can integrate the two approaches. This paper introduces a hybrid method that utilizes neural networks to enhance specific aspects of the statistical model in Bayesian MOT that have been identified as overly simplistic. By doing so, the performance of the prediction and update steps of Bayesian MOT is improved. To ensure tractable computation, our framework uses belief propagation to avoid high-dimensional operations combined with sequential Monte Carlo methods to perform low-dimensional operations efficiently. The resulting method combines the flexibility and robustness of model-based approaches with the capability to learn complex information from data of neural networks. We evaluate the performance of the proposed method based on the nuScenes autonomous driving dataset and demonstrate that it has state-of-the-art performance . I Introduction Multiobject tracking (MOT) [ 1 , 2 , 3 , 4 , 5 ] is an important capability in a variety of applications, including autonomous driving, ocean sciences, and aerospace surveillance. Most MOT methods follow the “detect-then-track” paradigm. Here, at each discrete time step, a measurement detector is applied to the raw sensor data. This detector provides the measurements for Bayesian MOT. Measurements are a representation of the position, motion, and shape of objects, e.g, they can be ranges and angles, range rate, positions in Cartesian coordinates, or bounding boxes. Applying a measurement detector to the raw sensor data reduces the data rate and facilitates the development of the measurement model. However, it typically also leads to a loss of object-related information and thus to a potential reduction of tracking performance. In MOT, the number of objects to be tracked is unknown, and there is a measurement-origin uncertainty, i.e., it is unknown which object generated which measurements. Thus, in addition to sequential Bayesian estimation, key aspects of MOT methods also include data association and object track management. Existing work on MOT can be divided into two types, i.e., model-based [ 1 , 2 , 3 , 5 , 4 , 6 , 7 , 8 ] and data-driven [ 9 , 10 ] methods. Fig. 1: Autonomous driving scenario used for performance evaluation. LiDAR measurements, ground truth object states, and object state estimates provided by the proposed method are shown. The orange dashed rectangles indicate the object state estimates and the black rectangles indicate ground truth object states. While both can achieve satisfactory results in various MOT scenarios, each has potential limitations. • Model-based methods face significant challenges using complex shape feature information of objects, e.g., as provided by LiDAR or camera measurements. In particular, it turns out to be challenging to describe this type of feature information using rather simple statistical models. In addition, for tractable modelling and estimation, model-based methods typically assume that objects move independently following Markovian state-transition models. This modelling approach ignores obvious complex interactions of object motion, as, e.g., induced by parallel lanes or intersections in an autonomous driving scenario . • Data-driven MOT methods typically ignore expert knowledge or any physical description of the underlying processes representing object motion and sensing. While this “a priori information” effectively constrains the problem, it can also provide a high degree of interpretability. The absence of any model-based information leads to a lack of interpretability of data-driven MOT methods. Furthermore, most data-driven MOT methods cannot provide any uncertainty quantification of estimation results, which may be limiting in real-world applications. Fig. 2: Block diagram of the proposed hybrid MOT approach that uses a neural network to enhance statistical motion and measurement models. In this paper, we address limitations of model-based and data-driven estimation by integrating neural networks into the statistical model used for Bayesian MOT. Using LiDAR and camera data from an autonomous driving scenario, we demonstrate that the resulting hybrid Bayesian method can improve the robustness and overall performance of MOT. The scenario used for performance evaluation, as well as a high-level block diagram of our approach, are shown in Figs. 1 and 2 , respectively. The proposed method combines the strengths of well-designed and physically motivated statistical models with the ability of neural networks to extract complex feature information and complex interactions of object tracks. This combination is achieved by enhancing the established motion and measurement model of Bayesian MOT by the output of a neural network that has been trained using annotated data. Details on the neural-enhancement of motion and measurement model are provided in what follows. The neural-enhanced motion model is obtained by learning a state-transition function that captures the influence of historical object states and object interactions with neighboring objects by making use of a recurrent neural network with hidden states and a weighting of neighbors inspired by the attention mechanism [ 11 ] . The neural-enhanced motion model is non-Markovian and makes it possible to consider temporal dependencies and spatial relationships between objects in the prediction step of MOT by propagating a high-dimensional hidden state that acts as a sufficient statistic for previous states of the object and objects in its vicinity. Sigma points (SPs) are used to perform the prediction step based on the neural-enhanced motion model efficiently [ 12 , 13 ] . A neural-enhanced measurement model is obtained by enhancing the model-based likelihood ratios used for data association and measurement update. Building on our previous work in [ 14 , 15 ] , our method provides two learnable factors referred to as “affinity factors” and “false positive rejection factors”. The affinity factors complement the affinity information provided by model-based likelihood ratios. In particular, affinity factors act as a surrogate for a statistical model of object size and shapes, which has been identified as notoriously challenging to establish. The false positive rejection factor dynamically adjusts the false positive PDF, individually for each measurement, using shape information extracted from raw sensor data. The neural network that provides these two factors can use different types of information as input, i.e., motion information provided by the prediction step, size measurements provided by the detector, and shape and appearance features learned from raw data . The overall structure of the statistical model is represented by a factor graph [ 16 ] , where certain factors representing motion and measurement model are enhanced by the neural network. Our method follows the belief propagation framework for MOT [ 4 ] combined with sequential Monte Carlo (SMC) techniques [ 17 ] for a fast and accurate computation of marginal posterior PDFs. The number of objects and their states are finally computed based on these marginal posterior PDFs. The proposed method is applied to the nuScenes autonomous driving dataset [ 18 ] and can achieve state of-the-art