Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification

Kumar Manas, Christian Schlauch, Adrian Paschke, Christian Wirth, Nadja Klein · 2025

Read Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification by Kumar Manas, Christian Schlauch, Adrian Paschke, Christian Wirth, Nadja Klein (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13111.txt.

Introduction

Kumar Manas and co-authors presented this 2025 method for forecasting where road users will move next. Self-driving systems need those forecasts, plus honest doubt when the scene is unclear. Deep models often fail on rare scenes and report confidence poorly. Their framework, SHIFT, rebuilds the task as sorting paths into classes rather than drawing one line.

The model separates doubt from lack of data and doubt from plain chance, using a guarded Gaussian process design. It also learns from driving rules written in plain words, such as stop rules, pulled from labels by a language model pipeline. On the nuScenes data set, SHIFT beat prior methods on doubt quality and path error. It did best at crossings, where doubt runs highest.

Catalog record

CREATORKumar Manas, Christian Schlauch, Adrian Paschke, Christian Wirth, Nadja Klein
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-13111

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification Kumar Manas, Christian Schlauch, Adrian Paschke, Christian Wirth, Nadja Klein Abstract Deep learning-based trajectory prediction models have demonstrated promising capabilities in capturing complex interactions. However, their out-of-distribution generalization remains a significant challenge, particularly due to unbalanced data and a lack of enough data and diversity to ensure robustness and calibration. To address this, we propose SHIFT ( S pectral H eteroscedastic I nformed F orecasting for T rajectories), a novel framework that uniquely combines well-calibrated uncertainty modeling with informative priors derived through automated rule extraction. SHIFT reformulates trajectory prediction as a classification task and employs heteroscedastic spectral-normalized Gaussian processes to effectively disentangle epistemic and aleatoric uncertainties. We learn informative priors from training labels, which are automatically generated from natural language driving rules, such as stop rules and drivability constraints, using a retrieval-augmented generation framework powered by a large language model. Extensive evaluations over the nuScenes dataset, including challenging low-data and cross-location scenarios, demonstrate that SHIFT outperforms state-of-the-art methods, achieving substantial gains in uncertainty calibration and displacement metrics. In particular, our model excels in complex scenarios, such as intersections, where uncertainty is inherently higher. Project page: https://kumarmanas.github.io/SHIFT/ . I Introduction Fig. 1 : Visualization of SHIFT’s trajectory prediction framework. Rule-based priors (blue dashed) generate trajectories without interaction modeling, while interaction-aware HetSNGP posteriors (pink solid) produce uncertainty-calibrated ego paths. Surrounding vehicles’ predictions (V1: green dashed, V2: purple dashed) with uncertainty regions (light green/purple shading) reveal multi-agent dynamics. Denser pink stripes (not size-based) highlight where vehicle interactions amplify prediction variance, showing how our model leverages static traffic rules with real-time uncertainty propagation across all actors. Trajectory prediction represents a fundamental challenge in autonomous driving and robotics, serving as a critical bridge between perception and planning systems [ 37 , 22 ] . As illustrated in Fig. 1 , autonomous vehicles must simultaneously reason about multiple possible future trajectories while accounting for road conditions, traffic rules, and multi-agent interactions. This complex decision-making process becomes particularly challenging in urban environments, where the interactions between vehicles, pedestrians, and infrastructure are fast-changing [ 25 , 5 ] . Deep learning approaches for trajectory prediction have demonstrated remarkable potential in tackling these challenges [ 22 ] . Nevertheless, the safety-critical nature of autonomous driving introduces requirements that current state-of-the-art deep learning-based trajectory predictors struggle to address comprehensively. First, models must generalize effectively from limited data, given the high cost of data collection and long-tail distribution of unique scenarios in urban environments. For example, most observed scenarios involve relatively simple behaviors, such as following a straight road, while rare safety-critical events, like near-accidents, are particularly challenging to capture due to their low frequency. Data efficiency of deep learning-based trajectory predictors has only recently gained attention in scientific investigations [ 16 , 56 ] . Second, models must be robust to shifts in road conditions, perception systems, and geographic locations. Social norms and traffic rules provide valuable context for predicting compliant agent behavior [ 7 ] . However, informed or knowledge-guided deep learning-based trajectory predictors have predominantly focused on physical constraints [ 12 , 1 , 27 , 55 ] , leaving broader contextual prior knowledge underexplored [ 53 ] . Third, trajectory prediction involves multiple sources of uncertainty that must be carefully quantified. Epistemic uncertainty arises from model limitations and incomplete training data, while aleatoric uncertainty stems from the stochasticity of agent behavior and environmental dynamics [ 23 ] . Although most trajectory prediction models address the predictive multi-modality resulting from these uncertainties (e.g., by framing it as a classification problem), many fail to provide sufficiently calibrated uncertainty estimates, leading to mode-collapse issues [ 2 ] . To address these requirements, we propose SHIFT ( S pectral H eteroscedastic I nformed F orecasting for T rajectories) to synergize uncertainty quantification with rules as soft constraints in a scalable framework as visualized in Fig. 2 : 1. We extend the Heteroscedastic Spectral-normalized Gaussian Processes (HetSNGP) by Fortuin et al. 2022 to the trajectory prediction using the CoverNet baseline architecture [ 39 ] . CoverNet frames the prediction as a classification problem. HetSNGPs enable simultaneous estimation of epistemic and aleatoric uncertainties through a distance-aware neural GP layer combined with input-dependent noise modeling, improving calibration and requiring minimal architectural change and computational overhead. 2. We model epistemic uncertainty while accounting for the inherent stochasticity of agent interactions through aleatoric uncertainty, enabling a sequential learning process for informative prior weight distributions. In stage 1, we encode traffic rules as synthetic training labels, capturing structured prior knowledge. These learned traffic rule priors are then imposed as regularization during stage 2, guiding model training on real-world observations. This approach soft-constrains trajectory predictions to align with traffic rules while preserving the flexibility to adapt to high-uncertainty and uncommon driving scenarios. 3. We employ a large language model (LLM) pipeline to semi-automatically generate synthetic training labels from natural language descriptions. This approach allows us to more easily scale the integration of multiple traffic rules, either as (a) a single “unified” prior learned from a single knowledge task or (b) a “chained” prior sequentially learned from multiple knowledge tasks in our first training stage. SHIFT increases data efficiency and robustness by integrating sets of rules into the training process. Through extensive experiments on the nuScenes dataset [ 6 ] , we demonstrate robust performance gains across varying training-set sizes (100% vs. reduced data) and challenging geographical splits (cross-location tests). Our results show particular strength in out-of-distribution scenarios, where the combination of uncertainty awareness and rule-based soft constraints provides more reliable predictions than conventional training methods. Fig. 2 : Concept of the SHIFT framework for uncertainty-aware trajectory prediction. Our model is based on the CoverNet architecture, which frames the prediction as trajectory anchor classification. The backbone is spectral-normalized, and the output layer is replaced by a heteroscedastic Gaussian process. Given the processed input, comprising trajectory histories and map features, the model is trained in a multilabel classification on traffic rule labels generated from an LLM-based rule-filtering approach (stage 1). The obtained informative prior is used to regularize the multi-class classification training on ground truth labels from the observed data (stage 2). The model outputs consist of the predicted trajectories and disentangled epistemic and aleatoric uncertainty estimates. Best viewed in color. II Related Work Trajectory Prediction Models: Existing models can be differentiated based on whether they predict the future behavior of all agents simultaneously (joint trajectory prediction) or a single agent at a time (marginal trajectory prediction), with most work focusing on the simpler latter problem [ 22 , 54 ] . Deep learning approaches have achieved state-of-the-art performance in in-distribution evaluation settings, surpassing traditional methods such as kinematic models [ 48 ] , Kalman filters and Bayesian Networks + [ 46 ] . Building on the work of Hagedorn et al. 2023 , deep learning-based trajectory predictors can be categorized based on their input representation, model backbone, and trajectory decoding methods. Input representations include birds-eye-view (BEV) image rasterizations [ 39 ] , sparse vector encodings [ 50 ] , graphs [ 57 ] , and parametric curves [ 55 ] . Model backbone encompass convolutional neural networks (CNN) [ 39 ] , recurrent neural networks [ 15 ] , graph neural

Read the complete text