ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving Conference: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 25–29, 2024; Barcelona, Spain Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD
Rongqing Li, Changsheng Li, Yuhang Li, Hanjie Li, Yi Chen, Ye Yuan, Guoren Wang · 2024
Read ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving Conference: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 25–29, 2024; Barcelona, Spain Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD by Rongqing Li, Changsheng Li, Yuhang Li, Hanjie Li, Yi Chen, Ye Yuan, Guoren Wang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07369.txt.
Introduction
Rongqing Li and co-authors predict vehicle paths from just two sightings in a 2024 paper. Self-driving cars must forecast where nearby agents will go, but classic predictors need long observed tracks. Real streets deny that luxury: a car can pop out from behind a truck, close and fast. Prediction then has to work from almost nothing.
Their plug-in method, ITPNet, first runs time backward. From two observed positions it imagines the unobserved past track, in feature form. Those imagined features then enrich the forecast of the future path. Noise and redundancy in the imagined history are handled inside the design. The approach targets the sudden-appearance cases where a slow, data-hungry predictor would fail outright.
Catalog record
| CREATOR | Rongqing Li, Changsheng Li, Yuhang Li, Hanjie Li, Yi Chen, Ye Yuan, Guoren Wang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-07369
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving Conference: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 25–29, 2024; Barcelona, Spain Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’24), August 25–29, 2024, Barcelona, Spain DOI: 10.1145/3637528.3671681 ISBN: 979-8-4007-0490-1/24/08 CCS: Applied computing Forecasting CCS: Computing methodologies Machine learning CCS: Computing methodologies Spatial and physical reasoning Rongqing Li, Changsheng Li, Yuhang Li, Hanjie Li, Yi Chen, Ye Yuan, Guoren Wang Abstract . Trajectory prediction of moving traffic agents is crucial for the safety of autonomous vehicles, whereas previous approaches usually rely on sufficiently long-observed trajectory (e.g., 2 seconds) to predict the future trajectory of the agents. However, in many real-world scenarios, it is not realistic to collect adequate observed locations for moving agents, leading to the collapse of most prediction models. For instance, when a moving car suddenly appears and is very close to an autonomous vehicle because of the obstruction, it is quite necessary for the autonomous vehicle to quickly and accurately predict the future trajectories of the car with limited observed trajectory locations. In light of this, we focus on investigating the task of instantaneous trajectory prediction, i.e., two observed locations are available during inference. To this end, we put forward a general and plug-and-play instantaneous trajectory prediction approach, called ITPNet. Specifically, we propose a backward forecasting mechanism to reversely predict the latent feature representations of unobserved historical trajectories of the agent based on its two observed locations and then leverage them as complementary information for future trajectory prediction. Meanwhile, due to the inevitable existence of noise and redundancy in the predicted latent feature representations, we further devise a Noise Redundancy Reduction Former (NRRFormer) module, which aims to filter out noise and redundancy from unobserved trajectories and integrate the filtered features and observed features into a compact query representation for future trajectory predictions. In essence, ITPNet can be naturally compatible with existing trajectory prediction models, enabling them to gracefully handle the case of instantaneous trajectory prediction. Extensive experiments on the Argoverse and nuScenes datasets demonstrate ITPNet outperforms the baselines by a large margin and shows its efficacy with different trajectory prediction models. 1. Introduction Predicting the future trajectories of dynamic traffic agents is a critical task for autonomous driving, which can be beneficial to the downstream planning module of autonomous vehicles. In recent years, many trajectory prediction methods have been proposed in computer vision and machine learning communities ( Wang et al. 2023 ; Park et al. 2023 ; Zhou et al. 2023 ; Zhu et al. 2023 ; Chen et al. 2021 ; Gu et al. 2022 ; Xu et al. 2022 ; Wang et al. 2022a ; Meng et al. 2022 ) . Among these methods, they usually need to collect sufficiently long observed trajectories (typically, 2 to 3 seconds) of an agent, in order to accurately predict its future trajectories. Recent advances have shown promising performance in trajectory prediction by learning from these adequate observations. However,in real-world self-driving scenarios, it is often difficult to accurately predict trajectories due to the limited availability of observed locations. For instance, due to the obstruction, a moving car might suddenly appear and be very close to the autonomous vehicle. At this moment, the autonomous vehicle does not have enough time to collect adequate observed locations of the car to accurately predict the vehicle’s future trajectories. Such a case will cause the collapse of the aforementioned prediction models due to the lack of information. To verify this point, we perform a typical trajectory prediction method, HiVT ( Zhou et al. 2022 ) , with different settings on the Argoverse dataset ( Chang et al. 2019 ) . As shown in the left part of Figure 1 (a), if we use 20 observed locations as the inputs of the prediction model during both training and test phases as in ( Zhou et al. 2022 ) , the prediction results are 0.698 and 1.053 in terms of minADE@6 and minFDE@6, respectively. However, if we set only 2 observed locations as the inputs of the model during testing, the model will degrade sharply, no matter if the number of observed locations is 2 or 20 during the training phase. Thus, it is essential to study the trajectory prediction task, when observed locations are very limited. Figure 1. (a) Results of HiVT ( Zhou et al. 2022 ) in terms of minADE@6 and minFDE@6 on the validation set of Argoverse ( Chang et al. 2019 ) with different observed locations as inputs during training and testing. The value in the horizontal axis denotes the number of observed locations. (b) Future predictions (shown in green) when utilizing different lengths of predicted unobserved trajectory locations. The observed trajectories are shown in orange, the predicted unobserved trajectories are shown in brown, the ground-truth unobserved trajectories are shown in blue, and the ground-truth future trajectories are shown in red. In light of this, we focus on studying the task of instantaneously predicting future trajectories of moving agents, under the assumption of only 2 trajectory locations available. Recently, The work in ( Sun et al. 2022b ) proposes a trajectory prediction method based on momentary observations. However, this method mainly focuses on the trajectory prediction of pedestrians, which has not been explored for other moving agents. In addition, the input of their model is the RGB image which usually contains abundant context and semantic information. Thus, it is much easier for the model to predict future trajectories using RGB images, compared to only several discrete trajectory locations. Moreover, ( Monti et al. 2022 ) design a knowledge distillation mechanism based on limited observed locations and achieves promising results. Since the method needs to pre-train a teacher model, and learns a student model distilling knowledge from the teacher model, which largely increases the computational complexities. To this end, we propose a general and principled approach, called ITPNet, for instantaneous trajectory prediction by only two observed trajectory locations. Specifically, ITPNet aims to train a predictor to backwardly predict the latent feature representations of unobserved historical trajectory locations of the agent based on its two observed trajectory locations. The additional information contained in the predicted unobserved trajectory features assists observed trajectory features in better predicting future trajectories. Nevertheless, we find that as we increase the number of backwardly predicted unobserved trajectory locations, the model’s performance initially improves but subsequently deteriorates (This is verified in Table 3 of Experiment section). We analyze two primary factors that impede the utilization of more unobserved trajectory features: One is the noise brought by inaccurate prediction of the unobserved trajectory features. The other is a negative impact on the trajectory prediction due to the intrinsic redundant information. Let’s consider a scenario where a vehicle travels straightly for a while and then suddenly executes a turn. In such a case, a longer historical trajectory may erroneously boost the model’s confidence in the vehicle continuing straight in the future, as depicted in the upper portion of Figure 1 (b). Conversely, a shorter unobserved historical trajectory with less redundancy tends to yield more accurate predictions because it maintains lower confidence in the vehicle’s persistence in a straight trajectory and,