Learning Generalized and Flexible Trajectory Models from Omni-Semantic Supervision

Yuanshao Zhu 1 , 2 , 3 ∗ {}^{1,2,3~\ast} , James Jianqiao Yu 4 † {}^{4~{\dagger}} , Xiangyu Zhao 2 † {}^{2~{\dagger}} , Xiao Han 2 , Qidong Liu 2 , Xuetao Wei 1 , Yuxuan Liang 3 † {}^{3~{\dagger}} · 2025

Read Learning Generalized and Flexible Trajectory Models from Omni-Semantic Supervision by Yuanshao Zhu 1 , 2 , 3 ∗ {}^{1,2,3~\ast} , James Jianqiao Yu 4 † {}^{4~{\dagger}} , Xiangyu Zhao 2 † {}^{2~{\dagger}} , Xiao Han 2 , Qidong Liu 2 , Xuetao Wei 1 , Yuxuan Liang 3 † {}^{3~{\dagger}} (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17437.txt.

Introduction

Yuanshao Zhu and co-authors build OmniTraj, a system for searching movement trajectories by meaning, not just by shape. Phones and sensors pour out path data, and finding the right paths fast is a core task in spatial data mining. Older methods lean on similarity scores and cannot take condition-based queries.

Their 2025 paper gives each of four views of a path its own encoder: the raw path, its topology, its road segments, and its regions. The four views are fused into one shared space, so a query can use any mix of them. Tests on two real-world data sets show the design handles large collections well and answers flexible queries that older rigid methods cannot.

Catalog record

CREATORYuanshao Zhu 1 , 2 , 3 ∗ {}^{1,2,3~\ast} , James Jianqiao Yu 4 † {}^{4~{\dagger}} , Xiangyu Zhao 2 † {}^{2~{\dagger}} , Xiao Han 2 , Qidong Liu 2 , Xuetao Wei 1 , Yuxuan Liang 3 † {}^{3~{\dagger}}
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17437

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Generalized and Flexible Trajectory Models from Omni-Semantic Supervision Conference: ; 2025 ACM, SIGKDD Thanks: ∗ Work done during the internship at HKUST(GZ) † Corresponding authors. Yuanshao Zhu 1 , 2 , 3 ∗ {}^{1,2,3~\ast} , James Jianqiao Yu 4 † {}^{4~{\dagger}} , Xiangyu Zhao 2 † {}^{2~{\dagger}} , Xiao Han 2 , Qidong Liu 2 , Xuetao Wei 1 , Yuxuan Liang 3 † {}^{3~{\dagger}} Abstract . The widespread adoption of mobile devices and data collection technologies has led to an exponential increase in trajectory data, presenting significant challenges in spatio-temporal data mining, particularly for efficient and accurate trajectory retrieval. However, existing methods for trajectory retrieval face notable limitations, including inefficiencies in large-scale data, lack of support for condition-based queries, and reliance on trajectory similarity measures. To address the above challenges, we propose OmniTraj , a generalized and flexible omni-semantic trajectory retrieval framework that integrates four complementary modalities or semantics—raw trajectories, topology, road segments, and regions—into a unified system. Unlike traditional approaches that are limited to computing and processing trajectories as a single modality, OmniTraj designs dedicated encoders for each modality, which are embedded and fused into a shared representation space. This design enables OmniTraj to support accurate and flexible queries based on any individual modality or combination thereof, overcoming the rigidity of traditional similarity-based methods. Extensive experiments on two real-world datasets demonstrate the effectiveness of OmniTraj in handling large-scale data, providing flexible, multi-modality queries, and supporting downstream tasks and applications. 1. Introduction The widespread availability of GPS-enabled devices and urban sensing infrastructure has resulted in the generation of vast amounts of spatio-temporal trajectory data, capturing the movement patterns of vehicles, pedestrians, and public urban services ( Wang et al. 2021 ; Chen et al. 2024 ; Deng et al. 2022 ; Zhu et al. 2024b ) . As a fundamental building block for smart city applications, trajectory retrieval enables semantically rich insight into mobility behaviors, ranging from traffic congestion prediction and emergency route planning to location-based recommendations ( Su et al. 2020 ; Chang et al. 2024 ; Fang et al. 2022 ) . For instance, urban planners might analyze trajectories passing through specific road segments to optimize traffic light scheduling, while logistics companies could query delivery routes that intersect designated regions to evaluate operational efficiency ( Su et al. 2013 ; Guo et al. 2020 ; Deng et al. 2011 ; Deng et al. 2024 ) . Figure 1. Comparison of trajectory retrieval methods. Our method can retrieve trajectories according to their different modalities and provides flexible retrieval solutions. Existing approaches to trajectory retrieval focusing on trajectory similarity measures that can be broadly categorized into heuristic methods and deep learning-based methods, face critical limitations in handling complex queries . Heuristic methods, such as Dynamic Time Warping (DTW) ( Müller 2007 ) and Hausdorff Distance ( Alt 2009 ) , focus on computing the similarity between full trajectories based on point-wise alignment. While effective for whole-trajectory matching, they suffer from prohibitive computational costs (e.g., \mathcal{O}(n^{2}) ) time complexity for DTW), especially when applied to large-scale datasets and are limited to supporting only full-trajectory similarity queries. Deep learning-based methods—including t2vec ( Li et al. 2018 ) and TrajCL ( Chang et al. 2023 ) —address efficiency issues by encoding trajectories into compact embeddings ( \mathcal{O}(n) time complexity when retrieving). However, these methods rely on holistic similarity computation and still do not address the support of feature-specific conditions like filtering trajectories that traverse designated road segments. Recent multimodal trajectory studies ( Lin et al. 2024a ; Lin et al. 2023 ) have attempted to integrate ancillary data (e.g., road networks or POIs), but have primarily targeted specific classification or prediction tasks rather than cross-modal retrieval. For example, models inspired by contrastive learning ( Zhou et al. 2024 ; Xu et al. 2024 ) align trajectories with other descriptions, but lack a mechanism for querying trajectories based on spatial semantic combinations. Overall, existing methods suffer from inefficiency in handling large-scale data, tunnel vision on trajectory similarity tasks, and unavailability to support conditional (or modality-based) queries. These restrictions hinder their applicability to real-world scenarios, where queries are inherently multi-faceted and demand both precision and efficiency at city scales. To address these issues, we propose to integrate diverse trajectory modalities, such as trajectory, topology, road segments, and regions, into a unified framework, enabling flexible and modality-supported queries. As shown in Figure 1 , existing methods are limited to matching trajectories as a single modality for trajectory point-wise alignment or using deep learning models for similarity learning. Meanwhile, a multi-modality trajectory query needs to construct full semantic self-supervised models for multiple modalities to realize flexible retrieval based on user-specified conditions. However, two fundamental challenges emerge in developing such a multi-modality retrieval model: (1) Semantic Disentanglement . How to represent trajectories in a modality-specific embedding space (e.g., road segments vs. regions) while maintaining their interrelationships? (2) Flexibility and Efficiency . How can multiple modalities during retrieval dynamically combine without sacrificing query efficiency while ensuring flexibility in supporting a wide range of complex queries? For example, DTW-based methods may retrieve trajectories matching a road segment but ignoring regional constraints, while deep learning models fail to isolate modality-specific features, yielding irrelevant results. Table 1. Comparison of OmniTraj with representative trajectory retrieval/similarity methods. Methods DTW t2vec TrajCL OmniTraj Retrieval Complexity \mathcal{O}(n^{2}) \mathcal{O}(n) \mathcal{O}(n) \mathcal{O}(n) Scalability ✗ ✓ ✓ ✓ Conditional Query ✗ ✗ ✗ ✓ Multi-Modality Support ✗ ✗ ✗ ✓ Application Support ✗ ✗ ✗ ✓ To bridge existing research gaps and address the challenges, we propose OmniTraj , a generalized and flexible omni-semantic trajectory retrieval framework designed to integrate diverse trajectory aspects (e.g., topology, road segments, and regions) into a unified system. Specifically, the primary innovation of the OmniTraj is its ability to handle both spatial and semantic complementary constraints and provide a richer understanding of trajectory data. Firstly, OmniTraj encodes the unique features of each trajectory modality individually, ensuring that the precise description of each modality is preserved during the retrieval process and supports a wide range of applications. The framework then employs a modular design while preserving their interrelationships without sacrificing scalability or query efficiency. By decoupling the encoding process of each modality and dynamically combining them during retrieval, OmniTraj enables flexible and efficient queries. In addition, the encoders for each modality can be used individually to support a wide range of downstream tasks. As highlighted in Table 1 , OmniTraj outperforms traditional methods by flexible and scalable design, and supporting condition-based queries while maintaining high efficiency on large-scale datasets. As a result, OmniTraj addresses the limitations of traditional trajectory similarity measures and provides a powerful solution that combines accuracy, efficiency, and versatility in trajectory retrieval. By enabling the handling of complex queries based on a combination of trajectory, spatial, and semantic information, OmniTraj sets a new perspective for the trajectory retrieval task. In summary, the contributions of our research are as follows: • We formalize the novel task of multimodal trajectory retrieval, enabling condition-based queries that span diverse trajectory semantics. This approach overcomes the limitations of current trajectory similarity measures and provides a fresh insight into how trajectory retrieval tasks can be approached. • We present OmniTraj, the first multimodal trajectory retrieval framework

Read the complete text