Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models

Letian Gong, Yan Lin, Xinyue Zhang, Yiwen Lu, Xuedi Han, Yichen Liu, Shengnan Guo, Youfang Lin, Huaiyu Wan · 2024

Read Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models by Letian Gong, Yan Lin, Xinyue Zhang, Yiwen Lu, Xuedi Han, Yichen Liu, Shengnan Guo, Youfang Lin, Huaiyu Wan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00823.txt.

Description

Abstract Location-based services (LBS) have accumulated extensive human mobility data on diverse behaviors through check-in sequences. These sequences offer valuable insights into users’ intentions and preferences. Yet, existing models analyzing check-in sequences fail to consider the semantics contained in these sequences, which closely reflect human visiting intentions and travel preferences, leading to an incomplete comprehension.

Catalog record

CREATORLetian Gong, Yan Lin, Xinyue Zhang, Yiwen Lu, Xuedi Han, Yichen Liu, Shengnan Guo, Youfang Lin, Huaiyu Wan
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE113 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00823

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models Letian Gong, Yan Lin, Xinyue Zhang, Yiwen Lu, Xuedi Han, Yichen Liu, Shengnan Guo, Youfang Lin, Huaiyu Wan Abstract Location-based services (LBS) have accumulated extensive human mobility data on diverse behaviors through check-in sequences. These sequences offer valuable insights into users’ intentions and preferences. Yet, existing models analyzing check-in sequences fail to consider the semantics contained in these sequences, which closely reflect human visiting intentions and travel preferences, leading to an incomplete comprehension. Drawing inspiration from the exceptional semantic understanding and contextual information processing capabilities of large language models (LLMs) across various domains, we present Mobility-LLM, a novel framework that leverages LLMs to analyze check-in sequences for multiple tasks. Since LLMs cannot directly interpret check-ins, we reprogram these sequences to help LLMs comprehensively understand the semantics of human visiting intentions and travel preferences. Specifically, we introduce a visiting intention memory network (VIMN) to capture the visiting intentions at each record, along with a shared pool of human travel preference prompts (HTPP) to guide the LLM in understanding users’ travel preferences. These components enhance the model’s ability to extract and leverage semantic information from human mobility data effectively. Extensive experiments on four benchmark datasets and three downstream tasks demonstrate that our approach significantly outperforms existing models, underscoring the effectiveness of Mobility-LLM in advancing our understanding of human mobility data within LBS contexts. 1 Introduction Location-based services (LBS) such as Gowalla, Weeplace, and Foursquare enable users to share and discover location information and nearby services. This results in the collection of extensive human mobility data, often presented in the form of check-in sequences. These sequences record users’ visits to different points of interest (POIs) like restaurants and hospitals at various times, reflecting significant semantics about their intentions and preferences. Analyzing these check-in sequences is crucial as it offers valuable information on human mobility data, which can positively impact individuals, businesses, and urban management. The key to effectively mining check-in sequences lies in understanding their rich semantics. Existing methods primarily focus on specific tasks, such as location prediction [ 10 , 58 , 51 , 25 ] , time prediction [ 39 , 44 ] , and trajectory user linking [ 33 , 13 , 61 ] , rather than delving into the semantics of human behaviors. This narrow focus often results in limited optimization goals and a shallow understanding of the semantics contained in check-in sequences. Recently, large language models (LLMs) have demonstrated impressive capabilities in semantic understanding and contextual information processing, demonstrating successful adaptability across different domains. LLMs trained on extensive corpora surpass task-specific models in their potential to understand semantic information. Inspired by this, we aim to utilize pre-trained LLMs as powerful check-in sequence learners. Nevertheless, LLMs encounter a significant obstacle in their inability to directly interpret check-in sequences. As typical sequential data, check-in sequences contain a wealth of semantic information that reflects various near-term regularities and inherent characteristics. The future intention of an individual is prone to be dictated by near-term regularities that are close to recent visits, termed visiting intentions . Furthermore, an individual’s inherent characteristics tend to persist over time and determine their travel preferences , which is necessary to analyze them across multiple domains for a comprehensive understanding. Hence, our main challenge is to enable LLMs to effectively extract semantics from check-in sequences and comprehensively understand human visiting intentions and travel preferences. To address this challenge, we present a novel unified framework called Mobility-LLM for various check-in sequence analysis tasks. It leverages pre-trained LLMs for general check-in sequence analysis. Our contributions can be summarized as follows: • We propose a unified framework called Mobility-LLM that uses a pre-trained LLM to achieve a SOTA or comparable performance across various check-in analysis tasks including location prediction, trajectory user link, and time prediction. We extract the semantics of check-in sequences to enable LLMs to gain a comprehensive understanding of human visiting intentions and travel preferences. • A visiting intention memory network (VIMN) is proposed for capturing users’ visiting intentions of users at each check-in record by prioritizing relevant check-in records. • A shared pool of human travel preference prompts (HTPP) in different domains is introduced, which enables a comprehensive understanding of human travel preferences and matches appropriate prompts from multiple domains. • Our model’s exceptional performance is validated through extensive experiments on four benchmark datasets involving three tasks. Our robust outcomes in cross-domain pre-training exhibit an average enhancement of 17.8% and an average of 23.6% to 38.3% on the few-shot scenario. 2 Related Works In this section, we provide short reviews of literature in the areas of mobility data mining and cross-domain applications of LLMs. We postpone the detailed discussion of works to the Appendix A , due to limited space. Mobility Data Mining has emerged as a promising research area due to the proliferation of location-based services. This has led to the development of three significant tasks that enhance service quality: next location prediction (LP), next time prediction (TP), and trajectory user link (TUL). The LP task aims to anticipate a user’s future location based on their historical movement. Several notable models have emerged as leading approaches in LP [ 10 , 58 , 50 , 47 , 5 , 22 , 57 ] . The TUL task focuses on establishing connections between different trajectories and users, facilitating the analysis of user movement patterns and uncovering valuable insights into their behavior [ 13 , 7 , 59 ] . The TP task focuses on estimating the time at which a user is likely to visit their next location. Various models have been developed to model the intensity function representing the rate or density of event occurrences, effectively making accurate time predictions [ 56 , 16 , 52 ] . Cross-domain Application of LLMs has gained attention in recent studies, which adopt large language models to address the challenge of limited training data. In the field of time series analysis, LLM4TS [ 4 ] is a pioneering method that aligns pre-trained large language models with temporal characteristics, introducing a two-level aggregation method to effectively incorporate multi-scale temporal data into pre-trained LLMs. One-Fits-All [ 62 ] is a unified framework that leverages a frozen pre-trained language model to attain state-of-the-art or comparable performance across various major types of time series analysis tasks. AutoTimes [ 29 ] facilitates the tokenization of time series into the embedding space of LLMs and intelligently utilizes the inherent token transitions to effectively predict time series in an autoregressive manner. In the field of computer vision, LM4VE [ 34 ] incorporates a frozen transformer block from an LLM as a general-purpose visual encoder layer. To tackle graph-related tasks, TAPE [ 18 ] utilizes semantic knowledge generated by LLMs to enhance the quality of initial node embeddings in GNNs. MoleculeSTM [ 26 ] aligns GNNs and LLMs within a shared vector space, integrating textual knowledge into graphs to enhance reasoning capabilities. 3 Preliminaries POI Visiting Record In the check-in datasets, a user’s visit to a certain place is represented by a POI visiting record R=(L_{p},t) generated by the user u . L_{p} indicates the visited POI at time t . L_{p} is

Read the complete text