Can We Predict Your Next Move Without Breaking Your Privacy?
Arpita Soni 1, Sahil Tripathi 2, Gautam Siddharth Kashyap 3, Manaswi Kulahara 4, Mohammad Anas Azeez 2, Zohaib Hasan Siddiqui 2, Nipun Joshi 5 (🖂), Jiechao Gao 6 · 2025
Read Can We Predict Your Next Move Without Breaking Your Privacy? by Arpita Soni 1, Sahil Tripathi 2, Gautam Siddharth Kashyap 3, Manaswi Kulahara 4, Mohammad Anas Azeez 2, Zohaib Hasan Siddiqui 2, Nipun Joshi 5 (🖂), Jiechao Gao 6 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08843.txt.
Introduction
Arpita Soni and co-authors propose a way to predict where a person will go next without taking their data off their device. Their 2025 paper joins federated learning, which trains models locally, with large language models. Location data is deeply personal, so central storage is a privacy risk.
Their system keeps user data on the device and uses a compact product trick to tap language model skill. It was tested on four public movement data sets drawn from location-sharing services. Scores led the field on each set, the authors report. The system also cut model size and memory use by large shares. The work speaks to app builders who want useful forecasts for maps and transport without pooling raw traces.
Catalog record
| CREATOR | Arpita Soni 1, Sahil Tripathi 2, Gautam Siddharth Kashyap 3, Manaswi Kulahara 4, Mohammad Anas Azeez 2, Zohaib Hasan Siddiqui 2, Nipun Joshi 5 (🖂), Jiechao Gao 6 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-08843
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can We Predict Your Next Move Without Breaking Your Privacy? Arpita Soni 1, Sahil Tripathi 2, Gautam Siddharth Kashyap 3, Manaswi Kulahara 4, Mohammad Anas Azeez 2, Zohaib Hasan Siddiqui 2, Nipun Joshi 5 (🖂), Jiechao Gao 6 Abstract We propose FLLL 3 M — Federated Learning with Large Language Models for Mobility Modeling —a privacy-preserving framework for Next-Location Prediction (NxLP). By retaining user data locally and leveraging LLMs through an efficient outer product mechanism, FLLL 3 M ensures high accuracy with low resource demands. It achieves state-of-the-art results on Gowalla (Acc@1: 12.55, MRR: 0.1422), WeePla-ce (10.71, 0.1285), Brightkite (10.42, 0.1169), and FourSquare (8.71, 0.1023), while reducing parameters by up to 45.6% and memory usage by 52.7%. † † tocauthor: Arpita Soni, Sahil Tripathi, Gautam Siddharth Kashyap, Manaswi Kulahara, Mohammad Anas Azeez, Zohaib Hasan Siddiqui, Nipun Joshi, and Jiechao Gao † † institute: 1 Eudoxia Research University, New Castle, USA 2 Jamia Hamdard, New Delhi, India 3 Macquarie University, Sydney, Australia 4 TERI School of Advanced Studies, New Delhi, India 5 Cornell University, New York, USA 6 Stanford University, CA, USA 1 Introduction The rise of smart devices has made understanding human mobility essential for applications such as navigation, transportation, and urban planning zhou2021self . Next-Location Prediction (NxLP) models feng2018deepmove enhance services like ride-sharing and traffic optimization. However, centralizing mobility data raises serious privacy concerns jeon2021lightmove . Federated Learning (FL) offers privacy by training models on local data micheli2024efficient , but suffers from reduced accuracy and high communication costs gupta2022learning . Large Language Models (LLMs) excel at modeling sequential data vaswani2017attention , yet integrating them into FL remains challenging due to data heterogeneity yuan2023spatio . To address this, we propose FLLL 3 M , a hybrid framework that integrates FL and LLMs via outer product for NxLP. It combines the privacy of FL with the contextual power of LLMs, enabling accurate, privacy-preserving mobility modeling. 2 Related Works Early models like Location2vec shoji2018location2vec and LBSN2vec yang2019revisiting utilize social media data for location prediction but lack generalizability to broader mobility contexts. Word2vec-based models yan2017itdl and CATAPE rahmani2019category focus on location semantics and personal preferences but struggle with data requirements and privacy. Embedding methods such as skip-gram and BERT vaswani2017attention offer strong representation power but require large datasets and high computational cost. Graph-based methods like Deepwalk , Node2vec , and LINE capture structural patterns but ignore privacy concerns. Whereas, mobility prediction focuses on three key tasks: Next-Location Prediction (NxLP) feng2018deepmove , Trajectory User Link (TUL) , and Next Time Prediction (NTP) . NxLP models often struggle with dynamic contexts, TUL faces challenges with scalability and heterogeneity, and NTP methods may generalize poorly to unseen data. 3 Methodology FLLL 3 M comprises three modules: (1) semantic mobility encoding of location-time pairs, (2) federated learning with local transformers sharing encrypted outer product representations, and (3) an LLM that integrates these into a GPT-style model for improved NxLP. 3.1 Preprocessing and Semantic Mobility Encoding Module The pipeline begins with user-specific mobility datasets \mathcal{D}_{i}=\{(l_{t}^{i},\tau_{t}^{i})\}_{t=1}^{T_{i}} , where l_{t}^{i} is the spatial location and \tau_{t}^{i} is the corresponding timestamp. Each data point is first cleaned using noise reduction filters (e.g., median filtering over GPS jitter) and normalized temporally to capture both short-and long-term patterns. To semantically enrich the locations, we employ a context-aware tokenizer (i.e. \Delta -IRIS) micheli2024efficient that maps each tuple into discrete semantic tokens from a location-time vocabulary \mathcal{V} : x_{t}^{i}=\text{Tokenize}(l_{t}^{i},\tau_{t}^{i})\in\mathcal{V} . These tokens are then converted to embeddings: \mathbf{e}_{t}^{i}=\phi_{\text{loc}}(x_{t}^{i})+\phi_{\text{time}}(\tau_{t}^{i}) , where \phi_{\text{loc}},\phi_{\text{time}}:\mathcal{V}\rightarrow\mathbb{R}^{d} are learnable embedding functions. The processed sequences \mathbf{E}_{i}=[\mathbf{e}_{1}^{i},\ldots,\mathbf{e}_{T_{i}}^{i}] are then chunked into sliding windows to capture short-term mobility context. These embedding windows serve as local training inputs for the next module—Federated Learning. 3.2 Federated Learning Module Each client retains its own embedding matrix \mathbf{E}_{i} and locally trains a lightweight transformer encoder f_{\theta} , with architecture inspired by GPT-style autoregressive modeling. At every step t , we predict the next embedding \mathbf{e}_{t+1}^{i} given the sequence prefix \{\mathbf{e}_{1}^{i},\ldots,\mathbf{e}_{t}^{i}\} using masked attention: \mathbf{h}_{t}^{i}=f_{\theta}(\mathbf{e}_{1}^{i},\ldots,\mathbf{e}_{t}^{i})\quad\text{with loss }\mathcal{L}_{i}=\sum_{t=1}^{T_{i}-1}\|\mathbf{h}_{t}^{i}-\mathbf{e}_{t+1}^{i}\|^{2} . Instead of sending full models to a central server, clients compute outer product representations between consecutive embeddings to capture second-order mobility dynamics: \mathbf{O}_{t}^{i}=\mathbf{e}_{t}^{i}\otimes\mathbf{e}_{t+1}^{i}\in\mathbb{R}^{d\times d} . Each \mathbf{O}_{t}^{i} is flattened and encrypted with local differential privacy noise \mathcal{N}(0,\sigma^{2}\mathbf{I}) before being transmitted: \tilde{\mathbf{o}}_{t}^{i}=\text{vec}(\mathbf{O}_{t}^{i}+\mathcal{N}(0,\sigma^{2}\mathbf{I})) . The server performs federated averaging over |\mathcal{U}| users to get the global outer product signal: \bar{\mathbf{o}}_{t}=\frac{1}{|\mathcal{U}|}\sum_{i\in\mathcal{U}}\tilde{\mathbf{o}}_{t}^{i} . This global latent vector encodes spatio-temporal transitions across users and is passed to the third stage for LLM integration. 3.3 Large Language Model Module To interface the federated representation \bar{\mathbf{o}}_{t}\in\mathbb{R}^{d^{2}} with a frozen GPT-style LLM, we first project it into the LLM’s embedding space \mathbb{R}^{d_{\text{LLM}}} using a residual MLP: \tilde{\mathbf{h}}_{t}=\Psi(\bar{\mathbf{o}}_{t})=\mathbf{W}_{1}\cdot\text{GELU}(\mathbf{W}_{0}\cdot\bar{\mathbf{o}}_{t}+\mathbf{b}_{0})+\mathbf{b}_{1} , where \mathbf{W}_{0}\in\mathbb{R}^{d_{1}\times d^{2}},\mathbf{W}_{1}\in\mathbb{R}^{d_{\text{LLM}}\times d_{1}} , and \Psi is kept small to ensure compute-efficiency. The transformed vector \tilde{\mathbf{h}}_{t} is concatenated with standard LLM token embeddings at an intermediate transformer layer l_{k} : \mathbf{z}_{t}^{(l_{k})}=\text{LLM}^{(l_{k})}(\mathbf{z}_{t-1}^{(l_{k})}+\tilde{\mathbf{h}}_{t}) . This injection acts as a semantic conditioning signal, steering the LLM to reason over spatio-temporal mobility patterns. The LLM outputs logits over the next predicted location token: \hat{y}_{t+1}=\text{Softmax}(\mathbf{W}_{\text{out}}\cdot\mathbf{z}_{t}^{(l_{k})}) , where \mathbf{W}_{\text{out}}\in\mathbb{R}^{|\mathcal{V}|\times d_{\text{LLM}}} is the output projection head. Because the LLM is frozen, only \Psi and \mathbf{W}_{\text{out}} are fine-tuned, keeping the model lightweight. Importantly, this outer product alignment strategy enables rich semantic prediction without requiring LLM backpropagation. Each module is tightly coupled, forming an end-to-end pipeline from raw mobility data to federated LLM-enhanced prediction. 4 Experimental Analysis 4.1 Dataset Analysis In our experiments, we utilize four real-world datasets: Gowalla 1 1 1 https://snap.stanford.edu/data/loc-Gowalla.html , WeePlace 2 2 2 Dataset obtained from the corresponding author , Brightkite 3 3 3 https://snap.stanford.edu/data/loc-brightkite.html , and Foursquare 4 4 4 https://sites.google.com/site/yangdingqi/home/foursquare-dataset . These datasets, sourced from location-based social networks, offer rich insights into user mobility and social interactions. Gowalla comprises 6,442,890 check-ins from 196,591 users (Feb 2009–Oct 2010), including user IDs, locations, timestamps, and a social graph with 950,327 edges. WeePlace aggregates 7,658,368 check-ins from 15,799 users across platforms like Facebook Places, Foursquare, and Gowalla, capturing time, location, category, and user metadata. Brightkite includes 4,491,143 check-ins by 58,228 users (Apr 2008–Oct 2010), with fields such as timestamp, location ID, latitude, and longitude. The Foursquare dataset contains 227,428 check-ins in New York and 573,703 in Tokyo (Apr 2012–Feb 2013), enriched with venue categories. To ensure uniformity, we applied a 120-day maximum historical window, filtering out users with fewer than 10 check-ins or venues visited less than 10 times. All datasets were randomly shuffled and split into training, validation, and test sets using a 6:2:2 ratio. 4.2 Hyperparameters For the FLLL 3 M , the following hyperparameters were tuned across the models: the location and time embeddings were set to d=128 , balancing model complexity and training efficiency. The local transformer encoder utilized a hidden size of 256 , with 4 attention heads and 6 layers to capture spatial-temporal dependencies. A learning rate of 10^{-4} was used with the Adam optimizer, which provided stable convergence across different datasets. For differential privacy, the standard deviation of the Gaussian noise was set to \sigma=0.1 , ensuring adequate privacy protection while maintaining model performance. A batch size of 64 was employed during FL, balancing memory usage and model updates. The residual MLP for LLM projection had d_{1}=512 and d_{\text{LLM}}=256 , ensuring smooth integration with the LLM. 4.3 Evaluation