TrajGANR : Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations
Maria Despoina Siampou, Gengchen Mai, Ni Lao, Jinmeng Rao, Neha Arora, Cyrus Shahabi, Shushman Choudhury · 2026
Read TrajGANR : Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations by Maria Despoina Siampou, Gengchen Mai, Ni Lao, Jinmeng Rao, Neha Arora, Cyrus Shahabi, Shushman Choudhury (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06990.txt.
Introduction
Maria Despoina Siampou and six co-authors argue that city prediction needs movement, not just maps and images. Congestion, travel demand, and road safety depend on how people actually travel through a place. Trajectory data carry that signal, but sparse GPS points rarely line up with street-view images.
Their framework, TrajGANR, turns each trajectory into a continuous field that can be queried at any location along the path. That lets mobility data align with images taken off the path itself. Across four road-level tasks in San Francisco and Porto, it beats recent geospatial and trajectory models. Its location encoder also works where no imagery or trajectories exist.
Catalog record
| CREATOR | Maria Despoina Siampou, Gengchen Mai, Ni Lao, Jinmeng Rao, Neha Arora, Cyrus Shahabi, Shushman Choudhury |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.06990
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TrajGANR : Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations Maria Despoina Siampou, Gengchen Mai, Ni Lao, Jinmeng Rao, Neha Arora, Cyrus Shahabi, Shushman Choudhury Abstract Multimodal self-supervised learning (MSSL) has emerged as a key paradigm for pretraining geospatial foundation models (GeoFMs). However, existing geospatial MSSL methods are mainly designed for static pairs of modalities, such as satellite imagery, street-view imagery, and text, where learning is driven by aligning observations from the same or nearby locations. This assumption breaks down for human mobility trajectories , which represent continuous movement along paths rather than discrete observations at individual locations. Although trajectories are important for urban understanding through their ability to capture human activity across roads, neighborhoods, and places over time, they remain largely underexplored in current geospatial MSSL frameworks. We present TrajGANR , a novel trajectory-centric geospatial MSSL framework that aligns continuous movement patterns with static, location-based observations. TrajGANR learns a continuous neural representation of trajectories at arbitrary points along each path , which enables fine-grained alignment with nearby street-view images, even when they are not co-located with any trajectory waypoints. We leverage this capability to introduce an MSSL objective that jointly aligns three modalities: trajectories, street-view images, and their geographic locations. We evaluate TrajGANR on four urban mobility and road understanding tasks, including average traffic speed prediction, road popularity prediction, area-of-interest function prediction, and hard braking event prediction. Across these tasks, TrajGANR consistently outperforms existing geospatial MSSL frameworks and a trajectory-specific foundation model. Ablation studies further demonstrate that our proposed MSSL objective and the multimodal learning framework are the primary drivers of these improvements, highlighting the importance of fine-grained geospatial alignment over coarser aggregation, as well as geospatial multimodal learning. 1 Introduction Multimodal self-supervised learning (MSSL) is a fundamental paradigm for learning transferable representations from heterogeneous data sources. By aligning information across modalities using large-scale unlabeled datasets, models such as CLIP ( 40 ) , SigLIP ( 58 ) , and ImageBind ( 17 ) have established a new standard for cross-modal representation learning. This success has extended to geospatial artificial intelligence (GeoAI), where modeling the physical world requires integrating heterogeneous data modalities, such as remote sensing imagery, street-view imagery, vector data, and trajectories ( 28 ) . In this context, recent geospatial MSSL frameworks ( 5 ; 21 ; 26 ) have highlighted the value of learning task-agnostic representations across complementary geospatial modalities, especially in settings where dense annotations are limited or costly to obtain ( 11 ; 50 ; 28 ) . Figure 1: Motivation for fine-grained alignment: Street-view images (SVIs) can lie along a trajectory path without co-locating with any of the trajectory’s sparse waypoint samples. The red line denotes one trajectory whose waypoints are highlighted as yellow dots. The diamond-shaped dots along this trajectory represent locations of different SVIs that are not colocated with any waypoints. The blue diamond-shaped dots are examples of SVIs whose images are shown. Despite this progress, current geospatial MSSL frameworks are largely designed for modalities that can be represented either as discrete, location-indexed samples or as regular spatial grids. This design limits their ability to incorporate continuous and dynamic signals, particularly human mobility trajectories , into multimodal pretraining. Existing contrastive pretraining objectives typically define positive pairs through co-location or spatial proximity. This strategy is well-suited to static modalities such as satellite imagery, street-view imagery, and location-based observations, where each sample is tied to a fixed geographic coordinate and can be aligned with observations from the same or nearby locations 31 ; 21 ; 5 ; 63 ; 3 ; 44 . Similarly, reconstruction-based objectives, such as masked autoencoding and autoregressive modeling, have been effective for remote sensing imagery 12 ; 35 ; 32 ; 18 ; 15 ; 10 and weather fields 34 ; 65 ; 2 ; 45 , whose inputs naturally form spatial grids or patches. Human mobility trajectories, however, do not naturally fit either paradigm, as they are continuous, regularly or irregularly sampled, and encode movement along paths rather than observations at isolated locations or values on a fixed grid. Simply reducing trajectories to discrete locations or grid cells would discard the path structure and localized movement context that make them informative. Consequently, existing geospatial MSSL objectives do not provide the necessary mechanisms for aligning trajectories with static urban observations at a fine-grained spatial granularity needed for trajectory-aware representation learning. Incorporating trajectories into geospatial MSSL frameworks is critical since they can enrich learned representations with dynamic behavioral signals that are largely absent from static geospatial modalities, as demonstrated by Zhao et al 62 and Siampou et al 46 . Mobility traces reveal how roads are used, where travel demand and congestion emerge, and how urban functions are connected through movement ( 9 ; 47 ) . Yet, integrating trajectories into geospatial MSSL frameworks is non-trivial. Existing trajectory encoders typically produce single representations of the whole trajectories or discrete sequences of trajectory GPS waypoint embeddings 68 ; 7 ; 64 ; 67 . But aligning trajectories with location observations such as street-view images requires trajectory embeddings localized where those observations occur. As illustrated in Figure 1 , a trajectory represents continuous movement in space and time, even though it is observed through sparse GPS waypoints (denoted as yellow dots), and street-view image (SVI) observations may lie along the trajectory path without co-locating with any of these waypoints. Achieving this requires a neural representation that preserves the continuous path structure of trajectories while supporting alignment with location observations . To this end, we propose TrajGANR , a trajectory-centric geospatial MSSL framework that integrates human mobility trajectories with street-view imagery and associated geographic locations. TrajGANR represents each trajectory as a path-conditioned neural implicit function that can be queried at arbitrary geographic positions along the trajectory. Rather than aligning a street-view image representation to the nearest trajectory waypoint embedding or to a coarse trajectory-level representation, TrajGANR uses the image location as a query point along the path and infers a localized trajectory embedding contextualized by the surrounding movement. Building on this capability, we introduce a novel MSSL objective that aligns each localized trajectory embedding with the representations of the street-view image and the geographic location associated with the same query point. Together, these components enable fine-grained geospatial multimodal alignment between continuous trajectories and static urban observations. We evaluate TrajGANR on four downstream tasks that highlight the value of incorporating trajectories into multimodal frameworks for road and urban understanding: average traffic speed prediction, road popularity prediction, area-of-interest function prediction, and hard braking event prediction. Across these tasks, TrajGANR consistently outperforms recent geospatial MSSL frameworks that do not incorporate trajectories and a unimodal trajectory foundation model 68 . Ablation studies further show that these performance gains are driven by two major drivers – the proposed fine-grained geospatial alignment MSSL objective and the geospatial multimodal learning framework for downstream tasks. We find that, compared with a model variant that skips multimodal alignment altogether and a coarse-level alignment variant based on road-segment co-location, TrajGANR better captures localized interactions between aggregate movement patterns, street-view imagery, and geographic location. This improvement highlights the importance of fine-grained alignment for continuous signals into multimodal geospatial representation learning. Moreover, ablation studies also show that our proposed geospatial multimodal learning framework