GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations

Zeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai · 2025

Read GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations by Zeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16683.txt.

Introduction

Zeping Liu and co-authors present a 2025 arXiv paper on teaching machines geography-aware vision across satellite photos, street photos, and map data. Standard image transformers give patch summaries but no fine detail at exact spots, which geographic work demands. An implicit representation module lets their system query features at any coordinate, aligning views of the same place across sources.

Pretraining is self-supervised and contrastive, needing no labels. Across nine task families and 22 datasets, the model beat leading geo-foundation models and strong generic pretraining recipes. The authors credit the fine-grained geographic alignment for the gains and share the code.

Catalog record

CREATORZeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16683

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations Zeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai Abstract Vision Transformer (ViT) has been widely used in computer vision tasks with excellent results by providing representations for a whole image or image patches. However, ViT lacks detailed localized image representations at arbitrary positions when applied to geospatial tasks that involve multiple geospatial data modalities, such as overhead remote sensing (RS) data, ground-level imagery, and geospatial vector data. Here high-resolution localized representations are vital for modeling geospatial relationships and alignments across modalities. We proposed to solve this representation problem with an implicit neural representation ( INR ) module extending ViT with Neural Implicit Local Interpolation, which produces a continuous RS image representation covering arbitrary location in the RS image. Based on the INR module, we introduce GAIR , a novel location-aware self-supervised learning (SSL) objective integrating overhead RS data, street view (SV) imagery, and their geolocation metadata. GAIR utilizes three factorized neural encoders to project different modalities into the embedding space, and the INR module is used to further align these representations geographically, which are trained with contrastive learning objectives from unlabeled data. We evaluate GAIR across 9 geospatial tasks and 22 datasets spanning RS image-based, SV image-based, and location embedding-based benchmarks. Experimental results demonstrate that GAIR outperforms state-of-the-art geo-foundation models (GeoFM) and alternative SSL training objectives (e.g., MoCo V3 and MAE) that do not use fine-grained geo-aligned spatial representations. Our results highlight the effectiveness of GAIR in learning generalizable geospatial representations across tasks, spatial scales, and temporal contexts. The project code is available at https://github.com/zpl99/GAIR . [orcid=0000-0003-2898-0023] [orcid=0000-0002-4034-7784] [orcid=0009-0004-4728-4458] [orcid=0000-0002-7272-8805] [orcid=0000-0002-7818-7309] † † highlights: GAIR bridges the scale gap between satellite and street-view imagery via a geo-aligned contrastive learning framework. A novel Neural Implicit Local Interpolation (NILI) module enables continuous, coordinate-level alignment across heterogeneous modalities. Pretrained on the new Streetscapes1M dataset, GAIR outperforms existing GeoFMs on 9 downstream tasks. keywords Self-supervised learning ,Implicit neural representation ,Cross-view alignment † † corresponding: Corresponding author 1 Introduction Figure 1 : Overview of the GAIR architecture. The model encodes three modalities: street view image s_{i} , geolocation x_{i} , and remote sensing image r_{i} . ViT-based encoders extract features g(s_{i}) and f(r_{i}) , while a location encoder maps x_{i} to e(x_{i}) . An INR module refines f(r_{i}) into a geo-aligned embedding z_{i}^{(q)} , which is used for contrastive learning with g(s_{i}) and e(x_{i}) . In the geospatial domain, there is a vast amount of unlabeled geospatial datasets such as satellite images, street view images, and user-generated geo-tagged data (e.g., Flickr images, geo-tagged tweets, iNaturalist species images, etc). In contrast, labeled geospatial data is typically scarce and highly imbalanced in terms of spatial, temporal, and class coverage ( Mai et al., 2023a ; Klemmer et al., 2023 ) due to the high cost of data annotation and the specialized domain expertise required. This scarcity of labeled data significantly limits the usage of these multimodal geospatial data in critical geospatial applications such as economic development prediction ( Jean et al., 2016 ) , species distribution modeling ( Mai et al., 2023b ; Cole et al., 2023 ) , crop yield estimation ( Azzari et al., 2017 ; You et al., 2017 ) , urban dynamics monitoring ( Cai et al., 2020 ) , geographic question answering ( Mai et al., 2020a ; Yu et al., 2025 ) , and climate extreme event detection ( Ham et al., 2019 ) . Furthermore, training AI models on these limited labeled datasets constrains their generalizability across space ( Li et al., 2022a ; Li et al., 2023a ; Wu et al., 2024 ) , time, and task ( Cong et al., 2022 ; Mai et al., 2025 ) . Meanwhile, inspired by the recent advancements of language and vision foundation models, geo-foundation models (GeoFMs) ( Mai et al., 2024 ; Hsu et al., 2024 ; Janowicz et al., 2025 ) are developed as task-agnostic pre-trained models to tackle the issue of limited supervision information. However, existing GeoFMs heavily rely on overhead remote sensing (RS) tasks ( Cong et al., 2022 ; Guo et al., 2024 ; Hong et al., 2024 ) , e.g., RS foundation models (RSFMs). Although multiple multimodal GeoFMs ( Fuller et al., 2024 ; Astruc et al., 2024 ) have been developed, they primarily focus on integrating different modalities of overhead RS data (e.g., optical, synthetic aperture radar (SAR), etc.) and/or language modality ( Kuckreja et al., 2024 ; Zhang et al., 2024 ) while overlooking other important geospatial data modalities such as ground-level images and geospatial vector data. This practice significantly limits the spatial reasoning capabilities of these models and hampers their generalizability across different tasks, regions, spatial scales, and temporal contexts. Furthermore, existing GeoFMs mainly rely on conventional language and vision foundation model objectives like temporal or spatial augmented embeddings ( Ayush et al., 2021 ; Stojnic and Risojevic, 2021 ; Guo et al., 2024 ) or image reconstruction ( Cong et al., 2022 ; Sun et al., 2022 ; Jain et al., 2022 ) , without explicitly considering the precise spatial alignment across different data modalities. This lack of explicit geo-alignment mechanism leads to suboptimal results for geospatial applications that require fine-grained spatial understanding. A key challenge for the current RS-based GeoFMs ( Guo et al., 2024 ; Hong et al., 2024 ) and geo-aware SSL approaches ( Ayush et al., 2021 ; Cong et al., 2022 ; Klemmer et al., 2023 ; Mai et al., 2023a ) lies in aligning geospatial data across diverse spatial scales and modeling spatial relations explicitly. For instance, overhead satellite imagery provides a broad spatial contextual view, while street view (SV) images capture fine-grained, ground-level details. This creates an intrinsic one-to-many spatial relationship, where a single 256 \times 256 overhead Sentinel-2 RS image can correspond to hundreds or even thousands of SV images at different locations. To bridge this gap, a common practice to align both modalities is to construct a set of geographically co-located RS-SV image pairs, which has been widely used in cross-view image geolocalization ( Workman et al., 2015 ; Wang et al., 2023b ; Li et al., 2025 ) and cross-view image synthesis ( Regmi and Borji, 2018 ; Wu et al., 2022 ; Ye et al., 2025 ) tasks. However, this strategy is fundamentally limited by resolution constraints, it typically requires very high-resolution RS images (e.g., 15 cm - 1 m spatial resolution) ( Workman et al., 2015 ; Li et al., 2025 ) to match SV images’ spatial scale. In easily accessible medium-resolution RS data (e.g., Sentinel-2), the viewshed of an SV image might collapse to a sub-pixel scale of RS data, making it fundamentally challenging to construct effective co-located image pairs without losing fine-grained spatial correspondences. Furthermore, such paring practice treats RS-SV image pairs as independent training samples and neglects the geospatial relation among them, while relying on neural networks to infer such structure implicitly. This practice will lead to poor geospatial reasoning ability and reduced geographic generalizability. To tackle the above challenges, in this paper, inspired by resolution-agnostic image super-resolution methods such as LIIF ( Chen et al., 2021b ) , we propose Geo-Aligned Implicit Representations, namely GAIR,

Read the complete text