Geospatial Foundational Embedder: Top-1 Winning Solution on EarthVision Embed2Scale Challenge (CVPR 2025)
Zirui Xu*, Raphael Tang*, Mike Bianco*, Qi Zhang, Rishi Madhok, Nikolaos Karianakis, Fuxun YuMicrosoftRedmond, WA, {v-ziruixu, v-raptang, v-mikebianco, rishi.madhok, nikarian, fuxunyu}@microsoft.com · 2025
Read Geospatial Foundational Embedder: Top-1 Winning Solution on EarthVision Embed2Scale Challenge (CVPR 2025) by Zirui Xu*, Raphael Tang*, Mike Bianco*, Qi Zhang, Rishi Madhok, Nikolaos Karianakis, Fuxun YuMicrosoftRedmond, WA, {v-ziruixu, v-raptang, v-mikebianco, rishi.madhok, nikarian, fuxunyu}@microsoft.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06993.txt.
Introduction
Zirui Xu and 6 co-authors describe the winning answer to a geospatial embedding challenge in a 2025 technical report. The EarthVision Embed2Scale challenge at CVPR 2025 aims to develop foundational geospatial models that embed hyperspectral geospatial data cubes, stacks of satellite imagery across many wavelengths, into embedding vectors. Such vectors should facilitate various downstream tasks, including sorting and regression.
Their new method, called the Geospatial Foundational Embedder, was the top-ranked winning answer on the challenge, according to the report. The work is presented as a technical report introducing the method rather than a full experimental study. It addresses data from the SSL4EO-S12 data set, a standard self-guided learning benchmark for Earth reading. Winning a shared challenge gives evidence the approach compared favorably against other submitted foundational embedding methods.
Catalog record
| CREATOR | Zirui Xu*, Raphael Tang*, Mike Bianco*, Qi Zhang, Rishi Madhok, Nikolaos Karianakis, Fuxun YuMicrosoftRedmond, WA, {v-ziruixu, v-raptang, v-mikebianco, rishi.madhok, nikarian, fuxunyu}@microsoft.com |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06993
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geospatial Foundational Embedder: Top-1 Winning Solution on EarthVision Embed2Scale Challenge (CVPR 2025) Zirui Xu*, Raphael Tang*, Mike Bianco*, Qi Zhang, Rishi Madhok, Nikolaos Karianakis, Fuxun YuMicrosoftRedmond, WA, {v-ziruixu, v-raptang, v-mikebianco, rishi.madhok, nikarian, fuxunyu}@microsoft.com Abstract EarthVision Embed2Scale challenge (CVPR 2025) aims to develop foundational geospatial models to embed SSL4EO-S12 hyperspectral geospatial data cubes ssl4eo into embedding vectors that faciliatetes various downstream tasks, e.g., classification, regression, etc. In this technical report, we introduce our proposed method for the Top-1 winning solution on the Embed2Scale Challenge. 1 Introduction The ability to effectively extract and analyze geospatial information from satellite images is essential for developing solutions that address global challenges including climate change, urbanization, disaster response. Geospatialtial embedding model tackles such tasks by transforming complex geospatial data into meaningful vector representations, facilitating a wide range of applications such as fine-grained scene classification and regression to enable rapid downstream finetuning and response. To facilitate that, the EarthVision Embed2Scale competition in CVPR 2025 aimed to advance the field of geospatial embedding by focusing on developing models capable of effectively compressing information from diverse-seasonal, hyperspectral imagery to improve predictive accuracy and robustness in downstream applications, as shown in Figure 1 . Figure 1 : SSL4EO dataset ssl4eo : Each embedding target in the challenge is one novel 4-season hypersepctral geospatial data cube with 4 x 17 = 128 channels. In this competition, we pretrained a series of foundational geospatial models on the given SSL4EO dataset and other external datasets (as we will introduce later). Benefited from the model quality, our embedding quality achieved the 1st place in the various downstream tasks, which underscores the effectiveness of our approach and its potential impact on the field of geospatial learning. In summary, our main contributions are as follows: • State-of-the-Art Pretraining Models: We adopt two families of powerful pretraining methods: CLIP clip , DINO dino . With adapted input channel numbers, we finetuned them on the EO4SSL dataset; • Geospatial Semantic Enrichment: By incorporating multiple auxiliary regression signals torchspatial such as elevation, forest coverage, population and nightlights, we enhance the holistic geospatial understanding and predictive capabilities of our models; • Geospatial Natural Language Supervision: Our approach uniquely adopts natural-language supervised geospatial CLIP georsclip , leveraging their rich semantic strengths to enhance overall embedding performance; • SVD-based Compressive Ensemble: We design an SVD + concat method to reduce dimensionality while preserving critical information, and compose the final embeddings through four distinct models, enabling more diverse representation capability. Table 1 : The Overview of Proposed Embedding Composition Solution. *: same model. Embedding Model Arch Input Dimension Training Dataset Training Method [0:128] ConvNeXt-XXL 128-channel SSL4EO + TorchSpatial CLIP w/ lat, lon, four regression targets [128:384] ViT-Huge 128-channel SSL4EO CLIP w/ lat, lon [384:512] ViT-Base 128-channel SSL4EO DINO [512:640] ViT-Huge* 3-channel RGB [Spring] RS5M GeoRSCLIP w/ natural language captions [640:768] ViT-Huge* 3-channel RGB [Summer] RS5M GeoRSCLIP w/ natural language captions [768:896] ViT-Huge* 3-channel RGB [Fall] RS5M GeoRSCLIP w/ natural language captions [896:1024] ViT-Huge* 3-channel RGB [Winter] RS5M GeoRSCLIP w/ natural language captions 2 The Winning Solution 2.1 CLIP and DINO Finetuning Contrastive Language-Image Pre-training (CLIP) has revolutionized multimodal learning by enabling the effective alignment of image and text embeddings through contrastive learning objectives. Complementarily, DINO is a powerful self-supervised vision training framework. Unlike CLIP’s reliance on paired textual descriptions, DINO utilizes self-distillation principles, encouraging a student network to emulate outputs from a momentum-updated teacher network on augmented views of the same image. This method excels at learning discriminative image embeddings without explicit annotations, resulting in robust representations to capture subtle visual distinctions. In our solution, we finetuned two powerful foundational model architectures from CLIP off-the-shelf pre-trained weights: ConvNext-XXLarge and ViT-H. We intentionally choose one Convolutional Neural Network (CNN) and one Vision Transformer (ViT) architecture to leverage their inductive bias differences: convolution spatial invariance v.s. transformer’s global attention mechanisms. For DINO method, we finetuned a smaller model, ViT-Base, due to the limitation from the off-the-shelf pretrained weights availability. All models’ first layer are modified from 3-channel to 128-channel, and we duplicate the 3-channel weights to 128-channel as well to match the new layer shape. 2.2 Geospatial Semantics Enrichment For the two aforementioned methods, DINO is a image-level self-contrasting method and can be hard to fuse further supervision information into training. Therefore, we leave the training setting unmodified for DINO. For CLIP, on the other hand, we considered multiple methods to enrich the geospatial semantics in both pretraining / finetuning stages. • Geospatial Location Semantics : The training data cubes from EO4SSL dataset comes with <lat,~lon> metadata. We adopt the numerical values and formatted as string: “ Latitute: lat, Longtitute: lon ” as one part of the supervision to the CLIP language encoder. • Geospatial Regression Semantics : We also collect four geospatial semantical information for the training data cubes based on their <lat,~lon> metadata. Specifically, we sourced the following four attributes from TorchSpatial torchspatial : Forest Cover, Elevation, Nightlights, and Population Density. And we format the string in the same way: “ Forest Cover: x1, Elevation: x2, Nightlights: x3, Population: x4 ” as the other part of the supervision to the CLIP language encoder. • Natural Language Semantics : Natural language supervision is hard to get for the given SSL4EO dataset. Therefore, we adopt a natural language geospatial model, GeoRSCLIP georsclip , that is pretrained on the RGB-based RS5M dataset. We use the model as-is on the Sentinel-2 L2A RGB channels of the SSL4EO data cubes for each of the four seasons separately. 2.3 GeoRSCLIP Unsupervised Finetuning During the development phase, we hypothesized that more discriminative, linearly separable features would improve end-task effectiveness because the test-time models were either linear or logistic regression, specifically without a bias term. Past work has also noted increased linear separability to improve out-of-distribution generalization for graphical embeddings baranwal2022graph . Thus, we constructed a set of pseudolabels using agglomerative clustering over the four-season concatenated GeoRSCLIP embeddings. Then, we applied a linear map (i.e., a square matrix) tied between the four seasons’ embeddings and fitted a linear model without a bias term (representing the test-time setup) on the subsequent linearly mapped embeddings to predict the pseudolabels. The intuition was that a full-rank square matrix should distort the embedding space to improve the “learnability” for linear models without overfitting to our limited dataset. Both the linear map and linear model were optimized end-to-end on the development set, and we used the fully optimized linear map to transform and produce our final GeoRSCLIP embeddings. 2.4 Compressive Ensemble Ensemble distinct models to leverage their different characteristics is the key to enhance the final embedding quality. However, each off-the-shelf model’s embedding size are very large: ConvNeXt-XXL / ViT-H has a final 1024 embedding size, and ViT-B has a 768 embedding size, etc. Therefore, we investigate different compressive ensemble methods. Various of methods are evaluated such as post-trained auto-encoder with different MSE / Lp-norm losses, or traditional PCA / SVD algorithms, etc. We empirically found the classic SVD algorithm tends to have less performance impact for the downstream tasks. Therefore, we adopt the SVD+Concat based compressive ensemble method on top of the different models’ embeddings. Table 1 lists the final models we used to compose the final embedding vector. Specifically, we have four models: •