Better Together: Evaluating the Complementarity of Earth Embedding Models

Thijs L van der Plas, Jacob JW Bakermans, Vishal Nedungadi, Gabrielė Tijūnaitytė, Marc Rußwurm, Ioannis N Athanasiadis · 2026

Read Better Together: Evaluating the Complementarity of Earth Embedding Models by Thijs L van der Plas, Jacob JW Bakermans, Vishal Nedungadi, Gabrielė Tijūnaitytė, Marc Rußwurm, Ioannis N Athanasiadis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18667.txt.

Introduction

Thijs van der Plas and five co-authors argue Earth embedding models should be judged together, not alone. Spatially aligned embeddings can be fused for richer detail per location, yet isolated tests miss that gain. They propose a complementarity index: the gain of fused embeddings over the best single model. Testing four models alone, in pairs, and jointly across six tasks, fused embeddings win four of six. Complementarity shifts with task, place, and land-cover scale. The 2026 paper reframes progress: the best gains may come from combinations.

Catalog record

CREATORThijs L van der Plas, Jacob JW Bakermans, Vishal Nedungadi, Gabrielė Tijūnaitytė, Marc Rußwurm, Ioannis N Athanasiadis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43760
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18667

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Better Together: Evaluating the Complementarity of Earth Embedding Models Thijs L van der Plas, Jacob JW Bakermans, Vishal Nedungadi, Gabrielė Tijūnaitytė, Marc Rußwurm, Ioannis N Athanasiadis Abstract Earth embedding models transform Earth observation data into embeddings uniquely tied to locations on the Earth’s surface. These models are typically evaluated in isolation, comparing the downstream task performance across different Earth embeddings. However, spatially aligned embeddings can naturally be fused, providing richer information per location, a capability that isolated evaluations fail to capture. We therefore propose assessing Earth embeddings by their complementarity : the performance gain of fused embeddings over the best single-model baseline. To operationalise this, we introduce an embedding complementarity index applicable to any embedding and task, and evaluate four Earth embedding models (AlphaEarth, Tessera, GeoCLIP, SatCLIP) in isolation, in all pairs, and jointly across six downstream tasks. Fused embeddings outperform the best single model in four out of six tasks, confirming that single-embedding evaluations often underestimate Earth embedding capabilities. Complementarity proves both task- and location-dependent. Further, for a land cover regression task, we find that complementarity is partially determined by the spatial scale of land cover classes. Complementarity reframes Earth embeddings: the greatest future gains may come not from any single Earth embedding model, but from combinations that are better together. 1 Introduction Earth embedding models construct compressed embeddings from rich, multimodal Earth observation data, which can be used for a variety of geospatial downstream tasks ( 19 ) . Compared to foundation model embeddings from other domains, such as vision or language, Earth embeddings have the unique property that any embedding is inherently linked to a location on the Earth’s surface. This trait has turned Earth embeddings into commodities, which are pre-computed once and then retrieved by coordinates alone, bypassing the need for model inference ( 8 ) . As a result of this ease of access, pre-computed Earth embeddings have now been used for a range of downstream tasks in agriculture, environmental sciences and ecology ( 39 ; 24 ; 37 ; 22 ; 3 ) . But while the differences in task performance between Earth embeddings are regularly assessed ( 39 ; 22 ; 3 ) , the complementarity between different Earth embeddings is not. This lack of complementarity assessments might stem from the notion that Earth embedding models are expected to encode all possible information about a location within a single vector. Yet, Earth embedding models are developed with varying training objectives, input data modalities and training datasets. For example, explicit geospatial foundation models map Earth observation images to pixels embeddings, and are trained using masked auto-encoders ( 5 ) or pixel-wise contrastive loss ( 32 ; 9 ) , while implicit neural representation models map coordinates to location embeddings via various contrastive loss objectives ( 35 ; 18 ; 7 ) . This variety of Earth embeddings suggests that each might capture a different “view" of the same location that is partially unique, and consequently they can be complementary sources of information. To test this, different Earth embeddings can effectively be treated as multiple (embedding) modalities and be fused into multi-Earth embeddings. In this work, we argue that Earth embeddings should therefore be evaluated both in isolation and in combination, and we demonstrate that for most tasks, Earth embeddings are better together . Because Earth embeddings are uniquely indexed by their location, i.e. , longitude/latitude coordinates, they are spatially aligned and can naturally be fused to obtain one multi-model representation per location. Across six different downstream tasks, we provide a systematic evaluation of four globally available, pre-computed Earth embeddings: AlphaEarth ( 5 ) and Tessera ( 9 ) (pixel embeddings at 10 m resolution), and GeoCLIP ( 35 ) and SatCLIP ( 18 ) (coordinate-based location embeddings). We find that embedding complementarity is task-specific, and we explore the mechanisms that drive embedding complementarity for a land cover regression task. In summary, our contributions are: • We propose a complementarity-based evaluation framework and index that quantifies the joint performance gain of fused Earth embeddings over the best single-model baseline. • We evaluate the complementarity of four Earth embedding models across six tasks, demonstrating that complementary embeddings substantially improve performance on four out of six tasks. • We show that embedding complementarity varies spatially and is task-dependent, and that for land cover regression it is partially explained by the spatial scale of land cover classes. 2 Related work Multimodal AI models leverage diverse data sources like text, images and depth to perform complex tasks, based off the premise that different modalities can provide complimentary and non-redundant information ( 1 ; 2 ; 26 ; 23 ) . Similarly, in the natural language processing (NLP) domain, cross-lingual alignment recovers divergent linguistic properties from similar semantic concepts ( 28 ) . Multimodal AI has become integral to geospatial AI, as inherent georeferencing facilitates the integration of diverse Earth observation and geospatial data sources. This has led to the widespread development of Earth embedding models that integrate multiple modalities into embedding vectors ( 19 ) . However, while Earth embedding model studies routinely quantify the intra-model complementarity of data modalities that feed into Earth embeddings ( 5 ; 16 ) , as well as one-to-one comparisons with other Earth embedding models ( 9 ; 14 ; 5 ; 16 ) , the complementarity between Earth embeddings is rarely assessed. Yet, preliminary evidence suggests that combining Earth embeddings can be highly effective. For example, 7 show that SatMAE pixel embeddings and SatCLIP location embeddings complement each other on a range of geospatial tasks, and 6 assess both one-to-one comparisons and the complementarity of S2Vec with GeoCLIP, SatCLIP and RS-MaMMUT. Further, location encoders are often used alongside geospatial covariates to boost performance on various geospatial applications, such as species distribution modelling ( 38 ) , estimating daily air quality ( 17 ) , satellite image super-resolution reconstruction ( 29 ) , and land cover/land use prediction ( 31 ) . Additionally, pixel embeddings can similarly provide additional context that improve predictions on geospatial tasks, including for species distribution modelling ( 13 ) and predicting groundwater contamination ( 36 ) . Despite these successful examples, a systematic overview that assesses the complementarity of different explicit and implicit Earth embeddings across a range of tasks is missing. Here, we address this by assessing two (explicit) pixel Earth embeddings and two (implicit) location Earth embeddings across six, diverse geospatial tasks. 3 Data 3.1 Downstream tasks We evaluate six downstream geospatial tasks, described below and summarised in Table 1 . These tasks encompass a wide range of Earth observation challenges, including crop type classification using the CropHarvest dataset ( 33 ) , biomass prediction using MMEarth-Bench ( 10 ) , land cover regression derived from Dynamic World ( 4 ) , bioclimatic variable prediction derived from WorldClim BIO ( 27 ) , population density prediction (Pop.) derived from WorldPop ( https://www.worldpop.org/ ) and distance to nearest road prediction (Dist. road) derived from GRIP4 ( 25 ) . More details regarding the sampling method and preprocessing for each task can be found in Supplementary A.1 . Table 1: Overview of downstream tasks. Task # target variables # locations Source License Crop type Multiclass classification 18 3,596

Read the complete text