GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis

Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis · 2025

Read GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis by Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09598.txt.

Introduction

Angelos Zavras and co-authors release GAIA, a global dataset pairing satellite images with careful text. Vision-language models train mostly on noisy web photos, so they stumble on remote sensing tasks. Existing earth datasets often give little more than date and place as text. Their 2025 paper builds the missing resource at scale.

GAIA holds over two hundred thousand image-text pairs across sensors, resolutions, and regions. The text describes the scene in scientific terms, covering land change, disasters, and other dynamic events. Coverage is balanced across space and time rather than clustered on rich regions. Remote sensing teams get training material that teaches models the vocabulary of the field, from floods to deforestation.

Catalog record

CREATORAngelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09598

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis Abstract The continuous operation of Earth-orbiting satellites generates vast and ever-growing archives of Remote Sensing (RS) images. Natural language presents an intuitive interface for accessing, querying, and interpreting the data from such archives. However, existing Vision-Language Models (VLMs) are predominantly trained on web-scraped, noisy image-text data, exhibiting limited exposure to the specialized domain of RS. This deficiency results in poor performance on RS-specific tasks, as commonly used datasets often lack detailed, scientifically accurate textual descriptions and instead emphasize solely on attributes like date and location. To bridge this critical gap, we introduce GAIA, a novel dataset designed for multi-scale, multi-sensor, and multi-modal RS image analysis. GAIA comprises of 201,005 meticulously curated RS image-text pairs, representing a diverse range of RS modalities associated to different spatial resolutions. Unlike existing vision-language datasets in RS, GAIA specifically focuses on capturing a diverse range of RS applications, providing unique information about environmental changes, natural disasters, and various other dynamic phenomena. The dataset provides a spatially and temporally balanced distribution, spanning across the globe, covering the last 25 years with a balanced temporal distribution of observations. GAIA’s construction involved a two-stage process: (1) targeted web-scraping of images and accompanying text from reputable RS-related sources, and (2) generation of five high-quality, scientifically grounded synthetic captions for each image using carefully crafted prompts that leverage the advanced vision-language capabilities of GPT-4o. We also release an automated processing framework developed for this purpose, enabling the broader research community to generate captions for RS images using the web-crawled image-text data. Our extensive experiments, including fine-tuning of CLIP and BLIP2 models, demonstrate that GAIA significantly improves performance on RS image classification, cross-modal retrieval and image captioning tasks, proving its value as a crucial resource for advancing the field. We make our dataset, automated processing framework and fine-tuned model weights publicly available on our project’s GitHub repository 1 1 1 https://github.com/Orion-AI-Lab/GAIA . I Introduction The Earth’s orbit is teeming with a growing constellation of Earth Observation (EO) satellites, continuously producing an unprecedented volume of diverse and complex information about our planet. According to the latest Copernicus Sentinel Data Access Annual Report 2 2 2 https://sentinels.copernicus.eu/web/sentinel/-/ninth-copernicus-sentinel-data-access-annual-report , Europe’s Copernicus satellite constellation alone features an average daily publication rate of more than 20 TiB of user-level data. More specifically, within the reporting period from January 1, 2023 to October 31, 2023, a total of more than 11.5 million user-level data were published, accounting for a total data volume of 6.04 PiB. To put this into perspective, the 6.04 PiB published during the aforementioned reporting period alone is more than ESA’s entire collection of EO data from the pre-Copernicus era, which amounted to 5.6 PB at the end of 2013. It is evident that this exponential growth in data production has outpaced our capacity for meaningful information extraction, prompting the Remote Sensing (RS) community to adopt Deep Learning (DL) approaches to extract meaningful insights for Earth observation. In this data-rich landscape [ 1 ] , language provides a natural interface to interact with and analyze these vast RS archives. Language provides a compressed representation of reality, efficiently encoding concepts. However, this compression is lossy and language alone lacks vital knowledge of the physical world that can only be acquired through sensory experience. To this end, Vision-Language models (VLMs) offer a solution by leveraging the richness of visual (image) data alongside the compression of textual information. By combining these modalities, VLMs are able to learn abstract representations that encapsulate comprehensive scene understanding. VLMs hold immense potential for the RS domain. By comprehending concepts from both visual and language modalities, VLMs can enable intuitive interfaces for RS data analysis (e.g. image captioning, cross-modal retrieval and visual question answering) or even serve as versatile building blocks for next-generation of RS foundation models. Data is regarded as the cornerstone for VLM training in the computer vision domain, a principle firmly established in the foundation models literature [ 2 ] , and the RS domain is no exception. Existing general purpose VLMs are trained on large amounts of web-crawled data, to encode general knowledge, without emphasizing on specialized domains such as RS and medical imagery, which as a matter of fact demonstrate fundamentally different distributions compared to natural images encountered during pre-training. Perhaps most critically, unlike natural image-text datasets, which can easily reach billion-scale [ 3 , 4 ] , specialized datasets hold a relatively limited scale due to their innate demand for highly targeted knowledge and substantial human labor, rendering it costly and often impractical. The advancements in VLMs have drawn significant attention in the RS field, leading to studies across different applications. While early attempts have shown the potential of applying VLMs to RS, it is still an emerging field with many unsolved challenges [ 5 ] . A challenging issue that impedes the development of VLMs in RS is the lack of large-scale aligned image-text datasets. Existing RSI datasets mostly focus on visual recognition tasks and do not provide language annotations. While there have been some efforts to create RS image-text paired datasets, their scale, quality, and diversity still leave room for improvement. Recent initiatives such as LAION-EO [ 6 ] have demonstrated the presence of RS-related samples within large-scale web-crawled image-text paired datasets like LAION-5B [ 3 ] . However, these samples frequently lack the depth and specificity needed for domain-specific RS applications. They primarily focus or are limited to basic attributes such as date and location, neglecting critical RS-specific semantic information including land cover types, geospatial context, and environmental features. This limits the applicability of these datasets for developing high-quality RS image analysis models, which require precise and context-rich in-domain annotations. Existing datasets for training VLMs in the RS domain suffer from significant limitations that hinder the development of effective in-domain models or the adaptation of existing pre-trained VLMs to RS tasks. To begin with, the predominant reliance on aerial and Very High-Resolution (VHR) commercial satellite imagery in widely used, yet limited-scale, datasets [ 7 , 8 , 9 , 10 ] introduces two primary challenges. First, the proprietary nature of this data restricts accessibility and reproducibility, contrasting with the open-access paradigm promoted by missions like the Copernicus Sentinel. Second, the finer spatial resolution inherent in these datasets, while advantageous for specific applications, creates a domain shift compared to data from non-commercial, often coarser-resolution satellite missions. On the other hand, more recent works exploit existing class-labeled datasets in order to automatically generate synthetic captions reaching up to million-scale capacity. Despite addressing the dataset scale aspect, these methods, similar to existing manually curated datasets, fail to adequately support the development of robust in-domain VLMs capable of generalizing across the diverse landscape of RS applications and data sources, due to their severely limited level of caption detail and lack of diverse, domain-specific vocabulary. To address these challenges, we release GAIA, a dataset of approximately 40,201 RS images, each accompanied by five diverse and informative captions, resulting in 201,005 image-text pairs. Specifically, less that 3% of GAIA’s images are found in compared web-crawled datasets. Moreover, we provide extensive benchmarks for RS image classification and

Read the complete text