Learned Image Compression for Earth Observation: Implications for Downstream Segmentation Tasks
Christian Mollière, Iker Cumplido, Marco Zeulner, Lukas Liesenhoff, Matthias Schubert, Julia Gottfriedsen · 2025
Read Learned Image Compression for Earth Observation: Implications for Downstream Segmentation Tasks by Christian Mollière, Iker Cumplido, Marco Zeulner, Lukas Liesenhoff, Matthias Schubert, Julia Gottfriedsen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01788.txt.
Introduction
Christian Molliere and co-authors test learned image compression for Earth observation with segmentation in mind. Satellite data volumes grow fast. Sending and storing it all strains links and archives. Learned compression can shrink files a lot, but it may drop details that later tasks need.
The team scores compression by downstream segmentation quality. Maps of land, water, and crops are the end use. A codec that keeps segmentation sharp while files shrink wins. That trade-off decides if learned codecs can serve real satellite pipelines. Segmentation-aware training bakes the end task into the codec, a shift from generic fidelity scores that often mislead archive planning.
Catalog record
| CREATOR | Christian Mollière, Iker Cumplido, Marco Zeulner, Lukas Liesenhoff, Matthias Schubert, Julia Gottfriedsen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01788
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learned Image Compression for Earth Observation: Implications for Downstream Segmentation Tasks Christian Mollière, Iker Cumplido, Marco Zeulner, Lukas Liesenhoff, Matthias Schubert, Julia Gottfriedsen Abstract The rapid growth of data from satellite-based Earth observation (EO) systems poses significant challenges in data transmission and storage. We evaluate the potential of task-specific learned compression algorithms in this context to reduce data volumes while retaining crucial information. In detail, we compare traditional compression (JPEG 2000) versus a learned compression approach (Discretized Mixed Gaussian Likelihood) on three EO segmentation tasks: Fire, cloud, and building detection. Learned compression notably outperforms JPEG 2000 for large-scale, multi-channel optical imagery in both reconstruction quality (PSNR) and segmentation accuracy. However, traditional codecs remain competitive on smaller, single-channel thermal infrared datasets due to limited data and architectural constraints. Additionally, joint end-to-end optimization of compression and segmentation models does not improve performance over standalone optimization. \workshoptitle Learned Image Compression for Earth Observation: Implications for Downstream Segmentation Tasks 1 Introduction Satellite-based Earth Observation (EO) systems have become essential for monitoring and managing environmental phenomena, particularly wildfires, which pose severe threats to ecosystems and human communities. However, the expanding spatial coverage and increased resolution from EO missions create significant challenges in terms of data transmission and storage. To address these challenges, well-designed image compression algorithms are key. Unlike traditional compression methods, which primarily seek to preserve visual fidelity, task-specific compression aims to retain essential features necessary for accurate downstream analysis, such as image segmentation. Effective task-specific compression would allow significant data volume reductions while maintaining the performance and interpretability of critical tasks like wildfire and cloud detection. The research question that motivates our work is therefore: What is the optimal EO compression regime for various downstream tasks, while maintaining a balanced rate-distortion trade-off? In this paper, we compare the performance of traditional image compression techniques, specifically JPEG 2000, with that of learned compression approaches across three representative EO segmentation tasks: cloud detection, fire detection, and building footprint extraction. By systematically evaluating these methods on diverse datasets differing in spectral modality (thermal infrared versus optical multispectral), dataset size, and spatial resolution, we identify optimal compression strategies tailored to specific EO scenarios. Figure 1a Samples of the datasets used in this work. The thermal segmentation tasks are evaluated monochromatically, whereas the optical task is providing three channels. Figure 1b Overview of experiment setup for task-specific raster data compression. Raw observations x are compressed onboard the satellite and decompressed at the ground station for segmentation analysis. 2 Experimental Setup Datasets We conducted experiments using three diverse EO datasets representing distinct segmentation tasks: Cloud detection, fire detection, and building footprint extraction. Collectively, these datasets allow comprehensive evaluation of compression strategies across various EO scenarios, ensuring our findings are suitable for real-world applications. Samples are shown in 1Figure 1a . • Fire Dataset ( 9 ) The fire dataset \mathcal{D}_{\text{fire}} originates from a commercial CubeSat platform using mid-wave infrared (MWIR) imagery at 3.8 µm wavelength, optimal for detecting active combustion. Initially comprising 40 full scenes, we augment the data by cropping 571 smaller 32×32 pixel patches around confirmed fire locations, each accompanied by expert-annotated binary fire masks. • Cloud Dataset ( 12 ) The cloud dataset \mathcal{D}_{\text{cloud}} comprises thermal infrared imagery also acquired from commercial CubeSat constellation. Each image has a ground sampling distance (GSD) of 200 m, capturing radiance in the long-wave infrared (LWIR) spectrum, centered at 11.5 µm. The dataset consists of 528 manually annotated scenes of cloud presence. Each scene is resized to 256×256 pixels. • Building Dataset ( 7 ) The building dataset \mathcal{D}_{\text{building}} is based on multispectral optical imagery from ESA’s Copernicus Sentinel-2 mission, featuring spatial resolutions of 10–20 m per pixel. We utilized 28,828 10-channel image-mask pairs, each resized to 128×128 pixels. The masks represent building footprints extracted from OpenStreetMap and Microsoft global building footprint layers, providing a substantial dataset suitable for training and evaluating the segmentation and compression models. Segmentation Baseline Methods To establish robust baseline performance, we employ a U-Net architecture ( 8 ), a widely adopted convolutional neural network designed specifically for segmentation tasks. We use two different encoder backbones, MobileNetV2 ( 10 ) and ResNet34 ( 5 ), selected for their proven effectiveness and computational efficiency. Models are trained with binary cross-entropy (BCE) loss and optimized using the Adam optimizer. Training incorporates data-specific augmentation and early stopping criteria based on validation performance to ensure stability and generalization. Compression Algorithms We compare two distinct image compression methods and systematically analyze the rate-distortion trade-off of each method using standard metrics. This includes the Peak Signal-to-Noise Ratio (PSNR) and bits per pixel (bpp), across different datasets. • JPEG 2000: A classical, wavelet-based compression method widely used for its efficiency, scalability, and support for both lossless and lossy compression. JPEG 2000 ( 11 ) serves as our baseline to benchmark learned compression performance. • Discretized Gaussian Mixture Likelihood: A variational autoencoder-based compression model employing advanced entropy modeling techniques, including Gaussian Mixture Likelihoods and attention mechanisms ( 3 ). It is specifically designed to reduce spatial redundancy in its latent representation, which is of importance for efficient coding when fewer spectral channels are available. It significantly outperforms JPEG 2000 when evaluated on natural RGB images. We selected this architecture from the CompressAI framework ( 2 ), despite newer SoTA architectures being available, given its accessibility among other implementations and its proven performance over our baseline. Optimization Methodology To investigate potential synergies between compression and segmentation, we implement a fully differentiable pipeline combining both models. This approach allows joint training, optimizing not only for reconstruction quality but also directly for segmentation performance. The combined loss function \mathcal{L} consists of three terms. The common rate-distortion trade-off balances the distortion term \mathcal{D}_{\text{MSE}} against the estimated bit-rate \mathcal{R} . For end-to-end optimization, we add the task-specific BCE segmentation loss \mathcal{T}_{\text{BCE}} as a third weighted term. The model is trained using separate Adam optimizers ( 6 ) for each of the three components (auto-encoder, entropy bottleneck and segmentation model), as the entropy bottleneck needs to be optimized using a lower learning rate. The task-specific weight \gamma is set to zero for standalone compression experiments. An overview of the experiment architecture is shown in 1Figure 1b . Further details on the used hyperparameters are given in Appendix A . \mathcal{L}=\lambda\cdot\mathcal{D}_{\text{MSE}}+\mathcal{R}+\gamma\cdot\mathcal{T}_{\text{BCE}} (1) 3 Results Segmentation Baseline Performance The best performing baseline segmentation models are summarized in Table 1 . The F1+ score denotes the F1 score of the positive class. It is reported alongside its macro F1 given the class imbalance of fire and building pixels in their respective datasets. Generally, the UNet architecture is able to provide a reasonable performing baseline for all three tasks. However, it is not able to resolve very fine detail contours in the building footprints, hence the comparatively low F1+ result in \mathcal{D}_{\text{building}} . Segmentation under Bit Rate Reduction Figure 2 shows the impact of bit rate reduction on segmentation performance (F1), when training on the decompressed image data. This is done to directly quantify the effect of compression on our selection of downstream tasks. For all task, we observe that a certain level of image quality is needed before reaching a plateau in segmentation performance. This is in agreement with existing work, as many