Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach
Francisco Mena, Diego Arenas, Andreas Dengel · 2025
Read Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach by Francisco Mena, Diego Arenas, Andreas Dengel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01132.txt.
Introduction
Francisco Mena, Diego Arenas, and Andreas Dengel tackle missing data in Earth watching, in a 2025 paper. Satellites and other sources give many views of the same place. Some views are often missing, which can hurt model forecasts. Past work in machine learning has used missing parts as a training trick.
Their methods train models on all mixes of present and missing views. Rather than fill gaps with a fixed number, the model simply skips the missing views. They test simple merging and a more complex merging based on a known network type. Trials on four Earth data sets show gains when some views are missing. Scores also rose when all views were present. One model can now handle any mix of views it is given.
Catalog record
| CREATOR | Francisco Mena, Diego Arenas, Andreas Dengel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-01132
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach Francisco Mena, Diego Arenas, Andreas Dengel Abstract Multi-view learning (MVL) leverages multiple sources or views of data to enhance machine learning model performance and robustness. This approach has been successfully used in the Earth Observation (EO) domain, where views have a heterogeneous nature and can be affected by missing data. Despite the negative effect that missing data has on model predictions, the ML literature has used it as an augmentation technique to improve model generalization, like masking the input data. Inspired by this, we introduce novel methods for EO applications tailored to MVL with missing views. Our methods integrate the combination of a set to simulate all combinations of missing views as different training samples. Instead of replacing missing data with a numerical value, we use dynamic merge functions, like average, and more complex ones like Transformer. This allows the MVL model to entirely ignore the missing views, enhancing its predictive robustness. We experiment on four EO datasets with temporal and static views, including state-of-the-art methods from the EO domain. The results indicate that our methods improve model robustness under conditions of moderate missingness, and improve the predictive performance when all views are present. The proposed methods offer a single adaptive solution to operate effectively with any combination of available views. A Preprint Keywords Multi-view Learning, Earth Observation, Missing Data, Data Augmentation, Robustness. 1 Introduction Nowadays, the usage of multiple data sources, sensors, or views in ML models has become a standard practice in various applications and domains [ Yan et al., 2021 ] . The reason is that by using multiple sources of information, the individual data can be ratified and complemented to enhance predictive models [ Hong et al., 2021 , Sainte Fare Garnot et al., 2022 ] . Earth Observation (EO) is one of the domains where Multi-View Learning (MVL) has been used to provide comprehensive insights in various applications [ Mena et al., 2024c ] . In our work, we refer to a view as all features in a specific data source. For example, a view can be an optical or radar Satellite Image Time Series (SITS) , weather conditions, topographic information, or various metadata, consisting of a heterogeneous scenario with different spatio-temporal resolutions. Thus, there are temporal views , with multi-temporal data and static views , with single-date data. This diversity distinguishes research done in EO from other ML domains such as vision and text [ Rolf et al., 2024 ] . Furthermore, EO views might not be a persistent source of information as researchers commonly assume. The EO domain faces challenges due to the finite lifespan of remote sensors, noise, and cloudy conditions in optical sensors [ Shen et al., 2015 ] . Besides, unexpected errors can affect the availability of the data, such as the failure of the Sentinel-1B satellite in 2021. This problem leads to scenarios with missing data, which hinder accurate predictions and introduce biases in ML models [ Choi and Lee, 2019 , Hong et al., 2021 , Sainte Fare Garnot et al., 2022 ] . For instance, [ Mena et al., 2024a ] evidence the negative predictive impact that missing views have in different vegetation applications, highlighting the lack of robustness of different MVL models. Even current ML models, like Transformers, are not naturally robust to missing views [ Ma et al., 2022 , Tseng et al., 2023 , Chen et al., 2024 ] . This leaves open questions such as how to increase the robustness of MVL models to missing views. Despite the negative impact of missing data in ML models, many studies actively incorporate it during learning. For instance, it has been used as a masking operator for self-supervision in text [ Devlin et al., 2018 ] , signal [ Cao et al., 2018 ] and vision domains [ He et al., 2022 ] . Another option is to use Missing data as Augmentation (MAug) techniques, such as the dropout layer [ Srivastava et al., 2014 ] , and augmentation operations in the vision domain (e.g. crop). In the EO domain, the MAug technique has been used in MVL models. One case is to drop all features from a random view (sensor) during training [ Wang et al., 2022 , Mena et al., 2024b ] . Another case corresponds to including all combinations of missing views, as experimented by [ Gawlikowski et al., 2023 ] in out-of-distribution detection. However, most of these works use a fixed-size merge function (like concatenation) and validate on EO datasets with only static views. In this work, we introduce methods based on two main components, all Combinations of Missing views (CoM) and a dynamic merge function at feature-level. The CoM acts as an augmentation technique, simulating all combinations of missing views in the training set, which relates to literature [ Gawlikowski et al., 2023 , Mena et al., 2024b ] . However, we apply the MAug technique at the feature-level instead of at the input [ Mena et al., 2024b ] and use dynamic merge instead of fixed functions [ Gawlikowski et al., 2023 ] . We find that integrating the CoM with a merge function that ignores the missing views enhances the predictions and robustness of MVL models. The dynamic merge can be a simple average or a more complex function. Inspired by literature on using ML models as aggregators [ Lee et al., 2019 , Bugueño and Mendoza, 2020 , Wang et al., 2021 , Guo et al., 2024 ] , we include some alternative functions based on gated fusion, cross-attention fusion, and memory fusion. We validate the proposed methods on four EO datasets with temporal and static views, and compare them to five state-of-the-art methods in the EO domain. For assessing the robustness to missing data, we simulate missing views during inference and compare the predictions to full-view data. The evidence suggests that our methods have better robustness than competing methods in cases of moderate missingness, and in some cases improve predictive performance when all views are available. Overall, our main contributions are as follows: i) we propose a MAug technique tailored to missing views in MVL at feature level ( CoM ), ii) adapt the MAug technique to ignore the features of the missing views by using a dynamic fusion, and iii) validate on EO datasets with classification and regression tasks, considering temporal and static views. The proposed methods, inspired by sensor invariant modeling [ Francis, 2024 , Mena et al., 2024b ] , allow a model to generate adaptive predictions based on the available views. Our code will be released at https://github.com/fmenat/com-views . 2 Related work MVL with EO data. Recently, there has been an increase in EO research using multiple data sources to enhance ML model predictions [ Camps-Valls et al., 2021 ] . The main difference in the MVL models investigated is how the data is fused [ Mena et al., 2024c ] . Input-level fusion has been the common choice for this, i.e. merge the data before feeding a ML model. For instance, [ Kussul et al., 2017 ] feed a CNN model with just the concatenation of different sensors (multi-spectral