MiTREE: Multi-input Transformer Ecoregion Encoder for Species Distribution Modelling

Theresa Chen, Yao-Yi Chiang · 2024

Read MiTREE: Multi-input Transformer Ecoregion Encoder for Species Distribution Modelling by Theresa Chen, Yao-Yi Chiang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18995.txt.

Introduction

Theresa Chen and Yao-Yi Chiang present MiTREE, a model that predicts where a species lives from satellite images plus environment data. Older machine learning models struggle to join spatial links between unlike inputs without warping them, and they often skip location and ecology facts. MiTREE is a multi-input Vision Transformer with an added ecoregion encoder.

The model computes cross-modal spatial links at native scale, and folds in where a place is and what its ecology is like. Tests used the SatBird summer and winter sets, which ask for bird species encounter rates at a site. MiTREE beat the leading baselines on that task, the authors report.

Catalog record

CREATORTheresa Chen, Yao-Yi Chiang
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18995

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MiTREE: Multi-input Transformer Ecoregion Encoder for Species Distribution Modelling Theresa Chen, Yao-Yi Chiang Abstract . Climate change poses an extreme threat to biodiversity, making it imperative to efficiently model the geographical range of different species. The availability of large-scale remote sensing images and environmental data has facilitated the use of machine learning in Species Distribution Models (SDMs), which aim to predict the presence of a species at any given location. Traditional SDMs, reliant on expert observation, are labor-intensive, but advancements in remote sensing and citizen science data have facilitated machine learning approaches to SDM development. However, these models often struggle with leveraging spatial relationships between different inputs- for instance, learning how climate data should inform the data present in satellite imagery- without upsampling or distorting the original inputs. Additionally, location information and ecological characteristics at a location play a crucial role in predicting species distribution models, but these aspects have not yet been incorporated into state-of-the-art approaches. In this work, we introduce MiTREE: a multi-input Vision-Transformer-based model with an ecoregion encoder. MiTREE computes spatial cross-modal relationships without upsampling as well as integrates location and ecological context. We evaluate our model on the SatBird Summer and Winter datasets, the goal of which is to predict bird species encounter rates, and we find that our approach improves upon state-of-the-art baselines. Conference: ; ; CCS: Computing methodologies Neural networks CCS: Applied computing Environmental sciences 1. Introduction The escalating effects of climate change pose serious threats to biodiversity through habitat loss and rapid changes within ecosystems. Biodiversity is essential for maintaining ecosystem services that benefit human well-being, such as pollination and nutrient cycling ( Haines-Young et al., 2010 ) . Planning for biodiversity conservation requires us to efficiently model areas that species’ are currently occupying. However, traditional methods to create species distribution models (SDMs) primarily rely on expert observation and knowledge, making the process very time and labor-intensive. Recently, the increasing availability of remote sensing images and citizen science data has led to the development of large-scale datasets that can facilitate machine learning methods for creating SDMs ( Teng et al., 2024 ; Lorieul et al., 2022 ) . Prior approaches have begun to harness the vast amount of information in large-scale species distribution datasets and have developed deep learning models that are customized for the task of species distribution modeling. For instance, several studies have utilized geographic locations as an indicator to predict species occurrence vectors ( Cole et al., 2023 ; Lange et al., 2024 ) . However, relying solely on geographic location may not provide accurate species distributions across diverse species, as environmental factors, such as climate or geography can influence spatial patterns ( Hirzel and Le Lay, 2008 ) . To achieve robust predictions and capture the full spectrum of influences on species distribution, models must be able to effectively integrate multi-modal geographic data. Currently, state-of-the-art (SOTA) methods that integrate multiple sources are computer vision models that have been adapted from their original source domains for species distribution modeling. For instance, studies have adapted ResNet to integrate red-green-blue (RGB) satellite imagery, the near-infrared (NIR) channel, and environmental data ( Teng et al., 2024 ; Lorieul et al., 2022 ) for the purpose of representation learning for a geographic area. These approaches require all modalities to be upsampled and aligned to a common resolution before they are concatenated and passed into the model. Upsampling prior to being passed through the neural network can affect the robustness of the representation as it sacrifices the fidelity of the original input and increases noise. Some approaches avoid upsampling by passing each input through an individual convolutional network and concatenating the learned representations for each input afterward ( Lorieul et al., 2022 ) . However, these models do not explicitly account for spatial cross-correlations between different data sources. For example, given a patch of trees in satellite imagery, the information in the corresponding patch in the climate data may help explain what kind of forest is present. Models that have been trained to quantify cross-modal relations between data from different sources have the potential to outperform baseline models on tasks that require robust representation for geographic areas. There exist several SOTA vision models that can perform multimodal learning with attention without upsampling ( Bachmann et al., 2022 ; Girdhar et al., 2023 ) . These models assume that images from different modalities have similar, if not the same, image dimensions (number of pixels on each side). However, when dealing with geographic data, one input’s image dimensions can be 20 to 30 times larger than another’s. Our aim is to utilize the base structures of SOTA multimodal vision models but adapt the architecture to account for inputs of vastly different resolutions. One challenge unique to geographic data is representing the geographic location of the data. Location is imperative in identifying underlying patterns in spatial data. Species data are particularly sensitive to geographic position, as movement limitations, such as maximum flight or foraging distances, often restrict species’ ranges. In the past, machine learning models have used location information to augment representations for classification tasks with a geographic component ( Mai et al., 2023 ) or clustered location information as a pre-text task during pre-training ( Ayush et al., 2021 ) . Using raw latitude and longitude values can present several issues, such as bias due to not accounting for the earth’s curvature and being too noisy ( Mai et al., 2022 ) . Having broader, region-level labels can help reduce the complexity of the location encoding task and simplify the model’s process for learning spatial relationships, as it now deals with a smaller number of labels rather than a vast range of raw coordinates. In our proposed model, we incorporate a map defining ecological regions in the contiguous U.S., compiled by domain experts ( Omernik and Griffith, 2014 ) , in order to make the model location-aware. By using ecoregions as location indicators, we address a common limitation of purely geographic clustering: locations that are close spatially but differ ecologically may still be grouped together if location alone is used. Ecoregions, however, are defined by shared environmental characteristics such as climate, vegetation, and topography ( Omernik and Griffith, 2014 ) . This approach ensures that regions are grouped not only by proximity but also by ecological similarity, allowing the model to better capture the environmental context necessary for species distribution predictions. For the task of species distribution modeling, we propose MiTREE, a multi-input ecoregion-aware vision Transformer (ViT) model that computes the attention between inputs of multiple resolutions without upsampling. We test our model on the species distribution modeling dataset SatBird, which utilizes imagery and environmental data of highly varied ground sampling distances (GSDs) (from 10 meters per pixel to 1,000 meters per pixel) to predict bird species distributions in the United States. Our contributions are as follows: • First, we present MiTREE , which builds upon the architecture of a multi-input ViT to jointly train satellite imagery and environmental data to predict species distributions across the United States. MiTREE introduces two key modifications to the original multi-input ViT. The first is modified patch projection layers tailored to the data present and number of pixels in each input,

Read the complete text