The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs

Jonathan Sauder, Viktor Domazetoski, Guilhem Banc-Prandi, Gabriela Perna, Anders Meibom, Devis Tuia, Environmental Computational Science and Earth Observation Laboratory, École Polytechnique Fédérale de Lausanne, Switzerland, Laboratory for Biological Geochemistry, École Polytechnique Fédérale de La · 2025

Read The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs by Jonathan Sauder, Viktor Domazetoski, Guilhem Banc-Prandi, Gabriela Perna, Anders Meibom, Devis Tuia, Environmental Computational Science and Earth Observation Laboratory, École Polytechnique Fédérale de Lausanne, Switzerland, Laboratory for Biological Geochemistry, École Polytechnique Fédérale de La (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20000.txt.

Introduction

Jonathan Sauder and co-authors release Coralscapes, the first large data set for teaching computers to read coral reef scenes. Reefs are declining under climate change and local stress. Watching them closely needs faster tools than expert divers.

The set holds over two thousand reef images. Experts labeled dozens of benthic classes across them. About 174 thousand segmentation masks mark each region. The design mirrors a famous city-scene data set closely. That parallel lets vision models benchmark on reefs fairly. Tests show transformer models lead the current field. Hard cases remain where classes blend together. Open data like this speeds reef monitoring worldwide. It is free for all teams to use.

Catalog record

CREATORJonathan Sauder, Viktor Domazetoski, Guilhem Banc-Prandi, Gabriela Perna, Anders Meibom, Devis Tuia, Environmental Computational Science and Earth Observation Laboratory, École Polytechnique Fédérale de Lausanne, Switzerland, Laboratory for Biological Geochemistry, École Polytechnique Fédérale de La
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-20000

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs Jonathan Sauder, Viktor Domazetoski, Guilhem Banc-Prandi, Gabriela Perna, Anders Meibom, Devis Tuia, Environmental Computational Science and Earth Observation Laboratory, École Polytechnique Fédérale de Lausanne, Switzerland, Laboratory for Biological Geochemistry, École Polytechnique Fédérale de Lausanne, Switzerland, Centre for Ecology and Conservation, University of Exeter, United Kingdom, School of the Environment, The University of Queensland, Australia, Center for Advanced Surface Analysis, University of Lausanne, Switzerland, Equal contribution Abstract Coral reefs are declining worldwide due to climate change and local stressors. To inform effective conservation or restoration, monitoring at the highest possible spatial and temporal resolution is necessary. Conventional coral reef surveying methods are limited in scalability due to their reliance on expert labor time, motivating the use of computer vision tools to automate the identification and abundance estimation of live corals from images. However, the design and evaluation of such tools has been impeded by the lack of large high quality datasets. We release the Coralscapes dataset, the first general-purpose dense semantic segmentation dataset for coral reefs, covering 2075 images, 39 benthic classes, and 174k segmentation masks annotated by experts. Coralscapes has a similar scope and the same structure as the widely used Cityscapes dataset for urban scene segmentation, allowing benchmarking of semantic segmentation models in a new challenging domain which requires expert knowledge to annotate. We benchmark a wide range of semantic segmentation models, and find that transfer learning from Coralscapes to existing smaller datasets consistently leads to state-of-the-art performance. Coralscapes will catalyze research on efficient, scalable, and standardized coral reef surveying methods based on computer vision, and holds the potential to streamline the development of underwater ecological robotics. 1 Introduction Coral reefs are under unprecedented stress from both global anthropogenic climate change and local human activities such as overtourism, pollution, or destructive fishing practices [ 56 ] . Coral reefs, which host more than a third of all marine biodiversity on less than 0.1% of the world’s oceans’ surface [ 31 ] , are among the most vulnerable ecosystems on the planet: under the current greenhouse gas emission trajectory, more than 99% of warm-water coral reefs are projected to lose significant area or suffer local extinction [ 57 , 27 ] . This could cascade into catastrophic impacts for the more than 500 million people that rely on coral reef-related ecosystem services through food security, coastal protection, or tourism [ 64 ] . However, there are regions, species, and genotypes within species that are more heat-resistant, holding promise to withstand the projected ocean warming induced by climate change [ 13 ] . The reefs of the northern Red Sea, for example, are projected to resist up to 5°C of warming [ 30 , 83 , 48 , 66 , 77 ] . These heat-resistant reefs must be protected from local stressors to ensure the survival of coral reef ecosystems on the planet. To inform effective conservation or restoration strategies, reefs must be monitored at the highest spatial and temporal resolution. As aerial and satellite remote sensing methods are limited in resolution, constrained to very shallow areas, and rely on good atmospheric, weather, and water conditions [ 52 , 5 , 19 , 44 ] , in situ surveying remains the de facto method for monitoring coral reefs. Computer vision emerges as a promising tool to scale coral reef monitoring, through tackling the bottleneck imposed by the needed expert time for identifying corals and estimating their abundance on images [ 8 , 9 ] , as well as the development of visual simultaneous localization and mapping (SLAM) systems [ 61 , 62 ] that can be used on autonomous underwater vehicles (AUVs) and other robots [ 60 , 33 ] . However, existing datasets in the coral reef domain are limited to narrow domains such as orthomosaics or photo quadrats [ 4 , 28 ] , limited to sparse point labels [ 8 , 9 ] or image classification [ 74 , 42 ] , or in their size and diversity, inhibiting the widespread deployment of computer vision in coral reef science and conservation. The main challenge to creating large-scale classification/segmentation datasets in coral reefs is the annotation process. Annotating coral reefs requires trained experts and cannot be done by crowd-sourced click-workers. Corals display high morphologic plasticity [ 4 ] , and the appearance of corals can vary drastically between biogeographic regions. In general, precise phylogenetic taxonomy of coral species or even genera can simply not be determined from images, even in close-up scenarios in good conditions, limiting the classification hierarchy to morphological attributes, such as growth forms (e.g. branching, meandering, massive). To add to the challenge, corals may appear bleached or dead, followed by various stages of degradation (e.g. overgrown by algae), or decomposing into rocky substrate or rubble. Besides corals, reefs host a wide range of life, including fish, invertebrates (sea urchins, sea cucumbers, anemones, sponges etc.), plants (algae, seagrass), and many other organisms. Finally, the identifiability of benthic classes in reefs is strongly impacted by the degradation of color and induced blur from the water column. Tackling these challenges to create a consistently labeled large-scale dataset requires a concerted effort by trained expert annotators. We propose the Coralscapes dataset for semantic scene understanding in coral reefs. Coralscapes serves two main purposes: i) to drive the development of computer vision applications in reef conservation, including automated surveying methods and underwater robotics, and ii) to allow benchmarking of state-of-the-art semantic segmentation on a domain that is underrepresented in large pre-training datasets. The Coralscapes dataset is: • The first general-purpose semantic segmentation dataset in the coral reef domain, including a wide range of reefs from thriving to heavily bleached/dead, as well as diverse camera angles and distances to the substrate. Unlike previous datasets, Coralscapes is not restricted to close-ups of corals, orthomosaics, or photo quadrats. • The largest expert-annotated semantic segmentation dataset spanning 2075 images at 1024 \times 2048px resolution, containing 174k polygons over 39 classes, labeled in a consistent and speculation-free manner. • The first dataset in the coral reef domain designed for fair evaluation of machine learning approaches, enforcing a spatial train/test split on images gathered from 35 dive sites in 5 countries in the Red Sea. Dataset # Images Annotation Type # Annotations # Annotated Pixels # Classes Comment Spatial Split Openly Available RSMAS [ 79 ] 776 Image 776 - 14 - No Yes Jamil et al. [ 42 ] 1582 Image 1582 - 3 - No Yes Raphael et al. [ 74 ] 5000 Image 5000 - 11 Coral only No No DeOhloNosCorais [ 32 ] 1411 Image + Foreground Map 1411 7.08M 21 - No Yes Eilat [ 10 ] 212 Sparse 1123 - 8 No Yes King et al. [ 46 ] 1807 Sparse 9511 - 10 - No No Moorea Labeled Corals [ 8 ] 2055 Sparse 400,000 - 9 Only photo quadrats No Yes Pacific Labeled Corals [ 9 ] 5090 Sparse 251,988 - 20 Only photo quadrats No Yes Benthoz-2015 [ 12 ] 9874 Sparse 407,968 - 147 Ortho-photos Yes Yes Seaview [ 34 ] 11387 Sparse 859,870 - 228 Ortho-photos Yes Yes BenthicNet [ 54 ] * 29603 Sparse 1,390,262 -

Read the complete text