An interpretable approach to automating the assessment of biofouling in video footage
Evelyn J. Mannix, Bartholomew A. Woodham · 2025
Read An interpretable approach to automating the assessment of biofouling in video footage by Evelyn J. Mannix, Bartholomew A. Woodham (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12875.txt.
Introduction
Evelyn Mannix and Bartholomew Woodham built in 2025 an interpretable computer vision system that grades marine growth on ship hulls from inspection video. Such growth spreads invasive species, so shipping rules increasingly demand proof of hull cleaning. Human review of diver and robot footage is slow. Earlier automated methods worked as black boxes, hiding their reasoning.
Their approach builds on a large vision model and an interpretable classification method. It beats previous convolutional systems while using far fewer parameters. Crucially, it shows which image regions drove each grade and which training examples it leaned on. All code, data, and model weights are public. The authors aim to make compliance checks faster and more trustworthy for regulators and operators.
Catalog record
| CREATOR | Evelyn J. Mannix, Bartholomew A. Woodham |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-12875
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An interpretable approach to automating the assessment of biofouling in video footage Evelyn J. Mannix, Bartholomew A. Woodham Abstract Biofouling—communities of organisms that grow on hard surfaces immersed in water—provides a pathway for the spread of invasive marine species and diseases. To address this risk, international vessels are increasingly being obligated to provide evidence of their biofouling management practices. Verification that these activities are effective requires underwater inspections, using divers or underwater remotely operated vehicles (ROVs), and the collection and analysis of large amounts of imagery and footage. Automated assessment using computer vision techniques can significantly streamline this process, and this work shows how this challenge can be addressed efficiently and effectively using the interpretable Component Features (ComFe) approach with a DINOv2 Vision Transformer (ViT) foundation model. ComFe is able to obtain improved performance in comparison to previous non-interpretable Convolutional Neural Network (CNN) methods, with significantly fewer weights and greater transparency—through identifying which regions of the image contribute to the classification, and which images in the training data lead to that conclusion. All code, data and model weights are publicly released. 1 Introduction The spread of exotic marine organisms and diseases pose a severe threat to people’s way of life, the global environment, and local economies. [ 1 ] Biofouling on vessels travelling internationally is a significant pathway for the anthropogenic spread and introduction of these species, [ 2 ] and can consist of a wide range of organisms adapted to colonising hard surfaces in water, such as barnacles, tubeworms and algae. [ 3 ] While the shipping industry prioritises the management of biofouling to reduce fuel costs, [ 4 ] the incentives to manage biofouling for hydrodynamic purposes do not provide sufficient coverage of biosecurity risks. [ 5 , 6 ] Several jurisdictions, such as Australia, New Zealand and California, [ 7 , 8 ] require internationally arriving vessels to provide pre-arrival information that documents the steps they have taken to manage their biofouling. By encouraging proactive biofouling management, the associated biosecurity risk from vessels to these countries is reduced. However, to assess the efficacy of management practices or provide evidence that a vessel does not pose an unacceptable biosecurity risk, the underwater hull of the vessel may need to be inspected. Further, incoming vessels may request or be asked to provide recent underwater footage that supports that their hull is clean and effectively managed. Manual assessment of footage from biofouling inspections by the appropriate decision maker can be a resource intensive exercise. There can also be a large degree of variability in biofouling assessment, with different experts having different thresholds for particular biofouling categories. [ 9 ] Previous work [ 10 ] has shown that it is feasible to use automated approaches to identify the presence and severity of biofouling on vessel hulls. However, these approaches are often not designed to identify fouling on commericial vessels—rather they are designed to identify particular groups of fouling organisms on test plates, that are used to study the behaviour of biofouling on surfaces. While there has been some work aiming to detect biofouling on vessel hulls, this has only focused on detecting biofouling in still images, and the approach was not extended to video data. [ 9 ] Previous deep learning approaches have also been black-box methods which are unable to explain their predictions. [ 11 ] Standard image classification approaches, such as training Convolutional Neural Networks (CNNs) with image annotations as done in prior works, [ 9 ] can often learn to detect classes using spurious features. [ 12 ] These are features that are correlated with the variable of interest that are used by a model to artificially improve training accuracy. In training a model to detect biofouling, these spurious features could be niche areas like sea chests or antifoulant coating damage, which are both known to promote biofouling growth. [ 13 ] In contrast, interpretable-by-design methods can (i) localise which regions of an image are important for making a prediction, and (ii) identify representative images from the training data that lead to a particular categorisation. [ 14 ] This can improve the trust of a biosecurity manager in the model and provide opportunities to identify flaws—such as if training data is incorrectly labelled or spurious features are important for a prediction. In this paper, we present the application of Component Features (ComFe), [ 15 ] a state-of-the-art interpretable-by-design image classification approach, to the challenge of identifying the presence of biofouling within footage of a vessel’s hull. The ComFe approach takes a pretrained foundation model, such as a model from the DINOv2 family, [ 16 ] and is able to identify key directions in the embedding space of this model that encode important concepts. [ 17 ] By clustering regions of the image and comparing the regional embeddings to these concepts, ComFe is able to identify where biofouling occurs in an image, and explain this by referring to key images in the training data. In this work, our main contributions are: • Presenting the first application of an interpretable computer vision model to the problem of identifying biofouling in imagery of vessel hulls, that is able to outperform previous large CNN ensembles while using significantly less parameters and being more efficient to compute. • Demonstrating the utility of the model to summarize videos taken using underwater Remotely Operated Vehicles (ROVs), to enable the potential biosecurity risk from biofouling on a vessel to be quickly evaluated. • The public release of the dataset used to train and evaluate the models, as well as the models themselves. In Section 2 we provide a high level explanation of the ComFe approach, describe the datasets used for training the models and the training process, and outline how the models are applied to video footage. In Section 3 we demonstrate the utility of ComFe for detecting biofouling in still image and video data, and discuss further considerations in Section 4 . Input Image Component Features Fouling class prototype Sim. score 1.0 , 0.99 No fouling class prototype Sim. score 1.0 , 0.96 No fouling class prototype Sim. score 0.65 Class Prototypes Prediction Fouling present Figure 1 : Illustration of ComFe. The image is first clustered into component features, which represent different regions with similar content. The green component feature shows the vessel hull, blue captures the sea chest grating, teal highlights the image tags, and the yellow and red component features identify the regions of the image with biofouling. These are then compared to class prototypes, which match as expected as visualised by the exemplars—training images in the dataset with similar embeddings to the fitted class prototypes. This comparison between component features and class prototypes is used to identify the salient parts of the image for predicting if biofouling is present, as shown by the class confidence heatmap in the final image, where the red regions show areas in the image with higher confidence of fouling being present. 2 Methods 2.1 The ComFe approach The ComFe [ 15 ] method explains a prediction through the use of component features and class prototypes, as outlined in Fig. 1 . The core components are a pretrained foundation model that is frozen during training, and the ComFe