Vision Transformers for Weakly-Supervised Microorganism Enumeration

Javier Ureña Santiago 1 , Thomas Ströhle 2 3 , Antonio Rodríguez-Sánchez 1 , Ruth Breu 2 · 2024

Read Vision Transformers for Weakly-Supervised Microorganism Enumeration by Javier Ureña Santiago 1 , Thomas Ströhle 2 3 , Antonio Rodríguez-Sánchez 1 , Ruth Breu 2 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02250.txt.

Introduction

Javier Ureña Santiago and co-authors wrote this 2024 computing paper on counting microorganisms in images with vision transformers. Counting microbes by hand is slow, yet the counts matter for checking contamination and surface cleanliness. Past automation leaned on segmentation or density maps.

The team tested whether vision transformers can count microbes under weak supervision, where full labels are not given. They trained several transformer versions as the feature backbone and compared them with the classic ResNet design. A transformer-based crowd counting model was also in the mix. Four microbiology data sets supplied the test images. ResNets won overall, but the transformers posted sound results on every set. The authors see promise for transformer counting in microbial image work.

Catalog record

CREATORJavier Ureña Santiago 1 , Thomas Ströhle 2 3 , Antonio Rodríguez-Sánchez 1 , Ruth Breu 2
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02250

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vision Transformers for Weakly-Supervised Microorganism Enumeration Javier Ureña Santiago 1 , Thomas Ströhle 2 3 , Antonio Rodríguez-Sánchez 1 , Ruth Breu 2 Abstract Microorganism enumeration is an essential task in many applications, such as assessing contamination levels or ensuring health standards when evaluating surface cleanliness. However, it’s traditionally performed by human-supervised methods that often require manual counting, making it tedious and time-consuming. Previous research suggests automating this task using computer vision and machine learning methods, primarily through instance segmentation or density estimation techniques. This study conducts a comparative analysis of vision transformers (ViTs) for weakly-supervised counting in microorganism enumeration, contrasting them with traditional architectures such as ResNet and investigating ViT-based models such as TransCrowd. We trained different versions of ViTs as the architectural backbone for feature extraction using four microbiology datasets to determine potential new approaches for total microorganism enumeration in images. Results indicate that while ResNets perform better overall, ViTs performance demonstrates competent results across all datasets, opening up promising lines of research in microorganism enumeration. This comparative study contributes to the field of microbial image analysis by presenting innovative approaches to the recurring challenge of microorganism enumeration and by highlighting the capabilities of ViTs in the task of regression counting. PubID: pubid: 979-8-3503-7903-7/24/$31.00 ©2024 European Union I Introduction Enumeration of microorganisms is crucial across diverse sectors including medicine, pharmaceutical quality control, or environmental monitoring [ 1 , 2 , 3 ] . This task is relevant in microbiology, and methods have been researched and improved for decades [ 4 ] . Traditional techniques are usually tedious, as counting manually (agar plate or hemocytometry) or the indirect estimations with turbidimetry [ 5 , 6 , 7 ] depends on specialized equipment, team, and time in order to improve efficiency. Hence, there has been a pursuit of automated and efficient enumeration techniques to improve this practice [ 8 ] . As a result, the efficiency of computer vision methods has improved microbial enumeration in later years. Machine learning, and later, deep learning, has made it possible to count the number of particles in images and extract various characteristic parameters of them, reducing workload and improving the accuracy of the analysis [ 9 , 10 ] . Research to improve numeration through deep learning is extensive and follows two lines of research: detection-based methods and regression-based methods, like image masks estimation, or density map regression [ 11 , 12 ] . These approaches have set the baseline and brought a new standard for efficiency all over the academic corpus. Fig. 1: Comparison of methodologies in deep learning regarding instance counting. Most approaches tackle the solution by density map estimation or instance segmentation (top and middle). Weakly-supervised counting (bottom) regresses the total number of instances in the image, removing the need for spatial-aware ground truth. Although density estimation and instance segmentation offer benefits, they are not always feasible due to the lack of detailed spatial information. For example, microbial swab testing assesses surface cleanliness by providing a total bacterial count without precise localization [ 13 ] . Spatial-aware datasets complicate enumeration by requiring detailed annotations and increasing computational burden [ 13 ] . Instead, focusing on aggregate counts provides a simpler, faster, and equally effective solution. To address this issue, weakly-supervised counting (WSC) is used as an approach that applies regression to images to predict the total number of instances without spatial information, as shown in Figure 1 . CNNs are the most common architectural choice to solve this problem [ 14 , 15 ] , but recent advances have shown the effectiveness of vision transformers (ViTs) for the same task, outperforming CNNs [ 16 , 17 ] . This is the result of the inherent self-attention mechanisms of ViTs, which, in contrast to CNNs, outperform in capturing global image context and contextual dependencies, proving effective in image classification and segmentation [ 18 ] . The goal of this study is to highlight the use of ViTs in weakly-supervised counting and make use of its applicability in the task of microorganism enumeration as an effective solution. To achieve this, we conducted a comprehensive analysis of ViT-based backbone regression architectures. We compared them to the most popular benchmark architectures: CNNs, ResNet50, and ResNet101, by training them under the same strategy (no use of pre-trained weights nor fine-tuning to optimize results) to achieve the task of microorganism enumeration. We used four different microscopic-based datasets: the Fluorescent Neuronal Cells dataset [ 19 ] , VGG-Cells dataset [ 20 ] , U2OS/HL60 Human Cancer Cells dataset [ 21 ] and a self-made artificial fluorescent bacteria dataset, that is composed of high-resolution images. We created this dataset for the task of WSC in order to cover the gaps the former three datasets have in regards of dataset size, density sparsity through the images, and image resolution. Our experimental evaluation indicates that although traditional architectures such as ResNet achieve better performance, ViTs, especially CrossViT, can achieve comparable results to ResNet. CrossViT also performed exceptionally well on homogeneously distributed datasets, outperforming other ViT variants and CNNs in terms of computational efficiency. These results underscore the need to explore the use of ViTs, as further research holds the potential to achieve effective architectures for the task of weakly-supervised microorganism enumeration. This study contributes to microorganism enumeration [ 8 ] by direct enumeration without the need for spatial information. We also analyze the use of vision transformers (ViTs) in regression counting. We evaluate popular and novel weakly-supervised counting methods, adapt them to microbial imaging, and evaluate ViTs in this context. Our analysis covers the capability of ViTs in a regression task, comparing different ViT approaches to identify the best one depending on the use case. By evaluating the performance of ViTs under the same training strategy as the other methods when configured for weakly-supervised counting, we provide insight into the limitations and potential of this architecture, thus contributing to the field of feature extraction using self-attention mechanisms [ 22 ] . We also contribute to the study by developing an artificial fluorescent bacteria dataset designed for the task of weakly-supervised microorganism enumeration. An implementation of the method and the artificial dataset generation tool are available at https://github.com/JavierUrenaPhDProjects/vits_for_WSC . II Related work II-A Machine Learning in microorganism enumeration Automating enumeration of microorganisms has been researched for decades, with traditional techniques like PCA, LDA or SVM and subsequently advancing to feature extraction with deep learning [ 8 , 23 ] . Two research strategies have been developed: detection-based and regression-based enumeration. Detection-based methods refer to the enumeration of instances once they are located in the image, being image segmentation as the most accurate method of instance detection. [ 24 , 11 , 25 ] extend the U-Net architecture [ 26 ] in its ability to count cells and bacteria by discretizing them from background and subsequently enumerating them. Regression-based methods solve the task either through density map estimation or direct regression without spatial details. Convolutional neural networks (CNNs) for density estimation treats image pixels as real-valued feature vectors and constructs density functions over pixel grids, enabling object count estimation by integrating over specific image regions [ 20 , 27 , 28 , 12 ] . The task is also accomplished with weak

Read the complete text