Democratising deep learning for microscopy with ZeroCostDL4Mic.

von Chamier L, Laine RF, Jukkala J, Spahn C, Krentzel D, Nehme E, Lerche M, Hernández-Pérez S, Mattila PK, Karinou E, Holden S, Solak AC, Krull A, Buchholz TO, Jones ML, Royer LA, Leterrier C, Shechtman Y, Jug F, Heilemann M, Jacquemet G, Henriques R. · 2021

Read Democratising deep learning for microscopy with ZeroCostDL4Mic. by von Chamier L, Laine RF, Jukkala J, Spahn C, Krentzel D, Nehme E, Lerche M, Hernández-Pérez S, Mattila PK, Karinou E, Holden S, Solak AC, Krull A, Buchholz TO, Jones ML, Royer LA, Leterrier C, Shechtman Y, Jug F, Heilemann M, Jacquemet G, Henriques R. (2021) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc8050272.txt.

Introduction

von Chamier and co-authors launch ZeroCostDL4Mic, a free platform that brings deep learning to microscopy. Deep learning now beats classic image processing on many tasks. But training networks demands hardware and coding skill most biologists lack. That barrier locks novices out. The platform runs on Google Colab's free cloud resources. No coding expertise is required to use it.

The platform trains and applies leading networks through guided notebooks. Segmentation uses U-Net and StarDist. Object detection runs on YOLOv2. Denoising comes from CARE and Noise2Void. Super-resolution and image translation tools round out the set. Each network ships with measures to judge its performance. Users can tune models before trusting them. The authors demonstrate the system on several biological processes. The aim is to democratize a powerful method.

Catalog record

CREATORvon Chamier L, Laine RF, Jukkala J, Spahn C, Krentzel D, Nehme E, Lerche M, Hernández-Pérez S, Mattila PK, Karinou E, Holden S, Solak AC, Krull A, Buchholz TO, Jones ML, Royer LA, Leterrier C, Shechtman Y, Jug F, Heilemann M, Jacquemet G, Henriques R.
YEAR2021
COLLECTIONPAPERS
FORMATTXT
FILE SIZE90 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC8050272

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Democratising deep learning for microscopy with ZeroCostDL4Mic By Lucas von Chamier, Romain F. Laine, Johanna Jukkala, Christoph Spahn, Daniel Krentzel, Elias Nehme, Martina Lerche, Sara Hernández-Pérez, Pieta K. Mattila, Eleni Karinou, Séamus Holden, Ahmet Can Solak Published in: Nature Communications DOI: 10.1038/s41467-021-22518-0 ABSTRACT Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes. INTRODUCTION Over the past decade, the amount and complexity of bioimages have increased exponentially, translating into a need for more complex image analysis. To combat this data challenge, new processing tools, including artificial intelligence (AI) methods, have been developed. In particular, deep learning (DL), a subset of AI that is capable of independently extracting relevant features from images to perform specific tasks, now often outperforms conventional image-processing strategies, as was demonstrated for image classification1 or segmentation2. Lately, DL is increasingly employed for high-performance image-analysis tasks such as object detection3,4, image segmentation2,5,6 and image restoration (improvement in image resolution or denoising)7,8. In particular, the ability to automatically recognise objects and features in images (for instance, cancer cells in biopsy samples) is well underway to revolutionise how clinical samples are analysed (digital pathology)4,9. These capabilities have also led to an increased interest in DL in standard image-analysis workflows such as nuclear segmentation5,10,11, a common task that can be a significant challenge if done manually12. However, the potentially game-changing capabilities of DL have, to date, remained out of reach for most researchers without a strong background in computer sciences. A classical DL pipeline requires a computer algorithm (called “artificial neural network” or “DL network” here for simplicity) to be trained on a training dataset to generate a “model” that can perform a specific task. The training step is crucial as it will dictate the specificity and performance of the model. Generally, there are two different approaches to this step, using either supervised or self-supervised training13. In the supervised approach, paired input and output images are required. The network learns by attempting to find a mapping from the input to the desired output images, on an image-by-image basis. In the self-supervised approach, networks learn implicit patterns in the data and therefore, do not require paired input and output images. For instance, in a supervised image restoration task, a DL network, such as content-aware image restoration (CARE)7, is trained using a dataset containing many examples of noisy and paired high-quality images. Once trained, the model can be used to denoise images similar to the noisy images encountered during training. Training these DL networks is computationally expensive and often requires coding and computational expertise. In many biomedical research laboratories, neither of these requirements are readily available, hindering the adoption of an ever-growing number of powerful DL methods. To train DL networks, computer scientists typically set up local servers with high computational power or purchase expensive DL-ready workstations (Fig. 1a). This constitutes a robust way to develop and deploy DL approaches but requires technical know-how and financial commitment to set up and maintain. An alternative is to purchase computational resources provided by cloud services to train DL models. Trained DL models can then be used directly in the cloud or downloaded and used locally (Fig. 1a). Several software packages taking these approaches have been developed for the bioimaging community, especially trying to simplify the interface with DL hardware (i.e., U-Net2,14, cDeep3m10, DeepCell Kiosk15, DeepMIB16, NucleAIzer17, YAPiC (https://yapic.github.io/yapic/), ImJoy18, ilastik19, CellProfiler20 and Noise2Void8 and DenoiSeg Fiji plugin11,21). Another solution is to take advantage of model “zoos” which provide trained DL models with high reusability and versatility potential (Fig. 1a). These can be used directly to obtain predictions from new data using web interfaces or ImageJ plugins (i.e., CellPose22, DeepImageJ23, StarDist Fiji plugin5,6). Using pretrained models alleviates the computational requirement of training and has therefore become popular for tasks such as cellular and nuclear segmentation6,17,22. However, using pretrained models bears the risk of predicting incorrect structures and artefacts if they are used on unseen data too dissimilar to the data they were originally trained on24,25. This is also supported by observations that models perform best on datasets very similar to the training datasets7,25–27 (see Supplementary Note 1 and Supplementary Fig. 1). Training will therefore often be necessary for users to achieve optimal performance of models on their own datasets. Given these considerations, there is a need for a tool that seamlessly enables users to train, validate and experiment with DL tools for various image-analysis tasks without the constraints of expensive resources and the potential drawbacks of pretrained models.Fig. 1Using DL for microscopy.a Paths to exploiting DL. Training on local servers and inference on local machines (or servers) (first row), cloud-based training and local inference (second row), cloud-based training and inference (third row) and pretrained networks on standard machines (fourth row). b Overview of ZeroCostDL4Mic. The workflow of ZeroCostDL4Mic, featuring data transfer through Google Drive, training, quality control and prediction via Google Colab. After running a network, trained models, quality control and prediction results can then be downloaded to the user’s machine. c Overview of the bioimage analysis tasks currently implemented within the ZeroCostDL4Mic platform. Datasets from top left to bottom right: U-Net—ISBI 2012 Neuronal Segmentation Dataset78,79, StarDist—nuclear marker (SiR-DNA) in DCIS.COM cells, YOLOv2—bright field in MDA-MB-231 cells, N2V—actin label (paxillin-GFP) in U-251-glioma cells, CARE—actin label Lifeact-RFP in DCIS.COM cells, Deep-STORM—actin-labelled glial cell, fnet—bright-field and mitochondrial label TOM20-Alexa Fluor 594 in HeLa cells, pix2pix—actin label Lifeact-RFP and nuclear labels in DCIS.COM cells, CycleGAN—tubulin label in U2OS cells. All datasets are available through Zenodo (see “Data availability”) or as indicated in the GitHub repository. Here, we developed ZeroCostDL4Mic, an entry-level, cloud-based DL-deployment platform (Fig. 1b) that simplifies DL use for microscopy (Supplementary Movie 1). ZeroCostDL4Mic is a unified collection of self-explanatory Jupyter Notebooks, featuring an easy-to-use graphical user interface (GUI) (Supplementary Fig. 2) that requires only a web browser and a Google account for a user to run any of our DL-based tasks (Fig. 1c). All calculations are performed in the cloud using Google Colaboratory (Colab for short), circumventing the need to purchase or install graphical processing units (GPUs) and associated software. Using ZeroCostDL4Mic does not require prior knowledge in coding. Researchers can, in a few mouse clicks and aided by a common workflow, install all needed software dependencies, upload their imaging data and run networks for training and prediction (Supplementary Movie 1). Within our framework, models are comprehensively evaluated for performance and reliability and can be directly applied to new data

Read the complete text