Deep Learning for BioImaging: What Are We Learning?

Ivan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell & Auguste Genovesio · 2026

Read Deep Learning for BioImaging: What Are We Learning? by Ivan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell & Auguste Genovesio (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13377.txt.

Introduction

Ivan Svatko and co-authors ask what deep learning models really learn from bioimaging data. Representation learning gave big gains for natural images, but microscopy may differ. Their study covers cell culture and tissue imaging, two widely used data types at key scales in biology. Simple baselines, even untrained models and structural representations, matched top methods in many test settings. That result shows common benchmark metrics can mask a lack of high-level biological features. Close comparison with baselines helps show where models are strong or weak.

Catalog record

CREATORIvan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell & Auguste Genovesio
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94643
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13377

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deep Learning for BioImaging: What Are We Learning? Ivan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell & Auguste Genovesio Abstract Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscopy imaging, however, it remains unclear what current representation learning methods actually learn. In this work, we conduct a systematic study of representation learning for the two most widely used and broadly available microscopy data types, representing critical scales in biology: cell culture and tissue imaging. To this end, we introduce a set of simple yet revealing baselines on curated benchmarks, including untrained models and simple structural representations of cellular tissue. Our results show that, surprisingly, state-of-the-art methods perform comparably to these baselines. We further show that, in contrast to natural images, existing models fail to consistently acquire high-level, biologically meaningful features. Moreover, we demonstrate that commonly used benchmark metrics are insufficient to assess representation quality and often mask this limitation. In addition, we investigate how detailed comparisons with these benchmarks provide ways to interpret the strengths and weaknesses of models for further improvements. Together, our results suggest that progress in microscopy image representation learning requires not only stronger models, but also more diagnostic benchmarks that measure what is actually learned. 1 Introduction Deep learning has transformed image analysis, supported by a large ecosystem of models, datasets, and benchmarks that make it easier to compare methods and reuse them for downstream applications. A similar shift is now happening in machine learning for biology, especially for microscopy imaging: data volumes are rapidly increasing ( 15 ) , and datasets are becoming more diverse in cell types, perturbations, and experimental settings. This scale has enabled the rise of foundation models trained on broad microscopy collections ( 38 ; 7 ) , with the promise of transferring to many biological tasks. As the field tests how useful these models really are for downstream biology, benchmarking becomes essential, both to rank models fairly and to understand which capabilities actually transfer. In parallel, new microscopy benchmarking efforts have started to emerge, including tasks focused on cell culture imaging ( 18 ; 8 ; 43 ) and histopathology ( 35 ; 27 ) . However, systematic benchmarking in microscopy imaging remains limited, and this gap is not only explained by missing datasets, it is also about how biological images are produced. Microscopy experiments are sensitive to experimental conditions, batch effects, and spatial or well and plate layout artifacts, and deep learning models can learn these signals instead of the underlying biology ( 62 ; 2 ; 30 ) . While these risks are widely recognized in principle, the field has not yet clearly demonstrated (using controlled, benchmark-focused analyses) how much such biases can distort model training and evaluation in microscopy. Importantly, a related biological modality has already made this issue concrete. In transcriptomics, several recent benchmarking studies have shown that confounding structure in the data can strongly influence performance estimates, sometimes to the point where simple statistical baselines rival or even outperform large foundation models ( 49 ; 6 ; 1 ; 20 ) . These results suggest that benchmark scores can reflect dataset-specific shortcuts rather than true biological generalization. Taken together, this motivates a careful look at whether microscopy benchmarks may face similar hidden failure modes, and whether current evaluations are reliably measuring what they intend to measure. In this work, we investigate what microscopy foundation models truly learn by evaluating fondation models on curated cell-culture and tissue benchmarks, using ImageNet-1k as a natural-image reference. To make performance easier to interpret, we introduce two simple but informative baselines. First, untrained models use post-processed features from randomly initialized networks to test how much signal comes from architectural inductive biases and weak pixel correlations rather than learned biological content. Second, disentangled tissue structures represent histology images through the spatial organization of cells, testing whether models capture biological information beyond tissue morphology. We then benchmark widely used foundation models on these tasks and compare them to both baselines, finding that many models fall short of expectations. 2 Related works Benchmarking pitfalls in biological machine learning. As biological machine learning shifts toward large pretrained models, evaluation has become a central bottleneck: reported gains can reflect confounding structures (batch, lab, protocol, cohort, or platform effects) rather than a transferable biological signal. In microscopy, this issue is especially acute because experimental design and acquisition artifacts can imprint strong variation that models may exploit, motivating benchmark designs that explicitly test robustness to batch structure and technical covariates ( 62 ; 2 ) . Closely related concerns have been demonstrated in transcriptomics, where several benchmarking studies show that confounders and dataset shortcuts can inflate performance estimates, and that simple baselines can match or outperform large models on perturbation prediction ( 6 ; 1 ; 20 ; 70 ; 69 ) . Importantly, these results highlight why strong simple baselines (e.g., linear models, cell-count/viability proxies, or random-feature embeddings) are not an afterthought: they calibrate what a score means and help detect when progress comes from benchmark-specific shortcuts rather than biological abstraction ( 54 ; 61 ) . Baselines for microscopy representations. Across computer vision, untrained networks can already impose powerful image priors ( 63 ; 32 ) , and can exhibit non-trivial selectivity driven by architecture and initialization ( 54 ; 3 ; 39 ) . Related observations extend to transformer architectures, where substantial behavior can arise even when large parts of attention are fixed or random ( 74 ; 22 ) . In microscopy, simple and interpretable baselines such as CellProfiler-derived morphology features ( 14 ) , cell-count / confluency summaries ( 68 ; 61 ) , and spatial-organization representations that emphasize cellular arrangement in addition to cellular morphology ( 65 ) often provide competitive reference points. Including these baselines is essential to calibrate benchmark’s difficulty and to relativize reported gains in presence of multiple sources of potential shortcuts. Training-light or training-free transfer pipelines for high-content screening further show that strong performance can arise from reusing generic representations with minimal task-specific training ( 19 ) . Microscopy imaging benchmarks. Benchmarking in microscopy spans diverse assay families and thus benefits from separating cellular-level and tissue-level organizational scales. For cell culture, widely used public resources include curated collections ( 50 ) such as BBBC ( 47 ) , perturbation imaging datasets designed to expose experimental batch effects such as RxRx1 and related releases ( 62 ; 55 ) , large-scale Cell Painting efforts such as JUMP-CP ( 15 ) , and more task-specific suites targeting heterogeneity across channels and acquisition settings ( 18 ) . Recent work has also moved toward making large industrial-scale screens more usable for benchmarking by publishing compressed subsets and standardized tasks ( 44 ; 59 ) . For tissue imaging, canonical benchmarks include challenge-style datasets such as CAMELYON16 and PANDA ( 4 ; 11 ) , alongside newer multimodal benchmarks that pair histology with molecular readouts (e.g., spatial transcriptomics) to evaluate cross-modal transfer ( 35 ; 27 ; 5 ) . Complementarily, distribution-shift benchmark frameworks provide standardized splits and protocols for robustness testing across domains, including settings relevant to both pathology and microscopy ( 42 ) . Microscopy imaging foundation models.

Read the complete text