DM-QPMNet: Dual-Modality Fusion Network for Cell Segmentation in Quantitative Phase Microscopy

Rajatsubhra Chakraborty, Ana Espinosa–Momox, Riley Haskin, Depeng Xu, Rosario Porras–Aguilar · 2025

Read DM-QPMNet: Dual-Modality Fusion Network for Cell Segmentation in Quantitative Phase Microscopy by Rajatsubhra Chakraborty, Ana Espinosa–Momox, Riley Haskin, Depeng Xu, Rosario Porras–Aguilar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00218.txt.

Introduction

Rajatsubhra Chakraborty and four co-authors tackle a practical imaging problem: how to outline single cells in pictures from a fast phase microscope. Older threshold methods break down when noise is high or cells sit close together. Simple deep learning tools also waste information, because they just stack two kinds of images together.

Their network, DM-QPMNet, reads the two image types in separate streams, one for light intensity and one for phase. An attention step then lets the streams share what each one sees best, such as edges and texture. Tests show clear gains over single-stream and simple stacking baselines, with stable training and little extra cost.

Catalog record

CREATORRajatsubhra Chakraborty, Ana Espinosa–Momox, Riley Haskin, Depeng Xu, Rosario Porras–Aguilar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00218

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DM-QPMNet: Dual-Modality Fusion Network for Cell Segmentation in Quantitative Phase Microscopy Rajatsubhra Chakraborty, Ana Espinosa–Momox, Riley Haskin, Depeng Xu, Rosario Porras–Aguilar Abstract Cell segmentation in single-shot quantitative phase microscopy (ssQPM) faces challenges from traditional thresholding methods that are sensitive to noise and cell density, while deep learning approaches using simple channel concatenation fail to exploit the complementary nature of polarized intensity images and phase maps. We introduce DM-QPMNet, a dual-encoder network that treats these as distinct modalities with separate encoding streams. Our architecture fuses modality-specific features at intermediate depth via multi-head attention, enabling polarized edge and texture representations to selectively integrate complementary phase information. This content-aware fusion preserves training stability while adding principled multi-modal integration through dual-source skip connections and per-modality normalization at minimal overhead. Our approach demonstrates substantial improvements over monolithic concatenation and single-modality baselines, showing that modality-specific encoding with learnable fusion effectively exploits ssQPM’s simultaneous capture of complementary illumination and phase cues for robust cell segmentation. † † address: 1 College of Computing and Informatics, University of North Carolina at Charlotte, NC, USA 2 Department of Physics and Optical Science, University of North Carolina at Charlotte, NC, USA 3 Center for TAIMing AI, University of North Carolina at Charlotte, NC, USA 1 Introduction Deep-learning cell segmentation has advanced rapidly through large, labeled fluorescence datasets paired with standardized training pipelines. Generalist models such as Cellpose learn robust appearance priors from heterogeneous corpora and transfer across laboratories and cell types [ 1 ] , while self-configuring frameworks like nnU-Net codify best practices for preprocessing, architecture scaling, and training [ 2 ] . Promptable foundation models such as SAM have further extended segmentation to novel distributions via interactive prompts [ 3 ] . These systems demonstrate impressive breadth but require abundant labeled images with conspicuous boundaries, typically achieved through fluorescence staining. Fluorescence microscopy is powerful yet operationally expensive and scientifically limiting. Assembling large, expertly annotated datasets (e.g., LIVECell) demands sustained curation effort [ 4 ] , and models often exhibit brittleness across different stains, instruments, and protocols [ 5 ] . Beyond annotation costs, fluorescence introduces photobleaching and phototoxicity that constrain live-cell imaging and longitudinal assays [ 6 ] , limiting throughput for routine cell analysis. Quantitative phase microscopy (QPM) offers a label-free alternative by measuring optical path length to reveal cell mass and morphology without exogenous contrast agents [ 7 ] . However, conventional QPM methods face practical limitations: phase-shifting techniques (DHM, DPM) [ 8 , 9 ] require temporal multiplexing or complex interferometric alignment, while transport-of-intensity approaches [ 10 ] necessitate mechanical scanning. These constraints increase system cost and reduce robustness. Single-shot quantitative phase microscopy (ssQPM) addresses these limitations by employing polarization-sensitive cameras that simultaneously record four interferograms at distinct polarization angles (0°, 45°, 90°, 135°) and reconstruct a pixel-aligned quantitative phase map in one exposure [ 11 ] . This common-path architecture operates under low-intensity illumination, exhibits inherent vibration robustness, and enables real-time acquisition at camera-limited frame rates [ 12 ] . Critically, ssQPM provides complementary multi-modal data : polarized intensities encode high-frequency edge and texture features, while the phase map captures low-frequency optical thickness and mass distribution both acquired simultaneously in a single shot. Traditional QPM segmentation relies on intensity-driven thresholding and morphological operations [ 11 ] , which exhibit sensitivity to speckle noise and cell density [ 13 ] . Learning-based approaches can overcome these limitations [ 7 ] , but most prior work treats QPM as a single-modality problem segmenting from phase maps alone or individual polarization angles. In medical imaging, multi-modal fusion architectures have consistently demonstrated that combining complementary channels improves segmentation accuracy [ 14 ] . Early fusion via input concatenation is straightforward but can dilute modality-specific statistics when channels exhibit different frequency characteristics or dynamic ranges [ 15 ] . Dual-encoder designs that learn modality-specific representations before interaction have proven effective for multi-sequence MRI and multi-stain histology [ 16 , 17 ] , with attention mechanisms enabling content-aware fusion at intermediate scales [ 18 , 19 ] . Frameworks like nnU-Net provide stable decoder topologies and training heuristics [ 2 ] , offering a robust foundation for architectural extensions. This work addresses single-shot QPM segmentation and asks: Can we effectively leverage complementary multi-illumination intensity and phase cues for accurate cell segmentation? Two naive baselines exist: (1) single-modality approaches that segment from phase maps alone or concatenated polarization angles only, and (2) early-fusion multi-modal approaches that concatenate all five channels (four polarization angles plus phase) into a standard nnU-Net. Single-modality methods discard either the high-frequency edge information (phase-only) or low-frequency thickness cues (angles-only), limiting segmentation accuracy. Early fusion, while incorporating both modalities, forces complementary signals through shared normalization and convolutional stems, potentially diluting modality-specific contributions. Polarized angles capture high-frequency boundary information while the phase map encodes low-frequency thickness distributions treating these as a homogeneous input overlooks their distinct statistical properties and spatial frequency content. We posit a dual-encoder architecture with late fusion at intermediate depth that respects modality-specific characteristics. Our model separates the four polarized intensity images and the quantitative phase map into distinct encoders, allowing each stream to develop appropriate feature representations independently. At an intermediate encoder depth, where receptive fields span cell-scale context while retaining spatial detail, we apply multi-head attention (MHA) to fuse features in a content-aware manner. This enables polarized edge and texture representations to selectively query the phase encoder for complementary optical thickness information. Following fusion, a shared encoder tail and standard nnU-Net decoder with deep supervision complete the segmentation pipeline. The design preserves nnU-Net’s training stability and decoder topology while introducing principled multi-modal integration at minimal parameter overhead. Our contributions are: • A dual-encoder nnU-Net with mid-encoder fusion via multi-head attention that treats polarized intensity and phase as distinct modalities; • Systematic ablation studies comparing fusion strategies (early vs. late, attention vs. concatenation), interaction depths, and modality contributions (single-modality vs. multi-modal); • Architecture-level framework demonstrating that multi-modal fusion with modality-specific encoding outperforms both single-modality and early-fusion baselines for ssQPM cell segmentation. 2 Methods 2.1 Dataset Following the data collection and processing in [ 11 ] , we obtain a single–shot quantitative phase microscopy (ssQPM) dataset acquired with a polarization–sensing camera that simultaneously records four interferograms at 0°, 45°, 90°, and 135°as seen in Fig. 1 and reconstructs a quantitative phase map in the same exposure. This common path system operates under low intensity illumination, is robust to vibrations, and is positioned as a cost effective alternative for routine cell analysis.It couples intensity maps from 4 polarization components with an aligned phase map in a single shot.We posit that these complementary cues enable more reliable cell segmentation than any single modality alone (angles or phase).In our experiments, we use 24 samples for training and 6 samples for testing, spanning high, medium, and low confluence regimes. Figure 1: ssQPM dataset inputs: four polarized intensity images (0°, 45°, 90°, 135°), one quantitative phase map, and the corresponding ground-truth binary cell mask. 2.2 Input Representation and Modality Handling We process the images obtained from ssQPM into four polarized intensity images and one quantitative phase map. Let \mathbf{A}\!\in\!\mathbb{R}^{H\times W\times 4} denote the stacked angles \{0^{\circ},45^{\circ},90^{\circ},135^{\circ}\} and \mathbf{P}\!\in\!\mathbb{R}^{H\times W\times 1} the phase map, pixel–aligned with \mathbf{A} . To respect modality statistics, we apply

Read the complete text