ReCoSeg++: Extended Residual-Guided Cross-Modal Diffusion for Brain Tumor Segmentation

Sara Yavari, Rahul Nitin Pandya, Jacob Furst · 2025

Read ReCoSeg++: Extended Residual-Guided Cross-Modal Diffusion for Brain Tumor Segmentation by Sara Yavari, Rahul Nitin Pandya, Jacob Furst (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01058.txt.

Introduction

Sara Yavari and co-authors outline brain tumors on MRI scans by first imagining a scan the patient never had. Their two-stage system starts with a diffusion model that synthesizes one contrast type from three others. Where the real scan disagrees with the invented one, the tumor likely sits, since healthy tissue is predictable and tumor tissue is not. Those disagreement maps then guide a small segmentation network.

The design needs no hand-drawn masks for its main objective, which suits messy multi-center data. On the large BraTS 2021 collection the method scored 93.02 percent on the standard overlap measure for whole tumors, beating its predecessor's marks on the earlier 2020 set. Filtering out empty slices and tuning thresholds balanced missed tumor against false alarms. The authors present the residual trick as a scalable way to find tumors where labeled training data are scarce.

Catalog record

CREATORSara Yavari, Rahul Nitin Pandya, Jacob Furst
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01058

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReCoSeg++: Extended Residual-Guided Cross-Modal Diffusion for Brain Tumor Segmentation Sara Yavari, Rahul Nitin Pandya, Jacob Furst Abstract Accurate segmentation of brain tumors in MRI scans is critical for clinical diagnosis and treatment planning. We propose a semi-supervised, two-stage framework that extends the ReCoSeg approach to the larger and more heterogeneous BraTS 2021 dataset, while eliminating the need for ground-truth masks for the segmentation objective. In the first stage, a residual-guided denoising diffusion probabilistic model (DDPM) performs cross-modal synthesis by reconstructing the T1ce modality from FLAIR, T1, and T2 scans. The residual maps, capturing differences between predicted and actual T1ce images, serve as spatial priors to enhance downstream segmentation. In the second stage, a lightweight U-Net takes as input the concatenation of residual maps, computed as the difference between real T1ce and synthesized T1ce, with T1, T2, and FLAIR modalities to improve whole tumor segmentation. To address the increased scale and variability of BraTS 2021, we apply slice-level filtering to exclude non-informative samples and optimize thresholding strategies to balance precision and recall. Our method achieves a Dice score of 93.02 % 93.02\% and an IoU of 86.7 % 86.7\% for whole tumor segmentation on the BraTS 2021 dataset, outperforming the ReCoSeg baseline on BraTS 2020 (Dice: 91.7 % 91.7\% , IoU: 85.3 % 85.3\% ), and demonstrating improved accuracy and scalability for real-world, multi-center MRI datasets. keywords : Brain Tumor Segmentation, Diffusion Models, Residual Maps, Semi-Supervised Learning. 1 Introduction Brain tumor segmentation from MRI scans is a critical task in neuro-oncology, facilitating accurate diagnosis, informed treatment planning, and long-term patient monitoring [ 1 ] . Deep learning models particularly U-Net [ 2 ] and its 3D extension, 3D U-Net [ 3 ] have become standard approaches for this application, achieving high segmentation accuracy through end-to-end supervised learning frameworks. However, these models typically require large volumes of annotated data and involve high computational costs [ 4 ] , which limits their scalability in real-world clinical environments where annotations may be scarce and imaging modalities inconsistently available. To address these challenges, recent research has turned to generative modeling techniques. Notably, Denoising Diffusion Probabilistic Models (DDPMs) have been employed for cross-modal synthesis to reconstruct missing MRI modalities [ 5 ] . For example, DDMM Synth [ 6 ] uses diffusion models to accurately generate the T1ce modality from other available sequences. Recently, DDPMS Known for their ability to generate high-quality, diverse images [ 7 ] , beyond image generation, they have shown strong performance in a range of vision tasks, DPMs have also demonstrated strong performance in various tasks such as image editing [ 8 ] , super-resolution [ 9 ] , and segmentation [ 10 ] , underscoring their flexibility and robustness. This shows that diffusion models are versatile and reliable, working well across many different computer vision tasks. Prominent large-scale diffusion models-such as DALL·E 2 [ 11 ] , Imagen [ 12 ] , and Stable Diffusion [ 13 ] have shown remarkable capabilities in high-quality image generation [ 14 ] ; [ 15 ] . At the core of diffusion models is a two-stage process structured as a Markov chain. In the forward stage, Gaussian noise is progressively added to the input image over multiple timesteps, eventually transforming it into pure noise. The reverse process then learns to gradually denoise this input, reconstructing the original image step-by-step [ 7 ] . This iterative denoising framework underpins the impressive performance of diffusion models in visual generation tasks. Originally, diffusion models were applied to domains where absolute ground truth data is unavailable or ill-defined. Motivated by the strengths of DPMs, this paper focuses on improving binary brain tumor segmentation in MRI using a semi-supervised approach that combines cross-modal synthesis with residual-based attention, following the approach described in previous work [ 16 ] . The proposed ReCoSeg++ framework adopts a two-step strategy to enhance brain tumor segmentation from MRI sequences, following the approach described in previous work [ 17 ] . In the first step, a DDPM synthesizes the T1ce modality from the available FLAIR, T1, and T2 scans to address the challenge of missing modalities and provide a more complete set of inputs for downstream analysis. The synthesized T1ce is then compared to its real counterpart to compute a residual map that highlights regions with structural discrepancies, often corresponding to tumor tissue. In the second step, this residual map is concatenated with the original MRI modalities and input into a lightweight 2D U-Net for segmentation. This residual-guided attention mechanism enables the model to focus on informative regions, improving segmentation accuracy while reducing reliance on densely annotated data, as illustrated in Figure 1 . Figure 1: Overview of the proposed ReCoSeg framework for brain tumor segmentation using multi-modal MRI. The method consists of two key stages: (1) A cross-modal synthesis stage, where a Denoising Diffusion Probabilistic Model (DDPM) is employed to generate the T1ce modality from the available FLAIR, T1, and T2 sequences. This addresses the challenge of missing modalities and enhances the completeness of input data; (2) A residual-guided segmentation stage, in which a residual map is computed by subtracting the synthesized T1ce from the real T1ce. This residual map highlights regions of structural discrepancy, often indicative of tumor tissue, and is concatenated with the original modalities. The combined inputs are then fed into a lightweight 2D U-Net, where the residuals act as attention cues to guide the segmentation process. This two-phase design improves boundary localization, reduces reliance on dense annotations, and enhances both interpretability and efficiency in clinical workflows. 2 Related work Magnetic Resonance Imaging (MRI) is an essential tool in modern radiology, especially for neurological evaluations, due to its exceptional soft tissue contrast and ability to acquire images across multiple planes and sequences [ 18 ] . Unlike Computed Tomography (CT), which relies on ionizing X-rays, MRI uses strong magnetic fields and radio frequency (RF) pulses to generate high-resolution cross-sectional images of brain anatomy [ 19 ] . A key advantage of MRI is its support for multi-sequence imaging-such as T1-weighted (T1w), T2-weighted (T2w), Fluid-Attenuated Inversion Recovery (FLAIR), and Diffusion-Weighted Imaging (DWI)-each providing unique, complementary insights critical for tumor detection and characterization [ 20 ] . The emergence of deep learning has significantly advanced medical image segmentation, particularly for MRI-based brain tumor analysis. Convolutional Neural Networks (CNNs) have driven this progress [ 21 , 22 ] , with architectures like U-Net and its variants effectively modeling spatial hierarchies for accurate tumor localization [ 23 ] . Models such as TransBTS [ 24 ] , which integrates attention mechanisms, and CANet [ 25 ] , which captures context-aware features, have improved performance, especially near tumor boundaries. Nonetheless, challenges remain in segmenting small or irregular tumors due to loss of spatial detail and limited feature expressiveness [ 20 ] . To overcome these issues, extensions like U-Net++ [ 26 ] with nested skip connections, 3D U-Net [ 27 ] , and V-Net [ 28 ] for volumetric imaging have been developed. Attention U-Net [ 29 ] , TransUNet [ 30 ] , and Swin-UNet [ 31 ] leverage attention and transformer mechanisms to capture both local details and global context essential for delineating complex boundaries. Despite these innovations, the encoder-decoder paradigm remains central

Read the complete text