Prior-Guided Residual Diffusion: Calibrated and Efficient Medical Image Segmentation
Fuyou Mao 1 , Beining Wu 2 , Yanfeng Jiang 3 , Han Xue 2 , Yan Tang 1 , and Hao Zhang 1 · 2025
Read Prior-Guided Residual Diffusion: Calibrated and Efficient Medical Image Segmentation by Fuyou Mao 1 , Beining Wu 2 , Yanfeng Jiang 3 , Han Xue 2 , Yan Tang 1 , and Hao Zhang 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01330.txt.
Introduction
Medical image segmentation, drawing the borders of organs or tumors, is often unsure at fuzzy edges, so a single answer can mislead. Fuyou Mao, with five co-authors, presents a diffusion-based framework that learns a full spread of likely segmentations for each voxel, the three-dimensional pixel. The method is guided by a prior, an early coarse guess.
Labels are embedded as one-hot targets in a continuous space to fit the diffusion process. The network then learns only the residual, the gap between the prior and the truth, which speeds training and improves calibration. Extra supervision at middle time steps steadies learning. On MRI and CT data sets, the method beat Bayesian, ensemble, and plain diffusion baselines on overlap scores and calibration measures, while needing fewer sampling steps.
Catalog record
| CREATOR | Fuyou Mao 1 , Beining Wu 2 , Yanfeng Jiang 3 , Han Xue 2 , Yan Tang 1 , and Hao Zhang 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01330
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Prior-Guided Residual Diffusion: Calibrated and Efficient Medical Image Segmentation Fuyou Mao 1 , Beining Wu 2 , Yanfeng Jiang 3 , Han Xue 2 , Yan Tang 1 , and Hao Zhang 1 Abstract Ambiguity in medical image segmentation calls for models that capture full conditional distributions rather than a single point estimate. We present Prior-Guided Residual Diffusion (PGRD), a diffusion-based framework that learns voxel-wise distributions while maintaining strong calibration and practical sampling efficiency. PGRD embeds discrete labels as one-hot targets in a continuous space to align segmentation with diffusion modeling. A coarse prior predictor provides step-wise guidance; the diffusion network then learns the residual to the prior, accelerating convergence and improving calibration. A deep diffusion supervision scheme further stabilizes training by supervising intermediate time steps. Evaluated on representative MRI and CT datasets, PGRD achieves higher Dice scores and lower NLL/ECE values than Bayesian, ensemble, Probabilistic U-Net, and vanilla diffusion baselines, while requiring fewer sampling steps to reach strong performance. I Introduction In modern clinical practice, medical image segmentation is an indispensable technology for critical procedures such as diagnosis, treatment planning, and disease monitoring [ 1 , 2 ] . However, the segmentation task often faces significant uncertainty due to inherent noise and artifacts in medical images, as well as the inter-observer variability among clinical experts manually delineating anatomical structures or lesions. Traditional deterministic segmentation models typically output a single, most probable class label for each pixel or voxel, forming a single segmentation map [ 3 , 4 ] . While intuitive, this approach has a fundamental flaw: it cannot express or quantify the uncertainty in its predictions. Furthermore, its predictions are often over-confident, which can introduce potential risks in safety-critical clinical decision-making. Therefore, it is of paramount importance to develop probabilistic segmentation models capable of capturing and modeling this uncertainty—that is, learning a conditional distribution over all plausible segmentation given an input image. To achieve probabilistic modeling, the academic community has explored various technical paths [ 5 ] . Mainstream methods can be broadly categorized into three types: (i) Bayesian Neural Networks [ 6 ] , which approximate the posterior distribution by sampling model parameters; (ii) Deep Ensembles [ 7 ] and Multi-Head Models [ 8 ] , which construct a structured hypothesis space by training multiple independent models or setting up multiple outputs within a single model; and (iii) generative models such as Generative Adversarial Networks (GANs) or Conditional Variational Autoencoders (CVAEs) [ 9 ] . Although these methods have, to some extent, enhanced the model’s ability to express uncertainty, they each have inherent limitations. For instance, Bayesian methods are computationally expensive and rely on the quality of approximate inference; Deep Ensembles consume substantial training and inference resources; and GAN/VAE-based generative models often suffer from training instability, mode collapse, or poor probability calibration. Consequently, there is an urgent need in the field for a novel solution that can balance the diversity of generated results, training stability, and probability calibration. Denoising Diffusion Probabilistic Models (DDPMs) [ 10 ] , or simply diffusion models, have garnered significant attention as a powerful class of generative models due to their outstanding performance and stable training process in high-dimensional data generation tasks, especially with natural images [ 11 ] . However, directly applying standard diffusion models to medical image segmentation presents two core challenges [ 12 , 13 ] : First, output space mismatch. Diffusion models naturally operate in a continuous data space, whereas image segmentation is fundamentally a pixel-wise or voxel-wise discrete classification problem. Effectively bridging the continuous diffusion process with discrete segmentation labels is a key challenge. Second, low computational efficiency. The generation process of diffusion models relies on an iterative reverse denoising chain, often requiring hundreds or even thousands of steps to produce high-quality samples. For time-sensitive clinical applications, the associated computational cost can be prohibitively high. To address these challenges, this paper proposes a novel diffusion model framework tailored for medical image segmentation that emphasizes both probability calibration and sampling efficiency—the Prior-Guided Residual Diffusion (PGRD) model. Our core idea is to decompose the complex task of segmentation map generation. First, we resolve the output space mismatch by embedding discrete segmentation labels as one-hot vectors, cleverly transforming them into diffusion targets that can be processed in a continuous space. Second, and our main innovation, we introduce a lightweight prior network that provides a coarse segmentation prediction at each step of the diffusion process. Consequently, the powerful diffusion model no longer needs to learn the entire complex segmentation map from pure noise but instead focuses on learning the residual between the ground truth and the coarse prior. This residual learning paradigm greatly simplifies the model’s learning objective, thereby significantly reducing the number of sampling steps required for convergence and improving the calibration of the final predictions. Furthermore, we have designed a Deep Diffusion Supervision (DDS) mechanism, which applies additional supervisory signals at intermediate steps of the denoising process to further stabilize model optimization and guide the model to more accurately recover the clean segmentation label throughout the entire chain [ 14 ] . The experiments in this paper will be conducted on several public Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) datasets. The results will demonstrate that, compared to various Bayesian methods, deep ensembles, Probabilistic U-Net, and standard diffusion model baselines, PGRD not only achieves superior performance in segmentation accuracy (e.g., Dice Similarity Coefficient) but also shows better performance on key probability calibration metrics (such as Negative Log-Likelihood, NLL, and Expected Calibration Error, ECE). Moreover, all of this is achieved while requiring fewer sampling steps. Our contributions can be seen as follows: • Segmentation-aware diffusion targets. We embed labels as one-hot vectors in a continuous space, enabling diffusion to learn voxel-wise distributions while remaining compatible with discrete evaluation. • Prior-guided residual learning (PGR). A light prior network provides coarse segmentations at each step; the diffusion model learns only the residual to the prior, reducing required sampling steps and improving calibration. • Deep diffusion supervision (DDS). We supervise intermediate time steps to stabilize optimization and encourage accurate recovery of the clean label along the chain. Fig. 1: Illustration of the Prior-Guided Residual Diffusion (PGRD) pipeline for probabilistic medical image segmentation. Our method decomposes the complex segmentation task into a two-step process. First, as shown in panel (a), a deterministic nnU-Net is pre-trained to provide a reliable but coarse prior g(x) . Then, in panel (b), this prior guides the reverse diffusion process. Instead of learning the entire segmentation from noise, our denoising U-Net focuses on the simpler task of learning the residual between the current noisy state y_{t} and the prior. This residual learning is further stabilized by our Deep Diffusion Supervision (DDS) mechanism. As depicted in panel (c), repeating this guided denoising process allows us to sample multiple plausible segmentation maps, effectively capturing the underlying uncertainty in the data and yielding a well-calibrated predictive distribution. II Method II-A Overview and Problem Setup Let x denote an input 2D/3D image volume and y the corresponding discrete segmentation. We represent y as a one-hot vector field y_{0} (per voxel), which serves as the diffusion target in a continuous space. A prior predictor g_{\varphi}(x) provides step-wise guidance; the diffusion model parameterized by \theta