Deep EM with Hierarchical Latent Label Modelling for Multi-Site Prostate Lesion Segmentation
Wen Yan 1,∗, Yipei Wang 1, Shiqi Huang 1, Natasha Thorley 2, Mark Emberton 2 ; Vasilis Stavrinides 3,4, Yipeng Hu 1, Dean Barratt 1 · 2026
Read Deep EM with Hierarchical Latent Label Modelling for Multi-Site Prostate Lesion Segmentation by Wen Yan 1,∗, Yipei Wang 1, Shiqi Huang 1, Natasha Thorley 2, Mark Emberton 2 ; Vasilis Stavrinides 3,4, Yipeng Hu 1, Dean Barratt 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14418.txt.
Introduction
Yan and co-authors present HierEM, a method for prostate lesion segmentation that learns from uneven hospital labels. Each site contours lesions by its own protocol, so networks overfit local styles and fail at new sites. HierEM treats each label set as a noisy view of one hidden clean mask. An EM loop swaps between guessing that mask and training a network on it as a soft target. Site-level sensitivity and specificity are estimated under a prior. Tests on three cohorts beat top rivals on cross-site reach.
Catalog record
| CREATOR | Wen Yan 1,∗, Yipei Wang 1, Shiqi Huang 1, Natasha Thorley 2, Mark Emberton 2 ; Vasilis Stavrinides 3,4, Yipeng Hu 1, Dean Barratt 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22486 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14418
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep EM with Hierarchical Latent Label Modelling for Multi-Site Prostate Lesion Segmentation Wen Yan 1,∗, Yipei Wang 1, Shiqi Huang 1, Natasha Thorley 2, Mark Emberton 2 ; Vasilis Stavrinides 3,4, Yipeng Hu 1, Dean Barratt 1 Abstract Label variability is a major challenge for prostate lesion segmentation. In multi-site datasets, annotations often reflect centre-specific contouring protocols, causing segmentation networks to overfit to local styles and generalise poorly to unseen sites in inference. We treat each observed annotation as a noisy observation of an underlying latent “clean” lesion mask, and propose a hierarchical expectation–maximisation (HierEM) framework that alternates between: (i) inferring a voxel-wise posterior distribution over the latent mask, and (ii) training a CNN using this posterior as a soft target and estimate site-specific sensitivity and specificity under a hierarchical prior. This hierarchical prior decomposes label-quality into a global mean with site- and case-level deviations, reducing site-specific bias by penalising the likelihood term contributed only by site deviations. Experiments on three cohorts demonstrate that the proposed hierarchical EM framework enhances cross-site generalisation compared to state-of-the-art methods. For pooled datasets evaluation, per-site mean DSC ranges from 29.50% to 39.69% across datasets; for leave-one-site-out generalisation, it ranges from 27.91% to 32.67%, yielding statistically significant improvements over comparison methods ( p < 0.039 p<0.039 ). The method also produces interpretable per-site latent label-quality estimates (e.g., sensitivity α ∼ [ 31.5 % , 47.3 % ] \alpha\sim[31.5\%,47.3\%] at specificity β ≈ 0.99 \beta\approx 0.99 ), supporting post-hoc analyses of cross-site annotation variability. These results indicate that explicitly modelling site-dependent annotation can improve cross-site generalisation. Code is available via https://anonymous.4open.science/r/HierEM-F2D3/ . 1 Introduction Multiparametric MRI (mpMRI), including T2-weighted (T2W), diffusion-weighted imaging (DWI), apparent diffusion coefficient (ADC) maps and contrast-enhanced (DCE) modalities, is widely used to assess suspected prostate cancer (PCa) [ 2 ] . Clinicians often contour lesions to guide treatment planning and follow-up. Automated deep-learning methods aim to reduce this workload, yet it faces a significantly high-variance ground truth challenge. This label variability is rooted in site-specific annotation protocols, the institutional style formed by local expert training and specific imaging protocols that influence how boundaries are perceived, which creates site-specific label bias. Inter-reader agreement for prostate lesion contours on mpMRI is only moderate, with reported Dice often around \sim 0.4 [ 5 , 8 ] . The current paradigm creates a critical bottleneck: segmentation networks easily overfit to the local contouring style of the training sites. When deployed to a new institution, the model generalises poorly, as single-site training achieves only 4%–28% Dice on a held-out site [ 10 ] . By contrast, testing on within-site achieves around \sim 20\%\sim 60\% Dice [ 16 , 3 , 13 , 1 ] with different datasets. This gap can be alleviated via test-site finetuning or calibration. However, such methods often lead to test-site-biased accuracy assessments. They force the model to match the local “observed” labels, which are themselves imperfect and biased. A truly multi-site training set (more than 10 sites) is expensive to curate for individual tasks that require coordination in labelling. PI-CAI is an example study with such a diverse collection of reference standards, but restricted to a primarily patient-level classification [ 12 ] . In many real-world applications, further finetuning or calibration at deployment may not be feasible or practical. While cross-site performance gaps may also stem from both imaging differences and label variation, pre-processing can partially mitigate the former. In this study, we focus on the impact of label-level variation. We propose a latent-label modelling approach to address cross-site annotation variability. Our model is inspired by the sensitivity/specificity formulation popularised by STAPLE [ 15 ] . STAPLE performs multi-annotator label fusion at inference time for a fixed image, jointly estimating each annotator’s sensitivity and specificity under conditional-independence assumptions. Our setting and goal differ. We do not fuse multiple labels for the same case, as typically only one annotation is observed per case. Instead, we treat each site’s annotation as a site-dependent noisy observation of an unobserved latent “clean” lesion mask during training. This yields a STAPLE-like observation model embedded within a segmentation-learning framework, where label variability is decomposed into three components: (1) global population factors capturing lesion characteristics shared across sites; (2) site-specific effects modelling systematic offsets due to local contouring and imaging protocols; and (3) case-level variability capturing intrinsic ambiguity (e.g., small or low-contrast lesions) that affects all annotators regardless of protocol. In summary, we use site-specific sensitivity/specificity to represent these hierarchical label-quality parameters. Learning is performed with an EM procedure [ 6 ] that alternates between inferring the latent consensus mask and updating model parameters. In the E-step, we compute a voxel-wise posterior over the latent consensus mask by combining the network’s image-based prior with the annotation likelihood under the current model. In the M-step, we update the UNet using this posterior as soft targets, and re-estimate the site-/case-level label-quality parameters by maximising their marginal likelihood under a hierarchical prior. Iterating these steps reduces dependence on any single centre’s contouring style, yields calibrated probabilistic predictions, and provides interpretable sens/spec estimates that summarise site tendencies and case-level ambiguity—improving robustness when deploying to unseen sites without requiring additional site-specific fine-tuning. 2 Method Let \Omega denote the voxel space of an mpMRI volume. For each case k\in\{1,\dots,K\} we observe an image X_{k} and a single binary lesion annotation Y_{k}\in\{0,1\}^{\Omega} provided by one site s_{k}\in\{1,\dots,S\} . The (unobserved) latent “clean” lesion mask is G_{k}\in\{0,1\}^{\Omega} . Our goal is to learn a segmentation model that is robust to site-specific annotation protocol, while also estimating site-/case-level latent label-quality parameters. Figure 1: Overview of proposed deep mixed EM framework with hierarchical latent label-quality parameters. The E-step infers a latent clean mask, and the M-step updates the segmentation network and latent label-quality parameters. segmentation network We model the latent consensus mask with an image-conditioned voxel-wise probability map predicted by the network: \pi_{k}(x)\;\triangleq\;p_{\theta}\!\left(G_{k}(x)=1\mid X_{k}\right)\;=\;\sigma\!\left(f_{\theta}(X_{k})_{x}\right), where x\in\Omega is a voxel in mpMRI, f_{\theta}(\cdot) is a neural network producing logits per voxel and \sigma(\cdot) is the sigmoid. 2.1 Hierarchical site-case label variability model We treat each institute as a distinct annotation protocol and model the observed label Y_{k}(x) as a noisy observation of G_{k}(x) . Conditioned on (G_{k},s_{k}) , voxels are assumed independent with site-case sensitivity and specificity: \displaystyle p(Y_{k}(x)=1\mid G_{k}(x)=1)=\alpha_{s_{k},k},\quad p(Y_{k}(x)=0\mid G_{k}(x)=0)=\beta_{s_{k},k},\ \forall x\in\Omega_{k}. (1) To stabilise site-/case-level label-quality estimates, we use a logistic-normal hierarchical prior that partially pools sens/spec toward a global mean \mu_{\alpha},\mu_{\beta} : \text{logit}(\alpha_{s,k})=\mu_{\alpha}+a_{s}+u_{k},\qquad\text{logit}(\beta_{s,k})=\mu_{\beta}+b_{s}+v_{k}, (2) where (a_{s},b_{s}) capture persistent site-specific contouring behavior and (u_{k},v_{k}) capture case difficulty. We place zero-mean Gaussian priors (equivalently, \ell_{2} penalties under MAP): a_{s}\sim\mathcal{N}(0,\sigma_{a}^{2}),\;\;b_{s}\sim\mathcal{N}(0,\sigma_{b}^{2}),\;\;u_{k}\sim\mathcal{N}(0,\sigma_{u}^{2}),\;\;v_{k}\sim\mathcal{N}(0,\sigma_{v}^{2}) For identifiability we constrain \sum_{s}a_{s}=0 and \sum_{s}b_{s}=0 . We further calculate site-level sen/spec as ( \sigma(\cdot) is sigmoid function): \alpha_{s}=\sigma(\mu_{\alpha}+a_{s}),\qquad\beta_{s}=\sigma(\mu_{\beta}+b_{s}) (3) EM learning with hierarchical sensitivity/specificity modelling We estimate segmentation network parameters \theta and the latent label-quality parameters \phi=\{\mu_{\alpha},\mu_{\beta},a_{1:S},b_{1:S},u_{1:K},v_{1:K}\} by maximizing the (regularized) marginal likelihood of \{Y_{k}\} with latent \{G_{k}\} . We optimise a MAP objective via an EM procedure. E-step: posterior of the latent mask. Given current (\theta,\phi) , the posterior factorizes over voxels: \displaystyle q_{k}(x) \displaystyle\;\triangleq\;p(G_{k}(x)=1\mid X_{k},Y_{k}(x)) (4) \displaystyle\;=\;\frac{\pi_{k}(x)\,p(Y_{k}(x)\mid G_{k}(x)=1)}{\pi_{k}(x)\,p(Y_{k}(x)\mid G_{k}(x)=1)+(1-\pi_{k}(x))\,p(Y_{k}(x)\mid G_{k}(x)=0)}. (5) With the sens/spec model, the likelihood terms are \displaystyle p(Y\!=\!1\mid G\!=\!1)=\alpha_{s,k},\quad p(Y\!=\!1\mid G\!=\!0)=1-\beta_{s,k},\quad (6) \displaystyle p(Y\!=\!0\mid G\!=\!1)=1-\alpha_{s,k},\quad p(Y\!=\!0\mid G\!=\!0)=\beta_{s,k}. (7) p(Y|G;\alpha,\beta)=\begin{cases}\alpha^{Y}(1-\alpha)^{1-Y},\qquad G=1,\\ (1-\beta)^{Y}\beta^{1-Y},\qquad G=0.\\ \end{cases}