DEMIST: DE coupled M ulti-stream latent d I ffusion for Quantitative Myelin Map S yn T hesis
Jiacheng Wang ⋆ , Hao Li ⋆ , Xing Yao ⋆ , Ahmad Toubasi † , Taegan Vinarsky † , Caroline Gheen † , Joy Derwenskus † , Chaoyang Jin † , Richard Dortch ‡ , Junzhong Xu † , Francesca Bagnato † , Ipek Oguz ⋆ · 2025
Read DEMIST: DE coupled M ulti-stream latent d I ffusion for Quantitative Myelin Map S yn T hesis by Jiacheng Wang ⋆ , Hao Li ⋆ , Xing Yao ⋆ , Ahmad Toubasi † , Taegan Vinarsky † , Caroline Gheen † , Joy Derwenskus † , Chaoyang Jin † , Richard Dortch ‡ , Junzhong Xu † , Francesca Bagnato † , Ipek Oguz ⋆ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12396.txt.
Introduction
Jiacheng Wang and eleven co-authors present DEMIST, a method to synthesize quantitative myelin maps from routine brain scans. A specialized magnetic resonance technique can produce a myelin-sensitive measure tied to multiple sclerosis assessment, but it needs long, dedicated scans. Standard T1-weighted and FLAIR images are far more common.
DEMIST uses a latent diffusion model in two stages. First, autoencoders learn aligned latent forms for the target maps and the body images. Then a conditional diffusion model works in that space. It sits on a frozen foundation backbone. Conditioning is split into semantic, spatial and adaptive parts, with edge-aware losses to keep lesion borders sharp. Evaluation uses a sizable set of scans and subjects.
Catalog record
| CREATOR | Jiacheng Wang ⋆ , Hao Li ⋆ , Xing Yao ⋆ , Ahmad Toubasi † , Taegan Vinarsky † , Caroline Gheen † , Joy Derwenskus † , Chaoyang Jin † , Richard Dortch ‡ , Junzhong Xu † , Francesca Bagnato † , Ipek Oguz ⋆ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-12396
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DEMIST: DE coupled M ulti-stream latent d I ffusion for Quantitative Myelin Map S yn T hesis Jiacheng Wang ⋆ , Hao Li ⋆ , Xing Yao ⋆ , Ahmad Toubasi † , Taegan Vinarsky † , Caroline Gheen † , Joy Derwenskus † , Chaoyang Jin † , Richard Dortch ‡ , Junzhong Xu † , Francesca Bagnato † , Ipek Oguz ⋆ Abstract Quantitative magnetization transfer (qMT) imaging provides myelin-sensitive biomarkers, such as the pool size ratio (PSR), which is valuable for multiple sclerosis (MS) assessment. However, qMT requires specialized 20-30 minute scans. We propose DEMIST to synthesize PSR maps from standard T1w and FLAIR images using a 3D latent diffusion model with three complementary conditioning mechanisms. Our approach has two stages: first, we train separate autoencoders for PSR and anatomical images to learn aligned latent representations. Second, we train a conditional diffusion model in this latent space on top of a frozen diffusion foundation backbone. Conditioning is decoupled into: (i) semantic tokens via cross-attention, (ii) spatial per-scale residual hints via a 3D ControlNet branch, and (iii) adaptive LoRA-modulated attention. We include edge-aware loss terms to preserve lesion boundaries and alignment losses to maintain quantitative consistency, while keeping the number of trainable parameters low and retaining the inductive bias of the pretrained model. We evaluate on 163 scans from 99 subjects using 5-fold cross-validation. Our method outperforms VAE, GAN and diffusion baselines on multiple metrics, producing sharper boundaries and better quantitative agreement with ground truth. Our code is publicly available at https://github.com/MedICL-VU/MS-Synthesis-3DcLDM . † † address: ⋆ Vanderbilt University † Vanderbilt University Medical Center ‡ Barrow Neurological Institute 1 Introduction Quantitative magnetization transfer (qMT) imaging can measure myelin content through biomarkers like the pool size ratio (PSR). PSR is highly specific to demyelination in multiple sclerosis (MS) [ 2 ] . However, qMT acquisitions are lengthy, motion-sensitive, and not routinely available, motivating PSR synthesis from ubiquitous T1w and FLAIR scans. Prior work [ 14 ] suggests PSR maps are partly predictable from clinical images, showing feasibility but also underscoring the need for stronger generative modeling. Existing synthesis methods based on GANs and UNets [ 10 , 19 ] can produce unrealistic features and lose anatomical detail. Diffusion models like DDPM and DDIM [ 4 , 16 ] are more stable and produce better results, but applying them directly in pixel space is too computationally expensive for 3D medical images. Latent diffusion models (LDMs) solve this problem by working in the compressed latent space of an autoencoder rather than directly on images, allowing them to generate 3D brain MRI and translate between modalities [ 12 , 6 , 11 , 9 ] . Diffusion-based image translation can be conditioned through concatenation, cross-attention [ 12 ] , spatial adaptive normalization [ 6 ] , or control branches like ControlNet [ 21 ] . ControlNet is particularly effective because it freezes the pretrained diffusion backbone and adds trainable parallel branches through zero-initialized connections, preserving learned priors while enabling task-specific control, such as segmentation-guided generation and cross-modality translation [ 7 , 20 , 13 ] . Beyond conditioning mechanisms, low-rank adapters (LoRA) [ 5 ] provide parameter-efficient modulation of attention projections. We propose DEMIST: 3D LDM uses both ControlNet and cross-attention for PSR synthesis (Fig. 1 ). The key idea is to decouple anatomical structure preservation (via ControlNet) from quantitative value generation (via cross-attention), while adapting a frozen diffusion foundation model with LoRA so that task-specific calibration does not compromise pretrained priors. We found that explicitly separating these objectives improves results because they require different types of conditioning information. Our main contributions are: Figure 1: Overview of our framework. Stage 1: A latent diffusion model learns in PSR latent space. Stage 2: A pretrained diffusion backbone receives three complementary streams derived from semantic tokens via cross-attention, spatial residual hints from a 3D ControlNet, and adaptive LoRA on attention projections while keeping backbone weights fixed. Edge-aware objective: gradient-magnitude maps and from a 3D Sobel operator, sharpening generated brain tissue structures. • Novel framework for simultaneous anatomy and intensity conditioning: Our goal is to preserve both anatomical structure and PSR intensities. We achieve this with separate conditioning pathways in our diffusion model. • Data-efficient multi-scale conditioning: We use separate autoencoders for each image type, apply ControlNet conditioning at multiple scales, and use LoRA to adapt a frozen diffusion foundation model, avoiding degradation of pretrained priors. This design works well even with limited training data. • Clinically driven evaluation: We include edge-preserving losses to maintain tissue boundaries, which are clinically important in MS. We validate our approach with standard image quality metrics as well as MS-specific metrics. 2 Methods Notation. Let x_{\mathrm{c}}\!\in\!\mathbb{R}^{2\times H\times W\times D} denote the input images (T1w+FLAIR) and y\!\in\!\mathbb{R}^{1\times H\times W\times D} the target PSR map. Two 3D AutoencoderKLs (same design, independent weights) define z_{\mathrm{c}}=E_{\mathrm{cond}}(x_{\mathrm{c}}) , z_{\mathrm{y}}=E_{\mathrm{PSR}}(y) , and \widehat{y}=D_{\mathrm{PSR}}(z_{\mathrm{y}}) , with spatial downsampling by 4\times so that any modality latent z_{\mathrm{mod}}\!\in\!\mathbb{R}^{L\times\frac{H}{4}\times\frac{W}{4}\times\frac{D}{4}} (we use L{=}8 ). We learn a conditional diffusion model entirely in the target latent space to approximate p(z_{\mathrm{y}}\mid z_{\mathrm{c}}) . Diffusion follows the standard noise-prediction parameterization: at time t the noisy latent x_{t} is obtained by adding Gaussian noise \varepsilon\!\sim\!\mathcal{N}(0,I) to z_{\mathrm{y}} , and the denoiser predicts \varepsilon from (x_{t},t) . Overview. Our proposed DEMIST framework has two stages. Stage 1 learns separate autoencoders for PSR and T1w+FLAIR to obtain modality-specific latents that preserve salient structure. Stage 2 initializes a diffusion UNet from an unconditional BraTS-pretrained LDM [ 1 , 11 , 8 ] and keeps it frozen to retain the score prior. It then fuses anatomical conditioning into the denoiser through multi-level interfaces. This design separates anatomical alignment from quantitative mapping and enables data-efficient PSR synthesis. We note that the unconditional 3D LDM pretrained on BraTS FLAIR volumes [ 11 ] in Stage 2 models p(z) without conditioning, whereas our task is conditional image-translation in the PSR latent, learning p(z_{\mathrm{y}}|z_{\mathrm{c}}) . Accordingly, we freeze the backbone to preserve the score prior and inject conditioning via three decoupled streams: a semantic stream providing cross-attention tokens (what to synthesize), a spatial stream using a zero-initialized 3D ControlNet to inject multi-scale residual hints at voxel level (where to place it), and an adaptive stream applying low-rank LoRA updates to selected attention projections (how to specialize the frozen prior to PSR with limited data). 2.1 Two-Stage Latent Diffusion Stage 1: Latent Autoencoder Alignment. We train two 3D KL autoencoders [ 11 ] with the same architecture but independent (non-shared) weights: one for the target PSR domain (E_{\text{PSR}},D_{\text{PSR}}) and one for the conditional T1w+FLAIR domain (E_{\text{cond}},D_{\text{cond}}) . Each encoder outputs mean \mu and standard deviation \sigma , defining a Gaussian posterior q(z|\boldsymbol{\cdot})=\mathcal{N}(\mu,\mathrm{diag}\,\sigma^{2}) . We reconstruct \widehat{u}=D_{\boldsymbol{\cdot}}(E_{\boldsymbol{\cdot}}(u)) for u\!\in\!\{y,x_{\text{c}}\} and optimize: \displaystyle\mathcal{L}_{\text{AE}} \displaystyle=\|u-\widehat{u}\|_{1}+\lambda_{\text{KL}}\,\text{KL}\!\big(q(z|u)\,\|\,\mathcal{N}(0,I)\big) (1) \displaystyle+\lambda_{\text{edge}}\,\|\nabla\widehat{u}-\nabla u\|_{1}+\lambda_{\text{adv}}\,\mathcal{L}_{\text{GAN}}(\widehat{u},u) Beyond the standard KL autoencoder objective, we add an edge-aware loss \mathcal{L}_{\text{edge}}=\|\nabla\widehat{u}-\nabla u\|_{1} based on first-order spatial gradients to encourage sharp tissue boundaries in reconstructions. After training both autoencoders, we freeze E_{\text{PSR}} and fine-tune E_{\text{cond}} to align the conditioning latent space with the target PSR latent space for Stage 2 diffusion process. Stage 2: Conditional Diffusion in the PSR Latent. We adopt the standard LDM framework, defining a forward noising process on the clean PSR latent z_{\text{y}}\!\sim\!q(\boldsymbol{\cdot}\mid y)