Generating Novel Brain Morphology by Deforming Learned Templates
Alan Q. Wang, Fangrui Huang, Bailey Trang, Wei Peng, Mohammad Abbasi, Kilian Pohl, Mert Sabuncu, Ehsan Adeli · 2025
Read Generating Novel Brain Morphology by Deforming Learned Templates by Alan Q. Wang, Fangrui Huang, Bailey Trang, Wei Peng, Mohammad Abbasi, Kilian Pohl, Mert Sabuncu, Ehsan Adeli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03778.txt.
Introduction
Wang and seven co-authors present a study from twenty twenty-five on making new brain scans by warping a learned template. Fake brain images must look plausible and match traits like age or disease. Direct image synthesis can miss fine shape details. Their method learns a template brain, then deforms it with a generated warp field. A diffusion model produces the warp rather than the image itself. An encoder blends image and template into a code that drives the warp. Alignment loss trains the whole system end to end.
Their method beats generative baselines on diversity and trait match. Shape measures on the fake brains also track real ones better. The code is posted in public for reuse. The authors aim at richer synthetic data for brain research.
Catalog record
| CREATOR | Alan Q. Wang, Fangrui Huang, Bailey Trang, Wei Peng, Mohammad Abbasi, Kilian Pohl, Mert Sabuncu, Ehsan Adeli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-03778
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generating Novel Brain Morphology by Deforming Learned Templates Alan Q. Wang, Fangrui Huang, Bailey Trang, Wei Peng, Mohammad Abbasi, Kilian Pohl, Mert Sabuncu, Ehsan Adeli Abstract Designing generative models for 3D structural brain MRI that synthesize morphologically-plausible and attribute-specific (e.g., age, sex, disease state) samples is an active area of research. Existing approaches based on frameworks like GANs or diffusion models synthesize the image directly, which may limit their ability to capture intricate morphological details. In this work, we propose a 3D brain MRI generation method based on state-of-the-art latent diffusion models (LDMs), called MorphLDM, that generates novel images by applying synthesized deformation fields to a learned template. Instead of using a reconstruction-based autoencoder (as in a typical LDM), our encoder outputs a latent embedding derived from both an image and a learned template that is itself the output of a template decoder; this latent is passed to a deformation field decoder, whose output is applied to the learned template. A registration loss is minimized between the original image and the deformed template with respect to the encoder and both decoders. Empirically, our approach outperforms generative baselines on metrics spanning image diversity, adherence with respect to input conditions, and voxel-based morphometry. Our code is available at https://github.com/alanqrwang/morphldm . Figure 1: (top) Graphical depiction of MorphLDM. The encoder takes in an image and a template that is the output of a template decoder, concatenated along the channel dimension. The decoder outputs a deformation field \bm{v} that is applied to the template. Encoder-decoder networks in blue are trained in the first stage on a registration loss (similar to LDM). The diffusion model (DM) in green is trained on the learned latents in the second stage. (bottom) Visualization of denoising steps t\in[T] of LDM (left) and MorphLDM (right). While LDM’s decoder maps from a synthesized latent to an image directly, MorphLDM’s decoder maps to a synthesized deformation field, which is subsequently applied to the learned template. 1 Introduction Morphology is an important characteristic of neuroimaging, known to be related to attributes and/or phenotypes such as age, sex, and disease state [ 20 , 31 ] . A growing literature aims to design generative models for structural brain magnetic resonance imaging (MRI), which synthesize realistic morphology and are specific to conditioned attributes such as age and sex [ 21 , 23 ] . These models offer the potential to augment limited training data [ 24 ] , generate counterfactuals [ 15 ] , and increase interpretability [ 30 ] . In computer vision, generative models have found success in natural image domains, driven largely by GANs [ 8 , 10 , 35 ] and diffusion models [ 13 , 26 ] . These models work by learning the data distribution p(\bm{x}) from which the data is assumed to be sampled. Although highly general and capable of capturing the variety inherent in naturalistic images, this approach applied to neuroimaging may fail to capture intricacies and subtleties crucial to morphology, especially in the context of high-dimensional 3D volumetric data and data-limited scenarios. Specific to neuroimaging, previous work has shown that much of the variability between individual anatomies can be captured by geometric and topological changes [ 4 , 5 ] . This motivates a line of work that, instead of directly synthesizing images, synthesizes dense deformation fields that are applied to a template image. However, in addition to using less powerful generative models like GANs, prior work either precomputes deformation fields using an expensive registration method [ 3 , 37 ] or fixes the template, thus biasing and limiting the model’s ability to adequately capture the image distribution [ 3 , 17 , 36 ] . In this work, we propose a 3D brain MRI generation method based on state-of-the-art latent diffusion models (LDMs), called MorphLDM, that generates novel images by applying synthesized deformation fields to a learned template. MorphLDM differs from LDMs in the design of the encoder/decoder. First, a learned template is outputted by a template decoder, optionally conditioned on image-level attributes. Then, an encoder takes in both an image and the template and outputs a latent embedding; this latent is passed to a deformation field decoder, whose output deformation field is applied to the template. Finally, a registration loss is minimized between the original image and the deformed template with respect to the encoder and both decoders. Subsequently, a diffusion model is trained on these learned latent embeddings. To synthesize an image, MorphLDM generates a novel latent in the same way as standard LDMs. The decoder maps this latent to its corresponding deformation field, which is subsequently applied to the learned template. We find that synthetic samples generated by our approach not only outperform powerful baselines on standard image generation metrics like FID, but also better capture input conditions known to be related to morphology, including age and sex. Additionally, we perform a voxel-based morphometry [ 33 ] analysis and find that our models generate more realistic structures in terms of regional volume compared to baselines. To the best of our knowledge, MorphLDM is the first MRI diffusion model that synthesizes deformation fields applied to a learned template. Our approach is efficient and data-driven in the sense that both the deformation fields and the template are learned simultaneously, without requiring precomputation of deformation fields or making a specific choice for the template. 2 Background 2.1 Brain MRI Generation Generative adversarial networks (GANs) and their extensions have been widely explored in the context of brain MRIs, including using Wasserstein GANs [ 11 ] , incorporating variational autoencoders (VAEs) and code discriminators [ 18 ] , and leveraging segmentations [ 8 ] . Autoregressive (AR) approaches generate samples in an autoregressive fashion and require discrete tokenization of images, typically done with a vector-quantized (VQ)-VAE [ 7 , 28 ] . More recently, latent diffusion models (LDMs) have become the state-of-the-art generative model for brain MRIs [ 21 , 23 ] , and are trained in two steps. In the first stage, an autoencoder \mathcal{D}(\mathcal{E}(\bm{x}))=\mathcal{D}(\bm{z}) is trained on a reconstruction loss \mathcal{L}_{sim} to learn a latent space. In the second stage, a diffusion model is trained on the latent space defined by \bm{z}=\mathcal{E}(\bm{x}) ; specifically, a network \epsilon_{\omega} is trained to perform time-conditioned denoising of the latent embeddings: \operatornamewithlimits{argmin}_{\omega}\mathbb{E}_{\mathcal{E}(\bm{x}),\epsilon\sim\mathcal{N}(0,1),t\in[T]}||\epsilon-\epsilon_{\omega}(\bm{z}_{t},t,\bm{c})||^{2}_{2}, (1) where \mathbb{E} is expected value, \mathcal{N}(0,I) is the standard normal distribution, and \bm{c} is any conditioning information associated with the image. 1 1 1 Typically, \epsilon_{\omega} is implemented as a time-conditional UNet [ 26 ] , where the conditioning information \bm{c} is included via cross-attention with the intermediate feature maps. 2.2 Templates Deformable templates, or atlases, are common in computational anatomy. Research is dedicated to constructing unbiased templates such that the deformation fields from these templates to individual images can be analyzed to understand population variability [ 2 , 4 , 9 ] . More recently, template learning approaches use neural networks to learn templates that are data-driven and attribute-specific [ 5 , 6 , 27 ] . These approaches aim to build decoders that learn templates conditioned on specified attributes. In this work, we take inspiration from template learning approaches, where our goal is to learn a universal template concurrently with a deformation field synthesizer