Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Models Karim Kadry, Abdalla Abdelwahed, Shoaib Goraya, Ajay Manicka, Naravich Chutisilp, Farhad R. Nezami, Elazer R. Edelman, [0.35em] MIT, Cambridge, MA, USA American University in Cairo, New Cairo, Egypt Abstract We present Anatomica: an inference-time framework for generating multi-class anatomical voxel maps with localized geo-topological control. During generation, we use cuboidal control domains of varying dimensionality, location, and shape, to slice out relevant substructures. These local substructures are used to compute differentiable penalty functions that steer the sample towards target constraints. We control geometric features such as size, shape, and position through voxel-wise moments, while topological features such as connected components, loops, and voids are enforced through persistent homology. Lastly, we implement Anatomica for latent diffusion models, where neural field decoders partially extract substructures, enabling the efficient control of anatomical properties. Anatomica applies flexibly across diverse anatomical systems, composing constraints to control complex structures over arbitrary dimensions and coordinate systems, thereby enabling the rational design of synthetic datasets for virtual trials or machine learning workflows. Figure 1 : Anatomica is a compositional diffusion-guidance framework for generating segmentations based on anatomical features that are localized within cuboidal control domains. Left : We generate voxel maps according to localized target geometry (size, shape, and position) visualized as red ellipsoids. Right : We generate voxel maps according to target topology (components, loops, and voids). 1 Introduction Anatomical form plays a vital role in dictating the function and dysfunction of physiological systems. By virtually modelling patient-specific organ systems as 3D voxelized segmentations, we can leverage numerical simulators to reveal structure-function relationships that inform clinical research and medical device design. Such use cases include the simulation of clinical trials to evaluate medical devices Sarrami-Foroushani et al. 2021 ; Viceconti et al. 2021 ; Abadi et al. 2020 , or simulating image-formation to create robust datasets for machine learning workflows Gopalakrishnan et al. 2024 ; Gopalakrishnan and Golland 2022 ; Dey et al. 2025 ; Billot et al. 2023 ; Fernandez et al. 2024 . Due to the sparsity and imbalances inherent to real-world datasets, there has been growing interest in augmenting anatomical datasets with synthetic data. A key advantage of using generative models over patient datasets lies in their controllability. Conditional generation of medical images based on anatomical or demographic information has been shown to improve the performance of machine learning classifiers and segmentation networks Ktena et al. 2024 ; Moroianu et al. 2025 ; Fernandez et al. 2024 . However, conditional generation of 3D multi-class segmentations based on anatomical features remains difficult. These features encompass both geometry (shape and size) and topology (connected components, loops, or voids). Moreover, such features are defined compositionally over multiple substructures within the segmentation, with varying dimensionality (e.g., 3D vs 2D), and across varying coordinate systems (e.g., Cartesian vs curvilinear). The ideal generative model must not only control such features in a precise and compositional manner, but also offer control mechanisms that are intuitive to use. We introduce Anatomica: an inference-time framework for controlling anatomical latent diffusion models based on arbitrarily localized properties related to geometry and topology. We formulate guidance through two key stages for each sampling step. First, we differentiably parse voxel-space segmentations to extract anatomical substructures with varying dimensionality over arbitrary coordinate systems. Second, we measure geometric and topological properties in a differentiable manner and apply potential functions to guide the reverse sampling process. Lastly, we adapt this guidance framework for latent diffusion models through neural field decoders which map arbitrary query points in latent space to voxel space, enabling the efficient measurement of anatomical properties from latent space. We advance the state-of-the-art in the following ways: • Differentiable and Localized Substructure Extraction : We introduce a modular method to differentiably parse localized and anatomically relevant substructures from voxel-space segmentations (V-parsing). We base our method on cuboidal control domains with varying scales, positions, and orientations. By arranging multiple control domains of varying dimensionality across relevant coordinate systems, we enable the characterization of a wide array of anatomical systems and structures. • Unified Geo-Topological Measurement and Guidance : We demonstrate that applying differentiable measurement and potential functions over anatomical substructures allows us to constrain localized properties through diffusion guidance. This includes geometric properties such as size, shape, position, and orientation, as well as topological properties such as the number of components, loops, or voids. We show that, by combining different control domains and potential, we unlock a rich design space for compositional anatomical control, within which a wide variety of structures can be controllably generated. • Latent Diffusion Guidance with Neural fields : We show that neural field decoders enable the efficient measurement of voxel-space properties within control domains directly from latent space during sampling (L-parsing). By exploiting the ability of neural fields to decode arbitrarily discretized point grids, we avoid the computational overhead of full-volume decoding. We introduce two partial decoding strategies: coarse L-parsing decodes globally at reduced spatial resolution, while localized L-parsing decodes local regions at high resolution. 2 Related Work Geometric Control for Generative Models of Anatomy Geometric features such as size and shape play a crucial role in biophysical dynamics Fabris et al. 2022 ; Kadry et al. 2021 . Modelling anatomy with simple shapes such as cylinders Aróstica et al. 2025 ; Madani et al. 2019 provides control over form but not realism. Statistical shape models Dou et al. 2022 ; Qiao et al. 2025 ; Williams et al. 2022 represent realistic variation via global shape vectors Hermida et al. 2024 ; Williams et al. 2022 but are not as interpretable or editable. To bridge this gap, recent studies conditionally train generative models based on size-based measures de Wilde et al. 2025 ; Kadry et al. 2024b . Recently, Kadry et al. 2025a proposed inference-time geometric guidance via differentiable geometry, expanding control to size, position, and shape, in a compositional manner over multi-class anatomy. However, this method was limited to globally defined geometric properties in 3D. In this work, we extend geometric guidance to arbitrarily localized attributes based on cuboidal control domains of varying scale, position, orientation, and dimensionality. By arranging control domains over non-Cartesian coordinate systems, we enable significantly more complex compositional control of physiologically relevant geometric features. Figure 2 : Differentiable measurement of anatomical properties from multi-class voxel maps. A : We differentiably parse relevant substructures from anatomical voxel maps for localized measurement. B : We spatially transform cuboidal primitives (template domains) into control domains that slice into anatomical structures (V-parsing). C : The substructure is then differentiably measured in terms of geometric properties; as well as D : persistent homology-based topological properties. Topological Deep Learning Topological properties such as the number of components, loops, or voids also play a crucial role in modulating biophysical dynamics Kong et al. 2024 . To regularize machine learning workflows in a differentiable manner, persistent homology (PH) can be used Brüel-Gabrielsson et al. 2019 to measure the continuous-valued persistence of topological features. PH-based topological losses have been used for the training Clough et al. 2020 and test-time adaptation Byrne et al. 2022 of segmentation networks. Similarly, PH has been used to conditionally train diffusion models of 2D binary label maps Gupta et al. 2025 and 3D surfaces Hu et al. 2024 . In contrast to using topological losses to update network weights, we use PH for inference-time control generative models that sample multi-class 3D anatomical segmentations without conditional training. This enables us to flexibly constrain topological features in a plug-and-play manner without retraining. Spatial Conditioning for Generative Models Spatial control of generative models relies on two main strategies. The first conditions models on mid-level representations (e.g., bounding boxes, ellipsoid parameters) Nie et al. 2024 ; Feng et al. 2025 ; Hertz et al. 2022 ; Koo et al. 2023 ; Anonymous 2024 . The second involves guidance methods, such as self-guidance Epstein et al. 2023 , which employs attention-based losses for basic geometric control (size, position) in text-to-image models, but it is not suited for multi-label segmentations, nor is it adapted for complex constraints needed to describe anatomical shape. In our work, we extend energy-based guidance to localized control over geometry and topology by introducing differentiable potentials for 3D multi-component anatomical voxel maps based on substructure-specific properties. We show that this enables a rich design space for anatomical control, within which a wide variety of organs can be controllably generated. 3 Methodology 3.1 Anatomical Latent Diffusion Models Autoencoder with Neural Field Decoder We develop our variational autoencoder based on hybrid implicit-explicit representations Peng et al. 2020 . Our dataset consists of 3D segmentation volumes \mathbf{V}\in\mathbb{R}^{C\times H\times W\times D} with C tissue channels and (H,W,D) spatial dimensions. During training, a convolutional encoder \mathcal{E} first encodes the voxelized segmentation map \mathbf{V} into a voxelized latent grid representation \mathbf{z}=\mathcal{E}(\mathbf{V}) , where \mathbf{z}\in\mathbb{R}^{c\times h\times w\times d} comprises c channels and spatial dimensions (h,w,d)=(H/f,\,W/f,\,D/f) for an integer downsampling factor f . To decode back into voxel space, a 3D query point grid \mathbf{X}^{q}\in\mathbb{R}^{H\times W\times D\times 3} is used to compute a latent point grid through the latent slice operator \mathcal{T}^{l}[\mathbf{X}^{q}]:\mathbb{R}^{3}\to\mathbb{R}^{c} which applies trilinear interpolation for each query point as in Jaderberg et al. 2015 . The resulting latent grid is then pointwise decoded with a neural field decoder \mathcal{F}:\mathbb{R}^{c}\to\mathbb{R}^{C} into the predicted voxel map \bar{\mathbf{V}}\in\mathbb{R}^{C\times H\times W\times D} : Figure 3 : Efficient parsing of anatomical substructures during diffusion guidance. A : During guidance, we parse relevant substructures directly from the clean latent prediction with a neural field decoder (L-parsing). B : In coarse L-parsing, we use a coarse grid to decode globally defined substructures at low spatial resolution. C : In localized L-parsing, we use a similar grid size but spatially transform the template point grid to decode localized substructures at high spatial resolution. \bar{\mathbf{V}}=\underbrace{\mathcal{F}[\mathbf{X}^{q}]}_{\text{Decode Latent}}\circ\underbrace{\mathcal{T}^{l}[\mathbf{X}^{q}](\mathbf{z})}_{\text{Slice Latent}}. (1) Here, \mathcal{F} is parametrized as a multi-layer perceptron that takes in the interpolated latent point grid and positionally encoded query points, and \circ denotes function composition. Unconditional Diffusion Model We use an unconditional latent diffusion model (LDM) as a prior over 3D anatomical segmentations. In the forward process, data samples are progressively corrupted by adding Gaussian noise \mathbf{n} through the relation \mathbf{z}_{\sigma}=\mathbf{z}+\mathbf{n} where \mathbf{n}\sim\mathcal{N}(\mathbf{0},\sigma^{2}\mathbf{I}) . Similar to Karras et al. 2022 , we aim to learn the score function \nabla_{\mathbf{z}_{\sigma}}\log p(\mathbf{z}_{\sigma};\sigma) that defines the reverse diffusion process: d\mathbf{z}_{\sigma}=-2\sigma\nabla_{\mathbf{z}_{\sigma}}\log p(\mathbf{z}_{\sigma};\sigma)\,dt+\sqrt{2\sigma}\,d\mathbf{w}\, (2) where d\mathbf{w} is the Wiener process. This score function \nabla_{\mathbf{z}_{\sigma}}\log p(\mathbf{z}_{\sigma};\sigma)=(D_{\theta}(\mathbf{z}_{\sigma};\sigma)-\mathbf{z}_{\sigma})/\sigma^{2} can be expressed via a denoising function D_{\theta} parametrized by a 3D U-Net. The neural network is trained by minimizing the clean data prediction objective L=\mathbb{E}_{\sigma,\mathbf{z},\mathbf{n}}\left[\omega(\sigma)\|D_{\theta}(\mathbf{z}_{\sigma};\sigma)-\mathbf{z}\|_{2}^{2}\right] , with \omega(\sigma) balancing loss contributions across noise levels. 3.2 Guidance via Anatomical Potential Functions We assume the voxel map \mathbf{V} contains K substructures \mathbf{S}_{k}\in[0,1]^{\alpha\times\beta\times\gamma} of interest, where \alpha,\beta,\gamma are grid size dimensions. Substructures represent anatomical regions of interest that may be comprised of single tissues (e.g., cross-sectional slices of an aorta) or combinations thereof (e.g., left and right atria). As shown in Fig. 2 , each substructure can be measured in terms of geometric properties (mass, position, size, shape, orientation) and topological properties (connectivity, presence of loops or voids). Differentiable Geometric Measurement Given a single parsed substructure \mathbf{S}_{k} , we aim to differentiably extract the geometric moments by numerically integrating the zeroth, first, and second-order moments of the substructure. Here, the zeroth moment represents the mass m_{k}\in\mathbb{R} , the first moment represents the centroid \mathbf{p}_{k}\in\mathbb{R}^{3} , and the second moment represents the covariance matrix \boldsymbol{\Sigma}_{k}\in\mathbb{R}^{3\times 3} . To do this, we follow Kadry et al. 2025a , but instead compute the moments for substructures inside arbitrarily sized cuboidal control domains, rather than globally. We first define \mathbf{\Omega}_{k}\in\mathbb{R}^{(\alpha\beta\gamma)\times 1} as the flattened substructure voxel grid \mathbf{S}_{k} and \mathbf{r}_{k}\in\mathbb{R}^{(\alpha\beta\gamma)\times 3} as the normalized voxel coordinates between 0 and 1. The moments are then computed as: \displaystyle m_{k} \displaystyle=\mathbf{1}^{T}\cdot\mathbf{\Omega}_{k}\hskip 10.00002pt\mathrm{and}\hskip 10.00002pt\mathbf{p}_{k}=\frac{\mathbf{\Omega}_{k}^{T}\mathbf{r}_{k}}{m_{k}}, \displaystyle\boldsymbol{\Sigma}_{k} \displaystyle=\frac{1}{m_{k}}\mathbf{r}_{k}^{T}\operatorname{diag}(\mathbf{\Omega}_{k})\,\mathbf{r}_{k}-\mathbf{p}_{k}\mathbf{p}_{k}^{T}. (3) Here, \mathbf{1}^{T} is the all-ones vector, and \operatorname{diag}(\cdot) refers to diagonal matrix embedding. Geometric Decomposition and Size Normalization As the covariance matrix implicitly contains information on mass, we aim to obtain a scale-normalized covariance matrix that relates only to orientation and relative aspect ratios. We decompose the covariance as \boldsymbol{\Sigma}_{k}=\mathbf{U}_{k}\tilde{\boldsymbol{\Lambda}}_{k}\mathbf{U}_{k}^{T}=v_{k}\mathbf{U}_{k}\boldsymbol{\Lambda}_{k}\mathbf{U}_{k}^{T} , where size v_{k}\in\mathbb{R} = \operatorname{tr}(\boldsymbol{\Sigma}_{k}) is the trace of the covariance matrix, shape \boldsymbol{\Lambda}_{k}\in\mathbb{R}^{3\times 3} = \tilde{\boldsymbol{\Lambda}}_{k}/v_{k} is the eigenvalue matrix \boldsymbol{\Lambda}_{k} normalized by the trace, and orientation \mathbf{U}_{k}\in\mathbb{R}^{3\times 3} is an orthonormal matrix. We thus define the scale-normalized covariance matrix as \boldsymbol{\Sigma}^{n}_{k}=\mathbf{U}_{k}\boldsymbol{\Lambda}_{k}\mathbf{U}_{k}^{T} . Geometric Potential Functions After computing the geometric features, we aim to penalize the deviations from the target geometric features \bar{\mathbf{G}}_{k} through a geometric potential function \mathcal{L}^{\text{geo}}_{k} . We formulate this potential function as a weighted combination of mean squared error losses \mathcal{L}_{\text{MSE}} : \displaystyle\mathcal{L}^{\text{geo}}_{k}=\lambda_{0}\mathcal{L}_{\text{MSE}}(m_{k},\bar{m}_{k})+\lambda_{1}\mathcal{L}_{\text{MSE}}(\mathbf{p}_{k},\bar{\mathbf{p}}_{k}) \displaystyle+\lambda_{2}\mathcal{L}_{\text{MSE}}(\boldsymbol{\Sigma}^{n}_{k},\bar{\boldsymbol{\Sigma}}^{n}_{k}). (4) Here, [\lambda_{0},\lambda_{1},\lambda_{2}] are the weighting factors for the mass, position, and normalized covariance losses, respectively. Adaptive Mass Weighting Given that geometric features are now defined locally, rather than globally, our potential functions must account for numerical instability in the centroid and covariance-based losses resulting from empty voxels. We address this by adaptively setting \lambda_{1}=\lambda_{2}=0 when the mass is below a specified threshold. Differentiable Topological Measurement We use persistent homology (PH) to measure the presence of topological features within the substructure \mathbf{S}_{k} considered as a cubical complex, similar to Gupta et al. 2025 . We consider super-level sets of \mathbf{S}_{k} , which are the set of voxels above a threshold value \tau . By decreasing \tau , we obtain a filtration of nested sets, all of which are used to extract topological features such as connected components (0D features), loops (1D features), and voids (2D features). The output of this process is a set of persistence points p\in\mathcal{X}_{k} which include the birth b and death d thresholds for all topological features. Intuitively, persistent features have a large interval between their birth and death thresholds. We use the Cubical Ripser library Kaji et al. 2020 to compute the PH of \mathbf{S}_{k} . Softmax Temperature Tuning As our substructure is derived from softmaxed multi-class probability maps, we found that the gradient of the topological potential was too low in regions where the probability was close to 0 or 1. To address this, when decoding the substructure from the latent space, we apply softmax with an increased temperature for topological guidance, enabling the gradient to pass through during backpropagation. Topological Potential Functions To enforce topological structures within \mathbf{S}_{k} during the reverse diffusion process, we partition the persistence set into disjoint sets \mathcal{X}_{k}=\mathcal{Y}_{k}\cup\mathcal{Z}_{k} consisting of points that should be preserved \mathcal{Y}_{k} or suppressed \mathcal{Z}_{k} based on a topological prior \mathcal{B}_{k}\in\mathbb{R}^{3} , which specifies the desired features for the components, loops, and voids, respectively. To differentiably compute the topological potential, we sample the voxel intensities from \mathbf{S}_{k} at the birth r^{p}_{b} and death r^{p}_{d} coordinates for each persistence point p . We then maximize or minimize the persistence of each feature through the following potential functions: Figure 4 : Geometric Control Tasks . We define a variety of relevant tasks by varying the selected tissues, template domain grid size, and control domain-specific spatial transforms. \scalebox{0.75}{ $\mathcal{L}^{\text{topo}}_{k}=-\sum_{p\in\mathcal{Y}_{k}}\underbrace{|\mathbf{S}_{k}(r^{p}_{b})-\mathbf{S}_{k}(r^{p}_{d})|^{2}}_{\text{Preserve Loss}}+\sum_{p\in\mathcal{Z}_{k}}\underbrace{|\mathbf{S}_{k}(r^{p}_{b})-\mathbf{S}_{k}(r^{p}_{d})|^{2}}_{\text{Suppress Loss}}$ }. (5) Gradient-Based Guidance Following Kadry et al. 2025a , we guide the diffusion process using the gradient derived from anatomical potential functions. At each sampling step, we first denoise the intermediately noised latent \mathbf{z}_{\sigma} to obtain a clean latent prediction \hat{\mathbf{z}}_{0}=D_{\theta}(\mathbf{z}_{\sigma};\sigma) , which is then parsed into K substructures \mathbf{S}_{k} as described in Section 3.3 . We compute a composite anatomical potential \mathcal{L}=\frac{1}{K}\sum_{k}(\lambda_{\text{geo}}\mathcal{L}^{\text{geo}}_{k}+\lambda_{\text{topo}}\mathcal{L}^{\text{topo}}_{k}) where \lambda_{\text{geo}} and \lambda_{\text{topo}} weight the geometric and topological potentials ( Fig. 3 , top row). The guided denoising step is then applied as follows: \underbrace{D_{\theta}^{w}(\mathbf{z}_{\sigma};\sigma)}_{\text{Guided Denoising}}=\underbrace{D_{\theta}(\mathbf{z}_{\sigma};\sigma)}_{\text{Uncond.\ Denoising}}-\underbrace{\sigma^{2}\cdot\nabla_{\mathbf{z}_{\sigma}}\mathcal{L}}_{\text{Anatomical Guidance}} . (6) 3.3 Substructure Parsing for Diffusion Guidance Voxel-Space Parsing Given a predicted voxel-space segmentation map \hat{\mathbf{V}} , we can parse K substructures \mathbf{S}_{k} with voxel-space substructure parsing ( Fig. 2 ), which we will refer to as V-parsing . V-parsing extracts a substructure by first using the boolean subset operator \mathcal{U}[\mathbf{u}]:\mathbb{R}^{C}\to\mathbb{R} ; parametrized by a selection vector \mathbf{u}\in\{0,1\}^{C} for elementwise extraction and recombination of tissues from \hat{\mathbf{V}} into a structure voxel map \hat{\mathbf{S}}_{k} . By taking different tissue combinations, we enable the control of various structures within the segmentation. The structure slicing operator \mathcal{T}^{s}[\mathbf{X}_{k}]:\mathbb{R}\to\mathbb{R} then samples voxelwise intensity values from \hat{\mathbf{S}}_{k} at locations specified by a cuboidal control domain \mathbf{X}_{k}\in\mathbb{R}^{\alpha\times\beta\times\gamma\times 3} discretized into a lattice-like point grid. The final relation is given by: \mathbf{S}_{k}=\underbrace{\mathcal{T}^{s}[\mathbf{X}_{k}]}_{\text{Slice Structure}}\circ\underbrace{\mathcal{U}[\mathbf{u}](\hat{\mathbf{V}})}_{\text{Subset Voxel}}. (7) Spatial Transformation of Template Domains Control domains \mathbf{X}_{k} are obtained by spatially transforming a template point grid \mathbf{X}^{\text{temp}}_{k}\in\mathbb{R}^{\alpha\times\beta\times\gamma\times 3} using affine transformation parameters \mathbf{A}_{k}=[\mathbf{R}_{k},\mathbf{s}_{k},\mathbf{t}_{k}] , where \mathbf{R}_{k}\in\mathbb{R}^{3\times 3} is a rotation matrix, \mathbf{s}_{k}\in\mathbb{R}^{3} is a scaling vector, and \mathbf{t}_{k}\in\mathbb{R}^{3} is a translation vector. The spatial transformation is defined as: \mathbf{X}_{k}=\mathbf{R}_{k}\operatorname{diag}(\mathbf{s}_{k})\mathbf{X}^{\text{temp}}_{k}+\mathbf{t}_{k}. (8) This formulation enables flexible substructure parsing across multiple design axes ( Fig. 4 ). For example, the template grid size controls the discretization (coarse to fine) and dimensionality (3D to 1D) of the parsed substructure. The affine parameters \mathbf{A}_{k} can be defined based on custom coordinate systems and enable the localization of substructures under varying extents, orientations, and positions in 3D space. Latent-Space Parsing Rather than decoding the entire voxel grid and subsequently slicing out substructures, we propose to use neural field decoders to directly parse substructures from latent space ( L-parsing ). This is accomplished by applying the latent slicing operator \mathcal{T}^{l}[\mathbf{X}_{k}] on the clean predicted latents \hat{\mathbf{z}}_{0} followed by the neural field decoder \mathcal{F}[\mathbf{X}_{k}] and the boolean subset operator \mathcal{U}[\mathbf{u}] : \mathbf{S}_{k}=\underbrace{\mathcal{U}[\mathbf{u}]}_{\text{Subset Voxel}}\circ\underbrace{\mathcal{F}[\mathbf{X}_{k}]}_{\text{Decode Latent}}\circ\underbrace{\mathcal{T}^{l}[\mathbf{X}_{k}](\hat{\mathbf{z}}_{0})}_{\text{Slice Latent}}. (9) Partial Decoding Strategies Neural field representations offer a key advantage in that they enable decoding from arbitrary point sets. We introduce two L-parsing strategies that apply partial decoding by using point grids with small grid sizes, enabling fast guidance at inference time. To efficiently measure global-level anatomical properties that are invariant to spatial resolution, we apply coarse L-parsing by using a template grid \mathbf{X}^{\text{temp}}_{k} with a low grid discretization [\alpha