LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators

Xinhe Zhang ​, Yuyang Zhang ​, Pengfei Jin, Arnau Marin-Llobet, Na Li, Quanzheng Li · 2026

Read LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators by Xinhe Zhang ​, Yuyang Zhang ​, Pengfei Jin, Arnau Marin-Llobet, Na Li, Quanzheng Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19060.txt.

Introduction

Xinhe Zhang and five colleagues build LiFT, a method for making three-dimensional medical scans from two-dimensional slice generators. Rather than modelling a whole volume at once, it treats the volume as an ordered path, tracking how body structures appear, shift, and vanish with depth. A loss function aligns generated slices with real scans, while a mixing step keeps neighbouring slices coherent across scan types. Tests on brain imaging show slice quality preserved at far lower computing cost.

Catalog record

CREATORXinhe Zhang ​, Yuyang Zhang ​, Pengfei Jin, Arnau Marin-Llobet, Na Li, Quanzheng Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54850
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19060

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators Xinhe Zhang ​, Yuyang Zhang ​, Pengfei Jin, Arnau Marin-Llobet, Na Li, Quanzheng Li Abstract High-resolution 3D medical image generation remains challenging because fully volumetric models are computationally expensive, while efficient 2D slice generators often fail to preserve anatomical consistency across the third dimension. We propose LiFT, a framework for Lifted inter-slice Feature Trajectories that factorizes 3D volume synthesis into per-slice image generation and inter-slice trajectory learning. Rather than modeling the volumetric distribution end-to-end, LiFT treats a volume as an ordered trajectory in feature space, capturing how anatomical structures appear, transform, and disappear across depth. A tri-planar drifting loss aligns the trajectory of generated slices with the trajectories of real volumes, enabling distributional learning over inter-slice progressions in unconditional generation; in paired translation, a bidirectional z z -context mixer trained against the registered target supplies through-plane coherence while preserving per-slice fidelity. We evaluate LiFT on BraTS 2023 (unconditional and missing-modality MR) and SynthRAD2023 (MR-to-CT). Across these settings, LiFT preserves per-slice quality, approaches the reported cWDM missing-MR reconstruction quality at ∼ \sim 135 × 135\times lower inference cost (without formal equivalence testing), and improves through-plane coherence on MR-to-CT relative to a no-mapper ablation, demonstrating that lightweight inter-slice trajectory learning is a viable route to high-resolution 3D medical synthesis. 1 Introduction Medical image synthesis is fundamentally a three-dimensional (3D) problem because the anatomical and pathological targets of interest are volumetric. Tumors, ventricles, edema, white-matter tracts, skull boundaries, air cavities, and bone interfaces are not interpreted as isolated two-dimensional (2D) slices, but through their spatial continuity across a volume. This volumetric structure is also required by many downstream clinical and computational workflows, including segmentation, registration, dose planning, longitudinal monitoring, and image-guided intervention [ 1 ] . Generative models for 3D magnetic resonance (MR) and computed tomography (CT) imaging can help address data scarcity and data-sharing constraints [ 2 ] , support augmentation and benchmarking [ 3 , 4 ] , and enable cross-modality synthesis, such as missing-modality MR completion [ 5 ] or MR-to-CT synthesis for MR-only radiotherapy [ 6 ] . Existing generative approaches address this volumetric requirement in different ways. Fully 3D generators model inter-slice consistency directly, but often require substantial compromises in memory, resolution, architecture, or training strategy. Prior work has therefore explored hierarchical sub-volume training [ 7 ] , dense 3D denoising [ 8 , 9 ] , latent-space generation [ 10 , 11 ] , wavelet-domain generation [ 12 , 13 ] , and patch-volume modeling [ 14 ] . Despite their differences, these methods share the assumption that volumetric coherence should be learned primarily by a 3D generator. In contrast, slice-wise 2D synthesis can exploit mature high-resolution image-generation backbones and is often more efficient at preserving native in-plane detail. However, independently generated slices do not necessarily form anatomically coherent volumes. Recent two-and-a-half-dimensional [ 15 ] , adjacent-slice [ 16 ] , multi-view [ 17 ] , slice-based latent-diffusion [ 18 ] , and triplane-aware [ 19 ] methods attempt to reduce this gap by injecting limited through-plane context into otherwise slice-oriented synthesis. This line of work suggests that the key challenge is not merely generating realistic slices, but coordinating them across depth. In many medical synthesis settings, high-resolution in-plane anatomy can be modeled effectively by 2D image-generation architectures. This is partly because clinical MR and CT volumes are often anisotropic, with substantially higher native resolution within each slice than across slices [ 20 , 21 ] , and partly because 2D backbones avoid the memory and optimization burden of fully volumetric generation [ 22 , 23 ] . However, anatomical structures do not evolve independently from slice to slice. Tumors, edema, ventricles, cortical boundaries, bone interfaces, and air cavities follow continuous spatial trajectories through the volume. As a result, purely slice-wise synthesis can preserve local image detail while still producing unrealistic through-plane discontinuities or inconsistent anatomy [ 15 , 16 ] . These observations motivate a factorized view of medical volume synthesis, in which high-resolution slice formation and through-plane anatomical organization are treated as related but separable problems. Figure 1: LiFT decouples high-resolution 2D in-plane synthesis from through-plane coherence modeling, adapting supervision to unconditional or paired volume settings. Motivated by these observations, we propose LiFT , a framework for Lifted inter-slice Feature Trajectories in medical volume synthesis. LiFT keeps the core synthesis pathway predominantly 2D while introducing lightweight through-plane modeling to impose volume-level coherence (Fig. 1 ). The form of volume-level supervision depends on the available data: in the unconditional setting, where no paired target volume exists for each generated sample, LiFT encourages coherence distributionally by matching tri-planar trajectories between generated and real volumes; in the paired translation setting with exact target volumes available, LiFT imposes coherence directly through supervised losses and depth-axis context mixing. Contributions. We propose LiFT, a 2D-to-volume synthesis framework for medical image generation and translation. Our main contributions are: • Factorized 2D-to-volume synthesis. LiFT lifts a 2D synthesizer into a volumetric synthesis model using a small learned depth-indexed feature trajectory. The framework adapts its volume-level supervision to the data regime: in the unconditional setting, LiFT-U uses a frozen 2D slice generator and trains a compact depth mapper with tri-planar distributional trajectory matching over completed volumes; in the paired setting, LiFT-C augments a native-resolution 2D translator with a lightweight inter-slice context module over per-slice bottleneck states, enabling direct supervised through-plane context modeling in a single forward pass. • Empirical validation across complementary clinical synthesis settings. We evaluate the same lifting principle across unconditional MR imaging generation, missing-modality MR synthesis, and MR-to-CT synthesis. These experiments assess whether lightweight through-plane organization can improve volume coherence while preserving the resolution and efficiency advantages of 2D synthesis, and compare it with volumetric and slice-wise baselines in terms of image quality, memory usage, inference cost, and through-plane coherence. 2 Related Work Volumetric and slice-wise medical synthesis. Volumetric methods enforce 3D structure by operating on 3D tensors or compressed 3D representations, including hierarchical sub-volume generative adversarial networks (GANs) [ 7 ] , 3D and conditional denoising diffusion probabilistic models (DDPMs) [ 8 , 9 ] , latent generative models [ 10 , 11 ] , wavelet-domain models [ 12 , 13 ] , and patch-volume latent diffusion models (LDMs) [ 14 ] . These approaches directly model inter-slice dependence, but their volumetric design often requires compromises in memory, spatial resolution, training strategy, or architectural complexity [ 23 ] . Slice-wise alternatives instead synthesize volumes by stacking independent 2D outputs from pix2pix-style or CycleGAN-style backbones [ 24 , 25 , 26 , 15 ] , with recent extensions adding adjacent-slice alignment, multi-view averaging, or slice-aggregation modules to recover coherence [ 16 , 17 , 18 ] . These methods preserve the efficiency and native in-plane resolution of 2D synthesis, but the resulting volumes can still suffer from discontinuities or inconsistent anatomy across depth [ 15 , 16 ] . Synthetic CT is a representative application of this tradeoff: SynthRAD2023 benchmarks MR-to-CT and cone-beam-CT-to-CT synthesis under image- and dose-based metrics [ 27 ] , and patch-based conditional GANs, CycleGANs, U-Nets, and drifting-model variants have all been applied to synthetic-CT generation [ 28 , 29 , 30 , 31 ] .

Read the complete text