UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound
Junxuan Yu, Yaofei Duan, Yuhao Huang, Yu Wang, Rongbo Ling, Weihao Luo, Ang Zhang, Jingxian Xu, Qiongying Ni, Yongsong Zhou, Binghan Li, Haoran Dou, Liping Liu, Yanfen Chu, Feng Geng, Zhe Sheng, Zhifeng Ding, Dingxin Zhang, Rui Huang, Yuhang Zhang, Xiaowei Xu, Tao Tan, Dong Ni, Zhongshan Gou ✉, Xin · 2025
Read UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound by Junxuan Yu, Yaofei Duan, Yuhao Huang, Yu Wang, Rongbo Ling, Weihao Luo, Ang Zhang, Jingxian Xu, Qiongying Ni, Yongsong Zhou, Binghan Li, Haoran Dou, Liping Liu, Yanfen Chu, Feng Geng, Zhe Sheng, Zhifeng Ding, Dingxin Zhang, Rui Huang, Yuhang Zhang, Xiaowei Xu, Tao Tan, Dong Ni, Zhongshan Gou ✉, Xin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23490.txt.
Introduction
Junxuan Yu and co-authors build digital twins of the heart from ordinary two-dimensional ultrasound views, in a 2025 paper. Ultrasound is the workhorse of heart exams. Flat images, though, hide the heart's true three-dimensional shape. True 3D ultrasound is scarce and grainy in practice. A twin rebuilt from several flat views could serve both worlds.
Reconstruction faces scarce paired training data and heavy image noise. Their method fuses multi-view scans into an anatomical model. Such twins support precise measurements and treatment planning. Surgeons could rehearse on the patient's own geometry. The paper reports progress toward routine twin generation. Cardiology gains a bridge between cheap imaging and rich models.
Catalog record
| CREATOR | Junxuan Yu, Yaofei Duan, Yuhao Huang, Yu Wang, Rongbo Ling, Weihao Luo, Ang Zhang, Jingxian Xu, Qiongying Ni, Yongsong Zhou, Binghan Li, Haoran Dou, Liping Liu, Yanfen Chu, Feng Geng, Zhe Sheng, Zhifeng Ding, Dingxin Zhang, Rui Huang, Yuhang Zhang, Xiaowei Xu, Tao Tan, Dong Ni, Zhongshan Gou ✉, Xin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23490
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound Junxuan Yu, Yaofei Duan, Yuhao Huang, Yu Wang, Rongbo Ling, Weihao Luo, Ang Zhang, Jingxian Xu, Qiongying Ni, Yongsong Zhou, Binghan Li, Haoran Dou, Liping Liu, Yanfen Chu, Feng Geng, Zhe Sheng, Zhifeng Ding, Dingxin Zhang, Rui Huang, Yuhang Zhang, Xiaowei Xu, Tao Tan, Dong Ni, Zhongshan Gou ✉, Xin Yang ✉ Abstract Echocardiography is routine for cardiac examination. However, 2D ultrasound (US) struggles with accurate metric calculation and direct observation of 3D cardiac structures. Moreover, 3D US is limited by low resolution, small field of view and scarce availability in practice. Constructing the cardiac anatomical twin from 2D images is promising to provide precise treatment planning and clinical quantification. However, it remains challenging due to the rare paired data, complex structures, and US noises. In this study, we introduce a novel generative framework UltraTwin , to obtain cardiac anatomical twin from sparse multi-view 2D US. Our contribution is three-fold. First, pioneered the construction of a real-world and high-quality dataset containing strictly paired multi-view 2D US and CT, and pseudo-paired data. Second, we propose a coarse-to-fine scheme to achieve hierarchical reconstruction optimization. Last, we introduce an implicit autoencoder for topology-aware constraints. Extensive experiments show that UltraTwin reconstructs high-quality anatomical twins versus strong competitors. We believe it advances anatomical twin modeling for potential applications in personalized cardiac care. 1 Introduction Accurate assessment of cardiac structure and function is vital for diagnosis, treatment planning, and surgical guidance. However, 2D ultrasound (US) lacks precise parameter evaluation and 3D anatomical context, bringing significant uncertainties in clinical decision-making, particularly for complex cardiac disease cases. Additionally, 3D US is hindered by motion-induced artifacts, restricted field of view, and limited clinical availability [ 8 ] . Hence, reconstructing cardiac anatomical twins directly from sparse 2D US is highly desired (see Fig. 1 ). These cardiac anatomical twins have the great potential to facilitate the personalized evaluation of cardiac function and improve treatment planning [ 6 , 11 , 1 ] . Figure 1: UltraTwin Workflow: from multi-view 2D US to 3D cardiac anatomical twin. Recently, several deep learning-based algorithms for cardiac reconstruction have been investigated. Laumer et al. [ 4 ] utilized single-view US for synthetic cardiac reconstruction. However, their approach could not differentiate between variations inherent to US imaging artifacts and actual cardiac pathologies due to the limited one-view information and lack of 3D ground truth (GT) validation. Stojanovsk et al. [ 10 ] then extended the approach to multi-view reconstruction by incorporating cardiac wall masks as guidance. While their method alleviated the single-view problem, it seriously depends on accurate mask segmentation, leading to potential error propagation. To capture the complexity and diversity of real clinical cases, research [ 5 ] attempted left ventricular reconstruction using multi-view US images. However, it overlooked crucial inter-chamber relationships and lacked validation with 3D GT. Reconstructing 3D cardiac structures from sparse 2D US views remains challenging, as multiple 3D cardiac shapes can correspond to the same set of 2D views, complicating personalized reconstruction. While Diffusion Models [ 3 ] offer promising solutions through distribution modeling, their data-intensive nature poses additional challenges especially when paired 2D-3D data is scarce. To address the above challenges, we propose UltraTwin, the first multi-view 3D reconstruction framework based on Diffusion Transformer (DiT) [ 9 ] in real-world cardiac imaging scenarios. We prospectively collected a real-world dataset of paired multi-view US images and corresponding CT scans for 3D modeling. UltraTwin encompasses three key designs: (1) Cardiac modeling pretraining with a novel pseudo-paired data generation strategy for robust data augmentation, enabling effective learning from limited clinical data. (2) A coarse-to-fine reconstruction pipeline that progressively refines cardiac structures through template-to-anatomy hierarchical feature learning, ensuring both global anatomical consistency and local anisotropic information extraction. (3) Implicit topology perception constraints that leverage cardiac structure priors to reduce noise in 2D US projections and enhance the anatomical plausibility of 3D reconstructions. Comprehensive validation on the large dataset demonstrates UltraTwin’s superior performance across multiple quantitative metrics, showing promising potential for clinical cardiac chamber volume measurements. 2 Methodology As shown in Fig. 2 , we propose UltraTwin to reconstruct personalized, high-fidelity 3D cardiac anatomical twins from sparse multi-view 2D US images under topological constraints. Our method combines pseudo-paired data pre-training, strictly paired data fine-tuning, a coarse-to-fine denoising scheme, and an implicit autoencoder for topology-aware reconstruction. Our baseline uses only coarse denoising scheme with condition injection. Figure 2: The pipeline of our proposed UltraTwin. In the coarse stage, a template selector (TS) outputs a template to guide denoising. The coarse output is patchified and refined anisotropically, with double dynamic fusion of branch results. 2.1 High-quality Dataset Construction To address the challenge of scarce paired data, we construct a high-quality dataset comprising both paired and unpaired multi-view 2D US images and 3D cardiac models. The unpaired data is processed using a novel Pseudo-paired Real Data Generation Strategy to enable effective pre-training. Paired Data Collection. The paired data includes multi-view 2D US images and ECG-gated CT scans. We use TotalSegmentator [ 12 ] to segment cardiac structures from CT scans, extracting 3D models at end-diastole (ED) and end-systole (ES) phases. For multi-view 2D US videos, ED and ES frames are manually annotated and temporally aligned, resulting in a strictly paired dataset. Pseudo-paired Real Data Generation Strategy. To address the scarcity of paired data, we leverage temporal-unpaired multi-view 2D US videos and chest CT scans to create additional training data for model pre-training. For each patient, the standard Apical 4-Chamber (A4C) view of 3D cardiac model is computed following [ 10 ] . Seven key parameters are calculated on this view. For US videos, pretrained structure and keypoint detectors extract cardiac motion curves and frame-level parameters. The A4C US frame with minimal parameter error compared to the 3D cardiac model is identified, and multi-view images in videos are temporally aligned based on their frame position in the motion curve. 2.2 Coarse-to-fine Conditional Diffusion Transformer To generate high-fidelity 3D cardiac models, we employ a coarse-to-fine conditional Diffusion Transformer (DiT), leveraging multiview feature fusion, cardiac template priors, and anisotropic refinement. Preliminaries. Following the original DiT-3D [ 7 ] , the input voxel tensor V_{i}\in\mathbb{R}^{N\times V\times V\times V} is divided into patches of size (p,p,p) , producing patch tokens t\in\mathbb{R}^{L\times N} , where L=(V/p)^{3} is total number of patches. In our study, N=6 denotes the number of classes (5 cardiac structures and one background). We refer readers to DiT-3D [ 7 ] for more details. To effectively utilize multi-view 2D US images, we introduce a Multi-view Feature Fusion module and a Conditional DiT (CDiT)-Block. A pre-trained ResNet50 [ 2 ] extracts view-level features, which are flattened into 2D vectors R\in\mathbb{R}^{d_{I}\times(h_{f}\cdot w_{f})} . We further inject view-level spacing information into the features using an MLP. As shown in Fig. 2 , these features are concatenated and fused via 1D convolutional blocks, then projected into a condition vector C\in\mathbb{R}^{L\times D} . And the CDiT-Block extends the DiT-Block with a cross-attention mechanism for condition injection. The input patch embeddings e_{v}\in\mathbb{R}^{L\times D} are used as the query ( q ), while the condition vector C serves as the key ( k ) and value ( v ). The cross-attention operation (CA) is defined as