3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance

Sapna Sachan, Amulya Kumar Mahto · 2026

Read 3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance by Sapna Sachan, Amulya Kumar Mahto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04856.txt.

Introduction

Sapna Sachan and Amulya Kumar Mahto build CT-like images from three-dimensional ultrasound scans of the kidney, aiming to guide operators in real time. CT gives clear reference views but uses ionising radiation. Ultrasound is safer and common, yet depends heavily on operator skill and lacks firm tissue measures.

Their system first aligns paired ultrasound and CT scans from the TRUSTED dataset using landmarks. A network mixing residual blocks with a transformer bottleneck then creates pseudo-CT volumes, checked by a discriminator for local realism. Scores for structural fidelity beat established baselines. The authors note the paired dataset is small, which may limit how widely the method generalises.

Catalog record

CREATORSapna Sachan, Amulya Kumar Mahto
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.04856

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance Sapna Sachan, Amulya Kumar Mahto Abstract Computed tomography (CT) is indispensable for clinical diagnosis and image-guided interventions but exposes patients to ionizing radiation, motivating the development of safer imaging alternatives. Ultrasound (US) is non-ionizing and widely accessible; however, it is highly operator dependent and lacks quantitative tissue characterization, often leading to diagnostic uncertainty and unnecessary CT examinations. This work presents a 3D ultrasound-derived pseudo-CT (UD-pCT) framework that generates CT-like anatomical reference volumes inferred from US, without aiming to reproduce physically accurate Hounsfield Units. Paired 3D kidney US and CT volumes from the TRUSTED dataset are first spatially aligned using a landmark-based multimodal registration pipeline, creating high-quality paired inputs for supervised training of an adversarial framework. The proposed Bottleneck Transformer Residual U-Net3D (BT-ResUNet3D) model employs a 3D residual encoder–decoder generator augmented with a transformer bottleneck, enabling effective modeling of fine-grained local anatomical structures as well as long-range volumetric dependencies, while a 3D Conditional PatchGAN discriminator enforces local structural realism in the synthesized pseudo-CT volumes. Quantitative evaluation using PSNR and SSIM demonstrates that the proposed method outperforms established baselines in structural fidelity and perceptual image quality. The UD-pCT volumes provide real-time anatomical reference for operator guidance, potentially reducing acquisition variability and unnecessary CT use. A limitation of this study is the relatively small paired dataset, which may limit the generalizability of the proposed model. All data preprocessing and testing codes are given as supplimentary material to ensure the reproducibility of the experimental results. I Introduction Imaging with computed tomography (CT) is a cornerstone of modern clinical practice, supporting diagnosis, treatment planning, and image-guided interventions. Despite its clinical utility, CT imaging relies on ionizing radiation, which is associated with a small but measurable risk of radiation-induced cancer for each scan. With an estimated 375 million CT examinations performed globally each year [ 5 , 6 ] , this risk becomes significant at the population level. At the same time, the global burden of cancer continues to rise, with approximately 20 million new cases reported in 2022 and projections reaching nearly 35 million by 2050 [ 1 , 2 ] . Earlier studies estimated that almost 29,000 future cancer cases in the United States could be linked to CT scans performed in 2007 alone [ 5 ] . More recent analyses suggest that, if current imaging practices continue, CT-associated cancers could account for up to \sim 5% of among all newly diagnosed cases [ 4 ] . These trends emphasize the urgency of minimizing CT examinations which can be avoided, particularly in clinical cases where some safe imaging modalities can give sufficient anatomical guidance. This motivates the development of some useful complementary approaches that reduce radiation exposure and also preserve the clinical utility of image-guided workflows. US imaging is a safe, non-ionizing, portable, and cost-effective alternative [ 7 ] . However, US has its own inherent limitations: its intensity values do not correspond directly to tissue electron density or Hounsfield Units (HU), which restricts quantitative interpretation, and quality of an image is highly dependent on the operator of the instrument. In particular, image quality gets significantly affected by the varying probe pressure applied by the operator, which again leads to incomplete anatomical visualization, potential misinterpretation, and, consequently, leading to unnecessary follow-up CT examinations. A encouraging strategy to address these limitations is by bridging the modality gap by generating 3D UD-pCT volumes using US data. It is important to note here that the goal of UD-pCT synthesis is not to replace diagnostic CT imaging, but rather to provide real-time anatomical reference information during US acquisition. By presenting a CT-like anatomical context, UD-pCT volumes can enhance effectiveness of operator by increasing the awareness of probe positioning, imaging orientation, and anatomical coverage. Such guidance can help steer acquisitions toward optimal imaging planes, improving consistency of imaging among operators, and diminishing unnecessary CT referrals, while fully preserving the safety and accessibility advantages of US. However, existing deep learning models such as 3D ResUNet-based models, while effective at local feature extraction, rely on convolutional receptive fields and struggle to capture global anatomical relationships across large 3D volumes. ResViT-style architectures, which replace or heavily integrate transformers throughout the encoder–decoder pathway, improve global context modeling but often incur high computational cost and may degrade fine-grained local anatomical detail—particularly problematic for noisy US data. Motivated by these developments, this work proposes BT-ResUNet3D model for UD-pCT volumes synthesis using paired kidney US and CT volumes from the TRUSTED dataset. Unlike ResViT models, our approach introduces a lightweight transformer exclusively at the bottleneck, where it models long-range volumetric dependencies without disrupting early-stage convolutional feature extraction. In contrast to standard 3D ResUNet, deep residual blocks are combined with global transformer context through residual fusion, enabling the network to preserve local anatomical fidelity while enhancing global structural coherence. The resulting UD-pCT volumes are designed to function as real-time anatomical references for operator guidance, rather than diagnostic CT substitutes. The main contributions of this work are summarized as follows: • We propose a robust and clinically motivated 3D multimodal data preprocessing pipeline , consisting of landmark-based US–CT registration followed by region-of-interest (ROI) cropping and field of view masking, to construct anatomically consistent and spatially aligned volumetric training pairs, which is critical for reliable supervised cross-modality synthesis. • We introduce a transformer-augmented bottleneck integrated within a 3D residual encoder–decoder(BT-ResUNet3D) architecture to explicitly capture long-range volumetric dependencies beyond convolutional receptive fields, while preserving local anatomical fidelity. • We incorporate a 3D conditional PatchGAN discriminator to enforce local structural realism and enhance high-frequency anatomical details in the synthesized UD-pCT volumes. • We stablish the clinical utility of UD-pCT volumes for operator guidance for US acquisition, with the ability to improve scan consistency and reduced unnecessary CT usage. • Quantitative and qualitative evaluation on the TRUSTED dataset shows that the proposed method outperforms baseline approaches on comparing in terms of PSNR and SSIM, leading to improved structural fidelity and perceptual quality. II Literature Survey II-A Operator Guidance in Ultrasound Imaging US imaging is highly operator dependent, and image quality strongly fluctuates with probe orientation, pressure, and anatomical window selection. Several works address these important challenges, focusing on variety of strategies to enhance quality of US imaging through artificial intelligence and multi-modal integration to provide real-time guidance to operators, which improves acquisition consistency and diagnostic reliability. Considering these issues, Ossaba et al. [ 14 ] validated a modular deep learning prototype that assists operator in acquiring diagnostic US images, for improved quality of captured planes. Similarly, Baloescu et al. [ 15 ] demonstrated that AI-assisted lung US helps non-experts to achieve performance comparable to trained clinicians. Recent reviews highlight that AI-guided acquisition, standard plane detection, and semantic segmentation are some of the critical components for reducing operator variability [ 16 ] . Hybrid approaches that integrate data-driven learning with physics-informed constraints further improve anatomical fidelity and acoustic consistency in US-based modeling [ 12 ] . Even though, deep regression model [ 8 ] shows promise in correcting for liver motion and improving anatomical alignment, fully mimicking expert-level adaptability across diverse clinical scenarios remains challenging. II-B US–CT Registration Methods Multimodal registration between US and CT is essential for cross‑modality synthesis

Read the complete text