DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging
Noe Bertramo, Gabriel Duguey, Vivek Gopalakrishnan · 2025
Read DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging by Noe Bertramo, Gabriel Duguey, Vivek Gopalakrishnan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06768.txt.
Introduction
Noe Bertramo, Gabriel Duguey, and Vivek Gopalakrishnan build a fake-ultrasound maker in a 2025 computing paper. Ultrasound guides many surgeries live, but its grainy images are hard to match with crisp scans taken before the operation. Their tool renders ultrasound-like views from MRI or CT volumes. Because the tool is built from smooth math, other software can tune it by gradient.
First, a learning step turns a scan into a map of how tissue bounces sound. Ray tracing then sends virtual sound beams through the map, with echoes bouncing more than once. The final image adds speckle and depth fade for realism. Such matched views could help line up live guidance with the surgical plan.
Catalog record
| CREATOR | Noe Bertramo, Gabriel Duguey, Vivek Gopalakrishnan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06768
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging Noe Bertramo, Gabriel Duguey, Vivek Gopalakrishnan Abstract Intraoperative ultrasound imaging provides real-time guidance during numerous surgical procedures, but its interpretation is complicated by noise, artifacts, and poor alignment with high-resolution preoperative MRI/CT scans. To bridge the gap between preoperative planning and intraoperative guidance, we present DiffUS, a physics-based, differentiable ultrasound renderer that synthesizes realistic B-mode images from volumetric imaging. DiffUS first converts MRI 3D scans into acoustic impedance volumes using a machine learning approach. Next, we simulate ultrasound beam propagation using ray tracing with coupled reflection-transmission equations. DiffUS formulates wave propagation as a sparse linear system that captures multiple internal reflections. Finally, we reconstruct B-mode images via depth-resolved echo extraction across fan-shaped acquisition geometry, incorporating realistic artifacts including speckle noise and depth-dependent degradation. DiffUS is entirely implemented as differentiable tensor operations in PyTorch, enabling gradient-based optimization for downstream applications such as slice-to-volume registration and volumetric reconstruction. Evaluation on the ReMIND dataset demonstrates DiffUS’s ability to generate anatomically accurate ultrasound images from brain MRI data. Our implementation is available at https://github.com/gduguey/DiffUS . 1 Introduction Ultrasound (US) imaging is valuable for intraoperative guidance given its speed, safety, and portability. However, US images are plagued by noise, artifacts, and poor contrast, complicating anatomical interpretation in delicate tasks such as tumor resection [ 17 ] . Furthermore, the difficulty in aligning intraoperative US and high-resolution preoperative MRI or CT scans forces surgeons to perform a complex mental mapping across modalities with disparate visual characteristics. This disconnect can make it difficult to accurately locate critical anatomical structures, identify residual pathology, and adjust surgical plans in real time. Generating realistic ultrasound images from preoperative volumetric scans could help overcome these challenges by improving spatial alignment between modalities and supporting more effective surgical guidance. To this end, we propose DiffUS, a physics-based, differentiable ultrasound renderer that models acoustic wave propagation through coupled reflection-transmission equations. Unlike previous ultrasound simulation methods that rely on random sampling [ 7 ] , our method employs deterministic ray propagation with sequential processing to capture temporal echo characteristics. Furthermore, our approach formulates wave propagation as a stable linear system solution, enabling efficient computation of multiple internal reflections that is faster than existing approaches [ 19 ] . The differentiable nature of our renderer facilitates gradient-based optimization for various downstream applications, including registration and reconstruction tasks. Our work contributes to the growing field of physics-based medical imaging simulation [ 2 , 8 , 12 ] by providing an interpretable, computationally efficient framework for ultrasound synthesis. DiffUS rapidly generates realistic ultrasound images towards the ultimate goal of supporting improved surgical outcomes through enhanced multimodal imaging integration. 2 Method DiffUS is a differentiable, physics-based US renderer that synthesizes realistic B-mode images by modeling acoustic wave propagation through volumetric MRI or CT scans. In the following sections, we detail the fundamental physics principles underlying our simulator (§ 2.1 ), our method for mapping CT/MRI intensities to acoustic impedances (§ 2.2 ), our formulation of wave propagation as a sparse linear system (§ 2.3 ), and the final image formation model (§ 2.4 ). 2.1 Physical Principles of Ultrasound Imaging US imaging captures the propagation of acoustic waves through biological tissues at frequencies ranging from 2-15 MHz. The physics of US is governed by the wave equation, with propagation characteristics determined by acoustic impedance Z=\rho c , where \rho denotes tissue density and c the speed of sound. At tissue interfaces, wave behavior follows the reflection and transmission coefficients: R_{1\rightarrow 2}=\left|\frac{Z_{2}-Z_{1}}{Z_{1}+Z_{2}}\right|,\quad T_{1\rightarrow 2}=\frac{2Z_{2}}{Z_{2}+Z_{1}} (1) where R and T represent reflection and transmission coefficients between media with impedances Z_{1} and Z_{2} . Clinical ultrasound systems operate under several simplifying assumptions [ 1 , 13 ] that we incorporate in DiffUS: (1) acoustic waves propagate along straight-line paths, (2) the speed of sound is constant across soft tissues ( c=1540 m/s), (3) acquisitions represent a single pulse-echo, (4) attenuation is uniform across media, and (5) the contribution of side-lobes is negligible. Under these conditions, the reflecting structure depth is determined by the time-of-flight measurement d=\frac{c}{2}(t_{2}-t_{1}) , where t_{1} and t_{2} represent pulse emission and echo reception times, respectively. 2.2 Generating Acoustic Impedance Volumes CT and MRI volumes do not directly encode impedance. Instead, the impedance of every voxel must be inferred through indirect relationships to underlying physical or biological properties. Therefore, to model acoustic wave propagation, we convert volumetric medical images into three-dimensional acoustic impedance maps Z(x,y,z) , which determine the intensity of both reflected and transmitted signals at tissue interfaces. CT-based impedance mapping. Voxels in CT are encoded using Hounsfield units (HU), calibrated quantities that reflect the local X-ray attenuation coefficient relative to water [ 16 ] . Because X-ray attenuation in biological tissues is dominated by Compton scattering at clinical energies (70–140 kVp), and Compton scattering scales approximately with electron density, HU values provide a strong proxy for physical density [ 18 ] . Therefore, we use an existing calibration method that fits a piecewise linear relationship between HU and density across typical biological tissues [ 18 , 21 ] . From this density map \rho(x,y,z) , we compute acoustic impedance as Z(x,y,z)=\rho(x,y,z)\cdot c(x,y,z) , where c is estimated directly from HU using previously reported regression fits [ 20 ] . This gives us a voxel-wise impedance map grounded in physical tissue characteristics. Figure 1: MLP-based acoustic impedance estimation from MRI T1 intensities. (a) The learned continuous mapping (blue curve) transforms T1 intensities to acoustic impedance values (MRayl). (b) Tissue-specific reference points (markers) used for supervised training. MRI-based impedance mapping. Unlike CT, MRI intensities do not correspond directly to physical quantities such as density or acoustic impedance. Instead, they reflect a complex combination of tissue-specific properties, including water content, fat concentration, and relaxation times, modulated by the chosen acquisition sequence. This makes impedance reconstruction from MRI fundamentally ill-posed [ 3 ] . To approximate impedance, we therefore adopt a supervised learning approach, fitting a multi-layer perceptron (MLP) to paired data consisting of MRI intensities and reference impedance values from literature [ 4 , 5 , 6 , 14 ] . While this provides a continuous and flexible estimate of Z(x,y,z) across the volume, it is a heuristic approximation that lacks the physical interpretability of CT-derived methods. The training data and learned mapping are illustrated in Figure 1 . 2.3 Wave Propagation Model We model the propagation of the ultrasound beam using a ray tracing approach where each acoustic beam traverses a linear sequence of voxels. At each voxel boundary, wave amplitudes undergo reflection and transmission according to local impedance contrasts. We define forward-traveling amplitude g_{i} and backward-traveling amplitude d_{i} at each interface i,i+1 , which evolve according to coupled recurrence relations: g_{i+1}=T_{i\rightarrow i+1}\cdot g_{i}+R_{i+1\rightarrow i}\cdot d_{i+1}\ \quad\quad d_{i}=R_{i\rightarrow i+1}\cdot g_{i}+T_{i+1\rightarrow i}\cdot d_{i+1} (2) These equations capture multiple internal reflections and transmissions throughout the layered medium, providing physically accurate wave amplitude evolution. The coupling logic between incident and reflected waves across a single boundary is illustrated in Figure 2 . We solve these coupled wave equations by formulating a sparse linear system \mathbf{A}\mathbf{x}=\mathbf{b} for N voxels. We arrange amplitude unknowns as \mathbf{x}=[g_{0},d_{0},g_{1},d_{1},\ldots,g_{N},d_{N}]^{T} (3) where boundary conditions impose unit pulse injection ( g_{0}=1 ) and far-field termination ( d_{N}=0 ). These boundary conditions