OSCAR: Occupancy-based Shape Completion via Acoustic Neural Implicit Representations

Magdalena Wysocki, Kadir Burak Buldu *, Miruna-Alexandra Gafencu, Mohammad Farid Azampour, Nassir Navab · 2026

Read OSCAR: Occupancy-based Shape Completion via Acoustic Neural Implicit Representations by Magdalena Wysocki, Kadir Burak Buldu *, Miruna-Alexandra Gafencu, Mohammad Farid Azampour, Nassir Navab (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08279.txt.

Introduction

Wysocki and colleagues propose OSCAR, an occupancy-based shape completion method that rebuilds complete 3D spine anatomy from partial sound views. Accurate 3D spine maps from ultrasound matter for guiding keyhole spine surgery, but sound shadows and view-dependent signal shifts make it hard. For surgery use, the method extracts the surface directly from the image, avoiding the need for labels during inference. It relies on a coupled latent space holding both image appearance and underlying shape.

Catalog record

CREATORMagdalena Wysocki, Kadir Burak Buldu *, Miruna-Alexandra Gafencu, Mohammad Farid Azampour, Nassir Navab
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20517
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08279

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OSCAR: Occupancy-based Shape Completion via Acoustic Neural Implicit Representations Magdalena Wysocki, Kadir Burak Buldu *, Miruna-Alexandra Gafencu, Mohammad Farid Azampour, Nassir Navab Abstract Accurate 3D reconstruction of vertebral anatomy from ultrasound is important for guiding minimally invasive spine interventions, but it remains challenging due to acoustic shadowing and view-dependent signal variations. We propose an occupancy-based shape completion method that reconstructs complete 3D anatomical geometry from partial ultrasound observations. Crucially for intra-operative applications, our approach extracts the anatomical surface directly from the image, avoiding the need for anatomical labels during inference. This label-free completion relies on a coupled latent space representing both the image appearance and the underlying anatomical shape. By leveraging a Neural Implicit Representation (NIR) that jointly models both spatial occupancy and acoustic interactions, the method uses acoustic parameters to become implicitly aware of the unseen regions without explicit shadowing labels through tracking acoustic signal transmission. We show that this method outperforms state-of-the-art shape completion for B-mode ultrasound by 80% in HD95 score. We validate our approach both in silico and on phantom US images with registered mesh models from CT labels, demonstrating accurate reconstruction of occluded anatomy and robust generalization across diverse imaging conditions. Code and data will be released on publication. 1 Introduction Ultrasound (US) image-guided spinal surgery offers significant benefits for both patients and clinicians, avoiding excessive ionizing radiation while retaining the benefits of minimally invasive approaches. Unfortunately, severe acoustic shadowing and reverberation significantly degrades image quality, so that only the near surface of hard tissue can be directly observed. To provide intra-operative guidance, then, expert sonographers rely on strong prior knowledge of spine anatomy to acquire multiple viewpoints and mentally infer the complete 3D geometry. Anatomical shape completion mimics this cognitive process, moving beyond visible surface reconstruction [ 2 , 16 ] to create the anatomical digital twin, e.g . , for robotic- and computer-assisted approaches transitioning from distinct (visible) to completed (occluded) [ 3 ] anatomical reconstruction. In the past, shape completion methods have used statistical shape models to approximate this expert capability [ 5 , 4 , 7 ] , but the reliance on explicitly annotated, discrete point clouds inherently ignores the acoustic image formation process, treating shadows as missing data and ignoring their view-dependent properties. Our goal is to provide robust, physics-informed shape completion by leveraging both anatomical and acoustic priors to reconstruct the complete vertebral geometry from fragmented B-mode images, thereby improving 3D spatial guidance for minimally invasive spine interventions and robotic surgery. Recently, Neural Implicit Representations (NIRs) have emerged as a powerful alternative to discrete surfaces for complex shape modeling. Foundational works in computer vision, such as DeepSDF [ 9 ] and Occupancy Networks [ 8 ] , demonstrated that NIRs can effectively encode complex 3D shapes. Within medical imaging, approaches like ImplicitAtlas [ 18 ] , alongside other recent implicit shape models [ 14 , 17 ] have further shown that these networks can learn rich, continuous shape priors, while shared latent spaces encode both anatomical shape and image appearance [ 12 , 13 ] . By training on both images and anatomical segmentations, these methods enable self-supervised test-time optimization (TTO), where the shared latent space is optimized using only a medical image to accurately infer the segmentation. Thus far, however, NIRs with coupled latent spaces have been ill-suited to the task of shape completion in ultrasound due to the severe occlusions of acoustic shadowing and view-dependent characteristics of B-mode images. Without a modality-specific model, incomplete and contradicting information from multiple views corrupts the reconstruction process. Here, we present Occupancy-based Shape Completion via Acoustic NIRs (OSCAR), a physics-aware framework for US-based spinal shape completion that integrates view-dependent acoustic properties of tissue into a coupled-latent NIR. By modeling acoustic image formation and signal propagation as a ray-tracing process in this NIR [ 1 , 15 , 10 ] , we enable the latent space to resolve multi-view pixel contradictions and reconstruct unobserved regions with information from disparate viewpoints. Further, this coupling facilitates bidirectional optimization. Given an inferred anatomical shape, our model can reconstruct a plausible acoustic space, allowing for novel view synthesis of B-mode images with known shape. We evaluate our approach both in silico and on real images of an anthropomorphic phantom, with ground truth from registered computed tomography (CT) imaging. Compared to the state-of-the-art in ultrasound shape completion SITD [ 5 ] , OSCAR improves the HD95 score of reconstructed shapes by 80% while maintaining anatomically plausible shapes across its latent space, in addition to enabling realistic image synthesis from known shape. 2 Method Figure 1: Proposed framework pipeline. Joint Training: The network couples an acoustic head ( g_{\phi} ) and a geometric head ( s_{\psi} ) via a shared latent space ( \mathcal{Z} ) and backbone ( f_{\theta} ). Supervised by both B-mode synthesis ( \mathcal{L}_{photo} ) and ground-truth shape occupancy ( \mathcal{L}_{occ} ), the ray-based rendering inherently models acoustic shadowing. Test-Time Optimization: At inference, network weights are frozen. A novel latent code \mathbf{z}^{*} is optimized directly from fragmented B-mode observations without shape labels ( \mathcal{L}_{tto} ). The optimized prior inherently extracts the complete 3D geometry o(\mathbf{x}\mid\mathbf{z}^{*}) . 2.1 Problem Formulation and Method Overview Given a sequence of partial B-mode ultrasound observations \mathcal{D}=\{(I_{k},T_{k})\}_{k=1}^{K} , with images I_{k} and corresponding probe poses T_{k}\in SE(3) , our goal is to reconstruct the complete 3D vertebral geometry, defined as a continuous occupancy field O:\mathbb{R}^{3}\to[0,1] . We propose a NIR-based framework (Fig. 1 ) that jointly models this geometry and acoustic properties. For a 3D coordinate \mathbf{x} and a subject-specific latent code \mathbf{z} , a shared backbone network f_{\theta} extracts a joint feature representation. This feature space is then processed by two distinct output heads. The acoustic head g_{\phi} maps the joint features to a localized acoustic property vector \mathbf{a}(\mathbf{x})=g_{\phi}(f_{\theta}(\mathbf{x},\mathbf{z})) , where \mathbf{a}(\mathbf{x})=[\beta(\mathbf{x}),\sigma(\mathbf{x}),\mu(\mathbf{x})]^{T} represents acoustic reflection, scattering, and attenuation. Simultaneously, the occupancy head s_{\psi} maps the same features to the continuous volume occupancy probability o(\mathbf{x})=s_{\psi}(f_{\theta}(\mathbf{x},\mathbf{z})) . To bridge these 3D predictions with the 2D observations, a differentiable ray-based forward model synthesizes the expected B-mode intensity \hat{I}_{k}(\mathbf{r})=\mathcal{F}(\mathbf{r},\mathbf{a}) . By explicitly calculating signal transmission \mathcal{T}(\mathbf{x}) along ray \mathbf{r} , the model gains an implicit awareness of occlusions, inherently capturing acoustic shadowing without explicit labels and learned latent prior completes the unobserved 3D shape o(\mathbf{x}\mid\mathbf{z}^{*}) . 2.2 Joint Acoustic and Geometric Representation To capture the codependence between anatomical structure and its acoustic signature, we define a unified latent space \mathcal{Z}\subset\mathbb{R}^{d} . For any 3D coordinate \mathbf{x}\in\mathbb{R}^{3} and subject latent code \mathbf{z}\in\mathcal{Z} , a shared backbone f_{\theta} learns a joint structural prior: \mathbf{h}(\mathbf{x})=f_{\theta}(\mathbf{x},\mathbf{z}) . This shared feature vector \mathbf{h}(\mathbf{x}) branches into two specialized heads. The acoustic head g_{\phi} outputs physical tissue properties for wave propagation, \mathbf{a}(\mathbf{x})=g_{\phi}(\mathbf{h}(\mathbf{x}))=[\beta,\sigma,\mu]^{T}\geq 0 . Concurrently, the occupancy head s_{\psi} yields the continuous surface occupancy, o(\mathbf{x})=s_{\psi}(\mathbf{h}(\mathbf{x}))\in[0,1] . By forcing both properties to decode from the shared representation \mathbf{h}(\mathbf{x}) conditioned on the same latent space \mathcal{Z} , the network intrinsically couples the spaces, projecting the acoustic optimization onto a manifold strictly constrained by the anatomical geometry. 2.3 Physics-Aware Ray-Based Rendering To optimize the latent code \mathbf{z} directly from B-mode images, we map the 3D acoustic field \mathbf{a}(\mathbf{x}) to the 2D image plane using the differentiable physical rendering formulation established in ultrasound NeRFs [

Read the complete text