Neural Acquisition & Representation of Subsurface Scattering

Arjun Majumdar 1 Raphael Braun 1 Hendrik Lensch 1 1 University of Tübingen, Germany · 2026

Read Neural Acquisition & Representation of Subsurface Scattering by Arjun Majumdar 1 Raphael Braun 1 Hendrik Lensch 1 1 University of Tübingen, Germany (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02292.txt.

Introduction

Majumdar and co-authors learn how light scatters below the surface of objects. Subsurface scattering shapes how many materials look. Their method acquires and estimates those properties in fine detail. It learns the pixel footprint response at each surface point. A 3D scanning input feeds a U-Net network for the task.

A stereo projector-camera setup captures the needed data. Phase-shifted profilometry patterns make that capture efficient. Varied scattering objects are measured with the setup. Dense pixel footprints allow relighting with arbitrary patterns. High resolution projector patterns can be used for relighting. The final output is a relit color image. Comparisons with real captured images show close matches. One model trains across multiple views and objects. Graphics users gain a way to capture material look.

Catalog record

CREATORArjun Majumdar 1 Raphael Braun 1 Hendrik Lensch 1 1 University of Tübingen, Germany
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.02292

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neural Acquisition & Representation of Subsurface Scattering Arjun Majumdar 1 Raphael Braun 1 Hendrik Lensch 1 1 University of Tübingen, Germany Abstract We present a method to acquire and estimate the sub-surface scattering properties of light transport at a highly detailed level by learning the pixel footprint response at each point on the object surface. The reconstruction leverages 3D scanning techniques as input to a U-Net CNN. A stereo projector-camera setup using phase-shifted profilometry (PSP) patterns efficiently captures the data for a variety of scattering objects. Reconstructing dense pixel footprints allows for relighting with arbitrary high-resolution projector patterns. The final output is a relit color image. Qualitative and quantitative comparison against illuminated real-world captured images demonstrate that the predicted footprints are almost identical to the actual responses. The same model is trained for multiple views across multiple objects such that the learned representations can be used to generalize to unseen sub-surface scattering materials as well. \WsPaper † † editors: B. Egger and T. Günther † † year: 2025 † † subject: \p@ConfName ( \p@ConfYear ) 1 Introduction In light transport subsurface scattering refers to the phenomena of light entering a surface, being scattered inside the material and leaving it at another location than where it entered. All non metal materials allow light to enter to various depths, which depends on the individual material. In case of glass like materials light passes through without being scattered. On the other side there are materials like wood, paper or concrete, which are considered opaque, as the light only penetrates to microscopic depths before being scattered back. Those extreme cases can be easily described with analytic BSSRDFs, for glass a clear dielectric model and for opaque materials a diffuse model. However in between there are materials such as wax, marble, organic objects, fruits, leaves, flowers, skin and others, which are not as straight forward. There are analytic models, that describe light transport inside the material as diffusion process [ JMLH01a ] , which in cases of geometry in form of semi-infinite slabs have exact solutions. Recently deep learning based approaches appeared which push the idea of closed form solutions to arbitrary curved surfaces [ VKJ19 ] while maintaining the efficiency of surface based models. Alternatively it is always possible to simulate the volumetric light transport inside a material via path tracing [ FHL∗18 , CFS∗18 ] , which however requires a detailed volumetric representation of the optical properties inside an object to get correct results for complex materials such as alabaster or any object with complex volumetric composition. Creating or measuring such a volumetric representation is challenging, as only the scattering result on the surface are directly observable, not the underlying density fluctuations and phase function changes. Instead of describing the complex light transport using surface and volume representation we use an image based rendering approach, where the impact of every light ray on the scene is captured as image. Here relighting an object just means adding together the correct images with the correct scaling factors. In practice it is however very costly to capture such an image dataset. The non-local nature of Subsurface Scattering requires the acquisition of thousands of images, even when the effect of multiple rays are captured in a single picture. In this paper, we propose a novel, completely data-driven method for image-based modeling of subsurface scattering objects over multiple views and across multiple objects, which after training can predict the required images for relighting without the need for excessive acquisitions. Instead, high-frequency phase-shifted sinusoidal patterns are used as input to a U-Net-based Convolutional Neural Network (CNN) [ RFB15 ] . These patterns are the same patterns that are used to capture precise 3D geometry of the object during 3D scanning. Our U-Net predicts a per pixel anisotropic footprint in image space expressing the subsurface scattering response to incident illumination at that point on the surface. It learns to generalize to unseen views and objects. Furthermore, our method can be trained from images of a single object and, unlike other methods, does not require to learn from many other objects, nor is it probabilistically generative in nature. As a result, it will not suffer from prior distribution bias. Also, since it consists of a single U-Net network, training it is relatively cheap, efficient and easy. We perform comprehensive qualitative and quantitative evaluations of our learned footprint responses to render the object under various projection patterns and compare the resulting images with the camera captured ground truth to demonstrate the performance of our model on objects with challenging material properties and geometry. Since the focus of our method is on learning and estimating the footprint response of each surface pixel, we perform extensive evaluations on relit results. In summary, we claim the following contributions: • We learn anisotropic pixel footprint responses to capture subsurface scattering properties by only using projected high-frequency phase-shift patterns which can at the same time be used for 3D scanning. • A neural network estimates footprint responses for each point on the object which then are used to perform relighting with spatially varying light patterns in RGB colorspace of the image. • Our neural network architecture is able to generalize across multiple views and multiple objects, which we verify both qualitatively and quantitatively. We compute image based sub-surface scattering (SSS) estimation. The actual material parameters for modeling the BSSRDF is not inferred. We only learn anisotropic SSS footprint responses which encodes both the geometry and the material properties and can be used for image based relighting. 2 Related Works The acquisition of material properties has a long-standing tradition, including the measurement of subsurface scattering parameters or general relightable representations, e.g. reflectance fields [ DHT∗00 ] . While initial works sampled directly the impulse response to incident illumination, fixed wavelet noise patterns [ PD03 ] , compressive sensing [ PML∗09 ] or adaptive acquisition schemes [ SCG∗05 , GTLL06 , OK10 , ORK12 ] exploiting the directionally duality of the light transport have been employed for faster acquisition of general light transport matrices. The required number of acquisition patterns still remains substantial. After some initial training over multiple objects, any new object and new view in our approach only requires about six input images to recover the spatially varying pixel footprint for subsurface scattering. After the introduction of the first practical BSSRDF model [ JMLH01b ] the specific estimation of subsurface scattering parameters has been performed using dedicated point-wise illumination [ JMLH01b , WMP∗06 ] , For entire objects, Goesele et al. [ GLL∗04 ] have presented a system, where a single laser beam scans every surface point to measure the resulting camera footprint, requiring thousands of individual measurements. A few neural network-based relighting approaches specifically focus on relighting of subsurface scattering objects [ ZSS21 , LTL∗22 , YGF∗23 , ZSB∗23 , DME∗25 ] from a smaller set of images. However, those approaches assume distant illumination and often use one-light-source-at-a-time (OLAT) as training data. Structured light scanning patterns, specifically high-frequency patterns, have been used to separate the observed reflected radiance wrt. being caused by local or global illumination [ NKGR06a ] . There are existing methods that leverage high-frequency PSP for sub-surface scattering materials in 3D scanning: [

Read the complete text