Snapshot Polarimetric Display Inverse Rendering
Seokjun Choi, Yunseong Moon, Kaizhang Kang, Hoon-Gyu Chung, Jin-Nyeong Kim, Giljoo Nam, Seung-Hwan Baek · 2026
Read Snapshot Polarimetric Display Inverse Rendering by Seokjun Choi, Yunseong Moon, Kaizhang Kang, Hoon-Gyu Chung, Jin-Nyeong Kim, Giljoo Nam, Seung-Hwan Baek (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24915.txt.
Introduction
Seokjun Choi leads a one-shot method recovering surface properties from one desktop capture. Each frame carries little data, so the task is tight. The setup shines a polarized color pattern from an LCD screen and records it with a polarization camera and quarter-wave plate. A feed-forward transformer maps readings to per-pixel direction, color, roughness, and metalness. To beat scarce training data, the team grows measured reflectance functions through a generative manifold. Tests on a real desktop rig beat prior methods across varied scenes.
Catalog record
| CREATOR | Seokjun Choi, Yunseong Moon, Kaizhang Kang, Hoon-Gyu Chung, Jin-Nyeong Kim, Giljoo Nam, Seung-Hwan Baek |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42311 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24915
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Snapshot Polarimetric Display Inverse Rendering Seokjun Choi, Yunseong Moon, Kaizhang Kang, Hoon-Gyu Chung, Jin-Nyeong Kim, Giljoo Nam, Seung-Hwan Baek Abstract Inverse rendering remains a core challenge in graphics and vision, especially in the snapshot configurations required for lightweight desktop workflows, where the per-frame information budget is highly constrained. Previous inverse rendering work explores various available dimensions for enriching the per-shot information, including temporal modulation, spectral encoding, and polarization. In this work, we introduce polarimetric display inverse rendering, using an LCD to project a linearly polarized RGB binary pattern and an RGB polarization camera augmented with a quarter-wave plate to acquire spectro-polarimetric measurements in a single shot. A feed-forward transformer maps these measurements to per-pixel normal, albedo, roughness, and metallicity. To overcome training data scarcity, we expand a limited set of measured polarimetric bidirectional reflectance distribution functions via a generative manifold. Evaluations on a real desktop setup demonstrate accurate inverse rendering across diverse scenes, outperforming existing approaches. We present a snapshot feed-forward inverse rendering method that leverages spectro-polarimetric reflectance cues. A simple display-camera setup (a) captures a single-shot image, which (b) is decomposed into nine measurements encoding three RGB lighting directions and three polarization states: unpolarized, linearly-polarized (LP), circularly-polarized (CP). From these measurements, our network (c) predicts PBR maps that enable robust relighting under diverse illumination conditions. 1 Introduction Inverse rendering–the estimation of geometry and appearance from images–has long stood as a foundational challenge in computer graphics and vision. While decades of research, accelerated by recent learning-based solutions Li et al. 2025b ; He et al. 2024 ; Liang et al. 2025 , have produced high-quality results from multi-frame inputs, the snapshot setting required by lightweight desktop workflows remains challenging. Prior work exploits different dimensions of light to increase the capability of a single shot: high-speed temporal modulation that captures multiple lighting conditions within one exposure Wenger et al. 2005 ; Sun et al. 2020 , encoding multiple lighting directions across spectral channels Lattas et al. 2022 ; Kampouris et al. 2018 ; Ikehata and Asano 2024 , and linear polarization that separates diffuse from specular reflectance Lattas et al. 2022 ; Kampouris et al. 2018 ; Choi et al. 2025 ; Ichikawa et al. 2024 . This line of work shows that different light properties can provide additional information, motivating us to integrate spectral and polarimetric cues within one capture on a desktop setup. In this work, we propose polarimetric display inverse rendering, a snapshot pipeline built around three coupled components: a display-camera imaging system for spectro-polarimetric acquisition, a feed-forward inverse rendering network, and a measured-reflectance dataset expansion method for training. First, we construct a display-camera imaging system to capture a scene into nine spectro-polarimetric measurements within a single snapshot. On the illumination side, we use a liquid crystal display (LCD) to project a linearly polarized binary RGB pattern Lattas et al. 2022 ; Kampouris et al. 2018 ; Ikehata and Asano 2024 , probing the scene simultaneously from multiple lighting directions across the R, G, and B channels. We then use an RGB polarization camera, which by itself records only the linear polarization components of the reflected light. We augment it with a quarter-wave plate (QWP), enabling single-shot acquisition of not only vertical linear polarization but also circular polarization, which provides cues for distinguishing dielectric and metallic reflectance. The resulting observation forms a 3 \times 3 spectro-polarimetric representation: each of the three spectral channels is decomposed into three polarization states: unpolarized, vertically linearly polarized, and circularly polarized, which are related to diffuse, specular, and metallic reflectance components, respectively. Second, we use a feed-forward network to estimate material parameters from these nine measurements. We modify a transformer-based architecture Li et al. 2026 to treat the spectro-polarimetric measurements as a token sequence and exploit self-attention to integrate cues across polarization states and spectral channels, outputting per-pixel normal, albedo, roughness, and metallicity in a single forward pass. Third, we develop a method that expands a limited set of measured polarimetric bidirectional reflectance distribution functions (pBRDFs) Baek et al. 2020 ; Moon et al. 2025 into a diverse, physically valid pBRDF dataset for training our neural network. Specifically, the measured pBRDFs are represented in a compact basis using principal component analysis (PCA), and a weight generator learns a generative manifold over the resulting PCA coefficients. New pBRDFs are synthesized by sampling weights from this manifold and reconstructing them through the PCA basis. The synthesized pBRDFs are then assigned to geometry assets, enabling synthetic dataset rendering for training and testing the inverse rendering network. By integrating these components, we demonstrate that our method enables accurate, single-shot inverse rendering on a desktop setup, outperforming baselines. Our main contributions are summarized as follows: • A display-camera imaging system that probes the scene from multiple lighting directions across spectral channels with a linearly polarized RGB pattern and encodes linear-circular polarization in a single shot, producing nine spectro-polarimetric measurements. • A feed-forward transformer that estimates per-pixel normal, albedo, roughness, and metallicity from these spectro-polarimetric measurements in a single forward pass. • A data-driven approach that expands a limited set of measured pBRDFs into a diverse and physically valid pBRDF dataset by sampling on the learned manifold. 2 Related Work Inverse Rendering from Conventional Images Existing inverse-rendering methods often rely on multi-view Zhang et al. 2021b ; Zhang et al. 2021a ; Zhang et al. 2022b ; Jin et al. 2023 , multi-light Li and Li 2022 , or both Zhang et al. 2022a ; Yang et al. 2022 ; Chung et al. 2024 ; Chung et al. 2025 types of measurements. Such settings have been widely adopted, particularly in NeRF- Mildenhall et al. 2021 and 3DGS-based Kerbl et al. 2023 methods, where iterative optimization has led to high-quality reconstruction performance. However, these approaches typically require repeated captures of static scenes, making them less suitable for limited-observation settings. In parallel, recent works have explored directly estimating scene properties from images using learned priors. Diffusion-based methods Lyu et al. 2023 ; He et al. 2024 ; Chen et al. 2025a ; Chen et al. 2025b ; Liang et al. 2025 generate plausible results through iterative denoising guided by generative priors learned from large-scale data, requiring relatively fewer constraints on the capture setup. However, they often fail to provide sufficient constraints for a physically valid disentanglement of geometry and reflectance. Rather than relying on passive captures, our method encodes spectro-polarimetric information at capture time, providing a rich physical signal that complements learned priors. Inverse Rendering by Spectro-polarimetric Imaging. Various active–illumination systems have been developed to provide informative observations for inverse rendering, including mobile flash cameras Lichy et al. 2021 , large-scale light stages Ghosh et al. 2009 ; Wenger et al. 2005 ; Sun et al. 2020 , and display–camera systems Aittala et al. 2013 ; Choi et al. 2024 ; Choi et al. 2025 ; Lattas et al. 2022 . Display-based approaches provide spatially programmable area illumination, enabling rich lighting cues to be multiplexed into fewer captures, but they still require multiple captures. Spectral multiplexing has been widely used to encode diverse lighting directions into a small number of measurements, using gradient Ma et al. 2007 ; Ghosh et al. 2009 ; Fyffe and Debevec 2015 , binary Kampouris et