DualMat: PBR Material Estimation via Coherent Dual-Path Diffusion Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland IS
Yifeng Huang, Zhang Chen, Yi Xu, Minh Hoai, Zhong Li † · 2025
Read DualMat: PBR Material Estimation via Coherent Dual-Path Diffusion Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland IS by Yifeng Huang, Zhang Chen, Yi Xu, Minh Hoai, Zhong Li † (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05060.txt.
Introduction
Yifeng Huang, Zhang Chen, Yi Xu, Minh Hoai, and Zhong Li estimate a surface's full material makeup from a single photo, in a 2025 paper. Physically based rendering needs albedo plus metallic and roughness maps, and guessing them under messy lighting is hard. One network forced to do everything tends to blur the different jobs.
Their DualMat framework splits the work across two diffusion paths. One path, working in image space, leans on pretrained visual knowledge to get color right. The other works in a compact space tuned for metal and roughness. Feature distillation during training keeps the two paths in agreement. A rectified flow design keeps inference fast, and patch-based estimation with cross-view attention handles large or multiple images. On standard and real-world data it beat prior methods clearly. The authors target image-to-3D pipelines hungry for believable materials.
Catalog record
| CREATOR | Yifeng Huang, Zhang Chen, Yi Xu, Minh Hoai, Zhong Li † |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05060
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DualMat: PBR Material Estimation via Coherent Dual-Path Diffusion Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland ISBN: 979-8-4007-2035-2/2025/10 DOI: 10.1145/3746027.3755237 CCS: Computing methodologies Computer vision Yifeng Huang, Zhang Chen, Yi Xu, Minh Hoai, Zhong Li † Abstract . We present DualMat, a novel dual-path diffusion framework for estimating Physically Based Rendering (PBR) materials from single images under complex lighting conditions. Our approach operates in two distinct latent spaces: an albedo-optimized path leveraging pretrained visual knowledge through RGB latent space, and a material-specialized path operating in a compact latent space designed for precise metallic and roughness estimation. To ensure coherent predictions between the albedo-optimized and material-specialized paths, we introduce feature distillation during training. We employ rectified flow to enhance efficiency by reducing inference steps while maintaining quality. Our framework extends to high-resolution and multi-view inputs through patch-based estimation and cross-view attention, enabling seamless integration into image-to-3D pipelines. DualMat achieves state-of-the-art performance on both Objaverse and real-world data, significantly outperforming existing methods with up to 28% improvement in albedo estimation and 39% reduction in metallic-roughness prediction errors. Our project can be found at yifehuang97.github.io/DualMatProjPage/ . 1. Introduction Physically-Based Rendering (PBR) materials are fundamental to realistic computer graphics, defining how rendered surfaces should interact with light through properties such as diffuse reflection, metallic behavior, and surface roughness. These materials form the cornerstone of modern graphics applications, from video games and digital content creation to augmented/virtual reality (AR/VR). Accurate PBR material estimation is crucial for faithfully replicating real-world materials in digital environments, yet recovering these properties from a single image remains challenging due to the complex interplay of material attributes, lighting conditions, and underlying geometry. Figure 1. Given a single input image (left), DualMat’s dual-path design combines an albedo-optimized path and a material-specialized path to achieve accurate PBR material estimation. The importance of accurate PBR material estimation has been further amplified by recent breakthroughs in image-to-3D generation. Modern image-to-3D pipelines aim to reconstruct complete 3D assets from images. Early works like DreamFusion ( Poole et al. 2022 ) distill a 3D representation with NeRF ( Mildenhall et al. 2021 ) from a 2D image diffusion model with Score Distillation Sampling loss. Recent works either employ large transformer architectures ( Hong et al. 2023 ; Li et al. 2023b ; Wang et al. 2023 ; Xu et al. 2024a ; Xu et al. 2024b ; Xu et al. 2023 ) to train a triplane representation in a data-driven manner, or explicitly generate multiview images and then reconstruct 3D models from the generated images ( Shi et al. 2023b ; Liu et al. 2023b ; Long et al. 2024 ; Shi et al. 2023a ; Liu et al. 2023c ; Li et al. 2024 ; Wang and Shi 2023 ; Tang et al. 2023 ; Wang et al. 2024 ) . However, these methods primarily focus on geometry and view-dependent appearance, often producing simplified or view-dependent textures that lack explicit material properties. This limitation severely restricts their applications in scenarios requiring physically accurate rendering, such as relighting under different environment maps or realistic integration into virtual environments. A robust PBR material estimation method could bridge this gap, enabling these 3D generation pipelines to produce assets with physically accurate rendering capabilities. To address these challenges, PBR material estimation is crucial and can be formulated as an image-to-2D map translation task. Traditionally, this task has been tackled with methods heavily reliant on deep convolutional networks ( Vecchio et al. 2021 ; Li et al. 2017 ; Ye et al. 2018 ; Li et al. 2018 ; Guo et al. 2021 ; Sang and Chandraker 2020 ; Zhou and Kalantari 2021 ; Deschaintre et al. 2018 ; Deschaintre et al. 2019 ; Ye et al. 2021 ; Guo et al. 2020b ; Wen et al. 2022 ; Henzler et al. 2021 ; Zhou et al. 2022 ; Hu et al. 2022 ; Guo et al. 2020a ; Liu et al. 2023a ; Li et al. 2023a ) . While methods such as SurfaceNet have demonstrated the ability to estimate spatially-varying bidirectional reflectance distribution functions (SVBRDF) from a single image, these convolution-based architectures face significant limitations: they struggle to capture high-frequency details, resulting in blurred or oversimplified material maps; their performance degrades in multi-view scenarios where consistency across viewpoints is critical; and they lack the expressive power needed to generate coherent outputs under varying lighting conditions. The limitations of these conventional approaches suggest the need for a more powerful generative framework. Recent advances in generative models, particularly diffusion models ( Ho et al. 2020 ; Ho et al. 2022 ; Karras et al. 2022 ; Rombach et al. 2022 ; Song et al. 2021 ; Liu et al. 2024 ; Xiong et al. 2025 ) , offer promising solutions to these challenges. These models have demonstrated remarkable success in high-quality image synthesis ( Esser et al. 2024 ; Podell et al. 2023 ) , material generation ( Vainer et al. 2024 ; Vecchio et al. 2024b ; Vecchio et al. 2024a ; Vecchio 2024 ; Sartor and Peers 2023 ) , super-resolution ( Saharia et al. 2022b ; Gao et al. 2023 ; Yue et al. 2024 ) , and image translation ( Su et al. 2022 ; Zhao et al. 2022 ; Sasaki et al. 2021 ) . Motivated by their ability to capture complex data distributions and generate high-fidelity outputs, we formulate PBR material estimation using a diffusion-based approach. Building on these insights, we present DualMat, a novel dual-path diffusion framework for estimating PBR materials from single images, as shown in Fig. 1 . Our approach operates in two distinct latent spaces to effectively capture material properties and ensure consistency with input images. The albedo-optimized path leverages pretrained visual knowledge through RGB latent space to generate high-quality albedo maps, while the material-specialized path operates in a compact latent space designed to precisely model metallic and roughness properties. To ensure coherent predictions between these complementary paths, we introduce a feature distillation loss during training that aligns intermediate representations. We further enhance computational efficiency by incorporating rectified flow, significantly reducing the number of denoising steps needed in both training and inference stages while preserving high-quality results. Our framework extends beyond single-image estimation through two crucial adaptations: a patch-based strategy for high-resolution inputs that preserves fine details while maintaining global consistency, and a cross-view attention mechanism for multi-view scenarios that ensures consistent material estimation across viewpoints. These capabilities position DualMat as a versatile PBR estimator that seamlessly integrates into existing image-to-3D generation pipelines, enabling applications from realistic relighting to textured mesh generation. In short, the primary contributions of our work are: (1) we propose a novel dual-path diffusion framework for PBR material estimation that leverages complementary latent spaces, consisting of an albedo-optimized path utilizing pre-trained visual knowledge and a material-specialized path dedicated to precise property estimation, combined with a feature distillation training strategy to ensure coherent outputs; (2) we design an efficient inference framework incorporating rectified flow sampling, patch-based high-resolution processing, and cross-view attention mechanisms, enabling high-quality material generation within just 2–4 steps and supporting seamless integration into