MUJICA: Reforming SISR Models for PBR Material Super-Resolution via Cross-Map Attention
2025
Read MUJICA: Reforming SISR Models for PBR Material Super-Resolution via Cross-Map Attention (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09802.txt.
Introduction
Xin Du, Maoyuan Xu, and Zhi Ying present MUJICA, an adapter that sharpens material texture maps for realistic rendering, in a research paper from 2025. Such materials use split maps for color, surface tilt, metal share, and roughness. Upscaling each map alone can make the maps clash and lose surface detail.
MUJICA attaches to a trained image upscaling model and leaves that model frozen. Attention across maps fuses their features while keeping the base skill of the model. Tests on three base models lift quality scores and keep maps in accord. Training stays light, which suits small labs. The tool reaches top results on material data sets.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-09802
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MUJICA: Reforming SISR Models for PBR Material Super-Resolution via Cross-Map Attention Thanks: Corresponding author. Affiliation: Xin Du, Maoyuan Xu, Zhi Ying Abstract Physically Based Rendering (PBR) materials are typically characterized by multiple 2D texture maps such as basecolor, normal, metallic, and roughness which encode spatially-varying bi-directional reflectance distribution function (SVBRDF) parameters to model surface reflectance properties and microfacet interactions. Upscaling SVBRDF material is valuable for modern 3D graphics applications. However, existing Single Image Super-Resolution (SISR) methods struggle with cross-map inconsistency, inadequate modeling of modality-specific features, and limited generalization due to data distribution shifts. In this work, we propose M ulti-modal U pscaling J oint I nference via C ross-map A ttention ( MUJICA ), a flexible adapter that reforms pre-trained Swin-transformer-based SISR models for PBR material super-resolution. MUJICA is seamlessly attached after the pre-trained and frozen SISR backbone. It leverages cross-map attention to fuse features while preserving remarkable reconstruction ability of the pre-trained SISR model. Applied to SISR models such as SwinIR, DRCT and HMANet, MUJICA improves PSNR, SSIM, and LPIPS scores while preserving cross-map consistency. Experiments demonstrate that MUJICA enables efficient training even with limited resources and delivers state-of-the-art performance on PBR material datasets. Introduction Physically Based Rendering (PBR) materials consist of different 2D texture maps such as basecolor, normal, roughness and metallic, that describe the appearance of virtual 3D shapes under arbitrary lighting in the form of spatially-varying bi-directional reflectance distribution function (SVBRDF). The industry continues to explore ways to integrate deep learning methods into modern PBR material production pipeline for various reasons. One major motivation is the requirement to upscale low-resolution legacy PBR materials to a higher resolution when developing remastered versions of games. Since legacy assets were created without using tools like Adobe Substance 3D Sampler, it is usually not possible to regenerate at arbitrary resolution. Another key factor is the growing adoption of diffusion models ( Podell et al. 2023 ; Ho, Jain, and Abbeel 2020 ; Sohl-Dickstein et al. 2015 ; Song and Ermon 2019 ; Song et al. 2020 ) to generate high-quality PBR materials ( Vecchio et al. 2024a ; Ye et al. 2024 ; Saravanan et al. 2025 ; Vecchio et al. 2024b ; Esser et al. 2024 ) . However, the native output resolution of these models is typically limited to 512\times 512 or 1024\times 1024 in current industrial practice due to limited computation resources, failing to reach industrial requirements such as 4096\times 4096 . Therefore, reconstructing high-resolution PBR materials from low-resolution inputs has become an important component of modern industry pipelines. In the task of Single Image Super-Resolution (SISR), methods based on Swin Transformer ( Liu et al. 2021 ) have achieved remarkable success over CNN-based methods ( Ledig et al. 2017b ; Lim et al. 2017 ; Li et al. 2020 ; Wang et al. 2018 ; Zhang et al. 2018b ) , but their direct application to PBR materials faces three main limitations: • Cross-Map Inconsistency. As shown in fig. 1 (a), SISR models process each PBR material map such as basecolor and normal separately, breaking consistency between maps. Such map inconsistency results in inconsistent renderings under varying lighting conditions. • Texture Distortion. Figure 1 (b) demonstrates that directly applying pre-trained state-of-the-art (SOTA) SISR models to PBR materials produces incorrect results like texture distortion. This is mostly due to the different data distribution between the natural images that SISR models are trained on and the PBR materials. • Limited Datasets. Available datasets for training PBR material super-resolution (SR) models are much more limited compared with those for SISR. To train our model, we employ MatSynth ( Vecchio and Deschaintre 2024 ) , the only publicly available SVBRDF dataset for our task, along with our in-house dataset. Remarkably, the textural and compositional complexity in single natural image often significantly surpasses single PBR material as shown in fig. 1 (c). Hence, training a PBR material SR model with limited data resources presents a significant challenge. Figure 1: Limitations of directly applying SISR models on PBR materials. To address the challenges above, we propose M ulti-modal U pscaling J oint I nference via C ross-map A ttention ( MUJICA ) in this paper to adapt SOTA SISR models like SwinIR ( Liang et al. 2021 ) , DRCT ( Hsu, Lee, and Chou 2024 ) and HMANet ( Chu et al. 2024 ) to PBR material SR task by cross-map feature fusion. To better capture the intricate interdependence among different PBR material maps, MUJICA adopts an attention-based fusion strategy. This design allows for both shared representation learning and flexible map specific refinement, facilitating robust multi-modal modeling under the complex structural priors inherent in PBR materials. Our contributions are summarized as below • Multi-modal Upscaling. MUJICA reformulates PBR material SR as a multi‑modal fusion problem, achieving SOTA performance on existing PBR material datasets. • Adapter for SISR models. MUJICA as an adapter to reform existing Swin‑transformer‑based SISR models to multi-modal SR models. Equipped with MUJICA, existing SISR models have the ability to deal with PBR materials while keeping their remarkable performance. • Efficient Training. By integrating MUJICA with any frozen, pre‑trained Swin‑transformer‑based SISR model as the backbone, only a small number of parameters need to be trained. Therefore, our proposed method can keep training requirements and hardware demands at a relatively low level while achieving SOTA performance even with limited training data. Experiments demonstrate that MUJICA achieves SOTA performance on existing PBR material datasets. For \times 2 SR task, MUJICAs outperform their SISR backbones with gains of up to 1.15dB in PSNR, 0.0069 in SSIM, and a reduction of 0.036 in LPIPS on renderings across datasets. For \times 4 SR task, MUJICAs improve metrics up to 0.76dB in PSNR, 0.0070 in SSIM, and 0.0695 in LPIPS. Related Work Single Image Super-Resolution In SISR task, transformer-based SISR methods ( Zhou et al. 2023 ; Chen et al. 2023b ; Liang et al. 2021 ; Hsu, Lee, and Chou 2024 ; Chu et al. 2024 ) achieve better performance than traditional CNN-based methods ( Ledig et al. 2017b ; Lim et al. 2017 ; Li et al. 2020 ; Wang et al. 2018 ; Zhang et al. 2018b ) . Swin-transformer-based Method. SwinIR ( Liang et al. 2021 ) , a classic Swin-transformer-based ( Vaswani et al. 2017 ; Liu et al. 2021 ) SISR model, performs self-attention within 8\times 8 local windows to extract deep features that significantly helps improve reconstructed high-resolution results. DRCT ( Hsu, Lee, and Chou 2024 ) adds dense-connections ( Huang et al. 2017 ) inside its residual blocks based on the architecture of SwinIR to mitigate the spatial information vanishing in deep feature extraction modules. HMANet ( Chu et al. 2024 ) aims to extract self-similarity of images, including local similarity between nearby regions and global similarity across distant areas, by adding grid-attention to window-based self-attention mechanism. GAN-based Method. Although GAN-based ( Wang et al. 2018 ; Ledig et al. 2017a ; Wu et al. 2017 ; Zhang et al. 2021 ) methods are able to generate visually richer textures, the adversarial loss tends to encourage the generator to produce plausible-looking details ( Ledig et al. 2017a ; Zhang et al. 2018a ) , even if those