PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution

Shuangfan Zhou # 1 {}^{1}\lx@frontmatter@keepsup{\#} Chu Zhou # 2 {}^{2}\lx@frontmatter@keepsup{\#} Youwei Lyu 1 Heng Guo * 1 {}^{1}\lx@frontmatter@keepsup{*} Zhanyu Ma 1 Boxin Shi 3,4 Imari Sato 2 1 School of Artificial Intelligence, Beijing University of Posts and Telecommunications 2 National Ins · 2025

Read PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution by Shuangfan Zhou # 1 {}^{1}\lx@frontmatter@keepsup{\#} Chu Zhou # 2 {}^{2}\lx@frontmatter@keepsup{\#} Youwei Lyu 1 Heng Guo * 1 {}^{1}\lx@frontmatter@keepsup{*} Zhanyu Ma 1 Boxin Shi 3,4 Imari Sato 2 1 School of Artificial Intelligence, Beijing University of Posts and Telecommunications 2 National Ins (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07758.txt.

Introduction

Shuangfan Zhou and co-authors present a joint method for cleaning up and enlarging polarised images, in a 2025 paper. Polarisation cameras capture light at several angles in one shot, but their raw output is a mosaic that must be rebuilt into full colour images. That rebuilding step adds errors in the very measures users care about, such as how strong and at what angle light is polarised.

Past methods either rebuild the image without raising its resolution, or enlarge a rebuilt image and carry its errors along. The new method does both jobs at once, straight from the raw mosaic. Tests on made-up and real data gave top results, the authors report, with more accurate polarisation measures. Downstream tasks that rely on polarisation also worked better with its output.

Catalog record

CREATORShuangfan Zhou # 1 {}^{1}\lx@frontmatter@keepsup{\#} Chu Zhou # 2 {}^{2}\lx@frontmatter@keepsup{\#} Youwei Lyu 1 Heng Guo * 1 {}^{1}\lx@frontmatter@keepsup{*} Zhanyu Ma 1 Boxin Shi 3,4 Imari Sato 2 1 School of Artificial Intelligence, Beijing University of Posts and Telecommunications 2 National Ins
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-07758

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution Shuangfan Zhou # 1 {}^{1}\lx@frontmatter@keepsup{\#} Chu Zhou # 2 {}^{2}\lx@frontmatter@keepsup{\#} Youwei Lyu 1 Heng Guo * 1 {}^{1}\lx@frontmatter@keepsup{*} Zhanyu Ma 1 Boxin Shi 3,4 Imari Sato 2 1 School of Artificial Intelligence, Beijing University of Posts and Telecommunications 2 National Institute of Informatics 3 State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University 4 National Engineering Research Center of Visual Technology, School of Computer Science, Peking University {zhoushuangfan, youweilv, guoheng, mazhanyu}@bupt.edu.cn [email protected] [email protected] [email protected] Abstract Polarization cameras can capture multiple polarized images with different polarizer angles in a single shot, bringing convenience to polarization-based downstream tasks. However, their direct outputs are color-polarization filter array (CPFA) raw images, requiring demosaicing to reconstruct full-resolution, full-color polarized images; unfortunately, this necessary step introduces artifacts that make polarization-related parameters such as the degree of polarization (DoP) and angle of polarization (AoP) prone to error. Besides, limited by the hardware design, the resolution of a polarization camera is often much lower than that of a conventional RGB camera. Existing polarized image demosaicing (PID) methods are limited in that they cannot enhance resolution, while polarized image super-resolution (PISR) methods, though designed to obtain high-resolution (HR) polarized images from the demosaicing results, tend to retain or even amplify errors in the DoP and AoP introduced by demosaicing artifacts. In this paper, we propose PIDSR , a joint framework that performs complementary P olarized I mage D emosaicing and S uper- R esolution, showing the ability to robustly obtain high-quality HR polarized images with more accurate DoP and AoP from a CPFA raw image in a direct manner. Experiments show our PIDSR not only achieves state-of-the-art performance on both synthetic and real data, but also facilitates downstream tasks. Figure 1 : Top: The concept of polarized image demosaicing (PID) and polarized image super-resolution (PISR). Mid: An example shows that the baseline (PID \rightarrow PISR) works in a sequential manner, where the AoPs calculated from the demosaicing results (produced by TCPDNet [ 23 ] ) and the HR polarized images (produced by PSRNet [ 8 ] ) suffer from severe artifacts. Bottom: An example shows that our PIDSR works in a complementary manner, where the calculated AoPs are more accurate. We choose k=4 here. 1 Introduction Polarization-based vision has benefited various applications, such as shape from polarization [ 1 , 2 ] , reflection removal [ 19 ] , image dehazing [ 41 ] , HDR imaging [ 42 ] , etc . By fully utilizing the physical clues encoded in the polarization-relevant parameters such as the degree of polarization (DoP) and angle of polarization (AoP), polarization-based methods often achieve higher performance compared with the image-based ones, showing promising potentials. To acquire the DoP and AoP, at least three polarized images with different polarizer angles are required. While a polarizer can be used for this purpose, it demands multiple shots, making the capture process quite inconvenient. Empowered by the division of focal plane (DoFP) technology, a polarization camera can capture four color polarized images with different polarizer angles ( 0^{\circ},45^{\circ},90^{\circ},135^{\circ} ) in a single shot, bringing convenience to the acquisition of the DoP and AoP. Since DoFP uses color-polarization filter array (CPFA) to record the color and polarization information simultaneously, the direct output of a polarization camera is a CPFA raw image. As shown in the left part of Fig. 1 (top), each pixel in a CPFA raw image contains information about only one color channel and one polarizer angle, which means that demosaicing is required to reconstruct the corresponding polarized images. Since unavoidable demosaicing artifacts tend to be amplified by the non-linearity of subsequent calculations, the DoP and AoP acquired from a polarization camera usually have a higher level of error than those acquired from a polarizer, making the physical clues less distinctive. Besides, limited by the hardware design, the resolution of a polarization camera is often much lower than that of a conventional RGB camera, restricting the fidelity of the recorded information. Thus, obtaining high-quality high-resolution (HR) polarized images with more accurate DoP and AoP from a polarization camera is of practical significance. Despite its importance, a practical and reliable approach to simultaneously achieve demosaicing and super-resolution of polarization images has yet to be developed. As shown in Fig. 1 (top), the most straightforward way is to sequentially perform polarized image demosaicing (PID) [ 20 , 22 , 23 ] and polarized image super-resolution (PISR) [ 8 , 37 ] on the CPFA raw image, i.e . , perform “PID \rightarrow PISR”. Defining \mathbf{R}\in\mathbb{R}^{1\times h\times w} ( h and w are the height and width respectively) as the CPFA raw image, \mathbf{I}_{\alpha_{1,2,3,4}}\in\mathbb{R}^{3\times h\times w} ( \alpha_{1,2,3,4}=0^{\circ},45^{\circ},90^{\circ},135^{\circ} are the polarizer angles) as the four full-color polarized images, and \mathbf{I}^{\text{HR}}_{\alpha_{1,2,3,4}}\in\mathbb{R}^{3\times kh\times kw} ( k is the SR scale) as the four HR polarized images respectively, the process of PID \rightarrow PISR can be written as \mathbf{I}_{\alpha_{1,2,3,4}}=\mathcal{D}(\mathbf{R})\;\text{first, then}\;\mathbf{I}^{\text{HR}}_{\alpha_{1,2,3,4}}=\uparrow(\mathbf{I}_{\alpha_{1,2,3,4}}), (1) where \mathcal{D} and \uparrow represent demosaicing and super-resolution (SR) respectively. However, PID \rightarrow PISR would produce degenerated results, reducing the accuracy of the DoP and AoP, as shown in Fig. 1 (mid). This is because existing PISR methods [ 8 , 37 ] usually assume that the inputs are free of demosaicing artifacts, while existing PID methods [ 20 , 22 , 23 ] cannot guarantee perfect outputs. Therefore, the essential question needs to be addressed is: How to robustly obtain \mathbf{I}^{\text{HR}}_{\alpha_{1,2,3,4}} from \mathbf{R} in a direct manner? We observe that the spatial resolution often correlates negatively with the severity of demosaicing artifacts, suggesting that enhancing resolution can benefit PID, while suppressing demosaicing artifacts could, in turn, improve the performance of PISR. The observation indicates that PID and PISR may be complementary, i.e . , optimizing both of them in a single framework can potentially enhance each other’s performance. This motivates us to propose PIDSR , a joint framework that performs complementary P olarized I mage D emosaicing and S uper- R esolution. As shown in Fig. 1 (bottom), given a CPFA raw image \mathbf{R} , our PIDSR can not only output the demosaicing results \mathbf{I}_{\alpha_{1,2,3,4}} with fewer artifacts, but also output the HR polarized images \mathbf{I}^{\text{HR}}_{\alpha_{1,2,3,4}} with higher quality, which can be described as \mathbf{I}^{\text{HR}}_{\alpha_{1,2,3,4}},\mathbf{I}_{\alpha_{1,2,3,4}}=\mathcal{D}^{\uparrow}(\mathbf{R}), (2) where \mathcal{D}^{\uparrow} denotes complementary demosaicing and SR. Here, it is non-trivial to carefully design the formulation of \mathcal{D}^{\uparrow} , since naively formulating \mathcal{D}^{\uparrow} as a combination of \mathcal{D} and \uparrow would result in error accumulation. To reduce the level of error, we propose to formulate \mathcal{D}^{\uparrow} as a series of polarized pixel reconstruction sub-problems, and introduce a two-stage pipeline to handle the intra-resolution and cross-resolution components of each sub-problem in a recurrent manner, fully exploiting the complementary aspects of \mathcal{D} and \uparrow to optimize each other jointly. Tailored to the pipeline, we design a neural network to explicitly inject the physical clues into both two stages to preserve the polarization properties, making full use of the Stokes-domain information of the polarized images. To summarize, this paper makes contributions by demonstrating: (1) PIDSR, a complementary polarized image demosaicing and super-resolution framework , including: (2) a two-stage recurrent pipeline to fundamentally reduce the level of error; and (3) a Stokes-aided neural network to preserve the

Read the complete text