Guided Lensless Polarization Imaging

Noa Kraicer, Erez Yosef, Raja Giryes · 2026

Read Guided Lensless Polarization Imaging by Noa Kraicer, Erez Yosef, Raja Giryes (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27357.txt.

Introduction

Kraicer and co-authors guide lensless polarization imaging with an ordinary RGB camera for sharper results. Polarization reveals cues the eye misses, but proper cameras are pricey and bulky. Lensless designs swap the lens for a diffuser and rebuild images computationally. Their system pairs a compact polarization-RGB sensor with a standard RGB camera. A physics-based inversion first recovers a rough polarization image per color channel. A transformer fusion network then refines it using the RGB guidance. Quality beat lensless-only baselines across datasets and conditions.

Catalog record

CREATORNoa Kraicer, Erez Yosef, Raja Giryes
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57141
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27357

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Guided Lensless Polarization Imaging Noa Kraicer, Erez Yosef, Raja Giryes Abstract Polarization imaging captures the polarization state of light, revealing information invisible to the human eye yet valuable in domains such as biomedical diagnostics, autonomous driving, and remote sensing. However, conventional polarization cameras are often expensive, bulky, or both, limiting their practical use. Lensless imaging offers a compact, low-cost alternative by replacing the lens with a simple optical element like a diffuser and performing computational reconstruction, but existing lensless polarization systems suffer from limited reconstruction quality. To overcome these limitations, we introduce a RGB-guided lensless polarization imaging system that combines a compact polarization-RGB sensor with an auxiliary, widely available conventional RGB camera providing structural guidance. We reconstruct multi-angle polarization images for each RGB color channel through a two-stage pipeline: a physics-based inversion recovers an initial polarization image, followed by a Transformer-based fusion network that refines this reconstruction using the RGB guidance image from the conventional RGB camera. Our two-stage method significantly improves reconstruction quality and fidelity over lensless-only baselines, generalizes across datasets and imaging conditions, and achieves high-quality real-world results on our physical prototype lensless camera without any fine-tuning. (a) Our optical setup (b) Self-fabricated polarization mask (c) Captured lensless polarization image (d) Reconstructed polarization result Figure 1 : RGB-guided lensless polarization imaging system: (a) optical setup; (b) custom polarization mask; (c) captured lensless image under front illumination with two orthogonally polarized projectors; and (d) reconstructed grayscale polarization result, visualized by mapping the 0^{\circ} , 45^{\circ} , and 90^{\circ} outputs to the R, G, and B channels. 1 Introduction Polarization is a fundamental property of light that describes the orientation and phase relationship between its orthogonal electric-field components. Because reflections and material anisotropy affect the polarization state, it encodes information about surface geometry, reflectance, and composition, details often inaccessible to standard intensity or RGB imaging 46 ; 21 . Figure 2 : Overview of the proposed RGB-guided reconstruction pipeline. The process consists of two stages: (1) polarization intensity images (color or grayscale) are reconstructed from lensless measurements using a physics-based algorithm (FISTA/ADMM); and (2) the initial reconstruction and a registered RGB image of the same scene are separately encoded and fused through cross-domain attention to produce a refined polarization reconstruction. For visualization, the grayscale reconstructions at three polarization angles ( 0^{\circ} , 45^{\circ} , 90^{\circ} ) are mapped to the R, G, and B channels. The pipeline is compatible with more general input configurations. Recent approaches to polarization imaging include division-of-focal-plane (DoFP) sensors, which assign different polarization filters to adjacent pixels; time-sequential polarimetry, which uses rotating or tunable elements to capture multiple polarization states over time 46 , and lensless polarization imaging, which replaces conventional lenses with spatially coded optical elements and reconstructs the image computationally. Despite its utility, polarization imaging has yet to achieve widespread adoption, largely due to the cost, size, and complexity of conventional polarization cameras. Lensless imaging offers a compelling alternative by replacing lenses with simple optical elements, such as diffusers or coded masks, and shifting hardware complexity to computation. Such schemes enable compact, low-cost, and scalable imaging systems 11 . Recent works have extended lensless imaging to polarization by combining polarization-sensitive components with various optical coding schemes 25 ; 16 ; 5 ; 47 . Yet, reconstruction quality remains limited due to the highly compressed measurements that jointly encode structural and polarization information, compounded by the inherently ill-posed nature of the inverse problem. Classical model-based solvers, such as the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) 8 and the Alternating Direction Method of Multipliers (ADMM) algorithm 13 , can recover coarse polarization estimates but often fail to recover high-frequency details and are sensitive to noise and deviations from the assumed imaging model. While deep learning models have significantly improved reconstruction quality in general lensless imaging tasks 11 , to the best of our knowledge, they have not yet been explored for lensless polarization imaging. RGB images provide complementary geometric and edge information consistent across polarization states, offering a natural way to regularize the ill-posed inversion and recover fine details. While adding an RGB camera increases system complexity compared to a fully lensless design, RGB cameras are compact, low-cost, and widely available, keeping the system simpler and more affordable than dedicated polarization cameras. This is particularly important for size- and cost-constrained applications such as compact microscopy and endoscopy 42 ; 38 ; 39 ; 52 . Leveraging the complementary geometric information provided by RGB images, we introduce the first RGB-guided lensless polarization imaging system that integrates physics-based reconstruction with data-driven refinement ( Figure 2 ), enabling accurate recovery of fine structures and details. The first stage performs a model-based inversion (e.g., FISTA or ADMM) to obtain a physically consistent initialization from raw lensless measurements. The second stage refines this estimate using a Swin Transformer–based fusion network, based on SwinFuSR 3 , which fuses polarization features from the initial reconstruction with RGB features from a lensed camera via alternating self- and cross-attention. Our approach generalizes across diverse scenes and imaging conditions, and achieves strong performance on real-world measurements without additional fine-tuning. Our main contributions may be summarized as follows: (i) We propose the first RGB-guided lensless polarization imaging system, combining simple and low-cost hardware with a reconstruction algorithm achieving state-of-the-art results for lensless polarization imaging; (ii) We design a two-stage reconstruction approach that integrates a physics-based solver (e.g., FISTA/ADMM) with an adapted version of Swin Transformer utilized for cross-modal fusion, enabling RGB-guided reconstruction of polarization intensity images through self- and cross-attention; (iii) We conduct extensive experiments on multiple simulated datasets, demonstrating consistent improvements over lensless-only baselines in PSNR, SSIM, and LPIPS, with strong generalization across datasets and unseen point-spread-functions (PSFs); (iv) We demonstrate promising real-world results on a prototype lensless polarization camera, validating the method’s practical feasibility without additional fine-tuning. Project page with code, pretrained models, and data: https://noa-kraicer.github.io/Guided-Lensless-Polarization-Imaging/ . 2 Related Work Polarization Imaging. Most existing polarization cameras are based on sequential filtering or division-of-focal-plane (DoFP) architectures 46 . DoFP sensors enable single-shot capture but suffer spatial resolution loss due to pixel subdivision. Sequential filtering preserves full resolution via multiple exposures with rotating or switching polarizers but introduces motion artifacts and mechanical complexity, limiting real-time use. Recent designs address these trade-offs using stacked polarizer architectures 41 and flat-optics or metasurface-based polarization elements 20 ; 65 ; 27 , signaling a shift toward miniaturized polarization cameras and motivating exploration of lensless variants. Lensless Imaging. Lensless imaging replaces traditional lenses with coded optics and computational reconstruction, enabling compact imaging systems. Several designs have been proposed, including FlatCam 4 , which uses a static amplitude mask; DiffuserCam 1 , based on a diffuser and compressive sensing; phase masks 10 ; and programmable optics 64 ; 35 ; 18 ; 62 ; 19 . A comprehensive review of lensless imaging systems can be found in 12 . Subsequent research has extended lensless imaging to additional modalities, including hyperspectral 36 , depth 6 , and temporal imaging 2 , highlighting the versatility of the approach. However, lensless polarization imaging has received limited attention and introduces a more complex forward model. Lensless Polarization Imaging Combining lensless imaging with polarization sensing enables compact, multi-modal imaging systems. Prior works have explored various hardware configurations for single-shot

Read the complete text