Optimal Multispectral Imaging using RGB Cameras

1 st Tomislav Matulić, 2 nd Ivan Škrabo, 3 rd Dubravko Babić, 4 th Damir Seršić · 2026

Read Optimal Multispectral Imaging using RGB Cameras by 1 st Tomislav Matulić, 2 nd Ivan Škrabo, 3 rd Dubravko Babić, 4 th Damir Seršić (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19460.txt.

Introduction

Matulić and colleagues present a physics-driven method for judging discrete spectral bands with a low-cost multispectral setup built from RGB cameras and narrow multi-band filters. Camera responses are written as linear mixes of unknown spectral parts, with mix coefficients set by the overlap between camera sensitivities and filter transmittances. For multi-camera setups, per-camera models stack into one global system fixed by wavelength allocation across camera-filter units. Wavelength allocation is treated as a design problem, and the configuration minimizing error is chosen.

Catalog record

CREATOR1 st Tomislav Matulić, 2 nd Ivan Škrabo, 3 rd Dubravko Babić, 4 th Damir Seršić
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45115
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19460

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optimal Multispectral Imaging using RGB Cameras Thanks: This work has been supported by the Croatian Government project NPOO.C3.2.R3-I1.04.0033 ”Scalable System of Cameras and Optical Filters for Industrial Applications (SKORPI)” 2024. 1 st Tomislav Matulić, 2 nd Ivan Škrabo, 3 rd Dubravko Babić, 4 th Damir Seršić Abstract We present a physics-driven framework for accurate evaluation of discrete spectral bands using a low-cost multispectral setup built from off-the-shelf RGB cameras and narrow multi-band optical filters. The approach starts by explicitly formulating a linear measurement model. The camera responses are expressed as linear mixtures of unknown spectral components, with mixing coefficients determined by the overlap between the camera spectral sensitivities and the filter transmittances. For a multi-camera configuration, the per-camera models are stacked into a single global system whose structure is fully determined by the allocation of target wavelengths across the camera–filter units. We pose wavelength allocation as a deterministic design problem and select the configuration that minimizes the spectral condition number of the resulting system matrix. Guided by a frame-theoretic interpretation, this criterion promotes numerical stability, maximizes worst-case output signal-to-noise ratio, and improves the robustness of spectral reconstruction. The design space is finite, enabling the evaluation of all feasible configurations under practical constraints. We demonstrate the method on a representative example with 12 target wavelengths and four triband filters, and identify the wavelength allocation that yields the most stable and noise-robust recovery. The proposed framework includes redundant configurations, in which individual wavelengths are measured by multiple cameras, thereby providing additional degrees of freedom that further improve noise robustness. University of Zagreb Zagreb, Croatia [email protected] I Introduction Multispectral imaging (MSI) refers to the acquisition of co-registered images of the same scene in a small number of distinct, discretely positioned spectral bands. By sampling the radiance of the scene at multiple wavelengths, MSI provides information that is inaccessible to standard RGB imaging, enabling improved material discrimination and quantitative spectral analysis. As a result, MSI has become an important tool in remote sensing and agriculture, medical imaging, food safety and quality inspection, cultural-heritage documentation and art conservation [ 17 , 5 , 15 , 11 , 7 ] . Despite these benefits, MSI hardware is often substantially more complex and costly than standard RGB imaging systems. Many acquisition concepts rely on additional wavelength-selection optics and associated calibration procedures, which increase cost and system complexity, and impose trade-offs in acquisition speed and robustness. A practical alternative is therefore to construct MSI systems from mass-produced RGB sensors combined with carefully designed narrow bandpass optical filters. In such configurations, the measured RGB values represent linear mixtures of the unknown spectral components, so spectral recovery becomes an inverse problem whose accuracy depends strongly on the forward model and its numerical conditioning. The primary motivation for using off-the-shelf RGB cameras combined with tri-band optical filters is the ability to customize the spectral response of the MSI system after the sensors and cameras have been manufactured. In contrast, many multispectral cameras rely on multi-zone (mosaic) filters deposited directly on the sensor (Bayer patterns), where 16, 25, or more wavelength channels are permanently assigned to pixels during fabrication. Because this wavelength layout is fixed at the manufacturing time, the selection of spectral bands cannot be modified once the camera is built. However, with external tri-band filters, the user can reconfigure the wavelength set simply by replacing the filters mounted in front of the RGB cameras. This post-manufacturing reconfigurability provides a level of flexibility that is typically not available in conventional mosaic-filter multispectral cameras. The central aim of this work is to enable accurate evaluation of spectral bands in an off-the-shelf multi-camera configuration by (i) explicitly formulating the measurement model and (ii) guiding the filter/camera wavelength allocation toward numerically stable and noise-robust reconstruction. I-A Related work Alternative low-cost MSI systems have been demonstrated in several hardware configurations, including multi-camera and filtered-RGB concepts [ 9 , 10 , 12 ] . These systems are naturally described by a linear forward model relating the scene spectral radiance to sensor responses. Recovering spectra from the measurements has therefore been widely studied, ranging from classical linear regression and basis-projection approaches to modern learning-based reconstruction methods [ 6 , 1 , 4 ] . While learning-based methods can achieve strong performance, physics-driven linear models remain attractive when calibration transparency, interpretability, and deterministic behavior are required. A critical factor in linear reconstruction is the conditioning of the system matrix. It maps spectral samples to observations: poorly conditioned systems amplify noise and yield unstable estimates. This has motivated filter selection and band allocation strategies that explicitly promote numerical stability. For example, Hardeberg et al. [ 3 ] proposed selecting filters by maximizing mutual orthogonality of response vectors, aiming to span the spectral space broadly, but without directly optimizing inversion stability. Li et al. [ 8 ] introduced a maximum linear independence (MLI) criterion that improves invertibility by minimizing the condition number (i.e., the ratio of the largest to the smallest singular value) of the sensitivity matrix. MLI is primarily used as a principled rule for choosing an informative subset of filters from a larger library, aiming to reduce ambiguity among channels and to stabilize subsequent spectral reconstruction. Efficient search procedures for exploring filter combinations under such criteria have also been proposed, such as Sippel’s fast binary search strategy [ 13 ] . In parallel, multi-camera MSI arrays built from off-the-shelf components have confirmed the practicality of extending spectral sensitivity via multiple filtered cameras [ 2 ] . A closely related concept was demonstrated by Themelis et al. [ 16 ] , who used standard color cameras combined with multiple-bandpass filters to obtain a calibrated linear mixing model allowing simultaneous recovery of several spectral band intensities via matrix inversion. I-B Proposed approach In contrast to task-specific or heuristic designs, we propose a systematic framework for multi-camera MSI design. We formulate a linear forward model for an arbitrary number of cameras, individual filter passbands, and target wavelengths, and derive analytic stability bounds linking numerical conditioning to worst-case noise amplification. Based on these insights, we enumerate feasible camera/filter configurations and select the wavelength allocation that minimizes the condition number of the global system matrix. This yields a fully deterministic and reproducible design procedure for allocating wavelengths to each camera’s filter such that spectral recovery is maximally noise-robust under the given hardware constraints. Whereas Li et al. [ 8 ] use the MLI criterion to select an informative filter subset primarily by minimizing the condition number of the filter transmittance matrix, we instead optimize the conditioning of the full camera–filter forward model for a prescribed set of target wavelengths and a general multi-camera wavelength-allocation setting. This shifts the objective from analyzing the filters in isolation to guaranteeing a well-posed, noise-robust inversion for the specific multi-camera configuration considered in this paper. Throughout the paper, we assume a predefined finite set of target wavelengths at which spectral information is to be recovered. For each camera, the combination of the sensor’s spectral sensitivities and a multi-band optical filter induces mixing coefficients determined by the overlap between the filter transmittances and the camera response functions. This yields a linear system that relates the desired spectral samples to the measured values. For instance, consider 6 target wavelengths and

Read the complete text