Learning Joint Denoising, Demosaicing, and Compression from the Raw Natural Image Noise Dataset

Benoit Brummer and Christophe De Vleeschouwer · 2025

Read Learning Joint Denoising, Demosaicing, and Compression from the Raw Natural Image Noise Dataset by Benoit Brummer and Christophe De Vleeschouwer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08924.txt.

Introduction

Benoit Brummer and Christophe De Vleeschouwer release a dataset of raw camera images and methods that clean, reconstruct, and compress them in one go, in a paper from twenty twenty-five. Camera pipelines usually polish finished pictures, discarding raw sensor data early. Raw files keep more information but need special handling.

The dataset pairs raw shots designed to help denoisers work across sensors, shooting styles, and development paths. One method cleans the raw mosaic directly for speed. Another works on linear color data for wider sensor reach. Both beat approaches that start from finished images. Merging denoising with compression at the raw stage also improved the size-versus-quality trade and cut computing cost. The authors argue for raw-first workflows.

Catalog record

CREATORBenoit Brummer and Christophe De Vleeschouwer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-08924

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Joint Denoising, Demosaicing, and Compression from the Raw Natural Image Noise Dataset Benoit Brummer and Christophe De Vleeschouwer Abstract This paper introduces the Raw Natural Image Noise Dataset (RawNIND) , a diverse collection of paired raw images designed to support the development of denoising models that generalize across sensors, image development workflows, and styles. Two denoising methods are proposed: one operates directly on raw Bayer data, leveraging computational efficiency, while the other processes linear RGB images for improved generalization to different sensors, with both preserving flexibility for subsequent development. Both methods outperform traditional approaches which rely on developed images. Additionally, the integration of denoising and compression at the raw data level significantly enhances rate-distortion performance and computational efficiency. These findings suggest a paradigm shift toward raw data workflows for efficient and flexible image processing. Fig. 1: Preview of the RawNIND ’s Bayer (top) and X-Trans (bottom) ground-truth images I Introduction Image denoising and compression are fundamental tasks in digital photography and image development, but current methods applied to developed images suffer from limitations in performance and generalization. These limitations arise because traditional compression methods and most of the literature in image denoising focus on developed images, which have undergone complex and often proprietary transformations applied by a camera image signal processor (ISP) or an image development software. This work presents a methodology that leverages raw sensor data and the Raw Natural Image Noise Dataset (RawNIND) , a contribution of this work, to achieve significant improvements in computational efficiency (four to sixteen times fewer operations), produce high-quality images from noisy raw data, and effectively generalize across diverse camera sensors and development workflows. In contrast, raw sensor data offers a more consistent and flexible representation of the captured scene. Unlike developed images, raw data preserves the linear sensor output and avoids artifacts introduced during image development, making it more suitable for denoising and compression tasks. However, raw image manipulation introduces its own challenges, as each sensor has unique characteristics, such as noise patterns and native color spaces. These variations have historically led to the assumption that models trained on one camera’s raw data would not generalize to others. To address these challenges, we introduce the Raw Natural Image Noise Dataset (RawNIND) , a collection of paired raw images from a variety of camera sensors. RawNIND enables the development of models that generalize across sensors, workflows, and noise conditions. Using this dataset, we explore two complementary denoising methods: • Raw Bayer Image Denoising: This approach offers significant computational efficiency (reducing processing by a factor of four) by denoising raw Bayer data directly, operating on one-quarter of the spatial dimensions and deferring demosaicing until the final step. • Linear RGB Denoising: This method operates on debayered images in a standardized linear RGB color space, maintaining high image quality and facilitating better generalization across different color filter arrays (CFAs) while maintaining the advantages of raw data. Both of the proposed denoising methods outperform approaches that take developed images as input, and they integrate seamlessly into any image development workflow, providing adaptability for any real-world application. Building on these denoising advancements, we then integrate denoising and compression into a unified framework that operates directly on raw sensor data. This joint approach significantly enhances both computational efficiency at the encoder stage and rate-distortion performance by addressing noise at the earliest stage of the image pipeline. Unlike conventional workflows which compress fully developed images, our method compresses raw data, preserving flexibility for subsequent editing. It also enables new compression paradigms where raw images are stored alongside lightweight XMP sidecar files, which contain all the necessary development instructions to produce the final image. This paper makes the following key contributions: • RawNIND Dataset: A novel dataset of paired raw images enabling generalized denoising and compression across diverse sensors and noise conditions. • Denoising Methods: Two high-performing approaches for raw image denoising—considering Bayer denoising for computational efficiency and linear RGB representations for different CFAs s—that generalize across all image development workflows. • Integrated Denoising and Compression: A joint model that significantly improves rate-distortion performance and computational efficiency compared to traditional sequential pipelines. • Compression Paradigm: We propose a paradigm for storing compressed raw images with metadata ( XMP sidecar files containing development instructions), enabling non-destructive editing and efficient storage by including instructions to convert the raw data into a developed image. The rest of the paper is organized as follows: Section II reviews related work on image denoising and compression. Section III describes our proposed methods and the creation of RawNIND. Section IV details the experimental setup and evaluation metrics, while Section V presents the results and discusses key findings. Finally, Section VI concludes the paper and outlines directions for future research. II Related Work II-A Image Denoising Image denoising aims to suppress noise while preserving important image details. Traditional methods, such as non-local means [ 1 ] (NLM) , wavelet-based denoising [ 2 ] , and BM3D [ 3 ] , rely on mathematical models to exploit redundancies in image structure. Among these, BM3D remains a widely used benchmark due to its high-quality results under controlled conditions, despite its computational intensity, while NLM and wavelet-based methods are implemented in raw image development software such as Darktable, due to their simplicity and efficiency. The emergence of deep learning techniques has shifted the focus to data-driven approaches. Convolutional neural networks [ 4 ] (CNNs) , including architectures like U-Net [ 5 ] and methods such as DnCNN [ 6 ] , and CBDNet [ 7 ] , have shown significant improvements over traditional methods. However, these models often rely on synthetic noise for training and are sensitive to mismatches between training and testing noise distributions. While datasets like SID [ 8 ] , SIDD [ 9 ] , and the Natural Image Noise Dataset [ 10 ] (NIND) provide real noisy-clean image pairs for training, their focus on specific sensors and development workflows limits their applicability to diverse real-world scenarios. Critically, most existing work evaluates denoising on developed images, which does not reflect real-world use cases. In practice, denoising is almost always performed by the camera’s ISP or raw development software, either directly on raw data or before applying non-linear transformations like sharpening. Post-development denoising is rare and generally suboptimal, as the introduction of non-linearities limits both denoising performance and its integration into the imaging pipeline. These limitations underscore the importance of addressing noise earlier in the processing workflow, such as directly on raw data. Methods combining denoising and demosaicing [ 11 , 12 , 13 ] have demonstrated improved noise suppression by leveraging raw data’s linear characteristics with respect to illumination. However, these studies often assume a single camera model, leaving cross-sensor generalization largely unexplored. This work addresses this gap by explicitly evaluating generalization across multiple sensors and workflows using raw image data. II-B Raw Image Development Pipelines Raw image development converts sensor data into visually interpretable images through steps such as demosaicing, color space transformations, and tone mapping. The variability in raw development pipelines arises from the unique characteristics of camera sensors and user-defined adjustments during development. In-camera ISPs apply proprietary pipelines optimized for speed and versatility, while raw development software like Darktable (illustrated in Figure 2 )

Read the complete text