Addressing the Depth-of-Field Constraint: A New Paradigm for High Resolution Multi-Focus Image Fusion

Luca Piano, Peng Huanwen, Radu Ciprian Bilcu, Huawei Technologies Oy (Finland) Company Ltd, Politecnico di Torino {luca.piano}@polito.it{penghuanwen}@huawei.com · 2025

Read Addressing the Depth-of-Field Constraint: A New Paradigm for High Resolution Multi-Focus Image Fusion by Luca Piano, Peng Huanwen, Radu Ciprian Bilcu, Huawei Technologies Oy (Finland) Company Ltd, Politecnico di Torino {luca.piano}@polito.it{penghuanwen}@huawei.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19581.txt.

Introduction

Luca Piano, Peng Huanwen, and Radu Ciprian Bilcu present a new approach to multi-focus image fusion. The task is to combine photos taken at different focus settings into one image that is sharp throughout. Ordinary lenses keep only a slice of a scene in focus. Fusion methods also struggle with scarce training data and faked synthetic examples.

Their method, VAEEDOF, uses a distilled variational autoencoder for faithful, efficient reconstruction. It can fuse up to seven images at once. To fix the data problem they built MattingMFIF, a synthetic 4K dataset that imitates realistic depth-of-field effects from real photographs. The system produced seamless, artifact-free results at the level of the best existing methods.

Catalog record

CREATORLuca Piano, Peng Huanwen, Radu Ciprian Bilcu, Huawei Technologies Oy (Finland) Company Ltd, Politecnico di Torino {luca.piano}@polito.it{penghuanwen}@huawei.com
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19581

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Addressing the Depth-of-Field Constraint: A New Paradigm for High Resolution Multi-Focus Image Fusion Luca Piano, Peng Huanwen, Radu Ciprian Bilcu, Huawei Technologies Oy (Finland) Company Ltd, Politecnico di Torino {luca.piano}@polito.it{penghuanwen}@huawei.com Abstract Multi-focus image fusion (MFIF) addresses the depth-of-field (DOF) limitations of optical lenses, where only objects within a specific range appear sharp. Although traditional and deep learning methods have advanced the field, challenges persist, including limited training data, domain gaps from synthetic datasets, and difficulties with regions lacking information. We propose VAEEDOF, a novel MFIF method that uses a distilled variational autoencoder for high-fidelity, efficient image reconstruction. Our fusion module processes up to seven images simultaneously, enabling robust fusion across diverse focus points. To address data scarcity, we introduce MattingMFIF, a new syntetic 4K dataset, simulating realistic DOF effects from real photographs. Our method achieves state-of-the-art results, generating seamless artifact-free fused images and bridging the gap between synthetic and real-world scenarios, offering a significant step forward in addressing complex MFIF challenges. The code, and weights are available here: \codelink 1 Introduction Multi-Focus Image Fusion (MFIF) is a significant challenge in image processing. Optical lenses are inherently limited by the Depth-of-Field (DOF) constraint, where only objects within a certain distance appear sharp, while objects outside this range become blurred. As a result, it is difficult to capture a scene in which all objects at varying distances are simultaneously in focus in a single shot aslantas2010fusion . To address this issue, various algorithms have been proposed to fuse multiple images with different focus points, producing an all-in-focus image. These fused images can enhance visualization, enable more effective feature extraction, and improve object recognition in computer vision tasks. While traditional methods have provided valuable solutions, they often have inherent limitations that hinder their effectiveness in more complex scenarios. In recent years, methods based on Deep Learning (DL) have gained significant attention as they offer greater flexibility and potential for improved fusion results. Despite their promising performance, they still present several unresolved challenges. One key issue is the lack of sufficient and diverse training data. Current publicly available datasets are limited in size and resolution, and synthetic data generation techniques often introduce domain gaps with respect to real images. Moreover, these datasets tend to lack challenging cases, which are crucial for developing robust models. Furthermore, the nature of the models themselves contributes to the difficulty of the problem. Decision mask-based approaches, which focus on selecting the sharpest regions for fusion, face challenges in dealing with areas where information is missing, for instance, when foreground elements block the view of background objects. However, current end-to-end methods struggle to produce high-quality fused images due to the limited number of training examples, making them prone to generating artifacts and failing to generalize well to real-world scenarios. Figure 1: Overview of the architecture. VAE is pretrained and frozen during the training. In this paper, we propose VAEEDOF to overcome these challenges. Our method leverages a Variational Autoencoder (VAE) distilled from Stable Diffusion 3 to balance high-fidelity image reconstruction with efficient speed. We introduce a Fusion U-Net module capable of fusing up to seven different images simultaneously, allowing for more comprehensive fusion across multiple focus points. Furthermore, we present a new 4k MFIF training and validation dataset created using real photos, simulating DOF effects with Blender. This dataset provides a realistic large-scale foundation for training models that addresses the scarcity of high-quality data for MFIF . Our approach aims to improve the quality of fused images, bridge the domain gap, and handle complex scenarios effectively. 2 Related Work This section initially delves into conventional techniques for MFIF . Following that, the discussion shifts to approaches rooted in DL methods and, subsequently, an evaluation of the existing dataset is conducted. 2.1 Multi-Focus Image Fusion MFIF was initially introduced in the late 1970s, leading to the development of numerous methods aimed at addressing this particular task, which can be broadly categorized into traditional methods and DL -based approaches. The traditional methods for MFIF are further divided into two main categories: transform- and spatial domain-based techniques. Classical transform domain fusion methods rely on multiscale transforms, including the Laplacian Pyramid, and the Discrete Wavelet Transform, among others. These methods first convert images into a different domain, perform fusion on the transformed coefficients, and then reconstruct the fused image using the inverse transform ma2019multi ; ma2017multi ; qiu2019guided ; liu2015multi ; shreyamsha2015image ; tian2011multi ; bai2015quadtree ; liu2020multi . In contrast, spatial domain-based methods focus on processing image pixels to extract and fuse relevant information. Pixel-based methods involve generating decision maps based on sharpness or focus criteria, which are refined using various filtering and comparison techniques ma2019multi ; ma2017multi ; qiu2019guided ; liu2015multi ; shreyamsha2015image ; tian2011multi ; bai2015quadtree . Block-based methods decompose source images into blocks and use specific measures to detect focussed regions from which the fused image is constructed bai2015quadtree . Region-based methods first identify the boundaries between the focussed and defocussed regions, segment the image accordingly, and calculate focus measures to generate a decision map for fusion zhang2017boundary . Although traditional methods provide a solid foundation for MFIF , DL -based techniques are gaining increasing attention due to their ability to learn complex features and fusion strategies from data. These techniques adeptly capture intricate data patterns, allowing effective integration of multi-focus information. These techniques can be broadly classified into decision map-based and generative methods zhang2021deep . The former use decision map generation and potential post-processing for refinement. The fused image is derived by blending the first image ( I_{0} ) and the second image ( I_{1} ) according to the decision mask ( M ), using the formula: I_{0}M+I_{1}(1-M) Liu et al. liu2017multi introduced one of the first DL -based methods, which uses a Convolutional Neural Network (CNN) to directly learn the mapping from the source images to the focus map. Then, a large number of scholar have joint the research. SESF-Fuse ma2021sesf trains an encoder–decoder network in an unsupervised manner to extract deep features from input images. The spatial frequency is then used to measure the activity levels of these features, followed by consistency verification to refine the decision map and generate the fused result. Wang et al. wang2021mfif introduce MFIF-GAN to mitigate Defocus Spread Effect (DSE) by generating focus maps with accurately sized foreground regions. It incorporates reconstruction and gradient regularization terms in the loss function to enhance boundary details and improve fusion quality. GACN ma2022end uses a cascade network trained end-to-end, eliminating the reliance on empirical postprocessing. To further enhance the output, it employs a gradient-aware loss function to preserve fine gradient details in the fused image. MSI DTrans zhai2024msi is a fusion method that combine multilayer semantic interaction with dynamic transformers. Integrates Haar wavelets into the network feature extraction process to combine the strengths of traditional and DL methods, enabling more comprehensive feature representation. Although these models have yielded promising results, they have a significant drawback: they cannot handle regions with missing information in the input images. In contrast, generative methods bypass the decision map, processing source images directly to output the fused image. IFCNN zhang2020ifcnn draws inspiration from transform-domain fusion algorithms, using convolutional layers to extract salient features from input images, which are then fused

Read the complete text