One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion

Chunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, ZhangyongTang, Hui Li, Zeyang Zhang, Sara Atito, Muhammad Awais, Josef Kittler · 2025

Read One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion by Chunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, ZhangyongTang, Hui Li, Zeyang Zhang, Sara Atito, Muhammad Awais, Josef Kittler (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19854.txt.

Introduction

Cheng and a large team build GIFNet in twenty twenty-five, one model that fuses images across many tasks. Fusion merges views, like heat and light, into one richer picture. Past methods chased high level missions and tangled on meaning gaps. Their turn is to fuse at the low level of pixels, guided by photo tasks.

Pixel level teaching gives strong, shared feature learning without abstract labels. The single model scores high on seen and unseen fusion jobs. It even sharpens single source images on its own. The authors pitch it as a flex tool for real imaging work. Code is promised online.

Catalog record

CREATORChunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, ZhangyongTang, Hui Li, Zeyang Zhang, Sara Atito, Muhammad Awais, Josef Kittler
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19854

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion Chunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, ZhangyongTang, Hui Li, Zeyang Zhang, Sara Atito, Muhammad Awais, Josef Kittler Abstract Advanced image fusion methods mostly prioritise high-level missions, where task interaction struggles with semantic gaps, requiring complex bridging mechanisms. In contrast, we propose to leverage low-level vision tasks from digital photography fusion, allowing for effective feature interaction through pixel-level supervision. This new paradigm provides strong guidance for unsupervised multimodal fusion without relying on abstract semantics, enhancing task-shared feature learning for broader applicability. Owning to the hybrid image features and enhanced universal representations, the proposed GIFNet supports diverse fusion tasks, achieving high performance across both seen and unseen scenarios with a single model. Uniquely, experimental results reveal that our framework also supports single-modality enhancement, offering superior flexibility for practical applications. Our code will be available at https://github.com/AWCXV/GIFNet . 1 Introduction Image fusion combines critical information from multiple sources to produce an output that is more informative and contextually rich, enhancing both human visual interpretation and the performance of downstream computer vision tasks Liu et al. 2018 ; Tang et al. 2024 . This technique has been shown to be valuable in remote sensing Javan et al. 2021 , medical imaging Azam et al. 2022 , and related fields Tang et al. 2024 ; Xu et al. 2020b . Typically, image fusion is divided into multi-modal fusion and digital photography fusion, based on the properties of the source images Karim et al. 2023 ; Li et al. 2025 . Multi-modal fusion combines complementary information from different sensors, such as Infrared and Visible Image Fusion (IVIF), where the infrared channel highlights targets against backgrounds and visible images convey texture details. Due to the lack of Ground Truth in such tasks, unsupervised approaches are commonly employed Zhang and Demiris 2023 ; Tang et al. 2023 . In contrast, digital photography fusion, such as Multi-Focus Image Fusion (MFIF) and Multi-Exposure Image Fusion (MEIF), addresses degradations caused by the limitations of the depth of field and inappropriate exposure within a single image Zhang 2021 . For these tasks, Ground Truth data can be generated artificially, for instance, by synthetically blurring regions or adjusting exposure, making supervised learning feasible. Figure 1 : A comparison of the versatility and efficiency of advanced multi-task fusion methods. The indices in the arrows are the fusion tasks validated by the corresponding methods. Currently, a task-interaction mechanism is widely used in advanced image fusion methods. A typical approach adopted by advanced multi-modal image fusion research is to draw on high-level visual tasks for supervisory signals to direct the fusion. As a downstream task, these high-level models are usually equipped with a fusion model and receive the fused image as input for iteratively optimising the fusion model and high-level models. High-level tasks, such as object detection or semantic segmentation Liu et al. 2022 ; Tang et al. 2022 ; Zhang et al. 2022 , introduce abstract semantic information that can be used to guide task-specific feature learning and improve the fusion performance. Meanwhile, the improved scene representation derived from the fused image also helps to realise the high-level task better by virtue of the mutual reinforcement mechanism established in this way. However, such high-level supervision is somewhat divorced from the underlying image fusion problem. As shown in Fig. 1 , when addressing a different image fusion task, this indirect formulation requires training a new fusion model that is tailored to specific features of each task. This additional requirement can hinder the deployment of image fusion algorithms on small footprint devices, e.g. , mobile phone, which have limited computational resources, as every application requires a different model to solve it. Figure 2 : Comparison of advanced multi-task fusion methods relying on high-level tasks and the proposed low-level task interaction paradigm. These semantic-focused paradigms cannot consistently ensure the robust fusion quality as our paradigm does, which provides the pixel-level supervision and presents clear texture details. Moreover, the semantic gap between a high level downstream task and the low level image fusion renders the high level supervision signal less than optimal for guiding pixel-focused fusion learning, as it tends to encode features related to object categories, shapes, and scene layouts, rather than fine-grained image details. This mismatch results in unwanted reliance on complex bridging modules Xu et al. 2023 or computationally intensive pre-trained models Cheng et al. 2024b for different fusion contexts (Fig. 1 (a) and (b)), both of which are resource-heavy and fail to generalise effectively across various fusion contexts. Also, as shown in Fig. 2 , paradigms (a) Liu et al. 2022 and (b) Yi et al. 2024 achieve increased saliency for detected objects or text-prompted regions, yet the fused images are often unable to consistently maintain high visual quality, which is significant for other fusion tasks. We argue that, this phenomenon can be attributed to the absence of pixel-level supervision. In this work, we aim to address these limitations by promoting cross-task interaction without relying on high-level semantics. Instead, we use low-level digital photography fusion tasks as a more natural alternative for providing supervisory signals. Digital photography fusion shares its characteristics with multi-modal fusion, emphasising the preservation of details and focusing on pixel-level feature alignment, rendering it better equipped for enhancing task-shared image features without the semantic mismatch inherent to interacting with high-level tasks. To this end, we introduce the Generalised Image Fusion Network (GIFNet), a three-branch architecture that supports low-level task interaction for effective fusion. GIFNet comprises a main task branch, an auxiliary task branch, and a reconciling branch. The main and auxiliary task branches, which alternately focus on the multi-modal and digital photography features, promote effective cross-task interaction. While the reconciling branch, centred on a shared reconstruction task, encourages the network to learn a universal feature representation Zhao et al. 2024a . This branch harmonises the optimisation directions of the multi-modal and digital photography branches, preventing divergent task-specific adaptations. Additionally, our model incorporates a cross-fusion gating mechanism that iteratively refines each task-specific branch, integrating multi-modal and digital photography features to deliver the fusion result. To minimise the data domain gap between multi-modal and digital photography tasks, we create an RGB-based joint dataset based on the augmentation technique. With the shared RGB modality derived from identical scenes, the proposed model can focus on consistent feature extraction across the adopted tasks in a unified context, thereby harmonising the training process. Finally, as shown in Fig. 1 (c), the limited computational cost of low-level fusion tasks in GIFNet reduces GFLOPs by more than 96% compared to the advanced image fusion method. Unlike current approaches that focus heavily on high-level vision tasks, prioritising a single category fusion, GIFNet’s integration of both multi-modal and digital photography tasks broadens its applicability across various fusion scenarios with a single model (task-agnostic image fusion). Besides, rather than amplifying task-specific features, our low-level task interaction enhances task-shared foundational features that are crucial for general image processing, allowing GIFNet to function as a versatile enhancer even for single-modality inputs. The main contributions of the proposed method include the following: • We uniquely demonstrate that collaborative training between low-level fusion tasks, a strategy whose importance was previously not

Read the complete text