EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning

Kailin Zhuang, Jiawei Wu, Zhi Jin · 2026

Read EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning by Kailin Zhuang, Jiawei Wu, Zhi Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22208.txt.

Introduction

Kailin Zhuang and co-authors present EvoIR-Agent, a self-evolving image restoration agent driven by experience. Multimodal agents pick tools and removal orders flexibly, yet zero-shot planning fails without experience. Training-based methods embed experience but cannot handle new tools, while training-free methods store it explicitly yet still waste trials. EvoIR-Agent formulates experience components, builds a hierarchical pool for coarse-to-fine guidance. It leads on full-reference metrics with a Pareto-optimal balance, running 61 percent more efficiently than state-of-the-art methods.

Catalog record

CREATORKailin Zhuang, Jiawei Wu, Zhi Jin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65300
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning Kailin Zhuang, Jiawei Wu, Zhi Jin Abstract Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios by flexibly selecting tools and determining removal orders. However, their zero-shot planning often fails without experience, necessitating severe trial-and-error overhead to achieve satisfactory outcomes. Currently, two paradigms are employed to address this issue, yet a dilemma persists: Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation. In contrast, training-free methods utilize explicit experience storage for compatibility but still incur trial-and-error overhead due to naive experience. To resolve the dilemma, we propose EvoIR-Agent , which first systematically formulates the experience components of a training-free image restoration agent. Subsequently, a hierarchical experience pool is constructed, which enables coarse-to-fine guidance for diverse tools and removal orders. Furthermore, a self-evolving mechanism is introduced to update the pool from scratch using accumulated records, thereby greatly improving performance and efficiency. Extensive experiments reveal that EvoIR-Agent achieves a significant lead in the full reference metrics and yields a remarkable Pareto-optimal balance between performance and efficiency compared to the state-of-the-art methods. Our codes are available at https://github.com/rightleft-123/EvoIRAgent 1 Introduction Image restoration (IR) is a low-level vision task that aims to recover a high-quality (HQ) image from a degraded low-quality (LQ) version. In recent decades, considerable progress has been made in addressing IR problems related to specific tasks [ 76 , 57 , 49 , 22 , 54 ] . However, in real-world scenarios, multiple degradations are often coupled, making restoration more challenging. A promising direction for addressing this challenge is all-in-one image restoration (AiOIR), which aims to handle multiple degradations with a unified model [ 19 ] . The majority of methods [ 29 , 52 , 10 , 21 , 32 , 61 ] aim to address the degradation coupling problem by learning more general degradation feature representations. While significant strides have been made, they are still limited to a few types of degradation and are difficult to extend to new degradations without retraining. Recent advances in Multimodal Large Language Models (MLLMs) [ 56 , 75 ] have inspired a new direction in IR, namely Image Restoration Agent (IRA). The key is to leverage a MLLM-driven agent to select restoration tools and determine removal orders. However, general-purpose MLLMs encounter difficulties with zero-shot planning [ 4 , 83 ] without experience-driven, which results in severe trial-and-error overhead to reach satisfied restoration result. To resolve this issue, existing methods can be divided into two distinct paradigms. Training-based methods [ 4 , 35 , 79 ] internalize intrinsic experience into model parameters, offering high inference efficiency but lacking compatibility for new tools or degradation. Conversely, training-free methods [ 83 , 20 , 84 ] utilize external containers for explicit experience storage, while highly compatible, they still result in considerable trial-and-error overhead owing to being guided by naive experience. Therefore, we aim to retain the compatibility of training-free methods while endowing the model with the inference efficiency of training-based methods. We identify that the considerable trial-and-error overhead in training-free IRAs stems from the underdeveloped experience mechanism. To systematically break this bottleneck, we revisit from two perspectives: (i) Do existing IRAs lack an experience mechanism altogether? Not entirely. Recent frameworks [ 83 , 20 , 84 ] already incorporated experience mechanism. However, their experience components remain simplistic and static, preventing the agent from accepting comprehensive and dynamic guidance. (ii) Is it possible to transfer well-established experience mechanisms from general-purpose agents to IRAs? It is possible. Recent advancements [ 81 , 77 , 66 , 14 ] have demonstrated that explicit experience drives self-evolving skills in general-purpose agents. However, it is unrealistic to directly apply to IRAs without a domain-specific redesign. Thereby, we introduce EvoIR-Agent , an image restoration agentic system explicitly designed to address the domain-specific needs. We first clearly identify the experience components in IRA as illustrated in Figure 1 . (1) Tool Selection : The optimal tool depends not merely on broad degradation type, but on specific degradation pattern [ 4 ] . Experience is designed to explicitly map these patterns to their optimal tool; (2) Degradation Removal Order : Similarly, in degradation coupling settings, the optimal order is also pattern-dependent; (3) Visual Quality : Due to inherent attributes in different tools, multiple paths may address the degradation, but result in significantly different visual outcomes. Overall, current experience mechanism focuses on removal order, with a few addresses on tool selection and showing an absence in focusing on visual quality. Moreover, they concentrate on the broad degradation type, while neglecting to delve into the degradation pattern at a finer granularity. Figure 1 : Illustration of the experience components in IRA. (Left&Center) We extract embeddings of various degraded images using an image encoder [ 17 ] . The scatter plots reveal that images sharing a degradation type may cluster into different degradation patterns. Crucially, the optimal Tool Selection and Degradation Removal Order are highly pattern-dependent. (Right) Visual Quality demonstrates that divergent tool paths can successfully achieve restoration but yield distinct visual quality. To systematically capture this coarse-to-fine information in conjunction with the aforementioned three experience components, we feature a hierarchical experience pool for storage. Inspired by [ 74 , 70 , 41 ] , this pool retrieves contextually relevant experience across multiple levels, providing the IRA with robust, prioritized experience guidance. Building upon this structural foundation, the pool is equipped with a self-evolving experience mechanism. Unlike prior static experiences, ours iteratively updates its experience pool using newly accumulated records. This dynamic evolution enables the system to adapt to tools and degradation on the data source, thereby promoting performance and inference efficiency. Our main contributions are summarized as follows: (1) Systematic Formulation. We reformulate the experience components in IRAs by introducing visual quality as an additional component. We also identify a limitation in current IRA experience: a singular focus on broad degradation types, with an absence of finer degradation patterns. (2) Self-Evolving Architecture. To effectively manage the formulated experience, we propose a hierarchical experience pool equipped with tailored retrieval methods. Building upon this structure, we introduce a self-evolving experience mechanism that updates the pool using accumulated records. (3) Empirical Evaluation. Extensive experiments demonstrate that EvoIR-Agent achieves a superior performance-efficiency trade-off compared to state-of-the-art methods. Furthermore, the results validate the profound impact of the experience mechanism and its capacity for self-evolving. 2 Related Work All-in-one Image Restoration. All-in-one image restoration aims to handle multiple degradations within a unified model [ 19 ] . AirNet [ 29 ] , and MiOIR [ 25 ] focus on designing a unified backbone to model diverse degradation distributions. PromptIR [ 40 ] uses learnable prompts to modulate the network for different degradations. DA-CLIP [ 37 ] exploits degradation-aware vision-language priors, and InstructIR [ 10 ] extends this idea by using human instructions for more flexible restoration control. Some methods leverage generative priors to enhance restoration quality. AutoDIR [ 21 ] , UniRestore [ 5 ] , and Defusion [ 36 ] formulate restoration in a diffusion framework. Later methods explore richer conditioning or more general restoration representation [ 61 , 48 ] . However, these works are constrained

Read the complete text