OPERA: An Agent for Image Restoration with End-to-End Joint Planning–Execution Optimization

Feng Zhu Shuyang Xie Yihan Zeng Ming Liu Wangmeng Zuo, Harbin Institute of Technology · 2026

Read OPERA: An Agent for Image Restoration with End-to-End Joint Planning–Execution Optimization by Feng Zhu Shuyang Xie Yihan Zeng Ming Liu Wangmeng Zuo, Harbin Institute of Technology (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22104.txt.

Introduction

Feng Zhu and co-authors present OPERA, an agent for image restoration with joint planning-execution optimization. Real-world restoration is hard because mixed degradations interact, and agent-based methods compose multiple task-specific tools. Their performance is limited by constrained planning spaces and poor coordination among pretrained tools. OPERA optimizes planning and execution jointly, using restoration quality as the reward, and adds agent-guided co-training so tools learn cooperative behaviors in sequence. On multi-degradation benchmarks it beats all-in-one models and existing agent methods.

Catalog record

CREATORFeng Zhu Shuyang Xie Yihan Zeng Ming Liu Wangmeng Zuo, Harbin Institute of Technology
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66005
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22104

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OPERA: An Agent for Image Restoration with End-to-End Joint Planning–Execution Optimization Feng Zhu Shuyang Xie Yihan Zeng Ming Liu Wangmeng Zuo, Harbin Institute of Technology Abstract Real-world image restoration is challenging due to complex and interacting mixed degradations. Recent agent-based approaches address this problem by composing multiple task-specific restoration tools. However, empirical analysis reveals that their performance is fundamentally limited by implicitly constrained planning spaces and the lack of coordination among independently pretrained tools. To address these issues, we propose OPERA (Optimized Planning-Execution Restoration Agent), a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. On the planning side, OPERA uses reinforcement learning to directly optimize tool composition over a combinatorial plan space, with the final restoration quality as the reward. On the execution side, OPERA introduces agent-guided co-training of restoration tools, enabling them to learn cooperative behaviors under sequential composition. Extensive experiments on multi-degradation benchmarks and real-world datasets demonstrate that OPERA consistently outperforms both all-in-one restoration models and existing agent-based methods across diverse and complex degradation scenarios. Codes are available at: https://github.com/xsyshuishui/Opera . 1 Introduction Real-world images often suffer from complex degradations involving multiple distortion types simultaneously, where noise, blur, haze, rain, and compression artifacts can coexist and interact in non-trivial ways. Unlike the clean, single-degradation scenarios commonly studied in benchmarks, mixed degradations pose significant challenges for image restoration systems, as the combined effects of multiple distortions cannot be modeled simply. Early efforts address this problem by developing all-in-one restoration models that aim to handle multiple degradation types within a single network zamir2022restormer ; Promptir ; airnet ; daclip . While conceptually appealing, these models inevitably face a trade-off between generalization and specialization: accommodating diverse degradation patterns often results in overly smooth outputs and the loss of fine details, especially in complex real-world settings chen2024restoreagent . Recently, agent-based image restoration methods have emerged as a promising alternative agenticir ; chen2024restoreagent . Instead of relying on a single model, these approaches leverage a collection of off-the-shelf, task-specific restoration tools to restore the image. By leveraging large language models (LLMs) or vision-language models (VLMs), they design agentic systems to dynamically select and compose appropriate tools for a given degraded input. By exploiting the strengths of specialized models, agentic systems have the potential to achieve strong task-specific performance while handling arbitrary combinations of degradations. This paradigm has opened a new direction for handling complex real-world degradations beyond the scope of conventional single-model designs. Despite this, existing agent-based methods suffer from two fundamental limitations that hinder their effectiveness in complex multi-degradation scenarios. • (1) Implicitly constrained planning space. Existing agent-based image restoration methods rely on implicit planning assumptions that significantly restrict the space of restoration plans. For example, many methods assume a one-to-one mapping between degradation types and restoration tools, planning by explicitly matching each detected degradation to a specific tool agenticir ; chen2024restoreagent . Some methods plan in a stepwise greedy manner, where each action is selected to locally improve an intermediate quality metric lu2025simplecall ; zhou2025q . These assumptions substantially narrow the planning space, limiting the agent’s ability to discover more complex yet effective cooperative restoration strategies. • (2) Static pretrained tools without coordination. Existing agent-based frameworks treat restoration tools as fixed, independently pretrained modules agenticir ; chen2024restoreagent . When these tools are composed sequentially, the output distribution of one tool becomes the input to the next, but these tools were never trained to cooperate. Applying a tool may alter the distribution of remaining degradations, adversely affecting subsequent restoration steps. Prior work has attempted to alleviate this issue by searching for optimal tool orderings, but we argue that ordering alone is insufficient. When tools lack coordinated training, no single ordering can consistently yield satisfactory results across diverse degradation combinations. To concretely understand how these limitations affect performance, we conduct an empirical study on cooperative multi-tool image restoration in a constrained setting. By exhaustively enumerating restoration plans, we analyze which tool combinations yield high-quality restoration outcomes. As illustrated in fig. 1 , many high-performing plans violate common planning assumptions. For example, effective restoration often involves out-of-scope tools as well as repeated tool applications. This also demonstrates the limitations of existing restoration tools when applied to this cooperative setting. The findings suggest that both a more expressive planning space and improved coordination among tools are necessary for effective agent-based restoration. Motivated by these observations, we propose OPERA , a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. At the planning level, OPERA departs from a hand-crafted, step-by-step decision-making workflow and instead trains an agent to generate a complete tool-invocation plan end-to-end. Given the combinatorial nature of the tool composition space, we use reinforcement learning to optimize the agent, with the final restoration quality serving as the reward signal. This formulation enables the agent to reason globally over tool compositions and discover non-obvious combinations. At the execution side, OPERA proposes agent-guided tool model training. Rather than treating restoration tools as static, independently pretrained modules, we jointly fine-tune tool models under the agent’s generated plans. In this process, the agent acts as a high-level planner that induces diverse tool compositions, while individual tools remain architecturally independent and are updated solely based on their contribution to downstream restoration quality. This co-training strategy enables tools to learn cooperative behaviors, effectively mitigating the distribution shift caused by sequential composition. Extensive experiments demonstrate that OPERA significantly outperforms both all-in-one restoration models and existing agent-based systems across most metrics. We further conduct analysis showing that the agent learns non-trivial planning strategies, while jointly trained tools adapt to operate robustly within agent-generated restoration plans. Finally, OPERA generalizes well to real-world datasets, highlighting the robustness of our joint planning-and-execution framework. Our main contributions are summarized as follows: • We present an empirical study of cooperative multi-tool image restoration, providing key insights into the design of the agentic image restoration system. • We propose OPERA , an end-to-end agent-based restoration framework that jointly optimizes tool composition planning via reinforcement learning and enables cooperative behavior through agent-guided tool training. • Extensive experiments on multi-degradation benchmark and real-world dataset show that OPERA significantly outperforms existing all-in-one models and agent-based methods across most metrics. 2 Related Work 2.1 All-in-One Image Restoration All-in-One Image Restoration (AiOIR) aims to develop unified models that handle diverse degradation types within a single framework jiang2025survey . Early approaches adopt shared encoder-decoder architectures with multi-scale processing zamir2022restormer . Recent advances explore various conditioning mechanisms: prompt-based methods Promptir use learnable prompts for task adaptation, degradation embedding approaches airnet ; daclip explicitly encode degradation representations, and Mixture-of-Experts architectures route inputs to specialized sub-networks. While these methods achieve strong performance on standard benchmarks, they may struggle with complex real-world degradation mixtures lu2025simplecall and face trade-offs between generalization and task-specific accuracy chen2024restoreagent . Moreover, incorporating new degradation types typically requires retraining the entire model, limiting extensibility agenticir . 2.2 Agent-based Image Restoration Agent-based methods address AiOIR limitations by employing intelligent controllers that dynamically select and sequence restoration tools. We review existing approaches along two dimensions: Planning Strategies. Existing agent-based methods universally adopt step-by-step planning with iterative execution. RestoreAgent chen2024restoreagent first leverages multimodal LLMs for degradation identification, introducing iterative replanning with rollback mechanisms to correct suboptimal decisions. AgenticIR

Read the complete text