Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent

Yixin Gao, Xin Li, Xiaohan Pan, Runsen Feng, Bingchen Li, Yunpeng Qi, Yiting Lu, Zhengxue Cheng, Zhibo Chen, Jörn Ostermann · 2025

Read Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent by Yixin Gao, Xin Li, Xiaohan Pan, Runsen Feng, Bingchen Li, Yunpeng Qi, Yiting Lu, Zhengxue Cheng, Zhibo Chen, Jörn Ostermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15243.txt.

Introduction

Yixin Gao and nine co-authors make image compression conversational in twenty twenty-five. Standard codecs offer few coding modes, chosen manually by engineers. That makes compression unfriendly for ordinary users with specific needs. Their system, Comp-X, is presented as the first interactive learned compression paradigm driven by a language model agent.

Three innovations define it. A multi-functional framework unifies coding modes for human viewing, machine perception, variable rates, and spatial bit allocation. An interactive agent learns from expert feedback, taught through augmented in-context learning. The agent interprets a user's request and selects settings accordingly. Readers in image coding get a glimpse of codecs operated by conversation, where expertise is supplied by the agent rather than demanded of the user.

Catalog record

CREATORYixin Gao, Xin Li, Xiaohan Pan, Runsen Feng, Bingchen Li, Yunpeng Qi, Yiting Lu, Zhengxue Cheng, Zhibo Chen, Jörn Ostermann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15243

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent Yixin Gao, Xin Li, Xiaohan Pan, Runsen Feng, Bingchen Li, Yunpeng Qi, Yiting Lu, Zhengxue Cheng, Zhibo Chen, Jörn Ostermann Abstract We present Comp-X, the first intelligently interactive image compression paradigm empowered by the impressive reasoning capability of large language model (LLM) agent. Notably, commonly used image codecs usually suffer from limited coding modes and rely on manual mode selection by engineers, making them unfriendly for unprofessional users. To overcome this, we advance the evolution of image coding paradigm by introducing three key innovations: (i) multi-functional coding framework , which unifies different coding modes of various objective/requirements, including human-machine perception, variable coding, and spatial bit allocation, into one framework. (ii) interactive coding agent , where we propose an augmented in-context learning method with coding expert feedback to teach the LLM agent how to understand the coding request, mode selection, and the use of the coding tools. (iii) IIC-bench , the first dedicated benchmark comprising diverse user requests and the corresponding annotations from coding experts, which is systematically designed for intelligently interactive image compression evaluation. Extensive experimental results demonstrate that our proposed Comp-X can understand the coding requests efficiently and achieve impressive textual interaction capability. Meanwhile, it can maintain comparable compression performance even with a single coding framework, providing a promising avenue for artificial general intelligence (AGI) in image compression. 1 Introduction Figure 1 : Image compression paradigm comparison between (a) manually inputting coding parameters and (b) our proposed Comp-X, which supports user input via natural language instructions. Lossy image compression is a cornerstone technology in visual signal communication, significantly reducing bandwidth and storage costs. In recent years, learned image compression (LIC) methods have achieved comparable or even superior rate-distortion performance to traditional image codecs, such as HEVC [ 69 ] and VVC [ 11 ] , through carefully designed techniques in transform [ 16 , 94 , 47 , 44 , 82 ] , quantization [ 89 , 29 , 26 ] , and entropy modeling [ 7 , 57 , 56 , 28 , 31 ] . Besides, taking advantage of the capability of end-to-end optimization with different loss metrics, many researchers have expanded the scope of LIC to support various scenarios like region-of-interest (RoI) coding [ 68 , 37 ] , human perception [ 54 , 59 , 87 , 13 ] , and machine perception [ 34 , 70 , 24 , 25 ] . Despite these achievements, LIC methods typically lack a unified and user-friendly usage mechanism, limiting their accessibility for common users. Most LIC approaches remain task-specific, optimized for distinct objectives rather than offering a flexible and generalizable framework. This limitation presents two major challenges: (i) the lack of a unified framework leads to inefficiencies in real-world deployment, and (ii) common users struggle to manually select the optimal compression parameters to meet their specific requirements. Recently, driven by the powerful reasoning capabilities of large language models (LLMs) in addressing various natural language processing tasks [ 12 , 61 , 76 , 5 , 18 , 74 ] , LLM-based agents have emerged as promising solutions to complex problems [ 30 , 80 , 52 , 67 , 88 , 50 ] . Typically, these agents employ LLMs as the core component of brain or controller and expand their perceptual and action space through strategies such as multimodal perception and tool utilization [ 60 , 90 , 66 , 51 , 78 ] . For instance, HuggingGPT [ 67 ] aims to automatically solve different user requests by prompting ChatGPT to act as a controller, conduct task planning, select suitable models in Hugging Face, and summarize responses based on execution results. These developments lead us to the following question: can we fundamentally evolve LIC into an interactive, user-friendly, and highly adaptable compression paradigm by leveraging the advanced reasoning and interaction capabilities of LLM agents? In this work, we present the first intelligently interactive image compression paradigm powered by LLM agent, Comp-X, aiming to enable common users to achieve customized image compression by simply inputting natural language instructions. Unlike most methods where users must manually configure image codec parameters for each scenario (Fig. 1 (a)), our proposed Comp-X (Fig. 1 (b)) employs an LLM agent to interpret user instructions, automatically selecting and adjusting coding parameters and tools to fulfill diverse user requirements. To achieve this, we first build a multi-functional image codec capable of flexibly scheduling various compression patterns, including human and machine perception, variable-rate coding, and spatial bit allocation, within a single unified framework. Specifically, we first build an image codec supporting variable bitrate (VBR) and spatial bit allocation, built upon a semantically structured bitstream [ 70 , 25 ] , achieving state-of-the-art rate-distortion performance. Next, to accommodate diverse user scenarios, we insert task-specific adapters [ 14 , 15 ] into our pre-trained MSE-optimized codec, fine-tuning it for perceptual quality and machine-vision tasks such as classification and segmentation. Based on this unified codec, we integrate an LLM agent to equip our image compression system with interactive, understanding, and reasoning capabilities. However, we observe that the pre-trained LLM [ 1 ] lacks specific knowledge about image compression techniques, leading to ambiguity when interpreting user requests. To address this challenge, we propose an augmented in-context learning method with coding expert feedback (ICL-EF). Rather than merely providing in-context examples [ 23 ] , our approach involves an iterative process where a coding expert reviews the output of the agent reasoning, identifies errors, and guides the agent through corrections until accurate conclusions are achieved. This mechanism enables the LLM agent to effectively internalize coding-related knowledge, significantly enhancing its ability to interpret user requests accurately, select optimal coding modes, and efficiently utilize available compression tools. Furthermore, to systematically evaluate the user request analysis capability of our Comp-X, we propose the first benchmark designed for I terative and I ntelligent image C ompression, called IIC-Bench. This benchmark contains a diverse collection of human requests, covering scenarios from simple to highly complex compression requirements. Extensive experiments demonstrate that Comp-X could accurately understand and interpret user requests, achieving an instruction parsing success rate exceeding 80%, while maintaining competitive compression performance across diverse application scenarios. The contributions of this paper can be summarized as follows: • To the best of our knowledge, we present the first intelligently interactive learned image compression paradigm. By leveraging LLM agent, our approach enables an automated and customizable image coding process tailored to user inputs in natural language. • We build a multi-functional image codec that integrates multiple coding modes into a single unified framework, including human-machine perception, variable coding, and spatial bit allocation, eliminating the need for multiple task-specific codecs. • We develop an augmented in-context learning approach with coding expert feedback to enhance the understanding of image compression knowledge priors of LLM. This mechanism significantly improves the ability of LLM agent to understand user requests, select appropriate coding modes, and effectively utilize coding tools. • We introduce IIC-Bench, the first dedicated benchmark designed to evaluate intelligently interactive image compression, featuring diverse real-world user requests with expert annotations, facilitating comprehensive performance assessment. • Extensive experiments demonstrate that Comp-X is capable of accurately interpreting user requests, achieving

Read the complete text