PICD: Versatile Perceptual Image Compression with Diffusion Rendering
Tongda Xu, Jiahao Li, Bin Li, Yan Wang, Ya-Qin Zhang, Yan Lu · 2025
Read PICD: Versatile Perceptual Image Compression with Diffusion Rendering by Tongda Xu, Jiahao Li, Bin Li, Yan Wang, Ya-Qin Zhang, Yan Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-05853.txt.
Introduction
Tongda Xu and five co-authors shrink screen images with a diffusion renderer in a 2025 paper. Looks first compression has grown sharp for photos at tiny file sizes, but text on screens smears under the same tools. Their codec, PICD, splits a screen shot into its text and its picture parts, squeezes each, then paints them back into one image with a diffusion model.
Guidance enters at three heights: the base model is tuned on screen content, an adaptor steers it with the squeezed parts, and per image hints polish the decode. Tests beat past looks first codecs on both text accuracy and visual quality. Stripped of text hints, the same codec serves photos well. The authors pitch one tool for both worlds.
Catalog record
| CREATOR | Tongda Xu, Jiahao Li, Bin Li, Yan Wang, Ya-Qin Zhang, Yan Lu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-05853
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PICD: Versatile Perceptual Image Compression with Diffusion Rendering Tongda Xu, Jiahao Li, Bin Li, Yan Wang, Ya-Qin Zhang, Yan Lu Abstract Recently, perceptual image compression has achieved significant advancements, delivering high visual quality at low bitrates for natural images. However, for screen content, existing methods often produce noticeable artifacts when compressing text. To tackle this challenge, we propose versatile perceptual screen image compression with diffusion rendering ( PICD ), a codec that works well for both screen and natural images. More specifically, we propose a compression framework that encodes the text and image separately, and renders them into one image using diffusion model. For this diffusion rendering, we integrate conditional information into diffusion models at three distinct levels: 1). Domain level: We fine-tune the base diffusion model using text content prompts with screen content. 2). Adaptor level: We develop an efficient adaptor to control the diffusion model using compressed image and text as input. 3). Instance level: We apply instance-wise guidance to further enhance the decoding process. Empirically, our PICD surpasses existing perceptual codecs in terms of both text accuracy and perceptual quality. Additionally, without text conditions, our approach serves effectively as a perceptual codec for natural images. † † ∗ The work was done when Tongda Xu was a full-time intern with Microsoft Research Asia. Figure 1 : For both screen and natural images, PICD demonstrates high text accuracy and superior visual quality simultaneously. Figure 2 : PICD works well for both screen and natural images. 1 Introduction With recent advancements in generative models, perceptual image compression has acquired the capability to compress natural images at low bitrate while maintaining high visual quality [ 35 , 11 ] . These approaches are driven by the rate-distortion-perception trade-off [ 7 , 8 ] and typically involve training a conditional generative model as a decoder, such as a generative adversarial network (GAN) [ 40 , 44 , 33 , 3 , 35 , 19 ] or a diffusion model [ 52 , 17 , 11 , 27 , 31 , 39 ] . Although these perceptual codecs are effective for natural images, they generally fall short for screen images. This limitation arises because current perceptual codecs focus on preserving marginal distributions rather than accurately reproducing text. For example, a screenshot containing the character ”a” might be decoded as the character ”c.” As long as the character ”c” is clear and visually coherent, the codec is deemed to be perceptually lossless. However, such reconstruction is obviously unacceptable for screen content. Conversely, existing screen content codecs prioritize text accuracy but disregard perceptual quality. [ 34 , 43 , 36 , 15 , 57 , 25 ] . They enhance text quality based on non-perceptual natural codec, and produce blurry reconstructions at low bitrates. To address the aforementioned challenges, we propose versatile perceptual screen content codec with diffusion rendering (PICD) for both screen and natural images. Specifically, we encode the text information losslessly and render them with a compressed image using a diffusion model to achieve high text accuracy and visual quality. To implement this diffusion rendering, we introduce a three-tiered conditioning approach: 1) Domain level: We fine-tune the base diffusion model using text content prompts. 2) Adaptor level: We develop an efficient adaptor that is conditioned on both text content and its corresponding location, in conjunction with compressed image. 3) Instance level: We employ instance-wise guidance during decoding to further enhance performance. Empirical results demonstrate that our proposed PICD surpasses previous perceptual codecs in terms of both text accuracy and perceptual quality. Furthermore, our PICD remains effective for natural images. • We propose PICD, a versatile perceptual codec for both natural and screen content. • We introduce a highly efficient conditional framework to transform a pre-trained diffusion model into a codec, with domain, adaptor, and instance level conditioning. • We demonstrate that PICD achieves both high visual quality and text accuracy for screen and natural images. 2 Preliminary: Perceptual Image Codec Perceptual image compression mains to maintain the visual quality of the decoded image. Specifically, denoted as X , the encoder as f_{\theta}(\cdot) , the bitstream as Y=f_{\theta}(X) , the decoder as g_{\theta}(\cdot) , and the reconstructed image as \hat{X}=g_{\theta}(Y) , driven by rate-distortion-perception trade-off [ 2 , 33 , 35 ] , perceptual image compression constrains the marginal distribution of \hat{X} to match that of the source X : \displaystyle p(\hat{X})=p(X). (1) The majority of perceptual image codecs use a conditional generative model as the decoder [ 2 , 33 , 35 , 17 , 11 ] . Specifically, these models train the decoder g_{\theta}(\cdot) to approximate the true posterior distribution p(X|Y) : \displaystyle\hat{X}=g_{\theta}(Y)\sim p(X|Y). (2) In this scenario, the marginal distribution p(\hat{X}) aligns with p(X) , thereby achieving the optimum perceptual quality. 3 PICD: Versatile Perceptual Image Compression with Diffusion Rendering Figure 3 : The overall framework of our proposed PICD. 3.1 Overall Framework and Rationale We first describe the PICD for screen images and then simplify PICD for natural images. The foundational framework of PICD for screen images is as follows: • Encoding : We first extract the text Z=h(X) using Optical Character Recognition (OCR) model h(.) and compress it losslessly. Then, we encode the image X with Z as a condition, into the bitstream Y=f_{\theta}(X|Z) . • Decoding : We first decode text Z then the image \bar{X}=g_{\theta}(Y|Z) . Next, we merge the \bar{X} with Z into one image using conditional diffusion model \hat{X}\sim p_{\theta}(X|\bar{X},Z) . We name this merging process Diffusion Rendering. In the rest of this section, we provide an explanation of the optimality of PICD. Specifically, we show that: 1) PICD is optimal for preserving text information Z ; 2) PICD achieves best perceptual quality. PICD is Optimal for Preserving Text Information Let h(.) denote an OCR algorithm extracting text content and its corresponding location, such that Z=h(X) represents the OCR output. We define the reconstructed text as accurate when the OCR output for the reconstructed image, \hat{Z}=h(\hat{X}) , corresponds exactly to Z : \displaystyle\hat{Z}=h(\hat{X})=h(g_{\theta}(Y))=Z. (3) At first glance, one might propose attaining this objective by integrating an additional loss function, such as ||h(\hat{X})-Z||^{2} , into the training process [ 25 ] . However, this approach proves to be ineffective at low bitrates (See Table 3 ). In this study, we introduce a more straightforward approach to tackle this challenge. We begin by encoding Z losslessly, followed by encoding X with Z as a condition. Subsequently, we render Z together with the compressed \bar{X} to produce a single decoded image, \hat{X} . The rendering is facilitated by a conditional diffusion model, which employs both the compressed image \bar{X} and the text Z , i.e., \displaystyle\bar{X}=g_{\theta}(Y|Z),\hat{X}\sim p(X|\bar{X},Z). (4) As the proposed approach works well for both screen and natural images using diffusion rendering, we name it versatile perceptual image compression using diffusion rendering (PICD). To comprehend why PICD achieves optimal bitrate for preserving text information, we observe that Z should be completely determined by Y as per Eq. 3 . This condition implies that the entropy of Z given Y is zero, i.e., \displaystyle H(Z|Y)=0. (5) Then according to Kraft’s inequality [ 13 ] , the optimal bitrate for compressing Z losslessly and then compressing Y given