Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors
Yunuo Chen, Chuqin Zhou, Jiangchuan Li, Xiaoyue Ling, Bing He, Jincheng Dai, Li Song, Guo Lu · 2026
Read Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors by Yunuo Chen, Chuqin Zhou, Jiangchuan Li, Xiaoyue Ling, Bing He, Jincheng Dai, Li Song, Guo Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15129.txt.
Introduction
Chen and co-authors present NeFIC, a method for image compression at ultra-low bitrates that borrows from video diffusion. Decoding starts from a compact anchor frame that keeps scene layout and meaning but drops fine detail. A pretrained video diffusion model then predicts the step from that anchor to the full image. Starting from a visible, faithful anchor improves both realism and fidelity. On the CLIC2020 test set, the method saved over half the bitrate versus DiffC on several quality metrics, with decoding up to five times faster.
Catalog record
| CREATOR | Yunuo Chen, Chuqin Zhou, Jiangchuan Li, Xiaoyue Ling, Bing He, Jincheng Dai, Li Song, Guo Lu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38337 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15129
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors Yunuo Chen, Chuqin Zhou, Jiangchuan Li, Xiaoyue Ling, Bing He, Jincheng Dai, Li Song, Guo Lu Abstract We present a novel paradigm for ultra-low-bitrate image compression (ULB-IC) that exploits the “temporal” evolution in generative image compression. Specifically, we define an explicit intermediate state during decoding: a compact anchor frame, which preserves the scene geometry and semantic layout while discarding high-frequency details. We then reinterpret generative decoding as a virtual temporal transition from this anchor to the final reconstructed image. To model this progression, we leverage a pretrained video diffusion model (VDM) as temporal priors: the anchor frame serves as the initial frame and the original image as the target frame, transforming the decoding process into a next-frame prediction task. In contrast to image diffusion-based ULB-IC models, our decoding proceeds from a visible, semantically faithful anchor, which improves both fidelity and realism for perceptual image compression. Extensive experiments demonstrate that our method achieves superior objective and subjective performance. On the CLIC2020 test set, our method achieves over 50% bitrate savings across LPIPS, DISTS, FID, and KID compared to DiffC, while also delivering a significant decoding speedup of up to × \times 5. Code will be released later. Figure 1 : (a) Some previous diffusion-based ULB-IC methods transmit latents that serve as conditions for image diffusion models during generative decoding. (b) Our method decodes a compact anchor frame and uses a video diffusion model to evolve this anchor into the target image via next-frame prediction. Radar plot compares our model with recent generative codecs on the CLIC2020 test set. 1 Introduction The explosive growth of image data has sharply increased storage and transmission burdens, motivating steady progress in high-performance lossy image compression. Learned image compression (LIC) has advanced rapidly under variational autoencoders (VAEs) and end-to-end optimization 4 ; 5 . Early models relied on CNN-based analysis-synthesis transforms 65 ; 25 ; 89 , while later Transformer- and Mamba-based architectures further improved the rate-distortion (RD) performance 107 ; 106 ; 58 ; 13 ; 72 ; 15 ; 46 ; 99 ; 14 ; 51 ; 103 ; 52 ; 60 ; 49 ; 18 . Concurrently, the rise of Generative Adversarial Network (GAN) 22 and diffusion models 79 ; 27 has reframed ultra-low-bitrate image compression as a pivotal frontier. By leveraging powerful generative priors, diffusion-based codecs can deliver striking perceptual realism at extremely low bitrates. Most existing approaches 62 ; 9 ; 53 control the multi-step denoising process via controlling networks (Fig. 1 (a)). However, the unclear influence of the conditional signals on diffusion process and the uncertainty of Gaussian initialization potentially induce semantic drift and weaken faithfulness to the source 11 ; 62 ; 37 ; 93 . In addition, the iterative denoising process incurs substantially higher computational cost and decoding latency than conventional VAE-style codecs. So far, the central challenge for ULB-IC is to achieve simultaneously: (i) strong perceptual realism, (ii) source-content faithfulness, and (iii) practical inference efficiency. To achieve these targets, we propose a novel paradigm for ultra-low-bitrate image compression that explores the dynamics in generative image decoding. Instead of treating image reconstruction as a static process, our framework reinterprets single-image generative decoding as a virtual temporal process, where a compact anchor evolves to a high-fidelity reconstructed image. We define an explicit intermediate state for ULB-IC: a visible anchor that preserves scene geometry, semantic layout, and coarse appearance while discarding high-frequency details. As shown in Fig. 1 (b), this anchor is encoded by a VAE-based Anchor Codec. At the decoder, we first decode the anchor from the bitstream and treat it as the initial state. The evolution from anchor to fine-grained reconstruction is then modeled as a video-like progression using a VDM. We construct a virtual two-frame video, where the anchor is the first frame and the original image is the second. This setup casts generative decoding as a next-frame prediction task conditioned on the anchor, guided by the dynamic priors of the VDM. This yields a reconstruction consistent with the anchor and perceptually aligned with the source without extra control networks, improving both realism and fidelity. However, two key challenges arise when applying video diffusion models to image compression. (1) Domain Gap Between Video Generation and Image Decoding. Video diffusion models are primarily trained for gradual temporal evolution across dozens of frames, and emphasize scene transitions and object motions. In contrast, our goal is to achieve high-fidelity detail synthesis through a single-interval transition: a virtual two-frame “video” consisting solely of the compact anchor and the target image. (2) High Computational Cost of Iterative Denoising. State-of-the-art VDMs often require 50 iterative denoising steps and start from standard Gaussian noise, causing high latency and injecting ambiguity that can harm fidelity 62 ; 11 . Our aim is to turn multi-step video diffusion into a one-step generative decoder that preserves both fidelity and realism. To address these challenges, we propose a two-stage training scheme that adapts VDMs for efficient and faithful ultra-low-bitrate image decoding: Stage I: Next-Frame Decoding Adaptation. We jointly finetune the pretrained VDM and the anchor codec to synthesize high-fidelity details from compact anchor frames while suppressing undesirable temporal artifacts, such as object motion and scene flicker. This adaptation aligns the generative priors of the VDM with the anchor-conditioned detail synthesis process, enabling accurate and perceptually faithful reconstruction through a single-step temporal evolution. Stage II: One-Step Generative Bypass. To eliminate the latency introduced by iterative denoising, we collapse the multi-step diffusion process to one-step generation by establishing a latent bypass between anchor compression and video generation. Specifically, we condition the encoding of the anchor Compression-VAE on generative latents obtained from the Video-VAE. We then introduce a Bypass Refinement module, which is a learned mapping from the compression latent space to the generative latent space. This bypass regularizes and aligns the latent spaces of the two VAEs, reducing generative uncertainty and enabling faithful one-step decoding. Our contributions can be summarized as follows. 1. A novel ULB-IC paradigm: We reinterpret generative image decoding as a virtual temporal transition from a compact anchor into a faithful realistic reconstruction. 2. Efficient generative decoding: Our two-stage training pipeline adapts VDMs for compression-aware next-frame prediction while enabling one-step generation. 3. State-of-the-art performance: Extensive experiments demonstrate that our model achieves superior perceptual realism and fidelity with relatively faster decoding. 2 Related Work 2.1 Generative Image Compression VAE- and GAN-based Codecs. Previous VAE-based learned image compression models 16 ; 25 ; 35 ; 34 ; 66 ; 65 ; 105 ; 32 ; 87 mainly optimize standard objective metrics, such as PSNR and MS-SSIM. Agustsson et al. 2 and HiFiC 64 start GAN-based methods for low bitrate image compression and aim to improve realism and fidelity. MR 1 connects generative and non-generative compression. MS-ILLM 68 introduces a local likelihood model to improve realism. EGIC 40 uses discriminators conditioned on segmentation. Recent studies 63 ; 95 ; 33 use vector quantization for extreme semantic compression. TACO 44 proposes a text-guided encoder to improve fidelity. Diffusion-based Codecs. Recent works 45 ; 41 ; 82 ; 19 ; 36 ; 56 ; 92 ; 96 ; 75 ; 94 ; 80 ; 104 ; 93 ; 54 ; 10 ;