Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment
Bryan Sangwoo Kim, Jeongsol Kim, Jong Chul Ye · 2025
Read Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment by Bryan Sangwoo Kim, Jeongsol Kim, Jong Chul Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18600.txt.
Introduction
Bryan Sangwoo Kim, Jeongsol Kim, and Jong Chul Ye present Chain-of-Zoom, a framework for extreme image enlargement. Their 2025 paper addresses a known failure: super-resolution models look sharp at their trained scale but collapse when asked to magnify far beyond it. Their method needs no extra training of the base model.
Chain-of-Zoom reuses one backbone model in a chain of mid-scale steps, each conditioned on the last. Text prompts that know the current scale, made by a vision-language model, guide each step as visual cues thin out. That prompt maker is itself tuned with reward-based learning against a critic model. A standard four-times model wrapped in the chain passed two hundred fifty-six times enlargement with strong perceptual quality.
Catalog record
| CREATOR | Bryan Sangwoo Kim, Jeongsol Kim, Jong Chul Ye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18600
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment Bryan Sangwoo Kim, Jeongsol Kim, Jong Chul Ye Abstract Modern single-image super-resolution (SISR) models deliver photo-realistic results at the scale factors on which they are trained, but collapse when asked to magnify far beyond that regime. We address this scalability bottleneck with Chain-of-Zoom (CoZ) , a model-agnostic framework that factorizes SISR into an autoregressive chain of intermediate scale-states with multi-scale-aware prompts. CoZ repeatedly re-uses a backbone SR model, decomposing the conditional probability into tractable sub-problems to achieve extreme resolutions without additional training. Because visual cues diminish at high magnifications, we augment each zoom step with multi-scale-aware text prompts generated by a vision-language model (VLM). The prompt extractor itself is fine-tuned using Generalized Reward Policy Optimization (GRPO) with a critic VLM, aligning text guidance towards human preference. Experiments show that a standard 4 × 4\times diffusion SR model wrapped in CoZ attains beyond 256 × 256\times enlargement with high perceptual quality and fidelity. Project Page: https://bryanswkim.github.io/chain-of-zoom/ . Figure 1: Extreme super-resolution of photorealistic images by CoZ with up to 64× magnification (top) and 256× magnification (bottom). Fine details such as textures on a wall, wrinkles on a flag, and leaf veins are clearly seen. 1 Introduction The field of generative modeling has witnessed remarkable progress, enabling the synthesis of highly realistic data across various modalities, including images, text, and audio. A key application benefiting from these advancements is single-image super-resolution (SISR), which aims to reconstruct high-resolution (HR) details from a low-resolution (LR) input image. Super-resolution is a problem of core interest for effectively bridging the gap between low-cost imaging sensors and high-fidelity visual information; its usages range from enhancing consumer photographs and legacy media to improving critical details in medical imaging, satellite surveillance, and scientific visualization [ 2 , 28 , 30 , 40 , 43 ] . The standard approach to SISR is based on the posterior probability distribution: p({\bm{x}}_{H}\mid{\bm{x}}_{L}) (1) where the goal is to sample a plausible HR image {\bm{x}}_{H} for a given input LR image {\bm{x}}_{L} . However, the mapping from {\bm{x}}_{L} to {\bm{x}}_{H} is highly complex and fundamentally ill-posed: a single LR image can correspond to a multitude of plausible HR images. This makes directly modeling the distribution extremely challenging for large magnification factors, and early attempts relying on interpolation or regression often produced blurry results [ 10 , 13 , 19 , 50 ] . Recent emergence of powerful generative models ( e.g. , diffusion-based models) has led to significant advancement in this task, providing strong generative priors over natural images that enable the synthesis of realistic textures and details consistent with the low-resolution input. Specifically, existing methods leveraging such generative priors largely fall into two categories. One line of work frames SR as an inverse problem, utilizing a pre-trained generative model as a prior during inference time to find a realistic HR image consistent with the LR input [ 5 , 6 , 7 , 8 , 20 , 21 ] . While such inverse problem-solving methods benefit from being training-free, they typically require lengthy iterative optimization or sampling processes at inference time to enforce data consistency ( i.e. , ensuring the downsampled HR prediction matches the original LR input), making them computationally expensive. Another line of work aims to incorporate this data consistency directly into the model’s training objective, thereby enabling much faster inference [ 31 , 42 , 47 , 48 , 54 , 55 ] . Modern state-of-the-art models within this category are capable of producing high-quality super-resolved images, even in a single inference step [ 47 , 55 ] . However, these fast, trained super-resolution models suffer from a significant limitation: they are inherently upper-bounded by their training configuration and tend to collapse when presented with inputs requiring magnification beyond what they were trained on [ 22 , 24 , 57 ] . This failure occurs because the model’s internal representations and learned restoration functions are tightly coupled to the specific scale and degradation seen during training [ 34 ] . Applying it outside this domain violates its learned assumptions, leading to severe artifacts, blurry outputs, or a complete failure to generate meaningful high-frequency details [ 11 , 13 , 19 ] . This lack of robustness severely restricts the practical applicability of these otherwise powerful models, demanding new models to be trained when the desired magnification factor exceeds what can be currently provided, which is highly inefficient. Figure 2: (a) Conventional SR. When an SR backbone trained for a fixed up-scale factor ( e.g. , 4 \times ) is pushed to much larger magnifications beyond its training regime, blur and artifacts are produced. (b) Chain-of-Zoom (ours). Starting from an LR input, a pretrained VLM generates a descriptive prompt, which—together with the image—is fed to the same SR backbone to yield the next HR scale-state. This prompt-and-upscale cycle is repeated, allowing a single off-the-shelf model to climb to extreme resolutions (16 \times –256 \times ) while preserving sharp detail and semantic fidelity. In this work, we therefore propose to solve a fundamental question: How can we effectively utilize super-resolution models to explore much higher resolutions than they were originally trained for? Solving this question is critical in that it addresses the practical need for flexible and arbitrary-scale super-resolution, allowing users to magnify images to desired levels without being constrained by model training specifics. Furthermore, training models for extremely high magnification factors ( e.g. , 16x, 32x) directly is often computationally prohibitive due to memory and time constraints [ 45 ] . Enabling the extension of existing, well-trained models ( e.g. , 4x SR models) to higher factors offers a significantly more resource-efficient pathway to achieving extreme resolutions. To address these fundamental challenges, we present Chain-of-Zoom (CoZ) , a novel framework for achieving extreme-resolution image generation beyond the training configurations of conventional super-resolution models. Specifically, we introduce intermediate scale-state modeling to bridge the gap between a low-resolution (LR) input and a high-resolution (HR) target image. These intermediate scale-states enable the decomposition of the conditional distribution in Eq. ( 1 ) into a series of tractable components, forming the basis of a scale-level autoregressive (AR) framework. Within this framework, models can progressively generate high-quality images at resolutions previously considered unattainable. In particular, building on the scale-level AR-2 model, we further propose a multi-scale-aware prompt extraction technique. This approach leverages Vision-Language Models (VLMs) to extract descriptive text prompts by attending to multiple scale-states throughout the zooming process, enabling semantically aligned and coherent super-resolution. This is from the observation that at extreme resolutions, conditioning provided by the original signal {\bm{x}}_{L} becomes insufficient, thus leading to unreasonable hallucinations by the SR model in cases. Furthermore, to obtain text prompts of even richer detail that aligns with human preference, we fine-tune the prompt-extraction VLM under a novel RLHF pipeline leveraging GRPO [ 33 ] . A core part of this pipeline is the utilization of a critic VLM to score the outputs of the prompt extraction VLM, thus guiding it to produce prompts more aligned to human preference. Incorporated into the CoZ framework, our final VLM model successfully guides the super-resolution process towards reasonable high-quality results. In summary, our contributions are as follows: • We present Chain-of-Zoom