Adversarial Error Correction for Visual Autoregressive Generation
Ligong Bi, Tao Huang, Jianyuan Guo, Chang Xu, [1.5ex] Shanghai Jiao Tong University · 2026
Read Adversarial Error Correction for Visual Autoregressive Generation by Ligong Bi, Tao Huang, Jianyuan Guo, Chang Xu, [1.5ex] Shanghai Jiao Tong University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24843.txt.
Introduction
Ligong Bi and co-authors offer AID-VAR, a plug-and-play fix for cascading errors in visual autoregressive generation. Small early mispredictions amplify and distort the final image. AID-VAR adds error correction inspired by GAN adversarial feedback: a discriminator diagnoses fidelity gaps at each scale change, plus a lightweight guidance injector. The module acts as a non-invasive adapter refining the backbone, steering generation toward real images without destabilizing the latent space. Tests show sharper details and fewer distortions, with 16 percent better FID for 3 percent more parameters.
Catalog record
| CREATOR | Ligong Bi, Tao Huang, Jianyuan Guo, Chang Xu, [1.5ex] Shanghai Jiao Tong University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63295 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24843
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adversarial Error Correction for Visual Autoregressive Generation Ligong Bi, Tao Huang, Jianyuan Guo, Chang Xu, [1.5ex] Shanghai Jiao Tong University Abstract Visual Autoregressive (VAR) models have emerged as a powerful paradigm for image synthesis by performing hierarchical next-scale prediction. However, VAR models are inherently prone to cascading error propagation, where subtle coarse-scale mispredictions are amplified across the hierarchy, ultimately distorting the final synthesis. To mitigate this, we propose AID-VAR, a plug-and-play framework that enhances pre-trained VARs through Adversarially Injected Diagnosis. Instead of a standard passive generation, AID-VAR introduces a proactive error-correction mechanism inspired by the adversarial feedback in GANs. We deploy a discriminator to diagnose fidelity gaps at each scale transition, coupled with a lightweight guidance injector. This module operates as a non-invasive adapter that refines the feature manifold of a frozen VAR backbone, effectively steering the generation toward the distribution of real images without destabilizing the pre-trained latent space. Furthermore, to rigorously evaluate this cross-scale progression, we introduce the Inter-Scale Consistency Score (ISCS), a novel metric that quantifies the fidelity and structural alignment between consecutive resolution scales. Experimental results across various backbones demonstrate that AID-VAR delivers sharper textural details and fewer structural distortions with negligible overhead. For instance, AID-VAR-d20 achieves a 16% improvement in FID with only a 3% increase in parameters. These results establish AID-VAR as a highly efficient and scalable pathway for upgrading large-scale VAR generators—enhancing global coherence and local detail without altering training data, base architectures, or sampling schedules. Code is available at https://github.com/bijiw515/AID-VAR . 1 Introduction Autoregressive modeling has recently emerged as a competitive paradigm for high-fidelity image synthesis, offering stable training, exact likelihood, and flexible conditioning [ 48 , 49 , 11 , 6 , 45 ] . In the visual autoregressive (VAR) [ 45 ] setting, an image is synthesized across a hierarchy of spatial scales; each stage conditions on previously generated content and predicts the next chunk of visual information. Despite their strong generative capacity, VARs suffer from a characteristic failure mode: local prediction errors accumulate across scales, compounding into detail loss, texture drift, and small-region distortions that undermine global coherence, as illustrated in Figure 1 and Figure 2 . These issues stem from exposure bias [ 2 , 34 , 43 , 13 ] and imperfect cross-scale transitions: once a local mistake enters the chain, subsequent decoders amplify rather than correct it [ 27 , 28 , 21 , 45 , 7 , 25 , 54 ] . Figure 1 : Conceptual illustration of our motivation. Left: The standard VAR model’s coarse-to-fine generation is prone to error accumulation, where initial inaccuracies (circled in red) are magnified across scales, leading to severe structural degradation. Right: Our AID-VAR framework introduces a guidance injector at each step to anticipate and correct these errors, ensuring a globally coherent and plausible final image. We argue that mitigating these cross-scale errors requires diagnostic feedback at each scale transition, capable of identifying “off-manifold” artifacts and nudging the generation back toward the natural image distribution. Adversarial learning [ 12 , 3 , 19 , 18 ] provides precisely such a signal: a discriminator trained on real v.s. generated images excels at detecting fine-grained visual pathologies ( e.g. , broken micro-geometry, aliasing, texture misalignment). However, naïvely adversarially fine-tuning the base VAR is often impractical: it introduces training instability, alters a large number of parameters, and demands substantial compute to retain the original model’s strengths. Figure 2 : Qualitative comparison between our AID-VAR and the standard VAR across generation scales. The comparison demonstrates that AID-VAR better preserves visual consistency and reduces error accumulation during progressive generation. This paper introduces AID-VAR (Adversarially Injected Diagnosis for VAR), a lightweight, plug-and-play guidance module that upgrades a pre-trained, frozen VAR with adversarial feedback, without modifying training data, the base architecture, or the sampling schedule. AID-VAR first trains a discriminator to detect visual errors and leverages it to supervise a small guidance injector attached to the VAR. During generation, the injector conditions on previously generated scales and injects a spatial guidance signal that steers the next scale’s prediction toward the natural-image manifold identified by the discriminator. Crucially, the base VAR remains frozen; only the tiny injector and the discriminator heads are trained, yielding stability, efficiency, and easy implementation across backbones. To implement the above error-fixing mechanism via adversarial guidance, a key question is where the discriminator should operate. While the VAR’s native latent space, where multi-scale tokens reside, seems natural, our experiments show that discriminators trained directly in this compressed space suffer from rapid collapse and weak gradients. Latent representations, though semantically rich, lack the explicit spatial and textural details needed to discriminate subtle visual artifacts. We therefore propose a differentiable pipeline that operates in RGB pixel space rather than latent space. Specifically, we employ a projected discriminator [ 39 ] that leverages a fixed, pretrained image encoder to extract robust perceptual features and learns only shallow classification heads on top. To bridge the gap between RGB-space supervision and latent-space optimization, we introduce soft-label based VQ-VAE decoding. Unlike standard hard quantization that blocks gradients, our soft decoding computes continuous weighted combinations of codebook vectors, providing stable gradients from pixel space back through the decoder to the injector. This maintains end-to-end differentiability while preserving the discrete structure of the VAR. Finally, to prevent the discriminator from overfitting to stale artifacts, we adopt a dynamic training refresh strategy. At each training iteration: (1) the frozen VAR paired with the current injector generates refined images via the soft-decoding path; (2) the discriminator is updated on real images versus these current refined samples; (3) the injector is updated using the discriminator’s adversarial signal. This on-the-fly refresh stabilizes the min-max game, reduces overfitting to outdated generation patterns, and improves sample efficiency. We further observe that hierarchical autoregressive generation calls for evaluation beyond final-image realism. Standard image-level metrics, including FID [ 14 ] , IS [ 37 ] , and precision/recall [ 20 ] , mainly measure the quality of completed images, but do not reveal whether each refinement stage remains faithful to its coarse-scale context. To fill this gap, we introduce the Inter-Scale Consistency Score (ISCS), which measures the fidelity of transitions between consecutive scales. ISCS serves as a targeted probe for exposure bias across the hierarchy: higher ISCS indicates that fine-scale refinements remain consistent with coarse-scale structure, aligning with perceptual coherence. Evaluated under ISCS, baseline VARs score low due to sensitivity to early errors and semantic drift; in contrast, our AID-VAR substantially boosts ISCS, indicating stabilized cross-scale refinement and reduced error propagation. Our main contributions are summarized as follows: • We propose a novel adversarial framework to address the cumulated error issue in VAR, by diagnosing and correcting cross-scale errors via a lightweight injector. • We make adversarial supervision effective and stable for VARs through an RGB-space discriminator and a differentiable soft-decode route that jointly deliver informative gradients to the injector, together with a dynamically refreshed training loop that maintains up-to-date fake samples. • We introduce ISCS to quantify cross-scale coherence, complementing FID/IS by directly targeting exposure-bias effects between adjacent scales. • Extensive experiments on ImageNet-1K [ 8 ] demonstrate that AID-VAR consistently improves multiple VAR backbones while introducing only modest additional overhead. 2 Related Work Autoregressive models. The application