Data Provenance for Image Auto-Regressive Generation

Bihe Zhao, Louis Kerner, Michel Meintz, Tameem Bakr, Franziska Boenisch, Adam Dziedzic · 2026

Read Data Provenance for Image Auto-Regressive Generation by Bihe Zhao, Louis Kerner, Michel Meintz, Tameem Bakr, Franziska Boenisch, Adam Dziedzic (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28386.txt.

Introduction

Bihe Zhao, Louis Kerner, Michel Meintz, Tameem Bakr, Franziska Boenisch, and Adam Dziedzic trace images back to the model that made them. Autoregressive image generators, which build pictures token by token like language models write text, now produce photorealistic results. Knowing which model synthesized an image matters for fighting misinformation, fraud, and harmful content.

Zhao and colleagues find that although such images look indistinguishable from real ones, the generation process leaves characteristic patterns in its outputs. Their post-hoc framework detects those patterns to establish provenance. Because nothing in the generator or its outputs is modified, the method works where watermarking cannot: on content already published unmarked and on models with no watermark built in. Tests across many autoregressive generators showed strong provenance tracing.

Catalog record

CREATORBihe Zhao, Louis Kerner, Michel Meintz, Tameem Bakr, Franziska Boenisch, Adam Dziedzic
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE108 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28386

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Data Provenance for Image Auto-Regressive Generation Bihe Zhao, Louis Kerner, Michel Meintz, Tameem Bakr, Franziska Boenisch, Adam Dziedzic Abstract Image autoregressive models (IARs) have recently demonstrated remarkable capabilities in visual content generation, achieving photorealistic quality and rapid synthesis through the next-token prediction paradigm adapted from large language models. As these models become widely accessible, robust data provenance is required to reliably trace IAR-generated images to the source model that synthesized them. This is critical to prevent the spread of misinformation, detect fraud, and attribute harmful content. We find that although IAR-generated images often appear visually identical to real images, their generation process introduces characteristic patterns in their outputs, which serves as a reliable provenance signal for the generated images. Leveraging this, we present a post-hoc framework that enables the robust detection of such patterns for provenance tracing. Notably, our framework does not require modifications of the generative process or outputs. Thereby, it is applicable in contexts where prior watermarking methods cannot be used, such as for generated content that is already published without additional marks and for models that do not integrate watermarking. We demonstrate the effectiveness of our approach across a wide range of IARs, highlighting its high potential for robust data provenance tracing in autoregressive image generation. Our code is available at https://github.com/sprintml/DataProvenanceIAR . 1 Introduction Recent progress in image autoregressive models (IARs) has led to significant advancements in image generation. Driven by advances in language modeling, these models produce high-quality images at rapid pace using the next-token prediction paradigm ( 42 ; 11 ) . As IAR-generated images become visually indistinguishable from natural content, several challenges, including the spread of misinformation, fraud, and harmful content dissemination, arise. Additionally, as generated data “pollutes” the data ecosystem, it is increasingly used to train new generative models, which degrade model performance (called model collapse ( 1 ; 36 ) ) and amplify existing biases ( 52 ) . Therefore, data provenance, i.e., identifying and attributing generated images to their generators, is highly important. Several provenance methods have been developed for generative vision models, including both watermarking ( 7 ; 22 ; 51 ; 61 ; 10 ; 17 ; 13 ; 43 ; 50 ) and fingerprinting ( 18 ; 56 ; 26 ) . Yet, these methods require the integration of additional signals into the models or into the images either during or after generation. This introduces perceptible or statistical changes, is not applicable to trace provenance for content that has already been published without marks, and often results in a trade-off between robustness, imperceptibility, and applicability. In this work, we propose the first post-hoc provenance framework for IAR-generated images that does not require any modification of the generation process, is model-agnostic, and applicable to previously published, and unmarked content. Our proposed framework builds on our intriguing observation that because IARs encode images as sequences of discrete tokens from a fixed codebook, i.e., their ”vocabulary”, they introduce a quantization step that leaves model-specific artifacts in the generated images as shown in Figure 1 . Specifically, token representations of generated images are consistently closer to the codebook entries than those of natural images. We refer to this as QuantLoss and show that it can be used to trace IAR-generated images back to their generators. Figure 1: Data Provenance: Token Space. Since the generated tokens of a given IAR are sampled from the codebook entries, the codebook acts as a key to distinguish the token representations of generated images from those of real images. We further enhance the reliability of our framework by amplifying existing signals and integrating additional, carefully designed ones. First, we train a model to approximate the inverse of the IAR’s decoder and use it to encode images whose provenance we want to test. Since the inverted decoder leads to a higher fidelity mapping from images to the tokens that the image was potentially generated from, it strengthens our provenance signals based on QuantLoss. Additionally, we introduce a novel token-search algorithm for the next-scale prediction paradigm, enabling more accurate tracking of generated images’ initial tokens and enabling a more meaningful comparison to the codebook tokens. Beyond QuantLoss, we also identify a complementary signal, EncLoss , which captures the deviation observed when encoding an image to the latent space with the inverse decoder and then decoding it back to the image space with the original decoder. This process yields low loss for generated images due to feature consistency, but typically greater loss for natural images. All these signals can eventually be combined to robustly trace the provenance of IAR-generated images. We evaluate our method on state-of-the-art IARs, including VAR ( 42 ) , RAR ( 57 ) , LlamaGen ( 38 ) , Taming ( 6 ) , and Infinity ( 11 ) , as well as a vector-quantized diffusion model (VQ-Diffusion) ( 9 ) . We are able to detect the images generated from these models with almost 100% success rate, which contributes to a reliable provenance tracing of generated content in IARs. We note that the post-hoc finetuning of the encoder has relatively small overhead compared to IAR training, especially for the newly developed IARs with increasing model scale and training data. We also analyze the robustness of our method to conventional image post-processing techniques and show that our method can still detect most of the generated content, significantly outperforming the existing methods. In summary, we make the following contributions: 1. We introduce the first post-hoc data provenance method for IARs that leverages generation-specific artifacts to reliably determine whether, and by which model, an image was generated. Our framework does not require any modifications to the model’s training or generation process, is model-agnostic, and applicable to already published generated content. 2. We show that combining carefully designed provenance signals derived from the generation-specific artifacts enables near-perfect detection of generated images and accurate attribution to their source model, consistently achieving nearly 100% TPR@1%FPR across a diverse set of IARs and outperforming all baselines. 3. We provide a thorough empirical evaluation of our framework on diverse models and with diverse datasets, assessing its robustness to image perturbations. Our results highlight that our framework can provide effective provenance tracing under real-world scenarios with non-perfect data, highlighting its practical applicability. 2 Background and Related Work Image Autoregressive Models (IARs). IARs have recently gained traction as a new architecture for image generation, following the success of generative adversarial networks ( 16 ; 3 ; 14 ) and diffusion models ( 29 ; 31 ; 27 ) . IARs inherit the next-token prediction paradigm from large language models (LLMs) by treating images (or their patches) as sequences of discrete tokens which enables them to generate images both quickly and with high quality. Building on the advances from LLMs also allows IARs to follow their clear power-law scaling ( 42 ) . In the last years, the progress in autoregressive image generation has moved from early pixel-space raster-scan autoregression ( 2 ; 46 ) to pioneering efforts with the next scale or resolution prediction (VAR) ( 42 ; 11 ) . Recent proposals opt for next-token prediction of randomized inputs permuted into different factorization orders with annealing probability (RAR) ( 57 ) , and

Read the complete text