Temporal Score Analysis for Understanding and Correcting Diffusion Artifacts

Yu Cao, Zengqun Zhao, Ioannis Patras, Shaogang Gong · 2025

Read Temporal Score Analysis for Understanding and Correcting Diffusion Artifacts by Yu Cao, Zengqun Zhao, Ioannis Patras, Shaogang Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16218.txt.

Introduction

Yu Cao, Zengqun Zhao, Ioannis Patras, and Shaogang Gong wrote this machine learning paper, published in 2025. It explains why AI image generators leave artifacts, warped hands and smeared faces, by watching how an image forms over time inside the model.

The authors split the image-building process into three phases: profiling, mutation, and refinement. Artifacts, they find, are born in the middle phase, where some regions show odd jumps in the model's guiding scores. That is why fixes that only inspect the finished image miss the trouble spots. Their method, ASCED, watches score behavior during generation, flags the bad regions, and repairs them on the fly with fresh noise. In tests it cut artifacts across many kinds of images, matching or beating detector-based fixes without extra training.

Catalog record

CREATORYu Cao, Zengqun Zhao, Ioannis Patras, Shaogang Gong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16218

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Temporal Score Analysis for Understanding and Correcting Diffusion Artifacts Yu Cao, Zengqun Zhao, Ioannis Patras, Shaogang Gong Abstract Visual artifacts remain a persistent challenge in diffusion models, even with training on massive datasets. Current solutions primarily rely on supervised detectors, yet lack understanding of why these artifacts occur in the first place. In our analysis, we identify three distinct phases in the diffusion generative process: Profiling, Mutation, and Refinement. Artifacts typically emerge during the Mutation phase, where certain regions exhibit anomalous score dynamics over time, causing abrupt disruptions in the normal evolution pattern. This temporal nature explains why existing methods focusing only on spatial uncertainty of the final output fail at effective artifact localization. Based on these insights, we propose ASCED (Abnormal Score Correction for Enhancing Diffusion), that detects artifacts by monitoring abnormal score dynamics during the diffusion process, with a trajectory-aware on-the-fly mitigation strategy that appropriate generation of noise in the detected areas. Unlike most existing methods that apply post hoc corrections, e.g . , by applying a noising-denoising scheme after generation, our mitigation strategy operates seamlessly within the existing diffusion process. Extensive experiments demonstrate that our proposed approach effectively reduces artifacts across diverse domains, matching or surpassing existing supervised methods without additional training. Project page: YuCao16.github.io/ASCED . Figure 1 : Why do diffusion models generate artifacts? We discover that a diffusion generative process necessarily undergoes three phases, we call them: (2) “Profiling” which recovers holistic mean templates, (2) “Mutation” which introduces local divergence, and (3) “Refinement” which rationalizes pixel-wise generation in spatial context. Four visual examples are shown: The first two rows are two examples of rational local mutations (in green boxes) either naturally integrated (Row 1) or reasonably eliminated (Row 2). The bottom two rows show two failure cases when mutations were trapped unreasonably (in red boxes), resisting refinement and resulting in artifacts. Phases are visualized in equal intervals for clarity; please zoom in for more details. 1 Introduction Figure 2 : Diagram of our framework . Denoising and Noising are using Eq. 5 and Eq. 1 , respectively. Diffusion models have emerged as powerful foundation models in computer vision [ 5 ] , achieving remarkable success in image generation [ 15 , 17 , 16 , 32 , 7 ] , image inpainting [ 35 , 27 , 24 ] , and text-to-image task [ 32 , 30 , 31 ] . However, even trained on large-scale datasets, diffusion generative images still exhibit two significant flaws: visual artifacts and hallucinations [ 43 , 47 ] . Visual artifacts appear as local irregularities in texture or structure, while hallucinations involve semantically incoherent content, e.g . , extra limbs or misplaced objects. In this work, we focus on addressing diffusion artifacts, which present a fundamental challenge to achieving reliable and high-quality image generation. Existing methods primarily treat visual artifact detection as a classification problem, i.e . , identifying problematic generations for filtering or reconstruction. These methods typically rely on a specialized classifier, either trained on manually annotated artifact datasets [ 43 ] or leveraging a pre-trained Large Multi-Modal Model (LMM) [ 22 ] . However, such post hoc interventions fail to address a fundamental problem: Why and when do artifacts emerge in diffusion models? To bridge this gap, we begin by examining the diffusion generation process itself. We discover that while diffusion process is guided by the same fundamental equation across time ( i.e . diffusion steps), in practice, the model exhibits different behavior that can be roughly categorized in three different temporal phases that we name Profiling, Mutation, and Refinement. In the “Profiling” phase, the model sketches the basic semantic global layout; in the “Mutation” phase it explores potential local pixel-wise variations to create local structure; in the “Refinement” phase it attempts to resolve these local pixel-wise variations into coherent visual details in context (see Fig. 1 for visual examples and Sec. 3.2 for a detailed analysis). This understanding of the generation process reveals that while visual artifacts may appear randomly, they follow systematic and identifiable temporal patterns during image formation. Recent uncertainty-based approaches try to identify artifacts by converting diffusion models into Bayesian networks and employ techniques such as Last Layer Laplace Approximation [ 9 ] to generate pixel-level variance matrices [ 20 ] . However, these uncertainty quantification analyses only capture spatial variations in the final output, i.e . , Var(x_{0}) , missing crucial temporal dynamics during the generation process. Our study shows that diffusion artifacts emerge when certain image regions exhibit abnormal evolution patterns over time , primarily during the Mutation phase of the generation process. Specifically, their pixel values stop updating properly while the surrounding areas continue to evolve. This phenomenon, which we formally define as “score traps” in Sec. 3.2 , explains why examining only the final output is insufficient and misplaced for effective diffusion artifact detection. Building on these insights, we propose ASCED: Abnormal Score Correction for Enhancing Diffusion, as shown in Fig. 2 . At the heart of the method is the estimation of a score, in this context the direction and magnitude of pixel-wise evolution at each diffusion generative step, and a scheme that analyses its temporal dynamics and detects abnormalities. We show both theoretically and experimentally that these abnormalities strongly correlate with artifact formation, making early detection possible at a stage where intervention is still feasible, before artifacts become irreversibly embedded in the generation process. We leverage this early detection capability by implementing a novel trajectory-aware correction mechanism that disrupts the evolution of artifact regions while preserving overall generation diversity. Importantly, ASCED operates in a fully unsupervised manner without requiring manual annotations or domain-specific training, making it readily applicable across various domains, , particularly valuable when training data may be limited or protected. Our contributions are: (1) We provide new insights into the formation of visual artifacts in the diffusion generative process, advancing the understanding of diffusion model internal mechanisms. (2) We introduce a novel method that detects potential artifact regions by monitoring abnormal score dynamics temporally, without any manually labeled training required. (3) We further develop a on-the-fly trajectory-aware correction mechanism that effectively mitigates artifacts while preserving image diversity. 2 Related Works Visual Artifact Detection initially targeted super-resolution artifacts, where upsampling operations are the main source [ 45 ] . These methods analyze either spatial domain characteristics to capture texture differences between real and generated images [ 41 , 23 ] , or frequency domain patterns to study artifact characteristics in high-frequency components [ 12 , 13 ] . More recent work has shifted focus to detecting artifacts in general image generation, developing specialized classifiers trained on manually annotated datasets [ 43 ] or utilizing pre-trained large vision models [ 22 ] . However, these supervised approaches require extensive manual labeling and may not generalize well across different domains. A parallel direction explores uncertainty quantification methods to understand visual artifacts. While various approaches including variational inference [ 4 , 18 ] , Laplace approximation [ 26 , 29 ] , and Markov Chain Monte Carlo [ 38 , 44 ] have been developed, their application to diffusion models remains limited [ 20 ] . BayesDiff [ 20 ] pioneers pixel-level uncertainty quantification in diffusion models using Last-layer

Read the complete text