Generalization of Diffusion Models Arises with a Balanced Representation Space

2025

Read Generalization of Diffusion Models Arises with a Balanced Representation Space (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20963.txt.

Introduction

Zekai Zhang, Qing Qu, and five co-authors explain, in terms of internal representations, why a diffusion model sometimes invents and sometimes photocopies. Studying a two-layer denoising autoencoder, they prove that memorization looks like raw training samples stored in the weights, producing localized, spiky representations. Generalization looks different: the model captures local statistics of the data, and its representations stay balanced across the space.

The same signatures show up in real unconditional and text-to-image diffusion models, so the toy-model theory carries practical weight. Two tools follow from the diagnosis: a representation-based detector that flags when a model is memorizing, and a training-free editing method that steers generation by nudging representations. Knowing which regime a model sits in is the first step to keeping generative systems on the creative side of the line.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20963

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generalization of Diffusion Models Arises with a Balanced Representation Space Abstract Zekai Zhang † , Xiao Li † , Xiang Li , Lianghe Shi , Meng Wu , Molei Tao 1 , Qing Qu ‡ University of Michigan \cdot 1 Georgia Institute of Technology † Joint first author ‡ Corresponding author Abstract Diffusion models excel at generating high-quality, diverse samples, yet they risk memorizing training data when overfit to the training objective. We analyze the distinctions between memorization and generalization in diffusion models through the lens of representation learning. By investigating a two-layer ReLU denoising autoencoder (DAE), we prove that: ( i ) memorization corresponds to the model storing raw training dataset in the learned weights for encoding and decoding, yielding localized, “spiky” representations; whereas ( ii ) generalization arises when the model captures local data statistics, producing “balanced” representations. Furthermore, we validate our theoretical findings on real-world unconditional and text-to-image diffusion models, demonstrating that the same representation structures emerge in deep generative models with significant practical implications. Building on these insights, we propose a representation-based method for detecting memorization and a training-free editing technique that allows precise control via representation steering. Together, our results highlight that learning good representations is central to novel and meaningful generative modelling . Keywords: Diffusion Models, Generalization, Representation Learning, Denoising AutoEncoders Date: August 10, 2026 Correspondence: [email protected] Resources: To be released https://deepthink-umich.github.io Figure 1 : Overview of theoretical findings. We analyze a two-layer ReLU DAE (middle row) to explain diffusion model behavior. We show that: ( i ) memorization occurs by storing raw samples in weights (top left), yielding spiky activations (bottom left); ( ii ) generalization arises by learning data statistics (top right), producing balanced , semantic codes (bottom right); and ( iii ) real-world models exhibit a hybrid regime due to data imbalance (top center). These insights unify representation and distribution learning , with empirical validation in Figure 2 . 1 Introduction Diffusion models [ 36 , 56 ] have rapidly emerged as the dominant class of generative models, powering state-of-the-art systems such as Stable Diffusion [ 66 ] , Flux [ 47 ] , and Veo [ 30 ] . By iteratively denoising random noise, they achieve unprecedented scalability, controllability, and fidelity. However, their empirical success raises a fundamental question: in principle, the standard training objective (e.g., denoising score matching) admits a closed-form solution that merely memorizes training examples [ 101 ] ; in practice, however, real-world models consistently produce novel and diverse outputs [ 108 , 39 ] . This distinct mismatch between theoretical expectation and observed behavior poses a critical gap in our understanding of diffusion model generalization , with direct implications for privacy, interpretability, and trustworthy deployment [ 72 ] . Addressing this question has drawn increasing attention in the machine learning community [ 108 , 52 , 88 , 39 , 31 , 11 , 107 , 8 , 106 ] , yet existing explanations remain far from satisfactory. Early works based on random feature models [ 50 , 29 ] provide useful insights but necessarily oversimplify model architectures. Analyses of linear models on Gaussian mixtures [ 52 , 88 , 86 ] shed light on generalization but cannot capture memorization. Another line of research explores inductive biases by constructing handcrafted closed-form solutions from empirical data to approximate U-Net performance [ 41 , 60 , 57 , 25 ] , attributing success to principles such as locality and equivariance. While these advances are valuable, the findings remain fragmented and phenomenological, and a more unified account of how diffusion models both memorize and generalize is still lacking (see Section 5 for a more detailed discussion of related work). To address these challenges, we develop a unified mathematical framework based on a theoretical analysis of a nonlinear two-layer ReLU denoising autoencoder (DAE) [ 84 ] . This framework not only unifies the characterization of memorization and generalization, but also bridges distribution learning with representation learning, offering profound practical implications. Specifically: ( i ) Memorization. We prove that when empirical samples are locally sparse, the network weights memorize and store individual training examples, leading to overfitting and hence memorization. ( ii ) Generalization. Conversely, when the empirical data are locally abundant, the weights effectively capture local data statistics, enabling the model to generate novel in-distribution samples. Crucially, our work provides a unique representation-centric perspective on generalization [ 79 ] , highlighting the pivotal role of bottleneck activations in DAE networks. This view is motivated by recent empirical evidence on the duality between distribution learning and representation learning in diffusion models [ 54 , 96 ] : they inherently learn informative features for downstream tasks [ 46 , 18 ] , and representation alignment regularization has been shown to accelerate training [ 102 ] . Our theory makes this connection explicit: memorized samples are encoded as spiky activations concentrated on a few neurons, whereas generalized samples yield balanced representations that reflect the underlying distribution. These contrasting modes of representation learning manifest in distinct generation behaviors in terms of memorization or generalization, which we comprehensively validated across a range of models, including EDM [ 43 ] , Diffusion Transformers (DiT) [ 62 ] , and Stable Diffusion v1.4 [ 66 ] (SD1.4). Moreover, our findings show that the representation space is not a byproduct but a crucial and controllable factor for generation. Specifically, we demonstrate two practical implications: ( i ) Memorization detection. Leveraging the spikiness of representations identified by our theory as a signature of memorization, we develop a theory-driven detector that achieves highly accurate and efficient performance in a prompt-free manner. ( ii ) Model steering. We propose an effective steering method based on additions in the representation space and reveal distinct behaviors between memorization and generalization: memorized samples are difficult to steer, whereas generalized samples are highly steerable owing to their balanced, semantically rich representations. Together, these applications illustrate the far-reaching implications of our representation-centric analysis for the privacy, interpretability, and controllability of diffusion models. Figure 2 : Diffusion models generalize while learning benign internal representations. Activations from intermediate network layers form a representation space , within which distinct patterns emerge: memorized samples produce spiky representations that make them detectable, whereas novel generations yield balanced, information-rich representations that support controllable generation via representation steering. Summary of contributions. Our main contributions are as follows: • Unified framework in a nonlinear ReLU setting. We analyze the optimal solutions of a two-layer nonlinear ReLU DAE under different empirical data sizes, providing a unified characterization of memorization and generalization that goes beyond prior random-feature or linear model analyses. • A representation-centric understanding of generalization. We establish a rigorous connection between representation structures and generalization, identifying distinct patterns that separate memorization from generalization and validating these insights across diverse model settings. • Theory-inspired tools for memorization detection and model steering. Building on our analysis, we propose simple yet effective methods for memorization detection and representation-space steering, revealing distinct behaviors of generalized versus memorized samples. 2 Problem Setup In this section, we first introduce the basics of diffusion models, and then describe our problem setup for theoretical studies in Section 3 . 2.1 A Denoising Perspective of Diffusion Models Basics of diffusion models. Diffusion models

Read the complete text