Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Models Nick Dodson ∗, Xinyu Gao ∗, Qingsong Wang ∗, Yusu Wang †, Zhengchao Wan †‡ Abstract Diffusion models generate high-quality samples but can also memorize training data, raising serious privacy concerns. Understanding the mechanisms governing when memorization versus generalization occurs remains an active area of research. In particular, it is unclear where along the noise schedule memorization is induced, how data geometry influences it, and how phenomena at different noise scales interact. We introduce a geometric framework that partitions the noise schedule into three regimes based on the coverage properties of training data by Gaussian shells and the concentration behavior of the posterior, which we argue are two fundamental objects governing memorization and generalization in diffusion models. This perspective reveals that memorization risk is highly non-uniform across noise levels. We further identify a danger zone at medium noise levels where memorization is most pronounced. In contrast, both the small and large noise regimes resist memorization, but through fundamentally different mechanisms: small noise avoids memorization due to limited training coverage, while large noise exhibits low posterior concentration and admits a provably near linear Gaussian denoising behavior. For the medium noise regime, we identify geometric conditions through which we propose a geometry-informed targeted intervention that mitigates memorization. 1 Introduction Diffusion models have achieved remarkable success in generative modeling, producing high-quality images but sometimes memorizing the training data, raising serious privacy and copyright concerns [ 21 , 4 , 27 ] . This memorization risk is inherent to the training objective, as it admits a unique global minimum, namely the empirical optimal solution, which can only generate the training data [ 9 ] . Consequently, it is both theoretically interesting and practically important to understand how and when a learned model can deviate from the empirical optimal to generalize beyond its training data. Some recent works have analyzed this question. In particular, [ 23 ] argues that the stochastic nature of diffusion model training helps prevent fitting the empirical optimal denoiser, and thus promotes generalization. Later, [ 2 ] contests this view, showing that for a large range of training times the regression target is essentially unique while model still generalizes. Recently, [ 22 ] point out that supervised training in diffusion models is limited to small regions which fails to cover most parts of the inference trajectory, and argues that the generalization behavior of the model stems from extrapolation. Our work. A key aspect to understand when models generalize is to understand when they memorize , which we focus on in this paper. To this end, we distinguish between trajectory-level memorization , which concerns whether full sampling trajectories collapse to the training set, and per-noise-level memorization , which characterizes memorizing denoiser behavior at a fixed noise scale. Empirically, we find that models exhibiting strong trajectory-level memorization may nonetheless behave benignly at many individual noise levels, indicating that memorization is not induced uniformly across the noise schedule (see Section 3 for more details). This observation motivates our central question: At which noise levels is memorization formed, and what geometric mechanisms govern its emergence? To address these questions, we analyze diffusion training through a geometric lens based on two quantities: posterior weight concentration and Gaussian shell coverage of the training data. This perspective naturally partitions the noise schedule into three regimes. At both small and large noise levels, the training lies on “calm ends” that are resistant to memorization, but for fundamentally different reasons: in the small noise regime, memorization is suppressed due to limited training coverage, while in the large noise regime due to weak posterior concentration and effectively linear denoising behavior. In contrast, a “danger zone” emerges at intermediate noise level – the “wild middle” – where these two effects align and memorization risk peaks sharply. Crucially, memorization formed in this regime can propagate along the inference trajectory and persist into the small noise stage, leading the model to memorize even when both ends of the schedule remain benign. This observation suggests that, to avoid memorization, it is most effective to control the behavior of diffusion models specifically within this intermediate “danger zone”. By analyzing how coverage and posterior concentration interact, we can predict the location of this danger zone directly from the dataset. Experiments in Section 5 confirm this picture and show that selectively undertraining the intermediate regime can substantially mitigate memorization while preserving generation quality. More related work. Several explanations have been proposed for diffusion model generalization. Notably, [ 15 ] shows that models in the generalization regime exhibit closeness to a Gaussian linear model, suggesting an inductive bias toward learning the covariance structure. Neural networks also exhibit spectral bias [ 19 , 25 , 10 ] , which may influence what diffusion models learn. Other factors known to promote generalization include early stopping before memorization occurs, limiting model capacity relative to dataset size [ 15 ] , increasing the amount of training data as the scaling behavior for generalization and memorization differ in terms of the number of data [ 3 ] , and conditional training with auxiliary noise classes to provide distributional contrast [ 28 ] . 2 Background Let p denote a data distribution on \mathbb{R}^{d} . Let \bm{X}\sim p denote a random variable following p and \bm{Z}\sim\mathcal{N}(0,I_{d}) a standard Gaussian random variable independent of \bm{X} . For the training and sampling of diffusion models, we mostly follow the description given in [ 12 ] . Training. In diffusion models, one of the central tasks is to denoise a noisy observation \bm{X}_{\sigma}=\bm{X}+\sigma\bm{Z} for various noise levels \sigma>0 in order to obtain \bm{X} . In particular, one popular diffusion training loss is defined as \mathcal{L}(\theta):=\mathbb{E}_{\begin{subarray}{c}\bm{X}\sim p\\ \bm{Z}\sim\mathcal{N}(0,I_{d})\\ \sigma\sim\lambda\end{subarray}}\left\|m^{\theta}_{\sigma}(\bm{X}_{\sigma})-\bm{X}\right\|^{2} (1) where \lambda denotes certain distribution on some chosen [\sigma_{\min},\sigma_{\max}]\subset(0,\infty) , and m_{\sigma}^{\theta}:\mathbb{R}^{d}\to\mathbb{R}^{d} is a neural network parameterized by \theta that aims to approximate the (optimal) denoiser m_{\sigma} that we will introduce below. We refer to m_{\sigma}^{\theta} as a trained denoiser in what follows. Optimal solution. The above loss admits a unique global minimum given by the posterior mean which we call the optimal denoiser : m_{\sigma}(x):=\mathbb{E}[\bm{X}\mid\bm{X}_{\sigma}=x] . In practice, a model is trained with finite samples \mathcal{D}=\{x_{i}\}_{i=1}^{N} assumed to be drawn i.i.d. from some data distribution p . In this case, the expectation over \bm{X}\sim p in the training loss Equation 1 can be replaced by \bm{X}\sim p_{\mathcal{D}} where p_{\mathcal{D}}:=\frac{1}{N}\sum_{i=1}^{N}\delta_{x_{i}} , and the resulting optimal denoiser, which we now call the empirical optimal denoiser , has a closed form given by m_{\sigma}(x)=\sum_{i=1}^{N}w_{i}(x,\sigma)x_{i} (2) where w_{i}(x,\sigma)=\mathbb{P}(\bm{X}=x_{i}\mid\bm{X}_{\sigma}=x) is called the posterior weight on x_{i} at the noise level \sigma and is given by w_{i}(x,\sigma)=\frac{\exp\left(-\frac{\|x-x_{i}\|^{2}}{2\sigma^{2}}\right)}{\sum_{j=1}^{N}\exp\left(-\frac{\|x-x_{j}\|^{2}}{2\sigma^{2}}\right)}. (3) Sampling. After training, one samples z\sim\mathcal{N}(0,I_{d}) and lets x_{\sigma_{\max}}=\sigma_{\max}z for some large \sigma_{\max} . New samples are generated by solving the following ODE: {dx_{\sigma}}/{d\sigma}=-({m_{\sigma}^{\theta}(x_{\sigma})-x_{\sigma}})/{\sigma} (4) from \sigma_{\max} to \sigma_{\min} . For any \sigma\in[\sigma_{\min},\sigma_{\max}] , we denote by \Psi^{\theta}_{\sigma}:\mathbb{R}^{d}\to\mathbb{R}^{d} the flow map induced by the denoiser m_{\sigma}^{\theta} , which maps x_{\sigma_{\max}} to x_{\sigma} by solving the ODE above. Figure 1 : MSE to Clean Image. Comparison of denoising quality across noise levels. Solid lines: training data; dotted lines: test data. EDM-1K shows a generalization gap in the mid- \sigma region. 3 Empirical Investigation of Memorization To understand where and how memorization emerges during diffusion model training, we begin with an empirical investigation that motivates our subsequent geometric framework. By comparing models with different memorization behaviors across the noise schedule, we find the intermediate noise level is the most susceptible to memorization. We start by comparing trained denoisers and the empirical optimal denoiser on the CIFAR-10 dataset [ 13 ] . We adopt the EDM from [ 12 ] to model m_{\sigma}^{\theta} and consider three denoisers: a full-data EDM denoiser trained on 50k images ( EDM-50K ), an EDM denoiser trained on only 1k images ( EDM-1K ) 1 1 1 Training on 1k CIFAR-10 images is known to lead to memorization [ 9 ] . We also validate this in Table 1 . The 1k data is the first 1k in CIFAR-10 which is randomly indexed. , and the empirical optimal denoiser under the same 1k training images ( EMP-1K ). One straightforward way to compare the three models is to evaluate their denoising losses via Equation 1 . For a fixed noise level \sigma , this objective reduces to the denoising mean squared error (MSE). We therefore evaluate the fixed- \sigma denoising MSE of the three models over \sigma\in[0.002,80] on both training and test data from CIFAR-10: \mathrm{MSE}_{\sigma}(p_{\bullet}):=\mathbb{E}_{\begin{subarray}{c}\bm{X}\sim p_{\bullet}\\ \bm{Z}\sim\mathcal{N}(0,I_{d})\end{subarray}}\left\|m_{\sigma}^{\bullet}(\bm{X}_{\sigma})-\bm{X}\right\|^{2} (5) where m_{\sigma}^{\bullet} denotes either a trained or an optimal denoiser and p_{\bullet} is either the training distribution p_{\mathcal{D}} or the test distribution p_{\mathcal{T}} . Intuitively, \mathrm{MSE}_{\sigma}(p_{\mathcal{D}}) measures how well a model can recover a training image from its noisy version at noise level \sigma , while \mathrm{MSE}_{\sigma}(p_{\mathcal{T}}) measures the corresponding performance on unseen test images. Taken together, these two quantities provide a coarse diagnostic of generalization behavior across noise scales: a small train–test gap suggests similar behavior on training and test data, whereas a large gap indicates degradation on test samples and may signal memorization. We emphasize, however, that train–test MSE alone is not a definitive measure of memorization, but serves here as a useful summary statistic. Figure 2 : Per-Noise-Level Memorization Rate. Fraction of denoised test images classified as memorized at each noise level. The results are shown in Figure 1 . Interestingly, we observe three distinct noise regions in which the three models exhibit qualitatively different behavior. At this stage, this partition is purely observational; in subsequent sections, we provide a geometric characterization that explains the emergence of these large-, intermediate-, and small noise regimes. 1. At large \sigma (i.e., large noise), all three denoisers behave similarly on both train and test. We will later explain in Section 4.2.2 that this is due to the fact that high noise denoising is dominated by coarse statistics. 2. At middle \sigma , EMP-1K sharply diverges from EDM-50K , achieving near-zero training error but large test error due to its nearest-neighbor nature, while EDM-50K continues to generalize well. 3. Interestingly, at small \sigma , although EDM-1K generates training data during sampling, it differs from EMP-1K (the empirical optimal denoiser) in its per \sigma -level denoising behavior on test data. This three-regime pattern motivates a closer examination of memorization for the three denoisers. In particular, instead of only considering the classic notion of memorization, we should also consider memorization risk at each noise level . While the MSE gap between train and test data may be a good metric for generalization, it is not directly capturing memorization. To resolve this issue, we propose the following metrics for evaluating memorization. Trajectory-Level memorization. For a denoiser m^{\bullet}_{\sigma} with noise schedule \sigma\in[\sigma_{\text{min}},\sigma_{\text{max}}] , let \Psi^{\bullet}_{\sigma} be the resulting flow map given by solving Equation 4 . Intuitively, the denoiser m^{\bullet}_{\sigma} is said to have memorized the data set \mathcal{D} if its generated samples through the flow map are close to one of the data points in \mathcal{D} . While there could be many ways of quantifying this intuition, we adopt the idea from [ 28 ] , in which they proposed to classify a generated sample as memorized if it is significantly closer to its nearest training neighbor than to the second nearest. Formally, we say \{m^{\bullet}_{\sigma}\}_{\sigma\in[\sigma_{\min},\sigma_{\max}]} is memorizing if for a large portion of noise samples \bm{Z}\sim N(0,I_{d}) , d_{1\mathrm{NN}}(\Psi^{\bullet}_{\sigma_{\text{min}}}(\sigma_{\text{max}}\bm{Z}))8.4 93.0% medium [0.14,8.4] 0.0% small \sigma<0.14 91.0% EDM-50K \to EDM-1K swap: large \sigma>8.4 0.0% medium [0.14,8.4] 92.2% small \sigma<0.14 0.0% Per-Noise-Level memorization. Motivated by the definition above, we identify a finer-grained notion of memorization at each noise level \sigma . Given a training set \mathcal{D}\subset\mathbb{R}^{d} sampled from a population distribution p , a denoiser m^{\bullet}_{\sigma} is said to have memorized \mathcal{D} at a noise level \sigma if for (nearly) all sample test points x_{\text{test}}\sim p and a large portion of noise samples \bm{Z}\sim\mathcal{N}(0,I_{d}) , we have d_{1\mathrm{NN}}(m_{\sigma}^{\bullet}(x_{\text{test}}+\sigma\bm{Z}))1}\frac{d_{K\mathrm{NN}}(x_{1})}{a_{K-1,\delta,q}} , then w_{1}(\bm{X}_{\sigma},\sigma)\leq q. 2. If \sigma\leq\frac{d_{2\mathrm{NN}}(x_{1})}{b_{\delta,q}} , then w_{1}(\bm{X}_{\sigma},\sigma)\geq q. where the constants a_{K,\delta,q} and b_{\delta,q} are defined by \displaystyle a_{K,\delta,q} \displaystyle:=F^{-1}(\delta/K)+\sqrt{\left(F^{-1}(\delta/K)\right)^{2}+2\log\left(\frac{Kq}{1-q}\right)} \displaystyle b_{\delta,q} \displaystyle:=\tilde{F}^{-1}(\delta/N)+\sqrt{\left(\tilde{F}^{-1}(\delta/N)\right)^{2}+2\log\left(\frac{Nq}{1-q}\right)} and where F is the CDF of the standard normal distribution and \tilde{F}:=1-F . (Note that d_{1\mathrm{NN}}(x_{1})=0 .) Remark 4.3 . We consider a dataset consisting of 1K images randomly sampled from the CIFAR-10 dataset. Using the bounds given by Theorem 4.2 , over a random sample of 20 points, the average lower bound needed to guarantee posterior concentration below 0.6 with probability 0.95 was \sigma\geq 22.54 . The average upper bound needed to guarantee posterior concentration above 0.95 with probability 0.95 was \sigma\leq 2.25 . This upper bound compares well with Figure 3 , where in the same setting \sigma=2.25 lies close to where the curve of the average max weight reaches 0.95. Remark 4.4 . In Section A.2 we use a variant of Theorem 4.2 to theoretically explain the sharp alignment between the optimal and conditional vector fields observed in high dimensional flow matching settings by [ 2 ] . Gaussian Shell Coverage. [ 22 ] proposes using Gaussian shells around training data points to model supervised regions during training. Following their intuition, we will use a theoretically tighter bound of Gaussian shells (cf. Lemma 4.5 ) and propose a novel notion of data coverage utilizing the supervised regions via Gaussian shells. The following high dimensional concentration result is key to our analysis in this paper (see proof in Appendix B ). Lemma 4.5 (Gaussian concentration [ 14 ] ) . Let d\geq 2 and fix c>0 . Define r^{\mathrm{in}}_{c,d}:=\sqrt{d-2\sqrt{cd}}, and r^{\mathrm{out}}_{c,d}:=\sqrt{d+2\sqrt{cd}+2c} . Let \bm{Z}\sim\mathcal{N}(0,I_{d}) . Then, \mathbb{P}\big(\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\big)\geq 1-2e^{-c}. This result states that Z\sim\mathcal{N}(0,I_{d}) will concentrate in a thin shell around the origin with radius \approx\sqrt{d} and thickness controlled by c . If we let c=5 , then the concentration probability is at least 0.9865. So we will simply let c=5 through this paper unless otherwise stated. Based on the concentration result, for any x\in\mathbb{R}^{d} and \sigma>0 , we define the Gaussian shell centered at x by S_{\sigma}({x}):=\Bigl\{x^{\prime}\in\mathbb{R}^{d}:\sigma r^{\mathrm{in}}_{c,d}\leq\|x^{\prime}-x\|\leq\sigma r^{\mathrm{out}}_{c,d}\Bigr\}. (6) As a direct consequence of Lemma 4.5 , we have that \mathbb{P}\big(\bm{X}_{\sigma}\in S_{\sigma}({x})\mid\bm{X}=x\big)\geq\ 1-2e^{-c}. This shows that a single Gaussian shell provides a high-probability description of where a noisy data point lies. Based on the notion of Gaussian shells, we next introduce the notion of data coverage. Definition 4.6 (Gaussian Shell Coverage) . Suppose \mathcal{D}=\{x_{i}\}_{i=1}^{N} is a finite dataset sampled from the data distribution p . We define the Gaussian shell coverage of the distribution p relative to \mathcal{D} at the noise level \sigma as follows: C_{\sigma}(p,\mathcal{D}):=\mathbb{P}\left(\bm{X}_{\sigma}\in\bigcup_{i=1}^{N}S_{\sigma}({x_{i}})\right),\quad\text{where }\bm{X}\sim p The quantity approximately quantifies the likelihood that a diffusion model encounters a noisy test x_{\text{test}}+\sigma z during training. In other words, C_{\sigma} measures how large a portion of the underlying ground-truth distribution is covered by the training set after the convolution p_{\sigma}=p*\mathcal{N}(0,\sigma^{2}I) . For brevity, in the remainder of the paper we will refer to the Gaussian shell coverage C_{\sigma}(p,\mathcal{D}) simply as “coverage” when no confusion can arise. Note that while in applications the data distribution p is not known, this quantity can be empirically estimated if a test dataset is available; see Figure 3 where we estimate C_{\sigma} for the CIFAR-10 dataset. Figure 3 also exhibits an interesting sharp phase transition for the coverage as the noise level varies. To provide a theoretical understanding why this happens, we find it better to treat the dataset as random and study the coverage in expectation. Interestingly, it turns out that controlling this coverage is closely related to the probability mass of intersection between just two shells. Below we provide a clean description of such a probability mass. Definition 4.7 . Let e_{1}=[1,0,\cdots,0]^{T}\in\mathbb{R}^{d} and define \Phi_{d,c}\left(t\right):=\mathbb{P}\!\left(\|\bm{Z}\|,\left\|\bm{Z}+te_{1}\right\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\right). The function \Phi_{d,c}(t) measures the probability of a Gaussian noisy observation for one data point lying also in the Gaussian shell over a data point at distance t and hence roughly represents the “probability of seeing a noisy data in the intersection of two Gaussian shells”. The following result shows that the coverage is closely controlled by \Phi_{d,c} . Theorem 4.8 . Let \bm{X},\bm{X}_{1},\dots,\bm{X}_{N}\overset{\text{i.i.d.}}{\sim}p , and let \bm{Z}\sim\mathcal{N}(0,I_{d}) be independent of (\bm{X},\bm{X}_{1},\dots,\bm{X}_{N}) . Fix a noise level \sigma>0 . Define the union of shells \mathcal{U}_{\sigma}:=\bigcup_{i=1}^{N}S_{\sigma}({\bm{X}_{i}}). Then the coverage probability satisfies \displaystyle\mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}) \displaystyle\geq\mathbb{E}\!\left[\Phi_{d,c}\!\left(\frac{d_{1\mathrm{NN}}(\bm{X})}{\sigma}\right)\right], \displaystyle\mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}) \displaystyle\leq 2e^{-c}+N\,\mathbb{E}\!\left[\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}^{\prime}\|}{\sigma}\right)\right] where \bm{X}^{\prime} is an independent copy of \bm{X} . The two bounds in the theorem are governed by the distribution of pairwise distances in the data as well as the ambient dimension d reflected by \Phi_{d,c} . The lower bound actually provides a very accurate estimation empirically and one may directly use it as a proxy to estimate the coverage; see Figure 6 for experimental results on CIFAR-10. 4.2 Characterization of the Noise Regimes With the two quantities of posterior weight and data coverage defined and explored, we are ready to characterize the noise regimes. In Figure 3 we plot two curves of the two quantities against noise levels, based on a 1k subset of the CIFAR-10 dataset. Note that in Figure 2 , EDM-1K exhibits a positive memorization ratio for noise levels range \sigma\in[0.6,12] , which closely aligns with the transition interval observed in Figure 3 . Based on all these observations, we now informally identify the three noise regimes as follows using the two quantities: Figure 3 : Gaussian shell coverage and max posterior weight on CIFAR-10. We plot empirical estimates of the Gaussian shell coverage C_{\sigma}(p,\mathcal{D}) and the max posterior weight W_{\sigma}(\mathcal{D}) as functions of the noise level \sigma , using a 1\mathrm{k} CIFAR-10 training subset and 1\mathrm{k} held-out test images. Regime Posterior weight Shell coverage Small noise High Low Medium noise Phase transition Phase transition -Danger zone- High High Large noise Low High While informal this description may serve as a guiding principal for classifying training dynamics across a full noise schedule. Precise estimation of noise ranges requires further work, but our Theorem 4.2 and Theorem 4.8 serve as a starting point. In what follows, we provide rationales for this characterization and further analysis within each regime. 4.2.1 Small Noise Regime The small noise (small \sigma ) regime is characterized by high posterior weight and low coverage. This is where the empirical optimal denoiser behaves almost like a nearest neighbor map. If a trained model well approximates the empirical optimal one, it will absolutely produce memorized training data and hence it is natural to suggest mitigation for memorization in this region [ 24 , 1 ] . However, although the empirical optimal denoiser exhibits strong memorization behavior (which is almost locally constant), the trained denoiser is unlikely to fit the empirical optimal denoiser since the supervision region during training, described by shell coverage, is very limited. We have empirically observed in Figure 2 that the small noise regime is quite safe from memorization risk in per-noise-level: while the empirical optimal denoiser exhibits nearly 100% memorization for \sigma<0.3 , the learned model ( EDM-1K ) shows minimal memorization in this regime. We also validate in the swapping experiment in Table 1 that the small noise regime in a trained diffusion model does not contribute to memorization. Within the small noise regime, when \sigma is small enough (e.g., <0.2 for the CIFAR-10), all Gaussian shells around data points will be disjoint. In this setting, we have the following description of the optimal solution to the training objective. See Theorem B.6 for the complete statement and proof. Theorem 4.9 (Informal theorem) . Suppose the training is only restricted to the union of shells \bigcup_{i=1}^{N}S_{\sigma}({x_{i}}) , which are also pairwise disjoint for small \sigma . Then, there are infinitely many global minimizers of the denoising objective sending S_{\sigma}({x_{i}}) to x_{i} for i=1,\ldots,N . In the small- \sigma regime, training supervision is concentrated within the small region consisting of the union of Gaussian shells. Then by Theorem 4.9 , the training objective alone does not constrain the denoiser outside this small region, so extrapolation behavior is largely determined by the model’s inductive bias. In one dimension, shallow network denoisers have been shown to interpolate the data with piecewise linear behavior [ 29 ] . In high dimensional and deep network settings, this behavior may not hold. Below we try to provide some evidence that certain locality inductive bias will appear for image datasets. For an image data such as CIFAR-10, we hypothesize that the locality increases when \sigma decreases: each output pixel depends mainly on a local patch. The task therefore reduces to patch denoising in a much lower-dimensional subspace, where the effective training data, the extracted patches, are plentiful, making the problem easier. This aligns with empirical findings in [ 11 , 17 , 18 ] : trained neural denoisers exhibit shrinking receptive fields as the noise level decreases. [ 11 , 17 , 26 ] finds the optimal denoiser under locality bias and [ 17 ] connects the localization to the data statistics. However, it was not clear what affects the spatial decay rate of the sensitivity. To study this analytically, we model the data as a Gaussian with circulant covariance—a natural idealization, since natural images exhibit approximately translation-invariant second-order statistics and stationary Gaussian models are a standard tool in their analysis [ 7 , 20 ] . We propose the following theorem; formal statements and proofs are in Appendix D . Theorem 4.10 (Informal theorem) . For a cyclic stationary Gaussian data distribution, at any noise level \sigma , the sensitivity between two pixels is bounded by \frac{T_{\sigma}}{m} , where m is their wrap-around distance and T_{\sigma} is a constant depending only on the data spectrum and \sigma . A more concentrated spectrum leads to faster decay of the sensitivity. 4.2.2 Large Noise Regime We characterize the large noise regime by low posterior weight and high coverage. The latter implies that the model m_{\sigma}^{\theta} can learn the empirical optimal on a region that covers most of the mass of p_{\sigma} , where inference takes place. In this regime, the empirical optimal denoiser does not exhibit per-noise-level memorization behavior, and hence even if the trained denoiser closely approximates the empirical optimal denoiser, per-noise-level memorization risk remains low. In fact, we can provide some more refined description of the empirical optimal denoiser within this regime. It is known that when \sigma is large, the denoiser m_{\sigma} will be close to the mean \mu of the data distribution p [ 24 ] . Empirically, it has also been observed that m_{\sigma} behaves like a linear map when \sigma is large [ 15 ] . In what follows, we reconcile these two observations by establishing the limiting behavior of m_{\sigma} when \sigma is large. Let \Sigma denote the covariance of p , and recall that \mu is the mean of p . Consider now the Gaussian distribution \mathcal{N}(\mu,\Sigma) that matches the mean and covariance of p . The denoiser m_{\sigma}^{G} of this Gaussian distribution is linear and has the closed form [ 15 ] : m_{\sigma}^{G}(x)=\mu+\Sigma(\Sigma+\sigma^{2}I)^{-1}(x-\mu). (7) Note that when \sigma is large, the linear term will be small and hence m_{\sigma}^{G}\approx\mu . It turns out that this optimal linear denoiser for Gaussian captures the limiting behavior of m_{\sigma} when \sigma is large (see Appendix C for a proof). Theorem 4.11 . Assume \mathrm{supp}(p)\subset B_{R}(0) , where B_{R}(0) is the open ball of radius R centered at the origin. Then, for any \delta\in(0,1) there exists a bounded continuous function H on \overline{B_{R}(0)}\times[-\delta,\delta] such that m_{\sigma}(x)=m_{\sigma}^{G}(x)+H(x,\sigma)(1+\sigma)^{-2}. (8) This theorem indicates that for large \sigma , the denoiser (the one either for the true data distribution p or for the empirical distribution over a dataset) is close to being a linear map determined by the mean and covariance of p . This validates the empirical observations in [ 15 ] . Figure 4 : Comparison of anti-memorization methods. Training curves showing FID score (solid lines) and memorization ratio (dashed lines) for three methods: 2k-Baseline, 2k-Dummy, and 2k-Gap. Gap training achieves the lowest memorization (0.7%) and best FID (2.35) at final checkpoint. 4.2.3 Medium Noise Regime We characterize the medium noise regime by simultaneous transitions in posterior weight concentration and data coverage: as sigma decreases from large to small, posterior weights concentrate on nearest neighbors while data coverage becomes incomplete. The interplay between these two effects creates a volatile training environment. There are two key observations we have regarding the medium noise regime. The first is the discovery of a danger zone for memorization: note that in Figure 3 there is a small region of \sigma in the medium regime where both coverage and posterior weights are high. In this case, the denoiser m_{\sigma}^{\theta} will tend to learn the empirical optimal m_{\sigma} in a region of \mathbb{R}^{d} where p_{\sigma} is concentrated (due to high Gaussian shell coverage) and furthermore, since this empirical optimal m_{\sigma} exhibits high per-noise-level memorization due to the high posterior concentration, m_{\sigma}^{\theta} will likely exhibit high per-noise-level memorization. The other observation is that generalization appears to arise primarily in the medium sigma regime. Although we do not yet have a theoretical explanation for this behavior, our empirical results provide strong evidence: swapping denoisers in this regime flips memorization behavior (Table 1 ), with qualitative examples shown in Figure 10 . This highlights the medium region as the key regime governing both memorization and generalization in diffusion models. More theoretical understanding of this regime is left to future work. In the section below, we provide interesting empirical findings regarding this regime: since the medium sigma regime is the most dangerous for memorization, we conjecture that avoiding it during training can mitigate memorization. We validate this idea experimentally in Section 5 . Figure 5 : Danger zone and gap configurations. The intersection of coverage and max weight curves identifies the danger zone. Memorization rates for different gap configurations are overlaid, showing that gaps targeting this region minimize memorization. 5 Targeted Noise Region Undertraining Our theoretical analysis has identified a “danger zone” in the medium- \sigma regime where memorization risk is highest due to the confluence of high posterior concentration and significant data coverage. A natural question arises: can we mitigate memorization by targeting this specific regime during training? While the swap experiments in Table 1 already provides some positive answer, in this section, we further test a rejection training idea on CIFAR-10 and CelebA. CIFAR-10 car subset. We consider a CIFAR-10 car subset (2k images) where both generalization and memorization occur [ 28 ] . As a baseline, we train standard EDM [ 12 ] with the full noise schedule \sigma\in[0.002,80] ( 2k-Baseline ). Building on the baseline, we consider two anti-memorization methods: 2k-Dummy [ 28 ] adds 2k random noise images as a second class to provide distributional contrast; and 2k-Gap (ours) excludes the danger zone \sigma\in[1.0,5.0] , identified empirically via coverage and posterior weight analysis (Appendix E.4 ). We train for 40k iterations, and evaluate at 8 checkpoints. At each checkpoint, we generate 1,024 samples and compute FID against 5k CIFAR-10 training cars and the memorization ratio. Figure 4 shows training trajectories. The 2k-Baseline exhibits severe memorization (78.8%) with FID 3.72. The 2k-Dummy method substantially reduces memorization to 9.0% with FID 2.58. Our 2k-Gap achieves the best results: 0.7% memorization and FID 2.35. Grayscale CelebA training data. Following [ 3 ] , we use downscaled grayscale CelebA data with 1024 training samples, which diffusion models can memorize. We compute the coverage and max weight curves (Figure 5 ) to identify the danger zone around \sigma\in[0.55,1.3] (approximately DDPM timesteps 125–325). We test multiple timestep gap configurations around this region to study the effect on memorization. We utilize the same DDPM architecture as in [ 3 ] and train for 100k iterations with batch size 512 and the baseline achieves 74.97% memorization. We then test nine timestep gap configurations, removing different ranges within [100–350] (the danger zone plus some buffer) during training. Results are summarized in Figure 5 . Skipping the full range [100–350] significantly reduces memorization from 75% to 9%. High max weight regions are more influential: the gap [100–200] spanning only 100 timesteps reduces memorization to 43%, while [100–275] achieves 13%. Visual comparisons of generated samples and their nearest training neighbors are provided in Appendix E.5 , showing high sample quality and meaningful generalization, which validates the memorization mitigation effect. We also include in Appendix E.5 a Pareto frontier analysis over all gap configurations, showing the trade-off between image quality (FID) and memorization; mid-range gaps (150–250, 167–275) achieve the best balance. 6 Discussion Our results indicate that the trajectory-level memorization in diffusion models arises from a specific medium range of noise levels, rather than from the small or large noise extremes. This per-noise-level analysis provides a fundamentally different perspective for analyzing memorization and generalization. From a practical perspective, this points to targeted control strategies that act selectively on the intermediate noise regime, such as modifying training emphasis, scheduling, or supervision strength at those scales. More broadly, our framework highlights the importance of incorporating data geometry and noise-scale interactions into the design and analysis of diffusion models. Appendix Contents Appendix A Posterior Concentration and Cosine Similarity Concentration A.1 Proof of Theorem 4.2 Proof. Fix \delta\in(0,1) and q\in(\frac{1}{2},1) . Recall that conditioning on \bm{X}=x_{1} gives \bm{X}_{\sigma}:=x_{1}+\sigma\bm{Z} where \bm{Z}\sim N(0,I_{d}) . By definition w_{j}(\bm{X}_{\sigma},\sigma)=\frac{\exp\left(-\frac{\|\bm{X}_{\sigma}-x_{j}\|^{2}}{2\sigma^{2}}\right)}{\sum_{k=1}^{N}\exp\left(-\frac{\|\bm{X}_{\sigma}-x_{k}\|^{2}}{2\sigma^{2}}\right)}. For ease of notation write w_{j}=w_{j}(\bm{X}_{\sigma},\sigma) and notice that since \sum_{j=1}^{N}w_{j}=1 we have w_{1}=\dfrac{1}{1+\sum_{j\neq 1}\frac{w_{j}}{w_{1}}}. (9) Let \hat{d}_{j}:=x_{j}-x_{1} for all x_{j}\in\mathcal{D} . By direct calculation we have \frac{w_{j}}{w_{1}}=\exp\Big(\frac{1}{\sigma}\langle\bm{Z},\hat{d}_{j}\rangle-\frac{1}{2\sigma^{2}}\|\hat{d}_{j}\|^{2}\Big). Define \bm{V}_{j}:=\frac{1}{\sigma}\langle\bm{Z},\hat{d}_{j}\rangle-\frac{1}{2\sigma^{2}}\|\hat{d}_{j}\|^{2} and without loss suppose that for all K\in\{1,\dots,N-1\} , we have \|x_{K+1}-x_{1}\|=d_{K} where d_{K}:=d_{(K+1)NN}(x_{1}) is the distance from x_{1} to its K^{\text{th}} nearest neighbor in \mathcal{D} . (Under this convention d_{1NN}(x_{1})=0 ). Fix a K\in\{2,\dots,N\} and let c_{K}:=\log\left(\frac{1-q}{qK}\right). Note c_{K}<0 since q\in(\frac{1}{2},1) and therefore (1-q)/q\in(0,1) . Define the event \mathcal{E}_{K}:=\bigcap_{j=2}^{K+1}\Big\{\bm{V}_{j}\geq c_{K}\Big\} so that on \mathcal{E}_{K} , \sum_{j\neq 1}\frac{w_{j}}{w_{1}}=\sum_{j\neq 1}e^{\bm{V}_{j}}\geq\sum_{j=2}^{K+1}e^{\bm{V}_{j}}\geq Ke^{c_{K}}=\frac{1-q}{q} which gives w_{1}\leq q via Equation 9 . We now bound the complement event \mathcal{E}_{K}^{c} . By union bound we have \mathbb{P}(\mathcal{E}_{K}^{c})\leq\sum_{j=2}^{K+1}\mathbb{P}(\bm{V}_{j}0 , hence for all j , \frac{\|\hat{d}_{j}\|}{\sigma}\leq a_{K,\delta,q}\quad\implies\quad\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c_{K}}{\|\hat{d}_{j}\|}\leq z. (13) Thus by Equation 12 and Equation 13 we have the following implication: If \sigma\geq\max_{2\leq j\leq K+1}\frac{\|\hat{d}_{j}\|}{a_{K,\delta,q}}=\frac{d_{K}}{a_{K,\delta,q}} then F\Big(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c_{K}}{\|\hat{d}_{j}\|}\Big)\leq\frac{\delta}{K} for all j\in\{2,\dots,K+1\} . By Equation 10 and Equation 11 this further implies \mathbb{P}(\mathcal{E}_{K}^{c})\leq K\cdot\frac{\delta}{K}=\delta. Since this construction holds for all K we have \sigma\geq\min_{K}\frac{d_{K}}{a_{K,\delta,q}}=\min_{K>1}\frac{d_{KNN}(x_{1})}{a_{K-1,\delta,q}} implies for K_{*}:=\arg\min_{K}(d_{K}/a_{K,\delta,q}) , \mathbb{P}(\mathcal{E}_{K_{*}})\geq 1-\delta and on \mathcal{E}_{K_{*}} , w_{1}\leq q. Plugging in the respective definitions for c_{K} and z into a_{K,\delta,q} completes the proof of the first implication. To prove the second implication, let c:=\log\left(\frac{1-q}{q(N-1)}\right) and notice that c<0 . Define the event \mathcal{F}:=\bigcap_{j\neq 1}\Big\{\bm{V}_{j}\leq c\Big\} so that on \mathcal{F} , \sum_{j\neq 1}\frac{w_{j}}{w_{1}}=\sum_{j\neq 1}e^{\bm{V}_{j}}\leq(N-1)e^{c}=\frac{1-q}{q} which gives w_{1}\geq q via Equation 9 . By union bound we have \mathbb{P}(\mathcal{F}^{c})\leq\sum_{j\neq 1}\mathbb{P}(\bm{V}_{j}>c). (14) For any fixed j\neq 1 , observe since \langle\bm{Z},\hat{d}_{j}\rangle\sim N(0,\|\hat{d}_{j}\|^{2}) , \mathbb{P}(\bm{V}_{j}>c)=\mathbb{P}\left(\left\langle\bm{Z},\frac{\hat{d}_{j}}{\|\hat{d}_{j}\|}\right\rangle>\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\right)=\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\right) (15) where \tilde{F}:=1-F . Put \tilde{z}:=\tilde{F}^{-1}(\delta/(N-1)) and note that since both \tilde{F} and \tilde{F}^{-1} are decreasing functions we may write \frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\geq\tilde{z}\iff\tilde{F}\Big(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\Big)\leq\frac{\delta}{N-1}. (16) Then solving the equation \frac{b}{2}+\frac{c}{b}=\tilde{z} yields a positive solution b^{\prime}_{\delta,q}:=\tilde{z}+\sqrt{\tilde{z}^{2}-2c} which is well defined since c<0 . By a similar argument to that used in the proof of the first implication, we have for all j\neq 1 , \frac{\|\hat{d}_{j}\|}{\sigma}\geq b^{\prime}_{\delta,q}\quad\implies\quad\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\geq\tilde{z}. (17) Since d_{1}\leq\|\hat{d}_{j}\| for all j , by Equation 16 and Equation 17 we have the following implication: If \sigma\leq\frac{d_{1}}{b^{\prime}_{\delta,q}} then \tilde{F}\Big(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma c}{\|\hat{d}_{j}\|}\Big)\leq\frac{\delta}{N-1} for all j\neq 1 . By the above union bound in Equation 14 and Equation 15 this further implies \mathbb{P}(\mathcal{F}^{c})\leq\delta. Lastly notice that since \tilde{F}^{-1} is a decreasing function we have \displaystyle b^{\prime}_{\delta,q} \displaystyle=\tilde{F}^{-1}(\delta/(N-1))+\sqrt{\Big(\tilde{F}^{-1}(\delta/(N-1))\Big)^{2}+2\log\Big(\frac{q(N-1)}{1-q}\Big)} \displaystyle\leq\tilde{F}^{-1}(\delta/N)+\sqrt{\Big(\tilde{F}^{-1}(\delta/N)\Big)^{2}+2\log\Big(\frac{qN}{1-q}\Big)}=:b_{\delta,q} which implies that \sigma\leq\frac{d_{1}}{b_{\delta,q}}=\frac{d_{2NN}(x_{1})}{b_{\delta,q}} yields the same implication. This completes the proof. ∎ A.2 Cosine Similarity Concentration The following lemma is similar to Theorem 4.2 with an additional bound on the distance from the denoiser m_{\sigma}(\bm{X}_{\sigma}) to x_{1} given that \bm{X}_{\sigma}=x_{1}+\sigma\bm{Z} . This demonstrates that for small \sigma the empirical optimal denoiser admits nearest neighbor behavior. Furthermore this result leads to Corollary A.2 , where following inspiration from [ 2 ] , we demonstrate a conditional bound on the cosine similarity between the optimal and conditional vector fields in the flow matching setting with linear schedules. Lemma A.1 . Let \mathcal{D}=\{x_{1},\dots,x_{N}\}\subset\mathbb{R}^{d} be a finite data set with uniform distribution p_{\mathcal{D}} . Define D:=\max_{1\leq i,j\leq N}\|x_{i}-x_{j}\|. let \hat{d}_{j}:=x_{j}-x_{1} . Fix \epsilon>0 and condition on \bm{X}=x_{1} , then with probability at least 1-\delta , w_{1}(\bm{X}_{\sigma},\sigma)\geq 1-\epsilon and consequently \|m_{\sigma}(\bm{X}_{\sigma})-x_{1}\|\leq D\left(1-w_{1}(\bm{X}_{\sigma},\sigma)\right)\leq D\epsilon (18) where \delta:=\sum_{j\neq 1}\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right) and \kappa_{\epsilon}:=\log\left(\frac{\epsilon}{(N-1)(1-\epsilon)}\right) and \tilde{F}:=1-F where F is the CDF of the standard normal distribution. Proof. Let \epsilon>0 and recall that \bm{X}=x_{1} gives \bm{X}_{\sigma}=x_{1}+\sigma\bm{Z} . We have w_{j}(\bm{X}_{\sigma},\sigma)=\frac{\exp\left(-\frac{\|\bm{X}_{\sigma}-x_{j}\|^{2}}{2\sigma^{2}}\right)}{\sum_{k=1}^{N}\exp\left(-\frac{\|\bm{X}_{\sigma}-x_{k}\|^{2}}{2\sigma^{2}}\right)}. For ease of notation write w_{j}:=w_{j}(\bm{X}_{\sigma},\sigma) . Since \sum_{j=1}^{N}w_{j}=1 we have w_{1}=\dfrac{1}{1+\sum_{j\neq 1}\frac{w_{j}}{w_{1}}}. (19) By direct calculation \frac{w_{j}}{w_{1}}=\exp\Big(\frac{1}{\sigma}\langle\bm{Z},\hat{d}_{j}\rangle-\frac{1}{2\sigma^{2}}\|\hat{d}_{j}\|^{2}\Big). Define \bm{V}_{j}:=\frac{1}{\sigma}\langle\bm{Z},\hat{d}_{j}\rangle-\frac{1}{2\sigma^{2}}\|\hat{d}_{j}\|^{2} and put \kappa_{\epsilon}:=\log\left(\frac{\epsilon}{(N-1)(1-\epsilon)}\right). Consider the event \mathcal{E}:=\bigcap_{j\neq 1}\left\{\bm{V}_{j}\leq\kappa_{\epsilon}\right\} so that on \mathcal{E} , \sum_{j\neq 1}\frac{w_{j}}{w_{1}}=\sum_{j\neq 1}e^{\bm{V}_{j}}\leq(N-1)e^{\kappa_{\epsilon}}=\frac{\epsilon}{1-\epsilon} which gives w_{1}\geq 1-\epsilon by Equation 19 . Since \left\langle\bm{Z},\hat{d}_{j}\right\rangle\sim\mathcal{N}(0,\|\hat{d}_{j}\|^{2}) we have by union bound \mathbb{P}(\mathcal{E}^{c})\leq\sum_{j\neq 1}\mathbb{P}\left(\left\langle\bm{Z},\frac{\hat{d}_{j}}{\|\hat{d}_{j}\|}\right\rangle>\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right)=\sum_{j\neq 1}\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right)=:\delta. Recall m_{\sigma}(\bm{X}_{\sigma})=\sum_{j=1}^{N}w_{j}(\bm{X}_{\sigma},\sigma)x_{j} so \|m_{\sigma}(\bm{X}_{\sigma})-x_{1}\|=\Big\|\sum_{j=1}^{N}w_{j}(x_{j}-x_{1})\Big\|\leq\sum_{j\neq 1}w_{j}\|x_{j}-x_{1}\|\leq D(1-w_{1}). Thus on \mathcal{E} , \|m_{\sigma}(\bm{X}_{\sigma})-x_{1}\|\leq D\epsilon which completes the proof. ∎ For the following setting let \bm{X}_{t}:=(1-t)\bm{Z}+t\bm{X} where \bm{Z}\sim\mathcal{N}(0,I_{d}) and \bm{X}\sim p_{\mathcal{D}} . Let u_{t} and u_{t}(\cdot\mid x_{1}) be the optimal and conditional vector fields respectively in the flow matching setting [ 16 ] with scheduling functions \alpha_{t}=t and \beta_{t}=1-t . Then conditional vector field is given by u_{t}\left(x\mid x_{1}\right)=\frac{x_{1}-x}{1-t}. (20) while the empirical optimal vector field admits a closed form u_{t}(x)=\sum_{i=1}^{N}w_{i}(x,t)\frac{x_{i}-x}{1-t} (21) where the weights w_{i}(x,t) are given by softmax weights w_{i}(x,t):=\frac{\exp\left(-\frac{\|x-tx_{i}\|^{2}}{2(1-t)^{2}}\right)}{\sum_{j=1}^{N}\exp\left(-\frac{\|x-tx_{j}\|^{2}}{2(1-t)^{2}}\right)}. Following notation from [ 24 ] , the empirical optimal denoiser in t , is then given by m_{t}\left(x\right):=\mathbb{E}\left[\bm{X}\mid\bm{X}_{t}=x\right]=\sum_{i=1}^{N}w_{i}(x,t)x_{i}. (22) Then with \sigma:=(1-t)/t , by direct calculation, the two empirical optimal denoisers have the following relationship: for any x\in\mathbb{R}^{d} and t\in(0,1) , one has the following straightforward observation (which was also implicitly shown in the proof of [ 24 , Proposition 2.2] ): m_{t}(tx)=m_{\sigma}(x) (23) and consequently, for all t\in(0,1) , m_{t}\left(\bm{X}_{t}\right)=m_{\sigma}(\bm{X}_{\sigma}). (24) The following corollary is designed to help explain the empirical result found in figure 1 of [ 2 ] , where in high dimensions sharp concentration of the cosine similarity between the conditional and empirical optimal vector fields was observed with respect to the flow matching setting with linear scheduling. Corollary A.2 . Assume the setting in Lemma A.1 for \mathcal{D}=\{x_{1},\dots,x_{N}\}\subset\mathbb{R}^{d} . Condition on \bm{X}=x_{1} and suppose x_{1}\neq 0 . Pick constants a,c>0 and fix \epsilon>0 . Then with probability at least 1-\delta_{\epsilon,a,c} \dfrac{\langle u_{t}(\bm{X}_{t}),u_{t}(\bm{X}_{t}\mid x_{1})\rangle}{\|u_{t}(\bm{X}_{t})\|\|u_{t}(\bm{X}_{t}\mid x_{1})\|}\geq 1-\dfrac{2D\epsilon}{(1-t)\sqrt{d-2\sqrt{dc}+\|x_{1}\|^{2}-a\|x_{1}\|}+D\epsilon} where \delta_{\epsilon,a,c}:=e^{-c}+\tilde{F}\left(\frac{a}{2}\right)+\sum_{j\neq 1}\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right) and \kappa_{\epsilon}:=\log\left(\frac{\epsilon}{(N-1)(1-\epsilon)}\right) and \tilde{F}:=1-F where F is the CDF of the standard normal distribution. Remark A.3 . The strong dependence on the dimension d should be noted in the bound. For empirical validation we use the CIFAR-10 dataset consisting of 50K images embedded into [-1,1]^{3072} in the standard manner. At t=0.4 ( equivalently \sigma=1.5 ) with \epsilon=0.01 , a=8 , and c=5 , averaging over a random sample of 500 images, Corollary A.2 yields an average lower bound for the cosine similarity of 0.940 with an average probability of 0.941 . Though stronger concentration is seen in figure 1 of [ 2 ] for smaller values of t , this result conveys the effect of high dimension and dataset diameter on alignment between u_{t}(\bm{X}_{t}) and u_{t}(\bm{X}_{t}\mid x_{1}) . Proof. Let \epsilon,a,c>0 . From \bm{X}=x_{1} we have \bm{X}_{t}=(1-t)\bm{Z}+tx_{1} . Plugging \bm{X}_{t} into Equation 20 and Equation 21 and then simplifying yields u_{t}(\bm{X}_{t}\mid x_{1})=x_{1}-\bm{Z}. and u_{t}(\bm{X}_{t})=\frac{m_{t}(\bm{X}_{t})-x_{1}}{1-t}+x_{1}-\bm{Z} Hence after expanding \displaystyle\dfrac{\langle u_{t}(\bm{X}_{t}),u_{t}(\bm{X}_{t}\mid x_{1})\rangle}{\|u_{t}(\bm{X}_{t})\|\|u_{t}(\bm{X}_{t}\mid x_{1})\|} \displaystyle=\dfrac{\|x_{1}-\bm{Z}\|^{2}+\frac{1}{1-t}\left\langle m_{t}(\bm{X}_{t})-x_{1},x_{1}-\bm{Z}\right\rangle}{\|\frac{1}{1-t}(m_{t}(\bm{X}_{t})-x_{1})+x_{1}-\bm{Z}\|\|x_{1}-\bm{Z}\|} \displaystyle\geq\dfrac{\|x_{1}-\bm{Z}\|-\frac{1}{1-t}\|m_{t}(\bm{X}_{t})-x_{1}\|}{\|x_{1}-\bm{Z}\|+\frac{1}{1-t}\|m_{t}(\bm{X}_{t})-x_{1}\|} \displaystyle=1-\dfrac{2\|m_{t}(\bm{X}_{t})-x_{1}\|}{(1-t)\|x_{1}-\bm{Z}\|+\|m_{t}(\bm{X}_{t})-x_{1}\|} (25) Where we used Cauchy–Schwartz and the triangle inequality. Recall x_{1}\neq 0 and define the events \mathcal{F}_{a}:=\left\{\left\langle\bm{Z},\frac{x_{1}}{\|x_{1}\|}\right\rangle\leq\frac{a}{2}\right\} and \mathcal{G}_{c}:=\left\{\|\bm{Z}\|\geq\sqrt{d-2\sqrt{dc}}\right\}. So that on \mathcal{F}_{a}\cap\mathcal{G}_{c} , \|x_{1}-\bm{Z}\|^{2}=\|\bm{Z}\|^{2}+\|x_{1}\|^{2}-2\langle x_{1},\bm{Z}\rangle\geq d-2\sqrt{dc}+\|x_{1}\|^{2}-a\|x_{1}\|. (26) Note since \langle\bm{Z},x_{1}\rangle\sim\mathcal{N}(0,\|x_{1}\|^{2}) we have \mathbb{P}(\mathcal{F}^{c}_{a})=\tilde{F}\left(\frac{a}{2}\right) where \tilde{F}=1-F and F is the CDF of the standard normal. Furthermore by the Laurent-Massart inequality [ 14 ] , we have with regrettable notation \mathbb{P}(\mathcal{G}_{c}^{c})\leq e^{-c}. Let \sigma:=(1-t)/t and so by Equation 24 we have \|m_{t}(\bm{X}_{t})-x_{1}\|=\|m_{\sigma}(\bm{X}_{\sigma})-x_{1}\| (27) where \bm{X}_{\sigma}=x_{1}+\sigma\bm{Z} . By the proof of Lemma A.1 we have \|m_{\sigma}(\bm{X}_{\sigma})-x_{1}\|\leq D\epsilon (28) on an event \mathcal{E} with \mathbb{P}(\mathcal{E}^{c})\leq\sum_{j\neq 1}\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right) where \kappa_{\epsilon}:=\log\left(\frac{\epsilon}{(N-1)(1-\epsilon)}\right). Define the event \mathcal{H}:=\mathcal{E}\cap\mathcal{F}_{a}\cap\mathcal{G}_{c} so that on \mathcal{H} , using the equality in Equation 27 we have by Equation 25 , Equation 26 and Equation 28 , \dfrac{\langle u_{t}(\bm{X}_{t}),u_{t}(\bm{X}_{t}\mid x_{1})\rangle}{\|u_{t}(\bm{X}_{t})\|\|u_{t}(\bm{X}_{t}\mid x_{1})\|}\geq 1-\dfrac{2D\epsilon}{(1-t)\sqrt{d-2\sqrt{dc}+\|x_{1}\|^{2}-a\|x_{1}\|}+D\epsilon} while using union bound and previous complement bounds yields, \mathbb{P}(\mathcal{H}^{c})\leq e^{-c}+\tilde{F}\left(\frac{a}{2}\right)+\sum_{j\neq 1}\tilde{F}\left(\frac{\|\hat{d}_{j}\|}{2\sigma}+\frac{\sigma\kappa_{\epsilon}}{\|\hat{d}_{j}\|}\right):=\delta_{\epsilon,a,c} which completes the proof. ∎ Appendix B Gaussian Shell and Coverage Proof of Lemma 4.5 . We first recall a standard concentration inequality for chi-square random variables due to Laurent and Massart [ 14 ] . Lemma B.1 . Let \bm{R}\sim\chi^{2}_{d} . Then for every c>0 , \displaystyle\mathbb{P}\!\left(\bm{R}-d\geq 2\sqrt{dc}+2c\right) \displaystyle\leq e^{-c}, (29) \displaystyle\mathbb{P}\!\left(d-\bm{R}\geq 2\sqrt{dc}\right) \displaystyle\leq e^{-c}. (30) Now we let \bm{S}:=\|\bm{Z}\|^{2}\sim\chi^{2}_{d} . By Lemma B.1 above, for every c\geq 0 , \mathbb{P}\big(\bm{S}\geq d+2\sqrt{cd}+2c\big)\leq e^{-c},\qquad\mathbb{P}\big(\bm{S}\leq d-2\sqrt{cd}\big)\leq e^{-c}. Therefore, \mathbb{P}\Big(d-2\sqrt{cd}\leq\bm{S}\leq d+2\sqrt{cd}+2c\Big)\geq 1-2e^{-c}. Since \big(r^{\mathrm{in}}_{c,d}\big)^{2}=d-2\sqrt{cd} and \big(r^{\mathrm{out}}_{c,d}\big)^{2}=d+2\sqrt{cd}+2c , taking square roots yields \mathbb{P}\big(\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\big)\geq 1-2e^{-c}, which proves the lemma. ∎ B.1 Proof of Theorem 4.8 We first introduce some terminology to assist with the proof. Let (\Omega,\mathcal{F},\mathbb{P}) be a probability space supporting random variables \bm{X},\bm{X}^{\prime},\bm{X}_{1},\dots,\bm{X}_{N}:\Omega\to\mathbb{R}^{d},\qquad\bm{Z}:\Omega\to\mathbb{R}^{d}, such that \bm{X},\bm{X}^{\prime},\bm{X}_{1},\dots,\bm{X}_{N} are i.i.d. with law p , and \bm{Z}\sim\mathcal{N}(0,I_{d}) is independent. We first recall the following result: Lemma B.2 (Example 4.1.7 in [ 5 ] ) . Let \bm{A} and \bm{B} be independent random variables and let \varphi be an integrable function such that \mathbb{E}[|\varphi(\bm{A},\bm{B})|]<\infty . Define g(a):=\mathbb{E}[\varphi(a,\bm{B})]. Then \mathbb{E}[\varphi(\bm{A},\bm{B})\mid\bm{A}]=g(\bm{A})\qquad\text{a.s.} Lemma B.3 . For each i=1,\dots,N , define I_{i}:=\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}})\}}. Then: (i) \mathbb{E}[I_{i}\mid\bm{X}_{\sigma}]=q_{\sigma}(\bm{X}_{\sigma})\qquad\text{a.s.,} where q_{\sigma}(y)=\mathbb{P}\bigl(y\in S_{\sigma}({\bm{X}^{\prime}})\bigr), and \bm{X}^{\prime} is an independent copy of \bm{X}_{1} . (ii) The family (I_{1},\dots,I_{N}) is conditionally independent given \bm{X}_{\sigma} . Consequently, conditional on \bm{X}_{\sigma} , the random variables I_{1},\dots,I_{N} are i.i.d. Bernoulli with parameter q_{\sigma}(\bm{X}_{\sigma}) . Proof. Since (\bm{X}_{1},\dots,\bm{X}_{N}) is independent of (\bm{X},\bm{Z}) and \bm{X}_{\sigma}=\bm{X}+\sigma\bm{Z} is measurable with respect to (\bm{X},\bm{Z}) , it follows that (\bm{X}_{1},\dots,\bm{X}_{N}) is independent of \bm{X}_{\sigma} . (i) Define \varphi(y,x):=\mathbf{1}_{\{y\in S_{\sigma}({x})\}} , I_{i}=\varphi(\bm{X}_{\sigma},\bm{X}_{i}) . Since \bm{X}_{\sigma} and \bm{X}_{i} are independent and \varphi is bounded, Lemma B.2 yields \mathbb{E}[I_{i}\mid\bm{X}_{\sigma}]=\mathbb{E}[\varphi(\bm{X}_{\sigma},\bm{X}_{i})\mid\bm{X}_{\sigma}]=g(\bm{X}_{\sigma}), where g(y)=\mathbb{E}[\varphi(y,\bm{X}_{i})]=\mathbb{P}\bigl(y\in S_{\sigma}({\bm{X}_{i}})\bigr) . Since \bm{X}_{i}\stackrel{{\scriptstyle d}}{{=}}\bm{X}^{\prime} , we have g(y)=q_{\sigma}(y) for all y . Hence \mathbb{E}[I_{i}\mid\bm{X}_{\sigma}]=q_{\sigma}(\bm{X}_{\sigma})\qquad\text{a.s.} (ii) Let J\subset\{1,\dots,N\} be finite and nonempty, and fix \varepsilon_{j}\in\{0,1\} for each j\in J . Define the measurable function \Phi:\ \mathbb{R}^{d}\times(\mathbb{R}^{d})^{J}\to\{0,1\},\qquad\Phi\bigl(y,(x_{j})_{j\in J}\bigr):=\prod_{j\in J}\mathbf{1}_{\{\mathbf{1}_{\{y\in S_{\sigma}({x_{j}})\}}=\varepsilon_{j}\}}. Then \mathbf{1}_{\cap_{j\in J}\{I_{j}=\varepsilon_{j}\}}=\Phi\bigl(\bm{X}_{\sigma},(\bm{X}_{j})_{j\in J}\bigr). Since \bm{X}_{\sigma} is independent of (\bm{X}_{j})_{j\in J} and \Phi is bounded, Lemma B.2 yields \mathbb{P}\!\left(\bigcap_{j\in J}\{I_{j}=\varepsilon_{j}\}\;\middle|\;\bm{X}_{\sigma}\right)=\mathbb{E}\!\left[\Phi\bigl(\bm{X}_{\sigma},(\bm{X}_{j})_{j\in J}\bigr)\,\middle|\,\bm{X}_{\sigma}\right]=G(\bm{X}_{\sigma}), where G(y)=\mathbb{E}\!\left[\Phi\bigl(y,(\bm{X}_{j})_{j\in J}\bigr)\right]. Using independence of (\bm{X}_{j})_{j\in J} , we factorize \displaystyle G(y) \displaystyle=\mathbb{E}\!\left[\prod_{j\in J}\mathbf{1}_{\{\mathbf{1}_{\{y\in S_{\sigma}({\bm{X}_{j}})\}}=\varepsilon_{j}\}}\right]=\prod_{j\in J}\mathbb{E}\!\left[\mathbf{1}_{\{\mathbf{1}_{\{y\in S_{\sigma}({\bm{X}_{j}})\}}=\varepsilon_{j}\}}\right] \displaystyle=\prod_{j\in J}\mathbb{P}\!\left(\mathbf{1}_{\{y\in S_{\sigma}({\bm{X}_{j}})\}}=\varepsilon_{j}\right)=\prod_{j\in J}\mathbb{P}\!\left(I_{j}=\varepsilon_{j}\,\middle|\,\bm{X}_{\sigma}=y\right). Hence \mathbb{P}\!\left(\bigcap_{j\in J}\{I_{j}=\varepsilon_{j}\}\;\middle|\;\bm{X}_{\sigma}\right)=\prod_{j\in J}\mathbb{P}\!\left(I_{j}=\varepsilon_{j}\,\middle|\,\bm{X}_{\sigma}\right), which proves that (I_{1},\dots,I_{N}) is conditionally independent given \bm{X}_{\sigma} . Finally, by part (i), \mathbb{P}(I_{i}=1\mid\bm{X}_{\sigma})=\mathbb{E}[I_{i}\mid\bm{X}_{\sigma}]=q_{\sigma}(\bm{X}_{\sigma}), so (I_{1},\dots,I_{N}) are conditionally i.i.d. Bernoulli with parameter q_{\sigma}(\bm{X}_{\sigma}) . ∎ Lemma B.4 . Fix \sigma>0 . For any deterministic x,x^{\prime}\in\mathbb{R}^{d} , define \bm{Y}_{x}=x+\sigma\bm{Z} with \bm{Z}\sim\mathcal{N}(0,I_{d}) . Then \mathbb{P}\big(\bm{Y}_{x}\in S_{\sigma}({x})\cap S_{\sigma}({x^{\prime}})\big)=\Phi_{d,c}\!\left(\frac{\|x-x^{\prime}\|}{\sigma}\right). Proof. Let \Delta:=x-x^{\prime} and t:=\|\Delta\|/\sigma . Note that \bm{Y}_{x}\in S_{\sigma}({x})\iff\|\bm{Y}_{x}-x\|\in[\sigma r^{\mathrm{in}}_{c,d},\sigma r^{\mathrm{out}}_{c,d}]\iff\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}], and \bm{Y}_{x}\in S_{\sigma}({x^{\prime}})\iff\|\bm{Y}_{x}-x^{\prime}\|\in[\sigma r^{\mathrm{in}}_{c,d},\sigma r^{\mathrm{out}}_{c,d}]\iff\|\bm{Z}+\Delta/\sigma\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]. Choose an orthogonal matrix Q\in O(d) such that Q(\Delta/\sigma)=te_{1} . Since \bm{Z}\sim\mathcal{N}(0,I_{d}) is rotation invariant, Q\bm{Z}\stackrel{{\scriptstyle d}}{{=}}\bm{Z} . Therefore, \displaystyle\mathbb{P}\Big(\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}],\ \|\bm{Z}+\Delta/\sigma\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\Big) \displaystyle=\mathbb{P}\Big(\|Q\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}],\ \|Q\bm{Z}+Q(\Delta/\sigma)\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\Big) \displaystyle=\mathbb{P}\Big(\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}],\ \|\bm{Z}+te_{1}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\Big)=\Phi_{d,c}(t). ∎ Lemma B.5 . Let (\Omega,\mathcal{F},\mathbb{P}) be a probability space supporting random vectors \bm{X},\bm{X}^{\prime}:\Omega\to\mathbb{R}^{d} and \bm{Z}:\Omega\to\mathbb{R}^{d} . Assume \bm{Z}\sim\mathcal{N}(0,I_{d}) and that \bm{Z} is independent of (\bm{X},\bm{X}^{\prime}) . Define the noisy point \bm{X}_{\sigma}:=\bm{X}+\sigma\bm{Z} and the indicator U:=\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}^{\prime}})\}}=\mathbf{1}_{\{\bm{X}+\sigma\bm{Z}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}^{\prime}})\}}. Then \mathbb{E}\big[U\mid\bm{X},\bm{X}^{\prime}\big]=\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}^{\prime}\|}{\sigma}\right)\quad\text{a.s.} Proof. Define \Psi(x,x^{\prime},z):=\mathbf{1}_{\{x+\sigma z\in S_{\sigma}({x})\cap S_{\sigma}({x^{\prime}})\}},\qquad(x,x^{\prime},z)\in\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{d}. Then U=\Psi(\bm{X},\bm{X}^{\prime},\bm{Z}) . For fixed (x,x^{\prime}) , by independence and the definition of S_{\sigma}({x}) , \mathbb{E}_{Z}[\Psi(x,x^{\prime},\bm{Z})]=\mathbb{P}\!\left(\|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}],\,\left\|\bm{Z}+\tfrac{x-x^{\prime}}{\sigma}\right\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}]\right). By Lemma B.4 , this probability depends only on \|x-x^{\prime}\|/\sigma , and equals \Phi_{d,c}\!\left(\frac{\|x-x^{\prime}\|}{\sigma}\right). Since \bm{Z} is independent of (\bm{X},\bm{X}^{\prime}) , we conclude by Lemma B.2 again that \mathbb{E}[U\mid\bm{X},\bm{X}^{\prime}]=\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}^{\prime}\|}{\sigma}\right)\quad\text{a.s.} ∎ Now we are ready to prove our theorem. Proof of Theorem 4.8 . By definition, \mathbf{1}_{\{\bm{X}_{\sigma}\notin\mathcal{U}_{\sigma}\}}=\prod_{i=1}^{N}\mathbf{1}_{\{\bm{X}_{\sigma}\notin S_{\sigma}({\bm{X}_{i}})\}}=\prod_{i=1}^{N}(1-I_{i}),\qquad I_{i}:=\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}})\}}. Using Lemma B.3 , we obtain almost surely \mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\notin\mathcal{U}_{\sigma}\}}\mid\bm{X}_{\sigma}\right]=\prod_{i=1}^{N}\mathbb{E}\!\left[1-I_{i}\mid\bm{X}_{\sigma}\right]=\prod_{i=1}^{N}(1-q_{\sigma}(\bm{X}_{\sigma}))=(1-q_{\sigma}(\bm{X}_{\sigma}))^{N}. Therefore, \mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\mid\bm{X}_{\sigma}\right]=1-(1-q_{\sigma}(\bm{X}_{\sigma}))^{N}. Applying the tower property of conditional expectation yields \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma})=\mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\right]=\mathbb{E}\!\left[\mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\mid\bm{X}_{\sigma}\right]\right]=\mathbb{E}\!\left[1-(1-q_{\sigma}(\bm{X}_{\sigma}))^{N}\right]. Now, we define the nearest-neighbor index i^{\star}(\omega):=\min\Big\{i:\ \|\bm{X}(\omega)-\bm{X}_{i}(\omega)\|=d_{1\mathrm{NN}}(\bm{X}(\omega))\Big\}, which is measurable by construction, and note that S_{\sigma}({\bm{X}_{i^{\star}}})\subseteq\mathcal{U}_{\sigma} pointwise. Hence, \mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\geq\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}_{i^{\star}}})}\}. Taking expectations and using the definition of probability as expectation, we obtain \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma})\geq\mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}_{i^{\star}}})}\}\right]. Now consider the conditional expectation with respect to \bm{X},\bm{X}_{i^{\star}} . By Lemma B.5 , \mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}_{i^{\star}}})}\}\;\middle|\;\bm{X},\bm{X}_{i^{\star}}\right]=\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}_{i^{\star}}\|}{\sigma}\right)=\Phi_{d,c}\!\left(\frac{d_{1\mathrm{NN}}(\bm{X})}{\sigma}\right). Applying the tower property once more gives \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma})\geq\mathbb{E}\!\left[\Phi_{d,c}\!\left(\frac{d_{1\mathrm{NN}}(\bm{X})}{\sigma}\right)\right], which establishes the lower bound. Write \mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\leq\mathbf{1}_{E^{c}}+\mathbf{1}_{E}\mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}},\qquad E:=\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\}. Taking expectations yields \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma})\leq\mathbb{P}(E^{c})+\mathbb{P}(E\cap\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}). By Lemma 4.5 , \mathbb{P}(E^{c})=\mathbb{P}(\|\bm{Z}\|\notin[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}])\leq 2e^{-c}. On the event E , if \bm{X}_{\sigma}\in\mathcal{U}_{\sigma} then \bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}}) for at least one i . Thus, \mathbf{1}_{E}\mathbf{1}_{\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\}}\leq\sum_{i=1}^{N}\mathbf{1}_{E}\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}})}\}. Taking expectations and using linearity, we have that \mathbb{P}(E\cap\{\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}\})\leq\sum_{i=1}^{N}\mathbb{P}(E\cap\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}})\}). Fix i . By Lemma B.5 and the tower property, \mathbb{P}(E\cap\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}_{i}})\})=\mathbb{E}\!\left[\mathbb{E}\!\left[\mathbf{1}_{\{\bm{X}_{\sigma}\in S_{\sigma}({\bm{X}})\cap S_{\sigma}({\bm{X}_{i}})}\}\;\middle|\;\bm{X},\bm{X}_{i}\right]\right]=\mathbb{E}\!\left[\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}_{i}\|}{\sigma}\right)\right]. Since \bm{X}_{i} is independent of \bm{X} and \bm{X}_{i}\stackrel{{\scriptstyle d}}{{=}}\bm{X}^{\prime} , this equals \mathbb{E}[\Phi_{d,c}(\|\bm{X}-\bm{X}^{\prime}\|/\sigma)] . Summing over i and adding the bound on \mathbb{P}(E^{c}) yields \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma})\leq 2e^{-c}+N\,\mathbb{E}\!\left[\Phi_{d,c}\!\left(\frac{\|\bm{X}-\bm{X}^{\prime}\|}{\sigma}\right)\right], which completes the proof. ∎ B.2 Proof of Theorem 4.9 We formalize that, when training is effectively restricted to disjoint Gaussian shells, the denoising objective is highly non-identifiable and admits infinitely many global optima. To isolate what the objective does (and does not) constrain on each shell, we idealize the Gaussian corruption by replacing the radially concentrated law of Z with a uniform distribution on the corresponding unit shell \mathcal{S}:=\{z\in\mathbb{R}^{d}:\ r^{\mathrm{in}}_{c,d}\leq\|z\|\leq r^{\mathrm{out}}_{c,d}\} . Theorem B.6 . Fix \sigma>0 and a finite dataset \mathcal{D}=\{x_{1},\dots,x_{N}\}\subset\mathbb{R}^{d} . Let S_{\sigma}({x_{i}}) be the Gaussian shells from Section 4.1 , and assume they are pairwise disjoint: S_{\sigma}({x_{i}})\cap S_{\sigma}({x_{j}})=\emptyset\qquad\text{for all }i\neq j. Let \bm{Z}\sim\mathrm{Unif}(\mathcal{S}) , where \mathcal{S}:=\{z\in\mathbb{R}^{d}:\ r^{\mathrm{in}}_{c,d}\leq\|z\|\leq r^{\mathrm{out}}_{c,d}\} , and consider the shell-only objective \mathcal{L}_{\sigma}(m):=\mathbb{E}_{\begin{subarray}{c}i\sim\mathrm{Unif}([N])\\ \bm{Z}\sim\mathrm{Unif}(\mathcal{S})\end{subarray}}\Bigl[\bigl\|m(x_{i}+\sigma\bm{Z})-x_{i}\bigr\|_{2}^{2}\Bigr] over measurable m:\mathbb{R}^{d}\to\mathbb{R}^{d} . Then, there are infinitely many global minimizers: any m such that m(y)=x_{i}\quad\text{for all }y\in S_{\sigma}({x_{i}}),\ \ i=1,\dots,N is a minimizer, and m can be defined arbitrarily on \mathbb{R}^{d}\setminus\bigcup_{i=1}^{N}S_{\sigma}({x_{i}}) . Proof. We first note that \mathcal{L}_{\sigma}(m)\geq 0 for every measurable m . Let m be any measurable map satisfying m(y)=x_{i}\quad\text{for all }y\in S_{\sigma}({x_{i}}),\ \ i=1,\dots,N. Then for every i\in[N] and every \bm{Z} with \|\bm{Z}\|\in[r^{\mathrm{in}}_{c,d},r^{\mathrm{out}}_{c,d}] , we again have x_{i}+\sigma\bm{Z}\in S_{\sigma}({x_{i}}) and therefore m(x_{i}+\sigma\bm{Z})=x_{i},\qquad\|m(x_{i}+\sigma\bm{Z})-x_{i}\|_{2}^{2}=0. Taking expectation shows \mathcal{L}_{\sigma}(m)=0 , hence every such m is a global minimizer. ∎ Appendix C Denoiser Behavior at Large Noise Levels In this section, we prove Theorem 4.11 . Instead of working with the \sigma parameter directly, we first work with a parameter t\in[0,1] from flow matching as already mentioned in Section A.2 . We use \bm{X}\sim p to denote a random variable. Since we have assumed p to have bounded support, we further assume that there is R>0 such that \mathrm{supp}(p)\subset B_{R}(0) . We next rewrite the denoiser m_{t} (cf. Equation 22 ) as follows. For x\in\mathbb{R}^{d} and t\in(-1,1) , define W_{t}(y;x)\;:=\;\exp\!\Big(-\frac{\|x-ty\|^{2}}{2(1-t)^{2}}\Big),\qquad D_{t}(x)\;:=\;\mathbb{E}[W_{t}(\bm{X};x)],\qquad N_{t}(x)\;:=\;\mathbb{E}[\bm{X}\,W_{t}(\bm{X};x)]. (31) Then, we have that for any t\in[0,1) m_{t}(x)\;=\mathbb{E}[\bm{X}\mid\bm{X}_{t}=x]=\;\frac{N_{t}(x)}{D_{t}(x)}. (32) Note that although \mathbb{E}[\bm{X}\mid\bm{X}_{t}=x] is only defined for t\in[0,1) , the formula N_{t}(x)/D_{t}(x) is well-defined for all t\in(-1,1) , and we will work with this extension of m_{t} in the proof. Note that 00 . Since |t|\leq\delta<1 , the factors (1-t)^{-m} are uniformly bounded. Hence there exists C>0 such that \sup_{\begin{subarray}{c}\|x\|\leq a\\ |t|\leq\delta\\ \|y\|\leq R\end{subarray}}|\partial_{t}^{k}W_{t}(y;x)|\leq C. Because \|\bm{X}\|\leq R a.s., we obtain |\partial_{t}^{k}W_{t}(\bm{X};x)|\leq C,\qquad\|\bm{X}\,\partial_{t}^{k}W_{t}(\bm{X};x)\|\leq RC\quad\text{a.s.} By dominated convergence, \partial_{t}^{k}D_{t}(x)=\mathbb{E}[\partial_{t}^{k}W_{t}(\bm{X};x)],\qquad\partial_{t}^{k}N_{t}(x)=\mathbb{E}[\bm{X}\,\partial_{t}^{k}W_{t}(\bm{X};x)], and these depend continuously on (x,t) . We have m_{t}(x)=N_{t}(x)/D_{t}(x) and D_{t}(x)>0 . Since N_{t} and D_{t} are C^{2} in t with continuous derivatives. The quotient rule gives the claim that \partial_{t}^{2}m_{t}(x) is continuous on \mathbb{R}^{d}\times[-\delta,\delta] . ∎ Now let \mu:=\mathbb{E}[\bm{X}] and \Sigma:=\mathrm{Cov}(\bm{X}) . Let G=\mathcal{N}(\mu,\Sigma) and define m_{t}^{G}(x) by the same formula Equation 31 – Equation 32 but with expectation taken under G . Recall from Equation 7 that in the Gaussian case Y\sim\mathcal{N}(\mu,\Sigma) with \Sigma\succeq 0 , we have a closed-form expression for m_{\sigma}^{G}(x) . Using Equation 23 , this yields the following closed form for m_{t}^{G} for every t\in(-1,1) : m_{t}^{G}(x)=\mu+t\,\Sigma\bigl(t^{2}\Sigma+(1-t)^{2}I\bigr)^{-1}(x-t\,\mu). (33) This formula follows from the conditional mean formula for multivariate Gaussian distributions. Then it is not hard to prove the following lemma. Lemma C.2 (Gaussian smoothness of m_{t}^{G} ) . For each fixed x\in\mathbb{R}^{d} , the map t\mapsto m_{t}^{G}(x) is C^{\infty} on (-1,1) . Moreover, for any a>0 and any \delta\in(0,1) , the map (x,t)\mapsto m_{t}^{G}(x) is C^{\infty} on B_{a}(0)\times[-\delta,\delta] . Lemma C.3 . For every fixed x\in\mathbb{R}^{d} , m_{0}(x)=\mu,\qquad\left.\frac{d}{dt}m_{t}(x)\right|_{t=0}=\Sigma x. The same identities hold for m_{t}^{G}(x) (with the same \mu,\Sigma ). Proof. At t=0 , the weight is constant in y : W_{0}(y;x)=\exp(-\|x\|^{2}/2)=:W_{0}(x). Therefore N_{0}(x)=W_{0}(x)\mathbb{E}[\bm{X}]=W_{0}(x)\mu and D_{0}(x)=W_{0}(x) , giving m_{0}(x)=\mu . For the first derivative, differentiate m_{t}=N_{t}/D_{t} m_{t}^{\prime}=\frac{N_{t}^{\prime}}{D_{t}}-\frac{N_{t}}{D_{t}}\frac{D_{t}^{\prime}}{D_{t}}. (34) It suffices to compute N_{0}^{\prime},D_{0}^{\prime} . A direct calculation yields \left.\partial_{t}\log W_{t}(y;x)\right|_{t=0}=-\|x\|^{2}+\langle x,y\rangle,\qquad\left.\partial_{t}W_{t}(y;x)\right|_{t=0}=W_{0}(x)\,(-\|x\|^{2}+\langle x,y\rangle). Hence D_{0}^{\prime}(x)=W_{0}(x)\mathbb{E}[-\|x\|^{2}+\langle x,\bm{X}\rangle],\qquad N_{0}^{\prime}(x)=W_{0}(x)\mathbb{E}\big[\bm{X}(-\|x\|^{2}+\langle x,\bm{X}\rangle)\big]. Substituting into the eq. 34 at t=0 and using m_{0}(x)=\mu gives m_{0}^{\prime}(x)=\mathbb{E}\big[\bm{X}\langle x,\bm{X}\rangle\big]-\mu\,\mathbb{E}[\langle x,\bm{X}\rangle]=\mathbb{E}\big[(\bm{X}-\mu)\langle x,(\bm{X}-\mu)\rangle\big]=\Sigma x. The same computation applies to G=\mathcal{N}(\mu,\Sigma) , so the identities hold for m_{t}^{G} as well. ∎ Theorem C.4 . Assume that \mathrm{supp}(p)\subset B_{R}(0) . Then for all \delta\in(0,1) , there exists a bounded function \tilde{H}:\overline{B_{R}(0)}\times[-\delta,\delta]\to\mathbb{R}^{d} such that for all x\in\overline{B_{R}(0)} and all |t|\leq\delta , m_{t}(x)=m_{t}^{G}(x)+t^{2}\tilde{H}(x,t). Proof. For x\in\overline{B_{R}(0)} , define f_{x}(t):=m_{t}(x)-m_{t}^{G}(x). By Lemma C.1 , the function f_{x} is of class C^{2} in t . Moreover, Lemma C.3 implies that f_{x}(0)=f_{x}^{\prime}(0)=0. Again by Lemma C.1 , the map (x,t)\mapsto f_{x}^{\prime\prime}(t) is continuous on \overline{B_{R}(0)}\times[-\delta,\delta] . Define \tilde{H}(x,t):=\int_{0}^{1}(1-s)\,f_{x}^{\prime\prime}(st)\,ds. Then the map (x,t,s)\mapsto(1-s)f_{x}^{\prime\prime}(st) is continuous on the compact set \overline{B_{R}(0)}\times[-\delta,\delta]\times[0,1], and therefore \tilde{H} is continuous on \overline{B_{R}(0)}\times[-\delta,\delta] . By compactness, \tilde{H} is uniformly bounded on this set. Applying Taylor’s theorem with integral remainder, we obtain for all |t|\leq\delta , f_{x}(t)=t^{2}\int_{0}^{1}(1-s)f_{x}^{\prime\prime}(st)\,ds=t^{2}\tilde{H}(x,t). Consequently, m_{t}(x)=m_{t}^{G}(x)+t^{2}\tilde{H}(x,t), which completes the proof. ∎ Now, we are going to translate the second-order expansion in the small parameter t into an asymptotic expansion in terms of the scale parameter \sigma . Proof of Theorem 4.11 . Let \sigma=\frac{1-t}{t} , then, by Equation 23 we have that m_{\sigma}(x)=m_{t}(tx),\qquad m_{\sigma}^{G}(x)=m_{t}^{G}(tx). By Theorem C.4 , for any |t|\leq\delta and x\in\overline{B_{R}(0)} , m_{t}(x)=m_{t}^{G}(x)+t^{2}\tilde{H}(x,t). Replacing x by tx , we obtain m_{t}(tx)=m_{t}^{G}(tx)+t^{2}\tilde{H}(tx,t). By the relationship between m_{t} and m_{\sigma} , we have m_{\sigma}(x)=m_{\sigma}^{G}(x)+t(\sigma)^{2}\tilde{H}(t(\sigma)x,t(\sigma)). Define H(x,\sigma):=\tilde{H}(t(\sigma)x,t(\sigma)). Since t=\frac{1}{1+\sigma}\to 0 as \sigma\to\infty , there exists \sigma_{0}>0 such that |t(\sigma)|\leq 1 for all \sigma\geq\sigma_{0} . \|tx\|\leq\|x\|\leq R, so tx\in\overline{B_{R}(0)} . Therefore, for \sigma\geq\sigma_{0} , (tx,t) lies in \overline{B_{R}(0)}\times[-\delta,\delta] , where \tilde{H} is bounded. Hence H is bounded on \overline{B_{R}(0)}\times[\sigma_{0},\infty) . Finally, since t=(1+\sigma)^{-1} , then t^{2}=(1+\sigma)^{-2} , which yields the stated form. ∎ Appendix D Sensitivity Localization We analyze the case when the data distribution p is a Gaussian distribution p=\mathcal{N}(\mu,\Sigma) on \mathbb{R}^{d} to establish a spatial sensitivity decay control. Recall from Equation 7 that the optimal denoiser for p is given by the closed-form m_{\sigma}(x)=\mu+\Sigma(\Sigma+\sigma^{2}I)^{-1}(x-\mu). In the following we only consider the case of a single channel. The multivariate case can be handled by the same arguments by treating each channel separately. We think of d=r\times r where r is the resolution and then each dimension represents a single-valued pixel. To quantify how much the denoiser at pixel i depends on observation x_{j} , we study the Jacobian \nabla_{x}m_{\sigma}(x) . By Tweedie’s formula [ 6 ] , we have that \nabla_{x}m_{\sigma}(x)=\frac{1}{\sigma^{2}}\,\mathrm{Cov}(\bm{X}\mid\bm{X}_{\sigma}=x)=\Sigma(\Sigma+\sigma^{2}I)^{-1}. The expression for \nabla_{x}m_{\sigma}(y) was derived in [ 17 ] , who also observed that \nabla_{x}m_{\sigma}(y)\to I as \sigma\to 0 . However, convergence to the identity does not by itself characterize how the off-diagonal entries decay with spatial distance. We now make the spatial decay claim precise under cyclic stationary. Lemma D.1 . For any i\neq j , we have that \left|\frac{\partial[m_{\sigma}]_{i}}{\partial x_{j}}\right|=\sigma^{2}\left|[(\Sigma+\sigma^{2}I)^{-1}]_{ij}\right|. (35) Proof. The identity ( 35 ) follows from the simple identity \Sigma(\Sigma+\sigma^{2}I)^{-1}=I-\sigma^{2}(\Sigma+\sigma^{2}I)^{-1} and the fact that I_{ij}=0 for i\neq j . ∎ Definition D.2 (Circulant covariance) . A matrix \Sigma\in\mathbb{R}^{d\times d} is called circulant if there exists a vector c=[c_{0},\dots,c_{d-1}]^{T} such that \Sigma_{ij}=c_{(i-j)\bmod d},\qquad i,j\in\{0,\dots,d-1\}. Equivalently, each row of \Sigma is a cyclic shift of its first row. Lemma D.3 . Let \Sigma\in\mathbb{R}^{d\times d} be circulant and let \omega=e^{2\pi i/d} . Then \Sigma is diagonalized by the discrete Fourier basis (\omega^{jk})_{j,k=0}^{d-1} , with eigenvalues (\lambda_{k})_{k=0}^{d-1} . In particular, for any function f applied spectrally to \Sigma , the matrix f(\Sigma) is circulant and its entries depend only on the wrap-around distance \mathrm{dist}(i,j):=\min\{|i-j|,\,d-|i-j|\} . Proof sketch. This is a classical property of circulant matrices: they are simultaneously diagonalized by the discrete Fourier transform, and functional calculus preserves circulant structure. See [ 8 ] for a detailed treatment of the spectral theory of circulant and Toeplitz matrices. ∎ Theorem D.4 . Assume \Sigma is circulant. Let \omega=e^{2\pi i/d} and let (\lambda_{k})_{k=0}^{d-1} be the eigenvalues of \Sigma in the discrete Fourier basis (Lemma D.3 ). Define h_{k}:=\frac{\sigma^{2}}{\lambda_{k}+\sigma^{2}},\qquad q_{r}:=\frac{1}{d}\sum_{k=0}^{d-1}h_{k}\,\omega^{rk},\qquad r\in\{0,\dots,d-1\}. For all i\neq j , let n:=\min\{|i-j|,\,d-|i-j|\} be the wrap-around distance. Then, \left|\frac{\partial[m_{\sigma}]_{i}}{\partial x_{j}}\right|=|q_{n}|. Moreover, for all n\in\{1,\dots,\lfloor d/2\rfloor\} , \left|\frac{\partial[m_{\sigma}]_{i}}{\partial x_{j}}\right|\leq\frac{\mathrm{TV}(h)}{4n},\qquad\mathrm{TV}(h):=\sum_{k=0}^{d-1}|h_{k+1}-h_{k}|\ \ (\text{indices mod }d). (36) Proof. Define the matrix Q_{\sigma}:=\sigma^{2}(\Sigma+\sigma^{2}I)^{-1}. Since \Sigma is circulant, so is \Sigma+\sigma^{2}I , and hence (\Sigma+\sigma^{2}I)^{-1} and Q_{\sigma} are circulant as well (Lemma D.3 applied to the spectral function f(t)=1/(t+\sigma^{2}) ). In the DFT basis, the eigenvalues of Q_{\sigma} are h_{k}=\frac{\sigma^{2}}{\lambda_{k}+\sigma^{2}},\qquad k=0,\dots,d-1. Therefore Q_{\sigma} has circulant kernel (q_{r})_{r=0}^{d-1} given by the inverse DFT: [Q_{\sigma}]_{ij}=q_{(i-j)\bmod d},\qquad q_{r}=\frac{1}{d}\sum_{k=0}^{d-1}h_{k}\,\omega^{rk}. Now fix i\neq j and let n:=\min\{|i-j|,\,d-|i-j|\} . By Lemma D.1 , \left|\frac{\partial[m_{\sigma}]_{i}}{\partial x_{j}}\right|=\sigma^{2}\left|[(\Sigma+\sigma^{2}I)^{-1}]_{ij}\right|=\left|[Q_{\sigma}]_{ij}\right|=|q_{n}|, which proves the first claim. For the decay bound, let (\Delta h)_{k}:=h_{k+1}-h_{k} (indices modulo d ). For n\not\equiv 0\pmod{d} , a cyclic summation-by-parts identity (discrete integration by parts) yields q_{n}=\frac{\omega^{n}}{d(1-\omega^{n})}\sum_{k=0}^{d-1}(\Delta h)_{k}\,\omega^{nk}. Taking absolute values and using the triangle inequality gives |q_{n}|\leq\frac{\sum_{k=0}^{d-1}|\Delta h_{k}|}{d\,|1-\omega^{n}|}=\frac{\mathrm{TV}(h)}{d\,|1-\omega^{n}|}. Finally, for 1\leq n\leq d/2 we have |1-\omega^{n}|=2|\sin(\pi n/d)|\geq\frac{4n}{d}, and substituting into the previous display yields ( 36 ). ∎ Appendix E Experiment Details E.1 Sampling Schedule Unless otherwise specified, the sampling schedule used in our experiments is the polynomial sampling schedule from [ 12 ] : \sigma_{i}=\Big(\sigma_{\max}^{1/\rho}+\frac{i}{N-1}\big(\sigma_{\min}^{1/\rho}-\sigma_{\max}^{1/\rho}\big)\Big)^{\rho}, for i=0,\ldots,N-1 , where \sigma_{\max}=80 , \sigma_{\min}=0.002 and N=18 is a popular choice for sampling on CIFAR-10. E.2 Empirical Shell Coverage on CIFAR-10 Figure 6 : Empirical shell coverage on CIFAR-10. We plot the lower bound, coverage, and upper bound from Theorem 4.8 . The lower bound curve (blue) is nearly indistinguishable from the coverage curve across the entire range of \sigma , as it almost perfectly overlaps with it. We empirically evaluate the lower and upper bounds in Theorem 4.8 and compare them with the Gaussian shell coverage probability on the CIFAR-10 dataset; see Figure 6 . All images are flattened to vectors in \mathbb{R}^{d} with d=3072 and linearly rescaled to lie in [-1,1]^{d} . For a fixed shell width parameter c=5 , we estimate the function \Phi_{d,c}(t)=\mathbb{P}\!\left(\|\bm{Z}\|\in[a_{d},b_{d}],\;\|\bm{Z}+te_{1}\|\in[a_{d},b_{d}]\right),\qquad\bm{Z}\sim\mathcal{N}(0,I_{d}), using Monte Carlo with 5000 i.i.d. Gaussian samples. We reuse the same Gaussian samples to evaluate \Phi_{d,c}(t) for all values of t encountered in the experiment. Noise schedule. We use the EDM polynomial noise schedule stated in Section E.1 with \sigma_{\max}=80 , \sigma_{\min}=0.002 , and a 40-step discretization ( N=40 ). Upper bound. To estimate the upper bound term N\,\mathbb{E}[\Phi_{d,c}(\|\bm{X}-\bm{X}^{\prime}\|/\sigma)] with N=1000 , we independently sample 12{,}000 pairs (\bm{X},\bm{X}^{\prime}) from the CIFAR-10 training set and compute the empirical average of \Phi_{d,c}(\|\bm{X}-\bm{X}^{\prime}\|/\sigma) for each noise level \sigma . Lower bound. To estimate the lower bound term \mathbb{E}[\Phi_{d,c}(d_{1\mathrm{NN}}(\bm{X})/\sigma)] , we perform 250 independent trials. In each trial, we sample a fresh subset \{\bm{X}_{1},\dots,\bm{X}_{N}\} of N=1000 training images and independently sample 8 test images \bm{X} . For each test image, we compute the nearest-neighbor distance d_{1\mathrm{NN}}(\bm{X})=\min_{1\leq i\leq N}\|\bm{X}-\bm{X}_{i}\| and evaluate \Phi_{d,c}(d_{1\mathrm{NN}}(\bm{X})/\sigma) . The results are averaged over test images and trials. Gaussian shell coverage. We estimate the true coverage probability \mathbb{P}(\bm{X}_{\sigma}\in\mathcal{U}_{\sigma}) using 350 independent trials. In each trial, we sample a fresh subset \{\bm{X}_{1},\dots,\bm{X}_{N}\} of size N=1000 and an independent test image \bm{X} . For each noise level \sigma , we draw 10 independent Gaussian noises \bm{Z} and check whether \bm{X}+\sigma\bm{Z} lies in at least one shell centered at the subset points. The coverage probability is estimated by averaging over noises and trials. E.3 Max vs Self Posterior Weight Figure 7 : Max vs. Sample Weight on 1K CIFAR-10. We plot average values of \max_{1\leq i\leq N}w_{i}(x_{1}+\sigma\bm{Z},\sigma) and w_{1}(x_{1}+\sigma\bm{Z}) across a range of \sigma . The two curves show that the max posterior weight can usually be well estimated by the sample weight, particularly at smaller noise levels where they coincide. Max vs Self weight estimation. For each noise level \sigma , we fix a subset of N=1000 images sampled uniformly without replacement from the CIFAR-10 dataset. We then draw 100 base points without replacement from the subset and for each base point x_{1} we compute for each i\in\{1,\dots,N\} , w_{i}(x_{1}+\sigma z,\sigma) over 400 independent Gaussian noises z . We then compute the averages for both \max_{i}w_{i}(x_{1}+\sigma z,\sigma) and w_{1}(x_{1}+\sigma z,\sigma) over all base points x_{1} and subsequent samples z . The resulting curves show the averages for each value of \sigma . E.4 Danger Zone Identification: 2k Car Subset In Figure 8 , we overlay the coverage and max posterior weight curves from Figure 3 for the 2k car training set and 1k car test set. The danger zone is identified as the region where both metrics are high, which in the case of 2k -car is a narrow interval containing the intersection point \sigma\approx 1.9 . Figure 8 : Coverage and max posterior weight vs noise level for 2k car training set and 1k car test set. The danger zone is where both metrics are high, spanning \sigma\in[1,5] with intersection near \sigma\approx 1.9 . E.5 Pareto Frontier of Training Configurations on 1024 Grayscale CelebA Images We compute the FID score of 10k samples against 10k held out test images. We visualize the Pareto frontier of different timestep gap configurations on the 1024 grayscale CelebA training set in Figure 9 . Each point represents a different gap configuration, plotting FID score against memorization rate (FMEM%). The dashed line connects non-dominated solutions forming the Pareto frontier. Figure 9 : Pareto frontier showing the trade-off between image quality (FID) and memorization (FMEM%). Each point represents a different timestep gap configuration. The dashed line connects non-dominated solutions forming the Pareto frontier. E.6 Denoiser Swapping Experiments To understand which noise levels are critical for sample quality, we perform denoiser swapping experiments. We generate samples using hybrid schedules that switch between EDM-1K and EDM-50K at different \sigma regions (large: \sigma>5.3 , medium: \sigma\in[0.06,5.3] , small: \sigma<0.06 ). Figure 10 shows representative samples from each condition. Key findings: • Swapping to EDM-50K in the medium region (steps 7-13) produces the largest quality improvement when starting from EDM-1K . • Swapping to EDM-1K in the medium region causes the largest quality degradation when starting from EDM-50K . • large and small region swaps have minimal impact on final sample quality. This confirms that the mid- \sigma region is where memorization most impacts generation quality, aligning with our one-step memorization analysis. Table 1 in the main text quantifies the memorization rates across all conditions. Figure 10 : Denoiser Swapping Samples. Top half: EDM-1K base with EDM-50K swapped in different regions. Bottom half: EDM-50K base with EDM-1K swapped in. Red boxes indicate memorized samples (where d_{1\mathrm{NN}}