Membership Inference Attack Against Music Diffusion Models via Generative Manifold Perturbation Yuxuan Liu, Peihong Zhang, Rui Sang, Zhixin Li, Yizhou Tan, Yiqiang Cai, Shengchen Li Abstract Membership inference attacks (MIAs) test whether a specific audio clip was used to train a model, making them a key tool for auditing generative music models for copyright compliance. However, loss-based signals (e.g., reconstruction error) are weakly aligned with human perception in practice, yielding poor separability at the low false-positive rates (FPRs) required for forensics. We propose the Latent Stability Adversarial Probe (LSA-Probe), a white-box method that measures a geometric property of the reverse diffusion: the minimal time-normalized perturbation budget needed to cross a fixed perceptual degradation threshold at an intermediate diffusion state. We show that training members, residing in more stable regions, exhibit a significantly higher degradation cost. With matched compute and a fixed threshold ( τ = P95 \tau{=}\mathrm{P95} ), LSA-Probe improves TPR@1% FPR by 3–8 percentage points over the best baseline across DiffWave and MusicLDM. These results indicate that local generative stability provides a reliable membership signal for audio diffusion models. † † address: Xi’an Jiaotong-Liverpool University, Suzhou, China 1 Introduction Diffusion models are revolutionizing music generation, synthesizing high-fidelity audio that achieves a high degree of perceptual realism [ 3 , 4 ] . However, the remarkable performance of these models is predicated on their training over vast datasets, often of uncertain provenance and consent [ 7 ] . This dependency creates a significant risk of copyright infringement and privacy violations for artists and rights-holders. Consequently, there is an urgent need for reliable auditing tools to determine whether a specific music clip was used to train a given model. Membership Inference Attacks (MIAs) are designed to exploit a model’s memorization of its training data to distinguish members (training-set samples) from non-members (held-out samples) [ 21 , 22 ] . In the domain of computer vision, such attacks operate on a core assumption: member samples yield a lower reconstruction loss than non-member samples [ 9 ] . However, a growing number of recent studies indicates that this fundamental assumption is severely challenged when applied to audio domains. For instance, Rossi et al. [ 20 ] demonstrated that standard MIAs falter on sequence models because they fail to adequately model the complex internal correlations within the data. Kong et al. [ 13 ] report that such endpoint, loss-only criteria are confounded and unstable for audio diffusion: they are sensitive to content complexity and timbral factors, exhibit high variance across noise seeds and timesteps. These findings have collectively catalyzed a paradigm shift in MIA research: a move away from relying on a single, static, endpoint signal (such as the reconstruction loss) towards exploiting more informative, dynamic, process-based signals [ 12 , 17 ] . This shift is motivated by an analogy to the well-established principle of ”flat minima”, where smoother regions of the loss landscape are strongly correlated with better generalization and robustness [ 12 ] . Matsumoto et al. [ 18 ] showed that members are more robust than non-members when adding the same adversarial perturbations. Inspired by this, we propose the Latent Stability Adversarial Probe (LSA-Probe), a novel white-box framework that directly interrogates the geometric stability of the generative process to infer membership. Our key hypothesis is that a model learns a smoother, more stable generative mapping in the local vicinity of its training members. We operationalize this hypothesis by quantifying the adversarial cost: the minimum perturbation budget required to induce a fixed level of perceptual degradation when applied to an intermediate latent state of the reverse diffusion process. Because members reside in more stable regions of the generative manifold, the adversarial cost required to degrade them is significantly higher than for non-members, providing a robust signal for our attack. The proposed method is evaluated on two types of music diffusion models: (i) a waveform Denoising Diffusion Probabilistic Model (DDPM) [ 10 ] , instantiated as DiffWave [ 14 ] ; and (ii) a Latent Diffusion Model (LDM), instantiated as MusicLDM [ 3 ] —across two datasets: the MAESTRO v3 [ 8 ] piano performance corpus and the Free Music Archive (FMA) Large subset [ 5 ] . Our experiments demonstrate a robust signal for membership that surpasses traditional baselines in the forensically crucial low-false-positive-rate regime. The primary contributions of this work are as follows: 1. To the best of our knowledge, this is the first systematic investigation of membership inference attacks for music diffusion models. 2. We propose the LSA-Probe, a white-box MIA that probes time-normalized latent stability along the reverse diffusion trajectory and scores samples by the adversarial cost needed to reach a fixed perceptual degradation. 3. We connect local generative stability to membership via a first-order analysis, yielding a practical score without requiring likelihoods or shadow models. Figure 1 : LSA-Probe overview (two-loop procedure at timestep t ). (a) Outer control loop. We search the budget \eta by binary search to find the adversarial cost C_{\mathrm{adv}}(x_{0};t,\tau)=\min\{\eta:\ D(\hat{x}_{0},\hat{x}_{0}^{\delta})\geq\tau\} , then use C_{\mathrm{adv}} as the decision score (higher \Rightarrow more likely member ); the dashed line indicates the decision threshold used for TPR@FPR analyses. (b) Inner loop (PGD on \tilde{\delta} ). Given a target waveform x_{0} , we form x_{t} by the forward process and inject a time-normalized latent perturbation \delta_{t}=\sigma_{t}\tilde{\delta} with \sigma_{t}=\sqrt{1-\bar{\alpha}_{t}} . We update \tilde{\delta} for K steps with projection onto the \ell_{p} ball \|\tilde{\delta}\|_{p}\leq\eta (sign step for \ell_{\infty} or normalized gradient for \ell_{2} ), while backpropagating through the deterministic reverse operator R_{t}(\cdot;\theta) and the chosen differentiable distance D (CDPAM, MR - STFT, log-mel MSE, or waveform MSE). The inner loop outputs the degradation D(\hat{x}_{0},\hat{x}_{0}^{\delta}) for the current \eta , which the outer loop uses to adjust \eta . For each (x_{0},t) we fix the forward noise \epsilon (seeded) to isolate the effect of \delta_{t} ; restarts and momentum are omitted from the diagram for clarity. 2 Proposed Method We study a white-box, developer-side setting: the adversary has read-access to model parameters, the reverse sampler, and gradients w.r.t. inputs at intermediate diffusion states. The goal is to decide membership for a given waveform x_{0} using only the audited model (no shadow models or training labels). An overview of our two-loop procedure (outer binary search for budget, inner PGD on a time-normalized perturbation) is shown in Fig. 1 and referenced throughout this section. 2.1 Preliminaries: forward/noise parameterization and reverse operator Let x_{0}\!\in\!\mathbb{R}^{n} denote a waveform. The forward noising at discrete step t\!\in\!\{1,\dots,T\} follows the usual DDPM parameterization x_{t}\;=\;\sqrt{\bar{\alpha}_{t}}\,x_{0}\;+\;\sqrt{1-\bar{\alpha}_{t}}\,\epsilon,\qquad\epsilon\!\sim\!\mathcal{N}(0,I), (1) with schedule \{\bar{\alpha}_{t}\}_{t=1}^{T} . We adopt a deterministic DDIM-style reverse operator with \sigma_{t}{=}0 . Given a noise-predictor \epsilon_{\theta}(\cdot,t) , define the stepwise denoised estimate \hat{x}_{0}(x_{t},t)\;=\;\frac{x_{t}-\sqrt{1-\bar{\alpha}_{t}}\,\epsilon_{\theta}(x_{t},t)}{\sqrt{\bar{\alpha}_{t}}}. (2) A single reverse step maps x_{t}\!\mapsto\!x_{t-1} via x_{t-1}=\sqrt{\bar{\alpha}_{t-1}}\,\hat{x}_{0}(x_{t},t)\;+\;\sqrt{1-\bar{\alpha}_{t-1}}\,\epsilon_{\theta}(x_{t},t). (3) Composing ( 3 ) from index t down to 1 yields a differentiable mapping R_{t}(\cdot;\theta):\;x_{t}\mapsto\hat{x}_{0}\in\mathbb{R}^{n} : \hat{x}_{0}\;=\;R_{t}(x_{t};\theta). (4) We index timesteps either by an integer t or by a ratio t_{\!\mathrm{ratio}}\in(0,1] , mapped as t=\lfloor t_{\!\mathrm{ratio}}(T{-}1)\rfloor{+}1 . 2.2 Time-normalized latent perturbation Perturbations at different t are not directly comparable because the forward noise variance is 1{-}\bar{\alpha}_{t} . We therefore inject a time-normalized perturbation at x_{t} : \delta_{t}\;=\;\sigma_{t}\,\tilde{\delta},\qquad\sigma_{t}=\sqrt{1-\bar{\alpha}_{t}},\quad\|\tilde{\delta}\|_{p}\leq\eta, (5) with p\!\in\!\{2,\infty\} and a global budget \eta{>}0 . The auxiliary variable \tilde{\delta} is drawn i.i.d. per sample from a Gaussian direction and is the optimization variable in our attack. 1 1 1 Rademacher directions gave similar trends in pilot runs; we fix Gaussian for reproducibility. The choice \delta_{t}{=}\sqrt{1-\bar{\alpha}_{t}}\,\tilde{\delta} matches the forward variance, equalizing the signal-to-noise ratio across timesteps and making budgets comparable. For each pair (x_{0},t) , we fix the forward \epsilon via a sample-specific seed so that paired evaluations isolate the effect of \delta_{t} ; across repetitions we average over K seeds. A second reverse pass from the perturbed latent yields \hat{x}_{0}^{\delta}\;=\;R_{t}\!\big(x_{t}+\delta_{t};\theta\big). (6) We quantify degradation by a differentiable distance D(\cdot,\cdot) computed on waveforms. Unless otherwise stated, we use the following metrics: CDPAM , multi-resolution STFT (MR-STFT) distance, log-mel MSE, and waveform MSE. Gradients of the primary metric are used only inside the attack optimization; all metrics are reported at inference for robustness. The inner-loop PGD and its interaction with R_{t} and D are depicted in Fig. 1 (b). 2.3 Objectives and threshold calibration We study two complementary objectives. For brevity, let \hat{x}_{0}\;=\;R_{t}(x_{t};\theta),\qquad\hat{x}_{0}^{\tilde{\delta}}\;=\;R_{t}\!\big(x_{t}+\sigma_{t}\tilde{\delta};\theta\big). (7) (O1) Fixed-budget maximal degradation \max_{\;\|\tilde{\delta}\|_{p}\leq\eta}\;\;D\!\big(\hat{x}_{0},\;\hat{x}_{0}^{\tilde{\delta}}\big). (8) (O2) Adversarial cost C_{\mathrm{adv}}(x_{0};t,\tau)\;=\;\inf\!\left\{\eta\!\geq\!0\ \middle|\ \exists\,\tilde{\delta},\;\|\tilde{\delta}\|_{p}\!\leq\!\eta,\;D\!\big(\hat{x}_{0},\hat{x}_{0}^{\tilde{\delta}}\big)\!\geq\!\tau\right\}. (9) Calibration of \tau . We pre-register \tau on a disjoint, unlabeled development non-member set \mathcal{D}_{\mathrm{dev}} . For each (x_{0},t)\!\in\!\mathcal{D}_{\mathrm{dev}} and L random unit Gaussian directions u , we evaluate D\!\Big(\hat{x}_{0},\;R_{t}\!\big(x_{t}+\sigma_{t}\,\eta_{\mathrm{ref}}\,u;\theta\big)\Big), (10) with the forward \epsilon fixed per (x_{0},t) . We set \tau to the 95th percentile over all samples and directions (default \eta_{\mathrm{ref}}{=}0.05 ). We fix \tau to P95 throughout experiments. The outer budget search yielding C_{\mathrm{adv}} and the subsequent membership decision are illustrated in Fig. 1 (a). 2.4 Optimization and compute parity We solve (O1) by projected gradient descent (PGD) on \tilde{\delta} ; (O2) uses an outer bisection on \eta\!\in\![0,\eta_{\max}] with inner PGD. Inner PGD employs momentum 0.9 , restarts r , and a step size tied to the current budget, \alpha\;=\;\beta\,\frac{\eta}{K},\quad\beta\in[0.2,0.3], with K steps. We project by \Pi_{\|\cdot\|_{2}\leq\eta}(\mathbf{z})=\eta\,\mathbf{z}/\max(\eta,\|\mathbf{z}\|_{2}) for p{=}2 , and elementwise clipping for p{=}\infty . Early stopping triggers when \Delta D/D\!<\!1\% for 3 consecutive steps or \|\nabla\|_{2}\!<\!10^{-6} . Outer bisection runs 10 steps. Gradients flow through R_{t} and the chosen D ; we enable gradient checkpointing across reverse steps. To ensure fair comparisons, we report UNet calls (forward/backward), wall-clock, and estimated FLOPs. With PGD( K ) and bisection( B ) the attack uses approximately (K{+}2)B reverse passes and (K{+}1)B metric evaluations per sample; baselines are repeated or multi- t to match total compute within \pm 5\% . Algorithm 1 Adversarial cost C_{\mathrm{adv}} at timestep t (DDIM, p\!\in\!\{2,\infty\} ) 01: x_{0} , schedule \{\bar{\alpha}_{s}\} , t , reverse R_{t} , metric D , norm p , max budget \eta_{\max} , steps K , restarts r , step-scale \beta , seed s 02: Fix forward \epsilon by seed s and form x_{t} via ( 1 ); set \sigma_{t}\leftarrow\sqrt{1-\bar{\alpha}_{t}} 03: Initialize [l,u]\leftarrow[0,\eta_{\max}] 04: for b\leftarrow 1 to 10 do \triangleright bisection (Fig. 1 a) 05: \eta\leftarrow(l{+}u)/2 ; D^{\star}\leftarrow 0 06: for j\leftarrow 1 to r do \triangleright PGD inner loop (Fig. 1 b) 07: Sample \tilde{\delta}^{(0)}\sim\mathcal{N}(0,I) and project to \lVert\cdot\rVert_{p}\leq\eta 08: for k\leftarrow 0 to K{-}1 do 09: \hat{x}_{0}\leftarrow R_{t}(x_{t}) ; \hat{x}_{0}^{\delta}\leftarrow R_{t}(x_{t}+\sigma_{t}\tilde{\delta}^{(k)}) 10: g\leftarrow\nabla_{\tilde{\delta}}\,D(\hat{x}_{0},\hat{x}_{0}^{\delta}) ;  update momentum 11: \tilde{\delta}^{(k{+}1)}\leftarrow\Pi_{\lVert\cdot\rVert_{p}\leq\eta}\!\bigl(\tilde{\delta}^{(k)}+\alpha\,\mathrm{step}(g)\bigr) , \alpha\leftarrow\beta\,\eta/K 12: end for 13: D^{\star}\leftarrow\max\!\bigl\{D^{\star},\,D\bigl(R_{t}(x_{t}),\,R_{t}(x_{t}{+}\sigma_{t}\tilde{\delta}^{(K)})\bigr)\bigr\} 14: end for 15: if D^{\star}\geq\tau then 16: u\leftarrow\eta 17: else 18: l\leftarrow\eta 19: end if 20: end for 21: return C_{\mathrm{adv}}\approx u 2.5 Relation to trajectory-based MIAs Trajectory reconstruction attacks (e.g., proximal-initialization style) recover a “ground-truth” DDIM trajectory point x_{t-t^{\prime}} from (x_{0},x_{t}) and compare it with a model-predicted x^{\prime}_{t-t^{\prime}} using an \ell_{p} distance. Our LSA-Probe instead injects a time-normalized perturbation at x_{t} and searches a worst-case direction via PGD. The two families are complementary under a white-box threat model; we re-implement the trajectory method [ 13 ] as a baseline for head-to-head comparison under matched compute. 2.6 Applicability to latent diffusion (MusicLDM) The probe applies unchanged to latent diffusion. In MusicLDM, diffusion operates on VAE latents z . For each waveform x_{0} , we compute z_{0}=\mathrm{Enc}(x_{0}) , forward-noise to z_{t} , and inject the time-normalized perturbation in latent space: z_{t}\mapsto z_{t}+\delta_{t} . The deterministic reverse R_{t}(\cdot;\theta) runs in latent space; the waveform scored by D is obtained with a frozen decoder: \hat{x}_{0}=\mathrm{Dec}\!\big(R_{t}(z_{t};\theta)\big),\qquad\hat{x}_{0}^{\tilde{\delta}}=\mathrm{Dec}\!\big(R_{t}(z_{t}+\sigma_{t}\tilde{\delta};\theta)\big). (11) Gradients backpropagate through R_{t} and the fixed decoder; when conditioning is used, the text prompt and guidance are held constant across paired evaluations. Compute parity counts both UNet and decoder calls. Table 1: Main results under matched compute (DDIM, t_{\mathrm{ratio}}{=}0.6 , p{=}2 , \eta_{\max}{=}0.8 ). We report TPR@ 1\text{\,}\mathrm{\%} FPR and AUC-ROC (mean \pm 95% CI). “Best Baseline” is the best among Loss / Trajectory / SecMI. \Delta is Ours minus Best Baseline. Model Dataset Best Baseline (TPR@1% / AUC) Ours (TPR@1% / AUC) \Delta (Ours - Best) – – TPR@1% AUC MusicLDM MAESTRO 0.10\ \bigl(0.07\text{--}0.12\bigr)/0.58\pm 0.02 \mathbf{0.13}\ \bigl(\mathbf{0.10}\text{--}\mathbf{0.15}\bigr)/\mathbf{0.61}\pm\mathbf{0.03} +0.03 +0.03 MusicLDM FMA-Large 0.08\ \bigl(0.05\text{--}0.10\bigr)/0.56\pm 0.01 \mathbf{0.14}\ \bigl(\mathbf{0.10}\text{--}\mathbf{0.16}\bigr)/\mathbf{0.59}\pm\mathbf{0.02} +0.06 +0.03 DiffWave MAESTRO 0.12\ \bigl(0.09\text{--}0.15\bigr)/0.63\pm 0.02 \mathbf{0.20}\ \bigl(\mathbf{0.16}\text{--}\mathbf{0.24}\bigr)/\mathbf{0.67}\pm\mathbf{0.02} +0.08 +0.04 DiffWave FMA-Large 0.11\ \bigl(0.08\text{--}0.14\bigr)/0.62\pm 0.02 \mathbf{0.18}\ \bigl(\mathbf{0.14}\text{--}\mathbf{0.22}\bigr)/\mathbf{0.66}\pm\mathbf{0.02} +0.07 +0.04 (a) ROC @ t_{\mathrm{ratio}}{=}0.6 (DiffWave / MusicLDM). (b) t_{\mathrm{ratio}}\in\{0.2,0.4,0.6,0.8\} . (c) Budget \eta\in[0.05,0.8] . (d) D\in\{ CDPAM, MR-STFT, log-mel MSE, wave MSE \} . Figure 2 : Key analyses under fixed \tau{=} P95 (DDIM, p{=}2 , \eta_{\max}{=}0.8 ). (a) ROC curves with 95% CIs; our LSA-Probe improves the low-FPR region. (b) Timestep ablation: mid-trajectory timesteps yield stronger separability. (c) Budget ablation: larger budgets help until mild saturation. (d) Metric robustness: perceptual metrics (CDPAM/MR-STFT) outperform training-aligned MSEs at low FPR. 3 Experiments We evaluate two music diffusion families: (i) a waveform DDPM instantiated as DiffWave [ 14 ] and (ii) a latent diffusion model instantiated as MusicLDM (unconditional/weakly conditioned LDM operating in VAE latents) [ 3 ] . Datasets are MAESTRO v3 (solo piano) [ 8 ] and FMA-Large (multi-genre) [ 5 ] . We adopt deterministic DDIM sampling, timestep ratio t_{\mathrm{ratio}}{=}0.6 , norm p{=}2 , maximum budget \eta_{\max}{=}0.8 , and the primary distance D is CDPAM; MR-STFT, log-mel MSE, and waveform MSE are also reported for robustness. Audio is trained at 22.05 kHz and metrics are computed at 16 kHz via in-graph resampling. Our threat model is white-box, developer-side with gradient access. According to previous research [ 2 , 1 , 16 ] , the decision threshold is fixed at \tau=\mathrm{P95} on a disjoint development non-member set. 3.1 Evaluation metrics and protocol We report AUC-ROC, TPR@FPR= 1\text{\,}\mathrm{\%} , TPR@FPR= 0.1\text{\,}\mathrm{\%} . AUC confidence intervals are computed with DeLong [ 6 ] ; other endpoints use sample-level bootstrap (10,000 resamples). Across timesteps and metrics, we control family-wise error via Holm–Bonferroni [ 11 ] . For membership labels, the member pool consists of training clips; the non-member pool is sampled from held-out validation/test with near-/cross-domain controls. We remove duplicates/covers using Chromaprint+LSH with manual spot checks. 3.2 Models, training, and splits DiffWave follows the official UNet architecture (200k steps, batch 32, AdamW, cosine LR). MusicLDM is trained unconditionally in the VAE latent space (stride \times 4 ); during attacks, the decoder is frozen and gradients flow through the latent reverse operator while distances are computed on decoded waveforms. We cut audio into 4 s clips, stratified by piece/artist (MAESTRO) or track/artist (FMA) to avoid leakage. Unless otherwise specified, samplers are deterministic (DDIM, \sigma_{t}{=}0 ). 3.3 Baselines and compute accounting Baselines include: loss-aligned (denoising/reconstruction losses at t and at the endpoint) [ 19 ] , trajectory (PIA/PIAN) [ 13 ] , SecMI [ 15 ] . To ensure fairness, we match compute within \pm 5\% using per-sample wall-clock on A100-80GB. Our adversarial cost C_{\mathrm{adv}} uses outer bisection ( B{=}10 ) with inner PGD ( K{=}12 steps, restarts r{=}5 ), which totals approximately (K{+}2)B reverse passes per sample. 3.4 Main results Table 1 reports the primary endpoint (TPR@ 1\text{\,}\mathrm{\%} FPR) and AUC-ROC under matched compute (DDIM, t_{\mathrm{ratio}}{=}0.6 , p{=}2 , \eta_{\max}{=}0.8 ). Across both datasets and model families, LSA-Probe consistently improves the low-FPR operating point over the strongest baseline: for MusicLDM (LDM), TPR@ 1\text{\,}\mathrm{\%} increases from 0.10\!\rightarrow\!0.13 on MAESTRO and 0.08\!\rightarrow\!0.14 on FMA-Large (absolute gains +0.03 and +0.06 ), with AUC lifts of +0.03 on both datasets; for DiffWave (DDPM), TPR@ 1\text{\,}\mathrm{\%} rises from 0.12\!\rightarrow\!0.20 (MAESTRO) and 0.11\!\rightarrow\!0.18 (FMA-Large) (gains +0.08 and +0.07 ), accompanied by larger AUC improvements of +0.04 on both datasets. Importantly, even with these gains, LDMs remain more robust to MIA than DDPMs. 3.5 Key analyses: ROC, timestep, budget, and metric Figure 2 consolidates the four most informative analyses into a single 4 \times 1 panel. (a) ROC: At t_{\mathrm{ratio}}{=}0.6 , LSA-Probe improves the low-FPR region across models, consistent with Table 1 . (b) Timestep: Sweeping t_{\mathrm{ratio}}\!\in\!\{0.2,0.4,0.6,0.8\} shows peak separability at mid-trajectory (0.6), aligning with the intuition that the reverse path transitions from coarse global layout to finer details. (c) Budget: Increasing \eta improves TPR@1% until mild saturation near \eta{\approx}0.6 – 0.8 , where saturation rates remain low. (d) Metric: Perceptual metrics (CDPAM, MR-STFT) provide stronger discrimination at low FPR than training-aligned MSEs, while the latter remain useful complementary views. 3.6 Implementation details We fix the forward noise per (x_{0},t) using a sample-specific seed so that paired evaluations isolate the effect of \delta_{t} ; across repetitions we average over S random seeds. The adversarial inner loop uses PGD with K{=}12 steps, momentum 0.9 , restarts r{=}5 , and step size \alpha=\beta\,\eta/K with \beta\!\in[0.2,0.3] ; Early stopping triggers at \Delta D/D{<}1\% for three consecutive steps or \|\nabla\|_{2}{<}10^{-6} . 4 Conclusion We proposed LSA-Probe , a white-box membership inference attack for music diffusion models that measures time-normalized latent stability along the reverse trajectory, using the adversarial budget required to exceed a fixed perceptual degradation threshold as the score. Across two model families (DiffWave and MusicLDM) and two corpora (MAESTRO v3, FMA-Large), LSA-Probe consistently improves low-FPR detection over strong baselines under matched compute. Ablations indicate that mid-trajectory timesteps and moderate budgets are most informative, and that perceptual metrics (CDPAM/MR-STFT) reveal stability gaps more reliably than MSE-based distances. 5