Adaptive Momentum Langevin Dynamics for Efficient Posterior Sampling Authors: Zhengbo Li, Jian Xu, Dingtao Peng, Shuang Hu Published: 2026 ABSTRACT Sampling from high-dimensional posterior distributions is a central challenge in Bayesian inference and noisy inverse problems. Standard first-order Langevin-based methods often suffer from slow convergence and sensitivity to step-size hyperparameters, particularly in annealed score-based inverse imaging pipelines. We propose Adaptive Momentum Langevin Dynamics (AMLD), a practical stochastic correction kernel that introduces a momentum variable into the annealed posterior sampling framework and equips it with an annealing-aware momentum retention schedule. The method is fully compatible with the SNIPS framework and retains its coordinate-wise adaptive step structure, acting as a lightweight drop-in replacement for the conventional first-order Langevin correction step. Extensive experiments on three representative image inverse problems—Gaussian deblurring, inpainting, and 4× super-resolution—demonstrate that AMLD consistently achieves strong PSNR and LPIPS performance, with competitive FID in most settings, compared to three state-of-the-art baselines (DDRM, DPS, SNIPS) under both nearly noiseless and noisy measurement conditions, while reaching target reconstruction quality using fewer sampling iterations. The proposed momentum-based sampler provides empirically improved exploration and robustness across evolving posterior landscapes, offering a practical and computationally efficient alternative to first-order annealed Langevin samplers in high-dimensional Bayesian inverse problems. 1. Introduction Sampling from high-dimensional posterior distributions is a central problem in Bayesian inference and inverse problems [1,2]. Langevin-based methods provide a principled framework, but often suffer from slow convergence and sensitivity to hyperparameters [3,4,5]. Recent approaches such as SNIPS [6] integrate learned score-based priors but rely on basic annealed Langevin dynamics with fixed friction, limiting exploration in multimodal landscapes. Concurrently, adaptive optimizers like Adam [7] have revolutionized non-convex optimization through gradient momentum. Our key insight is that these principles can be translated to overcome exploration bottlenecks in posterior sampling. Our Contributions:We introduce an annealing-aware momentum retention schedule that adapts momentum persistence to the evolving posterior geometry across noise levels, improving both early-stage stability and late-stage convergence.We design a momentum-based correction kernel that is fully compatible with the SNIPS framework and preserves its coordinate-wise adaptive step mechanism, serving as a lightweight drop-in replacement for the standard first-order Langevin update.We empirically demonstrate improved reconstruction quality and sampling efficiency on three benchmark image inverse problems under both noiseless and noisy conditions, and provide an ablation study quantifying the contribution of each component. We introduce an annealing-aware momentum retention schedule that adapts momentum persistence to the evolving posterior geometry across noise levels, improving both early-stage stability and late-stage convergence. We introduce an annealing-aware momentum retention schedule that adapts momentum persistence to the evolving posterior geometry across noise levels, improving both early-stage stability and late-stage convergence. We design a momentum-based correction kernel that is fully compatible with the SNIPS framework and preserves its coordinate-wise adaptive step mechanism, serving as a lightweight drop-in replacement for the standard first-order Langevin update. We design a momentum-based correction kernel that is fully compatible with the SNIPS framework and preserves its coordinate-wise adaptive step mechanism, serving as a lightweight drop-in replacement for the standard first-order Langevin update. We empirically demonstrate improved reconstruction quality and sampling efficiency on three benchmark image inverse problems under both noiseless and noisy conditions, and provide an ablation study quantifying the contribution of each component. We empirically demonstrate improved reconstruction quality and sampling efficiency on three benchmark image inverse problems under both noiseless and noisy conditions, and provide an ablation study quantifying the contribution of each component. Unlike Adam-type preconditioning, which rescales gradients in overdamped dynamics, our approach introduces a momentum retention variable within the annealed sampling framework. This leads to a different effect: instead of rescaling gradient magnitudes per coordinate, it adds directional persistence to the correction trajectory, thereby accelerating exploration and reducing random-walk behavior. We emphasize that AMLD is designed as a practical stochastic sampler rather than an exact MCMC method with strict theoretical guarantees. Our method is fully compatible with existing inference pipelines such as SNIPS, where it can replace the standard Langevin sampler without modifying other components. 2. Related Work Stochastic Gradient MCMC. SGLD [8] introduced stochastic gradient Langevin dynamics for scalable Bayesian learning, enabling posterior inference with mini-batch gradients. While theoretically well-founded [9], SGLD suffers from slow mixing in high-dimensional or multimodal settings. SGHMC [10] introduced momentum to improve exploration, drawing on Hamiltonian Monte Carlo. pSGLD [5] further proposed using gradient second-moment statistics to precondition the gradient in overdamped dynamics, effectively rescaling the step size per dimension. A key distinction from our method is that pSGLD performs coordinate-wise gradient preconditioning in the overdamped setting, whereas AMLD operates in the underdamped regime and adapts momentum retention at the annealing level rather than per dimension or per step. A complete characterization of valid SG-MCMC update rules is provided in [9], offering a unified perspective on the design space of stochastic gradient samplers. Adaptive MCMC [3] provides a general theoretical framework for adapting proposal distributions while preserving the target stationary distribution asymptotically. Score-Based Generative Models and Inverse Problems. Score-based generative models [11,12] learn the gradient of the log data density, enabling high-quality image synthesis via Langevin dynamics or stochastic differential equations. SNIPS [6] integrates these learned score functions as priors within an annealed Langevin sampling framework for solving noisy inverse problems, but relies on a fixed scalar friction coefficient throughout. DDRM [13] exploits the singular value decomposition of linear degradation operators to perform efficient diffusion-based restoration, but is restricted to linear measurement models. DPS [14] extends diffusion posterior sampling to general noisy and nonlinear inverse problems via an approximate likelihood score. A comprehensive survey [15] provides a systematic treatment of these approaches, highlighting trade-offs between computational efficiency, measurement consistency, and sample diversity. Langevin Dynamics and Posterior Sampling. Underdamped Langevin dynamics introduces an auxiliary momentum variable that enables more persistent state-space exploration than purely first-order overdamped dynamics [1]. The connection between Langevin-based dynamics and Bayesian posterior sampling is thoroughly discussed in [1,2], providing the theoretical motivation for momentum-based samplers in score-based inverse problems. AMLD adopts this momentum perspective within the SNIPS annealed sampling framework using a practical finite-step update that does not modify the coordinate-wise step mechanism inherited from SNIPS. Positioning of AMLD. The proposed method builds on two lines of research: it adopts the momentum-based underdamped perspective of SGHMC [10] to alleviate random-walk behavior, and it introduces an annealing-aware retention schedule tailored to the coarse-to-fine structure of score-based inverse problem samplers such as SNIPS [6]. Unlike pSGLD [5], which performs coordinate-wise preconditioning in the overdamped setting, AMLD operates in the underdamped regime and adapts at the annealing level rather than per step or per dimension. Relative to fixed-momentum underdamped samplers, the annealing-aware schedule provides a lightweight stability refinement that matches the changing posterior geometry across noise levels. 3. Method We present Adaptive Momentum Langevin Dynamics (AMLD) in a step-by-step manner. We first formulate the Bayesian linear inverse problem and define the annealed posterior target used in score-based inverse imaging. We then introduce reference continuous-time underdamped Langevin dynamics that motivates the use of momentum for posterior exploration. Based on this formulation, we develop an annealing-aware momentum schedule that adapts to the changing posterior geometry across noise levels, and derive the discrete update used in practice. Finally, we summarize the complete AMLD algorithm and clarify its methodological role as an efficient posterior correction kernel. Importantly, AMLD does not redefine the posterior target itself; rather, it improves the finite-step stochastic dynamics used to explore that target across annealing levels. AMLD is fully compatible with the SNIPS framework and retains its coordinate-wise adaptive step structure. 3.1. Problem Setup We consider a standard linear inverse problem in imaging. Let x∈Rd denote the unknown clean image and let y∈Rm denote the observed degraded measurement. The forward observation model is(1)y=Hx+n, where H∈Rm×d is a known linear degradation operator and n∈Rm denotes additive measurement noise. Direct inversion of Equation (1) is generally ill-posed: H may be ill-conditioned or rank-deficient, and the observation is corrupted by noise. Consequently, multiple clean images may explain the same degraded measurement. Instead of seeking a single point estimate, we adopt a Bayesian formulation and infer the posterior distribution of the unknown image given the observation. By Bayes’ rule,(2)p(x∣y)∝p(y∣x)p(x), where p(y∣x) is the likelihood induced by the forward model and p(x) is the prior distribution of natural images. For Gaussian measurement noise with standard deviation σn,(3)p(y∣x)∝exp−∥Hx−y∥222σn2. In score-based posterior sampling, rather than sampling directly from the final posterior in one step, one introduces a sequence of noise-conditioned intermediate targets indexed by annealing noise levels σl. At annealing level l, the corresponding annealed posterior energy is(4)Ul(x)=∥Hx−y∥222σn2−logpσl(x), where σl is the annealing noise level and pσl(x) denotes the Gaussian-smoothed prior distribution associated with the noise-conditioned score model. The annealing mechanism acts through the prior term, while the data-consistency term remains fixed by the observation model. In score-based generative models, the prior score is approximated by a pre-trained network: (5)sθ(x,σl)≈∇xlogpσl(x). Taking the gradient of Equation (4) and substituting Equation (5) gives the annealed posterior energy gradient: (6)∇Ul(x)=HT(Hx−y)σn2−sθ(x,σl). The derivation chain from the observation model to the posterior gradient isy=Hx+n︸(1)→Gaussian noisep(y|x)︸(3)→Bayesp(x|y)︸(2)→−logUl(x)︸(4)→∇+score∇Ul(x)︸(6). The first term in Equation (6) enforces consistency with the observation via the forward model, while the second incorporates the learned image prior. Posterior sampling is therefore used to generate reconstructions that satisfy both the measurement constraint and the learned prior distribution. This is particularly important for ill-posed inverse problems, where multiple plausible solutions naturally arise. The objective of AMLD is not to alter this posterior target, but to construct an efficient stochastic correction kernel that approximately explores it through a finite annealed refinement sequence. 3.2. Notation and Assumptions The main notation used throughout Section 3 is summarized in Table 1. We distinguish three representations of the image variable. The symbol x is the latent image in the Bayesian formulation; xt is the continuous-time process used to motivate the algorithm; and xk(l) is the discrete iterate at inner step k and annealing level l. All three refer to the same underlying quantity at different levels of abstraction. We also distinguish two types of noise. The measurement noise n is part of the physical observation model (Equation (1)), whereas σl is an algorithmic annealing parameter that controls the sequence of intermediate sampling targets. We assume throughout that (i) H is known; (ii) the score network provides a reasonable approximation of the prior score at each noise level; (iii) ∇Ul is sufficiently regular for the updates to be well defined; and (iv) all hyperparameters are chosen to maintain numerical stability. 3.3. Annealed Posterior Sampling Framework Score-based inverse problem methods sample from the sequence {pl(x)∝e−Ul(x)}l=1L of annealed posteriors, gradually refining samples from coarse to fine scales and decomposing posterior sampling into a sequence of easier stochastic correction sub-problems. At each annealing level l, starting from a state passed from the previous level, the sampler performs K correction steps intended to move the current state toward the annealed posterior before proceeding to the next level. The gradient required at each step is given by Equation (6). Following the SNIPS framework, each coordinate is associated with an individual step size encoded in the vector sl, derived from the singular value decomposition (SVD) of H and adapted to each noise level. This coordinate-wise step mechanism is preserved in full by AMLD. While the posterior score gk(l)=−∇Ul(xk(l)) is expressed in unified mathematical form, its practical computation follows the SNIPS piecewise coordinate partitioning based on the SVD of H: coordinates are partitioned according to whether σl times the corresponding singular value is larger than, smaller than, or zero at the measurement noise level σ0. This piecewise decomposition is inherited directly from SNIPS and provides numerical efficiency; it does not alter the mathematical definition of the posterior score. The role of AMLD is to improve the posterior correction step performed at each annealing level. Specifically, AMLD replaces the conventional first-order Langevin correction by a momentum-based alternative designed to improve empirical exploration efficiency and numerical stability. The resulting procedure should be understood as a practical finite-step sampler for the annealed posterior sequence rather than as an exact simulator of a single invariant diffusion. 3.4. Reference Continuous-Time Dynamics To motivate the use of momentum in AMLD, we consider a reference underdamped Langevin dynamics on an extended state space (xt,mt). With a scalar friction coefficient γ>0, the reference dynamics is (7)dxt=mtdt,(8)dmt=−γmtdt−∇Ul(xt)dt+2γdWt, where Wt is a standard Wiener process. The momentum variable is introduced to accelerate state-space exploration and alleviate the slow random-walk behavior of purely first-order overdamped Langevin dynamics. This continuous-time system serves as the conceptual motivation for AMLD. The friction term in the reference dynamics motivates momentum persistence, while the practical coefficient γl∈[0,1] acts as a momentum retention factor that controls what fraction of the previous momentum is carried to the next step; it plays an analogous but distinct role to the friction coefficient γ in the continuous reference model. To remain compatible with the coordinate-wise step mechanism of SNIPS, the practical discrete algorithm uses a simplified momentum update that does not aim to be an exact discretization of Equations (7) and (8). 3.5. Annealing-Aware Momentum Schedule Across annealing levels, the posterior landscape changes substantially: at high noise levels the energy surface is broad and requires careful exploration to avoid instability, while at low noise levels the landscape sharpens and larger momentum accelerates convergence toward fine posterior modes. A fixed momentum retention factor cannot adapt to this evolving geometry. AMLD therefore introduces an annealing-aware schedule that increases the momentum retention factor progressively across noise levels. Let γmin and γmax be the minimum and maximum momentum retention factors, and let r∈(0,1] be the warmup ratio. At annealing level l (l=1,…,L), the retention factor is(9)γl=γmin+(γmax−γmin)min1,(l−1)/Lr. During the warmup phase, the coefficient gradually increases from γmin toward γmax, reaching γmax shortly after the rL-th annealing level. For the remaining levels it stays fixed at γmax. Within each annealing level, γl remains constant across all K inner correction steps. This schedule assigns stronger attenuation (γl near γmin) in early high-noise stages to prevent momentum from accumulating over rough energy surfaces, and allows larger retention (γl=γmax) in later low-noise stages to accelerate convergence. 3.6. Discrete Sampling Updates We describe the discrete AMLD update used in practice. All operations use the coordinate-wise step vector sl to maintain full compatibility with SNIPS. Let xk(l) denote the position at inner step k and annealing level l, and let gk(l)=−∇Ul(xk(l)) denote the corresponding posterior score computed via Equation (6). A single auxiliary momentum vector m is maintained throughout the entire sampling procedure; it is initialized to zero once before the annealing schedule begins and carried continuously across all annealing levels without being reset between levels. The momentum update at each inner step is(10)m←γlm+sl⊙gk(l), where ⊙ denotes element-wise multiplication. The first term retains a fraction γl of the accumulated momentum from the previous step, providing directional persistence. The second term incorporates the current posterior score scaled by the coordinate-wise step size, directing momentum toward higher-probability regions of the annealed posterior. The position update is(11)xk+1(l)=xk(l)+m+2sl⊙ξk,ξk∼N(0,I). The position is advanced by the accumulated momentum and a stochastic exploration term whose variance is controlled coordinate-wise by sl. This differs from the classical underdamped Langevin discretization, where noise typically enters the momentum equation, and is adopted here as a practical SNIPS-compatible design choice rather than an exact invariant-preserving integrator. Injecting noise directly into the position update, rather than into the momentum update, prevents stochastic perturbations from accumulating in the momentum across steps. Because the discretization is finite-step and the retention factor varies across annealing levels, AMLD does not preserve the exact stationary measure of the reference dynamics in Equations (7) and (8). It is therefore used as a practical posterior correction kernel whose effectiveness is assessed empirically. 3.7. Complete AMLD Algorithm Algorithm 1 presents the complete AMLD procedure. Relative to the SNIPS baseline, the only structural changes are: (i) the addition of a persistent momentum vector m initialized once before the annealing schedule; (ii) the computation of the annealing-aware retention factor γl at the start of each noise level; and (iii) a two-line update replacing the single-line first-order Langevin step. All other components—the score network, annealing schedule, and SVD-based coordinate-wise step vector—are inherited unchanged from SNIPS. Algorithm 1 Adaptive Momentum Langevin Dynamics (AMLD)Require: Observation y, operator H, score network sθ, noise levels {σl}l=1L, step vectors {sl}l=1L, inner steps K, momentum bounds γmin, γmax, warmup ratio r 1:Initialize x0(1)∼N(0,σ12I) 2:Initialize momentum m=0 {initialized once; carried across all levels} 3:for l=1 to L do 4: Compute retention factor (Equation (9)): γl=γmin+(γmax−γmin)min1,(l−1)/(rL) 5: for k=0 to K−1 do 6: Compute posterior score (Equation (6)): gk(l)=sθ(xk(l),σl)−HT(Hxk(l)−y)/σn2 7: Sample ξk∼N(0,I) 8: Update momentum (Equation (10)): m←γlm+sl⊙gk(l) 9: Update position (Equation (11)): xk+1(l)=xk(l)+m+2sl⊙ξk 10: end for 11: if l