Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?

Emanuel Sommer, Kangning Diao, Jakob Robnik, Uroš Seljak, David Rügamer · 2026

Read Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise? by Emanuel Sommer, Kangning Diao, Jakob Robnik, Uroš Seljak, David Rügamer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06500.txt.

Description

Bayesian inference at scale needs sampling methods that work with small batches of data, not full passes over it. Emanuel Sommer and 4 co-authors ask whether microcanonical Langevin dynamics, a recent high-performing sampler, can exploit mini-batch gradient noise. Their study is the first systematic one of that question.

The answer involves two failure modes: a bias from uneven gradient noise across directions, and numerical instability in complex high-dimensional posteriors. The authors counter with a preconditioning scheme that tames the noise bias and an energy-variance tuner that sets step sizes and guardrails automatically. The resulting sampler proved robust and scalable on hard high-dimensional tasks, including Bayesian neural networks.

Catalog record

CREATOREmanuel Sommer, Kangning Diao, Jakob Robnik, Uroš Seljak, David Rügamer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06500

Browse the full library · About the Tower of Babel