Jeffreys Flow: Robust Boltzmann Generators for Rare Event Sampling via Parallel Tempering Distillation
Guang Lin, Christian Moya, Di Qi, Xuda Ye · 2026
Read Jeffreys Flow: Robust Boltzmann Generators for Rare Event Sampling via Parallel Tempering Distillation by Guang Lin, Christian Moya, Di Qi, Xuda Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05303.txt.
Introduction
Lin, Moya, Qi, and Ye introduce Jeffreys Flow, a sampler for physical systems whose rough energy landscapes trap ordinary methods in single valleys. Standard Boltzmann generators lean on a one-sided divergence that collapses onto some modes and misses others. The team instead distils samples from parallel tempering runs under a symmetric Jeffreys divergence, balancing precise local fit against coverage of all modes. Tests on rugged multi-peaked benchmarks show the collapse suppressed, with faster exact importance sampling for quantum thermal states.
Catalog record
| CREATOR | Guang Lin, Christian Moya, Di Qi, Xuda Ye |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65769 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.05303
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Jeffreys Flow: Robust Boltzmann Generators for Rare Event Sampling via Parallel Tempering Distillation Guang Lin, Christian Moya, Di Qi, Xuda Ye Abstract Sampling physical systems with rough energy landscapes is hindered by rare events and metastable trapping. While Boltzmann generators already offer a solution, their reliance on the reverse Kullback–Leibler divergence frequently induces catastrophic mode collapse, missing specific modes in multi-modal distributions. Here, we introduce the Jeffreys Flow, a robust generative framework that mitigates this failure by distilling empirical sampling data from Parallel Tempering trajectories using the symmetric Jeffreys divergence. This formulation effectively balances local target-seeking precision with global modes coverage. We show that minimizing Jeffreys divergence suppresses mode collapse and structurally corrects inherent inaccuracies via distillation of the empirical reference data. We demonstrate the framework’s scalability and accuracy on highly non-convex multidimensional benchmarks, including the systematic correction of stochastic gradient biases in Replica Exchange Stochastic Gradient Langevin Dynamics and the massive acceleration of exact importance sampling in Path Integral Monte Carlo for quantum thermal states. I Introduction Rare event sampling is a central challenge in statistical mechanics and computational physics 8 ; 54 ; 55 . Specifically, consider a target distribution \pi(x)\propto e^{-U(x)} on x\in\mathbb{R}^{d} , where the potential function U(x) exhibits multiple local modes separated by high energy barriers. Classical Monte Carlo methods, such as Metropolis–Hastings 13 ; 53 , Hamiltonian Monte Carlo (HMC) 6 ; 12 , and Langevin dynamics 42 , routinely suffer from poor performance, since the generated samples tend to become trapped within a few localized basins. This difficulty is formalized by the Eyring–Kramers formula 24 ; 33 , which dictates that the transition probability between metastable modes decays exponentially with the height of the intervening energy barrier 7 ; 35 , rendering such transitions as rare events for standard Monte Carlo methods. To simulate rare events, numerous enhanced Monte Carlo methods have been developed: Umbrella Sampling 62 ; 65 ; 31 , Simulated Annealing 64 ; 5 ; 47 , Sequential Monte Carlo (SMC) 21 ; 9 ; 14 ; 67 , Transition Path Sampling 16 , Metadynamics 34 , and Parallel Tempering (PT) 23 ; 43 ; 61 ; 18 . Among these, PT is widely adopted because it is simple and easy to implement. The core mechanism of PT is to simulate multiple replicas of the system in parallel across a temperature ladder. By frequently swapping configurations between adjacent replicas, PT allows low-temperature simulations to inherit the global ergodicity inherent to the high-temperature limits. The flow-based generative models emerge as another promising approach for sampling complex target distributions. Empowered by the rapid advancement of GPU architectures, these methods employ a sequence of invertible normalizing flows to push forward a simple base distribution \pi_{0}(x) towards the target distribution \pi(x) . This framework, known as the Boltzmann generator 49 ; 69 , is conceptually straightforward and primarily relies on minimizing the reverse Kullback–Leibler (KL) divergence. Formally, the goal is to find a transport map F that minimizes D_{\text{KL}}(F_{\#}\pi_{0}\|\pi) , where F_{\#}\pi_{0} denotes the pushforward distribution. In practice, F is composed of invertible layers parameterized by expressive architectures such as RealNVP 20 or Neural Spline Flows 22 . During inference, a trained Boltzmann generator has unique advantages over classical Monte Carlo methods. It circumvents repeated evaluations of the potential gradient, by leveraging the high memory bandwidth of GPUs to instantly generate massive statistically independent samples. Most importantly, Boltzmann generators produce unbiased samples from the target distribution via importance sampling reweighting. Although Boltzmann generators have been widely applied in computational physics 68 ; 1 ; 32 ; 15 ; 56 , their reliability in rare event sampling depends heavily on the loss function. When the target distribution \pi(x) has multiple metastable modes separated by high energy barriers, the pushforward samples of the trained flow often get trapped in a local basin, missing other significant modes. This phenomenon, known as mode collapse 44 ; 45 , remains the core challenge in designing efficient algorithms for rare event sampling. Figure 1: The overall architecture of the Jeffreys Flow. PT acts as the principal guide across the temperature ladder, providing reference samples ( \mu_{1},\dots,\mu_{M} ) to train the sequence of normalizing flows ( F_{1},F_{2},\dots,F_{M} ). Simultaneously, intermediate states ( \nu_{0},\nu_{1},\dots,\nu_{M} ) are progressively pushed forward and reweighted, intrinsically averting mode collapse without relying solely on the target potential function. We propose a new sampling strategy using the Jeffreys Flow, a Boltzmann generator specifically tailored for rare event sampling. As the name suggests, the Jeffreys Flow adopts the Jeffreys divergence (i.e., the symmetrized KL divergence) as its loss function. Because computing the forward KL divergence requires samples from the target distribution, our method utilizes reference samples generated via PT to train the normalizing flow. Thus, the Jeffreys Flow distills the sampling knowledge from PT. Like standard Boltzmann generators, the Jeffreys Flow maintains the benefits of generating statistically independent samples and allows unbiased reweighting. Most importantly, rather than forcing the flow to learn the complex landscape solely from the target potential, the Jeffreys Flow directly incorporates target samples, thereby impeding the severe mode collapse in rare event sampling. For visual overview of the Jeffreys Flow, see Figure 1 . While the original Boltzmann generator 49 also utilized the Jeffreys divergence, it did so only in the early stages of training with limited target samples. In contrast, our Jeffreys Flow leverages PT samples to guide the entire training pipeline. Although obtaining reference samples via PT incurs an upfront computational cost, these samples provide theoretical guarantees for training the flow across complex, multi-modal landscapes. Once the training is complete, the expensive PT simulation can be entirely discarded, allowing the trained flow to instantaneously generate statistically independent samples across the entire predefined temperature ladder in just a few computational steps. Beyond its intuitive design, the Jeffreys Flow is supported by stringent theoretical confirmation. Theorem 1 establishes that the pushforward distributions (\nu_{k}) generated by minimizing the Jeffreys divergence are strictly closer to the target distribution than the empirical reference samples (\mu_{k}) , underscoring the capacity of the Jeffreys Flow to successively correct inaccuracies inherent in the PT samples. In addition, Theorem 2 derives a rigorous concentration inequality showing that the probability of mode collapse diminishes to an arbitrarily small level when the Jeffreys divergence is sufficiently minimized. Together, these analytical results confirm that the Jeffreys Flow provides a principled and direct mechanism to impede mode collapse. In the numerical tests, we evaluate the performance of the Jeffreys Flow on benchmark multi-modal distributions up to 16 dimensions. Then we apply the Jeffreys Flow to two challenging applications: Replica Exchange Stochastic Gradient Langevin Dynamics (reSGLD) 17 ; 36 ; 37 for finite-sum potentials, and Path Integral Monte Carlo (PIMC) 29 ; 40 ; 10 ; 57 for quantum thermal sampling. In the reSGLD, we utilize importance weights to correct the stochastic gradient bias and enhance target accuracy. In the PIMC, we generate PT samples from the cheap classical Boltzmann distribution instead of the expensive ring-polymer quantum equilibrium. A low-dimensional flow is then trained on only the low-frequency normal modes, while the full quantum potential is used solely for importance sampling reweighting to guarantee unbiasedness. The paper is organized as follows. Section II below reviews related flow-based