SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows

Janik Kreit, Dominic Schuh, Kim A. Nicoli, Lena Funcke · 2025

Read SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows by Janik Kreit, Dominic Schuh, Kim A. Nicoli, Lena Funcke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19619.txt.

Introduction

Janik Kreit and co-authors teach normalizing flows to respect symmetry without being handcuffed by it, in a 2025 paper. Generative models that sample physics distributions work better when they know the system's symmetries. Strictly equivariant flows hard-wire those symmetries; real systems, though, often break them softly, and rigid models miss the truth.

Their method, SESaMo, wraps flows in stochastic modulation, a randomizing layer that injects symmetry knowledge as a bias rather than a law. The model can then learn exact symmetries where they hold and broken ones where they do not. Tests span a toy mixture of eight Gaussian blobs and two physics workhorses, scalar field theory and the Hubbard model. Flexibility, the results suggest, beats purity.

Catalog record

CREATORJanik Kreit, Dominic Schuh, Kim A. Nicoli, Lena Funcke
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19619

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows Janik Kreit, Dominic Schuh, Kim A. Nicoli, Lena Funcke Abstract Deep generative models have recently garnered significant attention across various fields, from physics to chemistry, where sampling from unnormalized Boltzmann-like distributions represents a fundamental challenge. In particular, autoregressive models and normalizing flows have become prominent due to their appealing ability to yield closed-form probability densities. Moreover, it is well-established that incorporating prior knowledge—such as symmetries—into deep neural networks can substantially improve training performances. In this context, recent advances have focused on developing symmetry-equivariant generative models, achieving remarkable results. Building upon these foundations, this paper introduces Symmetry-Enforcing Stochastic Modulation (SESaMo). Similar to equivariant normalizing flows, SESaMo enables the incorporation of inductive biases (e.g., symmetries) into normalizing flows through a novel technique called stochastic modulation . This approach enhances the flexibility of the generative model by enforcing exact symmetries while, for the first time, enabling the model to learn broken symmetries during training. Our numerical experiments benchmark SESaMo in different scenarios, including an 8-Gaussian mixture model and physically relevant field theories, such as the ϕ 4 \phi^{4} theory and the Hubbard model. 1 Introduction Sampling from unnormalized Boltzmann distributions is an ubiquitous yet challenging task across various fields, including physics ( 46 ) , chemistry ( 28 ) , and economics ( 23 ) . These distributions are typically of the form p(\boldsymbol{x})=\exp{(-f[\boldsymbol{x}])}/Z , where f[\cdot] is a functional representing, for example, the potential of a chemical compound or the action of a physical system, while Z , the normalization constant (or partition function), is often unknown. While f[\cdot] is usually available in closed form, as it describes the microscopic dynamics of the system under study, computing Z would require solving a functional or high-dimensional integral, which is generally intractable. In fact, for many systems of interest, sampling from Boltzmann distributions has been proven to be NP-hard ( 5 ) , making it highly unlikely that a polynomial-time algorithm exists for this problem. Due to this complexity, sampling from unnormalized Boltzmann distributions is traditionally performed using Markov Chain Monte Carlo (MCMC) methods ( 3 ) , where a randomly initialized Markov chain is guaranteed to converge to the target distribution. Despite numerous advanced MCMC techniques, significant challenges remain. In chemical and biological systems, for instance, sampling can be hindered by high-energy barriers separating metastable states, posing a major obstacle for tasks such as protein folding ( 25 ) . In physics, MCMC methods often suffer from slow convergence due to autocorrelations between samples, necessitating longer simulations to obtain statistically independent samples and thereby increasing computational costs ( 72 ) . Over the past decade, deep generative models ( 9 ) have achieved remarkable success in sampling from Boltzmann distributions within the framework of variational inference (VI) ( 7 ) . In particular, Ref. ( 54 ) introduced Boltzmann Generators (BGs), an approach in which a variational (parametrized) probability density q_{{\boldsymbol{\theta}}}\in\mathcal{Q} is learned, using a generative model, to approximate the target distribution 1 1 1 For notational convenience, we use the same symbol for a distribution and its density with respect to the Lebesgue measure. of a chemical system, i.e., q_{{\boldsymbol{\theta}}}\approx p . Around the same time, concurrent studies proposed similar ideas in the contexts of statistical physics ( 73 ; 53 ) and lattice quantum field theories ( 2 ; 50 ) . A distinctive feature of BGs is that they rely on generative models capable of providing the learned variational density in closed form. These include autoregressive neural networks ( 67 ; 68 ) and normalizing flows (NFs) ( 58 ; 35 ) , which are particularly suited for sampling discrete and continuous degrees of freedom, respectively. In the remainder of this work, we primarily focus on NFs, although extensions to other generative models that allow exact likelihood computation are also possible. Despite their potential to overcome some limitations of traditional MCMC sampling, deep generative models present challenges of their own. In particular, to ensure reliable sampling from the target density with suitable asymptotic guarantees ( 53 ) , these models must first be trained to a sufficiently high standard. Deep generative models, such as NFs, are parametrized by deep neural networks with numerous trainable parameters, which may require a substantial computational effort (training) to converge. To accelerate training, it has been shown that incorporating inductive biases, such as symmetry constraints, into the model architecture can lead to faster and more robust convergence. A seminal example are convolutional neural networks (CNNs), which exhibit built-in translational equivariance ( 39 ) . This concept has been generalized to arbitrary symmetry groups and manifolds ( 20 ; 18 ; 19 ) . Similar ideas have been extensively leveraged in scientific applications, where chemical and physical systems are often rich in symmetries ( 63 ; 8 ; 60 ; 41 ) . In this paper, we propose a general framework for embedding arbitrary symmetries into the training protocol of NFs, which we term Symmetry-Enforcing Stochastic Modulation (SESaMo). Our approach leverages the prior knowledge (symmetries) from the unnormalized log probability to train a NF and uses an independent random variable to infer the correct probability mass for each mode of the target distribution. Crucially, this approach holds promise for mitigating—and potentially overcoming—the fundamental challenge of mode collapse in variational inference ( 64 ; 52 ) . We focus on a setting in which no samples from the target distribution are available, and mode collapse is particularly prominent. However, we emphasize that our method is not limited to the data-free case. In summary, the contributions of this work are threefold: • We propose Symmetry-Enforcing Stochastic Modulation (SESaMo), a novel approach to incorporate continuous and discrete symmetries (broken and exact) into flow-based models. • We numerically enforce bijectivity by introducing a penalty term in the loss function. • We conduct extensive numerical experiments to validate our theory on both toy problems and real-world benchmarks for lattice quantum field theories. The remainder of this paper is organized as follows. In Sec. 2 , we introduce the necessary background on NFs and variational inference. We also discuss how symmetries can be incorporated into flow-based models and establish the notation used throughout the manuscript. In Sec. 3 , we present our stochastic modulation approach. Finally, in Sec. 4 , we validate our approach, both on a standard benchmark and on tasks of practical relevance, such as sampling lattice quantum field theories, including the \phi^{4} theory and the Hubbard model. In Sec. 5 , we discuss the limitations of our algorithm, and we conclude by summarizing our findings and outlining potential directions for future work in Sec. 6 . 1.1 Related Work The field of geometric deep learning ( 13 ) , which investigates the mathematical foundations of deep learning on geometric structures—particularly group-equivariant and gauge-equivariant neural networks—has advanced significantly in recent years. For a comprehensive review of common methodologies, we refer to Refs. ( 27 ; 12 ) . Köhler et al. ( 36 ) proposed a way to build NFs that are equivariant under the symmetries of the target p , ensuring that the variational distribution q_{{\boldsymbol{\theta}}} inherently respects these symmetries, thereby improving both accuracy and efficiency. This work laid the foundation for

Read the complete text