Scalable Multitemperature Free Energy Sampling of Classical Ising Spin States Ping Tuo, Zezhu Zeng, Jiale Chen, Bingqing Cheng Abstract Generative models have advanced significantly in sampling material systems with continuous variables, such as atomistic structures. However, their application to discrete variables, like atom types or spin states, remains underexplored. In this work, we introduce a discrete flow matching model, tailored for systems with discrete phase-space coordinates (e.g., the Ising model or a multicomponent system on a lattice). This approach enables a single model to sample free energy surfaces over a wide temperature range with minimal training overhead, and the model generation is scalable to larger lattice sizes than those in the training set. We demonstrate our approach on the 2D Ising model, showing efficient and reliable free energy sampling. These results highlight the potential of flow matching for low-cost, scalable free energy sampling in discrete systems and suggest promising extensions to alchemical degrees of freedom in crystalline materials. The codebase developed for this work is openly available at https://github.com/tuoping/alchemicalFES . 1 Introduction Estimating the free energy surface (FES) of the alchemical space of crystalline solids with different elements, which is isomorphic to an Ising spin system or a lattice model, has traditionally relied on stochastic sampling methods, such as Markov chain Monte Carlo (MCMC) simulations [ 1 ] . MCMC methods sample by constructing a Markov chain whose equilibrium distribution matches the target distribution, with common approaches like Metropolis-Hastings [ 2 ] , simulated annealing [ 3 ] , and replica exchange [ 4 , 5 ] helping to overcome challenges like metastability and slow convergence. These methods sample from the Boltzmann distribution in the long run, but many simulation steps are needed to produce an uncorrelated sample. One reason is that complex systems often have metastable states and the transitions between them are rare events. For instance, for the Ising model on a 24 \times 24 square lattice at temperature T=0.88\ T_{c} (below the critical temperature T_{c} ), more than 10^{9} MCMC steps are required to flip the overall magnetization direction; at a lower temperature T=0.79\ T_{c} , the flipping failed to happen after 10^{12} MCMC steps. Recently, deep generative models, such as normalizing flows, diffusion, and flow matching, have emerged as promising methods for estimating free energy surfaces (FES). By mapping the complex configurational space with a Boltzmann distribution to a latent space, in which the low-energy configurations of different states lie close to each other, these models enable more efficient sampling [ 6 , 7 , 8 ] . For instance, Wang et al. [ 9 ] used diffusion models to reduce the number of replicas required in replica exchange schemes. Olehnovics et al. [ 10 ] employed normalizing flows for more efficient reweighting in targeted free energy perturbation. Olsson et al. [ 11 ] used a flow matching model to simulate thermal interpolation. Herron et al. [ 12 ] trained a diffusion model to generate temperature-dependent attributes. While Dibak et al. [ 13 ] trained a temperature steerable normalizing flow to generate temperature-dependent attributes. Together, these examples underscore the versatility and potential impact of deep generative modeling for advancing the study of complex free energy landscapes. Although substantial progress has been made in applying generative models to continuous spaces, their use in discrete systems, such as spin lattices, remains comparatively limited. In particular, for the Ising model where the spins take values in \{-1,+1\} , the continuous-space formulations do not apply. A diverse but still emerging set of approaches has been developed, including masked modeling [ 14 , 15 ] , autoregressive models [ 16 , 17 , 18 ] , discrete diffusion models [ 19 , 20 , 21 , 8 , 22 , 23 , 24 ] , and discrete flow matching [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ] . Masked modeling techniques use multiple iterations of masking and filling to gradually reconstruct the entire input [ 14 , 15 ] . Autoregressive models generate discrete sequences element by element, capturing strong dependencies [ 16 , 17 , 18 ] . They can be effective in many settings, though scaling to high-dimensional data requires careful consideration. Discrete diffusion models extend denoising diffusion from continuous to categorical data [ 19 , 20 , 21 , 8 , 22 , 23 , 24 ] , leveraging iterative sampling steps to reconstruct complex distributions. In contrast, discrete flow matching learns a deterministic transformation from a simple distribution to the target distribution without iterative noise addition and removal, offering a faster inference process compared to iterative approaches [ 25 , 26 , 27 , 28 , 29 , 30 ] . In this work, we developed alchemicalFES, a flow matching model in the alchemical space. The model exhibits two key advancements in transferability: (1) it generates the FES across multiple temperatures using one trained model; (2) after being trained on the MCMC data of a small lattice, the model is scalable to lattices of arbitrary sizes. In Section 2 , we detail the specific FM formulations used in this work. After describing the algorithm, in Section 3 , we apply the model to generate the FES of a 2D square-lattice Ising model with the Hamiltonian H=-\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}J_{ij}s_{i}s_{j} where N is the number of lattice sites, s_{i}\in\{-1,+1\} , and J_{ij}=1 if sites i and j are nearest neighbors and 0 otherwise. Finally, in Section 3.1 , we show how to achieve multitemperature generation by using the classifier-free guidance technique [ 34 ] . 2 alchemicalFES Architecture We begin by reviewing foundational work on flow matching in Section 2.1 , where a neural network is employed to simulate an ordinary differential equation. In Section 2.2 , we extend this framework to flow matching based on Dirichlet distributions, a formulation naturally suited for discrete variables. The neural network architecture developed in this work is presented in Section 2.3 . Finally, Section 2.5 details how inference is performed through integration. 2.1 Definition of Flow Matching In flow matching (FM), let the time-indexed random vector \bm{x}\left(t\right)\in\mathbb{R}^{d} for t\in\left[0,1\right] have density p_{t}\left(\bm{x}\right) over \bm{x}\in\mathbb{R}^{d} . Let q\left(\bm{x}\right) denote the source/noise density and p_{\mathrm{data}}\left(\bm{x}\right) the target/data density; we impose the boundary conditions p_{0}=q and p_{1}=p_{\mathrm{data}} . We introduce a time-evolving conditional density p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right) of \bm{x}\left(t\right) conditioned on \bm{x}\left(1\right) with p_{0}\left(\bm{x}\mid\bm{x}\left(1\right)\right)=q\left(\bm{x}\right) and p_{1}\left(\bm{x}\mid\bm{x}\left(1\right)\right)=\delta\left(\bm{x}-\bm{x}\left(1\right)\right) , where \delta is the Dirac delta function. The corresponding marginal density is the mixture p_{t}\left(\bm{x}\right)=\int_{\mathbb{R}^{d}}p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)p_{\mathrm{data}}\left(\bm{x}\left(1\right)\right)\mathrm{d}\bm{x}\left(1\right). (1) For each \bm{x}\left(1\right) , let \bm{u}_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)\in\mathbb{R}^{d} be a conditioned velocity field transporting p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right) via the continuity equation \frac{\partial}{\partial t}p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)+\nabla_{\bm{x}}\cdot\left(p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)\bm{u}_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)\right)=0. (2) The induced marginal velocity field is the posterior average \bm{v}_{t}\left(\bm{x}\right)=\int_{\mathbb{R}^{d}}\bm{u}_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)\,\frac{p_{t}\left(\bm{x}\mid\bm{x}\left(1\right)\right)p_{\mathrm{data}}\left(\bm{x}\left(1\right)\right)}{p_{t}\left(\bm{x}\right)}\mathrm{d}\bm{x}\left(1\right). (3) In practice, we train a neural network to approximate \bm{v}_{t}\left(\bm{x}\right) . Sampling is then obtained by integrating the deterministic flow \frac{\mathrm{d}}{\mathrm{d}t}\bm{x}\left(t\right)=\bm{v}_{t}\left(\bm{x}\left(t\right)\right). (4) Thus, we can generate data \bm{x}\left(1\right)\sim p_{\mathrm{data}}\left(\bm{x}\right) from noisy samples \bm{x}\left(0\right)\sim q\left(\bm{x}\right) . 2.2 Flow Matching on the Simplex For an Ising model, each spin s_{i} at the i -th lattice site can take one of two states: \left\{-1,+1\right\} . We represent these two states with a categorical distribution, using probabilities \bm{x}_{i}=\left(x_{i(0)},x_{i(1)}\right) that satisfy x_{i(0)}+x_{i(1)}=1 , x_{i(0)},x_{i(1)}\geq 0 . Then, each spin state can take two values \bm{x}_{i}(t=1)=(1,0) or (0,1) . To describe the distribution of \bm{x}_{i} , we use the Dirichlet distribution, which has the probability density function [ 35 ] p\left(\bm{x}_{i};\bm{\alpha}\right)=\text{Dir}\left(\bm{x}_{i};\left(\alpha_{0},\alpha_{1}\right)\right)=\frac{\Gamma\left(\alpha_{0}\right)\Gamma\left(\alpha_{1}\right)}{\Gamma\left(\alpha_{0}+\alpha_{1}\right)}x_{i(0)}^{\alpha_{0}-1}x_{i(1)}^{\alpha_{1}-1} (5) with parameters \bm{\alpha}=\left(\alpha_{0},\alpha_{1}\right) , \alpha_{0},\alpha_{1}>0 , and the gamma function \Gamma\left(\cdot\right) [ 36 ] . We then define the noisy prior q to be the uniform distribution, or a Dirichlet distribution with parameter \bm{\alpha}=\left(1,1\right) : q\left(\bm{x}\right)=\text{Dir}\left(\bm{x};\bm{\alpha}=\left(1,1\right)\right)=1. (6) See Supporting Figure S1 (a) for an illustration of the Dirichlet distribution with different \bm{\alpha} parameters. Then, we define a conditional probability path by increasing one of the entries of \bm{\alpha} with time t\in\left[0,1\right] : \displaystyle p_{t}\left(\bm{x}\mid\bm{x}(1)\right)= \displaystyle\mathrm{Dir}\left(\bm{x};\bm{\alpha}=\left(1,1\right)+t\alpha_{\mathrm{max}}\cdot\bm{x}(1)\right) \displaystyle= \displaystyle\begin{cases}\mathrm{Dir}\left(\bm{x};\bm{\alpha}=\left(1+t\alpha_{\mathrm{max}},1\right)\right),&\text{if }\bm{x}(1)=\left(1,0\right)\\ \mathrm{Dir}\left(\bm{x};\bm{\alpha}=\left(1,1+t\alpha_{\mathrm{max}}\right)\right),&\text{if }\bm{x}(1)=\left(0,1\right)\end{cases} (7) where \alpha_{\mathrm{max}}>0 is a hyperparameter. When t\alpha_{\mathrm{max}}\rightarrow\infty , the distribution approaches a \delta distribution at either (1,0) or (0,1) , as shown in Supporting Figure S1 . Now that we have chosen a conditional probability path, we design a conditional velocity field that generates this conditional probability path by \bm{u}_{t}\left(\bm{x}\mid\bm{x}(1)\right)=\begin{cases}C\left(x_{(0)},t\alpha_{\mathrm{max}}\right)\left(\left(1,0\right)-\bm{x}\right),&\text{if }\bm{x}(1)=\left(1,0\right)\\ C\left(x_{(1)},t\alpha_{\mathrm{max}}\right)\left(\left(0,1\right)-\bm{x}\right),&\text{if }\bm{x}(1)=\left(0,1\right).\end{cases} (8) The C\left(x_{(k)},b\right) is derived to be [ 25 ] C\left(x_{(k)},b\right)=-\frac{\partial}{\partial t}I_{x_{(k)}}\left(b+1,1\right)\frac{\mathcal{B}\left(b+1,1\right)}{\left(1-x_{(k)}\right)x_{(k)}^{b}} (9) where I_{x} is the regularized incomplete beta function, and \mathcal{B} is the multivariate beta function \mathcal{B}\left(\alpha_{0},\alpha_{1}\right)=\frac{\Gamma\left(\alpha_{0}\right)\Gamma\left(\alpha_{1}\right)}{\Gamma\left(\alpha_{0}+\alpha_{1}\right)} . It can be proven that the conditional velocity field Eq. 8 and the conditional probability path Eq. 7 together satisfy the transport equation Eq. 2 and therefore constitute a valid flow matching framework [ 25 ] . Figure 1 (a) illustrates the schematic workflow of Dirichlet flow matching. Eqs. 7 - 8 provide an analytical solution for the variance-exploding path in discrete space. This inherently addresses the challenge of normalizing the probability path, a problem traditionally handled by introducing an explicit normalization term in the loss function and enforcing normalization during training [ 27 , 26 , 28 ] . Figure 1: Alchemical flow matching generator. (a) Workflow of the alchemical flow matching model. The spin state of a lattice site is represented as a two-dimensional vector \bm{x}(t)=\left(x_{(0)}(t),x_{(1)}(t)\right) , with \bm{x}(1)=(1,0) if s=-1 and \bm{x}(1)=(0,1) if s=1 . The initial state \bm{x}(0) is sampled from a Dirichlet distribution \text{Dir}\left(\alpha=(1,1)\right) , providing uniform random initialization. As t increases, the Dirichlet distribution sharpens toward the target states (1,0) or (0,1) . (b) A convolutional neural network (CNN) is trained to predict the classifier \bm{g}(t) that guides the probability flow. The input spin configuration \bm{x} and time t are separately featurized, then combined through convolutional layers. The lattice size is denoted as \sqrt{N}\times\sqrt{N} , and N equals the number of spins in a lattice. The model output \bm{g}(t) is trained against one-hot labels corresponding to the final spin states. The CNN model is functionally equivalent to a graph neural network. (c) The workflow for multitemperature flow matching by applying guidance facilitated by combining conditional and unconditional generation. The temperature-dependent parameter \gamma(T) controls the temperature of the generated ensemble. (d) Expanded view of the convolutional layers showing incorporation of conditional embeddings for conditioning variables (magnetization m and potential energy E ). These condition embeddings are added to the feature representations and processed through message-passing blocks. 2.3 Vector Field Model by CNN We train a neural network classifier to predict \bm{g}(t) to approximate \bm{x}(1) that chooses from the two cases of probability path in Eq. 7 . 2.3.1 CNN as a Spin Graph We use a convolutional neural network (CNN) to predict \bm{g}(t) . This CNN is functionally equivalent to a graph neural network, since convolution can be viewed as a specialized form of message passing over the spin graph. As shown in the model architecture of Figure 1 (b), each Ising configuration is represented as a graph, where spins correspond to nodes and undirected edges connect neighboring spins. 2.3.2 Spin State Featurization For each spin i , the input \bm{x}_{i} is fed into a ReLU-activated 1\times 1 convolutional layer that outputs a learnable node representation A_{i} of length N_{\mathrm{embedding}} . N_{\mathrm{embedding}} is chosen to be 128 . In the following, we use A_{i}^{(l)} to denote the node representation at message passing layer l . 2.3.3 Time Embedding Time is encoded by using a Gaussian Fourier projection \sigma_{t}=\left(\sin\left(2\pi\omega t\right),\cos\left(2\pi\omega t\right)\right) (10) where \omega is a learnable weight vector of length N_{\mathrm{embedding}}/2 and the equation outputs a learnable time embedding \sigma_{t} of length N_{\mathrm{embedding}} . 2.3.4 Message Passing by Convolutional Layers At each message-passing layer, \sigma_{t} is first mapped to a representation \mathcal{T}^{(l)} of size N_{\mathrm{embedding}} using a ReLU-activated 1\times 1 convolutional layer. Message passing is then performed using a 3\times 3 convolutional layer M_{i}^{(l)}=\sum_{j\in\mathcal{N}(i)\cup\{i\}}H^{(l)}\left(\vec{r}_{ji}\right)\left(A^{(l)}_{j}+\mathcal{T}^{(l)}\right), (11) where H^{(l)}\left(\vec{r}_{ji}\right) consists of nine matrices of dimension N_{\mathrm{embedding}}\times N_{\mathrm{embedding}} , corresponding to the eight neighbors considered for each spin, along with the self-interaction. The node representations are then updated as A^{(l+1)}_{i}=\text{ReLU}\left(M_{i}^{(l)}\right)+A^{(l)}_{i}. (12) 2.3.5 Readout After L=12 message-passing blocks, the node representations are aggregated using a ReLU-activated 3\times 3 convolutional layer, followed by a 1\times 1 convolutional layer with Softmax activation, yielding a two-class classifier \bm{g}_{i} for each spin i . 2.4 Loss Functions and Training 2.4.1 Cross Entropy Loss Training is conducted via a cross-entropy loss [ 25 ] \mathcal{L}_{\mathrm{CE}}=-\lambda_{\mathrm{CE}}\mathbb{E}_{t}\sum_{i}\left[\bm{x}_{i}(1)\cdot\ln\bm{g}_{i}(t)\right] (13) where \bm{x}_{i}(1) denotes the training data of spin i , represented as a one-hot vector taking values (1,0) or (0,1) . The prefactor \lambda_{\mathrm{CE}} adjusts the relative weight of the loss term during training. 2.4.2 Energy-Based Loss For the Boltzmann distribution, accurate modeling of the probabilities of configurations with multiple interacting spins can be achieved by training with an energy-based loss function [ 6 , 37 , 38 ] . To define the energy-based loss, we first define the energy as a function of spin states \bm{x}_{i} . We use \hat{x}=\{x_{i(k)}\mid i=1,2,\dots,N;k=0,1\} to denote the target state of a configuration with N lattice sites. Similarly, we use \hat{g}=\{g_{i(k)}\mid i=1,2,\dots,N;k=0,1\} to denote the generated classifiers for a configuration with N lattice sites. To calculate the energy E as a function of \hat{x} , we first determine the most likely spin state at each site by s_{i}=-1+2\ \text{argmax}_{k\in\left\{0,1\right\}}x_{i(k)}. (14) Then we compute the local energy for both spin states: E_{i}(s_{i}=-1)=\sum_{j\in\mathcal{N}(i)}s_{j}\quad\text{and}\quad E_{i}(s_{i}=1)=-\sum_{j\in\mathcal{N}(i)}s_{j} . The total energy of the lattice is obtained as E(\hat{x})=\sum_{\bm{x}_{i}\in\hat{x}}\left(\sum_{j\in\mathcal{N}(i)}s_{j}\right)\left(x_{i(0)}-x_{i(1)}\right). (15) The generated classifiers \hat{g}(t) and the target states \hat{x}(1) are used to compute the energies E(\hat{g}(t)) and E(\hat{x}(1)) respectively. Then, we use the energies to define a loss function \mathcal{L}_{\mathrm{E}}=\lambda_{\mathrm{E}}\mathbb{E}_{t}\left[e^{-E\left(\hat{x}(1)\right)/\tau}\left(E\left(\hat{g}(t)\right)-E\left(\hat{x}(1)\right)\right)/\tau\right]+\lambda_{\mathrm{MAE}}\mathbb{E}_{t}\left\|E\left(\hat{g}(t)\right)-E\left(\hat{x}(1)\right)\right\|_{1} (16) where \lambda_{E} is the prefactor and \tau=k_{B}T is the temperature. To avoid numerical instabilities from sharp energy landscapes, training begins at elevated \tau and gradually anneals toward k_{B}T . Here, we use a mean absolute error term \mathbb{E}\left\|E\left(\hat{g}(t)\right)-E\left(\hat{x}(1)\right)\right\|_{1} weighted by a prefactor \lambda_{\mathrm{MAE}} , since it encourages a tighter alignment of the predicted energy E\left(\hat{g}(t)\right) with the reference energy E\left(\hat{x}(1)\right) and accelerates training. 2.4.3 Reaction Coordinate Loss To help the model distinguish important energy degenerate states, we further use a reaction coordinate loss. For the square lattice Ising model, the magnetization m(\hat{x}) is used as reaction coordinate, which is determined in a similar way as the energy: the most likely spin state is determined by Eq. 14 , and the local contribution m_{i}(s_{i}=-1)=-1 and m_{i}(s_{i}=1)=1 are computed, then the total m(\hat{x}) of the lattice is obtained as m\left(\hat{x}\right)=\sum_{i=1}^{N}\left(x_{i(1)}-x_{i(0)}\right) . Then, the probabilities P\left(m\left(\hat{x}(1)\right)\right) and P\left(m\left(\hat{g}(t)\right)\right) are computed by batchwise kernel density estimation over the reaction coordinates of the training samples and the predicted classifiers respectively [ 6 ] . The reaction coordinate loss is defined as \mathcal{L}_{\mathrm{RC}}=\lambda_{\mathrm{RC}}\mathbb{E}_{t}D_{\mathrm{KL}}\left[P\left(m\left(\hat{x}(1)\right)\right)\|P\left(m\left(\hat{g}(t)\right)\right)\right] (17) where \lambda_{\mathrm{RC}} is the prefactor. 2.4.4 Training Models are trained with the Adam optimizer [ 39 ] using an initial learning rate of 5\times 10^{-4} over batch sizes of 1,024 lattice configurations. Details on the scheduling of prefactors for different loss functions are provided in the Supporting Section VI . 2.5 Flow Matching Inference At inference, we parameterize the marginal vector field using the model prediction \bm{g}_{i}(t) via [ 25 ] \bm{v}_{i}(t)=g_{i(0)}(t)\bm{u}_{t}\left(\bm{x}_{i}\mid\bm{x}_{i}(1)=(1,0)\right)+g_{i(1)}(t)\bm{u}_{t}\left(\bm{x}_{i}\mid\bm{x}_{i}(1)=(0,1)\right) (18) where \bm{u}_{t}\left(\bm{x}\mid\bm{x}(1)\right) is given in Eq. 8 . The marginal velocity is integrated over time to trace the marginal probability path by \bm{x}_{i}(1)-\bm{x}_{i}(0)=\int_{0}^{1}\bm{v}_{i}(t)\mathrm{d}t. (19) In practice, the probability path converges to the target distribution \delta(\bm{x}_{i}-\bm{x}_{i}(1)) at t\alpha_{\mathrm{max}}\geq 9 , as shown in Supporting Figure S1 , so we set \alpha_{\mathrm{max}}=9 . We employed 80 integration steps, as additional steps yield essentially the same results. 3 Reproducing the Free Energy Surface of the Ising Model We first apply alchemicalFES to generate the FES of the 2D square-lattice Ising model. The training set was generated using MCMC sampling of a 6\times 6 lattice Ising model at a given target temperature. Since the goal is to train the model to accurately capture the statistics of the dataset, the dataset must be large enough to contain a reasonable number of low-probability samples. In this work, we use a dataset with 1 million samples. The trained model was then utilized to generate samples for larger lattice sizes. Ensemble averaging was then used to construct the FES in an order parameter space. Specifically, we used two order parameters: the magnetization per spin ( m/N=\frac{1}{N}\sum_{i=1}^{N}s_{i} ) and the potential energy per spin ( E/N=-\frac{1}{N}\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}J_{ij}s_{i}s_{j} ). Figure 2: Free energy estimations from flow matching model (dots) for Ising lattice of different sizes, compared against the free energy surface from MCMC simulations (lines). Free energy as a function of E/N=-\frac{1}{N}\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}J_{ij}s_{i}s_{j} at (a) k_{B}T=4.0 , (d) k_{B}T=3.2 , (g) k_{B}T=2.2 ; free energy as a function of m/N=\frac{1}{N}\sum_{i=1}^{N}s_{i} at (b) k_{B}T=4.0 , (e) k_{B}T=3.2 , (h) k_{B}T=2.2 ; pair correlation function (PCF) at (c) k_{B}T=4.0 , (f) k_{B}T=3.2 , (i) k_{B}T=2.2 . There is a kink at E/N\approx-1.667 due to the finite-size effect of the Ising model. We explain this peculiar phenomenon in Supporting Section VIII . The results are shown in Figure 2 . For each lattice site, only eight neighbors are considered. As a result, the model prediction is size scalable under the limit of short-range correlations that rapidly diminish with distance. This assumption is valid at high temperatures T>T_{c} , where thermal fluctuations disrupt long-range order, as shown in Figure 2 (a,b,d,e). The predicted FES of k_{B}T=4.0 and k_{B}T=3.2 closely match the reference FES for various lattice sizes. Figure 2 (c,f) shows the pair correlation function (PCF). For both 6\times 6 lattices and 24\times 24 lattices, the predicted PCF matches the references very well. However, for low temperatures, the system exhibits long-range correlation and the naive flow matching approach no longer maintains size scalability, as shown in Figure 2 (h,i). 3.1 Size Scalable Multitemperature Generation In this section, we demonstrate how to generate samples across multiple temperatures with a single model by leveraging the classifier-free guidance technique and how this approach leads to size scalability even at low temperatures with long-range correlation. In the following, we briefly review the guidance technique and then describe how we adapted it to enable size-scalable generation across multiple temperatures. 3.1.1 Guidance Technique A key feature of iterative generative models is their ability to progressively bias the generative process toward specific target distributions, a concept known as guidance [ 40 , 34 ] . Guidance was originally introduced in the context of diffusion models to learn the score \bm{\epsilon}(\bm{x};t)=\nabla_{\bm{x}}\ln p_{t}(\bm{x}) of the distribution of noisy data. In particular, classifier guidance modifies the score by incorporating the gradient of an auxiliary classifier’s log-likelihood [ 40 ] \bm{\epsilon}\left(\bm{x},c;t\right)=\nabla_{\bm{x}}\ln p_{t}\left(\bm{x}\right)+\gamma\nabla_{\bm{x}}\ln p\left(c\mid\bm{x}\right) (20) where c denotes the desired class, and \gamma controls the strength of the classifier guidance. To remove the need for a separate classifier model, Ho and Salimans [ 34 ] introduced classifier-free guidance , which substitutes p\left(c\mid\bm{x}\right)=p_{t}\left(\bm{x}\mid c\right)p_{t}\left(c\right)/p_{t}\left(\bm{x}\right) into Eq. ( 20 ), leading to \bm{\epsilon}\left(\bm{x},c;t\right)=\gamma\nabla_{\bm{x}}\ln p_{t}\left(\bm{x}\mid c\right)+\left(1-\gamma\right)\nabla_{\bm{x}}\ln p_{t}\left(\bm{x}\right) (21) where we train a conditional model to learn \nabla_{\bm{x}}\ln p_{t}\left(\bm{x}\mid c\right) . 3.1.2 Relationship between Flow and Score For the Dirichlet probability path, the score can be obtained from the model posterior via the denoising score-matching identity [ 41 ] \bm{\epsilon}_{t}(\bm{x})=\bm{\epsilon}_{t}\left(\bm{x}\mid\bm{x}(1)=(1,0)\right)p_{t}\left(\bm{x}\mid\bm{x}(1)=(1,0)\right)+\bm{\epsilon}_{t}\left(\bm{x}\mid\bm{x}(1)=(0,1)\right)p_{t}\left(\bm{x}\mid\bm{x}(1)=(0,1)\right) (22) where p_{t}\left(\bm{x}\mid\bm{x}(1)\right) is given in Eq. 7 . Let \bm{p}=\left(p_{t}\left(\bm{x}\mid\bm{x}(1)=(1,0)\right),p_{t}\left(\bm{x}\mid\bm{x}(1)=(0,1)\right)\right) be a vector containing the two cases of Eq. 7 . We can differentiate \bm{p} w.r.t. \bm{x} to obtain a 2\times 2 Jacobian matrix \mathrm{diag}\left(\bm{\epsilon}\right)=D\mathrm{\ diag}\left(\bm{p}\right) where \bm{\epsilon}\in\mathbb{R}^{2} and D=\mathrm{diag}\left(t/\bm{x}\right)\in\mathbb{R}^{2\times 2} . We can rewrite the Jacobian equation to \bm{\epsilon}=D\bm{p} . Meanwhile, the computation of the marginal flow (Eq. 18 ) can also be written as a very similar matrix equation \bm{v}=U\bm{g} where \bm{g}=\left(g_{(0)}(t),g_{(1)}(t)\right) and the entries of U are given by Eq. 8 : U=\left[\begin{matrix}\bm{u}_{t}\left(\bm{x}\mid\bm{x}(1)=(1,0)\right)\\ \bm{u}_{t}\left(\bm{x}\mid\bm{x}(1)=(0,1)\right)\end{matrix}\right]^{T}\in\mathbb{R}^{2\times 2}. (23) Using \bm{g}\approx\bm{p} , we obtain \bm{v}=U\bm{g}=U\bm{p}=UD^{-1}\bm{\epsilon}. (24) Thus, a linear relationship exists between the marginal flow \bm{v} and the score \bm{\epsilon} arising from the model posterior \bm{p} . Suppose we have conditional and unconditional flow models \bm{v}\left(\bm{x};t\mid c\right) and \bm{v}\left(\bm{x};t\right) . Since a linear combination of scores results in a linear combination of flows, we similarly implement guidance to the flows by integrating \bm{v}_{\mathrm{CFG}}\left(\bm{x},c;t\right)=\gamma\bm{v}\left(\bm{x};t\mid c\right)+\left(1-\gamma\right)\bm{v}\left(\bm{x};t\right). (25) 3.1.3 Multitemperature Generation via Classifier-Free Guidance A straightforward way to enable multitemperature generation with a single model is to treat temperature as a conditioning variable. Schebek et al. [ 37 ] demonstrated a realization of this by using conditional normalizing flows, where temperature and pressure were used as input features to predict free energy differences between solid and liquid phases. However, they noted that this approach required more advanced model architectures and much longer training times compared to their earlier work, which addressed prediction under a single thermal condition. Moreover, extending it to reproduce the entire FES under multiple conditions would demand even greater training effort. In this work, we propose a two-step approach to accomplish multitemperature generation without increasing model capacity or incurring substantially greater training costs. Step 1: Using Order Parameters as Conditioning Variables. The first idea is to include more information in the conditioning variables. Rather than conditioning only on the temperature, we condition on the order parameters of a Ising model at the given temperature. For the Ising model defined by the Hamiltonian H=-\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}J_{ij}s_{i}s_{j} , we condition our flow model on two order parameters: the magnetization m=\sum_{i=1}^{N}s_{i} and the energy E=H . Figure 1 (d) illustrates the modified convolutional layers incorporating condition embeddings. Since the energy E and magnetization m of a 6\times 6 lattice Ising model take discrete values, we employed an embedding lookup of length N_{\mathrm{embedding}} to represent the E and m conditions as learnable embeddings e(E) and e(m) . The index set for m is \left\{n\mid-36\leq n\leq 36,n\equiv 0\pmod{2}\right\} , and the index set for E is \left\{n\mid-72\leq n\leq 72,n\equiv 0\pmod{4}\right\} . For larger lattices, we rescale E and m by N/36 to map them to the lookup range, where N denotes the number of lattice sites. Then, a ReLU-activated linear layer maps e(E) and e(m) to energy and magnetization features of length N_{\mathrm{embedding}} at message passing layer l : \mathcal{E}^{(l)} and \mathcal{M}^{(l)} . The message passing is augmented with energy and magnetization features, yielding M_{i}^{(l)}=\sum_{j\in\mathcal{N}(i)\cup\{i\}}H^{(l)}\left(\vec{r}_{ji}\right)\left(A^{(l)}_{j}+\mathcal{T}^{(l)}+\mathcal{E}^{(l)}+\mathcal{M}^{(l)}\right). (26) The conditional model was trained on the data of k_{B}T=3.2,\ 2.8,\ 2.4,\ 2.2,\ 2.0,\ 0.0 . Figure 3 shows the FES generated for a 6\times 6 lattice under this scheme. Figure 3: Conditional generation (dots) of 6\times 6 lattice Ising model using order parameter conditions, compared against MCMC data (lines). (a) Free energy as a function of E/N=-\frac{1}{N}\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}J_{ij}s_{i}s_{j} ; (b) free energy as a function of m/N=\frac{1}{N}\sum_{i=1}^{N}s_{i} ; and (c) pair correlation function (PCF). Step 2: Size-Scalable Generation with Temperature-Dependent Guidance. However, conditional generation with order parameter conditions alone cannot scale to larger system sizes, because we lack the order parameters for bigger lattices. To address scalability, we turn to the guidance technique. First, we define the guided score of the spin i as \begin{split}\bm{\epsilon}_{\mathrm{CFG},i}\left(\hat{x},E,m;t\right)&=\gamma\nabla_{\bm{x}_{i}}\ln P\left(\hat{x};t\mid E,m\right)+\left(1-\gamma\right)\nabla_{\bm{x}_{i}}\ln P\left(\hat{x};t\right)\\ &=\nabla_{\bm{x}_{i}}\ln\left[P\left(\hat{x};t\mid E,m\right)^{\gamma}P\left(\hat{x};t\right)^{1-\gamma}\right]\end{split} (27) where P(\hat{x},t) denotes the predicted probabilities of sample \hat{x} at time t . This implies the guided distribution P_{\mathrm{CFG}}\left(\hat{x},E,m;t\right)=\frac{P\left(\hat{x};t\mid E,m\right)^{\gamma}P\left(\hat{x};t\right)^{1-\gamma}}{Z_{\mathrm{CFG}}\left(t,E,m\right)} (28) where Z_{\mathrm{CFG}}\left(t,E,m\right) is a normalizing constant. The Boltzmann probability of a sample \hat{x} at a target temperature T is P_{T}\left(\hat{x}\right)=\frac{1}{Z_{T}}\exp\left(-\frac{E\left(\hat{x}\right)}{k_{B}T}\right) (29) where Z_{T} is a normalizing constant. The score of spin i is \nabla_{\bm{x}_{i}}\ln P_{T}\left(\hat{x}\right)=-\frac{1}{k_{B}T}\nabla_{\bm{x}_{i}}E\left(\hat{x}\right). (30) We denote the temperature of the ensemble generated by conditional FM by T^{\mathrm{cond}} , and that generated by unconditional FM by T^{\mathrm{uncond}} . Their respective scores at t=1 are \frac{1}{k_{B}T^{\mathrm{cond}}}\nabla_{\bm{x}_{i}}E\left(\hat{x}(1)\right) and \frac{1}{k_{B}T^{\mathrm{uncond}}}\nabla_{\bm{x}_{i}}E\left(\hat{x}(1)\right) respectively. Then, the guided score Eq. ( 27 ) at t=1 can be written as \bm{\epsilon}_{\mathrm{CFG},i}\left(\hat{x},E,m;1\right)=-\frac{\gamma}{k_{B}T^{\mathrm{cond}}}\nabla_{\bm{x}_{i}}E\left(\hat{x}(1)\right)-\frac{1-\gamma}{k_{B}T^{\mathrm{uncond}}}\nabla_{\bm{x}_{i}}E\left(\hat{x}(1)\right). (31) \bm{\epsilon}_{\mathrm{CFG},i}\left(\hat{x},E,m;1\right) coincides with the score of the Boltzmann distribution at the target temperature T if the guidance parameter \gamma is chosen to satisfy \frac{1}{k_{B}T}=\frac{\gamma}{k_{B}T^{\mathrm{cond}}}+\frac{1-\gamma}{k_{B}T^{\mathrm{uncond}}}, (32) i.e., \gamma=\frac{T^{\mathrm{cond}}\left(T^{\mathrm{uncond}}-T\right)}{T\left(T^{\mathrm{uncond}}-T^{\mathrm{cond}}\right)}, (33) thus ensuring that \bm{\epsilon}_{\mathrm{CFG},i}\left(\hat{x},E,m;1\right) reproduces the correct Boltzmann distribution at temperature T . Furthermore, because the Dirichlet flow follows a linear relationship with the score, we can obtain the corresponding flow using the guided score by Eq. 24 . The multitemperature generation algorithm is depicted in Figure 1 (c). Size scalability is maintained even at low temperatures under this scheme. When generating configurations at multiple temperatures for lattice sizes larger than those in the training set, we use as input conditions the order parameters of the magnetic states (i.e., all spins up or all spins down, whose magnetizations m and energies E are readily calculated), which leads to the generation of a low-temperature ensemble. The desired temperature can then be obtained by tuning the \gamma parameter in Eq. 31 according to Eq. 32 . The classifier-free guidance technique here allows us to reuse the same flow model to generate for multiple temperatures with minimal architectural changes or additional training effort. 3.1.4 Reproducing the Free Energy Surface of the Ising Model at Multiple Temperatures Figure 4: (a) Free energy estimations for 24\times 24 lattice Ising model at multiple temperatures obtained by the guided FM model, trained with the MCMC data of 6\times 6 lattice Ising model, are compared against the MCMC free energies (lines). The shaded region illustrates the 97.5% confidence interval of the estimated free energy. (b) Potential energy expectations at multiple temperatures predicted by the guided FM model (crosses), compared against MCMC data (lines). The vertical lines indicate the critical temperatures of phase transition for different lattice sizes, where T_{c}=\frac{2}{\ln(1+\sqrt{2})} in the infinite lattice limit [ 42 ] . T_{c}\approx 2.43 and T_{c}\approx 2.85 are estimated for 6\times 6 and 4\times 4 Ising lattice respectively [ 43 ] , by the renormalization group theory [ 44 , 45 , 46 ] . (c) Pair correlation function of multiple temperatures obtained by the guided FM model, compared against the MCMC data. Using the magnetic states as input conditions, the exact value of k_{B}T^{\mathrm{cond}} is determined by fitting the generated FES of a 6\times 6 lattice via Eq. ( 28 ) to a reference FES of the training data. Specifically, we iteratively adjust \gamma until the generated FES best matches the reference. Substituting the resulting \gamma back into Eq. ( 28 ) yields the desired k_{B}T^{\mathrm{cond}} . By fitting the FES at k_{B}T=2.2 , we obtained k_{B}T^{\mathrm{cond}}\approx 0.872 . The corresponding \gamma(T) values are shown in Supporting Figure S2 . Next, by applying Eq. 28 using the conditional model and the unconditional flow model of k_{B}T=3.2 , we were able to generate the full temperature range within 0.872