From Independent to Correlated Diffusion: Generalized Generative Modeling with Probabilistic Computers
Nihal Sanjay Singh, Mazdak Mohseni-Rajaee, Shaila Niazi, Kerem Y. Camsari · 2026
Read From Independent to Correlated Diffusion: Generalized Generative Modeling with Probabilistic Computers by Nihal Sanjay Singh, Mazdak Mohseni-Rajaee, Shaila Niazi, Kerem Y. Camsari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27996.txt.
Introduction
Singh and three co-authors generalize diffusion models by swapping plain noise steps for Markov chain sampling with built-in couplings. Standard diffusion returns when the couplings are set to zero. With Ising couplings active, the sampling uses spatial patterns that match the target system. The approach maps well onto probabilistic computers, which sample far faster and with less energy than GPUs. On a 2D Ising model and a 3D spin glass, the method tracks reference results more closely than standard diffusion.
Catalog record
| CREATOR | Nihal Sanjay Singh, Mazdak Mohseni-Rajaee, Shaila Niazi, Kerem Y. Camsari |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61476 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27996
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Independent to Correlated Diffusion: Generalized Generative Modeling with Probabilistic Computers Nihal Sanjay Singh, Mazdak Mohseni-Rajaee, Shaila Niazi, Kerem Y. Camsari Abstract Diffusion models have emerged as a powerful framework for generative tasks in deep learning. They decompose generative modeling into two computational primitives: deterministic neural-network evaluation and stochastic sampling. Current implementations usually place most computation in the neural network, but diffusion as a framework allows a broader range of choices for the stochastic transition kernel. Here, we generalize the stochastic sampling component by replacing independent noise injection with Markov chain Monte Carlo (MCMC) dynamics that incorporate known interaction structure. Standard independent diffusion is recovered as a special case when couplings are set to zero. By explicitly incorporating Ising couplings into the diffusion dynamics, the noising and denoising processes exploit spatial correlations representative of the target system. The resulting framework maps naturally onto probabilistic computers (p-computers) built from probabilistic bits (p-bits), which provide orders-of-magnitude advantages in sampling throughput and energy efficiency over GPUs. We demonstrate the approach on equilibrium states of the 2D ferromagnetic Ising model and the 3D Edwards-Anderson spin glass, showing that correlated diffusion produces samples in closer agreement with MCMC reference distributions than independent diffusion. More broadly, the framework shows that p-computers can enable new classes of diffusion algorithms that exploit structured probabilistic sampling for generative modeling. I. Introduction Diffusion models are a class of generative models inspired by non-equilibrium thermodynamics Sohl-Dickstein et al. 2015 . They operate through a forward stochastic noising process that progressively corrupts data, a training phase in which a neural network learns to reverse this corruption, and inference , in which new samples are generated by iteratively applying learned, stochastic denoising transitions Ho et al. 2020 ; Austin et al. 2021 . The probabilistic nature of diffusion models means they are inherently composed of two computational components: deterministic neural-network evaluation (a conditional estimator) and stochastic sampling (a transition kernel that realizes the next state). A central but often overlooked aspect of diffusion models is that their algorithmic structure imposes no fundamental constraint on how computational effort is divided between these two components. In current practice, the widespread availability of GPUs has pushed most of the cost into the neural-network evaluation, which is well matched to dense linear algebra, while the stochastic sampling is typically kept lightweight to fit what this hardware makes easiest to execute Hooker 2020 . This is a hardware-contingent design choice rather than a property of the diffusion framework, and it leaves underexplored a complementary regime in which the sampling step is fast enough to justify richer, more informative transition kernels. Probabilistic computers (p-computers) built from probabilistic bits (p-bits) Camsari et al. 2019 ; Chowdhury et al. 2023 and their Gaussian counterparts (g-bits) provide exactly this capability. These p-bits are binary stochastic units that naturally sample from Ising-type energy landscapes, and networks of p-bits implement Gibbs sampling of distributions defined by programmable couplings and biases. They thus constitute a form of Ising machine Mohseni et al. 2022 ; McMahon et al. 2016 : hardware designed to minimize Ising Hamiltonians, but extended here to support controllable stochastic sampling rather than energy minimization alone. Probabilistic computers built from p-bits have demonstrated strong performance in combinatorial optimization and machine learning Borders et al. 2019 ; Aadit et al. 2022 ; Niazi et al. 2024 , with native hardware implementations on FPGAs and projected stochastic magnetic tunnel junction (sMTJ) devices offering orders-of-magnitude advantages in sampling throughput and energy efficiency over conventional GPU-based implementations Aadit et al. 2022 (quantified in Section II , Table 1 ). The mapping of diffusion onto p-computers is particularly relevant because recent proposals for energy-based candidate selection Xu et al. 2025 and Monte Carlo tree search Yoon et al. 2025a ; Yoon et al. 2025b during inference further increase the demand for fast, efficient probabilistic sampling. For physical systems governed by known interactions, such as Ising models with coupling coefficients J_{ij} , there is a further opportunity: the sampling distributions within diffusion can be constructed directly from the correlations of the underlying system. Rather than sampling from independent noise at each diffusion step, the stochastic component can perform structure-aware Gibbs dynamics that respect the physical couplings. We show that allowing diffusion to sample from these physically motivated distributions improves the accuracy of generated samples, and that independent diffusion is recovered as a special case when the couplings are set to zero ( J_{ij}=0 ). Fig. 1 : Hybrid p-computer/GPU inference loop for correlated diffusion. System-level view of one reverse-diffusion iteration. The current noisy configuration ( s_{t} ) is passed from the probabilistic computer to the neural network on the GPU, which produces per-site denoising probabilities p=f_{\theta}(s_{t}) . A binary clean-state estimate \hat{s}_{0} is then sampled from these probabilities and sent back to the probabilistic computer, where it initializes an ensemble of forward Gibbs chains under the known couplings J_{ij} and the scheduled inverse temperatures. Each chain produces a candidate for s_{t-1} , and the candidates are reweighted by the one-step likelihood P(s_{t}\mid s_{t-1}) before one is sampled to continue the reverse trajectory. Repeating this procedure produces successive reverse states s_{t}\rightarrow s_{t-1}\rightarrow\cdots\rightarrow s_{0} , yielding the final sample s_{0} . For systems with known couplings, direct MCMC sampling is available but can require prohibitively long chains to decorrelate, particularly near phase transitions or in frustrated systems. Correlated diffusion offers an alternative route: once trained, the hybrid neural-network/Gibbs reverse process can generate decorrelated approximate samples intended to match the target distribution without requiring long equilibrium chains at inference time. In this sense, the framework is analogous to a Boltzmann generator Noé et al. 2019 for discrete systems, where the neural network proposes global denoising moves and the p-bit sampler enforces local consistency. Recent work on augmenting MCMC with learned proposals Bunaiyan et al. 2025 demonstrates the value of this approach; correlated diffusion extends it by embedding the physical structure directly into the generative dynamics. In this work, we formulate a generalized diffusion framework that incorporates Ising-structured Gibbs dynamics into both the forward noising and reverse inference processes. The framework naturally maps onto a hybrid architecture (Fig. 1 ) in which p-computers handle the stochastic sampling and GPUs handle the neural-network-based conditional estimation. More broadly, any Ising machine capable of controllable stochastic sampling (e.g., Ref. Mohseni et al. 2022 ; Böhm et al. 2022 ; Ng et al. 2022 ; Suzuki et al. 2013 ; Lee et al. 2025a ) can serve as the sampling backend in this framework. We introduce the probabilistic computing primitives, p-bits and g-bits (Fig. 2 ), that implement these dynamics, and quantify their sampling efficiency relative to GPUs (Table 1 ). We demonstrate the approach on two benchmark systems: the 2D ferromagnetic Ising model and the 3D Edwards-Anderson spin glass, showing that correlated diffusion produces samples that align more closely with MCMC reference distributions of equilibrium observables. Fig. 2 : Probabilistic computing hardware primitives. (a) p-bits and g-bits serve as the building blocks for Bernoulli and Gaussian sampling, respectively. (b) Multi-state Potts variables with integer states in [0,2^{N}-1] can be realized either as native Potts units or as networks of interconnected p-bits. (c) A p-bit circuit and candidate stochastic nanodevice realizations, including sMTJs, Zener diodes, and SPADs.