Scalable Maximum Entropy Population Synthesis via Persistent Contrastive Divergence

Mirko Degli Esposti · 2026

Read Scalable Maximum Entropy Population Synthesis via Persistent Contrastive Divergence by Mirko Degli Esposti (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27312.txt.

Introduction

Degli Esposti makes maximum entropy population synthesis scale with a persistent contrastive divergence solver. Synthetic ones are built from census totals without person-level records. Exact methods must sum over every attribute combination, which blows up past twenty fields. His GibbsPCDSolver keeps a pool of fake people instead. Gibbs sweeps update the pool at each gradient step to estimate the means. On an Italian demographic benchmark, constraint error stayed near three percent. They also stayed far more diverse than raked ones.

Catalog record

CREATORMirko Degli Esposti
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78288
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27312

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scalable Maximum Entropy Population Synthesis via Persistent Contrastive Divergence Mirko Degli Esposti Abstract Maximum entropy (MaxEnt) modelling provides a principled framework for generating synthetic populations from aggregate census data, without access to individual-level microdata. The bottleneck of existing approaches is exact expectation computation, which requires summing over the full tuple space 𝒳 \mathcal{X} and becomes infeasible for more than K ≈ 20 K\approx 20 categorical attributes. We propose GibbsPCDSolver , a stochastic replacement for this computation based on Persistent Contrastive Divergence (PCD): a persistent pool of N N synthetic individuals is updated by Gibbs sweeps at each gradient step, providing a stochastic approximation of the model expectations without ever materialising 𝒳 \mathcal{X} . We validate the approach on controlled benchmarks and on Syn-ISTAT , a K = 15 K{=}15 Italian demographic benchmark with analytically exact marginal targets derived from ISTAT-inspired conditional probability tables. Scaling experiments across K ∈ { 12 , 20 , 30 , 40 , 50 } K\in\{12,20,30,40,50\} confirm that GibbsPCDSolver maintains MRE ∈ [ 0.010 , 0.018 ] \mathrm{MRE}\in[0.010,0.018] while | 𝒳 | |\mathcal{X}| grows eighteen orders of magnitude, with runtime scaling as O ⁡ ( K ) O(K) rather than O ⁡ ( | 𝒳 | ) O(|\mathcal{X}|) . On Syn-ISTAT, GibbsPCDSolver reaches MRE = 0.03 \mathrm{MRE}{=}0.03 on training constraints and—crucially—produces populations with effective sample size N eff = N N_{\mathrm{eff}}=N versus N eff ≈ 0.012 ​ N N_{\mathrm{eff}}\approx 0.012\,N for generalised raking, an 86.8 × 86.8{\times} diversity advantage that is essential for agent-based urban simulations. 1 Introduction Urban digital twins and agent-based city models require synthetic populations: large collections of individual profiles whose joint statistical distribution matches the demographic structure of a real municipality. In practice, the only available data are aggregate contingency tables published by national statistical offices—marginal distributions and low-order cross-tabulations derived from census records. Individual-level microdata, when they exist at all, are subject to privacy restrictions that make them inaccessible for most research purposes. The maximum entropy (MaxEnt) principle provides the canonical solution to this inference problem ( Jaynes 1957 ) : given a set of empirical marginal targets, select the distribution over all possible individual profiles that matches those targets while remaining maximally non-committal about everything else. The resulting log-linear model is fully determined by the constraint targets, requires no microdata, and encodes only the dependencies explicitly supported by the data. Wu et al. 2018 formalise this framework for categorical population synthesis, and Pachet and Zucker 2026 provide a convex dual formulation with an efficient L-BFGS solver and a systematic comparison with the dominant practical baseline, generalised raking. The scalability bottleneck. The central computational challenge of MaxEnt population synthesis is that evaluating the model gradient requires computing expectations over the full tuple space \mathcal{X} : \hat{\alpha}_{j}(\bm{\lambda})=\sum_{\mathbf{x}\in\mathcal{X}}p_{\bm{\lambda}}(\mathbf{x})\,f_{j}(\mathbf{x}). For K categorical attributes with domain sizes d_{1},\ldots,d_{K} , we have |\mathcal{X}|=\prod_{k}d_{k} . With K{=}15 Italian demographic attributes, |\mathcal{X}|\approx 1.7\times 10^{8} —at the limit of feasibility. For realistic urban models with K\geq 20 attributes, |\mathcal{X}|>10^{13} and exact enumeration is impossible. Pachet and Zucker 2026 acknowledge this in their Section 5.1, noting Gibbs sampling as a direction for future work. Limitations of generalised raking. Raking (iterative proportional fitting, IPF) avoids the enumeration problem by reweighting a fixed sample rather than estimating a distribution. It is fast, simple, and converges to zero training error by construction. However, raking has two structural limitations that are not visible from the training MRE alone. First, for K\gtrsim 28 with ternary constraints, the sequential rescaling steps interfere destructively and raking diverges ( Pachet and Zucker 2026 ) . Second—and less widely appreciated—raking produces degenerate populations: after convergence, the effective sample size N_{\mathrm{eff}}=(\sum_{i}w_{i}^{2})^{-1} is a small fraction of the nominal population size N . In our experiments, raking achieves N_{\mathrm{eff}}\approx 0.012\,N with a Gini coefficient of 0.951 on the weight distribution. A synthetic city of 400{,}000 inhabitants drawn from this distribution would consist of repetitions of fewer than 1{,}200 distinct demographic archetypes, rendering agent-based dynamics unrealistic. This paper. We propose GibbsPCDSolver , a drop-in replacement for the exact expectation step in Algorithm 1 of Pachet and Zucker 2026 based on Persistent Contrastive Divergence (PCD) ( Tieleman 2008 ) . A persistent pool of N synthetic individuals is maintained and advanced by Gibbs sweeps at each gradient step; the pool’s empirical frequencies provide a stochastic approximation of \hat{\bm{\alpha}}(\bm{\lambda}) without ever materialising \mathcal{X} or computing Z(\bm{\lambda}) . The modification is surgical : one line of Algorithm 1 changes; the exponential family structure, the dual objective, and the Adam optimiser are unchanged. We make the following contributions: 1. Algorithm. GibbsPCDSolver ( section 3 ): the Gibbs conditional derivation, the attr_lookup precomputation for vectorised energy accumulation, the adaptive stopping rule, and practical hyperparameter guidelines derived from experiments. 2. Benchmark. Syn-ISTAT ( section 4 ): a K{=}15 Italian demographic benchmark with analytically exact marginal targets derived from ISTAT-inspired conditional probability tables, and a training/held-out split that tests generalisation to third-order interactions. 3. Empirical findings. GibbsPCDSolver converges cleanly at K{=}15 (MRE =0.030 after 535 iterations), outperforms raking on held-out ternary constraints involving high-degree graph nodes, and produces populations with effective sample size 86.8{\times} larger and entropy 3.15 nats higher than raking—essential properties for agent-based urban simulation. Organisation. Section 2 reviews MaxEnt population synthesis, raking, deep generative models, and PCD. Section 4 describes the Syn-ISTAT benchmark. Section 3 presents the algorithm, including the Gibbs conditional derivation, pseudocode, and validation on controlled experiments. Section 5 reports five experimental results: scaling beyond exact enumeration, held-out generalisation, population diversity, and hyperparameter sensitivity at K{=}8 and K{=}15 . Section 6 situates the findings within the competence map of Pachet and Zucker 2026 and discusses limitations and open questions. Full CPT specifications for the Syn-ISTAT benchmark are provided in appendix A . 2 Background 2.1 Maximum Entropy Population Synthesis The problem of generating a synthetic population from aggregate census data can be framed as a maximum entropy problem. Given a finite tuple space \mathcal{X}=D_{1}\times\cdots\times D_{K} of categorical attributes and a set of empirical marginal targets \{\alpha_{j}\}_{j=1}^{m} derived from published contingency tables, the MaxEnt principle ( Jaynes 1957 ) selects the unique distribution over \mathcal{X} that matches all targets while remaining maximally non-committal about everything else. The resulting model is the exponential family in eq. 3 ; its log-linear structure implies that observed correlations are modelled only to the extent required by the constraints. Wu et al. 2018 formalise this framework for population synthesis and propose an efficient iterative scaling algorithm exploiting the sparse structure of the constraint indicator matrix. Their experiments on US census data demonstrate that the MaxEnt model reproduces marginal statistics faithfully and produces synthetic populations suitable for downstream epidemic simulation. Pachet and Zucker 2026 substantially extend this line of work by introducing a convex dual formulation, an L-BFGS solver, and a systematic comparison with generalised raking. Their central empirical contribution is a competence map : a grid of relative accuracy over population size n and number of variables K , showing that MaxEnt overtakes raking when K\gtrsim 28 and ternary interactions are present. The present paper extends the Pachet–Zucker framework to the regime K>20 where exact expectation computation is infeasible, by replacing the exact sum in their Algorithm 1 (line 4) with a stochastic Gibbs estimate. The exponential family also arises

Read the complete text