SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion

Sedjro Salomon Hotegni, Sebastian Peitz · 2025

Read SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion by Sedjro Salomon Hotegni, Sebastian Peitz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21058.txt.

Introduction

Design often means trading off goals that pull in opposite ways. Sedjro Salomon Hotegni and Sebastian Peitz present SPREAD, a 2025 method for mapping that trade-off curve, called the Pareto front. Their method is based on diffusion models that generate candidate designs.

SPREAD first learns how sampled designs are spread through the decision space. As it generates, each step refines the current candidates in two ways. An adaptive gradient-style update speeds the move toward optimal trade-offs. A repulsion term, built from Gaussian kernels, keeps the candidates spread out along the front. The authors test SPREAD on standard multi-objective benchmarks, in both offline and Bayesian settings. The target is large, costly problems where each test of a design is expensive.

Catalog record

CREATORSedjro Salomon Hotegni, Sebastian Peitz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-21058

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion Sedjro Salomon Hotegni, Sebastian Peitz Abstract Developing efficient multi-objective optimization methods to compute the Pareto set of optimal compromises between conflicting objectives remains a key challenge, especially for large-scale and expensive problems. To bridge this gap, we introduce SPREAD, a generative framework based on Denoising Diffusion Probabilistic Models (DDPMs). SPREAD first learns a conditional diffusion process over points sampled from the decision space and then, at each reverse diffusion step, refines candidates via a sampling scheme that uses an adaptive multiple gradient descent-inspired update for fast convergence alongside a Gaussian RBF–based repulsion term for diversity. Empirical results on multi-objective optimization benchmarks, including offline and Bayesian surrogate-based settings, show that SPREAD matches or exceeds leading baselines in efficiency, scalability, and Pareto front coverage. 1 Introduction Multi-objective optimization (MOO) is fundamental in numerous scientific and engineering disciplines, where decision-makers often face the challenge of optimizing conflicting objectives simultaneously ( Rangaiah 2016 ; Malakooti 2014 ; Zhang et al. 2024b ) . The primary aim is to identify the Pareto front: a set of non-dominated solutions where improving one objective would deteriorate at least one other. Traditional methods for approximating the Pareto front include evolutionary algorithms ( Deb 2011 ; Zhou et al. 2011 ) , scalarization techniques ( Braun et al. 2015 ; Hotegni & Peitz 2025 ) , and multiple-gradient descent (MGD) ( Désidéri 2012 ; Sener & Koltun 2018 ) combined with multi-start techniques (i.e., using random initial guesses to obtain multiple points). However, these approaches may struggle with scalability, especially in high-dimensional or resource-constrained settings ( Cheng et al. 2021 ; Li et al. 2024a ) . As a workaround, domain-specific MOO algorithms have been developed to leverage domain knowledge to improve efficiency and solution quality in targeted settings (e.g., offline MOO ( Yuan et al. 2025b ) , Bayesian MOO ( Daulton et al. 2022 ) , or federated learning ( Hartmann et al. 2025 ) ), but at the expense of broader applicability. These challenges underscore the need for MOO methods capable of efficiently adapting to large-scale, high-dimensional, and computationally expensive problem settings. Developing such universal approaches would not only streamline the optimization process, but would also broaden the applicability of MOO techniques across different domains. Recent advancements in generative modeling have shown promise in addressing complex optimization problems ( Garciarena et al. 2018 ; Yuan et al. 2025a ) . In particular, diffusion models such as Denoising Diffusion Probabilistic Models (DDPMs), have demonstrated remarkable capabilities in generating high-quality samples across various domains ( Ho et al. 2020 ; Yang et al. 2023 ) . Their iterative refinement process aligns well with the principles of MOO, offering a potential pathway to efficiently approximate the Pareto front. In this work, we introduce SPREAD ( S ampling-based P areto front R efinement via E fficient A daptive D iffusion), a novel diffusion-driven generative framework designed to tackle multi-objective optimization across diverse problem settings. Our approach leverages the strengths of diffusion models to iteratively generate and refine candidate solutions, guiding them towards Pareto optimality. SPREAD applies a conditional diffusion modeling approach, where a conditional DDPM is trained on points sampled from the input space, allowing the model to effectively learn the underlying structure of the objective functions and steer the generation process toward promising regions. To further enhance convergence towards Pareto optimality, SPREAD incorporates an adaptive guidance mechanism inspired by the multiple gradient descent algorithm ( Désidéri 2012 ) , dynamically guiding the sampling process to regions likely to contain optimal solutions. Furthermore, to promote diversity among the generated solutions and ensure an approximation of the entire Pareto front, SPREAD utilizes a Gaussian RBF repulsion mechanism ( Buhmann 2000 ) that discourages clustering, mitigates mode collapse and encourages exploration in the objective space. We evaluate SPREAD on diverse MOO problems including two challenging, resource‑constrained scenarios: offline multi-objective optimization ( Xue et al. 2024 ) and Bayesian multi-objective optimization ( Knowles 2006 ) . In each case, we benchmark our method against state-of-the-art approaches specifically tailored for these settings. Our empirical results demonstrate that SPREAD not only achieves competitive performance but also offers superior scalability and adaptability across different problem domains. Our contributions can be summarized as follows: (a) We propose a novel diffusion-based generative framework for MOO that effectively approximates the Pareto front. (b) We introduce a novel conditioning approach along with an adaptive guidance mechanism inspired by MGD to improve convergence towards Pareto optimal solutions. (c) We implement a diversity-promoting strategy to ensure a comprehensive and well-distributed set of solutions. (d) We validate our approach on challenging MOO tasks, demonstrating its effectiveness and generalizability. 2 Related Work We now situate our approach within prior work on generative modeling for multi-objective optimization, and gradient-based methods relevant to our settings. An extended discussion of related work is provided in Appendix E . Generative Modeling for Multi-Objective Optimization Recent work explores alternatives to traditional search methods, such as evolutionary algorithms or acquisition-based optimization, by directly generating Pareto optimal candidates. ParetoFlow ( Yuan et al. 2025a ) uses flow-matching with a multi-objective predictor-guidance module to steer samples toward the front in the offline setting, showing that guided generative samplers can cover non-convex fronts efficiently. In parallel, PGD-MOO ( Annadani et al. 2025 ) trains a dominance-based preference classifier and uses it for diffusion guidance to obtain diverse Pareto optimal designs from data. For Bayesian settings, CDM-PSL ( Li et al. 2025a ) couples unconditional/conditional diffusion with Pareto set learning to propose candidate points under tight evaluation budgets. Our approach differs by conditioning a diffusion transformer on objectives and applying step-wise, MGD-inspired guidance together with an explicit diversity force, yielding both convergence and spread without a separate preference classifier. Gradient-based Methods for Pareto Set Discovery A complementary line of research explicitly profiles the Pareto set by moving a population with repulsive interactions or by smoothing scalarizations. PMGDA ( Zhang et al. 2025 ) extends the classical MGDA ( Désidéri 2012 ) by sampling multiple descent directions in a probabilistic manner, thus improving stability and coverage in high dimensions. Smooth Tchebycheff scalarization (STCH) provides a lightweight differentiable scalarization with favorable guarantees ( Lin et al. 2024 ) . Leveraging hypervolume gradients, HVGrad ( Deist et al. 2021 ) updates solutions toward the Pareto front while preserving diversity, while MOO-SVGD ( Liu et al. 2021 ) employs Stein variational gradients to transport particles and obtain well-spaced fronts. Extending this line of work, SPREAD integrates MGD directions into a DDPM denoising process, using them as adaptive guidance signals within diffusion sampling. 3 Preliminaries To set the stage for our method, we review the fundamental concepts on which our approach is based. 3.1 Denoising Diffusion Probabilistic Models (DDPMs) As powerful generative models, Denoising Diffusion Probabilistic Models excel at producing high-quality samples in a wide range of applications, such as image synthesis ( Dhariwal & Nichol 2021 ) , speech generation ( Kong et al. 2020 ) , and molecular design ( Hoogeboom et al. 2022 ) . These models operate by simulating a forward diffusion process, where Gaussian noise is incrementally added to data, followed by a learned reverse process that denoises the data step by step. In the

Read the complete text