Distributionally Robust Optimization via Diffusion Ambiguity Modeling

Jiaqi Wen, Jianyi Yang · 2025

Read Distributionally Robust Optimization via Diffusion Ambiguity Modeling by Jiaqi Wen, Jianyi Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22757.txt.

Introduction

Good decisions should hold up when data distributions shift. Jiaqi Wen and Jianyi Yang study robust optimization in 2025. Their frame guards against a set of nearby distributions. That set is called an ambiguity set. A good set stays close to the nominal data. It must also cover many adverse cases.

Their set is built with a diffusion model. Diffusion models can reach cases beyond the nominal support. Yet they stay consistent with the nominal law. The team then solves the inner worst case step. They prove the method settles at a stationary point. Tests show stronger out of distribution performance in a prediction task.

Catalog record

CREATORJiaqi Wen, Jianyi Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22757

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distributionally Robust Optimization via Diffusion Ambiguity Modeling Jiaqi Wen, Jianyi Yang Abstract This paper studies Distributionally Robust Optimization (DRO), a fundamental framework for enhancing the robustness and generalization of statistical learning and optimization. An effective ambiguity set for DRO must involve distributions that remain consistent with the nominal distribution while being diverse enough to account for a variety of potential scenarios. Moreover, it should lead to tractable DRO solutions. To this end, we propose a diffusion-based ambiguity set design that captures various adversarial distributions beyond the nominal support space while maintaining consistency with the nominal distribution. Building on this ambiguity modeling, we propose Diffusion-based DRO ( D-DRO ), a tractable DRO algorithm that solves the inner maximization over the parameterized diffusion model space. We formally establish the stationary convergence performance of D-DRO and empirically demonstrate its superior Out-of-Distribution (OOD) generalization performance in a ML prediction task. \optauthor University of Houston, USA keywords Distributionally Robust Optimization, Diffusion Models, OOD Generalization 1 Introduction Distributionally Robust Optimization (DRO) is a fundamental framework for enhancing the robustness of statistical learning and optimization problems, particularly under Out-of-Distribution (OOD) scenarios Liu et al. 2021 ; Arjovsky 2020 . DRO formulates a minimax optimization problem, where the inner maximization identifies the worst-case distribution within an ambiguity set, and the outer minimization optimizes the decision variable against this worst-case scenario Chen et al. 2020 ; Liu et al. 2022b ; Chen and Paschalidis 2018 ; Kuhn et al. 2019 ; Smirnova et al. 2019 . Unlike non-probabilistic robust optimization, DRO leverages probabilistic uncertainty modeling to enable improved generalization performance. This property has made DRO increasingly important in ML for addressing distribution shifts, noisy data, and adversarial conditions Quiñonero-Candela et al. 2022 ; Muratore et al. 2022 ; Ren and Majumdar 2022 ; Li et al. 2019 ; Veit et al. 2017 . The performance of DRO algorithms critically depends on the design of the ambiguity set, which must contain meaningful distributional variations around the nominal distribution. A common approach is to model the ambiguity set using \phi -divergences, such as the Kullback–Leibler (KL) divergence Hu and Hong 2013 ; Husain et al. 2023 ; Kirschner et al. 2020 ; Kocuk 2020 ; Liu et al. 2022b . Although such \phi -divergence-based formulations can sometimes yield closed-form solutions Hu and Hong 2013 ; Husain et al. 2023 , they require that any distribution P in the ambiguity set be absolutely continuous with respect to the nominal distribution P_{0} (denoted P\ll P_{0} ), meaning that for any measurable set \mathcal{A} , if P_{0}(\mathcal{A})=0 , then P(\mathcal{A})=0 . This implicit constraint limits robustness in scenarios with support shifts. In contrast, Wasserstein-DRO leverages the Wasserstein distance to define the ambiguity set, allowing for support shifts. However, solving Wasserstein-DRO over the infinite probability space is difficult. Some approaches Mohajerin Esfahani and Kuhn 2018 ; Chen et al. 2020 ; Gao and Kleywegt 2023 reformulate Wasserstein-DRO as a finite-dimensional optimization problem based on the convex assumptions, which typically do not hold in ML. Other methods approximate Wasserstein-DRO via adversarial optimization Staib and Jegelka 2017 ; Gao and Kleywegt 2023 , but such relaxations are overly conservative, limiting their ability to fully leverage the benefits of probabilistic uncertainty modeling. Recent advances have aimed to address the challenges of ambiguity modelling in DRO. For example, Liu et al. 2022a incorporates data geometric properties into the design of discrepancy metrics, thereby reducing the complexity of the ambiguity set. In addition, a recent work Yang et al. 2025 studies DRO with ambiguity sets defined by a generalized Sinkhorn distance, which enables modelling uncertainty across distributions with different support spaces. Another work Ren and Majumdar 2022 constructs a Wasserstein-based ambiguity set in the latent space of generative models and subsequently applies Wasserstein-DRO methods to solve the problem. Additional related studies are discussed in Appendix A . Different from these approaches, our work is the first to model the ambiguity set in the space of diffusion models, which offers several advantages: (1) Diffusion models have a strong capability to represent the underlying data distribution, ensuring that the distributions in the ambiguity set remain consistent with the nominal distribution. (2) Diffusion models are capable of producing diverse samples beyond the training support space, thereby enabling the discovery of worst-case distributions. (3) Diffusion models provide a finite, parameterized optimization space, avoiding the need to solve problems over an infinite probability space. Our main contributions are summarized below: (1) We introduce a novel Diffusion Ambiguity Set for DRO, which encompasses diverse distributions while preserving consistency with the nominal distribution. (2) We design an inner maximization procedure for DRO with the proposed Diffusion Ambiguity Set, enabling tractable iterative optimization within a finite, parameterized space. (3) We propose D-DRO (Algorithm 1 ), which solves the resulting minimax optimization problem, and formally establish its stationary convergence in Theorems 1 and 2 . (4) We demonstrate the superior performance of D-DRO on the challenging ML task of renewable energy prediction. 2 DRO and Ambiguity Modeling Distributionally Robust Optimization. DRO optimizes for the worst-case performance given an ambiguity set constructed on a nominal distribution P_{0} , which can be an empirical distribution S_{0} . Consider an objective function f(w,x) with the decision variable w\in\mathcal{W} and the random parameter x\in\mathcal{X} . Given the nominal distribution P_{0}(x) of the random parameter x , DRO solves the following minimax optimization problem. w=\min_{w\in\mathcal{W}}\max_{P\in\mathcal{B}(P_{0},\epsilon)}\mathbb{E}_{x\sim P}[f(w,x)], (1) where \mathcal{B}(P_{0},\epsilon) is the ambiguity set containing possible testing distributions which is typically modeled as a distribution ball \mathcal{B}(P_{0},\epsilon)=\left\{P\mid D(P,P_{0})<\epsilon\right\} given a distribution discrepancy measure D and an adversary budget \epsilon . Ambiguity modeling in DRO. The choice of the ambiguity set in DRO has a significant impact on both generalization performance and solution tractability. We observe that a well-designed ambiguity set in DRO should satisfy the following properties: First, it should include diverse distributions beyond the support of the nominal distribution, enabling the identification of various worst-case distributions. Second, the distributions within the ambiguity set should remain realistic and consistent with the nominal distribution, balancing the average-case and worst-case performance. Finally, the ambiguity set should facilitate a tractable solution of the DRO problem despite the infinite probability space. 3 Method 3.1 Diffusion Ambiguity Modeling Diffusion models . Diffusion models learn an underlying distribution P_{0} from its finite dataset S_{0} and can generate diverse samples from this distribution. The model training and inference are based on a forward process and a reverse process, detailed in Appendix B . The forward process begins with an initial sample x_{0}\in S_{0} and evolves according to a stochastic process to produce random variables x_{1},\dots,x_{T} with marginal distributions P_{t} , t\in[T] . The reverse process starts with x_{T} drawn from a prior distribution \pi that approximates P_{T} and reconstructs x_{T-1},\dots,x_{0} by following the reverse diffusion process, which depends on the score function (the gradient of the log-density of the underlying distribution \nabla_{x}\log P_{t}(x) ). The score-matching method employs an ML model s_{\theta}(x,t) with parameters \theta to approximate the distribution gradient \nabla_{x}\log P_{t}(x) by minimizing an empirical score-matching loss J(\theta,S_{0}) on the dataset S_{0} . Once the score model s_{\theta}(x,t) is obtained, we can generate samples x_{0},\dots,x_{T-1} according to the distribution P_{\theta}(x_{0:T})=\pi(x_{T})\prod_{t=1}^{T}P_{\theta}(x_{t-1}\mid x_{t}), where P_{\theta}(x_{t-1}\mid x_{t})=\mathcal{N}\big(x_{t-1};\mu_{\theta}(x_{t},t),\Sigma_{\theta}(x_{t},t)\big) where \mu_{\theta}(x_{t},t) and \Sigma_{\theta}(x_{t},t)\big) rely on the

Read the complete text