Soft Quality-Diversity Optimization

Saeed Hedayatian, Stefanos Nikolaidis · 2025

Read Soft Quality-Diversity Optimization by Saeed Hedayatian, Stefanos Nikolaidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00810.txt.

Introduction

Saeed Hedayatian and Stefanos Nikolaidis wrote a 2025 paper on a kind of optimization that values variety. Quality-diversity algorithms do not hunt for one best answer. They build a set of answers that are both strong and different from each other, covering many styles of solution.

Most current methods keep variety by cutting the behavior space into fixed boxes and filling each box. Hedayatian and Nikolaidis propose a softer version of that idea. A softer rule could keep diversity without forcing solutions into rigid bins. Such sets are useful in design and robotics, where having good options matters more than one winner.

Catalog record

CREATORSaeed Hedayatian, Stefanos Nikolaidis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00810

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Soft Quality-Diversity Optimization Saeed Hedayatian, Stefanos Nikolaidis Abstract Quality-Diversity (QD) algorithms constitute a branch of optimization that is concerned with discovering a diverse and high-quality set of solutions to an optimization problem. Current QD methods commonly maintain diversity by dividing the behavior space into discrete regions, ensuring that solutions are distributed across different parts of the space. The QD problem is then solved by searching for the best solution in each region. This approach to QD optimization poses challenges in large solution spaces, where storing many solutions is impractical, and in high-dimensional behavior spaces, where discretization becomes ineffective due to the curse of dimensionality. We present an alternative framing of the QD problem, called Soft QD , that sidesteps the need for discretizations. We validate this formulation by demonstrating its desirable properties, such as monotonicity, and by relating its limiting behavior to the widely used QD Score metric. Furthermore, we leverage it to derive a novel differentiable QD algorithm, Soft QD Using Approximated Diversity (SQUAD) , and demonstrate empirically that it is competitive with current state of the art methods on standard benchmarks while offering better scalability to higher dimensional problems. 1 Introduction Optimization in machine learning is typically cast as the search for a single solution that maximizes performance with respect to some objective. Quality-diversity (QD) optimization ( 42 ; 8 ) challenges this paradigm by instead discovering a collection of solutions that are both high-performing and behaviorally diverse. This perspective is especially powerful in domains with multiple useful optima, or where robustness and user choice matter as much as raw performance. To illustrate, consider the task of painting a portrait. A traditional optimizer might aim for the single image that is most similar to the subject. Conversely, a QD optimizer not only aims for high fidelity, but also explores a behavior space that captures stylistic dimensions like color palette, brushstroke texture, and degree of abstraction, yielding a set of portraits that all resemble the subject while spanning a spectrum of artistic expressions. Such diversity is valuable for human selection and for escaping the limitations of optimizers ( 43 ) and imperfect objectives. In recent years, QD optimization has grown from its roots in evolutionary computation into a broadly applicable machine learning paradigm. In reinforcement learning, QD has generated diverse policies that facilitate exploration and improve robustness, in both single ( 41 ; 13 ; 2 ) and multi-agent ( 27 ) settings. In the context of large foundation models, QD has been adopted for red-teaming and safety analysis ( 46 ; 54 ) as well as diverse content generation ( 7 ; 12 ) . Beyond these, QD has found applications in scenario generation ( 3 ; 17 ; 59 ) , creative design ( 35 ; 58 ) , engineering ( 47 ; 23 ) , robotics ( 25 ; 61 ) , and scientific discovery ( 5 ; 28 ) . QD algorithms typically operate by partitioning the behavior space into discrete cells and seeking the best solution in each cell (a tessellation, together with the stored solutions, is often referred to as the archive ). Progress on this objective is commonly measured by the QD Score ( 42 ) , which sums the performance of the best solutions across all occupied cells, thus capturing both quality and coverage. This approach has fueled much of the progress in QD, including recent advances that incorporate surrogate models ( 21 ; 60 ) , gradient information ( 39 ; 16 ; 18 ) , and alternative archive structures ( 51 ; 18 ; 36 ) . Yet, the formulation also presents two fundamental limitations. First, the non-differentiable nature of tessellations precludes direct optimization using gradient-based optimizers that dominate modern machine learning, except through heuristics ( 16 ; 39 ) . Second, discretizing the behavior space suffers from the curse of dimensionality in high-dimensional setting, as either the number or the volume of cells will grow exponentially, forcing methods to rely on dimensionality reduction techniques such as PCA or autoencoders ( 40 ; 22 ; 24 ) . As a result, QD optimization remains hindered in high-dimensional, gradient-rich machine learning domains, despite its promise. We address these challenges by rethinking the formulation of QD optimization itself. We introduce Soft QD Score as a new objective for QD optimization that measures how well a collection of solutions cover the behavior space with high-quality solutions, without discretizing the behavior space. Building on it, we derive SQUAD , a novel algorithm that leverages a tractable lower bound of the Soft QD score to enable end-to-end differentiable optimization. This approach has an intuitive interpretation as finding an equilibrium between attractive forces, which drive solutions toward higher quality, and repulsive forces, which spread them across the behavior space. Through experiments on both established and newly designed QD benchmarks, we demonstrate that SQUAD broadens the applicability of QD and provide further insights into its properties. Our contributions are threefold: 1. We introduce Soft QD, a new formulation of QD optimization and analyze its theoretical properties. 2. We develop SQUAD, a differentiable QD algorithm derived from the aforementioned formulation. 3. We evaluate SQUAD on multiple benchmarks, showing its effectiveness in high-dimensional and complex optimization settings. 2 Background The quality diversity (QD) problem aims to find a collection of high-quality solutions that are diverse in their behavior. A QD problem is defined by a solution space \Theta , an objective or quality function f:\Theta\to\mathbb{R} which measures a solution’s quality, and a behavior descriptor function \mathrm{desc}:\Theta\to\mathcal{B} that maps each solution to a point in the behavior space \mathcal{B} . The goal is to discover for each point in \mathcal{B} a high-quality solution that exhibits that specific behavior. We can formalize this objective as finding a set of solutions \bm{\theta}=\{\theta_{b}\}_{b\in\mathcal{B}} that maximizes the integral of their quality over the behavior space, \int_{\mathcal{B}}f(\theta_{b})\,\mathrm{d}b . The problem is referred to as Differentiable Quality-Diversity (DQD) ( 16 ) when both the objective and descriptor functions are differentiable. Since the behavior space \mathcal{B} is continuous, QD algorithms often partition it into n cells, \mathcal{A}=\{c_{1},\dots,c_{n}\} , known as an archive or tessellation . The QD objective is then framed as finding a high-quality solution for each cell. This is captured by the QD Score , which is the sum of the maximum quality found within each cell: \max_{\bm{\theta}}\mathrm{QD\,Score}_{\mathcal{A}}(\bm{\theta})=\sum_{c\in\mathcal{A}}\max\{f(\theta):\theta\in\bm{\theta},\,\mathrm{desc}(\theta)\in c\}. (1) Discretizing the behavior space introduces a fundamental challenge in high dimensions due to the curse of dimensionality. Grid archives (e.g., 11 ) divide the space evenly along each dimension, which makes the number of cells grow exponentially with the dimensionality of \mathcal{B} . This makes it infeasible to maintain a fine-grained grid when \mathcal{B} is high-dimensional. Centroidal Voronoi Tessellation (CVT) archives ( 51 ) address this by fixing the number of cells and defining them around a set of centroids. Each cell contains all points that are closer to its assigned centroid than to any other centroid, creating an almost uniform partitioning of the space. While this avoids the exponential increase of the number of cells, the volume of each CVT cell still grows exponentially with dimensionality. Large cells make it difficult to explore new regions by building on existing solutions (which is commonly

Read the complete text