Mixture-Model Preference Learning for Many-Objective Bayesian Optimization
Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang, Samuel Kaski · 2026
Read Mixture-Model Preference Learning for Many-Objective Bayesian Optimization by Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang, Samuel Kaski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28410.txt.
Introduction
Dubey and three co-authors learn decision makers' hidden preference types while running many-objective Bayesian optimization. Standard methods assume one utility function behind every comparison a person makes. Real people switch trade-off styles by context. The framework models preferences as a Dirichlet-process mix of archetypes. Its queries target two jobs in turn: spotting which archetype is active, then refining it. On synthetic tests and a chemical design case, it beat rival methods and recovered readable preference structure.
Catalog record
| CREATOR | Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang, Samuel Kaski |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65796 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.28410
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mixture-Model Preference Learning for Many-Objective Bayesian Optimization Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang, Samuel Kaski Abstract Preference-based many-objective optimization faces two obstacles: an expanding space of trade-offs and heterogeneous, context-dependent human value structures. Towards this, we propose a Bayesian framework that learns a small set of latent preference archetypes rather than assuming a single fixed utility function, modelling them as components of a Dirichlet-process mixture with uncertainty over both archetypes and their weights. To query efficiently, we designing hybrid queries that target information about (i) mode identity and (ii) within-mode trade-offs. Under mild assumptions, we provide a simple regret guarantee for the resulting mixture-aware Bayesian optimization procedure. Empirically, our method outperforms standard baselines on synthetic and real-world many-objective benchmarks, and mixture-aware diagnostics reveal structure that regret alone fails to capture. 1 Introduction Real-world problems involving design of materials and aditive manufacturing Hastings et al. 2025 ; Myung et al. 2025 , power systems (involving economic-emission dispatch) Ah King et al. 2005 , robotics and autonomous driving Li et al. 2023 ; Shen et al. 2025 are often associated with optimizing several objectives, a setting known as multi-objective optimization. Multi-objective optimization Deb et al. 2016 addresses problems in which several performance criteria must be optimized simultaneously. In realistic applications these criteria are often mutually conflicting, so a single design cannot satisfy all goals at once. The aim therefore shifts from finding one “best” solution to characterizing and selecting among trade-offs on the Pareto frontier—those designs for which improving one objective would necessarily degrade at least one other. For many applications the number of objectives grows into the many-objective regime (often L\!\geq\!4 ), which implies that the problem changes qualitatively. For instance, urban design frameworks include optimizing performance objectives like operational energy, comfort, daylight, cost and carbon Liu et al. 2023 . Similarly, multi-objective recommenders include optimizing user utility, diversity, fairness, revenue and long-term value Jannach and Abdollahpouri 2023 . It has been established in the literature that the complexity of computing the Pareto front dramatically increases when the number of objectives increase to four or more Ishibuchi et al. 2008 ; Binois et al. 2020 . Dominance relations weaken, many candidates are mutually non-dominating, so selection pressure diminishes and exploration requires substantially more evaluations to achieve comparable coverage. Summarization is also harder; beyond a few objectives, visualizing trade-offs, communicating them to stakeholders, and choosing a manageable set of representatives become nontrivial. Established indicators (e.g., hypervolume Zhang and Golovin 2020 ) are costly to compute and less discriminative in higher dimensions, further complicating algorithm design and evaluation. The burden shifts to both cognition and computation. For decision makers, comparing high-dimensional outcome vectors quickly exceeds human working-memory limits; judgments become noisy and inconsistent, especially when objectives have incommensurate units or span different risk horizons. In practice, people often default to heuristics while attending to a few salient objectives or applying non-compensatory rules. Hence, preferences expressed on the full objective set can be unstable and context dependent. Recent many-objective BO methods often collapse decisions into a single compromise solution or reduce dimensionality by removing redundant objectives Binois et al. 2020 ; Lin et al. 2024 ; Martín and Garrido-Merchán 2021 . These approaches assume calibrated objectives and a single global trade-off. In practice, however, decision making frequently exhibits multiple regimes (e.g., safety-first, cost-first, balanced), implying piecewise or multi-modal utilities rather than a single smooth scalarization. Many objectives also induce non-compensatory or conditional priorities that fixed scalarizations misrepresent. These limitations motivate methods that actively elicit preferences, model multiple trade-off modes, and focus sampling on decision-relevant regions of objective space. We build on this observation and propose an interactive, preference-based approach that treats heterogeneity as structure rather than noise. Instead of collapsing everything to one utility, we represent preferences as a mixture of archetypal trade-off modes. Each mode captures a distinct way of valuing objectives and is explicitly context-aware. A mode is represented by a weight vector \mathbf{w}_{k}\!\in\!\Delta^{L-1} that defines a Chebyshev utility over outcomes \mathbf{y}=\mathbf{f}(\mathbf{x}) . Under this view, preference elicitation naturally decomposes into two questions: which mode is currently active (identity) and how that mode trades objectives (shape). This decomposition directly informs query design. We introduce inter–mode queries that maximize mutual information about the active mode, intra–mode queries that refine the trade–off within a selected mode, and a hybrid policy that balances the two. Coupled with standard GP surrogates for the objectives, the procedure concentrates sampling where the decision-maker (DM) cares, learns a calibrated posterior over modes, and remains robust when users persist in a given style or switch across contexts. We show that treating user preferences as a mixture of archetypes and asking questions that are explicitly informative about that mixture, yields faster and more calibrated preference-driven many-objective Bayesian optimization, specially when real users exhibit persistent modes, which is true for real humans. Our contributions can be summarized as follows: • We formulate preference–based many–objective Bayesian optimization with a Dirichlet–process mixture over Chebyshev weights, abandoning the single–utility assumption and enabling multiple, distinct trade–off archetypes to co–exist. • We propose information-theoretic cluster-aware methods for active query selection in interactive preference learning. We derive inter, intra, and hybrid query rules that explicitly target information about mode identity versus within–mode shape, turning preference elicitation into learning both the identity and the shape of archetypes. • Beyond simple regret, we introduce diagnostics for mixture-aware evaluation like mode coverage, calibration of mixture weights that regret alone can hide. • Our experiments on simulated and real-world datasets show the effectiveness of the proposed method with respect to proposed mixture-aware diagnostics. 2 Problem Formulation We consider many-objective Bayesian optimization (MaO-BO) with L\geq 4 objectives. Let \mathcal{X}\subset\mathbb{R}^{d} be a design space and \mathbf{f}:\mathcal{X}\to\mathbb{R}^{L} an unknown vector-valued objective. Querying a design x\in\mathcal{X} returns a (possibly noisy) observation \mathbf{y}=\mathbf{f}(x)+\boldsymbol{\epsilon} where \boldsymbol{\epsilon}\sim\mathcal{N}(0,\sigma^{2}I) and evaluations are assumed expensive. Many-objective problems typically admit a large set of Pareto-optimal solutions. In human-in-the-loop settings, however, the goal is not to recover the entire Pareto front, but to identify solutions that are most desirable to a decision maker (DM). Direct evaluations of the DM’s utility are unavailable. Instead, the learner may query pairwise preferences between previously observed outcomes. Given two outcomes \mathbf{y}_{i} and \mathbf{y}_{i}^{\prime} , the decision maker returns a binary comparison indicating whether \mathbf{y}_{i}\succ\mathbf{y}_{i}^{\prime} . Further, we assume that the DM evaluates outcomes through an unknown utility function. To model heterogeneous trade-offs, we posit that the DM’s utility is governed by one of K latent preference modes. Each mode k\in[K] is parameterized by a weight vector \mathbf{w}_{k}\in\Delta^{L-1} , and modes occur with unknown mixture weights \boldsymbol{\eta}\in\Delta^{K-1} . This formulation captures either a single DM with context-dependent trade-offs or a heterogeneous population of DMs. We denote the scalar utility induced by mode k as U(\mathbf{f}(x);\mathbf{w}_{k}) . Our objective is to identify designs that maximize the DM’s mixture-expected utility, x^{\star}\in\arg\max_{x\in\mathcal{X}}\sum_{k=1}^{K}\eta_{k}\,\mathbb{E}\!\left[U(\mathbf{f}(x);\mathbf{w}_{k})\right], (1) where the expectation is taken over observation noise. The learner must therefore infer the latent mixture parameters \{\eta_{k},\mathbf{w}_{k}\}_{k=1}^{K} from pairwise feedback while sequentially selecting designs to evaluate. 3 Proposed Methodology In this section, we introduce a preference-aware multi-objective Bayesian optimization framework that jointly models uncertainty over objective functions and heterogeneous decision-maker trade-offs. The approach combines independent Gaussian process surrogates with a latent mixture model over preference scalarizations. We