QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization
Florian A. D. Burnat · 2026
Read QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization by Florian A. D. Burnat (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04267.txt.
Introduction
Burnat studies how to split a budget between testing solutions and asking the decision maker what they want. Interactive multi-objective optimization trades off competing goals, but each objective evaluation can be expensive, and preference questions also cost time. Cheap pairwise statements are noisy, while indifference adjustments are richer but costlier. QUIVER, a surrogate-assisted evolutionary optimizer, chooses at each step between evaluating objectives and asking either kind of question, picking whichever promises the most decision-quality gain per unit cost.
Across standard DTLZ and WFG benchmarks under simulated decision makers, QUIVER reached the lowest final utility regret on hard WFG problems, beating baselines by about twenty-five percent. The mix adapts to difficulty: on easy problems it asked pairwise questions eighty percent of the time, but on hard problems it shifted to thirty-five percent indifference adjustments. Cost-aware querying, the results suggest, beats any single question type.
Catalog record
| CREATOR | Florian A. D. Burnat |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04267
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization Florian A. D. Burnat Abstract Interactive multi-objective optimization systems face a budget allocation dilemma: one can spend resources on expensive objective evaluations or on eliciting decision-maker preferences that identify the relevant region of the Pareto set. Moreover, preference elicitation itself spans modalities with different information content and cognitive burden, ranging from cheap, noisy pairwise preference statements (PS) to richer but costlier indifference adjustments (IA). We study cost-aware optimization under an unknown scalarization and introduce QUIVER (Query-Informed Value Estimation for Regret), a surrogate-assisted evolutionary multi-objective optimizer that adaptively chooses between objective evaluations and heterogeneous preference queries. At each step, QUIVER selects the next action by maximizing the expected decision-quality improvement per unit total cost. Across DTLZ and WFG benchmarks under synthetic decision-maker models, QUIVER achieves the lowest final utility regret on challenging WFG problems (utility regret of 2.14 on WFG4, 2.82 on WFG9: a 25% improvement over baselines), outperforming all single-modality baselines. We analyze how the optimal mix of PS and IA adapts to problem difficulty: on easy problems (DTLZ2), QUIVER selects 80% PS queries; on hard problems (WFG9), it shifts to 35% IA queries. This adaptive modality selection demonstrates cost-aware preference learning in action. Keywords: interactive multi-objective optimization, surrogate-assisted EMO, preference elicitation, cost-aware optimization, evolutionary computation 1 Introduction Multi-objective optimization (MOO) seeks solutions that trade off multiple conflicting objectives. In many real-world applications, from engineering design to portfolio selection, objective evaluations are expensive and involve physical experiments, costly simulations, or time-consuming computations. This has motivated extensive research on surrogate-assisted and budgeted optimization methods that carefully allocate evaluation budget to efficiently approximate the Pareto front. However, the Pareto front is typically infinite, and ultimately, a decision-maker (DM) must select a single solution. This selection depends on an unknown scalar utility that captures the DM’s preferences over trade-offs. Without knowledge of these preferences, even a perfect approximation of the Pareto front may fail to identify the solution that the DM actually wants. This creates a fundamental tension: objective evaluations explore the Pareto set, whereas preference elicitation identifies which region of that set matters. The cost-aware allocation problem. Given a fixed total budget, how should an optimizer allocate resources between objective evaluation and preference elicitation? Spending too much on evaluations may yield a well-approximated front in a region that the DM does not care about; spending too much on preference queries may identify the target region precisely but leave it poorly explored. The optimal allocation depends on the relative costs and informativeness of each action, which vary across problem instances and over the course of optimization. Heterogeneous preference modalities. Preference elicitation spans modalities with different information content and cognitive burden. Pairwise preference statements (PS)—“Do you prefer A or B ?”—are cognitively cheap but yield only ordinal (1-bit) information. Indifference adjustments (IA)—“How much would you adjust objective k to make A and B equally preferred?”—are more demanding but reveal cardinal trade-off information, identifying the DM’s marginal rate of substitution. This heterogeneity raises a second allocation question: When should we use cheap, low-information queries versus expensive, high-information queries? Novelty and approach. Most interactive EMO methods assume a fixed preference query protocol (e.g., periodic pairwise queries at fixed generations), whereas most surrogate-assisted EMO methods focus on Pareto front exploration and treat the DM as an a posteriori selector. In contrast, we treat {objective evaluation (Eval), PS, IA} as a unified budgeted action space and introduce an adaptive controller that selects actions to maximize the expected decision-quality improvement per unit total cost . The value-of-information (VOI) perspective unifies objective exploration and preference learning under a single optimization criterion. Optimizer-agnostic design. We designed the VOI-per-cost controller to be optimizer-agnostic. In this study, we instantiate it with a surrogate-assisted NSGA-II and evaluate it using a protocol appropriate for evolutionary multi-objective optimization. The controller requires only a minimal backend interface: (i) a mechanism to generate candidate designs x , (ii) storage for evaluated points, (iii) optionally a surrogate providing predictive distributions, and (iv) the ability to interleave preference queries with evaluations. Any optimization backend satisfying these requirements, including Bayesian optimization, random search, or other population-based methods, can use the same VOI-per-cost action selection principle. Contributions. This study makes four contributions. 1. We formalize cost-aware interactive MOO with heterogeneous preference query modalities (PS and IA) under an unknown linear scalarization. 2. We derive a practical VOI-based action selection rule using Monte Carlo estimation of expected entropy reduction. 3. We propose QUIVER ( Qu ery- I nformed V alue E stimation for R egret), an evolutionary algorithm with NSGA-II backbone that implements this adaptive allocation policy via a particle-filter preference posterior. 4. We empirically evaluate on DTLZ and WFG benchmarks, demonstrating that QUIVER achieves lower utility regret per unit cost than fixed-schedule and single-modality baselines, with IA queries dominating when their information-per-cost ratio exceeds that of PS. 2 Related Work Surrogate-assisted evolutionary multi-objective optimization. Expensive black-box multi-objective optimization has motivated extensive research on surrogate-assisted evolutionary algorithms. ParEGO 12 scalarizes objectives via randomly sampled weight vectors and applies single-objective Bayesian optimization. More recent approaches integrate surrogates directly into population-based EMO frameworks, including MOEA/D-EGO 19 and K-RVEA 4 . Chugh et al. 5 categorized surrogate management strategies for expensive MOO. These methods focus on efficiently approximating the Pareto front, but typically do not learn DM preferences online. Interactive and preference-based EMO. Interactive evolutionary multi-objective optimization incorporates DM feedback to guide the search towards preferred regions. Preference-based EMO learns implicit utilities from comparisons or rankings 2 ; 3 . However, most approaches adopt fixed querying schedules and do not adaptively allocate budget between evaluations and preference elicitation. Existing interactive EMO typically assume a fixed query budget or schedule, whereas we study joint allocation across evaluation and heterogeneous query types under a single cost budget. Preference elicitation modalities and cognitive cost. Pairwise preference statements (PS) yield ordinal information and are commonly modelled using Bradley–Terry or probit likelihoods 1 . Indifference adjustments (IA) provide cardinal trade-off information and can identify marginal rates of substitution in multi-attribute utility models 11 . Recent empirical work by Haidinger et al. 8 quantifies this trade-off: among 83 roofing professionals, IAs require only \sim 15% more time and cognitive effort than PSs, yet provide substantially richer information about cardinal preferences. These differing cost–information trade-offs motivate adaptive strategies that choose query modalities based on the expected value per cost. Cost-aware Bayesian optimization. Our cost-aware acquisition builds on the value-of-information (VOI) principles from decision analysis 9 ; 16 . In active learning, cost-sensitive methods select queries that maximize the information gain per unit cost 18 . A recent study on cost-aware BO 13 proposed acquisition functions that explicitly account for evaluation costs. Our contribution extends cost-awareness to preference queries, unifying objective evaluation and preference elicitation under a single, budgeted decision framework. RLHF and preference optimization. Reinforcement learning from human feedback (RLHF) 14 and direct preference optimization 15 rely almost exclusively on pairwise comparisons. Our framework suggests that the strategic use of richer query types (analogous to IA) during critical training phases could reduce annotation budgets, which we discuss in Section 8 . 3 Problem Setup We consider the minimization of m objectives \mathbf{f}(x)\in\mathbb{R}^{m} over a feasible domain \mathcal{X} (maximization problems are handled via sign-flip).