MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions
Lam Ngo, Huong Ha, Jeffrey Chan, Hongyu Zhang · 2025
Read MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions by Lam Ngo, Huong Ha, Jeffrey Chan, Hongyu Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20872.txt.
Introduction
Lam Ngo, and three co-authors with him, propose MOBO-OSD, a Bayesian optimization method for problems with several goals at once. Bayesian optimization suits costly black box functions, but finding a diverse spread of best trade off points across goals remains hard.
Their method solves a batch of constrained subproblems along orthogonal search directions set against an estimated hull of single goal best points. A well spread set of directions covers the trade off front broadly instead of crowding one region. The algorithm aims at solution sets that show decision makers the full range of compromises, useful when cost, quality, and risk must be weighed together in expensive design tasks.
Catalog record
| CREATOR | Lam Ngo, Huong Ha, Jeffrey Chan, Hongyu Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20872
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions Lam Ngo, Huong Ha, Jeffrey Chan, Hongyu Zhang Abstract Bayesian Optimization (BO) is a powerful tool for optimizing expensive black-box objective functions. While extensive research has been conducted on the single-objective optimization problem, the multi-objective optimization problem remains challenging. In this paper, we propose MOBO-OSD, a multi-objective Bayesian Optimization algorithm designed to generate a diverse set of Pareto optimal solutions by solving multiple constrained optimization problems, referred to as MOBO-OSD subproblems , along orthogonal search directions (OSDs) defined with respect to an approximated convex hull of individual objective minima. By employing a well-distributed set of OSDs, MOBO-OSD ensures broad coverage of the objective space, enhancing both solution diversity and hypervolume performance. To further improve the density of the set of Pareto optimal candidate solutions without requiring an excessive number of subproblems, we leverage a Pareto Front Estimation technique to generate additional solutions in the neighborhood of existing solutions. Additionally, MOBO-OSD supports batch optimization, enabling parallel function evaluations to accelerate the optimization process when resources are available. Through extensive experiments and analysis on a variety of synthetic and real-world benchmark functions with two to six objectives, we demonstrate that MOBO-OSD consistently outperforms the state-of-the-art algorithms. Our code implementation can be found at https://github.com/LamNgo1/mobo-osd . 1 Introduction Multi-objective Bayesian Optimization (MOBO) has recently attracted much attention [ 5 , 13 , 52 , 44 , 1 ] . MOBO extends Bayesian Optimization (BO) — a powerful framework for optimizing expensive black-box functions — to settings where multiple, often conflicting, objectives must be optimized simultaneously. Applications of MOBO include but are not limited to machine learning [ 47 , 29 ] , material design [ 18 , 56 ] , agriculture [ 27 , 28 ] , robotics [ 54 , 34 ] and vehicle design [ 32 , 2 ] . There are several challenges when working with MOBO problems. First, the objective functions are often conflicting, meaning that improving one objective may deteriorate another. Hence the goal is not to identify a single optimal solution, but rather a set of Pareto optimal solutions . Second, maintaining the diversity among these Pareto optimal solutions is crucial to capture the wide range of trade-off across all objectives. This becomes more challenging under the limited evaluation budget in MOBO settings, where there is less budget to balance between exploration and exploitation while covering the Pareto front. Third, batch optimization plays an important role in reducing the time and cost through parallel evaluations of objective functions. However, in MOBO, it introduces additional complexity, as it requires modeling the interactions among unobserved points across multiple objective functions. Fourth, the computational cost remains a significant drawback, as the cost for training surrogate models and computing multi-objective acquisition functions, e.g., EHVI [ 13 ] , become increasingly expensive with the number of objectives. Despite the growing body of MOBO research, there exists substantial room for improvement. Many existing approaches rely on scalarization techniques [ 5 , 9 , 41 ] , which often fail to capture a diverse Pareto front, leading to suboptimal performance in terms of the hypervolume indicator. Moreover, several studies are unable to scale to a large number of objectives ( M>3 ) [ 33 ] or suffer from high computational cost in batch optimization settings [ 13 , 52 ] . In this paper, our goal is to develop a novel MOBO algorithm that prioritizes Pareto front diversity, resulting in an improvement in hypervolume performance and scalability to an arbitrary number of objectives in both sequential and batch optimization settings. To achieve this, we build upon the key insight of the Normal Boundary Intersection (NBI) method [ 11 ] : the intersection points between the boundary of the objective space and the vectors orthogonal to the convex hull of individual minima (CHIM) of the objectives could be Pareto optimal solutions. This geometric approach has been shown to generate a well-distributed ( diverse ) set of Pareto optimal solutions and scale well to a large number of objectives in multi-objective optimization problems with a large evaluation budget. To incorporate this idea under the limited evaluation budget in MOBO problems, we first introduce a technique to effectively approximate the CHIM. We then generate a well-distributed set of orthogonal search directions (OSDs) with respect to the approximated CHIM and define a tailored search procedure that solves a constrained single-objective optimization problem along each OSD ( MOBO-OSD subproblem ) to identify intersection points with the boundary that are likely to be Pareto optimal. By constructing a well-distributed set of OSDs, MOBO-OSD achieves broad coverage of the objective space, enhancing both solution diversity and hypervolume performance. To further enrich the set of Pareto optimal candidate solutions without the need to solve a large number of MOBO-OSD subproblems, we incorporate a Pareto Front Estimation (PFE) technique [ 46 ] that locally explores the neighborhood of existing Pareto optimal candidates. All candidates are aggregated and evaluated using the hypervolume improvement acquisition function [ 19 ] , and those with the highest scores are selected as the next batch of data points for evaluation. Finally, to support batch optimization, we incorporate the Kriging Believer technique [ 22 ] while considering the diversity of exploration regions, i.e., ensuring that selected points for observation originate from different exploration spaces. We name our method Batch M ulti- O bjective B ayesian O ptimization via O rthogonal S earch D irections (MOBO-OSD). Our thorough analysis and extensive experiments on various synthetic and real-world benchmark problems, in both sequential and batch optimization settings, show that MOBO-OSD outperforms the state-of-the-art MOBO methods. We summarize our contributions as follows. • We propose MOBO-OSD, a novel MOBO algorithm that generates a diverse set of Pareto optimal candidate solutions by solving multiple optimization subproblems defined along search directions orthogonal to the approximated CHIM of the objective space. To further enhance the diversity, we propose to locally explore the Pareto set for additional Pareto solutions via a PFE technique. • We develop MOBO-OSD in such a way that it can perform effectively in the batch optimization setting by leveraging the Kriging Believer technique and exploration space information. • We demonstrate that MOBO-OSD can outperform the state-of-the-art MOBO baselines on a comprehensive set of synthetic and real-world multi-objective benchmark problems with a wide range of numbers of objectives, in both sequential and batch settings. 2 Background 2.1 Multi-Objective Optimization We consider a multi-objective optimization (MOO) problem involving a vector-valued objective function \mathbf{f}:\mathcal{X}\rightarrow\mathcal{Y} with \mathbf{f}=(f_{1},\dots,f_{M}) , where \mathcal{X}\subset\mathbb{R}^{D} is a D -dimensional input space, and \mathcal{Y}\subset\mathbb{R}^{M} is an M -dimensional output space ( M>1 ). Without loss of generality, we assume the goal is to minimize all objectives of \mathbf{f} . In MOO, it is generally not possible to find a single solution that optimizes all objectives simultaneously. Instead, the aim is to identify the set of Pareto optimal solutions , where no objective can be improved without deteriorating at least one of the others. A solution \mathbf{f}(\mathbf{x}^{(i)}) is said to Pareto dominate another solution \mathbf{f}(\mathbf{x}^{(j)}) , i.e., \mathbf{f}(\mathbf{x}^{(i)})\succ\mathbf{f}(\mathbf{x}^{(j)}) , if f_{m}(\mathbf{x}^{(i)})\geq f_{m}(\mathbf{x}^{(j)})\ \forall m=1,\dots,M and there exists m^{\prime}\in\{1,\dots,M\} such that f_{m}(\mathbf{x}^{(i)})>f_{m}(\mathbf{x}^{(j)}) . The set of Pareto optimal solutions is called the Pareto front \mathcal{P}_{f}=\{\mathbf{f}(\mathbf{x})\