SMOG : Scalable Meta-Learning for Multi-Objective Bayesian Optimization

Leonard Papenmeier, Petru Tighineanu · 2026

Read SMOG : Scalable Meta-Learning for Multi-Objective Bayesian Optimization by Leonard Papenmeier, Petru Tighineanu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22131.txt.

Introduction

Leonard Papenmeier and Petru Tighineanu speed up optimization problems that pull in opposite directions. Multi-objective optimization balances competing goals with only black-box access to the system and a limited budget of measurements. Many teams hold historical data from related past tasks, which meta-learning could exploit.

Their 2026 preprint presents SMOG, a scalable meta-learning method for multi-objective Bayesian optimization. By learning from earlier tasks, the optimizer starts new problems already knowing where good trade-offs tend to live. That head start matters in drug design, engineering, and materials work. Where each measurement can cost days or thousands of dollars and budgets never stretch as far as the questions do.

Catalog record

CREATORLeonard Papenmeier, Petru Tighineanu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22131

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SMOG : Scalable Meta-Learning for Multi-Objective Bayesian Optimization Leonard Papenmeier, Petru Tighineanu Abstract Multi-objective optimization aims to solve problems with competing objectives, often with only black-box access to a problem and a limited budget of measurements. In many applications, historical data from related optimization tasks is available, creating an opportunity for meta-learning to accelerate the optimization. BO , as a promising technique for black-box optimization, has been extended to meta-learning and multi-objective optimization independently, but methods that simultaneously address both settings—meta-learned priors for multi-objective BO —remain largely unexplored. We propose SMOG , a scalable and modular meta-learning model based on a multi-output GP that explicitly learns correlations between objectives. SMOG builds a structured joint GP prior across meta- and target tasks and, after conditioning on metadata, yields a closed-form target-task prior augmented by a flexible residual multi-output kernel. This construction propagates metadata uncertainty into the target surrogate in a principled way. SMOG supports hierarchical, parallel training: meta-task GP are fit once and then cached, achieving linear scaling with the number of meta-tasks. The resulting surrogate integrates seamlessly with standard MOBO acquisition functions. 1 Introduction Many high-impact optimization problems are intrinsically multi-objective : engineers and machine-learning practitioners rarely optimize a single scalar, but rather trade off competing goals such as performance vs. cost, quality vs. speed, or accuracy vs. latency/energy. At the same time, these objectives are often expensive and noisy to evaluate, which naturally puts us in the low-data regime where BO ( BO )/ MOBO ( MOBO ) is particularly effective ( 30 ; 43 ; 27 ; 6 ; 7 ) . Crucially, such optimizations are rarely one-off: organizations accumulate logs of past runs on related products, machines, datasets, workloads, or environments. This makes meta-learning for expensive multi-objective optimization a natural setting: rather than starting each new optimization from scratch, we would like to leverage prior tasks to achieve good Pareto-optimal trade-offs with far fewer evaluations. Examples span industrial process tuning and calibration (where competing quality metrics must be balanced), scientific design (e.g., materials discovery and advanced manufacturing with multiple competing properties), and machine-learning system design (e.g., multi-objective hyperparameter and architecture tuning trading off accuracy, latency, and resource use) ( 11 ; 19 ; 23 ; 8 ; 18 ; 12 ) . Despite its promise, meta-learning for multi-objective optimization is technically subtle in the low-data regime. First, in multi-objective settings, “what to transfer” is not a single optimum but information about a Pareto set/front , and decision making typically depends on uncertainty-aware criteria (e.g., hypervolume-based utilities) ( 16 ; 6 ; 7 ) . Second, historical data are often scarce and heterogeneous across tasks; transfer must therefore account for meta-task uncertainty to avoid overconfident bias from weak or mismatched prior tasks ( 5 ; 34 ; 10 ; 32 ) . Third, multi-objective problems add another layer: objectives can be correlated, so efficiently using evidence often requires non-trivial probabilistic multi-output surrogates . Treating objectives independently can waste information precisely when evaluations are precious. These challenges leave a key gap: we need meta-learning methods that are (i) Bayesian uncertainty-aware, (ii) scalable across many meta-tasks, and (iii) able to exploit cross-objective correlations. Fully joint multi-task multi-output GP models are principled but quickly become infeasible at meta-learning scale ( 25 ; 1 ) , while most scalable alternatives are developed for single-objective transfer and treat objectives independently when adapted to MOBO . Contribution. We introduce SMOG ( S calable M eta-Learning and Multi- O bjective G P), a probabilistic framework that closes this gap by meta-learning a correlated multi-objective GP ( GP ) surrogate while retaining principled Bayesian uncertainty propagation from historical tasks. Building on the modular GP meta-learning perspective of 32 , SMOG constructs a structured multi-output prior that (i) remains fully Bayesian over all task data, (ii) learns correlations between objectives through vector-valued GP structure, and (iii) is scalable in the number of meta-tasks via a modular decomposition that enables parallel training and caching of meta-task posteriors. This yields a target-task surrogate that can be dropped into standard MOBO pipelines (e.g., hypervolume-based acquisition optimization) ( 6 ; 7 ; 16 ) , with coherent propagation of meta-data uncertainty to the target task. 2 Related Work Most work on meta-learning for BO focuses on learning a better surrogate for a single-objective target task. A principled approach is to build a joint Bayesian model across tasks (e.g., a multi-task GP ), which yields coherent uncertainty estimates but is computationally prohibitive—scaling cubically in the total number of observations and, at best, quadratically in the task-correlation hyperparameters ( 3 ; 4 ; 1 ; 31 ; 41 ; 14 ; 24 ; 29 ; 33 ) . To improve scalability, a number of methods rely either on heuristic combinations of per-task surrogates (e.g., GP ensembles) ( 10 ; 40 ; 5 ) or on building a parametric GP prior on the metadata ( 21 ; 26 ; 39 ; 36 ) . These scale better with the number of meta-tasks but sacrifice a joint Bayesian treatment and thus principled uncertainty propagation across tasks. A recent work addresses this tension by introducing assumptions that lead to a modular GP model: conditioning on meta-data exposes a modular decomposition into M independent meta-task GP posteriors and a target-task GP prior, enabling scalable and fully Bayesian transfer ( 32 ) . In contrast to the rich single-objective literature, meta-learning methods that directly target multi-objective optimization remain scarce. A notable exception is the task-similarity extension of MO-TPE by 37 , which transfers knowledge by reweighting the acquisition based on task similarity. While effective and scalable, it operates at the level of density models/acquisition rather than a unified probabilistic multi-output surrogate. Recent few-shot surrogate-assisted evolutionary methods for expensive multi-objective optimization meta-learn surrogates ( 42 ) . However, both approaches typically do not learn a correlated multi-output posterior and thus cannot exploit cross-objective dependencies or propagate meta-task uncertainty to the target surrogate in a principled way. Our work targets this underexplored regime by combining scalable GP meta-learning with a multi-output surrogate that models cross-objective dependencies. 3 Method Figure 1 : High-level view of SMOG : Meta tasks are modeled independently of the target GP . The target GP models correlations between objectives and uses the weighted means and covariances of the meta GP to learn an informative target-task prior. We aim to find the Pareto set of a target black-box function \bm{f}_{t}:\mathcal{X}\to\mathbb{R}^{O} , where \mathcal{X}\subset\mathbb{R}^{D} is the search space of dimensionality D and O is the number of objectives. Observations \bm{y}_{n}=(y_{n,o})_{o\in\mathcal{O}} of \bm{f}_{t} may be corrupted by independent zero-mean Gaussian noise, \bm{y}_{n}=\bm{f}_{t}(\bm{x}_{n})+\bm{\varepsilon}_{n} with \bm{\varepsilon}_{n}\sim\mathcal{N}\!\big(\bm{0},\mathrm{diag}(\sigma_{1}^{2},\ldots,\sigma_{O}^{2})\big) . We use the target data \mathcal{D}_{t}=\{(\bm{x}_{n},\bm{y}_{n})\}_{n=1}^{N_{t}} to build a probabilistic model. Specifically, we model \bm{f}_{t} with a multi-output GP prior with mean m_{o}(\cdot) and kernel k_{oo^{\prime}}(\cdot,\cdot) for o,o^{\prime}\in\mathcal{O}=\{1,\ldots,O\} . Conditioned on \mathcal{D}_{t} , the posterior is again a GP with mean and covariance ( 38 ) \begin{split}\hat{m}_{t,o}(\bm{x})&=m_{o}(\bm{x})+\bm{k}\!\left((\bm{x},o),\bm{X}_{t}\right)\\ &\quad\left[\bm{K}(\bm{X}_{t},\bm{X}_{t})+\bm{\Sigma}_{\varepsilon}\right]^{-1}\left(\bm{Y}_{t}-\bm{m}(\bm{X}_{t})\right),\\ \hat{k}_{t,oo^{\prime}}(\bm{x},\bm{x}^{\prime})&=k\!\left((\bm{x},o),(\bm{x}^{\prime},o^{\prime})\right)-\bm{k}\!\left((\bm{x},o),\bm{X}_{t}\right)\\ &\quad\left[\bm{K}(\bm{X}_{t},\bm{X}_{t})+\bm{\Sigma}_{\varepsilon}\right]^{-1}\bm{k}\!\left(\bm{X}_{t},(\bm{x}^{\prime},o^{\prime})\right),\end{split} (1) where \bm{X}_{t}=\big((\bm{x}_{n},o)\big)_{n=1,\ldots,N_{t},\;o\in\mathcal{O}} stacks the objective-augmented inputs and \bm{Y}_{t}=\big(y_{n,o}\big)_{n,o} the corresponding observations. We assume per-objective Gaussian noise \bm{\Sigma}_{\varepsilon}=\mathrm{diag}(\sigma_{1}^{2}\bm{I}_{N_{t}},\ldots,\sigma_{O}^{2}\bm{I}_{N_{t}}) . To leverage related tasks, we assume access to M\geq 1 meta-tasks with datasets \mathcal{D}_{1:M}=\bigcup_{m\in\mathcal{M}}\mathcal{D}_{m} , where \mathcal{M}=[M] . Each meta-task m provides \mathcal{D}_{m}=\{(\bm{x}_{m,n},\bm{y}_{m,n})\}_{n=1}^{N_{m}} with \bm{y}_{m,n}\in\mathbb{R}^{O} and per-objective Gaussian noise \bm{y}_{m,n}=\bm{f}_{m}(\bm{x}_{m,n})+\bm{\varepsilon}_{m,n} , \bm{\varepsilon}_{m,n}\sim\mathcal{N}\!\big(\bm{0},\mathrm{diag}(\sigma_{m,1}^{2},\ldots,\sigma_{m,O}^{2})\big) . For notational

Read the complete text