Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes

Yunyue Wei, Vincent Zhuang, Saraswati Soedarmadji, Yanan Sui · 2024

Read Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes by Yunyue Wei, Vincent Zhuang, Saraswati Soedarmadji, Yanan Sui (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20375.txt.

Description

A 2024 arXiv research paper by Yunyue Wei, Vincent Zhuang, Saraswati Soedarmadji, Yanan Sui. Bayesian optimization is an effective technique for black-box optimization, but its applicability is typically limited to low-dimensional and small-budget problems due to the cubic complexity of computing the Gaussian process (GP) surrogate. While various approximate GP models have been employed to scale Bayesian optimization to larger sample sizes, most suffer from overly-smooth estimation and focus primarily on problems that allow for large online samples.

Catalog record

CREATORYunyue Wei, Vincent Zhuang, Saraswati Soedarmadji, Yanan Sui
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-20375

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes Yunyue Wei, Vincent Zhuang, Saraswati Soedarmadji, Yanan Sui Abstract Bayesian optimization is an effective technique for black-box optimization, but its applicability is typically limited to low-dimensional and small-budget problems due to the cubic complexity of computing the Gaussian process (GP) surrogate. While various approximate GP models have been employed to scale Bayesian optimization to larger sample sizes, most suffer from overly-smooth estimation and focus primarily on problems that allow for large online samples. In this work, we argue that Bayesian optimization algorithms with sparse GPs can more efficiently allocate their representational power to relevant regions of the search space. To achieve this, we propose focalized GP, which leverages a novel variational loss function to achieve stronger local prediction, as well as FocalBO , which hierarchically optimizes the focalized GP acquisition function over progressively smaller search spaces. Experimental results demonstrate that FocalBO can efficiently leverage large amounts of offline and online data to achieve state-of-the-art performance on robot morphology design and to control a 585-dimensional musculoskeletal system. 1 Introduction Bayesian Optimization (BO) is a powerful approach for solving black-box optimization problems, demonstrating notable success in hyperparameter tuning ( Snoek et al., 2012 ) , reinforcement learning ( Calandra et al., 2016 ; Chen et al., 2018 ) , and scientific discovery ( Gómez-Bombarelli et al., 2018 ) . The efficacy of BO is attributed to its ability to model the unknown objective function using a surrogate model and to strategically select the next sample position by optimizing an acquisition function. Among the surrogate models, Gaussian Processes (GPs) ( Rasmussen, 2003 ) are usually favored due to their flexibility and robust uncertainty quantification. However, the computation of the posterior GP covariance matrix scales as \mathcal{O}(n^{3}) with the number of data points n , which can severely restrict the applicability of BO in handling large datasets. This poses a significant challenge for real-world applications with high-dimensional and heterogeneous function landscapes such as those in robot control, which often necessitate a substantial amount of data to adequately explore the vast search space. To extend the scope of BO to accommodate larger datasets (from long-horizon online trials and/or pre-collected offline datasets ( Trabucco et al., 2022 ) ), it is imperative to employ surrogate models that offer enhanced computational efficiency. Using sparse GP models is a popular method for reducing the computational cost of BO. Sparse GPs accomplish this by learning an approximation of the full GP, either by using a subset of data ( Lawrence et al., 2002 ) , ensemble of local models ( Snelson and Ghahramani, 2005 ) , or variational inference ( Titsias, 2009 ) . However, classical sparse GP models are typically tailored for regression tasks, and therefore are designed to fit to the entire function landscape. Given limited representational resources, the resulting posterior is likely to be overly smooth, which may negatively impact the performance of BO. This issue is exacerbated in the high-dimensional setting, in which accurately fitting the entire domain is a far more challenging task. As such, several works have proposed strategies to improve BO performance with sparse GP models by focusing promising regions ( McIntire et al., 2016 ; Moss et al., 2023 ) or advanced sparse GP models ( Jimenez and Katzfuss, 2023 ) . However, most of their empirical evaluations are only conducted under large online sample setting in low-dimensional problems with fewer than 20 variables. It is unclear whether existing methods can be generalized to large offline data or high-dimensional setting. In this work, we explore the application of sparse Gaussian processes for optimizing high-dimensional problems with large offline (and optionally large online) datasets. We argue that by iteratively identifying key sub-regions of the input space and focusing the modeling capacity on these areas, we can enhance the modeling fidelity of the sparse GP in regions that are most relevant, thereby improving the overall performance of the Bayesian optimization algorithm. To this end, we propose a novel loss function to train a variational sparse GP model (focalized GP) that emphasizes the fitting of local functional landscapes through weighting the training data. Along with focalized GP, we design a hierachical algorithm, FocalBO , to propose sample points via acquisition function optimization across varying scales of the search space. Experimental results demonstrate that FocalBO can improve upon commonly used acquisition functions in optimizing heterogeneous functions and can effectively utilize large offline datasets for efficient high-dimensional optimization. Furthermore, we showcase that FocalBO can efficiently optimize a policy with 585 parameters to control a musculoskeletal system, leveraging both offline and online data. To the best of our knowledge, FocalBO is the first sparse GP-based Bayesian optimization algorithm capable of efficiently optimizing high-dimensional problems under both large online sample and large offline data settings. Our main contributions: 1) We design FocalBO , which employs a hierarchical acquisition optimization strategy to achieve efficient optimization over high-dimensional problems with heterogeneous structure with limited representation capability. 2) Experimental results demonstrate the superior performance of FocalBO in leveraging large offline datasets for online optimization, and its capability to optimize high-dimensional musculoskeletal system control problems involving over 500 variables. 2 Related Work 2.1 Sparse Gaussian processes Scaling Gaussian processes to large datasets is an important topic ( Liu et al., 2020 ) . It can be broadly divided into global approximation strategies and local approximation strategies. Global sparse GPs perform distillation over the whole dataset to approximate the expensive full covariance matrix with a sparse representation. Several methods aim to choose a subset of representative training points from the whole dataset, and use the corresponding covariance matrix in place of the full covariance ( Hayashi et al., 2020 ; Lawrence et al., 2002 ; Seeger, 2002 ; Keerthi and Chu, 2005 ) . Sparse kernels aim at removing uncorrelated entries in the full covariance to obtain a compact matrix ( Gneiting, 2002 ; Melkumyan and Ramos, 2009 ; Buhmann, 2001 ; Wendland, 2004 ) . Sparse approximation methods use inducing variables to learn a low-rank representation of full covariance matrix ( Quinonero-Candela and Rasmussen, 2005 ; Smola and Bartlett, 2000 ; Seeger et al., 2003 ; Csató and Opper, 2002 ; Snelson and Ghahramani, 2005 ; Titsias, 2009 ; Hensman et al., 2013 ; Csató and Opper, 2000 ; Wilson and Nickisch, 2015 ) . Stochastic variational Gaussian process (SVGP) is a popular sparse GP method which employs variational inference to learn inducing variables and kernel hyperparameters jointly and enable training using stochastic gradient descent from mini-batch data ( Hensman et al., 2013 ) . Recently, nearest neighbor information has also been used to further improve the scalability of sparse GP over massive amount of data ( Wu et al., 2022 ; Tran et al., 2021 ) . In contrast, local sparse GPs divide the entire dataset and employ local GPs trained from different data subsets to approximate the full GP. For a given test set, the prediction can be extracted from one of the local GPs ( Kim et al., 2005 ; Datta et al., 2016 ) , mixture of GPs ( Yuksel et al., 2012 ; Masoudnia and Ebrahimpour, 2014 ) or product of GPs ( Hinton, 2002 ; Cohen et al.,

Read the complete text