Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces
Henry B. Moss, Sebastian W. Ober, Tom Diethe · 2025
Read Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces by Henry B. Moss, Sebastian W. Ober, Tom Diethe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03910.txt.
Introduction
Henry Moss, Sebastian Ober, and Tom Diethe rethink a popular optimization strategy in a 2025 machine learning paper. Bayesian optimization inside the latent space of a variational autoencoder is powerful for structured domains such as molecules. Existing methods, though, weld the surrogate model to the generative model, which can hurt when the latent space does not fit the task.
That coupling has spawned ever more complex algorithms. The authors instead explore a decoupled approach with a playful title, the return of the latent space COWBOYS. Decoupling may recover strong performance with a simpler method, a useful correction for the field.
Catalog record
| CREATOR | Henry B. Moss, Sebastian W. Ober, Tom Diethe |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03910
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces Henry B. Moss, Sebastian W. Ober, Tom Diethe Abstract Bayesian optimisation in the latent space of a VAE ( VAE ) is a powerful framework for optimisation tasks over complex structured domains, such as the space of scientifically interesting molecules. However, existing approaches tightly couple the surrogate and generative models, which can lead to suboptimal performance when the latent space is not tailored to specific tasks, which in turn has led to the proposal of increasingly sophisticated algorithms. In this work, we explore a new direction, instead proposing a decoupled approach that trains a generative model and a GP ( GP ) surrogate separately, then combines them via a simple yet principled Bayesian update rule. This separation allows each component to focus on its strengths— structure generation from the VAE and predictive modelling by the GP . We show that our decoupled approach improves our ability to identify high-potential candidates in molecular optimisation problems under constrained evaluation budgets. 1 Introduction First introduced by Gómez-Bombarelli et al. 2018 , BO ( BO ) over latent spaces has emerged as a powerful technique for optimising over structures. Rather than performing challenging combinatorial or high-dimensional optimisation directly on discrete structures — such as molecules or proteins — LSBO ( LSBO ) first maps inputs into a fixed-dimensional Euclidean latent space, where standard surrogate models and gradient-based acquisition routines can be employed. Candidate points selected by the optimiser are then decoded back into the original structured domain to yield new query points. Learning a complex mapping from variable-size structured inputs to fixed-size Euclidean representations requires more data than is typically gathered in LSBO itself. Therefore, LSBO relies on embeddings pre-trained on a related task with abundant unlabelled data (e.g., using a VAE latent space trained on a large set of valid molecules). However, it is well known ( Chu et al. 2024 ) that fitting surrogates in the latent space of a VAE can lead to suboptimal modelling; therefore, the recent LSBO literature has focused on introducing increasingly sophisticated heuristics to fine-tune the VAE on newly acquired experimental data. These approaches remain challenged by the persistent risk of overfitting when training neural network components of Bayesian models on limited datasets ( Ober et al. 2021 ) . In this work, we also make a new argument that the prevalent approach of limiting the search to a fixed box subset of the latent space — necessary to apply traditional BO methods — also limits the effectiveness of LSBO . Figure 1 : Unlike LSBO where GP are fit in a VAE’s latent space, COWBOYS ’s GP is fit in structure space, decoupled from its VAE . While the VAE is designed purely as a generative model, LSBO attempts to repurpose it for discrimination — a fundamental mismatch that gives rise to the pathologies discussed above. We propose an alternative approach that preserves the original motivation of the VAE as a generative model by coupling it with a separately trained surrogate GP model via a novel Bayesian formulation. Our primary goal is straightforward: develop a sampling strategy that increasingly favours structures with desirable objective values. We propose COWBOYS ( COWBOYS ), a novel principled framework that unifies GP and VAE within a BO loop. Although we focus on molecular search—the most popular application of LSBO —where we demonstrate that COWBOYS excels under heavily constrained optimisation budgets, our approach is general and applicable to any structured problem for which a suitable structural GP kernel can be defined (see our discussion in Section 8 ). 2 Background 2.1 Bayesian Optimisation BO ( BO ) ( Mockus 2005 ) is a powerful framework for the efficient optimisation of costly black-box functions f:\mathcal{X}\rightarrow\mathds{R} . For a successful application of BO , we typically need three components: a search space, a surrogate model, and an acquisition function. Given some initial data, BO constructs a surrogate model of the data, which is then used in tandem with an acquisition function to determine which point in the search space will be most valuable to acquire. Once this point is chosen, we query the black-box function at this point and update the surrogate model, repeating until the evaluation budget is exhausted. Search spaces . In order for optimisation to succeed, the space over which we attempt to find the optimal value must be suitably restricted. For instance, when dealing with a Euclidean space \mathds{R}^{d} , it is typical to restrict the search space to a box region [-\delta,\delta]^{d} , where \delta is typically chosen using some prior knowledge about the expected location of the optimum ( Hvarfner et al. 2022 ) . In this example, choosing too large a \delta will lead to slow convergence due to the need for more data so that the surrogate can be effective, whereas too small a \delta will risk missing the optimum. Later, we will discuss how choosing appropriate search spaces is a key bottleneck for LSBO and how COWBOYS addresses this. Surrogate models . BO requires surrogate models that accurately quantify uncertainty while retaining flexibility: as such, GP ( GP ) ( Rasmussen 2003 ) are a standard choice. A GP can be defined as an infinite collection of random variables, any finite number of which are Gaussian distributed, and is entirely defined by a mean function \mu(\cdot) and a kernel k(\cdot,\cdot) . Under a GP f\sim\mathcal{GP}(\mu,k) , making a prediction \hat{f} at a test point \hat{\bm{x}} amounts to conditioning on the observed data D : p(\hat{f}|D)=\mathcal{N}(\mu^{*},\Sigma^{*}) , where \hat{\mu} and \hat{\Sigma} can be computed in closed form via the properties of Gaussians with O(N^{3}) computational complexity and O(N^{2}) memory. Acquisition functions . Acquisition functions measure the utility of acquiring an unseen point \bm{x} according to the predictions of our surrogate model. Common acquisition functions measure the EI ( EI ), PI ( PI ) ( Bergstra et al. 2011 ) or UCB ( UCB ) ( Auer 2000 ) of candidate points, or use information theoretical arguments ( Hennig & Schuler 2012 ; Moss et al. 2021 ) , e.g., \displaystyle\alpha_{PI}(\bm{x};D)=\mathds{P}(f(\bm{x})>f^{*}|D), (1) where f^{*} is the best objective value observed so far. Covariance (kernel) functions . In this work, where we must build a surrogate model over the space of molecules, we use the Tanimoto kernel ( Tripp et al. 2023 ) : \displaystyle K_{T}(\bm{m},\bm{m}^{\prime})=\sigma^{2}\frac{\bm{m}\cdot\bm{m}^{\prime}}{\|\bm{m}\|^{2}+\|\bm{m}^{\prime}\|^{2}-\bm{m}\cdot\bm{m}^{\prime}}, where \bm{m} and \bm{m}^{\prime} are molecular fingerprints representation vectors of molecules and \sigma\in\mathds{R}^{+} is a tunable scaling factor. We follow the advice of Tripp & Hernández-Lobato 2024 and use count-based vectors that count specific molecular features ( Landrum 2013 ) . Structural kernels such as the Tanimoto kernel, but also string kernels ( Moss et al. 2020a ) and graph kernels ( Kriege et al. 2020 ) , can often outperform deep learning alternatives, particularly in low- to medium-data regimes ( Moss & Griffiths 2020 ; Griffiths et al. 2022 ) . 2.2 Variational AutoEncoders VAE ( VAE ) ( Kingma & Welling 2014 ; Rezende et al. 2014 ) are deep generative models that model data through a two-step process: sampling a latent variable \bm{z} (with dimension smaller than \mathcal{X} ) from