Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization Corresponding Author

Juan David Guerra, Thomas Garbay, Guillaume Lajoie, Marco Bonizzato · 2025

Read Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization Corresponding Author by Juan David Guerra, Thomas Garbay, Guillaume Lajoie, Marco Bonizzato (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11294.txt.

Introduction

Juan Guerra and four colleagues wrote a 2025 paper on a faster way to tune complex systems. Their setting is Bayesian optimization, a method that tests a system a few times and uses a statistical model to guess where the best settings lie. It suits problems where each test is slow or costly.

Their model is a hierarchical Gaussian process, which splits a problem into subtasks handled by linked sub-models. Older versions share information in one direction only, top-down or bottom-up. The new method, called Bidirectional Information Flow, lets the levels talk both ways at all times. The parent model learns from summaries sent up by its children. This two-way flow should reach good settings with fewer tests.

Catalog record

CREATORJuan David Guerra, Thomas Garbay, Guillaume Lajoie, Marco Bonizzato
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-11294

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization Corresponding Author Juan David Guerra, Thomas Garbay, Guillaume Lajoie, Marco Bonizzato Abstract Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing for different models to address each part, making them well-suited for problems with inherent hierarchical structure. However, typical H-GP models do not fully take advantage of this structure, only sending information up or down the hierarchy. This one-way coupling limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), an efficient H-GP framework that establishes bidirectional information exchange between parent and child models in H-GPs for online training. BIF retains the modular structure of hierarchical models — the parent combines subtask knowledge from children GPs — while introducing top-down feedback to continually refine children models during online learning. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. BIF outperforms conventional H-GP Bayesian Optimization methods, achieving up to 85 % 85\% and 5x higher R 2 R^{2} scores for the parent and children respectively, on synthetic and real-world neurostimulation optimization tasks. 1 Introduction Many real-world tasks require optimizing complex, structured parameter spaces under limited data and high-stakes constraints, by iterative trial-and-error. In team sports, success is not just the sum of individual optimal decisions, but emerges from the coordinated interplay between players. In aerospace engineering, Integrated Flight and Engine Control joins the flight control system (wing flaps, rudder, etc.) with the engine control system (throttle, fuel flow) for optimal performance ( Smith et al. 1991 ) . In neuroscientific engineering, designing multi-electrode neurostimulation protocols for a neural interface is costly and data-scarce ( Bonizzato et al. 2023 ) . Bayesian optimization (BO) provides a principled approach to such problems by using a surrogate model to guide the search for optimal parameters. Gaussian Processes (GPs), commonly used as BO surrogates, capture complex functions and uncertainty from small datasets, making them well-suited for these scenarios. However, when the parameter space has an inherent hierarchical structure, modeling it with a single GP can be inefficient. Hierarchical GP models address this limitation by decomposing the task: multiple child GPs model different subcomponents of the original problem, and a parent GP combines their outputs to drive the overall optimization procedure. This divide-and-conquer strategy leverages the problem’s structure and has shown promise in practice — for instance, it has been used to tune spatiotemporal neurostimulation parameters for treating neurological disorders ( Laferrière et al. 2020 ) , classification of glaucoma through medical images ( An et al. 2021 ) , drug development in the pharmaceutical industry ( Ruberg et al. 2023 ) , and many more ( Hensman et al. 2013 ; Fyshe et al. 2012 ; Fox et al. 2007 ) . Nevertheless, existing hierarchical GP frameworks suffer a key limitation: information flows only upward from children to parent ( Lawrence and Moore 2007 ; Fyshe et al. 2012 ; Laferrière et al. 2020 ) . The parent model treats child predictions as inputs or priors but has no mechanism to adjust or inform the child models in return. This one-way information flow means valuable insights at the parent level (such as interactions between sub-tasks) cannot propagate back down, leading to suboptimal sample efficiency and slow convergence. In this work, we introduce Bidirectional Information Flow (BIF) 1 1 1 Code and datasets: refer to our Github at https://anonymous.4open.science/r/BIF-D04F , a novel hierarchical GP framework that establishes two-way communication between parent and child models. BIF retains the modularity of the hierarchical approach — each child GP learns its individual subtask — while also incorporating top-down feedback from the parent to refine the children in an online learning process. By allowing the parent and children to continually inform each other, BIF reduces the number of experiments needed and improves the learning robustness. We claim that our Bidirectional Information Flow model is able to 1) improve performance efficiency compared to baseline models, 2) accurately train child models in the hierarchy using sub-responses entirely inferred from the parent model’s environment interaction, following a short initialization period and 3) boost performance of new combination problems with transfer learning of child modules. We show that BIF yields superior performance on both synthetic benchmarks and a real-world neurostimulation protocol design, achieving higher sample efficiency and more reliable learning than conventional unidirectional hierarchical methods. The remainder of this paper details the BIF methodology and its integration into Bayesian optimization, followed by experiments evaluating its performance. 2 Related Work Hierarchical implementations can be seen across many domains such as disease detection, drug design, neural stimulation, and many more due to the ability of hierarchical models to effectively break down complex tasks into easier to accomplish subtasks ( An et al. 2021 ; Ruberg et al. 2023 ; Laferrière et al. 2020 ; Hensman et al. 2013 ) . Not only do these models have the benefit of being more interpretable, seeing as it is easier to understand a step by step guide compared to a very low level explanation, they also tend to be sample efficient as models approximate state spaces using previously learned subtasks. We aim to take advantage of this structure and adapt it to use Gaussian Processes. Early hierarchical approaches, like the Hierarchical GP-LVM ( Lawrence and Moore 2007 ) , and subsequent Deep Gaussian Processes ( Damianou and Lawrence 2013 ) , utilize layered GP architectures for capturing intricate, multi-scale patterns from limited data. Multi-task GPs ( Bonilla et al. 2007 ) further exploit correlations across related tasks, enhancing data efficiency through shared covariance structures. Similarly, Fyshe et al. 2012 use low dimensional latent processes to classify magnetoencephalography (MEG) brain recordings of human subjects observing different words. Explicit hierarchical GP-BO methods directly address structured optimization problems. Laferrière et al. 2020 developed hierarchical GP-BO specifically for neurostimulation protocols, demonstrating improved sample efficiency and performance compared to traditional BO by modeling individual stimulation channels as child GPs feeding into a parent GP as a prior to the mean for the combined neurostimulation task. Other work in the field of hierarchical GP-BO was explored by ( Hensman et al. 2013 ) in the field of genomics. However, previous hierarchical GP frameworks predominantly adopt a one-way flow of information ( Lawrence and Moore 2007 ; Laferrière et al. 2020 ; Fyshe et al. 2012 ; Fox et al. 2007 ) . This limitation restricts adaptability and sample efficiency since parent-level insights cannot refine or inform the child models iteratively. Our proposed Bidirectional Information Flow (BIF) addresses this gap by introducing two-way interactions within the hierarchical GP framework, significantly enhancing both efficiency and adaptability. 3 Methodology Figure 1: Overview of the BIF framework with an arbitrary number of children. Colour coding demonstrates each child’s responsibility for a parent dimension. Arrows indicate the direction of information flow: red for prior map creation (child to parent, Section 3.2 ), and blue for sub-response inference via the contribution system (parent to child, Section 3.3 ). Steps (1–7) summarize Alg. 1 . We now present our novel algorithm - Bidirectional Information Flow (BIF). We implement a two layer hierarchical structure, wherein the first layer contains a singular parent Gaussian Process (GP) and the

Read the complete text