Amortized Bayesian Experimental Design for Decision-Making
Daolang Huang, Yujia Guo, Luigi Acerbi, Samuel Kaski · 2024
Read Amortized Bayesian Experimental Design for Decision-Making by Daolang Huang, Yujia Guo, Luigi Acerbi, Samuel Kaski (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02064.txt.
Description
A 2024 arXiv research paper by Daolang Huang, Yujia Guo, Luigi Acerbi, Samuel Kaski. Many critical decisions, such as personalized medical diagnoses and product pricing, are made based on insights gained from designing, observing, and analyzing a series of experiments. This highlights the crucial role of experimental design, which goes beyond merely collecting information on system parameters as in traditional Bayesian experimental design (BED), but also plays a key part in facilitating downstream decision-making . Most recent BED methods use an amortized policy network to rapidly design experiments.
Catalog record
| CREATOR | Daolang Huang, Yujia Guo, Luigi Acerbi, Samuel Kaski |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02064
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Amortized Bayesian Experimental Design for Decision-Making Daolang Huang, Yujia Guo, Luigi Acerbi, Samuel Kaski Abstract Many critical decisions, such as personalized medical diagnoses and product pricing, are made based on insights gained from designing, observing, and analyzing a series of experiments. This highlights the crucial role of experimental design, which goes beyond merely collecting information on system parameters as in traditional Bayesian experimental design (BED), but also plays a key part in facilitating downstream decision-making . Most recent BED methods use an amortized policy network to rapidly design experiments. However, the information gathered through these methods is suboptimal for down-the-line decision-making, as the experiments are not inherently designed with downstream objectives in mind. In this paper, we present an amortized decision-aware BED framework that prioritizes maximizing downstream decision utility. We introduce a novel architecture, the Transformer Neural Decision Process (TNDP), capable of instantly proposing the next experimental design, whilst inferring the downstream decision, thus effectively amortizing both tasks within a unified workflow. We demonstrate the performance of our method across several tasks, showing that it can deliver informative designs and facilitate accurate decision-making. 1 Introduction In a wide array of disciplines, from clinical trials ( Cheng and Shen,, 2005 ) to medical imaging ( Burger et al.,, 2021 ) , a fundamental challenge is the design of experiments to infer the distribution of some unobservable, unknown properties of the systems under study. Bayesian Experimental Design (BED) ( Lindley,, 1956 ; Chaloner and Verdinelli,, 1995 ; Ryan et al.,, 2016 ; Rainforth et al.,, 2024 ) is a powerful framework in this context, guiding and optimizing experimental design by maximizing the expected amount of information about parameters gained from experiments, see Fig. 1 (a). However, the ultimate goal in many tasks extends beyond parameter inference to inform a downstream decision-making task by leveraging our understanding of these parameters. For example, in personalized medical diagnostics, a model is built based on historical data to facilitate an optimal treatment for a new patient ( Bica et al.,, 2021 ) . This data typically comprises patient covariates, administered treatments, and observed outcomes. Since the query of such data tends to be expensive due to, e.g., privacy issues, we need to actively design queries to optimize resource utilization. Here, when the goal is to improve decisions, the strategy of experimental designs should not focus solely on inferring the parameters of the model, but rather on guiding the final decision-making for the new patient, to ensure that each query contributes maximally to the diagnostic decision. Traditional BED methods ( Rainforth et al.,, 2018 ; Foster et al.,, 2019 ; Foster et al.,, 2020 ; Kleinegesse and Gutmann,, 2020 ) do not take down-the-line decision-making tasks into account during the experimental design phase. As a result, inference and decision-making processes are carried out separately, which is suboptimal for decision-making in scenarios where experiments can be adaptively designed. Loss-calibrated inference , which was originally introduced by Lacoste-Julien et al. ( Lacoste-Julien et al.,, 2011 ) for variational approximations in Bayesian inference, provides a concept that adjusts the inference process to capture posterior regions critical for decision-making tasks. Rather than focusing on parameter estimation, the idea is to directly maximize the expected downstream utility, recognizing that accurate decision-making can proceed without exact knowledge of the posterior as not all regions of the posterior contribute equally to the downstream task. Inspired by this concept, we could consider integrating decision-making directly into the experimental design process to align the proposed designs more closely with the ultimate decision-making task. Figure 1: Overview of BED workflows. (a) Traditional BED, which iterates between optimizing designs, running experiments, and updating the model via Bayesian inference. (b) Amortized BED, which uses a policy network for rapid experimental design generation. (c) Our decision-aware amortized BED integrates decision utility in training to facilitate downstream decision-making. To pick the next optimal design, standard BED methods require estimating and optimizing the expected information gain (EIG) over the design space, which can be extremely time-consuming. This limitation has led to the development of amortized BED ( Foster et al.,, 2021 ; Ivanova et al.,, 2021 ; Blau et al.,, 2022 ; Blau et al.,, 2023 ) , a policy-based method which leverages a neural network policy trained on simulated experimental trajectories to quickly generate designs, as illustrated in Fig. 1 (b). Given an experiment history, these policy-based methods determine the next experimental design through a single forward pass, significantly speeding up the design process. Another significant advantage of amortized BED methods is their capacity to extract and utilize unstructured domain knowledge embedded in historical data. Unlike traditional methods, which never reuse the information from past data, amortized methods can integrate this knowledge to refine and improve design strategies for new experiments. In our problem setting, the benefits of amortization are also valuable where decisions must be made swiftly, such as when determining optimal treatment for patients in urgent settings. In this paper, we propose an amortized decision-making-aware BED framework, see Fig. 1 (c). We identify two key aspects where previous amortized BED methods fall short when applied to downstream decision-making tasks. First, the training objective of the existing methods does not consider downstream decision tasks. Therefore, we introduce the concept of Decision Utility Gain (DUG) to guide experimental design to better align with the downstream objective. DUG is designed to measure the improvement in the maximum expected utility derived from the new experiment. Second, to obtain the optimal decision, we still need to explicitly approximate the predictive distribution of the outcomes to estimate the utility. Current amortized methods learn this distribution only implicitly and therefore require extra computation for the decision-making process. To address this, we propose a novel Transformer neural decision process (TNDP) architecture with dual output heads: one acting as a policy network to propose new designs, and another to approximate the predictive distribution to support the downstream decision-making. This setup allows an iterative approach where the system autonomously proposes informative experimental designs and makes optimal final decisions. Finally, since our ultimate goal is to make optimal decisions at the final stage, which may involve multiple experiments, it is crucial that our experimental designs are not myopic or overly greedy by only maximizing next-step decision utility. Therefore, we develop a non-myopic objective function that ensures decisions are made with a comprehensive consideration of future outcomes. Contributions. In summary, the contributions of our work include: • We propose the concept of decision utility gain (DUG) for guiding the next experimental design with a direct focus on optimizing down-the-line decision-making tasks. • We present a novel architecture – Transformer neural decision process (TNDP) – designed to amortize the experimental designs directly for downstream decision-making. • We empirically show the effectiveness of TNDP across a variety of experimental design tasks involving decision-making, where it significantly outperforms other methods that do not consider downstream decisions. 2 Preliminaries 2.1 Bayesian experimental design BED ( Lindley,, 1956 ; Ryan et al.,, 2016 ; Rainforth et al.,, 2024 ) is a powerful statistical framework that optimizes the experimental design process. The primary goal of BED is to sequentially select a set of experimental designs \xi\in\Xi and gather outcomes y , to maximize the amount of information