Bayesian Design of Experiments in the Presence of Nuisance Parameters
Shirin Golchi † Luke Hagar † † Biostatistics, McGill University · 2025
Read Bayesian Design of Experiments in the Presence of Nuisance Parameters by Shirin Golchi † Luke Hagar † † Biostatistics, McGill University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03948.txt.
Introduction
Shirin Golchi and Luke Hagar build a Bayesian method for sizing experiments when extra unknown parameters cloud the design, in a paper from 2025. Classic design picks the smallest sample that guarantees a target power, often with tidy formulas. Bayesian design instead folds uncertainty about the parameters into the plan itself, but the math usually needs heavy simulation.
Their method pairs large-sample theory for the posterior with a flexible tree model to estimate operating traits fast. It covers fixed designs and adaptive ones that adjust as data arrive. By handling nuisance parameters at the design stage, the approach cuts computing cost sharply. The authors aim to make full Bayesian design practical for everyday studies.
Catalog record
| CREATOR | Shirin Golchi † Luke Hagar † † Biostatistics, McGill University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03948
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bayesian Design of Experiments in the Presence of Nuisance Parameters Shirin Golchi † Luke Hagar † † Biostatistics, McGill University Abstract Design of experiments has traditionally relied on the frequentist hypothesis testing framework where the optimal size of the experiment is specified as the minimum sample size that guarantees a required level of power. Sample size determination may be performed analytically when the test statistic has a known asymptotic sampling distribution and, therefore, the power function is available in analytic form. Bayesian methods have gained popularity in all stages of discovery, namely, design, analysis and decision making. Bayesian decision procedures rely on posterior summaries whose sampling distributions are commonly estimated via Monte Carlo simulations. In the design of scientific studies, the Bayesian approach incorporates uncertainty about the design value(s) instead of conditioning on a single value of the model parameter(s). Accounting for uncertainties in the design value(s) is particularly critical when the model includes nuisance parameters. In this manuscript, we propose methodology that utilizes the large-sample properties of the posterior distribution together with Bayesian additive regression trees (BART) to efficiently obtain the optimal sample size and decision criteria in fixed and adaptive designs. We introduce a fully Bayesian procedure that incorporates the uncertainty associated with the model parameters including the nuisance parameters at the design stage. The proposed approach significantly reduces the computational burden associated with Bayesian design and enables the wide adoption of Bayesian operating characteristics. Keywords: Assurance, Bayesian additive regression trees, Bayesian sample size determination, Decision theory, Group sequential design 1 Introduction Sample size determination is an important component of the design of experiments and studies. In classical statistics, sample size estimation follows the frequentist hypothesis testing framework, where a null hypothesis is defined with respect to the value of a quantity of interest. The size of a study is determined as the minimum amount of data required to reject the null hypothesis with a high probability (power) given a hypothesized value under the alternative hypothesis. When the power function of the test is available in analytic form, the sample size may be obtained analytically as is the case for many frequentist test statistics with known asymptotic sampling distributions. Bayesian methods have gained popularity in the design and analysis of scientific studies. For example, clinical trials which have traditionally been planned and analyzed using the frequentist framework, have recently adopted Bayesian methods at least partially in various settings ( Spiegelhalter et al., 1994 ; Spiegelhalter et al., 2004 ; Berry et al., 2010 ) . For analysis and decision making, the Bayesian approach relies on the posterior distribution and its summaries. A common hybrid Bayesian-frequentist approach is to assess a design that uses Bayesian decision rules with respect to frequentist properties such as frequentist power Berry et al., 2010 . Following Wang and Gelfand, 2002 , the current practice relies on simulation studies to estimate the sampling distribution of posterior summaries and determine the sample size as well as other design parameters in this hybrid setting. The term hybrid has also been used to refer to the cases where the Bayesian approach is taken at the design stage while the analysis remains frequentist. Advocated for by many researchers ( Spiegelhalter and Freedman, 1986 ; O’Hagan et al., 2005 ; Chuang-Stein, 2006 ; Gubbiotti and De Santis, 2011 ; Ren and Oakley, 2014 ; Best et al., 2024 ) , this approach incorporates the uncertainty about the parameter of interest via a probability distribution referred to as a design or sampling prior which is distinct from the analysis or fitting prior. The Bayesian approach toward design is particularly advantageous in problems with nuisance parameters as the design prior accommodates the uncertainty associated with these parameters while incorporating a hypothesized set of values for the parameter(s) of interest with their corresponding probabilities. The elimination of nuisance parameters in statistical inference has been extensively discussed Kalbfleisch and Sprott, 1970 ; Basu, 1977 ; Berger and Mortera, 1994 ; Berger et al., 1999 ; Flórez Rivera et al., 2024 . Several proposed approaches rely on integrating the likelihood function Basu, 1977 ; Berger et al., 1999 . The Bayesian framework has been noted by several authors as a natural approach to the problem where having assigned priors to all model parameters including the nuisance parameters, inference is made via the marginal posterior distribution of the parameter(s) of interest Basu, 1977 ; Berger and Mortera, 1994 ; Flórez Rivera et al., 2024 . We consider the problem of dealing with nuisance parameters from a design perspective which has received less attention. Recent work of note is that of Flórez Rivera et al., 2024 who considered the non-informative nuisance parameter principle (NNPP) Berger et al., 1988 which states the irrelevance of the nuisance parameters for inference about the parameter(s) of interest in the case that the likelihood is separable with respect to the two parameter types. Within this context, they revisited the mixed test approach of Pericchi and Pereira, 2016a which relies on a decision theoretic approach optimizing a weighted sum of errors as a loss function. Flórez Rivera et al., 2024 showed that the NNPP holds for the mixed test in discrete sample spaces. We are, however, concerned with the general problem of design in presence of (potentially informative) nuisance parameters with a special interest in accommodating Bayesian and decision theoretic approaches similar to that of Pericchi and Pereira, 2016a and its extensions Calderazzo et al., 2020 . When a design prior is used, the relevant operating characteristics are the integrated power or probability of success, also referred to as assurance or Bayesian assurance. In most realistic models, and especially in the case that a Bayesian analysis is performed, this integration is done numerically via Monte Carlo simulations. Typically, parameter values are generated from the design prior, data are generated from these parameter values, a decision is recorded based on the (frequentist or Bayesian) decision procedure, and this process is repeated to estimate the average power over the design prior and the sampling distribution. A large number of iterations is needed to achieve sufficient precision in the estimates of the operating characteristics. While there have been previous attempts in reducing the computational overhead required to estimate sampling distributions of posterior summaries ( Golchi, 2022 ; Golchi and Willard, 2024 ; Hagar and Stevens, 2025 ; Hagar and Golchi, 2025 ; Hagar et al., 2025 ) , efficient Bayesian design in the presence of nuisance parameters remains unaddressed. In this manuscript, we propose using the large-sample properties of the posterior distribution following from the Bernstein-von Mises (BvM) theorem ( van der Vaart, 1998 ) together with the asymptotic normality of the maximum likelihood estimator to obtain the sampling distribution of the commonly used posterior summaries in Bayesian testing procedures. This approximate posterior distribution depends on the true values of the nuisance parameters only through a scale parameter which is predicted across the parameter space from a set of initial evaluations generated by simulations and using Bayesian additive regression trees (BART). This approach will result in an analytic power function which can then be efficiently integrated over the design prior. The fundamental idea for our proposed approach is similar to that of Golchi, 2022 and