Approximate Bayesian computation.
Sunnåker M, Busetto AG, Numminen E, Corander J, Foll M, Dessimoz C. · 2013
Read Approximate Bayesian computation. by Sunnåker M, Busetto AG, Numminen E, Corander J, Foll M, Dessimoz C. (2013) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3547661.txt.
Introduction
Some statistical models are too complex for their likelihood to be computed, and Mikael Sunnaker and six colleagues explain approximate Bayesian computation in this 2013 primer. ABC sidesteps the likelihood altogether: simulate data from the model, keep the parameter values whose simulations land close to the real data, and treat those as the answer's distribution.
The trick widens Bayesian inference to models in genetics, ecology, and beyond that were previously out of reach. Approximations are unavoidable, and their cost must be checked; summary statistics that compress the data help efficiency when chosen well. The article lays out the logic, the algorithms, and the open problems in estimation and model choice, for a method spreading quickly.
Catalog record
| CREATOR | Sunnåker M, Busetto AG, Numminen E, Corander J, Foll M, Dessimoz C. |
|---|---|
| YEAR | 2013 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Approximate Bayesian Computation By Mikael Sunnåker, Alberto Giovanni Busetto, Elina Numminen, Jukka Corander, Matthieu Foll, Christophe Dessimoz, Shoshana Wodak Published in: PLoS Computational Biology DOI: 10.1371/journal.pcbi.1002803 ABSTRACT Approximate Bayesian computation (ABC) constitutes a class of computational methods rooted in Bayesian statistics. In all model-based statistical inference, the likelihood function is of central importance, since it expresses the probability of the observed data under a particular statistical model, and thus quantifies the support data lend to particular values of parameters and to choices among different models. For simple models, an analytical formula for the likelihood function can typically be derived. However, for more complex models, an analytical formula might be elusive or the likelihood function might be computationally very costly to evaluate. ABC methods bypass the evaluation of the likelihood function. In this way, ABC methods widen the realm of models for which statistical inference can be considered. ABC methods are mathematically well-founded, but they inevitably make assumptions and approximations whose impact needs to be carefully assessed. Furthermore, the wider application domain of ABC exacerbates the challenges of parameter estimation and model selection. ABC has rapidly gained popularity over the last years and in particular for the analysis of complex problems arising in biological sciences (e.g., in population genetics, ecology, epidemiology, and systems biology). "This is a “Topic Page” article for PLOS Computational Biology." HISTORY The first Approximate Bayesian computation (ABC)-related ideas date back to the 1980s. Donald Rubin, when discussing the interpretation of Bayesian statements in 1984 [1], described a hypothetical sampling mechanism that yields a sample from the posterior distribution. This scheme was more of a conceptual thought experiment to demonstrate what type of manipulations are done when inferring the posterior distributions of parameters. The description of the sampling mechanism coincides exactly with that of the ABC-rejection scheme, and this article can be considered to be the first to describe approximate Bayesian computation. Another prescient point was made when Rubin argued that in Bayesian inference, applied statisticians should not settle for analytically tractable models only but instead consider computational methods that allow them to estimate the posterior distribution of interest. This way, a wider range of models can be considered. These arguments are particularly relevant in the context of ABC. In 1984, Peter Diggle and Richard Gratton suggested using a systematic simulation scheme to approximate the likelihood function in situations where its analytic form is intractable [2]. Their method was based on defining a grid in the parameter space and using it to approximate the likelihood by running several simulations for each grid point. The approximation was then improved by applying smoothing techniques to the outcomes of the simulations. While the idea of using simulation for hypothesis testing was not new [3], [4], Diggle and Gratton seemingly introduced the first procedure using simulation to do statistical inference under a circumstance where the likelihood is intractable. Although Diggle and Gratton's approach had opened a new frontier, their method was not yet exactly identical to what is now known as ABC, as it aimed at approximating the likelihood rather than the posterior distribution. An article of Simon Tavaré et al. [5] was first to propose an ABC algorithm for posterior inference. In their seminal work, inference about the genealogy of DNA sequence data was considered, and in particular the problem of deciding the posterior distribution of the time to the most recent common ancestor of the sampled individuals. Such inference is analytically intractable for many demographical models, but the authors presented ways of simulating coalescent trees under the putative models. A sample from the posterior of model parameters was obtained by accepting/rejecting proposals based on comparing the number of segregating sites in the synthetic and real data. This work was followed by an applied study on modeling the variation in human Y chromosome by Jonathan K. Pritchard et al. [6] using the ABC method. Finally, the term Approximate Bayesian Computation was established by Mark Beaumont et al. [7], extending further the ABC methodology and discussing the suitability of the ABC-approach more specifically for problems in population genetics. Since then, ABC has spread to applications outside population genetics, such as systems biology, epidemiology, or phylogeography. METHOD Motivation A common incarnation of the Bayes theorem relates the conditional probability (or density) of a particular parameter value given data to the probability of given by the rule:where denotes the posterior, the likelihood, the prior, and the evidence (also referred to as the marginal likelihood or the prior predictive probability of the data). The prior represents beliefs about before is available, and it is often specified by choosing a particular distribution among a set of well-known and tractable families of distributions, such that both the evaluation of prior probabilities and random generation of values of are relatively straightforward. For certain kinds of models, it is more pragmatic to specify the prior using a factorization of the joint distribution of all the elements of in terms of a sequence of their conditional distributions. If one is only interested in the relative posterior plausibilities of different values of , the evidence can be ignored, as it constitutes a normalising constant, which cancels for any ratio of posterior probabilities. It remains, however, necessary to evaluate the likelihood and the prior . For numerous applications, it is computationally expensive, or even completely infeasible, to evaluate the likelihood [8], which motivates the use of ABC to circumvent this issue. The ABC Rejection Algorithm All ABC-based methods approximate the likelihood function by simulations, the outcomes of which are compared with the observed data [9]–[11]. More specifically, with the ABC rejection algorithm—the most basic form of ABC—a set of parameter points is first sampled from the prior distribution. Given a sampled parameter point , a dataset is then simulated under the statistical model specified by . If the generated is too different from the observed data , the sampled parameter value is discarded. In precise terms, is accepted with tolerance if:where the distance measure determines the level of discrepancy between and based on a given metric (e.g., the Euclidean distance). A strictly positive tolerance is usually necessary, since the probability that the simulation outcome coincides exactly with the data (event ) is negligible for all but trivial applications of ABC, which would in practice lead to rejection of nearly all sampled parameter points. The outcome of the ABC rejection algorithm is a sample of parameter values approximately distributed according to the desired posterior distribution and, crucially, obtained without the need of explicitly evaluating the likelihood function (Figure 1). [Figure: Parameter estimation by Approximate Bayesian Computation: a conceptual overview.] Summary Statistics The probability of generating a dataset with a small distance to typically decreases as the dimensionality of the data increases. This leads to a substantial decrease in the computational efficiency of the above basic ABC rejection algorithm. A common approach to lessen this problem is to replace with a set of lower dimensional summary statistics , which are selected to capture the relevant information in . The acceptance criterion in ABC rejection algorithm becomes: If the summary statistics are sufficient with respect to the model parameters , the efficiency increase obtained in this way does not introduce any error [12]. Indeed, by