Empirical Bayes for Data integration
Paul Rognon-Vael, Pompeu Fabra University and David Rossell, Pompeu Fabra University · 2025
Read Empirical Bayes for Data integration by Paul Rognon-Vael, Pompeu Fabra University and David Rossell, Pompeu Fabra University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08336.txt.
Introduction
A new study rarely starts from nothing: earlier work leaves summaries, estimates, and lists of promising variables behind. Paul Rognon-Vael and David Rossell build a transfer method on empirical Bayes, which lets past data set the starting beliefs for the current analysis. They spell out how that differs from full Bayesian updating and make it practical to compute.
Their theory shows the empirical version can still pick the right variables under weaker assumptions than full Bayes needs. It also settles on answers faster. In high-dimensional regression tests, full Bayes performed beautifully on its own. Borrowing strength through empirical Bayes still added a moderate, real improvement. The authors pitch the method at fields where raw old data are gone but summaries survive.
Catalog record
| CREATOR | Paul Rognon-Vael, Pompeu Fabra University and David Rossell, Pompeu Fabra University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 159 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08336
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Empirical Bayes for Data integration Paul Rognon-Vael, Pompeu Fabra University and David Rossell, Pompeu Fabra University Abstract . We discuss the use of empirical Bayes for data integration, in the sense of transfer learning. Our main interest is in settings where one wishes to learn structure (e.g. feature selection) and one only has access to incomplete data from previous studies, such as summaries, estimates or lists of relevant features. We discuss differences between full Bayes and empirical Bayes, and develop a computational framework for the latter. We discuss how empirical Bayes attains consistent variable selection under weaker conditions (sparsity and betamin assumptions) than full Bayes and other standard criteria do, and how it attains faster convergence rates. Our high-dimensional regression examples show that fully Bayesian inference enjoys excellent properties, and that data integration with empirical Bayes can offer moderate yet meaningful improvements in practice. Keywords: Data integration, Transfer learning, Empirical Bayes, Frequentist properties, Variable selection 1. Introduction Data integration is the task of combining multiple datasets or sources of information into a single analysis. It has the potential to improve inference and/or predictions, particularly in settings where data are scarce relative to the problem’s complexity (number of parameters, models or questions being considered). This is a standard statistical problem that is receiving renewed attention within the context of transfer learning. Therein, one wishes to transfer findings obtained from a setting where one has large or cheaply obtained data, to another setting where data are scarcer or costlier. We discuss the role of empirical Bayes methodology for data integration. In contrast to parameter estimation or prediction problems, which have been better studied, we focus on structural learning. By structural learning we refer to model selection tasks where one seeks to learn about structural features of the data-generating truth, say whether a parameter is zero, and we use variable selection as a canonical example. There are two main scenarios where one may seek to integrate data, illustrated in Figure 1 . The first (left panel) is when one has full access to multiple datasets. For example, one wishes to study multiple cancer types, and one has a separate dataset for each cancer type. Or, one has data from multiple schools and wants to learn common traits for low academic performance. We refer to this scenario as data integration with full data . There is a well-developed literature for this setting. A popular approach is to use hierarchical models that share information across datasets, see for instance Gelman and Hill 2007 for a monograph on mixed effects models. For data integration in covariance models see Avalos-Pacheco et al. 2022 ; Chandra et al. 2024 , in clustering see Lock and Dunson 2013 and in functional data analysis see Yang et al. 2015 . See also Jacob et al. 2017 ; Nott et al. 2023 and references therein for data integration strategies when the assumed model is misspecified. Our main focus is in the second scenario (Figure 1 , right), which occurs when one is interested in a single dataset \bm{y} , and one only has full access to that data. One also has other information that could be useful, for example summaries of previous datasets (denoted \bm{Z}=(\bm{z}_{1},\ldots,\bm{z}_{q}) in Figure 1 ). This second scenario is probably more common and certainly more challenging, because one does not have full access to the previous data and the provided summaries may be poor. An application that we illustrate later is a gene expression study in human colon cancer, where one has a list of genes that were found to be important in mice. The data analyst has full access to the human data, but only the list of genes from the mice experiment. We refer to this scenario as data integration with meta-covariates , i.e. with summaries \bm{Z} that may be informative as to what parameters are more likely to play an important role in the main data of interest. Although this scenario received less attention than data integration with full data, there is a literature on the topic. To mention some examples, Stingo et al. 2011 ; Cassese et al. 2014 proposed Bayesian variable selection for gene expression where prior inclusion probabilities depend on biological knowledge and meta-covariates. Chen et al. 2021 predicted disease outcomes by allowing LASSO penalties to depend on functional annotations. In causal analysis, Belloni et al. 2012 ; Antonelli and Dominici 2021 ; Papaspiliopoulos et al. 2025 penalized the inclusion of control covariates using their degree of association with the treatment of interest. See van de Wiel et al. 2019 for an overview on the use of meta-covariates in regression. Beyond regression, in Gaussian graphical models Peterson et al. 2016 and Jewson et al. 2023 used node and edge covariates to drive prior edge inclusion probabilities. And in factor models, Schiavon et al. 2022 used meta-covariates to determine non-zero loadings. Figure 1 . Data integration with full data (left) and with meta-covariates (right). With full data one observes q datasets with underlying parameters \bm{\theta}_{1},\ldots,\bm{\theta}_{q} with a hyper-parameter that allows sharing information. With meta-covariates one has access to summaries \bm{z}_{1},\ldots,\bm{z}_{q} extracted from unobserved data \tilde{\bm{y}}_{1},\ldots,\tilde{\bm{y}}_{q} . These summaries inform the parameter \bm{\theta} governing the dataset of interest \bm{y} Bayesian statistics provide a natural framework to integrate prior data such as meta-covariates into the analysis pipeline. In principle it suffices to set a prior on the parameters of interest ( \bm{\theta} in Figure 1 , right panel) that depends on the meta-covariates \bm{Z} and some hyper-parameters \bm{\omega} . There may be situations where an expert is willing to set such a prior subjectively. For example a biologist believes that if a gene is known to be involved in gastrointestinal cancer, then it has 0.9 prior probability of also being involved in colorectal cancer. Alas this situation is rather rare: experts do not have such a precise knowledge of \bm{\omega} , and it is cumbersome to specify such a prior when there are q>1 meta-covariates. This important because, if the prior grossly failed to reflect the true effect of \bm{Z} (i.e., the expert provided a poor prior for \bm{\omega} ), then one may obtain worse inference than if \bm{Z} were not used at all. Our goal is to explain how empirical Bayes can be a useful strategy for data integration. By modeling the relation between the meta-covariates \bm{Z} and the parameters \bm{\theta} governing the observed data \bm{y} , one may learn \bm{\omega} and hence integrate \bm{Z} in a way that’s less sensitive to misspecification. We remark that empirical Bayes can be seen as a frequentist method where the model-fitting criteria are based on Bayesian thinking, and as such it can help reconcile long-standing debates between frequentist and Bayesian inference, see for example Efron 2024 for a recent discussion. The paper is structured as follows. Section 2 reviews basic notions related to empirical Bayes and to the frequentist validity of Bayesian methods, in the more common setting of data integration with full data. We also point to some literature discussing the benefits of empirical Bayes, which largely focus on parameter estimation and the full data setting. In Section 3 we outline an empirical Bayes framework for data integration with meta-covariates. For concreteness we focus on