Quasi-Bayesian Variable Selection: Model Selection without a Model
Beniamino Hadj-Amar, Jack Jewson · 2026
Read Quasi-Bayesian Variable Selection: Model Selection without a Model by Beniamino Hadj-Amar, Jack Jewson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12767.txt.
Introduction
Beniamino Hadj-Amar and Jack Jewson put forward a way to pick key variables with less faith in a full model. The Bayesian route to this task is strong and well liked. It builds in a bias for short lists and states how unsure it is. Its picks can be sound and work well on small samples.
True doubt scores need the model to be right, though. When the model is wrong, its picks and scores can mislead. Their quasi Bayesian route keeps the style but drops the full model. It is framed as model choice with no model at all. The work serves fields where all models are known to be rough. Health and social data are two such fields.
Catalog record
| CREATOR | Beniamino Hadj-Amar, Jack Jewson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 122 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12767
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quasi-Bayesian Variable Selection: Model Selection without a Model Beniamino Hadj-Amar, Jack Jewson Abstract Bayesian inference offers a powerful framework for variable selection by incorporating sparsity through prior beliefs and quantifying uncertainty about parameters, leading to consistent procedures with good finite-sample performance. However, accurately quantifying uncertainty requires a correctly specified model, and there is increasing awareness of the problems that model misspecification causes for variable selection. Current solutions to this problem either require a more complex model, detracting from the interpretability of the original variable selection task, or gain robustness by moving outside of rigorous Bayesian uncertainty quantification. This paper establishes the model quasi-posterior as a principled tool for variable selection. We prove that the model quasi-posterior shares many of the desirable properties of full Bayesian variable selection, but no longer necessitates a full likelihood specification. Instead, the quasi-posterior only requires the specification of mean and variance functions, and as a result, is robust to other aspects of the data. Laplace approximations are used to approximate the quasi-marginal likelihood when it is not available in closed form to provide computational tractability. We demonstrate through extensive simulation studies that the quasi-posterior improves variable selection accuracy across a range of data-generating scenarios, including linear models with heavy-tailed errors and overdispersed count data. We further illustrate the practical relevance of the proposed approach through applications to real datasets from social science and genomics. Keywords: Variable Selection; Generalized-Bayes; Gibbs Posteriors; Quasi-Posteriors; 1 Introduction The Bayesian paradigm traditionally combines prior distributions for model parameters with the likelihood of data to produce posterior inferences. This framework is particularly desirable when the inference of interest is to identify the subset of possible predictors x\in\mathcal{X}\subseteq\mathbb{R}^{p} that are relevant for predicting a response y\in\mathcal{Y}\subseteq\mathbb{R} . Such an inference task is generally referred to as variable selection, a specific type of model selection. The prior encodes beliefs about the number of active predictors and the posterior quantifies the confidence that any given predictor is truly active. As a result, Bayesian variable selection naturally penalizes overly complex models, favoring simpler models that are well supported by the data. This enables consistent identification of the true model among a set of nested candidates ( Dawid 2011 , e.g.) , and has been shown to yield strong finite-sample performance in a variety of settings ( Rossell & Rubio 2018 , e.g.) . This article considers the class of generalised linear models (GLMs) ( McCullagh & Nelder 1989 ) that use a linear predictor to define \mathbb{E}\left[Y\mid X=x\right]=\mu(x^{T}\beta), (1) where \beta\in\mathbb{R}^{p} are model parameters capturing the extent to which each predictor affects the response y and \mu^{-1}(\cdot):\mathbb{R}\mapsto\mathcal{Y} is known as the GLM link function. In order to conduct variable selection, we introduce \gamma=(\gamma_{1},\ldots,\gamma_{p})\in\{0,1\}^{p} with \gamma_{j}=\mathbb{I}(\beta_{j}\neq 0) . Here, \gamma_{j}=0 means that variable x_{j} is not related to response y , i.e. is inactive. We refer to each unique configuration of predictors \gamma , as a model. A considerable drawback of the traditional Bayesian paradigm, however, is the need to specify a full likelihood for y\mid X in order to define the model posterior \pi(\gamma\mid y,X) . This requires ( 1 ) to be furnished with a distribution F(\cdot\mid X) , with density or mass function f , that describes in much greater detail how y depends on X than what is specified by ( 1 ). Recent works have demonstrated that misspecifying the model distribution F relative to the true data-generating distribution F_{0} can negatively affect the performance of Bayesian variable selection. Rossell & Rubio 2018 , Rossell & Rubio 2023 observed a decrease in finite-sample power to detect truly active covariates when the data have heavier tails than those assumed by F . Conversely, Grünwald et al. 2017 , Heide et al. 2020 demonstrated an increased rate of false positives relative to the predictive risk minimising model when the data contain inliers. Fontana & Sansó 2023 show that Bayesian variable selection tends to select unnecessarily complex models when ( 1 ) is also misspecified. Similar phenomenon have also been observed in Bayesian mixture models ( Miller & Dunson 2018 , Cai et al. 2021 ) and autoregressive time series models ( Miller & Dunson 2018 ) . In this paper, we demonstrate that the quasi-posterior ( Agnoletto et al. 2025 ) , a generalized Bayesian posterior ( Bissiri et al. 2016 ) based on the quasi-log-likelihood ( McCullagh 1983 , Wedderburn 1974 ) , allows for variable selection that retains the desirable properties of traditional Bayesian variable selection procedures while avoiding the need to correctly specify a full likelihood model F , thus mitigating the risk of misspecification. Instead, the quasi-likelihood enables calibrated inference on the regression parameters \beta and corresponding model \gamma that only requires specification of the mean and variance of the conditional distribution y\mid x . Furthermore, unlike other generalised Bayesian methods, there is no need to tune a ‘learning-rate’ parameter to achieve this. Existing approaches to address model misspecification in Bayesian inference can be broadly categorized into two philosophies: (i) modifying the model, or (ii) modifying the updating rule. Methods falling under (i) include those that specify more complex models with greater flexibility to adapt to the data, e.g. Bayesian non-parametric regression models ( De Iorio et al. 2004 , Müller & Quintana 2004 , e.g.) , or models specifically designed to be robust to particular types of misspecifications, such as outliers ( Berger et al. 1994 , Rossell & Rubio 2018 , e.g.) . However, more complex models require larger sample sizes and greater computational resources to fit, and are generally less interpretable than simpler structures such as ( 1 ). Further, even models designed to be robust are still inevitably misspecified to some degree. In contrast, category (ii) takes a different approach, modifying not the model itself, but the rule by which information from the data is incorporated into the prior. This includes methods that raise the likelihood to a power when conducting posterior inferences ( Walker & Hjort 2001 , Zhang 2006 , Grünwald et al. 2017 , Miller & Dunson 2018 , Lyddon et al. 2018 , Syring & Martin 2019 , Wu & Martin 2023 ) or update the prior using the exponential of a loss function rather than a likelihood ( Bissiri et al. 2016 , Syring & Martin 2017 , Jewson et al. 2018 , Chérief-Abdellatif & Alquier 2020 , Knoblauch et al. 2022 ) . However, departing from Bayes’ rule as the basis for posterior updating removes the principled probabilistic foundation of these methods. As a result, the validity of their uncertainty quantification often relies on carefully tuned parameters, which has so far limited their applicability to variable and model selection problems. To overcome these limitations, we establish the model quasi-posterior as a principled tool for Bayesian variable selection, by extending the framework of Agnoletto et al. 2025 from parameter inference to model uncertainty. We show that, in addition to delivering calibrated posterior uncertainty for parameters, the quasi-posterior provides calibrated uncertainty over models, in the sense that its model posterior distribution converges asymptotically to that of a correctly specified Bayesian model. This enables practitioners to quantify model uncertainty and perform variable selection using interpretable mean and variance structures relating y to x