High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO
Soham Ghosh, Sameer K. Deshpande · 2025
Read High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO by Soham Ghosh, Sameer K. Deshpande (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13007.txt.
Introduction
Soham Ghosh and Sameer Deshpande present a statistics method for studies with many mixed outcomes in a 2025 paper. A patient or site may yield both yes-no outcomes and measured ones, all possibly linked, while hundreds of candidate drivers compete to explain them. The team models the seen outcomes as glimpses of hidden continuous values.
Their method hunts for a sparse table of driver effects and a sparse web of links among outcomes at the same time. Special prior distributions push most entries to zero, and a tailored algorithm finds the best fit. Theory shows the method recovers the true nonzero effects as samples grow. Tests on simulated data and on three real problems, spanning medicine and ecology, performed well. The authors call the method mixed-mSSL.
Catalog record
| CREATOR | Soham Ghosh, Sameer K. Deshpande |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 124 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13007
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO Soham Ghosh, Sameer K. Deshpande Abstract We consider a high-dimensional multi-outcome regression in which q , q, possibly dependent, binary and continuous outcomes are regressed onto p p covariates. We model the observed outcome vector as a partially observed latent realization from a multivariate linear regression model. Our goal is to estimate simultaneously a sparse matrix ( 𝑩 \bm{B} ) of latent regression coefficients (i.e., partial covariate effects) and a sparse latent residual precision matrix ( Ω \Omega ), which induces partial correlations between the observed outcomes. To this end, we specify continuous spike-and-slab priors on all entries of 𝑩 \bm{B} and off-diagonal elements of Ω \Omega and introduce a Monte Carlo Expectation-Conditional Maximization algorithm to compute the maximum a posterior estimate of the model parameters. Under a set of mild assumptions, we derive the posterior contraction rate for our model in the high-dimensional regimes where both p p and q q diverge with the sample size n n and establish a sure screening property, which implies that, as n n increases, we can recover all truly non-zero elements of 𝑩 \bm{B} with probability tending to one. We demonstrate the excellent finite-sample properties of our proposed method, which we call mixed-mSSL , using extensive simulation studies and three applications spanning medicine to ecology. 1 Introduction In many applications spanning biology, ecology, and medicine, analysts are interested in understanding the effect of p covariates on q , possibly related outcomes. When all the outcomes are continuous, a multivariate linear regression model is a natural starting point. In this setting, jointly modeling the regression coefficient and residual covariance matrices has many advantages over fitting separate models to each outcome: estimates of the regression coefficients are asymptotically more efficient ( Zellner, 1962 ) and in high-dimensional settings, joint models can detect smaller covariate effects than separate models ( Deshpande et al., 2019 ) and yield improved predictions ( Li et al., 2023 ) . But when the outcomes are of mixed-type — that is, when some outcomes are continuous and others are discrete — specifying joint models capturing the residual dependence between the outcomes is much more complicated. While some authors decompose the joint outcomes into a product of conditional densities ( Fitzmaurice and Laird, 1997 , see, e.g.,) , many others rely on latent variables to induce dependence between discrete and continuous outcomes ( Dunson, 2000 ; McCulloch, 2008 ; Wagner and Tüchler, 2010 ; Canale and Dunson, 2011 ; Canale and Dunson, 2013 ; Canale and Dunson, 2014 ; Kowal and Canale, 2020 ; Bradley, 2022 , see, e.g.,) . In this work, we follow these latter examples and express our observed mixed-type outcomes as partially observed realizations from a multivariate linear regression model. More specifically, suppose that we observe n pairs (\bm{x}_{1},\bm{y}_{1}),\ldots,(\bm{x}_{n},\bm{y}_{n}) of a p -dimensional covariate vector \bm{x}\in\mathbb{R}^{p} and a q -dimensional outcome vector \bm{y}. To estimate simultaneously the effect of each covariate on each outcome and the residual dependence structure between the outcomes, we introduce a latent variable \bm{z}\in\mathbb{R}^{q} such that (i) \bm{z}\sim\mathcal{N}_{q}\left(\bm{B}^{\top}\bm{x},\Omega^{-1}\right) and (ii) \bm{y}=g(\bm{z}), where g is a deterministic function (defined formally in Section 2 ); \bm{B} is a p\times q matrix of unknown regression coefficients; and \Omega is a q\times q positive-definite precision matrix capturing the latent residual dependence structure. Our work is motivated by the following three applications. • Biomarkers for kidney disease: Ebiaredoh-Mienye et al., 2022 studied the effects of p=24 clinical covariates on a binary indicator of chronic kidney disease (CKD) status and a continuous measure of urine-specific gravity (SG) using data from n=400 patients. Here, we wish to identify which clinical covariates are most predictive of both CKD and SG and to quantify the residual dependence between these outcomes unexplained by these covariates. • Colorectal cancer and the microbiome: Wirbel et al., 2019 conducted a meta-study of five colorectal-cancer (CRC) cohorts totaling n=574 subjects and p=849 prevalent bacterial species in the gut microbiome. While their primary focus was on the binary CRC status, there is growing evidence that both Type-2 diabetes and body‐mass index (BMI) share overlapping microbial signatures with CRC risk ( Qin et al., 2012 ; Mandic et al., 2024 ) . In our re-analysis, we wish to identify those specifies most associated with CRC status, diabetes status, and BMI, while simultaneously quantifying their residual dependence. • Finnish birds: Lindström et al., 2015 surveyed n=137 breeding-season transects in Finland and recorded presence-absence for the q=50 most common bird species, together with four continuous climate covariates. In this context, entries in \bm{B} capture how changes in climatic conditions affect species’ presence while off-diagonal elements in \Omega may reflect biotic interactions such as competition or facilitation that cannot be explained by the measured climatic factors alone. For the motivating CRC and Finnish bird datasets, the total number of model parameters (pq+q(q-1)/2) exceeds the sample size n. In such high-dimensional settings, it is common to assume that \bm{B} and/or \Omega is sparse. Shrinkage priors that pull negligible parameter values towards zero while shrinking truly important values less aggressively are extremely popular in the Bayesian literature. Two complementary classes of such priors dominate current practice: (i) global-local shrinkage priors that model parameters as being drawn from continuous scale mixtures ( Bhadra et al., 2017 , e.g.,) and (ii) spike-and-slab priors that model parameters as being drawn from either a distribution sharply concentrated at zero (the “spike”) or a diffuse distribution (the “slab”) ( George and McCulloch, 1993 ; Ishwaran and Rao, 2005 , e.g.,) . In mixed-outcome regression, Wang et al., 2025 adopt the first approach: their mt-MBSP model combines a latent-variable construction with horseshoe-type priors to handle p\gg n cases, but it fixes the number of outcomes q and does not learn residual correlations. At the other end, the BS-MRMR framework of Chen et al., 2023 uses spike-and-slab priors with a point-mass spike to estimate both \bm{B} and \Omega , yet it is feasible only when n>p . Very recent algorithmic advances have begun to scale spike-and-slab learning to high dimensions ( Zhou et al., 2022 ) , motivating us to revisit spike-and-slab priors in the mixed-response setting. Motivated by the simultaneous needs to (i) model both binary and continuous outcomes and (ii) learn a sparse \bm{B} and a sparse \Omega when either p or q grows with n, while remaining computationally feasible, we propose a new method, termed mixed-mSSL . It represents the observed outcomes as a partially observed realization from a multivariate linear regression model with a sparse matrix of covariate effects \bm{B} and a sparse residual precision matrix \Omega. We further specify spike-and-slab LASSO ( Ročková and George, 2018 , SSLASSO ;) priors on each element of \bm{B} and the off-diagonal elements of \Omega. In this way, mixed-mSSL generalizes the univariate SSLASSO and its extension to the sparse multivariate linear regression setting ( Deshpande et al., 2019 , mSSL ;) . Like these methods, we compute the maximum a posteriori (MAP) estimate of \bm{B} and \Omega using an EM-like algorithm that iteratively solves a series of penalized maximum likelihood problems in which the penalties are adaptively and automatically updated. Because the latent variable \bm{z} in our model precludes