G-HIVE: Parameter Estimation and Approximate Inference for Multivariate Response Generalized Linear Models with Hidden Variables
Inbeom Lee Yang Ning · 2025
Read G-HIVE: Parameter Estimation and Approximate Inference for Multivariate Response Generalized Linear Models with Hidden Variables by Inbeom Lee Yang Ning (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00196.txt.
Introduction
Inbeom Lee and Yang Ning present G-HIVE in 2025, a framework for regression models with hidden variables and many outcomes. Hidden variables are unmeasured factors tied to both the outcome and the inputs, and they can bias estimates. Past work mostly covers linear models, where the link is straight.
Generalized linear models allow curved links, but hidden factors then cloud identification, estimation, and inference at once. G-HIVE rests on a factor model view and builds a modified quasi-likelihood from reweighted estimating equations. The key is a weight choice that removes the first-order bias from hidden factors. The unified frame handles multivariate outcomes in one pass.
Catalog record
| CREATOR | Inbeom Lee Yang Ning |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 164 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-00196
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
G-HIVE: Parameter Estimation and Approximate Inference for Multivariate Response Generalized Linear Models with Hidden Variables Inbeom Lee Yang Ning Abstract In practice, there often exist unobserved variables, also termed hidden variables, associated with both the response and covariates. Existing works in the literature mostly focus on linear regression with hidden variables. However, when the regression model is non-linear, the presence of hidden variables leads to new challenges in parameter identification, estimation, and statistical inference. This paper studies multivariate response generalized linear models (GLMs) with hidden variables. We propose a unified framework for parameter estimation and statistical inference called G-hive , short for G eneralized - HI dden V ariable adjusted E stimation. Specifically, based on factor model assumptions, we propose a modified quasi-likelihood approach to estimate an intermediate parameter, defined through a set of reweighted estimating equations. The key of our approach is to construct the proper weight, so that the first-order asymptotic bias of the estimator can be removed by orthogonal projection. Moreover, we propose an approximate inference framework for uncertainty quantification. Theoretically, we establish the first-order and second-order asymptotic bias and the convergence rate of our estimator. In addition, we characterize the accuracy of the Gaussian approximation of our estimator via the Berry–Esseen bound, which justifies the validity of the proposed approximate inference approach. Extensive simulations and real data analysis results show that G-hive is feasibly implementable and can outperform the baseline method that ignores hidden variables. Keywords : Generalized linear models, multivariate response data, non-linear regression, hidden variables, unmeasured confounders, parameter estimation, approximate inference. 1 Introduction In many regression problems, due to measurement limitations or ethical considerations, there often exist unobserved variables, also referred to as hidden variables. For example, in the analysis of high-throughput genomic data, researchers have long been aware that the measurements can be affected by many unobserved factors such as laboratory conditions, preparation time, and reagent lots ( Irizarry et al. 2005 ; Luo and Wei 2019 ) . These factors are called batch effects, which can be modeled as hidden variables ( Leek and Storey 2007 ) . Similarly, in biomedical studies, the onset of a disease is likely associated with several unmeasured variables, such as environmental factors or habitual patterns ( Katsaouni et al. 2021 ) . Ignoring hidden variables in the statistical analysis may introduce estimation bias and potentially lead to misleading scientific conclusions. Therefore, there is a pressing need to develop statistical methods that deal with hidden variables in a general regression framework, and that in particular, are applicable to binary or categorical data. This paper studies the multivariate response generalized linear model with hidden variables. Specifically, we assume that the M -dimensional response variable Y=(Y_{1},...,Y_{M})^{T} given the observed covariates X\in\mathbb{R}^{p} and hidden variables Z\in\mathbb{R}^{K} follows the generalized linear model (GLM) with the canonical link \displaystyle f(Y_{m}|X,Z)=\exp\big[\{Y_{m}\cdot(\Theta_{m}X+B_{m}Z)-b(\Theta_{m}X+B_{m}Z)\}/\phi\penalty\ +\penalty\ c(Y_{m},\phi)\big] (1) where b(\cdot) and c(\cdot) are known functions and \phi is the dispersion parameter. The parameters \Theta_{m} and B_{m} are the m -th row of coefficient matrices \Theta\in\mathbb{R}^{M\times p} and B\in\mathbb{R}^{M\times K} , respectively. Given n i.i.d copies of (Y,X) , we are interested in the estimation and inference of the coefficient matrix \Theta , the association between X and Y in the presence of hidden variables Z . In this work, we consider the regime where p,M,K are all allowed to grow with the sample size n , where p\leq n and K\leq M hold. In this work, we propose a unified framework for parameter estimation and statistical inference called G-hive , short for G eneralized - HI dden V ariable adjusted E stimation. Since the hidden variable Z is random and unobserved, the coefficient matrix \Theta is generally not identifiable. To make \Theta (asymptotically) identifiable and estimable, we impose a factor model in ( 2 ) that relates X and Z ( Bai 2003 ; Fan et al. 2013 ; Fan et al. 2008 ) . However, under these model assumptions, the distribution of Y given X does not follow a GLM and is indeed intractable, since we do not impose any parametric assumption on the distribution of Z . To overcome this challenge, we carefully construct a modified quasi-likelihood for a new estimand F^{*} , which is defined through a set of reweighted estimating equations. The rationale behind the reweighted estimating equations is that the resulting first-order approximation of the bias F^{*}-\Theta is shown to belong to the column space of B , which can be removed by estimating the projection matrix P_{B}=B(B^{T}B)^{-1}B^{T} . The intuition of our approach is explained in Section 2.2 . Under certain identifiability conditions, using F^{*} as a bridge, we introduce the estimator \hat{\Theta}=\hat{P}_{B}^{\perp}\hat{F} , where \hat{P}_{B} is obtained by applying PCA to a carefully constructed weighted covariance matrix and \hat{F} is the maximum modified quasi-likelihood estimator. Since our approach yields a tight pipeline, it is straightforward to implement in practice. In addition, we propose an approximate inference framework for uncertainty quantification. Unlike classical inference results for models without hidden variables, a new, unpleasant phenomenon of the estimator \hat{\Theta} is that the asymptotic bias may dominate the stochastic error. Consequently, the limiting distribution of the estimator \hat{\Theta} is no longer centered at \Theta with the \sqrt{n}- rate. To address this issue, we shift the target parameter from \Theta to P_{B}^{\perp}F^{*} (or F^{*} ), which corresponds to the second-order (or first-order) approximation of \Theta . Intuitively, P_{B}^{\perp}F^{*} can be viewed as the correct limiting value of \hat{\Theta} , and therefore a confidence interval based on the limiting distribution of \hat{\Theta} yields the desired coverage probability for P_{B}^{\perp}F^{*} . For this reason, we refer to this approach as second-order approximate inference. Theoretically, our first key result shows that the approximation bias satisfies \|F^{*}-\Theta\|_{F}/\sqrt{M}=O(1/\sqrt{p}) and \|P_{B}^{\perp}F^{*}-\Theta\|_{F}/\sqrt{M}=O(1/p) . An interesting implication of this result is that collecting more observed covariates can mitigate the approximation bias. Moreover, it also explains why P_{B}^{\perp}F^{*} (or F^{*} ) is called the second-order (or first-order) approximation of \Theta in our inference framework. Next, we establish the convergence rate of the estimator \hat{\Theta} . In particular, we show that, under mild conditions (e.g., M is large enough), the convergence rate of \hat{\Theta} is faster than that of \hat{F} , which corresponds to the baseline naive estimator that ignores the hidden variables. Finally, we characterize the accuracy of the Gaussian approximation of \hat{\Theta} via the Berry–Esseen bound, which justifies the validity of the proposed approximate inference approach. 1.1 Related Literature This work is most related to surrogate variable analysis (SVA) proposed by Leek and Storey 2007 , and more recently developed by Lee et al. 2017 ; Wang et al. 2017 ; McKennan and Nicolae 2019 ; Bing et al. 2022 ; Bing et al. 2023 , the last of which proposed a novel factor model based bias correction approach for multivariate response linear regression with hidden variables, which is a special case of GLMs. However, their approach is only applicable to linear regression. The challenge of extending their approach to GLMs is detailed in Section 2.1 . Compared to these works, our main methodological novelty is that we propose to calibrate the residual by an approximate inverse variance weighting scheme. Such a calibration step is essential under the GLM for parameter identification,