Sandwich regression for accurate and robust estimation in generalized linear multilevel and longitudinal models

Elliot H. Young, Rajen D. Shah · 2024

Read Sandwich regression for accurate and robust estimation in generalized linear multilevel and longitudinal models by Elliot H. Young, Rajen D. Shah (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06119.txt.

Introduction

Elliot Young and Rajen Shah propose a statistics method that stays accurate even when the model behind it is wrong. Their 2024 paper works with generalized linear models in grouped and repeated-measures settings. Common tools force the user to state how outcomes vary together, and a wrong guess wastes accuracy.

The new method, sandwich regression, picks the lowest-variance estimator inside a chosen family. It does so across all data patterns that fit a loose model where only the average response has a set form. Tests on simulated and real data show gains over mixed models and estimating equations. The authors frame the method as both robust and efficient for messy applied data.

Catalog record

CREATORElliot H. Young, Rajen D. Shah
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06119

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sandwich regression for accurate and robust estimation in generalized linear multilevel and longitudinal models Elliot H. Young, Rajen D. Shah Abstract Generalized linear models are a popular tool in applied statistics, with their maximum likelihood estimators enjoying asymptotic Gaussianity and efficiency. As all models are wrong, it is desirable to understand these estimators’ behaviours under model misspecification. We study semiparametric multilevel generalized linear models, where only the conditional mean of the response is taken to follow a specific parametric form. Pre-existing estimators from mixed effects models and generalized estimating equations require specificaiton of a conditional covariance, which when misspecified can result in inefficient estimates of fixed effects parameters. It is nevertheless often computationally attractive to consider a restricted, finite dimensional class of estimators, as these models naturally imply. We introduce sandwich regression, that selects the estimator of minimal variance within a parametric class of estimators over all distributions in the full semiparametric model. We demonstrate numerically on simulated and real data the attractive improvements our sandwich regression approach enjoys over classical mixed effects models and generalized estimating equations. 1 Introduction All models are wrong, but some are useful. When a large quantity of high-quality data is available, it is natural to be minimal with regards to the model assumptions being imposed when performing statistical analyses, with assumption-lean estimators typically requiring data-driven estimation of nonparametric nuisance functions. When dealing with smaller datasets, these nonparametric estimations become impractical, and instead often lead one to impose certain structural assumptions on the data generating process. The generalized linear model (glm) ( Nelder and Wedderburn,, 1972 ) has proved a popular structure to impose, where the distribution of an outcome Y\in\mathbb{R} conditional on covariates X\in\mathbb{R}^{1\times p} lies in a finite dimensional parametric class. Together with a random design on the covariates, this defines a class of distributions \mathcal{P}_{\mathrm{GLM}} on the pair (Y,X) . The maximum likelihood estimator corresponding to \mathcal{P}_{\mathrm{GLM}} enjoys consistency, asymptotic Gaussianity and efficiency within this model. This estimator remains efficient within a broader class of models ( McCullagh,, 1983 ) provided the first two conditional moments implied by the glm hold i.e. the model \mathcal{P}_{\mathrm{QGLM}}:=\Big\{P:g\big(\mathbb{E}_{P}[Y\,|\,X]\big)=X\beta,\;\mathrm{Var}_{P}(Y\,|\,X)=v(g^{-1}(X\beta))\Big\}, where \beta\in\mathbb{R}^{p} is a fixed effect parameter of interest, g is a strictly increasing link function, and v is the variance function of the glm. This model is termed the quasi generalized linear model ( Wedderburn,, 1974 ) , with corresponding estimator the quasi maximum likelihood (QML) estimator, and extends immediately to allow the variance function v to be an arbitrary function of the mean. The model \mathcal{P}_{\mathrm{QGLM}} therefore provides extra flexibility and robustness over \mathcal{P}_{\mathrm{GLM}} , but arguably is still restrictive in terms of the form of the conditional variance \mathrm{Var}_{P}(Y\,|\,X) being imposed. This restriction can be alleviated ( McCullagh and Nelder,, 1989 ; Nelder and Pregibon,, 1987 ) to allow the variance function to take an arbitrary function of the covariates \mathcal{P}_{v}:=\Big\{P:g\big(\mathbb{E}_{P}[Y\,|\,X]\big)=X\beta,\;\mathrm{Var}_{P}(Y\,|\,X)=\nu_{\gamma}(X),\;\gamma\in\Gamma\Big\}, for some class of variance functions \{\nu_{\gamma}:\gamma\in\Gamma\} where \Gamma\subseteq\mathbb{R}^{q} for some q\in\mathbb{N} . This setting is sometimes termed pseudo maximum likelihood estimation ( Gourieroux and Monfort,, 1993 ; Ziegler,, 2011 ) . Given i.i.d. realisations (Y_{i},X_{i})\in\mathbb{R}\times\mathbb{R}^{p} from a distribution (not necessarily in \mathcal{P}_{v} ), and for each \gamma\in\Gamma defining a law in \mathcal{P}_{v} , a QML estimator \tilde{\beta}(\gamma) for \beta can be constructed, thus implying a class \mathcal{E}_{\textrm{QML}} of estimators parametrised by the dispersion parameter \gamma\in\Gamma ; \mathcal{E}_{\textrm{QML}}=\bigg\{\tilde{\beta}(\gamma)\in\mathbb{R}^{p}:\;\sum_{i}\psi_{\text{QML}}(Y_{i},X_{i};\tilde{\beta}(\gamma))=0,\\ \psi_{\text{QML}}(Y_{i},X_{i},\beta)=\frac{(Y_{i}-g^{-1}(X_{i}\beta))}{g^{\prime}(g^{-1}(X_{i}\beta))\,\nu_{\gamma}(X_{i})}X_{i}^{T},\;\gamma\in\Gamma\bigg\}, When the ‘true distribution’ lies within the class \mathcal{P}_{v} , then provided the ‘true \gamma ’ - that is the \gamma\in\Gamma that correctly specifies the variance function - can be estimated consistently, the resulting QML estimator for \beta is also efficient. In the semiparametric literature this property is termed local efficiency ( van der Laan and Robins,, 2003 ; Tsiatis,, 2006 ; Zhang et al.,, 2008 ; van der Laan and Rose,, 2018 ) . In practice two prominent methods are commonly employed for dispersion estimation. The first is extended quasi maximum likelihood estimation (EQML) ( Nelder and Pregibon,, 1987 ; McCullagh and Nelder,, 1989 ) , that fits the pseudo-likelihood obtained if Y\,|\,X were normally distributed with variance \nu_{\gamma}(X) . Note that in practice the terms EQML and QML are often used synonymously; for avoidance of doubt here we use EQML to refer to dispersion parameter estimation and QML to refer to specifically \beta estimation given a fixed dispersion \gamma\in\Gamma . The second strategy is common in the literature for generalized estimating equations (GEE), which aims to estimate the dispersion parameter via a least squares regression of the squared Pearson residuals onto the hypothesised (scaled) variance model. When the true distribution lies in \mathcal{P}_{v} both these strategies consistently estimate the optimal dispersion, resulting in locally efficient estimation. However, local efficiency says nothing with regards to the variance of the estimator when the true distribution lies outside this model \mathcal{P}_{v} . We will study the broader model that only assumes the first conditional moment implied by the glm \mathcal{P}_{\mathrm{\mu}}:=\Big\{P:g\big(\mathbb{E}_{P}[Y\,|\,X]\big)=X\beta\Big\}. Within \mathcal{P}_{\mathrm{\mu}} every estimator in \mathcal{E}_{\textrm{QML}} remains consistent and asymptotically Gaussian, albeit with differing variances. We will see that locally efficient estimators with respect to the class \mathcal{P}_{v} need not even be the estimator in \mathcal{E}_{\textrm{QML}} of minimal variance when the true distribution lies within \mathcal{P}_{\mathrm{\mu}}\backslash\mathcal{P}_{v} i.e. when the conditional variance model is misspecified, and in fact this suboptimality can be arbitrarily large (see Proposition 1 ). We begin by exploring a simple motivating example of the above in the clustered linear model (see Section 1.2.1 ). First we introduce a few details specific to the linear model. Figure 1 : Schematic to describe the hierarchy of models \mathcal{P}_{\mathrm{GLM}}\subset\mathcal{P}_{\mathrm{QGLM}}\subset\mathcal{P}_{v}\subset\mathcal{P}_{\mathrm{\mu}} we will consider. In our setup we consider a class of estimators \mathcal{E}_{\textrm{QML}} given by the quasi maximum likelihood \beta estimators obtained as laws span over \mathcal{P}_{v} (i.e. variance functions spanning some potentially misspecified parametric class \{\nu_{\gamma}:\gamma\in\Gamma\} ). We will be selecting from this class of estimators that of minimal variance under only the assumption that the true law lies in \mathcal{P}_{\mathrm{\mu}} . 1.1 Grouped data The clustered quasi-Gaussian linear model is an example of a quasi generalized linear model for grouped data, asserting each cluster, consisting of a multivariate Y and matrix of covariates X , satisfies \mathbb{E}_{P}[Y\,|\,X]=X\beta,\qquad\mathrm{Cov}_{P}(Y\,|\,X)=\Sigma_{\gamma}(X)\;(\gamma\in\Gamma), (1) for some variance model defined in terms of a dispersion parameter \gamma\in\Gamma . Given i.i.d. data (Y_{i},X_{i})\stackrel{{\scriptstyle d}}{{=}}(Y,X) the class of QML estimators in the quasi-Gaussian linear model takes the form of the weighted least squares estimators \mathcal{E}_{\textrm{QML}}=\bigg\{\tilde{\beta}(\gamma)=\Big(\sum_{i}X_{i}^{T}\Sigma_{\gamma}(X_{i})^{-1}X_{i}\Big)^{-1}\Big(\sum_{i}X_{i}^{T}\Sigma_{\gamma}(X_{i})^{-1}Y_{i}\Big):\,\gamma\in\Gamma\bigg\}. (2) A popular dispersion structure imposed for clustered data is the linear mixed effects model, which asserts Y\,|\,X\sim N\big(X\beta\,,\;X^{(\text{sub})}\mathcal{V}X^{(\text{sub})T}+\sigma^{2}I\big), for some subcollection X^{\text{(sub)}} of the full covariates X , positive definite matrix \mathcal{V} and \sigma>0 , which motivates the class of weighted least squares estimators ( 2 ) with \Sigma_{\gamma}(X)=X^{\text{(sub)}}\mathcal{V}X^{\text{(sub)}T}+\sigma^{2}I,\quad\gamma=(\mathcal{V},\sigma). The quadratic conditional covariance introduced here is motivated through the imposition of fictitious ‘random effects’, whose aim is to motivate an estimator of reduced variance to the inefficient OLS estimator. For longitudinal datasets the ARMA model is often used to imply a practical dispersion structure. Given a working model \Sigma_{\gamma} , to estimate the dispersion parameter \gamma the EQML and GEE losses for the grouped linear model ( 1 ) correspond to \displaystyle\gamma_{P,\mathrm{EQML}} \displaystyle:=\underset{\gamma\in\Gamma}{\mathrm{argmin}}\,\mathbb{E}_{P}\Big[\log\Sigma_{\gamma}(X)+\varepsilon^{T}\Sigma_{\gamma}(X)^{-1}\varepsilon\Big] \displaystyle\,=\underset{\gamma\in\Gamma}{\mathrm{argmin}}\,\mathbb{E}_{P}\Big[\log\Sigma_{\gamma}(X)+\mathrm{tr}\big\{\Sigma_{\gamma}(X)^{-1}\mathrm{Cov}_{P}(Y\,|\,X)\big\}\Big], \displaystyle\gamma_{P,\mathrm{GEE}} \displaystyle:=\underset{\gamma\in\Gamma}{\mathrm{argmin}}\,\mathbb{E}_{P}\Big[\big\|\Sigma_{\gamma}(X)-\varepsilon\varepsilon^{T}\|_{F}^{2}\Big] \displaystyle\,=\underset{\gamma\in\Gamma}{\mathrm{argmin}}\,\mathbb{E}_{P}\Big[\big\|\Sigma_{\gamma}(X)-\mathrm{Cov}_{P}(Y\,|\,X)\|_{F}^{2}\Big], (3) at the population level, where \|\cdot\|_{F} denotes the Frobenius

Read the complete text