Simulations for estimation of random effects and overall effect in three-level meta-analysis of standardized mean differences using constant and inverse-variance weights
Elena Kulinskaya, David C. Hoaglin · 2024
Read Simulations for estimation of random effects and overall effect in three-level meta-analysis of standardized mean differences using constant and inverse-variance weights by Elena Kulinskaya, David C. Hoaglin (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00795.txt.
Description
Abstract We consider a three-level meta-analysis of standardized mean differences. The standard method of estimation uses inverse-variance weights and REML/PL estimation of variance components for the random effects. We introduce new moment-based point and interval estimators for the two variance components and related estimators of the overall mean.
Catalog record
| CREATOR | Elena Kulinskaya, David C. Hoaglin |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-00795
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Simulations for estimation of random effects and overall effect in three-level meta-analysis of standardized mean differences using constant and inverse-variance weights Elena Kulinskaya, David C. Hoaglin Abstract We consider a three-level meta-analysis of standardized mean differences. The standard method of estimation uses inverse-variance weights and REML/PL estimation of variance components for the random effects. We introduce new moment-based point and interval estimators for the two variance components and related estimators of the overall mean. Similar to traditional analysis of variance, our method is based on two conditional Q Q statistics with effective-sample-size weights. We study, by simulation, bias and coverage of these new estimators. For comparison, we also study bias and coverage of the REML/PL-based approach as implemented in rma.mv in metafor . Our results demonstrate that the new methods are often considerably better and do not have convergence problems, which plague the standard analysis. Keywords hierarchical model, inverse-variance weights, effective-sample-size weights, random effects, heterogeneity 1 Introduction When viewed as a multilevel model ( Raudenbush and Bryk [1985] , Goldstein et al. [2000] ), the standard random-effects model of meta-analysis is a two-level hierarchical model. Typically, the level-1 units comprise the sample within each study, and the level-2 units are the studies. A three-level model involves a further level of structure; for example, the studies may be nested within clusters. Konstantopoulos [2011] discusses the traditional analysis of a three-level model in detail. Cheung [2014] formulated a three-level model in the framework of structural-equation modelling. Multilevel meta-analysis is implemented in the rma.mv procedure in metafor [ Viechtbauer, 2010 ] . Traditional analysis uses REML-based estimation of the variance components for the random effects and profile-likelihood estimation of their confidence intervals. In a meta-analysis involving three levels, one can apply moment estimation similar to that in analysis of variance, separating the total variation into within and between components. Our approach uses constant weights based on effective sample sizes. In a related development, in multivariate meta-analysis, Jackson et al. [2024] proposed a method of moment-based estimation of the between-study covariance matrix (analogous to the heterogeneity variance in the usual random-effects model). In that method, the weights for estimating the overall effects involve an estimate of the covariance matrix, which is then updated to reflect the current estimate of the overall effects. By using constant weights, we avoid that complication. In a systematic review of multilevel meta-analyses, Fernández-Castilla et al. [2020] listed existing simulation studies (their Table 1) and the parameters from 178 actual meta-analyses in behavioral, social, biological, and medical sciences (their Tables 3–5). Our simulation scenarios were largely guided by those tables. Of those 178 meta-analyses, 162 fitted the three-level model, and 80 used the standardized mean difference (SMD) as the effect measure. For SMD as the effect measure, our simulation study evaluated our novel moment-based method of estimation in a three-level meta-analysis and compared it with the standard REML/PL-based methods implemented in rma.mv in metafor . 2 A three-level hierarchical model for meta-analysis We consider a 3-level hierarchical model involving M clusters of studies: K_{g} studies in cluster g ( g=1,\ldots,M ) and sample size n_{gi} for study i in cluster g ( i=1,\ldots,K_{g} ). The model includes random variation at each level: level-1, within-study random errors \epsilon_{g}\sim MVN(0,\hbox{diag}(\sigma_{gi}^{2})) ; level-2, within-cluster between-studies random effects \eta_{g}\sim MVN(0,\tau^{2}I_{K_{g}}) ; and level-3, between-clusters random effects \zeta\sim MVN(0,\omega^{2}I_{M}) ; where I_{K_{g}} and I_{M} are unit matrices of dimension K_{g} and M , respectively. In this model, the level-2 parameter, \tau^{2} , is not cluster-specific, so estimation can use the full data. Cluster-specific variances, \tau^{2}_{g} , would require within-cluster estimation. Our methods can handle the latter scenario, but we have not done any simulations for it. Similar to a two-level meta-regression model with random effects, our model for the T_{g} , the summary statistics in cluster g , involves \eta_{g} and \epsilon_{g} : T_{g}=a_{0g}1_{g}+Y_{g}\beta_{0}+\eta_{g}+\epsilon_{g}, (2.1) where a_{0g} is an effect of cluster g , 1_{K_{g}} is a vector of 1’s of dimension K_{g} , Y_{g} is a K_{g}\times p design matrix, \beta_{0} is a p -vector of fixed effects (without an intercept), and \eta_{g} and \epsilon_{g} represent independent between- and within-study variation. It follows that, conditionally, \hbox{Cov}(T_{g}|a_{0g})=\Sigma_{g}=\hbox{diag}(\tau^{2}+\sigma_{gi}^{2}) . To write the model in Equation ( 2.1 ) in terms of the stacked vector T=(T_{1}^{T},\ldots,T_{M}^{T})^{T} of length {\mathcal{K}}=\sum K_{g} , we define the parameter vector \beta=(a_{01},a_{02},\ldots,a_{0M},\beta_{0}^{T})^{T} of dimension (M+p) and the design matrices X_{g}=(\delta_{g1}1_{K_{g}},\ldots,\delta_{gM}1_{K_{g}},Y_{g}) , where \delta_{gt} is the Kronecker delta ( \delta_{gt}=1 if g=t and 0 otherwise). We form the {\mathcal{K}}\times(M+p) design matrix X by stacking the X_{g} . Thus, also using stacked error vectors \eta and \epsilon , Equation ( 2.1 ) becomes T=X\beta+\eta+\epsilon. (2.2) This model differs from the standard meta-regression model only by including in \beta the M -vector of random intercepts a_{0}=(a_{01},\ldots,a_{0M})^{T} ; the K_{g} studies in cluster g have the same a_{0g} . We model the intercepts a_{0g} via a mixed-effects meta-regression and a level-3 random effect: a_{0g}=W_{g}\gamma+Z_{g}\xi+\zeta_{g}, (2.3) where W_{g} is a 1\times q matrix of fixed effects, \gamma is a q -vector of unknown coefficients, Z_{g} is a 1\times r random-effects design matrix, \xi\sim MVN(0,\Omega) is an r -vector of random effects, and \zeta=(\zeta_{1},\ldots,\zeta_{M})^{T}\sim MVN(0,\omega^{2}I_{M}) . The variance is \hbox{Var}(a_{0g})=Z_{g}\Omega Z_{g}^{T}+\omega^{2} . Thus, the unconditional covariance matrix for cluster g is V_{g}=\hbox{Cov}(T_{g})=(Z_{g}\Omega Z_{g}^{T}+\omega^{2})J_{K_{g}}+\Sigma_{g}=(Z_{g}\Omega Z_{g}^{T}+\omega^{2})J_{g}+\hbox{diag}(\tau^{2}+\sigma_{gi}^{2}), (2.4) where J_{K_{g}} is the K_{g}\times K_{g} matrix of 1’s. The full covariance matrix of the stacked vector T , denoted by V , is the block-diagonal matrix of the M matrices V_{g} . 3 Estimation As in ANOVA.we approach estimation by subdividing the total variation into within and between components and using moments. Thus, we work with Equations ( 2.2 ) and ( 2.3 ) separately. 3.1 Estimating 2nd-level parameters (within-cluster analysis) At level 2, conditionally on the values of (a_{01},\ldots,a_{0M}) , we can estimate \beta and \tau^{2} as in a standard meta-regression. To accommodate various choices of weights, we introduce A , a symmetric {\mathcal{K}}\times{\mathcal{K}} matrix of full rank. Then from Equation ( 2.2 ), \hbox{E}(X^{T}AT)=X^{T}AX\beta and \hat{\beta}_{A}=(X^{T}AX)^{-1}X^{T}AT (3.1) is an unbiased estimator of \beta with conditional covariance \hbox{Cov}(\hat{\beta}_{A}|a_{0})=(X^{T}AX)^{-1}X^{T}A\Sigma AX(X^{T}AX)^{-1}, (3.2) where \Sigma=\hbox{diag}(\Sigma_{g}) . The maximum-likelihood-based estimator corresponds to A=\Sigma^{-1} (i.e., to inverse-variance weights). For ML/IV, \hat{\beta}_{A} given by Equation ( 3.1 ) needs to be calculated simultaneously with \hat{\tau}^{2} . For any fixed A , however, \hat{\beta} and \hat{\tau}^{2} can be estimated separately. The residuals based on \hat{\beta}_{A} are R_{A}=T-X\hat{\beta}_{A}=T-X(X^{T}AX)^{-1}X^{T}AT=(I-H_{A})T , where H_{A}=X(X^{T}AX)^{-1}X^{T}A is a hat matrix, similar to that in least-squares regression. As projections, H_{A} and (I-H_{A}) are idempotent. That is, H_{A}^{2}=H_{A} . Also, H_{A}^{T}A=AH_{A} . Moment estimation of the variance component \tau^{2} is based on the generalized Q statistic, a quadratic form in the residuals: Q_{A}=R_{A}^{T}AR_{A}=T^{T}(I-H_{A}^{T})A(I-H_{A})T=T^{T}A(I-H_{A})T (3.3) (discussed by DerSimonian and Kacker [2007] ). Given unbiased estimators v_{gi}^{2}=\hat{\sigma}_{gi}^{2} and a nonrandom A matrix, and denoting the weight matrix by C=A(I-H_{A}) , Q_{A}=T^{T}CT and \hbox{E}(Q_{A})=\sum_{g}\sum_{i}c_{(g)ii}(\tau^{2}+\hbox{E}(v_{gi}^{2})) , where the weight c_{(g)ii} is applied to T_{gi}^{2} in Equation ( 3.3 ). Hence, a moment estimator of \tau^{2} is \hat{\tau}^{2}=\hbox{max}((\sum\sum c_{(g)ii})^{-1}(Q_{A}-\sum\sum c_{(g)ii}\hbox{E}(v_{gi}^{2})),0). (3.4) The distribution of Q_{A} is that of a quadratic form in normal random variables, and we can obtain a confidence interval for \tau^{2} from percentage points of an approximation to this distribution under the hypothesis of no heterogeneity. In the