A Goodness-of-Fit Test for Mixed-Effects Logistic Regression
Ariel Linden · 2026
Read A Goodness-of-Fit Test for Mixed-Effects Logistic Regression by Ariel Linden (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19694.txt.
Introduction
Linden extends a grouping-based Wald test for mixed-effects logistic regression from random-intercept models to models with random slopes, covering sparse cluster settings. Observations are grouped by predicted probabilities within clusters, the model gains pooled group markers, and their joint significance is tested with a Wald statistic. A data-driven rule picks the group count as the smaller of 10 and the smallest cluster size, keeping estimation feasible. Simulations across 24 null scenarios show the test holds its stated error rate.
Catalog record
| CREATOR | Ariel Linden |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22673 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.19694
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Goodness-of-Fit Test for Mixed-Effects Logistic Regression Ariel Linden Abstract Mixed-effects logistic regression is widely used for analyzing binary outcomes in hierarchical data, yet formal goodness-of-fit tests remain limited to random intercept models and do not address sparse cluster settings. We extend a grouping-based Wald test to mixed-effects logistic models with random slopes. The procedure groups observations by their predicted probabilities within clusters, adds pooled group indicators to the model, and tests their joint significance using a Wald statistic. To accommodate small clusters, we introduce a data-driven rule for selecting the number of groups, G = min ( 10 , n min ) G=\min(10,n_{\min}) , where n min n_{\min} is the smallest cluster size. This prevents estimation failure when clusters contain fewer than ten observations. Simulation studies across 24 null scenarios show that the test maintains nominal Type I error in three-level random slope models, including at smaller sample sizes than previously studied. The test demonstrates increasing power to detect fixed-effects misspecification: power against omitted nonlinearity increased from 0.07 to 1.00 across effect sizes, and power against omitted interactions reached 0.87. As expected, the test has no power to detect an omitted clustering level, reflecting its focus on residual patterns in predicted probabilities. In sparse balanced designs, fixing G = 10 G=10 led to complete test failure, whereas the data-driven rule performed reliably. The method is implemented in the community-contributed Stata program mlm_gof after fitting a mixed-effects logistic regression model. Keywords: goodness-of-fit; mixed-effects logistic regression; multilevel models; random slopes; simulation study 1 Introduction Multilevel data structures are common in health services, epidemiologic, and social science research. Patients are nested within doctors, hospitals within regions, and repeated measurements within individuals. When the outcome is binary (e.g., mortality, disease onset, or treatment success) the appropriate analytic tool is the mixed-effects logistic regression model, which accounts for clustering and partitions variation across levels. The terms mixed-effects , multilevel , and hierarchical are used interchangeably in the literature; here we use mixed-effects . Similarly, we use random slopes to refer to models in which regression coefficients vary across clusters. The use of these models has increased substantially in recent decades across medicine, public health, and the social sciences. 1 A fitted model is only useful to the extent that it adequately describes the data. Before drawing inferences about covariate effects or variance components, it is essential to assess model fit. For single-level logistic regression, several goodness-of-fit tests are well established. The Hosmer–Lemeshow test 2,3 groups observations by predicted probabilities and compares observed and expected frequencies. Lipsitz et al. 4 extended this approach by adding group indicator variables to the model and testing their joint significance via a Wald statistic, making the method adaptable to correlated data. Further extensions have addressed survey-weighted models, 5 complex sampling designs, 6 and multinomial outcomes. 7 In contrast, goodness-of-fit testing for mixed-effects logistic regression has been limited. Cool et al. 8 highlighted the absence of suitable methods for higher-level multilevel binary models. Sturdivant and Hosmer 9 proposed residual-based statistics but did not evaluate power. Perera et al. 10 developed the first formal test for two-level models by adapting grouping-based methods to clustered data. Fernando and Sooriyarachchi 11 extended this framework to three-level models using limited-information approaches. Although these contributions represent important advances, they are restricted to random intercept models and do not address the more general case in which both intercepts and slopes vary randomly across clusters, which is the appropriate specification whenever the effect of a covariate may differ across clusters. A second limitation of existing tests is the fixed use of ten groups in the grouping procedure, following Hosmer and Lemeshow. 2 In mixed-effects settings, clusters may contain fewer than ten observations, making this choice infeasible: empty cells arise, indicator variables become degenerate, and the augmented model cannot be estimated. Existing methods do not address this issue or provide guidance for sparse cluster settings. This paper makes two contributions. First, we extend the grouping-based Wald test framework 10,11 to mixed-effects logistic regression models with random slopes. Second, we introduce a data-driven rule for selecting the number of groups, G=\min(10,n_{\min}) , where n_{\min} is the minimum cluster size. Simulation results show that this rule is necessary to avoid test failure when clusters are small. We evaluate the proposed method using an extensive simulation study covering Type I error, power under three types of misspecification, and a direct comparison of the data-driven grouping rule with the conventional choice of G=10 . The method is implemented in the community-contributed Stata program mlm_gof . 12 The remainder of the paper is organized as follows. Section 2 describes the model, test procedure, and simulation design. Section 3 reports simulation results. Section 4 presents an applied example. Section 5 discusses results and Section 6 concludes. 2 Methods 2.1 The Mixed-Effects Logistic Regression Model with Random Slopes Let y_{ij} denote a binary outcome for observation i in cluster j ( i=1,\ldots,n_{j} ; j=1,\ldots,J ). A two-level mixed-effects logistic regression model with a random intercept and a random slope on covariate x_{2ij} is \operatorname{logit}(p_{ij})=\beta_{0}+\beta_{1}x_{1ij}+\beta_{2}x_{2ij}+u_{0j}+u_{1j}\,x_{2ij}, (1) where p_{ij}=\Pr(y_{ij}=1\mid\mathbf{u}_{j}) , \beta_{0},\beta_{1},\beta_{2} are fixed effects, and (u_{0j},u_{1j})^{\top}\sim N(\mathbf{0},\boldsymbol{\Omega}) are cluster-specific random effects. The random intercept u_{0j} captures between-cluster variation in baseline log-odds, while the random slope u_{1j} allows the effect of x_{2} to vary across clusters. Conditional predicted probabilities are obtained as \hat{p}_{ij}=\operatorname{logit}^{-1}\!\bigl(\hat{\beta}_{0}+\hat{\beta}_{1}x_{1ij}+\hat{\beta}_{2}x_{2ij}+\tilde{u}_{0j}+\tilde{u}_{1j}\,x_{2ij}\bigr), (2) where \tilde{u}_{0j} and \tilde{u}_{1j} are empirical Bayes estimates. A three-level extension adds random effects at an intermediate level (e.g., subjects within families). This formulation generalizes prior work, which considered random intercept models only. 10,11 2.2 Derivation of the Goodness-of-Fit Test Statistic The test follows the model-based approach of Lipsitz et al., 4 extended to clustered data by Perera et al. 10 for the two-level case and by Fernando and Sooriyarachchi 11 for the three-level case. The null hypothesis is that the fitted model adequately describes the data. Under the null, the predicted probabilities \hat{p}_{ij} should capture the observed outcomes without systematic residual structure. Let G denote the number of groups formed from the distribution of \hat{p}_{ij} within each cluster. Define I^{g}_{ij}=1 if observation i in cluster j belongs to group g ( g=2,\ldots,G ). The augmented model is \operatorname{logit}(p_{ij})=\beta_{0}+\beta_{1}x_{1ij}+\beta_{2}x_{2ij}+\gamma_{2}I^{2}_{ij}+\cdots+\gamma_{G}I^{G}_{ij}+u_{0j}+u_{1j}\,x_{2ij}. (3) The null hypothesis is H_{0}\colon\gamma_{2}=\cdots=\gamma_{G}=0 . The joint Wald statistic is W=\hat{\boldsymbol{\gamma}}^{\top}\bigl[\widehat{\operatorname{Var}}(\hat{\boldsymbol{\gamma}})\bigr]^{-1}\hat{\boldsymbol{\gamma}}, (4) which is approximately \chi^{2} with G-1 degrees of freedom. 4 Large values of W indicate lack of fit. 2.3 The Step-by-Step Procedure Implemented in mlm_gof Step 1. Fit the baseline model. Estimate the mixed-effects logistic model using maximum likelihood and obtain predicted probabilities \hat{p}_{ij} . Step 2. Select the number of groups. Set G=\min(10,n_{\min}) , where n_{\min} is the minimum number of observations in any level-2 cluster. Step 3. Group observations. Within each level-2 cluster, sort \hat{p}_{ij} and assign observations to G approximately equal-sized groups using the asymptotic ranking approach of Rosner et al., 13 which preserves the overall ranking across clusters while enabling within-cluster grouping. Step 4. Create indicators. For each observation, define G-1 binary indicator variables I^{g}_{ij}=1 if observation i in cluster j belongs to group g ( g=2,\ldots,G ), with group 1 as the reference. Indicators for the same group g across clusters are pooled into a single variable for the full dataset. 10 Step 5. Refit