Penalized KLIC Model Selection for the Generalized Method of Moments in Longitudinal Data with Time-Dependent Covariates

Mahmud Hasan*, Mathias Nthiani Muia, Mous-Abou Hamadou, Niloofar Ramezani · 2026

Read Penalized KLIC Model Selection for the Generalized Method of Moments in Longitudinal Data with Time-Dependent Covariates by Mahmud Hasan*, Mathias Nthiani Muia, Mous-Abou Hamadou, Niloofar Ramezani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04457.txt.

Introduction

Mahmud Hasan and three co-authors propose better rules for choosing among models fitted to data gathered over time. The same subjects are measured again and again, and the predictors themselves change with time. Estimation by the generalised method of moments can offer many valid conditions, inviting overly complex models.

An existing information criterion for this setting can favour that complexity as parameters and conditions multiply. The authors add two penalised versions. One charges for the product of parameters and moment conditions. The other uses a log-based penalty. Computer studies with yes-no outcomes test whether the new criteria balance fit and simplicity more reliably.

Catalog record

CREATORMahmud Hasan*, Mathias Nthiani Muia, Mous-Abou Hamadou, Niloofar Ramezani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.04457

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Penalized KLIC Model Selection for the Generalized Method of Moments in Longitudinal Data with Time-Dependent Covariates Mahmud Hasan*, Mathias Nthiani Muia, Mous-Abou Hamadou, Niloofar Ramezani Abstract Model selection plays an important role in longitudinal data analysis, especially when models are estimated using the generalized method of moments (GMM) in the presence of time-dependent covariates. In this setting, the number of valid moment conditions can grow quickly and may lead to over-parameterized models. The Kullback–Leibler Information Criterion (KLIC) has been proposed as a model-selection tool for this framework; however, the original KLIC criterion may favor overly complex models when the number of parameters or valid moment conditions increases. To address this limitation, this study proposes two penalized versions of KLIC that incorporate penalties based on both the number of model parameters and the number of valid moment conditions. The proposed criteria are referred to as the Moment–Parameter Product Penalty KLIC (MPPP–KLIC) and the Logarithmic Penalty KLIC (LP–KLIC). These criteria provide a theoretically motivated mechanism for balancing model fit and model complexity in GMM-based longitudinal models. Through an extensive simulation study involving both binary and continuous response settings, the proposed criteria are shown to improve the ability of KLIC to distinguish among competing models and to reduce the selection of over-parameterized models. The performance of the proposed methods is further illustrated using the Filipino Child Morbidity dataset, a longitudinal study of child health in the Philippines. The results show that the proposed penalized criteria provide stable and interpretable model rankings and consistently identify age as the most important predictor of child morbidity. Overall, the proposed penalized KLIC criteria offer practical and theoretically grounded tools for model selection in GMM-based longitudinal data analysis with time-dependent covariates. Keywords: model selection; penalized KLIC; Kullback–Leibler divergence; generalized method of moments; longitudinal data; time-dependent covariates. 1 Introduction Longitudinal data arise when repeated measurements are collected on the same subjects over time using the same or comparable instruments. Such data are common in public health, education, economics, and the social sciences because they allow researchers to examine temporal change and within-subject variability. Well-known examples include the National Health and Nutrition Examination Survey (NHANES), which has followed a panel of participants annually since 1999 to record repeated health and nutritional measurements, and the National Education Longitudinal Study (NELS), initiated in 1988, which tracked a cohort of students across multiple survey waves to evaluate educational outcomes and development. Unlike cross-sectional data, longitudinal observations violate the assumption of independent and identically distributed observations because repeated measurements from the same subject are inherently correlated. As a result, statistical methods used for longitudinal data must explicitly account for within-subject correlation in order to obtain valid inference. Two major modeling frameworks are commonly used in this context: marginal models and conditional models. Marginal models provide population-averaged interpretations by modeling the mean response and the within-subject correlation separately [ 40 , 42 , 25 ] , whereas conditional or subject-specific models explain correlation through individual-level heterogeneity using random effects [ 41 , 27 , 29 ] . In marginal models, the expected response at time t is modeled as a function of covariates observed at the same time point. Marginal models are preferred when population-average interpretations are desired or when predictions rely only on current covariates [ 25 , 37 ] . One of the most widely used estimation methods for marginal models with correlated responses is the generalized estimating equations (GEE) approach introduced by [ 35 ] . The GEE method uses estimating equations based on a working correlation structure and requires only specification of the first two moments of the response rather than the full likelihood. Although GEE estimators are consistent even when the working correlation structure is misspecified, efficiency may be substantially reduced when time-dependent covariates are present and the independence working correlation is imposed [ 37 , 26 , 25 ] . A common feature of longitudinal studies is the presence of time-dependent covariates (TDCs), whose values change over repeated measurements. Examples include age, body mass index, treatment exposure, and other evolving subject characteristics. The temporal behavior of these covariates has important implications for estimation and inference. Prior research has shown that when time-dependent covariates are present, the use of GEE with non-independent correlation structures may lead to inconsistent estimators unless strong assumptions are satisfied [ 37 , 26 ] . To address this issue, [ 34 ] classified time-dependent covariates into several types according to the feedback structure between the response and covariate processes. In particular, Type I, Type II, and Type III covariates generate different sets of valid estimating equations depending on the temporal relationships between covariates and residuals. When covariates are of Type I or Type II, additional valid moment conditions exist that are not exploited by independence GEE. The Generalized Method of Moments (GMM), originally developed by [ 28 ] , provides a natural framework for incorporating these additional valid moment conditions. Instead of relying on likelihood or quasi-likelihood functions, GMM estimation is based on moment conditions whose expectations equal zero. By utilizing all valid moment conditions implied by the temporal structure of the covariates, GMM can yield more efficient parameter estimates than GEE when time-dependent covariates of Type I or Type II are present, while remaining consistent for Type III covariates [ 34 ] . As a result, GMM has become an attractive alternative for estimating marginal models in longitudinal studies with complex covariate structures. Despite these advantages, model selection for GMM-based longitudinal models remains relatively underdeveloped. In likelihood-based modeling, several widely used goodness-of-fit and model-selection criteria exist, including the model deviance [ 22 ] , Akaike’s Information Criterion (AIC) [ 23 , 24 ] , the corrected AIC (AICc) for small samples [ 39 , 31 ] , and the Bayesian Information Criterion (BIC) [ 38 ] . For models estimated using GEE, a modification of AIC known as the quasi-likelihood information criterion (QIC) and its variant QICu have been proposed for model selection [ 36 , 30 ] . However, these criteria cannot be directly applied to GMM models because they rely on the likelihood or quasi-likelihood functions, whereas GMM estimation is based solely on moment conditions and the minimization of a quadratic form. To overcome this limitation, an information-theoretic approach based on the Kullback–Leibler divergence [ 33 ] can be adapted to the moment-based setting. The resulting measure, referred to as the Kullback–Leibler Information Criterion (KLIC), evaluates the discrepancy between the model-implied moment structure and the underlying data-generating mechanism [ 32 , 28 ] . Similar to AIC and QIC, the KLIC is a scalar measure used for comparing competing models, and the preferred model is the one with the smallest KLIC value [ 3 ] . However, an important limitation of the standard KLIC is that it does not include an explicit penalty for model complexity. Consequently, when multiple models are compared, the unpenalized KLIC often favors larger models, particularly in settings involving many time-dependent covariates and numerous valid moment conditions [ 3 ] . This behavior resembles that of the ordinary coefficient of determination ( R^{2} ), which increases automatically as additional predictors are added, regardless of their true contribution to the model. To address this issue, a penalized version of

Read the complete text