Missing Data Sensitivity Analyses for Alcohol Research Authors: Craig K. Enders, Lara A. Ray Published: 2026 ABSTRACT ABSTRACTBackgroundIn alcohol research studies, nonresponse is plausibly related to unobserved drinking scores. These so‐called missing not at random (MNAR) processes can reflect situations in which missingness is associated with the underlying level of drinking or changes in drinking over time. Examples include participants experiencing heavy drinking, increasing use or symptoms who may be less likely to report their behavior, and participants whose drinking improves substantially who disengage after achieving their goals or no longer perceiving a need for continued participation.Methods and ResultsCalls for missing data sensitivity analyses have become increasingly common. To implement this approach, researchers compare models that make different assumptions about the missing data process and examine the stability of substantive conclusions across these alternatives. Sensitivity analyses are particularly valuable because assumptions about missing data are inherently untestable. However, clear guidance on how to implement them in practice remains limited. To address these recommendations and advance the application of sophisticated missing data methods in alcohol research, this paper provides a practical, step‐by‐step account of how to conduct such analyses. We focus on selection and shared parameter models because they align with the intuitive idea that a person's unobserved drinking behavior may influence the likelihood of nonresponse, either through individual change trajectories or recent episodic shifts in drinking behavior. Annotated scripts and output are provided in supplemental materials to support implementation. Although missing data assumptions did not appear to have a substantial impact, they easily could under slightly different conditions.ConclusionsOur sensitivity analysis strategy is applicable to a broad range of alcohol research settings. However, data‐specific features can meaningfully alter certain steps. We describe how aspects of our clinical trial illustration may or may not generalize to other alcohol research contexts, and we propose several recommendations for reporting the results of a sensitivity analysis. Introduction Missing data are a pervasive challenge in a wide range of alcohol research settings. For example, attrition rates across pharmacotherapy trials, behavioral interventions, and web‐based alcohol studies routinely range from roughly 20% to over 50% (Anton et al. 2006; Fertig et al. 2012; Johnson et al. 2007; Mason et al. 2014; Murray et al. 2013). This prevalence has prompted methodological literature evaluating analytic approaches (Hallgren et al. 2016; Hallgren and Witkiewitz 2013; Witkiewitz et al. 2014) and calls for improving reporting and analytic practices in alcohol research (Grigsby and McLawhorn 2019; Harris et al. 2013; Witkiewitz et al. 2015). One recommendation for improvement is to conduct sensitivity analyses (Hallgren et al. 2016; Hallgren and Witkiewitz 2013; Witkiewitz et al. 2015). This process involves comparing models that make different assumptions about the missing data process and examining the stability of substantive conclusions across these alternatives. Currently, clear guidance on how to implement sensitivity analyses in alcohol research remains limited. The purpose of this paper is to provide a step‐by‐step framework that researchers can readily adapt to their own data. Missing data sensitivity analyses require researchers to consider alternative assumptions about the process that produced the missing values. These assumptions are typically framed in terms of the missing data mechanisms described in various studies by Rubin and colleagues (Little and Rubin 2002; Mealli and Rubin 2016; Rubin 1976). Rubin's classification system introduces a missing data indicator that encodes whether a score is observed or missing (e.g., M = 1 if a drinking measure is missing and M = 0 if it is observed). This indicator is effectively a proxy for the myriad real‐world reasons underlying the missing values. Missing data mechanisms describe different ways in which nonresponse relates to the data: missingness may be unrelated to the data (missing completely at random; MCAR), related to the observed data only (missing at random; MAR), or to the missing values themselves (missing not at random; MNAR). The literature recommends that alcohol researchers use theoretically justified analytic approaches such as maximum likelihood, multiple imputation, and Bayesian Markov chain Monte Carlo (MCMC) estimation (Hallgren et al. 2016; Hallgren and Witkiewitz 2013; Harris et al. 2013; Witkiewitz et al. 2014; Witkiewitz et al. 2015). MAR is the standard working assumption for these methods. Although the term “random” appears in its name, MAR describes a systematic process in which missingness may depend on observed data; for two people with the same observed data profile, missingness is random. To emphasize this dependence on the observed data, Graham (2009) proposed the term conditionally missing at random (CMAR), which we adopt for clarity. Crucially, CMAR requires that missingness is unrelated to the unobserved drinking values themselves. Although this assumption cannot be verified from the observed data, many real‐world causes of missing data are likely unrelated to unobserved drinking scores. Such reasons include practical barriers (e.g., transportation, childcare, cost, scheduling conflicts, limited availability, or competing demands), life changes (e.g., relocating or starting a new job), and research‐specific factors (e.g., study fatigue or the burden of repeated assessments). In alcohol research studies, nonresponse is plausibly related to unobserved drinking scores. MNAR processes can reflect situations in which missingness is associated with the underlying level of drinking or changes in drinking over time. Examples include participants experiencing heavy drinking, increasing use, or withdrawal symptoms who may be less likely to report their behavior, as well as participants whose drinking improves substantially who disengage from assessments after achieving their goals or no longer perceiving a need for continued participation. Models for MNAR mechanisms have a long history in the biostatistics literature. These approaches introduce a secondary model for the nonresponse process. When correctly specified, this additional component can reduce or eliminate nonresponse bias when estimated alongside the focal model. Much of this work has focused on three frameworks. Selection models treat missingness as an outcome that depends on drinking behavior and measured covariates (Diggle and Kenward 1994; Kenward 1998; Molenberghs et al. 1997). Shared parameter models also treat missingness as an outcome but link it to changes in drinking scores over time through random effects (Albert and Follmann 2009; Follmann and Wu 1995; Vonesh et al. 2006; Wu and Carroll 1988). Finally, pattern mixture models treat the missing data indicator as a grouping variable, allowing responders and nonresponders to have different parameter values (Demirtas and Schafer 2003; Hedeker and Gibbons 1997; Pauler et al. 2003). We focus on selection and shared parameter models because they align with the intuitive idea that a person's unobserved drinking behavior may influence the likelihood of nonresponse, either through individual change trajectories or recent episodic shifts in drinking behavior (random‐coefficient‐dependent and outcome‐dependent processes; Little 1995). In contrast, pattern mixture models reverse the role of nonresponse, treating it as a grouping variable. A typical application involves classifying individuals as study completers or dropouts, then allowing these subgroups to have different drinking levels or treatment effects. A key challenge is that some subgroup parameters are not estimable from the observed data. For instance, it is not possible to estimate the endpoint mean or the treatment‐group mean difference among individuals who quit the study and have no drinking data. Consequently, pattern mixture models require researchers to supply plausible values for these parameters or impose identification constraints that make the model estimable. This process is arguably more challenging and less intuitive than conceptualizing nonresponse as an outcome predicted by drinking behavior. We refer interested readers to Hedeker and Gibbons (1997) and Enders (2022) for illustrations of pattern mixture models using data from a clinical trial. Calls for missing data sensitivity analyses have become increasingly common in alcohol research (Hallgren and Witkiewitz 2013; Hallgren et al. 2016; Witkiewitz et al. 2015); yet clear guidance on how to implement them in practice remains limited. To address these recommendations and advance the use of sophisticated missing data methods in alcohol research, this paper provides a practical, step‐by‐step account of how to conduct such analyses. Key elements include CMAR‐based analyses using theoretically justified methods; identifying and understanding auxiliary variables; identifying the predominant missing data pattern and coding the missingness indicator; modeling time‐related changes in missingness as a precursor to MNAR analyses; fitting shared parameter and selection models for trajectory‐ and outcome‐dependent processes; evaluating whether assumptions affect substantive conclusions; and considering strategies for reporting and interpreting results. To facilitate adoption and replication, the online materials at https://osf.io/tp3k6/ provide annotated analysis scripts and output. Materials and Methods We use data from an alcohol cessation trial to illustrate missing data sensitivity analyses (Ray et al. 2021). For full details on design, protocol, and outcomes, see the original publication. Briefly, the researchers conducted a 12‐week, Phase II, double‐blind randomized trial in a sample of N = 165 daily smokers who also engaged in heavy drinking. Subjects were assigned to receive either a combined medication regimen of varenicline with naltrexone or varenicline with a matched placebo. Our illustrations focus on the drinks per drinking day (DPDD) outcome measured at baseline and at Weeks 4, 8, 12, and 16. We chose this data set for our illustration because it is representative of clinical trial data sets on which we have conducted similar sensitivity analyses (Nieto et al. 2022; Ray et al. 2021, 2025), including comparable missing data patterns, dropout rates, and sample sizes. Importantly, the analytic framework we describe applies broadly to other alcohol research contexts, including behavioral treatment trials and prevention studies. We highlight any differences that arise in these other applications where relevant. Distributional assumptions are critical for MNAR models, especially selection models, where they provide key information for stable estimation. Witkiewitz et al. (2015) discuss normalizing transformations as a way to address skewed drinking outcomes. Because drinking scores were substantially positively skewed, we consider both the raw outcome and a log‐transformed version of the outcome, log(DPDD +1), in the sensitivity analyses. The log transformation is appealing in this context because it reduces the influence of extreme observations in the DPDD distribution and yields coefficients whose exponentiated values convert to proportional changes in drinking (e.g., a coefficient of −0.20 corresponds to approximately an 18% decrease in DPDD +1 values). Other transformations may be appropriate depending on the context and shape of the drinking distribution. We used linear mixed‐effects (multilevel) models because this framework is ubiquitous in alcohol research applications involving longitudinal data. Figure 1 shows DPDD means by experimental condition and time (assuming CMAR). On both metrics, the means follow a nonlinear pattern in which the two groups were equivalent at baseline (t = 0), exhibited differential change at the first follow‐up (t = 1), and followed roughly parallel trajectories thereafter. We employed a piecewise specification that represents this nonlinear pattern as two linear components that meet at a “knot” located at the first follow‐up. The level‐1 (within‐person) model is (1) DPDDti=β0i+β1iPREKNOTti+β2iPOSTKNOTti+εti where DPDDti is the drinks‐per‐drinking‐day score at occasion t for individual i, and β0i is the predicted baseline score. Coding PREKNOT as (0, 1, 1, 1, 1) defines β1i as an individual's linear change rate between baseline and the first follow‐up, and coding POSTKNOT as (0, 0, 1, 2, 3) defines β2i as the linear change after the knot. Finally, εti is a normally distributed residual with constant variation over time and people. Medication means by time. Group means by medication arm and time. The means assume a CMAR process in which missingness depends on the observed data but not on the unobserved drinking levels. The level‐2 (person‐level) models describe how the individual random intercepts and slopes differ as a function of treatment assignment (MED = 0 if varenicline plus placebo and MED = 1 if varenicline plus naltrexone). β0i=β0+β3MEDi+u0i (2) β1i=β1+β4MEDi+u1i β2i=β2+β5MEDi+u2i The β0, β1, and β2 coefficients represent the expected baseline score and linear change components for the MED = 0 condition (varenicline with placebo), and β3, β4, and β5 correspond to the mean differences for the MED = 1 group (varenicline with naltrexone). Although randomization justifies omitting the medication difference from the intercept equation, we include it here to maintain symmetry across the three growth parameters. Finally, the u terms are person‐level residuals that capture unexplained variation in the random coefficients. These random effects are assumed to follow a multivariate normal distribution. A key challenge in sensitivity analyses is determining whether parameter estimates change meaningfully across modeling assumptions. To gauge these differences, we computed standardized mean difference effect sizes at the 1‐ and 4‐month follow‐ups. Model‐predicted means were centered at the baseline mean and divided by the baseline standard deviation, defined as the square root of the total random‐intercept variance plus the within‐cluster residual variance. We used the total standard deviation because it is more stable across models; specifications may redistribute within‐ and between‐cluster variation, but the total should remain relatively consistent. Standardizing with only the within‐cluster standard deviation would yield larger but potentially less stable effect sizes. Yet another alternative is to use an invariant value like the within‐cluster standard deviation from the CMAR analysis. The goal is simply to express assumption‐related changes on a practical scale, and substantive conclusions are unlikely to depend strongly on the choice of standardization. Because the analysis model shapes the sensitivity analysis, it is useful to consider how this framework applies in other alcohol research contexts. We use a linear mixed‐effects model with a piecewise trajectory, though other settings may use polynomial or alternative within‐person specifications. Regardless of context, the focal model must include random effects, which provide the latent link between drinking and missingness in shared parameter models; without them, the model loses its key feature. Selection models do not require random effects and can be paired with a wider range of models. Both frameworks also accommodate other outcome types (e.g., binary outcomes such as abstinence or relapse). Finally, so that interested readers can get some practical experience running these models, we simulated an artificial data set that mimics the main features of our analysis example. To simplify the analysis, we used a linear growth trend and MNAR dropout. Apart from eliminating all terms involving β2i, the artificial practice data accommodate the same models we illustrate here. The online materials at https://osf.io/tp3k6/ provide a link to a GitHub repository with the analysis scripts and artificial data. Results Next, we performed a sensitivity analysis to evaluate the robustness of the treatment effect estimates under different assumptions about the missing data process. This section describes a step‐by‐step procedure for conducting these analyses. All models were fit using Bayesian MCMC estimation in the Blimp software program (Enders and Keller 2026; Keller and Enders 2021). The software is freely available at www.appliedmissingdata.com/blimp, and online materials at https://osf.io/tp3k6/ include detailed installation and quick start guides. CMAR Analysis In line with recommendations to use theoretically justified missing data methods (Witkiewitz et al. 2015), we first fit models that invoke the CMAR assumption. The top panel of Table 1 shows the effect size estimates, parameter standard deviations (“Bayesian standard errors”), and p values from the CMAR analyses. The negative point estimates indicate that the varenicline plus naltrexone (MED = 1) group had lower drinking scores than the varenicline plus placebo group. The outcome scale substantially influenced the results; the log‐transformed model produced larger effect sizes at both follow‐ups and trajectories that continued to decline after the knot. These larger effects reflect compression of the upper tail of the positively skewed drinking distribution. Because extreme values inflate variability more than means, this compression reduces the total standard deviation more than the between‐group mean difference, yielding larger standardized effects. TABLE 1Standardized effect sizes with a conditionally missing at random assumption.Outcome1‐month SMD4‐month SMD RM2 IterationsEst.SD p Est.SD p Model 1: CMARRaw DPDD−0.140.090.12−0.150.110.16NA20,000Log DPDD−0.250.150.10−0.320.190.09NA20,000Model 2: CMAR + auxiliary variablesRaw DPDD−0.120.090.19−0.140.100.19NA20,000Log DPDD−0.220.150.15−0.310.190.10NA20,000Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. Standardized effect sizes with a conditionally missing at random assumption. Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. Figure 2 shows the predicted means, standardized by subtracting the baseline mean and dividing by the baseline standard deviation. In Panel A, raw drinking levels decrease sharply from baseline to the first follow‐up and then remain relatively stable across later follow‐ups. In Panel B, log‐transformed drinking drops sharply from baseline to 1 month and then continues to decline more noticeably across later follow‐ups. The CMAR assumption serves as a standard working assumption for theoretically justified analytic approaches, and these trajectories provide a baseline for exploring the impact of alternative assumptions about missingness. Model‐predicted means from the CMAR analysis. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume a CMAR process in which missingness depends on the observed data but not on the unobserved drinking levels. Auxiliary Variables Methodological work often recommends including auxiliary variables that would not have been part of the focal analysis had the data been complete (Collins et al. 2001; Graham 2009; Witkiewitz et al. 2015). This inclusive strategy makes the CMAR assumption more defensible by expanding the observed data. The benefit depends on three conditions: whether the variable predicts missingness, correlates with residuals in the growth model, and is observed when drinking is missing. Reducing nonresponse bias requires all three, whereas improving power requires only the latter two (Collins et al. 2001; Enders 2022). In practice, useful auxiliary variables are difficult to identify because the focal model already includes the most informative predictors—earlier drinking scores. We are therefore less enthusiastic about their incremental value in longitudinal alcohol research. Researchers often compare complete and incomplete cases on baseline measures to assess whether attrition relates to observed variables. Differences do not imply bias or that a variable will function well as an auxiliary variable. What matters is whether a candidate correlates with the outcome's residuals, indicating added information outside the model. Identifying such residual correlations in multilevel models is challenging because auxiliary variable methods were developed for single‐level analyses (Graham 2009, 2003; Raykov and West 2015; Raykov and Marcoulides 2014). Because the payoff from a more detailed process is often limited, we suggest a pragmatic approach that identifies variables conceptually or statistically related to drinking. These are often baseline measures, as repeated measures are typically missing when the outcome is missing. To illustrate an inclusive strategy, we augmented the CMAR analysis with two person‐level auxiliary variables: biological sex and the Fagerström Test for Nicotine Dependence (FTND; Heatherton et al. 1991). Auxiliary variables should be incorporated without altering the definitions of the focal model parameters; adding them as covariates, for example, would redefine the fixed slopes as conditional effects adjusted for the auxiliary variables. Entering auxiliary variables as additional downstream outcomes preserves interpretations. In multilevel models, associations can be specified only among variables at the same level, so baseline auxiliary variables can relate to between‐person components but not directly to repeated measures. Accordingly, we regress the auxiliary variables on the random intercepts and slopes, allowing them to influence repeated measures indirectly through the random effects. The bottom panel of Table 1 shows the results from the auxiliary variable analyses. The models with and without the auxiliary variables were virtually indistinguishable. This pattern is not surprising and is consistent with our experience. Auxiliary variables may slightly sharpen predictions of the missing values, but the impact is often modest and not readily noticeable. This occurs because the focal analysis already contains the most salient sources of information about the missing drinking scores, namely the earlier measures of drinking. As an aside, Schafer and Graham (2002) recommended collecting auxiliary measures that capture missingness processes before they occur. In the context of alcohol research, a brief questionnaire administered at each occasion could assess barriers to continued participation, including transportation difficulties, scheduling conflicts, and competing demands, as well as motivational factors such as perceived treatment helpfulness and burden of participation. When collected prospectively, such measures can serve as auxiliary variables that strengthen the plausibility of the CMAR assumption and potentially inform the specification of MNAR models. Their recommendation is an intriguing possibility that could improve the rigor of missing data analyses. Preparing for MNAR Modeling: Coding the Missing Data Indicator As noted earlier, shared parameter and selection models pair the growth model with a regression for the binary missingness indicator. Choosing how to code this indicator is critical because it determines how missingness probabilities are interpreted and guides model specification. The optimal choice depends on the predominant missing data pattern, which encodes the location of missing values rather than the process that generates them. Missing data patterns are typically classified as dropout, intermittent, or some combination of the two. Dropout (also called monotone) patterns arise when participants discontinue entirely, whereas intermittent patterns reflect sporadic missing observations. Many applications involve a mixture of the two. In our clinical trial illustration, the missing data patterns reflect only attrition. In other settings, intermittent missingness or a mixture of patterns may be more common. Regardless of context, identifying the predominant pattern has direct implications for how MNAR models are specified. The upper panel of Table 2 shows the five missing data patterns from the clinical trial data. Approximately 73% of participants had complete data, with the remainder distributed across four groups. The location of the NAs within each column reflects permanent attrition. The largest group (about 14%) dropped out at the first follow‐up. These patterns require a binary indicator for permanent disengagement: M = 0 prior to dropout, M = 1 at dropout, and missing thereafter (rows after dropout can be omitted). The probabilities from this coding scheme reflect the conditional probability that a participant disengages from the trial at a given occasion, given that they were still participating at the previous occasion. This definition aligns with hazard probabilities in discrete‐time survival models (Masyn 2014; Singer and Willett 2003). TABLE 2Three types of missing data patterns and indicator coding schemes.Missingness patternsIndicator (M) coding1234512345MonthPermanent dropout0OOOOO000001OOOONA000012OOONANA0001NA3OONANANA001NANA4ONANANANA01NANANA% sample7316614MonthIntermittent missingness0ONAOONA010011OOONAO000102OONAOO001003NAONANAO101104ONAOONA01001% sample3412231020MonthMix of intermittent missingness and dropout0OONAOO00?001NAOOONA?00012ONAONANA0?01NA3OONANANA001NANA4ONANANANA01NANANA% sample405102025 Note: Each missing data pattern column reflects a subgroup with a particular configuration of observed (O) and missing (NA) values. The indicator columns show how to convert these patterns into binary codes. Three types of missing data patterns and indicator coding schemes. Note: Each missing data pattern column reflects a subgroup with a particular configuration of observed (O) and missing (NA) values. The indicator columns show how to convert these patterns into binary codes. The middle panel of Table 2 shows hypothetical missing data patterns from an alcohol study with sporadic missing values. The sample is again distributed across five subgroups, each characterized by intermittent missingness. These patterns require a binary indicator that codes whether a value is missing at a given occasion. The standard coding scheme for this application defines a binary indicator M that takes the value 1 when a value is missing and 0 otherwise. The corresponding predicted probabilities simply reflect the likelihood that a drinking outcome is missing at a given occasion. The bottom panel of Table 2 shows hypothetical missing data patterns from a study with a mixture of sporadic missing values and attrition. Dropout is coded the same as the top panel. The question marks denoting intermittent missing values can be handled in different ways. One option is to treat sporadic missingness as qualitatively distinct by creating a multicategorical variable in which 2 indicates an intermittent missing value (Albert et al. 2002; Gad 2011; Xie and Qian 2012). In practice, this strategy is challenging to implement because it requires sufficient instances of both patterns. Two other options are to code intermittent missing values as 0, effectively invoking CMAR assumptions in which missing values follow the observed data, or to code intermittent missing values as 1, such that any type of missing value arises from the same MNAR process. As a practical matter, the Blimp software includes built‐in functionality for creating different types of missing data indicators, so no preprocessing is required. Modeling Time‐Related Changes in Missingness Mitigating nonresponse bias with an MNAR model requires correctly specifying the missingness process, meaning that the regression equation for the binary indicator should include the appropriate predictors and functional forms. Although it is impossible to verify that the missingness model is correct, a reasonable starting point is a specification that reproduces the observed missing data rates over time. Modeling time‐related changes in missingness strengthens the CMAR component by conditioning on a salient observed determinant of missingness. This provides a baseline from which MNAR adjustments can be introduced. Incorporating temporal trends is also consistent with MNAR latent curve models (Enders 2011; Feldman and Rabe‐Hesketh 2012; Muthén et al. 2011; Xu and Blozis 2011), which naturally accommodate temporal changes in missingness via occasion‐specific intercepts. Figure 3A displays the progression of the observed dropout rates over time within treatment condition. As a reminder, these are hazard probabilities that represent the conditional probability that a participant disengages from the trial at a given occasion, given that they were still in the study at the previous occasion. Attrition is structurally absent at the first occasion, rises sharply at the first follow‐up, and then declines thereafter, albeit in an irregular fashion. The varenicline plus naltrexone group exhibited substantially higher disengagement at the first follow‐up, followed by a steady decrease across subsequent occasions. In contrast, the varenicline plus placebo condition showed a more erratic pattern, with dropout probabilities increasing and decreasing across time. Dropout by group and time. Panel A shows the observed dropout probabilities (hazard rates) by medication group and time. Dummy coding measurement occasions exactly reproduces these probabilities. Panel B shows the predicted dropout rates from a model with group‐specific quadratic trends, which misrepresent the observed probabilities. We begin with a flexible specification that is broad enough to capture a wide range of settings in which missing data rates change over time and differ by intervention or subgroup. Such applications are common in both pharmacological and behavioral trials, as well as in studies where missingness varies across observed covariates. The general strategy is to represent time using a set of dummy codes, include a grouping variable to capture overall differences in missingness rates, and add interactions between the group indicator and the time dummies to allow these trajectories to differ across groups. More formally, the missingness model regression equation is (3) Mti*=λ0+λ1MEDi+∑t=1T−1λt0Oti+λt1OtiMEDi+ui+zti where Oti is an occasion‐specific dummy code that contrasts occasion t versus baseline. We adopt a probit missingness model where Mti* is a normally distributed latent response variable that reflects individual i's continuous propensity for dropout at occasion t. The model incorporates a fixed threshold parameter (typically set to zero for identification) that divides each person's latent distribution into two regions, the areas of which correspond to their predicted probabilities of dropout versus non‐dropout, and zti is a standard normal residual term. Turning to the coefficients, λ0 determines the baseline missingness rate for MED = 0 at TIME = 0, and λ1 captures the mean difference between groups at baseline. Each λt0 represents the time‐specific change in missingness for MED = 0 at occasion t, and each λt1 represents how that change differs for MED = 1. Finally, ui is a random intercept that allows each person to have a unique propensity for missing data across the study period (i.e., more instances of Mti = 1). To reiterate, Equation (3) provides a flexible specification that accommodates a broad range of applications. However, it requires simplification when applied to dropout patterns like those in our example because some of its terms are not estimable. First, the random intercept term that defines person‐level missingness propensities (ui) must be eliminated because dropout indicators lack the within‐person repetition needed to estimate this variation (i.e., each person's indicator vector contains at most a single instance of Mti = 1). This random effect is only applicable to intermittent missing data patterns. Second, by definition, dropout cannot occur at baseline, resulting in constant values of 0 for the indicator at this occasion. Because the intercept encodes the predicted baseline probability, λ0 must be fixed at a value that induces a predicted probability very close to 0 (a probability of exactly 0 would require the intercept to approach negative infinity). Setting λ0 = −3 is typically effective, as that corresponds to a predicted probability of approximately 0.0014. When specifying temporal changes in the missingness model, researchers must balance parsimony against fidelity to the observed data. Because MNAR models are only weakly identified from the observed data, adding numerous unnecessary parameters to the missingness model can destabilize an already delicate part of the analysis. Dummy coding is attractive because it can accommodate irregular patterns, but this strategy increases the parameter count and can yield unstable or inestimable coefficients when late‐occasion dropout is rare. In some situations, polynomial functions of time may adequately reproduce the observed dropout rates. To illustrate, Figure 3B shows the predicted dropout rates from a probit model with group‐specific quadratic time effects. Although the polynomial model fails to reproduce the missingness patterns in our example, it may be viable in other applications. Simple graphical diagnostics like these are a valuable tool for evaluating alternative representations of time (Singer and Willett 2003; Ch. 11–12). The annotated scripts and outputs in the online materials (https://osf.io/tp3k6/) produce graphical displays like those in Figure 3. Model Checking: MCMC Diagnostics As mentioned previously, the validity of an MNAR model's estimates hinges on a correctly specified missingness model. We present a range of MNAR specifications that differ in complexity (and thus computational difficulty). Some of these models are weakly supported by the observed data and therefore rely heavily on distributional assumptions for estimation. Because MNAR models capture different sources of association between the outcome and missingness, there is no guarantee that all specifications will be supported by the data. In fact, several of the models we fit exhibited estimation difficulties. This is the norm, and there is no reason to believe that our application is unique in this regard. Although such models may still produce parameter estimates, the results may not be reliable and should be interpreted with caution or, in some cases, discarded. Although it is not possible to verify whether the model is correctly specified, MCMC diagnostics usually provide clear signals when a model lacks support from the data. The potential scale reduction factor (PSRF; Gelman and Rubin 1992) diagnostic evaluates whether separate MCMC chains have run long enough to yield stable parameter estimates with similar point estimates and spread. This diagnostic is useful for determining whether MCMC has converged (i.e., whether the warm‐up or burn‐in period is sufficiently long). For the PSRF, smaller values are better, with acceptable per‐parameter PSRFs below 1.05 (Asparouhov and Muthén 2010) or 1.10 (Gelman et al. 2014). If the highest (worst) PSRF across all parameters is greater than 1.05 at the end of the burn‐in period, then the number of warm‐up iterations should be increased. The effective sample size (ESS; Gelman et al. 2014) captures the extent to which autocorrelation across iterations reduces the amount of independent information in the parameter summaries. Here, “sample size” refers to the number of MCMC samples or iterations rather than the number of observations. This diagnostic is useful for determining whether the number of MCMC iterations is sufficient to summarize the parameter estimates. For the ESS, larger values are better, with acceptable per‐parameter ESSs exceeding 100 (Gelman et al. 2014). If the lowest (worst) ESS across all parameters is less than 100, the number of iterations after the warm‐up period should be increased. The need for very large numbers of MCMC iterations to achieve adequate PSRF and ESS values can indicate weak support from the data. What constitutes a large number of iterations is somewhat subjective and varies by model. In our analyses, we began with 20,000 iterations and increased this number as needed. In our experience, models that require 100,000–200,000 iterations to achieve adequate diagnostics warrant scrutiny, as this often signals that MCMC is struggling. For this reason, we stopped increasing the number of iterations at 250,000 and judged a model invalid if it had not produced adequate diagnostic indices. The missingness model's pseudo‐R 2 is often the clearest indicator of model quality. Values around 0.70 or higher suggest that the predictors nearly perfectly classify dropouts from completers—a condition known as near‐separation that indicates limited empirical support (Enders 2022). Trajectory‐Dependent Missingness: Shared Parameter Model Having established time‐ and treatment‐related effects on nonresponse, we next introduce MNAR processes where missingness depends on the unobserved drinking scores. We begin with the shared‐parameter model (Albert and Follmann 2009; Wu and Carroll 1988), where unobserved drinking relates to missingness via individual growth trajectories (random‐coefficient‐dependent missingness; Little 1995). For example, participants with steep declining trajectories that reflect rapid improvement may have a greater chance of missing data because they have achieved their treatment goals or are functioning well and no longer prioritize study participation. Conversely, flatter or worsening trajectories may signal that treatment is ineffective, leading some individuals to disengage from the study. The model derives its name from the fact that the random intercepts and slopes (the β0i, β1i, and β2i terms from Equation 1) appear on the right‐hand side of both equations—they are shared predictors that induce a correlation between nonresponse and the drinking scores. The specification of the shared parameter model depends on the missing data pattern and the coding of the missingness indicator. To begin, consider an application with intermittent missing data. Assuming there are missing values at every occasion, the random intercepts and slopes are free to influence nonresponse across the entire study span. Thus, the random intercepts and slopes simply enter Equation (3) as additional predictors. The model specification is slightly more involved for dropout patterns like those in the illustrative data. Because attrition can occur only at follow‐up visits, the influence of the trajectory components on missingness should be restricted to post‐baseline measurements. We accomplish this by including interactions between the random coefficients and the PREKNOT dummy variable from the focal model, coded as (0, 1, 1, 1, 1). These interactions act as switches that activate the influence of trajectory components at occasions with missing data. Applications with incomplete baseline data would not require these product terms. Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1β0iPREKNOTti+λ2β1iPREKNOTti+zti (4) Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1β0iPREKNOTti+λ2β1iPREKNOTti+λ3β2iPREKNOTti+zti The λ1, λ2 and λ3 terms are key because they induce an MNAR mechanism in which attrition is linked to unobserved drinking scores via the random intercepts and slopes. Practical experience suggests that building models in a sequence of increasing complexity is often useful. Accordingly, the first model included the random intercepts (latent baseline drinking) and pre‐knot change rates, and the second added linear change after the knot. The rationale for separating post‐knot change rates stems from the fact that this source of variation was very small relative to the initial change rates. Small amounts of variation make it difficult to link this feature to nonresponse. A potentially important feature of the previous models is that the effects of the random intercepts and slopes are unidirectional. That is, participants with steeply declining trajectories that reflect rapid improvement may have a greater chance of disengaging, or participants with flatter or worsening trajectories may have a greater chance. The linear model does not allow both possibilities simultaneously. The same is true for the random intercepts. Specifying bidirectional influences for the trajectory components requires polynomial effects. The quadratic model below includes the squared random intercepts and slopes as predictors, thereby allowing bidirectional influences. Although this model may appear more flexible—it allows both positive and negative outcomes to predict missingness—estimating nonlinear effects for latent variables is computationally challenging and generally requires more data. We anticipate that this model will be prohibitively complex in many contexts. (5) Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1β0iPREKNOTti+λ2β0i2PREKNOTti+λ3β1iPREKNOTti+λ4β1i2PREKNOTti+zti Table 3 shows the effect size estimates, parameter standard deviations (“Bayesian standard errors”), and frequentist p values for the six shared‐parameter models. The rightmost columns show the pseudo‐R 2 values and number of iterations for each model. The model with random intercepts and pre‐knot change predicting attrition (Model 3 in the table) converged rapidly and exhibited adequate diagnostics (all PSRF values < 1.05 and all ESS values > 100 after 20,000 iterations). The missingness model with post‐knot change rates as a predictor (Model 4) was substantially more difficult to estimate, especially for the raw drinking outcome, which required 250,000 iterations to achieve adequate diagnostics. This model also produced a very large pseudo‐R 2 of 0.86, strongly indicating that it is invalid. In contrast, the log‐transformed model achieved adequate diagnostics in fewer iterations and yielded a reasonable R 2. Finally, the bidirectional model (Model 5) did not converge or produce a valid solution after 250,000 iterations when applied to the raw drinking scores, but it did when applied to the log‐transformed scores. We attribute these estimation difficulties to the presence of outliers in the raw data. This point reinforces our earlier observation that data‐specific features affect the estimability of these models. TABLE 3Standardized effect sizes from MNAR shared parameter models.Outcome1‐month SMD4‐month SMD RM2 IterationsEst.SD p Est.SD p Model 3: Random intercepts and pre‐knot slopes predicting attritionRaw DPDD−0.140.090.11−0.150.10.150.4420,000Log DPDD−0.260.150.09−0.340.190.070.4320,000Model 4: All random intercepts and slopes predicting attritionRaw DPDD a −0.170.090.05−0.080.110.480.86250,000Log DPDD−0.270.150.07−0.250.210.220.5250,000Model 5: Random intercepts, pre‐knot slopes, and their squares predicting attritionRaw DPDD a −0.170.100.08−0.190.120.110.67250,000Log DPDD−0.200.220.35−0.320.260.220.62100,000Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. a Solution judged invalid. Standardized effect sizes from MNAR shared parameter models. Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. Solution judged invalid. Model 3 produced standardized mean difference effect sizes that were virtually indistinguishable from the CMAR results; Figure 4 shows the predicted means in standardized form. Model 4 showed a different pattern. Although the raw score model did not yield a valid solution, its predicted trajectories suggest a sharp reversion to higher drinking levels (Figure 5A). For the log‐transformed outcome, the 1‐month effect size was comparable to CMAR, but the trajectories remained flat rather than continuing to decline (Figure 5B). This represents the largest discrepancy among the valid shared parameter models. Model 5 also failed to converge for the raw outcome, but its trajectories were similar to the CMAR results (Figure 6A). For the log‐transformed outcome, the 1‐month effect size was slightly smaller than the CMAR estimate, but the 4‐month effect size was comparable (Figure 6B). Overall, the Model 5 trajectories did not differ meaningfully from the CMAR results. Finally, it is worth noting that although the significance test conclusions were consistent with the CMAR analyses, p values from the shared parameter models varied considerably (e.g., from 0.07 to 0.35). Although our conclusions about statistical significance were stable, this variability underscores that conclusions could easily change under slightly different conditions. Model‐predicted means from MNAR shared parameter Model 3. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume an MNAR process in which the random intercepts and pre‐knot change rates predict attrition. Model‐predicted means from MNAR shared parameter Model 4. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume an MNAR process in which the random intercepts, pre‐knot change rates, and post‐knot change rates predict attrition. Model‐predicted means from MNAR shared parameter Model 5. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume an MNAR process in which the random intercepts, pre‐knot change rates, and their squares predict attrition. Outcome‐Dependent Missingness: Diggle–Kenward Selection Model Next, we illustrate a selection model (Diggle and Kenward 1994) for an outcome‐dependent MNAR process (Little 1995) where missingness relates to one's current (unobserved) and previous (observed) drinking scores. For instance, a participant who suddenly experiences heavier drinking between assessments may be less likely to complete the next assessment. Alternatively, a participant who experiences a sudden improvement in drinking may also be less likely to complete subsequent assessments after achieving their treatment goals. Fitting the Diggle–Kenward selection requires a lagged dependent variable that shifts each person's drinking scores down by row. That is, each row of data pairs a person's drinking at occasion t with their previous score from occasion t – 1. The Blimp software provides built‐in facilities for creating and imputing lagged variables, so no preprocessing is required. The specification of the selection model again depends on the missing data pattern and the coding of the missingness indicator. To begin, consider an application with intermittent missing data. Assuming there are missing values at every occasion, the outcome and its lag are free to influence nonresponse across the entire study span. In that case, the pair of drinking scores simply enter Equation (3) as additional predictors. The model specification is slightly more involved for dropout patterns because the effects of the drinking scores on missingness should be restricted to post‐baseline measurements. We again use interactions between the outcomes and the PREKNOT dummy variable to activate their influence at occasions with missing data. Applications with incomplete baseline data would not require these product terms. Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1DPDDtiPREKNOTti+zti (6) Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1DPDDtiPREKNOTti+λ2DPDDti2PREKNOTti+zti Mti*=λ0+∑t=1T−1λt0Oti+λt1OtiMEDi+λ1DPDDtiPREKNOTti+λ2DPDDt−1,iPREKNOTti+zti As before, we recommend building selection models in a sequence of increasing complexity. Accordingly, the first model corresponds to an MNAR process in which dropout at follow‐up depends on one's concurrent, unobserved drinking score. In this model, the effect of drinking is unidirectional; the linear term allows sudden increases or decreases in drinking to affect attrition, but not both simultaneously. The second model incorporates a squared term that allows both positive and negative outcomes to predict missingness. Although this model may appear more flexible, we anticipate that it will be prohibitively complex in many contexts. Finally, the third model is the classic Diggle–Kenward selection model that includes a CMAR component adjusting for a person's drinking at the prior occasion. Table 4 shows the effect size estimates, parameter standard deviations (“Bayesian standard errors”), and frequentist p values for the six selection models. The rightmost columns show the pseudo‐R 2 values and number of iterations for each model. The models with current and previous drinking levels predicting attrition (Models 6 and 8 in the table) converged rapidly and exhibited adequate diagnostics (all PSRF values < 1.05 and all ESS values > 100 after 20,000 iterations). The missingness model with a bidirectional effect for DPDD (Model 7) did not converge or produce a valid solution after 250,000 iterations. The fact that the illustrative data did not support this model does not eliminate it from consideration in other applications. To reiterate, data‐specific features affect the estimability of all MNAR models. TABLE 4Standardized effect sizes from MNAR selection models.Outcome1‐month SMD4‐month SMD RM2 IterationsEst.SD p Est.SD p Model 6: Current drinking predicting attritionRaw DPDD−0.130.090.14−0.140.10.170.4520,000Log DPDD−0.250.150.11−0.320.190.090.4420,000Model 7: Current drinking and its square predicting attritionRaw DPDD a −0.240.100.02−0.280.120.021.00250,000Log DPDD a −0.280.180.12−0.320.210.130.85250,000Model 8: Current and previous drinking predicting attritionRaw DPDD−0.130.090.14−0.140.100.170.4420,000Log DPDD−0.240.160.11−0.320.190.100.4420,000Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. a Solution judged invalid. Standardized effect sizes from MNAR selection models. Abbreviations: p, frequentist probability value; RM2, attrition model pseudo‐R 2; SD, posterior standard deviation or “Bayesian standard error”; SMD, standardized mean difference. Solution judged invalid. Models 6 and 8 produced standardized mean difference effect sizes that were virtually indistinguishable from the CMAR results. To illustrate these results graphically, Figure 7 shows the predicted means from Model 8 in standardized form. Although Model 7 did not yield a valid solution, it is still informative to inspect its predictions about the progression of drinking. Relative to the corresponding CMAR analysis, the raw score model produced a larger effect size at the first follow‐up, and the mean difference increased slightly by the end of the study. The trajectories from the log‐transformed model were not meaningfully different from the CMAR analysis. Figure 8 shows the predicted means in standardized form. Finally, the significance test conclusions were unaffected, and p values from the selection models were quite similar to the CMAR analysis. Model‐predicted means from selection Model 8. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume an MNAR process in which the current (unobserved) and previous (observed) drinking scores influence attrition. Model‐predicted means from selection Model 7. Panels A and B show the predicted means for the raw and log‐transformed outcomes, respectively. The model‐predicted means are standardized by subtracting the baseline mean and dividing by the baseline standard deviation. Both sets of results assume an MNAR process in which the current (unobserved) drinking scores and their squares influence attrition. Neither solution converged and was judged as admissible. Discussion Missing data are a pervasive challenge in a wide range of alcohol research settings, with documented attrition rates ranging from roughly 20% to over 50% (Anton et al. 2006; Fertig et al. 2012; Johnson et al. 2007; Mason et al. 2014; Murray et al. 2013). Not surprisingly, the prevalence of missing data has motivated methodological literature evaluating analytic approaches (Hallgren and Witkiewitz 2013; Hallgren et al. 2016; Witkiewitz et al. 2014) and calls for improving reporting and analytic practices in alcohol research (Grigsby and McLawhorn 2019; Harris et al. 2013; Witkiewitz et al. 2015). One recommendation for improvement is to conduct sensitivity analyses that examine the stability of substantive conclusions across alternative assumptions about missingness (Hallgren et al. 2016; Hallgren and Witkiewitz 2013; Witkiewitz et al. 2015). Although calls for missing data sensitivity analyses are not new (Schafer and Graham 2002; Beunckens et al. 2007), alcohol researchers lack a step‐by‐step framework that can be readily applied to their own data. The purpose of this study was to provide such a tutorial. We illustrated two frameworks for integrating MNAR assumptions into a linear mixed (multilevel) model. The shared parameter model aligns with a process where an individual's overall change trajectory influences their likelihood of nonresponse. For example, participants with steep declining trajectories that reflect rapid improvement may have a greater chance of missing data because they have achieved their treatment goals or are functioning well and no longer prioritize study participation. Conversely, flatter or worsening trajectories may signal that treatment is ineffective, leading some individuals to disengage from the study. In contrast, the selection model reflects a process in which recent episodic shifts in drinking behavior influence missingness. For instance, a participant who suddenly experiences heavier drinking between assessments may be less likely to complete the next assessment. Alternatively, a participant who experiences a sudden improvement in drinking may also be less likely to complete subsequent assessments after achieving their treatment goals. We limit the tutorial to these approaches because both are consistent with the intuitive idea that unobserved drinking behavior may influence nonresponse. MNAR models can be particularly sensitive to distributional assumptions. For this reason, our sensitivity analysis included raw and log‐transformed DPDD outcomes. In fact, the metric was the most salient determinant of differences across analyses. For the raw outcome, change was primarily limited to the first follow‐up, and group trajectories were flat thereafter; in contrast, the log‐transformed outcome continued to decrease after the first follow‐up. Missing data assumptions had much less impact on the results. Considering only the valid solutions, the raw‐score MNAR models produced effect sizes that were indistinguishable from the CMAR analyses. The only noticeable differences occurred when applying shared parameter models to the log‐transformed data. In the model in which random intercepts and random slopes predicted attrition, the effect size at the first follow‐up was roughly the same as in the CMAR analysis, but the post‐knot trajectories were essentially flat instead of decreasing. In the model that incorporated bidirectional influences for the trajectories, the effect size at the first follow‐up was somewhat smaller (d = −0.20 vs. −0.25), but the groups achieved the same difference as in the CMAR analysis by the end of the study. Although missing data assumptions did not appear to have a substantial impact, they easily could under slightly different conditions. We relied on standardized mean difference effect sizes to compare models because they provide an apples‐to‐apples comparison across outcomes with different metrics. In contrast, p values could lead to a different conclusion. For example, the CMAR analyses of the log‐transformed data produced contrasts with p values just above the conventional 0.05 threshold (p = 0.10 and 0.09 at the first and last follow‐up), whereas the corresponding shared parameter models produced p values ranging from 0.07 to 0.35. Thus, it is easy to imagine that even a modest increase in power (e.g., from a slightly larger sample) could lead one analysis to give a statistically significant mean difference while another does not. It is important to consider how our clinical trial illustration may or may not generalize to other alcohol research contexts. Our sensitivity analysis strategy is broadly applicable to settings in which the focal model includes random effects, including generalized linear mixed models for binary outcomes like abstinence or relapse. However, data‐specific features can meaningfully alter certain steps. For example, the missing data pattern—intermittent missingness or attrition—fundamentally shapes how nonresponse is defined and how MNAR models are specified. Applying these analyses to clinical or behavioral trials with attrition is arguably more challenging because complete baseline data make it harder to model temporal trends in nonresponse. Similar challenges may arise in longer study spans or applications with more measurement occasions, especially when multiple occasions are (nearly) complete or nonresponse rates trend toward zero. Finally, our sample composition is a potential source of difference. Although the attrition rate was comparable to published rates (27% disengaged at some point), a sample of daily smokers may exhibit different nonresponse tendencies or levels of MNAR signal than other populations. At a high level, we would not expect our pattern of results—or our conclusion that missingness assumptions were not substantially impactful—to generalize across applications. Selection and shared parameter models capture fundamentally different sources of association between the outcome and missingness, and there is no reason to expect trajectory‐dependent and outcome‐dependent processes to operate in each setting or to exert the same level of signal across settings. Our goal was to present a palette of models that alcohol researchers can apply to their data. It is important to reiterate that data‐specific features affect the estimability of these models. Some models exhibited estimation difficulties that led us to judge the solutions as invalid, which is typical rather than exceptional. However, models that failed with our data may be viable in other settings, and those that performed adequately here may not do so elsewhere. For this reason, model checking with MCMC diagnostics is an essential step in a sensitivity analysis. In closing, we propose several recommendations for reporting the results of a sensitivity analysis. In our recent work, we have presented a CMAR analysis in the main text and have begun using online supplements to present MNAR analysis results (Nieto et al. 2022; Ray et al. 2021, 2025). The composition of the supplement could largely follow the key steps in our tutorial. For example, researchers should identify the predominant missing data pattern and code the missingness indicator appropriately, as this distinction determines whether nonresponse is expressed as occasion‐specific probabilities or dropout (hazard) probabilities. Next, describing the progression of missing data rates over time, and possibly within group, is useful because it informs how to model time‐related changes in missingness as a precursor to MNAR analyses (e.g., using dummy codes, as we do, or polynomial functions). Graphs like Figure 3A are useful in this regard. Next, researchers should describe the models included in the sensitivity analysis. This may involve CMAR analyses with auxiliary variables, shared parameter models for trajectory‐dependent missingness, and selection models for outcome‐dependent processes. The functional form matters, as linear effects imply unidirectional influences, whereas polynomial terms allow both positive and negative outcomes to affect missingness. Again, model checking is essential because convergence failures and computational difficulties are the norm rather than the exception. Researchers should report models that lack support from the data and document the diagnostics used to vet their solutions. Finally, reports should ideally present numeric or graphical summaries. The combination of standardized effect sizes and p values for contrasts or effects of interest is useful, as it provides practical metrics for evaluating assumption‐related changes. Finally, it is important to acknowledge that a vast number of MNAR processes are possible in any given setting, and even the most comprehensive sensitivity analysis can consider only a small subset of these possibilities. Many plausible MNAR specifications remain unexamined and could yield different patterns of results. Moreover, discrepant results across models should not be viewed as problematic, nor should consistent findings like ours be interpreted as confirmatory evidence for CMAR. Regardless of the pattern, the goal of a sensitivity analysis is to provide a more complete picture of the study's findings across a range of plausible assumptions about nonresponse. Funding This work was supported by National Institute on Drug Abuse award R01DA041226 and by National Institute on Alcohol Abuse and Alcoholism award K24AA025704. Conflicts of Interest The authors declare no conflicts of interest.