Tobit modeling for dependent-sample t-tests and moderated regression with ceiling or floor data Authors: Lijuan Wang, Ruoxuan Li Published: 2025 ABSTRACT Ceiling or floor effects pose analytic challenges in behavioral and psychological research. In this study, we developed novel Tobit modeling approaches, estimated using maximum likelihood (ML) or Bayesian methods, to address these effects for widely used statistical analyses, including the dependent-sample t-test and moderated regression. Simulation studies were conducted to compare the performance of the proposed modeling approaches to the conventional approach where ceiling or floor data are treated as if true values. The conventional approach was found to yield biased estimates, inflated Type I error rates, and poor confidence interval coverage, even with as little as 10% ceiling data. In contrast, the proposed approaches with either ML or Bayesian estimation provided accurate estimates and inference results across most studied conditions (e.g., with 30% ceiling data). Real data examples further illustrated the impact of modeling choices. To facilitate implementations of the proposed Tobit modeling approaches, we provide simulated datasets along with R and Mplus scripts online. Implications of the findings and future research directions were discussed.Supplementary InformationThe online version contains supplementary material available at 10.3758/s13428-025-02904-y. Introduction Ceiling or floor effects occur when measurement scales are too easy or too difficult and thus a substantial proportion of individuals obtain the maximum or minimum score (Uttl, 2005). These effects are prevalent in behavioral, psychological, and medical research. For example, Uttl (2005) found ceiling effects in 25% or more of 9- to 15-item verbal list learning tasks, and in 50% or more of the verbal paired-associations learning tasks used to assess memory in cognitive psychology. In social surveys, ceiling effects often appear in ’top-coding’ questions (Austin & Brunner, 2003), such as those asking about income, where the highest category represents an income greater than a certain threshold. Similarly, in medical research, ceiling effects have been observed in measurement scales like pain assessment tools used to evaluate outcomes in hip replacement surgery (Lim et al., 2015). Some researchers argued that the use of sum scores from measurement scales with too easy or too difficult Likert items can lead to ceiling or floor effects (Edwards & Soland, 2024; McNeish, 2024). For individuals with ceiling or floor effects, their true scores are not observed. Specifically, when a true score exceeds the maximum score (or the ceiling threshold), the observed sum score is the maximum score. Similarly, when a true score is smaller than the minimum score (or the floor threshold), the observed sum score is the minimum score. When ceiling or floor effects occur, there are mass points with dense frequencies at the maximum or the minimum in the distribution of observed data. Note that observed data from a semicontinuous variable ((Olsen & Schafer, 2001)) also have mass points – for example, alcohol or other substance usage often has a mass point at zero. In learning research, performance asymptotes (Miller, 1956) lead to distributions with mass points as well, such as at the upper and/or lower asymptotic values. Wang et al. (2008) noted that the mass point value in a semicontinuous variable (e.g., zero alcohol usage) is a true data value. Likewise, performance asymptote values are the greatest or lowest true values participants can demonstrate and thus are also true data values. In the current study, we follow the definition in Uttl (2005) and assume ceiling and floor effects occur before participants reach asymptotic values, making the mass point values from ceiling and floor effects proxies for larger or smaller true values. Consequently, statistical methods for handling ceiling or floor effects should be different from the two-part models used for semicontinuous variables (e.g., Olsen and Schafer, 2001) and models developed to data analyses with performance asymptotes (e.g., Feng et al., 2019; Singer and Willett, 2003). Liu and Wang (2021) conducted a literature review to study how psychological and educational researchers statistically handled ceiling or floor effects in independent-sample t-tests and ANOVA. They found that over 57% of the reviewed articles using t-tests and 70% of those using ANOVA ignored ceiling or floor effects, treating ceiling or floor data as if there were true scores in the analyses. In this study, we refer to this practice as the conventional approach due to its widespread use. The conventional approach underestimates marginal means in the presence of ceiling effects and overestimates them in the presence of floor effects. When both ceiling and floor effects occur, the extent to which observed means deviate from true means depends on the proportions of ceiling and floor data. Marginal variances are also underestimated under this approach. Researchers have demonstrated that the conventional approach may produce misleading statistical inference results under the presence of ceiling data (Liu & Wang, 2021; Wang et al., 2008). For instance, Liu and Wang (2021) observed biased estimates and inflated Type I error rates in independent-sample t-tests and ANOVA when ceiling effects existed in the outcomes. Similarly, Austin and Brunner (2003) found inflated Type I error rates in multiple regression analyses when ceiling effects existed in a predictor. In biostatistics or economics, censored data arise when the value of an observation is only partially known due to certain restrictions. For example, in biostatistics, right-censored data occur when a patient’s survival time is only known to exceed a certain point (e.g., 5 years) because they were still alive at the end of the study. Ceiling and floor data share similar statistical properties of right- or left-censored data in biostatistics and economics. As a result, Tobit modeling, introduced by Tobin (1958) to handle limited or censored data in economics, has been adapted to address ceiling and floor effects across various models and methods in psychology and behavioral research (Liu & Wang, 2021; Muthen, 1990; Wang et al., 2008; Wang & Zhang, 2011). When applying Tobit modeling to handle ceiling or floor effects, the key idea is to specify different likelihood functions for data with and without these effects. For example, when the underlying distribution is normal, the normal density function is used to specify the likelihoods for data without ceiling or floor effects, whereas the cumulative density function is used to specify the likelihoods for data with ceiling or floor effects. Maximum likelihood (ML) estimation has been employed to estimate Tobit models for addressing censored data in factor analysis (Muthen, 1989) and for handling ceiling or floor data in multiple regression (Austin & Brunner, 2003), longitudinal data analysis (Twisk & Rijmen, 2009), and independent-sample t tests and ANOVA (Liu & Wang, 2021). Additionally, Bayesian methods have been developed to address ceiling or floor effects in latent growth curve modeling (Wang et al., 2008) and mediation analysis (Wang & Zhang, 2011). Research has consistently shown that Tobit modeling, using either ML or Bayesian methods, outperforms the conventional approach when analyzing censored or ceiling/floor data. There is considerable discussion in the literature regarding the advantages of using Bayesian estimation over ML estimation, such as its ability to incorporate prior information and estimate complex random-effects or latent variable models (e.g., Du et al., 2017; McNeish, 2016; Zhang et al., 2008). However, no prior research has systematically compared the performance of Tobit modeling using ML estimation versus Bayesian estimation for handling ceiling or floor effects. To our knowledge, no prior research has developed or evaluated methods to address ceiling or floor effects for the dependent-sample t-test, a widely used statistical test. Although extending Tobit modeling from the independent-sample t-test to the dependent-sample t-test may seem straightforward, it presents practical challenges (to be discussed in the next section). Furthermore, in most existing studies, Tobit models have been developed to address ceiling or floor effects in dependent variables (e.g., outcomes, mediators). An exception is the work by Austin and Brunner (2003), who developed a likelihood function for multiple regression with two predictors, one of which exhibits ceiling effects. However, their approach is applicable to their studied multiple regression model and does not extend to more complex scenarios, such as multiple regression with moderators. Moreover, the likelihood function they proposed can be challenging for applied researchers to implement. For example, in moderated regression where the regression model involves moderators and thus interaction terms, the interaction terms are subject to ceiling or floor effects when their corresponding focal predictors and/or moderators are subject to these effects. However, how to statistically address these effects in interaction terms has not been studied previously. To fill in the research gaps, we developed novel modeling approaches integrating both Tobit modeling and latent variable modeling to handle ceiling or floor effects for two widely used statistical techniques: the dependent-sample t-test and moderated regression. In the proposed Tobit models, ceiling or floor effects in both the pre-test and post-test scores for the dependent-sample t-test and in both the outcome variable and predictors/moderators for moderated regression can be accounted for. For both the dependent-sample t-test and moderated regression, ML and Bayesian methods were developed to estimate the proposed Tobit models and implemented in Mplus and R JAGS, respectively. Furthermore, we conducted Monte Carlo simulation studies to evaluate and compare the performance of the proposed Tobit modeling approaches to that of the conventional approach as well as compare the performance of ML vs. Bayesian estimation for Tobit modeling. Real data analysis examples and simulated data with Mplus and R JAGS scripts were provided to illustrate the modeling approaches for both the dependent-sample t-test and moderated regression. In the remainder of the article, we will first introduce our proposed Tobit modeling approaches with ML and Bayesian estimation for the dependent-sample t-test. Then we will describe the design and results of a simulation study to evaluate the performance of different modeling approaches for handling ceiling or floor effects in dependent-sample t-tests, followed by a real data analysis example. After that, for moderated regression, we will present our proposed Tobit modeling approaches with ML and Bayesian estimation, describe the design and results from a separate simulation study for performance evaluation, and then a real data analysis example. Finally, we will conclude the article by discussing the implications of our findings and suggesting directions for future research. The dependent-sample t-test with ceiling or floor data The dependent-sample t-test (also known as the paired-sample t-test) is widely used to compare the means of two related samples, such as pre-test and post-test measurements or matched-pairs samples. For illustrative purposes, we consider the pre-test and post-test scenario as an example to introduce the modeling approaches. In such a research setting, researchers have observed data from individual i \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(i=1,2,..,N)$$\end{document}(i=1,2,..,N) at time point t (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t=$$\end{document}t= 1 for pre-test and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t=$$\end{document}t= 2 for post-test), denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}$$\end{document}yit. When ceiling or floor effects occur, the true scores \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}^{*}$$\end{document}yit∗ may not be fully observed. Specifically, when a true score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}^{*}$$\end{document}yit∗ falls below or equals the minimum score a (the floor threshold), the observed score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}$$\end{document}yit is constrained at a. When a true score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}^{*}$$\end{document}yit∗ exceeds the maximum score b (the ceiling threshold), the observed score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}$$\end{document}yit is capped at b. For individuals without ceiling or floor effects, their true scores are observed directly and fall within the range of a and b. Together, the observed scores \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{it}$$\end{document}yit can be expressed as:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} y_{it}={\left\{ \begin{array}{ll} a & y_{it}^{*}\le a\\ y_{it}^{*} & a