Corrected Version of “Simplicity Above Elegance: Another Look at the Asymptotically Correct Standardization of Snijders”

Sandip Sinharay, ETS · 2026

Read Corrected Version of “Simplicity Above Elegance: Another Look at the Asymptotically Correct Standardization of Snijders” by Sandip Sinharay, ETS (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00216.txt.

Introduction

Sandip Sinharay revisits a technical result in educational testing: how to standardize person-fit statistics, which flag test takers whose answer patterns look aberrant. Snijders had supplied a standardization that is correct in the large-sample limit, for a broad class of such statistics. Sinharay offers a different derivation of the same result.

The new derivation yields a simpler formula and a simpler description of several known statistics, including the well-known lz statistic. It also explains, on theoretical grounds, simulation findings reported by Snijders and by other earlier researchers. The paper is a work of statistical housekeeping: same destination, cleaner road, and a clearer view of why some older numerical results came out as they did.

Catalog record

CREATORSandip Sinharay, ETS
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00216

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

(WLE, 41 ) , or modal a posteriori (MAP) estimate, r_{i}(\theta) given by \displaystyle r_{i}(\theta)=\frac{P^{\prime}_{i}(\theta)}{P_{i}(\theta)\{1-P_{i}(\theta)\}}, (16) r_{0}(\hat{\theta}) is defined as r_{0}(\hat{\theta})=\left\{\begin{array}[]{l}0\mbox{ if }\hat{\theta}=\mbox{MLE},\\ \frac{d\log f(\hat{\theta})}{d\hat{\theta}}\mbox{ if }\hat{\theta}=\mbox{MAP}\\ \frac{J(\hat{\theta})}{2I(\hat{\theta})}\mbox{ if }\hat{\theta}=\mbox{WLE},\end{array}\right. where f(\theta) is the prior distribution on \theta , J(\theta)=\sum_{i}\frac{P^{\prime}_{i}(\theta)P^{\prime\prime}_{i}(\theta)}{P_{i}(\theta)(1-P_{i}(\theta))} , P^{\prime\prime}_{i}(\theta) is the second derivative of P_{i}({\theta}) , and the regularity conditions are satisfied for the common dichotomous IRT models. 32 proved that the results of 34 hold when \hat{\theta} is the posterior mean or expected a posteriori (EAP) estimate ( 2 ) for r_{i}(\theta) given by Equation 16 and also when \hat{\theta} is a robust ability estimate like the biweight ( 23 ) or Huber estimate ( 26 ) . 3 3 3 The expression of r_{i}(\theta) for robust estimates is somewhat cumbersome and is given in Appendix B. The asymptotically correct standardized version of the l_{z} statistic is popularly known as the l^{*}_{z} statistic (e.g., 40 ) and has a standard normal asymptotic distribution. Because of Equation 16 , c(\theta) provided in Equation 13 can be expressed as \displaystyle c(\theta)=\frac{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)}{\sum_{i}P^{\prime}_{i}(\theta)r_{i}(\theta)}=\frac{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)}{\sum_{i}\{P^{\prime}_{i}(\theta)\}^{2}/[P_{i}(\theta)\{1-P_{i}(\theta)\}]}=\frac{1}{I(\theta)}\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta), (17) because, for dichotomous IRT models, \displaystyle I(\theta)=\sum_{i}\frac{\{P^{\prime}_{i}(\theta)\}^{2}}{P_{i}(\theta)\{1-P_{i}(\theta)\}} (18) (e.g. 10 , p. 89) . Methods: An Alternative Derivation This section provides an alternative and arguably more direct derivation of the standardization of 34 . First, an expression of the asymptotic variance of W(\hat{\theta}) is derived, and then, the asymptotic distribution of W(\hat{\theta}) divided by the estimated square root of its approximate variance is derived. The Assumptions The derivations are predicated on the assumptions that \hat{\theta} is the MLE of ability or another ability estimator that is asymptotically equivalent to (or can be approximated well, for long tests, by) the MLE 4 4 4 Note that this assumption implies that \hat{\theta}\rightarrow\theta as n\rightarrow\infty . and that \displaystyle\sqrt{n}(\hat{\theta}-\theta)\overset{\scriptstyle d}{\longrightarrow}\mathcal{N}(0,\bar{I}^{-1}_{0}(\theta)), (19) where \bar{I}_{0}(\theta) is the (finite) limiting value, for n\rightarrow\infty , of I(\theta)/n , and I(\theta) is the Fisher information for the examinee ability. The assumption represented by Equation 19 is similar to the assumption of 34 that \sqrt{n}(\hat{\theta}-\theta) has a non-degenerate distribution as n\rightarrow\infty . It is also assumed, as in 34 , that P_{i}(\theta) and w_{i}(\theta) are twice differentiable and these functions and their second derivatives are uniformly bounded in a neighborhood of \theta ; these assumptions are satisfied for common dichotomous IRT models and several popular person-fit statistics. Approximating W(\hat{\theta}) by a Function of \theta Application of the first order Taylor-series expansion to l^{\prime}(\hat{\theta}) , which is the first derivative of l(\theta) defined in Equation 5 , at \theta=\hat{\theta} , leads to the result that for long tests, \displaystyle l^{\prime}(\hat{\theta})\approx l^{\prime}(\theta)+(\hat{\theta}-\theta)l^{\prime\prime}(\theta), (20) where l^{\prime\prime}(\theta) denotes the second derivative of l(\theta) , and hence \displaystyle\hat{\theta}\approx\theta+\frac{1}{l^{\prime\prime}(\theta)}\{l^{\prime}(\hat{\theta})-l^{\prime}(\theta)\}\cdot (21) The derivative l^{\prime}(\theta) is given by \displaystyle l^{\prime}(\theta)=\sum_{i=1}^{n}\frac{X_{i}-P_{i}(\theta)}{P_{i}(\theta)\{1-P_{i}(\theta)\}}P^{\prime}_{i}(\theta) (22) (e.g., 10 , p. 82) . Note that the MLE of ability is computed by solving the equation l^{\prime}({\theta})=0 (e.g. 10 , p. 83) —so l^{\prime}(\hat{\theta}) is equal to 0 when \hat{\theta} is the MLE. Because \hat{\theta} , when not equal to the MLE, is assumed to be asymptotically equivalent to the MLE, l^{\prime}(\hat{\theta})\rightarrow 0 as n\rightarrow\infty for these estimates. So, Equation 21 leads to the result that for long tests, \displaystyle\hat{\theta}\approx\theta+\frac{l^{\prime}(\theta)}{-l^{\prime\prime}(\theta)}\cdot (23) This result is very similar to one provided in Equation 42 of 20 . The quantity -l^{\prime\prime}(\theta) in Equation 23 is the observed information function for \theta and converges to its expected value, which is I(\theta) , the Fisher information for the examinee ability, by the Chebyshev’s theorem ( 25 , p. 112) given the aforementioned regularity conditions. 5 5 5 A detailed proof can be found in Lemma 1 of 20 as applied to the MLE. Therefore, Equation 23 leads to the result that for long tests, \displaystyle\hat{\theta}-\theta\approx\frac{l^{\prime}(\theta)}{I(\theta)}\cdot (24) Equations 2 , 10 , 22 , and 24 imply that \displaystyle W(\hat{\theta}) \displaystyle\approx \displaystyle W(\theta)-\frac{l^{\prime}(\theta)}{I(\theta)}\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\} (25) \displaystyle= \displaystyle\sum_{i}(X_{i}-P_{i}(\theta))w_{i}(\theta)-\frac{1}{I(\theta)}\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\}\sum_{i}\frac{X_{i}-P_{i}(\theta)}{P_{i}(\theta)\{1-P_{i}(\theta)\}}P^{\prime}_{i}(\theta) \displaystyle= \displaystyle\sum_{i}(X_{i}-P_{i}(\theta))w_{i}(\theta)-c(\theta)\sum_{i}\frac{X_{i}-P_{i}(\theta)}{P_{i}(\theta)\{1-P_{i}(\theta)\}}P^{\prime}_{i}(\theta) \displaystyle= \displaystyle\sum_{i}(X_{i}-P_{i}(\theta))\left(w_{i}(\theta)-c(\theta)\frac{P^{\prime}_{i}(\theta)}{P_{i}(\theta)\{1-P_{i}(\theta)\}}\right) \displaystyle= \displaystyle\sum_{i}(X_{i}-P_{i}(\theta))(w_{i}(\theta)-c(\theta)r_{i}(\theta)), for r_{i}(\theta) and c(\theta) provided in Equations 16 and 17 , respectively. Because of the local independence property of IRT models, the terms (X_{i}-P_{i}(\theta))(w_{i}(\theta)-c(\theta)r_{i}(\theta)),i=1,2,\ldots,n , are independent of each other given \theta , and consequently, the variance of W(\hat{\theta}) can be approximated by the variance of the right-hand side of Equation 25 as \displaystyle\mathop{\mathrm{Var}}\nolimits(W(\hat{\theta})) \displaystyle\approx \displaystyle\sum_{i}P_{i}(\theta)\{1-P_{i}(\theta)\}(w_{i}(\theta)-c(\theta)r_{i}(\theta))^{2} (26) \displaystyle= \displaystyle\sum_{i}\tilde{w}^{2}_{i}(\theta)P_{i}(\theta)\{1-P_{i}(\theta)\}=n\tau^{2}(\theta) for expressions of \tilde{w}_{i}(\theta) and \tau^{2}(\theta) provided in Equations 12 and 15 . The Standardized Statistic and its Asymptotic Distribution Define the standardized version of W(\hat{\theta}) as \displaystyle\tilde{\tilde{Z}}(\hat{\theta})=\frac{W(\hat{\theta})}{\sqrt{n}\tau(\hat{\theta})}\cdot (27) Let us denote the expression in the right-hand side of Equation 25 , which is the sum of n independent and non-identically distributed random variables, as G(\theta) . Because the individual terms of G(\theta) have mean of 0 and are uniformly bounded by the aforementioned regularity conditions, \displaystyle\frac{G(\theta)}{\sqrt{\mathop{\mathrm{Var}}\nolimits(G(\theta))}}=\frac{G(\theta)}{\sqrt{n}\tau({\theta})}\rightarrow\mathcal{N}(0,1) (28) by Equation 26 and the Lindeberg-Feller central limit theorem for independent but non-identical random variables (e.g., 25 , p. 128) . Equations 25 , 26 , and 28 imply that \displaystyle\frac{W(\hat{\theta})}{\sqrt{n}\tau({{\theta}})}\rightarrow\mathcal{N}(0,1)\cdot (29) The standardized statistic \tilde{\tilde{Z}}(\hat{\theta}) can also be expressed as \tilde{\tilde{Z}}(\hat{\theta})=\frac{W(\hat{\theta})}{\sqrt{n}\tau({{\theta}})}\frac{\tau({{\theta}})}{\tau({\hat{\theta}})}. Because \hat{\theta} converges to {\theta} , \frac{\tau({{\theta}})}{\tau({\hat{\theta}})}\rightarrow 1 because of the result that X_{n}\rightarrow c\Rightarrow g(X_{n})\rightarrow g(c) for any continuous function g where X_{n} is a sequence of random variables (e.g. 25 , p. 124) . Consequently, because \tilde{\tilde{Z}}(\hat{\theta}) is the product of two random variables and asymptotically, the first of the random variables is a standard normal random variable (by Equation 29 ), and the second is equal to the constant 1, the Slutsky’s theorem (e.g. 27 , p. 19) implies that asymptotically, \tilde{\tilde{Z}}(\hat{\theta}) follows the \mathcal{N}(0,1) standard normal distribution. An Alternative Variance Expression Note that by Equations 3 and for the expression of c(\theta) provided in 17 , \tau^{2}(\theta) can be expressed as \displaystyle\tau^{2}(\theta) \displaystyle= \displaystyle\frac{1}{n}\sum_{i}\tilde{w}^{2}_{i}(\theta)P_{i}(\theta)\{1-P_{i}(\theta)\} (30) \displaystyle= \displaystyle\frac{1}{n}\sum_{i}\left\{w_{i}(\theta)-r_{i}(\theta)c(\theta)\right\}^{2}P_{i}(\theta)\{1-P_{i}(\theta)\} \displaystyle= \displaystyle\frac{1}{n}\sum_{i}\left\{w_{i}(\theta)-\frac{r_{i}(\theta)}{I(\theta)}\sum_{k}P^{\prime}_{k}(\theta)w_{k}(\theta)\right\}^{2}P_{i}(\theta)\{1-P_{i}(\theta)\} \displaystyle= \displaystyle\sigma^{2}(\theta)+\frac{1}{n}\frac{\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\}^{2}}{I^{2}(\theta)}\sum_{i}r^{2}_{i}(\theta)P_{i}(\theta)\{1-P_{i}(\theta)\} \displaystyle- \displaystyle\frac{2}{n}\frac{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)}{I(\theta)}\sum_{i}w_{i}(\theta)r_{i}(\theta)P_{i}(\theta)\{1-P_{i}(\theta)\}\cdot Using Equations 16 and 18 , the second term above is equal to \frac{1}{n}\frac{\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\}^{2}}{I^{2}(\theta)}\sum_{i}\frac{\{P^{\prime}_{i}(\theta)\}^{2}}{P_{i}(\theta)\{1-P_{i}(\theta)\}}=\frac{1}{n}\frac{\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\}^{2}}{I(\theta)}, and the third term above is equal to \frac{2}{n}\frac{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)}{I(\theta)}\sum_{i}w_{i}(\theta)P^{\prime}_{i}(\theta)=\frac{2}{n}\frac{\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\}^{2}}{I(\theta)} Therefore, \tau^{2}(\theta) can be expressed as \displaystyle\tau^{2}(\theta) \displaystyle= \displaystyle\sigma^{2}(\theta)-\frac{1}{nI(\theta)}\left\{\sum_{i}P^{\prime}_{i}(\theta)w_{i}(\theta)\right\}^{2}\cdot (31) 16 derived a result like Equation 31 for the l^{*}_{z} statistic. Because of Equation 19 , for long tests, \mathop{\mathrm{Var}}\nolimits(\hat{\theta}) is approximately equal to I^{-1}({\hat{\theta}}) . Therefore, an asymptotic approximation of \tau^{2}(\hat{\theta}) is obtained as \displaystyle\tau^{2}(\hat{\theta})\approx\psi^{2}(\hat{\theta})=\sigma^{2}(\hat{\theta})-\frac{1}{n}\widehat{\mathop{\mathrm{Var}}\nolimits}(\hat{\theta})\left\{\sum_{i}P^{\prime}_{i}(\hat{\theta})w_{i}(\hat{\theta})\right\}^{2}, (32) where \widehat{\mathop{\mathrm{Var}}\nolimits}(\hat{\theta}) denotes an estimate of \mathop{\mathrm{Var}}\nolimits(\hat{\theta}) . Consequently, another expression of \tilde{\tilde{Z}}(\hat{\theta}) is given by \displaystyle\tilde{\tilde{Z}}(\hat{\theta})=\frac{W(\hat{\theta})}{\sqrt{n}\psi(\hat{\theta})}\cdot (33) The Ability Estimates for which the Asymptotic Normality of \tilde{\tilde{Z}}(\hat{\theta}) Holds It was mentioned earlier that the derivations in this paper underlie the assumptions that (and hence the asymptotic normality of \tilde{\tilde{Z}}(\hat{\theta}) holds when) \hat{\theta} is the MLE or asymptotically equivalent to the MLE and satisfies the condition stated in Equation 19 . The MLE of \theta satisfies the condition stated in Equation 19 for most common IRT models when it is reasonable to assume that the test is lengthened by adding strictly parallel forms (e.g. 17 ; 41 ; 28 ) . 6 6 6 17 stated that the latter condition is usually assumed to hold in IRT applications. So the asymptotic normality of \tilde{\tilde{Z}}(\hat{\theta}) holds for the MLE. A proof of the asymptotic equivalence of WLE and MLE under some regularity conditions 7 7 7 One condition is that the test is lengthened by adding strictly parallel forms. is given in 41 . The asymptotic equivalence of MLE and either of MAP or EAP, under some regularity conditions,

Read the complete text