A novel nonparametric framework for DIF detection using kernel-smoothed item response curves

Adéla Hladká, Patrícia Martinková · 2025

Read A novel nonparametric framework for DIF detection using kernel-smoothed item response curves by Adéla Hladká, Patrícia Martinková (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18963.txt.

Introduction

Adéla Hladká and Patrícia Martinková detect biased test items by a direct, model free comparison of response curves. A biased item favors one group over another at the same skill level, a flaw called differential item functioning. Their method smooths each group's item response curve with a kernel and compares the curves head on.

The work extends past curve comparison methods to binary answers, the norm in psychometrics. A new estimator gives the test statistic's long run variance, and weight functions are tuned to lift local power. Since the limit law is unknown, a wild bootstrap drives inference. Simulations show sound error control and power on par with the standard logistic method, with gains when curves cross more than once.

Catalog record

CREATORAdéla Hladká, Patrícia Martinková
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18963

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A novel nonparametric framework for DIF detection using kernel-smoothed item response curves Adéla Hladká, Patrícia Martinková Abstract This study introduces a novel nonparametric approach for detecting Differential Item Functioning (DIF) in binary items through direct comparison of Item Response Curves (IRCs). Building on prior work on nonparametric comparison of regression curves, we extend the methodology to accommodate binary response data, which is typical in psychometric applications. The proposed approach includes a new estimator of the asymptotic variance of the test statistic and derives optimal weight functions that maximise local power. Because the asymptotic distribution of the resulting test statistic is unknown, a wild bootstrap procedure is applied for inference. A Monte Carlo simulation study demonstrates that the nonparametric approach effectively controls Type I error and achieves power comparable to the traditional logistic regression method, outperforming it in cases with multiple intersections of the underlying IRCs. The impact of bandwidth and weight specification is explored. Application to a verbal aggression dataset further illustrates the method’s ability to detect subtle DIF patterns missed by parametric models. Overall, the proposed nonparametric framework provides a flexible and powerful alternative for detecting DIF, particularly in complex scenarios where traditional model-based assumptions may not be applicable. 1 Introduction Multi-item assessments play a significant role in various areas of our everyday lives, intervening in education, psychology, healthcare, and other applied fields ( 4 ; 11 ; 22 ) . In the educational context, they are used to assess academic performance, certify student qualifications, measure knowledge and skill proficiency, and administer admission tests, as well as national and international large-scale student exams. In psychology, these assessments help measure intelligence, personality traits, and attitudes. In health-related fields, they are employed for measuring outcomes such as fatigue, depression, pain, quality of life, and overall well-being. Similar instruments are also further applied in employment selection and promotion processes, peer reviews of academic work, and the evaluation of grant proposals. While decisions based on such assessments are often guided by total scores, a detailed analysis at the item level provides deeper insight. Item functioning can be described by Item Response Curve (IRC) , which expresses the probability of a specific response to an item as a function of an individual’s latent trait and, potentially, additional covariates. Parametric modelling frameworks, such as Item Response Theory (IRT) ( 35 ; 36 ) and score-based regression models ( 22 ) , are widely used to estimate IRCs . However, any parametric approach risks oversimplification when the underlying model omits critical information. In cases where the true model is highly complex or lacks a clear parametric form, nonparametric methods offer a more flexible alternative. Notable examples include the monotone homogeneity model and double-monotonicity model for binary items ( 23 ) , as well as the kernel-smoothing approach employing Nadaraya-Watson weights and rank-based ability estimates ( 28 ) . A crucial aspect of item functioning is that it may vary across different respondent groups. This phenomenon, known as Differential Item Functioning (DIF) , occurs when respondents from different groups but with the same underlying trait have different probabilities of giving a certain response to an item. Parametric approaches, such as IRT models and score-based regression methods, can be extended to account for the effect of covariates (e.g., group-membership variable) and applied for DIF detection ( 18 ; 27 ; 34 ; 9 ; 16 ; 15 ) . Yet, to our knowledge, a nonparametric DIF detection directly based on comparing IRCs has not been systematically explored. To address this gap, we introduce a new kernel-smoothing-based framework for the nonparametric comparison of IRCs across groups. Our method adapts and extends the general approach of 33 to the binary-item setting, developing it specifically for the purpose of detecting DIF . We further propose several methodological variants designed to accommodate different DIF types, and we demonstrate their performance in a systematic comparison with the widely used logistic regression method ( 34 ) . The remainder of this paper is structured as follows. Section 2 details the estimation procedure, the general test statistic, and several weight-function strategies, including optimal weights designed to maximise test power, along with their estimates and asymptotic properties of the resulting test statistic. Section 3 presents a simulation study evaluating the proposed framework in comparison to the traditional parametric approach based on logistic regression. Section 4 illustrates the methodology using an empirical dataset from a questionnaire about verbal aggression. Section 5 concludes with a discussion of the findings, practical recommendations, and directions for future work. 2 Methodology DIF and its detection are closely connected to the broader problem of describing the relationship between respondents’ responses \boldsymbol{Y_{i}}=(Y_{1i},\dots,Y_{ni}) to item i and their abilities \boldsymbol{\theta}=(\theta_{1},\dots,\theta_{n}) , where n is a number of respondents. This relationship can generally be expressed through a regression function m_{i} for item i : \displaystyle\boldsymbol{Y_{i}}=m_{i}(\boldsymbol{\theta})+\boldsymbol{\epsilon_{i}},\quad\E(\boldsymbol{\epsilon_{i}}|\boldsymbol{\theta})=\boldsymbol{0} This work focuses on binary outcomes \boldsymbol{Y_{i}} , in which case this relationship can be reformulated as \displaystyle\E(\boldsymbol{Y_{i}}|\boldsymbol{\theta})=\Prob(\boldsymbol{Y_{i}}=1|\boldsymbol{\theta})=m_{i}(\boldsymbol{\theta}). We estimate IRCs , i.e., the function m_{i}(\cdot) , using the nearest-neighbor kernel-smoothing approach of 33 across two groups: reference ( g=0 ) and focal ( g=1 ). Let \mathcal{P}_{g} denote the set of respondents in group g of size n_{g} , such that n_{0}+n_{1}=n is the total sample size. For a given item i , let Y_{pi} be the binary response of respondent p and \theta_{p} their ability (e.g., standardised total test score or other matching criterion). Define \hat{F}_{ig}(x) as the empirical distribution function of \theta_{p} in group g : \displaystyle\hat{F}_{g}(x)=\frac{1}{n_{g}}\sum_{p\in\mathcal{P}_{g}}\mathbf{1}[\theta_{p}\leq x]. The nearest neighbor estimate of the IRC m_{ig} for item i for group g is then given by: \displaystyle\hat{m}_{ig}(x)=\sum\limits_{p\in\mathcal{P}_{g}}Y_{pi}W_{pig}(x), (1) where the weights W_{pig}(x) are defined as: \displaystyle W_{pig}(x)=\frac{K\left(\frac{\hat{F}_{g}(\theta_{p})-\hat{F}_{g}(x)}{h}\right)}{\sum\limits_{k\in\mathcal{P}_{g}}K\left(\frac{\hat{F}_{g}(\theta_{k})-\hat{F}_{g}(x)}{h}\right)}. (2) K(\cdot) is a twice continuously differentiable, symmetric, non-negative kernel function, which is non-decreasing for u<0 with a compact support and \int K(u)\ \mathrm{d}u=1 . Examples include the Epanechnikov kernel K(u)=\frac{3}{4}(1-u^{2}),\left|{u}\right|\leq 1 ( 10 ) , and the uniform kernel K(u)=\frac{1}{2},\left|{u}\right|\leq 1 . The bandwidth parameter h satisfies nh^{3}\rightarrow\infty and nh^{4}\rightarrow 0 as n\rightarrow\infty (see 33 , p. 2042) . Therefore, the parameter h is assumed to take the value of n^{-\zeta} , where \zeta\in\left(\frac{1}{4},\frac{1}{3}\right) and n has the order of n_{0} and n_{1} . Kernel smoothing is advantageous here because it makes no assumptions about the functional form of m_{ig}(x) , making it applicable even when IRCs have complex shapes. For an illustration, for IRCs with multiple inflection points (Figure 1(a) ), the nearest neighbor estimate ( 1 ) applied to simulated binary data provides a closer match to the true curves (Figure 1(b) ) than the logistic regression approach (Figure 1(c) ). (a) Underlying IRCs (b) Nearest neighbours (c) Logistic regression Figure 1: Example of nearest neighbour and logistic regression estimates of IRCs; curves are accompanied by points representing empirical probabilities. 2.1 Test statistic Differences in ability distributions between groups can make direct curve comparison problematic. Following 33 , we define a common support by averaging the values of the matching criterion \theta_{p} from the two groups: \displaystyle\bar{X}_{p_{0}p_{1}}=\frac{\theta_{p_{0}}+\theta_{p_{1}}}{2},\quad p_{0}\in\mathcal{P}_{0},p_{1}\in\mathcal{P}_{1}. The proposed general test statistic for item i is then expressed as follows: \displaystyle\widehat{T}_{i}=\frac{1}{n_{0}n_{1}}\sum_{p_{0}\in\mathcal{P}_{0}}\sum_{p_{1}\in\mathcal{P}_{1}}W_{i}\left(\bar{X}_{p_{0}p_{1}}\right)\left[\hat{m}_{i0}\left(\bar{X}_{p_{0}p_{1}}\right)-\hat{m}_{i1}\left(\bar{X}_{p_{0}p_{1}}\right)\right], (3) where W_{i}(\cdot) is a twice continuously differentiable weight function for item i . \widehat{T}_{i} is the weighted average difference between the two IRCs across

Read the complete text