Sequential generalized kernel equating: Providing comparable scores across multiple test forms with nonequivalent groups and differently measured covariates

Michaela Vařejková, Patrícia Martinková, Eva Potužníková · 2026

Read Sequential generalized kernel equating: Providing comparable scores across multiple test forms with nonequivalent groups and differently measured covariates by Michaela Vařejková, Patrícia Martinková, Eva Potužníková (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28559.txt.

Introduction

Michaela Vařejková, Patrícia Martinková, and Eva Potužníková propose sequential generalized kernel equating for comparable scores across test forms with nonequivalent groups. When no anchor items exist, background traits bridge forms, but those traits may be measured on different forms, biasing results. Their method adjusts the trait first within the kernel framework, then adjusts scores. Tests show adjusting the trait cuts bias, above all when trait spreads differ and trait-score links are strong, and a national school-leaving exam example shows practical use.

Catalog record

CREATORMichaela Vařejková, Patrícia Martinková, Eva Potužníková
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54087
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28559

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sequential generalized kernel equating: Providing comparable scores across multiple test forms with nonequivalent groups and differently measured covariates Michaela Vařejková, Patrícia Martinková, Eva Potužníková Abstract Test equating using covariates may be applied to provide comparable scores from multiple test forms when no anchor items are available. However, its performance may be compromised if some of the covariates themselves are measured using different test forms. In this work, we propose sequential generalized kernel equating to account for possible differences in the distribution of covariates used in the NEC design. We evaluate the proposed approach through a simulation study within the kernel equating framework. Results indicate that equating the covariate reduces bias in equated test scores, particularly when the covariate distributions differ and the correlation between the covariate and the test score is strong. A real data example from a national high school leaving examination further demonstrates the practical application. Keywords: test score equating, nonequivalent groups design, kernel equating, background variables 1 Introduction Large-scale educational assessments often involve multiple test forms that are administered at different times or different places, but are intended to measure the same underlying knowledge and skills. Multiple test forms are typically needed to ensure that all test takers are assessed under the same conditions, without those tested later having an unfair advantage of knowing the content of past administrations. Other examples of alternate versions of the same test in large-scale assessment settings include different language variants, special educational needs accommodations, matrix-sampled tests, or computer adaptive tests [ 4 ] . Different test forms may also be needed to update the test items to reflect changes in the curriculum or the broader social context in which the assessment takes place [ 20 ] . Since the most common use of educational assessments is to compare the performance of testees, regardless of which specific form they took, equating scores from different test forms is necessary to make the comparison fair (American Educational Research Association et al., [ 2 ] ). Fair and accurate equating is therefore an important psychometric topic, and continued research on both the statistical tools and the computational aspects of equating remains essential [ 15 ] . A variety of test equating methods can be used, depending on the testing situation. While a common goal is to disentangle test form differences from the examinees’ differences in ability, the appropriate methods are chosen according to the data collection design [ 23 ] . Two common approaches implemented in large-scale assessment programs are the equivalent groups (EG) design which assumes that the groups of examinees taking different test forms do not differ in ability, and the nonequivalent groups with anchor test (NEAT) design, which requires the use of an anchor test, where a common set of items is administered to all test takers in addition to their specific test form to account for possible differences in ability between the groups [ 5 , 11 , 20 , 22 , 23 ] . However, in situations where the groups cannot be considered equivalent and the anchor test is not applicable or feasible, alternative methods must be used. One approach is to substitute the anchor test scores with covariates, for example, school grades or scores from other tests, and apply equating in a nonequivalent groups with covariates (NEC) design [ 25 , 24 ] . The critical assumptions are that the covariates can explain differences in ability and that the conditional distribution of test scores, given the covariates, is the same across all groups. These assumptions ensure that the covariates adequately explain the differences between groups. This equating method, however, is threatened if the relationship between test scores and covariates, or the meaning of the covariates themselves, differs between the groups. This may occur, for example, when some of the covariates are measured using multiple test forms. In such cases, the underlying assumption of the same conditional distributions may be violated, which can lead to biased equating results, as discussed in San Martín & González ( [ 19 ] ). In this work, we provide evidence via a simulation study of how violations of the assumption of population-invariant conditional distributions may bias the results of equating. We use the observed-score equating method of kernel equating [ 21 , 23 ] due to its flexibility, minimal distributional assumptions, and suitability for incorporating covariate information. We focus on the case where one of the covariates used in the equating process is measured using multiple test forms, exploring variants with similar and different test form difficulties, as well as those with stronger and weaker correlations between the covariate and the primary test score. For the NEC design when both the construct of interest and the covariates are measured using multiple test forms, we propose applying sequential equating to improve the accuracy of the equating process. The paper proceeds as follows: In Section 2, we summarize generalized kernel equating (GKE) with the NEC design and its underlying assumptions. We then propose and formalize the sequential generalized kernel equating approach. In Section 3, we present a simulation study to investigate the impact of assumption violation and to evaluate the proposed approach. Section 4 includes an empirical illustration. Finally, we provide a discussion and concluding remarks in Section 5. 2 Methods 2.1 Generalized kernel equating with the NEC design Generalized kernel equating (GKE) with the NEC design is a statistical method used to equate scores from different test forms while accounting for covariates. The kernel equating framework is flexible and general, encompassing several traditional equating methods—such as mean, linear, and equipercentile equating—as special cases [ 21 ] . The method can be described as a sequence of five steps: presmoothing, estimation of score probabilities, continuization, equating, and calculation of the standard error of equating [ 26 ] . We first introduce the notation and underlying framework that will be used throughout the article. We denote the random variables representing the total scores from two different test forms as X and Y , where both forms are designed to measure the same construct, and the vector of covariates as \boldsymbol{C} . The covariate vector \boldsymbol{C} can take values \boldsymbol{c_{l}} , l=1,\dots,L , where L represents the number of possible covariate categories or combinations of covariate values. If continuous covariates are included in \boldsymbol{C} , they are discretized into a finite number of intervals so that the number of covariate categories L remains finite. Furthermore, we consider two populations of students, populations P and Q . Generally, we assume that these populations differ in their ability. Our target population of interest T is assumed to be a mixture of these two populations: \displaystyle T=\omega P+(1-\omega)Q, (1) where 0\leq\omega\leq 1 is a constant representing the mixture weight between the populations, commonly determined based on the relative sample sizes of the populations. This mixture formulation is symbolic, representing how score distributions for T are derived. We further assume that a group of students who took test form X (with total scores represented by the random variable X ) was randomly selected from the population P , and a group of students who took test form Y (with total scores represented by the random variable Y ) was randomly selected from

Read the complete text