On the choice of using raw or demographically-corrected scores
Ignacio González-Pérez, Mats Julius Stensrud, Marco Piccininni · 2026
Read On the choice of using raw or demographically-corrected scores by Ignacio González-Pérez, Mats Julius Stensrud, Marco Piccininni (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31418.txt.
Introduction
González-Pérez and two colleagues ask when correcting test scores for age and education actually hurts. Such demographic corrections are routine in psychology, and z-score standardization is the standard tool. In cognitive screening, though, a corrected score can classify people worse than the raw score it replaced.
The team give sufficient conditions under which raw scores beat demographically corrected ones, with a plain reading of what those conditions mean. They also test the common claim that corrected scores give fairer decisions and find it wanting in the cases studied. Applied to the Mini-Mental State Examination in the OASIS-3 dataset, the results give clinicians a concrete warning: correct a screening score only when the setting meets the case for it.
Catalog record
| CREATOR | Ignacio González-Pérez, Mats Julius Stensrud, Marco Piccininni |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 152 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31418
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the choice of using raw or demographically-corrected scores Ignacio González-Pérez, Mats Julius Stensrud, Marco Piccininni Abstract . Demographic corrections are routinely performed in many disciplines, including psychology. Yet, there are ongoing debates about whether these corrections are appropriate and improve classification accuracy. Here, we focus on cognitive screening tests, and show that common demographic corrections, like the z-score standardization, can be detrimental for classification in some settings. Formally, we present sufficient conditions ensuring that raw scores outperform the demographically-corrected ones, and give a substantive interpretation of this result. We also investigate the claim that using demographically-corrected scores results in more fair decisions compared to using raw scores. We apply our results to the Mini-Mental State Examination in the OASIS-3 dataset. Keywords: cognitive screening, age-education correction, accuracy, demographic insensitivity 1. Introduction Demographic corrections are widely used in the health sciences, particularly in psychology. For example, tests assessing age-related cognitive impairment are usually corrected for age and education [ 68 , 71 , 53 ] . Such corrections are also used in the clinical definitions of cognitive impairment [ 9 ] , such as the DSM-5 definition of Mild Neurocognitive Disorder [ 2 ] and the older Mild Cognitive Impairment criteria [ 67 ] . The consequences of demographic corrections are particularly relevant to screening for cognitive impairment. Cognitive screening tests are used to decide whether an individual should be referred to further clinical assessments [ 14 , 4 , 60 ] . The validity of these tests is usually assessed by their classification accuracy, i.e., sensitivity, specificity, and area under the ROC curve, in categorizing individuals against the gold-standard clinical diagnoses of mild cognitive impairment and/or dementia [ 14 , 46 , 53 , 43 , 60 , 47 , 23 ] . Cognitive screening tests are designed to give a numerical result that summarizes the performance of the individual who undertakes the evaluation. This result, e.g., representing the number of correct answers or a rating, is commonly referred to as the “raw” test score [ 23 , 53 ] . Sometimes, a demographically-corrected score is preferred over the raw score [ 51 , 46 , 47 , 53 , 23 ] . Specifically, instead of interpreting raw scores directly, practitioners often draw conclusions by comparing the raw score to reference norms of individuals without cognitive impairment and with similar demographic characteristics [ 13 , 53 , 79 , 47 , 46 ] . In practice, this comparison is made by numerically transforming the raw score into a “corrected” version; this process is known as adjustment [ 5 , 46 , 51 ] or correction [ 61 ] . The most widespread type of demographic correction is the “z-score” correction [ 53 , 47 ] . While demographic corrections are ubiquitous in neuropsychology, empirical results suggest that demographically-corrected scores have lower classification accuracy compared to raw scores. For example, a large 1996 study of older persons found that demographic corrections did not improve classification accuracy for dementia compared to using raw scores [ 4 ] . Other empirical studies reported no benefit of demographic corrections on the classification accuracy of the Mini-Mental State Examination (MMSE) [ 43 ] and the modified 3MS [ 60 ] screening tests. In these studies, accuracy was measured with respect to clinical diagnoses. Similar results were found when assessing the prognostic accuracy of the test scores in longitudinal studies [ 31 , 71 ] , or when assessing the association between unadjusted or adjusted criteria and dementia biomarkers [ 29 ] . Some researchers have advocated the claim that the suboptimal accuracy of demographically-corrected scores can be explained by the fact that demographic characteristics influence not only the raw scores but also the probability of the condition of interest (e.g., cognitive impairment). Already in 1986, [ 5 ] used this argument to question the use of correction for age and education in cognitive screening. A decade later, [ 79 ] claimed that since age and education were both risk factors for dementia, eliminating their effect on the raw test scores resulted in lower classification accuracy of the test, which was later supported by simulations and real data analyses [ 61 ] . More recently, it has been shown that, under a simplified causal model and strong parametric assumptions, corrected scores have a lower area under the ROC curve compared to raw scores, if age and education have sufficiently strong effects on both the test performance and the risk of having cognitive impairment [ 69 ] . Simulations have suggested that using raw scores leads to higher marginal sensitivity (or specificity) for the same specificity (or sensitivity) level, while using demographically-corrected scores leads to more homogeneous performance across demographic groups [ 94 ] . These empirical and theoretical results have, however, been met with scepticism by practitioners. Some have questioned whether it is true that raw scores have better classification accuracy compared to corrected scores, because of the small differences found in practice and the inevitable uncertainty of finite sample inference [ 37 , 29 ] . Also, the existing simulation results have been questioned due to their strong assumptions, and the fact that there is no guarantee that the simulated data are realistic [ 37 ] . Finally, some researchers have informally argued that corrected scores should be preferred over raw scores even if their classification accuracy is lower, because in some sense they are more “fair” and avoid overdiagnosis in vulnerable demographic groups [ 37 , 1 ] . The aim of this work is to bring clarity to this debate. We will show, without relying on restrictive data generating models, parametric assumptions, or empirical results, that it is possible for raw scores to have better classification accuracy compared to corrected scores. Specifically, we will present sufficient conditions for the superiority in terms of classification accuracy of the raw scores over the corrected ones, and give an interpretation of this result. In addition, we will engage with the claims that corrected scores are more fair. In this context, we formalize the notion of a score being insensitive to demographics, and assess whether demographically-corrected scores fulfil this property. Finally, we apply our results to the OASIS-3 dataset [ 45 ] . In this dataset, we find evidence for the superiority of raw MMSE scores in terms of classification accuracy and that age-corrected scores are not insensitive to age amongst the cognitively impaired. 2. Comparison of classification accuracy 2.1. Notation and setup Consider a screening setting designed to assess whether an individual has a condition of interest ( D=1 ) or not ( D=0 ). As an example, let D indicate the presence of cognitive impairment, for example according to a clinical examination or some biomarkers. Let X indicate the raw score resulting from the administration of a cognitive screening test, for example the MMSE [ 18 ] , with support on a subset of the real line, \mathcal{X}\subseteq\mathbb{R} . The variable V represents the demographic characteristics of the individual. To simplify the exposition, assume that V is a univariate covariate supported on (a subset of) the real line, \mathcal{V} . For example, V could represent age, education, sex, or age-education stratum. We present our results