SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction

Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan · 2025

Read SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction by Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05129.txt.

Description

Alexander Scarlatos and co-authors present SMART, a method to predict question difficulty with fake students in 2025. Test makers need difficulty scores for new questions, yet field testing with real students costs a lot and cannot cover unseen items. Simulated students could fill the gap if their skill levels ring true.

The team tunes simulated students so their answer patterns match a real student model, called item response theory. Thousands of fake answers get scored by a language model, and a fit then yields difficulty scores. Tests on two real student data sets beat rival prediction methods. Good skill alignment in the fake class drove the gain.

Catalog record

CREATORAlexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05129

Browse the full library · About the Tower of Babel