SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction
Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan · 2025
Read SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction by Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05129.txt.
Description
Alexander Scarlatos and co-authors present SMART, a method to predict question difficulty with fake students in 2025. Test makers need difficulty scores for new questions, yet field testing with real students costs a lot and cannot cover unseen items. Simulated students could fill the gap if their skill levels ring true.
The team tunes simulated students so their answer patterns match a real student model, called item response theory. Thousands of fake answers get scored by a language model, and a fit then yields difficulty scores. Tests on two real student data sets beat rival prediction methods. Good skill alignment in the fake class drove the gain.
Catalog record
| CREATOR | Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05129