AI as a partner in assessment: generating situational judgment tests with large language models
Liming Jiang, Fang Luo, Xuetao Tian · 2025
Read AI as a partner in assessment: generating situational judgment tests with large language models by Liming Jiang, Fang Luo, Xuetao Tian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-12661831.txt.
Description
Liming Jiang, Fang Luo, and Xuetao Tian test whether language models can write good judgment tests. Their 2025 study uses situational judgment tests, which present a scenario and grade the response, a staple of ability assessment. The test case is emotional regulation.
Working with a leading Chinese model, the team tried two routes: inventing fresh items, and adapting items from existing tests. Experts rated the output, and nearly three hundred people took the tests. Fresh items matched or beat the quality of the existing bank on both expert and trial measures. Adapted items showed some flaws. The authors say the work proves models can serve as partners in building sound assessments.
Catalog record
| CREATOR | Liming Jiang, Fang Luo, Xuetao Tian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |