EduFairBench: reproducible evaluation of large language models for educational assessment
William Villegas-Ch, Aracely Mera-Navarrete, Fernando Zúñiga-Tello, Rommel Gutiérrez · 2026
Read EduFairBench: reproducible evaluation of large language models for educational assessment by William Villegas-Ch, Aracely Mera-Navarrete, Fernando Zúñiga-Tello, Rommel Gutiérrez (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13562039.txt.
Description
William Villegas-Ch and co-authors published this evaluation study in 2026. Language models are increasingly used to grade students' open-ended answers, but headline scores hide how stable or fair those grades are. The team presents EduFairBench, a protocol that probes grading behavior from several angles at once. It combines repeated runs, majority voting, uncertainty estimates, error patterns, and a check on the feedback text the model writes.
They ran the protocol on three public benchmarks totaling 2,000 student responses and 10,000 model gradings. Agreement with human graders was moderate. Repeated runs were highly consistent, yet the same systematic errors kept appearing in neighboring score categories, showing that consistency is not the same as correctness. Feedback grew more specific and better structured on longer tasks. The authors offer the protocol as a stricter standard for judging AI graders.
Catalog record
| CREATOR | William Villegas-Ch, Aracely Mera-Navarrete, Fernando Zúñiga-Tello, Rommel Gutiérrez |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |