Automated Benchmark Generation from Domain Guidelines Informed by Bloom’s Taxonomy
Si Chen, Le Huy Khiem, Annalisa Szymanski, Ronald Metoyer, Ting Hua, Nitesh V. Chawla · 2026
Read Automated Benchmark Generation from Domain Guidelines Informed by Bloom’s Taxonomy by Si Chen, Le Huy Khiem, Annalisa Szymanski, Ronald Metoyer, Ting Hua, Nitesh V. Chawla (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20253.txt.
Description
Si Chen and five colleagues build a way to generate test benchmarks automatically from domain guidelines. Open-ended question answering tests whether a model can reason in context, not just recall facts. In practice-based fields, knowledge is procedural and rests on professional judgment, yet human exam sets for testing are often missing.
The authors use Bloom's taxonomy, a classic ladder of thinking skills, to shape the questions they generate from guidelines. A reliable generator would let hospitals, firms, and agencies test models on their own rules. That matters where buying or writing an exam by hand is too slow or costly.
Catalog record
| CREATOR | Si Chen, Le Huy Khiem, Annalisa Szymanski, Ronald Metoyer, Ting Hua, Nitesh V. Chawla |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20253