Leveraging Large Language Models for Identifying Knowledge Components
Canwen Wang, Jionghao Lin, Kenneth R. Koedinger · 2025
Read Leveraging Large Language Models for Identifying Knowledge Components by Canwen Wang, Jionghao Lin, Kenneth R. Koedinger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09935.txt.
Description
Canwen Wang, Jionghao Lin, and Kenneth R. Koedinger explore whether large language models can identify knowledge components, the named skills behind test questions that adaptive learning systems depend on. Experts normally build these labels by hand, which is slow, and earlier automated attempts used tiny datasets and produced redundant labels. The team scaled a simulated textbook prompting strategy to 646 multiple-choice questions.
The raw model output performed worse than an expert-designed model and invented 569 components where experts used 101. The fix was merging labels whose meanings were highly similar: at the best threshold, the count fell to 428 and prediction error improved clearly. Generation alone is not enough, they conclude, but generation plus semantic merging is a viable path.
Catalog record
| CREATOR | Canwen Wang, Jionghao Lin, Kenneth R. Koedinger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 8 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-09935