Leveraging Large Language Models for Identifying Knowledge Components

Canwen Wang, Jionghao Lin, Kenneth R. Koedinger · 2025

Read Leveraging Large Language Models for Identifying Knowledge Components by Canwen Wang, Jionghao Lin, Kenneth R. Koedinger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09935.txt.

Description

Canwen Wang, Jionghao Lin, and Kenneth R. Koedinger explore whether large language models can identify knowledge components, the named skills behind test questions that adaptive learning systems depend on. Experts normally build these labels by hand, which is slow, and earlier automated attempts used tiny datasets and produced redundant labels. The team scaled a simulated textbook prompting strategy to 646 multiple-choice questions.

The raw model output performed worse than an expert-designed model and invented 569 components where experts used 101. The fix was merging labels whose meanings were highly similar: at the best threshold, the count fell to 428 and prediction error improved clearly. Generation alone is not enough, they conclude, but generation plus semantic merging is a viable path.

Catalog record

CREATORCanwen Wang, Jionghao Lin, Kenneth R. Koedinger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE8 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-09935

Browse the full library · About the Tower of Babel