Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis

Zhiang Dong, Jingyuan Chen, Fei Wu * · 2025

Read Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis by Zhiang Dong, Jingyuan Chen, Fei Wu * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05556.txt.

Introduction

Zhiang Dong, Jingyuan Chen, and Fei Wu bring language model knowledge into student diagnosis in a 2025 paper. Schools use cognitive diagnosis to infer what each student knows from exercise results. Rare students and rare exercises defeat these models because history is thin.

Large language models carry broad subject knowledge that could fill the gap. Mixing that knowledge with fine-grained answer patterns is the hard part. Their framework first lets the language model form its own diagnosis of students and items. A second stage aligns those judgments with the diagnosis model's number space. The design plugs into several existing diagnosis models unchanged. The goal is fair, accurate diagnosis even for students with only a handful of answers.

Catalog record

CREATORZhiang Dong, Jingyuan Chen, Fei Wu *
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05556

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis Zhiang Dong, Jingyuan Chen, Fei Wu * Abstract Cognitive Diagnosis Models (CDMs) are designed to assess students’ cognitive states by analyzing their performance across a series of exercises. However, existing CDMs often struggle with diagnosing infrequent students and exercises due to a lack of rich prior knowledge. With the advancement in large language models (LLMs), which possess extensive domain knowledge, their integration into cognitive diagnosis presents a promising opportunity. Despite this potential, integrating LLMs with CDMs poses significant challenges. LLMs are not well-suited for capturing the fine-grained collaborative interactions between students and exercises, and the disparity between the semantic space of LLMs and the behavioral space of CDMs hinders effective integration. To address these issues, we propose a novel K nowledge-enhanced C ognitive D iagnosis (KCD) framework, which is a model-agnostic framework utilizing LLMs to enhance CDMs and compatible with various CDM architectures. The KCD framework operates in two stages: LLM Diagnosis and Cognitive Level Alignment. In the LLM Diagnosis stage, both students and exercises are diagnosed to achieve comprehensive and detailed modeling. In the Cognitive Level Alignment stage, we bridge the gap between the CDMs’ behavioral space and the LLMs’ semantic space using contrastive learning and mask-reconstruction approaches. Experiments on several real-world datasets demonstrate the effectiveness of our proposed framework. Zhejiang University {dongza,jingyuanchen,wufei}@zju.edu.cn Introduction Cognitive diagnosis evaluates a student’s learning proficiency through his responses to a series of exercises, as shown in Figure 1 (a), which plays a fundamental role in intelligent education systems. The outcomes of cognitive diagnosis are crucial for various educational applications, such as educational recommendation ( Huang et al. 2019 ) and computerized adaptive testing ( Bi et al. 2020 ; Zhuang et al. 2022 ) . Consequently, the accuracy and reliability of cognitive diagnosis are essential for enhancing the effectiveness of these educational technologies. Figure 1: (a) An illustration of cognitive diagnosis. (b) Performance of warm and cold scenarios of NCD on PTADisc, exhibiting the limitations in cold scenario. Traditional cognitive diagnosis models (CDMs) are primarily grounded in psychometric theories, employing manually designed interaction functions inspired by principles from both psychometrics and educational theory. Examples include DINA ( De La Torre 2009 ) and MIRT ( Reckase 2009 ) models. Recent advancements in deep learning have enabled the development of innovative CDMs that leverage neural networks to model complex collaborative information ( i.e. , student-exercise interactions), thereby improving diagnostic accuracy and adaptability ( Wang et al. 2020 ; Gao et al. 2021 ) . However, these existing CDMs face significant challenges in diagnosing infrequent students and exercises, commonly referred to as the cold-start problem. This limitation arises primarily from the lack of prior knowledge within these models, which impairs their adaptability to unfamiliar students and exercises. As depicted in Figure 1 (b), experiments conducted on the PTADisc dataset ( Hu et al. 2023 ) indicate that existing CDMs exhibit poor performance in cold scenarios, thereby undermining overall diagnostic accuracy. Large language models (LLMs) have seen rapid advancements, showcasing remarkable capabilities in logical reasoning and text comprehension. Their success across various domains highlights the feasibility of this approach ( Wang et al. 2024a ; Xu, Zhang, and Qin 2024 ; Abbasiantaeb et al. 2024 ; Zhu, Huang, and Sang 2024 ; Zhang et al. 2024 ) . The extensive prior knowledge embedded in LLMs presents a promising solution for addressing the limitations of existing CDMs. Specifically, LLMs can leverage their understanding of concepts and relationships between different knowledge domains to provide insights into student learning behaviors and exercise characteristics. By incorporating this extensive prior knowledge, LLMs can simulate the reasoning of experienced human teachers, offering more accurate diagnoses in cold scenarios. Therefore, our objective is to effectively integrate LLMs with CDMs to enhance diagnostic performance. However, this integration is non-trivial due to several key factors. Firstly, LLMs are limited in their ability to model the fine-grained collaborative information crucial for understanding student-exercise interactions, as their input length constraints limit the inclusion of detailed textual information about relationships between exercises, knowledge concepts, and students. Furthermore, LLMs and CDMs operate in distinct representation spaces: LLMs process text-based data within a semantic space, whereas CDMs analyze student behavior within a behavioral space derived from interactions. Successful integration necessitates bridging the gap between these semantic and behavioral spaces. To address these challenges, we propose a novel Knowledge-enhanced Cognitive Diagnosis (KCD) framework that seamlessly integrates LLMs to enhance existing CDMs, aligning the semantic space of LLMs with the behavioral space of CDMs. The proposed KCD comprises two primary modules: LLM diagnosis and cognitive level alignment . The LLM diagnosis module leverages the capabilities of LLMs to simulate experienced human educators in diagnosing students’ learning status and the attributes of exercises, thereby enriching the prior knowledge of conventional CDMs. Specifically, during LLM diagnosis, collaborative information regarding students and exercises is gathered via LLMs, followed by an analysis of students’ response logs from both educational and psychological perspectives to generate textual diagnoses of students and exercises, revealing their cognitive status and attributes. Subsequently, the cognitive level alignment module aligns these textual diagnoses from the semantic space of LLMs with the behavioral representations of CDMs, resulting in more accurate cognitive representations of students. The contributions of this work are summarized as: • We propose the KCD framework, which is model-agnostic and leverages the combined strengths of LLMs and CDMs to achieve optimal diagnostic results. • We introduce the LLM diagnosis module that combines collaborative information and response logs to generate textual diagnoses of students and exercises. • We introduce the cognitive level alignment module, aligning the textual diagnoses from LLMs with behavioral representations from CDMs. • Experiments on several public datasets with different CDMs demonstrate the effectiveness of our framework. Our code and datasets are available at https://github.com/PlayerDza/KCD . Related Work Cognitive Diagnosis Cognitive diagnosis, which originated from educational psychology, is a fundamental task in the field of intelligent education. It characterizes students’ learning status and knowledge proficiency based on their responses to various questions ( Liu 2021 ) . Existing cognitive diagnosis methods are mainly divided into two main categories: psychometric theory-based methods ( Lord 1952 ; De La Torre 2009 ; Reckase 2009 ) and neural network-based methods ( Wang et al. 2020 ; Gao et al. 2021 ; Bi et al. 2023 ; Liu et al. 2024a ; Wang et al. 2023 ) . Psychometric theory-based methods, such as Item Response Theory (IRT) ( Lord 1952 ) , Multidimensional IRT (MIRT) ( Reckase 2009 ) , and Deterministic Inputs, Noisy And gate model (DINA) ( De La Torre 2009 ) , are designed to evaluate students’ proficiency through latent factors utilizing psychological theories. Neural network-based methods use deep neural networks to profile students’ learning status. NCD ( Wang et al. 2020 ) first incorporates neural networks into cognitive diagnosis to effectively capture the fine-grained student-exercise relationships. RCD ( Gao et al. 2021 ) and RDGT ( Yu et al. 2024 ) employ graph architectures to explore the relationships among exercises, knowledge concepts, and students. Recently, BETA-CD ( Bi et al. 2023 ) developed a reliable and rapidly adaptable cognitive diagnosis framework for new students through meta-learning. ACD ( Wang

Read the complete text