EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education

Guoqing Ma, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Jiawei Shen, Zilong Li, Yidan Liang, [0.5em] Zhejiang Normal University, Zhejiang, China · 2025

Read EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education by Guoqing Ma, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Jiawei Shen, Zilong Li, Yidan Liang, [0.5em] Zhejiang Normal University, Zhejiang, China (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00290.txt.

Introduction

Language models show promise for education, yet putting them in classrooms without checks risks real harm. Guoqing Ma and co-authors introduce EduEval in 2025, a benchmark for evaluating large language models in Chinese education. It is organized as a hierarchy of cognitive skills.

EduEval targets kindergarten through grade twelve education and tests models across subjects and thinking levels. Rigorous evaluation is needed because fluent answers can still be wrong, biased, or badly pitched for a child's level. Such benchmarks give schools and developers evidence for deciding where these models help and where they should stay out. Chinese classrooms give the benchmark a concrete national curriculum to test against.

Catalog record

CREATORGuoqing Ma, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Jiawei Shen, Zilong Li, Yidan Liang, [0.5em] Zhejiang Normal University, Zhejiang, China
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00290

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education Guoqing Ma, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Jiawei Shen, Zilong Li, Yidan Liang, [0.5em] Zhejiang Normal University, Zhejiang, China Abstract Large language models (LLMs) demonstrate significant potential for educational applications. However, their unscrutinized deployment poses risks to educational standards, underscoring the need for rigorous evaluation. We introduce EduEval, a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education. This benchmark makes three key contributions: (1) Cognitive Framework: We propose the EduAbility Taxonomy, which unifies Bloom’s Taxonomy and Webb’s Depth of Knowledge to organize tasks across six cognitive dimensions including Memorization, Understanding, Application, Reasoning, Creativity, and Ethics. (2) Authenticity: Our benchmark integrates real exam questions, classroom conversation, student essays, and expert-designed prompts to reflect genuine educational challenges; (3) Scale: EduEval comprises 24 distinct task types with over 11,000 questions spanning primary to high school levels. We evaluate 14 leading LLMs under both zero-shot and few-shot settings, revealing that while models perform well on factual tasks, they struggle with classroom dialogue classification and exhibit inconsistent results in creative content generation. Interestingly, several open source models outperform proprietary systems on complex educational reasoning. Few-shot prompting shows varying effectiveness across cognitive dimensions, suggesting that different educational objectives require tailored approaches. These findings provide targeted benchmarking metrics for developing LLMs specifically optimized for diverse Chinese educational tasks. The EduEval dataset and code are publicly available at https://github.com/Maerzs/E_edueval . 1 Introduction The rapid advancement of large language models (LLMs) in natural language processing (NLP) and artificial intelligence (AI) has opened new possibilities for their application in education. With strong capabilities in language understanding, reasoning, and text generation, LLMs have been widely adopted in intelligent tutoring, automated grading, and educational content generation, significantly enhancing personalized learning and instructional efficiency 9 ; 39 ; 46 . Their impact now extends from primary and secondary education to higher education and vocational training 19 ; 30 ; 40 , with great potential shown in real world teaching activities like lesson preparation assistance and interactive classroom support 30 ; 10 ; 18 ; 26 . However, evaluating LLMs in authentic educational settings remains a significant challenge 21 . The educational process is inherently dynamic and heterogeneous 31 , with diverse student cognitive profiles requiring constant adaptation 22 . Current mainstream evaluation methods rely heavily on static assessments 12 , such as standardized multiple choice questions, which fail to capture the complex reasoning, generative expression, and ethical decision making required in real world classrooms 2 ; 48 . As a result, these evaluations tend to underestimate the true educational potential of LLMs 43 . While general purpose benchmarks such as MMLU 23 , BIG-bench 43 , and HELM 34 cover broad abilities, they fall short in addressing the specific needs of educational applications. Similarly, Chinese benchmarks like E-Eval 25 , AGIEval 52 , and C-Eval 27 are still dominated by low level cognitive questions and single-turn formats, lacking systematic evaluation of higher-order abilities and providing limited diversity in task contexts and cognitive depth 36 . To bridge this gap, a robust benchmark must assess a spectrum of cognitive abilities grounded in authentic pedagogical contexts. Established frameworks like Bloom’s Taxonomy of Educational Objectives 32 and Webb’s Depth of Knowledge (DOK) model 37 provide a foundation for structuring cognitive levels, but a comprehensive evaluation requires a synthesized framework tailored specifically for the multifaceted demands of K-12 education. We introduce EduEval , a hierarchical benchmark designed to meet this need by systematically evaluating LLMs on Chinese K-12 educational tasks. EduEval prioritizes educational authenticity by incorporating tasks directly from teaching practice. Unlike traditional benchmarks, it includes classroom dialogue classification, student essay scoring, and teaching scenario design activities that reflect educators’ daily challenges. To ensure a comprehensive and authentic assessment, we developed a curriculum-balanced dataset spanning core subjects across developmental stages from primary to high school. All tasks derive from genuine educational sources, including examination archives, classroom transcripts, and expert-created materials. Our rigorous quality control employs a two-phase annotation protocol where practicing educators provide initial assessments before domain experts conduct thorough reviews, ensuring alignment with actual pedagogical standards rather than artificial benchmarking metrics. Our systematic evaluation of mainstream language models (including both general-purpose and education-specific systems) reveals a consistent performance gradient across cognitive dimensions. Models generally excel at knowledge-oriented tasks, with top performers achieving over 90% accuracy in areas like formula recall, but struggle significantly with contextual application in authentic educational scenarios, where accuracy can drop below 30% for tasks such as classroom dialogue classification. This pattern mirrors the complexity progression in human learning and highlights key areas for model improvement. By identifying these capability patterns across the EduAbility hierarchy, our benchmark provides not merely a scoring system but a developmental blueprint for creating AI systems that can meaningfully support the full spectrum of educational processes. In summary, we make three key contributions: 1) We propose the EduAbility Taxonomy, a novel and hierarchical cognitive framework that integrates Bloom’s Taxonomy and Webb’s Depth of Knowledge to systematically organize educational evaluation tasks across six cognitive dimensions. 2) We establish EduEval, a comprehensive and authentic benchmark for Chinese K-12 education , featuring over 11,000 questions across 24 task types derived from genuine classroom scenarios, including dialogue classification and essay scoring. 3) We present a holistic evaluation of 14 leading LLMs on this benchmark, providing a detailed analysis of their educational capabilities and a developmental blueprint for creating more effective AI systems for education. Figure 1: Overview of EduEval: concentric rings represent data sources, with data processing methodology (left) and educational cognitive framework hierarchy (right). 2 Related Work In recent years, large language models (LLMs) have demonstrated remarkable capabilities across various domains through mechanisms such as pre-training, supervised fine-tuning, and alignment with human or AI feedback 5 ; 13 ; 16 ; 17 . With the continuous advancement of natural language processing technology, benchmarks have expanded to encompass more complex tasks, including machine translation 8 and summarization 38 ; 24 . Comprehensive benchmarks like GLUE 45 and SuperGLUE 42 amalgamate various natural language understanding tasks, providing standards for evaluating model performance. Popular frameworks like MMLU 23 include tasks ranging from real-world exams to book knowledge, assessing ability in language understanding, common sense reasoning 6 ; 7 ; 41 , and code generation 11 ; 3 . With the burgeoning development of Chinese Large Models, an increasing number of Chinese benchmarks have emerged. CLUE 47 , an influential Chinese NLU benchmark, has been widely adopted. In addition, Chinese benchmarks similar to MMLU have surfaced, such as MMCU 51 , which encompasses medicine and education within its four main domains. AGIEval 52 focuses on standardized Chinese exams, while C-Eval 27 includes 13,948 questions on four levels of difficulty, from middle school to professional tests. M3KE 35 gathers 71 tasks from the Chinese education examination system, akin to the coverage of C-Eval. CMMLU 33 , designed for the Chinese language and cultural context, is a fully localized Chinese benchmark. These benchmarks mainly focus on low-level cognitive questions and single-turn formats, lacking systematic assessment of higher-order abilities and offering limited task context diversity and cognitive depth. EduEval actualizes this logical progression through its hierarchical cognitive framework, the EduAbility

Read the complete text