TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models

Zheng Li, Siyao Song, Jingyuan Ma, Rui Li, Ying Zeng, Minghao Li, Zhifang Sui, Peking University ByteDance BandAI, Institute of Automation, Chinese Academy of Sciences [email protected]; [email protected]; [email protected] · 2026

Read TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models by Zheng Li, Siyao Song, Jingyuan Ma, Rui Li, Ying Zeng, Minghao Li, Zhifang Sui, Peking University ByteDance BandAI, Institute of Automation, Chinese Academy of Sciences [email protected]; [email protected]; [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21375.txt.

Introduction

Zheng Li and co-authors build a test of teaching skill for language models. Models are often used as study helpers, yet most tests grade them on solving problems, not on teaching. A model can know an answer and still fail to help a student learn it.

Their framework, TeachBench, is grounded in real course syllabi. It grades a model by how much a student's results improve after several rounds of instruction. That shifts the target from right answers to real learning gains. The benchmark gives schools and builders a clearer way to judge AI tutors.

Catalog record

CREATORZheng Li, Siyao Song, Jingyuan Ma, Rui Li, Ying Zeng, Minghao Li, Zhifang Sui, Peking University ByteDance BandAI, Institute of Automation, Chinese Academy of Sciences [email protected]; [email protected]; [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21375

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models Zheng Li, Siyao Song, Jingyuan Ma, Rui Li, Ying Zeng, Minghao Li, Zhifang Sui, Peking University ByteDance BandAI, Institute of Automation, Chinese Academy of Sciences [email protected]; [email protected]; [email protected] Abstract Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving knowledge-centered teaching underexplored. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via student performance improvement after multi-turn instruction. By restricting teacher agents to structured knowledge points and example problems, the framework avoids information leakage and enables reuse of existing benchmarks. We instantiate the framework on Gaokao data across multiple subjects. Experiments reveal substantial variation in teaching effectiveness across models and domains: some models perform well in mathematics, while teaching remains challenging in physics and chemistry. We also find that incorporating example problems does not necessarily improve teaching, as models often shift toward example-specific error correction. Overall, our results highlight teaching ability as a distinct and measurable dimension of LLM behavior. 1 Introduction Large language models (LLMs) have demonstrated strong performance across a wide range of reasoning and problem-solving tasks. These advances have made LLMs promising candidates for AI-powered teaching assistants and automated educational systems. Recent work has begun to explore the use of LLMs in educational settings, such as interactive tutoring and learning support Nye et al. 2023 ; Liu et al. 2024b ; Patil et al. 2024 . However, despite increasing interest in these applications, systematic benchmarks for evaluating the teaching capability of LLMs remain largely underdeveloped. Several recent benchmarks have begun to explore educational scenarios. For example, datasets such as EducationQ Shi et al. 2025 and Teach2Eval Zhou et al. 2025 evaluate whether an LLM can guide a student toward solving a specific target problem. While valuable, these benchmarks primarily focus on problem-level guidance rather than teaching ability in a broader sense. In particular, whether a model can effectively act as a teacher—helping students learn underlying knowledge points and acquire problem-solving strategies—remains insufficiently evaluated. Moreover, directly teaching target questions introduces the risk of information leakage, which can invalidate evaluation results. In this work, we propose a teaching-centered evaluation framework that explicitly measures the instructional effectiveness of LLMs acting as teacher agents. Our framework is built around structured knowledge points derived from a syllabus, rather than target questions themselves. A teacher agent conducts multi-turn instruction based on these knowledge points, while a fixed-capability student agent serves as a proxy for a human learner. Teaching effectiveness is assessed by the student’s performance improvement after instruction, enabling controlled and fair comparison across different teacher agents. To support this evaluation, we construct a knowledge structure tree from exam syllabi and annotate each question with fine-grained knowledge paths. We further generate example problems of varying difficulty for each knowledge point, allowing us to study different teaching strategies under a unified protocol. Using data from the Chinese National College Entrance Examination (Gaokao) across multiple subjects, we conduct extensive experiments to evaluate teaching effectiveness under different interaction settings. Our experimental results reveal several key findings. First, among the evaluated models, Qwen3-235B-A22B-Instruct demonstrates strong teaching effectiveness, achieving a 7.63-point improvement in the mathematics domain. Second, teaching effectiveness varies substantially across domains: models perform better in subjects where knowledge points can be directly applied (e.g., mathematics, history, and politics), while teaching remains more challenging in domains requiring deeper integration of knowledge with complex problem contexts (e.g., physics and chemistry). Finally, we find that incorporating example problems does not necessarily enhance teaching effectiveness. Instead, current LLMs tend to shift from syllabus-grounded instruction to example-based error correction, which fragments the teaching process and ultimately weakens overall instructional quality. Together, these results highlight both the potential and limitations of current LLMs as teachers. More importantly, they show that teaching ability is a distinct and measurable dimension of LLM behavior, separate from raw problem-solving performance. We hope our framework provides a foundation for future research on LLM pedagogy and the development of models that can teach more effectively. Our main contributions are summarized as follows: • We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via post-instruction student performance, while preventing information leakage by restricting teachers to structured knowledge points and enabling reuse of existing benchmarks. • We construct a knowledge-structured benchmark based on Gaokao syllabi and questions, including fine-grained knowledge annotation and controlled Student–Teacher interactions across multiple subjects. • Through extensive experiments, we provide a systematic analysis of LLM teaching behaviors, revealing both effective teaching patterns and key limitations, particularly in example-based instruction. 2 Related Work Answer-centric benchmarks for knowledge and reasoning. Mainstream LLM evaluation has predominantly focused on answer correctness under static test settings, measuring knowledge coverage and reasoning skill rather than instructional competence. Representative suites include MMLU for broad academic/professional knowledge Hendrycks et al. 2020 , HELM for standardized multi-scenario and multi-metric reporting Liang et al. 2022 , and math-focused benchmarks such as GSM8K and MathBench for multi-step quantitative reasoning and hierarchical proficiency assessment Cobbe et al. 2021 ; Liu et al. 2024a . While these benchmarks are invaluable for characterizing “solver” capability, they do not operationalize whether a model can act as a teacher who improves a learner’s mastery. TechBench addresses this gap by shifting the evaluation target from solving to teaching: the model is assessed by its ability to organize instruction around a syllabus-derived knowledge structure and to induce measurable learning gains under controlled instructional inputs. Real-exam and Chinese-context evaluation. To increase ecological validity, several benchmarks evaluate LLMs using standardized exams and Chinese-context tests. AGIEval adopts a human-exam-centric design spanning multiple official exams Zhong et al. 2024 ; C-Eval and CMMLU provide large-scale Chinese multi-discipline multiple-choice suites across difficulty levels Huang et al. 2023 ; Li et al. 2024 ; and GAOKAO-Bench directly evaluates LLM performance on Gaokao questions, including subjective formats with grading considerations Zhang et al. 2023 . Despite their realism, these efforts remain largely solution-centric : they quantify how well a model answers exam questions, but not how well it can teach the underlying knowledge without being shown the test item. TechBench differs by explicitly instantiating a Gaokao-like teaching scenario: the teacher agent is not given the target question and must teach solely from knowledge points plus curated example problems, and success is defined by post-instruction improvement on held-out tests rather than raw exam scoring alone. Figure 1: Overview of the proposed LLM teaching evaluation framework. The workflow consists of three stages: (1) Knowledge Structuring , which transforms the raw syllabus and questions into a structured knowledge tree with associated exercises; (2) Teaching Loop , which simulates multi-turn, dialogue-based interactions and instruction between the target LLM acting as the teacher agent and a student agent; and (3) Evaluation , which quantifies student performance improvement by comparing results before and after the interaction, yielding a final teaching ability score. Tutoring datasets and pedagogy-oriented benchmarks. Recent work has begun to formalize pedagogical behaviors through tutoring dialogues and rubric-based evaluation. MathDial provides teacher–student tutoring dialogues grounded in multi-step math problems with pedagogical properties Macina et al. 2023 . The BEA shared tasks benchmark the generation and assessment of teacher responses in educational dialogues, highlighting

Read the complete text