Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches

Yuhang Zhou Xutian Chen Yixin Cao Yuchen Ni Yu He, Siyu Tian Xiang Liu Jian Zhang Chuanjun Ji Guangnan Ye Xipeng Qiu · 2025

Read Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches by Yuhang Zhou Xutian Chen Yixin Cao Yuchen Ni Yu He, Siyu Tian Xiang Liu Jian Zhang Chuanjun Ji Guangnan Ye Xipeng Qiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12259.txt.

Introduction

Yuhang Zhou and co-authors propose judging a language model by how well it teaches. Testing models head-on with fixed benchmarks now suffers from leaked data and narrow skills. Their method, Teach2Eval, takes a cue from a famous study trick: you learn by teaching.

In their setup, a strong model must teach weaker student models to do tasks well. The teacher's feedback turns open tasks into standard multiple-choice items. How much the students improve becomes the teacher's score. Grading teaching taps many skills at once, from clarity to error spotting. It also resists memorization, since the items are made fresh. The team argues this mirror gives a fairer, wider view of a model's mind. Their framework can be run at scale by machine.

Catalog record

CREATORYuhang Zhou Xutian Chen Yixin Cao Yuchen Ni Yu He, Siyu Tian Xiang Liu Jian Zhang Chuanjun Ji Guangnan Ye Xipeng Qiu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12259

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches Yuhang Zhou Xutian Chen Yixin Cao Yuchen Ni Yu He, Siyu Tian Xiang Liu Jian Zhang Chuanjun Ji Guangnan Ye Xipeng Qiu Abstract Recent progress in large language models (LLMs) has outpaced the development of effective evaluation methods. Traditional benchmarks rely on task-specific metrics and static datasets, which often suffer from fairness issues, limited scalability, and contamination risks. In this paper, we introduce Teach2Eval , an indirect evaluation framework inspired by the Feynman Technique. Instead of directly testing LLMs on predefined tasks, our method evaluates a model’s multiple abilities to teach weaker "student" models to perform tasks effectively. By converting open-ended tasks into standardized multiple-choice questions (MCQs) through teacher-generated feedback, Teach2Eval enables scalable, automated, and multi-dimensional assessment. Our approach not only avoids data leakage and memorization but also captures a broad range of cognitive abilities that are orthogonal to current benchmarks. Experimental results across 26 leading LLMs show strong alignment with existing human and model-based dynamic rankings, while offering additional interpretability for training guidance. 1 1 footnotetext: Contribute equally to this work. 2 2 footnotetext: The corresponding authors:Yixin Cao([email protected]), Guangnan Ye([email protected]) 1 Introduction Recent advances in large language models (LLMs) have significantly expanded their capabilities, enabling them to tackle a diverse range of general-purpose tasks beyond traditional natural language processing (NLP) applications OpenAI (2023) ; Touvron et al. (2023) . However, the progress in model evaluation has not kept pace. Evaluation has primarily relied on task-specific performance metrics, such as those from GSM8K Cobbe et al. (2021) , MATH Hendrycks et al. (2021) , MMLU Hendrycks et al. (2020) , and BigBench Srivastava et al. (2022) . These datasets are often prone to various fairness issues Wei et al. (2023) ; Zhou et al. (2023) , e.g., the testing data covers sufficient capabilities of interest in practice? how to fairly judge the open-ended responses? Figure 1: Comparing two evaluation methods. The left shows the use of static benchmarks to directly evaluate LLM, which may lead to data contamination and result in artificially high scores; On the right is Teach2Eval, which generates guidance to enhance the ability of the weak model as an indirect metric for LLM evaluation. In this paper, we propose Teach2Eval, an indirect evaluation method that targets scalable, quantifiable, and multi-dimensional evaluation of LLMs. Inspired by the Feynman Technique Feynman (2018) , which states "explaining or teaching concepts in a simple, concise manner deepens one’s understanding of the material", we introduce this principle as a tool for evaluating abilities beyond learning strategy. Unlike traditional evaluation methods, Teach2Eval does not directly assess LLMs through performance metrics on given datasets. Instead, we leverages the target to teach weak "student" models to complete tasks, as shown in Figure 1 , and the overall performance gains of students shall reflect multi-dimensional capabilities of the teacher. That is, better students, better teacher LLMs. There are several benefits of the above idea, which guides the design of such an indirect method. First, it transforms open-ended questions into multi-choice questions (MCQs) while preserving task difficulty, enabling more standardized and scalable assessment. As LLMs encounter increasing open-ended queries, traditional evaluation methods face challenges due to reliance on costly human judgment or inconsistent automated metrics Gu et al. (2024) ; Li et al. (2024) . While direct conversion to MCQs simplifies evaluation, it often fails to capture the generative capabilities of LLMs. In contrast, Teach2Eval leverages LLMs to generate feedback that guides weaker models to answer in MCQ format. We construct a benchmark of 60 datasets across four categories—Knowledge, Reasoning, Understanding, and Multilingual—and standardize them as MCQs with added misleading options derived from original answers. Evaluation is based on student model performance gains after teacher-guided learning. Importantly, the teacher model receives no access to MCQ choices, ensuring that evaluation reflects student accuracy rather than exposure to answer options. This approach enables cost-effective, fully automated evaluation for open-ended tasks. The second benefit of indirect evaluation is the involvement of comprehensive abilities. Traditional benchmarks usually focus on one or several targeted task-specific capabilities through the design of questions Cao et al. (2025) . While, Teach2Eval does not rely on the questions themselves (orthogonal to existing benchmarks). It grounds the evaluation in practice, and the teaching process naturally involves several practical abilities. We categorize them into four levels based on Bloom’s taxonomy Krathwohl (2002) : Application, Judgment, Guidance, and Reflection. Based on this, we design a multi-dimensional metric, and also prove that such a metric well supports the claims above. The third benefit is addressing the problem of data contamination. This interactive evaluation mechanism introduces dynamic randomness, and requires models to possess a deeper understanding of the knowledge in order to effectively teach the student model, memorizing answers becomes ineffective under this setup. To validate this, we designed a series of experiments based on the multi-dimensional cognitive abilities, demonstrating that Teach2Eval not only mitigates data contamination issue, but also provides fine-grained guidance for model training and development. To evaluate this approach, we test 26 latest LLMs and the results show our method’s results align closely with platforms like Chatbot Arena Chiang et al. (2024) , LiveBench White et al. (2024) , and GPT-as-a-Judge, with similarities of 0.91, 0.92, and 0.88, respectively. Note that our proposed evaluation is automated and cost-effective.Also, we evaluate the convergence and robustness of this method based on a random combination of student models and longer interaction turns. We present evidence and advantages of using this evaluation method to guide model training in the final, for instance, by analyzing the evolution of the two capabilities during model training, our method provides clear directions for preventing model overfitting. We make the following contributions: • We propose a novel, general indirect evaluation method Teach2Eval, which effectively addresses multiple issues of current evaluation approaches. We have open-sourced our code and data at https://github.com/zhiqix/Teach2Eval. • We define the metric based on students’ improvements, and demonstrates its effectiveness in reflecting several cognitive abilities of the teacher LLMs, these abilities are orthogonal to the current benchmarks. • Our approach not only provides an effective overall ranking of models at low cost, but also enables fine-grained capability analysis to guide model training and refinement. 2 Related Work 2.1 Benchmarks The most prevalent benchmarks for evaluating models are static, groundtruth-based ones, such as MCQs or QA tasks with predefined answers. These benchmarks can be categorized according to the 3H criteria Bai et al. (2022) , with a focus on helpful benchmarks. They cover a variety of topics, including Understanding, Knowledge, Reasoning, and Multilingual tasks, as illustrated in Figure 11 of Appendix B . Examples include MMLU Hendrycks et al. (2020) , GSM-8K Hendrycks et al. (2021) , BigBench Srivastava et al. (2022) , AGIEval Zhong et al. (2023) , and HellaSwag Zellers et al. (2019) . However, static benchmarks have limitations like contamination, saturation, overfitting, and a lack of human alignment Yang et al. (2023) ; Oren et al. (2023) . DynaBench Kiela et al. (2021) addressed these issues by introducing real-time benchmarks that incorporate human input. Our system adopts a similar approach, using perturbations from weak models to increase task difficulty, which can be extended to a variety of benchmarks. Figure 2:

Read the complete text