Is your multimodal large language model a good science tutor?

Ming Liu, Liwen Wang, Wensheng Zhang · 2025

Read Is your multimodal large language model a good science tutor? by Ming Liu, Liwen Wang, Wensheng Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06418.txt.

Introduction

Ming Liu and co-authors asked whether AI models can actually teach science in two thousand twenty-five. Answer accuracy alone misses teaching skill. A good tutor explains, checks, and adapts. Their framework grades tutoring with an education rubric. A simulated student judges each tutor's performance. Strong and weak tutors emerged from the scores.

Pairs of strong and weak outputs formed training data. Preference tuning then taught models to teach better. The source questions came from a science benchmark. The method needs no human labelers at scale. Rubric scores expose where tutors fail students. Explanation quality finally counts alongside correctness. Schools vetting AI tutors gain a yardstick.

Catalog record

CREATORMing Liu, Liwen Wang, Wensheng Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06418

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Is your multimodal large language model a good science tutor? Ming Liu, Liwen Wang, Wensheng Zhang Abstract Multimodal large language models (MLLMs) demonstrate impressive performance on scientific reasoning tasks (e.g., ScienceQA ). However, most existing benchmarks focus narrowly on the accuracy of the final answer while ignoring other metrics. In particular, when applying MLLMs to educational contexts, the goal is not only correctness but also the ability to teach . In this paper, we propose a framework that evaluates MLLM as science tutors using a comprehensive educational rubric and a simulated student model that judges the teaching performance of the tutors. Given a list of candidate MLLM science tutors, we use rubric-based student judgments to produce a range of tutor performance scores, identifying both strong and weak tutors. Using the training section of the ScienceQA dataset, we then construct a data set of pairwise comparisons between the outputs of strong and weak tutors. This enables us to apply multiple preference optimization methods to fine-tune an underperforming tutor model (Qwen2-VL-2B) into more effective ones. Our results also show that strong problem-solving skills do not guarantee high-quality tutoring and that performance optimization-guided refinements can yield more educationally aligned tutor models. This approach opens avenues for building MLLMs that serve not only as problem solvers, but as genuinely helpful educational assistants. 1 Introduction Multimodal Large Language Models (MLLM) are capable of understanding visual and textual content Li et al. (2024a) ; Wang et al. (2024) ; OpenAI (2024) . Models such as the LLaVA series Liu et al. (2023) ; Liu et al. (2024a) achieve impressive performance on various benchmarks Lu et al. (2022a) ; Yue et al. (2024) , demonstrating reasoning skills across scientific domains. However, current evaluations Lu et al. (2024) ; Lu et al. (2022a) only emphasize the correctness and complexity of the reasoning. In the educational domain, teaching quality —the ability to guide a student through conceptual understanding—is equally or more important than answering questions accurately. Figure 1: In our framework, we adopt an MLLM as tutor and another MLLM as student. Given a science question with visual input, the tutor is asked to give tutorial to help the student better understand the question and provide encouragement as well. As long as the student receives the tutorial, a feedback will be given in the form of a review number along with a remark. Effective tutoring involves more than providing a correct final answer. According to established educational and cognitive psychological principles, a qualified tutor breaks complex topics into comprehensible steps, adapts the explanations to the level of knowledge of the learner, and encourages the learner to reflect and reason independently Aleven and Koedinger (2002) ; Koedinger and Aleven (2007) ; Agarwal and Roediger (2018) ; Anderson et al. (1995) ; Aleven et al. (2006) . However, current MLLM benchmarks Lu et al. (2022a) ; Lu et al. (2024) ; Zhang et al. (2024) ; Lu et al. (2021) ; Fu et al. (2024) ; Yue et al. (2024) rarely measure these qualities, focusing solely on measuring task performance. To address this gap, we propose: 1. Rubric-based Tutoring Evaluation: We develop a rubric based on well-known educational principles to evaluate MLLMs as science tutors . Rather than judging correctness alone, we assess qualities such as conceptual clarity, scaffolding, and encouragement of reasoning. 2. Student-as-Judge Model: We introduce a simulated student model that receives tutorials from MLLM tutor candidates. This student, prompted to apply our tutoring rubrics, provides feedback on how well each tutor’s tutorials support learning. 3. From Evaluation to Improvement with DPO: After evaluating a list of tutors in the test section of ScienceQA and collecting rubric-based judgments from the student model, we obtain a comprehensive tutoring performance score for each tutor. We identify tutors who produce high tutoring scores (“strong tutors”) and those with low scores (“weak tutors”). By forming pairwise comparisons between their tutorials in the training section of ScienceQA , we create a data set suitable for preference optimization methods, such as Direct Preference Optimization (DPO) Amini et al. (2024) . Using these methods in the new data set, we transform a weak tutor model Qwen-VL-2B into a more educationally-aligned model, effectively distilling the “teaching capability” from a stronger tutor model. Our approach is based on recent progress in preference-based alignment methods Amini et al. (2024) ; Meng et al. (2024) ; Hong et al. (2024a) . Although existing work on alignment focuses mainly on scalar rewards (such as helpfulness or harmlessness), we incorporate educational criteria into the preferences dataset, thus aligning the model with educational principles rather than mere correctness. In summary, our contributions are as follows. • We developed a framework to assess the tutoring performance of MLLMs using a rubric following educational principles, implemented through a simulated student model that provides judgments on conceptual clarity, scaffolding, and encouragement. • We apply these rubric-based judgments to multiple candidate tutors, and produce a ranked set of tutoring capabilities. We then select top-performing and bottom-performing tutors to construct a preference data set for science tutoring. • We show that by applying preference optimization methods to this data set, we can significantly improve a weak tutor’s tutoring capacity and educational alignment. Our work pave the way for the building of MLLMs that solve problems accurately and function as reliable science tutors, ultimately contributing to the broader societal goal of using machine learning to benefit society. 2 Related Work Multimodal Large Langauge Model Research on language models has expanded rapidly in recent years, driven by advances in architectures, scaling, and training techniques Touvron et al. (2023) ; Kaplan et al. (2020) ; Ouyang et al. (2022) . Multimodal language models, which integrate textual and visual information, have demonstrated state-of-the-art performance in tasks ranging from image captioning and visual question answering to problem solving Li et al. (2024a) ; Wang et al. (2024) ; Hong et al. (2024b) ; Microsoft (2024) ; Bai et al. (2023) ; Meta AI (2024) ; Chen et al. (2024b) ; OpenAI (2024) ; OpenAI (2025) . However, these models are often assessed on the basis of correctness and raw reasoning ability Lu et al. (2022b) ; Lu et al. (2024) ; Yue et al. (2024) , rather than on their effectiveness in instructing or tutoring learners. Foundation Model for Education In the educational domain, there has been a growing interest in the use of large language models to help teach and learn Chevalier et al. (2024) ; Liu et al. (2024b) ; Kargupta et al. (2024) ; Li et al. (2024b) ; Macina et al. (2023) ; Yue et al. (2025) ; Yang et al. (2024) . Recently, many efforts have focused on evaluating MLLM’s capability to solve scientific questions, providing step-by-step reasoning Zhang et al. (2024) ; Lu et al. (2022b) ; Lu et al. (2024) . However, these studies only rely on the accuracy of the final task, overlooking important educational qualities such as clarity, scaffolding, and the participation of the learner. Preference Optimization Preference-based alignment methods Amini et al. (2024) ; Meng et al. (2024) ; Hong et al. (2024a) ; Ethayarajh et al. (2024) ; Azar et al. (2023) ; Liu et al. (2025) , which leverage pairwise preference

Read the complete text