Developing Authentic Simulated Learners for Mathematics Teacher Learning: Insights from Three Approaches with Large Language Models

Jie Cao ∗ Ha Nguyen ∗, Selim Yavuz, Boran Yu, Shuguang Wang, Pavneet Kaur Bharaj, Dionne Cross Francis · 2026

Read Developing Authentic Simulated Learners for Mathematics Teacher Learning: Insights from Three Approaches with Large Language Models by Jie Cao ∗ Ha Nguyen ∗, Selim Yavuz, Boran Yu, Shuguang Wang, Pavneet Kaur Bharaj, Dionne Cross Francis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04361.txt.

Introduction

Cao and colleagues evaluate three ways to make simulated pupils feel authentic for mathematics teacher training. Zero- or few-shot prompting often yields fake knowledge and language. Fine-tuning, multi-agent setups and Direct Preference Optimization all improve cognitive and linguistic authenticity over few-shot prompts. Interviews with eight pre-service teachers and researchers show distinct trade-offs. The fine-tuned model gives realistic, brief replies but fewer chances to stretch pupil thinking, while multi-agent and DPO versions spell out pupil reasoning. The paper weighs authenticity against instructional value.

Catalog record

CREATORJie Cao ∗ Ha Nguyen ∗, Selim Yavuz, Boran Yu, Shuguang Wang, Pavneet Kaur Bharaj, Dionne Cross Francis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17522
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04361

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Developing Authentic Simulated Learners for Mathematics Teacher Learning: Insights from Three Approaches with Large Language Models Jie Cao ∗ Ha Nguyen ∗, Selim Yavuz, Boran Yu, Shuguang Wang, Pavneet Kaur Bharaj, Dionne Cross Francis Abstract Large Language Model (LLM) simulations, where LLMs act as students with varying approaches to learning tasks, can support teachers’ noticing of student thinking. However, simulations using zero- or few-shot prompting often yield inauthentic knowledge and language, directing teachers to unrealistic reasoning. We evaluate three approaches (Fine-tuning, Multi-agent, and Direct Preference Optimization; DPO) to improve the authenticity and pedagogical utility of simulated students. All approaches improve cognitive and linguistic authenticity, compared with few-shot prompts. Interviews with elementary mathematics pre-service teachers and researchers ( n = 8) reveal distinct pedagogical affordances. The fine-tuned model produces realistic, brief responses but limits opportunities to extend students’ thinking. Meanwhile, the multi-agent and DPO approaches generate explicit reasoning behind student strategies. We discuss implications for designing LLM simulations that balance authenticity with instructional utility for teacher learning. * * footnotetext: These authors contributed equally to this work. 1 Introduction Responsiveness in mathematics education refers to professional noticing of students’ mathematical thinking with three interconnected skills: attending to students’ strategies, interpreting their emerging understanding, and deciding how to respond to build on that understanding [ 5 ] . Developing noticing skills is challenging for pre-service teachers (PSTs), who tend to prioritize guiding students to correct solutions rather than enriching their mathematical interpretations [ 7 ] . To support teacher noticing, practice-based teacher education (PBTE) decomposes noticing practices into component skills and allows teachers to rehearse in low-risk environments [ 4 ] . PSTs can practice teaching interactions in simulated classrooms using human role-playing [ 8 ] , mixed-reality [ 3 ] , and more recent Large Language Models (LLMs) student simulations as a scalable solution [ 16 , 25 ] . Researchers have used specific prompts to emulate students’ knowledge and motivation in LLM simulations [ 6 ] . However, LLMs might show characteristics of AI assistants or experts rather than learners [ 12 ] . They might drift across conversational turns and show inconsistencies in simulating cognition and language [ 11 , 13 ] , negatively impacting teacher engagement and the simulations’ effectiveness. To address these challenges, we explore approaches beyond simple prompting (e.g., zero-shot, few-shot), namely Fine-tuning [ 24 , 26 ] , Multi-agent systems [ 2 ] , and Direct Preference Optimization (DPO) [ 20 ] . We ask: RQ1: To what extent do Fine-tuning, Multi-agent, and DPO approaches reflect authentic student cognition and language, compared to few-shot prompts? RQ2: How do teachers perceive the authenticity and pedagogical utility of the simulated students? 2 Related work 2.1 Teacher Noticing in Mathematics Learning Responsive teaching in mathematical education requires a high degree of professional noticing of students’ mathematical thinking , including attending to students’ reasoning and supporting equitable, student-centered classroom discourse [ 5 , 23 ] . Although promising, many elementary teachers and PSTs struggle to enact professional noticing. They only pursue ideas that appear accurate and do not always connect students’ ideas and mathematical concepts [ 14 ] . PBTE advances professional noticing by allowing PSTs to rehearse authentic teaching [ 4 ] . Traditional PBTE approaches, such as shadowing practicing teachers and implementing instruction in field experiences, are short and can negatively impact student learning. Teacher educators have implemented alternative practices, including video reflections [ 18 ] , peer rehearsals [ 9 ] , and role-playing [ 14 ] . However, video reflections lack the urgency of real-time decision-making, and peer or role-play scenarios may not capture authentic student reasoning. 2.2 LLM-based Student Simulations LLMs-based simulations—where LLMs role-play as students—offer a promising PBTE approach that overcomes these limitations [ 1 , 6 , 16 ] . However, researchers have identified an authenticity gap . LLM simulations may inconsistently or inaccurately simulate students’ understanding and error patterns [ 12 , 27 ] and rely on overly verbose and complex language that does not reflect student talk [ 13 ] . To address these issues, we turn to three approaches: Fine-tuning, Multi-agent architectures, and DPO. Fine-tuning involves training LLMs on domain-specific data to align outputs with reference patterns. PSTs who interacted with LLMs fine-tuned with classroom discourse data reported that the interactions felt naturalistic and positively impacted how they would approach responsive teaching [ 1 , 27 ] . We also explore Multi-agent architectures , which employ multiple LLMs to collaborate, critique, and self-correct responses, to improve LLMs’ reasoning in complex tasks [ 10 ] . Finally, we use Direct Preference Optimization (DPO) , which provides paired preference data and steers the model’s output towards preferred behaviors [ 17 ] . DPO has shown promise in increasing the accuracy and pedagogical alignment of LLM-generated feedback in mathematics education [ 20 ] . Notably, these approaches differ in data requirements and adaptivity to feedback. To our knowledge, no prior work has systematically compared these approaches to examine the authenticity of student dialogues. Our study thus offers design-relevant evidence for scaling LLMs simulations. 3 Method 3.1 The Teaching Task and Simulated Student Profile In the simulation (Figure 1 ), PSTs engaged in one-on-one, text or voice interactions with an LLM agent (“Josh”), who role-played as a fifth-grader finding “a fraction between 2/3 and 7/8.” This task helped PSTs practice eliciting student thinking in working with fractions, a key topic in elementary classrooms. We implemented the simulation with 15 PSTs in a mathematics teaching methods class at a public university in the United States (Fall 2025). The PSTs interacted with the simulations five times (10-15 minutes/interaction), totaling 1438 talk turns. We used few-shot prompting in this initial implementation (left; Figure 1 ). PSTs reported that the simulated student sometimes appeared inauthentic (e.g., “talking too much”, “being too smart”), which affected their attempts to elicit or extend the agent’s thinking. These observations motivated us to identify instances where the agent might appear inauthentic and improve its output. Figure 1: Prompt and interface for the simulated agent (Josh) 3.2 Evaluation Framework of the Simulated Student Two researchers with mathematics teaching experience analyzed 20% of PST-agent interactions from the first implementation. They identified 40 interactions that appeared inauthentic and wrote analytic memos to document their reasoning. We inductively analyzed the memos to develop an evaluation framework, focusing on Cognition and Language (building on [ 11 , 13 ] ). Appendix A provides code descriptions 1 1 1 https://osf.io/5nv6u/overview?view_only=a0f57da0ddc746c58d1156fccb24d211 ). For Cognition , responses should align with the student profile in knowledge scope and show logical consistency in understanding across turns. It should not express inconsistent uncertainty . The explanation level should not be too complete to reflect fragmented or emerging understanding of fractions. Regarding Language , responses should reflect a natural student-like tone . They should avoid formal language (using disciplinary terminologies) and formulaic structure (repeating scripted phrases). 3.3 Three Approaches for Developing the Simulated Student Building on our evaluation (section 3.2), we explored three approaches to improve authenticity: Multi-agent (responder-evaluator-refiner), Fine-tuning (using real-world classroom data), and DPO (training from preference data; Figure 2 ). Figure 2: Overview of the approaches. LLM selection considering commercial (e.g., GPT) and open-source LLMs (Llama, Mistral) from testing-feedback in Summer-Fall

Read the complete text