Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators

Heejin Do, Shashank Sonkar, Mrinmaya Sachan · 2026

Read Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators by Heejin Do, Shashank Sonkar, Mrinmaya Sachan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12748.txt.

Introduction

Do, Sonkar and Sachan test whether language-model student simulators truly hold misconceptions or merely flatter the tutor. Such simulators fluently mimic student replies, making them attractive for training tutors and teachers. Yet they are usually judged by output similarity, not by whether they maintain a coherent wrong-belief state. The team introduces a framework for faithfulness to wrong beliefs, with a contrastive feedback protocol comparing targeted feedback against misaligned and generic controls.

Catalog record

CREATORHeejin Do, Shashank Sonkar, Mrinmaya Sachan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50414
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.12748

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators Heejin Do, Shashank Sonkar, Mrinmaya Sachan Abstract Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators. Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction. We introduce a controlled framework for evaluating misconception faithfulness , whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception. Central to our framework is a misconception-contrastive feedback protocol that compares targeted feedback against two controls: misaligned feedback (targeting a different but plausible misconception) and generic feedback (only identifying answer is wrong). We propose Selective Flip Score (SFS), which quantifies how much more often a simulator flips its answer under targeted feedback than under contrastive controls. Across seven LLMs (4B–120B), multiple datasets, and prompting strategies, simulators exhibit near-zero SFS, correcting their answers at similarly high rates regardless of feedback relevance. Further analyses reveal a sycophantic failure mode: models behave less like students with misconceptions but more like problem-solvers who treat any corrective signal as a cue to abandon the simulated belief and re-solve from internal knowledge. To address this, we develop a post-training pipeline spanning supervised fine-tuning (SFT), preference optimization, and reinforcement learning (RL) with an SFS-aligned reward; SFT yields notable gains up to +0.56, and SFS-aligned RL provides more consistent improvements than preference optimization. Our results establish misconception faithfulness as a challenging yet trainable property, motivating a shift from static output matching toward interactive, belief-aware student modeling. 1 Introduction Large language models (LLMs) have emerged as a promising foundation for simulating students in educational AI. Their ability to generate fluent and plausible responses makes them attractive as simulated students , i.e., virtual learners used to train and evaluate both AI tutors and human educators, including teachers and teaching assistants [ 12 , 17 , 16 , 20 , 27 ] . Compared to traditional student models, LLM-based simulators enable more flexible and scalable experimentation on how learners respond to instruction and feedback prior to deployment in real classrooms [ 17 , 20 , 16 ] . However, fluency does not imply faithfulness: whether these simulators behave like students, rather than merely sound like them, remains an open question. Acting like a student requires more than producing student-like answers. A useful simulator must capture the misconception that gives rise to those answers. Misconceptions are systematic patterns of incorrect reasoning [ 4 , 18 , 30 ] that shape how students interpret problems, respond to feedback, and revise their understanding. Effective instruction therefore depends not only on identifying that an answer is wrong, but also diagnosing why it is wrong; prior work shows that teachers’ Knowledge of Student Misconceptions is strongly associated with learning gains beyond subject-matter expertise alone [ 5 ] . Thus, for student simulators to support meaningful tutor and teacher training, they must enable realistic practice in diagnosing and responding to misconception-driven reasoning. Figure 1: Diagnostic framework for misconception faithfulness via misconception-contrastive feedback. Given a problem q with incorrect answer a_{w} arising from misconception m , we evaluate simulator behavior under three feedback conditions: targeted feedback f_{T} addressing m , misaligned feedback f_{M} targeting a different plausible misconception m^{\prime}\neq m , and generic feedback f_{G} indicating only that a_{w} is incorrect. Top: A faithful simulator should selectively flips to the correct answer only under f_{T} . Bottom: In practice, LLM simulators frequently flip across all feedback conditions, yielding near-uniform flip rates ( F_{T}\approx F_{M}\approx F_{G} ) and near-zero SFS, indicating sycophantic problem solving rather than misconception-faithful belief-state modeling. Despite this, current evaluations of LLM-based student simulators primarily rely on static metrics such as answer accuracy or similarity to real student responses [ 12 , 17 , 16 , 27 ] . While these metrics assess whether outputs appear student-like, they do not test whether the simulator maintains a coherent learner state during interaction. In particular, they cannot distinguish between a simulator that preserves a stable misconception versus one that simply recomputes the correct answer whenever corrective feedback is given. A faithful simulator should update selectively: targeted feedback should induce revision, whereas generic or misaligned feedback should maintain the underlying wrong belief state, uncertainty, or pushback. We formalize this property through a controlled framework for evaluating misconception faithfulness : whether a simulator preserves misconception-consistent belief state and selectively updates it only when feedback addresses the underlying misconception. Central to our approach is a misconception-contrastive feedback protocol consisting of three conditions for the same initial error: Targeted feedback addressing the true misconception, Misaligned feedback targeting a different but plausible misconception, and Generic feedback only indicating that the answer is incorrect. Based on this setup, we propose a new metric, Selective Flip Score (SFS), which measures how much more likely a student simulator is to revise its answer under targeted feedback than under the contrastive controls. Applying this framework to seven LLMs (4B–120B parameters) across multiple datasets and prompting strategies, we find a consistent failure of misconception faithfulness. LLM student simulators exhibit near-zero SFS, correcting their answers at similarly high rates regardless of whether feedback targets the true misconception, a different misconception, or merely indicates that the answer is wrong. This reveals a sycophantic failure mode: models behave less like students with stable misconceptions and more like sycophantic problem solvers treating any corrective signal as a cue to abandon the simulated misconception and recompute the answer from internal knowledge. The pattern persists under reflective prompting and multi-turn interaction, suggesting that prompting alone does not induce a stable misconception-conditioned learner state. Beyond diagnosis, we investigate whether selective updating can be induced through post-training. We develop an SFS-aligned training pipeline spanning supervised finetuning (SFT), preference optimization, and reinforcement learning (RL), where rewards encourage flips under targeted feedback while penalizing sycophantic flips under misaligned or generic feedback. SFT yields the strongest gains, improving SFS by up to +0.555 , while RL provides more consistent improvements than preference optimization. Overall, our results show that misconception faithfulness is difficult to elicit through prompting alone, but can be substantially improved when selective updating is made an explicit training objective. Our key contributions are: • We introduce a diagnostic framework for evaluating misconception faithfulness in LLM-based student simulators, centered on a misconception-contrastive feedback protocol and a new metric, Selective Flip Score. • We show that current LLM student simulators, across seven models from 4B to 120B parameters, systematically fail to exhibit misconception-faithful behavior during interaction. • We identify sycophantic problem solving as the dominant failure mode: models respond to corrective cues by abandoning the simulated misconception and recomputing the answer from internal knowledge. • We construct an SFS-aligned post-training pipeline, showing that selective update behavior can be improved via SFT and further refined with preference- and reward-based optimization. 2 Diagnostic Framework for Misconception Faithfulness Our diagnostic framework evaluates whether LLM-based student simulators exhibit misconception-faithful behavior during interaction. Rather than assessing surface-level plausibility, we ask whether a simulator maintains a coherent misconception-driven belief state during interaction. We formalize this through selective updating behavior: a faithful student simulator should correct its answer when feedback directly addresses

Read the complete text