Tracing Mathematical Proficiency Through Problem-Solving Processes

Jungyang Park, Suho Kang, Jaewoo Park, Jaehong Kim, Jaewoo Shin, Seonjoon Park, Youngjae Yu · 2025

Read Tracing Mathematical Proficiency Through Problem-Solving Processes by Jungyang Park, Suho Kang, Jaewoo Park, Jaehong Kim, Jaewoo Shin, Seonjoon Park, Youngjae Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00311.txt.

Introduction

Intelligent tutoring systems work best when they know what a student understands right now. Jungyang Park and co-authors trace mathematical proficiency through problem-solving processes in 2025, rather than through answers alone. Traditional knowledge tracing models rely only on whether responses were correct.

Correctness hides the rich information inside a student's steps, false starts, and strategies. Using the process itself should improve predictions of future performance and make the model's judgments explainable to teachers. Better tracing supports tutoring that responds to how a student thinks, not just to a string of right and wrong marks. Problem-solving traces can be captured by tutoring software without extra testing time. That makes the approach cheap to add to systems already in classrooms.

Catalog record

CREATORJungyang Park, Suho Kang, Jaewoo Park, Jaehong Kim, Jaewoo Shin, Seonjoon Park, Youngjae Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00311

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tracing Mathematical Proficiency Through Problem-Solving Processes Jungyang Park, Suho Kang, Jaewoo Park, Jaehong Kim, Jaewoo Shin, Seonjoon Park, Youngjae Yu Abstract Knowledge Tracing (KT) aims to model student’s knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems. However, traditional KT methods face fundamental limitations in explainability, as they rely solely on the response correctness, neglecting the rich information embedded in students’ problem-solving processes. To address this gap, we propose Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP), which incorporates students’ problem-solving processes to capture the multidimensional aspects of mathematical proficiency. We also introduce KT-PSP-25, a new dataset specifically designed for the KT-PSP. Building on this, we present StatusKT, a KT framework that employs a teacher-student-teacher three-stage LLM pipeline to extract students’ MP as intermediate signals. In this pipeline, the teacher LLM first extracts problem-specific proficiency indicators, then a student LLM generates responses based on the student’s solution process, and a teacher LLM evaluates these responses to determine mastery of each indicator. The experimental results on KT-PSP-25 demonstrate that StatusKT improves the prediction performance of existing KT methods. Moreover, StatusKT provides interpretable explanations for its predictions by explicitly modeling students’ mathematical proficiency. 1 Introduction Knowledge Tracing (KT) is a technique that models a learner’s evolving knowledge state over time ( 10 ; 24 ) . However, directly obtaining the true knowledge state remains inherently challenging. In practice, KT addresses this limitation by exploiting learners’ historical interaction data to predict the likelihood of correctness in subsequent exercises, as illustrated in Figure 1 (a). Early KT approaches, such as Bayesian Knowledge Tracing (BKT) 10 and Item Response Theory (IRT) 15 , introduced interpretable probabilistic frameworks but struggled to capture long-term dependencies. With the advent of neural networks, deep learning-based KT (DLKT) models have demonstrated notable improvements by employing architectures such as recurrent neural networks ( 34 ; 29 ) , and attention mechanisms ( 32 ; 14 ) . Beyond architectural innovations, subsequent studies have further enriched KT models by integrating diverse contextual signals, such as item difficulty ( 45 ; 5 ) , student attempt history 28 , and knowledge concept structure 37 , thereby enhancing their performance and interpretability. Figure 1: (a) Conventional KT utilizes limited metadata, lacking interpretability. (b) By incorporating problems, students reasoning, and teacher comments, StatusKT captures granular proficiency like a teacher. Although KT has advanced in various ways, most approaches rely on outcome-based signals, such as answer correctness, knowledge concepts, problem-response time, and question difficulty ( 41 ; 39 ; 45 ; 23 ) . However, such signals often fail to reflect students’ understanding. In real-world educational contexts, as illustrated in Figure 1 (b), teachers evaluate understanding through the problem-solving process rather than outcomes alone ( 7 ; 39 ; 21 ) . Nevertheless, despite the importance of this process-oriented perspective, current KT approaches and datasets remain underexplored for incorporating it ( 12 ; 27 ; 20 ) . To address this limitation, we propose Knowledge Tracing with Problem-Solving Process ( KT-PSP ), a new formulation of the KT task that explicitly incorporates students’ problem-solving processes into the interaction sequence. To facilitate research on KT-PSP , we further introduce KT-PSP-25 , a novel dataset that contains real-world problem-solving processes for each student-problem interaction. While the original handwritten images cannot be released due to privacy and policy constraints, we provide OCR transcriptions of the solution processes. Alongside these process-level texts, the dataset includes rich problem metadata (knowledge concepts, difficulty, question text, solution text) and interaction attributes (selected answer, duration, correctness) Furthermore, we introduce StatusKT , a KT framework that incorporates students’ mathematical proficiency (MP)–defined in 13 as comprising five components–extracted from their problem-solving processes as an intermediate signal for modeling student knowledge, leveraging this rich information as shown in Figure 1 (b). Inspired by 11 , who assessed four observable MP dimensions using a structured, step-based evaluation, StatusKT adopts a similar teacher-student-teacher LLM pipeline to extract MP signals. Specifically, a teacher LLM derives problem-specific MP indicators; a student LLM responds to them based on the student’s solution process; and a teacher LLM evaluates these responses to produce problem-level MP signals. These MP signals are then integrated into a KT backbone as auxiliary inputs, enabling the model to incorporate proficiency-level information, thereby enhancing both its predictive accuracy and interpretability. Our experiments on KT-PSP-25 demonstrate that StatusKT consistently improves the performance of strong DLKT baselines across multiple metrics. In addition to higher predictive accuracy, StatusKT provides interpretable proficiency signals that offer fine-grained insights into students’ learning progress. Our contributions are as follows: • We introduce KT-PSP , a new KT task formulation that incorporates students’ problem-solving processes into the interaction sequence, enabling process-aware modeling of knowledge. • We release KT-PSP-25 , a first mathematical KT dataset containing real-world problem-solving processes. • We propose StatusKT , which is a novel KT framework that extracts MP signals from students’ problem-solving process through teacher-student-teacher LLM pipeline and integrates them as auxiliary representations. • Experiments on KT-PSP-25 show that StatusKT consistently improves prediction performance over DLKT baselines while providing interpretable signals. 2 Related Works 2.1 Knowledge Tracing KT has played a crucial role in ITS by enabling accurate assessment of students’ progressive knowledge states. A well-studied approach in KT relies on students’ learning interaction data to infer and predict their understanding of specific concepts in an auto-regressive manner over time ( 34 ; 44 ; 29 ; 17 ; 47 ) . Other models employ memory-augmented neural networks for concept embedding, using a static key memory for soft attention and a dynamic value memory to update students’ knowledge states ( 46 ; 2 ) . Moreover, graph neural network (GNN) based approaches ( 30 ; 43 ) model relationships between questions and KCs, aggregating current embeddings with historical states and performing graph-structured updates—often over implicit/learned edges—to yield time-series, node-level predictions. Attention-based approaches in KT have shown strong potential in capturing the relationship between students’ past interactions and future problems ( 32 ; 8 ; 14 ; 25 ; 22 ) . While effective, these models can be sensitive to noisy or confounded behavioral signals. For robustness, it is crucial to model the key factors that shape students’ knowledge states. 16 decouple stable cognitive patterns from error-prone factors that induce anomalous outcomes, such as slips and incidental question-level mistakes. However, key determinants remain underexplored, most notably students’ problem-solving processes and their underlying mathematical proficiency, which likely mediate the mapping from interaction traces to knowledge states. Accordingly, we propose a method that leverages fine-grained signals from students’ problem-solving processes to infer their mathematical proficiency ( 13 ; 38 ) , thereby enhancing knowledge tracing. 2.2 Knowledge Tracing Dataset Question Question Question Student’s Dataset Difficulty Timestamp Text Type PSP ASSISTments2009 ✓ ✗ ✗ ✗ ✗ ASSISTments2014 ✓ ✗ ✗ ✗ ✗ Junyi2015 ✓ ✓ ✗ ✗ ✗ KDDcup2010 ✗ ✓ ✗ ✗ ✗ EdNet ✗ ✓ ✗ ✗ ✗ DBE-KT22 ✓ ✓ ✓ ✗ ✗ XES3G5M ✗ ✓ ✓ ✓ ✗ ES-KT-24 ✗ ✓ ✓ ✓ ✗ KT-PSP-25 ( Ours ) ✓ ✓ ✓ ✓ ✓ Table 1: Comparison of educational datasets commonly used in KT (ASSISTments ( 12 ; 33 ) , Junyi Academy 4 , KDD2010, EdNet 9 , DBE-KT22 1 ,

Read the complete text