Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education CCS: Human-centered computing Empirical studies in HCI
Tianyu Shao, Miguel Feijóo-García, Yi Zhang, Hugo Castellanos, Tawfiq Salem, Alejandra Magana, Tianyi Li · 2026
Read Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education CCS: Human-centered computing Empirical studies in HCI by Tianyu Shao, Miguel Feijóo-García, Yi Zhang, Hugo Castellanos, Tawfiq Salem, Alejandra Magana, Tianyi Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10400.txt.
Introduction
Shao and six colleagues trace how students prompt chat tools in a first Python class. From chats and final code of 163 students, they map paths from full hand off to step by step rework. Most learners pasted machine made code straight into their work, yet many shaped it through rounds of fixes. Prompt paths lined up with course marks and hinted at how well each learner steered their own study. The authors urge tools that coach joint work between students and machines instead of mere copying.
Catalog record
| CREATOR | Tianyu Shao, Miguel Feijóo-García, Yi Zhang, Hugo Castellanos, Tawfiq Salem, Alejandra Magana, Tianyi Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71939 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.10400
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education CCS: Human-centered computing Empirical studies in HCI Tianyu Shao, Miguel Feijóo-García, Yi Zhang, Hugo Castellanos, Tawfiq Salem, Alejandra Magana, Tianyi Li Abstract . As AI tools such as ChatGPT enter programming classrooms, students encounter differing rules across courses and instructors, which shape how they use AI and leave them with unequal capabilities for leveraging it. We investigate how students engaged with AI in an introductory Python assignment, analyzing student–LLM chat histories and final code submissions from 163 students. We examined prompt-level strategies, traced trajectories of interaction, and compared AI-generated code with student submissions. We identified trajectories ranging from full delegation to iterative refinement, with hybrid forms in between. Although most students directly copied AI-generated code in their submission, many students scaffolded the code generation through iterative refinement. We also contrasted interaction patterns with assignment outcomes and course performance. Our findings show that prompting trajectories serve as promising windows into students’ self-regulation and learning orientation. We draw design implications for educational AI systems that promote personalized and productive student-AI collaborative learning. 1. Introduction The rapid rise of Large Language Models (LLMs) such as ChatGPT, Gemini, and Claude has reshaped programming education by making on-demand code generation, conceptual explanations, and debugging support widely accessible ( 1 ; 14 ; 45 ) . This availability offers significant benefits, but also poses new challenges for educators: understanding how students actually learn with LLMs, identifying their evolving needs, and adapting instruction accordingly. At the same time, industry increasingly expects graduates to use LLMs effectively in their work, creating pressure for curricula to integrate these tools. Yet concerns persist that instant access to solutions may bypass the cognitive struggle and problem-solving processes essential for mastering programming. As a result, many undergraduate courses continue to discourage or restrict LLM use, and course-level policies vary widely. Studies have shown that while LLMs lower barriers to code generation and debugging, they also risk undermining essential learning processes. Novice programmers often struggle to craft effective prompts, which exposes fragile programming skills and limited prompt literacy ( 32 ; 25 ) . In some cases, students have passed exams with ChatGPT’s help despite weak preparation, a pattern that suggests reliance on AI can replace foundational practice rather than complement it ( 40 ) . Reliance on LLMs for debugging has also been shown to reduce opportunities for developing independent problem-solving skills ( 35 ) , while hallucinations and partial accuracy cast doubt on the reliability of AI support in STEM domains ( 12 ; 5 ) . These findings point to a central tension: LLMs provide powerful cognitive scaffolding, yet they can displace the productive struggle that is critical to mastering programming. Recent research has begun to examine this emerging landscape from multiple perspectives. One line of work explores how students use LLMs in programming courses, analyzing prompt types, usage patterns, and motivations for turning to AI ( 10 ; 22 ; 4 ) . These studies reveal that students seek a range of support, from complete solutions to targeted fixes or explanations, but they largely capture single points of interaction rather than full sessions. In practice, prompting behaviors shift dynamically within a task, yet few studies trace how these behaviors evolve across the entire problem-solving process. Another thread investigates how to help students engage more effectively with LLMs, for example, by scaffolding prompt formulation or encouraging reflection on AI outputs rather than treating them as black-box solutions ( 25 ; 32 ) . These interventions highlight the value of prompt literacy and metacognitive awareness but usually emphasize local improvements rather than holistic interaction trajectories. A third line of work designs collaborative interfaces that assign LLMs roles such as feedback providers, brainstorming partners, or debugging assistants ( 11 ; 13 ; 45 ) . While these efforts illustrate the potential of AI as an interactive learning companion, they often assume fixed roles, whereas actual student use blends delegation, repair, and reflection within a single task. Nonetheless, we still know little about how students actually use LLMs to navigate an entire problem-solving episode under authentic constraints. Most existing studies either isolate fragments of interaction or spread usage over extended timeframes, leaving unexamined the moment-to-moment strategies students deploy when solving a complete task from start to finish. Yet this end-to-end perspective is essential. Timed, single-session activities simulate the realities of student practice, where work is often completed under time pressure and efficiency takes precedence over exploration. Such contexts reveal how students manage cognitive load, shift between delegation and independent effort, and converge—or diverge—in their final solutions. By zooming in on a single, time-bounded assignment, we can trace interaction trajectories as they unfold, offering unique insight into how LLMs are appropriated as cognitive partners when the stakes of immediacy, completion, and correctness are most salient. We investigated student-LLM interaction trajectories within an in-class programming activity in an introductory Python course (Spring 2025) where 163 students were asked to solve a number-guessing game within a single lecture session, with the option of using an LLM for support. We collected both their chat transcripts and their final code submissions, enabling us to trace the full arc of interaction from initial prompt to completed solution. Our analysis addresses three research questions: RQ1: How do students’ prompts to LLMs evolve, and what recurring interaction trajectories emerge? RQ2: How closely do students’ submissions align with LLM-generated code, and to what extent do their solutions converge with one another? RQ3: How do interaction trajectories relate to code similarity, and how do these patterns affect their final code submission quality? Across 146 valid submissions and 662 prompts, we identify eight distinct interaction trajectories that reflect how students distributed cognitive effort between themselves and the LLM. We find strong convergence in student code, with over 80% directly reusing AI-generated solutions, though meaningful variation persisted at the margins. Students who fully outsourced to the LLM often performed as well as those who engaged more strategically, highlighting tensions between efficiency and reflection. Our results reveal how LLM-supported programming increasingly resembles a form of rapid prototyping: generate drafts, test them against requirements, and iterate through cycles of adaptation or debugging. This work makes the following contributions: (1) A typology of interaction trajectories. From 662 prompts across 146 valid submissions, we identify eight distinct trajectories that capture how students delegated, adapted, or debugged with LLMs across a single, full problem-solving process. (2) A dataset of paired LLM transcripts and student submissions. To our knowledge, this is among the first classroom datasets that capture prompt-level interactions and final code under a single and complex problem-solving process from start to end. (3) Design implications for LLM-supported learning. We reframe LLM-supported programming as cycles of prototyping, offering implications for assignment design, assessment, and equity that emphasize iteration and reflection as crucial learning goals. Positionality Statement This study was conducted by an interdisciplinary team with backgrounds in Human–Computer Interaction (HCI), Engineering Education, Natural Language Processing (NLP), and Computer Vision (CV). The course instructor (a CV researcher) and the teaching assistant (an NLP researcher) designed and administered the assignment and facilitated data collection. Data analysis was led by two HCI researchers, and the two engineering education students assisted with