Personalized Parsons Puzzles as Scaffolding Enhance Practice Engagement Over Just Showing LLM-Powered Solutions Conference: Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2; February 26-March 1, 2025; Pittsburgh, PA, USA Proceedings of the 56th ACM Technical Symposi
Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson · 2025
Read Personalized Parsons Puzzles as Scaffolding Enhance Practice Engagement Over Just Showing LLM-Powered Solutions Conference: Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2; February 26-March 1, 2025; Pittsburgh, PA, USA Proceedings of the 56th ACM Technical Symposi by Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09210.txt.
Introduction
Xinying Hou and co-authors test a gentler way to help programming students who ask AI for help, in a 2025 paper. Modern chatbots will happily hand a stuck student a finished solution, and the student learns little. The team instead offered personalised Parsons puzzles: the lines of a correct program, shuffled, which the student must put back in order. The puzzle is built from an AI solution tuned to the student's own attempt.
In an eighty-minute classroom study of Python practice, students were split at random into two help styles. One group got the puzzles as scaffolding whenever they asked for help. The other saw the complete solution at once, copying the usual chatbot reply. The puzzle group stayed engaged in practice far longer than the solution group. The authors argue for AI help that makes students work, not just watch.
Catalog record
| CREATOR | Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 9 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-09210
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Personalized Parsons Puzzles as Scaffolding Enhance Practice Engagement Over Just Showing LLM-Powered Solutions Conference: Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2; February 26-March 1, 2025; Pittsburgh, PA, USA Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2 (SIGCSE TS 2025), February 26-March 1, 2025, Pittsburgh, PA, USA DOI: 10.1145/3641555.3705227 ISBN: 979-8-4007-0532-8/25/02 CCS: Social and professional topics Computing education Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson Abstract . As generative AI products could generate code and assist students with programming learning seamlessly, integrating AI into programming education contexts has driven much attention. However, one emerging concern is that students might get answers without learning from the LLM-generated content. In this work, we deployed the LLM-powered personalized Parsons puzzles as scaffolding to write-code practice in a Python learning classroom (PC condition) and conducted an 80-minute randomized between-subjects study. Both conditions received the same practice problems. The only difference was that when requesting help, the control condition showed students a complete solution (CC condition), simulating the most traditional LLM output. Results indicated that students who received personalized Parsons puzzles as scaffolding engaged in practicing significantly longer than those who received complete solutions when struggling. 1. Introduction Writing code is challenging for most novices. Thanks to the development of large language models (LLMs) and generative AI techniques, students now have the opportunity to generate code directly based on a natural language description ( Kazemitabaar et al. 2023 ) . Therefore, some novices can simply use AI code generation tools to complete their short programming task homework without even engaging with the programming practice ( Kazemitabaar et al. 2023 ) . This raises growing concerns about over-utilizing generative AI tools when learning programming, which can harm students’ programming skill development. To support struggling students in programming practice while keeping them engaged, built upon the previous work ( Hou et al. 2024 ) , this work explored the use of personalized Parsons puzzles as write-code scaffolding in a real classroom lecture setting. Parsons puzzles are an increasingly popular active programming exercise that requires students to arrange a set of drag-and-drop code blocks to solve a problem. It can have distractor blocks that are not needed in a correct solution. The personalized Parsons puzzle applied as programming scaffolding applied two levels of personalization when offered as scaffolding ( Hou et al. 2022 ) . The delivered puzzle was tailored to students’ existing code at both the code solution level and the block level ( Hou et al. 2024 ) . To understand the effectiveness of this scaffolding technique, we deployed it to a real undergraduate classroom setting and conducted a randomized between-subjects classroom experiment. The control condition offered students a complete AI-generated code solution when requesting help, a typical output of most AI code-generation tools. Students in both conditions could choose to ask for the provided scaffolding or solve the write-code practice independently. This work presents the preliminary results addressing the key RQ: Are there condition differences in terms of students’ practice engagement? 2. Methods The study was conducted in the winter semester of 2024 at a large public research university in the northern United States. This course covered programming concepts including Python basic data structures, object-oriented programming concepts, and more. Two study conditions Students in both conditions practiced traditional short programming tasks in an online system. Each task contained a natural language description, a programming area, and a "Save & Run" button to execute the code and display the unit test results. It also included a "Help" button to trigger the support. Students could ask for support at any time or choose to solve the write-code problems independently. In the puzzle-scaffolding condition (PC), students received personalized Parsons puzzles as scaffolding. After clicking the "Help" button, PC students received a personalized Parsons puzzle with no indentation-level requirements and immediate feedback. Upon completing a puzzle, they could copy the solution to the clipboard with the "Copy Answer to Clipboard" button or retyping it. Students could also regenerate a new personalized Parsons puzzle at any point. Students in the control condition (CC) practiced programming with the same programming interface. However, after clicking "Help", they received a complete personalized correct code solution. They could also copy the solution to the clipboard with the "Copy Answer to Clipboard" button. Both two types of support conducted the same personalization on the solution level ( Hou et al. 2024 ) . This study used the institution-protected GPT-4, which protected data appropriately. Figure 1. Two conditions: Puzzle Scaffolding condition (PC) and Control Condition (CC) Participants and Procedure This one-session classroom study was conducted during an 80-minute lecture period in week six. This study was introduced to students as an in-class practice. Students were instructed to do the sections in order and answer questions to the best of their ability without outside help. Students were randomly assigned to one of two support conditions. The practice topic was nested dictionaries in Python . Students were given four write-code practice problems in each condition. Each section had no time restrictions, allowing students to progress through the materials at their own pace until the end of the lecture. 3. Results There were 118 students (51 PC students and 67 CC students) who worked on all four practice questions as instructed and did not encounter any technical difficulties. We used the Mann–Whitney U test instead of the t-test in cases where the data was not normally distributed. Results showed that students in both conditions had similar levels of pretest performance ( U = 1628.5, p = .649, CLES = .48) and self-efficacy levels ( U = 1595.5, p = .540, CLES = .47) before the study, suggesting that the conditions were comparable. To answer the RQ, we investigated students’ practice time (in minutes), which indicated the duration of the practice engagement. Students’ practice time was calculated as the sum of the time they spent on each practice question. Results showed that students in the PC condition ( M = 22.7, SD = 10.1, Median = 22.8) spent significantly more time with the practice than those in the CC condition ( M = 15.8, SD = 12.7, Median = 11.7), U = 2368.0, p < .001, CLES = .69. Specifically, PC students spent approximately 7 more minutes on average in the practice compared to students in the CC condition. On the contrary, six students in the CC condition finished practice in less than two minutes. Some mainly opened the programming help immediately after receiving the question, copied the solution to the write-code box, and immediately submitted it. However, according to the students’ perceptions, we found some PC students felt that the puzzles still provided too much support, and they would rather have a more lightweight approach, like subtle hints. However, another group of PC students required more guidance on how the blocks should be rearranged. They asked for additional help when they did not know where to place the code blocks. For CC students, some were satisfied with providing a solution as a quick and accurate way to deliver support, but some found providing a full solution prevented them from thinking independently and learning anything besides getting the