Feedback That Clicks: Introductory Physics Students’ Valued Features in AI Feedback Generated From Self-Crafted and Engineered Prompts
Amogh Sirnoorkar, N. Sanjay Rebello · 2025
Read Feedback That Clicks: Introductory Physics Students’ Valued Features in AI Feedback Generated From Self-Crafted and Engineered Prompts by Amogh Sirnoorkar, N. Sanjay Rebello (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08516.txt.
Introduction
Amogh Sirnoorkar and N. Sanjay Rebello study what physics students value in AI feedback in a 2025 education paper. Large classes cannot give each student a careful reply. Since 2022, AI tools have promised fast, tailored feedback at that scale. The catch is skill. A student must ask the tool well before the reply can be good.
The study has three aims. First, how sharp are the prompts students write when they seek feedback on their arguments? Second, which traits of the AI reply do they prize most? Third, what trends run across groups of students? Prompt craft is now a study skill in its own right. These answers tell teachers what to teach about AI help, and tell tool builders what students actually use.
Catalog record
| CREATOR | Amogh Sirnoorkar, N. Sanjay Rebello |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08516
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Feedback That Clicks: Introductory Physics Students’ Valued Features in AI Feedback Generated From Self-Crafted and Engineered Prompts Preprint: APS/123-QED Amogh Sirnoorkar, N. Sanjay Rebello Abstract Since the advent of GPT-3.5 in 2022, Generative Artificial Intelligence (AI) has shown tremendous potential in STEM education, particularly in providing real-time, customized feedback to students in large-enrollment courses. A crucial skill that mediates effective use of AI is the systematic structuring of natural language instructions to AI models, commonly referred to as prompt engineering. This study has three objectives: (i) to investigate the sophistication of student-generated prompts when seeking feedback from AI on their arguments, (ii) to examine the features that students value in AI-generated feedback, and (iii) to analyze trends in student preferences for feedback generated from self-crafted prompts versus prompts incorporating prompt engineering techniques and principles of effective feedback. Results indicate that student-generated prompts typically reflect only a subset of foundational prompt engineering techniques. Despite this lack of sophistication, such as incomplete descriptions of task context, AI responses demonstrated contextual intuitiveness by accurately inferring context from the overall content of the prompt. We also identified 12 distinct features that students attribute the usefulness of AI-generated feedback, spanning four broader themes: Evaluation, Content, Presentation, and Depth. Finally, results show that students overwhelmingly prefer feedback generated from structured prompts, particularly those combining prompt engineering techniques with principles of effective feedback. Implications of these results such as integrating the principles of effective feedback in design and delivery of feedback through AI systems, and incorporating prompt engineering in introductory physics courses are discussed. I Introduction Providing customized student-facing feedback in large enrollment courses has been one of the emerging topical areas in physics education research Sirnoorkar and Rebello 2025 . Feedback is any information provided by an agent (such as people or material artifacts) about one’s performance Hattie and Timperley 2007 . The sources of feedback can be external as well as internal. In the context of student learning, information from peers, teachers, algorithms, etc. can act as external sources whereas one’s lived experiences, self-monitored observations, etc. can act as internal sources of feedback. This information can take various forms such as corrective, directive, alternative, or affective Burke and Pieterick 2010 . Effective feedback plays a crucial role in improving students’ learning along with enhancing their skills and academic performance Azevedo and Bernard 1995 . Effective feedback is also instrumental in motivating students to actively engage with content knowledge, with qualitative feedback considered more effective than quantitative feedback Giamos et al. 2024 . Researchers thus have called for students’ active involvement in the feedback process by facilitating them in interpreting, internalizing, and taking actions based on the provided information Nicol and Macfarlane-Dick 2006 . Facilitating effective and engaging external feedback poses a major challenge for educators. Resource demands such as human expertise, material costs, and time constraints make the process challenging. Additional barriers such as the need for timely delivery of information and the incorporation of design features that motivate students to meaningfully engage with and follow up on the information also compound the challenges. Consequently, numerous studies have explored ways to reliably automate the design and delivery of customized feedback based on student responses Sirnoorkar et al. 2023 ; Laverty et al. 2022 ; Allen et al. 2025 . Recent advancements in Generative Artificial Intelligence (henceforth referred as ‘AI’) promise a huge potential in addressing these barriers. This promise stems from AI’s ability to process multi-modal data and generate real time, coherent, customized, responses based on user inputs Polverini and Gregorcic 2024a ; Polverini and Gregorcic 2024b . The user inputs are natural language instructions, referred to as ‘prompts’, that guide the behavior and output of AI models. With carefully crafted prompts significantly enhancing the AI output, ‘prompt engineering’ has evolved into an empirical and growing body of research focused on ways of systematically structuring prompts to better interact with AI systems. Broadly, prompt engineering techniques can be categorized into two categories: foundational and advanced Chen et al. 2025 . Foundational techniques emphasize clarity, structure, and organization of prompts to optimize outputs. Advanced techniques involve more complex interactions with dynamically responding to AI outputs to yield precise, accurate, and high-quality results (refer Section II.1 for details). In physics education, researchers have explored the potential of prompt engineering in enhancing accuracy of AI responses to physics assessments Polverini and Gregorcic 2024a and prompting approaches of elementary pre-service teachers while seeking appropriate representational formats of physics concepts Hamed et al. 2025 . Despite concerns on content accuracy and students’ bias towards language sophistication, evolving models have significantly improved in responding to introductory physics content often times even with no emphasis on prompting Kortemeyer 2025 ; Kortemeyer 2023 ; Sirnoorkar et al. 2024 . The process of feedback generation broadly involves an initial assessment or grading of student responses (typically to an assessment) based on a predetermined rubric, followed by communication of results through performance reports. In physics education, considerable attention has been devoted to the former, i.e., AI-assisted grading of student responses. Studies in this area have leveraged AI models to grade students’ handwritten solutions Kortemeyer et al. 2024 , written descriptions of strategies to conceptual questions Kortemeyer 2024 ; Kortemeyer and Nöhl 2025 ; Lee et al. 2024 ; Chen and Wan 2025 , and solutions to numerical problems Wei et al. 2025 . This strand of research broadly notes that while AI models are becoming increasingly effective at grading, their accuracy varies depending on the combination of the employed prompting strategies. Moreover, they also highlight the continued need for active human oversight, particularly when grading problems that involve complex reasoning processes, hand-drawn representations, or responses at the lower end of the score spectrum. Research on AI-generated feedback reports in physics education is gradually evolving with studies increasingly focusing on leveraging prompting techniques to design customized, student-facing feedback. Instructors and students alike have appreciated the feedback content Chen and Wan 2025 both in terms of correctness and usefulness Wan and Chen 2024 . Dai et al . Dai et al. 2025 observed that mandated versus on-demand use of AI feedback by high school students had varied influences on their academic performance. Additional investigations have thus unpacked students’ perspectives, where students have described AI-generated feedback as useful, actionable, and clearly written, with particular appreciation for suggested improvements and rewritten examples Mills et al. 2025 ; Allen et al. 2025 . The above literature on design and delivery of feedback through AI systems highlight several gaps. Firstly, compared to the literature on AI grading (and to benchmarking the performance of AI models on physics assessments), research on AI-generated feedback in physics education remains relatively underexplored. Secondly, studies that focus on AI feedback have primarily employed empirical approaches in designing feedback with little to no focus on embedding the principles of effective feedback established in the education research literature. Thirdly, research on features of AI-generated feedback perceived as valuable by students while reasoning about physics problems remains relatively unexplored. Lastly, despite the reinforcing results on the significance of prompting approaches on the performance of AI models, coupled with students’ increased use of AI, there is a need for investigating the extent of sophistication in student-generated AI prompts when interacting