VArsity: Can Large Language Models Keep Power Engineering Students in Phase?
Samuel Talkington, Daniel K. Molzahn · 2025
Read VArsity: Can Large Language Models Keep Power Engineering Students in Phase? by Samuel Talkington, Daniel K. Molzahn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20995.txt.
Introduction
Samuel Talkington and Daniel Molzahn report a classroom experiment with AI in power engineering in a 2025 paper. In two offerings of a Georgia Tech course on power system analysis, students were graded partly on finding and fixing errors in ChatGPT's answers to power factor correction problems.
With the older GPT-4 in fall 2023, most students caught the errors. With the stronger o1 model in spring 2025, the mistakes grew much harder to spot. The case study uses that shift to argue that language models now challenge how engineering courses assess learning: if students must audit the AI, the audit itself gets harder as models improve. The authors call for more study of the model's role in teaching and testing.
Catalog record
| CREATOR | Samuel Talkington, Daniel K. Molzahn |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20995
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VArsity: Can Large Language Models Keep Power Engineering Students in Phase? Samuel Talkington, Daniel K. Molzahn Abstract This paper provides an educational case study regarding our experience in deploying ChatGPT Large Language Models (LLMs) in the Spring 2025 and Fall 2023 offerings of ECE 4320: Power System Analysis & Control at Georgia Tech. As part of course assessments, students were tasked with identifying, explaining, and correcting errors in the ChatGPT outputs corresponding to power factor correction problems. While most students successfully identified the errors in the outputs from the GPT-4 version of ChatGPT used in Fall 2023, students found the errors from the ChatGPT o1 version much more difficult to identify in Spring 2025. As shown in this case study, the role of LLMs in pedagogy, assessment, and learning in power engineering classrooms is an important topic deserving further investigation. I Introduction In just a few short years, large language models (LLMs) have been ubiquitously deployed in a broad array of fields, including power engineering [ 1 ] . The impacts of these models have been rapid and widespread, especially in higher education [ 2 , 3 , 4 ] . From engineering [ 4 ] , operations research [ 5 ] , and algorithms [ 6 ] , academic communities are asking critical questions about the role of LLMs. These questions are also being raised in power engineering education [ 7 ] . The power engineering community has had limited discussion on the impacts of LLMs. Early studies report both promise and peril: while LLMs accelerate ideation, they also confidently generate physically inconsistent or numerically incorrect answers [ 2 , 3 ] . Electric power systems, as critical infrastructure, may be particularly vulnerable to the risks posed by LLMs, and these risks are especially poignant in the context of power engineering education. If the next generation of power engineers is trained in a way that validates or even promotes frequent errors in units, sign conventions, or phasor arithmetic, this can propagate directly into megawatt-scale mistakes. Recognizing that LLMs are likely here to stay in the classroom [ 2 ] , there is an urgent need for community discussion on governance and best practices for their use. Prior work emphasizes that teachers and learners must develop new competencies (e.g., prompt strategies, fact checking) to harness LLMs responsibly [ 3 , 4 ] . In this spirit, prompt design and assessment techniques that integrate LLMs into coursework in a way that reinforces fundamental engineering skills, rather than short-circuiting them, are poised to become essential. This issue became salient to the authors during our experience teaching the Spring 2025 and Fall 2023 iterations of ECE 4320: Power System Analysis & Control , a first course in power systems at Georgia Tech that is typically taken by senior undergraduate and first-year graduate students. In this course, we experimented with deploying OpenAI’s ChatGPT LLM as part of exam assessments. Specifically, students were tasked with interpreting and correcting errors in the solutions produced by ChatGPT to various power systems problems. Using power factor correction problems as a case study topic, the contributions of this paper are: 1. Assessment protocol : Case study examples using LLM outputs in power engineering course assessments, including a grading rubric based on error identification and correction. 2. Empirical data : A retrospective analysis student success in identifying and correcting LLM output errors. 3. Perspectives and guidelines: Our findings in Section V suggest an opportunity for power engineering educators to refine their assessment practices by harnessing LLMs to promote critical engineering skills. The remainder of this paper is organized as follows. Section II surveys related work and the ECE 4320 course. Section III details the problem and prompt design. Section IV explains the solution output, the errors it contains, and corrections. Section V presents empirical results. Section VI concludes the paper. II Background II-A Related work Reference [ 1 ] maps research questions for LLMs in the electric energy sector, including operation and planning. Although [ 1 ] did not investigate domain-specific educational impacts, the work of [ 7 ] evaluated GPT-3.5 on economic dispatch homework problems, finding that 58% of its numerical outputs violated basic power balance constraints. Two key gaps in prior literature are: 1. Problem setting : The prior education task explored in [ 7 ] , while valuable, was a short, single-step economic dispatch problem. The economic dispatch problem, including the formulation in [ 7 ] , typically uses a highly simplified model that does not encode the physics that governs the underlying power system. Presently, the literature lacks an understanding of how LLMs understand multielement linear circuits in power engineering assignments. 2. Students in the loop : We currently know little about whether students can identify and correct LLM mistakes in electrical engineering contexts, and which scaffolds help. As LLM workflows become increasingly intertwined in classrooms and engineering practice, it is essential to improve our understanding of students’ LLM interactions. In adjacent communities, [ 5 ] explores the use of LLMs in optimization education, covering the computation of dual linear programs. Reference [ 6 ] covers the performance of ChatGPT o1 for a proof-based analysis of algorithms course, finding that it performed at a passing undergraduate level. II-B Overview of the course and the Georgia Tech experience At Georgia Tech, ECE 4320 is typically one of the final courses taken by undergraduate ECE students who choose the Electric Energy Systems Thread , which is a structured curriculum for students who wish to specialize in electric power systems or power electronics. For many such students, the ECE 4320 classroom is the last time they will be exposed to formal power system concepts in classroom settings. This course introduces students to key power engineering concepts, such as the power flow equations, the optimal power flow problem, unit commitment, and state estimation. The course thus lays the foundation for a career in power engineering and introduces core concepts for graduate education and research in power systems. The foundational nature of this course means that understanding LLMs’ impact is essential. III A Case Study: Power Factor Correction In the Spring 2025 iteration of ECE 4320, both students and ChatGPT version o1 were presented with a power factor correction problem based on the single-phase, 60 Hz circuit in Fig. 1 . We focus on this problem as a case study on LLMs in power engineering education. Providing a comparison to an earlier version of ChatGPT (version GPT-4 ), the Fall 2023 iteration of ECE 4320 posed related power factor correction and AC power flow formulation problems described in Appendices -A and -B , respectively. ChatGPT version o3 has vastly improved performance on all problems; see Appendix -C . III-A Problem formulation In Fig. 1 , the active power consumed by the load, P_{d} , is a constant value of 50 W. The reactive power consumed by the load, Q_{d} , continuously varies between a minimum value of {-}36 VAr and a maximum value of 60 VAr. As shown in the circuit above, the voltage source has an RMS value of 100 V. The students were instructed to act as circuit designers—they could choose (fixed) values for both capacitive reactance X_{l} and