Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build

Sina Rismanchian, Hasan Uzun, Jeffrey Matayoshi, Eric Cosyn, Eyad Kurd-Misto · 2026

Read Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build by Sina Rismanchian, Hasan Uzun, Jeffrey Matayoshi, Eric Cosyn, Eyad Kurd-Misto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21629.txt.

Introduction

Sina Rismanchian and co-authors ask how AI has shifted learning and its effects. Surveys show little change, while studies report AI use without scale or duration to measure consequences. Using a decade panel of over three million ALEKS interactions, they find learning time on AI-susceptible problems falls two point eight percent per quarter for college students after ChatGPT. This cumulates to twenty-six point nine percent over eleven quarters. Under proctoring the divergence vanishes. Proctored retention items show a twenty-five percent decline in odds of correct response.

Catalog record

CREATORSina Rismanchian, Hasan Uzun, Jeffrey Matayoshi, Eric Cosyn, Eyad Kurd-Misto
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77384
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21629

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build Sina Rismanchian, Hasan Uzun, Jeffrey Matayoshi, Eric Cosyn, Eyad Kurd-Misto Abstract How much have students’ ordinary learning processes shifted in response to generative AI, and how does that affect their durable learning outcomes? Self-report surveys show little change, while small-scale behavioral studies report widespread AI use without the scale or duration to measure learning consequences. We address both questions using a ten-year panel of 3.2 3.2 million ALEKS learning interactions for the time-on-task analysis, complemented by ALEKS PPL placement-assessment data for the proctoring and retention analyses, with a quasi-experimental design exploiting within-curriculum variation in AI susceptibility: text-based word problems transcribable into AI prompts serve as the treated group; graph-based problems requiring interactive platform manipulation as the comparison. Learning time on AI-susceptible problems declines 2.8 % 2.8\% per quarter among college students after ChatGPT’s release, cumulating to 26.9 % 26.9\% over eleven quarters; high-schoolers show 31.3 % 31.3\% , middle-schoolers 9.0 % 9.0\% , and Grade 5 students no detectable change. The divergence vanishes entirely under proctoring for college students, making general efficiency gains unlikely. Logistic fixed-effects models on randomly assigned proctored retention items yield a 25 % 25\% cumulative decline in odds of correct response; the same estimator on non-proctored assessment produces a large opposite-signed increase — inconsistent with any platform, cohort, or curriculum explanation. These results are among the first large-scale behavioral and outcome evidence that generative AI has altered how students study and the knowledge they build — the population-level indicator of cognitive surrender , with direct implications for educational research, assessment governance, and AI policy. Significance Statement Whether generative AI has changed how students study and what they learn has been hard to answer: surveys show little change, and small experiments cannot detect longitudinal shifts. Using 3.2 million mathematics learning interactions over a decade, we document a large post-ChatGPT decline in time spent on AI-susceptible problems which is absent under proctoring. The shift carries a durable learning cost: on randomly assigned proctored retention items, odds of correct response fall by 25\% , while the same estimator on non-proctored assessment yields a large opposite-signed increase — a reversal that is unlikely for non-AI mechanism to produce. 1 Introduction The arrival of ChatGPT and generative AI chatbots ( 24 ) placed a capable school-level mathematics problem-solver ( 3 ) in the hands of millions of students with essentially zero marginal cost. It is widely assumed that this shift has changed how students study, nonetheless, reliable evidence on how much student behavior has actually moved is scarce. Large-scale self-report surveys of secondary and postsecondary students in the years following ChatGPT’s release find little to no change in the rate at which high school students report using AI for academically dishonest behaviors relative to pre-ChatGPT baselines ( 15 ; 6 ) . This stability is at odds with widespread intuitions about AI adoption, and perhaps reflects limitations of self-report: social-desirability bias, normative ambiguity about what counts as AI misuse, and a lack of shared vocabulary for describing the increasingly fine-grained ways students interact with AI tools ( 20 ) . Other survey-based measurements that use sophisticated methods to mitigate social desirability bias find that 20% to 34% of college-level students report using AI to cheat ( 22 ; 25 ) ; however, it remains unclear whether these respondents engaged in such behavior only once or did so repeatedly. On the other hand, small-scale behavioral studies of AI use in instructional contexts have shown widespread concerning AI usage among undergradute students but are limited by sample sizes and short timeframes ( 27 ) . The net result is that educators, administrators, and researchers have been formulating AI policy largely in the absence of quantitative evidence about the magnitude of behavioral change that has actually occurred. We address this gap using large-scale behavioral trace data from ALEKS, a widely deployed adaptive mathematics learning and assessment platform serving more than four million students annually. For the behavioral analyses, we draw on two time-based data sources observed over a ten-year window spanning the pre- and post-ChatGPT periods: 3.2 million ALEKS learning interactions for the time-on-task analysis and 12.2 million ALEKS PPL placement-assessment response-time observations for the proctoring analysis. Unlike self-reports, interaction logs capture students’ actual behavior: how long they spend on each problem, which problem formats they accelerate on, and whether those patterns persist under supervision. Our primary behavioral outcome is time-on-task, a basic signal of engagement with the material ( 8 ) . The central analytical lever is within-curriculum variation in AI susceptibility. Text-based word problems are easily transcribed into natural-language prompts and can therefore be solved with AI assistance at low friction. Graph-based problems, by contrast, require visual interpretation and interactive manipulation of platform widgets, making them substantially harder to outsource to AI. This differential susceptibility creates a within-student, within-quarter comparison between AI-susceptible and AI-resistant problems, helping absorb platform-wide trends, cohort differences, and calendar-time shocks. Having established whether student behavior has shifted, we then ask whether that shift matters for learning. To do so, we complement the behavioral analyses with 6.7 thousand ALEKS PPL retention observations that measure whether students retain the relevant knowledge under supervised testing conditions. This retention analysis allows us to test whether reduced engagement with AI-susceptible problems is associated with weaker durable knowledge. The cognitive science of active learning ( 7 ; 14 ) posits that active cognitive engagement at study time produces deeper understanding, higher mastery, and more durable retention than passive processing — and recent randomized experimental evidence supports this prediction in the context of generative AI. 1 showed that unrestricted GPT-4 access during mathematics practice improved assisted performance by 48\% but reduced unassisted exam scores by 17\% in a single-semester experiment with approximately 1{,}000 students. Whether this performance reversal scales to millions of students over multi-year timeframes in authentic learning environments — without any experimental manipulation — is an open question that only large-scale observational data can answer. We therefore investigate four research questions (RQ) in sequence. RQ1 , is there evidence in large-scale behavioral trace data that student study time has shifted on AI-susceptible problems following generative AI chatbots’ proliferation, and if so, how large is the shift and how does it vary by student age? RQ2 , does the behavioral shift disappear in proctored assessment settings where AI access is restricted — a falsification test that distinguishes AI-use-driven from general-efficiency-driven explanations? RQ3 , does the behavioral shift have measurable consequences for durable knowledge, as indexed by retention performance on the same problem types under conditions that restrict AI at the point of testing? RQ4 , does a second falsification by applying the same retention estimator to non-proctored assessments produce the opposite-signed result predicted by an AI-use mechanism but by no other candidate explanation? Our answers, in order, are: (1) yes and the shift is substantial: cumulative learning-time reductions of 26.9\% in college and 31.3\% in high school over eleven post-ChatGPT quarters, concentrated in AI-susceptible formats, with only a small effect in middle school ( -9.0\% ) and a null effect in Grade 5, an age gradient consistent with autonomous AI access; (2) yes, the divergence vanishes entirely

Read the complete text