Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers

Shahin Hossain · 2026

Read Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers by Shahin Hossain (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28749.txt.

Introduction

Hossain studies how undergraduate students lean on large language models, the AI chat tools many now use while writing essays. His study ran at a public university that serves many minority students, and it draws on 382 survey responses, 14 interviews and hundreds of written answers. Earlier tools judged reliance only by how often students used AI, which he argues misses what students actually contribute themselves.

Four kinds of reliance emerged from the data. Strategic users, about a third of the group, used AI most thoughtfully, while a small dependent group, about 4.5 percent, leaned on it heavily. Oddly, the strategic group scored lowest on standard measures, which Hossain reads as a flaw in those measures. He also found about 13 percent of students refuse AI for ethical reasons, a group current frameworks ignore.

Catalog record

CREATORShahin Hossain
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE106 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28749

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers Shahin Hossain Abstract . Although most undergraduates now use Large Language Models (LLMs) for academic writing, there are no clear or validated methods to distinguish among the various ways students rely on AI. Existing tools assess AI reliance exclusively by frequency of use, a measure that, as this study demonstrates, inadvertently rewards students for depending on AI rather than recognizing their individual intellectual contributions. Conducted at a public minority-serving university and grounded in the AI Literacy Framework, Expectancy-Value Theory, and Biggs’s 3P Model, the research involved 382 undergraduates, 14 interviews, and 396 open-ended responses from two survey items. Four distinct types of AI reliance were identified and confirmed: Strategic (34.3%), Instrumental (30.9%), Dialogic (30.4%), and Dependent (4.5%). Students’ beliefs about the value and difficulty of using AI affect the extent of their reliance on AI, while their actual AI skills determine the specific type of AI they use, indicating that differentiated approaches are necessary to support students effectively. A key finding was that Strategic users, those who use AI most thoughtfully, scored the lowest on standard assessment measures, a discrepancy that reflects limitations in current evaluation methods, which prioritize AI-driven outcomes over actual writing quality and thereby penalize students demonstrating greater independent thinking. Analysis also revealed an additional group, approximately 13 percent, who decline AI use for ethical reasons rather than practical considerations, and whom current frameworks fail to account for. These findings carry significant implications for AI literacy programs, the measurement of student learning outcomes, and equitable AI policies at minority-serving institutions. A Mixed-Methods Study at a Minority-Serving R1 University † † cc-license: by 1. Introduction The public release of ChatGPT in November 2022 significantly altered undergraduate academic writing practices, surpassing the capacity of institutional frameworks to respond effectively. ChatGPT, a generative artificial intelligence (GenAI) model developed by OpenAI, is “specifically designed to generate human-like text in a conversational style” ( 14 , p. 228) . Within three academic years of its introduction, survey data show that approximately 88% of undergraduates reported using AI tools in their assessments ( 22 ) . Nevertheless, pedagogical, assessment, and policy frameworks governing such use have remained largely static. Academic research has paralleled the rapid adoption of these technologies, producing an expanding body of literature on prevalence, academic integrity risks, student perceptions, and the ethics of AI-generated text ( 14 ; 39 ) . Despite this proliferation, the literature lacks a theoretically grounded analysis of what it means, in qualitatively distinct terms, to rely on large language models (LLMs) during academic writing. The discipline learned to count how much students use these tools long before it learned to distinguish the kinds of use that deepen thought from those that quietly replace it. The limitation is structural. Existing research conceptualizes reliance on LLMs as a single, continuous frequency construct, categorizing students as users or non-users, or as frequent or infrequent users, and measuring outcomes by the extent to which GenAI contributes to their work. This approach obscures distinctions that are educationally significant. For instance, two students may submit the same research paper. One student may use GenAI to verify citations, assess the coherence of a drafted paragraph, and challenge claims about which she is uncertain. Another student may begin the assignment by prompting the AI and constructing the paper’s intellectual framework based on the AI’s output. According to all currently published measurement frameworks, these students are distinguished, if at all, only by frequency counts on a Likert scale. However, their cognitive engagement, developmental trajectories as writers, metacognitive regulation, and academic integrity practices differ fundamentally. Reliance on frequency measures alone cannot capture these distinctions. Frequency is the easiest property of reliance to measure and the least important to understand. The empirical consequences of this gap are increasingly apparent. Recent experimental studies demonstrate substantial cognitive costs associated with AI-assisted writing. For instance, students who utilize LLMs exhibit reduced neural connectivity and experience difficulty recalling content from essays they have just composed, a pattern referred to as accumulating “cognitive debt” ( 30 , p. 151) . Furthermore, AI-assisted writers engage in significantly fewer metacognitive activities compared to their unaided counterparts, a phenomenon termed “metacognitive laziness” ( 18 , p. 492) . Frequent AI use is also negatively correlated with critical thinking, as increased cognitive offloading reduces students’ “engagement in deep, reflective thinking” ( 23 , p. 2) . Although these findings are noteworthy, they share a common methodological limitation: each study compares users with non-users without a framework for distinguishing qualitatively different forms of AI engagement. Cognitively preserving AI engagement, which maintains metacognitive oversight and authorial control, is analytically indistinguishable from engagement that results in cognitive debt unless the measurement instrument is specifically designed to capture this distinction. A field that cannot distinguish cognitively preserving engagement from cognitive abdication will, in time, mistake the one for the other and build its evidence base upon the confusion. This research addresses the distinction by utilizing the AI Literacy Framework ( 35 ; 39 , p. 1) , Expectancy-Value Theory ( 54 ) , and Biggs’ ( 8 ) Presage-Process-Product Model. Using a sequential explanatory mixed-methods design with 382 undergraduates at a public minority-serving research university, the study identifies and validates four empirically distinct types of LLM reliance: Strategic, Instrumental, Dialogic, and Dependent. It further models the individual-difference predictors that determine both the type and intensity of reliance adopted by students. The research makes three primary contributions to the literature. First, it introduces a theoretically grounded and empirically validated typology that distinguishes qualitatively different forms of LLM reliance, thereby addressing the conceptual limitations of frequency-based frameworks. Second, it demonstrates that two distinct predictive systems govern reliance: AI literacy determines type membership, while Expectancy-Value beliefs determine intensity. These systems require separate and non-interchangeable interventions. Third, the study identifies a fundamental validity issue in existing LLM outcome measurement, the resolution of which is essential for advancing interpretive progress in the field. Why students rely on AI and how they rely on it answer to different forces, and an intervention aimed at one will not move the other. The third contribution warrants explicit articulation at the outset. The most theoretically significant finding of this study is not the typology itself, but rather the observation that Strategic users, those who are deliberate, critically evaluative, and academically principled, consistently score lowest on all outcome measures. A superficial interpretation may suggest that principled AI restraint is less effective than uncritical reliance. However, structural analysis indicates that existing outcome instruments in the LLM literature predominantly assess statements such as “I use generative AI tools to achieve X,” thereby measuring AI usage rather than writing quality or cognitive development. Students who intentionally restrict AI involvement in their work score low on these measures by design, not due to weak writing, but because their principled restraint results in lower AI-attributed achievement. Qualitative data in this study support this reinterpretation. What has been labeled as “outcomes” in the field is, at best, an indicator of AI’s contribution to student work and, at worst, a metric that systematically rewards dependence while mis-characterizing the most cognitively sophisticated students as the weakest performers. An instrument that scores the magnitude of AI’s contribution

Read the complete text