Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing Publication type: Original Article Journal: Journal Volume: 00

Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan · 2025

Read Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing Publication type: Original Article Journal: Journal Volume: 00 by Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04398.txt.

Introduction

Jin, Yang, Martinez-Maldonado, Gašević, and Yan study in twenty twenty-five how the design of an AI writing helper shapes student writing. Medical and nursing students wrote data-based essays with one of two helpers. One helper answered only when asked. The other took initiative, posing staged questions and feedback along the way.

Proactive help tied conceptual reasoning, evidence use, and engagement more tightly together. Reactive help stayed more factual and step-based. Students' own AI literacy predicted how well they wrote alone after help was removed, above all for using visual data. The literacy gap showed under the reactive design but not the proactive one. Learners asked for feedback that fits context and dialogue.

Catalog record

CREATORYueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-04398

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing Publication type: Original Article Journal: Journal Volume: 00 Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan Abstract Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments. While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students’ GenAI literacy influences their independent multimodal writing skills or how chatbot interaction strategies (passive reactive vs. proactive scaffolding) impact learning. This study examined 79 higher education students’ multimodal academic writing performance using a comparative research design. Students completed writing tasks integrating visual data under two chatbot-assisted conditions (passive vs. proactive) and subsequently without AI assistance. Their writing performance was rigorously evaluated across five dimensions, including insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Ordinal logistic regression and correlation analyses revealed that higher levels of GenAI literacy significantly predicted stronger independent multimodal writing performance immediately after AI assistance removal, particularly for students using passive chatbots requiring active prompting. These results highlight the critical role of GenAI literacy and specific chatbot interaction strategies in shaping students’ capacities for independent multimodal academic writing. Our findings emphasise the need for purposeful integration of GenAI literacy training into curricula and balancing external scaffolding support with autonomous learning opportunities. This research offers valuable recommendations for educators leveraging AI-enhanced pedagogies to optimise student writing outcomes and technological engagement strategies. keywords generative AI literacy, academic writing, chatbot interaction, multimodal, skill development † † copyright-year: 2025 † † startpage: 1 † † titlemark: Do Students Write Better Post-AI Support? † † corresponding: Corresponding author Lixiang Yan, Centre for Learning Analytics Monash (CoLAM), Faculty of Information Technology, Monash University, \state Victoria, 3168, Australia Jin et al 1 * Practitioner Notes What is currently known about this topic • Generative AI (GenAI) tools are increasingly used by learners in their academic writing tasks, offering new opportunities and challenges. • Effective use of advanced AI tools depends partly on students’ technological literacy and their ability to strategically engage with AI-generated content. • The academic writing process is becoming increasingly multimodal, requiring integration of textual and visual data, yet limited research exists on how GenAI tools influence learner performance in such multimodal contexts. What this paper adds • Provides empirical evidence that learners’ GenAI literacy significantly predicts their independent multimodal academic writing performance, especially after AI supports are removed. • Demonstrates that passive chatbot interactions benefit students with higher GenAI literacy more strongly than proactive chatbot interactions, highlighting interaction style as a crucial moderating variable. • Confirms GenAI literacy as a distinct predictor of writing performance, separate from domain-specific knowledge, indicating its importance for successfully integrating visual information and critical-thinking components into AI-assisted writing tasks. Implications for practitioners • Educators should explicitly integrate GenAI literacy training within academic curricula, focusing on skills like formulating effective prompts, critically evaluating AI-generated output, and independently integrating this assistance into academic work. • Practitioners need to carefully select chatbot scaffolding strategies appropriate for students’ GenAI literacy, balancing proactive guidance with opportunities for learners’ active inquiry, metacognitive awareness, and autonomous problem-solving. • Instructional designers should strategically balance GenAI-driven scaffolding with independent practice opportunities for students to independently apply and develop their GenAI literacy skills. 2 Introduction Academic writing is an essential practice in higher education. It helps students develop critical thinking, effective communication, and the ability to deal with complex ideas and information [ 33 , 46 ] . Students need to interpret and synthesise data, create logical arguments, and assess different perspectives [ 13 , 47 ] . Increasingly, academic writing also involves multimodal literacy, requiring students to accurately interpret, synthesise, and incorporate varied visual data sources, such as graphs, charts, or images, into their arguments. This skill reflects the increasingly multimodal nature of contemporary academic discourse and demands additional capabilities concerning visual-data integration [ 34 , 46 , 27 ] . Academic writing is crucial for both academic success and professional competency [ 53 , 59 ] . Employers value strong academic writing skills as they help individuals write persuasive reports, create coherent presentations, and solve problems effectively [ 12 ] . Yet, academic writing is linked to cognitive and metacognitive processes. It encourages students to monitor and control their thought processes, facilitating metacognitive awareness and self-regulation [ 18 ] . This engagement is essential for students’ ability to plan, evaluate, and reflect on their learning strategies, leading to a deeper understanding and better retention of information [ 21 ] . As technology advances, academic writing can evolve from being solely an individual process to one supported by various tools [ 45 ] . With the increasing availability of generative AI (GenAI) tools (e.g., ChatGPT and Gemini), students have more opportunities to enhance their academic writing performance [ 62 , 4 ] . Using GenAI tools in writing is becoming more common [ 30 ] , making it urgent to understand how these external supports impact students’ skill development [ 39 ] . GenAI tools can provide feedback, generate content ideas, or assist in structuring arguments, offering new ways for students to engage with writing tasks [ 7 ] . However, while these tools present exciting possibilities, it is critical to examine their influence on learners’ development of independent writing skills [ 63 ] . Understanding this dynamic will guide educators in integrating technology effectively into writing instruction, ensuring that while students benefit from technological aids, they also continue to grow their critical thinking and independent writing abilities [ 45 , 7 ] . Indeed, emerging studies are beginning to explore how GenAI can produce high-quality writing and support student writing processes [ 38 , 30 , 39 ] . Most of these studies focus on how GenAI can assist humans in writing and the patterns that emerge from this interaction [ 45 , 62 ] . However, existing research predominantly addresses text-only writing contexts, leaving uncertainties about the role of GenAI in supporting multimodal academic writing involving visual information [ 27 , 34 ] . There is also little empirical evidence on how GenAI tools affect students’ academic writing performances, especially when these tools are taken away after they have been used. Using GenAI tools might merely create the illusion of performance enhancement , but it might not necessarily lead to learning and skill improvements [ 16 , 63 ] . Additionally, few studies have thoroughly examined how different interaction strategies of GenAI, such as passively replying versus proactively scaffolding, impact student writing. Understanding interaction strategy is important because it could shape how students engage with these tools and influence the development of their writing skills [ 45 , 29 ] . A passive interaction might mean the tool only responds to commands, while proactively scaffolding could guide students step-by-step and provide feedback [ 63 ] . Each style can affect how independent and critical students become in their writing. Thus, it is crucial to understand if dependence on GenAI affects students’ writing skills and independence. Research in this area can help educators balance the use of GenAI technology, making sure students can take advantage of GenAI while still growing

Read the complete text