Analyzing Security and Privacy Challenges in Generative AI Usage Guidelines for Higher Education
Bei Yi Ng, Jiarui Li, Xinyuan Tong, Kevin Ye, Gauthami Yenne, Varun Chandrasekaran, Jingjie Li · 2025
Read Analyzing Security and Privacy Challenges in Generative AI Usage Guidelines for Higher Education by Bei Yi Ng, Jiarui Li, Xinyuan Tong, Kevin Ye, Gauthami Yenne, Varun Chandrasekaran, Jingjie Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20463.txt.
Introduction
Bei Yi Ng and co-authors read university rulebooks for generative AI with a security eye, in a 2025 paper. The tools reshaping study are run by a few private firms with murky data habits, while students often share sensitive material unknowingly. Universities have answered with usage guidelines meant to balance innovation against integrity, privacy, and safety.
The team analyzed such guidelines from forty-three universities in twelve countries, hunting the privacy and security content. Gaps recur: schools struggle to deploy real privacy measures and to adopt known security frameworks. The findings lay groundwork for guidelines that protect as well as permit. Policy, the authors imply, is still catching up with practice.
Catalog record
| CREATOR | Bei Yi Ng, Jiarui Li, Xinyuan Tong, Kevin Ye, Gauthami Yenne, Varun Chandrasekaran, Jingjie Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-20463
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Analyzing Security and Privacy Challenges in Generative AI Usage Guidelines for Higher Education Bei Yi Ng, Jiarui Li, Xinyuan Tong, Kevin Ye, Gauthami Yenne, Varun Chandrasekaran, Jingjie Li Abstract . Educators and learners worldwide are embracing the rise of Generative Artificial Intelligence (GenAI) as it reshapes higher education. However, GenAI also raises significant privacy and security concerns, as models and privacy-sensitive user data, such as student records, may be misused by service providers. Unfortunately, end-users often have little awareness of or control over how these models operate. To address these concerns, universities are developing institutional policies to guide GenAI use while safeguarding security and privacy. This work examines these emerging policies and guidelines, with a particular focus on the often-overlooked privacy and security dimensions of GenAI integration in higher education, alongside other academic values. Through a qualitative analysis of GenAI usage guidelines from universities across 12 countries, we identify key challenges and opportunities institutions face in providing effective privacy and security protections, including the need for GenAI safeguards tailored specifically to the academic context. 1. Introduction Generative AI (GenAI) is rapidly transforming higher education; students and educators use it for writing, programming, research, and data analysis ( jurenka-arxiv24 ) . Its versatility in addressing open-ended queries has made it a core tool in academic workflows ( hu2025generative ) . As adoption grows, traditional educational technology providers are embedding GenAI into services like writing support and virtual collaboration ( chang2024survey ) . In parallel, extensive research is underway to better understand AI-human collaboration in education—ranging from the design of intuitive interfaces to investigations of GenAI’s pedagogical impact to the broader shifts it introduces in teaching and learning paradigms ( GenAI2025HigherEd ; Quartararo2023GenAIImpact ; Cornell2023GenAIReport ) GenAI systems continually improve by leveraging vast datasets ( liu2024understanding ) , including (potential) user interactions ( OpenAI2024DataControls ) . In academic settings, GenAI is integrated into tools like AI-enhanced learning management systems and adaptive tutoring applications. However, GenAI providers’ (e.g., OpenAI) data practices often lack transparency, raising serious concerns ( Hardinges2024Transparency ) . For example, the collapse of AllHere Education—the company behind Los Angeles Unified School District’s AI chatbot ‘Ed’—sparked questions about the fate of sensitive student data it collected ( edsource-url25 ) . Such incidents highlight the need for stronger oversight in handling educational data. In the U.S., the Family Educational Rights and Privacy Act (FERPA) ( ferpa2011 ) regulates how student data is accessed and shared, while similar protections exist globally: the EU’s General Data Protection Regulation (GDPR) ( GDPR-url25 ) enforces strict data minimization and consent requirements, and Australia’s Privacy Act 1988 ( oaic-url25 ) governs the handling of personal information in schools. These regulatory frameworks compel institutions to implement robust safeguards when adopting GenAI. While the benefits of GenAI in higher education are undeniable—enhancing productivity, creativity, and efficiency—its adoption is accelerating within a “lemon market,” where users lack clear visibility into model quality, risks, and trade-offs ( chan-ijethe23 ) . Students and educators often rely on GenAI tools without a full understanding of their limitations, such as susceptibility to misinformation ( huang2025survey ) , bias ( kadaruddin-ijble23 ) , and over-reliance ( bastani2024generative ) . As policy lags behind technological advancement, universities have a vital role to play in shaping responsible GenAI adoption through clear institutional guidelines, safeguards, and pedagogical frameworks ( chan-ijethe23b ) . Apart from principles of academic integrity, recent studies identify privacy and security as key motivators in universities’ and instructors’ policies about GenAI usage ( Jin_Yan_Echeverria_Gašević_Martinez-Maldonado_2025 ; McDonald_Johri_Ali_Collier_2025 ; ali2025-cse25 ; jiao-nature25 ) . However, an in-depth understanding of how universities approach and assess these privacy and security risks of GenAI, and in particular how they communicate the trade-offs between potential mitigation strategies and benefits, is still missing. Our study aims to bridge this gap. By analyzing GenAI usage guideline documents published online by universities globally, we aim to address the following three research questions: RQ1. How do university guidelines address privacy and security risks of GenAI, such as sensitive data breach and technical vulnerabilities? RQ2. How do universities evaluate and suggest different measures to mitigate the aforementioned risks? RQ3. How are privacy and security positioned among other guiding principles in policy design? We conducted qualitative content analysis for GenAI guidelines from 43 universities spanning 12 countries. Our study reveals in-depth findings corresponding to the privacy and security issues of GenAI in academic settings. We summarize the main findings corresponding to our research questions below: F1. Universities broadly recognize the privacy and security risks posed by GenAI, particularly those that arise from sensitive data usage and vulnerabilities of GenAI’s system integration in academic settings. Yet, the guidelines can often be vague, conflating technical terms that refer to sensitive data and omitting specific discussions of security risks and malicious attacks in academic settings. F2. University guidelines present varying degrees of clarity and implementation consistency. They do this for mitigation strategies of privacy and security risks through technical safeguards, operational measures at the institutional level, and individual right and responsibility. Their privacy and security measures build on existing technical solutions and frameworks, while they recognize their limitations, e.g., reliability and difficulty in enforcement. F3. Institutional policies tend to frame privacy and security within broader discussions of academic values and ethics. Beyond institutional values, universities in different countries are also influenced by external authorities—national agencies, inter-university networks, and policy exemplars—highlighting both a reliance on existing frameworks and the potential for collaborative knowledge transfer. However, the alignment of privacy and security with other academic values varies across different universities and is subject to individual interpretation. Based on our findings, we propose several recommendations to improve privacy and security for universities, educators, and students adopting GenAI. These include improving privacy and security assessment frameworks that considered GenAI-specific threats and risks, tailoring technical safeguards to academic contexts, as well as reducing infrastructure and communication disparities for safeguarding GenAI across universities and academic disciplines. 2. Related Work GenAI in education. GenAI has rapidly gained traction across the higher education sector, permeating both student and faculty practices ( chan2023students ; yusuf2024generative ; francis2025generative ; tillmanns2025mapping ; luo2024critical ) . Students are leveraging GenAI tools for drafting essays, solving mathematical problems, summarizing readings, generating code, translating texts, and preparing for exams through simulated question-answering ( Giannakos_Azevedo_Brusilovsky_Cukurova_Dimitriadis_Hernandez-Leo_Järvelä_Mavrikis_Rienties_2024 ) . Simultaneously, educational professionals are incorporating GenAI into teaching workflows—using it to develop lecture materials, generate quiz banks, provide formative feedback, assist in grading, and personalize instruction through intelligent tutoring systems and adaptive learning platforms ( udeh2025role ; qi2025role ) . Despite this growing adoption, the understanding of GenAI technologies within educational contexts remains fragmented. Many users—students and instructors alike—engage with GenAI tools without a clear grasp of their operational limitations or risks in education. One major concern is the generation of false or misleading content (i.e., “hallucinations”), which can misinform students or degrade the quality of instruction when integrated into teaching materials ( elsayed2024impact ) . This is particularly problematic in activities requiring factual accuracy where hallucinated outputs can introduce conceptual errors or propagate misinformation that is plausible-sounding ( huang2025survey ; xie2024survey ) . Another concern is in coding and programming contexts, where GenAI tools may produce code that compiles successfully but fails at runtime, silently introducing logic