Investigating generative AI models and detection techniques: impacts of tokenization and dataset size on identification of AI-generated text
Haowei Hua, Co-Jiayu Yao · 2024
Read Investigating generative AI models and detection techniques: impacts of tokenization and dataset size on identification of AI-generated text by Haowei Hua, Co-Jiayu Yao (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11611853.txt.
Description
Haowei Hua and Co-Jiayu Yao investigate how well detectors can spot AI-generated student essays, in a study from twenty twenty-four. Generative models now help students across subjects, raising cheating concerns. Existing detectors struggle with modified AI text and with second-language writers. The study focuses on high-stakes writing assessment.
Classical machine learning models were trained to separate AI essays from student writing. Transformer models including BERT and RoBERTa were compared against them. Several generators were tested in multiple versions, from base to latest. Paraphrasing tools that humanize AI text were thrown at the detectors. The authors introduce a synonym-based method to catch humanized text. They also explore how dataset size changes detection performance, informing future data collection.
Catalog record
| CREATOR | Haowei Hua, Co-Jiayu Yao |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |