Detecting the corruption of online questionnaires by artificial intelligence
Benjamin Lebrun, Sharon Temtsin, Andrew Vonasch, Christoph Bartneck · 2024
Read Detecting the corruption of online questionnaires by artificial intelligence by Benjamin Lebrun, Sharon Temtsin, Andrew Vonasch, Christoph Bartneck (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-10869497.txt.
Description
Online surveys are cheap and easy, but language models now let cheats fill them in automatically, complete with plausible free-text answers. Benjamin Lebrun and co-authors test whether such AI-written responses can be caught. Human judges spotted the machine authorship with seventy-six percent accuracy — better than chance, yet far below what clean data would need.
Automatic AI detectors, by contrast, were completely unusable. The authors warn that researchers currently stay safe mainly because fraudsters lack interest, not because defences work. If fake submissions grow common, checking them could cost more than the survey saves, and simple attention checks will not rescue data quality. The fix, they argue, must come from the crowdsourcing platforms themselves, since neither human vigilance nor detector software can carry the load.
Catalog record
| CREATOR | Benjamin Lebrun, Sharon Temtsin, Andrew Vonasch, Christoph Bartneck |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 115 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |