Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers.

Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT. · 2023

Read Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. by Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT. (2023) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc10133283.txt.

Introduction

Gao and colleagues test whether ChatGPT-written research abstracts can be told apart from real ones, in a 2023 study. They gathered abstracts from five leading medical journals and asked ChatGPT to write new ones on the same topics. The question was whether detectors and human experts could spot the fakes.

Blinded reviewers and detection tools each tried to sort real from generated text. The study reports how often each method was fooled. The results feed a wider debate on trust and honesty. They also bear on AI use in science writing.

Catalog record

CREATORGao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT.
YEAR2023
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC10133283

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers By Catherine A. Gao, Frederick M. Howard, Nikolay S. Markov, Emma C. Dyer, Siddhi Ramesh, Yuan Luo, Alexander T. Pearson Published in: NPJ Digital Medicine DOI: 10.1038/s41746-023-00819-6 ABSTRACT Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. We gathered fifth research abstracts from five high-impact factor medical journals and asked ChatGPT to generate research abstracts based on their titles and journals. Most generated abstracts were detected using an AI output detector, ‘GPT-2 Output Detector’, with % ‘fake’ scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% ‘fake’ [12.73%, 99.98%] compared with median 0.02% [IQR 0.02%, 0.09%] for the original abstracts. The AUROC of the AI output detector was 0.94. Generated abstracts scored lower than original abstracts when run through a plagiarism detector website and iThenticate (higher scores meaning more matching text found). When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, though abstracts they suspected were generated were vaguer and more formulaic. ChatGPT writes believable scientific abstracts, though with completely generated data. Depending on publisher-specific guidelines, AI output detectors may serve as an editorial tool to help maintain scientific standards. The boundaries of ethical and acceptable use of large language models to help scientific writing are still being discussed, and different journals and conferences are adopting varying policies. The release of OpenAI’s free tool ChatGPT1 on November 30, 2022 demonstrated the ability of artificial intelligence models to generate content, with articles quickly published on its possible uses and potential controversies2–4. Early adopters have shared their experiences on social media, with largely positive sentiments5. Articles are bemoaning the death of the traditional school essay assignment4,6,7, as ChatGPT has been shown to generate high-scoring papers8, correctly answer USMLE questions9, and even articulate critical thinking10. The ethical and acceptable boundaries of ChatGPT’s use in scientific writing remain unclear11, although some publishers are beginning to lay down policies12–14. Large language models (LLM) are often complex neural network-based transformer models that can generate tone and content-defined text. These are trained on enormous amounts of data to predict the best next text element, which produces a product that reads naturally. ChatGPT is built on Generative Pre-trained Transformer-3 (GPT-3), which is one of the largest of these types of models, trained with 175 billion parameters15. These models generate coherent and fluent output, that can be difficult to distinguish from text written by humans16,17. Artificial intelligence (AI) has numerous applications in medical technologies18, and the writing of biomedical research is no exception, with products such as the SciNote Manuscript Writer19 or Writefull20 that help with scientific writing. However, with the release of ChatGPT, this powerful LLM technology is now available to all users for free, and millions are engaging with the new technology. The user base is likely to continue to grow. Thus, there is an urgent need to determine if ChatGPT can write convincing medical research abstracts. We gathered 50 abstracts from five high-impact journals as our control corpus of well-written abstracts. We asked ChatGPT to generate 50 scientific abstracts based on the titles and specific journals from this list (example subset in Supplementary Data 1). While all the output appeared superficially to be formatted as a scientific abstract, only 8 (16%) correctly used the headings particular to the specific journal in the prompt (e.g., Nature Medicine’s paragraph-style without headings, as opposed to specific headings such as ‘Design, Setting, and Participants’ for JAMA, see Supplementary Note 1 for examples). The patient cohort sizes were a similar order of magnitude between the original abstracts and the generated abstracts, with a Pearson correlation of the logarithmic cohort sizes of r = 0.76, p < 0.001 (Fig. 1).Fig. 1Generated abstracts have a similar patient cohort size as original abstracts.Cohort sizes from original abstracts (x-axis) and generated abstracts (y-axis) plotted on a logarithmic 10 scale. The AI output detector we used, ‘GPT-2 Output Detector’21,22, found a high probability of AI-generated output (higher % ‘fake’ score indicating more likely to be AI-generated text) in the generated abstracts with median [IQR] of 99.98% [12.73%, 99.98%] compared with very low probability of AI-generated output in nearly all the original abstracts with median [IQR] of 0.02% [0.02%, 0.09%] (Fig. 2a). The AI output detector had an area under the receiver operating characteristics (AUROC) curve of 0.94 for detecting generated abstracts (Fig. 2b). At the optimal cutoff maximizing sensitivity and specificity, 1.46%, the AI output detector had a sensitivity of 86% and a specificity of 94% at differentiating original versus generated abstracts.Fig. 2Many generated abstracts can be detected using an AI output detector.a AI detection scores as [% ‘fake’] per GPT-2 Output Detector for original abstracts and generated abstracts. Higher score indicates more likely to be generated by AI. b The AI output detector ROC curve for discriminating between original and generated abstracts, with AUROC of 0.94. We ran both original and generated abstracts through a free plagiarism-detection website, Plagiarism Detector20, and a paid professional similarity checker, iThenticate23. For both platforms, a higher score indicates more matching text was found. Original abstracts scored higher on the Plagiarism Detector website with median ‘plagiarized’ score 62.5% [IQR 43.25%, 84.75%] compared with generated abstracts with median ‘plagiarized’ score of 0% [IQR 0, 0] (Fig. 3a). Original abstracts also scored higher on iThenticate with median similar index of 100 [IQR 100, 100] compared with generated abstracts that had with median similarity index of 27 [IQR 19, 40.75] (Fig. 3b).Fig. 3Generated abstracts score lower than original abstracts on plagiarism detectors.a Plagiarism scores from plagiarism detector website, with higher % ‘plagiarized’ score indicating more matching text was found. b iThenticate Similarity Index for original abstracts and generated abstracts [%], with higher value meaning more similar text was found. Blinded human reviewers (FMH, NSM, ECD, SR) were given a mixture of real and generated abstracts and asked to provide a binary score of real or generated (Table 1). They were able to correctly identify 68% of generated abstracts as being generated, and correctly identified 86% of original articles as being original. They incorrectly identified 32% of generated abstracts as being real, and 14% of original abstracts as being generated. Our reviewers commented that abstracts they thought were generated by ChatGPT were superficial and vague, and sometimes focused on details of original abstracts such as inclusion of Clinical Trial Registration numbers and alternative spellings of words. The AI output detector scores were not statistically different (p = 0.45 by MWW) between the abstracts that reviewers correctly identified as generated and ones that they failed to identify as generated (Fig. 4).Table 1Human reviewer scoring for whether abstracts were real or generated, along with truth.TruthOriginalGeneratedReviewer guessOriginal4316Generated734Fig. 4Reviewers use criteria different than the AI output detector for flagging abstracts as either generated or original.The AI detection scores for generated abstracts were not significantly different (p = 0.45) between abstracts that human reviewers identified as

Read the complete text