Comparing the Accuracy of Two Generated Large Language Models in Identifying Health-Related Rumors or Misconceptions and the Applicability in Health Science Popularization: Proof-of-Concept Study

Yuan Luo, Yiqun Miao, Yuhan Zhao, Jiawei Li, Yuling Chen, Yuexue Yue, Ying Wu, Amaryllis Mavragani · 2024

Read Comparing the Accuracy of Two Generated Large Language Models in Identifying Health-Related Rumors or Misconceptions and the Applicability in Health Science Popularization: Proof-of-Concept Study by Yuan Luo, Yiqun Miao, Yuhan Zhao, Jiawei Li, Yuling Chen, Yuexue Yue, Ying Wu, Amaryllis Mavragani (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11627524.txt.

Description

Yuan Luo and co-authors compare, in a 2024 proof-of-concept study, two large language models on spotting health rumors in Chinese. False health claims spread fast online and can trigger panic or bad choices. The team tested GPT-4 and Baidu's ERNIE Bot 4.0 on the task.

Twenty rumors and ten true statements were fed to both models, which had to judge each and explain why. Each model then wrote a short popular-science essay. Judges scored accuracy, quality, readability, and practical use. The head-to-head shows where chatbots can already help health communicators debunk myths, and where their answers still need a human check before reaching the public.

Catalog record

CREATORYuan Luo, Yiqun Miao, Yuhan Zhao, Jiawei Li, Yuling Chen, Yuexue Yue, Ying Wu, Amaryllis Mavragani
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC11627524

Browse the full library · About the Tower of Babel