Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media

Zhen Sun Zongmin Zhang Xinyue Shen Ziyi Zhang, Yule Liu Michael Backes Yang Zhang Xinlei He · 2024

Read Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media by Zhen Sun Zongmin Zhang Xinyue Shen Ziyi Zhang, Yule Liu Michael Backes Yang Zhang Xinlei He (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18148.txt.

Introduction

Zhen Sun and co-authors measure how much of social media text is now written by AI. Machine text can spread false claims and bend public debate, so its scale matters. Their 2024 study gathers about two and a half million posts from three big platforms. They also build a test set from human posts and text made by twelve language models.

With a tuned detector, the team tracked the AI share from early 2022 to late 2024. On two platforms the share soared from about two percent to nearly forty. On the third it crept up only a bit. Machine posts differed from human ones in style, topics, reach, and who posts them. The authors say the trends call for steady watching.

Catalog record

CREATORZhen Sun Zongmin Zhang Xinyue Shen Ziyi Zhang, Yule Liu Michael Backes Yang Zhang Xinlei He
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18148

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media Zhen Sun Zongmin Zhang Xinyue Shen Ziyi Zhang, Yule Liu Michael Backes Yang Zhang Xinlei He Abstract Social media platforms are experiencing a growing presence of AI-Generated Texts (AIGTs). However, the misuse of AIGTs could have profound implications for public opinion, such as spreading misinformation and manipulating narratives. Despite its importance, it remains unclear how prevalent AIGTs are on social media. To address this gap, this paper aims to quantify and monitor the AIGTs on online social media platforms. We first collect a dataset ( SM-D ) with around 2.4 ​ M 2.4M posts from 3 3 major social media platforms: Medium, Quora, and Reddit. Then, we construct a diverse dataset ( AIGTBench ) to train and evaluate AIGT detectors. AIGTBench combines popular open-source datasets and our AIGT datasets generated from social media texts by 12 12 LLMs, serving as a benchmark for evaluating mainstream detectors. With this setup, we identify the best-performing detector ( OSM-Det ). We then apply OSM-Det to SM-D to track AIGTs across social media platforms from January 2022 to October 2024, using the AI Attribution Rate (AAR) as the metric. Specifically, Medium and Quora exhibit marked increases in AAR, rising from 1.77 % 1.77\% to 37.03 % 37.03\% and 2.06 % 2.06\% to 38.95 % 38.95\% , respectively. In contrast, Reddit shows slower growth, with AAR increasing from 1.31 % 1.31\% to 2.45 % 2.45\% over the same period. Our further analysis indicates that AIGTs on social media differ from human-written texts across several dimensions, including linguistic patterns, topic distributions, engagement levels, and the follower distribution of authors. We envision our analysis and findings on AIGTs in social media can shed light on future research in this domain. Our code and dataset are publicly available. 1 1 1 Code & Dataset: https://github.com/TrustAIRLab/AIGT_on_Social_Media . Introduction The rapid development of Large Language Models (LLMs) has markedly enhanced the quality of AIGTs, enabling the use of models like GPT-3.5 [ 42 ] in daily life to produce high-quality texts, such as in academic writing [ 14 ] , question-answering [ 23 ] , and translation [ 65 ] . These AIGTs are often indistinguishable from Human-Written Texts (HWTs), presenting AIGT detection as a crucial yet challenging task for effective classification. On social media platforms, the use of LLMs to answer questions can contribute to the spread of misinformation [ 72 ] . Furthermore, AIGTs may be deliberately used for information manipulation or the dissemination of fake news, potentially resulting in serious societal impacts [ 16 ] . To better understand the prevalence of AIGTs on social media platforms, we aim to quantify and monitor their presence, addressing the question: On social media, are we already interacting with AI-generated texts? Figure 1 : Pipeline for quantifying AIGTs on social media: SM-D (2.4M posts), AIGTBench (training benchmark), OSM-Det (optimal detector). Currently, numerous detectors have been developed to detect AIGTs. According to the MGTBench [ 17 ] , these detectors are broadly divided into two categories: metric-based [ 12 , 39 ] and model-based detectors [ 21 , 54 , 4 ] , some of which have shown high accuracy and robustness. While these detectors have been applied in controlled settings, recent studies have explored their effectiveness in real-world scenarios. Hanley and Durumeric [16] conduct AIGT detection on news website articles, with a primary focus on content generated by GPT-3.5 and others from Turing benchmark, which includes various pre-2022 models [ 62 ] . Furthermore, Liu et al. [31] detects ChatGPT-generated content on arXiv papers. However, academic and news writing are formal and tailored to specific audiences, whereas social media content is more interactive, making it a better domain for observing AIGTs’ impact on daily life. Moreover, previous studies do not account for recent popular LLMs, while we consider a broader range of models in our efforts to detect AIGTs on social media. To quantify and monitor AIGTs on social media, we collect textual data from 3 popular platforms spanning January 1, 2022, to October 31, 2024, as most LLMs are released after 2022. After data preprocessing, we obtain 1,170,821 posts from Medium, 245,131 answers from Quora, and 982,440 comments from Reddit. We name it as SM-D , short for S ocial M edia D ataset. To identify the most effective detector, we construct a dataset named AIGTBench , which consists of public AIGT/Supervised-Finetuning (SFT) datasets and our own AIGT datasets generated from social media data. AIGTBench includes AIGTs generated by 12 different LLMs, such as GPT Series [ 44 ] ) and Llama Series [ 60 , 61 , 10 ] ), totaling around 28.77M AIGT and 13.55M HWT samples. We then benchmark AIGT detectors on AIGTBench and leverage the best-performing detector as our primary detector, which achieves an accuracy of 0.979 and an F1-score of 0.980 . To better reflect its application in detecting AIGTs on online social media, we rename it as OSM-Det ( O nline S ocial M edia Det ector). Based on OSM-Det , we quantify and monitor the texts across the 3 platforms and use the AI Attribution Rate (AAR) to represent the rate of posts classified as AI-generated (The pipeline is shown in Figure 1 ). We observe several noteworthy phenomena: (1) A sharp rise in AI-generated content begins in December 2022, with distinct AAR trends emerging across platforms. Before December 2022, the AAR across platforms remains stable. However, starting in December, Medium and Quora show significant surges, while Reddit shows only a slight increase. This suggests the widespread and diverse LLM adoption on social media; (2) Linguistic analysis shows similar AAR trends and exhibits stylistic features in AIGTs/HWTs. Based on the word-level analysis, we find that the usage trend of top-frequency AI-preferred words aligns closely with LLM adoption trends. With sentence-level analysis, we also reveal that AIGTs tend to be more objective and standardized, whereas HWTs are more flexible and informal; (3) Technology-related topics drive higher AARs on Medium. Topics like “Technology” and “Software Development” show the highest AARs, indicating that users with a strong technical background are more likely to adopt LLMs; (4) Predicted HWTs receive more engagement than AIGTs. On Medium, the content predicted as HWTs receives more average “Likes” and “Comments” than AIGTs. This suggests that users are more inclined to engage with HWTs; and (5) Authors with fewer followers are more likely to produce AIGTs. On Medium, users with no more than one thousand followers tend to produce content that has the highest mean AAR at 54.02\% . In contrast, as the follower count increases, the AAR gradually shifts toward the lower range ( \leq 25.00\% ). Our contributions are summarized as follows: • We are the first to conduct a systematic study to quantify, monitor, and analyze AIGTs on social media. To achieve this, we collect a large-scale dataset SM-D , which includes around 2.4M posts from three platforms, spanning from January 2022 to October 2024. • We construct AIGTBench , a dataset for benchmarking AIGT detectors. AIGTBench can be divided into two parts: one derived from open-source datasets and the other generated by 12 LLMs based on platform-specific characteristics. Leveraging AIGTBench ,

Read the complete text