Generative Propaganda (Working Paper)

Madeleine I. G. Daepp, Alejandro Cuevas, Robert Osazuwa Ness, Vickie Yu-Ping Wang, Bharat Kumar Nayak, Dibyendu Mishra, Ti-Chung Cheng, Shaily Desai, Joyojeet Pal · 2025

Read Generative Propaganda (Working Paper) by Madeleine I. G. Daepp, Alejandro Cuevas, Robert Osazuwa Ness, Vickie Yu-Ping Wang, Bharat Kumar Nayak, Dibyendu Mishra, Ti-Chung Cheng, Shaily Desai, Joyojeet Pal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-19147.txt.

Introduction

Madeleine I. G. Daepp and eight co-authors, in a working paper, study how generative AI is actually used to sway public opinion. They interviewed defenders such as fact-checkers, journalists, and officials in Taiwan, and creators such as influencers and political consultants in India and Taiwan. Both places see heavy online propaganda. The word deepfakes, they find, dominates defenders' fears and skews which countermeasures get priority.

The authors build a taxonomy that splits uses by whether AI involvement is obvious or hidden, and whether the aim is to promote or to tear down. Deception was rarely the point: Indian creators often flaunted AI use to cut legal risk, seeking persuasion instead. The real gains were efficiency, working across languages and formats, and slipping past human and algorithmic detection. They urge security researchers to widen their threat models beyond deepfakes.

Catalog record

CREATORMadeleine I. G. Daepp, Alejandro Cuevas, Robert Osazuwa Ness, Vickie Yu-Ping Wang, Bharat Kumar Nayak, Dibyendu Mishra, Ti-Chung Cheng, Shaily Desai, Joyojeet Pal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-19147

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generative Propaganda (Working Paper) Madeleine I. G. Daepp, Alejandro Cuevas, Robert Osazuwa Ness, Vickie Yu-Ping Wang, Bharat Kumar Nayak, Dibyendu Mishra, Ti-Chung Cheng, Shaily Desai, Joyojeet Pal Abstract Generative propaganda is the use of generative artificial intelligence (AI) to shape public opinion. To characterize its use in real-world settings, we conducted interviews with defenders (e.g. factcheckers, journalists, officials) in Taiwan and creators (e.g. influencers, political consultants, advertisers) as well as defenders in India, centering two places characterized by high levels of online propaganda. The term “deepfakes”, we find, exerts outsized discursive power in shaping defenders’ expectations of misuse and, in turn, the interventions that are prioritized. To better characterize the space of generative propaganda, we develop a taxonomy that distinguishes between obvious versus hidden and promotional versus derogatory use. Deception was neither the main driver nor the main impact vector of AI’s use; instead, Indian creators sought to persuade rather than to deceive, often making AI’s use obvious in order to reduce legal and reputational risks, while Taiwan’s defenders saw deception as a subset of broader efforts to distort the prevalence of strategic narratives online. AI was useful and used, however, in producing efficiency gains in communicating across languages and modes, and in evading human and algorithmic detection. Security researchers should reconsider threat models to clearly differentiate deepfakes from promotional and obvious uses, to complement and bolster the social factors that constrain misuse by internal actors, and to counter efficiency gains globally. I Introduction Generative AI can now produce text, audio, and images that most people cannot distinguish from real content ( Frank et al. 2024 ; Diel et al. 2024 ) . Major breakthroughs occurred just ahead of 2024—dubbed “the biggest election year in history”—when more than half the world’s population participated in major elections ( The Economist 2023 ) . Elections are consistently targeted by information manipulation and interference campaigns ( Bradshaw and Howard 2019 ) , leading to widespread predictions that 2024 would see a “deepfake deluge,” an inundation of malicious and misleading AI-generated depictions of people or events ( Rifkind 2024 ; Goldman 2024 ; World Economic Forum 2024 ) . The potential of generative AI to produce deepfakes is unsettling. There is a long-standing history of sudden and massive reversals in political fortunes based on single pieces of information ( Reyes 2006 ; Twomey et al. 2023 ) . High-profile cases of viral AI-generated content have amplified concerns, including a 2019 video of former U.S. President Barack Obama (specifically intended to highlight the technology’s risks in politics) ( Silverman 2018 ) and a 2023 image of a bombed Pentagon that catalyzed a stock-market sell-off (though the market quickly recovered) ( Sorkin et al. 2023 ) . In the global 2024 elections cycle, however, the use of generative AI to convince people of candidate interactions and events that did not, in fact, occur, was limited relative to researchers’ expectations ( Schneier and Sanders 2024 ) . 1 1 1 There are several important exceptions in which fraudulent deepfakes were both used and considered impactful ( Meaker 2023 ; Factcheck Bureau 2024 ) ; for a detailed review of AI incidents in 2024, see Trauthig et al. 2025 . While there is little doubt that deepfakes in and of themselves are concerning, this article shows that the focus on antagonistic deception obscures a much broader space of generative AI’s use and misuse in political communications. Through in-depth study of generative AI’s use in Taiwan and India, two places with national elections in 2024 and subject to high levels of computational propaganda ( Lührmann et al. 2019 ; World Economic Forum 2024 ) , we show that observed uses were a superset of those that journalists, fact-checkers, and other front-lines defenders expected. Developing a taxonomy that differentiates obvious versus hidden and promotional versus derogatory use, we find that influencers, political consultants, and other creators make AI’s use obvious or promotional in order to minimize potential legal and reputational costs; social context further serves as a key bottleneck on whether content resonates or attains reach. Notably, these explanations contrast with common technologically determinist explanations that center limitations on quality, cost, and distribution as defining constraints on deepfakes’ use ( Simon et al. 2023 ; Kapoor and Narayanan 2024 ; Kapoor and Narayanan 2023 ) . We further provide evidence of additional adversarial uses beyond deceptive representations. In particular, creators and defenders describe the use of “AIPasta”—the AI-based perturbation of online communications to introduce authentic-seeming variation ( Dash et al. 2025 ) —and “Precision Propaganda”, or AI’s use to tailor messaging to narrow target audiences. The primary risk, we find, lies not in photorealism but in new efficiency gains—–lower detectability, multilingual reach, and multimodal output—–that enable actors to scale information manipulation across places and platforms, and for which existing social defenses provide little restraint. Our work broadens the evidence of AI’s use beyond representational deepfakes to highlight growing evidence of its value both for obvious and promotional representations as well as for adversarial efforts to attain narrative dominance. To respond effectively, security researchers should account for uses and misuses beyond deepfakes, prioritize interventions that match the scale of AI’s efficiency gains, and recognize that social and technical defenses will serve as necessary complements in robust defensive systems. Our article makes four contributions. 1. We collected 64 hours of interviews with 72 participants across two of the places most affected by computational propaganda globally, documenting front-line perspectives on the use and misuse of a major emerging technology. 2. We provide a taxonomy to differentiate adversarial deepfakes from other representational uses, showing how persuasion and distortion motivated observed patterns motivated applications beyond antagonistic deception. 3. We document observed threat actors and their constraints, including legal fears, reputational costs, and social context—factors that differ from leading prior theories. 4. We provide detailed insights to inform threat models and interventions, highlighting the importance of tailoring mitigations to the sociotechnical context of a given place. II Literature Review Our study is rooted in three related bodies of work: (1) studies monitoring global information disorder; (2) interdisciplinary examinations of generative AI’s misuse in digital communications, particularly via deepfakes; and (3) qualitative research developing human-centered frameworks for threat modeling in the security literature. II-A Global Information Disorder Global information disorder refers to the widespread use of computational propaganda—political messaging produced and amplified via automated, algorithmic, and surveillance technologies ( Bradshaw and Howard 2019 ) —to manipulate digital public spheres, the online spaces where public opinions are formed ( Habermas 2023 ) . Although early research focused on misinformation and disinformation—-unintentional and intentional deception, respectively ( Jack 2017 ) —researchers increasingly recognize that antagonistic falsehoods are just one subset of the broader class of efforts to manipulate collective sensemaking processes online ( Starbird 2024 ) . The set of countries affected has expanded rapidly, more than doubling from 28 in 2017 to 70 in 2019 ( Bradshaw and Howard 2019 ) ; effectively, all democracies now confront some form of online information disorder. Despite conflicting research on the efficacy of such efforts in effectively deceiving targets ( Budak et al. 2024 ) , there is now strong evidence to show that existing schisms in society are not just

Read the complete text