Aletheia : Combating Social Media Influence Campaigns with Graph Neural Networks

Mohammad Hammas Saeed, Isaiah J. King, Howie Huang · 2025

Read Aletheia : Combating Social Media Influence Campaigns with Graph Neural Networks by Mohammad Hammas Saeed, Isaiah J. King, Howie Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-21391.txt.

Introduction

Mohammad Hammas Saeed, Isaiah J. King, and Howie Huang present ALETHEIA, a system built to detect coordinated influence campaigns on social media. Such campaigns use networks of malicious or troll accounts to sway public discussion, and moderators need tools that can spot the accounts and predict what they will do next.

ALETHEIA represents a campaign as a graph and combines network structure with language features, feeding them to graph neural networks that can scale to large networks. Tested on campaigns from several countries on Reddit and X, it improves detection scores over standard methods that use only interaction and user features. A second component stacks a graph network over a recurrent network to forecast future links, predicting which users trolls will target next with high accuracy.

Catalog record

CREATORMohammad Hammas Saeed, Isaiah J. King, Howie Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-21391

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Aletheia : Combating Social Media Influence Campaigns with Graph Neural Networks Mohammad Hammas Saeed, Isaiah J. King, Howie Huang Abstract Influence campaigns are a growing concern in the online spaces. Policymakers, moderators and researchers have taken various routes to fight these campaigns and make online systems safer for regular users. To this end, our paper presents Aletheia , a system that formalizes the detection of malicious accounts (or troll accounts ) used in such operations and forecasts their behaviors within social media networks. We analyze influence campaigns on Reddit and X from different countries and highlight that detection pipelines built over a graph-based representation of campaigns using a mix of topological and linguistic features offer improvement over standard interaction and user features. Aletheia uses state-of-the-art Graph Neural Networks (GNNs) for detecting malicious users that can scale to large networks and achieve a 3.7% F1-score improvement over standard classification with interaction features in prior work. Furthermore, Aletheia employs a first temporal link prediction mechanism built for influence campaigns by stacking a GNN over a Recurrent Neural Network (RNN), which can predict future troll interactions towards other trolls and regular users with an average AUC of 96.6%. Aletheia predicts troll-to-troll edges (TTE) and troll-to-user edges (TUE), which can help identify regular users being affected by malicious influence efforts. Overall, our results highlight the importance of utilizing the networked nature of influence operations (i.e., structural information) when predicting and detecting malicious coordinated activity in online spaces. 1 Introduction Influence campaigns are coordinated online efforts to alter public opinion, craft malicious narratives, and push agendas in online spaces. Extensive prior research highlights influence operations and their activities on various social media platforms, e.g., Twitter (now X) and Reddit [ 55 , 57 , 58 , 59 ] . However, as it stands, malicious influence operations still remains a pertinent issue in cyberspace [ 22 , 45 , 7 ] . These operations are often carried out through dedicated accounts (also referred to as troll accounts ). Troll accounts operate by intermingling with real users on a platform, often replying and operating in groups together to simulate a real conversation [ 33 , 48 ] . This aspect of imitating regular behavior makes it difficult to detect troll behavior, since they are distinct from standard social media bots in that they create conversation like real users instead of simply broadcasting template messages, although that might be one of their goals [ 43 ] . Motivation. Prior research shows that troll accounts are loosely coordinated actors that rely on pushing each other’s messages [ 42 ] , and often maintain online personas to interact with the general public [ 26 ] . They strategically interject themselves into social media spaces and strategize to spread their agendas, e.g., excessive retweeting in groups and networks of accounts [ 50 ] . Since these accounts are multi-intent actors, i.e., they reshare information, interact with users and also work together in groups, their reliance on collaborative efforts makes them ideal candidates to be modeled in the form of a network. Therefore, in this work, we posit the detection of troll accounts and predicting their future behavior as node detection and link prediction tasks in a social media graph. Research Problem. Although past work highlights the various characteristics of influence campaigns [ 53 , 39 , 50 ] and uses some of their characteristics to develop detection systems [ 18 , 2 , 42 ] , it is equally important to develop systems that can predict future troll behavior for preemptive measures given the velocity of information flows in today’s online ecosystem. In this work, we present Aletheia , a system that models influence campaigns in the form of a network and utilizes a mix of graph and linguistic properties to improve detection performance of malicious spreaders for multiple campaigns on both Reddit and X. Aletheia consists of a novel framework that stacks a graph neural network (GNN) over a recurrent neural network (RNN) to accurately predict future troll interactions. The goal is to utilize network information (e.g., real-world social media connections of users) and identify malicious users and behaviors. Research Questions. We seek to answer the following research questions. • RQ1: Can state-of-the-art graph based approaches improve node classification scores for accounts involved in influence campaigns? • RQ2: Can we effectively predict future interactions of a troll account with other troll accounts as well as regular users? To answer the research questions, we study influence campaigns on both Reddit and X from multiple countries. We use ground truth datasets of accounts released by the platforms. Our approach is to model the network of social media accounts and identify anomalous coordinated activity. Aletheia first models the troll interactions in the form of a graph using sharing patterns (e.g., replying , resharing and mentions ). Next, Aletheia comprises of a multi-modal pipeline built upon GraphSAGE [ 20 ] that uses the network graphs to perform node detection, highlighting the efficacy of topological features in combination with language embeddings when representing malicious campaigns. Our results are consistent across platforms and campaigns showing the importance of structural features in modeling troll activity. Aletheia also performs a first-of-its-kind temporal link prediction for influence operations over the network and predicts future links with an average AUC of 96.6%. These results are significant since we can use existing snapshots of the network and predict likely users that will be impacted by the troll accounts or when troll accounts interact with one another for amplification of narratives. Contributions. Overall, we make the following contributions in our work. • We show that modeling influence campaigns as a graph (network) and utilizing topological features (e.g., degree and centrality) to represent nodes in the graph can result in improvement of detection performance for both Reddit and X operations (with an average F1-score of 96.44% and 97.9% respectively). • Our results show that developing a multi-modal pipeline, where text embeddings augment a graph-based approach improves detection performance, with a 1.53% increase in AUC and 2.06% increase in F1 score from our pipeline built on GraphSAGE. • We present, to the best of our knowledge, the first application of temporal link prediction to influence campaigns where future links can be predicted effectively between troll-troll users and troll-regular user. Aletheia uses a combination of GraphSAGE and RNN to effectively predict troll interactions with an average AUC of 96.6%. Implications. Our work paves the way for improved detection and preemptive identification of malicious and manipulative behavior in online spaces. Developing multi-modal pipelines that can forecast and detect influence efforts in a scalable manner can help curb malicious influence on regular users on social media platforms ensuring user and platform safety. Disclaimer. We present several case studies that highlight nuanced influence through coordinated operations. The presented examples might be offensive, harmful and toxic to readers: therefore, reader discretion is advised. 2 Data We use data from both Reddit and Twitter (now X) for our analyses, which are popular social media platforms that have been targeted by large-scale influence operations in the past [ 55 , 57 , 58 , 59 ] . 2.1 Reddit Reddit is primarily a news aggregation and discussion website where users can

Read the complete text