MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media
Muhammad Arslan Manzoor, Ruihong Zeng, Dilshod Azizov, Preslav Nakov & Shangsong Liang · 2024
Read MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media by Muhammad Arslan Manzoor, Ruihong Zeng, Dilshod Azizov, Preslav Nakov & Shangsong Liang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10467.txt.
Introduction
Manzoor, Zeng, Azizov, Nakov, and Liang rate news outlets for bias and factual reliability by looking at who shares their audience. Text-only models miss links between outlets, while graph models stumble when the audience graph breaks into islands and labels are scarce. Their 2024 system, MediaGraphMind, wraps both model types in a variational loop that lets each teach the other.
Instead of trusting only near neighbors, the system borrows features, structure, and label hints from similar outlets anywhere in the graph. That lets the graph model capture long-range ties and feeds structure back into the text model. Tests set new best scores on the profiling task. The authors publish their data, code, and notes for reuse.
Catalog record
| CREATOR | Muhammad Arslan Manzoor, Ruihong Zeng, Dilshod Azizov, Preslav Nakov & Shangsong Liang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-10467
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media Muhammad Arslan Manzoor, Ruihong Zeng, Dilshod Azizov, Preslav Nakov & Shangsong Liang Abstract * * footnotetext: Equal contribution. $\dagger$ $\dagger$ footnotetext: Corresponding author. In the current era of rapidly growing digital data, evaluating the political bias and factuality of news outlets has become more important for seeking reliable information online. In this work, we study the classification problem of profiling news media from the lens of political bias and factuality. Traditional profiling methods, such as Pre-trained Language Models (PLMs) and Graph Neural Networks (GNNs) have shown promising results, but they face notable challenges. PLMs focus solely on textual features, causing them to overlook the complex relationships between entities, while GNNs often struggle with media graphs containing disconnected components and insufficient labels. To address these limitations, we propose MediaGraphMind (MGM), an effective solution within a variational Expectation-Maximization (EM) framework. Instead of relying on limited neighboring nodes, MGM leverages features, structural patterns, and label information from globally similar nodes. Such a framework not only enables GNNs to capture long-range dependencies for learning expressive node representations but also enhances PLMs by integrating structural information and therefore improving the performance of both models. The extensive experiments demonstrate the effectiveness of the proposed framework and achieve new state-of-the-art results. Further, we share our repository 1 1 1 https://github.com/marslanm/MGM_code which contains the dataset, code, and documentation. 1 Introduction The rise of the Internet has offered many opportunities to publish information and to express opinions Mehta and Goldwasser 2023b . Concurrently, this easy means of distribution has accelerated the spread of misinformation and disinformation online which resembles news in form but lacks the journalistic standards that ensure its quality Fairbanks et al. 2018 . Vosoughi et al. 2018 has found that “fake news” spreads six times faster and reaches much farther than real news. Any delay in profiling in rapidly evolving digital landscapes can lead to unchecked distribution of misleading content Liu et al. 2022 . Profiling news outlets through NLP pipelines offers a proactive approach by enabling the early detection of potentially unreliable sources as soon as they publish content. Since outlets with a history of biased or false information are more likely to do so again, profiling the media in advance allows us to quickly identify probable “fake news” by evaluating the reliability of the source itself. Nakov et al. 2024 . Early studies on automatic media profiling relied solely on text characteristics Battaglia et al. 2018 ; Pérez-Rosas et al. 2017 , which has proven particularly challenging. The complexity increases when the text features contain indeterminate noise, leading to classification errors Baly et al. 2018 ; Baly et al. 2020a . Moreover, traditional methods struggle to capture the intricate relationships between entities, such as media outlets, content they publish, and audiences. Panayotov et al. 2022 constructed media graphs: nodes represent media, and edges represent audience overlap between media. They proposed a framework that captures both inherent and implicit information about media through interactive learning within the media ecosystem, addressing the limitations of relying solely on textual features. We analyze these media graphs and identify two key challenges: disconnected components and label sparsity. Disconnected components prevent GNNs from capturing long-range dependencies, limiting their ability to learn expressive node representations for classification tasks Longa et al. 2024 ; Zhang et al. 2024 . Prior studies Yin et al. 2024 ; Tang et al. 2024 address similar issues by using memory-based approaches that store global information throughout the graph using external memory modules. However, these methods require significant memory to store all node embeddings. To tackle these challenges, we present MGM, a novel method based on a variational Expectation-Maximization (EM) framework that augments existing Graph Neural Networks (GNNs) to capture and exploit global information in media graphs. MGM seamlessly integrates local and global patterns, node features, and labels from globally similar nodes to enhance performance. Unlike Graph Attention Networks (GATs) Veličković et al. 2018 , which focus solely on local neighborhoods, MGM employs an external memory module to store precomputed node representations of all nodes. This approach not only reduces computational costs Fey et al. 2021 but also facilitates efficient node embedding retrieval. Furthermore, MGM optimizes memory usage by focusing on a small set of candidate nodes, guided by a Dirichlet prior distribution He et al. 2020 . The experimental results show that MGM substantially enhances the performance of baseline GNNs, delivering a 10% increase across all evaluation measures on the Media Bias/Fact Check (MBFC) 2 2 2 www.mediabiasfactcheck.com data feature in the ACL-2020 Baly et al. 2020b and the EMNLP-2018 Baly et al. 2018 datasets. Despite the lack of rich node features in the media graph, we enhance the dataset by scraping Articles and Wikipedia descriptions for ACL-2020. Pre-trained language models (PLMs) such as BERT Devlin et al. 2018 , RoBERTa Liu et al. 2019b , DistilBERT Sanh et al. 2019 , and DeBERTaV3 He et al. 2021 are fine-tuned to predict political bias and factuality. Where textual data of media are inaccessible, MGM’s representation-based probabilities compliment the gap. Moreover, integrating MGM’s probabilities with PLMs enhance the performance for both tasks. Our contributions are as follows: • We introduce MGM, an efficient and expressive approach that enhances GNNs for reliable news media profiling by leveraging global information and minimizing memory requirements via a sparse distribution. • We illustrate that MGM consistently outperforms vanilla GNNs for the detection of factuality and political bias across all baselines. • We validate that integrating the MGM features with the PLMs enhances performance and yields state-of-the-art results. 2 Related Work 2.1 Political Bias and Factuality of Media Early research on political bias detection focused on the analysis of textual content Afroz et al. 2012 ; Battaglia et al. 2018 ; Pérez-Rosas et al. 2017 ; Conroy et al. 2015 . To improve the performance, subsequent research added contextual information Baly et al. 2020b ; Hounsel et al. 2020 ; Castelo et al. 2019 ; Fairbanks et al. 2018 , including the nuances of multimedia production Huh et al. 2018 , the associated infrastructure Hounsel et al. 2020 , and the social context Baly et al. 2020b . Guo et al. 2022 used BERT Devlin et al. 2019 to model the linguistic political bias in news articles. Fan et al. 2019 used annotated media from Budak et al. 2016 , analyzing articles for political bias using distant supervision. Various methods measured political bias, including analyzing Twitter interactions An et al. 2012 ; Stefanov et al. 2020 , often using small datasets only in English Da San Martino et al. 2023 ; Nakov et al. 2023a ; Nakov et al. 2023b ; Barrón-Cedeño et al. 2023a ; Barrón-Cedeño et al. 2023b ; Azizov et al. 2023 ; Spinde et al. 2022 . Lei et al. 2022 improved political bias detection through discourse structures, Liu et al. 2019a detected frames in gun violence reporting, and Lee et al. 2022 proposed framework for neutral summaries. Bang et al. 2023 proposed a polarity minimization loss to reduce framing bias in multi-document summarization. Liu et al. 2023 addressed framing bias in event understanding with a neutral event graph induction framework