Predictively Combatting Toxicity in Health-related Online Discussions through Machine Learning This research work has been funded by MICIU/AEI/10.13039/501100011033 and ESF+ (FPU21/05783), ERDF A way of making Europe (PID2019-109238GB-C22, PID2023-147404OB-I00), and by the Xunta de Galicia (ED431C 2
Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas and Carlos Eiras-Franco · 2025
Read Predictively Combatting Toxicity in Health-related Online Discussions through Machine Learning This research work has been funded by MICIU/AEI/10.13039/501100011033 and ESF+ (FPU21/05783), ERDF A way of making Europe (PID2019-109238GB-C22, PID2023-147404OB-I00), and by the Xunta de Galicia (ED431C 2 by Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas and Carlos Eiras-Franco (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17068.txt.
Introduction
Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas, and Carlos Eiras-Franco aim to stop online toxicity before it happens. Their paper appeared in two thousand twenty-five. Health discussions online often turn hostile or spread dangerous claims. Platforms usually react after toxic comments appear. Removal can backfire and inflame threads further. Prediction offers a gentler lever.
Their method borrows collaborative filtering from recommender systems. It forecasts which user and subcommunity pairings are likely to turn toxic. COVID-era Reddit conversations supply the test data. Predictive scores pass eighty percent on key metrics. Platforms could steer risky pairings apart in advance. Prevention replaces the delete button as the first response.
Catalog record
| CREATOR | Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas and Carlos Eiras-Franco |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17068
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Predictively Combatting Toxicity in Health-related Online Discussions through Machine Learning This research work has been funded by MICIU/AEI/10.13039/501100011033 and ESF+ (FPU21/05783), ERDF A way of making Europe (PID2019-109238GB-C22, PID2023-147404OB-I00), and by the Xunta de Galicia (ED431C 2022/44) with the European Union ERDF funds. CITIC, as Research Center accredited by Galician University System, is funded by “Consellería de Cultura, Educación e Universidade from Xunta de Galicia”, supported in an 80% through ERDF Operational Programme Galicia 2021-2027, and the remaining 20% by “Secretaría Xeral de Universidades” (ED431G 2023/01) Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas and Carlos Eiras-Franco Abstract In health-related topics, user toxicity in online discussions frequently becomes a source of social conflict or promotion of dangerous, unscientific behaviour; common approaches for battling it include different forms of detection, flagging and/or removal of existing toxic comments, which is often counterproductive for platforms and users alike. In this work, we propose the alternative of combatting user toxicity predictively, anticipating where a user could interact toxically in health-related online discussions. Applying a Collaborative Filtering-based Machine Learning methodology, we predict the toxicity in COVID-related conversations between any user and subcommunity of Reddit , surpassing 80% predictive performance in relevant metrics, and allowing us to prevent the pairing of conflicting users and subcommunities. I Introduction In recent years, the decentralisation of online communication and the expansion of social media has boosted the popularity of public health-related topics in online conversations; more than 80% of US citizens use online media for medical consultations before visiting a doctor. Our reliance on digital media to inform itself about important issues such as health and disease implies an overriding need to ensure the good nature of conversations and information about these issues on the internet. However, social media has also been characterised by a steep rise in misinformation and toxic behaviours, such as hate speech, bullying, etc. On public health issues, such as COVID, this toxicity can fuel social and political conflict, or promote unscientifically based dangerous behaviours [ 1 ] , [ 2 ] . Among social networks, Reddit is popular in work in medical AI, e.g. for early detection of self-injury and depression [ 3 ] or anorexia [ 4 ] . The hierarchical and decentralised structure made Reddit a hub of heated debate during the onset of the COVID pandemic, with over 200,000 related posts per day. On the other hand, Reddit is known for its high percentage of toxic interactions, made by more than 15 percent of users [ 5 ] , these being the most read and visited [ 1 ] . Surprisingly, more than 80% of users vary their degree of toxicity between different Reddit subcommunities or subreddits , indicating that the toxicity of a user in a subcommunity does not depend solely on the characteristics of the user or the subreddit , but on the interaction of both [ 5 ] . Efforts to combat this toxicity during the disinformation episodes in the COVID pandemic focused on detecting toxic subcommunities or comments [ 6 ] [ 7 ] and banning users who exhibited these behaviours [ 8 ] . These reactive measures have become the most prevalent methods to combat toxicity in large-scale online platforms, and are supported either by professional or volunteer content moderators [ 9 ] . However, these detect-and-react moderation approaches come with a myriad of issues. Professional content moderation incurs high operational costs, recently causing platforms like X (formerly Twitter ) or Facebook to migrate to volunteer-centered moderation [ 10 ] . Conversely, volunteer-based moderation is generally more susceptible to bias and under-moderation, depending on the platform’s audience. Furthermore, both professional and volunteer-based reactive moderation of online toxicity has serious negative psychological effects on the moderating individuals [ 11 ] , and incite toxic users to accumulate in radical communities with more potential for misinformation and public risk [ 5 ] . In the grand scheme, reactive approaches to combatting online toxicity only avoid critical situations, can be ineffective and biased, and involve the assistance of human operators; they do not solve the underlying problem, and instead tend to worsen it in the long term. As a shift in paradigm, we propose the alternative of combatting toxicity predictively: instead of detecting who and where has been toxic, we propose to predict who and where will be toxic. In this work, we explore the anticipation of toxic behaviour in COVID conversations in Reddit , predicting the expected toxicity of any user in any subcommunity. Our approach models the latent toxicity characteristics of users and subcommunities using Matrix Factorization and Natural Language Processing and could allow, for example, to avoid recommending users to subcommunities where they are presumed to act in a toxic way, helping to reduce the occurrence of potentially dangerous online content on public health issues. The main contributions of this research are: • The proposal of a novel approach to combatting online user toxicity; in contrast with classic reactive approaches, which are ineffective, resource-intensive and incite user radicalization, we introduce a predictive approach to pre-emptively anticipate online user toxicity, which platforms could use to reduce its surfacing and the subsequent resources required to detect and moderate it. • The development of a Matrix Factorization-based Collaborative Filtering technique capable of predicting the future toxicity of a user when interacting with a new online subcommunity, in the fashion of the aforementioned toxicity approach. • The design of an adapted Leave Out Last Item data partitioning method suitable for binary classification-oriented Collaborative Filtering tasks. • The evaluation of our toxicity prevention algorithm in a real-world dataset of COVID-related discussions in Reddit , achieving >80 % predictive performance in relevant metrics, proving our method can prevent the pairing of conflicting users and subcommunities. II Data Acquisition and Analysis To work with real toxic and non-toxic user interactions, we collected comment data from Reddit through the archiving platform PushShift [ 12 ] ; suitable comment data for other platforms could not be obtained due to their recent decisions towards discontinuing API data access for research purposes [ 13 ] . The data acquisition, filtering and pre-processing process used, shown in Figure 1 , is as follows: Figure 1: Acquisition, filtering and pre-processing processes for online Reddit comment data. 1. We obtain comments posted on subreddit r/coronavirus during March 2021 (approximating the start of worldwide COVID vaccination campaigns). 2. For each user authoring comments obtained in 1), we extract up to 1000 of their comments across Reddit between January and June 2021 and add them to the set. 3. We remove comments on subreddits where \leq 20 users have commented, to discard those without sufficient conversation. 4. We remove “generic comments‘’ from the set, i.e. those unrepresentative of the subreddit they belong to and unrelated to COVID, using a keyword-based method for each subreddit s : (a) Identify P(s) , the set of the 100 most popular words in s . (b) Identify P(\overline{s}) , the set of the 100 most popular words outside s . (c) Calculate S(s)=P(s)\setminus P(\overline{s}) , i.e. the popular words specific to s . (d) Tag comments as related to COVID if they contain keywords such as ‘COVID’, ‘vaccine’, ‘pfizer’, ‘lockdown’.... 1 1 1 Authors have temporarily removed this