Utilising Large Language Models for Generating Effective Counter Arguments to Anti-Vaccine Tweets

Utsav Dhanuka, Soham Poddar, Saptarshi Ghosh · 2025

Read Utilising Large Language Models for Generating Effective Counter Arguments to Anti-Vaccine Tweets by Utsav Dhanuka, Soham Poddar, Saptarshi Ghosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16359.txt.

Introduction

Utsav Dhanuka, Soham Poddar, and Saptarshi Ghosh explore using language models to answer anti-vaccine tweets in real time. False claims about vaccines spread fast on social media and chip away at trust in health advice. Spotting them is now routine, but crafting good replies at scale is not.

The team tests prompting tricks and fine-tuning recipes for writing rebuttals, and sorts tweets into worry types first. Those types cover fears about side effects, doubts the shots work, and political angles, so replies can meet each claim head-on. Human judges, model judges, and automatic scores largely agree on which replies land. Adding label descriptions and structured fine-tuning clearly strengthens the counters. The work sketches a scalable way to push back on vaccine falsehoods where they spread.

Catalog record

CREATORUtsav Dhanuka, Soham Poddar, Saptarshi Ghosh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-16359

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Utilising Large Language Models for Generating Effective Counter Arguments to Anti-Vaccine Tweets Utsav Dhanuka, Soham Poddar, Saptarshi Ghosh Abstract In an era where public health is increasingly influenced by information shared on social media, combatting vaccine skepticism and misinformation has become a critical societal goal. Misleading narratives around vaccination have spread widely, creating barriers to achieving high immunization rates and undermining trust in health recommendations. While efforts to detect misinformation have made significant progress, the generation of real-time counter-arguments tailored to debunk such claims remains an underexplored area. In this work, we explore the capabilities of LLMs to generate sound counter-argument rebuttals to vaccine misinformation. Building on prior research in misinformation debunking, we experiment with various prompting strategies and fine-tuning approaches to optimize counter-argument generation. Additionally, we train classifiers to categorize anti-vaccine tweets into multi-labeled categories—such as concerns about vaccine efficacy, side effects, and political influences—allowing for more context-aware rebuttals. Our evaluation, conducted through human judgment, LLM-based assessments, and automatic metrics, reveals strong alignment across these methods. Our findings demonstrate that integrating label descriptions and structured fine-tuning enhances counter-argument effectiveness, offering a promising approach for mitigating vaccine misinformation at scale. 1 Introduction The rapid proliferation of social media platforms such as Twitter and Reddit has significantly transformed the way information is disseminated and consumed. However, this transformation has also facilitated the spread of misinformation, with anti-vaccine rhetoric becoming a prominent example. Such misinformation can erode public trust in vaccines, impede public health efforts, and contribute to the resurgence of preventable diseases. Addressing these challenges requires innovative solutions that go beyond manual moderation and fact-checking due to the sheer scale of content generated daily. The COVID-19 pandemic, which began in 2020, has underscored the critical importance of mass vaccination as the most effective means to achieve herd immunity and mitigate the virus’s impact. Despite the availability of safe and effective vaccines, misinformation campaigns have fueled skepticism and resistance, endangering public health initiatives worldwide. To counter this, there is an urgent need for automated systems capable of responding to anti-vaccine claims with persuasive and factual counter-arguments in real time. While efforts to detect misinformation have made significant progress, the generation of real-time counter-arguments tailored to debunk such claims remains an underexplored area. Our works seeks to bridge that gap by focusing on the automatic generation of counter-arguments for anti-vaccine tweets. By providing timely, informative, and convincing rebuttals, we can reduce the spread of misinformation and promote trust in vaccination efforts. Prior studies have explored the theoretical underpinnings of countering anti-vaccination attitudes, highlighting both cognitive mechanisms and dual-process theories to address resistance to vaccination messages ( Okuhara et al. 2020 ; Horne et al. 2015 ) . However, these works largely focus on the psychology of persuasion and broad message framing rather than on the generation of targeted counterarguments addressing specific user concerns. To the best of our knowledge, our work is the first to systematically tackle the problem of generating fine-grained, concernspecific counter-arguments for anti-vaccine content using large language models. In this work we focus on developing a comprehensive approach to generate counter-arguments for anti-vaccine tweets by leveraging recent advancements in artificial intelligence and natural language processing. Key contributions of our work include: • First , we establish an effective strategy for addressing anti-vaccine rhetoric through prompt engineering techniques applied to GPT-4o-Mini. To ensure the generated counter-arguments are both relevant and impactful, we employ a rigorous evaluation framework that incorporates human judgment, LLM-based assessments, and automatic metrics to assess dimensions such as factual accuracy, clarity, coverage, and persuasiveness. • Second , we develop classifiers capable of generating label descriptions for anti-vaccine tweets, which are then used to prompt smaller language models (SLMs) for producing effective and context-aware rebuttals. We further enhance this process by designing a Chain-of-Thought (CoT) reasoning approach, where the model is guided to first infer relevant concerns or themes before generating the counter-argument, leading to more coherent and targeted responses. • Finally , we fine-tune small language models (SLMs) on high-quality counter-arguments generated by GPT-4o to enable them to operate independently. These models are evaluated using standard automatic metrics such as ROUGE and BERTScore. Our approach highlights the practicality of using SLMs, which offer a favorable trade-off between performance and computational efficiency, making them suitable for deployment in real-world, resource-constrained settings. 2 Related Work In this section, we review prior research on counter-argument generation, highlighting its significance in online discourse and exploring how effective counter-arguments should be constructed. We also discuss related efforts in countering misinformation, hate speech, and other adversarial content using language models and prompting strategies. Approaches to Counter-Narratives: Recent work has shown that techniques developed for political argumentation and counterspeech can inform strategies to counter vaccine misinformation effectively. Palmer and Spirling 2023 demonstrate that Large Language Models (LLMs) are capable of producing persuasive arguments on politically polarizing topics that are often as convincing as human-authored ones. However, they also find that human evaluators display bias against LLM-generated arguments when informed of their origin, highlighting the challenges of trust in AI-generated persuasion. Extending LLM capabilities to toxic online discourse, Saha et al. 2022 propose COUNTERGEDI, a controllable generation framework that produces counterspeech with specific attributes such as politeness, emotional tone, and detoxification. Their model significantly improves the quality and acceptability of counterspeech, offering a promising avenue for generating nuanced responses to online hate, which is often closely intertwined with misinformation. Similarly, Saha and Srihari 2024 propose a hybrid framework that adapts counter-hate strategies to misinformation domains, combining annotated response strategy classifiers with an argument-graph-guided LLM generator. Their approach demonstrates improvements in factual accuracy, naturalness, and persuasiveness. Alshomary et al. 2021 proposed argument undermining, a novel approach that identifies weak premises in arguments and generates responses that directly challenge those specific points, demonstrating improved quality in generated counter-arguments. Saha et al. 2024 investigated zero-shot counterspeech generation using models like GPT-2, DialoGPT, ChatGPT, and Flan-T5, highlighting the efficacy of strategic prompting and model scaling while raising concerns around toxicity. A complementary study by Islam and Goldwasser 2024 introduced an LLMs-in-the-loop framework to uncover latent arguments from social media discourse, which supports dynamic argument discovery in evolving domains such as COVID-19 vaccine hesitancy and climate change. These works collectively provide useful methods that inform our LLM-based counter-argument generation pipeline for addressing anti-vaccine discourse, ranging from premise analysis and dataset curation to prompting strategies and model-assisted argument extraction. Studies on Vaccine-related Arguments: Recent studies provide crucial insights into the psychological and communicative strategies needed to construct effective counter-arguments against vaccine misinformation. Okuhara et al. 2020 emphasize that anti-vaccine rhetoric often appeals to fast, emotional reasoning (System 1), while pro-vaccine messages typically rely on slower, analytical thinking (System 2). They argue that persuasive messages should engage both systems—combining emotional appeal with factual evidence. Complementing this, Germani and Biller-Andorno 2022 stress the importance of filling “information voids” where misinformation thrives, advocating for proactive public health messaging and increased digital literacy to build long-term resilience. Horne et al. 2015 offer another important perspective, focusing on how the framing of counter-arguments influences their effectiveness. Their work reveals that direct myth-debunking strategies—such as presenting scientific facts that contradict anti-vaccine claims—often fail to change minds and may even backfire. Instead, they demonstrate that highlighting the real-world consequences

Read the complete text