Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine

Kateryna Akhynko, Oleksandr Kosovan, Mykola Trokhymovych · 2025

Read Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine by Kateryna Akhynko, Oleksandr Kosovan, Mykola Trokhymovych (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24028.txt.

Introduction

Kateryna Akhynko, Oleksandr Kosovan, and Mykola Trokhymovych present their entry in a shared task on spotting tricks in social media posts. The posts targeted Ukrainian Telegram users during Russia's 2022 invasion of Ukraine. Systems had to sort the posts and also mark the exact spans of text that do the tricking.

For sorting, they fine-tuned a large language model with small added adapters. A second sorter, built on extra features with tuned cutoffs, came on top of that. For span marking, they used a model trained to label single tokens. Their system took second place in sorting and third in span marking. The work shows a working pipeline for flagging hidden persuasion.

Catalog record

CREATORKateryna Akhynko, Oleksandr Kosovan, Mykola Trokhymovych
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24028

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine Kateryna Akhynko, Oleksandr Kosovan, Mykola Trokhymovych Abstract This paper presents one of the top-performing solutions to the UNLP 2025 Shared Task on Detecting Manipulation in Social Media. The task focuses on detecting and classifying rhetorical and stylistic manipulation techniques used to influence Ukrainian Telegram users. For the classification subtask, we fine-tuned the Gemma 2 language model with LoRA adapters and applied a second-level classifier leveraging meta-features and threshold optimization. For span detection, we employed an XLM-RoBERTa model trained for multi-target, including token binary classification. Our approach achieved 2nd place in classification and 3rd place in span detection. † † To appear in UNLP’25. 1 Introduction In times of war, information can have the same power as weaponry. During the 2022 Russian invasion of Ukraine, Telegram emerged not only as a battlefield communication tool but also as the primary source of information for 44% of Ukrainians. Its speed, reach, and anonymity became an important tool for civilians and military actors. However, these features — particularly minimal content moderation and user anonymity — have also made Telegram a favorable environment for influence operations Vorobiov 2024 . Manipulation on social media is a complex and nuanced phenomenon. It includes not just factual distortions (i.e., disinformation) but also rhetorical strategies, emotional appeals, and narrative framing that are designed to influence perception or behavior subtly. In this paper, we present the solution 1 1 1 https://github.com/akhynkokateryna/manipulative-narrative-detection to the UNLP 2025 Shared Task, 2 2 2 https://github.com/unlp-workshop/unlp-2025-shared-task focused on manipulative narratives detection, which is defined as the intentional use of language and messaging tactics aimed at influencing beliefs, emotions, or attitudes, without providing clear factual support. The task includes several challenges that make it particularly complex. First, it focuses exclusively on the textual content of social media posts without incorporating metadata such as user history or engagement metrics. Second, the dataset presents multiple layers of complexity: it is imbalanced across manipulation types, multilingual (primarily Ukrainian and Russian), and multi-label, meaning that a single post can include several manipulation techniques simultaneously. Finally, the span detection subtask requires identifying the exact textual fragments responsible for the manipulation, often implicit, rhetorical, or emotionally charged language that is difficult to isolate. Given these challenges, we developed a system that achieved second place in manipulation techniques classification and third place in span detection subtasks (see Figure 1 ). For classification, we fine-tuned the Gemma 2 language model using LoRA adapters and introduced a second-level classifier that leveraged meta-features and custom threshold optimization. For span detection, we trained an XLM-RoBERTa model capable of multi-target, token-level binary classification to locate manipulative spans within posts. Figure 1: Sketch of manipulation techniques classification and span detection problems 2 Related Work Our research is based on a growing body of work in detecting propaganda and misinformation analysis. Numerous studies have focused on identifying propaganda techniques in news articles, particularly in the context of SemEval-2020 Task 11. Da San Martino et al. 2020 explored detecting propaganda techniques in news articles through span identification and technique classification tasks. Similarly to previous research, the UNLP 2025 Shared Task includes two subtasks: manipulation technique classification (a multi-label classification) and span detection (a token classification). Within this framework, research from SemEval-2020 Task 11 demonstrated BERT’s remarkable capabilities for propaganda technique identification Altiti et al. 2020 . Further advancing this line of inquiry, Da San Martino et al. 2020 showcased RoBERTa’s performance in addressing both tasks simultaneously. At the same time, the nature of propaganda on social media evolves continuously, adapting to specific circumstances to remain undetected. Solopova et al. 2023 explored this process by combining machine learning and linguistic analysis to reveal how pro-Kremlin propaganda evolved in the context of the 2022 Russian invasion of Ukraine. In this context, it is important to note that while our work has a similar goal, we focus specifically on detecting manipulative narratives regardless of the factual support of the claim. This distinguishes our approach from fact-checking or knowledge manipulation detection methods Trokhymovych and Saez-Trumper 2021 ; Trokhymovych et al. 2025 . In our case, we are dealing with multilingual Telegram data containing Ukrainian and Russian texts. In this scenario, fine-tuning a multilingual model, such as XLM-RoBERTa, appears to be a more productive approach, as demonstrated in research on hostility identification for low-resource Indian languages Sai et al. 2021 . Moreover, XLM-RoBERTa-based models have demonstrated cross-lingual strengths in other downstream tasks, including those involving Ukrainian and Russian languages Mehta and Varma 2023 ; Trokhymovych et al. 2024 . While Sprenkamp et al. 2023 discovered that fine-tuned RoBERTa outperformed zero and few-shot learning approaches with LLMs for propaganda detection, newer advances in large language models show considerable promise. Recent innovations have developed methods to transform decoder-only LLMs into effective text encoders suitable for classification tasks BehnamGhader et al. 2024 . Models such as Gemma offer particularly interesting customization potential for classification challenges Team et al. 2024 . Notably, Gemma-family models enable fine-tuning with LoRA adapters and support quantization techniques, making them viable options even with limited computational resources. Building on this foundation, Kiulian et al. 2024 ventured into fine-tuning both Gemma and Mistral specifically to enhance Ukrainian language representation, providing valuable insights that directly inform our approach to detecting manipulative narratives within Telegram content from the region. 3 Data The UNLP shared task dataset contains more than 9,500 text samples collected from Telegram channels, with 68% of these collected samples containing manipulative narratives. This dataset forms the basis for a dual-task challenge: classifying manipulation techniques and identifying corresponding text spans. The data is divided into training and testing sets, with 3,822 samples allocated for training and 5,735 for testing. Among the 3,822 training samples, 2,147 (56%) are in Ukrainian and 1,675 (44%) are in Russian. At the same time, the testing set does not include language labels. Notably, the testing set is further split into public and private sets for leaderboard evaluation. Figure 2: Co-occurrence of manipulation techniques in the combined training and testing sets Each post is annotated for both classification and span detection tasks. Specifically, every sample is labeled with one or more of ten predefined manipulation techniques, detailed in Appendix A . Manipulative text segments are also defined, irrespective of the specific technique involved. Figure 2 illustrates manipulation techniques’ co-occurrence patterns across training and testing sets. As the distribution of labels is similar in both subsets, we present them together for clarity. 4 Methodology In this section, we present our approaches for solving the technique classification and span identification subtasks. 4.1 Technique Classification The manipulation technique detection task is formulated as a multi-label text classification problem, where each input text may contain multiple manipulation strategies. Each sample is annotated with any number of 10 predefined manipulation techniques. Figure 3: Pipeline of technique classification solution Our best-performing solution involves multi-stage fine-tuning of the instruction-tuned Gemma 2 2B IT model. 3 3 3 https://huggingface.co/google/gemma-2-2b-it The complete fine-tuning pipeline schema is presented in Figure 3 . Firstly, we fine-tune the model using a causal language modeling (CLM) objective, where the model learns to predict the next token given a left-to-right context. Specifically, we employed the AutoModelForCausalLM class from HuggingFace Transformers .

Read the complete text