Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool

Milena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannies Illies · 2025

Read Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool by Milena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannies Illies (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06734.txt.

Introduction

Milena Pustet and co-authors describe a monitoring tool being built with, not just for, civil society groups. The tool watches Telegram for anti-democratic movements and uses language models to sort the flood of posts. As platforms spend less on moderation, watchdogs need such help, yet few open tools join AI sorting to a working monitor. The team's stance is that the groups who know harmful content best should shape the tool itself.

In their design, the partner groups feed judgments back into the system, and the classifier adapts from that feedback. The work is still in progress and is being developed openly with the stakeholders at the table. The authors frame the project as a test of whether feedback-driven design can keep an AI monitor aligned with the values of the people who rely on it.

Catalog record

CREATORMilena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannies Illies
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE9 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-06734

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool Milena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannies Illies Abstract 1 Introduction The role of civil society organizations (CSOs) in monitoring harmful online content is increasingly crucial, especially as platform providers reduce their investment in content moderation Kayyali 2025 . AI tools can assist in detecting and monitoring harmful content at scale. However, few open-source tools offer seamless integration of AI models and social media monitoring infrastructures. Given their thematic expertise and contextual understanding of harmful content, CSOs should be active partners in co-developing technological tools, providing feedback, helping to improve models, and ensuring alignment with stakeholder needs and values, rather than as passive ‘consumers’. However, collaborations between the open source community, academia, and civil society remain rare, and research on harmful content seldom translates into practical tools usable by civil society actors. This work in progress explores how CSOs can be meaningfully involved in an AI-assisted open-source monitoring tool of anti-democratic movements on Telegram, which we are currently developing in collaboration with CSO stakeholders. Telegram has become a key platform for far-right and other extremist groups, enabling fast, large-scale dissemination of harmful, increasingly multi-modal content Baumgartner et al. 2020 ; Urman and Katz 2022 . Using conspiracy theory classification as a scenario, we discuss two concrete approaches to such collaboration. 2 Related Work Several CSOs in German-speaking countries, such as CeMAS and ISD, use in-house and closed-source monitoring tools for Telegram and other platforms. Open-source scripts for Telegram analysis exist, but remain inaccessible to most non-technical users. GESIS offers a web data collection service with a clear focus on academic researchers. An early contribution by Schinas et al. 2017 offered a modular, locally deployable monitoring system with basic NLP and relevance feedback, though it is no longer maintained. Existing open-source tools such as Telegram Monitor Júnior et al. 2022 or those developed by Primig and Fröschl 2024 ; Silva 2024 ; Ruscica et al. 2025 mainly focus on data extraction, search and statistical analyses, while 4CAT Peeters and Hagen 2022 integrates NLP techniques and prioritizes transparency. AI-based analysis and user feedback integration are still missing across tools. In adjacent domains like fact-checking, there is a growing demand for AI assistance, particularly for monitoring social media Juneja and Mitra 2022 ; Hrckova et al. 2024 . Despite the availability of capable models (e. g., for transcription), their integration remains difficult for non-technical users. Importantly, fact-checkers and researchers emphasize that AI must complement, not replace human expertise Procter et al. 2023 , calling for adaptive systems that incorporate user feedback Juneja and Mitra 2022 and ensure data ownership and privacy Wolfe and Mitra 2024 . 3 Feedback-Driven Model Integration Figure 1: User feedback integration workflow for fine-tuned and prompt-based content classification. The software is designed for use by individual CSOs in their own dedicated environments. It enables users to search for content within Telegram channels and presents the results in a feed format, where each post is assigned a binary CT classification label along with a corresponding confidence score. Since the classifier’s output is only one of several features, its evaluation is not the primary objective but is intended to be embedded within the main monitoring workflow. In line with this design, user feedback should focus on the binary labels and be integrated directly into the search interface. There are currently two main approaches for integrating classification models into monitoring systems: fine-tuning (FT) a domain-specific model or prompting (P) an LLM. We use a well-performing BERT-based model for CT classification trained on Telegram data Pustet et al. 2024 as a solid starting point for FT-based classification. Figure 1 outlines a workflow integrating both approaches. During monitoring, users provide feedback on the class labels displayed alongside each message. This feedback, stored with the original classifications and feedback-related metadata, forms a growing gold-standard dataset for evaluation and model refinement. User feedback supports refinement in both approaches: it informs fine-tuning in FT and guides prompt revision or few-shot updates in P. In the FT pathway, concept drift, detected by comparing new data to the training corpus, can additionally trigger model retraining. Updated models or prompts are then re-evaluated with accumulated feedback and iteratively deployed. Several challenges must be addressed. First, due to negativity bias, users of the software are more likely to provide feedback when they disagree with a label, while agreement often goes unmarked . However, positive signals are also important for model evaluation and refinement, especially in the FT approach. One option is to collect representative explicit feedback through dedicated data views, in which users are asked to rate a full sample of posts. However, this introduces additional workload and deviates from users’ regular monitoring routines. An alternative is to infer implicit affirmation by treating posts that were seen or clicked but left unmarked as signals of agreement, potentially weighting different user actions (e.g., scrolling vs. clicking) accordingly. Implementing this strategy would require not only experimental validation but also continuous tracking of which users have seen which posts, posing both technical and data privacy challenges. Second, when multiple users are involved, conflicting feedback must be resolved . While explicit input should take precedence over implicit signals, disagreements between users may need to be routed to a dedicated conflict resolution interface. Retraining schedules for FT and prompt revision cycles for P also need to be defined, along with protocols for prompt governance. Both approaches require validation before deployment , using part of the growing gold-standard dataset for test and validation sets. In FT, staged rollouts or A/B testing may be needed; in P, prompt changes can be applied immediately, but must still be monitored. Finally, all components must operate in secure, cost-efficient environments . This requirement limits the use of external services, as sending data to third-party APIs raises privacy and compliance concerns. At the same time, running internal (L)LMs on local infrastructure typically requires GPU-intensive hardware, which most, especially smaller, CSOs do not possess. These constraints suggest that, regardless of the chosen approach, smaller or distilled models should be prioritized to balance performance, cost, and accessibility. Table 1 compares FT and P across flexibility, control, resources, and adaptability. FT offers stable performance, high output consistency, and control, but retraining is technically demanding, especially in low-resource settings. On the other hand, practical issues such as output parsing and prompt formatting remain challenging in the P approach, particularly in multilingual or noisy settings like Telegram. P is more adaptable to evolving discourse and allows easier task switching. While growing user familiarity with LLM prompting may lower entry barriers, evaluating prompt effectiveness and refining outputs still requires structured feedback, technical guidance, and robust evaluation methods. For example, few-shot prompting highly depends on the number and choice of examples Chae and Davidson 2025 ; Yoshida 2024 , requiring automated experiments evaluated on the validation set. Table 1: Comparison of FT and P. Symbols: ++ = very good, + = acceptable, 0 = limited Aspect FT P Flexibility Adaptability to task variations 0 ++ Handling long input 0 ++ Flexibility across task scopes 0 ++ Robustness to raw inputs 0 ++ Control & Output Output consistency ++ 0 Explainability 0

Read the complete text