Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey
Katerina Korre †, Dimitris Tsirmpas †‡, Nikos Gkoumas †, Emma Cabalé, Danai Myrtzani ‡, Theodoros EvgeniouIon Androutsopoulos ‡† , John Pavlopoulos ‡† † Archimedes, Athena Research Center, Greece ( {k.korre,n.goumas}@athenarc.gr ) ‡ Athens University of Economics and Business, Greece ( {dim.tsirmpas · 2025
Read Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey by Katerina Korre †, Dimitris Tsirmpas †‡, Nikos Gkoumas †, Emma Cabalé, Danai Myrtzani ‡, Theodoros EvgeniouIon Androutsopoulos ‡† , John Pavlopoulos ‡† † Archimedes, Athena Research Center, Greece ( {k.korre,n.goumas}@athenarc.gr ) ‡ Athens University of Economics and Business, Greece ( {dim.tsirmpas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01513.txt.
Introduction
Katerina Korre and co-authors survey how online discussions can be judged and guided in the age of large language models. Online talk aims at shared understanding but often slides into hate and abuse, which strains society. The catalog creator field is garbled with symbols, affiliations, and emails, so the clean head byline of eight authors governs here. Their survey appeared in 2025.
Language models now let software agents do more than delete bad posts: they can step in and lift the quality of a thread. The survey joins ideas from computing and social science into one map. It offers a new scheme for scoring discussion quality, a review of intervention styles, and a sorting of the data sets used to train facilitators. A closing roadmap lists good practice and open questions, both technical and social, for building fair discussion spaces.
Catalog record
| CREATOR | Katerina Korre †, Dimitris Tsirmpas †‡, Nikos Gkoumas †, Emma Cabalé, Danai Myrtzani ‡, Theodoros EvgeniouIon Androutsopoulos ‡† , John Pavlopoulos ‡† † Archimedes, Athena Research Center, Greece ( {k.korre,n.goumas}@athenarc.gr ) ‡ Athens University of Economics and Business, Greece ( {dim.tsirmpas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01513
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey Katerina Korre †, Dimitris Tsirmpas †‡, Nikos Gkoumas †, Emma Cabalé, Danai Myrtzani ‡, Theodoros EvgeniouIon Androutsopoulos ‡† , John Pavlopoulos ‡† † Archimedes, Athena Research Center, Greece ( {k.korre,n.goumas}@athenarc.gr ) ‡ Athens University of Economics and Business, Greece ( {dim.tsirmpas,dan.myrtzani,ion,annis}@aueb.gr ) Abstract We present a survey of methods for assessing and enhancing the quality of online discussions, focusing on the potential of LLM ( LLM ). While online discourses aim, at least in theory, to foster mutual understanding, they often devolve into harmful exchanges, such as hate speech, threatening social cohesion and democratic values. Recent advancements in LLM enable artificial facilitation agents to not only moderate content, but also actively improve the quality of interactions. Our survey synthesizes ideas from NLP ( NLP ) and Social Sciences to provide (a) a new taxonomy on discussion quality evaluation, (b) an overview of intervention and facilitation strategies, (c) along with a new taxonomy of conversation facilitation datasets, (d) an LLM -oriented roadmap of good practices and future research directions, from technological and societal perspectives. 1 Introduction Discussions, especially of complex or controversial topics, are a cornerstone of collective decision-making Burton et al. (2024) . In contrast to initial hopes of promoting mutual understanding Rheingold (2000) , online discussions (especially in social media) often degenerate into hate speech, personal attacks, promoting conspiracy theories or propaganda – to the extent that they can even be considered a threat to social cohesion and democracy Tucker et al. (2018) ; Mathew et al. (2019) . Natural Language Processing ( NLP ) and ML ( ML ) can potentially help improve the quality of online discussions. For example, automatic classifiers Bang et al. (2023) ; Molina and Sundar (2022) are already being used to help or even replace human moderators, by flagging posts that violate the law or policies of online discussion fora Saeidi et al. (2021) . Social Science provides theories and applications for the facilitation of a discussion, but in specific contexts, such as teaching Mansour (2024) or clinical discussions Gelula (1997) , without much research devoted to online discussions. While prior NLP studies have explored LLM -facilitated discussions Burton et al. (2024) ; Aher et al. (2023) ; Beck et al. (2024) ; Schroeder et al. (2024) ; Small et al. (2023) ; Cho et al. (2024) , rarely does Social Science work examine how facilitation can be automated Gimpel et al. (2024) . Figure 1: A conceptualization of this survey. We explore approaches from different disciplines, which recommend their own ways of evaluating and improving discussions. In this survey, we combine LLM -based methods, with ideas from Social Science (e.g., Deliberative Theory) when discussing how to evaluate online discussions, and when exploring intervention strategies. Figure 1 provides a high-level conceptualization of our work. The main research question of this survey is can LLM be used effectively as facilitators in online discussions? Focusing on threaded discussions (§ 2 ), we explore three key areas: (1) methods (potentially also LLM -based) for evaluating aspects of online discussions, (2) intervention strategies for facilitation, and (3) available data resources which can be used to analyze human facilitation and train LLM equivalents. Specifically, we survey discussion evaluation aspects and introduce a new taxonomy (§ 4 ). We map tasks suited for ML models, LLM , and humans, aggregate multidimensional insights on facilitation strategies (§ 5 ), and outline future possibilities for LLM (§ 6 ). Additionally, we aggregate and compare all major relevant datasets in literature, dividing them into categories per task (§ 7 ). Our findings show that (a) many discussion evaluation dimensions coexist in the literature; (b) LLM advancements show significant promise in improving the quality and timeliness of facilitation methods; (c) while surveying the existing datasets, we notice a scarcity of datasets for studying facilitation. We posit that LLM -generated discussions, could become an asset to develop and test automatic facilitation strategies in diverse artificial discussions, before testing the strategies and the LLM -based facilitator agents in more costly experiments with human participants. 2 Terminology Given the numerous aspects to consider regarding discussion quality and facilitation, we clarify the terminology we use. We highly recommend consulting the Terminology Section of Appendix C and, especially, Table 3 , where we explain our findings with regard to the terms used in the literature. Facilitation vs. Moderation The term ‘moderation’ is more commonly used in NLP Argyle et al. (2023) , typically referring to the flagging and/or removal of unwanted content (‘content moderation’), while ‘facilitation’ is more prevalent in the Social Sciences, where it encompasses a broader scope, including active interventions Vecchi et al. (2021) ; Kaner et al. (2007) ; Trenel (2009) . Given the limited attention to facilitation in NLP and the survey’s grounding in Social Science, we distinguish between the terms, even though they are sometimes used interchangeably in the literature. Ex-Post moderation This survey mainly focuses on ‘Real-Time, Ex-Post-moderation’, i.e., moderation happening just after the user has posted some content. This is different from pre-moderation approaches, such as nudging users before they post harmful content Argyle et al. (2023) , or delaying the posting of user content until a moderator has had the chance to check it. Discussion, Deliberation, Dialogue, Debate The definitions of these terms often vary across literature Russmann and Lane (2016) ; Goñi (2024) . We focus on discussions , a general term for verbal/written exchanges Russmann and Lane (2016) , and deliberations , a term for structured discussions focusing on opinion sharing Degeling et al. (2015) ; Lo and McAvoy (2023) . This is in contrast to the (at least in theory) collaborative nature of dialogues Rose-Redwood et al. (2018) ; Bawden (2021) ; Goñi (2024) and the competitive and organized nature of debates Lo and McAvoy (2023) . Tree-style discussions (or ‘threads’) are discussions which start from an Original Post (OP) with subsequent comments replying to either the OP or to other comments Seering (2020) . 3 Comparison to Other Surveys Only two studies have surveyed the field of NLP while also considering ideas from Social Science. However, they focus mainly on AM ( AM ). These are the studies of Wachsmuth et al. (2024) and Vecchi et al. (2021) . Wachsmuth et al. (2024) focus primarily on discussion evaluation disregarding its relation to facilitation, which is one of the main goals of our survey. The survey of Vecchi et al. (2021) argues that advancing AM for social good requires a collaborative effort between AM and Social Science. They point out that traditional AM has prioritized the logical structure and soundness of arguments, while overlooking other important dimensions, such as civility, respectfulness, inclusiveness, originality, and the broader impacts of discussions—such as encouraging mutual understanding and problem-solving. Building on these notions, we incorporate ideas from Social Science into NLP -based approaches, discussing both discussion evaluation and facilitation, both with a focus on the potential of LLM . 4 Discussion Quality Evaluation Improving online discussions presupposes being able to define and measure discussion quality . While there have been attempts to provide frameworks for discussion quality evaluation Kies (2022) ; Gerber et al. (2018) , none of them