Forecasting Conversation Derailments Through Generation

Yunfan Zhang, Kathleen McKeown, Smaranda Muresan · 2025

Read Forecasting Conversation Derailments Through Generation by Yunfan Zhang, Kathleen McKeown, Smaranda Muresan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08905.txt.

Introduction

Yunfan Zhang, Kathleen McKeown, and Smaranda Muresan teach language models to forecast whether an online conversation is about to turn hostile. Moderators would rather see a fight coming than clean it up afterward, yet models that spot insults already on the page do poorly at predicting future blowups.

Their approach imagines the future instead. A tuned model samples several ways the conversation might continue, and the forecast follows the consensus of those imagined paths. Signals about the social dance of the exchange, such as who is responding to whom, can guide the imagination. On standard English test sets the method set new records, and removing pieces of it cost clear accuracy. The authors see uses in moderation, negotiation, and conflict resolution.

Catalog record

CREATORYunfan Zhang, Kathleen McKeown, Smaranda Muresan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-08905

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Forecasting Conversation Derailments Through Generation Yunfan Zhang, Kathleen McKeown, Smaranda Muresan Abstract Forecasting conversation derailment can be useful in real-world settings such as online content moderation, conflict resolution, and business negotiations. However, despite language models’ success at identifying offensive speech present in conversations, they struggle to forecast future conversation derailments. In contrast to prior work that predicts conversation outcomes solely based on the past conversation history, our approach samples multiple future conversation trajectories conditioned on existing conversation history using a fine-tuned LLM. It predicts the conversation outcome based on the consensus of these trajectories. We also experimented with leveraging socio-linguistic attributes, which reflect turn-level conversation dynamics, as guidance when generating future conversations. Our method of future conversation trajectories surpasses state-of-the-art results on English conversation derailment prediction benchmarks and demonstrates significant accuracy gains in ablation studies. 1 Introduction Figure 1: An example conversation from the BNC dataset, including background and the future turn. Offensive speech is highlighted in red. Our task requires forecasting whether the derailment would occur in the future based on the conversation so far. Predicting future derailments from ongoing conversations has a wide range of real-world applications. For instance, in online moderation, the ability to forecast if a discussion thread might devolve into offensive exchanges allows moderators to intervene preemptively. Similarly, in conflict resolution, political hearings, and business negotiations, a system that warns participants of impending conflicts could help mitigate disputes before they escalate. Despite its utility, predicting future conversation derailments presents significant challenges. As shown in examples in Figure 1 , unlike the detection of offensive conversational turns Warner and Hirschberg (2012) ; Poletto et al. (2021) ; Fortuna and Nunes (2018) ; Nobata et al. (2016) , which focuses on identifying harmful speech within a given conversation transcript, our task requires forecasting whether offensive turns will occur later in the conversation. This makes our task significantly more challenging: the model has to predict potential future derailments based on an otherwise benign conversation history, and this demands a nuanced understanding of the conversation’s progression. Our task is further complicated by the inherently diverse nature of human interactions: given the same conversational history, there are multiple possible future trajectories, each with varying topics, styles, and tones. To address these challenges, we propose a novel approach that forecasts future derailments by generating potential continuations of the given conversation . We define conversation derailments as conversations that end with offensive speech or ad hominem attacks, with dataset-specific criteria and annotation procedures explained in Section 4.1 . Our approach is based on the observation that while Large Language Models (LLMs) may not be effective classifiers for forecasting conversation breakdowns directly, they excel at generating conversation continuations. Thus, we adopt a generate-then-predict approach for forecasting conversation derailments. We first fine-tune an LLM on modeling online conversations. We then sample multiple conversation continuations from this fine-tuned LLM, conditioned on the existing conversation history. We feed each generated continuation along with the existing conversation history to a binary conversation derailment classifier and obtain multiple predictions. We determine the final conversation outcome by taking a majority vote of these individual predictions. By sampling multiple plausible continuations and aggregating the individual predictions with majority vote, we reduce the variability of stochastic LLM outputs, leading to more robust and accurate forecasting of conversation derailments. Additionally, inspired by prior work Morrill et al. (2024) and the Circumplex Theory Leach et al. (2015) ; Jonathan et al. (2005) ; Koch et al. (2016) in Psychology, we explore whether the use of social orientation labels can guide the generation of conversation continuations . Social orientation labels (e.g., Assertive, Confrontational) are socio-linguistic attributes that reflect conversation dynamics and emotional states (Figure 2 ), and have been shown to help computational models better capture the flow of conversation Morrill et al. (2024) . We validate our approach on two datasets: the Conversation Gone Awry Wiki Split (CGA-Wiki) dataset Zhang et al. (2018) ; Chang and Danescu-Niculescu-Mizil (2019) , derived from Wikipedia editor discussions, and the Before Name Calling (BNC) dataset Habernal et al. (2018) , based on Reddit’s r/changemyview threads. Our method demonstrates significant accuracy improvements compared to the previous state-of-the-art and few-shot GPT-4o. On the CGA-Wiki dataset, we achieve a 4-7% absolute accuracy improvement, while on the BNC dataset, our method yields an 18-20% improvement. Extensive ablation studies further confirm the effectiveness of our proposed approach. In sum, our contributions are: • A novel approach for predicting conversation outcomes (derailment or not) by generating multiple potential continuations given the conversation so far. • A thorough experimental setup that shows that our proposed approach significantly outperforms prior state-of-the-art methods and powerful new models such as GPT-4o with in-context learning (few-shot, k=4 ). • An exploration of whether social orientation labels can guide the generation of conversation continuations, with mixed impact on derailment prediction accuracy, depending on the conversation genre. Our datasets and models are available through this GitHub repository . 2 Problem Statement and Motivation Our objective is to estimate the likelihood of future conversation derailment based on the benign conversation history up to a certain point. We define conversation derailments as offensive speech and ad hominem attacks, in line with previous work Zhang et al. (2018) ; Habernal et al. (2018) . Formally, let C=\{c_{1},\dots,c_{n}\} represent the conversation history consisting of n turns, where c_{i} denotes the i -th turn in the dialogue. Let y(\{c_{1},\dots,c_{i}\}) be a binary indicator function for conversation derailment over the turns \{c_{1},\dots,c_{i}\} , where y=1 indicates the presence of derailments. Our goal is to estimate the likelihood of future derailments given the first k benign turns, expressed as: \displaystyle P(y(\{c_{k+1},\dots,c_{n}\})=1\mid\{c_{1},\dots,c_{k}\}) \displaystyle\text{subject to }k<n,y(\{c_{1},\dots,c_{k}\})=0 CGA-Wiki BNC Methods Acc. Prec. Rec. F1 Acc. Prec. Rec. F1 BART, No Offensive Turns in Input 65.4 63.9 70.5 67.0 84.2 85.6 82.3 83.9 BART, All Turns 95.5 94.2 96.9 95.5 92.7 91.7 93.8 92.8 Table 1: Accuracy, precision, recall, and F1 scores on CGA-Wiki and BNC . Modern Language Models (LMs) can easily identify offensive speech present in the conversation. However, we notice a significant drop in accuracy when only the benign speech is given and the model is trained to forecast offensive speech in future exchanges. To illustrate the challenges of forecasting conversation derailment, we conducted an experiment comparing the performance of a language model in two settings: detecting offensive speech from a full conversation transcript and predicting future derailments based only on the conversation history prior to any offensive speech. We used BART-Base Lewis et al. (2020) as the underlying model, trained with a Binary Cross Entropy loss to directly predict the conversation derailment. The results, shown in Table 1 , demonstrate that even smaller LMs such as BART perform well in detecting offensive speech when given full transcripts including the offensive turns, achieving over 90% accuracy on both the CGA-Wiki and BNC datasets. However, the model’s performance drops substantially when forecasting future derailments from only benign turns. Specifically, BART’s accuracy falls to 65% on CGA-Wiki and 84% on BNC, representing absolute reductions of 30% and 8%, respectively. These findings emphasize a key challenge in this task: while LMs excel at identifying offensive speech present in the transcript, they struggle significantly when predicting upcoming derailments based solely on benign

Read the complete text