Conflicts in Texts: Data, Implications and Challenges

Siyi Liu, Dan Roth · 2025

Read Conflicts in Texts: Data, Implications and Challenges by Siyi Liu, Dan Roth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19472.txt.

Introduction

Siyi Liu and Dan Roth survey conflicts inside texts and the trouble they cause for language technology, in this 2025 paper. AI models both read and produce clashing claims, from web pages that disagree to annotations that encode human disputes. Ignoring such clashes can quietly rot a model's reliability.

The survey sorts conflicts into three zones: natural web text, human-labeled training data, and outputs produced when models run. Each zone gets its causes, harms, and fixes reviewed. The authors argue for conflict-aware systems that can reason over rival claims instead of flattening them. Open challenges and research directions close the survey.

Catalog record

CREATORSiyi Liu, Dan Roth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19472

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Conflicts in Texts: Data, Implications and Challenges Siyi Liu, Dan Roth Abstract As NLP models become increasingly integrated into real-world applications, it becomes clear that there is a need to address the fact that models often rely on and generate conflicting information. Conflicts could reflect the complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs. In all cases, disregarding the conflicts in data could result in undesired behaviors of models and undermine NLP models’ reliability and trustworthiness. This survey categorizes these conflicts into three key areas: (1) natural texts on the web , where factual inconsistencies, subjective biases, and multiple perspectives introduce contradictions; (2) human-annotated data , where annotator disagreements, mistakes, and societal biases impact model training; and (3) model interactions , where hallucinations and knowledge conflicts emerge during deployment. While prior work has addressed some of these conflicts in isolation, we unify them under the broader concept of conflicting information , analyze their implications, and discuss mitigation strategies. We highlight key challenges for developing conflict-aware and robust NLP systems, and propose concrete research directions to address them. 1 Introduction The rapid advancement of natural language processing (NLP), particularly with the rise of large language models (LLMs), has led to their widespread adoption in daily tasks, information retrieval, and decision-making processes. However, the increasing complexity of these models reveals various types of conflicts at multiple stages, including training, annotation, and model interaction, affecting the reliability and trustworthiness of downstream applications. For example, training models on data containing factual contradictions, annotation disagreements, or prompts that contradict a model’s parametric knowledge can introduce inconsistencies with unpredictable consequences Pavlick and Kwiatkowski (2019) ; Sap et al. (2019) . Figure 1: Examples of the three different areas of conflicts discussed in this work. The first example describes a case where two different entities of the same name are found naturally on the web , the second example elaborates the annotation disagreement in a sentiment analysis task, and the third showcases a knowledge conflict between the context and memory of LLMs during model interactions . Existing work on conflicts in NLP tends to focus on specific issues, such as annotation disagreements Uma et al. (2021) ; Klie et al. (2023) , hallucinations and factuality Zhang et al. (2023) ; Wang et al. (2023) , and knowledge conflicts Xu et al. (2024) ; Feng et al. (2024) , without synthesizing these problems into a broader perspective. In this survey, we conceptualize these diverse challenges under the umbrella of conflicting information and analyze their origins, implications, and mitigation strategies. {forest} t Figure 2: Taxonomy of conflicts in texts. To ensure comprehensive and representative coverage of conflicts in NLP, we first establish a high-level categorization encompassing three primary sources: (1) natural conflicts present in web data, (2) conflicts arising from human annotation, and (3) conflicts emerging from model interactions. Notably, conflicts found in natural web texts and human-annotated datasets are primarily present in the training data—i.e., the inputs to models—whereas conflicts involving model interactions can arise in various forms, such as inconsistencies between model outputs and their inputs, contradictions among multiple outputs, or conflicts within the outputs themselves. For each category, we identify influential and widely cited survey papers as initial seed works Uma et al. (2021) ; Klie et al. (2023) ; Zhang et al. (2023) ; Xu et al. (2024) ; Feng et al. (2024) ; Wang et al. (2023) . Building upon these seeds, we systematically trace and incorporate the most impactful and representative studies for each type of conflict through citation chaining and targeted literature searches across major databases. This approach enables us to synthesize developments in each category and connect them, thereby providing an integrated discussion of current challenges, impacts on downstream tasks, and promising future directions for conflict-aware AI systems (See Appendix B for more discussion about our survey methodology). The abundance of online data is accompanied by inherent conflicts, stemming from diverse sources, interpretations, and biases. These conflicts manifest as factual conflicts , such as semantic ambiguities Pavlick and Tetreault (2016) ; Min et al. (2020) 1 1 1 The previous version on ACL Anthology has a wrong paper title for a reference here. and factual inconsistencies Pham et al. (2024) ; Liu et al. (2024) , or as conflicts in opinions related to political ideologies Entman (1993) ; Recasens and et al. (2013) and perspectives Chen et al. (2019) ; Liu et al. (2021) . Factual conflicts are particularly prevalent in open-domain question answering (QA) and retrieval-augmented generation (RAG) systems Chen et al. (2017) , where aggregating knowledge from multiple sources introduces inconsistencies Liu et al. (2024) . These challenges highlight the need for conflict-aware retrieval and reasoning mechanisms to improve model reliability Xie et al. (2024) . Unlike factual conflicts, opinionated disagreements reflect the variability in human interpretation, beliefs, and ideological stances Chen et al. (2019) ; Fan et al. (2019) . The presence of conflicting viewpoints complicates tasks such as summarization, sentiment analysis, and dialogue generation, where maintaining coherence and neutrality is crucial Liu et al. (2021) ; Lee et al. (2022) . Furthermore, the uneven distribution and biases of web data also affects models to behave from a Western perspective Ramaswamy et al. (2023) ; Mihalcea et al. (2025) . Another significant conflict arises in human-annotated data . For instance, annotation disagreements persists in both subjective and seemingly objective NLP tasks Mostafazadeh Davani et al. (2022) . Disagreements are widespread in sentiment analysis Wan et al. (2023) , hate speech detection Sap et al. (2022) , and even natural language inference (NLI) Pavlick and Kwiatkowski (2019) . Models trained on aggregated (e.g. majority-vote) labels struggle with ambiguous or high-disagreement examples, often treating them as hard-to-learn or mislabeled Anand et al. (2024) . Pavlick and Kwiatkowski (2019) also find that standard NLI models’ uncertainty does not reflect the true ambiguity present in human opinions, leading to overconfidence in contentious cases. In addition, annotation biases —such as those related to race, gender, and geography—skew model predictions and reinforce societal biases Buolamwini and Gebru (2018) ; Sap et al. (2022) ; Pei and Jurgens (2023) . These issues highlight the need for fair and representative annotations that capture the complexity of human disagreement. Conflicts also emerge during interactions with models , manifesting as knowledge conflicts between model memories and contexts, and hallucinations in generated outputs. Knowledge conflicts arise when a model’s internal memory contradicts external contextual evidence, as shown by Longpre et al. (2021) , who found that models often overly depend on memorized knowledge, leading to hallucinations. Neeman et al. (2023) proposed separating parametric and contextual knowledge to improve interpretability, while Xie et al. (2024) examined LLMs’ confirmation bias, showing how models inconsistently handle contradictory evidence. Additionally, hallucinations—ranging from factual inconsistencies Lin et al. (2022) ; Ouyang et al. (2022) to contextual hallucinations Maynez et al. (2020) ; Kryscinski et al. (2020) —further undermine model reliability. Various mitigation strategies have been proposed, including retrieval augmentation Lewis et al. (2020) ; Shuster et al. (2021) , hallucination detection Manakul et al. (2023) , and knowledge graph-based verification Guan et al. (2024) . In this survey, we systematically examine

Read the complete text