Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue DOI: 10.1145/3805689.3812264 The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, Canada ISBN: 979-8-4007-2596-8/2026/06 CCS: Computing methodologies Natural language processing CCS: Social and professional topics Computing / technology policy
Hoffmann Muki, Olukunle Owolabi · 2026
Read Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue DOI: 10.1145/3805689.3812264 The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, Canada ISBN: 979-8-4007-2596-8/2026/06 CCS: Computing methodologies Natural language processing CCS: Social and professional topics Computing / technology policy by Hoffmann Muki, Olukunle Owolabi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04177.txt.
Introduction
Hoffmann Muki and Olukunle Owolabi test whether large language models label conflict events fairly. They compare four general open models and two models adapted for African conflict news against ACLED, a checked data set of events in Nigeria and Cameroon.
The general models often turned real battles into attacks on civilians, while the adapted models were much more even on that measure. Yet all models still favoured state actors over non-state actors in the same tactical setting, and wording tied to place could shift the labels. The results warn against using such tools without close human checks.
Catalog record
| CREATOR | Hoffmann Muki, Olukunle Owolabi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 112 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04177
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue DOI: 10.1145/3805689.3812264 The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, Canada ISBN: 979-8-4007-2596-8/2026/06 CCS: Computing methodologies Natural language processing CCS: Social and professional topics Computing / technology policy Hoffmann Muki, Olukunle Owolabi Abstract . As LLMs enter conflict monitoring, understanding systematic distortions in their outputs is critical for humanitarian accountability. We evaluate four vanilla open-weight models Gemma 3 4B, Llama 3.2 3B, Mistral 7B, and OLMo 2 7B and two domain-adapted models, AfroConfliBERT and AfroConfliLLAMA, on Nigeria and Cameroon conflict-event classification against ACLED, a gold-standard dataset with multi-stage verification. We find a bifurcated divergence in normative directionality. Open-weight models exhibit statistically significant False Illegitimation bias: Gemma misclassifies to 18.29% of legitimate battles as civilian-targeted violence while making zero False Legitimation errors. By contrast, AfroConfliBERT and AfroConfliLLAMA achieve near-directional neutrality, with Legitimization Bias differences indistinguishable from zero. Yet domain adaptation does not eliminate actor-based selection bias. Both adapted models show statistically significant actor bias comparable to vanilla LLMs; in Nigeria, state actors are legitimized 36.5% more often than non-state actors in identical tactical contexts. Open-weight outputs are also fragile to geography-specific lexical framing: delegitimizing phrases produce flip rates up to 66.7% in Cameroon and 34.2% in Nigeria, while perturbations salient in one context may not matter in another. Error trace profiling shows models mask normative bias through unfaithful rationale confabulations. In contrast, AfroConfliBERT and AfroConfliLLAMA are largely robust, with near-zero flip rates across perturbation categories. Overall, current models are not ready for unsupervised deployment in conflict monitoring. We call for fairness-aware fine-tuning to reduce actor-based selection bias, mandatory adversarial robustness evaluation against lexical manipulation, and context-specific human-in-the-loop oversight calibrated to regional difficulty. 1. Introduction Growing interest in applying Large Language Models (LLMs) to humanitarian, security, and policy-making contexts has prompted recent exploration of their potential as analytical engines for automated conflict event extraction ( 30 ; 36 ) . As this exploration grows, so does their potential role in the decision-making pipelines of non-governmental organizations and international security bodies. Yet the application of LLMs to conflict monitoring raises critical questions about their normative neutrality: whether model outputs faithfully reflect the empirical realities of political violence or introduce systematic distortions that could undermine the credibility of human rights documentation and humanitarian response ( 4 ; 30 ; 36 ) . The utility of any automated monitoring system in this domain depends on its adherence to a rigorous, operationalizable conflict taxonomy. This study utilizes the Armed Conflict Location & Event Data (ACLED) project as our foundational ground truth. ACLED is widely regarded as the gold standard for near-real-time conflict monitoring due to its independent, impartial, and highly granular coding methodology, which provides the necessary technical distinctions required for legal and security analysis ( 33 ) . Critically, the classification of conflict events is not merely a linguistic task but a normative one. Prior work demonstrates that LLM outputs vary systematically with the framing and language used to describe conflict events ( 39 ) , and that alignment procedures may introduce representational asymmetries across geographic and demographic contexts ( 35 ) . However, how such patterns manifest when LLMs are applied to conflict data, whether they vary systematically across model architectures, and whether domain adaptation can mitigate them, have not been addressed heretofore. This study addresses those questions through a meticulous empirical investigation of general-purpose and domain-adapted models on conflict event data from Nigeria and Cameroon, via equalized odds, counterfactual analysis, few-shot prompting and error trace profiling. We investigate these questions using conflict event data from Nigeria and Cameroon, evaluating four general-purpose LLMs: Gemma 3 4B, Llama 3.2 3B, Mistral 7B, and OLMo 2 7B, alongside two domain-adapted models introduced in this study: AfroConfliBERT, a domain-adapted version of ConfliBERT fine-tuned on 81,104 ACLED African conflict events, and AfroConfliLLAMA, a Llama-based decoder fine-tuned on the same corpus via LoRA. For the sake of brevity, we will henceforth refer to the vanilla LLMs as Gemma, Llama, Mistral and Olmo respectively. Our primary contributions are five-fold: • We provide the first systematic comprehensive evaluation of LLM bias on ACLED, bridging the gap between computational linguistics and empirical conflict studies, through a targeted analysis of Cameroon and Nigeria. • We offer a rigorous quantification of normative directionality in model errors, examining whether observed divergences across architectures reflect systematic actor-based bias. • We evaluate the effect of few-shot in-context learning (ICL) and lexical framing on vanilla LLMs, examining whether few-shot examples and lexical cues shift normative directionality in model outputs, or whether such factors merely redistribute rather than eliminate underlying bias. • We perform a granular error-trace analysis using Rationale-Flip Concordance ( 45 ) and Layer Integrated Gradients ( 40 ) , demonstrating that apparent gains in model calibration often mask underlying confabulation, where models justify noteworthy flips through untrustworthy explanations. • We introduce AfroConfliBERT and AfroConfliLLAMA, domain-adapted versions of ConfliBERT and Llama fine-tuned on ACLED African conflict event data, investigating whether domain adaptation mitigates the observed biases and what limitations remain. The remainder of this paper is organized as follows. Section 2 situates our work within the socio-technical literature on LLMs in conflict analysis, followed by a detailed account of our methodology and experimental design in Sections 3 and 4 . Section 5 presents our empirical findings and their interpretation, while Section 6 concludes by discussing the implications for the responsible deployment of AI in international security policy. To provide further technical depth, the Appendix includes per-class model metrics (Section A ), formal definitions for legitimization and fairness (Sections B and D ), and an analysis of model calibration (Section C ). We also examine model sensitivity to contextual length in Section E and conclude with a qualitative audit of the primary model disagreements in Section F . Figure 1 . Top ten conflict actors in Nigeria (2014–2024), illustrating the diverse landscape of conflict with participation from state actors, community militias, and other non-state groups. Top ten conflict actors in Nigeria Top ten conflict actors in Nigeria (2014–2024), illustrating the diverse landscape of conflict with participation from state actors, community militias, and other non-state groups. 2. Background and Related Work Actor Definitions Analysis of political violence requires a precise taxonomy of the participating entities. Following the Weberian tradition, we define state actors as formal agents of a recognized government, such as military and police forces, that successfully claim a monopoly on the legitimate use of physical force within a defined territory ( 33 ) . Conversely, non-state actors are organized, armed groups that operate independently of central government control, ranging from rebel movements seeking national power to communal militias focused on local security or identity ( 20 ) . Context of Conflict in Nigeria and Cameroon Conflict in Africa is characterized by a distinctive landscape that differs fundamentally from patterns observed in the Global North. The continent hosts 46% of the world’s internally displaced persons, with the majority fleeing from violence rather than natural disasters ( 22 ) . Unlike conflicts in Western contexts, African conflicts are