Managing Escalation in Off-the-Shelf Large Language Models

Sebastian Elbaum, Jonathan Panter · 2025

Read Managing Escalation in Off-the-Shelf Large Language Models by Sebastian Elbaum, Jonathan Panter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01056.txt.

Introduction

Sebastian Elbaum and Jonathan Panter test simple ways to curb escalation advice from language models, in this 2025 paper. Security agencies now use public-style models for planning. Past studies found such models often urge escalatory moves in war-game prompts.

The pair added two plain, non-technical interventions to a published war-game design. Escalation fell sharply across the game with those changes in place. The authors argue calls to bar models from security work are premature, since agencies will use them regardless. Their stance is to guide use with workable guard steps that align outputs with security goals, including calm escalation control.

Catalog record

CREATORSebastian Elbaum, Jonathan Panter
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01056

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Managing Escalation in Off-the-Shelf Large Language Models Sebastian Elbaum, Jonathan Panter Abstract U.S. national security customers have begun to utilize large language models, including enterprise versions of “off-the-shelf” models (e.g., ChatGPT) familiar to the public. This uptake will likely accelerate. However, recent studies suggest that off-the-shelf large language models frequently suggest escalatory actions when prompted with geopolitical or strategic scenarios. We demonstrate two simple, non-technical interventions to control these tendencies. Introducing these interventions into the experimental wargame design of a recent study, we substantially reduce escalation throughout the game. Calls to restrict the use of large language models in national security applications are thus premature. The U.S. government is already, and will continue, employing large language models for scenario planning and suggesting courses of action. Rather than warning against such applications, this study acknowledges the imminent adoption of large language models, and provides actionable measures to align them with national security goals, including escalation management. 1 Introduction Public concern about the risks of military artificial intelligence (AI) has focused predominantly on “battle management” software for making tactical and operational decisions in warfare. Proponents of such software argue that it will provide a decisive advantage in tomorrow’s high-speed, high-complexity conflicts. Critics, meanwhile, argue that removing the human “from the loop” could place civilians at risk or exacerbate escalation dynamics [ 2 , 7 , 8 , 11 , 1 ] . Despite such concerns, the use of AI for tactical and operational decision-making is well under way. Applying AI to even higher-level questions in national security – that is, political and strategic questions – appears to be the next frontier. Applications using large language models (LLMs) are already under development to assist senior officials with scenario planning and course-of-action development [ 4 , 5 , 14 ] . LLMs comprise a specific type of AI. They are very large, deep neural networks trained on vast amounts of data, which process natural-language input queries to generate text responses. Recent research has raised alarms about the alleged proclivity of LLMs to escalate when applied to national security decision-making. In this paper, we test two simple, non-technical user interventions for managing escalation using “off-the-shelf” LLMs – the kind available to the public, such as ChatGPT or Llama, that are often the baselines for enterprise models offered to government customers. We find that these interventions substantially reduce the risks of escalation identified in recent research. If such interventions can reduce model escalation and output variation to such a high degree, then the cause for alarm may not be as great as recent reports intimate. 2 LLMs in National Security Applications LLMs offer great promise for strategic decision-making. AI models can ingest more data (particularly unstructured data), and process it more rapidly, than humans can. Moreover, they can generate ideas without human bias or emotion. LLMs are particularly useful for wargaming, as planners can run many more simulations than is possible with humans alone. In the future, LLMs might support political and strategic decision-making by aggregating and summarizing documents, predicting adversary moves, or generating recommended actions [ 3 , 6 ] . For example, a staff officer at U.S. European Command could feed an LLM data about recent Russian troop movements, and prompt it to suggest deterrent options for the combatant commander. However, employing AI for such decisions may prove even more controversial than the use of AI for battle management. Strategic applications intrude on realms that require human insight, empathy, and creativity, such as interpreting an adversary’s intentions, or devising methods to signal national resolve. In addition, LLMs exhibit behaviors and limitations that could destabilize diplomatic bargaining and signaling, interactions which are already notoriously delicate and risky. First, current LLMs suffer from “hallucinations” (generating false information) and errors of omission. Second, LLMs are only as good as their training data and process, as they cannot themselves determine what is true or false, or “know” if pertinent context is missing. Third, many of the purported strengths of AI – such as its ability to act without emotion, or to act extremely quickly – may also be weaknesses in crises. For instance, crisis bargaining frequently requires leaders to put themselves in their opponents’ shoes, comprehend the non-material costs of war, or slow down a problem to give time for cooler heads to prevail [ 13 , 10 ] . Recent research illuminates these risks. Rivera et al. [ 12 ] deployed five off-the-shelf LLMs in wargame simulations, and found that all were prone to sudden, even nuclear, escalation. Each simulation provided one of three starting conditions – neutral, invasion, or cyber-attack – to eight LLM “nation agents”, representing countries with distinct identities and histories. Over the course of a simulated 14-day crisis, agents responded to one another by selecting from 27 pre-set options, ranging from negotiations to initiating conflict. Across all scenarios and all models, the authors observed an initial escalation. None of the models deescalated over the course of a simulation, and all seven were prone to arms-racing. There were no predictable patterns in escalation decisions. One model in particular, GPT-4-Base, escalated more than the others. This was the only model without reinforcement learning from human feedback (RLHF), a fine-tuning method whereby humans supervise a model’s training and score the quality of its outputs, then re-train the model by rewarding preferred behavior. Jensen et al. [ 9 ] compared the responses of seven LLMs to 400 geopolitical scenarios and 66,000 question-answer pairs. The scenarios covered four domains of international relations: escalation, intervention, cooperation, and alliances. Models were provided with state identities (e.g. the United States or Russia) and asked to select from pre-set responses. The authors observed substantial variation among model outputs and across the assigned actor identities. For instance, Llama 3.1 8B Instruct escalated far more than Claude 3.5 Sonnet, and models acted the most aggressively when assigned to play the U.S. or United Kingdom. 3 Study The studies from Rivera et al. and Jensen et al. conclude that deploying LLMs for national security applications poses grave risks, and recommend further research on safeguards. Heeding these calls, we demonstrate two simple interventions to control off-the-shelf models’ escalatory tendencies. Critically, these methods are available to practitioners with no technical background, and at no cost. Building on Rivera et al.’s base design, we construct a wargame with LLM nation agents played by a single off-the-shelf model, Mistral-7B-Instruct-v0.3, run over 14 days. We apply two categories of treatments: temperature variation and prompt engineering. Temperature is a parameter that regulates the randomness of a model’s output, and hence the “creativity” of its generated text. Rivera et al. and Jensen et al. used two different temperatures, employed consistently across their simulations. Notably, in the Jensen et al study, variation among LLMs in escalation decisions occurred despite a consistent temperature setting of 0 across models. However, neither study examined temperature as an independent variable to achieve leverage on escalation outputs. Here, we examine the effects of temperature variation on escalation, as users may desire different levels of output creativity based on the task or scenario they face. Prompt engineering refers to user efforts to write instructions and queries carefully and deliberately, to elicit LLM responses more aligned with the user’s goals.

Read the complete text