High-quality generation of dynamic game content via small language models: A proof of concept ISBN: 978-1-4503-XXXX-X/2018/06

Morten I. K. Munk, Arturo Valdivia, Paolo Burelli · 2026

Read High-quality generation of dynamic game content via small language models: A proof of concept ISBN: 978-1-4503-XXXX-X/2018/06 by Morten I. K. Munk, Arturo Valdivia, Paolo Burelli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23206.txt.

Introduction

Morten I. K. Munk, Arturo Valdivia, and Paolo Burelli generate dynamic game content with small language models. Large models promise rich, changing game worlds, but they suffer narrative incoherence and high running costs. Their size also forces cloud access, which offline games cannot assume.

The proof of concept shows small models can produce high-quality content locally. On-device generation keeps games playable without a connection and cuts server bills. It also keeps player data on the machine, a privacy plus for studios. Small models answer the offline problem directly, since they can run on the player's own machine without cloud access.

Catalog record

CREATORMorten I. K. Munk, Arturo Valdivia, Paolo Burelli
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23206

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

High-quality generation of dynamic game content via small language models: A proof of concept ISBN: 978-1-4503-XXXX-X/2018/06 Morten I. K. Munk, Arturo Valdivia, Paolo Burelli Abstract . Large language models (LLMs) offer promise for dynamic game content generation, but they face critical barriers, including narrative incoherence and high operational costs. Due to their large size, they are often accessed in the cloud, limiting their application in offline games. Many of these practical issues are solved by pivoting to small language models (SLMs), but existing studies using SLMs have resulted in poor output quality. We propose a strategy of achieving high-quality SLM generation through aggressive fine-tuning on deliberately scoped tasks with narrow context, constrained structure, or both. In short, more difficult tasks require narrower scope and higher specialization to the training corpus. Training data is synthetically generated via a DAG-based approach, grounding models in the specific game world. Such models can form the basis for agentic networks designed around the narratological framework at hand, representing a more practical and robust solution than cloud-dependent LLMs. To validate this approach, we present a proof-of-concept focusing on a single specialized SLM as the fundamental building block. We introduce a minimal RPG loop revolving around rhetorical battles of reputations, powered by this model. We demonstrate that a simple retry-until-success strategy reaches adequate quality (as defined by an LLM-as-a-judge scheme) with predictable latency suitable for real-time generation. While local quality assessment remains an open question, our results demonstrate feasibility for real-time generation under typical game engine constraints. 1. Introduction Artificial intelligence has long been integral to the video game industry ( 35 ) , with a particular emphasis on adaptive narrative elements, such as believable NPCs ( 4 ; 17 ) . Game elements that provide clear feedback and consistently respond to player agency correlate with increased engagement ( 13 ; 25 ) . Recent advances in generative AI have opened up new avenues for dynamic and adaptive game content generation using large language models (LLMs). For instance, approaches have been explored for NPC dialogue ( 21 ; 22 ) and quest generation ( 1 ) . In this paper, we explore an alternative paradigm for narrative AI in games, which we believe is more suited for the practical reality of integration into video games: An agentic system of aggressively fine-tuned small language models (SLMs), each deliberately constrained to narrow, well-defined generation tasks. The paper is organized as follows: Section 2 outlines our proposed framework and the technical constraints under consideration. Section 3 presents DefameLM, our proof-of-concept implementation, and details both the game loop design and model training. Section 4 contains a quantitative evaluation of DefameLM’s output quality at different levels of quantization, as well as the generation efficiency. In Section 5 we discuss limitations and future directions, and in Section 6 offer conclusions. 1.1. Obstacles of LLMs for dynamic game content Although holding much promise, LLMs appear to have difficulty forming a coherent understanding of complicated worlds, as exemplified by Tsai et al. in a recent study, which showed that a straightforward application of the current strongest LLMs is insufficient for playing a text-based game, Zork I (1977) ( 28 ) . Specifically, they tasked ChatGPT-4 with playing the game with access to the game manual as well as few-shot examples, but cited an inability of the LLM to successfully infer and utilize knowledge about the game world and formulate goals. This suggests that naive applications of LLMs to control NPCs in more complicated game situations will likely fail to adhere to the world and maintain logical coherence. Since maintaining narrative comprehension is essential for player immersion in story-driven games ( 24 ) , the LLMs’ difficulty in maintaining narrative cohesion underscores a significant challenge to LLM integration in narratively-driven games. Müller-Brockhausen et al. ( 18 ) found that even superficial "chatter" (or "barks" in game developer terminology) can struggle to stay in context, highlighting the challenge of full dynamic NPC dialogue. Indeed, lack of consistency and guard-railing of LLMs is an often cited shortcoming for AI generated narrative content ( 6 ; 26 ) . Maintaining narrative consistency requires LLMs to track extensive game state and character information, which can be managed through techniques such as RAG or complex prompting. However, increasing prompt complexity and constraint specificity beyond a certain point has been shown to reduce narrative coherence and creativity ( 2 ) . This creates tension between the need for consistency (requiring more context/constraints) and the need for creative, engaging narratives. 1.2. Are agentic LLM frameworks the solution? Recent studies have suggested that some of the above challenges can be mitigated by adopting an agentic framework, where complex tasks decompose into smaller subtasks handled by separate LLM calls, reminiscent of AI chaining ( 34 ) and chain-of-thought reasoning ( 36 ; 32 ) . As an example, suppose an NPC has to perform a complex task, such as making a decision based on recent events and its current psychological state. This task can then be broken into several separate tasks, some of which can be carried out ahead of time to reduce the final generation time, such as summarizing recent events and evaluating the NPCs state of mind. Jeong et al. ( 12 ) applied this approach, and while players reported high levels of engagement due to the high degree of freedom, the problem of coherence remained, and because the system provided no explicit goals for the LLM, players in turn reported a lack of direction and increased confusion. This points towards a fundamental challenge with narrative content generation, namely that narratives comprise a wide section of incompatible operations, and as such it is necessary to break down the generation by first analysing the specific use-case and narrative structure carefully and being very deliberate about what is generated and why, a point stressed by Gervás et al. ( 8 ) . What they suggest is exactly to separate concerns, e.g. by having separate LLM calls working out the valid plot structures from those simulating characters’ behaviours. Conversely, handling tasks monolithically would require encoding the entire narrative framework into the prompt. For instance, the logic for tracking and developing narrative tension, managing valid transitions, and triggering character development must be handled while maintaining consistency across all these dimensions. An example of this difficulty is quest generation, which touches all these aspects simultaneously. Designing such a prompt would be a monumental task, and whether current LLMs are able to successfully follow those instructions is an open question. The end result would also rely on the LLM’s reasoning rather than the game developer’s, introducing considerable risk for less open-ended applications. For these reasons, approaches that decompose complex tasks into separate calls hold the most promise. One fascinating recent example of story generation using a very deliberate narrative structure in this way is given by Wen et al. ( 33 ) , who constructed coherent branching stories using a directed acyclic graph (DAG) approach as well as a reductionist story framework of rising and falling story beats. The authors observed that the narratological beats were recognizable by their users and that the application of these story beats greatly increased the players’ immersion. One main shortcoming was that the

Read the complete text