Real-Time World Crafting: Generating Structured Game Behaviors from Natural Language with Large Language Models

Austin Drake, Hang Dong · 2025

Read Real-Time World Crafting: Generating Structured Game Behaviors from Natural Language with Large Language Models by Austin Drake, Hang Dong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16952.txt.

Introduction

Austin Drake and Hang Dong let players program games in plain words in a 2025 paper. Players type natural language, and the game gains new behaviors live. Letting a language model write raw code would be unsafe. Their design instead translates commands into a restricted language.

That restricted language configures a standard game structure at runtime. The team tested the setup inside a two-dimensional spell-crafting game. Models from three major families tried various prompting strategies. A checked model judge rated how well outputs matched creative intent. Bigger models captured intent better than smaller ones. Step-by-step prompting helped creative tasks, while examples suited harder scripts. The pattern offers game builders a safer route to player-made content.

Catalog record

CREATORAustin Drake, Hang Dong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-16952

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Real-Time World Crafting: Generating Structured Game Behaviors from Natural Language with Large Language Models Austin Drake, Hang Dong Abstract We present a novel architecture for safely integrating Large Language Models (LLMs) into interactive game engines, allowing players to “program” new behaviors using natural language. Our framework mitigates risks by using an LLM to translate commands into a constrained Domain-Specific Language (DSL), which configures a custom Entity-Component-System (ECS) at runtime. We evaluated this system in a 2D spell-crafting game prototype by experimentally assessing models from the Gemini , GPT , and Claude families with various prompting strategies. A validated LLM judge qualitatively rated the outputs, showing that while larger models better captured creative intent, the optimal prompting strategy is task-dependent: Chain-of-Thought improved creative alignment, while few-shot examples were necessary to generate more complex DSL scripts. This work offers a validated LLM-ECS pattern for emergent gameplay and a quantitative performance comparison for developers. 1 Introduction Figure 1: A flow diagram of the developed architecture. The LLM interface (left) handles translating Natural Language (NL) into DSL code. The ECS (center) consists of data output by the LLM and logical systems operating on that data. The I/O container (outside edges) is the mode of content delivery and interaction for the player. A long-standing goal in Human-Computer Interaction (HCI) and in interactive art forms is to allow users to control complex simulations in real-time using only natural language, which requires translating between ambiguous human intent and the rigid logic of computer systems Manaris 1998 . While Large Language Models (LLMs) offer a promising approach to this problem, allowing an LLM to generate and execute arbitrary code in a general-purpose language poses significant risks to system stability and security Majumdar and Vogelsang 2024 . To mitigate these risks, we propose a novel software architecture that uses an LLM to interpret natural language commands into a constrained, human-readable Domain-Specific Language (DSL). This intermediate layer provides a safe and verifiable instruction set for a video game simulation, addressing a gap in research on the real-time translation of creative instructions and producing physically plausible behaviors in interactive systems. We use a 2D physics-based video game as a testbed to evaluate this architecture. While many games use LLMs for narrative generation, our work focuses on driving more realized, physically-based systems. Our proposed architecture relies on the Entity-Component-System (ECS) pattern Romeo 2016 ; Wiebusch and Latoschik 2015 , whose flexible, compositional design is well-suited for dynamically modifying object behaviors at runtime based on non-deterministic LLM outputs. Our prototype, Latent Space , features a spell-crafting system and a cellular automata sandbox to test the LLM’s ability to generate both compositional and logic-based DSL code. This paper investigates how effectively LLMs can translate creative commands into structured, executable instructions. We experimentally evaluate the performance of models from the Gemini, GPT, and Claude families with various prompt engineering strategies to determine which factors most influence the generation of logically consistent and physically plausible game mechanics. Our contributions include: (1) a validated, DSL-mediated architecture for safely integrating LLMs into interactive systems; (2) a quantitative comparison of model performance on creative and logical generation tasks; (3) an analysis of prompting strategies for maximizing creative throughput in language-driven interfaces; and (4) an evaluation of the architecture, including both human and automated LLM-as-judge ratings. Our demo, experiment implementation, and generated data are available as a GitHub repository 1 1 1 https://github.com/austin-the-drake/real-time-world-crafting-wordplay-demo . 2 Background 2.1 Historical Challenges in Games Natural language interfaces in interactive systems and art forms have historically been limited by the challenges of linguistic ambiguity and the prohibitive authoring cost required to support wide-reaching player freedom. Early parser-based systems such as Zork Lebling et al. 1979 , while revolutionary, were also very rigid, and frustratingly rejected any inputs that did not exactly match their pre-programmed vocabularies. Later NLP-driven experiences including Façade Mateas and Stern 2003 ; Mateas and Stern 2005 addressed this problem by mapping free-form user input to the closest available pre-written options, but this comes with an immense workload of manually authoring responses to maintain the illusion of natural conversation. These challenges have led modern games to often rely on systems such as “dialogue wheel”, which restrict player input to lessen the authoring burden at the cost of player expressiveness Taylor-Giles 2020 . Figure 2: A pair of screenshots from Latent Space’s Battle Mode. Natural language magic descriptions, e.g., “Singeing arrow volley,” entered into the text interface (left), can have explosive consequences (right). Each team’s remaining players and their health (red/purple) are visible above, as is the amount of time remaining for each turn. 2.2 LLM-Driven Architectures LLMs can simultaneously interpret ambiguous text and generate novel content, which presents a potential solution to both historical problems at once. Contemporary research is moving from using models to select from pre-written options (as in Façade ) towards generating fully dynamic outcomes in real-time Gallotta et al. 2024 . Applications range from generating dynamic narratives in co-creative storytelling and role-playing games Sun et al. 2023 ; Sanders 2024 ; Triyason 2023 to driving complex agent behaviors in simulations and strategy games Park et al. 2023 ; Yim et al. 2024 ; Ito and Takahashi 2024 . A promising direction for LLMs is to support natural language as a free-form scripting interface for game mechanics. Several existing projects constrain player input to make this task more tractable BigFrogg 2025 ; However, this project investigates the challenge of translating fully unconstrained language into game logic. Architectural inspiration for this task can be drawn from domains like robotics and data science Liu et al. 2024 ; Li et al. 2025a ; Sah et al. 2024 . A common pattern involves an LLM translating a high-level command into a structured, machine-readable representation (e.g., API calls, JSON), which is then executed by a deterministic backend Perrault and Grosz 1988 ; Sah et al. 2024 . Robotics frameworks like Interactive Task Planning (ITP) use this to break abstract goals down into concrete actions or update plans based on new context Li et al. 2025a . A major focus is grounding the LLM’s output in the state and capabilities of the target system Bernardi et al. 2025 . 3 System Architecture and Implementation Our demo, Latent Space , is designed to translate natural language into executable game mechanics. The primary design challenge was to safely and reliably use the non-deterministic output of an LLM within a deterministic game engine. To achieve this, we developed a modular architecture (Figure 1 ) that sandboxes the LLM’s influence. Instead of generating general-purpose code, which is unconstrained and error-prone, the LLM’s role is strictly limited to generating structured data in a custom JSON-based DSL. The architecture’s layers are: (1) an LLM interface that translates a player’s request into the DSL; (2) a custom ECS framework that parses the DSL to instantiate and configure the corresponding game entities and their components; and (3) a commercial game engine that acts as an I/O container, rendering the ECS state and capturing player input. This separation of concerns improves system stability and also allows the LLM’s translation performance to be tested in isolation. 3.1 Prototype and DSL Design Latent Space is designed to provide players with unconstrained, free-form natural language input,

Read the complete text