Agents Play Thousands of 3D Video Games
Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, Wei Yang · 2025
Read Agents Play Thousands of 3D Video Games by Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, Wei Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13356.txt.
Introduction
Zhongwen Xu and co-authors present PORTAL in 2025, a framework that lets AI agents play thousands of 3D video games. Instead of training by trial and error, which costs huge computing time, their agents write their own behavior plans in a special game language. A large language model turns the play problem into a writing problem.
The plans are behavior trees that mix fixed rules with neural parts, keeping strategy while allowing quick adaptation to new games. One framework thus spans many game worlds without retraining from scratch for each. The authors argue the method keeps depth of play at far lower cost. Game AI builders get a route to broad, flexible agents that can be inspected as readable trees.
Catalog record
| CREATOR | Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, Wei Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13356
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agents Play Thousands of 3D Video Games Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, Wei Yang Abstract Abstract: We present PORTAL 1 1 1 Portal symbolizes a gateway that provides access to thousands of game worlds and it represents connection and transition between different gaming domains. , a novel framework for developing artificial intelligence agents capable of playing thousands of 3D video games through language-guided policy generation. By transforming decision-making problems into language modeling tasks, our approach leverages large language models (LLMs) to generate behavior trees represented in domain-specific language (DSL). This method eliminates the computational burden associated with traditional reinforcement learning approaches while preserving strategic depth and rapid adaptability. Our framework introduces a hybrid policy structure that combines rule-based nodes with neural network components, enabling both high-level strategic reasoning and precise low-level control. A dual-feedback mechanism incorporating quantitative game metrics and vision-language model analysis facilitates iterative policy improvement at both tactical and strategic levels. The resulting policies are instantaneously deployable, human-interpretable, and capable of generalizing across diverse gaming environments. Experimental results demonstrate PORTAL’s effectiveness across thousands of first-person shooter (FPS) games, showcasing significant improvements in development efficiency, policy generalization, and behavior diversity compared to traditional approaches. PORTAL represents a significant advancement in game AI development, offering a practical solution for creating sophisticated agents that can operate across thousands of commercial video games with minimal development overhead. Experiment results on the 3D video games are best viewed on our project website . An agent must learn from its experience. – Rich Sutton 1 Introduction Games have served as foundational testing grounds for artificial intelligence development since the field’s inception [ 15 , 9 , 17 , 26 , 2 , 27 ] . They provide ideal environments for measuring and advancing cognitive capabilities including pattern recognition, memory utilization, reasoning, sophisticated planning and generalization. The pursuit of creating AI systems capable of generalizing effectively across diverse game environments represents a significant and ongoing challenge at the intersection of academic AI research and the commercial game industry. The gaming landscape underwent a profound transformation with the rise of user-generated content (UGC) platforms, most notably pioneered by Roblox. UGC platforms have demonstrated unprecedented scale in both creation and consumption. By allowing users to design, build, and share their own games within a unified ecosystem, UGC games fundamentally altered the game development paradigm. The explosive growth of UGC gaming created a unique challenge for traditional game-playing AI systems. While conventional AI approaches had made remarkable progress in mastering specific games with well-defined rulesets – from Go [ 17 ] to StarCraft II [ 26 ] – they faced inherent limitations in environments characterized by constant flux and unbounded creativity. Traditional game AI typically relies on extensive domain-specific engineering, carefully crafted heuristics, or reinforcement learning methodologies that require millions/billions of training iterations within a single game environment. Such approaches simply cannot scale to match the pace and diversity of user-created content, where thousands of new games with unique mechanics, aesthetics, and objectives emerge daily. This fundamental mismatch between traditional AI capabilities and the rapidly expanding universe of user-generated games highlights a critical need for more adaptive, generalizable AI approaches – a gap that we are aiming to bridge here. This paper introduces PORTAL( P olicy O ptimization and R easoning for T actical A rtificial L earning), a novel approach to generating game-playing artificial intelligence (AI) agents capable of operating across thousands of 3D video games. By leveraging large language models (LLMs) [ 10 , 1 , 22 , 23 , 24 ] to generate specialized behavior trees (BTs) [ 3 ] expressed in domain-specific language (DSL), we establish a new paradigm for AI agent development that combines the strategic reasoning of LLMs with the real-time performance requirements of commercial video games. Unlike traditional reinforcement learning approaches [ 26 , 2 , 32 , 5 , 14 ] that require massive simulation resources and distributed training, our method decouples the tactical planning process from execution, enabling rapid development and deployment of sophisticated agents. This decoupling allows for the generation of diverse, adaptable, and human-like behaviors while maintaining the computational efficiency necessary for commercial application. PORTAL represents a significant advancement in the field by transforming complex decision-making problems into language modeling challenges. Through an innovative hybrid architecture that integrates rule-based nodes with neural network components, we demonstrate how LLM-generated policies can be effectively executed in dynamic, real-time environments without the latency limitations that typically constrain LLM-based agents. On methodology, we propose a meta algorithm for decision-making problems that fundamentally reimagines the role of large language models (LLMs) in game AI systems. Unlike prior works [ 27 , 8 , 1 ] where LLMs directly function as actor policies , our approach positions these models as architects that generate policy structures . This important innovation creates a hybrid architecture that leverages the strengths of both paradigms: LLMs design sophisticated policy frameworks composed of interconnected basic building blocks, with each node representing discrete functionality. These nodes can then be implemented as neural networks, enabling dynamic adjustment to changing game environments. The nodes can also be implemented with rule-based codes, where it brings exact control over behaviors. The resulting system maintains the strategic sophistication and broadly contextual awareness of LLMs while incorporating the adaptability and responsiveness of neural network approaches. The remainder of this paper is organized as follows: Section 2 reviews related work in behavior trees, reinforcement learning, and language models for game playing AIs. Section 3 details our methodology, including the hybrid policy structure, DSL representation, and an agent-environment loop with reflection feedbacks. Section 4 presents experimental results, and Section 5 discusses implications and potential applications beyond gaming. We conclude our work and point out directions for future research. 2 Related Work Behavior Trees : Behavior Trees (BTs) [ 3 ] have established themselves as a predominant paradigm for artificial intelligence in modern game development, particularly within sophisticated engines such as Unreal Engine 5 2 2 2 Unreal Engine 5 Documentation on Behavior Trees and Unity. Their popularity stems from their capacity to organize complex AI behaviors in a structured, modular, and visually intuitive manner. The foundational components of BTs include: • Selector Nodes: Implement prioritized decision-making by evaluating child nodes sequentially until one succeeds, enabling fallback behaviors. • Sequence Nodes: Execute a series of actions in order, succeeding only if all component actions succeed, facilitating complex, multi-step behaviors. • Task Nodes: Perform atomic game actions such as movement, attack sequences, or environmental interactions. • Condition Nodes: Evaluate environmental states to determine execution paths. This architecture is complemented by a Blackboard system that maintains state information accessible to all nodes within the tree. In commercial implementations such as Unreal Engine 5, a dedicated AI controller periodically evaluates the behavior tree against current game state, determining appropriate actions for non-player characters (NPCs). The modular structure of BTs facilitates code reuse, simplifies debugging, and enables designers to implement sophisticated behaviors without extensive programming expertise. However, traditional BTs rely primarily on manually crafted rules and conditions, limiting their adaptability to novel or highly dynamic scenarios. Our work extends this established framework by incorporating neural components and LLM-based generation, preserving the interpretability and modularity of classical