Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing
Raihana Ferdous, Fitsum Kifetew, Davide Prandi, Angelo Susi · 2025
Read Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing by Raihana Ferdous, Fitsum Kifetew, Davide Prandi, Angelo Susi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14606.txt.
Introduction
Game bugs hide in corners testers never reach, and Raihana Ferdous and co-authors send a curious team of learning agents to hunt them, in work from twenty twenty-five. Solo agents stall on big three-dimensional worlds. Their team splits the search, driven by curiosity rewards for new ground.
Across levels of one such game, the team beat a solo learner on three coverage yardsticks. It also finished faster. Teamwork plus curiosity, the authors argue, beats either alone for game testing. Studios drowning in world size gain a tireless, self-guided play crew.
Catalog record
| CREATOR | Raihana Ferdous, Fitsum Kifetew, Davide Prandi, Angelo Susi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14606
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing Raihana Ferdous, Fitsum Kifetew, Davide Prandi, Angelo Susi Abstract Recently testing of games via autonomous agents has shown great promise in tackling challenges faced by the game industry, which mainly relied on either manual testing or record/replay. In particular Reinforcement Learning (RL) solutions have shown potential by learning directly from playing the game without the need for human intervention. In this paper, we present cMarlTest , an approach for testing 3D games through curiosity driven Multi-Agent Reinforcement Learning (MARL). cMarlTest deploys multiple agents that work collaboratively to achieve the testing objective. The use of multiple agents helps resolve issues faced by a single agent approach. We carried out experiments on different levels of a 3D game comparing the performance of cMarlTest with a single agent RL variant. Results are promising where, considering three different types of coverage criteria, cMarlTest achieved higher coverage. cMarlTest was also more efficient in terms of the time taken, with respect to the single agent based variant. I Introduction Software testing has long been recognized as a critical aspect of the development process and accounts for a significant portion of the overall budget. Over the years researchers have developed numerous techniques and tools to automate as much as possible the testing process, hence reducing the associated cost. Despite research efforts, testing remains a primarily manual task for certain software systems, such as computer games [ 1 ] . Computer games, and particularly 3D games, bring several challenges to the table when considering their automated testing. They are highly interactive, involving continuous input spaces, as well as visual elements that are not easy to deal with in an automated manner. Consequently, automated testing of games has been mostly limited to capture-replay based solutions, which tend to produce brittle tests. Recent research using Reinforcement Learning (RL) has shown promising results [ 2 , 3 , 4 , 5 , 6 , 7 ] , where RL is used to train agents that can play the game and win it. There have been recent efforts in the automated testing of the wider scope of software applications that fall under the category of Extended Reality (XR) systems [ 8 ] , which include 3D games as well. In a previous work, we proposed using single agent RL for testing games where we obtained encouraging results [ 9 ] . We also observed that while RL works reasonably well and serves as a good starting point, it faces a number of challenges that are difficult to tackle with a single agent. One such challenge is the fact that modern 3D games are only partially observable. Hence, a single agent faces difficulty in properly assessing the effect(s) of its actions. For instance, at a certain point in the game, if the agent flips a light switch that controls several light bulbs distributed in different rooms/corridors (some of which could be behind closed doors), it is extremely difficult for the agent to observe the effect of the switch (i.e., which bulbs are alight and which not). Similarly, the size of the game is also another challenge where the agent would need to cover a lot of ground to fully explore the game level [ 9 ] . To address the aforementioned challenges, in this paper, we propose cMarlTest , a testing approach for 3D games exploiting MARL where two (or more) agents work collaboratively to explore the game world and achieve the desired testing objectives. cMarlTest uses a curiosity driven reward mechanism in such a way that the agents are driven towards unexplored areas of the game world, maximizing coverage of the game. This work builds on our previous work [ 9 ] using a similar setup and environment but instead, it applies a multi-agent RL approach to address some of the limitations of the aforementioned work. The use of a similar setup and environment enables a direct comparison of the results of the single agent and multi agent RL in the same context. This paper presents details about cMarlTest and the experiment we carried out to assess its performance with respect to a single agent RL based approach. Specifically, we aim to answer the following research questions: RQ1 (Effectiveness) What is the effectiveness of cMarlTest for testing 3D games with respect to the single agent RL approach? Effectiveness is measured by calculating the coverage of the 3D game achieved by the compared approaches. RQ2 (Efficiency) What is the efficiency of cMarlTest for testing 3D games with respect to the single agent RL approach? Efficiency is measured by calculating the amount of time required by the compared approaches to achieve their final coverage. The rest of the paper is organized as follows: in Section II we introduce a running example we use in the rest of the paper. In Section III we present cMarlTest , followed by the experimental setup in Section IV . The results of the experiment are presented in Section V . Section VI discusses related works, and finally Section VII concludes the paper and outlines future work. II Running Example This section introduces an example taken from Lab Recruits , an open source 3D game 1 1 1 https://github.com/iv4xr-project/labrecruits where the player navigates a maze-like environment composed of rooms and corridors connected by doors. The doors are connected to buttons that open/close one or more doors. The game level may also contain other entities, such as desks, chairs, fire hazards, extinguishers, etc. Lab Recruits allows the user to design and load a game level. Our example, illustrated in Figure 1 , is a simple level in Lab Recruits composed of three doors (door1, door2, door3), four buttons (bttn1, bttn2, bttn3, bttn4), and other non interactable entities such as desks. Lab Recruits could be played by either human players or test agents [ 10 ] by controlling the avatar (labelled as ‘agent’ in Figure 1 ). Our approach uses the test agents for interacting with the game. Fig. 1 : Level L1 in Lab Recruits . III cMarlTest : Curiosity Driven MARL for Game Testing Our proposed approach is to employ MARL where multiple agents work collaboratively to achieve better exploration of the game world. In the literature, MARL based approaches have been applied in the context of game play [ 11 , 12 , 13 , 14 ] , where the main objective is typically to train agents that can play and eventually win the game. In our case the objective is not to train agents capable of winning a game, but rather agents that can effectively explore the game world from the perspective of software testing where the ultimate goal is to achieve full coverage of the desired coverage criterion. Different notions of coverage could be considered, depending on the game and desired testing level [ 15 ] . In our experiments, we consider three coverage criteria, i.e., entity coverage , entity connection coverage , and spatial coverage (as detailed in Section IV-C ). However, the approach can easily be extended to include other coverage criteria [ 15 ] . Deployment of the MARL architecture for testing of 3D games is difficult [ 16 ] , one of the