Software Testing for Extended Reality Applications: A Systematic Mapping Study
Ruizhen Gu, José Miguel Rojas, Donghwan Shin · 2025
Read Software Testing for Extended Reality Applications: A Systematic Mapping Study by Ruizhen Gu, José Miguel Rojas, Donghwan Shin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08909.txt.
Introduction
Ruizhen Gu, José Miguel Rojas, and Donghwan Shin map the research on testing extended reality software in a 2025 study. Virtual and augmented reality apps move in six directions and immerse the user, which breaks the testing habits built for flat screens. Theirs is the first systematic map of this young field.
They selected thirty-four studies for close reading and sorted them by technique, test focus, and evaluation style. The map shows where work clusters and where it thins out. They also built a public repository of the data sets and tools the studies used. The paper ends with open challenges and concrete directions, aiming to help researchers and builders raise XR quality.
Catalog record
| CREATOR | Ruizhen Gu, José Miguel Rojas, Donghwan Shin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-08909
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Software Testing for Extended Reality Applications: A Systematic Mapping Study Ruizhen Gu, José Miguel Rojas, Donghwan Shin Abstract Extended Reality (XR) is an emerging technology spanning diverse application domains and offering immersive user experiences. However, its unique characteristics, such as six degrees of freedom interactions, present significant testing challenges distinct from traditional 2D GUI applications, demanding novel testing techniques to build high-quality XR applications. This paper presents the first systematic mapping study on software testing for XR applications. We selected 34 studies focusing on techniques and empirical approaches in XR software testing for detailed examination. The studies are classified and reviewed to address the current research landscape, test facets, and evaluation methodologies in the XR testing domain. Additionally, we provide a repository summarising the mapping study, including datasets and tools referenced in the selected studies, to support future research and practical applications. Our study highlights open challenges in XR testing and proposes actionable future research directions to address the gaps and advance the field of XR software testing. keywords Software testing, extended reality, systematic mapping 1 Introduction The global market for Extended Reality (XR) has grown significantly in recent years ---estimated at USD 77 bn in 2024--- and is expected to continue its rapid expansion to cross USD 3 tn by 2037 1 1 1 Market forecast available at: https://www.researchnester.com/reports/extended-reality-market/4863 , reflecting the increasing adoption and technological maturation of XR across multiple sectors. The industry continues to evolve with major technology companies investing heavily in this space. In late 2024, Google announced Android XR, a dedicated XR operating system built for next-generation computing experiences 2 2 2 https://blog.google/products/android/android-xr/ . The platform is developed in collaboration with Samsung for their forthcoming headset (expected in 2025), which might represent a significant shift in the XR landscape, further accelerating mainstream adoption. While entertainment–particularly video games–remains the most popular application domain for XR technologies ( Rodriguez and Wang, 2017 ) , various other fields have also benefited from its rapid development, including education ( Kavanagh et al, 2017 ) , engineering ( Tadeja et al, 2020 ) , military ( Lele, 2013 ) , and medicine ( Kim et al, 2017b ) . This broad spectrum of applications underscores the transformative potential of XR technologies beyond consumer entertainment. XR is an umbrella term encompassing Augmented, Mixed and Virtual Reality (resp. AR, MR and VR). XR applications (hereafter, XR apps) are software programs designed to run on XR-compatible devices. These apps typically feature virtually organised spaces populated with virtual objects and interactive elements, allowing users to explore scenes and engage with digital content. For instance, Pokémon Go 3 3 3 https://pokemongolive.com , a phenomenal AR mobile game, utilises GPS and cameras of mobile devices to overlay virtual content onto real-world locations. More immersive experiences are offered through head-mounted displays (HMDs), such as VR headsets (e.g., PlayStation VR2 4 4 4 https://playstation.com/ps-vr2 ) and AR headsets (e.g., Apple Vision Pro 5 5 5 https://apple.com/apple-vision-pro and Meta Quest 3 6 6 6 https://meta.com/gb/quest/quest-3 ). As XR apps become increasingly prevalent across diverse and critical domains and multiple platforms and devices, their development and testing have grown significantly more complex ( Andrade et al, 2020 ) . XR apps possess unique characteristics that distinguish them from traditional apps, such as mobile 2D apps. These include real-time responsiveness and complex interactions, enabling users to select and manipulate virtual objects or navigate through virtual environments ( Doerner et al, 2022 ) . These differences pose unique challenges for software testing. For example, in the context of generating test sequences, Android apps have finite interaction paths when navigating between different activities (i.e., individual screens of an app) ( Su et al, 2017 ) . In contrast, XR apps involve virtually infinite interaction possibilities; even a simple task, such as moving towards and interacting with a virtual object, requires accounting for countless variations in interaction sequences ( Andrade et al, 2023 ) . These complexities necessitate advanced software testing methods to ensure that XR apps operate reliably and meet user expectations. Many XR platforms now include simulation capabilities that allow developers to test and debug apps without requiring physical headset usage, such as Meta XR Simulator 7 7 7 https://developers.meta.com/horizon/documentation/unity/xrsim-intro and Unity XR Device Simulator 8 8 8 https://docs.unity3d.com/Packages/[email protected]/manual/xr-device-simulator-overview.html . For instance, Meta XR Simulator supports Meta Quest app development by enabling keyboard, mouse, or game controller simulation of XR interactions. The simulator also features a valuable record and replay function that captures input sequence and verifies consistent behaviour across executions. While record-and-replay is a common testing approach for GUI apps that simplifies the automation of complex usage scenarios ( Hu et al, 2015 ; Modarressi et al, 2024 ) , it has limitations, such as poor maintainability, where the captured test frequently breaks when the app’s UI changes, requiring substantial manual updates to remain effective ( Lam et al, 2017 ) . Unlike traditional software, which benefits from well-established surveys covering various testing practices ( Zein et al, 2016 ; Garousi et al, 2013 ) , to the best of our knowledge, there are currently no available comprehensive systematic review studies dedicated to XR software testing. Critical aspects, such as testing practices, tools, frameworks, and general testing guidelines, remain largely unexplored. To address this gap, we present a systematic mapping study on XR software testing. Systematic mapping is a methodology designed to survey the literature, provide a comprehensive overview of a topic, identify research gaps, and offer insights into future research directions. By carefully following the guidelines proposed by Petersen et al (2015) , we selected a total of 34 primary studies as the subjects for this mapping (see Appendix A for the details of the studies). We systematically classified and extracted data from these studies to investigate the current research status in XR software testing, explore key testing facets (e.g., activities and objectives), and examine the evaluation methodologies used. To facilitate future research, we compile and present the tools and datasets used in the studies. Finally, we identify the limitations and challenges in XR software testing and highlight potential avenues for advancing the field. The main contributions of this systematic mapping study are as follows: • We provide an in-depth survey of the current software testing methods for XR apps, shedding light on the state-of-the-art in this emerging domain. The data extraction template used to derive these findings is included as part of the study. • We compile a repository of existing tools and datasets used in XR software testing to support future research in the field. The repository, along with the data extraction results, is publicly available at: sites.google.com/view/xr-testing . • We identify critical challenges in XR software testing and outline potential research directions to address these challenges and advance the field. This paper is structured as follows. Section 2 presents the background of this work, key definitions and a primer on XR user interaction. Section 3 discusses the motivation behind this work and summarises relevant related studies. Section 4 details our methodology, including the process for searching and selecting relevant literature. Section 5 presents the results of our study and answers to our research questions. Section 6 discusses the findings and explores their implications for the field of