XR Reality Check: What Commercial Devices Deliver for Spatial Tracking

Tianyi Hu Department of Electrical and Computer Engineering Duke University, Tianyuan Du Department of Electrical and Computer Engineering Duke University, Zhehan Qu Department of Computer Science Duke University, Maria Gorlatova Department of Electrical and Computer Engineering Duke University · 2025

Read XR Reality Check: What Commercial Devices Deliver for Spatial Tracking by Tianyi Hu Department of Electrical and Computer Engineering Duke University, Tianyuan Du Department of Electrical and Computer Engineering Duke University, Zhehan Qu Department of Computer Science Duke University, Maria Gorlatova Department of Electrical and Computer Engineering Duke University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08642.txt.

Introduction

Four Duke University researchers, Tianyi Hu, Tianyuan Du, Zhehan Qu, and Maria Gorlatova, benchmark how well five retail headsets track motion in space. Poor tracking makes virtual objects shake or drift, which breaks the feel of extended reality and can make users uneasy.

Their new test rig runs all the devices at once under the same room conditions and the same movements, across sixteen test scenes. Error rose sharply in plain rooms with few visual cues, and the gap between the best and worst devices was wide. The team also tested whether one top headset can stand in for a costly motion capture rig when grading the others, with mixed results.

Catalog record

CREATORTianyi Hu Department of Electrical and Computer Engineering Duke University, Tianyuan Du Department of Electrical and Computer Engineering Duke University, Zhehan Qu Department of Computer Science Duke University, Maria Gorlatova Department of Electrical and Computer Engineering Duke University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08642

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

XR Reality Check: What Commercial Devices Deliver for Spatial Tracking Tianyi Hu Department of Electrical and Computer Engineering Duke University, Tianyuan Du Department of Electrical and Computer Engineering Duke University, Zhehan Qu Department of Computer Science Duke University, Maria Gorlatova Department of Electrical and Computer Engineering Duke University Abstract \onlineid 1619 \vgtccategory Research \vgtcinsertpkg \crefformat footnote#2 1 1 footnotemark: 1 #3 \teaser (a) XR devices and data collection apparatus, and (b) data collection environment. keywords Human-centered computing—Ubiquitous and mobile computing design and evaluation methods; Mixed/augmented reality; Computing methodologies—Computer vision problems; Tracking; Empirical studies 1 Introduction The rapid advancement of Extended Reality (XR) technologies has generated significant interest across research, development, and consumer domains. Contemporary XR systems predominantly leverage inside-out tracking methodologies that employ onboard camera arrays and inertial measurement units (IMU) to simultaneously estimate user motion and reconstruct environmental geometry [ 1 , 2 , 3 , 4 , 5 ] . This approach allows on-device tracking of user pose, which offers improved portability and usability compared to traditional outside-in tracking systems that require fixed external infrastructure [ 6 , 7 , 8 ] . However, inherent limitations persist in visual-inertial odometry (VIO) and visual-inertial SLAM (VI-SLAM) implementations, particularly under challenging operational conditions including high rotational velocities, low-light environments, and textureless spaces. These factors can induce tracking errors that manifest as drift and misalignment of virtual content, resulting in positional instability that ultimately degrades the immersive experience [ 7 , 9 , 10 ] . While recent advancements have improved tracking robustness [ 6 , 11 , 12 ] , SOTA XR devices continue to demonstrate performance limitations, as evidenced by empirical user reports and developer documentation [ 13 , 14 , 15 ] . A rigorous quantitative evaluation of XR tracking systems is critical for developers optimizing immersive applications and users selecting devices. However, three fundamental challenges impede systematic performance analysis across commercial XR platforms. Firstly, major XR manufacturers do not reveal critical tracking performance metrics, sensor (tracking camera and IMU) interfaces, or algorithm architectures. This lack of transparency prevents independent validation of tracking reliability and limits decision-making by developers and end users alike. Secondly, existing studies [ 7 , 9 , 16 , 17 , 8 , 18 ] have either focused on single-device analysis or evaluated multiple devices individually without standardized methodologies for cross-device comparisons under controlled environmental and kinematic conditions. Thirdly, existing evaluations focus on trajectory-level performance but omit correlation analyses at timestamp level that link pose errors to camera and IMU sensor data. This omission limits the ability to analyze how environmental factors and user kinematics influence estimation accuracy. \textcolor blackFinally, most prior work does not share testbed designs or experimental datasets, limiting reproducibility, validation, and subsequent research, such as efforts to model, predict, or adapt to pose errors based on trajectory and sensor data. In this work, we propose a novel XR spatial tracking testbed that addresses all the aforementioned challenges. The testbed enables the following functionalities: (1) synchronized multi-device tracking performance evaluation under various motion patterns and configurable environmental conditions; (2) quantitative analysis among environmental characteristics, user motion dynamics, multi-modal sensor data, and pose errors; and (3) open-source calibration procedures, data collection frameworks, and analytical pipelines. We validate our testbed through comprehensive experiments with five commercial XR devices, including Apple Vision Pro [ 19 ] , Meta Quest 3 [ 20 ] , HoloLens 2 [ 21 ] , Magic Leap 2 [ 22 ] , and XReal Air 2 Ultra [ 23 ] , across diverse operational scenarios. Furthermore, our analysis reveal that the Apple Vision Pro’s tracking accuracy (with an average relative pose error (RPE) of 0.52 cm, which is the best among all) enables its use as a ground truth reference for evaluating other devices’ RPE without the use of a motion capture system. The full implementation, including hardware designs, software code, benchmarking tools, \textcolor blackand experimental dataset, is publicly accessible via an open-source repository 1 1 1 \href https://github.com/Duke-I3T-Lab/XR_Tracking_Evaluationgithub.com/Duke-I3T-Lab/XR_Tracking_Evaluation to promote reproducibility and standardized evaluation in the XR research community. Our main contributions are as follows: • Designed a novel testbed enabling simultaneous evaluation of multiple XR devices under the same environmental and kinematic conditions. This testbed achieves accurate evaluation via time synchronization precision and extrinsic calibration. • Conducted the first comparative analysis of five SOTA commercial XR devices (four headsets and one pair of glasses), quantifying spatial tracking performance across 16 diverse scenarios. Our analysis reveals that average tracking errors vary by up to 2.8× between devices under identical challenging conditions, with errors ranging from sub-centimeter to over 10 cm depending on devices, motion types, and environment conditions. • Performed correlation analysis on collected sensor data to quantify the impact of environmental visual features, SLAM internal status, and IMU measurements on pose error, demonstrating that different XR devices exhibit distinct sensitivities to these factors. • Presented a case study evaluating the feasibility of using Apple Vision Pro as a substitute for traditional motion capture systems in tracking evaluation. Our results show that relative pose error measurements from Apple Vision Pro strongly correlate with those from the motion capture system ( R^{2}=0.830 ). However, while the correlation for absolute pose error is substantially lower ( R^{2}=0.387 ), this suggests that Apple Vision Pro provides a reliable reference for local tracking accuracy, making it a practical tool for many XR evaluation scenarios despite its limitations in assessing global pose precision. The remainder of this paper is organized as follows: Section 2 reviews related work. Section 3 details the design of our testbed system. Section 4 describes the experimental setup, and Section 5 presents the experimental results. Section 6 provides a case study demonstrating the potential of using the Apple Vision Pro as a ground truth reference in place of a mocap system. Section 7 discusses the implications of our findings, while Section 8 addresses the limitations of this work and suggests directions for future research. Finally, Section 9 concludes the paper. 2 Related Work Factors influencing XR tracking performance : Modern visual-inertial odometry (VIO) and visual-inertial simultaneous localization and mapping (VI-SLAM) systems face notable challenges in tracking accuracy, which are predominantly shaped by environmental and kinematics conditions. Texture-deficient surfaces impair environmental perception, limiting the ability of VI-SLAM to extract meaningful features for localization [ 24 , 6 , 25 , 26 , 27 ] . In contrast, environments with high structural complexity and distinct visual features significantly improve pose estimation accuracy [ 28 , 29 , 30 , 31 , 32 , 33 ] . Kinematic factors, such as abrupt rotations or translations of the headset, exacerbate performance degradation by inducing motion blur in visual data streams, which leads to feature point tracking loss [ 34 , 35 , 36 , 37 , 12 ] . To address these limitations, our evaluation framework systematically incorporates operational constraints to quantitatively assess XR device tracking performance under diverse controlled conditions. Existing XR tracking evaluation methods : Reliable spatial tracking is foundational for most modern XR systems, and extensive research exists on evaluation methodologies of spatial tracking. Among them, three approaches are widely adopted. Robotic actuation platforms : Robotic arms execute predefined kinematic profiles with high repeatability [ 9 , 7 , 16

Read the complete text