ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task Journal: PACMHCI Volume: 10 5 MHCI7668 8 DOI: 10.1145/3821671 CCS: Human-centered computing Mixed / augmented reality CCS: Human-centered computing Empirical studies in HCI CCS: Computer systems organization Robotics
Christian Fronk, Hanting Ye, Zhehan Qu, Maria Gorlatova · 2026
Read ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task Journal: PACMHCI Volume: 10 5 MHCI7668 8 DOI: 10.1145/3821671 CCS: Human-centered computing Mixed / augmented reality CCS: Human-centered computing Empirical studies in HCI CCS: Computer systems organization Robotics by Christian Fronk, Hanting Ye, Zhehan Qu, Maria Gorlatova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25202.txt.
Introduction
Augmented reality overlays can feed a warehouse worker live information about a robot partner, but the graphics can also hide the real world at the wrong moment. Christian Fronk and three colleagues present ARTOO-DARTU, an AR system for warehouse human robot collaboration with an obstruction detection and mitigation pipeline built in.
The system shows situated analytics about the robot and accepts control input. It also watches for overlays that block important real elements, moving or fading them. For evaluation the team built Pocket MonstARs, a game like stand in for inventory picking where virtual monsters mark pick targets among labelled real boxes. In a study with 34 participants, the AR analytics raised task efficiency by 46 percent, but only with the mitigation pipeline active. Visibility preservation, not the analytics alone, made the gain.
Catalog record
| CREATOR | Christian Fronk, Hanting Ye, Zhehan Qu, Maria Gorlatova |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25202
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task Journal: PACMHCI Volume: 10 5 MHCI7668 8 DOI: 10.1145/3821671 CCS: Human-centered computing Mixed / augmented reality CCS: Human-centered computing Empirical studies in HCI CCS: Computer systems organization Robotics Christian Fronk, Hanting Ye, Zhehan Qu, Maria Gorlatova Abstract . Human-robot collaboration (HRC) often requires robot intentions and internal states to be conveyed to users for task efficiency and safety. Recently, augmented reality (AR) situated analytics provide such real-time robot feedback in HRC contexts. However, AR situated analytics can obstruct important real-world elements, posing safety and usability risks, especially when content is dynamically positioned relative to movements of mobile robots in a warehouse HRC scenario. In this paper, we introduce the A ugmented R eality T echnique O f O bstruction D eterrence while A iding R obotic T eaming for U sers (ARTOO-DARTU), an AR system tailored specifically for warehouse HRC that enables real-time robot situated analytics and control while preserving visibility of the real-world through an obstruction detection and mitigation pipeline (ODM) that is uniquely suited for AR-HRC. To evaluate ARTOO-DARTU, we developed Pocket MonstARs, a controlled gamified abstraction of HRC warehouse inventory picking in which virtual monsters serve as proxies for pick targets, while labeled and object-marked boxes preserve the real-world identification demands of the picking task. In a 34-participant user study, we found that our designed AR situated analytics yielded a 46% increase in efficiency on the overall HRC task, but only when the ODM was active. Participants with the ODM active were also 61% faster on subtasks requiring visibility of the real world. Our findings demonstrate that, when paired with our developed ODM to prevent real-world obstructions, the situated analytics in ARTOO-DARTU can significantly enhance efficiency and user experience in AR-HRC warehouse scenarios. Figure 1. (a) The area allocated for our developed Pocket MonstARs task, where users collaborate with a robot to complete gamified picking targets represented as monsters. The task abstracts warehouse inventory picking by requiring users to identify target locations using physical labels, objects, and AR cues. Twelve labeled inventory boxes populate the 2.4 \times 2.4 m task area, along with four additional boxes containing YOLO-recognizable objects. (b) AR view showing robot “thought bubble” and location indicator situated analytics when real-world elements are unobstructed. (c) AR view after obstruction mitigation via the ODM: thought bubble and location indicator become fully transparent when they overlap a real-world object and text. Composite figure introducing the Pocket MonstARs study environment and ODM behavior. Panel (a) shows the physical Pocket MonstARs task area used in the user study. A participant wearing an AR headset points toward a small 2.4 by 2.4 meter area containing rows of cardboard boxes with printed labels and objects on top of some boxes. A robot is located in the task area. Panels (b) and (c) show first-person AR views of the robot near inventory boxes. In panel (b), the robot’s AR thought bubble is visible and readable because it does not block important real-world information. In panel (c), obstruction mitigation is active: the thought bubble and robot location indicator have become transparent where they would otherwise overlap real-world boxes and labels, allowing the physical objects and text to remain visible. The figure contrasts an unmitigated AR analytics display with an obstruction-mitigated display. † † cc-license: by 1. Introduction Augmented reality (AR) can superimpose virtual content onto the real world to effectively guide a user in completing tasks, such as placing directional overlays for navigation ( 48 ) . Beyond this, AR is also recognized for its potential in human-robot collaboration (HRC) scenarios ( 30 ; 40 ; 34 ; 16 ; 25 ) , where it can enhance task efficiency by providing both human and robot with better awareness of each other’s relative positions, objectives, and intentions. This mutual awareness is especially essential in some shared HRC environments like warehouses ( 17 ) , where collaborative tasks often involve numerous mobile ground robots working alongside workers, such as in inventory picking (locating and retrieving items to fulfill orders) ( 1 ; 13 ) . Though valuable, improperly placed AR content can negatively impact worker efficiency because inventory picking depends on real-world visibility: workers must be able to see inventory labels and physical objects to complete their tasks. These AR view management challenges are exacerbated in HRC scenarios, where virtual content is often displayed as if it is “attached” to a robotic collaborator, and so is subject to their movement and decisions. To assess how often AR-induced obstructions occur in an HRC warehouse environment, we developed and conducted a simulated case study set in an AR-HRC warehouse. The results indicate that obstructions caused by AR content are a significant concern in this context. Motivated by this unique challenge of AR-HRC view management in dynamic and complex warehouse environments, this paper introduces the A ugmented R eality T echnique O f O bstruction D eterrence while A iding R obotic T eaming for U sers (ARTOO-DARTU), an AR system for the Microsoft HoloLens 2 that delivers detailed, real-time situated analytics about a mobile ground robot in a warehouse setting, and allows users to control this robot. The analytics of the system are designed to be informative and easily understood at a glance by a user involved in a warehouse task, with intuitive graphics and personable messages that portray a robot’s position, intent, and internal state. To address the problem of obstruction caused by AR content, ARTOO-DARTU introduces ODM, which, to our knowledge, is the first obstruction detection and mitigation pipeline designed specifically for robot-coupled situated analytics in AR-HRC. Rather than assuming a known environment with predefined elements, the ODM operates in environments with previously unseen real-world elements such as labels and objects. The ODM detects obstructions and mitigates them while preserving the spatial coupling between analytics and the robot, ensuring real-world information remains visible without breaking the alignment of content with a robotic collaborator. To investigate users’ reactions to real-world obstructions caused by situated analytics in a warehouse-style task and evaluate ARTOO-DARTU, we develop Pocket MonstARs, a gamified task scenario that captures key aspects of warehouse inventory picking without directly replicating a full warehouse. In the task, monsters serve as gamified representations of inventory picking targets, enabling a more engaging repeated-trial experience for participants while preserving the aspects of warehouse picking most relevant to AR obstruction: identifying targets using real-world cues, coordinating with a robotic collaborator, and maintaining awareness of both real and virtual elements within a structured evaluation setting. Thus, Pocket MonstARs is intended as a controlled evaluation task for obstructions caused by AR content during warehouse AR-HRC, rather than a direct recreation of all warehouse operations. We use Pocket MonstARs to evaluate ARTOO-DARTU and its ODM through an IRB-approved 2\times 3 mixed factorial user study with 34 participants. In summary, our main contributions are the following: • We created an AR system for HRC with mobile ground robots in warehouse settings, offering enhanced situated analytics and robot control. 83% of study participants preferred our solution over a baseline with no analytics. • We introduce ODM, an obstruction detection and mitigation pipeline for robot-coupled AR situated analytics in HRC , and integrate it with