A Vision for AI-Driven Adaptation of Dynamic AR Content to Users and Environments Price: 15.00 DOI: 00.0000/0000000.0000000 tbd ISBN: 000-0-0000-0000-0/00/00 Conference: Workshop Proceedings of the 2025 Everyday AR through AI-in-the-Loop Workshop; April 2025; Yokohama, Japan tbd CCS: Human-centered

Julian Rasch, Florian Müller, Francesco Chiossi · 2025

Read A Vision for AI-Driven Adaptation of Dynamic AR Content to Users and Environments Price: 15.00 DOI: 00.0000/0000000.0000000 tbd ISBN: 000-0-0000-0000-0/00/00 Conference: Workshop Proceedings of the 2025 Everyday AR through AI-in-the-Loop Workshop; April 2025; Yokohama, Japan tbd CCS: Human-centered by Julian Rasch, Florian Müller, Francesco Chiossi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16562.txt.

Introduction

Julian Rasch, Florian Muller, and Francesco Chiossi sketch a future for augmented reality in a 2025 vision paper. Such systems lay digital content over the real world around a user. Most current systems place that content in fixed ways, even as the user moves and the room changes.

The authors picture a smart system that shifts content as needs change. Part of the display could sit in the room, while the rest stays fixed in view. That split could cut the load on the user's mind. The paper maps uses from city walks to work tasks and class rooms, and sets out what to build next.

Catalog record

CREATORJulian Rasch, Florian Müller, Francesco Chiossi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE10 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16562

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Vision for AI-Driven Adaptation of Dynamic AR Content to Users and Environments Price: 15.00 DOI: 00.0000/0000000.0000000 tbd ISBN: 000-0-0000-0000-0/00/00 Conference: Workshop Proceedings of the 2025 Everyday AR through AI-in-the-Loop Workshop; April 2025; Yokohama, Japan tbd CCS: Human-centered computing Ubiquitous and mobile devices CCS: Human-centered computing Interaction techniques CCS: Human-centered computing Interactive systems and tools Julian Rasch, Florian Müller, Francesco Chiossi Abstract . Augmented Reality (AR) is transforming the way we interact with virtual information in the physical world. By overlaying digital content in real-world environments, AR enables new forms of immersive and engaging experiences. However, existing AR systems often struggle to effectively manage the many interactive possibilities that AR presents. This vision paper speculates on AI-driven approaches for adaptive AR content placement, dynamically adjusting to user movement and environmental changes. By leveraging machine learning methods, such a system would intelligently manage content distribution between AR projections integrated into the external environment and fixed static content, enabling seamless UI layout and potentially reducing users’ cognitive load. By exploring the possibilities of AI-driven dynamic AR content placement, we aim to envision new opportunities for innovation and improvement in various industries, from urban navigation and workplace productivity to immersive learning and beyond. This paper outlines a vision for the development of more intuitive, engaging, and effective AI-powered AR experiences. 1. Motivation The integration of AI capabilities into AR technologies presents new opportunities to solve a key challenge in spatial computing, namely adapting AR content to dynamic users and environments as they move through space. Traditional AR systems often use static content placement or fixed positioning within users’ field-of-view, limiting their potential in dynamic, real-world scenarios ( Lages and Bowman 2019 ) . Recent research has explored various approaches to this challenge. Tanaka et al. ( Kohei Tanaka et al. 2008 ) proposed a layout method for AR displays in changing environments. More recent systems like SituationAdapt ( Li et al. 2024 ) use large language models (LLMs) to optimize mixed reality interfaces based on context. Further, UI positioning in AR has seen advancements, with studies exploring user preferences for video call windows in different environments ( Chang et al. 2024 ) , user performance in dual-task walking scenarios with varying AR content anchoring around the user’s body ( Rasch et al. 2025 ) , and techniques for UI transitions between static, dynamic, and "self entities" ( Pei et al. 2024 ) . Research has also identified areas where AI enhances AR, including pattern detection and improved experiences through neural networks and natural language processing ( Reiners et al. 2021 ; Hirzle et al. 2023 ) . While these examples show progress on individual aspects of AR interfaces, it is often limited to these specific scenarios. For AR to become our primary interaction medium, its interfaces must be suitable for a wide range of everyday situations, including indoors and outdoors, while walking or standing, when alone, or in crowded spaces. To bridge the gap between these localized solutions and create more generalizable, adaptable interfaces that can seamlessly integrate with dynamic environments and users, our work proposes an AI-driven system for dynamic AR content placement that adapts to these changes This approach manages content distribution between environmental projections and head-mounted displays (HMDs), offering more nuanced and context-aware placement. 2. Problem Space When studying and designing AR interactions for users in everyday scenarios, we can separate use cases based on static or dynamic characteristics of users and environment into a matrix with four distinct scenarios as shown in Figure 1 . In each given scenario, the AR content, can be static, dynamic, or also combine AR elements of both characteristics. Adaptation of UI elements might be necessary because of visual changes in the user’s field of view. These changes occur in particular as a result of spatial changes — either by the users themselves (through head rotation, locomotion, etc.) or by elements of the environment (cars, people, etc.). Figure 1. The design space for everyday AR interactions categorizes them into use cases based on user mobility and environmental stability each with dynamic and static AR content. Static User, Static Environment Both user and environment are fixed, e.g., in scenarios like inspecting static AR models. Static content supports applications where consistent information delivery is important, such as text labels on artifacts or fixed guidance displays in training. Dynamic content, if used, could introduce adaptive layers for exploration but may not always be necessary when precision and stability are prioritized. Static User, Dynamic Environment A stationary user interacts in a changing environment, such as viewing AR content in public transport. Static content is beneficial for persistent notifications or content providing general information, such as route maps or advertisements. However, dynamic content could enhance situational relevance by responding to environmental changes like crowd density or lighting conditions. Dynamic User, Static Environment A moving user in a stable environment interacts with fixed AR content, such as navigation waypoints in museums or airports. Static content, like directional arrows or labels, remains effective for guiding users without overwhelming them. The challenge lies in maintaining visibility across varying perspectives. Dynamic content can offer adaptive interfaces, such as repositioning navigation cues or dynamically adjusting information displays based on user location or movement patterns. Dynamic User, Dynamic Environment Both user and environment are in motion, such as AR shopping in busy spaces. Static content, such as price tags or simple notifications, remains useful for persistent information. However, its effectiveness may decline as the environment becomes more chaotic. Applications like interactive shopping assistants or context-based offers highlight how dynamic content can personalize the user experience without relying solely on environmental adaptation. Each scenario in Figure 1 comes with unique challenges. However, out of these combinations, a dynamic user in a dynamic environment with dynamic AR content is the least researched and most challenging field. The central question here is: How can we design an AR system that can seamlessly integrate with the dynamic environment and provide a coherent and engaging user experience? Specifically, what content should be displayed as part of the augmented environment, where and how should it be overlaid, and what should remain static within the user’s field of view? 3. Vision for an AI-powered AR System for Dynamic AR Content for Users in Dynamic Environments For this most challenging case of dynamic users in dynamic environments with dynamic AR content, multiple solutions approaches exist that could be integrated into an AI-powered AR system, which we envision as follows. The system can create contextual content for users utilizing computer vision for scene understanding and LLMs for the generation of contextual information and 3D content ( De La Torre et al. 2024 ) . Traditional AR content placement often focuses on either the user’s perspective or static environmental elements. In contrast, the AI-powered AR system has environmental awareness and considers both the user’s movement and changes in the surrounding environment, creating a more holistic and responsive system. Unlike traditional methods that often rely on static content placement or fixed positioning within the user’s field of view, the envisioned system could dynamically adapt content placement based on real-time user movement and environmental changes. The system employs a multi-modal approach to manage content distribution between environmental

Read the complete text