VisGuardian: A Lightweight Group-based Privacy Control Technique For Front Camera Data From AR Glasses in Home Environments Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Fa

Shuning Zhang, Qucheng Zang, Yongquan ‘Owen’ Hu, Jiachen Du, Xueyang Wang, Yan Kong, Xinyi Fu, Suranga Nanayakkara, Xin Yi, Hewu Li · 2026

Read VisGuardian: A Lightweight Group-based Privacy Control Technique For Front Camera Data From AR Glasses in Home Environments Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Fa by Shuning Zhang, Qucheng Zang, Yongquan ‘Owen’ Hu, Jiachen Du, Xueyang Wang, Yan Kong, Xinyi Fu, Suranga Nanayakkara, Xin Yi, Hewu Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19502.txt.

Introduction

Shuning Zhang and nine colleagues tackle a privacy problem created by smart glasses. The glasses can sense their surroundings at all times, and their front cameras record what the wearer sees. In a home, that camera may catch private objects and family members who never agreed to be recorded. Simple on-off permission switches cannot handle such shifting, context-based risks.

Their answer is VisGuardian, a light control method built around groups. The idea is to set privacy rules for sets of people and places, not just one user at a time. A home is full of shared spaces and daily routines, so rules need to fit that setting. The work points toward camera controls that respect the people nearby, not only the person wearing the glasses.

Catalog record

CREATORShuning Zhang, Qucheng Zang, Yongquan ‘Owen’ Hu, Jiachen Du, Xueyang Wang, Yan Kong, Xinyi Fu, Suranga Nanayakkara, Xin Yi, Hewu Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-19502

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VisGuardian: A Lightweight Group-based Privacy Control Technique For Front Camera Data From AR Glasses in Home Environments Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3790288 ISBN: 979-8-4007-2278-3/2026/04 CCS: Security and privacy Usability in security and privacy CCS: Security and privacy Privacy protections CCS: Computing methodologies Computer vision Shuning Zhang, Qucheng Zang, Yongquan ‘Owen’ Hu, Jiachen Du, Xueyang Wang, Yan Kong, Xinyi Fu, Suranga Nanayakkara, Xin Yi, Hewu Li Abstract . Always-on sensing of AI applications on AR glasses makes traditional permission techniques ill-suited for context-dependent visual data, especially within home environments. The home presents a highly challenging privacy context due to the high density of sensitive objects, and the frequent presence of non-consenting family members, and the intimate nature of daily routines, making it a critical focus area for scalable privacy control mechanisms. Existing fine-grained controls, while offering nuanced choices, are inefficient for managing multiple private objects. We propose VisGuardian, a fine-grained content-based visual permission technique for AR glasses. VisGuardian features a group-based control mechanism that enables users to efficiently manage permissions for multiple private objects. VisGuardian detects objects using YOLO and adopts a pre-classified schema to group them. By selecting a single object, users can efficiently obscure groups of related objects based on criteria including privacy sensitivity, object category, or spatial proximity. A technical evaluation shows VisGuardian achieves mAP50 of 0.6704 with only 14.0 ms latency and a 1.7% increase in battery consumption per hour. Furthermore, a user study (N=24) comparing VisGuardian to slider-based and object-based baselines found it to be significantly faster for setting permissions and was preferred by users for its efficiency, effectiveness, and ease of use. Figure 1. VisGuardian is a fine-grained permission technique for AR glasses, enabling group-based privacy management with minimal task disruption. (1) The reminding window pops up when sensitive objects of new classes are detected or when an app requests visual permissions. The streaming continues, and sensitive objects are highlighted with bounding boxes, (2) users can directly click the bounding box of different identified sensitive objects to hide or reveal them. When selected, (3) the interface pops up with the choices to obscure or reveal similar objects (including three choices based on: privacy sensitivity, object type, or spatial proximity). (4) After hiding the objects, VisGuardian remembers the users’ choice until new objects emerge or the user wants to manually change the settings. † † cc-license: by 1. Introduction The landscape of head-worn wearables is rapidly evolving towards a converging paradigm of Artificial Intelligence (AI) applications on Augmented Reality (AR). While early market distinguished between ‘‘smart glasses’’ (e.g., Ray-Ban Meta 1 1 1 https://www.meta.com/ai-glasses/ray-ban-meta/ ), which focused on AI assistants and capture, and ‘‘AR headsets’’ (e.g., HoloLens 2 2 2 https://learn.microsoft.com/en-us/hololens/hololens-commercial-features ), which focused on immersive display, recent industry innovations, exemplified by prototypes like Meta’s Orion 3 3 3 https://about.fb.com/news/2024/09/introducing-orion-our-first-true-augmented-reality-glasses/ , point towards a unified future: wearable glasses that integrate always-on visual sensors with spatial AR displays ( Rao et al. 2020 ; Suzuki et al. 2025 ) . These devices 4 4 4 We used AR glasses to denote these devices in the latter sections of the paper. leveraged advanced AI to proactively sense and interpret the user’s environment, offering an always-on interface for visual understanding ( Cai et al. 2025 ; Zhao et al. 2024 ) . However, the persistent sensing capability introduces profound privacy risks, which are particularly acute within home environments. Unlike public or workplace settings, the home is a highly sensitive environment, a space for intimate family moments ( Zeng and Roesner 2019 ) , private routines ( Chiang et al. 2020 ) , and sensitive personal data ( Lenhart et al. 2023 ) . Furthermore, the frequent co-presence of bystanders ( Marky et al. 2020 ) (e.g., family members, guests) complicates the ethics and logistics of consent for continuous data collection. The home, therefore, represents a critical high-risk context that demands robust visual privacy controls. The privacy risks extend beyond inadvertent data leakage. A primary threat that motivates our work emerges from semi-trusted applications that provide services to the user but may extract and leak their private information. This information could also be exploited for user profiling or targeted advertising, thereby compromising user privacy. Prior research has demonstrated that sensitive attributes, such as a user’s gender ( Zhang et al. 2025b ; Zhang et al. 2025a ) , behavioral characteristics ( Berkovsky et al. 2019 ; Hoppe et al. 2018 ) , and emotional state ( Hickson et al. 2019 ) , can be inferred in real time from the data captured by these visual sensors. For example, continuous sensing via the front camera could track a user’s dietary habits from what they eat for breakfast, infer their emotional state from their facial expressions during a video call with a family member, or reveal their lifestyle habits from the books and documents on their desk. While permission systems are the standard data control mechanism for AI applications on AR glasses ( Google Android 2022 ; Oculus 2022 ) , their design is inadequate for the continuous data collection paradigm introduced by these devices ( Abraham et al. 2024b ) . These sensors create a heightened privacy-utility tension, where the binary “all-or-nothing” nature of permissions forces an untenable choice ( Felt et al. 2011 ; Kelley et al. 2012 ) , a decision further complicated by users’ poor comprehension of the implications ( Lin et al. 2012 ; Kelley et al. 2013 ; Johnstone et al. 2022 ) . Prior work has attempted to provide granular control ( Abraham et al. 2024b ; Venugopalan et al. 2024 ) . One stream of research focuses on multi-level settings ( Android 2022 ; Abraham et al. 2024b ; Nair et al. 2023 ) to move beyond simple binary choices. While these methods, such as privacy sliders, acknowledge the privacy-utility trade-off, they often lack the expressiveness to manage specific contextual objects and can impose a high cognitive load on users. Another stream of research involves automatic control paradigms ( Roesner et al. 2014b ; Steil et al. 2019 ) that configure protection policies or shutter cameras without user intervention. However, these paradigms did not consider users’ preferences regarding privacy-utility trade-offs, and may compromise the task utility through shuttering the whole camera ( Steil et al. 2019 ) . The third stream enables direct data access control ( Aditya et al. 2016 ; Raval et al. 2016 ; Jana et al. 2013 ; Lee et al. 2024 ; Venugopalan et al. 2024 ) . These approaches offer great precision, evolving from manual policy configuration ( Lee et al. 2024 ) to automatic detection based on user selections ( Venugopalan et al. 2024 ) . However, they often require users to manage permissions on an object-by-object basis. This approach, especially when systems are designed around single salient objects such as credit cards ( Venugopalan et al. 2024 ) , does not scale well to complex scenarios, leading to high interaction costs. Therefore, a critical research gap exists for a technique that provides

Read the complete text