Continual Human-in-the-Loop Optimization

Yi-Chi Liao, Paul Streli, Zhipeng Li, Christoph Gebhardt, Christian Holz · 2025

Read Continual Human-in-the-Loop Optimization by Yi-Chi Liao, Paul Streli, Zhipeng Li, Christoph Gebhardt, Christian Holz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05405.txt.

Introduction

Yi-Chi Liao and co-authors introduce Continual Human-in-the-Loop Optimization in a 2025 paper. Ideal input settings differ by user, thanks to motor skill and taste. Tuning by hand is slow, and one-by-one optimization wastes past users data. Their idea carries knowledge from earlier users to each new one.

A Bayesian method with a neural surrogate holds the shared population traits. It adapts quickly to each fresh user while keeping the shared core. A replay strategy guards old knowledge from being forgotten. The team tuned a virtual reality keyboard for direct touch typing. New users reached good settings faster. The pattern suits many personal devices.

Catalog record

CREATORYi-Chi Liao, Paul Streli, Zhipeng Li, Christoph Gebhardt, Christian Holz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05405

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Continual Human-in-the-Loop Optimization Yi-Chi Liao, Paul Streli, Zhipeng Li, Christoph Gebhardt, Christian Holz Abstract . Optimal input settings vary across users due to differences in motor abilities and personal preferences, which are typically addressed by manual tuning or calibration. Although human-in-the-loop optimization has the potential to identify optimal settings during use, it is rarely applied due to its long optimization process. A more efficient approach would continually leverage data from previous users to accelerate optimization, exploiting shared traits while adapting to individual characteristics. We introduce the concept of Continual Human-in-the-Loop Optimization and a Bayesian optimization-based method that leverages a Bayesian-neural-network surrogate model to capture population-level characteristics while adapting to new users. We propose a generative replay strategy to mitigate catastrophic forgetting. We demonstrate our method by optimizing virtual reality keyboard parameters for text entry using direct touch, showing reduced adaptation times with a growing user base. Our method opens the door for next-generation personalized input systems that improve with accumulated experience. Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706598.3713603 ISBN: 979-8-4007-1394-1/25/04 CCS: Computing methodologies Machine learning CCS: Human-centered computing Interaction techniques † † cc-license: by 1. Introduction Due to the diverse motor abilities, preferences, and behaviors among users, optimal settings for input interactions in virtual reality (VR) and augmented reality (AR) vary significantly between individuals ( Shen et al., 2022 ; Yu et al., 2024 ; Schenkluhn et al., 2023 ) . Today’s interactive systems either commonly overlook this variability, or rely on manual user adjustment and explicit calibration procedures that can result in inefficient interactions or increased setup time. Human-in-the-loop optimization (HiLO) presents an alternative approach that optimizes interactions based on the user’s past performance with specific design parameters. HiLO has been shown to be effective across a wide range of applications, including target selection ( Liao et al., 2023 ; Chan et al., 2022 ) , text input ( Shen et al., 2022 ) , visual design ( Koyama and Goto, 2022 ; Koyama et al., 2020 ) , wearable devices ( Kim et al., 2017 ) , game development ( Khajah et al., 2016 ) , and animation rendering ( Brochu et al., 2010 ) . While computational optimizers help avoid exhaustive testing of every design option ( Shahriari et al., 2016 ) and aim to identify optimal solutions with minimal user trials—thereby reducing the time users spend with suboptimal settings ( Kim et al., 2017 ) — a significant barrier to HiLO’s broader adoption remains its relatively low sample efficiency. Without informed priors, HiLO often relies on initial random searches to explore the problem space, still requiring numerous trials to converge to optimal solutions. Moreover, each user must start the optimization process from scratch ( Bai et al., 2023 ) . For instance, Chan et al. ( Chan et al., 2022 ) found that optimizing the transfer function for 3D selection can take 60 to 90 minutes per user. While individual users may have distinct preferences and performance with different input settings, shared traits across the user population could be leveraged to improve the optimizer efficiency across users ( Liao et al., 2024a ) . Ideally, optimization would become more sample-efficient as data from prior users accumulates, allowing subsequent users to benefit from earlier optimization experiences while maintaining sufficient flexibility to ensure the discovery of optimal solutions tailored to individual needs. Thus, in this paper, we investigate the question: Can an optimizer continually learn from prior user experiences to improve its efficiency over time? Despite the potential, current computational methods do not support a continually improving optimization for HiLO. Moreover, the problem itself and the corresponding challenges have not been thoroughly formulated in the existing literature. One related concept is meta-Bayesian optimization ( Feurer et al., 2018a ; Volpp et al., 2019 ; Wistuba et al., 2018 ; Wistuba and Grabocka, 2021 ) , which combines meta-learning and Bayesian optimization. Liao et al. (2024a) demonstrated the use of meta-Bayesian optimization for online HiLO, where a batch of “prior users” needs to go through a full optimization process from scratch, enabling the optimizer to perform more efficiently with subsequent “end-users.” However, this approach suffers from a key limitation: its computation time during deployment increases with the number of prior users, leading to scalability issues. Beyond a certain point, it becomes impractical, as users would experience significant delays during adaptation. Additionally, this method assumes that “prior users” can dedicate sufficient time to undergo a thorough optimization, which may not be feasible in practice. Therefore, these methods are not suitable for direct transformation into a continual learning framework. Another closely related concept is continual learning ( Wang et al., 2024 ; Cossu et al., 2021 ; Awasthi and Sarawagi, 2019 ) , where a model improves its predictive capabilities by accumulating knowledge across different tasks (in the context of HiLO, a task is optimizing for a specific user). A few recent works have emerged that address continual learning within the context of optimization ( Schur et al., 2023 ; Kassraie et al., 2022 ) , but their target problems are limited to linear (1D) discrete bandit problems, where typical HiLO tackles interactions with continuous and multiple-dimensional parameter spaces. The unique challenges and the corresponding methods of continual learning for HiLO remain unaddressed and unexplored. Figure 1. Illustration of Continual Human-in-the-Loop Optimization (CHiLO) : The optimizer, parameterized by \theta , continuously evolves by accumulating experience from optimizations for different users. This enables more efficient adaptation to new users over time. To fill in the research gap, we formulate the problem and concept of Continual Human-in-the-Loop Optimization (CHiLO) — a computational optimizer capable of continuously improving its efficiency and performance by leveraging accumulated experience from previous users (illustrated in Figure 1 ). We further identify the key technical challenges that are aligned with those in continual learning ( Wang et al., 2024 ) , including scalability , catastrophic forgetting , the stability-plasticity dilemma , and model instability due to uneven data distribution of observations. These challenges guide the design principles for building CHiLO methods. Finally, we propose our novel method, the Population-Informed Continual Bayesian Optimization (ConBO) . ConBO’s core is a Bayesian Neural Network (BNN) ( MacKay, 1995 ) trained on data synthesized from individual models of previous users, each representing a unique set of user characteristics. This method facilitates more stable and robust continual optimization by progressively integrating population-level user experiences with each new user. We validate the efficacy and generalizability of ConBO using a series of standard benchmarking optimization functions, demonstrating that ConBO enhances optimization efficiency as more user data accumulates. Finally, we apply ConBO to optimize mid-air keyboard configurations for text entry using direct touch in VR. Our evaluations show a significant improvement in both user performance and convergence time as the number of users increases, compared to optimizing from scratch for each individual user. In summary, we contribute (1) The concept of Continual Human-in-the-Loop Optimization (CHiLO). We formally define the problem, identify potential challenges, and provide design principles for future methods. (2) Population-informed Continual Bayesian Optimization (ConBO), a first approach to CHiLO.

Read the complete text