Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants

Eirini Schoinas, Adyah Rastogi, Anissa Carter, Jacob Granley, Michael Beyeler · 2025

Read Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants by Eirini Schoinas, Adyah Rastogi, Anissa Carter, Jacob Granley, Michael Beyeler (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00177.txt.

Introduction

Eirini Schoinas, Adyah Rastogi, Anissa Carter, Jacob Granley, and Michael Beyeler wrote a study from 2025 on eye implants that aim to restore some sight. Their method asks the user to give feedback as the system runs. In each round, the user picks which of two test images looks better. The system then adjusts how it sends signals to the eye.

The team tested the method with sighted people who viewed fake implant vision on a screen. They ran three kinds of tests, with changed settings and with values drawn from outside the usual range. In all three, people liked the tuned images more than those from two simpler setups. The team also found that real people chose in ways that tests run only on a computer did not predict.

Catalog record

CREATOREirini Schoinas, Adyah Rastogi, Anissa Carter, Jacob Granley, Michael Beyeler
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-00177

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants Thanks: 1 ES is with the College of Creative Studies at the University of California, Santa Barbara, CA 93106, USA. Thanks: 2 AR, JG, and MB are with the Department of Computer Science, University of California, Santa Barbara, CA 93106, USA. Thanks: 3 AC is with the Department of Communication, University of California, Santa Barbara, CA 93106, USA. Thanks: 4 MB is with the Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA 93106, USA. Thanks: The authors confirm contribution to the paper as follows. Study conception and design: ES, AC, JG, MB; computational model development: JG, MB; experiment coding and development: ES, AR, AC, JG; data collection: AC, ES, AR; all authors analyzed the data, wrote and approved the final version of the manuscript. Thanks: Supported by the National Library of Medicine of the National Institutes of Health (NIH) under Award Number DP2LM014268. The authors would like to thank Tori LeVier for her support in recruiting and managing participants. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Eirini Schoinas, Adyah Rastogi, Anissa Carter, Jacob Granley, Michael Beyeler Abstract Human-in-the-loop optimization (HILO) is a promising approach for personalizing visual prostheses by iteratively refining stimulus parameters based on user feedback. Previous work demonstrated HILO’s efficacy in simulation, but its performance with human participants remains untested. Here we evaluate HILO using sighted participants viewing simulated prosthetic vision to assess its ability to optimize stimulation strategies under realistic conditions. Participants selected between phosphenes generated by competing encoders to iteratively refine a deep stimulus encoder (DSE). We tested HILO in three conditions: standard optimization, threshold misspecifications, and out-of-distribution parameter sampling. Participants consistently preferred HILO-generated stimuli over both a naïve encoder and the DSE alone, with log odds favoring HILO across all conditions. We also observed key differences between human and simulated decision-making, highlighting the importance of validating optimization strategies with human participants. These findings support HILO as a viable approach for adapting visual prostheses to individuals. Clinical Relevance— Validating HILO with sighted participants viewing simulated prosthetic vision is an important step toward personalized calibration of future visual prostheses. I Introduction Visual prostheses are being developed to restore vision for individuals with incurable blindness by electrically stimulating functional cells along the visual pathway [ 1 , 2 ] . Retinal [ 3 , 4 , 5 ] and cortical [ 6 , 7 , 8 ] prostheses have enabled tasks such as object localization and supported mobility. However, the quality of vision provided by these implants remains limited. Phosphenes—the artificial visual percepts evoked by stimulation—vary widely across individuals [ 9 , 10 , 6 , 11 ] and often do not combine linearly [ 12 , 13 , 14 ] , suggesting the presence of a nonlinear transfer function between electrical stimuli and perceptual outcomes. To address these nonlinearities, computational models have been developed to predict perceptual responses to electrical stimulation. These forward models use user-specific parameters to capture how stimulus properties affect the brightness and shape of phosphenes, as well as their interaction across electrodes [ 15 , 9 , 16 , 17 , 18 ] . While forward models enable a mechanistic understanding of prosthetic vision, their utility in real-world applications hinges on the ability to invert them: that is, to determine the optimal stimulus parameters required to elicit a desired percept. Deep stimulus encoders (DSEs) have been proposed for this purpose (Fig. 1 A), leveraging deep learning to approximate the inverse mapping from percepts to stimuli [ 19 , 20 , 21 , 17 , 22 ] . However, DSEs require precise knowledge of user-specific parameters, which is often infeasible due to limited data availability or the inherent variability across users [ 23 ] . Moreover, interviews with blind prosthesis users have highlighted gaps between laboratory models and real-world experiences, underscoring the importance of designing technologies that adapt to individual needs and usability constraints [ 24 ] . To optimize stimulus strategies from limited human feedback, the human-in-the-loop optimization (HILO) framework was introduced to the field by Fauvel and Chalk [ 25 ] (Fig. 1 B). Their work applied HILO to a linear phosphene model, showing that parameter estimation was feasible using pairwise comparisons of simulated percepts ( duels ). Granley et al. [ 23 ] extended this approach by incorporating DSEs and testing it under more realistic conditions, including model misspecifications and noisy user feedback. Their results suggested that HILO could effectively personalize stimulus encoding, even when simulated users selected randomly in two out of three trials. However, these simulations relied on predefined probabilistic rules, where choices were more likely when a percept had a lower computed error metric. Human decision making, by contrast, is inherently more variable and may not align with model assumptions, leaving open the question of whether HILO would perform similarly when real users provide feedback. Fig. 1: A) Deep stimulus encoder (DSE). A forward model ( f ) predicts the perceptual response to visual stimuli based on user-specific parameters ( \phi ), while an encoder ( f^{-1} ) learns to minimize the perceptual error between predicted and target percepts. B) Human-in-the-loop optimization (HILO). The parameters from the DSE are refined using user preferences, collected through 60 binary comparison trials per condition. New parameter pairs are adaptively selected to efficiently converge on the most preferred percept. The target percept changes each iteration. Adapted under CC-BY from https://doi.org/10.48550/arXiv.2306.13104 . C) Example duels used to infer user preferences are presented to sighted participants on a computer monitor. Participants selected the preferred stimulus based on both shape and brightness. In this study, we take an intermediate step toward real-world validation by incorporating sighted participants viewing simulated prosthetic vision (SPV) stimuli on a monitor (Fig. 1 C). This approach enables systematic testing of whether HILO can adapt to individual perceptual variability, maintain robustness to model misspecifications, and remain practical under real-world decision-making constraints. Importantly, we evaluated HILO not only under standard conditions but also in out-of-distribution scenarios, testing its generalizability beyond the training range. We also systematically varied key phosphene properties, such as size and elongation, highlighting how perceptual distortions affect optimization performance and motivating future efforts to improve phosphene focality. By replacing simulated user decisions with human feedback, this study provides critical evidence for the utility of HILO in personalizing neuroprosthetic devices. II Methods II-A Participants Seventeen sighted undergraduate students from the University of California, Santa Barbara, participated in the study (10 female, 7 male; ages 18–21; M=19.6 , SD=1.12 ). Participants were recruited through university-wide announcements and provided informed consent prior to participation. All participants reported normal or corrected-to-normal vision and no history of neurological or visual impairments. Participants were briefed on the study’s purpose and tasks before beginning the experiment. The study was approved by the Institutional Review Board at the University of California, Santa Barbara. II-B Task Participants viewed simulated prosthetic vision on a monitor and completed a series of trials aimed at optimizing the mapping between simulated electrical stimuli and perceived visual representations. Each trial, referred to as a “duel,” presented two full percepts generated for a target image. Participants selected

Read the complete text