Personalizing Exposure Therapy via Reinforcement Learning

Athar Mahmoudi-Nejad, Matthew Guzdial, Pierre Boulanger · 2025

Read Personalizing Exposure Therapy via Reinforcement Learning by Athar Mahmoudi-Nejad, Matthew Guzdial, Pierre Boulanger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14095.txt.

Introduction

Athar Mahmoudi-Nejad, Matthew Guzdial, and Pierre Boulanger tailor fear therapy with a learning program. Their paper, from twenty twenty-five, works in virtual reality. Patients with spider fear face fake spiders that grow more intense over time. A human therapist would tune that pace by feel, which takes skill and time.

Their system reads body signals from the patient as it runs. A reward-based learner then builds the next spider to fit that person. In a study with human subjects, the system beat a fixed rule-based method. The authors say the same idea could shape other kinds of graded therapy.

Catalog record

CREATORAthar Mahmoudi-Nejad, Matthew Guzdial, Pierre Boulanger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14095

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Personalizing Exposure Therapy via Reinforcement Learning Athar Mahmoudi-Nejad, Matthew Guzdial, Pierre Boulanger Abstract Personalized therapy, in which a therapeutic practice is adapted to an individual patient, can lead to improved health outcomes. Typically, this is accomplished by relying on a therapist’s training and intuition along with feedback from a patient. However, this requires the therapist to become an expert on any technological components, such as in the case of Virtual Reality Exposure Therapy (VRET). While there exist approaches to automatically adapt therapeutic content to a patient, they generally rely on hand-authored, pre-defined rules, which may not generalize to all individuals. In this paper, we propose an approach to automatically adapt therapeutic content to patients based on physiological measures. We implement our approach in the context of virtual reality arachnophobia exposure therapy, and rely on experience-driven procedural content generation via reinforcement learning (EDPCGRL) to generate virtual spiders to match an individual patient. Through a human subject study, we demonstrate that our system significantly outperforms a more common rules-based method, highlighting its potential for enhancing personalized therapeutic interventions. Introduction Anxiety disorders are the most prevalent mental illness, with an estimated 301 million people living with them in 2019 ( Yang et al. 2021 ) . Anxiety disorders result in notable challenges including decreased well-being, low productivity, reduced quality of life, and frequent medical visits ( Öst et al. 2023 ) . Cognitive-behavioral therapy, particularly Exposure Therapy (ET), is effective in treating anxiety by gradually exposing individuals to feared situations to reduce sensitivity ( Kaczkurkin and Foa 2022 ) . Virtual Reality Exposure Therapy (VRET) is a modern form of ET that immerses individuals in virtual environments, offering safe and practical exposure to feared stimuli, such as virtual spiders for arachnophobia ( Carl et al. 2019 ; Maples-Keller et al. 2017 ; Valmaggia et al. 2016 ) . VRET effectively induces physiological anxiety responses (e.g., blood pressure, muscle vibration, heart rate, and sweating) ( Bun et al. 2017 ; Šalkevicius et al. 2019 ) . VRET applications are mainly non-adaptive, wherein all users experience the same content. For example, all users see the same virtual spider for arachnophobia or encounter the same high-standing location for fear of heights. However, personalized treatment has shown greater benefits for individuals ( Smits, Powers, and Otto 2019 ) . Adaptive VRET attempts to select specialized content to elicit the user’s desired anxiety levels in an individual, offering a more personalized experience. Current adaptive VRET is mostly either therapist-guided ( Kampmann et al. 2016 ; Balan et al. 2020 ) or rules-based ( Bian et al. 2019 ; Repetto et al. 2013 ) , limiting scalability and personalization ( Smits, Powers, and Otto 2019 ) . Therefore, an automatic framework that effectively adapts to individuals to achieve desired anxiety levels without extensive therapist intervention or predefined rules is desirable. This work introduces an Experience-Driven Procedural Content Generation (EDPCG) framework using Reinforcement Learning (RL) for VRET targeting arachnophobia ( Yannakakis and Togelius 2011 ) . The framework aims to automatically generate and adjust virtual spiders to achieve desired anxiety levels without extensive therapist intervention. This approach represents the first PCGRL application in rehabilitation, differing from previous rules-based methods by adapting to individual user responses. Focusing on arachnophobia, the RL agent dynamically generates spiders to elicit specific anxiety levels defined by therapists. To evaluate the effectiveness of our framework, we conducted a human subject study in which we evaluated our framework in comparison to a rules-based method. We found that our framework was effective at achieving desired anxiety levels, indicating that it may be feasible in clinical settings. However, we only tested the system’s ability to reach desired anxiety levels, not its effect on therapeutic outcomes. Finally, we conducted an experiment to demonstrate the necessity of personalizing spiders to each user. We present the following contributions: • Developing an end-to-end personalized EDPCGRL framework, which incorporates real-time human interaction to automatically adapt a VR environment to a user. • Conducting a human subject study to evaluate the effectiveness of our framework compared to a rules-based method by collecting both subjective and objective measures during the experiments. • Demonstrating that different participants exhibit anxiety responses to different attributes of spiders, emphasizing the necessity for personalized treatment for each participant. Related Work Physiological Measures: To assess anxiety levels in real-time, studies have utilized physiological measures such as heart rate variability (HRV) ( Cheng et al. 2022 ; Harrewijn et al. 2018 ; Alvares et al. 2013 ; Chalmers et al. 2014 ; Gaebler et al. 2013 ) and electrodermal activity (EDA) ( Christian et al. 2023 ; Kimani and Bickmore 2019 ; Kritikos et al. 2019 ; Lonsdorf et al. 2017 ; Šalkevicius et al. 2019 ; Sevil et al. 2017 ; Nikolić et al. 2018 ; Owens and Beidel 2015 ) . EDA refers to the autonomic changes in the electrical properties of the skin. It consists of Skin Conductance Response (SCR) and Skin Conductance Level (SCL), both of which correlate with anxiety and emotional responses ( Boucsein 2012 ) . Studies have shown strong correlations between SCL and self-reported anxiety in VRET ( Wilhelm et al. 2005 ; Mühlberger et al. 2008 ) , and SCR amplitudes have been used to approximate stress and arousal ( Suso-Ribera et al. 2019 ; Pick, Mellers, and Goldstein 2018 ; Yee and Shiota 2015 ; Albayrak et al. 2023 ; Sevinç 2018 ) . In this paper, we employed SCL to assess participants’ anxiety. Procedural Content Generation (PCG) for Rehabilitation: Procedural Content Generation (PCG) refers to the algorithmic generation of content. PCG algorithms have been employed in adaptive rehabilitation. Examples include a ski-slalom game for balance improvement ( Dimovska et al. 2010 ) , a self-adaptive shooter game for amblyopia ( Correa et al. 2014 ) , and adaptive tasks for upper limb rehabilitation ( Hocine et al. 2015 ) . Other applications involve VR labyrinths for emotional self-awareness ( Badia et al. 2018 ) and gait rehabilitation customization ( Lyu and Bidarra 2023 ) . These studies typically use constructive or rules-based PCG methods ( Guzdial, Snodgrass, and Summerville 2022a ) , relying on predefined rules under the assumption of homogeneity among subjects, i.e., they can predict all subjects’ behaviors or interactions with the system. In contrast, we assume that subjects are unknown and exhibit different behaviors; therefore, the system needs to generate and adapt game content dynamically based on each subject’s needs. Our approach is based on PCG via reinforcement learning (PCGRL) which leverages reinforcement learning (RL) for dynamic, personalized content generation ( Guzdial, Snodgrass, and Summerville 2022b ; Khalifa et al. 2020 ) . Virtual Reality Exposure Therapy (VRET) for Arachnophobia: VRET has been effective in reducing fear of spiders and avoidance behaviors ( Côté and Bouchard 2009 ; Michaliszyn et al. 2010 ; Minns et al. 2019 ; Miloff et al. 2019 ; Dyrdal and Sanner 2022 ) . Early studies showed significant improvements in VRET groups compared to controls ( Michaliszyn et al. 2010 ; Miloff et al. 2019 ) . While most studies pre-design scenarios with attributes like spider size and behavior, Kritikos, Alevizopoulos, and Koutsouris 2021 introduced an adaptive VR environment using rules-based methods to adjust spider

Read the complete text