Emotion Detection in Older Adults Using Physiological Signals from Wearable Sensors Conference: Great Lakes Symposium on VLSI 2025; June 30-July 2, 2025; New Orleans, LA, USA Great Lakes Symposium on VLSI 2025 (GLSVLSI ’25), June 30-July 2, 2025, New Orleans, LA, USA DOI: 10.1145/3716368.3735280 ISBN: 979-8-4007-1496-2/2025/06 CCS: Computing methodologies Machine learning approaches CCS: Human-centered computing HCI design and evaluation methods Md. Saif Hassan Onim, Andrew M. Kiselica, Himanshu Thapliyal Abstract . Emotion detection in older adults is crucial for understanding their cognitive and emotional well-being, especially in hospital and assisted living environments. In this work, we investigate an edge-based, non-obtrusive approach to emotion identification that uses only physiological signals obtained via wearable sensors. Our dataset includes data from 40 older individuals. Emotional states were obtained using physiological signals from the Empatica E4 and Shimmer3 GSR+ wristband and facial expressions were recorded using camera-based emotion recognition with the iMotion’s Facial Expression Analysis (FEA) module. The dataset also contains twelve emotion categories in terms of relative intensities. We aim to study how well emotion recognition can be accomplished using simply physiological sensor data, without the requirement for cameras or intrusive facial analysis. By leveraging classical machine learning models, we predict the intensity of emotional responses based on physiological signals. We achieved the highest 0.782 r 2 r^{2} score with the lowest 0.0006 MSE on the regression task. This method has significant implications for individuals with Alzheimer’s Disease and Related Dementia (ADRD), as well as veterans coping with Post-Traumatic Stress Disorder (PTSD) or other cognitive impairments. Our results across multiple classical regression models validate the feasibility of this method, paving the way for privacy-preserving and efficient emotion recognition systems in real-world settings. † † cc-license: by 1. Introduction Understanding emotion is vital to understanding human behavior, especially in older adults, as emotional well-being is significantly related to cognitive and physical health. Research has shown that changes in emotional patterns can serve as early indicators of neurodegenerative diseases such as Alzheimer’s disease and related dementia (ADRD) ( Rhodus et al. 2024 ) . They also help diagnose mental health conditions such as PTSD, depression, and anxiety disorders in veterans and other vulnerable populations. Traditional emotion recognition approaches are primarily based on facial expression analysis, speech recognition, and behavioral assessments ( Din et al. 2024 ; Badawi et al. 2024 ) . They often require camera-based monitoring or active user participation. Although effective, these methods raise concerns about privacy, intrusiveness, and real-world applicability. Recent advancements in wearable technology and physiological sensing have introduced a promising alternative to detect mental health conditions and relevant cognitive impairments ( Onim et al. 2023 ; Delmastro et al. 2020 ; Kikhia et al. 2016 ) . Researchers employ both obtrusive and non-obtrusive instruments to gather physiological signals that serve as biometric indicators ( Onim et al. 2024 ; Cheong et al. 2020 ; Ferreira et al. 2014 ) . Beyond emotion recognition, previous studies have explored stress detection using these physiological signals. For example, research that uses EDA and HRV has shown promising results in identifying acute stress responses in real-time settings. Other studies have used Empatica E4 to assess the cognitive load, anxiety, and stress during educational tasks ( Onim and Thapliyal 2023 ; Onim and Thapliyal 2024 ; Belk et al. 2016 ) . These works reveal strong correlations between physiological changes and fluctuations in mental state and highlight the versatility of physiological sensors. Table 1 . Overview of Recent Literature on Emotion Detection from Physiological Signals Authors & Year Data Acquisition Detected Emotions Emotion Stimuli Requires Video for Detection ? Target Age Group of Participants Wei et al . ( Wei et al. 2017 ) 2017 EEG Positive and Negative Pictures ✗ Adults Hui et al . ( Hui and Sherratt 2018 ) 2018 PPG, EDA, SKT and EMG Happiness, Anger, Fear, Disgust and Sadness Pictures and Videos ✓ Adults Quiroz et al . ( Quiroz et al. 2018 ) 2018 HR Neutral, Happy and Sad Musics and Videos ✗ Young Adults Heinisch et al . ( Heinisch et al. 2018 ) 2018 EMG, EDA, TMP, EEG, BVP, RR and ECG HPVHA, HNVLA, HPVLA HNVHA and Neutral Pictures ✗ Young Adults Ragot et al . ( Ragot et al. 2018 ) 2018 EDA and HR Valence and Arousal Pictures ✗ Young Adults Xiefeng et al . ( Xiefeng et al. 2019 ) 2019 HS and ECG Relaxed, Happy, Sad and Angry Musics and Videos ✗ Young Adults Costa et al . ( Costa et al. 2019 ) 2019 EDA, PPG, IMU and GSR Anger, Contempt, Disgust, Fear, Happiness, Neutral, Sadness and Surprise Pictures ✓ Adults Jiang et al . ( Jiang et al. 2019 ) 2019 EEG and Motion Fear, Frustrated, Sad, Satisfied, Pleasant and Happy Musics, Pictures and Videos ✓ Adults Zhang et al . ( Zhang et al. 2020 ) 2020 HR, EDA, BVP, TEMP, PD, SA, and SV Valence and Arousal Not Mentioned ✓ Young Adults Kim et al . ( Kim et al. 2020 ) 2020 GSR and PPG Pride, Elation, Joy, Satisfaction, Relief, Hope, Interest, Surprise, Sadness, Fear, Shame, Guilt, Envy, Disgust, Contempt and Anger Musics, Pictures and Videos ✓ Adults Shu et al . ( Shu et al. 2020 ) 2020 HR Neutral, Happy and Sad Musics, Pictures and Videos ✓ Young Adults Miranda et al . ( Miranda et al. 2021 ) 2021 ECG, GSR, EEG and SKT Fear and Non-fear Videos ✓ Adults Chen et al . ( Chen et al. 2021 ) 2021 SP Happiness, Sadness, Anger and Fear Videos ✗ Adults Badawi et al . ( Badawi et al. 2024 ) 2024 PR, PRV, RR, SCL Agitation and Aggression Indoor Activities ✓ Older Adults Proposed Work 2025 EDA, BVP, IBI Positive, Negative and Neutral TSST Protocol ✗ Older Adults SA: Saccadic amplitude, SV: Saccadic velocity, HS: Heart Sound, PPG: Photoplethysmography, SKT: Skin Temperature, EMG: Electromyography, RR: Respiration Rate, PR: Pulse Rate, ECG: Electrocardiogram, EEG: Electroencephalography, EDA: Electrodermal Activity, SCL: Skin Conductance Level, IMU: Inertial Measurement Unit, PD: Pupil Dilation,HR: Heart Rate, TMP: Temperature, SC: Step Counter, BVP: Blood Volume Pressure, IBI: Inter Beat Interval, SP: Skin Potential, PRV: Pulse Rate Variability Table 1 provides a detailed overview of recent research on emotion detection using physiological signals, showcasing various methodologies, detected emotions, and participant demographics. Wei et al . ( Wei et al. 2017 ) and Ragot et al . ( Ragot et al. 2018 ) employed EEG and EDA/HR, respectively, to detect broad emotional states such as positive/negative emotions and valence/arousal without relying on video data. In contrast, Hui et al . ( Hui and Sherratt 2018 ) and Kim et al . ( Kim et al. 2020 ) combined physiological data with video-based methods to detect complex emotions like fear, shame, and guilt. Costa et al . ( Costa et al. 2019 ) and Jiang et al . ( Jiang et al. 2019 ) integrated multi-modal data such as EEG, GSR, and motion data with stimuli like pictures, videos, and music to enhance emotion recognition. While these studies focus primarily on adults and young adults, Badawi et al . ( Badawi et al. 2024 ) addressed older adults by detecting agitation and aggression using physiological signals during indoor activities. Usually, detections are made based on biometric signals such as electrodermal activity (EDA), heart rate variability (HRV), blood volume pulse (BVP), and skin temperature (ST). However, most existing works integrate facial emotion recognition ( Din et al. 2024 ; Badawi et al. 2024 ; Saganowski et al. 2022 ) alongside physiological data for improved accuracy or focus on detecting a limited set of emotions rather than a comprehensive range of affective states ( Schmidt et al. 2018 ) . Previous studies often depend on deep learning models and cloud-based processing, which significantly limit the feasibility for real-time edge deployment ( Wijasena et al. 2021 ; Ba and Hu 2023 ) . This is of concern, particularly for older adults who may be uncomfortable or unable to participate in such assessments. Thus, the practical use of such a framework in privacy-sensitive settings such as nursing homes or veterans’ care facilities is often underexplored ( Rhodus et al. 2024 ) . To address these challenges, we propose an edge-based, non-intrusive emotion detection framework that relies solely on wearable physiological sensor data without requiring cameras or facial analysis. Our proposed work focuses specifically on older adults. These groups are often overlooked in existing studies. Unlike previous research that often integrates facial recognition or video-based methods, our framework uses only physiological signals. Thus, our method eliminates privacy concerns and enables real-time edge-based deployment. This design enhances the practicality and scalability of our method for real-world use in settings such as nursing homes, clinical care, and home environments. Our contribution to this paper is as follows: • We used a data set collected from 40 older adults, where emotions were recorded through physiological signals from the Empatica E4 and Shimmer3 GSR+ wristband. • Participants’ facial expression was captured with live video and labeled using iMotion’s Facial Expression Analysis (FEA). • We detected the intensity for three distinct emotions: Neutral, Positive, and Negative with only the labeled sensor data. We used machine learning for regression tasks and reported the performance on the test set. • By focusing on classical models rather than deep learning, we ensured computational efficiency, interpretability, and compatibility with edge-based deployment. • We demonstrated that emotion recognition can be accurately achieved using only sensor data, eliminating the need for visual or behavioral inputs. The paper is organized as follows: Section 2 explains the stimulus protocol for emotion and the data recording. The suggested method for emotion detection is presented in Section 3 . The results of the study and their analysis are presented in section 4 . Section 5 presents the conclusion with future direction. 2. Stimulus Protocol TSST and Data Recording In this study, we selected 40 healthy elderly individuals aged between 60 and 80, comprising 28 females and 12 males. Due to the corruption of one participant’s data, 39 participants’ data were ultimately utilized. Before enrollment, participants underwent screening for any pre-existing medical conditions to ensure an unbiased dataset. The stimulus for emotions was imposed based on Trier Social Stress Test (TSST) protocol. Figure 1 . Stimulus Protocol TSST for Capturing Facial Emotion The TSST is a renowned experimental procedure recognized for its effectiveness in causing stress within a naturalistic setting ( Birkett 2011 ) . Thus, we chose the TSST protocol as our external stimulus to impose emotions. Figure 1 presents the stages involved in the experimental process. The TSST protocol encompasses the waiting period, pre-stress phase, stress phase, and recovery phase. During the waiting period, participants complete demographic questionnaires and can ask any questions regarding the study procedures. Baseline data are collected in the pre-stress phase, which directly follows the waiting period. Together, these phases last 20 minutes (T1-T2). The 20-minute stress phase proceeds after the pre-stress phase (T2-T3) and consists of a 10-minute anticipatory stress segment (AS) and a 5-minute segment comprising both speech and mental arithmetic tasks (M). During the AS phase, participants are tasked with delivering a continuous 5-minute speech in front of an audience. The mental arithmetic component requires participants to solve basic addition and subtraction problems under observation. The complexity of these arithmetic tasks increases as participants correctly solve each problem. Finally, the study concludes with two 20-minute recovery phases (T3-T5). For recording the physiological sensor data, two commercially available smart devices called the Empatica E4 and Shimmer3 GSR+ were used. These devices have EDA, PPG, and ST sensors inbuilt. The recording mode keeps collecting data that can be synchronized after each use. Features collected from the sensors are Yaw, Pitch and Roll from the accelerometer, Temperature, Internal ADC voltage, GSR Resistance, Heart Rate and GSR Conductance. 3. Proposed Method for Emotion Detection Figure 2 . Proposed Emotion Detection Method In this section, we will discuss our proposed method for emotion detection with machine learning-based regression models. An overview of the proposed method is shown in Fig. 2 . The three main components are (i) Data Preprocessing, (ii) Emotion ground truth labeling with iMotion’s Facial Expression Analysis (FEA), and (iii) regression with classical machine learning models. In this section, we will explain them in detail. 3.1. Preprocessing and Ground Truth Labeling The dataset is preprocessed for labeling after data collection. A 60-sample moving average window was used to scan the time series data during preprocessing. This assisted in eliminating unwanted artifacts and noise. To prevent data leakage, the train and test sets of participants are later divided and normalized independently using z-score normalization. In our study, ground truth labeling for emotion recognition is established using iMotion’s Facial Expression Analysis (FEA) module ( iMotions A/S 2024 ) , which employs an advanced implementation of the Facial Action Coding System (FACS). This is a widely used framework in affective computing that can identify facial muscle movements known as Action Units (AUs). Each combination of such action units can correspond to specific emotional expressions. The FEA module leverages computer vision and deep learning algorithms to analyze these AUs in real time. Table 2 . Emotion Baseline and Percentage Outside 1st Standard Deviation for Labeled Dataset Emotion Baseline % outside 1st std Emotion Baseline % outside 1st std Joy -0.6629763 24.28 Sadness -0.515366 27.38 Anger 0.3638698 41.11 Neutral -0.0408471 34.96 Surprise -1.859378 7.61 Positive -0.6629763 24.28 Fear -0.8419589 15.12 Negative 0.3638698 18.49 Contempt -0.1459146 33.83 Confusion 0.809446 46.93 Disgust -0.3184329 21.58 Frustration 0.5876518 42.75 \star In This work, we will detect only Neutral, Positive and Negative emotion This automated analysis ensures a standardized, objective method for labeling emotions, reducing potential bias associated with self-reporting. By synchronizing physiological sensor data (EDA, BVP, and IBI) with facial expression-derived emotion labels, we created a robust dataset that enables training and validation of our edge-based emotion recognition framework. After the labeling, a distribution of Emotion intensity is shown in Table 2 . Again, Fig. 3 shows a 2D projection of all the emotion classes after labeling. The centroid of the cluster shows how all the emotional states can overlap and are hard to distinguish. In this work, we plan to detect only 3 sets of emotions: Positive, Negative and Neutral . Figure 3 . 2D Projection of the Labeled Dataset with PCA Showing the Cluster Centers where Each Color Represents Different Emotion 3.2. Machine Learning Models for Regression To evaluate the performance of our framework, we implemented seven classical machine learning methods with one 10-layer Multi-Layer Perceptron (MLP) and a 3-hidden-layer Dense Neural Network. The correlation heatmap shown in Fig. 4 summarizes that traditional statistical modeling will make it difficult to make an emotion detection method. Thus, machine learning models that can handle such small correlations will prove to be well-suited for such problems. The most simplistic regression model is the linear regression. It assumes a direct linear relationship between physiological features and emotional intensity. It estimates coefficients that best fit the data by minimizing the sum of squared errors. Ridge Regression is a linear model that incorporates L2 regularization, which penalizes large coefficients to prevent overfitting. Bayesian Ridge Regression extends Ridge Regression by incorporating a probabilistic framework, which assigns prior probability distributions to the model coefficients. K-Nearest Neighbors (KNN) Regression is a non-parametric model that estimates emotion intensity based on the average intensity values of the K most similar physiological responses. This approach assumes that individuals with similar physiological reactions experience similar emotional intensities. Decision Tree Regression is a rule-based model that recursively splits the data into subsets based on the feature values that minimize the prediction error at each node. Another prominent method is the Random Forest Regression which is an ensemble learning method. It constructs multiple decision trees during training and averages their predictions for a more accurate and robust outcome. Gradient Boosting Regression is a more complex model. It is an advanced ensemble technique that sequentially builds multiple weak learners (typically decision trees), where each new tree corrects the errors of the previous ones. This method reduces both bias and variance, leading to highly accurate predictions. Multi-Layer Perceptron (MLP) Regression is a type of feedforward neural network consisting of multiple hidden layers. Each layer processes the input data using weighted connections and non-linear activation functions. Finally, A Dense Neural Network (DNN) consists of multiple fully connected layers that transform input features through non-linear activation functions. By learning hierarchical representations, DNNs can capture complex interactions between physiological signals and emotional intensity levels. Figure 4 . Correlation Heatmap of the Features and Emotion of Dataset 3.3. Model Training and testing As we had participant-specific data, there were multiple ways to generalize the machine learning models based on the dataset. To avoid data leakage, we split the dataset based on participants before normalizing them. We performed Leave One Sample Out (LOSO) cross-validation keeping one subject for testing and others for training. We used some widely used regression models as our base machine learning model mentioned in Section 3.2 . For each of the regression models, the set of hyperparameters used is shown in Table 3 . The values for these parameters were chosen based on the best-performing set on test data after multiple trial and error. Table 3 . Hyper-parameters Used to Train Machine Learning Models Machine Learning Model Hyper-parameters Random Forest no of estimators=100, criterion=squared error Dense Neural Network no of hidden layers=3, loss=mse, output layer activation=ReLU, optimizer=Adam, epochs=50, Learning rate = 0.0001 Decision Tree criterion=squared error K-Nearest Neighbors (KNN) no of neighbors=3, weights=uniform, distance metrics=minkowski Gradient Boosting no of estimators=100, Learning rate=0.1, loss=squared error,criterion=friedman mse Multi Layer Perceptron no of hidden layers=10, loss=mse, output layer activation=ReLU, optimizer=Adam, Learning rate = 0.001, maximum iteration=500 Ridge Regression maximum iteration=1000, tolerance= 10^{-4} , solver=svd Bayesian Ridge Regression maximum iteration=1000, tolerance= 10^{-3} , \alpha_{1},\alpha_{2},\lambda_{1},\lambda_{2}=10^{-6} Linear Regression N/A 4. Result Analysis and Discussion In this section, we will evaluate the performance of machine learning models with our dataset with respect to the estimated ground truth mentioned in Section 3.1 . We have evaluated the model’s performance using r^{2} score and MSE with the equations shown in Eq. ( 1 ) and ( 2 ) where, y_{i} = actual value, \hat{y}_{i} = predicted value and \bar{y}_{i} = mean of actual value. The r^{2} score of a model says how much of the variability in the target that model can explain in terms or normalized distance. Again, MSE refers to the squared difference between the model’s predicted values and the actual values. The scores are expressed as the mean of all folds for LOSO cross-validation. (1) R^{2}\ score=1-\frac{\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}}{\sum_{i=1}^{n}(y_{i}-\bar{y})^{2}} (2) \text{MSE}=\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2} The results indicate that Random Forest outperforms other models as evidenced by its highest r^{2} scores in Table 4 and lowest Mean Squared Error (MSE) in Table 5 across all emotion categories. That is because the ensemble nature of Random Forest helps reduce overfitting and enhances generalization. This indicates that the Random Forest model was able to explain a substantial portion of the variance in emotional states with reasonable confidence. Table 4 . r^{2} Scores for Various ML Models (Higher is better) ML Model Negative Neutral Positive Random Forest 0.782 0.7636 0.8033 Dense Network 0.5657 0.5339 0.5813 Decision Tree 0.5377 0.5253 0.5762 K-Nearest Neighbors (KNN) 0.5217 0.5021 0.5673 Gradient Boosting 0.3545 0.3644 0.2997 Multi Layer Perceptron 0.3088 0.2568 0.2655 Ridge Regression 0.0752 0.0371 0.0527 Bayesian Ridge Regression 0.0752 0.0370 0.0526 Linear Regression 0.0752 0.0370 0.0526 In contrast, simpler models such as Ridge Regression, Bayesian Ridge Regression, and Linear Regression perform poorly. They showed significantly lower r^{2} scores with high MSE. This implies that linear models fail to capture the intricate dependencies between physiological signals and emotion states. Since physiological signals are often highly nonlinear and affected by multiple interacting factors, simpler models struggle to provide meaningful predictions. Table 5 . MSE for Various ML Models (Lower is better) ML Model Negative Neutral Positive Random Forest 0.0006 0.0014 0.0019 Dense Network 0.0012 0.0029 0.0030 Decision Tree 0.0012 0.0030 0.0030 K-Nearest Neighbors (KNN) 0.0013 0.0031 0.0044 Gradient Boosting 0.0017 0.0042 0.0050 Multi Layer Perceptron 0.0018 0.0047 0.0052 Ridge Regression 0.0025 0.0060 0.0067 Bayesian Ridge Regression 0.0025 0.0060 0.0067 Linear Regression 0.0025 0.0060 0.0067 Neural network-based models, such as the Dense Network and Multi-Layer Perceptron (MLP), perform moderately well. While they achieve better r^{2} scores than linear models, their higher MSE values suggest a higher degree of variance in their predictions. However, their ability to outperform purely linear models suggests that with further optimization and larger datasets, deep learning approaches could become more competitive in emotion detection from physiological signals with more complexity as that would make them harder to deploy in the edge. 5. Conclusion This study demonstrates the feasibility of emotion detection using physiological signals alone that eliminates the need for intrusive facial recognition methods. Our results show that classical machine learning models can perform better in predicting emotion intensities. These findings validate the effectiveness of wearable sensor data in capturing emotional states that can offer a privacy-preserving and edge-compatible solution for real-world applications. This has significant implications for individuals with Alzheimer’s Disease and Related Dementia (ADRD), veterans with Post-Traumatic Stress Disorder (PTSD), and other cognitive impairments. This approach can be integrated into hospital and assisted living settings to monitor emotional well-being without requiring active participation from individuals. This is especially beneficial for patients with limited verbal or cognitive abilities. Future work can expand on this foundation by incorporating deep learning models, transfer learning, and larger datasets to enhance performance further. This study lays the groundwork for real-time, privacy-conscious emotion detection systems which is viable both for healthcare monitoring and human-centered AI.