Human-Centered Non-Intrusive Driver State Modeling Using Personalized Physiological Signals in Real-World Automated Driving CCS: General and reference Surveys and overviews CCS: Human-centered computing User models CCS: Human-centered computing User centered design CCS: Computing methodologies Modeling methodologies CCS: Applied computing Transportation
David Puertas-Ramirez, Raul Fernandez-Matellan, David Martin Gomez, Jesus G. Boticario · 2026
Read Human-Centered Non-Intrusive Driver State Modeling Using Personalized Physiological Signals in Real-World Automated Driving CCS: General and reference Surveys and overviews CCS: Human-centered computing User models CCS: Human-centered computing User centered design CCS: Computing methodologies Modeling methodologies CCS: Applied computing Transportation by David Puertas-Ramirez, Raul Fernandez-Matellan, David Martin Gomez, Jesus G. Boticario (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11549.txt.
Introduction
Puertas-Ramirez and three colleagues test driver-state models tuned to each driver, using body signals read without skin contact in automated driving. In an SAE Level 2 vehicle, an Empatica E4 wrist device captured skin conductance, heart rate, heat, and motion from four drivers. Signals were recast as images and run through a multi-input ResNet50-based setup. Models tuned to each driver averaged 92.68 percent accuracy, while general models fell to 54 percent, exposing wide differences between drivers and weak transfer across drivers. Future monitoring must adapt to each driver.
Catalog record
| CREATOR | David Puertas-Ramirez, Raul Fernandez-Matellan, David Martin Gomez, Jesus G. Boticario |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47502 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11549
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Human-Centered Non-Intrusive Driver State Modeling Using Personalized Physiological Signals in Real-World Automated Driving CCS: General and reference Surveys and overviews CCS: Human-centered computing User models CCS: Human-centered computing User centered design CCS: Computing methodologies Modeling methodologies CCS: Applied computing Transportation David Puertas-Ramirez, Raul Fernandez-Matellan, David Martin Gomez, Jesus G. Boticario Abstract . In vehicles with partial or conditional driving automation (SAE Levels 2-3), the driver remains responsible for supervising the system and responding to take-over requests. Therefore, reliable driver monitoring is essential for safe human–automation collaboration. However, most existing Driver Monitoring Systems rely on generalized models that ignore individual physiological variability. In this study, we examine the feasibility of personalized driver state modeling using non-intrusive physiological sensing during real-world automated driving. We conducted experiments in an SAE Level 2 vehicle using an Empatica E4 wearable sensor to capture multimodal physiological signals, including electrodermal activity, heart rate, temperature, and motion data. To leverage deep learning architectures designed for images, we transformed the physiological signals into two-dimensional representations and processed them using a multimodal architecture based on pre-trained ResNet50 feature extractors. Experiments across four drivers demonstrate substantial interindividual variability in physiological patterns related to driver awareness. Personalized models achieved an average accuracy of 92.68%, whereas generalized models trained on multiple users dropped to an accuracy of 54%, revealing substantial limitations in cross-user generalization. These results underscore the necessity of adaptive, personalized driver monitoring systems for future automated vehicles and imply that autonomous systems should adapt to each driver’s unique physiological profile. 1. Introduction The division of driving tasks and responsibilities is evolving as vehicles become more automated. According to SAE International ( 43 ) , Level 0 vehicles are fully manual, while Level 1 is termed Driver Assistance. Level 2, or Partial Driving Automation, provides sustained control of both steering and speed, yet requires the driver to continuously supervise the system and the driving environment. At Level 3, vehicles can handle most driving tasks, but occasionally issue a Take-Over Request (TOR) to the driver. Level 4 systems operate autonomously within a restricted Operational Design Domain (ODD), and Level 5 represents a vision of complete autonomy in all driving conditions. Current implementations consist of vehicles with Partial Driving Automation (SAE Level 2), and Level 4 "RoboTaxis" deployments limited in geo-fenced areas ( 38 ) . This focuses attention on the complexities of the intermediate levels where driving tasks demand collaboration ( 43 ) . A fundamental challenge for both Level 2 and Level 3 is the well-known paradox of the "irony of automation" ( 3 ) , which states that the more reliable a system is, the harder it is for the human operator to maintain attention. At Level 2, this applies to the task of continuous supervision. At Level 3, it applies to the driver’s ability to maintain a "fallback-ready" state for TORs, a dynamic that also introduces complex legal and ethical questions about determining responsibility at any given moment ( 17 ) . Resolving the challenges of both sustained supervision and safe control transitions fundamentally depends on accurately understanding the driver’s state. Existing Driver Monitoring Systems (DMS) often rely on user-agnostic models to assess attention or alertness ( 9 ) , overlooking significant inter-individual variability in physiological signals, cognitive load, and behavior, especially under complex driving conditions ( 49 ) . This generalization can compromise accuracy, particularly for diverse drivers. Current research has begun exploring the physiological state of users inside Autonomous Vehicles (AVs) using simulation environments, as this way the complete environment can be controlled and dangerous situations can be tested safely ( 42 ; 4 ; 36 ; 2 ) . While simulations provide valuable insights and a controlled environment for testing, they have limitations and may not fully capture the intricacies, challenges, and unexpected situations inherent in real-world data, which are essential for gaining a deeper understanding of the problem ( 1 ; 33 ; 41 ) . Our work addresses these two distinct gaps: we propose a novel approach that not only uses physiological sensors in a real-world driving environment but also develops personalized models to accurately reflect each individual’s state. To guide our investigation, we formulate the following hypotheses: • H1: A personalized modeling approach, using data from non-intrusive physiological sensors in real-world driving, can accurately classify distinct driver states. • H2: The physiological patterns of driver awareness exhibit significant and measurable inter-individual variability. • H3: Incorporating data from other drivers into a user-specific training set degrades the model’s predictive accuracy compared to a model trained exclusively on that user’s own data. The experiments for this study were conducted using an experimental SAE Level 2 vehicle in real-world driving conditions on public highways. To model the driver’s state, we used data from non-intrusive physiological sensors. This real-world methodology ensures that participant responses are natural and immersion is high, enabling the identification of unexpected situations and potential problems relevant to future commercial deployments. The core of our modeling technique utilizes Convolutional Neural Networks (CNNs), which are well-established for complex pattern recognition. Since CNNs are designed for image data, we first transform the raw physiological signals into 2D image representations suitable for this type of analysis. The value of personalization is well-established in fields like precision medicine and adaptive learning, where user-specific models outperform generalized ones ( 44 ; 14 ; 19 ; 28 ; 48 ; 12 ; 7 ) . Adopting this approach shifts the experimental focus from broad data collection to intensive, longitudinal data from fewer participants to effectively model individual differences. Contributions. This work investigates the feasibility and implications of personalized driver-state modeling using non-intrusive physiological sensing in real-world automated driving. The main contributions of this paper are: (1) Real-world physiological sensing for automated driving. We present a driver monitoring study conducted in a real-world SAE Level 2 automated vehicle, using non-intrusive wearable sensors to capture multimodal physiological signals under natural driving conditions. (2) A signal-to-image modeling pipeline for physiological data. We propose a multimodal architecture that converts physiological time-series signals into image representations and leverages pre-trained convolutional neural networks ( ResNet50 ) for feature extraction and driver state classification. (3) Empirical evidence of strong inter-individual variability in driver awareness. Through extensive experiments across four drivers, we demonstrate that physiological patterns related to driver awareness vary significantly between individuals, leading to substantial performance degradation when models are applied across users. (4) Evidence supporting personalized driver monitoring systems. Our results show that personalized models dramatically outperform generalized approaches (92.68% vs. 54% accuracy), suggesting that adaptive personalization is a key design requirement for future human-centered driver monitoring systems in automated vehicles. The remainder of the paper is organized into the following sections: 2 Related Work, 3 Methodology, 4 Experimental results, 5 Discussion, and 6 Conclusions and Future Work. 2. Related Work Driver State Monitoring in Conditional Automation The necessity of maintaining situational awareness in SAE Level 2 and 3 vehicles is well-documented ( 3 ; 11 ) . Research has largely bifurcated into detecting physiological alertness such as drowsiness ( 27 ) , fatigue ( 30 ) , and distraction ( 52 ; 6 ) , and modeling the driver’s relational dynamics with the vehicle, including trust ( 25 ; 24 ) , acceptance ( 53