SENSORY CONFLICT IN SIMULATOR SESSIONS – MEASURING BIOSIGNALS TO PREDICT THE ONSET OF DISORDERLY SYMPTOMS: A BRIEF LITERATURE REVIEW Authors: Przemysław Wojciechowski, Konrad Wojtowicz, Jan Błaszczyk Published: 2024 ABSTRACT The global virtual reality (VR) market is growing surprisingly fast. As VR applications continue to expand into various areas of life, attention is being paid to issues related to user well-being. The danger lurking for users is the occurrence of simulator sickness and artificial reality sickness, collectively referred to as sensory conflict. As early as the 1950s, an attempt was made to study simulator sickness. Unfavorable psychophysical symptoms occurred in pilots using the first flight simulators. With the development of technology, the graphic and simulation capabilities of the various types of simulators are increasing. Easier access to simulators using first person view (FPV) and thus more outstanding research capabilities allow new studies related to the incidence of this disease to compare symptoms occurring during simulator sessions with those occurring during real-world endeavors. The primary purpose of the review is to bring together the latest reports on different types of sensory conflict concerning factors that are symptomatic in prediction and diagnosis. Heart rate, brain activity, stomach activity, and skin conductance seem to be the most adequate, objective indicators of subjects' susceptibility to this phenomenon. In addition, it is intended to systematize concepts related to sensory conflict in the broadest sense. INTRODUCTION In recent years, there has been a notable surge in the utilization of virtual reality (VR) across a multitude of domains. In 2022, the market was estimated at USD 59.96 billion and is projected to grow by 27.5% annually from 2023 to 2030. In the USA alone, revenue from the VR market grew from USD 124 million in 2016 to USD 2.9 billion in 2021 [1]. As technology advanced, the influence of such developments on the human body came to be the subject of increasing scrutiny. One of the research areas receiving particular attention is that of sensory conflict. Sensory conflict is a phenomenon that is increasingly becoming a subject of research in medicine, psychology, engineering, and behavioral sciences. Known by various terms, e.g., simulator sickness, visually induced motion sickness, or simulator-induced sickness, this conflict is a phenomenon in which a person experiencing a simulation, such as a flight simulator, virtual reality (VR), or computer games, manifests sickness symptoms that can be similar to those of motion sickness or migraine. However, unlike traditional motion sickness, this disease is triggered by sensations received using simulators or virtual environments rather than by the vehicle's movement. Early examples The first reports of related symptoms date back to the Second World War when pilots training in flight simulators began complaining of symptoms similar to motion sickness. Symptoms included nausea, dizziness, and general malaise. However, for understandable reasons, understanding or studying this phenomenon was not a priority at the time, so no attempt was made to explain the causes of pilots' malaise [2]. Development of research Systematic research into the diagnosis began in the 1950s and 1960s when flight simulator technology began to develop, which more pilots had access to them. Pilots and training staff noticed some trainees experiencing unwanted side effects when using flight simulators. Researchers began documenting these cases and experimenting with various factors that could affect the symptoms [2]. In the 1950s, the Bell Aircraft Corporation built a helicopter simulator that had already been reported to affect pilots during demonstration tests (dizziness) negatively. In the 1970s, the development of simulators closer to those seen today can be observed, while in the 1980s, the statement of simulator sickness can already be read in the U.S. Army reports. In 1987, Jaron Lanier first used the term “virtual reality,” and as early as 1995, Kay Stanney recommended the word “cybersickness” to describe the side effects caused by exposure to virtual reality [3]. Over the years, scientists have tried to distinguish between simulator sickness and cybersickness [4]. A distinction was made between the symptoms, the duration and timing of the disease, the mechanism of each symptom, and the conditions that must be met to minimize the impact of the diseases on the subjects. Over time, the expressions “virtual reality sickness,” “visually induced motion sickness,” or, more generally, “sensory conflict/vestibular-visual conflict” also began to be used. Some definitions evolved from others, while subcategories were drawn from others. This review also examines the current understanding of the various terms used for the conflict. Symptoms Sensory conflict disorders have a variety of symptoms that are often physical. The most commonly cited symptoms are nausea, vomiting, dizziness, headaches, feelings of fatigue, sweating, numbness in the extremities, and an accelerated heartbeat [3]. These symptoms can be difficult to bear and significantly affect the comfort of simulator users. Moreover, they disrupted basic flight training by making it impossible to determine how prepared pilots were for specific missions. Besides physical symptoms, these diseases can also lead to psychological symptoms, such as anxiety, restlessness, and spatial disorientation. Individuals experiencing this phenomenon may experience disorientation and a loss of a sense of reality, which can lead to increased stress and negatively affect the performance of the assigned task [5]. Mechanism of onset Sensory conflict, psychological factors, response delays, sensory adaptation, tunnel effect, individual sensitivity, or degree of simulation realism – this is the most common terminology used to describe the mechanisms underlying the generation of symptoms associated with the impact of the simulator on a human being. Simulators, virtual reality, or computer games often provide users with visual stimuli that conflict with sensory information other senses receive, such as balance maintenance or proprioceptor signals (the sense that enables the perception of body position in space). This conflict is called visual-vestibular conflict [6,7]. Some studies suggest that psychological factors, such as individual personality traits, anxiety, stress, or negative expectations, can influence the severity of simulator sickness. Individuals who are more prone to anxiety and stress may experience more disorientation while using VR, which increases the risk of simulator symptoms sickness [7,8]. Delays between a user's movement and VR's response to that movement can affect the onset of simulator sickness. The longer the delay, the higher the risk of disorientation and loss of balance. Proper technical and hardware optimization of VR can help reduce this problem [9]. Long-term use of VR technology can affect the user's sensory adaptation to an environment of this type. After some time, the body may adapt and elicit less response to visual stimuli, which reduces the intensity of simulator sickness [10,11]. Some VR applications use a tunnel effect that limits the user's field of vision. This effect can affect feelings of disorientation and cause subjects to experience symptoms of simulator sickness [10]. Simulator sickness is not equally experienced by all simulators or VR users. Individual sensitivity to it varies; some people are more susceptible to its onset than others. This fact suggests that genetic and psychological factors may play a role in its etiology [12]. Perhaps the most crucial factor affecting the occurrence of simulator sickness is the degree of realism of the simulation. The more realistic and immersive the simulation is, the more likely the user experiences symptoms. It explains why the disease is reported more often in advanced flight simulators or VR games [13,14]. Sensory conflict is a significant phenomenon that requires an interdisciplinary approach to understand its causes and consequences. It is a problem that can be looked at from a medical and engineering point of view. It has implications for the developing technologies of simulators and virtual reality. Research on simulator sickness remains active, and the results are helping to improve simulations and make users comfortable and safe when using modern technology. METHODS Publications available in PubMed, Elsevier, and Multidisciplinary Digital Publishing Institute (MDPI) databases were reviewed, and papers on the study of flight simulators' effects on the pilot's body were selected. Keyword combinations were “simulator,” “cybersickness,” “simulator sickness,” “virtual reality,” and “sensor conflict.” Only current works were included in the analyses, limiting the years of publication to 2014–2023. Studies conducted with healthy adults without physical or mental illnesses were included. Publicly available papers in both Polish and English were analyzed. The review was conducted at the Military University of Technology, Warsaw, Poland, in December 2023. The authors excluded works where full content was unavailable, did not analyze psychophysical reactions caused by virtual reality influence, and conducted studies in groups of people with diagnosed motion sickness. RESULTS During the period from January 2014 to the end of September 2023, 87 papers were published in the PubMed publication database referring to systematic search with the following query: (“cybersickness” OR “simulator sickness” OR “sensor conflict”) AND (“virtual reality” OR “simulator”) identified in the keywords of the publication in the Elsevier database the number of articles was 255, while in the MDPI database, there were 236 articles. Content analysis of the abstracts or full texts made it possible to identify 74 papers meeting the criteria (Figure 1). Study selection process for the review on measuring biosignals for the prevention of sensor conflict in simulator sessions in 2014–2023 Current understanding of basic definitions Cybersickness Cybersickness is a set of symptoms resulting from excessive use of electronic devices such as computers, televisions, and smartphones (Figure 2). Researchers at Coventry University have dubbed the phenomenon a “cybersickness.” Classification of sickness types according to the mechanism of occurrence Symptoms are similar to motion sickness and can include nausea, headaches, and other discomforts, often occurring during rapid or prolonged scrolling of content displayed on mobile devices. Other symptoms can include dizziness, fatigue, trouble concentrating, and the post-exposure period, contributing to sleep disturbances. The duration of cybersickness symptoms can vary depending on the subjects' predispositions and the intensity of screen exposure [15–18]. Simulator sickness Simulator sickness is one of a subset of visual-vestibular conflict disorders (Figure 2). Respondents typically experience it from a first-person perspective when playing video games. It was discovered in the context of airplane pilots, who extensively use flight simulators during training. Although the exact date of discovery is not available, it is known that research on a similar phenomenon, known as simulator sickness, was conducted as early as 1958 [2,3]. It is, in many ways, similar to motion sickness but occurs in simulated environments and can be induced without actual movement [19]. Symptoms of simulator sickness include discomfort, apathy, drowsiness, confusion, fatigue, and nausea. These symptoms negatively affect the effectiveness of simulator training in flight training and can consequently lead to its reduction for some pilots. What is more, pilots who have experienced simulator sickness are less likely to use simulators, which can affect their level of training, increasing their reaction time in emergencies, for example [20]. The duration of symptoms of simulator sickness can vary depending on the person and the intensity of exposure to the simulator. For those who suffered from severe symptoms (total Simulator Sickness Questionnaire (SSQ) score of >60), recovery time was >30 min. On the other hand, for those who experienced only mild symptoms (total SSQ score of ≤25), it took ≤5 min to recover from the symptoms of the simulator sickness [21–25]. Virtual reality sickness Virtual reality sickness occurs when exposure to a virtual environment causes symptoms similar to motion sickness. It is a particular form of simulator sickness directly related to the use of VR (Figure 2). The most common symptoms include general discomfort, eye strain, headache, increased stomach activity, nausea, vomiting, pallor, sweating, fatigue, drowsiness, confusion, and apathy [26–28]. The disease was first observed as early as the 1960s and 1970s when the first users of VR goggles spent too much time in virtual reality. However, it was only after the release of the Oculus Rift DK goggles (Oculus, Irvine, California, USA), when mostly new users of VR technology complained of dizziness and nausea after just a few minutes spent in the VR world, that the disease began to be taken seriously [29]. The duration of virtual reality sickness symptoms can vary depending on the person and the intensity of exposure to the virtual environment. One study found that over half of people playing a virtual reality game using a set of Oculus Rift goggles felt sick in as little as 15 min. Another study, which assessed the relationship between exposure time and the severity of virtual reality sickness symptoms during a 7.5-hour virtual immersion, found that the severity of virtual reality sickness symptoms was positively correlated with exposure time: the longer participants were exposed to the VR environment, the more severe the sickness symptoms they felt [18,29–31]. Visually induced motion sickness Visually induced motion sickness (VIMS) is a phenomenon similar to traditional motion sickness that is often observed in users of technologies that use all types of visualization of spatial situations, such as simulators or virtual reality goggles. It occurs when physically immobile individuals observe a convincing vision of movement (Figure 2). It can also occur when there are detectable delays between head movements and the conversion and presentation of the projected image in head-mounted displays [32]. Symptoms of VIMS can include nausea, vomiting, increased stomach activity, sweating and facial pallor (sometimes referred to as “cold sweats”), salivation, increased sweating, dizziness, drowsiness (also referred to as “sopite syndrome”), sometimes headache, loss of appetite and increased sensitivity to odors [33,34]. Basic biosensors to verify the presence of simulator sickness Heart rate The primary method of measuring heart rate is electrocardiography (ECG) (Figure 2). There is no clear answer to how virtual reality affects heart rate. Studies [26,35–37] suggest that heart rate can increase or decrease. Other, more consistent results were obtained by Garcia-Agundez et al. [38,39]. They found that heart rate decreases in people exposed to simulator or VR sickness. Moreover, some works present a completely different position, according to which the heart rate increases with exposure to simulator sickness [40,41]. Further, one can find a study [42] denying the existence of any relationship between heart rate and the occurrence of simulator sickness. It should be noted that the main factor influencing the change in heart rate is stress caused by external stimuli, which has been confirmed in works such as [43,44]. Therefore, it is necessary to investigate whether these changes are related to possible stress related to the content displayed on the screen or in the simulator goggles rather than the sensations of using virtual reality. Bioelectrical activity of the brain Before interpreting electroencephalography (EEG) data, one must know which brain parts are involved in receiving and analyzing visual information, movement, and balance [45] (Figure 3). In addition, it is necessary to consider whether areas responsible for controlling the digestive system can provide clues about potential susceptibility to simulator sickness. According to a recent study [46], the visual-prefrontal network in the medial cortex consists of the visual cingulate sulcus (CSv), prefrontal motor (PcM), and V6 complex. The V6 complex is a motion-sensitive area that mainly represents the peripheral visual field, and its cells are susceptible to translational motion [47]. The study also found that 2 additional brain areas adjacent to the corpus callosum show similarities in location and response to vestibular and visual stimuli with self-motion-related brain areas recently described in primates. The vestibular pericallosal sulcus (vPCS) processes vestibular information, while the motion-sensitive region in the retrosplenial complex (mRSC) is associated with the activation of (radial) visual motion information [48,49]. Placement of biosensors used to predict the occurrence of sensory conflict in simulator sessions One of the fundamental theories describing the basis of simulator sickness is the sensory conflict [50], arising when comparing visual and vestibular signals in the cerebral cortex. By recording and using EEG, electrical impulses, and brain waves, individual signals can be separated and assigned to different groups. According to the study [51], observed frequencies are mainly categorized into 4 groups: –delta (0.5–4 Hz),–theta (4–8 Hz),–alpha (8–13 Hz),–beta (13–30 Hz). –delta (0.5–4 Hz), delta (0.5–4 Hz), –theta (4–8 Hz), theta (4–8 Hz), –alpha (8–13 Hz), alpha (8–13 Hz), –beta (13–30 Hz). beta (13–30 Hz). However, according to the literature, an additional gamma group [52] and even a “mu rhythm” group [53] are identified. As a result of changes in the power of signals from different frequency bands, some changes in the activity of individual brain areas of the study participants can be distinguished. The energy ratios of the gamma band are related to the level of simulator sickness [54,55]. In 2016, Wibirama and Hamamoto [55] clearly stated that beta and theta frequencies are the best for detecting simulator sickness. An additional method used for diagnostic purposes is the analysis of event-related potentials (ERPs), which measure brain activity during a potential event that may cause simulator sickness [56,57]. Stomach activity Electrogastrography (EGG) is a non-invasive method for measuring gastric myoelectrical activity (Figure 3). It is a promising method for measuring gastric myoelectrical activity by placing epidermal electrodes on the abdomen in the gastric region [58]. Changes in muscle tone, and consequently changes in gastric activity, can indicate an autonomic nervous system response triggered by an external stimulus. In 2016, Dennison et al. [33] found that tachygastric frequency increases with cybersickness while gastric bradygastric activity decreases. One of the main shortcomings of the study was insufficient exposure time to the VR environment because the frequency of EGG measurements was only 3 cycles/min. Potential indicators from the EGG signal are not limited to changes in the ratio of gastric frequencies. In 2021, there was a study that showed that an increase in dominant frequency (DF), percentage of power spectrum density crest factor (CF), and decrease in high power spectral density (PSD) are closely related to cybersickness [59]. It should be underlined that the increase in EGG amplitude occurred simultaneously with the reported nausea. This study lasted 45 min and was divided into 3 equal parts, in which the subjects watched recordings of 3 driving simulations with different route profiles and varying degrees of maneuverability. Electrogastrography is still not widely used. More studies are required to accurately determine the correlation of the signal from the EGG system electrodes and simulator sickness/VR sickness. Brain-gut interaction is undoubtedly an exciting direction for developing research on VR and simulator sickness [60,61]. Skin conductance Various studies often mention the skin conductance (SC) (Figure 3) level or galvanic skin response (GSR). Unfortunately, the available results are ambiguous [15,20,42,50,62]. Some studies have shown a significant relationship between the level of skin conductance and simulator sickness [42], while others have noticed an increase in skin conductance but attributed it mainly to increased arousal, not to cybersickness [63]. The correlation between the level of skin conductance and the occurrence of simulator sickness should be noticeable and easy to examine. Unfortunately, it should be remembered that GSR is nothing more than a measurement of the change in electrical resistance of the skin. It depends on the degree of skin moisture caused by sweating, which the sympathetic system controls. However, it is a challenge to link directly the nature of the change in skin conductance with a specific trigger causing simulator sickness. Verification based on the subjective feelings of respondents It is important to note that the results obtained from biosensors in the tests may not always be the most reliable indicator of disease susceptibility. In addition, the subjective assessment of the subjects' psychophysical state and their feelings about the comfort of the simulators should also be considered [64]. Currently, the most common questionnaires used to assess the psychophysical state of respondents are: –Simulator Sickness Questionnaire (SSQ),–Motion Sickness Assessment Questionnaire (MSAQ),–Cybersickness Questionnaire (CSQ),–Visually Induced Motion Sickness Susceptibility Questionnaire (VIMSSQ, also known as the VIMSSQ-short). –Simulator Sickness Questionnaire (SSQ), Simulator Sickness Questionnaire (SSQ), –Motion Sickness Assessment Questionnaire (MSAQ), Motion Sickness Assessment Questionnaire (MSAQ), –Cybersickness Questionnaire (CSQ), Cybersickness Questionnaire (CSQ), –Visually Induced Motion Sickness Susceptibility Questionnaire (VIMSSQ, also known as the VIMSSQ-short). Visually Induced Motion Sickness Susceptibility Questionnaire (VIMSSQ, also known as the VIMSSQ-short). Simulator Sickness Questionnaire is a tool for assessing simulator sickness. It is used in post-exposure studies where symptoms such as nausea, confusion, and oculomotor disturbances are assessed. The SSQ is used in tests conducted on training simulators of all types of vehicles and aircraft. The SSQ is well received by test subjects, who rate it as not requiring much time and easy to understand [65–68]. Motion Sickness Assessment Questionnaire is a motion sickness assessment tool. It is used to assess responses to various aspects of motion sickness induced by VR. The MSAQ is a crucial tool for assessing motion sickness and is used in studies that evaluate the effects of motion sickness on sleep quality and task performance ratings [66,67]. Cybersickness Questionnaire is a tool for assessing digital illness, a side effect of virtual reality (VR) technology. The CSQ allows the assessment of cybersickness digital disease during VR exposure and uses the pupil size test, a biomarker of cybersickness digital disease. The CSQ has significantly better internal consistency than the SSQ and VRSQ, and CSQ scores have significantly better psychometric properties in detecting temporary performance decline caused by digital illness [69–73]. Visually Induced Motion Sickness Susceptibility Questionnaire is a tool for assessing visually induced motion sickness. It is used to assess individual susceptibility to visually induced motion sickness. The VIMSSQ has good psychometric properties and is a valuable addition to the MSAQ in predicting visually induced motion sickness [74]. CONCLUSIONS Critically analyzing the results of various studies related to simulator sickness and VR sickness, it is evident that while there are established tools and questionnaires for assessing and quantifying these phenomena, more comprehensive research is needed to understand the exact nature and underlying causes of simulator sickness. Additionally, the impact of simulator sickness on a wide range of users, including those with different levels of experience with virtual reality, needs to be further explored. The existing questionnaires and assessment tools provide valuable insights into the symptoms and manifestations of simulator sickness. However, standardized protocols and procedures for conducting susceptibility tests are needed to ensure consistency and reliability across different studies. It facilitates more accurate comparisons of results and enhances the generalizability of findings. Furthermore, including physiological measures such as ECG, EEG, and GSR examinations, in addition to subjective questionnaires, can provide a more comprehensive understanding of the psychophysical responses to virtual environments. This multidimensional approach can help identify individual differences in susceptibility to simulator sickness and contribute to developing targeted interventions and preventive measures. For instance, ECG can provide insights into heart rate variability, EEG can indicate changes in brain activity, and GSR can measure changes in skin conductance, all of which can be correlated with subjective reports of discomfort or sickness. It is also essential to consider the potential impact of task complexity, exposure duration, and the nature of the virtual environment (e.g., whether it is a game, a training simulation, or a therapeutic environment) on the onset and severity of simulator sickness. Understanding these contextual influences can aid in designing VR experiences that minimize the risk of simulator sickness while optimizing user engagement and performance. Moreover, the long-term effects of repeated exposure to virtual environments on susceptibility to simulator sickness need to be explored. It is particularly relevant in VR-based training and rehabilitation programs, where individuals may be exposed to virtual environments for extended periods. Longitudinal studies can provide valuable insights into the adaptive mechanisms of the human body, such as habituation or desensitization, and potential habituation effects over time, which can help inform the development of safer and more effective VR experiences. In conclusion, while existing research has shed light on the symptoms and correlates of simulator sickness, there is a pressing need for further investigations that delve into the underlying mechanisms, individual differences, and contextual factors associated with this phenomenon. By addressing these gaps in knowledge, researchers can contribute to developing safer and more effective virtual reality experiences for diverse user populations. Despite the large number of articles on the topic of simulator sickness and VR sickness, there is a legitimate need for increased work on the exact nature of this phenomenon and the qualifications of those immune to its impact. Given the high interest in using virtual reality in both entertainment and training and the low level of knowledge related to simulator disease among users, it seems necessary to conduct training that plays both an informative and preventive role. However, with the potential for advancements in virtual reality technology, it is possible look forward to the future where these issues are better understood and effectively addressed. From an analysis of various studies of the problem related to sensory conflict, it can be noted that all the psychophysical symptoms studied in connection with simulator sickness, VR sickness, or cybersickness are directly related to those indicating the presence of stress in the broadest sense. It is essential to consider whether the stress occurring in the subjects is related to the fear of failure to perform a given task or the process of the test being conducted. It seems that an indispensable element in correctly performed research related to susceptibility to simulator sickness is the psychophysical verification of the test subject before and after the tests (ECG, EEG, GSR examination) have conducted an in-depth analysis of the topic of simulator sickness and VR sickness. There is a legitimate need for increased research on the exact nature of this phenomenon and the qualifications of those immune to its impact. The interest in using virtual reality in entertainment and training is high, yet users need more knowledge of simulator sickness. Therefore, it seems necessary to conduct training that plays both an informative and preventive role. An analysis of various studies related to sensory conflict shows that all the psychophysical symptoms studied in connection with simulator sickness/VR or cybersickness are directly related to those indicating the presence of stress in the broadest sense. It is essential to consider whether the stress occurring in the subjects is related to the fear of failure to perform a given task or the process of the test being conducted. Prolonged exposure to the test causes fatigue and increases the subject's stress level, leading to ambiguity in the results. Therefore, a shorter but more complex, structured exposure to a light stimulus (moving image) may prove more accurate in assessing susceptibility to sensory conflict disease. In critically analyzing the results of various studies related to simulator sickness and VR sickness, it is evident that while there are established tools and questionnaires for assessing and quantifying these phenomena, more comprehensive research is needed to understand the exact nature and underlying causes of simulator sickness. Additionally, the impact of simulator sickness on a wide range of users, including those with different levels of experience and susceptibility, should be further explored. Furthermore, developing standardized protocols for simulator sickness susceptibility tests is crucial to ensure consistency and comparability across different studies. It helps establish reliable benchmarks for assessing and comparing individuals' susceptibility to simulator sickness. It is also important to consider the practical implications of simulator sickness, especially when using VR for training and entertainment. Understanding the factors contributing to simulator sickness and developing effective preventive measures is crucial in maximizing VR technology's potential while minimizing its adverse effects on users. It seems reasonable to analyze the use of electromyography (EMG) to verify muscle fatigue and, on this basis, determine the optimal duration of the test [75]. The prolonged exposure to the test causes fatigue and affects the subject with an increase in stress level, which, as a result, can lead to ambiguity in the results. A shorter but more complex, structured exposure to a light stimulus (moving image) may prove more accurate in assessing susceptibility to virtual reality sickness.