A virtual reality system for court interpreting education and its effects on motivation and fluency based on self determination theory Authors: Chan-in Sio, Andrew K F Cheung, Yao Yao, Rui Xie, Xian Wang, Chi Sun, Lik-Hang Lee Published: 2026 ABSTRACT AbstractCourt interpreting is critical for ensuring fairness in the justice system. However, traditional training methods often lack engagement and fail to provide interpreters with sufficient hands-on experience, relying heavily on passive learning approaches. To address these limitations, we designed and implemented MetaCourt, a virtual reality (VR) environment for court interpreting training. Using a small-scale participatory design approach, we developed an immersive platform that aligns with the Player Experience of Needs Satisfaction (PENS) model under self-determination theory. We recruited 21 undergraduate interpreting students to evaluate MetaCourt’s effectiveness. Results showed that the VR environment significantly enhanced fluency compared to traditional digital methods, while accuracy remained consistent. PENS analysis revealed high levels of autonomy, intuitive controls, and presence, supporting learners’ psychological needs. Additionally, NASA Task Load Index (NASA-TLX) and Igroup Presence Questionnaire (IPQ) surveys indicated reduced mental workload and acceptable user experiences, reinforcing MetaCourt’s usability. This study represents the first integration of virtual environments with the PENS paradigm in court interpreting training. The findings highlight MetaCourt’s potential to improve fluency, enhance virtual learning experiences, and address critical gaps in traditional training, offering a promising alternative for advancing court interpreting education.Keywords: Metaverse, Virtual Reality (VR) , Immersive learning , Self-Determination Theory (STD), Player Experience of Needs Satisfaction (PENS)Subject terms: Computer science, Information technology, Software Introduction With the advancement of global migration and the expansion of international trade, the linguistic diversity among court service users, including defendants, witnesses, and litigants, has significantly increased. The role of court interpreters is essential in ensuring equitable access for all parties and in safeguarding fairness within the justice system. In Hong Kong, the qualification for a court interpreter requires only an undergraduate degree, often in a non-relevant major, coupled with proficient language skills. Court interpreting encompasses both simultaneous and consecutive interpreting techniques, which are typically acquired through a passive learning approach, including shadowing and classroom exercises. However, practical trial experience is essential for developing interpreting proficiency. Due to limited training resources1, many novice court interpreters lack opportunities to gain sufficient courtroom experience.Traditional approaches like shadowing and classroom exercises often lack the realism and unpredictability of live courtrooms, leaving novice interpreters unprepared for the pressures of real cases. This insufficient preparation increases the risk of interpretation errors, which can lead to procedural delays, misunderstandings, or even miscarriages of justice where individuals are wrongfully convicted or denied fair hearings. Novice interpreters are often thrust into real court cases with minimal preparation, adopting a sink-or-swim approach that risks undermining the integrity of justice. This sink-or-swim approach places intense pressure on novices, forcing them to learn under high-stakes conditions without sufficient scaffolding. Such stress can undermine confidence, impair learning, and lead to uneven skill development, compromising both interpreter well-being and case outcomes. Errors in interpreting caused by inadequate training can have serious implications, potentially leading to miscarriages of justice. The need for innovative approaches to court interpreter training is increasingly recognized within both the general public and the legal profession2. Advances in educational technology, particularly virtual reality (VR), offer promising tools for enhancing interpreting and translation training3. However, applications of VR in legal contexts remain largely unexplored4. Existing literature on court interpreting predominantly emphasizes a listening-and-translating approach5,6, often neglecting the immersive potential of VR7. Despite the growing use of virtual environments, the integration of theory-based motivational models in court interpreting training is rare. This study employs self-determination theory (SDT)8 to examine how 3D VR environments meet psychological needs such as autonomy, competence, and relatedness9, satisfying these needs fosters long-term engagement and improved behavioral outcomes. By focusing on the intersection of VR and SDT, this work advances the understanding of how psychological factors influence court interpreting training in virtual settings. Specifically, the PENS model is used to assess how MetaCourt satisfies learners’ needs for autonomy (control over tasks), competence (sense of effectiveness), and relatedness (feeling of connection), which are critical for sustaining motivation, engagement, and skill development in virtual learning environments. To address these gaps, we developed MetaCourt, a VR system that replicates courtroom scenarios and provides immersive, interactive training for court interpreters, particularly novices. Using VR’s interactive and context-rich environment, MetaCourt enables learners to engage with courtroom dynamics, practice interpreting within realistic procedural settings, and experience the linguistic and procedural complexity of legal proceedings. An overview of the MetaCourt is demonstrated in Fig. 1. Figure 1 provides an overview of MetaCourt and the courtroom roles with their corresponding perspectives. The central image offers a bird’s-eye view of the courtroom layout: the upper section features the judge’s bench; the right side, marked by a pig avatar, represents the witness box; the middle section, with a zebra avatar, indicates the clerk’s bench, which also serves as the interpreter’s seat; the lower section shows the lawyers’ bench; and the left side, identified by a bear avatar, marks the jury bench. Photo 1 presents the user’s perspective from the judge’s seat, while Photo 2 shows the view from the jury bench. Photo 3 depicts the perspective from the witness box, and Photo 4 illustrates the view from the court clerk/interpreter’s seat. Finally, Photo 5 displays the interpreter’s screen, which simulates the presentation of courtroom exhibits. Participatory design workshops were conducted to identify specific user requirements, informing the development of cartoon-like avatars representing courtroom roles such as judges, lawyers, and witnesses. MetaCourt also simulates key criminal trial procedures, including the prosecution opening and examination-in-chief, enabling users to gain hands-on experience in court interpreting. An overview of MetaCourt and various roles and their point-of-view: (1) Judge; (2) Court Clerk, i.e., Interpreter; (3) Jury; (4) Witness; (5) Exhibit. Images were screen-captured from the MetaCourt environment developed in Unity (version 2022.3.3f1, https://unity.com/). Layout was composed using Canva for Mac https://www.canva.com/. This work presents a VR-based court interpreting training system designed to address the limitations of traditional methods. We investigated three questions: How do traditional and immersive environments influence interpreting performance? How do traditional and immersive environments influence interpreting performance? How do psychological needs satisfaction and cognitive workload differ across modes? How do psychological needs satisfaction and cognitive workload differ across modes? How does interaction with the virtual environment shape perceptions of presence and involvement? How does interaction with the virtual environment shape perceptions of presence and involvement? Using four measures: General Scoring Technique, PENS, NASA-TLX, and IPQ, we evaluated MetaCourt with 21 participants. Results show improved fluency, higher autonomy, lower cognitive workload, and stronger presence in VR compared to PC. Semi-structured interviews further indicated that MetaCourt enhances immersion and is especially beneficial for beginners, offering a supportive alternative for those unfamiliar with court interpreting procedures. Related work This section reviews the existing literature on the intersection of court interpreting, VR technology, and innovative teaching theories, elucidating key trends and research findings that inform our study. The growing importance of court interpreting Recent scholarship underscores the pivotal role of court interpreting in bridging linguistic and legal gapsand ensuring procedural fairness. Interpreters are essential for addressing legal discrepancies and safeguarding the right to a fair trial, especially for linguistic minorities. Li10 emphasizesthat precision and fairness ininterpretating are critical to uphold justice,while Slocum11 notes both potential and limitations of corpus linguisticsin clarifying legal meanings. Training challenges remain pronounced in this field. Novice interpreters often rely on passive learning approaches, such as shadowing and classroom exercises, which lack hands-on practice and fail to simulate the pressures of real courtroom work. This increases the risk of inaccuracies, which can have serious legal ramifications, including the potential for miscarriages of justice. Li10 advocates for the active participation of interpreters to ensure that justice is not only delivered but also perceived as equitable. Advances in court interpreting education include methodologies like discourse analysis, ethnography, pragmatics12, and action research13,which improve student outcomes and engagement. Klabal14 emphasizesthe importance of enhancing interpreting techniques to prepare students for thecomplex demands of legal proceedings.Technology-assisted learning, including computer-based simulation and remote interpreting platforms, is also gaining traction as a modern training tool. Historical reviews, such as those by Monteoliva-Garcia15 and Hale12, highlight ongoing challengesincluding insufficient regulation,limited certification standards and the undervaluation of interpreting professionals6,16. Hertog17 and Shlesinger18 stress the importance of maintaining high standards and expanding access to competent interpreting services, especially within the European Union. Comparative studies on statutory interpretation, including the Law19 and Safonov20 provide insights into the legal frameworks affecting interpreting practices. As global migration intensifies and remote interpreting becomes more prevalent21, trainingprogrammes must evolve to addressnot only linguistic proficiency, legal knowledge, and the growing prevalence of remote interpreting22,but also cultural competence and digital skills. Thisstudy aims to lay a foundation forintegrating virtual reality into court interpreting education to meet these emerging challenges. Education in virtual reality The integration of virtual reality (VR) in education has emerged as a transformative approach, offering enhancements to traditional teaching methods, greater student engagement, and improved critical thinking and problem-solving skills23,24.The immersive qualities of VR have proven beneficial in diverse fields such as medical education25, scientific experimentation26, and art instruction27, where interactivity and depth significantly enhance the learning experience28. However, the adoption of virtual reality in education faces notable challenges, including high costs, limited accessibility, and technical barriers29. Proposed solutions to mitigate these barriers include using affordable VR headsets, developing modular VR applications, and integrating VR as a complement to, it is not a replacement for traditional teaching methods. Despite these hurdles, research consistently highlights VR’s positive impact on learning outcomes and academic performance, making a strong case for overcoming these limitations30. In higher education, efforts have focused on optimizing VR design elements and incorporating learning theories to maximize their effectiveness, demonstrating the potential to revolutionize traditional education paradigms through dynamic and interactive experiences31,32. Specific sectors such as art, medical, and game education have shown promising results with VR integration, achieving advancements in instructional delivery and student receptivity33. Despite these advances, gaps remain in applying VR to specialized fields such as legal training and interpreter education, where research is still limited. Highlighting these gaps positions this study as one of the first to explore VR’s potential in court interpreting education. Given VR’s ability to create immersive, context-rich environments, its adaptation to court interpreting presents unique opportunities to address traditional training limitations. This study pioneers the exploration of VR in court interpreting education, aiming to leverage its transformative potential to enhance training methods and prepare interpreters for the complex demands of legal proceedings. Interpreting training: toward emerging teaching theory in VR Recent advances in the training of interpreters emphasize the complex and multidisciplinary nature of interpretation, supported by innovative teaching theories. Grieshofer34 underscores the need for structured methodologies to address interpretation’s intrinsic complexities effectively. Similarly, Lee35 highlights the application of philosophical hermeneutics, advocating for a classroom approach that embraces diverse perspectives and enriches the learning process. Abdel Latif36focuses on leveraging interpreting studies to enhance teaching strategies, aligning with broader educational theories that link effective pedagogy to improved learning outcomes. Korthagen37 introduces a paradigm in teacher education integrating theory and practice, while Roehe38 explores the dynamic interplay between teaching strategies and student learning. These contributions are complemented by Short39, who promotes inquiry-based learning, and Ackland40, who addresses practical challenges in implementing innovative pedagogy. To strengthen learner motivation and skill development, this study incorporates the Player Experience of Needs Satisfaction (PENS) model, a psychological framework rooted in Self-Determination Theory (SDT)41,42. SDT posits that autonomy (sense of control), competence (sense of mastery), and relatedness (sense of connection) are fundamental for fostering intrinsic motivation and sustained engagement, both critical in interpreter training contexts. The PENS model evaluates how immersive environments fulfill psychological needs (autonomy, competence, and relatedness) which are essential for motivation and engagement43. Validated across various scenarios44,45, the PENS model provides a robust framework for assessing VR-based educational tools9. In interpreter education, this framework helps explain how VR can create immersive, scaffolded learning environments that support gradual skill acquisition, reduce cognitive overload, and address interpretation complexities by offering repeated practice in realistic scenarios. This study bridges the gap in applying the PENS model to court interpreting training, exploring how VR can satisfy learners’ psychological needs while enhancing their interpretation skills. By evaluating the interplay between motivational theory, educational scaffolding, and practical implementation, this research contributes to the evolving field of interpreting education in virtual reality. Summary of related work Court interpreting plays a critical role in the justice system, yet its training faces challenges due to limited resources and outdated methodologies. Recent research highlights the importance of interpreters as linguistic and cultural mediators, emphasizing the need for innovative training approaches10. Advances in VR technology present opportunities to transform traditional training approaches by providing immersive and interactive environments, as demonstrated across various fields such as medical and scientific education25,26. Despite its potential, VR remains underutilized in legal education and interpreting training3. The PENS model, rooted in Self-Determination Theory, offers a framework for evaluating how VR supports autonomy, competence, and relatedness43,44. By applying this lens, this study shows how VR can enhance interpreting skills while meeting core psychological needs, positioning VR as a promising, learner-centered alternative to traditional training. System design and implementation Digital twins of court environment MetaCourt, a virtual reality (VR) environment, was developed to replicate real courtroom settings for court interpreting training. Inspired by the layout of a common law Magistrates’ Court in Hong Kong, the design aimed to provide an authentic learning experience. The environment was created using Unity (version 2022.3.3f1), with Meta Quest 3 serving as the hardware to display the immersive simulations. Participatory design During the design stage, a participatory design approach was adopted to align MetaCourt with user needs, ensuring the system met the evolving requirements of interpreting students. Five students, including undergraduate and postgraduate students, participated in a two-stage iterative process. Before joining the participatory design, all of them had observations of real criminal trials involving court interpreting. Initial focus group discussions and in-class observations identified preferences and learning needs, informing the development of interfaces and learning content. For example, Participant 1 requested that interpreting content be displayed directly in front of them while still being able to observe interactions among courtroom characters. Participant 5 emphasized the importance of having clear visual cues to guide interpreting procedures, noting that this would help learners quickly familiarize themselves with the standard workflow. Based on this feedback, we iteratively refined the user interface, improving visual prompts, navigational ease, and overall system responsiveness to better support learners’ cognitive and procedural needs. In the second iteration, students engaged with the refined environment, providing additional feedback for improvements. Wireframe sketches were used to optimize interface design. This iterative methodology ensured MetaCourt aligned with learners’ expectations and enhanced usability. Design highlights of MetaCourt In MetaCourt, courtroom scenarios and roles are recreated to reflect real participants in legal proceedings. Users may select different roles and learning scenarios. Figure 2 illustrates the user interface for entering the courtroom. The roles include the judge, court clerk, witness, prosecutor, defense counsel, defendant, jury, and members of the public. Each character has its own point of view in the courtroom, providing participants with a fully immersive experience. The starting procedures: (1) Choose Offline Mode (2) Choose Judge as Character (3) Choose Scene Plea and Sentence (4) Enter MetaCourt’s virtual space. A key design choice was the decision to represent characters using cartoon-style heads (Fig. 3(L)), instead of realistic human avatars. Feedback from a previous participatory study46,47, in which participants indicated that realistic human characters increased stress levels during the learning process, influenced this decision. The cartoon characters help reduce this stress while preserving the authoritative and professional nature of the courtroom environment. This balance allows students to focus on interpreting without feeling overwhelmed by the intensity of the simulation48,49. (Fig. 3L) Cartoon-style heads:(1) Judge-Elephant; (2) Court Clerk/Court Interpretor-Zebra (3) Jury-Dog (4) Audience-Cat (5) Defence Counsel-Fox (6) Witness-Pig (7) Defendant-Wolf (8) Prosecutor-Panda, and (Fig. 3R) Witness’s Point of View & displaying the trial’s transcript. Learning content for court interpreting In the pre-experiment lecture for MetaCourt, participants were introduced to the structure and process of a Hong Kong criminal court through the use of a Lego court model, which visually represented the courtroom setting. The roles of key courtroom figures were explained in detail. We also introduced the criminal trial procedure, covering phases such as the prosecution’s opening, examination-in-chief, and cross-examination. Special emphasis was placed on the role and significance of court interpreters, highlighting the differences between court interpreting and other forms of interpretation. Additionally, participants were familiarized with the glossary of legal terms that would appear in MetaCourt, ensuring they understood terms they might not be familiar with. Finally, for those with no prior VR experience, we provided instructions on how to use the VR headset and navigate the MetaCourt environment using the provided controls. The VR environment incorporates pre-recorded courtroom proceedings, allowing learners to listen to and interpret real-life cases (Fig. 3(R)). The proceedings used in this work are from a criminal trial in Hong Kong involving a drug-related charge. In this case, the defendant is accused of possessing dangerous drugs–specifically, a pack of cigarettes containing cannabis, which is illegal in Hong Kong. It was chosen due to its serious nature and impact on society. The portion of the trial included in the simulation focuses on the examination-in-chief of the defendant, during which the defense counsel questions the defendant about the events at the material time and why he claims innocence. This segment of the trial was chosen to give learners a challenging and realistic task for interpretation. The learners can obtain hands-on experience by performing interpretation in an accurate and fluent manner. The pre-recorded proceedings are controlled using a simple interaction mechanism: participants press the “X” button on the Meta Quest controller to advance to the next sentence or press “Y” to return to the previous sentence. This interactive feature allows participants to control the pace of the trial, enabling them to practice interpreting without feeling rushed. The pre-recorded dialogue features a mix of languages, with the defense counsel’s questions spoken in Chinese and the defendant’s responses given in English. To further challenge participants, the system utilizes text-to-speech technology to generate the voices, employing a normal Chinese accent for the defense counsel and an Indian English accent for the defendant. This choice was made to increase the difficulty of the interpreting task, simulating real-world conditions where interpreters often work with speakers of different accents and linguistic backgrounds. Table 1 shows the first part of the interpreting task. The Defence Lawyer questioned in Chinese, and here is the suggested English interpretation. The Defendant answered in English. In the experiment, participants were required to interpret the answers into Chinese. Meanwhile, the participants were given a glossary of the phrases that were not common but important in the trial. Table 1Interpreting materials used in both the MetaCourt and PC setups: A case of criminal trial.PartiesQuestions and answersDefense Lawyer (English translation)Witness, please introduce yourself briefly.Defendant:My name is Amir Kan, I am a secondary school student in Hong Kong Secondary School.Defense Lawyer (English translation)How old are you this year? Which grade are you currently in?DefendantI am now 18. I am in form 6, I am taking the DSE exam this year.Defense Lawyer (English translation)Do you have the habit of smoking?DefendantNot very often. Sometimes when I feel stressful, especially when preparing for the exam, I will smoke.Defense Lawyer (English translation)Is the cigarette you smoke a normal one? Where do you get it?DefendantI smoke hand-rolled cigarettes. I usually buy the cut tobacco and rolling paper from a store near my home.Defense Lawyer (English translation)Do you smoke marijuana?DefendantNo. I tried once in Thailand and I felt very sick. So I think it is not my cup of tea.Defense Lawyer (English translation)I will now switch topics. On May 1, 2024, where were you?DefendantI finished all my DSE exams. And therefore I went to a bar in Tsim Sha Tsui to drink with my friends to relax and celebrate.Defense Lawyer (English translation)Did anything special happen that day?DefendantA group of police came to the bar and checked everyone’s ID. I gave my ID card to a policeman.Defense Lawyer (English translation)Witness, please slow down a bit. Okay, continue.DefendantThen the policeman asked me whether I smoke, I said yes. And he further asked me which brand I usually smoke, I said Hong Kong Panda.Defense Lawyer (English translation)What happened next?DefendantThe policeman took something from the sofa where I sat. It was a box of Hong Kong Panda. He sniffed it and then showed a weird face.Defense Lawyer (English translation)And then?DefendantHe showed me the cigarette inside, it was a hand-rolled cigarette. But it is not one I rolled.Defense Lawyer (English translation)What did the policeman do after that?DefendantHe suddenly said he would arrest me for possession of weed. I was so surprised because that box did not belong to me.Defense Lawyer (English translation)Did you tell him?DefendantYes, of course. I told him right away. But he did not believe me and then he took me to the police station.Defense Lawyer (English translation)Do you know whose hand-rolled cigarettes those were?DefendantI don’t know. Actually, I had just arrived at the bar, and I saw some other people sitting there. I really don’t know.Defense Lawyer (English translation)Alright. I have no further questions. Please remain seated, the prosecutor will cross-examine you. Interpreting materials used in both the MetaCourt and PC setups: A case of criminal trial. Methods Participants and apparatus Twenty-one participants (20 females, 1 male, aged 21–25; = 21.47, SD = 1.167) from an undergraduate Chinese/English interpreting class at a university in China were recruited for the study. Among them, 5 had prior VR experience, and 10 had basic legal knowledge acquired through school lectures or social media. All participants passed China’s College English Test Band 4 (CET-4), with 15 also achieving CET-6, demonstrating high English proficiency. Participants were compensated with a 20 RMB gift card for completing the experiment and surveys. The experiments were conducted in a quiet laboratory environment. Participants used noise-canceling headphones during VR and PC-based tests to minimize distractions. Participants were divided into two groups: the experimental group used VR, and the control group used personal computers (PC). The VR setup employed Meta Quest 3 headsets, providing participants with an immersive experience, while the control test used desktop computers, replicating traditional audio-based interpreting methods. Both conditions followed identical pre-recorded courtroom proceedings. The sample size of 21 participants in this study is consistent with prior VR-based educational research, where sample sizes typically range from 10 to 40 participants depending on study objectives50 and a median of approximately 23 participants has been reported across studies51. While our sample size aligns with these precedents, we acknowledge that the small size may limit statistical power. Future studies should aim to expand participant recruitment, potentially through multi-institutional collaboration, to improve the generalizability and robustness of findings. The participant sample in this study was predominantly female (20 females and 1 male), which may limit the generalizability of the findings across genders. Prior research has identified gender differences in VR learning environments, with males often reporting stronger spatial presence and involvement52,53, while females have shown higher engagement, particularly when virtual agents are integrated into the narrative54. Future research should aim to recruit more gender-balanced samples to investigate whether these gender-related differences influence learning outcomes and engagement in VR-based interpreter training. Experimental procedure In the VR condition, participants used the Meta Quest 3 headset to log into MetaCourt as court clerks performing interpreting tasks. Within the virtual courtroom, they listened to pre-recorded courtroom proceedings from a mock criminal trial. Participants interpreted Chinese questions into English and translated English answers into Chinese. Each question and answer was played only once, and there was no time limit for interpreting and simulating real-time courtroom demands. The duration is around 15 minutes for each participant. In the control condition, participants completed identical tasks in a standard computer lab setup. Using headsets and PC, they interpreted identical pre-recorded courtroom proceedings. There was no time limit for translation; the duration of the control experiment was around 12 minutes for each participant. A Latin Square design was used to counteract the learning effects between the two conditions. Participants were divided into two groups: one started with the VR condition, while the other began with the control condition. Both groups experienced the same pre-recorded content, ensuring consistent task conditions across both setups. Measurements To evaluate the effectiveness of MetaCourt in court interpreting training, we employed five distinct measurement methods, integrating both quantitative and qualitative approaches. First, participants’ interpreting performances in VR and PC environments were recorded and evaluated by a professional court interpreter using the General Scoring technique55,56. This method provided scores out of 100 points for two key metrics: accuracy and fluency. Accuracy was assessed based on the correct use of legal terminology, the absence of mistranslations, and fidelity to the source message. Fluency was evaluated by examining the smoothness of delivery, coherence of interpreted speech, and appropriate pacing. These detailed scoring criteria allowed for a meaningful comparison of participants’ interpreting skills across the immersive VR setting and the traditional PC-based condition, providing insight into the practical impact of MetaCourt on performance outcomes. Second, participants completed the Player Experience of Needs Satisfaction (PENS) questionnaire, the NASA Task Load Index (NASA-TLX), and the Igroup Presence Questionnaire (IPQ). All questionnaires were presented in a bilingual format (Chinese and English) on the same form, with Chinese translations provided for reference. Importantly, no modifications were made to the original items or scoring procedures of any of the scales. The Player Experience of Needs Satisfaction (PENS) questionnaire, rooted in Self-Determination Theory (SDT)43, was used to measure how well MetaCourt satisfied participants’ psychological needs. This adapted, bilingual questionnaire employed a 7-point scale to assess autonomy, competence, relatedness, presence, and intuitive controls: Autonomy assessed participants’ control and freedom in managing tasks within MetaCourt. Autonomy assessed participants’ control and freedom in managing tasks within MetaCourt. Competence reflected their sense of mastery over interpreting tasks. Competence reflected their sense of mastery over interpreting tasks. Relatedness captured their feelings of connection to the virtual courtroom and its characters. Relatedness captured their feelings of connection to the virtual courtroom and its characters. Presence measured immersion, enhanced by realistic visuals and interactions. Presence measured immersion, enhanced by realistic visuals and interactions. Intuitive controls evaluated the ease of navigating the virtual environment, ensuring task focus. Intuitive controls evaluated the ease of navigating the virtual environment, ensuring task focus. The NASA Task Load Index (NASA-TLX) was administered to assess cognitive workload in both VR and PC tasks. This tool measured mental demand, physical effort, and frustration, providing insights into the cognitive challenges posed by each environment. The Igroup Presence Questionnaire (IPQ) was used to evaluate participants’ sense of presence within MetaCourt after the VR task. As a standard metric for VR settings, the IPQ provided insights into the immersive quality of MetaCourt as a training tool. Finally, semi-structured interviews were conducted with 9 participants to gather qualitative feedback. These interviews explored immersive experience, visual design, user comfort, educational value, and court interpreting insights. The experiment was scheduled for a single day, and the VR sessions exceeded their planned duration; therefore, only participants who were available and willing to stay after completing the experiment took part in the interviews. Although this smaller subset limited the breadth of qualitative data, it provided valuable insights into user experiences and perceptions of MetaCourt. Collectively, these measurement methods offered a comprehensive framework for evaluating MetaCourt, integrating objective performance metrics with subjective user feedback to assess its educational effectiveness. All methods were carried out in accordance with relevant guidelines and regulations. The experimental protocol was approved by the Hong Kong Polytechnic University Institutional Review Board, approval number HSEARS20240109002. Informed consent was obtained from all participants prior to their involvement in the study. Results and implications In this experiment, five different dimensions were used to measure the effectiveness of MetaCourt in enhancing court interpreting education. These methods included assessing participants’ interpreting performance in both the VR and PC environments for accuracy and fluency, administering the PENS questionnaire to evaluate psychological needs satisfaction, using the NASA-TLX to gauge cognitive workload, utilizing the IPQ to measure the sense of presence in the VR setting, and conducting semi-structured interviews to gather qualitative feedback. This section presents the data collected from these various measures. Statistical analyses for this study were performed using R, version 4.2.0. Accuracy and fluency in interpreting performance In the VR-PC comparison study, we aimed to evaluate user performance in both environments based on two key metrics: accuracy and fluency. The interpreting records from the VR and PC groups were anonymized and sent to a professional court interpreter for evaluation. Each participant’s work was assessed, and scores were assigned based on the accuracy of the interpreted content and the fluency of the delivery. Once evaluated, the records were de-anonymized and categorized back into their respective groups–VR and PC–for further analysis. We ran a paired t-test for two metrics separately. No statistical difference exists between the two environments (VR-PC) for accuracy scores (, ). In contrast, a significant effect of environments was found for fluency (, ), implying a significant improvement in fluency in the VR environment compared to PC. Implications One of the key objectives was to determine whether the VR environment could improve participants’ accuracy and fluency in interpreting tasks. The results revealed no significant difference in accuracy between VR and PC settings, likely due to the fact that accuracy improvements require stronger foundational language skills57, which are not easily improved in an hour-long experiment. However, fluency scores were notably better in the VR environment compared to the PC. This improvement can be attributed to the feedback cues (e.g., visual and audio) in the MetaCourt environment. The cognitive load differences might reinforce this interpretation, as evidenced by the NASA-TLX results. Participants experienced lower mental demand in VR than in the PC, which allowed for smoother and more fluent interpreting performance in the immersive environment. Psychological Needs Satisfaction (PENS) The 20-question PENS questionnaire is divided into five dimensions, with each dimension comprising several sub-dimensions (see Fig. 4(L)): Autonomy (3Qs) – VR: = 5.57, SD = 1.44; PC: = 4.52, SD = 1.48; Competence (3Qs) – VR: = 4.21, SD = 1.44; PC: = 5.09, SD = 1.44; Relatedness (3Qs) – VR: = 4.53, SD = 1.63; PC: = 4.44, SD = 1.73; Presence (9Qs) – VR: = 4.65, SD = 1.54; PC: = 4.15, SD = 1.41; and Intuitive Controls (3Qs) – VR: = 4.2, SD = 1.38; PC: = 4.29, SD = 1.14. A paired samples t-test was conducted to determine the effect of different technologies on PENS value. The results indicate a significant difference in Autonomy and Competence sub-dimension. The results show that VR significantly improves Autonomy (, ), but PC is significantly more effective in Competence (, ). (Fig.4L) Box plot of PENS questionnaire. (Note: *** marked at Autonomy and Competence means and (Fig. 4R) PENS Pearson Correlation Heatmap for MetaCourt. Furthermore, Fig. 4(R) depicts the Pearson correlation coefficients between the PENS dimensions, providing further insight for the MetaCourt condition, with the following three highlights. First, Autonomy shows a strong positive correlation with both Presence (0.80) and Intuitive Controls (0.82), indicating that when participants felt more autonomous, they also experienced a higher sense of immersion and found the controls more intuitive. Second, Competence is moderately correlated with Autonomy (0.45), Presence (0.51), and Relatedness (0.47), suggesting a relationship between participants’ feelings of competence and their experiences of control, immersion, and connection. Third, Presence and Intuitive Controls are highly correlated (0.88), indicating that the more intuitive the controls were, the more immersed participants felt in the VR environment. The strong correlations between Autonomy, Presence, and Intuitive Controls suggest that improving one of these areas could positively impact the others. In contrast, Competence and Relatedness show weaker relationships, indicating that these areas may need separate attention for improvement. Implications: The PENS test examined whether MetaCourt can satisfy the psychological needs of participants based on Self-Determination Theory (SDT). This theory highlights three essential needs: autonomy, competence, and relatedness, which are critical for motivation and well-being9. The results showed that autonomy was the most positively rated need, with participants feeling a high level of control and choice during their tasks. Competence scores were moderate in VR, indicating that while participants felt capable, there was variation, some found the tasks either too easy or too challenging. It is important to note that Competence scores were higher in the PC condition, which may reflect participants’ prior familiarity with PC-based interpreting–a common and traditional training format. Relatedness scores, albeit with the absence of statistical significance, were also moderate, suggesting that while participants felt somewhat connected to the courtroom environment and characters, there is room for improvement in creating a stronger sense of social presence. NASA-TLX The modified NASA Task Load Index58 was administered to measure the participants’ perceived workload under two conditions: VR-assisted mode (VR) vs Traditional mode (PC). A paired t-test was conducted with mode (VR vs. PC) as the independent variables and the six NASA-TLX dimensions, i.e. Mental Demand (VR: = 3.33, SD = 1.39 vs PC = 2.09, SD = 1.34), Physical Demand (VR: = 3.19, SD = 1.29 vs PC = 2.95, SD = 1.36), Performance (VR: = 3.24, SD = 0.94 vs PC = 3.23, SD = 0.94), Temporal Demand (VR: = 3.14, SD = 0.96 vs PC = 2.29, SD = 1.27), Effort (VR: = 3.33, SD = 1.24 vs PC = 3.296, SD = 1.34) and Frustration (VR: = 2.24, SD = 1.51 vs PC = 2.05, SD = 1.28), as dependent variables. The results indicate that mental demand was significantly lower in the VR mode compared to the PC mode (, ), suggesting that participants perceived the VR mode as less mentally taxing. Also, participants reported a significantly higher sense of success in completing the task in the VR mode than in the PC mode (, ). No significant differences were found between the two modes in terms of effort, frustration, interpreting pace, and physical demand (Fig. 5(L)). (Fig. 5L)NASA-TLX: Self-perceived Rating across Dimensions(L) (Note: * marked at Mental Demand and Performance means and (Fig. 5R) IPQ Questionnaire Results: G1 = Sense of Being There; INV = Involvement; REAL = Experienced Realism; SP = Spatial Presence. Implications: The NASA-TLX results indicate that users still perceived their performance positively in the MetaCourt environment, despite VR imposing higher mental demands compared to PC. The lack of significant differences in physical demand, temporal demand, effort, and frustration suggests that VR is not more burdensome than traditional PC environments in these areas. This implies that VR can be seamlessly integrated into various workflows without additional strain, making it a usable platform for tasks requiring high interaction and immersion. However, careful task design is needed to balance the cognitive load and optimize user performance in VR environments. Igroup Presence Questionnaire (IPQ) We leverage IPQ to understand the performance of the MetaCourt condition, but the control condition is omitted due to the nature as an audio-based learning module. As depicted in (Fig. 5(R)), we report the four dimensions of the IPQ, with mean and standard values: Sense of being there ( = 3.76, SD = 1.26), Spatial presence ( = 2.95, SD = 1.41), Involvement ( = 3.12, SD = 1.20), and Experienced realism ( = 3.48, SD = 0.99). Implications: The IPQ results indicate that participants experience varying levels of presence in virtual environments, with some categories being more pronounced. The significant differences across categories suggest that elements such as sense of presence (G1) and involvement (INV) are perceived differently by participants. Since a high sense of presence is crucial for creating immersive and engaging virtual experiences, the findings further imply that the MetaCourt design should focus on maintaining a decent level of sense of presence. At the same time, improving other three factors (e.g., user involvement) can potentially lead to greater user satisfaction (i.e., these dimensions can improve from 4 to 5 point). After-task semi-structured interview The post-experiment interview results reveal several important insights into participants’ experiences with VR technology in court interpreting learning. A majority of participants expressed high satisfaction with the learning experience, noting that while VR effectively helped them understand court interpreting procedures, its impact on improving their interpreting skills was less significant. P4 reflected: “The MetaCourt is suitable for beginners like us, and I feel I was situated in a court environment. The use of VR technology can replicate genuine courtroom situations, therefore creating a sense of immersion and suspense like that of a game” Also, P3 made an aligned comment: “MetaCourt helps with learning court interpreting procedures. I perceived this VR setting as more appropriate for novices and found it potentially conducive to proficiently kick-starting their interpreting abilities. I currently see MetaCourt has a limited impact on improving interpreting skills.” On the other hand, the immersive environment created by VR elicited a sense of both nervousness and realism, closely resembling real-world scenarios, yet it was also described as engaging and relaxing. P7 mentioned: “The MetaCourt provides participants with a sense of having a companion during practice.” P5 also said: “I like the animal avatar in MetaCourt that provides a relaxed atmosphere, and I feel enjoyable in learning.” A number ofparticipants found the experience beneficial for beginners and appreciated the sense of having a companion during practice, which made the learning process feel more supportive. However, participants also suggested that introducing a wider variety of accents could increase the difficulty level, making the experience more reflective of the diverse linguistic challenges faced in real court interpreting. P8 emphasized: “textitIt would be advantageous to incorporate a diverse range of accents (e.g., USA and South Europe) and other practical circumstances, such as business negotiation. The above findings imply that while VR-based MetaCourt aids in learning court interpreting procedures, further enhancements are needed to improve its effectiveness in developing interpreting skills for diversified scenarios. Discussion This study aimed to investigate the use of VR to enhance court interpreting education, a field where simulated training is often lacking. By designing a VR-based court environment, MetaCourt, we sought to provide interpreting students with a more immersive and engaging learning experience. 21 participants, although it is a small participant number, were involved in the study, and they performed court interpreting tasks within the MetaCourt setting. The results suggest that VR offers specific benefits for learning court procedures, supporting findings from prior research that demonstrate VR’s potential to assist learning through immersive environments59–61. Importance of court interpreting Court interpreting plays a critical role in ensuring fairness and accessibility in legal proceedings, making it an essential component of the judicial system. Effective training for court interpreters is crucial but often difficult to achieve effectively in traditional educational settings. The inadequate practical training in university interpreting courses presents considerable obstacles. A significant number of university interpreting students encounter inadequate training for the stringent requirements of court interpreting. Further, interpreters often fail to maintain the pragmatic force and illocutionary effect of the original utterances, instead focusing only on the propositional content62. MetaCourt was designed to address this gap by providing an immersive, interactive environment where learners can practice and refine their interpreting skills. The primary aim of this study was to test whether MetaCourt could assist in improving court interpreting through five distinct measurement methods. This comprehensive approach allowed for both objective and subjective evaluation of its effectiveness. Fluency improvement, cognitive load reduction, and participant feedback Participants demonstrated improved fluency in the VR environment compared to the traditional PC setting, highlighting MetaCourt’s potential to enhance fluency.In this study, accuracy was defined as the correct use of legal terminology and faithful transfer of propositional meaning without mistranslations, while fluency referred to the smoothness, coherence, and natural pacing of delivery. For example, accurately interpreting a complex legal phrase like ‘cross-examination’ into its correct Chinese legal equivalent without hesitation reflected both dimensions.This improvement is likely due to the reduced cognitive load in VR, as supported by NASA-TLX results showing lower mental demand in VR tasks. Post-task interviews further revealed positive participant feedback, with many praising the immersive quality of MetaCourt for aiding understanding of court procedures. These findings align with prior research suggesting that reduced cognitive load and engaging learning environments, like VR, foster better performance and motivation63–65. While fluency improved significantly in the VR condition, accuracy did not show notable gains. This may be because fluency benefits more immediately from reduced cognitive load and immersive cues, whereas accuracy relies on deeper foundational knowledge, such as legal terminology, cultural competence, and stress management–areas that require sustained practice over time. The discrepancy between overwhelmingly positive qualitative feedback and more moderate quantitative outcomes suggests that learners’ subjective satisfaction may not always align with measurable performance gains. Future work should combine self-reports, performance analytics, and biometric data to better understand this gap and refine instructional design. Self-determination theory and learning motivation This study examined learning motivation in MetaCourt through Self-Determination Theory (SDT). The PENS results revealed autonomy as the highest-rated psychological need, indicating participants felt a sense of control and freedom in the VR environment, which are the key factors linked to increased motivation and engagement. This aligns with recent studies emphasizing autonomy, competence, and relatedness as critical drivers of learner satisfaction in virtual settings9,43. Given the lack of formal training programs for court interpreters in many jurisdictions, integrating SDT principles into VR education offers a promising approach to enhancing motivation and learning outcomes.These findings have important theoretical implications, suggesting that VR environments that satisfy basic psychological needs can foster motivation, persistence, and skill development among novice interpreters. Integrating SDT-based strategies in interpreter education could thus provide a scalable framework to enhance learner engagement and resilience. Design effectiveness and immersion The results from the NASA-TLX and IPQ suggest that MetaCourt is reasonably effective for supporting learning. Participants reported moderate levels of immersion and manageable cognitive load, indicating that MetaCourt effectively balances engagement with mental demand. These results are consistent with prior research highlighting the importance of presence and intuitive controls in VR-based training systems66–68. Improving certain aspects of the visual and auditory elements could further enhance the sense of presence, as suggested by participants during the interviews.The moderate presence scores suggest that while participants felt reasonably immersed, further design refinements, such as enhancing real-time feedback, adding collaborative interpreting tasks, and introducing more challenging scenarios, could strengthen engagement and learning outcomes. An important limitation of this study is the reliance on self-reported measures for cognitive load and presence. Future research could incorporate objective measures, such as EEG recordings or physiological tracking, to complement subjective data and offer a more comprehensive assessment of learner experience. Triangulating self-reports with behavioral and biometric data would strengthen the validity of findings. Generalizability of the VR interpreting education This study highlights the effectiveness of MetaCourt in enhancing court interpreting training through immersive simulations, promoting autonomy, competence, and relatedness while ensuring cognitive and physical comfort66,69. Its success indicates potential applications in other high-stakes fields like medical, diplomatic, and conference interpreting, where precise communication is critical, and errors carry significant consequences70,71. The realism offered by VR surpasses traditional methods such as textbooks or recordings72, making it a versatile training tool. Moreover, VR’s flexibility enables adaptation to diverse languages, accents, and cultural nuances, broadening its applicability across legal and non-legal contexts73. Adaptive features can tailor difficulty to individual learners’ skill levels, providing continuous and meaningful challenges74. Collaborative elements could allow multiple users to engage in team-based scenarios, mirroring real-world legal settings. Expanding VR applications to areas like legal negotiations and mediation could further demonstrate its value in specialized interpreter training. Limitations and conclusion This study represents a first step in applying VR technology to court interpreting education, but several limitations must be acknowledged. First, the sample size was relatively small (n = 21) with a clear gender imbalance (20 females, 1 male), limiting the generalizability of the results. Future studies should recruit larger, more diverse participant groups to improve statistical power and representativeness. Second, participants were drawn exclusively from an interpreting course, introducing potential selection bias, as they may not reflect the broader population of novice or professional court interpreters. Broader recruitment strategies, such as involving participants from professional training programs or certified interpreter pools, are recommended. Third, the short duration of the experimental intervention limits the ability to assess long-term learning outcomes. Future research should incorporate longitudinal designs to examine skill retention and transfer over time. Fourth, the pre-experiment preparation, i.e. a brief lecture and glossary, may not fully capture the complexities of real-world interpreter training. Extending the training duration, providing more realistic preparatory materials, or including participants with prior court interpreting experience would improve ecological validity. Finally, expanding MetaCourt to incorporate diverse linguistic challenges (e.g., multiple accents), enhancing interface design, and adding collaborative learning features are important next steps to improve engagement and training effectiveness. The importance of court interpreting is underscored by its recognition in Article 14(3) of the International Covenant on Civil and Political Rights (ICCPR)75, which emphasize the right to an interpreter in ensuring justice. Traditional training methods often fall short in preparing interpreters for the initial complexities of courtroom work. MetaCourt demonstrates that VR can serve as a valuable introductory platform, particularly for beginners, by providing an immersive environment to practice basic procedures and develop early-stage fluency. While MetaCourt does not yet replicate the full complexity of real-world trials, this study establishes a foundation for integrating VR into interpreter education. Future developments may extend its capability toward more advanced, procedurally complex learning tasks. Acknowledgements The authors acknowledge Mr Alan Tang, Mr Ifan Cheung, Ms Lawrence Diao, Ms Kaifusai Julaiti and Ms Virginia Leung for their assistance with the experiment in this project. Author contributions C. SIO wrote the main manuscript text, conceived the experiment and prepared the figures and tables; Y. YAO, R. XIE and C. SIO conducted the experiment and conducted the data analysis; X. WANG, C. SUN and C. SIO developed the systems; A. CHEUNG and L. LEE supervised the research project; all authors reviewed the manuscript. Funding This research is supported by the Hong Kong Polytechnic University’s Start-up Fund for New Recruits (Project ID: P0046056) and a collaborative research project (Project ID: P0047685). Data availability The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.