The effect of training methods on newly graduated physical therapists in Japan: A study of on-the-job and off-the-job training Authors: Tetsurou Watari, Soichiro Koyama, Ikuo Motoya, Yoshikiyo Kanada, Hiroaki Sakurai Published: 2025 ABSTRACT BACKGROUND:Newly graduated physical therapists require effective training to enhance their clinical skills. On-the-job training (OJT) and off-the-job training (Off-JT) are commonly used methods, but their comparative effectiveness remains unclear. This study aimed to compare the effectiveness of OJT and Off-JT in improving the clinical competencies of newly graduated physical therapists.MATERIAL AND METHODS:A crossover study was conducted involving 13 newly graduated physical therapists. Participants alternated between OJT and Off-JT interventions. The objective-structured clinical examination assessed four key skills—joint range of motion measurement, muscle testing, assistance in getting up, and assistance in transferring. Evaluations were conducted at four time points. A Mann–Whitney U-test was used to compare the change in average scores between the two groups from Assessment 1 to Assessment 2. A two-way analysis of variance was conducted to analyze the interaction effects of training methods and assessment timing.RESULTS:Off-JT was more effective than OJT in improving clinical skills, especially in muscle testing and patient assistance tasks. Both training methods enhanced joint range of motion measurement skills; however, no significant differences were found between the groups. Some tasks showed performance declines in the final assessment, indicating the necessity of continued practice.CONCLUSIONS:Off-JT provides a more effective approach for improving clinical skills, particularly for tasks that require specialized knowledge and structured learning. However, OJT remains valuable for reinforcing routine clinical tasks. Future research should explore the long-term effects of these training methods on various clinical competencies. Introduction Exploring workplace mentorship for newly graduated physical therapists presents a significant challenge. Wells et al. reported that newly graduated physical therapists are perceived to bring research-informed and contemporary knowledge but struggle with clinical reasoning in diagnosis, assessment, and treatment.[1] Similarly, Martin et al. indicated that these therapists require training in practical skills, including physical assessments, effective communication, treatment methods, and clinical reasoning.[2] Black et al. reported that the clinical environment has a significant impact on the early career development of newly graduated physical therapists. Particularly, a lack of mentoring and educational programs in the workplace inhibits newcomers’ clinical skill development and professional confidence.[3] Notably, environments without adequate support limit learning opportunities and make it more difficult to adjust to the workplace, resulting in an increased risk of disengagement. To maintain and enhance their clinical expertise, ongoing postgraduate education is indispensable for physical therapists. Educational activities that incorporate active learning elements—such as practical feedback, peer assessment, and individualized mentoring during patient interactions—have been reported.[45] Moreover, the concept of the “master adaptive learner,” which emphasizes a systematic process of enhancing clinical capabilities through four stages—planning, learning, assessment, and adjustment—provides a theoretical framework that underscores the importance of fostering an environment supportive of experiential learning, reflection, and structured feedback rather than merely transmitting information.[46] Furthermore, when interactive sessions and structured feedback are integrated into various educational modalities, whether face-to-face, blended, or online, not only is knowledge acquisition enhanced, but improvements in clinical behavior and, in some cases, patient outcomes are also observed.[78] Collaborative learning environments, grounded in constructivist and experiential learning theories, are also recognized as critical for promoting peer-to-peer interaction and deepening learning.[9] In Japan, on-the-job training (OJT) and off-the-job training (Off-JT) are utilized as workplace mentoring methods for newly graduated physical therapists.[10111213] OJT is mainly responsible for skill improvement through practical work and contributes to the development of skills that can be immediately used in the field, such as “environmental coordination” and “multidisciplinary cooperation.” On the one hand, Off-JT supplements systematic knowledge, enhances clinical reasoning, and effectively boosts motivation for learning in the early stages of training. Furthermore, it has been reported that the combination of OJT and Off-JT improves the clinical competence assessment score (i.e. CEPT) of newcomers, with this effect being particularly pronounced in the first and second years. On the other hand, OJT has some disadvantages; instructors need to have sufficient knowledge to train newly graduated physical therapists, resulting in variations in instructional content. Additionally, it is associated with high human and time costs.[14] Previous studies have reported the effects of OJT and Off-JT separately; however, comparative studies on their educational outcomes in training newly graduated physical therapists within the workplace are lacking. Unlike these studies, which focused on either OJT or Off-JT independently, this study directly compares both methods in a controlled crossover design, providing new insights into their relative effectiveness in clinical skill acquisition and investigating which of the two—OJT or Off-JT—is more effective for educating newly graduated physical therapists and enhancing their clinical skills. Materials and Methods Study design and setting This study employed a crossover design to compare the effectiveness of OJT and Off-JT in enhancing the clinical skills of newly graduated physical therapists. Study participants and sampling The study included 13 newly graduated physical therapists with a mean age of 21.6 years (±0.5 standard deviation [SD]). Participants were randomly assigned to two groups (Group 1: n = 6; Group 2: n = 7). Each group alternated between OJT and Off-JT training in a crossover design. Participants received detailed information about the study and provided written informed consent in accordance with the Declaration of Helsinki. Data collection tool and technique Clinical competencies were evaluated using the objective-structured clinical examination (OSCE), specifically designed for physical therapists.[1516] The OSCE assesses both attitude (therapist’s demeanor) and skills (appropriate techniques). Each clinical task is evaluated using a three-point scale: good (2 points), fair (1 point), and poor (0 points), with a 5-min time limit per task. The OSCE covers four key clinical skills essential for physical therapists: joint range of motion measurement, muscle testing methods, assistance in getting up, and assistance in transferring. These skills were selected based on their relevance to routine clinical practice in physical therapy. Experimental procedure The evaluation process involved trained raters and a standardized simulated patient to ensure assessment reliability. The raters were two experienced physical therapists, each with an average of 10.5 ± 0.7 years of clinical experience, while the simulated patient was also a licensed physical therapist with four years of clinical experience. To enhance inter-rater reliability, the raters conducted a pre-assessment session before the experiment began. This session standardized scoring criteria, assessment methods, and evaluation protocols, ensuring consistency in ratings. Furthermore, the simulated patient’s behavior was predefined and standardized, minimizing variations in responses during OSCE evaluations. This standardization ensured that all participants were assessed under identical conditions and that their performance was evaluated based on consistent criteria. Clinical skills assessment using the OSCE was conducted at four time points to evaluate changes in clinical competency over time. The first assessment (Assessment 1) was conducted at the beginning of the experiment to establish a baseline measurement of each participant’s clinical skills. After this initial evaluation, the 13 newly graduated physical therapists were randomly assigned to one of two groups: Group 1 (n = 6) and Group 2 (n = 7). During the first training phase, which lasted one month, Group 1 received Off-JT (Training A), while Group 2 underwent OJT (Training B). At the end of this period, all participants were reassessed in Assessment 2 to determine the effectiveness of the training. Following the second assessment, the training methods were switched in a crossover design. Group 1, which had previously received Off-JT, transitioned to OJT, while Group 2, which had undergone OJT, began Off-JT. This second training phase also lasted one month to ensure that each participant experienced both training methods under comparable conditions. After completing the second training phase, Assessment 3 was conducted to evaluate whether switching training methods impacted clinical skill development. To assess the long-term retention of training effects, Assessment 4 was conducted three months after Assessment 3. This final evaluation was intended to determine whether the skills acquired through OJT and Off-JT were maintained over time. The training phases and assessment schedule are presented in Figure 1, and the detailed group allocation is summarized in Table 1. Intervention protocol for this study. The study was conducted using the ABAB crossover design Table 1Number and age of participants and skill scores at the time of Assessment 1GroupGender: nAge (mean±SD)Assessment 1 score (%)ROM-tMMTGetting upTransferGroup 1Male: 3Female: 321.5±0.557.173.139.958.3Group 2Male: 6Female: 121.7±0.552.755.231.146.0SD=Standard deviation, ROM=range of motion, MMT=manual muscle testing Number and age of participants and skill scores at the time of Assessment 1 SD=Standard deviation, ROM=range of motion, MMT=manual muscle testing On-the-job training The OJT program was designed based on established methodologies from the previous research.[17] The training provided real-time, hands-on clinical skills instruction under the supervision of an experienced clinical physical therapist mentor. The OJT sessions were integrated into participants’ daily clinical practice, ensuring that they acquired practical skills in a real-world setting while actively engaging with patients. Each participant was paired with a dedicated mentor who provided one-on-one supervision and direct feedback throughout the training period. The mentoring sessions focused on improving specific clinical competencies, including patient assessment, treatment planning, therapeutic interventions, and communication with patients and healthcare teams. The training was highly interactive, allowing participants to receive immediate guidance and correct mistakes in real time. Additionally, mentors facilitated case-based discussions, enabling participants to reflect on their clinical encounters, analyze their decision-making processes, and develop a deeper understanding of patient management [Figure 2]. Theoretical framework of on-the-job training The OJT sessions were conducted during regular working hours, ensuring that participants applied their newly acquired skills in an authentic clinical environment. Each session lasted 1–2 h/day, depending on case complexity and individual learning needs. The training spanned a total duration of one month, providing ample time for skill reinforcement and competency development. Off-the-job training The Off-JT implemented in this study was structured to provide a comprehensive learning experience through a combination of theoretical instruction and practical application. Various Off-JT methodologies have been documented in previous research, including vestibule training, role-playing, simulations, lectures, management games, audiovisual training, programmed instruction, and case studies.[14] Based on the objectives of this study, a lecture-based learning component and a simulation-based training component were selected to facilitate skill acquisition in newly graduated physical therapists. During the lecture component, the instructor, a licensed physical therapist with over 10 years of clinical experience, delivered oral explanations focusing on fundamental clinical concepts and best practices. The lectures covered detailed procedural guidelines for patient assessments and therapeutic interventions. Participants were encouraged to engage in discussions, ask questions, and clarify doubts to ensure a deep understanding of the theoretical foundations behind each clinical technique. The lecture sessions lasted approximately 20 min per session. Following the lecture, the simulation-based training provided participants with an opportunity to apply theoretical knowledge in a controlled environment. Under the instructor’s supervision, participants practiced joint range of motion measurement, muscle-testing methods, assistance in getting up, and assistance in transferring—key competencies required in clinical practice. The instructor offered real-time feedback, corrections, and performance assessments, ensuring that participants refined their techniques and improved their proficiency. Each simulation session lasted approximately 40 min [Figure 3]. Theoretical framework of off-the-job training Off-JT sessions were conducted twice a week, with each session lasting 1 h, for a total of eight sessions over the training period. The structured approach of Off-JT ensured that participants had ample opportunity to integrate theoretical knowledge with hands-on practice, thereby enhancing their overall clinical competence before applying these skills in a real-world setting. Statistical analysis The OSCE task scores were converted into percentage values, with a maximum score of 100%. To evaluate the effectiveness of the training methods, a Mann–Whitney U-test was used to compare score changes between Assessment 1 and Assessment 2 for the two groups. Additionally, a two-way analysis of variance (ANOVA) was performed to assess the interaction effects of training methods and assessment timing. To further analyze the effects of the independent variables (training method and assessment timing) on OSCE performance, a two-way factorial analysis of covariance (ANCOVA) was conducted. This analysis allowed for the control of baseline differences, thereby providing a more accurate evaluation of the training effects. The results of Assessment 1 were used as a covariate in ANCOVA to account for initial differences between the groups. Before conducting ANOVA and ANCOVA, normality was assessed using the Shapiro–Wilk test, and homogeneity of variance was tested using Levene’s test. Since all data met the statistical assumptions, no transformations were necessary. Multiple post-hoc comparisons were conducted using the Bonferroni correction to control for type I errors. All statistical analyses were performed using IBM SPSS Version 26 (IBM, Chicago, Illinois, USA), with statistical significance set at P < 0.05. Ethical considerations This study was approved by the ethics committee of the research institution (approval number HM24-117). Results Figure 4 illustrates the effects of OJT and Off-JT, showing changes in the average scores between Assessments 1 and 2. Table 2 summarizes the average percentage of points, SDs, interactions, and main effects of the four OSCE tasks. Figure 5 illustrates the average percentage of OSCE task scores for both groups at each assessment time point. Change in the average percentage of scores between Assessment 1 and Assessment 2. The vertical axis shows the change in the average rate. The horizontal axis shows on-the-job training and off-the-job training Evolution of assessment results for each objective-structured clinical examination task. The black plot represents Group 1, and the white plot represents Group 2. Solid lines indicate on-the-job training interventions and dotted lines indicate off-the-job training interventions Table 2Objective-structured clinical examination score rate means and standard deviations for each group at each assessment period, and results of a two-way analysis of varianceTaskPercentage score, mean (SD)Time factorInteraction Assessment 1Assessment 2Assessment 3Assessment 4 F P F P Range of motion Group 157.1 (19.5)89.7 (11.9)93.7 (5.3)93.7 (4.3)38.6<0.051.60.21 Group 252.7 (14.7)70.4 (14.6)87.8 (7.7)83.7 (11.2)Muscle testing Group 173.1 (9.4)92.1 (4.1)93.1 (7.1)88.0 (7.5)66.1<0.055.20.005 Group 255.2 (11.1)62.7 (18.8)86.5 (5.6)81.7 (10.1)Getting up Group 139.9 (24.2)78 (10.4)91.1 (4.9)76.8 (13.6)50.9<0.058.6<0.05 Group 231.1 (11.2)32.7 (15.9)82.7 (15.0)66.8 (19.3)Transfer Group 158.3 (27.8)80.7 (13.6)86.5 (11.8)77.6 (12.6)42.7<0.058.7<0.05 Group 246.0 (11.2)47.8 (11.5)86.6 (9.3)79.5 (14.6)SD=Standard deviation Objective-structured clinical examination score rate means and standard deviations for each group at each assessment period, and results of a two-way analysis of variance SD=Standard deviation The “joint range of motion measurement” task displayed a significant effect of assessment timing, but no interaction effect was observed (F(3,33) = 1.59, P = 0.21). Assessment timing factors exhibited significant differences between Assessments 1 and 2 in both groups (P < 0.01). The “muscle testing methods,” “assistance in getting up,” and “assistance in transferring” tasks displayed an interaction [“muscle testing methods” task: (F(3,33) = 5.2, P < 0.05); “assistance in getting up” task: (F(3,33) = 8.6, P < 0.05); “assistance in transferring” task: (F(3,33) = 8.7, P < 0.05)]. Post-hoc analysis identified significant differences in the “muscle testing methods” task between Assessments 1 and 2 (P < 0.05), Assessments 1 and 3 (P < 0.05), and Assessments 1 and 4 (P < 0.05) for Group 1; and between Assessments 1 and 3 (P < 0.05), Assessments 1 and 4 (P < 0.05), Assessments 2 and 3 (P < 0.05), and Assessments 2 and 4 (P < 0.05) for Group 2. In the “assistance in getting up” task, there were significant differences between Assessments 1 and 2 (P < 0.05), Assessments 1 and 3 (P < 0.05), and Assessments 1 and 4 (P < 0.05) for Group 1; and between Assessments 1 and 3 (P < 0.05), Assessments 1 and 4 (P < 0.05), Assessments 2 and 3 (P < 0.05), and Assessments 2 and 4 (P < 0.05) for Group 2. In the “assistance in transferring” task, there were significant differences between Assessments 1 and 2 (P < 0.05) and Assessments 1 and 3 (P < 0.05) in Group 1; and between Assessments 1 and 3 (P < 0.05), Assessments 1 and 4 (P < 0.05), Assessments 2 and 3 (P < 0.05), and Assessments 2 and 4 (P < 0.05) in Group 2. Discussion This study evaluated the comparative effectiveness of OJT and Off-JT in improving the clinical competencies of newly graduated physical therapists. The findings revealed that Off-JT was significantly more effective than OJT in improving specific clinical skills, particularly in tasks that required structured knowledge and standardized procedural execution, such as muscle testing and patient assistance techniques. In contrast, both training methods contributed to the improvement of joint range of motion measurement skills, with no significant difference observed between the two groups. These results suggest that engaging in Off-JT sessions outside of regular clinical hours provides a structured and focused learning environment, which may be particularly beneficial for newly graduated physical therapists who require explicit guidance and repetition to develop procedural proficiency. This supports the notion that Off-JT allows for targeted skill acquisition in a controlled setting, which may be essential for mastering complex assessment and treatment techniques. This aligns with previous research indicating that structured clinical training programs, such as simulation-based learning and problem-based learning, significantly contribute to skill acquisition and retention. Sahu et al. emphasized the importance of early clinical skills training and hands-on practice opportunities as key factors in competency development, particularly for healthcare professionals transitioning from academic training to real-world clinical practice.[18] However, the role of OJT in reinforcing hands-on experience in real clinical settings should not be overlooked, as it facilitates skill application in actual patient care contexts. Focusing specifically on the “joint range of motion measurement” task, no interaction was observed between the group and assessment timing factors. OJT appears to modestly enhance the clinical skills of newly graduated physical therapists. However, concerns have arisen regarding the instructor’s workload and variability in teaching effectiveness attributed to differences in teaching methods. The effectiveness of OJT may depend largely on the quality and consistency of the instructor’s guidance, highlighting the need for a more structured OJT framework to ensure consistent training quality across different settings. Consistent with this study, Yoshida et al. identified that OJT in an educational program for nurses significantly improved knowledge and skills in endoscopic assessment techniques, ensuring proper and safe practice.[19] Drake et al. also noted improvements in OSCE results for midwives trained in neonatal resuscitation using a structured OJT approach.[20] However, Sree et al. highlighted potential issues in OJT when trainers lack sufficient skills or time to deliver high-quality training.[21] This implies a significant limitation of OJT: its reliance on the mentor’s expertise and availability, which may vary considerably across clinical settings. We demonstrated that Off-JT is significantly more effective than OJT in enhancing the skills in assisting patient movements. The combination of understanding lecture content, engaging in simulated activities, and receiving feedback appears crucial for improving clinical skills. Nitta et al. reported that Off-JT involving the simulation of a vascular anastomosis procedure significantly reduced task completion time and improved clinical skills.[22] Similarly, Belykh and Byvaltsev identified significant improvements in microsurgery skills after one month of simulation training.[23] A recent meta-analysis by Tarrahi et al. analyzed nine clinical trials and reported that simulation-based training in obstetric emergencies significantly improves skills in managing dystocia and postpartum hemorrhage compared to traditional lecture-based methods. These findings highlight the potential benefits of simulation training in enhancing clinical competency.[24] Additionally, they reinforce the idea that Off-JT provides a more controlled and intensive learning experience that may lead to faster and more substantial improvements in complex clinical tasks. There was no significant difference in the effectiveness of OJT and Off-JT for “joint range of motion measurement” tasks, suggesting that both training methods can contribute to skill improvement in standardized tasks. “Joint range of motion measurement” tasks are typically well established, and the techniques are uniform, which may explain the similar outcomes of both training methods. For such tasks, procedure consistency may reduce the need for highly specialized training, allowing both OJT and Off-JT to be equally effective.[2526] Both groups exhibited a decline in performance in “assistance in getting up” and “assistance in transferring” tasks during Assessment 4, suggesting that these skills may require more frequent practice. Particularly, Group 1 exhibited no significant difference in scores between Assessments 1 and 4 for the “assistance in transferring” task, indicating a possible decline in skill retention. This is consistent with Gradl-Dietsch et al.’s findings, which reported a significant decline in acquired skills among medical students six months after training in manipulation techniques.[27] Offiah et al. evaluated the retention rate of clinical skills among medical students and reported that skills practiced on a daily basis were retained at a high rate, whereas those with limited opportunities for practice showed a significant decline.[28] Furthermore, Govender et al. and de Ruijter et al. found that the skills of medical professionals begin to decline within three months of training. They also emphasized that skill retention is influenced by factors such as the learning environment, motivation, and the use of simulations for low-frequency procedures.[2930] These findings highlight the importance of ongoing practice and reinforcement in maintaining clinical competency, particularly skills that may not be frequently used in clinical settings. The findings of this study highlight the importance of structured clinical training programs in the development of newly graduated physical therapists. From a health policy perspective, structured training methods such as Off-JT align with national guidelines for clinical education and workforce development. In Japan, the Ministry of Health, Labour and Welfare emphasizes the need for both theoretical and practical training to ensure the competency of healthcare professionals, reinforcing the relevance of this study. Additionally, the World Health Organization stresses that well-structured training programs are essential for achieving Universal Health Coverage by ensuring that healthcare professionals are adequately trained. The results of this study suggest that an integrated approach combining both OJT and Off-JT can enhance clinical competency and contribute to the development of a sustainable and highly skilled healthcare workforce. Future studies should explore the long-term impacts of these training methods on patient outcomes and healthcare system efficiency. While this study provides valuable insights into the effectiveness of different training methods for newly graduated physical therapists, it also has certain limitations, with several aspects warranting further investigation. First, the sample size was relatively small, and the study was conducted within a specific clinical setting. Future research should involve a larger cohort of participants across diverse clinical environments to enhance the generalizability of the findings. Second, this study focused on short-term skill acquisition. To determine the long-term retention of training effects, follow-up assessments at six months or one year post-training would be beneficial. Finally, this study primarily examined newly graduated physical therapists. Further research should explore whether similar training effects can be observed in experienced therapists or those undergoing specialization training. Despite these limitations, this study has several notable strengths. It employed a crossover design, which allowed for within-subject comparisons to minimize inter-individual variability. Additionally, the study provides valuable insights into the comparative effectiveness of OJT and Off-JT for newly graduated physical therapists, contributing to the development of evidence-based training strategies. Moreover, the findings highlight the importance of structured learning environments in skill acquisition, offering practical implications for designing future clinical training programs. Limitations and recommendations While this study provides valuable insights into the comparative effects of OJT and Off-JT for newly graduated physical therapists, several limitations should be acknowledged. The relatively small sample size and single-institution setting may limit the generalizability of the findings. Moreover, although the instructional content was designed based on established models, some variation in delivery among mentors and instructors could not be fully controlled. Additionally, the assessment focused on four OSCE tasks, which, while relevant to clinical practice, do not capture the full range of clinical skills required in diverse clinical settings. The relatively short follow-up period also restricts our understanding of the long-term retention of training effects. To build on these findings, future research should aim to include larger, more diverse participant groups, implement more standardized instructional protocols, assess a broader set of clinical skills, and incorporate long-term follow-ups to evaluate the sustainability of training outcomes. Conclusions This study demonstrated that Off-JT was more effective than OJT in enhancing the clinical skills of newly graduated physical therapists, particularly in tasks requiring specialized techniques and structured learning. While both training methods contributed to skill development, Off-JT provided a more focused educational environment that facilitated faster skill acquisition. These findings suggest that incorporating structured Off-JT components into early career training programs may strengthen clinical skills among new graduates. Future training initiatives should consider a balanced integration of OJT and Off-JT to support comprehensive professional development in clinical settings. Conflicts of interest There are no conflicts of interest.