Unravelling Orthopaedic Surgeons’ Perceptions and Adoption of Generative AI Technologies Authors: Matthias Schmidt, Yasmin B Kafai, Adrian Heinze, Monica Ghidinelli Published: 2024 ABSTRACT ABSTRACTThis mixed-methods study investigates the adoption of generative AI among orthopaedic surgeons, employing a Unified Theory of Acceptance and Use of Technology (UTAUT) based survey (n = 177) and follow-up interviews (n = 7). The research reveals varying levels of AI familiarity and usage patterns, with higher adoption in research and professional development compared to direct patient care. A significant generational divide in perceived ease of use highlights the need for tailored training approaches. Qualitative insights uncover barriers to adoption, including the need for more evidence-based support, as well as concerns about maintaining critical thinking skills. The study exposes a complex interplay of individual, technological, and organisational factors influencing AI adoption in orthopaedic surgery. The findings underscore the need for a nuanced approach to AI integration that considers the unique aspects of orthopaedic surgery and the diverse perspectives of surgeons at different career stages. This provides valuable insights for educational institutions and healthcare organisations in navigating the challenges and opportunities of AI adoption in specialised medical fields.KEYWORDS: Generative AI, orthopedic surgery, technology adoption, UTAUT, continuing medical education, human-AI collaboration Introduction & Context The rapid advancement of artificial intelligence (AI) technologies, particularly large language models (LLMs) and generative AI is poised to transform healthcare, including orthopaedic surgery. Generative AI represents a paradigm shift from previous AI systems, demonstrating capabilities that include creating novel content, performing knowledge assessments at a level comparable to that of medical residents, and automating clinical workflows [1,2]. These advancements promise to alleviate administrative burdens on clinicians, potentially allowing more time for patient care [3]. However, integrating AI into clinical practice presents complex challenges, including ethical considerations, regulatory uncertainties, and concerns about patient trust and care quality [4,5]. Recent studies have highlighted the potential benefits and barriers to AI adoption in healthcare settings [6,7]. Additionally, Henning et al. [8] emphasise the urgent need to include advanced digital skills in medical education, including CME, to address the growing gap between technological advancements and healthcare professionals’ preparedness. The field of orthopaedic surgery presents a unique context for studying AI adoption. Faced with workforce shortages and the imperative for evidence-based practice, orthopaedic surgeons are navigating the potential benefits and risks of integrating AI into their workflows [9,10]. Understanding how these professionals perceive and adopt AI technologies is crucial for developing effective strategies to enhance surgical outcomes, improve efficiency, and maintain high standards of patient care. Studies by Giorgino et al. [11] and Cuthbert & Simpson [12] have begun to explore the implications of generative AI in orthopaedic practice, highlighting both the transformative potential and the implementation challenges. As AI evolves, questions arise about its impact on medical education and clinical decision-making. There is a pressing need to explore whether AI enhances learning and practice or risks undermining the acquisition of critical thinking skills essential for high-stakes medical decisions [13,14]. This research aims to shed light on the current state of AI adoption in orthopaedic surgery, providing insights to inform better the development of continuing medical education (CME) and continuing professional development (CPD) programmes. This work addressed specifically these questions: What is the current level of familiarity with and usage of different types of AI among orthopaedic surgeons What is the current level of familiarity with and usage of different types of AI among orthopaedic surgeons What are the barriers to adopting this technology? What are the barriers to adopting this technology? Methods This study employed a mixed methods approach to explore the adoption of generative AI among orthopaedic surgeons within the AO Foundation Network, which comprises over 20,000 orthopaedic surgeons globally. This methodology aligns with recommendations by Venkatesh et al. [15] for studying complex phenomena in emerging technological domains. Participants and Recruitment Purposive sampling was used to select participants, ensuring the inclusion of surgeons actively using or considering generative AI and those in the early stages of exploration. Recruitment strategies involved distributing flyers at AO Davos courses (n = 40), AO subspeciality newsletters (n = 52), posts in faculty networks at the University of Pennsylvania Medical School and the University of Rochester Medical School, and orthopaedic interest groups on LinkedIn, Reddit, and Facebook (n = 85). Data Collection Data was collected from November 2023 to March 2024 utilising a survey based on the Unified Theory of Acceptance and Use of Technology (UTAUT) framework [16]. The survey employed a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree) to capture nuanced responses. We collected 177 qualified responses, with 82 participants opting in for follow-up interviews. Due to resource constraints, only seven in-depth interviews were conducted, following the guidelines of Creswell and Creswell [17] for qualitative sampling in mixed methods research. Data Analysis Data analysis involved descriptive and exploratory inferential statistics [18] using Python scripts. The qualitative phase employed thematic analysis of interview data to provide context and depth to the quantitative findings [19], mainly focusing on barriers to adoption and the practical use of generative AI. Ethical Approval The Study Adhered to IRB Protocol #854794 from the University of Pennsylvania and was Preregistered on Aspredicted.Org (Protocol #153425). Results The study aimed to assess orthopaedic surgeons’ familiarity with and adoption of AI technologies. Among the 177 orthopaedic surgeons who fully completed the survey, 87% were male, 12.4% were female, and 0.5% did not specify it. The sample represented various age groups (range 18–74) and geographies (Europe 35.6%, Asia 26.5%, North America 18.5%, South America 11.8%, Africa 4.5%, and Oceania 2.8%) from 56 different countries. Surgeons were primarily familiar with artificial intelligence in general (mean = 3.58, SD = 1.65), followed by machine learning (mean = 3.43, SD = 1.65) (Figure 1). Generative AI and large language models showed lower familiarity levels (mean = 3.14, SD = 1.69 and mean = 3.26, SD = 1.71, respectively) (Figure 1), suggesting a potential knowledge gap in these more recent AI technologies. Familiarity with AI technologies. Self-reported familiarity with different AI technologies (7-point likert scale: 1 – not familiar at all, 7 – extremely familiar), n = 177. To gain further insights into surgeons’ understanding of generative AI, the survey included an open-ended question asking respondents to define generative AI in their own words. A thematic analysis of the responses revealed that 33.3% provided focused definitions (Figure 2) aligning with generative AI’s core capabilities of creating novel content based on learned patterns [20], and 13.9% classified it as a subset of AI. Notably, 20.9% offered vague descriptions, and 9.3% expressed unfamiliarity (Figure 2), indicating a significant variance in understanding this emerging technology among orthopaedic professionals. Conceptions of generative AI. Thematic analysis of an open-ended question asking respondents to define generative AI in their own words. N = 129. We then analysed the self-reported usage of generative AI across clinical workflows (Figure 3). The highest usage was reported in clinical research (mean = 3.03, SD = 2.00) and professional development (mean = 2.73, SD = 1.83), with 14% and 9% of surgeons using these applications weekly, respectively. In contrast, surgeons showed limited adoption of generative AI in direct patient care activities, with intra-operative assistance being the lowest (mean = 1.88, SD = 1.71), with 71% of surgeons never using it in this context. Pre-consultation and treatment planning showed moderate adoption (mean = 2.14, SD = 1.78 and mean = 2.16, SD = 1.76, respectively). The adoption of generative AI was relatively low among surgeons with significantly higher use for treatment planning, post-operative care monitoring, and professional development in the younger age groups. Self-reported usage frequency of generative AI. How frequent do you use generative AI tools in the following aspects of your clinical workflow? N = 177. We additionally analysed effort expectancy, which refers to the perceived ease of technology use across age groups using a one-way ANOVA (Figure 4). Residuals were normally distributed and there were neither outliers nor variance heterogeneity. We found a significant main effect (F(4,172) = 4.51, p = 0.002, η2 = 0.095) and conducted Tukey’s post-hoc tests, correcting for multiple testing, which revealed that the youngest age group showed significantly higher effort expectancy than the two oldest age groups (56–65 and > 65) (Figure 4). All other comparisons were not significant (all adjusted p > 0.05). This indicates that younger surgeons perceived generative AI as easier to use than their more senior counterparts, suggesting a potential generational gap in technology adoption that might need to be addressed in future training initiatives. Distribution of effort expectancy across age groups. Comparative analysis of the effort expectancy averaging three statements: “I believe that generative AI is easy to use”, “learning to operate generative AI will be easy for me”, and “interacting with generative AI will be clear and understandable” (7-point likert scale: 1 – strongly disagree to 7 – strongly agree). N = 177. P-values show the significant pairwise comparisons with tukey-hsd. The thematic analysis of the interviews provided insights into applications of generative AI in practice and highlighted its potential in clinical use cases, education enhancement, and improving patient communication (Figure 5). Surgeons reported using generative AI to create case studies, personalise learning experiences, and generate more empathetic patient communications. The thematic analysis strongly emphasised AI’s role in automating administrative tasks and enhancing educational content, aligning with the quantitative findings of higher adoption in research and professional development contexts. Applications of generative AI in practice. Thematic analysis of interview transcripts. N = 7. The barriers to the adoption of generative AI were multifaceted (Figure 6), including systemic and institutional barriers, technological limitations, and concerns about patient acceptance and trust. Surgeons highlighted challenges such as managerial focus on financial constraints over innovation, regulatory uncertainties, and the need for reliable information validation. The qualitative data also revealed concerns about the potential overreliance on AI and the importance of maintaining critical thinking skills in medical practice. Barriers of generative AI adoption. Thematic analysis of interview transcripts. N = 7. These findings paint a complex picture of generative AI adoption in orthopaedic surgery, revealing potential benefits and significant challenges. Discussion [500] This study assessed the familiarity and adoption of AI technologies among orthopaedic surgeons. While surgeons recognise the potential of AI technology, particularly in research and education, practical implementation in clinical settings still needs to be improved. The analysis of effort expectancy revealed a generational gap, with younger surgeons perceiving generative AI as easier to use than their more senior counterparts. Barriers to adoption were multifaceted, including systemic, technological, and patient acceptance challenges. Concerns about overreliance on AI and maintaining critical thinking skills were also mentioned. These results underscore the need for targeted CME/CPD educational initiatives and institutional support to bridge the knowledge gap and facilitate more widespread adoption of generative AI in orthopaedic practice. The observed variance in perceptions of ease of use between older and younger surgeons is particularly noteworthy, as it highlights a generational divide in technology acceptance that could significantly impact the integration of AI tools in clinical practice. Younger surgeons show a greater willingness to use AI in treatment planning, post-operative care monitoring, and professional development, reflecting their adaptability and openness to leveraging new technologies. This presents challenges and opportunities and suggests that traditional approaches to technology training in medical education may need to be re-evaluated. The concept of reverse mentoring, where youngersurgeons with greater technological affinity guide their more senior colleagues in navigating new technologies, could be a valuable strategy to address the immediate need for technology adoption and to foster intergenerational learning and collaboration, potentially enhancing overall surgical practice [21]. The qualitative insights around barriers to adoption and actual use cases provide crucial context for understanding the slow pace of integration of generative AI in direct patient care. The emphasis on evidence-based practice in surgery creates a high bar for implementing new technologies. As noted by Greenhalgh et al. [22], adopting innovations in healthcare organisations is a complex process influenced by multiple factors, including the perceived benefits and risks of the innovation. In the case of generative AI, the lack of robust clinical evidence supporting its efficacy in improving patient outcomes may be a significant barrier to adoption, particularly among more senior surgeons. The concept of “human-in-the-loop”, i.e. the integration of human oversight in AI generative processes, emerges as a critical factor in the potential adoption of AI technologies in the operating room. This aligns with the work of Hemmer et al. [13], who emphasised the importance of maintaining human oversight and decision-making in AI-assisted healthcare. The integration of AI should aim to augment rather than replace human expertise, particularly in tasks requiring complex judgement and psychomotor skills that are currently beyond the capabilities of AI systems. The differential appetites for generative AI usage among surgeons at various career stages present intriguing implications for professional development and workflow optimisation. Younger surgeons show an inclination to apply AI for research and professional development, which aligns with the findings of Cheng et al. [6], who noted the potential of AI in enhancing medical education and research. Conversely, the interest of older surgeons in workflow automation suggests a pathway for AI integration that focuses on reducing administrative burdens, potentially allowing more time for patient care. The findings that older surgeons use AI less frequently for treatment planning and post-operative monitoring suggest that tailored support and training could enhance their engagement with these technologies for more steps in the clinical workflow. However, the challenges in leveraging AI for CPD among highly experienced surgeons represent a significant limitation of current AI applications. This echoes the findings of Esmaeilzadeh [4], who highlighted the importance of tailoring AI solutions to healthcare professionals’ specific needs and expertise levels. Future research should explore how AI can be leveraged to enhance the learning and development of surgeons across all experience levels, perhaps by focusing on rare case presentations or emerging surgical techniques. This study’s findings underscore the need for a nuanced, multifaceted approach to integrating generative AI in orthopaedic surgery. While the technology shows promise in research and administrative task automation, its adoption in direct patient care still needs to be improved. This suggests that future development and implementation strategies should focus on creating AI tools that demonstrably enhance surgical outcomes and integrate seamlessly into existing workflows while recognizing the imperative of human expertise and judgementin surgical effective practice. Limitations Although substantial for a specialised field like orthopaedic surgery, the sample size may limit generalisability. In addition, in this study there is the possibility of selection bias, as participants were drawn mainly from internet-based special interest groups. This may have resulted in a sample skewed towards individuals who are more technologically inclined, and therefore not fully representative of the broader surgical community. Therefore, it is important to acknowledge that the results may reflect the perspectives of a subgroup of surgeons with a higher technological affinity. The rapid evolution of AI technologies during data collection could affect the currency of some findings. Additionally, the self-reported nature of usage data and potential self-selection bias in participant recruitment may influence results [23,24]. Conclusion The findings from this study offer valuable insights that could inform development of CME and CPD programmes in orthopaedic surgery. While the small sample may represent the perspectives of a subgroup of surgeons with a higher technological affinity, the identified generational gap in AI adoption highlights the need for targeted educational initiatives that address both the enthusiasm of younger surgeons and the apprehensions of their more experienced counterparts. Continued research is necessary to address the challenges of AI integration and leverage its potential to enhance orthopaedic surgical practice and education. Acknowledgments We thank all the participating surgeons who generously contributed their time and insights to this study. Disclosure Statement No potential conflict of interest was reported by the author(s). References