How artificial intelligence is disrupting the field of medical knowledge and evaluation: a narrative review Authors: Kathrin Cresswell, Robin Williams Published: 2026 ABSTRACT AbstractAlthough artificial intelligence (AI) shows considerable promise in healthcare, its introduction and safe and effective exploitation involve several challenges. We argue that to maximise the benefits of AI and understand, track and mitigate potential harms there must be a shift in how AI is evaluated. Evaluation should be more closely linked to the local contexts in which AI is deployed and used and better integrated with clinical practice.To support this shift, we propose three interrelated levels of evaluation: evaluation of implementation, which examines transformations in contextually situated practices; evaluation of optimisation, which explores changes to clinical pathways over extended periods; and evaluation of scaling, which focuses on coordination through regional and national structures. This approach offers significant opportunities to advance both healthcare practice and the epistemology of medical knowledge.Keywords: Artificial intelligence, Evidence-Based Medicine, Health Information Systems WHAT IS ALREADY KNOWN ON THIS TOPIC Artificial intelligence (AI) has considerable potential to improve healthcare, but its performance is highly dependent on local contexts, workflows and populations. Existing evaluations focus largely on technical validation and often fail to capture how AI interacts with clinical practice and organisational processes over time. Artificial intelligence (AI) has considerable potential to improve healthcare, but its performance is highly dependent on local contexts, workflows and populations. Existing evaluations focus largely on technical validation and often fail to capture how AI interacts with clinical practice and organisational processes over time. WHAT THIS STUDY ADDS This paper proposes a three-level framework for evaluating AI-enabled healthcare that addresses implementation, ongoing adaptation and optimisation, and coordination and scaling across health systems. It argues that evaluation should be continuous, embedded in practice and closely linked to organisational learning. This paper proposes a three-level framework for evaluating AI-enabled healthcare that addresses implementation, ongoing adaptation and optimisation, and coordination and scaling across health systems. It argues that evaluation should be continuous, embedded in practice and closely linked to organisational learning. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY The framework provides a practical approach for generating evidence that is better aligned with the pace and complexity of AI deployment in healthcare. It highlights the need for policies and investments that support evaluation infrastructure, governance and cross-organisational learning to ensure AI is adopted safely, effectively and equitably. The framework provides a practical approach for generating evidence that is better aligned with the pace and complexity of AI deployment in healthcare. It highlights the need for policies and investments that support evaluation infrastructure, governance and cross-organisational learning to ensure AI is adopted safely, effectively and equitably. Artificial intelligence (AI) is becoming pervasive across health and care settings, accompanied by high expectations that it will help achieve the quintuple aim: improving population health, enhancing patient experience, reducing per capita costs, supporting the well-being of health and care professionals and advancing health equity.1 2 In many respects, the challenges associated with AI mirror those observed in complex digital health technologies, extensively described in the sociotechnical systems literature.3–7 Here, implementation studies repeatedly highlight the substantial effort required to align technologies with organisational processes and optimise work practices within developing health system landscapes.8 These insights appear to have been overlooked amid powerful claims based on AI’s laboratory performance and promises of technology-driven rationalisation. At the same time, AI exhibits several characteristics that distinguish it from other forms of health information technology. AI systems evolve rapidly, meaning that regulatory approaches designed for ‘conventional’ slowly-changing technologies may carry opportunity costs by delaying adoption of later, potentially safer and more beneficial innovations.9 Ethical and governance frameworks remain underdeveloped and struggle to keep pace with the speed of technological change.10 Formal evaluations of real life experience show that current AI performance can vary substantially across contexts, datasets and populations, with risks of degradation when systems are deployed beyond their original training environments and over time.11 Beyond this are less well-characterised but potentially much larger and more significant variations that reflect wider structural inequalities in access to data infrastructures, computational resources, evaluation capacity and commercially developed AI models. As access to frontier AI systems increasingly depends on licensing arrangements, proprietary application programming interfaces and specific technological capabilities, there is a risk that some health systems, particularly low- and middle-income settings, may be excluded from participating fully in AI-enabled healthcare transformation.12 13 New unanticipated issues are emerging such as, for example, those surrounding transparency, bias and hallucinations.14 Blind user reliance on AI outputs can introduce new safety hazards, underscoring the importance of maintaining human oversight.15 Ongoing post-market surveillance is therefore essential, with real-time monitoring required for high-risk or critical applications. Fully autonomous systems are unlikely to become a reality in the near future, primarily due to issues surrounding reliability and safety, transparency and accountability, bias and equity, public acceptance, impacts on patients and healthcare professionals, data privacy and security and the potential for malicious use.16 Furthermore, the pursuit of increasingly autonomous healthcare systems raises important ethical and equity considerations, including the risk that access to human-led care and clinician interaction could become disproportionately concentrated in better-resourced settings, potentially making human-delivered medicine an increasingly scarce or premium resource. We are still at an early stage of learning how to optimally exploit AI to enhance the quality and efficiency of care. As a result, we have limited understanding of what routine, system-wide integration of AI looks like in practice, or of its medium- and long-term effects on care delivery, clinical interactions and patient pathways.17 These dynamics raise fundamental questions about how a pervasively AI-enabled health service will emerge and the consequences this will have for medical expertise and practice. We primarily focus on clinician-facing AI systems designed to support healthcare delivery and clinical decision-making, including clinical decision support tools, diagnostic assistance systems and generative AI applications embedded within clinical workflows. While healthcare AI encompasses a broader range of administrative, public health and patient-facing technologies, the proposed evaluation framework is principally intended for AI systems integrated into routine clinical practice and organisational care processes. The manuscript is based on a narrative synthesis of literature spanning health information technology evaluation (including AI), implementation science, information infrastructures and sociotechnical systems. References to support the argument were selected purposively to capture key conceptual and practical debates relevant to evaluation in AI-enabled healthcare systems. This paper has two core aims. First, we argue that AI has the potential to transform how care is delivered, but that its safe and effective operation depends on sustained evaluation across locations and over time. AI performance is contingent on contexts, depending on interactions between algorithms, local infrastructures, clinical workflows, care processes and human judgement. Assessing processes and optimising outcomes therefore requires ongoing, carefully designed evaluation activities that generate information about implementation and adoption in real-world settings. Second, we suggest that there are broader changes in how medical knowledge is developed and mobilised. AI-enabled healthcare will be permeated by interconnected research and evaluation activities, bringing clinical practice, service delivery and knowledge production closer together. As medicine increasingly includes multiple interconnected variables (eg, genetic, environmental and lifestyle factors) alongside organisational and workflow dynamics, the boundary between scientific research and medical practice becomes blurred. Contemporary healthcare therefore must be seen as a data-intensive and knowledge-intensive environment, supported by expanding data infrastructures and pervasive AI tools. Within this setting, short-term adjustments in AI practice and evaluation contribute to longer-term shifts in the character of medical knowledge. Traditional medical research approaches have emphasised hypothesis-driven designs, controlled conditions and repeatability in the search for effectiveness.18 However, such approaches are often slow, costly and poorly equipped to capture contextual variation, system learning over time or emergent ethical and governance challenges. By privileging performance testing in laboratory or controlled trial settings, they risk producing evidence that is misaligned with real-world practice. However, research methodologies are not neutral. Methodological choices shape what can and cannot be discovered.19 In line with this, applying established tools such as randomised controlled trials (RCTs) to rapidly evolving AI systems can lead to a focus on validating existing tools and care delivery practices rather than improving these. Methodologies further carry assumptions that may not apply to AI, or indeed other complex technologies that transform the way care is provided rather than automate specific processes. For example, a common view of AI-driven transformation implies a one-to-one replacement of human actor by machine.20 However, this view does not account for accompanying adjustments in the organisation and the wider health and care system (eg, changes to care pathways, professional roles, provider–patient relationships and documentation).21–24 While much of the current literature focuses on evaluating AI systems themselves, we argue that AI adoption reinforces the need for broader changes in the theory and practice of evaluation within healthcare. Existing frameworks such as FUTURE-AI have made important contributions by articulating principles for trustworthy AI development, validation, deployment and governance.11 Our proposed framework complements these approaches by focusing more explicitly on the longitudinal, context-sensitive and practice-embedded evaluation activities needed to support ongoing adaptation, organisational learning and coordination of AI-enabled care across settings and over time. Evaluation encompasses a range of activities with differing purposes and intensities. Within the machine learning literature, distinctions are often drawn between verification, validation, local calibration, evaluation and post-market monitoring. Our use of the term evaluation focuses on broader processes of sociotechnical assessment of AI adoption and use. It refers primarily to longitudinal, context-sensitive and practice-embedded forms of assessment that combine elements of local calibration, implementation evaluation and ongoing monitoring within real-world healthcare settings. The focus is therefore not solely on technical model performance, but also on how AI systems embed with organisational processes, workflows and evolving care practices over time and within a broader political and economic context. To date, emphasis has largely been placed on demonstrating impact and value to justify investment and secure further funding.25 The current AI landscape, however, requires evidence that is more closely aligned with real-world contexts, emergent changes in relationships and care provision and decision-making timelines. Such evaluations are especially critical for AI, where systems and practices co-evolve, divisions of labour shift and care pathways are reconfigured through social learning.26 Many AI systems are currently being deployed without robust evaluation because traditional research designs are inappropriate, too slow and too expensive. This does not imply that speed should compromise rigour. Local short-term assessments are valuable but not necessarily generalisable. Rather, different forms of evaluation and evidence may be appropriate at different stages of AI deployment and for different decision-making purposes. Rapid, embedded evaluations can support implementation processes and help to identify risks, while larger-scale controlled studies remain important for establishing robust evidence of broader impact. Timelier although perhaps imperfect, evaluation evidence could support more informed choices by implementers and enable mitigation of risks. However, in resource-constrained settings, evaluation capacity may be limited though arguably even more vital due to shortcomings in local digital infrastructure, implementation capabilities and AI tools. In such contexts, ‘minimum viable evaluation’ approaches focused on core dimensions such as safety, quality and efficiency may represent a pragmatic starting point for supporting responsible adoption. Current efforts by the WHO in developing evaluation guidelines integrating ethical principles across countries with various levels of income have an important role in shaping this agenda.27 We call for a reconceptualisation of evaluation as an integral concurrent component of implementation and optimisation activities that can feed into future model enhancement and wider transformation of care. While evaluation is widely advocated, substantial gaps remain between evaluation theory and practice, perhaps illustrating the long-standing disconnect between those that commission and implement systems and those that evaluate their performance.28 29 What is needed now is not just more evaluation but evaluation that is better suited to the changing needs of health systems and rapidly developing technologies. To date, evaluation has served as a catch-all for activities of varying purpose and sophistication. We must now clarify its role and how it should be conducted to help ensure that AI in medicine serves the interests of the health system and does not create unnecessary risks. We argue that therefore achieving evaluation that is better aligned with practice requires three interrelated levels (figure 1). This proposed framework was developed based on our previous and ongoing research in health information technology evaluation (including AI), implementation science, information infrastructures and sociotechnical systems.3 8 25 30–33 The three interrelated evaluation levels emerged conceptually from identifying recurring tensions between static evaluation paradigms and the dynamic, context-dependent nature of AI-enabled healthcare systems. First, there is a need to develop new evaluation methods that explore transformation of existing practices—and how these are contextually situated, and locally driven, with feedback loops between evaluators and implementers (evaluating implementation of AI-enabled care). Second, evaluation must continue over longer timeframes that go beyond implementation towards optimisation of AI and associated pathways (evaluating adaptation and optimisation). Here, optimisation refers not only to technical refinement of AI models, but more broadly to the iterative changes in human-AI interactions and the organisation and delivery of care. Third, mechanisms are needed to orchestrate knowledge aggregation and dissemination beyond traditional timeframes of digital change programmes and associated evaluation activities. Real-time learning and coordinated sharing at regional or national levels could reduce duplication, prevent repeated mistakes and streamline piloting and testing (evaluating coordination and scaling). However, achieving such coordination will require sustained investment in infrastructure, governance and workforce capabilities, raising important questions about who bears the costs of evaluation and scaling, and which actors (eg. care providers, vendors, health researchers) should be involved. This should also include direct scrutiny of system performance and training data across populations and over time. At all levels, the viewpoint of evaluation must change from detached researchers to embedded evaluators, and from academic settings to implementation environments bringing evaluative insight closer to operational and strategic decision-making in an agile manner. However, greater evaluator embeddedness also raises important questions surrounding independence, accountability and conflicts of interest. Embedded evaluation models therefore require clear governance arrangements, transparency regarding evaluative roles and assumptions and mechanisms for external scrutiny and knowledge aggregation to ensure that locally situated findings can appropriately inform broader system-level learning and decision-making. Three shifts to better align evaluation with practice in AI-enabled healthcare. AI, artificial intelligence. Unfortunately, existing evaluations often begin too late and conclude too early to support the long-term adaptation of technologies and organisations. Moving evaluation closer to real-world settings and the clinical frontline will challenge longstanding assumptions about the arm’s-length, objective researcher that underpin much of modern medical science. Despite these limitations, large-scale controlled studies like RCTs remain important for detecting bias and addressing questions of best practice, particularly for large-scale interventions such as screening programmes.34 Rather than dismissing these approaches, we argue that other forms of knowledge are necessary and that evaluation activity should not be viewed as secondary to traditional biomedical research. It follows that existing evaluation frameworks must evolve. Rather than treating assessment as a discrete, pre-implementation exercise, evaluation must become longitudinal, context-sensitive and embedded within care delivery itself. Such an approach is essential not only for safety and accountability, but also for the optimisation and sustainable adoption of AI in medicine (and indeed other transformative technologies). While framed through the lens of AI, the challenges discussed here reflect broader tensions in how healthcare interventions are evaluated within complex and rapidly evolving systems. AI therefore serves not only as an object of evaluation, but also as a catalyst for rethinking evaluation practices more broadly, including how health systems address questions of equity, access, governance, and participation in increasingly data-intensive models of care. Delivering this form of evaluation will require collaboration across multiple stakeholders, including healthcare organisations, clinicians, developers, regulators and policymakers, with shared responsibility for ensuring that AI-enabled healthcare systems are evaluated safely, contextually and equitably throughout their lifecycle and across various contexts of use. While this shift may encounter resistance, it aligns with broader changes that extend beyond AI alone in the longer-term reconfiguration of medical knowledge. Advances in clinical research are increasingly focused on modelling interactions between multiple variables (such as genetics, lifestyle and environment) in both health and treatment. Personalised medicine in particular is challenging traditional approaches based on controlled trials of a limited number of risk factors. Within an increasingly data-intensive and knowledge-intensive healthcare environment enabled by expanding information infrastructures and pervasive AI, these developments offer significant opportunities to reshape both medical practice and the foundations of medical knowledge. In this context, policy has a critical role in shaping evaluation methods that attend to the specificities of AI deployment across different settings and over time, while coordinating focus, supporting skills development and enabling learning and spread to ensure that AI contributes safely, effectively, and equitably to health and care systems. Footnotes Contributors: KC and RW conceived this paper and drafted the manuscript together. We have used AI for streamlining wording only. The content is our own. Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. Competing interests: None declared. Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research. Provenance and peer review: Commissioned; externally peer reviewed. Ethics statements Patient consent for publication Not applicable. Ethics approval Not applicable. References