Testing the limits: can enhanced external validity promote translation? Authors: Aileen MacLellan, Manoj M Lalu Published: 2024 Efficient translation of therapies from “bench-to-bedside” is a priority for researchers, funders, industry, regulators, and patients alike. Yet this can be immensely challenging, with development of new therapies requiring substantial time and resources.1 Studies, both old and new, have suggested a relatively static rate of successful translation (∼5%) despite advances in research methods.2,3 Metaresearch—the study of research itself—offers a path forward to address this inertia. By analyzing methods and outcomes in studies across a research field, metaresearch can reveal patterns, systemic issues, and opportunities for improvement that may not be apparent within individual studies alone. This approach has already shaped strategies to accelerate translation. For instance, metaresearch has shown that rigorous methods, such as randomization and blinding, make a significant difference in preclinical animal studies. Without these practices, studies are prone to inflated effect sizes and produce results that fail to replicate in human trials.4 This has led to widespread calls for preclinical researchers to implement these methods that increase internal validity, with the goal of improving the reliability of preclinical findings.5 In the December 2024 issue of eBioMedicine, Berg and colleagues6 extend this approach, demonstrating the value of metaresearch in understanding how translation may be influenced by external validity (i.e., generalizability of findings across species, sex, models etc.). Through a systematic review and meta-analysis of preclinical multiple sclerosis (MS) studies, the authors compared external validity in animal studies investigating drugs that successfully translated to humans versus those that did not. Their findings characterise the complexity of the translational pipeline, highlight existing disconnects between clinical and preclinical research, and offer key insights into how the biomedical research community can work to bridge these gaps. A key finding in this study is that none of the factors investigated by Berg and colleagues6 predicted successful translation. Specifically, when assessing animal studies that were conducted prior to the initiation of early-phase clinical trials, the authors found no differences in the diversity of experimental approaches applied for therapies that successfully translated to humans versus those that did not. However, this work also revealed a general lack of variation in the experimental conditions used in these preclinical studies. For example, prior to approval, no therapies were tested across multiple MS animal models and only one had been evaluated in both sexes. This lack of heterogeneity may explain the null result, suggesting that the MS research community is currently working within too narrow a range of experimental conditions to observe any meaningful relationships between external validity and translation. Thus, this study does not definitively reject the hypothesis that enhanced external validity in animal research promotes translation. Instead, it highlights key limitations of the current evidence base. Critically, other results of this systematic review suggest further investigation of the role of external validity in preclinical animal studies is worthwhile. When examining all animal studies, including those conducted after translation to humans, therapies that successfully translated were also effective across a wider variety of experimental contexts. Of course, this is not sufficient evidence to conclude that findings with greater external validity (i.e., similar results across species, sexes, labs) will reliably translate to humans. However, these findings do support the value of using animal models and provide indirect evidence suggesting the importance of external validity in preclinical animal research. Moving forward, a more rigorous test of this hypothesis will require incorporation of biologically relevant heterogeneity into preclinical animal experiments.7 Preclinical investigators should seek to test the generalizability of findings across diverse disease models and species, and also include both sexes in experiments (Table 1). Aiming to generate results that can be replicated across independent laboratories should also be a priority, with multilaboratory preclinical studies offering a powerful mechanism to assess external validity, and identify robust, generalizable findings.8,9 Table 1Variables to manipulate and evaluate within experimental designs to increase external validity in animal studies.VariableExamplesSexMales and femalesSpeciesMultiple species (e.g., mice, rats, pigs, primates)AgeRelevant age groups or ranges for the disease of interest (e.g., aged versus middle aged animals)Disease modelApply multiple disease induction methodsComorbidityAnimals with relevant comorbid conditions (i.e., particularly those that are common in affected human populations)Experimenter identityMultiple individuals conduct experimental procedures independentlyHousingAnimals housed in conventional conditions, and with resources (e.g., environmental enrichment) that improve welfareLaboratoryMultiple independent laboratories conduct experiments Variables to manipulate and evaluate within experimental designs to increase external validity in animal studies. Beyond examination of external validity, another striking finding from Berg and colleagues6 was that 91% of MS animal studies were conducted after therapies had already gained regulatory approval. Moreover, for 57% of the therapies, no animal studies were published prior to early-phase clinical trials. While surprising, this pattern is consistent with findings from another metaresearch investigation that reviewed novel therapies from seven disease areas (87% of animal studies were conducted after early-phase clinical trials; for 35% of therapies, early-phase clinical trials were conducted prior to publication of any animal studies10). These findings challenge “linear” conceptions of the translational pipeline, and instead reflect the iterative, cyclical nature of biomedical research. Yet they also raise questions regarding the justification of animal studies conducted post-regulatory approval. Here, Berg and colleagues6 found that the majority of post-approval animal studies were conducted for “legitimate” aims such as understanding underlying mechanisms, or to benchmark a novel candidate against the approved therapy. However, they also found that some preclinical researchers may be unaware of clinical progress, with 4% of studies performed after approval aimed at translating a potential therapy, and 3% aimed at establishing proof-of-principle efficacy. This highlights gaps between preclinical and clinical research, and also suggests there is room to improve existing mechanisms to protect against redundancy (e.g., ethical and peer review processes). Taken together, this innovative study by Berg and colleagues6 highlights the value of metaresearch in accurately characterising the biomedical research landscape and pinpointing priority areas to enhance translational efficiency. Addressing these challenges will require coordinated efforts from both preclinical and clinical research communities to close existing gaps, incorporate external validity considerations into experimental design, and ultimately work towards improved translation. Contributors Conceptualization: MML and AM; Writing—original draft: AM; Writing—review and editing: AM and MLL; Supervision: MML. Declaration of interests AM is supported by a Mitacs Elevate postdoctoral fellowship with the Canadian Council on Animal Care. The authors report no other competing interests related to this study. Acknowledgements No funding source has been involved in the writing of the manuscript or the decision to submit it for publication. References