Assessing uptake of the macular degeneration core outcome set in clinical trials: a cross-sectional study Authors: Eli Oldham, R Hunter Hall, Garrett Jones, Jay Modi, Shaelyn Ward, Trevor Magee, Kyle Fitzgerald, Kimberly Magana, Griffin Hughes, Alicia Ito Ford, Matthew Vassar Published: 2025 ABSTRACT Abstract AbstractPurposeEstablishing comparability between measured outcomes in clinical trials poses a significant obstacle for systematic reviewers. Core outcome sets (COSs) were developed to address this issue. The macular degeneration (MD) COS is designed to standardise outcome measurement across clinical trials for MD. This study investigates the uptake of the MD COS in standardising outcome measurement across clinical trials.DesignCross-sectional analysisMethodsWe conducted a search on ClinicalTrials.gov to locate MD clinical trials that were registered 5 years prior to COS publication through the search date of 26 June 2023 and obtained a pool of 2152 registered studies. After applying various inclusion and exclusion criteria, we analysed 159 trials. We then analysed the COS uptake using an interrupted time series analysis (ITSA) and performed performed analyses of variance (ANOVAs) and Pearson correlations to evaluate associations between trial characteristics and outcome measurement.ResultsITSA showed no significant change in uptake following the MD COS (2016): mean percentage of completion of the COS increased by 0.24% per month before publication (p=0.27) and by 0.07% per month after publication (p=0.62), indicating no meaningful post-publication slope change in COS use. For context, visual acuity was most commonly measured, while several patient-reported and disutility domains were infrequently captured.ConclusionNo discernible patterns in COS usage for MD trials were observed. We recommend further collaboration between regulators and COS developers to help with COS uptake. Additionally, we suggest that further studies analyse adherence to COSs in respect to regulatory recommendations.Keywords: Clinical trials, OPHTHALMOLOGY, Clinical Trial, Cross-Sectional Studies, Treatment Outcome, Patient Reported Outcome Measures STRENGTHS AND LIMITATIONS OF THIS STUDY This study used a pre-registered protocol with predefined inclusion and analysis methods to enhance reproducibility. This study used a pre-registered protocol with predefined inclusion and analysis methods to enhance reproducibility. Screening and data extraction were conducted independently and in duplicate to minimise bias. Screening and data extraction were conducted independently and in duplicate to minimise bias. All materials (protocol, data and code) are publicly available on the Open Science Framework to ensure transparency. All materials (protocol, data and code) are publicly available on the Open Science Framework to ensure transparency. Because trial identification was limited to ClinicalTrials.gov, our sample may under-represent studies registered in non-US registries (eg, the EU Clinical Trials Register/EudraCT, ISRCTN, ANZCTR), particularly European age-related macular degeneration (AMD) trials. Because trial identification was limited to ClinicalTrials.gov, our sample may under-represent studies registered in non-US registries (eg, the EU Clinical Trials Register/EudraCT, ISRCTN, ANZCTR), particularly European age-related macular degeneration (AMD) trials. Registry-based extraction from ‘Outcome Measures’ may under-represent ‘complications of treatment’ where adverse events were recorded only in other modules. Registry-based extraction from ‘Outcome Measures’ may under-represent ‘complications of treatment’ where adverse events were recorded only in other modules. Introduction Macular degeneration (MD) is a retinal condition that ranks among the most common causes of vision loss worldwide.1 Recent estimates report roughly 200 million individuals currently live with age-related MD (AMD), with a prevalence of one in eight individuals over age 60 in developed countries.1 Furthermore, as life expectancy continues to increase worldwide, the prevalence of AMD can be expected to increase, with current projections nearing 300 million individuals by 2040.1 The vision loss associated with MD also comes at great societal cost as it decreases independence, negatively affects mental health and increases the likelihood of falls and fractures, resulting in individual impairment with downstream economic effects.2 For example, in 2021, it was estimated that there was $411 billion (0.3% gross domestic product [GDP]) in annual productivity loss worldwide due to vision impairment from diseases like MD.3 The increasingly high incidence and prevalence of a disease with such detrimental effects on quality of life, coupled with its economic burden, reinforce the importance of physicians needing reliable clinical evidence for effective patient care. Systematic reviews (SRs) play a crucial role in informing clinical practice by aggregating the results of multiple clinical trials. This approach helps to overcome limitations inherent in individual studies.4 However, the reliability of SRs relies on the quality of the clinical trials included within the review.5 For common eye diseases, such as MD, the inconsistent reporting of outcomes poses a significant challenge, affecting the reliability of SRs.6 7 Solomon et al’s8 SR on MD, which included 16 trials, highlighted the inconsistent measurement of visual function and quality of life. Only five trials measured visual function, and an equal number measured quality of life.8 This inconsistency in outcome measurement hinders trial comparability and contributes to reporting bias, ultimately compromising the quality of SRs.9 Core outcome sets (COSs) have emerged as a solution to address the inconsistencies in outcome measurement across clinical trials.10 A COS is a standardised and agreed upon minimum set of outcomes that should be measured in clinical trials for a specified disease or condition. Developed through the collaboration of patients, researchers and physicians,10 COSs aim to ensure consistency and comparability in outcome measurement. While COSs have been developed across a wide range of medical conditions, their effectiveness in standardising outcome measurement remains a topic of inquiry.11 Previous evidence suggests that the COS for rheumatoid arthritis trials has been effective in standardising outcome measurement; in particular, trials that measured the full COS have doubled since its inception in 1994.12 However, it is unclear whether COSs for other conditions will yield similar outcomes. Therefore, the aim of this study is to evaluate the uptake of the COS in MD clinical trials (both atrophic (non-neovascular) and neovascular AMD) and provide insight into which outcomes are most often measured. Methods Reproducibility and study design Before conducting this study, a pilot test of the search strategies, inclusion criteria and data extraction materials was carried out. We reported this cross-sectional observational study in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology guideline. We retained a Preferred Reporting Items for Systematic Reviews and Meta-Analyses-style flow diagram solely as a descriptive aid for the registry search and selection steps.13 This investigation assessed the uptake of the MD COS in clinical trials. We used a similar methodology conducted by Kirkham et al.12 Our methodology was strictly followed as outlined in the protocol which was made available to the public on Open Science Framework (OSF) (dataset).14 Prior to the start of the study, we submitted the protocol to the Institutional Review Board who determined that the study did not qualify as human subjects research. This study used only anonymised data from publicly available clinical trial registries. No individual participant data were accessed, and therefore informed consent was not required. Search string Using the Core Outcome Measures in Effectiveness Trials (COMET) Initiative database, the ICHOM AMD Standard Set published by Rodrigues et al in 2016 was identified and selected for uptake analysis.11 While the ICHOM AMD Standard Set was developed for routine clinical care, it is catalogued by COMET as a COS for AMD; accordingly, we operationalised it as a COS for this analysis and mapped registered trial outcomes to its seven domains. The COMET Initiative is an organisation founded in 2010 that brings together physicians, researchers and other delegates who are interested in the development and application of standardised COSs.15 The search was conducted on 26 June 2023. ClinicalTrials.gov, an electronic clinical trial registry, was used to identify any phase III/IV clinical trials regarding MD. ClinicalTrials.gov automatically searches synonyms and other names for the condition or disease, which have been made publicly available on OSF (dataset).14 The following filters were applied to the ClinicalTrials.gov search: “conditions: Macular Degeneration”, “study type: interventional studies”, “phase: 3 and 4” and “date: 08/01/2011 to 06/26/2023”. There were no restrictions applied to recruitment status. We restricted our search to phase III/IV clinical trials as they are known to include efficacy and/or safety of interventions.12 The search date range was chosen to permit pre-publication and post-publication analysis of COS uptake. We used a single-registry strategy (ClinicalTrials.gov) to ensure consistent fields for extraction and interrupted time series analysis (ITSA); however, this approach may miss trials registered only in other regional registries. Training Prior to extraction, all authors received training on COS development using training videos and presentations from the COMET Initiative COS tutorial website.16 Investigators also reviewed the COMET initiative handbook and engaged in group discussion.17 To ensure consistency in the extraction of data, all investigators received training on the methodology and purposes of COS uptake. Screening/eligibility criteria To qualify for inclusion, the trial registry record must have (1) studied MD as the primary topic (including either atrophic (non-neovascular) or neovascular AMD), (2) assessed the effectiveness or efficacy of interventions, (3) been registered between 5 years prior to the publication of the COS until 26 June 2023 and (4) been defined as a phase III/IV clinical trial. Trials were excluded for the following: (1) those not exclusively focused on MD, (2) those solely focused on drug pharmacokinetics or pharmacodynamics and (3) those solely focused on diagnostic test accuracy. The clinical trials identified from the comprehensive search were then compiled into a Google Sheet. In the initial screening, two authors (RHH, EO) independently evaluated the clinical trial registries of all search results to assess potential inclusion. All screening for this study was performed in a duplicate and masked manner. After independent completion of clinical trial registry screening, both authors convened to reconcile decisions regarding study inclusion/exclusion. If an agreement for a particular study could not be reached, a third-party mediator was consulted (GJ). Data extraction For each included trial, two investigators (RHH, EO) used a pilot-tested Google Form to extract general study characteristics and completeness of COS uptake. Data extraction was completed in a masked, duplicate fashion. The following characteristics were recorded for each clinical trial: trial duration, phase of trial, recruitment status, National Clinical Trial number, year of trial start date, funding type, trial continent affiliation(s), if before/after COS publication, enrolment number and type of intervention. The Google Form also included entries for the reporting of specific outcomes as previously defined by the authors of the COS, which includes ‘visual functioning and vision-related quality of life’, ‘disutility of care’ and ‘disease control’ (further information regarding the COS can be found in online supplemental etable 1). If a trial measured ‘adverse events’ as an outcome, we recorded the COS outcome ‘complications of treatment’ in the ‘disutility of care’ section as a ‘yes’. Abstraction was restricted to items explicitly listed under the ClinicalTrials.gov ‘Outcome Measures’ field. Adverse events reported only in other registry modules (eg, ‘Adverse Events’) were not counted towards COS uptake unless also specified as an outcome. In the event trialists used an established screening instrument (ie, Visual Function Questionnaire-25), authors consulted the referenced scale to determine if the appropriate core outcomes were included in the screening instrument. Two investigators (RHH, EO) extracted data independently for the first five clinical trials included in the sample for the purpose of training and then resolved any discrepancies through discussion. A third investigator (GJ) was used for mediation, if necessary. The same investigators then completed extraction of the data from the remaining clinical trials in the sample. In the event a consensus could not be made, the same third investigator was used for resolution. For any unresolved decisions, the third reviewer (GJ) performed an independent assessment of the registry record before any discussion; if disagreement persisted after GJ’s independent decision, a brief triad discussion was held to finalise inclusion/exclusion. Data analysis We used descriptive statistics to calculate the frequency of trial characteristics of our sample. The main analysis evaluated the uptake of the COS prior to and after its publication through an ITSA. The Newey-West method was implemented to estimate standard errors, as this method corrects for autocorrelation (the degree of correlation between variable’s value at different time intervals) and heteroskedasticity (the variability in errors that can make predictive values less reliable). The analysis was carried out to evaluate the per cent of COS adherence over time. To assess the adherence to the COS for each trial, we calculated the percentage of completion of the COS (%-COS-completion) by dividing each individual study’s total number of COS outcomes measured (n) by the total number of outcomes (7) listed in the COS (%-COS-completion = n/7). We then calculated a monthly mean %-COS-completion for all trials published in their respective month. Because the ITSA only allows for one data point per time period, the mean calculation was necessary to perform the analysis. Additionally, we ran a secondary analysis that evaluated the effects of ‘continent’, ‘funding type’ and ‘recruitment status’ on the variation of the mean %-COS-completion and a Pearson correlation between ‘enrolment number’ and the %-COS-completion. For this exploratory analysis, an alpha value of less than 0.001 was determined to be the significant threshold. Our analyses were computed using Stata/BE V.17.0 (StataCorp, LLC, College Station, Texas), R (V.4.2.1) and RStudio. All original data, final reconciled data and statistical analysis approaches have been uploaded to OSF (dataset).14 Results Clinical trial inclusions Our initial search returned 2152 trials from the ClinicalTrials.gov database. After filtering by date and trial phase, 364 trials remained for screening. Following screening of the 364 remaining trials, 205 additional trials were excluded. Of the 205 excluded, 197 were excluded for being the wrong disease and 8 were excluded for not being exclusively about MD. Our final sample consisted of 159 included trials for data extraction. A flow diagram of our study selection process can be found in figure 1. Flow diagram of inclusions and exclusions. MD, macular degeneration. Trial characteristics Most clinical trials we assessed occurred in 2014 (22/159; 13.8%), while 2011 had the fewest (3/159; 1.9%). The trials had a median enrolment of 205 participants and a median trial duration of 31 months. The vast majority of intervention types used were intravitreal anti-vascular endothelial growth factor (VEGF) treatments (132/159; 83.0%), while retinal radiation treatment (1/159; 0.6%) and surgical treatments (1/159; 0.6%) were the least used. Regarding recruitment status, 52.2% (83/159) trials were fully completed, while 1.3% (2/159) trials continued to enrol. Funding was most commonly provided by industry (126/159; 80.5%) and least commonly by private entities (2/159; 1.3%). Additional trial characteristics can be found in table 1. Table 1Trial characteristicsCharacteristicN=159Year, median (IQR)2017.0 (2014.0–2020.0)Phase, n (%) 393 (58.5) 466 (41.5)Continent, n (%) Multiple44 (27.7) North America38 (23.9) Asia37 (23.3) Europe28 (17.6) Not listed5 (3.1) Australia3 (1.9) South America3 (1.9) France1 (0.6)Recruitment status, n (%) Completed83 (52.2) Recruiting24 (15.1) Unknown18 (11.3) Terminated14 (8.8) Active, but no recruiting12 (7.5) Withdrawn4 (2.5) Enrolling by invitation2 (1.3) Not yet recruiting2 (1.3)Funding type, n (%) Industry110 (69.2) Multiple without industry18 (11.3) Multiple with industry16 (10.1) University10 (6.3) Hospital3 (1.9) Private2 (1.3)Enrolment number, median (IQR)205 (47–487)Trial duration, median (IQR)31 (23–40)Type of intervention, n (%) Intravitreal anti-VEGF treatment132 (83.0) Multiple15 (9.4) Other10 (6.3) Retinal radiation therapy1 (0.6) Surgical1 (0.6)VEGF, Vascular Endothelial Growth Factor. Trial characteristics VEGF, Vascular Endothelial Growth Factor. Analysis of COS Uptake There was no statistically significant change in uptake following publication of the MD COS. The mean %-COS-completion increased by 0.24% per month before publication (p=0.27) and by 0.07% per month after publication (p=0.62), indicating no meaningful post-publication slope change in COS use. All data in figure 2 were shown to be statistically non-significant (see online supplemental efigure 1 for descriptive yearly means). Mean percent completion per month versus date of trial initiation. The least commonly measured outcomes were ‘mobility and independence’ (30/159; 18.9%), ‘number of treatments’ (32/159; 20.1%), ‘emotional well-being’ (33/159; 20.8%) and ‘reading and accessing information’ (33/159; 20.8%). The most commonly measured items were ‘visual acuity’ (139/159; 87.4%), ‘complications of treatment’ (73/159; 46.8%) and ‘presence of fluid, oedema and haemorrhage’ (55/159; 35.3%). Because extraction was limited to the ‘Outcome Measures’ field, the estimate for ‘complications of treatment’ may be conservative if some trials recorded adverse events solely outside this field. Additional information regarding the frequency of outcome set uptake for MD clinical trials can be found in table 2. Table 2Frequency of core outcome measurement in macular degeneration clinical trialsOutcome set domain and item, n (%)N=159Visual functioning and vision-related quality of lifeDistance visual acuity139 (87.4)Mobility and independence30 (18.9)Emotional well-being33 (20.8)Reading and accessing information33 (20.8)Disutility of careNumber of treatments32 (20.1)Complications of treatment73 (46.8)Disease controlPresence of fluid, oedema or haemorrhage55 (35.3)Three trials indicated ‘complications of treatment’ and ‘presence of fluid/oedema/haemorrhage’ were not applicable (non-neovascular/dry AMD or geographic atrophy contexts). Percentages exclude these trials from the denominator.AMD, age-related macular degeneration. Frequency of core outcome measurement in macular degeneration clinical trials Three trials indicated ‘complications of treatment’ and ‘presence of fluid/oedema/haemorrhage’ were not applicable (non-neovascular/dry AMD or geographic atrophy contexts). Percentages exclude these trials from the denominator. AMD, age-related macular degeneration. Relationship between trial characteristics and outcome measures Our results demonstrated a statistically non-significant relationship between mean %-COS-completion for ‘continent’ (F=1.34, p=0.24, η2=0.05), ‘funding type’ (F=1.74, p=0.13, η2=0.06) and ‘recruitment status’ (F=1.33, p=0.24, η2=0.05). No statistically significant associations were observed between %-COS-completion and continent, funding type or recruitment status and the correlation with enrolment was small (r=0.26). Detailed statistics are provided in online supplemental etable 2. Discussion Our study demonstrates that the uptake of COS outcomes in MD clinical trials was inconsistent and that the release of the COS for MD has not significantly affected the measured outcomes in registered clinical trials. The most frequently reported outcome was visual acuity, with nearly 90% of trials measuring this domain; however, the remaining outcomes were reported in less than half of our sample. These findings were consistent both before and after the publication of the MD COS, as there was relatively no change in %-COS-completion overall. These results suggest that researchers conducting MD clinical trials are not considering the COS when deciding on outcomes to measure, indicating potential barriers to its use. Previous literature, such as Matvienko-Sikar et al, has reasoned that the lack of adherence to COSs is due to trialists’ lack of COS awareness, disagreement with COSs and inadequate resources for measuring core outcomes.18 The authors have also suggested that greater adherence to the standardised outcomes may occur in clinical trials over time, as the COS is disseminated across relevant healthcare fields.18 Williamson et al have also acknowledged many of these same barriers in their review concerning the relevancy of COSs within clinical research.19 A potential barrier to the uptake of COSs that has yet to be examined is the development of preferred outcomes by regulatory agencies such as the Food and Drug Administration (FDA) and European Medical Association (EMA). Despite studies observing a potential relationship between adherence to COSs and these agencies’ endorsed outcomes, there has yet to be a statistical analysis of the effects of these regulatory agencies on COSs.18,20 Future studies in this area are warranted. As previously mentioned, Kirkham et al were able to illustrate an increase in usage for the rheumatoid arthritis COS; however, they also noted that the rheumatoid arthritis COS had been endorsed shortly after its publication by two large governmental agencies—the FDA and the EMA.12 21 22 An additional analysis by Smith et al demonstrated a decrease in adherence to the osteoarthritis COS over time by clinical trialists, which they speculated to be due to the EMA changing their recommended outcomes that had previously aligned with those of the COS.20 23 Many of these clinical trials are being funded by drug companies that are seeking approval to commercialise their drugs, which has been shown to make trialists more likely to measure the outcomes endorsed by the regulatory agencies that grant the sought approval.12 24 25 Consistent with this conclusion, over 80% of the trials in the Kirkham et al analysis received funding from the medical industry, further substantiating the idea that regulators’ preferred outcomes are behind the patterns seen in COS usage.12 Although over 79% of the MD trials we reviewed received industry funding, there had been no official recommendations by a regulatory body for us to compare to the MD COS. Additionally, the current COS for MD did not report taking into account US or European regulatory agencies’ considerations when developing their guidelines.11 Currently, the FDA has an unofficial draft of its first ‘Guidance for Industry’ for age-related wet MD.26 Despite this document having various recommendations, the only endorsed domain that explicitly coincides with the MD COS is ‘distance visual acuity,’ our most commonly measured outcome.26 It is currently unknown whether subsequent updates to the COS in response to the final publication from the FDA will be made; and if there are none, it will be intriguing to see if future trials then show a corresponding decrease in COS adherence. We suggest that further studies analysing COS adherence take into account the role of these regulatory agencies. Additionally, studies analysing clinical trial outcomes in relation to FDA and EMA recommendations should be done to further explore the idea that regulatory agencies have a greater effect on measured domains than COSs. We also suggest that COS developers advocate for endorsement of the COS by relevant regulatory agencies (potentially via attendance at summits, such as the National Eye Institute/FDA endpoints workshop, or feedback on drafts of the industry guidance documents).27 Conclusion There were no discernible patterns in COS adherence in MD clinical trials before or after the publication of the MD COS. The only consistently reported outcome was visual acuity, while all other outcomes were measured in less than half of our sample. We speculate that previous patterns recorded concerning COSs were a result of regulatory agencies endorsing the use of certain outcomes rather than the COSs themselves. We suggest further studies be conducted on COSs in relation to the recommended outcomes of regulatory agencies, and we also recommend that those developing COSs seek regulatory agency endorsements for the use of their COS. Supplementary material Footnotes Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2025-104705). Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Ethics approval: Not applicable. Patient and public involvement: Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research. Data availability statement Data are available in a public, open access repository. References Supplementary Materials Data Availability Statement Data are available in a public, open access repository.