Role of video review for sport-related concussion identification: a systematic review Authors: Dominic Charles Townsend, Abigail Saker, Katie Scandrett, Matthew Green, Michael Brownlow, Paddy Riley, Mark Gillett, Antonio Belli Published: 2025 ABSTRACT AbstractObjectiveTo determine the frequency and diagnostic utility of visible signs identified through video review for sport-related concussion detection and to evaluate the role of injury mechanism in predicting concussions.DesignSystematic review.Data sourcesMEDLINE, SPORTDiscus, Web of Science and Cochrane Library from 2001 to June 2024.Eligibility criteriaOriginal, peer-reviewed studies focusing on sport-related concussion, where video review identified visible signs or the mechanism of injury relevant to clinical decision-making. Studies were included irrespective of study design but had to be published in English and use a clinical diagnosis or consensus definition of sport-related concussion as a reference standard.ResultsOut of 1001 records screened, 29 studies involving 3281 sport-related concussions were included for quantitative synthesis. Only eight studies had a low risk of bias. All studies used clinical diagnosis as a reference standard; 26 were conducted in professional competitions, with two studies including female athletes. The most frequently observed visible sign was ‘slow to get up’ (37.6%), followed by ‘motor incoordination’ (26.7%). Sensitivity of visible signs was generally low (<50%), while specificity was high (>90%) across most studies. Four studies across three sports examined mechanism of injury characteristics, identifying common themes, such as high speed of impact, potentially associated with sport-related concussion occurrence. Inter-rater and intrarater reliability varied, with indications of higher reliability among expert raters.ConclusionsExpert video review can reliably identify visible signs of sport-related concussion. However, due to their low frequency and limited diagnostic sensitivity, the diagnosis of sport-related concussion should be supported by a multimodal assessment. WHAT IS ALREADY KNOWN ON THIS TOPIC Sport-related concussion is a challenging clinical diagnosis, emphasising the importance of early identification to mitigate potential adverse outcomes and guide management. Sport-related concussion is a challenging clinical diagnosis, emphasising the importance of early identification to mitigate potential adverse outcomes and guide management. Overt signs of sport-related concussion, such as loss of consciousness, are reported in less than 10% of cases. Overt signs of sport-related concussion, such as loss of consciousness, are reported in less than 10% of cases. The combined approach of video review and multimodal assessment has shown benefits in aiding the clinical diagnosis of sport-related concussion across various contact sports. However, an evidential summary of the findings and utility of video review for the diagnosis of sport-related concussion has not been performed. The combined approach of video review and multimodal assessment has shown benefits in aiding the clinical diagnosis of sport-related concussion across various contact sports. However, an evidential summary of the findings and utility of video review for the diagnosis of sport-related concussion has not been performed. WHAT THIS STUDY ADDS Among the 29 studies meeting inclusion criteria, 6 studies (718 sport-related concussions) evaluated the diagnostic accuracy of individual visible signs. Most visible signs showed low sensitivity (<50%) and high specificity (>90%). Among the 29 studies meeting inclusion criteria, 6 studies (718 sport-related concussions) evaluated the diagnostic accuracy of individual visible signs. Most visible signs showed low sensitivity (<50%) and high specificity (>90%). 10 studies reported on the occurrence of visible signs using similar definitions. The two most frequently observed visible signs were ‘slow to get up’ (37.6%) and ‘motor incoordination’ (26.7%). This was followed by ‘no visible sign seen’ in almost a quarter of diagnosed concussions (23.6%). 10 studies reported on the occurrence of visible signs using similar definitions. The two most frequently observed visible signs were ‘slow to get up’ (37.6%) and ‘motor incoordination’ (26.7%). This was followed by ‘no visible sign seen’ in almost a quarter of diagnosed concussions (23.6%). Given the low frequency and sensitivity of visible signs, this adjunctive tool should be complemented by a multimodal assessment for the diagnosis of sport-related concussion. Future studies should address methodological limitations, including the inclusion of female athletes and consider how sport-specific mechanisms of injury influence risk-prediction models. Given the low frequency and sensitivity of visible signs, this adjunctive tool should be complemented by a multimodal assessment for the diagnosis of sport-related concussion. Future studies should address methodological limitations, including the inclusion of female athletes and consider how sport-specific mechanisms of injury influence risk-prediction models. Introduction The assessment and management of athletes who have sustained a sport-related concussion has developed significantly in recent years, supported by an international consensus definition and frequent systematic reviews of scientific literature guiding research efforts.1 Despite these advancements, sport-related concussion remains a challenging clinical diagnosis. Early identification is crucial to mitigate potential adverse outcomes and facilitate timely assessment before implementing appropriate management strategies for a safe return to play.2 Identifying sport-related concussion early is challenging as overt signs, such as loss of consciousness, are present in less than 10% of cases.3 Moreover, beyond the infrequent occurrence of visible signs, additional challenges include the fast-paced and often chaotic environment of elite-level sports, where medical staff may have obscured views or be distracted by injuries elsewhere on the field.2 In the absence of a definitive pitch-side biomarker for identifying sport-related concussion, the recognition of suspected cases is best approached using a multimodal side-line assessment tool like the Sport Concussion Assessment Tool (SCAT), complemented by video review capabilities.2 When available, video review allows clinicians to review incidents in detail, potentially capturing brief early signs of sport-related concussion following impact. Consequently, many professional contact sports have incorporated video review into their sport-related concussion management protocols.4 The adoption of video review processes varies across sports, influenced more by resource availability and sport-specific demands rather than strictly scientific evidence.4 However, the effectiveness of video review is influenced by the availability and quality of footage, which is often limited at the non-professional level due to fewer camera angles and lower resolution.5 Nevertheless, the combination of video and SCAT-based assessments has shown benefits in diagnosing sport-related concussion in various contact sports.6 7 Despite differing practical protocols, there are commonalities across sports in the visible signs examined through video review.4 International consensus definitions of six primary concussion video signs have provided clarity and promoted consistent research, with the objective of enhancing match-day detection and standardised recognition practices.8 This systematic review aims to evaluate the existing body of evidence on: The prevalence of visible signs observed through video review in sport-related concussions. The prevalence of visible signs observed through video review in sport-related concussions. Inter-rater reliability in identifying visible signs of sport-related concussion via video review. Inter-rater reliability in identifying visible signs of sport-related concussion via video review. The diagnostic accuracy of visible signs identified in sport-related concussion cases. The diagnostic accuracy of visible signs identified in sport-related concussion cases. The role of injury mechanisms in predicting sport-related concussion. The role of injury mechanisms in predicting sport-related concussion. This review includes original research articles from various sports and observers with diverse levels of experience. Methods Study design and reporting This systematic review follows the evidence-based minimum standards outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA) (online supplemental eFigure 1) and implementing PRISMA in Exercise, Rehabilitation, Sports medicine and SporTs science (PERSiST) guidelines.9 10 The review was not preregistered prior to it being conducted. Eligibility criteria Studies eligible for inclusion had to be peer-reviewed, written in English, provide full-text articles (not abstract only), report ethical approval and participant consent where applicable, and meet the eligibility criteria (online supplemental eTable 1). In this systematic review, the primary focus was on concussion sustained during sports competition or training (sport-related concussion), where video review was used to identify visible signs or mechanism of injury relevant to clinical decision-making. Studies were required to employ either a clinical diagnosis by a medical doctor or apply a consensus definition of sport-related concussion as the reference standard for diagnosing sport-related concussions. Search strategy The Population, Index test, Target condition framework guided the identification of key topics for inclusion in the search (online supplemental eTable 2).11 Synonymous search terms related to these key topics were identified to ensure comprehensive coverage (online supplemental eTable 3). To enhance sensitivity, additional search terms were incorporated after screening titles, abstracts and full texts of relevant studies initially identified. These terms were trialled in pilot searches before integration into the final search strategy. The inclusion start date was set as 1 January 2001, aligning with the establishment of the first consensus definition of sport-related concussion.12 Searches were independently conducted on MEDLINE, SPORTDiscus, Web of Science and Cochrane Library from 1 November 2022 to 30 November 2022 by the principal reviewers (DCT and AS). To maintain currency, an updated search was conducted up to and including June 2024. Boolean search command “OR” was used to combine the main concept with associated synonyms, and the command “AND” was used to include all three main concepts within the search. The “*” symbol was used for wildcard character searching as needed. Screening strategy and study selection All citations identified by the search strategy were managed in EndNote (Clarivate Analytics, Philadelphia, Pennsylvania, USA). Duplicate studies were identified and removed. Screening initially involved reviewing titles and abstracts. If inclusion could not be determined from these alone, the full text was consulted briefly. Two reviewers (DCT and AS) independently performed duplicate removal and initial screening. Any disagreements on inclusion were resolved through discussion. If consensus could not be reached, a third reviewer (AB) made the final decision. Full texts of all the remaining articles underwent a second screening phase. Two reviewers (DCT and AS) independently assessed each article against the eligibility criteria to ensure correct inclusion decisions. Any discrepancies were resolved through discussion or by consulting the third reviewer (AB) as necessary. The study selection process is visually represented in the PRISMA flow chart (figure 1). PRISMA flow chart for the screening process. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Data extraction Data from the selected studies were extracted into an Excel spreadsheet by the lead reviewer (DCT). Extracted data encompassed methodological characteristics, participant demographics, study characteristics and study findings (table 1). For this review, ‘expert’ raters were defined as personnel with relevant prior experience or training in video review, otherwise, they were defined as ‘novice’ raters. Table 1Data extraction details by categoryCategoryData extraction detailsMethodological characteristics Study aims and designLevel of evidenceStudy environment (including which sport the study was conducted in) Participant demographics GenderAgeParticipant numberPlaying positionStandard of competition Study characteristics Concussion: definition, method of case confirmationVideo analysis: technology used, application of analysis, sign description, sign occurrence, mechanism of injury characteristics, sensitivity and specificity, likelihood ratios, diagnostic ORs, predictive models, receiver operating characteristic curve dataRater: level of experience, intrarater and inter-rater reliabilityOutcome measures: diagnosis, player removal from competition/training, duration of absence, return to play progression, postconcussion syndrome Study findings Main resultsLimitations Data extraction details by category Study aims and design Study aims and design Level of evidence Level of evidence Study environment (including which sport the study was conducted in) Study environment (including which sport the study was conducted in) Gender Gender Age Age Participant number Participant number Playing position Playing position Standard of competition Standard of competition Concussion: definition, method of case confirmation Concussion: definition, method of case confirmation Video analysis: technology used, application of analysis, sign description, sign occurrence, mechanism of injury characteristics, sensitivity and specificity, likelihood ratios, diagnostic ORs, predictive models, receiver operating characteristic curve data Video analysis: technology used, application of analysis, sign description, sign occurrence, mechanism of injury characteristics, sensitivity and specificity, likelihood ratios, diagnostic ORs, predictive models, receiver operating characteristic curve data Rater: level of experience, intrarater and inter-rater reliability Rater: level of experience, intrarater and inter-rater reliability Outcome measures: diagnosis, player removal from competition/training, duration of absence, return to play progression, postconcussion syndrome Outcome measures: diagnosis, player removal from competition/training, duration of absence, return to play progression, postconcussion syndrome Main results Main results Limitations Limitations A second reviewer (AS) independently verified the data extraction process. Any discrepancies were resolved through consensus or consultation with a third reviewer (AB). Attempts were made to obtain missing data by contacting study authors via email. Risk of bias assessment Each reviewer (DCT and AS) independently performed a risk of bias assessment, with discrepancies resolved by a third reviewer (AB). Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies, version 2 (QUADAS-2) framework.13 Studies were classified as having a ‘low risk of bias’ if they were rated as ‘low’ in all domains. Studies were considered ‘at risk of bias’ or had ‘concerns regarding applicability’ if one or more domains were rated as ‘high’ or ‘unclear’.13 Statistical analysis Data synthesis and statistical analysis were performed based on available study data. Results are mostly presented descriptively. For diagnostic test accuracy data, forest plots were generated to illustrate individual study estimates of sensitivity and specificity for each of the visible signs of sport-related concussion (the index tests), along with 95% CIs. A bivariate random-effects logistic regression model was fitted for each index test separately using the ‘meqrlogit’ command in Stata V.16.0 to estimate summary sensitivity and specificity values, as recommended in the Cochrane handbook for systematic reviews of diagnostic test accuracy.14 We planned to conduct a comparative meta-analysis to compare index tests with a summary sensitivity and specificity greater than 50%. All analyses were conducted using RevMan V.5.4 and Stata V.16.0. Equity, diversity and inclusion statement All eligible studies were included in this review without exclusion based on participant characteristics such as sex, gender, ethnicity, socioeconomic and competition level. The discussion section addresses the importance of enhancing diversity within the evidence base on this topic. The research team comprises individuals of diverse backgrounds, including men and women from various educational backgrounds, professional disciplines and levels. Results Study selection and characteristics The initial search across electronic databases yielded 1001 articles. After removing duplicates and applying eligibility criteria, 29 studies were included (figure 1) with a combined total of 3281 sport-related concussions undergoing video review. From the information available in the studies, the sport-related concussion cases included in them are not known to overlap. Table 2 presents a summary of key study characteristics. Table 2Characteristics of the included studies in the final reviewFirst author, yearStudy typeSportCompetitionSexNumber of concussions(n)Criteria used for analysisRater experienceBlindedInter-rater reliability consideredMechanism of injury consideredVisible signs consideredAthiviraham et al17RCCSBBMLBM9BCSExpertYesNoYes (Helmet hit-by-pitch only)NoAustin et al37RCSAFNCAA Division 1M44ICDExpertNRNoYesYesBruce et al18RCSIHNHLM202Echemendia et al21 Hutchison et al32NRNRNoYesYesCassoudesalle et al38RCSSCRLigue 1, Ligue 2M36ICDExpertNRYesYesYesClark et al31RCSAFNFLM429BCSNRNRNoYesNoCross et al19RCCSRUThe English Premiership, The Pro 12, The Rugby World CupM182BCSExpertNRNoYes (Tackles only)NoDavis and Makdissi20PCSAUSAFLM45Hierarchical flow chartExpertYesNoNoYesDavis et al8QQAUS, CK, BB, AF, IH, RL, RUAFL, CA, MLB, NFL, NHL, NRL, WRM0QuestionnaireExpertNRNoNoYesDavis et al4QQAUS, CK, BB, AF, IH, RL, RUAFL, CA, NFL, NHL, NRL, WRM0QuestionnaireExpertNRNoNoYesEchemendia et al21RCSIHNHLM202NHL protocolExpertYesYesNoYesElbin et al22PCSSAFNFLM211Makdissi and Davis41ExpertNRNoNoYesGardner et al39RCSRLNRLM20BCSExpertNRNoYesYesGardner et al23RCSRLNRLM60ORACExpertYesYes (and intrarater reliability)NoYesGardner et al40RCSRLNRLM25ORACExpert and noviceNRYesYesYesGardner et al24RCSRLNRLM20BCSExpertNRNoNoYesGardner et al25RCSRLNRLM60ORACNRYesNoYesYesGardner et al26RCSRUSuper RugbyM34HIA protocolExpertYesNoNoYesGardner27PCSSRLNRLM106ICDExpertYesNoNoYesHutchison et al32PCSIHNHLM199Heads-up checklistExpert and noviceNRYesYesNoHutchison et al33RCSIHNHLM197Heads-up checklistNRNRNoYesNoKent et al34RCSAFNCAA Division 1M42BCSExpertNRNoYesNoLessley et al35RCSAFNFLM367BCSExpertYesNoYesNoMakdissi and Davis28PCSAUSAFLM45Makdissi and Davis41ExpertYesYes (and intrarater reliability)NoYesMakdissi and Davis 41PCSAUSAFLM82BCSExpertYesYesYesYesO'Halloran et al29PCSCKFirst-Class County Championship, One-Day 50 over, T20 cricket and Men’s senior national sideM16ICD, McCrory et al45 Concussion protocols of the ICC, WR and FIFANRNRNoYes (Helmet strikes only)YesSpiegelhalter et al15RCSRLWSLF12Hopkinson et al46ExpertNRNoYes (Tackles only)NoSunderland et al16RCSAUSAFL, AFLWMF252/44BCSNRNRYes (MOI only)YesNoSuzuki et al30RCSRUCollegiate competitionM33Suzuki et al47NRNRNoYes (Tacklers only)NoZuckerman et al36RCS+PCSAFNFLM307Hierarchical flow chart20Expert and noviceNRYesNoYesStudy type coded as: RCCS, RCS, PCS, PCSS, QQ.Sport coded as: BB, AF, IH, SCR, RU, AUS, CK, RL.Number of concussions indicates the number of clinically diagnosed concussions.Criteria used for analysis coded as: BCS, ICD,8 NHL protocol, ORAC, HIA protocol.Blinded to clinical outcome refers to; ‘yes’ if the study methodology actively states the rater was blinded, ‘no’ if the study methodology actively states the rater was not blinded and ‘NR’ if the study methodology does not mention either.AF, American Football; AFL, Australian Football League; AUS, Australian Football; BB, baseball; BCS, Bespoke criteria to study; CA, Cricket Australia; CK, cricket; HIA, World Rugby Head Injury Assessment protocol; ICC, intraclass correlation coefficient ; ICD, International Consensus Definitions; IH, Ice Hockey; MLB, Major League Baseball; MOI, mechanism of injury; NCAA, National Collegiate Athletic Association; NFL, National Football League; NHL, National Hockey League; NR, not reported; NRL, National Rugby League; ORAC, Observational Review and Analysis of Concussion; PCCS, prospective cross-sectional study; PCS, prospective cohort study; QQ, Qualitative Questionnaire; RCCS, retrospective case–control study; RCS, retrospective cohort study; RL, Rugby League; RU, Rugby Union; SCR, Association Football/Soccer; WR, World Rugby. Characteristics of the included studies in the final review Study type coded as: RCCS, RCS, PCS, PCSS, QQ. Sport coded as: BB, AF, IH, SCR, RU, AUS, CK, RL. Number of concussions indicates the number of clinically diagnosed concussions. Criteria used for analysis coded as: BCS, ICD,8 NHL protocol, ORAC, HIA protocol. Blinded to clinical outcome refers to; ‘yes’ if the study methodology actively states the rater was blinded, ‘no’ if the study methodology actively states the rater was not blinded and ‘NR’ if the study methodology does not mention either. AF, American Football; AFL, Australian Football League; AUS, Australian Football; BB, baseball; BCS, Bespoke criteria to study; CA, Cricket Australia; CK, cricket; HIA, World Rugby Head Injury Assessment protocol; ICC, intraclass correlation coefficient ; ICD, International Consensus Definitions; IH, Ice Hockey; MLB, Major League Baseball; MOI, mechanism of injury; NCAA, National Collegiate Athletic Association; NFL, National Football League; NHL, National Hockey League; NR, not reported; NRL, National Rugby League; ORAC, Observational Review and Analysis of Concussion; PCCS, prospective cross-sectional study; PCS, prospective cohort study; QQ, Qualitative Questionnaire; RCCS, retrospective case–control study; RCS, retrospective cohort study; RL, Rugby League; RU, Rugby Union; SCR, Association Football/Soccer; WR, World Rugby. Among the included studies, 27 focused exclusively on male athletes, one on female athletes15 and one compared both sexes.16 All studies used a clinical sport-related concussion diagnosis as the reference standard. 15 studies also reviewed head impact events that did not result in a clinical diagnosis (n=22 076) in addition to the sport-related concussion cases; this was primarily for comparison purposes.151730 The majority of studies were retrospective (n=19) and performed in professional sporting competitions (n=26). The number of concussion cases varied greatly (ranging from 0 to 429), as did the framework used for video analysis—most commonly performed by expert raters (n=19). 10 studies exclusively analysed the mechanism of injury through video review,151719 30 while 11 studies focused solely on visible signs,48 2024 26 and 8 studies examined both elements in combination.1825 29 3741 Risk of bias and applicability On review, eight studies were categorised as having a ‘low’ risk of bias,48 20 23 2628 41 while the remaining studies were either deemed ‘at risk of bias’ (n=11)151719 24 25 31 32 34 35 39 or had ‘concerns regarding applicability’ (n=10)1821 22 29 30 33 3638 40 (online supplemental eTable 4). Instances categorised as ‘at risk of bias’ were predominantly related to case selection procedures, where enrolment criteria were inadequately described, or a significant proportion of injury video footage was not available. ‘Concerns regarding applicability’ largely stemmed from uncertainties surrounding rater blinding. 10 studies clearly stated whether raters were blinded to the reference standard, whereas others did not provide clear information on this aspect (table 2). Frequency of visible signs occurrence Studies varied in their approach to assigning visible signs to individual cases of sport-related concussion. In a greater number of studies (n=6), raters assigned multiple visible signs to each case,212327 36 39 compared with others assigning only a single visible sign (n=5).2024 3638 Table 3 illustrates the frequency of occurrence of the visible signs across studies, showing considerable variability. This is with the exception of ‘impact seizure’, ‘tonic posturing’ and ‘visible facial injury’, which also demonstrate relatively low occurrence frequencies across the studies. Notably, on average ‘slow to get up’ (37.6%) and ‘motor incoordination’ (26.7%) were the most frequently occurring visible signs. These were followed by ‘no visible sign seen’ (23.6%). Table 3Concussion counts and visible sign types and frequenciesFirst author, yearSportNumber of concussion (n)Lying motionless (%)Motor incoordination (%)No protective action–floppy (%)Impact seizure (%)Tonic posturing (%)Blank/ vacant look (%)Slow to get up (%)Clutching head/ face (%)Visible facial injury (%)No visible sign seen (%)Austin et al37AF449.111.4––2.3–11.429.5–22.7Cassoudesalle et al38SCR3611.1–––––––––Davis and Makdissi20AUS45416252–7229114Echemendia et al21IH2026.417.3–––239.67.9553Elbin et al22AF21118.828.417.65.7–1.765.929.70.726Gardner et al39RL20306645––53–35–0Gardner et al23RL604058.3–5–7510063.3–0Gardner et al40RL2055––––––––20Gardner27RL10620.836.820.82.85.753.8–––43.4Zuckerman et al36 (novice raters)AF969.5155.50–021.54.5043.5Zuckerman et al36 (expert raters)AF21118.528.517.56–1.56629.50.526.5Average percentage %(range %)21.2(4–55)26.7(11.4–66)19.1(5.5–45)2.9(0–6)5.3(2.3–5.7)20.5(0–75)37.6(11.4–100)23.4(4.5–63.3)2.2(0–11)23.6(4–53)Sport coded as: AF, IH, SCR, AUS, RL.Number of concussions indicates the number of clinically diagnosed concussions.Studies where the visible sign was either not considered or not reported is denoted by ‘–’.AF, American Football; AUS, Australian Football; IH, Ice Hockey; RL, Rugby League; SCR, Association Football/Soccer. Concussion counts and visible sign types and frequencies Sport coded as: AF, IH, SCR, AUS, RL. Number of concussions indicates the number of clinically diagnosed concussions. Studies where the visible sign was either not considered or not reported is denoted by ‘–’. AF, American Football; AUS, Australian Football; IH, Ice Hockey; RL, Rugby League; SCR, Association Football/Soccer. Two additional studies reported the occurrence of visible signs,26 29 but these studies took a different approach to categorisation and defining so were not included in table 3. One of these studies reported the presence or absence of visible signs rather than individual occurrences, finding visible signs in 55.9% of 34 concussion cases.26 The others used different descriptive terms for visible signs in their cohort of 16 concussion cases, with the most frequent being ‘player removes or checks helmet’ (75%) and ‘more than 4 s to resume’ (68.8%).29 Visible signs: assessment of diagnostic test accuracy The diagnostic test accuracy of individual visible signs was assessed in six studies included in this review,182123 27 28 which were chosen for their consistent and detailed definitions of visible signs. Key diagnostic accuracy measures consistently reported across these studies included sensitivity, specificity, positive predictive value and negative predictive value. Notably, the study by Elbin et al22 contributed two separate data sets to the analysis, considering diagnostic accuracy measures independently for each rater involved. Figure 2 presents the diagnostic test accuracy data in forest plots, with studies ordered based on sensitivity. Summary sensitivity and specificity values for each of the visible signs are presented in table 4. None of the studies achieved a summary sensitivity and specificity greater than 50%, precluding a comparative meta-analysis. For most visible signs, sensitivity was low (<50%) and specificity was high (>90%). The opposite was true for ‘slow to get up’ with a summary sensitivity of 80% and a summary specificity of 27%. Forest plots displaying study sensitivity and specificity of individual visible signs. FP, false positive; FN, false negative; TN, true negative; TP, true positive. Table 4Sensitivity and specificity of the visible signsVisible signsStudies (n)SRC cases/total number of casesSummary sensitivity (95% CI)Summary specificity (95% CI)Lying motionless7929/37600.20 (0.13 to 0.30)0.96 (0.92 to 0.98)Motor incoordination7929/37600.32 (0.24 to 0.42)0.91 (0.87 to 0.94)No protective action–floppy4573/15470.18 (0.13 to 0.23)0.96 (0.91 to 0.98)Impact seizure5633/19870.05 (0.04 to 0.07)1.00 (0.97 to 1.00)Blank/vacant look6835/28990.10 (0.02 to 0.39)0.99 (0.95 to 1.00)Slow to get up5729/19560.80 (0.46 to 0.95)0.27 (0.14 to 0.45)Clutching head/face5729/19560.27 (0.14 to 0.46)0.69 (0.55 to 0.80)Visible facial injury4669/15730.03 (0.01 to 0.12)0.92 (0.77 to 0.97)SRC, sport-related concussion. Sensitivity and specificity of the visible signs SRC, sport-related concussion. Another study examined the diagnostic test accuracy of individual visible signs; however, it used different descriptive terms and definitions and did not report all diagnostic accuracy measures, so it was not included in this analysis.29 In that study, the visible sign with the highest sensitivity was ‘player removes or checks helmet’ (sensitivity 0.92, specificity 0.60) and ‘more than 4 s to resume’ (sensitivity 0.92, specificity 0.52).29 Four additional studies investigated the diagnostic test accuracy of multiple visible signs rather than individual visible signs21 22 25 27 (online supplemental eTable 5). The limited number of studies precluded the calculation of summary sensitivity and specificity values. However, a general trend observed was that as the number of visible signs present increased, sensitivity tended to decrease while specificity increased. Similarly, positive predictive values and negative predictive values generally increased with the number of visible signs present. Mechanism of injury characteristics Four studies examined how characteristics of the mechanism of injury were statistically associated with a diagnosis of sport-related concussion.18 19 25 30 Online supplemental eTable 6 outlines the mechanism of injury characteristics that either significantly increased or decreased the likelihood of a sport-related concussion diagnosis across these studies. The studies encompassed three sports—ice hockey, rugby league and rugby union—highlighting challenges in directly comparing mechanisms of injury due to sport-specific differences. Nonetheless, common themes emerged: mechanism of injury characteristics such as high speed of impact, direct head contact with a solid object and foul play consistently increased the likelihood of sport-related concussion. Conversely, lower speed and reduced overall force of impact were associated with decreased likelihood of sport-related concussion. In addition, two studies focused on the frequency of mechanism of injury characteristics, both involving female athletes.15 16 In women’s rugby league, sport-related concussion incidence was higher among tacklers (3.5/1000 head contacts, 95% CI 0.9 to 9.5) and during knee-to-head contact (25.6/1000 knee-to-head contact events, 95% CI 16.7 to 37.7), with minimal influence from team standard or time in the match.15 Comparatively, in Australian Football competitions, female athletes had higher sport-related concussion rates following actions like bumping (incidence rate ratio (IRR)=8.11, 95 % CI 1.36 to 48.53), being bumped (IRR=4.63, 95 % CI 2.05 to 10.46) or being tackled (IRR=2.66, 95 % CI 1.17 to 6.03).16 Female athletes also showed increased sport-related concussion rates following contact with the ground (IRR=2.79, 95% CI 1.45 to 5.37), opposing player’s head (IRR=3.29, 95% CI 1.64 to 6.61) or pelvis/hip (IRR=5.53, 95% CI 1.92 to 15.91).16 Reliability Six studies examined the inter-rater reliability of individual visible sign identification21 23 28 36 40 41 (online supplemental eTable 7). Among these, two studies explored agreement between novice and expert raters,36 40 while another two focused on intrarater reliability.23 28 Cohen’s kappa coefficients (Κ) were most frequently chosen as the statistical measure of inter-rater reliability (n=4).23 36 40 41 Across these studies, experts generally demonstrated higher levels of agreement compared with novices.36 40 However, variability in performance was observed, with Κ values for novice raters ranging from 0.44 (weak) to 0.68 (moderate), and for expert raters ranging from 0.08 (none) to 0.89 (strong). Notably, one study41 significantly influenced expert rater variability, showing minimal to no agreement across all visible signs using the McHugh Κ agreement classification.42 Higher inter-rater agreement was consistently observed for visible signs such as ‘lying motionless’, ‘slow to get up’ and ‘clutching head/face’, while ‘blank/vacant look’ exhibited lower agreement levels. One study exclusively reported inter-rater reliability for mechanism of injury characteristics, indicating ‘almost perfect’ agreement (Κ=0.95, 95% CI 0.83 to 1.00).16 Intrarater reliability was assessed for individual visible signs among expert raters only.23 28 In Australian Football, the intraclass correlation coefficient (ICC) demonstrated varied agreement levels, with ‘visible facial injury’ showing the highest ICC (rater 1=1.00, rater 2=0.93) and ‘blank/vacant’ look the lowest (rater 1=0.63, rater 2=0.36).28 In rugby league, K demonstrated moderate to perfect intrarater reliability for all visible signs identified by two expert raters, with ‘impact seizure’, ‘slow to get up’ and ‘clutching head/face’ showing perfect agreement.23 The visible sign with the lowest level of intrarater agreement, although still moderate, was ‘blank/vacant look’.23 Discussion The integration of video review into protocols for identifying sport-related concussion has become integral across major international sporting organisations.4 However, variations in definitions for individual visible signs among protocols across different sports were noted, encompassing a total of 19 distinct visible signs. The only universally recognised signs across all protocols were ‘loss of consciousness’ or ‘lying motionless’ and ‘motor incoordination’.4 In many sports, the identification of these signs on video mandates permanent removal of the athlete from play (eg, Australian Football League, World Rugby, National Football League, Cricket Australia, National Rugby League), while in others, such as the National Hockey League, removal for further assessment is required.4 Despite many sports using the term ‘loss of consciousness’, it is recognised that a clinical assessment is required to diagnose this sign and cannot be done on video review alone. The preferred term therefore when performing video review is ‘lying motionless’.4 This systematic review represents the first comprehensive assessment of the scientific evidence investigating the role of video review in sport-related concussion identification. The increasing adoption of this technology in professional sports organisations’ protocols underscores its importance in guiding clinicians’ decision-making regarding athlete management on the field of play. Therefore, this review will in part help to inform the application of video review. With many of the studies identified being retrospective in design, it is recognised that future studies exploring real-time use are required to better inform practice. Despite growing scientific and clinical interest in sport-related concussion over recent decades, our review identified a relatively modest total of 29 studies. Most of these studies were retrospective and included professional athletes. This number contrasts with 98 studies included in a review focusing on more advanced technologies for sport-related concussion assessment.43 Only two studies included female athletes,15 16 and sample sizes across studies were generally small, with heterogeneous criteria for video review. Given the critical role of video review in current protocols and clinical decision-making, we expected a larger and more consistent evidence base underpinning its application. The size of the evidence base may reflect limited access to video review technology due to resource constraints in research and applied settings, outside of professional competitions. Additionally, anonymising the injury video footage is challenging, introducing further ethical considerations for researchers. Concerns regarding bias and applicability were notable, with only 28% of studies considered to have low risk of bias. The remaining 72% were either at high risk of bias or had concerns regarding applicability. This raises questions about the generalisability of findings across different sport settings and for female athletes, who may be at higher risk of concussion but are underrepresented in research.44 Addressing methodological issues such as including all consecutive cases, studies conducted in both sexes, ensuring rater blinding, performing real-time review and adopting international visible signs consensus definitions to standardise assessments is important for future research efforts to overcome deficiencies currently present and establish a more homogenous evidence base.8 Future studies should also consider how adopting this technology and definitions impacts clinical practice and outcomes. Detailed comment on para-sport participants goes beyond the scope of this review, but it is important that dedicated studies of this athlete group are completed as this body of literature develops. In diagnosing sport-related concussion, athletes may disclose symptoms, display overt visible signs or exhibit findings on examination, or a combination thereof, following a direct or indirect head impact. Historically, loss of consciousness was pivotal in confirming sport-related concussion diagnoses, but a broader range of visible signs is now recognised.8 Recent studies indicate that loss of consciousness occurs in less than 10% of cases,3 yet our review suggests a higher occurrence of lying motionless in professional male settings (average 21.2%, range 4%–55%). However, the most frequent visible signs identified were ‘slow to get up’ and ‘motor incoordination’. Notably, ‘no visible sign seen’ occurred in 23.6% of cases, and with the non-specific nature of ‘slow to get up’, this highlights that video review cannot be solely relied on to inform clinical decision-making and should be supported by a multimodal sideline assessment.1 2 Factors contributing to variations in the presence of visible signs across studies and sports may include helmet use, mechanisms of injury variations across sports and potential strategic advantages in sports for remaining on the ground.23 Examining the diagnostic test accuracy of individual visible signs, our review identified only six studies suitable for direct comparison due to inconsistent definitions of visible signs.182123 27 28 Overall, no single visible sign demonstrated both high sensitivity and specificity, indicating no clinically useful standalone test for the diagnosis of sport-related concussion. Most visible signs had a low sensitivity (<50%) and a high specificity (>90%), suggesting their utility in prompting athlete removal while cautioning against ruling out sport-related concussion in their absence. ‘Slow to get up’, however, showed relatively high sensitivity (80%) and low specificity (27%), suggesting its usefulness in identifying potential sport-related concussion cases following a significant direct or indirect head impact, and the need for athletes to receive comprehensive multimodal sideline assessment. Even fewer studies considered combined visible signs to diagnose sport-related concussion. Those that did demonstrated that as the number of visible signs increased, sensitivity decreased while specificity, positive predictive value and negative predictive value increased.21 22 25 27 Future studies should explore the role of mechanism of injury characteristics alone or in combination with visible signs to enhance diagnostic accuracy or help identify athletes that require subsequent clinical assessment. Four studies examined how mechanism of injury characteristics alone may influence the likelihood of diagnosing sport-related concussion.18 19 25 30 The applicability of these study results across different sports may be limited, as two studies focused exclusively on specific sports-related mechanisms of injury.19 30 Overall, mechanisms of injury that increase or decrease the likelihood of a subsequent sport-related concussion diagnosis are related to the force or speed of impact. It is evident that mechanisms of injury differ between sexes;16 therefore, future sport-specific and sex-specific research is warranted when studying mechanisms of injury. An inherent limitation of visible signs is that they are subject to human error, as evidenced by studies examining inter-rater reliability.21 23 28 36 40 41 Variability in reliability may not solely stem from the performance of the individual raters. Contributing factors include the quality of available video footage, including both the visual quality and number of camera angles, and the precision of visible signs definitions.41 Recommendations for sideline medical personnel include access to real-time, multiangle camera feeds including broadcast footage for accurate review, with the ability to replay at various speeds.4 International consensus definitions may increase precision if widely adopted, as they facilitate a degree of standardisation which is important as the clinical diagnosis is the same regardless of setting.8 Using expert raters appears beneficial, given their demonstrated higher reliability,36 40 with many sports organisations also requiring these individuals to be experienced doctors.4 It could, therefore, be advantageous for individuals to demonstrate both characteristics. Studies on intrarater reliability have shown acceptable levels,23 28 suggesting that while individual reliability may be adequate, ongoing education in visible signs identification could enhance consistency across raters. Several international sporting organisations (National Football League, National Hockey League, National Rugby League, World Rugby) already offer specific educational modules for this purpose.4 Recommendations for future research We propose several recommendations for future research aimed at addressing the current limitations in the evidence base and enhancing the use of video review in sport-related concussion identification within organisational protocols: Enrolment of larger and representative samples: Include larger and more diverse samples, encompassing female athletes, a broader spectrum of sports and various competition levels. Enrolment of larger and representative samples: Include larger and more diverse samples, encompassing female athletes, a broader spectrum of sports and various competition levels. Prospective, blinded study designs: Employ prospective, consecutive study designs where raters are blinded to the clinical outcome. Prospective, blinded study designs: Employ prospective, consecutive study designs where raters are blinded to the clinical outcome. Investigate the impact on clinical practice of standardised visible signs definitions: Possibly adopting the international consensus definitions for this purpose.8 Investigate the impact on clinical practice of standardised visible signs definitions: Possibly adopting the international consensus definitions for this purpose.8 Exploration of cross-sport patterns: Investigate whether there are overarching patterns or mechanistic explanations for the variability in visible signs occurrence across different sports. Exploration of cross-sport patterns: Investigate whether there are overarching patterns or mechanistic explanations for the variability in visible signs occurrence across different sports. Diagnostic test accuracy studies: Conduct diagnostic accuracy studies on both individual and combined visible signs across a wider array of sports to inform decisions regarding mandatory athlete removal or further assessment. Diagnostic test accuracy studies: Conduct diagnostic accuracy studies on both individual and combined visible signs across a wider array of sports to inform decisions regarding mandatory athlete removal or further assessment. Sport-specific mechanism of injury characteristics: Gain a better understanding of sport-specific mechanism of injury characteristics and their role in risk-prediction models. Sport-specific mechanism of injury characteristics: Gain a better understanding of sport-specific mechanism of injury characteristics and their role in risk-prediction models. Improving inter-rater reliability: Explore the reasons behind variations in inter-rater reliability and evaluate interventions aimed at enhancing reliability. This could include stepwise video review process guidance or educational programmes. Improving inter-rater reliability: Explore the reasons behind variations in inter-rater reliability and evaluate interventions aimed at enhancing reliability. This could include stepwise video review process guidance or educational programmes. Study limitations Several limitations should be considered regarding this review. Eligibility criteria may have inadvertently excluded relevant studies, such as those not accessible in English or without full-text access. This review does not encompass evidence specific to individuals under 18 years old or participants in para-sports, and only two studies included female athletes. Despite efforts to standardise definitions, subtle variations across studies introduce potential errors when calculating summary sensitivity and specificity values. Finally, potential publication bias may influence overall results and conclusions. Conclusions This systematic review aims to enhance the understanding of the role of video review in sport-related concussion identification and its optimal integration into sporting organisations’ management protocols. The results support the use of expert-led video review for visible signs identification. However, given its limited diagnostic accuracy, this approach should complement a comprehensive sideline multimodal assessment. Further research should address the current methodological limitations to improve generalisability, explore standardised visible signs definitions and the potential benefits of incorporating the mechanism of injury characteristics. Supplementary material