Mixed Martial Arts: Comparing the King-Devick and Sport Concussion Assessment Tool 5 in knockouts, technical knockouts and choke holds

Abstract

Objective

To compare validity indices of the King-Devick (KD) test and Sport Concussion Assessment Tool 5 (SCAT5) for traumatic events in MMA, and to determine if perfusion events (alterations in consciousness as the result of choke holds) cause similar changes in KD/SCAT5 scores.

Design

A prospective cohort study in MMA fighters who completed KD and SCAT5 assessments before and after a match. Outcomes were categorized as non-event, traumatic event, or perfusion event. KD/SCAT5 changes were compared between all athletes.

Participants

One hundred forty MMA athletes (7 women, 133 men), mean age=27.1 ± 4.9 years.

Intervention

N/A

Main outcome measures

King-Devick (KD) test and Sport Concussion Assessment Tool 5 (SCAT5)

Results

Among the 140 athletes, 19 sustained traumatic and 15 perfusion events. Testing provided sensitivities/specificities of 21.05%/93.39% (KD) and 77.78%/52.99% (SCAT5) in detecting a traumatic event. KD and SCAT5 Symptom Severity scores differed between athletes with and without traumatic events (P=.041 and .014). KD and SCAT5 Symptoms Score changes were observed between athletes with and without traumatic events (P=.023 and .042). Neither KD nor SCAT5 differed significantly between athletes with and without perfusion events.

Conclusions

The KD test provides high specificity and the SCAT5 demonstrates reasonable sensitivity when detecting a traumatic event. Of the SCAT5, symptoms-related scores may most effectively identify a traumatic event. A traumatic event may cause KD/SCAT5 changes similar to a concussion, while perfusion events did not.

Sports-related concussions (SRC) have an annual incidence of approximately 1.6-3.8 million, according to the Centers for Disease Control.¹ Concussion, also commonly known as a mild traumatic brain injury (TBI), consists of a complex pathophysiological process caused by traumatic biomechanical forces, inducing a neurometabolic cascade that may lead to a constellation of nonspecific clinical symptoms.^2,3 These symptoms are variable and include, among others, cognitive slowing, altered sensorium, headache, fatigue, photophobia, phonophobia, balance problems, nausea, vomiting, emotional lability, and sleep disturbance. SRC may not present with immediate onset of symptoms, and a precocious or inadvertent return to play may predispose to catastrophic injury (eg, second impact syndrome) or, potentially, long-term adverse effects.⁴ As a result, much attention has been focused on the development of objective screening tools to assist with identification of SRC.

Although computerized vestibular, neurologic and cognitive testing are used for concussion detection, these are time consuming, require specific equipment, and may need to be completed in an office setting, making them impractical for rapid sideline use.⁵ The Sports Concussion Assessment Tool (SCAT), with the most recent iteration being the SCAT5, has been widely adopted as a standardized sideline evaluation tool.⁶ It includes a post-concussion symptom scale, neurologic signs report, Glasgow Coma Scale (GCS), Standardized Assessment of Concussion (SAC), and Modified Balance Error Scoring System (M-BESS). Several limitations need to be considered within this test. A ceiling effect has been observed because of its relative simplicity.⁷ Participants with more cognitive reserve may be able to complete the test error-free despite significant cognitive injury,⁸ and scores can be affected by hydration status, depression, and gender.⁹ Lastly, it takes approximately 10 minutes to complete, limiting its utility in some sports environments. For these reasons, a simple, rapid, cost-effective, and reliable SRC assessment tool is needed.

The King-Devick (KD) test can be utilized as a SRC assessment tool by detecting subtle changes involved in vision, attention, and processing speeds.¹⁰ It is a non-invasive, brief reading instrument that tests the speed and accuracy of the athlete reading aloud a series of random, single-digit numbers from left to right, as one would read a book. It requires no special equipment, takes approximately 2 minutes to complete, is not affected by hydration status, depression, or gender,⁹ and has excellent test-retest reliability.11, 12, 13 Studies have demonstrated a slight learning effect with repeat KD testing due to improvement in times on subsequent testing^11,14, 15, 16 of approximately 2-5 seconds (s),^10,16, 17, 18 and that a delay (worsening) in KD test times by 5 s or greater, or an increase in test errors, is specific for SRC and TBI.^11,14

Several studies have reported that the KD test more accurately detects acute, mild TBI than does the SCAT.^19,20 It has been utilized in various sports including football, boxing, hockey, and rugby, among others.^10,17 Despite the frequency of head trauma sustained in Mixed Martial Arts (MMA) along with the sport's growing popularity in recent decades, the sensitivity and specificity of the KD test has yet to be fully assessed in MMA athletes. Meanwhile, the potential for perfusion injuries via choke holds in MMA poses a unique potential for brain injury not commonly seen in other sports²¹ and has yet to be widely studied. Application of the KD test in MMA athletes may improve the accuracy of detecting acute SRC, aid post-injury treatment, and guide decisions regarding return to sport.

This study had 2 specific goals. First, to compare validity indices (sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios [+LR and -LR]) of the KD and SCAT5 tests for traumatic events (knockout [KO]/technical knockout [TKO]) in MMA athletes after a match. Second, to evaluate if perfusion events (choke holds) cause similar changes in KD and SCAT5 scores to those seen after traumatic events.

Methods

Design and subjects

The study was approved by the institution's Institutional Review Board (IRB). Recruitment took place during pre-fight physicals at Minnesota MMA event centers. Athletes who expressed an interest in participating were provided a verbal and written description of the study. Inclusion criteria included men and women between the ages of 18-60 years who participated in a sanctioned MMA match between August 2018 and July 2020. Exclusion criteria included the presence of disease affecting balance or cognition, inability to read numbers, poor English fluency, or an inability to perform the tests for other physical or cognitive reasons. Athletes who met criteria and wished to participate were asked to sign an informed consent form prior to enrollment in the study.

The KD test was performed on an iPad for all participants. Athletes began with 1 practice (demonstration) page followed by 3 different, progressively more challenging test pages on which they were asked to read the numbers as quickly as possible without making any errors. Test times and errors were recorded as the sum of the 3 pages. Testing was administered in the quietest location possible in or near the athletes’ dressing rooms. Each test was performed by experienced health care providers, and the same health care provider performed the pre- and post-match KD and SCAT5 assessments on each athlete. Healthcare providers administering the assessments were not blinded to the outcomes of the matches.

Athletes retook the KD and SCAT5 tests within 15 minutes of completing their match and compared to pre-match scores. The outcome of the match was recorded for each athlete and categorized as either (1) traumatic event; (2) perfusion event; or (3) non-event, as indicated in table 1. An event was defined as an outcome leading to early cessation of the match¹⁰ and included KOs/TKOs (traumatic event) or choke holds (perfusion event). Perfusion events encompassed instances of choking that culminated in the athlete losing consciousness, as well as instances where the athlete voluntarily submitted by tapping out. Submissions due to non-choke holds (ie, arm bars, ankle locks, etc) were considered a non-event. A positive KD test, indicating SRC, was defined as either an increased testing time of 5 s or more or an increase in number of errors on post-match testing.^11,14 SCAT5 cutoffs consisted of a worsening of 3 or more points on the Symptoms Score, 5 or more on the Symptom Severity score, 3 or more on the SAC, 3 or more on M-BESS, 4 or more on the Tandem stance section of M-BESS, any errors on the double-leg stance of M-BESS, and any abnormality on the neurologic exam.22, 23, 24 For the purpose of data analysis in this study, these definitions, outlined in table 1, were determined prior to the start of the study. After their match, each athlete was classified as having either 1 of 3 event outcomes (traumatic, perfusion, or non-event) and assigned a “yes” or “no” value for the KD and SCAT5 to indicate a positive or negative test. Athletes were then contacted via telephone follow-up 1 week after the match to repeat the Symptoms Score and Symptom Severity score components of the SCAT5.

Table 1.

Definitions of events, positive KD test, and positive SCAT5 test. KD and SCAT5 test changes are based off pre-to-post match scores.

Event	Positive KD test	Positive SCAT5 test
Traumatic event: Any event in which the athlete suffered a TKO or KO resulting in early match cessation	Increase in 5 or more seconds to complete	Increase of 3 or more points on the Symptoms Score (ie,: number of symptoms)
Perfusion event: Any event in which the athlete suffered a choke hold resulting in early match cessation	Increase in number of errors	Increase of 5 or more points on the Symptom Severity score
Non-event: Neither of the above events occurring		Worsening of 3 or more points on the SAC
		Increase of 3 or more points on the M-BESS
		Increase of 4 or more points on the Tandem stance section of the M-BESS
		Any errors on the coordination test or the double-leg stance of the M-BESS
		Any abnormality on neurologic examination

Sample size

We determined sample size requirements using a nomogram from Malhotra & Indrayan.²⁵ Assuming the prevalence of “events” in MMA fighters is approximately 30%, and that worsening KD scores will have a sensitivity of 90% in distinguishing between concussed and non-concussed athletes,¹⁷ while accepting a ±10% level of precision in the sensitivity estimate, it was determined that at least 125 participants were required.

Data analysis

Descriptive statistics (means and SDs for continuous data, frequencies for categorical data) were calculated. Pre-to-post match changes in KD and SCAT5 scores were compared between athletes with and without events with a Mann-Whitney U test. Validity indices of measures from the KD test and SCAT5 were examined with receiver operating characteristic (ROC) curve analyses. Inferential statistics were conducted at alpha=.05. Data were analyzed with IBM SPSS Statistics Version 25^a and MedCalc diagnostic test evaluation software.^b

Results

The sample included 140 athletes (7 women, 133 men), mean age=27.1 ± 4.9 years. There were 34 total events that resulted in the early cessation of a match. These included 19 traumatic events (KO/TKO) and 15 perfusion events (choke holds), resulting in a combined event rate of 24.3% (95% CI=17.4% to 32.3%).

King-Devick

Mean pre-to-post match KD times for all athletes were 47.1 ± 7.8 and 44.8 ± 7.7 s, respectively, resulting in a mean improvement of 2.3 s. Post-match scores differed significantly between the traumatic event, perfusion event, and non-event groups (table 2, P=.023). Pre-to-post match changes in KD time improved among those with a non-event (mean change=-2.9 ± 4.2 s) but did not improve in athletes who experienced a traumatic event (mean change=0.0 ± 4.9 s, P=.028). The pre-to-post match change in athletes who experienced a perfusion event (mean change=-1.7 ± 3.4 s) did not differ significantly from either the traumatic event or non-event groups (P=.737 and .257, respectively).

Table 2.

Mean ± SD pre-to-post match KD and SCAT5 scores in the non-event, traumatic event (KO/TKO), and perfusion event (choke hold) groups.

	Pre-match	Post-match	Pre-Post Difference
Non-event group (n = 106)
KD time (s)	47.5 (8.5)	44.6 (8.4)	−2.9 (4.2)*ref.
SCAT5 Symptom Score (#)	2.2 (3.3)	2.2 (3.7)	0.0 (3.7)*ref.
SCAT5 Symptom Severity	3.4 (5.7)	4.3 (9.4)	0.9 (9.3)*ref.
SCAT5 SAC	26.4 (2.3)	25.5 (2.3)	−0.9 (2.5) ref.
SCAT5 M-BESS	2.9 (3.0)	3.4 (2.6)	0.5 (3.2) ref.
Traumatic event (KO/TKO) group (n = 19)
KD time (s)	44.6 (4.1)	44.6 (5.6)	0.0 (4.9)*P=.028
SCAT5 Symptom Score (#)	1.9 (2.9)	4.3 (5.0)	2.4 (5.1)*P=.043
SCAT5 Symptom Severity	2.8 (4.1)	8.3 (9.0)	5.5 (9.2)*P=.013
SCAT5 SAC	26.9 (2.2)	25.4 (2.4)	−1.5 (2.8) P=.549
SCAT5 M-BESS	2.4 (2.5)	3.9 (2.9)	1.5 (1.7) P=.095
Perfusion event (choke hold) group (n = 15)
KD time (s)	48.1 (5.0)	46.4 (4.7)	−1.7 (3.4) P=.257
SCAT5 Symptom Score (#)	1.2 (1.7)	1.2 (1.7)	0.0 (2.3) P=.786
SCAT5 Symptom Severity	1.9 (3.1)	2.6 (4.3)	0.7 (5.5) P=1.000
SCAT5 SAC	25.7 (3.7)	25.4 (3.8)	−0.3 (2.1) P=.410
SCAT5 M-BESS	4.0 (3.3)	5.2 (3.2)	1.2 (3.1) P=.553

NOTE: Values reported as mean ± SD.

Abbreviations: SAC, Standardized Assessment of Concussion; M-BESS, Modified Balance Error Scoring System; ref., reference.

^⁎Statistically significant (P<.05) differences between event groups and ref. values.

Based on ROC curve analysis, pre-to-post match changes in KD times differentiated between athletes who did and did not experience a traumatic event (area under the curve [AUC]=.646, 95% CI=.514-.779, P=.041) (fig 1A). However, pre-to-post match changes in KD time did not differentiate between athletes with and without a perfusion (choke hold) event (AUC=.568, 95% CI=.423-.713, P=.390) (fig 1B). Based on this study's pre-determined definition of a positive KD test as either a 5 s increase in time or any increase in errors, the KD test differentiated traumatic events from non-events with a sensitivity of 21.05% and a specificity of 93.39%, with a +LR of 3.18 and accuracy of 83.57% (table 3).

Fig 1.

ROC Curve of KD test comparing (A) traumatic events (KO/TKO) to non-events (AUC=.646, 95% CI=.514-.779, P=.041) and (B) perfusion events (choke holds) to non-events (AUC=.568, 95% CI=.423-.713, P=.390).

Table 3.

Validity indices of positive KD and SCAT5 tests in traumatic events.

	KD		SCAT5
Statistic	Value	95% CI	Value	95% CI
Sensitivity	21.05%	6.05% to 45.57%	77.78%	52.36% to 93.59%
Specificity	93.39%	87.39% to 97.10%	52.99%	43.55% to 62.28%
+LR	3.18	1.06-9.55	1.65	1.21-2.26
-LR	0.85	0.67-1.07	0.42	0.17-1.01
Disease prevalence*	13.57%	8.37% to 20.38%	13.33%	8.10% to 20.25%
Positive Predictive Value*	33.33%	14.29% to 60.00%	20.29%	15.69% to 25.82%
Negative Predictive Value*	88.28%	85.60% to 90.52%	93.94%	86.53% to 97.40%
Accuracy*	83.57%	76.38% to 89.29%	56.30%	47.50% to 64.81%

^⁎These values are dependent on disease prevalence.

ROC curve analysis provided an indication of which cutoff in KD change scores best balanced sensitivity and specificity to yield an optimized positive likelihood ratio (table 4). Using a negative 3 s or greater (improvement) pre-to-post match change in KD times or any increase in errors, for example, discriminated between athletes with and without events with 79.0% sensitivity (95% CI=54.4% to 94.0%) and 43.0% specificity (95% CI=34.0% to 52.3%), producing a +LR=1.38 (95% CI=1.05 to 1.83) and a –LR=0.49 (95% CI=0.20 to 1.20). A cutoff of 0 s or any increase in errors was 42.1% sensitive (95% CI=20.3% to 66.5 %) and 76.0% specific (95% CI=67.4% to 83.3%), producing a +LR=1.76 (95% CI=0.95 to 3.25) and a –LR=0.76 (95% CI=0.51 to 1.13). Using a 3 s cutoff or any increase in errors provided a sensitivity of 31.6% (95% CI=12.6% to 56.6%) and a specificity of 91.7% (95% CI=85.3% to 96.0%) producing a +LR=3.82 (95% CI=1.57 to 9.30) and a –LR=0.75 (95% CI=0.55 to 1.02).

Table 4.

Validity indices of various cutoffs in KD time changes (or increase in errors, with 95% confidence intervals) in traumatic events.

Statistic	-3 s change	0 s change	3 s change
True positives	15	8	6
False positives	69	29	10
True negatives	52	92	111
False negatives	4	11	13
Sensitivity	79.0% (54.4%-94.0%)	42.1% (20.3%-66.5%)	31.6% (12.6%-56.6%)
Specificity	43.0% (34.0%-52.3%)	76.0% (67.4%-83.3%)	91.7% (85.3%-96.0%)
+LR	1.38 (1.05-1.83)	1.76 (0.95-3.25)	3.82 (1.57-9.30)
-LR	0.49 (0.20-1.20)	0.76 (0.51-1.13)	0.75 (0.55-1.02)
Positive Predictive Value*	17.9% (14.1%-22.3%)	21.6% (13.0%-33.8%)	37.5% (19.8%-59.4%)
Negative Predictive Value*	92.9% (84.2%-97.0%)	89.3% (84.9%-92.6%)	89.5% (86.2%-92.1%)
Accuracy*	47.9% (39.4%-56.5%)	71.4% (63.2%-78.7%)	83.6% (76.4%-89.3%)

^⁎These values are dependent on disease prevalence.

SCAT5

Pre-to-post match changes in SCAT5 Symptoms Score (number of symptoms) showed a statistically significant mean change in score between the non-event, traumatic event, and perfusion event groups (table 2, P=.045). Pre-to-post match changes in Symptoms Score were greater in athletes who experienced a traumatic event (mean change=2.4 ± 5.1 symptoms) than those in the non-event group (mean change=0.0 ± 3.7 symptoms, P=.042). None of the other SCAT5 components demonstrated a statistically significant mean change pre-to-post match or between event groups (all P>.05). Overall, the SCAT5 differentiated traumatic events from non-events with a sensitivity of 77.78%, specificity of 52.99%, -LR of 0.42, and accuracy of 56.30%.

Based on ROC curve analysis, pre-to-post match changes in Symptom Severity score on the SCAT5 statistically differentiated between athletes with and without a traumatic event (fig 2A, AUC=.679, 95% CI=.555-.804, P=.014). Pre-to-post match changes in all other SCAT5 components did not differentiate between athletes with and without traumatic events (fig 2A), and none of the SCAT5 components differentiated between those with and without perfusion events (fig 2B, all P>.05).

Fig 2.

ROC Curve of SCAT5 test comparing (A) traumatic events (KO/TKO) to non-events. Only changes in SCAT5 Symptom Severity scores provided statistically significant differentiation (AUC=.679, 95% CI=.555-.804, P = .014) between athletes with and without traumatic events, while other components did not (all P>.05). (B) SCAT5 test comparing perfusion events (choke holds) to non-events. Pre-to-post match changes in SCAT5 measures did not differentiate between athletes with and without perfusion events (all P>.05).

At 1-week telephone follow-up, 105 of the 140 athletes (75.0%) were available for response. Of the 19 traumatic event athletes, 12 responded, and 0 reported worsened SCAT5 symptom scores. Of the 15 perfusion event athletes,10 responded, and 0 reported worsened symptom SCAT5 scores. 81 of the 106 non-event athletes were contacted at 1-week follow-up, and 2 (2.5%) reported worsened SCAT5 symptom scores.

Discussion

This study investigated the use of the KD and SCAT5 tests to evaluate traumatic and perfusion injury events in a series of MMA fighters. Our results demonstrated that KD and SCAT5 tests can successfully differentiate a traumatic event from a non-event. Based on ROC curve analysis and validity indices, the KD test demonstrated greater accuracy compared to the SCAT5 when differentiating athletes with and without traumatic events. Our data also provide insight on how a clinician can use the KD test as a tool to determine the clinical diagnosis of a possible concussion using different time cutoffs to define a positive test. We did not find statistically significant differences in KD or SCAT5 scores in the perfusion event group compared to the traumatic event or non-event groups.

King-Devick

Defining a positive KD test as either a 5 s increase in time or any increase in errors, our study showed that a KD test demonstrated a sensitivity of 21.05% and specificity of 93.39% (table 3). Other studies suggest a sensitivity closer to 62%-86%.^17,18 However, this may be secondary to different cutoffs being used. For example, Naidu et al used a cutoff of any time (0 s) to indicate a positive test, resulting in a sensitivity of 62%.¹⁸ In our cohort, a 0 s cutoff resulted in a sensitivity of 42.1%. We observed a 2.9 s improvement in healthy individuals on subsequent KD testing, comparable to previous studies that range from an improvement of 1.9 to 5.0 s.^10,17,18 Because of this improvement, one could argue that any athlete who falls short of an expected improvement (for example, -3 s cutoff) could be considered for further concussion evaluation.

The KD test cannot be used as a standalone test to assess for concussion and rather needs to be considered as a component of the clinical diagnosis. If the goal is to more confidently rule in a concussion-related injury, a cutoff of 3 to 5 s as the operational definition of a positive test yields the greatest accuracy rate while maximizing the specificity and positive likelihood ratio in our cohort. If the provider's goal is to use the KD test to more confidently rule out a concussion-related injury, the -3 s change threshold optimizes sensitivity and negative likelihood ratio. However, a more pragmatic approach may be to use a 0 s cutoff, as this has shown reasonable sensitivity in other studies as well and is an easy number for health care providers to compare to.

In a prior prospective cohort study investigating the effect of an ‘event’, defined as a KO, TKO, choke hold, or submission, on KD tests, 92 MMA athletes underwent KD testing prior to and after a workout or match. Results revealed KD tests worsened in a majority of athletes after an ‘event’ (n=21). KD tests improved by a mean of 5.0 s after a standard workout or after a match in which no event occurred in a majority of cases (n=69). The conclusions suggested KD tests may assist with identifying MMA athletes who have suffered any event related to brain injury.¹⁰

SCAT5

Like the KD test, the SCAT test can be interpreted in several ways. Our SCAT5 data suggest that symptoms-related components of the SCAT5 may be the most clinically relevant, and other literature suggests these components demonstrate the highest test-retest reliability.²⁶ The test as a whole provides reasonable sensitivity but lower specificity, and this has been supported in other studies as well.²⁷ This suggests the SCAT5 may be most useful as a “rule out” sideline concussion tool.

Currently, there is a need for widely established, validated rules or guidelines for interpreting change in performance from baseline on the SCAT5 or SCAT3 in professional or amateur athletes.²² This study used cutoffs based on prior work by Hanninen et al., who evaluated interpreting changes in various aspects of the SCAT3 in Finnish professional hockey players.^10,22, 23, 24 They determined normative values and test-retest reliability based on preseason baseline SCAT3 testing during two-consecutive seasons.22, 23, 24 Based on the results of the study, 10% or fewer of the athletes showed the following worsening of SCAT3 retest scores compared to baseline: an increase of 3 or more points on the Symptoms Score, an increase of 5 or more points on the Symptom Severity score, a worsening of 3 or more points on the SAC, an increase of 3 or more error points on the M-BESS, or an increase of 4 or more s on the Tandem Gait. Any errors on the Coordination test and the double-leg stance of the M-BESS were considered abnormal.²³ The final series of the Finnish study compared the previously determined normative values of SCAT3 testing to those found the day of concussion. The percentage of players with uncommon day-of-injury performance compared to their preseason baseline for Symptoms Score, Symptoms Severity score, SAC, M-BESS, Tandem Gait, and coordination were, respectively, 96%, 96%, 33%, 46%, 18%, and 8%.²²

Limitations

This study has several limitations. As this was a prospective cohort study, randomization as well as a controlled comparison were lacking. Because of the nature of the study design and the regional MMA events, health care administrators were unable to be blinded to the outcome of the match results. Sample size was relatively small, with only 34 events available for analysis. Only 1 KD test was performed at baseline; some studies suggest that baseline KD testing should incorporate the mean of 2 tests to minimize the known learning effect associated with subsequent testing.⁷ A thorough past medical history was not available to exclude confounding factors such as prior concussion or brain injury, migraine, ADHD, depression/anxiety, and other learning disabilities that have been associated with delayed recovery from concussion.²⁸ Further research is needed on KD and SCAT5 sideline concussion testing, as well as further investigation within the subgroups of traumatic vs perfusion injuries in MMA.

Conclusions

This study demonstrates that the KD test is a reasonable and specific sideline test to consider when evaluating for traumatic brain events, while the SCAT5 provides high sensitivity. Using a cutoff of 3-5 s to signify a positive KD test provides a high +LR to identify a traumatic event. The KD test may more accurately identify traumatic events compared to the SCAT5. Of the various SCAT5 components, the Symptoms Score and Symptoms Severity scores may most effectively identify a traumatic event. MMA athletes suffering a traumatic event may have sustained a brain injury similar to a concussion, while perfusion events do not cause similar changes in KD or SCAT5 scores compared to traumatic events.

Suppliers

• a.IBM SPSS Statistics Version 25; IBM Corp.
• b.MedCalc diagnostic test evaluation software; MedCalc Software Ltd.