Do Sideline Concussion Assessments Predict Subsequent Neurocognitive Impairment After Sport-Related Concussion?

Abstract

Context: 

Comprehensive, multidomain assessment is the standard of care after sport-related concussion. However, the relationship between performance on sideline concussion-assessment tools and in-office computerized neurocognitive testing has received little attention, and the prognostic utility of sideline measures is unknown.

Objective: 

To evaluate concurrent impairment on commonly used concussion measures 24 to 48 hours postinjury while also determining the predictive utility of sideline measures on computerized neurocognitive testing in the acute to subacute recovery periods postinjury.

Design: 

Case-control study.

Setting: 

High school and collegiate athletics.

Patients or Other Participants: 

A total of 125 high school and college-aged athletes (85 males, 40 females) 14 to 23 (16.8 ± 2.21) years old.

Main Outcome Measure(s): 

Participants were administered sideline concussion-assessment measures (ie, Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT], Standardized Assessment of Concussion [SAC], and Balance Error Scoring System [BESS]) 24 to 48 hours postinjury and completed ImPACT and the Post-Concussion Symptom Scale 5 to 7 and 10 to 14 days postinjury. Outcome measures were the ImPACT composite (verbal memory, visual memory, reaction time, visual-motor speed), SAC, and BESS scores and total symptom score on the Post-Concussion Symptom Scale.

Results: 

Participants demonstrated heterogeneous patterns of impairment on measures 24 to 48 hours postinjury, with the most common pattern being impairment on ImPACT and the SAC. Performance on the SAC and BESS at 24 to 48 hours after injury did not distinguish between those with and those without impairment on ImPACT at 5 to 7 days postinjury (χ² = 5.076, P = .079) or 10 to 14 days postinjury (χ² = 2.04, P = .361).

Conclusions: 

More than 90% of athletes were impaired on at least 1 sideline or neurocognitive measure 24 to 48 hours after sport-related concussion. Although sideline measures are useful for concussion diagnosis, they are not suitable for prognostication of impairment or the presence of symptoms 1 to 2 weeks postinjury.

Key Points

• Using multiple sideline measures for concussion assessment improved the diagnostic yield compared with any standalone measure.

• After concussion, athletes were most likely to have deficits on both sideline and computerized cognitive measures 24 to 48 hours postinjury.

• Although sideline measures were useful for identifying and diagnosing concussion, they were not effective in prognosticating neurocognitive impairment or symptom reports at 1 to 2 weeks postinjury.

Current estimates suggest that 1.6 million to 3.8 million sport- and recreation-related concussions (SRCs) occur every year in the United States.¹ Between 1997 and 2007, emergency room visits for SRCs increased 100% for 8- to 13-year-olds and 200% for 14- to 19-year-olds.² The SRC is a heterogeneous injury^3,4 that requires a multifaceted assessment approach, including measures of neurocognitive functioning, balance, and self-reported symptom inventories. A comprehensive assessment approach is used for diagnosis and clinical management throughout recovery until the athlete is cleared to return to play. Although research is inconsistent regarding which measures possess the best sensitivity for diagnosing concussion, there is consistent support for the increased sensitivity of comprehensive assessment batteries compared with single, stand-alone measures.^5–7 For example, Broglio et al⁶ found that the sensitivity of a comprehensive test battery exceeded 90%. Similarly, Resch et al⁵ demonstrated that a multidimensional approach incorporating neurocognitive testing, balance testing, and symptom reports correctly identified 80% to 100% of concussed athletes as injured, whereas when each test was evaluated separately, up to 47.5% of the sample was misclassified. Using scores and performance information from multiple assessments, a clinician can reduce the tendency to rely too heavily on self-reported symptoms, which can result in missed concussion diagnoses and premature return-to-play decisions.⁸ It is important to note that studies of comprehensive assessment batteries have primarily focused on acute assessment for the purpose of diagnostics rather than clinical management (eg, using scores to prognosticate recovery or referral for specific treatment) postinjury.

Although the multifaceted approach to concussion has been widely advocated via consensus statements⁹ and has been substantiated by research,⁵ the relationship among commonly used tools administered at the same time or at different times after recovery has not received much attention. This is surprising, given the varying postinjury time windows in which it is appropriate to use various measures.^9,10 For example, sideline tools are generally most sensitive up to 72 hours postconcussion and are not appropriate for use as stand-alone measures for return-to-play decisions due to limited sensitivity and specificity outside this window. Similarly, it is not feasible to use computerized neurocognitive testing on the sideline of a sporting event. However, understanding the relationships among these measures may offer valuable, practical information regarding the athlete's anticipated recovery and promote realistic expectations for a recovery course and return to play.

In theory, performances on sideline and office-based tools may be related if they measure the same constructs of brain functioning, such as the neurocognitive abilities of attention and memory. Furthermore, in some instances, similar tests are incorporated into sideline and in-office evaluations. For example, both the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) and the Standardized Assessment of Concussion (SAC) include a list-learning task that measures immediate and delayed memory for words. Whereas ImPACT demands reading skills and visual processing and tests recognition, the SAC involves a shorter, orally administered list-learning task that requires executive functioning and free recall. In short, these tests assess both similar and distinct constructs, and the relationship between the 2 tests warrants additional research. Although cognition and balance are largely considered different assessment domains, performance measures of balance were found to be significantly correlated with memory scores on computerized neurocognitive testing among patients completing vestibular therapy,¹¹ suggesting a relationship between cognitive and vestibular functioning that warrants further research. Given the overlap in these constructs, further investigation is needed into the concurrent impairment and predictive utility of cognitive and balance measures.

The purpose of our study was to evaluate concurrent impairment on commonly used concussion measures 24 to 48 hours postinjury while also determining the predictive utility of sideline measures on computerized neurocognitive testing in the acute to subacute recovery periods postinjury. Specifically, our objectives were to describe the relationships among frequently used tools, including ImPACT, the SAC, and the Balance Error Scoring System (BESS) at 24 to 48 hours postinjury and to evaluate the prognostic utility of sideline tools (ie, SAC, BESS) in the acute phase (24–48 hours postinjury) to predict return to baseline on ImPACT at 5 to 7 and 10 to 14 days postinjury. We hypothesized that most athletes would have at least 1 impairment at 24 to 48 hours postconcussion, given the acute nature of injury and research suggesting that multiple assessment measures increase the likelihood of detecting impairment. We also hypothesized that worse scores on the SAC and BESS would predict ongoing impairment on ImPACT and persistent symptoms 5 to 7 and 10 to 14 days postinjury.

METHODS

Participants and Procedures

Between August 2009 and December 2011, we collected prospective, repeated-measures SRC data on high school and college-aged athletes who were part of a multisite study in California, Louisiana, Michigan, and Pennsylvania. Data for all measures were collected at baseline (ie, preinjury), <24 hours, 24 to 48 hours, 5 to 7 days, and 10 to 14 days postinjury, with the exception of ImPACT, which was not administered at <24 hours postinjury. Study inclusion criteria were age 14 to 22 years, participation in a scholastic sport, complete data for all follow-up intervals, and valid baseline computerized neurocognitive data and symptom assessment. We obtained self-reported demographic data from all participants for age, sex, and number of previous concussions. Exclusion criteria, documented by a brief medical history questionnaire, were a history of any of the following: moderate to severe traumatic brain injury, brain surgery, major psychiatric or neurologic disorder, or substance abuse. Athletes with a history of concussion; migraine; or learning disability or attention–deficit/hyperactivity disorder (LD/ADHD) were included. The study was approved by the coordinating site's university institutional review board under an expedited protocol. After a study informational meeting and before enrollment in the study, participants and their parents (if the participants were minors) provided written informed consent.

Definitions and Measures

Sport-Related Concussion.

A concussion was defined as a “complex pathophysiological process affecting the brain, induced by biomechanical forces.”⁹ Athletes with suspected concussions were diagnosed by a licensed medical professional (eg, physician, neuropsychologist, certified athletic trainer) trained in the assessment and treatment of concussion using the following criteria: (1) clear mechanism of injury, (2) presence of signs at time of injury (eg, loss of consciousness, posttraumatic amnesia, disorientation, or confusion), (3) current symptoms (eg, headache, dizziness, nausea), and (4) one or more areas of cognitive impairment (eg, memory, processing speed, reaction time). All concussions occurred during sport participation.

Concussion Symptoms.

The Post-Concussion Symptom Scale (PCSS) was used to assess SRC symptoms. The PCSS contains 22 self-report SRC symptoms (eg, headache, fogginess, dizziness) rated on a 0- (none) to 6- (severe) point Likert scale. The total PCSS score ranges from 0 to 132. For the purpose of this study, an asymptomatic score was defined as a score within the reliable change of baseline and a symptomatic score reflected an athlete with more symptoms than at baseline.

Neurocognitive Testing.

The ImPACT is a computer-based neurocognitive test battery comprising 6 subtests designed to examine neurocognitive impairment in individuals with an SRC: (1) verbal memory, (2) design memory, (3) X's and O's, (4) symbol matching, (5) color matching, and (6) three-letter memory. Data from the 6 subtests are collapsed into 4 composite scores: verbal memory (percentage correct), visual memory (percentage correct), visual-motor processing speed (a higher number is a better score), and reaction time (seconds). The ImPACT takes approximately 20 to 25 minutes to administer. For the purpose of this study, not impaired performance on ImPACT was defined as a score within the reliable change of baseline and impaired was defined as below baseline, outside of the reliable change.

The SAC is a brief cognitive screening measure that is used for sideline assessment in the sport setting. It measures orientation, immediate memory, concentration, and delayed memory.¹² This measure has also been shown to be sensitive in detecting impairments up to 48 hours after SRC. On the basis of prior performance measures among healthy participants,^12,13 we defined impairment as a conservative cutoff of <26 points. As such, not impaired performance was defined as a SAC score of 26 to 30, and impaired referred to a score <26.

Balance Testing.

The BESS is a clinical balance assessment developed to evaluate static and dynamic postural stability after SRC.¹⁴ A trained observer assesses 6 balance conditions (3 conditions with feet on the floor and 3 conditions with feet on a foam pad) while the participant's eyes are closed. The total BESS score is determined by counting the number of errors across all conditions, with higher scores representing worse balance. A comprehensive description of and detailed psychometric properties for the BESS are provided elsewhere.¹⁵ Interrater reliability was not calculated. Given the variability in normal performance across age and sex, we adopted a conservative cutoff score of 20 errors to identify impairment.¹³ For the purpose of this study, <20 errors across all 6 conditions was defined as not impaired and ≥20 errors as impaired performance.

Data Analysis

All analyses were conducted using SPSS software (version 22.0; IBM Corp, Armonk, NY). Statistical significance of P < .05 was used for all noncorrected analyses. Due to established differences in cognitive functioning among athletes with a history of LD/ADHD¹⁶ and mixed findings for concussion history,¹⁷ we conducted independent-samples t tests and χ² analyses to explore any potential impairment differences between groups and to determine whether these subpopulations were overrepresented at any postinjury time point. Descriptive statistics, including frequencies, were used to note the number of athletes impaired on each measure at 24 to 48 hours postinjury. Logistic regression analyses were used to analyze the predictive utility of acute measures of concussion impairment (ie, SAC and BESS at 24–48 hours postinjury) on subacute measures of neurocognitive impairment (ie, ImPACT at 5–7 days and 10–14 days postinjury) and symptom reports (5–7 days and 10–14 days postinjury).

RESULTS

Demographics

A total of 125 high school and college-aged athletes (85 males, 40 females) participated in this study. Participants were 16.8 ± 2.21 years old (range, 14–23 years; Table 1) and had typical impairments consistent with SRC at 24 to 48 hours postinjury (Table 2). No differences in age were present between those who were measured as impaired or not impaired on ImPACT at 5 to 7 days (t = 0.19, P = .85) or 10 to 14 days (t = 0.06, P = .81) postinjury. Forty-seven participants (37.6%) reported no prior concussions, whereas the remaining participants reported an average of 1.0 ± 1.1 previous concussion (range, 0–5 concussions). The previous number of concussions did not differ between impairment groups at either 5 to 7 days postinjury (t = 1.62, P = .11) or 10 to 14 days postinjury (t = 0.54, P = .46). Of the 125 participants, 6 reported being diagnosed with ADHD and 3 reported being diagnosed with LD. In addition, no differences were observed between impaired and not-impaired groups on ImPACT for the presence of ADHD at 5 to 7 days postinjury (χ² = 0.08, P = .77) or 10 to 14 days postinjury (χ² = 1.06, P = .30). Similarly, no differences were evident between groups for the diagnosis of an LD at 5 to 7 days postinjury (χ² = 1.96, P = .16) or 10 to 14 days postinjury (χ² = 2.81, P = .09). Fifteen athletes reported being treated by a physician for migraines; these athletes were equally represented between groups at 5 to 7 days postinjury (χ² = 1.10, P = .30) and 10 to 14 days postinjury (χ² = 3.40, P = .07).

Table 1. .

Participant Demographics (N = 125)

Variable	Mean ± SD
Age, y	16.8 ± 2.2
	n (%)
Sex
Males	85 (68.0)
Females	40 (32.0)
History of concussion	78 (62.4)
Treatment for migraine	18 (14.4)
Diagnosed learning disability	3 (2.4)
Diagnosed attention-deficit/hyperactivity disorder	6 (4.8)
Current sporta
Football	71 (57.3)
Women's soccer	12 (9.7)
Volleyball	8 (6.5)
Women's basketball	8 (6.5)
Men's soccer	5 (4.0)
Wrestling	5 (4.0)
Men's basketball	5 (4.0)
Women's ice hockey	5 (4.0)
Softball	4 (3.2)
Men's ice hockey	1 (0.8)

^aN = 124; data were missing for 1 participant.

Table 2. .

Measures of Concussion Assessment at 24 to 48 Hours Postinjury (N = 125)

Variable	Mean ± SD
Neurocognitive scores
Verbal memory	74.55 ± 14.91
Visual memory	64.30 ± 16.21
Motor processing speed	34.74 ± 9.38
Reaction time	0.65 ± 0.13
Impulse control	9.06 ± 8.26
Total symptom score	25.24 ± 22.04
Standardized Assessment of Concussion total score	25.68 ± 2.93
Balance Error Scoring System total score	17.13 ± 7.82

Acute Measures of Concussion

The means and standard deviations of performance across measures at 24 to 48 hours are summarized in Table 2. Whereas 21 of 125 athletes (18.4%) were impaired on all 3 measures (ie, ImPACT, SAC, and BESS), 114 of 125 (91.2%) were impaired on at least 1 measure (Figure). Impairment was captured most frequently on ImPACT: 83 of the 125 athletes (66.4%) were identified as impaired. The SAC captured impairment in 79 of the 125 athletes (63.2%), and the BESS showed impairment in 44 of the 125 athletes (35.2%). The most common pattern involved impairment on ImPACT and SAC (28.0%). It is notable that 69 athletes (60.5%) remained impaired on ImPACT at 5 to 7 days postinjury, but only 24 athletes (21.1%) were considered symptomatic (ie, symptom score outside reliable change compared with baseline). At 10 to 14 days postinjury, 40 athletes (42.1%) were impaired on at least 1 ImPACT composite, whereas only 7 athletes (7.4%) were symptomatic.

Figure.

Relationship among concussion-assessment measures 24 to 48 hours postinjury. Abbreviations: BESS, Balance Error Scoring System; ImPACT, Immediate Post-Concussion Assessment and Cognitive Testing; SAC, Standardized Assessment of Concussion.

Predictive Utility of Sideline Testing for Neurocognitive Impairment

We used logistic regression analyses to assess the predictive utility of acute measures (ie, SAC, BESS) administered at 24 to 48 hours after injury on impairment status (ie, score within reliable change, not impaired; score outside reliable change, impaired) for ImPACT at 5 to 7 days and 10 to 14 days postinjury. For the first analysis, a test of the full model, including SAC and BESS performance at 48 hours postinjury, against a constant-only model was not statistically significant, indicating that performance on the SAC and BESS at 24 to 48 hours after injury did not distinguish between those with and those without impairment on ImPACT at 5 to 7 days postinjury (χ² = 5.076, P = .079; Table 3). In the second analysis, the full model was again tested against the constant-only model to explore the predictive power of 48-hour SAC and BESS performance on impairment on ImPACT at 10 to 14 days postinjury. The model was not statistically significant in predicting impairment (χ² = 2.04, P = .361; Table 3).

Table 3. .

Logistic Regression Examining Predictive Utility of Acute Measures on Neurocognitive Impairment at 5 to 7 and 10 to 14 Days Postinjury

Variables	β	SE	P Value	Odds Ratio (95% Confidence Interval)
Impaired at 5–7 d
SAC	0.134	0.074	.071	1.14 (0.99, 1.32)
BESS	0.046	0.026	.079	1.05 (0.99, 1.10)
Impaired at 10–14 d
SAC	−0.064	0.073	.385	0.94 (0.81, 1.08)
BESS	0.022	0.028	.425	1.02 (0.97, 1.08)

Abbreviations: BESS, Balance Error Scoring System; SAC, Standardized Assessment of Concussion.

Predictive Utility of Sideline Testing for Symptom Impairment

Logistic analyses were used again to determine whether acute measures (ie, SAC, BESS) at 24 to 48 hours postinjury predicted impairment status (ie, return to baseline) for symptom reports at 5 to 7 and 10 to 14 days postinjury. For the first analysis, SAC and BESS performance at 48 hours after injury did not predict being symptomatic or asymptomatic at 5 to 7 days postinjury (χ² = 3.470, P = .176) or 10 to 14 days postinjury (χ² = 1.185, P = .553).

DISCUSSION

The purpose of our study was to evaluate concurrent impairment on commonly used concussion measures 24 to 48 hours postinjury while also determining the predictive utility of sideline measures (ie, SAC, BESS) on computerized neurocognitive testing and symptom reporting in the acute (5–7 days) to subacute (10–14 days) recovery periods postinjury. Participants demonstrated heterogeneous patterns of impairment on measures 24 to 48 hours postinjury, as indicated by a majority of athletes (65.6%) displaying impairment on 2 or 3 of the 3 measures administered, and 91.2% of the sample demonstrating at least 1 impaired performance. Second, sideline measures (ie, SAC, BESS) administered 24 to 48 hours postinjury did not predict impairment status on computerized neurocognitive testing or symptom status at 5 to 7 or 10 to 14 days postinjury.

The pattern of impairment on commonly used concussion measures in our study highlights the heterogeneity of injury and is consistent with other authors' reports of increased sensitivity for a comprehensive battery compared with individual stand-alone measures,⁶ lending support to the calls of previous researchers^5,6 for a comprehensive assessment of SRC. On a comprehensive battery, a majority of athletes (91.2%) will be detected as impaired on at least 1 objective measure of concussion at 24 to 48 hours postinjury. These findings also support the heterogeneous, clinical profiles–based approaches to concussion proposed by investigators.^3,4,18–20 Our study highlights one of the most common clinical profiles, neurocognitive dysfunction. It is interesting that approximately one-third of the sample demonstrated the most common pattern, impairment on both ImPACT and SAC at 24 to 48 hours postinjury. This is consistent with the results of researchers²¹ who reviewed cognitive deficits in the acute stage of concussion recovery.

Our study was unique in being the first to attempt to use sideline assessment measures to predict return to baseline on computerized neurocognitive testing and symptom severity reports at a later time point. Investigating these relationships is important for understanding the prognosis and developing a concussion-management plan. In contrast to our hypothesis, scores on the SAC and BESS at 24 to 48 hours postinjury did not predict neurocognitive recovery (ie, performance within reliable change on ImPACT) at 5 to 7 or 10 to 14 days postinjury. These findings highlight the individual nature of recovery among different domains or aspects of brain functioning being measured (eg, balance versus cognition). Henry et al²² reported that different domains, such as cognitive, vestibular, and oculomotor symptoms, recovered at different postinjury time points. The findings of Henry et al²² together with ours support a comprehensive and repeatable assessment battery approach. In addition, the results suggest that acute concussion assessments such as the SAC and BESS are most appropriate for use in the first few days after injury. Although overlap in impairment on cognitive assessments (ie, ImPACT, SAC) was the most common pattern in the acute stage of the injury, impairment on the SAC did not predict later performance on ImPACT. Although ImPACT and the SAC both measure cognition, the SAC is a brief screen that was developed for sideline use and intended to help a health care provider determine whether the athlete should be removed from play rather than to provide detailed information regarding the severity of injury. In contrast, ImPACT was designed similarly to other neuropsychological tests, with normative data reflecting a broader range of functioning (eg, impaired, low average, high average). As such, our results contrast with those of authors who identified particular patterns of impairment on computerized neurocognitive testing or symptom patterns that predicted worse outcomes,²³ including protracted recovery.^24,25 Also, a relationship between acute cognitive deficits and deficits at 1 to 2 weeks postinjury may be difficult to detect given the decreased effect sizes of neurocognitive dysfunction at 1 week postinjury and beyond.²⁶ Another hypothesis for our findings relates to the difference in the underlying brain function being measured. Specifically, ImPACT is a computerized test that requires strong visual skills, and worse performance has been linked to oculomotor dysfunction postinjury.¹⁹ The SAC is delivered orally and does not rely on oculomotor function. This may explain why ImPACT and SAC share variance but also capture impairment in different individuals when evaluated concurrently and at different time points. Despite the widespread use and established diagnostic utility of sideline tools,^14,27 it does not seem plausible to prognosticate recovery on computerized neurocognitive testing based on the degree of impairment on sideline measures during the acute stage of injury. Furthermore, scores on the SAC and BESS at 24 to 48 hours postinjury did not predict return to baseline on symptom report (ie, symptom report within reliable change of baseline) at 5 to 7 or 10 to 14 days postinjury. Although the initial symptom burden has been linked to longer recovery,²³ it appears that the degree of impairment on sideline measures was not associated with persistent symptoms.

Some aspects of our results are not surprising given the paradoxical findings of acute presentation and recovery outcomes. For example, brief loss of consciousness and posttraumatic amnesia, which clearly are alarming signs of injury in the acute setting, are not always linked to longer recovery times or more significant deficits postinjury.^23,25 We find it interesting that researchers²⁸ have also reported “delayed onset” of symptoms and impairments as being linked to protracted recovery; perhaps our first study time point was too early to fully detect all impairments that may have manifested as a result of injury. This further supports the need for a multifaceted and repeatable assessment approach.

Our study is novel in that we attempted to predict neurocognitive recovery (ie, return to baseline) from scores on sideline measures administered in the first 2 days postinjury. A strength of our study was the concise time intervals at which participant data were collected. Although reliable change from the baseline ImPACT result was used to determine impairment and recovery, the SAC and BESS were not administered at baseline. Therefore, we relied on normative cutoffs, which have been inconsistent in the literature,¹³ to determine impairment on these measures. On the other hand, baseline tests were completed within 2 years of the current sport season and changes in symptom presentation may have occurred for which we were unable to account. A recognized limitation of our methods was the lack of a control group. In addition, several individuals at different study sites administered the BESS, and interrater reliability was not recorded.

CONCLUSIONS

We examined concurrent performance on commonly used assessment measures after SRC and found that athletes were most likely to have deficits on both sideline and computerized cognitive measures 24 to 48 hours postinjury. These results reinforce the need for a multifaceted approach to concussion assessment, the global nature of impairment and symptoms in the first week postinjury, and the heterogeneous nature of this injury. We also evaluated the predictive utility of sideline measures (ie, SAC and BESS) administered 24 to 48 hours postinjury, which suggested that performance on sideline measures could not predict either return to baseline performance on ImPACT at 5 to 7 or 10 to 14 days postinjury or return to baseline symptom severity. Although sideline measures are useful for identifying and diagnosing concussion, the current results indicate they are not effective in predicting neurocognitive impairment or symptom reports at 1 to 2 weeks postinjury. As such, we advocate for a comprehensive approach to assessing concussion across different domains using measures that are appropriate for each postinjury time point.