Computerized Neurocognitive Testing for the Management of Sport-Related Concussions

Abstract

Objectives:

To describe the prevalence of computerized neurocognitive testing for the assessment of high school athletes who sustain concussions, and to describe associations between using computerized neurocognitive tests, timing of return-to-play, and medical provider managing the athlete.

Methods:

Concussions recorded in the High School Reporting Information Online injury surveillance system during the 2009–2010 academic year were included. Measures of association between use of computerized neurocognitive testing and outcomes were analyzed. A questionnaire was sent to athletic trainers (ATs) querying the use of computerized neurocognitive testing. χ² analyses were conducted for categorical variables. Logistic regression analyses were used to adjust for potential confounders. Statistical significance was set at P < .05.

Results:

High School Reporting Information Online recorded 1056 concussions. Athletes who underwent computerized neurocognitive testing were less likely to be returned to play within 10 days of injury (38.5% vs 55.7%, P < .001) and more likely to be returned to play by a physician (60.9% vs 45.6%, P < .001). We had a response rate of 97.3% for the survey. Of respondents, 39.9% used computerized neurocognitive testing; 93.0% of those used ImPACT. Tests were most often interpreted by ATs (78.9%) and/or physicians (78.8%), as opposed to neuropsychologists (16.9%).

Conclusions:

Approximately 40% of US high schools that employ an AT use computerized neurocognitive tests when managing sport-related concussions. Tests are most often interpreted by ATs and physicians, as opposed to neuropsychologists. Computerized neurocognitive tests are significantly associated with the timing of return-to-play.

What’s Known On This Subject:

Neurocognitive testing is recommended for the assessment of sport-related concussions. Computerized neurocognitive tests are more sensitive and more efficient than traditional neuropsychological testing in assessing sport-related concussions.

What This Study Adds:

We describe the current prevalence of computerized neurocognitive testing, the relative use of the various computerized programs, the types of clinicians interpreting test scores, and associations of computerized tests with timing of return-to-play and medical provider type managing the athlete.

The field of neuropsychology has made significant contributions to the assessment and management of sport-related concussions. By documenting quantifiable brain dysfunction after sport-related concussions,^1–4 neuropsychologists have given new legitimacy to what many clinicians previously dismissed as a trivial injury. Although not without controversy,^5,6 neuropsychological testing has been called “the cornerstone” of concussion assessment and management.^7,8 The use of neuropsychological testing in the management of sport-related concussions has been recommended in several major consensus statements.^7–9

Due to availability, convenience, accuracy, and sensitivity, computerized paradigms have become the preferred method of neurocognitive testing for assessing the cognitive functioning of athletes at high-risk for concussion.^10–12 Computerized neurocognitive assessments use specific tasks, often based on traditional neuropsychological tests, to measure verbal memory, visual design memory, concentration, visual processing speed, and reaction time.^13–15 These paradigms have proven sensitive for the detection of sport-related concussion and, when used in conjunction with symptom reporting and clinical assessment, more sensitive in detecting injury and monitoring recovery than symptom reporting alone.^14,16–19 Ideally, athletes undergo baseline neurocognitive assessments before the start of an athletic season. These baseline tests are then used for comparison after a concussion has been sustained to determine the presence of clinically significant changes in brain functioning.^20,21

Few authors have investigated the frequency with which computerized neurocognitive tests are used when assessing sport-related concussions. A survey of athletic trainers (ATs) published in 2001 revealed that only 1.9% of respondents used neurocognitive testing when making return-to-play decisions.²² Distinctions were not made between computerized paradigms and more traditional testing. In a survey of primary care providers published in 2006, only 16% of respondents had access to neurocognitive testing within a week of injury.²³ More recently, we have reported that computerized neurocognitive testing was used in assessing 25.7% of the sport-related concussions recorded in the High School Reporting Information Online (HS RIO) injury surveillance system during the 2008–2009 academic year.²⁴

We are aware of no other prospective studies assessing the frequency with which computerized neurocognitive testing is used in the assessment and management of athletes with concussions. Furthermore, we are aware of no study assessing the type of medical personnel that interpret these tests. There are few data regarding the effects these tests have on the timing of return-to-play. In this study, we used the HS RIO injury surveillance system and its associated network of ATs to determine the frequency of computerized neurocognitive testing in the management of sport-related concussions occurring in US high schools. Furthermore, we describe the frequency with which the various types of computerized neurocognitive tests are used, the type of provider interpreting test scores, and associations between computerized neurocognitive testing and return-to-play, duration of recovery, and the type of medical personnel deciding to return athletes to play.

Methods

HS RIO has been described elsewhere in detail.^25–27 In brief, high schools that employ at least 1 certified AT who is affiliated with the National Athletic Trainers Association and has a valid e-mail address were invited to participate. ATs from participating high schools received a small financial incentive to log onto the HS RIO Web site weekly throughout the academic year to report injury incidence and athletic exposure data. For each injury reported, ATs completed a detailed injury report form providing information on the injured athlete (eg, age, position played), the injury itself (eg, body site injured, diagnosis), and the injury event (eg, mechanism, activity at the time of injury).

During the 2009–2010 academic year, 192 US high schools reported data for athletes participating in 20 sports (boys’ sports: baseball, basketball, football, ice hockey, lacrosse, soccer, swimming and diving, track and field, volleyball, and wrestling; girls’ sports: basketball, field hockey, gymnastics, lacrosse, soccer, softball, swimming and diving, track and field, volleyball, and cheerleading). The number of schools reporting for each sport varied from 158 schools reporting girls’ basketball data to 27 schools reporting boys’ volleyball data. Only boys’ volleyball and girls’ gymnastics had <50 schools reporting data; 12 of the 20 sports had over 100 schools reporting data. The surveillance system is designed to capture all injuries that result in an absence from 1 or more practices or games, as well as any fracture, any concussion, and any dental injury, regardless of whether it resulted in restriction of the student athlete’s participation. All concussions that (1) occurred during organized high school athletic practice or competition, (2) resulted in the athlete receiving care from a medical provider, and (3) were brought to the attention of the AT were recorded. An athletic exposure was defined as 1 athlete participating in 1 organized high school athletic practice or competition.

In addition, 183 ATs participating in HS RIO were sent a questionnaire at the start of the academic year querying the use of computerized neurocognitive testing at their schools. Data regarding the type of testing used, the sports for which athletes were tested, the frequency of baseline and postinjury testing, the type of medical personnel interpreting the scores, and whether the scores were used to assist in making return-to-play decisions were recorded.

All statistical analyses were performed by using SPSS software (SPSS version 16.0; SPSS Inc, Chicago, IL). χ² analyses were performed for all categorical variables. Logistic regression analyses were used to adjust for duration of symptoms when measuring associations between the use of computerized neurocognitive testing and the timing of return-to-play and type of provider returning the athlete to play. Statistical significance was set at P < .05.

Results

Survey Results

The current study had a 97.3% response rate, with 178 schools returning the questionnaire. Of the respondent schools, 39.9% used computerized neurocognitive testing in the management of athletes with concussions. Of those using computerized neurocognitive testing, the vast majority (93.0%) used ImPACT (ImPACT Applications Inc, Pittsburgh, PA; Fig 1).The majority (69.9%) of schools using computerized neurocognitive testing only tested athletes in certain sports, whereas the remainder (30.1%) tested all of their athletes. Table 1 shows the sports for which computerized neurocognitive testing was used among the 51 schools that only test athletes playing certain sports. Most schools (85.9%) performed both baseline and postinjury tests. An additional 12.7% of schools only performed postinjury tests. Although 4 respondent schools left the question unanswered, all other schools that used computerized neurocognitive testing reported that athletes’ scores were used to assist in making return-to-play decisions.

FIGURE 1.

Specific computerized neurocognitive test programs used by surveyed HS RIO schools. ^a Cogsport has since changed its name to AxonSports.

TABLE 1.

The Sports for Which Computerized Neurocognitive Testing Was Obtained Among Schools Only Testing Athletes Playing Certain Sports

Sport	Schools Testing Athletes, No. (%)
Football	19 (37.3)
Boys’ soccer	10 (19.6)
Girls’ soccer	10 (19.6)
Wrestling	8 (15.7)
Baseball	5 (9.8)
Softball	5 (9.8)
Boys’ basketball	4 (7.8)
Girls’ basketball	3 (5.9)
Boys’ lacrosse	3 (5.9)
Girls’ lacrosse	3 (5.9)
Cheerleading	3 (5.9)
Field hockey	2 (3.9)
Boys’ ice hockey	2 (3.9)
Girls’ ice hockey	2 (3.9)
Volleyball	2 (3.9)
Gymnastics	1 (2.0)
Pole vault	1 (2.0)

For the majority of schools using computerized neurocognitive testing, the scores are interpreted by an AT (78.9%) and/or a physician (78.8%). Table 2 lists the various clinicians interpreting computerized neurocognitive test scores. As athletes with concussions are often cared for by more than 1 type of clinician, multiple clinicians often interpreted test scores.

TABLE 2.

Clinicians Interpreting Computerized Neurocognitive Test Scores

Clinicians Interpreting Neurocognitive Test Scores	Tests Interpreted, %
AT	78.9
Physician	78.8
Neuropsychologist	16.9
Registered nurse	4.2
Physician assistant	1.4
Nurse practitioner	1.4

Percentages do not add up to 100, as multiple clinicians may be interpreting scores for a given school.

Effects of Using Computerized Neurocognitive Testing

ATs from the 192 schools participating in HS RIO recorded 1056 concussions during the 2009–2010 academic year. Data regarding the use of computerized neurocognitive testing were entered for 1041 (98.6%) concussions. Of those, computerized neurocognitive testing was used in the assessment and management of 429 (41.2%). This represents an increase from the previous academic year, during which 25.7% of concussions were managed by using computerized neurocognitive testing.²⁴

Those athletes with symptoms lasting longer than 7 days were more likely to undergo computerized neurocognitive testing than those whose symptoms resolved within 7 days (56.9% vs 36.0%; P < .001). Athletes who had computerized neurocognitive testing were less likely to return-to-play within 10 days of injury (38.5% vs 55.7%; P < .001) and more likely to have a physician (as opposed to an AT) decide when they should return-to-play (60.9% vs 45.6%; P < .001) than those athletes who did not have testing.

We considered that prolonged symptoms (>7 days) might be a potential confounder, associated with the exposure, the use of computerized neurocognitive testing, and the outcomes: athletes returning to play within 10 days, and physician deciding to return the athlete to play. Thus, logistic regression analyses were performed to adjust for the duration of symptoms. Each outcome remained associated with the use of computerized neurocognitive testing, even when adjusted for the duration of symptoms (Table 3); athletes who received testing remained less likely to return-to-play within 10 days of their injury and less likely to be cleared for play by an AT, as opposed to a physician.

TABLE 3.

Odds Ratios for Use of Computerized Neurocognitive Testing, Adjusted for Duration of Symptoms (>7 days)

Outcome	Adjusted Odds Ratio (95% CI)
Return-to-play within 10 d	0.725 (0.538–0.976)
AT (as opposed to a physician) returned athlete to play	0.600 (0.457–0.788)

Schools that offer computerized neurocognitive testing were less likely to return athletes with concussions to play within 10 days of their injury than schools that did not offer computerized testing (41.7% vs 53.0%; P = .002). However, even at schools that offer it, some concussions were managed without the use of computerized neurocognitive testing. Thus, we were able to assess which differences in management were due to the use of the computerized neurocognitive tests themselves, as opposed to other characteristic differences between schools that offered testing and schools that did not, such as more conservative management styles. When only those concussions managed without the use of computerized neurocognitive testing were analyzed, there was no significant difference in the proportion of athletes with concussions returned to play within 10 days between schools that offer testing and schools that did not offer testing.

Schools that offer computerized testing were slightly more likely to have a medical professional return an athlete with a concussion to play than schools that did not offer testing (99.2% vs 96.5%; P < .01). Furthermore, schools that offer computerized neurocognitive testing were more likely to have a physician (as opposed to an AT or other medical professional) decide when to return an athlete to play than schools that did not offer testing (63.1% vs 46.1%; P < .001). This association remained even when only those athletes who did not undergo computerized testing were assessed; analyses using only those concussions for which computerized neurocognitive testing was not used revealed that schools that offer testing were more likely to have a physician decide when to return the athlete to play than an AT (59.8% vs 43.0%; P = .02).

Athletes attending schools that offer computerized neurocognitive testing were more likely to remain symptomatic for longer than 7 days than those athletes attending a school that did not offer testing (27.6% vs 16.7%; P < .001). However, when only those students who did not undergo computerized neurocognitive testing despite attending a school that offered it were analyzed, no statistically significant association was found between athletes attending a school that offers neurocognitive testing and duration of symptoms.

Discussion

Approximately 40% of US high schools that employ at least 1 certified AT use computerized neurocognitive testing to assess athletes who sustain sport-related concussions. The percentage of concussions for which computerized neurocognitive testing is used has risen from 25.7% in the 2008–2009 academic year²⁴ to 41.2% in the 2009–2010 academic year. Given the flurry of media attention devoted to sport-related concussions in the last several years,^28–31 this number is likely to continue to increase. In addition, several states have recently legislated return-to-play requirements for athletes who have sustained sport-related concussions.³² This may also increase the proportion of athletes who undergo neurocognitive assessments. At the time these data were collected, however, only a few states had passed such legislation.

As might be expected, 100% of schools offering computerized neurocognitive testing to their athletes report that these scores are used in making return-to-play decisions.

However, the National Athletic Trainer’s Association reports that only 42% of high schools in the United States use the services of a certified AT.³³ It is likely that a smaller proportion of schools that do not employ an AT use computerized neurocognitive testing to manage sport-related concussions.

A previous survey by Covassin et al¹³ revealed that 94.7% of institutions, both high schools and colleges, listed on the ImPACT Web site administered baseline tests. Our study, which included high schools exclusively, revealed that only 86% of the schools using computerized neurocognitive testing obtain baseline tests. This is less than ideal. There is significant variability in neurocognitive functioning among athletes. Without baseline assessments administered before the athlete is concussed, it can be difficult to discern whether performance on a postinjury test represents diminished cognitive functioning or the athlete’s usual capabilities.^10,20 Compared with normative samples, some athletes may perform relatively poorly on these tests when they are healthy, especially if they have learning disabilities, difficulties with attention, or put forth inconsistent efforts. Alternatively, some athletes may perform so well on these tests at baseline that even when they are injured their scores may be at the higher end of the normative values. Thus, without a baseline score, there is a greater potential for the results to be misinterpreted.

Our study revealed that in high school athletics, most computerized neurocognitive test scores are interpreted by ATs and physicians, as opposed to neuropsychologists. This likely reflects the overall availability of ATs and physicians compared with neuropsychologists. In our study, only 17% of schools had neuropsychologists interpreting the scores. This finding is consistent with the survey conducted by Pleacher and Dexter²³ that revealed only 16% of primary care physicians had access to a neuropsychologist within 1 week of a concussion occurring. While ideally, all teams and all clinics would have access to enough trained neuropsychologists to interpret every neurocognitive test, this is unlikely to occur in the near future. It is estimated that as many as 3.8 million sport-related traumatic brain injuries occur each year,³⁴ the vast majority being concussions. The frequency of concussive brain injuries in sports and the limited availability of neuropsychologists underscore the importance of training other medical professionals to administer and interpret computerized neurocognitive tests for the limited purpose of assessing sport-related concussions.³⁵ There is no doubt that trained neuropsychologists familiar with these computerized paradigms will be able to glean more information from these tests than nonneuropsychologists. Furthermore, there are many instances where the interpretation of these tests will be complicated, either by the lack of baseline data, concurrent medical problems, psychiatric issues, or other conditions that have the potential to affect performance. In these circumstances, the expertise of a trained neuropsychologist remains invaluable.

Some concerns about computerized neurocognitive tests must be addressed. Schools with limited resources might be tempted to purchase the neurocognitive assessment software without the added expense of training their personnel. It should be emphasized that no AT, physician, neuropsychologist, or other medical professional should attempt to interpret computerized neurocognitive test scores without first undergoing proper training. The potential exists for misuse of these tests. Athletes might purposely perform poorly on preseason, baseline tests, in the hopes that they might perform equally well after a concussion. To avoid mistaking such tests as true baseline measurements, computerized assessments are only considered valid if certain criteria have been met. An untrained administrator would be unaware of these criteria and might clear an athlete on the basis of erroneous information. Furthermore, untrained individuals may be tempted to use the computerized tests as a sole determinant of recovery and clear athletes for return-to-play when they have re-achieved their baseline scores, despite remaining symptomatic. Schools should train personnel or identify qualified personnel in their community with whom to partner, before purchasing computerized neurocognitive assessments. One can train on these programs by taking the courses offered by the companies who developed them, working in a clinical setting under the guidance of a trained and approved clinician using one of these programs, reviewing the clinical literature, and studying the user manuals.

When computerized neurocognitive testing is used, athletes are less likely to return-to-play within 10 days of injury. This association remains even after controlling for duration of symptoms. Although it is possible that schools offering testing have more conservative practice styles, this association was only observed when testing was actually used. Athletes whose concussions were assessed without the use of computerized neurocognitive testing despite attending a school that offered such testing were just as likely to be returned to play within 10 days of injury as athletes attending schools that did not offer computerized neurocognitive testing. Thus, it is the computerized neurocognitive tests themselves that are associated with this outcome. Presumably, the tests were able to detect persistent symptoms or cognitive deficits that were not detected by the remainder of the clinical assessment. Because the results of computerized neurocognitive test scores were not collected, however, we cannot state this with certainty.

Similarly, a higher proportion of athletes for whom neurocognitive tests were used remained symptomatic beyond 7 days after injury. It is likely that the longer an athlete has symptoms, the more likely she or he is to undergo computerized neurocognitive testing. In addition, combining neurocognitive testing with other clinical assessments, such as symptom reporting and postural control, can increase sensitivity and diagnostic accuracy.^16,17,36 The addition of neurocognitive testing may have allowed ATs to detect symptoms and cognitive deficits that would not have been detected by other clinical assessments alone.

As others have emphasized, neurocognitive assessment should never be used by itself, but rather, as a part of the overall assessment of the athlete with a concussion.^8,37,38 A comprehensive concussion management program should include a thorough history, a complete neurologic screening examination, a standardized assessment of symptoms, a standardized balance assessment, and an assessment of neurocognitive function.⁸ These data points must be interpreted in their clinical context and used to guide return-to-play decisions. None should be used in isolation or as a sole determinant of recovery.

All schools included in the HS RIO surveillance system employ at least 1 AT. Therefore, our findings are limited to schools that employ an AT. In addition, some athletes may have taken tests at school that were interpreted offsite by an MD or neuropsychologist, perhaps resulting in a delay in return-to-play timing.

Conclusions

Approximately 40% of US high schools that employ an AT use computerized neurocognitive testing in the assessment of athletes with concussions. Most often, computerized neurocognitive tests used in this setting are interpreted by ATs or physicians, as opposed to neuropsychologists. Athletes assessed with computerized neurocognitive tests are less likely to be returned to play within 10 days of injury than athletes with concussions managed without such testing. When used appropriately by trained personnel, computerized neurocognitive assessments are a valuable component of a comprehensive concussion management program.

Glossary

AT

athletic trainer

HS RIO

High School Reporting Information Online