A Normative Study of the Sport Concussion Assessment Tool (SCAT2) in Children and Adolescents

Abstract

Recent clinical practice parameters encourage systematic use of concussion surveillance/management tools that evaluate participating athletes at baseline and after concussion. Office-based tools (Sports Concussion Assessment Tool [SCAT2]) require accurate baseline assessment to maximize utility but no normative data exist for children on the SCAT2, limiting identification of ‘normal’ or ‘impaired’ score ranges. The purpose of this study was to develop child and adolescent baseline norms for the SCAT2 to provide reference values for different age groups. A community-based approach was implemented to compile baseline performance data on the SCAT2 in 761 children aged 9 to 18 to create age- and sex-graded norms. Findings indicate a significant age effect on SCAT2 performance such that older adolescents and teenagers produced higher (better) total scores than younger children (ages 9 to 11) driven by age differences on individual components measuring cognition (SAC), postural stability (BESS), and symptom report. Females endorsed greater numbers of symptoms at baseline than males. Normative data tables are presented. Findings support the SCAT2 as a useful clinical tool for assessing baseline functioning in teenagers, but suggest clinical utility may be limited in children under age 11. Follow-up studies after incident concussion are needed to confirm this assumption.

Mild traumatic brain injury (concussion) affects at least 1.7 million people annually, a large proportion of whom are children involved in youth sports (Centers for Disease Control and Prevention, 2011; Faul, Xu, Wald, & Coronado, 2010). Concussion symptoms occur in several domains including cognition, emotional regulation, and physical well-being and can impact children’s academic and social functioning (Broglio, Moore, & Hillman, 2011; Erlanger et al., 2003; Fazio, Lovell, Pardini, & Collins, 2007; Guskiewicz, 2001). Young children and adolescents are more vulnerable to concussion for several reasons, including the fact that they are in a critical period for physiological and cognitive development, which affects postural stability and also decision-making about risk taking behavior (McKeever & Schatz, 2003). Not only is this population at higher risk for sustaining a concussion (Prins & Giza, 2012), but they may also be more sensitive to post-concussive effects due to their developing neural status, and, therefore, may exhibit protracted recovery times compared to adults and show greater long-term neurobehavioral impairment (Benz, Ritz, & Kiesow, 1999; Giza & Hovda, 2001)

Objective diagnosis of concussion depends upon identification of characteristic signs and symptoms, which can be complicated as definitive neuroimaging and neurological findings are typically absent (Jacobs et al., 2010; McCrory et al., 2009). The Sport Concussion Assessment Tool (SCAT2) is an open-source, multi-component assessment approach for concussion that is endorsed by 3^rd International Conference on Concussion in Sport for standard clinical assessment in sport (McCrory et al, 2009) and has demonstrated value in detecting post-concussion decrements from baseline measurement in domains of physical, cognitive, and postural stability (Mayfield, Shepherd, McLeod, & Bay, 2013). In a position statement from 2010, the American Academy of Pediatrics identified the SCAT2 as a recommended tool for sideline and in-office assessment for concussion (Halstead, Walter, The Council on Sports Medicine and Fitness, 2010). The SCAT2 provides brief concussion assessment and evaluates symptom report together with measures of cognition (Standardized Assessment of Concussion; SAC) (McCrea et al., 1998) and postural stability (Balance Error Scoring System; BESS) (Guskiewicz, Ross, & Marshall, 2001).

Concussion assessment may benefit from including individual baseline evaluation at pre-season (Moser et al., 2007; Lovell & Collins, 1998), but not all athletes are presented with the opportunity to receive baseline concussion testing. Therefore, establishing a normative comparison group based on age and sex is useful for improving clinical concussion diagnostics, especially for cases where individual baselines are unavailable. Additionally, norms can aid interpretability in cases where athletes may be purposefully underperforming on preseason baseline testing (“sandbagging”) or demonstrating learning effects (Valovich McLeod et al., 2004). Current normative research on the SCAT2 has focused primarily on high school-aged athletes (Jinguji et al., 2012; Valovich McLeod, Bay, Lam, & Chhabra, 2012), but the SCAT2 is purportedly useful in children as young as 10 years old (McCrory et al., 2009). In order to understand the utility of the SCAT2 in a pre-teenaged as well as high school population, the current study collected baseline data on children aged 9 to 18 years to examine the effect of age and sex on test performance.

Methods

This study was approved by institutional review boards from University of Florida, Florida State University, Tallahassee Memorial Hospital, Arnold Palmer Medical Center, and Florida Hospital. Patient data for the normative study were collected in the context of the Health IMPACTS for Florida: a unique research collaborative between University of Florida and Florida State University. This practice-based research network consists of University-affiliated healthcare providers who administered SCAT2s during routine, qualifying office visits for youth athletes in their community. Providers in this network included medical doctors, registered nurse practitioners (ARNP), physician assistants and medical assistants located in cities across Florida including Orlando, Tallahassee, and Jacksonville as well as surrounding rural communities. As part of the study protocol, student-athlete participants who attended these medical practices for qualifying physical examinations agreed to return to the examining physician if and when the participant sustains a concussion during season play for follow-up SCAT2 testing to assist in concussion diagnosis and management. Network healthcare providers were trained to administer the SCAT2 through an online portal provided by the Health IMPACTS for Florida website accessible through project-administered passwords. Physician training data were collected and managed using REDCap electronic data capture tools hosted at University of Florida. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources (Harris et al., 2009). Informed consent was obtained for all participants with the exception of those in the retrospective sample for whom a waiver was approved by the University of Florida IRB.

SCAT2 data were collected through several additional approaches to supplement the provider network. First, researchers administered the SCAT2s during “athlete round-ups” facilitated by community clinics that organized a group approach to provide qualifying sports physicals for large numbers of youth athletes in a short time period. Second, community-based concussion-testing events were created that functioned outside of the network as a free service to the community. Finally, a corpus of retrospective clinical data from area high schools was included from SCAT2s collected before the 2012–2013 academic year by athletic trainers in the graduate Athletic Training Program at the University of Florida. As part of the program’s graduate student curricula, supervised trainees are stationed in area high schools throughout Alachua County to provide athletic training services for students. All providers and research volunteers except for the athletic trainers participated in a standardized training protocol that included video demonstrations of administration and scoring for all components of the SCAT2. Research assistants who were not medical providers underwent a separate, in-person certification session conducted by the research coordinator after completing the initial training protocols in order to verify their ability to administer the SCAT2 in a standardized manner.

Participants

This study analyzed data from 761 total subjects ages 9 to 18 across all collection methods. As the goal of this project was to collect a normative sample, the only exclusion criteria were that participants could not have had a concussion diagnosis within the past three months. The average age of the participant group was 14.77 (SD = 2.3) years, and participants were recruited from rural and urban locations throughout north and central Florida. Of the total sample, retrospective clinical data accounted for 63.9%, the Health IMPACTS for Florida network accounted for 23.1% and community based concussion testing (termed athlete “round ups”) accounted for 13%. The sample included 656 (86.2%) males and 105 (13.8%) females, and a total of 20 team and individual sports were included, most commonly, American football (72.1%), cheerleading (5.3%), basketball (4.1%), and baseball (3.2%). Participants were divided into five groups based on age (9 to 10, 11 to 12, 13 to 14, 15 to 16, and 17 to 18) for two reasons: 1) based on graphical analysis of the outcome scores that showed trending differences in the 9 and 10 year olds compared to older peers and 2) these age groups roughly corresponded to academic grade level.

SCAT2

Individual SCAT2s were administered in one of two formats: paper and pencil and iPad-based using an application designed by Innova, Inc. A demographic questionnaire was also administered during the athlete round-ups and community testing events that included questions about race/ethnicity status and previous concussion history (number of concussions diagnosed by a medical professional). A positive history of concussion was defined as a head injury diagnosed as such by a medical or healthcare professional, and responders were also asked to identify the number of confirmed concussions. Parents were the preferential source for completing this background questionnaire; however, questionnaire data were not available for the majority of participants as most of the sample was collected retrospectively.

Psychometric properties of the SCAT2 are not well established for the tool as a whole, but the SAC and BESS components have demonstrated validity and reliability in previous studies (Barr & McCrea, 2001; Broglio, Zhu, Sopiarz, & Park, 2009; Riemann, Guskiewicz, & Shields, 1999; Valovich McLeod, Barr, McCrea, & Guskiewicz, 2006) and the reliability of self-reported graded symptom scales for baseline concussion assessment in adolescents has been established (Mailer, McLeod, & Bay, 2008). A maximum of 100 total points are possible and errors on SAC and BESS components or symptom endorsement lower the overall score (see Figure 1 for detailed SCAT2 scoring information). The SCAT2 begins with a symptom report section containing 22 possible symptoms with graded response options (0 through 6) to indicate symptom severity. Symptom Score was calculated by subtracting the number of symptoms endorsed from the total symptoms possible and Symptom Severity was calculated by summing the total symptom severity ratings (maximum of 132). For incident concussion assessment, (not used in our baseline assessments) physical signs, Glasgow Coma Scale (GCS), and Maddocks Questions provide additional information about basic neurological, motor, and response functioning after injury. These sections were omitted for baseline assessment, and participants automatically received maximum points for physical signs and GCS. Analyses were conducted using the IBM Statistical Package for the Social Sciences (SPSS) (Storrow & Bifano, 1997).

Figure 1.

SCAT2 scoring

Results

Baseline Norms

A normative table of baseline SCAT2 scores, organized by age and sex is presented in Table 1. A one-way univariate Analysis of Variance (ANOVA) was conducted using the age group divisions as a five-leveled independent variable and SCAT2 total score as the dependent variable. Overall, there was a significant effect of age on the SCAT2 total score, F(4, 295) = 9.05, MSE = 32.4,p < .001, with younger ages linearly associated with lower SCAT2 total scores. A series of Games-Howell corrected post-hoc analyses were conducted to identify significant differences among specific age groups, which showed that the youngest age group (ages 9 and 10) scored significantly lower on the SCAT2 compared to children ages 15 and older, and children in the oldest age group (ages 17 and 18) produced significantly higher scores than those in the youngest three groups (ages 9 to 14).

Table 1.

Sport Concussion Assessment Tool (SCAT2) Total and Component Scores by Age Group

	Ages 9–10 n = 51		Ages 11–12 n = 76		Ages 13–14 n = 176		Ages 15–16 n = 251		Ages 17–18 n = 202		p	Effect Sizea
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
SCAT2 Total	84.35	7.55	87.25	5.47	87.35	5.87	88.77	5.31	89.43	5.53	<.001*	.05
SAC	24.22	2.61	25.26	2.48	24.7	3.01	25.26	2.83	25.47	2.7	.01*	.02
BESS	23.73	3.57	25.03	3.43	24.86	3.4	25.28	3.58	25.4	3.48	.01*	.02
Symptom Score	18.59	4.5	19.09	3.77	19.85	3.17	20.28	2.73	20.42	3.06	.002*	.03
Male	19.09	2.59	19.09	3.64	20.18	2.78	20.41	2.59	20.57	2.81	-	-
Female	18.18	5.62	19.09	4.16	17.75	4.48	18.80	3.76	17.40	5.64	-	-
Symptom Severity	5.71	9.17	4.41	7.15	3.73	7.3	3.1	5.58	2.7	5.5	.06	.02
Male	4.13	4.18	4.74	8.19	3.19	6.83	2.86	5.31	2.34	4.65	-	-
Female	7.00	11.73	3.65	3.88	7.17	9.22	5.95	7.67	9.6	12.83	-	-

Note. Sport Concussion Assessment Tool 2 (SCAT2), Standardized Assessment of Concussion (SAC), Balance Error Scoring System (BESS)

^aEta-Squared

^*Denotes significant difference at α < .05

Separate one-way ANOVAs run for each of the component scores indicated that the age effect on SCAT2 total scores was driven by age differences on component scores. Younger children scored lower (made more errors) across the cognition and balance components, F(4, 754) = 3.34, MSE = 7.81,p= .01; F(4, 756) = 3.35, MSE = 12.23,p = .01, respectively. Bonferroni-corrected post-hocs indicated that the youngest age group (9 to 10) was again significantly lower on cognition scores than the oldest age group (17 to 18), and postural stability (BESS) follow-ups found that the youngest age group (9 to 10) committed significantly more balance errors than the each of the two oldest age groups, (15 to 16 and 17 to 18). Symptom Score and Symptom Severity indices also demonstrated age effects. Since both of these outcome measures violated the assumption of homogeneity of variances and were significantly non-normal, the Brown-Forsythe robust test of equality of means was used, which produced a significant effect of age for Symptom Score, F(4, 262.98) = 4.38,MSE = 10.08,p = .002, but not Symptom Severity in that younger age was associated with higher symptom endorsement at baseline.

A univariate Analysis of Covariance (ANCOVA) was conducted on SCAT2 total and component scores (cognition and balance) to isolate the effect of age group covarying for sex and data collection setting. Only test setting as a covariate was significantly related to SCAT2 total score, F(1, 749) = 16.01,MSE = 31.8,p < .001), and the effect of age group on SCAT2 total score persisted after controlling for the effect of sex and test setting, F(4, 749) = 4.68, MSE = 31.8,p = .001. For cognition, both covariates were significantly related to SAC score, (sex, F(1, 752) = 6.97,MSE = 7.63,p = .008; test setting, F(1, 752) = 18.02, MSE = 7.63,p >.001), and the effect of age group on SAC score was no longer significant after controlling for these two variables. For the balance component, both covariates were significantly related to BESS score, sex, F(1, 754) = 3.92, MSE = 12.05,p < .05, test setting, F(1, 754) = 5.15, MSE = 12.05,p =.02, and the effect of age group on BESS was still significant after controlling for sex and test setting, F(4,754) = 6.17, MSE = 12.05,p< .001.

Sex

Independent samples t-tests were used to compare males and females on each of the SCAT2 outcome scores (see Table 2). Results indicated that sex is not associated with significant differences on SCAT2 total scores. No sex differences were detected for the cognition and balance components, but there was a significant sex difference in symptom reporting and symptom severity ratings: Symptom Score, U(760) = 24,974.5, z = -4.83, p < .001; Symptom Severity, U(760) = 43,840, z = 4.79, p< .001. Both Symptom Score and Symptom Severity were tested using the non-parametric independent Wilcoxon rank-sum test due to a significant violation of normality. Females endorsed more symptoms and rated those symptoms as higher in severity at baseline than males. A two-way univariate ANOVA was run with age group and sex as the independent variables to investigate possible interaction effects but was not significant.

Table 2.

Sport Concussion Assessment Tool (SCAT2) Total and Component Scores by Sex

	Males n = 656		Females n = 105		p	Effect Sizea
	Mean	SD	Mean	SD
SCAT2 Total	88.36	5.61	86.97	6.97	.05	.17
SAC	25.1	2.78	25.19	3.04	.77	.01
BESS	25.07	3.53	25.57	3.45	.18	.05
Symptom Score	20.25	2.82	18.32	4.69	<.001*	.18
Symptom Severity	2.98	5.79	6.35	9.25	<.001*	.17

Note. Sport Concussion Assessment Tool 2 (SCAT2), Standardized Assessment of Concussion (SAC), Balance Error Scoring System (BESS)

^aEta-Squared

^*Denotes significant difference at α < .05

Data Setting Analysis

Since our data was collected in various settings, we wanted to determine whether significant differences existed as a function of testing site. The contribution of test setting (a 3-level variable comprised of retrospective clinical data, in-office healthcare provider, and community-based assessment settings) on baseline SCAT2 scores was examined through a one-way ANOVA. Brown-Forsythe F-adjustment was used due to violation of the homogeneity of variance assumption and showed that there is a significant effect of administration setting on SCAT2 scores, F(2, 292,57) = 19.77, MSE = 31.98,p <.001. Games-Howell-corrected follow-ups elaborated that the retrospective clinical data produced significantly higher SCAT2 baseline total scores than either the athlete “round-ups” or in-office provider assessments. However, this finding is complicated by the fact that the retrospective clinical setting assessed only high school age children (ages 13 to 18) producing a higher mean age tested in this group compared to the athlete “round-ups” and in-office provider settings, where the full range of ages was tested (9 to18). The significant effect of test setting on SCAT2 total scores persisted after selecting for only high school aged children, ages 14 to 18 using the Brown-Forsythe test, F(2, 84.89) = 12.26, MSE = 12.26,p < .001. Results of post-hoc analysis of component scores by test setting are presented in Figure 2.

Figure 2.

Post-hoc differences across SCAT2 component scores by test setting

Discussion

Attempts to establish SCAT2 normative comparison groups in non-adults under the age of 18 have appeared only in the past year (Jinguji et al., 2012; Valovich McLeod et al., 2012), but they have focused exclusively on high school-age youths and do not offer normative data on younger, elementary and middle school aged children. The current study is the first to define normative population estimates for pre-season, baseline SCAT2 in younger children, ages 9 to 18. Our findings present differences in baseline SCAT2 scores based on age, which is consistent with previous normative attempts and a priori expectations based on physiological and neurocognitive developmental literature. In this study, younger age was associated with lower overall SCAT2 scores, and this pattern held true across component measures of cognition (SAC), balance (BESS), and symptom report (Symptom Score). The youngest age group (ages 9 and 10), in particular, made more errors on component measures resulting in significantly lower scores on the BESS and SAC compared to the oldest age groups. Regarding symptom report, younger children (9 and 10) also tended to identify significantly more symptoms at baseline than adolescents ages 15 and up. Although these findings are significant, the effect sizes across all detected age differences are small, which could attenuate their clinical meaningfulness; however, further examination into the relationship of concussion testing at baseline to that conducted after injury is needed to determine the impact of these baseline age differences.

Compared to the normative analyses of the SCAT2 performed by Jinguji et al. (2012) and Valovich McLeod et al. (2012) who identified mean SCAT2 baseline scores as roughly 90 out of 100 for high school aged adolescents (14 to 18 years), the present study found slightly lower mean baseline SCAT2 score for this age group (M= 88.6), and an even lower mean baseline score for those younger than high school age (86.8). One limitation is that, in addition to developmental factors, this discrepancy may be the result of using baseline tests obtained across three different settings and/or a product of the lack of exclusionary criteria in this study compared to Jinguji et al., which excluded participants with a history of learning disability, attention deficit hyperactivity disorder, or six-month concussion history and Valovich McLeod et al., which required a pre-participation physical examination clearance before enrolling in their normative study. Consideration should also be given to the fact that this sample is over-represented by male participants; thus, normative values for females should be interpreted cautiously.

The age differences demonstrated on SCAT2 component scores demonstrated by this study are consistent with developmental literature showing that performance on the BESS and SAC is affected by maturation. Regarding balance, research has shown that vestibular function is the slowest of all multimodal sensory systems to develop, and that postural stability does not reach full maturational development until the mid-teenage years (Hirabayashi & Iwasaki, 1995; Steindl, Kunz, Schrott-Fischer, & Scholtz, 2006). For the SAC, age effects are also noted, such that younger age is associated with significantly lower scores on the SAC in healthy, non-patient populations (Grubenhoff, Kirkwood, Gao, Deakyne, & Wathen, 2010). Cognitive development is in great flux between the ages of 9 to 18, so it is expected that neurocognitive scores on the SAC produced by a 17 year old would not be comparable to that of a 9 or 10 year old, which is further supported by our findings.

Systematic variability based on age and sex on the self-rated symptom scale is also corroborated by the literature. Sex differences in self-reported physical symptoms are well-documented across symptom checklists and other health surveys, and the effect of females reporting a higher frequency and greater severity of symptoms is especially true for healthy individuals at baseline (Van Wijk & Kolk, 1997). It is unclear why the youngest group of children in this study indicated higher baseline experience of physical symptoms, since this finding could be influenced by several factors such as difficulty understanding the checklist wording, less motivation to present favorably to the test administrator, inadequate comprehension of the rating scale, or greater willingness to report feelings and symptoms. The influence of these factors along with broader questions about the nature of base-rate symptom reporting in healthy children should be further investigated.

Although not a main focus of this project, data collection setting emerged as a possible influence on the normative data trends, since the retrospective clinical data from local high schools contained significantly higher SCAT2 baseline scores than the other two data collection settings. One limitation of this study is that it is impossible to know if the older average age of this sample fully explains their higher scores since a different cohort of examiners (graduate athletic trainers) collected this data. Interestingly, student-athletes from the high school setting reported significantly fewer baseline physical symptoms when compared to in-office healthcare provider and athlete “round-up” settings. This finding could have resulted from an intentional effort by high school athletes to more positively portray their physical functioning to athletic trainers who, essentially, control athletes’ sports participation eligibility. Another possibility is that differences in the athletic trainers’ SCAT2 training may have affected the athletes’ responses to the assessment, since this cohort did not undergo the same training protocol that the other examiners did. Subjects might also be more inclined to report physical symptoms to health care providers in health care settings where such issues are the focus.

Overall, these findings suggests that the SCAT2 may have less clinical utility in children under the age of 11 since variance in component scores for these children may be too limited to detect changes after a concussion has been sustained. However, this observation remains at the assumption level only since diagnostic characteristics of the SCAT2 after an incident concussion have not yet been established in children. A longitudinal study of children with baseline data who sustain incident concussions would be needed to address this issue. Another implication of these age trends is the necessity of repeated, annual pre-season baseline testing to accurately reflect maturational changes in youth athletes that manifest through improved neurocognitive ability and postural stability. The natural extension of the present study is to document SCAT2 scores on incident concussions, ideally matched with pre-season baseline, to better understand the clinical utility of the SCAT2 in children. The potential influence of administrator type on SCAT2 scores, particularly with respect to symptom report, should be incorporated as a qualitative consideration for baseline interpretation; however, future research is clearly needed to determine if quantitative adjustment of baseline scores based on test administrator are appropriate for certain populations, such as high school students, who may be motivated to minimize symptom presentation, and how this adjustment should be clinically applied. Future research investigations should also consider the impact of learning disorders and other conditions like attention-deficit hyperactivity disorder, that are known to be associated with decreased performance on heavily-weighted aspects of the SCAT2 such as attention, memory, and, possibly, balance.

Since the conclusion of this project, an updated version of the SCAT2 was released, the Sport Concussion Assessment Tool, 3^rd version (SCAT3) and Child Sport Concussion Assessment Tool (Child-SCAT3) (McCrory et al., 2013). The data presented here can be mapped with reasonable accuracy onto the SCAT3 since they are almost identical, especially with respect to the version of the SCAT3 intended for ages 13 and older. The total score has been eliminated from both versions of the SCAT3 and presents component scores for individual interpretation. The SAC component is unchanged on the updated SCAT3, but the Child-SCAT3 simplifies the concentration and orientation sub-component questions to be more age-appropriate. Regarding balance (BESS), the updated version includes the same basic stances as the SCAT2, but eliminates the single-leg stance for the child’s version and introduces another measure, the tandem walk. Normative data have not been presented for the full scope of the age recommendations covered by the Child-SCAT3, which is proposed for use in children as young as 5 years old. Thus, this study provides interim data for this younger population for the unchanged components and a comparison group from which to verify the age-appropriate modifications made to the Child-SCAT3. Practically, physicians could use the data provided in Table 1 to identify scores that fall below 1 or 2 standard deviations of the normative age/sex means as those that might “raise suspicion” of head injury or an invalid baseline performance as the result of sub-optimal effort or other confounding variable such as learning disorder. However, this approach would need to be externally validated before implemented as a standard clinical interpretation protocol. The results of the present study also emphasize the need for further investigation into the appropriateness and sensitivity of lowered age-recommendations for detecting post-injury change on the Child-SCAT3.