Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 12.
Published in final edited form as: J Int Neuropsychol Soc. 2020 Aug 7;27(2):113–123. doi: 10.1017/S1355617720000685

Neuropsychological Change After a Single Season of Head Impact Exposure in Youth Football

Arthur Maerlender 1,*, Eric Smith 2, P Gunnar Brolinson 2,5, Joseph Crisco 6, Jillian Urban 3, Amaris Ajamil 4, Steven Rowson 2, Eamon T Campolettano 2, Ryan A Gellner 2, Srinidhi Bellamkonda 6, Emily Kieffer 2, Mireille E Kelley 3, Derek Jones 3, Alex Powers 7, Jonathan Beckwith 4, Joel Stitzel 3, Richard M Greenwald 4, Stefan Duma 2
PMCID: PMC7867662  NIHMSID: NIHMS1604717  PMID: 32762785

Abstract

Objectives

Head impact exposure (HIE) in youth football is a public health concern. The objective of this study was to determine if one season of HIE in youth football was related to cognitive changes.

Method

Over 200 participants (ages 9–13) wore instrumented helmets for practices and games to measure the amount of HIE sustained over one season. Pre- and post-season neuropsychological tests were completed. Test score changes were calculated adjusting for practice effects and regression to the mean and used as the dependent variables. Regression models were calculated with HIE variables predicting neuropsychological test score changes.

Results

For the full sample, a small effect was found with season average rotational values predicting changes in list-learning such that HIE was related to negative score change: standardized beta (β)=−.147, t(205) = −2.12, and p = .035. When analyzed by age clusters (9–10, 11–13) and adding participant weight to models, the R2 values increased. Splitting groups by weight (median split), found heavier members of the 9–10 cohort with significantly greater change than lighter members. Additionaly, significantly more participants had clinically meaningful negative changes: X2 = 10.343, p = .001.

Conclusion

These findings suggest that in the 9–10 age cluster, the average seasonal level of HIE had inverse, negative relationships with cognitive change over one season that was not found in the older group. The mediation effects of age and weight have not been explored previously and appear to contribute to the effects of HIE on cognition in youth football players.

Keywords: Youth football, Head impact exposure, Neuropsychological test, Cognition, NIH Toolbox, Reliable change

BACKGROUND

In 2014, the Institute of Medicine (IOM) and National Research Council (NRC) reported that more study was needed at the youth level to better understand the shortand long-term effects of concussions and repetitive sub-concussive head impacts (impacts that do not result in the clinical signs or symptoms of concussion) (IOM and NRC, 2013). At the time, most studies of head impact exposure (HIE) had focused on collegiate and high-school athletes (Broglio et al., 2011; Broglio, Surma, & Ashton-Miller, 2012; Crisco et al., 2011; Duma et al., 2005; Jacobson, Buzas, & Morawa, 2013; Kerr, Hayden, Dompier, & Cohen, 2015; Rosenthal, Foraker, Collins, & Comstock, 2014; Urban et al., 2013).

Recent youth football studies have shown that younger players are generally exposed to lower levels (frequency and magnitude) of HIE than high-school or college football players (Broglio et al., 2012; Cobb et al., 2013; Daniel, Rowson, & Duma, 2012; Urban et al., 2013). However, youth can sustain head impacts comparable to those at the high school and collegiate level, and some youth may experience total season HIE levels approaching the levels found in high-school and college football (Cobb et al., 2013; Daniel et al., 2012; Kelley et al., 2017). Despite educational programs in youth football tackling (Kerr et al., 2015), there is still concern, and much is still unknown about 1 the consequences of repetitive head impacts in football for all ages.

Studies of football athletes have found measurable changes in neuroimaging and neuropsychological testing done at pre-season to post-season intervals that correlated with HIE metrics (linear and/or rotational force measures) in groups with non-cocnussed participants (Bahrami et al., 2016; Bazarian et al., 2007; Bazarian et al., 2014; Davenport et al., 2014; McAllister et al., 2014). However, the nature of the relationship between HIE and neuropsychological changes is not yet fully understood. Studies showing effects of HIE in high school and youth have often confused clinical and statistical significance and/or used improper analysis of test score changes that rendered the results questionable (Cobb et al., 2013; Munce et al., 2014; Nauman et al., 2015; Talavage, Nauman, && Yoruk, 2014). In addition, biosensor technology is challenging to implement and requires considerable technical skill and oversight that is not as available as the products themselves are. Thus, questionable interpretations and methodological variations warrant further careful investigation of the effect of sub-concussive impacts, especially in the youth population (Institute of Medicine (IOM) and National Research Council (NRC), 2014; Munce et al., 2014; Tarnutzer, Straumann, Brugger, & Feddermann-Demont, 2017; Tsushima, Geling, Arnold, & Oshiro, 2016).

Recently, Rose et al published two studies of youth football players using newer heat-transducing HIE technology in helmets (Rose et al., 2018, 2019). This technology does not provide specific measurements of HIE but rather categorical labels of force and location. In the most recent paper, they described effects of two seasons of football in 55 youth aged 9 to 18 (Rose et al., 2019). Their results demonstrated a general lack of detectable declines in neurocognitive outcomes, assessed by a comprehensive battery of paper and pencil tests and the CogState computerized test. These findings were consistent with their previous study of only one season of play (Rose et al., 2018).

Although the Rose studies are encouraging from a safety perspective, specific values and the long-term effects of repetitive HIE are not well understood. Concern for neurodegenerative disease has been put forth as a possible consequence of the accumulation of HIE over time, although this theory has been criticized for lack of specific evidence regarding the extent of the problem or the nature of the risks (e.g., how much HIE, for whom, and for how long) (Carman et al., 2015; for review, see McAllister & McCrea, 2017). Only a few studies have attempted to evaluate the long-term effects of sub-clinical HIE in youth football, and there is conflicting evidence on whether participation in football prior to high school is related to cognitive impairments later in life (Alosco et al., 2017; Montenigro, et al., 2016; Solomon et al., 2016).

The goal of this study was to determine if pre- to post-season changes in neuropsychological test scores of youth football players (aged 9–13) were related to HIEs using a well-utilized technology (HIT System, Lebanon, NH, USA) and a carefully designed analytic procedure. Participants were selected after their first season in the 5-year project. Data were collected from two teams at each of three sites over 4 years. At both pre-season and post-season, each participant completed a brief battery of neuropsychological tests. Participants wore helmets instrumented with biomechanical sensors to capture the frequency, magnitude, and location of HIE for all practices and games. The primary hypothesis was that greater HIE would be related to greater negative changes in neuropsychological test scores, thus reflecting an effect of HIE on cognition.

METHODS

This study is part of a larger National Institutes of Health (NIH)-funded project: “The biomechanical basis of concussions in youth football.” Institutional Review Board approval was obtained through Virginia Tech as the lead University.

Recruitment

Participants were recruited at the initial team meetings or at equipment handouts at the beginning of each football season by a research assistant trained for human subjects research by their respective institutions. Interested youth and parent(s) attended an information session with presentations by study staff at the end of which the informed consent and assent process was completed. See Figure 1 for recruitment flow diagram.

Fig. 1.

Fig. 1.

Open in a new tab

Recruitment and enrollment flow diagram. Note. NP = neuropsychological; HIE = head impact exposure; samples for each NP test: Pattern Comparison n = 218, Flanker n = 217, List Sort n = 216, and Rey Auditory Verbal Learning Test (RAVLT) n = 206.

Participants

Football players (contact group)

Participants were primarily male youth, aged 9 to 13 (mean 11.23, s.d. 1.27), with several racial/ethnic groups represented, the predominant membership of Caucasian and African–American groups. Only two females were represented and they were left in for inclusiveness. Demographics including psychiatric diagnoses, number of previous concussions, and previous years of contact sports are reported in Table 1. Between the years 2015 and 2018, two teams each from three states were recruited from community youth football leagues (Rhode Island, Virginia, NC, USA), for a total of 24 team seasons. Participants and teams were not consistent across all 4 years. That is, both teams and players may have participated in more than 1 year and not necessarily in contiguous years. Participation was also limited by the number of instrumented helmets available per program (n = 168) across the three sites. The number of those who declined participation was not recorded. Thirty-four (34) dropped out across the 18 teams. Of these participants, the primary reasons given have been league weight or age requirements (10), 5 were due to helmet-sensor discomfort, 4 due to having a recent injury, and 15 for unknown reasons. Of the 168 participants, 134 completed demographic information (80%). Only athletes in their first season in the study were included to minimize multiple test exposures at pre- and post-season.

Table 1.

Demographic characteristics

Characteristics N/frequency
Sex
 Male 217
 Female 2
Season mean head impact exposure 238 (s.d. 227)
Test intervals
 Days between pre-season to post-season test (median) 89
 Days from end of season to post-test (median) 11
Race/Ethnicity %
 Native American 1 <.01
 Asian 2 0.01
 African American 62 0.32
 Caucasian 113 0.58
 Mixed 9 0.05
 Total 187 0.96
 Missing 8 0.04
 Hispanic origin 8 0.04
Psychiatric – Developmental
 Picture Vocabulary Test 150 SS = 104.13 (s.d. =13.01)
 ADHD 26 0.13
 Language disability 1 0.01
 Reading disability 7 0.04
 Writing disability 2 0.01
 Math disability 3 0.02
 Autism 2 0.01
 Taking medication 15 0.08
 Average years of contact sport 1.49 3.97
 Previous concussions (one persion with two concussions, all the rest =1) 16 0.07
Maternal level of education:
 Less than high school 1 0.01
 High-school graduate/GED 25 0.17
 Some college 38 0.25
 Bachelor’s degree 49 0.32
 Master’s degree 27 0.18
 Doctoral degree 7 0.05
 Professional degree 4 0.03
 Total 151 1.00
Open in a new tab

Note. M = mean; s.d. = standard deviation; SS = Standard Score with mean= 100, standard deviation = 15; ADHD = Attention Deficit Hyperactivity Disorder; GED = General Education Diploma.

Controls (non-contact group)

As per recommendations (Heilbronner et al., 2010; Johnson Dow, Lynch, Hermann, 2006), data from a control group were collected to establish test–retest parameters for the regression-based Z score of change in the contact group. Thirty-five non-contact sport male participants, aged 9 to 13, were recruited from the community and consented to complete pre- and post-season assessments. All were male, and they were not directly matched beyond the age groups. The mean interval between test sessions was 154 day. This difference was slightly longer, but not statistically different than the 142-day interval of the contact group.

Measures

Demographics

Demographic variables collected included age, sex, height, weight, race/ethnicity, mother’s level of education, study site, psychiatric and learning diagnoses, time intervals between test 1 and test 2 and test 2 and assessment, the number of previous concussions, and the number of years playing a contact sport.

Neuropsychological tests

NIH Toolbox was used for neuropsychological tests due to excellent reliability and validity (Carlozzi, Beaumont, Tulsky, & Gershon, 2017; Carlozzi, Tulsky, Kail, & Beaumont, 2013; Tulsky, Carlozzi, Beaumont, & Mugas, 2013; Tulsky et al., 2017). Before the 2016 season, the Toolbox testing platform was changed from a desktop computer to an iPad. The two platforms were assessed and where there were differences, scoring adjustments were made so that scores were comparable (National Institutes of Health and Northwestern University; 2017; Slotkin, 2017).

Four tests were chosen: two for memory functions and two for processing speed functions (List Sorting, Rey Auditory Verbal Learning Test (RAVLT), Pattern Comparison, Flanker, and Picture Vocabulary). Tests were individually administered in distraction-free environments by trained research technicians according to standard protocols in laboratories at the respective sites. Testing was conducted at the beginning and end of each team’s season (the median interval from last recorded HIE to testing was 5 days, with range 2 to 51). The test descriptions are from the relevant manuals.

List sort

The List Sort test requires immediate recall and sequencing of different visually and orally presented stimuli. Pictures of different foods and animals are displayed with accompanying audio recording and written text (e.g., “elephant”), and the participant is asked to say the items back in size order from smallest to largest, first within a single dimension (either animals or foods, called 1-List) and then on two dimensions (foods, then animals, called 2-List).

Rey Auditory Verbal Learning Test

The Rey is a word-list learning task in which 15 unrelated words are presented orally by examiner or via audio recording over three consecutive learning trials. After each presentation, the participant is asked to recall as many of the words as he can.

Pattern comparison

This test measures speed of processing by asking participants to discern whether two side-by-side pictures are the same or not. The items are presented one pair at a time on the computer screen, and the participant is given 90 s to respond to as many items as possible (up to a maximum of 130).

Flanker

The Flanker task measures both a participant’s attention and inhibitory control and is thus a measure of efficiency of processing. The test requires the participant to focus on a given stimulus, while inhibiting attention to stimuli flanking it. Sometimes the middle stimulus is pointing in the same direction as the “flankers” (congruent) and sometimes in the opposite direction (incongruent). Twenty trials are conducted.

Picture vocabulary

The NIH Toolbox Picture Vocabulary Test is a measure of receptive vocabulary administered in a computer-adaptive test format. The participant is presented with four pictures on the iPad screen and an audio recording saying a word. The participant is instructed to touch the picture that most closely shows the meaning of the word. After the participant makes a choice, another set of pictures automatically appears with the next item and associated audio recording. For this study, the Picture Vocabulary was used as a proxy for full-scale IQ/general developmental level (Zelazo et al., 2013).

Sample neuropsychological test reliability

There were significant increases in all of the neuropsychological test scores from pre- to post-season (all paired sample t-test values p <.004). A two-way mixed effects model was used to calculate intraclass correlations (ICCs) as indicators of test–retest reliability. All measures demonstrated acceptable levels of test–retest reliability: RAVLT = .699, List Sort = .738, Pattern Comparison = .743, and Flanker = .801.

Neuropsychological test performance measures

Although formal performance validity tests or metrics were not obtained, test administrators completed behavioral rating observation forms and an administration fidelity form to assess performance validity. Fidelity measures included use of trained personnel, use of scripts for instructions, and having an appropriate setting for testing. For each test administered testing personnel kept track of distractions or disruptions: 1. Requests to repeat directions or asking for help understanding task; 2. Being distracted by external factors (e.g., fidgeting with phone or looking away from iPad to focus on something, outside phone ringing, talk in hallway outside of testing room); 3. Taking a break between tasks (e.g., bathroom break, getting snack, or water); 4. Needing encouragement, prompts for best effort; 5. Other distractions such as technical or computer problems (e.g., computer/iPad freezing). These instruments were developed by the authors and not formally validated.

Biomechanical measurement of HIE

Head impact measuring devices allow researchers to study on-field head impacts and collect kinematic data of concussive and sub-concussive head impacts (Brennan et al., 2017). The Head Impact Telemetry System (HIT) was developed to continuously monitor HIE in real time in football and ice hockey (Broglio et al., 2011; Cobb et al., 2013; Duma et al., 2005; Gysland et al., 2012; Wilcox et al., 2014). The HIT system has also been validated through several on-field and laboratory studies (Beckwith et al., 2012; Crisco, Chu, & Greenwald, 2018; Duma et al., 2005; Funk, Duma, Manoogian, & Rowson, 2007; Funk, Rowson, Daniel, & Duma, 2012; Manoogian, McNeely, Duma, & Brolinson, 2006). Research assistants attended each practice or game session to collect biomechanical HIE data and maintain the equipment. Standard video analysis of HIE events included deeming impacts valid if they occurred during organized team sessions (games or practices) and invalid if they occurred during water breaks or outside organized team sessions. Further, all impacts over 40g’s were inspected to assure accurate recording.

The biomechanical head impact variables analyzed were the total number of HIE impacts for the season, mean of the linear acceleration values, mean of the rotational acceleration values, and the mean HITsp. The HITsp metric is a transformation of the computed head impact measures of peak linear and peak angular acceleration into a single latent variable using principal component analysis and applies a weighting factor based on impact location (Greenwald, Gwin, Chu, & Crisco, 2008).

Analyses

Statistics were performed with the Statistical Package for Social Sciences, V 22 (SPSS, Chicago, IL, USA). Demographic variables were checked for relationship to dependent variables using graphical displays and correlation analysis with a p < .05 threshold for statistical significance. Categorical variables were analyzed using chi-square statistics.

Neuropsychological change scores were generated using test–retest data of a sample of non-contact youth of the same age and at similar test intervals (all males). Regression parameters of test 1 predicting test 2 were obtained and used to calculate a Z score for each pre-season to post-season test score difference of the contact sample. The actual transformation followed (Johnson et al., 2006) and is supported by multiple studies (Chelune, 2003; Heilbronner et al., 2010; Jacobson & Truax, 1991; Temkin, Heaton, Grant, & Dikmen, 1999). An advantage of regression-based Z scores over reliable change intervals is that Z scores are continuous variables and can be used in analyses as such. The test–retest data were also analyzed for reliability (ICC). A two-way mixed effects model is recommended for test–retest reliability with absolute agreement definition and the average of the two measures (Koo & Li, 2016; Shrout & Fleiss, 1979).

Biomechanical variables were initially evaluated using graphical displays of data. For all analyses, residuals were checked graphically and with tests for normality. Two broad functions of HIE values were analyzed to characterize (1) seasonal load and (2) maximum magnitude of exposures. Four values representing the total;season load were assessed: the mean value of the sum of seasonal linear, rotational and HITsp values, and the total number of HIE for the season. Except for the total number of HIE, the values were found to be adequately modeled as normally distributed in the sample of players. The frequency of head impacts was log-transformed and found to be normally distributed. The mean values were of interest as they represent a cumulative value of magnitude modified by the frequency and allows for direct comparison across subjects with varying degrees of exposure. To assess maximum HIE load, three sets of data were assessed: the maximum value of linear acceleration, rotational acceleration and HITsp in the participant’s season, the maximum value for the three metrics in the last week of the season, and the number of values for the three metrics that exceeded the relevant age groups 90th percentile. Non-parametric (Kendall’s tau) and parametric correlations to assess the effect of these high-magnitude HIE were calculated.

To answer the primary question, a series of linear regression models were constructed for each of two age-cluster groups (9–10 and 11–13 age groups). Each neuropsychological test change score was the dependent variable, and independent biomechanical variables were entered in a stepwise manner in one step, with weight and height entered in a second step (stepwise entry). Residual plots were used following model-fit to check for violations of assumptions. Multivariate analysis of variances (MANOVAs) were calculated to confirm results using the full set of HIE variables.

For any demographic variables contributing a significant amount of variance to the change scores, these variables were added as a second step in the regression model.

RESULTS

Demographic Analysis

This sample was primarily male (only two females); by age, the sample sizes were 9 = 31, 10 = 31, 11 = 51, 12 = 70, 13 = 36. The average number of HIE for the season was 238; however, there was considerable variability (s.d. = 227). Table 1 presents the demographic information. Analysis of the demographic variable’s effect on the change scores found no effect for any variable, except for Attention Deficit Hyperactivty Disorder (ADHD). The difference in RAVLT change scores between ADHD and non-ADHD was significant: t(205) = 2.416, p = .017, d = .448. The mean difference was .51 with the ADHD mean value lower. However, adding presence of ADHD as a dummy variable did not explain any significant amount of variance in the regression model, so it was dropped.

Analysis of Independent Variables

The creation of two age clusters was supported by MANOVA analysis of the three HIE values across the five ages: MANOVA F (3, 200) = 4.49, p<.001, ƞp2 = .071. Post hoc analysis found that across all three HIE variables age 9 was not different than age 10, while age 9 was different than ages 11 and up. Also, there were few significant differences between the 11 and 13 ages.

Differences between these two age clusters by each of the HIE variables were all significant: mean linear acceleration t(205) = 4.898, p<.001, d = .735, mean rotational acceleration t(205) = 2.741, p<.001, d = .411; mean HITsp t(205) = 3.169, p= .002, d = .475. Subsequent analyses were conducted on both the full sample and by age cluster for each neuropsychological test change score, including age group and weight. Table 2 presents statistics for HIE variables by age cluster.

Table 2.

Characteristics of head impact exposure variables by age cluster

Variable Age cluster N Mean s.d. Mean difference 95% CI
Lower Upper
Mean linear 9–10 62 20.74 2.6
11–13 157 22.04 2.51
Full sample 219 21.67 2.6 −1.301 −2.052 −0.551
Mean rot. 9–10 62 924.67 138.64
11–13 157 1026.32 139.28
Full sample 219 997.54 146.17 −101.641 −142.765 −60.518
Mean HITsp 9–10 62 13.91 1.39
11–13 157 14.57 1.43
Full sample 219 14.38 1.44 −0.652 −1.071 −0.234
Open in a new tab

Note. All age-cluster differences significant p < .003; s.d. = standard deviation; 95% CI = 95th %ile Confidence Interval; Mean rot. = mean rotational acceleration.

An analysis of variance of weight by age group was significant: F(4,168) = 12.223, p < .001, ƞp2 = .215. Post hoc analysis identified two distinct groups with ages 9 and 10 not significantly different from each other by weight, and ages 11, 12, and 13 not significantly different from each other, but both 9 and 10 significantly different from 11, 12, and 13. While height was significantly different across all ages, body mass index was similar to weight with ages 9 and 10 similar, and ages 11, 12, and 13 similar.

Season HIE Values and Neuropsychological Change Scores

The regression-based change scores (RBz) were the dependent measures in the regression models with HIE values in the first step and weight in the second; the RBz means are presented in Table 3. Negative mean scores indicate change that was less than would be expected given the test–retest changes.

Table 3.

Regression-based change score (RBz) means and standard deviations (s.d.) by age cluster

Variable 9–10
 11–13
 Total
n Mean s.d n Mean s.d. n Mean s.d.
RAVLT RBz 58 −0.19 1.18 149 −0.17 0.95 207 −0.17 1.02
List Sort RBz 60 −0.18 0.90 157 −0.04 1.04 217 −0.08 1.01
Pat Com RBz 62 0.00 0.69 156 0.17 0.66 218 0.12 0.67
Flanker RBz 62 0.15 0.81 156 0.13 0.98 218 0.13 0.94
Open in a new tab

Note. RAVLT = Rey Auditory Verbal Learning Test; Pat Com = Pattern Comparison.

There were no significant findings within the older cohort (aged 11–13) for any model. However, all three HIE metrics predicted significant changes only in RAVLT scores in the younger age cluster.

In the age 9–10 cohort, the three seasonal mean HIE values with an age by weight interaction term were each significantly related to RAVLT score changes in the hypothesized direction (see Table 4). The first step in each was significant, but R2 values increased significantly with the addition of the interaction term.

Table 4.

Regression models predicting RAVLT change scores for younger and older age clusters

9–10 age cluster
 11–13 age cluster
Variable Model B SE B β Model B SE B β
Linear 1 −0.176 0.079 −0.338* 1 0.025 0.032 0.068
Linear 2 −0.128 0.068 −0.244 2 0.027 0.032 0.073
Age × Weight 2 −0.003 0.001 −0.525 2 0.00 0.00 0.059
R2 = 0.381 R2 = 0.00
F = 11.714 df = 2,38 p < .001 F = 0.52 df = 2, 129 p = .593
Rotational 1 −0.005 0.001 −0.478** 1 0.00 0.001 −0.031
Rotational 2 −0.004 0.001 −0.376 2 0.00 0.001 −0.023
Age × Weight 2 −0.003 0.001 −0.351 2 0.00 0.001 −0.023
R2 = 0.438 R2 = 0.003
F = 14.787 df=2,38 p<.001 F = 0.208 df = 2, 129 p=.723
HITsp 1 −0.52 0.123 −0.56*** 1 0.031 0.056 0.049
HITsp 2 −0.358 0.125 −0.386 2 0.034 0.056 0.052
Age × Weight 2 −0.002 0.001 −0.402 2 0.00 0.00 0.056
R2 = 0.444 R2 = 0.005
F = 15.197 Df = 2,38 p < .001 F = 0.353 df = 2, 129 p=.703
Open in a new tab

Notes. B = Unstandardized beta; SE B = Standard error of beta; β = standardized beta; Step 1 p-values

*

<.05.

**

<.01

***

<.001.

To better understand the effect of weight on the change scores, groups were created based on a median split of weight within each age cluster;. The RBz scores for RAVLT were then compared and significant differences emerged in the younger age cluster, with the higher weight subgroup showing more significant negative change (RBȥ = −.961, n = 19) and the lower weight showing a non-significant positive change (RBz = .335, n = 22): t(42) = 3.806, p <.001, d = .115. No significant weight by RBȥ differences was found in the older age cluster. Thus, the weight differences accounted for the increased effect of the biomechanical measures on neuropsychological score (Figure 2).

Fig. 2.

Fig. 2.

Open in a new tab

Comparison of regression-based Z scores (RBz) by weight subgroup and age cluster.

Further, in the younger cluster, the heavier subgroup experienced significantly more exposures (frequency of HIE) over the course of the season: 185 HIE (lighter) versus 311 HIE (heavier), t(43) = 2.09, p = .042, d = .638. The number of HIE’s for the season was not different between age clusters: t(217) = .916, p = .361, d = .124; nor was it different between heavier and lighter in the older age cluster: t(138) = .718, p = .474, d = .122.

Effects of High-Magnitude HIE on Neuropsychological Change Scores

No relationships were identified comparing the high-magnitude HIE metrics and neuropsychological change scores (i.e., number of linear, rotational, or HITsp values greater than 90th %ile for age). Analyses were conducted for the whole sample first and then the two age clusters. Morphological variables were also unrelated to these high-magnitude HIE metrics.

Clinical Significance

When unexpected negative changes in test scores are encountered, there are two issues that must be addressed. First, are the changes clinically meaningful, and second, does the number of such changes exceed the base rate? A 90% confidence interval (CI) was used as the threshold for significant clinical change. Using regression-based change statistics, a one-tailed 90% CI would reflect a change score of less than −1.28. In the lighter cohort, only three of 21 cases had RAVLT RBz scores =/< −1.28 (14%); however, for the heavier subgroup, 9 of 18 cases (50%) had significantly low RBz scores: X2 = 5.804, p = .016, φ = −.386.

In terms of base rate, the full sample found 33 of 208 cases with significantly low RBz scores (15%). The odds ratio (OR) was 2.05, 95% CI 0.957–4.403 but not significant (p = .06). While the CI included 1.00, the upper range suggests an effect.

DISCUSSION

In this sample of football players aged 9 to 13, the accumulation of HIE during one season of football had a negative effect on cognition as indexed by the change in a list-learning test from pre-season to post-season. While a subtle effect of rotational acceleration on negative changes in RAVLT scores was noted for the full sample, this did not hold for the older age cluster (aged 11–13) but was verified in the younger age subset (aged 9 and 10). The relationships between HIE values and RAVLT change scores were inversely proportional: higher mean seasonal HIE values predicted significantly less than expected improvement in RAVLT change scores. Further, an unexpected but significant mediating effect was identified for participant weight in the younger sample. Additional analysis revealed that the heaviest half of the younger group had significantly greater negative test score changes and a higher rate of clinically meaningful results. The differential effect in the heavier subgroup indicated that weight and age increased the effect and was not just an effect of weight or of age independently.

It is possible that the heavier young participants were selected to play more often and thus had more opportunity for exposure. The high incidence of clinically meaningful changes in the subgroup participants was noteworthy. While the heavier weight subgroup of younger athletes was made up primarily of 10-year olds, the effect was not found when analyzing the 10-year olds by themselves. Further, the effect of just HIE on change score was significant, if small for the younger age clustter, without the interaction term. Thus, a small but statistically significant risk was conferred on the 9–10 group as a whole.

In general, the current study’s findings pointed to risk for heavier, younger athletes. In youth football, positions are fluid so they were not reliably tracked. However, in older cohorts, it is established that linemen receive the highest frequency counts of HIE (Broglio et al., 2011; Crisco et al., 2011). The heavier athletes here had slightly higher mean HIE frequencies (not significant).

It was not the case here that the amount of high-magnitude HIE in the last week of play was related to cognitive change as was observed in a college sample (McALlister et al., 2012). These findings also differ from two recent studies by Rose et al. who failed to find any relationship between neuropsychological function and HIE in youth football over one and two seasons. However, there were several important differences between the studies. The Rose studies used a new head impact measurement device for documenting HIE that registers HIE by category which limited both measurement specificity and analytic power. Additionally, they did not separate ages, likely because of the small cell sizes that would have been generated within the nine age groups.

While this brief battery cannot claim to measure specific neurocognitive constructs, it does represent the cognitive framework employed in most concussion testing (i.e., speed and memory; see Schatz and Maerlender 2013). List-learning activates a network of neurocognitive functions that most certainly includes working memory, that has a long history of disruption in mild traumatic brain injury (McDonald et al. 2012). The list-learning task reflects the ability to encode auditory-verbal lists of words over several trials. Although the specific findings were different in a previous study, McAllister et al. found relationships between the California Verbal Learning Test 2nd Ed. and somewhat different HIE metrics (McAllister et al., 2012). In that study, a decline in neuropsychological test scores was related to high-magnitude impacts in the last week of play (McAllister et al., 2012). This relationship was further supported by diffusion tensor imaging (McAllister et al., 2014).

There are several limitations to this study that must be acknowledged. Given the observational nature of the results, there may be other explanations than those offered here that warrant further study or confirmation. We did not have the sample size nor design to tease out some potentially relevant causative factors (e.g., coach behavior and years of football experience). All of the demographic and background information was from parent report including years of contact sport, number of previous concussions, and psychiatric diagnoses. The accuracy of these reports were not established and may have not been completely accurate.

The HIT system measurement devices have been used in multiple studies but the technology can be criticized if not used with proper experimental controls (e.g., video confirmation and regular checking of the output). The values of rotational accelerations are estimated and not directly measured, increasing measurement error. However, the consistency in these data gives support to the findings.

As suggested, the scope of the project did not allow for assessment of coaching practices that might offer further information regarding the etiology of differences in exposure. Although considerable controls were in place for data collection (biomechanical data and neuropsychological data), the multi-site nature of the project also injected likely random errors, including an inablility to obtain complete data for all cases. In addition, other variables and factors not measured in the study might be important (e.g., more complete race/ethnicity data, genetics such as APOE4++, multiple season exposures, and HIE intervals).

A sensitivity analysis eliminating the two female participants resulted in no appreciable change in mean scores for any primary study variable. Their data were left in the analyses out of respect for their participation.

The generalizability of the findings is also unclear. While age and weight effects have some intuitive appeal as a factor in the effects of HIE, other factors may be at play such as football experience, actual playing time, and the aforementioned coaching practices. And it bears repeating that this was a predominantly male sample.

With those caveats in mind, these results point to a differential effect of repetitive HIE on cognition in younger athletes with particular risk to those young players who are heavier and younger. The amount of change in the heavier subgroup was clinically meaningful and significantly frequent. We noted that all of the age groups of this study were heavier than census data for these ages (Kuczmarski et al., 2002). That we only found these changes in a young-heavy subgroup raise interesting possibilities. At such young ages, the effect of heavier weight on the younger frame provides less structural integrity when absorbing HIE. This pre-adolescent stage is one of changing neural development: under these conditions, repetitive forces may cause a disruption of neurogenesis and thus lowering higher-order cognitive processes (Casey, Tottenham, Liston, & Durston, 2005; Casey & Jones, 2010). Brain effects may be secondary to increased cardiovascular reactivity (Len et al. 2011). Without replication, we cannot rule out a spurious finding. Further, these data do not say whether the changes are permanent or temporary.

It is critical to note that replication of these findings is needed before strong conclusions about the safety of youth football are drawn. Further, the HIE values presented here should not be used as benchmarks for clinical use until validation has been provided. Whether or not this effect persists through successive seasons needs to be determined, although the data indicate that older cohorts are potentially at less risk.

ACKNOWLEDGMENTS

Research reported in this publication was supported by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under Award Number R01NS094410. We would like to thank the youth, parents, and coaches for their time and effort participating in this project; also, thanks to Tina Strates for database assistance.

Footnotes

CONFLICTS OF INTEREST

Joseph J. Crisco,Richard M. Greenwald,Jonathan G. Beckwith, and Simbex have a financial interest in the instruments (HIT System, Sideline Response System, Riddell, Inc) that were used to collect the biomechanical data reported in this study.

References

  1. Alosco ML, Kasimis AB, Stamm JM, Chua AS, Baugh CM, Daneshvar DH,::: Stern RA. (2017). Age of first exposure to American football and long-term neuropsychiatric and cognitive outcomes. Translational Psychiatry, 7(9), e1236 10.1038/tp.2017.197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bahrami N, Sharma D, Rosenthal S, Davenport EM, Urban JE, Wagner B, Jung Y, Vaughan CG, Gioia GA, Stitzel JD, Whitlow CT, & Maldjian JA (2016). Subconcussive head impact exposure and white matter tract changes over a single season of youth football. Radiology, 281(3), 919–926. 10.1148/radiol.2016160564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bazarian JJ, Zhong J, Blyth B, Zhu T, Kavcic V, & Peterson D (2007). Diffusion tensor imaging detects clinically important axonal damage after mild traumatic brain injury: a pilot study. Journal of Neurotrauma, 24(September), 1447–1459. 10.1089/neu.2007.0241. [DOI] [PubMed] [Google Scholar]
  4. Bazarian JJ, Zhu T, Zhong J, Janigro D, Rozen E, Roberts A,:::Blackman EG. (2014). Persistent, long-term cerebral white matter changes after sports-related repetitive head impacts. PLoS ONE, 9(4). 10.1371/journal.pone.0094734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Beckwith JG, Greenwald RM, & Chu JJ (2012). Measuring head kinematics in football: correlation between the head impact telemetry system and hybrid III headform. Annals of Biomedical Engineering, 40(1), 237–248. 10.1007/s10439-011-0422-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brennan JH, Mitra B, Synnot A, McKenzie J, Willmott C, McIntosh AS,:::Rosenfeld JV. (2017). Accelerometers for the assessment of concussion in male athletes: a systematic review and meta-analysis. Sports Medicine, 47(3), 469–478. 10.1007/s40279-016-0582-1. [DOI] [PubMed] [Google Scholar]
  7. Broglio SP, Eckner JT, Martini D, Sosnoff JJ, Kutcher JS, & Randolph C (2011). Cumulative head impact burden in high school football. Journal of Neurotrauma, 28(10), 2069–2078. 10.1089/neu.2011.1825. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Broglio SP, Surma T, & Ashton-Miller JA (2012). High school and collegiate football athlete concussions: a biomechanical review. Annals of Biomedical Engineering, 40(1), 37–46. 10.1007/s10439-011-0396-0. [DOI] [PubMed] [Google Scholar]
  9. Carlozzi NE, Beaumont JL, Tulsky DS, & Gershon RC (2017). The NIH toolbox pattern comparison processing speed test: normative data. Mongraphs of the Society for Research in Child Development, 30(May 2015), 359–368. 10.1093/arclin/acv031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carlozzi NE, Tulsky DS, Kail RV, & Beaumont JL (2013). NIH toolbox cognition battery(CB): measuring processing speed. Mongraphs of Thw Society for Research in Child Development, 78(4), 88–102. 10.1111/mono.12036.VI. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carman AJ, Ferguson R, Cantu R, Comstock RD, Dacks PA, DeKosky ST,::: Fillit HM. (2015). Expert consensus document: mind the gaps—advancing research into short-term and long-term neuropsychological outcomes of youth sports-related concussions. Nature Reviews Neurology, 11(4), 230–244. 10.1038/nrneurol.2015.30. [DOI] [PubMed] [Google Scholar]
  12. Casey BJ, & Jones RM (2010). Neurobiology of the adolescent brain and behavior: implications for substance use disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 49(12), 1189–1285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Casey BJ, Tottenham N, Liston C, & Durston S (2005). Imaging the developing brain: what have we learned about cognitive development? Trends in Cognitive Sciences, 9(3), 104–110. [DOI] [PubMed] [Google Scholar]
  14. Chelune G (2003). Assessing reliable neuropsychological change, In Franklin RD (Ed.), Predictio in forensics and neuropsychology:sound statostocal practices, (pp. 123–147). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. [Google Scholar]
  15. Cobb BR, Urban JE, Davenport EM, Rowson S, Duma SM, Maldjian JA, Whitlow CT,::: Stitzel JD (2013). Head impact exposure in youth football: elementary school ages 9–12 years and the effect of practice structure. Annals of Biomedical Engineering, 41(12), 2463–2473. 10.1007/s10439-013-0867-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Crisco JJ, Wilcox BJ, Beckwith JG, Chu JJ, Duhaim A-C, Rowson S, Duma SM,::: Greenwald RM (2011). Head impact exposure in collegiate football players. Journal of Biomechanics, 44(15). 10.1016/j.jbiomech.2011.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Crisco JJ, Chu JJ, & Greenwald RM (2018). An algorithm for estimating acceleration magnitude and impact location using multiple nonorthogonal single-axis accelerometers. Journal of Biomechanical Engineering, 126(December 2004), 849–854. 10.1115/1.1824135. [DOI] [PubMed] [Google Scholar]
  18. Daniel RW, Rowson S, & Duma SM (2012). Head impact exposure in youth football. Annals of Biomedical Engineering, 40(4), 976–981. 10.1007/s10439-012-0530-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Davenport EM, Whitlow CT, Urban JE, Espeland MA, Jung Y, Rosenbaum DA, Gioia GA, Powers AK, Stitzel JD, & Maldjian JA (2014). Abnormal white matter integrity related to head impact exposure in a season of high school varsity football. Journal of Neurotrauma, 31(19), 1617–1624. 10.1089/neu.2013.3233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Duma SM, Manoogian SJ, Bussone WR, Brolinson PG, Goforth MW, Donnenwerth JJ,::: Crisco JJ (2005). Analysis of real-time head accelerations in collegiate football players. Clinical Journal of Sport Medicine, 15(1), 3–8. 10.1097/00042752-200501000-00002. [DOI] [PubMed] [Google Scholar]
  21. Funk JR, Duma SM, Manoogian SJ, & Rowson S (2007). Biomechanical risk estimates for mild traumatic brain injury. Annual Proceedings/Association for the Advancement of Automotive Medicine, 51, 343–361. [PMC free article] [PubMed] [Google Scholar]
  22. Funk JR, Rowson S, Daniel RW, & Duma S (2012). Validation of concussion risk curves for collegiate football players derived from HITS data. Annals of Biomedical Engineering, 40(1), 79–89. 10.1007/s10439-011-0400-8. [DOI] [PubMed] [Google Scholar]
  23. Greenwald RM, Gwin JT, Chu JJ, & Crisco JJ (2008). Head impact severity measures for evaluating mild traumatic brain injury risk exposure. Neurosurgery, 62(4), 789–798. 10.1227/01.NEU.0000311244.05104.96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gysland SM, Mihalik JP, Register-Mihalik JK, Trulock SC, Shields EW, & Guskiewicz KM (2012). The relationship between subconcussive impacts and concussion history on clinical measures of neurologic function in collegiate football players. Annals of Biomedical Engineering, 40(1), 14–22. 10.1007/s10439-011-0421-3. [DOI] [PubMed] [Google Scholar]
  25. Heilbronner RL, Sweet JJ, Attix DK, Krull KR, Henry GK, & Hart RP (2010). Official position of the American academy of clinical neuropsychology on serial neuropsychological assessments: the utility and challenges of repeat test administrations in clinical and forensic contexts. Clinical Neuropsychologist, 24(8), 1267–1278. 10.1080/13854046.2010.526785. [DOI] [PubMed] [Google Scholar]
  26. Institute of Medicine (IOM) and National Research Council (NRC). (2014). Sports-related concussions in youth: improving the science, changing the culture. Washington, DC: The National Academies Press. [PubMed] [Google Scholar]
  27. Jacobson NS, & Truax P (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. 10.1037/0022-006X.59.1.12. [DOI] [PubMed] [Google Scholar]
  28. Johnson EK, Dow C, Lynch RT, & Hermann BP (2006). Measuring clinical significance in rehabilitation research. Rehabilitation Counseling Bulletin, 50(1), 35–45. 10.1177/00343552060500010501. [DOI] [Google Scholar]
  29. Kelley ME, Urban JE, Miller LE, Jones DA, Espeland MA, Davenport EM,::: Stitzel JD. (2017). Head Impact Exposure in Youth Football: Comparing Age- and Weight-Based Levels of Play. Journal of Neurotrauma, 34(11), 1939–1947. 10.1089/neu.2016.4812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kerr ZY, Yeargin S, Valovich McLeod TC, Nittoli VC, Mensch J, Dodge T,::: Dompier TP. (2015). Comprehensive coach education and practice contact restriction guidelines result in lower injury rates in youth American football. Orthopaedic Journal of Sports Medicine, 3(7), 1–8. 10.1177/2325967115594578. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Koo TK, & Li MY (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. 10.1016/j.jcm.2016.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kuczmarski RJ, Ogden CL, Guo SS, Grummer-Strawn LM, Flegal KM, Mei Z, Wei R, Curtin LR, Roche AF, Johnson CL (2002). 2000 CDC growth charts for the United States: methods and development. Vital Health Statistics, 11(246), 1–190. [PubMed] [Google Scholar]
  33. Len T, Neary JP, Asmundson G, Goodman D, Bjornson B, Bhambani Y (2011). Cerebrovascular reactivity impairment after sport-induced concussion. Medicine & Science in Sports & Exercise, 43(12), 2241–2248. 10.1249/MSS.0b013e3182249539. [DOI] [PubMed] [Google Scholar]
  34. Manoogian S, McNeely D, Duma S, & Brolinson PG (2006). Head acceleration is than less than 10 percent of helmet acceleration in football impacts. Biomedical Sciences Instrumentation, 42, 383–388. [PubMed] [Google Scholar]
  35. McAllister T, & McCrea M (2017). Long-term cognitive and neuropsychiatric consequences of repetitive concussion and head-impact exposure. Journal of Athletic Training, 52(3), 309–317. 10.4085/1062-6050-52.1.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. McAllister TW, Flashman LA, Maerlender A, Greenwald RM, Beckwith JG, Tosteson TD,:::Turco JH. (2012). Cognitive effects of one season of head impacts in a cohort of collegiate contact sport athletes. Neurology, 78(22). 10.1212/WNL.0b013e3182582fe7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. McAllister TW, Ford JC, Flashman LA, Maerlender A, Greenwald RM, Beckwith JG,::: & Jain S. (2014). Effect of head impacts on diffusivity measures in a cohort of collegiate contact sport athletes. Neurology, 82(1). 10.1212/01.wnl.0000438220.16190.42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. McDonald BC, Saykin AJ, & McAllister TW (2012). Functional MRI of mild traumatic brain injury (mTBI): progress and perspectives from the first decade of studies. Brain Imaging and Behavior, 6(2), 193–207. 10.1007/s11682-012-9173-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Montenigro PH, Alosco ML, Martin BM, Daneshvar DH, Mez J, Chaisson CE,::: Triposdis Y. (2016). Cumulative head impact exposure predicts later-life depression, apathy, executive dysfunction, and cognitive impairment in former high school and college football players. Journal of Neurotrauma, 34, 328–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Munce TA, Dorman JC, Thompson PA, Valentine VD, & Bergeron MF (2014). Head impact exposure and neurologic function of youth football players. Medicine and Science in Sports and Exercise, 14, 1567–1576. 10.1249/MSS.0000000000000591. [DOI] [PubMed] [Google Scholar]
  41. National Institutes of Health and Northwestern University. (2017). NIH toolbox ® for assessment of neurological and behavioral function administrator’s manual. Evanston, IL: Northwestern University Press. [Google Scholar]
  42. Nauman EA, Breedlove KM, Breedlove EL, Talavage TM, Robinson ME, & Leverenz LJ (2015). Post-season neurophysiological deficits assessed by ImPACT and fMRI in athletes competing in American football. Developmental Neuropsychology, 40(2), 85–91. 10.1080/87565641.2015.1016161. [DOI] [PubMed] [Google Scholar]
  43. Rose SC, Yeates KO, Fuerst DR, Ercole PM, Nguyen JT, & Pizzimenti NM (2018). Head impact burden and change in neurocognitive function during a season of youth football. Journal of Head Trauma Rehabilitation, 34(2), 87–95. 10.1097/HTR.0000000000000441. [DOI] [PubMed] [Google Scholar]
  44. Rose SC, Yeates KO, Nguyen JT, McCarthy MT, Ercole PM, & Pizzimenti NM (2019). Neurocognitive function and head impact burden over two seasons of youth tackle football. Journal of Neurotrauma, 36(19), 2803–2809. 10.1089/neu.2019.6519. [DOI] [PubMed] [Google Scholar]
  45. Rosenthal JA, Foraker RE, Collins CL, & Comstock RD (2014). National high school athlete concussion rates from 2005–2006 to 2011–2012. American Journal of Sports Medicine, 42(7), 1710–1715. 10.1177/0363546514530091. [DOI] [PubMed] [Google Scholar]
  46. Schatz P, & Maerlender A (2013). A two-factor theory for concussion assessment using impact: memory and speed. Archives of Clinical Neuropsychology, 28(8). 10.1093/arclin/act077. [DOI] [PubMed] [Google Scholar]
  47. Shrout P, & Fleiss J (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. [DOI] [PubMed] [Google Scholar]
  48. Slotkin J (2017). NIH toolbox equivalency study. Retrieved from http://www.healthmeasures.net/images/LearnMore/2017_User_Conference_Slides/Track_D/Breakout_2/D2.4_-_Equivalency_Toolbox_Slotkin.pdf.
  49. Solomon GS, Kuhn AW, Zuckerman SL, Casson IR, Viano D, Lovell MR, & Sills AK (2016). Participation in pre –high school football and neurological, neuroradiological, and neuropsychological findings in later life a study of 45 retired national football league players. American Journal of Sports Medicine, 44 10.1177/0363546515626164. [DOI] [PubMed] [Google Scholar]
  50. Talavage TM, Nauman EA, & Yoruk U. (2014). Functionally-detected cognitive impairment in high school football players without clinically diagnosed concussion. Journal of Neurotrauma, 31(4), 327–338. 10.1089/neu.2010.1512. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Tarnutzer AA, Straumann D, Brugger P, & Feddermann-Demont N (2017). Persistent effects of playing football and associated (subconcussive) head trauma on brain structure and function: a systematic review of the literature. British Journal of Sports Medicine, 51(22), 1592–1604. 10.1136/bjsports-2016-096593. [DOI] [PubMed] [Google Scholar]
  52. Temkin NR, Heaton RK, Grant I, & Dikmen SS (1999). Detecting significant change in neuropsychological test performance: a comparison of four models. Journal of the International Neuropsychological Society, 5(4), 357–369. 10.1017/S1355617799544068. [DOI] [PubMed] [Google Scholar]
  53. Tsushima WT, Geling O, Arnold M, & Oshiro R (2016). Are there subconcussive neuropsychological effects in youth sports? an exploratory study of high- and low-contact sports. Applied Neuropsychology: Child, 5(2), 149–155. 10.1080/21622965.2015.1052813. [DOI] [PubMed] [Google Scholar]
  54. Tulsky D, Carlozzi N, Beaumont J, & Mugas. (2013). NIH toolbox cognitive functioning battery: measuring working memory. Mongraphs of the Society for Research in Child Development, 78(4), 70–87. 10.1111/mono.12035.NIH. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Tulsky D, Carlozzi N, Holdnack J, Heaton R, Wong A, Goldsmith A, & Heinemann A (2017). Using the NIH toolbox cognition battery (NIHTB-CB) in individuals with traumatic brain injury. Rehabilitation Psychology, 62, 413–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Urban JE, Davenport EM, Golman AJ, Maldjian JA, Whitlow CT, Powers AK,::: Rose KL. (2013). Head impact exposure in youth football: high school ages 14 to 18 years and cumulative impact analysis. Annals of Biomedical Engineering, 41(12), 2474–2487. 10.1007/s10439-013-0861-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Wilcox BJ, Beckwith JG, Greenwald RM, Chu JJ, McAllister TW, Flashman LA,::: Crisco JJ. (2014). Head impact exposure in male and female collegiate ice hockey players. Journal of Biomechanics, 47(1). 10.1016/j.jbiomech.2013.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Zelazo PD, Anderson JE, Richler J, Wallner-allen K, Beaumont JL, & Weintraub S (2013). II. NIH toolbox cognition battery (CB): measuring executive function and attention. Mongraphs of the Society for Research in Child Development, 78,16–33. [DOI] [PubMed] [Google Scholar]

RESOURCES