Skip to main content
Journal of Neurotrauma logoLink to Journal of Neurotrauma
. 2017 Jan 15;34(2):328–340. doi: 10.1089/neu.2016.4413

Cumulative Head Impact Exposure Predicts Later-Life Depression, Apathy, Executive Dysfunction, and Cognitive Impairment in Former High School and College Football Players

Philip H Montenigro 1,,2, Michael L Alosco 1, Brett M Martin 3, Daniel H Daneshvar 1, Jesse Mez 1,,4, Christine E Chaisson 1,,3,,5, Christopher J Nowinski 1,,6, Rhoda Au 4,,7, Ann C McKee 1,,4,,8,,9, Robert C Cantu 1,,6,,10,,11, Michael D McClean 12, Robert A Stern 1,,2,,4,,10,,*,, Yorghos Tripodis 1,,5,,*
PMCID: PMC5220530  PMID: 27029716

Abstract

The term “repetitive head impacts” (RHI) refers to the cumulative exposure to concussive and subconcussive events. Although RHI are believed to increase risk for later-life neurological consequences (including chronic traumatic encephalopathy), quantitative analysis of this relationship has not yet been examined because of the lack of validated tools to quantify lifetime RHI exposure. The objectives of this study were: 1) to develop a metric to quantify cumulative RHI exposure from football, which we term the “cumulative head impact index” (CHII); 2) to use the CHII to examine the association between RHI exposure and long-term clinical outcomes; and 3) to evaluate its predictive properties relative to other exposure metrics (i.e., duration of play, age of first exposure, concussion history). Participants included 93 former high school and collegiate football players who completed objective cognitive and self-reported behavioral/mood tests as part of a larger ongoing longitudinal study. Using established cutoff scores, we transformed continuous outcomes into dichotomous variables (normal vs. impaired). The CHII was computed for each participant and derived from a combination of self-reported athletic history (i.e., number of seasons, position[s], levels played), and impact frequencies reported in helmet accelerometer studies. A bivariate probit, instrumental variable model revealed a threshold dose-response relationship between the CHII and risk for later-life cognitive impairment (p < 0.0001), self-reported executive dysfunction (p < 0.0001), depression (p < 0.0001), apathy (p = 0.0161), and behavioral dysregulation (p < 0.0001). Ultimately, the CHII demonstrated greater predictive validity than other individual exposure metrics.

Keywords: : behavior, cognition, concussion, football, long-term impairment, subconcussive impacts

Introduction

The term “repetitive head impacts” (RHI) refers to the cumulative exposure to recurrent concussive and subconcussive events.1–6 A concussive event is a direct or indirect impact to the head of sufficient intensity to produce overt clinical symptoms that typically resolve within days or weeks,5 but can persist for months7 or ≥1 year.8 The mechanisms of concussion are believed to involve shearing and tensile forces to axons that occur during the impact-induced acceleration, deceleration, and rotational forces of the brain.9 A subconcussive event is similar in that it also involves a transfer of mechanical energy to the brain at enough force to injure axonal integrity, but does not result in clinical symptoms.1–4,10,11 However, the intensity of impact necessary to trigger this potential neuronal damage is not known, and whether there are levels of impact that are benign to neuronal functioning has also not been determined.

The investigation of RHI has most commonly been in contact sports, such as American football. More than 4,500,000 amateur athletes participate in tackle football each year,12,13 and this sport has one of the highest rates of concussion.14,15 Surveys of high school and college athletes show that ∼50% of football players sustain a concussion each year, and >30% sustain multiple concussions.14,16,17 Subconcussive events are likely even more frequent, as helmet-based accelerometer studies estimate that amateur football players average 600 subconcussive impacts per season in high school and >1000 at the collegiate level.3 The high prevalence of concussive and subconcussive events in amateur football players is concerning, given their reported association with acute18–20 and chronic21–25 neurological consequences. Repetitive subconcussive blows (measured by helmet accelerometer sensors recording events that exceed 14.4g10) are associated with pre- to post-season cognitive decline,10,26 functional brain alterations (e.g., reduced neurophysiological health),10,26,27 and microstructural white matter brain changes28 in high school football players.

Although research on this topic still is limited, cumulative concussive as well as subconcussive impacts may be a key contributor to later-life neurological consequences, including the neurodegenerative disease, chronic traumatic encephalopathy (CTE).29 Importantly, 16% of pathologically confirmed cases of CTE have no reported history of concussion,30 highlighting the potential long-term risks of subconcussive injury. In addition, although CTE has been described predominantly in former professional contact sport athletes, a recent study of deceased amateur athletes and controls found that a history of contact sport involvement was the greatest risk factor for CTE neuropathology.31 Various exposure metrics (e.g., age of first exposure to football,32–34 duration of football play,11,35–37 concussion history21–25) have been linked to later-life cognitive and neurobehavioral disturbances in former football players and other contact sport athletes. These different metrics may reflect different aspects of RHI exposure, each with slightly different effects on the brain.11,35

A direct relationship between RHI and long-term clinical outcomes has been difficult to formally test because of the lack of validated tools to quantify cumulative RHI exposure.38 Quantifying RHI exposure is methodologically challenging, given that it involves a self-reported assessment of multiple events that occur throughout one's athletic career. Thus far, research on RHI and long-term outcomes has relied on single, indirect metrics based on a subject's history of traumatic brain injury (TBI), which involves retrospective self-reports or proxy reports using a structured interview containing validated scales.39–41 These scales include those recommended by the National Institute of Neurological Disorders and Stroke (NINDS) common data elements and the Centers for Disease Control (CDC).42 Despite known limitations, numerous studies have demonstrated the usefulness of retrospective report in predicting long-term consequences following multiple concussions.21,22,43 Although short-term head impact exposure could be quantified prospectively with the placement of accelerometers in the helmets of athletes, this approach does not estimate cumulative exposure over one's athletic career, which is speculated to be of primary importance for predicting later-life impairments and CTE.44,45 In response to this need, Kerr and coworkers recently proposed the Head Impact Exposure Estimate (HIEE),46 which estimates a football player's total hours of contact exposure, excluding exposure prior to high school, using self-report interview. However, the HIEE has not been validated in a model with clinical outcomes.

In our study, we developed and validated a metric to estimate an athlete's total cumulative exposure to RHI from football, referred to as the “cumulative head impact index” (CHII). To derive the CHII we combined two sources of information: 1) individual self-report measures of athletic exposure, and 2) extrapolated objective measures based on position(s) played, obtained from published helmet accelerometer studies. Our metric includes estimates for youth, high school, and collegiate level exposure, and incorporates percentages for all positions played (i.e., primary, secondary, tertiary). The purpose of this study was to evaluate the relationship between the CHII and later-life cognitive, behavioral, and mood impairment. Additionally, we compared the predictive validity of the CHII against three other individual exposure metrics; namely, total season/years played, age at first exposure (AFE), and overall concussion history.21–23,25,32,33,43,47 We hypothesized that cumulative head impact exposure would have a measurable threshold, above which the risk of developing later life clinically meaningful cognitive, mood, and behavioral impairment increases significantly.

Methods

Study design

This present sample was part of a larger ongoing study, the Longitudinal Examination to Gather Evidence of Neurodegenerative Disease (LEGEND) study, at the Boston University Alzheimer's Disease and Chronic Traumatic Encephalopathy (CTE) Center. LEGEND is a longitudinal study to assess potential risk factors for short- and long-term consequences of RHI. Participation involves annual telephone-based cognitive assessments; web-based measures of mood, behavior, and cognition; and saliva sampling for APOE genotyping. The LEGEND research protocol was approved by the Institutional Review Board of Boston University Medical Campus and written consent was obtained from all LEGEND participants. Study participation was open to adults, ≥18 years of age, who were either active or former athletes, across all sports and levels of play. There is no active recruitment program for LEGEND. Rather, this is a convenience sample in which potential subjects learn of the study through descriptions on the investigators' web sites and through word of mouth. Therefore, the LEGEND sample should not be viewed as being representative of all athletes. Detailed descriptions of the LEGEND protocol have been previously published.25,47,48

Participants

Of the 800 participants in the LEGEND data set at the time of analysis, 93 former amateur football players met the following inclusion and exclusion criteria: 1) highest level of football played was at high school or college, 2) no concussion was sustained in the year prior to their initial evaluation (to diminish potential effect of acute brain injuries on clinical outcome measures), and 3) they did not participate in another contact sport (i.e., amateur wrestling, boxing, bull riding, diving, horse jumping, ice hockey, karate, lacrosse, martial arts, mixed martial arts, entertainment wrestling, rugby, or soccer). The final sample included 17 former high school football players and 76 former collegiate football players. Demographic characteristics of the sample are shown in Table 1.

Table 1.

Demographics

  Total sample n = 93 Mean (SD) High school n = 17 Mean (SD) College n = 76 Mean (SD) *p value
Age (years) 47.3 (13.9) 43.6 (11.6) 47.7 (14.2) 0.2760
Formal education (years) 16.9 (2.1) 15.3 (2.1) 17.2 (2.0) 0.0005
Education (terminal degree n [%])
 High School/GED 11(11.8) 6 (35.3) 5 (6.6) 0.0018
 Bachelor's/Associates 50 (53.8) 7 (42.2) 43 (56.6)  
 Master's or Doctorate 32 (34.4) 4 (23.5) 28 (36.8)  
Working full or part time n (%) 63 (67.7) 10 (58.8) 53 (69.8) 0.673
Marital status, married n (%) 39 (41.9) 5 (62.5) 34 (63.0) 0.658
Body mass index 31.5 (5.6) 31.5 (6.7) 31.5 (5.3) 0.994
Handedness, right n (%) 74 (79.6) 13 (76.5) 61 (80.3) 0.484
Military history n (%) 14 (15.1) 5 (29.4) 9 (11.8) 0.0671
*

Wilcoxon two sample test. Numbers in parentheses are standard deviations except when otherwise indicated.

Health and athletic history

Participants were administered a structured questionnaire that has been used previously,25,32,47 designed to collect retrospective information about the participant's lifetime athletic experience, past medical history, and concussion history. Questions about athletic experience captured variables regarded by the literature as being potential predictors of brain trauma, such as: 1) sports played,11,49 2) AFE to tackle football,32–34 3) levels of play50–53 (youth, high school, college, professional), 4) number of seasons played at each level,27,28,54 5) total years played,23,35–37 6) all positions55–58 played for each sport (first, second, third) at each level, 7) percentage of game time played at each position,46,58 and 8) age at retirement from the sport.36

The self-reported number of concussions was obtained, after participants were read a “modern” definition of concussion, based on the CDC statement on sports-related concussion59 and the Third International Conference on Concussion in Sport held in Zurich.60 The concussion history characteristics of the sample are provided in Table 2. Because the distribution of self-reported concussions is highly skewed, we used the log of concussions in all subsequent analyses to normalize the distribution. Participants were also asked to report their AFE to tackle football and the total number of seasons they played (i.e., duration of football play) (Table 2). For our analysis, participant's AFE was converted into a dichotomous variable: AFE before age 12 and AFE at age 12 or above. Age 12 was selected as the cutoff, based on our previous work on AFE, as an exposure metric associated with later-life clinical and structural changes.32,33 For duration of play, we used seasons rather than years because for some athletes, a year of football involved both a fall and spring season.

Table 2.

Exposure Variables

  Total sample n = 93 Mean (SD) High school n = 17 Mean (SD) College n = 76 Mean (SD) p value
Concussion history median (IQR)a 20 (3) 10 (13) 20 (39) 0.0431*
CHII 5805.7 (3091.2) 3237.6 (1932.4) 6384.4 (3015.4) <.0001*
Age at first exposure (years) 10.9 (2.7) 10.8 (2.7) 11.0 (2.7) 0.884
Average years of youth football 3.3 (2.5) 3.4 (2.3) 3.3 (2.5) 0.931
Seasons of football play 11.1 (4.2) 7.1 (3.1) 12.0 (3.1) <0.0001
Time since retirement (years) 26.3 (14.2) 26.4 (12.2) 26.3 (14.4) 0.971

Numbers in parentheses are standard deviations except when otherwise indicated.

a

Interviewers read participants a current definition of concussion (methods described elsewhere47) and asked to estimate their concussion history based on that definition.

*

Wilcoxon two sample test.

IQR, interquartile range; CHII, Cumulative Head Impacts Index.

CHII

The CHII was developed as an estimate of an athlete's total cumulative exposure to RHI from football. To derive an individual's CHII we combined data from two sources of information: 1) individual self-report measures of athletic exposure obtained from the LEGEND questionnaire at each level played (youth, high school, college), such as number of seasons played, positions played at each level (primary, secondary, tertiary), and the proportion of the season's total game time played at each position; and 2) a measure of estimated head impacts received per season, based on data from published helmet accelerometer studies that report the frequency of head impacts per season by position and level of play (See following section and Table 3).55–57,61,62

Table 3.

Summary of Data Collected from Review of Helmet-Accelerometer Studies

Level Position Study Minimum acceleration n Mean/ Median Weighted mean impacts per season
College DL Crisco et al. 201057 14.4g 29 1086 871
    Mihalik et al. 200755 10.0g 13 965  
    Crisco et al. 201162 10.0g 49 718  
  LB Crisco et al. 201057 14.4g 29 846 685
    Mihalik et al. 200755 10.0g 9 655  
    Crisco et al. 201162 10.0g 47 592  
  DB Crisco et al. 201057 14.4g 34 487 417
    Mihalik et al. 200755 10.0g 12 731  
    Crisco et al. 201162 10.0g 55 306  
  OL Crisco et al. 201057 14.4g 46 960 728
    Mihalik et al. 200755 10.0g 22 921  
    Crisco et al. 201162 10.0g 75 543  
  OB/RB Crisco et al. 201057 14.4g 23 459 412
    Mihalik et al. 200755 10.0g 12 589  
    Crisco et al. 201162 10.0g 37 326  
  WR Crisco et al. 201057 14.4g 16 305 237
    Mihalik et al. 200755 10.0g 5 501  
    Crisco et al. 201162 10.0g 30 157  
  QB Crisco et al. 201057 14.4g 8 305 206
    Crisco et al 201162 10.0 g 14 149  
High School QB Broglio et al. 201161 15.0g 4 467 467
  WR/DB Broglio et al. 201161 15.0g 28 372 372
  RB/LB Broglio et al. 201161 15.0g 27 619 619
  DL/OL Broglio et al. 201161 15.0g 41 868 868
Youth All positions Daniel et al. 201256 10.0g 7 107 107

Player positions selected by our study participants were grouped, whenever necessary, in accordance with position groups as reported in the selected helmet-accelerometer studies and cross-referenced with position groups as characterized by relevant and authoritative publications.58,114

DL, defensive linemen; LB, linebackers; DB, defensive backs; OL, offensive linemen; OB/RB, offensive backs or running backs; WR, wide receivers; QB, quarterbacks; WR/DB, wide receivers and defensive backs (cornerbacks and safeties); DL/OL, linemen. For more detailed information see references listed in table.

Published accelerometer studies used to determine frequency of head impacts by position and level of play

To obtain estimates of the frequency of head impacts by position and level of play, we performed a systematic review of the literature (using PubMed) on helmet accelerometer studies, using the following key words/search terms: HITS system, 6DOF, accelerometers, helmet-sensor, football, youth, high school, and college. Studies that satisfied the following a priori inclusion criteria were selected.

  • 1. Head Impacts were measured during every practice and game for the entire season.

  • 2. Level of play (youth, high school, college) was identified.

  • 3. Head impact frequencies were reported for positions of play.

  • 4. Any impact event with a peak linear acceleration <10g was excluded from analysis. A minimum cutoff of 10g ensures the elimination of nonimpact events (e.g., jumping) from the calculation of head impact frequency.37, 56,63–66

Table 3 provides a summary of key data points obtained from each accelerometer study that met our inclusion criteria. The participants in the studies summarized in Table 3 were active in both games and practices. Because most players at the youth level play at multiple positions, we include a single number to reflect exposure for all seasons each player spent at the youth level. There was a single high school study identified that grouped together certain positions, such as linesmen, regardless of whether or not they played at offensive or defensive positions. Because many players at the high school level would interchangeably play at both offense and defense, using a similar exposure metric for these positions is a realistic assumption.61 Impact frequencies from these studies were pooled and weighted, whenever possible, to derive averages weighted by each study's sample size. These weighted averages are estimates of the impact frequencies per position at the different levels of play (youth, high school, college).

The CHII was calculated from the self-report variables and weighted impact frequencies using the equations outlined in Table 4. A hypothetical case is provided to illustrate the calculation of the CHII.

Table 4.

Calculation of the Cumulative Head Impacts Index

  % games played at first position X Position's weighted average no. impacts per season X Total no. of youth seasons  
  +  
Youth % games played at second position X Position's weighted average no. impacts per season X Total no. of youth seasons  = [A]
  +  
  % games played at third position X Position's weighted average no. impacts per season X Total no. of youth seasons  
+
  % games played at first position X Position's weighted average # impacts per season X Total no. of youth seasons  
  +  
High school % games played at second position X Position's weighted average no. impacts per season X Total no. of youth seasons  = [B]
  +  
  % games played at third position X Position's weighted average no. impacts per season X Total no. of youth seasons  
+
  % games played at first position X Position's weighted average no. impacts per season X Total no. of youth seasons  
  +  
College % games played at second position X Position's weighted average no. impacts per season X Total no. of youth seasons  = [C]
  +  
  % games played at third position X Position's weighted average no. impacts per season X Total no. of youth seasons  

Cumulative Head Impacts Index = [A] + [B] + [C]

Mr. A is a 42-year-old male who reports having participated in football at the youth, high school, and collegiate levels.

  • 1. In college, Mr. A reported that he had played a total of three seasons. His primary position for his college team was linebacker (LB); he reported having no secondary or tertiary positions of play. He estimated having participated in 85% of game downs as a linebacker. Therefore, his college CHII was: (85%) × (685 impacts per season for LB from Table 5) × (3 seasons) = 1747.

  • 2. In high school, Mr. A reported that he had played for all four seasons. His primary position for his high school team was also LB; he reported having a secondary position playing the offensive line (OL) as a guard. Of all the games in high school, he estimated having participated in 40% of game downs as a LB and 30% as OL. Therefore, his high school CHII was: [(40%) × (619 impacts per season for LB) × (4 seasons)] + [(30%) × (868 impacts per season for OL) × (4 seasons)] = 2032.

  • 3. Lastly, Mr. A reported that he had played four seasons of football prior to high school. He reported having played as an OL throughout his youth participation. He estimated that he participated in 90% of game downs for all 4 seasons. Therefore, his youth CHII was: [(90%) × (107 impacts per season for any position) × (4 seasons)] = 385.

  • 4. His overall CHII = 1747 + 2032 + 385 = 4164.

Table 5.

Behavior, Mood, and Cognition Outcome Measures

  Total sample n = 93 Mean (SD) High school n = 17 Mean (SD) College n = 76 Mean (SD) p value
Behavioral Regulation Index of the BRIEF-A68 64.1 (15.9) 64.4 (16.0) 64.0 (16.0) 0.9288
Metacognition Index of the BRIEF-A68 64.5 (16.9) 62.2 (14.8) 65.0 (17.4) 0.5493
Global Executive Composite of the BRIEF-A68 65.3 (16.9) 64.2 (15.0) 65.6 (17.3) 0.7563
Center for Epidemiologic Studies Depression Scale69 21.7 (15.5) 21.7 (17.9) 21.7 (15.0) 0.9854
Apathy Evaluation Scale70 35 (11.5) 34.2 (12.5) 35.2 (11.3) 0.7455
Brief Test of Adult Cognition by Telephone79 0.04 (0.9) −0.11 (1.05) 0.08 (0.90) 0.4359

BRIEF-A, Behavior Rating Inventory of Executive Function – Adult Version.

Clinical outcomes and measures

The following set of instruments administered to LEGEND participants were selected for the current study: 1) Brief Test of Adult Cognition by Telephone (BTACT)67; 2) Behavior Rating Inventory of Executive Function - Adult Version (BRIEF-A)68; 3) Center for Epidemiologic Studies - Depression Scale (CES-D)69; and 4) Apathy Evaluation Scale (AES).70 Each of these measures was chosen a priori based on its previous use in studies of TBI and concussion (BRIEF-A,25,71,72 AES,73–75 CES-D,76–78 BTACT79), and on the availability of validated cutoff scores suggesting clinically meaningful impairment.68,73,80–82

BTACT

The BTACT67,79 is an objective measure of cognitive function administered by telephone. The benefits and validity of cognitive test batteries administered by telephone are well documented.79,83 The BTACT requires 20 min to complete and consists of six subtests that measure episodic verbal memory (Immediate and Delayed Rey Auditory-Verbal Learning Test), working memory (Digits Backward), verbal fluency (Animals Categorical Fluency), task switching (Red/Green Test), inductive reasoning (Number Series), and processing speed (Backward Counting).67 A global composite score was derived with the bi-factor approach, which has an overall improved validity over the single-factor approach and is better able to distinguish among persons with a lifetime history of head injury.79 Age- and gender-corrected scores were scaled relative to a healthy normative sample. Objective cognitive impairment was defined as 1.5 standard deviations (SDs) below the normative mean.82,84

BRIEF-A

Participants completed an online version of the BRIEF-A questionnaire,68 a well-validated 75 item measurement of executive function behavior in activities of daily living. Participants rate “how often each of the [75] following behaviors has been a problem?” in the past month on a three point Likert scale (1 = never, 2 = sometimes, 3 = often); higher scores indicate a greater degree of dysfunction. We used the global measure of executive function (Global Executive Composite [GEC]), as well as two factor-based measures of cognitive regulation (Meta-cognition Index [MI]) and behavioral-emotional regulation (Behavioral Regulation Index [BRI]). Raw scores were converted into standardized age-adjusted T scores (mean = 50, SD = 10). T-scores ≥1.5 SDs of the normative mean (T scores ≥65) are considered to indicate clinical impairment.68

CES-D

The CES-D69 is a 20 item self-report measure of depression symptoms that was developed and validated by the National Institute of Mental Health.69 Participants rate their depression symptom severity in the past week on a four point Likert scale that ranges from “none of the time” to “all of the time.” Higher scores indicate more severe depressive symptoms, with an established total CES-D cutoff score ≥16 reliably indicating clinically meaningful depression.80,81

AES

The AES70 is an 18 item self-report measure of apathy over the previous 4 weeks. Participants rate their apathetic emotions, thoughts, and behaviors in the past 4 weeks on a four point Likert scale that ranges from “not at all characteristic” to “very characteristic.” Higher total AES scores indicate worse apathy, with an established cutoff total AES score ≥34 reliably indicating clinically meaningful apathy.73

Statistical analysis

Group comparisons

The former high school and collegiate football player groups were compared using two sample t tests for continuous normal variables, Wilcoxon two sample tests for non-normal continuous variables, and χ2 tests for categorical or dichotomous variables.

Regression modeling

To test our study hypothesis, we modeled each dichotomous outcome measure (probability of impairment or not) with our predictor metric (CHII). This model is illustrated in Figure 1. We first identified a point (threshold dose) at which the magnitude of the relationship (slope) changes from a zero magnitude (B1 = 0) to a non-zero magnitude (B2 > 0). The change point threshold shows the conversion of the relationship from a baseline risk of playing football to a dose-response relationship above which higher exposure to head impacts can lead to higher risk of impairment.85 The change point was identified using a Bayesian hierarchical model estimated by Markov Chain Monte Carlo (MCMC) method with 30,000 simulations implemented in PROC MCMC in SAS 9.4.85 An important contribution in modeling head impact exposure to long-term outcomes is the incorporation of concussion history into the model. Because a head impact is a necessary condition for a concussion, we cannot estimate the effect of head impacts by controlling for concussions in a simple regression setup. This relationship between predictors where a value of one variable is directly caused by another is often described as endogenous.86 If endogeneity exists between variables within our model, the following two problems would occur: 1) a linear regression model that includes both the CHII and concussion history would give invalid inference, and 2) the estimates of the slopes from such a regression model would be biased. To address the inference and bias problems caused by endogeneity we used an “instrumental variable model” with log concussions as the instrument.87 The instrumental variable model can be described as a two stage regression. In the first stage regression, we estimated the effect of the log of the number of concussions on the CHII, whereas in the second stage regression we used the predicted values from the first stage to estimate the effect of the CHII on the probabilities of impairment for each of the outcomes. Both regressions included age and education as covariates. Next, we applied the instrumental variable model using a bivariate probit model. The estimated effect of cumulative head impacts derived from this instrumental variable model is equivalent to an effect estimated among the compliers from a study where exposure was prospectively randomized.88 Therefore, an instrumental variable model allows for causal inference from observational data with large measurement error.88,89 Further, the instrumental variable approach reduces the measurement error of the estimates of frequency of head impacts by position and level of play. Because the studies used to estimate these frequencies are independent from the LEGEND study, their measurement error is uncorrelated from the self-reported concussions in LEGEND. Therefore, in this instance, concussions become an ideal instrument to calibrate and reduce the measurement error of head impact exposure. Similar approaches in different settings have been used in econometric and clinical therapeutic studies.90,91

FIG. 1.

FIG. 1.

Schematic of the dose-response model with a constant baseline risk of later-life impairment (baseline gradient of slope = 0) below the cumulative head impact threshold dose and with increasing probability of impairment (dose-response gradient of slope >0) above that threshold dose. Color image is available online at www.liebertpub.com/neu

Comparison with other exposure metrics

We also examined total seasons played and AFE as simple exposure metrics that have demonstrated a significant relationship with clinical outcomes in other studies.11,32,33,35,36 Total season played and AFE were independently added to our bivariate probit model, with age and education as covariates. Next, we modeled clinical outcomes with concussion history in a separate univariate probit model, with age and education included as covariates.

Results

Participant demographics and group comparisons

Participant demographics (Table 1), exposure variables (Table 2), and outcome measures (Table 5) were compared across the two highest levels of play; that is, high school and college. The college level group had significantly more years of education (t test = 3.58, p value = 0.0005), more seasons played (t test = 4.85, p value < 0.0001), a greater number of concussions (Wilcoxon = 592.5, p value = 0.0431), and a higher CHII (t test = 2.47, p value = 0.0156). All other group comparisons were nonsignificant. Mean scores on the BTACT, an objective measure of cognitive function, indicated that the entire sample was, on average, cognitively normal (Table 5).

CHI exposure and risk of impairment

The CHII was calculated for all 93 participants, and the means for former high school and collegiate football players are listed in Table 2.

The CHII change points listed in Table 6 indicate the threshold number of impacts, above which a dose response relationship is initiated between exposure (CHII) and the risk of impairment. Figure 2 shows the predicted probabilities of impairment for different doses of exposure. The baseline risk of impairment significantly increases linearly after a change point as exposure increases in all outcomes. (See Table S1 for a tabular summary of the data depicted in Fig. 2) (see online supplementary material at http://www.liebertpub.com)

Table 6.

Change Point Thresholds from Baseline Constant Risk to Dose-Response Relation Between CHII and Risk of Impairment

Outcome measure Clinical domain Threshold*mean CHII for dose-response 95% CI
Behavioral Regulation Index of the BRIEF-A Behavior 2216 1537-8120
Metacognition Index of the BRIEF-A Metacognition 2393 1571-6682
Global Executive Composite of the BRIEF-A Executive Function 1850 1523-2011
Center for Epidemiologic Studies Depression Scale Depression 1801 1514-2010
Apathy Evaluation Scale Apathy 2160 1536-5754
Brief Test of Adult Cognition by Telephone Cognition 7251 1754-9788
*

The Mean and 95% CI from 30,000 Markov Chain Monte Carlo simulations. Change-point thresholds represent the mean number of impacts above which there is a predictive dose-response relationship between head impacts and clinically meaningful measures of impairment.

BRIEF-A, Behavior Rating Inventory of Executive Function – Adult Version.

FIG. 2.

FIG. 2.

Each dichotomous outcome measure (A–F) was fit using a bivariate probit, an instrumental variable model adjusted for both age and education as well as our predictor variable of interest (cumulative head impact index [CHII]). Baseline risk refers to the risk of impairment at the CHII dose range below the thresholds identified in Table 6. Above these thresholds, a significant linear dose-response relationship between CHII exposure and later-life clinical impairment was found for all outcomes. See Table S1 for tabular data (see online supplementary material at http://www.liebertpub.com)

Specifically, we find that the risk of impairment increases steadily every 1000 impacts, or about twice the sample's average's season number of impacts (545) above the baseline change point. For all outcomes, the risk of developing clinically meaningful impairments in mood, behavior, and cognition increased considerably with two additional seasons worth of head impacts. For example, we found that adding 10 seasons of impacts above the baseline threshold increased a subject's risk of developing objective cognitive impairment by 9-fold.

Other exposure metrics and risk of impairment

We added alternate exposure metrics to our bivariate probit model to test whether other metrics might be better at predicting clinical outcomes and/or whether they might eliminate the predictive significance of the CHII. Total seasons of play and AFE did not add independently to our bivariate probit model, nor did they eliminate the significance of the CHII for predicting clinical outcomes. When added to the model with the CHII, participants with AFE <12 showed some increase in the risk for impairment but did not achieve significance on any outcome after adjusting for CHI: GEC (β = 0.41, p value = 0.2542), MI (β = 0.36, p value = 0.3456), BRI (β = 0.56, p value = 0.073), CES-D (β = 0.49, p value = 0.1771), AES (β = 0.33, p value = 0.4170), and BTACT (β = 0.81, p value = 0.1427). Interestingly, when controlling for CHI, increasing the total number of seasons played reduced the risk for clinical impairment: BRI (β = −0.11, p value = 0.0183), MI (β = −0.09, p value = 0.0847), GEC (β = −0.07, p value = 0.1586), CES-D (β = −0.09, p value = 0.1224), AES (β = −0.07, p value = 0.2011), and BTACT (β = −0.03, p value = 0.6390). The significant negative association between total seasons played and BRI is consistent with previous research demonstrating a neurobehavioral benefit from regular exercise.92–94 The betas (β indicates the magnitude and direction of the relationship among the predictor, AFE or total seasons, and the risk for impairment in a model that also takes into account a quantitative estimate of CHI exposure. For example, positive betas indicate a positive association between exposure and impairment, such that increasing exposure was associated with an increased probability of impairment.

Lastly, we explored the role that concussion history had on predicting clinical outcomes, and compared its predictive power to that of the CHII. Because of the endogeneity problem previously described (i.e., that head impacts are a necessary factor for both the CHII and concussions) we could not simply include concussion history as a covariate in our CHII model. The Spearman correlation between concussion history and the CHII was 0.22 (p value = 0.02). Therefore, we ran a univariate probit model with age and education as covariates, in order to evaluate the predictive power of concussions on the probability of impairment. The estimates were: BRI (β = 0.50, p value = 0.0109), MI (β = 0.41, p value = 0.0255), GEC (β = 0.48, p value = 0.0138), CES-D (β = 0.48, p value = 0.0195), AES (β = 0.30, p value = 0.0717), and BTACT (β = 0.43, p value = 0.1526). Compared with the CHII, concussion history was limited to predicting self-reported changes in behavioral regulation (BRI), cognition (MI), executive dysfunction (GEC), and depression (CES-D), but not apathy (AES), or objective cognitive dysfunction (BTACT). Even for the significant outcomes (BRI, MI, GEC, CES-D), concussion history was found to have less predictive power than the CHII.

Discussion

We developed a metric of cumulative exposure to RHI from football, the CHII, a measure that includes self-reported exposure and estimated quantitation of head impacts based on published helmet accelerometer studies. The CHII was used to examine the relationship between cumulative RHI exposure and later-life cognitive, mood, and behavioral impairment in a sample of former football players whose highest level of play was either high school or college. Subjects denied having played any other contact sport at any time. The mean CHII of 5806 total impacts for our sample is consistent with the range of cumulative impacts expected for former high school and college football players.3 We found that the CHII strongly predicts later-life clinical outcomes, outperforming other individual metrics such as concussion history, AFE to tackle football, and total duration of play, suggesting that it is a useful metric to estimate lifetime RHI exposure.

We view this study as an initial examination of the CHII, and our findings of the relationship between earlier RHI exposure and later life impairments should be considered preliminary because of several limitations described subsequently. Moreover, this was not a study of CTE or neurodegeneration; our outcome measures focused on clinically meaningful levels of cognitive, mood, and behavior impairment and did not include any biomarkers of underlying disease or injury. This study does, however, underscore the importance of subconcussive trauma in the development of later life neurological impairment. The number of self-reported concussions predicted impairment in fewer outcomes than the CHII. Compared with the CHII, concussion history did not significantly predict apathy or objective cognitive dysfunction (BTACT). These findings are consistent with previous imaging studies of amateur athletes that demonstrated that changes in brain functional imaging occurred after a single season of cumulative asymptomatic impacts without concussion,10,26,27 and postmortem studies that indicated that 16% of former contact sport athletes with pathologically verified CTE had no concussion history.30,36 These studies together with our current findings highlight the critical need to evaluate prospectively the potential risk and later-life consequences of exposure to repetitive asymptomatic blows to the head (i.e., subconcussive impacts).

This study is the first in the literature to demonstrate a threshold dose-response relationship between estimated cumulative head impact exposure from football and later-life risk for cognitive and neurobehavioral impairment. We found that after a threshold, the risk for impairment increased with additional impacts. Similar findings of a threshold dose-response from RHI have been reported in studies of soccer and boxing.35,95–97 Specifically, our results show that the risk of developing behavioral dysregulation, executive dysfunction, depression, and apathy nearly doubled with 2800 additional impacts above the threshold. Our findings also show that increasing the CHI dose from 6500 to >12,000 increased the risk for objective cognitive impairment by a considerable 25-fold.

Interestingly, the dose-response threshold for cognitive function was much higher relative to other outcomes. Although this is speculative, it is possible that changes in mood and behavior reflect a different underlying mechanism or areas of structural impairment than cognitive changes, and that the cognitive changes reflect the evolution and progression of underlying CTE pathology.36,98 This is consistent with previous imaging studies in soccer players, wherein a lower threshold for detecting microstructural brain changes from soccer heading was identified compared with the threshold for memory impairment on neuropsychological evaluation.95 Further, in CTE, individuals with cognitive symptoms present later in the clinical course than those with behavioral impairments.98

Our findings indicate that below identified thresholds, accumulated impacts have no apparent effect on the risk for cognitive or neurobehavioral impairment. However, individuals had a baseline constant risk below the threshold. This baseline risk does not suggest that safety below the threshold is assumed. Moreover, our study design does not allow us to determine a safe time to cease RHI exposure; that is, quit football. Further, the baseline risk and the threshold likely depend upon several subject-specific factors such as possible genetic susceptibility, body mass index, socioeconomic status, and cognitive reserve.99 Large, well-controlled, prospective, longitudinal research that distinguishes threshold effects and baseline risk in football is clearly needed to identify maximum exposure levels for each player's safety. Such research would ultimately facilitate the development of safety guidelines that could minimize the risk of adverse effects on the brain in football.

Considering the public health implications of our study, there is also a need to investigate any causal evidence between RHI exposure and clinical impairment. Establishing cause indicates the possibility of intervention. We utilized the Bradford–Hill criteria for evaluating causal inference in observational studies,100 and our findings are suggestive of a causal relationship between cumulative head impact exposure and later-life clinical impairment. We analyzed eight Bradford–Hill criteria in our model: strength, dose-response, consistency, plausibility, coherence, experiment, reversibility, temporality, and specificity. Strength of association between our predictor, the CHII and all six measures of clinical impairment was very strong and higher than any other exposure predictor. The dose-response criterion is of particular importance in disorders that exhibit a latent onset, and this was supported by the present study.101 Regarding consistency of association, the dose-response relationship has also been shown in boxers.94,102,103 The plausibility of association is supported by human and animal studies that show cumulative subconcussive impacts and axonal Injury and blood–brain barrier damage.10,11,28,93,104–107,108 Since 1928, a relationship has been hypothesized between RHI and neurological disease, supporting the notion of coherence of association.106 Animal models have shown that a reduced number of impacts results in reduced negative consequences to support experimentation of association and the reversibility of association.109,110 Lastly, exposure must precede the disease, which conforms to observations in all previous studies and demonstrates temporality of association.

These findings do not suggest that every individual with a history of RHI – even exposure above the reported thresholds – will have later-life cognitive, mood, or behavioral impairments or develop a neurodegenerative disease, such as CTE. Support of a causal relationship does not imply a universal relationship. For example, a causal relationship between smoking tobacco and subsequent lung cancer has long been accepted. However, not all smokers develop lung cancer; some develop other conditions and diseases causally linked to smoking (e.g., stroke, heart disease, emphysema), and still others remain healthy throughout their lives. We hope that the use of the CHII (and subsequent iterations of this exposure metric) will result in greater clarification and validation of the causal relationship between football-related RHI and later-life clinical impairment.

The current study has clear limitations. Although all the accelerometer studies used to measure RHI discarded linear accelerations <10g, some of the studies used a slightly higher minimum cutoff to register a hit. However, small differences in cutoff points are likely not meaningful given that the majority of recorded impacts had mean accelerations well above the minimum cutoff points (as supported by the data summarized in Table 3). For example, the frequency of impacts per season reported by Mihalik and coworkers55 was lower than the frequency reported by Crisco and coworkers,57 despite having used a lower cutoff (Mihalik 10g; Crisco 14.4g). Additional research using both helmet-based and head-based accelerometers is needed to develop standardized, consensus based cutoff points.66 In addition, within the CHII, it is assumed that players are active in both games and practice, and does not take into account players who do not participate in games. Participants in our study were active players in games in high school and in college. For example, former college players in our study averaged 3.6 years of play and 50% game involvement. Therefore, because of the possible difference in RHI exposure between reserve players (who participate in all practices but have limited game involvement) and starters (who participate in all practices and have extensive game involvement), and because of the potential discrepancy in the frequency and severity of RHI exposure between practices and games, our findings may not accurately reflect differences in long-term consequences between starters and reserve players. Moreover, our study evaluated only certain aspects of exposure (i.e., cumulative impacts) and did not use other possible biomechanical metrics, such as cumulative linear and rotational accelerations. The sample size of the present study was also small because of the strict inclusion and exclusion criteria. The present results will need replication in larger studies. Further, despite our analytic approach, the study design is cross-sectional, which limits the extent to causal inference. Our convenience sample may induce a self-selection bias limiting the external validity of our findings. However, our instrumental variable analytic approach reduced the measurement error bias and potential confounding effects.111 Specifically, this method allowed us to provide unbiased estimates of causal effects in our nonrandomized sample88,89 and is increasingly utilized in clinical studies112,113 particularly when there are obstacles to performing a randomized controlled trial.

Future studies will be necessary to validate the CHII and improve our understanding on the long-term clinical consequences of RHI exposure. Larger studies are needed to investigate the effect of CHI exposure by age and level. Studies utilizing objective fluid and neuroimaging biomarkers will allow for a better understanding of the underlying etiology associated with CHI exposure. Case-controlled postmortem studies will also be necessary to examine the association between the CHII and CTE (and other) neuropathology. There is also the need to examine additional potential risk factors that may modify the relationship between RHI exposure and later-life cognitive and neurobehavioral impairment, including, but not limited to, genetics, diet, exercise, substance use (including performance enhancing drugs), and cardiovascular risk. Finally, there is a need for similar models of cumulative RHI exposure for other contact sports, once accelerometer or some other objective measure of head impacts is more widely available.

Conclusion

We developed the CHII, a tool to quantify retrospective cumulative exposure to RHI, including subconcussive impacts. Using the CHII, we showed that RHI exposure among amateur football players is associated with later-life cognitive and neurobehavioral consequences. Although our findings raise safety concerns for participation in amateur football, prior to changes in policy and rules, prospective longitudinal research in larger samples is needed to validate the CHII and replicate and extend our associated findings.

Supplementary Material

Supplemental data
Supp_Table1.pdf (25.1KB, pdf)

Acknowledgments

This study was funded in part by a grant from the Concussion Legacy Foundation, as well as National Institutes of Health (NIH) grant U01NS093334. Dr. Montenigro received partial support for this study from a Boston University School of Medicine Medical Student Summer Research Program Scholarship and from a Graduate Medical Sciences Student Organization Community Service Award. Dr. Alosco is supported by the T32-AG06697 post-doctoral fellowship. The authors thank the following individuals: Sarah Fairfield, Dr. Brandon Gavett, Kaitlin Perry, David Riley, Clifford Robbins, and members of the Boston University Alzheimer's Disease and CTE Center. The authors also extend their gratitude and appreciation to all the participants who made this study possible.

Author Disclosure Statement

Dr. Stern is a paid consultant to Amarantus Bioscience, Avanir Pharmaceuticals, and Biogen. He also serves as an expert advisor to attorneys for cases pertaining to the long-term consequences of repetitive brain trauma. He receives royalties from Psychological Assessment Resources for the publication of neuropsychological tests. Dr. Cantu receives compensation from the National Football League as Senior Advisor to the Head, Neck, and Spine Committee; from the National Operating Committee on Safety of Athletic Equipment as Chairman of the Scientific Advisory Committee; and from Concussion Legacy Foundation as co-founder and medical director for some talks given and research conducted. He receives royalties from Houghton Mifflin Harcourt and compensation from expert legal opinion. All other authors report no competing financial interests.

References

  • 1.Erlanger D.M. (2014). Exposure to sub-concussive head injury in boxing and other sports. Brain inj. 29, 171–174 [DOI] [PubMed] [Google Scholar]
  • 2.Dashnaw M.L., Petraglia A.L., and Bailes J.E. (2012). An overview of the basic science of concussion and subconcussion: where we are and where we are going. Neurosurg. Focus 33, E5. [DOI] [PubMed] [Google Scholar]
  • 3.Bailes J.E., Petraglia A.L., Omalu B.I., Nauman E., and Talavage T. (2013). Role of subconcussion in repetitive mild traumatic brain injury: a review. J. Neurosurg. 119, 1235–1245 [DOI] [PubMed] [Google Scholar]
  • 4.Belanger H.G., Vanderploeg R.D., and McAllister T. (2015). Subconcussive blows to the head: a formative review of short-term clinical outcomes. J. Head Trauma Rehabil. [Epub ahead of print] [DOI] [PubMed] [Google Scholar]
  • 5.Belanger H.G., and Vanderploeg R.D. (2005). The neuropsychological impact of sports-related concussion: a meta-analysis. J. Int. Neuropsychol. Soc. 11, 345–357 [DOI] [PubMed] [Google Scholar]
  • 6.Carman A.J., Ferguson R., Cantu R., Comstock R.D., Dacks P.A., DeKosky S.T., Gandy S., Gilbert J., Gilliland C., and Gioia G. (2015). Mind the gaps-advancing research into short-term and long-term neuropsychological outcomes of youth sports-related concussions. Nat. Rev. Neurol. 11, 230–244 [DOI] [PubMed] [Google Scholar]
  • 7.Ponsford J., Willmott C., Rothwell A., Cameron P., Kelly A., Nelms R., Curran C., and Ng K. (2000). Factors influencing outcome following mild traumatic brain injury in adults. J. Int. Neuropsychol. Soc. 6, 568–579 [DOI] [PubMed] [Google Scholar]
  • 8.Bigler E.D. (2008). Neuropsychology and clinical neuroscience of persistent post-concussive syndrome. J. Int. Neuropsychol. Soc. 14, 1–22 [DOI] [PubMed] [Google Scholar]
  • 9.Giza C.C., and Hovda D.A. (2001). The neurometabolic cascade of concussion. J. Athl. Train. 36, 228–235 [PMC free article] [PubMed] [Google Scholar]
  • 10.Talavage T.M., Nauman E.A., Breedlove E.L., Yoruk U., Dye A.E., Morigaki K.E., Feuer H., and Leverenz L.J. (2014). Functionally-detected cognitive impairment in high school football players without clinically-diagnosed concussion. J. Neurotrauma 31, 327–338 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Montenigro P.H., Bernick C., and Cantu R.C. (2015). Clinical features of repetitive traumatic brain injury and chronic traumatic encephalopathy. Brain Pathol. 25, 304–317 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Guskiewicz K.M., Weaver N.L., Padua D.A., and Garrett W.E. (2000). Epidemiology of concussion in collegiate and high school football players. Am. J. Sports Med. 28, 643–650 [DOI] [PubMed] [Google Scholar]
  • 13.Powell J.W., and Barber–Foss K.D. (1999). Traumatic brain injury in high school athletes. J.A.M.A. 282, 958–963 [DOI] [PubMed] [Google Scholar]
  • 14.Daneshvar D., Nowinski C., McKee A., and Cantu R. (2011). The epidemiology of sport-related concussion. Clin. Sports Med. 30, 1–17 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Guthrie R.M. (2015). Emerging data on the incidence of concussion in football practice at all levels of amateur play. Phys. Sportsmed. 43, 333–335 [DOI] [PubMed] [Google Scholar]
  • 16.Langburt W., Cohen B., Akhthar N., O'Neill K., and Lee J.-C. (2001). Incidence of concussion in high school football players of ohio and pennsylvania. J. Child Neurol. 16, 83–85 [DOI] [PubMed] [Google Scholar]
  • 17.Llewellyn T., Burdette G.T., Joyner A.B., and Buckley T.A. (2014). Concussion reporting rates at the conclusion of an intercollegiate athletic career. Clin. J. Sport Med. 24, 76–79 [DOI] [PubMed] [Google Scholar]
  • 18.Guskiewicz K.M., McCrea M., Marshall S.W., Cantu R.C., Randolph C., Barr W., Onate J.A., and Kelly J.P. (2003). Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA concussion study. J.A.M.A. 290, 2549–2555 [DOI] [PubMed] [Google Scholar]
  • 19.Collins M.W., Lovell M.R., Iverson G.L., Cantu R.C., Maroon J.C., and Field M. (2002). Cumulative effects of concussion in high school athletes. Neurosurgery 51, 1175–1181 [DOI] [PubMed] [Google Scholar]
  • 20.Iverson G.L., Gaetz M., Lovell M.R., and Collins M.W. (2004). Cumulative effects of concussion in amateur athletes. Brain Inj. 18, 433–443 [DOI] [PubMed] [Google Scholar]
  • 21.Guskiewicz K.M., Marshall S.W., Bailes J., McCrea M., Cantu R.C., Randolph C., and Jordan B.D. (2005). Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery 57, 719–726 [DOI] [PubMed] [Google Scholar]
  • 22.Guskiewicz K.M., Marshall S.W., Bailes J., McCrea M., Harding H.P., Matthews A., Mihalik J.R., and Cantu R.C. (2007). Recurrent concussion and risk of depression in retired professional football players. Med. Sci. Sports Exerc. 39, 903. [DOI] [PubMed] [Google Scholar]
  • 23.Singh R., Meier T.B., Kuplicki R., Savitz J., Mukai I., Cavanagh L., Allen T., Teague T.K., Nerio C., Polanski D., and Bellgowan P.S. (2014). Relationship of collegiate football experience and concussion with hippocampal volume and cognitive outcomes. J.A.M.A. 311, 1883–1888 [DOI] [PubMed] [Google Scholar]
  • 24.De Beaumont L., Lassonde M., Leclerc S., and Théoret H. (2007). Long‐term and cumulative effects of sports concussion on motor cortex inhibition. Neurosurgery 61, 329–337 [DOI] [PubMed] [Google Scholar]
  • 25.Seichepine D.R., Stamm J.M., Daneshvar D.H., Riley D.O., Baugh C.M., Gavett B.E., Tripodis Y., Martin B., Chaisson C., McKee A.C., Cantu R.C., Nowinski C.J., and Stern R.A. (2013). Profile of self-reported problems with executive functioning in college and professional football players. J. Neurotrauma 30, 1299–1304 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Breedlove E.L., Robinson M., Talavage T.M., Morigaki K.E., Yoruk U., O'Keefe K., King J., Leverenz L.J., Gilger J.W., and Nauman E.A. (2012). Biomechanical correlates of symptomatic and asymptomatic neurophysiological impairment in high school football. J. Biomech. 45, 1265–1272 [DOI] [PubMed] [Google Scholar]
  • 27.Abbas K., Shenk T.E., Poole V.N., Breedlove E.L., Leverenz L.J., Nauman E.A., Talavage T.M., and Robinson M.E. (2015). Alteration of default mode network in high school football athletes due to repetitive subconcussive mild traumatic brain injury: a resting-state functional magnetic resonance imaging study. Brain Connect. 5, 91–101 [DOI] [PubMed] [Google Scholar]
  • 28.Davenport E.M., Whitlow C.T., Urban J.E., Espeland M.A., Jung Y., Rosenbaum D.A., Gioia G., Powers A.K., Stitzel J.D., and Maldjian J.A. (2014). Abnormal white matter integrity related to head impact exposure in a season of high school varsity football. J. Neurotrauma 31, 1617–1624 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Montenigro P.H., Corp D.T., Stein T.D., Cantu R.C., and Stern R.A. (2015). Chronic traumatic encephalopathy: historical origins and current perspective. Annu. Rev. Clin. Psychol. 11, 309–330 [DOI] [PubMed] [Google Scholar]
  • 30.Stein T.D., Alvarez V.E., and McKee A.C. (2015). Concussion in chronic traumatic encephalopathy. Curr. Pain Headache Rep. 19, 1–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Bieniek K.F., Ross O.A., Cormier K.A., Walton R.L., Soto–Ortolaza A., Johnston A.E., DeSaro P., Boylan K.B., Graff–Radford N.R., Wszolek Z.K., Rademakers R., Boeve B.F., Mckee A.C., and Dickson D.W. (2015). Chronic traumatic encephalopathy pathology in a neurodegenerative disorders brain bank. Acta Neuropathol. 130, 877–889 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Stamm J.M., Bourlas A.P., Baugh C.M., Fritts N.G., Daneshvar D.H., Martin B.M., McClean M.D., Tripodis Y., and Stern R.A. (2015). Age of first exposure to football and later-life cognitive impairment in former nfl players. Neurology 84, 1114–1120 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Stamm J.M., Koerte I.K., Muehlmann M., Pasternak O., Bourlas A.P., Baugh C.M., Giwerc M.Y., Zhu A., Coleman M.J., Sylvain Bouix, Fritts N.G., Martin B.M., Chaisson C., Michael D. McClean Alexander P., Lin Robert C, Yorghos Tripodis C., Stern R.A., and Shenton M.E. (2015). Age at first exposure to football is associated with altered corpus callosum white matter microstructure in former professional football players. J. Neurotrauma 32, 1768–1776 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Jedliński J., Gatarski J., and Szymusik A. (1971). Encephalopathia pugilistica (punch drunkeness). Acta Med. Pol. 12, 443. [PubMed] [Google Scholar]
  • 35.Bernick C., and Banks S. (2013). What boxing tells us about repetitive head trauma and the brain. Alzheimers Res. Ther. 5, 1–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.McKee A.C., Stern R.A., Nowinski C.J., Stein T.D., Alvarez V.E., Daneshvar D.H., Lee H.S., Hall G., Wojtowicz S.M., Baugh C.M., Riley D.O., Kubilus C.A., Cormier K.A., Jacobs M.A., Martin B.R., Abraham C.R., Ikezu T., Reichard R.R., Wolozin B.L., Budson A.E., Goldstein L.E., Kowall N.W., and Cantu R.C. (2013). The spectrum of disease in chronic traumatic encephalopathy. Brain 136, 43–64 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Gysland S.M., Mihalik J.P., Register–Mihalik J.K., Trulock S.C., Shields E.W., and Guskiewicz K.M. (2012). The relationship between subconcussive impacts and concussion history on clinical measures of neurologic function in collegiate football players. Ann. Biomed. Eng. 40, 14–22 [DOI] [PubMed] [Google Scholar]
  • 38.Bernick C., Banks S., Phillips M., Lowe M., Shin W., Obuchowski N., Jones S., and Modic M. (2013). Professional fighters brain health study: rationale and methods. Am. J. Epidemiol. 178, 280–286 [DOI] [PubMed] [Google Scholar]
  • 39.Corrigan J.D., and Bogner J. (2007). Screening and identification of TBI. J. Head Trauma Rehabil. 22, 315–317 [DOI] [PubMed] [Google Scholar]
  • 40.Corrigan J., and Bogner J. (2007). Initial reliability and validity of the ohio state university TBI identification method. J. Head Trauma Rehabil. 22, 318–329 [DOI] [PubMed] [Google Scholar]
  • 41.Corrigan J.D., and Bogner J. (2015). Why self-report? Ohio Valley Center for Brain Injury Prevention and Rehabilitation. Available at: http://ohiovalley.org/informationeducation/screening/why/ (last accessed January15, 2016)
  • 42.Grinnon S.T., Miller K., Marler J.R., Lu Y., Stout A., Odenkirchen J., and Kunitz S. (2012). National institute of neurological disorders and stroke common data element project–approach and methods. Clin, Trials 9, 322–329 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Kerr Z.Y., Marshall S.W., and Guskiewicz K.M. (2012). Reliability of concussion history in former professional football players. Med. Sci. Sports Exerc. 44, 377–382 [DOI] [PubMed] [Google Scholar]
  • 44.Stern R.A., Gavett B.E., Baugh C., Nowinski C.J., Cantu R.C., and McKee A.C. (2011). Recurrent sports-related traumatic brain injury and tauopathy, in: Nutrition and Traumatic Brain Injury: Improving Acute and Subacute Health Outcomes in Military Personnel. Pillsbury L. Oria M., and Erdman J. (eds.). National Academies Press: Washington, DC, pps. 305–310 [PubMed] [Google Scholar]
  • 45.Baugh C.M., Stamm J.M., Riley D.O., Gavett B.E., Shenton M.E., Lin A., Nowinski C.J., Cantu R.C., McKee A.C., and Stern R.A. (2012). Chronic traumatic encephalopathy: neurodegeneration following repetitive concussive and subconcussive brain trauma. Brain Imaging Behav 6, 244–254 [DOI] [PubMed] [Google Scholar]
  • 46.Kerr Z.Y., Littleton A.C., Cox L.M., DeFreese J.D., Varangis E., Lynall R.C., Schmidt J.D., Marshall S.W., and Guskiewicz K.M. (2015). Estimating contact exposure in football using the head impact exposure estimate. J. Neurotrauma 32, 1083–1089 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Robbins C.A., Daneshvar D.H., Picano J.D., Gavett B.E., Baugh C.M., Riley D.O., Nowinski C.J., McKee A.C., Cantu R.C., and Stern R.A. (2014). Self-reported concussion history: impact of providing a definition of concussion. Open Access J. Sports Med. 5, 99–104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Riley D.O., Robbins C.A., Cantu R.C., and Stern R.A. (2015). Chronic traumatic encephalopathy: contributions from the boston university center for the study of traumatic encephalopathy. Brain Inj. 29, 154–163 [DOI] [PubMed] [Google Scholar]
  • 49.Bär K.-J., and Markser V.Z. (2013). Sport specificity of mental disorders: the issue of sport psychiatry. Eur. Arch. Psychiatry Clin. Neurosci. 263, 205–210 [DOI] [PubMed] [Google Scholar]
  • 50.Dompier T.P., Kerr Z.Y., Marshall S.W., Hainline B., Snook E.M., Hayden R., and Simon J.E. (2015). Incidence of concussion during practice and games in youth, high school, and collegiate american football players. J.A.M.A. Pediatr. 169, 659–665 [DOI] [PubMed] [Google Scholar]
  • 51.Grindel S.H., Lovell M.R., and Collins M.W. (2001). The assessment of sport-related concussion: The evidence behind neuropsychological testing and management. Clin. J. Sport Med. 11, 134–143 [DOI] [PubMed] [Google Scholar]
  • 52.Cantu R.C., and Mueller F.O. (2003). Catastrophic spine injuries in american football, 1977–2001. Neurosurgery 53, 358–363 [DOI] [PubMed] [Google Scholar]
  • 53.Harmon K.G., Drezner J.A., Gammons M., Guskiewicz K.M., Halstead M., Herring S.A., Kutcher J.S., Pana A., Putukian M., and Roberts W.O. (2013). American medical society for sports medicine position statement: concussion in sport. Br. J. Sports Med. 47, 15–26 [DOI] [PubMed] [Google Scholar]
  • 54.Cantu R.C. (2013). Role of diffusion tensor imaging MRI in detecting brain injury in asymptomatic contact athletes. World Neurosurg. 80, 792–793 [DOI] [PubMed] [Google Scholar]
  • 55.Mihalik J.P., Bell D.R., Marshall S.W., and Guskiewicz K.M. (2007). Measurement of head impacts in collegiate football players: an investigation of positional and event‐type differences. Neurosurgery 61, 1229–1235 [DOI] [PubMed] [Google Scholar]
  • 56.Daniel R.W., Rowson S., and Duma S.M. (2012). Head impact exposure in youth football. Ann. Biomed. Eng. 40, 976–981 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Crisco J.J., Fiore R., Beckwith J.G., Chu J.J., Brolinson P.G., Duma S., McAllister T.W., Duhaime A.-C., and Greenwald R.M. (2010). Frequency and location of head impact exposures in individual collegiate football players. J. Athl. Train 45, 549–559 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Baugh C.M., Kiernan P.T., Kroshus E., Daneshvar D.H., Montenigro P.H., McKee A.C., and Stern R. (2014). Frequency of head impact related outcomes by position in NCAA division I collegiate football players. J. Neurotrauma 32, 314–326 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Centers for Disease Control And Prevention (2013). Concussion in sports. Available at: http://www.cdc.gov/concussion/sports/ Accessed February28, 2014
  • 60.McCrory P., Meeuwisse W., Johnston K., Dvorak J., Aubry M., Molloy M., and Cantu R. (2009). Consensus statement on concussion in sport – the 3rd international conference on concussion in sport held in Zurich, November 2008. PM R 5, 406–420 [DOI] [PubMed] [Google Scholar]
  • 61.Broglio S.P., Eckner J.T., Martini D., Sosnoff J.J., Kutcher J.S., and Randolph C. (2011). Cumulative head impact burden in high school football. J. Neurotrauma 28, 2069–2078 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Crisco J.J., Wilcox B.J., Beckwith J.G., Chu J.J., Duhaime A.-C., Rowson S., Duma S.M., Maerlender A.C., McAllister T.W., and Greenwald R.M. (2011). Head impact exposure in collegiate football players. J. Biomech. 44, 2673–2678 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Ng T.P., Bussone W.R., and Duma S.M. (2006). The effect of gender and body size on linear accelerations of the head observed during daily activities. Biomed. Sci. Instrum. 42, 25–30 [PubMed] [Google Scholar]
  • 64.Duma S.M., Manoogian S.J., Bussone W.R., Brolinson P.G., Goforth M.W., Donnenwerth J.J., Greenwald R.M., Chu J.J., and Crisco J.J. (2005). Analysis of real-time head accelerations in collegiate football players. Clin. J. Sport Med. 15, 3–8 [DOI] [PubMed] [Google Scholar]
  • 65.Guskiewicz K.M., Mihalik J.P., Shankar V., Marshall S.W., Crowell D.H., Oliaro S.M., Ciocca M.F., and Hooker D.N. (2007). Measurement of head impacts in collegiate football players: relationship between head impact biomechanics and acute clinical outcome after concussion. Neurosurgery 61, 1244–1253 [DOI] [PubMed] [Google Scholar]
  • 66.Broglio S.P., Sosnoff J.J., Shin S., He X., Alcaraz C., and Zimmerman J. (2009). Head impacts during high school football: a biomechanical assessment. J. Athl. Train. 44, 342–349 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Tun P.A., and Lachman M.E. (2006). Telephone assessment of cognitive function in adulthood: the brief test of adult cognition by telephone. Age Ageing 35, 629–632 [DOI] [PubMed] [Google Scholar]
  • 68.Gioia G., Isquith P.K., Guy S.C., and Keutworthy L. (2000). Test review: Behavior rating inventory of executive function. Child Neuropsychol. 6, 235–238 [DOI] [PubMed] [Google Scholar]
  • 69.Radloff L.S. (1977). The CES-D scale a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1, 385–401 [Google Scholar]
  • 70.Marin R., Biedrzycki R., and Firinciogullari S. (1991). Reliability and validity of the apathy evaluation scale. Psychiatry Res. 38, 143–162 [DOI] [PubMed] [Google Scholar]
  • 71.Waid–Ebbs J.K., Wen P.-S., Heaton S.C., Donovan N.J., and Velozo C. (2012). The item level psychometrics of the behaviour rating inventory of executive function-adult (BRIEF-A) in a TBI sample. Brain Inj. 26, 1646–1657 [DOI] [PubMed] [Google Scholar]
  • 72.Gioia G.A., and Isquith P.K. (2004). Ecological assessment of executive function in traumatic brain injury. Dev. Neuropsychol. 25, 135–158 [DOI] [PubMed] [Google Scholar]
  • 73.Kant R., Duffy J., and Pivovarnik A. (1998). Prevalence of apathy following head injury. Brain Inj. 12, 87–92 [DOI] [PubMed] [Google Scholar]
  • 74.Lane–Brown A., and Tate R. (2009). Measuring apathy after traumatic brain injury: Psychometric properties of the apathy evaluation scale and the frontal systems behavior scale. Brain Inj. 23, 999–1007 [DOI] [PubMed] [Google Scholar]
  • 75.Andersson S., and Bergedalen A.-M. (2002). Cognitive correlates of apathy in traumatic brain injury. Neuropsychiatry Neuropsychol. Behav. Neurol. 15, 184–191 [PubMed] [Google Scholar]
  • 76.McCauley S.R., Pedroza C., Brown S.A., Boake C., Levin H.S., Goodman H.S., and Merritt S.G. (2006). Confirmatory factor structure of the center for epidemiologic studies-depression scale (CES-D) in mild-to-moderate traumatic brain injury. Brain Inj. 20, 519–527 [DOI] [PubMed] [Google Scholar]
  • 77.Appaneal R.N., Levine B.R., Perna F.M., and Roh J.L. (2009). Measuring postinjury depression among male and female competitive athletes. J. Sport Exerc. Psychol. 31, 60–76 [DOI] [PubMed] [Google Scholar]
  • 78.Scheibel R.S., Newsome M.R., Troyanskaya M., Steinberg J.L., Goldstein F.C., Mao H., and Levin H.S. (2009). Effects of severity of traumatic brain injury and brain reserve on cognitive-control related brain activation. J. Neurotrauma 26, 1447–1461 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Gavett B.E., Crane P.K., and Dams–O'Connor K. (2013). Bi-factor analyses of the brief test of adult cognition by telephone. NeuroRehabilitation 32, 253–265 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Radloff L.S., and Locke B.Z. (1986). The community mental health assessment survey and the Ces–D scale, in: Community Surveys of Psychiatric Disorders. Weissman M.M. Myers J.K. Ross C.E. (eds). Rutgers University Press, New Brunswick, NJ, pps. 177–188 [Google Scholar]
  • 81.Husaini B.A., Neff J.A., Harrington J.B., Hughes M.D., and Stone R.H. (1980). Depression in rural communities: validating the ces‐d scale. J. Community Psychol. 8, 20–27 [Google Scholar]
  • 82.Diane B. Howieson P., and Muriel D. Lezak P. (2008). The neuropsychological evaluation, in: The American Psychiatric Publishing Textbook of Neuropsychiatry and Behavioral Neurosciences. Yudofsky S.C., and Hales R.E. (eds.). American Psychiatric Publishing: Washington, DC, p. 223 [Google Scholar]
  • 83.Wilson R.S., and Bennett D.A. (2005). Assessment of cognitive decline in old age with brief tests amenable to telephone administration. Neuroepidemiology 25, 19–25 [DOI] [PubMed] [Google Scholar]
  • 84.Petersen R.C., and Morris J.C. (2005). Mild cognitive impairment as a clinical entity and treatment target. Arch. Neurol. 62, 1160–1163 [DOI] [PubMed] [Google Scholar]
  • 85.Carlin B.P., Gelfand A.E., and Smith A.F. (1992). Hierarchical bayesian analysis of changepoint problems. Appl Statist 41, 389–405 [Google Scholar]
  • 86.Heckman J.J. (1979). Sample selection bias as a specification error. Econometrica 47, 153–161 [Google Scholar]
  • 87.Leigh J.P., and Schembri M. (2004). Instrumental variables technique: cigarette price provided better estimate of effects of smoking on SF-12. J. Clin. Epidemiol. 57, 284–293 [DOI] [PubMed] [Google Scholar]
  • 88.Angrist J.D., Imbens G.W., and Rubin D.B. (1996). Identification of causal effects using instrumental variables. J Am Stat Assoc 91, 444–455 [Google Scholar]
  • 89.Winship C., and Morgan S.L. (1999). The estimation of causal effects from observational data. Annu. Rev. Sociol. 25, 659–706 [Google Scholar]
  • 90.Angrist J.D., and Keueger A.B. (1991). Does compulsory school attendance affect schooling and earnings? Q. J. Econ. 106, 979–1014 [Google Scholar]
  • 91.Wang P.S., Schneeweiss S., Avorn J., Fischer M.A., Mogun H., Solomon D.H., and Brookhart M.A. (2005). Risk of death in elderly users of conventional vs. atypical antipsychotic medications. N. Engl. J. Med. 353, 2335–2341 [DOI] [PubMed] [Google Scholar]
  • 92.Smith P.J., Potter G.G., McLaren M.E., and Blumenthal J.A. (2013). Impact of aerobic exercise on neurobehavioral outcomes. Ment. Health Phys. Act. 6, 139–153 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 93.Harrison P.A., and Narayan G. (2003). Differences in behavior, psychological factors, and environmental factors associated with participation in school sports and other activities in adolescence. J. Sch. Health 73, 113–120 [DOI] [PubMed] [Google Scholar]
  • 94.Fredricks J.A., and Eccles J.S. (2006). Is extracurricular participation associated with beneficial outcomes? Concurrent and longitudinal relations. Dev. Psychol. 42, 698. [DOI] [PubMed] [Google Scholar]
  • 95.Lipton M.L., Kim N., Zimmerman M.E., Kim M., Stewart W.F., Branch C.A., and Lipton R.B. (2013). Soccer heading is associated with white matter microstructural and cognitive abnormalities. Radiology 268, 850–857 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Stiller J.W., Yu S.S., Brenner L.A., Langenberg P., Scrofani P., Pannella P., Hsu E.B., Roberts D.W., Monsell R.M., Binks III S.W., Guzman A., and Postolache T.T. (2014). Sparring and neurological function in professional boxers. Front. Public Health 2, 69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 97.Bernick C., Banks S., Mahmoud S., Shin W., Jones S., Lowe M., Phillips M., and Modic M. (2012). The threshold effect of repeated head trauma on brain structure and cognition: the professional fighters brain health study. Neurology, AAN Annual Meeting Abstracts 79, e87–e91 [Google Scholar]
  • 98.Stern R.A., Daneshvar D.H., Baugh C.M., Seichepine D.R., Montenigro P.H., Riley D., Fritts N.G., Stamm J.M., Robbins C.A., McHale L., Simkin I., Stein T.D., Alvarez V.E., Goldstein L.E., Budson A.E., Kowall N.W., Nowinski C.J., Cantu R.C., and McKee A.C. (2013). Clinical presentation of chronic traumatic encephalopathy. Neurology 81, 1122–1129 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Stern R.A., Riley D.O., Daneshvar D.H., Nowinski C.J., Cantu R.C., and McKee A.C. (2011). Long-term consequences of repetitive brain trauma: chronic traumatic encephalopathy. PM R 3, S460–S467 [DOI] [PubMed] [Google Scholar]
  • 100.Hill A.B. (1965). The environment and disease: amssociation or causation? Proc. R. Soc. Med. 58, 295. [PMC free article] [PubMed] [Google Scholar]
  • 101.Pettygrove S. (2008). Dose-response relationship, in: Encyclopedia of Epidemiology. Boslaugh S., and McNutt L.-A. (eds.). SAGE Publications, Inc: Thousand Oaks, CA, pps. 283–285 [Google Scholar]
  • 102.Jordan B.D., Matser E., Zimmerman R.D., and Zazula T. (1996). Sparring and cognitive function in professional boxers. Phys. Sportsmed. 24, 87–98 [DOI] [PubMed] [Google Scholar]
  • 103.Bernick C., Banks S.J., Shin W., Obuchowski N., Butler S., Noback M., Phillips M., Lowe M., Jones S., and Modic M. (2015). Repeated head trauma is associated with smaller thalamic volumes and slower processing speed: the professional fighters' brain health study. Br. J. Sports Med. 49, 1007–1011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Goldstein L.E., Fisher A.M., Tagge C.A., Zhang X.L., Velisek L., Sullivan J.A., Upreti C., Kracht J.M., Ericsson M., Wojnarowicz M.W., Goletiani C.J., Maglakelidze G.M., Casey N., Moncaster J.A., Minaeva O., Moir R.D., Nowinski C.J., Stern R.A., Cantu R.C., Geiling J., Blusztajn J.K., Wolozin B.L., Ikezu T., Stein T.D., Budson A.E., Kowall N.W., Chargin D., Sharon A., Saman S., Hall G.F., Moss W.C., Cleveland R.O., Tanzi R.E., Stanton P.K., and McKee A.C. (2012). Chronic traumatic encephalopathy in blast-exposed military veterans and a blast neurotrauma mouse model. Sci. Transl. Med. 16, 1–16 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 105.Goldstein L.E., McKee A.C., and Stanton P.K. (2014). Considerations for animal models of blast-related traumatic brain injury and chronic traumatic encephalopathy. Alzheimers Res. Ther. 6, 1–10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 106.Robinson M.E., Shenk T.E., Breedlove E.L., Leverenz L.J., Nauman E.A., and Talavage T.M. (2015). The role of location of subconcussive head impacts in fMRI brain activation change. Dev. Neuropsychol. 40, 74–79 [DOI] [PubMed] [Google Scholar]
  • 107.Marchi N., Bazarian J.J., Puvenna V., Janigro M., Ghosh C., Zhong J., Zhu T., Blackman E., Stewart D., and Ellis J. (2013). Consequences of repeated blood–brain barrier disruption in football players. PLoS One 8, e56805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 108.Martland H.S. (1928). Punch drunk. J.A.M.A. 91, 1103–1107 [Google Scholar]
  • 109.Fidan E., Lewis J., Kline A.E., Garman R.H., Alexander H., Cheng J.P., Bondi C.O., Clark R., Dezfulian C., and Kochanek P.M. (2015). Repetitive mild traumatic brain injury in the developing brain: effects on long-term functional outcome and neuropathology. J. Neurotrauma 32, 1–11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 110.Angoa–Perez M., Kane M.J., Briggs D.I., Herrera–Mundo N., Viano D.C., and Kuhn D.M. (2014). Animal models of sports-related head injury: bridging the gap between preclinical research and clinical reality. J Neurochem. 129, 916–931 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 111.Rassen J.A., Schneeweiss S., Glynn R.J., Mittleman M.A., and Brookhart M.A. (2009). Instrumental variable analysis for estimation of treatment effects with dichotomous outcomes. Am. J. Epidemiol. 169, 273–284 [DOI] [PubMed] [Google Scholar]
  • 112.Schneeweiss S., Solomon D.H., Wang P.S., Rassen J., and Brookhart M.A. (2006). Simultaneous assessment of short‐term gastrointestinal benefits and cardiovascular risks of selective cyclooxygenase 2 inhibitors and nonselective nonsteroidal antiinflammatory drugs: an instrumental variable analysis. Arthritis Rheum. 54, 3390–3398 [DOI] [PubMed] [Google Scholar]
  • 113.Katz M.H. (2010). Evaluating Clinical and Public Health Interventions: A Practical Guide to Study Design and Statistics. Cambridge University Press: San Francisco [Google Scholar]
  • 114.Redding R. (2015). 2015 NCAA® Football Rules and Interpretations. The National Collegiate Athletic Association: Indianapolis [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental data
Supp_Table1.pdf (25.1KB, pdf)

Articles from Journal of Neurotrauma are provided here courtesy of Mary Ann Liebert, Inc.

RESOURCES