Refinement of a Preliminary Case Definition for Use in Traumatic Brain Injury Surveillance

Abstract

Objective:

Current methods used to measure incidence of traumatic brain injury (TBI) underestimate its true public health burden. The use of self-report surveys may be an approach to improve these estimates. An important step in public health surveillance is to define a public health problem using a case definition. The purpose of this article is to outline the process that the Centers for Disease Control and Prevention undertook to refine a TBI case definition to be used in surveillance using a self-report survey.

Setting:

Survey.

Participants:

A total of 10 030 adults participated via a random digit-dial telephone survey from September 2018 to September 2019.

Main Measures:

Respondents were asked whether they had sustained a hit to the head in the preceding 12 months and whether they experienced a series of 12 signs and symptoms as a result of this injury.

Design:

Head injuries with 1 or more signs/symptoms reported were initially categorized into a 3-tiered TBI case definition (probable TBI, possible TBI, and delayed possible TBI), corresponding to the level of certainty that a TBI occurred. Placement in a tier was compared with a range of severity measures (whether medical evaluation was sought, time to symptom resolution, self-rated social and work functioning); case definition tiers were then modified in a stepwise fashion to maximize differences in severity between tiers.

Results:

There were statistically significant differences in the severity measure between cases in the probable and possible TBI tiers but not between other tiers. Timing of symptom onset did not meaningfully differentiate between cases on severity measures; therefore, the delayed possible tier was eliminated, resulting in 2 tiers: probable and possible TBI.

Conclusion:

The 2-tiered TBI case definition that was derived from this analysis can be used in future surveillance efforts to differentiate cases by certainty and from noncases for the purpose of reporting TBI prevalence and incidence estimates. The refined case definition can help researchers increase the confidence they have in reporting survey respondents’ self-reported TBIs as well as provide them with the flexibility to report an expansive (probable + possible TBI) or more conservative (probable TBI only) estimate of TBI prevalence.

Millions of Americans sustain a traumatic brain injury (TBI) annually.^1,2 However, current national estimates of nonfatal TBI are largely based on hospital administrative data sets, which underestimate the burden of TBI in the United States.³ These estimates do not account for TBIs treated outside of hospital settings (eg, those that are treated in primary care offices, concussion clinics, urgent care facilities) and those that are untreated or those that go undiagnosed.^4–8 An alternative approach that could improve estimates of TBI in the United States is to use self-reported survey data, as discussed in the companion article in this journal (Note: see “Rationale for the Development of a Traumatic Brain Injury Case Definition for the pilot National Concussion Surveillance System,” published in JHTR separately). However, given the complexities of assessing TBI, there is a need to establish a case definition that explicitly describes how self-report data are evaluated to produce a TBI determination and calculate TBI incidence and prevalence estimates.

Current self-reported data collections of TBI have limitations, in part due to constraints on the amount of space for TBI-related questions. Examples of these include questions that require the respondent to report only about TBIs for which they received a diagnosis,⁹ include only TBIs that resulted in a loss of consciousness (LOC),¹⁰ or ask about TBIs using single question that includes a list of multiple symptoms^11,12 that frequently do not include the full range of symptoms that may be experienced following a TBI. This has likely led to underreporting, particularly in the case of mild TBI, if the respondent did not seek care for the injury or their symptoms were not among those assessed. On the other hand, more generalized symptoms such as headaches, nausea, or dizziness may be falsely attributed to a TBI, resulting in false-positive cases.

There have been recent efforts by the Centers for Disease Control and Prevention (CDC) and others to represent the level of confidence in making a TBI determination by the amount of evidence (eg, signs/symptoms documented in the medical record, positive imaging findings) supporting such a diagnosis.^13–15 As part of the CDC’s effort to pilot the feasibility in implementing a National Concussion Surveillance System (NCSS), a tiered TBI case definition was created (see Figure 1).

Figure 1.

Original proposed TBI case definition. TBI indicates traumatic brain injury.

OBJECTIVE

While the original tiered TBI case definition developed for the NCSS pilot (Note: see “Rationale for the Development of a Traumatic Brain Injury Case Definition for the Pilot National Concussion Surveillance System,” published in JHTR separately) was based on expert input informed by clinical experience and knowledge of the scientific literature, no previous research had tested a case definition for the purpose of producing surveillance estimates using self-report survey data. It is of critical public health importance for researchers and the public to have confidence in the estimates produced from surveys—that the respondents who reported experiencing a TBI did indeed experience a TBI. Therefore, we sought to determine whether this version of the TBI case definition could be further refined to better capture self-reported TBI and differentiate the levels of certainty (eg, possible, probable) that a TBI occurred. The purpose of this article is to summarize the process of evaluating and refining the TBI case definition. Empirically refining a TBI case definition is a key step in differentiating TBI cases from noncases when analyzing self-report survey data and accurately determining how many adults and children experience a TBI each year in the United States. Use of a clearly specified, transparent case definition is critical for the reliability of the calculated estimates.

We hypothesized that TBIs classified as tier 3 (probable TBI) would have the highest level of medical evaluation, longest time to symptom resolution, and highest self-reported impact on social and work functioning. These 4 indicators of severity theoretically correspond to greater certainty that a TBI occurred. We predicted that tier 2 (possible TBI) would measure lower on the severity measures than tier 3; tier 1 (delayed possible) TBIs would measure lower on the severity measures than tier 2; and head injuries without symptoms (noncases) would measure lowest on the severity measures.

METHODS

Survey

In collaboration with academic and federal partners with expertise in TBI and public health surveillance, the CDC developed a survey that was piloted as a random-digit-dial telephone survey that took place over the course of a full year (September 2018 to September 2019). NCSS data collection utilized computer-assisted telephone interviewing and followed a protocol designed to minimize bias (eg, calling people both during the day and in the evenings) while maximizing completeness (eg, specific call back protocols designed to reach every eligible respondent). The NCSS sampling frame included the noninstitutionalized population aged 18 years and older residing in the 50 states and the District of Columbia. Eligible respondents were US adults who resided in a household with a landline telephone (ie, landline sample) or possessed a cell phone (ie, cell phone sample). The combined response rate for survey participants was 8.4% for completed surveys. Additional details about the survey methodology can be found in a previous publication.³ Study procedures were approved by the institutional review board of ICF International, Inc, and were consistent with ethical guidelines for human subjects research.

Measures

TBI sustained in the preceding 12 months

Respondents were initially asked 2 questions to determine whether they had sustained a head injury in the past 12 months (see Supplemental Digital Content Appendix, available at: http://links.lww.com/JHTR/A734):

1. In the last year, that is since (insert date 1 year before the interview date), were you examined in a doctor’s office, clinic, hospital, or elsewhere because of a head injury?

2. In the last year, that is since (insert date 1 year before the interview date), did you experience any other injuries to your head that you did not see a doctor about?

Respondents were then asked to report the number of head injuries they experienced in the past month; however, contextual information (eg, the indicators of severity) was assessed only for the most recent injury reported. This was done to both decrease the survey length for respondents and because it is more likely that respondents would be able to recall detailed information about the injuries that had occurred in the more recent past.

Symptoms

If the respondent answered “yes” to either of the aforementioned questions, they were then asked a series of 12 yes or no questions regarding signs or symptoms that may have occurred because of the head injury, and for certain questions, whether the symptoms were experienced “immediately or minutes after” the head injury or “later on.” Signs and symptoms included being dazed, confused, or having trouble thinking straight; difficulty remembering what happened just before or after injury; LOC; nausea or vomiting; headache; dizziness, clumsiness, or balance problems; blurred or double vision; trouble concentrating; difficulty learning or remembering new things; sensitivity to light or noise; change in mood or temperament; and changes in sleep or being more tired than usual.

Indicators of severity

Four indicators of TBI severity were assessed in relation to the most recent head injury reported. They included the following: whether medical evaluation was sought (yes/no), time to symptom resolution (number of days), and self-reported impact on social and work functioning (“To what extent did the head injury interfere with your normal social activities with family, friends, neighbors or groups?” and “To what extent did the head injury interfere with your normal work [including work outside the home and housework]?”). The mean was calculated on the basis of a Likert scale with response options “not at all,” “slightly,” “moderately,” “quite a bit,” or “extremely”.

Given that certainty and severity vary together,¹⁶ construct validity was tested using the hypothesis that greater severity corresponds to greater certainty that a TBI occurred. TBI outcomes that serve as indicators of severity include time to symptom resolution or persistent symptoms,¹⁷ receipt of post-TBI examination,¹⁸ and impact to self-reported social and work functioning subsequent to the head injury.^19,20

Analytic steps

Survey weights were computed to correct for disproportionate sampling probabilities introduced by the sampling design, including the dual-user overlap of landlines and cell phones, and to correct for differential nonresponse by calibrating the demographic characteristics observed in the sample to match the population on sex, age, race/ethnicity, educational attainment, marital status, and home ownership. Estimates are presented as unweighted counts and weighted percentages or means. The pilot NCSS survey, from which this study’s data is drawn, was designed to support TBI case ascertainment and measure development. Estimates were weighted with the intent of comparing TBI prevalence among subpopulations derived from different databases. Weighted estimates from this pilot work were not specifically intended to produce nationally representative estimates of TBI. All analyses were performed in SAS 9.4 (SAS Institute, Cary, North Carolina) survey procedures, which calculate variances that account for the complex survey design using Taylor Series Linearization. Although there is no one-size-fits-all approach for conducting a validation study, this study followed suggested steps to validation within the constraints of the study parameters.^21,22

Refinement phase 1: examine original case definition tiers in relation to indicators of severity

Descriptive statistics are reported for placement of a head injury in 1 of the 3 tiers, based on the signs and symptoms reported, and the 4 indicators of severity. A series of t tests were conducted post hoc to test for pairwise differences in the means (continuous severity indicators) and the column percentages at each level of the categorical severity indicators. Time to symptom resolution was examined in 2 ways: percentage of head injuries in a tier experiencing 1 day of symptoms (to indicate lower severity) and percentage of head injuries still experiencing symptoms or having had 8+ days of symptoms (to indicate greater severity).^* A series of t tests and χ² tests were conducted post hoc to test for pairwise differences in the means for continuous severity indicators and distributions for categorical severity indicators, respectively.

Refinement phase 2: bivariate association of individual signs and symptoms with indicators of severity

To determine whether and how individual TBI signs and symptoms were statistically associated with the 4 indicators of TBI severity (sought medical evaluation, time to symptom resolution, social and work functioning), a series of logistic (categorical severity indicators) or linear regression (continuous severity indicators) models examined each sign or symptom as the independent variable and each indicator of severity as the outcome of interest.

Refinement phase 3: iterative process of modifying the 2-tiered case definition

Based on the strength of bivariate associations examined in phase 2, it was determined that a 2-tiered case definition was optimal (see the “Results” section later). After this decision was made, a series of modifications, each of which employed a 2-tiered case definition, were created in a stepwise fashion (see Supplemental Digital Content Appendix 1, available at: http://links.lww.com/JHTR/A735). The first step included testing modifications that included only 1 single change (eg, moving a symptom up or down). After these first modifications were applied, the modifications that demonstrated improved associations based on the criteria from steps 1 to 3 below were then further modified (eg, if a modification that moved 1 symptom up a tier showed improved associations with the indicators of severity, this new modification was further adapted to further improve the associations). The modifications included moving individual symptoms up (ie, promoting a symptom from the possible to the probable tier) or down (ie, demoting a symptom from the probable to the possible tier) a tier and increasing or decreasing the symptom counts (ie, number of symptoms reported) necessary to move up a tier. To empirically determine whether any of the case definition modifications performed better than the original case definition, the following steps were undertaken:

1. For each modification, effect sizes (tetrachoric or polychoric correlations where appropriate) were calculated examining the association between placement of head injuries into a particular tier and indicator of severity variables.

2. Effect size differences were tested using the method from Meng and colleagues.²³

   1. The differences were converted using the Fisher Z-transformation before testing and then back-transformed to correlations to aid interpretation.

   2. The 95% confidence intervals (CI) that did not contain zero were considered statistically significant from the original definition.

3. With a large sample size, there is a risk of detecting small differences that may not be meaningful. Therefore, significant effect size differences were classified as meaningfully or not meaningfully significant. For example, given Cohen’s convention for describing small effects sizes,²⁴ a small effect of r value of less than 0.14 would explain less than 2% of the variability in either the placement of head injury or the indicator of severity. Therefore, a statistical difference in the effect size of 0.15 or more in the expected direction (eg, higher tiers demonstrated worse severity) was chosen as a cut point for practical significance.²⁴

4. Changes in the definition were retained if they improved the effect size by 0.15 or more for 2 out of the 4 indicators of severity, and if the difference was in a direction that signified an improved association compared with the original definition.

5. For those modified case definitions that were retained as potential candidates, the next step was to examine differences using pairwise t tests or χ² tests between the modified definition and the original (ie, were the tiers better differentiated from each other [larger separation either in the percentage or mean between tiers] in the modified definition). Based on the aforementioned criteria, an optimal modified TBI case definition was selected.

For modified TBI case definitions that were retained as potential candidates, steps 2 and 3 were repeated to determine whether a particular modified TBI case definition was better than other potential candidates. If several case definitions performed similarly, weight was given to those that more closely resembled the original case definition as the original was based on expert and theoretical consensus.

Refinement phase 4: consultation with TBI subject matter experts

A select panel of 3 TBI subject matter experts (SMEs) who were consulted during the development of the initial case definition (see “Rationale for the Development of a Traumatic Brain Injury Case Definition for the pilot National Concussion Surveillance System,” submitted to JHTR separately) was consulted again to provide feedback on the revised TBI case definition. The experts represent the fields of rehabilitation psychology, clinical child psychology, and pediatrics/sports medicine. Individual interviews were conducted with each expert. They were presented with the results of the analysis and the proposed modifications to the case definition. This was not an official consensus-building process; instead, we asked them to provide feedback on the analysis and the face validity of the new case definition and their perception of how the modified case definition would be received in the larger scientific community.

A summary graphic detailing the analytic refinement process of our theoretical case definition is included in Supplemental Digital Content Appendix 2, available at: http://links.lww.com/JHTR/A736.

RESULTS

Sample

The current study includes data on a sample of 10130 adults; 1364 adult respondents (13.5%) reported sustaining at least 1 head injury in 12 months prior to their interview. The sample of respondents reporting 1 or more injuries was 52.5% male and 47.3% female (data not shown).^† About 19.7% of the sample were aged 18 to 24 years, 39.7% were aged 25 to 44 years, 28.9% were aged 45 to 64 years, 6.6% were aged 65 to 74 years, and 4.1% were 75 years of age or older. About two-thirds (65.0%) of the sample reported being non-Hispanic White; 9.0% were non-Hispanic Black, 18.6% were Hispanic, 3.2% were non-Hispanic Asian or Pacific Islander, and 2.9% were non-Hispanic other. About a quarter (25.2%) of the injured sample had a high school diploma while 22.7% had a bachelor’s degree or higher. More than one-third (38.7%) of the sample was married at the time of the interview and 30.8% were never married. As noted, for respondents who reported experiencing more than 1 head injury in the past 12 months, only their most recent head injury was included in most analyses (exceptions are noted later). Among respondents who reported any head injuries in the past year (n = 1152), 723 had just 1 head injury (63%) and 429 had multiple head injuries (37%). The distribution of respondents with multiple head injuries is as follows: 1 head injury = 723 (63%), 2 head injuries = 221 (19%), and 3 or more head injuries = 208 (18%). Symptom frequencies among all reported head injuries are presented in Table 1. The most commonly reported signs or symptoms among persons with a head injury were immediate headache (64.7%) and being dazed, confused, or trouble thinking straight (58.8%), while the least commonly reported signs or symptoms were delayed blurred or double vision (7.2%) and delayed nausea or vomiting (8.8%).

Table 1.

Frequency of signs and symptoms among all reported head injuries, injured people, National Concussion Surveillance System Pilot^a

	All injuries, n = 1691		Probable TBI (tier 3), n = 1072		Possible TBI (tier 2), n = 490		Delayed possible TBI (tier 1), n = 129
	Unweighted, n	Weightedb, %	Unweighted, n	Weightedb, %	Unweighted, n	Weightedb, %	Unweighted, n	Weightedb, %
Probable TBI signs (tier 3)
Dazed, confused, or trouble thinking straight	948	58.8	948	89.1	...	...	...	...
Difficulty remembering what happened just before/after injury	363	24.2	363	36.6	...	...	...	...
Loss of consciousness	327	22.6	327	34.3	...	...	...	...
Three tier 2 symptoms	366	22.4	366	34.0	...	...	...	...
Possible TBI symptoms (tier 2)
Immediate nausea or vomiting	234	14.5	199	18.9	35	7.3	...	...
Immediate headache	1077	64.7	671	63.5	406	82.8	...	...
Immediate dizziness, clumsiness, or balance problems	513	31.7	447	41.9	66	14.9	...	...
Immediate blurred or double vision	338	21.5	308	30.2	30	5.8	...	...
Immediate trouble concentrating	418	25.9	385	36.9	33	5.6	...	...
Three tier 1 symptoms	538	35.5	470	48.3	68	13.1	...	...
Delayed possible TBI symptoms (tier 1)
Difficulty learning or remembering new things	245	16.5	226	23.2	17	3.6	2	3.9
Sensitivity to light or noise	640	39.6	523	50.9	103	19.7	14	9.4
Change in mood or temperament	665	43.3	508	52.6	115	23.9	42	30.9
Changes in sleep or more tired than usual	659	41.1	537	52.8	95	18.9	27	17.3
Delayed nausea or vomiting	134	8.8	116	12.0	15	2.7	3	3.1
Delayed headache	356	22.3	237	23.8	52	11.6	67	51.9
Delayed dizziness, clumsiness, or balance problems	242	15.1	191	19.2	41	7.5	10	5.8
Delayed blurred or double vision	106	7.2	96	10.2	9	1.2	1	1.0
Delayed trouble concentrating	204	13.0	170	17.1	31	5.8	3	1.2

Abbreviation: TBI, traumatic brain injury.

^aThis includes all head Injuries in which a respondent reported 1 or more signs or symptoms. This Includes multiple head Injuries If a respondent Included more than 1 in a 12-month period.

^bEstlmates were weighted to correct for disproportionate sampling probabilities Introduced by the sampling design and to correct for differences in the demographic characteristics of the sample versus the population. Weighted estimates from this pilot work were not specifically Intended to produce nationally representative estimates of TBI.

Refinement phase 1: examine original case definition tiers in relation to indicators of severity

There were statistically significant differences between tier 3 (probable TBI) and tier 2 (possible TBI) for all of the indicators of severity (see Table 2). For example, 46.5% of adults with a probable TBI indicated that they were still experiencing symptoms, or they had 8+ days of symptoms, compared with 18.9% of adults with a possible TBI. However, there were few statistically significant differences between possible TBI and delayed possible TBI and between delayed possible TBIs and noncases (ie, those who reported a head injury with no symptoms) for all measures of severity. For example, while noncase injuries were significantly different from probable TBI on all 4 severity measures, noncase injuries were not significantly different from delayed possible TBI cases on any severity indicator.

TABLE 2.

TBI indicators of severity by TBI tier.; most recent head injury, National Concussion Surveillance System Pilot^a

	Probable TBI (tier 3), n = 716			Possible TBI (tier 2), n = 343			Delayed Possible TBI (tier 1), n = 93			Noncases (tier 0), n = 212			Statistically significant differences between tiers
	Count	Weightedb, % or mean	95% Cl	Count	Weightedb, % or mean	95% Cl	Count	Weightedb, % or mean	95% Cl	Count	Weightedb, % or mean	95% Cl
Evaluated for TBI Yes	319	45.4%	40.8%-50.1%	74	20.9%	15.5%-26.2%	21	23.8%	12.6%-35.1%	32	14.5%	8.6%-20.4%	c, d, e
Time to symptom resolution 1 day of symptoms	133	15.8%	12.7%-18.8%	141	41.5%	34.9%-48.2%	41	42.0%	29.9%-54.2%	f	f	f	c, d, e
Still having symptom OR had symptoms for 8+ days	338	46.5%	41.9%-51.1%	72	18.9%	14.0%-23.9%	14	15.8%	5.8%-25.8%	f	f	f	c, d, e
Extent of Impact: social functioning (mean)	657	2.1	2.0–2.2	327	1.3	1.2–1.4	85	1.1	1.0–1.2	207	1.0	1.0–1.1	c, d, e, g, h
Extent of Impact: work functioning (mean)	659	2.2	2.1–2.3	329	1.4	1.3–1.5	83	1.2	1.1–1.3	207	1.1	1.0–1.1	c, d, e, g, h

Abbreviations: Cl, confidence interval; TBI, traumatic brain Injury.

^aPalrwlse differences between groups were tested using a univariate f statistic for continuous severity Indicators and χ² tests for categorical severity Indicators.

^bEstlmates were weighted to correct for disproportionate sampling probabilities Introduced by the sampling design and to correct for differences in the demographic characteristics of the sample versus the population. Weighted estimates from this pilot work were not specifically Intended to produce nationally representative estimates of TBI.

^cStatistically significant difference (P < .05) between tiers 3 and 2.

^dStatistically significant difference (P < .05) between tiers 3 and 1.

^eStatlstlcally significant difference (P < .05) between tiers 3 and 0.

^fNot applicable. No analysis was conducted because no data were collected for these cases.

^gStatistical ly significant difference (P < .05) between tiers 2 and 1.

^hStatistically significant difference (P < .05) between tiers 2 and 0.

Refinement phase 2: association of individual symptoms with measures of TBI severity

Contrary to the hypothesis that experiencing a sign or symptom immediately following a head injury would provide greater certainty that the individual had a TBI, the data were not supportive of this hypothesis (see Table 3). Specifically, the strength of the association between delayed symptoms and indicators of severity was generally stronger (eg, larger odds ratio or point estimate for all indicators of severity except 1 day of symptoms; for 1 day of symptoms there was a smaller odds ratio and point estimate in the expected direction) than that of the associations between symptoms experienced immediately after a head injury. For example, those respondents who reported experiencing delayed blurred or double vision were more likely to still be experiencing symptoms or to have had symptoms for 8 or more days (adjusted OR [aOR] = 5.20; 95% CI, 2.7–10.1). While those who had immediate blurred or double vision after head injury were also more likely to still be experiencing symptoms or to have had symptoms for 8 or more days (AOR = 2.16; 95% CI, 1.5–3.2), the strength of the association was weaker than it was for delayed blurred or double vision (AOR = 5.20; 95% CI, 2.7–10.1). Based on this generally consistent pattern of results, the distinction between symptom onset was removed (eg, immediate headache and delayed headache were combined to just be “headache”) and bivariate associations between individual signs and symptoms were reanalyzed (see Table 4).

Table 3.

Bivariate association of individual signs and symptoms for a 3-tiered TBI case definition with indicators of severity, most recent injury, National Concussion Surveillance System Pilot^a

	Evaluation		Time to symptom resolution
	Yes		1 day of symptoms		Still having symptoms or had symptoms for 8+ days		Extent of impact: social functioning (continuous variable)				Extent of impact: Work functioning (continuous variable)
	Odds ratio	95% confidence interval	Odds ratio	95% confidence interval	Odds ratio	95% confidence interval	R 2	Estimate	t value	P	R 2	Estimate	t value	P
TBI sign or symptom
Probable TBI (tier 3)
Difficulty remembering what happened just before/after Injury	4.78	3.3–70	0.12	0.1–0.2	3.95	27–5.9	0.16	1.2	8.7	<.0001	0.12	1.1	7.5	<.0001
Dazed, confused, or trouble thinking straight	2.82	2.1–3.8	0.35	0.3–0.5	2.99	2.2–4.1	0.13	0.8	10.8	<.0001	0.12	0.8	10.2	<.0001
Loss of consciousness	6.10	4.1–9.2	0.15	0.1–0.3	2.66	1.8–4.0	0.11	1.0	7.2	<.0001	0.10	1.0	6.9	<.0001
Three tier 2 symptoms	1.87	1.3–2.7	0.47	0.3–0.8	2.30	1.6–3.3	0.08	0.9	6.4	<.0001	0.08	0.9	6.4	<.0001
Possible TBI (tier 2)
Immediate nausea or vomiting	1.85	1.2–2.9	0.36	0.2–0.7	2.42	1.5–3.9	0.05	0.8	4.6	<.0001	0.04	0.8	4.4	<.0001
Immediate headache	0.93	0.7–1.2	1.39	1.0–1.9	0.87	0.6–1.2	0.01	0.3	3.3	.0011	0.02	0.3	3.6	.0004
Immediate dizziness, clumsiness, or balance problems	2.01	1.5–2.8	0.64	0.4–0.9	1.91	1.4–2.6	0.07	0.7	6.4	<.0001	0.05	0.6	5.7	<.0001
Immediate blurred or double vision	1.84	1.2–2.7	0.53	0.3–0.9	2.16	1.5–3.2	0.05	0.7	5.1	<.0001	0.03	0.6	4.6	<.0001
Immediate trouble concentrating	1.95	1.4,27	0.57	0.4–0.9	2.10	1.5–3.0	0.09	0.8	7.2	<.0001	0.09	0.9	7.3	<.0001
Three tier 1 symptoms	4.62	3.3–6.4	0.06	0.03–0.1	6.06	4.3–8.6	0.34	1.5	14.2	<.0001	0.27	1.4	12.6	<.0001
Delayed possible TBI (tier 1)
Difficulty learning or remembering new things	4.66	2.9–74	0.01	0.002–0.1	9.73	5.5–17.2	0.26	1.8	12.2	<.0001	0.21	1.7	10.4	<.0001
Sensitivity to light or noise	2.64	1.9–3.6	0.28	0.2–0.4	3.21	2.3–4.4	0.20	1.1	11.5	<.0001	0.18	1.1	10.8	<.0001
Change In mood or temperament	3.00	2.2–4.1	0.33	0.2–0.5	3.57	2.6–4.9	0.22	1.1	11.9	<.0001	0.15	1.0	9.6	<.0001
Changes In sleep or more tired than usual	3.58	2.6–4.9	0.11	0.1–0.2	5.32	3.8–74	0.24	1.2	12.6	<.0001	0.22	1.2	12.2	<.0001
Delayed nausea or vomiting	3.63	2.1–6.2	0.08	0.03–0.2	1.94	1.1–3.3	0.05	0.9	4.4	<.0001	0.04	0.9	4.5	<.0001
Delayed headache	1.95	1.4–2.8	0.60	0.4–0.9	1.28	0.9–1.8	0.01	0.2	2.3	.0214	0.01	0.2	2.1	.039
Delayed dizziness, clumsiness, or balance problems	3.68	2.4–5.5	0.10	0.04–0.2	3.36	2.2–5.1	0.09	1.0	7.0	<.0001	0.07	0.9	6.1	<.0001
Delayed blurred or double vision	3.97	2.1–76	0.04	0.01–0.2	5.20	2.7–10.1	0.05	1.1	5.1	<.0001	0.06	1.3	5.1	<.0001
Delayed trouble concentrating	3.24	2.1–5.1	0.12	0.05–0.3	4.39	2.7–72	0.12	1.2	7.5	<.0001	0.08	1.1	6.0	<.0001

Abbreviation: TBI, traumatic brain injury.

^aValues in boldface are those that had an odds ratio (OR) of more than 4.72 (when the OR is >1), or OR of less than 0.21 (when the OR Is <1), or an R² value of 0.25 and more. These values were chosen on the basis of the cutoff for a strong effect size (refer to Table 1 for P₀ = .05 in Chen et al²⁵ for odds ratios and Cohen criteria for R² values).

Table 4.

Bivariate association of individual signs and symptoms for a 2-tiered TBI case definition with indicators of severity, most recent injury. National Concussion Surveillance System Pilot^a

	Evaluation		Time to symptom resolutionb
	Yes		1 day of symptoms		Still having symptom or had symptoms for 8+ days		Extent of impact: social functioning (continuous variable)				Extent of impact: work functioning (continuous variable)
	Odds ratio	95% confidence interval	Odds ratio	95% confidence interval	Odds ratio	95% confidence interval	R 2	Estimate	t value	P	R 2	Estimate	t value	P
TBI sign or symptom
Probable TBI
Difficulty remembering what happened just before/after Injury	4.78	3.3–70	0.12	0.1–0.2	3.95	2.7–5.9	0.16	1.2	8.7	<.0001	0.12	1.1	7.5	<.0001
Dazed, confused, or trouble thinking straight	2.82	2.1–3.8	0.35	0.3–0.5	2.99	2.2–4.1	0.13	0.8	10.8	<.0001	0.12	0.8	10.2	<.0001
Loss of consciousness	6.10	4.1–9.2	0.15	0.1–0.3	2.66	1.8–4.0	0.11	1.0	7.2	<.0001	0.10	1.0	6.9	<.0001
Three tier 2 symptoms	4.12	3.0–5.6	0.20	0.1–0.3	4.78	3.4–6.8	0.26	1.1	15.6	<.0001	0.20	1.1	13.1	<.0001
Possible TBI
Nausea or vomiting	2.76	1.9–4.0	0.21	0.1–0.4	2.49	1.7–3.6	0.11	1.0	7.0	<.0001	0.10	0.9	6.8	<.0001
Headache	1.79	1.3–2.5	0.89	0.6–1.4	1.13	0.8–1.7	0.04	0.5	8.0	<.0001	0.05	0.6	8.2	<.0001
Dizziness, clumsiness, or balance problems	3.62	2.7–4.9	0.31	0.2–0.4	3.58	2.6–4.9	0.19	1.0	11.8	<.0001	0.15	1.0	10.2	<.0001
Blurred or double vision	2.59	1.8–3.7	0.34	0.2–0.5	3.28	2.3–47	0.10	0.9	7.6	<.0001	0.09	0.9	7.2	<.0001
Trouble concentrating	3.01	2.2–4.1	0.31	0.2–0.5	3.96	2.9–5.5	0.25	1.2	12.6	<.0001	0.21	1.2	11.5	<.0001
Difficulty learning or remembering new things	4.66	2.9–74	0.01	0.002–0.1	9.73	5.5–17.2	0.26	1.8	12.2	<.0001	0.21	1.7	10.4	<.0001
Sensitivity to light or noise	2.64	1.9–3.6	0.28	0.2–0.4	3.21	2.3–4.4	0.20	1.1	11.5	<.0001	0.18	1.1	10.8	<.0001
Change In mood or temperament	3.00	2.2–4.1	0.33	0.2–0.5	3.57	2.6–4.9	0.22	1.1	11.9	<.0001	0.15	1.0	9.6	<.0001
Changes In sleep or more tired than usual	3.58	2.6–4.9	0.11	0.1–0.2	5.32	3.8–74	0.24	1.2	12.6	<.0001	0.22	1.2	12.2	<.0001

Abbreviations TBI, traumatic brain injury.

^aValues in boldface are those that had an odds ratio (OR) of more than 4.72 (when the OR is >1) or OR of less than 0.21 (when the OR Is <1) or an R² value of 0.25 and greater. These values were chosen on the basis of the cutoff for a strong effect size (refer to Table 1 for P₀ = 0.05 in the study by Chen et al²⁵ for ORs and Cohen criteria for R² values).

^bRespondents were asked to report how long It took for all of their head injury-related symptoms to go away. They were allowed to report in number of days, weeks, or months, anywhere from 1 day to 12 months. For the purposes of our analysis, we looked only at the percentage of respondents who reported 1 day of symptoms (to Indicate those with less severe TBIs) and those who reported experiencing 8 or more days or who were still experiencing symptoms at the time of the Interview (to Indicate those with more severe TBIs).

In addition, some signs and symptoms within the same tier demonstrated a stronger association with severity indicators than other symptoms. For example, associations between being dazed or confused, or trouble thinking straight and most of the indicators of severity were lower than they were for the other 2 signs included in the probable TBI tier (ie, LOC and difficulty remembering what happened just before or after the injury). Similar patterns suggested that the case definition could potentially be strengthened by moving particular symptoms up or down a tier.

Refinement phase 3: iterative process of modifying the 2-tiered case definition

Because of the removal of symptom onset as a consideration, tier 2 and tier 1 were combined into a single tier, resulting in 2 tiers: probable TBI and possible TBI. In addition, when looking at all injuries (n = 1691), dazed, confused, or trouble thinking straight was one of the most commonly reported symptoms (see Table 1) but generally not as strongly associated with indicators of severity as noted in Table 4. Conversely, difficulty learning or remembering new things, while initially placed in tier 1, showed very strong associations with the indicators of severity (see Table 4). These findings suggested that these 2 symptoms (dazed, confused, or trouble thinking and difficulty learning or remembering new things) may be good targets for modifying the TBI case definition.

In making iterative changes to the case definition, the following changes were tested: (a) moving dazed, confused, or trouble thinking down a tier as it generally had lower associations with severity indicators compared with other symptoms in the probable TBI tier, (b) modifying the symptom count in the “probable TBI” tier by removing it altogether, increasing to 4 or more, and decreasing to 2 or more, and (c) moving difficulty learning or remembering new things up a tier, from possible TBI to probable TBI, as this symptom had higher associations with severity indicators compared with other symptoms in the possible TBI tier (see Supplemental Digital Content Appendix 1, available at: http://links.lww.com/JHTR/A735). This process led to a total of 10 modified definitions to test:

1. Dropping down dazed, confused, or trouble thinking straight to possible TBI.

2. Removing symptom count for probable TBI.

3. Decreasing symptom count for probable TBI.

4. Increasing symptom count for probable TBI.

5. Moving up difficulty learning or remembering new things to probable TBI.

6. Dropping down dazed, confused, or trouble thinking straight to possible TBI and removing symptom count for probable TBI.

7. Dropping down dazed, confused, or trouble thinking straight to possible TBI and decreasing symptom count for probable TBI.

8. Dropping down dazed, confused, or trouble thinking straight to possible TBI and increasing symptom count for probable TBI.

9. Dropping down dazed, confused, or trouble thinking straight to possible TBI and moving up difficulty learning or remembering new things to probable TBI.

10. Dropping down dazed, confused, or trouble thinking straight to possible TBI, increasing symptom count for probable TBI, and moving up difficulty learning or remembering new things to probable TBI.

Next, steps 1 to 4 were repeated using the modified definitions, and the modified definitions that passed the criteria outlined in these steps (ie, improved the effect size by 0.15 or more for 2 out of the 4 indicators of severity compared with the original 3-tiered definition) were kept (see Table 5). At the end of this step, 4 modifications were retained. Based on steps 2 (testing effect size differences) and 3 (classifying significant effect sizes by criteria chosen for practical significance), these models were not considered different from each other (see Table 6).

Table 5.

Effect size^a differences in selected, outcomes between the original 3-tierecl TBI definition and modified 2-tiered, is, most, recent, head, injury.; National Concussion Surveillance System Pilot^b

	Effect size differences (95% Cl) between indicators of severityc
		Time to symptom resolution
Modified definition	Evaluationd Yes	1 day of symptomse	Still having symptom OR had symptoms for 8+ daysd	Extent of impact; social functioningd (continuous variable)	Extent of impact: work functioningd (continuous variable)
Modification 1	−0.09 (−0.10 to −0.07)	0.17 (0.15 to 0.19)	−0.15 (−0.16to —0.13)	−0.17 (—0.19 to —0.15)	−0.13 (−0.15 to-0.11)
Modification 2	−0.01 (−0.02 to-0.01)	0.04 (0.04 to 0.04)	0.01 (0.01 to 0.02)	0.06 (0.05 to 0.06)	0.03 (0.03 to 0.04)
Modification 3	−0.03 (−0.04 to −0.02)	0.04 (0.03 to 0.06)	−0.13 (−0.14 to −0.11)	−0.16 (−0.18 to −0.15)	−0.10 (−0.11 to −0.09)
Modification 4	−0.05 (−0.06 to −0.05)	0.15 (0.15 to 0.15)	− 0.13 (−0.13 to −0.12)	−0.14 (−0.14 to −0.13)	−0.11 (−0.11 to −0.11)
Modification 5	−0.05 (−0.05 to −0.05)	0.14 (0.13 to 0.14)	−0.11 (−0.11 to −0.11)	−0.14 (−0.14 to −0.14)	−0.10 (−0.10 to −0.10)
Modification 6	−0.10 (−0.10 to −0.10)	0.22 (0.22 to 0.22)	−0.01 (−0.01 to −0.01)	0.05 (0.05 to 0.06)	0.08 (0.08 to 0.08)
Modification 7	−0.05 (−0.07 to-0.02)	0.06 (0.03 to 0.09)	−0.11 (−0.14 to −0.09)	−0.15 (−0.18 to −0.12)	−0.10 (−0.13 to −0.08)
Modification 8	−0.11 (−0.13 to −0.09)	0.29 (0.27 to 0.31)	−0.21 (−0.23 to −0.19)	−0.21 (−0.23 to −0.19)	−0.13 (−0.15 to-0.11)
Modification 9	−0.08 (−0.10 to −0.06)	0.18 (0.16 to 0.20)	−0.14 (−0.15 to −0.12)	−0.16 (−0.18 to −0.14)	−0.13 (−0.15 to-0.11)
Modification 10	−0.09 (−0.11 to −0.07)	0.24 (0.22 to 0.25)	−0.13 (−0.15 to −0.11)	−0.21 (−0.23 to −0.19)	−0.11 (−0.14 to-0.09)
	Modification retained?f	Changes that were made from the original TBI definitiong
Modified definition
Modification 1	X	Drop down dazed/confused/trouble thinking
Modification 2		Remove symptom count in tier 3
Modification 3		decrease symptom count in tier 3
Modification 4		Increase symptom count in tier 3
Modification 5		Move up difficulty learning or remembering new things
Modification 6		Drop down dazed/confused/trouble thinking and remove symptom count from tier 3
Modification 7		Drop down dazed/confused/trouble thinking and decrease symptom count in tier 3
Modification 8	X	Drop down dazed/confused/trouble thinking and increase symptom count for tier 3
Modification 9	X	Drop down dazed/confused/trouble thinking and move up difficulty learning or remembering new things
Modification 10	X	Drop down dazed/confused/trouble thinking, increase symptom count for tier 3 and move up difficulty learning or remembering new things

Abbreviations: Cl, confidence interval; TBI, traumatic brain injury.

^aFor each modification, effect sizes (tetrachoric or polychoric correlations when appropriate) were calculated examining the association between placement of head injuries into a particular tier and outcome variables.

^bValues in boldface are those that passed the 0.15 cut point for practical significance and performed better than the original definition (eg, higher tiers demonstrated worse outcomes).

^cTo determine whether the modified definition was significantly different than the original, effect size differences were tested using the method from Meng and colleagues²³ (1992). The differences were converted using the Fisher z-transformation before testing and then back-transformed to correlations to aid interpretation. 95% confidence intervals that did not contain 0 were considered statistically significant from the original definition.

^dThe expected direction for a better model for the difference in effect size is a negative value.

^eThe expected direction for a better model for the difference in effect size is a positive value.

^fModel was retained for the next step if 2 out of the 4 outcomes were significant, they passed the 0.15 cut point for practical significance and performed better than the original definition.

^gAfter these first modifications were created (modifications 1–5), the modifications that demonstrated approved associations based on criteria from the “Model Retained” were then further modified again. The different shades of gray represent modifications that were chunked together. For example, Modification 1 was the only model that was retained from Modifications 1–5; thus, the next set of modifications all included “dropping down dazed/confused/trouble thinking” when creating a new modification.

Table 6.

Effect size^a differences in selected outcomes between the modified 2-tiered TBI definitions that were retained, most recent head injury, National Concussion Surveillance System Pilot^b

Modified definition comparisons	Effect size differences (95% Cl) between indicators of severityc					Best modeld Yes/No
	Evaluatione Yes	Time to symptom resolution
		1 day of symptomsf	Still having symptom OR had symptoms for 8+ dayse	Extent of impact: social functioninge Continuous variable	Extent of impact: work functioninge Continuous variable
Modification 1g vs 8h	−0.03 (−0.03 to −0.03)	0.13 (0.13 to 0.13)	−0.07 (−0.07 to −0.07)	−0.04 (−0.04 to −0.04)	0.00 (−0.00 to 0.00)	No
Modification 1 vs 9i	0.01 (0.00 to 0.01)	0.01 (0.01 to 0.01)	0.01 (0.01 to 0.01)	0.01 (0.01 to 0.01)	0.00(0.00 to 0.00)	No
Modification 1 vs 10i	−0.00 (−0.01 to 0.00)	0.07 (0.06 to 0.08)	0.02 (0.01 to 0.03)	−0.04 (−0.05 to -0.03)	0.02 (0.01 to 0.02)	No
Modification 8 vs 9	0.03 (0.03 to 0.03)	−0.12 (−0.12 to −0.12)	0.08 (0.08 to 0.08)	0.05 (0.05 to 0.05)	0.00 (0.00 to 0.00)	No
Modification 8 vs 10	0.02 (0.02 to 0.02)	−0.06 (−0.06 to -0.06)	0.09 (0.09 to 0.09)	0.00 (0.00 to 0.00)	0.01 (0.01 to 0.02)	No
Modification 9 vs 10	−0.01 (−0.01 to −0.01)	0.06 (0.06 to 0.06)	0.01 (0.01 to 0.01)	−0.05 (−0.05 to −0.04)	0.01 (0.01 to 0.01)	No

Abbreviation: Cl, confidence interval.

^aFor each modification, effect sizes (tetrachoric or polychoric correlations when appropriate) were calculated examining the association between placement of head injuries into a particular tier and outcome variables.

^bValues in boldface are those that passed the 0.15 cut point for practical significance and performed better than the original definition (eg, higher tiers demonstrated worse outcomes).

^cTo determine whether the modified definition was significantly different than the original, effect size differences were tested using the method from Meng and colleagues.²³ The differences were converted using the Fisher z-transformation before testing and then back-transformed to correlations to aid interpretation. 95°/o CIs that did not contain 0 were considered statistically significant from the original definition.

^dModel was considered better than the comparison model if 2 out of the 4 outcomes were significant, they passed the 0.15 cut point for practical significance and performed better than the comparison model.

^eThe expected direction for a better model for the difference in effect size is a negative value.

^fThe expected direction for a better model for the difference in effect size is a positive value.

^gModification dropped down dazed/confused/trouble thinking.

^hModification dropped down dazed/confused/trouble thinking and increased symptom count for tier 3.

ⁱModification dropped down dazed/confused/trouble thinking and moved up difficulty learning or remembering new things.

^jModification dropped down dazed/confused/trouble thinking and increased symptom count for tier 3 and moved up difficulty learning or remembering new things.

Finally, we went through the remaining 4 case definitions and examined differences between the modified definition and the original. For example, in the original case definition, 46.5% of adults with a probable TBI indicated that they were currently experiencing symptoms or had symptoms for 8+ days, compared with 18.9% of adults with a possible TBI and 15.8% of adults with a delayed possible TBI (see Table 7). In the Modification 1 definition, there was better separation between the tiers; 47.5% of adults with a probable TBI indicated that they were currently experiencing symptoms or had symptoms for 8+ days, while only 14.3% of adults with a possible TBI reported currently experienced symptoms or 8+ days of symptoms. From this phase, a modified TBI case definition (Modification 1) was selected.

Table 7.

Traumatic brain injury (TBI) indicators of severity by TBI tier and case definition modifications^a, most recent head injury, National Concussion Surveillance System Pilot^b

	Evaluated for TBI		Time to symptom resolution
	Yes Weighted %	Statistically significant differences between tiers	1 day Weighted %	Statistically significant differences between tiers	Still having symptom OR had symptoms for 8+ days Weighted %	Statistically significant differences between tiers
Original definition
Probable TBI (tier 3)	45.4	c	15.8	c	46.5	c
Possible TBI (tier 2)	20.9	d	41.5	d	18.9	d
Delayed possible TBI (tier 1)	23.8	e	42.0	f	15.8	f
Noncase (tier 0)	14.5	f	...		...
Modification 1
Probable TBI (tier 3)	46.9	c	14.5	c	475	c
Possible TBI (tier 2)	16.9	d	46.1	c	14.3	c
Noncase (tier 0)	14.5	f	...		...
Modification 8
Probable TBI (tier 3)	52.9	c	8.8	c	54.6	c
Possible TBI (tier 2)	20.0	d	42.2	c	172	c
Noncase (tier 0)	14.5	f	...		...
Modification 9
Probable TBI (tier 3)	46.6	c	14.4	c	472	c
Possible TBI (tier 2)	171	d	46.6	c	14.5	c
Noncase (tier 0)	14.5	f	...		...
Modification 10
Probable TBI (tier 3)	50.1	c	11.1	c	50.6	c
Possible TBI (tier 2)	19.4	d	43.4	c	17.7	c
Noncase (tier 0)	14.5	f	...		...
		Extent of impact: Social functioning		Extent of impact: Work functioning
	Mean	Statistically significant differences between tiers	Mean	Statistically significant differences between tiers
Original definition
Probable TBI (tier 3)	2.1	c	2.2	c
Possible TBI (tier 2)	1.3	c	1.4	c
Delayed possible TBI (tier 1)	1.1	e, g	1.2	e, g
Noncase (tier 0)	1.0	f, h	1.1	f, h
Modification 1
Probable TBI (tier 3)	2.1	c	2.2	c
Possible TBI (tier 2)	1.2	c	1.2	c
Noncase (tier 0)	1.0	c	1.1	c
Modification 8
Probable TBI (tier 3)	2.4	c	2.5	c
Possible TBI (tier 2)	1.2	c	1.3	c
Noncase (tier 0)	1.0	c	1.1	c
Modification 9
Probable TBI (tier 3)	2.1	c	2.2	c
Possible TBI (tier 2)	1.2	c	1.2	c
Noncase (tier 0)	1.0	c	1.1	c
Modification 10
Probable TBI (tier 3)	2.3	c	2.4	c
Possible TBI (tier 2)	1.2	c	1.3	c
Noncase (tier 0)	1.0	c	1.1	c

Abbreviation: TBI, traumaticbrain injury.

^aThe TBI definitions presented in the table are those that were retained after going through phase 3, step 3 of the refinement process.

^bPairwise differences between groups were tested using a univariate t statistic for continuous severity indicators and x² tests for categorical severity indicators. Ellipses indicate not applicable and also indicate that no analysis was conducted because no data were collected for these cases.

^cStatistically significant difference (P < .05) between all tiers.

^dStatistically significant difference (P < .05) between tiers 3 and 2.

^eStatistically significant difference (P < .05) between tiers 3 and 1.

^fStatistically significant difference (P < .05) between tiers 3 and 0.

^gStatistically significant difference (P < .05) between tiers 2 and 1.

^hStatistically significant difference (P < .05) between tiers 2 and 0.

Refinement phase 4: consultation with TBI SMEs

Modification 1 TBI case definition was chosen for several reasons. First, it met the criteria for passing the cut point for 2 out of the 4 indicators of severity (evaluated for TBI and the time to symptom resolution). Second, it was parsimonious. The only modification made from the original definition, outside of condensing the original 3-tiered definition into 2 tiers, was moving the symptom of dazed, confused, or trouble thinking straight down to possible TBI. All other elements of the original case definition were retained. Third, it fit well with the literature and what would seem to be acceptable to the wider TBI research community.

TBI SMEs that were consulted expressed general support for the revised case definition. They also agreed with the decision to remove symptom timing as a factor used in the case definition. Beyond the lack of empirical support for incorporating symptom timing, not having to assess symptom timing will make for a less complex survey. One SME expressed concern about moving one of the cardinal TBI symptoms (being dazed, or confused, or having trouble thinking straight) down to possible TBI. However, we found that about 76% of all probable TBIs did include dazed/confused/having trouble thinking straight; most were elevated to the probable TBI tier by virtue of its inclusion in the “3 or more symptoms” specification. Ultimately, the TBI SMEs agreed that our proposed revised TBI case definition had good face validity and would be generally well accepted in the field.

Final revised TBI case definition

Results of the 4 refinement phases generated a 2-tiered TBI case definition (see Figure 2). Persons who have a probable TBI are defined as individuals who sustained a hit to the head and who experienced difficulty remembering what happened just before or after the injury, or loss or consciousness, or 3 or more symptoms individually indicative of possible TBI. Persons who have a possible TBI are defined as individuals who sustained a hit to the head and who experience 1 or more of the following signs or symptoms: dazed, confused, or who have trouble thinking straight; nausea or vomiting; headache; dizziness, clumsiness, or balance problems; blurred or double vision; trouble concentrating; difficulty learning or remembering new things; sensitivity to light or noise; change in mood or temperament; and/or change in sleep or more tired than usual. Using the original TBI case definition, a total of 12.1% of adults in the NCSS sample would be classified as having sustained a TBI in the past 12 months; 8.2% experienced a probable TBI; 3.2% experienced a possible TBI; and 0.7% experienced a delayed possible TBI (see Figure 3). Under the revised TBI case definition, 12.1% of adults would be classified as having had a TBI in the past 12 months; 8.4% experienced a probable TBI; and 3.7% experienced a possible TBI.

Figure 2.

Revised TBI case definition. TBI indicates traumatic brain injury.

Figure 3.

Comparison of the percentage of adults who sustained a TBI using the original versus revised proposed TBI case definition, pilot National Concussion Surveillance System, 2018–2019. TBI indicates traumatic brain injury.

DISCUSSION

This report presents the process undertaken by the CDC to empirically refine a TBI case definition for analysis with self-reported data. The original TBI case definition was based on expert opinion, informed by prior TBI case definitions and clinical and research expertise. The tiers within the case definition were an attempt to represent the continuum of evidence that exists to support a TBI determination. The case definition was empirically tested against a hypothesis that those TBIs placed into a higher tier of certainty would have a stronger statistical association with the indicators of severity, specifically, whether medical evaluation was sought, time to symptom resolution, and self-reported impact on social and work functioning.

While the originally proposed 3-tiered TBI case definition adequately differentiated TBIs by level of certainty, the refinement process revealed several improvements. First, the analysis demonstrated that there was no empirical basis for retaining 3 tiers; having 2 tiers —probable TBI and possible TBI—was sufficient and parsimonious. Second, the analysis suggested that the commonly reported symptom of being dazed, confused, or having trouble thinking straight was less associated with the measured indicators of severity than the other highest tier symptoms (ie, LOC and difficulty remembering what happened just before or after injury). Consequently, being dazed, confused, or having trouble thinking straight was moved down to the possible tier and performance of the case definition improved.

A tiered case definition for TBI, which allows for varying certainty in whether a TBI occurred, such as the one presented here, allows for flexibility and transparency in reporting TBI incidence and prevalence. A more conservative approach would be to report only the probable TBIs—those we have more certainty about, given the constellation of signs and symptoms reported—while a more liberal approach would be to report the incidence/prevalence of TBI with probable and possible combined. Case definitions, such as the one examined here, provide transparency as they articulate exactly how a surveillance estimate was calculated. If researchers disagree with how a case definition produced an estimate, they can factor the case definition into how they interpret the estimate, or they can recalculate an estimate in a different way when data become publicly available.

As we noted (Note: see “Rationale for the Development of a Traumatic Brain Injury Case Definition for the Pilot National Concussion Surveillance System,” published in JHTR separately), tiered case definitions for TBI have previously been proposed and developed by other researchers,^13,14 including the 2023 American Congress of rehabilitation medicine definition (ACRM) for mild TBI,²⁶ but their use was confined to clinical encounters and were not necessarily applied to surveillance efforts. Subject matter experts who were consulted in the development of this case definition noted that previous TBI definitions^13,14,16 proposed by other groups typically include clinical assessments, such as balance testing, imaging, and a medical chart review. This type of information is not available for use in surveillance that relies entirely on self-report data. While we were not able to use these other TBI case definitions due to limitations of the measures collected, nor was it possible to precisely “test” these definitions given our step-by-step process of how symptoms were moved up and down tiers to create new case definition modifications, we did create a modification that closely mimics the criteria for signs and symptoms of the 2023 ACRM definition to the best of our ability. While the results demonstrated that this was a good definition in general (this definition was either similar to [eg, social and work impairment] or better than [eg, evaluation and 1 day of symptoms] the original case definition), this definition would not have passed our criteria of an effect size difference of 0.15 or more for at least 2 indicators of severity (data not shown). Future research could identify whether patterns similar to those identified in this analysis hold even when other information such as clinical assessments is available to make a TBI determination.

Limitations

There are at least 6 limitations to this study. First, given the self-reported nature of the data, issues with social desirability, telescoping, and recall bias are possible. Second, data analysis was limited to the adult sample of the pilot NCSS, and the applicability of this case definition to a pediatric population is unknown. Third, while the pilot NCSS queried respondents on a relatively large number of TBI-related symptoms, it is possible that the respondents might have experienced symptoms that were not included in the survey (eg, irritability, feeling sluggish), and inclusion of these other symptoms could have an impact on the empirically derived case definition. Fourth, the response rate of 8.4% is relatively low. Response rates for random digit dialing surveys have been declining over time and are likely to continue to decline. While the pilot NCSS was still able to obtain a nationally representative sample and the research team does not have concerns about nonresponse bias, future iterations may consider different methodologies (eg, address-based sampling). Finally, the case definition was tested in relation to a set of severity indicators based on the hypothesis that a tiered case definition can represent, to some extent, the level of certainty that a TBI has been sustained. An underlying hypothesis was that TBIs for which there is greater evidence (eg, more symptoms, presence of certain cardinal symptoms) will have more severe outcomes. However, greater evidence or certainty of a TBI is not the same as severity, and the degree to which they are different represents an unknown error. Furthermore, a broader set of severity indicators may have revealed different results. Some of our severity indicators, in particular, whether medical evaluation was sought, are likely driven by noninjury factors (eg, access to care or insurance coverage) which may have affected our analysis.

CONCLUSION

The TBI case definition generated from this analysis can be used within self-report surveys to incorporate flexibility and transparency in assessing the burden of TBI in the United States. Estimates of TBI prevalence and incidence from self-report surveys complement existing administrative sources, both in terms of expanding the scope and in providing more contextual information. Consequently, the results from this type of data collection can provide critical information to federal, state, and local decision makers, about TBI trends and how to target prevention strategies and estimate the resources needed for TBI prevention and management.