Comparison of US TBI-related emergency department visits from two data sources: NEISS-AIP and HCUP-NEDS

Abstract

Background

Traumatic brain injury (TBI) surveillance in the USA has historically used hospital administrative datasets and vital records to determine the number and rate of people who are hospitalised or die from a TBI. However, gaps exist in obtaining timely and accurate estimates of emergency department (ED)-treated TBIs. The purpose of this paper is to compare the number of TBI-related ED visits derived from two national datasets.

Methods

We used 2016–2021 data from the National Electronic Injury Surveillance System–All Injury Program (NEISS-AIP) and the Healthcare Cost and Utilization Project – Nationwide ED Sample (HCUP-NEDS). Estimates over time were compared.

Results

Applying a broad TBI case definition to NEISS-AIP that included concussions, skull fractures and internal injuries of the head, an average of 3.0 million TBI-related ED visits occurred on an annual basis, with the number of visits ranging from 2.8 million to 3.2 million. When using the broadest definition of TBI in HCUP-NEDS, which includes the unspecified injury of the head code, there were an average of 2.7 million TBI-related visits per year (estimates ranged from 2.4 million to 2.9 million). However, the narrower definition of TBI, which did not include unspecified injury of the head, found an average of 1.1 million visits per year, ranging from 1.0 million to 1.1 million.

Discussion and conclusion

When using the broadest definitions of TBI for both datasets, the number of annual TBI-related ED visits is similar. Deciding which dataset to use for TBI surveillance will depend on the project’s goals.

BACKGROUND

Traumatic brain injury (TBI) surveillance in the USA has historically relied on hospital administrative datasets and vital records.¹ On an annual basis, the U.S. Centers for Disease Control and Prevention (CDC) publishes the number and rate of TBI-related hospitalisations and deaths in the USA.² However, these estimates do not include people who seek care in other settings such as the emergency department (ED). In comparison to the number of people who seek care in an ED or other setting for their suspected TBI, relatively few people are hospitalised or die from a TBI. Examining TBI-related ED visits therefore helps create a more comprehensive understanding of TBI burden in the USA.

However, due to previous concerns about the International Classification of Diseases (ICD)-10-Clinical Modification (CM)-based TBI case definition,³ CDC has not published TBI-related ED visit estimates since using 2014 data (the last full year ICD-9-CM coding was in effect). Briefly, during the change from ICD-9-CM to ICD-10-CM, the new TBI case definition excluded an ‘unspecified injury to the head’ code, which several studies subsequently found was capturing a large proportion of TBI-related visits.^3–6 Therefore, excluding the code would result in large underestimates of ED-treated TBIs, while including the unspecified code would result in capturing many ‘false positives’.³ These challenges led to a stalling of national TBI surveillance of ED-treated TBI, with surveillance efforts largely relying on hospitalisation and death data.

A potential way forward is to provide an estimated range of TBI-related ED visits generated using definitions that vary in scope. There are two existing national datasets from which estimates of TBI-related ED visits might be obtained: the Agency for Healthcare Research and Quality’s Healthcare Cost and Utilization Project – Nationwide ED Sample (HCUP-NEDS) and the National Electronic Injury Surveillance System – All Injury Program (NEISS-AIP). HCUP-NEDS uses ICD-10-CM codes while NEISS-AIP uses medical record abstractors to obtain injury-related information including diagnoses. Each data source comes with its own strengths and limitations. The objectives of this paper are to: (1) estimate and compare numbers of TBI-related ED visits using definitions that vary in scope regarding how to define a TBI for each of the two data sources and (2) discuss strengths and limitations of each dataset to obtain current TBI-related ED visit estimates.

METHODS

Data sources

This project used 2016–2021 data from HCUP-NEDS and NEISS-AIP to produce estimates. At the time of our analysis, 2021 was the last year available for HCUP-NEDS, and therefore, the most recent data available for comparisons across both data sets. HCUP-NEDS is operated by the Agency for Healthcare Research and Quality and contains data from over 990 hospitals, which represent approximately 20% of visits to hospital-owned EDs. These visits are then weighted to produce national estimates.⁷

NEISS-AIP, operated by the US Consumer Product Safety Commission and CDC, contains data on patients treated in hospital EDs drawn from a nationally representative, stratified probability sample of hospitals. Data from approximately 66 hospitals are weighted by the inverse probability of selection to provide national estimates.^{8 9}

TBI identification and case definitions

To estimate the number of TBI cases in HCUP-NEDS, a definition based on ICD-10-CM codes was used. In the beginning of the 2016 federal fiscal year (which was 1 October 2015), hospitals switched from ICD-9-CM to ICD-10-CM codes. We estimated the number of persons with TBI using the National Center for Health Statistics (NCHS) ICD-10-CM-based¹⁰ TBI case definition (the ICD-10-CM codes for TBI included: S02.0 (fracture of vault of skull), S02.1– (fracture of base of skull), S02.80X–S02.82X (fracture of other specified skull and facial bones), S02.91 (unspecified fracture of skull), S04.02 (injury of optic chiasm), S04.03– (injury of optic tract and pathways), S04.04– (injury of visual cortex), S06.0– (concussion), S06.1– (; traumatic cerebral oedema), ;S06.2– (diffuse TBI), S06.3– (focal TBI), S06.4– (epidural haemorrhage), S06.5– (traumatic subdural haemorrhage), S06.6– (traumatic subarachnoid haemorrhage), S06.8– (other specified intracranial injuries), S06.9– (unspecified intracranial injury), S06.A– (traumatic brain compression and herniation), S07.1 (crushing injury of skull) and T74.4 (shaken infant syndrome)). Persons with an S09.90 (head injury unspecified) code only were also analysed, as this code was historically included in the prior ICD-9-CM case definition for TBI¹¹ but was excluded in the ICD-10-CM definition based on concerns about its accuracy of representing TBIs.³ Because of the recent change in the NCHS TBI case definition and the concerns about the head injury unspecified code, this analysis presents three sets of TBI estimates for HCUP-NEDS: (1) the NCHS ICD-10-CM TBI case definition, (2) head injury unspecified only and then (3) a combined estimate (case definition+unspecified head injury).

To estimate the number of TBI cases in NEISS-AIP, a definition was applied to cases based on if the coded body part injured was the head and the primary diagnosis was listed as concussion, internal organ injury or fracture. Due to the increased certainty that concussion and skull fracture represent ‘true’ TBIs and less certainty about the internal organ injury of the head diagnoses (similar to the head injury unspecified code in HCUP-NEDS), this analysis also presents three sets of TBI estimates: (1) a combined estimate of concussions and skull fractures, (2) internal injury of the head only and then (3) a total combined estimate of all three codes. In 2018, the secondary body part injured and secondary diagnosis fields were added and these cases were also included in the analysis if they met the TBI definition.

For both datasets, only initial visits for an injury were included, and patients who were dead on arrival or died in the ED were excluded.

Patient and public involvement

In this secondary data analysis, patient and public involvement was not conducted due to the nature of the study design. The analytic approach limited opportunities for engaging patients or the public in shaping research questions, selecting outcomes or planning dissemination strategies. While we acknowledge the significance of patient and public involvement in enhancing research relevance and impact, it was not feasible to incorporate their perspectives within the framework of this analysis.

This study involves analysis of previously collected, deidentified and publicly available data from human participants; therefore, institutional review board review was not required. However, this study was conducted consistent with ethical guidelines for the conduct of research.

Analysis

Estimates and 95% confidence intervals (CIs) were calculated using complex survey procedures in SAS V.9.4 (SAS Institute, Cary, North Carolina, USA). Combined totals for HCUP-NEDS and NEISS-AIP were compared using z-tests and considered significantly different when the p value was <0.05.

RESULTS

HCUP-NEDS estimates

Based on HCUP-NEDS data from 2016 to 2021, the overall number of TBI-related ED visits, under the combined definition, ranged from 2 388 910 in 2020 to 2 883 726 in 2017 (table 1), with an average of 2.7 million visits annually (data not shown). Head injury unspecified (ICD-10-CM S09.90) made up the majority of these visits: 59–61% of TBI-related visits each year. When examining just the NCHS TBI case definition alone, which excludes ICD-10-CM S09.90 (head injury unspecified), the number of TBI-related ED visits ranged from 969 832 in 2020 to 1 145 250 in 2017, with an average of 1.1 million visits annually.

Table 1.

Number of TBI-related ED visits in HCUP-NEDS, 2016–2021

	Combined*		Proposed TBI definition		Head injury unspecified only (S09.90)
	Estimate	95% CI	Estimate	95% CI	Estimate	95% CI
2016	2 866 272	(2 715 366 to 3 017 177)	1 120 862	(1 062 762 to 1 178 961)	1 745 410	(1 633 855 to 1 856 965)
2017	2 883 726	(2 724 795 to 3 042 657)	1 145 250	(1 086 442 to 1 204 057)	1 738 476	(1 624 155 to 1 852 798)
2018	2 827 302	(2 683 165 to 2 971 438)	1 099 857	(1 044 193 to 1 155 521)	1 727 444	(1 622 050 to 1 832 839)
2019	2 858 635	(2 721 250 to 2 996 019)	1 129 714	(1 075 636 to 1 183 792)	1 728 920	(1 629 110 to 1 828 731)
2020	2 388 910	(2 269 948 to 2 507 872)	969 832	(927 346 to 1 012 319)	1 419 078	(1 327 999 to 1 510 157)
2021	2 534 283	(2 416 567 to 2 651 999)	1 012 548	(969 382 to 1 055 713)	1 521 736	(1,432 346 to 1 611,125)

^*Includes cases with TBI code in any field; excludes deaths that occur prior to arrival or in the emergency department. ED, Emergency Department; HCUP-NEDS, Healthcare Cost and Utilization Project – Nationwide Emergency Department Sample; TBI, traumatic brain injury.

NEISS-AIP estimates

In comparison, based on NEISS-AIP data from 2016 to 2021, the overall estimated number of TBI-related ED visits, using the combined definition, ranged from 2 788 507 in 2020 to 3 216 072 in 2019 (table 2), with an average of 3.0 million visits annually (data not shown). Internal injury of the head cases made up the majority of these visits, ranging from 82% of the head injury-related diagnoses in 2017 to 85% of the head injury-related diagnoses in 2020.

Table 2.

Number of TBI-related ED visits in NEISS-AIP, 2016–2021

	Combined*		Concussion and/or skull fracture		Internal injury only
	Estimate	95% CI	Estimate	95% CI	Estimate	95% CI
2016	3 037 258	(2 214 699 to 3 859 818)	523 850	(372 299 to 675 401)	2 513 408	(1 778 708 to 3 248 109)
2017	2 909 455	(2 267 499 to 3 551 411)	517 168	(389 780 to 644 556)	2 392 287	(1 814 788 to 2 969 786)
2018	2 992 733	(2 322 451 to 3 663 015)	488 781	(380 020 to 597 542)	2 503 952	(1 879 378 to 3 128 525)
2019	3 216 072	(2 564 318 to 3 867 825)	518 792	(409 461 to 628 124)	2 697 279	(2 087 787 to 3 306 772)
2020	2 788 507	(2 133 941 to 3 443 074)	410 741	(317 538 to 503 944)	2 377 767	(1 773 067 to 2 982 467)
2021	2 987 021	(2 296 585 to 3 677 457)	448 010	(361 005 to 535 014)	2 539 012	(1 904 211 to 3 173 812)

^*Includes cases where either the primary or secondary body part-diagnosis combination is body part injured of head and diagnosis of concussion, fracture, or internal organ injury; excludes patients that died.

ED, emergency department; NEISS-AIP, National Electronic Injury Surveillance System–All Injury Program; TBI, traumatic brain injury.

Comparison of estimates

Within HCUP-NEDS, the NCHS TBI case definition captures between 39–41% of the number of combined TBI-related ED visits per year (table 1). Within NEISS-AIP, concussion and skull fracture captures between 15% and 18% per year of the number of the combined TBI-related ED visits (table 2). Combined estimates in HCUP-NEDS and NEISS-AIP are close; there were no significant differences, based on the z-tests, in the combined TBI-related ED visit totals between HCUP-NEDS and NEISS-AIP in any data year.

DISCUSSION

Our findings demonstrate similar estimates for TBI-related ED visits between HCUP-NEDS and NEISS-AIP when using combined case definitions in each (ie, in HCUP-NEDS, combining the NCHS case definition with the unspecified head injury cases and in NEISS-AIP, combining concussion and skull fracture cases with internal injury of the head cases). This is despite the datasets’ different sample sizes and purpose, methods of data collection and different variables used (ie, ICD-10-CM codes in HCUP-NEDS and body part and primary diagnosis in NEISS-AIP). Across the study period, the estimated overall annual number of TBI-related ED visits ranged from 2.4 million to 2.9 million using HCUP-NEDS data and 2.8 million to 3.2 million using NEISS-AIP data. However, there were no significant differences in the combined total number of visits between the two datasets for any year. Each dataset contains its own strengths and limitations, and there are different implications of using each of these national data sources for TBI-related ED estimates (table 3).

Table 3.

Comparison of features of HCUP-NEDS and NEISS-AIP for estimating TBI-related ED visits

Area of focus or indicators of interest	HCUP-NEDS	NEISS-AIP
Sample size of hospitals	Large	Moderate
Sampled to be nationally representative	Yes	Yes
Data collected for the sole purpose of injury surveillance	No	Yes
Includes a patient narrative that describes the injury	No	Yes
Stratification by demographic characteristics	Yes	Mostly yes, with the exception of race/ethnicity
Information on sports and recreation	No	Yes
Calculations of trends over time	Yes, after 2015	Yes, after 2001
Geographic information on patients	Yes	No
Payer information	Yes	No
Timeliness of data release	3 years	18 months

ED, emergency department; HCUP-NEDS, Healthcare Cost and Utilization Project – Nationwide Emergency Department Sample; NEISS-AIP, National Electronic Injury Surveillance System – All Injury Program; TBI, traumatic brain injury.

Benefits of HCUP-NEDS for estimating TBI

HCUP-NEDS uses a very large sample of hospitals, nearly 1000, is nationally representative and due to its reliance on ICD-10-CM coding, has TBI cases that are standardised across hospitals. HCUP-NEDS also provides information on sociodemographic variables and injury characteristics such as sex, age, race/ethnicity, geographic information (eg, rural/urban status of patient and hospital), intent and mechanism of injury and payer information, making it an appropriate data source for producing national TBI estimates and for stratifying TBI numbers and rates by demographic characteristics.¹²

Limitations of HCUP-NEDS for estimating TBI

Surveillance based on HCUP-NEDS must rely on administrative billing data, which were not designed for surveillance purposes, to produce its national estimates. Plus, there is concern about the unspecified injury to the head code. This code (959.01) was included in the ICD-9-CM case definition for TBI¹³ but was dropped for the ICD-10-CM case definition for TBI (S09.90).¹⁰ A study³ examining ED medical records using ICD-10-CM coding in four states found that between 60–74% of sampled medical records assigned S09.90 contained evidence of TBI; this means that between 26–40% of records contained no evidence of a TBI. However, it is important to note that a limitation of that study is that ‘no evidence’ of a TBI does not rule out that a TBI occurred, just that there was no documented signs or symptoms associated with a TBI in the medical record. Therefore, it is possible that the ICD-9-CM TBI case definition could have overestimated TBI incidence, while dropping the code, as was done for the ICD-10-CM TBI case definition, almost certainly underestimates TBI-related ED incidence.

Because the case definition for TBI changed between ICD-9-CM and ICD-10-CM in 2015, this also makes TBI estimates pre-2015 and post-2015 incomparable. Multiple studies^4–6 demonstrated that 50–58% of TBI-related ED and hospitalisation records contained the ICD-9-CM code 959.01 (unspecified injury to the head), demonstrating the popularity of this code and how TBI ED estimates are likely affected by the change in case definition. Our findings show that S09.90 makes up the majority (59–61%) of TBI-related ED visits when this code is included in the TBI case definition. Other limitations of using ICD coding to define TBI include misclassification (particularly for mild TBI), with validation studies reporting 55–72% sensitivity and 80–85% specificity in the case definition.^14–16

Implications of using HCUP-NEDS to estimate TBI

Given these benefits and drawbacks, HCUP-NEDS may be particularly well-suited for surveillance of TBI when uniformity across sites is desired and when there is a need for analysis of estimates by demographic characteristics, such as race and ethnicity of patients. HCUP-NEDS also contains information on payer and some geographic information, which may be helpful for research on TBI burden and differences across these key characteristics.

Benefits of NEISS-AIP for estimating TBI

NEISS-AIP, while comprising a much smaller sample of hospitals (around 80 beginning in 2022), can also be used to produce nationally representative estimates of TBI. A major benefit is that these data are collected specifically for injury surveillance purposes and include patient narratives, which allows researchers to better understand circumstances of how each injury occurred.¹⁷ For example, NEISS-AIP also collects information on whether injuries were related to sports-related and recreation-related activities, which is particularly relevant for TBIs (eg, sports concussion) and not well-captured in ICD-10-CM codes. In fact, NEISS-AIP has historically been used to track the number and rate of sports-related and recreation-related concussion in the USA^18–20 as well as other demographic data such as sex, age, intent and mechanism of injury and products associated with the injury. NEISS-AIP data are also released in a timelier fashion than HCUP-NEDS (18-month time lag compared with 3-year time lag)¹⁷ and do not rely on ICD-10-CM codes, thus potentially avoiding the unspecified injury to the head and other misclassification issues.

Limitations of NEISS-AIP for estimating TBI

It is important to note that NEISS-AIP does not capture ‘TBIs’; it only allows for the estimation of concussions, skull fractures and internal injuries of the head, which have been grouped together for this study and called ‘TBI’. Considering the similarity in the TBI estimates across the two datasets, it is likely that some of the head injuries in NEISS-AIP would be classified as ‘head injury unspecified’ if they appeared in HCUP-NEDS, so there is also the possibility that using the internal injury of the head cases to classify TBIs may include non-TBI cases. Additionally, NEISS-AIP only contains two body part-diagnosis combination fields while HCUP-NEDS has 40 diagnosis fields. Therefore, NEISS-AIP may underestimate TBIs in situations where there may be multiple injuries such as motor vehicle crashes. NEISS-AIP may also not allow for the stratification of estimates by race/ethnicity given the relatively high missingness of that variable.²¹

Implications of using NEISS-AIP to estimate TBI

Given these benefits and drawbacks, NEISS-AIP may be particularly well-suited for overall national estimates of ED-treated TBI, surveillance of sports-related and recreation-related TBIs, TBI stratified by select demographics (eg, age and sex), and when more timely estimates are needed. Additionally, because the diagnosis codes have remained consistent across the years, NEISS-AIP is also appropriate when longer-term trend analyses of injuries are desired.

Study limitations

Both datasets only contain the estimated number of TBIs evaluated in the ED, which underestimates the true burden of TBI. These estimates do not include TBIs that are treated in other settings (eg, primary care offices, urgent care settings, etc), and TBIs where medical care was not sought. Plus, it can be difficult to identify TBIs at the time of initial evaluation in the ED given some signs and symptoms do not develop immediately (eg, difficulty concentrating, changes in mood/temperament, difficulty sleeping) or they may be perceived as a symptom of another health condition. Additionally, there is no universally agreed on surveillance definition for TBI. For this analysis, TBI case definitions were used for both HCUP-NEDS and NEISS-AIP, but these definitions may miss other valid TBI cases, such as those that occur in multitrauma incidents but where a head injury is not noted or diagnosed. As mentioned, there is some uncertainty regarding the use of ‘head injury unspecified’ codes in HCUP-NEDS and whether they represent true TBIs, which could inflate estimates of TBI-related ED visits. Similarly, there is some uncertainty in NEISS-AIP that the internal injury of the head designation constitutes a TBI, where we surmise cases with the ‘head injury unspecified’ would be coded to if they appeared in HCUP-NEDS. This study did not use NEISS-AIP narratives to classify TBIs; it is likely that there may be greater certainty that the TBI cases in NEISS-AIP are truly TBI cases if the narratives were reviewed as they can provide data on signs and symptoms and mechanism of injury. Future studies may investigate whether using the content of NEISS-AIP narratives can improve what constitutes a TBI. For NEISS-AIP, a second diagnosis-body part field combination was added during the study period (2018) which may have resulted in higher estimates for those years. However, the estimates were similar before and after the addition, and the secondary fields only represented 10–12% of the estimated cases. While the overall estimates were similar between the two datasets, it is possible that introducing demographic stratification, for example, sex and age, could reveal diverging estimates. It may be valuable for future research to compare yearly national estimates stratified by sex, age and other variables.

CONCLUSIONS

HCUP-NEDS and NEISS-AIP produce similar TBI-related ED estimates when using their broadest definitions of TBI. Given historical underestimation of TBI (due to injuries not seen in healthcare settings, unreported symptoms and missed diagnoses), using a transparent, broad definition of TBI may be warranted. In addition to definition selection, researchers must select a data source for estimation. Both sources examined in this study (ie, HCUP-NEDS and NEISS-AIP) have multiple benefits and some drawbacks for TBI surveillance and deciding which source to use depends on the goals of the project (eg, timeliness of data, mechanisms of injury being considered, characteristics required for stratified analyses, long-term trend analyses requiring a consistent case definition).

WHAT IS ALREADY KNOWN ON THIS TOPIC

⇒ There are gaps in traumatic brain injury (TBI) surveillance when solely relying on hospital administrative datasets and national vital records; missing are the number and rate of people who are evaluated in an emergency department (ED) for a TBI.

WHAT THIS STUDY ADDS

⇒ Comparing the number of TBI-related ED visits derived from two national datasets shows that between 2.7 million and 3 million TBI-related ED visits occur on an annual basis when using the broadest definitions of TBI for both datasets.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

⇒ Deciding which source to use for TBI surveillance and research depends on the goals of the project (eg, timeliness of data, mechanisms of injury being considered, characteristics required for stratified analyses, long-term trend analyses requiring a consistent case definition).

Data availability statement

Data are available in a public, open access repository.