Multisite medical record review of emergency department visits for traumatic brain injury

Abstract

Background

In 2016, the CDC in the USA proposed codes from the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) for identifying traumatic brain injury (TBI). This study estimated positive predictive value (PPV) of TBI for some of these codes.

Methods

Four study sites used emergency department or trauma records from 2015 to 2018 to identify two random samples within each site selected by ICD-10-CM TBI codes for (1) intracranial injury (S06) or (2) skull fracture only (S02.0, S02.1-, S02.8-, S02.91) with no other TBI codes. Using common protocols, reviewers abstracted TBI signs and symptoms and head imaging results that were then used to assign certainty of TBI (none, low, medium, high) to each sampled record. PPVs were estimated as a percentage of records with medium-certainty or high-certainty for TBI and reported with 95% confidence interval (CI).

Results

PPVs for intracranial injury codes ranged from 82% to 92% across the four samples. PPVs for skull fracture codes were 57% and 61% in the two university/trauma hospitals in each of two states with clinical reviewers, and 82% and 85% in the two states with professional coders reviewing statewide or nearly statewide samples. Margins of error for the 95% CI for all PPVs were under 5%.

Discussion

ICD-10-CM codes for traumatic intracranial injury demonstrated high PPVs for capturing true TBI in different healthcare settings. The algorithm for TBI certainty may need refinement, because it yielded moderate-to-high PPVs for records with skull fracture codes that lacked intracranial injury codes.

Introduction

Traumatic brain injury (TBI) is a temporary or permanent disruption of normal brain function due to an external force.^{1 2} Public health agencies in the USA have conducted TBI surveillance using healthcare billing datasets coded in International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM), at least since the 1990s.^3–5 On 1 October 2015, healthcare systems in the USA implemented International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). For TBI surveillance, the CDC proposed use of ICD-10-CM diagnosis codes for intracranial injury; skull fracture without crushing skull injury; injury to optic chiasm or pathways, injury to visual cortex; crushing injury of skull; and shaken infant syndrome in preparation for this change and before data coded in ICD-10-CM was available to validate their proposed code set.⁶

This study is the first to estimate the PPV of CDC-proposed TBI codes, one component of a comprehensive validation. The objective was to estimate PPV of TBI for the ICD-10-CM diagnosis codes specific to intracranial injury and skull fracture in emergency department (ED) billing records. This study focused on less severe TBI, such as concussion, that does not require inpatient hospitalisation.

Methods

The study design was a retrospective, cross-sectional review of medical records at ED sites in four states: Colorado, Kentucky, Maryland and Massachusetts. To assess PPV for TBI, each study site team selected two random samples of ED billing records based on any ICD-10-CM diagnosis code for: (1) only skull fracture without other proposed TBI codes or (2) intracranial injury (S06) with or without other proposed TBI codes.⁶ (In Maryland, one site of the two sites was a neurotrauma referral centre, not an ED, and its trauma registry was used for sampling.) To replicate methods of US public health surveillance of TBI-related ED visits, the sampling frame was limited to records that documented initial medical encounters for patients treated in EDs and discharged home during 2015–2018 (Faul et al ¹, p. 49). Reviewers recorded signs and symptoms related to TBI and head imaging findings from the medical records for each sample.

Case definitions

The study focused on ICD-10-CM codes for skull fracture and intracranial injury as the potential TBI cases of interest in ICD-10-CM-coded ED discharge records, grouped as follows:

1. Fracture of skull bones: ICD-10-CM codes S02.0, S02.1-, S02.8- or S02.91 for initial encounter (seventh character in the ICD-10-CM code of ‘A’ or ‘B’ or missing) in any discharge diagnostic field (first-listed or secondary diagnosis), without concurrent ICD-10-CM diagnosis code for intracranial injury (S06); and

2. Intracranial injury: ICD-10-CM codes S06 for initial encounter (seventh character of ‘A’ or missing) in any diagnostic field, with or without a code for skull fracture or other proposed TBI codes (S04.02 for injury to optic chiasm, S04.03- for injury to optic pathways, S04.04- for injury to visual cortex, S07.1 for crushing injury of skull or T74.4 for shaken infant syndrome).⁶

TBI, especially mild TBI treated in the ED, can manifest in transient signs and symptoms. Therefore, the study team developed a case definition that represented levels of certainty regarding TBI based on the presence of TBI-related clinical documentation in the medical record. With input from TBI experts at CDC, the study team adapted an unpublished, self-reported definition for concussion surveillance and a clinical TBI definition to create rank-ordered levels of certainty of TBI based on the number and type of symptoms present in the medical record and imaging findings (table 1).⁷ This study definition used the same symptoms as the CDC concussion definition except time of symptom onset after the injury event, because the records lacked precise time of onset.

Table 1.

Certainty of TBI based on signs, symptoms and imaging result: multisite review of ICD-10-CM codes for intracranial injury and skull fracture treated in EDS

Level of certainty of TBI	Injury event confirmed by reviewer AND
	Group 1*: signs or symptoms	Group 2†: signs or symptoms	TBI result from head imaging‡
Highest	1 or more (OR)	3 or more (OR)	Positive
Medium	0 (AND)	0–2 (AND)	Suspected TBI
Medium	0 (AND)	2 (AND)	Negative
Lowest	0 (AND)	1 (AND)	Negative
None	0 (AND)	0 (AND)	Negative

*Group 1: dazed/foggy/confused, memory problems, any loss of consciousness following an injury event.

†Group 2: nausea or vomiting, headache/pressure in head, dizziness/poor balance, change in vision, poor concentration, sensitivity to noise/light, Irritable/change in mood, drowsiness/change in sleep, speech problems, hearing problems, weakness/numbness following an injury event.

‡Almost all of the imaging of the head, when done, was a CT scan. The online supplemental file 1) lists the specific text in the radiology reports that indicated positive for TBI, suspected TBI and negative for TBI.

ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; TBI, traumatic brain injury.

Shown in table 2, the study team implemented the study in various types and numbers of hospitals in the four states, reflecting differences in public health authority in the states and the large geographic size and distribution of hospitals in one state (Colorado). The study site in Kentucky was a two-hospital system with a Level 1 trauma centre that cares for a large majority of patients with brain injury in the eastern half of Kentucky. The study sites in Maryland were the R. Adams Cowley Shock Trauma Center, part of the University of Maryland Medical System, and the Johns Hopkins Hospital. Due to triage and transfer protocols in Maryland, emergency medical services take the majority of adults with TBI directly to the Shock Trauma Center, the primary adult neurotrauma referral centre for Maryland, and most paediatric patients with TBI to Johns Hopkins Hospital.

Table 2.

Setting, locations, relevant dates and other study characteristics: multisite review of ICD-10-CM codes for intracranial injury and skull fracture treated in EDs

Study sites	One shock trauma, One ED in referral centres	Two EDs in university system	EDs in level I, II, III trauma hospitals	EDs in all acute care hospitals
State of study site	Maryland	Kentucky	Colorado	Massachusetts
Setting: type of hospital	Large academic hospitals*	Acute care hospitals	Acute care hospitals in 11 urban counties†	All acute care hospitals in the state
Number of hospital EDs in the study	2	2	29	73
Percentage of state population covered by hospitals (catchment area)	15	38	83	100
Dates of the ED visits in the billing dataset	January 2016- December 2018	January 2016- June 2018	January 2017-December 2017	October 2015-September 2016
Number of diagnosis fields in the ED billing dataset	30	25	30	34
Type of medical record reviewer	18 clinical researchers	2 trauma nurses	1 professional MR coder	3 professional MR coders
Access to electronic medical record	Yes	Yes	ED report and EMS transport‡	ED report and EMS transport‡

*In Maryland, one of the two sites was a neurotrauma referral centre, not an ED, and its trauma registry was used for sampling.

†The Colorado team selected the 29 acute care hospitals that had a trauma designation of level I, II or III, and located in Colorado’s 11 most populous counties. These 29 hospitals had 77% of all ED visits for intracranial injury and 81% of all ED visits for skull fractures (without an intracranial injury) in the state during the study time period.

‡Additional medical documents requested in Colorado: face sheet, radiology reports, toxicology reports. Massachusetts: face sheet, radiology reports, lab work/lab notes, triage notes, history and physical.

ED, emergency department; EMS, emergency medical services; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; MR, medical record.

Study population and data source

The study population was persons treated in the ED or shock trauma centre for an intracranial injury or skull fracture and discharged home. The study used two data sources: (1) administrative billing records for ED visits to select eligible records and (2) the corresponding original medical record, to collect TBI signs, symptoms and imaging findings. Staff limited the ED billing records to records for patients who were state residents. Selecting state residents replicated selection criteria used in TBI surveillance (Faul et al ¹, p49). The sites had different lag time in receiving ED billing data or needed a longer time to accumulate at least 385 eligible ED records for the review, resulting in different study periods for each site.

Variables of interest from medical records

The reviewers at the study sites collected TBI-related signs and symptoms as well as findings from any imaging of the head by X-ray, CT scan or MRI. Reviewers confirmed that the medical record indicated a recent injury event preceding the signs and symptoms. Signs and symptoms variables were those typical of TBI presentation: loss of consciousness; being dazed, foggy or confused; other memory problems; nausea/vomiting; headache or pressure in head; dizziness/poor balance; change in vision; poor concentration; sensitivity to noise or light; irritability or change in mood or personality; drowsiness or change in sleep (particularly in infants and very young children); speech problems; hearing problems; or weakness/numbness. Due to the typically short time between an injury event and patient arrival at an ED, some signs and symptoms might have resolved or could have occurred after discharge from the ED. The Kentucky study team added a variable that asked their clinical nurse reviewers, based on the complete record review, to give their clinical opinion by selecting one of these choices: No TBI, Possible TBI, Probable TBI or TBI.

To reduce systematic differences (observer bias) among the reviewers and over time, the study team developed a common abstraction form to record the variables of interest from the original medical record and a detailed reviewer manual with definitions and examples, including clinical terms. Based on input from subject matter experts, the study team developed a list of clinical terms and phrases to assist medical record reviewers in determining whether findings from a CT, MRI or X-ray image of the head were positive, suspect or negative for TBI (see online supplemental file for list.)

Study size

To determine the desired sample size, the study team used 90% as the expected PPV of TBI for both intracranial injury and skull fracture, based on unpublished data from states’ CDC-funded TBI surveillance during the ICD-9-CM era. Each study site chose a random sample of records to provide 385 reviews and achieve a margin of error no larger than 3% for the 95% CI of the estimated PPV.

Statistical methods

Staff from the four states independently calculated the counts and percentage of reviewed medical records that met the criteria for each level of TBI certainty, using uniform categorisation of TBI certainty (table 1). Staff calculated certainty of TBI for the sample with an ICD-10-CM code for intracranial injury and repeated this analysis with the sample for skull fracture. Although they had not been validated, the levels of TBI certainty provided a method of sensitivity analysis in this study to examine the extent to which the constellation of signs, symptoms and diagnostic imaging affects the PPV and to determine whether there is a level of certainty at which the PPV was similar across the four study sites, given that two sites used clinical researchers or nurses and had access to the full electronic health record in those hospitals. Data analysis was performed using SAS V.9.3 or V.9.4, and IBM SPSS V.23.0.

Results

Table 3 presents the results of the signs and symptoms inventories for the traumatic intracranial injury samples. The majority of ED medical records at all four study sites had documentation of at least one of the following signs or symptoms after an injury event: dazed/foggy/confused, memory problems or loss of consciousness. However, the proportion of each sign or symptom varied by study site. Loss of consciousness was the most common of these three signs and symptoms at the study sites in Maryland, Kentucky and Colorado. The study sites in Maryland (referral centres) had the largest proportion with loss of consciousness. Headache/pressure in the head and nausea or vomiting were the most common signs and symptoms among the ones given less weight in the level of TBI certainty at all four study sites. In the Maryland study site, 88.6% of the intracranial injury records documented imaging of the head, and 33.6% of the findings were positive for TBI. In the Kentucky study site, 71.2% of the medical records documented imaging of the head, and 19.5% had positive or suspected TBI findings. In Colorado, 61.4% had imaging and only 4.3% (of all 376 records) had a positive finding for TBI. In Massachusetts, 61.1% had imaging and only 5.5% (of all 388 records) had a positive finding for TBI.

Table 3.

TBI-related signs and symptoms and imaging findings among a random sample of medical records assigned traumatic intracranial injury ICD-10-CM codes*, by study site

Study sites	One shock trauma, one ED in referral centres (Maryland) n=456		Two EDs in university system (Kentucky) n=385		EDs in level I, II, III trauma hospitals (Colorado) n=376‡		EDs in all acute care hospitals (Massachusetts) n=388
Known or suspected signs and/or symptoms	Count	%	Count	%	Count	%	Count	%
In group 1
Dazed/foggy/confused	163	35.7	115	29.9	110	29.3	118	30.4
Memory problems	154	33.8	81	21.0	68	18.1	46	11.9
Loss of consciousness	308	67.5	168	43.6	145	38.6	99	25.5
Any group 1 sign/symptom	361	79.2	230	59.7	214	56.9	202	52.1
No group 1 signs/symptoms	88	19.3	155	40.3	162	43.1	186	47.9
Total	449	98.5	385	100.0	376	100.0	388	100.0
In group 2
Nausea or vomiting	83	18.2	94	24.4	145	38.6	171	44.4
Headache/pressure in head	202	44.3	238	61.8	250	66.5	319	82.2
Dizziness/poor balance	68	14.9	69	17.9	113	30.1	127	32.7
Change in vision	27	5.9	34	8.8	65	17.3	69	17.7
Poor concentration	19	4.2	†	†	17	4.5	†	†
Sensitivity to noise/light	†	†	15	3.9	31	8.2	37	9.6
Irritable/change in mood	52	11.4	†	†	21	5.6	†	†
Drowsiness/change in sleep	57	12.5	34	8.8	40	10.6	37	9.6
Speech problems	33	7.2	11	2.9	17	4.5	15	3.9
Hearing problems	†	†	†	†	11	2.9	†	†
Weakness/numbness	27	5.9	†	†	34	9.0	31	8.1
Imaging of the head
Imaging performed	404	88.6	274	71.2	231	61.4	237	61.1
Positive imaging findings	153	33.6	75	19.5	16	4.3	14	5.5

The bold values indicate that those two characteristics were exclusive (meaning a patient could not have both) and their total counts or percentages were shown in the "total" row below them.

*Intracranial injury ICD-10-CM codes began with ‘S06’ and ‘A’ in seventh character of the code or missing.

†Suppressed because the count was less than 11.

‡Nine of the 385 sampled medical records in Colorado indicated that the patient was not a state resident, was not discharged home, did not have an injury event or was admitted as an inpatient.

ED, emergency department; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; TBI, traumatic brain injury.

When defining PPV based on the highest certainty of TBI (table 4), the TBI PPV of the sample selected using ICD-10-CM intracranial injury codes was 87.7% (95% CI: 84.8% to 90.9%) in Maryland and 74.0% (95% CI: 69.3% to 78.3%) in Kentucky, 74.5% (95% CI: 69.8% to 78.8%) in Colorado and 70.4% (95% CI: 65.5% to 74.9%) in Massachusetts. Based on high and medium certainty levels, PPV in three sites (82.1%–86.1%) approached but did not reach the 91.7% PPV for TBI in Maryland. The Kentucky clinical reviewers, based on complete record review assessment, determined that 218 (76.5%) patients whose record had a traumatic intracranial injury code had a TBI or probable TBI out of the 285 records with high certainty of TBI based on the study algorithm (data not shown). Similarly, in the clinical reviewer’s opinion, 26 (83.9%) patients had a TBI or probable TBI out of the 31 records that had a medium certainty of TBI based on the algorithm.

Table 4.

PPV for TBI based on a review of ED records with traumatic intracranial injury ICD-10-CM codes*, by study site

Study sites	One shock trauma, one ED in referral centres (Maryland) n=456		Two EDs in university system (Kentucky) n=385		EDs in level I, II, III trauma hospitals (Colorado) n=376‡		EDs in all acute care hospitals (Massachusetts) n=388
Level of certainty of TBI	Count	%	Count	%	Count	%	Count	%
Highest†	400	87.7	285	74.0	280	74.5	273	70.4
Medium	18	3.9	31	8.1	43	11.4	61	15.7
Lowest	17	3.7	43	11.2	36	9.6	41	10.6
Highest + medium levels	418	91.7	316	82.1	323	85.9	334	86.1
All levels of certainty	435	95.3	359	93.3	359	95.5	375	96.6
No TBI documentation	19	4.2	26	6.8	17	4.5	13	3.4

*Intracranial injury ICD-10-CM codes began with ‘S06’ and ‘A’ in seventh character of the code or missing.

†The proportion of highest certainty of TBI is statistically significant by study site. Χ²=44.8266, p<0.0001.

‡Nine of the 385 sampled medical records in Colorado indicated that the patient was not a state resident, was not discharged home, did not have an injury event or was admitted as an inpatient and is not included in this table.

ED, emergency department; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; TBI, traumatic brain injury.

Table 5 presents signs, symptoms and imaging results for the skull fracture sample. The proportion of the skull fracture sample with documented loss of consciousness or any Group one or any Group two signs or symptoms was lower compared with the intracranial injury sample. The most common Group two symptom was headache/pressure in the head. Almost all skull fracture records in Maryland, Kentucky and Colorado study sites indicated imaging and the proportion with imaging findings positive for TBI varied: 23.1%, 34.5%, and 64.8% respectively. In Massachusetts, 83.8% had imaging, and 72.5% of the 388 records had positive findings.

Table 5.

TBI-related signs and symptoms and imaging findings among a random sample of medical records assigned skull fracture ICD-10-CM codes*, by study site

Study sites	One shock trauma, one ED in referral centres (Maryland) n=428		Two EDs in university system (Kentucky) n=385		EDs in level I, II, III trauma hospitals (Colorado) n=368‡		EDs in all acute care hospitals (Massachusetts) n=388
Known or suspected signs and/or symptoms	Count	%	Count	%	Count	%	Count	%
In group 1
Dazed/foggy/confused	48	11.2	69	17.9	38	10.3	22	5.7
Memory problems	40	9.3	59	15.3	49	13.3	15	3.9
Loss of consciousness	154	35.9	105	27.3	103	28.0	50	12.9
Any group 1 sign/symptom	173	40.4	140	36.4	128	34.8	72	18.6
No group one signs/symptoms	222	51.8	245	63.6	240	65.2	316	80.9
Total	395	92.3	385	100.0	368	100.0	388	100.0
In group 2§
Nausea or vomiting	32	7.5	42	10.9	51	13.9	41	10.6
Headache/pressure in head	102	23.8	118	30.6	115	31.3	211	54.4
Dizziness/poor balance	23	5.4	18	4.7	31	8.4	32	8.2
Change in vision	56	13.1	38	9.9	45	12.2	66	17.0
Irritable/Change in mood	17	4.0	11	2.9	20	5.4	†	†
Drowsiness/change in sleep	15	3.5	29	7.5	13	3.5	14	3.6
Imaging of the head
Imaging performed	406	94.9	345	89.6	353	95.9	325	83.8
Positive imaging findings	99	23.1	133	34.5	228	64.8	235	72.5

The bold values indicate that those two characteristics were exclusive (meaning a patient could not have both) and their total counts or percentages were shown in the "total" row below them.

*Skull fracture ICD-10-CM diagnosis codes beginning with ‘S02.0’, ‘S02.1’, ‘S02.8’ or ‘S02.91’ in any diagnosis field and seventh character of ‘A’ ‘B’ or missing. The billing record could not also have an intracranial injury code beginning with ‘S06’.

†Suppressed because the count was less than 11.

‡Sensitivity to noise/light, speech problems, hearing problems and weakness/numbness were documented in 0.5% to 3.0% of the skull fracture sample in the study sites, except the Maryland site had 17 records (4.2%) with speech problems and Massachusetts had 31 records (8.0%) with weakness.

§Seventeen of the 385 sampled medical records in Colorado indicated that the patient was not a state resident, was not discharged home, did not have an injury event or was admitted as an inpatient.

ED, emergency department; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; TBI, traumatic brain injury.

Based on the highest certainty of TBI (table 6), PPVs for the sample selected using ICD-10-CM codes for skull fracture ranged from 54.2% (95% CI: 49.4% to 59.0%) for Maryland to 77.7% (95% CI: 73.1% to 81.9%) in Colorado. Variability in PPV across study sites remained when PPV was expanded to include medium certainty of TBI, with a PPV of 57.0% in Maryland, 61.0% in Kentucky, 84.5% in Colorado and 82.0% in Massachusetts. Based on a complete record review assessment, the Kentucky clinical reviewers determined that 106 (48.4%) patients had a TBI or probable TBI out of the 219 records with a high certainty of TBI based on the study algorithm (data not shown), and 5 (31.3%) patients had a TBI or probable TBI out of the 16 records with a medium certainty of TBI.

Table 6.

PPV for TBI based on a review of ED records with skull fracture ICD-10-CM codes*, by study site

Study sites	One shock trauma, one ED in referral centres (Maryland) n=428		Two EDs in university system (Kentucky) n=385		EDs in level I, II, III trauma hospitals (Colorado) n=368*		EDs in all acute care hospitals (Massachusetts) n=388
Level of certainty of TBI	Count	%	Count	%	Count	%	Count	%
Highest‡	232	54.2	219	56.9	286	77.7	259	66.8
Medium	12	2.8	16	4.2	25	6.8	59	15.2
Lowest	47	11.0	41	10.7	21	5.7	40	10.3
Highest + medium levels	244	57.0	235	61.0	311	84.5	318	82.0
All levels of certainty	291	68.0	276	71.7	332	90.2	358	92.3
No TBI documentation	111	25.9	109	28.3	36	9.8	30	7.7

*Skull fracture ICD-10-CM diagnosis codes beginning with ‘S02.0’, ‘S02.1’, ‘S02.8’ or ‘S02.91’ in any diagnosis field and seventh character of ‘A’ ‘B’ or missing. The billing record could not also have an intracranial injury code beginning with ‘S06’.

†The proportion of highest certainty of TBI is statistically significant by study site. Χ²=57.0761, p<0.0001.

‡Seventeen of the 385 sampled medical records in Colorado indicated that the patient was not a state resident, was not discharged home, did not have an injury event or was admitted as an inpatient and are excluded.

ED, emergency department; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; TBI, traumatic brain injury.

Discussion

Surveillance strategies that yield accurate findings support the development of sound public health policy and effective allocation of resources. In this multisite study, the PPVs of intracranial injury ICD-10-CM codes to identify TBI demonstrate an acceptable level of accuracy to justify using these codes for public health surveillance. The TBI PPV of intracranial injury codes stabilised at 82%–92% across the four sites when PPV included both high and medium evidence of TBI. TBIs treated only in the ED are likely mild TBIs, which justify calculating PPV based on signs, symptoms and imaging findings representing high and medium certainty of TBI.

A traumatic skull fracture without an intracranial injury is not a clinical TBI. However, public health surveillance of TBI included skull fracture diagnoses only, due to the proportion that represent missed TBI and to limit the risk of under-reporting TBI.^{1 7} This multisite study found more than half of records with a skull fracture code but no code for traumatic intracranial injury had evidence of a TBI with high certainty. The skull fracture records from the two study sites that used clinical reviewers had lower PPVs for TBI and had a larger proportion without any evidence of TBI, compared with results from the two sites that used professional coders. These multisite findings suggest that the untested algorithm used retrospectively to assign certainty of TBI may yield an excessive number of false-positive cases. The additional information from one site on clinical impression further supports that the algorithm or its application identified a large proportion of false-positive TBIs. In developing the algorithm, the study team sought feedback from fewer clinicians than public health practitioners on the level of certainty method and the criteria for positive findings from imaging the head (details are in the online supplemental file 1). As a result, the analytical approach towards a broad public health definition prevailed based on previous public health surveillance, as indicated by the algorithm categorising positive findings of skull fractures from head imaging as a TBI.¹

This study was limited by the absence of a validated method for retrospective categorisation of transient signs, symptoms and imaging into levels of certainty that a TBI has occurred, reflecting the clinically challenging nature of TBI’s presentation and functional deficits.^{2 7} Our method assigned more weight (‘high certainty’ of TBI) to any of three signs and symptoms, two of which (loss of consciousness and memory problems) had a higher prevalence in the intracranial injury samples in the two study sites with university hospitals as referral centres and clinical reviewers. Without an objective diagnostic aid for TBI and a validated consensus clinical definition, the nature and extent of bias (false positive or false negative TBIs) are unknown. Our study highlights the need for further validation and testing of the algorithm to improve the methodology for medical record case confirmation studies on TBI.

This study had limited scope. The study assessed only PPV and not NPV, sensitivity and specificity to fully validate the proposed ICD-10-CM codes for identifying TBI. The findings in a related review of medical records for ED visits with ICD-10-CM diagnosis codes for unspecified head injury suggest that there are missed cases of TBI.⁸

This study did not assess all potential threats to validity of the results, for example, clinical documentation practices and the influence of medical or electronic health record software on documentation across the four study areas. The two-hospital study sites with more uniform electronic health records might have more consistent clinical documentation. The impact of record type inconsistency on the validity of documentation or diagnosis coding is unknown. The reviewer type and access to medical information varied across areas: the clinical reviewers in two study areas accessed electronic health records, while a professional coder in another study area had access to electronic health records or scanned copies of medical records, and professional coders in the fourth area accessed copies of medical records. The coders in the latter two areas did not always receive all the medical documents requested, notably the emergency medical services pre-hospital report. This study did not assess ICD-10-CM coding training or hospital coding practices.

Differences among study settings justified not pooling study results. Results across the four study areas must be compared with caution, considering the known differences in the study areas (table 2). However, knowing the range of estimated TBI PPV of ICD-10-CM codes in a variety of ED settings can be a strength. For example, regardless of whether the study reviewers were clinical or professional coders, the reviewers found documentation consistent with a TBI in the records with ICD-10-CM codes for intracranial injury.

This study has implications for public health practice. This study found that the ICD-10-CM codes for intracranial injury identified TBIs in the four samples at an acceptable level of accuracy (PPV 82%–92%), despite differences in study sites and study limitations. The PPV for the ICD-10-CM skull fracture codes with no concurrent intracranial injury code was lower than that for the sample with intracranial injury codes. One might expect the TBI PPV of the skull fracture codes to be even lower, if the original medical coders assigned accurately all the ICD-10-CM diagnosis codes to reflect all the physician documentation in the medical record. This finding suggests that before using the ICD-10-CM codes for skull fractures, applied epidemiologists will want to consider the number and type of hospital EDs represented in their ED billing dataset and select a similar study site, if comparable. The PPV finding for skull fracture or the lack of a comparable study site can inform the epidemiologist’s decision to use skull fracture codes to identify TBI in their ED billing dataset and to better interpret results of any skull fracture analysis. Further research can address the limitations of this study, especially validating a method for retrospective confirmation of a true TBI and assessing the sensitivity and specificity of the CDC’s proposed TBI ICD-10-CM codes. Public health practice would benefit from such research.

What is already known on the subject.

• On 1 October 2015, the USA implemented the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM).

• Before ICD-10-CM coded data were widely available in the USA, the CDC proposed in 2016 a set of ICD-10-CM codes for population surveillance of traumatic brain injury that included codes for traumatic intracranial injury and skull fracture.

What this study adds.

• In study sites in four states, more than two-thirds of randomly selected emergency department records with ICD-10-CM diagnosis code(s) for traumatic intracranial injury included documentation of (1) loss of consciousness, confusion or memory problems after an injury event; (2) a positive finding from imaging of the head; or (3) three or more signs or symptoms suggestive of a traumatic brain injury. This definition of ‘high’ certainty of TBI resulted in PPVs of TBI for intracranial injury codes ranging from 70% to 89%.

• The PPVs of the ICD-10-CM diagnosis code(s) for traumatic intracranial injury ranged from 82% to 92% in samples of emergency department records from study sites in four states, based on high certainty and medium certainty (fewer signs or symptoms or imaging results indicated suspected TBI).

• In study sites in four states, about a third of randomly selected emergency department records with ICD-10-CM diagnosis code(s) for skull fracture (without a code for traumatic intracranial injury) documented loss of consciousness, confusion or memory problems after an injury event.

• The PPVs of the ICD-10-CM codes for skull fracture (without a code for traumatic intracranial injury), based on documented evidence for medium-to-high certainty of TBI, was 57% and 61% in two sites with clinical reviewers in university/trauma hospitals and 82% and 85% for two study sites with professional medical coders reviewing statewide or nearly statewide samples. These findings suggest the untested algorithm used retrospectively to assign certainty of TBI may yield an excessive number of false-positive cases.