Validation of International Classification of Disease-10 Code for Identifying Children Hospitalized With Coronavirus Disease-2019

To the Editor: In December 2019, a cluster of cryptogenic severe pneumonia cases occurred in Wuhan, China. The etiology was identified as severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) in January 2020, and the resultant illness was termed coronavirus disease of 2019 (COVID-19) [1]. SARS-CoV-2 has since spread globally with >42,500,000 cases and >1,100,000 deaths by 10/24/2020 [2]. Most of the published clinical studies on COVID-19 have been on small cohorts. Large administrative datasets will likely be used to more comprehensively describe the epidemiology and outcomes of patients hospitalized with COVID-19. An emergency International Classification of Disease 10 (ICD-10) code, “U07.1 COVID-19, virus identified,” was introduced on April 1, 2020, and was expanded to include “multisystem inflammatory syndrome in children (MIS-C)” under the same code on May 14, 2020 [3]. However, the accuracy of this ICD-10 code in identifying patients with COVID-19 is unknown. In a small, early study of military data, only 20.3% of known COVID-19 cases were assigned this ICD-10 code [4].

We performed a single-center validation study of pediatric inpatients. Some data from this center have previously been reported [5]. Universal admission screening for SARS-CoV-2 began at our institution on April 1, 2020. We considered a patient to have COVID-19 if he/she tested positive for SARS-CoV-2 by polymerase chain reaction (PCR) or had a clinical diagnosis of MIS-C. A clinical diagnosis of MIS-C was made if a patient met Centers for Disease Control (CDC) diagnostic criteria [6] and a multidisciplinary group of physicians decided this was the most likely diagnosis. After obtaining Institutional Review Board (IRB) approval at our academic, tertiary pediatric medical center, the institution’s electronic health record (EHR) was queried for all admitted patients tested for SARS-CoV-2 by a PCR-based assay on a nasopharyngeal or endotracheal aspirate or were assigned a primary or secondary diagnosis code of “U07.1” from April 1, 2020, to April 8, 2020. Any patient with a diagnosis of MIS-C from May 14, 2020, to August 1, 2020, was added if they were included on a comprehensive list maintained by our institution’s MIS-C team (Supplementary Table 1). Only initial hospital admission encounters were included for each unique subject. Subjects were classified as “U07.1 positive” if they were assigned an ICD-10 code of U07.1 or as “U07.1 negative” if that code was not assigned. Student’s t-test and Chi-square test were used to compare the demographic characteristics of subjects who were COVID-19 positive vs COVID-19 negative. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the U07.1 code were assessed to identify the true-positive (ie, PCR positive and/or clinical diagnosis of MIS-C) and true-negative (ie, PCR negative and/or no MIS-C diagnosis) subjects. Receiver operating characteristic curve (ROC) analysis was performed. EHR review of all COVID-19-positive patients and U07.1-positive patients was performed to confirm results.

A query of the EHR on August 21, 2020, revealed 32,907 patient encounters during the study period. After applying the above criteria, 3905 inpatients were included; 117 (3.0%) were COVID-19 positive and 110 (2.8%) were U07.1 positive. Five patients were U07.1 positive but negative for SARS-CoV-2 by PCR and did not have an MIS-C diagnosis. Of the 117 positive for COVID-19, 21 were diagnosed with MIS-C (10 were PCR positive; others were diagnosed by clinical criteria). For COVID-19-positive and COVID-19-negative subjects, demographic characteristics are provided in Table 1.

Table 1.

Descriptive Statistics Comparing COVID-19-Negative and COVID-19-Positive Patients^a

	Category	COVID-19 Negative	COVID-19 Positive
Study population, n (%)		3788	117
Sex	Female	1879 (49.6%)	54 (46.2%)
	Male	1907 (50.3%)	62 (53.0%)
	Unknown	2 (0.1%)	1 (0.9%)
Age in years, mean (SD)**		8.3 (7.6)	10.7 (12.4)
Race*	Black/African American	986 (26.0%)	51 (43.6%)
	White	1907 (50.4%)	38 (32.5%)
	Other	894 (23.6%)	28 (23.9%)
Ethnicity*	Not Hispanic/Latino	3339 (88.1%)	93 (79.5%)
	Hispanic/Latino	449 (11.9%)	24 (20.5%)
Insurance**	Commercial	1980 (52.3%)	33 (28.2%)
	Government	1404 (37.1%)	69 (59.0%)
	Other/not available	404 (10.7%)	15 (12.8%)

Abbreviations: COVID-19, coronavirus disease 2019; MIS-C, multisystem inflammatory syndrome in children; PCR, polymerase chain reaction; SD, standard deviation.

^aA patient is defined as COVID-19 positive if he/she had a positive PCR test for SARS-CoV-2 and/or met the clinical criteria for MIS-C.

*P < .05, **P < .001.

The sensitivity of the U07.1 ICD-10 code for identifying those with a positive SARS-CoV-2 PCR test was 90.5% (95% confidence interval [CI]: 82.8%-94.6%; Table 2). The specificity of the U07.1 ICD-10 code was 99.9% (95% CI: 99.7%-100%). Together, these yielded an area under the ROC curve of 0.952 (95% CI: 0.920%0.976%). The PPV of the U07.1 code was 96.3% (95% CI: 89.7%-98.5%). The NPV was 99.7% (95% CI: 99.4%-99.8%).

Table 2.

A 2 × 2 Table Showing Epidemiologic Calculations for U07.1 as a Predictor for Disease Due to COVID-19

		COVID-19			Calculations (95% CI)
		(+)	(–)		Sensitivity	89.7% (82.8–94.6%)
U07.1	(+)	105	5	110	Specificity	99.9% (99.7–100%)
	(–)	12	3783	3795	PPV	95.5% (89.7–98.5%)
		117	3788	3905	NPV	99.7% (99.4–99.8%)
					Area under ROC	0.948 (0.920–0.976)

Abbreviations: NPV, negative predictive value; PPV, positive predictive value.

In conclusion, ICD-10-based identification demonstrated high sensitivity and specificity for COVID-19 patients, which together yield an excellent area under the ROC curve. Our results may underestimate the accuracy of the U07.1 code in future studies because it may be more precisely used over time. Limitations of our study include that it was a single-center study at a US children’s hospital, and thus the generalizability of our findings to other settings and populations, especially with our use of universal admission screening, is unknown. Another limitation is that patients identified as false positives may have been tested for SARS-CoV-2 at an outside institution, though this would have been rare since our institution is the primary regional pediatric referral center. Our results suggest that the U07.1 ICD-10 code may be a valid means of identifying patients with COVID-19 in administrative databases. Further studies are needed to confirm these results.

Notes

Acknowledgments. The authors thank the entire staff of the Children’s Hospital of Philadelphia for their ongoing compassionate care during the COVID-19 pandemic.

Financial support. This work was supported, in part, by a National Institute of General Medical Science T32 grant to A. M. B. (T32 GM75766-13) and a Eunice Kennedy Shriver National Institute of Child Health and Human Development T32 grant to W. R. O. (NIH T32-HD043021).

Potential conflicts of interest. A. M. B., W. R. O., X. L., J. S. G.: No reported conflicts of interest. M. Z. D.: served as a consultant for GSK and Baxter. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.