Abstract
International Classification of Diseases diagnostic codes are used to estimate acute gastroenteritis (AGE) disease burden. We validated AGE-related codes in pediatric and adult populations using 2 multiregional active surveillance platforms. The sensitivity of AGE codes was similar (54% and 58%) in both populations and increased with addition of vomiting-specific codes.
Keywords: surveillance, validation, acute gastroenteritis, methods
Population-based studies estimate that approximately 179 million cases of acute gastroenteritis (AGE) occur each year in the United States, resulting in approximately 600 000 hospitalizations and 5000–11 000 deaths [1–3]. These estimates of the numbers of hospitalizations and deaths are based on International Classification of Diseases, Clinical Modification (ICD-CM), codes from AGE-associated medical encounters and death certificates. ICD-CM codes have also been used to estimate AGE-associated medical encounters and deaths due to specific AGE-causing pathogens, such as norovirus and rotavirus [1, 3–9].
Previous studies estimating AGE burden have used a similar set of AGE-related ICD-CM codes [1, 3, 5]. This AGE code set did not include ICD-CM codes for isolated vomiting. To guide improved disease-burden estimates, we assessed both the validity of previously used AGE ICD-CM codes, and the validity of those same codes with the addition of vomiting-specific AGE ICD-CM codes, using 2 different multiregional AGE active surveillance platforms.
METHODS
Study Populations
The New Vaccine Surveillance Network (NVSN) included 7 participating pediatric hospitals: The Children’s Mercy Hospital (Kansas City, Missouri), University of California at San Francisco Benioff Children’s Hospital (Oakland, California), Texas Children’s Hospital (Houston, Texas), Seattle Children’s Hospital (Seattle, Washington), Cincinnati Children’s Hospital Medical Center (Cincinnati, Ohio), Vanderbilt University Medical Center (Nashville, Tennessee), and the University of Rochester Medical Center (Rochester, New York) and was described previously [10, 11]. An AGE case was defined as a patient younger than 5 years old enrolled in the hospital or emergency department (ED) with 3 or more episodes of loose stool or 1 or more episode of vomiting within a 24-hour period, per caregiver report. Cases with AGE enrolled from 1 December 2011 to 30 June 2016 were included.
The Surveillance Platform for Enteric and Respiratory Infectious Organisms in the Veterans Affairs (VA) population (SUPERNOVA) included 5 participating sites: Atlanta VA Medical Center (Atlanta, Georgia), James J. Peters VA Medical Center (Bronx, New York), VA Greater Los Angeles Health System (Los Angeles, California), the Michael E. DeBakey VA Medical Center (Houston, Texas), and the VA Palo Alto Health Care System (Palo Alto, California) and was described previously [12–14]. Due to the time frame of the study, the fifth site was not included in our analyses. An AGE case was defined as a patient aged 18 years or older enrolled in the hospital or ED with 3 or more episodes of diarrhea, 2 or more episodes of vomiting, or 1 or more episode of both diarrhea and vomiting in a 24-hour period. Patients with AGE hospitalized from 1 December 2016 to 28 March 2018, or admitted to the ED only (single site) from 1 December 2016 to 30 September 2017, were included. Age- and time-matched hospitalized veteran controls without AGE were also enrolled.
Data Collection and ICD-CM Codes
ICD-CM codes and demographic and clinical data were obtained for all study participants. We abstracted the first 10 ICD-CM codes from NVSN encounters and the first 25 ICD-CM codes from SUPERNOVA encounters. The AGE ICD-CM code set included ICD-CM codes classified as cause-unspecified or cause-specified AGE using a previously described set of ICD, revision 9 (ICD-9), and ICD, revision 10 (ICD-10), codes [1, 3, 4]. The revised AGE ICD-CM code set included vomiting codes in addition to the previously described AGE ICD-CM code set (Supplementary Table S1)
Statistical Analyses
The sensitivity of AGE codes among cases with AGE and, in SUPERNOVA, their specificity among controls without AGE were calculated with exact (Clopper-Pearson) 95% confidence intervals (CIs). Multivariable logistic regression analyses were conducted to identify predictors of accurate AGE coding. An a priori decision was made to include sex, race, and ethnicity in all models. Site, healthcare setting, presence of vomiting alone (no diarrhea) during illness, presence of severe AGE (defined as modified Vesikari score ≥11, NVSN only [15]), and the presence of immunosuppression (SUPERNOVA only) were examined for model inclusion. Final models were obtained using backward selection (criteria for removal = P > .05). Statistical analyses were conducted using SAS version 9.4 (SAS Institute, Inc).
Institutional review board approvals were obtained from the Centers for Disease Control and Prevention and from each study site.
RESULTS
New Vaccine Surveillance Network
ICD-CM codes were collected from 13 916 enrolled children (Table 1). Of these children, 46% were female, and the median age was 18 months. The majority of children were enrolled in the ED (80%) and 30% presented with vomiting only (no diarrhea).
Table 1. .
Characteristics of Patients With Acute Gastroenteritis (AGE) and Associations With Accurate AGE Coding and Sensitivity Estimates, by Surveillance Platform
| Demographics | NVSN (N = 13 916a) | SUPERNOVA (N = 635a) | ||||
|---|---|---|---|---|---|---|
| With AGE Code (n = 7572), n (%) | Without AGE Code (n = 6344), n (%) | aORb (95% CI) | With AGE Code (n = 368), n (%) | Without AGE Code (n = 267), n (%) | aORc (95% CI) | |
| Male | 4166 (55.0) | 3325 (52.4) | … | 340 (92.4) | 243 (91.0) | … |
| Female | 3406 (45.0) | 3019 (47.6) | … | 28 (7.6) | 24 (9.0) | … |
| Median aged (25th IQR, 75th IQR) | 18 (9, 31) | 18 (9, 33) | 1.01 (1.01, 1.02) | 63 (53, 70) | 64 (55, 71) | … |
| Ethnicity | n = 7563 | n = 6337 | n = 367 | n = 267 | ||
| Non-Hispanic | 4468 (59.1) | 4299 (67.8) | … | 324 (88.3) | 221 (82.8) | … |
| Hispanic | 3095 (40.9) | 2038 (32.2) | … | 43 (11.7) | 46 (17.2) | … |
| Race | n = 6374 | n = 5935 | n = 358 | n = 257 | ||
| White | 3148 (46.8) | 2606 (43.9) | … | 182 (50.8) | 119 (46.3) | … |
| Black | 2474 (36.7) | 2381 (40.1) | … | 163 (45.5) | 126 (49.0) | … |
| Asian/NH/PI | 183 (2.7) | 121 (2.0) | … | 2 (0.6) | 1 (0.4) | … |
| AI/AN | 14 (0.2) | 11 (0.2) | … | 1 (0.3) | 1 (0.4) | … |
| Other | 915 (13.6) | 816 (13.8) | … | 10 (2.8) | 10 (3.9) | … |
| Setting | ||||||
| Inpatient | 1692 (22.4) | 1162 (18.3 | Ref | 322 (87.5) | 252 (94.4) | Ref |
| ED | 5880 (77.7) | 5182 (81.7) | 0.81 (0.73, 0.90) | 46 (12.5) | 15 (5.6) | 2.24 (1.16, 4.32) |
| AGE symptoms documented | n = 7564 | n = 6324 | ||||
| Diarrhea with or without vomiting | … | … | Ref | 357 (97.0) | 228 (86.0) | Ref |
| Vomiting alone | 763 (10.1) | 3408 (53.9) | 0.08 (0.07, 0.09) | 11 (3.0) | 37 (14.0) | 0.21 (0.10, 0.42) |
| Immunocompromising condition presente | NA | NA | … | 85 (23.1) | 53(19.9) | … |
| Severe AGEf | n = 6621 | n = 5658 | NA | NA | ||
| Yes | 3508 (53.0) | 1796 (31.8) | … | NA | NA | … |
| Siteg | ||||||
| 1 | 1020 (13.5) | 1389 (21.9) | Ref | 72 (19.6) | 32 (12.0) | Ref |
| 2 | 360 (4.8) | 754 (11.9) | 0.58 (0.48, 0.69) | 26 (7.1) | 38 (14.2) | 0.30 (0.15, 0.61) |
| 3 | 1190 (15.7) | 1309 (20.6) | 1.31 (1.14, 1.51) | 221 (60.1) | 129 (48.3) | 0.68 (0.41, 1.12) |
| 4 | 331 (4.4) | 220 (3.5) | 2.37 (1.85, 3.00) | 49 (13.3) | 68 (25.5) | 0.35 (0.20, 0.63) |
| 5 | 1855 (24.5) | 943 (14.9) | 2.75 (2.40, 3.15) | … | … | … |
| 6 | 1564 (20.7) | 1247 (19.7) | 1.88 (1.63, 2.17) | … | … | … |
| 7 | 1252 (16.5) | 482 (7.6) | 4.08 (3.36, 5.00) | … | … | … |
| Surveillance yearh | ||||||
| 2011 | 1225 (16.2) | 1105 (17.4) | Ref | NA | NA | … |
| 2012 | 1803 (23.8) | 1437 (22.7) | 1.05 (0.91, 1.20) | NA | NA | … |
| 2013 | 1822 (24.1) | 1496 (23.6) | 1.27 (1.10, 1.45) | NA | NA | … |
| 2014 | 1916 (25.3) | 1471 (23.2) | 1.26 (1.09, 1.44) | NA | NA | … |
| 2015 (incomplete) | 806 (10.6) | 835 (13.2) | 1.11 (0.94, 1.30) | NA | NA | … |
| 2016 | NA | NA | … | 306 (83.2) | 259 (63.3) | … |
| 2017 (incomplete) | NA | NA | … | 62 (16.9) | 150 (36.7) | … |
All crude ORs and year-specific aORs are provided in Supplementary Table 4.Abbrevations: AGE, acute gastroenteritis; AI/AN, American Indian/Alaska Native; aOR, adjusted odds ratio; CI, confidence interval; ED, emergency department; IQR, interquartile range; NA, not applicable; NH/PI, Native Hawaiian/Pacific Islander; NVSN, New Vaccine Surveillance Network; Ref, reference; SUPERNOVA, Surveillance Platform for Enteric and Respiratory Infections in the Veterans Affairs Population.
aUnless otherwise specified.
bNVSN model adjusted for age, sex, race, ethnicity, year, site, healthcare setting (inpatient vs ED visit), presence of vomiting without diarrhea, and severe AGE.
cSUPERNOVA model adjusted for age, sex, race, ethnicity, site, healthcare setting, and presence of vomiting without diarrhea.
dAge in months for NVSN and age in years for SUPERNOVA.
eImmunocompromising condition present defined as organ transplant, cancer, HIV, and immunosuppressive therapy
fSevere AGE defined as modified Vesikari score ≥11 [15].
gFour SUPERNOVA sites were included in the study: Atlanta VA Medical Center, James J. Peters VA Medical Center, VA Greater Los Angeles Health System (Los Angeles, California), and the Michael E. DeBakey VA Medical Center (Houston, Texas).
hSurveillance year is 1 December–30 November for NVSN and 1 December–30 September for SUPERNOVA.
The sensitivity of the AGE code set was 54% (7572 of 13 916; 95% CI, 54–55%) but varied considerably across sites (32–72%) (Table 2). When the AGE with vomiting code set was used, the sensitivity of all-cause AGE codes was 75% (10 484 of 13 916; 95% CI, 75–76%). Sensitivity estimates were higher in the inpatient as compared with the ED setting (59% vs 53%, respectively; chi-square test, P < .0001). Of the encounters with an AGE ICD code present, the most common type of AGE codes was unspecified AGE (95%) (Supplementary Table S2).
Table 2.
Sensitivity Estimates of Acute Gastroenteritis ICD-CM Coding
| NVSN, n/N (%; 95% CI) | SUPERNOVA, n/N (%; 95% CI) | |
| ICD code grouping | ||
| All-cause AGE | 7572/13 916 (54.4; 53.6, 55.2) | 368/635 (58.0; 54.0, 61.8) |
| All-cause AGE with vomitinga | 10 484/13 916 (75.3; 74.6, 76.1) | 386/635 (60.8; 56.9, 64.6) |
| Setting | ||
| Inpatient | 1692/2854 (59.3; 57.5, 61.1) | 322/574 (56.1; 51.9, 60.2) |
| ED | 5880/11 062 (53.2; 52.2, 54.1) | 46/61 (75.4; 62.7, 85.5) |
| Site | ||
| 1 | 1020/2409 (42.3; 40.4, 44.3) | 72/104 (69.2; 59.4, 77.9) |
| 2 | 360/1114 (32.3; 29.6, 35.2) | 26/64 (40.6; 28.5, 53.6) |
| 3 | 1190/2499 (47.6; 45.6, 49.6) | 221/350 (63.1; 57.9, 68.2) |
| 4 | 331/551 (60.1; 55.8, 64.2) | 49/117 (41.9; 32.8, 51.4) |
| 5 | 1855/2798 (66.3; 64.5, 68.0) | … |
| 6 | 1564/2811 (55.6; 53.8, 57.5) | … |
| 7 | 1252/1734 (72.2; 70.0, 74.3) | … |
Abbrevations: AGE, acute gastroenteritis; ED, emergency department; ICD-CM, International Classification of Diseases, Clinical Modification; NVSN, New Vaccine Surveillance Network; SUPERNOVA, Surveillance Platform for Enteric and Respiratory Infections in the Veterans Affairs Population.
aRevised all-cause AGE includes vomiting-specific codes.
In regression analyses, clinical presentation with vomiting alone during illness significantly decreased the likelihood of receiving an AGE code (adjusted odds ratio [aOR], 0.08; 95% CI, 0.07–0.09). An ED setting also independently decreased the likelihood that an AGE code would be present upon discharge (aOR, 0.81; 95% CI, 0.73–0.90) (Table 1, Supplementary Table S3).
SUPERNOVA
ICD-CM codes were collected from 777 enrolled adult veterans (635 cases with AGE and 142 controls) (Supplementary Table S4). Among adults enrolled as cases, 8% were female and the median age was 63 years. Only 1 site enrolled in the ED setting, accounting for 10% of all enrolled veterans included in the study. Eight percent presented with vomiting only (no diarrhea) during AGE illness.
The sensitivity of the AGE code set was 58% (368 of 635; 95% CI, 54–62%), ranging across sites from 41% to 69%, with a specificity of 100% (142 of 142). Of the encounters with an AGE ICD code present, the most common type of AGE codes were unspecified AGE (58%). Sensitivity and specificity estimates were similar when using the AGE with vomiting code set (61% [95% CI, 57–65%] and 100% [95% CI, 96–100%]). For the site that enrolled ED patients, sensitivity estimates were higher in the ED setting as compared with the inpatient setting (75.4% vs 56.1%, respectively; chi-square test, P = .005.)
In regression analyses, clinical presentation with vomiting only independently decreased the likelihood that an AGE code would be present upon discharge (aOR, 0.21; 95% CI, 0.10–0.42), and ED setting increased the likelihood that an AGE code would be present upon discharge (aOR, 2.24; 95% CI, 1.16–4.32).
DISCUSSION
Based on these 2 regionally diverse active surveillance platforms, AGE codes were relatively insensitive in identifying children or adults with AGE; our analyses estimate a sensitivity of 54% in the pediatric population and 58% in the adult population. The AGE codes were extremely specific in the adult population (100%). These findings suggest that US AGE disease-burden estimates that rely on ICD-CM codes to identify cases with AGE likely underestimate the true number of cases with AGE in both the pediatric and adult populations and are consistent with previous studies validating cause-specific AGE ICD-CM codes [9, 16].
In the pediatric population, the addition of the vomiting unspecified ICD-CM code to the AGE code set improved its sensitivity (sensitivity estimate increased from 54% to 75%), indicating that using the AGE with vomiting code set to identify children hospitalized with AGE would capture more cases with AGE. This was not surprising, as the case definition for AGE included 1 or more vomiting episodes, and 30% of children presented with vomiting only. Because the NVSN cohort in this analysis did not include controls, and thus specificity could not be calculated, it is unclear how often AGE codes are inappropriately assigned to pediatric patients without AGE.
Among adult veterans, the addition of the vomiting unspecified ICD-CM code to the AGE code set did not improve its sensitivity; however, this is likely due to the low proportion of adults hospitalized with vomiting alone (8%) and the more stringent AGE case definition. In adjusted analyses, the presence of vomiting alone decreased the likelihood of AGE coding by 83%.
Our study is subject to a number of limitations. Enrolled subjects were seen in either the network facilities’ ED or hospital; the validity of AGE codes in the primary care setting and from death certificates was not examined. Furthermore, in the pediatric population, up to 10 ICD-CM codes were abstracted, but AGE codes may have been missed if present after the 10th code. However, only 1.3% (179 of 13 916) of pediatric subjects had 10 ICD-CM codes associated with their medical encounter; hence, the extraction of additional ICD-CM codes beyond the first 10 would minimally impact these findings. Finally, our adult population was from a VA network, which may not be generalizable to other adult populations [14] and the number of control subjects included to evaluate specificity of the AGE codes was small. Future studies should additionally assess the validity of AGE codes in adult nonveteran populations and the specificity of these codes in pediatric and adult population-based samples.
Conclusions
Across 2 multiregional AGE active surveillance platforms, one enrolling children and one enrolling adults, the estimated sensitivity of all-cause AGE codes was low. This suggests that current national estimates for AGE disease burden may be underestimating the true burden of AGE pathogens in the United States and emphasizes the importance of active, prospective surveillance. The presence of vomiting alone decreased the likelihood of identifying a cases with AGE in both the pediatric and adult populations. These findings should be considered when using AGE codes to estimate AGE disease burden and explored further.
Supplementary Data
Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Notes
Acknowledgments. The authors acknowledge the following individuals for their important contributions to this study: Parvin H. Azimi, University of California at San Francisco Benioff Children’s Hospital, Oakland, California; Leila C. Sahni, Texas Children’s Hospital, Houston, Texas; Christopher J. Harrison, The Children’s Mercy Hospital, Kansas City, Missouri; Mary Mofatt, The Children’s Mercy Hospital, Kansas City, Missouri; Peter G. Szilagyi, University of California, Los Angeles (UCLA), California; Blanca Vargas, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, Texas; Karen V. Evangelista, VA Greater Los Angeles Healthcare System and David Geffen School of Medicine at UCLA, Los Angeles, California; Kathryn L. Meagley, Atlanta VA Medical Center, Atlanta, Georgia; Adrienne Perea, James J. Peters VA Medical Center, Bronx, New York; Johane Simelane, James J. Peters VA Medical Center, Bronx, New York; and Guery A. Perez, James J. Peters VA Medical Center, Bronx, New York.
Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC).
Financial support. This work was supported by the CDC.
Potential conflicts of interest. J. A. E. reports research support from GlaxoSmithKline, Merck, Chimerix, MedImmune, and Novavax and personal fees from Sanofi Pasteur outside the submitted work. All other authors report no potential conflicts. The authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1. Scallan E, Griffin PM, Angulo FJ, Tauxe RV, Hoekstra RM. Foodborne illness acquired in the United States–unspecified agents. Emerg Infect Dis 2011; 17:16–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hall AJ, Curns AT, McDonald LC, Parashar UD, Lopman BA. The roles of Clostridium difficile and norovirus among gastroenteritis-associated deaths in the United States, 1999–2007. Clin Infect Dis 2012; 55:216–23. [DOI] [PubMed] [Google Scholar]
- 3. Lopman BA, Hall AJ, Curns AT, Parashar UD. Increasing rates of gastroenteritis hospital discharges in US adults and the contribution of norovirus, 1996–2007. Clin Infect Dis 2011; 52:466–74. [DOI] [PubMed] [Google Scholar]
- 4. Gastañaduy AS, Zabaleta J, Li L, Bégué RE. Norovirus as the cause of medically attended gastroenteritis: a hospital-based experience. Epidemiol Infect 2016; 144:2773–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Hall AJ, Lopman BA, Payne DC, et al. . Norovirus disease in the United States. Emerg Infect Dis 2013; 19:1198–205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rha B, Lopman BA, Alcala AN, Riddle MS, Porter CK. Incidence of norovirus-associated medical encounters among active duty United States military personnel and their dependents. PLoS One 2016; 11:e0148505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Cortes JE, Curns AT, Tate JE, et al. . Rotavirus vaccine and health care utilization for diarrhea in U.S. children. N Engl J Med 2011; 365:1108–17. [DOI] [PubMed] [Google Scholar]
- 8. Cortese MM, Tate JE, Simonsen L, Edelman L, Parashar UD. Reduction in gastroenteritis in United States children and correlation with early rotavirus vaccine uptake from national medical claims databases. Pediatr Infect Dis J 2010; 29:489–94. [DOI] [PubMed] [Google Scholar]
- 9. Hsu VP, Staat MA, Roberts N, et al. . Use of active surveillance to validate International Classification of Diseases code estimates of rotavirus hospitalizations in children. Pediatrics 2005; 115:78–82. [DOI] [PubMed] [Google Scholar]
- 10. Payne DC, Boom JA, Staat MA, et al. . Effectiveness of pentavalent and monovalent rotavirus vaccines in concurrent use among US children <5 years of age, 2009–2011. Clin Infect Dis 2013; 57:13–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Payne DC, Staat MA, Edwards KM, et al. . Active, population-based surveillance for severe rotavirus gastroenteritis in children in the United States. Pediatrics 2008; 122:1235–43. [DOI] [PubMed] [Google Scholar]
- 12. Cardemil CV, Kambhampati A, Grytdal S, et al. . Incidence of norovirus and rotavirus from multisite active surveillance in Veteran’s Affairs hospitals, December 2016–February 2018: results from the SUPERNOVA network. Open Forum Infect Dis 2018; 5(Suppl 1):S49. [Google Scholar]
- 13. Kambhampati AK, Vargas B, Mushtaq M, et al. . Active surveillance for norovirus in a US Veterans Affairs Patient Population, Houston, Texas, 2015–2016. Open Forum Infect Dis 2019; 6:ofz115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Grytdal S, Browne H, Collins N, et al. . Trends in incidence of norovirus-associated acute gastroenteritis in four Veterans Affairs medical center populations in the United States, 2011–2015. Clin Infect Dis 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Wikswo M, Payne D, Parashar U. Validation of a modified Vesikari score in US children with gastroenteritis. Poster presentation at the 11th International Rotavirus Symposium (New Delhi). 2014.
- 16. Scallan E, Griffin PM, McLean HQ, Mahon BE. Hospitalisations due to bacterial gastroenteritis: a comparison of surveillance and hospital discharge data. Epidemiol Infect 2018; 146:954–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.