Abstract
We performed a retrospective analysis of the changes in accuracy of International Classification of Diseases (ICD-CM) diagnosis codes for colectomy and hysterectomy surgical site infection surveillance. After the transition from ICD-CM 9th to 10th edition codes, there was no significant change in the accuracy of these codes for SSI surveillance.
INTRODUCTION
Surgical Site infections continue to remain a significant cause of patient morbidity and increased healthcare costs after operative procedures.1,2 To assist in SSI prevention several surveillance systems, including the National Healthcare Safety Network (NHSN), were developed to assist hospitals in the identification and reporting of these complications.3 However, these surveillance systems rely on labor-intensive manual chart review, which limits the generalizability and feasibility of this approach.
The use of administrative billing codes such as the International Classification of Diseases Clinical Modification (ICD-CM) diagnosis codes has been proposed as a tool to help assist in the surveillance of surgical site infections.4 Several studies have demonstrated a wide range for sensitivity (10-100%) and positive predictive value (PPV) (11-95%) of these codes depending on the procedures evaluated.4 Given the wide variability in the performance of ICD-CM codes between hospitals, their use for surveillance of SSIs has been limited. However, on September 30, 2015, the Centers for Medicare and Medicaid Services (CMS) transitioned from the ICD-CM 9th edition to the ICD-CM 10th edition codes.5 This change resulted in an increase from 13,000 9th edition codes to 68,000 10th edition codes.5 Several studies have examined the impact of this vocabulary expansion on several condition-specific diagnoses in administrative discharge data; however the effect on the accuracy of ICD-CM codes for the detection of SSIs is unknown.6 Therefore, the goal of the present study is to evaluate the change in accuracy of the ICD 10th edition codes compared to the ICD 9th edition codes for the detection of SSIs in a single healthcare system.
METHODS
The data in the present study were obtained from Intermountain Healthcare, a not-for-profit healthcare system located in the intermountain west. Patients were included in the present study if they underwent a National Healthcare Safety Network defined colectomy and abdominal hysterectomy procedures between January 1, 2012, and April 1, 2019 based on CMS defined denominator procedure codes.3 The reference standard was an NHSN defined surgical site infection, including superficial, deep, and organ/space, as identified by infection preventionists as part of mandatory hospital SSI reporting practices.7 For patients included in the study, we obtained all ICD diagnosis codes within 30 days of the operative procedure. We included diagnosis codes from both the index admission and any readmission that occurred within 30 days of the operative procedure. If a patient underwent a surgical procedure on or before September 30, 2015, ICD 9th edition diagnosis codes were utilized, and if the procedure occurred on or after October 1, 2015, ICD 10th edition codes were utilized. ICD codes were grouped into a narrow-range (using only the ICD code for postoperative infection) and expanded-range SSI category as previously described by the Pennsylvania Health Care Cost Containment Council (PHC4) and defined in the Supplementary Appendix.8
The performance of the narrow and expanded range ICD 9th and 10th edition codes were compared to the reference standard and evaluated for sensitivity, specificity, positive predictive value, negative predictive value. We utilized bootstrapping with 2000 iterations to generate confidence intervals for each point estimate. The analysis was performed using the R v4.0 Statistical Software Package.
RESULTS
A total of 16,714 procedures were included in the present study, 10,315 (62%) procedures utilizing ICD-9 codes, and 6,399 (38%) procedures utilizing ICD-10 codes. Colon procedures comprised 52% of the cohort and hysterectomy procedures accounted for the remainder 48%. There was a significantly higher prevalence of SSI in the ICD-9 cohort compared to the ICD-10 cohort (4.7% vs 3.6%, OR [95%CI]: 0.75 [0.64-0.88], p<0.001) and in the colectomy procedures compared to the hysterectomy procedures (6.2% vs. 2.1%, OR [95%CI]: 2.95 [2.48-3.51], p<0.001).
The classification performance for the ICD 9 and 10 codes is shown in Table 1. Overall there was a small decrease in the sensitivity of ICD 10 codes compared to ICD 9 codes using either a narrow (difference [95%CI]: −0.03 [−0.12-0.04]) or expanded (difference [95%CI]: −0.08 [−0.16-0.00]) code range. In addition, there were small increases in specificity of ICD 10 codes compared to ICD9 codes using either narrow (difference [95%CI]: 0.007 [0.002-0.01]) or expanded (difference [95%CI]: 0.005 [−0.002-0.01]) code range.
Table 1.
Classification Performance of International Classification of Diseases Clinical Modification 9th and 10th Edition Codes for Surveillance of Surgical Site Infections.
| Sensitivity |
Specificity |
||||||
|---|---|---|---|---|---|---|---|
| Procedure | ICD Range | ICD-9 (95% CI) | ICD-10 (95% CI) | Difference (95% CI) | ICD-9 (95% CI) | ICD-10 (95% CI) | Difference (95% CI) |
| Colectomy | Narrow | 0.45 (0.39-0.50) | 0.38 (0.31-0.452) | −0.06 (−0.15-0.03) | 0.97 (0.96-0.97) | 0.98 (0.98-0.99) | 0.02 (0.01-0.02) |
| Hysterectomy | Narrow | 0.50 (0.42-0.58) | 0.60 (0.45-0.74) | 0.10 (−0.10-0.30) | 0.99 (0.98-0.99) | 0.99 (0.98-0.99) | 0.002 (−0.004-0.008) |
| All | Narrow | 0.46 (0.42-0.51) | 0.43 (0.36-0.49) | −0.03 (−0.12-0.04) | 0.98 (0.97-0.98) | 0.98 (0.98-0.99) | 0.007 (0.002-0.01) |
|
| |||||||
| Colectomy | Expanded | 0.60 (0.54-0.65) | 0.49 (0.42-0.60) | −0.11 (−0.20- −0.02) | 0.91 (0.90-0.92) | 0.94 (0.93-0.95) | 0.03 (0.02-0.04) |
| Hysterectomy | Expanded | 0.64 (0.57-0.72) | 0.71 (0.57-0.83) | 0.07 (−0.11-0.26) | 0.97 (0.97-0.98) | 0.96 (0.96-0.97) | −0.008 (−0.2-0.0) |
| All | Expanded | 0.61 (0.57-0.65) | 0.53 (0.47-0.60) | −0.08 (−0.16-0.00) | 0.95 (0.94-0.95) | 0.95 (0.94-0.96) | 0.005 (−0.002-0.01) |
CI, Confidence Interval; ICD, International Classification of Diseases
On subgroup analysis, for colectomy procedures using the ICD-10 expanded code range we observed a small decrease in the sensitivity for SSI detection compared to the ICD9 expanded code range (difference [95% CI]: −0.11 [−0.2 - −0.02]). For hysterectomy procedures, observed a small increase sensitivity for SSI detection with ICD 10 codes compared to ICD 9 codes using an expanded code range (difference [95% CI]: 0.07 [−0.11-0.26]).
The predictive value of information for the ICD codes is shown in Table 2. There was small decreases in the PPV between the narrow range codes (difference [95% CI]: −0.01 [−0.09-0.08]) and the expanded range (difference [95%CI]: −0.07 [−0.12- −0.02])
Table 2.
Positive and Negative Predictive Values of International Classification of Diseases Clinical Modification 9th and 10th Edition Codes for Surveillance of Surgical Site Infections
| PPV |
NPV |
||||||
|---|---|---|---|---|---|---|---|
| Procedure | ICD Range | ICD-9 (95% CI) | ICD-10 (95% CI) | Difference (95% CI) | ICD-9 (95% CI) | ICD-10 (95% CI) | Difference (95% CI) |
| Colectomy | Narrow | 0.52 (0.47-0.57) | 0.50 (0.43-0.58) | −0.02 (−0.12-0.09) | 0.96 (0.95-0.96) | 0.97 (0.97-0.97) | 0.02 (−0.02- −0.007) |
| Hysterectomy | Narrow | 0.46 (0.39-0.53) | 0.47 (0.37-0.59) | 0.02 (−0.16-0.18) | 0.99 (0.99-0.99) | 0.99 (0.99-1.00) | 0.01 (−0.01-0.0) |
| All | Narrow | 0.50 (0.46-0.54) | 0.50 (0.44-0.55) | −0.01 (−0.09-0.08) | 0.97 (0.97-0.98) | 0.98 (0.98-0.98) | 0.01 (−0.01-0.0) |
|
| |||||||
| Colectomy | Expanded | 0.35 (0.32-0.38) | 0.29 (0.25-0.33) | −0.06 (−0.13-0.003) | 0.97 (0.96-0.97) | 0.97 (0.97-0.98) | 0.01 (−0.02- −0.001) |
| Hysterectomy | Expanded | 0.36 (0.32-0.41) | 0.26 (0.21-0.32) | −0.10 (−0.21-0.01) | 0.99 (0.99-0.99) | 1.00 (0.99-1.00) | −0.004 (−0.008-0.001) |
| All | Expanded | 0.35 (0.33-0.38) | 0.28 (0.25-0.32) | −0.07 (−0.13−-0.02) | 0.98 (0.98-0.98) | 0.98 (0.98-0.98) | −0.001 (−0.006-0.003) |
PPV, positive predictive value; NPV, negative predictive value, CI, confidence interval. ICD, International Classification of Diseases
DISCUSSION
In the study of colectomy and hysterectomy procedures in a single healthcare system, we identified several small but clinically questionable changes in the performance of ICD-CM 10th edition codes compared to ICD-CM 9th edition codes. There were small decreases in sensitivity for SSI after the ICD-CM transition and there was a small decrease in the PPV for expanded ICD-CM codes.
Several previous studies have defined the performance of ICD-CM 9th and 10th edition codes or the surveillance of SSIs. Stevenson utilized the PHC4 expanded ICD-CM 9th edition codes and identified a sensitivity of 60% for SSI surveillance.9 Other groups outside the U.S. have utilized ICD-CM 10th edition codes for SSI Surveillance. Kanerva utilized data from Finnish acute care hospitals and demonstrated a sensitivity of 45% for and expanded ICD-CM 10th edition codes.2 Our work builds upon the previous work by identifying the performance of ICD-CM 9th and 10th edition codes in a single health care system using active surveillance of colectomy and abdominal hysterectomy procedures. Like others, we found limited performance for SSI surveillance using ICD-CM codes. The poor performance did not improve after the transition to ICD-CM 10th edition. Interestingly, the direction of change was different for the two procedures studies, in that sensitivity increase with hysterectomy procedure and decreased with colectomy procedures. These data suggest the impact of ICD vocabulary expansion on SSI surveillance is not uniform across procedures and has implications for ICD-based surveillance of other procedures including cardiovascular, neurosurgical, or orthopedic procedures.
Our study has several limitations. First, we limited to mandatory NHSN reporting colon and hysterectomy procedures and did not include other surgical procedures such as orthopedic or neurosurgical procedures where the SSI risk extends beyond 30 days. Second, NHSN reporting may have missed SSIs which the ICD codes identified, thus some false positives may be true positive cases. Lastly, the decrease in the prevalence of SSIs during the study period may partially explain the decrease in PPV of both the narrow and expanded code sets.
CONCLUSIONS
In conclusion, we observed no significant changes in the performance of ICD-CM codes for the detection of SSI in colectomy and abdominal hysterectomy procedures after transitioning from the 9th edition to 10th edition codes. ICD-CM codes for SSI detection continue to have low sensitivity and PPV. We cannot recommend the utilization of ICD-CM 10th edition as a screening tool for SSI surveillance. Predictive models utilizing ICD-CM codes in combination with clinical data are needed to assist in SSI surveillance.
Supplementary Material
FUNDING/SUPPORT
BTB received grant support from the Agency for Healthcare Research and Quality (1K08HS025776).
ROLE OF THE FUNDER/SPONSOR
The Agency for Healthcare Research and Quality had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The views expressed are those of the authors and do not necessarily represent the views or opinions of the U.S. Government or the U.S. Department of Veterans Affairs.
Footnotes
CONFLICT OF INTEREST DISCLOSURE
The authors report no disclosures.
ACCESS TO DATA AND DATA ANALYSIS
Dr. Bucher had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data.
REFERENCES
- 1.Weber WP, Zwahlen M, Reck S, et al. Economic burden of surgical site infections at a European university hospital. Infect Control Hosp Epidemiol 2008;29:623–629. [DOI] [PubMed] [Google Scholar]
- 2.Kanerva M, Ollgren J, Virtanen MJ, Lyytikäinen O. Estimating the annual burden of health care-associated infections in Finnish adult acute care hospitals. Am J Infect Control 2009;37:227–230. [DOI] [PubMed] [Google Scholar]
- 3.West N, Eng T. Monitoring and reporting hospital-acquired conditions: a federalist approach. Medicare Medicaid Res Rev 2014;4:E1–E16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.van Mourik MSM, van Duijn PJ, Moons KGM, Bonten MJM, Lee GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open 2015;5:e008424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Khera R, Dorsey KB, Krumholz HM. Transition to the ICD-10 in the United States: An Emerging Data Chasm. JAMA 2018;320:133–134. [DOI] [PubMed] [Google Scholar]
- 6.Mainor AJ, Morden NE, Smith J, Tomlin S, Skinner J. ICD-10 Coding Will Challenge Researchers: Caution and Collaboration may Reduce Measurement Error and Improve Comparability Over Time. Med Care 2019;57:e42–e46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Berrios-Torres SI, Umscheid CA, Bratzler DW, et al. Centers for Disease Control and Prevention Guideline for the Prevention of Surgical Site Infection, 2017. JAMA Surg 2017;152:784–791. [DOI] [PubMed] [Google Scholar]
- 8.Julian KG, Brumbach AM, Chicora MK, et al. First year of mandatory reporting of healthcare-associated infections, Pennsylvania: an infection control-chart abstractor collaboration. Infect Control Hosp Epidemiol 2006;27:926–930. [DOI] [PubMed] [Google Scholar]
- 9.Stevenson KB, Khan Y, Dickman J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health care-associated infections. Am J Infect Control 2008;36:155–164. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.