Validation of intensive care and mechanical ventilation codes in Medicare data

Abstract

Objective

To assess the reliability of codes relevant to critically ill patients in administrative data.

Design

Retrospective cohort study linking data from APACHE Outcomes, a clinical database of intensive care unit (ICU) patients with data from Medicare (MedPAR). We linked data based on matching for sex, date of birth, hospital, and date of admission to hospital.

Setting

46 hospitals in the US participating in APACHE Outcomes.

Patients

All patients in APACHE Outcomes ≥65 years of age who could be linked with hospitalization records in MedPAR from 1 January 2009 through 30 September 2012.

Measurements & Main Results

Of 62,451 patients in the APACHE Outcomes dataset, 80.1% were matched with data in MedPar. All but 2.7% of APACHE Outcomes ICU patients had either an ICU or coronary care unit charge in MedPAR. In APACHE Outcomes, 37.0% received MV during the ICU stay versus 24.1% in MedPAR. The MedPAR procedure codes for MV had high specificity (96.0%; 95% Confidence Interval (CI) 95.8–96.2), but only moderate sensitivity (58.4%; 95% CI 57.7–59.1), with a positive predictive value of 89.6% (95% CI 89.1–90.1) and negative predictive value of 79.7% (95% CI 79.4–80.1). For patients with MV codes, MedPAR overestimated the percentage with a duration >96hrs (36.6% versus 27.3% in APACHE Outcomes). There was discordance in the hospital discharge status (alive or dead) for only 0.47% of all linked records (kappa=1.00).

Conclusions

MedPAR data contains robust information on hospital mortality for patients admitted to the ICU, but has limited ability to identify all patients who received mechanical ventilation during a critical illness. Estimates of use of mechanical ventilation in the United States should likely be revised upward.

The care of critically ill patients in the United States represents a large proportion of healthcare spending (1). Medicare data, and other administrative datasets such as the Nationwide Inpatient Sample are often used to assess the epidemiology of critically ill patients and understand the burden of disease in the US (1–3). Many studies focus on identifying and assessing patients who receive mechanical ventilation (MV), as this group of critically ill patients is at particularly high risk of death (4, 5). In particular, recent studies have focused on use of “prolonged mechanical ventilation”, often defined in administrative data using the code for “continuous invasive mechanical ventilation for 96 hours or more” (3, 6). However, the reliability of key codes, such as the codes for intensive care and MV are not known. We therefore sought to validate these codes in an administrative dataset: the Medicare Provider Analysis and Review (MedPAR) data files against data from a clinical audit database to determine their reliability for identifying critically ill patients and associated mechanical support.

Methods

Data were obtained from the Acute Physiology and Chronic Health Evaluation (APACHE) Outcomes database (Cerner Corporation, Kansas City, MO) for ICU admissions from 1 Jan 2009 through 30 September 2012. Participation in APACHE Outcomes is voluntary: participating ICUs pay a set fee for use of the software. The resultant database was generated based on records of medical care and collected using software that supports automated and computer based manual entry. For all patients age 65 years and older, we attempted deterministic matching with MedPar (Centers for Medicare & Medicaid Services) with exact matches for sex, date of birth, hospital, and date of admission to hospital. The MEDPAR file contains data from claims for services to beneficiaries admitted to Medicare certified inpatient hospitals who have Medicare fee-for-service insurance coverage. We used the APACHE Outcomes data as the clinical standard for patient characteristics, interventions and outcomes. We assessed coding for intensive care unit (ICU) and coronary care unit (CCU) days in MedPAR, defined using 020X and 021X accommodation revenue codes. For MedPAR we assessed the sensitivity, specificity and positive and negative predictive value of billing codes for mechanical ventilation (defined using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes 96.7x, and then stratified by greater or less than 96 hours of MV (96.72 and 96.71 respectively), and by individual hospitals. We further compared the patient characteristics of the patients identified as receiving MV in APACHE Outcomes, but not in MedPAR and compared them with patients who were coded as receiving MV in MedPAR.

We also compared ICU and hospital length of stay and assessed the agreement for coding of hospital mortality using the kappa statistic. ICU length of stay was calculated in APACHE Outcomes based on date and time stamps for ICU admission and discharge; MedPAR ICU length of stay was based on days with 020X and 021X revenue codes. Due to the discrepancy noted in ICU length of stay, we post hoc assessed the difference with a number of exploratory analyses: (1) recoding lengths of stay in APACHE Outcomes as total days based on any time spent in the ICU from midnight to midnight; (2) including only patients admitted from either the emergency room or general ward and discharged from ICU to the ward (based on APACHE data). Finally, we assessed patient characteristics of the patients with concordant versus discordant ICU length of stay in APACHE Outcomes and MedPAR. This research was exempt from IRB review by Albert Einstein College of Medicine.

Results

Of 62,451 patients ≥ 65 years of age in the APACHE Outcomes dataset, 55,019 (80.1%) were matched with data in MedPar. Of these patients, 48,487 (88.1%) had ICU bed charges in MedPar and 16,890 (30.7%) had CCU bed charges. When combined, only 1,463 (2.7%) of APACHE Outcomes ICU patients had neither an ICU nor CCU charge in MedPar (Table 1).

Table 1.

Comparison of ICU codes, duration of mechanical ventilation, lengths of stay and hospital mortality in APACHE Outcomes versus MedPAR

Variable	APACHE Outcomes n=55,019	MedPAR n=55,019
Coded as admission to ICU or CCU, n (%)	NA	53,556 (97.3)
Any mechanical ventilation, n (%)	20,347 (37.0)	13,257 (24.1)
Duration of mechanical ventilation, n (%)*
<96 hours	14,779 (72.6)	8,383 (63.2)
>96 hours	5,564 (27.3)	4,851 (36.6)
Unknown	4 (0.0)	23 (0.2)
Length of ICU stay, median (IQR)**	2 (1,4)	4 (2,8)
Length of hospital stay, median (IQR)	7 (4,12)	7 (4,12)
Hospital mortality, n (%)	7,842 (14.3)	7,689 (14.0)

ICU = intensive care unit; IQR = interquartile range

^*Duration of mechanical ventilation coding: APACHE Outcomes: date and time stamps; MedPAR: defined using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes 96.7x, and then stratified by greater or less than 96 hours of MV (96.72 and 96.71 respectively)

^**APACHE Outcomes: date and time of ICU discharge minus date and time of ICU admission, rounded to nearest day; MedPAR: billing days for intensive care (all hospital days with a 020X level revenue center code)

In APACHE Outcomes, 20,343 (37.0%) received MV during the ICU stay versus 13,257 (24.1%) in MedPAR. The MedPAR procedure codes for MV had high specificity (96.0%; 95% confidence interval (CI) 95.8–96.2), but only moderate sensitivity (58.4%; 95% CI 57.7–59.1), with a positive predictive value of 89.6% (95% CI 89.1–90.1) and negative predictive value of 79.7% (95% CI 79.4–80.1; Table 2). Specificity was consistently high across individual hospitals, but sensitivity varied (Appendix Table 1). For patients with MV codes, MedPAR overestimated the percentage with a duration >96hrs (36.6% versus 27.3% in APACHE Outcomes) (Table 1). A comparison of patients who had MV codes in APACHE Outcomes but not in MedPAR revealed that these patients were primarily surgical (70%), and with much lower hospital mortality compared with patients in MedPAR who had an MV code (11% versus 36%, p<0.001; Appendix Table 2)

Table 2.

Assessment of mechanical ventilation codes in MedPAR

MedPar – Mechanical ventilation	APACHE Outcomes – Mechanical Ventilation		PPV/NPV
	Yes	No
Yes	11,882 (21.6)	1,375 (2.5)	PPV = 89.6% (95% CI 89.1–90.1)
No	8,465 (15.4)	33,301 (60.5)	NPV = 79.7% (95% CI 79.4–80.1)
	Sensitivity = 58.4% (95% CI 57.7–59.1)	Specificity = 96.0% (95% CI 95.8–96.2)

CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value

Length of ICU stay was overestimated using MedPAR billing data (Table 1), with a median length of stay twice that reported in APACHE Outcomes. Further assessment demonstrated that restriction of analysis to patients who did not spend time in other ICUs or on stepdown beds yielded more similar ICU lengths of stay (Appendix Table 2). Overall, approximately 50% of patients had concordant ICU length of stay (Appendix Table 3). These patients were less likely to be discharged to stepdown from the ICU and were more likely to have died in the ICU or during hospitalization.

Length of hospital stay, and hospital mortality were very similar in the two datasets. Hospital mortality was accurately recorded in MedPAR; there was discordance in the hospital discharge status (alive or dead) for only 261 (0.47%) of all linked records (kappa=1.00).

Discussion

For Medicare beneficiaries with hospitalizations linked with data from a high quality clinical database of intensive care admissions, a combination of ICU and CCU codes allows for accurate identification of admission to intensive care. Hospital mortality for patients is also accurate. Mechanical ventilation procedure codes in Medicare are very specific (96.0%) but with only moderate sensitivity (58.4%), and overestimate the proportion of patients with an extended length of mechanical ventilation (>96 hours). This finding is substantially different from the validation of mechanical ventilation codes in administrative data from Canada that found much higher sensitivity (87.0%) (7), but in line with the validation performed by Wonneberger et al using data from the University of Pennsylvania and Kaiser Permanente, finding high specificity and low sensitivity (8).

The information regarding mechanical ventilation codes in Medicare has large implications for epidemiological studies of critically ill patients in the United States. Our findings suggest that MedPAR may be appropriate for cohort studies of mechanically ventilated patients that focus on outcomes, with the recognition that the data are capturing a slightly more severely ill group of mechanically ventilated patients with longer duration of mechanical ventilation (5, 9) and in particular, are excluding post-surgical patients who require a short period of MV. However, these administrative data substantially underestimate the population “burden” of mechanical ventilation and the resource use across the United States, suggesting these estimates should be revised upward (2). Similarly, the estimated ICU length of stay for patients in MedPAR appears to be high compared with clinical data, suggesting that we may similarly be overestimating the number of ICU bed-days for patients across the United States. Much of this overestimation may be due to the practice in MedPAR of combining ICU and intermediate ICU days.

Our study has important limitations. First, chart review to determine use of MV would represent the gold standard for assessment of MedPAR data. Due to the difficulty of this task, we used APACHE Outcomes as our “clinical” standard, but recognize that we do not have validation of this coding. Assessment is limited to patients with Medicare as their primary insurance and to those patients who are over the age of 65 and in APACHE hospitals, which may be larger and more academic than the majority of US hospitals. Moreover, because APACHE Outcomes only includes a small subset of ICU patients, we were unable to fully assess the sensitivity and specificity of ICU and CCU codes in MedPAR. Second, some patients may have accrued more ICU days in other ICUs in the APACHE data, or during readmissions to ICU not captured in this analysis, limiting our ability to fully explore the discrepancy we found in ICU days between the datasets. Finally, it is possible that other administrative datasets may have different reliability of these codes; however, since most of these datasets rely on billing data, these findings may be applicable to other data sources, but this remains speculative.

In conclusion, MedPAR data contains robust information on hospital mortality for patients admitted to the ICU, but with limited ability to identify all patients who received mechanical ventilation during a critical illness. Estimates of use of mechanical ventilation in the United States should likely be revised upward.