Validation of an administrative definition of intensive care unit admission using revenue center codes

Abstract

Objective

Describe the operating characteristics of a proposed set of revenue center codes (RCCs) to correctly identify ICU stays among hospitalized patients.

Design

Retrospective cohort study. We report the operating characteristics of all ICU-related RCCs for intensive and coronary care, excluding nursery, intermediate, and incremental care, to identify ICU stays. We use a classification and regression tree (CART) model to further refine identification of ICU stays using administrative data. The gold standard for classifying ICU admission was an electronic patient location tracking system.

Setting

The University of Pennsylvania Health System in Philadelphia, Pennsylvania, United States.

Patients

All adult inpatient hospital admissions between July 1, 2013 and June 30, 2015

Interventions

None.

Measurements and Main Results

Among 127,680 hospital admissions, the proposed combination of RCCs had 94.6% (95% CI 94.3% - 94.9%) sensitivity and 96.1% (95% CI 96.0% - 96.3%) specificity for correctly identifying hospital admissions with an ICU stay. The CART algorithm had 92.3% (95% CI 91.6% - 93.1%) sensitivity and 97.4% (95% CI 97.2% - 97.6%) specificity, with an overall improved accuracy (χ² = 398; p < 0.001).

Conclusions

Use of the proposed combination of RCCs has excellent sensitivity and specificity for identifying true ICU admission. A CART algorithm with additional administrative variables offers further improvements to accuracy.

Introduction

Validated administrative definitions exist for some specific types of critically ill patients, such as those with severe sepsis (1) and those undergoing mechanical ventilation (2). However, validated definitions of ICU admission for all types of critically ill patients are lacking in the United States (US). This raises questions regarding the utility of many analyses of ICU utilization and outcomes, which commonly rely on large administrative databases (3-7). We therefore sought to describe the accuracy of a proposed administrative definition of ICU admission in the US using UB-92 Revenue Center Codes (RCCs; See Supplementary Table 1) across three hospitals with different specialized ICUs.

Materials and Methods

Patient population

We retrospectively examined administrative and clinical records for all patients at least 18 years of age who were admitted to any of three hospitals in the University of Pennsylvania Health System (UPHS) between July 1, 2013 and June 30, 2015. RCCs associated with each hospital admission were obtained from the UPHS financial offices. Patient location data, billing codes, and demographic information were obtained from the UPHS clinical database.

Defining ICU admission

We classified true ICU admissions based on patient location data in the NaviCare Patient Flow System (Hill-Rom, Chicago, IL) used for all inpatients at UPHS hospitals. We defined the gold standard for a true ICU admission as having spent at least two consecutive hours in any ICU. To confirm the validity of this gold standard, two investigators (G.E.W. and R.K.) independently reviewed the daily progress notes for a randomly selected sample of 50 hospital admissions.

In the proposed administrative definition of ICU admission, we chose to examine all RCCs for both intensive and coronary care. We excluded codes for intermediate care, which may represent a type of care distinct from the ICU (6); incremental nursing care, as these were neither clearly defined nor used in prior studies of ICU utilization; and nursery care. Thus, our RCC-based definition of ICU admission included any hospital admission with one or more of the following RCCs: 200, 201, 202, 203, 204, 207, 208, 209, 210, 211, 212, 213, and 219 (Supplementary Table 1). Using this definition, we report sensitivity, specificity, positive (PPV) and negative predictive values (NPV), and accuracy of RCCs in identifying true ICU admissions.

Classification algorithm using CART

To determine whether accuracy could be improved relative to the simple RCC approach, we developed a classification and regression tree (CART) model using RCCs and other commonly available variables in administrative datasets. The CART algorithm was developed using the rpart package (8) with a 75% sample used for model training with 10-fold cross-validation, and a 25% sample used to test the final model performance. Predictors were chosen based on likely availability in a large, administrative dataset. They included age, modified Elixhauser score, and hospital length of stay as linear, covariates, and the following as categorical variables: gender, race, discharge status (alive/dead), admission type, severe sepsis, mechanical ventilation, vasopressor use, and diagnosis-related group (DRG) codes that are commonly associated with ICU admission. See the digital supplement for details of these administrative definitions. For the testing and training set we report the same performance measures as for the RCCs alone.

All analyses were conducted using R version 3.3.0 (9). McNemar's chi-squared test with a type I error rate of 0.05 is used to report differences in performance measures between the two models. This study was approved by the Institutional Review Board of the University of Pennsylvania.

Results

We identified 133,505 inpatient hospital admissions for patients from the UPHS financial database. Incomplete information or errors in patient-identifying data were discovered in 78 (0.06%) hospital admissions, and these records were excluded. For the remaining 133,427 hospital admissions, we used a combination of unique patient and admission identifiers to match 127,680 (95.7%) administrative records to the patient location data. The manual chart review of daily progress notes of 50 hospital admissions demonstrated that the patient location data derived from the electronic health record was 100% (95% CI 92.9% – 100.0%) accurate for identifying ICU admission during hospitalization. The patient location data identified 24,512 (19.2%) hospital admissions in which the patient spent at least 2 consecutive hours in an ICU. Details of the study population are shown in Table 1.

Table 1. Characteristics of hospital admissions.

Count (percent)
Characteristic	All	True ICU Admissiona	No ICU Admissiona
Hospital admissions (n)	127,680	24,512 (19)	103,168 (81)
Age (years), median (IQR)	56.6 (37.4 - 68.9)	63.8 (52.8 – 73.9)	54.3 (35.0 – 67.3)
Race
White	67,739 (53)	14,588 (11)	53,151 (42)
Black	48,908 (38)	7,328 (6)	41,580 (33)
Other	6,983 (5)	1,196 (1)	5,787 (5)
Unknown	4,050 (3)	1,400 (1)	2,650 (2)
Gender
Female	73,587 (58)	10,732 (8)	62,855 (49)
Male	54,093 (42)	13,780 (11)	40,313 (32)
Hospital length of stay (days), median (IQR)	3.0 (2.0 - 6.0)	7.0 (4.0 – 13.0)	3.0 (2.0 – 5.0)
ICU length of stayb (days), median (IQR)	2.0 (1.1 - 4.2)	2.0 (1.1 - 4.2)	0.0 (0.0 – 0.0)
Admission type
Medical	76,596 (60)	9,757 (8)	66,839 (52)
Surgical	47,494 (37)	14,298 (11)	33,196 (26)
Unclassifiable/Unknown	3,590 (3)	457 (<1)	3,133 (2)
Modified Elixhauser comorbidity score, median (IQR)	2 (0.0– 10.0)	7.0 (0.0 – 15.0)	0.0 (0.0 – 7.0)
Severe sepsis	12,400 (10)	6,214 (5)	6,186 (5)
Intubation or mechanical ventilation	5,979 (5)	5,470 (4)	509 (<1)
Vasopressors	605 (<1)	575 (<1)	30 (<1)
High-riskdiagnosis-related group code	12,226 (10)	3,675 (3)	8,551 (7)
Discharge status
Alive	125,242 (98)	22,510 (18)	102,732 (80)
Dead	2,333 (2)	1,988 (2)	345 (<1)
Unknown	105 (<1)	14 (<1)	91 (<1)
Admitting hospital
Hospital of the University of Pennsylvania (Quaternary-care academic referral center)	66,471 (52)	13,531 (11)	52,940 (41)
Pennsylvania Hospital (Community-based academic affiliate)	34,610 (27)	4,528 (4)	30,082 (24)
Penn Presbyterian Medical Center (Tertiary-care academic affiliate)	26,599 (21)	6,453 (5)	20,146 (16)
Associated revenue center codesc
Intensive care revenue center codes
200 – General	6,413 (5)	4,452 (3)	1,961 (2)
201 – Surgical	14,708 (12)	12,736 (10)	1,972 (2)
202 – Medical	2,996 (2)	2,981 (2)	15 (<1)
Coronary care revenue center codes
210 – General	4,092 (3)	4,060 (3)	32 (<1)

IQR = interquartile range.

^aThe percentages for these columns are calculated based on the entire study population.

^bThe ICU length of stay is calculated only for those patients with a confirmed ICU admission.

^cRevenue center codes included in the analysis but with zero occurrences in the billing data are omitted from the table.

The proposed combination of RCCs had 94.6% (95% CI 94.3% - 94.9%) sensitivity and 96.1% (95% CI 96.0% - 96.3%) specificity for correctly identifying hospital admissions with an ICU stay. The PPV and NPV were 85.4% (95% CI 84.9% - 85.8%) and 98.7% (95% CI 98.6% - 98.8%), respectively. The overall accuracy was 95.8% (95% CI 95.7% - 96.0%).

The final CART model used the following variables: presence of any ICU-related RCCs, age at admission, hospital length of stay, patient gender, and presence of a “high risk” DRG code. The overall accuracy in the training and testing samples were 96.4% (95% CI 96.3% - 96.5%) and 96.4% (95% CI 96.2% - 96.6%), respectively, representing a small but statistically significant improvement in overall accuracy compared to the RCC-only approach (McNemar's χ² = 398; p < 0.001). In the testing sample, the CART model had sensitivity 92.3% (95% CI 91.6% - 93.1%), specificity 97.4% (95% 97.2% - 97.6%), PPV 89.4% (95% CI 88.5% - 90.2%), and NPV 98.2% (95% CI 98.0% -98.3%).

Figure 1 provides a visual depiction of the final CART. The results of an error analysis of factors associated with misclassification using only RCCs are found in the supplement.

Figure 1. Classification of hospital admissions as having or not having an intensive care unit (ICU) admission using commonly available administrative data.

Abbreviations: ICU = intensive care unit, RCC = revenue center code, DRG = diagnosis related group. High-risk DRG codes include: 64, 65, 189, 193, 208, 247, 280, 287, 291, 292, 309, 310, 313, 378, 638, 682, 871, and 918.

Discussion

The proposed RCC-based definition of ICU admission has excellent sensitivity and specificity for identifying ICU stays among hospital admissions using administrative data. The performance of this RCC-based definition of ICU admission compares favorably to other US-based administrative definitions of specific features of critical illness, including severe sepsis (7) and mechanical ventilation (8). This first study to examine such codes in the US provides researchers with a validated tool to study ICU utilization, and outcomes among patients truly admitted to ICUs, using administrative datasets.

The modest improvements in accuracy offered by the CART model relative to the RCC-only definition was expected once we observed the high accuracy of the RCC-only approach. Nonetheless, because this approach requires only modestly greater analytic complexity, investigators who prefer limiting false-positive identification of ICU patients may consider its use in lieu of RCC codes alone.

Strengths of this study include the large sample size, inclusion of data from multiple hospitals, a highly reliable gold standard of ICU admission, and rich administrative data to develop a highly accurate classification tree. Additionally, assignment of RCCs is based on nationally standardized finance and accounting methods which improves the generalizability of these findings. Nonetheless, these results should be considered in light of some limitations. First, the results derive from a single health system, in which all hospitals serve the same geographic region, and none of which offer a dedicated burn unit. Although the patient populations and utilization patterns studied here may not generalize to other health systems, all systems are similarly incentivized to record RCCs accurately. Relatedly, though we did not distinguish between claims reimbursed in fee-for-service or bundled payment arrangements, assignment of RCCs is uniform across all three hospitals and is independent of payer and reimbursement type. Finally, our analysis did not include data on time of death. This precludes an error analysis of coding patterns for those patients who die quickly on arrival to the ICU, and may or may not have ICU-related RCCs or meet the 2-hour threshold used in this study.

Conclusions

Using the RCCs defined here will enable investigators to accurately identify ICU admissions in administrative datasets. These findings merit validation in a more heterogeneous sample of hospitals, but suggest that even if performance of RCC-only classification approaches were lower elsewhere, investigators working with detailed claims data may be able to use additional administrative variables to augment classification.