Evaluating the Use of the Case Mix Index for Risk Adjustment of Healthcare-Associated Infection Data: An Illustration using Clostridium difficile Infection Data from the National Healthcare Safety Network

Abstract

BACKGROUND.

Case mix index (CMI) has been used as a facility-level indicator of patient disease severity. We sought to evaluate the potential for CMI to be used for risk adjustment of National Healthcare Safety Network (NHSN) healthcare-associated infection (HAI) data.

METHODS.

NHSN facility-wide laboratory-identified Clostridium difficile infection event data from 2012 were merged with the fiscal year 2012 Inpatient Prospective Payment System (IPPS) Impact file by CMS certification number (CCN) to obtain a CMI value for hospitals reporting to NHSN. Negative binomial regression was used to evaluate whether CMI was significantly associated with healthcare facility-onset (HO) CDI in univariate and multivariate analysis.

RESULTS.

Among 1,468 acute care hospitals reporting CDI data to NHSN in 2012, 1,429 matched by CCN to a CMI value in the Impact file. CMI (median, 1.49; interquartile range, 1.36–1.66) was a significant predictor of HO CDI in univariate analysis (P < .0001). After controlling for community onset CDI prevalence rate, medical school affiliation, hospital size, and CDI test type use, CMI remained highly significant (P < .0001), with an increase of 0.1 point in CMI associated with a 3.4% increase in the HO CDI incidence rate.

CONCLUSIONS.

CMI was a significant predictor of NHSN HO CDI incidence. Additional work to explore the feasibility of using CMI for risk adjustment of NHSN data is necessary.

Healthcare-associated infection (HAI) surveillance data are essential for hospital infection prevention and quality improvement programs, especially now that HAI data are used for state and federal quality measurement and public reporting.^1,2 Variation in HAI measures is intended to reflect differences in healthcare quality. However, this variation may also reflect differences in non-modifiable patient risk factors (eg, age, comorbidity, severity of illness) in the populations served. Several measures used in the Centers for Medicare and Medicaid Hospital Inpatient Quality Reporting Program, for example, the 30-day readmission and mortality measures,³ are risk adjusted at the patient-level using claims-based data to account for differences among patient populations and to facilitate accurate comparisons of performance between hospitals. To optimize interpretation and use of HAI data, it is ideal to include risk adjustment for known differences in relevant patient characteristics.^4–10

The Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) uses surveillance data to generate a variety of HAI measures, several of which are used for quality reporting purposes.² The measures generated are routinely calculated and risk-adjusted using various facility, unit, or patient-level characteristics, which are reported directly into to the NHSN by trained staff from participating facilities.^8,11 Inclusion of pertinent patient-level characteristics, as for NHSN surgical site infection surveillance,⁸ is desirable to facilitate meaningful interfacility comparisons.^4–10,12,13 However, identification of the relevant set of characteristics is challenging, and routine collection of data for all individual patients within a hospital can be impractical and burdensome.

Case mix index (CMI) based on the Medicare Severity diagnosis-related group (DRG) coding system, the most widely used inpatient billing and reimbursement standard in the United States, has been used as an indicator of facility-level disease severity when patient-level data are unavailable.^14–16 After discharge, patients are assigned to a DRG based on diagnosis and procedure codes, the presence of complications or comorbidity, age (infant or adult), gender, and discharge status. Each DRG has a relative weight assigned (and updated annually), and CMS payments are made proportional to this weight; higher DRG weights receive higher reimbursement. CMI represents the average DRG weight for a hospital and is the sum of the weights of a facility’s DRGs divided by the number of admissions during the time frame of interest (eg, a month, quarter, or year). Importantly, CMI is intended to reflect the clinical complexity and resource consumption of the population of patients within a hospital¹⁴; it is not a specific indicator of severity of illness. Derived from administrative data, CMI is affected by variations in documentation and coding practices between institutions and over time.^12,15,17 Despite these limitations, given the absence of individual patient-level data, we sought to evaluate the potential for CMI to be used as a facility-level disease severity proxy for risk adjustment of HAI data. We used NHSN facility-wide Laboratory identified (LabID) Clostridium difficile infection (CDI) event data as an initial example.

METHODS

NHSN Laboratory-Identified Clostridium difficile Infection Event Definitions and Variables

Standard NHSN LabID event definitions¹⁸ were applied to categorize reported events as healthcare facility-onset (HO), community-onset (CO), or community-onset healthcare facility-associated (CO-HFCA). Hospital CDI incidence (HO events) rate per 10,000 patient days was used as the outcome (dependent variable) to evaluate the following independent variables for risk adjustment: hospital size (number of beds), hospital teaching status, CDI test type used, and the CDI prevalence (CO events) rate per 100 patient admissions (a measure of the volume of the community-onset CDI presenting to a hospital). As is routine for the NHSN,¹⁸ CO-HFCA events, which represent the CO events attributed to the reporting hospital, were removed from the calculation of the CDI prevalence rate, leaving only the CO events presenting from other facilities in the CDI prevalence rate. The existing model used for risk adjustment of facility-wide LabID CDI events data is described in detail elsewhere.¹⁹

Data Sources and Study Population

Two data sources were used for this analysis: LabID CDI event data for 2012 from the NHSN, and the CMS Acute Inpatient Prospective Payment System (IPPS) Impact file for the fiscal year 2012²⁰ to obtain CMI by facility CMS certification number (CCN). CCN was included in both datasets and used as the primary variable to merge the NHSN data with an associated CMI value. Additional information common in both datasets such as state, city, and zip code were used as secondary

variables to verify the matches on CCN were successful. Notably, many-to-one relationships between NHSN data and the CMI value were identified. NHSN data are reported for an individual hospital using a unique NHSN identifier, but CMI values in the CMS Impact file are reported by CCN. To illustrate, a healthcare system consisting of multiple hospitals has multiple unique NHSN identifiers (1 for each individual hospital) but could share a single CCN and, thus, have a common CMI value. All hospitals in the NHSN reporting a CCN, regardless of whether the CCN was associated with multiple hospitals within a system, were included in the analysis.

To be eligible hospitals were required to report facility-wide LabID CDI data to NHSN during 2012 and classified as an acute care hospital; long-term acute care hospitals and critical access hospitals were excluded. Facilities with incomplete or missing denominator data, or with outlier values for CDI incidence or CDI prevalence (n = 4), defined as 5 times the interquartile range above the 75^th percentile (IQR5), were also excluded. Additionally, hospitals that did not report a CCN to NHSN were excluded (n = 32). A total of 36 acute care hospitals (2.4%) were excluded from the analysis; the final number of facilities eligible for further analysis was 1,468.

Statistical Approach

Negative binomial regression, a generalization of Poisson regression suitable for the analysis of count data with overdispersed outcome variables,²¹ was used to analyze the data. Each independent variable was evaluated to identify the best parameterization for inclusion in the model and to determine whether there was a significant association with CDI incidence. All variables with P < .20 in univariate analyses were included in the final model. A multivariate model was built using stage-wise selection, with the sequential addition of variables to identify the best final model. Variables were retained in the final model if P < .05. The fit of various models was compared using the Akaike information criterion (AIC), with smaller values indicating better fit, and by calculating pseudo-R² based on model dispersion parameter suitable for use with negative binomial regression.^21–23 While the interpretation is not the same as for the R² measure in linear regression (ie, the proportion of variability explained by the model), pseudo-R² measures are valid and useful in evaluating multiple models predicting the same outcome from the same dataset, as in this analysis. Their values range from 0 to 1 and take on higher values for better-fitting models. All statistical analyses were performed using SAS version 9.3 software (SAS Institute Inc, Cary, North Carolina). All reported P values are 2-sided, and P < 0.05 was considered statistically significant.

RESULTS

Participants

Of the 1,468 inpatient acute care hospitals reporting data to the NHSN that met the inclusion criteria, 1,429 (97.3%) had matching records in the CMS IPPS Impact file. Most often (for 92% of 1,429 hospitals), there was a single CCN for each hospital in the NHSN. For the remaining 8%, between 2 and 5 unique hospitals in the NHSN were associated with a single CCN and consequently had a common CMI value with those hospitals sharing a CCN.

Among the 1,429 hospitals, a total of 39,725 HO CDI events and 54,472,047 patient days were reported, with a pooled mean incidence of 7.293 per 10,000 patient days. The variability in NHSN facility characteristics is summarized in Table 1.

TABLE 1.

Characteristics of Acute Care Hospitals Reporting Laboratory-identified Clostridium difficile Infection (CDI) Events to the National Healthcare Safety Network, 2012

Characteristic	No. (%)a
State or territory, No. hospitals
California	306 (21.4)
New York	170 (11.9)
Illinois	127 (8.9)
Tennessee	104 (7.3)
Otherb	722 (50.5)
Hospital size, No. beds
Median (IQR)	175 (96–303)
Medical School Affiliation, No. hospitals
Major	203 (14.2)
Graduate	233 (16.3)
Undergraduate	49 (3.4)
Non-teaching	944 (66.1)
CDI test type, No. hospitals
Nucleic acid amplification test	804 (56.3)
Enzyme immunoassay	574 (40.2)
Other	51 (3.6)
CDI prevalence per 100 admissions
Median (IQR)	0.272 (0.132–0.453)
CDI incidence per 10,000 patient days
Median (IQR)	5.399 (2.710–8.457)

NOTE. IQR, interquartile range.

^aUnless otherwise noted.

^b47 states or territories, range, 1–63 hospitals.

Case Mix Index

Among the 1,429 hospitals, the median hospital CMI was 1.49 (interquartile range, 1.307–1.664), and ranged from 0.8 to 3.8 (Figure 1); most hospitals (94.1%) had a CMI value between 1.0 and 2.0.

FIGURE 1.

Distribution of case mix index (CMI) for acute care hospitals reporting facility-wide laboratory-identified Clostridium difficile infection events to the National Healthcare Safety Network, 2012.

Univariate Analysis

A list of the variables available for analysis is summarized in Table 2. Each of the variables had P < .05 and were thus included in multivariate modeling. The variable CMI was significantly associated with CDI incidence; a 0.1 unit increase in CMI was associated with a 6.4% increase in CDI incidence.

TABLE 2.

Predictors of Healthcare Facility-Onset Clostridium difficile Infection (CDI) Events by Univariate and Multivariate Analyses among Inpatient Acute Care Hospitals Reporting Laboratory-Identified Events to the National Healthcare Safety Network, 2012

	Univariate			Multivariate
Parameter	Est.	Rate Ratio (95% CI)	P Value	Est.	Rate Ratio (95% CI)	P Value
CDI prevalence rate, per 0.1 unit increase	0.146	1.157 (1.143–1.172)	<.0001	0.138	1.148 (1.134–1.162)	<.0001
Case mix index, per 0.1 unit increase	0.062	1.064 (1.049–1.079)	<.0001	0.033	1.034 (1.021–1.047)	<.0001
CDI test type
NAAT vs. EIA/Othera	0.497	1.643 (1.536–1.758)	<.0001	0.211	1.235 (1.163–1.312)	<.0001
Hospital size, No. beds		…	<.0001b		…	<.0001b
≥254 vs <120a	0.382	1.466 (1.340–1.603)	<.0001	0.204	1.227 (1.127–1.335)	<.0001
120–253 vs <120a	0.192	1.211 (1.105–1.328)	<.0001	0.113	1.119 (1.035–1.210)	.0047
Medical school affiliation:		…	<.0001b		…	.0115b
Major vs undergraduate/Non-teachinga	0.199	1.221 (1.110–1.342)	<.0001	0.110	1.117 (1.033–1.207)	.0054
Graduate vs undergraduate/Non-teachinga	0.102	1.107 (1.009–1.214)	.0308	0.069	1.071 (0.997–1.151)	.0597

NOTE. NAAT, nucleic acid amplification test; EIA, enzyme immunoassay.

^aReferent group.

^bMain effect P value.

Multivariate Analysis

In multivariate modeling, all of the variables remained significant, independent predictors of CDI incidence (Table 2). The predictor of CDI incidence with the best fit (lowest AIC) and the highest rate ratio was CDI prevalence. When controlling for the other variables in the model, with every 0.1 unit increase in CDI prevalence (increase of 0.1 per 100 admissions), there was a 14.8% increase in the CDI incidence rate. The second best fitting predictor with the next highest rate ratio was CMI; every 0.1 unit increase in CMI (on a scale of 0.000 to 4.000) resulted in a 3.4% increase in the CDI incidence rate, after controlling for CDI prevalence, CDI test type use, hospital size, and medical school affiliation (Figure 2).

FIGURE 2.

Modelled change in National Healthcare Safety Network healthcare facility-onset. *Clostridium difficile infection incidence rate ratio per 0.1 unit increase in hospital case mix index (CMI).

Model Fit

The multivariate model including only the 4 NHSN variables (ie, CDI prevalence per 100 admissions, CDI test type use, hospital size, and medical school affiliation) had an AIC of 9,195.46 and pseudo-R² = 0.4980. The model with the 4 NHSN variables and CMI had a lower AIC of 9,172.59 and a marginally higher pseudo-R² of 0.5142, indicating that the model fit with CMI was improved over the fit of the model without CMI.

DISCUSSION

Our findings suggest that hospital CMI is a significant independent predictor of NHSN HO CDI incidence. After controlling for CDI prevalence rate (the amount of CDI presenting to a hospital), CDI test type, hospital size, and medical school affiliation, hospitals with a higher CMI had a higher CDI incidence rate. To illustrate, given equivalent values of CDI prevalence, CDI test type, hospital size, and medical school affiliation, a hospital with a CMI of 1.66 (the 75th percentile of CMI for hospitals included in this analysis), would be predicted to have a 12% higher incidence rate of HO CDI compared with a hospital with a CMI of 1.31 (the 25th percentile). This finding is important because use of CMI for CDI risk adjustment represents the use of a factor beyond those routinely collected in NHSN, and one that represents the patient population served by a hospital.^6,10 Additionally, CMI is attractive for risk adjustment of HAI data because it is a hospital-level metric already known to administrators and clinicians, is a single value obtained easily from administrative data, and its use requires no additional patient-level data collection by infection preventionists.

CMI was designed for hospital payment purposes, and as such, CMI is higher when patients use more resources and care is more expensive, but it is not necessarily higher for patients with the most severe disease.¹⁴ Nevertheless, CMI has often been used as an indicator to represent facility-level severity of illness or acuity,^{13–16,24–27} even though its role in the risk adjustment of HAI data is not yet clear. In a study of bloodstream infection (BSI) in a single long-term acute care hospital, no change in CMI was noted over a 24-month intervention period,²⁵ perhaps suggesting that CMI is more useful for interfacility rather than intrafacility comparisons. In a group of Swiss hospitals, antibiotic use (defined as daily doses per 100 bed days) had a significant, positive correlation with CMI at both the department level and the hospital level.¹⁶ In a study of 70 US academic medical centers,²⁶ hospital-wide adult antibacterial drug use (defined as days of therapy and length of therapy per 1,000 patient days) was not significantly correlated with CMI. In a report of hospital methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant Enterococcus (VRE) BSI data from the California Department of Public Health,²⁷ hospital CMI values were included along-side their BSI incidence rates in an attempt to “… indicate whether a hospital serves patients with higher or lower severity of illness.” While risk adjustment was not performed, the report indicates an association between CMI and VRE BSIs, but not MRSA BSI. Differences in the settings, outcomes under study, variables evaluated, and methods used have generated inconsistent findings with respect to the use of CMI. Given its appeal as a simple proxy measure of hospital-level severity of illness, the use of CMI for risk adjustment of HAI data warrants further evaluation.

There are several limitations to report. First, some NHSN facility types were excluded from this analysis due to a difference in the distribution of CDI incidence rates (Long-term Acute Care and Critical Access Hospitals) or because a CMI value was not available in the CMS IPPS Impact files (Indian Health Service, Military, or Veterans Affairs hospitals). It is, therefore, unknown whether our findings are applicable outside of acute care hospitals. Second, 8% of hospitals included shared a CMI value with 1 or more other hospitals under a single CNN. This overlap most likely limits the ability of CMI to predict CDI incidence, and, therefore, the impact of CMI on risk adjustment should be considered somewhat conservative. Third, it is possible that patients with healthcare-facility onset CDI were assigned to a DRG with a higher weight, potentially increasing hospital CMI value; however, we anticipate that any increase in CMI was negligible. Fourth, the CMI obtained from the CMS IPPS Impact file were calculated using DRGs only from Medicare discharges; these values may not be equivalent to a CMI based on all patients within a hospital. While other measures of CMI exist, such as the All-Patient Refined DRG system,^15,28,29 they were not available for our analysis.

Exploring the use of CMI represents the initiation of work at the CDC to identify additional factors outside of the NHSN that may be suitable candidates for risk adjustment of HAI data. The use of CMI for risk adjustment of other NHSN measures, such as LabID MRSA bacteremia and antibiotic use measures, is underway. Additional work is necessary to determine the feasibility of routinely obtaining and incorporating hospital CMI into NHSN risk adjustment approaches. Though much debate continues around which set of patient-level factors is appropriate for risk adjustment of various healthcare quality measures,^{4–10,13,30,31} ultimately, it is the intent of the NHSN to move toward the use of more patient-level data for risk adjustment to improve the accuracy of the HAI measures generated.^5,10 To minimize the burden of such additional data collection, NHSN has developed infrastructure for hospitals and electronic health record (EHR) vendors³² to enable automated collection and collation of data for NHSN reporting. However, until such patient-level data are available, facility-level proxies of clinical complexity like CMI will continue to be attractive for risk adjustment purposes.

CMI is a single measure that reflects the clinical complexity and resource consumption of patients within a hospital. Its use for risk adjustment is intended to represent differences in the patient populations served. We found CMI to be significant predictor of NHSN HO onset CDI for acute care hospitals, even after adjustment for other NHSN variables (CDI prevalence rate, CDI test type, hospital size, and medical school affiliation). Further exploration of methods to improve risk adjustment of NHSN data, including but not limited to CMI, will continue.