Risk-adjusted outcomes of inpatient medicare medical admissions

Supplemental Digital Content is available in the text

Abstract

It is important that actual outcomes of care and not surrogate markers, such as process measures, be used to evaluate the quality of inpatient care. Because of the heterogenous composition of patients, risk-adjustment is essential for the objective evaluation of outcomes following inpatient care. Comparative evaluation of risk-adjusted outcomes can be used to identify suboptimal performance and can provide direction for care improvement initiatives.

We studied the risk-adjusted outcomes of 6 medical conditions during the inpatient and 90-day post-discharge period to identify the opportunities for care improvement. The Medicare Limited Dataset for 2012 to 2014 was used to identify acute myocardial infarction (AMI), chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), pneumonia (PNEU), cerebrovascular accidents (CVA), and gastrointestinal hemorrhage (GIH). Stepwise logistic predictive models were developed for the adverse outcomes (AOs) of inpatient deaths, 3-sigma prolonged length-of-stay outliers, 90-day post-discharge deaths, and 90-day readmissions after unrelated events were excluded. Observed and predicted AOs were determined for each hospital with ≥75 cases for each of the 6 medical conditions. Z-scores and risk-adjusted AO rates for each hospital permitted comparative analysis of outcomes after adjusting for covariance among the medical conditions.

There were a total of 1,811,749 patients from 973 acute care hospitals with the 6 medical conditions. A total of 41% of all patients had ≥1 AO events. One or more readmissions were identified in 29.8% of patients. A total of 64 hospitals (6.4%) were 2 standard deviations better than the mean for risk-adjusted outcomes, and 72 (7.4%) were 2 standard deviations poorer. The best performing decile of hospitals had mean AO rates of 35.1% (odds ratio = 0.766; 95% confidence interval (CI) CI: 0.762–0.771) and the poorest performing decile a mean AO rate of 48.5% (odds ratio = 1.357; 95% CI: 1.346–1.369). Volume of qualifying cases ranged from 670 to 9314; no association was identified for increased volume of patients (P < .40).

Risk-adjusted AO rates demonstrated nearly a 14% opportunity for care improvement between top and suboptimal performing hospitals. Hospitals must be able to benchmark objective measurement of outcomes to inform quality initiatives.

1. Introduction

Value-based care (VBC) has become the goal of government and private payers of healthcare services. The concept of VBC places a premium on the desired outcomes that are delivered at a fair price. The Centers for Medicare and Medicaid Services (CMS) have adopted VBC models of accountable care organizations and alternative payment strategies. These and evolving alternative payment models will require planning and care redesign efforts to improve outcomes for patients and better value for payers.

More recently, CMS created the Total Performance Score (TPS) for hospitals.^[1] The TPS ranges from 0 to 100 and is a composite score of metrics that include mortality rates, process measures, patient satisfaction evaluations, patient safety indicators, infection rates, and an efficiency/cost reduction domain. Hospitals will sustain future Medicare payments penalties of 2% of base Medicare-Severity Diagnosis-Related Group (MS DRG) payments based upon their TPS for FY 2017. Penalty revenue will be given as incentive payments to hospitals with favorable TPS. The TPS is the foundation upon which the Hospital-Compare system has been created to direct patient selection of hospitals for inpatient care.^[2]

To use hospital performance metrics that are based solely upon objective measures of clinical outcomes and not opinion surveys or process measures, we examined 6 inpatient major medical conditions. Aggregation of risk-adjusted outcomes for these six separate medical conditions provides an objective assessment of individual performance of hospitals. Risk-adjusted outcomes provide hospitals with benchmarks to identify specific areas where care redesign is necessary for improvement.

2. Methods

The design of this research effort was to create risk-adjusted models for each of the 4 outcomes among 6 different medical admissions. Prediction models were then used to compare observed and predicted risk-adjusted adverse outcome events among Medicare patients in hospitals meeting minimum volumes of cases.

To achieve this research objective, we used the Medicare Limited Dataset for 2012 to 2014 to identify 6 major groups of medical hospitalizations. Only patients classified within medical Medicare-Severity, Diagnosis-Related Groups (MS-DRGs) for each of these categories were included. Classification by MS-DRG avoided inclusion of associated procedural interventions that could affect outcomes. The 6 groups were acute myocardial infarction (AMI; MS-DRGs 280–285), chronic obstructive pulmonary disease (COPD; MS-DRGs 190–192), congestive heart failure (CHF; MS-DRGs 291–293), pneumonia (PNEU; MS-DRGs 177–179, 193–195), cerebrovascular accidents (CVA; MS-DRGs 061–066), and gastrointestinal hemorrhage (GIH; MS-DRGs 377–379). A flow chart of case exclusions at each step of the analysis is presented in Fig. 1. Patient age <65 years were excluded because they are disability patients which are younger and commonly have a single illness that is a major risk factor. Including them increases the heterogeneity of the study population. All Medicare Advantage patients were excluded. Other exclusions were patients with missing critical data (e.g., patient or hospital identifier), patient transfers to-or-from another acute care hospital, or discharges against medical advice. Finally, patients admitted during the last 3 months of the study period had 90-day follow up that extended into the year 2015 were excluded from this analysis.

Figure 1.

A flow diagram illustrates the starting population of patients and identifies the changes in the population as selected cases were excluded.

This is a retrospective and observational study of administrative data. The research is compliant with all federal guidelines in our data use agreement (DUA), especially those relating to reporting small cells of data that might compromise patient confidentiality. Patient informed consent was not obtained for the over 1.8 million study cases. Patient informed consent is waived since this is a retrospective database without patient identifiable information. The appropriate use of the data is covered by the DUA. Institutional Review Board approval was not obtained for this study, since this is not required for use of this dataset with a current DUA with CMS.^[3] We have a current DUA with Medicare (LDSS-2012-23573) for the use of this data.

2.1. Predictive models

Separate predictive models for each of the 6 medical conditions were designed for each of the 4 dependent variables of adverse outcomes (AOs). The 4 dependent AOs variables of interest were inpatient deaths (IpD), inpatient risk-adjusted length-of-stay outliers (prLOS), 90-day post-discharge deaths without readmission (PD90), and 90-day hospital readmissions (RA90). The 90-day period for post-discharge events was selected since it is the period of cost attribution for participating hospitals and providers in both the Bundled Payments for Care Improvement (BPCI) program,^[4] and the recently introduced BPCI Advanced program by CMS.^[5] The IpD model was developed with all hospital admissions for each condition. The prLOS model used live discharges from the index hospitalization. The PD90 model used live discharges without readmission, and the RA90 model used live discharges that survived for 90-days. Excluded readmissions for the dependent variable in the RA90 model were consistent with those identified in the CMS BPCI program and are detailed in Supplemental Digital Table 1.

For each medical condition, risk models were developed using stepwise logistic regression after previously reported methods.^[6–8] Because present-on-admission (POA) coding is important in the separation of risk factors from complications in administrative data for acute medical conditions, only hospitals with quality POA coding scores were used for model development.^[9] We used over 500 individual/aggregated present-on-admission codes to identify the independent risk factors. All candidate risk factors were presented for model evaluation with only those variables with P < .01 retained in final models. Hospital dummy variables were employed to remove hospital effects.^[10] PrLOS variables were used in the PD90 and RA90 models to account for effects of prolonged inpatient length-of-stay on post-discharge AOs. Schwarz criterion was used to avoid over-fitting models.^[11] C-statistics were used to evaluate the discrimination of final models. Final models were then applied to all hospitals that met minimum volume criteria.

We have used prLOS outliers as a surrogate indicator for major inpatient complications rather than the coded complications in the hospital discharge records. A perverse incentive exists for the coding of all potential complications during the hospitalization to enhance reimbursement under the Medicare prospective payment model. Differences in rates of coded complications often are the result of individual hospital coding practices rather than the true differences in clinically relevant events. We have found that many coded complications are not associated with any prolongation of hospitalization.^[12] Thus, we employed a linear model to define the appropriate risk-adjusted length-of-stay for patients without any complications. We used that linear model with a moving-range control chart to identify cases with a risk-adjusted length-of-stay that were 3-σ greater than the upper control limit to serve as a surrogate for major complication rather than the coded entries.^[13,14] SAS software (Version 9.4, SAS Institute, Cary, NC) was used for all analyses except where indicated otherwise.

2.2. Readmissions

The total population of patients used for model development were evaluated for causes of readmissions in each of the 6 medical categories. The MS-DRG of the initial readmission was identified and categorized by the duration of time after discharge within the 90-day study period. Because multiple readmissions occur for the same patients within the 90-days following discharge from the index hospitalization, these were separately evaluated by specific MS-DRG and by the time of readmission.

2.3. Hospital-level outcomes

Eligible hospitals for this analysis were those with a minimum of 75 cases during the 3-year study period for each of the 6 medical categories. The minimum requirement of 75 cases was used in all categories so that general acute care hospitals were included, and to avoid the presence of specialized facilities with a special interest in 1 category (e.g., heart hospitals), but limited volume and interest for other medical conditions.

A patient was deemed to have had an overall adverse outcome if ≥1 of the 4 individual AO events occurred. From the individual prediction models, the probability of ≥1 AO was calculated for each condition. The total number of patients with ≥1 AO was then computed within each medical group. The total predicted AOs for the entire population was then set equal to the total number of observed patients with ≥1 AO events, by multiplication of the predicted values by (Observed AOs÷Predicted AOs) × (AO rate of the whole population of study hospitals), which is (o÷p) × (AO rate of the whole population of study hospitals). The p of ≥1 AO combined with the number of cases (N) within each hospital permits calculation of the variance (V) that is unique to the risk profile of the hospital population (V = N∗p∗[1–p]) for each medical condition.

Hospital performance was compared by computation of the z-score (Z), where Z = (Observed AOs – Predicted AOs) ÷ (Standard Deviation [SD] of Predicted values). The SD equals the square root of the sum of the 6 V-values of each medical condition, assuming they are mutually independent. However, outcomes of the medical conditions may not be independent because of a common hospital and common hospital physicians.

To adjust for potential dependency, the covariance was computed for the relationship of each of the 6 medical conditions using Excel (Covariance-S) with the observed AO rate for each hospital.^[15] Each pair of the 6 medical conditions had a covariance coefficient (CC) computed, for a total of 15 separate pairs. For each pair of medical conditions, the covariance adjustment = 2∗N₁∗N₂∗CC_1,2, where N equals the number of cases studied within each of the medical conditions.^[16] The 15 covariance adjustments of the paired medical conditions were then added to the summed values of the normally computed 6 variance values (V = N∗p∗(1–p)) of each medical condition. The final z-score for the performance of each hospital was then computed by . The z-score indicated whether hospital overall performance for all 6 medical conditions was better (negative value) or poorer (positive value) compared with the population of all study hospitals. The risk-adjusted AO rate was estimated by the conventional method of (o/p)∗(AO rate of the population) for each hospital.^[17]

3. Results

3.1. Predictive models

The total number of patients from all hospitals that met eligibility criteria were 209,548 for AMI; 640,000 for COPD; 781,844 for CHF; 788,188 for PNEU; 432,270 for CVA; and 413,555 for GIH. From this total database of 3,265,405 patients, only patients from good coding hospitals (n = 2,562,787; 78.5%) regardless of volume of cases in individual hospitals were used in the development of predictive models for each of the 6 medical conditions. The significant risk factors and their respective odds ratios derived from only good coding hospitals are detailed for each medical condition and for each of the AO events in Supplemental Digital Tables 2–5. The total number of cases from good coding hospitals for model development of each of the 6 medical conditions, total variables in each model, and respective C-statistics of models are identified in Table 1. The poorest discrimination was the prediction model for readmissions which has been the general experience.^[18] This likely relates to non-medical variables that influence readmissions.

Table 1.

Details of the number of variables and the C-statistic for each of the models for the 4 AOs in each of the 6 medical conditions.

3.2. Readmissions

A total of 29.8% of index hospitalizations from all 6 medical conditions in the total database had ≥1 readmission within the 90-day post-discharge period, and represented the biggest factor in the overall AO rate. The MS DRGs of the first readmission within each medical category are detailed in Table 2, and the MS DRGs of all repeat readmissions are detailed in Table 3. Only MS DRGs that occurred in >1% of readmissions are identified in Tables 2 and 3. The first readmissions for each medical category commonly were for MS DRGs of one of the other 6 medical groups. Acute renal failure, sepsis, cardiac arrhythmias, and gastrointestinal events were also common for medical first readmissions.

Table 2.

The total number of MS DRGs that occurred in ≥1% of first readmissions in the 6 groups of index medical admissions.

Table 3.

The total number of MS DRGs that occurred in ≥1% of repeat readmissions in the 6 groups of index medical admissions.

First readmissions occurred in the initial 30 days following discharge in 50% to 60% of readmitted patients (Table 4). As noted in Table 4, total readmissions exceeded the total first readmissions because selected patients had multiple readmission events in the 90-day observation period.

Table 4.

The readmission profile of all patients in the total dataset are presented.

3.3. Hospital outcomes

A total of 973 hospitals met the criteria of 75 cases for each of the 6 medical conditions over the 3-year study period. This resulted in 1,811,759 patient admissions in the final study of hospital performance. There was a mean of 1862 and a median of 1628 patient admissions per hospital. The range of volume by hospital was 670 to 9314 cases. The number of patients in each medical condition and the frequency for each of the 4 AOs identified within each medical group is presented in Table 4, including additional deaths during or following readmission. The total deaths that included inpatient and 90-days of post-discharge follow-up were 29,992 (22.1%) for AMI; 31,512 (9.4%) for COPD; 81,577 (18.5%) for CHF; 58,333 (15.3%) for PNEU; 44,677 (16.8%) for CVA; and 26,261 (10.6%) for GIH. Among all 6 medical groups, patients with ≥1 AO was 41.0% (Table 5).

Table 5.

Total adverse outcomes among the 6 categories of medicare medical admissions in the 973 hospitals that met the minimum required number of cases.

A total of 62 hospitals (6.4%) had z-scores were <(–) 2 which reflected the best performing hospitals, and had median risk-adjusted AO rates of 34.5%. There were 72 hospitals (7.4%) that had z-scores >(+) 2 and median risk-adjusted AO rates of 48.6% which reflected suboptimal risk-adjusted performance.

Figure 2 presents the risk-adjusted AO rates for study hospitals that are presented by deciles of hospital performance. The range of risk-adjusted AO rates was from 30.7% in the best performing facility to 54.4% in the poorest. The first decile of hospitals had a median risk-adjusted AO rate of 35.1% (mean = 34.7%, odds ratio = 0.766; 95% CI [0.762, 0.771]) and the poorest performing decile was 47.8% (mean = 48.5%, odds ratio = 1.357; 95% CI [1.346, 1.369]). There was no relationship between the volume of cases admitted by hospital during this 3-year study period and the risk-adjusted AO rates by linear regression (R² = 0.0007; P = .40).

Figure 2.

This figure illustrates the risk-adjusted outcomes of each decile of hospitals in the study. The Error Bars represent the interquartile range of hospital outcomes for each decile.

4. Discussion

The results of this study validate the premise that risk-adjusted outcomes can be measured and that benchmarking hospital-specific results can identify opportunities for care improvement. Considerable variability in the risk-adjusted outcomes of hospitals that met minimum criteria for the volume of cases in the 6 medical conditions was observed. The range of AOs was 14% difference between the top performing facilities that were 2 z-scores better than average compared with those that were 2 z-scores poorer. The overall risk-adjusted AO rate was 41% with a 29.8% readmission rate being the major component. Over 50% of first readmissions for these medical cases occurred in the initial 30 days following discharge, but were common events in the 31 to 90-day post-discharge period, as has been reported by others.^[19] The results identify the reduction of readmissions as the most formidable challenge in the improvement of adverse outcomes in the Medicare population.

High rates of all-cause readmission within 90 days of discharge for both medical and surgical Medicare patients have been reported at 34%.^[20] The high rates of readmission for these medical conditions have resulted in some arguing that these events are the consequence of the fundamental severity of the patient's disease and are not effective measures of the quality of care that has been received by the patient.^[21,22] Recent studies indicate that hospital quality is a factor in readmission rates at 30-days following discharge.^[23] The results of our current study indicates significant differences in the rates of readmissions between top and suboptimal performing hospitals, and indicate that better results can be achieved. The interrelationship of readmissions among the 6 diagnoses studied here, and with selected other diagnoses (e.g., acute renal failure) suggests that global strategies be employed for improvement rather than those oriented toward a specific diagnosis.^[24] Similar patterns of readmissions have previously been reported for AMI, CHF, and PNEU.^[25]

We have used the prolonged risk-adjusted length-of-stay as a surrogate for inpatient complications. Our observations have been that complications of inpatient care are the major impetus for prolonged hospitalization. Coded complications are so common in the Medicare population of patients that they are of limited value when a modifier is absent to identify the severity of the complication. PrLOS in these medical patients had a significant odds ratio in predicting post-discharge deaths and readmissions, and appeared to be a valid measure of major complications in these medical conditions. The use of length-of-stay of the index hospitalization alone without risk-adjustment cannot be used as a metric for outcomes without including post-discharge readmissions. In an era of reduced length of stay for all hospitalizations, increased readmissions and post-discharge mortality must be evaluated to account for premature discharges and the shift of inpatient complications of care not being identified until after discharge.

The outcomes in these 6 medical conditions have many similarities. Best performing and suboptimal performing hospitals clearly had collinearity in performance and this required covariance coefficients to compute an accurate variance. Readmission over the 90-days following discharge had similar patterns for timing of first readmissions. The 45% rate of overall first readmissions using the Medicare criteria in the 31 to 90 days post-discharge time span is evidence of the severity of the primary disease and associated comorbidities in these patients. It will be a formidable challenge in chronic disease management to alter readmissions for these medical conditions. Yet another confounding factor will be the role of excluded readmissions for unrelated events and how interval non-associated readmissions will influence those readmissions that are attributed to the index hospitalization.

Many publications have emphasized better outcomes with increased volumes of specific cases for hospitals.^[26–30] We required a minimum of 75 cases for the 3-year study period for each of the 6 medical conditions for the hospital to be included. Among these hospitals no advantage was identified in risk-adjusted outcomes as a function of hospital volume. Hospitals with much smaller case volumes than we have studied may demonstrate a different relationship.

Like all studies with administrative data,^[31] our study has limitations. Inaccuracy in coding comorbidities and complications of care affects the precision needed for risk adjustment. All would agree that comprehensive clinical abstraction of the record would provide more accurate information. The addition of admission laboratory information and other easily imported elements of clinical information from the electronic medical record has the promise to enhance prediction models without expensive clinical abstraction of the medical record.^[32–34] The incorporation of socioeconomic data may provide the necessary non-medical variables to refine mortality and readmission models.^[35] While it is fashionable to criticize studies with administrative data, these data do provide an accurate accounting of adverse events following discharge. Clinical registries have their own limitations of being self-reported, commonly having limited duration of post-discharge follow-up, and do not routinely capture deaths and adverse events at other facilities.

Thus, the development of hybrid data bases that merge administrative and readily available clinic data together have the promise of enhanced predictive modeling for future outcomes-based research. Other future research prospects based upon the findings of this study include extension of outcomes in other medical conditions and the correlation of risk-adjusted costs for both the uncomplicated cases and for those with major adverse outcomes. Future research will need to better characterize the role of non-medical variables (e.g., patient income and educational level) especially in the post-discharge AOs.

The data in this study indicate that considerable room for improvement in outcomes among these medical admission groups is possible. Improvement depends on hospitals and physicians examining higher inpatient length-of-stay outliers and readmission rates when their performance is benchmarked to the national standard. While the policy road has been uneven in the pursuit of alternative payment models, prospective and episode-based payment models remain attractive and provide an intrinsic reward for fewer AOs. Reductions in the cost of care require better results. An examination of best performing hospitals compared with suboptimal performance in medical and surgical admissions demonstrates the biggest promise for lowering the cost of inpatient care in the Medicare population.

Acknowledgment

This manuscript was totally prepared by the authors without any other writers. The research reported herein was totally funded as a developmental project by MPA Healthcare Solutions without any external funding.

Author contributions

Conceptualization: Donald E. Fry, Susan M. Nedza, Michael Pine.

Data curation: Agnes M. Reband, Chun-Jung Huang.

Formal analysis: Donald E. Fry, Susan M. Nedza, Michael Pine, Agnes M. Reband, Chun-Jung Huang.

Funding acquisition: Gregory Pine.

Investigation: Donald E. Fry, Susan M. Nedza, Michael Pine.

Methodology: Donald E. Fry, Susan M. Nedza, Michael Pine, Agnes M. Reband, Chun-Jung Huang.

Project administration: Gregory Pine.

Resources: Gregory Pine.

Software: Chun-Jung Huang.

Supervision: Donald E. Fry, Gregory Pine.

Validation: Donald E. Fry, Susan M. Nedza, Agnes M. Reband, Chun-Jung Huang.

Visualization: Donald E. Fry.

Writing – original draft: Donald E. Fry.

Writing – review & editing: Donald E. Fry, Susan M. Nedza, Michael Pine, Agnes M. Reband, Chun-Jung Huang, Gregory Pine.

Donald E. Fry orcid: 0000-0002-5308-1671