Abstract
Purpose
We sought to examine variation in long-term acute care hospital (LTACH) quality based on 90-day in-hospital mortality for patients admitted for weaning from mechanical ventilation.
Methods
We developed an administrative risk-adjustment model using data from Medicare claims. We validated the administrative model against a clinical model using data from LTACHs participating in a 2002 to 2003 clinical registry. We then used our validated administrative model to assess national variation in 90-day in-hospital mortality rates in LTACHs from 2013.
Results
The administrative risk-adjustment model was derived using data from 9,447 patients admitted to 221 LTACHs in 2003. The model had good discrimination (C statistic=0.72) and calibration. Compared to a clinically derived model using data from 1,163 patients admitted to 14 LTACHs, the administrative model generated similar performance estimates. National variation in risk-adjusted mortality was assessed using data from 20,453 patients admitted to 380 LTACHs in 2013. LTACH-specific risk-adjusted mortality rates varied from 8.4% to 48.1% (median: 24.2%, interquartile range: 19.7% – 30.7%).
Conclusions
LTACHs vary widely in mortality rates, underscoring the need to better understand the sources of this variation and improve the quality of care for patients requiring long-term ventilator weaning.
Keywords: Intensive care, critical care, chronic critical illness, mechanical ventilation, hospitals
INTRODUCTION
Approximately 10% of critically ill patients develop persistent organ failures necessitating prolonged organ support, a condition known as chronic critical illness [1]. Chronic critical illness places an extraordinarily large burden on the health system. According to recent estimates approximately 380,000 patients in the United States develop chronic critical illness each year, accounting for over $26 billion in annual hospital spending [2]. Long-term outcomes for these patients are extremely poor, with only half of patients alive after one year [3–5]. As a consequence, developing novel strategies of care for these patients is a high priority for clinicians, health administrators, and health policy makers [6].
One such strategy is the transfer of patients with chronic critical illness from a traditional short-stay hospital to a long-term acute care hospital (LTACH). LTACHs are specialized facilities that focus on the care of patients with chronic critical illness. Defined by the United States Centers for Medicare and Medicaid services as hospitals with a geometric mean length of stay ≥25 days, LTACHs provide holistic medical care and rehabilitation services to patients requiring long-term weaning from mechanical ventilation and those with other types of prolonged acute conditions [7]. Due to favorable reimbursement models and increases in the incidence of chronic critical illness, LTACHs are among the fastest growing segments of acute care in the United States [8]. At last count there were 426 United States LTACHs in operation accounting for over $5.3 billion in Medicare spending overall [9].
Despite the growing importance of LTACHs in the United States health care system, little is known about them in terms of the quality of care they provide [10,11]. What research exists has focused on the effectiveness of LTACHs compared to other sites of care for chronic critical illness, such as short stay hospitals and skilled nursing facilities [12]. This research generally shows that outcomes after LTACH admission are comparable to outcomes in short stay hospital ICUs [12]. However, extremely little is known about how LTACHs compare to each other in terms of whether or not they vary in the quality of care they provide. Such information is essential to understanding not only the quality of LTACHs but also the quality of care for patients with chronic critical illness in general. To the degree that LTACHs vary in quality, information on care delivery in high-performing LTACHs could be used to develop a set of evidence-based practices that can be used to improve care in other LTACHs.
To better understand this issue, we set out to quantify variation in outcomes across US LTACHs. We focused on patients transferred from short stay hospitals to LTACHs for weaning from mechanical ventilation. Such patients are the archetypal chronic critical illness patients and are the single most common type of patient admitted to LTACHs. Additionally, a focus on weaning from long-term mechanical ventilation is among the most distinguishing features of LTACHs, such that understanding the quality of care for these patients is central to understanding LTACH quality overall.
METHODS
Overview of methodological approach
We used administrative claims data from fee-for-service Medicare beneficiaries to estimate LTACH-specific risk-adjusted 90-day in-hospital mortality rates for patients transferred from short stay hospitals for weaning from mechanical ventilation. We used Medicare data because they are the only national data source for LTACH admissions. We chose 90-day in-hospital mortality as our outcome of interest because it is patient-centered and easily measurable in both administrative and clinical data.
To perform our study in Medicare claims it was first necessary to develop and validate an administrative model suitable for profiling LTACHs on their 90-day in-hospital mortality for ventilated patients. To accomplish this, we developed a novel administrative risk-adjustment model which we then validated by (a) comparing LTACH-specific risk-adjusted mortality rates derived from our administrative model to rates for those same LTACHs derived from a granular clinical dataset; and by (b) examining LTACH-specific risk-adjusted mortality rates derived from our administrative model over time. Under this approach, a valid administrative model would produce similar mortality estimates to a clinical model and would produce mortality estimates that are stable from year to year. This same approach has been successfully applied by Medicare contractors in several other populations [13,14].
Data sources
For our clinical model, we used data from the Ventilation Outcomes Study, a previously published national evaluation of LTACH patient outcomes [15,16]. The Ventilation Outcomes Study database contains detailed demographic and clinical variables on 1,419 ventilated patients admitted to 23 LTACHs across a 12 month period in 2002 and 2003. The data include laboratory values and physiological results from the day of admission as well as in-hospital mortality out to one year from the day of admission. Due to the terms of the data use agreement and to assure reliable risk-adjustment, we restricted the cohort to only LTACHs that consented to allow their de-identified data to be repurposed for this study and that had sufficient case volumes defined as at least 10 patients in the data set, resulting in 1,163 patients from 14 LTACHs.
For our administrative model, we used data from the Medicare Provider Analysis and Review (MedPAR) files. MedPAR contains the final action claims for all hospitalized fee-for-service Medicare beneficiaries in the United States, including hospitalizations at both traditional short stay hospitals and LTACHs. We used MedPAR data from three years: 2003 and 2004 (to approximate the timeframe of the Ventilator Outcomes Study), and 2013 (to approximate a recent year). MedPAR was used to obtain dates of admission and discharge, patient demographics, and International Classification of Diseases—version 9.0—clinical modification (ICD-9-CM) diagnosis and procedure codes for each hospitalization. MedPAR files were linked to the Medicare Healthcare Cost Report Information System (HCRIS) to obtain hospital characteristics. LTACHs and short-stay hospitals were identified using hospital identifiers in HCRIS [17].
Clinical model development
We used data from the Ventilator Outcomes Study to build a robust clinical risk-adjustment model for 90-day risk-adjusted in-hospital mortality. We identified candidate risk-adjustment variables from the list of available elements in the Ventilator Outcomes Study data and prior studies of the clinical factors associated with risk-adjusted outcome in critically ill patients [18,19]. Candidate variables were limited to those collected on the day of LTACH admission and included age, pre-LTACH admission hospital length of stay, co-morbid conditions (congestive heart failure, coronary artery disease, chronic obstructive lung disease, and stroke), vital signs (heat rate, blood pressure, temperature, and respiratory rate), laboratory values (PaO2, PaCO2, hematocrit, white blood cell count, creatinine, blood urea nitrogen, sodium, albumin, bilirubin, and glucose), Glasgow coma scale (excluding verbal points since all patients were receiving invasive mechanical ventilation), and 24-hour urine output.
We treated age and pre-hospital length-of-stay as quadratic splines. We categorized continuous variables, other than age and prehospital length-of-stay, based on visual inspection of locally weighted scatterplot smoothing plots and imputed missing variables into the normal category, as previously performed [19]. We treated these variables and other categorical variables as indicator covariates. We then performed step-wise logistic regression based on Akaike’s Information Criterion to create a parsimonious model for 90-day risk-adjusted in-hospital mortality [20]. For the final model we examined discrimination using the C-statistic and calibration using the slope and intercept of a fitted plot of observed versus predicted mortality [21].
Lastly, we used hierarchical logistic regression to create hospital-specific risk-adjusted in-hospital mortality rates based upon the final model. To create the risk-adjusted mortality rates we divided hospital-specific predicted mortality (estimated using logistic regression including a hospital-specific random effect) by hospital-specific expected mortality (estimated using logistic regression) and then multiplied by the average mortality in the sample [22]. This approach creates hospital-specific mortality rates that are both risk-adjusted, in that they account for variation in base-line risk across hospitals, and reliability-adjusted, in that they account for low reliability in small hospitals by moving the estimate toward the population average. This approach is similar to the method that Medicare uses to profile hospitals based on their risk-adjusted mortality [13,14].
Administrative model development
We used MedPAR to build a robust administrative model for 90-day risk-adjusted in-hospital mortality. We identified patients that were transferred from short-stay hospitals to LTACHs for ventilator weaning using a previously validated method [17]. Specifically, we identified mechanical ventilation using ICD-9-CM procedure codes [23] and diagnosis related group codes, and considered patients to be transferred to an LTACH for weaning when there were temporally adjacent admissions at a short stay hospital and at an LTACH, both of which involved mechanical ventilation [17]. To prevent interdependence of observations, for patients with more than one eligible admission within an analysis year, we selected one random admission per year. As in the clinical data set, we limited the analysis to LTACHs with at least 10 admissions within the analysis year. We identified candidate variables from the set of variables typically available in administrative claims data, specifically focusing on variables found to be associated with long-term mortality in a prior multicenter cohort study [5]. Candidate variables were restricted to those available on the day of LTACH admission and included age, pre-LTACH admission hospital length of stay, co-morbid conditions defined in the manner of Elixhauser using codes from the originating short stay hospital [24], and selected diagnoses and procedures from the originating short stay hospital including coronary artery-bypass grafting, heart valve replacement, shock, thrombocytopenia, traumatic injury, ischemic stroke, and hemorrhagic stroke, all based on ICD-9-CM codes [25].
We treated continuous variables (age and pre-hospital length-of-stay) as quadratic splines. We treated categorical variables (all others) as indicator covariates. We then performed logistic regression to estimate a model for 90-day risk-adjusted in-hospital mortality. We did not perform any variable reduction steps for this model. Instead we retained all covariates hypothesized to be related to mortality in prior work [5]. We examined discrimination using the C-statistic and calibration using the slope and intercept of a fitted plot of observed versus predicted mortality [21].
As with the clinical model, we used hierarchical logistic regression to create hospital-specific risk-adjusted in-hospital mortality based on the administrative model. We separately fit the model in 2003, 2004, and 2013 data, allowing year-specific model covariates and yielding three sets of hospital-specific risk-adjusted mortality rates.
Administrative model validation
We validated our administrative model in two ways. First, we compared risk-adjusted mortality rates from the 2003 clinical model and the 2003 administrative model in the 14 hospitals participating in the Ventilator Outcomes Study using Spearman’s rank correlation. We hypothesized that with a valid administrative model the risk-adjusted mortality rates and hospital rankings based on those rates would be similar across the two models. Second, we compared risk-adjusted mortality rates from the 2003 administrative model and the 2004 administrative model for all US LTACHs with data in both years using Spearman’s rank correlation. We hypothesized that with a valid administrative model the risk-adjusted mortality rates and hospital rankings based on those rates would be similar across the two years.
Variation in mortality rates across LTACHs
Finally, we used the 2013 risk-adjusted 90-day in-hospital mortality rates from the administrative model as a marker of LTACH performance, examining the variation in risk-adjusted mortality across hospitals. We graphically examined variation using a caterpillar plot in which we plotted each LTACH’s risk-adjusted mortality rate against its relative rank. Confidence intervals for the risk adjusted mortality rate were derived using the bootstrap. We also tested the relationship between LTACH-specific risk-adjusted 90-day mortality rates and key hospital characteristics, including ownership status (non-profit, for-profit, or government), co-location status (free-standing or co-located within a short-stay hospital), size (based on total number of beds), and percent of patients receiving mechanical ventilation (as identified in MedPAR). The goal of this analysis was to provide additional construct validity, since under a valid performance assessment model we would expect performance to vary by key LTACH characteristics [26]. To perform these tests, we compared the mean RAMR across categories using analysis of variance.
All analyses were performed with either SAS 9.2 (SAS Institute, Cary, North Carolina) or Stata 15.0 (StataCorp LLC, College Station, Texas). We considered a p-value of 0.05 to be statistically significant. This work was reviewed and approved by the University of Pittsburgh Human Research Protection Office.
RESULTS
Clinical model
The clinical model was developed in 1,163 patients in 14 hospitals. Of these patients, 239 (20.6%) died in the hospital by day 90. Demographic and clinical characteristics of both survivors and decedents are shown in Supplementary Table S1. The number of patients per hospital ranged from 11 to 280 (median: 53; interquartile range: 30 – 104). Unadjusted 90-day in-hospital mortality across LTACHs ranged from 0.0% to 41.7% (median: 18.2%; interquartile range: 8.0% – 23.9%). The final risk-adjustment model included the following variables: gender, age, short-stay hospital length-of-stay, white blood count, 24-hour urine output, blood urea nitrogen, albumin, glucose, and Glasgow Coma Scale. The full model results are shown in Supplementary Table S2. The model C-statistic was 0.76, the slope of the fitted calibration line was 1.07 (p=0.41 for the test that the slope is different than 1.0), and the intercept of the fitted calibration line was −0.01 (p=0.56 for the test that the intercept is different than 0). Based on this model, hospital-specific risk-adjusted 90-day in-hospital mortality rates across LTACHs ranged from 12.2% to 41.7% (median: 20.0%; interquartile range: 16.3% – 23.2%).
Administrative model
In the 2003 MedPAR data there were 9,447 eligible patients in 221 LTACHs. Of these patients, 3,283 (34.8%) died in the hospital by day 90. Demographic and clinical characteristics of both survivors and decedents are shown in Table 1. The number of patients per hospital ranged from 10 to 223 (median: 35; interquartile range: 22 – 54). Unadjusted 90-day in-hospital mortality across LTACHs ranged from 0% to 65.2% (median: 30.2%; interquartile range: 20.7% – 42.1%). The full model results are shown in Table 2. The model C-statistic was 0.72, the slope of the fitted calibration line was 1.08 (p=0.05 for the test that the slope is different than 1.0), and the intercept of the fitted calibration line was −0.03 (p=0.08 for the test that the intercept is different than 0).
Table 1.
Demographic and clinical characteristics of patients in the 2003 administrative cohort, by 90-day in-hospital mortality
| All patients (n=9,447) |
Survivors (n=6,164) |
Decedents (n=3,283) |
|
|---|---|---|---|
| Age | 72.2 ± 11.0 | 70.7 ± 11.4 | 75.0 ± 9.7 |
| Male Gender | 4,672 (49%) | 2,981 (48%) | 1,691 (52%) |
| Short stay hospital length of stay | 27.1 ± 16.8 | 26.7 ± 16.8 | 27.8 ± 16.9 |
| Short stay procedures and diagnoses | |||
| CABG | 742 (8%) | 485 (8%) | 257 (8%) |
| Dialysis | 473 (5%) | 266 (4%) | 207 (6%) |
| Hypotension | 660 (7%) | 400 (6%) | 260 (8%) |
| PTCA | 115 (1%) | 72 (1%) | 43 (1%) |
| Hemorrhagic Stroke | 264 (3%) | 176 (3%) | 88 (3%) |
| Ischemic Stroke | 548 (6%) | 346 (6%) | 202 (6%) |
| Thrombocytopenia | 253 (3%) | 182 (3%) | 71 (2%) |
| Trauma | 766 (8%) | 538 (9%) | 228 (7%) |
| Valve Replacement | 345 (4%) | 203 (3%) | 142 (4%) |
| Comorbidities | |||
| AIDS | 13 (0%) | 6 (0%) | 7 (0%) |
| Alcohol abuse | 250 (3%) | 172 (3%) | 78 (2%) |
| Deficiency Anemias | 557 (6%) | 375 (6%) | 182 (6%) |
| Rheumatoid arthritis/collagen vas | 106 (1%) | 70 (1%) | 36 (1%) |
| Chronic blood loss anemia | 269 (3%) | 156 (3%) | 113 (3%) |
| Congestive heart failure | 3,519 (37%) | 2,171 (35%) | 1,348 (41%) |
| Chronic pulmonary disease | 3,747 (40%) | 2,421 (39%) | 1,326 (40%) |
| Coagulopathy | 950 (10%) | 581 (9%) | 369 (11%) |
| Depression | 100 (1%) | 80 (1%) | 20 (1%) |
| Diabetes w/o chronic complications | 1,004 (11%) | 676 (11%) | 328 (10%) |
| Diabetes w/ chronic complications | 262 (3%) | 159 (3%) | 103 (3%) |
| Drug abuse | 67 (1%) | 48 (1%) | 19 (1%) |
| Hypertension | 2,244 (24%) | 1,452 (24%) | 792 (24%) |
| Hypothyroidism | 221 (2%) | 148 (2%) | 73 (2%) |
| Liver disease | 119 (1%) | 67 (1%) | 52 (2%) |
| Lymphoma | 52 (1%) | 25 (0%) | 27 (1%) |
| Fluid and electrolyte disorders | 3,722 (39%) | 2,384 (39%) | 1,338 (41%) |
| Metastatic cancer | 207 (2%) | 87 (1%) | 120 (4%) |
| Other neurological disorders | 1,107 (12%) | 773 (13%) | 334 (10%) |
| Obesity | 170 (2%) | 136 (2%) | 34 (1%) |
| Paralysis | 357 (4%) | 291 (5%) | 66 (2%) |
| Peripheral vascular disease | 237 (3%) | 141 (2%) | 96 (3%) |
| Psychoses | 180 (2%) | 130 (2%) | 50 (2%) |
| Pulmonary circulation disease | 340 (4%) | 229 (4%) | 111 (3%) |
| Renal failure | 973 (10%) | 575 (9%) | 398 (12%) |
| Solid tumor w/out metastasis | 178 (2%) | 87 (1%) | 91 (3%) |
| Peptic ulcer Disease × bleeding | 4 (0%) | 3 (0%) | 1 (0%) |
| Valvular disease | 882 (9%) | 536 (9%) | 346 (11%) |
| Weight loss | 1,841 (19%) | 1,195 (19%) | 646 (20%) |
Values are frequency (percent) or mean ± standard deviation
CABG = coronary artery bypass grafting; PTCA = percutaneous transluminal coronary angioplasty; AIDS =acquired immunodeficiency syndrome
Table 2.
Results of the administrative risk adjustment model for 90-day in-hospital mortality from the 2003 cohort
| OR | SE | 95% CI | T | |
|---|---|---|---|---|
| Male | 1.24 | (0.06) | [1.13 – 1.36] | 4.45 |
| Short stay hospital length of stay (linear spline term) | ||||
| ≤10 days | 1.02 | (0.14) | [0.77 – 1.35] | 0.14 |
| 10 to 20 days | 1.01 | (0.05) | [0.92 – 1.11] | 0.17 |
| 20 to 30 days | 1.01 | (0.04) | [0.94 – 1.10] | 0.34 |
| 30+ days | 1.00 | (0.00) | [0.99 – 1.01] | −0.39 |
| Short stay hospital length of stay (quadratic spline term) | ||||
| ≤10 days | 1.00 | (0.01) | [0.98 – 1.02] | 0.00 |
| 10 to 20 days | 1.00 | (0.00) | [0.99 – 1.01] | −0.10 |
| 20 to 30 days | 1.00 | (0.00) | [0.99 – 1.01] | 0.09 |
| 30+ days | 1.00 | (0.00) | [1.00 - 1.00] | 0.55 |
| Age (linear spline term) | ||||
| ≤70 | 0.99 | (0.05) | [0.90 – 1.09] | −0.26 |
| 70 to 75 | 1.02 | (0.08) | [0.87 – 1.19] | 0.23 |
| 75 to 80 | 1.16 | (0.10) | [0.98 – 1.37] | 1.70 |
| 80+ | 1.07 | (0.03) | [1.02 – 1.13] | 2.55 |
| Age (quadratic spline term) | ||||
| ≤70 | 1.00 | (0.00) | [1.00 - 1.00] | 0.91 |
| 70 to 75 | 1.00 | (0.02) | [0.97 – 1.03] | 0.15 |
| 75 to 80 | 0.98 | (0.02) | [0.95 – 1.01] | −1.09 |
| 80+ | 1.00 | (0.00) | [0.99 – 1.00] | −1.17 |
| CABG | 0.86 | (0.08) | [0.72 – 1.04] | −1.54 |
| PTCA | 1.55 | (0.17) | [1.25 – 1.92] | 3.97 |
| Valve replacement | 1.08 | (0.23) | [0.72 – 1.63] | 0.39 |
| Dialysis | 1.28 | (0.16) | [0.99 – 1.64] | 1.89 |
| Hypotension | 1.09 | (0.17) | [0.80 – 1.48] | 0.55 |
| Thrombocytopenia | 1.18 | (0.17) | [0.89 – 1.58] | 1.16 |
| Trauma | 1.16 | (0.12) | [0.95 – 1.41] | 1.49 |
| Ischemic Stroke | 0.81 | (0.12) | [0.60 – 1.09] | −1.41 |
| Hemorrhagic Stroke | 0.79 | (0.07) | [0.66 – 0.95] | −2.58 |
| Comorbidities | ||||
| AIDS | 4.13 | (2.56) | [1.22 – 13.94] | 2.28 |
| Alcohol abuse | 0.92 | (0.14) | [0.68 – 1.24] | −0.55 |
| Deficiency Anemias | 0.98 | (0.10) | [0.80 – 1.20] | −0.20 |
| Rheumatoid arthritis/collagen vas | 1.43 | (0.32) | [0.93 – 2.21] | 1.62 |
| Chronic blood loss anemia | 1.24 | (0.17) | [0.95 – 1.62] | 1.60 |
| Congestive heart failure | 1.16 | (0.06) | [1.05 – 1.27] | 2.96 |
| Chronic pulmonary disease | 1.05 | (0.05) | [0.95 – 1.16] | 0.99 |
| Coagulopathy | 1.10 | (0.15) | [0.85 – 1.42] | 0.70 |
| Depression | 0.68 | (0.18) | [0.40 – 1.16] | −1.41 |
| Diabetes w/o chronic complications | 1.05 | (0.08) | [0.90 – 1.23] | 0.67 |
| Diabetes w/ chronic complications | 1.35 | (0.19) | [1.02 – 1.78] | 2.13 |
| Drug abuse | 1.17 | (0.34) | [0.65 – 2.08] | 0.52 |
| Hypertension | 0.92 | (0.07) | [0.80 – 1.06] | −1.13 |
| Hypothyroidism | 1.07 | (0.17) | [0.78 – 1.46] | 0.43 |
| Liver disease | 2.04 | (0.42) | [1.36 – 3.04] | 3.48 |
| Lymphoma | 2.26 | (0.67) | [1.27 – 4.05] | 2.75 |
| Fluid and electrolyte disorders | 1.11 | (0.05) | [1.00 – 1.22] | 2.05 |
| Metastatic cancer | 2.84 | (0.43) | [2.10 – 3.83] | 6.82 |
| Other neurological disorders | 0.92 | (0.07) | [0.79 – 1.07] | −1.09 |
| Obesity | 0.80 | (0.17) | [0.53 – 1.21] | −1.06 |
| Paralysis | 0.55 | (0.08) | [0.41 – 0.73] | −4.01 |
| Peripheral vascular disease | 1.32 | (0.19) | [0.99 – 1.75] | 1.92 |
| Psychoses | 1.17 | (0.21) | [0.82 – 1.67] | 0.87 |
| Pulmonary circulation disease | 1.10 | (0.14) | [0.86 – 1.41] | 0.75 |
| Renal failure | 1.37 | (0.14) | [1.12 – 1.66] | 3.14 |
| Solid tumor w/out metastasis | 2.22 | (0.36) | [1.61 – 3.05] | 4.85 |
| Peptic ulcer Disease × bleeding | 0.47 | (0.58) | [0.04 – 5.21] | −0.62 |
| Valvular disease | 1.16 | (0.09) | [0.99 – 1.36] | 1.80 |
| Weight loss | 1.03 | (0.06) | [0.92 – 1.16] | 0.57 |
| Constant | 0.07 | (0.10) | [0.00 – 1.16] | −1.86 |
OR = odds ratio; SE = standard error; CI = confidence interval; CABG = coronary artery bypass grafting; PTCA = percutaneous transluminal coronary angioplasty; AIDS =acquired immunodeficiency syndrome
Spearman’s rank correlation comparing risk-adjusted mortality rates between the clinical model and the 2003 administrative model for the 14 hospitals with estimates from both models was 0.49 (p=0.08), suggesting modest correlation (Figure 1). Spearman’s rank correlation comparing risk-adjusted mortality rates between the 2003 administrative model and the 2004 administrative model for the 216 hospitals with data in both models was 0.70 (p <0.001), suggesting that the administrative model produced results that were stable across time (Figure 2). Together these analyses suggested that the administrative model produced valid estimates of LTACH performance based on risk-adjusted 90-day in-hospital mortality.
Figure 1.
Scatterplot of risk-adjusted mortality rates for the 14 hospitals in the Ventilator Outcomes Study, comparing rates from the clinical model (Y axis) to rates from the administrative model (X axis). Each dot represents one hospital. RAMR = risk-adjusted mortality rate
Figure 2.
Scatterplot of risk-adjusted mortality rates for the 216 hospitals in both the 2003 and 2004 administrative data, comparing rates from the 2004 model (Y axis) to rates from the 2003 model (X axis). Each dot represents one hospital. RAMR = risk-adjusted mortality rate
Variation in mortality rates
Hospital-specific risk-adjusted 90-day in-hospital mortality rates for 380 LTACHs in the 2013 MedPAR data are shown in Figure 3. Mortality rates varied widely, ranging from 8.4% to 48.1% (median: 24.2%; interquartile range: 19.7% – 30.7%). Examining risk-adjusted 90-day in-hospital mortality rates across hospital types, we found lower mortality rates among non-profit LTACs, free-standing LTACs, and LTACHs with a higher percentage of patients receiving mechanical ventilation compared to a lower proportion (Table 3). There was no significant association between mortality rates and the total bed count or the total number of mechanically ventilated patients.
Figure 3.
Caterpillar plot of risk-adjusted mortality rates from the 2013 administrative model. Each long-term acute care hospital is represented as one dot on the plot along with its 95% confidence interval.
Table 3.
Relationship between long-term acute care hospital characteristics and risk-adjusted 90-day in-hospital mortality rates for 380 hospitals in the 2013 administrative data.
| Count | Mean (SD) | p-value | |
|---|---|---|---|
| Ownership | |||
| Non-profit | 65 | 0.229 (0.079) | 0.025 |
| For-profit | 301 | 0.257 (0.074) | |
| Government | 14 | 0.239 (0.070) | |
| Co-location status | |||
| Free-standing | 123 | 0.237 (0.074) | 0.035 |
| Co-located | 179 | 0.258 (0.074) | |
| Unknown | 78 | 0.259 (0.077) | |
| Total Beds | |||
| Quartile 1: 16 – 35 | 97 | 0.242 (0.066) | 0.530 |
| Quartile 2: 36 – 50 | 102 | 0.255 (0.077) | |
| Quartile 3: 51 – 68 | 86 | 0.255 (0.074) | |
| Quartile 4: 69 – 667 | 95 | 0.255 (0.083) | |
| Count of ventilated patients | |||
| Quartile 1: 10 – 35 | 98 | 0.245 (0.065) | 0.306 |
| Quartile 2: 36 – 54 | 92 | 0.246 (0.075) | |
| Quartile 3: 55 – 87 | 95 | 0.252 (0.083) | |
| Quartile 4: 88 – 439 | 95 | 0.263 (0.078) | |
| Percent of patients ventilated | |||
| Quartile 1: 0.0243 – 0.1391 | 95 | 0.266 (0.071) | <0.001 |
| Quartile 2: 0.1398 – 0.2154 | 95 | 0.260 (0.080) | |
| Quartile 3: 0.2172 – 0.3008 | 95 | 0.251 (0.076) | |
| Quartile 4: 0.3019 – 0.6761 | 95 | 0.230 (0.071) |
SD = standard deviation
DISCUSSION
Using a newly-developed and validated administrative risk-adjustment model, we found wide variation in hospital-specific risk-adjusted 90-day in-hospital mortality rates for patients admitted to LTACHs for ventilator weaning. At the best performing LTACH in our sample, only 8.4% of patients were dead at 90 days adjusting for risk and model reliability, while at the worst performing LTACH in our sample 48.1% of patients were dead at 90 days. These findings underscore the need to better understand the sources of this variation and improve the quality of care for patients requiring long-term ventilator weaning in an LTACH setting.
At present there are few evidence-based practices known to be associated with improved survival in long-term ventilator patients. A randomized trial of two weaning strategies found that a strategy of daily trach-collar trials may confer a survival benefit compared to a strategy of titrated pressure support ventilation in LTACHs [27]. However, there are few other studies to guide clinicians that care for long-term ventilator patients, either in the specific processes of care or the overall structure of care. A much larger body of evidence exists for patients receiving mechanical ventilation in acute care ICUs, where numerous care structures or processes are known to be associated with improved clinical outcomes. These include intensivist physician staffing, lower nurse to patient ratios, high nurse education levels, daily interprofessional rounds, daily interruption of continuous sedation, and the use of rounding checklists, among others [28–33]. Whether or not these practices are equally beneficial in the LTACH setting is unknown.
Identifying the best clinical and organizational practices in LTACHs will require a comprehensive and diverse program of comparative effectiveness research [11], including both randomized clinical trials and observational studies [34]. Observational research in LTACHs could take the form of large multi-center cohort studies or positive-deviance studies in which the best-practices at high-performing LTACHs are used to create a performance improvement framework that can be applied at low-performing LTACHs [35]. By quantifying variation in LTACH performance based on risk-adjusted mortality rates, our study provides a valuable starting place for such work.
In addition to quantifying LTACH performance based on risk-adjusted mortality rates, we found that non-profit LTACHs and LTACHs caring for a higher proportion of mechanically ventilated patients demonstrated lower risk-adjusted mortality. The findings about for-profit status mirror the literature in nursing homes, in which for-profit nursing homes generally provide lower quality care compared to non-profit nursing homes [36]. The finding about proportion of mechanically ventilated patients mirrors the literature in acute care ICUs, in which higher volume hospitals generally provide a higher quality of care [37,38]. Although these findings should prompt further investigations into potential mechanisms of action, it is important not to over-interpret them since we cannot rule out the possibility that these factors are just markers for other determinants of performance. We examined the relationship between LTACH characteristics and RAMRs in order to help provide additional construct validity to our performance assessment strategy. Providing definitive evidence of these relationships would require more directed study.
In interpreting our results it is important to consider that outcomes beyond mortality may be of interest when assessing the quality of LTACHs. Other important outcomes may include receipt of evidence-based care processes; physical functioning and emotional well-being among survivors; and a high-quality death among decedents. The latter issue is particularly salient among the LTACH population, since many patients would prefer to avoid high-intensity treatment at the end-of-life, and LTACHs should not be adversely benchmarked for providing end-of-life care. This problem is common to all mortality-based performance assessment and will require future research to solve [39].
It is also important to note that although we quantified variation in LTACH performance, we did not seek to demonstrate performance outliers, i.e. we did not assess whether any one LTACH’s peformance was statistically signficantly different than the average. Although such information can be used to inform hospital choice and is a standard part of United States CMS quality reporting, we were did not feel that this approach would provide substantial new insight into variation in performance above our present analyses. Rather, our goal was to demonstrate that variation exists in a way that can help guide LTACH performance improvement, regardless of whether there are statistically significant performance outliers.
Our study has several limitations. The Ventilator Outcomes Study data which we used to validate our administrative model are now over 15 years old, and data were only available for a small number of hospitals, limiting their utility in terms of validation. However, there are no other multi-center clinical datasets in the LTACH population, making these data uniquely valuable for this purpose. Moreover, although we validated our administrative risk-adjustment model, it is likely that some of the variation in risk-adjusted mortality rates we observe are due to unmeasured variation in base-line risk. However, no risk-adjustment model can perfectly account for risk, and model accuracy is similar to other models used to assess hospital performance in administrative data. That said, stronger correlation between our administrative and clinical model would have provided stronger support for the validity of the model. Additionally, we only examined patients transferred to LTACHs for ventilator weaning. In the administrative data set our approach for identifying these patients is unlikely to be perfectly accurate, creating the possibility for bias. And although these patients represent an important demographic in the LTACH population, LTACHs provide care for many patient that do not receive mechanical ventilation [40]. Our study does not provide insight into LTACH performance for these patients.
Despite these limitations, our study provides an important roadmap for efforts to improve the quality of care in LTACHs and improve outcomes for patients with chronic critical illness. Future work should build off our findings by identifying the clinical and organizational factors common in higher-performing LTACHs and using these to develop a framework for evidence-based health care for patients receiving prolonged mechanical ventilation.
Supplementary Material
HIGHLIGHTS.
Long-term acute care hospitals (LTACHs) are an increasingly common site of care for patients requiring prolonged mechanical ventilation.
We developed and validated an administrative risk-adjustment model using data from the United States Medicare program.
LTACH-specific risk-adjusted mortality rates varied from 8.4% to 48.1%.
Non-profit hospitals and hospitals with a relatively higher percentage of mechanically ventilated patients tended to have lower risk-adjusted mortality rates.
Future work should attempt to understand the sources of this variation and improve the quality of care for patients requiring long-term ventilator weaning.
Acknowledgments
No individuals substantially contributed to this study outside the listed authors.
Funding: This work was supported by the National Institutes of Health (R01HL096651).
Dr. Kahn has received travel reimbursement and speaking honoraria from the National Association of Long Term Care Hospitals.
LIST OF ABBREVIATIONS
- AIDS
acquired immunodeficiency syndrome
- CABG
coronary artery bypass grafting
- CI
confidence interval
- HCRIS
Healthcare Cost Report Information System
- ICD-9-CM
International Classification of Diseases—version 9.0—clinical modification
- LTACH
long-term acute care hospital
- MedPAR
Medicare Provider Analysis and Review
- RAMR
risk-adjusted mortality rates
- OR
odds ratio
- PTCA
percutaneous transluminal coronary angioplasty
- SE
standard error
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Disclosures: Other authors have nothing to disclose.
References
- 1.Nelson JE, Cox CE, Hope AA, Carson SS. Chronic critical illness. American Journal of Respiratory and Critical Care Medicine. 2010;182:446–54. doi: 10.1164/rccm.201002-0210CI. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kahn JM, Le T, Angus DC, Cox CE, Hough CL, White DB, et al. The epidemiology of chronic critical illness in the United States. 2015;43:282–7. doi: 10.1097/CCM.0000000000000710. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cox CE, Carson SS, Lindquist JH, Olsen MK, Govert JA, Chelluri L, et al. Differences in one-year health outcomes and resource utilization by definition of prolonged mechanical ventilation: a prospective cohort study. Crit Care. 2007;11:R9. doi: 10.1186/cc5667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Rose L, Fraser IM. Patient characteristics and outcomes of a provincial prolonged-ventilation weaning centre: a retrospective cohort study. Can Respir J. 2012;19:216–20. doi: 10.1155/2012/358265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Carson SS, Kahn JM, Hough CL, Seeley EJ, White DB, Douglas IS, et al. A multicenter mortality prediction model for patients receiving prolonged mechanical ventilation. 2012;40:1171–6. doi: 10.1097/CCM.0b013e3182387d43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lamas D. Chronic critical illness. N Engl J Med. 2014;370:175–7. doi: 10.1056/NEJMms1310675. [DOI] [PubMed] [Google Scholar]
- 7.Liu K, Baseggio C, Wissoker D, Maxwell S, Haley J, Long S. Long-term care hospitals under Medicare: facility-level characteristics. Health Care Financ Rev. 2001;23:1–18. [PMC free article] [PubMed] [Google Scholar]
- 8.Kahn JM, Benson NM, Appleby D, Carson SS, Iwashyna TJ. Long-term acute care hospital utilization after critical illness. Jama. 2010;303:2253–9. doi: 10.1001/jama.2010.761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Medicare Payment Advisory Commission. Report to the congress: Medicare payment policy. Washington, D.C.: MedPAC; 2017. Long-term care hospital services. [Google Scholar]
- 10.de Lissovoy G, Pronovost PJ, Faden R. Long-term acute care hospitals: a clinical, economic, and ethical dilemma. Medical Care. 2012;51:1–3. doi: 10.1097/MLR.0b013e31827a8cb7. [DOI] [PubMed] [Google Scholar]
- 11.Kahn JM, Carson SS. Generating evidence on best paractice in long-term acute care hospitals. Jama. 2013;309:719–20. doi: 10.1001/jama.2013.848. [DOI] [PubMed] [Google Scholar]
- 12.Kahn JM, Werner RM, David G, Have Ten TR, Benson NM, Asch DA. Effectiveness of Long-term Acute Care Hospitalization in Elderly Patients With Chronic Critical Illness. Medical Care. 2013;51:4–10. doi: 10.1097/MLR.0b013e31826528a7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006;113:1693–701. doi: 10.1161/CIRCULATIONAHA.105.611194. [DOI] [PubMed] [Google Scholar]
- 14.Bratzler DW, Normand S-LT, Wang Y, O'Donnell WJ, Metersky M, Han LF, et al. An administrative claims model for profiling hospital 30-day mortality rates for pneumonia patients. PLoS ONE. 2011;6:e17401. doi: 10.1371/journal.pone.0017401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Scheinhorn DJ, Hassenpflug MS, Votto JJ, Chao DC, Epstein SK, Doig GS, et al. Ventilator-dependent survivors of catastrophic illness transferred to 23 long-term care hospitals for weaning from prolonged mechanical ventilation. Chest. 2007;131:76–84. doi: 10.1378/chest.06-1079. [DOI] [PubMed] [Google Scholar]
- 16.Scheinhorn DJ, Hassenpflug MS, Votto JJ, Chao DC, Epstein SK, Doig GS, et al. Post-ICU mechanical ventilation at 23 long-term care hospitals: a multicenter outcomes study. Chest. 2007;131:85–93. doi: 10.1378/chest.06-1081. [DOI] [PubMed] [Google Scholar]
- 17.Kahn JM, Iwashyna TJ. Accuracy of the discharge destination field in administrative data for identifying transfer to a long-term acute care hospital. BMC Res Notes. 2010;3:205. doi: 10.1186/1756-0500-3-205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–10. doi: 10.1007/BF01709751. [DOI] [PubMed] [Google Scholar]
- 19.Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today's critically ill patients. 2006;34:1297–310. doi: 10.1097/01.CCM.0000215112.84523.F0. [DOI] [PubMed] [Google Scholar]
- 20.Akaike H. Fitting autoregressive models for prediction. Annals of the Institute of Statistical Mathematics. 1969;21:243–7. [Google Scholar]
- 21.Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35:2052–6. doi: 10.1097/01.CCM.0000275267.64078.B0. [DOI] [PubMed] [Google Scholar]
- 22.Normand S, Glickman ME. Statistical methods for profiling providers of medical care: issues and applications. Journal of the American Statistical Association. 1997;92:803–14. [Google Scholar]
- 23.Wunsch H, Kramer A, Gershengorn HB. Validation of Intensive Care and Mechanical Ventilation Codes in Medicare Data. Crit Care Med. 2017;45:e711–4. doi: 10.1097/CCM.0000000000002316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Medical Care. 2005;43:1130–9. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 25.Wunsch H, Harrison DA, Rowan K. Health services research in critical care using administrative data. J Crit Care. 2005;20:264–9. doi: 10.1016/j.jcrc.2005.08.002. [DOI] [PubMed] [Google Scholar]
- 26.Carson SS. Know your long-term care hospital. Chest. 2007;131:2–5. doi: 10.1378/chest.06-2513. [DOI] [PubMed] [Google Scholar]
- 27.Jubran A, Grant BJB, Duffner LA, Collins EG, Lanuza DM, Hoffman LA, et al. Effect of Pressure Support vs Unassisted Breathing Through a Tracheostomy Collar on Weaning Duration in Patients Requiring Prolonged Mechanical Ventilation: A Randomized Trial. Jama. 2013:1–7. doi: 10.1001/jama.2013.159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Wilcox ME, Chong CAKY, Niven DJ, Rubenfeld GD, Rowan KM, Wunsch H, et al. Do Intensivist Staffing Patterns Influence Hospital Mortality Following ICU Admission? A Systematic Review and Meta-Analyses. 2013;41:2253–74. doi: 10.1097/CCM.0b013e318292313a. [DOI] [PubMed] [Google Scholar]
- 29.Tarnow-Mordi WO, Hau C, Warden A, Shearer AJ. Hospital mortality in relation to staff workload: a 4-year study in an adult intensive-care unit. The Lancet. 2000;356:185–9. doi: 10.1016/s0140-6736(00)02478-8. [DOI] [PubMed] [Google Scholar]
- 30.Kelly DM, Kutney-Lee A, McHugh MD, Sloane DM, Aiken LH. Impact of critical care nursing on 30-day mortality of mechanically ventilated older adults. Crit Care Med. 2014;42:1089–95. doi: 10.1097/CCM.0000000000000127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kim MM, Barnato AE, Angus DC, Fleisher LA, Fleisher LF, Kahn JM. The effect of multidisciplinary care teams on intensive care unit mortality. Arch Intern Med. 2010;170:369–76. doi: 10.1001/archinternmed.2009.521. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Girard TD, Kress JP, Fuchs BD, Thomason JWW, Schweickert WD, Pun BT, et al. Efficacy and safety of a paired sedation and ventilator weaning protocol for mechanically ventilated patients in intensive care (Awakening and Breathing Controlled trial): a randomised controlled trial. Lancet. 2008;371:126–34. doi: 10.1016/S0140-6736(08)60105-1. [DOI] [PubMed] [Google Scholar]
- 33.Weiss CH, Moazed F, McEvoy CA, Singer BD, Szleifer I, Amaral LAN, et al. Prompting physicians to address a daily checklist and process of care and clinical outcomes: a single-site study. American Journal of Respiratory and Critical Care Medicine. 2011;184:680–6. doi: 10.1164/rccm.201101-0037OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Carson SS, Goss CH, Patel SR, Anzueto A, Au DH, Elborn S, et al. An official American Thoracic Society research statement: comparative effectiveness research in pulmonary, critical care, and sleep medicine. American Journal of Respiratory and Critical Care Medicine. 2013;188:1253–61. doi: 10.1164/rccm.201310-1790ST. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Bradley EH, Curry LA, Ramanadhan S, Rowe L, Nembhard IM, Krumholz HM. Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009;4:25. doi: 10.1186/1748-5908-4-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hillmer MP, Wodchis WP, Gill SS, Anderson GM, Rochon PA. Nursing home profit status and quality of care: is there any evidence of an association? Med Care Res Rev. 2005;62:139–66. doi: 10.1177/1077558704273769. [DOI] [PubMed] [Google Scholar]
- 37.Kahn JM, Goss CH, Heagerty PJ, Kramer AA, O'Brien CR, Rubenfeld GD. Hospital volume and the outcomes of mechanical ventilation. N Engl J Med. 2006;355:41–50. doi: 10.1056/NEJMsa053993. [DOI] [PubMed] [Google Scholar]
- 38.Nguyen MD M Yên-Lan, Wallace MD M, MD YY, PhD LT, MSc JB David J, Angus MD MF Derek C, et al. The Volume-Outcome Relationship in Critical Care. Chest. 2015;148:79–92. doi: 10.1378/chest.14-2195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Walkey AJ, Barnato AE, Wiener RS, Nallamothu BK. Accounting for Patient Preferences Regarding Life-Sustaining Treatment in Evaluations of Medical Effectiveness and Quality. American Journal of Respiratory and Critical Care Medicine. 2017;196:958–63. doi: 10.1164/rccm.201701-0165CP. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Koenig L, Demiralp B, Saavoss J, Zhang Q. The Role of Long-term Acute Care Hospitals in Treating the Critically Ill and Medically Complex: An Analysis of Nonventilator Patients. Medical Care. 2015;53:582–90. doi: 10.1097/MLR.0000000000000382. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.