A multicenter mortality prediction model for patients receiving prolonged mechanical ventilation

Abstract

Objective

Significant deficiencies exist in the communication of prognosis for patients requiring prolonged mechanical ventilation after acute illness, in part because of clinician uncertainty about long-term outcomes. We sought to refine a mortality prediction model for patients requiring prolonged ventilation using a multicentered study design.

Design

Cohort study.

Setting

Five geographically diverse tertiary care medical centers in the United States (California, Colorado, North Carolina, Pennsylvania, Washington).

Patients

Two hundred sixty adult patients who received at least 21 days of mechanical ventilation after acute illness.

Interventions

None.

Measurements and Main Results

For the probability model, we included age, platelet count, and requirement for vasopressors and/or hemodialysis, each measured on day 21 of mechanical ventilation, in a logistic regression model with 1-yr mortality as the outcome variable. We subsequently modified a simplified prognostic scoring rule (ProVent score) by categorizing the risk variables (age 18–49, 50–64, and >65 yrs; platelet count 0–150 and >150; vasopressors; hemodialysis) in another logistic regression model and assigning points to variables according to β coefficient values. Overall mortality at 1 yr was 48%. The area under the curve of the receiver operator characteristic curve for the primary ProVent probability model was 0.79 (95% confidence interval, 0.75–0.81), and the p value for the Hosmer-Lemeshow goodness-of-fit statistic was .89. The area under the curve for the categorical model was 0.77, and the p value for the goodness-of-fit statistic was .34. The area under the curve for the ProVent score was 0.76, and the p value for the Hosmer-Lemeshow goodness-of-fit statistic was .60. For the 50 patients with a ProVent score >2, only one patient was able to be discharged directly home, and 1-yr mortality was 86%.

Conclusion

The ProVent probability model is a simple and reproducible model that can accurately identify patients requiring prolonged mechanical ventilation who are at high risk of 1-yr mortality.

Many patients who survive the first few days of critical illness do so with multiple persistent organ failures, ultimately becoming dependent on mechanical ventilation for prolonged periods (1). Up to 10% of patients who develop acute respiratory failure require prolonged mechanical ventilation (PMV) (2). The number of patients receiving PMV has increased in recent years, likely as a result of improvements in acute management and supportive care for critically ill patients (2, 3). As the population ages, it is expected that this number will increase further, because advanced age is a key risk factor for PMV (3, 4). One-yr mortality for patients receiving PMV is high (5–9), and only 11% percent of patients are functionally independent and living at home by 1 yr (9, 10). During the year of their illness, 74% of the patients’ days alive are spent in a hospital, postacute care facility, or receiving home health care.

Recent empirical studies have documented serious shortcomings in the process of decisionmaking about life support for patients on PMV. Up to 93% of families and surrogate decisionmakers do not receive any information about expected long-term survival despite explicit wishes to have this information (11, 12). In one study, 93% of surrogate decisionmakers had high expectations for survival of patients on PMV compared with only 44% of physicians for the same patients (12). These deficiencies are problematic for two reasons. First, they are a threat to patient-centered care because existing data suggest that patients often prefer treatment focused on palliation in the setting of a poor prognosis (13–15). Second, patients receiving PMV are among the highest consumers of healthcare resources (16) and, from a societal perspective, it is important to ensure that the provision of this very expensive resource is reserved for patients who would choose such treatments after a careful discussion of the risks and benefits.

Although there are likely several reasons for suboptimal discussions about prognosis between physicians and families, one important reason is clinicians’ uncertainty about the long-term outcomes of patients on PMV (17–20). This is perhaps not surprising because most intensive care unit (ICU) clinicians have little opportunity to follow patients after they leave the ICU and therefore little opportunity to refine their prognostic abilities regarding long-term outcomes. To address this gap for patients on PMV, the ProVent model was developed and internally validated at a single tertiary care medical center to predict 1-yr mortality for patients receiving at least 21 days of mechanical ventilation after acute illness (5). Using four easily identified clinical variables (age, platelet count, on-going use of vasopressors, and hemodialysis), the ProVent model had very good discrimination (area under the curve [AUC] of the receiver operator characteristic curve 0.81) and calibration for identifying patients who were at high risk of death after PMV. To establish broader applicability, we sought to refine the ProVent model and provide external validity using data from a heterogeneous group of patients from multiple hospitals across the United States.

MATERIALS AND METHODS

In a retrospective cohort design, patients were enrolled from five tertiary care centers including the University of Washington, University of California at San Francisco, Denver Health Medical Center, the Hospital of the University of Pennsylvania, and Duke University Medical Center. Centers were selected based on geographic distribution and access to a broad range of medical, surgical, and trauma patients requiring PMV. None of the centers had contributed data to the original development model. The research protocol was approved by institutional review boards at each of the five centers as well as the coordinating center at the University of North Carolina.

Patients receiving mechanical ventilation in 2005 for at least 14 days after acute illness, uninterrupted by >48 hrs of unassisted breathing, were followed, and patients who were still receiving mechanical ventilation by day 21 were included in the study. Exclusion criteria included age <18 yrs old; diagnosis of acute or chronic neuromuscular disease such as Guillain-Barré syndrome, muscular dystrophy, or myasthenia gravis; patients sustaining extensive burn injuries; and requirement for chronic mechanical ventilation before acute admission. These inclusion and exclusion criteria are the same criteria used for the original model development. Patients were identified by screening records of mechanical ventilation for all patients admitted to adult medical, neurologic, surgical, cardiac, or trauma ICUs. Either consecutive samples or random samples of patients were enrolled at each center depending on the number of patients who were eligible.

Data were abstracted from medical records by two trained individuals at each site. One abstractor who was blinded to patient outcome determined eligibility and collected data on demographic variables and risk factors. The other abstractor collected data on hospital outcomes. The principal investigator at each site reviewed the first ten charts that were abstracted and a random sample of ten subsequent charts to confirm accuracy of data and identify errors that would prompt review of additional charts and correction.

Descriptive variables included age, admission source, primary ICU service, ICU admission diagnoses, and comorbidities based on a modified Charlson score (21). Race and ethnicity as listed in medical records were abstracted to provide information regarding generalizability. We assessed severity of illness on ICU admission using the Acute Physiology and Chronic Health Evaluation III score (22) determined using the worst values measured within the first 24 hrs of index ICU admission. Because the objective of this study was to provide external validity for the mortality prediction model developed at a single center, we only included the four original predictive variables in the probability model. The four predictor variables collected on day 21 of mechanical ventilation included age, platelet count, and requirement for vasopressors and/or hemodialysis.

Requirement for hemodialysis was defined as provision of any form of renal replacement therapy on or within 48 hrs of day 21 of mechanical ventilation. The primary outcome variable was 1-yr mortality using death dates obtained by linking patient records to the National Death Index or the Washington State Death Database. We also assessed several in-hospital outcome variables, including duration of mechanical ventilation, liberation from mechanical ventilation in the hospital defined as unassisted breathing for 7 consecutive days, ICU and hospital length of stay, and hospital mortality. For patients who died during the index hospitalization, records were reviewed for use of mechanical ventilation, vasopressors, and hemodialysis within 72 hrs of death as well as mechanical ventilation, vasopressors, or cardiopulmonary resuscitation on the day of death.

Analysis

Descriptive statistics are presented using mean ± sd for normally distributed continuous variables, median with interquartile range for nonnormally distributed continuous variables, and proportions for categorical variables. To validate the predictive capabilities of the four ProVent predictor variables, we included all variables in a logistic regression model (ProVent probability model) with 1-yr mortality as the outcome variable. We assessed model discrimination using the AUC and model calibration using the Hosmer-Lemeshow goodness-of-fit statistic comparing observed mortality with predicted mortality for each decile of predicted risk. Because a second external cohort was not available, a bootstrap method was used to validate the model by repeating 1000 random samples consisting of 60% of the cohort to provide a 95% confidence interval for the AUC.

After validation of the primary ProVent probability model that used the risk variables as they were measured, we categorized the risk variables and included them in a second logistic regression model. Before initiation of data collection, the investigators elected to modify the cut point for age from the original ProVent score (5). Specifically, two cut points were included for age (age 50 and 65 yrs) rather than one at age 50 yrs to better reflect the higher risk associated with advancing age. Other categorical variables remained the same. We then created a new ProVent score by assigning points to each risk factor according to the β coefficients in the logistic regression model. Long-term survival based on the range of cumulative scores was represented by Kaplan-Meier curves, and the performance of the ProVent score was assessed in a third logistic regression model.

Data were analyzed using SAS software (SAS Institute Inc., Cary, NC). Kaplan-Meier curves were drawn using Stata 8.0 software (Stata, College Station, TX).

RESULTS

A total of 289 patients were enrolled from the five centers. Of those, 260 patients (90%) had complete data for risk variables and were included in analyses. Patient characteristics and outcomes are shown in Table 1. The mean age ± sd of patients was 55 ± 17 yrs, and 41% were female. Patients were diverse in diagnosis, admission source, and primary critical care service including medical, surgical, trauma, and neurologic units. Median (interquartile range) duration of mechanical ventilation was 30 (25–40) days, and median ICU and hospital lengths of stay were 34 (28 – 48) and 44 (33–70) days, respectively. Twenty-eight percent of patients died in the hospital and 12% were discharged home. Of the patients who died in the hospital, 90% were receiving mechanical ventilation and 46% were receiving vasopressors within 72 hrs of death. Only 8% of patients received cardiopulmonary resuscitation at the time of death. Patients who died in the hospital received a median of 32 (26 – 43) days of mechanical ventilation before death. One-yr mortality for the cohort was 48%. The 29 patients not included in analyses as a result of incomplete data for risk variables were similar in mean age (57 ± 15 yrs), gender (39% female), comorbidity score (median, 1 [0 –3]), and 1-yr mortality (48%).

Table 1.

Patient characteristics and outcomes

Patient Characteristics	n = 260
Age, yrs, mean ± sd, [range]	55 ± 17 [18–90]
Female, no. (%)	102 (41%)
Race, no. (%)
Native American or Alaska Native	2 (1%)
Asian or Pacific Islander	16 (6%)
Other nonwhite	10 (4%)
Black	49 (19%)
White	152 (58%)
Unknown or not reported	31 (12%)
Ethnicity, no. (%)
Hispanic or Latino	7 (3%)
Not Hispanic or Latino	207 (80%)
Unknown or not reported	46 (17%)
Primary intensive care unit service, no. (%)
Medicine	90 (35%)
Cardiology	7 (3%)
General surgery/trauma	80 (31%)
Cardiac surgery	31 (12%)
Thoracic surgery	8 (3%)
Neurology/neurosurgery	31 (12%)
Transplant surgery	4 (1%)
Other	9 (3%)
Comorbidity score, median (IQR)	1.0 (0–3)
Admission Acute Physiology and Chronic Health    Evaluation III score, mean ± sd	83 ± 29
Intensive care unit admission diagnoses, no. (%)
Cardiovascular	30 (11%)
Pulmonary including pneumonia	42 (16%)
Gastrointestinal	18 (7%)
Neurologic	30 (12%)
Endocrine	2 (1%)
Hematologic or malignancy	8 (3%)
Infection other than pneumonia	20 (8%)
Surgery	45 (17%)
Trauma	65 (25%)
Hospital outcomes
Duration of MV, median (IQR)	30 (25–40)
Duration of MV if died in the hospital, median (IQR)	32 (26–43)
Liberation from MV, no. (%)	134 (52%)
Intensive care unit length of stay, median (IQR)	34 (28–48)
Hospital length of stay, median (IQR)	44 (33–70)
Discharge disposition, no. (%)
Died in the hospital	71 (28%)
Long-term acute hospital	51 (20%)
Rehabilitation facility	58 (22%)
Skilled nursing facility	45 (17%)
Home with assistance	15 (6%)
Home independent	16 (6%)
Other	4 (2%)
One-yr mortality, no. (%)	124 (48%)

IQR, interquartile range; MV, mechanical ventilation.

Some percentages do not add to 100 as a result of rounding.

In the ProVent probability logistic regression model (see subsequent equation), each of the four ProVent variables was independently associated with 1-yr mortality, including age (odds ratio [OR], 1.04; 95% confidence interval [CI], 1.03– 1.06) for each additional year of age, platelet count (0.996; 0.994–0.998) for each increase of 1 × 10⁹/L, vasopressors (2.96; 1.03–8.46) relative to no vasopressors, and hemodialysis (2.52; 1.00–6.34) relative to no hemodialysis. Enrollment center was not an independent predictor when included as a model variable. Discrimination as measured by the AUC was 0.79 (95% CI, 0.75–0.81). In comparison, the AUC for the Acute Physiology and Chronic Health Evaluation III score measured at ICU admission and 1-yr mortality was 0.63. A comparison of observed vs. predicted mortality for the model is shown in Table 2. The Hosmer-Lemeshow goodness-of-fit statistic was 3.58 with 8 df (p = .89).

Table 2.

Calibration of the ProVent probability model

Partition for the Hosmer-Lemeshow Test (p = .89)
		Died in 1 Yr		Alive in 1 Yr
Group	Total	Observed	Expected	Observed	Expected
1	26	1	1.80	25	24.20
2	26	3	3.91	23	22.09
3	26	8	6.50	18	19.50
4	26	11	9.37	15	16.63
5	26	11	11.54	15	14.46
6	26	15	13.53	11	12.47
7	26	16	15.74	10	10.26
8	26	16	17.72	10	8.28
9	26	18	19.89	8	6.11
10	26	24	23.01	2	2.99

Deciles of expected mortality according to the ProVent probability equation in the equation in “Results.” Hosmer-Lemeshow goodness-of-fit statistic 3.58 with 8 df (p = .89).

Using the ProVent probability model, the predicted probability of death within 1 yr can be calculated using the following equation:

$$\begin{array}{cc}\hfill \text{Prob}& \text{(death in 1-year}\mid \text{A,P,V,H)}\hfill \\ \hfill & =\frac{{\exp }^{(-1.7401+0.0435\mathrm{A}-0.00363\mathrm{P}+1.0835\mathrm{V}+0.925\mathrm{H})}}{1+{\exp }^{(-1.7401+0.0435\mathrm{A}-0.00363\mathrm{P}+1.0835\mathrm{V}+0.925\mathrm{H})}}\hfill \end{array}$$

where A = person’s age (in years); P = platelet count (in 10^9/L units), V = 1 if on vasopressors or = 0 if not, and H = 1 if on hemodialysis or = 0 if not; “exp” is the exponential constant (2.71828). Variables are measured on day 21 of mechanical ventilation. Requirement for hemodialysis is defined as provision of hemodialysis on or within 48 hrs of day 21 of mechanical ventilation.

The second logistic regression model with categorized variables had an AUC of 0.77, and the Hosmer-Lemeshow goodness-of-fit statistic was 5.70 with 5 df (p = .34). Point values were assigned according to the β values from the second model as shown in Table 3 to generate the ProVent score. Two points were assigned to age ≥65 yrs, and 1 point was assigned to each of the other risk factors including age 50–64 yrs, platelet count ≤150 × 10⁹/L, and requirement for vasopressors or hemodialysis on day 21 of mechanical ventilation. Scores could range from 0 to 5 points. The third logistic regression model using the cumulative ProVent score had an AUC of 0.76, and the Hosmer-Lemeshow goodness-of-fit statistic was 1.86 with 3 df (p = .60). Table 4 and Figure 1 show 1-yr mortality and long-term survival for patients according to their ProVent score. For patients in the highest risk groups (ProVent score >2 points), hospital mortality was 43%, yet only one patient was discharged home, and 1-yr mortality was 86%.

Table 3.

Model with categorized risk variables

Categorical Variable	No. (%)	Odds Ratio (95% Confidence Interval)	Beta Value	Points
Age ≥65 yrs	80 (31%)	7.6 (3.8–15.5)	2.03	2
Age 50–64 yrs	88 (34%)	2.0 (1.0–3.9)	0.67	1
Platelets ≤150 × 109/L	65 (25%)	1.9 (0.9–3.9)	0.65	1
Vasopressors	35 (13%)	4.4 (1.6–12.6)	1.49	1
Hemodialysis	34 (13%)	2.4 (1.0–6.0)	0.89	1

Table 4.

ProVent score and observed 1-yr mortality

ProVent Score	No.	Observed Mortality Percent (95% Confidence Interval)
0	72	20 (10–29)
1	60	36 (24–48)
2	78	56 (45–68)
3	36	81 (67–94)
4 or 5	14	100 (77–100)

The ProVent score is calculated by summing the point values assigned according to the presence of risk variables listed in Table 3 when measured on day 21 of mechanical ventilation.

Figure 1.

Kaplan-Meier curve of survival for patients by ProVent score.

DISCUSSION

In a multicenter cohort study, the primary ProVent probability model accurately predicted risk of 1-yr mortality for patients requiring at least 21 days of mechanical ventilation. The cohort included a racially diverse group of patients from medical, surgical, and neurologic ICUs. The model has good discrimination and excellent calibration for patients at all levels of risk. The ProVent model uses only four variables that are easily measured on day 21 of mechanical ventilation. The model does not require subjective assessments such as the Glasgow Coma Scale that can be affected by sedation practices or primary admission diagnosis, which can be uncertain in patients presenting with multiorgan failure (23). Predicted mortality for patients can be obtained by using the prediction equation provided. Alternatively, the model has been converted to a simple scoring rule (ProVent score) to aid in clinical application at the bedside if a computer or hand-held device is not available to complete the probability equation. Less than 15% of patients with ProVent scores >2 were alive after 1 yr. The Model for End-Stage Liver Disease score, which uses three objective variables to predict survival in patients with advanced liver disease, provides a clear example of how simple prediction rules can gain wide general use in the acute care setting for purposes of risk prediction and scarce resource allocation (24, 25).

Prognostication is not straightforward in many clinical conditions. PMV presents unique challenges for long-term prognostication because few inpatient clinicians participating in ICU decisionmaking have experience with patient outcomes beyond hospital discharge. Existing severity of illness measures using variables measured on the day of ICU admission do not perform well in the PMV population as demonstrated in previous analyses (26) and in the current assessment of the Acute Physiology and Chronic Health Evaluation III system in this cohort. Therefore, a model specific to the PMV population is necessary. Published outcome studies provide mean outcomes for large cohorts (5–9) but are not sufficiently tailored to individual patient characteristics to reliably inform clinical prognostication.

This validated prediction model for long-term outcome can: 1) standardize illness severity in observational and interventional studies of chronically critically ill patients; 2) help determine appropriate levels of postacute care (27–29); and 3) increase clinicians’ confidence in responding to informational needs of patients, families, and surrogate decision-makers (30, 31). It is yet to be determined whether the ProVent models are more accurate than physician estimates of high risk, and like with any prognostic model, the ProVent models are intended to complement the a priori assessments of an experienced clinician rather than replacing clinical judgment (32). Given the inherent limitations in translating data on population-level outcomes to individual risk estimates, the use of scoring systems as a sole guide to making decisions about whether to initiate or continue to provide intensive care is inappropriate by current ethical standards (33). However, the data derived from these systems can provide relevant information for decisionmaking, especially when combined with physician estimates of outcome.

Another consideration is whether clinicians will use prognostic information from the ProVent model. The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatment (34) was a large randomized controlled trial in which physicians were given prognostic estimates for individual patients based on a sophisticated prognostic model. The intervention had no significant impact on the main outcomes, in part because only 20% of physicians disclosed the prognostic information to surrogates. These data suggest that to meaningfully impact care, the ProVent score may need to be part of a more sophisticated decision support process that is acceptable to clinicians. Examples of decision support interventions that could incorporate ProVent data and may benefit patients on PMV include structured family meetings led by intensivists or palliative care-trained clinicians (35–38) or formal decision support tools that can be shared with patient surrogates in a formal setting (39). Future iterations of the ProVent model should involve measurement of variables before 21 days of mechanical ventilation to aid decisionmaking earlier in the course of ICU care.

Multiple studies have suggested that intensivist perceptions of extremely poor prognosis are associated with less aggressive or invasive care (40–42). Prognoses in the intermediate range may be less likely to impact decisionmaking, but intermediate prognoses are still valuable in the setting of prolonged ventilation and chronic critical illness. For example, patients with a ProVent score of 3 have predicted 1-yr mortality of 81% (95% CI, 67–94). Although clinicians and families will not perceive this as hopeless, it is likely to help focus their attention on the patient’s desires for prolonged invasive care in the context of lower expectations for survival, a universally high symptom burden (43), and poor expected functional outcomes in long-term survivors (6 – 8).

Our study has several limitations. Although we refined our model in a geographically diverse population, we conducted our study primarily in large tertiary centers. However, previous literature indicates that large centers take care of the majority of patients requiring PMV as a result of the greater complexity of their patient populations and transfer practices from smaller community hospitals (44). The confidence interval around the AUC and measures of calibration for the primary probability model using the original continuous variables are excellent. However, further validation of the modified scoring rule (ProVent score) in a larger external sample is indicated. The retrospective study design could have introduced bias in ascertainment of data, but patient eligibility and risk variables were easily identified in medical records, and investigators measuring risk variables were blinded to patient outcomes. Our study also did not assess long-term functional status, an important factor in decisionmaking for many patients (41), because the study design did not allow for measurement of those outcomes. Because some patients or surrogates opted not to pursue full life support throughout their entire course, the model likely predicts an interplay of physiological and social factors rather than the bare natural history of disease (15, 45). This is true of all mortality models derived from clinical populations.

CONCLUSION

The ProVent probability model is a simple and reproducible model that can accurately identify patients requiring PMV who are at high risk of 1-yr mortality. When paired with clinical judgment, this model may increase clinicians’ ability to discuss the likely outcomes of treatment and to tailor care to achieve patient-centered goals. Future studies should examine similar models using variables measured earlier in the course of prolonged ventilation and outcomes that include long-term functional status.