Outcome prediction in mechanically ventilated neurologic patients by junior neurointensivists

Abstract

Objective:

Physician prediction of outcome in critically ill neurologic patients impacts treatment decisions and goals of care. In this observational study, we prospectively compared predictions by neurointensivists to patient outcomes at 6 months.

Methods:

Consecutive neurologic patients requiring mechanical ventilation for 72 hours or more were enrolled. The attending neurointensivist was asked to predict 6-month 1) functional outcome (modified Rankin scale [mRS]), 2) quality of life (QOL), and 3) whether supportive care should be withdrawn. Six-month functional outcome was determined by telephone interviews and dichotomized to good (mRS 0–3) and poor outcome (mRS 4–6).

Results:

Of 187 eligible patients, 144 were enrolled. Neurointensivists correctly predicted 6-month functional outcome in 80% (95% confidence interval [CI], 72%–86%) of patients. Accuracy for a predicted good outcome was 63% (95% CI, 50%–74%) and for poor outcome 94% (95% CI, 85%–98%). Excluding patients who had life support withdrawn, accuracy for good outcome was 73% (95% CI, 60%–84%) and for poor outcome 87% (95% CI, 74%–94%). Accuracy for exact agreement between neurointensivists' mRS predictions and actual 6-month mRS was only 43% (95% CI, 35%–52%). Predicted accuracy for QOL was 58% (95% CI, 39%–74%) for good/excellent and 67% (95% CI, 46%–83%) for poor/fair. Of 27 patients for whom withdrawal of care was recommended, 1 patient survived in a vegetative state.

Conclusions:

Prediction of long-term functional outcomes in critically ill neurologic patients is challenging. Our neurointensivists were more accurate in predicting poor outcome than good outcome in patients requiring mechanical ventilation ≥72 hours.

GLOSSARY

BI

= Barthel index;

CI

= confidence interval;

E-GOS

= extended Glasgow outcome scale score;

ICU

= intensive care unit;

IQR

= interquartile range;

mRS

= modified Rankin scale;

QOL

= quality of life.

Approximately 1 in 5 deaths in the United States occurs during or shortly after receiving care in the intensive care unit (ICU).^1,2 Withdrawing life support and instituting comfort care precede the majority of these ICU deaths.³ The decision to transition to palliative care is shared between the physician and the patient, often represented by the family. Thus, the doctor's opinion on outcome often strongly influences family decisions. Physicians rate poor prognosis in regards to survival, quality of life (QOL), and functional outcomes as important factors when making a decision to withdraw supportive treatment.^4–6 Physician perception of outcome plays a significant role in these life-or-death decisions because an overly pessimistic approach may lead to increased mortality due to premature withdrawal of support.⁷

The prognosis of critically ill patients with neurologic diseases or complications, up to one third of ICU admissions, presents a unique challenge because not only are prognostic predictions made in regard to the chance of survival, but also in regard to functional recovery.^8–10 Information on how accurately neurointensivists can predict long-term neurologic outcome in the individual patient is limited. We designed this prospective observational study to assess our attending neurointensivists' accuracy in predicting 6-month functional outcome in a consecutive series of neurologic patients who had been intubated and mechanically ventilated for 72 hours or more.

METHODS

Study protocol.

This study was conducted in a 52-bed university hospital adult ICU from January 2005 to August 2007. Patients with a neurologic illness or a severe neurologic complication of critical illness were eligible for enrollment once they were intubated and mechanically ventilated for ≥72 hours. Five attending neurointensivists participated in the study. For each patient enrolled, the neurointensivist caring for the patient was asked to 1) predict the patient's 6-month functional outcome by the modified Rankin scale¹¹ (mRS: 0 = no symptoms at all; 1 = no significant disability despite symptoms, able to carry out usual duties and activities; 2 = slight disability, unable to carry out all previous activities but able to look after own affairs without assistance; 3 = moderate disability, requiring some help but able to walk without assistance; 4 = moderately severe disability, unable to walk without assistance and unable to attend to own bodily needs without assist; 5 = severe disability, bedridden, incontinent, and requiring constant nursing care and attention; 6 = death), 2) predict the patient's 6-month view of their QOL on a simple qualitative scale: poor, fair, good, or excellent, and 3) provide their opinion on whether withdrawal of support should be recommended assuming the patient's and families' wishes were unknown. Questionnaires were completed anonymously by the attending neurointensivists in an effort to promote the most accurate opinion free from discriminatory labeling.

If patients survived their hospitalization, they or their surrogates were consented for a follow-up 6-month telephone interview (∼30 minutes) to assess their mRS, QOL, extended Glasgow outcome scale score^12,13 (E-GOS; a scale of 1, death, to 8, minimal or no disability), Barthel index^14,15 (BI; a scale to measure independence of activities of daily living: 0, bedridden with complete dependence, to 100, independent), and their opinion on whether in retrospect they feel that life support should have been withdrawn. For the QOL assessment, physicians and patients (or their surrogates) were asked to simply rate QOL on a qualitative scale including excellent/good/fair/poor. The telephone interviews were completed by 2 investigators, who were not allowed to be part of the pool of evaluators after any of the data had been categorized or analyzed. Further, both interviewers were kept blind to patient status and no review of the patient's chart was permitted prior to the 6-month phone call. If the patient could not participate in the interview, the surrogate's answers were used.

Standard protocol approvals, registrations, and consents.

This study received approval from our institutional ethical standards committee on human experimentation for any experiments using human subjects. Written informed consent was obtained for the follow-up telephone interview.

Outcome measures.

The neurointensivist's predictions were compared to the actual patient's 6-month outcome. Patients with incomplete outcome forms were excluded only from the pertinent comparison item. Accuracy measurements between predicted outcome and 6-month outcome were made by dichotomizing the mRS into good (mRS 0–3) and poor outcome (mRS 4–6). Predictions were considered correct if the prediction fell into the correct range of the patient's 6-month outcome. For example, if the evaluator predicted a 6-month mRS score of 3 and the patient's 6-month mRS score was 2, the evaluator was considered correct. In addition, neurointensivists' mRS predictions were also assessed for exact agreement in each of the 6 categories.

The QOL assessment was dichotomized into 1) poor or fair and 2) good or excellent responses. Deceased patients were excluded. Regarding withdrawal of support, patients with questionnaires marked yes to withdrawal of support by the neurointensivist were followed for outcomes of 1) whether withdrawal of life support occurred, 2) death other than withdrawal of life support, and 3) 6-month's mRS and the patient's (or surrogate's) 6-month response to the question of whether support should have been withdrawn in retrospect.

Data analysis.

Accuracy results for the dichotomized mRS and QOL outcomes were measured by calculating the predictive values for good and poor outcome with a 95% confidence interval (CI). Predictive values for the mRS were calculated for 1) all patients, 2) patients who were not taken off life support, and 3) patients who were consented prior to hospital discharge. The χ² test was utilized to compare the predictive value between groups. A p value <0.05 was considered significant. Data were analyzed using computer software (SPSS, version 16.0, SPSS, Inc.; Chicago, IL).

RESULTS

Enrollment.

A total of 187 consecutive patients were eligible for the study and 144 were enrolled. Forty-three patients were not included because of lack of informed consent. One patient was lost to follow-up and another one withdrew from the study. Follow-up data were collected from clinic visits in 13 cases. Baseline patient and survivor characteristics are listed in table 1. The most common diagnoses were subarachnoid or intracranial hemorrhage (53%), ischemic stroke (18%), and status epilepticus (11%).

Table 1 Baseline characteristics in all patients (n = 142) and in survivors (n = 73)

Sixty-nine (49%) patients died. Of these, 49 (71%) were taken off life support based on a perceived poor prognosis. The presumed cause of death in the 20 patients who were not taken off life support included infection (4), malnutrition (2), cancer (1), cardiopulmonary arrest (3), brain death (3), and recurrent intracranial hemorrhage (2), and was unknown in 5 patients. The median 6-month mRS of the entire enrollment group was 5 (interquartile range [IQR] 3–6) and of the survivors 3 (IQR 2–4). The BI and E-GOS were obtained on 62 (85%) survivors. The median BI was 80 (IQR 41–100) and the median E-GOS was 4 (upper severe disability) (IQR 3–6).

Questionnaires were completed at a median of 4 days (IQR 3–6) after ICU admission. Mean age of the 5 participating neurointensivists was 36 ± 4 years and their mean years in practice since fellowship were 2 ± 3 years.

Modified Rankin Scale outcomes.

The distribution of neurointensivist mRS predictions and actual 6-month outcomes are shown in figure 1. Neurointensivists predicted a lower percentage (54%) of poor outcomes (mRS 4–6) compared to the actual 6-month outcomes (68%), particularly with regard to death (26% vs 49%).

Figure 1 Predicted and observed 6-month modified Rankin scale (mRS) scores (n = 142)

Comparison of neurointensivists (NI) mRS predictions and actual 6-month outcome in 142 critically ill neurologic patients intubated for 72 hours or more in the intensive care unit. Dark line represents dichotomization used for analysis.

Overall, for the dichotomized analysis the neurointensivists correctly predicted functional outcomes in 80% (95% CI, 72%–86%) of patients (table 2). Accuracy for predicting good outcome (mRS 0–3) was 63% (95% CI, 50%–74%) and for poor outcome 94% (95% CI, 85%–98%). After excluding patients who were taken off life support, the accuracy for predicting good outcome was 73% (95% CI, 60%–84%) and for poor outcome 87% (95% CI, 74%–94%). Accuracy for predicting good outcome increased to 85% (95% CI, 74%–93%) after excluding patients who had died.

Table 2 Accuracy of neurointensivists' (NI) predictions of functional outcome and quality of life at 6 months

To assess for potential bias resulting from the overrepresentation of dead patients in our enrollment group, due to their known 6-month outcomes, we performed the same analysis in the subset of patients for whom we had consent prior to hospital discharge (n = 46). The observed 6-month mortality was significantly lower in this group (15%) compared to the entire enrollment group (49%, p < 0.001). There was no difference in the accuracy of the neurointensivists' outcome predictions between this group and the entire enrollment group. Overall accuracy in the select group was 80% (95% CI, 66%–90%). The accuracy for predicting a good outcome was 78% (95% CI, 57%–90%) in this group compared to 63% (p = 0.17) in the entire enrollment group; and the accuracy for predicting a poor outcome was 84% (95% CI, 60%–96%) compared to 94% (p = 0.35). However, in contrast to the entire enrollment group, the predicted mortality by neurointensivists in this group (13%) was almost identical to the observed mortality (15%).

Accuracy for exact agreement between neurointensivists' mRS predictions and actual 6-month mRS was 43% (61/142, 95% CI, 35%–52%). The majority of incorrect predictions were too optimistic (74%, 60/81). However, after excluding patients who died (mRS 6), there was an equal number (50%, 21/42) of too optimistic and too pessimistic predictions (figure 2).

Figure 2 Predicted and observed 6-month modified Rankin scale (mRS) scores for each patient (n = 142)

Histogram comparing predicted and actual 6-month mRS outcome for individual patients.

Quality of life.

Of the 73 survivors, 60 patients (median mRS 3 [IQR 2–4]) were included in the QOL evaluation. For 8 patients the surrogate's answer was used. The neurointensivists' overall accuracy in predicting QOL was 62%. For good/excellent responses, prediction accuracy was 58% (95% CI, 39%–74%) and for poor/fair responses 67% (95% CI, 46%–83%) (table 2). Thirty-eight percent (14/37) of patients with a 6-month mRS of 0–3 reported their QOL to be fair or poor, whereas 17% (4/23) of patients with a mRS of 4 or 5 described their QOL as good or excellent.

Withdrawal of life support.

Neurointensivists recommended withdrawal of care for 27 (19%) patients, which was instituted in 23. Of the remaining 4 patients, 3 died and 1 survived to 6 months in a vegetative state. The son of this patient reported that his father would be glad to have survived in this state. Conversely, family members of 2 patients who were not recommended for withdrawal of care early during their ICU stay wished in retrospect they had withdrawn support. One of these patients died of a second intracranial hemorrhage. The other was bedridden with severe disabilities at 6 months.

DISCUSSION

Accurate outcome prediction in critically ill neurologic patients is an important issue, because the physician's perceived outcome impacts decisions regarding whether to provide aggressive care or to withdraw life support and let the patient die. Overly optimistic outcome prediction may cause patients to survive in severely disabled states that are unwanted, whereas overly pessimistic outcome prediction may lead to death in those who may have had a chance of survival with an acceptable long-term functional outcome. Prospective data on the accuracy of outcome prediction by neurointensivists of not only survival, but also long-term functional outcome are scarce.

We found that neurointensivists at our institution had a relatively high predictive accuracy (94%) at 3 days or later in determining which critically ill neurologic patients were likely to have a poor 6-month functional outcome. Also after excluding patients who had life support withdrawn, i.e., avoiding the potential bias of a self-fulfilling prophecy, the accuracy for predicting poor outcome remained relatively high (87%). This relatively high accuracy is unlikely the result of the high prevalence of poor outcomes in our study group as a whole, because even in the subgroup of patients who were consented prior to hospital discharge, a group with a mortality rate of only 15%, accuracy for poor outcome prediction remained high at 84%.

We felt that determining prognosis based on a range of functional outcomes dichotomized at a mRS of 0–3 vs 4–6 was reasonable in this critically ill neurologic patient population. Very few of these patients are likely to survive without any neurologic deficits. Similar cutoffs have been used in other studies including severely ill neurologic patients.^16,17 When speaking to families about prognosis, many physicians express a degree of uncertainty and give a range of potential outcomes (and the likelihood of each one of them) rather than one specific functional outcome. Our results assessing for exact agreement in predicted and actual functional outcome in each category of the mRS (accurate in only 43% of patients) underscore the difficulty in making a precise prediction. However, it would be an unrealistic goal for neurointensivists to exactly predict 6-month functional outcome in critically ill neurologic patients. In general, our neurointensivists tended to underestimate mortality, which may in part be caused by an overrepresentation of patients who died in our study sample. Although some patients had unexpected deaths from cardiac arrest or massive pulmonary embolism, many patients died as a consequence of their neurologic illness.

Why the accuracy for predicting exact outcomes and good functional (mRS 0–3) outcomes was not higher may be that we simply do not know or measure all of the factors that affect the brain's capacity to recover after injury. Other potential reasons may include lack of physician experience, limited exposure to patient long-term outcome, interval medical events affecting prognosis, and perhaps the admission of medically complicated patients at a tertiary hospital, making it challenging to accurately predict functional outcomes. In spite of the development of prediction models, precise outcome prediction in individual patients remains difficult in ICU-based populations.^5,18,19 Further, there is a paucity of validated outcome scales that predict functional outcome rather than mortality in neurologic patients. We found that both overoptimism and overpessimism contributed to our incorrect predictions.

We also found poor accuracy in regards to assessing future QOL. However, since we measured QOL in a simplistic way, one needs to exert caution with interpreting this result. Similar to a previous study,²⁰ we found that the majority of patients with poor functional outcome reported having a poor QOL and vice versa; however, the fact that 38% of patients with relatively good functional outcomes (mRS scores of 0–3) rated their QOL as poor or fair and that 17% of patients with poor functional outcomes (mRS 4 or 5) rated their QOL as good or excellent suggests that QOL is not directly linked to functional outcome in every individual. The presence of depression may account for part of this dissociation.²¹ Furthermore, it is conceivable that good physical recovery alone may not predict good outcome in some patients. This notion may deserve attention when families and physicians discuss goals of care assuming that a certain clinical examination or brain imaging pattern will correlate or predict a certain QOL. Better determinants of an acceptable QOL are needed in this setting. Future studies are needed to investigate this seeming disassociation between functional outcome and QOL at different time points in the recovery process. After all, whether life in the long term is worth living to the individual patient should be the most important measure of our treatment success in the neurointensive care unit.

Our study population represents the sickest neurology patients in the ICU. We purposely chose to only include critically ill neurologic patients who were intubated for ≥72 hours in this study, as these are the patients for whom we can actually remove life support if the perceived long-term outcome is felt to be undesirable. Thus, it is no surprise that the 6-month mortality in our enrollment group was almost 50%. Other studies have reported mortality ranges from ∼20%–60% among ICU patients with neurologic illnesses or complications.^6–9 Since our study design facilitated the inclusion of patients who had died, the high observed mortality may have resulted in part from selection bias. Nevertheless, even if all 187 eligible patients would have been enrolled, the observed mortality would have been at least 37% (69/187). Further, it is plausible that the diagnostic heterogeneity of our enrollment group, which included both primary neurologic illnesses and ICU patients with neurologic complications, may have contributed to the high mortality. A higher mortality has been reported in ICU patients with neurologic complications than in ICU patients with a primary neurologic diagnosis.⁹

This study has limitations. First, the generalizability of the results is very limited as this was a single center study with a small number of junior academic neurointensivists that included neurology patients who were mechanically ventilated with a variety of primary and secondary neurologic diagnoses. Study of more homogenous patient populations and more experienced neurointensivists may have yielded different results. Further, we lack information on individual raters' religious or ethical background and their use of mortality scoring systems, factors which likely affect outcome prediction.¹⁰ Therefore, our study results do not apply to other practices unless duplicated elsewhere. Third, alterations in the grouping of mRS and QOL outcomes to judge prognostic accuracy may have changed the interpretation of the results. Fourth, our methods of measuring QOL were simplistic. A more comprehensive measurement scale might bring out differences in QOL that were due to depression vs disability. Fifth, the timing of study eligibility, after 72 hours of ventilatory support, excluded many patients, which may have affected our results. Accuracy of prognostication on ICU day 4 may differ from ICU admission. We chose a time window in which we felt that outcome prediction is most relevant. It is precisely for patients who do require prolonged ventilatory support that we face the decision whether to continue life support. Patients with devastating neurologic injuries on hospital admission, who die early, and those who do not require prolonged intubation are not included. Thus, our results only apply to patients who are mechanically ventilated for at least 3 days. Finally, our study design and results do not address the appropriate usage and timing of institution of treatment limitations in critically ill neurologic patients and should not be used for this purpose.

AUTHOR CONTRIBUTIONS

Statistical analysis was conducted by Dr. Michael Mlynash.

DISCLOSURE

Dr. Finley Caulfield receives research support from the NIH (RO1 HL089116-01A2 [Sub-I]) and has received research support from the Stanford University/Program in Organizing Neuroethics Education and Research (PIONEAR) for this study. L. Gabler receives research support from the University of Oxford (Department of Public Health and Primary Care) and from the Rhodes Trust. Dr. Lansberg has received research support from Stanford University (BioX grant) for this study and presently receives research support from the NIH/NINDS (K23 NS051372 [PI] and NS044876-05 [local PI]). Dr. Eyngorn and Dr. Mlynash report no disclosures. Dr. Buckwalter receives research support from the NIH/NINDS (1K08 NS050304-01A2 [PI]) and from the Weston Haven Foundation. Dr. Venkatasubramanian reports no disclosures. Dr. Wijman receives research support from the NIH (RO1 NS034866-08 [PI] and RO1 HL089116-01A2 [PI]) and has received research support from the American Heart Association (043275N [PI]) for this study.

Address correspondence and reprint requests to Dr. Anna Finley Caulfield, 701 Welch Road, Suite B325, Palo Alto, CA 94304 afinley@stanford.edu

Editorial, page 1086

Disclosure: Author disclosures are provided at the end of the article.

Received May 29, 2009. Accepted in final form December 18, 2009.