The Accuracy of Nurses’ Predictions for Clinical Outcomes in the Chronically Critically Ill

Abstract

Background

Accurately predicting survivorship of patients in the intensive care unit is known to be difficult. Previous research has shown that nurses are more likely to recognize futile medical care than other disciplines. The purpose of this study was to describe the accuracy of nurse’s predictions for survival of patients who are chronically critically ill (CCI).

Methods

Using a secondary data analysis from a longitudinal, descriptive study, we evaluated nurses’ predictions for survival at admission and until ICU discharge or patient death.

Results

Nurses were able to more accurately predict survival of CCI patients than death and prognostication appeared to improve with prolonged ICU stays.

Conclusion

This is the first longitudinal study that focused on nurse prognostication in a chronically critically ill patient population. Future research should explore the relationship between ICU nurse characteristics and the accuracy of survival predictions.

INTRODUCTION

Intensive care unit (ICU) nurses have a critical role in the care of patients and their family members, including participation in discussions of health outcomes and prognosis. While several studies have explored survival predictions in the ICU, there is wide variability in clinicians’ ability to accurately discriminate between likely survivors and non-survivors^[1–4].

Nurses demonstrate prognostication skills that are similar to physicians^[5] and often are more accurate when compared to available prognostic tools. However, a recent study evaluated the consistency of daily survival predictions by ICU nurses and found low consistency (inter-observer reliability) in these expectations^[6]. The variance was found to be higher when looking at predictions made by different nurses compared to the variance within observers over time.

Both physicians and nurses are concerned with the futility of medical care in ICU settings, though nurses appear more likely to perceive care to be futile^[7,8]. The tendency to underestimate the probability of survival could be associated with moral distress as well as influencing communication with the family. Communication of the prognosis is a key aspect to providing guidance and meaningful discussions with families^[9,10]. We know that families get their input from many members of the health care team and the ability for nurses to provide accurate estimates for survival is critical to these discussions^[9,11].

Previous research reveals the difficulty in providing accurate survival predictions in the intensive care setting, but this is the first nurse-focused longitudinal study to be conducted with the chronically critically ill (CCI) patient population. The purpose of this study was to describe the accuracy of nurse’s predictions for survival of patients who experience mechanical ventilation and long stays in the ICU.

METHODS

Design

This descriptive, longitudinal study was a secondary analysis of data collected from a larger study (R01NR013322) examining the extent to which transitions to End-of-Life (EOL) care can be predicted in the intensive care unit. The parent study used complexity science to examine the influences on these decisions. A comprehensive description of the parent study has been previously published in the literature^[12]. Quantitative data were collected from February, 2012–April, 2014.

Sample

This study was implemented within three intensive care units (medical, surgical, neuroscience) at University Hospitals Cleveland Medical Center, a tertiary medical center in the Midwest. One hundred and thirty-six nurses and 266 family decision makers were enrolled in the study. Each family surrogate was identified as the decision maker for a CCI patient who was unable to make health care decisions due to his/her cognitive state. The 36 nurses were considered the subjects for this study.

Procedures

Approval for this study was obtained from the hospital’s institutional review board. Subjects were recruited in the first week of the ICU patient’s hospitalization and data were collected every 5 days thereafter until ICU discharge or death. Nursing data were collected within 24 h of when data were obtained from the family subjects. All data collection was done electronically by trained research assistants using an iPad (^©Apple). The subjects had the option of answering the questions through self-report or an interview by the research assistant. To accomplish the study objective, using a method devised by Cox et al.^[13] nurses of CCI patients were asked to answer one question regarding their expectations for patient survival: “Given your patient’s current condition, what do you expect in terms of survival at six months post-discharge”? The 4 answer options for the survival questions were dichotomized and collapsed whereas “very unlikely” and “not likely”=not likely to survive and “probable” and “very likely”=likely to survive.

Data Analysis

Data were entered and analyzed using Version 24 SPSS software. Descriptive statistics including measures of central tendency were used to address the primary research question. T-tests and Chi-square were used to analyze differences in the accuracy of nurses’ prognostication for patient outcomes (survival or non-survival) at two intervals ‒ study enrollment and last point of data collection (for those with more than one data collection point). This last date varied for the subjects depending on date of death or discharge from the hospital, and ranged from 1 to 83 days after enrollment.

RESULTS

Demographic data of the nurses included in the sample can be found in Table 1. The majority of the nurses (80.1%) were female and Caucasian (88%) with an average of 6.1 years of experience in the ICU (Table 1).

Table 1.

ICU nurse characteristics (n=136).

	N (%)	Mean (SD)
Age (years)		32.1 (8.9)
Years in practice		7.0 (7.5)
Years in ICU practice		6.1 (6.9)
Gender
Female	109 (80.1)
Male	27 (19.9)
Race
Caucasian	120 (88.0)
African American	8 (6.0)
Other	8 (6.0)

Nurses’ Accuracy for Patients’ Clinical Outcomes Over Time

In terms of accuracy, at enrollment (n=262), nurses correctly predicted the outcome (survival or death) in 172 (66%) cases (Figure 1). Accuracy examined by actual outcome showed that predictions for patients who survived were more likely to be accurate (76% correctly predicted) than for those patients who died (52%) (X²=16.5, p=0.000). By the last time point, nurses’ accuracy improved and predictions were correct in a total of 120 (76%) of cases. Again, there was greater accuracy in predictions for surviving patients compared to predictions for patients who died (82% vs. 66%) (X²=5.61, p=0.02).

Figure 1.

Nurses’ clinical outcome predictions overtime.

Independent t test were run to determine whether nurses’ age and years of nursing experience at enrollment were factors in predicting patient outcomes. Neither nurse age (p=0.35) nor years of nursing experience (p=0.41) were statistically significant in accurately predicting survival.

DISCUSSION

This study revealed that nurses correctly predicted survival of CCI patients in this ICU setting the majority of the time. The accuracy of nurses’ predictions appeared to improve from the time of enrollment to the last time point of data collection. This suggests that the accuracy for predicting patient survival improves as the patient’s length of stay in the ICU increases. This finding is consistent with previous studies conducted with physician respondents^[3]. Additionally, chi-square analyses demonstrated that nurses were significantly more likely to correctly predict survivorship than death over time. Again, this is similar to previous findings that accurate prognostication of non-survival is difficult in the intensive care setting^[14], due to the complexity and acuity of the patients. Interestingly, increasing experience does not appear to impact the accuracy with which nurses were able to predict patient outcomes in this population. Given these findings, the more salient temporal element is time spent with patients since prognostication accuracy improved over time.

This longitudinal study addressed a gap in the knowledge of nurse-specific prognostication in CCI patients across time. Our findings demonstrate that nurses are somewhat likely to make accurate predictions of survival outcomes, but have a tendency to err on the side of survival. However, there are important limitations to these findings. First, the homogeneity of our sample may limit the generalizability of the findings to other populations of nurses. Second, the data available for analysis were limited by what was collected in the parent study so additional questions such as the degree of agreement between nurses, remain unanswered by this study alone. Finally, there are some nurses who cared for the patients on multiple occasions and others who only met the patient at only one point in time. This likely affected their ability to accurately predict survival.

Future studies should examine other characteristics of ICU nurses that may impact the accuracy of survival predictions for CCI patients. Likewise, more research is needed to explore the complexity of accurately predicting non-survival in the ICU in order to support nurses in their important role of counseling family decision makers.