Identifying Nursing Documentation Patterns Associated with Patient Deterioration and Recovery from Deterioration in Critical and Acute Care Settings

Abstract

Objectives:

Nursing documentation behavior within electronic health records may reflect a nurse’s concern about a patient and can be used to predict patient deterioration. Our study objectives were to quantify variations in nursing documentation patterns, confirm those patterns and variations with clinicians, and identify which patterns indicate patient deterioration and recovery from clinical deterioration events in the critical and acute care settings.

Methods:

We collected patient data from electronic health records and conducted a regression analysis to identify different nursing documentation patterns associated with patient outcomes resulting from clinical deterioration events in the intensive care unit (ICU) and acute care unit (ACU). The primary outcome measures were whether patients were discharged alive from the hospital or expired during their hospital encounter. Secondary outcome measures were clinical deterioration events.

Results:

In the ICU, the increased documentation of heart rate, body temperature, and withheld medication administrations were significantly associated with inpatient mortality. In the ACU, the documentation of blood pressure, respiratory rate with comments, singular vital signs, and withheld medications were significantly related to inpatient mortality. In contrast, the documentation of heart rate and “as needed” medication administrations were significantly associated with patient survival to discharge in the ACU.

Conclusion:

We successfully identified and confirmed the clinical relevancy of the nursing documentation patterns indicative of patient deterioration and recovery from clinical deterioration events in both the ICU and ACU.

1. INTRODUCTION

Preventable adverse events in a hospital setting may be directly related to inpatient mortality.[1,2] Previous research has reported that 210,000 to 440,000 deaths per year are associated with preventable harm in hospitals.[1] These deaths may be avoidable if clinicians could detect and proactively respond to early warning signs of deterioration.[1,3] Effective recognition and observation of at-risk inpatients is an integral first step in identifying deterioration and proper management.[4] Modified Early Warning Score (MEWS) has been used as a standard practice in many hospitals to predict clinical deterioration.[5,6] Despite years of modification, MEWS has had only a modest impact on outcomes.[7,8] However, nurses frequently detect patient deterioration through intuitive perception influenced by familiarity with their patients and experiential understanding of the clinical course of a disease process or condition.[4,9,10] Also, these nurses’ intuitive perceptions or observations of patients may not be formally communicated or documented. Altogether, recognizing deterioration in patients and subsequently escalating care is a highly complex process requiring skill, experience, and confidence. [4,11]

In our prior work, we found that nurses use optional documentation in electronic health record (EHR) flowsheets to contextualize and highlight abnormal clinical data and record their concerns about and increased surveillance of worrisome patients.[12] Therefore, it may be possible to leverage nursing documentation behavior within EHR data to predict patient deterioration.[13] From this previous study, our team also identified a difference between the communication patterns in intensive care units (ICUs) and acute care units (ACUs) among care team members.[12] Some communication patterns occurred between nurses and physicians when nurses noticed subtle changes in patient status or were concerned about patient condition.

Interestingly, we found that the nursing documentation practices focused on communicating concerns were associated with survival in ICUs but a greater risk of mortality in ACUs.[12] While it is well understood that ICU patients are more critically ill, more frequently monitored, and more rigorously treated than ACU patients, we found that ICU patients were more likely to survive than ACU patients when the same documentation practices and patterns were performed.[12] The differences in outcomes between settings may be related to barriers to patient care in the ACU that are not present in the ICU.

In this study, we investigated the nature of these barriers by quantifying variations in nursing documentation patterns and confirming with nurses and physicians how those variations reflect clinical practice. We also identified which patterns were indicative of patient deterioration and recovery from clinical deterioration events such as cardiac arrest, respiratory arrest, and sepsis. We explored the possibility of using the identified documentation patterns to drive improved decision-making to improve patient outcomes at the point of care in the acute care setting.

2. MATERIALS AND METHODS

2.1. Study Design

This multi-method study used qualitative and quantitative approaches, including focus groups, semi-structured interviews, and data mining of EHR data, to identify statistically significant and clinically relevant variables. The motivations for EHR data analytics included comparing different care settings and care processes through nursing documentation patterns that could explain the variation observed in outcomes for patients who experience clinical deterioration in the ICU and ACU.

This study had three phases: 1) initial EHR data analysis, 2) focus groups and semi-structured interviews, and 3) refined EHR data analysis and confirmation of the final findings (Figure 1).

Figure 1:

A Multi-Method Study with Study Phases

2.2. Phase 1: Initial EHR Data Analysis

2.1.1. Data Collection

We collected data from a vendor-based EHR at Mass General Brigham (MGB) on patients admitted to any one of five MGB hospitals between 2015 and 2018. We included all adult patients who had one or more clinical deterioration events (i.e., cardiac arrest, respiratory arrest, sepsis, unplanned transfer to the ICU, and activation of rapid response system [RRS]) during admissions of 24 hours or longer. Patients in the emergency department, oncology, observation units, or palliative care were excluded from the analysis due to the variability of the unique practices in these settings. If a patient experienced multiple clinical deterioration events, the first event was considered the event for analysis.

For patient encounters containing events, independent data variables (e.g., vital sign frequency and comment frequency in nursing flowsheets) were extracted from the EHR and stratified by unit type (ACU vs. ICU). The independent data variables were selected based on the previous research [12,13] and found in two main data types: nursing flowsheets and medication administration records.

2.2.2. Data Analysis

We analyzed EHR data recorded by nurses to determine documentation patterns predictive of clinical deterioration events.

Outcome Measures:

The primary outcome measures were patients who were discharged alive from the hospital or expired during their hospital encounters. Secondary outcome measures were clinical deterioration events. The operational definitions of clinical deterioration events are summarized in Appendix 1. These data variables were found in patient demographic data, claims data, medication orders, and event records.

Statistical Analysis:

A regression analysis using the generalized linear model (GLM) procedure was employed to predict patient outcomes. Discharged alive from the hospital was used as the “survival” endpoint, as opposed to the discharged as the expired or “death” endpoint in our statistical analyses. We utilized sub-sampling, cross-validation, and control of confounders to increase reliability and validity.

For our prediction, we stratified the dataset between the patients who had an event in an ACU and those who had an event in an ICU. Data from the 24 hours preceding the time of each event were collected and transformed into the independent data variables used in the predictive model (e.g., “HR” and “HR comment” represent the total number of heart rate entries and heart rate comment entries in the 24 hours preceding an event). A robust scaler was used to scale each of these independent data variables so that they could be compared to each other without losing outlier effects in the data. A GLM was trained on 70% of this dataset, with 30% used as ten-fold cross-validation.

2.3. Phase 2: Focus Groups and Semi-structured Interviews

We conducted a focus group and semi-structured interviews with nurses and physicians to understand clinicians’ perspectives of recovery activities to mitigate patient deterioration and increase survival-to-discharge in the acute care setting. A focus group guide was developed based on current literature and expert opinion (Appendix 2). [14,15] The first part of the focus group and interview sessions was spent asking participants to discuss behaviors related to recovery activities that clinicians perform to help mitigate patient deterioration. In the second part of the sessions, we presented the initial Phase 1 findings and asked participants for their perspectives and expert opinions on the explanatory variables and identified patterns found in our data analysis.

Qualitative data from interviews were audio-recorded and transcribed. Two researchers (KOS and JG) analyzed the transcribed data using thematic analysis to identify and extract common concepts and themes. [15,16] We conducted consensus coding, member checks, and peer debriefings to ensure our findings’ credibility and dependability (validity and reliability). The main themes were finalized after obtaining consensus between the two coders.

2.4. Phase 3: Refined EHR Data Analysis and Confirmation of the Final Findings

Based on the findings from the thematic analysis in Phase 2, we conducted a refined analysis to exclude RRS activation as a clinical deterioration event. We found that the RRS was activated based on nurses’ personal decisions, and it may not always be indicative of a true state of clinical deterioration in our hospital settings. In other words, nurses could activate the RRS when they were significantly concerned about patients or witnessed an acute change in mental status [17,18], which may indicate a high risk of deterioration. At the same time, nurses could activate the RRS as a preventive measure to avoid clinical deterioration based on their perceptions and experience. After reviewing the raw clinical data as a first step of the analysis, we found that some RRT activation data were not related to clinical deterioration. We presented the results from the refined analysis to the Phase 2 focus group and interview participants to confirm our final findings.

3. RESULTS

3.1. Identification of Nursing Documentation Patterns Associated with Patient Deterioration

The overall data sample contained 8,552 patients. Out of the 1,162 patients admitted to the ICU, 248 patients (22.8%) experienced cardiac or respiratory arrest, 841 patients (77.2%) experienced sepsis, and 85 patients (7.3%) activated the RRS (Out of 85, six patients activated the RRS and then had a cardiac or respiratory arrest). Of the 7,390 patients admitted to the ACU, 347 patients (6.0%) experienced cardiac or respiratory arrest, 3,106 patients (53.6%) experienced sepsis, 1,827 patients (24.7%) activated the RRS (115 patients activated the RRS and then had cardiac arrest sepsis or unplanned transfer to ICU), and 2,340 patients (40.4%) experienced an unplanned transfer to the ICU. A total of 5,108 ACU patients and 774 ICU patients were discharged alive. (Appendix 3).

In the ICU, increases in heart rate and temperature documentation in the nursing flowsheet and withheld medications were significantly associated with inpatient mortality. In the ACU, increases in blood pressure, body temperature, and oxygen saturation documentation, as well as oxygen saturation comments in the nursing flowsheet, were significantly associated with inpatient mortality (Appendix 3).

Pro re nata (PRN) medication (a.k.a. “as needed” medication) administration was associated with an increased likelihood of survival in the ICU. In contrast, increases in respiratory rate entries and PRN medication administration were associated with an increased likelihood of survival in the ACU.

3.2. Understanding of Correlation between Recovery Activities and Documentation Patterns

We conducted three focus groups and two semi-structured interviews with nurses and physicians from ICUs and ACUs. A total of 29 providers participated. The study participants’ characteristics are summarized in Appendix 4. A thematic analysis revealed four main themes related to behaviors when clinicians are concerned about patients: 1) documentation, 2) monitoring, 3) taking vital signs, and 4) communication among the care team (Figure 2).

Figure 2. Behaviors when Clinicians are Concerned about Patients.

*ICU: Intensive Care Unit, ACU: Acute Care Unit, MD: Doctor of medicine, EHR: Electronic Health Record, EKG: Electrocardiogram

Documentation was the most frequently reported activity performed when clinicians were concerned about patient deterioration. Most participants mentioned that they documented nursing notes when the care team was notified about patients who were concerned. The second and third most reported activities performed were monitoring and taking vital signs. In both care environments, nurses reported increases in general monitoring and the monitoring of specific patient deterioration signs. There was a great variety in the types of monitoring reported between the ICU and ACU. Another common theme was communication. Most participants stated that they notified someone else on the care team when they were concerned that a patient might deteriorate.

Overall, participants agreed with our statistical analysis findings, with some disagreement and discussion surrounding PRN medications given in the ICU and respiratory rate in the ACU.

3.3. Finalization of the Clinically Relevant Set of Variables

The Phase 3 data sample analysis contained 6,882 patients. Out of the 1,089 patients admitted to the ICU, 248 patients (22.8%) experienced cardiac or respiratory arrest, and 841 patients (77.2%) experienced sepsis. Of the 5,793 patients admitted to the ACU, 347 patients (6.0%) experienced cardiac or respiratory arrest, 3,106 patients (53.6%) experienced sepsis, and 2,340 patients (40.4%) experienced an unplanned transfer to the ICU. A total of 5,108 ACU patients and 774 ICU patients were discharged alive.

In the ICU, increases in heart rate and temperature documentation in the nursing flowsheet and withheld medications were significantly associated with inpatient mortality. In the ACU, increases in blood pressure and oxygen saturation documentation, as well as respiratory rate comments in the nursing flowsheet, were significantly associated with inpatient mortality (Table 1). Increases in the entry of only singular vital signs in the nursing flowsheet and withheld medications were also significantly associated with inpatient mortality in the ACU.

Table 1.

Independent Data Variables for Patients Who Had Events in the ICU and ACU (Phase 3)

	Events in the ICU			Events in the ACU
Data Variables	Coefficients	Std_error	P_value	Coefficients	Std_error	P_value
HR	5.986	3.077	0.052	−5.576	2.199	0.011
RR	−0.774	0.934	0.407	0.033	1.126	0.977
BP	−1.235	2.802	0.659	1.955	0.728	0.007
BT	3.565	1.448	0.014	−0.535	1.227	0.663
SpO2	−1.143	2.281	0.616	4.425	1.222	<0.001
HR_comment	0.765	1.366	0.575	0.802	2.023	0.692
RR_comment	0.669	0.537	0.213	2.486	1.058	0.019
BP_comment	−0.269	0.856	0.753	−0.886	1.042	0.395
BT_comment	−0.958	1.860	0.607	−0.434	0.956	0.650
SpO2_comment	0.270	0.723	0.709	1.264	0.760	0.096
One_vital	−0.495	1.818	0.785	2.368	1.004	0.018
Set_vital	−0.277	1.410	0.844	−1.040	0.675	0.124
PRN	−1.020	0.660	0.122	−1.801	0.553	0.001
Withheld meds	1.716	0.510	0.001	1.919	0.333	<0.001

HR: heart rate, RR: respiratory rate, BP: blood pressure, BT: body temperature, SpO₂: oxygen saturation. One_vital: single vital sign was documented, Set_vital: A set of vital sign measures were documented, PRN: Pro re nata medication administration. Withheld meds: withheld medication administrations

No independent data variables were associated with an increased likelihood of survival in the ICU, while increases in heart rate entries and PRN medication administration were associated with an increased likelihood of survival in the ACU.

After refining the data analysis, the findings revealed different results from those of the initial Phase 1 analysis (Appendix 3). We confirmed the final findings with the participants and identified heart rate and PRN administration as the only significant data variables associated with an increased likelihood of survival in the ACU (Table 2).

Table 2.

Confirming the Study Findings

ICU	Phase 1 Analysis	Phase 3 Analysis	Summary Opinions of Findings
Predictive for Death
HR	X	X	Participants agreed with this finding. In particular, they are noticing when changes in the patient are more acute, and they are more at risk. Some participants also expressed that they take vitals (e.g., blood pressure or heart rate) more frequently when somebody is on pressors.
BT	X	X	Some participants agreed with the nursing practice of measuring temperature more frequently so that clinicians can get continuous data. Others asked why body temperature was considered a significant feature for predicting mortality in the ICU while respiratory rate, blood pressure, and oxygen level were not. Some participants commented on body temperatures were manually measured, while other data were coming from mechanical ventilators, which may explain the aspects of clinical practice for patient deterioration in the ICU.
Withheld meds	X	X	Some agreed with the findings since withholding a medication generally means the patient is too sick to receive it. A few participants argued with this finding, though, saying if patients are sicker, then they would receive more medications, such as a bolus or antibiotics.
Predictive for Survival
PRN administration	X	-	Some were unsure why an increase in PRN medications given was found to be associated with survival to discharge in both the ICU and ACU. Others wondered which types of medications were considered “PRN” compared to standard, planned orders. Some participants stated that if more analgesic was given for temperature or pain, it would be associated with survival. A few participants expressed that nurses often titrate medications for at-risk patients instead of giving PRN medications. Furthermore, some participants did not understand our interpretation of a PRN-given, while others questioned our methods of analysis and inclusion of PRN-given data in the analytic model.
ACU	Phase 1 Analysis	Phase 3 Analysis	Summary Opinions of Findings
Predictive for Death
SpO2	X	X	Participants agreed with the ACU-related findings. Specifically, some agreed that increases in blood pressure and oxygen saturation entries are signs of increased concern for patients in the ACU. “If you’re checking SpO2 and blood pressures more often, you’re more worried about the patient. If you’re commenting about the SpO2, that’s bad.”
BP	X	X	Participants agreed with the ACU-related findings. Specifically, some agreed that increases in blood pressure and oxygen saturation entries are signs of increased concern for patients in the ACU.
BT	X	-	Only a few participants stated that if patients were septic, body temperature would be frequently monitored. Others disagree with this, saying taking temperature would not be a focus when concerned about a patient.
Predictive for Survival
RR	X	-	A fraction of the clinicians were uncertain that increases in respiratory rate entries were significant in predicting patient survival in the ACU. They noted that respiratory rate is a unique vital sign, and a value within the normal range for respiratory rate was an estimated value, not a measured value, is entered in the EHR by nurses and nursing assistants. One participant felt that within the context of our study, the value of respiratory rate, and not its frequency, was a more suitable proxy for nursing concern.
PRN administration	X	X	They also agreed with the association of increases in PRN medications given with being discharged alive from the ACU. These clinicians stated that for a provider to treat a patient with a PRN medication, such as pain medications, a patient must have a stable enough mental status to complain about a symptom. Additionally, pain is associated with many complications, such as agitation and tachycardia, especially in the ACU. Thus, PRNs given are a good sign of treating pain to lead to better outcomes.

X: Significant, Non-significant, HR: heart rate, BT: body temperature, Withheld meds: withheld medication administrations, PRN: Pro re nata medication administration, SpO₂: oxygen saturation, BP: blood pressure, RR: respiratory rateICU: Intensive Care Unit, ACU: Acute Care Unit

The focus group and interview participants reported that the study findings could be incorporated into tools to help prevent patient clinical deterioration. For instance, a patient status tool could help improve clinicians’ awareness of patients at increased risk of deteriorating before any deterioration occurs. Participants expressed that they would prefer to get a clinical deterioration prediction alert from a patient status tool 24 to 72 hours in advance of a predicted deterioration event. The most common response from the study subjects was a preference to receive updates about patient deterioration risk at handoff between every shift instead of real-time alerts because the alert would be able to show a trend in patient deterioration sometime soon but not observe the physical signs of the deterioration.

4. DISCUSSION

We successfully identified an association between nursing documentation and clinical outcomes when patients experience clinical deterioration and confirmed clinical relevance with clinicians in both the ICU and ACU. There was some disagreement about the initial analysis results, which led to a reconsideration of analysis methods. However, the strength of our study method was that it provided an opportunity to improve the accuracy of the predictive data variables and develop an algorithm to predict patient deterioration. The results of the final analysis generally corresponded to the clinicians’ feedback, and we confirmed that the analysis data aligned with clinical practice. For example, clinicians were unsure why an increase in PRN medications given was found to be associated with survival to discharge in the ICU. Some clinicians stated that if more analgesic was given for temperature control or pain, it would be associated with survival. In the final analysis data, the PRN medication given was not a significant data variable associated with an increased likelihood of death in the ICU. Another example is that some clinicians were uncertain that increases in respiratory rate entries were significant in predicting patient survival in the ACU. Respiratory rate is a unique vital sign and taking the patient’s respiratory rate is not a high priority when the patient shows signs of deterioration in the acute care setting. After the phase 3 analysis, the respiratory rate no longer increased the likelihood of survival in the ACU, which aligned with the feedback from the acute care clinicians. We observed variation in the patterns predicting care outcomes between the ICU and ACU settings. Accounting for this variation is crucial for understanding and promoting the standard use of recovery activities that support the care of at-risk patients across care settings. In our previous ACU study, we concluded that despite increased documentation of concerns about patients, treatment may still be delayed, likely due to the complexity of team communication, resulting in failure to rescue. [19,20] In the current study, our findings aligned with these previous results, but the independent data variables were significantly varied. Our initial data analysis (Phase 1) showed that increases in respiratory rate entries were significantly associated with inpatient mortality in the ACU; however, this finding was contradicted by ACU clinicians’ feedback (Phase 2). After conducting our subsequent analysis (Phase 3), we found that respiratory rate comments in the nursing flowsheet were significantly associated with inpatient mortality. Our findings support prior research showing that an abnormal respiratory rate is an important predictor of serious events such as cardiac arrest and transfer to intensive care. Still, the respiratory rate is often not recorded as frequently as other vital signs. [21] While the frequency of recorded respiratory rate may not adequately reflect nursing concern, the documentation of respiratory rate with comments appears to strongly indicate nurses’ concerns about patient deterioration.

Regarding increased documentation of body temperature in the ACU, we speculated that nurses tended to transfer patients with sepsis to the ICU. This hypothesis may explain our finding that increased frequency in documenting body temperature was associated with patient mortality in the ICU but not in the ACU.

It is feasible to mitigate patients’ risk of deterioration and improve patient recovery from deterioration events in the ACU by quantifying and applying nursing documentation trends related to positive and negative outcomes. Our predictive model showed it could define nursing practice patterns and predict patient outcomes; therefore, our study findings can be used to identify patients at risk of clinical deterioration and develop an algorithm to create decision-support interventions to drive recovery care processes.[22]

4.1. Limitations

Our study had some limitations. First, due to the exploratory nature of the data analytics methods, our findings could be biased by the data values used in the analysis. In the case of sepsis, there is no gold standard for diagnosis, so the sepsis data could have varied depending on the definition. [23,24] Second, we analyzed EHR data from multiple hospitals, but qualitative analysis data from the interviews and focus groups were limited to clinicians from one hospital. Additionally, the number of participants was limited, but most participants agreed with our final findings. Even though we saw agreement between the qualitative analysis data and clinicians’ feedback, the qualitative analysis data may represent an interpretation the participants had not previously considered and challenged implicit cultural norms; hence, the interpretation may have been accurate. This point should be further considered to develop an algorithm to detect clinical deterioration. Further investigation is warranted to validate our analytical methods and identify variability in the results that could be attributed to specific hospitals.

5. CONCLUSIONS

We successfully identified and confirmed nursing documentation patterns that are indicative of patient deterioration and recovery from clinical deterioration in both the ICU and ACU settings. Our next steps are to develop an algorithm to predict clinical deterioration and create a clinical decision support intervention that will be implemented in acute care settings to help clinicians mitigate patient deterioration, leading to better patient outcomes.

Summary Table.

What was already known on the topic

• The frequency of nursing-documentation-related nursing concerns for patients is highly associated with increases in cardiac arrest and mortality.

• There are differences in nursing practice and care team communication patterns between ACUs and ICUs.

What this study added to our knowledge

• Nursing documentation patterns were indicative of recovery patterns for patients at risk of deterioration, but the independent data variables varied between the ICU and ACU.

• We confirmed the clinical relevancy of the quantitative analysis results through qualitative focus groups and interviews with clinicians.