“Reading Between the Lines” of Flowsheet Data: Nurses' Optional Documentation Associated with Cardiac Arrest Outcomes

Abstract

Historically, short ‘comments’ on paper-based flowsheets conveyed the patient's overall clinical state. We analyzed the content and documentation patterns of electronic health record flowsheets for 201 cardiac arrest patients. Free-text comments were associated with the abnormality of clinical measurements (p<0.05). The documentation of 3 or more comments for acute care patients was associated with a greater likelihood of dying by discharge (p<0.01). Documentation of ICU vital signs greater than the minimum hourly requirement was associated with increased survival of a cardiac arrest (p<0.05). Further analysis of such patterns may be useful for the measurement of nursing knowledge and surveillance activities, interdisciplinary communication tools, and clinical decision making.

1. Background

The effective use of clinical documentation in electronic health records (EHRs) for information exchange may reduce risks to patient safety and increase the efficiency of communication about a patients clinical status and plan [1, 2]. However, poorly designed systems may increase information overload by providing an abundance of raw data with minimal associated context or significance [3]. This information overload may lead to an increased risk of patient harm [3]. Nurses perform surveillance activities to detect potential threats or risks to patients and recognize subtle cues and responses from the patient before physiological changes are evident [4]. The nurses' knowledge gained from performing these surveillance activities may be documented in the EHR in the form of narrative comments associated with clinical measurements [5]; likewise, the nurses level of concern about a patient's clinical state may be evident in patterns of increased documentation. Therefore, the use of the EHR as a tool to communicate a nurse's concern may increase patient safety. Specifically, this may be achieved through the extraction, visualization, and display of synthesized and contextualized knowledge from the EHR. Such intelligent displays may assist clinicians in quickly identifying impending problems as well as solving problems that could not be solved if they were only presented with raw patient data [3].

Narrative nursing notes for hospitalized patients contain distilled knowledge that “tells the patient's story” [6]. Nurses use paper or computer-based documentation flowsheets to organize and structure clinical measurements such as vital signs, intake and output fluid balance and patient care interventions. Flowsheets can facilitate recognition of trends and timely decision-making by clinicians, and they contain the information most commonly sought by physicians [7]. Twenty years ago, Gurman et al. discussed how nurses annotated paper-based flowsheet values with comments written directly on the flowsheet so that “one can get the whole picture of the patient's condition at a certain hour” [5]. Today, EHR flowsheets provide a mechanism for documenting clinical measurements in a structured format. Most commercial EHRs allow for nurses to enter optional narrative (free-text) comments associated with a clinical measurement. For example, in one widely-used commercial EHR, a nurse can right-click on any data entry field within a flowsheet to add a free-text comment that will be associated with that field. An asterisk appearing in the corner of the field indicates that a comment has been entered. To view a comment, clinicians may right-click on the field.

Structured data in the EHR enables automated calculation of physiological risk scores such as the Acute Physiology and Chronic Health Evaluation (APACHE) [8]. However, algorithms using structured data to identify patients at risk of clinical deterioration have historically lacked specificity [9]. One reason for the poor performance of these systems is the inaccuracy (i.e., lack of proper context) associated with the raw data being fed to the algorithms.

Measuring nurses' surveillance activities and recognition of subtle changes in the patient's condition may help identify patients' at risk of deterioration and inform clinical decision making. Technological advances have made automated analysis of documented comments and measurements for display to clinicians through an EHR possible. Therefore, the aim of this research was to analyze EHR flowsheet comments and clinical measurements for content and patterns associated with clinical outcomes. Knowledge derived from this analysis may be useful as a complimentary parameter to physiological data to identify a patient's clinical state. Additionally, this analysis is an incremental step in measuring nursing surveillance activities, nursing knowledge development, and building well-designed EHRs to support clinical decision making.

The specific research questions addressed by this study were: 1) What is the content of nurses' optional EHR documentation?; and 2) What is the association of nurses' optional documentation with a patient's clinical measurements and outcomes?

2. Methods

Data was collected for hospitalized patients who experienced a cardiac arrest during their hospital stay at a large urban teaching hospital that uses a commercial EHR. Institutional review board approval was obtained prior to data collection. Patient's less than 18 years of age were excluded. All hospitalized patients that experienced a cardiac arrest between September 2008 and December 2009 were included. At this hospital, when a cardiac arrest occurs a “Cardiac Arrest Event Note” is entered by a resident into the EHR. For this study, all of the “Cardiac Arrest Event Notes” entered between September 2008 and December 2009 were retrieved from the clinical data warehouse. A critical care nurse (SC) reviewed the content of each note describing the event to verify that an arrest had occurred and the number of arrest patients retrieved from the clinical data warehouse was verified with data from the hospital cardiac arrest committee. Vital Sign flowsheet data were extracted from our institution's clinical data warehouse. In some cases, nursing assistants entered data into the flowsheets that were assessed and verified by a registered nurse. The data set was constrained to include narrative comments entered and the clinical measurements in the nurses' vital sign flowsheets from 0-48 hours prior to the patient's arrest. The methods consisted of two separate analyses: 1) Content analysis of comments documented on vitals sign flowsheets, 2) Statistical analysis of patterns of documentation of comments and clinical measurements on vital sign flowsheets.

2.1 Content Analysis of Comments

Content analysis consisted of iterative coding of all of the comments. The nursing process (assessment, diagnosis, plan, intervention, and evaluation) was used as a coding framework and additional codes were developed as needed based on the data to capture the content of all the codes. Iterative coding development continued until saturation of the coding occurred and no new codes were identified. After the coding framework was developed, inter-coder reliability was performed by a physician informaticist on a random sample of 10% of the comments. The physician was presented with the final coding framework, including definitions and examples for each code, and asked to use the framework to code the comments. The physician's coding was compared to the coding done by the critical care nurse for that same data items and Cohen's Kappa statistic was calculated. A physician informaticist was chosen for the inter-coder reliability because flowsheets have been cited as the most common physician-reported information source [7] and we anticipate our coding of flowsheet comments to inform the development of an informatics tool, or redesign of EHR flowsheets, to facilitate information exchange between nurses and physicians.

2.2 Pattern Analysis of Comment Documentation Events

Descriptive statistics were used to describe the cardiac arrest patients and the number of comments documented. The Welch two-tailed T-test was used to calculate the mean difference for the number of comments between the patients that died and those that survived. Due to sample sizes, Fisher's Exact Test and relative risk with 95% confidence intervals (CI) were calculated using SPSS statistical software to detect statistically significant associations between the frequency of flowsheet comment documentation and: 1) 7 types of vital sign flowsheet clinical measurements, 2) the 7 clinical measurements being normal versus abnormal, 3) survival of cardiac arrest, and 4) survival to discharge. The characterization of normal versus abnormal clinical measurements was based on clinical literature and established guidelines [8, 10-12]. The 7 clinical measurements from the Vital Signs flowsheet were chosen because they were the flowsheet measurements that had at least one comment associated with them. These 7 measurements were: Temperature, blood pressure, heart rate, respiratory rate, pulse oximeter oxygen saturation (SpO2), blood glucose, and central venous pressure (CVP). Specifically, normal ranges were defined as follows: 1) Temperature between 96.8°F and 101.12°F; 2) Systolic blood pressure between 90 and 130 mm Hg; 3) Heart rate between 50 and 99 beats/minute; 4) Respiratory rate between 12 and 24 breaths/minute; 5) SpO2 between 93 and 100%; 6) Blood glucose between 70 and 150 mg/dl, and 7) CVP 3-12 cm H₂O. We also used Fisher's Exact Test and calculated relative risk to detect statistically significant associations between the frequency of documentation of the 7 clinical measurements and: 1) survival of cardiac arrest, and 2) survival to discharge. To account for patients that stayed less than 48 hours, the number of comments and clinical measurements documented per patient was normalized by the patient's length of stay during the 48 hour period. Institutional Review Board approval was obtained for this study.

3. Results

Two hundred and one patients experienced an in-hospital cardiac arrest between September 2008 and December 2009. At the time of cardiac arrest, 145 (72%) of the patients were on a general medical/surgical floor, 42 (21%) of the patients were on an Intensive Care Unit (ICU), 12 (6%) of the patients were at an off-floor procedure (e.g., interventional radiology, CAT scan), and 2 (1%) of the patients were on a step-down unit.

There were a total of 388 comments. At least one comment was documented for 94 (47%) of the cardiac arrest patients. For the total sample, an average of 1.93 comments were entered per patient (range 0-15, standard deviation 3.059). For patients with at least one comment, an average of 4 comments were entered per patient (range 1-15, standard deviation 3). At least one comment was entered for 71 (49%) of general medical/surgical floor patients, 17 (40.5%) of ICU patients, 5 (41.7%) of patients at an off-floor procedure, and 1 (50%) of the 2 step-down unit patients. Of the 201 patients, 151 (75%) died and 50 (25%) survived to discharge. The 151 patients that died had an average of 2.19 comments (range 0-15, standard deviation 3.30) and the 50 patients that survived had an average of 1.14 comments (range 0-9, standard deviation 1.99) entered in the 48 hours prior to the arrest. This is a mean difference of 1.05 comments (95% CI: 0.28 - 1.82; Welch two-tailed T-test, P = 0.0077).

3.1 Content Analysis of Comments

Iterative coding resulted in eight coding categories for documented comments (see Table 1). Three of the coding categories were based on categories from the nursing process: 1) assessment; 2) plan; 3) intervention. The remaining 5 coding categories were iteratively developed based on the data to capture all of the content within the comments. The 5 developed categories were: 1) physician/nurse practitioner/physician's assistant (MD/NP/PA) aware; 2) equipment information; 3) reference to other shift or location; 4) registered nurse (RN) aware; and, 5) reference to other documentation. Cohen's Kappa for inter-coder reliability of a random sample of 10% (39) of the comments was 0.841 (95% confidence interval: 0.693 - 0.988). Eighty-nine comments contained multiple concepts and were categorized as having up to 4 codes. Fifty comments had 2 codes, 31 comments had 3 codes, and 8 comments had 4 codes. The most frequent combinations were ‘assessment and MD/NP/PA aware’ (n=30) and ‘assessment, MD/NP/PA aware and intervention’ (n=24). For example, the comment that contained the text, “pt alternating nsr 80's and afib 120's. MD X aware. Pt given coumadin 2.5 as ordered. VSS. Lopressor given via duo tube”, was coded as 1) assessment, 2) MD/NP/PA aware, and 3) intervention.

Table 1. Comment Codes.

Comment Code	Frequency$	% of code in total comments (n=388)‡
Intervention	231	59.5
MD/PA/NP aware	143	36.9
Assessment	68	17.5
Equipment information	27	6.9
RN aware	22	5.6
Plan	19	4.9
Reference to other shift or location	10	2.6
Reference to other documentation	4	1.0
Total Codes	524	NA

^$Total Codes is greater than total comments (n=388) because a comment may have multiple codes;

^‡Percentages are of presence of each code in the 388 total comments, not of the 524 total codes, therefore the sum is greater than 100% due to some comments having multiple codes.

A total of 231 (59.5%) comments discussed an intervention related to the associated structured flowsheet measurement. One hundred forty-three (36.9%) indicated that the nurse notified a physician, nurse practitioner or physician's assistant. These comments were typically entered in the form of ‘MD/NP/PA aware’ or ‘MD/NP/PA notified’. The 68 (17.5%) assessment comments provided further detail about the documented measurement. For example, the comment “Unable to obtain” for the blood pressure value of zero indicated that the assessment was done but a blood pressure was not detected (subsequently the nurse manually obtained the blood pressure using Doppler equipment). The 27 (6.9%) equipment comments described information related to the functioning of monitoring equipment. Twenty of the 27 equipment comments described that the calibration of pressure monitoring equipment was done, indicating that the flowsheet value accurately reflected the patient's state and was not an artifact caused by an inaccurate setting or equipment malfunction. Twenty-two (5.6%) comments were documented by a nursing assistant and indicated that the RN was aware about the associated vital sign measurement.

The 19 (4.9%) plan comments discussed the nurses' plan of care specifically related to the clinical flowsheet measurement that was associated with the comment. These comments included plans such as the further assessment of vital signs, administration of a diuretic medication, or titration of a continuous intravenous vassopressor medication to achieve a specified blood pressure goal (e.g., “titrating to map >65”). Ten (2.6%) of the comments referenced other shifts or patient care locations. For example, one hour before the nurses' change of shift, a comment stated: “MD made aware, pt given losartan and hydralazine at this time, day RN notified that these medications were given already.” Finally, 4 (1.0%) comments referenced other documentation, such as “a-fib, see note”.

As a subanalysis, we coded the intervention comments at a finer level of granularity (see Table 2). Fifty-three percent (n=123) of the 231 intervention comments specified the amount and method of supplemental oxygen administered, including 4 comments which specified that the patient was given a nebulizer. Furthermore, 32 (13.9%) of the intervention comments described the settings of a mechanical ventilator or that a patient was intubated. The third most frequent type of intervention was insulin administration (n=25, 10.8%), followed by cardiac medications (n=16, 6.9%). The remaining intervention categories, which each accounted for less than 5% of the intervention comments, were: fluid bolus (n=11, 4.8%), low blood glucose (n=11, 4.8%), no action performed (n=4, 1.7%), cardiac procedure (n=4, 1.7%), laboratory test (n=2, 0.9%), physical therapy (n=2, 0.9%), and pain medication administration (n=1, 0.4%). The categories insulin and cardiac medication include continuous intravenous medications. In the intensive care unit, an order for a continuous intravenous medication typically includes a dosage range within which the nurse could increase or decrease the dose (i.e., titrate) to maintain blood glucose or blood pressure within a specified range. The comments that referred to continuous intravenous medications typically detailed contextualizing factors at the time of a dosage change that influenced the nurse's decision to increase or decrease the medication dose. For example, one comment stated: “Levophed increased to 13 when MAP [blood pressure] decreased after starting CVVH [dialysis].”

Table 2. Sub-analysis of Intervention Codes.

Intervention Code	N	%
Supplemental oxygen	123	53.2
Mechanical ventilation	32	13.9
Insulin	25	10.8
Cardiac medication	16	6.9
Fluid bolus	11	4.8
Low blood glucose	11	4.8
No action performed	4	1.7
Cardiac procedure	4	1.7
Laboratory test	2	0.9
Physical therapy	2	0.9
Pain medication	1	0.4
Total	231	100

Analysis of the EHR system indicated that except for no action performed, each type of intervention comment (supplemental oxygen administered, mechanical ventilator, insulin administration, fluid bolus, low blood glucose, cardiac procedure, laboratory test, physical therapy, and pain medication) entered in the Vital Signs Flowsheet could have been explicitly documented by a nurse in a structured format elsewhere within the EHR. The information contained in these intervention comments could have been documented in the: 1) Medication Administration Record; 2) Assessment Flowsheet; 3) Treatment Flowsheet; 4) Respiratory Flowsheet; or, 5) Blood Administration Note. Therefore, in these 231 instances, the nurse either double-documented or only documented the intervention information in the optional comment field.

3.2 Pattern Analysis of Comment Documentation Events

The 388 comments were associated with seven types of clinical measurements (see Table 3). SpO2 (n=168), blood pressure (n=105), and blood glucose (n=77) were the three most frequent clinical measurements for which a comment was documented and, combined, accounted for over 90% of the comments (see Table 3). The analysis with which comment documentation events were associated with normal and abnormal measurement values are presented in Table 4.

Table 3. Types of Documentation Events Containing Comments.

Type of Structured Flowsheet Measurement	Comments N (%)
SpO2	168 (43.3)
Blood Pressure	105 (27.0)
Blood Glucose	77 (19.8)
Heart Rate	21 (5.4)
Central Venous Pressure	13 (3.4)
Respiratory Rate	4 (1.0)
Total	388(100)

Table 4. Comment Association with Normal and Abnormal Measurements.

Clinical Measurement† or Comment Code§	Association with Normal or Abnormal Value	P-value$
†SpO2	Normal	<0.0001
†Blood Pressure	Abnormal	<0.0001
†Blood Glucose	Abnormal	<0.0001
†Heart Rate	Abnormal	<0.0001
†Respiratory Rate	Abnormal	0.039
§MD/NP/PA aware	Abnormal	<0.0001

^$Fisher's Exact Test

Comments documented for the clinical measurement Sp02 were statistically more likely to be associated with normal clinical values (112 normal measurements versus 56 abnormal measurements, p<0.0001, Fisher's exact test) (see Table 4). All of the SpO2 comments were directly related to the patient's oxygen status and respiratory function. Twenty-eight of the SpO2 comments included MD/NP/PA aware, 150 SpO2 comments included an SPO2 intervention (e.g., “2 Liters nasal cannula”, “nebulizers given”, “100% non rebreather”, and “ABGs [arterial blood gas] sent went up to 75% Fio2 [fraction of inspired oxygen ventilator setting]”), and 19 SpO2 comments included a respiratory assessment (e.g., denies SOB [shortness of breath]”, “no respiratory distress”, “patient SaO2 sat remains 92-93 when previously 96%”, “SOB upon physical exertion”).

Comments were statistically more likely to be associated with abnormal clinical values for the clinical measurements: blood pressure (70 abnormal measurements versus 34 normal measurements, p<0.0001, Fisher's exact test), blood glucose (65 abnormal measurements versus 12 normal measurements, p<0.0001, Fisher's exact test), heart rate (18 abnormal measurements versus 3 normal measurements, p<0.0001, Fisher's exact test), and respiratory rate (3 abnormal measurements versus 1 normal measurement, p=0.039, Fisher's exact test). There were no comments documented for temperature measurement values. There was no statistical association between the documentation of a comment and a CVP value being normal or abnormal (p=0.130, Fisher's exact test). However, the comments coded as MD/NP/PA aware were statistically more likely to be associated with abnormal clinical measurements (107 abnormal measurements versus 36 normal measurements, p<0.0001, Fisher's exact test).

Overall, there was no difference in survival for patients that had zero comments versus patients that had at least one comment; however, general medical/surgical patients were 1.27 times more likely to have died at discharge if 3 or more comments were documented in the 48 hours prior to cardiac arrest (95% CI: 1.06 - 1.51; Fisher's Exact Test, p=0.019,). There was no statistically significant association between the documentation of comments and survival immediately after cardiac arrest for general medical/surgical patients. Additionally, there was no association between comment documentation and survival for ICU cardiac arrest patients.

It is standard ICU practice to document vital signs once per hour. Analysis of clinical measurement documentation revealed that ICU patients were 1.96 times more likely to survive a cardiac arrest if the nurse documented on average greater than 1 Sp02 measurement per hour in the 48 hours prior to arrest (95% CI: 1.06 - 3.59; Fisher's Exact Test, p=0.029). Furthermore, ICU patients were 1.97 times more likely to survive cardiac arrest if the nurse documented on average greater than or equal to 1.5 blood pressure measurements per hour in the 48 hours prior to arrest (95% CI: 1.16 - 3.31, Fisher's Exact Test, p=0.033). There were no statistically significant associations between the documentation frequency of other clinical measurements and cardiac arrest survival or survival to discharge for ICU patients. There were no statistically significant associations between the documentation frequency of any clinical measurements and survival for general medical/surgical floor patients.

4. Discussion

Nurses' perform surveillance activities to detect potential threats to patients and recognize subtle cues and responses from the patient before physiological changes are evident [4, 13]. This study demonstrated that the patterns and content of flowsheet comments and clinical measurement documentation may provide information about nursing knowledge of the patient's clinical state. This knowledge, beyond statistical patterns of a patient's physiological data, may be used to balance the sensitivity and specificity of clinical decision support and automated surveillance systems to detect clinical deterioration [9].

Coding comments documented by nurses in EHR flowsheets was useful to understand the types of clinical information documented by nurses as they perform clinical surveillance activities. The content of comments documented by nurses indicate assessments, actions, concerns, and contextualization about a patient's state, which, overall, may be described as nurses' judgments based on surveillance activities and may be useful in future research for a variety of measures such as nurses' cognitive processes and nurses' workload.

Furthermore, this study demonstrated that patterns of nursing documentation are associated with patterns of normal and abnormal clinical measurements and cardiac arrest outcomes. Therefore, the vital sign flowsheet is rich with valuable nursing knowledge beyond mere clinical measurements. A significant portion of comments associated with blood pressure, blood glucose, heart rate and respiratory rate measurements highlighted abnormal values. Most comments related to SpO2 discussed the amount of supplemental oxygen that patients were receiving. Guidelines define hypoxemia in relation to the amount of supplemental oxygen a patient is receiving to maintain their blood oxygen saturation level. For example, one guideline states: “SpO₂ of less than 90% in subjects breathing room air” (Note: we defined a stricter normal Sp02 as 93% or greater for this study based on current literature) [14]. Kallstrom et al's guideline demonstrates that the proper context of the amount of supplemental oxygen administered to the patient is an important factor for respiratory status evaluation. For example, a patient that is receiving 100% oxygen through a non-rebreather face mask to maintain SpO2 greater than 95% may be considered more unstable than a patient that is receiving 2 liters/minute of oxygen through a nasal cannula with SpO2 between 90 and 95% [15]. It is typical practice on a general medical/surgical floor or ICU for a nurse to independently make clinical decisions regarding adjustments to the amount of supplemental oxygen administered to a patient. The documentation of these changes in relation to the patient's Sp02 may properly contextualize the patient's state beyond a clinical measurement. This nursing practice of communicating the patient's state through the use of comments to contextualize vital signs and other clinical measurements may be defined as a form of interdisciplinary communication through the EHR. However, 36.9% of the comments state that the physician, NP, or PA was notified through another mode of communication and it is unknown how often comments are read and by whom. Additionally, it is unknown if other clinicians are aware that a nurses' comments and documentation patterns may be a reflection of the patient's clinical state.

5. Implications for nursing practice

Providing nurses with intelligent documentation tools that explicitly capture and display clinical patterns and knowledge and provide a forum for communication through the EHR may increase collaborative awareness of the patient's clinical state. Furthermore, increasing charting efficiency via automatic data collection from bedside devices may allow more time for nurses to evaluate, highlight, and annotate data with the proper context needed for an accurate clinical interpretation of the patient's state [16].

Nursing documentation problems, and barriers to nursing knowledge discovery through documentation, have been associated with the perception that documentation is for legal or regulatory purposes and not to communicate information that may be used for clinical decision making [17, 18]. Interestingly, our study demonstrated that the optional information nurses chose to document and the patterns of that optional documentation were associated with the abnormality of clinical measurement values and cardiac arrest patients' outcomes. Therefore, in some instances, nurses' autonomy to use their clinical judgment to choose what information is significant to document and how it should be documented may be a rich source of documented clinical expertise and nursing knowledge. A future research challenge will be to harness and measure this knowledge and these decision-making processes with the appropriate informatics tools and methods to inform clinical decision making at the point of care.

Limitations

This study was conducted at one large urban teaching hospital that uses a commercial EHR. The sample size was limited to vital sign flowsheet documentation for patients that experienced a cardiac arrest. Future analysis should include other types of nursing flowsheets and documentation and comparison to a cohort of non-cardiac arrest patients. Additionally, future work should evaluate flowsheet comments in relation to their role as a clinical communication tool and develop user-friendly comment fields in EHRs to increase their visibility.

6. Conclusion

A total of 388 optional comments for 201 cardiac arrest patients were coded into 8 categories. This study found that documentation within the EHR vital sign flowsheet is rich with patterns and content of nurses' judgments, concerns, and contextualization about a patient's state that are associated with the abnormality of clinical measurement values and cardiac arrest outcomes. These patterns may be useful for the measurement of nurses' surveillance activities, clinical decision-making, and workload. Further analysis of the patterns and content of flowsheet comment fields and clinical measurements may be helpful for predicting patient complications and to inform the development of EHR flowsheets and documentation that support nursing practice, interdisciplinary communication, and clinical decision making.