A Computerized Decision Support System Improves the Accuracy of Temperature Capture from Nursing Personnel at the Bedside

Abstract

Objective

To assess the effect of a computerized decision support system (CDSS) on the accuracy of patient temperature recording at the bed side.

Design

This is a randomized, controlled trial comparing nurses assigned to an intervention group that received CDSS whenever they attempted to store a low temperature (≤ 96.4°F) or a control group that received no CDSS.

Measurements

The computer recorded temperatures that would trigger reminders equally in both control and intervention groups. It also logged the reactions of nurses who received reminders and whether they retook the patient’s temperature or chose to store the original low value.

Results

We analyzed the temperature data over a 10-month period tracking a total of 44339 temperatures taken by the control group and 45823 temperatures taken by the intervention group. We showed a 51% relative reduction in the number of erroneous low temperatures stored by the intervention versus the control group.

Conclusion

CDSS are effective with nursing personnel in improving the accuracy of temperature capture at the bedside.

INTRODUCTION

Many studies have shown that computerized decision support systems (CDSS) can improve physician compliance with recommended care practices and reduce cost, and/or improve the quality of patient care.^1–6 Excepting some early work done in the 1980’s ^7–10 the effect of computer reminders on nursing personnel has not been well studied.^{11, 12}

A number of clinicians in Wishard hospital have long suspected that patient body temperatures are often recorded inaccurately low on patients admitted to the routine wards (non-critical care areas). In the process of implementing a bedside computer workstation and a device for automatically capturing blood pressures, pulse rates, temperature and other data, we confirmed that low body temperature readings (Below 96.5ºF) do occur frequently – in more than 7.7% of all temperature measurements on ward services. Such a rate can not be explained by text book causes of hypothermia such as hypotension and exposure in that setting. We also observed a few body temperatures of 89°F recorded on patients who were alert, oriented and comfortable. Shivering begins to occur in most people with core body temperatures below 95°F, amnesia and dysarthria develop below 93°F, and patients become stuporous below 90°F¹³ We therefore suspected the vast majority of these low temperatures on the routine wards were measurement or recording errors. We performed a preliminary investigation on 13 patients with temperatures recorded that were below 96.5 ºF to determine the degree to which temperatures below this threshold were due to measurement problems, and found that more than 92 % of these low measurements could be attributed to a confounding factor such as such as poor probe placement; a cold patient mouth from sucking on ice or drinking ice-water;¹⁴ smoking; recent bathing; or a non-cooperative patient with mental status changes.^15–19 When we repeated these measures by the same or different route (e.g. from oral to rectal) within 30 minutes of the original measure, the repeated measure was body temperature was above 96.4°F in 92% of these cases.

With this data in hand, we feel confident that a large proportion of these recorded low body temperatures (≤96.4°F) were likely artifacts of the measuring process and not accurate. Wishard Hospital’s ready access to CDSS tools ²⁰ and our new bedside computer workstations provided an ideal opportunity to study the effect of a bedside nursing personnel CDSS. We hypothesized that providing feedback to nurses at the bedside could reduce the number of inaccurately recorded low temperatures. Here we report the results of a randomized, controlled trial designed to test this hypotheses on a routine medical-surgical (non-critical care) hospital ward.

To generate the electronic reminders, we used the new bedside computer workstation in our hospital’s medical-surgical (non-critical care) wards. This system consists of a Datascope Accutor Plus patient monitor (Datascope Corp., Paramus, NJ) and a bed side PC (keyboard & LCD screen). The DataScope delivers its results (Blood pressure, pulse, body temperature and Oximetry) to the PC via an RS-232 serial interface that uses a vendor developed, data-exchange protocol. To obtain a patient measurement, the nurse operates the Datascope in the customary manner to obtain a BP, pulse, temperature, and/or oximetry. When measurement is complete, the DataScope send the results’ data to the bedside PC where it is displayed on the LCD screen for the nurse to verify. The nurse is given a chance to correct and/or repeat the any measures, repeat the data capture process, and when they are satisfied with the measurement, press the confirm button on the PC to commit the measurement(s) to the patient’s record. The system can also accept any of the previously mentioned data as well as other bedside measurements including respiratory rate, finger-stick blood glucose, pain scale, patient weight, fluid input and output, and many others using keyboard entry. This allows nurse the capability type in a temperature or other bedside data if measurements were made with other equipment or when the patient was not in the room for example.

The CDSS identifies low temperature values (≤96.4ºF) as soon as they are produced (but before they are verified by the nurse and committed to the patient’s medical record.) In such a case, a pop-up window appears that warns the user about the low value and suggests repeating it at the same or a different body site. This pop-up window also provides the user with some suggestions on how to correctly retake the patient’s temperature. The user can take the computer’s advice and repeat the measurement or ignore it and commit the initial measurement to the patient’s medical record. In the latter case the computer displays a second window to ask the user to provide a reason he or she is ignoring the computer suggestion. Users are required to provide a reason to over-ride the reminder by either selecting an answer from a menu or by typing in their own free text answer. At this point, a user still has the option to back out of this screen and repeat the temperature again or manually type in a value using the keyboard.

Our bedside system is installed in the Wishard Memorial Hospital which is a county-run, teaching hospital with close affiliation to both Indiana University’s Medical School and the Regenstrief Institute, Inc. Wishard is a level-one trauma center with a high volume emergency department, four critical care areas, a step-down unit, and 96 medical-surgical beds. 70 bedside computer workstations are installed on the medical-surgical wards that serve either one or two beds (depending on whether they are installed in a private or semiprivate room.) Systems that service semiprivate rooms are located on the head wall between the two beds (Figure 1) while systems in private rooms are located on the headwall just to the door-side of the bed.

Figure 1.

The Bedside “Gopher” Workstation

Experimental Design

User-IDs are created when nursing personnel are first hired and consists of part of his or her last name plus the first initial or employee number. We randomized nurses to the control or intervention groups based on the check digit (0–9) of their user ID’s. The check digit is generated using an alphanumeric version of the Luhn Algorithm ²¹ that uses all the ASCII characters that compose the user-ID. We assigned the study groups (intervention versus control) based on a coin flip before the study began. Nursing personnel whose user IDs had even number (0,2,4,6,8) were assigned to the control group and those with an odd numbered check digits (1,3,5,7,9) to the intervention group. Nurses in the intervention group received reminders about low temperatures and suggestions to repeat the measurement. Nursing personnel in the control group received no such reminders.

The standard of care of the institution is to allow the nurse to use his or her clinical judgment when determining what route to obtain the original and any repeat temperatures. By policy, if the temperature is low, nursing personnel are required to either retake the temperature or notify the patient’s physician. The computer logged all low temperature events in both groups. When a low temperature event occurred in the intervention group, the computer logged whether the nurse retook the patient’s temperature or chose to override the warning and store the original, low value.

Nursing personnel consisted of Registered Nurses (RN’s), Licensed Practical Nurses (LPN’s), nursing aides, and nursing students. On a typical shift, each 12-patient ward is staffed by one RN, two LPN’s or nursing aides, and sometimes nursing students. Because of the staffing patterns and work processes, most temperature measurements are taken by aids. When nursing students take vital signs, they are supervised by nursing school faculty or other employed nursing personnel. Two subjects changed professional category during the course of the study. One nursing student in the control group took a job as a nursing aide while one nursing aide in the intervention group became an LPN No user ID’s changed during the study so there was no cross over from control to intervention groups or vice versa. We analyzed all results using on an intention-to-treat basis.

IRB Approvals

We received approval from the Institutional Review Board of the Indiana University Medical center to automatically enroll all nurses who worked at least one shift on the Medical-Surgical wards with a waiver of informed consent.

Statistical Analyses

Because each nurse took a variable number of patient temperatures, we calculated total counts and relevant percentages of the number of temperatures recorded, the number of reminders, and the number of low temperatures stored permanently. To estimate and test the effectiveness of the reminder system, we used a nonlinear mixed model with the logit of the proportion of low temperatures stored per nurse as the outcome. The predictors included the group assignment and nurse type as fixed effects and the nurses as random effects. We also examined the interaction between the group and nurse type. Statistical analyses were performed with SAS (V9.1)

RESULTS

We collected data from 12:00 AM, February 11, through midnight, November 30, 2003. During this 293-day period, a total of 337 unique nurses (Table 1) recorded 90,162 temperature readings for an average of 268 temperature readings per measurement taker. The overall distribution of these temperature measurements is shown in Table 2. Nurses’ aids recorded 76% of all of these measurements, RN’s 15%, LPN’s 6%, and nursing students 3%. Note that 88.71 % of the temperatures recorded in the control group and 91.23% recorded in the intervention group were in the range between 97–101.5ºF. Temperatures above 100ºF (11.8% control and 11.1% intervention) were more common than were temperatures below 97ºF (10.3% control and 8.4% intervention) in both groups. A total of 575 temperatures recorded in the control group (1.3%) and 248 in the intervention group (0.05%) were at impossible levels at both high and low ends of the range (< 95ºF or > 110ºF). The most frequently stored temperature was 98.4ºF in both the control (3214) and intervention (3158) groups. The average temperature recorded by the control group was 96.4ºF and in the intervention group was 97.7ºF, and the number of instances of low temperature measurements was 14.5 per subject in the control group and 7.8 per subject in the intervention group.

Table 1.

Unique Nursing Personnel Subjects by Nursing Type

	Control Group (no CDSS)			Intervention Group (with CDSS)
Nursing Type	Subjects (%)		All Recorded Temp’s	Subjects (%)		All Recorded Temp’s
RN	56	(32)	6074	43	(26)	7001
LPN	24	(14)	2646	21	(13)	2577
Aides	45	(26)	33668	51	(31)	35296
Students	48	(28)	1951	49	(30)	949

Table 2.

Distribution of All Temperatures Recorded

	Control Group		Intervention Group
Temperature (°F)	N	%	N	%
< 80	9	0.02	9	0.02
80–90	32	0.07	6	0.01
90.1–95.0	504	1.14	212	0.46
95.1–94.6	1971	4.45	1046	2.28
96.5–98	12501	28.19	14753	32.20
98.1–99.0	16495	37.20	17158	37.44
99.1–100	8371	18.88	8264	18.03
100.1–102	3957	8.92	3894	8.50
102.1–104	446	1.01	446	0.97
104.1–106	23	0.05	13	0.03
106.1–110	0	0	1	0
>110	30	0.07	21	0.05
Total	44339	100	45823	100

N: Total number of temperatures in range given

%: Percent of total of temperatures in group

Almost the same number of low body temperatures was collected by both groups on the first attempt (2451 in the intervention group and 2516 in the control.) However, the intervention group subjects responded to 48% of the reminders to repeat the measurements before committing a final value to the patient’s medical record. Eighty four percent of the measurements taken by intervention subjects in response a reminder were stored as non-low values. The net result was that 5.7% of the temperatures stored by control subjects were below 96.4ºF while only 2.8% of the temperatures recorded by Intervention subjects were that low.

The nonlinear mixed model estimated the computerized reminders reduced the proportion of low temperatures stored by a factor of 0.37 (p<0.0001). Although the breakdown by type of nursing personnel (Table 3) shows trends toward a lesser reminder effect among nurses aids compared to higher trained nurses, these results did not reach statistical significance. A graphical analysis did not show any visible seasonal variation in the number of low temperatures (adjusted for the number of bed-days).

Table 3.

Temperatures Recorded by Nursing Personnel Type

	Recorded Low Temperatures
Nursing Type	Control (%)		Intervention (%)		AR
RN	359	(5.9)	131	(1.9)	4.0
LPN	133	(5.0)	50	(1.9)	3.1
Aides	1881	(5.6)	1066	(3.0)	2.6
Students	143	(7.3)	26	(2.7)	4.6
Total	2516	(5.7)	1273	(2.8)	2.9

AR: Absolute Reduction of low temperature in percent

(%): Percent of all recorded temperatures in group

DISCUSSION

Most studies of CDSS thus far have been done with physicians “..to the near exclusion of other clinicians or potential users.”¹¹ Clearly there is a need for more of this type of research focused on health care workers other than just physicians. Like Andrews showed with respiratory therapists,⁷ our results suggest that CDSS have the potential to affect the behavior medical personnel other than physicians.

Our results show that a CDSS can successfully be employed with nursing personnel to improve the accuracy of temperature collection and this strategy could potentially be used to improve the capture of other bedside data. On one occasion the initial low temperature that triggered the reminder was 20ºF while the repeat value stored was 102.1ºF. This instance illustrates the potential benefit of this kind of bedside decision support system. However, because our study did not assess patient outcomes, it is not clear how improved accuracy in temperature acquisition or other bedside data actually affects quality of care or cost. Also, our study does not prove whether a similar strategy would be successful for use with vital signs other than temperature or other bedside data although intuitively this would seem to be the case. These areas are ripe for future work.

Temperatures values stored that were less than 80 and greater than 110 are obvious data entry errors (and beyond the instrumentation limit of the equipment.) Although the percentage of temperatures stored by both groups below 80ºF was identical, there was a difference between the groups for temperatures greater than 110 ºF that seems to favor the group with the reminder. Because the reminder did not fire for these high temperatures, this may represent a training effect of the electronic reminders on the intervention group to pay more attention to the accuracy of recording bedside data.

If most of the recorded low temperatures are false positive, we believe this may be desensitizing providers to the signal of low temperatures similar to how normal respiratory rates are commonly charted as 20 when that is above the average normal respiratory rate in adults (8–16 breaths/min.)²² As Gardner has shown, the charted clinical data physicians incorporate into their decision making process are often significantly different than the actual clinical readings as recorded by automated systems.⁶ Bedside CDSS may have a significant impact on this problem as we have shown here with patient temperature.

There is not agreement in the literature on what a normal, healthy, adult body temperature is. In fact, Mackowiak and others provided experimental evidence that the 98.6°F value should be abandoned in favor of 98.2°F.²³ For clinical hypothermia, there seems to be a general consensus that hypothermia begins at core body temperatures below 95°F.²⁴ When we analyze the data and compare the number of clinically hypothermic (<95ºF) temperature values stored by both groups, we see the electronic reminders had a 59.5% relative reduction in the number of low temperatures (in the clinically hypothermic range.) This may indicate our choice to fire the reminder at the threshold of 96.4°F may have been a little too high, making our reminder a little too sensitive. This may also indicate the effectiveness of this kind of reminder may be related to the severity of the low value first entered by the nurse. In addition, this could indicate many nurses do not feel temperatures less than 96.5°F but greater than 95°F are clinically inappropriate for this setting.

Implementation of reminders for nursing personnel also requires the appropriate technology to be available at the bedside, right where the data is collected. The results of work done by Pryor and others showed, the “...overwhelming desire by the nurse in favor of the bedside terminal”⁸ over centrally located workstations.^{25, 26} Most hospitals do not yet have this capability. As technology advances and computers become more ubiquitous in the bedside environment, the importance of this type of work is only likely to increase. Regardless of how advanced the field of Biomedical Informatics becomes, it can only be as good as the data it has to work with.