The Effect of Computer-generated Reminders on Charting Deficiencies in the ICU

Abstract

Objective: To examine the effect of computer-generated reminders on nurse charting deficiencies in two intensive care units.

Design: Nurses caring for a group of 60 study patients received patient-specific paper reminder reports when charting deficiencies were found at mid-day. Nurses caring for a group of 60 control patients received no reminders. A group of 60 retrospective patients was also formed.

Measurements: The average numbers of charting deficiencies at the end of the shift in each of the three groups were compared using two planned orthogonal contrasts.

Results: The average in the study group patients was 1.02 deficiencies per day per patient, whereas the control group the average was 1.40 deficiencies per day per patient (p = 0.001). The average number of end-of-shift deficiencies in the pooled prospective (study/control) population was 1.21 deficiencies per day per patient, compared with the average in the retrospective group of 1.56 deficiencies per day per patient (p < 0.001).

Conclusion: The decrease was likely due both to the appropriate response of the nurses to the reminders and to a learned attentiveness to the tasks on the part of the nurses who cared for study patients. Greater gains were hindered by incomplete "coupling" of the reminders to the end-of-shift deficiencies and by inaccuracies in the reminders.

During each shift, the critical care nurse is responsible for performing patient assessments; administering medications; monitoring a multitude of patient parameters; performing hygiene and physical therapy activities; assisting with procedures; consulting with physicians, respiratory therapists, and other caregivers; keeping family members informed; and documenting care. With so many responsibilities and so much information to assimilate, it is not surprising that at times certain tasks escape the nurse’s attention.

At LDS Hospital in Salt Lake City, Utah, intensive care unit (ICU) leaders sought a means of encouraging nurses to perform certain routine activities deemed important to the recovery of the critically ill patient, yet sometimes overlooked. These activities included sedation and level of consciousness monitoring in the form of two scoring systems, oral hygiene, and routine patient turning. Careful monitoring of sedation levels is included in the Society of Critical Care Medicine’s guidelines for sedation.¹ The goal of monitoring is to provide assistance in determining the required level of sedation, while avoiding oversedation with its concomitant increase in ICU and hospital days. Additionally, prolonged use of sedation also may increase the risk of nosocomial pneumonia.²

Although the specific, most effective oral care procedures have not been determined, it is believed that regular oral hygiene in the ICU is important not only in preventing the deterioration of oral health, but also in reducing the risk of aspiration leading to nosocomial pneumonia.^3,4 There are few data in the literature about prevention of pressure ulcers in critical care and in particular how patient turning should be performed. Nevertheless, based on their experience, the clinical leaders of the LDS Hospital Critical Care Department saw regular patient turning as a simple yet potentially effective deterrent to the development of decubitus ulcers and their consequent prolonged care.

The ICU leaders created a Respiratory Failure Care Process Model—a standard process for the management of patients with acute respiratory failure. Performance of the aforementioned activities was a key component of the model. The leaders hoped that adherence to the Care Process Model would expedite movement of the patients through the system while also preventing complications and improving outcomes. Although the Care Process Model was initially defined for patients with acute respiratory failure, the clinical leaders deemed the activities discussed in this research important to the entire ICU population. The first step, however, was ensuring that the activities prescribed by the model were being performed. Thus a standard of care would be established; its effects could then be measured and the effects of changes to the standard could also be measured. Meetings were held with staff nurses to convey the process and solicit buy-in.

Computerized reminders presented a possible aid to ensuring that the standards were met. Alerts and reminders have been shown to have a positive effect on clinician behavior and patient care.^5–12 Most applications, however, have addressed the effect of alerts and reminders on physicians in the outpatient setting. Inpatient reminder studies include the study by Overhage et al. of “corollary orders,”¹⁰ the discussion by White et al. of alerts for digoxin intoxication,¹³ Young’s lab ordering study,¹⁴ and Shea et al.’s length of stay information messages.¹⁵ A search of the literature uncovered only one report of the effects of alerts and reminders on inpatient nurses.¹⁶ Broadening the definition of an alert or reminder would increase the number of citations. The cues and prompts generated by a computerized nursing documentation or care plan system also may be considered a form of “alert” or “reminder.” Their role in motivating more complete charting is mentioned in the literature more often.^17–20

This research reports on a trial implementation of computerized reminders directed toward the ICU nurses and an evaluation of the reminders’ effect on ICU charting deficiencies.

Methods

Two LDS Hospital ICUs were involved in the study: the Shock/Trauma/Respiratory ICU (STRICU) and the Medical/Surgical ICU (MSICU). The STRICU has 12 beds surrounding a central nursing station; the MSICU has 16 beds, also around a central station. The two units care for the same case mix of patients. The difference between the units’ populations lies in the severity of illness; patients in the STRICU are generally more acutely ill than those in the MSICU. The bedside nurse-to-patient ratio in both units is one to one or one to two, depending upon the severity of the patient illness. The nurses work 12-hour shifts, from 7:00 to 19:00 and from 19:00 to 7:00. The day shift, from 7:00 to 19:00, was chosen as the subject of this research for convenience. During each shift, each unit has one charge nurse on duty who is responsible for overseeing nursing care in the unit. No patient is assigned to the charge nurse.

Nurses in the units document care using the Health Evaluation through Logical Processing (HELP) system,^21–22 the hospital’s clinical information system. Nurse charting takes place through character-based screens; the nurses are encouraged to chart their activities as they perform them. Routine activities are charted in the computer every two hours. A charting terminal is available in each patient room. Several terminals are also available at the central nurses’ station. The system has been in use for over two decades, and the nurses are quite facile with the menus and their options.

The research addressed four routine nursing tasks: assessment of Motor Activity Assessment Scale (MAAS) at least once per shift (7:00 to 19:00) for patients receiving sedation or narcotics, assessment of Glasgow Coma Score (GCS) at least once per shift, turning of the patient at least three times per shift, and performance of routine oral care at least twice per shift. These standards were viewed by nursing leadership as established responsibilities of the bedside nurses caring for the population of the two ICUs.

For the purposes of this research, an absence of documentation indicating that any of the four nursing standards had been met was deemed an “end-of-shift deficiency.” For each of the four possible end-of-shift deficiencies, a mid-day (13:00) condition was identified that might predict occurrence of the end-of-shift deficiency, barring subsequent action. The mid-day conditions and end-of-shift deficiencies are shown in Table 1▶.

Table 1 .

Mid-day Reminders and End-of-shift Deficiencies

Mid-day Conditions	End-of-shift Deficiency
Sedation or narcotic use and no Motor Activity Assessment Scale charted between 7:00 and 13:00	Sedation or narcotic use and no Motor Activity Assessment Scale charted between 7:00 and 19:00
No Glasgow Coma Score charted between 7:00 and 13:00	No Glasgow Coma Score charted between 7:00 and 19:00
< 2 turns and no PT activity between 7:00 and 13:00	< 3 turns and no PT activity between 7:00 and 19:00
No routine oral care or teeth brushing charted between 7:00 and 13:00	< 2 routine oral care or teeth brushing charted between 7:00 and 19:00

The mid-day conditions and the end-of-shift deficiencies were translated into new rules in the Table-driven Clinical Rules (TCR) system at LDS Hospital. The TCR system stores rule definitions in a series of relational tables. A rule engine parses the contents of the tables, piecing together rule definitions and executing the rules against the patient data of the HELP system. New rules can be added to the system by creating new entries in the tables without writing procedural programming code.

A program that formed printable reminders from the results of executing the mid-day condition rules was developed. It was hypothesized that delivering these patient-specific reminders to the nurses at mid-day would result in a reduction of end-of-shift deficiencies.

The rate at which end-of-shift deficiencies were occurring in a retrospective population was calculated. A power analysis revealed that a sample size of 60 patients per group would be required to detect between a 20% and 25% reduction in the average number of end-of-shift deficiencies per day per patient with 80% power.

Consecutively admitted patients were assigned to two prospective groups according to the ICU room to which they were admitted; patients admitted to the east side of the STRICU (rooms 7 through 12) and the west side of the MSICU (rooms 38 through 45) constituted the study group, whereas patients admitted to the west side of the STRICU (rooms 1 through 6) and the east side of the MSICU (rooms 30 through 37) constituted the control group. Patients were assigned rather than randomized to reduce crossover contamination. On any given day, a nurse rarely cares for both a patient on the east side of the unit and a patient on the west side; assigning east-side and west-side patients to different groups thus averted a situation in which a nurse would be caring for both a control patient and a study patient on the same day. There was no reason to believe that any differences existed between patients admitted to the east side of a unit and patients admitted to the west side.

Patients with a hospital-admitting diagnosis indicating spinal injuries were excluded from the study because they legitimately would not be turned. Patients less than 18 years of age were excluded because of nonclinical Institutional Review Board issues. Patients who had experienced a previous stay in the STRICU or the MSICU were not enrolled again in order to ensure that no single patient would contribute more than one piece of data to the study.

A patient’s data became eligible for inclusion in the analysis after the patient had been in the unit for one full day. Enrollment continued in the prospective groups until each group had 60 patients with eligible data. To ensure an expeditious end to the experiment, a 24-day limit was set on patient involvement; only the first 24 days of a patient’s stay would be included in the examination.

In addition to the prospective groups, a retrospective group of 60 patients was formed from patients consecutively admitted to, and subsequently discharged from, the STRICU and the MSICU in the period immediately before the prospective assignment of patients. Exclusions were performed as in the prospective groups. Patients who were admitted before commencement of prospective group assignation but discharged after commencement of group assignation were also excluded from the retrospective group.

An enrolling computer program ran automatically each day at 13:00. Based on the previous day’s admit/transfer data, the program added and subtracted from the experiment’s working patient list as necessary to reflect the current state of the STRICU and MSICU units. The program assigned any new patients to the study or control group according to the assignation scheme previously described.

A few minutes later, the TCR system executed the rules representing the mid-day conditions of Table 1▶ by examining the control and study patients’ HELP data. The system stored any mid-day reminders generated (on both control and study patients) in a database. Next, for each patient in the study group, the program previously mentioned generated a reminder report. The report listed the patient room, patient number, patient name, the date, and any reminders that had been generated for the patient. Figure 1▶ is an example of a report showing all four reminders. Monday through Saturday during the experiment any reports containing reminders were delivered to the STRICU and MSICU charge nurses. The charge nurses had been instructed in meetings before the experiment to deliver the reports to the bedside nurses and address the reminders with the nurses.

Figure 1.

Example of mid-day reminder report. “Data updated” is an indicator of whether the data were properly updated before generation of the reminder report. MAAS = Motor Activity Assessment Scale, PT = physical therapy.

At the end of each shift, the TCR system executed the rules representing the end-of-shift deficiencies on the study and control patients. The end-of-shift deficiency rules and the mid-day reminder rules were also executed on the data of the 60 retrospective group patients.

After the study period, the statistic mean number of end-of-shift deficiencies per day was calculated for each patient in the three groups (study, control, and retrospective) by dividing the patient’s total number of deficiencies by his number of valid days in the study. A valid day was a day on which reminders were distributed (Monday through Saturday), excluding the patient’s first and last days. A mean for each group was calculated by averaging each group’s 60 individual patient means. The three resulting means were compared using two planned orthogonal contrasts. The two contrasts performed were (1) the retrospective group mean vs. the mean of the pooled control group and study group, and (2) the study group mean vs. the control group mean.

The TCR system is an experimental extension operating outside the HELP system. Consequently, hand-delivered printed reminders were used as opposed to on-screen reminders integrated into the nurse charting application. Such integration is certainly possible in the future, as more experience is gained with inpatient nurse reminders.

The experiment was conducted with the approval of the Institutional Review Boards of both LDS Hospital and the University of Utah.

Results

The length of time from first enrollment day to last discharge day was 59 days in the study group (4/25/2000 to 6/22/2000), 45 days in the control group (4/25/2000 to 6/8/2000), and 29 days in the retrospective group (3/25/2000 to 4/22/2000). On day 59 of the prospective period, the experiment was terminated with one study group patient still enrolled. This patient had been in the unit for 24 days, and the rule that limited enrollment to 24 days was invoked. No other invocations of the rule were necessary during the experiment.

Fifty-one nurses cared for patients in the control group, 58 nurses cared for study patients, and 61 nurses cared for patients in the retrospective group. Forty nurses cared for at least one patient in both the study and control groups. Fifty-one nurses cared for at least one patient in the retrospective group and at least one patient in one of the prospective groups.

The groups were compared with respect to gender, first day APACHE II score, and days in the study. No statistically significant difference among groups was discovered.

Reminder reports were delivered to charge nurses on 46 of the 59 study days. Day one was the initial enrollment day and reminders were not distributed. Eight of the 59 days were Sundays, on which no reminders were given. On an additional four days, the system detected no conditions that would generate a reminder on any patient.

The mean number of end-of-shift deficiencies for each of the three groups and the results of the planned comparisons are shown in Table 2▶. The differences in both comparisons were statistically significant.

Table 2 .

Results of Orthogonal Comparisons of Average Number of End-of-shift Deficiencies per Day per Patient

Comparison	Mean (Deficiencies/Day/Patient)	SD	n	Student’s T p/Significance
Study	1.02	0.66	60	p = 0.001/yes
Control	1.40	0.60	60
Retrospective	1.56	0.63	60	p < 0.001/yes
Pooled study and control	1.21	0.65	60

SD = standard deviation.

Discussion

Reduction in End-of-shift Deficiencies

The mid-day reminders appeared to reduce deficiencies in the nurses’ charting, as evidenced by the statistically significant difference between the average number of deficiencies in the control and study groups. Apparently, because of the nurses’ many responsibilities, reminders are helpful in focusing the nurses’ attention on specific duties.

A concern foreseen at the time of the experiment’s design was the possible effect of crossover contamination and learning between the two prospective groups. As noted, 40 nurses cared for both study and control patients at some time during the study. A nurse who had at some point received a reminder while caring for a study group patient might be inclined to remember to perform and chart the activities thereafter, even when caring for control group patients on subsequent days. Unfortunately, no practical means of preventing the crossover was discovered. A randomized nurse design would not control for any variability between patients and would also not be sufficiently powerful. A design that controlled for both nurse- and patient-variability would be even less powerful.

To explore the possibility that some other factor besides the reminders was responsible for the change between the retrospective and prospective periods, a post-hoc comparison between the retrospective and control groups was performed. If the difference truly was attributable to the intervention, there should be no significant difference between retrospective and control group means. No significant difference was discovered (p = 1.51).

Effectiveness

The percentage of mid-day reminders that were not followed by charting deficiencies at the end of the shift was calculated for each patient in each of the three groups. This calculation yields a measure of the effectiveness of the reminders. In the retrospective group, the average reminder effectiveness among the group’s 60 patients was 12%—quite low, as would be expected, because reminders were not delivered and hence never acted on. Any avoidance of end-of-shift deficiencies following mid-day reminder conditions occurred by chance. In the control group, the average effectiveness was 16%—also low, as would be expected. In the study group, the average effectiveness was 29%—considerably higher than in either of the other two groups because in the study group, reminders were delivered to the nurses and they responded to them.

Nevertheless, 29% effectiveness indicates that, on average, 71% of the reminders delivered to the charge nurses failed to prevent an end-of-shift deficiency. At least three possible explanations exist. First, it was observed that the reminders and the end-of-shift deficiencies were incompletely “coupled.” For instance, in response to the reminder “No routine oral care or teeth brushing between 7:00 and 13:00,” the nurse might immediately perform and chart a “routine oral care.” However, to avoid the associated end-of-shift deficiency“ < 2 routine oral care or teeth brushings between 7:00 and 19:00,” at least one more charting would be required at some point before 19:00. In other words, a direct response to the reminder was not always sufficient to prevent an end-of-shift deficiency; an improved and sustained attentiveness during the remainder of the shift would have also been required. If such sustained attention failed to materialize, an end-of-shift deficiency would have occurred despite an immediate response to the reminder.

A second possible reason for the suboptimal effectiveness of the reminders was that the nurses viewed them as inaccurate or not applicable and consequently ignored them. There may have been legitimate reasons for not performing the prescribed activities (for instance, if the patient was on transport or in surgery for most of the day). The reminders were oblivious to many of these legitimate reasons and generated clinical “false alarms.” (In such cases, the end-of-shift deficiencies that followed also would have been “false alarms.”) In addition, conversations with individual nurses and responses to a questionnaire distributed to the nurses after the experiment revealed that not all nurses agreed that the tasks were important—or they agreed that they were important, but not at the frequencies specified. Such disagreement undoubtedly caused others of the mid-day reminders to be intentionally ignored.

Finally, it is possible that the bedside nurses did not receive the reminder reports from the charge nurses, since the delivery of the reports to the bedside nurses by the charge nurse was not verified during the experiment. However, failure to receive the reports probably occurred only in isolated instances. Often the charge nurse would begin distributing the reports immediately on receiving them. Also, a questionnaire was delivered to all nurses who should have received reminder reports during the study period (based on the nurses’ schedule book). The questionnaire asked that nurses who had received a report complete the questionnaire. The great majority were completed.

Rates of Generation of Mid-day Reminders

The reduced rate of end-of-shift deficiencies in the study group of patients has been attributed to the response of the nurses to receiving reminders. A nurse received a reminder at mid-day, indicating that the activities (or their chartings) had not yet been performed, and acted to rectify the situation. Another possible explanation for the differences, however, may have been that the nurses caring for study patients complied with the task requirements from the beginning of the shift (as opposed to complying after receiving reminders) more often than the control nurses. To examine this possibility, the number of mid-day reminders per day was determined for each patient and a group mean was calculated for each of the three groups. The mean number of mid-day reminders per day per patient in the study group was 1.03, whereas in the control group the mean was 1.37. The retrospective group mean was 1.56. Two points of interest regarding these means prompted further exploration.

The first point of interest was that the mid-day reminder mean in the study group was considerably lower than in the control or retrospective groups, seemingly in support of the conjecture that nurses who cared for study patients were more vigilant from the beginning of the shift, not just after reminders were issued. A comparison of the groups’ mid-day reminder averages was performed using pair-wise post-hoc nonorthogonal contrasts, adjusted using Tukey’s HSD (Table 3▶). In the study group, the mid-day reminder per day average was indeed significantly lower than in either of the two other groups.

Table 3 .

Results of Post-hoc Comparisons of Mean Mid-day Reminders per Day per Patient

Comparison	Mean (Mid-day Reminders/Day/Patient)	SD	n	Student’s T* p/Significance
Study	1.03	0.66	60	p = 0.017/yes
Control	1.37	0.67	60
Study	1.03	0.66	60	p < 0.001/yes
Retrospective	1.56	0.73	60
Control	1.37	0.67	60	p = 0.287/no
Retrospective	1.56	0.73	60

*Significance level is 0.05, with a Tukey’s HSD correction.

SD = standard deviation.

There are several possible explanations. First, the nurses who cared for study patients may have been by chance more vigilant in the performance of the tasks. Because of Institutional Review Board concerns, tracking of individual nurse performance was not permitted, and this conjecture was unverifiable. It is improbable, however, in light of the following three observations:

1. When examined separately, both ICUs participating in the experiment exhibited the same characteristics; the study group’s average end-of-day deficiencies per day per patient was lower than both the other groups’ averages, and there was less difference between the control group and the retrospective group. It is highly unlikely that in both units the more compliant nurses happened to take care of the study patients.

2. As stated previously, 40 of the 51 nurses who cared for at least one control patient also cared for at least one study patient, and 51 of the 61 nurses who cared for at least one retrospective patient also cared for at least one patient in one or both of the prospective groups. Such mixing also makes it unlikely that especially compliant nurses cared for the study patients.

3. If the compliant nurses by chance cared for the study patients, leaving the noncompliant nurses to care for the control patients, one would expect to see greater disparity between the study group’s statistic and the control group’s statistic, with that of the retrospective group falling between, somewhere near the middle. Instead, the retrospective mean was statistically similar to the control group’s mean, whereas both the control group and the retrospective group were statistically dissimilar from the study group.

Another possible reason for the lower mid-day reminder rate in the study group may have been that the nurses caring for study patients experienced the Hawthorne effect (i.e., the tendency of subjects to improve their performance when they know they are being observed). In other words, did the nurses caring for study patients know they would receive reminders if they did not perform their tasks and consequently performed them before the reminders were generated, whereas the nurses caring for control patients knew they would not receive reminders regardless of their performance and so disregarded the requirements? No sure conclusion is possible, but anecdotal evidence suggests that nurses generally were not aware of the existence of a study and control group. Nurses attending a STRICU staff meeting subsequent to the study were asked if they had been aware that only nurses caring for patients on the east side of the unit were receiving reminders. They replied that they had not. Other bedside nurses in both units gave the same response. The existence of two groups could have been detected without much difficulty, but apparently nurses were so preoccupied with their work that they did not pay attention.

A last possible explanation is related to work practices in the units. A nurse in the ICUs commonly works two or three consecutive days and is usually, when possible, assigned the same patients on those days. Therefore, a nurse caring for a study patient may have received a reminder or reminders on the first day that he or she cared for the patient. On subsequent days, as the nurse continued to care for the study patient, he or she may have become more cautious about the four activities addressed by the reminders because of the reminder received earlier. In other words, learning may have taken place not between study and control groups, as discussed earlier with regard to contamination, but within the study group.

To investigate this hypothesis, calculations were performed to yield the average number of reminders per patient that occurred on day one of a patient’s stay, the average on day two of a patients’ stays, the average on day three of a patients’ stays, and so forth, for each of the three groups. The results for the three groups are shown graphically in Figure 2▶.

Figure 2.

Average number of mid-day reminders per patient, by patient’s ICU day.

As the figure shows, on days two and three mean reminders per day drop sharply in the study group. This seems to support the conjecture that if a nurse caring for a study patient received reminders, he or she may have been more motivated to perform the related activities on subsequent days he or she cared for that patient.

Of interest, even in the control and retrospective groups the rate of mid-day reminders experienced a decrease over time, albeit not as pronounced as in the study group. This may be due to an increasing chance for detection, as the patient’s stay continued, of the nonperformance of the activities addressed by the reminders. For instance, if the patient is not being turned appropriately, it is increasingly likely as the patient’s stay continues that someone will notice—either by reviewing documentation or by physical manifestations of neglect on the patient’s skin. Similarly, the effects of irregular oral care may eventually begin to be noticed in the patient’s mouth. These discoveries may trigger better compliance in the later days of the patient’s stay.

Another possible explanation is that the nature of the activities is such that they are clinically less important to perform before the patient has been in the unit for a certain amount of time. However, the clinical leaders judged the activities of this experiment to be of importance from the outset of the patient’s stay, making this explanation doubtful.

Figure 2▶ also illustrates that the retrospective group’s daily means were consistently higher than not only the study group’s but also the control group’s through the first three days. This pattern may indicate some degree of Hawthorne effect in the control group as well as the study group. Nurses caring for patients in either prospective group—study or control—were more keenly aware of their duties because of the reminders.

Another point of interest in the rates of mid-day reminder generation was that the mean number of mid-day reminders per day per patient in the study group (1.03) is virtually the same as the group’s mean number of end-of-shift deficiencies per day per patient (1.02). The same was also true in the control and retrospective groups, but in those groups the observation is not surprising. Some reminders were not followed by end-of-shift deficiencies, and some end-of-shift deficiencies were not preceded by mid-day reminders. The two effects in the control and retrospective groups may tend to negate each other, resulting in approximately equal rates of generation of mid-day reminders and end-of-shift deficiencies.

In the study group, however, nurses were acting on reminders; thus, one would expect the end-of-shift deficiency level to be substantially lower than the mid-day reminder rate. This was clearly not the case. The study group patients experienced a greater number of mid-day reminders not followed by end-of-shift deficiencies than did the other two groups. But they also saw enough end-of-shift deficiencies not preceded by mid-day reminders and end-of-shift deficiencies that occurred despite mid-day reminders to negate the gains earned by the effectiveness of the mid-day reminders.

The incomplete “coupling” of reminders and deficiencies, discussed earlier, may explain this negation. The memory of previous reminders may have caused study nurses to be more diligent in the mornings, thus lowering mid-day reminder rates. But end-of-shift deficiencies would be prevented only through an increased and sustained attentiveness throughout the remainder of the day. If the study nurses were not also substantially more diligent than control nurses during the remainder of the day, their gains would have been negated. Perhaps the memory of reminders was able to cause nurses to perform as necessary in the morning to avoid a mid-day reminder but was not always forceful enough to cause the sustained attentiveness throughout the day that would forestall an end-of-shift deficiency.

This hypothesized effect of incomplete coupling poses an important question in the implementation of reminders: When, how frequently, and in what situations is it best to remind? Common sense suggests that the best time to remind would be at the time that the task is required, but the nature of the nurse’s work processes is conducive only to frequency requirements for most tasks (e.g., “perform twice per shift” rather than “perform at 10:00 and 14:00”). Reminders, however, can be tied to some other activity known to be performed with certain frequency. For instance, if computer charting is required four times per shift, a nurse’s entry into the computer charting program could trigger reminders regarding tasks that are required four times per shift.

The end of the shift may be a good time to display reminders for tasks that need to be performed once during the shift but not at a particular time. However, it is of course an inappropriate time to remind about tasks that need to be done repeatedly throughout the shift. For instance, it would be inappropriate to remind at the end of the shift that turning had not been performed all day—it would be too late to perform the day’s turnings.

End-of-shift reminders also may be appropriate regarding tasks for which real-time or near-real-time charting is not expected. However, LDS Hospital encourages near-real-time charting of nursing activities. The hospital has a long history of implementing clinical decision support that depends on near-real-time documentation. Furthermore, the ability of a nurse to remember at the end of the shift how many times he or she turned the patient during the past 12 hours is questionable. As the demands on the nurse’s time increase, the balance between the benefits of real-time charting and the nurse’s time needs to be constantly reassessed, but for the time being these activities are expected to be documented in as near real-time as possible.

As discussed, reminders once per shift may have been insufficient to produce the desired effect. Too frequent reminders, however, could easily prove annoying and cumbersome. The consequences of failure to perform the addressed task must be weighed against the cost of introducing more reminders.

Finally, the situations in which reminders are used must be carefully determined. Because reminders represent both an implementation expense and a possible imposition on caregivers, their implementation should be reserved for situations in which the clinical or financial consequences merit it.

No data were collected to evaluate the effect of this specific intervention on clinical outcomes; the reduction in charting deficiencies cannot be directly translated to patient outcomes. However, since institution of the Care Process Model, the ICU clinical leaders have seen decreases in their hospital, ICU, and ventilator days and reductions in nosocomial infection rates. They believe that the nursing activities addressed in this research are an important part of that model. Future investigation may establish more firmly the link between the particular activities and outcomes, and between computer reminders and the performance (rather than the charting) of the activities.

Assumptions

Implicit assumptions in the implementation of this experiment’s reminders (and, indeed, of any effective reminders) are that (1) recipients have internalized the importance of the tasks about which they are reminded and (2) they simply need their attention focused in order to comply. These assumptions were not always valid during the experiment.

With regard to assumption (1), as has been noted, it became evident during the study that some nurses did not agree with what they were asked to do. Although the clinical leaders had sought since the formulation of the Care Process Model to gain consensus among the staff nurses and had discussed the activities on multiple occasions, some nurses continued to disagree. To these nurses, the reminders probably led only to resentment rather than to increased compliance. Reminders are most effective when the recipients value the importance of the task that they are reminded to perform. Violation of this assumption hampers the effectiveness of the reminders and has the potential of fostering a negative attitude toward computerized reminders in general.

Assumption (2) was probably true only to the extent that assumption (1) was true. Furthermore, even in nurses who agreed with the task requirements, the reminders apparently were able to elicit a short-term response or to modify behavior on subsequent mornings but were not always able to elicit sustained attentiveness to the tasks throughout a shift.

Another assumption in this particular experiment was that the end-of-shift charting deficiency was a valid measure of clinical care. Such was not always the case. Some nurses welcomed the reminders, commenting that they were a good means of jogging their memories to perform the tasks. In such cases, the reduction in end-of-shift charting deficiencies likely represented an increase in the performance of the tasks. On the other hand, one nurse complained that “any good nurse” knows that turning a patient is important, but he did not see the need to document each turning. And another nurse felt that the reminders were useless because they would not influence nursing care, but only its documentation. She predicted that nurses who did not perform the tasks would simply chart that they had done so to avoid receiving reminders. These comments highlight an important distinction—that between the performance of the nursing activities and the documentation of those activities. Certainly, the performance of the tasks is the weightier matter, and ideally the experiment would have addressed the performance, not the documentation, of the activities. The correspondence between the two (i.e., the proportion of the deficiencies due to nurses not performing the tasks as opposed to performing them but not documenting them), was unknown. An interesting investigation, outside the scope of this experiment, would be to assess that correspondence. For the purposes of this experiment, charting was determined to be a pragmatic, if not infallible, indicator of care. However, until further experiments are performed, the results cannot be extrapolated to conclude firmly that the reminders resulted in more appropriate performance of the tasks as opposed to more appropriate charting.

If not assumed, it had at least been hoped that the reminders would be viewed by the nurses as an aid to performing their jobs better. In many nurses this seemed to be the case. Other nurses, however, saw the reminders as “policing” or as criticisms of their care. These nurses especially resented any “false alarms”—reminders that were generated although there were legitimate reasons for not performing the activities.

Such attitudes have important implications for plans to implement permanently the reminders of this experiment or to implement any reminders. The reminders had a positive effect on preventing end-of-shift deficiencies, but the impact of permanent implementation on morale and culture cannot be ignored. Technical changes can assist the process. The reminders can and should be refined to reduce the “false-alarm” rate. The most effective and best accepted means of reminding (e.g., paper report delivered to the nurse, messages sent to the computer monitor) can be explored. But as is often the case in medical informatics innovations, the organizational and sociological changes necessary for a successful implementation may prove as difficult as the technical changes, if not more so.

Conclusions

The computer-generated reminders had a significant effect in decreasing end-of-shift charting deficiencies. The data suggest that the decrease was due in part to nurses’ responses after receiving reminders (as evidenced in the study group by greater “effectiveness” as defined in the study) and in part to the fact that nurses in the study group attended better to the four tasks from the beginning of the day. Incomplete coupling of the reminders to the end-of-shift deficiencies (and the resultant lack of a sustained effect on the nurses through the remainder of the shift) as well as inaccuracies in the reminders probably precluded a larger effect. Nevertheless, the results suggest that implementation of computerized, patient-specific reminders is a viable means of affecting positive change in the behaviors of ICU nurses.