Leveraging Computerized Sign-Out to Increase Error Reporting and Addressing Patient Safety in Graduate Medical Education

SUMMARY

Setting

Electronic reporting systems are important components of the patient-safety movement but have been problematic particularly in capturing information from physicians.

Objective

In an attempt to increase error reporting in our community hospital residency program, we devised a convenient safety log integrated into the computerized resident-patient sign-out.

Measurements and Main Results

Safety-log events are discussed in weekly safety reports with emphasis on developing a safety culture and preventing further events. We report our experience implementing the safety log. The program quadrupled our contribution to the hospital quality assurance system and has led to significant system innovations. Challenges have included sharing information and improving accountability without humiliating participants or alienating co-workers, as well as sustaining the project over time.

Conclusion

Residents are uniquely placed to provide insight into hospital systems. An error-reporting system based in a resident sign-out can leverage this role into an important tool for safety education and care improvement.

INTRODUCTION

Patient safety and quality improvement are crucial concerns in graduate medical education. Current literature bemoans the lack of physician contribution to error reporting.^1,2,3 Many studies have cited that physicians are less likely than other care providers to report near misses or adverse effects. Barriers to incident reporting usually stem from uncertainty about reporting needs and mechanisms, time constraints, and fear of retribution.³ Teaching hospitals have the advantage of physicians-in-training that are providers on the “front line” and have a unique point of view from which to observe patient-safety-related events.^4,5 Most trainees are responsive to the patient-safety movement and believe adverse events are preventable.⁶ Thereby, residents provide a unique and valuable resource for adverse-event reporting. Educating physicians early in their training about safety also encourages a culture of error reporting in the future. We report on the implementation of a program to integrate a safety log in an extant electronic sign-out system, which enables residents to report adverse effects and near misses in a timely and convenient manner and with mechanisms to process, share, and retain this data in an efficient manner.

PROGRAM DESCRIPTION

In July 2004, we began a quality-improvement process directed at improving the safety practices of our 32-member residency program, based in a 300-bed community hospital. It became immediately apparent that a major barrier to improvement was the lack of effective real-time reporting of events. The residents reported observing multiple problems in patient care but felt inhibited by the time constraints and conflicts that arose around the traditional incident-report system. Verbal reports, initially monthly and then weekly, were poorly remembered and tended to focus on emotionally charged events and conflicts. Establishing web-based reports and emails proved to be more difficult than expected, apparently because they were outside the resident’s usual workflow and expectations.

In January 2006, we re-designed our multiple sclerosis (MS) Access patient sign-out database to help improve patient safety. The foundation for our database is the now widely used open-source tool from St. Agnes Hospital, Baltimore.^7,8,9 We improved the user interface to focus the residents on specific safety-related parameters, such as potential for delirium, falls, clotting, and volume overload. The capacity to print progress notes directly from the sign-out was added to encourage regular updates of the data. Participation was mandatory.

On the main data entry form, we placed a conspicuous button with a link to a pop-up window showing our safety event form (see Fig. 1). We added an additional step to discharge asking whether care had been optimal. If not, the safety form would appear. Significant events detected by nursing or pharmacy were entered by the program staff.

Figure 1.

Non-optimal care and safety event form. Screenshot of a MS Access form linked to a resident sign-out database. Each record corresponds to a single admission. Multiple events for a single admission are combined on the same form. Frequency and impact radio buttons are converted into a 1–9 scale with higher numbers corresponding to increased severity. DC Discharge; VTE venous thromboembolism; ED emergency department.

The prompt screen encourages the house staff to quickly capture the following data:

• Date of the safety event

• Time elapsed between the safety event and detection of the event (in hours)

• The person who detected the event (to provide positive feedback)

• Estimation of how frequently the problem occurs (ranging from rare/yearly to frequent/daily)

• Effect of the safety event (ranging from no harm to patient’s death)

• Typical effects of the event (ranging from no harm to patient’s death)

• Free-text area for description of the safety event

• Free-text area for suggestions to prevent similar events

• Classification of the event

  • o Medical care, e.g., incomplete history and physical, missed medication allergy, fall, hospital acquired infection, etc.
  • o Systems issues, e.g., faulty hospital procedure, equipment breakdown, training deficiency

After entry, the data becomes a part of the patient’s data within the sign-out database. Safety data is kept completely confidential and does not enter the hospital’s patient electronic medical record.

Simultaneously, we instituted a weekly “Safety Report” in lieu of a Morning Report conference. At the Safety Report, we extract information on non-optimal care from the sign-out database for the previous week. Adverse events or near misses recognized by ward attendings, pharmacy, and nursing are also discussed. Additional cases generated by discussion but not captured by the system were added to the log after the report. We encourage a round-table discussion mediated by the Program Director and attended by the house staff, on-service attendings, the nurse manager for quality control, and a nurse educator. We seek to constructively discuss and interpret the data—personal culpability is deemphasized. If a safety event appears to be particularly informative, we perform a rudimentary root–cause analysis. Furthermore, we formulate a plan to address the non-optimal care or safety event. This can involve education, systems change, or a combination thereof. All cases are also sent to the hospital quality assurance (QA) committee by the quality control nurse manager to be evaluated for further action. The safety log project itself has been continuously evaluated as part of safety rounds with attention to evaluating trends and impact of changes in procedure.

PROGRAM EVALUATION

Between 12/2/2006 and 4/28/2007, the house staff made 59 reports over 1,983 patient days, averaging 11.8 per month or 30 per 1,000 patient days. This number is substantially higher than what had been historically reported by our housestaff (approximately three per month). Approximately 44% of interns and 38% of all residents were given credit for discovery of at least one case. About 11 cases were contributed by attendings. House staff most frequently reported failures in hospital systems, transition-of-care, and implementation of orders. However, they rarely reported treatment complications or inadequate physical exams, which were not infrequent according to the recollection of faculty. Residents frequently reported their own errors, and these discussions led to a number of direct changes in how residents function on the floor. These included designating a “Team Safety Officer”, usually the pre-call intern; focusing supervision on high-risk transitions in care [e.g., transfers from the intensive care unit (ICU) to the medical floor], monitoring drug interactions and improving communication between team members.

Table 1 shows a portion of the safety log chosen and sorted by severity index. The entries are minimally edited for clarity, leaving the subjective and sometimes judgmental flavor of the entries. The severity index is the product of the resident’s estimate of problem frequency and potential impact. We have found the severity index to be helpful in prioritizing targets for system change and in helping develop curricula to focus new housestaff on the most frequent and significant events. Table 1 demonstrates the variety of system insights and changes directly triggered by the safety log and report. Case 1 helped us uncover that the emergency department (ED) triage vital signs protocols were inconsistently applied to the sickest patients who by-passed triage and were taken directly into the ED. Case 5 revealed a flaw in how the lab reports elevations in cardiac enzymes. Other cases highlight flaws in resident practice, in particular how discharge medications and transfer orders are written, as in case 2. Yet another set of cases bring attention to high-risk medical situations that residents need to be vigilant for, as in case 4, where the team initiating treatment for thrombosis did not appreciate a patient’s history of menorrhagia.

Table 1.

Top Safety Events Sorted by Severity Index*

	Case description from Safety Log	Safety rounds findings, disposition, and follow-up	D	F	I	TI	SI
1	Pt had a delay in diagnosis of neuroleptic malignant syndrome because high temperature not noted at Emergency (ED) triage—still in ICU—carrying a very poor prognosis	Discovered that in ED triage, very sick patients by-pass the vital signs check and are placed in a bed directly. Subsequent assessment is then less standardized, especially with temperatures. Referred to hospital and ED quality assurance	20	9	3	6	54
2	After HTN emergency, patient discharged from CCU. All 5 Rx’s miswritten by intern (no #, dose, name of pt etc). Patient missed all meds for 1 day. Also, no discharge instructions in EMR for me to review	Review of Rx writing. Review of discharge procedures lead to a formal study of discharge prescription errors showing 30% error rate. Eventually, a review by team of discharge medications implemented	24	9	3	6	54
3	Serum phosphorus was 0.7 at 4.30 a.m. The pt had not received any replacement until the pt was transfer to ICU service around 2 p.m.	Delay related to transition and wrong attending name. Plan to improve lab cross-checking and medication reconciliation during transfers	10	5	4	8	40
4	History of fibroids and heavy bleedings was started on treatment for DVT. Presented later with symptomatic anemia and syncope	Resident awareness. Referred to ED quality assurance	–	9	4	4	36
5	Admission to MICU for SOB, 1st cardiac enzymes ~wnl, but 3rd set significantly elevated (1240/92/1.16) but no one called intern/resident to address. Not discovered until a.m.	Patient without chest pain and healthiest patient in MICU. Referred to hospital. Discovered that lab does not call cardiac enzymes currently. Educated residents	12	4	7	9	36
6	Atenolol was written for twice daily and only given once daily since admission	Referred to hospital. Residents educated about double checking meds with atypical sig	48	6	1	6	36
7	Patient with DVT post hospitalization. Did not understand instructions. Stopped LMWH without monitoring. Subtherapeutic	Residents educated regarding importance of careful patient education and follow up	–	5	1	6	30
8	Amitriptyline and tegretol were not restarted on transfer to the medical floor	Discussed with residents. Led to development of electronic transfer orders	72	5	1	6	30
9	Wrong dosing for i.v. metoprolol. Ratio is 2.5:1 or 5 mg i.v. q6	Educated about pharmacy assistance with conversions from PO to i.v.	4	6	1	5	30
10	Difficulty finding pager number, no response on pager for 1 h. Patient with new findings on colonoscopy	Led to creation of list of resident assignment abstracted from sign-out database and linked to the EMR. List includes a back-up call protocol	1	7	1	4	28
11	Cardiologist recommended decreasing metoprolol dose by 25 mg increments from 100 bid; when I went to write order, another consultant (not cards) had already written order to change to 50 mg bid	Forwarded to the Chairman to assist in reinforcing policy that orders should be written by housestaff, or at least communicated to them	1	7	1	4	28

D Delay in hours between event occurrence and detection, F frequency as estimated by the resident ranging from 1 (rarely) to 9 (frequently), I actual impact from 1 (no harm) to 9 (death), TI typical impact as estimated by the resident, SI severity index which is the product of frequency and typical impact

*Selected cases from the safety log. Cases involving common communication and transcription errors or simple failures of implementation were excluded. Approximately 17% of cases fell into this category. Although of less interest to a general readership, they should be noted as a valued source of feedback tool for the hospital.

HTN: hypertension, CCU: cardiac care unit, EMR: electronic medical record, DVT: deep venous thrombosis, MICU: medical intensive care unit, SOB: shortness of breath, wnl: within normal limits, sig: dosing, LMWH: low molecular weight heparins, PO: by mouth

Table 1 also demonstrates several areas in which the safety log was problematic. First, the structure of the data fields caused problems. To lower the barriers to implementation and make the process feel easy, the description field was purposely truncated at 250 characters—meant to be an identifier rather than data for analysis. The resulting data, however, is telegraphic and fragmentary. Occasionally at safety rounds, we found cases to be entirely unintelligible. Frequently, ultimate root–cause analysis demonstrates a problem not recognized during the safety rounds. Bringing cases back to safety rounds after final disposition has been illuminating in these cases.

Second, the case descriptions often frame the problem in a way that is judgmental. In case 2 of Table 1, the resident focuses on the act of an intern miswriting a prescription rather than on a team’s review of medications at a transition or the residency’s failure in educating the intern in prescription writing. At safety rounds, we had to refocus the discussion away from the specific intern to prevent unnecessary humiliation and address the larger issues. The training issue for the intern was referred privately to his advisor. The skill and attention of the moderator, the program director, or chief resident is crucial to intercepting inflammatory issues. We found that successfully doing so helped create opportunities to discuss the practice of a just culture.

Third, as approximately 30% of cases documented a failure in order implementation, we anticipated the potential for creating tension between residents and nursing at a time when we were pursuing improvement of collegiality and collaboration. Indeed, despite active efforts to include nursing in the early months of safety rounds, no representation was forthcoming outside of the hospital QA department. The only feedback we received was a complaint from managers that cases were being forwarded too late and that a 24-h turnaround would allow more active intervention. In April 2007, nursing leadership formally assigned an experienced nurse educator. The presence of a nursing voice that could educate the residents about nursing concerns and constraints changed the flavor of safety rounds considerably and valuably. Subsequently, a specific project to improve resident–nursing communication on rounds has been initiated.

Fourth, sustaining interest and energy in the project has required significant resources. Although all aspects of medical care arise in the safety report, devoting 20% of morning reports to this format is a significant investment. Intern reporting rates appeared inversely related to morale problems. Several seniors complained of insufficient didactic content. Chief interventions that helped fight fatigue included frequently returning to evidence of changes to patient care resulting from their work and rotating the format of safety rounds. As many errors were outside their control, we chose to focus on reducing the time interval between event occurrence and detection—an area which residents could have significant impact. We set a goal of detecting errors within 4 h. The goal helped sustain interest in recurrent problems, which might otherwise have seemed frustrating. Another particularly effective intervention involved the development of a monthly safety prize (a night off of call) given to the intern with the best “catch”. The surprise nights off were provided by a senior resident chosen randomly from a list of residents who had not completed their procedure documentation on time. Residents were selected by the interns for superior safety efforts and rewarded with additional book funds.

Finally, the project has had a number of technical hurdles. Designed and maintained by residents, early versions of the database on which the safety log was built had a tendency to fail. Residents not appreciating the security benefit of individually logging into the server created alternate copies of the database. This compromised comprehensiveness and accessibility. Many residents did not have the skill to assess whether a shortcut linked to the correct database, and so significant information was lost. Some residents also wrote over old records in the database when encountering repeat admissions. As the hospital recognized the value in the safety log, it has devoted increasing technical support to solving these problems. We expect that these issues resulted in decreased reporting, and improvements should make the project even more robust.

DISCUSSION

In our experience, housestaff are readily accepting of safety-event monitoring and review in accordance with the findings of Petersen et al.¹⁰ Recording this data as an intrinsic part of a patient sign-out database provides convenience and encourages participation. Physicians-in-training have the unique advantage of discerning adverse effects before actual patient harm has conspired. The importance of creating a hospital system that effortlessly and continuously collects safety data cannot be overemphasized. However, the greatest benefits to integration of safety issues into the resident sign-out are improved physician compliance and better data collection, open discussions about non-optimal care, and increased disclosure of safety events without the fear of retribution. All these innovations help better identify areas for improvement and empower physicians-in-training as patient advocates. The clinical impact of the new system merits further study.