Development of a Web-based Event Reporting System in an Academic Environment

Abstract

In pursuit of a strategy for patient safety and error reduction, The Ohio State University Health System developed and implemented a standardized voluntary event reporting system. The Web-based application is user friendly as well as context-sensitive and encompasses a broad range of errors, events, and near misses. A full organizational transformation was required to effectively implement the system, which involved process reengineering for event entry and for postentry automated workflows. This system serves as the foundation for efficient and consistent reporting processes, which are essential for encouraging a culture of commitment to patient safety.

The Institute of Medicine (IOM) has identified adverse events arising out of system failures in health care institutions as one of the nation's leading causes of injury and death, contributing to more than 44,000 deaths in U.S. hospitals every year and costing between $17 billion and $29 billion.¹ Although physicians in general do not appear to feel a particular sense of urgency about reporting errors, and the magnitude of the problem is being debated, the IOM emphasizes the importance of health care safety to help prevent injury and death and reduce costs.² From an accreditation perspective, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) identifies safety as a primary focus. Medical and nonmedical literature contain several examples of anonymous nonpunitive reporting systems proving more effective results than mandatory programs for tracking errors.^3,4 The importance of voluntary error reporting in a “blame-free” organizational culture as a way to improve systems and enhance patient safety is also emphasized by the Institute for Safe Medication Practices.⁵

At The Ohio State University Health System (OSUHS), as in many health care organizations, the reporting of medical errors historically has been accomplished by disparate paper-based processes. These processes do not provide a mechanism for anonymity, nor do they lend themselves to quick notification or easy statistical analysis. Additionally, the processes are effort-intensive and expensive to maintain. This report describes the design and implementation of a Web-based event-reporting and tracking tool that overcomes these difficulties. The event-reporting system described serves as the foundation and infrastructure for reporting errors and events in a blame-free culture.

Background

The OSUHS is a large health care system and academic medical center with a staff of more than 1,700 physicians and 5,000 employees. The health system includes The Ohio State University Hospital, a tertiary medical-surgical care facility; the Arthur G. James Cancer Hospital and Richard J. Solove Research Institute, a National Cancer Institute comprehensive cancer center; Dodd Hall, an acute rehabilitation facility; OSU and Harding Behavioral Health, a neuropsychiatric hospital; OSU Hospitals East, a community hospital; and numerous clinics and physician offices. The inpatient facilities at OSUHS contain 993 beds with over 30,000 annual admissions and 750,000 outpatient visits.

The first step in creating a cultural shift in reporting and processing errors, events, and near misses at OSUHS was to determine why errors tend to remain unreported. Uribe et al.⁶ identified a number of specific barriers to reporting events, such as lack of anonymity, time requirements, fear of recriminating lawsuits, and perception that the organization does not make use of the reports. Fear of malpractice claims is particularly pronounced among physicians.⁷ Cohen⁸ cites similar barriers in his report.

While a number of systems and methods for instituting anonymous reporting have been documented, including national medication error reporting tools, the medical literature shows less focus on the organizational processes associated with event reporting follow-up (i.e., verifying the usefulness of the report).⁹ Uribe et al.⁶ suggest that caregivers' apprehension about reporting events can be reduced if they are reassured that organizational processes related to reported events will actually influence change. Ovretveit¹⁰ also supports the importance of focusing on all associated personnel and the entire workflow related to errors and near misses. Ninety-two percent of physicians believe that more training in handling errors is needed.⁷ In addition, Weingart et al.¹¹ suggest that a front-line voluntary system would be feasible based on physician interviews after adverse events had been documented. These observations define the attributes of a reporting system that addresses the issues.

Prior to the development of the OSUHS event reporting system, a design team analyzed the source of events from the existing paper-based reporting system by examining specific results of a broad enterprise survey. This analysis indicated that only 10% of events were reported by physicians and more than half by nurses. Also, a survey of all hospital employees indicated that 63% of staff felt comfortable reporting an event without fear of reprisal (▶). These observations made it important to design a system that met the needs of the reporting constituency. Event reporting baseline information gathering was limited to staff comfort with reporting of events due to disparate and inconsistent processes. Consequentially, quantifying tangible baseline reporting volumes at a health system level would not have been considered valid and, therefore, was not done.

Figure 1.

Staff comfort with reporting events and a breakdown by role of who is reporting events. ^*Patient care resource managers (PCRM) assist physicians in patient throughput and are mostly nurses.

A fundamental objective of the event-reporting initiative was to assure caregivers that the health system could and would respond to reported events. Key goals for the initiative were to develop a user-friendly system that maintained anonymity of the reporter, sustained a nonpunitive approach, and provided a process to rapidly address reported events. In such an environment, individuals reporting events could have the confidence that their time and effort might influence positive change in the institution.

In early 2002, the health system assembled, spanning multiple hospitals, a multidisciplinary task force (design team) of physicians, other clinical staff, quality improvement experts, and software engineers. The team was asked to do the following: develop a reporting system with the capability of analyzing errors and near misses; reengineer the organization's processes concerning such events; and create a culture of increased voluntary reporting. The design team established the fundamental attributes needed to maximize reporting including anonymous access, fast and easy entry of an event (less than 3 minutes); access across the enterprise, inclusion of all event types (including near misses and customer complaints), and integration of process and workflow associated with event follow-up.

To develop the event reporting system, the design team analyzed existing workflow processes. First, the team examined the current paper-based reporting process. All paper forms were reviewed to determine how questions from the various forms could be combined, simplified, and clarified. Second, the team examined the form-routing processes with a focus on problems such as bottlenecks and misrouting of information. Finally, the team gathered information about how various levels of management reviewed and responded to specific events and trends. The evaluation included a variety of people, ranging from nursing unit managers to quality improvement directors and risk managers. The analysis captured a broad perspective of the institution's reporting practices and overall capability to address reported errors and events. The weaknesses associated with all aspects of the paper event-reporting infrastructure of OSUHS were highlighted in the process, which included inaccurate and incomplete information as well as inconsistent practices and/or perception of practices.

The design team then recommended the development of an all-encompassing Web-based event-reporting system to lead an organizational shift to a blame-free reporting environment and a more efficient method of addressing reported events. The recommended event-reporting system design would offer anonymous and easy reporting via a user-friendly Web-based application and would automatically distribute entered events electronically to the appropriate investigative teams.

System Status

The implementation of event reporting began with a pilot phase spanning 12 weeks and included six nursing units consisting of 122 inpatient beds, which averaged 14.6 reported events per week (SD = 7.04). A full implementation rollout began on January 12, 2003. For the following 16 weeks post-live, the same units averaged 16.2 reported events per week (SD = 4.32). Data taken from seven other units, 207 inpatient beds, for the same postlive time period averaged 15.1 reported events per week (SD = 6.14, ▶). The average time to enter an event was 7 minutes and 40 seconds (SD = .20). Medication errors and a generic category of “other” represented the two most common types of events reported, each representing 27% of the total volume (▶).

Figure 2.

Number of reported events.

Figure 3.

Event type breakdown.

Event reporting also offered the ability to track the efficiency of event throughput. Two key times indicating the organization efficiency in detailing events were measured: the “event open time,” which spanned from event entry until an area manager became aware of it, and the “manager complete time,” from event entry until the area manager completed the investigation and information gathering. The averages of these benchmark times were compared in weeks 1 through 20 versus weeks 21 through 30. A statistically significant reduction (p < 0.02) in event open time was observed. The average hours to open decreased by 44.1% from 115.2 hours (n = 966, SD = 42.47) in weeks 1 through 20 to 64.37 hours (n = 1,159, SD = 42.40) in weeks 21 through 30. Likewise, a statistically significant reduction (p < 0.001) in manager complete time was observed. The hours to complete were reduced by 71.2%, from 459.25 hours (n = 1,141, SD = 225.63) in weeks 1 through 20 to 132.1 hours (n = 1,159, SD = 85.56) in weeks 21 through 30. We did not begin capturing hours to open until week 13, thus, the different sample sizes in the week 20 measurement. ▶ displays the complete trend of the benchmark times throughout the first 30 weeks of implementation.

Figure 4.

Event throughput efficiency.

Even though the system allowed and encouraged anonymous reporting, 64% of all entered events individually disclosed their identity, affording the health system to provide individualized feedback.

A policy decision had been made to not repeat the baseline survey results regarding staff comfort with reporting events, which had originally been done in the context of broadly assessing culture and attitudes. Since multiple-year strategies that affect that cultural transformation are still under way, it was decided that a premature assessment would raise problems with methodology and comparability.

Usability Issues

Event reporting is available via a nonpassword-protected link on the OSUHS Intranet homepage. The event reporter is never required to enter any personal information. However, the event reporter may optionally enter basic contact information if he or she would like to follow the status of the investigation.

In reviewing a number of known taxonomies for classifying medical errors, including Victoroff/ASIPS, Dovey/AAFP, Eindhoven, and Harvard, the design team determined that the extensive training required to prepare users to use such classification models was too great, and they decided to utilize already-familiar house language to prioritize/escalate or minimize the acuity of an event.¹² Key functionality requirements of the OSUMC classification system include a simplified number of options, ability to update event attributes dynamically, a broad range of error and event types crossing all patient care environments, and the need to create internally defined communication rules. The design team appointed the quality and risk management departments to define acuity and escalation parameters based on established OSUHS-specific work processes.

Key Functions

Event reporting was written in the Web-based programming language, Cold Fusion, that runs on any standard Web server. The underlying storage is a standard relational database. Data integration and enhanced data trending capabilities were considered by integrating event reporting with the computerized patient record system, physician order entry, information warehouse, electronic mail, and the risk management system (▶).

Figure 5.

Event reporting architecture.

The event-reporting system allows for flexibility in altering content as necessary, such as changing event types, associated details, and business logic. The event reporting system is also used as a vehicle to solicit comments and suggestions for improvement of the system. These comments are collected by the event reporting manager and discussed periodically with the Clinical Safety Committee, which has oversight responsibility for patient safety. In this way, as the needs of the institution evolve, changes can be made by the Clinical Safety Committee without programming intervention from Information Systems. The need for a system that could be adapted rapidly was self-evident, as paper-based forms and processes continually reflected outdated iterations throughout the institution.

Entering an Error or Event

A single standardized method was adopted for reporting all issues, regardless of type or location. This method ensures clear and consistent processes by eliminating multiple steps and replacing them with an uncomplicated and straightforward workflow. Processes for reporting near misses and customer complaints are also incorporated into the design.

The event reporter navigates through a total of only four menu-driven data collection screens. The type of event entered on the first screen determines the content on subsequent screens; details, injuries, contributing factors, and patient interventions differ depending on the type of event entered (▶). For example, a medication error results in different data collection options for a fall event (▶). Physician search capability and formulary integration ensure correct entry of specific information-related errors, which can be used for future reporting and trend mapping. This functionality provides for a user-friendly environment, ease of entry, and integration with other systems. The third screen allows the event reporter to enter optional narrative comments. A summary/confirmation page pops up as a fourth and final screen.

Figure 6.

The first event entry screen.

Figure 7.

Event detail screen for a medication error.

To ensure a minimal time commitment for the event reporter, only essential information is required. Additionally, no personal information is required, thus, preserving anonymity. To eliminate the traditional multistep notification process, the system offers a single reporting method, providing rapid and straightforward notification of even the most critical cases to the quality improvement department or risk management department.

The design team defined four broad categories (details, injuries, contributing factors, and patient interventions) to provide a consistent structure for the user to rapidly enter additional information about any type of event. The categories are event specific to keep the options simple, to help ensure accurate data collection, and to prevent display of inappropriate fields (e.g., display of a medication field when the event being reported is a patient fall).

The event-reporting tool accommodates events with a wide range of event acuities. Predetermined protocols in the design allow immediate notification of specific “critical” events. These protocols—or predetermined criteria—might differ from department to department. For example, the quality improvement department might associate certain attributes of a reported event with a need for immediate notification very differently from the hospital administration department, based on the departments' respective roles in the event investigation process. Finally, the design also allows for free text events to be entered in a category labeled “Other” for situations that would not otherwise fit the structure described above.

Investigation of an Event

The event reporting system allows the workflow and business processes to extend beyond an event report and complete the reporting cycle by incorporating the subsequent investigative phases. The various paper processes that had been used previously to report items such as falls, medication errors, quality inquiries, and delays in service were completely reengineered and streamlined in the event reporting application. The entire event process flow is shown in ▶.

Figure 8.

Event reporting process diagram.

Key Users

Three fundamental groups required accommodation regarding the workflow of reported events. The first group included the individuals closest to the issue who would be involved in the investigation and collection of information, i.e., unit or area managers. Typically, a nurse manager would be responsible for verifying the completeness and accuracy of the information and for adding details appropriate for a specific event. The second group consisted of various ancillary areas that potentially could be involved in an event follow-up, depending on the event type and associated details. The third group included the quality improvement and risk management departments, which function at a health system level and receive event reports from nurse/area managers. All of these groups required notification of new events, which they could review easily by navigating quickly through the event lists. To facilitate efficient use, a display of events would need to show all pertinent details in a condensed format as well as the overall investigation status of all parties involved.

Noncritical events were to be directed to the quality improvement and risk departments following the completion of an initial assessment by the nursing unit. Critical events would require immediate notification, and confidential and sensitive information would not be circulated until the investigation and feedback to the reporting parties had been accomplished. The system also would allow the quality improvement and risk management departments to send an event back to the original nursing unit if incomplete information had been entered.

A high-level event status view summarizes the current events and each group's progress status (i.e., “new,” “in progress,” or “done”). Users may view all events and event statuses appropriate to their areas. A status is symbolized by the first letter of the area conducting the investigation and is color-coded to indicate the status: Red = New, Yellow = In Progress, and Green = Done (▶). Unit level investigators can view the status of events in quality improvement and risk management, but to protect legal privileges for the information, they cannot view comments from the respective areas.

Figure 9.

Event display queue.

Event reporting includes further event routing features whereby the user can rapidly request additional information or forward an event to other areas. For example, if the quality department needs more details about the unit investigation, the event can simply be resent to the original unit manager with a request for additional information. The events forwarding feature also facilitates rapid and easy communication. For example, if an event initially does not appear to have any pharmacy-related components, but investigation finds medication involvement, the event can simply be forwarded to pharmacy and added to its working queue.

If an event report was found to lack sufficient information to draw meaningful conclusions, the quality improvement department carefully noted specific omissions in the report and routed it back to the original owner. In this way, event reporters learned what was needed for a complete and effective report.

Area Managers

Nurse/area managers also identified specific needs. They required the ability to add details and information to an event as well as to change or update the information that was originally entered while preserving the original content. For example, if an event entered for a patient fall indicated that the patient injury was a simple bruise, but a later x-ray showed a hip fracture, it would be appropriate for the nurse manager to update the injury information accordingly. To facilitate the practical aspects of event follow-up and investigation, it was also determined that follow-up duties could be performed by multiple individuals—nurse managers, assistant nurse managers, or clinical nurse specialists.

Ancillary area support services, such as pharmacy or blood services, had the same fundamental requirements from an event-queue display perspective. Additionally, ancillary areas needed a method whereby they could be alerted to their potential role in an event as well as a means to enter additional follow-up information concurrently with nurse managers.

User Access and Security

Event reporting's security structure allows key people in leadership positions to control individual access rights to the respective areas of responsibility. Situations involving system access such as new or cross-coverage staff members can then be addressed in a real-time fashion by the appropriate authorized administrator who can make account changes directly into event reporting. No intervention from Help Desk or Information Systems personnel is required.

Although no user authentication is required to enter an event, once an event is entered it can only be viewed by the appropriate management and quality personnel secured by a username and password. Subsequent viewing or updates to an event are tracked in the database. To protect the scenario of a partially entered event left unattended at a public workstation, a timeout function clears the screen and browser memory after 5 minutes of inactivity.

Discussion

The importance of an effective voluntary and anonymous event reporting method is universally acknowledged.^3,4,5 However, the paper-based forms and procedures that typically support event reporting involve cumbersome event reporting steps and result in inefficient organizational processes when attempting to use the information for overall improvement. Thus, confidence in the effectiveness of the procedure may be diminished, perpetuating the cycle of poor event reporting. The initial 30 weeks of reporting volume show a relatively consistent volume of reported events, although the long-term impact will need to be evaluated. These sustained volumes from the pilot and nonpilot units suggest that event reporting had not been significantly affected by a Hawthorne effect, which often is associated with such initiatives. These volumes were maintained despite the more than 7 minutes' average enter time, which exceeded the original goal of less than 3 minutes. It seemed that the clinicians were willing to spend additional time to elaborate on narrative information to add sufficient details to augment the menu selections.

Sixty-four percent of event reporters voluntarily self-reported, affirming the idea that caregivers are comfortable and confident in the reporting mechanisms and their associated confidentiality. In an environment such as ours, in which our caregivers have a high level of comfort with reporting, these results are not surprising. Further, this offered the opportunity of direct feedback, thus, improving overall confidence about the process.

As part of a larger patient safety strategy, the OSUHS event-reporting application promotes a culture of confidence in a blame-free error reporting structure by overcoming the barriers to traditional reporting methods, such as lack of anonymity, excessive time demands, and delayed or no response to a reporting event. Drastically simplifying the steps and reducing the time required to report are essential to organizational transformation. The simplification is based on the fundamental concepts of making the system user friendly (and Web based), requiring only essential information and anonymity anywhere in the health system, thus, eliminating the need to take notes and transfer it to a “form.”

Because the OSUHS event-reporting system is Web-based and was developed in house, it can be adapted to future reporting needs. One example under consideration involves patient/family complaints or compliments. Large institutions with numerous caregivers often struggle with these issues frequently confusing and intimidating patients and their families. The event reporting system could allow a patient or family member to easily enter a complaint or compliment and, based on the nature of the issues, the appropriate workflow and communication would be triggered. Another feature being considered for event reporting is an online patient satisfaction questionnaire, which would eliminate another complex paper trail.

Reporter confidence in the organization's ability to respond to events is also essential for sustained increase in reporting. To simply increase reporting without process and workflow redesign could result in a higher volume of issues being run through an inadequate structure incapable of handling the additional volume. Quick turnaround of reported events is facilitated by the efficient structure of the OSUHS Event Reporting system. When a caregiver can observe a response to a reported event within hours or days versus weeks or even months, he or she is more likely to report future events.