Abstract
Problem
To improve the timeliness, efficiency, and effectiveness of occurrence reporting.
Setting
Baylor University Medical Center, a 1000-bed tertiary facility, and other components of the Baylor Health Care System, all located in Dallas and the surrounding area.
Strategies for improvement
Designing a custom Web-based patient occurrence reporting system through the efforts of the Center for Quality and Care Coordination and Information Services and training staff not only in using the system but also in viewing reporting as a key element for quality and safety rather than as an individual performance or disciplinary measure.
Effects of change
Both the number of occurrences reported and the timeliness of reports improved: prior to implementation of the Web forms, an average of 128 paper patient occurrence reports were received per month an average of 7.6 days afterthe occurrence; in the 12 months afterwards, an average of 175 patient occurrence reports were submitted, with 82% reported within 24 hours of the occurrence. Anecdotal comments about the system from end users have been positive.
Conclusion
Paperless Web-based systems are timely, efficient, and effective. The system developed at Baylor met its goals, including being secure, offering online helps and formal education, creating a risk stratification model, remaining adaptable for system needs, and serving as a tool for data analysis. Efforts to change the culture around patient occurrence reporting are ongoing and an essential element for success.
Occurrence, incident, or variance reports, regardless of title, are familiar to hospital staff. Facilities use such reports to internally document unusual events. Analysis and trending of report data provide a means to drive process improvement and patient safety initiatives. Reporting also meets certain external agency requirements. For example, the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) requires an appropriate response to certain events in its accreditation standards.
Some external agencies specify a response time in their standards. If the occurrence meets JCAHO's criteria for a sentinel event, a facility must perform a root cause analysis and prepare a process improvement plan within 45 days of the occurrence. If the event involves a medical device and meets reporting criteria, a facility has 10 days to file a MedWatch report with the Food and Drug Administration according to the Safe Medical Devices Act. Complete and accurate data capture and timeliness of reporting can have a major impact on a facility's ability to use this tool for process improvement, risk management, and regulatory compliance.
The traditional reporting method is manual: when an unusual event is discovered, a staff member completes a paper form. The manager then reviews the form and forwards it to a quality or risk management department. There, the information is entered into an electronic format and the data are analyzed. Multiple factors can delay the interval between the occurrence and the availability of the data for electronic analysis. For example, paper forms may not be up-to-date, readily available, or easy to use, and information on the paper forms may be illegible or incomplete. Further, paper forms are not very secure as they can be lost, misplaced, or photocopied.
In fall 1999, the Center for Quality and Care Coordination (CQCC) at Baylor University Medical Center assessed the facility's process for patient occurrence reporting. Medication variances were already being captured electronically, and defined processes for capturing visitor and employee occurrence data existed. What was missing was a timely, efficient, and effective method to capture all other patient occurrences (i.e., nonmedication events).
This article describes the improvements that were implemented: a custom Web-based format for patient occurrence reporting accompanied by education of staff. The Web-based form was made available not only to Baylor University Medical Center—a teaching hospital with almost 1000 licensed beds-but also to other components of Baylor Health Care System.
METHODS
In February 2000, a team consisting of 3 CQCC staff members and 3 staff members from the Department of Information Services (IS) began to meet weekly. They identified the following goals for a Web-based patient occurrence reporting system:
To create a secure, confidential, and internal method to accurately report any unusual, unanticipated, or unexpected patient occurrences
To provide online helps for the end user along with a test site for training and education
To increase the efficiency and effectiveness of the current process to capture higher-quality data To provide various levels of security access based on roles and areas of responsibility
To update the current data model and standardize it across the health care system
To create a risk stratification model with an associated alert mechanism
To make individual and composite data available on a real time basis to administration and management via the intranet
To comply with various external regulatory guidelines and standards
To serve as a tool for researching trends and benchmarking
To build an application with scalability to grow with the organization and be responsive to its needs
To change the culture and mind-set regarding patient occurrence reporting
Development of the Web form and related system components
When choosing a host for the Web-based patient occurrence reporting system, CQCC and IS decided to use the existing framework: Baylor's intranet, which contains departmental Web pages, online resources, and news. The intranet was readily available to all staff from any personal computer across the health care system. The team also decided to develop the program in house. The major advantages were complete control of sensitive data on Baylor's server, the ability to direct and modify the application based exclusively on the needs of the health care system, and the opportunity for individual and composite systemwide reporting and analysis. The custom-made approach avoided vendor issues such as contracts, service agreements, fees, and vendor reliability or longevity. Technological aspects of the system are described in Table 1.
Table 1.
Technological features of the patient occurrence reporting system developed for Baylor University Medical Center
| • Access: through the intranet from any work station within the Baylor Health Care System |
| • Hosting services: provided by Baylor Information Services, with servers maintained 24 hours a day, 7 days a week by technical personnel and with incremental backups 6 days a week and complete system backups once a week |
| • Platform: Microsoft NT4, SP6 using Microsoft's most advanced Internet information server (Microsoft IIS) and the Microsoft SQL 6.5 database* |
| • Programming language: ColdFusion 4.0† |
| • Security: NT Authentication servers are located behind the Baylor Health Care System network firewall |
| • Analysis: real-time risk analysis of collected data by designated managers; graphical and tabular data analysis; standard and user-controlled ad hoc query and reporting |
| • Modifications: options for local system administrators to view/edit details for their respective facilities as well as a global administrative level for viewing/editing details systemwide including, but not limited to, adding new forms or editing/archiving existing forms, questions, or answer options |
*Servers will soon be upgraded to Windows 2000 Active Server, SQL Server 2000.
†ColdFusion will be upgraded to version 5.0 when the servers are upgraded.
The data elements needed on the Web form were evaluated. Some questions on the paper forms were not useful; others needed reformatting or different answer options; still others needed to be added. Input from key individuals from various departments was sought during this process. Additionally, prior root cause analyses were scrutinized for key items to add. IS, Risk Management, and legal counsel reviewed the revised form and ensured security and confidentiality of privileged data. Some screen shots from the system appear in Figure 1.
Figure 1.
A screen shot of (a) the Web forms home page, (b) the header of a form, and (c) sample questions in another occurrence form.
To enhance data completeness and accuracy, decision rules were embedded into many questions. For example, an end user could not future date or time an occurrence, and certain questions were mandatory. Each question also links to an online dictionary and provides end users immediate access to operational definitions, further explanations, or examples while completing a Web form. Electronic forms can use branching logic and capture data in a coded form, yielding higher quality than manual forms.
The CQCC and IS team developed related components as well. First, since occasionally someone may want to report an occurrence anonymously, a hotline was implemented, with the telephone number of the hotline appearing on the Web forms. When calling in, the individual has only to describe the event. Second, based on a request from nursing administrators, Web forms were created to collect key data for auditing nursing quality.
The system was set up so that users access the intranet, go to the Web forms by entering their unique user name and password, select the needed form, click on the selected answer options, and submit it. Once submitted, a unique record number is assigned, and all information is immediately written to the server's database and is ready for review, retrieval, and analysis. Web forms can be accessed from any personal computer connected to the intranet and are available at all times. Access to the database for data retrieval and analysis, however, is highly restricted and limited to individuals based on their assigned security rights. Any manager or administrator can have these rights for his or her defined areas of responsibility. Others, such as departmental quality representatives, biomedical engineers, and risk managers, also have this access based on a legitimate need to know.
Rollout and expanded capabilities of the system
The system went live on July 5, 2000, for 4 facilities: Baylor University Medical Center, Baylor-Tom Landry Center, Baylor Specialty Hospital, and Our Children's House. During the initial months of use, a systemwide workgroup reviewed the application and reached consensus about a risk stratification model, data set standardization, and rollout for the community medical centers.
In January 2001, phase II was implemented, giving managers and administrators unique security rights to a new online reports link. Site administrators set these rights based on managers' areas of responsibility. At any time, select managers can look at composite results, isolate an individual report, and review new, pending, and closed records. They can also append follow-up comments without compromising the integrity of the original data; appended data can be tracked through automatic stamping of the manager's name and append date and time. Realtime access to information is a distinct advantage to managers, administrators, and others because follow-up can be initiated more quickly.
Phase III was implemented in July 2001. This phase consisted of implementing the automated risk stratification model that uses a decision algorithm to assign the risk level, e-mail notification of managers and administrators for events classified as high or critical risk, and additional online reporting options for managers.
Education and cultural changes
Over 80 in-service training sessions for hospital staff were presented on various days and shifts using demonstrations, and IS established a test site where individuals could practice using the forms. Concise instruction sheets were provided, including various screen printouts. These educational sessions also afforded the opportunity to start to change the culture from reporting errors only to reporting the 3 U's: anything unusual, unexpected, or unanticipated. There has been a continuing educational effort to emphasize reporting as a means to quality, safety, and process improvement, not just to document errors or assign blame. No staff incentives, monetary or nonmonetary (e.g., movie tickets), were used to improve reporting rates.
Measurement
To determine whether the original goals of timeliness, efficiency, and effectiveness had been met, statistics on patient occurrence reporting were compared before and after Web form implementation. In addition, feedback was obtained during a campus-based quality fair in which the Web forms project was displayed.
RESULTS
Prior to implementation of the Web forms, CQCC received an average of 128 paper patient occurrence reports per month 7.6 days after the occurrence; in the 12 months afterwards, it received an average of 175 patient occurrence submissions, with 82% reported within 24 hours of the occurrence (Figure 2). Between 13 months and 17 months afterwards, the average monthly number of reports decreased slightly, but the gain in timely reporting was sustained (Figure 2). Plotting these data on a statistical process control chart showed no special cause variation related to the decrease, and the number of Web forms received in January 2002 exceeded 200.
Figure 2.
Number and timeliness of nonmedication patient occurrence reports before and after implementation of Web-based reporting at Baylor University Medical Center.
Data from the reporting quarters immediately before and 9 to 12 months after implementation show similar numbers: an 83% increase in submissions and a 79% decrease in submission time (Table. 2).
Table 2.
Patient occurrence reporting statistics at Baylor University Medical Center before and after implementation of Web-based forms
| Method | Time period | Average number of submissions per month | Average days from event to submission | Percentage submitted within 24 hours |
| Paper forms | April–June 2000 | 121 | 7.8 | 17.6% |
| Web forms | April–June 2001 | 222 | 1.6 | 83.3% |
During a recent quality fair, anecdotal comments from end users were positive, citing the ease and speed of submitting a Web form compared with manually completing a paper form. Users liked clicking on answer options. One member noted that she no longer had to search the unit for a form; all she needed was the nearest computer. It typically takes <3 minutes to complete a Web form, but the time may vary based on the amount of free text the end user chooses to enter.
Improvements in report timeliness have allowed for quicker interventions, notifications of key personnel, and assistance in meeting external agency timelines. Further, data are immediately available for analysis, and issues of completeness and legibility have been eliminated.
DISCUSSION
The Web-based occurrence forms have met the objectives initially identified by the project team. Their development has led to computerization of other forms as well: currently there are 10 patient occurrence forms and 11 nursing quality audit forms in operation. In addition, since implementation of the Web-based occurrence forms, the 4 medication variance forms have been adapted to this same application and have the same look; all are available from a single intranet screen. Some forms, like the patient occurrence Web forms, meet organizational needs, whereas others, like the nursing quality audit forms, meet individual departmental initiatives. CQCC has also used the application to build Web forms unique to specific unit needs, such as the postanesthesia care unit delay of transfer form and the bone marrow infusion audit form.
The broad spectrum of computer proficiency among staff posed some challenges. A recommendation is to include some type of basic computer skills assessment or education during employee orientation. Some individuals who were computer proficient would change the uniform intranet settings (i.e., home page default) on a computer in their department, causing confusion for those who were still computer novices. Close monitoring and continued reinforcement of system policies are required.
Increased functionality continues to be defined for this project. Items are prioritized and incorporated into the project plan, and a scope of work is developed that includes deliverable dates. A Web-based suggestion form has also been implemented to capture user suggestions. Current research is under way for using this application to capture visitor and employee occurrences. However, since processes for reporting these occurrences already exist, it is important to carefully evaluate needs in order to avoid duplication of effort or data capture.
Reporting via the intranet remains robust. Use of the hotline, however, has been extremely rare. One potential difficulty with anonymous reporting is that it may be difficult to research the event due to insufficient information. This type of reporting is probably best for pure process improvement versus risk management, where further investigation and follow-up are required. A critical lesson is that regardless of the level of sophistication used in reporting, technology alone does not guarantee reporting.
To sustain and improve reporting, all levels of the organization must continually reinforce the message that reporting is not about blame or punishment. Rather, it is key to promoting a culture of quality and safety. All employees must see reporting as a routine responsibility of their jobs and feel secure in reporting. Such a mind-set requires not only ongoing formal education but also demonstration of this commitment in both words and actions by all members of the organization, especially the leadership teams. Gathering data provides knowledge about a subject, but it alone does not drive the improvement process. Improvement is still the responsibility of individuals.
The Web forms project has made a major contribution to the quality and safety initiatives of Baylor Health Care System's mission. After implementation, additional community medical centers converted to the application, including Baylor Institute for Rehabilitation, Baylor Medical Center at Irving, Baylor Irving-Coppell, and Baylor Medical Center at Ellis County. The rollout of the system to community medical centers allows for facility and system-level composite reporting, trending, and analysis.
Acknowledgment
The authors would like to acknowledge Traci Graves for her preparation of the paper patient occurrence forms summary data and the Information Services Department for its contributions in making this project successful.