There is an App for That: Mobile Technology Improves Complication Reporting and Resident Perception of Their Role in Patient Safety

Background:

Morbidity and mortality (M&M) conferences are rooted within the culture of medicine. They serve a role in every training program and have been mandated by the Accreditation Council for Graduate Medical Education in surgical programs since 1983. Despite the patient safety improvements and educational benefits of these conferences, many adverse events are grossly under-reported.

Methods:

We developed a web-based, Health Insurance Portability and Accountability Act-compliant, M&M reporting mobile application based on Research Electronic Data Capture. The list of possible complications was based on the American Board of Orthopaedic Surgery complications list for part II. The interface is accessible through all mobile platforms. All residents were encouraged to use the application for real-time reporting of complications. Using an unpaired T-test, we compared the reporting before and after the implementation of the mobile application. Residents were surveyed using the Agency for Healthcare Research and Quality Patient Safety Culture Survey before and after implementation to evaluate resident perception of the department's culture of safety

Results:

The application was launched in August 2017. All reported events were tallied from August 2016 through July 2019. Before the implementation of the application, there were 54 adverse events reported, with a mean of 4.0 per month. In the Post-App cohort, a total of 176 adverse events were reported in year 1, with a mean of 14.76 per month, and 236 adverse events were reported in year 2, with a mean of 19.66 per month. Residents were significantly more likely to feel that their input on patient safety was valued by attendings after the implementation of the app (p = 0.0243).

Conclusions:

An anonymous mobile reporting method for M&M significantly increased the reporting of both major and minor complications and improved resident perception of their role in patient safety efforts. This suggests that traditional methods of M&M reporting may grossly underestimate the complication rates which can negatively affect patient safety and quality improvement efforts and that reducing barriers to the reporting of complication may improve resident engagement in patient safety.

Morbidity and mortality conferences have served as a cornerstone of surgical programs for decades and to this day have evolved into a vital component of resident and professional education. To enhance surgical care and patient safety, morbidity and mortality (M&M) cases and adverse events are openly presented and discussed in hopes of improving learning opportunities for those in attendance. In 1983, the Accreditation Council for Graduate Medical Education mandated that all surgical programs participate in scheduled case presentations of morbidity and mortality at their respective institutions¹.

Despite the educational value of these conferences, it is possible that the reporting of complications may result in negative repercussions for the reporter or the development of a persecutorial environment, particularly in controversial cases. This can result in under-reporting of complications by both resident and attending physicians². The Institute of Medicine described this problem in reference to patient safety, “the biggest challenge to moving toward a safer health system is changing the culture from one of blaming individuals for errors to one in which errors are treated not as personal failures, but as opportunities to improve the system and prevent harm”³.

The desire to avoid blame and scrutiny leads to an under-reporting of medical errors⁴. The objective of this project was to prospectively evaluate a user-friendly mobile application that allowed for real-time simple and anonymous reporting of adverse events. We hypothesized that the use of this novel method of reporting would increase the number of events reported and resident perception on the culture of safety before and after the dissemination of the mobile application

Methods

A web-based, Health Insurance Portability and Accountability Act-compliant, and anonymous M&M reporting mobile application based on Research Electronic Data Capture (REDCap) was developed and disseminated to all residents and attendings in the orthopaedic department at a Level I, urban, academic medical center^5,6. The institutional review board (IRB) was consulted, and IRB approval was not required because the application and project are both exempt under the designation of quality improvement. The application is a survey that includes a patient identifier (medical record number), the date of the event, the service which the event occurred (trauma, arthroplasty, sports, etc), a list of complications, and finally, whether the reporter felt the event was significant enough to warrant a formal discussion in the M&M conference (Fig. 1). The survey is accessed as a mobile web application with no login or user identifier required to maintain reporter anonymity. No reported event can be attributed to the individual who completed the survey. With the completion of the survey, the selections are transmitted in an encrypted fashion and stored in the REDCap database. The list of possible complications was derived from the American Board of Orthopaedic Surgery complications list for part II (Table I). The complications were then categorized as either major or minor to evaluate the changes more accurately in reporting. The interface is accessible through all mobile platforms and traditional web browsers to facilitate ease of use in multiple environments. All residents and attendings were encouraged to use the application for real-time reporting of complications and also regularly reminded during meetings. On a monthly basis, using REDCap's “Advanced Data Export” function, the data collected were exported in a deidentified Excel format which included all reported complications. Each reported event was assigned a unique identifier, “Record ID,” to avoid duplicate entries and for the purpose of retrospective validation. At the completion of the study period, all reported events and categorization of major and minor complications were individually reviewed for accuracy. The total number of cases performed within the orthopaedic department each year during the study period was collected to ensure reliability of any differences noted in complication reporting.

Fig. 1.

Application in use—Mockups of the existing application screen demonstrating the straightforward user experience. M&M, morbidity and mortality.

TABLE I.

The American Board of Surgery Complication List: Major and Minor Complications^*

General Complications		Surgical/Technical Complications		Medical Complications
Major	Minor	Major	Minor	Major	Minor
Unplanned reoperation	Delay in diagnosis	Compartment syndrome	Infection	Pulmonary embolism	Pneumonia
Death	Delay in treatment	Limb ischemia	Skin ulcer	Myocardial infarction	Congestive heart failure
	Regional anesthesia complication	Nerve palsy	Tendon injury	Cerebral vascular accident	Renal failure
	Unplanned readmission	Dislocation	Ligament injury	Deep vein thrombosis	Arrhythmia
	Emergency room visit	Vascular injury	Wound healing delay	Medication error	Urinary tract infection
		Bone fracture	Wound healing failure	Respiratory failure	Confusion
		Wrong site	Hematoma		Delirium
		Spinal cord injury	Seroma		Gastrointestinal bleeding
		Implant failure	Graft related problem		Hypoxia
		Loss of reduction	Pain		Dermatologic
		Failure of tendon	Fall		Pressure ulcer

^*Complications were identified through the American Board of Orthopaedic Surgery complication list and further sorted into major and minor complications.

Before the use of the application, complications were solicited in a retrospective manner via email by the chief resident on the trauma service on a monthly basis in preparation for the M&M conference. These submissions were then collected by the residency coordinator and stored. The application was launched in August 2017. All reported events tallied between August 2016 and July 2017 were compiled and defined as the Pre-App cohort. Events reported between August 2017 and July 2019 were compiled and defined as the Post-App cohort.

To assess the impact of the survey on department culture, the Hospital Survey on Patient Safety Culture by the Agency for Healthcare Research and Quality was modified to be resident-specific and administered to all residents 1 week before the widespread implementation of the app in July 2017⁷. In late June 2018, after 11 months of implementation, the same anonymous survey was readministered to capture the outgoing residency cohort. An unpaired T-test was used to compare safety survey results and the volume of Pre-App and Post-App complication reporting. A p value <0.05 was considered significant.

Results

In the Pre-App cohort, there were 54 adverse events reported of a total of 3,443 surgeries performed (1.6%), with a mean of 4.0 events per month. In the Post-App cohort, a total of 176 adverse events were reported in year 1 of a total of 3,409 surgeries performed (5.2%), with a mean of 14.76 per month and 236 adverse events were reported in year 2 of a total of 3,825 surgeries performed (6.2%), with a mean of 19.66 per month (Fig. 2). In an unpaired T-test comparison, there was a statistically significant difference in events reported between the Pre-App cohort and the Post-App cohort with a p < 0.0001. Before the implementation of the app, 31 of 54 (57%) were considered major events. After the implementation, 248 of 412 (year 1: 104 of 176, year 2: 144 of 236) (60%) were considered major events. A χ² test did not show a statistically significant difference between the category of event reported before and after the application (p = 0.946).

Fig. 2.

This figure illustrates the number of reported complications at our trauma center over the course of 3 years. The major complications are marked in blue, and the minor complications are marked in gray.

The resident survey included 38 total questions, 19 of which pertained to error reporting and the culture of promoting patient safety. The survey was completed by 17 of 20 residents before the application and 20 of 20 after the application. Seventeen of 19 (89.47%) were generally more positive after the introduction of the application but only 1 statement, “My attendings consider residents' suggestions for improving patient safety,” reached statistical significance (p = 0.0243).

Discussion

The purpose of this project was to evaluate the effect of implementing an alternative, anonymous, and easier method of reporting complications. This study demonstrates a significant increase in the number of both major and minor events reported through implementation of a mobile application, as compared to traditional methods of event reporting. This is important both for providing ample cases to be discussed in the M&M conference and identifying numerous patient-safety opportunities. The app demonstrated the capability of capturing both large and smaller events, without creating a predilection for one or the other, as demonstrated by the consistent proportion of major events reported before and after implementation of the application.

This project demonstrated that traditional methods of soliciting complications may result in gross under-reporting of adverse events, which is not a unique finding⁸. The Institute of Medicine in To Err Is Human recommended expansion in the reporting of adverse events and medical errors; investigators have studied these events at the department level because it relates to morbidity and mortality conferences⁹. McVeigh et al. instituted a prospective means of collecting adverse events in their department of surgery by including a paper-based pro forma in all medical charts to be filled out by house staff. They showed a 73% increase in morbidities reported and 10.81% increase in mortalities reported¹⁰. The application in this study was used in a similar manner but has the added benefit of being digital and therefore more easily accessible.

Despite the many strengths of this study, it does however have some limitations that must be addressed. One potential limitation of this study may be the psychological phenomenon of the Hawthorne effect. Because the application was disseminated, participants understood there was more of a focus on reporting adverse events and this could have affected their reporting habits independent of the application. It is possible that if the same amount of emphasis and regular reminders had been placed in the 12 months earlier (i.e., the pre-M&M app period), there would have been a surge in event reporting during that time as well. We do, however, believe this is less likely because the relative ease of the app is much less time consuming than traditional methods and provides important anonymity. In addition, although it is possible that the number of complications actually increased in the Post-App period, this is unlikely because there were no major changes in the clinical protocols, faculty, staffing, residency, or overall case volume.

The preapplication and postapplication resident survey demonstrated an improvement in resident perception of the culture of safety, which in turn could make reporting more effective. Many previous studies have investigated the concept of speaking up for patient safety¹¹. Hierarchical structure in surgical training programs has been a barrier to open communication, even when patient safety is concerned. Belyansky et al. surveyed both attendings and residents in surgical training programs and showed that all attendings believed they encouraged residents to question their intraoperative decision-making but only 55% of residents agreed¹². In our study, resident-attending communication regarding patient safety improved and resident perception of how their input was valued reached statistical significance. This may be a result of the encouragement by attendings to residents to add events to the M&M application and a reduction in the stigma of openly discussing adverse events. Other questions (89.5%) in the survey showed improvement but did not reach statistical significance. The strong pre-existing culture of safety in the department may have made it difficult to have statistically significant improvements.

In conclusion, the culture of M&M conferences may ironically create an environment that results in under-reporting of complications and subsequently mitigates educational and patient-safety opportunities. This study demonstrates a low-cost, efficient, and anonymous method of reporting adverse events which improved both the reporting of complications and resident perception of attending engagement. This has the potential to not only maximize physician education but also provides opportunities to improve patient safety.