What Can Apologies in the Electronic Health Record Tell Us About Health Care Quality, Processes, and Safety?

Supplemental digital content is available in the text.

Abstract

Introduction

Apologizing to patients is an encouraged practice, yet little is known about how and why providers apologize and what insights apologies could provide in improving quality and safety.

Objective

The aim of the study was to determine whether provider apologies in the electronic health record could identify patient safety concerns and opportunities for improvement.

Methods

After performing a free-text search, we randomly selected 100 clinical notes from 1685 available containing terminology related to apology. We categorized the reason for apology, presence and classification of medical error, level of patient harm, and practice improvement opportunities. We compared patient events discovered from apologies in the medical record to standard patient incident report logs.

Results

Of 100 randomly selected apologies, 37 were related to a delay in care, 14 to misunderstanding, 11 to access to care, and 8 to information technology. For apologies related to delay, the median delay was 6 days (mean = 8.9, range = 0–41). Twenty-four (65%) of the 37 delays were related to diagnostic testing.

Medical errors were associated with 46 (46%) of the 100 apologies. Sixty-four (64%) of the 100 apologies were associated with actionable opportunities for improvement. These opportunities were classified into 37 discrete issues across 8 broad categories. When apology review was compared with standard incident report logs, 27 (73%) of the 37 discrete issues identified by patient apology review were not found in incident reporting; both methods identified similar rates of patient harm.

Conclusions

Review of apologies in the electronic health record can identify patient safety concerns and improvement opportunities not apparent through standard incident reporting.

It is now an expected practice that providers apologize when a patient experiences a medical error.^1–4 In the past, apologies were often regarded as an admission of guilt and a potential risk for medical malpractice.^5,6 In contrast, apologies are now an important consideration in everyday medical practice as well as in training future healthcare providers.^7,8 Some consider an apology a moral imperative and a necessary catharsis that preserves providers' professionalism.^6,9 Apologies are also an important tool in risk mitigation, and many states now have laws written to protect providers who apologize to patients.^3,4,10–12 The circumstances, methods, and art of apology in medical practice continue to evolve and mature.^13,14 Best practices for responding to and communicating with patients after an adverse event have been described in the quality and safety literature.¹⁵

Despite an increasing emphasis on apology, little is known about the frequency, reason for apology, or how providers choose to apologize (online messaging, through surrogate, in person or over the telephone). A systematic exam of the typical patterns of apology in everyday practice could serve to highlight opportunities to improve the quality and safety of care delivered.

New tools within electronic health records (EHRs) now allow users to perform text searches of clinical documentation using strings of key words.¹⁶ For example, text searches have been used to examine the accuracy of patient-reported symptoms and the quality of blood pressure measurement reporting through patient secure online messaging.^17,18 Using free-text searches, users can now screen thousands of secure messages and clinical notes for text strings including terms such as “apologize.” Using this technology to examine apologies can provide a different perspective in examining process and safety concerns. While mining of free text in the medical record has been used in several different clinical and research capacities,^19,20 to our knowledge, it has not been used to examine characteristics of patient apologies.

In this study, we performed text searches of secure messages and clinical notes to determine patterns of provider apology and whether analysis of apologies could have implications for practice improvement.

METHODS

Setting

The study took place within Mayo Clinic's primary care practice in Rochester, Minnesota. The adult primary care practice provides care for approximately 120,000 community members and employs approximately 51 staff internal medicine physicians, 25 family medicine physicians, 20 nurse practitioners or physician’s assistants, 100 internal medicine residents, and 21 family medicine residents. These clinics are contained within a large, academic medical center and an integrated, regional healthcare system.

Secure online messaging through our EHR is an important part of our workflows; approximately 60% of our primary care patients have signed up for patient online services accounts, and 77% of those patients regularly access their account. Through secure online messaging, patients are able to view their medical record, including laboratory test results, imaging reports, and clinical notes. In addition, patients can initiate secure messages to their care teams. Communication by secure message between patients and their providers is captured and searchable in the EHR.

Identification of Apologies

To identify apologies in the EHR, we used an institutionally created search tool called Advanced Cohort Explorer (ACE). Advanced Cohort Explorer can perform multistep free-text searches of clinical documents, including all clinical notes and patient secure online messages. For this study, we used ACE to search for user-defined text strings in the medical record that indicated an apology had taken place.

Data Sources

Our data source for apologies was clinical notes in the EHR. There are multiple clinical note types, which document face-to-face encounters, test results communication, miscellaneous notes, and secure online messages transmitted between patient and provider. Our searches included the entire text fields of those clinical notes. Baseline patient characteristics were collected from the ACE database.

We used employee entered incident report logs filed within the same time interval as the free-text search. Our incident reporting system is an online system where employees can anonymously provide pertinent details of safety events they encounter. The purpose of comparing free-text review of apologies to incident reports from the same period was to assess the qualitative and quantitative differences in the types of safety events identified through each method.

Identification of Search Terms for Apology

An iterative process was used to formulate search strategies. We examined clinical notes entered between January 1 and November 1, 2016, examining searches containing the root terms apologize, sorry, and mistake. The search was limited to patients older than 18 years. Figure 1 contains additional details.

FIGURE 1.

Search and clinical note selection flow.

Review of documentation containing the term “sorry” revealed many (8209) entries, but review of a 10-chart sample did not suggest a consistent connection to quality or safety. Many notes were sympathetic, for example, “I'm sorry to hear you are not feeling well.” The search for the term “mistake” revealed 325 results (16 exclusions), but we decided to instead limit analysis to text searches related to “apologies” given the higher number of results identified. Our apologies search identified 1685 unique clinical notes available for review. Preliminary review of 10 clinical notes with terminology containing the term “apology” suggested relevance to quality and safety.

Data Collection After Identification of Apology Sample

For the 100 note sample collected, we retrieved patient age, provider type (nurse practitioner or physician assistant, resident, nurse or staff physician), communication method for apology delivery (secure online message versus phone versus office visit), demographic information, and note type. The only data collected that required chart review beyond viewing the retrieved note were in calculating length of delays.

Determination of Apology Characteristics, Medical Error, and Patient Harm

Two reviewers (J.M. and F.N.) analyzed the clinical notes containing the search terms. The apologies were categorized as shown in Table 1; these categories were mutually determined by the authors while reviewing the random sample of apologies. The reviewers assessed medical errors and evaluated the level of potential harm using the definitions in Appendix 1, http://links.lww.com/JPS/A167. These independent responses were entered into a REDCap database.²¹

TABLE 1.

Baseline Demographic Data of Reviewed Apologies (N = 100)

Determination of Delay in Care

Delays were determined by a single author (J.C.M.) performing a manual chart review; time of delay was determined by counting the days from index point of contact with the patient to the time of apology documentation. Both authors subsequently categorized the delays and determined whether they were time sensitive (Table 2). Time sensitive was defined based on the reviewer's clinical judgment as to whether the delay could have resulted in an adverse event. For example, delayed reporting of a normal colonoscopy was not considered time sensitive, but a delay in reporting a positive urine culture was.

TABLE 2.

Apology Categorization (100 Charts Reviewed, 142 Entries)*

Determination of Medical Error

Medical error and associated harm have been traditionally difficult to study.²² After reviewing several definitions of medical error,^23–25 we agreed to use a working definition of medical error as “an act of omission or commission in planning or execution that contributes or could contribute to an unintended result”; this was adapted from several sources including the Institute of Medicine report “Crossing the Quality Chasm.”^5,22,26 Both authors applied this definition when reviewing an apology and independently determined whether the clinical note identified through the apology search suggested a medical error had taken place.

An apology could contain multiple errors that were not mutually exclusive or repetitive. For example, a delayed communication of a test result coupled with a wrong test ordered would be counted as two episodes of medical error within the same apology. We also graded the level of harm for all 100 records, based on the framework of Panesar et al.²⁷ Appendix 1, http://links.lww.com/JPS/A167 describes in greater detail how those definitions were applied in determination of error and harm.

Improvement Opportunities

We reviewed whether a particular episode of apology suggested actionable improvement opportunities within our practice. This determination was based on each reviewer's assessment of the clinical situation, perception of local practice conditions, processes, and factors, which led to that apology. We categorized these findings in Table 3.

TABLE 3.

Classification of Medical Error (n = 46)*

Review Reconciliation

After completion of an initial, independent evaluation both authors reviewed each apology together and consensus was reached in categorizing the nature of apology, presence of medical error, harm determination, and improvement insights gleaned. Agreement was reached in all instances.

Comparison With Patient Incident Report Logs

Employee entered incident reporting is our standard method of outpatient safety surveillance. When an employee observes a safety concern, they may anonymously report the details of that event to the office of patient safety through a secure electronic platform. These incidents are then reviewed, categorized, and catalogued by the patient safety office and returned to the clinical practice as incident report logs. To discover whether examining apologies could provide new insights relative to our current incident reporting system, we additionally reviewed 10 months (134 reports) of incident report logs covering the exact same period and from the same practice area and patient population as the apologies review. We manually compared descriptions of events around the reviewed apologies with descriptions of events in our incident report logs. The purpose of comparing the free-text apologies search with incident reports from the same period was to assess the qualitative and quantitative differences in the safety and quality issues identified between the two search strategies (Table 4).

TABLE 4.

Practice Improvement Opportunities (n = 64)*

Data Collection and Statistical Analysis

We used JMP 12.0 (SAS Institute, Cary, NC) for calculating Cohen κ and Fisher exact test for determining interrater agreement and categorical differences, respectively.

Study Conduct

This study was approved by the Mayo Clinic Institutional Review Board.

RESULTS

Of 272,730 eligible clinical notes from the study period, 1685 (0.6%) contained our preselected search terms (Fig. 1) indicating that an apology had been documented. Of the 100 randomly sampled apologies, 19 (19%) were apologies delivered by the patient, whereas 81 were delivered by a member of the primary care team. Seventy-six percent of apologies were delivered via patient online services, 19% through documentation of a phone conversation, and 5% through documentation of an in-person office visit.

Apology Characteristics

Clinical and demographic characteristics of patients identified from our search are displayed in Table 5.

TABLE 5.

Comparison With Standardized Incident Report Logs

Categorization of Apologies

Table 1 shows the apology categories and counts. When an apology was related to a delay in care, the median delay was 6 days. Delays were classified as related to testing, appointment availability, treatment, or others; several apologies were counted in multiple categories. Of the 37 total delays identified, 13 (35%) were determined to be time sensitive.

Medical Error and Patient Harm Categorization

Using literature-guided definitions (Appendix 1, http://links.lww.com/JPS/A167), 46 (46%) notes containing an apology identified a medical error. Of these 46 medical errors, 36 (78%) represented an error of execution and 10 (22%) represented an error of planning. Further subcategorization of error is shown in Table 2. In determining risk of harm, 35 reviewed apologies were classified as having no potential for harm, 50 were no harm events, 2 as low harm events, and 2 as moderate harm events. Both moderate harm events were potentially avoidable emergency department visits. Eleven apology events were classified as unclear or requiring additional information apart from what could be gleaned from the clinical note returned from the search.

In 64 (64 %) of apologies reviewed, we identified improvement opportunities within our practice (Table 3). Interrelater agreement across categories of apology classification varied with Cohen κ values ranging from 0.12 for assessment of whether a delay was time sensitive to 0.74 for categorizing an apology as related to delay in care. Consensus was attained following joint review.

Appendix 2, http://links.lww.com/JPS/A167 shows the variety of improvement opportunities identified from the review of 100 apologies. We collapsed 37 specific process improvement opportunities identified from the apology review into eight broad categories. New insights and opportunities, not present in review of our standard incident report logs for a similar period, are denoted with an asterisk.

Comparison With Standard Incident Reporting Process

Seventy-three percent of the event types identified by apology review were not present in the incident report logs (Appendix 2, http://links.lww.com/JPS/A167). A more general comparison is shown in Table 4. Incident report logs predominantly identified events, which occurred while the patient was physically present in the clinic. Examples included mislabeled specimen containers, falls in the clinic, or improperly administered medications. In contrast, review of apologies predominantly identified issues arising outside of any face-to-face encounter. Examples included delayed responses to patient inquiries, inadequate integration of outside medical records, and dissatisfaction with care provided in specialty clinics. Notably, the free-text apologies search identified numerous (21) events where patients brought care process concerns to light, whereas this was uncommon in incident reports. Similar rates of events leading to harm using our aforementioned definitions were found in both modes of safety surveillance.

DISCUSSION

Using free-text EHR searches to explore instances of apology can be used to identify medical error, quality and safety concerns, unsafe processes, and practice improvement opportunities. Seventy three percent of improvement opportunities found by apology search were not found in standard incident reporting. Most apologies found in the EHR were associated with secure online messages.

Our study shows that examining apologies can broaden the scope of opportunity for practice improvement when compared with review of incident reports. The information obtained from apology review often revealed practice concerns in quality and safety domains not otherwise identified by incident reports. Although incident reporting systems remain an important part of outpatient safety surveillance, there are well-known limitations to wide spread usage particularly surrounding local reporting culture.²⁸ Free-text searching does not seem to be subject to many of these limitations. Because many events identified by the apologies search were not found in our incident reporting system, healthcare institutions could use apologies or other free-text searches to examine the breadth and quality of information obtained from their incident reporting systems.

The apologies search captured many of the challenges encountered within the nonvisit care processes commonly employed in a modern, primary care practice. In our practice, incident log reports commonly capture specific events occurring whereas the patient was present in the clinic for a face-to-face encounter. As outpatient practices increasingly use secure online messages and other nonvisit approaches to provide care,²⁹ it will be important to examine these messages for events that can escape the attention of a healthcare system still more attuned to the care provided within the traditional face-to-face visit.

Because many apologies were delivered through patient online services, one may presume that this mode of communication provides a forum more comfortable to both patient and provider compared with a face-to-face visit or phone discussion. Perhaps use of secure online messaging allows circumvention of logistical challenges, which previously may have limited the use of apology for more routine issues and concerns. We also recognize that only two events were graded as causing moderate or severe harm. This may be due to inherent limitations of using secure online messaging for delivering apologies to patients. It is plausible that clinicians are not comfortable documenting apologies in this manner. The appropriateness of using this forum for apology delivery and the effect of secure online messaging on patient experience and perception of apology is also worthy of additional study.

It is notable that nearly 25% of the apologies reviewed were in response to events or errors discovered by patients, most commonly communicated to their care team through secure online message. This patient engagement in quality and safety via secure message was an important finding in our study. Cataloging issues most relevant to patients through this type of search could create a more robust system of patient engagement in quality and safety. Because we strive toward systems, which engage patients as equal partners in their care, modalities, which enable patients to directly voice safety concerns, are needed. Searching for apologies within secure online messages may be one way to identify patient generated concerns. Future design of secure online messaging systems can facilitate patient-directed communication of quality and safety concerns. For example, when logging on to patient online services, there should be an option for a patient to submit a quality or safety concern either perceived or experienced.

Reviewing apologies seems to be sustainable. Although we did not determine the length of time spent on reviewing each apology, it is likely that most apologies could be effectively reviewed in only a few minutes. Whereas reviews in this study were performed by physicians, many components could be completed by other team members. In addition, the time, training, and cost required may be minimal once the technology infrastructure is in place. We acknowledge low rates of reviewer agreement on assessment of whether delays in care were categorized as time sensitive. Agreement rates could be improved by developing more specific definitions of what constitutes a time-sensitive delay.

Limitations

Our study has limitations. We limited the apology search and incident report review to a single primary care practice. As such, our practice mode, patterns of apology, and incident reporting may affect generalizability. The categorization, error and harm determination, and improvement opportunity analysis were performed by two physician reviewers. Limiting this review to two physicians may exclude other healthcare professional's perspectives on safety and improvement opportunities. Our current process relies on physician time. However, with future development, it is desirable that searches become automated to readily provide actionable data.

Not all errors or harm involve an apology, and other language should be considered. Other permutations in free-text searches that reflect geographical, cultural, and linguistic traditions could lead to different insights. As such, reviewing apologies in a single-practice setting may lead to a sample overrepresenting certain events or certain providers based on relative differences in apology threshold and style of delivery.

There are some technical limitations. This type of search platform may not be widely available. Although some institutions may not have a free text search platform available, the prevalence of these tools seems to be increasing.³⁰ Our rate of secure online message registration and usage is likely higher than other institutions. This high rate of secure online message usage may create a larger data sample for reviewing apologies, and the breadth of practice insights we found may not be found in institutions with lower rates of secure online messaging usage. We also only compared the incident report logs during the same timeframe as the apologies. If we reviewed incident report logs retrospectively for several years to increase the sample of events, differences in events identified between the incident report logs and apologies may be less drastic.

Concerns regarding effects on the patient-provider relationship will need to be considered. If providers know that their documentation is being searched for specific text strings, it could affect practices surrounding apology. Even organizations with a strong culture of disclosure and advanced systems of patient safety would need to carefully monitor a program of this nature for counter-balancing effects.

CONCLUSIONS

Apologies in the EHR can provide a new source of information about the quality and safety of patient care provided. Further work is needed to determine whether identifying and analyzing apologies should be incorporated into a comprehensive quality and safety review process. This study suggests that apologies could be identifying important and complementary information, which can serve to enhance ongoing surveillance efforts.