Calling on the Patient’s Perspective in Emergency Medicine: Analysis of 1 Year of a Patient Callback Program

Abstract

Background:

Patient-centered approaches in the evaluation of patient experience are increasingly important priorities for quality improvement in health-care delivery. Our objective was to investigate common themes in patient-reported data to better understand areas for improvement in the emergency department (ED) experience.

Methods:

A large urban, tertiary-care ED conducted phone interviews with 2607 patients who visited the ED during 2015. Patients were asked to identify one area that would have significantly improved their visit. Transcripts were analyzed using content analysis, and the results were summarized with descriptive statistics.

Results:

The most commonly cited themes for improvement in the patient experience were wait time (49.4%) and communication (14.6%). Related, but more nuanced, themes emerged around the perception of ED crowding and compassionate care as additional important contributors to the patient experience. Other frequently cited factors contributing to a negative experience were the discharge process and inability to complete follow-up plan (8.0%), environmental factors (7.9%), perceived competency of providers in the evaluation or treatment (7.4%), and pain management (7.4%).

Conclusions:

Wait times and perceptions of ED crowding, as well as provider communication and compassionate care, are significant factors identified by patients that affect their ED experience.

Introduction

Elements of consumerism and service quality have long been incorporated in the discussion of health-care delivery (1 –4). However, it has been the widespread adoption of tools such as the Hospital Consumer Assessment of Healthcare Providers and Systems Survey that has led to the prioritization of the patients’ perception of their care as a publicly reported benchmark for hospital quality and performance (5).

The patient experience is not just about providing service and hospitality. Increasing literature supports the notion that better patient experience has a positive impact on adherence to treatment and evidence-based guidelines (6 –8), clinical outcomes (7), readmission (9), profitability (10), medicolegal risk (11 –14), emergency department (ED) utilization (15), and willingness to return (16).

Although ED-specific psychometric surveys are in early phases of development (17), the absence of a validated ED-specific tool (16,18,19) has led to heterogeneous literature specific to the ED patient experience. Previous studies have identified the provision of information to patients (20 –23) and perceived wait times (24 –26) as recurrent themes associated with patient satisfaction. However, all of these studies were conducted over a decade ago.

Furthermore, these ED studies have largely focused on quantitative data in the form of Likert scale responses to patient surveys. Although 2 studies in the pediatric ED literature incorporated qualitative analysis of 1 or 2 open-ended questions (27,28), one was limited by a small sample size, and both were restricted to pediatric populations. Although multiple prior studies have employed content analysis to look at patient complaints outside of the ED (13,29 –36), no study to date has systematically evaluated qualitative data around patient-reported experience in the adult ED. Qualitative analysis of patient-reported outcome measures allows for the assessment of the broad range of patient experiences that Likert scales cannot capture and creates a more patient-centered focus for this field of research (37).

To that end, we looked to perform a large, systematic analysis of free-text responses to ED patient-reported experience. Our objective was to code and analyze 1 year of patient callback data as a hypothesis-generating analysis to better understand areas for improvement in the ED experience from the patient perspective.

Methods

Study Setting and Subjects

This analysis was performed at an urban, university-affiliated, tertiary-care ED as part of a larger patient safety and quality improvement initiative at the hospital. The time period assessed was January 1, 2015, to December 31, 2015. As a quality improvement effort, it was exempt from human subject review by the institutional review board.

Data Collection and Analysis

As a part of routine care, an ED registered nurse selects a random sample of eligible patients to participate in a telephone survey within 24 to 48 hours following ED discharge. This nurse-driven callback program has been previously described (38). The adult ED is divided into 4 separate treatment areas: a fast-track low acuity area, a seated treatment area with chairs, a stretcher area with monitored beds, and a high-acuity/trauma area. All patients discharged from the lowest 2 treatment areas were included in the callback program and served as the denominator for the study population. This population was chosen in particular, as most patients treated in these 2 areas are discharged, and these areas are where the ED has the most control over and can thus address the concerns brought up by the calls. Furthermore, these are the areas with the most patient experience issues, and thus a natural focus for the ED leadership. Program exclusions included: patients with limited English proficiency, patients on an acute psychiatric detainment, patients with abusive behavior as noted in the medical record, and patients who were discharged to a skilled nursing or rehabilitation facility.

Historically, approximately 65% of patients discharged from the 2 lower acuity areas are eligible for the callback program. Calls within 24 to 48 hours of discharge are then attempted for approximately 63% to 70% of this subgroup, depending on staffing for the calls and how much outreach is needed to meet the needs of patients postdischarge. Of these attempted calls, typically about 62% are completed.

In addition to the 6 questions with binary or open-ended responses, the nurse asks patients, “We’re always looking to improve our services. Do you have any suggestions for things we could do better?” Patients’ answers to this question are transcribed by the nurse into a database.

As part of a departmental quality improvement effort, a set of codes related to patient experience and patient safety had been previously developed using content analysis of patient-initiated complaints and staff-initiated safety reports. In keeping with the method of content analysis, this taxonomy was then used to systematically identify and count episodes described in the free text of this telephone survey with the goal of producing a quantitative description of the content (39). To begin, 4 coders, all emergency medicine trained physicians, independently coded 10% (260) of the sample callback data using the initial taxonomy. New codes were developed for content not captured in the initial taxonomy, and coding disagreements were resolved through consensus discussion. After the 260 comments had been coded, we revised and condensed the taxonomy, and the final codebook included 35 codes.

The same 4 coders then used content analysis to independently apply the codes to all transcripts using Dedoose (SocioCultural Research Consultants, LLC, Manhattan Beach, CA) (40). Each coder reviewed a separate 25% sample of the available transcripts. Multiple codes could be applied to a single response as appropriate. Descriptive statistics were then generated to summarize the data and quantify the frequency of codes.

Results

During the period of January 1, 2015, to December 31, 2015, there were 109 234 ED visits. Of these visits, 47 474 patients were evaluated and discharged from the 2 lower acuity areas. In our sample of eligible individuals, 2607 patients responded to the open-ended question being studied. The average age of respondents was 50.6 years (standard deviation: 19.3) and 57.8% were female (Table 1).

Table 1.

Sample Population Characteristics.

Patient Characteristic	Study Sample (n = 2608)
Mean age (SD), years	50.6 (19.3)
Sex: female, %	57.8
Insurance, %
Medicare	26.9
Medicaid	7.9
Private	59.7
Other	5.5

A total of 3195 codes were applied. Illustrative patient statements and the associated codes utilized are provided (Table 2). The most common themes of areas for improvement in the ED patient experience were related to wait times (1289/2607; 49.4%) and communication (381/2607; 14.6%; Table 3). Other frequently cited factors were ED bed availability (211/2607; 8.1%), discharge process and inability to complete follow-up plan (209/2607; 8.0%), environmental factors (207/2607; 7.9%), perceived competency of providers or inadequate evaluation or treatment (194/2607; 7.4%), and pain management (184/2607; 7.1%).

Table 2.

Codes From the Qualitative Analysis of Responses to the Open-Ended Question.

Code	Illustrative Quote
Wait time: general wait time	The wait was so long.
Wait time: wait to be seen	They kept telling us that a doctor would be here shortly but then after an hour and no one came I was starting to get upset.
Wait time: wait to be discharged	Then I had to wait for the nurse to discharge me and every step took so long.
Wait time: wait for results/data/imaging	Waiting for CT results was very long.
Communication: provider-to-patient	I would have liked better communication as to what was going on with me. I had to keep on asking the next steps.
Communication: not otherwise specified	The communication was terrible with everyone.
Communication: provider-to-provider (handoff)	I believe the pass-off was poor from the doctor to doctor.
ED bed availability	I sat in a chair for 5 hours and it would have been nice to lay down in a bed.
Discharge: discharge process	I felt it was so in impersonal when I was discharged and I wanted more explanation.
Discharge: inability to complete follow-up clinic or plan	The urology clinic can’t see me [for months], and the discharge instructions said to follow-up with them.
Environment: general environment	The monitors and lights. The noise level was just very loud.
Environment: food	I’m a diabetic and there wasn’t any food available in the main ED.
Environment: cleanliness	The room I was in didn’t seem to be clean and bed not made up.
Perceived competency or inadequate evaluation/treatment	My symptoms managed to get messed up from doctor to doctor. I was tested for some things that did not really make sense for what symptoms I was presenting with.
Pain management	I wished they had gotten my pain more under control.
Compassion/treating with kindness	As soon as they realize you don’t have a serious condition they try to kick you out as soon as they can. I wish they would be more human, more sensitive to patients and not just come in and say OK you are fine you have to leave.
Privacy	Privacy was an issue. I was in a room with lots of people and heard everyone’s problems.
Perceived lack of an MD/role clarity	I really never knew who was taking care of me and I kept moving from one area to the next and that was very confusing.
Staffing ratio	You need more nurses and every nurse was running around like crazy. It took the nurse about 2 hours to get me discharged.
Call bell/responsiveness	I pressed the called button and it took 1 hour to get someone to help me to the bathroom.

Abbreviation: ED, emergency department.

Table 3.

Callback Responses.

Theme	N of Codes Applied (3195)a	% of Total Callback Reports (2607)
Wait time	1289	49.4%
General wait time	767
Wait to be seen	276
Wait to be discharged	37
Wait for results/data/imaging	50
Communication	381	14.6%
Communication: provider–patient	263
Communication: not otherwise specified	91
Communication: provider–provider (handoff)	27
ED bed availability	211	8.1%
Discharge	209	8.0%
Discharge process	82
Inability to complete follow-up clinic or plan	127
Environment	207	7.9%
General environment	118
Food	63
Cleanliness	26
Perceived competency or inadequate evaluation/treatment	194	7.4%
Pain management	184	7.1%
Compassion/treating with kindness	139	5.3%
Privacy	136	5.2%
Perceived lack of an MD/role clarity	32	1.2%
Staffing ratio	32	1.2%
Call bell/responsiveness	27	1.0%
Other categories (each <1.0%)	102	3.9%

Abbreviation: ED, emergency department.

^a Callback reports could have multiple codes applied to one report, and not all callback reports were found to have codes represented.

Conversely, infrequently cited areas for improvement were related to error: medication or radiology-related error (22/2607; 0.8%), technical error (21/2607; 0.8%), near miss (4/2607; 0.2%), and misdiagnosis (3/2607; 0.1%).

Discussion

To gain insight into the ED-specific patient experience, we performed a qualitative analysis of patient-reported areas for improved experience in the ED. Our results demonstrate clear themes focused on patients’ perception of wait times and inadequate communication.

Finding that nearly half of patients (49.4%) cite wait time as an area for improvement comes as little surprise. Past studies have similarly examined the role of ED wait times and length of stay on measures of patient satisfaction (20,23,41). Although actual ED wait times may be difficult to dramatically improve secondary to increasing ED utilization, more extensive ED testing, and other external factors, further emphasis may be placed on managing patient expectations of wait time, as prior studies have demonstrated that patients who are informed of delays and expected wait times are significantly more satisfied with their ED visits (42). Likewise, since prior studies have shown that perceived wait time, rather than actual wait time, more significantly predicts ED patient satisfaction, departments could consider strategies to improve the waiting experience through enhanced privacy, seating, computerized media, and nourishment options, among other factors (19,43).

Much like wait time, on deeper examination, other factors cited by the patients in this study are also, in fact, patient-centered indicators of ED crowding. For example, 8.1% of patients comment on bed availability as a focus for improvement. These responses were often in reference to waiting for results or further care while assigned to the internal treatment area after an initial evaluation. We also found that the majority of responses coded under “General Environment” (118/2607; 4.4%) were descriptions of how crowded the ED felt to the patient while already receiving care. For example, patients would comment that “the ER is crazy and lots of people there” or that the environment was “totally insane with so many patients.” Studies have suggested that increased ED crowding may be associated with the risk of preventable medical errors and negative effects on clinically oriented outcomes (44,45). Our results provide additional evidence that patients themselves also perceive ED crowding as a priority area of focus.

The second most commonly cited theme was communication (14.6%). The open-ended format allowed us to gain insight into the complexity of this issue. Prior studies on ED patient satisfaction that evaluated aspects of communication have mostly focused on the provision of information to patients at triage or upon discharge (20 –23). In this study, the majority of these responses (263/381; 69%) specifically identified communication between the provider and the patient as an area in need of improvement. Most of these concerns referenced problematic communication during the post-triage ED course (n = 207), such as the patient not knowing “what’s going on” or the plan, what was pending, or how long the process would take. This lack of information, often experienced as a desire for more frequent updates, led several patients to remark on a feeling of neglect. Further, many wished for more face time with the physician, specifically. A smaller group (n = 17) found communication of the discharge plan to be an issue, with questions on medication management and proper wound care cited multiple times. Notably, the communication style of the provider was also a significant, albeit less frequent, factor (n = 36), where patients recalled providers who they perceived as rushed (n = 9), rude (n = 8), did not listen (n = 6), or otherwise were not as adept at explaining worrisome information as they would have hoped.

Although communication has been widely recognized as a critical process of improving quality of care (46), there is little literature to support ED-based communication programs. However, models of effective programs in other health-care settings do exist and have shown promising results, and contain elements that could be implemented in an ED setting (47).

This study was undertaken as a part of a broader quality improvement effort aimed at identifying a pressing area of need to improve patient safety and quality improvement. This finding that provider–patient communication was an area of significant concern to patients matched findings from the other arm of this quality improvement effort aimed at understanding the concerns of staff (48). To that end, these findings were used to garner institutional support for the development and dissemination of a communication training curriculum for all clinical staff. To date, over 400 staff have been trained, and the results of this training are being analyzed.

In addition to communication, another related theme important to the patients was compassionate care (139/2607; 5.3%). Although there is little empiric evidence around how to increase compassionate care, clinical communication is known to play a pivotal role (49). The development of effective provider education around patient communication and compassionate care will be an important focus moving forward.

Interestingly, these patient-reported factors do not highlight the role of medical error and misdiagnosis as contributors to a negative ED experience. Both medical error and misdiagnosis are 2 major areas of concern in the field of quality and safety (50,51). The literature suggests that when people are asked specifically about medical error, they can identify that an error was made (52). However, our study suggests that in the absence of this directed question, when asked about what could be improved in their care, diagnostic accuracy is not a common theme identified. This study was not, however, designed to evaluate this apparent discrepancy and the rate of misdiagnosis in the study sample is not known.

The insights gained from this qualitative study provide an example for the importance of engaging patients directly in quality improvement efforts. By asking patients what they would improve, we were then able to identify important areas of focus that otherwise were not a part of our quality improvement agenda. We should engage patients in agenda setting in quality and safety, and look to their comments in aggregate, not only as individual voices on hospital committees such as a Patient and Family Advisory Council.

Limitations

Our study has several limitations. First, it was conducted at a large tertiary care hospital that often operates at or above capacity. As a result, wait time may have emerged as a more predominant theme than is reflective of all EDs. Additionally, the data are limited to the recorded patient responses to the open-ended question. Follow-up questioning or clarification was not available. This limits our understanding, for example, of which of the multiple possible points of contact with a provider during the ED course—triage, initial evaluation, reassessments, review of results, and discharge—are the most impactful on the patient experience in terms of communication. This limited our ability to analyze subsets of themes for more specific lessons.

Conclusion

Our study reveals the strengths and challenges of a systematic qualitative study of the ED patient experience. Our findings confirm the persistence of previously known patient concerns around wait times, but also reveal previously unreported nuances of patient perceptions of ED crowding that extend beyond wait time or length of stay. Furthermore, the less explored concerns of inadequate communication between the provider and the patient, as well as compassionate care, are also influential themes that may provide an opportunity to develop effective provider education.

With patient-centered and relationship-centered care emerging as a central focus in health-care delivery, qualitative analysis can prove important in generating new hypotheses, providing richer data than traditional quantitative approaches, and conceptualizing how we make assessments and progress in this field. However, qualitative methods can be a labor-intensive process. Until natural language processing is more widely accessible, investigators must consider the amount of resources required with the potential insights revealed.

Author Biographies

Shaw Natsui, MD, MPA, is a graduate of the Harvard Affiliated Emergency Medicine Residency. He is a postdoctoral fellow at the National Clinician Scholars Program.

Emily L Aaronson, MD, MPH, is the Assistant Chief Quality Officer at Massachusetts General Hospital and the Edward P. Lawrence Center for Quality and Safety. She is an Assistant Professor of Emergency Medicine at Harvard Medical School.

Tony A Joseph, MD, is a graduate of the Harvard Affiliated Emergency Medicine Residency. He is an emergency physician at Kaiser Permanente San Francisco.

Andrew J Goldsmith, MD, MBA, is a resident physician at the Harvard Affiliated Emergency Medicine Residency.

Jonathan D Sonis, MD, is a graduate of the Harvard Affiliated Emergency Medicine Residency. He is a administrative fellow in the Department of Emergency Medicine at Massachusetts General Hospital.

Ali S Raja, MD, MBA, MPH, is the Executive Vice Chairman of the Department of Emergency Medicine at Massachusetts General Hospital and an Associate Professor at Harvard Medical School.

Benjamin A White, MD, is an attending physician at the Massachusetts General Hospital Department of Emergency Medicine. He is an Assistant Professor of Emergency Medicine at Harvard Medical School.

Ines Luciani-Mcgillivray, RN, BSN, CEN, is a staff RN at the Massachusetts General Hospital Department of Emergency Medicine.

Elizabeth Mort, MD, MPH, is the Senior Vice President of Quality and Safety and Chief Quality Officer at Massachusetts General Hospital and Massachusetts General Physicians Organization.