Medical Students as Systems Ethnographers: Exploring Patient Experiences and Systems Vulnerabilities in the Emergency Department

Abstract

Objectives

The objectives were 1) to design systems ethnography roles for first‐year medical students that could enhance student learning with regard to healthcare systems, ethnography, and systems thinking and 2) to describe strategies for other education programs seeking to implement systems ethnography roles in clinical settings.

Methods

Fourteen medical students were educated about ethnography and systems thinking and linked with patients in the emergency department (ED) for 12 to 15 hours to observe patient experiences and clinical processes. Students submitted written assignments, participated in a debriefing exercise with ED and medical education leadership, and completed an electronic survey regarding educational benefits and perceived clinical value conferred to the ED using 5‐point Likert‐scale questions. Qualitative methods were used to analyze both students’ assignments and notes taken during the debriefing session, including small‐group report‐outs and discussions, and to identify vulnerabilities in the patient experience.

Results

Students identified one overarching theme of the patient experience—prolonged waiting in close proximity to the fast‐paced, hectic “world” of the ED. Four key categories of systems vulnerabilities were identified through student observations: 1) patient experience; 2) communication and collaboration; 3) processes, physical space, and resources; and 4) professionalism. Students reported improved appreciation for challenges experienced by patients (3.92/5), importance of communication between providers and patients (3.92/5), and improved understanding of the patient experience while receiving care (3.77/5).

Conclusions

These results demonstrate how innovative systems ethnography experiences for medical students can provide unique educational opportunities while at the same time adding value by highlighting shortcomings in the care environment that can be used for system improvement.

Health systems in the United States are seeking to optimize care delivery processes to achieve the quadruple aim, which focuses on improving 1) the patient experience of care, 2) population health, 3) cost, and 4) the work life of healthcare clinicians and staff.1 At the same time, the education and clinical missions are building closer partnerships to catalyze long‐term success by enhancing student education in health systems science competencies and engaging students in roles that can potentially add value to the clinical care environment.2, 3, 4, 5, 6, 7

Integrated with these efforts, medical educators are seeking to enhance students’ learning with higher‐order skills and competencies beyond basic and clinical sciences.6, 7, 8, 9 These competencies include 1) ethnography, the observation and “writing” of culture that allows for critical reflection and deeper learning, and 2) systems thinking, the process of examining the interrelatedness of parts within a larger system.8, 10, 11, 12 Ethnography refers to an immersive process whereby one seeks to empirically understand and articulate the experience of individuals and/or cultures using the self as the instrument of knowing.13 This process uses participant‐observation and field notes to understand the lived realities of a population of people within a discrete environment. Those undertaking fieldwork are taught to be thoroughly self‐reflective and transparent, maintaining awareness of their relationship to their subjects and subjective role in influencing observations (i.e., how one's biases, assumptions, and theoretical underpinnings may color their perceptions). Similarly, systems thinking encompasses “the ability to analyze systems as a whole, including the recognition of essential interrelationships within the system and between subsystems, and any changes and patterns that arise out of the networks of relationships and interactions,” and is a foundational construct underlying the systems‐based practice competency applicable to undergraduate, graduate, and continuing medical education.5, 8, 11, 14, 15 Given that systems thinking requires the ability to observe, collect, and accurately understand complex systems, ethnography would seem to be a tool that can inform this process. Despite recommendations to promote skill development in ethnography and systems thinking and develop learner roles that can add value to health systems, there are few descriptions of programs that have been implemented to accomplish these goals.16, 17

In Fall 2016, we implemented a new educational program that immersed a subset of first‐year medical students into systems ethnography roles in our emergency department (ED). We had two primary goals in this work: 1) to design systems ethnography roles for first‐year students that could enhance learning with regard to healthcare systems, ethnography, and systems thinking and 2) to describe strategies for other education programs seeking to implement systems ethnography roles in clinical settings.

Methods

Setting

In 2013, Penn State College of Medicine (PSCOM) received a grant from the American Medical Association to implement a new systems navigation curriculum for all 150 students in the first‐year class, with the primary goal of aligning medical education with emerging health system needs.18 The new curriculum has two components: 1) a health systems course lasting 17 months and focusing on health systems science topics such as insurance, cost, care coordination, population and public health, high‐value care, teamwork, leadership, ethnography, and systems thinking and 2) experiences in the health system designed to create motivation and context for the classroom‐based course.8, 19 Students are randomly selected to participate in different experiential roles, one of which is a systems ethnography role in the ED. The first year of medical school was chosen for these experiences because the literature is identifying this time period as ideal for students to be integrated within communities of practice of the health system unencumbered by the pressures of clinical clerkships.20 The Penn State Hershey Medical Center is a 501‐bed university‐based acute care hospital in central Pennsylvania. It is a Level I adult and Level I pediatric trauma center with almost 600 residents and fellows training in 64 disciplines (including an accredited residency in emergency medicine). The ED occupies 34,000 square feet, has 60 patient care beds (including nine bays used as a “fast track”), and cares for approximately 75,000 patients per year. As the tertiary care referral center for south‐central Pennsylvania and the primary ED for Derry Township, students rotating through the ED could expect to see the entire spectrum of illness and injury that could present for emergent care. The only patients excluded in this ethnography experience were those who were clinically unstable (ESI triage category 1), incarcerated or with psychiatric or organic cognitive disorders.

Description of the Educational Program

We sought to enhance students’ education within the social context of healthcare delivery, develop their knowledge and skills in ethnography and systems thinking, improve their ability to adopt a patient‐centered perspective, and engage in opportunities to change the health system. Prior to the ethnography experience, all students received a 2‐hour primer on ethnographic theory and methods (facilitated by DRG, a medical anthropologist by training) and systems thinking (facilitated by JDG and DRW). Practice examples were used to demonstrate how ethnographers have creatively engaged with biomedicine, advance students’ understanding of how ethnographic techniques (particularly observation and self‐reflexivity) might be applied in a clinical setting, and illustrate how systems thinking can be applied to understand the root causes of healthcare process issues and inform potential high‐leverage solutions.21 The design team, including three medical educators, Office of Medical Education staff, and two physician leaders within the ED, met several times to discuss the design and goals of the experience, anticipated challenges, and evaluation components.

Based on these meetings, objectives and features of the ethnography experience were agreed upon. Students would: 1) be embedded in the ED for 12 to 15 hours total, observing healthcare delivery and linking with one or more specific patients; 2) practice the use of ethnography and systems thinking skills in conducting these observations and interactions; and 3) work in teams to explore the facilitators, insufficiencies, and potential improvements in the ED. After agreeing upon these features, the approaches, challenges and strategies were identified (Table 1). The experiences were primarily facilitated by a staff member in the Office of Medical Education (DG). An estimated 50 hours were required for Office of Medical Education staff to plan the program, communicate with students, facilitate the clinical placements, arrange the debriefing session, and coordinate the evaluation. Each student was assigned a total of three half‐day observation sessions (4–5 hours each session) within a 6‐week period during which they were linked with individual patients and mentored by ED nursing and physician leadership. Students were provided standard field notebooks to record their observations and personal reactions in real time. Data Supplement 1.

Table 1.

Objectives, Approaches, Challenges, and Strategies For Implementing Systems Ethnography Roles for Medical Students in the ED

Objectives	Approaches	Challenges	Strategies
To immerse students into authentic care delivery processes, and link them with patients to observe processes related to patient care.	Identify appropriate time and space in the curriculum to integrate systems ethnography roles. Collaborate with ED and nursing leadership in the design phase of the experience.	Staff may be unfamiliar with the students’ roles, the purpose of students’ presence, and potential for clinical and educational benefit .	Ongoing communication with students, staff, and ED leadership to ensure smooth transitions in the experiences.
To provide students the opportunity to practice the use of ethnography and systems thinking skills in conducting observations.	Provide students the prerequisite education and knowledge/skills related to ethnography and systems thinking in the clinical setting. Assist students with regular guidance and “check‐ins” during their ED visits (i.e., reviewing field notes, providing exemplars of best practice).	Educational sessions will likely require collaboration with other educators to develop new strategies that may not be part of the current curriculum. Students may be new to the concepts of ethnography and systems thinking and therefore unfamiliar with its use and applicability.	Assume a developmental model of education by focusing on knowledge/concepts and beginner skills within the experience. Provide rationale of goals and examples of applicability of ethnography and systems thinking to their careers as physicians.
To allow students to work in teams to explore the facilitators, insufficiencies, and potential improvements in the ED.	Highlight the team‐based approach to the students throughout all phases of the educational experience. Design an end‐of‐experience debriefing session to allow students to reflect, explore, and articulate observations of insufficiencies and brainstorm potential improvement efforts.	Students may have limited experience or knowledge with the complexities of the healthcare delivery system, thereby limiting discussions. Students may be apprehensive to report insufficiencies to health system and ED leaders.	Integrate an interdisciplinary team of educators and providers into the debriefing session design. Create a safe environment from the beginning of the experience by highlighting the goals and allowing health system leaders to identify their purpose in participating in the experience.

Evaluative Measures

We developed several measures to evaluate whether we met our goals for the curriculum. The data used in this study were approved as part of the education research registry, which was approved by the PSCOM Institutional Review Board (#STUDY00000123).

Systems Vulnerabilities Within the ED

The primary evaluative measure was the identification of systems vulnerabilities in providing patient‐centered care within the ED. At the completion of the experience, each student was required to submit a one‐ to two‐page, single‐spaced assignment that captured their “observations, reactions, interpretations, assumptions, or questions with specific reference to the source of those observations” and included a description of “the patient experience and the healthcare team in the ED (i.e. discuss the process and flow, healthcare professionals, dynamics of the team, with particular emphasis on viewing these from the patient perspective.).”

At the end of the program, a 3‐hour debriefing session was held with all students, course leadership, and ED leadership, including the department chair, vice chair for academic affairs, and nurse manager. Data Supplement 2. In this session, students were divided into groups and asked to refer to their ethnographic field notes and collectively draw a visual representation of the patient experience, healthcare team functioning, and barriers/facilitators to optimal care in the ED. Floor plans of the ED were provided as a reference. These groups then reported the results of their work to the larger group, followed by discussion. Finally, small groups reconvened to: 1) discuss barriers to care that they identified in the ED and 2) develop and prioritize three solutions that could be implemented to improve care. This was followed by report‐outs and group discussion with ED leadership. Two education members took notes during all aspects of the debriefing session.

Educational Value of Systems Ethnography Roles

To determine students’ perceptions of the educational value of the experience, we developed and administered an electronic survey 2 weeks after the debriefing session (November 2016). We made several efforts to optimize content and response process validity. First, survey items were similar in structure as those used in other course evaluations at PSCOM. Second, all items were created and reviewed by our education leadership team to ensure that questions aligned with the goals of the program. Finally, the survey was pilot tested with four Office of Medical Education staff to assess if respondents would interpret the questions appropriately, which resulted in several modifications to improve clarity.22 Items included perceptions of the educational value of the program, improvement in their understanding of the healthcare system and patients’ experience of care, and benefit to their career. Participants responded using a Likert‐scale (“1” representing “strong disagreement” or “not at all,” and “5” representing “strong agreement” or “to a large extent”) and open‐ended text boxes.

Data Analysis

To better understand the system vulnerabilities that could be impacting patient care, we performed a thematic analysis of students’ submitted assignments and all notes taken from the debriefing session, including small‐group report‐outs and discussions.23, 24 Two students did not consent to have their educational data used for research and their data were excluded from qualitative analysis. Two members of our research team (JG, DG) with experience in qualitative research led the analysis phase.3 We used previously published conceptual frameworks to categorize systems vulnerabilities.25, 26, 27 The two investigators first reviewed two student responses using a process of constant comparative analysis to identify initial themes and generate a preliminary codebook.28, 29 One investigator (DG) then independently coded open‐ended responses, followed by a cross‐check of all data by another investigator (JG). Both investigators held regular adjudication sessions to allow for additional code creation, elimination, and refinement. NVivo 10 was used to facilitate the analysis. Coinvestigators discussed overarching themes and representative quotations. The survey data are presented descriptively.

Results

A total of 14 students completed the ethnography experience, which included three separate observations in the ED over a 6‐week period (estimated 12 hours/student). The primary overarching theme that emerged was the contrast between the generally monotonous patient experience and the fast‐paced, hectic “world” of the ED. Four categories of systems vulnerabilities were identified. These findings are described below.

The “Two Worlds” of the ED

Students observed that the patient experience of the ED is in marked contrast to the environment occurring outside of the patient's room or location. Similarly, students identified the “culture” of the ED could cause it to be seen as an intimidating place with potential to alienate or confine /patients. Figure 1 is a student group's drawing created during the debriefing session to depict the patient experience. Students commented about the disparity that exists between these two perspectives (see Data Supplement S3 [available as supporting information in the online version of record of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10038/full] for additional quotations). For instance, one student commented:

Figure 1.

Students’ drawing depicting the patient experience in the ED. Student ethnographers collectively drew this picture of a patient's experience in the ED. The left side of the picture depicts a feature of Salvador Dali's painting “The Persistence of Memory (1931),” also known as “Clocks.” Patients experience a pseudo dream state with the lengthy passage of time while waiting in the ED, separated by a wall, viewing the hurried activities occurring outside of the room (depicted by a tornado on the right side of the drawing).

The potentially high‐stress and fast pace experienced by the ED staff is in stark contrast to the extreme limbo faced by most patients. While the main station is filled with busy computer monitors and various beeping sounds, the isolated ED rooms yield a sense of confinement and seemingly eternal boredom.

Systems Vulnerabilities Within the ED

Students identified several key themes of systems vulnerabilities occurring within the ED, which include: 1) patient experience; 2) communication and collaboration; (3) processes, physical space, and resources; and (4) professionalism (see Table 2 for representative quotations). Students identified numerous issues relating to the patient's experience of care, many of which diminished patients’ satisfaction, dignity, or understanding. Patients frequently were asked to repeat their presenting story to several providers, waited for long periods of time to see providers or have tasks completed, experienced a large number of providers interacting with their care often not knowing their names or roles, and experienced some violations of their privacy. Communication and collaboration between providers was often viewed as less than optimal, with patients being conveyed different care plans from different providers and providers informing patients they “would be back” only never to return to the room. ED processes were identified as slow (from the patient perspective of their own care), untimely, noisy, and often superfluous or redundant (e.g., a CT scan of the chest being followed by a repeat CT scan of the abdomen). Finally, several professionalism incidents were noted between patients and providers as well as among providers, such as staff gossiping, irritability of providers in front of others, and a nurse calling a male patient “honey” in front of his wife.

Table 2.

Systems Vulnerabilities to Patient‐Centered Care in the ED as Identified by First‐year Medical Students in Systems Ethnography Roles

Categories	Description	Example Quotation
Patient experience	Issues related to the patient's experience of care received, which primarily focused on the diminishing of patient satisfaction, dignity, or understanding of care processes or plans.	For every patient with whom I was paired, the waiting was a problem. Combined with the few number of physicians I actually observed interact with my patients (i.e., two), if I were to receive a satisfaction survey, I would be rating 90% of the day on time not spent with a healthcare provider. Once a patient is triaged in a room, there is a flurry of action. The patient is generally seen somewhat quickly by a physician. After that, things tend to slow down. Labs are ordered, meds are started, imaging studies scheduled. In the meantime, the patient could be unattended for a while. Physicians claim they will be “right back,” [but] do not reappear for hours. The presence of a new face in the room just means repeating answers to the same questions that a patient already answered for the previous person. From the patient's perspective, repetition results in frustration, as patients feel they spend the first few hours of their visit describing symptoms to various physicians without receiving any answers or relief.
Communication and collaboration	Issues related to breakdowns or insufficiencies in communication and collaboration between providers or between providers and patients.	With the volume overload, hours of waiting, and the unobservable hierarchy, I would definitely not be able to distinguish the roles of each team member. It felt more like a watercolor painting in there, sort of a blurry transient flow of patients and staff with an absence of demarcated lines. The team informed the family that the patient was “brain dead.” The family dealt with this information as best they could, and requested a chaplain before the patient was to be removed from life support. However, 20 minutes later, a physician informed the family their plan was to observe the patient for another 72 hours. Perhaps the [two] teams discussed the patient's case and agreed on this plan, but the family was very confused. [Patients] were walked into the room, and numerous tests were done, all without explaining [their purpose]. I observed multiple patients nodding along. When the doctor left, they would get together with their families and ask: “Why are they doing that?” or “I don't know why they'd be giving me this test, it doesn't seem to have anything to do with my problem. Many patients felt that there was NO communication between the healthcare team. They felt like they had to repeat themselves every time a new doctor, nurse or physician entered the room.
Processes, physical space, and resources	Issues related to insufficient ED processes, which were described as slow, untimely, or redundant.	The singularly most used verb in my observations is ‘wait,’ which is hardly a verb at all. In the roughly twelve hours I spent next to the patients’ bedsides, I saw two physicians. I used to think that the most time‐consuming part of the ED visit was seeing the physician, getting labs drawn/tests, and waiting for the results. I was surprised to see that the discharge took longer than everything else. But I was in the patient's room so I'm not sure how the discharge process works, and when I went back to the triage area, the ED had certainly gotten busier.
Professionalism	Issues related to professionalism lapses or incidents that occurred between providers or between providers and patients.	The patient's wife requested that I step out of the bay while her husband had his physical exam. Someone scribing on a cart approached me and said, “Do you know how he hurt himself? I always feel so awkward going and asking over and over again.” She took my word for it. It felt like I had stepped into the Twilight Zone. For the majority of the visit, there were two nurses gossiping about their financial circumstance and complaining about their jobs in the emergency room.

Educational Value of Systems Ethnography Roles

Based on quantitative data, overall the students believed the experience was valuable for their education but only marginally (3.93/5). Ten of 14 students reported their ethnography skills improved to a “moderate” or “large” extent (mean = 3.64/5). The strongest agreement statement related to students’ improvement in appreciating the challenges and/or barriers experienced by patients in the healthcare system, and the importance of communication among providers and patients, with five of 14 students reporting “to a large extent” for both items (mean = 3.93/5). Students did also report an increased understanding of the patient experience (3.79/5; see Table 3 for complete results).

Table 3.

Attitudes and Perceptions of First‐year Medical Students Regarding the Educational Value of Systems Ethnography Experiences in the ED

Survey Items	Response Categories (n of 14 Total Responses)					Total Response, Mean (±SD)
	Category 1	Category 2	Category 3	Category 4	Category 5
Rate your perception of the ED ethnography experience overalla	0	1	3	10	0	3.64 (±0.63)
The ethnography experience was valuable for my educationb	0	0	3	9	2	3.93 (±0.62)
The ethnography experience helped improve care provided in the emergency roomb	0	0	11	1	2	3.36 (±0.74)
Comparing before the experience to after, please respond to the following items:c
My appreciation for challenges and/or barriers experienced by patients in the healthcare system has improved.	0	1	4	4	5	3.93 (±1.00)
My knowledge of healthcare delivery has improved.	0	1	4	7	2	3.71 (±0.83)
My understanding of the patient experience while receiving care in the healthcare system has improved.	0	0	6	5	3	3.79 (±0.80)
My appreciation for the importance of teamwork and collaboration among providers has improved.	0	1	8	3	2	3.43 (±0.85)
My appreciation for the importance of communication amongst providers and patients has improved.	0	1	4	4	5	3.93 (±1.00)
My knowledge and skills in systems thinking have improved.	0	1	5	5	3	3.71 (±0.91)
My ethnography skills (observing environments without prematurely reacting) have improved.	1	0	3	9	1	3.64 (±0.93)

^aLikert‐scale responses: 1 = poor, 2 = fair, 3 = average, 4 = good, 5 = excellent.

^bLikert‐scale responses: 1 = strongly disagree, 2 = somewhat disagree, 3 = neither agree nor disagree, 4 = somewhat agree, 5 = strongly agree.

^cLikert‐scale responses: 1 = not at all, 2 = to a small extent, 3 = to some extent, 4 = to a moderate extent, 5 = to a large extent.

Qualitative analysis revealed enhanced appreciation of care delivery within the ED, nuanced challenges in communication between providers and patients (e.g., lack of knowledge of providers’ names and roles from the patient perspective), and patients’ worry about diagnoses and prognoses. Students additionally demonstrated their ability to ethnographically and objectively observe processes and reflect on these observations. Ethnography and systems thinking skills are demonstrated in the following comment:

The overall structure of the ED is made up of several constituents—some of those are palpable, such as the physical space and layout, while others can only be observed, such as the social dynamics and culture of the ED. It is important to take into account as many of these components as possible when analyzing the ED because they are all interconnected in one way or another. Three aspects that stood out the most to me [were] the physical space of the ED, the subject of time, and communication.

Discussion

The systems ethnography experience was generally well received by students, a majority of whom believed that it was beneficial for their education and career development. Ethnography and systems thinking are fairly new to the medical education literature, but are emerging as higher‐order competencies for all healthcare professionals.7, 8, 10, 14 Learning about the healthcare system and patient‐centered care is increasingly recognized as a critical, if undervalued, component of undergraduate and graduate medical education.8, 16 With the emergence of education programs in health systems science, educators are exploring novel experiential roles for students that allow for authentic engagement within healthcare communities of practice, particularly during early years of their education.20, 30 Our program can provide a potential blueprint for other programs seeking to expand student experiences that can be beneficial for learning health systems science while also adding value to clinical microsystems.31 Such a model also validates the role medical humanities can play in clinical care, creating valuable opportunities for students to develop and apply skills in observation and ethnography.

Moreover, medical schools are increasingly faced with concern for efficiency and productivity in clinical settings and looking to develop and implement new student experiences that can add value to health systems while also enhancing learning.2, 3, 4, 9, 31, 32, 33 Proposed “value‐added” roles include patient navigators, health coaches, and using students as systems ethnographers and analysts.31 However, there are limited descriptions of such programs in the literature. Our learners reported modest agreement/neutral perceptions of “adding value” to ED processes, but did not report low scores, suggesting that they did not perceive their presence as a burden. Our qualitative analysis was planned with the anticipation that systems vulnerabilities exist within the ED, some of which are fully recognizable and some that may be unidentified by ED leadership and staff. Student ethnographers identified several critical issues influencing patient care, including breakdowns in communication, insufficient collaboration, and unprofessional events. These identified systems issues were not novel for the ED literature, as prior studies have identified factors that influence the patient experience in the ED, including the emotional impact of being in an urgent situation, staff–patient interactions, waiting for services, and the ED environment.34, 35 Additionally, key themes of the patient experience in the ED include the need for reciprocal communication and information sharing, interpersonal skills and professionalism, care environment, emotional support, discharge planning and process, and correct treatment goals.27 These findings do suggest the exercise was successful in having students use ethnographic techniques to evaluate the ED's clinical microsystem, and the results are similar to studies suggesting that ethnographic research methods can successfully be used in medical education and clinical settings.12, 36, 37 We believe that there is potential for this type of role in other clinical settings, particularly in areas that may be underexplored—it is in these areas where students may add the most value.

Successful implementation of such a program depends on several factors. First, the buy in of key leadership, with investment both in education and systems improvement and change, is critical. Second, although financial costs were minimal, activities and debriefing sessions require considerable planning and support and should be considered in the process of allocating resources and creating job descriptions. We have highlighted several of the challenges we encountered in fulfilling our main objectives and strategies that can be considered in overcoming these barriers (Table 1).

Limitations

There are several limitations to our study. First, the single‐center implementation calls into question how well this can be replicated in other institutions. Although we present general strategies and approaches for implementation to maximize generalizability, we suspect that modifications will be necessary when implementing these types of experiences in other medical schools. Second, although the surveys were pilot tested for content validity, the construct validity was not rigorously assessed. Finally, we did not perform a preassessment of students’ perceptions, only assessed students’ perceptions of the experience, and did not measure more objective educational outcomes or changes in their behavior or the perspectives of the ED staff involved with the activity.

Conclusions

Recommendations have been made for new clinical immersion roles for students to enhance education in healthcare delivery concepts while also contributing to systems of care, but little scholarly work has advanced this field. We successfully designed, implemented, and studied a novel “systems ethnography” activity for first‐year medical students in the ED that directly addresses this need. We believe this study can provide a template for other medical schools seeking to engage students in communities of practice to learn and make a difference within the clinical setting.

The authors thank all staff in the emergency department who supported the experience and the medical students for their engagement in the project.

Supporting information

Data Supplement S1. Ethnography program process, tasks, and assignment.

Data Supplement S2. Systems ethnography debriefing session outline.

Data Supplement S3. Representative quotations from overarching theme of the patient's experience of two worlds of the emergency department.

AEM Education and Training 2017;1:225–233.