Developing a Simulation to Study Conflict in Intensive Care Units

Abstract

Rationale: Although medical simulation is increasingly being used in healthcare education, there are few examples of how to rigorously design a simulation to evaluate and study important communication skills of intensive care unit (ICU) clinicians.

Objectives: To use existing best practice recommendations to develop a medical simulation to study conflict management in ICUs, then assess the feasibility, acceptability, and realism of the simulation among ICU clinicians.

Methods: The setting was a medical ICU of a tertiary care, university hospital. Participants were 36 physicians who treat critically ill patients: intensivists, palliative medicine specialists, and trainees. Using best-practice guidelines and an iterative, multidisciplinary approach, we developed and refined a simulation involving a critically ill patient, in which the patient had a clear advance directive specifying no use of life support, and a surrogate who was unwilling to follow the patient’s preferences. ICU clinicians participated in the simulation and completed surveys and semistructured interviews to assess the feasibility, acceptability, and realism of the simulation.

Measurements and Main Results: All participants successfully completed the simulation, and all perceived conflict with the surrogate (mean conflict score, 4.2 on a 0–10 scale [SD, 2.5; range, 1–10]). Participants reported high realism of the simulation across a range of criteria, with mean ratings of greater than 8 on a 0 to 10 scale for all domains assessed. During semistructured interviews, participants confirmed a high degree of realism and offered several suggestions for improvements.

Conclusions: We used existing best practice recommendations to develop a simulation model to study physician–family conflict in ICUs that is feasible, acceptable, and realistic.

Medical simulation is an established strategy to help clinicians acquire new procedural skills in clinical medicine (1–5). There are also reasons to think it may be a valuable training method for difficult communication skills in intensive care units (ICUs). First, there is growing evidence that communication skills training in particular requires careful, in-the-moment feedback followed by opportunities to retry the skill—something that is not possible in real-life settings (6). This is consistent with principles of adult learning theory that deliberate practice in a controlled environment is a crucial aspect of mastering complex skills (7). Second, simulation offers an advantage over real-life experimentation because it allows for a standardized scenario that helps control for sources of external variation (8). Third, simulation does not entail the kinds of risks and burdens to human subjects that arise when trying to study communication behaviors in patients with advanced critical illness (e.g., emotional distress and conflict from individuals attempting to practice and deploy underdeveloped communication skills and the burdens of research participation on families of critically ill patients [9, 10]). Fourth, simulation-based studies are far more efficient to conduct than studies on patients with advanced critical illness, in terms of both costs and logistical difficulty.

Despite the potential importance of using simulation in communication skills training, there is a paucity of research describing how to develop a “communication simulation” according to best practice recommendations. One recent study assessing the effects of a simulation-based communication skills intervention found no effect on quality of communication (11). This highlights the importance of developing and studying simulation in a rigorous manner to determine if it is an appropriate tool to teach communication. Existing guidelines for simulation development and deployment suggest certain key steps in development, such as obtaining feedback, conducting storyboarding, and integrating with existing curriculum, among others (12–14).

However, most existing publications on “communication simulations” do not detail the development process, and consequently there are few examples of how to rigorously design and deploy a simulation to evaluate and study important communication skills of ICU clinicians. This is a problematic gap in the literature, because although simulation is used in many training programs to meet Accreditation Council for Graduate Medical Education core competency requirements for communication and interpersonal skills, there are few clear data about the most effective way to develop useful simulations.

One particularly common, high-stakes communication scenario is that of conflict in the ICU (15, 16). If poorly handled, conflict in the ICU contributes to poor outcomes (17, 18), low staff morale, and clinician burnout (19, 20). However, skillful management may help parties better understand each other’s perspectives, identify and correct misperceptions, and develop mutually agreeable solutions (21, 22). There are no published reports describing the development of a simulation exercise to study clinician–family conflict in ICUs.

Therefore, we describe the application of existing best practice recommendations in the development of a simulation exercise to study how physicians manage conflict with surrogates in the ICU.

Methods

Case Development

We designed our case using evidence-based guidelines for simulation in health care (12–14, 23), which include the following: (1) selection of an opportunity to assess and study a particular set of skills, (2) performance of a formal needs analysis, (3) organization and storyboarding of case content matched to learning objectives, (4) formal and rigorous measurement of performance, and (5) incorporation of collaboration and continual feedback. The design and development of this simulation exercise took approximately 16 weeks.

Selection of a specific opportunity to study a particular set of skills

Because physician–surrogate conflict in the ICU is a specific and common scenario, we chose to develop a case to study communication skills in conflict management to better understand how clinicians manage conflict with surrogates. The scenario is a patient with advanced cancer who suffers an acute event with a high probability of death (Figure 1). To reliably provoke conflict, we created a situation in which the patient had an advance directive declining the use of any advanced life-sustaining therapies because of his concerns about poor quality of life and functional impairment. The surrogate refused to follow the patient’s stated preferences, instead opting to continue life support with the goal of life prolongation. We developed a detailed case background (see online supplement).

Figure 1.

Virtual patient case scenario.

Performance of a formal needs assessment

To enhance face and content validity of the case, we gathered input about the case design using an iterative, multi-stakeholder needs assessment that included experts in health communication, critical care medicine, palliative medicine, bioethics, and decision science. The perspectives of these experts were used to develop and iteratively refine the simulation case.

Organization of case content to match learning objectives

To enhance realism, we developed detailed medical records using the exact format of the institution’s electronic medical record. Clinical data included recent vital signs; laboratory data; progress notes from critical care medicine, neurosurgery, and neurooncology services; and dictated radiology reports. Additionally, the medical record contained the patient’s Five Wishes advance directive (24), filled out by hand and signed by the patient and surrogate. In addition, we conducted the simulated family meetings in the conference room located within the medical ICU that is typically used for clinician–family discussions.

An experienced medical actor portrayed the patient’s daughter. Training for this case included giving the actor written summaries of the patient’s and daughter’s backgrounds, values, and beliefs, and rehearsals with study staff. In the rehearsal we role-played various behaviors so the actor would have a good sense of what physicians might do. Because we wanted to maintain the clarity of the conflict, the actor was instructed to clearly articulate, if prompted, that she knew the patient’s wishes but she did not believe that following his wishes was in his best interests. To ensure fidelity to the case, study staff listened to encounters and provided immediate feedback to the actor.

Measurement of performance

Performance of the simulation was measured by self-report from the participants. Self-report data included a questionnaire assessing several aspects of realism as well as a validated measure of perceived conflict, described further below in the study procedures.

Incorporation of feedback

After the simulation, participants completed a semistructured interview and facilitated debriefing session about the simulation, further described below in study procedures. This facilitated debriefing offered participants an opportunity to reflect on their experience and what they may have considered doing differently. This debriefing allowed us to understand what the participant was thinking rather than just interpreting their behavior.

Study Procedures

We recruited a convenience sample of physicians who routinely participate in family meetings in ICUs: critical care and pulmonary/critical care physicians, trainees (critical care and pulmonary/critical care fellows, internal medicine residents), and hospice and palliative medicine physicians. We chose these groups for two reasons: (1) to have a representative sample of those performing family meetings in the ICU, and (2) to later assess the ability of the model to discriminate between the conflict management of palliative clinicians compared with non–palliative care clinicians. We recruited participants with e-mailed invitations. All participants completed written informed consent and were compensated $25. The institutional review board approved study procedures.

Participating physicians completed a presimulation questionnaire eliciting demographic information and self-assessed skill in handling difficult conversations with surrogate decision makers. They then had the opportunity to review the medical records and advance directive. After reviewing clinical data, participants conducted an audio-recorded family conference with the actor portraying the surrogate decision maker. Conferences were limited to 30 minutes.

Participants completed a postconference questionnaire—developed by the research team and not formally validated—assessing clinical verisimilitude with questions about diagnostic and prognostic facts. We assessed realism with six questions, scored on a 0 to 10 scale, with 0 indicating “completely different” and 10 indicating “almost identical” to the participant’s prior experience. Various elements of the simulated case were assessed for realism: overall case realism, the actor’s portrayal of a family member, the emotions expressed during the conference, the medical record, the conference setting, and the disagreement with the surrogate. We assessed whether the physician perceived conflict with the surrogate using a one-item conflict scale developed by Abernethy and Tulsky and previously used in multiple prior studies of ICU decision making (25–28).

After the simulation, participants completed a semistructured interview and debriefing about the simulation. Using principles of cognitive interviewing (29), a trained interviewer explored each participant’s experience with the following elements: the conference in general, realism, and whether participants would have done anything differently compared with a real family conference. The interviewer used open-ended probes to gain further insight into issues affecting realism and any recommendations participants provided to improve the simulation. At the end of the interview, the interviewer debriefed the participant by explaining the specific purpose of the study and allowing the participant to ask questions.

Experienced medical transcriptionists transcribed the audio-recorded conferences and interviews verbatim, and study staff listened to a subset of audio recordings to ensure fidelity of the transcription to the recordings. Transcriptionists deidentified all transcripts for analysis.

Data Analysis

We generated summary statistics for participants’ responses to questionnaire items. All quantitative data analysis was performed using Stata 12 (College Station, TX).

Two investigators (R.A.S. and N.C.E. or J.C.) independently read transcripts of the simulated conferences and identified the point at which conflict arose. Three points when conflict arose were identified from our iterative data analysis process: (1) both the physician and the surrogate acknowledged knowing the content of the advance directive and thus that the current plan of care violated the patient’s preferences; (2) one of the participants made a direct statement disagreeing with the other’s opinion about the use of life-sustaining therapy; or (3) the use of any overtly confrontational language, whether or not it related to the use of life-sustaining therapy or the patient’s advance directive. We resolved all disagreements about the start of the conflict by discussion among investigators.

We used the modified grounded theory approach as described by Crabtree and Miller (30) to analyze the semistructured interviews. This method is well suited for the practical application of qualitative data in a medical setting. We constructed a codebook through an iterative process using constant comparisons (31). We began analysis shortly after the first interviews were completed. Through close readings and discussion of the interview transcripts, we developed a comprehensive coding scheme, which we iteratively modified through a series of investigator meetings. We reached thematic saturation where all new data could be easily assigned to existing themes. Once we finalized the codebook, one investigator (R.A.S.) applied the codes to all transcripts. To ensure reliability of the coding scheme, a second investigator (N.C.E.) independently applied the codes to a subset of passages, and we assessed interrater reliability (kappa = 0.89). We used Atlas.ti for qualitative data management (Scientific Software, Berlin, Germany).

The codebook contained the following three major themes brought up by participating clinicians: (1) simulation realism, which included overall impressions, realistic elements, and unrealistic elements; (2) negotiation tactics, which included interpreting the daughter’s decision, responding to the daughter’s decision, and using the advance directive; and (3) normative ethics, which included such topics as educating about surrogate decision making and emphasizing the patient’s values.

Results

A convenience sample of 43 physicians volunteered to participate in response to e-mailed invitations. Of these physicians, seven could not complete the simulation due to scheduling conflicts. Thirty-six physicians were enrolled (84% enrollment), comprising 10 intensivists, 12 hospice and palliative medicine specialists, and 14 trainees (9 fellows and 5 residents). Figure 2 depicts participant enrollment. Table 1 details participant characteristics.

Figure 2.

Study enrollment flow diagram. IM = internal medicine.

Table 1.

Characteristics of participants

	n (%)
Sex
Male	16 (44)
Female	20 (56)
Race
White	21 (58)
Asian/Pacific Islander	10 (28)
Black/African American	2 (6)
Other	1 (3)
Declined to answer	2 (6)
Training level
Attending	22 (61)
Fellow	9 (25)
Resident	5 (18)
Primary specialty*
Internal medicine	24 (67)
Emergency medicine	5 (14)
Anesthesia	2 (6)
Neurology	1 (3)
Family medicine	2 (6)
None	5 (14)
Subspecialty*
Pulmonary and critical care medicine	5 (14)
Critical care medicine	5 (14)
Hospice and palliative medicine	12 (33)
Infectious diseases	1 (3)
Geriatrics	2 (6)
Nephrology	1 (3)
None	13 (36)
	Mean (SD)
Age, yr	37 (9.2)
Years in practice	8.0 (8.4)

N = 36.

^*Sums add to >100% because some answered more than one specialty.

Feasibility and Acceptability

All participants who enrolled completed the simulation. The average length of the conferences was 22 minutes. The average length of the debriefing sessions was 12 minutes. All participants perceived conflict with the surrogate. Participants perceived a mean level of conflict of 4.2 on a scale of 0 to 10 (SD, 2.5; range, 1–10). Participants rated their skill in handling the conflict on a 0 to 10 scale, with a mean of 7.2 (SD, 1.4).

Table 2 contains several examples of how the conflict initially became apparent during the simulation.

Table 2.

Examples of physician–surrogate conflict

Physician: So, it sounds like you and your dad had some conversations at that time, and I see that you also completed an advance directive. So help me understand some of the conversations and what he was thinking at that time?

Surrogate: Well he, he doesn’t want to be on machines when he dies. He wants to be—it’s important for him to be around his family, because when my mom passed away, [voice quivers] he never really felt like she knew that he was there. And being with family is really im—

Physician: This is really hard. There’s nothing harder than to see someone that you love [Surrogate sniffles] really—

Surrogate: I can’t imagine—

Physician: …dying.

Surrogate: [sobs] Um, I’m certainly not ready to throw in the towel yet on my father.

Physician: Well, I can tell you, kind of, you know, it seems like the Five Wishes document is what you’d use if you weren’t able to make your own decision. And right now we are really at a situation where we need to kind of decide the kind of care that we want to give because he’s not able to help make our decisions right now, unfortunately. And so it sounds like, and I mean, I really do, I really do wish I had different, uh, a different way to say this, but I think the point that we’re at right now is probably this situation over here. Um…

Surrogate: I understand. I understand that my dad wanted this, and…

Physician: It looks like you’re—

Surrogate: I don’t want to take him off the machines; I really don’t. I want to keep him on there because I really want to get him home.

Physician: I don’t think that he will leave this hospital a—alive. I think the question is, is when, how that happens. If this happens because his heart stops or because we start to, we, we withdraw some of the machines and he, and he dies kind of, uh, naturally from, from his illness. I think that he, for example, right now if we turn the ventilator off, suppose you were to, you wanted to, if you thought about him going home, his lungs can’t support him without a ventilator assisting him. His heart can’t support his blood pressure without certain medications. So I don’t think that, that him going home is something that will, will happen. It, it, I think it, it can’t. And I think that you, it’s a whole lot to process and I think that what…

Surrogate: I, I, I just really want to keep him on the machines. I know what you’re saying. I just, I just want to give him some sort of hope. [sighs]

Realism and Clinical Verisimilitude

As summarized in Table 3, participants rated the simulation’s realism highly on all six domains assessed (8.4–9.7 on a 0–10 scale). There were no differences in perceived realism by trainees versus more experienced physicians (P = 0.38) or by palliative care clinicians versus non–palliative care physicians (P = 0.56).

Table 3.

Realism of simulation elements

Domain	Mean (SD), 0–10 Scale
Realism of actor portraying surrogate	9.0 (1.1)
Realism of emotions expressed during conference	9.1 (0.9)
Realism of conflict with surrogate	8.4 (1.5)
Realism of environment of conference	9.7 (0.6)
Realism of medical records	9.3 (1.2)
Realism of conference as a whole	8.7 (1.1)

N = 36.

During the semistructured interview, all physicians reported that the simulation was realistic and similar to their usual practice of ICU family conferences. Some representative participants’ quotes include:

“It felt very much like a regular family meeting up in [the medical ICU], or any other ICU. I think it matched the real experience.”

“I thought that [the actor] was very real in what she was experiencing, the shock, the fear, so I can’t help but say it really felt like a truly, a real encounter. I forgot that it was a simulation.”

“[A]s far as the clinical condition and the setting,…I thought was astoundingly spot-on. I definitely was in the moment, definitely.”

Confirming that participating clinicians’ clinical assessments were consistent with the content of the case, all participants appropriately identified one or more of the factors contributing to the patient’s respiratory failure. Participants’ mean estimate of the patient’s probability of surviving the hospitalization if life-sustaining treatment was continued was 17.6% (SD, 16.1%; range, 0–70%), and their mean estimate of the patient’s probability of surviving for 3 months was 8.7% (SD, 12.2%; range, 0–50%).

Suggestions for Improvement

Participants identified specific realistic elements, including the actor’s abilities, the clinical facts of the case, and the medical records. Participants also provided several suggestions to improve the realism of the simulation: the addition of other family members or clinicians, rather than only the physician and one surrogate; an advance directive that was less explicit about the limitations on life-sustaining therapy; and the ability to examine a flesh-and-blood patient to better identify with the case.

Discussion

We used an iterative, multidisciplinary approach to develop and test a simulation model to study clinician–surrogate conflict in ICUs. The simulation reliably led to clinician–surrogate conflict and was perceived to be acceptable and realistic by participants.

Our effort represents one example of how to translate best-practice recommendations into the development and deployment of a simulation exercise in the ICU. Perhaps the most important implication of our findings is that simulations, when rigorously developed and pilot tested, may be a valuable adjunct to the more traditional methods of teaching end-of-life communication skills—didactic instruction, direct observation of mentor behavior, and supervised practice on real patients. We found high perceived realism and acceptability, which, taken together with similar findings in two recent studies in the intensive care setting (32, 33), strengthen the argument that simulation is a valuable adjunct to other training methods for difficult communication skills.

A strength of this effort relative to prior studies in the field is our use of a rigorous mixed-methods approach to develop and assess the feasibility and acceptability of the simulation. Although quantitative approaches to assess realism have been used in surgical and procedural simulations (34–37), we are aware of no prior simulations about medical communication to use such instruments. Our study suggests that quantitative assessments of realism can provide evidence of validity in novel communication simulations. Semistructured interviewing is a valuable qualitative method of facilitated debriefing after the simulation. Debriefing is a recommended practice in simulation and provides important data regarding realism, suggests improvements, and offers participants an opportunity to reflect on their experience and what they may have considered doing differently (13).

Participants in our study provided valuable feedback on ways to improve the simulation. Most participants indicated that their usual experience of clinician–family conferences includes more attendees than one physician and one surrogate; generally a bedside nurse, unit care coordinator, and/or social worker also attend. Although funding constraints limited our ability to involve multiple actors in each simulation, future efforts should seek to recreate the interdisciplinary environment that is the norm for ICU family conferences. This is in line with existing recommendation on simulation approaches to team training (13). Second, several physicians noted that the advance directive was more clear and applicable than is often the case. We piloted versions with more ambiguous advance directives but found that they did not reliably evoke conflict.

Our study has several limitations. The favorable reception to this simulation exercise was at a single center and may not generalize to all other institutions. Additionally, sample sizes in the different groups of participants were small, prohibiting detailed between-group comparisons of the perceived realism by different ICU physicians. We used one type of conflict in the simulation—whether to continue life-sustaining therapy for a critically ill patient with an advance directive. There are numerous other types of conflict that could be studied; we plan to build on the foundation of the work described herein to develop other conflict scenarios for use in training interventions. Finally, as quantitative measures of realism have not been previously used in this type of simulation, we are unable to compare our participants’ ratings with prior ratings.

In conclusion, our medical simulation targeting communication skills for conflict management in the ICU lays the groundwork for future simulation-based education efforts in this area. A uniform standard for simulation development, testing, and integration with existing curricula is vital in realizing the potential impact of simulation on continuing medical education.