Hospital-based physicians’ intubation decisions and associated mental models when managing a critically and terminally ill elder

Abstract

Background

Variation in the intensity of acute care treatment at the end-of-life is influenced more strongly by hospital and provider characteristics than patient preferences.

Objective

We sought to describe physicians’ mental models (i.e., thought processes) when encountering a simulated critically and terminally ill elder and to compare those models based on whether their treatment plan was patient preference-concordant or preference-discordant.

Methods

73 hospital-based physicians from three academic medical centers engaged in a simulated patient encounter and completed a mental model interview while watching the video recording of their encounter. We used an “expert” model to code the interviews. We then used Kruskal Wallis tests to compare the weighted mental model themes of physicians who provided preference-concordant treatment versus with those who provided preference-discordant treatment.

Results

66 (90%) physicians provided preference-concordant treatment and 7 (10%) provided preference-discordant treatment (i.e., they intubated the patient). Physicians who intubated the patient were more likely to emphasize the reversible and emergent nature of the patient situation (z=−2.111, p=.035), their own comfort (z=−2.764, p=.006), and rarely focused on explicit patient preferences (z=2.380, p=.017).

Limitations

Post decisional interviewing with audio/video prompting may induce hindsight bias. The expert model has not yet been validated and may not be exhaustive. The small sample size limits generalizability and power.

Conclusions

Hospital-based physicians providing preference-discordant used a different mental model for decision making for a critically and terminally ill simulated case. These differences may offer targets for future interventions to promote preference-concordant care for seriously ill patients.

Variation exists in the triage(1) and treatment(2) of seriously-ill elders at the end of life. Research suggests that this variation cannot be fully explained by illness characteristics(3) or patient preferences.(4,5) Instead, characteristics of the region or the hospital to which the patient is admitted appear to drive these variations.(6) The mechanism of treatment variation at the hospital level is still a subject of speculation.

One comparative qualitative case study of two academic medical centers that varied in end-of-life (EOL) treatment intensity suggested that differences in norms of physician decision making regarding life-sustaining treatment are one potential mechanism driving variation in patient care. Differing norms included the goals of life-sustaining treatment, the definition of a “dying” patient, the locus of decisional control, and the relative importance of harms of commission versus omission.(7) A companion simulation study identified how norms might affect hospital-based physicians’ decision making heuristics, case perceptions, and the consequent diagnosis and treatment of a seriously ill patient.(8)

A deeper understanding of hospital-based physicians’ decision making processes might help further explain variation in EOL treatment intensity that is attributable to providers (rather than patient preferences) and thereby identify opportunities for interventions to promote preference-concordant treatment. To this end, the purpose of the current study is to describe the mental model (i.e., thought process) of hospital-based physicians from three academic medical centers who made preference-concordant versus preference-discordant treatment decisions for a simulated critically and terminally ill elder with informed, stable treatment preferences.

MATERIALS AND METHODS

Simulation

We analyzed physician interviews from two parent studies exploring hospital-based physician contributions to variation in EOL treatment intensity using a mixed-methods mental model approach. Details regarding participant recruitment, simulation procedures, and data collection have been previously reported.(8,9) Briefly, we designed a scenario of a man in his late 70’s with metastatic cancer and progressively worsening vital signs, accompanied by his caregiver wife, by combining Sim-Man technology vital signs tracings with experienced and trained standardized patients. Physician subjects received a chart prior to entering the room, including a discharge summary from a recent 2-month hospital stay, a report of 1-week old CT scan, and the assessment and plan from his presentation to the ED. The chart contained no advance care plan.

The patient had a do not intubate order documented at the skilled nursing facility from which he had been admitted, but it was not transferred to the hospital. The patient and his wife knew there were no further curative treatments available and expected him to live no longer than 3–6 more months. If asked during the course of the encounter, the patient and his wife would reveal their knowledge of the cancer prognosis, preference for avoiding re-admission to the ICU, or intubation, and to receive comfort-focused treatment. The husband’s role preference for decision making was to make his own decisions independent of the physician (however, he is dyspneic and unable to speak more than 1–2 words). The wife was aware of this role preference but is ambivalent about her husband’s treatment preference and has a more passive role preference for decision making. Given a choice between treatment alternatives, she will ask the doctor for a recommendation. If the doctor makes a treatment plan recommendation she will accept it. If the doctor makes a directive treatment plan without assessing treatment preferences, she will acquiesce. If, on the other hand, the doctor offers a choice between two treatment options, she will choose the least intensive option.

The scenario was designed to induce an experience of time pressure for decision making in two ways. First, the patient’s vital signs meet standard criteria for a “rapid response team” upon entry into the room. Second, they steadily deteriorate over the course of the simulation; however, the patient does not frankly arrest. The simulation ended when the physician makes a treatment plan or 30 minutes elapse, whichever came first.

Mental Models Approach

The term “mental models” refers to a methodological approach to understand decision-making processes, represented in the form of propositions or diagrams. Common diagrammatic mental model approaches include concept mapping, semantic web formation and decision-analysis.(10–12) Decision-analysis based mental models can be used to describe a target population’s understanding and conceptualization of decision processes, as characterized through direct or indirect elicitation.(10,13,14) Our mental model analysis fits the latter category by using a “think-aloud” interview (e.g., indirect elicitation) that prompted physician research participants to recall their thought process during a simulated patient encounter while watching a video playback of the encounter. The subject’s answers were analyzed for key topics and references linking two topics. We then arranged the identified topics into a diagram with visual connections representing ideological connections.

Participants

Participants included emergency medicine, hospitalist, and intensivist physicians from three major academic medical centers. These medical centers reflected a range of end-of-life treatment intensity, as measured by ICU use among chronically-ill Medicare fee-for-service decedents.(15) Institution A had the least intensive ICU use (3.6 ICU days per Medicare decedent in the last 6 months of life; 23% of their terminal admissions involving an ICU stay), Institution C had the most intensive ICU use (13.9 ICU days per decedent; 41% of terminal admissions involving an ICU stay), and Institution B more closely mirrored the national average (4.7 ICU days per decedent; 26% of terminal admissions involving an ICU stay).(15)

Eighty-seven participants completed the simulation encounter, 73 (84%) of whom had complete debriefing interviews. In the remainder of instances, study time elapsed before the participant finished watching their encounter video with the interviewer and therefore the interview was incomplete. In qualitative research using semi-structured interviews, 10–15 interviews per group is typically sufficient to achieve thematic saturation.(16) This secondary analysis seeks to understand intubation decisions, which occurred in 10% of the simulations. Based on a sample size of 73, we had sufficient power to detect only very large differences in the mental model concept category weights between intubators and non-intubators (i.e., 80% power to detect an odds ratio of approximately 7).

Simulation Outcome

As previously described(9), we identified intubation and other treatment decisions by live coding during the encounter (i.e., statements to the patient or surrogate regarding the treatment plan) and from orders written after the encounter. A decision to intubate the patient would be preference-discordant and was considered a medical error in this simulation.

Physician Characteristics

We recorded physician demographics and validated measure of reactions to uncertainty (17) and risk tolerance (18) in a web-based survey completed after the simulation.

Interview Procedure

After completing the simulation, participants completed a mental model debriefing interview while watching a video of their encounter. The debriefing interview followed a semi-structured format to allow interviewees to express ideas naturally, in their own words. The interview consisted of five sections corresponding to the main areas of interest: the physician’s recollection of their goal when entering the patient’s room, the thought process during each two minute segment of the scenario, their through process in response to actors’ scripted statements, inquiry of factors supporting an intention to intubate the patient, differences between the simulation patient and patients with a similar prognosis, and the perception of the patient’s preferences regarding intubation. One of four investigators conducted each interview after being oriented by a social scientist experienced in the method.

Expert model diagram formation

We generated an expert model diagram based on discussion with a panel of experts on the predicted topics of relevance to the physician’s thought process during the simulation (Figure 1). The panel included expertise in decision sciences, communication, critical care, and palliative care. The model informed the codebook and contains 12 topics with a total of 44 subtopics grouped by meaning. Connections between concepts are demonstrated by a single line (e.g., between “physician evaluation” and “physician action”), and do not occur between all topics.

Figure 1.

Expert mental model of key concepts described by physicians encountering a terminally and critically ill elder. Concepts or ideas were identified and grouped by an expert panel into concept categories with a descriptive heading. For example the concept category “Patient Understanding” has two related sub-concepts termed “see reality” and “competent”. The grouping is represented in this figure by the encompassing box. In this figure, lines connecting the concept category boxes represent ideological links or relationships between the concept categories (e.g., instances in which concepts were mentioned in the same statement during the interview.) In this expert model, not all concept categories were thought to have ideological links and are not connected by a line in this figure.

Content analysis

Three raters coded transcribed interviews. The coding process parsed responses to interview questions into statements identified as complete thoughts. These varied from a phrase to a few sentences. For each distinct statement, any particular code was applied only once. Training involved raters independently coding transcripts, then discussing and negotiating disagreements, until successfully completing 2 subsequent interviews at 80% or higher agreement. Thereafter, two raters independently coded a subset of 20% of all interviews to allow the computation of reliability statistics. The raters were in agreement when assigning codes to the same phrases 84% of the time, suggesting sufficient reliability.

Statistical analyses

We used descriptive statistics to summarize the physician characteristics of physician subjects drawn from closed-ended survey responses. We tested whether physicians’ demographics or their attitudes towards uncertainty or risk were associated with intubation using Kruskall-Wallis or Mann-Whitney tests. Further analysis using multivariate logistic regression was impractical due to collinearity of several variables. We also tested the correlation between physicians other actions during the simulation (e.g., prescribing opioids for symptoms, eliciting treatment preferences) and intubation.

We summarized the percentage of interviewees who mentioned each concept or a relationship between concepts from the expert model at least once in the entire interview. We then generated a weighted category score for each concept group. This weighting process adjusts for redundancy introduced by talkativeness and differences in model saturation (i.e., the extent to which the subject mentions all of the concepts in the mental model). (19) For example, redundancy due to talkativeness or concept repetition would artificially increase the value of the concept if weighting were not performed (see the online appendix for details). We used Kruskal Wallis procedures to test for differences in mental model weighted category scores due to the non-normal distribution of the data.

Interested investigators may contact the senior author for access to a de-identified data set.

RESULTS

Physician characteristics

The 73 participants were ages 29–70 years old and most were white men (Table 1). On average they had 13 years in medical practice. Most had primary board certification in internal medicine (56%) or emergency medicine (27%), and 40% worked in an ICU setting. Our measures of physician’s attitudes towards uncertainty were similar to the physicians involved in scale construction (17) with the exception of a higher level of discomfort when disclosing uncertainty to patients. Also similar to other samples (18), physicians were more willing to gamble when the status quo involved certain loss.

Table 1.

Physician characteristics (n=73)

Characteristic category	Characteristic	Measure
Demographics	Male Gender, n (%)	54 (74)
	Age, mean (range)	40 (29 – 70)
	Years since graduation from medical school, mean (range)	12.8 (3 – 42)
	Primary role in the hospital
	Intensivist, n (%)	29 (40)
	Emergency, n (%)	20 (33)
	Hospitalist, n (%)	24 (32)
	Home institution
	Institution A, n (%)	20 (27%)
	Institution B, n (%)	32 (44%)
	Institution C, n (%)	21 (29%)
Attitudes towards uncertainty	Anxiety from diagnostic uncertainty score*, mean (range)	19 (8–29)
	Concern about bad outcomes score*, mean (range)	12 (3–18)
	Reluctance to disclose uncertainty to patients score*, mean (range)	23 (14–29)
	Reluctance to disclose mistakes to physicians score*, mean (range)	10 (6–12)
	Risk tolerance - willingness to gamble when risk framed as gain†, n (%)	2/70 (3)
	Risk aversion – willingness to gamble when risk framed as loss†, n (%)	24/70 (34)

^*Data available for 71 of 73 study physicians. Subscales of Gerrity et al physician reactions to uncertainty subscales. Higher scores indicate greater anxiety, concern about bad outcomes, relutance to disculos uncertainty to ptients, and reluctance to disclose mistakes to physicians, respectively. The maximum ranges that can be measured by the scales are 5 to 30 for the anxiety and disclosing uncertainty to patients scales, 3 to 18 for the bad outcomes scale, and 2 to 12 for the disclosing mistakes to physicians scale.

^†Data available for 70 of 73 study physicians. Physicians are asked to choose, for a hypothetical patient with hypertension, between therapies which offer (a) a certain gain of five years’ life expectancy, or (b) a 50/50 gamble of gaining ten or zero year’s life expectancy (certain gain frame). Then physicians are asked to choose, for a hypothetical patient with cancer, between therapies which offer (a) a certain loss of five years’ life expectancy, or (b) a 50/50 gamble of losing ten or zero year’s life expectancy (certain loss frame).

Treatment decisions

Most physicians (n=44; 60%) treated the patient with comfort-focused care. The remaining physicians did not, including 7 physicians (10%) who actually intubated the patient. We defined intubation as preference-discordant treatment.

Predictors of treatment decisions

No physician demographic characteristics were associated with the decision to intubate. (Table 2) Physician reluctance to disclose uncertainty to patients t=−2.152, df=69, p=.03) and reluctance to disclose mistakes to physicians (z=−1.943, p=.05) were associated with intubation. Intubation was association with other physician simulation actions, including eliciting treatment preferences (2/7 (23%) vs 62/66 (71%), p<.001) and administering opiates for symptom management (0/7 (0%) vs 43/66 (65%), p<0.001).

Table 2.

Univariable relationship between physician characteristics and the decision to intubate

Characteristic category	Characteristic	Test statistic, df	P-value	Effect size
Demographics	Gender*	χ2 = 0.026, df = 1	>0.99	0.021
	Age†	z = −0.854	0.39	−0.102
	Years since graduation from medical school†	z = −1.126	0.26	−0.132
	Primary role in the hospital*	χ2 = 4.508, df = 1	0.09	0.251
	Home institution*	χ2 = 2.395, df = 1	0.39	0.181
Attitudes towards uncertainty	Anxiety from diagnostic uncertainty§	t = −1.498, df = 69	0.14	0.603
	Concern about bad outcomes†	z = −1.562	0.12	0.192
	Reluctance to disclose uncertainty to patients§	t = −2.152, df = 69	0.03	1.033
	Reluctance to disclose mistakes to physicians†	z = − 1.943	0.05	0.232
	Risk tolerance – willingness to gamble when risk framed as gain*	χ2 = 4.509, df = 1	0.17	0.251
	Risk aversion – willingness to gamble when risk framed as loss*	χ2 = 3.054, df = 1	0.17	0.211

df – degrees of freedom

^*Fisher’s exact test (Note: reported chi-squared test statistic value, and Fisher’s exact test p value)

^†Wilcoxon-Mann-Whitney test

^§Student’s t-test

Effect size formula

¹ϕ or Cramer’s V = sqrt(χ2/n)

²Z/sqrt(n)

³Cohen’s d = (M1 – M2)/SD_pool

Mental model outcomes

We compared the mental model concept category scores of physicians who did and did not intubate the patient. We present the unweighted frequencies of the mental model concepts of physicians who did (intubators) and did not intubate the patient (non-intubators) in Figure 2. Concepts and concept categories (categories within the concept) with greater frequencies have denser outlines. We present the comparison of weighted concept categories for intubators and non-intubators in Table 3. We provide example quotations associated with mental model concepts and concept categories in on online appendix.

Figure 2.

Summary mental model representing the thought process of physicians who intubated versus those who did not intubate a simulated patient. These models represent the unweighted average frequency of concepts mentioned by physicians who (a) intubated or (b) did not intubate the terminally ill simulation patient. The thickness of each outline or line corresponds to the frequency that the concept or link between concepts was mentioned during the interview. Heavily weighted lines represent concepts which were frequently mentioned. This does not account for repetition or talkativeness.

Table 3.

Mental model concept category mean score^*, by physician decision to intubate

Mental model concept category†	Intubated (n=7) Preference discordant	Did not intubate (n=66) Preference concordant	Test statistic§	P-value	Effect size
Physician evaluation (1)	32.1	19.2	−1.281	0.200	−0.15
Patient situation (2)	26.7	10.3	−2.111	0.035	−0.25
Physician action (3)	17.9	11.2	−1.056	0.291	−0.12
Physician perception (4)	11.2	2.3	−2.764	0.006	−0.32
Physician goal (5)	34.3	38.5	−0.753	0.452	−0.09
Patient preferences (6)	2.9	29.9	2.380	0.017	0.28
Physician explanation (8)	2.9	6.0	0.505	0.614	0.06
Physician questioning (9)	2.9	6.3	0.632	0.527	0.07
Patient history (10)	3.6	6.8	0.478	0.633	0.06
Patient understanding (11)	2.9	7.3	0.570	0.569	0.07
Surrogate (12)	0.0	13.4	1.724	0.085	0.20

^*Concept category scores are adjusted for overall statement count and frequency of linked categories per participant. The concept category scores were multiplied by a factor of 100 for convenience.

^†Numbers adjacent to the category name correspond to the same category box in Figure 1. The category “Demographics” was not mentioned during any debriefing interview and is not included in this table.

^§Wilcoxon-Mann-Whitney test

Effect size = z/sqrt(n)

Similarities

Intubators and non-intubators emphasized physician goals and physician evaluation during the debriefing of their simulation encounter. Neither group mentioned patient demographics.

Differences

We identified three statistically significant differences in mental model concept emphasis: patient situation, physician perception, and patient preferences (Table 3).

Patient’s situation (Figure 2, concept category 2)

Intubators emphasized the patient’s situation more often than non-intubators (z=−2.111, p=0.035). Moreover, intubators more often described the patient situation using terms related to “emergency”, “reversibility” and “imminence”, whereas non-intubators more often described the patient situation as “end-of-life”. For example, one intubator explained: “…if I intubate him today and let us say that he has pneumonia or he is septic… a couple of days of antibiotics can reverse all of that stuff and he can come back.”(Subject BX2) These perceptions of reversibility directly impacted treatment plans, as another intubator explained: “I do not at this point…address the end-of-life issues in this case, because my feeling in my own mind was that we had a very more tangible at least short-term remedy in this case.” (Subject B04) In contrast, one non-intubator explained: “…if they have limited time, we can do all the critical care measures, but I do not know if that is going to change or increase his lifespan. So that is why at this kind of stage of the game I sort of go by what the patient wants.” (Subject B16)

Physician’s perceptions. (Figure 2, concept category 4)

Intubators emphasized their own perceptions of the case more than non-intubators (z=−2.764, p= 0.006). Intubators’ own comfort level was commonly mentioned in association with communication actions. Discomfort with the case was mentioned in context of increased caution. As one intubator explained: “it’s a bit uncomfortable approaching that [end of life care] particularly in the presence of the patient, hence the hesitation.” (Subject B14). Additionally, intubators were also more likely to mention their own preferences for treatment in the situation.

Patient preferences (Figure 2, concept category 6)

Intubators focused less on patient preferences (z-2.380, p = 0.017). Moreover, when intubators did discuss patient preferences, they more frequently mentioned “inferred” over “explicitly stated” patient preferences. Specifically, intubators inferred consent when the patient or surrogate did not question or correct the physician’s treatment plan. As one intubator explained: “[I] basically said in very small increments ‘I am escalating care’, and would look at him and the family member [implying the question] ‘is this okay with you?’… It felt like I had no resistance …towards aggressive care, so I continued to get more aggressive and escalate care.” (Subject BX8) Another intubator explained: “She [the surrogate] knew what intubation was and when you told her that is what likely we are going to have to do, she didn’t say that’s not what he wanted or anything.” (Subject BX1)

DISCUSSION

In this study of hospital-based physicians from three US academic medical centers, we found that the mental model of physicians who provided preference-discordant treatment for a simulated critically and terminally ill elder with stable treatment preferences differed in three key ways from those who provided preference-concordant treatment. The physicians providing preference-discordant treatment focused on the reversibility or emergent nature of the situation, on inferred rather than explicitly stated patient preferences, and on their own comfort level and preferences.

A physician’s mental model may reflect their general approach to patient care. In this study, the physicians who provided preference-discordant treatment focused on the imminently unstable but reversible situation. Their descriptions are reminiscent of the Biomedical Model (20) coined by Ronald Laing. The Biomedical Model reduces the doctor-patient encounter to the discovery and treatment of physiologic derangements and is the cornerstone of Western medical education. This model is echoed in other descriptions of medical decision making such as the “fix-it model”, where treatments are offered or chosen in order to restore a patient to normal health or functional state.(21) In contrast, the physicians who provided preference-concordant care gave descriptions reminiscent of William Oslers’ Aequanimitas or Humanist Model (22,23) or the more recent Biopsychosocial Model.(24) The Biopsychosocial Model was proposed by George Engel (25) in 1977 and provides a holistic view of the patient including their social, psychological and medical influences on their health state. This model is increasingly taught in Western medical schools and may be the foundation for the mental model of physicians who focused their attention on explicit elicitation of treatment preferences by prompting a longer-range view of the patient’s situation. This long range view is echoed in an “Outcomes model” of medical decision making (26) where the focus is not only life expectancy but also quality of life. The grading of health outcomes can only be evaluated in the context of a patient’s preferences and values. The two mental models described in this study, and their resulting short versus longer-range focus, mirror the differences described in our qualitative case study of ICU decision making regarding goals of life-sustaining treatment, the definition of dying, and the interpretation of advance directives.(7)

Alternately, a physician’s mental model may not represent a fixed philosophical approach to patient care, but may change in response to certain contextual cues. In the case of a critically ill patient, physicians may default to one of the two mental models (“save the patient” versus “elicit treatment preferences for end of life care”) depending on contextual influences, such as a patient’s age or underlying diagnosis. These “cues” may trigger different behavior or thought processes after some threshold is met.(27) For example, the non-intubators in our study might have had a mental model focus more similar to intubators if the patient had been a healthy 30 year old rather than a 78 year old with end-stage cancer.

Internal, emotional cues may also influence decision making. Indeed, physicians who intubated the patient were more likely to discuss their own comfort. These physicians reported greater discomfort with disclosing uncertainty and mistakes on the validated questionnaire. Prior studies have found that physician intolerance for uncertainty is associated with increased medical spending.(30) Intrinsic differences in physician’ risk tolerance may also influence physician decision making. Prior studies have found greater risk-aversion to certain losses increase the likelihood of intubating a seriously ill patient.(18)

Whether the differences in the mental models were due to fixed philosophical differences or are potentially flexible in response to different internal or external “cues” has implications for interventions to improve shared decision making. If external, contextual cues can trigger different clinical decisions or behaviors in physicians, then directive education on the recognition of triggers and appropriate patient centered responses may be helpful. One simple method could be the employment of well-structured communication protocols.(31, 32) Protocol use in other areas of medicine is common and especially helpful for complex multi-component processes (33–35) or ensuring evidence based care during time pressured or emotionally charged situations such as the provision of ACLS for cardiac arrest.(36) However, it is also plausible that physicians’ mental models are more stable and may undermine protocol adherence, if for example, some physicians neglect elicitation of patient preference in favor of a more diagnostic approach. In this case, novel experiential or simulated learning might be used to recalibrate heuristics, for example, using compelling narratives to create an availability bias and associated emotional cues regarding harms of failing to elicit patient preferences. If philosophical differences are driving the underlying mental model structures, the associated beliefs may have been acquired during training and new education may be ineffective to create lasting behavioral change. In this case, education efforts should be focused on the training process of young physicians in training.

Although this study has important strengths, including the use of high-fidelity simulation and the involvement of providers from three academic medical centers in different geographic locations with different EOL practice patterns, it also has several weaknesses. First, the small sample limits generalizability and power to detect differences between the two groups. Second, behavior and reflection in a simulation setting may not generalize to physicians’ mental models in the “real world.” Third, the use of post-decisional mental model analysis relied on immediate recall and could be subject to hindsight bias. Generally, however, the mental models approach has proven to be predictive of future decisions.(13) Fourth, our expert model may not be exhaustive and has not been validated. We did not use the expert model to guide the interview questions, but instead used the videotaped encounter itself to elicit participant responses. In other work, the expert model provides a comparator for decisions made by subjects or laypersons.(19,37) In this study, we used initial observations of the encounter to build the expert model, with input from experts in communication, decision sciences, critical care, and palliative care prior to coding the interview transcripts. Nevertheless, our physician sample discussed concepts outside of the expert model only 10% of the time, and the majority of these unmatched statements were related to the believability of the simulation or patient rather than the decisions being made. We also appreciate that our interviews were not exhaustive and may not have uncovered a complete set of related concepts or constructs. Finally, although we achieved satisfactory levels of agreement between coders, semi-structured interview coding is vulnerable to measurement error.

In conclusion, hospital-based physicians in our study who intubated the patient against his preferences were less likely to elicit the patient’s treatment preferences, focusing instead on short-term treatment goals, inferred preferences, and their own comfort and preferences for the case. Identifying these mental model differences and their associated behaviors (e.g., elicitation of preferences) may offer targets for interventions to promote preference-concordant treatment for seriously ill patients.

APPENDIX

Statistical methods

To generate the weighted category score for each concept we include mentions of all the concepts in the category and the associated links to the category. For example, to calculate a category score for “Patient Understanding” (node 11 in Figure 1), we included mentions of the sub-concepts “see reality” and “competent” as well as expressed relationships between “Patient Understanding” and “Patient History”. The relationship between two concepts in our mental model is referred to as a link and is represented by a line connecting two concept categories in Figure 1. Each link has a calculated ratio (termed the relative frequency) consisting of a numerator representing the number of mentions of the link and a denominator generated from a summation of the all the mentioned links and the concept categories as a binary. The numerator represents the connectedness of a concept category. The denominator represents both the extent to which the subject uses the expert model concepts (model saturation) and the connectedness of the categories to each other. A higher number could reflect either greater use of the expert model or a highly connected but focused use of the model. A lower number reflects a more focused use of the model (i.e., a more focused thought process). Each link related to the concept category in question is then summed to create the category score which represents an emphasis of the concept category during the interview. To demonstrate this calculation in detail, we have included the raw data and resulting calculation for the category concept “Patient Preferences” for Subject A14 in Appendix Figure 2.

Appendix Table 1.

Debriefing Interview Questions

Section 1:	Physician goals when entering the simulation room
	“What were your goals when you entered the room?”
Section 2:	Thought process at two minutes intervals (Stopping video playback every 2 minutes and/or at select scripted patient/surrogate statements)
	“What is going on? / What are you thinking?”
	“Did that change your assessment of the situation?”
Section 3:	Intention to intubate patient
	“Was there anything about this patient that made you think that you might need to intubate him?”
	“What made you think that?”
Section 4:	Difference between simulated patient and those with similar prognosis
	“Was there anything about this particular patient that might be different from patients with a similar prognosis?”
	“What made you think that?”
Section 5:	Patient preference for intubation
	“Was there anything about this patient that made you think he might not want to be intubated?”
	For those who stated that they thought the patient might not want to be intubated: “At what point did you first think he might not want to be intubated?”

Appendix Table 2.

Descriptive interview quotes demonstrating the application of concept categories and sub-concept codes

Concept Category	Concept Category Item	Example text or explanation
Physician evaluation (1)†	Reading chart examining patient looking at patient looking at surrogate looking at monitors talking to nurse	Physician would report they were performing the coded behavior
Patient situation (2)	End-of-life	Subject C11: “…he is, you know end-stage, and I am trying to figure out do they sort of want everything done or they don’t want anything done, and if they really understand, you know this is probably terminal. I mean definitely terminal, but how they want to sort of proceed from now on.”
	Emergency	Subject BX3: “…that this patient seems to be in a crisis, and the crisis could either be averted by going all the way or you allow the patient to have the comfort measures and die peacefully.”
	Reversibility	Subject B14: ”He’s suffering from an irreversible deterioration of his already chronic irreversible condition.”
	Imminence	Subject C10: “…I did not address end of life or what their resuscitation preferences were because I was in an acute situation and that is generally not the time to have that discussion. Those discussions should be had when things are relatively stable if possible and the other thing is that, the other reason that may have influenced me not going there is this patient looked hale and hearty really except for his breathing trouble.”
Physician action (3)	Admit to ICU No ICU admission Curative therapies Palliative therapies Activate emergency response system Intubated Do not intubate	Physician would report they were performing the coded behavior
Physician perception (4)	Comfort level	Interviewer: [asks if they have discussed end of life care] Subject B14: “It is always difficult to approach…It’s not always difficult, well, it is not always difficult, but it’s a bit uncomfortable approaching that particularly in the presence of the patient, hence the hesitation.”
	Confidence	Subject B29 “…and then again I am worried, am I moving too quickly down this road? That is what I am thinking.”
	Own preferences	Subject BX6: “…mainly his problem was acute pneumonic process on top of cancer. Had he been my father I would have intubated him and taken him to the ICU.”
Physician goal (5)	Diagnosis	Subject BX2: “Right now I am trying to see what can be the cause of the condition.”
	Curative treatment	Subject BX6: “So if I could fix the hypoxemia by giving him oxygen we can then give him volume, then I can raise his blood pressure…”
	Palliative treatment	SubjectB37: “So in my mind the next thing is to control his pain.”
	Obtain code status	Subject BX4: “Yeah, um basically just to assess him to see his level of stability and then also to find out what their desires were with regard to end-of-life care and the ultimate measure to which he wanted to be resuscitated because it seemed like he was unstable.”
Patient preferences (6)	Explicitly stated	Subject BX6: “So then now I am discussing with them clearly at this point they do not want intubation, they do not want CPR.”
	Inferred	Subject B14: “Even that act of coming from the nursing home. Really tells you that they want something done.”
Physician explanation (8)	Goal implications	Subject BX13: “ I think at this point I just want to just be clear with the wife in terms of the things that we were doing for him, making him comfortable and that they may change how is able to interact with her as well so that she was clear in terms of, even though we were going to be focusing on his comfort that there may be some side effects, I guess to a certain degree or some other consequences for that that would change other parameters of how he was able to interact with her.”
	Explain options	Subject B14: “But I can intubate you, I can start pressors, and I can stop them. We can always, and one thing I always make clear to my patients is that all that we are doing we can stop it at any time.”
	Empathic reactions	Subject B04: “Sometimes they [surrogates] end up with guilt feelings or did we do the right thing or not do the right thing and I think just a positive reinforcement that they decided that it is, you know, that is what we decided and the doctor to tell them that was the right decision, is much more reassuring than somebody says “Oh, you know what, you should not have done that and your decision was not the right one.”
	Explain situation	Subject C19: “…clarifying that basically he is going to die. Making sure that she understood this was imminent and not going to happen necessarily a few days from now…”
	Recommendation	Subject B54: “I think that is where when I was telling them, you know, maybe we should just really focus on the comfort measures now.”
Physician questioning (9)	Eliciting goals	Subject BX3: “And so what I was trying to get at was, you know, he will die of his cancer and then so if he has got limited time, how does he want to spend that time, you know, with his family”
	Treatment preferences	Subject BX2: “So, I’m trying to flush out exactly what the discussions had been between the two of them about what the preferences would be since he’s, you know, I decided in this instance that having a discussion with him was going to be too difficult and that he’d probably be too hypercarbic and just too short of breath to carry on a meaningful conversation about something this complicated, and so I focused on his immediate surrogate and to see what kind of conversations they had had.”
	Asking about support	Subject BX6: “In my experience, I find that the family is very grateful when I start getting the priest and social services involved.”
	Eliciting questions	Interviewer: “Now you asked her if she had any questions?” Subject A14: “Yes…Well, you know, we use a lot of jargon and so if there are questions about what we think might be wrong or what are the things that we’re going to do. People sometimes have questions about whether we think they’re going to live.”
Patient history (10)	From chart	Subject BCX5: “The chart did tip me that the CAT scan was indicative of lymphangitic spread of the tumor in multiple areas of the lung.”
	From questioning	Subject A21: And the reason I asked him if he has been in the ICU before and been intubated is patients who have been through that experience they are more likely to be willing to do it again and if that experience was a very negative one and they have made a very clear decision against it, they make it very clear to me right then and there as well.”
Patient understanding (11)	See reality	Subject BX5: “My overall sense was that they did not have as good a handle on what was going on with him, his underlying problems, as I would hope.”
	Competent	Subject C14: “I was just assessing his mental status and whether, again, just further gathering evidence that what his clinical status was. Was he hypoxic to the point of being altered, he was not but it does not really change the fact that his vital signs were terrible and it makes it so that I can feel like I can talk to him and have a reasonable conversation, you know, like he knows, you know, is capable of understanding what I am talking about.”
Surrogate (12)	Trustworthy	Subject BX6: “I don’t put as much store in a sister as a wife, I am not really sure why, as I have been married for 28 years, but usually because siblings can often be detached from their sibling, but where as though there can be secondary gain or loss, basically a spouse tend to have a much more understanding this early of their spouse.”
	Knows patient preferences	Subject C04: “… you have either the patient and the family have not thought about it or even though they have thought about it, they have not talked about it. They [actors in scenario] had been so clear about their wishes and been so on the same page.”
	See reality	Subject B11: “Well, again I do not know. If he has had cancer for 5 years, it would be different than if he just had it for 3 weeks because their level of understanding and acceptance are different between the two cases”

^†The adjacent number corresponds to the identification number in Figure 1.

Appendix Figure 1.

Chronologic use of mental model concepts during simulation debriefing interview: an illustrative sample of physicians who intubated and did not intubate the simulated patient. Each concept mentioned during the debriefing interview is noted by a box with a number corresponding to a concept number in Figure 1, in the order which is was mentioned during the debriefing interview. The colors represent post analysis grouping of ideas and processes. A color key is provided in the corresponding table.

Appendix Figure 2.

Example calculation for the concept category “Patient Preferences” for Subject A14. Raw data is listed on the left for the coded interview for Subject A14. The box and line figure demonstrates the category concept of interest in the calculation (“Patient Preferences”) and the associated category concepts connected by ideological relationships or links. The weight of the outlines or links represents the raw frequencies coded during this interview. Heavier outlines or links were mentioned more frequently.

A visual inspection of the raw data shows this physician used most of the expert model concept categories (11 out of 12), but their use of concept linking was focused (only using 5 out of 17 possible) on the concept category “Patient Preferences” (4 of the 5 mentioned links). Calculation of the relative frequencies of the links tempers the repeated mentions of links to physician goal and surrogate. The category score for “Patient preferences” is a summation of the five relative frequencies of the associated links to “Physician Questioning”, “Physician Explanation”, “Patient Understanding”, “Physician Goals”, and “Surrogate”.