A randomized trial of the effect of patient race on physician ICU and life-sustaining treatment decisions for an acutely unstable elder with end-stage cancer

Abstract

Objective

To test whether hospital-based physicians made different ICU and LST decisions for otherwise identical African American (AA) and European American (EA) patients with end-stage cancer and life-threatening hypoxia.

Design

We conducted a randomized trial of the relationship between patient race and physician treatment decisions using high-fidelity simulation. We counterbalanced the effects of race and case by randomly alternating their order using a table of random permutations. Physicians completed two simulation encounters with AA and EA patient simulators diagnosed with prognostically-identical end-stage gastric or pancreatic cancer and life-threatening hypoxia and hypotension, followed by a self-administered survey of beliefs regarding treatment preferences by race. We conducted within-subjects analysis of each physician’s matched-pair simulation encounters, adjusting for order and case effects, and between-subjects analysis of physicians’ first encounter, adjusting for case.

Setting

Peter M. Winter Institute for Simulation Education and Research at the University of Pittsburgh.

Subjects

33 hospital-based attending physicians, including 12 emergency physicians, 8 hospitalists, and 13 intensivists from Allegheny County, Pennsylvania.

Intervention

Race of patient simulator.

Measurements and Main Results

Measurements included physician treatment decisions recorded during the simulation and documented in the chart and beliefs about treatment preference by race. When faced with an AA versus a EA patient, physicians did not differ in their elicitation of intubation preferences (within-subject comparison (WS): 28/32 (88%) vs. 28/32 (88%), p=0.589; between-subject comparison (BS): 13/17 (87%) vs. 13/17 (76%), p=0.460), ICU admission (WS: 14/32 (44%) vs. 12/32 (38%), p=0.481; BS: 8/15 (53%) vs. 7/17 (41%), p=0.456), intubation (WS: 5/32 (16%) vs. 4/32 (13%), p=0.567; B: 1/15 (7%) vs. 4/17 (24%), p=0.215) or initiation of comfort measures only (WS: 16/32 (50%) vs. 19/32 (59%), p=0.681; BS: 6/15 (40%) vs. 8/17 (47%), p-0.679). Physicians believed that an AA patient with end-stage cancer was more likely than a similar EA patient to prefer potentially-life prolonging chemotherapy over treatment focused on palliation (67% vs. 64%, z= − 1.79, p=.07) and to want mechanical ventilation for 1 weeks’ life extension (43% vs. 34%, z= − 2.93, p=.003), and less likely to want a DNR order if hospitalized (51% vs. 60%, z= 3.03, p=.003).

Conclusions

In this exploratory study, hospital-based physicians did not make different treatment decisions for otherwise identical terminally ill AA and EA elders despite believing that AA patients are more likely to prefer intensive, life-sustaining treatment, and they grossly overestimated the preference for intensive treatment for both races.

While African American patients receive less preventive and early curative cancer care than European American patients, once end-stage, they receive more intensive and expensive treatment before death. For example, African American patients with metastatic cancer are more likely than similar European American patients to be admitted to an ICU and to receive life-sustaining treatments (LSTs) such as mechanical ventilation and hemodialysis before death, even after adjusting for African Americans’ use of higher intensity hospitals(1, 2, 3 ). This finding has been attributed to differences in treatment preferences by race/ethnicity. Indeed, local and national surveys confirm that African Americans are more likely to prefer aggressive life-sustaining treatment than European Americans(4–7). Yet the majority of African American elders, like European American elders, do not prefer such treatment(8).

Like other statistical phenomena, the higher relative rate among African Americans for end-of-life ICU and LST use, compared to European Americans, could lead physicians to assume larger differences in base rates than actually exist. This may lead to cognitive biases in decision-making, particularly under conditions of time pressure and uncertainty(9–14). We hypothesized that some of the observed racial differences in end-of-life ICU and LST use could be attributable to cognitive biases that lead to different treatment decisions by race among hospital-based physicians.

The purpose of this exploratory study was to test whether hospital-based physicians made different ICU and LST decisions for otherwise identical African American and European American patients with end-stage cancer and life-threatening hypoxia and whether differences, if they exist, were due to race-based beliefs about treatment preferences.

MATERIALS AND METHODS

We conducted a randomized factorial trial of the relationship between patient race and physician treatment decisions using high-fidelity simulation. We counterbalanced the effects of race and case by randomly alternating their order using a table of random permutations. The simulation has been validated to mimic the time-pressured decisions regarding ICU admission and intubation faced by hospital-based physicians for an acutely unstable terminally ill elder.(15) We reduced race to the phenotypic expression of a patient’s heritage (e.g., predominantly skin color); the patient simulators did not differ in preferences, socioeconomic status, or verbal and nonverbal behaviors.

Subject Recruitment

We used a combination of probability sampling and convenience sampling to recruit emergency physicians, hospitalists, and intensivists (see Appendix).

Simulation

Each physician subject participated in one simulated clinical encounter with an AA patient and one with an EA patient at the University of Pittsburgh Peter M. Winter Institute for Simulation Education and Research (WISER). The encounters were separated by a distracting task lasting 10 minutes. Following the approach we developed previously(15), the clinical encounters took place on a hospital set (emergency department set for emergency physicians, ward set for hospitalists and intensivists) by trained patient simulators playing the patient and his wife or sister. We provided clinical data in an electronic medical record-based chart and bedside vital signs monitor. A real nurse executed the physician orders at the bedside and an investigator running the simulation answered any unanticipated physician diagnostic or therapeutic requests through an intercom speaker from the observation room.

We trained 4 African American and 4 European American actors to use a combination of scripted answers to anticipated questions and response principles (See Appendix 1). We designed the response principles such that the physicians, not the actors, would lead discussions about decision making, and practiced using 12 hours of role-play, including formal dress rehearsal with 3 volunteer physicians. We used multiple actors in equal frequencies to minimize potential for an actor effect (Figures 1 and 2).

Figure 1. Actors portraying the role of the patient.

Two black and two white patient simulators portrayed the roles of Mr. Jenkins, a 78 year-old man with metastatic gastric cancer and a chief complaint of dyspnea and Mr. Thomas, a 76 year-old man with metastatic pancreatic cancer and a chief complaint of abdominal pain. The vital signs tracings on a bedside monitor next to the patient were identical, regardless of case.

Figure 2. Actors portraying the role of the caregiver/surrogate.

Two black and two white patient simulators portrayed the roles of Emma Jenkins, Mr. Jenkins’ wife caregiver/surrogate, or Ruth Thomas, Mr. Thomas’ sister caregiver/surrogate.

The two cases that we randomly alternated with race differed in non-informative social details and were considered clinically to be prognostically identical. The clinical conditions had parallel differential diagnoses: likely progression of cancer with a (potentially reversible) infectious process as a possible coincident or alternate cause. Mr. Jenkins, accompanied by his caregiver/surrogate wife, was a 78-year old retired Catholic English teacher with metastatic gastric cancer admitted from skilled nursing for shortness of breath; a spiral CT ruled out pulmonary embolism and suggested lymphangitic spread of the tumor as the most likely cause of the shortness of breath, although the admitting team had begun empiric antibiotic therapy for pneumonia. His daughter lives in Hawaii with her family, and was last home during her father’s recent complicated admission for gastrectomy. Mr. Thomas, accompanied by his caregiver/surrogate sister, was a 76-year old retired Protestant bank manager with metastatic pancreatic cancer admitted from skilled nursing for abdominal pain; an abdominal CT and laboratory values suggested biliary obstruction due to liver metastasis as the most likely cause of the abdominal pain, although the admitting team had begun empiric antibiotic therapy for cholangitis. His son is deployed in Iraq, and was provided leave for his father’s recent complicated admission for pancreaticoduodenectomy. We asked subjects to imagine a nurse had summoned them to the bedside, some 8–12 hours after initial presentation, to assess the patient’s identical deteriorating vital signs: tachypnea, tachycardia, hypotension, and hypoxia.

In both cases, the chart contained no prognostic or treatment information from the oncologist or an advance care plan. If probed during the course of the simulation, the patient and his caregiver would reveal they know the tumor to be widely metastatic, that the treating oncologist said he was too week to receive chemotherapy and he might die within 6 months, and that they prefer not to be admitted to the ICU or to receive mechanical ventilation or cardiopulmonary resuscitation.

Data Collection

Two physician investigators (AB and DM) independently ran one of each subjects’ two simulation encounters. We recorded treatment decisions made during the simulation encounter on a standardized form. After each encounter, the physician subject wrote a brief chart note and orders using a web-based survey in an adjacent conference room. We used data from treatment decisions observed during the encounter and recorded in the chart note and orders to create 9 non-mutually-exclusive individual treatment decisions (provision of opiates for symptom palliation during the simulation encounter; a trial of NIMV during the simulation; elicitation of intubation preferences; documentation of a do not intubate (DNI) order in the chart; admission to the ICU; respiratory intubation and mechanical ventilation; initiation of comfort measures only (CMO); palliative intent; and palliative care consultation).

We also collected physician subject demographic, training, and employment information, and their diagnosis, prognosis, and perceptions of the treatment goals for each case. After simulation and interview completion, we collected subjects’ beliefs about treatment preferences among black and white patients with metastatic pancreatic cancer. Finally, we asked subjects whether they had read the investigators’ previously published pilot feasibility study and invited them to venture a guess regarding the hypothesis under testing.

Analyses

We descriptively summarized the characteristics of the physician subjects and their prognostic assessments of the two cases. We assessed the bivariate relationship between predictors (case, race) and each of the 9 dichotomous decisions using McNemar’s test for within-subject comparison (matched African American vs. European American patient pairs in which each physician served as their own control) and the Fisher exact test for between-subject comparison (limiting the African American vs. European American patient comparison to each physician’s first encounter to eliminate contamination effects). We used logistic regression with subject-level random effects to assess the multivariable relationship between case, race, and order and treatment decisions for the within-subject comparisons, and logistic regression to assess the multivariable relationship between case and race and treatment decisions for the between-subject analyses. We used the Wilcoxon signed rank test to compare physicians’ beliefs about preference, by race, for chemotherapy, mechanical ventilation, and a DNR order. This was an exploratory study without an a priori estimate of effect size (See Discussion for post-hoc power calculation). Any associations at the p≤0.1 level were considered suggestive of a positive association. We used Stata 10.0 (College Station, TX) for all analyses.

Human Subjects

The protocol was approved by the University of Pittsburgh Institutional Review Board, which required deliberate omission of the specific study purpose from the consent form (race-based differences in end-of-life decision making and communication). Subjects completed written informed consent with the understanding that they were participating in a study of treatment decisions for critically ill patients made by hospital-based providers who do not have an established relationship with the patient. Subjects received $200 for their 2-hour participation.

RESULTS

Subjects

We recruited 15 subjects through medical society and hospital lists and 18 through professional contacts (See Appendix 2). We report the characteristics of the 33 physician subjects in Table 1. None of the study subjects read the manuscript describing our pilot work before participation. Three (9%) guessed that the hypothesis we were testing was about patient race.

Table 1.

Characteristics of the study physicians (N=33 subjects)

Characteristic	Mean or frequency
Age, μ (range), y	42 (29–70)
Years since medical school graduation, μ (range)	16 (3–42)
Months on service annually, μ (range)	8 (2–12)
Female, n/N (%)	5/33 (15%)
Race, n/N (%)
White	18/32 (56%)
Black	2/32 (6%)
Asian	10/32 (31%)
Hispanic	2/32 (6%)
Emergency physician, n/N (%)	12/33 (36%)
Hospitalist, n/N (%)	8/33 (24%)
Intensivist, n/N (%)	13/33 (39%)
Major teaching hospital	21/33 (64%)
Minor teaching hospital	12/33 (36%)

We removed one subject’s treatment decision data for both of his encounters because of errors in actor responses that may have biased the doctor’s treatment decision. An independent non-study physician who watched the encounter affirmed this judgment. This reduced our sample of subjects for analysis of treatment decisions from 33 to 32.

Simulation Treatment Decisions

In crude and adjusted within-subject and between-subject comparisons of treatment decisions for the African American versus European American patient, there were no differences in provision of an opiate during the simulation, a trial of noninvasive mechanical ventilation (NIMV), elicitation of intubation preferences, chart documentation of preferences, ICU admission, intubation, comfort measures only, palliative intent, and palliative care consultation (Table 2).

Table 2.

Physician treatment decisions for black and white patients, using within-subject and between-subject comparisons (N=32 subjects)

	Within-subject analysis				Between-subject analysis
	Black	White	Crude p-value1	Adjusted p-value2	Black	White	Crude p-value3	Adjusted p-value4
Opiate for symptoms	19/32 (59%)	20/32 (65%)	.796	.686	9/15 (60%)	9/17 (53%)	.735	.686
Trial of non-invasive MV	14/32 (44%)	15/32 (47%)	.763	.716	7/15 (47%)	8/17 (47%)	>..99	.974
Elicited preferences	28/32 (88%)	28/32 (88%)	>.99	.826	13/15 (87%)	13/17 (76%)	.659	.460
Documented preferences	25/32 (78%)	26/32 (81%)	.706	.511	12/15 (80%)	11/17 (65%)	.444	.330
Admitted to ICU	14/32 (44%)	12/32 (38%)	.527	.481	8/15 (53%)	7/17 (41%)	.723	.456
Intubated	5/32 (16%)	4/32 (13%)	.655	.567	1/15 (7%)	4/17 (24%)	.338	.215
Comfort measures only	16/32 (50%)	17/32 (53%)	.763	.681	6/15 (40%)	8/17 (47%)	.735	.679
Palliative intent	16/32 (50%)	19/32 (59%)	.405	.373	7/15 (47%)	9/17 (53%)	>.99	.704
Consulted palliative care	9/32 (28%)	9/32 (28%)	>.99	.734	5/15 (33%)	5/17 (29%)	>.99	.812

¹McNemar’s test

²Logistic regression with subject-level random effects, adjusted for case (pancreatic or gastric cancer) and order (first or second)

³Fisher’s exact test

⁴Logistic regression, adjusted for case (pancreatic or gastric cancer)

In crude and adjusted within-subject and between-subject comparisons of treatment decisions for the gastric cancer vs. pancreatic cancer case, physicians were more likely to provide an opiate for the pancreatic cancer patient with pain and more likely to attempt a trial of NIMV for the gastric cancer with dyspnea (Table 3). In the between-subject comparisons only, physicians were more likely to admit the pancreatic cancer patient to the ICU and less likely to treat with palliative intent (Table 3).

Table 3.

Physician treatment decisions for gastric and pancreatic cancer cases, using within-subject and between-subject comparisons (N=32 subjects)

	Within-subject analysis				Between-subject analysis
	Gastric	Pancreatic	Crude p-value5	Adjusted p-value6	Gastric	Pancreatic	Crude p-value7	Adjusted p-value8
Opiate for symptoms	15/32 (47%)	24/32 (75%)	.020	.036	9/17 (53%)	9/15 (60%)	.735	.686
Trial of non-invasive MV	20/32 (63%)	9/32 (28%)	< .001	.014	11/17 (65%)	4/15 (27%)	.042	.037
Elicited preferences	29/32 (91%)	27/32 (84%)	.317	.261	15/17 (88%)	11/15 (73%)	.383	.289
Documented preferences	27/32 (84%)	24/32 (75%)	.257	.281	14/17 (82%)	9/15 (60%)	.243	.165
Admitted to ICU	12/32 (38%)	14/32 (44%)	.527	.504	5/17 (29%)	10/15 (67%)	.074	.039
Intubated	4/32 (13%)	5/32 (16%)	.655	.670	2/17 (12%)	3/15 (20%)	.645	.518
Comfort measures only	17/32 (53%)	16/32 (50%)	.763	.684	9/17 (53%)	5/15 (33%)	.308	.266
Palliative intent	19/32 (59%)	16/32 (50%)	.405	.378	11/17 (65%)	5/15 (33%)	.156	.081
Consulted palliative care	11/32 (34%)	7/32 (22%)	.103	.156	6/17 (35%)	4/15 (27%)	.712	.601

⁵McNemar’s test

⁶Logistic regression with subject-level random effects, adjusted for race and order (first or second)

⁷Fisher’s exact test

⁸Logistic regression, adjusted for case race

Beliefs about Preferences by Race

Based upon survey self-report, physician subjects believed that a black patient with metastatic pancreatic cancer was more likely than a similar white patient to prefer potentially-life prolonging chemotherapy over treatment focused on palliation (67% vs. 64%, z= − 1.79, p=.07) and to want mechanical ventilation for 1 weeks’ life extension (43% vs. 34%, z= − 2.93, p=.003), and less likely to want a DNR order if hospitalized (51% vs. 60%, z= 3.03, p=.003).

Perceptions about Case

More physician subjects attributed the pancreatic cancer patient’s deterioration to infection (16; 50%) than to cancer (15; 47%); this pattern was reversed for the gastric cancer patient, with 21 (66%) attributing the deterioration to the cancer and 7 (22%) to infection. Nonetheless, prognostic estimates for the pancreatic and gastric cancer cases were similar: 21 (66%) and 25 (78%) predicted < 10% 3-month survival; 12 (38%) and 13 (41%) predicted < 10% survival to discharge with no treatment limitation.

DISCUSSION

In this exploratory randomized factorial simulation trial we demonstrated significant variation in the use of the ICU, life-sustaining treatment, and palliative care for a critically ill elder with end-stage cancer. This variation, however, did not appear to be influenced by the race of the patient. This finding was particularly notable given physician subjects’ self-reported beliefs that African American patients with metastatic cancer prefer more life-prolonging treatment than European American patients.

This is the first study to use high-fidelity simulation to explore the relationship between patient race and physicians’ end-of-life decision making. Previous studies using video vignettes alone(16) and in conjunction with implicit association tests(17) in the area of cardiovascular disease reveal that physician treatment decisions for otherwise identical African American and European American patients differ and that some, but not all, of the difference is explained by physician priors about risk and benefit by race. In another study of physician-reported diagnoses after patient consultation, differences in plausible priors explained racial differences in diagnosis of hypertension and diabetes(11). Like the physicians in those studies, our subject physicians held prior race-based beliefs. Indeed, their beliefs about blacks’ higher likelihood for preferring life-prolonging drugs, mechanical ventilation, or a do-not-resuscitate orders corresponded with reports in the literature, although they overestimated by several-fold the absolute probabilities of preferences for intensive treatments for both races (8, 18–20). We did not have the occasion to explore whether these priors mediated race-based differences in treatment because we did not find treatment differences.

There are several plausible explanations for lack of race-based differences in physician treatment decisions. First, the physician’s priors may not distort their decision making. This could be because, unlike the diagnostic and treatment decisions explored in these prior studies, in which the “true” state may not be directly observed without further (delayed) diagnostic testing, the preferences for life-sustaining treatment in our simulation could “tested” right away by asking the patient a question. Although in 8 encounters the subject physician failed to ask about intubation preferences (and so intubated the patient), most physicians did ask, and there was no difference in the elicitation of intubation preferences by patient race.

Second, our findings might not reflect “real world” race-based differences in physician decision making because we oversimplified the construct of race. That is, by isolating race to appearance and standardizing away social and cultural variables that sometimes confound race, such as socioeconomic status, religiosity, trust, and level of advance care planning, we may have stripped away the real stimuli that moderate race-based differences in physician behavior. For example, it is possible that physicians respond differently to African American and European American mistrust, and that this complexity might give rise to treatment differences by race. Yet we argue that ours is the necessary first step in deconstructing this complex social question as others have sought to do by teasing out confounders in observational samples (21, 22). Given the feasibility of simulation demonstrated by the current study, our future studies could experimentally manipulate sociocultural confounders to explore their impact on physician decision making.

Third, and perhaps most importantly, our study was underpowered. There were no prospective estimates of expected rates of ICU admission and intubation among African American and European American elders with end-stage cancer and acute respiratory failure to use as effect size estimates, let alone estimates of provider-attributable differences. Assuming a 10% provider-attributable difference to be “clinically important,” the information on concordance from our sample of 32 evaluable subjects suggests that 124 and 301 providers, respectively, would have been required to find a clinically important difference in intubation and ICU admission using a within-subjects design. Nonetheless, we can place the low power for a race finding in the context of the unexpected finding that the particular type of cancer (gastric vs. pancreatic) and presenting symptoms of infiltration (lung vs. liver) – what we believed were non-informative aspects of the case – affected physician decision making. Specifically, if there are race-based differences in decision-making, they must be smaller than those caused by case for them not to have been identified in the current study. The cause of the differences in decision making by case are unknown. We conjecture that the pancreatic cancer patient’s chief complaint of pain prompted more opiate use during the simulation (even though opiates are also effective for dyspnea) and that the gastric cancer patient’s chief complaint of dyspnea prompted more use of NIMV (even though the pancreatic cancer patient’s respiratory status was identical). We also conjecture that the physicians were more likely to admit the pancreatic cancer patient to the ICU and less likely to treat with palliative intent because they more commonly perceived that the proximate cause of the pancreatic cancer patient’s compromise was infection (rather than cancer). This may have influenced their perception of the proximate condition’s “reversibility,” even though the physicians’ hospital- and 3-month prognostic estimates for the two cases were identical.

Our study has several strengths. By using simulation we were able to study a decision context (ICU triage) that is difficult to study in the “real world” because it is unscheduled and time-pressured. Simulation also allowed us to experimentally manipulate the independent variable – race. Despite the artificial setting and experimental manipulations, physicians suspended disbelief and seldom guessed that the study was about race. Nevertheless, we cannot rule-out a Hawthorne effect. It is possible that physicians’ behavior in the simulation may not reflect their actual practice(23). By being on their “best behavior” during observation, they may have suppressed any race-based differences in behavior even if they did not know that the study was about race. Yet we studied a heterogeneous group of physicians from major and minor teaching hospitals. Also, we explored our findings using two analytic designs: within- and between-subject comparisons. Within-subject comparisons, by allowing each subject to serve as his or her own control, make it easier to attribute differences in responses to the experimental condition, rather than to potentially confounding individual subject characteristics, but are subject to carryover effects such as fatigue (increasing error variance), anchoring (bias toward the null), contrast effects (bias toward finding difference) and demand effects (decreasing validity). Between-subject comparisons avoid carryover effects but may require unfeasibly large sample sizes. Although ours was an exploratory study without pre-specified effect sizes, neither analytic strategy suggested an independent effect of race, and we did find evidence of potential cross-case contamination, including a carryover effect.

There are other threats to generalizability beyond our isolation of the effect of race based upon appearance alone. First, we designed the clinical scenario to be relatively simple. We removed the diagnostic challenge with a fully worked-up patient. We limited the decision makers to two parties – the patient and his surrogate – who both understood the underlying cancer prognosis and had previously discussed life-sustaining treatment preferences. Such conditions may be uncommon. If the diagnosis were more opaque, there were more decision makers involved, less insight into prognosis or more ambivalence about treatment preferences, we likely would have observed more intensive treatment and perhaps greater variation. Second, we recruited subject physicians from one region and only 2 of them were black. Although the proportion of Pittsburgh’s patient population that is black is similar to the nation overall (13%), the existence race-based differences in physician end-of-life decision making may vary across regions.

In conclusion, we found that patient race alone did not appear to influence treatment decisions by hospital-based physicians in response to life-threatening hypoxia in an elderly patient with end-stage cancer and stable preferences to avoid intensive care. Future work should explore the interaction of race, socioeconomic, and cultural variables in end-of-life decision making across varying preference conditions in larger, more regionally-diverse physician samples.

SIMULATION DETAILS

Case development and actor training

A multi-disciplinary team of physicians, including specialists in general internal medicine, palliative care, oncology, critical care, and emergency medicine contributed to the development of a second clinical case that was similar in short- and long-term prognosis to the previously validated case. We trained 4 white and 4 black actors to use a combination of scripted answers to anticipated questions and response principles (See Appendix). We designed the response principles such that the physicians, not the actors, would lead discussions about decision making, and practiced using 12 hours of role-play, including formal dress rehearsal with 3 volunteer physicians. We used multiple actors in equal frequencies to minimize potential for an actor effect (Figures 1 and 2). To promote simulation consistency, the actors observed one another during the simulation dress rehearsals, debriefed with an investigator and patient simulation trainer/director after each encounter, and periodically reviewed videos of each others’ simulations with the investigators and the trainer/director.

Clinical scenarios

Mr. Jenkins, accompanied by his caregiver/surrogate wife, was a 78-year old man with metastatic gastric cancer admitted from skilled nursing for shortness of breath; a spiral CT ruled out pulmonary embolism and suggested lymphangitic spread of the tumor as the most likely cause of the shortness of breath, although the admitting team had begun empiric antibiotic therapy for pneumonia. Mr. Thomas, accompanied by his caregiver/surrogate sister, was a 76-year old man with metastatic pancreatic cancer admitted from skilled nursing for abdominal pain; an abdominal CT and laboratory values suggested biliary obstruction due to liver metastasis as the most likely cause of the abdominal pain, although the admitting team had begun empiric antibiotic therapy for cholangitis. These cases have parallel differential diagnoses: likely progression of cancer with a (potentially reversible) infectious process as a possible coincident or alternate cause. A medical oncologist reviewed all case materials for verisimilitude and prognostic consistency.

In both cases, we asked subjects to imagine a nurse had summoned them to the bedside, some 8–12 hours after initial presentation, to assess the patient’s identical deteriorating vital signs: tachypnea, tachycardia, hypotension, and hypoxia. Prior to entering the room, subjects received an electronic medical record-based chart that included a discharge summary from a recent complicated 2-month hospital stay for resection of the patient’s cancer requiring prolonged mechanical ventilation. In both cases, the chart contained no prognostic or treatment information from the oncologist or an advance care plan. However, if probed during the course of the simulation, the patient and his caregiver would explain that they saw the oncologist one week ago, that the tumor was widely metastatic, that there was no additional chemotherapy or radiation to offer, and that he “might not make it until Christmas” (6 months hence). Also if directly probed, the patient and his caregiver would reveal their preference for avoiding re-admission to the ICU, mechanical ventilation, and cardiopulmonary resuscitation. The caregiver at the bedside held the durable power of attorney for health care, and an adult child living remotely was unavailable to participate in bedside decision making. The bedside nurse implemented physician orders and answered questions, but did not otherwise make any requests or treatment recommendations. Any additional diagnostic tests, if ordered, would be uninformative. All interventions, including non-invasive mechanical ventilation (NIMV), fluid boluses, and vasopressors, were ineffective in reversing the deteriorating vital signs. The simulation ended when the physician articulated a plan and left the room to write a chart note and orders.

APPENDIX 1 – Case description, script and response principles for actors

Barnato et al. “Physician ICU admission, intubation, and palliation decisions for black and white elders with end-stage cancer: A simulation study.”

Purpose: The purpose of the simulation is to observe how physicians interact with the patient and his surrogate decision maker, and whether there is a difference in the interaction depending on whether they are African American or Caucasian. We will analyze the physicians’ communication style and treatment decisions.

Decisions: The 3 main treatment decisions that we will document are:

• Does the doctor admit the patient to the ICU? (yes/no)

• Does the doctor initiation palliation for the patient’s dyspnea (shortness of breath) and anxiety? (yes/no). This may be direct treatment, such as starting a morphine drip, or indirect, in the form of a palliative care consultation.

• Does the doctor document the patient’s code status in the chart? (yes/no). This is an indication of whether the patient would want to be intubated or have CPR if their heart should stop.

Standardization: The key to testing our hypothesis is having the white and African American patient and surrogate behave identically. Please follow all scripts and response principles. Use caution when ad-libbing. Less is more!

Case overview: There are two cases with identical prognoses. Specifically, the patient is terminally and critically ill. He is likely to die in the next 24–48 hours if he does not receive life-sustaining treatment in the ICU; even with this treatment, his life expectancy is less than 3 months. The decision that the physician has to make is how to treat the patient right now, given that he is critically ill. The “default” approach in the hospital is to “do everything;” the alternative would be to initiate palliative care, which might include a morphine drip and preparation for death (e.g., attending to comfort, and to family and spiritual issues). During the simulation we will record the physicians’ test ordering and specific treatment decisions. All simulations will run until a decision is made or 30 minutes elapse.

The day of admission: The patient was in the skilled nursing home/rehabilitation hospital where the surrogate had been visiting daily. He’d been there trying to regain his strength after his operation 3 months ago, which was complicated and involved a 2-month hospital stay. It’s been around 3 weeks since he was discharged to the hospital and he’s been generally dwindling. He seemed a little unwell yesterday, but the surrogate thought he was just a little tired. After she left yesterday, they called her and said he was sick (short of breath/feverish) and they were sending him to the hospital. She met him in the emergency department in the wee hours of the morning.

Current location: The location will depend on the type of physician being tested. If the physician is an emergency medicine physician, the location is the Emergency Department. The patient and his surrogate will be waiting in the Emergency Department for a bed up on a hospital ward after having been fully evaluated by one of the emergency physician’s colleagues during the last shift when he begins to get sicker. If the physician is a hospitalist or an intensivist, the location is a hospital ward. He will have been admitted from the emergency department a few hours ago. In both instances, the patient will have first arrived to the emergency department in the wee hours of the morning – a total of 8 hours have passed since he arrived from the nursing home/rehabilitation hospital.

Patient role: He is awake but sick (Jenkins with gastric cancer has shortness of breath, which is gradually increasing over the last hour, associated with anxiety; Thomas with pancreatic cancer has abdominal pain, which is gradually increasing over the last hour and is now associated with some lightheadedness). The patient is also a little scared about what is happening. His main role is to answer questions in 1–3 word answers and gestures. He relies a lot on his surrogate to provide detail on his medical history, since he is very sick. The only treatment that will reduce his shortness of breath (Jenkins) or his abdominal pain (Thomas) is a narcotic.

Surrogate role: She is worried about the patient’s symptoms (explicit, volunteered); worried that he is dying (implicit, unspoken). Her main role is to express this worry, to answer questions about the patient’s past medical history, and then to either answer questions or use mirroring statements in response to treatment recommendations (see “Rules of Engagement”). She should engage with the patient throughout the simulation, since he is awake, with emotion relevant to the evolution of the encounter. This last item will be challenging. We need to ensure standardization, but we want to allow the actors to respond to the communication style of the physician. For example, if the physician attends to emotion, the actor can decrease her anxiety a little. If the physician discusses dying, the actors can express some grief. The key is for the emotional range to be the same in both families in response to similar stimuli.

Understanding of illness: The patient and his surrogate understand from the oncologist that there is no chemotherapy or radiation that will prolong his life. So, if the doctor asks the patient and his wife what they understand about the cancer or the cancer’s prognosis, they can say: “The doctor said he didn’t get all the cancer out” and “Last week the oncologist told us that it had spread to his lungs and liver,” and “He said there wasn’t anything more that he could do.” If asked what they know about how much time he has to live, they can express their knowledge in vague terms to suggest that they know he’s not likely to live long “The oncologist said he probably wouldn’t make it to another Christmas” or “He said I wouldn’t have too much longer before the cancer got me” or something like that (rather than “he said 3 months”). None of the patient’s physicians has raised the option of hospice. If the doctor provides information, such as “you know you are going to die,” then patient and his wife will respond by saying “yes, we understand.”

Preferences: Regarding life-sustaining treatments, like intubation and mechanical ventilation or CPR, the patient and his surrogate have discussed that the “doesn’t want to die hooked to machines” and he has a living will at home saying that. In the nursing home, he actually had a signed “do not resuscitate (DNR) order;” documentation of this order did not come with his medical chart from the nursing home. The patient and his surrogate should not volunteer any of this information unless asked directly.

The patient had a bad experience during his last hospitalization and does not want to go back to the ICU. If asked about his intubation preferences, he will say that “no tube.” His surrogate will report, “He said he doesn’t want to die hooked to machines.” If asked about CPR (chest compressions or shocks to re-start the heart), his surrogate can report, “No, he doesn’t want any of that.” The feeling to create here is that he doesn’t want intensive care or the ventilator (because he had a prior bad experience), and that he knows for sure he doesn’t want to “die hooked to machines”, but neither do they necessarily know that he is dying right now. The do not have the clarity about his current prognosis, nor specific knowledge of alternatives to request palliative care. They are in a situation of wanting to be helped (Mr. Jenkins is short of breath and scared, Mr. Thomas is in pain and scared) – so ready to take the doctor’s recommendation – but also knowing that he doesn’t want to “die hooked up to machines.”

If the doctor disregards his preferences and kind of bullies the patient into going to the ICU [or makes a unilateral or paternalistic decision] then the family should acquiesce (and not fight).

Understanding Palliative Care: The patient and his surrogate will not know what “comfort measures” or “palliative care” mean.

Doctor: “We can start comfort measures.”

Surrogate: “What does that mean?”

Doctor: “We can call palliative care.”

Surrogate: “What is that?”

Doctor: “We can focus on making him comfortable.”

Surrogate: “Yes, please, I want him to be comfortable.” [but without really understanding the subtext of this statement, which is that we won’t use life-sustaining treatment.]

General Rules of Engagement: The patient and his surrogate will respond to direct questions. Specifically, they will not volunteer information. Regarding their treatment preferences, they will answer questions if they are asked directly, and will use the scripted answers whenever possible. The patient is competent, and can answer questions, but he is sick and has oxygen on, so he speaks in very short sentences and use some head and hand gestures.

Let the doctor lead. If the doctor is up front and uses language like death, survive, dying, etc., then the patient and his surrogate can follow, indicating that they understand what the doctor is saying and that it is consistent with what they have come to understand from their oncologist. If the doctor uses euphemism, or avoids talking about the cancer altogether, then the patient and the surrogate also should not directly acknowledge that the patient is dying. One of the key variables we are interested is the degree to which doctors assume that the patient and wife know what he’s talking about, especially when talking about death and only using vague terminology.

Examples:

Doctor: “Do you have a living will?”

Surrogate: “Yes, we have one of those at home.”

Doctor: “What does it say?”

Surrogate: “He says he doesn’t want to die hooked up to machines.”

Patient: “No tubes.”

Doctor: “Have the two of you discussed what kind of treatment you would want if [the patient] was to get as sick as he was last time he was in the hospital?”

Surrogate: “What do you mean?”

Doctor: “Like if he needed to be on the breathing machine again?”

Surrogate: “He had a terrible experience last time; he said he never wanted that tube again.”

Patient: “No tubes”

Doctor: “Has your oncologist talked to you about hospice?”

Surrogate: “No.”

Doctor: “Your oxygen is so low that we may need to put a tube in to help you breathe. Mr. XX, is that something that you want us to do?”

Patient: “No tube.”

Surrogate: “He said he never wanted the tube again.”

If the physician does not ask about treatment preferences, but only provides treatment alternatives, the wife is to mirror the physician’s statement, and ask if that is the doctor’s treatment recommendation.

Doctor: “If we admit your husband to the ICU, we can fix his low oxygen level and help his breathing

Surrogate: “Is that what you are recommending?”

Doctor: “Some people don’t want intensive care, and might want hospice care/palliative care instead.”

Surrogate: “Is that what you’re recommending?” etc.

If the doctor is very directive, as they may be in ordering tests, the wife does not need to “mirror” and can just agree:

Example:

Doctor: “I’m going to do an arterial blood gas test to check his oxygen levels.”

Surrogate: “Okay.”

Nurse: “The test was performed, the results are unchanged from the one we drew half an hour ago.”

If the doctor gives a choice of options, the wife will ask “what are you recommending?” If the doctor responds that “it’s up to you,” then provide one more prompt to push it back to the doctor (“He’s having such a hard time.”). If truly given two treatment options, choose the least aggressive option. But if the doctor shows a clear preference for a certain course of action (e.g., seems to be making a treatment recommendation), they will go with what the doctor is recommending. If the doctor says “I want you to talk about it,” provide one more prompt to push it back to the doctor (“He’s having such a hard time.”) If asked again to think about/talk about what they want, then you can acknowledge that you have talked about it in the past.

Examples:

Doctor: “There are two main options: we can leave him here, and give him medicine to make him more comfortable or we can put the tube in his throat to help him breathe.”

Surrogate: “What are you recommending?”

Doctor: “I’m not recommending one or the other. It depends on what you and your husband want to do.”

Surrogate: “He’s having such a hard time.”

Doctor: “I know. I want you to know that whatever you decide, we will make sure that he is more comfortable.”

Surrogate: “Well, he said he doesn’t want to die hooked up to machines.”

Patient: “No tube.”

Doctor: “There are two main options: we can leave him here, and give him medicine to make him more comfortable or we can put the tube in his throat to help him breathe.”

Surrogate: “What are you recommending?”

Doctor: “I really think it’s safer to move him to the ICU. We can keep a closer eye on him there.”

Surrogate: “Okay.”

Doctor: “There are two main options: we can leave him here, and give him medicine to make him more comfortable or we can put the tube in his throat to help him breathe.”

Surrogate: “What are you recommending?”

Doctor: “I’m not recommending one or the other. It depends on what you and your husband want to do. I think that the two of you should talk about it and decide.”

Surrogate: “He’s having such a hard time.”

Doctor: “Let me get the chaplain, and then give you some time to talk about what you want.”

Surrogate: “We talked about it a lot after he got out of the ICU. He said he doesn’t want to die hooked up to machines.”

Patient: “No tube.”

Nurse Role and Anticipated Treatments

The nurse’s role is to give the “bullet” of the patient’s condition outside of the hospital room before entry. The specific bullet depends on the type of physician

Howard Jenkins (gastric cancer)

Hospitalist or Intensivist

“Mr. Howard Jenkins is a 78 yo man with metastatic gastric cancer who was admitted early this morning from skilled nursing with shortness of breath. He had no evidence of aspiration. He had blood cultures and levaquin and azithromycin administered IV in ED, and felt symptomatically improved on 6L O2 by nasal cannula and was admitted to a monitored bed on the wards. Now his oxygen requirement to 50% face mask, with sats in the 90–92% range and he’s in moderate respiratory distress. He is awake, alert and oriented, and his current vitals are: Temperature 38; HR 125; BP 100/60 mm Hg; Respiratory rate 34/min. An ABG done 30 minutes ago was: PH 7.35; PCO2 46; PO2 62. This is his medical chart. His wife is in there with him now.”

ED Physician

“Mr. Howard Jenkins is a 78 yo man with metastatic gastric cancer who was transferred to the ED from skilled nursing early this morning with shortness of breath. He had no evidence of aspiration. He had blood cultures and levaquin and azithromycin administered IV by a colleague prior to shift change, and felt symptomatically improved on 6L O2 by nasal cannula. While waiting for a bed, his oxygen requirement has increased to 50% face mask, with sats in the 90–92% range, and he’s in moderate respiratory distress. He is awake, alert and oriented, and his current vitals are: Temperature 38; HR 125; BP 100/60 mm Hg; Respiratory rate 34/min. An ABG done 30 minutes ago was PH 7.35; PCO2 46; PO2 62. This is his medical chart. His wife is in there with him now.”

Raymond Thomas (pancreatic cancer)

Hospitalist or Intensivist

“Mr. Raymond Thomas is a 76 yo man with metastatic pancreatic cancer who was admitted early this morning from skilled nursing with slight fever, mild abdominal pain and elevated LFTs. He had blood cultures drawn in the ED and was started on Unasyn and admitted to a monitored bed on the wards. He’s now tachycardic and hypotensive. Vitals are: Temperature 37.5; HR 120; BP 100/60 mm Hg; Respiratory rate 30/min, with sats in the 90–92% range on on 6L by nasal cannula. An ABG drawn 30 minutes ago was pH 7.49 PCO2 30 PO2 68. He is awake, alert and oriented. This is his medical chart. His sister, who is his main caregiver, is in there with him now.”

ED Physician

“Mr. Raymond Thomas is a 76 yo man with metastatic pancreatic cancer who was admitted early this morning from skilled nursing with slight fever, mild abdominal pain and elevated LFTs. He had blood cultures drawn in the ED and was started on Unasyn. He’s waiting for a bed upstairs. He’s now tachycardic and hypotensive. Vitals are: Temperature 37.5; HR 120; BP 100/60 mm Hg; Respiratory rate 30/min, with sats in the 90–92% range on on 6L by nasal cannula. An ABG drawn 30 minutes ago was pH 7.49 PCO2 30 PO2 68. He is awake, alert and oriented. This is his medical chart. His sister, who is his main caregiver, is in there with him now.”

In the room her job is to implement the doctor’s orders and to answer questions. Generally this will be in the form of mirroring. If the doctor asks for a treatment, the nurse should say “The [treatment] has been administered.” Of if the doctor asks for a test, the nurse should say “The test was performed, and the results are unchanged from those obtained upon admission [or, in the case of an arterial blood gas, from the one obtained 30 minutes ago.”

The doctor running the room will provide answers to questions that have not been anticipated.

If there is a question asked that the nurse/actor doesn’t know the answer to, simply say “Let me find out [raise right hand].”

The nurse/actor and the doctor running the room from behind the two-way mirror will work in collaboration, so that the nurse/actor will provide a sense of the seriousness of the patient’s condition (by not leaving the bedside), but the doctor running the simulation can provide information from the “outside” that the nurse/actor cannot.

From the pilot, the most common orders for Mr. Jenkins were:

• Turn up the oxygen – nurse will lean over the bed and turn up the oxygen nob.

• Obtain a lab test or an x-ray – nurse will provide the test results. Typically, this will be “the ---- is unchanged from those obtained upon admission.”

• Administer an albuterol nebulizer (“neb” or breathing treatment) – nurse will report that the treatment has been administered. “He’s completed his nebulizer treatment, doctor.”

• BiPAP (a face mask that blows oxygen into the airways – it’s like a ventilator without the tube down into the trachea. It is often used for patients who are either unlikely to get off a ventilator if they are put on, or for patients who might be bridged through an acute episode of lung problems and who don’t want to be on a ventilator). The patient and his surrogate should be willing to try this. The doctor running the room will say “It’s been 3 hours since we started BiPAP and his vital signs are not improving.” The vital signs will stay on the same trajectory.

• Narcotics (e.g., morphine, fentanyl) – the nurse will say “It’s in.” The patient should become less agitated, breathe less rapidly, and become a little sleepy. He should indicate that he feels a little better.

• Fluid (e.g., a 500 cc “bolus”) – the nurse will say “It’s in.”

From the pilot, the most common questions/other requests were:

• What access does he have?

  • “16-gauge in the right antecube.”

• What was his pressure when he came in?

  • It’s on the chart.” [You can memorize whatever history you think would be appropriate for the nurse to know]

• Can you page the patient’s doctor (could be the primary care doctor, oncologist, or surgeon)?

  • He’s not answering the page.”

• Call palliative care

  • Go to the phone in the room and pretend to make a call, then report: “I’ve paged them; what should I tell them when they call back?”

We do not know for sure what the orders will be for Mr. Thomas; however, many of the requests will be similar. Again, if there is anything that the nurse did not anticipate, try to make it up using clinical knowledge.

If the doctor calls a Condition C or a calls for a medical emergency team (MET), the investigator running the simulation room will tell the doctor to pretend like s/he is the doctor who is leading the medical emergency team that responds to the Condition C. The nurse will continue to play her role implementing orders/answering questions with the support of the investigator running the simulation room.

Summary of Psychosocial and Clinical Aspects of the Two Cases

Mr. Howard Jenkins	Mr. Raymond Thomas
Chief Complaint: Shortness of breath and anxiety	Chief complaint: Abdominal pain.
Reason for summoning the doctor: From the nurse’s perspective: increasing oxygen requirement and respiratory distress; from the surrogate’s perspective: difficulty breathing is gradually worsening over the last hour and he seems even more anxious.	Reason for summoning the doctor: From the nurse’s perspective: tachycardia and hypotension; from the surrogate’s perspective: gradually increasing abdominal pain in the right upper abdomen – the same pain as brought him in originally – and a feeling of lightheadedness or “wooziness.”
Diagnosis: Metastatic gastric cancer; being treated acutely with antibiotics for a possible pneumonia.	Diagnosis: Metastatic pancreatic cancer; being treated acutely with antibiotics for a possible biliary tract infection. Also has a bedsore on his backside.
Past Medical History: diabetes, chronic renal insufficiency, high blood pressure	Past Medical History: heart disease (bypass surgery 10 years ago), stroke, high cholesterol, arthritis.
Age: 78	Age: 76
Occupation: Retired English teacher	Occupation: Retired bank manager
Home: South Side Slopes, 18th Street, near St. Paul’s Monastery	Home: Point Breeze, Meade Street, near Construction Junction
Religion: Catholic	Religion: Protestant
Surrogate: Emma (wife)	Surrogate: Ruth (sister)
Family: Daughter, 42, Collette, lives in Hawaii with her husband Dennis and two young children. She cannot travel quickly to Pittsburgh. She visited Pittsburgh around the time of her dad’s transfer from the hospital to the nursing home 3 weeks ago. She understands the severity of her father’s illness and discussed the fact that he might soon die during the visit, when the family also met with Father Ken. Emma is in touch with Collette by phone every few hours.	Family: Wife is dead (died 4 years ago from diabetes and heart failure). Son, Ray Jr., is deployed in Iraq as a Captain in the U.S. Army. He is not married. He came home when his dad was in the ICU last time, and he understands the severity of his father’s illness. Ray Jr. has been in touch with his mom and dad by Skype frequently since returning to Iraq, and has made a joke out of saying to his dad at the end of those calls: “Just in case both of us bite it before next time, let’s meet at the corner bar in heaven.” Ruth does not think this is very funny; but Ray Sr. does.
Skilled Nursing Facility: Presbyterian SeniorCare (Oakmont)	Skilled Nursing Facility: Heritage Place (Squirrel Hill)
PCP: Dr. Robertson	PCP: Dr. Marks
Oncologist: Dr. Smith	Oncologist: Dr. Jones
Surgeon: Dr. Lee	Surgeon: Dr. Hughes
Medical facts: Mr. Jenkins was transferred from a skilled nursing facility to the hospital in the wee hours of the morning because of shortness of breath. His wife met him there. The x-rays of the lungs showed metastatic cancer, a possible pneumonia, but no sign of a blood clot. He was given a diuretic and a breathing treatment, and was started on antibiotics for a possible pneumonia. He felt a little better in the first few hours he was treated.	Medical facts: Mr. Thomas was transferred from a rehabilitation facility to the hospital in the wee hours of the morning because of mild abdominal pain and a low-grade temperature. His sister met him there. The x-rays showed metastatic cancer in the liver and the lab tests showed elevated liver function tests. He was started on IV antibiotics “just in case he had an infection” and given some oxycodone for his abdominal pain. He felt a little better in the first few hours he was treated.
In the last hour his shortness of breath is gradually worsening, and his oxygenation is dropping. The doctor who is called to see him will be given laboratory and vital signs data that suggest Mr. Jenkins will soon (possibly in 4–6 hours) lose consciousness and could soon die without medical intervention, such as being put on a ventilator.	In the last hour his abdominal pain is gradually increasing and now he feels woozy. The doctor who is called to see him will be given laboratory and vital signs data that suggest Mr. Jenkins will soon (possibly in 4–6 hours) lose consciousness and could soon die without medical intervention, such as being put on a ventilator.
Three months ago Mr. Jenkins underwent a partial gastrectomy (removal of the stomach) for gastsric (stomach) cancer. The operation was not curative – there were positive margins (cancer at the edge of the removed stomach) and lymph nodes with cancer in them (suggesting metastasis).	Three months ago, Mr. Thomas underwent a Whipple procedure (removal of the pancreas) for pancreatic cancer. The operation was not curative – the cancer had spread regionally to a large blood vessel and lymph nodes.
After the initial surgery, Mr. Jenkins was in the ICU for 2 months. He needed the breathing machine and hemodialysis. Three weeks ago he finally got off the ventilator, his tracheostomy was removed, and he was discharged to a skilled nursing facility because he was too weak to go home.	After the initial surgery, Mr. Thomas was in the ICU for a 2 months because he got an infection of his blood; he needed a breathing machine and hemodialysis, and suffered a stroke. A month ago he finally got off the ventilator, his tracheostomy was removed, and he was discharged to a skilled nursing facility because he was too weak to go home.
One week ago he had a follow-up appointment with his oncologist; a CT scan done at that time revealed that the cancer had spread to the lungs and the liver. The oncologist felt that he was too weak to receive any chemotherapy at this time and explained to the family that, even with such treatment, Mr. Jenkins wouldn’t live very long (he did not give a specific time frame).	One week ago he had a follow-up appointment with his oncologist; a CT scan done at that time revealed that the cancer had spread to the liver and he had some fluid in his lungs. The oncologist felt that he was too weak to receive any chemotherapy at this time and explained to the family that, even with such treatment, Mr. Thomas wouldn’t live very long (he did not give a specific time frame).
Other: Ask to see Father Ken” if the doctors offers spiritual support or asks if they need “anything else.”	Other: Ask to see Reverend Craig Barnes if the doctor offers spiritual support or asks if they need “anything else.”

APPENDIX 2

Barnato et al. “Physician ICU admission, intubation, and palliation decisions for black and white elders with end-stage cancer: A simulation study”

Figure A1. Physician sample.

We initially sought to recruit a probability sample of emergency medicine, hospitalist, and intensive care physicians in approximately equal proportions from Allegheny County. We mailed invitations to participate to a stratified random sample of physicians drawn from Allegheny County Medical Society lists of board-certified emergency physicians, internists, and critical care physicians, augmented by hospitalist and intensivist lists provided by Allegheny County hospitals. In blocks of 20, we contacted physicians in each specialty group by letter containing a postage paid response card, followed by an average of 1.5 follow-up telephone calls. Additionally, we contacted all black physicians in the sampling frame. Due to a low recruitment yield using these methods, we switched to convenience sampling using two mechanisms: culling the sampling frame for physicians personally known to any of the investigators for recruitment and having influential colleagues send e-mails to distribution lists of regional emergency physician, hospitalist, and intensivist practice groups. From the original sampling frame of 505 physicians, we sampled and sought to contact 177. Twenty-six were willing to participate (15%), but only 15 could be scheduled (8%). Forty were confirmed ineligible (23%), 69 (39%) were confirmed to be in practice but did not return phone calls, 26 (15%) refused, and 16 (9%) could not be located (Figure 3). We recruited 18 additional physicians who were not in the original sampling frame through professional contacts.

Figure A2. Factorial allocation of subjects.

The factorial allocation of subjects was closely balanced with 9 subjects in one permutation (white/gastric followed by black/pancreatic) and 8 subjects in the other 3 (white/pancreatic followed by black/gastric; black/gastric followed by white/pancreatic; black/pancreatic followed by white/gastric), but it became less balanced when we removed one study subject from the analysis of treatment plan in the black/pancreatic followed by white/gastric permutation.