Abstract
Objective
To evaluate the effect of (1) patient values as expressed by family members, and (2) a requirement to document patients’ functional prognosis on intensivists’ intention to discuss withdrawal of life support in a hypothetical family meeting.
Design
A 3-armed, randomized trial
Setting
179 U.S. hospitals with training programs in critical care accredited by the Accreditation Council for Graduate Medical Education
Subjects
630 intensivists recruited via e-mail invitation from a database of 1,850 eligible academic intensivists
Interventions
Each intensivist was randomized to review ten, on-line, clinical scenarios with a range of illness severities involving a hypothetical patient (Mrs. X). In control-group scenarios, the patient did not want continued life support without a reasonable chance of independent living. In the first experimental arm, the patient wanted life support regardless of functional outcome. In the second experimental arm, patient values were identical to the control group, but intensivists were required to record the patient’s estimated three-month functional prognosis
Measurements and Main Results
Response to the question: “Would you bring up the possibility of withdrawing life support with Mrs. X’s family?” answered using a five-point Likert scale. There was no effect of patient values on whether intensivists intended to discuss withdrawal of life support (P = 0.81), but intensivists randomized to record functional prognosis were 49% more likely (95% confidence interval: 20%–85%) to discuss withdrawal.
Conclusions
In this national, scenario-based, randomized trial, patient values had no effect on intensivists’ decisions to discuss withdrawal of life support with family. However, requiring intensivists to record patients’ estimated 3-month functional outcome substantially increased their intention to discuss withdrawal.
Keywords: Decision Making, Intensive Care, Resuscitation Orders, Prognosis, Withholding Treatment, Life Support Care
INTRODUCTION
Consensus statements recommend that interdisciplinary family meetings occur for all ICU patients within five days of admission and include discussions about patient values, prognosis, and the options of comfort care or withdrawing life support when death is possible.1–4 However, whether surrogates of critically ill patients are offered comfort care as a treatment option depends primarily on whether their physician believes life support should be withheld or withdrawn.5,6 Previous work has documented that physicians are reluctant to discuss outcomes for critically ill patients in the face of prognostic uncertainty, and frequently do not ask surrogates about patient values.7–11 How physicians decide whether to discuss withdrawing life support as an option under these circumstances is unclear.
In this study, we asked academic intensivists to evaluate a single hypothetical patient, across each of 10 scenarios representing a wide range of illness severities (probability of in-hospital mortality 0.35–0.98), and report whether they would discuss withdrawing life support with her family. Our objectives were to evaluate the effect of (1) patient values as expressed by family members, and (2) a requirement that intensivists record patients’ 3-month functional prognoses, on intensivists’ intent to discuss the option of withdrawing life support in a planned family meeting. We hypothesized that intensivists would be responsive to patient wishes and would be more likely to discuss withdrawing life support after recording a functional prognosis.
MATERIALS AND METHODS
Subject Recruitment
A previously described database of academic intensivists was used to recruit faculty from hospitals with training programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) in Internal Medicine – Critical Care Medicine, Anesthesiology – Critical Care Medicine, and Surgical Critical Care.12 The database was updated in 2012 to include demographic and electronic contact information for 2,482 physicians. Physicians were excluded from randomization if they 1) lacked electronic contact information (11%), 2) had been invited to participate in the pilot study (11%), 3) had made a previous request not to be contacted (3%), or 4) contributed to study design or survey development (<1%) (Figure 1). All randomized physicians received an invitation via e-mail on November 20th, 2012 to participate in a short survey administered using the Qualtrics© on-line survey platform as part of a study about ICU physicians’ attitudes about life support. Recruitment ended December 21st, 2012. The institutional review board of Johns Hopkins University approved the study.
Figure 1. Recruitment, randomization, and exclusion criteria.
“No Response” is defined as never clicking the web link to the study survey in the invitation e-mail. “Not completed” means that the web link to the study survey was clicked, but not all survey questions were answered. “Not practicing” indicates that the physician selected “No” in response to the survey question: “Have you treated patients in the ICU setting in the last two years?”
Clinical Scenarios
All study participants reviewed 10 clinical scenarios involving a 76-year-old female patient (Mrs. X) admitted to the ICU from the emergency room 72 hours previously with severe, community-acquired pneumonia and mechanically ventilated for the past 48 hours. The hypothetical patient was not able to communicate or participate in decision-making, but her adult children had participated in a family meeting at ICU admission and specified “full-code” status. In each clinical scenario, background information about the patient and family, as well as vital signs and lab work from 72-hours post admission, were displayed. The vital signs and lab work in the 10 scenarios were identical in each study arm, and represented a wide range of illness severities and associated probabilities of in-hospital mortality. To view the vital signs and lab work from 3 of the 10 scenarios see Supplemental Digital Content – Appendix 1 Figure 2. Inhospital mortality probabilities for each scenario were estimated using the Mortality Probability Model II-72 hours (MPMII-72),15,16 but these probabilities were not displayed to participants. The 10 scenarios were presented in a randomly generated order, and respondents were not permitted to skip scenarios or return to previously viewed scenarios before answering the primary outcome question: “Would you bring up the possibility of withdrawing life support with Mrs. X’s family?” during a follow-up family meeting at 72 hours. Participants answered this question using a 5-point Likert scale containing the answers “Definitely No”, “Probably No”, “Maybe”, “Probably Yes”, and “Definitely Yes.”
Trial Interventions
Blocking on physician sex, specialty (medicine, anesthesiology, or surgery), years since residency, and geographic region of residency was performed prior to participant randomization.13,14 Within blocks, physicians were randomized to either the control arm, or one of two experimental arms (Table 1).
Table 1.
Experimental Design
| Control Arm | 1st Experimental Arm | 2nd Experimental Arm | |
|---|---|---|---|
| Group Name | Control | Values | Prognosis |
| Scenarios reviewed | 10 | 10 | 10 |
| Severity of illness in the 10 scenarios | Vital signs and lab values consistent with a probability of in-hospital mortality ranging from 0.35 to 0.98as calculated using the MPMII-72† | Same vital signs and lab values as control arm | Same vital signs and lab values as control arm |
| Family statements about patient values described in scenario background information | Mrs. X would not want to continue life-sustaining treatment if there was not a reasonable chance that she would eventually return to independent living in her own home. | Mrs. X has always been a would receive help with her activities of daily living. | Same as control |
| 3-month functional prognosis required? | No | No | Yes |
| Outcome measures | Response to: “Would you bring up the possibility of withdrawing life support with Mrs. X’s family?” | Same as control | Same as control |
In the control arm of this study, the background information stated that, at ICU admission, the family “did not believe Mrs. X would want to continue life-sustaining treatment if there was not a reasonable chance that she would eventually return to independent living in her own home.” In the first experimental arm (“values” group), Mrs. X’s family stated that “Mrs. X has always been a ‘fighter’ and would want life-sustaining therapy even if her best possible outcome is transfer to a nursing home where she would receive help with her activities of daily living.” In the second experimental arm (“prognosis” group) Mrs. X’s family made statements identical to the control arm. For the complete background descriptions from each study arm see Supplemental Digital Content - Appendix 1 Figure 1.
In the prognosis group, after reviewing each scenario and before indicating whether they intended to discuss withdrawing life support, intensivists were asked an additional question: “What do you believe is Mrs. X’s most likely outcome three months after discharge based on the information above?” The additional question was answered by selecting one of five possible answers: 1) Able to live independently with no physical or cognitive decline from pre-ICU status, 2) Able to live independently with some physical and cognitive decline from pre-ICU status, 3) Dependent in at least one activity of daily living (ADL) and unable to live independently, 4) Dependent in all ADLs and unable to live independently, or 5) In-hospital death.
Statistical analysis
Multivariable regression was used to model physician intent to discuss withdrawal of life support and to estimate the impact of intervention group on intention to discuss withdrawal, defined as selecting “Probably Yes” or “Definitely Yes” in response to the primary outcome question. The model was adjusted for intensivist sex, specialty, region of first residency, years since first residency, and scenario. Sex, specialty, region, scenario, and intervention group were modeled as categorical variables. Generalized estimating equations, with an exchangeable correlation matrix, were used to account for the clustering of responses, by intensivist, across the 10 scenarios.17 Because bringing up withdrawal was not a rare outcome, odds ratios were not close approximations of risk ratios between intervention groups. Therefore, we used the COPY method (C = 1,000) to estimate risk ratios and improve interpretability.18,19 Among study participants included in the analysis, region of first residency was missing for 6.2% and year of first residency completion was missing for 15.6%. Missing values for these two variables were imputed using imputation for chained equations and included in regression analyses.20 All other variables had no missing data.
Additional sensitivity analyses were performed to ensure model estimates were robust to the threshold chosen for the outcome definition (e.g. including “Maybe” as an indication of discussion), and to assess the generalizability of the findings to the full population of academic intensivists.21,22 To assess whether nonresponse bias might limit the generalizability of our results, we generated propensity scores to predict trial participation using boosted classification and regression trees (CART) and all available intensivist covariates.23 With these methods, each participant was assigned a weight (median = 1.02; range = 0.34 – 2.26), defined as the marginal probability of completing the questionnaire divided by their propensity score. The weights were incorporated into multivariable regression models to assess whether population-weighted effect estimates differed from unweighted estimates. Analyses were performed using the R programming language (version 3.0.1; R Development Core Team, including the twang, mice, and geepack packages).24–27
RESULTS
Participants
Of the 1,850 physicians invited to participate, 726 (39%) from 179 hospitals responded. Respondents were excluded from analysis if they did not answer all study questions (3%), or responded “No” to the question: “Have you treated patients in the ICU setting in the last two years?” (2%). A total of 630 academic intensivists were successfully recruited and included in analysis with 205, 215, and 210 in the control, “values”, and “prognosis” groups respectively (Figure 1). Among the analyzed intensivists, 79% were male, 60% specialized in internal medicine, and 50% completed their first residency at least 18 years ago (Table 2).
Table 2.
Intensivist Demographics
| Updated database (N = 2,482) | Invited to participate* (n = 1,850 ) | Analyzed sample† (n = 630) | Non-responders (n = 1,124 ) | |
|---|---|---|---|---|
| Male | 80% | 80% | 79% | 81% |
| Specialty‡ | ||||
| Medicine | 63% | 63% | 60% | 65% |
| Surgery | 21% | 20% | 20% | 21% |
| Anesthesia | 15% | 16% | 20% | 14% |
| Region of residency‡,§ | ||||
| Northeast | 34% | 33% | 33% | 32% |
| Midwest | 23% | 22% | 22% | 22% |
| South | 22% | 22% | 23% | 21% |
| West | 11% | 11% | 12% | 12% |
| International | 5% | 5% | 4% | 6% |
| Unknown | 6% | 7% | 6% | 7% |
| Years since residency|| | 20 | 20 | 18 | 21 |
| median (IQR) | (13 – 28) | (13 – 28) | (11 – 26) | (14 – 28) |
Physicians in the database were not invited to participate if their e-mail address was not available, if they were invited to participate in the pilot study, if they had previously requested not to be contacted, or if they contributed to designing the survey.
Respondents who answered “No” to the question Have you treated patients in the ICU setting in the last two years? and those who did not complete all questions in the survey were excluded from analysis.
Percentages may not add to 100% due to rounding.
Region is defined as the US census region.
Among physicians for whom year of residency completion was available.
Outcomes
The in-hospital mortality rate, as predicted by the MPMII-72, ranged from 35% to 98% across the 10 randomly sequenced scenarios. The proportion of intensivists selecting “Probably Yes” or “Definitely Yes” ranged from 3.7% (95% confidence interval (CI): 1.2%-6.3%) in the values group responding to the scenario with the lowest predicted mortality, to 85.2% (95% CI: 80.4%–90.1%) in the prognosis group responding to the scenario with highest predicted mortality (Figure 2). The proportion of intensivists in the control and values groups intending to discuss withdrawal of life support was similar across all scenarios. In every scenario, the proportion raising withdrawal was greatest in the prognosis group. The proportion reporting they would “Maybe”, “Probably,” or “Definitely” discuss withdrawal was significantly greater in the prognosis group than in the control group in scenarios 3 through 7. The proportion reporting they would “Probably” or “Definitely” discuss withdrawal was significantly greater in scenarios 3,4, and 6 (Figure 2).
Figure 2. Intensivist response to the question “Would you bring up the possibility of withdrawing life support with Mrs. X’s family?”.
Study participants responded using a 5-point Likert scale containing: Definitely No, Probably No, Maybe, Probably Yes, and Definitely Yes. Probability of in-hospital mortality was calculated using the Mortality Probability Model II – 72 Hours (MPMII –72). Bars depict 95% confidence intervals for the proportion of intensivists selecting Maybe, Probably Yes, or Definitely Yes for plot on left and Probably Yes or Definitely yes for plot on right. The X-coordinates of over-lapping confidence intervals have been jittered to avoid overlap. Lines are loess curves generated using a bandwidth of 0.5.
In the prognosis group, the majority of intensivists expected the patient to survive and return to independent living in the two scenarios with the lowest predicted mortality, to die in-hospital in the two scenarios with the highest predicted mortality, and to survive but be unable to live independently in all other scenarios (Figure 3). Intensivists in the prognosis group indicated they would discuss withdrawal of life support in 88% of the scenarios in which they expected the patient to die in the hospital (Table 3).
Figure 3. Three-month, function prognosis as predicted by 210 intensivists.
The 210 intensivists in the second experimental arm of the study (prognosis group) recorded estimates of the most likely 3-month, functional outcome for each of 10 scenarios using a scale with the options: 1) Able to live independently with no physical or cognitive decline from pre-ICU status, 2) Able to live independently with some physical and cognitive decline from pre-ICU status, 3) Dependent in at least one ADL and unable to live independently, 4) Dependent in all activities of daily living and unable to live independently, or 5) In-hospital death. Scenarios are displayed by MPMII-72 probability of in-hospital mortality from left to right.
Table 3.
Responses to the question: “Would you bring up the possibility of withdrawing life support with Mrs. X’s family?” by 3-month outcome
| Response to “Would you bring up withdrawing life support?” | Expected 3-Month Functional Outcome
|
||
|---|---|---|---|
| Able to live independentlya | Unable to live independently*,† | In-hospital death† | |
|
Probably or Definitely Yes Number (% of column total) |
35 (6%) | 456 (46%) | 449 (88%) |
|
Maybe Number (% of column total) |
111 (18%) | 325 (33%) | 45 (9%) |
|
Probably or Definitely No Number (% of column total) |
461 (76%) | 200 (20%) | 18 (4%) |
| Total | 607 (100%) | 981 (100%) | 512 (100%) |
The responses “Able to live independently with some physical and cognitive decline from pre-ICU status” and “Able to live independently with no physical or cognitive decline from pre-ICU status” were grouped as “Able to live independently.” The responses “Dependent in all activities of daily living (ADLs) and unable to live independently” and “Dependent in at least one ADL and unable to live independently” were grouped as “Unable to live independently.”
Percentages in column do not add to 100% due to rounding.
There was no statistically significant association between any of the known intensivist characteristics and the primary outcome in the adjusted regression model (Table 4). Moreover, comparison of results from the randomized “values” experimental group versus the control group, revealed no significant effect of the family’s statements about the patient’s wish for continued use of life support on the proportion of intensivists reporting they would discuss withdrawal of life support (adjusted RR = 1.03; 95% CI: 0.81–1.32; P = 0.81). However, in the adjusted analyses, intensivists in the “prognosis” versus control group were 49% (95% CI: 20%–85%; P <0.001) more likely to report “probably” or “definitely” discussing withdrawal of life support. These results remained significant in a sensitivity analyses in which the outcome definition also included the “Maybe” response option (P < 0.001). After weighting trial participants to reflect the distribution of demographic covariates among the 2,482 physicians in the database, results of the “prognosis” versus control group comparison remained significant (P <0.001) and the “values” versus control group comparison remained statistically insignificant (P = 0.33).
Table 4.
Association between intensivist characteristics and selecting “Probably yes” or “Definitely yes” response to bringing up withdrawing life support*
| Model 1 Complete cases only Risk Ratio N = 511 | Model 2 Imputed covariates† Risk Ratio N = 630 | |
|---|---|---|
| Sex | ||
| Male | Ref | Ref |
| Female | 1.11 (0.90,1.36) | 1.15 (0.96,1.39) |
| Specialty | ||
| Medicine | Ref | Ref |
| Surgery | 1.16 (0.94,1.43) | 1.18 (0.97,1.43) |
| Anesthesia | 0.72 (0.53,0.96) | 0.81 (0.63,1.04) |
| Region of Residency | ||
| Northeast | Ref | Ref |
| Midwest | 0.95 (0.75,1.20) | 0.98 (0.79,1.22) |
| South | 0.93 (0.74,1.18) | 0.90 (0.72,1.12) |
| West | 0.85 (0.61,1.18) | 0.87 (0.66,1.15) |
| International | 0.75 (0.36,1.57) | 0.81 (0.44,1.48) |
| Years since residency | ||
| 1 year increase | 0.99 (0.98,1.00) | 0.99 (0.98,1.00) |
| Survey | ||
| Control | Ref | Ref |
| Values | 0.95 (0.72,1.26) | 1.03 (0.81,1.32) |
| Prognosis | 1.45 (1.13,1.85) | 1.49 (1.20,1.85) |
Model estimated using the COPY method with C = 1,000 and adjusted for scenario. Clustering of responses within intensivists accounted for using generalized estimating equations with an exchangeable correlation matrix.
Region of first residency was missing for 6.2% of intensivists and year of first residency completion was missing for 15.6%. Missing values were imputed using imputation for chained equations in model 2.
DISCUSSION
In this scenario-based, randomized trial of 630 academic intensivists from 179 hospitals, statements made by family members about a hypothetical patient’s desire to continue the use of life support contingent on her expected functional outcomes had no effect on intensivists’ self-reported intention to discuss withdrawing life support in a planned family meeting. However requiring intensivists to record the patient’s 3-month, functional outcome made them significantly more likely to report intent to discuss withdrawal of life support.
In psychology the tendency to place more importance on a factor that a person was prompted to consider is known as the “focusing effect”.28,29 Our results suggest that cueing physicians to consider relevant factors, such as functional outcomes or a patient’s goals of care, when faced with clinical decisions may be an effective use of the focusing effect phenomenon. Eliciting functional prognoses early in the course of an ICU stay is already a component of a previously described communication-based model to improve end-of-life care.30
The lack of association between family statements about patient wishes and intensivists’ intention to discuss withdrawal of life support was unexpected. In a prospective study of mechanically ventilated patients, physician perception that a patient would not want life support was strongly associated with actual withdrawal of life support.31 If physician perception of patient wishes is truly influential, it may be that intensivists in this scenario-based trial did not believe family statements were accurate representations of patient wishes, especially since surrogates incorrectly predict treatment preferences for almost one-third of patients.32 However there is evidence suggesting that when patients directly express their wishes through advance directives, it does not substantially influence their care at the end of life.33,34
Although the proportion of intensivists bringing up withdrawal of life support was very similar across all scenarios in the control versus “values” groups, the difference between the control versus “prognosis” group varied substantially between scenarios (Figure 2). For the first two scenarios, in which the probability of in-hospital death was <50%, and in the last two scenario in which the probability of in-hospital death was >90%, there was no significant difference in the proportion of intensivists in the control versus prognosis groups responding “probably” or “definitely” yes. The impact of prognosticating a 3-month functional outcome appeared greatest for scenarios in which the majority of intensivists expected the patient to survive, but become dependent in activities of daily living.
Additionally, unique attributes of scenarios likely influenced physician response. For example, the proportion of intensivists planning to discuss withdrawal was particularly high, relative to the MPMII-72 predicted mortality, in scenarios 4 and 6 when compared to scenario 5 (Figure 2). Similarly, the proportion of intensivists in the prognosis arm of the study who selected “in-hospital death” as the most likely outcome was greater in scenarios 4 and 6 than in scenarios 3, 5, or 7 (Figure 3). We suspect this is because scenarios 4 and 6 included chronic liver disease as a comorbidity.
Our study has potential limitations. First, intensivist responses to scenarios may not accurately reflect real-world behavior. In the clinical setting, recording a functional prognosis might have more or less impact on intensivist actions depending on where such a prognosis is recorded, how it is used, and who is able to access the information. However, the use of a hypothetical patient scenario was necessary to permit a randomized trial design for isolating the potential cause-effect association of documenting functional prognosis on intention to discuss withdrawal of life support. Second, we cannot assume that our findings are generalizable to clinical situations not represented by our study scenario. For example, we do not know how our results might have differed if the patient had been younger, male, chronically ill, or ventilated for a longer duration, or if a prognosis had been required for a patient whose family preferred aggressive treatment regardless of expected outcome. Third, we were limited to recruiting participants from an existing database of academic intensivists from hospitals with training programs in critical care which limits generalizability. Finally it is possible that intensivists recruited into the study were more interested in patient-physician communication or end-of-life care than non-participants. The effects of patient values or reporting a functional prognosis could have been different for physicians with less interest.35 However, such nonresponse bias is lessened by two observations: (1) measured physician characteristics were similar across non-respondents, early respondents, and late respondents (see Supplemental Digital Content - Appendix 1 Table 1), and (2) analyses weighted for physicians’ propensities to respond produced similar results as the original unweighted analyses.36,37
CONCLUSION
In summary, this scenario-based, randomized trial found no effect of family statements about patient values on intensivists’ self-reported intent to discuss withdrawal of life support in a hypothetic family meeting. However, requiring intensivists to record the patient’s most likely 3-month functional outcome significantly increased the proportion of intensivists reporting they would discuss withdrawal of life support. Future studies in clinical settings are warranted to explore the effects of interventions that focus clinicians on functional outcomes prior to discussing treatment options with critically ill patients and their families.
Supplementary Material
Acknowledgments
We thank all the physicians who participated in the study as well as Cristi O’Connor, BA, Cheyenne McCray BA, and W. Chase Hamilton, for help with data collection, and Ann Parker, MD for assistance with scenario development.
AET is supported by the Johns Hopkins University Sommer Scholars Program and a postdoctoral training grant from the National Institute on Aging, T32AG000247.
Footnotes
All authors contributed to the conception and design of the study. AET and JRK analyzed the data. AET drafted the article. All authors contributed to the interpretation of analyses, critically revised the article for important intellectual content and gave final approval of the manuscript version to be published. AET is responsible for the overall content as guarantor.
The remaining authors have disclosed that they do not have any potential conflicts of interest.
Copyright Form Disclosures: Dr. Turnbull received support for article research from NIH (supported by a postdoctoral training grant from the National Institute on Aging, T32AG000247). Dr. Krall received grant support from the National Institute on Aging (T32-AG000247) and received support for article research from NIH (T32-AG000247).
References
- 1.Nelson JE, Mulkerin CM, Adams LL, Pronovost PJ. Improving comfort and communication in the ICU: a practical new tool for palliative care performance measurement and feedback. Qual Saf Health Care. 2006;15(4):264–271. doi: 10.1136/qshc.2005.017707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Truog RD, Campbell ML, Curtis JR, et al. Recommendations for end-of-life care in the intensive care unit: a consensus statement by the American College [corrected] of Critical Care Medicine. Crit Care Med. 2008;36(3):953–963. doi: 10.1097/CCM.0B013E3181659096. [DOI] [PubMed] [Google Scholar]
- 3.Nelson JE, Bassett R, Boss RD, et al. Models for structuring a clinical initiative to enhance palliative care in the intensive care unit: a report from the IPAL-ICU Project (Improving Palliative Care in the ICU) Crit Care Med. 2010;38(9):1765–1772. doi: 10.1097/CCM.0b013e3181e8ad23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. [Accessed March 23, 2012];Care and Communication Quality Measure at the National Quality Measures Clearing house by the Agency for Healthcare Research and Quality. Available at: http://qualitymeasures.ahrq.gov/
- 5.Danis M, Mutran E, Garrett JM, et al. A prospective study of the impact of patient preferences on life-sustaining treatment and hospital cost. Crit Care Med. 1996;24(11):1811–1817. doi: 10.1097/00003246-199611000-00009. [DOI] [PubMed] [Google Scholar]
- 6.Schenker Y, Tiver GA, Hong SY, White DB. Association between physicians’ beliefs and the option of comfort care for critically ill patients. Intensive Care Med. 2012;38(10):1607–1615. doi: 10.1007/s00134-012-2671-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.White DB, Engelberg RA, Wenrich MD, Lo B, Curtis JR. Prognostication during physician-family discussions about limiting life support in intensive care units. Crit Care Med. 2007;35(2):442–448. doi: 10.1097/01.CCM.0000254723.28270.14. [DOI] [PubMed] [Google Scholar]
- 8.Cox CE, Martinu T, Sathy SJ, et al. Expectations and outcomes of prolonged mechanical ventilation. Crit Care Med. 2009;37(11):2888–2894. doi: 10.1097/CCM.0b013e3181ab86ed. quiz 2904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Heyland DK, Barwich D, Pichora D, et al. Failure to Engage Hospitalized Elderly Patients and Their Families in Advance Care Planning. JAMA Intern Med. 2013:1–10. doi: 10.1001/jamainternmed.2013.180. [DOI] [PubMed] [Google Scholar]
- 10.White DB, Malvar G, Karr J, Lo B, Curtis JR. Expanding the paradigm of the physician’s role in surrogate decision-making: an empirically derived framework. Crit Care Med. 2010;38(3):743–750. doi: 10.1097/CCM.0b013e3181c58842. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Corke CF, Stow PJ, Green DT, Agar JW, Henry MJ. How doctors discuss major interventions with high risk patients: an observational study. BMJ. 2005;330(7484):182. doi: 10.1136/bmj.38293.435069.DE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Halpern SD, Hussen SA, Metkus TS, Ward NS, Luce JM, Curtis JR. Development of an e-mail database of US intensive care physicians. J Crit Care. 2007;22(1):28–31. doi: 10.1016/j.jcrc.2006.12.006. [DOI] [PubMed] [Google Scholar]
- 13.US Census Regions. [Accessed July 1, 2013]; Available at: http://www.census.gov/geo/www/us_regdiv.pdf.
- 14.Imai K, King G, Stuart EA. Misunderstandings between experimentalists and observationalists about causal inference. J R Stat Soc Ser A Stat Soc. 2008;171(2):481–502. doi: 10.1111/j.1467-985X.2007.00527.x. [DOI] [Google Scholar]
- 15.Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J. Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA J Am Med Assoc. 1993;270(20):2478–2486. [PubMed] [Google Scholar]
- 16.Lemeshow S, Klar J, Teres D, et al. Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study. Crit Care Med. 1994;22(9):1351–1358. doi: 10.1097/00003246-199409000-00003. [DOI] [PubMed] [Google Scholar]
- 17.Heagerty PJ, Zeger SL. Marginal Regression Models for Clustered Ordinal Measurements. J Am Stat Assoc. 1996;91(435):1024–1036. doi: 10.2307/2291722. [DOI] [Google Scholar]
- 18.Lumley T, Kornmal R, Ma S. Relative risk regression in medical research: models, contrasts, estimators, and algorithms. [Accessed June 13, 2013];UW Biostatistics Working Paper Series. 2006 Vol Paper 293. Available at: http://www.bepress.com/uwbiostat/paper293.
- 19.Savu A, Liu Q, Yasui Y. Estimation of relative risk and prevalence ratio. Stat Med. 2010;29(22):2269–2281. doi: 10.1002/sim.3989. [DOI] [PubMed] [Google Scholar]
- 20.Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res. 2011;20(1):40–49. doi: 10.1002/mpr.329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cole SR, Stuart EA. Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. Am J Epidemiol. 2010;172(1):107–115. doi: 10.1093/aje/kwq084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Stuart EA, Cole SR, Bradshaw CP, Leaf PJ. The use of propensity scores to assess the generalizability of results from randomized trials. J R Stat Soc Ser A Stat Soc. 2011;174(2):369–386. doi: 10.1111/j.1467-985X.2010.00673.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lee BK, Lessler J, Stuart EA. Improving propensity score weighting using machine learning. Stat Med. 2010;29(3):337–346. doi: 10.1002/sim.3782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hojsgaard S, Halekoh U, Yan J. The R Package geepack for Generalized Estimating Equations. J Stat Softw. 2006;15(2):1– 11. [Google Scholar]
- 25.Van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Softw. 2011;45(3):1–67. [Google Scholar]
- 26.Ridgeway G, McCaffrey D, Morral A, Griffin BA, Burgette L. twang: Toolkit for Weighting and Analysis of Nonequivalent Groups. 2013 Available at: http://CRAN.R-project.org/package=twang.
- 27.R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. Available at: http://www.R-project.org/ [Google Scholar]
- 28.Kahneman D, Krueger AB, Schkade D, Schwarz N, Stone AA. Would you be happier if you were richer? A focusing illusion. Science. 2006;312(5782):1908–1910. doi: 10.1126/science.1129688. [DOI] [PubMed] [Google Scholar]
- 29.Schkade DA, Kahneman D. Does Living in California Make People Happy? A Focusing Illusion in Judgments of Life Satisfaction. Psychol Sci. 1998;9(5):340–346. doi: 10.1111/1467-9280.00066. [DOI] [Google Scholar]
- 30.Mosenthal AC, Murphy PA. Interdisciplinary model for palliative care in the trauma and surgical intensive care unit: Robert Wood Johnson Foundation Demonstration Project for Improving Palliative Care in the Intensive Care Unit. Crit Care Med. 2006;34(11 Suppl):S399–403. doi: 10.1097/01.CCM.0000237044.79166.E1. [DOI] [PubMed] [Google Scholar]
- 31.Cook D, Rocker G, Marshall J, et al. Withdrawal of mechanical ventilation in anticipation of death in the intensive care unit. N Engl J Med. 2003;349(12):1123–1132. doi: 10.1056/NEJMoa030083. [DOI] [PubMed] [Google Scholar]
- 32.Shalowitz DI, Garrett-Mayer E, Wendler D. The accuracy of surrogate decision makers: a systematic review. Arch Intern Med. 2006;166(5):493–497. doi: 10.1001/archinte.166.5.493. [DOI] [PubMed] [Google Scholar]
- 33.Nicholas LH, Langa KM, Iwashyna TJ, Weir DR. Regional variation in the association between advance directives and end-of-life Medicare expenditures. JAMA J Am Med Assoc. 2011;306(13):1447–1453. doi: 10.1001/jama.2011.1410. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Bischoff KE, Sudore R, Miao Y, Boscardin WJ, Smith AK. Advance Care Planning and the Quality of End-of-Life Care in Older Adults. J Am Geriatr Soc. 2013;61(2):209–214. doi: 10.1111/jgs.12105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kühne F, Behmann M, Bisson S, Walter U, Schneider N. Non-response in a survey of physicians on end-of-life care for the elderly. BMC Res Notes. 2011;4:367. doi: 10.1186/1756-0500-4-367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Yessis J, Rathert C. Initial versus Prompted Responders to Patient Satisfaction Surveys: Implications for Interpretation and Pateint Feedback. J Appl Manag Entrep. 2006;11(4):49–64. [Google Scholar]
- 37.Halbesleben JRB, Whitman MV. Evaluating Survey Quality in Health Services Research: A Decision Framework for Assessing Nonresponse Bias. Health Serv Res. 2013;48(3):913–930. doi: 10.1111/1475-6773.12002. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.