Abstract
Objective
To determine whether groups of surrogates for patients with severe acute brain injury (SABI) with poor prognosis can be identified based on their prioritization of goals-of-care (GOC) decisional concerns, an online survey of 1,588 adults recruited via a probability-based panel representative of the US population was conducted.
Methods
Participants acted as a surrogate for a GOC decision for a hypothetical patient with SABI and were randomized to 1 of 2 prognostic scenarios: the patient likely being left with a range of severe functional disability (SD) or remaining in a vegetative state (VS). Participants prioritized a list of 12 decisional concerns via best-worst scaling. Latent class analysis (LCA) was used to discover decisional groups.
Results
The completion rate was 44.6%; data weighting was conducted to mitigate nonresponse bias. For 792 SD respondents, LCA revealed 4 groups. All groups shared concerns regarding respecting patient wishes and minimizing suffering. The 4 groups were otherwise distinguished by unique concerns that their members highlighted: an older adult remaining severely disabled (34.4%), family consensus (26.4%), doubt regarding prognostic accuracy (20.7%), and cost of long-term care (18.6%). For the 796 VS respondents, LCA revealed 5 groups. Four of the 5 groups had similar concern profiles to the 4 SD groups. The largest (29.0%) expressed the most prognostic doubt. An additional group (15.8%) prioritized religious concerns.
Conclusions
Although surrogate decision makers for patients with SABI are concerned with respecting patient wishes and minimizing suffering, certain groups highly prioritize other specific decisional factors. These data can help inform future interventions for supporting decision makers.
Many intubated patients with severe acute brain injury (SABI) require goals-of-care (GOC) discussions regarding life-sustaining therapy (LST, e.g., tracheostomy and/or gastrostomy placement) vs comfort measures only (CMO).1 These decisions are often challenging for surrogate decision makers, not only because of prognostic uncertainty2 but also because surrogates often wrestle with multiple personal concerns and values.1,3 Intensive care unit (ICU) decision support to improve psychological outcomes among surrogates needs to address the personal concerns and values of surrogates directly.4
A number of studies conducted locally at academic centers have qualitatively explored the concerns that surrogates have in general when deciding on LST for critically ill patients with poor prognosis,5–8 including some with SABI.9 Expanding such research beyond academic centers and characterizing specific groups of people across the United States who share similar concerns and values when making such decisions are critical to designing interventions appropriately tailored to improve outcomes among these groups. The hypothesis of this survey study was that members of the general US population—when placed in the hypothetical situation of determining the GOC for an adult patient with SABI—can be classified into groups based solely on their decision-making concerns. Latent class analysis (LCA)10 was used to segment US residents into distinct clusters based on their prioritization of concerns. To explore the impact of varying prognoses, participants were randomized between 2 versions of the survey that differed only with respect to the severity of the poor prognosis presented for a hypothetical patient, with separate latent class analyses conducted for each survey.
Methods
The survey versions for this study were developed in 2015, with the main data for this study then collected between February 18 and 29, 2016, from a nationally representative online survey of US adults aged 30 years and older. The following sections outline the survey development process first, followed by descriptions of the sample, data collection, and statistical design for the main study.
Standard protocol approvals, registrations, and patient consents
The study was approved by the Yale Human Investigation Committee (protocols #1406014207 and #1505015893).
Survey instrument development
Two versions of the survey instrument were drafted by the authors that both asked a participant to consider an 85-year-old, previously healthy, close family member admitted to an ICU following a severe intracerebral hemorrhage and remaining intubated and poorly responsive despite 2 weeks of care without limitations (full surveys available in Dryad, doi.org/10.5061/dryad.z8w9ghx78, appendix e-1). Both versions of the survey then asked the participant to imagine that they were the patient's surrogate decision maker for possible tracheostomy and gastrostomy placement (i.e., LST).
The 2 survey versions that were developed differed with regard to the long-term poor prognosis presented. One version—the more common real-life scenario—stated that the patient would most likely remain unresponsive, with a small but unlikely chance that the patient could gradually recover the ability to speak, eat, recognize others, and/or travel in a wheelchair with time (the severe disability [SD] survey). The other version stated a less common but more severe scenario that the patient would remain bedbound and unresponsive long term with certainty (the vegetative state [VS] survey). Both survey versions specified that the patient would be ventilator-free following tracheostomy and that the patient did not have an advance directive.
For both versions of the survey, a list of concerns that a surrogate could prioritize when deciding on LST vs CMO for an intubated patient with SABI was also developed (list available in Dryad, doi.org/10.5061/dryad.z8w9ghx78, table e-1). To review the drafted hypothetical scenarios and to create the list of decision-making concerns, semistructured interviews were conducted with 11 family members of former patients in the Yale New Haven Hospital (YNHH) Neuroscience ICU (6 of whom had requested the ICU make their relatives CMO). Family members who provided input were recruited by telephone within 8 months of their neuroscience ICU experience. Detailed notes were taken as the family members (1) provided feedback on the wording of the drafted hypothetical scenarios and (2) listed what their own top decisional concerns would be if acting as a surrogate decision maker. They were also asked whether they agreed or disagreed with concerns that other families had expressed in prior published qualitative research regarding similar ICU GOC decisions.5
Interviews with family members were conducted until thematic saturation regarding the decision-making concerns was achieved, with a total of 10 concerns attained. Fifteen national experts—in the fields of neurology, critical care, palliative care, geriatrics, ethics, and medical decision making with publication track records on the topic of ICU GOC decision making—were then contacted by email, given revised copies of the hypothetical scenarios and the list of 10 concerns elicited from the family interviews, and asked to provide additional feedback.
The final list of 12 decision-making concerns in table e-1 were used to create a 12-question best-worst scaling (BWS) exercise used in both survey versions to allow participants to prioritize their concerns based on the hypothetical scenario presented. A BWS question asks respondents to choose the best (or most concerning) and the worst (or least concerning) item from a series of sets containing different combinations of items from a master list. An example is provided in appendix e-1. BWS questions are easy to understand, and respondents from diverse socioeconomic backgrounds have been shown to provide reliable data. The format avoids scale-related response bias and is thus more efficient than rating scales.11 BWS has been used to assess patients' preferences in a wide variety of health care settings.12
The question design used in the BWS application was based on a Youden balanced incomplete block design.13,14 Each of the 12 questions presented 4 potential decision-making concerns, with each concern presented an equal number of times in each version of the survey, and each concern presented with each of the other 11 concerns an equal number of times.
Following the BWS exercise, both survey versions asked participants about their decision regarding requesting LST vs CMO for the hypothetical patient and their certainty about the decision on a 4-point scale.
Surveys were designed using Sawtooth Software, Inc. (SSI) Web v8.2. Both versions of the survey underwent cognitive testing among a total of 11 ethnically diverse subjects recruited from the local community around YNHH. The final survey versions incorporated feedback and were both expected to take at least 8 minutes to complete based on testing.
Sample
Following survey instrument development, participants for the main study were recruited from the GfK (now Ipsos) KnowledgePanel, a probability-based online panel designed to be representative of all noninstitutionalized civilian adults living in US households. The 55,000 active panel members cover 97% of US households, and the panel sample is updated every 3 months through address-based sampling to include those who are served only by cellphones or have no telephone service. Selected panel members who do not have computers or internet service are provided with both.15
Because the minimum age of real-life surrogate decision makers in previous studies in neuro-ICUs was approximately 30 years, this age was used as the minimum age for recruitment of participants from the survey panel.16,17
Data collection
Following a 2-day operational test period with 36 participants, the KnowledgePanel members accessed the surveys using a link sent by the company and were randomized to 1 of the 2 survey versions, with their differing prognostic scenarios. Any respondent completing a survey in less than 8 minutes was excluded. The KnowledgePanel provided the following basic demographic information regarding participants: age, sex, race, education, household internet access, region of residence, marital status, employment status, and income. All other demographic variables were self-reported by respondents directly in response to additional survey questions.
Statistical analysis
Data from the SD and VS survey versions were analyzed separately. To minimize bias due to differential nonresponse in the KnowledgePanel recruitment process and nonresponse during the field period, iterative proportional fitting was used to weight the survey data and geodemographically match the US population aged 30 years and older, based on the 2015 Current Population Survey.18
LCA10 was performed on the data from the surveys' BWS exercise using Sawtooth Software's Latent Class Segmentation Module v4 to estimate part-worth utilities (i.e., importance scores for each of the concerns listed in the surveys), scaled from 0 to 100, and to determine previously unmeasured class membership among participants. Class solutions were replicated 20 times from random starting seeds, with a maximum of 5 groups to allow for possible heterogeneity of survey responses.
For both versions of the survey, the Bayesian Information Criterion19 and Akaike Information Criterion20 decreased as the number of groups increased, suggesting improving fit. However, for the SD survey, 2 of the groups in the 5-group solution had utility distributions that were nearly identical (data available in Dryad, doi.org/10.5061/dryad.z8w9ghx78, figure e-1), which made 4 groups the best practical solution.
For both survey versions, associations between participant characteristics and group membership were explored by comparing univariate data and subsequently performing multinomial logistic regression including only those surveys with all questions answered. Analyses were conducted using SAS software v14 (SAS Institute, Inc., Cary, NC).
Sample size
Factors affecting the performance of latent class analyses include the number of items in the BWS exercise, the manner in which different concerns cluster among participants, and the eventual size of each group in the best solutions.10 Meaningful segmentation analyses have been performed in the medical literature with as few as 183 respondents for a survey that contained as many as 18 items to prioritize.21 In comparison, recruitment for this study obtained nearly 800 respondents for each of two 12-item BWS exercises.
Data availability
Anonymized data from this study can be shared by request by any qualified investigator for the purposes of replicated procedures and results.
Results
Enrollment
Figure 1 contains enrollment information. Overall, 44.6% (1,588/3,559) of potential participants completed their survey in 8 minutes or longer (the time needed to complete the survey in credible fashion during the cognitive testing portion of survey development) and were included in the analysis. There were no demographic differences between the participants randomized to the SD vs VS survey, and the samples for both versions reflected the geodemographic distribution of the US population aged 30 years or older after weighting (data available in Dryad, doi.org/10.5061/dryad.z8w9ghx78, table e-2).
Figure 1. Study enrollment.
SD = severe disability; VS = vegetative state.
Results for the SD survey
Latent class analysis
Figure 2 shows the 4-group solution for the SD survey. All groups ranked the following 4 concerns among their top 5: (1) “Doing what my loved one would have wanted”; (2) “Wanting my loved one to experience as natural and comfortable a death as possible”; (3) “Worrying that my loved one would suffer if kept alive with tubes”; and (4) “Worrying that my loved one would suffer while dying if tube feeding and IV fluids were stopped.”
Figure 2. Latent class analysis results for the severe disability survey (n = 792).
The percentage next to each group number title is the weighted percentage of the entire survey cohort that best fits into that group. The numbers within each group pie chart are the part-worth utilities for each concern, with larger values representing greater concern. Utilities have been rescaled such that the utilities for all concerns within any given group sum to 100. Exploded portions of each pie chart are major concerns that each group found to be more concerning relative to the other groups.
Beyond these 4 shared concerns, each group was distinguished by a different distinct concern included within their top 5 concerns. The largest group of respondents (group 1, n = 272, weighted percentage 34.4%) believed that “Worrying that my loved one is too old to be kept alive and severely disabled” was relatively concerning. The second largest group (group 2, n = 216, 26.4%) was concerned with “Getting all of my family members to agree on what to do,” relative to other groups. The third largest group (group 3, n = 169, 20.7%) had a relative concern that “The doctor's opinion that my loved one would not improve is wrong.” This group was also more concerned about guilt and religious beliefs compared with the other groups. The smallest group (group 4, n = 135, 18.6%) had the highest concern for “Paying for long-term care for my loved one” but otherwise discriminated among the various concerns the least.
Demographics associated with groups
Table 1 compares demographics across these 4 groups of participants. Religious affiliation and frequency of attending religious services were associated with group membership (p < 0.0001 for χ2 test for both variables), with more than half of respondents (62.4%) in group 1 attending religious services only once a year or less. In a multinomial logistic regression model, all denominations of Christianity were independently associated with being classified outside of group 1 (table 2). Increased frequency of attending religious services independently increased the odds of being in group 3 (odds ratio 3.48 vs group 1, 95% confidence interval 1.80–6.75, for “Once a week or more” compared with “Once a year or less”).
Table 1.
Demographics and GOC decisions across the 4 groups of participants taking the severe disability survey (n = 792)
Table 2.
Multinomial logistic regression across the 4 groups of the severe disability survey participants (n = 784)
GOC decisions
Among all participants taking the SD survey, 104 of 785 (15.0%) selected LST as their GOC choice for the hypothetical patient. Table 1 reports GOC choices among the 4 decisional groups. Group 1 was least likely to select LST (5.2%) in pairwise comparisons with the other 3 groups. Group 3 appeared to be the most inclined toward LST (31.1%), but with the least degree of certainty (“Very certain” = 34.2%).
Results for the VS survey
Latent class analysis
Figure 3 shows the 5-group solution for the VS survey. Four of the 5 groups had concern profiles similar to the 4 groups found for the SD scenario. An additional, separate group (group 4) was more concerned about “Acting according to my religious beliefs” compared with the other groups and comprised 15.8% (n = 138) of respondents. In contrast to the SD survey respondents, the largest group among those taking the VS survey ranked prognostic accuracy and guilt as relatively concerning (group 1, n = 226, 29.0%), but self-reported little concern for religious beliefs.
Figure 3. Latent class analysis results for vegetative state survey (n = 796).
The percentage next to each group number title is the weighted percentage of the entire survey cohort that best fits into that group. The numbers within each group pie chart are the part-worth utilities for each concern, with larger values representing greater concern. Utilities have been rescaled such that the utilities for all concerns within any given group sum to 100. Exploded portions of each pie chart are major concerns that each group found to be more concerning relative to the other groups.
Demographics associated with groups
Table 3 compares demographics across the 5 groups of the VS survey participants. Variables significantly associated with group membership again included religious affiliation and frequency of attending religious services (in a pattern similar to the SD survey), as well as sex, household internet access, income, and years in the United States (p < 0.001 for all variables).
Table 3.
Demographics and GOC decisions across the 5 groups of participants taking the vegetative state survey (n = 796)
Table 4 contains the multinomial logistic regression results, with the group with the highest concern about the patient being an older adult (group 3) selected as the reference, as was done in the analysis for the SD survey analysis. In contrast to the SD survey results, increased frequency of attending religious services was not independently associated with the group most concerned with prognostic accuracy (group 1), but rather with group 4. Although male sex was associated with increased likelihood of being in group 5 and low income was associated with membership in groups 4 and 5, household internet access and years in the United States were not associated with group membership in the multivariate model.
Table 4.
Multinomial logistic regression across the 5 groups of the vegetative state survey participants (n = 782)
GOC decisions
Among all VS survey version respondents, 81 of 793 participants (11.4%) selected LST as their GOC choice for the hypothetical patient (compared with 15.0% of the SD survey respondents, p = 0.07). Similar to the SD survey results, the decisional group with the highest concern for prognostic accuracy (group 1) trended toward the highest percentage of members selecting LST (17.5%) but the lowest percentage of members who were “Very certain” (34.7%) (table 3).
Discussion
When asked to act as a surrogate for an intubated older adult relative with a severe brain injury and a prognosis of a range of very severe functional disability, most Americans shared concerns about respecting the patient's wishes and minimizing suffering. However, 1 in 4 surveyed otherwise ranked family consensus as the most important concern. One in 5 specified doubt in prognostic accuracy as otherwise most concerning, whereas nearly 1 in 5 were most concerned about paying for long-term care. When respondents were told in more narrow terms that the same patient instead was likely to be unresponsive long term (i.e., vegetative), closer to 1 in 3 participants rated doubt in prognostic accuracy as their most important decision-making factor aside from respecting the patient's wishes and minimizing suffering—concerns that were once again shared by all decisional groups.
Previous research has described factors that surrogate decision makers consider when making GOC decisions on behalf of patients in general ICUs, including the importance of family dynamics,5 prognostic uncertainty,22 guilt,6 and cost of care.8 These past studies have mostly consisted of qualitative interviews at a limited number of academic medical centers and, with few exceptions,9 have not focused specifically on neurologic patients. More importantly, these studies have not attempted (1) to define how groups of surrogates for patients with SABI ultimately choose to prioritize their various (and sometimes competing) concerns when making GOC decisions or (2) to describe how prevalent these decisional groups may be, relative to one another. It is possible that attempting to characterize such groups has not been done before because (1) it is difficult to create an exhaustive list of possible decisional concerns that any given surrogate could have, (2) many of these concerns may be shared among all surrogates to a near-universal degree, and (3) ultimately clinicians recognize the need to avoid blanket characterizations and carefully attend to each individual surrogate's unique set of concerns. Nevertheless, currently, there is a clear need for new interventions to improve neurologic shared decision-making processes,23,24 and evidence is emerging that it is difficult to demonstrate the impact of interventions such as decision aids if they are simply deployed across all types of ICU surrogates,25 regardless of their main decisional concerns. The analyses of this study enable the future design of educational tools and/or interventions for improving shared decision making for patients with SABI—e.g., programs to enhance communication,26–28 decision aids,25,29,30 or other novel approaches—to focus on a target decision group (or groups) that comprises a significant portion of the US population with similar top concerns. For example, in the case of a cognitive decision aid—one that mostly provides information on concepts such as tracheostomy, feeding tube placement, long-term care, and comfort care—targeting such an intervention toward the group of surrogates most concerned with patients' advanced age and comfort may prove to be higher yield than administering the same tool to surrogates in other decisional groups, which may require other strategies to supplement shared decision making.
The finding that significant proportions of participants in both versions of the survey were concerned about the medical team's prognostic accuracy is consistent with the literature that many ICU families consider numerous sources outside of clinician opinion in formulating personal impressions of prognosis.31 In particular, the association between having concerns regarding prognostic accuracy and acting according to religious beliefs among the SD survey respondents is consistent with previous research.32 However, among the VS survey respondents, the group with the highest concern regarding prognostic accuracy represented a higher proportion of the overall sample, and—unlike the SD survey results—membership in this group was not associated with increased frequency of attending religious services. This observation may suggest that many surrogates, irrespective of religiosity, may simply question how clinicians can provide a prediction of neurologic outcome as narrow as a long-term VS with certainty.33,34 How best to address surrogates' doubt regarding clinicians' prognostic estimates is a complicated problem in neurology, but it is possible that providing a range of best and worst possible outcomes in such discussions may help take the focus of the surrogate off of a concern regarding the clinician's ability to predict an exact future and direct the surrogate toward working through other important concerns.35
A major strength of this study is that survey coverage bias was mitigated by using a respondent panel specifically designed to represent a true probability sample of the US population. This panel captured geodemographic groups whose decisional priorities might be hard to ascertain, especially those who may not have otherwise had access to a computer and the internet.15 A second strength of this study is that the BWS format that was used in the surveys avoided scale-related response bias. This design allowed for efficient assessment of the degree to which respondents prioritized multiple concerns, despite that respondents were asked to make a complex multifactorial decision.11
Regarding the study sample, limitations include the fact that 44.6% of invited respondents began and completed 1 of the 2 survey versions. Those participants who took the survey more quickly than was judged to be the time necessary to formulate well thought-through responses (based on piloting) were excluded, an approach that helped to preserve the validity of the data analyzed but that contributed to a reduction in the reported overall survey completion rate. Collecting additional data on nonresponders and those participants who took the survey in less than 8 minutes could have provided additional insight on the nature of any sampling bias. However, these biases for this study were ultimately addressed by weighting the survey data to match the sample geodemographically with the US population.18 A larger sample size for both surveys would have potentially allowed for more detailed analysis of various racial groups.
The use of a hypothetical scenario for the purposes of eliciting respondents' decision-making concerns is also a potential limitation. This survey design enabled the study to reach a large probability sample of the US population that would otherwise have been challenging but raises the issue of experimental realism—the question regarding whether respondents might have different priorities when placed in real-world situations.36 Several points may mitigate this concern. First, a substantial proportion of survey participants (36.9%) self-reported having made a similar decision in the past (table e-2). Second, the hypothetical scenarios and list of concerns that were presented in the survey were validated among families of neuroscience ICU patients and content experts during the survey development process. Third, multiple previous studies have used hypothetical scenarios to provide generalizable insights into the nature of GOC decisions to avoid the ethical challenges of surveying families in the midst of or immediately following actual life-or-death decisions for their loved ones.37,38
Of note, the hypothetical patient in each survey was not presented as having any preexisting disabilities or major comorbid conditions in an effort to isolate respondents' concerns regarding SABI with poor prognosis, in and of itself. Conversely, the patient in each survey was presented as being of advanced age, as opposed to a younger patient with more uncertain prognosis, so as not to have the fundamental uncertainty of each scenario overwhelm respondents' prioritization of concerns. Appropriate caution should be taken in extrapolating the decisional groups discovered in this study to other clinical scenarios.
Future educational tools and ICU-based interventions aimed at improving shared decision making for surrogates of intubated patients with SABI with poor prognosis will need to go beyond simply asking surrogates to exercise substituted judgment on behalf of incapacitated patients and addressing near-universal concerns about how patient suffering might be minimized. To be high yield, interventions may also need to attend to concerns of achieving family consensus, trusting a poor prognosis (either for religious or nonreligious reasons), and understanding financial implications of prolonging care. Neurologists should continue to recognize that any individual surrogate may prioritize their concerns in a manner that overlaps these groups. However, the results of this study provide an empiric framework that may help tailor future tools and interventions toward decisional groups known to be highly prevalent across the United States and thus help maximize their effectiveness.
Acknowledgment
The authors thank Dianne Rucinski (Office of Minority Health, US Department of Health and Human Services) for her assistance in initiating this project. They also thank Venkatesh Aiyagari (UT Southwestern), James Bernat (Dartmouth Geisel School of Medicine), Christopher Cox (Duke University School of Medicine), Claire Creutzfeldt (University of Washington Medicine), Alex Dubov (Loma Linda University), Terri Fried (Yale School of Medicine), Adam Kelly (University of Rochester Medical Center), Barbara Lutz (University of North Carolina–Wilmington School of Nursing), Kazuma Nakagawa (The Queen's Medical Center), Maisha Robinson (Mayo Clinic Jacksonville), and Darin Zahuranec (University of Michigan) for their input during survey creation. Finally, they thank collaborators at GfK/Ipsos for their expertise in the survey data collection.
Glossary
- BSW
best-worst scaling
- CMO
comfort measures only
- GOC
goals-of-care
- ICU
intensive care unit
- LCA
latent class analysis
- LST
life-sustaining therapy
- SABI
severe acute brain injury
- SD
severe disability
- VS
vegetative state
- YNHH
Yale New Haven Hospital
Appendix. Authors


Footnotes
Editorial, page 813
Podcast: NPub.org/hpe2a3
Study funding
This study was funded by an American Brain Foundation Practice Research Training Fellowship, a Neurocritical Care Society Research Training Fellowship Grant, and a philanthropic grant from the Apple Pickers Foundation in Westerly, RI (D.Y. Hwang). Research reported in this publication was also supported by the Rheumatology Research Foundation Innovative Research Grant program and the National Institute of Arthritis and Musculoskeletal and Skin Diseases, under award number AR060231-06 (L. Fraenkel), as well as the National Heart, Lung, and Blood Institute under award number K24 HL148314 (D.B. White).
Disclosure
D.Y. Hwang reports no disclosures, aside from study funding. A.K. Knies, D. Mampre, S. Kolenikov, M. Schalk, H. Hammer, D.B. White, R.G. Holloway, and K.N. Sheth report no disclosures. L. Fraenkel reports no disclosures, aside from study funding. Go to Neurology.org/N for full disclosures.
References
- 1.Cai X, Robinson J, Muehlschlegel S, et al. Patient preferences and surrogate decision making in neuroscience intensive care units. Neurocrit Care 2015;23:131–141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lazaridis C. Withdrawal of life-sustaining treatments in perceived devastating brain injury: the key role of uncertainty. Neurocrit Care 2019;30:33–41. [DOI] [PubMed] [Google Scholar]
- 3.Rid A, Wendler D. Can we improve treatment decision-making for incapacitated patients? Hastings Cent Rep 2010;40:36–45. [DOI] [PubMed] [Google Scholar]
- 4.Tannenbaum AM, Halpern SD. Knowing is (only) half the battle. Ann Intern Med 2019;170:338–339. [DOI] [PubMed] [Google Scholar]
- 5.Meeker MA, Jezewski MA. Metasynthesis: withdrawing life-sustaining treatments: the experience of family decision-makers. J Clin Nurs 2009;18:163–173. [DOI] [PubMed] [Google Scholar]
- 6.Schenker Y, Crowley-Matoka M, Dohan D, et al. I don't want to be the one saying “we should just let him die”: intrapersonal tensions experienced by surrogate decision makers in the ICU. J Gen Intern Med 2012;27:1657–1665. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Limerick MH. The process used by surrogate decision makers to withhold and withdraw life-sustaining measures in an intensive care environment. Oncol Nurs Forum 2007;34:331–339. [DOI] [PubMed] [Google Scholar]
- 8.Braun UK, Beyth RJ, Ford ME, et al. Voices of African American, Caucasian, and Hispanic surrogates on the burdens of end-of-life decision making. J Gen Intern Med 2008;23:267–274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Quinn T, Moskowitz J, Khan MW, et al. What families need and physicians deliver: contrasting communication preferences between surrogate decision-makers and physicians during outcome prognostication in critically ill TBI patients. Neurocrit Care 2017;27:154–162. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.McCutcheon AL. Latent Class Analysis. Beverly Hills: Sage Publications; 1987. [Google Scholar]
- 11.Flynn TN, Marley AJ. Best-worst scaling: theory and methods. In: Handbook of Choice Modeling. Hess S, Daly A, editors. Cheltenham: Edward Elgar Publishing Limited, 2014:178–201. [Google Scholar]
- 12.Flynn TN, Louviere JJ, Peters TJ, et al. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ 2007;26:171–189. [DOI] [PubMed] [Google Scholar]
- 13.Youden WJ. Use of incomplete block replications in estimating tobacco-mosaic virus. Contrib Boyce Thompson Inst Plant Res 1937;8:41–48. [Google Scholar]
- 14.Youden WJ. Experimental designs to increase accuracy of greenhouse studies: contributions. Boyce Thompson Inst Plant Res 1940;11:219–228. [Google Scholar]
- 15.Ipsos. KnowledgePanel® Overview [Internet]. Ipsos; 2018. Available at: www.ipsos.com/sites/default/files/18-11-53_Overview_v3.pdf. Accessed January 15, 2019. [Google Scholar]
- 16.Hwang DY, Yagoda D, Perrey HM, et al. Assessment of satisfaction with care among family members of survivors in a neuroscience intensive care unit. J Neurosci Nurs 2014;46:106–116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Weber U, Johnson J, Anderson N, et al. Dedicated afternoon rounds for ICU patients' families and family satisfaction with care. Crit Care Med 2018;46:602–611. [DOI] [PubMed] [Google Scholar]
- 18.KnowledgePanel®. Recruitment and Sample Survey Methodologies [Internet]. GfK; 2019. Available at: gfk.com/fileadmin/user_upload/dyna_content/US/documents/KnowledgePanel_Methodology.pdf. Accessed January 15, 2019. [Google Scholar]
- 19.Schwarz G. Estimating dimension of a model. Ann Stat 1978;6:461–464. [Google Scholar]
- 20.Akaike H. New look at statistical-model identification. Ieee T Automat Contr 1974;19:716–723. [Google Scholar]
- 21.Waschbusch DA, Cunningham CE, Pelham WE, et al. A discrete choice conjoint experiment to evaluate parent preferences for treatment of young, medication naive children with ADHD. J Clin Child Adolesc Psychol 2011;40:546–561. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Apatira L, Boyd EA, Malvar G, et al. Hope, truth, and preparing for death: perspectives of surrogate decision makers. Ann Intern Med 2008;149:861–868. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Davidson JE, Aslakson RA, Long AC, et al. Guidelines for family-centered care in the neonatal, pediatric, and adult ICU. Crit Care Med 2017;45:103–128. [DOI] [PubMed] [Google Scholar]
- 24.Creutzfeldt CJ, Kluger B, Kelly AG, et al. Neuropalliative care: priorities to move the field forward. Neurology 2018;91:217–226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Cox CE, White DB, Hough CL, et al. Effects of a personalized web-based decision aid for surrogate decision makers of patients with prolonged mechanical ventilation: a randomized clinical trial. Ann Intern Med 2019;170:285–297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Shaw DJ, Davidson JE, Smilde RI, et al. Multidisciplinary team training to enhance family communication in the ICU. Crit Care Med 2014;42:265–271. [DOI] [PubMed] [Google Scholar]
- 27.White DB, Angus DC, Shields AM, et al. A randomized trial of a family-support intervention in intensive care units. NEJM 2018;378:2365–2375. [DOI] [PubMed] [Google Scholar]
- 28.Curtis JR, Treece PD, Nielsen EL, et al. Randomized trial of communication facilitators to reduce family distress and intensity of end-of-life care. Am J Respir Crit Care Med 2016;193:154–162. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Van Scoy LJ, Chiarolanzio PJ, Kim C, et al. Development and initial evaluation of an online decision support tool for families of patients with critical illness: a multicenter pilot study. J Crit Care 2017;39:18–24. [DOI] [PubMed] [Google Scholar]
- 30.Muehlschlegel S, Shutter L, Col N, et al. Decision aids and shared decision-making in neurocritical care: an unmet need in our NeuroICUs. Neurocrit Care 2015;23:127–130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Boyd EA, Lo B, Evans LR, et al. It's not just what the doctor tells me: “factors that influence surrogate decision-makers” perceptions of prognosis. Crit Care Med 2010;38:1270–1275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.White DB, Ernecoff N, Buddadhumaruk P, et al. Prevalence of and factors related to discordance about prognosis between physicians and surrogate decision makers of critically ill patients. JAMA 2016;315:2086–2094. [DOI] [PubMed] [Google Scholar]
- 33.Evans LR, Boyd EA, Malvar G, et al. Surrogate decision-makers' perspectives on discussing prognosis in the face of uncertainty. Am J Respir Crit Care Med 2009;179:48–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Smith AK, White DB, Arnold RM. Uncertainty—the other side of prognosis. NEJM 2013;368:2448–2450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Holloway RG, Gramling R, Kelly AG. Estimating and communicating prognosis in advanced neurologic disease. Neurology 2013;80:764–772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Aronson E, Carlsmith JM. Experimentation in social psychology. In: The Handbook of Social Psychology, Lindzey G, Aronson E, editors. 2nd ed. Reading: Addison-Wesley; 1968:1–79. [Google Scholar]
- 37.Zier LS, Sottile PD, Hong SY, et al. Surrogate decision makers' interpretation of prognostic information: a mixed-methods study. Ann Intern Med 2012;156:360–366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Chapman AR, Litton E, Chamberlain J, et al. The effect of prognostic data presentation format on perceived risk among surrogate decision makers of critically ill patients: a randomized comparative trial. J Crit Care 2015;30:231–235. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Anonymized data from this study can be shared by request by any qualified investigator for the purposes of replicated procedures and results.









