Abstract
Background:
Although goals-of-care discussions are important for high-quality palliative care, this communication is often lacking for hospitalized older patients with serious illness. Electronic health records (EHR) provide an opportunity to identify patients who might benefit from these discussions and promote their occurrence, yet prior interventions using the EHR for this purpose are limited. We designed two complementary yet independent randomized trials to examine effectiveness of a communication-priming intervention (Jumpstart) for hospitalized older adults with serious illness.
Methods:
We report the protocol for these 2 randomized trials. Trial 1 has two arms, usual care and a clinician-facing Jumpstart, and is a pragmatic trial assessing outcomes with the EHR only (n = 2000). Trial 2 has three arms: usual care, clinician-facing Jumpstart, and clinician- and patient-facing (bi-directional) Jumpstart (n = 600). We hypothesize the clinician-facing Jumpstart will improve outcomes over usual care and the bi-directional Jumpstart will improve outcomes over the clinician-facing Jumpstart and usual care. We use a hybrid effectiveness-implementation design to examine implementation barriers and facilitators.
Outcomes:
For both trials, the primary outcome is EHR documentation of a goals-of-care discussion within 30 days of randomization; additional outcomes include intensity of end-of-life care. Trial 2 also examines patient- or family-reported outcomes assessed by surveys targeting 3–5 days and 4–8 weeks after randomization including quality of goals-of-care communication, receipt of goal-concordant care, and psychological symptoms.
Conclusions:
This novel study incorporates two complementary randomized trials and a hybrid effectiveness-implementation approach to improve the quality and value of care for hospitalized older adults with serious illness.
Clinical Trials Registration:
STUDY00007031-A and STUDY00007031-B.
Keywords: Goals-of-care, Serious illness communication, Palliative care, Health services research
1. Introduction
Communication about goals of care is an important aspect of palliative and end-of-life care, yet our relative inability to conduct and document this communication with seriously ill patients is a shortcoming in our healthcare system [1–6]. While the value of advance care planning about hypothetical future health states with healthy individuals is a topic of debate, there is consensus about the value of improving goals-of-care discussions for those with serious illness facing difficult treatment decisions [7–12]. Goals-of-care communication has been associated with improved patient and family outcomes as well as reduced intensity of care at the end of life [1,13]. Electronic health records (EHR) provide a key opportunity to identify patients who would most benefit from goals-of-care discussions and to promote these discussions, yet prior interventions have not used the EHR to accomplish these goals. This gap was highlighted in two National Institute on Aging (NIA) consensus conferences [14,15].
Goals-of-care discussions are particularly salient for older adults with chronic, life-limiting illness who are especially vulnerable to inappropriately intensive end-of-life care [16,17]. Intensive care unit (ICU) admissions may worsen mortality for some patients [18] and, for the growing numbers of adults with Alzheimer’s disease and related dementias (ADRD) [19], end-of-life care is increasingly marked by intensive care that confers no survival benefit [17,20,21]. We were particularly interested in examining the effect of the intervention for patients with ADRD given the increasing numbers of patients with ADRD and the rising use of intensive care for this population [17,20,21].
This protocol builds on two successful programs, one using the EHR to identify seriously ill patients and documentation of goals-of-care discussions [22–27], and the other an intervention shown to promote and improve these discussions among outpatients with serious illness [2,3]. In the first program, we developed natural language processing methods (NLP) to automatically identify EHR documentation of goals-of-care discussions for patients with serious illness [22]. In the second, we conducted a randomized trial that successfully prompted outpatient clinicians to complete and document goals-of-care discussions for patients with serious illness using a bi-directional “Jumpstart” intervention. The bi-directional (delivered to both patients and clinicians) Jumpstart is a communication-priming intervention that uses pre-encounter patient survey responses to populate one-page guides that summarize patient-specific information about preferences for communication and care, as well as tips to improve this communication [2,28]. This guide is delivered to patients (to prepare them to talk with clinicians) and clinicians (to give tips for goals-of-care communication). This intervention increased goals-of-care discussions from 31% to 74% of routine clinic visits (p < 0.001) and increased patient-assessed quality of communication (p < 0.001) [2,3]. However, surveying patients or family members prior to the intervention makes this intervention challenging to implement in routine practice due to the resources required and may not be necessary to achieve some of the benefit of this intervention.
In this protocol paper, we describe 2 complementary yet independent trials among hospitalized older adults with serious illness, including adults with ADRD. The first, Trial 1, is a large pragmatic trial (n = 2000) comparing usual care with a patient-specific, EHR-populated clinician-facing Jumpstart designed to prompt clinicians to initiate goals-of-care discussions. The second, Trial 2, is an effectiveness trial with three arms that compares a) this EHR-populated, clinician-facing Jumpstart, b) the survey-populated, bi-directional Jumpstart, and c) usual care (n = 600). We hypothesize the clinician-facing Jumpstart will improve outcomes compared to usual care, and the bi-directional Jumpstart will improve outcomes compared to the clinician-facing Jumpstart and also to usual care. In both trials, we are using a Type 1 hybrid effectiveness-implementation approach to examine implementation of the interventions [29].
2. Methods
2.1. Setting
The trials are being conducted at three UW Medicine hospitals: University of Washington Medical Center (UWMC)-Montlake, Harborview Medical Center, and UWMC-Northwest. UWMC-Montlake is a quaternary-care university hospital and academic medical center that provides subspecialty care to the Pacific Northwest region; it has 529 acute care beds and 75 ICU beds. Harborview Medical Center is a county-owned tertiary care hospital and regional referral center, and the sole Level 1 Trauma Center for a five-state region; it has 413 acute care beds and 94 ICU beds. UWMC-Northwest is an academically-affiliated community hospital with 218 acute care beds and 15 ICU beds, and serves a large geriatric and nursing home resident population.
2.1.1. Patient population
Eligible patients will be hospitalized, 55 years of age or older, and identified by ICD-10 codes documented in the EHR during the 2 years prior to the hospitalization that indicate one or more of the nine chronic conditions used by the Dartmouth Atlas to study end-of-life care [30]: dementia, cancers of poor prognosis, chronic pulmonary disease, coronary artery disease, heart failure, chronic liver disease, chronic renal disease, diabetes with end-organ damage, and peripheral vascular disease. These 9 conditions account for 90% of deaths among Medicare beneficiaries in the US [31,32]. To increase inclusivity of important and under-studied populations, we also include all hospitalized patients over age 80 [33,34]. Among patients meeting any of these criteria, we include only those with no identified documentation of goals-of-care discussions during the current admission prior to randomization as determined through daily screening of hospitalized patients [22–27]. In trial 2, eligibility for patients include the ability to speak English well enough to complete surveys. If patients are unable to participate in Trial 2, eligibility for family members include being a legal next of kin and ability to speak English well enough to complete surveys.
Trial 1 is a pragmatic design using a waiver of informed consent to enroll all eligible patients. The rationale for a waiver of informed consent is that the intervention is designed to promote standard of care. We estimate 20% of the planned sample to have ADRD. For Trial 2, patients or their legal surrogate decision-maker consent and complete a baseline survey without formal over-sampling but with a focus on recruiting eligible patients with ADRD first to provide some pragmatic over-sampling of patients with ADRD. We estimate enrolling participants such that up to 40% have ADRD. Trial 1 has the advantage of enrolling all eligible patients and therefore results will be more generalizable than Trial 2. However, Trial 2 has the advantage of including survey-based outcomes providing more information about effectiveness of the intervention. These trials could have been combined into a single trial, but this approach would not have allowed Trial 1 to capitalize on the pragmatic feature of enrolling all eligible patients due to the requirement of informed consent for survey administration. These two trials are independent and sequential with completion of Trial 1 first. Fig. 1 shows the overall trial design (Fig. 1). For Trial 2, study staff use a brief six-item screening tool to assess cognitive impairment among patients [35]. If patients do not pass the cognitive screen, the legal surrogate decision-maker is asked to complete the surveys.
2.1.2. Randomization
Patients are randomized in a 1:1 ratio in Trial 1 and 1:1:1 ratio in Trial 2 using variable size blocks and stratified for hospital and ADRD vs. no ADRD. Participating family members or legal surrogate decision makers are assigned the same arm as the corresponding patient. Study coordinators conducting screening and enrollment are blinded to assignment until screening and enrollment are complete.
2.2. Interventions
2.2.1. Clinician-facing Jumpstart
First, we use automated methods to examine inpatient and outpatient EHR notes prior to the current admission, identifying current code status as well as all prior Physician Orders for Life-Sustaining Treatments (POLST) forms and advance directives; this information is included on Jumpstart Guides to inform discussions. Second, we deliver the Jumpstart Guide to the primary hospital team (all attending and resident physicians and advanced practice providers) via secure email and a page alerting the physicians to the presence of the Jumpstart Guide in their email, with the addition of in-person delivery of a paper version in Trial 2 only (see online appendix for example Jumpstart guides).
2.2.2. Bi-directional Jumpstart
First, information about the patient is abstracted from the EHR in the same way as for the Clinician-facing Jumpstart. Second, patients or their legal surrogate decision-maker complete baseline survey items assessing three domains: a) preferences for discussions about goals of care; b) barriers and facilitators for having such discussions; and c) current goals of care. Third, using the EHR and baseline survey, we use the automated algorithm from our prior trial [2] adapted to the hospital setting using human-centered design methods [36] to create a survey-informed Jumpstart Guide to prompt and guide goals-of-care discussions between the patient and hospital team or, if the patient isn’t able, the family member and the hospital team (see supplement for sample Jumpstart Guide). Finally, in the fourth step, we deliver the Jumpstart Guides to the primary team via secure email similar to Trial 1, as well as in-person delivery to members of the team. We also provide a survey-informed Jumpstart Guide to the patient or family, adapted with language specifically for the patient and family. All Jumpstart Guides are delivered on the day of randomization with the goal of prompting a goals-of-care discussion early during hospitalization, as supported by the National Quality Forum [37]. The bi-directional Jumpstart includes both EHR- and patient-tailored suggestions for conducting goals-of-care discussions based on survey responses. The suggestions are guided by the educational experience of VitalTalk, a nationally-acclaimed program for teaching serious illness communication, and adapted to the inpatient setting [38,39], as well as by a human-centered design exercise with hospital clinicians [36].
2.2.2.1. Outcomes from the EHR and Death Certificates (Trials 1 and 2).
The primary outcome for both trials is EHR documentation of goals-of-care discussions within 30 days after randomization. Our rationale for this as the primary outcome is that this is the primary target for the interventions and important to diverse stakeholders including patients and their families [7,22,40–42]. We will use supervised-machine-learning-based NLP to measure the primary outcome. In this approach, a dataset of EHR records external to the trial is first annotated by human abstractors to identify passages representing documented goals-of-care discussions. The annotated data are then used to train an NLP model to predict the presence or absence of documented goals-of-care discussions in unannotated EHR text collected from the trial itself [43,44]. Our research group has developed and reported the performance of bag-of-words logistic-regression models trained on 3183 EHR notes (689 positive) collected from previous trials [2,45] that suggested an area under the receiver operating characteristic curve (AUCROC) of 0.943 for classifying the presence or absence of documented goals-of-care discussions in a given clinical note [46]; and, hybrid rule- and bag-of-words logistic-regression models trained on 4391 EHR notes (99 positive) collected from 150 participants in a previous trial [47] of a communication-priming intervention that suggested an AUCROC up to 0.932 for classifying the cumulative incidence of documented goals-of-care discussions in a given patient-hospitalization [48]. Although this degree of accuracy is likely adequate for the proposed trial, we are actively developing deep learning methods that should improve the performance and generalizability of NLP toward this task [48–50]. We will manually review the EHR for goals-of-care discussions using standard EHR abstraction methods [46,48] for a randomly selected subset of patients in each trial to evaluate potential misclassification by NLP. We will also have the option of manually confirming NLP-identified goals-of-care discussions if NLP performance is suboptimal, as has been done by others [51].
Additional outcomes for both trials, obtained from the EHR, include utilization metrics associated with intensity of care (i.e., any ICU admissions, any ED visits, any palliative care consultations, and ICU- and hospital-free days); these outcomes will be assessed at 30 and 90 days after randomization. ICU- and hospital-free days are defined as the number of days alive and outside of the ICU (or hospital) within the specified time period after randomization (i.e. 30 days or 90 days) [52,53]. We will also examine the following outcomes: 1) time to first goals-of-care discussion during the 30 days after randomization; 2) occurrence of any hospital readmissions within 7 and 30 days after discharge from the index hospitalization; and 3) mortality status at 90 days and 1 year after randomization. Costs of care during hospital admission and 30- and 90-days following randomization will be obtained from institutional billing systems. Washington State death certificate data will be used to examine mortality after hospital discharge (Table 1).
Table 1.
Major outcome measures | concept | Data collection: source & time |
---|---|---|
| ||
Aims 1 and 2 outcomes | ||
EHR documentation of goals-of-care discussion (Primary outcome for both Trials) | Goals-of-care discussion | EHR: 30 days post randomization |
ICU use, ICU and hospital free days | Intensity of care | EHR: 30 days post randomization |
7- and 30-day ICU and hospital readmissions | Intensity of care | EHR: 7 and 30 days following hospital discharge |
Costs of care | Intensity of care/ intervention costs | EHR: During hospital stay, 30 and 90 days post-randomization. |
All-cause mortality at 90 days and 1 year | All-cause mortality | Washington State death certificates |
Aim 2 outcomes (not used in Aim 1 since the Trial 1 is a pragmatic trial without contact with patients or family members) | ||
Patient/family-reported discussion of goals [2,28] | Goals-of-care discussion occurrence | Survey: 3–5 days & 4–6 weeks post-randomization |
Quality of Communication (QOC) [54,55,57] | Quality of communication | Survey: 3–5 days post randomization |
SUPPORT question [60] | Goal-concordant care | Survey: 3–5 days & 4–6 weeks post-randomization |
HADS – anxiety and depression [65,66] | Symptoms of anxiety & depression | Survey: 4–6 weeks post-randomization |
EQ-5D-5L | Health-related QOL | Survey: 4–6 weeks post-randomization |
Patient/family reported ED, hospitalization and outpatient utilization | Healthcare utilization | Survey: 4–6 weeks post-randomization |
CollaboRATE [95] | Shared decision-making | Survey: 3–5 days post randomization |
2.2.2.2. Outcomes derived from patient- and family-reports (Trial 2 only).
Additional outcomes for Trial 2 will be obtained from patient or family surveys. Surveys will be completed targeting three time points: 1) baseline; 2) 3–5 days after randomization; and 3) 30 days after randomization (see online appendix for survey examples). Surveys may be completed in person, online, by mail, or by phone, based on respondents’ preferences.
Occurrence and quality of discussions (timepoint 2):
We use previously validated items to assess the occurrence and quality of goals-of-care communication during the hospitalization after randomization [2,28,54–59]. Communication occurrence is assessed with a single item [2,28]. Quality of goals-of-care communication is assessed with the end-of-life communication scale (QOC_eol) of the Quality of Communication (QOC) survey, developed from qualitative interviews and focus groups with a diverse group of patients, families, and clinicians [54,55,57].
Goal-concordant care (timepoint 1, timepoint 2, and timepoint 3):
Concordance between the care patients want and the care they are receiving will be measured with two questions from the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments [60]. The first question defines patients’ priorities for extending life or ensuring comfort. The second question assesses patients’ perceptions of their current treatment using the same two options [60]. Concordance is defined as a match between preference for care and the type of care currently received, as reported by patients or families. Although most patients want both quality and life-extending care, requiring respondents to pick one is a useful way to identify patients’ top priority [61–63]. If patients are unable to respond, goals of care are elicited from family as they would be in clinical practice [64].
Symptoms of anxiety and depression (timepoint 1 and timepoint 3):
Patient and family symptoms of anxiety and depression are assessed with the Hospital Anxiety and Depression Scale (HADS) [65,66]. Patients and families will complete these surveys for themselves only; we do not ask for surrogate report of patients’ psychological symptoms. The goal is not to diagnose the clinical syndromes of anxiety or depression, but rather to identify the burden of symptoms.
Utilization (timepoint 3):
In addition to measuring hospital readmissions through the EHR, we will use patient or family reports of patient emergency department visits, hospitalizations, and outpatient visits following hospital discharge. By using both sources of data, we expect to capture utilization that occurs outside of UW Medicine.
2.2.2.3. Implementation outcomes (Trials 1 and 2).
Assessment of the implementation of the interventions in Aim 3 is guided by the RE-AIM Framework for implementation research [67–70] and the Consolidated Framework for Implementation Research (CFIR) [71]. RE-AIM is a multidimensional framework for evaluating the public health impact of efforts to translate research into practice [68]. The five dimensions of RE-AIM are reach of the intervention within the target population, effectiveness of the intervention, adoption by target staff members or settings, implementation consistency and quality, and maintenance of intervention delivery and effects [67–70]. CFIR is a pragmatic meta-theoretical framework that synthesizes constructs related to implementation of evidence-based interventions. The five overarching domains are intervention characteristics, outer setting, inner setting, characteristics of individuals, and process, and include a total of 37 constructs that can be used to understand what works, and why, in a certain setting [71]. We collect quantitative and qualitative data on reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) of the intervention. Quantitative data are collected as routine tracking as part of the implementation of both trials, with data on participation, intervention use, fidelity to the intervention, and changes over time (Table 2). Qualitative data are collected through short, semi-structured interviews (10–30 min) guided by the CFIR domains. The interviews are conducted in-person or by phone with patients (n = 20) and family members (n = 20) from Trial 2, and clinicians (n = 40) from either Trial 1 or 2 after study involvement. All participants are selected using purposive sampling to ensure a diverse group based on level of participation, race, ethnicity, age, gender and, for clinicians, specialty, and year of training. A trained qualitative interviewer will interview participants using an interview guide, and interviews will be audio-recorded and transcribed [41,72–88].
Table 2.
Domain | Existing Knowledge Gaps | Quantitative Data from Trials 1 and 2 | Qualitative Data from Interviews |
---|---|---|---|
| |||
REACH: Proportion willing to participate in the intervention | • Extent clinicians, patients, and family willing to participate in Jumpstart intervention unknown • Uptake on institutional level unknown |
% of eligible patients and families participating (Trial 2) | Factors influencing acceptability of intervention to patients, families, and clinicians |
EFFECTIVENESS: Ability to improve outcomes | • Prior trials confirmed Jumpstarts effectiveness in the outpatient setting, but not inpatient • “Real world” effectiveness still to be determined |
Impact of intervention on outcomes (Aim 1&2) | Explore patient, family, and clinician experiences with the effectiveness of intervention |
ADOPTION: Proportion who actually use the intervention | • High-level interest from health system leaders, but adoption by frontline clinicians unknown • Will patients and family members accept the inpatient Jumpstart |
Patient and family participation across sites, units, services (Trial 2) | Explore barriers and facilitators to “real world” adoption and variability across units, services, & hospitals |
IMPLEMENTATION: Fidelity and consistency of use | • Fidelity of Jumpstart high in outpatient clinics, but unclear about the more hectic inpatient settings • Unclear if fidelity would vary by unit or hospital |
Use* across sites, units, services (Trial 1&2) | Assessment of clinician, patient, and family experience of intervention fidelity |
MAINTENANCE: Consistency over time and settings | • Maintenance of the two interventions is unknown and may be higher in Trial 1 • Maintenance may vary by unit or hospital |
Use* of interventions over time (Trials1&2) | Assess patient-, unit-, service-, & hospital-level maintenance over duration of Trials 1 and 2 |
Use assessed as proportion of patients for whom a Jumpstart form was opened and reviewed by a clinician on the primary team.
2.2.2.4. Analysis. We will follow the intention-to-treat principle for all analyses.
Primary Outcome (presence of goals of care discussion within 30 days after randomization): The effect of intervention on the primary outcome will be quantified by the difference in proportions and evaluated with a linear regression model with robust standard errors. The predictor of interest is randomization arm (Clinician-facing Jumpstart or usual care for Trial 1; or Clinician-facing Jumpstart, Bi-directional Jumpstart, or usual care for Trial 2). The model will adjust for hospital site and ADRD status, since randomization is stratified on these factors. This model assumes the effect of intervention is the same for patients with and without ADRD. We will also include an interaction between randomization arm and ADRD, which allows the effect of intervention to vary by ADRD status and allows evaluation of the effect among those with and without ADRD. We will evaluate the timing of goals-of-care discussions with a Cox proportional hazards model.
2.2.2.5. Additional outcomes.
For the analysis of the other outcomes, we will use a strategy similar to that for the primary outcome. For continuous outcomes (e.g., ICU-free days, HADS score), the effect of intervention will be quantified by a difference in means. For survey outcomes which are collected at more than one time point after randomization, we will use a mixed model to account for the correlation between repeated measures. Our initial model will allow the average response to be different at each time point, but assume the intervention has the same effect at each time. We will also allow the effect of intervention to be different across time by including an interaction between time and intervention. The advantage of using the data at the multiple time points and a mixed model approach is that we can gain precision; it also allows for missing responses, assuming responses are missing at random. Missing data are more of an issue for the survey outcomes than the primary outcome; we will quantify the amount and type of missing data, evaluate associations of missingness with participant characteristics, and apply appropriate methods to account for missing data [89].
2.2.2.6. Evaluate implementation and identify barriers and facilitators to future implementation.
We will perform thematic content analysis of transcribed interviews to explore feedback on the intervention, ways to improve intervention implementation, and aspects of care not adequately addressed by the intervention [90–92]. Interview guides and analyses will be guided by the RE-AIM and CFIR frameworks as described above [67–71]. Qualitative data will be imported to analytic software (Dedoose), where investigators will perform the following analytic steps using an iterative approach to thematic analysis [93]: 1) initially code material, devising a coding framework and using that framework to reduce the text into smaller segments; 2) identify themes from the coded text; 3) construct thematic networks that include basic themes, organizing themes, and global themes; 4) describe and summarize thematic networks; and 5) interpret patterns that have emerged in and across thematic networks.
2.2.2.7. Sample size considerations for the primary outcome.
The focus for sample size considerations is the primary outcome: proportion of patients with documented goals-of-care discussions within 30 days after randomization.
Trial 1:
With a total sample size of 2000 (1000 per group), two-sided significance level (α) of 0.05, and a variance estimate based on a proportion of 0.54, we have 80% power to detect a difference in proportions between those randomized to Clinician-facing Jumpstart and usual care of at least 0.06. We assumed a proportion of 0.54 based on the proportion among all participants in a prior trial of the Jumpstart guide [2]. If the total number of patients with ADRD in Trial 1 is 400 (200 per group), we would have 80% power with α = 0.05 to detect a difference in proportions of 0.14 among those with ADRD.
Trial 2:
With a total sample size of 600 (200 per Clinician-facing Jumpstart, 200 per Bi-directional Jumpstart, and 200 per usual care), we have 80% power to detect a difference in proportions of 16% for each of the 3 pairwise comparisons assuming an overall α = 0.05 and a Bonferroni adjustment for the 3 comparisons (α = 0.017 for each comparison) and variance based on a proportion of 0.54 as above [2].
2.2.2.8. Sample size for qualitative analyses.
For Aim 3 qualitative analyses, it is important to achieve theoretical saturation (no new themes emerging) [92,94]. Based on our prior studies, we anticipate achieving saturation by 80 interviews for understanding patients/families and clinician perspectives [41,75,83–88]. We will monitor for saturation and will recruit additional participants if needed.
3. Discussion
The interventions described in this protocol paper use the EHR to identify eligible patients and prompt and guide goals-of-care discussions with either: a) a clinician-facing prompt and guide for clinicians only, along with information about prior advance care planning completed prior to the hospitalization; or b) a bi-directional, patient-informed intervention that provides patient-specific support to clinicians, patients, and family members. We anticipate that both interventions will be effective compared to usual care, and that this study will provide important options for healthcare systems. Economic analyses will allow us to evaluate the effect on costs of care, after factoring in the costs of implementing these interventions, to enhance dissemination. If either or both of these interventions are not effective, the results of Aim 3 will provide important information to shape, direct and deliver future interventions.
This study has several potential limitations. First, this study occurs in a single healthcare system which may limit generalizability. Second, goals-of-care discussions may be misclassified for two reasons: 1) the sensitivity and specificity of NLP for the outcome is not perfect; and 2) documentation of goals-of-care discussions in the EHR does not perfectly reflect actual discussions. We will assess the accuracy of NLP against manual EHR review in a sample of patients to evaluate the extent of misclassification. Third, it is possible that this intervention might change behavior for clinicians caring for patients randomized to usual care. Our prior studies suggest that most clinicians require a patient-specific prompt to have timely goals-of-care discussions, which may mitigate this concern [2,28]. However, we will assess for an increase in goals-of-care discussions in the usual care groups over time, which might signify contamination or temporal trends, but could be used to assess the potential degree of contamination if present. Contamination would bias the results toward the null hypothesis and only be a major issue for a negative study. Fourth, although we do not expect missing data for our primary outcome (presence of goals of care discussion within 30 days after randomization according to the EHR), outcomes from surveys are likely to be missing from some participants which could lead to bias and reduce precision of our estimate of intervention on patient-reported outcomes. Fifth, although we are powered to detect a difference between each of the 3 arms in Trial 2, a difference between the 2 interventions is not of interest if neither is superior to usual care and we are not powered to detect non-inferiority or equivalence (of the Clinician-facing Jumpstart compared to the bi-directional Jumpstart). In addition, there is no established “minimal clinically important difference” for this outcome. Sixth, cost assessments are limited to costs available at UW Medicine, and we will be limited in our ability to assess costs from other healthcare systems after hospital discharge. Most of the benefits we anticipate for this intervention will occur during the hospitalization, although there may be ongoing reductions in costs after hospitalization related to changes in the goals of care as a result of the intervention. Finally, we acknowledge that these two trials could have been conducted as a single trial without a waiver of informed consent which would have reduced resources required, but also highlight that this would not have had the advantage of Trial 1 of enrolling all eligible patients without risk of response bias.
In summary, we report here the protocol for two complementary yet independent trials to evaluate an intervention to prime and guide goals-of-care discussions for seriously ill hospitalized patients. The first is a large pragmatic trial comparing a clinician-facing intervention to usual care. The second is an effectiveness trial comparing the clinician-facing Jumpstart, a bi-directional (clinician-facing and patient- or family-facing) Jumpstart, and usual care. Both trials use a Type 1 hybrid effectiveness-implementation design to explore barriers and facilitators for similar interventions in the future.
Supplementary Material
Funding acknowledgements
Funded by the National Institute on Aging (R01 AG062441).
Footnotes
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.org/10.1016/j.cct.2022.106879.
Data availability
No data was used for the research described in the article.
References
- [1].Wright AA, Zhang B, Ray A, et al. , Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment, JAMA 300 (14) (2008) 1665–1673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Curtis JR, Downey L, Back AL, et al. , Effect of a patient and clinician communication-priming intervention on patient-reported goals-of-care discussions between patients with serious illness and clinicians: a randomized clinical trial, JAMA Intern. Med. 178 (7) (2018) 930–940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Fakhri S, Engelberg RA, Downey L, et al. , Factors affecting Patients’ preferences for and actual discussions about end-of-life care, J. Pain Symptom Manag. 52 (3) (2016) 386–394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Heyland DK, Dodek P, You JJ, et al. , Validation of quality indicators for end-of-life communication: results of a multicentre survey, CMAJ 189 (30) (2017) E980–E989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Silveira MJ, Kim SY, Langa KM, Advance directives and outcomes of surrogate decision making before death, N. Engl. J. Med. 362 (13) (2010) 1211–1218. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Teno JM, Gruneir A, Schwartz Z, Nanda A, Wetle T, Association between advance directives and quality of end-of-life care: a national study, J. Am. Geriatr. Soc. 55 (2) (2007) 189–194. [DOI] [PubMed] [Google Scholar]
- [7].Sudore RL, Lum HD, You JJ, et al. , Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel, J. Pain Symptom Manag. 53 (5) (2017) 821–832 e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Childers JW, Back AL, Tulsky JA, Arnold RM, REMAP: a framework for goals of care conversations, J Oncol Pract 13 (10) (2017) e844–e850. [DOI] [PubMed] [Google Scholar]
- [9].Halpern SD, Goal-concordant care - searching for the holy grail, N. Engl. J. Med. 381 (17) (2019) 1603–1606. [DOI] [PubMed] [Google Scholar]
- [10].Morrison RS, Meier DE, Arnold RM, What’s wrong with advance care planning? JAMA 326 (16) (2021) 1575–1576. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Curtis JR, Three stories about the value of advance care planning, JAMA 326 (21) (2021) 2133–2134. [DOI] [PubMed] [Google Scholar]
- [12].Periyakoil VS, Gunten CFV, Arnold R, Hickman S, Morrison S, Sudore R, Caught in a loop with advance care planning and advance directives: how to move forward? J. Palliat. Med. 25 (3) (2022) 355–360. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].Detering KM, Hancock AD, Reade MC, Silvester W, The impact of advance care planning on end of life care in elderly patients: randomised controlled trial, BMJ 340 (2010), c1345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Tulsky JA, Beach MC, Butow PN, et al. , A research agenda for communication between health care professionals and patients living with serious illness, JAMA Intern. Med. 177 (9) (2017) 1361–1366. [DOI] [PubMed] [Google Scholar]
- [15].Aslakson RA, Reinke LF, Cox C, Kross EK, Benzo RP, Curtis JR, Developing a research agenda for integrating palliative care into critical care and pulmonary practice to improve patient and family outcomes, J. Palliat. Med. 20 (4) (2017) 329–343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].Curtis JR, Engelberg RA, Teno JM, Understanding variability of end-of-life care in the ICU for the elderly, Intensive Care Med. 43 (1) (2017) 94–96. [DOI] [PubMed] [Google Scholar]
- [17].Teno JM, Gozalo P, Khandelwal N, et al. , Association of Increasing use of mechanical ventilation among nursing home residents with advanced dementia and intensive care unit beds, JAMA Intern. Med. 176 (12) (2016) 1809–1816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Guidet B, Leblanc G, Simon T, et al. , Effect of systematic intensive care unit triage on long-term mortality among critically ill elderly patients in France: a randomized clinical trial, JAMA 318 (15) (2017) 1450–1459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Alzheimer”s Association, Alzheimer’s Disease Facts and Figures, Alzheimer’s Association, 2017. https://www.alz.org/facts/. [Google Scholar]
- [20].Gozalo P, Teno JM, Mitchell SL, et al. , End-of-life transitions among nursing home residents with cognitive issues, N. Engl. J. Med. 365 (13) (2011) 1212–1221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [21].Teno JM, Gozalo PL, Mitchell SL, et al. , Does feeding tube insertion and its timing improve survival? J. Am. Geriatr. Soc. 60 (10) (2012) 1918–1921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [22].Curtis JR, Sathitratanacheewin S, Starks H, et al. , Using electronic health Records for Quality Measurement and Accountability in Care of the Seriously ill: opportunities and challenges, J. Palliat. Med. 21 (S2) (2018) S52–S60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [23].Hicks K, Downey L, Engelberg RA, et al. , Predictors of death in the Hospital for Patients with chronic serious illness, J. Palliat. Med. 21 (3) (2018) 307–314. [DOI] [PubMed] [Google Scholar]
- [24].Lavin K, Davydow DS, Downey L, et al. , Effect of psychiatric illness on acute care utilization at end of life from serious medical illness, J. Pain Symptom Manag. 54 (2) (2017) 176–185 e1. [DOI] [PubMed] [Google Scholar]
- [25].Sathitratanacheewin S, Engelberg RA, Downey L, et al. , Temporal trends between 2010 and 2015 in intensity of care at end-of-life for patients with chronic illness: influence of age under versus over 65 years, J. Pain Symptom Manag. 55 (1) (2017) 75–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Brown CE, Engelberg RA, Sharma R, et al. , Race/ethnicity, socioeconomic status, and healthcare intensity at the end of life, J. Palliat. Med. 21 (9) (2018) 1308–1316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [27].Steiner JM, Kirkpatrick JN, Heckbert SR, et al. , Identification of adults with congenital heart disease of moderate or great complexity from administrative data, Congenit. Heart Dis. 13 (1) (2018) 65–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [28].Au DH, Udris EM, Engelberg RA, et al. , A randomized trial to improve communication about end-of-life care among patients with COPD, Chest 141 (3) (2012) 726–735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29].Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C, Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact, Med. Care 50 (3) (2012) 217–226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].The Dartmouth Institute for Health Policy and Clinical Practice, The Dartmouth Atlas of Healthcare. www.dartmouthatlas.org.
- [31].Wennberg JE, Bronner K, Skinner JS, Fisher ES, Goodman DC, Inpatient care intensity and patients’ ratings of their hospital experiences, Health Aff (Millwood) 28 (1) (2009) 103–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [32].Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK, Use of hospitals, physician visits, and hospice care during last six months of life among cohorts loyal to highly respected hospitals in the United States, BMJ 328 (7440) (2004) 607. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [33].Zaslavsky O, Zelber-Sagi S, LaCroix AZ, et al. , Comparison of the simplified sWHI and the standard CHS frailty phenotypes for prediction of mortality, incident falls, and hip fractures in older women, J. Gerontol. A Biol. Sci. Med. Sci. 72 (10) (2017) 1394–1400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [34].Fried LP, Tangen CM, Walston J, et al. , Frailty in older adults: evidence for a phenotype, J. Gerontol. A Biol. Sci. Med. Sci. 56 (3) (2001) M146–M156. [DOI] [PubMed] [Google Scholar]
- [35].Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC, Six-item screener to identify cognitive impairment among potential subjects for clinical research, Med. Care 40 (9) (2002) 771–781. [DOI] [PubMed] [Google Scholar]
- [36].Abedini NC, Merel SE, Hicks KG, et al. , Applying human-centered design to refinement of the jumpstart guide, a clinician- and patient-facing goals-of-care discussion priming tool, J. Pain Symptom Manag. 62 (6) (2021) 1283–1288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [37].Forum NQ. National Quality Forum, Measure 1626, Patients Admitted to ICU who Have Care Preferences Documented within 48 hours (RAND Corporation; ). https://www.qualityforum.org/News_And_Resources/Endorsement_Summaries/Endorsement_Summaries.aspx, 2022. [Google Scholar]
- [38].Back AL, Arnold RM, Baile WF, et al. , Efficacy of communication skills training for giving bad news and discussing transitions to palliative care, Arch. Intern. Med. 167 (5) (2007) 453–460. [DOI] [PubMed] [Google Scholar]
- [39].Back AL, Arnold RM, Tulsky JA, Baile WF, Fryer-Edwards KA, Teaching communication skills to medical oncology fellows, J. Clin. Oncol. 21 (12) (2003) 2433–2436. [DOI] [PubMed] [Google Scholar]
- [40].Sudore RL, Schillinger D, Knight SJ, Fried TR, Uncertainty about advance care planning treatment preferences among diverse older adults, J. Health Commun. 15 (Suppl. 2) (2010) 159–171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [41].Curtis JR, Wenrich MD, Carline JD, Shannon SE, Ambrozy DM, Ramsey PG, Understanding physicians’ skills at providing end-of-life care: perspectives of patients, families, and health care workers, J. Gen. Intern. Med. 16 (2001) 41–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [42].Steinhauser KE, Christakis NA, Clipp EC, McNeilly M, McIntyre LM, Tulsky JA, Factors considered important at the end of life by patients, family, physicians, and other care providers, JAMA 284 (2000) 2476–2482. [DOI] [PubMed] [Google Scholar]
- [43].Nadkarni PM, Ohno-Machado L, Chapman WW, Natural language processing: an introduction, J. Am. Med. Inform. Assoc. 18 (5) (2011) 544–551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [44].Yim WW, Yetisgen M, Harris WP, Kwan SW, Natural language processing in oncology: a review, JAMA Oncol 2 (6) (2016) 797–804. [DOI] [PubMed] [Google Scholar]
- [45].Curtis JR, Treece PD, Nielsen EL, et al. , Randomized trial of communication facilitators to reduce family distress and intensity of end-of-life care, Am. J. Respir. Crit. Care Med. 193 (2) (2016) 154–162. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [46].Lee RY, Brumback LC, Lober WB, et al. , Identifying goals of care conversations in the electronic health record using natural language processing and machine learning, J. Pain Symptom Manag. 61 (1) (2021) 136–142 e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [47].Lee RY, Kross EK, Downey L, et al. , Efficacy of a communication-priming intervention on documented goals-of-care discussions in hospitalized patients with serious illness: a randomized clinical trial, JAMA Netw. Open 5 (4) (2022), e225088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [48].Uyeda AM, Curtis JR, Engelberg RA, et al. , Mixed-methods evaluation of three natural language processing modeling approaches for measuring documented goals-of-care discussions in the electronic health record, J. Pain Symptom Manag. 63 (6) (2022) e713–e723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [49].Chan A, Chien I, Moseley E, et al. , Deep learning algorithms to identify documentation of serious illness conversations during intensive care unit admissions, Palliat. Med. 33 (2) (2019) 187–196. [DOI] [PubMed] [Google Scholar]
- [50].Udelsman BV, Moseley ET, Sudore RL, Keating NL, Lindvall C, Deep natural language processing identifies variation in care preference documentation, J. Pain Symptom Manag. 59 (6) (2020) 1186–1194 e3. [DOI] [PubMed] [Google Scholar]
- [51].Lindvall C, Deng CY, Moseley E, et al. , Natural language processing to identify advance care planning documentation in a multisite pragmatic clinical trial, J. Pain Symptom Manag. 63 (1) (2022) e29–e36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [52].Khandelwal N, Brumback LC, Halpern SD, Coe NB, Brumback B, Curtis JR, Evaluating the economic impact of palliative and end-of-life care interventions on intensive care unit utilization and costs from the hospital and healthcare system perspective, J. Palliat. Med. 20 (12) (2017) 1314–1320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [53].Auriemma CL, Taylor SP, Harhay MO, Courtright KR, Halpern SD, Hospital-free days: a pragmatic and patient-centered outcome for trials among critically and seriously ill patients, Am. J. Respir. Crit. Care Med. 204 (8) (2021) 902–909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [54].Engelberg R, Downey L, Curtis JR, Psychometric characteristics of a quality of communication questionnaire assessing communication about end-of-life care, J. Palliat. Med. 9 (5) (2006) 1086–1098. [DOI] [PubMed] [Google Scholar]
- [55].Curtis JR, Engelberg RA, Nielsen EL, Au DH, Patrick DL, Patient-physician communication about end-of-life care for patients with severe COPD, Eur. Respir. J. 24 (2004) 200–205. [DOI] [PubMed] [Google Scholar]
- [56].Curtis JR, Patrick DL, Caldwell E, Greenlee H, Collier AC, The quality of patient-clinician communication about end-of-life care: a study of patients with AIDS and their primary care clinicians, AIDS 13 (1999) 1123–1131. [DOI] [PubMed] [Google Scholar]
- [57].Janssen DJ, Curtis JR, Au DH, et al. , Patient-clinician communication about end-of-life care for Dutch and US patients with COPD, Eur. Respir. J. 38 (2) (2011) 268–276. [DOI] [PubMed] [Google Scholar]
- [58].Hanson LC, Zimmerman S, Song MK, et al. , Effect of the goals of care intervention for advanced dementia: a randomized clinical trial, JAMA Intern. Med. 177 (1) (2017) 24–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [59].White DB, Angus DC, Shields AM, et al. , A randomized trial of a family-support intervention in intensive care units, N. Engl. J. Med. 378 (25) (2018) 2365–2375. [DOI] [PubMed] [Google Scholar]
- [60].Teno JM, Fisher ES, Hamel MB, Coppola K, Dawson NV, Medical care inconsistent with patients’ treatment goals: association with 1-year Medicare resource use and survival, J. Am. Geriatr. Soc. 50 (3) (2002) 496–500. [DOI] [PubMed] [Google Scholar]
- [61].Coast J, Huynh E, Kinghorn P, Flynn T, Complex valuation: applying ideas from the complex intervention framework to valuation of a new measure for end-of-life care, Pharmacoeconomics 34 (5) (2016) 499–508. [DOI] [PubMed] [Google Scholar]
- [62].Finkelstein EA, Bilger M, Flynn TN, Malhotra C, Preferences for end-of-life care among community-dwelling older adults and patients with advanced cancer: a discrete choice experiment, Health Policy 119 (11) (2015) 1482–1489. [DOI] [PubMed] [Google Scholar]
- [63].Flynn TN, Bilger M, Malhotra C, Finkelstein EA, Are efficient designs used in discrete choice experiments too difficult for some respondents? A case study eliciting preferences for end-of-life care, Pharmacoeconomics 34 (3) (2016) 273–284. [DOI] [PubMed] [Google Scholar]
- [64].Appelbaum PS, Clinical practice. Assessment of patients’ competence to consent to treatment, N. Engl. J. Med. 357 (18) (2007) 1834–1840. [DOI] [PubMed] [Google Scholar]
- [65].Zigmond AS, Snaith RP, The hospital anxiety and depression scale, Acta Psychiatr. Scand. 67 (6) (1983) 361–370. [DOI] [PubMed] [Google Scholar]
- [66].Bjelland I, Dahl AA, Haug TT, Neckelmann D, The validity of the hospital anxiety and depression scale. An updated literature review, J. Psychosom. Res. 52 (2) (2002) 69–77. [DOI] [PubMed] [Google Scholar]
- [67].Gaglio B, Shoup JA, Glasgow RE, The RE-AIM framework: a systematic review of use over time, Am. J. Public Health 103 (6) (2013) e38–e46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [68].Glasgow RE, Vogt TM, Boles SM, Evaluating the public health impact of health promotion interventions: the RE-AIM framework, Am. J. Public Health 89 (9) (1999) 1322–1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [69].Harden SM, Gaglio B, Shoup JA, et al. , Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review, Syst Rev 4 (2015) 155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [70].Shoup JA, Gaglio B, Varda D, Glasgow RE, Network analysis of RE-AIM framework: chronology of the field and the connectivity of its contributors, Transl. Behav. Med. 5 (2) (2015) 216–232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [71].Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC, Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science, Implement. Sci. 4 (2009) 50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [72].Engelberg RA, Wenrich MD, Curtis JR, Responding to families’ questions about the meaning of physical movements in critically ill patients, J. Crit. Care 23 (4) (2008) 565–571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [73].Engelberg RA, Downey L, Wenrich MD, et al. , Measuring the quality of end-of-life care, J. Pain Symptom Manag. 39 (6) (2010) 951–971. [DOI] [PubMed] [Google Scholar]
- [74].Howell AA, Nielsen EL, Turner AM, Curtis JR, Engelberg RA, Clinicians’ perceptions of the usefulness of a communication facilitator in the intensive care unit, Am. J. Crit. Care 23 (5) (2014) 380–386. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [75].Curtis JR, Engelberg R, Young JP, et al. , An approach to understanding the interaction of hope and desire for explicit prognostic information among individuals with severe chronic obstructive pulmonary disease or advanced cancer, J. Palliat. Med. 11 (4) (2008) 610–620. [DOI] [PubMed] [Google Scholar]
- [76].Curtis JR, Engelberg RA, Wenrich MD, Au DH, Communication about palliative care for patients with chronic obstructive pulmonary disease, J. Palliat. Care 21 (3) (2005) 157–164. [PubMed] [Google Scholar]
- [77].Curtis JR, Engelberg RA, Wenrich MD, et al. , Studying communication about end-of-life care during the ICU family conference: development of a framework, J. Crit. Care 17 (2002) 147–160. [DOI] [PubMed] [Google Scholar]
- [78].Curtis JR, Engelberg RA, Wenrich MD, Shannon SE, Treece PD, Rubenfeld GD, Missed opportunities during family conferences about end-of-life care in the intensive care unit, Am. J. Respir. Crit. Care Med. 171 (8) (2005) 844–849. [DOI] [PubMed] [Google Scholar]
- [79].Reinke LF, Engelberg RA, Shannon SE, et al. , Transitions regarding palliative and end-of-life care in severe chronic obstructive pulmonary disease or advanced cancer: themes identified by patients, families, and clinicians, J. Palliat. Med. 11 (4) (2008) 601–609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [80].Reinke LF, Shannon SE, Engelberg RA, Young JP, Curtis JR, Supporting hope and prognostic information: nurses’ perspectives on their role when patients have life-limiting prognoses, J. Pain Symptom Manag. 39 (6) (2010) 982–992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [81].West HF, Engelberg RA, Wenrich MD, Curtis JR, Expressions of nonabandonment during the intensive care unit family conference, J. Palliat. Med. 8 (4) (2005) 797–807. [DOI] [PubMed] [Google Scholar]
- [82].White DB, Engelberg RA, Wenrich MD, Lo B, Curtis JR, Prognostication during physician-family discussions about limiting life support in intensive care units, Crit. Care Med. 35 (2) (2007) 442–448. [DOI] [PubMed] [Google Scholar]
- [83].Back AL, Young JP, McCown E, et al. , Abandonment at the end of life from patient, caregiver, nurse, and physician perspectives: loss of continuity and lack of closure, Arch. Intern. Med. 169 (5) (2009) 474–479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [84].Carline JD, Curtis JR, Wenrich MD, Shannon SE, Ambrozy DM, Ramsey PG, Physicians’ interactions with health care teams and systems in the care of dying patients: perspectives of dying patients, family members, and health care providers, J. Pain Symptom Manag. 25 (2003) 19–28. [DOI] [PubMed] [Google Scholar]
- [85].Curtis JR, Patrick DL, Caldwell E, Collier AC, Why don’t patients with AIDS and their clinicians talk about end-of-life care? Barriers to communication for patients with AIDS and their primary care clinicians, Arch. Intern. Med. 160 (2000) 1690–1696. [DOI] [PubMed] [Google Scholar]
- [86].Curtis JR, Wenrich MD, Carline JD, Shannon SE, Ambrozy DM, Ramsey PG, Patients’ perspectives on physicians’ skills at end-of-life care: differences between patients with COPD, cancer, and AIDS, Chest 122 (2002) 356–362. [DOI] [PubMed] [Google Scholar]
- [87].Wenrich MD, Curtis JR, Ambrozy DM, et al. , Provision of emotional support and personalized care by physicians to patients nearing the end of life, J. Pain Symptom Manag. 25 (2003) 236–246. [DOI] [PubMed] [Google Scholar]
- [88].Wenrich MD, Curtis JR, Shannon SE, Carline JD, Ambrozy DM, Ramsey PG, Communicating with dying patients within the spectrum of medical care from terminal diagnosis to death, Arch. Intern. Med. 161 (2001) 868–874. [DOI] [PubMed] [Google Scholar]
- [89].Little R, Rubin D, Statistical Analysis with Missing Data, Third ed., Wiley, New York, 2019. [Google Scholar]
- [90].Hsieh HF, Shannon SE, Three approaches to qualitative content analysis, Qual. Health Res. 15 (9) (2005) 1277–1288. [DOI] [PubMed] [Google Scholar]
- [91].Corbin J, Strauss A, Methods of Qualitative Research, Sage Publications, Thousand Oaks, CA, 1993. [Google Scholar]
- [92].Strauss AL, Corbin J, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications, Thousand Oaks, 1998. [Google Scholar]
- [93].Attride-Stirling J, Thematic networks: an analytic tool for qualitative research, Qual. Res. 1 (2001) 385–405. [Google Scholar]
- [94].Giacomini M, Cook DJ, For the evidence-based medicine working group. Qualitative research in health care: what are the results and how do they help me care for my patients? JAMA 284 (2000) 478–482. [DOI] [PubMed] [Google Scholar]
- [95].Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM, Developing CollaboRATE: a fast and frugal patient-reported measure of shared decision making in clinical encounters, Patient Educ. Couns. 93 (1) (2013) 102–107. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
No data was used for the research described in the article.