Abstract
Background
Failure to deliver care near the end of life that reflects the needs, values and preferences of patients with advanced cancer remains a major shortcoming of our cancer care delivery system.
Methods
A mixed-methods comparative effectiveness trial of in-person advance care planning (ACP) discussions versus web-based ACP is currently underway at oncology practices in Western Pennsylvania.
Patients with advanced cancer and their caregivers are invited to enroll. Participants are randomized to either (1) in-person ACP discussions via face-to-face visits with a nurse facilitator following the Respecting Choices® Conversation Guide or (2) web-based ACP using the PREPARE for your care™ web-based ACP tool. The trial compares the effect of these two interventions on patient and family caregiver outcomes (engagement in ACP, primary outcome; ACP discussions; advance directive (AD) completion; quality of end-of-life (EOL) care; EOL goal attainment; caregiver psychological symptoms; healthcare utilization at EOL) and assesses implementation costs. Factors influencing ACP effectiveness are assessed via in-depth interviews with patients, caregivers and clinicians.
Discussion
This trial will provide new and much-needed empirical evidence about two patient-facing ACP approaches that successfully overcome limitations of traditional written advance directives but entail very different investments of time and resources. It is innovative in using mixed methods to evaluate not only the comparative effectiveness of these approaches, but also the contexts and mechanisms influencing effectiveness. Data from this study will inform clinicians, payers and health systems seeking to adopt and scale the most effective and efficient ACP strategy in real-world oncology settings.
1. INTRODUCTION
Mounting evidence suggests major shortcomings in end of life (EOL)i care for over 600,000 patients who die in the U.S. every year from cancer.1-3 National organizations—including the National Academy of Medicine and the American Society of Clinical Oncology—have called for an increased focus on advance care planning (ACP)j to improve the delivery of EOL care consistent with patients’ needs, values, and preferences.2-5 However, there are several unanswered questions about the ACP approach most likely to impact clinical care for patients with advanced cancer. A critical barrier to progress in the field is a lack of evidence about the most effective and efficient ACP strategy to improve treatment decisions near EOL and ensure patients’ wishes are honored.
Two different patient-facing ACP interventions are widely used but entail sizeable differences in costs and complexity to deploy: (1) in-person ACP discussions with trained facilitators6,7 and (2) web-based ACP using interactive videos.8,9 These interventions may impact clinical care by encouraging patients to consider, discuss, and document their goals and preferences. Both approaches can overcome barriers associated with simply asking patients to complete traditional written advance directives (ADs).2,10-12 In-person ACP enables patients to have a face-to-face conversation about preferences and goals with a trained facilitator, while web-based ACP allows patients to engage in ACP in stages, independently and at home.6,9 These approaches have never been compared directly. It is therefore unclear whether one form of ACP is more effective—and if so, for whom, how, and under what circumstances. Understanding the relative effectiveness of in-person, facilitated versus web-based ACP is important because in-person, facilitated ACP requires far more resources. Oncology practices must decide whether and in what circumstances an in-person, facilitated approach is worth the additional cost and administrative burden when compared to a web-based alternative.
We therefore designed the Patient-centered and efficacious advance care planning in cancer (PEACe-compare) trial, a single-blind, patient-level randomized trial to compare the effectiveness of in-person ACP discussions with trained facilitators versus web-based ACP using interactive videos. Funded by the National Cancer Institute (1R01CA235730), PEACe-compare incorporates mixed methods to understand contexts and circumstances influencing ACP effectiveness, yielding data to inform clinicians, payers, and health systems seeking to adopt and scale the best ACP strategy in real-world oncology settings. In this manuscript, we describe the PEACe-compare trial protocol and key design considerations.
2. MATERIALS and METHODS
Study Overview
This study is a single-blind, patient-level randomized trial comparing in-person ACP discussions with trained facilitators to web-based ACP using interactive videos. The trial is designed to compare the effect of these two interventions on patient and family caregiver outcomes, including engagement in ACP (primary outcome), ACP discussions with family caregivers and physicians, advance directive (AD) completion, quality of end-of-life (EOL) care, EOL goal attainment, and caregiver symptoms of anxiety, depression, and post-traumatic stress. To inform future dissemination efforts, we compare implementation costs and effects on healthcare utilization at EOL. Using in-depth interviews with patients, caregivers, and clinicians, we identify contexts and mechanisms that influence the effectiveness of each approach. The research protocol was approved by the University of Pittsburgh Institutional Review Board (STUDY19080337), and the trial is registered on ClinicalTrials.gov (NCT03824158). The full study protocol is included as an appendix (see Appendix A).
Setting/Participants
The PEACe-compare trial is conducted at the UPMC Hillman Cancer Center, one of the largest integrated community networks of oncologists in the United States with over 50 locations throughout Western Pennsylvania, New York, and Ohio. We chose clinic locations within this network that (1) represent a mix of academic and community practice models and (2) include a range of clinic sizes. Oncologists at participating sites and cancer center leaders are strongly supportive of this research.
We designed eligibility criteria to be broadly representative of the population with cancer for whom advance care planning is most important (see Table 1).13 We have used these criteria successfully in prior and ongoing work to identify patients with sufficiently advanced disease to be at high risk for facing end of life decisions without being too ill to participate.14,15 Currently, there is lack of good evidence regarding the optimal timing of ACP in advanced cancer,12,16 but in most prior studies patients and clinicians preferred to delay the introduction of ACP to later in the illness trajectory, rather than at time of diagnosis.17-19 The validated “would not be surprised question” is a simple and effective method for identifying patients with cancer at high risk of dying in one year (Hazard Ratio 7.8).20,21
Table 1.
Inclusion and exclusion criteria for PEACe-compare patient and caregiver participants
| Patient eligibility criteria | |
|---|---|
| Inclusion Criteria | 18 years of age or older |
| Solid tumor (any type) | |
| The oncologist ‘would not be surprised’ if the patient died within the next year | |
| Eastern Cooperative Oncology Group performance status (ECOG PS) of 0, 1, or 2 | |
| Planning to receive ongoing care at a participating oncology clinic | |
| Willing to participate in either a web-based or in-person advance care planning program | |
| Exclusion Criteria | Does not speak English |
| Inability to consent, using a validated teach-back method. | |
| Hematologic malignancy | |
| No phone for additional study contacts and follow-up interviews | |
| Unable to participate in advance care planning, as assessed by clinician | |
| Unable to complete the baseline interview | |
| Caregiver eligibility criteria | |
| Inclusion Criteria | 18 years of age or older |
| Family member or friend of an eligible patient | |
| Primary person involved in patient’s care and best able to participate in the study, as assessed by patient | |
| Exclusion Criteria | Does not speak English |
| No phone for additional study contacts and follow-up interviews | |
| Unable to complete the baseline interview | |
We do not include patients with hematologic malignancies because these diseases have very different trajectories and treatment options near the end of life,22,23 and the majority of patients seen at participating practices have solid tumors. We considered excluding patients who have previously completed an AD, but ultimately we decided to include this group because preferences change and patients may benefit from additional engagement in ACP with changing health contexts.16 We do require that participants be willing to participate in either ACP intervention as a component of study participation. We exclude patients < 18 years of age because children with incurable cancer have unique ACP needs best served by clinicians experienced working with pediatric populations, and our study is conducted at adult oncology practices. We also exclude patients unable to consent to treatment using a validated teach-back method designed to ensure comprehension of study procedures among vulnerable populations (see Recruitment/Informed Consent, below).24
Based on our prior work, we estimate that most eligible patients will be able to identify and enroll a caregiver, designated by the patient as the primary person involved in their care and best able to participate in the study.15,25-27 Patients unable to identify a caregiver will not be excluded because these patients also face treatment decisions near end of life and may benefit from ACP.
We enroll clinician participants for the subset of patients who participate in an in-depth interview (see below section, In-depth interviews to identify factors that influence ACP effectiveness). Patients are asked to identify the clinician (oncologist, primary care provider, nurse practitioner, etc.) most involved in decisions about their care.
Recruitment/Informed Consent
We have created a practical and systematic approach to recruitment, with the goal of minimizing administrative costs, disruption to clinical activities, and selection bias. Our research team reviews administrative lists and clinic schedules with participating oncologists and/or their advance practice provider on a weekly basis to identify potentially eligible patients with solid tumors who meet the ‘would not be surprised’ criteria. Potentially eligible patients are first approached in-person by a clinical member of their oncology team and offered a 1-page study information sheet. We do not introduce the study at the first appointment in which a new diagnosis of metastatic disease is discussed because this is a time of heightened anxiety when patients are unlikely to be receptive to ACP research.19,25 If desired by the patient, this introduction is followed by a detailed, in-person explanation of the study from a trained research assistant, who obtains written informed consent from all patient participants and tracks reasons for non-participation. A second invitation to participate is offered to patients who decline initial participation (for example, because they are feeling too overwhelmed) but give permission to be re-contacted (either in-person or by phone) at a later date.
A consent teach-back is administered by the trained research assistant to ensure comprehension of study procedures, risks, and benefits (see Appendix A “PEACe-compare Protocol” Section 3.4). In a question-answer format adapted from prior work by Sudore, et al.,24 potential participants are asked eight key questions (see Table 2). Patients who are unable to provide the correct answer to each question after three tries are deemed ineligible to participate.
Table 2.
Informed Consent Teach-back Questions for PEACe-compare
| 1) “In your own words, can you tell me why we are doing this study?” |
| 2)“What will happen if you take part in this study? What will we ask you to do?” |
| 3) “Do you have a choice about taking part in this research study?” |
| 4) “What are the risks of being in this study?” |
| 5) “What are the benefits of being in this study?” |
| 6) “Can you stop being in this study at any time?” |
| 7) “As part of this study, will you be expected to complete participation in either a web-based or in-person program at home or in clinic?” |
| 8) “What should you do if you have questions about the study?” |
Caregivers of consented patients, who have been identified by the patient as the primary person involved in the patient’s care, are approached about study participation either in-person during the patient’s clinical visit or by telephone. The option for caregivers to provide verbal informed consent is designed to maximize participation for caregivers who may not be present when the patient is initially approached.
Randomization
Upon completion of baseline patient and caregiver surveys, patient-caregiver pairs (or patients without an enrolled caregiver) are randomized 1:1 at the patient level to receive either (1) in-person, facilitated ACP or (2) web-based ACP. We chose patient-level randomization, consistent with most prior ACP trials,7,8 because both interventions are at the patient level. The risk of contamination is extremely low because both interventions will be conducted in private and are unlikely to influence systematic changes in physician practices. To further ensure against any potential contamination, we ask participants not to discuss either intervention with other patients or caregivers.
Permuted block randomization with random block size is stratified by health literacy (determined by a single validated question concerning confidence with medical forms28) and whether patients have previously completed an advance directive (AD). We chose these variables for stratification to ensure equal distribution between intervention groups, because health literacy and previous AD completion may be most strongly associated with our outcomes of interest.29,30
3. PEACe-compare INTERVENTIONS
In-person, facilitated ACP
Respecting Choices is a facilitated ACP model developed in La Crosse, Wisconsin in the 1990s. Respecting Choices has been vetted by patients, caregivers, and clinicians from diverse communities and cultures and successfully replicated across entire communities31 and healthcare systems worldwide.32-34 In prior randomized efficacy trials, facilitated ACP with Respecting Choices improved family understanding of patient goals,35 increased goal-concordant EOL care,7 and decreased stress, anxiety, and depression in surviving relatives as compared to usual care.7
Patients randomized to this arm participate in in-person facilitated ACP discussions led by a nurse facilitator. Two nurse facilitators completed a three-day Respecting Choices First Steps® training in La Crosse, Wisconsin held April 29-May 1, 2019 and were certified by Respecting Choices faculty. These facilitators also participated in the program’s train-the-trainer sessions, to allow for the possibility of training more nurses in the event of staff turn-over. We chose to train nurses because these non-physician members of the clinical team are typically trained as ACP facilitators and are widely available in outpatient oncology settings, maximizing feasibility of future dissemination. We selected facilitators who do not provide other clinical duties at participating sites to guard against potential contamination.
Participants in the in-person, facilitated ACP arm are given an introductory packet that includes a welcome letter, the Making Choices® Information Card for Healthcare Agents, the Making Choices® Advance Care Planning Guide, and a blank copy of the Pennsylvania Advance Healthcare Directive, which is used across University of Pittsburgh Medical System clinics and hospitals. The facilitator contacts the patient to schedule the ACP discussion. Face-to-face visits are conducted either in a patient’s private home, a private setting in clinic, or a private office in the research offices at University of Pittsburgh. Patients are encouraged to attend the ACP discussion with their medical decision maker, when available, as well as any additional family members they wish to include.
Facilitated ACP discussions follow the First Steps® ACP Conversation Guide for Adults with Chronic Illness. This particular conversation guide is geared towards individuals with chronic illness who are otherwise healthy. We chose it because 1) we anticipate this encounter being the first time most patients will have had an ACP discussion and 2) our intent is to normalize ACP. This structured interview tool provides a discussion roadmap for choosing a medical decision maker, exploring serious illness understanding and experiences, identifying goals and values, and making future treatment decisions. Facilitators assist patients with completing advance directives and providing a copy to clinical staff, when appropriate, and they make recommendations for communicating goals and sharing written preferences. Typically the ACP discussion is completed in a single visit; however, up to three visits may be scheduled, if needed, to facilitate advance directive completion. Patients do not pay a co-pay for participating in facilitated ACP.
All Respecting Choices visits are audio-recorded. The nurse facilitators listen to each other’s visits and meet weekly to debrief and provide feedback on visit content. The nurse facilitators also have regular phone meetings with a faculty consultant from the Respecting Choices program, during which they evaluate, critique, and discuss recorded Respecting Choices conversations.
Web-based ACP
PREPARE for your care™ is a web-based ACP tool launched in 2013 based on extensive formative work demonstrating the need for patient-centered tools to overcome barriers to ACP, especially for vulnerable populations.10,11 PREPARE uses interactive videos and a simple, 5-step process that asks patients about their values and helps them make a commitment to engaging in each ACP step.9 PREPARE can be viewed on a computer, tablet, or phone in the privacy of a patient’s own home. Each step takes five to ten minutes;9 participants can work at their own pace, save their answers, and continue working where they left off. Extensive research has demonstrated that PREPARE is feasible and acceptable in diverse populations, including patients with cancer.9,36 In two prior trials, PREPARE increased ACP engagement and documentation when compared to written ADs alone.8,37 These effects were achieved without additional clinician or system-level interventions.8,9 To date, the PREPARE website (www.prepareforyourcare.org) has been visited by over 150,000 unique users from 150 countries.
Patients randomized to the web-based ACP arm receive an introductory packet with information about how to use the website, the PREPARE pamphlet detailing each of the ACP steps, a blank copy of the easy-to-read (i.e., 5th –grade reading level) Pennsylvania Advance Health Care Directive, and instructions for how to have this AD added to their medical record. Participants are directed to a study-specific sign-up page where they are asked to create a PREPARE account using a pre-assigned unique user name. Participants are also given the option of using the PREPARE website on a study-supplied tablet at their oncology practice. We chose to allow this options because low-income patients are less likely to have home internet access.38
Usage activity for all visits to the secure PREPARE website is monitored by study staff, with a weekly report indicating when each participant first and most recently signed into PREPARE. More detailed quarterly reports show the steps each participant visited and how much time was spent on each step. Study staff conduct up to three follow up calls with patients who have not logged on. These calls, spread out over 12 weeks, serve to remind participants to complete PREPARE within the study timeline, assess reasons for non-participation, troubleshoot any technical difficulties, and offer participation on a tablet in clinic for patients who do not want to participate at home
4. Outcome Measures
Successful ACP influences multiple important outcomes (see Figure 1). Our choice of outcomes was based on the organizing framework developed by an international panel of ACP experts.39 Within this framework, we selected priority outcomes related to patients, caregivers, and healthcare utilization for which validated and parsimonious measures have been developed for use with seriously-ill participants (see Table 3). The primary outcome is patient-reported engagement in ACP, measured using a 15-item validated survey with each item scored from 0 to 5, with higher scores indicating higher engagement.40 We chose this this validated measure as our primary outcome because it reflects a patient- and caregiver-centered understanding of ACP as a complex process involving multiple behaviors, including discussions with surrogates and medical providers about wishes for medical care, rather than simply completion of an advance directive.40 We selected a 12-week time-point for patient and caregiver assessments to balance the need to allow sufficient time for ACP to occur without incurring excessive loss to follow up in a seriously-ill population. For patients who die during the study period, we conduct brief additional interviews with bereaved caregivers 12 weeks after a patient’s death, allowing assessment of EOL care and caregiver bereavement adjustment while minimizing participant burden. We review electronic medical records for documented care goals and EOL healthcare utilization and assess ACP implementation costs by tracking staff time spent on each intervention arm. All outcome assessments are conducted by blinded research staff.
Figure 1:
Longitudinal Outcomes of ACP assessed in PEACe-compare
Table 3.
Patient and Family Caregiver Outcomes Measures for the PEACe-compare trial
| Description | Source | Timing |
|---|---|---|
| 15-item ACP engagement survey assessing ACP processes related to choosing a medical decision maker, discussing and documenting preference for care at EOL, flexibility for surrogate decision making, and asking questions of medical providers. Single summary score (range 0-5 with higher scores indicating higher engagement) has high internal consistency (Cronbach’s alpha=0.92) and responsiveness to change.40 | patients | Baseline, 12 weeks |
| “Have you talked with your family or friends about the kind of medical care you would want if you were very sick or near the end of life?”40,62,63 | patients | Baseline, 12 weeks |
| “Have you talked with your doctor about the kind of medical care you would want if you were very sick or near the end of life?”40,62,63 | patients | Baseline, 12 weeks |
| Single question will assess AD completion. We will additionally assess documented care goals by reviewing medical records for any new ACP documentation, including AD forms or documented discussion. | patients, medical record | Baseline, 12 weeks |
| Two validated caregiver-reported questions have been used to measure receipt of goal-concordant EOL care for patients with cancer: (1) “In your opinion, to what extent were [the patient’s] wishes followed in the medical care received in the last month of life?”; and (2) Caregivers asked about patient’s preferred and actual places of death, with questions separated in the survey to minimize conscious comparison.64 | caregivers | Bereavement |
| The Caregiver Evaluation of Quality of End-of-Life Care (CEQUEL) scale is a 13- item instrument (range of scores 13-26) with demonstrated reliability and convergent validity among cancer caregivers.65 Strengths include brevity and clinical relevance, with inclusion of 4 distinct but related factors (prolongation of death, perceived suffering, shared decision-making, and preparation for death) associated with caregiver bereavement outcomes.65 | caregivers | Bereavement |
| Hospital Anxiety and Depression Scale (HADS) is a widely-used 14-item instrument measuring symptoms of depression and anxiety.66 It has been extensively validated for screening emotional distress among advanced cancer patients67 and family members.68 Score of ≥ 8 on either domain indicate significant symptoms of depression or anxiety with good sensitivity and specificity.6 | caregivers | Baseline, 12 weeks, Bereavement |
| Impact of Events Scale is a 22-item self-report instrument for PTSD symptoms with 3 subscales measuring intrusion (8 items), avoidance (8 items) and hyperarousal (6 items).70 Subscales have high internal consistency (Cronbach’s alpha ranging from 0.79-0.92).71 A score of 30 or higher is considered a clinically significant burden. | caregivers | Bereavement |
| American Society of Clinical Oncology’s Quality Oncology Practice Initiative (QOPI)72 and National Quality Forum (NQF)73 -endorsed measures: chemotherapy administered within the last 2 weeks of life; hospitalizations (>1) or ICU admission in the last month of life; > 1 emergency room visit in the last month of life; hospice enrollment; hospice enrollment within 3 days of death; and number of days in hospice. | caregivers, medical record | Bereavement |
Statistical Analysis
We will evaluate the statistical properties of baseline and follow-up outcome measures, including potential outliers, normality, and missing data, using summary statistics and graphical tools. Appropriate transformations may be applied as necessary. To assess the effectiveness of randomization, we will compare distributions of baseline characteristics for patients and caregivers between the two intervention groups. All analyses for treatment group comparisons will use an intention-to-treat approach. Results will be reported following the CONSORT guideline.41
The primary outcome of this study is ACP engagement. We will test the effect of treatment assignment on the primary outcome using linear mixed models. The model will include treatment group, randomization stratification factors (health literacy, whether the patient has previously completed an advance directive), and baseline ACP engagement as fixed effects. To adjust for possible clustering of patients seen by the same facilitator or provider, we will include random effects for facilitators (for patients in the in-person, facilitated ACP group only) and oncologists.
The same analytic approach will be used for continuous secondary outcomes, perceived quality of EOL care and caregiver symptoms of anxiety, depression and PTSD. Binary outcomes such as ACP discussions, AD completion, documented care goals, and receipt of goal-concordant EOL care will be compared using mixed effect logistic regression with the same set of fixed effects and random effects as in the models for continuous outcomes. We will also compare rates of intervention completion between the two intervention groups, using the same model as for other binary outcomes. Analyses of health care utilization data will use GLMM, with logit link (binary distribution) for dichotomous (yes/no) health care outcomes such as chemotherapy within last 2 weeks of life. Again, we will use intervention group, randomization stratification factors, and baseline ACP as fixed effects and oncologist and clinic as random factors.
Missing Data
We prevent and monitor missing data by using an eSYSDM, which will automatically produce an error message notifying the user that the field must be completed when required fields are left blank. We will compare baseline characteristics between patients with complete follow-up to those without by randomization group, in order to assess potential biases that may exist in the complete case analysis. We also record and report all reasons for study drop-out using a withdrawal/termination form to assess the missing data mechanism (missing completely at random, missing at random, or non-ignorable missingness, meaning the data missingness is related to the actual value). We will conduct sensitivity analyses for primary and secondary outcomes using several validated methods: (1) complete case analyses, which assumes missing completely at random; (2) multiple imputation using M=10 imputations, which assumes missing at random; and (3) assigning poor scores and good scores for missing values differentially by treatment group, which aligns with non-ignorable missingness.
Sample Size
We selected our sample size to provide sufficient power to assess differences in the primary outcome—ACP engagement—as well as secondary patient, caregiver, and healthcare utilization outcomes. To adjust for possible clustering effect within each oncologist, we used the design effect (DE=1+[(1+CV2)m-1)]ρ), where CV=the coefficient of variation for cluster sizes42 and m=the number of patients per oncologist. Conservatively estimating a 25% loss to follow-up for patient-reported outcomes, n=400 patients will provide >98% power to detect a moderate effect size (Cohen’s d=0.5) in our primary outcome.40 While any increase in ACP engagement may be associated with clinically meaningful improvements in EOL outcomes,40 a minimal clinically important difference has not been established. A moderate effect size (0.35 on a 5-point score, based on a SD of 0.7)40, ensures that we will be able to detect a difference comparable to the changes seen with ACP interventions in prior trials.9,40
In-depth Interviews to Identify Factors that Influence ACP Effectiveness
Qualitative methods are ideally suited for understanding how and why events occur, illuminating the experiences of different stakeholders to explore new theories and enhance understanding of complex phenomena.43,44 This contextual evaluation component of PEACe-compare expands on the comparative effectiveness data to answer the guiding questions (1) “what makes each ACP intervention work,” and (2) “for whom and under what circumstances is each ACP intervention most effective.”45 We maximize the added value of qualitative research to this trial by ensuring that qualitative data collection and analyses are pre-planned, adequately resourced, and conducted by investigators with extensive qualitative research experience.46-48
A subset of patients and caregivers are invited to participate in in-depth interviews after completing trial outcome assessments (12-week and bereavement) to avoid impacting trial integrity.48 We recruit equal numbers of participants from each ACP intervention arm. Within each intervention arm, we include participants for whom the intervention was effective (defined as an increase of at least 0.2 in the average score of the 15-item ACP engagement survey from baseline to the 12 week timepoint) and not effective (defined as less than 0.2 or decrease in the average score of the survey from baseline to the 12 week timepoint). Only qualitative research staff conducting in-depth interviews have access to information about intervention effects. In addition to stratifying participation based on intervention arm and effect, patients are selected for participation based on variation in sex, race, and age to explore how these variables impact patients’ experiences and outcomes. Recruitment into this subset is rolling and continues throughout the entire study data collection period. This sampling strategy ensures elicitation of a spectrum of experiences regarding what makes each intervention work (or not work) for changing ACP behaviors. We aim for linked patient-caregiver pairs because caregivers may lend additional perspective on patient EOL experiences. Because ACP also involves clinicians and healthcare systems, we ask patients to identify a clinician most involved in decisions about their care and interview these stakeholders after patient and caregiver interviews have been completed. As recommended for qualitative research, the sample size is not fixed. Rather, we will conduct interviews until thematic saturation is reached, meaning no new themes emerge from the data. Based on our prior work, we anticipate that this will involve approximately 60 patients, 60 caregivers, and 60 clinicians.49-52 All participants will provide informed consent.
Our preliminary, semi-structured interview guides are informed by (1) a theoretical framework for ACP based on behavior change theory9,53 and (2) the literature on factors that may affect the ability of ACP interventions to improve ACP. We will interview a subset a caregivers after the patient has died, using a separate interview guide designed for bereaved caregivers. All guides are refined and pilot tested with two to three representative stakeholders prior to use. All interviews are overseen by a PhD qualitative researcher with extensive in-depth interviewing experience, audio-recorded, and transcribed verbatim. Interview questions are modified as new themes emerge from the data. The full interview guides are included in the appendix (see Appendix B).
We will perform iterative, thematic analysis54 to illuminate key contexts and mechanisms influencing outcomes for each ACP strategy. Preliminary coding will be done by the interdisciplinary investigative team, with robustness assessed through a kappa statistic. Once the coding scheme is standardized, the remainder of coding will be conducted by research staff using qualitative analysis software, with supervision from investigators and cross-checking to ensure confirmability. Regular interdisciplinary meetings will be held to identify themes emerging from the data and revise the initial conceptual model. The final product will be a detailed conceptual model providing plausible explanations about how, for whom, and in what circumstances each ACP intervention influences outcomes.
5. DISCUSSION
PEACe-compare is a single-blind, patient-level randomized trial that incorporates mixed methods to compare the effectiveness of in-person, facilitated ACP vs web-based ACP among 400 patients with advanced cancer and their family caregivers. By testing the effect of these two interventions on patient and family caregiver outcomes; comparing implementation costs and healthcare utilization at end of life; and identifying contexts and mechanisms that influence the effectiveness of each approach, this trial will provide rich data about whether one form of ACP is more effective—and if so, for whom and under what circumstances.
Our trial addresses a major public health problem that arises in the care of patients with incurable malignancies: failure to deliver end-of-life (EOL) care consistent with patients’ needs, values and preferences. Decades of research highlight the challenges with the process and outcomes of treatment decisions near end of life (EOL) in advanced cancer. The minority of patients discuss EOL wishes with their oncologists,18 and documentation of care goals is poor.19,20 Families are often forced to make decisions about life sustaining treatments without knowing what their loved one would want.2,3 While most patients with incurable illness prefer to focus on comfort, minimize days in the hospital, and die at home,21-23 many patients with advanced cancer continue to receive aggressive treatments in the last month of life.24,25 Aggressive treatments near EOL for patients with advanced cancer may not reflect patient preferences,26 have been associated with worse caregiver bereavement outcomes,27,28 have not been shown to improve survival, 29 and are associated with escalating health care costs.30 The National Academy of Medicine highlights the failure of patients with advanced cancer to “receive end-of-life care consistent with their needs, values and preferences” as a key shortcoming of our current cancer care delivery system.3
Evidence suggests that ACP helps patients to understand and share their personal values, life goals, and preferences for future medical care,2,36,37 and may lead to receipt of care near the end of life that reflects their values and preferences,7,38,39 decreased use of unwanted EOL treatments,28,40,41 and decreased burden on surrogate decision makers and bereaved family members.28,42,43 Among patients with advanced cancer, earlier EOL discussions are associated with less aggressive medical care near death.44 In contrast, more aggressive EOL cancer care is associated with worse bereavement outcomes in caregivers.27,28,45 Recognition that traditional written ADs fail to improve EOL care for many patients has led to the development of more patient-centered ACP approaches focused on promoting discussions about preferences and goals. These interventions do not directly involve oncologists but may improve advanced cancer care by encouraging patients to communicate their goals and preferences.
The best approach to ACP is currently not known. To date, most ACP studies have relied on observational designs,55,56 used hypothetical vignettes rather than true clinical outcomes,57,58 or compared an ACP intervention to usual care.59,60 Head-to-head comparisons of different ACP interventions are lacking. To improve care, leaders in healthcare research and quality have called for evaluations that move beyond traditional “cause and effect” analyses to provide insight into how, why and when interventions work.61 This study will provide comparative effectiveness and contextual data to fill a particularly important evidence gap for oncology, where failure to engage patients in ACP and provide patient-centered EOL care is widely recognized as a critical public health problem for which solutions are urgently needed.3
Data from this study will inform clinicians, payers and health systems seeking to adopt and scale the most effective and efficient ACP strategy in real-world oncology settings, with the potential to improve care for the over 600,000 Americans who die annually from cancer and their families.
Supplementary Material
HIGHLIGHTS.
Advance care planning (ACP) helps ensure medical care reflects patients’ goals.
In-person and web-based ACP differ in costs, scalability, and implementation.
It is unknown which ACP method is most impactful for patients with advanced cancer.
This mixed-methods trial compares effectiveness of in-person versus web-based ACP.
ACKNOWLEDGEMENTS AND FUNDING
This trial is supported by R01CA235730 from the National Cancer Institute. This project also uses the Hillman clinical facilities and staff that are supported in part by the National Cancer Institute award P30CA047904. Dr. Douglas B. White’s effort was partially funded by National Institutes of Health K24HL148314-01.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
EOL: End of Life
ACP: Advance Care Planning
REFERENCES
- 1.American Cancer Society. Cancer Facts and Figures 2019. Available at https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2019.html [access date January 27, 2020].
- 2.Institute of Medicine. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: The National Academies Press; 2014. Available at https://www.nap.edu/read/18748/chapter/1 [access date January 27, 2020]. [PubMed] [Google Scholar]
- 3.Institute of Medicine. Delivering High Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: National Academies Press; 2013. Available at https://www.nap.edu/read/18359/chapter/1 [access date January 27, 2020]. [PubMed] [Google Scholar]
- 4.Bestvina CM, Polite BN. Implementation of Advance Care Planning in Oncology: A Review of the Literature. J Oncol Pract. October 2017;13(10):657–62. DOI: 10.1200/JOP.2017.021246. PMID: 28586259. [DOI] [PubMed] [Google Scholar]
- 5.Kline RM, Bazell C, Smith E, Schumacher H, Rajkumar R, Conway PH. Centers for medicare and medicaid services: using an episode-based payment model to improve oncology care. J Oncol Pract. March 2015;11(2):114–6. DOI: 10.1200/JOP.2014.002337. PMID: 25690596. [DOI] [PubMed] [Google Scholar]
- 6.Briggs L Shifting the focus of advance care planning: using an in-depth interview to build and strengthen relationships. J Palliat Med. April 2004;7(2):341–9. DOI: 10.1089/109662104773709503. PMID: 15130215. [DOI] [PubMed] [Google Scholar]
- 7.Detering KM, Hancock AD, Reade MC, Silvester W. The impact of advance care planning on end of life care in elderly patients: randomised controlled trial. BMJ. March 23, 2010;340:c1345 DOI: 10.1136/bmj.c1345. PMCID: PMC2844949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Sudore RL, Boscardin J, Feuz MA, McMahan RD, Katen MT, Barnes DE. Effect of the PREPARE Website vs an Easy-to-Read Advance Directive on Advance Care Planning Documentation and Engagement Among Veterans: A Randomized Clinical Trial. JAMA Intern Med. August 1, 2017;177(8):1102–9. DOI: 10.1001/jamainternmed.2017.1607. PMCID: PMC5710440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sudore RL, Knight SJ, McMahan RD, et al. A novel website to prepare diverse older adults for decision making and advance care planning: a pilot study. J Pain Symptom Manage. April 2014;47(4):674–86. DOI: 10.1016/j.jpainsymman.2013.05.023. PMCID: PMC4111443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.McMahan RD, Knight SJ, Fried TR, Sudore RL. Advance care planning beyond advance directives: perspectives from patients and surrogates. J Pain Symptom Manage. September 2013;46(3):355–65. DOI: 10.1016/j.jpainsymman.2012.09.006. PMCID: PMC4111444. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sudore RL, Fried TR. Redefining the "planning" in advance care planning: preparing for end-of-life decision making. Ann Intern Med. August 17, 2010;153(4):256–61. DOI: 10.7326/0003-4819-153-4-201008170-00008. PMCID: PMC2935810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Johnson S, Butow P, Kerridge I, Tattersall M. Advance care planning for cancer patients: a systematic review of perceptions and experiences of patients, families, and healthcare providers. Psychooncology. April 2016;25(4):362–86. DOI: 10.1002/pon.3926. PMID: 26387480. [DOI] [PubMed] [Google Scholar]
- 13.National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology: Palliative Care. February, 2019. Available at https://www.nccn.org/professionals/physician_gls/default.aspx#supportive [access date January 27, 2020.] [DOI] [PubMed] [Google Scholar]
- 14.Becker CL, Arnold RM, Park SY, et al. A cluster randomized trial of a primary palliative care intervention (CONNECT) for patients with advanced cancer: Protocol and key design considerations. Contemp Clin Trials. March 2017;54:98–104. DOI: 10.1016/j.cct.2017.01.005. PMCID: PMC5331884. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schenker Y, White D, Rosenzweig M, et al. Care management by oncology nurses to address palliative care needs: a pilot trial to assess feasibility, acceptability, and perceived effectiveness of the CONNECT intervention. J Palliat Med. March 2015;18(3):232–40. DOI: 10.1089/jpm.2014.0325. PMCID: PMC4347888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Walling A, Lorenz KA, Dy SM, et al. Evidence-based recommendations for information and care planning in cancer care. J Clin Oncol August 2008;26(23):3896–902. DOI: 10.1200/JCO.2007.15.9509. PMID:18688058 [DOI] [PubMed] [Google Scholar]
- 17.Fried TR, O'Leary JR. Using the experiences of bereaved caregivers to inform patient- and caregiver-centered advance care planning. J Gen Intern Med. October 2008;23(10):1602–7. DOI: 10.1007/s11606-008-0748-0. PMCID: PMC2533358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lamont EB, Siegler M. Paradoxes in cancer patients' advance care planning. J Palliat Med. Spring 2000;3(1):27–35 DOI: 10.1089/jpm.2000.3.27. PMID:15859719. [DOI] [PubMed] [Google Scholar]
- 19.Barnes K, Jones L, Tookman A, King M. Acceptability of an advance care planning interview schedule: a focus group study. Palliat Med. January 2007;21(1):23–8. DOI: 10.1177/0269216306073638. PMID: 17169956. [DOI] [PubMed] [Google Scholar]
- 20.Moss AH, LunNey JR, Culp S, et al. Prognostic significance of the "surprise" question in cancer patients. j Palliat Med. July 2010;13(7):837–40. DOI: 10.1089/jpm.2010.0018. PMID: 20636154. [DOI] [PubMed] [Google Scholar]
- 21.Billings JA, Bernacki R. Strategic targeting of advance care planning interventions: the Goldilocks phenomenon. JAMA Intern Med. April 2014;174(4):620–4. DOI: 10.1001/jamainternmed.2013.14384. PMID: 24493203. [DOI] [PubMed] [Google Scholar]
- 22.Hui D, Didwaniya N, Vidal M, et al. Quality of end-of-life care in patients with hematologic malignancies: a retrospective cohort study. Cancer. May 15, 2014;120(10):1572–8. DOI: 10.1002/cncr.28614. PMCID: PMC4013204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.LeBlanc TW, Abernethy AP, Casarett DJ. What is different about patients with hematologic malignancies? A retrospective cohort study of cancer patients referred to a hospice research network. J Pain Symptom Manage. March 2015;49(3):505–12. DOI: 10.1016/j.jpainsymman.2014.07.003. PMID: 25116911. [DOI] [PubMed] [Google Scholar]
- 24.Sudore RL, Landefeld CS, Williams BA, Barnes DE, Lindquist K, Schillinger D. Use of a modified informed consent process among vulnerable patients: a descriptive study. J Gen Intern Med. August 2006;21(8):867–73. DOI: 10.1111/j.1525-1497.2006.00535.x. PMCID: PMC1831581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Schenker Y, Bahary N, Claxton R, et al. A pilot trial of early specialty palliative care for patients with advanced pancreatic cancer: challenges encountered and lessons learned. J Palliat Med. January 2018; 21(1): 28–36. DOI: 10.1089/jpm.2017.0113. PMCID: PMC5757080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ferrell B, Sun V, Hurria A, et al. Interdisciplinary palliative care for patients with lung cancer. J Pain Symptom Manage. December 2015;50(6):758–67. DOI: 10.1016/j.jpainsymman.2015.07.005. PMCID: PMC4666729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Sun V, Grant M, Koczywas M, et al. Effectiveness of an interdisciplinary palliative care intervention for family caregivers in lung cancer. Cancer. October 2015;121(20):3737–45. DOI: 10.1002/cncr.29567. PMCID: PMC4592403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Chew LD, Griffin JM, Partin MR, et al. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. May 2008;23(5):561–6. DOI: 10.1007/s11606-008-0520-5. PMCID: PMC2324160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hong M, Yi EH, Johnson KJ, Adamek ME. Facilitators and Barriers for Advance Care Planning Among Ethnic and Racial Minorities in the U.S.: A Systematic Review of the Current Literature. J Immigr Minor Health. October 2018; 20(5): 1277–1287. DOI: 10.1007/s10903-017-0670-9. PMID: 29124502. [DOI] [PubMed] [Google Scholar]
- 30.Harrison KL, Adrion ER, Ritchie CS, Sudore RL, Smith AK. Low Completion and Disparities in Advance Care Planning Activities Among Older Medicare Beneficiaries. JAMA Intern Med. December 2016;176(12):1872–5. DOI: 10.1001/jamainternmed.2016.6751. PMCID: PMC5304942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hammes B, Rooney BL, & Gundrum J A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc. July 2010;58(7):1249–55. DOI: 10.1111/j.1532-5415.2010.02956.x. PMID: 20649688. [DOI] [PubMed] [Google Scholar]
- 32.Pecanac KE, Repenshek MF, Tennenbaum D, Hammes BJ. Respecting Choices(R) and advance directives in a diverse community. J Palliat Med. March 2014;17(3):282–7. DOI: 10.1089/jpm.2013.0047. PMID: 24325558. [DOI] [PubMed] [Google Scholar]
- 33.in der Schmitten J, Lex K, Mellert C, Rotharmel S, Wegscheider K, & Marckmann G Implementing an advance care planning program in German nursing homes. Dtsch Arztebl Int. January 2014;111(4):50–7. DOI: 10.3238/arztebl.2014.0050. PMCID: PMC3950824. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Teo W-SK, Raj AG, Tan WS et al. . Economic impact analysis of end-of-life programme for nursing home residents. Palliat Med. May 2014;28(5):430–7. DOI: 10.1177/0269216314526270. PMID: 24651709. [DOI] [PubMed] [Google Scholar]
- 35.Kirchhoff KT, Hammes BJ, Kehl KA, Briggs LA, Brown RL. Effect of a disease-specific planning intervention on surrogate understanding of patient goals for future medical treatment. J Am Geriatr Soc. July 2010;58(7):1233–40. DOI: 10.1111/j.15325415.2010.02760.x. PMCID: PMC2964863. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Cresswell MA, Robinson CA, Fyles G, Bottorff JL, Sudore R. Evaluation of an advance care planning web-based resource: applicability for cancer treatment patients. Support Care Cancer. March 2018;26(3):853–60. DOI: 10.1007/s00520-017-3901-7. PMID: 28929291. [DOI] [PubMed] [Google Scholar]
- 37.Sudore RL, Schellinger D, Katen MT, Boscardin WJ, Osua S, Barnes DE. Engaging Diverse English- and Spanish-Speaking Older Adults in Advance Care Planning: The PREPARE Randomized Clinical Trial. JAMA Intern Med. December 1, 2018;178(23):1616–25. DOI: 10.1001/jamainternmed.2018.4657. PMCID: PMC6342283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Anderson M Digital divide persists even as lower-income American make gains in tech adoption. Washington, DC: Pew Research Center; May 7, 2019. Available at http://www.pewresearch.org/fact-tank/2017/03/22/digital-divide-persists-even-as-lower income-americans-make-gains-in-tech-adoption/ [accessed January 27, 2020]. [Google Scholar]
- 39.Sudore RL, Heyland DK, Lum HD, et al. Outcomes that Define Successful Advance Care Planning: A Delphi Panel Consensus. J Pain Symptom Manage. February 2018:55(2):245–255. DOI: 10.1016/j.jpainsymman.2017.08.025. PMCID: PMC5794507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Sudore RL, Heyland DK, Barnes DE, et al. Measuring Advance Care Planning: Optimizing the Advance Care Planning Engagement Survey. J Pain Symptom Manage. April 2017;53(4):669–81.e8. DOI: 10.1016/j.jpainsymman.2016.10.367. PMCID: PMC5730058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. June 1 2010;152(11):726–32. DOI: 10.7326/0003-4819-152-11-201006010-00232. PMID: 20335313. [DOI] [PubMed] [Google Scholar]
- 42.Zimmermann C, Swami N, Krzyzanowska M, et al. Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial. Lancet. May 17, 2014;383(9930):1721–30. DOI: 10.1016/S0140-6736(13)62416-2. PMID: 24559581. [DOI] [PubMed] [Google Scholar]
- 43.Sofaer S Qualitative methods: what are they and why use them? Health Serv Res. December 1999;34(5 Pt 2):1101–18. PMCID: PMC1089055. [PMC free article] [PubMed] [Google Scholar]
- 44.Kazi M Realist evaluation for practice. British Journal of Social Work. 2003;33:803–18. [Google Scholar]
- 45.Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in london. Milbank Q. June 2009;87(2):391–416. DOI: 10.1111/j.1468-0009.2009.00562.x. PMCID: PMC2881448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.O'Cathain A, Goode J, Drabble SJ, Thomas KJ, Rudolph A, Hewison J. Getting added value from using qualitative research with randomized controlled trials: a qualitative interview study. Trials. June 9, 2014;15:215 DOI: 10.1186/1745-6215-15-215. PMCID: PMC4059032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.O'Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Goode J, Hewison J. Maximising the value of combining qualitative research and randomised controlled trials in health research: the QUAlitative Research in Trials (QUART) study--a mixed methods study. Health Technol Assess. June 2014;18(38):1–197, v-vi. DOI: 10.3310/hta18380. PMCID: PMC4781055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Cooper C, O'Cathain A, Hind D, Adamson J, Lawton J, Baird W. Conducting qualitative research within Clinical Trials Units: avoiding potential pitfalls. Contemp Clin Trials. July 2014;38(2):338–43. DOI: 10.1016/j.cct.2014.06.002. PMID: 24937019. [DOI] [PubMed] [Google Scholar]
- 49.Schenker Y, Crowley-Matoka M, Dohan D, Tiver GA, Arnold RM, White DB. I don't want to be the one saying 'we should just let him die': intrapersonal tensions experienced by surrogate decision makers in the ICU. J Gen Int Med. December 2012;27(12):1657–65. DOI: 10.1007/s11606-012-2129-y. PMCID: PMC3509291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Schenker Y, Crowley-Matoka M, Dohan D, et al. Oncologist factors that influence referrals to subspecialty palliative care clinics. J Oncol Pract. March 2014;10(2):e37–44. DOI: 10.1200/JOP.2013.001130. PMCID: PMC3948709. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Schenker Y, White DB, Crowley-Matoka M, Dohan D, Tiver GA, Arnold RM. "It hurts to know... and it helps": exploring how surrogates in the ICU cope with prognostic information. J Palliat Med. March 2013;16(3):243–9. DOI: 10.1089/jpm.2012.0331. PMCID: PMC3583249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Guest BA, Johnson L. How Many Interviews Are Enough? An Experiment with Data Saturation and Variability. Field Methods. 2006;18(1):59–82. [Google Scholar]
- 53.Prochaska JO, Velicer WF. The transtheoretical model of health behavior change. Am J Health Promot. Sep-Oct 1997;12(1):38–48. DOI: 10.4278/0890-1171-12.1.38. PMID: 10170434. [DOI] [PubMed] [Google Scholar]
- 54.Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci. September 2013; 15(3):398–405. DOI: 10.1111/nhs.12048. PMID: 23480423. [DOI] [PubMed] [Google Scholar]
- 55.Silveira MJ, Kim SY, Langa KM. Advance directives and outcomes of surrogate decision making before death. N Engl J Med. April 1, 2010;362(13):1211–8. DOI: 10.1056/NEJMsa0907901. PMCID: PMC2880881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Nicholas LH, Langa KM, Iwashyna TJ, Weir DR. Regional variation in the association between advance directives and end-of-life Medicare expenditures. JAMA. October 5, 2011;306(13):1447–53. DOI: 10.1001/jama.2011.1410. PMCID: PMC3332047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Thompson T, Barbour R, Schwartz L. Adherence to advance directives in critical care decision making: vignette study. BMJ. November 1, 2003;327(7422):1011 DOI: 10.1136/bmj.327.7422.1011. PMCID: PMC261651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Hardin SB, Yusufaly YA. Difficult end-of-life treatment decisions: do other factors trump advance directives? Arch Intern Med. July 26, 2004;164(14):1531–3. DOI: 10.1001/archinte.164.14.1531. PMID: 15277284. [DOI] [PubMed] [Google Scholar]
- 59.Houben CH, Spruit MA, Groenen MT, Wouters EF, Janssen DJ. Efficacy of advance care planning: a systematic review and meta-analysis. J Am Med Dir Assoc. July 2014;15(7):477–89. DOI: 10.1016/j.jamda.2014.01.008. PMID: 24598477. [DOI] [PubMed] [Google Scholar]
- 60.Brinkman-Stoppelenburg A, Rietjens JA, van der Heide A. The effects of advance care planning on end-of-life care: a systematic review. Palliat Med. September 2014;28(8):1000–25. DOI: 10.1177/0269216314526272. PMID: 24651708. [DOI] [PubMed] [Google Scholar]
- 61.Berwick DM. The science of improvement. JAMA. March 12, 2008;299(10):1182–4. DOI: 10.1001/jama.299.10.1182. PMID: 18334694. [DOI] [PubMed] [Google Scholar]
- 62.Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. October 8, 2008;300(14):1665–73. DOI: 10.1001/jama.300.14.1665. PMCID: PMC2853806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG. End-of-life discussions, goal attainment, and distress at the end of life: predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. March 1, 2010;28(7):1203–8. DOI: 10.1200/JCO.2009.25.4672. PMCID: PMC2834470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Wright AA, Keating NL, Ayanian JZ, et al. Family Perspectives on Aggressive Cancer Care Near the End of Life. JAMA. January 19, 2016;315(3):284–92. DOI: 10.1001/jama.2015.18604. PMCID: PMC4919118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Higgins PC, Prigerson HG. Caregiver evaluation of the quality of end-of-life care (CEQUEL) scale: the caregiver's perception of patient care near death. PLoS One. June 6, 2013;8(6):e66066 DOI: 10.1371/journal.pone.0066066. PMCID: PMC3675191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. June 1983;67(6):361–70. DOI: 10.1111/j.1600-0447.1983.tb09716.x. PMID: 6880820. [DOI] [PubMed] [Google Scholar]
- 67.Vodermaier A, Millman RD. Accuracy of the Hospital Anxiety and Depression Scale as a screening tool in cancer patients: a systematic review and meta-analysis. Support Cancer Care. December 2011;19(12):1899–908. DOI: 10.1007/s00520-011-1251-4. PMID: 21898134. [DOI] [PubMed] [Google Scholar]
- 68.Pochard F, Darmon M, Fassier T, et al. Symptoms of anxiety and depression in family members of intensive care unit patients before discharge or death. A prospective multicenter study. J Crit Care. March 2005;20(1):90–6. DOI: 10.1016/j.jcrc.2004.11.004. PMID: 16015522. [DOI] [PubMed] [Google Scholar]
- 69.Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res. February 2002;52(2):69–77. DOI: 10.1016/s0022-3999(01)00296-3. PMID: 11832252. [DOI] [PubMed] [Google Scholar]
- 70.Sundin EC, Horowitz MJ. Horowitz's Impact of Event Scale evaluation of 20 years of use. Psychosom Med. Sep-Oct 2003;65(5):870–6. DOI: 10.1097/01.psy.0000084835.46074.f0. PMID: 14508034. [DOI] [PubMed] [Google Scholar]
- 71.Sundin EC, Horowitz MJ. Impact of Event Scale: psychometric properties. Br J Psychiatry. March 2002;180:205–9. DOI: 10.1192/bjp.180.3.205. PMID: 11872511. [DOI] [PubMed] [Google Scholar]
- 72.Campion FX, Larson LR, Kadlubek PJ, Earle CC, Neuss MN. Advancing performance measurement in oncology: quality oncology practice initiative participation and quality outcomes. J Oncol Pract. May 2011;7(3 Suppl):31s–5s. DOI: 10.1200/JOP.2011.000313. DOI: 10.1200/JOP.2011.000313. PMCID: PMC3092462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.National Quality Forum. National Voluntary Consensus Standards for Quality of Cancer Care. Washington, DC; 2009. Available at: http://www.qualityforum.org/publications/2009/05/National_voluntary_consensus_stand rds_for_Quality_of_Cancer_Care.aspx [access date 1/27/2020]. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.

