Abstract
Background and Objectives
While advance care planning (ACP) is critical for ensuring optimal end-of-life outcomes among individuals with mild cognitive impairment (MCI), many individuals who may benefit from ACP have not initiated this process. This article aims to describe the iterative design of an MCI group visit-based intervention and evaluate the feasibility and acceptability of the intervention.
Research Design and Methods
We used human-centered design, rapid-cycle prototyping, and multiple methods to adapt an ENgaging in Advance Care planning Talks (ENACT) Group Visits intervention. We convened an advisory panel of persons with MCI and care partners (n = 6 dyads) to refine the intervention and conducted a single-arm pilot of 4 MCI ENACT intervention prototypes (n = 13 dyads). We used surveys and interviews to assess outcomes from multiple perspectives.
Results
The advisory panel affirmed that ACP is a priority for individuals with MCI, described the need for ACP in a group setting, and suggested refinements to ACP resources for the MCI ENACT intervention. Feasibility of recruitment was limited. MCI ENACT intervention participants strongly agreed that group discussions provided useful information and recommended the intervention. Themes supporting acceptability included (a) feedback on acceptability of the intervention, (b) previous experiences with ACP, and (c) reasons for participation, including desire for discussions about MCI and how it relates to ACP.
Discussion and Implications
Despite stakeholders’ positive ratings of acceptability of the MCI ENACT intervention, future work is needed to enhance the feasibility of recruitment to support implementation into clinical settings.
Keywords: Dementia, End-of-life care planning, Stakeholder engagement
Background
Advance care planning (ACP) is a process where patients, families, and health care practitioners reflect, discuss, and make decisions related to the person’s goals and preferences for future medical care (Sudore, Lum et al., 2017). ACP may benefit patients, families, and health care systems through increased autonomy, dignity, and peace, reduced intensity of family grieving, and decreased resource utilization (McMahan et al., 2021; Steinhauser et al., 2000; Wright et al., 2008). ACP is important for individuals with mild cognitive impairment (MCI) due to higher risk for progressive cognitive and functional decline than those without MCI (deLima Thomas et al., 2018).
ACP is critical for improving end-of-life outcomes by allowing individuals with MCI to participate in decision making about future medical care before potentially losing decision-making capacity and communication abilities (Dixon et al., 2018; Mitchell et al., 2009). ACP can be difficult for individuals with MCI due to individual/family barriers such as difficulty understanding the diagnosis of MCI, ambiguity related to disease trajectory, and uncertainty about the need for future care planning (deLima Thomas et al., 2018; Gomersall et al., 2015). ACP among individuals with MCI is low, with one study indicating less than half of individuals with MCI (39%) had any type of ACP within 5 years following diagnosis (Garand et al., 2011).
To promote ACP discussions in clinical settings, previous research has tested ACP interventions and resources such as websites, visuals, and games (McMahan et al., 2021); however, challenges to implementation into clinical settings have been identified (Lund et al., 2015). We previously began addressing this challenge by developing an ACP group visit intervention called ENgaging in Advance Care planning Talks (ENACT Group Visits intervention) and tested this model in primary care among older adults without cognitive impairment (Lum et al., 2020). The overarching goal of the current research is to refine the ENACT Group Visits intervention for individuals with MCI using a participatory research process to develop an MCI ENACT intervention and evaluate if it is feasible and acceptable.
Theoretical Background
Group visits bring patients together for medical care, education, and patient engagement (Wadsworth et al., 2019). Group visits for individuals with MCI have focused on the effects of a group-based intervention on neurological outcomes (Jeong et al., 2016); but there are no existing ACP group visits for persons affected by MCI. The ENACT Group Visits intervention integrates ACP Engagement Theory (patient-level), Collaborative Learning Theory (interpersonal-level), and strengths of group visits. ACP Engagement Theory describes ACP as a health behavior that includes factors affecting change including knowledge, contemplation, self-efficacy, and readiness (Sudore et al., 2013). Collaborative Learning Theory describes mechanisms for learning in the context of group visits, including the central tenets: learning is a shared experience, learners are diverse (e.g., cultural backgrounds, learning styles), and learners have personal experiences which can lend expertise alongside factual or expert-provided information (Bruffee, 1993). Through the ENACT Group Visits intervention, clinician facilitators promote patient education and ACP goal setting through an interactive group discussion that aims to promote ACP behavior change.
The ENACT Group Visits intervention involved groups of community-dwelling older adults (aged 65+) who participated in two group sessions, 1 month apart, consisting of between 8 and 12 participants, facilitated by a physician and social worker, using a protocolized implementation manual (i.e., systematic procedures for conducting ENACT Group Visits; Lum et al., 2016). The discussions of ACP topics are interactive and supported by ACP resources. These older adults rated the group visits as better than usual clinic visits for talking about ACP, thought they received useful information, felt comfortable talking about ACP, and found talking with peers about ACP to be helpful. A randomized controlled trial conducted in a geriatric primary care clinic indicated that patients in the ENACT Group Visits intervention compared to patients receiving ACP resources by mail had significantly higher rates of advance directives and of medical decision-maker documentation in the electronic health record (Lum et al., 2020).
Objectives
Because the ENACT Group Visits intervention was designed for older adults without cognitive impairment, we sought to engage older adults with MCI and family care partners in a participatory research process and user testing in which these stakeholders informed an MCI ENACT intervention designed by and for individuals affected by MCI (Corrado et al., 2020; Ideo.org., n.d.). The purpose of this article is to describe the iterative design and evaluation process, specifically, the key refinements to the ENACT Group Visits intervention that resulted in the MCI ENACT intervention, and to report implementation outcomes of feasibility and acceptability of the MCI ENACT intervention.
Research Design and Methods
Study Design
We engaged in an iterative human-centered design process, whereby we gathered information from stakeholders about needed intervention changes, revised the intervention with iterative prototyping, tested the prototypes, and evaluated the feasibility and acceptability of the prototypes using multiple methods and triangulation (Figure 1; Lyon & Koerner, 2016). This design process is grounded in human-centered design principles (Corrado et al., 2020; Design Kit), which allows for development of new or refined interventions by incorporating the perspectives and ideas from the stakeholders for whom the intervention is designed.
Figure 1.
Overview of mild cognitive impairment (MCI) ENgaging in Advance Care Planning Talks (ENACT) intervention design process. Using a participatory research process, an Advisory Panel met three times to iteratively inform refinements to an MCI ENACT group visit prototype. Four MCI ENACT prototypes were conducted over 9 months. Quantitative and qualitative data were collected after MCI ENACT prototypes, shared at Advisory Panel meetings, and used to iteratively test another MCI ENACT prototype. Arrows represent the interactive process of using input from advisory panel meetings to design MCI ENACT prototypes and subsequently sharing quantitative and qualitative data from the prototypes with the advisory panel.
As an overview, to refine the ENACT Group Visits intervention for individuals with MCI, we convened an advisory panel of individuals with MCI and their family care partners (family care partners could be a partner, spouse, or friend). The role of the advisory panel was to provide input into the design of the refined MCI ENACT intervention and respond to the qualitative input from participants who tested the prototype MCI ENACT intervention. We also conducted four MCI ENACT intervention prototypes tests, with adaptations made between each test. We collected data from the test participants to explore the feasibility and acceptability of MCI ENACT intervention prototypes for individuals with MCI and family care partners. The changes for each MCI ENACT prototype were guided by multilevel stakeholder input, including feedback from the advisory panel, quantitative and qualitative data from prototype test participants, as well as research team considerations to develop a pragmatic intervention for individuals with MCI.
Setting and Participants
For the advisory panel and the MCI ENACT intervention prototypes, individuals with MCI and family care partners were recruited from research and clinical settings associated with an academic medical center in Aurora, CO, as well as community-based self-referrals. We included family care partners based on previous research indicating the importance of including a support system for individuals with MCI (Lingler et al., 2016). Participants were referred from an unrelated longitudinal study of MCI at the University of Colorado Alzheimer’s and Cognition Center and from a University geriatrics clinic. Community- or self-referrals were also potentially eligible. The Alzheimer’s Association of Colorado disseminated the recruitment flyer via an online email listserv for “early-stage” clients and the Alzheimer’s Association TrialMatch. Diagnosis of MCI was based on the National Institute on Aging and Alzheimer’s Association workgroup clinical criteria for MCI but departed from the core clinical criteria in that we focused specifically on individuals with self- or family care partner-reported memory impairment. Study criteria for MCI included (a) cognitive complaint that reflects a decline in cognition, (b) preservation of independence in functional activities, and (c) not meeting criteria for dementia.
We recruited participants for the advisory panel and then for the four MCI ENACT interventions prototypes. Dyads who enrolled in the advisory panel were eligible to participate in an MCI ENACT intervention prototype if desired (one dyad participated in both the advisory panel and an MCI ENACT intervention). Recruitment goals were to have six dyads recruited for the advisory panel and three to five dyads for each of the four MCI ENACT intervention prototypes. Patients referred to the study from the unrelated longitudinal study and the geriatrics clinic were mailed a letter explaining the study along with a copy of the consent form. Participants were recruited over the phone by research assistants who screened for eligibility and reviewed the consent form with participants.
The same inclusion criteria were applied to the advisory panel and MCI ENACT prototype participants and consisted of (a) documented or self-reported diagnosis of MCI, (b) age 60 years or older, and (c) family care partner who could participate. Exclusion criteria were (a) three or more errors (out of 10) on the Short Portable Mental Status Questionnaire (only applied to individuals referred from the University geriatrics clinic and community self-referrals; Pfeiffer, 1975); (b) hearing issues that would limit group discussion; (c) inability to travel to the study site; or (d) inability to demonstrate capacity to consent (Sudore et al., 2006).
Ethics
To ensure ethical enrollment and inclusion of individuals with MCI, we used an informed consent process that required a functional demonstration of decision-making capacity through a “teach-to-goal” method developed for vulnerable older adults (Sudore et al., 2006; details are given in Supplementary Material). This maximized the ability of cognitively impaired adults to participate and protected participants who could not demonstrate understanding of the study purpose (Prusaczyk et al., 2017). All participants signed the consent form at a study meeting. This research was approved by the Colorado Multiple Institutional Review Board. All participants received a $50 stipend for their participation. Longitudinal cohort members received a $50 stipend per advisory panel meeting.
Human-Centered Design With Advisory Panel
To identify MCI ENACT intervention refinements, the advisory panel participated in three meetings over 9 months (February–October 2019) for ongoing input for the four MCI ENACT intervention prototypes (Figure 1). The meetings were designed to elicit feedback about how to refine the original ENACT Group Visits intervention for individuals with MCI. Each advisory meeting was cofacilitated by the study PI and a study staff member, lasted approximately 2 h, and was audio-recorded for transcription.
The first advisory panel meeting occurred prior to the MCI ENACT intervention prototypes and aimed to elicit input on how to refine the first two MCI ENACT intervention prototypes by introducing the topic of ACP and describing the ENACT Group Visits intervention to patient–care partner dyads. Using Table 1 as a guide, we asked the advisory panel to reflect on potential alignment and impact of the MCI ENACT interventions for individuals with MCI and family care partners. The second meeting centered on a review of ACP tools, resources, and advance directives previously used in the first two MCI ENACT interventions and additional ACP materials for consideration for use in the final two prototypes. The ACP materials were selected based on input from the first advisory panel meeting about the need for acceptable materials for individuals with MCI. The final meeting provided overall feedback on the design of the MCI ENACT intervention. We presented a summary table of the ACP resources and tools provided to participants of the first three MCI ENACT interventions along with proposed resources for the fourth prototype (Table 1).
Table 1.
Refinement of MCI ENACT Intervention Prototypes
| Intervention component | MCI ENACT prototype 2.1 N = 4 dyads |
MCI ENACT prototype 2.2 N = 3 dyads |
MCI ENACT prototype 2.3 N = 3 dyads |
MCI ENACT prototype 2.4 N = 3 dyads |
|---|---|---|---|---|
| Number of sessions | One session | Two sessions (4 weeks apart) | Two sessions (4 weeks apart) | Two sessions (2 weeks apart) |
| ACP videos | Choose flexibility for your decision makerb | ABC TV World News Clipb Choose flexibility for your decision makere |
Be Open, Be Ready, Be Hearda Ask doctors the right questionsc Choose flexibility for your decision makere |
Be Open, Be Ready, Be Hearda |
| Mailed ACP materials | One-page MDPOAb PREPARE pamphlete |
One-page MDPOAb Conversation starter kitb |
One-page MDPOAb Conversation starter kitb |
One-page MDPOAb Five steps for ACPf |
| In-person ACP materials | One-page MDPOAb risks, benefits, and burdens cardf | One-page MDPOA MOST formd |
One-page MDPOAb PREPARE pamphlete Risks/benefits/burdens cardf ACP wallet cardb PREPARE easy to read advance directivee MOST formd |
One-page MDPOAb Five steps for ACP Go wish cardsc PREPARE easy to read advance directivee |
Notes: MCI = mild cognitive impairment; ENACT = ENgaging in Advance Care planning Talks; ACP = advance care planning; MD = medical doctor; MDPOA = Medical Durable Power of Attorney.
References: awww.advancecareplanning.org.au; bhttps://theconversationproject.org/; cwww.gowish.org; dwww.polst.org; ewww.prepareforyourcare.org; favailable upon request.
Iterative MCI ENACT Intervention Prototypes
Four unique MCI ENACT interventions were conducted based on input from the advisory panel, feedback from the MCI ENACT intervention participants (Figure 1), and researcher input using iterative prototyping principles (Lyon & Koerner, 2016). The MCI ENACT intervention prototypes explored refinements such as number of group participants, recruitment of dyads (instead of the patient only), number of sessions (one vs. two group visits), ACP resources, and types of advance directives available during the groups. For example, one planned refinement was to invite three to five dyads (n = 6–10 total participants) to each MCI ENACT intervention, rather than 8–12 participants based on published MCI and dementia group visits with five participants (Jeong et al., 2016; Khandelwal et al., 2015).
Data Collection
We collected different types of implementation outcome data (i.e., feasibility and acceptability) from multiple sources as we sought to increase the credibility of the study through triangulation (Thurmond, 2001). Quantitative and qualitative data related to the MCI ENACT interventions were collected separately. During each advisory panel meeting, a researcher recorded observations and debriefing documentation.
Feasibility of recruitment: MCI ENACT prototype testing
Feasibility of recruitment was defined as the enrollment rate among patients who were referred to the study from the unrelated study of MCI and from the University geriatrics clinic. We were unable to track the number of patients who received the recruitment materials through community sources. We tracked recruitment using REDCap to determine if recruiting patients with MCI for the prototypes was feasible (Harris et al., 2009). For prototype testing, we also measured feasibility of intervention completion based on intervention retention, defined as participation in all planned sessions of the MCI ENACT intervention prototypes.
Acceptability (quantitative data): survey after MCI ENACT intervention prototypes
As a quantitative measure of acceptability of the MCI ENACT intervention, each participant with MCI and family care partner completed a seven-item evaluation of their experience in the intervention after the MCI ENACT prototype (Lum et al., 2016). All items were rated on a 5-point Likert scale from 1 (strongly disagree 1) to 5 (strongly agree).
Acceptability (qualitative data): interviews after MCI ENACT intervention prototypes
Each prototype was audio-recorded and professionally transcribed. Within 2 weeks of participation, both participants and their family care partners were contacted via telephone for semistructured interviews (n = 13 dyads). The interview explored acceptability of the MCI ENACT intervention prototypes. We aimed to interview the participants with MCI first and separately from the family care partner to limit potential bias, for a total of 20 interviews. The interview guide (available upon request) focused on understanding the dyads’ reasons for participating in the intervention, if it met their expectations, and any recommended changes to the intervention.
Data Analysis
Quantitative: All statistical analyses used SAS version 9.4 (SAS Institute, Cary, NC). Mean and standard deviations were used to summarize the intervention acceptability items. To aid our understanding of the quantitative data on intervention acceptability, we further explored acceptability by integrating with qualitative data from participant interviews (described below).
Qualitative: Qualitative data were collected from three advisory panel meetings, observations from four MCI ENACT interventions, and 20 MCI ENACT participant interviews. Employing multiple methods from many sources allowed for analysis of diverse insights, comparison between findings, and ultimately triangulation of the qualitative data, as well as integration with the quantitative data (Thurmond, 2001). For the qualitative analysis, we sorted our qualitative data based on the data source (i.e., focus groups, observations, and interviews; Farmer et al., 2006). Next, we used a rapid qualitative analysis process described by Hamilton (2013) to analyze the participant interviews to identify iterative refinements of the MCI ENACT intervention and perspectives on participant acceptability. The rapid assessment process allowed for a team-based qualitative inquiry using iterative data analysis to quickly develop a preliminary understanding from the participants’ perspective (Beebe, 2001). See Supplementary Material for details of the rapid qualitative analysis process including creation of summary templates and a data matrix integrating participant summaries and additional data from focus groups and prototype observations.
As triangulation can aid in reducing discrepancies that may occur from applying a singular method (Thurmond, 2001), the qualitative data analysis offered an opportunity to expand on the quantitative acceptability data. Themes describing participants’ experiences and acceptability of the intervention were compared with the quantitative acceptability results wherein we began with the quantitative acceptability items and sought to find similar or different perspectives within the participants’ qualitative results related to acceptability. This process was achieved by using an intuitive approach where researchers compared information between the data sources and intuitively related information that demonstrated a connection across the data sets (Farmer et al., 2006). A summary of findings and recommendations for the MCI ENACT intervention were then organized thematically.
Results
Advisory Panel Input on Intervention Design
As part of the human-centered design approach, stakeholders participated in the three advisory panel meetings and provided their feedback on the MCI ENACT intervention. Six dyads participated in three meetings over 9 months. Due to weather, the first meeting was rescheduled, and four dyads were able to attend; two dyads participated in interviews by phone to discuss the same questions. For the second and third advisory panel meetings, four of six dyads participated. For the six participants with MCI who participated in the advisory panel, the mean age was 76.2 years, half were male, all identified as White, were married, and reported postgraduate education and excellent self-rated health (Table 2).
Table 2.
Persons With MCI Who Participated in the Advisory Panel or MCI ENACT Intervention Prototypesa
| Advisory panel, n (%) (n = 6 individuals with MCI) | MCI ENACT prototypes, n (%) (n = 13 individuals with MCI) | |||||
|---|---|---|---|---|---|---|
| Participant characteristic | M (SD) | N | % | M (SD) | N | % |
| Age | 76.2 (6.6) | 78.7 (5.8) | ||||
| Gender | ||||||
| Female | 3 | 50% | 2 | 23% | ||
| Race/ethnic backgrounds | ||||||
| African American | 0 | 0% | 1 | 8% | ||
| White | 6 | 100% | 10 | 76% | ||
| Hispanic | 0 | 0% | 1 | 8% | ||
| Other | 0 | 0% | 1 | 8% | ||
| Education | ||||||
| High school graduate | 0 | 0% | 3 | 23% | ||
| Some college | 0 | 0% | 1 | 8% | ||
| College graduate | 1 | 17% | 2 | 15% | ||
| Any postgraduate | 5 | 83% | 7 | 54% | ||
| Relationship status | ||||||
| Married/with partner | 6 | 100% | 10 | 77% | ||
| Widowed | 0 | 0% | 1 | 8% | ||
| Divorced/separated | 0 | 0% | 2 | 15% | ||
| Self-rated health | ||||||
| Excellent | 5 | 83% | 8 | 62% | ||
| Good | 1 | 17% | 3 | 23% | ||
| Fair | 0 | 0% | 2 | 15% | ||
| Family care partners’ relationship to a person with MCI | ||||||
| Spouse | 4 | 67% | 9 | 69% | ||
| Partner | 1 | 17% | 1 | 8% | ||
| Child | 1 | 17% | 3 | 23% |
Note: MCI = mild cognitive impairment; ENACT = ENgaging in Advance Care planning Talks.
aDemographic information was not collected for family care partners. Data include one individual with MCI who participated in both the advisory panel and an MCI ENACT prototype.
Table 3 summarizes findings from each meeting, which are labeled 1.1, 1.2, or 1.3, including advisory panel topics, themes, and recommendations for the MCI ENACT intervention. Overall, the advisory panel found the MCI ENACT intervention acceptable for individuals with MCI.
Table 3.
Advisory Panel Topics, Themes, and Recommendations for the MCI ENACT Intervention Prototypes
| Advisory panel meeting component | Meeting 1.1 | Meeting 1.2 | Meeting 1.3 |
|---|---|---|---|
| Planned topics | Discussion of MCI Introduction to ACP Discussion of experience with ACP |
Overview of the format of the ENACT Group Visits intervention Review of MCI ENACT prototypes Advance directive discussion |
Review of MCI ENACT prototypes Discussion of MCI ENACT intervention purpose and format |
| Themes from the discussion | Diagnosis of MCI Experiences with ACP |
Communication MCI ENACT intervention recommendations Difficulty of disclosing MCI diagnosis with loved ones Limitations MCI might impose on planning and engaging in ACP because of difficulty in prognostication Importance of ACP before further cognitive decline |
Acceptability of MCI ENACT interventions |
| Recommendations for MCI ENACT interventions | Keep group size to up to six individuals (three dyads) Having care partner present helpful to a person with MCI One or two MCI ENACT interventions are acceptable Recommended a discussion of MCI diagnosis, medications, possible treatments for MCI, long-term care living arrangements, financial planning |
Limit the number of resources provided to participants | Consensus that MCI ENACT interventions are appropriate for individuals with MCI MCI ENACT interventions are better for ACP than a one-on-one visit with a provider Because having multiple advisory panel meetings improved comfort with one another, they anticipated having multiple sessions for the MCI ENACT intervention would have the same effect Felt the intervention would allow more time to discuss ACP than possible with health care providers. |
Note: MCI = mild cognitive impairment; ENACT = ENgaging in Advance Care planning Talks; ACP = advance care planning.
Feasibility of MCI ENACT Intervention Prototype Tests
There were 36 referrals to the MCI ENACT interventions from the unrelated longitudinal study of MCI of which 12 dyads enrolled (33% recruitment rate). The University geriatrics clinic referred 63 patients of which nine dyads enrolled (14% recruitment rate). There were 14 self-referrals from the Alzheimer’s Association; the number of people who viewed recruitment materials is unknown. None of the self-referrals were enrolled (0% recruitment rate): three were ineligible based on cognitive status being too impaired, five declined after learning more about the goals of the study, two had already completed advance directives, and four were unable to be reached. Supplementary Figure shows a complete description of recruitment rates.
Thirteen dyads were enrolled in the four MCI ENACT interventions. For participants with MCI (n = 13), the mean age was 78.7 years, 11 (77%) were male, including one (8%) African American, one (8%) Hispanic, and one (8%) identified as being from other racial/ethnic backgrounds (Table 2). The first prototype had four dyads and the remaining three interventions had three dyads. One dyad from the longitudinal cohort participated in one of the MCI ENACT intervention prototypes.
Overall feasibility of recruitment of participants with MCI was judged as low because recruitment rates ranged from 0% to 33%, was based on referral source, and recruitment took a lengthy amount of time from the researchers’ perspectives. Feasibility of intervention completion was based on 12 of 13 dyads completing all scheduled MCI ENACT intervention sessions (92% retention rate). Of four dyads recruited for the third MCI ENACT intervention, one dyad did not attend the first session of the ENACT intervention.
Acceptability of MCI ENACT Interventions
On the seven-item quantitative evaluation of the MCI ENACT intervention, participants and family care partners from the four prototypes provided overall high ratings of acceptability and satisfaction with the intervention (Table 4). Each of the items had a mean score above 4 on a 5-point Likert scale (agree to strongly agree), suggesting the potential for participant acceptability bias. From the MCI ENACT intervention interviews, three themes emerged related to acceptability of the MCI ENACT intervention and participants’ backgrounds and values that influence their perspectives on the intervention, including (a) feedback on acceptability of the intervention, (b) previous experiences with ACP, and (c) reasons for participating in the intervention. Within the first theme, we present quantitative and qualitative findings alongside each other.
Table 4.
MCI ENACT Participants’ Evaluation of Acceptability
| Acceptability item | Prototype 2.1 (n = 8) | Prototype 2.2 (n = 6) | Prototype 2.3 (n = 6) | Prototype 2.4 (n = 6) | Total mean (SD) |
|---|---|---|---|---|---|
| 1. The group visit setting is better for talking about advance care planning than a normal visit with my doctor. | 4.13 | 4.67 | 4.67 | 4.5 | 4.46 (0.86) |
| 2. The group discussion gave me useful information. | 4.75 | 4.83 | 4.83 | 4.83 | 4.81 (0.40) |
| 3. I felt comfortable talking about advance care planning in the group setting. | 4.75 | 4.83 | 4.67 | 4.67 | 4.73 (0.53) |
| 4. Talking with other people about advance care planning was helpful. | 4.86 | 4.83 | 5.00 | 4.5 | 4.62 (1.02) |
| 5. I feel the group visit addressed my specific questions. | 4.13 | 4.67 | 4.67 | 4.17 | 4.38 (0.70) |
| 6. I feel able to discuss advance care planning with my regular health care provider. | 4.63 | 4.67 | 4.33 | 4.33 | 4.50 (0.58) |
| 7. I would recommend these group visit sessions to a friend. | 4.71 | 5.00 | 4.83 | 4.83 | 4.65 (1.06) |
Notes: MCI = mild cognitive impairment; ENACT = ENgaging in Advance Care planning Talks. Persons with MCI and family care partners independently, rather than as a dyad, rated measures of acceptability of the MCI ENACT prototype after participating on a scale from 1 (strongly disagree) to 5 (strongly agree).
Theme 1—Feedback on acceptability of the intervention
As a global perspective on acceptability, participants noted an overall positive rating of whether they would recommend the group visits to a friend (Item 7, 4.65 out of 5). In seeking to further understand participants’ perspectives on acceptability of the MCI intervention, there were five subthemes from the qualitative interviews: More discussion of MCI, How to have conversations about ACP, Number of sessions, Group format, and Helpful to hear others’ stories.
Item 5 on the acceptability evaluation (I feel the group visit addressed my specific questions) was the lowest-rated item (4.38 out of 5). The subtheme, More discussion of MCI, related to how participants felt about the lack of discussion on MCI throughout the ENACT intervention and was a key issue affecting intervention acceptability. A few participants mentioned they would have liked some discussion on the diagnosis in addition to learning on ACP. Care partner, prototype 2.4, recommended MCI education as a supplement to the ACP information provided:
I would’ve preferred [MCI education] not necessarily instead of, but in addition to. Because … I found [ACP education] very important and good and positive and thought-provoking and motivating. It was just a little different from I had thought it might be.
Item 2 on the acceptability evaluation (The group discussion gave me useful information), in contrast, had the highest overall rating (4.81 out of 5). The subtheme, How to have conversations about ACP, offers further insight regarding what information was perceived as useful. Participants shared positive feedback surrounding the discussion of how to start and have a conversation about ACP. Participants especially liked the videos demonstrating how to initiate a conversation about ACP. Care partner, prototype 2.1, remarked:
What I think is gonna be very important is up until that meeting that we had with you, my main goal was to make everybody feel good. Well, you can’t make everybody feel good when you’re talking about end-of-life decisions. It isn’t going to be a wonderful, fabulous, everybody’s gonna have a happy time …. The key to my success … is going to be to concentrate on the fact that I am trying to convey to them how I feel. I’ve never done that. I don’t do that.
Item 3 allowed participants to rate their comfort talking about ACP in the group setting and had the second highest overall rating (4.73 out of 5). Similarly, Item 6 had relatively positive ratings (4.5 out of 5) of being able to discuss ACP with their health care provider. From the qualitative results, the subtheme, Number of sessions, reflected participant comfort in the group setting and specifically provided feedback on the frequency/quantity of the sessions. We compared a group visit consisting of one session (prototype 2.1) with others that featured two sessions held 2–4 weeks apart (Table 1). Interviewees who participated in the two-session prototypes described they felt more comfortable in the second session and would recommend designing future MCI ENACT interventions to always have two sessions. Three or more sessions were described as excessive. A person with MCI, prototype 2.4, summarizes the general preference for two sessions:
I thought twice was just perfect. I think more than that would be overkill, but I think to just get together once—I felt that we were able to talk with each other much more comfortably in the second session than in the first session.
The subtheme, Group format, offered additional insight on Item 1 of the acceptability evaluation, “The group visit setting is better for talking about advance care planning than a normal visit with my doctor,” which had a mean rating of 4.46 out of 5. From the interviews, we were able to gain perspective on why the group setting was viewed as more favorable for discussion of ACP, while also reflecting the need for an appropriate group size. Specifically, interviewees explained that the group format was beneficial because it allowed them to hear from others on a difficult subject. They also described that a group size of about four dyads was ideal. Smaller group size was described to benefit intimate conversations, create a comfortable atmosphere conducive to participation, and allow for greater discussion. Nearly all interviewees indicated that having a family care partner present was helpful rather than only including the individual. For example, person with MCI, prototype 2.3, remarked on the group format:
I thought it was very helpful, because you can’t think of everything on your own. You need input from other people. And that’s what the group helped me with, was thinking of those things that we need to look at and take care of.
Item 4 from the acceptability evaluation, “Talking with other people about advance care planning was helpful,” was rated at 4.62 out of 5. The subtheme, Helpful to hear others’ stories, embodies feedback related to additional benefits from the group setting and sheds light on what attributes of the group setting may have contributed to participants’ greater comfort levels. When assessed in conjunction with the subtheme, Group format, we find that key functions of the group setting that participants may find most helpful are the interactive discussions with other peers. Person with MCI, prototype 2.2, explained the group members helped advance their understanding of ACP:
I learned some things and there were—the other people in the group were intelligent people and their opinions were useful either corroborating or some new ideas.
Theme 2—Previous experiences with ACP
The theme of previous experiences with ACP is represented by three subthemes: Completed ACP documents, Started but not completing ACP, and Conversations of goals and preferences for future medical care. The subtheme, Completed ACP documents, captured participant responses indicating they had completed advance directives or had already established burial and funeral arrangements. Care partner, prototype 2.1, confidently described:
Oh, we have everything. We have power of attorney. We have executors. We have living will; we had this done before we met with you.
The subtheme, Started but not completing ACP, reflects participant responses describing previously initiating ACP with their lawyers, as an example, but not feeling they had completed the process. Although several participants described previously meeting with a lawyer related to ACP, they expressed that these meetings had occurred in the distant past and as such, still felt they desired to engage further in ACP. Care partner, prototype 2.4, described:
Several years ago, we had discussions and met with a lawyer and drew up some papers including an advanced directive, but we haven’t really talked much about it since … in preparation for this class, trying to find the papers, neither of us can find any advanced directive paperwork.
The subtheme, Conversations of goals and preferences for future medical care, represents descriptions of either having or not having prior conversations about end-of-life wishes with family. Person with MCI, prototype 2.1, explained:
We talked about several things, and I think one of the things that I remember our talking about was “do I want to be buried,” or “do I want to have a cremation?”
Theme 3—Reason for participation
Interviewees stated a variety of reasons for participating in the MCI ENACT intervention and five subthemes emerged: Altruism, Clarifications, Reassurance, Previous experience with end of life, and Wanted to learn more about MCI. The subtheme, Altruism, captured participant discussion on their involvement with the intervention was based on a desire to use their experiences and knowledge to benefit others. Person with MCI, prototype 2.1, ruminated:
I’m also part of another study, and there’s nothing I can do to stop this disease, but I can help others by getting more information out there, whether it be medications or whether it be ways of living with the disease and doing what’s helpful.
The subtheme, Clarifications, represented participants’ interest to gain additional information on ACP. Care partner, prototype 2.1, cited motivation to participate stemmed from an interest to advance their learning through information gleaned from others in the group setting:
I thought it would benefit us both to have someone bring it [ACP] up and give us some depth of information … and in a group setting, it makes it easier to kind of assimilate exactly what it is.
The subtheme, Reassurance, consisted of participant responses desiring to gain verification of ACP processes in an effort to bolster confidence in their understanding. Care partner, prototype 2.4, described viewing the MCI ENACT intervention as an opportunity to ensure completeness in the family’s ACP endeavors:
We wanted to make sure we had everything in place and make sure … what we had done is correct.
The subtheme, Previous experience with end of life, embodies participants’ feedback indicating their motivating factor for participation was their previous experience with death and dying and wanting to feel more prepared for their own end-of-life planning. Person with MCI, prototype 2.3, shared a personal experience that alerted a need for ACP:
It wasn’t interest that did it. It was the fact that my 41-year-old son-in-law died suddenly .… That we need to get a will done, and get things put in place, because I’m getting up there in age...Well, he died without a will, and he has four kids … he wasn’t ready to go … he did like everybody does … figured he was too young, and it wouldn’t happen. And it did.
Similarly, the subtheme, Wanted to learn more about MCI, encapsulated responses hoping the intervention would offer more information about the diagnosis of MCI and what to expect as the disease progresses. Person with MCI, prototype 2.4, explained:
I had just been recently dealing with issues around memory and memory loss and having gone through that whole thing with my dad, because he died of Alzheimer’s, we just decided, it’s good for us to be doing some thinking about this. It seemed like a real great opportunity to get some professional assistance in terms of things that we might not think about ourselves.
As also noted in the theme related to feedback on the intervention, the MCI ENACT intervention did not explicitly discuss MCI during the group visits. While the research team chose not to include a discussion of MCI because it would be too significant of an adaptation by the research team, participant feedback clearly noted a desire for attention to MCI as part of the intervention.
In summary, the quantitative and qualitative findings provided an opportunity for greater understanding related to the feasibility and acceptability of a newly designed MCI ENACT intervention. The advisory panel provided iterative input on ACP materials and formatting for the intervention prototypes. Despite feasibility challenges related to participant recruitment, the feasibility of intervention participation (retention for the second group visit, when appropriate) was high. By analyzing the quantitative and qualitative data, we identified specific feedback from participants that influenced intervention acceptability.
Discussion
This research used human-centered design and rapid-cycle prototyping to refine the ENACT Group Visits intervention for individuals with MCI and their family care partners. This research sought to determine if an MCI ENACT intervention is feasible and acceptable. Importantly, consistent with the importance of using a participatory research approach when considering interventions for older adults, the advisory panel reported ACP as a priority for individuals with MCI and described the need for ACP in a group setting (Corrado et al., 2020). Similarly, MCI ENACT intervention participants also rated the intervention as acceptable by reporting they found the group visit setting helpful to hear others’ perspectives on ACP and beneficial to initiating conversations about end-of-life planning. Most felt the format and size of the MCI ENACT intervention were acceptable, the videos and resources were helpful, and would recommend the intervention to others. These findings align with ACP group visits among older adults without MCI (Lum et al., 2016). However, feedback from both the advisory panel and MCI ENACT intervention participants indicated that they would like more candid conversations about the diagnosis of MCI and how it relates to ACP. This feedback specifically addresses the known need of persons with MCI and family care partners to reduce ambiguity in the context of a diagnosis of MCI (Gomersall et al., 2015). While a discussion of MCI was not the goal of the MCI ENACT intervention, using the group setting may be helpful for others to share how they are coping with their diagnosis of MCI and future care planning.
The feasibility of the MCI ENACT intervention was limited by recruitment of participants to the prototypes. Most participants recruited for this study came from the longitudinal study of MCI conducted at the University of Colorado Alzheimer’s and Cognition Center, and therefore, the results may not be generalizable to wider community settings. These individuals were already involved in research and thus, we were able to recruit those individuals more easily. Our recruitment rate from the University geriatric clinic (14%) resembled the original ENACT Group Visits intervention research (13%; Lum et al., 2020), which may confirm recruitment challenges for group visit intervention studies in clinical settings, or recruitment for research more broadly.
The qualitative interviews with dyads who participated in the MCI ENACT interventions indicated that many participants were unclear what the MCI ENACT interventions would entail and what to expect. To help with future feasibility of recruitment, we specifically asked for recommendations on how to make the description of the intervention clearer. However, interviewees were unable to provide any suggestions. In addition, we were unable to successfully recruit any individuals who were self-referrals. A few referrals from the Alzheimer’s Association had more advanced stages of dementia that prevented their ability to self-consent. Among those who declined after learning about the study goals, a few described they were hoping to participate in research that might help find a cure for MCI and dementia. Given we targeted the recruitment of individuals in the early stages of dementia, this population may be particularly difficult to recruit as they may be adjusting to the diagnosis and perhaps are not ready to talk about ACP or end-of-life care.
Limitations
There were several limitations to this research. Generalizability is limited because participants in the advisory panel and the four MCI ENACT intervention prototypes were mostly White, highly educated, and motivated to participate in research. The relative lack of diversity among study participants likely results in missed opportunities to understand important feedback and perspectives on the design of an MCI ENACT intervention that can be acceptable to individuals from diverse cultural and socioeconomic backgrounds. Generalizability is also limited due to the potential for reporting bias by participants (i.e., tending toward reporting high level of acceptability) and by researchers (i.e., lack of independence between intervention team and evaluation team) which may have influenced the findings. For example, the qualitative themes and quantitative acceptability ratings tend to focus on the benefits of the intervention. Second, while we aimed to recruit individuals with MCI due to possible Alzheimer’s disease, to maximize feasibility, we did not require comprehensive neuropsychological testing or neuroimaging; thus, it is possible that some participants had nonprogressive etiologies. Lastly, participants and family care partners were asked in surveys what their future ACP goals were. However, this research did not follow up to see if those goals were met. Future research should examine whether ACP goal-setting resulted in ACP actions following the MCI ENACT intervention (e.g., 3–6 months after the ENACT intervention), and the evaluation should be conducted independently of the intervention implementation.
Implications
Use of a pragmatic research approach with rapid prototyping and multilevel stakeholder engagement allowed testing of different ACP resources and group visit formats aimed at helping individuals with MCI and their family care partners discuss ACP. Overall, individuals with MCI and their family care partners found the MCI ENACT intervention helpful and would recommend others to participate in future group visits. Specifically, engaging in these conversations before individuals with MCI lose capacity may increase the likelihood that an aging, cognitively impaired population can participate in important health care planning. However, despite stakeholder acceptability, a refined MCI ENACT intervention to specifically reach dyads affected by MCI may have limited feasibility for implementation into usual care in a clinical setting, as recruitment of individuals with MCI proved difficult for this study and would likely face significant barriers to implementation in real-world practice.
Supplementary Material
Acknowledgments
This project would not have been possible without the help of Sue Felton, Sarah Jordan, and Adreanne Brungardt who helped with recruitment, facilitation of the advisory panel meetings and the ENACT group visits, and their assistance with the qualitative interviews. We would also like to thank Joanna Dukes for getting this work approved through the Colorado Multiple Institutional Review Board and for setting up the data collection tools within REDCap, and Jessica Cassidy for critical review of the manuscript.
Contributor Information
Andrea E Daddato, Division of Geriatric Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA.
Elizabeth W Staton, Department of Family Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA.
Brianne M Bettcher, Alzheimer’s and Cognition Center, University of Colorado School of Medicine, Aurora, Colorado, USA.
Prajakta Shanbhag, Division of Geriatric Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA.
Hillary D Lum, Division of Geriatric Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA; Geriatric Research Education and Clinical Center, VA Eastern Colorado Health Care System, Aurora, Colorado, USA.
Funding
This work was supported by the National Institute on Aging of the National Institutes of Health (K76AG054782) and the Colorado Clinical & Translational Sciences Institute (CCTSI) with the Development and Informatics Service Center (DISC) grant support (NIH/NCRR Colorado CTSI Grant Number UL1 RR025780) for use of REDCap-based data management. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The sponsors had no role in the design, methods, or preparation of the manuscript. The views in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Conflict of Interest
None declared.
References
- Beebe, J. (2001). Rapid assessment process: An introduction: AltaMira Press. [Google Scholar]
- Bruffee, K. A. (1993). Collaborative learning: Higher education, interdependence, and the authority of knowledge. Johns Hopkins University Press. [Google Scholar]
- Corrado, A. M., Benjamin-Thomas, T. E., McGrath, C., Hand, C., & Laliberte Rudman, D. (2020). Participatory action research with older adults: A critical interpretive synthesis. The Gerontologist, 60, e413–e427. doi: 10.1093/geront/gnz080 [DOI] [PubMed] [Google Scholar]
- deLima Thomas, J., Sanchez-Reilly, S., Bernacki, R., O’Neill, L., Morrison, L. J., Kapo, J., Periyakoil, V. S., & Carey, E. C. (2018). Advance care planning in cognitively impaired older adults. Journal of the American Geriatrics Society, 66, 1469–1474. doi: 10.1111/jgs.15471 [DOI] [PubMed] [Google Scholar]
- Dixon, J., Karagiannidou, M., & Knapp, M. (2018). The effectiveness of advance care planning in improving end-of-life outcomes for people with dementia and their carers: A systematic review and critical discussion. Journal of Pain and Symptom Management, 55, 132–150.e1. doi: 10.1016/j.jpainsymman.2017.04.009 [DOI] [PubMed] [Google Scholar]
- Farmer, T., Robinson, K., Elliott, S. J., & Eyles, J. (2006). Developing and implementing a triangulation protocol for qualitative health research. Qualitative Health Research, 16, 377–394. doi: 10.1177/1049732305285708 [DOI] [PubMed] [Google Scholar]
- Garand, L., Dew, M. A., Lingler, J. H., & DeKosky, S. T. (2011). Incidence and predictors of advance care planning among persons with cognitive impairment. The American Journal of Geriatric Psychiatry, 19, 712–720. doi: 10.1097/JGP.0b013e3181faebef [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gomersall, T., Astell, A., Nygård, L., Sixsmith, A., Mihailidis, A., & Hwang, A. (2015). Living with ambiguity: A metasynthesis of qualitative research on mild cognitive impairment. The Gerontologist, 55, 892–912. doi: 10.1093/geront/gnv067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamilton, A. (2013). Qualitative methods in rapid turn-around health services research. Department of Veteran Affairs. http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780 [Google Scholar]
- Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42, 377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ideo.org. (n.d.). Design Kit. https://www.designkit.org/
- Jeong, J. H., Na, H. R., Choi, S. H., Kim, J., Na, D. L., Seo, S. W., Chin, J., Park, S. A., Kim, E. J., Han, H. J., Han, S. H., Yoon, S. J., Lee, J. H., Park, K. W., Moon, S. Y., Park, M. H., Choi, M. S., Han, I. W., Lee, J. H., … Kim, J. Y. (2016). Group- and home-based cognitive intervention for patients with mild cognitive impairment: A randomized controlled trial. Psychotherapy and Psychosomatics, 85(4), 198–207. doi: 10.1159/000442261 [DOI] [PubMed] [Google Scholar]
- Khandelwal, C., Prentice, A., Fisher, J., Parrott, R., & Sloane, P. D. (2015). Treating dementia with shared group visits. Family Practice Management, 22, 16–21. Available on website: https://www.aafp.org/fpm/2015/0500/fpm20150500p16.pdf [PubMed] [Google Scholar]
- Lingler, J. H., Terhorst, L., Schulz, R., Gentry, A., & Lopez, O. (2016). Dyadic analysis of illness perceptions among persons with mild cognitive impairment and their family members. The Gerontologist, 56, 886–895. doi: 10.1093/geront/gnv029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lum, H. D., Dukes, J., Daddato, A. E., Juarez-Colunga, E., Shanbhag, P., Kutner, J. S., Levy, C. R., & Sudore, R. L. (2020). Effectiveness of advance care planning group visits among older adults in primary care. Journal of the American Geriatrics Society, 68, 2382–2389. doi: 10.1111/jgs.16694 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lum, H. D., Jones, J., Matlock, D. D., Glasgow, R. E., Lobo, I., Levy, C. R., Schwartz, R. S., Sudore, R. L., & Kutner, J. S. (2016). Advance care planning meets group medical visits: The feasibility of promoting conversations. Annals of Family Medicine, 14, 125–132. doi: 10.1370/afm.1906 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lund, S., Richardson, A., & May, C. (2015). Barriers to advance care planning at the end of life: An explanatory systematic review of implementation studies. PLoS One, 10, e0116629. doi: 10.1371/journal.pone.0116629 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lyon, A. R., & Koerner, K. (2016). User-centered design for psychosocial intervention development and implementation. Clinical Psychology, 23, 180–200. doi: 10.1111/cpsp.12154 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McMahan, R. D., Tellez, I., & Sudore, R. L. (2021). Deconstructing the complexities of advance care planning outcomes: What do we know and where do we go? A scoping review. Journal of the American Geriatrics Society, 69, 234–244. doi: 10.1111/jgs.16801 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mitchell, S. L., Teno, J. M., Kiely, D. K., Shaffer, M. L., Jones, R. N., Prigerson, H. G., Volicer, L., Givens, J. L., & Hamel, M. B. (2009). The clinical course of advanced dementia. The New England Journal of Medicine, 361, 1529–1538. doi: 10.1056/NEJMoa0902234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pfeiffer, E. (1975). A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. Journal of the American Geriatrics Society, 23, 433–441. doi: 10.1111/j.1532-5415.1975.tb00927.x [DOI] [PubMed] [Google Scholar]
- Prusaczyk, B., Cherney, S. M., Carpenter, C. R., & DuBois, J. M. (2017). Informed consent to research with cognitively impaired adults: Transdisciplinary challenges and opportunities. Clinical Gerontologist, 40, 63–73. doi: 10.1080/07317115.2016.1201714 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steinhauser, K. E., Christakis, N. A., Clipp, E. C., McNeilly, M., McIntyre, L., & Tulsky, J. A. (2000). Factors considered important at the end of life by patients, family, physicians, and other care providers. Journal of the American Medical Association, 284, 2476–2482. doi: 10.1001/jama.284.19.2476 [DOI] [PubMed] [Google Scholar]
- Sudore, R. L., Landefeld, C. S., Williams, B. A., Barnes, D. E., Lindquist, K., & Schillinger, D. (2006). Use of a modified informed consent process among vulnerable patients: A descriptive study. Journal of General Internal Medicine, 21, 867–873. doi: 10.1111/j.1525-1497.2006.00535.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sudore, R. L., Lum, H. D., You, J. J., Hanson, L. C., Meier, D. E., Pantilat, S. Z., Matlock, D. D., Rietjens, J. A. C., Korfage, I. J., Ritchie, C. S., Kutner, J. S., Teno, J. M., Thomas, J., McMahan, R. D., & Heyland, D. K. (2017). Defining advance care planning for adults: A consensus definition from a multidisciplinary Delphi panel. Journal of Pain and Symptom Management, 53, 821–832.e1. doi: 10.1016/j.jpainsymman.2016.12.331 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sudore, R. L., Stewart, A. L., Knight, S. J., McMahan, R. D., Feuz, M., Miao, Y., & Barnes, D. E. (2013). Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors. PLoS One, 8, e72465. doi: 10.1371/journal.pone.0072465 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thurmond, V. A. (2001). The point of triangulation. Journal of Nursing Scholarship, 33, 253–258. doi: 10.1111/j.1547-5069.2001.00253.x [DOI] [PubMed] [Google Scholar]
- Wadsworth, K. H., Archibald, T. G., Payne, A. E., Cleary, A. K., Haney, B. L., & Hoverman, A. S. (2019). Shared medical appointments and patient-centered experience: A mixed-methods systematic review. BMC Family Practice, 20, 97. doi: 10.1186/s12875-019-0972-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wright, A. A., Zhang, B., Ray, A., Mack, J. W., Trice, E., Balboni, T., Mitchell, S. L., Jackson, V. A., Block, S. D., Maciejewski, P. K., & Prigerson, H. G. (2008). Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. Journal of the American Medical Association, 300, 1665–1673. doi: 10.1001/jama.300.14.1665 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.