Abstract
Background
There is an increasing number of interactive web-based advance care planning (ACP) support tools, which are web-based aids in any format encouraging reflection, communication, and processing of publicly available information, most of which cannot be found in the peer-reviewed literature.
Objective
This study aims to conduct a systematic review of web-based ACP support tools to describe the characteristics, readability, and quality of content and investigate whether and how they are evaluated.
Methods
We systematically searched the web-based gray literature databases OpenGrey, ClinicalTrials.gov, ProQuest, British Library, Grey Literature in the Netherlands, and Health Services Research Projects in Progress, as well as Google and app stores, and consulted experts using the following eligibility criteria: web-based, designed for the general population, accessible to everyone, interactive (encouraging reflection, communication, and processing of information), and in English or Dutch. The quality of content was evaluated using the Quality Evaluation Scoring Tool (score 0-28—a higher score indicates better quality). To synthesize the characteristics of the ACP tools, readability and quality of content, and whether and how they were evaluated, we used 4 data extraction tables.
Results
A total of 30 tools met the eligibility criteria, including 15 (50%) websites, 10 (33%) web-based portals, 3 (10%) apps, and 2 (7%) with a combination of formats. Of the 30 tools, 24 (80%) mentioned a clear aim, including 7 (23%) that supported reflection or communication, 8 (27%) that supported people in making decisions, 7 (23%) that provided support to document decisions, and 2 (7%) that aimed to achieve all these aims. Of the 30 tools, 7 (23%) provided information on the development, all of which were developed in collaboration with health care professionals, and 3 (10%) with end users. Quality scores ranged between 11 and 28, with most of the lower-scoring tools not referring to information sources.
Conclusions
A variety of ACP support tools are available on the web, varying in the quality of content. In the future, users should be involved in the development process of ACP support tools, and the content should be substantiated by scientific evidence.
Trial Registration
PROSPERO CRD42020184112; https://tinyurl.com/mruf8b43
Keywords: advance care planning, systematic review, web-based tools, health communication, quality of online content
Introduction
Via a European consensus process, advance care planning (ACP) has been defined as a process that enables individuals to define goals and preferences for future medical care, discuss these preferences with family and health care providers, and record these preferences and choices [1,2]. In recent decades, the concept of ACP has changed considerably, shifting from a clinician-led and documentation-focused process that emphasizes the need for advance directives to a broader concept of ongoing communication regarding various aspects of future care and treatment planning [3,4]. In the recent public health literature, the concept has been broadened further by emphasizing the opportunities that ACP conversations offer for normalizing and reshaping how we think, talk, and make decisions about the last chapters of our lives [5,6].
To support people in having such conversations, a wide variety of ACP support tools have been developed in several formats such as print or websites. They exist in different kinds of modalities (guides, card games, and videos) and for different target groups: people with specific diseases and their families, family caregivers, or the general public [7,8]. With the growing use of the internet and international efforts to promote ACP [9], more web-based ACP support tools are becoming publicly available [10]. An advantage of these web-based tools is that they are easily accessible to a large audience, are often interactive, and can thus be tailored to the needs of the user. Interactive elements include, for example, questions or exercises to encourage reflection and communication and process the information provided [10-12].
A recent review of the published peer-reviewed studies assessed the feasibility and effectiveness of interactive web-based ACP support tools for adult patients, relatives, and healthy individuals and found that users considered the tools easy to use and not burdensome. It also demonstrated that they could improve a user’s knowledge of ACP, ACP communication with relatives and health care professionals, and ACP documentation [8]. However, this review was not able to include all ACP support tools available to the general public, as many exist only on the web and have not been published in academic journals [7,8]. Reviewing these web-based tools is important as the quality of web-based content can vary widely or be based on personal opinions and experiences rather than on scientific evidence [13-16]. This can be particularly problematic with regard to ACP, as the content may be biased in favor of or against certain medical interventions [15], whereas the primary purpose of ACP should be to promote choices based on individual values and preferences [1,2].
Currently, there is no comprehensive overview of interactive ACP tools available on the internet. Therefore, this systematic review aims to answer the following research questions:
What are the characteristics and functionalities of interactive web-based ACP support tools?
How is ACP addressed in these tools?
What is the readability and quality of their content, and have they been evaluated in a study, and if so, what is their level of evidence?
Methods
Review Design and Protocol Registration
We conducted a systematic review of web-based interactive ACP support tools (hereafter called tools) following the 4 search strategies for web-based gray literature by Godin et al [17]. Search, selection, and data synthesis were performed between September 2020 and January 2021. The protocol for this systematic review was registered with PROSPERO (International Prospective Register of Systematic Reviews; CRD42020184112).
Eligibility Criteria
We searched for tools that met the following inclusion criteria:
Designed to support the general population; that is, people with or without serious illnesses and their families
Available on the internet
Accessible to whoever visits the tool and can be used by anyone
Interactive; that is, encourage the user to reflect, communicate, formulate decisions, or document wishes [10-12]
In English or Dutch
Tools that exclusively aimed to support health care professionals in the ACP process were excluded.
Search Strategies
We systematically searched for tools using the four search strategies recommended by Godin et al [17]: (1) web-based gray literature databases, (2) search engines, (3) app stores, and (4) expert consultation. The first three search strategies were conducted separately by 2 researchers (CD and FM), who both used the same search combinations (Multimedia Appendices 1-3) on different computers with clean (deleted cookies and history) browsers without being logged into a Google account. Because, as with peer-reviewed databases, every search database (gray literature databases, search engines, and app stores) has its own search functionalities and filters, we adapted the search terms to fit into the search fields of each database.
Furthermore, as search engines have their own algorithms for showing relevant results, we used several different search combinations (ie, combining the search terms, their permutations, and trending keywords) in search strategy 2 to minimize the risk of omitting relevant sources (Multimedia Appendix 2). As a search engine can give an overwhelming amount of hits, we screened the first 15 pages (first 150 hits) of each search combination.
We screened the available content of the results, such as executive summaries, the webpage about, or the explanation of the tool—when available—until we could answer the following question: is this tool potentially a web-based ACP support tool for the general population?. If the answer was yes, we included the tool for full screening and transferred the results to a Microsoft Excel file (with the name of the tool or website if there was no specific name for the tool).
For the first search strategy (gray literature databases), we used the following databases: OpenGrey, ClinicalTrials.gov, ProQuest, British Library, Grey Literature in the Netherlands, and Health Services Research Projects in Progress. For the second search strategy, we used the Google search engine, and for the search in app stores, we used Google Play Store and Apple App Store.
The fourth search strategy entailed consultation with experts on ACP. We identified experts via frequently listed and cited authors of the relevant literature, known stakeholders, and suggestions from other key informants. We consulted these experts by email and, to achieve saturation, asked them whether they knew any other tools that we had not found using the first three search strategies. The responses were added to the Microsoft Excel file and saved for the final selection.
Selection of Web-Based ACP Support Tools
For the final selection, the 2 researchers (CD and FM) separately screened all available content of each potentially relevant tool against the eligibility criteria. In cases of disagreement about whether to include a tool, a third reviewer (TS) screened it and made the final decision on including it. We used Archive.is to archive the home page or the first page of the tools.
Evaluation of the Readability and Quality of the Content of the Included Tools
Readability was evaluated using web-based readability analysis tools. These readability tools calculate several readability scores based on the characteristics of the text, such as the number of syllables per word and number of words per sentence. As the web-based readability analysis tools were exclusively for one language, we used two tools: one for the English tools [18] and one for the Dutch tools [19]. To determine readability, we used the Common European Framework of Reference for Languages (CEFR) levels [20,21]. The CEFR levels can be used to determine both English and Dutch readability and are calculated by the algorithm of web-based readability analysis tools and look at the number of words per sentence, number of pronouns and prepositions in a sentence, and number of simple words [22]. CEFR comprises six reading levels (A1, A2, B1, B2, C1, and C2), with A1 as the easiest level and C2 as the most difficult. The recommended readability standard for CEFR is B1 [22].
To evaluate the quality of the content of the tools, we used the validated Quality Evaluation Scoring Tool (QUEST) [23]. This quality assessment tool can be used to assess web-based health content by evaluating 7 items, each assigned with a weighted score. Six items have a possible score between 0 to 1 or 0 to 2 and weight between 1 and 3: authorship (score 0-2×1), conflicts of interest (score 0-2×3), complementarity (whether they support the patient-physician relationship; score 0-1×1), currency (if the content is frequently updated; score 0-2×1) and the tone of the content (whether the content was fully supported using strong vocabulary such as cure, guarantee, and easy; mainly supported where the authors mainly support their claims but with more cautious vocabulary; or balanced/cautious support with statements of limitations or contrasting findings; score 0-2×3). The seventh item is attribution (whether and what kind of sources are used to create the content) and is measured in two steps: first, identifying the presence of references to scientific studies (score 0-3×3) and, second, when scoring >1, identifying the type of studies referred to in vitro, observational studies, or meta-analyses or clinical trials (score 0-2×1) [23,24]. Each tool was evaluated for each of the 7 items. The scores for each item were summed to create a total quality score ranging between 0 and 28, with higher scores indicating a better quality of the content in the tool.
Readability was evaluated by 2 researchers (CD and FM) who copy-pasted the text of a full webpage into the text fields of the web-based readability score calculator tools. The same 2 researchers assessed the content of the tools to determine the QUEST score for each tool. Any disagreements on readability and QUEST scores were discussed to reach a consensus. If no consensus was found, a third researcher (TS) made the final decision.
Evaluation of the Level of Evidence of the Included Tools
To assess whether the included tools had been evaluated as part of a research study, we (1) screened each one for any information on evaluation; (2) we searched the gray literature databases OpenGrey, ClinicalTrials.gov, ProQuest, British Library, Grey Literature in the Netherlands, and Health Services Research Projects in Progress; and (3) we searched for publications of primary peer-reviewed studies in PubMed, Web of Science, PsycINFO, and CINAHL. Search terms in the database included the names of the tools, websites, or persons or organizations involved in the development of the tool.
We screened the available content (for scientific article abstracts or, when needed, full texts) to check whether the result was a primary study on one of the included tools. If it was, where possible, we determined the level of evidence using the Hierarchy of Evidence from the National Health and Medical Research Council [25,26] to determine how the tools were evaluated.
Two researchers (CD and FM) independently assessed the full texts of the peer-reviewed studies to determine the level of evidence and assess what was evaluated (usability and effectiveness). Any disagreements were discussed to reach a consensus. If no consensus was reached, a third researcher (TS) made the final decision.
Data Synthesis and Outcomes of Interest
To answer the research questions, we developed 4 data extraction tables. For the first 3 extraction tables, the 2 researchers (CD and FM) independently assessed the content of the tools to summarize the addressed characteristics, functionalities, and key elements of ACP. Any disagreements were discussed to reach a consensus. If no consensus was reached, a third researcher (TS) made the final decision.
The first extraction table was used to assess the aim; target group; available languages; format; and where, by whom, and how the tools were developed. We evaluated the used functionalities by developing a second extraction table based on a review of peer-reviewed studies by van der Smissen et al [8]. We slightly changed the 12 assessed functionalities by removing can be used without assistance, as in our review, we only included tools that were designed to be used by the general population, and we added the functionalities predetermined path and possibility to save input and return as these allow the user to conduct the ACP process at their own pace and input can be printed as this functionality can increase the accessibility of websites [27]. Therefore, in this review, we extracted the following data with regard to the functionalities of ACP tools:
The tool is free of charge.
Registration is needed.
There is a save and return option.
It is possible for the user to give input.
The tool is tailored to the user.
The tools provide feedback based on the input of the user.
The input from the user can be printed.
The tool suggests a predetermined ACP path.
The tool gives an indication of the progress of the user.
Videos are used, as well as hyperlinks to other web pages.
There is a text-to-speech option.
A privacy policy is mentioned.
Data log analysis is mentioned.
The manner in which ACP is addressed in the tools was extracted using 14 ACP key elements (the third extraction table) as an analytic framework. The 14 key elements were self-developed and based on the recommendations of Rietjens et al [1], the consensus definition of ACP of Sudore et al [2], and the review by van der Smissen et al [8]. From these definitions, we aimed to extract all relevant elements that can be part of the ACP; that is, the following elements:
Providing information on ACP
Providing information on legal frameworks
Address readiness and timing for ACP
Stimulates to explore personal values and goals
Stimulates to explore preferences regarding future care
Stimulates to explore uncertainties and consequences
Stimulates to explore preferences regarding the last days of life
Stimulates to explore preferences for a possible proxy decision-maker
Encourages to appoint a proxy decision-maker
Encourages to discuss ACP with family
Encourages to discuss ACP with health care professionals
Encourages to document ACP
Encourages to generate that document (in the tool)
Encourages to share that document.
The readability, quality of the content, information about the evaluation of the tools, and their level of evidence were summarized in the fourth extraction table.
Results
Selection and Inclusion of the Tools
We found 436 tools using the first 3 search strategies. After removing duplicates (the tools retrieved and the removed duplicates per search strategy can be found in Multimedia Appendix 4), a list of 96 potential interactive web-based ACP support tools for the general population was sent to 15 experts on ACP. Of the 15 experts, 14 (93%) replied, and together they identified 35 additional tools. Two researchers (CD and FM) subsequently screened the 131 tools against the eligibility criteria. Approximately 69.5% (91/131) were excluded (Figure 1), and there were no conflicts to resolve regarding exclusion. The remaining 30.5% (40/131) of tools were fully screened (ie, all text available in the tool for data extraction), and we ended up excluding 10 more in an agreement between both researchers. Thus, the total number of included tools was 30. An overview of screening, selection, and reasons for exclusion and inclusion is shown in Figure 1.
Figure 1.
PRISMA (Preferred Reporting Item for Systematic Reviews and Meta-Analyses) flowchart of the selection process.
Characteristics of the Tools
Approximately 80% (24/30) of tools targeted the general population, although 20% (6/30) also aimed to target health care professionals (Multimedia Appendix 5 [28-57]). For instance, the app ACDCare could be used by patients to make an advance care directive that can be uploaded to their health record and to which the health care professional has access [28]. The tools Mydirectives, Be My Voice, My Living Will, Speak Up, and Considering your own future health care not only supported the general population in ACP but also provided information on ACP for health care professionals [29-33]. Of the 30 tools, 24 (80%) did mention a clear aim; 7 (23%) supported reflection or communication, 8 (27%) supported people in making decisions, 7 (23%) provided support to document decisions, and 2 (7%) aimed to achieve all these objectives.
Of the 30 tools, 15 (50%) were websites, 10 (33%) were web-based portals with the possibility of logging in and returning to personal information, 3 (10%) were apps, and 2 (7%) had a combination of formats. For example, PREPARE for Your Care (PREPARE) can be used via the website or by logging in to a web-based portal [34].
Of the 30 tools, 12 (40%) were developed in the United States, 6 (20%) in Canada, 4 (13%) in Australia, 3 (10%) in the Netherlands, 3 (10%) in the United Kingdom, 1 (3%) in New Zealand, and 1 (3%) in Belgium. Approximately 83% (25/30) were owned by nonprofit organizations. Approximately 23% (7/30) provided information on their development; all were developed in collaboration with health care professionals, and 10% (3/30) were also developed with end users.
Functionalities of the Tools
Of the 30 tools, 27 (90%) were free of charge, and 17 (57%) were available without registration (Multimedia Appendix 6 [28-57]). Approximately 97% (29/30) of tools offered users the option of providing input by responding to a question or statement, for example, by asking them to write in an empty box. With regard to the flow of the tool (the steps or path a user must take to go through it), 97% (29/30) used a predetermined path [28-44,46-57]. For example, if there are x number of steps in a tool, the user needs to go through these steps to finish the ACP process provided in the tool. One of the tools (the Go Wish card game) does not have a set path but uses a mechanism in which users can sort cards on particular wishes and preferences to stimulate reflection and communication [45]. Approximately 17% (5/30) used the input of the user to tailor information, redirect them to a specific webpage with more information, or ask for clarification on the input. For example, in My Values, when identifying not wanting to be a burden as important, a screen pops up asking them to explain briefly (by typing) what becoming a burden means to them [49]. One also uses input to give feedback (Multimedia Appendix 6 [28-57])—PREPARE not only provides information based on input but also provides tips when, for example, the user indicates not ready to choose a proxy decision-maker [34].
ACP Elements Addressed in the Tools
All tools contained at least seven of the 14 ACP key elements identified in our analytical framework, and 20% (6/30) of tools comprised all (Multimedia Appendix 7 [28-57]). With regard to the information available, 40% (12/30) provided both information on ACP and on the legal frameworks of ACP. Of the 30 tools, readiness and timing of ACP were addressed in 15 (50%), encouraging people to explore personal values and goals in 28 (93%), and preferences regarding future care in 29 (97%) tools; uncertainties and consequences of hypothetical serious illness scenarios were addressed in 25 (83%), preferences regarding the last days of life in 27 (90%), and the possibility of appointing a proxy decision-maker in 21 (70%) tools. Of the 30 tools, 20 (67%) encouraged appointing a proxy decision-maker, and 27 (90%) encouraged discussing ACP with family or a health care professional and documenting ACP outcomes (for example, using an advance directive); in 27 (90%) tools, it was possible to document one’s wishes and preferences within the tool itself and share this document with others (by printing, via email, or via a direct link; Multimedia Appendix 7 [28-57]).
Readability and Quality of the Content
The readability of the ACP tools varied (Table 1); however, 83% (25/30) of ACP tools had a B1 or lower CEFR level. The QUEST scores of the tools varied between 11 and 28 (theoretical scale scores between 0 and 28; Table 2). Most of the lower-scoring tools did not refer to any sources to support the information they contained, were not current (ie, not updated in the past 5 years), or did not provide any information on authorship. The tools that scored ≥21 used at least one reference to a scientific study to support the information in the tool.
Table 1.
Readability scores of the interactive, web-based ACPa support tools.b
| Tools | CEFRc scale |
| My decisions | A1 |
| My living voice | A1 |
| Speak up | A1 |
| PREPAREd | A2 |
| Go Wish card game | A2 |
| MyDirectives | A2 |
| Plan Your Life Span | A2 |
| Lets Think Ahead—My ACP | A2 |
| Advance Care Planning: Should I Stop Treatment That Prolongs My Life? | B1 |
| The Letter project Advance Directive | B1 |
| Advance Care Planning: Should I Receive CPRe and Life Support? | B1 |
| MyWishes | B1 |
| My Living Will | B1 |
| Everplans | B1 |
| Considering your own future health care | B1 |
| Tijdig nadenken over het levenseinde | B1 |
| Oog in Oog | B1 |
| Verken uw wensen voor zorg en behandeling | B1 |
| Dying to Talk | B1 |
| Be my voice | B1 |
| ACDCare | B1 |
| Five Wishes | B1 |
| MyValues | B1 |
| Dementia Values and Priorities Tool | B1 |
| Advance Care Planning: Should I Have Artificial Hydration and Nutrition? | B2 |
| Planning for Your Future | B2 |
| Beslishulp—Vroegtijdige zorgplanning | B2 |
| Cake | B2 |
| NVLivingWill | B2 |
| Advance Care Planning: Should I Stop Kidney Dialysis? | C1 |
aACP: advance care planning.
bRanked from lowest (A1) to highest score (C2) possible.
cCEFR: Common European Framework of Reference for Languages.
dPREPARE: PREPARE for Your Care.
eCPR: cardiopulmonary resuscitation.
Table 2.
QUESTa scores of the interactive, web-based ACPb support tools.c
| Tools | QUEST score |
| PREPAREd | 28 |
| Go Wish card game | 23 |
| Considering your own future health care | 20 |
| Everplans | 20 |
| Oog in Oog | 20 |
| Dementia Values and Priorities Tool | 19 |
| Dying to Talk | 19 |
| My Living Will | 19 |
| MyDirectives | 19 |
| NVLivingWill | 19 |
| Plan your Life Span | 19 |
| Cake | 17 |
| Five Wishes | 17 |
| Lets Think Ahead–My ACP | 17 |
| Planning for Your Future | 17 |
| Be my voice | 16 |
| Beslishulp-Vroegtijdige zorgplanning | 16 |
| MyValues | 16 |
| MyWishes | 16 |
| My decisions | 14 |
| Advance Care Planning: Should I Have Artificial Hydration and Nutrition? | 13 |
| Advance Care Planning: Should I Receive CPRe and Life Support? | 13 |
| Advance Care Planning: Should I Stop Kidney Dialysis? | 13 |
| Advance Care Planning: Should I Stop Treatment That Prolongs My Life? | 13 |
| My living voice | 12 |
| ACDCare | 11 |
aQUEST: Quality Evaluation Scoring Tool.
bACP: advance care planning.
cRanked from lowest (0) to highest score (28) possible.
dPREPARE: PREPARE for Your Care.
eCPR: cardiopulmonary resuscitation.
Evaluated Tools and Their Level of Evidence
Of the 30 included tools, 5 (16%) tools had been evaluated in a study, all of which were published in the peer-reviewed literature (Table 3). Verken uw wensen voor zorg en behandeling [57] is under evaluation; however, the results have not yet been published. MyDirectives and NVLivingWill were studied using a posttest design [58-60], and PREPARE, Plan Your Life Span, and The Letter project Advance Directive were studied in at least one properly designed randomized controlled trial [61-71]. The study on NVLivingWill evaluated the ease of use of the tool [58]; the studies on PREPARE evaluated ease of use, effectiveness, acceptability, and understandability (information in the tool was easy to read) [63-71]; Plan Your Life Span evaluated effectiveness [61]; and the study on The Letter project Advance Directive evaluated understandability [72].
Table 3.
Level of evidence of the interactive, web-based advance care planning support tools.
| Tool, authors, and title of the study | The hierarchy of evidencea | |||
| MyDirectives [29] | Evidence obtained from a posttest | |||
|
|
Holland et al [59] | Nurse-led patient-centered advance care planning in primary care |
|
|
|
|
Fine et al [60] | Early experience with digital advance care planning and directives, a novel consumer-driven program |
|
|
| NVLivingWill [51] | Evidence obtained from a posttest | |||
|
|
Klugman and Usatine [58] | An evaluation of 2 online advance directive programs |
|
|
| Plan Your Life Span [53] | Evidence obtained from at least one properly designed randomized controlled trial | |||
|
|
Ramirez-Zohfeld et al [62] | Longitudinal follow-up of long-term care planning using PlanYourLifespan.org |
|
|
|
|
Lindquist et al [61] | PlanYourLifeSpan.org—an intervention to help seniors make choices for their fourth quarter of life: results from the randomized clinical trial |
|
|
| PREPAREb [34] | Evidence obtained from at least one properly designed randomized controlled trial | |||
|
|
Howard et al [71] | Effect of an interactive website to engage patients in advance care planning in outpatient settings |
|
|
|
|
Freytag et al [70] | Empowering older adults to discuss advance care planning during clinical visits: the PREPARE randomized trial |
|
|
|
|
Lum et al [69] | Improving a full range of advance care planning behavior change and action domains: the PREPARE randomized trial |
|
|
|
|
Zapata et al [68] | Feasibility of a video-based advance care planning website to facilitate group visits among diverse adults from a safety-net health system |
|
|
|
|
Sudore et al [67] | Engaging diverse English- and Spanish-speaking older adults in advance care planning: the PREPARE randomized clinical trial |
|
|
|
|
Sudore et al [2,66] | Effect of the PREPARE website vs an easy-to-read advance directive on advance care planning documentation and engagement among veterans |
|
|
|
|
Cresswell et al [65] | Evaluation of an advance care planning web-based resource: applicability for cancer treatment patients |
|
|
|
|
Ouchi et al [64] | Preparing older adults with serious illness to formulate their goals for medical care in the emergency department |
|
|
|
|
Sudore et al [63] | A novel website to prepare diverse older adults for decision-making and advance care planning: a pilot study |
|
|
| The Letter project Advance Directive [55] | Evidence obtained from at least one properly designed randomized controlled trial | |||
|
|
Periyakoil et al [72] | A randomized controlled trial comparing the letter project advance directive to traditional advance directive |
|
|
aHierarchy of evidence from the National Health and Medical Research Council was assessed per tool. Level I: evidence obtained from a systematic review of all relevant randomized controlled trials; level II: evidence obtained from at least one properly designed randomized controlled trial; level III-1: evidence obtained from well-designed pseudorandomized controlled trials (alternate allocation or some other method); level III-2: evidence obtained from comparative studies with concurrent controls and allocation not randomized (cohort studies), case control studies, or interrupted time series with a control group; level III-3: evidence obtained from comparative studies with historical control, ≥2 single-arm studies, or interrupted time series without a parallel control group; level IV: evidence obtained from case series, either posttest or pretest, and posttest [25,26].
bPREPARE: PREPARE for Your Care.
Discussion
Principal Findings
This review included 30 ACP tools developed in North America, Europe, and Oceania. Most tools mention a clear aim (ie, to support reflection and communication, support people in making decisions, or support documenting decisions); however, only 7% (2/30) aimed to achieve all 3 aims. Of the 30 tools, 7 (23%) were developed in collaboration with health care professionals, but only 3 (10%) also involved end users. All tools, except 1, encouraged users to follow steps in a predetermined order to go through the ACP process. With regard to the ACP elements, almost all tools stimulated the user to explore personal values, goals, and preferences regarding future care; 40% (12/30) provided both information on ACP and its legal frameworks. Of the 3 tools, 2 (67%) also encouraged the user to appoint a proxy decision-maker. Most of the ACP tools had a good readability score; however, the quality of the content varied between 11 and 28 on the QUEST scale. Most of the included ACP tools had not been evaluated in a study.
We found great variety among the tools available on the web in terms of their aims, functionalities, approaches to addressing ACP, and quality. However, the included tools also shared important commonalities.
First, we found that many tools did not provide information about their development process. If they did, they involved health care professionals such as physicians, experts on end-of-life care, ethicists, or lawyers. End users were only involved in the development process of 10% (3/30) of tools, although this is highly recommended in the literature on developing web-based technologies [73-75]. Research on developing new technologies shows that the involvement of end users inevitably yields improvements in usability and quality and ensures that the tool is tailored to the needs of prospective end users [73-75].
Second, most tools stimulated the user to explore personal values, goals, and preferences regarding future care and the last days of life, which is the primary purpose of ACP [1,2], and most also encouraged the appointment of a proxy decision-maker and discussion with family and health care professionals. However, all but 1 tool in this review used an approach in which all users are encouraged to follow a predetermined path or step to go through the ACP process. ACP is a process of exploration; discussion; and recording of preferences, wishes, and decisions. How and when to best perform these aspects of ACP depends, among other things, on personal barriers, needs, preferences, and readiness [76,77]. Using steps in a predetermined order suggests that ACP is linear instead of an iterative process [1]. Moreover, using predetermined steps may not be appropriate for all users, as some may just want to explore possibilities without making decisions, whereas others, for instance, because of their illness, may prefer to focus on anticipatory decision-making [76,78].
Third, with regards to the quality of the content of the included tools, as rated by the QUEST score, we found that 20% (6/30) scored ≥20, of which 17% (1/6; PREPARE) scored a maximum of 28 points. All other tools had a medium to low quality score as they did not refer to any information sources and were not up to date; that is, they had not been updated in at least 5 years. Especially with regard to ACP, having evidence-based information that can be verified by the user is important as people use this information to plan and make health-related care decisions [79,80]. We also found that most of the included tools had not been evaluated in a study. Only the PREPARE tool had been evaluated for its ease of use, effectiveness, acceptability, and understandability. When people look for support in ACP, they may use tools that have a low or even nonexistent level of evidence regarding their usability and effectiveness [13-16,81].
Strengths, Limitations, and Future Research
This is the first systematic review to provide an overview of interactive web-based ACP support tools available on the internet for the general population. Previous reviews have provided an overview of web-based ACP support tools that could be found in the peer-reviewed literature and emphasized the absence of an overview of those available in the gray literature [7,8]. Furthermore, our review is the first to assess the quality of the content of web-based ACP support tools [7,8]. This study had some limitations. First, as we limited our third search strategy (ie, search engine) to 150 hits per search combination, there is a possibility that some existing tools were not included in our review. However, we consider this unlikely, as our search was conducted systematically by 2 researchers using a broad range of search terms, and we searched more hits than recommended (recommended 100) by Godin et al [17]. We also consulted experts and asked them whether they knew of any other tools that we had not yet found. Furthermore, although we archived the tools when performing data synthesis using Archive.is, there is a possibility that new content or functionalities have been added to the tools included in our review or that new tools have been released since the search was conducted.
For future developments, users should be involved in aligning their preferences and needs with the content and functionalities of the tools. Involving users early in the development process can improve the usability of tools and increase their uptake [82]. In addition, we would recommend following a thorough design process using existing road maps when developing new ACP tools, rigorously evaluating through usability and effectiveness testing before deployment, and transparently reporting on development and evaluation. Second, the ACP content provided to users should be regularly updated and supported by sources; hence, we recommend that the content in ACP tools should be substantiated with the most recent scientific literature. Moreover, future research should focus on how ACP tools are used by the general population and how they can support ACP in the medical context; that is, between patients and health care professionals. Finally, it would be interesting to know how the general population would assess these tools.
Conclusions
There are numerous interactive web-based ACP support tools that are publicly available, varying in terms of their characteristics, functionalities, readability, quality of content, and level of evidence. Most tools were not codeveloped with end users; were of low or medium quality; and, with a few exceptions, had not been evaluated in research. In the future, users should be involved in the development of ACP support tools, and their content should be substantiated by scientific evidence. In addition, we recommend that developers should follow a rigorous design process and evaluate the usability and effectiveness of tools before their deployment. Future research should focus on how tools are used by the general population and how they can support ACP in the medical context; that is, between patients and health care professionals.
Acknowledgments
The authors would like to thank all the experts for their time and effort in assisting in data collection and Jane Ruthven for English editing.
This study is part of the project CAPACITY: Flanders Project to Develop Capacity in Palliative Care Across Society, a collaboration among the Vrije Universiteit Brussels, Ghent University, and the Catholic University Leuven, Belgium. This study was supported by a grant from the Research Foundation, Flanders (file number S002219N).
Abbreviations
- ACP
advance care planning
- CEFR
Common European Framework of Reference for Languages
- PREPARE
PREPARE for Your Care
- PROSPERO
International Prospective Register of Systematic Reviews
- QUEST
Quality Evaluation Scoring Tool
Search terms for the web-based gray literature databases.
Search terms for the Google search engine.
Search terms for app stores.
Tools retrieved per search strategy.
Characteristics of the included interactive and web-based advance care planning support tools.
Functionalities of included interactive and web-based advance care planning support tools.
Advance care planning elements addressed in interactive web-based advance care planning support tools.
Footnotes
Authors' Contributions: All authors have substantially contributed to the research and approved the version to be published. The search, screening, selection, and synthesis were performed by CD and FM. The original draft was prepared by CD. TS, LP, ADV, CVA, and LVdB were involved in the review and editing of the manuscript. TS and LVdB were involved in supervision.
Conflicts of Interest: None declared.
References
- 1.Rietjens JA, Sudore RL, Connolly M, van Delden JJ, Drickamer MA, Droger M, van der Heide A, Heyland DK, Houttekier D, Janssen DJ, Orsi L, Payne S, Seymour J, Jox RJ, Korfage IJ, European Association for Palliative Care Definition and recommendations for advance care planning: an international consensus supported by the European Association for Palliative Care. Lancet Oncol. 2017 Sep;18(9):e543–51. doi: 10.1016/S1470-2045(17)30582-X.S1470-2045(17)30582-X [DOI] [PubMed] [Google Scholar]
- 2.Sudore RL, Lum HD, You JJ, Hanson LC, Meier DE, Pantilat SZ, Matlock DD, Rietjens JA, Korfage IJ, Ritchie CS, Kutner JS, Teno JM, Thomas J, McMahan RD, Heyland DK. Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel (S740) J Pain Symptom Manage. 2017 May;53(5):821–32.e1. doi: 10.1016/j.jpainsymman.2016.12.331. https://linkinghub.elsevier.com/retrieve/pii/S0885-3924(16)31232-5 .S0885-3924(16)31232-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sinclair JB, Oyebode JR, Owens RG. Consensus views on advance care planning for dementia: a Delphi study. Health Soc Care Community. 2016 Mar;24(2):165–74. doi: 10.1111/hsc.12191. [DOI] [PubMed] [Google Scholar]
- 4.Van den Block L. Advancing research on advance care planning in dementia. Palliat Med. 2019 Mar;33(3):259–61. doi: 10.1177/0269216319826411. [DOI] [PubMed] [Google Scholar]
- 5.Prince-Paul M, DiFranco E. Upstreaming and normalizing advance care planning conversations-a public health approach. Behav Sci (Basel) 2017 Apr 12;7(2):18. doi: 10.3390/bs7020018. https://www.mdpi.com/resolver?pii=bs7020018 .bs7020018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Tishelman C, Eneslätt M, Menkin ES, Van Den Block L. Tishelman et al's response to Morrison: advance directives/care planning: clear, simple, and wrong (DOI: 10.1089/jpm.2020.0272) J Palliat Med. 2021 Jan;24(1):16–7. doi: 10.1089/jpm.2020.0540. [DOI] [PubMed] [Google Scholar]
- 7.Butler M, Ratner E, McCreedy E, Shippee N, Kane RL. Decision aids for advance care planning: an overview of the state of the science. Ann Intern Med. 2014 Sep 16;161(6):408–18. doi: 10.7326/M14-0644. https://www.acpjournals.org/doi/abs/10.7326/M14-0644?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .1891307 [DOI] [PubMed] [Google Scholar]
- 8.van der Smissen D, Overbeek A, van Dulmen S, van Gemert-Pijnen L, van der Heide A, Rietjens JA, Korfage IJ. The feasibility and effectiveness of web-based advance care planning programs: scoping review. J Med Internet Res. 2020 Mar 17;22(3):e15578. doi: 10.2196/15578. https://www.jmir.org/2020/3/e15578/ v22i3e15578 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Narang AK, Wright AA, Nicholas LH. Trends in advance care planning in patients with cancer: results from a national longitudinal survey. JAMA Oncol. 2015 Aug;1(5):601–8. doi: 10.1001/jamaoncol.2015.1976. http://europepmc.org/abstract/MED/26181909 .2383145 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Green MJ, Levi BH. The era of "e": the use of new technologies in advance care planning. Nurs Outlook. 2012;60(6):376–83.e2. doi: 10.1016/j.outlook.2012.08.005. http://europepmc.org/abstract/MED/23141197 .S0029-6554(12)00231-X [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, Dew MA. User-centered design and interactive health technologies for patients. Comput Inform Nurs. 2009;27(3):175–83. doi: 10.1097/NCN.0b013e31819f7c7c. http://europepmc.org/abstract/MED/19411947 .00024665-200905000-00011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cline RJ, Haynes KM. Consumer health information seeking on the internet: the state of the art. Health Educ Res. 2001 Dec;16(6):671–92. doi: 10.1093/her/16.6.671. [DOI] [PubMed] [Google Scholar]
- 13.Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, Kanouse DE, Muñoz JA, Puyol JA, Lara M, Watkins KE, Yang H, McGlynn EA. Health information on the internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285(20):2612–21. doi: 10.1001/jama.285.20.2612. http://europepmc.org/abstract/MED/11368735 .joc02274 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Seçkin G, Yeatts D, Hughes S, Hudson C, Bell V. Being an informed consumer of health information and assessment of electronic health literacy in a national sample of internet users: validity and reliability of the e-HLS Instrument. J Med Internet Res. 2016 Jul 11;18(7):e161. doi: 10.2196/jmir.5496. https://www.jmir.org/2016/7/e161/ v18i7e161 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Powell J, Inglis N, Ronnie J, Large S. The characteristics and motivations of online health information seekers: cross-sectional survey and qualitative interview study. J Med Internet Res. 2011 Feb 23;13(1):e20. doi: 10.2196/jmir.1600. https://www.jmir.org/2011/1/e20/ v13i1e20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Fahy E, Hardikar R, Fox A, Mackay S. Quality of patient health information on the Internet: reviewing a complex and evolving landscape. Australas Med J. 2014;7(1):24–8. doi: 10.4066/AMJ.2014.1900. http://europepmc.org/abstract/MED/24567763 .20141900 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015 Oct 22;4:138. doi: 10.1186/s13643-015-0125-0. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-015-0125-0 .10.1186/s13643-015-0125-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Readability score | Readability test | Reading level calculator. Readable. [2022-02-17]. https://readable.com/
- 19.Leesniveau. Accessibility Foundation. [2022-02-17]. https://www.accessibility.nl/tools/leesniveau .
- 20.Bletsis PP, Egeler S, van der Lei B, Lee BT, Lin SJ. Readability of Dutch online patient-directed health information on breast reconstruction. Eur J Plast Surg. 2019 Aug;42(4):343–50. doi: 10.1007/s00238-019-1498-4. [DOI] [Google Scholar]
- 21.Meppelink CS, van Weert JC, Brosius A, Smit EG. Dutch health websites and their ability to inform people with low health literacy. Patient Educ Couns. 2017 Nov;100(11):2012–9. doi: 10.1016/j.pec.2017.06.012.S0738-3991(17)30358-0 [DOI] [PubMed] [Google Scholar]
- 22.North B, Piccardo N. Common European framework of reference for languages: learning, teaching, assessment?; companion volume. Strasbourg, France: Council of Europe Publishing; 2020. Jul, pp. 55–59. [Google Scholar]
- 23.Robillard JM, Jun JH, Lai JA, Feng TL. The QUEST for quality online health information: validation of a short quantitative tool. BMC Med Inform Decis Mak. 2018 Oct 19;18(1):87. doi: 10.1186/s12911-018-0668-9. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0668-9 .10.1186/s12911-018-0668-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Roberts L. Health information and the Internet: the 5 Cs website evaluation tool. Br J Nurs. 2010;19(5):322–5. doi: 10.12968/bjon.2010.19.5.47075. [DOI] [PubMed] [Google Scholar]
- 25.National Health and Medical Research Council . A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra, Australia: Commonwealth of Australia; 1999. pp. 15–17. [Google Scholar]
- 26.Anderson CF. Guide to clinical preventive services: an assessment of the effectiveness of 169 interventions. Mayo Clin Proc. 1989 Dec 1;64(12):1566–7. doi: 10.1016/s0025-6196(12)65716-4. [DOI] [Google Scholar]
- 27.Make web content printer friendly. Health Literacy Online. [2022-01-12]. https://health.gov/healthliteracyonline/display/section-3-10/
- 28.ACDCare. GP Partners Australia. [2021-03-15]. http://archive.today/2021.04.16-082031/https://apps.apple.com/uy/app/acdcare/id1500681626 .
- 29.MyDirectives. [2021-03-15]. http://archive.today/2021.04.16-083317/https://mydirectives.com/
- 30.Be My Voice. Capital Health Network. [2021-03-15]. http://archive.today/2021.04.16-083021/http://www.bemyvoice.com.au/Home.aspx .
- 31.My Living Will. Vital Decisions. [2021-03-15]. http://archive.today/2021.04.16-083023/https://www.mylivingwill.org.uk/static/wizard?Type=Statement .
- 32.Speak up. Canadian Hospice Palliative Care Association. [2021-03-15]. http://archive.today/2021.04.16-083319/https://www.advancecareplanning.ca/my-plan/workbook/
- 33.Considering your own future health care. Health Quality & Safety Commission, New Zealand. 2018. [2021-03-15]. http://archive.today/2021.04.16-082903/https://www.hqsc.govt.nz/our-programmes/advance-care-planning/publications-and-resources/
- 34.PREPARE for your care. The Regents of the University of California. 2018. [2021-03-15]. http://archive.today/2021.04.16-083623/https://prepareforyourcare.org/welcome .
- 35.Advance Care Planning: Should I Have Artificial Hydration and Nutrition? Healthwise and Alberta. 2019. [2021-03-15]. http://archive.today/2021.04.16-082516/https://www.healthlinkbc.ca/health-topics/tu4431 .
- 36.Advance Care Planning: should I receive CPR and life support? Healthwise and Alberta. 2019. [2021-03-15]. http://archive.today/2021.04.16-082531/https://www.healthlinkbc.ca/health-topics/tu2951 .
- 37.Advance Care Planning: should I stop kidney dialysis? Healthwise and Alberta. 2019. [2021-03-15]. http://archive.today/2021.04.16-082541/https://myhealth.alberta.ca/Health/Pages/conditions.aspx?hwid=tu6095 .
- 38.Advance Care Planning: should I stop treatment that prolongs my life? Healthwise and Alberta. 2019. [2021-03-15]. http://archive.today/2021.04.16-082605/https://myhealth.alberta.ca/Health/pages/conditions.aspx?hwid=tu1430&lang=en-ca .
- 39.Vroegtijdige zorgplanning (voor patiënt) - Beslishulp. Centrum voor Evidence-Based Medicine (Cebam) [2021-03-15]. http://archive.today/2021.04.16-083644/https://www.gezondheidenwetenschap.be/beslissingshulp/vroegtijdige-zorgplanning .
- 40.Cake. [2021-03-15]. http://archive.today/2021.04.16-084034/https://www.joincake.com/
- 41.Dementia Values and Priorities Tool. Compassion & Choices. [2021-03-15]. http://archive.today/2021.04.16-084038/https://values-tool.compassionandchoices.org/
- 42.Dying to Talk - Discussion Starters. Palliative Care Australia. [2021-03-15]. https://dyingtotalk.org.au/discussion-starters/
- 43.Everplans. 2012. [2021-03-15]. http://archive.today/2021.04.16-084153/https://webapps.everplans.com/%23/auth/sign-up .
- 44.Five Wishes. [2021-03-15]. http://archive.today/2021.04.16-084416/https://fivewishes.org/shop/order/product/five-wishes-online .
- 45.Go Wish card game online version. CODA Alliance. 2018. [2021-03-15]. http://archive.today/2021.04.16-083901/http://www.gowish.org/gowish/gowish.html .
- 46.Lets Think Ahead - My ACP. University of the West Scotland. [2021-03-22]. http://archive.today/2021.04.16-082949/https://www.scet.co/fullscreen-page/comp-iqtuyfne/e2469d5e-3e56-4452-803d-ea83e55b2f32/4/%3Fi%3D4 .
- 47.Compassion in Dying charity. My Decisions. [2021-03-15]. http://archive.today/2021.04.16-084547/https://mydecisions.org.uk/
- 48.My Living Voice. Vital Decisions. [2021-03-15]. http://archive.today/2021.04.16-084549/https://vitaldecisions.mylivingvoice.com/
- 49.My Values. Barwon Health. [2021-03-15]. http://archive.today/2021.04.16-083754/https://www.myvalues.org.au/
- 50.MyWishes. [2021-03-15]. https://www.app.mywishes.co.uk/register .
- 51.NVLivingWill Living Will Online Completion. Nevada Center for Ethics & Health Policy. [2021-03-15]. http://archive.today/2021.04.16-084642/http://www.nvlivingwill.com/
- 52.Bureau MORBidee Oog in Oog. Google Play. [2021-03-15]. http://archive.today/2021.04.16-084820/https://play.google.com/store/apps/details?id=com.eyetoeye.eye_to_eye&hl=en_IE .
- 53.Plan your Life Span. Northwestern University. [2021-03-15]. http://archive.today/2021.04.16-084828/https://www.planyourlifespan.org/
- 54.Prepare an advance directive. People's Law School. 2019. [2021-03-15]. https://www.peopleslawschool.ca/prepare-advance-directive/
- 55.Use the Letter Project form to complete your Advance Directives in Minutes! Stanford Medicine Letter Project. 2015. [2021-03-15]. https://med.stanford.edu/letter/advancedirective.html .
- 56.Tijdig nadenken over het levenseinde [Timely thinking about the end of life] NPV Zorg voor het leven. [2021-03-15]. https://ehealth-npvzorg.serviant.nl/login/?back=https://ehealth-npvzorg.serviant.nl/client/module%3FtrajectId%3D3692%26q%3DlTTLQmY .
- 57.Verken uw wensen voor zorg en behandeling [Explore your care and treatment needs] Dutch College of General Practitioners, Erasmus MC and thuisarts.nl. [2021-03-15]. https://www.thuisarts.nl/keuzehulp/verken-uw-wensen-voor-zorg-en-behandeling .
- 58.Klugman CM, Usatine RP. An evaluation of 2 online advance directive programs. Am J Hosp Palliat Care. 2013 Nov;30(7):657–63. doi: 10.1177/1049909112463116.1049909112463116 [DOI] [PubMed] [Google Scholar]
- 59.Holland DE, Vanderboom CE, Dose AM, Ingram CJ, Delgado A, Austin CM, Green MJ, Levi B. Nurse-led patient-centered advance care planning in primary care. J Hosp Palliat Nurs. 2017;19(4):368–75. doi: 10.1097/NJH.0000000000000358. [DOI] [Google Scholar]
- 60.Fine RL, Yang Z, Spivey C, Boardman B, Courtney M. Early experience with digital advance care planning and directives, a novel consumer-driven program. Proc (Bayl Univ Med Cent) 2016 Jul;29(3):263–7. doi: 10.1080/08998280.2016.11929430. http://europepmc.org/abstract/MED/27365867 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Lindquist LA, Ramirez-Zohfeld V, Sunkara PD, Forcucci C, Campbell DS, Mitzen P, Ciolino JD, Gregory D, Kricke G, Cameron KA. PlanYourLifeSpan.org - an intervention to help seniors make choices for their fourth quarter of life: Results from the randomized clinical trial. Patient Educ Couns. 2017 Nov;100(11):1996–2004. doi: 10.1016/j.pec.2017.06.028. http://europepmc.org/abstract/MED/28689855 .S0738-3991(17)30376-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Ramirez-Zohfeld V, Seltzer A, Ramirez A, Muhammad R, Lindquist LA. Longitudinal follow-up of long-term care planning using PlanYourLifespan.org. J Appl Gerontol. 2021 May;40(5):536–40. doi: 10.1177/0733464820943066. http://europepmc.org/abstract/MED/32686545 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Sudore RL, Knight SJ, McMahan RD, Feuz M, Farrell D, Miao Y, Barnes DE. A novel website to prepare diverse older adults for decision making and advance care planning: a pilot study. J Pain Symptom Manage. 2014 Apr;47(4):674–86. doi: 10.1016/j.jpainsymman.2013.05.023. https://linkinghub.elsevier.com/retrieve/pii/S0885-3924(13)00386-2 .S0885-3924(13)00386-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Ouchi K, Knabben V, Rivera-Reyes L, Ganta N, Gelfman LP, Sudore R, Hwang U. Preparing older adults with serious illness to formulate their goals for medical care in the emergency department. J Palliat Med. 2017 Apr 01;20(4):404–8. doi: 10.1089/jpm.2016.0109. http://europepmc.org/abstract/MED/27797638 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Cresswell MA, Robinson CA, Fyles G, Bottorff JL, Sudore R. Evaluation of an advance care planning web-based resource: applicability for cancer treatment patients. Support Care Cancer. 2018 Mar;26(3):853–60. doi: 10.1007/s00520-017-3901-7.10.1007/s00520-017-3901-7 [DOI] [PubMed] [Google Scholar]
- 66.Sudore RL, Boscardin J, Feuz MA, McMahan RD, Katen MT, Barnes DE. Effect of the PREPARE website vs an easy-to-read advance directive on advance care planning documentation and engagement among veterans: a randomized clinical trial. JAMA Intern Med. 2017 Aug 01;177(8):1102–9. doi: 10.1001/jamainternmed.2017.1607. http://europepmc.org/abstract/MED/28520838 .2627902 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Sudore RL, Schillinger D, Katen MT, Shi Y, Boscardin WJ, Osua S, Barnes DE. Engaging diverse English- and Spanish-speaking older adults in advance care planning: the PREPARE randomized clinical trial. JAMA Intern Med. 2018 Dec 01;178(12):1616–25. doi: 10.1001/jamainternmed.2018.4657. http://europepmc.org/abstract/MED/30383086 .2708194 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Zapata C, Lum HD, Wistar E, Horton C, Sudore RL. Feasibility of a video-based advance care planning website to facilitate group visits among diverse adults from a safety-net health system. J Palliat Med. 2018 Jun;21(6):853–6. doi: 10.1089/jpm.2017.0476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Lum HD, Barnes DE, Katen MT, Shi Y, Boscardin J, Sudore RL. Improving a full range of advance care planning behavior change and action domains: the PREPARE randomized trial. J Pain Symptom Manage. 2018 Oct;56(4):575–81.e7. doi: 10.1016/j.jpainsymman.2018.06.007. https://linkinghub.elsevier.com/retrieve/pii/S0885-3924(18)30312-9 .S0885-3924(18)30312-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Freytag J, Street Jr RL, Barnes DE, Shi Y, Volow AM, Shim JK, Alexander SC, Sudore RL. Empowering older adults to discuss advance care planning during clinical visits: the PREPARE randomized trial. J Am Geriatr Soc. 2020 Jun;68(6):1210–7. doi: 10.1111/jgs.16405. http://europepmc.org/abstract/MED/32157684 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Howard M, Robinson CA, McKenzie M, Fyles G, Sudore RL, Andersen E, Arora N, Barwich D, Bernard C, Elston D, Heyland R, Klein D, McFee E, Mroz L, Slaven M, Tan A, Heyland DK. Effect of an interactive website to engage patients in advance care planning in outpatient settings. Ann Fam Med. 2020 Mar;18(2):110–7. doi: 10.1370/afm.2471. http://www.annfammed.org/cgi/pmidlookup?view=long&pmid=32152014 .18/2/110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Periyakoil VS, Neri E, Kraemer H. A randomized controlled trial comparing the letter project advance directive to traditional advance directive. J Palliat Med. 2017 Sep;20(9):954–65. doi: 10.1089/jpm.2017.0066. [DOI] [PubMed] [Google Scholar]
- 73.Fischer B, Peine A, Östlund B. The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist. 2020 Sep 15;60(7):e513–23. doi: 10.1093/geront/gnz163. http://europepmc.org/abstract/MED/31773145 .5644100 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Shah SG, Robinson I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int J Technol Assess Health Care. 2007;23(1):131–7. doi: 10.1017/S0266462307051677.S0266462307051677 [DOI] [PubMed] [Google Scholar]
- 75.de Beurs D, van Bruinessen I, Noordman J, Friele R, van Dulmen S. Active involvement of end users when developing web-based mental health interventions. Front Psychiatry. 2017;8:72. doi: 10.3389/fpsyt.2017.00072. doi: 10.3389/fpsyt.2017.00072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Zwakman M, Jabbarian LJ, van Delden J, van der Heide A, Korfage IJ, Pollock K, Rietjens J, Seymour J, Kars MC. Advance care planning: a systematic review about experiences of patients with a life-threatening or life-limiting illness. Palliat Med. 2018 Sep;32(8):1305–21. doi: 10.1177/0269216318784474. https://journals.sagepub.com/doi/10.1177/0269216318784474?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Piers R, Albers G, Gilissen J, De Lepeleire J, Steyaert J, Van Mechelen W, Steeman E, Dillen L, Vanden Berghe P, Van den Block L. Advance care planning in dementia: recommendations for healthcare professionals. BMC Palliat Care. 2018 Jun 21;17(1):88. doi: 10.1186/s12904-018-0332-2. https://bmcpalliatcare.biomedcentral.com/articles/10.1186/s12904-018-0332-2 .10.1186/s12904-018-0332-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Slyer J, Archibald E, Moyo F, Truglio-Londrigan M. Advance care planning and anticipatory decision making in patients with Alzheimer disease. Nurse Pract. 2018 Jun 11;43(6):23–31. doi: 10.1097/01.NPR.0000532763.68509.e4.00006205-201806000-00006 [DOI] [PubMed] [Google Scholar]
- 79.Sadasivam RS, Kinney RL, Lemon SC, Shimada SL, Allison JJ, Houston TK. Internet health information seeking is a team sport: analysis of the Pew Internet Survey. Int J Med Inform. 2013 Mar;82(3):193–200. doi: 10.1016/j.ijmedinf.2012.09.008.S1386-5056(12)00192-X [DOI] [PubMed] [Google Scholar]
- 80.Lenhart A. Pew Research Center. 2007. Jun 27, [2021-03-30]. https://www.pewresearch.org/internet/2007/06/27/about-the-pew-internet-american-life-project/
- 81.Kaufman D, Roberts WD, Merrill J, Lai TY, Bakken S. Applying an evaluation framework for health information system design, development, and implementation. Nurs Res. 2006;55(2 Suppl):S37–42. doi: 10.1097/00006199-200603001-00007.00006199-200603001-00007 [DOI] [PubMed] [Google Scholar]
- 82.van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011 Dec 05;13(4):e111. doi: 10.2196/jmir.1672. https://www.jmir.org/2011/4/e111/ v13i4e111 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Search terms for the web-based gray literature databases.
Search terms for the Google search engine.
Search terms for app stores.
Tools retrieved per search strategy.
Characteristics of the included interactive and web-based advance care planning support tools.
Functionalities of included interactive and web-based advance care planning support tools.
Advance care planning elements addressed in interactive web-based advance care planning support tools.