Communication in decision aids for stage I–III colorectal cancer patients: a systematic review

Abstract

Objectives

To assess the communicative quality of colorectal cancer patient decision aids (DAs) about treatment options, the current systematic review was conducted.

Design

Systematic review.

Data sources

DAs (published between 2006 and 2019) were identified through academic literature (MEDLINE, Embase, CINAHL, Cochrane Library and PsycINFO) and online sources.

Eligibility criteria

DAs were only included if they supported the decision-making process of patients with colon, rectal or colorectal cancer in stages I–III.

Data extraction and synthesis

After the search strategy was adapted from similar systematic reviews and checked by a colorectal cancer surgeon, two independent reviewers screened and selected the articles. After initial screening, disagreements were resolved with a third reviewer. The review was conducted in concordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. DAs were assessed using the International Patient Decision Aid Standards (IPDAS) and Communicative Aspects (CA) checklist.

Results

In total, 18 DAs were selected. Both the IPDAS and CA checklist revealed that there was a lot of variation in the (communicative) quality of DAs. The findings highlight that (1) personalisation of treatment information in DAs is lacking, (2) outcome probability information is mostly communicated verbally and (3) information in DAs is generally biased towards a specific treatment. Additionally, (4) DAs about colorectal cancer are lengthy and (5) many DAs are not written in plain language.

Conclusions

Both instruments (IPDAS and CA) revealed great variation in the (communicative) quality of colorectal cancer DAs. Developers of patient DAs should focus on personalisation techniques and could use both the IPDAS and CA checklist in the developmental process to ensure personalised health communication and facilitate shared decision making in clinical practice.

Strengths and limitations of this study.

• This is the first large-scale and comprehensive systematic review on stage I–III decision aids (DAs) for patients with colorectal cancer.

• Both academically tested DAs as well as DAs that patients can find online were included to create a more accurate picture of the quality of DAs patients can find and use in clinical practice.

• Although the included DAs are freely available to patients, we cannot conclude based on this review whether they are actually being used in clinical practice.

Introduction

Colorectal cancer (CRC) is the third most common cancer in the world.¹ With emerging knowledge and availability of technology, the therapeutic options for these patients are increasing. For instance, selected early-stage CRC may now be removed with a minimally invasive endoscopic approach.² However, these tumours carry a small risk of metastatic spread to the regional lymph nodes which are left behind after endoscopic treatment.³ This risk may be lowered by removal of these lymph nodes but this in turn requires additional surgery with its inherent risks for postoperative complications. Similar considerations come into play with regard to adjuvant chemotherapy after curative resection of high-risk CRC⁴ and whether or not to treat with neo-adjuvant radiotherapy in rectal cancer.^{5 6} Recently, a ‘watch-and-wait’ approach is gaining popularity in patients with complete clinical response after radio-chemotherapy as an alternative to radical surgery.⁷

In all these scenarios, the potential beneficial effect on oncological outcome of a more radical approach should be weighed against the possible negative effects on long-term quality of life.⁸ To help weigh the pros and cons of these treatment decisions, so-called patient decision aids (DAs) have been developed. Such tools specifically aim to assist patients and clinicians with decision making so that the patient has a better understanding of the treatment options and has insight in their personal preferences regarding treatments. In turn, patient and clinician discuss these personal preferences during consultation, so that they jointly decide which treatment is best. This process is called ‘shared decision making’.⁹

A recent systematic review assessing the quality of such DAs for various diseases concluded that patients using DAs have (1) a better knowledge of treatments, (2) are better informed about treatments and (3) have a better understanding of their personal values compared to patients in usual care.¹⁰ This seems promising, but patients also increasingly use the internet as an important source of health information¹¹ and the DAs found through the web are not included in such large-scale systematic reviews. Therefore, conclusions drawn from such reviews do not necessarily reflect clinical reality. Two recent systematic reviews that did include DAs found through the web—as well as academically developed DAs—found that the quality of DAs for breast and prostate cancer is relatively low.^{12 13} The authors conclude that there is a lot of variation between individual DAs, and assessment of DAs in other cancer domains is necessary to have a more accurate reflection of what is happening in clinical practice. Especially since many CRC patients have low literacy skills,¹⁴ it seems crucial to assess the information and communication in DAs aimed at supporting patients with CRC with shared decision making. The focus of the current review is on stage I–III CRC DAs, where curative treatment is the main goal (and are therefore distinctly different from stage IV DAs). Currently, we are aware of only one systematic review that focusses on CRC DAs for treatment. However, this small review¹⁵ only included three academically developed DAs. Another systematic review assessed the usefulness of metastatic CRC nomograms¹⁶ (N=14). Both reviews conclude that quality of DAs for CRC is generally low and few patient DAs for CRC have been developed. The aims of this systematic review are therefore to (1) create a larger corpus of all existing treatment DAs for stage I–III CRC found both through scientific literature as online searches, (2) to get a deeper understanding of the general quality of CRC DAs and (3) to assess the communicative quality of such DAs.

Materials and methods

Patient and public involvement statement

With this study, we aimed to create a more accurate depiction of clinical practice for patients by not only including academically validated DAs, but also DAs that patients could find online. No patients were involved in the design or production, or in any other aspect of this systematic review.

Search strategy

To identify DAs for patients with stage I–III CRC, a systematic academic and online literature search was performed in concordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.¹⁷ The MEDLINE, Embase, CINAHL, Cochrane Library and PsycINFO databases were searched from 2006 (as this is the launch date of the International Patient Decision Aids Standards, one of our assessment instruments) to February 2019. The search strategy (see online supplemental appendix 1) was adapted from earlier systematic reviews for prostate¹² and breast¹³ cancer DAs and checked by a CRC clinician (IdH). References and author names of the studies found were checked for additional eligible DAs. The Ottawa Decision Aid Library and The International Database for Support in Medical Choices (Med-Decs) were also consulted. Languages included were Dutch, German and English.

To ensure that all DAs accessible to patients were incorporated into our analysis, we also performed a Google and Bing search in Dutch, German and English (search date: 16 April 2019). Search terms were: “colon/rectal/colorectal cancer” (DU: “(dikke)darm/anus/endeldarm-kanker”; GER: “Darm-/Mastdarm-krebs)’ + “decision aid” (DU: “keuzehulp”, GER: “Entscheidungshilfe”). We searched the first 100 hits.

Selection criteria

For the academic literature search, studies that were published in peer-reviewed scientific journals between 2006 and 2019 and that were written in English, Dutch or German could be included. Papers that described randomised controlled trials, experiments, the development or evaluation of DAs could be selected.

For both the academic articles and the online search, only tools aimed at supporting the decision-making process of colon, rectal and patients with CRC stage I–III were eligible for selection. Tools targeted only towards metastatic colon, rectal or CRC patients were excluded from the analysis, as were tools aimed at screening decisions for patients with CRC. These tools are focused on inherently different decisions (eg, ‘should I get a screening test’ or ‘which treatments can help with quality of life for my final stages of life’) and therefore require different communication strategies. Appropriate formats for DAs were considered paper-based DAs (booklets or pamphlets), web-based DAs (websites), computer-based DAs (computer programs) and videos. Additionally, DAs had to be freely available, refer to at least two treatments, and written in English, German or Dutch. Nomograms as well as focus groups and question prompt sheets were not included as they cannot be analysed with the assessment instruments we use.

Data extraction

Two reviewers (SH and FC) screened all the retrieved articles and selected eligible articles based on titles and abstracts. After initial screening, disagreements were resolved via discussions with a third reviewer (RV). Full articles were independently assessed using a predefined selection checklist (see online supplemental appendix 2) by two reviewers (SH and FC), and final decisions about inclusion were made jointly with a third reviewer (RV). Inter-rater agreement was substantial between the reviewers (κ=.79). The data extraction forms (online supplemental appendix 3) were filled out independently by two reviewers (SH and FC). Both the selection and the data extraction forms were based on earlier systematic reviews^{12 13} to ensure consistency between outcomes.

Assessment instruments

Two instruments were used to assess the quality of the communication within DAs: the International Patient Decision Aid Standards (IPDAS) and Communicative Aspects (CA) checklist. Five teams of coders, containing two reviewers each, were responsible for the assessment (see online supplemental appendix 4 for a full overview). This way, each DA was reviewed by two coders using both the IPDAS and CA checklist. To calculate inter-rater agreement between team members, we used the Kappa statistic (κ). Although there has been some debate about the assumptions underlying the kappa statistic,^{18 19} we decided to keep the measure as they are well understood and frequently used to compute inter-rater agreement. We have, however, also provided the agreement matrices so other agreement indices may be calculated (see online supplemental appendix 5).

The IPDAS instrument consists of 36 items (see table 1). It was developed by a group of clinical researchers, practitioners and stakeholders²⁰ to ensure that DAs adhere to certain quality standards²¹ and has been validated (for more detailed information on the validation process see: Elwyn et al and the associated website http://ipdas.ohri.ca).^20–22 The instrument is divided into nine key components: information, outcome probabilities, clarifying values, decision guidance, developmental process, using evidence, disclosure and transparency, plain language and evaluation. As the validity of DAs was not assessed academically for all DAs, the evaluation dimension was excluded from analysis. Items could have the values ‘yes’ (1) or ‘no’ (0). Final scores were converted to percentages of the total number of items.

Table 1.

Results from the International Patient Decision Aids Standards (IPDAS) of colorectal cancer patient decision aids

Item	IPDAS dimension	Item description	n	%
1	Information	The DST describes the health condition or problem (intervention, procedure or investigation) for which the index decision is required	17	94
2		The DST described the decision that needs to be considered (the index decision)	18	100
3		The DST described the options available for the index decision	18	100
4		The DST describes the natural course of the health condition or problem, if no action is taken	8	44
5		The DST describes positive features (benefits or advantages) of each option	8	44
6		The DST describes negative features (harms, side effects or disadvantages) of each option	13	72
7		The DST makes it possible to compare the positive and negative features of the available options	3	17
8		The DST shows the negative and positive features of options with equal detail	6	33
9	Outcome probabilities	The DST provides information about outcome probabilities associated with the options (ie, the likely consequences of decisions)	16	89
10		The DST specifies the defined group (reference class) of patients for which the outcome probabilities apply	10	56
11		The DST specifies the event rates for the outcome probabilities	8	44
12		The DST specifies the time period over which the outcome probabilities apply	9	50
13		The DST allows the user to compare outcome probabilities across options using the same denominator and time period	5	28
14		The DST provides information about the levels of uncertainty around event or outcome probabilities	11	61
15		The DST provides more than one way of viewing the probabilities	9	50
16		The DST provides balanced information about event or outcome probabilities to limit framing bias	5	28
17	Clarifying values	The DST describes the features of options to help patients imagine what it is like to experience physical effects	13	72
18		The DST describes the features of options to help patients imagine what it is like to experience the psychological effects	12	67
19		The DST describes the features of options to help patients imagine what it is like to experience social effects	10	56
20		The DST asks patients to think about which positive and negative features of the options matters most to them	8	44
21	Decision guidance	The DST provides a step-by-step way to make a decision	12	67
22		The DST includes tools like worksheets or lists of questions to use when discussing options with a practitioner	11	61
23	Developmental process	The DST (or associated paper) mentions that the development process included finding out what clients or patients need to prepare them to discuss a decision	3	17
24		The DST (or associated paper) mentions that the development process included finding out what health professionals need to prepare them to discuss a specific decision with patients	1	6
25		The DST (or associated paper) mentions that the development process included expert review by clients/patients not involved in producing the DST	6	33
26		The DST (or associated paper) mentions that the development process included expert review by health professionals not involved in producing the DST	11	61
27		The DST (or associated paper) mentions that the DST was field tested with patients who were facing the decision	1	6
28		The DST (or associated paper) mentions that the DST was field tested with practitioners who counsel patients who face the decision	0	0
29	Using evidence	The DST (or associated paper) provides citations to the studies selected	5	28
30		The DST (or associated paper) describes how research evidence was selected or synthesised	2	11
31		The DST (or associated paper) provides a production or publication rate	9	50
32		The DST (or associated paper) provides information about the proposed update policy	9	50
33		The DST (or associated paper) describes the quality of the research evidence used	3	17
34	Disclosure and transparency	The DST (or associated technical documentation) provides information about the funding used for development	12	67
35		The DST includes author/developer credentials or qualifications	15	83
36	Plain language	The DST (or associated paper) reports readability levels (using one or more of the available scales)	3	17

DST, Decision support technology.;

The CA checklist was developed and validated by an interdisciplinary team of communication researchers and medical psychologists^{12 13} to create an in-depth quality assessment of the communicative quality within DAs (see table 2). With ‘communicative quality’ they mean the assessment of whether or not there is evidence that ‘the communicative process in which shared decision-making occurs (Vromans et al, p.2)’ is sufficient. The checklist consists of 76 items and has questions relating to seven main domains (1) information presentation, (2) information control, (3) personalised information, (4) interaction, (5) accessibility, (6) suitability and (7) source of information. Valid responses are ‘yes’ (1) or ‘no’ (0), and final scores are computed in percentages of the total number of items. The total number of items for paper-based DAs was 70 (as not all items were applicable) and 76 for web-based/video-based DAs.

Table 2.

Results from the Communicative Aspects checklist of colorectal cancer patient decision aids (DAs)

Item	IPDAS dimension	Item description	n	%
1	Information presentation	No of DAs that included probabilistic information	18	100
		Methods used to communicative probabilistic information:
2		Verbal
		Absolute risk descriptions	18	100
		Relative risk descriptions	11	61
3		Numerical
		Percentages	6	33
		Natural frequencies	13	72
		Absolute risks	8	44
		Relative risks	3	17
		Absolute risk reduction	0	0
		Relative risk reduction	3	17
		No needed to treat/harm	0	0
4		Visual
		Pie chart	1	6
		Bar chart	2	11
		Line graph	0	0
		Icon array	3	17
		Risk scale	0	0
5		No of DAs that described uncertainties around probabilities	16	89
		Methods used to communicate uncertainties (n=16):
6		Verbal
		Textual descriptions	16	100
7		Numerical
		Numerical range	6	38
8		Visual
		CIs	0	0
		Coloured pictograms	0	0
9		No of DAs that included disease-related information	17	94
		Methods to communicate this information (n=17):
10		Verbal (text)	17	100
11		Visual (illustrations)	12	71
12*		Audiovisual (video clips) (n=3)	2	67
13*		Audio (audio clips) (n=3)	2	67
14		No of DAs that included information about the procedures of treatments	17	94
		Methods used to communicate this information (n=17):
15		Verbal (text)	17	100
16		Visual (illustrations)	10	59
17*		Audiovisual (video clips) (n=3)	2	67
18*		Audio (audio clips) (n=3)	2	67
19		No of DAs that presented the information in a balanced and unbiased way	2	11
		Methods used for balanced and unbiased information:
20		Uses roughly the same amount of text for each option	8	44
21		Displays statistics in the same way for each option (n=13)	3	23
22		Uses similar fonts for each option	17	94
23		Uses language that is not biased in favour of a specific option	9	50
24		Presents equal no of positive features of each option (n=9)	1	11
25		Presents equal no of negative features of each option (n=16)	1	6
26		Keeps the order of positive and negative features constant (n=9)	7	78
27	Information control	The decision aid allows for patients to only receive information that they want to read	2	11
28		The decision aid provides a step-by-step way to move through the decision aid	12	67
29		The decision aid provides the patient the opportunity to read more about a specific topic of interest	12	67
30		The decision aid provides access to external sources	16	89
31		The decision aid provides access to internal sources	3	17
32		The decision aid allows for patients to search for key words	16	89
33*		The decision aid makes it easy for patients to return to previous parts of the decision aid (n=12)	7	39
34	Personalised information	Tailoring in general towards type of treatment	4	22
35		Tailoring in general towards specific populations	0	0
36		Tailoring in general towards specific disease factors	5	28
37		Tailoring in general towards specific stage of disease	4	22
38		Probability tailoring	0	0
39		Content tailoring	1	6
40		Mode of presentation tailoring	1	6
41	Interaction	No of decision aids that help patients to consider personal values and preferences	11	61
		Methods used to consider or assess values and preferences (n=11):
		Passive
42		Recommends patients to think about their values and preferences	10	91
		Asks patients for their personal values and preferences
		Active	7	64
43		Weighting exercises	2	18
44		Sliders to assign values to preferences	1	9
45		No of decision aids that help allow for comparison of positive and negative features of treatment options	4	22
		Methods used to compare positive and negative features of options (n=4):
46		Ranking or rating scale	0	0
47		Table to compare positive and negative features	3	75
48		Verbal comparisons	4	100
49		Discrete choice task	0	0
50		No of decision aids that provide patients the most suitable treatment option	0	0
		Methods used to provide feedback:
51		The decision aid shows the progress of the decision aid	4	22
52		The decision aid provides patients a summary of their values and preferences	1	6
53		The decision aid permits printing as a single document	16	89
54		The decision aid provides space for note taking	9	50
55		The decision aid includes a short knowledge test	2	11
56	Accessibility	The decision aid is freely available on the web	17	94
57		The decision aid requires no login code	18	100
58		The decision aid is not purely computer based	17	94
59		The decision aid requires no access to the internet for its use	17	94
60		The decision aid reports last update	16	89
61		The decision aid reports update frequency	6	33
62		The decision aid requires no staff assistance	17	94
63		The decision aid is self-administered	15	83
64		The decision aid can be used on multiple devices	18	100
65	Suitability	The decision aid contains less than 10 (web) pages	1	6
66*		The decision aid contains videos with a length of less than 1 min (n=)	0	0
67		The decision aid has a conversational (writing) style	13	72
68		The decision aid has irrelevant illustrations	8	44
69	Source of information	No of decision aids that mentioned on which datasets the probabilistic information are based on	1	6
		Types of datasets (n=1)
		Observational data	0	0
		Randomised controlled trials	1	100
		Patient reported outcomes data	0	0
		Data combined from different studies	1	100
		Types of outcome probabilities reported by the decision aid:
70		Mortality rate	5	28
		Survival rate	6	33
71		Incidence rate	4	22
		Progression free survival
72		Treatment side effects	13	72
73		Quality of life	8	44
		Types of information about the data(sets) provided by the decision aid (n=)
74		About what scale the patient data have been collected	0	0
75		About the no of patients on which the data are based on	0	0
		About the characteristics of patients on which the data are based on	0	0
76		About the period of time of data collection	2	11

*This item does not apply to paper-based decision aids.

Note that for both assessment instruments (IPDAS and CA) a higher score does not necessarily mean that the quality of such a DA is higher, it merely indicates that more aspects have been taken into account.

Results

Study selection

Initially, 5645 unique studies were found through the systematic literature search (see figure 1 for a flow chart of the complete study selection). After eligibility checks through abstract and full-text screening, 121 studies were selected. In the end, 18 DAs were identified through the academic literature search (n=1) and online sources (n=17). These numbers compare to earlier systematic reviews using the same assessment instruments.^{12 13}

Figure 1.

Flow chart of the study selection for academic literature and online search.

Additionally, we updated the search between submission and revision. On 16 February 2021, we ran the Google Search again in German, Dutch and English. No new DAs were found in this search in Dutch and English. We did find some updated versions of DAs in German (namely: DA2, DA4 and DA5) but after careful comparison we concluded that no changes were made that impacted the scores of the original DAs so we decided not to replace them. We also ran our academic search in PubMed once more for the period of April 2019–February 2021. This search identified 378 articles, but after title and abstract screening, none of the articles were selected for inclusion. Reasons for exclusion were: ‘no DA discussed’, ‘not a treatment DA for CRC stage I–III’ (eg, a DA about screening decisions or metastatic cancer) and ‘not a DA but a nomogram’.

Table 3 shows a detailed description of the DA characteristics including titles of the DAs, developing organisations, country of origin (AUS/USA/IE/CAN=10, GER=6, NL=2), target audiences, treatments discussed, year of publication, DA format (web=1, video=1, paper/PDF=16) and length of the DA (min=2 pages, max=127 pages).

Table 3.

Characteristics of included decision aids

ID	Title	Organisation	Country	Audience	Year	Treatments	Format	Length
1	Should I have my bowels ‘hooked up’ (anestomosis) when removing my rectal cancer? A decision aid for patients with rectal cancer	The Ottowa Hosipital General Campus, Wu et al (2014; 2016)40 41	CAN	Rectal cancer	2014	APR; (L)AR	PDF	11 pages
2	Patientratgeber Darmkrebs	Bayerische Krebsgesellschaft EV	GER	Colorectal cancer	2011	APR; (L)AR; AM; CA; COL; CT; IT; LNS; RT; TT	PDF	44 pages
3	Patienteninformation Darmkrebs im frühen Stadium	Ärztliches Zentrum für Qualität in der Medizin (ÄZQ)	GER	Early stage CRC	2016	COL; CT; RT	PDF	2 pages
4	Die blauen Ratgeber Darmkrebs. Antworden. Hilfen. Perspektiven	Deutsche Krebshilfe (DK) & Deutshe Krebsgesellschaft (DKG)	GER	Colorectal cancer	2018	APR; (L)AR; AM; COL; CT; IT; LNS; LS; RT	PDF	63 pages
5	Darmkrebs im frühen Stadium. Ein Ratgeber für Patientinnen und Patienten	DKG, DK & AWMF (Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften)	GER	Early stage CRC	2014	COL; CT; LS; RT; WW	PDF	67 pages
6	Darmkrebs	Institut für Qualität und Wirtenschaftlichkeit im Gesundheitswesen (IQWIG)	GER	Colorectal cancer	2018	COL; CT	PDF	13 pages
7	Beratungsordner. Informationen und Dokumentationen herausgegeben Darmzentrum Ortenau	Ortenau Klinikum	GER	Colorectal cancer	?	COL; CT; IT; LS; RT	PDF	89 pages
8	Patienteninformatie. ‘Wait&See’ beleid. Niet opereren na bestraling en chemotherapie voor endeldarmkanker.	Maastricht UMC+	NL	Rectal cancer	2016	COL; WW	PDF	3 pages
9	Keuzehulp darmkanker stadium two hoog risico	Patient+	NL	Colorectal cancer stage II high risk	2017	CT; WW	Web	12 web pages
10	Ottowa Rectal Cancer Decision Aid	The Ottowa Hospital & University of Ottowa	CAN	Rectal cancer	2017	APR; (L)AR; COL	Video	13:09 mins
11	Colorectal Cancer. Information Guide and Personal Record	The Champlain Regional Cancer Programme & The Ottowa Hospital Cancer Programme	CAN	Colorectal cancer	2015	APR; (L)AR; COL; CT; LS; RT; WW	PDF	124 pages
12	A practical guide to understanding cancer. Understanding colon cancer	MacMillan	IE	Colon cancer	2017	APR; (L)AR; COL; CT; LS; LNS; TT	PDF	127 pages
13	Understanding. Cancer of the Colon and Rectum (Bowel). Caring for people with cancer	Irish Cancer Society	IE	Colorectal cancer	2015	COL; CT; RT; TT	PDF	41 pages
14	Understanding Bowel Cancer. A guide for people with cancer, their families and friends	Cancer Council Australia	AUS	Colorectal cancer	2019	APR; (L)AR; COL; CT; LNS; LS; RT	PDF	80 pages
15	Treatment update: Colorectal Cancer	Cancercare	USA	Colorectal cancer	2019	COL; CT; IT; LNS; TT	PDF	24 pages
16	Your guide in the fight	Fight Colorectal Cancer	USA	Colorectal cancer	2017	APR; (L)AR; COL; CT; IT; RT	PDF	112 pages
17	NCCN Guidelines for patients. Colon Cancer	National Comprehensive Cancer Network Foundation (NCCN)	USA	Colon cancer	2018	COL; CT; IT; LNS; RT; TT	PDF	88 pages
18	NCCN Guidelines for patients. Rectal Cancer	National Comprehensive Cancer Network Foundation (NCCN)	USA	Rectal cancer	2018	APR; (L)AR; COL; CT; IT; LNS; RT; TT	PDF	88 pages

AM, complementary medicine Medicine; APR, abdominoperineal resection; CA, cryoablation; COL, colectomy/colostomy; CT, chemotherapy; IT, immunotherapy; (L)AR, (lower) anterior resection; LNS, lymph node surgery; LS, Laparoscopic surgery; RT, radiation therapy; TT, targeted therapy; WW, watch and wait.

The IPDAS results

Inter-rater agreements (κ) between teams ranged from fair to substantial agreement (κ=.32 to κ=.60) for IPDAS. As is visible from table 4, IPDAS scores for individual DAs ranged from 28% to 78% (mean=48%, SD=14.15%, first quartile=38%, third quartile=58%, median=47%). The best performing DA was DA1, which also was the only DA with an associated research paper. Three DAs (DA6, DA11 and DA15) only scored 28%, which means that they met 10 of the 36 IPDAS items. In figure 2, a visualisation of the IPDAS results is shown.

Table 4.

Individual IPDAS item scores per DA

IPDAS item		Decision AID
		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	%
Information	Des. Cond.	·	·	·	·	·	·	·		·	·	·	·	·	·	·	·	·	·	94
	Index dec.	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	100
	Des. Opt.	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	·	100
	Nat. course				·	·	·		·	·		·	·	·						44
	Positive f.	·	·	·	·	·		·	·	·										44
	Negative f.	·	·		·	·		·	·	·	·		·	·	·	·	·			72
	Fair comp.	·							·	·										17
	Equal details	·	·		·	·		·												33
Outcome probabilties	Out. Probs.	·	·	·	·	·		·	·	·	·	·	·	·	·		·	·	·	89
	Ref. class	·	·	·	·	·	·			·	·		·			·				56
	Event rates	·		·		·	·			·	·		·				·			44
	Time period	·		·		·	·			·	·		·	·	·					50
	Same den.	·					·				·					·	·			28
	Uncertainty	·	·	·	·	·			·	·	·						·	·	·	61
	Mult. Meth.	·		·		·			·	·	·		·			·	·			50
	Bal. Info.	·			·	·								·	·					28
Values	Exp. Phys.	·	·			·		·	·	·	·			·	·	·	·	·	·	72
	Exp. Psycho.		·	·	·	·	·	·			·			·	·		·	·	·	67
	Exp. Social		·		·	·		·			·			·	·		·	·	·	56
	Matters most	·				·				·				·	·		·	·	·	44
Dec. Guid.	Step-by-step	·			·	·		·		·		·	·	·	·		·	·	·	67
	Worksh./q’s				·	·		·		·		·	·	·	·		·	·	·	61
Development	Patient needs	·									·	·								17
	Doctor needs											·								6
	Rev. patients	·		·							·			·				·	·	33
	Rev. doctors	·	·	·						·	·	·	·	·			·	·	·	61
	Test. Patients	·																		6
	Test. Doctors																			0
Evidence	Citations	·		·							·			·	·					28
	Sel. Evi.	·								·										11
	Pub. Rate			·					·	·			·	·	·		·	·	·	50
	Update pol.			·			·			·			·	·	·		·	·	·	50
	Qual. Evi.	·		·		·														17
D&T	Funding	·	·	·	·	·					·		·		·	·	·	·	·	67
	Authors/dev.	·	·	·	·	·		·		·	·		·	·	·	·	·	·	·	83
PL	Plain lang.	·								·	·									17
IPDAS Score		28	15	19	17	23	10	13	11	23	21	10	17	21	18	10	20	17	17
%IPDAS Score		78	42	53	47	64	28	36	31	64	58	28	47	58	50	28	56	47	47

DA, decision aid; D&T, Disclosure and transparency; IPDAS, International Patient Decision Aid Standards; PL, Plain language.

Figure 2.

Violin plot of the IPDAS results. IPDAS, International Patient Decision Aid Standards.

All IPDAS items can be found in table 4. In total, there are 18 DAs and 36 IPDAS items. IPDAS scores are the sum of all 36 items per individual DA. The %IPDAS score is the percentage of IPDAS items met per individual DA (max=100).

Information

All DAs (N=18) described the health condition, the index decision and the options available for that decision. However, less than half (n=8, 44%) described the natural course of the disease if no action was taken. Positive features of specific treatment(s) at hand were shown by 8 DAs (44%), whereas 13 DAs (72%) offered negative features of treatment(s). Only three DAs (DA1, DA8 and DA9) (17%) allowed for a fair comparison between treatment options, and six DAs (33%) explained the different treatments with equal detail.

Outcome probabilities

Almost all DAs (n=16, 89%) described the likely consequences of the decisions (the outcome probabilities). More than half (n=11, 61%) explained uncertainty around probabilities. Additionally, 50% of DAs provided the reference class, used multiple methods to view the probabilities and specified the time period over which the outcome probabilities applied. Eight DAs (44%) discussed event rates, and only five DAs (28%) provided the outcomes probabilities in a balanced way and used the same denominator for the outcome probabilities.

Clarifying values

About 70% of DAs clarified to patients what it is like to experience the physical (72%) and psychological (67%) consequences of certain treatments. The social consequences were explained in 56% of DAs, and even fewer DAs (44%) expressed that patients had to think about what positive or negative features of the decisions matters most to them.

Decision guidance

Decision guidance was provided by leading patients in a step-by-step way through the decision (67% of DAs) and/or providing a list of questions to ask their clinician (61% of DAs).

Developmental process

Although 61% of DAs reported that the DA was reviewed by clinicians, only 33% mentioned the review involvement of patients in this process. Only three DAs (17%) mentioned that patients were asked about their needs for the DA, and one DA (DA10) mentioned that clinicians were asked about their needs for the DA. Similarly, only one DA mentioned that it was tested with patients (DA1), and none of the DAs mentioned that they were tested with doctors.

Using evidence

Half of the DAs provided a publication rate and an update policy for the DA. Only 28% of DAs (n=5) provided the reader with citations of the evidence used in the DA. Less than 20% of DAs reported about the quality of the evidence used (17%) and how the evidence was selected (11%).

Disclosure and transparency

More than 80% of DAs (83%) provided information about the authors and developers. About 70% (67%) also provided information about the funding related to the DA.

Plain language

Only 17% of DAs (DA1, DA9 and DA10) reported reading levels related to plain language.

The CA results

Inter-rater agreement (κ) for the CA checklist ranged from fair to substantial (κ=.38 to κ=.79). Results for the CA checklist ranged from 28% to 58% (mean=41%, SD=6.2%, first quartile=38%, third quartile=43%, median=41%). The DA that scores highest on the CA scale was DA9, while DA6 was the lowest on this scale. Table 2 shows an overview of the CA results, in figure 3, these results are visualised.

Figure 3.

Violin plot of the CA results. CA, communicative aspects.

Information presentation

All DAs provided probabilistic information. As for the methods used to express them, all DAs reported verbal statistics (eg, ‘It is likely that you experience nausea’ or ‘most people have side effects’). Of the DAs that also included numerical probabilities (n=15, 83%), most reported natural frequencies (72%) (eg, ‘1 in 10 people …’). About one-third reported percentages (33%) (eg, ‘70% of the population …’). Absolute risks (eg, ‘The chance of recurrence of 60%, with chemotherapy this is 40%’) were given in 44% of DAs. Relative risks and relative risk reductions (eg, ‘compared with chemo, it is five times as likely to …’) were given in 17% of DAs. Five DAs provided the information visually (6% used a pie chart, 11% had a bar chart and 17% showed icon arrays).

Most DAs (n=16) provided information about the uncertainty around the information. All of the DAs that communicated uncertainty did this verbally, and 6 DAs also showed a numerical range (eg, ‘1 or 2 out of 10’ or ‘10%–20% of people’).

All but one DA (DA8) provided disease related information (eg, explain what (colorectal) cancer is), and 71% of DAs also included visuals to do so. Additionally, all but one DA (DA6) included information about the procedure of treatments discussed, and 59% also used visuals to explain this. There were two non-paper based DAs (DA9 and DA10), of which 1 offered audio and audiovisual stimuli to explain disease related information and procedures. There was also one paper-based DA that offered this (by providing web links) (DA12).

Almost all DAs (94%) used consistent fonts throughout the DA. Half of the DAs used unbiased language, and about two-fifth of DAs (44%) used roughly the same amount of text for each treatment option. Seventy-eight percent of DAs kept the order of positive and negative features of the treatments consistent. Only one DA that mentioned positive features of treatments (n=9) showed these with equal detail (DA8). Similarly, for the DAs that mentioned negative features of treatment options (n=16), the same DA (DA8) discussed them equally. Overall, 78% of DAs that mentioned both negative and positive options (n=9) kept the order in which they discussed these consistent.

Information control

Most DAs provided access to external sources (n=16). Also, most DAs (n=16) allowed patients to search for key words (as CTRL+F is always an option in PDF). Two-thirds of DAs (67%) provided patients with a step-by-step way through the DA and gave patients the opportunity to read more about specific topics. Less than half of the DAs (n=7) made it easy to return to previous sections of the DA (eg, by providing clickable links to earlier content or providing a contents ruler on each page). Three DAs provided access to internal sources (eg, ‘read/learn more’ sections) and two DAs (DA9 and DA14) provided patients with the option to only receive information that they would want to have (eg, by making it easy to skip sections).

Personalised information

In general, DAs contained few options to personalise information. Only five DAs (28%) were tailored towards specific disease factors and four DAs (22%) had tailored information for specific stages of the disease or the type of treatment(s) patients were eligible for. There was one DA (DA7) that made it possible to tailor the content and one DA (DA12) provided the option to change the mode of presentation (eg, by providing the same content in audio, video and text).

Interaction

More than half of the DAs (61%) mentioned that patients needed to assess their own personal values and preferences for the different treatment options. Of the DAs that offered this assistance (n=11), almost all did so in a passive way by either recommending patients to think about their personal preferences (n=10, 91%) and/or by asking patients for their preferences (64%, n=7). There were also 2 DAs that provided active interactions by giving weighting exercises (DA1 and DA9) and sliders (DA9).

There were four DAs (22%) that allowed for the comparison of negative and positive treatment options in an active way by verbally comparing the options (n=4) or providing a table with negative and positive features (n=3, 75%).

Four DAs (22%) showed the progress of the DA, whereas one DA (6%) provided a summary of the values and preferences of patients (DA9).

All the paper-based DAs (n=16) could be printed as one document, half of the DAs provided space for note taking and two DAs (11%) provided the patient with a short knowledge quiz (DA1 and DA9).

Accessibility

No DA required a login code and all DAs could be used on multiple devices. Almost all DAs were freely available on the web, not purely computer based and did not require internet access or staff assistance (n=17, 94%). Most DAs reported the last update (89%). Finally, 83% (n=15) of DAs were self-administered. However, only 33% of DAs (n=6) reported the update frequency.

Suitability

Although 72% of DAs (n=13) had a conversational style, up to 44% (n=8) contained irrelevant illustrations (eg, showing random people without providing any context). Also, almost all DAs were lengthy (lengthy >10 pages/5 min; n=15; min: 2; max; 127; M=58 pages), the video DA was 13:09 min (DA10)).

Source of information

There was one DA that reported on which dataset(s) the probabilistic information was based (DA10), the rest of the DAs did not report this. The most reported statistic was treatment side effects (n=13, 72%), followed by quality of life information (n=8, 44%), survival rate (n=6, 33%), mortality rate (n=5, 28%) and incidence (n=4, 22%). None of the DAs mentioned at what scale patient data have been collected, the number of patients the data are based on or the patient characteristics of the evidence used. Only two DAs (11%) mention the time period of the data collection.

Discussion

Our systematic review of 18 patient CRC DAs shows that the communicative quality of these DAs varies substantially between individual DAs.

Our results are in line with previous systematic reviews on CRC DAs in general¹⁵ and CRC Decision Support Systems for stage IV¹⁶ as both conclude that evidence for the quality of CRC DAs is too limited to recommend their use in clinical practice today. Additionally, conclusions can be drawn for the quality of communication in DAs between prostate,¹² breast¹³ and CRC, as all reviews indicate that there are substantial differences in the communicative quality between individual DAs and overall quality seems to be low.

Strengths of this systematic review include the wide scope of our search, but also the in-depth analysis on the kind of information given in DAs for CRC. Our analysis showed that in most CRC DAs, probabilities are only communicated verbally. This is problematic, as research shows that people have a hard time interpreting verbally communicated statistical information^23–26 such as ‘there is a big chance of …’. Additionally, information seems to be generic and lengthy in CRC DAs, whereas providing patients with personalised health information is recommended²⁷ as this reduces the information overload patients may experience.²⁸ Especially since many CRC patients have low health literacy skills,¹⁴ it seems crucial that information is (also) visualised^{29 30} and communicated in plain language.²⁷ However, our analysis shows that this is often not the case. Finally, as in previous systematic reviews on treatment DAs,^{28 31} we found that many do not provide citations for the evidence used and they often seem to rely on anecdotal evidence instead.

We conjecture that many of these issues can be addressed using Natural Language Generation,³² an AI technique which automatically converts data into fluent and coherent text (possibly combined with automatically generated pictures), tailored to individual readers. A recent example harnessing these techniques for personalised DAs is a prototype decision support tool that generates personalised probabilities for effects on quality of life after chemotehrapy.³³ In short, the support tool relies on the PROFILES³⁴ registry data set, consisting of over 21 000 patients with cancer within the Netherlands Cancer Registry. With latent class analysis,³⁵ the tool can predict which outcome scenario is most applicable for a new patient based on individual prognosis data and the PROFILES data set. This way, patients can view symptom-related quality of life outcomes such as the probability of becoming nauseous, but also social or financial implications of chemotherapy. We are currently evaluating the tool with different patients to see how we can communicate the different outcomes in a personal and accessible way.

There are also several limitations to this study. It should be noted that our review did not take measures of the effectiveness of DAs, such as decisional conflict or participation in shared decision-making, into account as this was not within the aims of our study. It should also be noted that although IPDAs and CA can be used to guide the design process of DA developers, using these tools does not ensure (communicative) quality. We, therefore, stress that DAs should also always be evaluated with clinical experts and patients.³⁶ Finally, as we included several countries within our review, results appear to apply to all different countries. However, it seems to be the case that plain language use was harder to establish for the German DAs which might be because of the formal sentence structures in German. Additionally, it seemed that German patients were less encouraged to participate in shared decision making (‘listen closely to your doctor’) then, for example, American patients (‘make decisions you want to make’). Although it has been demonstrated that culture might impact the effectiveness of health communication between doctors and patients of different cultural background,^{37 38} studying cultural differences between (European) countries remains challenging as theories and methods for assessing differences vary between countries.³⁹ Future reviews could look into systematic differences between DAs from different countries more to see if shared decision making is a globally agreed on goal.

Conclusion

This review is—to the best of our knowledge—the first to perform a large-scale analysis of the quality of communication in treatment CRC patient DAs. The findings highlight the variety of communicative quality in DAs and the lack of support that many DAs are able to provide to both patients and clinicians in shared decision making in a clinical setting. It calls for personalising information in CRC treatment DAs in order to facilitate patient participation in shared decision making. To ensure this, both the IPDAS instrument and CA checklist can be useful tools to guide DA developers in such a way that they are made aware of certain aspects and can take them into account. Future research should focus on evaluation of such personalised tools to test their usefulness in the clinical practice.

Data availability statement

Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. All reviewed materials were made freely available on the internet by their developers. A full list of the DAs and their titles can be found in table 3. Individual assessments of each reviewer are stored in a secure data repository at Tilburg University Dataverse and is available on request by contacting the main author.