Do choosing wisely tools meet criteria for patient decision aids? A descriptive analysis of patient materials

Abstract

Objectives

Choosing Wisely is a remarkable physician-led campaign to reduce unnecessary or harmful health services. Some of the literature identifies Choosing Wisely as a shared decision-making approach. We evaluated the patient materials developed by Choosing Wisely Canada to determine whether they meet the criteria for shared decision-making tools known as patient decision aids.

Design

Descriptive analysis of all Choosing Wisely Canada patient materials.

Data source

In May 2015, we selected all Choosing Wisely Canada patient materials from its official website.

Main outcomes and measures

Four team members independently extracted characteristics of the English materials using the International Patient Decision Aid Standards (IPDAS) modified 16-item minimum criteria for qualifying and certifying patient decision aids. The research team discussed discrepancies between data extractors and reached a consensus. Descriptive analysis was conducted.

Results

Of the 24 patient materials assessed, 12 were about treatments, 11 were about screening and 1 was about prevention. The median score for patient materials using IPDAS criteria was 10/16 (range: 8–11) for screening topics and 6/12 (range: 6–9) for prevention and treatment topics. Commonly missed criteria were stating the decision (21/24 did not), providing balanced information on option benefits/harms (24/24 did not), citing evidence (24/24 did not) and updating policy (24/24 did not). Out of 24 patient materials, only 2 met the 6 IPDAS criteria to qualify as patient decision aids, and neither of these 2 met the 6 certifying criteria.

Conclusions

Patient materials developed by Choosing Wisely Canada do not meet the IPDAS minimal qualifying or certifying criteria for patient decision aids. Modifications to the Choosing Wisely Canada patient materials would help to ensure that they qualify as patient decision aids and thus as more effective shared decision-making tools.

Strengths and limitations of this study.

• This study assessed patient materials developed by Choosing Wisely Canada using the modified 16-item minimum criteria for qualifying and certifying patient decision aids developed by International Patient Decision Aid Standards (IPDAS), the most credible and internationally recognised measure for evaluating patient decision aids.

• Characteristics of patient materials were extracted independently by four reviewers to reduce risk of bias; only the minimum IPDAS criteria were applied and ambiguous IPDAS wordings were generously interpreted to the benefit of the assessed patient materials.

• To the best of our knowledge, this study is among the first to evaluate these patient materials. Therefore, our results, while reproducible, cannot be compared to other studies.

• We did not contact the authors for supplementary information or validation of our evaluation.

Introduction

Choosing Wisely is a remarkable physician-led campaign to reduce the provision of unnecessary or harmful services in healthcare and help physicians and patients discuss the necessity of such tests and treatments.¹ Numerous studies have shown that certain healthcare services can be useless or potentially harmful, such as cardiac screening for asymptomatic adults.^2–4 Choosing Wisely addressed this problem by asking specialist societies to generate a list of the most prevalent low-value services in their field. Patient materials were then developed to help them and their patients engage in conversations regarding the necessity of such tests and treatments.⁵ More recently, the Choosing Wisely campaign has spread worldwide, including to Canada, where the patient materials have been slightly modified to fit the Canadian context (eg, omitting service costs). A recently published study performed in the USA on the impact of the campaign indicated that out of seven low-value services analysed, changes showed a desirable decrease for two recommendations (imaging for headache and cardiac imaging for low-risk patients) at the population level.⁶

Some healthcare professionals identify Choosing Wisely as a shared decision-making campaign^{5 7 8} and use the patient materials as patient decision aids. In shared decision-making, in addition to the clinician sharing information about the risks and benefits of treatment or about screening alternatives with the patient, the patient in turn shares with the clinician all relevant personal information that might tilt the balance in favour of one option over another. The patient also expresses what role he or she wishes to play in the decision-making process, following which both parties come to a mutual health-related decision.⁹ The decision is not only based on the best available evidence but is also congruent with the patient's values and preferences.¹⁰

Patient decision aids are tools that facilitate shared decision-making.^11–15 They are aimed at helping patients and their physicians to discuss options in a clear and unbiased way. Especially in the grey areas of medicine, they are intended to help patients clarify personal values and preferences regarding the benefits and harms of each option, guide them in deliberation and communication, and help them make decisions that are in keeping with their values and preferences.¹¹ A Cochrane review of 115 randomised controlled trials of patient decision aids showed improved knowledge, values-choice agreement and patient participation in decisions about treatment and screening.¹¹ Regarding cost savings, one of the aims of the Choosing Wisely campaign, there is evidence to show that patients who are better informed opt for more conservative treatment options,¹¹ and that decision support interventions do not increase spending,¹⁶ but not enough to suggest that patient decision support interventions lead to system-wide cost savings.^{16 17}

As the patient-friendly materials developed by Choosing Wisely Canada were designed to help physicians discuss treatment and screening options with patients, our objective was to determine whether these patient materials meet International Patient Decision Aid Standards (IPDAS) minimal criteria for patient decision aids¹⁸ and whether they can support shared decision-making.

Methods

Study design, data sources and search strategies

We conducted a descriptive analysis of all Choosing Wisely Canada patient materials available from their official website on 26 May 2015. We used one source of data: the patient materials section of Choosing Wisely Canada's official website (http://www.choosingwiselycanada.org/materials/). Patient materials were downloaded in PDF format and printed.

Data selection

We included all patient materials that were developed by the campaign and that were available on Choosing Wisely Canada's official website on the specified date. We excluded duplicates of patient materials (eg, summaries of available patient materials). Materials were available in English and French, but as they were equivalent in content, only the English versions were assessed.

Data extraction

Four reviewers independently extracted characteristics of patient materials using the modified 16-item minimum criteria for qualifying and certifying patient decision aids. The IPDAS is a checklist of criteria and a tool that payers, patients, practitioners, developers, and researchers can use to assess patient decision aids on criteria related to content, process of development and effectiveness.^{19 20} The IPDAS checklist was rigorously developed in a two-stage Delphi process in 2005 using online ratings that enabled extensive international collaboration. A total of 122 individuals from four stakeholder groups (researchers, practitioners, patients and policymakers) representing 14 countries reviewed evidence and rated the importance of 80 criteria in 12 quality dimensions. The checklist was continuously updated and improved over the succeeding 10 years.²¹ We are unaware of the existence of any other tool for quantitative measurement of the quality of decision support tools.¹⁸ To qualify as a patient decision aid, a tool must (1) explicitly state the decision being considered; (2) provide evidence-based information about a health condition, the options, associated benefits, harms, probabilities and scientific uncertainties; and (3) help patients to recognise the value-sensitive and preference-sensitive nature of the decision, and to clarify, either implicitly or explicitly, the value they place on the benefits, harms and scientific uncertainties. According to IPDAS, there are two ways to clarify values: an implicit approach, such as a description of what it is like to experience the consequences of the options, and an explicit approach, such as asking the patient which positive and negative features of the options matter most to them.¹¹

IPDAS has now developed a modified 16-item minimum criteria checklist for qualifying and certifying patient decision aids.²² Although the original IPDAS checklist contains 44 items, patient materials need only to meet 16 minimum criteria to be considered patient decision aids: 6 qualifying criteria and, to minimise harmful bias, 10 certification criteria (6 items for treatment decisions and 4 more for screening decisions).²³ The remaining 28 items provide added information about the quality of the patient decision aid.²² We also extracted the following descriptive characteristics: (1) number of pages; (2) type of health service (screening, prevention or treatment); and (3) clinical context (chronic disease, acute disease or no specified disease) and medical specialty.²⁴ A screening test was defined as any testing procedure designed to detect early evidence of disease; prevention was defined as any procedure or precaution taken by healthy people to reduce the risk of contracting an illness or developing a disease/condition.

Data synthesis and analysis

We performed a descriptive analysis of the data using frequency counts of criteria that were met for each patient material using the IPDAS modified 16-item minimum standard criteria (IPDASi V.4.0).²² Discrepancies were resolved through discussion among third-party team members. Of the 16 minimum criteria, 7 were found to be vague in their wording and were therefore interpreted following a consensus discussion among the four raters:

• 1.1.2—‘The DA explicitly states the decision that needs to be considered’ was interpreted as a non-directive statement identifying a preference-sensitive decision. For example, the statement ‘Antibiotics for urinary tract infections in older people: When you need them—and when you don't’ was rated as ‘No’ as it is implicitly directive. For a ‘yes’ rating, the statement would have required non-directive presentation of the options. Removal of the long dash would suffice, as in ‘When you need it and when you don't’. The long dash gives added emphasis to the second option, or ‘pops the point’, as in a blurb for a TV cop show: ‘Was he an honest man—or was he a thief?’

• 1.1.4 and 1.1.5—‘DA describes the positive features (benefits/advantages) of each option’ and ‘DA describes the negative features (harms, side effects, or disadvantages) of each option’. This was interpreted more broadly as describing the positive and negative features of the main option presented, instead of the positive and negative features of each option.

• 1.2.1—‘The document describes what it is like to experience the consequences of the options (eg, physical, psychological, social)’ was interpreted more broadly as clarifying values in terms of what matters most to the patient (ie, implicit or explicit value clarification). We determined that value clarification was explicit when the patient materials explicitly asked the patient about their values and explicitly stated the physical, psychological and social values related to each option. We determined that value clarification was implicit when the patient materials did not ask the patient which values were most important to them but did list some consequences of the options, thus making an attempt to illustrate what values could be involved in such a decision.

• 2.2.4—The item ‘Provides information about uncertainty around event or outcome probabilities’ was understood to mean that the patient material contains information that includes qualitative probabilities, such as ‘they can increase the risk of death’, or ‘in some cases, dialysis will not relieve your symptoms’; or quantitative probabilities, such as ‘about one in four people who take antibiotics will have side effects’.

• 2.3.1—The item ‘DA (or associated documentation) provides information about the funding source used for development’ was rated with ‘yes’ if there was any information about funding, without requiring details about whether funding was used for the development of the patient materials.

• 2.2.1—The item ‘provides citations to the evidence selected’ was rated with a ‘yes’ if the materials cited primary sources of evidence. References to websites were not considered citations to evidence as they are not primary sources.

Results

Patient material selection

Of the 25 patient materials available on Choosing Wisely Canada's official website, 24 met the eligibility criteria for inclusion in the study. One was not eligible because it was a summary of five other patient materials that we assessed individually (figure 1).

Figure 1.

Flow chart of patient material selection. IPDAS, International Patient Decision Aid Standards.

Characteristics of patient materials

Of the 24 patient materials assessed, 12 were for treatment services,^25–36 11 were for screening services^37–47 and 1 was for prevention services.⁴⁸ Of the 24 patient materials assessed, 16 related to a chronic disease,^{25–32 34 35 38–43} 4 related to an acute illness^{33 36 47 48} and 4 did not relate to any specific disease (table 1).^{37 44–46}

Table 1.

Characteristics of patient materials

Short title	Objective	Number of pages	Clinical context	Medical specialty
Prevention
Preventing infections48	Reduce the unnecessary use of urinary catheters and ulcer drugs	2	Acute disease	Internal medicine
Screening topics
Check-ups37	Reduce the unnecessary use of annual health check-ups	2	No disease specified	Family medicine
Lower back pain38	Reduce the unnecessary use of imaging tests for lower back pain	2	Chronic disease	Physical therapy and rehabilitation
Bone-density tests39	Reduce the unnecessary use of bone-density tests	2	Chronic disease	Orthopaediology
Colonoscopy40	Reduce the unnecessary use of colonoscopy	2	Chronic disease	Oncology
Electrocardiogram41	Reduce the unnecessary use of electrocardiogram	2	Chronic disease	Cardiology
Pap tests42	Reduce the unnecessary use of pap tests	2	Chronic disease	Oncology
Low-risk prostate cancer43	Reduce the unnecessary use of treatments such as surgery and radiation and to increase the use of ‘active surveillance’ for people with low-risk prostate cancer	2	Chronic disease	Oncology
Heart tests before surgery44	Reduce the unnecessary use of heart tests before surgery	2	No disease specified	Cardiology
Laboratory tests before surgery45	Reduce the unnecessary use of laboratory tests before surgery	2	No disease specified	Medical biochemistry
Vitamin D tests46	Reduce the unnecessary use of vitamin D tests	2	No disease specified	Medical biochemistry
Imaging tests for headaches47	Reduce the unnecessary use of imaging tests for headaches	2	Acute disease	Neurology
Treatment topics
Chronic kidney disease25	Reduce the unnecessary use of anaemia drugs and dialysis for people with chronic kidney disease	3	Chronic disease	Nephrology
Feeding tubes26	Reduce the unnecessary use of feeding tubes for people with Alzheimer's disease	2	Chronic disease	Gastroenterology
Dementia27	Reduce the unnecessary use of antipsychotics for treating disruptive behaviour in people with dementia	2	Chronic disease	Psychiatry
End of life28	Reduce the unnecessary use of treatment for patients with advanced cancer and to promote end-of-life care	2	Chronic disease	Oncology
GERD29	Reduce the unnecessary use of PPI for treating heart burn and GERD	2	Chronic disease	Gastroenterology
Osteoarthritis30	Reduce the unnecessary use of popular supplements such as glucosamine and chondroitin sulfate for treating osteoarthritis of the knee	2	Chronic disease	Rheumatology
Pain medicines31	Reduce the unnecessary use of NSAIDs to relieve pain for people with heart problems or kidney disease	2	Chronic disease	Family medicine
Palliative care32	Increase the use of palliative care for people with a serious illness	2	Chronic disease	Multiple specialities
Sinusitis33	Reduce the unnecessary use of antibiotics for treating sinusitis	2	Acute disease	Family medicine
Insomnia and anxiety34	Reduce the unnecessary use of sleeping pills for treating insomnia and anxiety in older people	2	Chronic disease	Psychiatry
ED35	Reduce the unnecessary use of testosterone treatment for erection problems	2	Chronic disease	Urology
UTIs36	Reduce the unnecessary use of antibiotics to treat UTIs	2	Acute disease	Urology

ED, erectile dysfunction; GERD, gastro-oesophageal reflux; NSAID, non-steroidal anti-inflammatory drug; PPI, proton-pump inhibitor; UTI, urinary tract infection.

IPDAS minimum criteria results

The median score for patient materials using the minimum standard criteria for certifying patient decision aids was 10/16 (range: 8–11) for patient materials related to screening and 6/12 (range: 6–9) for patient materials related to prevention and treatment. Two patient materials (treatment for chronic kidney and feeding tubes for people with Alzheimer's disease) met all six qualifying criteria. No patient materials met all 10 certifying criteria. The criteria that were least fulfilled were (1) making the decision explicit (n=3); (2) citing evidence (n=0); (3) updates (n=0); and (4) balanced presentation of information on options, benefits and harms (n=0) (table 2).

Table 2.

Appraisal of Choosing Wisely Canada patient materials (n=24) using IPDASi V.4.0

Item		Check-ups37	Preventing infections48	Lower back pain38	Bone-density tests39	Colonoscopy40	Electrocardiogram41	Pap tests42	Low-risk prostate cancer43	Heart tests before surgery44	Laboratory tests before surgery45	Vitamin D tests46	Chronic kidney disease25	Feeding tubes26	Dementia27	End of life28	Imaging tests for headaches47	GERD29	Osteoarthritis30	Pain medicines31	Palliative care32	Sinusitis33	Insomnia and anxiety34	ED35	UTIs36	Total
Qualifying criteria
1.1.1	Describes health condition or problem	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	24
1.1.2	Explicitly states the decision that needs to be considered								√				√	√												3
1.1.3	Describes the options available	√		√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√		√	√	√	√	22
1.1.4	Describes the positive features (benefits/advantages) of each option		√	√	√	√	√	√	√	√	√	√	√	√		√				√	√	√	√		√	18
1.1.5	Describes the negative features (harms, side effects, or disadvantages) of each option	√	√	√	√	√	√	√		√	√	√	√	√	√		√	√	√	√		√	√	√	√	21
1.2.1	Describes what it is like to experience the consequences of the options (physical, psychological, social)					√			*√				√	√	√	√					*√	√	√	√	√	11
Certification criteria
2.1.1	Shows the negative and positive features of options in equal detail																									0
2.2.1	Provides citations to the evidence selected†																									0
2.2.2	Provides a publication date	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	24
2.2.3	Provides an update policy																									0
2.2.4	Provides information about the levels of uncertainty around event or outcome probabilities	√	√	√	√	√	√	√	√	√	√		√	√	√		√	√	√		√	√	√		√	20
2.3.1	Provides information about the funding source used for development	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	√	24
2.4.1	Describes what the test is designed to measure	√	N/A		√	√	√	√	√	√	√	√	N/A	N/A	N/A	N/A	√	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	10
2.4.2	Describes the next steps typically taken if the test detects the condition	√	N/A		√	√	√	√	√	√	√	√	N/A	N/A	N/A	N/A	√	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	10
2.4.3	Describes the next steps if the condition is not detected		N/A		√	√							N/A	N/A	N/A	N/A		N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	2
2.4.4	Has information about the consequences of detecting the condition that would never have occurred if screening had not been done (lead time bias)	√	N/A	√			√	√		√	√	√	N/A	N/A	N/A	N/A	√	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	8
		9	6	8	10	11	10	10	10	10	10	9	9	9	7	6	9	6	6	6	6	8	8	6	8

*Included explicit preferences and values clarification.

†Or associated information.

ED, erectile dysfunction; GERD, gastro-oesophageal reflux; IPDASi, modified International Patient Decision Aid Standards; N/A, not applicable; UTI, urinary tract infection.

Only two of the patient materials met the criterion for explicitly clarifying values and preferences because they explicitly asked the patient to think about their personal values and preferences, and to talk about them with their family and/or their doctor.

Discussion

We evaluated 24 patient materials developed by Choosing Wisely Canada to determine whether they met the standards for patient decision aids by assessing whether they fulfil IPDAS minimum criteria and, consequently, whether they can support shared decision-making. The median score for patient materials using the IPDAS minimum standard criteria for certifying patient decision aids was 10/16 (range: 8–11) for patient materials related to screening, and 6/12 (range: 6–9) for patient materials related to prevention and treatment. Two patient materials met all qualifying criteria and none of the patient materials met all certifying criteria. Our results lead us to make three main observations.

First, none of the patient materials presented the negative and positive features of each option in equal detail, provided references to evidence or mentioned an update policy. In other words, none presented balanced information about the options that patients needed to consider. One of the most important features of shared decision-making is to present the negative and positive features of the available options in a balanced manner.^{18 49} We observed that these patient materials often promoted one positive or negative feature prominently. Indeed, the strong declarative tone against certain tests would make clear which option the physician favours. This makes it difficult to describe the materials as shared decision-making tools. The physician's favoured option may shift the patient towards a specific choice that may not be in line with their own preferences or values. Where tests or treatments are demonstrably useless or harmful, it is clear that they should not be offered to patients at all, rather than the healthcare professional or professional society simply discouraging patients from choosing them. In this case, the campaign should not be directed solely at patients, but also at healthcare systems, specialist societies and other organisations that produce clinical practice guidelines (CPGs). If the system is still offering ‘unnecessary’ options, these options must be considered on an equal footing with other options and discussed with the patient in a neutral fashion through a shared decision-making process, as what the physician may consider ‘low value’ might not be considered low value by the patient. In addition, patient materials provided no references to the evidence in order to verify the information presented. This may be of growing importance as an increasing number of patients are using the internet to inform their healthcare choices.⁵⁰ They are exposed to much information from questionable sources and these tools may not always be accurate or objective.

Second, less than half of the patient materials reviewed provided an opportunity to clarify patient values and preferences. Nine patient materials described what it is like to experience the consequences of the options, but only two provided for explicit value clarification. In most cases, there is no single best choice of treatment or screening options, because the evidence about their usefulness is equivocal. Among 3000 treatments appraised, Clinical Evidence classified 50% as having insufficient evidence, 24% as likely to be beneficial, 7% as requiring ‘trade-offs between benefits and harms’, 5% as unlikely to be beneficial, 3% as likely to be ineffective or harmful and only 11% as being clearly beneficial.⁵¹ Thus for the 97% that are not clearly ineffective or harmful, patient decision aids are needed not only to help patients consider the evidence (even for the 11% of treatments that show a clear benefit), but also to clarify the values they place on the benefits, harms and scientific uncertainties. Value-sensitive and preference-sensitive decisions can affect patient-reported outcomes that matter, such as decisional conflict and decision regret.^52–54 This, in turn, can lead to many unwanted consequences such as lack of adherence to the decision made.^{11 55–57} In addition, many simple healthcare decisions lead to subsequent healthcare decisions, each with different consequences, complexities and value-laden choices.⁵⁸ For example, a positive screening test may lead to additional diagnostic testing (eg, prenatal screening for Down syndrome). Yet out of 11 patient materials related to screening tests, only 2 described the next steps if the condition or problem is not detected: for example, the patient materials about colonoscopy suggested it was necessary every 5 years if a condition was detected, and every 10 years if not; while materials relating to cervical screening made no recommendations, if screening was negative. It is essential for clinicians to present a balanced account of the benefits and consequences of each available option in order to engage patients in a truly informed and shared decision-making process.

Finally, while the Choosing Wisely campaign has been considered a shared decision-making approach,^{5 7 8} we observed that its patient materials do not fulfil the minimum IPDAS criteria for patient decision aids.²² Not all shared decision-making (SDM) tools should be rejected because they do not score 16/16 on the qualifying and certifying criteria of the IPDAS scale. Also other SDM tools that do not describe themselves as decision aids have been shown to increase patient engagement in shared decision-making.^{11 59} The Choosing Wisely Canada materials are patient-oriented tools that may help patients and physicians to ‘engage in conversations regarding the necessity’ of tests and treatments, but in their current form, they do not provide enough information or take patient values and preferences into consideration sufficiently to facilitate shared decision-making with physicians.^{11 18 60} With some adjustment, these materials could enable patients to participate meaningfully in decisions about their healthcare and to achieve evidence-based and value congruent decisions. We propose that the best way to do so is to modify the existing materials to meet IPDAS criteria.

Our study has some limitations. We relied only on materials found on the website. We did not contact the authors for supplementary information or validation of our evaluation. As such, we were unable to judge the quality of the evidence underlying the patient materials. Our objective was to provide an informed, transparent and evidence-based assessment of the Choosing Wisely Canada patient materials based on the IPDAS criteria. However, the IPDAS criteria have been in use for 10 years, and ambiguity in the wording of some criteria could have resulted in differing interpretations that would have changed the results of the evaluation. For example, we evaluated the patient materials with respect to the main option, but identifying the main option is somewhat subjective. For the patient material on low-risk prostate cancer, we decided that the main option was active surveillance, but it could also be treatment for low-risk prostate cancer. However, in order to account for the possible differences in interpretation, four reviewers independently evaluated all patient materials before reaching consensus under the mentorship of an IPDAS co-lead (DS). In order to account for differences, we were as generous as possible in our evaluation to ensure that we were conservative in our analysis.

Conclusion

Choosing Wisely Canada is an initiative whose remarkable goal is to reduce overuse of unnecessary or harmful treatments. However, the patient materials presented unbalanced information and most did not provide an opportunity to clarify patient values and preferences. Therefore, the Choosing Wisely Canada patient materials do not meet the criteria for patient decision aids, and thus lack important components necessary to shared decision-making. By adapting the materials in such a way as to provide balanced information on options, cite primary evidence sources and help patients clarify their values and preferences, the Choosing Wisely Canada patient materials could facilitate shared decision-making involving patients and physicians. Finally, our results suggest a need for further clarification and education about shared decision-making tools such as patient decision aids among the many medical and surgical specialty societies that have engaged in this campaign, as well as a need for a certification process.⁶¹ This could bring about the implementation of effective shared decision-making in routine clinical practice and ensure that patients make informed value-based decisions.⁶²