Decision Aids: the Effect of Labeling Options on Patient choices and Decision-Making

Abstract

Background

Conscious and unconscious biases can influence how people interpret new information and make decisions. Current standards for creating decision aids, however, do not address this issue.

Method

Using a 2×2 factorial design we sent surveys to members of two Internet survey panels containing a decision scenario involving a choice between aspirin or a statin drug to lower risk of heart attack and a decision aid. Each aid presented identical information about reduction in heart attack risk and likelihood of a major side effect. They differed in whether the options were labeled and amount of decisional guidance: information only (a balance sheet) versus information plus values clarification (a multicriteria decision analysis). After using the decision aid, participants indicated their preferred medication. Those using a multicriteria decision aid also judged differences in the comparative outcome data provided for the two options and the relative importance of achieving benefits versus avoiding risks in making the decision.

Results

The study sample size is 536. Participants using decision aids with unlabeled options were more likely to choose a statin: 56% versus 25%, p < 0.001. The type of decision aid made no difference. This effect persists after adjusting for differences in survey company, age, gender, education level, health literacy, and numeracy. Participants using unlabeled decision aids were also more likely to interpret the data presented as favoring a statin with regard to both treatment benefits and risk of side effects (p ≤ 0.01). There were no significant differences in decision priorities (p=0.21).

Conclusion

Identifying the options in patient decision aids can influence patient preferences and change how they interpret comparative outcome data.

Introduction

Conscious and unconscious biases can influence how people interpret new information and make decisions. [1–3] A well-known example is the differences in how people assess items when they are identified versus when they are not in research studies. For this reason blinding is recommended to increase the validity of research findings. [4, 5]

The goal of patient decision aids is to improve decision quality which has been defined as the extent to which patients accurately comprehend pertinent information about the decision options and their expected outcomes and receive treatments consistent with their goals and preferences. [6] The majority of patient decision aids currently available identify the options under consideration. [7] Although this practice could influence their effectiveness by triggering cognitive biases, its impact on patient decision making is unknown. Current standards for creating decision aids recommend steps to minimize framing bias but they do not address this issue. [8, 9]

Patient decision aids can provide different amounts of decisional guidance. Simple decision aids such as balance sheets and option grids present comparative information about decision options but provide no guidance for how to judge which is most preferred. [10, 11] More complex decision aids both present information and guide the patient through a series of values clarification exercises that can include focused comparisons regarding the pros and cons of the decision options and assessment of decision priorities. Current standards for patient decision aids recommend inclusion of values clarification exercises. However, poor uptake of decision aids in routine practice settings has sparked renewed interest in simpler information-based aids that may be easier to implement. [12, 13] It is currently unknown whether labeling options in a decision aid differentially affects patient decisions made using simple versus more complex decision aids.

The goal of this study was to address these gaps in our knowledge. We conducted a controlled trial designed to meet four study aims. The first was to confirm previous findings that patients make different choices when using a decision aid that contains labeled options, i.e., identified by name, than when using an identical decision aid with unlabeled options. The second aim was to extend these findings by determining if labeling options affects patient choices for both simple and more complex decision aids. The third and fourth aims were to explore possible explanations for how labeling options affect patient decisions by examining whether patients evaluate comparative outcome data differently or alter their decision priorities when using decision aids with labeled versus unlabeled options.

Methods

Participants

Study participants consisted of members of Internet survey panels who responded to requests made through two Internet survey companies: SurveyMonkey and Fluidsurveys. [14, 15] All surveys were targeted to obtain a demographically representative sample of the general US population aged 18 and over. Because the surveys were written in English, all respondents had to be able to complete an English-language survey. There were no other exclusion criteria.

Study intervention

The study intervention consisted of a series of surveys containing a decision scenario involving a choice between two medications, a decision aid, and questions about preferred medication.

The decision scenario asked participants to imagine they have 12% chance of having a heart attack in the next 10 years and to choose between two medication options capable of decreasing this risk: aspirin or a statin. Aspirin, acetylsalicylic acid, is an analgesic and antipyretic medication that also lowers heart attack risk by inhibiting platelet aggregation. Statins, HMG-CoA reductase inhibitors, are a class of drugs that have been shown to lower risks of heart attacks by lowering cholesterol and preventing thrombus formation.

The study used a 2×2 factorial design to investigate the effects of two factors: option labeling and whether the decision aid provided information only or information plus decisional guidance. The resulting four decision aids differed in whether the medication options were identified as aspirin or Medication A and a statin or Medication B and in the format of the decision aid: either a balance sheet or an Analytic Hierarchy Process (AHP) multi-criteria decision analysis that provided both information and decisional guidance. Figure one shows examples of these two formats.

Figure 1.

Blinded decision aid example. The upper panel shows the balance sheet decision aid. The AHP-based decision aid included all of the elements shown in the figure. The balance sheet was used to present the comparative outcome information. Pairwise comparison judgments regarding differences between the options’ benefits and risks were entered using the interactive sliders shown in the middle of the figure. Sliders were also used to compare the priorities of benefits versus risks in making the decision. Results of the analysis are summarized in the chart and table shown below. The unblinded decision aids were identical except that aspirin was denoted as Medication A and statin was denoted as Medication B.

The balance sheet consisted of a table showing the expected benefit of each option in terms of lowering risk of heart attack and a single side effect. [11] The aspirin side effect was described as “severe internal bleeding requiring hospitalization and/or blood transfusion”. The statin side effect was described as “side effects severe enough to stop taking the medicine, mostly muscle aches and pains”. The data presented regarding reduction in heart attack risk and frequencies of side effects were chosen to be consistent with the current literature. [16–20]

The AHP-based decision aid consisted of the same balance sheet combined with a values clarification exercise that asked the participants to explicitly compare the benefits and risks of the two options and the relative importance of achieving treatment benefits versus avoiding treatment risks in choosing a preferred option. These comparisons were made using a 9 point scale ranging from −4 to +4 and converted into preference scores for the alternatives and priority weights for decision criteria using the Analytic Hierarchy Process. [21–23] These scores and weights were then combined to derive overall scores for the two decision alternatives that were presented at the bottom of the decision aid in the form of a bar graph and accompanying table.

After the reviewing the decision aid, all participants were asked to indicate their preferred medication. They also completed the one question subjective health literacy measure and the subjective numeracy scale, validated measures of health literacy and numeracy respectively. [24, 25] We considered respondents who indicated they were either extremely or quite a bit confident when filling out forms by themselves as having adequate health literacy.

Data collection

Each of the four survey versions was sent to both the SurveyMonkey and Fluidsurvey Internet panels resulting in 8 surveys altogether. Because the survey results from both Internet panels were similar in all cases, we combined them for analysis.

Analysis

We summarized the data using standard descriptive statistics. To assess the impact of labeling options on patient choices of preferred outcome and whether this effect was influenced by the amount of decisional guidance provided by decision aid, we used Fisher’s exact test to compare differences in preferred option between groups using decision aids with labeled versus unlabeled decision options with and without controlling for the type of decision aid: balance sheet or AHP.

To explore mechanisms underlying differences in patient choices related to option labeling, we used the same analytic strategy for the groups who used an AHP-based decision aid to compare pairwise comparison judgments regarding decision priorities and the options’ benefits and risks made with decision aids containing labeled versus unlabeled options.

For both sets of analyses, we used logistic regression to adjust for possible imbalances among our study groups due to survey company, age, gender, education level, health literacy, and numeracy. To facilitate the interpretation of these analyses, we combined categorical response categories for education and literacy.

We performed all statistical analyses using JMP 10.0 and defined a statistically significant result as p ≤ 0.05. This study was approved by Research Subjects Review Board of the University of Rochester.

Results

The characteristics of the study population are summarized in Table 1. The total study sample size is 536, 62% female, with a mean of 50.7 years. Eighty two percent have education beyond the high school level, 78% report adequate health literacy skills, and the mean subjective numeracy level on a 6 point scale is 4.64, indicating that, as a group, study participants had moderately high to high numeracy skills.

Table 1.

Characteristics of the study population

	AHP unlabeled N = 128	AHP labeled N = 129	Table unlabeled N = 139	Table labeled N = 140	Overall n = 536	P value
Mean age (95% CI), years	47.5 (44.4 to 50.6)	49.7 (47.0 to 52.5)	46.4 (43.9 to 49.0)	58.9 (31.2 to 86.6)	50.7 (43.5 to 57.9)	0.11
missing	3	5	0		12
Gender
female	74 (59%)	80 (65%)	86 (62%)	83 (61%)	323 (62%)	0.82
male	52 (41%)	44 (35%)	53 (38%)	53 (39 %)	202 (38%)
missing	2	5	0	4	11
Education level
less than high school	4 (3%)	1 (1%)	0	2 (1%)	7 (1%)	0.16
high school	22 (14%)	20 (16%)	31 (22%)	16 (12%)	89 (17%)
some college*	31 (25%)	38 (31%)	45 (32%)	43 (32%)	157 (30%)
college	40 (32%)	44 (35%)	42 (30%)	52 (38%)	178 (34%)
post-graduate	29 (23%)	21 (17%)	21 (15%)	23 (17%)	94 (18%)
Missing	2	5	0	4	11
Subjective health literacy†
extremely	48 (38%)	63 (49%)	62 (45%)	62 (44%)	235 (44%)	0.42
quite a bit	51 (40%)	43 (33%)	46 (33%)	44 (31%)	184 (34%)
somewhat	15 (12%)	11 (9%)	17 (12%)	19 (16%)	62 (12%)
a little bit	14 (11%)	11 (9%)	10 (7%)	12 (9%)	47 (9%)
Not at all	0	1 (1%)	4 (3%)	3 (2%)	8 (1%)
Subjective Numeracy Score, Mean (95% CI)	4.69 (4.51 to 4.86)	4.69 (4.5 to 4.9)	4.70 (4.51 to 4.9)	4.49 (4.31 to 4.67)	4.64 (4.5 to 4.7)	0.21
missing	1	4	0	0	5

^*category includes associate’s degree

^†responses to the question “How confident are you filling out medical forms by yourself”

The effects of labeling the decision options are illustrated in Figure 2. Participants using decision aids with unlabeled options were more than twice as likely to choose a statin as those using decision aids with labeled options: 56% versus 25%, p < 0.001. The type of decision aid used – the balance sheet or the AHP – made no difference. The strong effect of labeling on decision choice persists after adjusting the analysis for possible differences among the study groups. The adjusted odds of choosing a statin were 3.9 times higher in participants using an unlabeled than a labeled decision aid, p < 0.001, 95% confidence interval 2.6 to 5.5. The only other factor affecting medication choices was subjective numeracy. For every one point increase in SNS score, the odds of choosing Medication B or Statins were 0.8 times higher, p = 0.004, 95% confidence interval 0.6 to 0.92.

Figure 2.

Preferred medications. This figure illustrates the overall differences in preferred medications between decision aids with labeled versus unlabeled options and differences within each type of decision aid. All differences are statistically significant, p < 0.01.

The effects of labeling on pairwise comparisons between the options’ benefits and risks and the relative priorities of lowering heart attack risk versus avoiding medication side effects in making a decision are summarized in Table 2. Participants using unlabeled decision aids were more likely to interpret the data presented as favoring statins with regard to both treatment benefits (p < 0.01) and risk of side effects (p < 0.01). Labeling the treatment options did not have a statistically significant effect on how participants judged the relative importance of benefits versus risks in making a treatment decision between the two alternatives (p = 0.21)

Table 2.

Effect of blinding on pairwise comparison judgments.

	Number*	Mean†	Std Dev	Lower 95%	Upper 95%	P value
Benefits
Unlabeled	122	0.47	2.33	0.05	0.89	p = 0.01
Labeled	114	−0.20	2.09	−0.59	0.19
Side effects
Unlabeled	122	0.47	2.55	0.01	0.92	p < 0.0001
Labeled	114	−1.12	2.10	−1.51	−0.73
Criteria priorities
Unlabeled	122	0.34	2.15	−0.04	0.73	p = 0.21
Labeled	114	−0.04	2.31	−0.47	0.38

^*Data only include study participants who used an AHP-based decision aid.

^†Judgments were made using a 9 point scale ranging from −4 to 4. For the comparisons between the alternatives benefits and side effects, −4 indicated that Medication A or aspirin was strongly preferred, 4 indicated that Medication B or Statins was strongly preferred and 0 indicated both options were equal. For the criteria priorities, negative numbers indicated that benefits were considered more important and positive numbers that avoiding side effects was more important.

Discussion

These results demonstrate that identifying the decision options in a patient decision aid can greatly influence which option patients prefer regardless of whether the aid provided explicit decisional guidance in addition to structured information. They also suggest that this effect is more likely to be due to differences in how patients compare data regarding the options’ expected outcomes than altering how they make trade-offs between competing decision objectives.

One explanation for the differences in preferred options between labeled and unlabeled decision aids is that knowing the options allows patients to incorporate considerations not included in the decision aid into their deliberations. These additional considerations could improve the quality of the decision by bringing important information about an option not specified in the decision aid into the decision making process. For example, a patient may recognize that one of the options is a medication they have been told to avoid because it interacts with a drug they are already taking. However, identifying options in a decision aid could also trigger cognitive biases that, if acted on, would decrease the quality of the decision and diminish the effectiveness of the decision aid. For example, a patient may mistakenly believe that a treatment is risk-free and therefore discount outcome data showing a clear association with documented adverse effects.

This finding is consistent with previous studies that have examined the impact of labeled versus unlabeled options on patient decisions. In a trial comparing three different types of decision aids to help men make prostate cancer screening decisions, study participants using aids that specifically identified the PSA screening test were 3.7 to 1.8 times more likely to prefer screening than participants shown and an unlabeled, but equivalent, screening test option. [26] Similarly, in a discrete choice experiment, study participants making trade-offs between labeled colorectal cancer screening test options were more likely to make choices based on a single attribute than those shown unlabeled test options suggesting they were making greater use of external information. [27] Moreover, these findings are not unique to patients. Labeling has also been shown to impact how trained researchers interpret information. [28]

Current standards for assessing the quality of patient decision aids recommend that they include both information comparing the characteristics of decision options and guidance in making trade-offs between their advantages and disadvantages. [8] Concerns about poor uptake of conventionally formatted patient decision aids in practice settings have led to the development of simplified decision aids suitable for use within the context of a doctor patient consultation. [12, 13] This development raises the question of the relative merits of decision aids that provide information plus decision support versus simpler ones that provide information alone. The former approach typically uses a decompositional strategy that breaks a complicated decision down into a series of smaller, simpler considerations that are easier work with, whereas the latter relies on normal conversational and intuitive decision making processes that consider the decision holistically. Studies comparing decomposed and holistic approaches to decision making have found that decomposed approaches consistently result in better decisions. [29, 30] These findings suggest that the addition of decision guidance to a decision aid, as currently recommended, should increase its effectiveness in promoting high quality clinical decisions. We therefore hypothesized that labeled options would have less effect on preferences generated using a decision aid that provided information plus decision guidance than preferences generated based on information alone. However, we found that the type of decision aid made no difference.

The analysis of the differences in how study participants compared the benefits and risks of the labeled and unlabeled options provides one possible explanatory mechanism for the effects of option labeling on patient decisions. These results suggest that the study subjects were either systematically altering how they interpreted the comparative outcome information to fit pre-existing mental models, responding to emotions evoked by the labels, or a combination of these factors. [31, 32] Another possible explanation that is particularly germane in this context is that our patients may have been more familiar with aspirin than statin drugs. Research on cognitive processing fluency suggests that unfamiliarity with an item will alter how people perceive it and, in this case, result in a higher perceived risk of side effects for the less familiar option. [33–35]

The only patient characteristic that attenuated the effects of unblinded decision options was subjective numeracy. Participants with higher levels of subjective numeracy were less likely to prefer different options in the unlabeled versus labeled study groups. This finding is consistent with research showing that people who score higher on an objective test of numeracy skills are less susceptible to common decision making biases. [36, 37] It also extends these findings to suggest that high subjective numeracy, consisting of perceived numeracy skill and preference for working with numeric information, has a similar effect. [25]

This study is subject to several limitations. Our study participants were working with a standardized, imaginary decision scenario rather than facing a real decision. This study design allowed us to control the information presented and thereby better address the study aims but could affect how accurately the results reflect the effect sizes that exist in real-world decision tasks. The effects of labeling on choice of treatment may also have been exaggerated because we used a simple decision model that could have excluded considerations that would normally be included in a patient decision aid. Finally, the options we used are likely to be familiar to many people. We were therefore unable to determine the effect of labeling less well-known medical interventions.

The goal of a patient decision aid is to help patients become informed about decisions they face and make better choices among the options available. Despite its limitations, our study, along with the existing body of literature regarding the potential negative effects of cognitive biases on judgment and decision making, suggests that identifying the options included in patient decision aids can mitigate their effectiveness. Of particular concern is the biasing effect of labeling on the interpretation of data regarding risks and benefits.

Although preliminary, our findings demonstrate that labeling options in a decision aid can decrease its effectiveness in improving decision quality by impairing its ability to help patients interpret pertinent comparative outcome data consistently. They warrant additional research to identify the extent to which option labeling affects decision aids addressing more complicated decisions involving less familiar alternatives, particularly in real world settings. Confirmation would make the development of methods to reduce the negative impact of option labeling in decision aids a high priority for shared decision making research. One strategy would be to adopt a two-stage format in which an unlabeled assessment of options precedes a labeled one. Such a strategy would allow an unbiased assessment of options to be incorporated into a patient’s background experience that could potentially minimize decision biases while still addressing the needs of real-world decision makers and retain the possible beneficial impact of background knowledge. We have successfully used blinded options in a study of patient decision making regarding colorectal cancer screening options. [38] However, more information is needed regarding the effectiveness and feasibility of this approach before it could be recommended for routine use in patient decision aids.