Abstract
Background. Presenting numeric data alone may result in patients underappreciating clinically significant benefits. Contextualizing statements to counter this may raise concern about absence of neutrality. These issues arose during construction of a decision aid for sacubitril-valsartan, a heart failure medication associated with a ∼3% absolute reduction in 2-year mortality that carries high out-of-pocket cost. A contextualizing statement framing this as a “pretty big benefit” was incorporated. The impact of statements like this within decision aids is unknown. Objective. This online Qualtrics survey sought to deepen understanding of benefit framing by testing the impact of varying contextualizing statements within a decision aid for sacubitril-valsartan. Design. Participants were randomly assigned to receive one of six abbreviated versions of a decision aid for sacubitril-valsartan that varied only by contextualizing statement (ranging from strongly neutral to strongly positive and using relative and absolute risk reductions). Participants were asked to answer questions regarding the likelihood of taking the medication at a cost of $50/month and their perception of the drug’s benefits. Results. A total of 1873 participants who were demographically similar to the heart failure population completed the survey. Fifty-four percent were willing to take sacubitril-valsartan at $50/month. Each of the five experimental contextualizing statements was compared with the baseline version; no significant differences were observed in reported likelihood of taking sacubitril-valsartan. After controlling for demographics and covariates, group assignment did not predict likelihood of taking the medication. Higher income, better self-reported health status, and younger age were associated with increased likelihood of taking sacubitril-valsartan. Limitations. This study used a hypothetical scenario and evaluated one method of delivering contextualizing statements. Conclusions. Contextualizing statements as tested within this decision aid did not affect decision making.
Keywords: decision aid, heart failure, framing
Introduction
Effective shared medical decisions integrate medical evidence with patients’ preferences, values, and circumstances. Engaging patients in this process requires communicating information regarding relevant benefits and placing this information in the appropriate context. The process can go wrong if patients recall medical facts but fail to grasp their proper meaning or implications. 1 It is also well-known that framing of medical information can profoundly influence patient perceptions and ultimate decisions. 2 The use of survival-based versus death-based framing, absolute versus relative risks, and different time frames have all been shown to affect patients’ assessments of particular clinical benefits.3–5 Other contextualizing or labelling statements—such as labeling results as normal or abnormal—have also been shown to affect decisions and interpretation of quantitative information.6–8
There has been debate in the context of shared decision making as to whether ideal benefit communication involves providing patients with numerical data only or numerical data accompanied by contextualizing language. To provide patients with numerical benefit information only and allow them to interpret this information on their own may be seen as more neutral and less biased, but this approach may not optimize comprehension or provide appropriate context for patients.5,9,10 In contrast, providing contextualizing statements along with numerical data may enhance their ability to use the information in shared decisions but may exert too strong an influence and bias their interpretations.11,12 Although there is some support for adding textual contextualization to numerical data within decision aids to enhance understanding of probabilistic information, the impact of contextualizing language on patients’ decisions is in many instances still unknown. 13 It has not been studied whether varying forms of contextualization influence how patients make tradeoffs involving cost and benefit, for example.
Sacubitril-valsartan is a guideline-directed medical therapy for heart failure and provides an ideal case in which to study the impact of contextualizing statements. This drug has demonstrated significant reductions in morbidity and mortality compared with generic alternatives (e.g., angiotensin-converting enzyme [ACE] inhibitors and angiotensin receptor blockers).14,15 Despite its benefits, however, the higher out-of-pocket cost associated with sacubitril-valsartan can make this decision preference-sensitive for many patients.16,17 A decision aid was created for sacubitril-valsartan presenting its benefits and risks and highlighting cost as an important consideration. 18 During development of the decision aid, clinicians expressed concern that patients would undervalue the drug—even if they correctly understood the numeric benefit—if presented only with a pictographic demonstration of a 3% absolute risk reduction of mortality over 2 years. To address these concerns and communicate the perspective of many clinicians, the decision aid incorporated a contextualizing statement describing this as a “pretty big benefit” compared with many other treatments. 19 This study was designed to examine how changes in benefit framing and contextualization affect patient decisions by testing the impact of various contextualizing statements within this decision aid on individuals’ likelihood of taking sacubitril-valsartan and perceptions of the drug’s benefits.
Methods
Design
We conducted an online survey during July 2020. Participants were randomly assigned to receive one of six versions of a contextualizing statement within an abbreviated version of the currently available DA for sacubitril-valsartan (Table 1). The contextualizing statement included in the actual decision aid for sacubitril-valsartan was considered the baseline (“Compared to other medicines, this is actually a pretty big benefit”). The experimental contextualizing statements differed in several domains, including the strength of the medical recommendation and the inclusion of relative versus absolute risk information. Participants were instructed to imagine themselves as patients diagnosed with heart failure who were making medication decisions with their doctor. The study was determined to be exempt from review by the Emory University IRB Institutional Review Board.
Table 1.
The Six Versions of the Contextualizing Statement Included in the Decision Aid
| Version | Statement |
|---|---|
| #1: Baseline | “Compared to other medicines, this is actually a pretty big benefit.” |
| #2: Re-statement of Absolute Risk Reduction + Baseline | “This means 3 (out of 100) fewer patients died after two years on Entresto. Compared to other medicines, this is actually a pretty big benefit.” |
| #3: Re-statement of Absolute Risk Reduction-Alone | “This means 3 (out of 100) fewer patients died after two years on Entresto.” |
| #4: Re-statement of Absolute Risk Reduction + Physician Recommendation | “This means 3 (out of 100) fewer patients died after two years on Entresto. Most doctors consider this to be a very significant improvement.” |
| #5: Re-statement of Absolute Risk Reduction + Neutral Patient Preference Statement | “This means 3 (out of 100) fewer patients died after two years on Entresto. You can decide whether this is a significant benefit for you.” |
| #6: Re-statement of Relative Risk Reduction + Baseline | “This means 15% fewer patients died after two years on Entresto. Compared to other medicines, this is actually a pretty big benefit.” |
Setting and Participants
The survey was conducted by Qualtrics, which aggregates multiple online panels to recruit respondents and uses a quota-based sampling strategy. Participants were drawn from a general public sample, but demographic quotas for age (10% 30-44; 30% 45-64; 60% >65), race/ethnicity (30% African American, 51% White, 14% Hispanic, 4% Asian, 0.5% American Indian/Alaska Native), and sex (50% female) were chosen to reflect the heart failure patient population, with oversampling of African Americans because African Americans bear disproportionate levels of mortality and morbidity related to cardiovascular disease.20,21
Panelists received payment from Qualtrics. No identifiable information was collected.
Survey Development
The survey (Appendix 1) was developed by the investigators and presented to participants with an abbreviated version of the currently available DA focusing on benefits of sacubitril-valsartan (Appendix 2). The baseline DA displayed a pictographic representation of the drug’s absolute mortality risk reduction of 3% over a 2-year period with a contextualizing statement immediately adjacent to the figure containing the language, “Compared to other medicines, this is actually a pretty big benefit.” For this study, five experimental contextualizing statements were created. The statements varied in their presentation of sacubitril-valsartan’s benefit data, employing differing combinations of textual restatement of absolute versus relative mortality risk reduction and/or adding a qualifying sentence (which varied in valence).
All five experimental statements included a textual restatement of the numerical risk reduction data presented in the pictographic. Four of the five statements presented the absolute risk reduction associated with sacubitril-valsartan, while one used relative risk reduction in order to test the documented phenomenon of relative risks being interpreted as more significant. In order to evaluate the impact of restating the absolute risk, one statement added re-statement of absolute risk reduction to the baseline statement, while another presented absolute risk reduction alone without the baseline statement. Finally, in order to examine the impact of the strength or valence of the contextualizing statement, one statement presented absolute risk plus a strong positive valence (“Most doctors consider this to be a very significant improvement.”), while another used a qualifying sentence with a strongly neutral valence (“You can decide whether this is a significant benefit for you.”). We intentionally tested experimental statements that many may consider to run counter to IPDAS guideline recommendations to avoid bias. 22
Each participant was randomized to receive the DA with one of six different contextualizing statements (baseline or an experimental statement) placed next to the pictograph (Table 1). All other content was identical. The survey used the trade name of sacubitril-valsartan (Entresto) and a single ACE inhibitor (Enalapril) to avoid acronyms and increase readability for participants. The survey was pretested using Amazon Mechanical Turk and CloudResearch to assess comprehension and to solicit feedback. 23 Questions were edited to minimize misunderstanding, and further pretesting was conducted to ensure adequacy of administration and accuracy of quota-based sampling.
Survey Administration
Self-reported demographic data were obtained to facilitate quota-based sampling. Participants who met demographic requirements were randomly assigned to receive one of the six contextualizing statements within the DA followed by the rest of the survey. As an attention check, participants were asked to identify the disease treated by these medicines. To ensure quality, we excluded participants who incorrectly answered the attention check question, 24 completed the survey in less than 1.75 minutes (one half of the median response time), or did not complete the entire survey.
Outcome and Measurements
The primary outcome was self-reported likelihood of taking sacubitril-valsartan at a price of $50 per month (the approximate median out-of-pocket cost for Americans covered by Medicare Part D who are taking this drug), 16 as assessed by the question,
Based on what you read, if your insurance covers Entresto and you would need to pay $50 per month for it (versus no co-pay for Enalapril), how likely would you be to take Entresto if your doctor recommended it?
Participants were asked to respond using a 4-point Likert-type scale in which 1 indicated very likely and 4 indicated very unlikely. Participants were able to refer back to the decision aid with the data display while answering this question.
Secondary outcome measures included perception of the relative value of sacubitril-valsartan over ACE inhibitors, decisional conflict, and comprehension of the benefit of sacubitril-valsartan versus ACE inhibitors. Decisional conflict was assessed using the Uncertainty Subscale of the Decisional Conflict Scale.25,26 Comprehension of benefit was assessed using a single open-ended question in which participants were asked to identify and provide the absolute risk reduction cited in the decision aid.
Additional potential predictor measures were assessed, including age, gender, race, educational attainment, and income. Participants were also asked about their financial situation, self-reported health status, and the number of medications they take daily. Finally, participants’ approach to medical decision making was assessed using a 6-point Likert-type scale in which 1 was labelled “Lean toward waiting and seeing” and 6 was labelled “Lean toward taking action” (with lower scores indicating being a “medical minimizer” and higher scores being a “medical maximizer”).27,28
Statistical Analysis
The primary, prespecified analysis included five pairwise chi-square tests to determine whether stated likelihood of taking sacubitril-valsartan was different between those participants presented with the original contextualizing statement and those presented with each of the five experimental contextualizing statements. 29 A sample size of 1800 participants was chosen to provide 80% power to detect a 10% difference in any of these comparisons (adjusted for multiple comparisons using Bonferroni correction). Likelihood of taking sacubitril-valsartan was dichotomized into “likely” (1–2) and “unlikely” (3–4) for this analysis.
To test the null hypothesis of no impact of contextualizing statements on decisions, Bayes factors were calculated for each of the pairwise comparisons.30–32 A Bayes factor was calculated for each pairwise comparison using the R package “BayesFactor,” which is a ratio of the likelihood probability of the alternative hypothesis to the likelihood probability of the null hypothesis. Under the null hypothesis, the same multinomial probabilities of taking sacubitril-valsartan were assumed between groups. A lower Bayes factor indicates a higher degree of evidence for the null hypothesis.
In addition, the impact of each contextualizing statement was examined using a non-dichotomized primary outcome (an ordinal variable with four levels) to assess for more subtle effects on likelihood of taking sacubitril-valsartan using ordinal logistic regression. Chi-square tests and ANOVA were conducted to assess the impact of contextualizing statements and participant characteristics on participants’ attitudes and understanding about sacubitril-valsartan. Logistic regression models were constructed to adjust for key demographic characteristics (age, race, sex, income, and education) and other covariates (self-rated health status and approach to medical decision making) hypothesized to affect decision making.
Finally, we compared the impact of contextualizing statement on the likelihood of taking sacubitril-valsartan within quartiles of time to completion in order to assess whether there were time-dependent effects present for the primary outcome. All analyses were conducted using SAS 9.4 (Cary, NC) and R 4.0.3. P values less than 0.05 were considered as statistically significant.
Results
Participants
The survey completion rate was 61.5% (1873 of 3048 randomized participants; Figure 1); 258 survey responses were excluded due to a failed attention check and 917 due to incomplete data. As specified in the quota-based sampling approach, participants reflected the age and sex distribution of the national heart failure population, with planned overrepresentation of African Americans (59.9% 65 years or older; 30.1% African American, 50.9% White; Table 2). Educational attainment among participants was relatively high, with 48.9% of participants having a bachelor’s degree or higher, compared with 36% of the US population 25 and older. 33 Income was well-distributed. Just under 20% reported fair or poor health status, and 48% reported taking three or more medications daily. All participant characteristics were well-balanced across the six groups.
Figure 1.
Participant enrollment and exclusion.
Table 2.
Participant Characteristics
| Characteristic | Overall (n = 1873), n (%) | DA #1—Baseline (n = 303), n (%) | DA #2—ARR + Baseline (n = 315), n (%) | DA #3—ARR Alone (n = 308), n (%) | DA #4—ARR + Doctor Recommendation (n = 318), n (%) | DA #5—ARR + Neutral Statement (n = 313), n (%) | DA #6—RRR + Baseline (n = 316), n (%) | P Value |
|---|---|---|---|---|---|---|---|---|
| Age a | 0.9990 | |||||||
| 30–44 years | 186 (9.9) | 30 (9.9) | 29 (9.2) | 31 (10.1) | 29 (9.1) | 32 (10.2) | 35 (11.1) | |
| 45–64 years | 566 (30.2) | 88 (29.0) | 95 (30.2) | 97 (31.5) | 98 (30.8) | 93 (29.7) | 95 (30.1) | |
| 65+ years | 1121 (59.9) | 185 (61.06) | 191 (60.6) | 180 (58.4) | 191 (60.1) | 188 (60.1) | 186 (58.9) | |
| Gender | 0.4330 b | |||||||
| Female | 942 (50.3) | 140 (46.2) | 171 (54.3) | 160 (52.0) | 157 (49.4) | 159 (50.8) | 155 (49.1) | |
| Male | 923 (49.3) | 162 (53.5) | 143 (45.4) | 147 (47.7) | 160 (50.3) | 150 (47.9) | 161 (50.9) | |
| Prefer not to answer | 8 (0.4) | 1 (0.3) | 1 (0.3) | 1 (0.3) | 1 (0.3) | 4 (1.3) | 0 | |
| Race a | 0.9694 b | |||||||
| American Indian or Alaska Native | 11 (0.6) | 3 (1.0) | 2 (0.6) | 3 (1.0) | 0 | 1 (0.3) | 2 (0.6) | |
| Asian | 75 (4.0) | 9 (3.0) | 17 (5.4) | 15 (4.9) | 10 (3.1) | 11 (3.51) | 13 (4.1) | |
| Black or African American | 564 (30.11) | 96 (31.68) | 88 (27.9) | 90 (29.2) | 100 (31.5) | 94 (30.0) | 96 (30.4) | |
| Hispanic/Latino(a) | 260 (13.9) | 40 (13.2) | 46 (14.6) | 41 (13.3) | 42 (13.2) | 41 (13.1) | 50 (15.8) | |
| Middle Eastern or North African/Mediterranean | 1 (0.1) | 0 | 1 (0.3) | 0 | 0 | 0 | 0 | |
| Native Hawaiian, or Pacific Islander | 2 (0.1) | 0 | 0 | 0 | 2 (0.6) | 0 | 0 | |
| White | 953 (50.9) | 155 (51.2) | 159 (50.5) | 159 (51.6) | 163 (51.3) | 163 (52.1) | 154 (48.7) | |
| More than one race | 6 (0.3) | 0 | 2 (0.63) | 0 | 1 (0.3) | 3 (1.0) | 0 | |
| Other | 1 (0.1) | 0 | 0 | 0 | 0 | 0 | 1 (0.3) | |
| Education | 0.5891 | |||||||
| Less than high school | 26 (1.4) | 10 (3.3) | 3 (1.0) | 4 (1.3) | 3 (1.0) | 4 (1.3) | 2 (0.6) | |
| High school graduate or GED | 249 (13.3) | 37 (12.2) | 38 (12.1) | 46 (14.9) | 45 (14.2) | 38 (12.1) | 45 (14.2) | |
| Some college (associates, trade school, etc.) | 682 (36.4) | 107 (35.3) | 130 (41.3) | 110 (35.7) | 111 (34.9) | 109 (34.8) | 115 (36.4) | |
| Bachelor’s degree | 566 (30.2) | 87 (28.7) | 91 (28.9) | 93 (30.2) | 94 (29.6) | 102 (32.6) | 99 (31.3) | |
| Master’s, professional, or doctoral degree | 350 (18.7) | 62 (20.5) | 53 (16.8) | 55 (17.9) | 65 (20.4) | 60 (19.2) | 55 (17.4) | |
| Income | 0.4649 | |||||||
| Less than $25,000 | 314 (16.8) | 47 (15.5) | 66 (21.0) | 43 (14.0) | 53 (16.7) | 46 (14.7) | 59 (18.7) | |
| $25,000 to $50,000 | 499 (26.6) | 76 (25.1) | 72 (22.9) | 93 (30.2) | 80 (25.2) | 100 (32.0) | 78 (24.7) | |
| $50,000 to $75,000 | 398 (21.3) | 63 (20.8) | 61 (19.4) | 63 (20.5) | 80 (25.2) | 58 (18.5) | 73 (23.1) | |
| $75,000 to $100,000 | 262 (14.0) | 41 (13.5) | 54 (17.1) | 38 (12.3) | 43 (13.5) | 48 (15.3) | 38 (12.0) | |
| $100,000 to $175,000 | 237 (12.7) | 47 (15.5) | 35 (11.1) | 42 (13.6) | 35 (11.0) | 35 (11.2) | 43 (13.6) | |
| $175,000 or greater | 93 (5.0) | 16 (5.3) | 17 (5.4) | 16 (5.2) | 18 (5.7) | 14 (4.5) | 12 (3.8) | |
| Prefer not to answer | 70 (3.7) | 13 (4.3) | 10 (3.2) | 13 (4.2) | 9 (2.8) | 12 (3.8) | 13 (4.1) | |
| How often does this statement apply to you: I have money left over at the end of the month. | 0.0949 | |||||||
| Always | 548 (29.3) | 82 (27.1) | 99 (31.4) | 77 (25.0) | 89 (28.0) | 93 (29.7) | 108 (34.2) | |
| Often | 401 (21.4) | 67 (22.1) | 62 (19.7) | 72 (23.4) | 68 (21.4) | 60 (19.2) | 72 (22.8) | |
| Sometimes | 482 (25.7) | 87 (28.7) | 67 (21.3) | 77 (25.0) | 91 (28.6) | 79 (25.2) | 81 (25.6) | |
| Rarely | 315 (16.8) | 42 (13.9) | 58 (18.4) | 61 (19.8) | 51 (16.0) | 64 (20.5) | 39 (12.3) | |
| Never | 127 (6.8) | 25 (8.3) | 29 (9.2) | 21 (6.8) | 19 (6.0) | 17 (5.4) | 16 (5.1) | |
| In situations where it’s not clear, do you tend to lean toward taking action or do you lean toward waiting and seeing if action is needed. (scale 1–6) | ||||||||
| Mean (SD) | 3.6 (1.5) | 3.7 (1.5) | 3.5 (1.5) | 3.7 (1.5) | 3.6 (1.5) | 3.7 (1.4) | 3.6 (1.6) | 0.8367 |
| In general, would you say your health is | 0.4251 | |||||||
| Excellent | 144 (7.7) | 24 (7.9) | 20 (6.4) | 21 (6.8) | 19 (6.0) | 26 (8.3) | 34 (10.8) | |
| Very good | 589 (31.5) | 87 (28.7) | 100 (31.8) | 98 (31.8) | 100 (31.5) | 94 (30.0) | 110 (34.8) | |
| Good | 772 (41.2) | 118 (38.9) | 131 (41.6) | 139 (45.1) | 136 (42.8) | 136 (43.5) | 112 (35.4) | |
| Fair | 319 (17.0) | 65 (21.5) | 55 (17.5) | 46 (14.9) | 53 (16.7) | 49 (15.7) | 51 (16.1) | |
| Poor | 49 (2.6) | 9 (3.0) | 9 (2.9) | 4 (1.3) | 10 (3.1) | 8 (2.6) | 9 (2.9) | |
| How many different medications do you take daily? | 0.4009 | |||||||
| 0 | 400 (21.4) | 67 (21.3) | 76 (24.7) | 59 (18.6) | 65 (20.8) | 77 (24.4) | 56 (18.5) | |
| 1–2 | 568 (30.3) | 94 (29.8) | 98 (31.8) | 101 (31.8) | 94 (30.0) | 90 (28.5) | 91 (30.0) | |
| 3–4 | 455 (24.3) | 78 (24.8) | 66 (21.4) | 78 (24.5) | 85 (27.2) | 82 (26.0) | 66 (21.8) | |
| 5–20 | 450 (24.0) | 76 (24.1) | 68 (22.1) | 80 (25.2) | 69 (22.0) | 67 (21.2) | 90 (29.7) |
ARR, absolute risk reduction; DA, decision aid; RRR, relative risk reduction.
Variables with predetermined quotas.
Categories with sparse cell counts were not included in chi-square analysis.
Impact of Contextualizing Statements on Likelihood of Taking Sacubitril-Valsartan
Overall, 54.2% of participants reported they would likely take sacubitril-valsartan at $50/month, with a range across the varying contextualizing statements (CS) from 52.5% (CS#4, Re-statement of Absolute Risk Reduction + Physician Recommendation) to 57.3 (CS#6, Re-statement of Relative Risk Reduction + Baseline; Table 3). Among the pairwise comparisons between the baseline contextualizing statement and each of the five experimental contextualizing statements, there were no significant differences. Bayes factors ranged from 0.09 to 0.13, indicating support for the null hypothesis (Table 4). There were no statistically significant effects observed by contextualizing statement when examining responses using the entire ordinal scale (Figure 2; P = 0.18).
Table 3.
Participant Attitudes Toward Sacubitril-Valsartan and Certainty Regarding Their Decision
| Outcomes | DA #1—Baseline (n = 303), n (%) | DA #2—ARR + Baseline (n = 315), n (%) | DA #3—ARR Alone (n = 308), n (%) | DA #4—ARR + Doctor Recommendation (n = 318), n (%) | DA #5—ARR + Neutral (n = 313), n (%) | DA #6—RRR + Baseline (n = 316), n (%) | Overall (n = 1873), n (%) | P Value |
|---|---|---|---|---|---|---|---|---|
| Based on what you read, if your insurance covers most of the cost of Entresto but you would need to pay $50 per month for it (versus $ 0 co-pay for Enalapril), how likely would you be to take Entresto if your doctor recommended it? | P = 0.89 | |||||||
| Likely | 162 (53.5) | 170 (54.0) | 166 (53.9) | 167 (52.5) | 169 (54.0) | 181 (57.3) | 1015 (54.2) | |
| Unlikely | 141 (46.5) | 145 (46.0) | 142 (46.1) | 151 (47.5) | 144 (46.0) | 135 (42.7) | 858 (45.8) | |
| On a scale of 1 to 10, with 1 being no better and 10 being a lot better, how much better do you think Entresto is for patients with heart failure compared to Enalapril? | P = 0.09 | |||||||
| Mean (SD) | 6.27 (2.3) | 6.23 (2.2) | 5.95 (2.3) | 6.09 (2.3) | 5.75 (2.4) | 6.15 (2.5) | 6.07 (2.3) | |
| Decisional Conflict Scale—3-question uncertainty subscore | P = 0.52 | |||||||
| Mean (SD) | 30.8 (21.9) | 29.5 (21.7) | 29.2 (21.1) | 30.8 (23.0) | 33.8 (23.7) | 26.6 (22.6) | 30.1 (22.4) | |
| According to the information you read, if 100 people were to take Entresto for 2 years, how many more people would be alive than if the same 100 people took Enalapril? | P = 0.72 | |||||||
| 3 (Correct answer) | 127 (41.9) | 140 (44.4) | 118 (38.3) | 131 (41.2) | 129 (41.2) | 125 (39.6) | 770 (41.1) | |
| Incorrect (0–2, 4–100) | 176 (58.1) | 175 (55.6) | 190 (61.7) | 187 (58.8) | 184 (58.8) | 191 (60.4) | 1103 (58.9) |
ARR, absolute risk reduction; DA, decision aid; RRR, relative risk reduction.
Table 4.
Pairwise Comparisons between Contextualizing Statement on Likelihood of Taking sacubitril-valsartan
| Contextualizing Statement Comparison | Percent Likely Comparison | Chi-square P Value | Bayes Factors |
|---|---|---|---|
| CS #1 v. CS #2 | 53.47 v. 53.97 | 0.90 | 0.10 |
| CS #1 v. CS #3 | 53.47 v. 53.90 | 0.92 | 0.09 |
| CS #1 v. CS #4 | 53.47 v. 52.52 | 0.81 | 0.10 |
| CS #1 v. CS #5 | 53.47 v. 53.99 | 0.90 | 0.10 |
| CS #1 v. CS #6 | 53.47 v. 57.28 | 0.34 | 0.13 |
Figure 2.
Likelihood of taking sacubitril-valsartan by contextualizing statement.
In multivariable analysis, after controlling for demographics and covariates, none of the experimental contextualizing statements were significant predictors of likelihood of taking sacubitril-valsartan when compared with the baseline (Table 5).
Table 5.
Multivariable Logistic Regression Model of Likelihood of Taking Sacubitril-Valsartan (n = 1796)
| Effect | Sacubitril-Valsartan | ||
|---|---|---|---|
| Odds Ratio | 95% Confidence Limit | ||
| Lower | Higher | ||
| Contextualizing statement | |||
| CS #2 v. CS #1 | 1.124 | 0.801 | 1.578 |
| CS #3 v. CS #1 | 1.012 | 0.72 | 1.422 |
| CS #4 v. CS #1 | 1.005 | 0.718 | 1.407 |
| CS #5 v. CS #1 | 0.979 | 0.698 | 1.374 |
| CS #6 v. CS #1 | 1.276 | 0.908 | 1.792 |
| Age | |||
| <45 v. 65+ | 2.772 | 1.84 | 4.177 |
| 45–64 v. 65+ | 1.308 | 1.023 | 1.673 |
| Gender | |||
| Male v. Female | 1.15 | 0.92 | 1.438 |
| Race | |||
| Black v. White | 1.128 | 0.88 | 1.447 |
| Hispanic v. White | 1.007 | 0.716 | 1.414 |
| Other v. White | 0.88 | 0.55 | 1.407 |
| Income | |||
| 25,000–50,000 v. <25,000 | 1.424 | 1.052 | 1.926 |
| 50,000–100,000 v. <25,000 | 1.703 | 1.263 | 2.298 |
| >100,000 v. <25,000 | 2.122 | 1.474 | 3.056 |
| Education | |||
| Some college v. High school or less | 0.861 | 0.635 | 1.168 |
| Bachelor’s degree v. High school or less | 0.906 | 0.652 | 1.257 |
| Graduate degree v. High school or less | 1.034 | 0.718 | 1.491 |
| Self-rated health | |||
| Excellent/very good/good v. Fair/poor | 1.403 | 1.096 | 1.796 |
| Medical decision approach (minimizer, maximizer) | |||
| 2 or 3 v. 1 | 2.18 | 1.525 | 3.116 |
| 4 v. 1 | 2.735 | 1.886 | 3.966 |
| 5 or 6 v. 1 | 3.76 | 2.608 | 5.422 |
Impact of Contextualizing Statements on Perceptions of Benefit
Overall, 41.1% of participants correctly identified that three more people out of 100 would be alive after taking sacubitril-valsartan for 2 years, and participants’ identification of this benefit did not differ between variations in the contextualizing statement (P = 0.72; Table 3). Among participants who correctly identified the benefit, there was no significant difference in likelihood of taking sacubitril-valsartan between participants exposed to any of the experimental contextualizing statements compared to the baseline statement (baseline CS#1 53.5%, range 49.2% to 61.6%).
Only the contextualizing statement with a deliberately neutral statement (CS#5) demonstrated a statistically significant reduction compared to the baseline contextualizing statement (CS#1) in participants’ assessment of “how much better” sacubitril-valsartan is compared to enalapril (mean 5.8 [CS#5] v. 6.3 [CS#1], P = 0.01).
Predictors of Likelihood of Taking Sacubitril-Valsartan
In multivariable logistic regression analysis, survey respondents who leaned more toward taking action when making medical decisions (“medical maximizers”) 28 were more likely to take sacubitril-valsartan than those who leaned more toward waiting and seeing (“medical minimizers”; Table 5). On a single-item self-rated health status, participants who described their health status as “Excellent,”“Very good,” or “Good” were also more likely to take sacubitril-valsartan (odds ratio: 1.4, 95% confidence interval = 1.1–1.8), compared with those with fair or poor health. Higher income and age less than 45 years old were also positively associated with likelihood of taking sacubitril-valsartan. Race, sex, and education were not statistically significant predictors of likelihood of taking sacubitril-valsartan.
To examine whether there were differences in likelihood of taking sacubitril-valsartan based on the time to complete the survey, time was divided into quartiles. There were no significant differences in likelihood among quartiles (P = 0.23).
Discussion
The impact of different forms of contextualizing language on patients’ understanding of benefits and medical choices is not well understood. A currently available decision aid for patients considering sacubitril-valsartan explicitly includes a contextualizing statement that was added due to clinicians’ concern that patients would fail to appreciate the significance of a 3% absolute risk reduction in mortality over 2 years. 16 These data provide novel insights into whether this or other contextualizing statements might be effective tools in decision aid construction.
Overall, these data suggest that the contextualizing statements we used within a traditionally constructed decision aid had no discernable impact on patient decision making. Despite testing contextualizing statements with markedly different language and valence, varying from outright positive (Most doctors consider this to be a very significant improvement) to strongly neutral (You can decide whether this is a significant benefit for you), we saw no significant impact on likelihood of taking sacubitril-valsartan, a drug associated with a benefit most clinicians consider to be important. Even factors typically found to influence and drive benefit perception, such as presentation of relative risk reduction versus absolute risk reduction, had minimal effect on the primary outcome in our analysis. Only the strongly neutral contextualizing statement had an impact on the perceived benefit of the drug. In addition to the absence of any significant relationship in the primary analysis, Bayes factors ranging from 0.09 to 0.13 indicate evidence in favor of the null hypothesis, that there is no association between willingness to take sacubitril-valsartan and group assignment.
The finding that these contextualizing statements had minimal impact on decisions in this context is humbling. Significant time and effort goes into creating decision aids in order to optimally present treatment options to patients in ways that promote value-choice concordance. Relevant decisions include what data to present, what to leave out, and what contextual information is most helpful. These results suggest some of the details on which decision aid constructors spend considerable effort, and about which strong assumptions may exist, may not influence actual decisions in predicted ways. In this study, even objectively nonneutral contextualizing statements—which would likely raise concern as out of line with standards such as the International Patient Decision Aid Standards (IPDAS) criteria due to potential bias—did not affect likelihood of taking sacubitril-valsartan. 22 Other ways to contextualize these data and other ways to integrate these statements may have more significant impact, and we did not assess whether no contextualizing statement at all would change decisions. However, these findings reinforce the importance of empirical evaluation of decision aid construction rather than relying on conventional wisdom or assumptions about the impact of language choice.
In addition to having no discernible impact on decisions themselves, different contextualizing statements did not significantly affect comprehension of numerical benefits, in contrast to other studies that have found decision aid components to significantly influence comprehension.34,35 Regardless of how the benefit of sacubitril-valsartan over ACE inhibitor was contextualized, there was an overall low level of correct identification of benefit data. Increasing benefit comprehension is an important goal of decision aids. Our results suggest that contextualizing statements about the magnitude of health benefit, at least as tested here, may not be a particularly effective mechanism for accomplishing this goal.
Although various contextualizing statements had no significant differential impact on the likelihood of taking sacubitril-valsartan at a $50 per month price point, younger age and better self-reported health status were statistically significant predictors of likelihood of taking the medication. Individuals with worse self-reported health status might be predicted to have an increased appreciation for their own mortality risks. However, in this study, they were less likely to take sacubitril-valsartan. These findings echo a prior study of heart failure patients regarding this decision and may have implications for clinicians. 17 Specifically, these results suggest a need for careful discussion of this medication with older and sicker patients to ascertain whether decreased willingness to take the medication is a function of differential valuation of the benefits, concerns about cost, general views about medications, or a combination of those considerations. In contrast, the higher likelihood of taking sacubitril-valsartan in higher income groups is not surprising but further demonstrates increased cost-sensitivity among patients with lower income. 17
Several limitations of this study must be acknowledged. First, this study involved a hypothetical scenario; the results may not reflect the choices of actual patients with heart failure with reduced ejection fraction making actual decisions about their health or the way such patients would actually read or use a decision aid. Second, selection bias could be present with the use of an online panel of respondents, though our sample contained well-balanced subgroups and utilized participant-level randomization. Third, the study featured a brief description of the considerations associated with choosing to start sacubitril-valsartan over an ACE inhibitor. Participants may have lacked sufficient topical experience to understand the nuances involved in the decision to start sacubitril-valsartan. We deliberately shortened the decision aid to focus on the quantitative benefit information, but it still contained an appreciable amount of information in the form of text and multiple images. Fourth, we did not explore general attitudes toward taking medications among participants as a possible explanation for why certain demographic groups were more or less likely to take the medications. Finally, only one mechanism of inserting a contextualizing statement (a box adjacent to a pictograph) was assessed at one price point, and we did not test a decision aid in which no contextualizing statement was present. Other displays could be more impactful, and contextualizing statements may play a different role in other types of decisions or at different prices. However, the persistent null result among participants who correctly identified the 3% mortality reduction, strongly negative Bayes factors, and the fact that the study did detect other predictors of likelihood of taking sacubitril-valsartan (i.e., higher income and younger individuals were more likely to pay for sacubitril-valsartan), adds strength to the overall null result regarding the impact of contextualizing statements as studied.
Conclusion
Despite null results, this study provides important insights. Specifically, it raises questions regarding the limitations of manipulating contextualizing statements as a tool for framing benefits within decision aids. It may assuage concerns that these statements are heavy-handed or too strongly influential, but it suggests they may not be particularly meaningful, at least as implemented in this context. In this respect, continued reflection, evaluation, and humility are needed to learn more about what does and does not impact patients’ decisions and perceptions.
Supplemental Material
Supplemental material, sj-docx-1-mpp-10.1177_23814683211041623 for Framing Benefits in Decision Aids: Effects of Varying Contextualizing Statements on Decisions About Sacubitril-Valsartan for Heart Failure by Mary C. Thomson, Larry A. Allen, Scott D. Halpern, Yi-An Ko, Daniel D. Matlock, Andrea R. Mitchell, Miranda A. Moore, Alanna A. Morris, Birju R. Rao, Laura D. Scherer, Candace D. Speight, Peter A. Ubel and Neal W. Dickert in MDM Policy & Practice
Footnotes
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Allen reports grant funding from AHA, NIH, and PCORI, and reports consulting fees from Abbott, ACI Clinical, Amgen, Boston Scientific, Cytokinetics, and Novartis.
Dr. Dickert reports receiving research funding from AHRQ, NIH, PCORI, and the Greenwall Foundation. Dr. Moore reports receiving research funding from AHRQ, HRSA, and the Ardmore Institute of Health. Dr. Morris reports receiving research funding from NIH/NHLBI, AHRQ, the Woodruff Foundation, and the Association of Black Cardiologists. Dr. Rao reports research support from the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Numbers UL1TR002378 and TL1TR002382 and support from the Bryon Williams Jr, MD Fellowship Fund. Ms. Thomson, Dr. Halpern, Dr. Ko, Dr. Matlock, Ms. Mitchell, Dr. Scherer, Ms. Speight, and Dr. Ubel report no financial conflict of interests related to this research. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided by a grant from the Agency for Healthcare Quality and Research (1RO1HS026081-01). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
ORCID iDs: Daniel D. Matlock 
https://orcid.org/0000-0001-9597-9642
Miranda A. Moore https://orcid.org/0000-0002-1455-6099
Laura D. Scherer https://orcid.org/0000-0002-8660-7115
Neal W. Dickert https://orcid.org/0000-0003-4415-3861
Supplemental Material: Supplementary material for this article is available on the Medical Decision Making Policy & Practice website at https://journals.sagepub.com/home/mpp.
Contributor Information
Mary C. Thomson, Medical College of Georgia, Augusta, Georgia
Larry A. Allen, Department of Medicine, University of Colorado School of Medicine, Aurora, Colorado
Scott D. Halpern, Palliative and Advanced Illness Research (PAIR) Center and Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania
Yi-An Ko, Rollins School of Public Health, Emory University, Atlanta, Georgia.
Daniel D. Matlock, Department of Medicine, University of Colorado School of Medicine, Aurora, Colorado VA Eastern Colorado Geriatric Research Education and Clinical Center, Denver, Colorado.
Andrea R. Mitchell, Department of Medicine, Division of Cardiology, Emory University School of Medicine, Atlanta, Georgia
Miranda A. Moore, Department of Family and Preventive Medicine, Emory University School of Medicine, Atlanta, Georgia
Alanna A. Morris, Department of Medicine, Division of Cardiology, Emory University School of Medicine, Atlanta, Georgia
Birju R. Rao, Department of Medicine, Division of Cardiology, Emory University School of Medicine, Atlanta, Georgia
Laura D. Scherer, Department of Medicine, University of Colorado School of Medicine, Aurora, Colorado
Candace D. Speight, Department of Medicine, Division of Cardiology, Emory University School of Medicine, Atlanta, Georgia
Peter A. Ubel, Duke University Fuqua School of Business, Sanford School of Public Policy, Durham, North Carolina
Neal W. Dickert, Department of Medicine, Division of Cardiology, Emory University School of Medicine, Atlanta, Georgia; Department of Epidemiology, Emory University Rollins School of Public Health, Atlanta, Georgia.
References
- 1. Reyna VF. A theory of medical decision making and health: fuzzy trace theory. Med Decis Making. 2008;28(6):850–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. In: Connolly T, Arkes HR, Hammond KR, eds. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge University Press; 2000. p 424. [Google Scholar]
- 3. McNeil B, Pauker S, Sox H, Jr, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med. 1982;306:1259–62. [DOI] [PubMed] [Google Scholar]
- 4. Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk. J Gen Intern Med. 1993;8(10):543–8. [DOI] [PubMed] [Google Scholar]
- 5. Navar AM, Wang TY, Mi X, et al. Influence of cardiovascular risk communication tools and presentation formats on patient perceptions and preferences. JAMA Cardiol. 2018;3(12):1192–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Marteau TM, Senior V, Sasieni P. Women’s understanding of a “normal smear test result”: experimental questionnaire based study. BMJ. 2001;322(7285):526–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Zikmund-Fisher BJ, Fagerlin A, Keeton K, Ubel PA. Does labeling prenatal screening test results as negative or positive affect women’s responses? Am J Obstet Gynecol. 2007;197(5):528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Wolfe CR, Reyna VF, Smith RJ. On judgments of approximately equal. J Behav Decis Mak. 2018;31(1):151–63. [Google Scholar]
- 9. Waters EA, Fagerlin A, Zikmund-Fisher BJ. Overcoming the many pitfalls of communicating risk. In: Diefenbach MA, Miller-Halegoua S, Bowen DJ, eds. Handbook of Health Decision Science. Springer Science; 2016. p 265–77. [Google Scholar]
- 10. Cameron LD, Sherman KA, Marteau TM, Brown PM. Impact of genetic risk information and type of disease on perceived risk, anticipated affect, and expected consequences of genetic tests. Health Psychol. 2009;28(3):307–16. [DOI] [PubMed] [Google Scholar]
- 11. Fagerlin A, Ubel PA, Smith DM, Zikmund-Fisher BJ. Making numbers matter: present and future research in risk communication. Am J Health Behav. 2007;31(suppl 1):S47–S56. [DOI] [PubMed] [Google Scholar]
- 12. Trevena LJ, Zikmund-Fisher BJ, Edwards A, et al. Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers. BMC Med Inform Decis Mak. 2013;13(suppl 2):S7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making. 2007;27(5):696–713. [DOI] [PubMed] [Google Scholar]
- 14. McMurray JJV, Packer M, Desai AS, et al. Angiotensin–neprilysin inhibition versus enalapril in heart failure. N Engl J Med. 2014;371(11):993–1004. [DOI] [PubMed] [Google Scholar]
- 15. Yancy CW, Jessup M, Bozkurt B, et al. 2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America. Circulation. 2017;136(6):e137–e161. [DOI] [PubMed] [Google Scholar]
- 16. DeJong C, Kazi DS, Dudley RA, Chen R, Tseng CW. Assessment of national coverage and out-of-pocket costs for sacubitril/valsartan under Medicare Part D. JAMA Cardiol. 2019;4(8):828–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Smith GH, Shore S, Allen LA, et al. Discussing out-of-pocket costs with patients: shared decision making for sacubitril-valsartan in heart failure. J Am Heart Assoc. 2019;8(1):e010635. doi: 10.1161/JAHA.118.010635 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. CardioSmart. A decision aid for renin-angiotensin inhibitor drug options for patients with heart failure [cited December 17, 2020}. Available from: https://www.cardiosmart.org/docs/default-source/assets/decision-aid/heart-failure-drug-options.pdf?sfvrsn=aaff9c98_1.GoogleScholar
- 19. Venechuk GE, Allen LA, Doermann Byrd K, Dickert N, Matlock DD. Conflicting perspectives on the value of neprilysin inhibition in heart failure revealed during development of a decision aid focusing on patient costs for sacubitril/valsartan. Circ Cardiovasc Qual Outcomes. 2020;13(9):e006255. [DOI] [PubMed] [Google Scholar]
- 20. Virani SS, Alonso A, Benjamin EJ, et al. Heart disease and stroke statistics-2020 update: a report from the American Heart Association. Circulation. 2020;141(9):e139–e596. [DOI] [PubMed] [Google Scholar]
- 21. Heidenreich PA, Trogdon JG, Khavjou OA, et al. Forecasting the future of cardiovascular disease in the United States: a policy statement from the American Heart Association. Circulation. 2011;123(8):933–44. [DOI] [PubMed] [Google Scholar]
- 22. Elwyn G, O’Connor A, Stacey D, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006. Aug 26;333(7565):417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Litman L, Robinson J, Abberbock T. TurkPrime.com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behav Res. 2017;49(2):433–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Oppenheimer D, Meyvis T, Davidenko N. Instructional manipulation checks: detecting satisficing to increase statistical power. J Exp Soc Psychol. 2009;45:867–72. [Google Scholar]
- 25. O’Connor AM. User manual—Decisional Conflict Scale [cited August 11, 2021]. Available from: https://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_decisional_conflict.pdf
- 26. O’Connor AM. Validation of a decisional conflict scale. Med Decis Making. 1995;15(1):25–30. [DOI] [PubMed] [Google Scholar]
- 27. Scherer LD, Caverly TJ, Burke J, et al. Development of the Medical Maximizer-Minimizer Scale. Health Psychol. 2016;35(11):1276–87. [DOI] [PubMed] [Google Scholar]
- 28. Scherer LD, Kullgren JT, Caverly T, et al. Medical maximizing-minimizing preferences predict responses to information about prostate-specific antigen screening. Med Decis Making. 2018;38(6):708–18. [DOI] [PubMed] [Google Scholar]
- 29. Wharton Credibility Lab, University of Pennsylvania. As predicted: Benefit Framing and Shared Decision-Making Survey (#43098) [cited August 11, 2021]. Available from: https://aspredicted.org/blind.php?x=uv64bu
- 30. Gunel E, Dickey J. Bayes factors for independence in contingency tables. Biometrika. 1974;61(3):545–57. [Google Scholar]
- 31. Goodman SN. Toward evidence-based medical statistics. 2: the Bayes factor. Ann Intern Med. 1999;130(12):1005–13. [DOI] [PubMed] [Google Scholar]
- 32. Lee M, Wagenmakers E. Bayesian Cognitive Modeling: A Practical Course. Cambridge University Press; 2014. [Google Scholar]
- 33. United States Census Bureau. US Census Bureau releases new educational attainment data [cited January 19, 2021]. Available from: https://www.census.gov/newsroom/press-releases/2020/educational-attainment.html
- 34. Martin RW, Brower ME, Geralds A, Gallagher PJ, Tellinghuisen DJ. An experimental evaluation of patient decision aid design to communicate the effects of medications on the rate of progression of structural joint damage in rheumatoid arthritis. Patient Educ Couns. 2012;86(3):329–34. [DOI] [PubMed] [Google Scholar]
- 35. Martin RW, Enck RD, Tellinghuisen DJ, Eggebeen AT, Birmingham JD, Head AJ. Comparison of the effects of a pharmaceutical industry decision guide and decision aids on patient choice to intensify therapy in rheumatoid arthritis. Med Decis Making. 2017;37(5):577–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-docx-1-mpp-10.1177_23814683211041623 for Framing Benefits in Decision Aids: Effects of Varying Contextualizing Statements on Decisions About Sacubitril-Valsartan for Heart Failure by Mary C. Thomson, Larry A. Allen, Scott D. Halpern, Yi-An Ko, Daniel D. Matlock, Andrea R. Mitchell, Miranda A. Moore, Alanna A. Morris, Birju R. Rao, Laura D. Scherer, Candace D. Speight, Peter A. Ubel and Neal W. Dickert in MDM Policy & Practice