Improving Communication of Breast Cancer Recurrence Risk

Abstract

Purpose

Doctors commonly use genomic testing for breast cancer recurrence risk. We sought to assess whether the standard genomic report provided to doctors is a good approach for communicating results to patients.

Methods

During 2009–2010, we interviewed 133 patients with stages I or II, node-negative, hormone-receptor positive breast cancer and eligible for the Oncotype DX genomic test. In a randomized experiment, patients viewed 6 vignettes that presented hypothetical recurrence risk test results. Each vignette described a low, intermediate, or high chance of breast cancer recurrence in 10 years. Vignettes used one of five risk formats of increasing complexity that we derived from the standard report that accompanies the commercial assay or a sixth format that used an icon array.

Results

Among women who received the genomic recurrence risk test, 63% said their doctors showed them the standard report. The standard report format yielded among the most errors in identification of whether a result was low, intermediate or high risk (i.e., the gist of the results), while a newly developed risk continuum format yielded the fewest errors (17% vs. 5%; OR, 0.23; 95% CI, 0.10 to 0.52). For high-recurrence risk results presented in the standard format, women made errors 35% of the time. Women rated the standard report as one of the least understandable and least liked formats, but they rated the risk continuum format as among the most understandable and most liked. Results differed little by health literacy, numeracy, prior receipt of genomic test results during clinical care and actual genomic test results.

Conclusion

The standard genomic recurrence risk report was more difficult for women to understand and interpret than other formats. A less complex report, potentially including the risk continuum format, would be more effective in communicating test results to patients.

Introduction

As researchers develop new technologies for estimating disease risk, many will likely do what others have done before: create new ways to communicate the results of their new tests. Genomic testing for recurrence of early-stage breast cancer is a case in point. Oncotype DX (Genomic Health, Redwood City, California) is a genomic test that estimates 10 year risk of distant recurrence of stage I and II, node-negative, estrogen receptor-positive breast cancer, based on the expression of 21 genes [1–4]. The results from this test help early-stage breast cancer patients and their doctors make decisions about adding adjuvant chemotherapy to endocrine therapy [4–8]. This test is now widely used in clinical cancer care.

While use of genomic tests is increasing, the best way to communicate results of these sometimes complex tests is not known. The American company that performs the Oncotype DX assay, Genomic Health, Inc, has created a detailed report for clinicians, but anecdotal reports suggest that clinicians often provide it to their patients. Their report provides a recurrence score between 1 and 100 that is transformed into a 10 year recurrence risk (assuming endocrine treatment) with 95% confidence interval which is then further categorized as a low, intermediate or high risk (Figure 1a). The report also includes a complex graph showing the confidence intervals and the three risk categories as well as information about the clinical research that is the basis for the test. This contrasts dramatically with the approach of the European company that makes Mammaprint, a 70 gene profile [9–12] that also estimates recurrence risk. Their test yields only two results: low risk or high risk.

Figure 1.

Figure 1a. Standard Oncotype DX report (risk format 5)

Figure 1b. Patients’ preferred risk format (risk format 3). Recurrence risk with verbal descriptor, risk continuum graphic and confidence interval

Figure 1c. Icon array format (risk format 6)

Is a simpler or more complex approach better for communicating results of genomic tests to patients? Surprisingly, we have no data to adequately answer this question or to help us make an educated guess other than to say that the standard Oncotype DX report seems more complicated than is likely helpful [13]. Previous studies have focused on communicating risk as a percentage, frequency or graphic [14,15] and other research has shown that confidence intervals may undermine accurate understanding [16]. However, the ambitious multifactor presentation adopted for Oncotype DX is so complex that we sought to identify aspects of the format that work well and others that do not. To this end, we conducted an experiment with early-stage breast cancer patients that compared risk communication formats of varying complexity that used elements from the Oncotype DX report. We also included a format not used in the standard report, an icon array, as previous research has identified this as an effective way to present breast cancer and other risks [14,15]. We hypothesized that simpler risk formats would be easier for patients to understand and thus elicit fewer interpretation errors.

Methods

Participants and Recruitment

Participants were women being treated for early-stage breast cancer at the University of North Carolina Breast Clinic (Chapel Hill, NC) between May 2009 and November 2010, who were eligible to receive genomic recurrence risk testing by Oncotype DX (diagnosed with stages I or II, node-negative, hormone-receptor positive breast cancer). We excluded women who were non-English speaking, were incarcerated, had a second primary cancer diagnosis or other life-threatening co-morbid disease, or had a history of a serious psychiatric diagnosis. We interviewed eligible women who received the genomic test and those who were eligible but did not receive the test as part of their clinical care.

We approached women at their next scheduled medical oncology appointments. For women who received the genomic test, we usually interviewed them immediately after the appointment during which they received their test results. We mailed study materials to women who did not complete questionnaires in clinic.

Patients read and signed a written informed consent form. They received $15 and a parking pass for participating. The University of North Carolina institutional review board approved the study.

Recurrence Risk Vignettes

We constructed vignettes that presented hypothetical recurrence risk test results. Each vignette described a risk near the median of the three categories visually depicted on the standard Oncotype DX report: low risk magnitude (7% or 9% chance of recurring in 10 years), intermediate (14% or 18% chance of recurring in 10 years), or high (27% or 30% chance of recurring in 10 years). Each vignette used one of six risk formats described below. Participants each viewed 6 of the 36 possible vignettes. We counterbalanced the vignettes such that we assigned each vignette to one of the 6 risk magnitudes randomly without replacement and to one of the 6 risk formats using a Latin square.

Risk format 1 was the simplest risk format. It described women’s chance of breast cancer recurrence in 10 years as a percentage and interpreted the percentage as being a low, intermediate, or high chance of recurrence (e.g., “Breast cancer like yours has a 7% chance of recurring in 10 years with hormone therapy. This is considered a low chance”). Risk format 2 had the same statement and added a risk continuum graphic. The graphic included an x-axis that ranged from 0–100% chance of breast cancer recurrence in 10 years, partitioned into low, intermediate, and high chance of recurrence, with an arrow that pointed to the test result. Risk format 3 included the previous statement with risk continuum graphic but added text reporting a 95% confidence interval with a verbal translation description below the graphic (see Figure 1b).

Risk format 4 added two elements from the standard genomic recurrence risk report, a recurrence score and a two-dimensional graph that transformed the recurrence score into a recurrence risk. Risk format 5 was an actual Oncotype DX report with patient name and information removed (Oncotype DX is a trademark of Genomic Health, Inc.). The standard report provides a recurrence score, a recurrence risk, a graph, a 95% confidence interval, plain language risk categories (e.g., “low risk”), an assay description, and miscellaneous information about the test and Genomic Health (see Figure 1a). Risk format 6 was an icon array with a 10 by 10 grid of rectangles, adapted from a design shown to improve comprehension of breast cancer risk information (see Figure 1c) [14,15]. Shaded boxes indicated the number of breast cancer patients, out of 100, who will have a recurrence in the next 10 years. Accompanying text described the number of breast cancer patients with recurrence in 10 years and labeled this a low, intermediate, or high chance of recurrence.

Measures

Five questions followed each vignette, usually at the bottom of the page, to assess errors in interpreting the result and attitudes toward the risk format.

Errors

The first item assessed whether women inaccurately identified the “gist” [17] of the recurrence risk presented. Response options were “low chance”, “intermediate chance”, or “high chance”. The second item assessed whether women inaccurately identified the verbatim recurrence risk. Response options were 0% to 100%.

Attitude toward the test results

We averaged two items to form a single continuous measure of women’s perceptions of how well they understood the test results (r=.89). The items read “How confident are you that you understand this test result?” and “How easy was this test result to understand?”) A 4-point response scale accompanied the items that ranged from “not at all” to “very”. An item that assessed trust read, “How much do you trust that this test result is accurate?” and was accompanied by a 4-point response scale ranging from “not at all” to “quite a lot”.

Other measures

At the end of the questionnaire, participants viewed all 6 risk formats and marked one they liked most and one they liked least. They indicated whether a doctor had ever shown them, or given them to take home, a test report that looked like the sample Oncotype DX report in the questionnaire. Women interviewed in clinic completed the health literacy assessment [18], if clinic flow permitted it. We assessed numeracy with a 3 item scale [19] (range 0–3 correct answers).

Data Analysis

Analyses adjusted for multiple observations within subjects. We used generalized estimating equations to examine the influence of risk format and risk magnitude on accurate recall of recurrence risk, understanding of test results, and trust. We tested for effect moderation by including interaction terms in separate models. We tested for mediation using the four-step Baron and Kenny procedure [20]. Analyses omitted 18 observations from a vignette that contained a layout error but retained data from these participants’ other vignettes. As preliminary analyses yielded an effect of serial position in which patients viewed risk formats on gist and verbatim accuracy, final analyses involving accuracy controlled for risk format serial position. Analyses used two-tailed tests, with a critical alpha of .05. We conducted analyses using SAS 9.2 (Cary, NC).

Results

Of 225 women we invited, 143 participated in the study (64%) (Figure 2). Women took the questionnaires in clinic (22%), at home (56%), or in both locations (22%). As 10 women did not complete the risk format questionnaire, the final sample size was 133. Women’s median age was 59 years (range 34 to 85); most women were white (89%) (Table 1). Many (36%) did not have a college degree. Though 79% correctly answered all health literacy items, only 32% correctly answered all numeracy items. About 41% reported an annual household income of less than $60,000. About half (48%) received the Oncotype DX test as part of their clinical care: Of these women, 63% (40/64) recalled that their doctors showed them their report, and 56% (36/64) received a copy that they could take home. About 39% of women were planning to receive chemotherapy treatment.

Figure 2.

Study flow diagram

Table 1.

Participant characteristics (n=133)

	n (%)
White race/ethnicity	117 (89)
Married or living as married	93 (70)
College degree	85 (64)
Annual household income, < $60,000	50 (41)
Worked for pay	69 (53)
Insured	123 (93)
High health literacy (8 of 8 correct answers)	45 (79)
High numeracy (3 of 3 correct answers)	42 (32)
Received Oncotype DX test	64 (48)
Oncotype DX risk category
Low	31 (55)
Intermediate	19 (34)
High	6 (11)
Received/planning to receive treatment
Mastectomy	66 (51)
Radiation	84 (63)
Chemotherapy	51 (39)

Note. 57 patients received the health literacy assessment.

Oncotype DX risk category applies only for women who received test (n=64). Risk category information was not available for 8 women.

Accuracy

Overall, 11% of women’s responses inaccurately identified the gist of the test results, defined as incorrectly identifying a result as low, intermediate or high recurrence risk. Gist errors were lower when women viewed results in a risk continuum graphic format than in the standard Oncotype DX report format (5% vs. 17%; OR, 0.23; 95% CI, 0.10 to 0.52; Table 2.) About 13% of patients’ responses reflected verbatim errors, defined as stating an incorrect numerical risk. Verbatim errors were lower when women viewed results in any of the three simpler risk formats (1–3) than in the standard report format (all P<.001). Gist errors were higher for vignettes describing intermediate and high recurrence risk compared to low recurrence risk (all P<.01); verbatim errors were equivalent for vignettes describing low, intermediate and high risk (all P>.05). For high recurrence risk results presented in the standard format, women made gist errors 35% of the time (8% for verbatim errors).

Table 2.

Errors in interpretation of hypothetical test results

	Gist errors n (%)	OR (95% CI)	Verbatim errors n (%)	OR (95% CI)
Risk Format
RF1: Percent recurrence risk with verbal descriptor	17/133 (13)	0.57 (0.31, 1.06)	10/130 (8)	0.34 (0.18, 0.64) **
RF2: RF1 + risk continuum graphic	8/133 (6)	0.27 (0.12, 0.58) **	9/133 (7)	0.29 (0.16, 0.52) **
RF3: RF2 + confidence interval	7/133 (5)	0.23 (0.10, 0.52) **	11/129 (9)	0.39 (0.21, 0.74) **
RF4: RF3 + risk score	11/114 (10)	0.54 (0.28, 1.03)	17/111 (15)	0.77 (0.46, 1.30)
RF5: Oncotype DX report	22/133 (17)	Ref	23/128 (18)	Ref
RF6: Icon array	21/133 (16)	0.79 (0.44, 1.44)	27/127 (21)	1.18 (0.72, 1.91)
Risk Magnitude
Low	12/268 (4)	Ref	39/262 (15)	Ref
Intermediate	32/266 (12)	3.47 (1.69, 7.13)*	30/258 (12)	0.73 (0.48, 1.11)
High	43/246 (17)	6.24 (3.04, 12.80)**	28/238 (12)	0.84 (0.52, 1.37)

Note. Ns reflect multiple observations within subjects. Gist = interpretation of whether the result indicated low, intermediate, or high risk. Verbatim = stating exact percentage risk. RF = risk format.

^*P < .01;

^**P < .001

Attitudes toward Risk Communication Formats

The standard Oncotype DX report was among the most disliked risk formats, chosen by 31% of women as the format they liked least, disapprobation surpassed only by the icon array (39%) (Table 3). When asked for the risk format they liked most, the risk continuum with confidence interval was the most popular (39%) followed by the Oncotype DX standard report (21%). This positive ranking of the standard report was largely driven by those who saw it during appointments. More women who recalled seeing the standard report during their clinic visits liked it than those who received Oncotype DX testing but did not see the report (40% vs. 10%, P<.05). Women said they found it easier to understand risk formats that had less detail than the standard Oncotype DX format. This format received lower understandability ratings than any other format (all P<.05) (Figure 3).

Table 3.

Least and most preferred risk formats (n=97)

Risk Format	Liked the Least	Liked the Most
	n (%)	n (%)
RF1: Percent recurrence risk with verbal descriptor	12 (12)	9 (9)
RF2: …and risk continuum graphic	0 (0)	8 (8)
RF3: …and confidence interval	0 (0)	38 (39)
RF4: …and recurrence score	9 (10)	15 (15)
RF5: Oncotype DX report	30 (31)	20 (21)
RF6: Icon array	38 (39)	3 (3)

Note. Analyses excluded data for 36 participants (18 received a risk format layout error and 18 selected multiple likes/dislikes for the questionnaire item on preferred risk formats).

Figure 3.

Perceived understandability of risk formats and trust in test results. Error bars report standard errors. *P <.05 for comparison to risk format 5 (standard Oncotype DX report); ** P <.01. Key for abbreviations (RF1-6) appears in first column of Table 2.

It was harder for women to understand vignettes describing intermediate compared to low recurrence risk (P=.02), but vignettes describing high and low recurrence risk were equivalent (P=.22), regardless of format. Higher understandability ratings were associated with fewer errors, for both gist and verbatim measures (P<.01 for each). The association of risk format to errors dropped in magnitude but remained statistically significant when we controlled for rating of understandability; this pattern of results is consistent with partial mediation, suggesting that perceived understandability is one explanation for the benefit of the simpler risk formats [20]. Women reported lower trust in the icon array format (P=.01, Figure 3) but equivalent trust in the other risk formats, regardless of recurrence risk magnitude.

Health Literacy, Numeracy, Prior Receipt of Oncotype DX and Test Results

Health literacy was not associated with either accuracy measure or trust (P=.14, P=.89, and P=.21), though lower health literacy was associated with lower perceived understanding of test results overall (P=.01). Lower numeracy was associated with lower accuracy for both measures (all P<.001), lower perceived understanding of test results (P<.001) and lower trust (P=.003). These outcomes were not associated with prior receipt of Oncotype DX testing during clinical care or the recurrence risk indicated by this test (low, intermediate or high) (all P>.05).

Associations of risk format to errors, perceived understanding and trust did not vary by health literacy, numeracy, receipt of Oncotype DX testing during clinical care, their test results, or whether women recalled their physicians showing them the Oncotype DX report, except as noted below. The association of risk format to understandability was moderated by numeracy (interaction, P <.001). Women with higher numeracy said they understood the icon array format better than the standard format (mean 3.6 and 3.1 respectively, P<.01). However, women with lower numeracy gave both risk formats equivalent and low ratings of understandability (mean 2.8 and 2.8 respectively, P=.98).

Conclusions

Accurate understanding of results from medical tests, including genomic tests, is fundamental to informed patient decision making in cancer care. However, the best way to communicate results of new genomic tests to patients is not always obvious. While doctors use many different ways to communicate risk [21,22], and their opinions about treatment may carry special weight, the materials they provide to patients may also affect patients’ decisions. Our study found that over half of breast cancer patients who received the Oncotype DX test at one institution recalled seeing the standard report as part of their clinical care. About 35% of the time, breast cancer patients in our study misunderstood the gist of hypothetical high risk results presented in the standard Oncotype DX report format. The standard report yielded more errors than other simpler formats, and breast cancer patients gave it among the lowest ratings of understandability and liking.

Understanding the gist of recurrence risk (i.e., whether the risk is low, intermediate or high) is important for good decision making, because it is often associated with clinical decision rules[23]. Indeed, one of the key purposes of Oncotype DX is to allow women with low recurrence risk to safely forgo chemotherapy and its potential side effects. In contrast, understanding the precise numerical risk may be especially important when recurrence risk falls near the border between risk categories. Although study findings were generally similar for the gist and verbatim accuracy measures, patients made more gist errors when presented with information indicating intermediate and high recurrence risk compared to low risk. Less accurate interpretation of higher risk results may be due to resistance to risk information, a well documented finding [24,25]. Conversely, verbatim errors were about the same across the low, intermediate and high risk formats. Biases in gist, but not verbatim, accuracy is a common finding, showing that the people often see one thing but believe another [17].

Our results suggest that less is more [13]: a simple graphic presenting recurrence risk information on a continuum was the superior communication method. Women found the risk continuum easier to understand, and they objectively made fewer errors as a result. Such misunderstanding can affect adjuvant treatment decisions. In one study, breast cancer patients were more likely to endorse adjuvant chemotherapy treatment if they received relative rather than absolute risk reduction information [26]. Although simpler formats do not always elicit more accurate understanding of risk, people often prefer less cognitively taxing depictions of risk information [27,28]. Other explanations for the risk continuum format’s superior performance may be the inclusion of a highly visible numerical risk estimate (bolded and accompanied by a large arrow) accompanied by risk category boundaries and bolded labels.

We found two exceptions to our “less is more” conclusion. First, we had expected a drop off in performance for formats that included a 95% confidence interval. However, the two formats that differed only with respect to presenting a confidence interval (i.e., risk formats 2 and 3) performed similarly well [29]. We suspect that this is because we added the confidence interval in the text at the bottom of the risk format, allowing it to be available for women interested in it and ignored by the rest. Had we added this information to the main risk continuum graphic, we suspect that our findings for the confidence interval format might have been less favorable.

Second, regarding “less is more,” some studies have favored icon arrays as an effective method of risk communication [14–15, 30], because of their intuitive appeal derived from their use of frequencies rather than percentages [31,32]. Our results showed that participants generally liked and trusted this method the least, and they performed poorly using it. Our results echo those of other studies, which generally find little evidence for the superiority of frequencies over percentages [28,32,33–34]. The icon array’s poor performance may be due in part to its difference from the other formats, whose similarity to one another could have facilitated understanding. Further, unlike the other graphical risk formats, the icon array did not include visual markers of low, intermediate and high risk categories.

Receipt of Oncotype DX testing did not affect study outcomes. This is important as it suggests that our findings may generalize across patients naïve to Oncotype DX testing and for those who had experience with it. For women who received Oncotype DX, familiarity with the report format and actual test results also did not influence study outcomes. While health literacy played a surprisingly small role [35,36], patients with higher numeracy skills tended to make fewer errors in comprehension and reported better understanding and more trust in the risk communications. As patients generally receive counseling from their doctors about their genomic test results, patients may make fewer errors than our study suggests. However, our previous work has found that one third of patients who have received Oncotype DX did not recall their recurrence risk estimate [21], leaving open the possibility that many women have to rely on their fallible understanding of the report they take home.

Strengths of the study are the use of an experiment, a powerful within subjects design, and clinical populations for whom recurrence risk testing was especially relevant. A limitation is that the use of hypothetical test results may have yielded responses different from what patients might do with actual results while deciding about their own care. Our study did not address patients’ understanding of the impact of clinical variables such as tumor size on recurrence risk, another potential source of misunderstanding [17]. The within-subjects design may have affected reactions to some formats as women became used to the questionnaire, though we controlled statistically for serial position. While many women had reviewed Oncotype DX reports with their doctors prior to answering our survey, familiarity with the Oncotype DX report did not influence our main study findings. The sample had relatively high levels of education and other indicators of socioeconomic status; accuracy of patient understanding may even be lower among a more diverse sample. Since we began our study, the standard Oncotype DX report expanded to include information on relative risk reduction from adjuvant treatment. While the new format will need to be tested, we believe that our results suggest that less rather than more information will be useful. Further research is needed to replicate our findings with more diverse samples as well as to develop best practices around communication of genomic test information in clinical care.

We suggest making a less complex version of the standard Oncotype DX report for patients, as women made fewer errors when viewing test results in simpler risk formats. Emphasizing the gist of information should not involve simply presenting fewer facts, but, rather, presenting the facts in a format that makes them interpretable. The revised report will require testing to ensure accurate interpretation of risk information by patients and overall ease of use. In the interim, health professionals communicating results of this genomic test to patients should take special care with use of the standard report. Additionally, clinicians may adopt and test the risk continuum in their practice if they think it would be helpful in communicating with patients. As genomic tests rapidly expand into other areas of health, we should be cognizant of developing effective and efficient ways to communicate this critical information to our patients.