How do women trade-off benefits and risks in chemotherapy treatment decisions based on gene expression profiling for early-stage breast cancer? A discrete choice experiment

Abstract

Objectives

Gene expression profiling (GEP) of tumours informs baseline risk prediction, potentially affecting adjuvant chemotherapy decisions for women with early-stage breast cancer. Since only 15% will experience a recurrence, concerns have been raised about potential harms from overtreatment and high GEP costs in publicly funded healthcare systems. We aimed to estimate preferences and personal utility of GEP testing information and benefit–risk trade-offs in chemotherapy treatment decisions.

Design, setting and intervention

Based on literature review and findings from our qualitative research (focus groups, interviews with patients with breast cancer and medical oncologists), we developed a discrete choice experiment (DCE) survey and administered it via an internet panel. The DCE included 12 choice tasks with 5 attributes and 3 alternatives considering orthogonality, D-efficiency and level balance.

Participants

The DCE survey was administered to 1004 Canadian women from the general population.

Main outcome measures

Preferences were analysed using conditional logit and hierarchical Bayes and evaluated for goodness of fit. We conducted simulation analyses for alternative scenarios.

Results

GEP test score indicating likely benefit from chemotherapy was the most important attribute. Doctor's clinical estimate of the risk of cancer returning, trust in your cancer doctor and side effects of chemotherapy (temporary and permanent) were relatively less important but showed significant differences among levels. In the scenario analyses, 78% were likely to choose chemotherapy in a high-risk scenario, 55% in a moderate-risk scenario and 33% in a low-risk scenario, with the other attributes held constant. A high GEP score was more important in influencing the choice of chemotherapy for those at intermediate clinical risk.

Conclusions

GEP testing information influences chemotherapy treatment decisions in early-stage breast cancer and varies depending on clinical risk. Clinicians should be aware of these differences and tailor the use of GEP testing accordingly.

Strengths and limitations of this study.

• The first utility-based survey to obtain quantitative benefit–risk data on the value of gene expression profiling (GEP) in breast cancer treatment decisions.

• The hypothetical choice tasks had high face validity, demonstrated by the directionality, magnitude and consistency of responses, and high rate of confirmation about chemotherapy decisions.

• Large sample of women from the general population who may be at risk of breast cancer and are major stakeholders in a publicly funded healthcare system.

• Possibility of incentive bias when using an online survey panel.

• The choice tasks were hypothetical scenarios, and respondents’ choices may be different when faced with these decisions in real life.

Introduction

Breast cancer is the most common form of cancer in women in North America.^{1 2} Women with early-stage breast cancer are faced with challenging treatment decisions that require balancing risks and benefits. Current guidelines suggest that women with human epidermal growth factor receptor 2 (HER2)-negative, oestrogen receptor-positive, lymph node-negative cancer should be offered adjuvant chemotherapy.^3–6 However, it is estimated that only 15% of these cancers will recur.^3–6 As a result, some patients may be treated without benefit but are at risk of side effects from chemotherapy, and the associated treatment costs are borne by patients, caregivers and the healthcare system.

In recent years, concerns have been raised regarding overdiagnosis and overtreatment which can be harmful to patients’ health and increase burden on the healthcare system.^{7 8} Cultural beliefs that more treatment is better,⁹ as well as fear and uncertainty for patients and doctors may contribute to disease overtreatment.¹⁰ It is increasingly important to understand how individuals weigh the benefits and risks of treatment alternatives.

Gene expression profiling (GEP; eg, Oncotype DX), a form of personalised medicine, predicts the likelihood of cancer recurrence and can help identify women who may not benefit from chemotherapy, sparing them from associated toxicity.^11–15 GEP is increasingly used in clinical practice to support clinical decisions by oncologists and is specified in treatment guidelines as a complement or replacement to clinical algorithms that inform treatment decisions.^{6 16 17} Several studies have examined how receiving GEP results changes chemotherapy treatment decisions by women with breast cancer and their oncologists.^18–26 However, it remains unclear how women value GEP when weighing the benefits and risks in chemotherapy decisions.

Although attitude surveys provide important information, preference studies require respondents to trade-off risks and benefits of treatment or testing, which allows for a quantification of preferences as a measure of personal utility (the meaning and worth an individual gives to a genomic or genetic test from their personal perspective).^{27 28} A common method for measuring preferences is using a discrete choice experiment (DCE), in which respondents evaluate alternative scenarios and the relative importance of specific attributes in each scenario.²⁹ DCEs have been recommended as a method to measure personal utility in the area of genetics and genomics.^{30 31}

Without evidence of GEP's value for informing treatment decisions, it is unclear whether or not it should be supported and reimbursed through a publicly funded healthcare system. Given the considerable cost of GEP, ∼US$4000, the value of testing needs to be considered in the context of allocating limited healthcare resources where public values are increasingly incorporated into policy decisions.³² To our knowledge, no studies have measured preferences and the personal utility of GEP for informing chemotherapy decisions in early-stage breast cancer using utility-based methods. We aimed to estimate preferences and the personal utility of GEP testing information in chemotherapy decisions based on benefit–risk trade-offs using a DCE.

Methods

Study sample

We administered a DCE survey online to a sample of Canadian women (18 years and older) from the general population, recruited through an internet-based survey panel (Research Now). Age, education and province of residence quotas were set to ensure the sampled population was representative of the Canadian population. Research Now recruits panel members by invitation only, which yields high level of panel quality and representativeness, guards against duplication, fraudulent respondents and professional survey takers.³³

We estimated our minimum required sample size using the following calculation based on the number of choice tasks, attributes and levels: (N_resp×N_scen)/N_att>500,^{34 35} where N_resp is the number of respondents, N_scen is the number of choice tasks each respondent completes and N_att is the maximum number of levels for any single attribute. In our DCE survey, there were 10 random choice tasks and the maximum number of attribute levels was 4. As such, the minimum sample size required was n=200 respondents to achieve a minimum acceptable level of statistical precision (SE of all attributes <0.05) when estimating preferences. To allow us to explore variation by demographic subgroups with precise statistical analysis, we obtained a sample of 1004 respondents.

Informed consent was obtained from all respondents via an online consent form prior to commencing the survey.

Study design

Following good research practice recommendations, we designed a DCE survey to elicit preferences and estimate the value of GEP testing information in treatment decisions for early-stage breast cancer in terms of personal utility.^{27 29} DCEs use robust quantitative methods grounded in economic theory to estimate the utility (value) of alternative choices involving hypothetical risk–benefit trade-offs for specific attributes; the higher the utility, the greater the value of the attribute in the decision.

The development of key attributes and levels was informed by a literature review and qualitative research. We conducted focus groups and interviews (in English), details reported elsewhere,^{17 36 37} with n=28 women with a history of breast cancer,³⁶ interviews with n=14 oncologists.^{17 37} We also consulted the medical oncologists on our study team (MT, NL). Through this process, we identified themes that led to the development of five key attributes (each with three to four levels). The medical oncologists on our study team confirmed the clinical face validity of the proposed attributes and levels. The survey and DCE were then pretested in a sample of women from our focus groups (n=3) and in a convenience sample from the general population (n=11) using interviews and cognitive testing (one-on-one ‘think aloud’ methods). Pretesting was done to test understanding of survey content and obtain feedback on survey questions; preferences were not collected during survey pretesting. On the basis of pretesting feedback, we changed and simplified some of the terminology and wording of the background information, survey questions, attributes and levels. The updated survey was pilot tested in an online sample of n=55 women from the general population. Pilot testing was conducted to test online survey programming, duration to complete the survey, ensure respondents were able to complete the survey online and that there were no questions where dropouts were consistently occurring. Minor changes were made based on pilot test findings. The final attributes and levels used in the survey are presented in table 1. The final survey was translated into French and reviewed by French-language speakers to ensure the translation was correct. The final survey consisted of four main parts: (1) experiences with cancer, breast cancer and chemotherapy; (2) background information discussing breast cancer, chemotherapy and GEP (how GEP works and how to interpret results; figures 1, 2 and 3); (3) the DCE (figure 4 and table 1) and (4) demographics. The survey was administered in both French and English.

Table 1.

Attributes and levels used in the discrete choice experiment

Attribute	Levels
1. Your doctor's estimate of risk of cancer returning (without using a GEP test)	Low risk of cancer returning (eg, lymph node negative, the cancer is not responsive to Herceptin, the cancer is highly responsive to hormone therapy, tumour size is between 1.1 and 2.0 cm) Intermediate risk of cancer returning (eg, lymph node negative, the cancer is not responsive to Herceptin, the cancer is less responsive to hormone therapy, tumour size is between 2.1 and 3 cm) High risk of cancer returning (eg, lymph node negative, the cancer is responsive to Herceptin, the cancer is less responsive to hormone therapy, tumour size is between 3.1 and 5 cm)
2. Trust in your cancer treatment doctor	Do not trust Somewhat trust Totally trust
3. Likelihood of temporary side effects (may commonly occur during chemotherapy—may include nausea, vomiting, numbness or tingling in fingers, hair loss, fever and infection)	Low Moderate High
4. Likelihood of permanent side effects (may occur after chemotherapy and last permanently—may include leukaemia (blood cancer), damage to the heart muscle and early menopause)	Low Moderate High
5. GEP test score Likely benefit from chemotherapy	9, Low benefit 22, Uncertain benefit 44, High benefit GEP test not available

GEP, gene expression profiling.

Figure 1.

Breast cancer background information that was presented to respondents prior to completing the discrete choice experiment section of the survey. HER2, human epidermal growth factor receptor 2.

Figure 2.

‘Your doctor's estimated risk of recurrence’ attribute background information that was presented to respondents prior to completing the discrete choice experiment section of the survey. HER2, human epidermal growth factor receptor 2.

Figure 3.

‘GEP test score and likely benefit from chemotherapy’ attribute background information that was presented to respondents prior to completing the discrete choice experiment section of the survey. GEP, gene expression profiling.

Figure 4.

A sample choice task from the discrete choice experiment survey examining chemotherapy treatment decisions for early-stage breast cancer in the sample of n=1004 women from the general population. GEP, gene expression profiling.

We used Sawtooth Software SSI Web V.7.0.26 to create 50 survey versions with 12 choice tasks (10 random, 2 fixed) (Sawtooth Software SSI Web v. 7.0.26 [program]. v. 7.0.26 version. Orem, UT: Sawtooth Software Inc., 2011). We generated the choice task experimental design using the balanced overlap method to permit estimation of main effects and interactions, and D-optimal procedures to maximise statistical efficiency (Sawtooth Software Inc, 2011).³⁸ Since our sample size for the pilot test was small, we did not use the priors as a basis to inform the experimental design of the final survey. For each choice task, the respondent was asked to choose one situation out of three alternatives in which they would most likely select chemotherapy. Subsequently, to assess whether the alternatives were considered as realistic choices, there was an opt-out question in conjunction with each choice task where they were asked to indicate if they would actually choose chemotherapy if that was really their situation (figure 4).

Statistical analysis

Descriptive statistics (means for continuous variables, percentages for categorical variables) were used to examine the demographic and breast cancer experience variables. There were no missing data.

Hierarchical Bayes (HB) methods were used in the final analysis to estimate the part-worth utility of each attribute for individual survey respondents. HB was used because it provides part-worth utility coefficients for each respondent and greatly reduces the independence from irrelevant alternatives phenomenon inherent in the multinomial logit. HB uses the Markov chain Monte Carlo iterative algorithm to estimate individual respondent coefficients at the lower level of the hierarchical model using a multinomial logit algorithm and then a sharing of information among respondents at the higher level using a multivariate normal algorithm. The HB algorithm was set to run through 10 000 iterations during the burn-in phase and 20 000 iterations during the estimation period. The attribute part-worth utility coefficients were rescaled to be zero centred, with higher values denoting stronger preferences. We used χ² tests to determine if choice patterns deviated from uniform and paired t tests to determine if the mean part-worth utilities were different within attributes. The Brown-Forsythe test was used to test for statistically significant differences in part-worth utilities for multilevel covariates and t values were used for bilevel covariates. p Values <0.05 were considered statistically significant.

We used simulations and sensitivity analyses to estimate the relative influence of the five attributes, and various combinations of the attributes, on the likelihood of choosing chemotherapy treatment in alternative scenarios. The option to opt out of choosing chemotherapy is embedded within the simulation and sensitivity analyses.

Results

A total of 1004 respondents completed the survey. The mean age of respondents was 49 years (median, 51 years; table 2). Two-thirds of respondents (n=637) had at least some postsecondary education, over half (n=580) were employed, 67% were married or living common-law, 67% were without children, 42% were middle income ($20 000–79 999) and 84% were white (table 2). Overall, our sample is older and more educated than reports by Statistics Canada.^39–43

Table 2.

Demographics and cancer experiences of respondents (n=1004, 5 of which were in French)

Variable	n	Per cent
Average age	1004	49 years
Education
Less than high school/high school diploma	367	37
Apprenticeship or college diploma(s)/some university	358	36
University degree(s)	279	28
Employment status (current paid work)
Yes (outside the home and self-employed)	580	58
No (unemployed, retired, other)	424	42
Population centre
Small population centre (<29 999)	230	23
Medium population centre (30 000–99 999)	187	19
Large urban population centre (≥100 000)	587	58
Marital status
Married/living common-law	675	67
Widowed/separated/divorced/single	329	33
Children living at home
Yes, with children	335	33
Race*
White	841	84
First Nations, Metis or Inuit	19	2
Chinese	61	6
Black	7	1
All other	54	5
2011 Household income*
<$19 999	36	4
$20 000–79 999	421	42
≥$80 000	348	35
Cancer experiences
Know someone who had chemotherapy for cancer	791	79
Cared for someone else with cancer	348	35
Know of a relative(s) who has, or had, breast cancer	489	49
Know someone other than a relative who has, or had, breast cancer	740	74
Know someone who had GEP testing for breast cancer	51	5

*Respondents could also choose ‘prefer not to answer’.

GEP, gene expression profiling.

More than 75% of respondents know someone who had chemotherapy for cancer but fewer than 40% reported they had cared for someone with cancer (table 2). Approximately half of respondents know of a relative who had/has breast cancer and more than 70% know someone other than a relative who had/has breast cancer. Only 5% know someone who had GEP testing for breast cancer (table 2).

On average, 85% of respondents confirmed their choice about chemotherapy in the survey choice tasks based on their responses to the opt-out question about having chemotherapy, indicating that the choices presented in the survey were realistic and meaningful to the respondents. The part-worth utilities for attribute levels are plotted in figure 5. The GEP test score and likely benefit from chemotherapy attribute has the greatest utility relative to the remaining attributes—doctor's estimate of the risk of cancer returning (without using a GEP test), trust in your cancer doctor and likelihood of side effects (temporary and permanent).

Figure 5.

The part-worth utilities of the levels for each attribute (with 95% confidence limits), representing the relative influence of each on the choice of undergoing chemotherapy for early-stage breast cancer in the sample of n=1004 women from the general population. GEP, gene expression profiling; SE, side effects; MD, cancer treatment doctor.

The part-worth utility estimates of GEP test scores were in line with expectations, with a low GEP score of 9 having the lowest (and negative) part-worth utility (and associated with a low likelihood of choosing chemotherapy) and a high GEP score of 44 having the highest part-worth utility (and a high likelihood of choosing chemotherapy) (table 3). The part-worth utilities of the GEP test not being available and the GEP score being uncertain are similar and it is likely that neither influences the decision to choose chemotherapy.

Table 3.

Mean part-worth utility values (rescaled as zero centred) for the n=1004 respondents

Attributes and levels	Mean part-worth utility	SE	t Values	Mean significantly different from 0
MD's clinical risk estimate
Low risk	−29.81	2.55	−11.68	***
Intermediate risk	0.20	0.89	0.22	NS
High risk	29.61	2.48	11.95	***
Trust in MD
Do not trust	−25.54	0.98	−26.10	***
Somewhat trust	−0.73	0.67	−1.09	NS
Totally trust	26.27	0.92	28.55	***
Likelihood of temporary SE
Low likelihood	2.47	0.75	3.27	**
Moderate likelihood	7.55	0.60	12.60	***
High likelihood	−10.02	0.75	−13.34	***
Likelihood of permanent SE
Low likelihood	33.75	1.10	30.59	***
Moderate likelihood	12.25	0.76	16.13	***
High likelihood	−46.00	1.22	−37.66	***
GEP score, likely benefit from chemotherapy
GEP not available	−17.34	0.86	−20.18	***
9, Low benefit	−43.39	1.21	−35.73	***
22, Uncertain benefit	−6.83	0.89	−7.65	***
44, High benefit	67.56	1.52	44.48	***
General preference for chemotherapy vs no chemotherapy	−49.53	5.74	−8.63	***

Conducting paired t-test among attribute levels within each attribute, all pairs were significantly different at p<0.001.

**Significant at p<0.01; ***Significant at p<0.001.

GEP, gene expression profiling; MD, cancer treatment doctor; NS, not significant; SE, side effects.

Likelihood of permanent side effects was next most influential on respondents’ likelihoods of choosing chemotherapy; a treatment having a higher likelihood of permanent side effects would lower the likelihood of the respondent choosing chemotherapy. The doctor's estimate of risk of cancer returning (without using a GEP test) also strongly influenced chemotherapy decisions of respondents and demonstrated a monotonically increasing effect as the doctor's estimate of risk increased from low to intermediate and then to high risk. The effects were well spaced and were statistically different (p<0.001). Trust in your cancer treatment doctor followed a similar pattern as the doctor's estimate of risk of cancer returning. The likelihood of temporary side effects was least influential in the chemotherapy decision.

The simulation examines three profiles with trust in the cancer treatment doctor and likelihood of temporary and permanent side effects held constant at the moderate level in all profiles:

• High-risk profile—the doctor's estimate of risk of cancer returning (without a GEP test) is high and the GEP test score and likely benefit from chemotherapy was also high (44).

• Moderate-risk profile—the doctor's estimate of risk of cancer returning (without using GEP) was intermediate and the GEP test score and likely benefit from chemotherapy was uncertain (22).

• Low-risk profile—the doctor's estimate of risk of cancer returning (without using GEP) was low and the GEP test score and likely benefit from chemotherapy was low (9).

The simulation estimates that 78% would choose chemotherapy in the high-risk profile, whereas only 55% would choose chemotherapy in the moderate-risk profile and 33% in the low-risk profile (figure 6).

Figure 6.

Likelihood of choosing chemotherapy for early-stage breast cancer under different doctor's estimate of risk and GEP test score simulation scenarios in the sample of n=1004 women from the general population. All other attribute levels were held constant (trust in cancer treatment doctor=somewhat trust, and both likelihood of temporary and permanent side effects=moderate). GEP, gene expression profiling; SE, side effects; MD, cancer treatment doctor.

The graphs summarising likelihood of choosing chemotherapy were similarly shaped, but of different levels of magnitude, among the three levels for the doctor's estimate of risk (figure 6). In each scenario, the lowest likelihood of choosing chemotherapy was for the lowest GEP score of 9. The likelihoods were similar between the ‘GEP tests not available’ and ‘22, uncertain benefit’ levels over the three doctor's estimate of risk levels. As expected, the highest likelihoods of choosing chemotherapy occurred when the GEP score was 44. While the likelihoods increased with the level of clinical risk when the GEP test score was 44, the likelihood for high clinical risk was similar to intermediate clinical risk (77.9 vs 77.7). This suggests that the GEP score has more weight at the high GEP score level for those at intermediate clinical risk.

The part-worth utilities were significantly different in only a few of the attribute levels when split by several covariates (online supplementary appendix table S1). There were no statistically significant differences for employment, province of residence, age or language. There were only a few statistically significant differences for ‘know someone who has/had breast cancer’, income, ‘relative who has/had breast cancer’, marital status and whether there were ‘children in the household’ with no particular pattern overall (online supplementary appendix table S1).

Supplementary table

Tests for differences in part-worth utilities for covariates using Brown Forsythe F for multi-level covariates and t-value for bi-level covariates for the n=1004 respondents.

bmjopen-2015-010981supp_table.pdf ^{(116.4KB, pdf)}

Discussion

Treatment of patients when the likelihood of benefit is low and there is a risk of side effects can cause harm and also generate unnecessary costs to the healthcare system.^{7 8} Concerns about overtreatment have grown as less severe forms of disease are treated and treatment benefits are overestimated.⁹ GEP is a form of personalised medicine that can help inform effective treatment decisions. We estimated the value of GEP in chemotherapy treatment decision-making for early-stage breast cancer in the context of benefit–risk trade-offs. We found that GEP is highly valued and strongly influences chemotherapy treatment decisions in a sample of women from the general population.

The strengths of our study are fourfold. To our knowledge, this is the first utility-based survey to obtain quantitative benefit–risk data on the value of GEP testing in breast cancer treatment decisions. Second, the hypothetical choice tasks had high face validity, demonstrated by the directionality, magnitude and consistency of responses, and the high rate of confirmation about chemotherapy decisions. Third, we had a large sample of women from the general population who may be at risk of breast cancer and are major stakeholders in a publicly funded healthcare system, although they were older and more educated. Individuals who are more educated may be more interested in receiving GEP information and older individuals may be inclined to more heavily weighted decisions based on doctor's estimate of risk. Last, our results can be generalised to the female population at risk of breast cancer in Canada who may ultimately face a decision about GEP.

Our study had some limitations. There is the possibility of incentive bias when using an online survey panel. However, this method and the use of incentives are common, and research exploring online surveys has demonstrated validity and reliability that is comparable to traditional methods.^{44 45} Second, the choice tasks were hypothetical scenarios and respondents’ choices may be different when faced with these decisions in real life. Last, the information provided within the survey was comprehensive and complex, which required a high level of understanding to complete the survey. It is possible that some respondents did not fully understand the survey. However, our pretesting confirmed that respondents were understanding the survey. Additionally, based on our analysis of the DCE opt-out question that asks if they would actually choose chemotherapy if that was really their situation, it would appear that the choice tasks were realistic and meaningful to respondents. However, it is well known that risk is complex to communicate.^{46 47} In this era of complexity of clinical decision-making that is introduced by GEP and other types of testing for personalised medicine, it is challenging for clinicians to communicate this information to patients. There may be an increased role for decision aids eliciting patient preferences and communicating risk to inform personalised treatment decisions.

GEP testing information has significant influence on the potential stated uptake of chemotherapy treatment but varies depending on clinical risk. Recent findings from the TAILORx trial reported that about 16% of women with breast cancer who met established guidelines for adjuvant chemotherapy based on clinicopathologic features had a recurrence score of 10 or lower (low risk) and had very low rates of recurrence at 5 years with endocrine therapy alone.⁴⁸ This is a group of women who could potentially be ‘restratified’ to no chemotherapy treatment based on GEP results. Our simulations of preferences examining the influence of GEP test scores on the likelihood of choosing chemotherapy suggest that GEP testing had the greatest influence for those at intermediate clinical risk. For example, a high GEP score would increase the likelihood of choosing chemotherapy from 55% to 78%, and a low GEP score would decrease the likelihood of choosing chemotherapy from 55% to 39%. From a population perspective, this ‘restratification’ of patients could reduce uncertainty for patients with intermediate risk and could reduce the use of adjuvant chemotherapy, side effects and the associated costs for some patients who would likely not benefit from treatment, as has been observed in a recent retrospective cohort study.⁴⁹

Public and patient preferences have implications for the use of personalised medicine in clinical practice; personalised medicine tests are more likely to be adopted if they are valued by patients, the public and physicians. The findings from our study of public preferences are consistent with studies that have examined patient attitudes towards GEP testing. Bombard et al³⁶ found that patients valued GEP because ‘it provided them with certainty amidst confusion, with options and a sense of empowerment and with personalized, authoritative information’. In a study of patients with breast cancer, GEP testing is found to be acceptable and more than 60% indicated that the main reason for undergoing GEP testing was to ‘access the most appropriate treatment’.⁵⁰ Many studies have also found that GEP influences or changes chemotherapy treatment decisions.^18–26 Furthermore, patients with cancer are willing to pay increasing amounts to avoid chemotherapy side effects.⁵¹

Health policymakers require evidence that demonstrates the value of GEP to support policy decisions regarding funding and reimbursement, and are increasingly incorporating public values into these decisions.³² We found that GEP is highly valued and strongly influences chemotherapy treatment decisions in a sample of women from the general population but varies depending on clinical risk. Specific individuals may be more or less likely to change their chemotherapy treatment decisions. Clinicians should be aware of these differences and consider the use of GEP testing accordingly. Future research should examine preferences and the value of GEP in different population groups, such as patients with breast cancer, medical oncologists and subgroups of these populations who may have different preferences, to understand heterogeneity within these populations.