Long-Term Health Outcomes of a Decision Aid: Data from a Randomized Trial of Adjuvant! in Women with Localized Breast Cancer

Abstract

Purpose

Women with localized breast cancer face difficult decisions about adjuvant therapy. Several decision aids are available to help women choose between treatment options. Decision aids are known to affect treatment choices and may therefore affect patient survival. The authors aimed to model the effects of the Adjuvant! decision aid on expected survival in women with early stage breast cancer.

Patients and Methods

Data were obtained from a randomized trial of Adjuvant! (n =395). To calculate the effects of the decision aid on survival, the authors used the Adjuvant! survival predictions as a surrogate endpoint. Data from each arm were entered separately into statistical models to estimate change in survival associated with receiving the Adjuvant! decision aid.

Results

Most women (~85%) chose a treatment option that maximized predicted survival. The effects of the decision aid on outcome could not be modeled because a small number of women (n =12, 3%) chose treatment options associated with a large (5%–14%) loss in survival. These women—most typically estrogen receptor positive but refusing hormonal therapy—were equally divided between Adjuvant! and control groups and were not distinguished by medical or demographic factors.

Conclusions

Expected benefit from treatment is a key variable in understanding patient behavior. A small number of women refuse adjuvant treatment associated with large increases in predicted survival, even when they are explicitly informed about the degree of benefit they would forgo. Investigation of the effects of decision aids on cancer survival is unlikely to be fruitful due to power considerations.

Women with localized breast cancer must make difficult decisions about adjuvant therapy, trading off possible survival benefits against decreases in short-term quality of life. Studies have consistently shown that breast cancer patients vary as to the improvement in survival that they would require to opt for adjuvant therapy, although many women would choose adjuvant treatment even for a very small increase in survival.^1–3

Several decision aids have been developed to help women make decisions about treatment for localized breast cancer. Decision aids are interventions designed to help patients make specific and deliberative choices among treatment options by providing information, often quantitative in nature, about the options and their likely outcomes.⁴ Several decision aids have been evaluated in randomized trials. In a typical study, Whelan and others⁵ randomized 176 women with early stage breast cancer to a standard medical consultation or to a consultation plus a “decision board” that gave quantitative information about the effects of adjuvant therapy on survival. A number of similar trials have been summarized in a Cochrane Collaboration systematic review.⁴ Patients randomized to decision aids tended to have greater knowledge, more realistic expectations, and lower decisional conflict compared with controls. Evidence was less clear on the effects of decision aids on anxiety scores and health care choices.

We are interested in the effects of decision aids on long-term health outcomes. We have previously reported that, overall, breast cancer patients using a decision aid were less likely to use adjuvant therapy than controls.⁶ It is possible that this would have effects on cancer survival: if a large group of women who stand to gain only a small benefit from adjuvant therapy accordingly decide against such treatment after using a decision aid, at least some will experience a cancer recurrence and die prematurely as a result. On the other hand, we have reported that patients with high-risk cancer are more likely to choose adjuvant therapy after exposure to a decision aid.⁶ This suggests that use of a decision aid might, overall, improve survival rates.

Between 1998 and 2001, two of us (LAS, PBP) conducted a trial in which women with early stage breast cancer and no contraindications to either chemotherapy or tamoxifen were randomized to receive a decision aid known as Adjuvant! or a pamphlet that provided general information about adjuvant therapy for breast cancer (control group). The initial results of our trial on decision making have been previously reported. In brief, patients found the decision aid easy to understand (>95%), and ~85% reported that it influenced their decision making; physicians reported that it helped them to understand both patient preferences (75%) and patient prognosis (~80%).⁷ In the current study, we aimed to model the effects of the Adjuvant! decision aid, compared to a general information pamphlet, on survival in women with early stage breast cancer, using the Adjuvant! survival predictions as a surrogate endpoint. The justification for using this surrogate is that Adjuvant! has been shown to be well calibrated⁸: for example, in a group of 100 women given a 10-year survival probability by Adjuvant! of, say, 92%, close to 92 will be alive after 10 years.

METHODS

Study Design

The study was conducted at 14 practices in 2 large metropolitan communities in 2 states. Stratified by community v. academic status, physicians’ practices were randomized to either Adjuvant! or to a control group that used a general informational pamphlet. Patients were eligible for the study if they had been diagnosed with breast cancer, completed their primary surgical treatment, were candidates for adjuvant therapy (chemotherapy, hormonal, or combination therapy), and had no prior history of breast cancer. All patients eligible for the study were asked to participate in the study prior to seeing their medical oncologist for the first time. Written informed consent was obtained from all patients and physicians participating in this study. A total of 395 patients and 56 oncologists completed the data collection process. This study was conducted under the supervision of all relevant institutional review boards. The consultations were observed and audiotaped. A detailed description of the trial and its methods is reported elsewhere.^6,7

Decision Tool

The data presented to patients were from Adjuvant! Version 2.2, originally developed by Ravdin and Siminoff⁹ to help physicians estimate the benefits of adjuvant therapy. The program is designed to produce predictions of outcome with and without therapy, based on estimates of individual patient prognosis and estimates of the efficacy of different adjuvant therapy options. Estimates of benefit are obtained by calculating the reduction of risk of negative outcomes for individual patients. This is accomplished by estimating a patient’s risk of negative outcome (death or relapse) and then multiplying by the proportion of negative events that a given adjuvant therapy is known to prevent. Estimates of prognosis are based mainly on Surveillance, Epidemiology, and End Results Registry (SEER) estimates for breast cancer patients in the general population in the United States. Estimates of the efficacy of therapy are based mainly on the proportional risk reductions from the 1998 Early Breast Cancer Trialists Collaborative Group meta-analysis of the efficacy of adjuvant therapy for breast cancer.⁹

The computer program presents this information on the screen for the physician and as printed pages for use with patients. For example, having input patient and tumor characteristics, a patient might be informed that, for every 100 similar women who received no adjuvant therapy, 48 would be alive, 43 dead from cancer, and 9 dead from other causes in 10 years; however, if they chose hormone therapy, there would be “10 out of 100 women alive because of added therapy.” The program and its use have been described in detail elsewhere.⁹

Statistical Analysis

There were 2 key endpoints in our principal analysis: adjuvant treatments and deaths. In our comparison of Adjuvant! with control, we considered 2 scenarios. The 1st was that the decision aid would reduce the number of patients opting for adjuvant therapy but increase deaths. This might occur if treatment decisions were similar between groups except for women with only a small survival advantage from adjuvant therapy, a smaller proportion of whom opted for treatment in the Adjuvant! group (scenario 1 in Table 1). The 2nd scenario was that the Adjuvant! decision aid would reduce both the number of patients receiving adjuvant therapy and the number of deaths. This might be the case if not only did women with a small survival advantage forgo chemotherapy but, among patients who would benefit greatly from adjuvant therapy, a larger proportion decided to undergo treatment after using the decision aid (scenario 2 in Table 1). In the latter scenario, Adjuvant! would “dominate” the control intervention, and we would recommend use of the decision tool. In the case of scenario 1, where there is a tradeoff between lowered intervention rates but increased deaths, decision-analytic methods could be used to determine the value of the Adjuvant! decision aid.

Table 1.

Example Scenarios for the Effects of a Decision Aid on Outcome

Number of Women	Predicted Survival without Adjuvant Therapy, %	Predicted Survival with Adjuvant Therapy, %	Proportion Choosing Adjuvant Therapy in Decision Aid Group	Proportion Choosing Adjuvant Therapy in the Control Group	Predicted Deaths in Adjuvant! Group, n	Predicted Deaths in Control Group, n
Scenario 1
350	79	80	55%	85%	71.6	70.5
350	70	75	88%	88%	89.6	89.6
150	60	70	92%	92%	46.2	46.2
150	30	45	95%	95%	83.6	83.6
1000			781	886	291	290
Scenario 2
350	79	80	55%	85%	71.6	70.5
350	70	75	92%	88%	88.9	89.6
150	60	70	96%	92%	45.6	46.2
150	30	45	98%	95%	83	83.6
1000			806	886	289	290

In the first scenario, the only effect of the decision aid is that some women with little to gain from adjuvant therapy forgo treatment. This leads to an additional 1 death per 1000 patients. In scenario 2, the decision aid also increases the proportion of high-risk women choosing treatment, leading to a decrease in the death rate. The number of deaths is calculated using the following formula: number of women ×proportion choosing adjuvant therapy ×(1 − survival with adjuvant therapy) +number of women ×(1 − proportion choosing adjuvant therapy) ×(1 − survival without adjuvant therapy).

Determining the effects of Adjuvant! on the number of patients choosing adjuvant therapy is straightforward, as we know the treatment choices made by women in the randomized trial. To calculate the effects of Adjuvant! on survival, we used an approach based on counterfactuals. We first used data from the Adjuvant! group to predict the probability of each woman’s treatment choice—combined chemotherapy and hormonal therapy, chemotherapy alone, hormonal therapy alone, no treatment—using as the independent variables the Adjuvant! predictions and, in the case of hormonal therapy, estrogen receptor status. We then applied this model to the patients in the control group. Multiplying the predicted probability of each treatment choice by the predicted probability of survival associated with each treatment choice gave the predicted survival of each control patient, had she received the decision aid. This can be compared with predicted survival given the actual treatment choice, to calculate a difference in predicted survival associated with randomized allocation. This process was then reversed to obtain the difference in predicted survival for patients in the Adjuvant! decision aid group, had they been assigned to control.

As a secondary analysis, we investigated whether Adjuvant! affected decision making as expected. We first created 2 variables for each patient: potential treatment benefit was defined as the predicted improvement in survival associated with chemotherapy plus hormonal therapy compared with no adjuvant therapy, and actual treatment benefit was calculated as the predicted improvement in survival associated with a patient’s actual treatment choice compared with no therapy. Thus, actual and potential treatment benefits were equal for patients who chose combined therapy; actual treatment benefit was zero for patients who refused all adjuvant treatment. Our hypothesis was that for any given level of potential treatment benefit, actual treatment benefit would be different in the Adjuvant! group compared with controls. For example, we thought that a patient who was given an Adjuvant! prediction of a 20% improvement in survival from chemotherapy and hormonal treatment would be more likely to use adjuvant therapy than a control patient given less quantitative information, such as being informed only that she was at high risk and would benefit greatly from treatment. Conversely, we thought that a patient given an Adjuvant! prediction of a 1% improvement in survival from treatment would be less likely to use adjuvant therapy than a control patient. The statistical hypothesis is that the interaction term between potential treatment benefit and assignment to Adjuvant! decision aid or control in a model predicting actual treatment benefit would be positive in sign. All analyses were conducted using Stata 9.2 (Stata Corp., College Station, Texas).

RESULTS

Baseline data on the study sample have been reported previously.⁶ Patients had a median age of 62 years (interquartile range 54, 70), were predominately white (81%), were married (64%), and had at least a high school education (84%; 30% college educated). Approximately half of the sample (196, 51%) had private health insurance, with 143 (37%) covered by Medicare and the remainder by a variety of other public sources; no patient was without insurance.

Treatment choices by study allocation are given in Table 2: slightly more patients in the control group used at least one form of adjuvant therapy (91% v. 84%; P =0.045 by Fisher’s exact test). The interaction term between potential treatment benefit and allocation was, as hypothesized, positive in sign (0.073; 95% confidence interval [CI]: −0.0049, 0.15; P =0.066). The interaction term met conventional levels of statistical significance if we used nonlinear terms to model the effects of expected benefit on treatment decisions (P =0.027 for quadratic regression; P =0.035 for restricted cubic splines). Figure 1 shows the benefit of treatment in the control group subtracted from the benefit of treatment in the Adjuvant! group graphed against the maximum benefit of treatment. Women with little to gain from adjuvant treatment tended to use less therapy in the Adjuvant! decision aid group compared with controls: where maximum benefit of treatment is less than 5%, the difference in benefit of treatment chosen is negative. Conversely, women with a large increase in predicted survival from adjuvant tended to use more therapy in the Adjuvant! group: above 5%, the difference in benefit of treatment chosen is positive. This suggests that the Adjuvant! decision aid affected decisions in an appropriate direction.

Table 2.

Treatments by Randomized Group

	Control Group, n (%)	Adjuvant! Decision Aid, n (%)
No adjuvant therapy	14 (9)	36 (16)
Hormonal therapy	73 (46)	96 (42)
Chemotherapy	28 (18)	34 (15%)
Combined chemotherapy and hormones	45 (28)	60 (27)
Total	160 (100)	226 (100)

Figure 1.

Difference between groups in predicted outcome of treatment choice by maximum benefit. Light gray lines indicate 95% confidence intervals.

The models used to predict treatment choice had good discrimination: area under the curve (AUC) was between 0.85 and 0.94 for all models, with the exception of the model predicting use of no adjuvant treatment in controls (AUC of 0.74). However, initial data analyses showed that the data were very skewed, such that our models were of questionable validity. Although the predicted change in survival associated with the Adjuvant! decision aid was less than 1% for approximately 80% of control patients, a small number had very large differences, such as 12% or 13%. Similarly, a small number of patients in the Adjuvant! group were predicted to have had large improvements in survival had they been randomized to control.

To understand these findings, we analyzed the difference between predicted survival with combined therapy and predicted survival with the treatment choice made by the patient (Table 3). Approximately 60% of women chose a treatment option that maximized survival, and a further 25% chose a treatment option where predicted survival was only 1% less than maximum. However, 12 women (3%) chose treatment options associated with an absolute loss in predicted survival of 5% or more percentage points.

Table 3.

Effects on Predicted Survival of Treatment Choice Made, by Randomized Group

	Control Group, n (%)	Adjuvant! Decision Aid, n (%)
Maximized predicted survival	106 (66)	132 (58)
≤1% loss in predicted survival	31 (19)	58 (26)
2% loss in predicted survival	9 (6)	22 (10)
3%–4% loss in predicted survival	8 (5)	8 (4)
≥5% loss in predicted survival	6 (4)	6 (3)
Total	160 (100)	226 (100)

It was an unexpected finding that a nontrivial number of women (n =12) would not choose treatment associated with an important improvement in survival. Characteristics of the 12 women are shown in Tables 4 and 5. There were no obvious differences between this subgroup and the rest of the sample in terms of demographic factors or insurance (Table 4). Moreover, the group is equally split between the Adjuvant! and control groups. Table 5 shows that the most common scenario was for an estrogen receptor–positive woman to receive chemotherapy alone, even though combination therapy would have substantially improved her predicted survival. In a number of cases, patients chose chemotherapy when hormonal monotherapy would have been more effective.

Table 4.

Demographic Characteristics of the 12 Women Choosing Treatment Options Associated with a 5% or Greater Loss in Predicted Survival

Allocation	Ethnicity	Age, y	Years of Education	Marital Status	Doctor Code	Insurance	Treatment Chosen	Loss in Predicted Survivala
Control	Hispanic	43	13	Married	2036	Private	Chemotherapy only	6
Control	White	50	16	Married	1043	Private	Chemotherapy only	13
Control	White	57	12	Married	2060	Social Security disability	Chemotherapy only	13
Control	White	58	12	Married	1069	Private	Chemotherapy only	8
Control	White	62	12	Widowed	1007	Private	Chemotherapy only	12
Control	Hispanic	69	13	Married	2036	Medicare	No adjuvant therapy	5
Decision aid	White	43	18	Married	1502	Private	Chemotherapy only	5
Decision aid	African American	51	16	Divorced	2544	Private	Chemotherapy only	14
Decision aid	White	51	12	Divorced	2526	Government	No adjuvant therapy	5
Decision aid	Hispanic	52	3	Married	2568	Medigap	Chemotherapy only	6
Decision aid	White	62	12	Married	2518	Private	No adjuvant therapy	6
Decision aid	White	77	9	Widowed	2544	Medicare	No adjuvant therapy	5

^aCalculated by subtracting predicted survival on treatment chosen by patient from predicted survival had the patient received chemotherapy plus hormonal therapy.

Table 5.

Clinical Characteristics of the 12 Women Choosing Treatment Options Associated with a 5% or Greater Loss in Predicted Survival

Allocation	Age, y	ER Status	Tumor Size (cm)	Number of Positive Nodes	Tumor Differentiation	Predicted Survival				Treatment Chosen	Loss in Predicted Survivala
						No Adjuvant	Chemotherapy	Hormones	Combined
Control	43	Positive	5	0	Missing	62	74	71	80	Chemotherapy	6
Control	50	Positive	4	5	Moderate	32	36	46	49	Chemotherapy	13
Control	57	Positive	10	13	Moderate	4	10	15	23	Chemotherapy	13
Control	58	Positive	4	0	Poor	60	63	69	71	Chemotherapy	8
Control	62	Positive	3	7	Poor	29	34	42	46	Chemotherapy	12
Control	69	Positive	3	0	Poor	57	58	61	62	No adjuvant	5
Decision aid	43	Positive	3	0	Moderate	76	78	82	83	Chemotherapy	5
Decision aid	51	Positive	10	9	Poor	14	21	15	35	Chemotherapy	14
Decision aid	51	Positive	1	0	Moderate	86	88	89	91	No adjuvant	5
Decision aid	52	Positive	1	1	Moderate	73	77	79	83	Chemotherapy	6
Decision aid	62	Negative	3	0	Poor	70	76	70.5	76	No adjuvant	6
Decision aid	77	Negative	2	1	Moderate	44	49	45	49	No adjuvant	5

^aCalculated by subtracting predicted survival on treatment chosen by the patient from predicted survival had the patient received chemotherapy plus hormonal therapy.

DISCUSSION

In this study, we attempted to model the effects of a decision aid on long-term survival in women with breast cancer. We have found that a small number of breast cancer patients, approximately 3% in our sample, forgo adjuvant therapy predicted to increase markedly their chance of survival. Most typically, these were women with estrogen receptor–positive breast cancer who received chemotherapy without hormonal therapy. We found no obvious differences between these women and the rest of the sample with respect to measured covariates. In particular, a decision aid that gave an individualized risk prediction did not have an apparent effect on predicted survival. In other words, some women refused hormonal therapy even if they were given explicit, quantitative information that not using hormonal therapy would considerably increase their risk of death.

The efficacy of adjuvant systemic therapy in reducing the risk of breast cancer recurrence has been well demonstrated.¹⁰ Despite this, a proportion of patients do not receive adjuvant therapy. Use of adjuvant chemotherapy has been shown to decline substantially with age, independent of comorbidity and disease characteristics.^11–13 Although use of tamoxifen and other hormonal therapies has increased over time,^14,15 one study found that 20% of women with estrogen receptor–positive disease aged 65 or older received no adjuvant therapy.¹⁶

Omission of adjuvant systemic therapy may be reasonable when it is due to contraindications, low risk of recurrence, or patient preference. However, several previous authors have raised concerns about the failure of physicians to recommend adjuvant therapy to patients who are likely to benefit from it. Patients may not receive a recommendation for adjuvant treatment when they are not referred to a medical oncologist,¹⁷ when physicians believe that existing evidence of efficacy is not applicable,¹⁸ or when physicians are concerned about comorbid conditions and functional status.¹⁹

Previous authors generally have examined choice of adjuvant therapy independent of the benefit expected from treatment. We think that expected benefit is a key variable to understand patient behavior. For example, it is not difficult to understand why a woman might refuse chemotherapy if her expected survival benefit was only 1% to 2%. Indeed, we have found case reports in which clinicians have suggested that a 1% to 2% benefit might be insufficient to warrant the toxicities associated with chemotherapy.²⁰ A decision to refuse hormonal therapy at the expense of, say, an absolute 14% decrease in predicted survival likely requires an entirely separate set of explanations.

A key limitation of our study is that we do not know the exact reasons adjuvant therapy was not administered to any particular woman. Given the context of a randomized trial of a decision aid, we have assumed that this was a matter of the woman’s personal choice: all women were eligible for adjuvant therapy, had no contraindications to adjuvant therapy, had adequate insurance, and were seeing oncologists who routinely prescribed hormonal therapy or chemotherapy. Nonetheless, we have no way to confirm the degree to which a woman’s decision might have been influenced by her physician or by other factors. Moreover, we are unable to determine the degree to which any woman refusing highly beneficial adjuvant therapy did so despite fully appreciating her decrease in predicted survival or because she failed to understand the decision aid. A second, natural limitation is that our results are specific to the Adjuvant! decision aid and may not generalize to other types of decision tools used to counsel patients with breast cancer.

Our findings have several implications for research. First, it appears that a direct investigation of the effects of a decision aid on cancer survival is unlikely to be fruitful. This is because almost all women (85%) either chose the adjuvant therapy regimen that maximized their predicted survival or an option associated with only a 1% decrease. As the effects of decision aids on patient decision making are subtle, and the effects of the decision on survival are moderate, the effects of a decision aid on outcomes are likely to be too small to be detected by a trial of feasible size: detecting a 0.5% difference in survival at 10 years, for example, would require a trial accruing well in excess of 100,000 patients. Nonetheless, we recommend that investigators examine the effects of decision aids on health outcomes for endpoints other than cancer survival.

A more fruitful line of investigation with respect to cancer outcomes would be an in-depth examination of the small number of women who forgo importantly beneficial treatment. Given the public emphasis on the toxicities of chemotherapy—from hair loss and premature menopause to cognitive dysfunction and the risk of second cancers—we were somewhat surprised that avoidance of hormonal therapy was the most common cause of decreases in predicted survival. This cannot be explained in terms of contraindications, as eligibility for both chemotherapy and hormonal therapy was an inclusion criterion for the trial. Moreover, women forgoing important treatment benefit had no obvious medical or demographic features that distinguished them from the rest of the sample. Accordingly, the decisions made by this group of women can only be understood by carefully examining the consultations between patients and physicians and by asking patients in more detail about the reasons for their choices.