Feasibility of Measuring Preferences for Chemotherapy among Early Stage Breast Cancer Survivors using a Direct Rank Ordering Multi-Criteria Decision Analyses versus a Time Trade-Off

Abstract

Objectives:

Chemotherapy is increasingly a preference-based choice among women diagnosed with early stage breast cancer. Multi-criteria decision analysis (MCDA) is a promising but underutilized method to facilitate shared decision-making. We explored the feasibility of conducting a MCDA using direct rank ordering versus a time tradeoff (TTO) to assess chemotherapy choice in a large population-based sample.

Methods:

We surveyed 904 early stage breast cancer survivors who were within 5 years of diagnosis and reported to the Western Washington State Cancer System and Kaiser Permanente Northern California registries. Direct rank ordering of 11 criteria and TTO surveys were conducted from September 2015-July 2016; clinical data were obtained from registries or medical records. Multivariable regressions estimated post-hoc associations between the MCDA, TTO, and self-reported chemotherapy receipt, considering covariates.

Results:

Survivors ranged in age from 25–74 and 73.9% had stage 1 tumors. The response rate for the rank ordering was 81.0%; TTO score was 94.2%. A one-standard deviation increase in the difference between the chemotherapy and no chemotherapy MCDA scores was associated with a 75.1% (95% CI 43.9%−109.7%, p<0.001) increase in the adjusted odds of having received chemotherapy; no association was found between the TTO score and chemotherapy receipt.

Conclusions:

A rank-order based MCDA was feasible and associated with chemotherapy choice. Future research should consider developing and testing this MCDA for use in clinical encounters. Additional research is required to develop a TTO-based model and test its properties against a pragmatic MCDA to inform future shared decision-making tools.

Précis:

Multi-criteria decision analysis but not time trade off based preferences were associated with chemotherapy receipt but had a lower response rate (81% versus 94%).

INTRODUCTION

Multi-criteria decision analysis (MCDA) is of collection of methods that provides decision makers with a structured transparent approach to consider multiple attributes simultaneously when choosing between treatment options [1,2]. MCDA measures how each alternative treatment option performs on several dimensions of value and summarizes the performance of each alternative into a single score, using preference weights provided by the decision maker. In healthcare, MCDA has experienced limited but growing applications in the areas of system-wide investment in prioritization [3], value-based health technology assessment [4], and facilitating shared-decision making [5–7]. MCDA’s limited uptake in healthcare has been attributed to respondent burden, and a lengthy and complex development process [4, 8]. Most MCDA frameworks exhibit a trade-off between sophisticated theory-based procedures that require more respondent time versus simpler easier to use a-theoretical assessments with potential methodological flaws [9]. A review of 41 MCDA studies in healthcare suggested that current applications favored easier-to-use weighting methods [10]. Rank ordering, which involves ranking decision criteria from most to least important, is a simple assessment procedure and has been recommended for use in shared decision making in busy practice settings [7]. However, recent work suggests that simple additive MCDA models used in conjunction with these weighting methods are inadequate for shared decision making and advocates the use of QALY models instead [11]. With growing interest in practical MCDA applications compared to direction preference elicitation for shared decision making, the real-life feasibility of various methods and their ability to assess the health care decisions of patients has great importance.

The decision to pursue adjuvant chemotherapy among patients newly diagnosed with early stage breast cancer provides a compelling yet untested context to explore the feasibility of conducting MCDA using direct rank ordering versus traditional direct preference elicitation such as the time-trade off (TTO) [12]. The adjuvant chemotherapy choice has long been recognized as a preference-sensitive decision requiring patients to trade-off the probability of therapeutic success, severity of side effects, logistics and costs of treatment, among other factors.[13–16]. However, the decision-making context has changed as new evidence suggests that only women with cancers that have a higher probability of recurrence based on gene expression profile testing may benefit [17,18]. A recent review of patient decision aids for early-stage breast cancer treatment found many current tools lacked simple, transparent methods to elicit patient values or preferences (e.g., value-clarification exercise)[19], contributing to their low uptake in practice among patients and physicians [20]. Currently, no breast cancer decision aids use MCDA with a direct rank ordering or QALY models although both these approaches are highly transparent and fulfill the required elements of shared decision making [11]. The TTO model is a standard approach to assessing patient ‘weights’ or utility values for QALY models and has recently been used to assess preferences for adjuvant chemotherapy among older breast cancer patients [15]..

The objective of this study is to explore the feasibility of conducting MCDA using direct rank ordering versus a time tradeoff (TTO) to assess chemotherapy choice in a large population-based sample of women with early stage breast cancer. We hypothesized that an additive MCDA with direct ranking order would be associated with chemotherapy receipt because it can account for multiple decision attributes, thus enhancing capture of a larger array of factors that may account for observed heterogeneity in patient preferences for chemotherapy.

METHODS

Setting and Population

The data for the current study is from a multi-site, collaborative National Cancer Institute-funded study evaluating preferences for gene expression profile testing (GEP) and chemotherapy among women with early stage breast cancer [21–24]. Participants were recruited from two geographically distinct populations: (1) the SEER Western Washington State Cancer Surveillance System (WA-CSS) catchment area [part of the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute]; (2) those enrolled in Kaiser Permanente Northern California (KPNC). The populations and practices are diverse in terms of race and ethnicity, represent a range of insurance types and health care settings, and include community and academic practices. Breast cancer survivors were eligible if they met criteria for GEP testing at the time of the study [i.e., stage I or II, lymph node negative, estrogen-receptor positive, and human epidermal growth factor 2 negative (HER2)], were between the ages of 25 and 74 at diagnosis and diagnosed between January 2, 2012 and May 22, 2016. Participants were identified from the KPNC tumor registry and electronic medical records and from the WA-CSS registry. For this survey, we selected an age-stratified random sample of GEP test-eligible survivors from the KPNC and WA-CSS groups, aiming for an equal proportion of those tested vs. non-tested, as well as representation from varying age groups. Informed consent was obtained from all individual participants included in the study. All procedures in the study involving human participants were in accordance with the ethical standards of site Institutional Review Boards and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Data Collection

We conducted cognitive pre-testing of the survey in a convenience sample of 8 breast cancer patients at Kaiser Permanente. All participants received a mailed recruitment letter with information about the study, a URL address linked to a web-based questionnaire and contact information for assistance. The KPNC letter also included a postage-paid return postcard with options to request a paper survey or decline participation. Non-respondents were mailed a second recruitment letter; and follow-up courtesy calls were placed. At KPNC, participants who did not respond on-line were automatically sent paper surveys. At WA-CSS, survivors who did not complete the survey online received courtesy calls starting one week after mailing. WA-CSS survivors who informed study staff during courtesy calls that they wished to complete paper-based surveys were sent questionnaires by mail. At both sites, individuals who completed the survey received a $20 gift card. Surveys were collected between September 2015 and July 2016.

Information regarding age, AJCC tumor stage (I or II), ER and PR status, and diagnosis year (2012–2016) was captured from the SEER registry in the WA-CSS sample and from medical records in the KPNC sample.

TTO and MCDA Preference Elicitation Measures and Score Determination

The TTO measured preferences for the number of healthy years a woman would be willing to give up to avoid experiencing the common side effects of chemotherapy [25]. These side effects included nausea and vomiting, fatigue, itchy and peeling skin, hair loss, and moderate joint pain (Appendix Table S1 in Supplemental Materials). Based on a ping-pong response pattern, women were asked to choose between 10 years of remaining life with the common side effects of chemotherapy and a range of 1 to 10 years in perfect health. Women who were indifferent between 1 year of perfect health and 10 years with side effects (TTO score = 1) had the lowest preference for chemotherapy (greater dislike for the side effects). Women who were indifferent between 10 years of perfect health (TTO score = 10) and 10 years of additional life with side effects had the highest preference. One additional scenario compared 10 years of perfect health against that same 10 years of perfect health plus an additional 10 years with side effects. The TTO score was set to 0 for the additional scenario, indicating the respondent placed no value on the additional 10 years of side effects. Therefore, TTO score ranged from 0 to 10, with a higher score representing a greater preference for chemotherapy.

The MCDA portion of the survey asked women to respond to the question: “Chemotherapy is an important issue for me because of the following:” Women ranked 11 criteria from most important (rank =1) to least important (rank = 11). Following ISPOR 2 guidelines [26, 27], the eleven criteria were identified from several sources including prior preference surveys, clinical input, and recurring themes that came up in focus group interviews [16, 18–20]. The common side effects of chemotherapy included Nausea/Vomiting, Hair Loss, and Fatigue, while the remaining criteria included trips to the hospital, out of pocket costs, reducing cancer recurrence risk, lifestyle concerns, and long-term effects of chemotherapy.

To calculate the MCDA scores for both the chemotherapy and no chemotherapy option, each respondent’s rankings from 1 to 11 were first converted into rank order centroid (ROC) weights (Appendix Table S2 in Supplemental Materials) [28, 29].The ROC weights ranged from approximately 0.27 (rank 1) to 0.008 (rank 11) and summed to 1 for each respondent The ROC methodology has low user burden and a high reliability compared to other approaches that rank order information [30]. In addition, it has been recommended for use in MCDA methods facilitating shared decision making [7]. Next, for each criteria, a utility weight was determined for both the chemotherapy and the no chemotherapy treatment options. The utility weights were derived from a combination of literature review and clinical consultation in the absence of literature [26,27]. All utility weights were scaled from 0 to 100 (100=highest utility) (Appendix Table S3 in Supplemental Materials) [28]. Lastly, each respondent’s ROC weights were multiplied by the criteria’s chemotherapy utility weight and added together to form a chemotherapy score. The ROC weights were also multiplied by each criteria’s no chemotherapy utility weight to form a no chemotherapy score. The differenced MCDA scores for each respondent was calculated as her chemotherapy score minus her no chemotherapy score. (Appendix S4 in Supplemental Materials). Equal chemotherapy and no chemotherapy scores (e.g. differenced MCDA score = 0) indicated a respondent was ‘indifferent’ between the two treatment options; a differenced MCDA score greater than (less than) zero indicated a stronger (weaker) preference for chemotherapy. For comparability with the TTO score, we rescaled the differenced MCDA score from 0 to 10, which mapped the indifferent cut point to 7.9.

Outcome Variable

The primary outcome was chemotherapy receipt (Yes, No) ascertained via patient-self report from the survey.

Covariates

We measured several covariates that might affect the relationship between the preference elicitation measures and our outcome. Covariates included age, self-identified race (Non-Hispanic White, Non-White), education (Some college or less, Associate or Bachelor degree, Graduate degree), and AJCC stage (I or II) at diagnosis [24]. We also controlled for self-reported receipt of a gene-expression profile test (Performed, Not Performed, Unknown) and time since diagnosis. We included an indicator for site to control for differences in care settings and regional practice patterns, and an indicator to control for the survey format chosen by the respondent (Paper, Electronic).

Analysis

We first compared characteristics of responders and non-responders to the survey using X² tests; among survey responders we examined missing or incomplete responses to the TTO and the MCDA items. The relationship between the TTO and the MCDA scores was evaluated using Spearman’s rank correlation. We evaluated the pairwise associations between chemotherapy receipt and the TTO and MCDA scores based X² and t-tests. Multivariable logistic regression was then used to model chemotherapy receipt (Yes, No) adjusted for the control variables listed above and TTO score only, MCDA score only, and both the TTO and the MCDA score. The Akaike and Bayesian Information Criterion was used to determine goodness of fit. We included log-likelihood ratio tests comparing the model with MCDA and TTO vs. TTO alone or vs, MCDA alone to examine whether models including both preference elicitation measures had a statistically significant improvement in fit over the model that included just one. As a sensitivity analysis, we tested a binary specification of the TTO (TTO = 0 vs TTO = 1–10) and a piecewise linear specification (knot at TTO=1) [20]. All analyses were completed in SAS version 9.4.

RESULTS

Study Population

A total of 1990 eligible survivors were contacted: 795 from northern California and 1,195 from Washington State. The overall survey completion rate was 45.4% (53.1% from northern California and 40.3% from Washington; Figure 1). Comparing responders with non-responders, women who completed the survey did not significantly differ from those who did in terms of age, stage, and diagnosis year. However non-responders were more likely to be Non-White (20.2% vs. 12.2%; P<0.001) and from the WA-CSS sample (65.7% vs. 53.3%; P<0.001).

Fig. 1: Cohort Diagram.

TTO = Time Trade Off. MCDA = Multi-Criteria Decision Analysis. GEP Test = Gene Expression Profile Test Receipt.

Valid responses to the MCDA item required a complete and correct ranking of all 11 criteria. Partial rankings (e.g. only ranking the 6 criteria) were excluded as were rankings that gave two different criteria the same number.

Demographic variables included age, race, education, AJCC stage, and years since diagnosis

From the sample of 904 respondents, 52 (5.8%) had a missing or partial response to the TTO item; 172 (19.0%) for the MCDA item. Compared with responders, non-responders to the TTO item were more likely to be 60 years or older (75.0% vs. 50.0%; P<0.001), from Washington State (67.3% vs. 52.5%; P<0.04), have completed some college or less (57.7% vs. 33.8%; P<0.001), and have completed a paper survey (78.8% vs. 13.8%; P<0.001). In contrast, non-responders to the complete MCDA item were more likely to be non-white (27.9% vs. 18.0%; P<0.01), from northern California (56.4% vs. 44.4%, P=0.01), and have completed some college or less (42.4% vs. 33.8%; P=0.03) (Appendix Table S5 in Supplemental Materials). In addition to excluding the 172 missing responses to the MCDA item and the 52 missing responses to the TTO question, we omitted 4 (0.4%) with a missing response to the chemotherapy receipt question and 1 (0.1%) additional respondent with a missing demographic question.

The final analytic sample (N=694) was largely (81.4%) non-Hispanic white, approximately half (48.4%) were at least 60 years old, and over half (55.8%) had earned a Bachelor’s degree or higher. In addition, most women were one to three years from diagnosis (76.4%) and had been diagnosed with Stage I cancer (73.9%) (Table 1).

Table 1:

Characteristics, Chemotherapy Preferences, and Receipt of Chemotherapy Among Hormone Receptor Positive, Early Stage Disease Breast Cancer Survivors

	Survivor Characteristics (N = 694)
Age
25–49		145 (20.9%)
50–59		213 (30.7%)
60–69		266 (38.3%)
70–74		70 (10.1%)
Race
White (Non-Hispanic)		565 (81.4%)
Non-White		129 (18.6%)
Education
Some College or Less		223 (32.1%)
Associate or Bachelor’s Degree		270 (38.9%)
Graduate or Professional Degree		201 (29.0%)
Care Setting
Integrated Care California		315 (45.4%)
Washington SEER		379 (54.6%)
AJCC Stage
Stage 1		513 (73.9%)
Stage 2		181 (26.1%)
Self-Reported GEP Test Receipt
Performed		300 (43.2%)
Not performed		187 (26.9%)
Unknown		207 (29.8%)
Years Since Diagnosis
< 1		18 (2.6%)
1 – 2		150 (21.6%)
2 – 3		380 (54.8%)
3 – 4		18 (17.0%)
> 4		28 (4.0%)
Survey Format
Paper		93 (13.4%)
Electronic		601 (86.6%)
	Chemotherapy Preferences (N = 694)
	Mean (SD)	Median [IQR]
Diff. MCDA Score (0 – 10) a	6.6 (2.4)	7.6 [5.0, 8.5]
TTO Score (0 – 10)	3.4 (3.8)	0.0 [ 0.0, 7.0]
Chemotherapy Preferences by Chemotherapy Receipt
	Chemotherapy (N=167)	No Chemotherapy (N=527)
Diff. MCDA Score (0 – 10) a, Mean (SD)	7.5 (1.2)	6.4 (2.4)b
TTO Score (0 – 10), Mean (SD)	3.9 (3.9)	3.2 (3.7)c

TTO = Time Trade Off Score, MCDA = Multi-Criteria Decision Analysis. IQR = Interquartile Range. See Appendix 1 for survey items.

^aDifferenced MCDA score was calculated as the chemotherapy score – no chemotherapy score, re-scaled from 0 – 10. An increasing differenced MCDA score corresponds to an increasing preference for chemotherapy.

^bP value <0.001

^cP value = 0.05

MCDA and TTO Preferences for Chemotherapy

The highest ranked (most important) criteria in the MCDA item were “Fighting my Cancer” and “A Chance to Reduce my Risk for the Cancer Coming Back” (median rank = 2) (Table 2), followed by “Long Term Effects of Chemotherapy” (median rank = 3). The lowest ranked criteria were “Trips to Hospital/Clinic for Treatment” and “Out of Pocket Cost for Chemotherapy” (median rank = 9). The three criteria describing common side effects of chemotherapy, including “Probability of Nausea/Vomiting”, “Probability of Hair Loss”, and “Probability of Fatigue”, were ranked in the middle (median rank = 6, 8, 6, respectively).

Table 2:

Survivor Rankings of the Eleven Criteria in the Multi-Criteria Decision Analysis Item for Chemotherapy Preference (N=694)

Criteria	Median (IQR)
Fighting my Cancer	2 (1, 4)
A Chance to Reduce my Risk for the Cancer Coming Back	2 (1, 4)
Long Term Effects of Chemotherapy	3 (2, 6)
Probability of Fatigue	6 (4, 7)
Probability of Nausea/Vomiting	6 (4, 8)
Inability to Maintain Lifestyle	6 (4, 8)
Caring for Family	7 (4, 9)
Probability of Hair Loss	8 (5, 10)
Inability to Work	8 (5, 10)
Trips to Hospital/Clinic for Treatment	9 (7, 10)
Out of Pocket Cost for Chemotherapy	9 (7, 11)

IQR = Interquartile Range

To calculate the differenced MCDA score, each respondent’s rankings from 1 to 11 were converted into rank order centroid (ROC) weights. For each criteria, a utility weight was determined for both the chemotherapy and the no chemotherapy treatment options. Each respondent’s ROC weights were multiplied by the criteria’s chemotherapy utility weight and added together to form a chemotherapy score. The ROC weights were also multiplied by each criteria’s no chemotherapy utility weight to form a no chemotherapy score. The differenced MCDA score was calculated as the chemotherapy score minus the no chemotherapy score, rescaled from 0 to 10.

Next, we converted the criteria rankings into the differenced MCDA score (i.e., chemotherapy MCDA score minus no chemotherapy MCDA score; range 0–10). We found the the mean (SD) of the differenced MCDA score was 6.6 (2.4) (Table 1). To provide context, women with a differenced MCDA score of 7.9 were ìndifferent’ between choosing chemotherapy or not (Appendix S4 in Supplemental Materials). In our sample, approximately 56.8% of respondents preferred to avoid chemotherapy as indicated by a differenced MCDA score less than 7.9.

The mean (SD) of the TTO score was 3.4 (3.8), however, there was a large skew in the results (Table 1). Approximately half of the respondents (51.9%) reported no preference for chemotherapy (TTO=0), indicating they were not willing to trade any years of perfect health for 10 years of life expectancy with the common side effects of chemotherapy. In comparison, women with some preference for chemotherapy (48.1%), were willing to trade 10 years with the common side effects of chemotherapy for between 1–10 years of perfect health.

The TTO score was weakly but significantly correlated with the MCDA score (Spearman’s rank correlation = 0.11: P<0.01). Figure 2 displays the nearly independent relationship between the TTO and the MCDA scores and their associations with chemotherapy receipt. The figure illustrates that among the 300 women who had an MCDA score greater than 7.9 (some preference for chemotherapy), 148 of them indicated no preference for chemotherapy according to the TTO score (TTO=0).

Fig. 2: Scatter Plot of the Time Trade Off vs the Differenced Multi-Criteria Decision Analysis Scores for Chemotherapy Preference.

TTO = Time Trade Off. Diff. MCDA = Differenced Multi-Criteria Decision Analysis Score (0–10). N=694

Associations of Preferences and Chemotherapy Receipt by Preference Elicitation Measure

Approximately one quarter (24.1%; 167/694) of women reported receiving chemotherapy. Overall, the relationship between the TTO score and chemotherapy receipt was not statistically significant in either the unadjusted (Table 1) or the adjusted analysis (OR 1.03, 95% CI 0.98–1.08, p=0.29). (Table 3, Model 1). In sensitivity analyses, neither the binary specification nor the piecewise linear specification of the TTO were statistically significant (Appendix Table S6 in Supplemental Materials).

Table 3:

Adjusted Odds of Chemotherapy Receipt Associated with Time Trade Off and/or Differenced Multi-Criteria Decision Analysis Scores for Chemotherapy Preference (N=694)

Preference Elicitation Method	Model 1: TTO Score Only		Model 2: Diff. MCDA Score Only		Model 3: TTO and Diff. MCDA Scores
	Odds Ratio a (95% CI)	P-value	Odds Ratio a (95% CI)	P-value	Odds Ratio a (95% CI)	P-value
TTO Score	1.03 (0.98–1.08)	0.30	--	--	1.00 (0.95–1.07)	0.77
Diff. MCDA Score b	--	--	1.32 (1.18–1.46) c	<0.001	1.32 (1.19–1.46) c	<0.001
AIC	665		633		635
BIC	751		720		726
Likelihood Ratio Test (Model 3 vs. Model 1) p <0.001 Likelihood Ratio Test (Model 3 vs. Model 2) p = 0.83

TTO = Time Trade Off (0–10). Diff. MCDA = Differenced Multi-Criteria Decision Analysis Score (0–10). AIC = Akaike Information Criterion. BIC = Bayesian Information Criteria.

^aChemotherapy vs. No Chemotherapy Receipt. Adjusted for age, race, education, healthcare delivery site, AJCC stage, gene expression profile receipt, years since diagnosis, and survey format.

^bSee Table 2 for the calculation of the differenced MCDA score. An increasing differenced MCDA score corresponds to an increasing preference for chemotherapy.

^cPer 1-point increase in the differenced MCDA score

In contrast, the differenced MCDA scores were significantly higher in unadjusted analysis for women who received chemotherapy compared to those that did not (7.5 vs. 6.4; P<0.001) (Table 1) and higher differenced MCDA scores were significantly associated with a higher adjusted odds of chemotherapy receipt (OR = 1.32; 95% CI 1.18–1.46, P<0.001) (Table 3, Model 2). A one-standard deviation (2.4 point) increase in the differenced MCDA score was associated with a 75.1% (95% CI 43.9%−109.7%, p<0.001) increase in the odds of receiving chemotherapy. Figure 3 illustrates the predicted odds of chemotherapy receipt as a function of the differenced MCDA score. The addition of the TTO score to the differenced MCDA score did not alter the association between the MCDA score and chemotherapy receipt (Table 3, Model 3).

Fig. 3: Predicted Likelihood of Chemotherapy Receipt as a function of the Differenced Multi-Criteria Decision Analysis Score.

Differenced MCDA score was calculated as the chemotherapy MCDA score – no chemotherapy score, re-scaled from 0 – 10. Predicted for the following respondent: aged 60–69 years old, non-Hispanic white, Bachelor’s Degree, Washington SEER region, Stage 1, received gene expression profile test, 2–3 years from diagnosis. N=694

DISCUSSION

This preliminary study examined the feasibility of conducting MCDA using direct rank ordering versus a time tradeoff (TTO) capturing the common side effects of chemotherapy to assess adjuvant chemotherapy choice in a large population-based sample of women with early stage breast cancer. Our MCDA results suggested that on average women ranked the longer-term impact of their treatment decision (e.g. “Fighting my cancer”, “Long term effects of chemotherapy”) as more important than the side effects experienced during treatment, while the trips to the hospital and out-of-pocket costs were their lowest priority. We found that MCDA-based preferences, but not TTO-based preferences, were significantly associated with chemotherapy receipt. However, the response rate to our 11-criteria rank ordering exercise was lower than the TTO (81% versus 94%).

Our direct rank ordering and TTO based preferences were largely consistent with the previous patient preference studies, but with some notable exceptions [13–16]. Our finding that on average women ranked the longer-term impact of their treatment decision as the most important criteria supports the literature that many patients judged small survival benefits sufficient to make chemotherapy worthwhile [16,31]. However, estimates of the proportion of women prioritizing length of life varies across studies and may be as high as 50% [14] One the other hand, the TTO illustrated that while approximately half of patients were willing to trade length of life for quality of life (e.g. TTO >0), 52% would decline chemotherapy if it provided an additional 10 years of life with the side effects of chemotherapy. Previous research found that the estimates of patients who would refuse chemotherapy regardless of the benefits were as high as 24% [16]

In light of the direct rank ordering results, our finding that the differenced MCDA scores exhibited greater association with adjuvant chemotherapy receipt than TTO scores should be viewed in the context of several considerations. First, the different associations may be explained by the different criteria included in the respective methods. Specifically, the MCDA results were impacted by the high ranking women placed on the longer-term impact of their treatment decision, which were not captured in the TTO score.. Thus, our inclusion of different criteria may have disadvantaged the TTO score. Our finding that the TTO score was not associated with chemotherapy receipt in women with early stage breast cancer contrasts with two previous studies [13,14], which asked women to choose between a fixed survival time without chemotherapy and a variably longer survival time with chemotherapy. In these studies, many women required only modest survival gains (< 12 months) to choose chemotherapy. Behavioral science suggests that reframing the TTO question as a loss of survival time versus a gain may influence patient responses [32]. There is currently no scientific consensus on the optimal specification of the TTO task, but differences in the specification may result in incomparable health state values [33]. In addition, the TTO score includes individual only derived utility scores and while the MCDA combined individual ROC weights and population mean utility scores raising additional complexity in comparing these two approaches. Future research could explore how a TTO score that models a gain in survival time or a wider chemotherapy experience and includes life years to estimate a QALY model performs compared to this MCDA model.

Several limitations should be considered in evaluating our findings. We relied on self-report of chemotherapy. The respondents in this study were more likely to be white and more highly educated than then general US population of women with early stage breast cancer. As this was a preliminary study, we did not conduct the full lengthy and complex development process as recommended by ISPOR guidelines [26,27]. Our preferences were assessed after the treatment experience, which may limit their utility in pre-treatment counseling. We used an online and in person approach to administer both the TTO and MCDA questions, per patient preference. The use of two survey methods was chosen to facilitate response but may have influenced our findings. Nearly one fifth of the survey respondents either skipped the MCDA item or did not complete it correctly, suggesting that ranking 11 criteria may be a difficult cognitive task. Currently, there is no scientific consensus on the optimal number of criteria [26,27]. Our decision to include 11 criteria was informed by a recent review which found an average of 8.2 criteria were used to assess interventions, with the number of criteria ranging from 3 to 19 [10], and our pilot testing. Future iterations may consider fewer criteria. Following the recommendations of the previous literature [28,30], our testing of the MCDA also only considered the direct ranking for eliciting women’s relative value weights. Other pragmatic approaches for eliciting values (e.g. direct scoring, cumulative voting) could improve the potential of MCDA models for use in clinical and other decision-making contexts [34–36].

CONCLUSION

These preliminary results suggest that despite the potential theoretical limitations, this pragmatic MCDA approach is feasible among a general population of breast cancer patients and yields results associated with chemotherapy receipt. We view these results as a preliminary step in the lengthy and complex process to develop a tool for shared decision making that leverages the strengths of MCDA to facilitate patient and clinician adoption [37]. Further research is required to develop a TTO-based model and test is properties against a pragmatic MCDA to inform future shared decision-making tools.

KEY POINTS FOR DECISION MAKERS.

Multi-criteria decision analysis (MCDA) applications and time tradeoff (TTO) offer two approaches to assess preferences for adjuvant chemotherapy to facilitate shared decision making among early stage breast cancer patients.

An MCDA- but not TTO-based preferences were significantly associated with breast cancer patients’ decision to receive chemotherapy. However, the ranking exercise in the MCDA appeared to be burdensome, with a lower response rate than the TTO (81% versus 94%).

The study suggests that MCDA may have value for use in clinical encounters. Further research is required to develop MCDA surveys that are both pragmatic to administer and informative as shared decision-making tools.

Data availability

The datasets generated and analyzed during the current study are not publicly available due to IRB protocol, but may be available from the corresponding author on reasonable request.