Development and Validation of a Simulation Model–Based Clinical Decision Tool: Identifying Patients Where 21-Gene Recurrence Score Testing May Change Decisions

PURPOSE

There is a need for industry-independent decision tools that integrate clinicopathologic features, comorbidities, and genomic information for women with node-negative, invasive, hormone receptor–positive, human epidermal growth factor receptor-2–negative (early-stage) breast cancer.

METHODS

We adapted an extant Cancer Intervention and Surveillance Modeling Network simulation model to estimate the 10-year risk of distant recurrence, breast cancer–specific mortality, other-cause mortality, and life-years gained with chemoendocrine versus endocrine therapy. We simulated outcomes for 1,512 unique patient subgroups based on all possible combinations of age, tumor size, grade, and comorbidity level; simulations were performed with and without 21-gene recurrence scores (RSs). Model inputs were derived from clinical trials, large US cohort studies, registry, and claims data. External validation was performed by comparing results to observed rates in two independent sources. We highlight results for one scenario where treatment choice may be uncertain.

RESULTS

Chemoendocrine versus endocrine therapy in a 65-69-year-old woman with a small (≤ 2 cm), intermediate-grade tumor, and mild comorbidities provides a 1.3% absolute reduction in 10-year distant recurrence risk, with 0.23 life-years gained. With these tumor features, a woman like this will have a 28% probability of having an RS 16-20, 18% RS 21-25, and 11% RS 26+. If testing is done, and her RS is 16-20, chemoendocrine therapy reduces 10-year distant recurrence risk to 1%, with 0.20 life-years gained, a similar result as without testing. The absolute benefits would increase to 4.8%-5.5% if the RS was 26+. The model closely reproduced observed rates in both independent data sets.

CONCLUSION

Our validated clinical decision tool is flexible, readily adaptable to include new therapies, and can support discussions about genomic testing and early breast cancer treatment.

INTRODUCTION

More than 180,000 women in the United States are annually diagnosed with hormone-receptor–positive (HR+), human epidermal growth factor receptor-2–negative (HER2–), axillary lymph node–negative breast cancer.¹ Up to 85% of these women could remain distant recurrence-free at 10 years with adjuvant endocrine therapy alone, and the remainder may benefit from chemoendocrine therapy. Historically, adjuvant chemotherapy decisions have been guided by individual clinicopathologic features such as tumor grade and size, considering the woman's age and comorbidities.² However, clinicopathologic features may provide inconsistent prognostic information^3-5 and may not always be sufficient to identify women who could benefit from chemotherapy. Gene-expression profile tests such as the 21-gene recurrence score (RS) assay (Oncotype DX test; Exact Sciences UK Ltd, London, United Kingdom) have shown to provide more predictive and prognostic information to guide chemotherapy decisions than the use of clinical and pathologic features alone.^3-7

CONTEXT

• Key Objective

• The goal of this study was to develop and validate an independent clinical decision tool that integrates individual, clinical, and pathologic characteristics with 21-gene recurrence score assay information to guide adjuvant chemotherapy treatment decisions in women diagnosed with node-negative, invasive, hormone receptor–positive, human epidermal growth factor receptor-2–negative (early-stage) breast cancer.

• Knowledge Generated

• The novel simulation model–based web clinical decision tool (BTxChoice) provides the 10-year risk of distant recurrence and life-years gained with chemoendocrine versus endocrine therapy considering a woman's age, tumor size, tumor grade, and comorbidities with and without 21-gene recurrence score test results. The model closely reproduced observed rates in two independent data sets.

• Relevance

• The BTxChoice tool provides useful information to facilitate shared decision making about the use of genomic testing and adjuvant chemotherapy in early-stage breast cancer.

The landmark Trial Assigning Individualized Options for Treatment (TAILORx)⁸ demonstrated that the 21-gene RS test could stratify women into groups that would benefit from chemotherapy.^6,8 However, TAILORx did not provide outcomes for all possible combinations of RS results, patient (age and comorbidities) and clinical-pathologic characteristics. Clinicopathologic features may be especially important when a woman's RS is close to the TAILORx cutpoints and/or when tumor features such as tumor size and grade suggest a different prognosis than the RS result. Furthermore, in practice, not all women receive genomic testing.

In this context, the ASCO now recommends providing data for use in shared decision making about chemotherapy.⁹ There are presently several treatment decision tools available to clinicians,^10-14 but only one industry-based tool includes 21-gene RS and clinicopathologic data.¹⁰ Unfortunately, that tool requires the RS results and does not consider comorbidity that could limit benefits and increase harms of chemotherapy.

To fill this gap, we adapted and validated a Cancer Intervention and Surveillance Modeling Network breast cancer simulation model to power a new clinical decision tool (BTxChoice, Washington, DC).^15-17 This independent tool provides estimates of 10-year risk of distant recurrence, breast cancer–specific mortality, other-cause mortality, and life-years gained with chemoendocrine versus endocrine therapy for individual women based on age, tumor size, grade, and comorbidity level with and without 21-gene RS results. This tool is flexible, includes the range of uncertainty for each outcome, fills current clinical needs, and facilitates personalized treatment decisions in HR+, HER2–, node-negative breast cancer.

METHODS

We adapted an extant breast cancer model (Model-GE)¹⁵ developed within the Cancer Intervention and Surveillance Modeling Network to evaluate breast cancer outcomes in 1,512 subgroups representing possible combinations of individual (age and comorbidities) and clinicopathologic characteristics (tumor size and grade), considering estimates for each subgroup with and without 21-gene RS results. This model has been used previously to simulate and replicate TAILORx and other clinical trials, and the modeling approach is described in detail elsewhere.^15-17 The study was approved by the Georgetown University Institutional Review Board and was considered as exempt research based on use of deidentified data.

Model Inputs

The model input parameters and data sources are summarized in Table 1. Individual and clinicopathologic characteristics (eg, age, grade, tumor size, estrogen receptor or progesterone receptor status, and RS) were based on the joint distributions observed in individual-level deidentified trial data provided by the Eastern Cooperative Oncology Group–American College of Radiology Imaging Network Cancer Research Group.⁸

TABLE 1.

Input Parameters

Time-to-distant recurrence was defined as time from diagnosis to tumor recurrence at a distant site according to the Standardized Definitions for Efficacy End Points in Adjuvant Breast Cancer Trials (STEEP) criteria.^4,20,24 Contralateral disease, second primary cancers, and death without distant recurrence were considered censoring events.^4,20,24 Competing-risk models for distant recurrence and time-to-distant recurrence conditional on therapy and patient and clinicopathologic attributes were fitted to the National Surgical Adjuvant Breast and Bowel Project (NSABP) B14/B20 clinical trial data set (including only HER2– cases).^4,7,20,21 The competing-risk models included treatment (chemoendocrine v endocrine therapy), age (continuous), tumor size (≤ 1 cm; 1.1-2 cm; or > 2 cm), grade (low; intermediate; or high), RS (continuous), and two-way interaction terms between RS and age; RS and tumor size; and RS and grade for RS < 26. A separate competing-risk model was used for subgroups with RS of 26 and above that included an additional interaction term between treatment and RS based on previous literature indicating a greater chemotherapy benefit in RS 26+.^10,25 The subhazard ratios for predictive patient and tumor attributes were calculated using Fine-Gray methods.²⁶ A proportional-hazards cumulative incidence function was estimated from each competing risk model for a designated combination of reference values of the predictive patient and tumor attributes. These cumulative incidence functions are semiparametrically dependent on various attributes of the patient (age) and the tumor (tumor size, grade, and RS).

Similarly, competing risk models for breast cancer death and time-to-breast cancer death were fitted to the B14/B20 trial data set. Subhazard ratios and cumulative incidence functions for a set of reference values for the predictive attributes of breast cancer death were estimated from the competing risk models.

Since the NSABP trials were conducted several decades earlier, we used data from TAILORx^5,6,8 and the Oxford Overview^22,23 to adjust treatment effects to reflect current therapy. Specifically, we adjusted 5 years of endocrine therapy effects from tamoxifen to effects seen with aromatase inhibitors conditional on age,²³ and chemotherapy effects from cyclophosphamide, methotrexate, fluorouracil regimens to effects seen with anthracycline- and taxane-based regimens.²²

The prevalence of comorbidities conditional on age, and competing other-cause mortality conditional on age and comorbidities were simulated using a published input parameter derived from Medicare data for women age 65+ years,¹⁸ and observational data from cohort studies of patients with breast cancer for women age < 65 years.¹⁹ We assumed that comorbidity level was independent of RS. The comorbidity parameter was derived from the Charlson comorbidity index using Medicare claims data²⁷ and has four levels including none, mild, moderate, and severe comorbidities (Data Supplement, online only).

Model Overview

The simulation model uses an empiric Bayesian analytical approach that captures uncertainty in all predictors' effects on outcomes and the sampling variation. We modeled women from the point of having newly diagnosed breast cancer to death. The population were women with HR+, HER2–, invasive, node-negative breast cancer with tumor size ≤ 5.0 cm that have received lumpectomy (with radiotherapy) or mastectomy.

First, we simulated individual characteristics for each virtual woman based on the joint distributions of age, tumor size, grade, hormonal status (estrogen receptor or progesterone receptor), comorbidity level, and RS.¹⁶ Each woman could remain event-free, experience a distant recurrence, die of breast cancer or other causes conditional on her treatment (5 years of endocrine therapy or chemoendocrine therapy), age, RS, tumor size, grade, and comorbidity level. A woman with distant recurrence could die of breast cancer or other causes. Time-to-events for each virtual woman was identified by applying the subhazard ratios of the predictive attributes to the baseline cumulative incidence functions estimated using the competing-risk models described above. Distant recurrence and breast cancer death were modeled separately because of limited data that were available to model the distribution of time from distant recurrence to breast cancer death (Data Supplement).

We simulated the effects of endocrine versus chemoendocrine therapy for 1,512 subgroups defined by combinations of age (within 5-year bands), tumor size (≤ 2 cm or > 2 cm), grade (low; intermediate; or high), and comorbidity level (no; mild; moderate; or severe), all with RS (0-100 in increments of 5) or without RS.¹⁶ We chose 5-year age bands based on current computational capacity; using individual years of age and RS would have required more than 87,000 subgroups.

Analysis

First, we calculated the proportion of women belonging to each RS category within each subgroup defined by age (5-year bands), tumor size, and grade. Next, we estimated the 10-year distant recurrence rates and 95% CIs for chemoendocrine versus endocrine therapy using Kaplan-Meier curves for each subgroup with clinical-pathologic features alone (age, tumor size, and grade), and then repeated the analysis with clinical-pathologic features combined with RS results. Since chemotherapy generally prevents most recurrences within 5 years after diagnosis,⁶ only recurrences up to 10 years postdiagnosis were estimated.

Average life-years were calculated from the date of diagnosis to date of breast cancer death or age- or comorbidity-specific other-cause death with chemoendocrine or endocrine therapy. Life-years gained were estimated by calculating the difference in average life-years for chemoendocrine therapy versus endocrine therapy for each subgroup.

Uncertainty

We quantified the uncertainty related to sampling variability for any given input parameter value by replicating the simulation for each of the 1,512 subgroups up to 1,000 times. The replicates were each randomly assigned its own set of effects sampling from the prior distribution of the subhazard ratios derived from the competing-risk survival models. Since the precision for any given model input value depends on the number of simulations,¹⁶ we simulated samples sizes so that the variability (ie, SE) for the rate differences comparing chemoendocrine versus endocrine therapy was never greater than ± 1.5 SEs.

Model Validation

Independent validation of results was performed to confirm model accuracy.²⁸ The clinical advisors (J.A.S., A.W.K., and C.I.) reviewed the face validity of the model structure, inputs, and results. We assessed the external validity of the model using two independent data sources. First, we compared the 10-year distant recurrence rates stratified by RS, age, tumor size, and grade from the Kaiser Permanente Pathways data set¹⁹ (n = 2,071) to model-based estimates among women diagnosed with node-negative, HR+, HER2–, invasive breast cancer. Pathways is the largest modern US cohort study of distant breast cancer recurrence risk. Second, we compared model-based estimates with 10-year breast cancer–specific mortality rates by RS, age, tumor size, and grade among women diagnosed with node-negative, HR+, HER2–, invasive breast cancer from 2004 to 2013 in the SEER registry (n = 59,826).²⁹ The probability distribution of 21-gene RS results was also validated against the SEER data set.²⁹

RESULTS

The model-based tool BTxChoice is designed to provide estimates for women with and without RS results. This feature is useful where RS testing is not available or to illustrate how often obtaining the RS might lead to different treatment choices. For example, if the clinician is seeing a woman age 65-69 years with a 1.5-cm tumor with intermediate grade, it could be useful to know the probability that she has an RS of 26-30 or above. The tool provides these data.

Personalized Estimates of Absolute Chemotherapy Benefit for an Older Woman

The BTxChoice tool generates estimates for chemoendocrine versus endocrine therapy benefit for 1,512 subgroups of women; we summarize one example for an older woman. We selected a woman age 65-69 years for the first illustration because older women are under-represented in trials and treatment choices can be more complex for older versus younger women. In this example, the woman has mild comorbidities, and was diagnosed with a small (≤ 2 cm), intermediate grade, HR+ and HER2– tumor. If this woman decides to undergo RS testing, Figure 1A shows the probability distribution of RS results for women with her tumor characteristics: 20%, 69%, 6%, or 5% chance of belonging to a 0-10, 11-25, 26-30, or 31+ RS category, respectively.

FIG 1.

Personalized estimates for a woman age 65-69 years with mild (mild comorbidity includes life expectancy associated with a history of myocardial infarction, ulcer, or rheumatologic disease) comorbidities, diagnosed with a small tumor (≤ 2 cm), and intermediate-grade breast cancer. (A) The probability distribution of 21-gene RSs; (B) 10-year risk of distant recurrence for chemoendocrine versus endocrine therapy with and without 21-gene RS test results; (C) absolute chemotherapy benefit on 10-year risk of distant recurrence with and without 21-gene RS test results; and (D) average life-years (life-years calculated considering breast cancer–specific mortality conditional on treatment, tumor grade, tumor size, age, and 21-gene RS; and other-cause mortality conditional on age and comorbidity level) gained for chemoendocrine versus endocrine therapy with and without 21-gene RS test results. RS, recurrence score.

Based on this woman's age, tumor size, and grade, the 10-year risk of distant recurrence with endocrine therapy alone is 3.8% (SE: 0.26) (Fig 1B). With chemoendocrine therapy, the 10-year risk is reduced to 2.5% (SE: 0.21), resulting in an absolute chemotherapy benefit of 1.3% (SE: 0.37) (Fig 1C). On average, she could potentially gain 0.23 life-years (SE: 0.13) (84-days) with the addition of chemotherapy to endocrine therapy (Fig 1D). The overall results represent a weighted average across all RS results for women with these clinicopathologic characteristics.

These results may prompt this woman and her clinician to order a 21-gene RS test. If the RS result was 0-10, this woman's absolute reduction in 10-year risk of distant recurrence with chemoendocrine versus endocrine therapy alone is < 1% (SE: 0.51), which is lower than the average benefit of 1.3% (Fig 1C). If the RS result was 26-30, this woman's absolute reduction in 10-year risk of distant recurrence with chemoendocrine versus endocrine therapy alone is 4.8% (SE: 0.44), which is higher than the average benefit of 1.3%.

Finally, the tool provides the variation of life-years gained with chemoendocrine versus endocrine therapy. This woman could gain 0.63 life-years (approximately 230-days) (SE: 0.13) with chemoendocrine versus endocrine therapy if her RS result was 26-30 (Fig 1D).

Personalized Estimates of Absolute Chemotherapy Benefit for a Young Woman

The choice of chemoendocrine versus endocrine therapy can also be complex for women younger than age 45 years. This group has a long life-expectancy and can experience early menopause and other long-term treatment effects. There are also fewer clear-cut results on chemotherapy benefits for all possible combinations of clinicopathologic features and RS groups for younger women. In this example, we show results for a woman age 40-44 years with a small (≤ 2 cm), intermediate-grade tumor and no comorbidities.

There is an 18% chance that this woman could have an RS ranging from 21 to 25 and an 11% chance of having an RS > 25 (Fig 2A). Based on her age, tumor grade, and size alone, this woman has a 2.6% (SE: 0.21) absolute reduction in 10-year risk of distant recurrence with chemoendocrine versus endocrine therapy (Fig 2C).

FIG 2.

Personalized estimates for a woman age 40-44 years with no comorbidities, diagnosed with a small tumor (≤ 2 cm), and intermediate-grade breast cancer. (A) The probability distribution of 21-gene RSs; (B) 10-year risk of distant recurrence for chemoendocrine versus endocrine therapy with and without 21-gene RS test results; (C) absolute chemotherapy benefit on 10-year risk of distant recurrence with and without 21-gene RS test results; and (D) average life-years (life-years calculated considering breast cancer–specific mortality conditional on treatment, tumor grade, tumor size, age, and 21-gene RS; and other-cause mortality conditional on age and comorbidity level) gained for chemoendocrine versus endocrine therapy with and without 21-gene RS test results. RS, recurrence score.

If this woman decides to get tested and has an RS 21-25, chemoendocrine therapy will provide a 6.1% (SE: 0.44) absolute reduction in 10-year distant recurrence risk and 1.47 (SE: 0.13) added life-years (approximately 536-days) versus those with endocrine therapy alone (Fig 2D).

Model Validation

Model-based estimates were compared with observed 10-year distant recurrence and breast cancer–specific mortality rates in subgroups defined by age, RS, grade, and tumor size whenever the Pathways or SEER data sets had adequate sample sizes to provide stable estimates for the subgroup. The model-based estimates were similar to the observed rates in both the external data sources (Fig 3). In younger women, within each subgroup, the average estimated 10-year distant recurrence rate provided by the model approximates the cohort Kaplan-Meier estimate in the Pathways study, with a Lin concordance correlation of 0.968. In the SEER data set, the Lin concordance correlation was 0.957 and 0.968 for younger and older women, respectively.

FIG 3.

Model validation: distant recurrence rates at 10 years in the Kaiser Permanente Pathways study (n = 2,071) versus model-based estimates in women (A) age ≤ 50 years and (B) age > 50 years. Breast cancer–related death rates at 10 years in the SEER registry (n = 59,826) versus model-based estimates in women (C) age ≤ 50 years and (D) age > 50 years.

DISCUSSION

There are several web-based tools to support treatment decisions in breast cancer.^10-14 However, none of these tools integrate genomic results or comorbidity-specific life expectancy with patient clinical and tumor features. The recently developed RSClin tool¹⁰ provides the effects of chemoendocrine versus endocrine therapy on the 10-year risk of distant recurrence considering a woman's 21-gene RS and clinicopathologic characteristics. By contrast, our independently developed simulation model–based clinical decision tool BTxChoice provides additional data including the distribution of probable RS results, given a woman's age, tumor size, and grade; life-years gained considering the woman's age, tumor size, grade, and comorbidities; and 10-year distant recurrence risk with and without RS test results. This additional information could potentially help guide treatment decisions when a woman requests therapy at variance with her clinicopathologic features, comorbidities, or RS results.

This novel simulation model–based clinical decision tool can be useful in clinical practice in several ways. First, this tool can be used to guide treatment decisions even if RS testing is not available. In a resource-poor setting, these data could also potentially help guide the choice of when to use RS testing.^30,31 Second, when RS testing is available, the tool provides additional information about RS-category–specific benefits, given a woman's age, comorbidities, and clinicopathologic characteristics. Third, to our knowledge, none of the existing clinical tools provides treatment outcomes based on a woman's age- and comorbidity-specific life expectancy. The use of life-years gained as a complement to absolute risk reduction places the magnitude of chemotherapy benefit in terms of remaining life expectancy. This information could inform treatment decisions in older women where tumor features alone may not be sufficient. Fourth, the chemotherapy benefit shown in our model for RS 26 or higher is consistent with a recently published validation of the 21-gene RS test in a HER2– subset of the NSABP B-20 cohort.²⁵ While the TAILORx trial reported that there was no chemotherapy benefit on average in women with RS results of 11-25, subgroup analysis showed some chemotherapy benefit in younger women age ≤ 50 years with RS results of 16-20 and 21-25.⁸ Some of this benefit may be due, in part, to chemotherapy-induced early menopause.^5,10,32 The current tool provides individualized estimates of absolute chemotherapy benefit for women diagnosed with node-negative, invasive, HR+, HER2– breast cancer that are consistent with these previous studies. Ultimately, the decision about chemoendocrine versus endocrine therapy is a personal choice based on a woman's priorities and circumstances and the weight she places on the benefits and harms of treatment.

The summary of the tool presented here should be considered within the context of the limitations of the data sources used to inform model development. The distribution of distant recurrence and breast cancer–specific survival were derived from historical (B14/B20) trials. These trials do not have information related to individual comorbidities, so we relied on observational and claims data to incorporate comorbidities. The trials also did not have race-specific data, and racial minorities are under-represented in cancer trials.³³ It will be important to add race-specific data in future tool versions. Furthermore, we were unable to include menopausal status because of lack of this information in the historical trials used to derive model input parameters. Previous studies suggest that the absolute chemotherapy benefit in younger women (ie, ≤ 50 years) with an RS 16-25 may have resulted, in part, from chemotherapy-induced early menopause.^5,10,32 Also, we were unable to consider breast cancer outcomes for women who opt for 10 years of endocrine therapy or outcomes by mode-of-detection and chemotherapy regimen or treatment toxcities.^22,34,35 As more data become available in the future, the flexible model structure allows us to add new tool features such as the effects of specific types of chemotherapy and endocrine therapy and their effects on comorbidities. At present, the tool provides useful and validated information to support shared decision-making discussions. This tool will be tested for usability, acceptability, ease of interpretation, and utility in clinical encounters. After this testing, the tool will be made publicly available and disseminated to the clinical community via professional groups such as ASCO and the National Cancer Institute–funded cooperative groups.

Overall, this study demonstrates how simulation modeling can be used to combine evidence from various data sources to create a calculation engine for a clinical decision tool. This tool could potentially assist shared decision making and communication about treatment in node-negative, HR+, HER2– breast cancer during clinical encounters. In the future, as more clinical data become available, the tool will be further extended to reflect current information relating to the impact of chemotherapy conditional on age and RS in node-positive disease,³² other factors affecting recurrence risk (eg, obesity), and the effects of novel treatments (eg, cyclin-dependent kinase 4/6 inhibitors) in early-stage breast cancer to support the translation of rapidly evolving knowledge into clinical practice.

AUTHOR CONTRIBUTIONS

Conception and design: Jinani Jayasekera, Joseph A. Sparano, Suzanne O'Neill, Claudine Isaacs, Clyde B. Schechter, Jeanne Mandelblatt

Administrative support: Jinani Jayasekera

Provision of study materials or patients: Jinani Jayasekera, Joseph A. Sparano

Collection and assembly of data: Jinani Jayasekera, Jeanne Mandelblatt

Data analysis and interpretation: Jinani Jayasekera, Young Chandler, Claudine Isaacs, Allison W. Kurian, Lawrence Kushi, Clyde B. Schechter, Jeanne Mandelblatt

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Development and Validation of a Simulation Model–Based Clinical Decision Tool: Identifying Patients Where 21-Gene Recurrence Score Testing May Change Decisions

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).