Breast Medical Oncologists' Use of Standard Prognostic Factors to Predict a 21-Gene Recurrence Score

The ability of oncologists to predict the Oncotype DX® recurrence score using standard prognostic criteria is examined.

Abstract

Background.

Half of all breast cancers are early stage, lymph node negative, and hormone receptor positive. A 21-gene (Oncotype DX®; Genomic Health, Inc., Redwood City, CA) recurrence score (RS) is prognostic for recurrence and predictive of chemotherapy benefit. We explored the ability of oncologists to predict the RS using standard prognostic criteria.

Methods.

Standard demographic and tumor prognostic criteria were obtained from patients with an available RS. Two academic pathologists provided tumor grade, histologic type, and hormone receptor status. Six academic oncologists predicted the RS category (low, intermediate, or high) and provided a recommendation for therapy. The oncologists were then given the actual RS and provided recommendations for therapy. Analysis for agreement was performed.

Results.

Thirty-one cases, including nine additional cases with variant pathology reads, were presented. There was substantial agreement in oncologists' ability to discriminate between true low or true intermediate and true high (κ = 0.75; p < .0001). Predictions between low and intermediate were not consistent. The most common discrepancies were predictions of a low RS risk when cases were true intermediate and predictions of an intermediate RS risk when cases were true low. The actual RS resulted in a change in the treatment recommendations in 19% of cases. Of the 186 scenarios and six oncologists in aggregate, five fewer chemotherapy recommendations resulted with the actual RS.

Conclusions.

Oncologists are able to differentiate between a low or intermediate RS and a high RS using standard prognostic criteria. However, provision of the actual RS changed the treatment recommendations in nearly 20% of cases, suggesting that the RS may reduce chemotherapy use. This effect was observed in particular in intermediate-risk cases. Prospective clinical trials are necessary to determine whether decisions based on the RS change outcomes.

Introduction

More than 200,000 cases of breast cancer are diagnosed in the U.S. annually [1]. Of these, about half are early-stage, lymph node–negative, hormone receptor–positive tumors. More public health campaigns for regular mammographic screening have increased detection of tumors in early stages, before regional or distant metastasis has occurred [2]. Although adjuvant clinical trials have clearly demonstrated that adjuvant chemotherapy prolongs survival for patients with early-stage breast cancer [3], retrospective analyses of some of these trials suggest that the benefit may be restricted to a subset of patients with either poor or negative expression of estrogen receptor (ER) [4] or patients with human epidermal growth factor receptor (HER)-2⁺ breast cancer [5].

Gene-expression profiling is commonly used as both a research tool and a clinical tool; this technology has been used by multiple independent groups to develop gene profiles that are associated with breast cancer recurrence [6, 7]. One of the most widely validated gene-expression profiles, the recurrence score (RS), is performed by Oncotype DX® (Genomics Health, Inc., Redwood City, CA). This is a 21-gene profile recommended by the American Society of Clinical Oncology as a biomarker for the identification of the risk for distant recurrence in patients with surgically treated, ER⁺, stage I or II, node-negative breast cancer [8]. Additionally, its use was suggested by the St. Gallen Consensus Panel [9] and the National Comprehensive Cancer Network (NCCN) [10]. An important limitation remains the absence of any prospective clinical studies reporting outcomes with its use. Because studies investigating the clinical utility of the RS and other profiles (e.g., MammaPrint and the breast cancer gene expression ratio) are ongoing, it remains important to consider whether conventional prognostic markers can suffice for providing clinical practice recommendations instead of the use of gene profiles [11].

The RS has been repeatedly validated as a prognostic marker [7, 12, 13] and additionally determined to be predictive of chemotherapy benefit [12, 14]. Independent retrospective analyses of the National Surgical Adjuvant Breast and Bowel Project B20 chemotherapy trial [14] and the Southwest Oncology Group 8814 clinical trial [12] have found that the benefit of adjuvant chemotherapy is most clearly shown in the subset of patients whose tumors were measured to have a “high-risk” RS.

Because the RS is derived from the expression of genes that measure ER, progesterone receptor (PR), HER-2, and proliferation, it was hypothesized that oncologists could predict the RS by assessing standardized immunohistochemical measures of ER, PR, and proliferation (grade). Therefore, we sought to determine whether academic oncologists specializing in the treatment of breast cancer, when presented with standard prognostic and predictive criteria as determined by academic breast pathologists, could identify patients independently determined to be high risk by means of the RS. Additionally, this study sought to determine whether knowledge of the RS would affect oncologists' recommendations for or against adjuvant chemotherapy in addition to hormonal therapy.

Methods

Using an institutional review board–approved protocol, patients from the Rochester, MN and Jacksonville, FL campuses of the Mayo Clinic with an available Oncotype DX® test result or who had the test performed at another institution but reviewed at these campuses were identified by conducting an electronic patient record database query. The medical records from these patients were reviewed to identify those patients with hormone receptor–positive, lymph node–negative invasive breast cancer. Tumor slides from these cases were reviewed by two academic breast pathologists who were blinded to previous pathology interpretations, the actual RS, and each other's analysis. Histologic type, Nottingham grade, and percentage ER⁺ and PR⁺ were recorded. Hormone receptor positivity was reported as 0%, 1%–10%, 11%–50%, 51%–90%, or >90%.

Patient characteristics, including patient age, tumor size, percentage ER⁺ and PR⁺, and HER-2/neu status, were presented to six academic oncologists from various institutions for review. While being blinded to the actual RS, reviewers were asked to predict each patient's RS category as representing either low (RS, 0–17), intermediate (RS, 18–30), or high (RS, 31–100) risk.

Additionally, each oncologist was queried as to their enthusiasm for recommending chemotherapy (reported as yes or no) in addition to hormonal therapy. Three weeks later, the case order was randomly mixed and then the clinical characteristics were provided along with the RS to the oncologists, with the question regarding their recommendation (yes or no) for chemotherapy in addition to hormonal therapy. In those cases in which there was a discrepancy between the pathologists' interpretations (e.g., in terms of tumor grade or percentage ER⁺), an “alternate read” was provided along with the primary read. These alternate reads were scrambled, along with the primary reads, as individual cases. This study was conducted in 2006. Patient characteristics were summarized using descriptive statistics (medians and ranges for continuous variables and counts and frequencies for categorical variables). Agreement among the six oncologists as well as between each oncologist and the actual RS risk group (low or intermediate versus high) was assessed via κ statistics [15].

To assess the effect on chemotherapy recommendations associated with being given the actual RS, the pairs of chemotherapy recommendations (without and with being given the actual RS) for individual cases for each oncologist were entered into a repeated measures general estimation equation (GEE) analysis [15]. To assess the effect of clinical characteristics and the actual RS risk group on the likelihood of a chemotherapy recommendation change, the change in chemotherapy recommendation (yes versus no) was modeled using logistic regression. Variables included in the model were oncologist, age (as a continuous variable), histology (ductal versus lobular), grade (I versus II of III), tumor size (as a continuous variable), HER-2/neu status, ER⁺ (<50% versus ≥50%), PR⁺ (≤10% versus >10%), and actual Oncotype DX® RS risk group (low versus intermediate versus high). Categories of grade, percentage ER⁺, and percentage PR⁺ were collapsed because of small sample sizes in some categories. Interaction between each clinical characteristic and oncologist was assessed in a separate model that contained a main effect for the oncologist, a main effect for the clinical characteristic being investigated, and an interaction effect between oncologist and the clinical characteristic. Because the interaction effect was not statistically significant in any model, no interaction effects between the oncologist and clinical characteristics were included in the multivariate model.

In all logistic regression models, model fit was assessed via the deviance. p-values < .05 were considered statistically significant throughout.

Results

Thirty-one distinct cases were identified with characteristics of these tumors outlined in Table 1. The median age of the patients was 53 years (range, 42–82 years). All cases were either ductal or lobular histology, with lobular comprising 23% of cases. All but two cases were HER-2/neu⁻. The median actual RS was 14 (range, 7–74) and the RS risk was classified as low in 18 (58%), intermediate in 10 (32%), and high in three (10%) cases. Cases presented to the surveyed oncologists are displayed in Table 2.

Table 1.

Clinical characteristics

Table 2.

Summary of 31 cases (original pathology read)

^a HER-2 for this case was read as 2+ by immunohistochemistry; fluorescence in situ hybridization studies showed “HER-2/neu to D17Z1 ratio of 3.4, 10% population with amplification … most of the remaining nuclei had HER-2/neu duplication with 2–3 copies of the chromosome 17 centromere and 3–10 copies of the HER-2/neu gene.”

Abbreviations: ER, estrogen receptor; HER-2, human epidermal growth factor receptor 2; Neg, negative; ODX, Oncotype DX®; PR, progesterone receptor; RS, recurrence score.

Frequency of Predicted RS Risk by Oncologists

Figure 1 demonstrates the percentage of RS risk groups predicted by each oncologist along with the actual percentage of RS risk groups. The percentages of predicted RS risk groups were in the range of 39%–77%, 16%–52%, and 3%–10% for the low-, intermediate-, and high-risk groups, respectively, for the six oncologists. There was a high level of agreement among oncologists (κ = 0.75; p < .0001) in predicting actual low or intermediate risk versus high risk. However, there was, at best, a moderate level of agreement between each oncologist and the actual RS risk level (range, 0.22–0.48) (low or intermediate versus high). The most frequent types of discrepancies were predicting a patient's risk to be intermediate when the true RS indicated a low risk (31 of 80 total discrepancies) and predicting a patient's risk to be low when the true RS indicated an intermediate risk (29 of 80 total discrepancies) (Table 3). In considering patients separately by actual RS risk group, the percent agreement between the actual and predicted scores was excellent for low-risk patients (low or intermediate versus high, 100%), moderate for intermediate-risk patients (low or intermediate versus high, 87%), and poor for high-risk patients (low or intermediate versus high, 33%).

Figure 1.

Percentage of predicted and actual Oncotype DX® (ODX) recurrence score (RS) risk groups by oncologists.

Table 3.

Types of discrepancies in predicting Oncotype DX® recurrence score

Recommendation for Chemotherapy

Based on the clinical data without the RS, five oncologists recommended chemotherapy in 16%–23% of the 31 cases, with one oncologist recommending chemotherapy in 52% of the 31 cases. Oncologists generally followed the rule of almost never (eight of 115, 7%) recommending chemotherapy in predicted low-risk cases and always (14 of 14, 100%) recommending chemotherapy in predicted high-risk cases. Predictions of intermediate risk were associated with chemotherapy recommendations in 25 of 57 (44%) cases.

When given the RS, the same five oncologists recommended chemotherapy in 10%–19% of the 31 cases, with one oncologist recommending chemotherapy in 58% of the 31 cases. Overall, 19% of the cases involved changes in management, with the most frequent change being from yes to no (Table 4) when given the RS. There were two cases (RSs of 14 and 32) in which five of the six oncologists changed their chemotherapy recommendation (Table 2) (case #6 and #24), three cases (RSs of 14, 22, and 40) in which four of six oncologists changed their chemotherapy recommendation (cases #3, #7, and #22), and 15 cases in which no oncologist changed their chemotherapy recommendation. When given the actual RS risk groups, oncologists recommended chemotherapy in 10 of 108 (9%) low-risk cases, 15 of 60 (25%) intermediate-risk cases, and 17 of 18 (94%) high-risk cases. In total, after being given the RS, there were five fewer recommendations for chemotherapy (47 recommendations when not given the RS versus 42 recommendations when given the RS in 186 scenarios). There was no significant association (p = .63) between a chemotherapy recommendation and the predicted or actual RS.

Table 4.

Types of changes in chemotherapy recommendation

In the repeated measures GEE analysis of the chemotherapy recommendation without and with being given the RS, there was no significant within-oncologist effect on the chemotherapy recommendation when given the RS (p-value = .56). The only significant parameter in the model was that of the one oncologist who recommended chemotherapy in substantially more cases than the other five oncologists without and with being given the RS (p-value < .0001).

In modeling the likelihood of a change in chemotherapy recommendation (Table 5), age, grade (I versus II or III), tumor size, and high RS risk were significantly associated with the likelihood of a change in chemotherapy recommendation (all p-values < .05). Higher age and grade I were associated with a lower likelihood of a change in chemotherapy recommendation, whereas larger tumor size and high RS risk were associated with a greater likelihood of a change in chemotherapy recommendation.

Table 5.

Logistic regression model of likelihood of change in chemotherapy recommendation

Model fit: deviance = 117.7 (degrees of freedom = 171; p-value = .9994).

^a Oncologist #6 was chosen as being the most accurate in predicting the actual RS.

^b p-value < .05.

Abbreviations: ER, estrogen receptor; HER-2/neu, human epidermal growth factor receptor 2; ODX, Oncotype DX®; PR, progesterone receptor; RS, recurrence score.

To explore this further, four oncologists were surveyed again 6 weeks after the initial analysis with the same cases in an unplanned fashion. When comparing these responses with those provided earlier, changes in the predicted RS risk were seen in 15% of cases, resulting in changes in the chemotherapy recommendation in 10% of cases. These data suggest that some of the changes observed may be associated with random day-to-day practice variability in prescribing patterns. Studies are needed to further characterize these types of variation in clinical practice and determine whether or not adherence to practice guidelines will improve patient outcomes.

Changes from Alternate Reads

In nine of 31 cases, the breast pathologists differed in their interpretation of the histologic findings. In one case, they differed in their interpretation of histology (ductal versus lobular); in four cases, they differed in their interpretation of grade (I versus II); and in eight cases, they differed in their interpretation of ER⁺ (51%–90% versus >90%). Additionally, in eight cases, differences in percentage PR⁺ were reported. The alternative pathology reads were associated with a change in a given oncologist's predicted RS in 19 of 54 (54%) scenarios. Eighteen changes were from low risk to intermediate risk or vice versa, and one change was from intermediate risk to high risk. The alternative pathology reads were associated with seven changes in the chemotherapy recommendation within the cumulative 54 cases (13%).

Discussion

This case series demonstrates that academic breast oncologists may be reasonably accurate at using standard prognostic criteria to distinguish high-risk patients from low- or intermediate-risk patients as determined by the RS. High agreement was observed among oncologists in predicting the Oncotype DX® RS, especially when designating an RS of low or intermediate versus high risk. There still remains significant inability to discern low- from intermediate-risk cases, as evidenced by the large number of discrepancies that occurred.

In this analysis, most oncologists did not recommend chemotherapy for predicted low RS patients, whereas they always recommended chemotherapy for predicted high RS patients. Predicted intermediate scores led to wide variability in treatment recommendations, but most oncologists did not recommend chemotherapy in patients whom they felt to be at intermediate risk. These practice patterns were also seen in a retrospective review of 285 patients in which RS and chemotherapy decisions were recorded in a multicenter, community-based, health care system [16]. In the current study, the added information of the actual RS changed the chemotherapy recommendation in 19% of cases, resulting in an overall decrease of five clinical scenarios for which chemotherapy would be recommended. Cases with alternative pathology reads were associated with changes in predicted RS risk in over one third of scenarios and were associated with changes in the chemotherapy recommendation in >10% of cases.

Others have performed similar analyses. Acs et al. [17] recently presented their findings on 154 patients with early-stage, ER⁺ breast cancer and an available RS in which surgical oncologists, medical oncologists, and pathologists were asked to estimate the risk for recurrence. Risk estimates agreed with the RS 54% of the time, with the most common discrepancy being overestimation (32%). Using the assumption that patients with a low- or intermediate-risk RS do not benefit from chemotherapy, and are thus “overtreated,” they concluded that 82% and 69% of patients would unnecessarily receive chemotherapy without and with the use of the RS, respectively (p = .03). Lo et al. [18] recently reported the results of RS predictions and treatment recommendations before and after the RS was obtained from 17 medical oncologists in 89 patients. Oncologists changed the treatment recommendation in 32% of cases after knowing the RS results. Similar to the current study findings, the greatest change was from a pretreatment recommendation of chemotherapy to a post-test recommendation of hormonal therapy alone. The authors allowed an answer of “equipoise,” reflecting neither superiority nor inferiority for either treatment choice. If one assumes that an answer change to equipoise reflects ambiguous conclusions of superiority but no actual treatment change, then the percentage of treatment recommendations changed by the RS may be 26%, close to our findings.

How much variation in treatment recommendations can be attributed to chance alone is not well understood. Studies have shown that the practice patterns of medical oncologists in managing patients with early-stage breast cancer vary and are often discordant with national guidelines [19, 20]. Based on repeated measures GEE analysis of chemotherapy recommendations, the effect of being given the RS on an oncologist's chemotherapy recommendation was not statistically significant (i.e., the change within a given oncologist in the chemotherapy recommendation was not larger than that attributed to random fluctuation in an oncologist's recommendation).

These data lead to several important observations. First, conventional prognostic criteria are useful to identify the risk for distant recurrence as assessed by the RS. As expected, the trend to recommend against chemotherapy in predicted scenarios of low RS risk and to recommend chemotherapy in predicted scenarios of high RS risk did not significantly change after being given an actual low and high RS. However, differences emerged in the frequency of chemotherapy recommendation when not given the RS versus being given the RS in the intermediate-risk group (44% versus 25%; p = .035). Thus, up to 19% of patients would have been misclassified and not received chemotherapy, with clinically important implications.

A recent economic analysis of the Oncotype DX® test concluded that the RS can actually augment the classification made by the NCCN system. In that study [21], 28% of patients stratified by the NCCN criteria as low risk were reclassified by the RS as intermediate or high risk. Conversely, almost 50% of patients classified as high risk were reclassified by the RS as low risk. Assuming that these newly classified patients would not receive chemotherapy, RS testing would increase the quality-adjusted life-years by 8.6 years and produce a 5% decline in overall health care–associated costs. These results suggest that the clinical utility of the RS is in reducing costs, morbidity, and mortality of chemotherapy without changing outcome. Our data show similar results in the number of cases predicted to be low risk reclassified to intermediate or high risk (20%), but no predicted high-risk cases were truly low risk by the Oncotype DX® RS.

There are several limitations to this study. The relatively low number of cases studied is a reflection of the lack of national consensus on the use of the RS at the time of the study. The issues of experience and expertise are important to note. The ability to generalize the current findings to settings outside tertiary-care cancer centers with dedicated breast oncologists and highly experienced breast pathologists is likely limited. Even though two well-experienced breast pathologists, blinded to outcomes, read the cases, this study demonstrated that even slight differences in the quantification of hormone receptor expression may change the predicted RS and chemotherapy recommendation. To a lesser extent, this may also be true of small differences in histologic grade. But even considering this, minor changes in pathologic reads were associated with changes in the predicted RS risk in 54% of cases and resulted in a change in the chemotherapy recommendation in 13% of cases. This underscores the issue that even minor discrepancies in pathologic reads can change clinical decisions in an important number of clinical scenarios.

Finally, it should be noted that the RS is weighted using information regarding HER-2 and proliferation genes (e.g., Ki-67). Although the oncologists in this study were provided information regarding HER-2 (standard immunohistochemistry [IHC] and fluorescence in situ hybridization [FISH] if appropriate), they were not provided with an immunhistochemical marker of molecular grade (Ki-67). Therefore, it is possible that the addition of this information may have further improved the ability of academic oncologists to discriminate among RS categories. Notably, only two cases were considered HER-2⁺ by standard IHC and/or FISH criteria, and the current NCCN guidelines do not recommend using the RS for tumors that are HER-2⁺.

Conclusion

In summary, this case series demonstrates that academic breast oncologists interpreting standard prognostic criteria provided by an academic breast pathologist may be able to distinguish high- from low- or intermediate-risk patients as identified by the RS. However, the observation of a management change in ∼20% of cases after the addition of the RS suggests that it may be an important tool for better standardization of chemotherapy recommendations, especially in the intermediate-risk group. Prospective clinical trial data are necessary to further define its utility in improving patient outcomes.

Author Contributions

Conception/Design: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz

Provision of study material or patients: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz

Collection and/or assembly of data: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, Xochiquetzal J. Geiger, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz

Data analysis and interpretation: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, Amylou C. Dueck, Xochiquetzal J. Geiger, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz

Manuscript writing: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, Amylou C. Dueck, Xochiquetzal J. Geiger, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz

Final approval of manuscript: Arif H. Kamal, Charles L. Loprinzi, Carol Reynolds, Amylou C. Dueck, Xochiquetzal J. Geiger, James N. Ingle, Robert W. Carlson, Timothy J. Hobday, Eric P. Winer, Matthew P. Goetz