The value of multigene predictors of clinical outcome in breast cancer: an analysis of the evidence

Abstract

Objective

Multigene predictors are being used increasingly in early stage breast cancer patients for prediction and prognosis. However, one consequence of the increased use of multigene predictors, and the heightened efforts towards their incorporation into routine clinical practice, is the potential for future malpractice litigation. It is therefore important to ascertain the strength of the evidence for using the different commercially available multigene predictor assays clinically. We evaluated the literature for evidence of clinical validity of four currently available gene signatures, and to assess the influence of the 21-gene-expression assay Oncotype DX^® (Genomic Health Inc., Redwood City, CA, USA) on changes in treatment recommendations.

Methods

A systematic search of the peer-reviewed literature from January 2002 to March 2014 for multigene predictor assays was carried out, and a meta-analysis was conducted.

Results

The adjusted Cox hazard ratio average for studies that met eligibility criteria was 3.538 (95% CI: 1.513 – 8.469). The 21-gene signature showed the highest stability in the estimation of likelihood of distant risk of recurrence. Using recurrence scores resulted in changes in treatment recommendations in 31.8% of all patients in the studies.

Conclusion

This study may provide insight about the use of multigene predictors in clinical practice for prediction and prognosis in breast cancer.

INTRODUCTION

Although advances in molecular diagnostics and therapeutic interventions have led to declines in breast cancer mortality, particularly over the last 15 years, breast cancer remains a frequently diagnosed cancer among women [1]. It has been estimated that a total of 232,670 new cases will be diagnosed in the United States in 2014 [1].

A key challenge for physicians and their patients in the management of breast cancer in making decisions about treatment options relates to the use of adjuvant chemotherapy [2,3] following surgical resection of the tumor. Adjuvant chemotherapy has been shown to increase recurrence-free and overall survival in early-stage patients as a whole [4], but it is difficult to identify the patients who would not have experienced recurrence in the first place and do not need chemotherapy. The effectiveness of these interventions are based on risk stratification of patients and has involved the use of validated computer-based models using algorithms designed to calculate the rates for recurrence of the disease such as the Adjuvant! Online (www.adjuvantonline.com) developed using Surveillance, Epidemiology, and End Results (SEER) registry data; the Nottingham Prognostic Index - NPI [5]; and/or the utilization of panel recommendations or clinical guidelines issued by scientific or professional organizations such as the St. Gallen Consensus [6] and the National Comprehensive Cancer Network - NCCN [7].

In more recent years, a number of molecular multigene predictor gene expression profiling (GEP) assays have been developed and increasingly used to identify early-stage breast cancer patients who are most likely to benefit from adjuvant chemotherapy [8].

As a consequence of the increased use of multigene predictors and other molecular companion diagnostics, clinicians, policy-makers and other decision-makers are working towards developing and managing healthcare systems that incorporate personalized genomic medicine into routine clinical practice. This increased use of multigene predictors and other molecular diagnostics thus has the potential to affect future malpractice litigation risks [9,10].

It is therefore important to ascertain: (1) the strength of the evidence for using the different commercially available GEP assays based upon whether these tests provide clinically valid and useful information; and, (2) the influence of the 21-gene-expression assay, commercially marketed as Oncotype DX^® (Genomic Health Inc., Redwood City, CA, USA) on physician treatment recommendations, since this is the most frequently used multi-gene predictor GEP assay in the US.

Several studies have been published evaluating the use of the Oncotype DX^® recurrence score (RS) on physician recommendations for treatment options (adjuvant chemotherapy and/or hormonal therapy) [11], or whether the gene signatures of interest were comparable in terms of their clinically validity of the estimations for risk of recurrence. However, these studies primarily evaluated how knowledge of the RS affects management of treatment for eligible early-stage breast cancer patients, and did not analyze gene signature adjusted multivariate Cox hazard ratios to allow more direct comparisons of the prognostic value of the gene signature for the likelihood of 10-year relapse-free survival and overall survival of breast cancer between published reports. We systematically reviewed the peer-reviewed published literature and conducted a meta-analysis in order to determine whether (a) sufficient evidence exists in the literature to substantiate the clinical validity of gene-expression profiling to predict recurrence; and, (b) assess the influence of the 21-gene-expression assay on changes in physician treatment recommendations. Given the increasing need for clinicians and patients, and potentially representatives of the legal community, to interpret findings from the literature to make informed decisions about the use of gene expression profiling assays in breast cancer care, we conducted a systematic review of the literature and meta-analysis aimed at summarizing and synthesizing the current evidence.

METHODS

Study Identification and Selection

We systematically searched the peer-reviewed literature from January 2002 through March 2014 for multigene predictor assays. Searches were performed using several databases including PubMed Medline, Ovid (including Medline and Journals@Ovid Full Text) and EMBASE for studies of multigene predictors of clinical outcome in breast cancer. We independently reviewed a list of titles and abstracts for relevance to identify any articles that describe either a validation of any gene-expression profiling assay for early-stage breast cancer and/or the use of the RS to make or change treatment recommendations in breast cancer. Articles reviewed were limited to those published in the English language, and were excluded if they lacked relevance to gene-expression profiling for breast cancer or did not deal with human populations. The full text of any article that passed this initial screening was retrieved for further review. Full articles were also retrieved for any articles for which it was not possible to determine relevance from the abstract in order to ensure a comprehensive search. The references of relevant published articles were also manually searched to identify other potentially relevant articles.

Studies were included for further review if they met the following criteria: original research studies published in peer-reviewed journals; studies focusing on validation of any studies on any of the specific gene-signatures; studies of clinical utilization of the 21-gene expression assay (Oncotype DX^®; Genomic Health, Redwood City, CA) recurrence score to make treatment recommendations for patients. Reviews of the literature or meta-analyses that did not include original research were excluded.

Data Extraction and Analysis

Data from each of the articles that were selected for further review was abstracted on to a structured abstraction form. Abstracted data included: a) authors; b) year of publication; c) total number of study participants; d) gene signature adjusted multivariate Cox model hazard ratio and respective 95% confidence intervals; e) total number of patients with changes using RS for treatment recommendations; g) total number of patients changing from a combination of chemotherapy (CHT) plus hormone therapy (HT), CHT-HT, to HT alone or h) from HT alone to a combination of CHT-HT. Data was also extracted about i) the type of study (prospective or retrospective); j) which clinical practice guidelines (if this information was available) and/or algorithm-based estimators of risk (such as Adjuvant! Online and/or the Nottingham Prognostic Index), were used for initial treatment recommendations.

The information available for adjusted multivariate Cox hazard ratios and their 95% confidence intervals (CI) for different gene-signatures was pooled, averaged by gene signature and weighted (using a random effects model). Statistical heterogeneity was calculated using the χ2 test for heterogeneity and the I² percentage. A probability level for the χ2 statistic less than or equal to 10% (p≤0.10) and/or an I² greater than 50% would be considered indicative of statistical heterogeneity.

Data about differences between treatment recommendations before and after the availability of recurrence scores was taken directly from the reviewed articles, pooled, and averaged to ascertain how RS were used to change initial treatment recommendations. Effect sizes for differences in treatment recommendations were calculated using Cohen's d and the two-tailed correlation coefficient, r_Y, using means and standard deviations of two independent groups of equal size. Data were analyzed using IBM^® SPSS® Statistics Release 21, (Armonk, New York).

RESULTS

Our initial broad search of the literature yielded 590 articles (figure 1). Of the remaining 52 studies that met eligibility criteria, twenty-five studies were eligible for a meta-analysis of the gene signature associated standardized multivariate Cox hazard ratios. Table 1 summarizes the characteristics of the included studies. These 25 studies used four gene-signatures: Mammaprint® (44%; n=11); Mammostrat® (8.0%; n=2); Oncotype DX^® (24.0%; n=6); and the Rotterdam signature (24.0%; n=6). The sample size of these studies ranged between 86 to 4,598 subjects [12,13] with a mean of 702 patients enrolled. We contrasted the prognostic value of the likelihood of 10-year relapse-free survival and overall survival of breast cancer by gene signatures included in this meta-analysis by comparing the reports of adjusted multivariate Cox hazards ratios. The adjusted Cox hazard ratio average for the series of 25 studies was 3.538 (95% CI: 1.513 – 8.469; table 1). Point estimates and their 95% confidence intervals showed some degree of overlap. In our analysis, the 21-gene signature Oncotype DX^® showed the highest stability in the estimation of likelihood (Mean: 2.70; 95% CI: 1.56-4.72). Around this range of variation in the adjusted multivariate Cox hazards ratios, the closest one to Oncotype DX^® was for Mammaprint^® (Mean: 3.43; 95% CI: 1.43-8.64), followed by Mammostrat^® (1.61; 95% CI: 1.30-2.02). The Rotterdam 76-gene signature showed the farthest variation overall (Mean: 5.21; 95% CI: 18.50) (Figure 2).

Figure 1.

Search strategy flowchart.

Table 1.

Weighted Adjusted Multivariate Cox Hazard Ratio (HR) and 95% Confidence Intervals (CI)

Study	Patient Enrollment	Cox HR	SE	Weighted CI lower	Weighted CI upper
Mook et al, 2010 [41]	964	3.450	0.050	3.333	3.567
Knauer et al, 2010 [42]	541	0.210	0.020	0.171	0.249
Kok et al, 2009 [31]	364	2.210	0.078	2.057	2.363
Buyse et al, 2006 [43]	307	1.500	0.067	1.363	1.637
van de Vijver et al, 2002 [44]	295	4.600	0.125	4.355	4.845
Bueno-de-Mesquita et al, 2009 [45]	274	5.400	0.142	5.125	5.675
Mook et al, 2009 [46]	241	2.990	0.111	2.772	3.208
Saghatchian et al, 2013 [47]	173	2.698	0.124	2.453	2.943
Knauer et al, 2010 [48]	168	5.780	0.185	5.416	6.144
Glas et al, 2006 [49]	162	5.600	0.186	5.236	5.964
Mook et al, 2009 [50]	148	4.400	0.172	4.062	4.738
Avg. Mammaprint	330.6	3.531	0.115	3.304	3.757
Bartlett et al, 2012 [13]	4,598	1.910	0.020	1.870	1.950
Ross et al, 2008 [55]	1,294	1.400	0.032	1.336	1.464
Avg. Mammostrat ®	2,946.0	1.655	0.026	1.603	1.707
Partin et al, 2011 [18]	1,674	2.160	0.036	2.090	2.230
Gray et al, 2011 [52]	1,436	1.430	0.062	1.368	1.492
Dowsett et al, 2010 [53]	1,372	4.360	0.042	4.250	4.470
Tang et al, 2011 [21]	1,319	2.370	0.052	2.287	2.453
Paik et al, 2006 [54]	651	1.780	0.086	1.678	1.882
Albain et al, 2010 [38]	367	2.640	0.151	2.474	2.806
Avg. Oncotype Dx	1,136.5	2.457	0.071	2.358	2.556
Zhang et al, 2009 [55]	300	6.130	0.143	5.850	6.410
Wang et al, 2005 [56]	286	5.550	0.139	5.277	5.823
Desmedt et al, 2008 [20]	198	5.110	0.161	4.795	5.425
Desmedt et al, 2007 [57]	198	2.800	0.119	2.567	3.033
Foekens et al, 2006 [58]	180	11.360	0.251	2.670	48.400
Prat et al, 2012 [59]	1,380	1.490	0.015	1.426	1.554
Avg. Rotterdam 76	423.7	5.407	0.138	3.764	11.774
Effect Summary	1,209.2	3.262	0.088	2.757	4.949

Figure 2. Gene Signature Reported Adjusted Multivariate Cox Hazard Ratios and 95% confidence intervals.

Forest plots showing disease-free survival. Squares: Reported hazard ratios from each study; Diamonds: Each gene signature's average and the overall average of the hazard ratio.

Our search of the literature for evidence of clinical utility of GEP in breast cancer care ultimately yielded 27 studies, all focused on the 21-gene expression-profiling assay, Oncotype DX^®. We limited our analysis to studies that reported changes in treatment recommendations from initial recommendations based on using the Oncotype DX^® RS. Twenty-seven studies reporting changes in initial treatment recommendations following the use of RS, met eligibility criteria (Table 2), and included 6,655 patients with a mean of 246.5 participants per study (SD: 377.2; 95% CI: 104.2 – 388.8). We found wide variation in the number of participants, from the largest sample of 1,822 patients [14] to the smallest sample of 22 patients[15]. More than two thirds of the studies (70.4%; n=19) were published between 2011 and 2014. Three-fourths of the studies were retrospective (74.1%; n=20). Studies reported that initial treatment recommendations for adjuvant chemotherapy were formulated following clinical practice guidelines endorsed by professional associations, mainly NCCN (35.0%; n=14), St. Gallen Consensus (15.0%; n=6) or ASCO (12.5%; n=5). Several studies reported using algorithms for risk estimation such as Adjuvant! Online (17.5%; n=7) and the Nottingham Prognostic Index (5.0%; n=2).

Table 2.

Proportional Changes to Treatment Recommendations following Utilization of the 21-gene signature Recurrence Score (RS^®)

Study	Type of Study	Country of Pub.	Number of cases/Study Participants	Guidelines or Decision Algorithm Used Pre-RS	Receipt of Adjuvant Therapy
					% Change CT-HT to HT	% Change HT to CT-HT	% Change Total
Oratz et al 2007 [60]	Retrospective	USA	68	AO	25.0	25.0	25.0
Erb et al 2007 [61]	Retrospective	USA	124	NCCN	30.0	n.a.	30.0
Lo et al 2010 [25]	Prospective	USA	89	n.a.	22.5	3.4	25.9
Asad et al 2008 [35]	Retrospective	USA	85	NCCN	38.8	4.7	43.5
Henry et al 2009 [62]	Retrospective	USA	29	AO	25.9	7.4	33.3
Geffen et al 2009 [63]	Retrospective	Israel	25	AO	24.0	12.0	36.0
Klang et al 2010 [64]	Retrospective	Israel	313	NCCN & ASCO	33.5	6.4	39.9
Ademuyiwa et al 2011 [65]	Retrospective	USA	276	AO & NPI	31.9	6.2	38.0
Partin et al 2011 [18]	Retrospective	USA	169	St Gallen; NCCN & AO	71.0	3.0	74.0
Oratz et al 2011 [66]	Prospective	USA	89	NCCN & ASCO	35.0	n.a.	35.0
Joh et al 2011 [67]	Retrospective	USA	154	NCCN & ASCO	24.9	n.a.	24.9
Haas et al 2011 [29]	Retrospective	USA	364	NCCN	61.8	0.3	62.1
Hassett et al 2012 [16]	Retrospective	USA	1,111	St Gallen & NCCN	9.6	8.0	17.6
Eiermann et al 2012 [24]	Prospective	Germany	366	n.a.	21.6	10.6	33.0
Albanell et al 2012 [26]	Retrospective	Spain	107	St Gallen & ESMO	20.6	11.2	31.8
Kamal et al 2012 [17]	Retrospective	USA	186	St Gallen & NCCN	10.8	8.1	18.9
Schneider & Khalil 2012 [28]	Retrospective	USA	89	NCCN; ASCO & AO	39.3	4.5	43.8
Malo et al 2012 [68]	Retrospective	USA	118	NCCN	42.4	0.8	43.2
deBoer et al 2013 [23]	Prospective	Australia	151	St Gallen	35.8	15.5	24.5
Chen et al 2013 [14]	Retrospective	USA	1,822	NCCN & ASCO	23.3	3.1	26.4
Holt et al 2013 [69]	Prospective	United Kingdom	142	AO & NPI	18.3	8.5	26.8
Stemmer et al 2013 [70]	Retrospective	Israel	282	St Gallen	45.6	n.a.	45.6
Davidson et al 2013 [71]	Prospective	Canada	150	n.a.	20.0	15.0	35.0
Yamauchi et al 2014 [72]	Prospective	Japan	124	NCCN & Japanese	32.3	5.6	37.9
Fried & Moskovitz et al 2014 [73]	Retrospective	Israel	111	n.a.	11.7	9.9	21.6
Toole et al 2014 [15]	Retrospective	USA	22	n.a.	27.3	9.1	36.4

AO:Adjuvant! Online; NCCN: National Comprehensive Cancer Network; ASCO: American Society of Clinical Oncology; NPI: Nottingham Prognostic Index; St Gallen: St Gallen International Consortium; CT: Chemotherapy; HT: Hormonal Therapy

Using recurrence scores resulted in changes in treatment recommendations from initial treatment recommendations in 31.8% (n=2,113) of all patients in the studies. A reduction in the administration of chemotherapy but maintenance of hormonal therapy was observed in the majority of study patients (83.3%; n=1,761). However, addition of chemotherapy to hormonal therapy was found in less than one-fifth of the pooled study patients (16.4%; n=346). These changes from the initial treatment recommendations were statistically significant (paired t(₂₆)=5.82, p<0.001, 2-tailed; Cohen's d=2.0; effect size r=0.71.5; coefficient of determination r²=51.1%) suggesting that treatment recommendations based on use of the Oncotype DX^® recurrence scores resulted in less administration of chemotherapy. Reported percentages in changes from initial treatment recommendations about systemic adjuvant chemotherapy varied between less than 20% up to 74% of all pooled study patients [16,17,18].

DISCUSSION

We evaluated the evidence in the peer-reviewed literature for whether: (1) the currently available gene-signatures for early-stage breast cancer are associated with the likelihood of distant recurrence of breast cancer based upon reported adjusted multivariate Cox hazard ratios; and (2) the strength of evidence for influence of the Oncotype DX^® RS from the 21-gene expression profiling assay on changes in physician treatment recommendations from initial recommendations.

We found that the four gene signatures evaluated appear to be associated with predicting distant recurrence, and point estimates and their 95% confidence intervals showed some degree of overlap. This supports the suggestion by some authors that perhaps combining one or more of the already developed mulitgene predictors will likely provide more precise prognostic and predictive information about recurrence risk with concomitant therapeutic outcomes and one such attempt has been reported [19-21]. In our analysis, the 21-gene signature Oncotype DX^® showed the highest stability in the estimation of likelihood of risk of recurrence. This suggests that the estimation was less variable and/or less susceptible to variations in predictors of recurrence, including tumor genomics (e.g. Ki67, HER2 biomarker status). It further suggests the clinical validity of Oncotype DX® as a predictive and prognostic diagnostic for making treatment decisions for breast cancer [22]. Although other authors have identified these gene signatures as predicting clinical outcome [22], to the best of our knowledge, this study is the first to pool together and statistically analyze standardized adjusted Cox hazard ratios, which are estimations of the probability of 10-year recurrence.

Our findings that treatment recommendations were changed based on using the RS in more than 30% of patients in the pooled analysis of studies are consistent with other individual studies which found treatment recommendations changed in some 24% to 30% for women with node-negative cancer following RS use [23-26].

Our analysis demonstrated that changes in treatment recommendations mostly resulted in proportionally reduced administration of chemotherapy in the majority of cases, and less frequently in addition of chemotherapy. This result is not surprising, since chemotherapy is an accepted standard therapeutic intervention in breast cancer [27]. However, prior individual studies had only found changes in treatment recommendations varied from 9.6 to 71 with a mean of 29.8 [28,29]. Our finding may have possible policy-relevant consequences for clinical utilization, testing costs, and potential future liability cases.

Although several reviews have recently evaluated the literature related to multigene predictors of clinical outcome in breast cancer [30,31,32], these did not specifically statistically pool and analyze the proportional changes in treatment recommendations that were made as a result of using recurrence scores.

Our study suggests that GEP tests, and particularly use of the 21-gene assay, Oncotype DX^® are increasingly influencing how breast cancer patients are treated.

Although studies that directly show an association with outcomes following use of multigene predictors are limited, evidence is accumulating that tests such as the 21-gene signature (Oncotype DX^®) and the 70-gene signature (MammaPrint^®) contribute additional prognostic value beyond conventional clinical indicators [26, 33,34, 70]. Thus, treatment recommendations for adjuvant chemotherapy should include gene expression profiling as an integral part of any decision algorithm [19,35].

It is important for clinicians to be aware of how these multigene predictive assays are increasingly being used in practice [36] and of their limitations. In addition to factors such as risk of distant recurrence that affect treatment, clinicians should also consider potential risks as well as patient quality of life. Since this field, and what constitutes standard of care appears to be continually evolving, it is equally important for clinicians to avail themselves of opportunities to learn about the latest clinical practice guidelines and accepted standards in order to avoid any potential litigation [37] even though it maybe premature to consider litigation at this time.

Since the standard of care for multigene predictors is still fraught with uncertainty, this presents with a “damned if you do, damned if you don't” dilemma for physicians [37]. A physician that fails to utilize such a test and recommends against chemotherapy on other grounds may face a lawsuit by a patient with recurring breast cancer who argues after the-fact that the physician was negligent in not relying on a multigene predictor test. Alternatively, if the physician recommends a multigene predictor assay and no chemotherapy based on the results of that test, the physician may face a lawsuit alleging premature reliance on a test that was not yet subsequently validated [37]. In either case, with the advantage of perfect hindsight, the physician could be portrayed by a skilled trial lawyer as having acted too slowly or too quickly to rely on multigene predictor assays.

While uncertainty and rapid change in medical technologies create an inherent liability risk for physicians, the best way to protect against such liability is for the physician to document in the patient's medical record at the time of decision that the physician was aware of the available and uncertain evidence on the clinical utility of GEP assays, and made an informed professional judgment in the face of such uncertainty.

We recognize that policy regarding the use of multigene predictors should ideally be governed by evidence from prospective randomized clinical trials [22, 38]. Until the results of prospective clinical trials such as the TAILOR-X [39] and the MINDACT [40] become available, systematic assessments and meta-analyses such as this study ought to be useful for providing insight to clinicians, policymakers, and for better understanding potential liability in this burgeoning field.

EXPERT COMMENTARY

Multigene predictors, including gene expression profiling, are being used increasingly for risk stratification to identify early stage breast cancer patients who are most likely to benefit from adjuvant chemotherapy. During the past decade, a number of studies have been published focused on the analytical and clinical validity, and clinical utility of at least 4 multigene predictors, of which the most frequently used in US populations is the 21-gene signature Oncotype DX^®. Although incorporation of multigene predictor assays into routine clinical breast cancer practice has not yet been fully realized, gene-expression profiling for prediction and prognosis is increasingly being used in clinical settings, with a consequent interest on the part of clinicians and policymakers towards developing and managing health care systems that incorporate personalized genomic medicine as a standard of medical practice. This has the potential to affect future malpractice litigation risks. This study suggests that multigene predictor assays, and particularly use of the 21-gene assay, are increasingly being used in practice. It appears that this trend towards adoption of multigene predictors and other molecular genomic technologies will continue as more advances in technologies for genomic profiling are made. As what constitutes standard of care continues to evolve, it is important for physicians to document in the patient's medical record at the time of decision that the physician was aware of the available and uncertain evidence on the clinical utility of multigene predictor assays, and made an informed professional judgment in the face of such uncertainty.

FIVE-YEAR VIEW

We believe that as multigene predictors and other genomic technologies continue to evolve, it is likely that the potential for malpratice litigation will increase as well. Historically, there are precedents for increased litigation with increased use and adoption of novel technologies, primarily due to increased patient expectations. Thus, we can expect that in the next 5 to 10 years, as genomic technologies such as multigene predictor assays become more routinely integrated into clinical practice and health systems, physicians will likely be dealing with the first wave of malpractice litigation.

KEY ISSUES.

• Multigene predictors, including gene expression profiling, are being used increasingly for risk stratification to identify early stage breast cancer patients who are most likely to benefit from adjuvant chemotherapy.

• One consequence of the increased use of GEPs and other molecular companion diagnostics, and the heightened efforts towards their incorporation into routine clinical practice, is the potential for future malpractice litigation.

• A systematic search of the literature from Jan 2002 to March 2014 and meta-analyses were performed.

• The analysis of Cox hazard ratios demonstrated that gene signatures appear to be associated with predicting distant recurrence.

• Point estimates and their 95% confidence intervals showed some degree of overlap between the 4 gene signatures assessed, however, the 21-gene signature Oncotype DX^® showed the highest stability in the estimation of likelihood of distant risk of recurrence.

• Using recurrence scores resulted in changes in treatment recommendations from initial treatment recommendations in 31.8% of all patients in the studies.

• The results of this study may provide insight about the use of multigene predictors in clinical practice for prediction and prognosis in breast cancer, and improved understanding of potential liability issues in the field of personalized genomic medicine.