Practical implications of gene-expression-based assays for breast oncologists

Abstract

Gene-expression profiling has had a considerable impact on our understanding of breast cancer biology, and more recently on clinical care. Two statistical approaches underlie these advancements. Supervised analyses have led to the development of gene-expression signatures designed to predict survival and/or treatment response, which has resulted in the development of new clinical assays. Unsupervised analyses have identified numerous biological signatures including signatures of cell type of origin, signaling pathways, and of cellular proliferation. Included within these biological signatures are the molecular subtypes known as the ‘intrinsic’ subtypes of breast cancer. This classification has expanded our appreciation of the heterogeneity of breast cancer and has provided a way to sub-classify the disease in a manner that might have clinical utility. In this Review, we discuss the clinical utility of gene-expression-based assays and their technical potential as clinical tools vis-a-vis the performance of breast cancer biomarkers that are the current standard of care.

Introduction

Breast cancer is a devastatingly common disease and there is a considerable need to improve approaches for prevention, diagnosis and treatment.¹ One of the biggest barriers to progress is the extreme clinical and genetic heterogeneity of the disease. Rapid progress in diagnostic capabilities has occurred over the past decade, however, breast cancer management is caught between two worlds: the old world of familiar groupings defined by estrogen receptor (ER) and HER2 status and a new world of seemingly endless and complex ways to classify breast cancers for treatment individualization. Although enthusiasm for new genomic discovery tools should be tempered, many studies conducted using conventional gene-expression profiling should be vaunted as identifying successful biomarkers. Over the past decade, we have learnt many valuable lessons from gene-expression profiling studies. The clinical utility of gene-expression-based assays has been demonstrated with increasing confidence, particularly with respect to tailoring treatment for the current standard of care in chemotherapy and hormonal treatment.^2–4 In this Review, we present the current status of clinically applicable gene-expression-based assays, and we discuss their reproducibility as well as the reproducibility of the biomarkers that are standard of care in breast cancer today.

Clinical gene expression-based assays

A plethora of prognostic and some predictive gene-expression signatures have been identified for breast cancer; however, most of these signatures identify breast cancer subtypes that have a similar biology, which encompasses ER status, HER2 status, and proliferation.⁵ Improvements offered by gene-expression-based assays must, therefore, go beyond these known variables, or at least encompass them in ways that are more quantitative, reproducible, and/or objective than the existing immunohistochemistry (IHC)-based assays. The main characteristics of five gene-expression-based predictors are shown in Table 1. Although there are considerable similarities between these predictors,⁶ there are enough statistically significant differences across them such that each should be considered to be unique.⁷

Table 1.

Main features of the commercially available multi-gene signatures in breast cancer

Assay name	Genes (n)	Source	Platform	Training dataset	Output data
Biology driven
PAM50 subtype-predictor	55*	FFPE/FF	qRT-PCR/microarray/nCounter	Breast cancer-based cohort	Luminal A Luminal B HER2-E Basal-like Normal-like
Survival driven
PAM50 risk of relapse (ROR)	55*	FFPE/FF	qRT-PCR/microarray/nCounter	RFS; ER+/ER−/node− breast cancer patients receiving no adjuvant systemic treatment	Continuous Variable Low–medium–high
Oncotype DX®	21	FFPE	qRT-PCR	Survival: largely ER+/node− breast cancer patients receiving tamoxifen-only adjuvant treatment	Continuous Variable Low–intermediate–high
MammaPrint®	70	FF	Microarray	DRFS: largely ER+/node− breast cancer patients receiving no adjuvant systemic treatment	Continuous Variable Good–bad
Pathology driven
MapQuant Dx™	97	FF	Microarray	Grade 1 vs 3 in ER+ patients with breast cancer	Continuous Variable Low–high
Survival and pathology driven
Breast Cancer Index	7	FFPE	qRT-PCR	MGI: grade 1 vs 3, HOXB13/IL17BR ratio DRFS: ER+ patients with breast cancer receiving tamoxifen-only treatment	Continuous Variable Low–intermediate–high

^*5 genes included for expression normalization. Abbreviations: DRFS, distant relapse-free survival; FF, fresh-frozen; FFPE, formalin-fixed paraffin-embedded; MGI, molecular grade index; qRT-PCR, quantitative reverse transcription PCR; RFS, relapse-free survival.

Oncotype DX® recurrence score

Oncotype DX® (Genomic Health, Redwood City, CA) is a 21-gene quantitative reverse transcription (qRT)-PCR-based assay and is the most widely used clinical gene-expression assay in the USA. It is based on the levels of gene expression of 16 cancer-related genes and five genes for expression normalization using RNA from formalin fixed-paraffin embedded (FFPE) tumor tissues.^8,9 The genes in the assay were selected from 250 candidates that were tested for association with survival in a cohort of 447 tumor samples, including 233 (52%) ER-positive samples from the tamoxifen-treated and node-negative cases of the National Surgical Adjuvant Breast and Bowel Project (NSABP) B-20 clinical trial.¹⁰ The final 16-gene list is weighted by biological processes including proliferation, HER2 and ER signaling. The resulting recurrence score (RS) is 0 to 100, which translates into three risk-group categories: low (RS <18), intermediate (RS from 18 to <31) and high (RS ≥31).

In a seminal study, Paik and colleagues¹¹ validated Oncotype DX® in a large cohort of ER-positive, node-negative tamoxifen-treated patients with breast cancer enrolled in the NSABP B-14 trial; in this study, the rates of distant recurrence at 10 years were 6.8%, 14.3% and 30.5% for the low-risk, intermediate-risk and high-risk groups, respectively. In a subsequent validation study,¹² Oncotype DX® RS was strongly associated with the risk of death from breast cancer in a similar cohort and also in patients who did not receive adjuvant systemic therapy. Importantly, the associations of Oncotype DX® with survival have been found to be independent from standard clinicopathological variables,^11,12 thus showing that this assay adds information beyond ER and HER2 status, stage, and grade.

More recently, Oncotype DX® has been endorsed by ASCO³ and two expert panels: NCCN Breast Cancer Clinical Practice Guidelines² and the 2011 St Gallen International Expert Consensus.⁴ These panels consider Oncotype DX® to be useful for patients with ER-positive, node-negative breast cancer as an aid to decision making for administering adjuvant chemotherapy. Since its commercialization, over 175,000 assays have been ordered by more than 7,500 physicians worldwide.

MammaPrint®

MammaPrint® is a microarray-based gene-expression profiling assay that was developed by Agendia (Amsterdam, the Netherlands) based on the Agilent (Agilent technologies, Santa Clara, CA) microarray-based platform that analyzed data from 78 patients with node-negative breast cancers.¹³ The vast majority of these 78 patients had ER-positive tumors and did not receive adjuvant systemic therapy. The 70 genes that comprise the MammaPrint® assay are proliferation genes and genes associated with invasion and angiogenesis. Since 2002, this signature has been validated on numerous cohorts of node-negative patients,^14–16 and has been shown to provide independent prognostic information beyond standard clinico-pathological variables such as age, histological grade and pathological stage.¹⁴ MammaPrint® is also the first FDA-approved gene-expression-based assay to be used as a prognostic test for women with node-negative breast cancers. Unlike the Oncotype DX® assay, this test is based on microarrays results and hence requires high-quality RNA from freshly collected tissues.

MapQuant Dx™ genomic grade

MapQuant Dx™ (Ipsogen, Marseille, France) is a predictor test that was derived by identifying 97 differentially expressed genes from grade 1 and 3 breast tumors using a training set of 64 ER-positive tumors;¹⁷ most of the genes in this signature are involved in cell-cycle regulation and proliferation. In validation data sets, this ‘genomic grade index’ (GGI) was strongly associated with risk of recurrence among patients with grade 2 tumors.^17,18 Similar to MammaPrint®, this assay is microarray based and requires freshly prepared tissues and high-quality RNA.

Breast Cancer Index^SM (BCI)

The Breast Cancer Index^SM (BCI; BioTheranostics, San Diego, CA) prognostic assay provides an assessment of the likelihood of distant recurrence in patients diagnosed with ER-positive, node-negative breast cancer. In development and validation studies, BCI stratified nearly 50% of tamoxifen-treated ER-positive, node-negative patients with breast cancer into a low-risk group for 10-year distant recurrence.^19,20 Unlike the other commercialized tests, BCI has been developed from the combination of two indices: the HOXB13:IL17BR two-gene ratio, which predicts distant recurrence in patients with ER-positive disease treated with tamoxifen,²¹ and a proliferation-related five-gene molecular grade index (MGI),¹⁹ which discriminates grade 1 from grade 3 disease (similar to the MapQuant Dx™ assay). The BCI molecular assay is based on qRT-PCR using RNA from FFPE tissues.

Intrinsic subtypes and PAM50 assay

Over the past 12 years, global gene-expression analyses have identified at least four breast molecular subtypes known as the ‘intrinsic subtypes’ (luminal A, luminal B, HER2-enriched and basal-like),^22–25 which show significant differences in epidemiological risk factors,²⁶ baseline prognoses,^6,27,28 and responses to systemic therapies.^28–32 Information provided by the intrinsic subtypes, using an IHC-based approach, has also been adopted by the 2011 St Gallen Consensus Conference (Table 2).⁴ The biological and clinical features of the intrinsic subtypes of breast cancer have been extensively discussed in the literature and hence we will focus on the most recent version of the intrinsic subtyping assay, known as the PAM50 assay.^28,33 The training set for the PAM50 subtype predictor was composed of tumor samples from 220 patients who had ER-positive or ER-negative tumors and HER2-positive or HER2-negative tumors and from whom the prototypic samples of each molecular subtype were identified using a list of 1,906 intrinsic genes; it also included 12 true normal breast samples that constituted the normal breast-like centroid.²⁸ A final minimized set of 50 genes was derived from these prototypic samples, which comprises genes involved in proliferation, ER and ER-regulated genes, HER2, and basal and myoepithelial characteristics.

Table 2.

2011 St Gallen consensus recommendations of systemic treatment⁴

IHC Subtype	Definition	Type of adjuvant therapy
Luminal A	HR+/HER2−/Ki67low	Endocrine therapy alone*
Luminal B	HR+/HER2−/Ki67high	Endocrine therapy ± cytotoxic therapy
Luminal B	HR+/HER2+	Cytotoxics + anti-HER2 + hormonal therapy
HER2-positive	HR−/HER2+	Cytotoxics + anti-HER2 therapy
Triple-negative	HR−/HER2−	Cytotoxic therapy

^*A few patients require cytotoxics (such as high nodal status or other indicator of risk). Abbreviation: HR, hormone receptor.

The PAM50 assay provides a risk of relapse (ROR) score prognostic of relapse-free survival for patients with node-negative tumors who did not receive adjuvant systemic therapy.²⁸ The ability of the ROR score to predict prognosis (as a continuous variable or as a three-class variable) has been confirmed in an independent test set of 786 patients with ER-positive disease treated only with adjuvant tamoxifen.³³ In this cohort, the ROR together with tumor size outperformed standard clinicopathological variables that included expression of the proliferation marker Ki67, progesterone receptor (PR) and histological grade.³³ In addition, a ROR model that contained a proliferation component (derived using 11 of 50 PAM50 genes associated with cell-cycle function) improved the original model, showing that the best model may contain variables for molecular subtype, tumor size and proliferation status. Interestingly, this 11-gene proliferation signature has been found to have more prognostic value than the expression of Ki67 assessed by IHC.³³ The PAM50 assay is compatible with FFPE-derived RNA and is currently being developed for clinical use on the NanoString nCounter™ Analysis System (Nanostring technologies, Seattle, WA).³⁴ It is also offered using a qRT-PCR format from ARUP Laboratories (Salt Lake City, UT) as a laboratory-developed test (that is, developed, evaluated, and validated within one particular laboratory) waived by the CLIA (Clinical Laboratory Improvement Amendments).

The intrinsic molecular subtypes are correlated with standard pathological markers based on gene-expression analysis.^28,35–38 For example, most luminal tumors are ER-positive and/or PR-positive, and most HER2-enriched and basal-like tumors are typically HER2-amplified and triple-negative tumors, respectively.^28,35–38 It should be noted that intrinsic subtyping by gene expression compared with standard pathological biomarkers are not interchangeable classifications,^28,35,37 which demonstrates that intrinsic subtypes are not simple combinations of ER, PR, and HER2.

Predicting early-stage outcomes

Prognostic signatures in breast cancer consist mainly of changes in proliferation and/or ER-regulated genes.^5,6,27 These two biological processes in breast cancer are critical for distinguishing the relatively good prognosis of the luminal A subtype (which is characterized by low proliferation and high expression of ER-regulated genes) from the worse prognosis of the luminal B, HER2-enriched, and basal-like subtypes (which are characterized by a higher proliferation status).^22–25 Thus, the low-risk groups defined by Oncotype DX®, MammaPrint®, MapQuant Dx™ and PAM50-ROR are essentially composed of a subset of luminal A tumors.^7,18 This observation explains why the prognostic ability of these different gene expression-based assays is similar, as most of them are differentiating luminal A from all other subtypes.^6,27

Fan et al.²⁷ evaluated 323 different gene signatures (also called modules), including research versions of Oncotype DX®, MammaPrint®, MapQuant Dx™ and PAM50-ROR, on a microarray-based cohort of 550 patients with node-negative breast cancer who did not receive adjuvant systemic therapy. This exploratory study showed that it is only within ER-positive, luminal disease that robust prognostic predictions can be made, but not within ER-negative, or within HER2-positive disease.²⁷ Other investigators have identified immune-related signatures that might be able to predict survival within HER2-positive or triple-negative tumors; however, the ‘good’ prognostic group still showed a relatively high likelihood of relapse.^39,40 Thus, from a clinical utility perspective, prognostic gene-expression-based assays are only indicated for patients with ER-positive, node-negative disease for whom it might be adequate to say that hormonal and local therapies are sufficient, and that chemotherapy may not be necessary.

The clinical utility of Oncotype DX® versus standard parameters in ER-positive, node-negative disease has been prospectively examined.^41,42 In a relatively small multicenter study, treatment recommendations were changed for 31% of the patients based upon Oncotype DX® outcomes versus standard pathology-based risk assessments, with the largest change being the use of hormonal therapy only, with the omission of chemotherapy in 22.5% of cases. Similar data have been reported from an independent study of 135 patients.⁴² Thus, although the costs of these gene-expression-based tests might be high relative to the costs of ER, PR and HER2 testing, the additional reduction in adjuvant chemotherapy administration is likely to result in an overall cost saving,^43,44 as well as reduced toxicity and quality-of-life improvements for patients.

The clinical utility of Oncotype DX® and the other gene-expression-based assays in ER-positive, node-positive breast cancer remains investigational. Although Oncotype DX®, MammaPrint® and PAM50-ROR are able to identify different predicted risk groups with statistically distinct outcomes, for those with ER-positive, node-positive disease, the low-risk groups do not show a sufficiently low rate of recurrence (that is fewer than 10% relapse rates at 8–10 years) to consider excluding adjuvant chemotherapy based on prognosis alone.^{7,15,33,45,46} However, when the subset of patients with one-to-three positive lymph nodes are considered, the low-risk groups of Oncotype DX® and MammaPrint® have shown outstanding outcomes with hormonal therapy alone (that is <10% relapse rate at 5–10 years), which is similar to the outcomes previously observed in similar groups with node-negative disease.^45–47

Two large prospective clinical trials are underway to further establish the clinical utility of Oncotype DX® and MammaPrint®.^9,16 In the fully accrued TAILORx trial, patients with ER-positive, HER2-negative, node-negative disease with a RS of 10 or less were given hormonal therapy alone, patients with a RS of 11–25 were randomly assigned to receive adjuvant hormonal therapy with or without chemotherapy, and those patients with a RS >25 received hormonal therapy and chemotherapy (standard of care). The primary end point is to determine if adjuvant chemotherapy plus hormonal therapy is better than hormonal therapy alone in those with a RS between 11 and 25. In addition, this trial will confirm whether patients with a low RS have an extremely low risk of recurrence that excludes the need for chemotherapy based on prognosis alone. In the MINDACT trial assessing the MammaPrint® assay in patients with ER-positive, node-negative disease in the adjuvant treatment setting will be assessed by standard clinicopathological prognostic factors using Adjuvant!Online and MammaPrint®. The primary objective of MINDACT is to confirm that patients with a ‘low risk’ prognosis according to MammaPrint® and ‘high risk’ prognosis according to clinical features can be spared chemotherapy without affecting survival outcomes. On the basis of the recent results of MammaPrint® in patients with positive lymph nodes,⁴⁷ the MINDACT trial has also been amended to include patients with one-to-three positive lymph nodes.

Predicting treatment response

In the neoadjuvant setting, pathological complete response (pCR) in the breast has been used to determine the systemic response to chemotherapy agents, although pCR is only achieved in 20–30% of patients.⁴⁸ Across the intrinsic subtypes, pCR rates to standard anthracycline-based and taxane-based chemotherapy have been found to be approximately 7% for luminal A, 17% for luminal B, 36% for HER2-enriched, and 43% for basal-like.⁴⁸ Thus, the vast majority of patients that achieve a pCR have non-luminal A tumors. This explains, in part, why gene-expression signatures that largely discriminate luminal A from non-luminal A tumors can also predict pCR.^49–51

The benefit of neoadjuvant hormonal therapies is also an area of interest, and a non-genomic prognostic index based on tumor size, nodal status, Ki67 levels and ER status after treatment with neoadjuvant hormonal therapy (known as the preoperative endocrine prognostic index [PEPI]⁵²) has been developed. This index was developed in the P024 letrozole versus tamoxifen trial,⁵³ and validated as predictive of relapse-free survival in an independent study of 203 postmenopausal women in the IMPACT trial.⁵² Interestingly, in the neoadjuvant aromatase inhibitor trial ACOSOG Z1031, luminal A tumors (defined using PAM50) achieved significantly higher rates of PEPI score zero (good prognosis) than the luminal B tumors,⁵⁴ consistent with luminal A tumors being associated with superior outcome when treated with hormonal therapy alone.

In the adjuvant setting, prediction of chemotherapy and/or hormonal therapy efficacy relies on survival end points from clinical trials in which patients have been randomized to different treatments. The best data are from retrospective studies in which Oncotype DX® has been evaluated;^45,55 in the NSABP-B20 trial,⁵⁵ patients with ER-positive, node-negative breast cancer were randomly assigned to receive tamoxifen or tamoxifen plus chemotherapy (cyclophosphamide, methotrexate and 5-fluorouracil [CMF]). On the one hand, the low-RS Oncotype DX® group did not derive a survival benefit from the addition of chemotherapy and the relapse rates with or without chemotherapy were <5% at 12 years. On the other hand, the high-RS Oncotype DX® group had a large benefit from chemotherapy with an absolute decrease in 10-year distant recurrence of 27.6%. Less clear was the benefit of chemotherapy in the intermediate RS group in NASBP-B20, hence the design of the TAILORx trial, which randomized treatment to chemotherapy versus observation in this group.

In the SWOG 8814 trial,⁴⁵ the Oncotype DX® assay was applied to 219 patients with ER-positive, node-positive disease randomly assigned to receive adjuvant fluorouracil, doxorubicin and cyclophosphamide (FAC) followed by tamoxifen versus tamoxifen alone. The benefit of FAC was only seen in the high-RS group. Importantly, further subanalysis within the group with one-to-three positive nodes indicated that the relapse rates at 5 years in the low-RS group was <10% regardless of the type of chemotherapy use. These results, together with the TransATAC findings in patients with one-to-three positive nodes,⁴⁶ has led to the initiation of the RxPONDER clinical trial (NCT01272037) where around 4,000 patients with ER-positive breast cancer with one to three positive lymph nodes and a RS ≤25 will be randomized to hormonal therapy alone or chemotherapy followed by hormonal therapy.

Reproducibility of breast biomarkers

The reproducibility of genomic biomarkers has drawn considerable attention, which prompted us to investigate the issue of current biomarker reproducibility. Our review of the literature suggests that standardized gene-expression-based assays such as Oncotype DX® and MammaPrint® are more reproducible^8,56,57 than current methodologies for ER, PR and HER2 testing.^58,59 Currently offered gene-expression-based assays are single platform or single technology based (and sometimes even carried out within a single laboratory as is the case for Oncotype DX®) and follow a single protocol. This high level of standardization should be considered as a strength of these gene-expression-based assays as the existing breast cancer biomarker field is plagued by nonstandardized tests that do not require, or achieve, high levels of inter-laboratory concordance.

Since 2007, ASCO and the College of American Pathologists (CAP) have issued recommendation guidelines for ER, PR and HER2 testing.^58,59 These recommendations summarized the various validated assays and provide a common scoring algorithm; however, many standardization issues remain. Indeed, ASCO and CAP note that 20% of ER, PR and HER2 measurements in the clinical setting are possibly inaccurate.^58,59 This should be taken seriously as the status of these biomarkers dictates therapeutic decision making. In fact, if a new genomic or proteomic assay was to be put forth today with a 20% discordance rate, it likely would not be approved for clinical use.

In Table 3, we present a summary of the literature regarding reproducibility and concordance for the current breast cancer biomarker tests, and we distinguish the concordance between two identical protocols (inter-observer and inter-laboratory) and between two different methods (inter-method). A commonly used measure of reproducibility or concordance is the kappa (κ) statistic,⁶⁰ which has an output range from −1 (total discordance) to +1 (total concordance). The following three categories based on κ values are used to define levels of concordance using concordance correlation coefficients: 0.40–0.60 (moderate concordance—defined here as level 3), 0.61–0.80 (substantial concordance—level 2) and ≥0.81 (almost perfect agreement—level 1).^61–63

Table 3.

Reproducibility data of currently used biomarkers in breast cancer

Level of agreement	Type of test	Identical protocols (inter-laboratory and/or inter-observer)	Different protocols or technologies (inter-method)
Level 1 (κ ≥0.81) Almost perfect agreement	Pathology test	ER IHC testing, same antibody (+ vs −)64–70 HER2 FISH testing (+ vs −)71–76 HER2 IHC testing, same antibody (+ vs −)65,77–79	NA
	Gene-expression-based test	Oncotype DX® and MammaPrint®8,56,57 Basal-like and HER2E subtypes by supervised HC using different (microarray-based) intrinsic gene lists62	Basal-like by different gene list and microarray platform intrinsic subtype centroid predictors (microarray-based)63,107–109
Level 2 (κ 0.60–0.80) Substantial agreement	Pathology test	HER2 IHC testing, same antibody (+ vs −)71,73,76,79 Histological grade92,94,99 PR IHC testing (+ vs −)65,66 ER IHC, 3 or more groups64	ER IHC by different antibodies69,85,87 ER by IHC vs LBA64,69,84–86 HER2 IHC by different antibodies104,115–117 HER2 by IHC vs FISH77,101–104
	Gene-expression-based test	Overall intrinsic molecular subtypes identied by supervised HC using different intrinsic gene lists (microarray-based)62 Luminal A/luminal B/normal-like identied by supervised HC using different intrinsic gene lists (microarray-based)62	Overall intrinsic molecular subtypes by different gene list and microarray platform intrinsic subtype centroid predictors63,107–109
Level 3 (κ 0.40–0.59) Moderate agreement	Pathology test	Histological grade91–98 ER IHC with 4 or more groups64,67,90 PR IHC with 4 or more groups90	PR by IHC vs LBA84 PR IHC by different antibodies68,87,118 ER IHC by different antibodies72,118 HER2 by IHC/FISH vs FISH72
	Gene-expression-based test	NA	Risk category groups* comparing Oncotype DX® vs MammaPrint® and Oncotype DX® vs PAM50-ROR Overall intrinsic molecular subtypes by different gene list and microarray platform intrinsic subtype centroid predictors63,107–109 HER2E/luminal A/luminal B/normal-like by different gene list and microarray platform intrinsic subtype centroid predictors (microarray-based)63,107–109

^*All risk category groups have been calculated in the NKI295 dataset.¹⁵ For κ value calculation, low and intermediate groups of Oncotype DX® have been combined into a low-risk group when compared with two risk groups from MammaPrint®. For intrinsic molecular subtyping, a higher level of concordance is achieved when performing microarray platform to platform normalization, which was not performed by Weigelt et al.⁶³

Abbreviations: ER, estrogen receptor; FISH, fluorescence in situ hybridization; HC, hierarchical clustering; HER2E, HER2-enriched; IHC, immunohistochemistry; LBA, ligand-binding assay; NA, not applicable; PR, progesterone receptor.

When two observers, or laboratories, use identical protocols (inter-observer and inter-laboratory studies), level 1 agreement is achieved for ER status defined by IHC and HER2 status defined by fluorescence in situ hybridization (FISH) when scored as a two-class variable (variable of two elements: positive versus negative).^64–76 For HER2 status, which is usually scored as a three-class variable (0/1+, 2+ and 3+), level 1 agreement is observed in most inter-observer studies,^65,77–79 whereas level 2 agreement is observed in inter-laboratory studies.^71,73,76,79 The lower agreement achieved when the same HER2 IHC protocol is carried out across two different laboratories might be due to pre-analytical variables such as fixation times and/or in the critical area of subjectivity in IHC interpretation, especially for the HER2 2+ cases.^80,81 In the NCCTG-N9831 trial, Perez et al.⁸² reported that 18.4% of 636 samples that were identified as HER2 3+ by the FDA-approved HercepTest™ (Dako, Glostrup, Denmark) were not confirmed centrally by the same assay; similar findings were noted in the NSABP-B-31 trial.⁸³

Concordance between two different ER or PR testing methods—such as the concordance between the ligand-binding biochemical assay and IHC^{64,69,84–86}—or between two different antibodies for ER,^69,85,87 falls between level 2 and 3 (Table 3). It should be noted that different antibodies are allowed in the USA for ER and PR testing, and thus, level 2–3 is the current standard of reproducibility. For example, Rhodes et al.⁸⁸ assessed histological sections from breast cancers with low, medium and high levels of ER expression and sent the samples to 200 laboratories for IHC staining where in-house techniques were used. The results were then centrally evaluated. The inter-method staining results showed that 80% of laboratories were able to demonstrate ER positivity in tumors with medium and high expressing ER levels, whereas only 37% of laboratories scored adequately for the low-expressing tumors. The low concordance within low ER-expressing tumors, together with data suggesting that these tumors might still derive benefit from hormonal therapy,^86,89 has led the ASCO–CAP guidelines to recommend 1% positive cells or greater as the cutoff for identifying a sample as ER-positive.⁵⁹ Multi-class scoring of ER and PR into four or more classes (using one antibody) by two pathologists shows level 3 concordance.^64,67,90 Lastly, level 2 and 3 concordances are typical for histological grade as estimated by two or more pathologists;^91–99 these levels of concordance are considered sufficient by some investigators to recommend using tumor grade to help make therapeutic decisions.¹⁰⁰

Concordance of HER2 status by different IHC based-antibodies,^69,85,87 or IHC versus FISH,^77,101–104 also falls between level 2 and 3 (Table 3). In the NSABP B-31 trial, Paik et al.⁸³ found that 45% of 24 samples that had a community-based HER2 3+ score using a non-HercepTest™ IHC test could not be confirmed by central HercepTest™. Interestingly, concordance of HER2 FISH and HER2 gene expression, as determined by central laboratories, falls within level 1 agreement,^105,106 indicating that HER2 gene expression might be a more reliable surrogate of HER2 amplification than HER2 protein by IHC.

Overall, these data show that level 1 concordance can only be achieved in the clinical setting when identical protocols are used (for example, the same IHC antibody); however, this is not the current standard;^58,59 instead, the current standards for ER, PR and HER2 testing allow for different antibodies to be used, or allow different technologies to be used (FISH versus IHC), and in these cases, level 2–3 concordance is observed and accepted for therapeutic decision making. These results highlight the need to improve the reproducibility of current diagnostic tests for patients with breast cancer, and we suggest that medical practice should strive for level 1 concordance.

Reproducibility of gene-expression biomarkers

The Oncotype DX® and MammaPrint® assays have shown level 1 κ concordance values and/or concordance correlation coefficients >0.90 when tested using the same starting materials and when repeated on different days, with different reagent batches, and by different technicians.^,8,56,57 Clinical implementation of the 50-gene PAM50 subtype predictor is currently underway, and thus, no published data exist to judge its reproducibility; however, over the past decade, various research laboratory-based intrinsic subtype predictors have been developed using the most advanced DNA microarrays and statistical methods that are currently available. The identification of the intrinsic subtypes by multiple investigators across the world, using many different expression technologies, has been accomplished and has had an enormous impact, so that it is now commonplace to carry out research-based intrinsic subtyping as a means of anchoring new findings to the literature.

The inter-observer concordance of different intrinsic subtype classifications using DNA microarrays has been recently evaluated.⁶² Mackay et al.⁶² carried out 15 hierarchical clustering analyses using five independent breast cancer microarray datasets and three previously published intrinsic gene lists, and then estimated the agreement in identifying the intrinsic subtypes (luminal A, luminal B, HER2-enriched, basal-like and normal-like) across five observers. The results showed that the overall agreement was 0.64–0.70 (level 2), and was >0.81 (level 1) for the basal-like and HER2-enriched subtypes, and substantial (κ = 0.6–0.80, level 2) for the other subtypes (luminal A, luminal B and normal-like). Thus, the inter-observer concordance of the intrinsic subtypes in the research setting by hierarchical clustering analysis showed level 2 agreement (thus equivalent to current ER and HER2 testing), except for the basal-like and HER2-enriched subtypes that showed even better level 1 agreement.

The inter-method concordance of different molecular subtype classifications using DNA microarrays was evaluated by Weigelt et al.⁶³ who compared three different intrinsic subtype prediction methods, each of which differed in three technological means (different gene lists, different tumor training data sets, and different microarray platforms); these analyses showed an overall κ value agreement of 0.24–0.74, and level 1 agreement for the basal-like subtype, whereas the other subtypes had level 2–3. This experiment would be similar to testing ER concordance using two different antibodies and the ligand-binding assay, determining cut points using three different patient training sets, and then comparing classification concordance results on yet other data sets. In addition, these substantial inter-method concordance values were obtained despite the fact that the Weigelt et al.⁶³ analyses had several bioinformatic issues including a lack of microarray platform normalization.^107–109 In the research setting, when using two different normalization procedures on the same data set before PAM50 intrinsic subtype classification, the κ concordance value for the five subtypes will drop from level 1 to level 2 (A. Prat and C. M. Perou, unpublished observations).

Similar to ER IHC and HER2 FISH, level 1 concordance for any gene-expression-based assay will only be achieved when using a single technology platform and a standardized protocol. This protocol is the current standard for Oncotype DX® and MammaPrint®, and the future protocol for intrinsic subtyping via the PAM50 assay. Overall, these results show that intrinsic subtyping, as carried out in the research setting, is reproducible across platforms and gene lists, and shows κ concordance values equivalent to current clinical testing for histological grade, ER, PR and HER2 testing. The next challenge for gene-expression- based assays will be to develop distributed assays that can maintain a similar level of analytical reproducibility and clinical utility, as has been demonstrated by single laboratories. Interestingly, a recent multisite reproducibility study of an FDA-approved microarray-based gene-expression test for identifying the tissue site of origin for tumors of unknown origins, known as Pathwork® (Pathwork Diagnostics, Redwood City, CA), obtained κ concordance values >0.81 when testing 149 paired samples.¹¹⁰

The next decade

In the 12 years after the first publication of a gene-expression- based study on breast tumors,²⁵ it is clear that the information provided by gene-expression-based assays is useful and deserves incorporation into clinical decision making. In fact, the term ‘molecular portraits’ was coined based on the fact that in the vast majority of tumors, different pieces of the same tumor, or primary tumor-metastasis pairs, are paired together when using gene-expression data and a hierarchical clustering analysis.^22–24,111 To further illustrate this point, we re-analyzed 43 tumors with two or more distinct tumor aliquots or fragments (for example, one primary tumor cut into two fragments or a tumor-metastasis pair assessed with two RNA preps, and two microarrays) that were independently arrayed using data from Harrell et al.,¹¹² and found that 41 of 43 were paired together (Figure 1). If we call pairing a successful classification (that is, inter-individual concordance), this 43 class classification analysis yields a κ value of 0.95 (near perfect agreement). Thus, gene-expression analysis not only accurately classified samples into common groups, but is also able to identify the true individuality of each tumor, thus showing that each tumor is in a unique and reproducible class by itself.

Figure 1.

Identification of tumor individuality using global gene-expression analyses. a | Supervised hierarchical clustering of breast cancer data from Harrell et al.¹¹² of 367 breast samples using 1,900 intrinsic genes. Paired tumor samples are highlighted by the red lines in the array tree, with 41 out of 43 possible pairs being paired. The two paired samples that did not pair together are shown in green and light blue color. Note that one of each paired sample was a primary tumor sample that was present within the normal-like cluster, which suggests that both were highly contaminated with true normal breast tissue. b | A subset of the basal-like array tree showing the different array names. Note that some tumors had more than two samples, and in all cases, these were all grouped together. Abbreviations: INT, second partner of an intrinsic pair (that is, two distinct pieces of the same primary tumor); LN, lymph-node metastasis; Met, distant metastasis; Meta, metaplastic.

Conclusions

Current clinical recommendation for Oncotype DX® from ASCO–CAP and NCCN guidelines and the 2011 St Gallen consensus report support the use of gene- expression-based assays in the management of breast cancer. In the coming decade it is also likely that further improvements in gene-expression-based assays will occur and make a clinical impact, as will the discovery and implementation of new gene signatures. In addition, new and exciting technologies will also make a clinical diagnostic impact, with the most immediate possibilities being based on next-generation sequencing technologies, proteomics, or in the isolation and characterization of special cell populations such as circulating tumor cells. The first complete breast-cancer genomes have been published and revealed a myriad of somatic mutations,^113,114 some of which may be therapeutically targetable. These are exciting times and we look forward to continued refinements of existing assays and new discoveries in the decade to come, so that outcomes for patients with breast cancer will continue to improve.

Key points.

• Gene-expression-based assays provide independent prognostic information beyond standard clinical-pathological variables; however, tumor and nodal stage remain important and must be taken into account in the final prognostic assessment

• Gene-expression-based assays identify patients with ER-positive node-negative disease at low risk of relapse after treatment with hormonal therapy and who might be spared from chemotherapy

• Clinical use of gene-expression-based assays for the prediction of chemotherapy benefit in node-positive disease, and in ER-negative disease, is currently experimental

• Current methodologies for ER, PR and HER2 testing might benefit from additional protocol standardizations, but may still be less reproducible than standardized gene-expression-based assays

• Non-standardized research-based identification of the intrinsic subtypes shows concordance values equivalent to current clinical testing for histological grade, ER, PR and HER2

• For daily clinical use, we recommend the highest level of reproducibility/concordance (Level 1), which will only be achieved for pathology and gene-expression-based tests by using a single platform and standardized protocol