Predictive biomarker validation in practice: lessons from real trials

Abstract

Background

With the advent of targeted therapies, biomarkers provide a promising means of individualizing therapy through an integrated approach to prediction using the genetic makeup of the disease and the genotype of the patient. Biomarker validation has therefore become a central topic of discussion in the field of medicine, primarily due to the changing landscape of therapies for treatment of a disease and these therapies purported mechanism(s) of action.

Purpose

In this report, we discuss the merits and limitations of some of the clinical trial designs for predictive biomarker validation using examples from ongoing or completed clinical trials.

Methods

The designs are broadly classified as retrospective (i.e., using data from previously well-conducted randomized controlled trials (RCT)) versus prospective (enrichment or targeted, unselected or all-comers, hybrid, and adaptive analysis). We discuss some of these designs in the context of real trials.

Results

Well-designed retrospective analysis of prospective RCT can bring forward effective treatments to marker defined subgroup of patients in a timely manner. An example is the KRAS gene status in colorectal cancer – the benefit from cetuximab and panitumumab was demonstrated to be restricted to patients with wild type status based on prospectively specified analyses using data from previously conducted RCTs. Prospective enrichment designs are appropriate when compelling preliminary evidence suggests that not all patients will benefit from the study treatment under consideration; however, this may sometimes leave questions unanswered. An example is the established benefit of trastuzumab as adjuvant therapy for breast cancer; a clear definition of HER2-positivity and the assay reproducibility have, however, remained unanswered. An all-comers design is optimal where preliminary evidence regarding treatment benefit and assay reproducibility is uncertain (e.g., EGFR expression and tyrosine kinase inhibitors in lung cancer), or to identify the most effective therapy from a panel of regimens (e.g., chemotherapy options in breast cancer).

Limitations

The designs discussed here rest on the assumption that the technical feasibility, assay performance metrics, and the logistics of specimen collection are well established and that initial results demonstrate promise with regard to the predictive ability of the marker(s).

Conclusions

The choice of a clinical trial design is driven by a combination of scientific, clinical, statistical, and ethical considerations. There is no one size fits all solution to predictive biomarker validation.

Introduction

Biomarkers provide the possibility to estimate disease-related patient trajectories (prognostic signatures) and/or to predict patient-specific outcomes in response to specific treatments (predictive signatures) [1–10]. Both prognostic and predictive signatures are becoming increasingly common in cancer treatment to monitor disease severity and to predict the outcome to different treatments [1–10]. A prognostic marker is a single trait or signature of traits that separates a population with respect to the outcome of interest in the absence of treatment, or despite non targeted ‘standard’ treatment. A predictive marker, on the other hand, is a single trait or signature of traits that separates a population with respect to the outcome of interest in response to a particular (targeted) treatment.

Prognostic marker validation is relatively straightforward, and can be established using the marker and outcome data from a cohort of uniformly treated patients with a certain disease having adequate follow-up [11,12]. In the case of a predictive biomarker, the goal is to prospectively identify patients who are likely to have a favorable clinical outcome such as improved survival and/or decreased toxicity to a specific treatment. Predictive biomarker validation therefore is more complex and requires the same standards of evidence as is needed to adopt a new therapeutic intervention [11,12]. This implies that a predictive marker validation is prospective in nature, and the obvious strategy is to conduct a prospectively designed randomized controlled trial (RCT) to test for a marker by treatment interaction. In some instances, where a prospective RCT is not possible due to ethical and logistical (large trial and long time to complete) considerations, a well-conducted retrospective validation can also aid in bringing forward effective treatments to marker defined patient subgroups in a timely manner [12].

The use of a RCT, both in the prospective or retrospective validation setting, is fundamentally essential for predictive marker validation as it assures that the patients who were treated with the agent for whom the marker is purported to be predictive are comparable to those who were not. A nonrandomized design will render the separation of any causal effect of the marker on therapeutic efficacy from the multitude of other factors impossible. One such example is the study that reported on the predictive utility of tumor microsatellite instability for the efficacy of 5-fluorouracil (5-FU)-based chemotherapy in colon cancer [13,14]. The data came from a cohort of nonrandomized patients where the median age of the treated patients was 13 years younger than those of the nontreated patients, thus rendering any meaningful statements about the predictive value of the marker impossibly confounded and therefore questionable [13,14].

In this article, we focus on clinical trial design for predictive marker validation, under the assumption that the methods for assessment of the biomarker are well-established and the initial results show promise with regard to the predictive ability of the marker(s). We discuss the relative merits and limitations of each design using examples from ongoing or completed clinical trials.

Trial designs for predictive marker validation

Several designs have been proposed and utilized in the field of cancer biomarkers for validation of predictive markers. Broadly speaking, these designs can be either retrospective or prospective, where the prospective designs can be further classified into the following four categories: (a) enrichment or targeted designs, (b) all-comers designs, which include the sequential testing strategy designs and marker-based designs, (c) hybrid designs, and (d) adaptive analysis designs. In this report, we discuss the retrospective validation as well as the enrichment and all-comers designs for prospective validation. We refer the readers to other work for a detailed discussion on the hybrid and adaptive analysis designs [11,12,15–17].

Retrospective trial designs

Testing for the predictive effect of a marker through the use of utilizing data collected from previously conducted RCT comparing therapies for which a marker is proposed to be predictive is a feasible and timely option [10]. This strategy maybe a reasonable alternative to a prospective trial when: (1) a prospective RCT is ethically impossible based on results from previous trials, and/or (2) a prospective RCT is not logistically feasible (large trial and long time to complete). The essential elements for such a retrospective analysis to be valid are given below:

• Availability of samples on a large majority of patients to avoid selection bias in the patients who have/do not have the samples.

• Prospectively stated hypothesis, analysis techniques, and patient population.

• Precisely stated algorithm for assay techniques and scoring system.

• Upfront sample size and power justification for all subgroup analyses.

If such a retrospective validation can be demonstrated in data from two independent RCTs, this provides, in our opinion, strong evidence for a robust predictive effect. An example of a marker that has been successfully validated using data collected from previous RCTs is KRAS as a predictor of efficacy of panitumumab and cetuximab in advanced colorectal cancer [18–24]. All ongoing clinical trials sponsored by the U.S. National Cancer Institute (NCI) with these agents in colorectal cancer have been modified to only include KRAS wild type patients, and the label for panitumumab monotherapy has been restricted to KRAS wild type patients in both the United States and Europe. The prospective validation of the retrospectively assessed KRAS gene status in colorectal cancer using data from previously conducted RCTs is discussed below.

In a prospectively specified analysis of data from a previously conducted randomized Phase III trial of panitumumab versus best supportive care (BSC), KRAS status was assessed on 92% (427/463) of the patients enrolled, with 43% having the KRAS mutation [18]. The hazard ratio for treatment effect comparing panitumumab versus BSC on progression free survival in the wild type and mutant subgroups was 0.45 and 0.99, respectively, with a significant treatment by KRAS status interaction (p<0.0001), consistent with data from multiple other phase II trials [18,19]. Similarly, prospectively specified analyses of KRAS status and efficacy of cetuximab from multiple previously conducted Phase III and Phase II trials demonstrated a statistically significant advantage in progression-free or overall survival for patients with wild type KRAS, with no benefit in patients with KRAS mutant status [20–24]. These data consistently demonstrated that the benefit from panitumumab and cetuximab is restricted to patients with wild type KRAS status, with no clinical benefit for patients with mutant KRAS.

Prospective trial designs

While a well conducted retrospective validation study may be acceptable as a marker validation strategy in certain instances, the gold standard for predictive marker validation continues (appropriately) to be a prospective RCT. The strength of the preliminary evidence has a major role in the choice of a design of a prospective marker validation trial. A key component is the hypothesized effectiveness of the new treatment: is it effective in all patients regardless of the marker status or only within certain marker-defined subgroups? We discuss the rationale behind the choice of an enrichment design strategy versus an all-comers design strategy using specific clinical trials as examples.

An enrichment design screens patients for the presence or absence of a marker or a panel of markers, and then only includes patients who either have or do not have a certain marker characteristic or profile. This design is based on the paradigm that not all patients will benefit from the study treatment under consideration, but rather that the benefit will be restricted to a subgroup of patients who either express or do not express a specific molecular feature. This design was utilized in the two large randomized trials of trastuzumab in the adjuvant setting for breast cancer in which only human epidermal growth factor receptor 2 (HER2)-positive patients were eligible on the basis of strong preliminary data. These trials succeeded in identifying a subgroup of patients who received a significant benefit from trastuzumab combined with paclitaxel after doxorubicin and cyclophosphamide treatment [25]. Subsequent post-hoc analyses have, however, raised the possibility of a beneficial effect of trastuzumab in a more broadly defined patient population than that defined in the two trials as well as raised issues regarding the reproducibility and the validity of the assay. Specifically, post-hoc central testing for HER2 expression from the available tumor tissue blocks demonstrated that patients with fluorescent in situ hybridization (FISH) negative tumors and who had less than immunohistochemical (IHC) 3+ staining by central testing also derived benefit from trastuzumab [26,27]. This brings into question the definition of HER2 positivity based on FISH or IHC for the adjuvant disease setting, and whether trastuzumab therapy may benefit a potentially larger group than the approximately 20% of patients defined as HER2 positive in these two trials [26,27]. Additionally, there was a high degree of discordance (approximately 25%) in the HER2 results between central and local testing for IHC and FISH [28]. However, since patients deemed HER2 negative based on the local evaluation were not enrolled onto the trials, questions of assay reproducibility arising from local versus central testing for HER2 status were left unanswered.

While the enrichment strategy did clearly and quickly define an effective treatment for a subset of patients, several other questions regarding the predictive utility of HER2 are left unanswered. Unless there is compelling preliminary evidence that not all patients will benefit from the study treatment under consideration (such as the case with KRAS gene status in colorectal cancer [18–24]), it is prudent to include and collect specimens and follow-up from all patients (since all patients are screened anyway) in the trial to allow for future testing for other potential prognostic markers in this population as well as other marker assessment techniques. This paradigm of collecting specimens from all patients is presently being used in several large ongoing trials in lung cancer, colon cancer, and breast cancer, where the primary aim is to validate a biomarker in either the entire population or only within a marker-defined subgroup [29–33]. In the HER2-trastuzumab setting, if indeed trastuzumab had a beneficial effect in a more broadly defined patient population, an all-comers design strategy, which is discussed below, including both HER2 positive and negative patients may have provided a more definitive answer regarding the predictive utility of HER2.

In the all-comers design, all patients meeting the eligibility criteria are entered into the trial. The ability to provide adequate tissue may be an eligibility criterion for these designs, but not the specific biomarker result or the status of a biomarker characteristic. These designs can be broadly classified as sequential testing strategy designs or marker-based designs, each differentiated from the other by the protocol specified approach to the pre-specified type I and type II error rates (influencing sample size), analysis plans (including a single hypothesis test, multiple tests, or sequential tests), and randomization schema [12].

Sequential testing designs utilize a single primary hypothesis that is either tested in the overall population first and then in a prospectively planned subset, or in the marker-defined subgroup first, and then tested in the entire population if the subgroup analysis is statistically significant. The approach of first testing in the subgroup defined by marker status was implemented in the recently closed (as of November 2009) US-based Phase III trial testing cetuximab in addition to FOLFOX as adjuvant therapy in stage III colon cancer (N0147) (Figure 1). The primary analysis will be conducted at the 0.05 level in the KRAS wild-type patients. Based on the closed testing procedure, if this analysis is statistically significant at p = 0.05, then the efficacy of the regimen in the entire population will also be tested at level 0.05, using the data from approximately 800 patients KRAS mutant tumors who were previously enrolled on this trial prior to the amendment for including only wild type KRAS patients.

Figure 1.

N0147 trial design

An alternate analysis plan for the ‘all-comers’ strategy is the marker by treatment interaction design, which uses the marker status as a stratification factor (i.e., assumes that the overall population can be split into marker defined subgroups) and randomizes patients to treatments within each marker subgroup [12,34,35]. The fundamental difference between this design and a single large RCT is that the marker by treatment interaction design is clearly a prospective (and a definitive) marker validation trial. In addition, in the marker by treatment interaction design, only patients with a valid marker result are allowed to be randomized, the sample size is prospectively specified separately within each marker-based subgroup, and the randomization is stratified by marker status [12]. An example of the marker by treatment interaction design is the biomarker validation study (MARVEL – Marker Validation of Erlotinib in Lung Cancer; Figure 2) of second-line therapy in patients with advanced nonsmall cell lung cancer randomized to receive pemetrexed or erlotinib (N0723). As of December 2009, this trial was permanently closed due to poor accrual partly due to the changes in the standard of care for first and second-line treatment for advanced nonsmall cell lung cancer.

Figure 2.

MARVEL trial design

MARVEL was motivated by (1) the need to obtain prospective evidence to address the conflicting results from several retrospective analyses regarding the predictive role of epidermal growth factor receptor (EGFR) amplification by FISH in the setting of treatment with chemotherapy and EGFR tyrosine kinase inhibitors, and (2) the fact that EGFR FISH represents a poor prognostic factor in untreated NSCLC patients [3,36–42]. The FISH status of the patient was assessed prior to randomization (to ensure adequate number of patients with FISH (+) and FISH (−) status) in a central location (to address issues regarding standardization of assay techniques, reproducibility, and interpretability of assay results). Analysis of survival outcomes at the end of the trial among patients with or without the prespecified molecular markers would have helped to ascertain the presence or absence of true differences in objective clinical endpoints due to treatment, and not due to inherent tumor behavior conferred by the molecular marker. If there was a true difference, the randomization of treatment would have revealed superiority of one treatment arm over the other based on the prespecified molecular marker. Given that the primary goal of this trial was to establish the predictive value of EGFR FISH for selection of an EGFR inhibitor therapy (i.e., Erlotinib), an enrichment design strategy utilizing only the FISH positive subgroup would have failed to identify patients who do not benefit from the therapy. Specifically, patients deemed FISH negative would have never been studied in an enrichment design, thus the benefit or no benefit of the therapy in that subgroup could not have been established.

The marker-based strategy design randomizes patients to have their treatment either based on or independent of the marker status [12,35]. This design includes patients treated with the same regimen on both the marker-based and the nonmarker-based arms resulting in a significant overlap (driven by the prevalence of the marker) in the number of patients receiving the same treatment regimen in both arms. As a consequence, the overall detectable difference in outcomes between the two arms is reduced (depending on the marker prevalence), thus resulting in a comparatively larger trial. A marker-based strategy design is utilized in the ongoing RCT led by the Duke Comprehensive Cancer Center to evaluate the genomic-guided-based chemotherapy for neoadjuvant treatment of breast cancer [43]. A research biopsy is obtained to assess the genomic signature, and subsequently patients are randomized to either the genomics guided arm (marker-based) or the unguided arm (nonmarker-based), with the estrogen receptor status and tumor size as the stratification factors (Figure 3). This marker-based design strategy has the ability to identify the most effective therapy based on a patient’s genetic profile from amongst a panel of standard of care regimens.

Figure 3.

Marker-based strategy design for genomic-guided chemotherapy

Summary

Critical components required for the validation of biomarkers include the choice of an appropriate clinical trial design, the choice of an adequate marker assessment method (IHC, FISH, real time polymerase chain reaction), high dimensional microarray and proteomics-based classifiers, etc.), the reliability and reproducibility of the assay, the logistics and feasibility of obtaining biospecimens, and the costs involved with assessing marker status. The ultimate clinical utility of a biomarker hinges on the added value of the marker assessment in every patient in relation to the prevalence of the marker, specifically the incremental benefit of treatment selection based on the marker compared with the added costs and complexity induced by the measurement of such markers. Moreover, the question of the effectiveness of the new treatment in all patients regardless of the marker status (the magnitude of benefit may differ within the marker-defined subgroups) versus just in the marker defined subgroup(s) needs careful consideration. The translation of the biomarker information to the clinic will require prospective planning, independent validation and availability of data from both retrospective and in some cases prospective trials.