Strength of Validation for Surrogate End Points Used in the US Food and Drug Administration’s Approval of Oncology Drugs

Abstract

Objective

To determine the strength of the surrogate-survival correlation for cancer drug approvals based on a surrogate.

Participants and Methods

We performed a retrospective study of the US Food and Drug Administration (FDA) database, with focused searches of MEDLINE and Google Scholar. Among cancer drugs approved based on a surrogate end point, we examined previous publications assessing the strength of the surrogate-survival correlation. Specifically, we identified the percentage of surrogate approvals lacking any formal analysis of the strength of the surrogate-survival correlation, and when conducted, the strength of such correlations.

Results

Between January 1, 2009, and December 31, 2014, the FDA approved marketing applications for 55 indications based on a surrogate, of which 25 were accelerated approvals and 30 were traditional approvals. We could not find any formal analyses of the strength of the surrogate-survival correlation in 14 out of 25 accelerated approvals (56%) and 11 out of 30 traditional approvals (37%). For accelerated approvals, just 4 approvals (16%) were made where a level 1 analysis (the most robust way to validate a surrogate) had been performed, with all 4 studies reporting low correlation (r≤0.7). For traditional approvals, a level 1 analysis had been performed for 15 approvals (50%): 8 (53%) reported low correlation (r≤0.7), 4 (27%) medium correlation (r>0.7 to r<0.85), and 3 (20%) high correlation (r≥0.85) with survival.

Conclusions

The use of surrogate end points for drug approval often lacks formal empirical verification of the strength of the surrogate-survival association.

The US Food and Drug Administration (FDA) may grant oncology drugs either accelerated (provisional) (AA) or traditional (full) (TA) marketing approval.¹ Accelerated approvals are given based on a surrogate end point that is “reasonably likely to predict” true clinical efficacy, ie, survival or quality of life.^2–4 Traditional approvals are granted when a drug demonstrates “a longer or better life or a favorable effect on an established surrogate for a longer or better life.”⁴ Surrogate end points, thus, play a prominent role in oncology drug approvals, with the strength of the surrogate end point guiding the pathway of approval.

When relying on surrogates to guide clinical and regulatory decisions, it is important that the surrogate-survival correlation is robust to avoid the approval of toxic drugs with no benefit. Bevacizumab received AA in 2008 based on data that it markedly improved progression-free survival (PFS).⁵ However, by 2011, that approval was withdrawn when multiple studies found that the drug did not improve overall survival (OS) and carried toxicity and that gains in PFS were smaller than initially appreciated.⁶ In retrospect, the approval and subsequent withdrawal of bevacizumab in metastatic breast cancer is not surprising given that multiple validation studies found that this specific surrogate-survival association is weak.⁷

The validation of surrogate end points in oncology is an increasingly important field, with different statistical methods used.^8–12 We favor a clear and simple hierarchy to grade the strength of surrogate-survival correlations.^7,13 In this model, level 3—the lowest level—requires the surrogate-survival correlation to be only biologically plausible. Level 2 and level 1 analyses require clinical data. Although level 2 analysis shows that the surrogate is associated with the final outcome across groups, level 1 analysis addresses the clinically relevant question of whether improving the surrogate end point is associated with improvements in survival across many randomized studies. Typically, regression analysis is performed in level 1 studies. The x coordinate reflects the change in surrogate end point, and the y coordinate reflects the change in final end point. Correlation coefficients (r) closer to 1 signify stronger associations. As such, the validation of surrogate-survival associations in oncology exists along an established hierarchy.

We set out to characterize the nature of FDA approvals in oncology from 2009 through 2014. Specifically, what percentages of approvals were accelerated and traditional? Among TAs, what percentage were made based on a surrogate end point? For all approvals granted on the basis of surrogates, what is the documented strength of the surrogate-survival association? Finally for drugs approved based on surrogates, have subsequent trials demonstrated improvements in survival or quality of life? In short, we set out to empirically describe the strength of evidence for 6 years of FDA cancer drug approvals.

METHODS

Data Source

The FDA provides a record of hematology and oncology drug approvals and safety notifications on their website (http://www.fda.gov/Drugs/InformationOnDrugs/ucm279174.htm) and in related links. Each relevant webpage was downloaded and is provided in the Supplemental Figure 1 (available online at http://www.mayoclinicproceedings.org). Further information for each approval was obtained from the Drugs@FDA website, which includes information regarding the approval of new oncology drugs as well as expanded indications for currently approved drugs, date of approval, basis of approval, and a summary of the clinical review that supported the approval.

Study Sample

We identified all oncology drugs approved by the FDA between January 1, 2009, and December 31, 2014, the last complete year at the time of this investigation. Oncology drugs were approved based on improvements in OS or one of the following surrogate end points: improvements in disease response rate (eg, hematologic, pathologic, or tumor response) or delay in progression (eg, improved PFS or recurrence-free survival). We included data on new oncology drugs and on new indications for previously approved oncology drugs.

End Points Extracted

We ascertained the total number of AAs and TAs. We noted the efficacy end point leading to approval. When drugs were approved on the basis of improvement in OS or quality of life—measures of patient-centered benefit—we performed no further investigation. When drugs were approved based on a surrogate end point, we investigated formal analyses of the surrogate-survival correlation and whether subsequent publications have found an OS benefit.

Literature Search

We sought to ascertain the strength of the surrogate-survival correlation. In other words, as the criteria for AA and TA based on surrogates are “reasonably likely to predict” and “established,” respectively, we sought to evaluate the practical meaning of these terms.

For each surrogate drug approval, we performed a focused review of the literature to identify available surrogate-survival association studies. Surrogate association studies are widely performed in oncology to assess the strength of the surrogate end points.¹⁴ These studies are often meta-analyses of randomized controlled trials conducted in the same setting as the particular indication of the drug approval. For example, if one wants to know whether PFS correlates with OS in metastatic castrate-resistant prostate cancer, one begins by collecting all randomized controlled trials in this setting. Then one plots whether the hazard ratio or change in PFS (x coordinate) predicts the hazard ratio or change in OS (y coordinate). Regression analysis is conducted across trials to demonstrate the general correlation between the surrogate and survival. For each specific surrogate drug approval identified, we performed a review of the literature to locate such analyses. Multiple searches were performed, and all the search terms used and databases searched are listed in Supplemental Table 1 (available online at http://www.mayoclinicproceedings.org). Two CONSORT diagrams show (1) the number of articles retrieved and the percentage included for AAs and (2) AAs and TAs combined (Supplemental Figures 2 and 3, available online at http://www.mayoclinicproceedings.org).

Grading the Strength of Correlation

We scored the strength of correlation for level 1 studies based on a modification of criteria proposed by the Institute for Quality and Efficiency in Health Care,¹⁵ as we have done previously⁷: low correlation (r≤0.7), medium correlation (r>0.7 to r<0.85), and high correlation (r≥0.85). The specific cutoff points were adapted to function even when confidence intervals were not presented. If coefficients of determination (R²) were given instead of correlation coefficients (r), we calculated the r by taking the square root. Where multiple level 1 studies existed, the median r was used for scoring. We repeated the analysis using the best r, which did not materially change the results (data not shown). We could not score level 2 studies because a variety of analyses and reported measures were used.

Subsequent Publications

For all drugs approved on the basis of a surrogate end point, we performed a focused search of the published literature to identify subsequent publications that report whether the drug improved OS, as we have also done previously, albeit in a set of different years (2008–2012).¹⁶ We credited a drug for improving OS if that drug improved survival in any combination (even beyond a combination that received approval) or in any line of treatment (eg, if the drug was approved for second line but improved survival in first line, we would credit the drug as improving survival). The search terms and databases included are also given in Supplemental Table 2 (available online at http://www.mayoclinicproceedings.org).

RESULTS

Between January 1, 2009, and December 31, 2014, the FDA approved marketing applications for 83 oncologic indications: 25 (30%) were AAs and 58 (70%) were TAs. Of the AAs, 24 (96%) were based on response rate (or duration of response) and 1 (4%) was based on PFS. Of the TAs, 28 (48%) were based on either improved OS or quality of life—patient-centered end points—and 30 (52%) were based on a surrogate end point, such as response rate (7 indications, 12% of TAs) or PFS or disease-free survival (23 indications, 40% of TAs). Figure 1 shows each approved drug, the year of approval, the indication for approval, the pathway for approval (traditional or accelerated), and the clinical end point supporting claims of efficacy at the time of approval.

FIGURE 1.

All indications receiving Food and Drug Administration marketing authorization for oncology drugs between 2009 and 2014. Approvals are grouped based on traditional or accelerated authorization and the efficacy end point met to garner approval. *Drugs approved based on bioequivalence (mercaptopurine (2014): ALL, asparaginase Erwinia chrysanthemi (2011): ALL) were removed from the analysis. ALL = acute lymphoblastic leukemia; CLL = chronic lymphocytic lymphoma; CML = chronic myeloid leukemia; CRC = colorectal carcinoma; CRPC = castration-resistant prostate cancer; CTCL = cutaneous T-cell lymphoma; DFS = disease-free survival; GEJ = gastroesophageal junction; GIST = gastrointestinal stromal tumor; HER2 = human epidermal growth factor receptor 2; NET = neuro-endrocine tumor; NSCLC = non-small cell lung cancer; OS = overall survival; PFS = progression-free survival; Ph = Philadelphia chromosome; PTCL = peripheral T-cell lymphoma; RCC = renal cell cancer; RR = response rate; SLL = small lymphocytic lymphoma.

Accelerated approvals are made on the basis of a surrogate that is reasonably likely to predict true clinical efficacy. As shown in Figure 2, A, we could not find any formal analyses (studies that assessed level 1 or level 2 surrogacy) of the strength of the surrogate-survival correlation in 14 drug approvals (56%). In 7 instances (28%), a level 2 analysis (but not a level 1 analysis) could be found.^17–22 The specific correlations established in those analyses are shown in Table 1. In 4 cases (16%), a level 1 analysis had been performed,^{24,25,28–33,41,42} with all studies reporting low correlation (r≤0.7).

FIGURE 2.

The strength of evidence between a surrogate-survival correlation for accelerated (A) and traditional (B) drug approvals based on surrogate end points. Level 1 studies were scored based on a modification to criteria proposed by the Institute for Quality and Efficiency in Health Care: low correlation (r≤0.7), medium correlation (r>0.7 to r<0.85), and high correlation (r≥0.85). No level 1 or 2 means that we could not identify a single association study in the literature. Where multiple level 1 studies exist, the median r was used for scoring.

TABLE 1.

Published Surrogate Correlation Studies for Accelerated Approvals

Tumor type	Approved drug (year)	Basis of accelerated approval	Studies assessed level 2 surrogacy	Studies assessed level 1 surrogacy
Breast cancer	Pertuzumab (2013)	pCR	Kong et al,23 2011: OROS vs pCR=3.44, P<.00001	Berruti et al,24 2014: rHR OS vs pCR=0.3 Cortazar et al,25 2014: rHR OS vs pCR=0.49
Breast cancer	Lapatinib (2010)	PFS	Ciccarese et al,26 2007 (abstract): rOS vs DFS=0.77 Matsubara et al,27 2011: HROS vs PFS=0.93, P=.31	Hackshaw et al,28 2005: rHR OS vs TTP=0.75 Burzykowski et al,29 2008: rHR OS vs PFS=0.48 Miksad et al,30 2008: rHR OS vs. PFS=0.7 for anthracyclines Miksad et al,30 2008: rHR OS vs PFS=0.59 for taxanes Ng et al,31 2008: rdiff OS vs DFS=0.62 Sherrill et al,32 2008: rHR OS vs TTP=0.55 Beauchemin et al,33 2014: rdiff OS vs PFS/TTP=0.43
Chronic lymphocytic leukemia	Ibrutinib (2014) Ofatumumab (2009)	ORR	-	-
Chronic myeloid leukemia	Ponatinib (2012) Omacetaxine (2012) Dasatinib (2010) Nilotinib (2010)	MCyR MaHR CCyR MMR	Rosti et al,21 2003: rOS vs MCyR=0.66 Oriana et al,20 2013: P=.02 at 5 y, CCyR vs no CCyR (no report on coefficient) Jain et al,19 2013: P=.01 at 3 y, CCyR vs no CCyR	-
Follicular lymphoma or small lymphocytic lymphoma	Idelalisib (2014)	ORR	-	-
Glioblastoma	Bevacizumab (2009)	ORR	Jaeckle et al,18 2008: P=.01 (no report on coefficient)	-
Hodgkin lymphoma and anaplastic large B cell	Brentuximab vedotin (2011)	ORR	-	-
Mantle cell lymphoma	Ibrutinib (2013)	ORR	-	-
Melanoma	Nivolumab (2014) Pembrolizumab (2014) Trametinib and dabrafenib (2014)	ORR	-	-
Multiple myeloma	Pomalidomide (2013) Carfilzomib (2012)	ORR	van de Velde et al,22 2007: P<.0001 (no report on coefficient) Gay et al,17 2011: OS HRCR vs PR =0.08, P<.001	-
Non-small cell lung cancer	Crizotinib (2011) Ceritinib (2014)	ORR	Splinter,34 1991: rOS vs ORR=0.57 Paesmans et al,35 1997: rOS vs ORR=2.20, P<.0001 Sekine et al,36 1998: rOS vs ORR=0.504 Bruzzi et al,37 2007 (abstract): OS HRresponse vs. no response=0.50, P<.001 Tsujino et al,38 2009: rOS vs ORR=0.258 Mandrekar et al,39 2010: OS HRresponse vs no response=0.66, P=.009 Li et al,40 2012: rOS vs ORR=0.91	Blumenthal et al,41 2015: rHR OS vs ORR=0.3 Johnson et al,42 2006: rdiff OS vs ORR=0.4
Ph— acute lymphoblastic leukemia	Blinatumomab (2014)	CR/reduction in MRD	-	-
Ph— acute lymphoblastic leukemia	Liposomal vincristine (2012)	CR/CRi	-	-
Peripheral T-cell lymphoma	Belinostat (2014) Pralatrexate (2009)	ORR	-	-
Tuberous sclerosis with renal angiomyolipoma	Everolimus (2012)	ORR	-	-
Tuberous sclerosis with subependymal giant cell astrocytoma	Everolimus (2012)	ORR	-	-

CCyR =complete cytogenetic response; CR =complete response; CRi =complete response with incomplete blood count recovery; DFS =disease-free survival; HR =hazard ratio; MaHR =major hematologic response; MCyR =major cytogenetic response; MMR =major molecular response; MRD =minimal residual disease; OR =odds ratio; ORR =objective response rate; OS =overall survival; pCR =pathologic complete response; PFS =progression-free survival; PR =partial response; TTP =time to progression.

When direct patient benefit has not been reported at the time of approval, TAs may be granted on the basis of an established surrogate for clinical benefit. Among 30 TAs based on a surrogate, we could not find any formal analyses of the strength of the surrogate-survival correlation in 11 instances (37%) (Figure 2, B). In 4 cases (13%), a level 2 analysis had been performed.^{19–21,43,44} The specific correlations established in those analyses are shown in Table 2. In 15 approvals (50%), a level 1 analysis had been performed to evaluate the strength of the surrogate-survival association, with 8 (53%) reporting low correlation (r≤0.7),^{28–33,41,42,45,49–53} 4 (27%) reporting medium correlation (r>0.7 to r<0.85),^56–59 and 3 (20%) reporting high correlation (r≥0.85) with survival.⁴⁶ In only 7 of 30 TAs (23%) did a level 1 analysis demonstrate medium or high correlation with survival.^{46,56,57,59,60}

TABLE 2.

Published Surrogate Correlation Studies for Traditional Approvals Based on a Surrogate

Tumor type	Approved drug (year)	Basis of traditional approval	Studies assessed level 2 surrogacy	Studies assessed level 1 surrogacy
Basal cell carcinoma	Vismodegib (2012)	ORR	-	-
Breast cancer	Pertuzumab (2012)	PFS	Ciccarese et al,26 2007 (abstract): rOS vs DFS=0.77 Matsubara et al,27 2011: HROS vs PFS=0.93, P=.31	Hackshaw et al,28 2005: rHR OS vs TTP=0.75 Burzykowski et al,29 2008: rHR OS vs PFS=0.48 Miksad et al,30 2008: rHR OS vs PFS=0.7 for anthracyclines Miksad et al,30 2008: rHR OS vs PFS=0.59 for taxanes Ng et al,31 2008: rdiff OS vs DFS=0.62 Sherrill et al,32 2008: rHR OS vs TTP=0.55 Beauchemin et al,33 2014: rdiff OS vs PFS/TTP=0.43
Chronic lymphocytic leukemia	Idelalisib (2014) Ofatumumab (2014) Obinutuzumab (2013)	PFS	-	-
Chronic myeloid leukemia	Bosutinib (2012)	MCyR Complete hematology response Overall hematologic response	Rosti et al,21 2003: rOS vs MCyR=0.66 Oriana et al,20 2013: P=.02 at 5 y, CCyR vs no CCyR (no report on coefficient) Jain et al,19 2013: P=.01 at 3 y, CCyR vs no CCyR	-
Cutaneous T-cell lymphoma	Romidepsin (2009)	ORR	-	-
Differentiated thyroid cancer	Sorafenib (2013)	PFS	-	-
Follicular lymphoma	Rituximab (2011)	PFS	-	-
Gastroenteropancreatic neuroendocrine tumor	Lanreotide (2014) Sunitinib (2011) for pancreatic NET Everolimus (2011) for pancreatic NET	PFS	-	Singh et al,45 2014: rHR OS vs PFS/TTP=0.17
Giant cell tumor	Denosumab (2013)	ORR and DOR	-	-
Mantle cell lymphoma	Lenalidomide (2013)	ORR and DOR	-	-
Medullary thyroid carcinoma	Cabozantinib (2012) Vandetenib (2011)	PFS	-	-
Melanoma	Trametinib (2013) Dabrafenib (2013) Peginterferon-alpha 2b (2013)	PFS	-	Flaherty et al,46 2014: rHR OS vs PFS=0.89
Non-small cell lung cancer	Nab paclitaxel (2012)	ORR	Splinter,34 1991: rOS vs ORR=0.57 Paesmans et al,35 1997: rOS vs ORR=2.20, P<.0001 Sekine et al,36 1998: rOS vs ORR=0.504 Bruzzi et al,37 2007 (abstract): OS HRresponse vs no response=0.50, P<.001 Tsujino et al,38 2009: rOS vs ORR=0.258 Mandrekar et al,39 2010: OS HRresponse vs no response=0.66, P=.009 Li et al,40 2012: rOS vs ORR=0.91	Blumenthal et al,41 2015: rHR OS vs ORR=0.3 Johnson et al,42 2006: rdiff OS vs ORR=0.4
	Crizotinib (2014) Afatinib (2014) Erlotinib (2013)	PFS	Hayashi et al,47 2012: rPFS vs OS=0.43 Sekine et al,48 1999: rPFS vs OS=0.80	Blumenthal et al,41 2015: rHR OS vs PFS=0.28 Suzuki et al,49 2014: rHR OS vs PFS=0.41 Hayashi et al,50 2013: rdiff OS vs PFS=0.29 Hotta et al,51 2013: rHR OS vs PFS=0.37 Mauguen et al,52 2013: rHR OS vs DFS=0.96 Hotta et al,53 2009: rOS ratio vs TTP ratio=0.57
Ovarian cancer	Olaparib (2014)	ORR	Rose et al,44 2010: rOS vs ORR=0.56	-
Ovarian/fallopian/primary peritoneal cancer	Bevacizumab (2014)	PFS	Rose et al,44 2010: rOS vs PFS=0.66	-
Renal cell carcinoma	Axitinib (2012) Pazopanib (2009) Bevacizumab (2009) Everolimus (2009)	PFS	Halabi et al,54 2014: HROS vs progression at 6 mo=2.8, P<.0001 Heng et al,55 2011: rOS vs PFS=0.66	Bria et al,56 2015: r12 mo OS vs 3 mo PFS=0.82, r12 mo OS vs 6 mo PFS=0.85 Johnson et al,57 2015: rdiff OS vs PFS=0.7 Petrelli and Barni,58 2013: rdiff OS vs PFS=0.36 Delea et al,59 2012: rdiff OS vs PFS/TTP=0.53 rHR OS vs PFS/TTP=0.79
Soft-tissue sarcoma	Pazopanib (2012)	PFS	Penel et al,43 2013: rOS vs PFS=0.4	-

CCyR =complete cytogenetic response; CRi =complete response with incomplete blood count recovery; DFS =disease-free survival; DOR =duration of response; HR =hazard ratio; MaHR =major hematologic response; MCyR =major cytogenetic response; MMR =major molecular response; MRD =minimal residual disease; NET =neuroendrocine tumor; ORR =objective response rate, OS =overall survival; pCR =pathologic complete response; PFS =progression-free survival; TTP =time to progression.

When an oncology drug is approved based on a surrogate, subsequent studies or longer follow-up for ongoing studies may report OS benefits. Table 3 summarizes the results of this review of subsequent publications for all marketing indications approved on the basis of a surrogate. Although 10 of 55 approvals (18%) were later found to carry an OS benefit, 15 (27%) were found not to improve OS. Most marketing approvals 30 of 55 (55%) remain untested. Traditional approvals based on surrogates were more likely not to show survival benefits than AAs (40% vs 12%; P=.02). The use of crossover occurred in 48% of these studies and did not vary among trials that found a survival advantage vs those that did not (4 of 10 [40%] vs 8 of 15 [53%]; P=.52).

TABLE 3.

Surrogate-Based Approvals for Which Subsequent Trials Report an OS Benefit or a Lack of Survival Benefit or for Which No Trials Exist Showing or Refuting a Survival Benefit

Indication	Approvals (No. [%])
	Proven OS benefit	No OS benefit	OS benefit unknown
Total (n=55)	10 (18.2)	15 (27.3)	30 (54.5)
Accelerated approval (n=25)	6 (24.0)	3 (12.0)	16 (64.0)
Traditional approval (n=30)	4 (13.3)	12 (40.0)	14 (46.7)

OS =overall survival.

Combining the results of the 2 analyses, we find that among 55 drugs approved on the basis of a surrogate, 10 (18.2%) have shown OS benefits, and another 2 (3.6%) have not shown OS benefits, but were approved on the basis of a high-strength correlation in a level 1 analysis. Thus, only 12 (21.8%) of 55 surrogate approvals were made on the basis of a strong surrogate-survival correlation or later showed OS benefits.

DISCUSSION

We found that most cancer drug approvals (55 of 83 [66%]) are based on a surrogate end point. Although the FDA grants TA based on established surrogate end points, this standard is lax. Only 3 of 30 such approvals (10%) have shown high correlation in a level 1 surrogate analysis, widely considered a prerequisite for clinical or regulatory decisions.¹⁵ Of concern, 11 of 30 TAs (37%) had no formal analyses of the surrogate-survival correlation. Accelerated approvals are granted on the basis of a surrogate that is reasonably likely to predict clinical benefit; but again, practically, this standard has not been enforced, with 56% of approvals (14 of 25) made without any formal analysis of surrogacy.

The frequent use of surrogate end points in FDA approval is paralleled by a rise in the use of these end points as the primary end point of clinical trials. Examining randomized controlled trials from 1974 through 2009 for non-small cell lung cancer, breast cancer, and colorectal cancer in 5 major journals, PFS increased in frequency as the primary end point of trials from 0% (1975–1984) to 20% (2005–2009).⁶¹ Sacher et al⁶² found, in an exhaustive look at trials for non-small cell lung cancer, that OS declined as the primary end point of lung cancer trials from 97% (1980–1990) to 96% (1991–2000) to 81% (2001–2010). This trend was accompanied by a rise in the PFS end point (a surrogate) as the primary end point. The improvement of a surrogate end point in oncology can be used to petition for drug approval^4,63 and to expand clinical guidelines, which in many cases obliges insurers and Medicare to cover drugs. It is likely that the validation of surrogate end points in cancer would benefit from a formal set of guidelines, as was done decades ago for general statistical presentations in medicine.⁶⁴

There are at least 2 potential reasons why surrogates may not correlate with OS for new oncology drugs. The first is that in contrast with OS, where the date of death is precise and can be ascertained for all patients, surrogates are prone to reader interpretation, measurement error, evaluation bias, and attrition bias.^65,66 These artifacts may create spurious surrogate benefits. The second explanation is a biological one: a cancer drug with favorable surrogate effects may affect changes in tumor growth or aggressiveness or may increase off-target deaths.^67–70 For these reasons, it is important that a surrogate is validated in a precise context (eg, whether disease-free survival correlates with OS for cytotoxic agents in the adjuvant treatment of colorectal cancer). Validation is performed for the unique combination of disease setting (adjuvant), tumor type (colorectal), class of agents (cytotoxic), and particular surrogate end point (disease-free survival).

Regulatory language appreciates these concerns and allows AA for surrogates reasonably likely to predict true efficacy and TA for surrogates that are established. Although there is clearly flexibility in this language, we believe that, at a minimum, the language implies some previous formal analysis of the surrogate-survival relationships. Yet, in 25 of 55 drug approvals based on a surrogate end point, we found no published analysis assessing the robustness of the surrogate-survival correlation. Although it is possible that the FDA has conducted internal and unpublished analyses in these cases, we consider that unlikely because the FDA has announced and published other analyses they have commissioned seeking to assess a surrogate.²⁵ Without any formal analysis, we contend that it is incorrect to consider a surrogate as established.

Subsequent studies do not lend much clarity to surrogate approvals. In the present analysis, we found that less than half of the surrogate approvals (25 of 55, 45%) had a subsequent analysis of survival, and, when they did, such analyses concluded approximately 3 to 2 (15:10) that the drug did not improve survival. The present results regarding subsequent trials in the present data set are similar to the fate of drugs approved from 2008 through 2012, as we reported previously.¹⁶

A crossover design was used in 48% of these studies, and they did not vary among trials that found a survival advantage vs those that did not. As such, the present data provide further evidence to question the common narrative concerning crossover: that it masks OS benefits that truly exist.^71,72 Alternative explanations include that crossover obscures the fact that survival benefits do not exist and prevents the ability to observe late toxicity. Moreover, despite crossover, some drugs have shown survival benefits.⁷³

Finally, note that the sizable use of unvalidated (and altogether untested) surrogates to approve cancer drugs may further undermine the ability to conduct definitive trials of precision medicine. Once drugs are available, patients are naturally reluctant to participate in trials assessing their fundamental efficacy. Instead, we increasingly have to rely on case reports,⁷⁴ a notoriously unreliable way to assess claims of efficacy.

There are several limitations to this analysis. Whether an end point is truly a surrogate or a measure of clinical benefit continues to be subject to debate in oncology.⁶⁸ We considered improvements in patient-reported outcomes, quality of life, and OS to constitute a patient-centered benefit and improvements in all other outcomes as surrogate to this. Others may believe that under certain circumstances radiographic PFS becomes clinically meaningful. However, because this end point includes events that patients may not physically be aware of (radiographic progression), as such, we believe that our classification is technically accurate. Future research is needed to delineate the relationship between PFS and quality of life across tumor settings.

The use of follow-up studies remains inadequate among cancer drugs approved based on a surrogate, with 16 of 25 AAs and 14 of 30 TAs lacking a subsequent study reporting on OS. Given the recent nature of the drugs we investigated (2009–2014), it is likely that additional studies will become available in the future. However, this may be more true among AAs, where such postmarketing studies are a requirement for continued authorization, than among TAs, which often do not entail further postmarketing efficacy commitments. Thus, it is possible that some of the estimates change as future data become available. At the same time, it must be acknowledged that the FDA’s enforcement of postmarketing commitments has historically been poor⁷⁵ and that future data will not change the known strength of surrogate-survival correlations at the time of approval, which remains the regulatory basis for approval.

Another limitation is that although the search strategy for surrogate-survival association studies involved the use of 2 search engines, it is possible that we missed such associational studies. However, previously we performed an exhaustive mixed-methods search for surrogate survival correlations,⁷ and, thus, we believe that we have captured most such papers that exist in the biomedical literature.

A final limitation—not necessarily of the study itself but of the topic that we are studying—is that surrogate validation studies likely have publication and selective reporting bias. Previous work⁷ has found that few level 1 validation studies (5 of 36) survey both published and unpublished trials, and when they do, they are able to retrieve and use data from only 51.5% of studies. Although this concern does not pertain to situations in which surrogate validation studies were absent in the present investigation, it suggests that in situations in which validation was present, the strength of correlation may be different based on a more comprehensive analysis using all trials conducted on a topic.

CONCLUSION

Most new cancer drugs are approved on the basis of surrogate end points. The standard for such approvals is that surrogates are reasonably likely to predict clinical efficacy or established in the case of AA and TA, respectively. We found that, practically, this standard is lax, with 56% and 37% of AAs and TAs, respectively, based on surrogates made without any formal analysis of the strength of the surrogate-survival correlation. Additional follow-up to existing trials or new trials were unlikely to be completed or to confirm survival benefits. The present study suggests that the use of surrogate end points for drug approval often lacks formal empirical verification. This practice should be reconsidered.

Abbreviations and Acronyms

AA

accelerated approval

ALL

acute lymphoblastic leukemia

CCyR

complete cytogenetic response

CLL

chronic lymphocytic lymphoma

CML

chronic myeloid leukemia

CRC

colorectal carcinoma

CR

complete response

CRi

complete response with incomplete blood count recovery

CRPC

castrate-resistant prostate cancer

CTCL

cutaneous T-cell lymphoma

DFS

disease-free survival

DOR

duration of response

FDA

Food and Drug Administration

GEJ

gastroesophageal junction

GIST

gastrointestinal stromal tumor

HER2

human epidermal growth factor receptor 2

HR

hazard ratio

MaHR

major hematologic response

MCyR

major cytogenetic response

MMR

major molecular response

MRD

minimal residual disease

NET

neuroendrocine tumor

NSCLC

non-small cell lung cancer

OR

odds ratio

ORR

objective response rate

OS

overall survival

pCR

pathologic complete remission

PFS

progression-free survival

Ph

Philadelphia chromosome

PR

partial remission

PTCL

peripheral T-cell lymphoma

RCC

renal cell cancer

RR

response rate

SLL

small lymphocytic lymphoma

TA

traditional approval

TTP

time to progression