Analysis of Impact of Post-Treatment Biopsies in Phase I Clinical Trials

Abstract

Purpose

The use of biopsy-derived pharmacodynamic biomarkers is increasing in early-phase clinical trials. It remains unknown whether drug development is accelerated or enhanced by their use. We examined the impact of biopsy-derived pharmacodynamic biomarkers on subsequent drug development through a comprehensive analysis of phase I oncology studies from 2003 to 2010 and subsequent publications citing the original trials.

Methods

We conducted a search to identify and examine publications of phase I oncology studies including the use of biopsy-derived pharmacodynamic biomarkers between 2003 and 2010. Characteristics of those studies were extracted and analyzed, along with outcomes from the biomarker data. We then compiled and reviewed publications of subsequent phase II and III trials citing the original phase I biomarker studies to determine the impact on drug development.

Results

We identified 4,840 phase I oncology publications between 2003 and 2010. Seventy-two studies included a biopsy-derived pharmacodynamic biomarker. The proportion of biomarker studies including nondiagnostic biopsies increased over time (P = .002). A minimum of 1,873 tumor biopsies were documented in the 72 studies, 12 of which reported a statistically significant biomarker result. Thirty-three percent of studies (n = 24) were referenced by subsequent publications specifically with regard to the biomarkers. Only five positive biomarker studies were cited subsequently, and maximum tolerated dose was used for subsequent drug development in all cases.

Conclusion

Despite their increased use, the impact of biopsy-derived pharmacodynamic biomarkers in phase I oncology studies on subsequent drug development remains uncertain. No impact on subsequent dose or schedule was demonstrated. This issue requires further evaluation, given the risk and cost of such studies.

INTRODUCTION

The era of molecularly targeted antineoplastic therapies has been accompanied by a corresponding surge in phase I clinical trials.¹ Many of these trials include nondiagnostic biopsies intended to elucidate biologic effects in response to pharmacologic interventions. The arguments supporting such studies are that they yield proof of mechanism and define the optimal biologic dose. There has been optimism that using biopsy-derived pharmacodynamic biomarkers for these purposes will accelerate drug development while minimizing the risk of toxicities.^2,3 As a result, the proportion of phase I studies with biomarkers has increased steadily from 1990 to 2007.^4,5 The aforementioned benefits have been cited as driving factors for the increased use of biopsy-derived biomarkers in early-phase clinical trials.

Despite the increased use of biomarkers, a paucity of data demonstrating a beneficial impact on drug development exists. Phase I pharmacodynamic studies rarely inform subsequent phase II dosing, which remains driven by toxicity, that is, maximum tolerated dose (MTD). Only 13% of phase I studies with biomarkers from 1991 to 2002 reported pharmacodynamic data that affected the phase II drug dose.⁵ Moreover, the proportion of phase II trials shaped by phase I pharmacodynamic data may also be inflated by publication bias. A recent study on early-phase clinical trials with biopsy-derived pharmacodynamic biomarkers revealed that 22% were never published after abstract presentation.⁶ Of those that were published, pharmacodynamic data were frequently absent (23%) or incomplete (30%).

Nondiagnostic tissue collection is associated with significant costs and potentially deters patients from enrollment in early-phase clinical trials. The per-patient cost for an additional biomarker endpoint in a trial was estimated to be $6,675 in 2004.⁷ In addition, 36% of cancer patients in one survey stated that mandatory research-related biopsies would deter them from enrolling in a clinical trial.⁸ In the same study, 42% of patients believed that a nondiagnostic biopsy would influence their health and care, even when they were clearly informed that it was for research purposes only. These data indicate that cancer patients often misinterpret the risk-benefit profile of biopsies in early-phase clinical trials.

Given these concerns, it is critical to understand with precision the potential benefits of such studies. Thus, we aimed to ascertain the impact of biopsy-derived pharmacodynamic biomarkers on subsequent drug development. We systematically identified phase I oncology clinical trials that used cancer biomarkers published from 2003 to 2010. We then compiled and analyzed subsequent phase II and III studies citing those trials to quantify whether the results of these studies informed subsequent drug development.

METHODS

We identified published phase I oncology studies that involved the procurement of an additional invasive, nondiagnostic, post-treatment tumor biopsy for pharmacodynamic biomarkers over an 8-year period between 2003 and 2010.

Study Design

Publications of phase I clinical trials in oncology that incorporated a biomarker were identified by searching PubMed using a broad search algorithm. As an initial screen, a query was conducted using the National Library of Medicine Medical Subject Headings terms antineoplastic agent; clinical trial, phase I; and neoplasms, as well as the text term oncology. Studies incorporating a post-treatment biopsy were identified through manual review. Results were limited to phase I clinical trials over an 8-year period, from 2003 to 2010. The 2010 cutoff allowed adequate time for publication of subsequent trials citing the phase I trial. Duplications were removed, and data were exported for further analysis.

Identification of Biopsy-Derived Biomarker Studies

To extract phase I trials in oncology incorporating post-treatment tumor biopsies, we further refined the data set by conducting a text search of full abstracts. We compiled studies that included the terms biomarker, pharmacodynamic, surrogate, or biopsy. The target population for this analysis was adult patients with solid tumors receiving oral or intravenous systemic antineoplastic therapies. Therefore, articles covering pediatric populations, hematologic malignancies, topical/local therapies, gene therapy, immunotherapy, or radiation therapy were excluded. Articles on phase 0 trials were also excluded, as were review articles and editorial publications. The remaining articles were downloaded, and one of the authors (R.F.S) manually reviewed each full text for the presence or absence of an additional nondiagnostic post-treatment invasive tumor biopsy. Studies with urine, peripheral blood, imaging, or normal tissue biopsies were collected but were not further analyzed for this study. Studies including post-treatment biopsies for pharmacodynamic biomarkers were then reviewed by at least two of the three authors.

Data Abstraction

We developed an article abstraction form that included three broad domains: study characteristics, biomarker features and results, and citing article data. The first domain included descriptive characteristics of each study, including article year (time of indexing), funding source, study location, and drug category (cytotoxic chemotherapy, small molecule, biologic, or natural product/extract).

Next, we abstracted biomarker features and results. We noted the number of different biomarker modalities, such as peripheral blood draw, tumor biopsy, normal tissue biopsy, urine collection, or imaging. We then focused on tumor biopsies and assessed several performance metrics. Biomarker test reproducibility was the first characteristic assessed. Consideration of interpatient or intrapatient variability, standard error, population-based validation, test duplication for reproducibility, or quality control was considered positive. If a reference to a citation with relevant information on the assay was provided, that article was then reviewed and assessed for the same measures. The use of appropriate positive and negative controls alone was not considered sufficient to account for reproducibility.

The second characteristic analyzed was the presence or absence of a prespecified statistical plan. Description of a power analysis or comment on the expected number of patients necessary to detect a significant result in the context of a biomarker was considered positive. A prespecified plan regarding traditional phase 1 endpoints (toxicity or efficacy) without consideration of the biomarker was not considered positive. A description of post hoc statistics used to characterize groups alone was not considered sufficient. Next, we noted the number of laboratory tests per biopsy sample and determined whether multiple comparison testing was addressed in the Methods section, such as the application of a Bonferroni correction or the use of a more stringent P value. The results and conclusions were then reviewed for reporting of a statistically significant result. If present, we noted whether a correction for multiple comparisons was mentioned. Finally, we assessed whether the article included a conclusion regarding the biomarker and whether the authors suggested that the biomarker data would affect future studies.

Lastly, we used PubMed and Google Scholar to compile articles citing the original phase I trials. For each study, the resulting list was filtered to include only phase II or III studies to assess the impact of the biomarker on subsequent drug development. We then noted the total number of studies that referenced each biomarker. Those articles were characterized by the context of the biomarker reference (mechanism of action, recommended drug dose/schedule, or both).

Statistics

Descriptive statistics were used to analyze the studies. We used one-way analysis of variance or Fisher’s exact test, as appropriate, to evaluate the significance of relationships between study characteristics and biomarker-specific outcome. The Cochran-Armitage test was used to assess linear trends over time. All statistical analyses were carried out using JMP software (version 11.1.1; SAS Institute, Cary, NC). Significance was defined as P < .05 for all tests.

RESULTS

Data Sample

We identified 4,840 publications between 2003 and 2010 that met initial screening criteria for phase I oncology trials (Fig 1). Abstract keyword analysis narrowed those results to 584 studies that included biomarkers, which were manually reviewed by one of the authors (R.F.S.). Seventy-two studies included post-treatment, nondiagnostic invasive tumor biopsies and were included in our final analysis (Data Supplement).

Fig 1.

Schema depicting the study design for identification of phase I oncology trials including a biopsy-derived pharmacodynamic biomarker.

Characteristics of Biopsy-Derived Biomarker Studies

The characteristics of the 72 phase I studies that included nondiagnostic biopsies are shown in Table 1. The proportion of biomarker studies that included nondiagnostic biopsies increased over time (P = .002, Fig 2A). The median number of biopsies per study was 16, with a range of 2 to 149 (Fig 2B). A total of 1,873 tumor biopsies were documented within the 72 studies. This total is underestimated, because 14 studies had additional biopsies not explicitly quantified within the text of the publication or supplementary materials. The number of different tests/assays performed per biopsy ranged from one to 21, with a median of three. The majority of studies with biopsy-derived biomarkers (71%, n = 51) also included biomarkers derived from other modalities (peripheral blood draw, normal tissue biopsy, urine collection, or imaging).

Table 1.

Study Characteristics, Performance Metrics, and Citation Data

Study Characteristic	No.	%
	n = 72
Therapy type
Biologic	42	58
Cytotoxic	20	28
Small molecule	10	14
Primary funding source
NIH or national government	33	46
Industry	23	32
Both government and industry	13	18
Not disclosed	3	4
Location
United States	38	53
Multinational	19	26
Non-US, single country	15	21
Performance metrics
Description of test characteristics	16	22
Prespecified statistical plan	0	0
Multiple biomarker tests performed	68	94
Address multiple testing	3	4
Biomarker results
Statistically significant findings	12	17
Address multiple testing	0	0
Conclusion on biomarker	62	86
Impact on future studies reported	20	28
Citations
Phase II or III study cites article	43	60
Biomarker referenced in citing text	24	33
Biomarker impacted
Drug dose/schedule	10	14
Mechanistic knowledge	24	33

Abbreviation: NIH, National Institutes of Health.

Fig 2.

(A) Number of biomarker studies identified each year. The proportion of studies including a biopsy increased over time (P = .002, Cochran-Armitage test for linear trend). (B) Distribution of biopsies per study (n = 1,873). Range was two to 149, with a median of 16 and a mean of 26 biopsies per study. Quantile box plot depicts 10th and 90th percentiles (short hash marks), median with first and third quartiles (box), and mean with confidence intervals (diamond).

Methodology and Impact of Biopsy-Derived Biomarker Studies

Methods of biopsy-based biomarker assays were reviewed for each study and characterized by predefined performance metrics and impact on drug development (Table 1). A minority of studies included a description of biopsy test characteristics, such as reproducibility, assay validation, or standard error. Although the vast majority of studies tested multiple biomarkers, few described a correction for multiple testing in the methods. No study included a prespecified statistical plan or power analysis in reference to hypothesized biomarker outcomes. The vast majority of studies made conclusions regarding the biomarker results, but only a minority suggested that the biomarker results would affect subsequent drug development. Although most studies were cited by at least one subsequent phase II or III publication, the citing studies inconsistently mentioned the prior biomarker results.

Characterization of Studies With a Positive Biomarker Result

A statistically significant biomarker result was reported in 17% of studies (n = 12). These studies were deemed “positive biomarker studies” and were further analyzed by compiling and reviewing the citing phase II and III studies. From 2003 to 2007, the proportion of publications that were considered positive biomarker studies increased from 0% to 37% (P = .006 for trend). Positive biomarker studies were more likely to have assay characteristics defined in the methods (P = .02) and a higher number of biopsies (median, 32 v 15, P = .01). The mean number of tests performed per biopsy was 3.4. No positive biomarker studies included correction for multiple testing. Only five positive biomarker studies were cited by subsequent phase II or phase III trial publications in regard to the biomarker (range, 1-10 citing publications per study).

Final Assessment of Overall Impact

To assess the overall impact of including biopsy-derived biomarkers in phase I oncology trials, we focused on the five positive biomarker studies that were cited by subsequent phase II or III trial publications (Table 2). General impact was first assessed through quantification of citations. Three of the studies (60%) were cited exclusively by a single phase II article published by the same authors. The remaining two studies by Baselga et al and Tabernero et al were cited by unaffiliated investigators and were cited more frequently (cited by three and 10 phase II or III studies, respectively).

Table 2.

Phase II/III Citations of Phase I Positive Biomarker Studies With Derivation of Dosing

Phase I Study PMID	Year	Recommended Dose From Phase I Study	Citations—PMID	Description	Derivation of Dose
17925555	2007	MTD, decitabine 90 mg/m2 + carboplatin AUC 6	24642620*	Negative phase II study of combination with carboplatin in ovarian cancer—reduced efficacy of carboplatin	MTD, decitabine 90 mg/m2 + carboplatin AUC 6
18332469	2008	MTD, everolimus 10 mg/d or 50 mg/wk	23357973	Negative phase II study of everolimus and temozolomide in melanoma	MTD, everolimus 10 mg/d, 5 of 7 days + temozolomide 200 mg/m2/d for 5 days each cycle
			22928480	Negative phase II study of everolimus and bicalutamide in prostate cancer	MTD, everolimus 10 mg/d, 5 of 7 days + bicalutamide 50 mg/d
			19687332	Phase II study of everolimus in breast cancer	MTD, everolimus 10 mg/d v 70 mg/wk
			19306412	Phase II study of everolimus in renal cancer	MTD, everolimus 10 mg/d
			19380449*	Phase II study of neoadjuvant letrozole + everolimus v placebo in breast cancer	MTD, everolimus 10 mg/d
			22824201	Phase III study of everolimus in renal cancer	MTD, everolimus 10 mg/d
			19047305	Negative phase II study in pancreatic cancer	MTD, everolimus 10 mg/d
			20038218	Negative phase II study in mastocytosis	MTD, everolimus 10 mg/d
			20630061	Negative phase II study in pancreatic cancer	MTD from a separate phase I study, everolimus 30 mg + erlotinib 150 mg/d
			23743569*	Negative phase II study in colorectal cancer	MTD, everolimus 10 mg/d and 70 mg/wk
19047122	2008	MTD, rucaparib (AG014699) 12 mg/m2/d with temozolomide 200 mg/m2/d for 5 days every 28 days	23423489*	Phase II study in melanoma	MTD, rucaparib (AG014699) 12 mg/m2/d with temozolomide 200 mg/m2/d for 5 days every 28 days
19553641	2009	MTD, olaparib 400 mg twice per day	21862407*	Phase II in ovarian and triple-negative breast cancer	MTD, olaparib 400 mg twice per day
20805299	2010	MTD, saracatinib 175 mg once per day	23342270	Phase II biomarker-driven study closed prematurely without providing useful information	MTD, saracatinib 175 mg once per day
			23151808	Negative phase II study in melanoma	MTD, saracatinib 175 mg once per day
			21400081	Negative phase II study in gastric cancer	MTD, saracatinib 175 mg once per day

Abbreviations: AUC, area under the curve; MTD, maximum tolerated dose; PMID, PubMed indexed for Medline.

^*Indicates self-citation, defined as at least one overlapping author.

Subsequent publications were then reviewed for context in regard to biomarker reference. In all citing phase II or III publications, the biomarker was considered to have validated the biologic mechanism of action or confirmed the desired targeted molecular effect. Furthermore, in each study, a comment was made referencing biologic activity at the selected dose or schedule of drug. However, MTD was still used for dose selection in all cases, including the two most frequently cited articles.

DISCUSSION

The addition of biopsy-derived biomarkers to phase I oncology trials appears to have low yield in enhancing drug development. Over an 8-year period in which 72 studies were conducted with at least 1,873 nondiagnostic biopsies, only five studies resulted in a statistically significant biomarker result that was cited in subsequent publications. Furthermore, only two studies were cited independently (ie, by other research groups).

Our formal analysis confirms that the use of biomarkers, including those that are biopsy derived, is increasing over time. Only three studies were identified with biopsy-derived biomarkers from 2003, the first year analyzed, compared with 19 from 2010, the final year included in this analysis. Furthermore, the references to biopsy-derived biomarkers only confirmed expected target activation and did not actually affect dose selection or go/no-go decision making. Taken together, these findings call into question the increasing use of such biomarkers in phase I studies without concomitant improvement in the design of such studies.

It is important to highlight our finding that a statistically significant biomarker result was more likely to occur in a study with assay characteristics clearly delineated. This result is logical and appears consistent with prior observations. The clinical application of biomarkers is complex and requires both scientific and technical validation.⁹ When designing such trials in the future, the use of validated assays and carefully designed measures of pharmacodynamic effect are likely to maximize the potentially derived knowledge value. This output is critical as the sole measure that can be weighed against the risks and costs of nondiagnostic invasive biopsies.

The most heavily cited biomarker study was by Tabernero et al and involved the drug everolimus. This publication included a thorough description of assay validation, blinding during assessment, appropriate positive and negative controls, and statistics.¹⁰ A clear demonstration of drug activity was shown through biomarkers in this study. Interestingly, an accompanying editorial by Sleijfer and Wiemer still concluded that “there are too many uncertainties to solely rely on biomarkers for dose selection.”^11(p1577) This point was reiterated in one of the citing articles by Ellard et al, who stated that “maximum-tolerated dose remains an important end point and cannot be supplanted with predictions of effective biologic dose.”^12(p4539) Nonetheless, this phase I study highlighted the appropriate consideration of numerous factors of pharmacodynamic biomarker trial design that have been previously recommended.^2,13 Consideration of such features should be incorporated into guidelines for the design of phase I studies involving biomarkers.

Biomarker studies have been used after a drug has been successfully developed to explore the impact of changes in dose or schedule, as with cetuximab.^14,15 Those studies used paired skin biopsies, not tumor biopsies, exemplifying the complexity of identifying an optimal dose of a relatively nontoxic monoclonal antibody. Alternatively, randomized dose-ranging studies using tumor size as an endpoint can also be used in this context.¹⁶

Given our findings, we recommend that the use of invasive biopsies for the purpose of pharmacodynamics be limited to those clinical trials in which the primary objective is to ascertain the effect of the drug(s) on the biomarker. This may be particularly relevant when there is uncertainty as to whether to proceed to phase II development (ie, a pivotal biomarker study). In this context, one needs to be rigorous in the design, ideally incorporating randomization to accurately measure the effect of the drug treatment on the biomarker. For example, patients can be randomly assigned to either two pretreatment biopsies or one pretreatment and one post-treatment biopsy after the MTD is reached. With this design, any differences can be clearly attributed to the drug. Such pivotal biomarker studies should have appropriate power to detect an effect of pharmacologic significance. Given that the intraindividual variability of the biomarker may be unknown, the target sample size may need to be adjusted once this variability is assessed (ie, in the control arm). Alternatively, patients can be randomly assigned across a range of doses and/or schedules, ideally using a single post-treatment biopsy (if the baseline assay variability is high and the post-treatment result is hypothesized to be highly dependent on the pretreatment value, then two biopsies should be used). Again, confidence in the assayed biomarker is necessary, and the hypothesis should be clearly framed with a statistical design appropriate for the impact of the question, given the potential mortality of post-treatment tumor biopsies.¹⁷

Several limitations to our analysis existed. First, our data only included studies published as full publications in journals indexed through PubMed. Unpublished data could not be quantified or assessed and may have affected our findings. However, this publication bias is more likely to overinflate the positive impact of these studies, because negative results are less likely to be published.^6,18 Second, despite our attempt to identify all phase I oncology trials with biomarkers, there is a possibility that some were not captured by our search algorithm, despite its broad scope. Similarly, our analysis specifically excluded studies involving radiation and hematologic malignancies, which limited the interpretation to phase I studies of solid tumors. Immunotherapy studies were also excluded. Those studies involved vaccines, cytokines, or adoptive cellular therapies, which warrant special consideration, given that the target cellular effect is on the host rather than on the tumor. Given recent advances in immune checkpoint blockade, we did requery our dataset and found four studies involving CTLA-4 or PD-1/PDL-1 targeted therapy. However, none of those studies included tumor biopsy-derived biomarkers; therefore, no relevant data were missed. The analysis ended in 2010, which was intentionally chosen to allow adequate time for subsequent citation by phase II and III studies. These citation data were critical to our final analysis of impact. To gauge the quantity of uncaptured data from recent years, we screened studies from 2014 and found the number of biomarker studies to be only slightly higher than that in 2010, the final year in our analysis (123 v 102, respectively). Thus, it seems that a proportional number of additional studies would be added by including 4 more years. The incremental addition of those studies to the analysis would be of low benefit, given the previously mentioned limitations to identification of citing studies.

Finally, even though dose and schedule do not seem to be affected by biopsy-derived biomarkers, many of the citations discussed confirmation of target engagement. Proof-of-concept could potentially serve as a basis for continued development of a drug class, when one particular drug may not show clinical activity. Our study did not address this point because it is difficult, if not impossible, to quantify the positive impact such knowledge of biologic underpinnings might have on future drug development.

AUTHOR CONTRIBUTIONS

Conception and design: Randy F. Sweis, Mark J. Ratain

Collection and assembly of data: All authors

Data analysis and interpretation: Randy F. Sweis, Mark J. Ratain

Manuscript writing: All authors

Final approval of manuscript: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Analysis of Impact of Post-Treatment Biopsies in Phase I Clinical Trials

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or jco.ascopubs.org/site/ifc.

Randy F. Sweis

No relationship to disclose

Michael W. Drazer

No relationship to disclose

Mark J. Ratain

Stock or other ownership: Biscayne Pharmaceuticals, Venaxis

Consulting or advisory role: AbbVie, Biscayne Pharmaceuticals, Cyclacel, Genentech/Roche, Shionogi, Biomarin, Drais Pharmaceuticals

Research funding: Dicerna (Inst)

Patents, royalties, other intellectual property: Royalties related to UGT1A1 genotyping for irinotecan (Self, Inst) and pending patent related to a genomic prescribing system (Self, Inst)

Expert testimony: Multiple generic drug companies