Prospective clinical trials of biotherapies in solid tumors: a 5-year survey

Abstract

Purpose: To review the content and quality of prospective clinical trials of biotherapies in solid tumors. Methods: Data were collected from the literature between 1990 and 2002 on general study characteristics, patient and disease factors, study methodology, and factors related to completeness of reporting. Quality of phase II studies was evaluated by an ad hoc questionnaire. Descriptive statistics, contingency tables, and the χ-square test were applied. Results: A total of 334 studies were selected, of which about three quarters were multicenter, with 42.5% reporting phase I, 42.2% phase II or I/II, and 11.9% phase III or II/III studies. Only 13.7% were randomized, and a study design emphasizing statistical analysis was lacking in as many as one third. The assessment of biological endpoints was stated as the primary or secondary goal in half of these studies. Melanoma (17.1%), renal carcinoma (11.1%), gastrointestinal neoplasms (11.1%), and lymphomas (6.3%) were the most studied diseases. Immunotherapies accounted for 182 studies; the remaining 152 reported other biotherapies. Patients with (1) advanced disease (P=0.003), (2) heavily pretreated neoplasms (P<0.0001), (3) poor performance status (PS<2) (P<0.0001), were more frequently enrolled in studies of biotherapy. Biotherapies were less frequently evaluated in phase III studies (7/152) compared with immunotherapies (33/182) (P<0.0001). A statistical study design was more frequently identified in biotherapy trials (127/152) compared with immunotherapy trials (98/182) (P<0.0001). Biological endpoints were less frequently evaluated in phase III studies in both biotherapies (100% no vs 0% yes) and immunotherapies (81.8% no vs 18.2% yes) (P=0.01, for biotherapies; P<0.0001, for immunotherapies). Phase I immunotherapy studies more frequently applied biological or molecular criteria for patient selection (41.1%) than phase II (29.3%) and III (3.1%) studies (P<0.0001). Conclusions: The very wide diversity in modalities of conducting and reporting clinical trials of biotherapies of solid tumors and the presence of some methodological pitfalls suggest that the methodological standards for conducting and publishing clinical trials in biotherapies should be improved to enhance the reliability of the body of published data.

Introduction

New compounds with biologically oriented mechanisms of action are enriching the therapeutic possibilities in oncology. Novel agents have been shown in nonclinical models to inhibit tumor growth, invasion, angiogenesis, and metastasis, or to potentiate the immune response to the neoplasms (immunotherapy). Mostly the effects of these new compounds are the inhibition of tumor growth rate and/or the decrease of tumor progression-associated phenomena [1, 2]. The present survey was prompted by the consideration that several factors may affect the clinical evaluation of a biological drug. Several authors have already identified problems regarding the methodology of clinical trials involving these agents [3–10]. Phase II studies deserve particular attention because of the high volume of published phase II studies in the literature, the wide variety of statistical methodology applied [11, 12], and because of their particular importance in evaluating the antitumor effects of biotherapies. Interpretation of published study results in biotherapy can be difficult for clinicians. Factors that could help the oncologists’ evaluation of studies of these agents would include a clear presentation of rationale and preclinical evidence, patient characteristics including prior treatments, valid biological evaluation, statistical design and sample size, and clear description and discussion of endpoints.

We undertook this survey to describe the content and quality of prospective clinical trials of biotherapies in solid tumors. Here we report an extensive review of papers focused on the biotherapy of solid tumors, published between 1998 and 2002 in five specialty journals with impact factor over 3 in the cited period. Three hundred thirty-four papers were selected. Our results showed that (a) most of the papers considered patients with advanced disease and poor performance status, (b) few manuscripts classified patients on a biological and/or molecular profile, (c) few manuscripts reported a biological endpoint, (d) one third of papers did not report a statistical study design with an “a priori” estimate of the sample size.

Methods

Selections of articles

All prospective clinical trials of biotherapy (biologically oriented or target-oriented) in solid tumors (with the exception of hormone therapy for breast and prostate cancer) published between 1998 and 2002 were selected by hand-searching five distinguished (i.e., with an impact factor above 3 during the selected time period) specialty journals. Studies on cytotoxic agents—free or immunoconjugated (immunotoxins, radioimmunotherapy)—were excluded, unless they were combined with biotherapies.

Data source

Manuscripts were the source of our data. Each article was read by two investigators (A.O., G.D.L., M.D.M., C.P., E.B., R.T., M.P.), and a study report form (SRF) was completed. Divergent evaluations reported on SRFs were resolved by consensus discussion, and a database was generated when all authors agreed on the issues.

Collected information and database characteristics

For each article, four classes of information were collected regarding (1) the general characteristics of the studies (journal, year of publication, number of study arms, study phase, type of tumors, type of drug by mechanism of action, statistical design, number of patients enrolled, claimed endpoints, conclusions), (2) patient and disease factors (stage of disease, performance status, selection of patients bearing the molecular target, number and/or type of metastatic sites, number of previous systemic treatments, type of previous chemotherapy, informed consent), (3) study methodology (i.e., phase of the study, use of time-to-event descriptions, use of biological endpoints, statistical study design and number of participating centers), and (4) factors related to completeness of reporting (presence of information on follow-up compliance with treatments such as indication of median or total number of cycles/doses administered compared with the planned ones or the number of patients who discontinued treatments). When no explicit indication of study phase was reported, this information was deduced from the text.

Each record of the database corresponds to a study. Articles describing more than one study (i.e., two different studies in the same report) were treated as different, consecutive records. Multiarm trials were treated as a single record. This choice was based on the consideration that the variables analyzed (patients’ selection criteria, assessment of biological endpoints) were expected to be the same among different arms of each multiarm study.

Preclinical evidence reporting index

To verify the modality and quality of biologic rationale referred by each article we designed a score according to the sum of the impact factors of the journals cited by the authors when discussing the preclinical background (lower <15 and higher ≥15). This score is indicated in the text and tables as the preclinical evidence reporting index (PERI). For the cases in which the authors also reported on their own preclinical evidence, the PERI included the impact factor of the journal in which the article was published.

Quality of phase II studies

For phase II biological studies we designed an ad hoc quality evaluation questionnaire (Table 1), which was based on literature review [2–10, 11]. Each trial was given a score with a maximum of 100 points by two independent reviewers who were not included in the authorship and blinded to the journal and authors’ names.

Table 1.

Criteria list for methodological quality assessment of selected phase II studies of biological therapies of solid tumors

Study population  A. Patient selection   Zero points if patients are not selected on a validated biological/molecular basis; 10 points if they are  B. Trial size   2 points if total number of patients is <30; 6 points if >30 or <50; 10 points if >50

Analysis and measurement of biological effect  C. Study design   Zero points if not reported; 10 points if yes  D. Biological endpoints   Zero points if not reported; 10 points if evaluations of biological endpoints are planned

Modality of response and toxicity reporting  E. Response   Zero points if criteria for response assessment are not reported; 10 points if yes  F. Toxicity   Zero points if criteria for toxicity evaluation are not reported; 10 points if yes

Modality of preclinical evidence and data reporting  G. PERI   Zero points if preclinical evidence is not cited; 2 points if PEI<15; 8 points if PEI>15  H. Sites of disease   Zero points if not reported; 8 points if reported  I. Compliance with treatment   Zero points if not reported; 8 points if reported  J. Time-to-event descriptions   Eight points if overall-survival and/or disease-free survival are reported  K. Follow-up   Eight points if any information about follow-up is reported (duration, lost to follow-up)

Treatments

The articles were divided into two main groups on the basis of the mechanism of action of the drug: (1) biotherapy (inhibitors of the proliferative signals, antiangiogenetic agents, differentiating agents, inhibitors of metastasis/invasiveness, drugs with multiple mechanisms of action) and (2) immunotherapy (vaccines, cytokines, adoptive immunotherapy, antibodies stimulating the immune system, drugs with multiple mechanisms of action). Studies evaluating biotherapy plus chemotherapy were also selected. Studies evaluating biological therapy plus any locoregional treatments (embolization, radiotherapy, surgery) were treated as single biological therapy. This choice was based on the consideration that the systemic effects of the treatment were not related to the locoregional treatment.

Analysis

Consistent with our aim, all the analyses performed in this survey were descriptive. Associations between type of therapy and all the others variables were evaluated by the χ-square test. P values ≤0.05 were considered statistically significant.

Results

General characteristics of the studies

The distribution of the articles during the 5-year period for each type of therapy is indicated in Fig. 1. A total of 334 studies were reviewed; general contents are shown in Table 2. Forty-one studies were published in 1998 (12.3%), 66 in 1999 (19.8%), 65 in 2000 (19.5%), 81 in 2001 (24.3%), and 81 in 2002 (24.3%). Most of the studies (78.4%) were multicentered. One hundred forty-two articles (42.5%) reported phase I studies, 141 (42.2%) phase II or I/II, and 40 (11.9%) phase III or II/III. The study phase was missing in 11 (3.3%) articles. Forty-six studies (13.7%) were randomized. A statistical study design and an “a priori” estimate of the study sample were not reported in 109 articles (32.6%). The assessment of biological effects (inhibition of an enzyme, assessment of the stimulation/inhibition rate of receptors/trasduction signal pathways, analysis of soluble molecules in the serum, any biological evaluation of tumor tissues before and after therapy, etc.) was stated as primary or secondary endpoint in 166 studies (49.7%).

Fig. 1.

The distribution of the articles during the 5-year period for each type of therapy

Table 2.

General characteristics of the studies reviewed (n=334)

Variable	No.	Percentage
Journal, in alphabetical order
Annals of Oncology	26	7.8
British Journal of Cancer	42	12.6
Cancer	30	8.9
Clinical Cancer Research	116	34.7
Journal of Clinical Oncology	120	35.9
Year of publication
1998	41	12.3
1999	66	19.8
2000	65	19.5
2001	81	24.3
2002	81	24.3
Participating centers
Single	72	21.6
Multiple	262	78.4
Study phase
I	142	42.5
II and I/II	141	42.2
III and II/III	40	11.9
Not identifiable	11	3.3
Type of study
Single arm	288	86.3
Randomized	46	13.7
Statistical study design
Not identifiable	109	32.6
Identifiable	225	67.4
Use of biological endpoints
Yes	166	49.7
No	168	50.3
Type of treatment
Monobiotherapy	262	78.4
Polybiotherapy	72	21.6
Association with chemotherapy
Yes	88	26.3
No	246	73.7
Tumors, in alphabetical order
Breast	17	5.0
Central nervous system	4	1.2
Gastrointestinal	37	11.1
Head and neck	9	2.7
Lymphoma	21	6.3
Lung and pleura	16	4.8
Melanoma	57	17.1
Ovarian	8	2.4
Prostate	14	4.2
Renal	37	11.1
Sarcomas	8	2.4
Others	18	5.4
More than one tumors or all solid tumors	88	26.3
Mechanism of actions, in alphabetical order
Biotherapy	152	45.5
Antiangiogenetic drugs	32	9.6
Differentiating agents	19	5.7
Inhibitors of invasiveness	20	6.0
Inhibitors of proliferation	53	15.9
Multiple actions	28	8.4
Immunotherapy	182	54.5
Adoptive cellular immunotherapy	11	3.3
Antibodies (stimulating immunity)	17	5.0
Cytokines	91	27.2
Multiple actions and/or multiple drugs	21	6.3
Vaccines	42	12.6

Two hundred sixty-two studies (78.4%) evaluated monobiotherapy, 72 studies (21.6%) evaluated polybiotherapy. Eighty-eight studies (26.3%) were not disease oriented. Melanoma (17.1%), renal carcinoma (11.1%), gastrointestinal neoplasms (11.1%), and lymphomas (6.3%) were the most studied diseases. The effects of the association with chemotherapy were treated in 88 studies (26.3%).

Modality of conducting studies and reporting results

Studies were divided into two main groups. One hundred eighty-two studies reported the effect of immunotherapies (immunotherapy group, IG), while 152 studies reported the effect of biotherapies (biotherapy group, BG). Characteristics of the studies by treatment modalities are shown in Table 3. In addition, differences between treatment groups were studied with the χ-square test. Patients with (1) advanced disease (P=0.003), (2) heavily pretreated neoplasms (P<0.0001), or (3) poor PS (PS≤2) (P<0.0001) were more frequently enrolled in studies of biotherapy. The selection of patients on a biological basis was adopted more frequently in IG studies (50/182) compared with BG studies (27/152) (P=0.049). Metastatic sites were more frequently reported in immunotherapy studies (97/182 for IG vs 49/152 for BG; P<0.0001). The analysis of study phase by treatment group showed that biotherapies were less frequently evaluated in phase III studies (7/152) compared with immunotherapies (33/182) (P<0.0001). PERI was lower (≤15) in immunotherapy studies compared with biotherapy studies (P=0.002). Time-to-event related outcomes were more frequently reported in immunotherapy trials (74/182 for IG vs 26/152 for BG; PS<0.0001). A statistical study design was more frequently identified in biotherapy trials (127/152) compared with immunotherapy trials (98/182) (P<0.0001).

Table 3.

Characteristics of the studies by type of therapy (biotherapy, 152 studies; immunotherapy, 182 studies)

	Numbers		P (χ2 test)
	Biotherapies	Immunotherapies
Setting of disease
Advanced	143	151	0.003
Nonadvanced	9	31
No. of previous treatments
<3	51	104	<0.0001
≥3	70	44
Not identifiable	31	34
PS of patients
≤1	35	72	<0.0001
≤2	96	72
≤3	10	7
Not reported	11	31
PS scale
WHO/ECOG	115	135	0.624
KPS	32	37
Others	5	10
Selection of patients bearing the molecular target
Yes	27	50	0.049
No	125	132
No. of metastatic sites
Reported	49	97	<0.0001
Not reported	103	85
Type of previous chemotherapy
Yes	20	31	0.408
No	132	151
Informed consent
Yes	141	159	0.149
Not reported	11	23
Phase of the study
Phase I	91	51	<0.0001
Phase II or I/II	52	89
Phase III or II/III	7	33
Not identifiable	2	9
PERI
Lower	66	111	0.002
Higher	86	71
Use of time-to-event descriptions
Yes	51	120	<0.0001
No	101	62
Follow-up
Yes	26	74	<0.0001
No	126	108
Compliance with treatment
Yes	123	138	0.322
No	29	44
Use of biological endpoints
Yes	75	90	0.928
No	77	92
Statistical study design
Not identifiable	25	84	<0.0001
Identifiable	127	98

Quality of phase II studies

Phase II studies presented a very heterogeneous quality index (QI) assessment ranging from 18 to 86 points (median QI of all studies: 53). A QI<50 points was calculated for 37.1% of immunotherapy and 36.5% of biotherapy phase II studies, respectively (Fig. 2). No significant increase of QI scores was registered during the analyzed period (data not shown). A significant association was found between QI of the articles (<50 vs ≥50) and impact factor of the journals (P=0.001).

Fig. 2.

Dot-plot distribution of QI of articles by treatment modality

Distribution of subject inclusion criteria, biological endpoints and time-to-event reporting by study phase

The distribution of variables related to subject inclusion criteria, biological endpoints, and time-to-event reporting was analyzed by study phase to explore possible variations that could occur during the clinical development of these drugs. The results are shown in Table 4. The enrollment of patients with advanced disease was predominant in phase I (90.1% advanced vs 9.9% nonadvanced disease) and phase II immunotherapy studies (88.8% advanced vs 11.2% nonadvanced disease) (P=0.001). The use of time-to-event descriptions was predominantly adopted in phase III studies in both biotherapy (100% yes vs 0% no) (P<0.0001) and immunotherapy (93.9% yes vs 6.1% no) (P<0.0001). Biological endpoints were less frequently evaluated in phase III studies both in biotherapies (100% no vs 0% yes) and in immunotherapies (81.8% no vs 18.2% yes) (P=0.01, for biotherapies; P<0.0001, for immunotherapies). Phase I immunotherapy studies more frequently applied biological or molecular selection criteria of patients (41.1%) than phase II (29.3%) and III (3.1%) studies (P<0.0001).

Table 4.

Variables related to selection criteria, biological assessments, and time-related outcomes by study phase in biotherapy and immunotherapy. Data reported are column percentages. Correlations in bold are statistically significant (see text for P values)

	No.	Biotherapies (phase of the studya)			No.	Immunotherapies (phase of the studyb)
		Phase I (n=91)	Phase II (n=52)	Phase III (n=7)		Phase I (n=51)	Phase II (n=89)	Phase III (n=33)
Setting of disease
Advanced	143	95.6	92.3	100	151	90.1	88.8	63.6
Nonadvanced	9	4.4	7.7	–	31	9.9	11.2	36.4
PS of patients
≤1	35	21.9	26.9	14.9	72	49.1	34.8	39.4
≤2	96	65.9	61.5	42.8	72	29.4	49.4	36.3
≤3	10	6.6	3.8	27.4	7	5.9	3.3	3.0
Not reported	11	5.6	7.8	14.9	31	15.6	12.5	21.3
Use of time-to-event descriptions
Yes	51	9.9	65.3	100	120	27.5	77.6	93.9
No	101	90.1	34.6	–	62	72.5	22.4	6.1
Use of biological endpoints
Yes	76	47.2	59.6	–	90	78.4	43.9	18.2
No	76	52.7	40.4	100	92	21.6	56.1	81.8
Selection of patients bearing the molecular target
Yes	27	18.6	17.4	–	50	41.1	29.3	3.1
No	125	81.3	82.6	100	132	58.9	70.7	96.9

^aThe study phase was not identifiable in two studies

^bThe study phase was not identifiable in nine studies

Discussion

Interest in biotherapies is increasing in clinical cancer research. In fact, in the years taken into consideration, the number of studies published in the selected journals doubled (12.3% in 1998, to 24.3% in 2002). Most studies are multicenter, single-arm, phase I or II and investigate the effects of monobiotherapies. Although predominantly descriptive, some findings reported in this survey merit further discussion.

Disease and patient factors

Most of these studies (294/334, 88%) enrolled patients with advanced disease. We reviewed only 9 (2.7%) biotherapy studies and 31 (9.3%) immunotherapy studies that enrolled patients with either localized disease or who were receiving adjuvant treatment (P=0.003). Nearly half of all patients in the BG studies (70/152, 46%) had been treated with three or more lines of systemic therapies, compared with 78/182 (43%) for IG studies (P<0.0001). The number of studies which adopted an inclusion criteria of PS≤2 were 96/152 (63.1%) for the BG studies and 72/182 (39.6%) for the IG studies (P<0.0001). Furthermore, this data was probably underestimated in the BG, since in the studies in which the performance status or the numbers of previous treatments were not formally reported, the authors stated that the patients had been “heavily pretreated.”

Compared to BG, the IG patients received fewer lines of treatments and had a better performance status, even though advanced disease continued to be predominant. Because the IG studies were performed on a population less stressed by several lines of treatment, thus reducing the probablilty of the presence of more aggressive tumor cell clones, the patient population could hypothetically be considered more responsive than that of the BG. These data confirm that clinical researchers consider biotherapies as the last possible treatment for patients with cancer (“last chance” effect). This despite the fact that the administration of biological treatments after several lines of chemotherapy, in most cases, was not justified by strong evidence of efficacy of chemotherapy or of sequential chemobiotherapy. Further, by administering biotherapy to a subset of heavily pretreated patients with poor performance status and advanced disease, the researcher lacks the necessary (or desired) conditions in line with the rationale of biotherapy in the first place. We recommend instead that studies dealing with biological therapies should include patients with a good PS who have not been heavily pretreated, in order to obtain the best “interaction” with the immune system, to avoid the impact of a more aggressive cell population selected by continuous switch of malignant versus cell clones resistant to any therapy, and to better clarify toxicities without the confounding effect of “end-of-life” morbidities. These changes could also lead to improved treatment compliance and the ability to study the effects of the drug with long-term administration.

Study methodology

Another interesting finding was that only 166 of 334 (49.7%) of the studies assessed any biological endpoints and only 77/334 (23%) studies selected patients on the basis of biological and/or molecular characteristics (expression of receptors, kinases, specific antigens, HLA settings, etc.). We noted a significant statistical difference between the two groups with regards to selection of patients on the basis of bearing a molecular target, with 27/152 (17.8%) for BG studies and 50/182 (27.5%) for IG studies (P=0.049). Moreover, the presence of a molecular target was verified in most cases at the conclusion of the study (or after enrollment) and only subsequently correlated with the efficacy of the treatments. The selection of patients on a biological basis is particularly important for the correct planning of a clinical trial of a target-based or biologically oriented drug, and we recommend that it be done before enrollment of patients. This allows for proper prospective analysis, and avoids retrospective analysis that can produce different results.

Furthermore, a statistical study design was not present in 109/334 (32.6%) of all studies. A statistical study design was not identifiable more frequently in IG (25/152, 16.5%) than BG (84/182, 46.1%) (P<0.0001). Phase II studies lacked a formal statistical plan more frequently than phase I and III in both BG and IG (data not shown). This is a methodological shortcoming that may drammatically affect interpretation of results [11, 12].

Interestingly, the amount and the quality of the preclinical literature cited by different authors regarding the same drug, in the same study phase, were highly variable. PERI was higher in BG than IG (P=0.002). In some cases, there was no discussion of preclinical evidence, evidence that should provide readers with the biological/molecular basis and rationale of the anticancer action. In fact, about 12% of phase I studies and 35% of phase II studies make no mention of preclinical evidence (PERI=0) in any parts of the paper. Some papers dealing with biologically oriented new drugs seemed to have optimistic and unreasonable expectations of efficacy not consistent with the preclinical background. We recommend that authors clearly cite the preclinical rationale for biotherapy studies.

From the above analysis we can speculate that IG studies seem better designed than BG studies with respect to PS of enrolled patients, number of previous treatments, statistical study design, study conduction (follow-up, time-to-event description, etc). This is probably due to the greater experience of physicians involved in this field of research. In fact, more than 56% of all IG studies were dedicated to three specific tumors in which immunotherapy has been studied for several years: melanoma, renal cell, and lymphoma. However, the criticisms remain that there are a high number of IG studies in patients with advanced disease, preclinical data is poorly considered, and biological endpoints are neither frequently nor adequately evaluated.

Quality of phase II studies

In addition to the detailed description of contents of the selected studies, we attempted to estimate and report on the quality of phase II studies. Although the QI assessment should be considered with caution in the absence of a validated measurement tool, we believe that we have raised two important issues. The first is that the phase II studies were quite heterogeneous. The second is that the quality scores did not improve over the 5-year period. Both of these issues could, in part, be due to the lack of guidelines available to the oncologist conducting biotherapy clinical research.

Other recommendations

Lastly, moving from the above analysis we would like to propose some criteria for designing and evaluating biological studies. This checklist should be helpful to clinicians as well as to journal reviewers in improving the planning, conducting, reporting, and the relevance of international trials in clinical practice. To this purpose, we suggest the following quality items for consideration:

1. Was preclinical evidence and rationale of the study adequately and comprehensively described?

2. Were the previous treatments and the clinical characteristics of the patients (performance status, sites of disease) adequately described?

3. Were the enrolled patients selected on a validated biological/molecular basis?

4. Was the clinical trial adequately planned as regards statistical design and sample size?

5. Are all the endpoints adequately described and discussed?

The present survey reviewed the contents of clinical trials of biotherapies in solid tumors as they appeared in distinguished specialty journals over a 5-year period with the goal of providing oncologists with a comprehensive overview on this topic. To the best of our knowledge this is the largest survey ever presented in the field of the anticancer treatment. Although predominantly descriptive, this survey suggests that changes in selection criteria of patients (better PS, biological characteristics), and in modalities of conducting and reporting cancer clinical trials (appropriate endpoints) of biotherapy drugs could improve the reliability of such studies.