Impact of industry collaboration on randomized controlled trials in oncology

Abstract

Background

Industry funders can simply provide money or collaborate in trial design, analysis, or reporting of clinical trials. Our aim was to assess the impact of industry collaboration on trial methodology and results of randomized controlled trials (RCT).

Methods

We searched PubMed for oncology RCTs published May 2013 to December 2015 in peer-reviewed journals with impact factor > 5 requiring reporting of funder role. Two authors extracted methodologic (primary endpoint; blinding of the patient, clinician, and outcomes assessor; and analysis) and outcome data. We used descriptive statistics and two-sided Fisher exact tests to compare characteristics of trials with collaboration, with industry funding only, and without industry funding.

Results

We included 224 trials. Compared to those without industry funding, trials with collaboration used more placebo control (RR 3·59, 95% CI [1·88–6·83], p<·0001), intention-to-treat analysis (RR 1·32, 95% CI [1·04–1·67], p=·02), and blinding of patients (RR 3·05, 95% CI [1·71–5·44], p<·0001), clinicians (RR 3·36, 95% CI [1·83–6·16], p=<·001), and outcomes assessors (RR 3·03, 95% CI [1·57–5·83], p=·0002). They did not differ in use of overall survival as a primary endpoint (RR 1·27 95% CI [0·72–2·24]) and were similarly likely to report positive results (RR 1·11 95% CI [0·85–1·46], p=0·45). Studies with funding only did not differ from those without funding.

Conclusions

Oncology RCTs with industry collaboration were more likely to use some high-quality methods than those without industry funding, with similar rates of positive results. Our findings suggest that collaboration is not associated with trial outcomes and that mandatory disclosure of funder roles may mitigate bias.

Introduction

Involvement of the pharmaceutical and device industries in clinical research may lead to bias in the evidence base.¹ Industry sponsorship has been associated with practices used to distort evidence² and with positive clinical trial outcomes,^3–5 although methodological rigor has been shown to be similar in published funded and unfunded studies.⁶ When industry sponsors trials, the nature of their involvement varies; companies can simply provide funding or may be involved in study design, data interpretation, or manuscript preparation. These different degrees of involvement may have different effects. A recent study sought to differentiate the impacts of funding alone versus collaboration (defined as participation in study design, analysis, or reporting) on randomized controlled trials (RCTs) across medical specialties, and found that industry collaboration was associated with a higher likelihood of reporting a positive primary outcome (i.e. in favor of the study drug) compared no industry involvement.⁷ Industry funding alone without collaboration was not associated with a positive primary outcome.

Industry plays a particularly important role in funding and conducting clinical trials in oncology and is critical to the continued development of new therapeutics. The role of industry in oncology trials has expanded^8–10 and in 2011 industry funded over half of oncology clinical trials.¹¹ There has been concern about bias related to industry involvement in oncology trials from American Society of Clinical Oncology (ASCO) and others,^12,13 and ASCO has called for clinical trials to focus on overall survival (OS) as the most clinically meaningful outcome.^14,15 In addition, there have been broad calls for increased transparency,¹⁶ and many journals now require authors of clinical trial reports to disclose the role of the funding source.¹⁷

Several studies have examined the relationship of industry funding to positive clinical trial outcomes specifically in oncology, with mixed results.^9,18 Further, funded trials have similar quality of study design⁶ and perhaps higher rates of appropriate blinding¹⁸ compared to unfunded trials. However, studies have not differentiated trials with industry funding alone from funded trials in which industry collaborated in the design, analysis, or reporting.

Given the importance of industry in the development of new cancer therapies and the potential different impact of industry collaboration in clinical trials versus simple funding, we set out to determine the specific impact of industry collaboration on the design and results of oncology RCTs. We hypothesized that collaboration would be associated with a higher rate of positive outcomes, similar quality of study methodology, and similar use of the outcome of OS, compared with no industry involvement. We further hypothesized that industry funding alone would not be associated with positive trial outcomes compared with no industry involvement.

Methods

Journal and Study Selection

We searched Web of Science for journals that publish in oncology-relevant categories (e.g. Immunology, Hematology) and selected journals with 5-year impact factor greater than 5 and a requirement that authors report the role of the funder. We conducted a power calculation based on previous findings^5,7 and estimated that 250 studies were required to provide 80% power to detect differences at a significance level of 0·05 between studies with no industry funding and studies with industry collaboration. We searched PubMed for oncology RCTs published in the selected journals. The search strategy is shown in Appendix Figure 1 in the Supplement.

We included all RCTs evaluating drugs or devices in patients diagnosed with cancer. We excluded studies with unclear industry collaboration, preventative trials, surgical trials, behavior trials, trials comparing dosing regimens, post-hoc analyses, unplanned interim or follow-up analyses, follow-up studies evaluating secondary endpoints of the original trial, and single-arm studies. Beginning with the most recent articles (published December 3, 2015), we reviewed articles for inclusion in reverse chronological order by publication date until including an adequate number based on our power calculation. The oldest included articles were published in May 2013.

Data Abstraction

All articles underwent primary review by one of two authors (A.L., A.Y.) and an independent secondary review by the other. During primary review we extracted basic study demographics and information on industry involvement (no funding, funding only, collaboration), defining collaboration as industry involvement in study design, data analysis, or reporting of results. In addition, we recorded information on the primary study outcome and aspects of study design that could impact bias including blinding (of patients, clinicians, and outcomes assessors), type of control (placebo, active, or both), trial design (superiority, non-inferiority, neither, safety or efficacy), and analysis method (intention-to-treat, modified intention-to-treat, neither/per-protocol). We defined outcomes assessors as those who determine whether the patient experienced a primary or secondary outcome. If blinding of outcomes assessors was not specifically described, we assumed assessors were not blinded. We defined studies as using modified intention-to-treat (ITT) when all patients were analyzed in the group to which they were randomized but there were randomized patients excluded from the analysis.^19,20 We defined positive results as those favoring the study drug or device.

We extracted key characteristics on secondary review, including industry involvement status, blinding, analysis method, and primary result. Any disagreements between the primary and secondary reviewer (n=23) were resolved by a third author (D.K.).

Study methodologic quality

We evaluated the quality of study methodology and primary outcomes. We assessed study methodology in 2 domains. We considered blinding of the patient, clinician, and outcome assessor and ITT analysis as high quality features²¹, with modified ITT and per-protocol (as-treated) analyses considered lower quality.²² We defined the primary outcome of OS as a high value outcome based on the ASCO High Value Framework¹⁵.

Statistical Analysis

Study characteristics are summarized using descriptive statistics. We compared trial characteristics based on funding and collaboration using Fisher exact tests. All tests were two-sided and p-values < 0·05 were considered statistically significant. All analyses were conducted using R version 3·1·1.

Results

We screened 947 articles for inclusion in the study and included 224 from 14 high-impact journals (see Appendix Figure 2 in the Supplement). Most trials assessed a drug (n=221, 99%) and were Phase III trials (n=143, 64%); many were published in Lancet Oncology (n=138; 62%) (Table 1). Seventy-one (32%) had no industry funding, 41 (18%) had industry funding alone, and 112 (50%) had industry collaboration (Table 2). Compared to those without funding, trials with industry collaboration were more likely to use a placebo control (RR 3·59, 95% CI [1·88–6·83], p<·0001), use an ITT analysis method (RR 1·32, 95% CI [1·04–1·67], p=·02), and blind the patient (RR 3·05, 95% CI [1·71–5·44], p<·0001), the clinician (RR 3·36, 95% CI [1·83–6·16], p=<·001), and outcomes assessors (RR 3·03, 95% CI [1·57–5·83], p<·001). Compared to trials without funding, trials with industry collaboration were similarly likely to use OS as a primary outcome (RR 1·27, 95% CI [0·72–2·24]) and to report a positive result (RR 1·11, 95% CI [0·85–1·46]). There was no difference between unfunded trials those with funding alone (Table 2).

Table 1.

Characteristics of included studies (n=222)

	No. (%)
Year of Publication
2015	92 (41)
2014	81 (37)
2013	49 (22)
Object of Study
Drug	219 (99)
Device	3 (1)
Phase
II	54 (24)
III	143 (64)
II / III	1 (0.5)
Not applicablea	23 (10)
Not specifiedb	1 (0.5)
Journal
Lancet Oncology	138 (62)
European Journal of   Cancer	33 (15)
European Urology	16 (7)
Lancet	13 (6)
Journal of the American   Medical Association   (JAMA)	7 (3)
JAMA Oncology	4 (2)
American Journal of   Gastroenterology	3 (1)
Other	8 (4)
Study design
Superiority	198 (89)
Noninferiority	18 (8)
Neither	2 (1)
Safety / Efficacy	4 (2)
Analysis design
Intention-to-treat	140 (63)
Modified intention-to-treat	77 (35)
Neither / per protocol	5 (2)
Placebo Control	66 (30)
Primary endpoint
Overall survival	52 (23)
PFS / DFS	94 (42)
Objective response	17 (8)
Other	39 (18)
Composite	20 (9)
Patients blinded	70 (32)
Clinicians blinded	69 (31)
Outcomes assessors blinded	63 (28)
Primary result
Positive	118 (53)
Negative	97 (44)
Mixed	7 (3)

^aRefers to trials of approved drugs or devices, or follow-up studies

^bRefers to preliminary trials of novel compounds for therapy

PFS=progression-free survival; DFS=disease-free survival

Table 2.

Industry Involvement Status and Trial Design and Outcome

Trial Variable	No funding (n= 70)		Funding only (n = 41)			Collaboration (n = 111)
	No. (%)	RR (95% CI)	No. (%)	RR (95% CI)	P Value	No. (%)	RR (95% CI)	P Value
	ITT analysis	39 (55·7)	1 (ref)	21 (51·2)	0·92 (0·64–1·32)	0·70	81 (73·0)	1·31 (1·03–1·65)	0·023
Placebo Control used	9 (12·9)	1 (ref)	7 (17·1)	1·33 (0·53–3·30)	0·58	51 (45·9)	3·57 (1·88–6·79)	<0·001
Patient blinded	11 (15·7)	1 (ref)	7 (17·1)	1·09 (0·46–2·58)	0·99	53 (47·7)	3·04 (1·71–5·41)	<0·001
Clinician blinded	10 (14·3)	1 (ref)	7 (17·1)	1·20 (0·49–2·90)	0·79	53 (47·7)	3·34 (1·82–6·13)	<0·001
Outcome assessor blinded	9 (12·9)	1 (ref)	11 (26·8)	2·09 (0·95–4·61)	0·077	43 (38·7)	3·01 (1·57–5·79)	<0·001
Primary endpoint: OS	14 (20·0)	1 (ref)	10 (24·4)	1·22 (0·60–2·49)	0·64	28 (25·2)	1·26 (0·72–2·22)	0·47
Positive result	37 (52·9)	1 (ref)	18 (43·9)	0·83 (0·55–1·25)	0·43	64 (57·7)	1·09 (0·84–1·43)	0·54

ITT=intention-to-treat; OS=Overall survival; PFS=progression-free survival; DFS=disease-free survival

Discussion

We found that half of oncology RCTs in high-impact journals were performed in collaboration with industry, and that these trials were more likely to have high quality methodology than those without funding. Unlike other studies, our study found that the likelihood of finding in favor of the study drug did not differ based on industry collaboration. Trials with industry funding in which industry did not collaborate did not differ from those with no industry funding.

Our finding that oncology RCTs with industry collaboration were more likely to describe rigorous methodology may not be surprising. A 2013 systematic review of the relationship between funder and outcome in cancer clinical trials similarly noted that industry funded studies were more likely than those without funding to have adequate allocation concealment and to describe blinding and patient withdrawals.¹⁸ However, other studies have found that funded trials are more likely than those without industry funding to use a modified ITT compared with an ITT analytic approach,²³ though whether this results in bias is controversial.^19,24 While prior authors did not differentiate studies with industry collaboration from those with simple funding, we found higher rates of blinding and modified ITT only in collaborative studies; there was no difference between studies with industry funding alone and unfunded trials. This may be because studies with industry collaboration are designed strategically with regulatory processes for drug approval in mind, or it may reflect the fact that in collaborative studies industry provides the resources necessary to optimize study design. Regardless of the reason, our findings regarding methodology are reassuring and suggest that industry collaborations do not result in sub-optimal RCT methodology. Insofar as some of these methodological elements reduce risk for bias, the publishing community may need to explore non-regulatory levers for encouraging optimal study design when Food and Drug Administration (FDA) approval is not the ultimate goal.

We found that relatively few published RCTs used OS as a primary outcome, with no differences based on industry collaboration or funding. ASCO¹⁴ and others²⁵ have called for the use of OS in oncology trials, noting its importance in assessing value. However, evaluating OS may require a larger sample size than evaluations of disease-free survival or objective response.²⁶ Currently the FDA does not require that studies evaluate OS,²⁶ but moving forward the oncology community should find regulatory and other levers to encourage funders and investigators to evaluate this most important outcome.

We found similar rates of positive results among trials with industry collaboration and unfunded trials. This finding disproved our hypothesis and has several possible explanations. First, it may reflect a general evolution over time with the impact of industry involvement on study results waning in recent years.¹⁸ Most prior studies of the association between funder and outcome reflected older publications^3,4,27 and the single study demonstrating higher rates of positive results among trials with industry collaboration included trials published in 2011– 2012, whereas our study included trials published between 2013 and 2015. A possible shift in recent years may be related to a number of interventions to increase transparency and reduce bias. Since 2005, the International Committee of Medical Journal Editors (ICMJE), on behalf of its participating journals, has required prospective registration of all clinical trials²⁸; this move was designed in part to limit publication bias that may have led to a lower likelihood of publication of negative studies in which industry collaborated or simply funded, and has been broadly supported.^29,30 Although trial registration remains incomplete³¹ and its impact on publication of negative trials has been disappointing,³² among the more recent publications we evaluated the ICMJE policy may be blunting the previously documented effect of industry collaboration on positive results. Our negative finding may also relate to other changes regarding author transparency. Journals have increasingly required disclosures related to potential conflicts of interest including the role of the sponsor in trial execution. These disclosure requirements are designed both to discourage related bias and to allow the reader to detect such bias,³³ though the effectiveness of disclosure has been challenged.^34,35 However, our findings imply that at least among journals requiring disclosure of the role of the funder, to which our study is by nature limited, the need for disclosure may mitigate biases in study design and reporting that could arise from industry involvement. Thus, at least in oncology, disclosure may actually be effective at accomplishing its goal of discouraging bias.

Our findings may also be explained by a different effect of industry collaboration in oncology compared to other clinical areas. Collaborations among researchers, clinicians, and the pharmaceutical industry are particularly important to drug development in oncology,¹² and both research collaborations and payments to individual clinicians are common.^15,36 These relationships are likely relevant in studies with industry collaboration as well as in those that are not industry funded, which may blunt the impact of collaboration itself. However, simple industry funding has been associated with positive results in oncology trials,¹⁸ with a similar effect size as in the literature overall,⁵ so a fundamental difference between oncology and other areas is unlikely.

Our study has important limitations. First, we only included trials published in journals that require authors to report both funding source and the role of the funder. In journals that do no require such disclosure, the author may opt to disclose this information. However, including these select studies from non-requirement journals would have biased our sample. Thus, we only included RCTs from journals that require reporting of the role of the sponsor. For this reason, we were unable to include RCTs from several high impact journals that influence oncology practice (e.g. the New England Journal of Medicine) and our sample was dominated by studies from a small group of journals. Among studies with industry collaboration there may be selective publishing of those with higher-quality methodology and negative results in such journals. Our result, then, may be applicable only to RCTs published in journals that require disclosure of the role of the sponsor. However, even if this is the case it implies that the policy of requiring such disclosure can successfully eliminate any effect of industry collaboration on positive results, implying that broader implementation of these policies at other journals would be an effective intervention to minimize bias in the oncology literature.

Our study has other relevant limitations. We were unable to account for the role of publication bias. While there is evidence that negative studies are less likely to be published both in general³⁷ and specifically among industry-sponsored trials,³⁸ the impact of study funding and industry collaboration on publication bias is not clear. The fact that we found that studies with industry collaboration were not more likely to have a positive result argues against a large role for publication bias. Like studies of the general literature,⁷ we found no difference between trials with industry funding without collaboration and those without funding. However, the number of studies with funding alone was small, so we may have missed small differences. In addition, we chose to restrict our search to journals with a high impact factor. Previous research has shown that positive oncology trials and those using ITT analysis are more likely to be published in journals with high impact factor.³⁹ Our focus on high-impact journals may affect the generalizability of our findings regarding the higher quality of studies with industry collaboration, although it also means that our findings are applicable to RCTs that are most likely to impact care. Finally, because of exclusions during the review process, our study sample included fewer than our target of 250 studies. For this reason we may have been underpowered to detect significant results for studies with industry funding alone; in particular a larger sample size may have allowed us to detect a significantly higher rate of outcome assessor blinding among studies with funding without collaboration, for which the p-value was 0·07. However, our sample allowed us to detect many important differences associated with collaboration; a larger sample is unlikely to have substantially impacted our overall results.

Conclusion

In this cross-sectional study of oncology RCTs in high-impact journals requiring disclosure of the role of the funder, we found that pharmaceutical industry collaboration, but not funding alone, was associated with certain elements of methodological quality but not with results in favor of study drug. These findings are reassuring that industry collaboration in oncology, which is common, seems not to threaten the integrity of the evidence.

Highlights.

• Industry may collaborate in trial design and execution or simply provide funds.

• Trials with industry collaboration differ from those without industry funding.

• Industry collaboration is associated with placebo control and more blinding.

• Industry collaboration is not associated with higher rate of positive results.

• Trials with industry funding only are similar to those without funding.