Analyzing factors associated with clinical trial publication in radiation oncology

Abstract

Clinical trials are considered the gold standard of clinical research and are sought in the medical literature for the goal of providing quality care. To identify factors associated with successful or unsuccessful publication of clinical trials in radiation oncology, data on trial characteristics were collected from the National Institutes of Health database on clinicaltrials.gov. To assess studies that had adequate time to accrue, trials between 2000 and 2005 were extracted by filtering for “radiation oncology”. Studies were excluded if they were incomplete, observational, Phase 4, or lacked sufficient method descriptions. Included studies underwent independent samples t-tests and Pearson Chi-Square bivariate analyses. 538 studies were candidates for analysis of clinical trial characteristics. United States (US) origin, multi-center sites, government funding, Phase III status, and randomized allocation were factors associated with increased publication rate. The number of study arms, study length, and number of participants were significantly greater in published trials. The review's results demonstrate potential barriers or facilitators to publication, and they suggest that publication status may be influenced by geographic, financial, and temporal characteristics of clinical trials. Understanding trial background factors that may impact publication improve data visibility and clinical advancements for all.

1. Introduction

Of the study types in medical literature, clinical trials are considered the pinnacle of evidence-based medicine. [1] Formative studies across medical specialties since the mid-1990s have confirmed the existence of publication bias and found that likelihood of being published in a medical journal is directly associated with having positive results, using novel therapies, or having results that support primary hypotheses. [2] However, exploration of phenomena or factors influencing publication should extend beyond publication bias. Radiation oncology is a starting point to investigate trial characteristics given that the specialty's trials are less likely to be published in high-impact medical journals than other types of therapeutic interventions in cancer research. [3] A field with lower relative publication rates and heavy reliance on evidence-based medicine would benefit from identification of factors contributing to publication. Findings in the field of radiation oncology may not only shed light on unique ways to combat publication disparities within the specialty, but they may also identify trends that are present among other specialties.

2. Methods and materials

To explore factors associated with successful publication of clinical trials in radiation oncology, we collected data from the National Institutes of Health (NIH) database on clinicaltrials.gov and completed analyses by January 2021. Publication status was determined by the presence or absence of citations below the “More Information” section of each study's NIH page, indicating whether trial data were published. Publication rates were compared by study origin, single or multi-center characterization, funding source, trial phase, randomized or non-randomized allocation, length, number of arms, and number of participants. To assess studies that had adequate time to accrue, trials that were started between January 1, 2000 to December 31, 2005 were extracted with the search term “radiation oncology” given that the average duration of oncology clinical trials is approximately 13 years. [4] Studies were excluded if they were incomplete, observational, Phase 4, or lacked sufficient methodology descriptions. A total of 538 studies underwent statistical analyses with the JMP, version 16 (Fig. 1). Independent samples t-tests were used to compare the means of numerical variables, and bivariate analyses using the Pearson Chi-Square test was used to compare nominal variables. A P < 0.05 was considered statistically significant.

Fig. 1.

Study flow chart.

3. Results

Of 538 total studies, multi-center studies and single-center studies were published at significantly different rates of 56% and 32%, respectively. Government funded studies had a higher rate of publication (54%) than studies funded by industry (27%) or other entities (44%). Rates of publication increased by progression of trial phase from Phase I/II (25%), Phase II and Phase II/IIII (33%), to Phase III (62%). Randomized trials were published at a rate (59%) almost twice that of non-randomized trials (35%). There was a near significant difference between the publication rates of studies based on origin, indicating that US studies were more likely to be published than international studies (P = 0.0506). In published studies, there was a greater frequency of 2 or more study arms, while 67% of unpublished studies had 1 study arm. There was an approximate two-year difference in the lengths of published (7.461, CI 95%, 6.973–7.950) and unpublished studies (5.851, CI 95%, 5.441–6.261). Published studies’ average number of participants (331.360, CI 95%, 234.226–428.4495) was almost three times greater than that of unpublished studies (107.869, CI 95%, 79.350–136.387) (Table 1 and Fig. 2).

Table 1.

Results of statistical analyses.

Characteristics	Published N (%)	Not Published N (%)	P value
Origin
International	82 (39%)	130 (61%)	0.0506
US	154 (53%)	172 (47%)
Sites
Multi-center	152 (56%)	121 (44%)	<0.0001
Single-center	84 (32%)	181 (68%)
Funding
Government	121 (54%)	105 (46%)	<0.0001
Industry	36 (27%)	96 (73%)
Other	79 (44%)	101 (56%)
Trial Phase
Early Phase I & Phase I	1 (100%)	0 (0%)	<0.0001
Phase I/II	18 (25%)	54 (75%)
Phase II & Phase II/III	20 (33%)	41 (67%)
Phase III	82 (62%)	50 (38%)
Allocation
Randomized	116 (59%)	81 (41%)	<0.0001
Non-randomized	120 (35%)	221 (65%)
Study Arms
1	106 (45%)	202 (67%)	<0.0001
2	95 (40%)	90 (30%)
3	18 (8%)	0 (0%)
4	10 (4%)	5 (2%)
5	2 (<1%)	0 (0%)
6	0 (0%)	2 (<1%)
7	4 (2%)	1 (<1%)
8	1 (<1%)	0 (0%)
Average Length of Study in Years	7.462	5.851	<0.0001
Average Number of Participantsa	331.360	107.869	<0.0001

^aOne outlier excluded.

Fig. 2.

Histogram of number of participants. *Four trials excluded due to lack of data.

4. Discussion

Completed late phase trials had higher publication rates, which is expected as these trials are conducted after robust preclinical and early clinical study (Fig. 3). It is reassuring that studies that are likely to influence clinical practice and in extension, impact the health of patients, have robust research methods. While it is expected that completed early phase trials were less likely to be published for this reason, these findings imply that early phase trials may be rejecting hypotheses and demonstrating failed efficacy more often than late phase trials. [5] Separately, the publication and citation of positive and not negative studies leads to an “unnatural selection” in what clinicians and scientists read and ultimately perceive. [6] Harboring this bias stigmatizes negative or unequivocal findings, and an increasingly expensive publication process disincentivizes funding sources to invest in these studies. [7,8] The exposure of only a select group of studies reduces transparency and propagates confirmation bias. It is important to note that all objectively derived results are vital to the scientific literature. Lack of publications entails researchers being unable to learn from “failed” trials, repeating unnecessary experiments that ultimately waste resources and may be harmful to participants.

Fig. 3.

Publication rates by trial phase. Early Phase I/Phase I was not included in this figure as only 1 published study and 0 unpublished studies made up this category.

With regards to funding, radiation oncology is a specialty that heavily relies on technology for the delivery and advancement of its practice. There is substantial pressure to use equipment and resources made available through industry-funded services to generate revenue, making radiation oncology especially susceptible to industry influence and interests. [9] One study found that in 10 high-impact medical journals, systemic therapy trial articles were more likely to have industry funding and successful publication than their local therapy counterparts in oncology research. [3] The findings of our review, however, demonstrate that trials with government or other types of funding were approximately twice as likely to be successfully published than those with industry funding (Fig. 4). The results suggest that despite predispositions to report industry-centric outcomes, government funding is associated with successful publication at a higher percentage. The reason for this trend is unclear, but it is important to note that government funding involves taxpayer money. Therefore, successful trials are the fruits of taxpayer-funded labor, while failed trials may be perceived to have “wasted” taxpayer money. This perception parallels our previous discussion on positivity bias.

Fig. 4.

Publication rates by funding source.

Demographic descriptors of researchers are also presumed contributors to publication status. The number of international trial sites and investigators has been growing for decades, and most United States Food and Drug Administration (FDA) drug and biologic applications rely on international trials for data. However, the FDA inspected less than 1% of international sites while the trials in our analysis were being conducted or presumably completed. [10] This difference in publication frequency based on geographic origin can likely be explained by the FDA's oversight on international and early phase trials. Another review found that a study was more likely to be published if its author were from North America, had an identifiable academic rank, or had a previously established record of academic productivity. Women are also generally underrepresented in academic medicine, as they have lower manuscript submission rates and face more institutional barriers to achieving senior faculty rank in radiation oncology. Academic rank, previous research productivity, and male gender have been shown to be associated with increased rate of publication. [3,11] However, these variables were too difficult to accurately describe during our review. The principal investigators' (PI) positions and genders were never explicitly stated on the NIH database. Individuals navigating the database must, therefore, subjectively decipher from search engines if they have identified the correct PI and their title. This limitation demonstrates how publication bias can be elusive: we know of its existence, yet we encounter obstacles to describing it objectively and comprehensively.

The results of our review holistically demonstrate potentially multifaceted barriers to publication. Less than half of the included clinical trials (43.9%) in radiation oncology from 2000 to 2005 were published. This percentage does not account for trials that did not meet inclusion criteria for our review. Completion status is another factor to consider, as analyses of incomplete trials may have further decreased the percentage of successfully published trials. There can be various reasons for discontinuation of studies, some of which were noted in a review of the literature of head and neck cancers, including committee recommendations, drug toxicities, corporate logistics and strategy changes, and positive results from other studies. [12] Many of these reasons are preventable causes that hinder trial completion across medical specialties. However, identifying causes of discontinuation is incredibly challenging, despite the potential to minimize research waste and poor trial designs.

5. Conclusions

We suspect that publication bias is not the only phenomenon driving disparities in medical literature visibility. In other words, there is further room for exploration and expansion of biases as multifaceted phenomena in scientific research. We believe that it is inadequate and inappropriate to recommend specific interventions to address publication disparities currently, such as registering all clinical trials or blinding journal editors to demographics that are irrelevant to studies. It is not possible develop efforts against publication barriers when substantive efforts have not been made to investigate them. We cannot fix a problem that we do not fully understand. While we acknowledge that there are challenges to addressing publication barriers, we denounce dismissing the elephant in the room that stomps on the transparency of clinical trials. We call for future investigations to delve deeper into and expand beyond the topics we have discussed here. It is a puzzle that cannot assembled en masse, rather piece by piece. There is an immense pool of trials and data that is lost or invisible to the medical community because of the cultural and systemic barriers in academia. Therefore, ascertaining facilitative and inhibitory factors to publication may bring the medical community closer improving clinical advancements and the scientific discovery process for all.

Funding

None.

CRediT authorship contribution statement

Newsha Nikzad: Writing – original draft, Conceptualization. Shraddha M. Dalwadi: Writing – review & editing, Visualization. Michelle S. Ludwig: Writing – review & editing, Supervision, Visualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.