This systematic review assesses the rate of discontinuation or nonpublication of phase 3 and 4 randomized clinical trials involving patients with head and neck cancer.
Key Points
Question
What are the rates of discontinuation or nonpublication of randomized clinical trials (RCTs) about head and neck cancer, and what are the potential factors associated with trial discontinuation and nonpublication?
Findings
In this study of 130 RCTs, 92 trials (70.8%) were completed and 38 (29.2%) were discontinued, with committee recommendations as the most common reason for discontinuation of trials. Of the 130 RCTs, 67 (51.3%) were published and 63 (48.5%) were unpublished, with trials funded by other sources more likely to reach publication than industry-funded RCTs.
Meaning
The rate of discontinuation and nonpublication of RCTs of head and neck cancer appears to be high, which may thwart scientific advancements, dissuade patient participation in trials, and unnecessarily expose patients to unproven and potentially harmful interventions.
Abstract
Importance
Randomized clinical trials (RCTs) play an important role in clinical decision-making, and discontinuation or nonpublication of these trials are causes of great concern. The extent of discontinued or unpublished RCTs about head and neck cancer remains unclear.
Objective
To assess the rate of discontinuation or nonpublication of RCTs involving patients with head and neck cancer. This objective was measured by observing 3 domains: discontinuation of trial, nonpublication of trial data, and feasibility of contacting trial investigators of aforementioned trials.
Evidence Review
For this study, the sample was derived using the ClinicalTrials.gov advanced search feature on March 18, 2019, to locate completed and discontinued RCTs pertaining to head and neck cancer registered before this date. Trials were analyzed to identify reasons for trial discontinuation and publication status of each trial. If publication status or reason for trial discontinuation was not allocated through the systematic search of ClinicalTrials.gov, the corresponding author was emailed to determine publication status.
Findings
After exclusions, 130 RCTs were included. Of these trials, 92 (70.8%) were completed and 38 (29.2%) were discontinued for various reasons. The most common reason for discontinuation of trials was committee recommendations. Of the 130 analyzed trials, 67 (51.5%) were published in a peer-reviewed journal and 63 (48.5%) were unpublished trials. Of the 92 completed trials, 55 (59.8%) were published and 37 (40.2%) remained unpublished 3 or more years after trial completion. Trials funded by other sources (private, nonprofit, or the National Institutes of Health) were more likely to reach publication than industry-funded RCTs (unadjusted odds ratio, 4.3 [95% CI, 1.3-14.0]; adjusted odds ratio, 4.1 [95% CI, 1.2-14.3]).
Conclusions and Relevance
Of RCTs in head and neck cancer, 29.2% were discontinued and 40.2% completed trials never reached publication. The findings suggest that needs exist for RCT guidance of head and neck cancer. The reporting of reasons for trial discontinuation appears to be lacking, and trial publication rates were low. This study is relevant to many physicians and researchers because it identifies potential sources of decreased research productivity and ethics.
Introduction
Head and neck cancers (HNCs) are among the most common cancers worldwide.1 On an annual basis, approximately 65 410 new cases of HNC are diagnosed and nearly 14 620 patients die of HNC in the United States.2,3 In 2010, the total direct medical cost of HNC in the United States was $3.64 billion.4 Because of the high prevalence and financial burden associated with HNC, researchers conducting randomized clinical trials (RCTs) addressing HNC should be mindful of any potential research waste. The estimated prevalence of research waste is 85% of all biomedical research, translated to potential waste of $170 billion of the $200 billion spent on research worldwide.5 Two factors contributing to this financial burden of research waste are studies that never reach publication and studies that are unnecessarily discontinued.
Unpublished research regardless of the nature of study findings hinders scientific advancement, and up to half of RCTs never result in publication.6,7 One contributor to nonpublication is publication bias, defined as a bias toward publishing studies with statistically significant results. Publication bias is a pervasive problem in medical research in general and in otolaryngology research in particular.8 Publication bias represents one of the worst threats to the validity of scientific research, and it may have adverse effects for patients greater than those arising from falsified data.9,10 In addition to the financial burden associated with unnecessary research, nonpublication also presents ethical concerns with regard to investigators and patients. For example, both investigators and patients volunteer their time with the intent to advance medical knowledge and to help other patients with the same disease.11 As a result, nonpublication limits scientific progress and may compromise the health of patients who are exposed to interventions for which efficacy is incompletely understood because the information is not available.
Furthermore, scientific advancement may be thwarted because of discontinued trials. Although it may sometimes be necessary to discontinue a trial for reasons pertaining to trial efficacy, patient safety, or feasibility, RCTs are too often discontinued for preventable reasons.12,13 For example, a 2016 systematic review found that 76% of discontinued RCTs ended prematurely for reasons that could have been prevented.14 The Declaration of Helsinki16 states that ending clinical trials for financial or personal reasons is unethical because such discontinuations may ultimately compromise the safety of participants as well as contribute to research waste.15,17
A study exploring the underlying reasons for discontinued or nonpublication of RCTs would contribute new knowledge regarding the extent and nature of these issues. Our primary objective in the present study was to evaluate the prevalence, characteristics, and publication history of HNC RCTs and to investigate factors that may be associated with nonpublication or discontinuation.
Methods
Objective and Outcome Measures
The primary objective of this systematic review was to assess the rate of discontinuation or nonpublication of RCTs involving patients with HNC with use of a cross-sectional design. Data were collected from ClinicalTrials.gov and published trial reports.
Locating Clinical Trials on HNC
We performed a focused search of registered trials related to HNC by using ClinicalTrials.gov, an online clinical trial registry provided by the US National Library of Medicine. All US investigators are required to register their trial before commencement and to provide detailed updates as the trial progresses. A unique national clinical trial number is assigned to each trial entry, and the data collected include the recruitment status, study design, intervention, participants, funding, and other details pertaining to each trial. We used the advanced search function on ClinicalTrials.gov and deployed the search with the following keywords: head and neck cancer, neoplasm of head and neck, carcinoma of head and neck, and cancer of the head and neck. Keywords were chosen for their high sensitivity and the pragmatic likelihood of locating most registered trials related to HNC. Furthermore, ClinicalTrials.gov contains an automated term-mapping capability, which added several related terms to our search. Our initial search was conducted on March 18, 2019, and included all RCTs registered before this date on ClinicalTrials.gov (ClinicalTrials.gov began registering trials in 2000). Search returns from ClinicalTrials.gov were then downloaded and formatted into Microsoft Excel, version 16.30 (Microsoft Corporation) for analysis.
Study Inclusion and Exclusion Criteria
Our search was designed to specifically locate trials that were completed or discontinued. This strategy only included trials with an updated trial status (completed, suspended, terminated, withdrawn, or unknown). All studies with the status of completed formed the completed group, and all studies with the status of terminated, withdrawn, unknown, or suspended formed the discontinued group (eTable 1 in the Supplement). We excluded trials with a status of active, not recruiting, or enrolling by invitation. One of us (A.L.J.) screened the search results by registered title, condition, study design, and completion date. Studies were excluded if they were found not to be relevant to HNC, were not a phase 3 or 4 clinical trial, were not an RCT, or were completed after March 1, 2016. We chose phase 3 and 4 RCTs because they assess long-term outcomes in diverse patient populations with the intent of subsequent publication, and they constitute the highest level of evidence, which likely affects clinical decision-making.18,19,20 Trials that were combined phase 2 and 3 were also included in the sample. We excluded phase 1 and 2 trials because these are typically performed for safety and precautionary purposes and publication is less likely. The exclusion date of March 1, 2016, was chosen to allow the investigators 36 months from the date of trial completion for publication.21,22 Because HNC can occur in any patient population, no limitation was enforced for patient age or demographics.
Locating Publications of Completed Trials
Two of us (I.F. and J.M.A.) reviewed ClinicalTrials.gov to identify the trial publication status of each included trial. If no publication was linked to the ClinicalTrials.gov listing, 2 of us (I.F. and J.M.A.) then searched MEDLINE via PubMed, Embase, and Google Scholar for trial title, authors, or national clinical trial numbers to identify any publication. We allowed the included trials a minimum of 36 months from the latest completion date (March 1, 2016) to the time of our search (March 18, 2019) before considering their publication status. The definition of publication used in this study was an available release of trial results in a peer-reviewed journal in the form of a complete manuscript.
Contacting Authors When No Publication Could Be Located
If no publication was located using our systematic search, we attempted to find a contact email address for the corresponding author. To obtain contact information for the corresponding author, we searched ClinicalTrials.gov, the institution website of the corresponding author, Google, and PubMed publications listing the trial's corresponding author. If an email address was found, a standardized email was sent with several prespecified responses addressing the reason for nonpublication (eTable 2 in the Supplement). Our standardized response for nonpublication contact was based on a systematic review by Song et al.23 We repeated this email strategy once a week for 3 weeks. If no response was received within 8 weeks or the email was returned as inactive, we deemed the author to be noncontactable. This method of email contact to obtain information about relevant trials has been deployed with success in similar studies.21,22 If both the search process and email contact did not provide evidence of publication, we considered the trial to be unpublished.
Statistical Analysis
Summary statistics (frequencies and proportions) were calculated using Google Sheets (Google LLC). The association of trial characteristic variables with discontinuation and publication status using unadjusted and adjusted odds ratio binary logistic regression were calculated using Stata, version 15.1 (StataCorp). We specified which variables to include in the logistic regression model a priori. Our model included trial status (0 indicated discontinued and 1, completed) or publication status (0 indicated unpublished and 1, published) as criterion variables and source of funding, intervention, and sample size (continuous) as variables.
Results
Study Characteristics
The initial search yielded 1356 returns related to HNC, with 869 completed and 487 discontinued trials. Of the initial 1356 trials, our final analysis included 130 RCTs after exclusions. Fifty-three trials were excluded owing to completion after March 1, 2016, and 1173 were excluded because they were not phase 3 or 4 trials (Figure 1). Our sample included 15 combined phase 2 and 3 trials, 93 phase 3 trials, and 22 phase 4 trials. The included trials most frequently investigated pharmacologic (chemotherapy and nonchemotherapy) interventions for HNC (50 of 130 trials [38.5%]). Most RCTs in the sample (41 of 130 trials [31.5%]) were funded by a combination of industry, hospital or university, the National Institutes of Health, nonprofit, or private trial sponsors. Table 1 provides further details regarding trial characteristics. The trial registration dates ranged from February 25, 1991, to January 13, 2016. Six continents were represented with regard to the primary site or affiliation listed by the principal investigator, and the median number of study enrollments or participants was 130 patients (interquartile range, 51.5-325.3 patients; range, 0-1993 patients). There were 47 RCTs (36.2%) with a sample size less than 100, 68 (52.3%) with a sample size at least 100, and 15 (11.5%) that did not report a sample size.
Figure 1. Search Strategy for Locating Included Trials.
Table 1. Characteristics of Completed vs Discontinued Trials and Published vs Unpublished Trials.
| Characteristic | No. (%) [95% CI] | ||||
|---|---|---|---|---|---|
| Completed (n = 92) | Discontinued (n = 38) | Published (n = 67) | Unpublished (n = 63) | Total (n = 130) | |
| Intervention | |||||
| Behavioral or dietary | 14 (15.2) [7.9-22.6] | 4 (10.5) [0.8-20.3] | 8 (11.9) [4.2-19.7] | 10 (15.9) [6.8-24.9] | 18 (13.8) [7.9-19.8] |
| Device or procedure | 12 (13.0) [6.2-19.9] | 7 (18.4) [6.1-30.7] | 9 (13.4) [5.3-21.6] | 10 (15.9) [6.8-24.9] | 19 (14.6) [8.5-20.7] |
| Pharmacologic | 38 (41.3) [31.2-51.4] | 12 (31.6) [16.8-46.4] | 25 (37.3) [25.7-48.9] | 25 (39.7) [27.6-51.8] | 50 (38.5) [30.1-46.8] |
| Pharmacologic and radiotherapy | 24 (26.1) [17.1-35.1] | 12 (31.6) [16.8-46.4] | 21 (31.3) [20.2-42.5] | 15 (23.8) [13.3-34.3] | 36 (27.7) [20.0-35.4] |
| Radiotherapy | 4 (4.3) [0.2-8.5] | 3 (7.9) [0-16.5] | 4 (6.0) [0.3-11.6] | 3 (4.8) [0-10.0] | 7 (5.4) [1.5-9.3] |
| Funding source | |||||
| Industry | 24 (26.1) [17.1-35.1] | 10 (26.3) [12.3-40.3] | 12 (17.9) [8.7-27.1] | 22 (34.9) [23.1-46.7] | 34 (26.2) [18.6-33.7] |
| Mixed | 30 (32.6) [23.0-42.2] | 11 (28.9) [14.5-43.4] | 25 (37.3) [25.7-48.9] | 16 (25.4) [14.6-36.1] | 41 (31.5) [23.6-39.5] |
| Hospital or university | 22 (23.9) [15.2-32.6] | 13 (34.2) [19.1-49.3] | 16 (23.9) [13.7-34.1] | 19 (30.2) [18.8-42.5] | 35 (26.9) [19.3-34.5] |
| Othera | 16 (17.4) [9.7-25.1] | 4 (10.5) [0.77-20.3] | 14 (20.9) [11.1-30.6] | 6 (9.5) [2.3-16.8] | 20 (15.4) [9.2-21.6] |
Other: private (n = 10), nonprofit (n = 8), and the National Institutes of Health (n = 2). Trials did not report participant enrollment, or the report was null.
Discontinuation
Of the 130 included trials, 92 (70.8%) were completed and 38 (29.2%) were discontinued. Of the 92 completed trials, 29 (31.5%) had results posted on ClinicalTrials.gov. Of the 38 discontinued trials, 5731 patients were recruited, which represents 19.8% (5731 of 28 956) of the total study participants. Of the 38 discontinued trials, 18 (47.4%) were terminated, 5 (13.2%) were withdrawn, and 15 (39.5%) were suspended. Before email responses, 23 of 38 discontinued trials (60.5%) did not provide reasons for trial discontinuation on ClinicalTrials.gov. Contact emails were identified for 9 of 23 discontinued trials (39.1%) that did not list a reason for trial discontinuation, and responses were received for 4 of 9 emails (44.4%). Final analysis after email responses identified 19 of 38 trials (50.0%) providing a reason for trial discontinuation (Figure 2). Reasons identified for discontinuation of trials provided from all sources (8 of 19 trials [42.1%]) included committee recommendations (such as corporate reasons unrelated to safety and efficacy, changes in company strategy, toxic effects associated with study drug, positive preliminary results from other studies, elimination of head and neck committee for the studies, or because of recommendation from the data monitoring committee), withdrawn funding or support from the trial sponsor (5 of 19 [26.3%]), low accrual or insufficient recruitment (5 of 19 [26.3%]), or changes in the squamous cell carcinoma cause of HNC (1 of 19 [5.3%]). In addition to providing reasons for trial discontinuation, email responses identified 4 discontinued trials that were published but not recorded on ClinicalTrials.gov or found during a literature search. Of the 38 discontinued trials, 6 (15.8%) were combined phase 2 and 3 trials, 28 (73.7%) were phase 3 trials, and 4 (10.5%) were phase 4 trials. The registration years of discontinued trials ranged from 2003 to 2014, with no year having a considerably higher percentage of discontinued trials. The median number of study enrollment or participants in discontinued RCTs was 56 patients (interquartile range, 5.3-148.0 patients, range, 0-1200 patients).
Figure 2. Trial Discontinuation Flowchart.
A total of 19 reasons for discontinuation were found.
Nonpublication of Completed Studies
Our initial search identified 41 unpublished completed trials, for which 14 email addresses (34.1%) were identified and the researcher contacted, leaving the remaining 27 (65.9%) contact emails unknown. Nine of 14 (64.3%) emails led to responses to our query during the 8 weeks allotted for replies. Email responses identified 4 completed trials that were published but not recorded on ClinicalTrials.gov or found during a literature search.
Overall Publication Status of Completed and Discontinued Studies
In our final analysis of the 130 RCTs through hand-searching trial registries, MEDLINE, and email response, we found that 67 (51.5%) were published in a peer-reviewed journal and 63 (48.5%) were unpublished trials. Nine of 63 unpublished trials (14.3%) were combined phase 2 and 3 trials, 42 (66.7%) were phase 3 trials, and 12 (19.0%) were phase 4 trials. Of 38 discontinued trials, 12 (31.6%) reached publication. In addition, 55 of 92 completed trials (59.8%) were published, and 37 (40.2%) remained unpublished after trial completion (Figure 3) (eFigure in the Supplement). The registration years of unpublished trials ranged from 2003 to 2016, with no year having a considerably higher percentage of unpublished trials. Of the 63 unpublished trials, 58 (92.1%) included a location where the trial was conducted, with 35 of 58 trials (60.3%) having been conducted internationally and 23 (39.7%) in the United States. Table 1 shows trial characteristics according to publication status.
Figure 3. Trial Nonpublication Flowchart.
A total of 67 publications were identified, including 55 completed trials and 12 discontnued trials.
A total of 28 956 study participants were recruited during the included 130 trials, with 11 684 participants (40.4%) in trials that did not reach publication (calculated from actual or target registration data in ClinicalTrials.gov). Trials funded by other sponsors (private [n = 10], nonprofit [n = 8], or the National Institutes of Health [n = 2]) were more likely to reach publication than industry-funded RCTs (unadjusted odds ratio, 4.3 [95% CI, 1.3-14.0]; adjusted odds ratio, 4.1 [95% CI, 1.2-14.3]). Further univariate or multivariate logistic regression did not reveal any additional associations with discontinued or unpublished RCTs and intervention type, funding source, or sample size (Table 2).
Table 2. Logistic Regression of Trial Discontinuation and Nonpublicationa.
| Characteristic | Discontinued Trials (n = 38) | Unpublished Trials (n = 63) | ||||
|---|---|---|---|---|---|---|
| No. (%) | Odds Ratio (95% CI) | No. (%) | Odds Ratio (95% CI) | |||
| Unadjusted | Adjusted | Unadjusted | Adjusted | |||
| Intervention | ||||||
| Pharmacologic | 12 (31.6) | 1 [Reference] | 1 [Reference] | 25 (39.7) | 1 [Reference] | 1 [Reference] |
| Behavioral or dietary | 4 (10.5) | 1.1 (0.3-4.0) | 2.0 (0.5-8.4) | 10 (15.9) | 0.8 (0.3-2.4) | 0.6 (0.2-2.1) |
| Device or procedure | 7 (18.4) | 0.5 (0.2-1.7) | 0.7 (0.2-2.5) | 10 (15.9) | 0.9 (0.3-2.6) | 0.8 (0.3-2.5) |
| Pharmacologic and radiotherapy | 12 (31.6) | 0.6 (0.2-1.6) | 0.5 (0.2-1.3) | 15 (23.8) | 1.4 (0.6-3.3) | 1.0 (0.4-2.7) |
| Radiotherapy | 3 (7.9) | 0.4 (0.1-2.2) | 0.3 (0.0-2.0) | 3 (4.8) | 1.3 (0.3-6.6) | 0.7 (0.1-3.8) |
| Funding source | ||||||
| Industry | 10 (26.3) | 1 [Reference] | 1 [Reference] | 22 (34.9) | 1 [Reference] | 1 [Reference] |
| Mixed | 11 (29.0) | 1.1 (0.4-3.1) | 1.2 (0.4-3.6) | 16 (25.4) | 2.9 (1.1-7.4) | 3.0 (1.1-8.6) |
| Hospital or university | 13 (34.2) | 0.7 (0.3-1.9) | 0.6 (0.2-2.1) | 19 (30.2) | 1.5 (0.6-4.1) | 2.4 (0.8-7.4) |
| Otherb | 4 (10.5) | 1.7 (0.4-6.2) | 1.8 (0.5-7.5) | 6 (9.5) | 4.3 (1.3-14.0) | 4.1 (1.2-14.3) |
| Enrollment | ||||||
| Trial participants sample size, continuous | NA | 1.0 (1.0-1.0) | 1.0 (1.0-1.0) | NA | 1.0 (1.0-1.0) | 1.0 (1.0-1.0) |
Logistic regression adjusted for funding source, intervention, and sample size.
Other: private (n = 10), nonprofit (n = 8), and the National Institutes of Health (n = 2).
Discussion
Approximately 1 in 4 RCTs on HNC were discontinued, and 1 in 3 completed trials never reached publication. Our analysis of 63 unpublished trials revealed that 11 684 study participants may have been exposed to potentially harmful interventions, yet the data from these trials remain unpublished and difficult to access. This data accessibility barrier may raise ethical concerns because participants may have volunteered their time in hopes of contributing to advances in medicine and the betterment of medical care for others, only to have trial discontinuation and nonpublication potentially negate their valued contribution. If trials are consistently discontinued or fail to reach publication, the existing difficulty of recruiting participants may be exacerbated because potential trial participants might feel that their efforts would be in vain.
Our finding that 50% of RCTs about HNC were discontinued for unknown reasons was particularly limiting and may have been attributed to a low response rate from corresponding authors and incomplete trial information on ClinicalTrials.gov. These results merit further research to define contributing factors because they are not consistent with previous works in other medical specialties. For example, a 2018 orthopedic study21 found reasons for trial discontinuation for 29 of 30 (97%) discontinued trials related to osteoarthritis. Moreover, all 273 osteoarthritis RCTs (97%) reported the reasoning for trial discontinuation on ClinicalTrials.gov. A similar association was observed in a general surgery analysis in which 67 of 81 RCTs listed a reason for discontinuation on ClinicalTrials.gov.22 Comparatively, our analysis found that only 15 of 38 RCTs (39.5%) on HNC provided a reason for trial discontinuation on ClinicalTrials.gov. The lower rate of trial reporting found in HNC compared with other medical specialties suggests a lack of transparency in HNC RCTs. Furthermore, high incidence of discontinuation and inconsistent reporting of reasons for trial discontinuation in public registries might potentiate problems for future HNC RCTs, including the waste of research resources.
High discontinuation rates have been shown to contribute to the waste of already scarce resources available for medical research.24,25 Although discontinuation is necessary in certain circumstances, RCTs are consistently discontinued for reasons that are considered to be preventable.12 By addressing discontinuation attributable to preventable causes, such as insufficient recruitment, financial or personal reasons, and poor study design, research waste may be minimized.14 Although not quantified by our study, this wastage may represent a poor use of financial resources for funders, host institutions, and commissioning bodies. Moreover, increasing RCT completion rates in HNC may result in an increase in published data because discontinuation has been shown to be a risk factor for nonpublication in other medical specialties.24
Nonpublication causes substantial concern in the medical community because obtaining data from unpublished studies can be difficult. Although we used multiple search methods, we were unable to find email contacts for two-thirds of 63 unpublished trials included in our study. Consistent with our findings, Scott et al.21 had similar difficulties when attempting to locate contact information of the investigators, only finding 39.4% of email addresses.20 Up-to-date monitoring of trial status on trial registries and ensuring that each trial publication and principal investigator’s contact information are linked to the registry might alleviate some difficulties in accessing data from published and nonpublished RCTs. Without progressive steps to increase the transparency in medical research reporting, future investigators may make the same preventable mistakes that occurred in earlier studies. Furthermore, we found that other funded trials (private, nonprofit, or the National Institutes of Health) were more likely to reach publication than industry-funded trials. Industry-sponsored clinical trials, unlike government-funded trials, are known to be more susceptible to financial and business influences; however, the effects of these potential forms of bias were unclear in our study.26 It is possible that multiple factors may contribute to the failure of RCTs to reach publication, with some being more ethically problematic than others.
Recommendations for Future Studies
Exploring solutions to address the challenges that prevent studies from reaching completion or publication should be a priority, yet the optimum framework to do so remains unknown. To increase the ease of trial data and status accessibility, adherence to a more robust reporting standard and regular updates of current trial status on public registries may serve to decrease underreported interventions and wasted participant data. Requiring clinical investigators to first provide empirical evidence through pilot trials showing successful methods, including sample size targets, site locations, and acceptable recruitment periods, might minimize the discontinuation rate of trials citing reasons that are considered to be preventable. If an RCT is terminated prematurely per request of trial sponsors, investigators should remain free to publish trial data sets of their findings without risk of the sponsor’s veto.27
Given that journal editors and peer reviewers act as the gatekeepers for scientific literature, they play a critical role in limiting bias in trial outcome reporting and publication. To reduce publication bias, HNC-related journals should consider including a negative results section, as has been done in other fields of medicine.28,29 In addition, HNC-related journals and conferences should encourage authors to submit their research regardless of the direction or nature of their findings.
Limitations
Although every standard measure to determine the publication status of each RCT was used, some may still have been unobserved, which could affect the results of this study. In addition, our sample was limited to include only phase 3 and 4 RCTs and thus may not be generalizable to other clinical trial types. Because we used the pragmatic search strategy limited to only a few keywords, we might not have captured all HNC RCTs in this study. To locate all HNC RCTs, we chose the keywords based on the recommendation of a practicing HNC specialist (B.R.M.) in the field of otolaryngology. Moreover, publication bias may have contributed to trial nonpublication; thus, our results are presumably tainted by this systemic issue in medical research practice. Although reasonable effort was made to find author email addresses and make contact, we were unable to find a working email address for all authors. To solve this problem, we suggest that a permanent contact listing on trial registries be made available to the public to increase the availability of data of unpublished trials. For the aforementioned reasons, readers should be cautious in interpreting our study and should consider findings as a lower bound estimate of trial discontinuation and nonpublication.
Conclusions
This review found that reporting of reasons for trial discontinuation was lacking and that trial publication rates were low. Discontinuation and nonpublication of findings may subject patients to unnecessary treatment interventions, may result in duplication of efforts and thus exacerbate the waste of scarce financial resources, and may prevent results from being included in future systematic reviews.
eTable 1. Bar Graph Demonstrating the Distribution of Trial Status (Completed vs Discontinued) and Publication Status (Published vs Unpublished)
eTable 2. Standardized Email Used to Evaluate Rates of Publication of Completed and Discontinued Trials
eFigure. Bar Graph Demonstrating the Distribution of Trial Status (Completed vs Discontinued) and Publication Status (Published vs Unpublished)
References
- 1.Parkin DM, Bray F, Ferlay J, Pisani P. Global cancer statistics, 2002. CA Cancer J Clin. 2005;55(2):74-108. doi: 10.3322/canjclin.55.2.74 [DOI] [PubMed] [Google Scholar]
- 2.Fitzmaurice C, Allen C, Barber RM, et al; Global Burden of Disease Cancer Collaboration Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 32 cancer groups, 1990 to 2015: a systematic analysis for the Global Burden of Disease Study. JAMA Oncol. 2017;3(4):524-548. doi: 10.1001/jamaoncol.2016.5688 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA Cancer J Clin. 2019;69(1):7-34. doi: 10.3322/caac.21551 [DOI] [PubMed] [Google Scholar]
- 4.Mariotto AB, Yabroff KR, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst. 2011;103(2):117-128. doi: 10.1093/jnci/djq495 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86-89. doi: 10.1016/S0140-6736(09)60329-9 [DOI] [PubMed] [Google Scholar]
- 6.Schmucker C, Schell LK, Portalupi S, et al. ; OPEN Consortium . Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One. 2014;9(12):e114023. doi: 10.1371/journal.pone.0114023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chan A-W, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257-266. doi: 10.1016/S0140-6736(13)62296-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kaper NM, Swart KMA, Grolman W, Van Der Heijden GJMG. Quality of reporting and risk of bias in therapeutic otolaryngology publications. J Laryngol Otol. 2018;132(1):22-28. doi: 10.1017/S0022215117002407 [DOI] [PubMed] [Google Scholar]
- 9.Onishi A, Furukawa TA. Publication bias is underreported in systematic reviews published in high-impact-factor journals: metaepidemiologic study. J Clin Epidemiol. 2014;67(12):1320-1326. doi: 10.1016/j.jclinepi.2014.07.002 [DOI] [PubMed] [Google Scholar]
- 10.Murad MH, Chu H, Lin L, Wang Z The effect of publication bias magnitude and direction on the certainty in evidence. BMJ Evid Based Med. 23(3):84-86. doi: 10.1136/bmjebm-2018-110891 [DOI] [PMC free article] [PubMed]
- 11.Truong TH, Weeks JC, Cook EF, Joffe S. Altruism among participants in cancer clinical trials. Clin Trials. 2011;8(5):616-623. doi: 10.1177/1740774511414444 [DOI] [PubMed] [Google Scholar]
- 12.Schulz KF, Altman DG, Moher D; CONSORT Group . CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: 10.1136/bmj.c332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lièvre M, Ménard J, Bruckert E, et al. Premature discontinuation of clinical trial for reasons not related to efficacy, safety, or feasibility. BMJ. 2001;322(7286):603-605. doi: 10.1136/bmj.322.7286.603 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Briel M, Olu KK, von Elm E, et al. A systematic review of discontinued trials suggested that most reasons for recruitment failure were preventable. J Clin Epidemiol. 2016;80:8-15. doi: 10.1016/j.jclinepi.2016.07.016 [DOI] [PubMed] [Google Scholar]
- 15.Psaty BM, Rennie D. Stopping medical research to save money: a broken pact with researchers and patients. JAMA. 2003;289(16):2128-2131. doi: 10.1001/jama.289.16.2128 [DOI] [PubMed] [Google Scholar]
- 16.World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. Jahrbuch für Wissenschaft und Ethik. 2009;14(1). doi: 10.1515/9783110208856.233 [DOI] [Google Scholar]
- 17.US Department of Health and Human Services. Protection of human subjects: subpart A—basic HHS policy for protection of human research subjects. Electronic Code of Federal Regulation; 45 CFR 46.102(d). In effect July 19, 2018. https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=&SID=83cd09e1c0f5c6937cd9d7513160fc3f&pitd=20180719&n=pt45.1.46&r=PART&ty=HTML#se45.1.46_1102. Accessed July 19, 2018.
- 18.US National Library of Medicine. ClinicalTrials.gov. Glossary of common site terms. https://clinicaltrials.gov/ct2/about-studies/glossary. Accessed March 18, 2019.
- 19.Atkins D, Best D, Briss PA, et al. ; GRADE Working Group . Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490. doi: 10.1136/bmj.328.7454.1490 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Huang YQ, Traore K, Ibrahim B, Sewitch MJ, Nguyen LHP. Reporting quality of randomized controlled trials in otolaryngology: review of adherence to the CONSORT statement. J Otolaryngol Head Neck Surg. 2018;47(1):34. doi: 10.1186/s40463-018-0277-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Scott J, Cooper CM, Checketts JX, et al. An observational analysis of discontinuation and non-publication of osteoarthritis trials. Osteoarthritis Cartilage. 2018;26(9):1162-1169. doi: 10.1016/j.joca.2018.05.019 [DOI] [PubMed] [Google Scholar]
- 22.Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A. Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ. 2014;349:g6870. doi: 10.1136/bmj.g6870 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Song F, Hooper L, Loke YK. Publication bias: what is it? how do we measure it? how do we avoid it? Open Access J Clin Trials. 2013;2013(5):71-81. doi: 10.2147/OAJCT.S34419 [DOI] [Google Scholar]
- 24.Rosenthal R, Kasenda B, Dell-Kuster S, et al. Completion and publication rates of randomized controlled trials in surgery: an empirical study. Ann Surg. 2015;262(1):68-73. doi: 10.1097/SLA.0000000000000810 [DOI] [PubMed] [Google Scholar]
- 25.Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104. doi: 10.1136/bmj.f6104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Chopra SS. MSJAMA: Industry funding of clinical trials: benefit or bias? JAMA. 2003;290(1):113-114. doi: 10.1001/jama.290.1.113 [DOI] [PubMed] [Google Scholar]
- 27.Graf C, Battisti WP, Bridges D, et al. ; International Society for Medical Publication Professionals . Research methods & reporting. Good publication practice for communicating company sponsored medical research: the GPP2 guidelines. BMJ. 2009;339:b4330. doi: 10.1136/bmj.b4330 [DOI] [PubMed] [Google Scholar]
- 28.Evangelou E, Siontis KC, Pfeiffer T, Ioannidis JPA. Perceived information gain from randomized trials correlates with publication in high-impact factor journals. J Clin Epidemiol. 2012;65(12):1274-1281. doi: 10.1016/j.jclinepi.2012.06.009 [DOI] [PubMed] [Google Scholar]
- 29.Dirnagl U, Lauritzen M. Fighting publication bias: introducing the negative results section. J Cereb Blood Flow Metab. 2010;30(7):1263-1264. doi: 10.1038/jcbfm.2010.51 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable 1. Bar Graph Demonstrating the Distribution of Trial Status (Completed vs Discontinued) and Publication Status (Published vs Unpublished)
eTable 2. Standardized Email Used to Evaluate Rates of Publication of Completed and Discontinued Trials
eFigure. Bar Graph Demonstrating the Distribution of Trial Status (Completed vs Discontinued) and Publication Status (Published vs Unpublished)
