Cross-sectional study of Pfizer-sponsored clinical trials: assessment of time to publication and publication history

Abstract

Objective

To estimate the proportion of Pfizer-sponsored clinical trials that completed in 2010 and are published as manuscripts in the peer-reviewed literature, and to assess the manuscript development history.

Design

Retrospective, cross-sectional analysis.

Setting

Clinical trials registered in ClinicalTrials.gov that completed in 2010 for approved, Pfizer prescription products in patients or vaccines in healthy participants.

Main outcome measures

The proportion of studies for which the primary outcome(s) was published and the median time from study completion to publication. The manuscript development history included the number of times a manuscript was submitted before it was accepted for publication.

Results

Among registered clinical trials for which Pfizer was the sponsor that completed in 2010, 76 met all inclusion criteria. The primary outcome(s) for 65 (85%) studies was published in 71 manuscripts; the median time to publication was 31 months (range 3–63 months). Of the remaining 11 studies, 2 had been submitted to at least one journal, 2 had not yet been submitted and 7 had no plans to publish because the study had terminated early due to recruitment challenges. Manuscripts accepted at the first choice journal were published at median time of 28 months (range 8–63, n=31), those accepted at second choice journal were published at 32 months (3–45, n=19), and for those accepted at third choice journal, it was 40 months (range 24–53, n=13).

Conclusions

The publication rate and median time to publication from study completion for Pfizer-sponsored studies were comparable to those previously reported for combined analyses of industry and non-industry sectors. Opportunities exist for sponsors, authors and journals to explore ideas that would facilitate more timely publication for clinical trial results. However, to be effective, such changes may need to revisit the entire publication process.

Strengths and limitations of this study.

• The major strength of this study was our access to the publication development and submission history for each study in the analysis.

• Long-term follow-up of a company's sponsored study publications, at least 52 months.

• Generalisability is limited by the analysis only including studies that were completed in one calendar year (2010).

Introduction

Treatment decisions made by healthcare professionals are informed by the results of clinical trials published in peer-reviewed journals. Research conducted on studies that were sponsored by the pharmaceutical industry and completed more than a decade ago highlighted issues of delayed, incomplete or biased publication of clinical trial results.^1–3 For example, between 22%⁴ and 57%⁵ of studies supporting approval of products by the US Food and Drug Administration (FDA) remained unpublished ∼5 years after product approval and those with favourable primary outcomes were more likely to be published.^{4 5}

Concerns about incomplete, inaccurate and/or distorted reporting of outcomes for marketed drugs led to the implementation of guidelines and regulations to ensure greater transparency.⁶ In 2004, the International Committee of Medical Journal Editors (ICMJE) published a landmark editorial that mandated study registration in a public trials registry prior to or at the onset of patient enrolment as a prerequisite for publishing in their journals.⁷ In 2007, the Food and Drug Administration Amendment Act (FDAAA) was enacted, requiring registration on the ClinicalTrials.gov website of phase II–IV interventional studies of an FDA-regulated drug, biologic or device (whether or not approved for marketing).⁸ Summary results from clinical trials covered by FDAAA are required to be posted to ClinicalTrials.gov website; in particular, the results of studies for approved products were to be posted within 1 year of study completion. In the European Union, a new regulation (EU number 536/2014) was enacted in 2014 that includes measures designed to increase transparency of clinical trial data.⁹ Although posting of study results in registries improves transparency, it is not a replacement for publishing the results of trials in peer-reviewed journals. Peer-review journals remain an important source of information for healthcare providers because they place the results in context of the existing literature and describe the strengths and limitations of the studies.

Recently, the WHO has issued a position paper calling for results of clinical trials to be submitted to a journal within 12 months of study completion and to be made publically available within 24 months.¹⁰ Industry organisations have also responded. For example, the International Federation of Pharmaceutical Manufacturers and Associations issued a position paper in 2010 in which their member companies committed to submitting a manuscript for publication for all phase III clinical trials as well as trials with results of significant medical importance, regardless of outcome.¹¹ For approved products, the position paper calls for the manuscript to be submitted within 12–18 months after completion of the trial. Despite these initiatives, concerns have been raised that the situation has not improved, and there has been a call to the pharmaceutical industry to demonstrate that reporting of industry-sponsored clinical trial results has improved.¹²

Research has been conducted on industry’s adherence to FDAAA's requirements for posting results, but to the best of our knowledge, little information exists on the publication rate of industry studies in the medical literature for individual companies.¹³ We therefore conducted this analysis to determine the performance of Pfizer with respect to publishing clinical trials with the greatest potential to inform healthcare decisions (ie, interventional clinical trials in patients involving an approved product). We selected a window of time that was sufficiently recent such that trial registration would have been required, and also allowed sufficient time such that a publication might already have been expected. We also examined the details of the publication history (eg, time from study completion to manuscript submission, number of different journals to which a manuscript was submitted) so that we might identify key factors affecting publication timelines.

Methods

Analysis data set

The ClinicalTrials.gov website (accessed on 2 November 2012) was searched using the Advanced Search feature. The following information was entered in the Advanced Search form fields: ‘study type’ as interventional studies, ‘sponsor (lead)’ as Pfizer and ‘first received from’ as 9/13/2005 (ie, registered on or after the ICMJE deadline of 13 September 2005). All other search fields were left blank (see online supplementary appendix 1). The results were downloaded as a csv file and then saved as an Excel file for subsequent analyses. The fields included registration number and date, study title, recruitment status, trial phase, study completion date and presence/absence of study results.

Previous studies found that, for published studies, the median time from study completion to publication was 23–27 months.^{14 15} Therefore, we chose to assess studies with a study completion date in ClinicalTrials.gov in 2010; this should allow sufficient time from completion of the study to publication in a peer-reviewed journal for the majority of studies.

Studies that did not have a study completion date between 1 January and 31 December 2010 were excluded from the file (ClinicalTrials.gov defines ‘study completion date’ as the date on which the final data for a clinical study were collected because the last study participant made the final visit to the study location). Studies that did not involve a Pfizer product or a product co-marketed by Pfizer were excluded. Studies that did not involve an intervention with an approved Pfizer prescription product (as of November 2012) were identified by determining the approval status of the ‘intervention’ using the Drugs@fda.gov and http://www.ema.europa.eu/ema/ websites and were excluded. The following were also excluded on the basis that they had limited impact on healthcare decisions: studies of over-the-counter products, studies that involved products that were withdrawn from the market and studies with a recruitment status of ‘withdrawn’ (ie, the clinical study stopped before enrolling the first participant). Because studies in healthy participants are typically not conducted to test efficacy, those studies were identified by reviewing the ClinicalTrials.gov study ‘title’ and the ‘eligibility’ section. All healthy participant studies were excluded except for vaccine studies, for which pivotal studies are usually conducted in healthy participants with immunological efficacy end points.

Outcomes

The primary outcomes of this analysis were the proportion of studies published and the median time from study completion to publication. Secondary outcomes assessed for published manuscripts were: the number of submissions to different journals; the time from completion date to receipt, acceptance and publication by the journal; and the time from acceptance by the journal to publication.

Publication status and time to publication

Publication status for each study was initially determined by searching Pfizer's internal publication management system (Datavision (Envision Pharma Group, Glastonbury, Connecticut, USA)—last censored April 2015). The status was assigned to one of four categories as follows: ‘published’: primary outcome(s) was published in a peer-reviewed journal; ‘submitted’: primary outcome(s) of study had been submitted to one or more peer-reviewed journals; ‘not submitted’: records indicated that a manuscript was under development but had not yet been submitted; and ‘no plans to publish’: the records provided no indication that there was a plan to publish the primary outcome(s). To determine the reproducibility of the search results for ‘published’ studies, PubMed and Google Scholar were searched for all published manuscripts (search terms used included NCT number, intervention, condition and study location). For studies published in languages other than English, searches of Google were performed by Pfizer offices in the relevant country.

If the Datavision record indicated that the primary outcome(s) had been published, this was confirmed by obtaining the published manuscript(s) from the journal(s) and matching to the corresponding record in ClinicalTrials.gov using the following parameters: NCT number, study intervention, primary outcome measures, timeframe of primary outcome measure(s) and numbers of enrolled participants. For studies published in languages other than English, copies of the articles were sent to Pfizer offices in the relevant country to assist with matching these to the ClinicalTrials.gov records. Confirmation of the status of studies as ‘submitted’ was performed by obtaining records of correspondence with the journal(s). The status of studies in the ‘not submitted’ and ‘no plans to publish’ categories were confirmed by the Pfizer colleague responsible for the study.

For studies classified as ‘published’, the time to publication was assessed by calculating the duration of time between the completion date of the study (from ClinicalTrials.gov) and publication. For studies published in PubMed-indexed journals, the earlier of the e-pub (ahead of print) or print date was used as the publication date. For studies published in all other journals (ie, non-PubMed indexed), the date indicated by the publisher was used.

Manuscript development history

For studies that were categorised as published, the history of the development process for each manuscript was evaluated, including number of sequential submissions to different journals; the time from study completion date (from ClinicalTrials.gov) to receipt, acceptance and publication by the journal; and the time from acceptance to publication by the journal. The dates of receipt and acceptance were obtained where possible from the published manuscript; where those data were not available from the journal, Datavision records were recorded and used in calculations. The publication date was obtained as described above. For calculation of time to publication, if the primary outcomes of the trial were divided among more than one manuscript, the time to the publication of the final manuscript was used.

The number of submissions required prior to acceptance by a journal was determined by reviewing the Datavision records and confirmed by reviewing correspondence with the journals. Manuscripts were categorised as published at the authors' first, second, third or fourth choice journal.

If the primary outcomes of the trial were divided among more than one manuscript, the data pertaining to the final manuscript was used for the time to study publication and the manuscript development history analyses (if the primary outcomes were split across two manuscripts and both manuscripts were published on the same day, the data from both manuscripts were included in the manuscript development history analysis).

Statistical analysis

All data were assessed using descriptive statistics. Time was calculated in months. The median was used to estimate the central tendency.

Results

Among all interventional studies registered on or after the ICMJE deadline with a completion date in 2010 and for which Pfizer was the sponsor (lead), 76 studies met all the inclusion criteria including involvement of an approved Pfizer prescription product in patients or a vaccine in healthy participants (trials of unapproved indications for approved products were included; figure 1). The majority of the studies (63%) enrolled more than 100 participants. All phases of development were represented, although the largest proportion was phase III (41%). The studies covered a wide range of conditions and study designs (table 1).

Figure 1.

Flowchart of included studies.

Table 1.

Characteristics of studies included in the analysis

	N (%) (n=76)
Conditions
Cancer	12 (16)
Vaccine	12 (16)
Immunological diseases	11 (14)
Pain	8 (11)
Psychiatric disorders and addiction	6 (8)
Bacterial/fungal diseases	4 (5)
Cardiovascular disease	4 (5)
Nervous system diseases	4 (5)
Urological conditions	4 (5)
Other	11 (14)
Phase
1	4 (5)
2	19 (25)
2/3	3 (4)
3	31 (41)
4	18 (24)
Not specified	1 (1)
Allocation
Randomised	44 (58)
Non-randomised	27 (36)
Not specified	5 (7)
End point classification
Efficacy	8 (11)
Pharmacodynamics	1 (1)
Pharmacokinetics	3 (4)
Safety	5 (7)
Safety/efficacy	57 (75)
Not specified	2 (3)
Interventional model
Cross-over	6 (8)
Parallel group	40 (53)
Single arm	30 (39)
Masking
Double-blind	32 (42)
Open label	44 (58)
Primary purpose
Basic science	1 (1)
Prevention	15 (20)
Treatment	58 (76)
Not specified	2 (3)
Age group
Adult	7 (9)
Senior	1 (1)
Child	8 (11)
Adult/senior	52 (68)
Adult/child	3 (4)
Adult/child/senior	5 (7)
Recruitment
Completed	67 (88)
Terminated	9 (12)
Participants enrolled
1–100	28 (37)
101–500	31 (41)
>500	17 (22)

Publication status and time to publication

By April 2015 (last censoring date), all studies had at least 52 months follow-up since study completion. Sixty-five (85%) studies were identified as having been published in 71 manuscripts (see online supplementary appendix 2). Primary outcome(s) for 61 studies was published in one manuscript each; 4 studies had primary outcome(s) reported in multiple manuscripts (2 studies had 2 manuscripts each, and 2 studies had 3 manuscripts each). All were published in English except for two that were published in Japanese (studies conducted in Japan). All English language manuscripts were found in searches of PubMed and Google Scholar; the Japanese language manuscripts were found in searches of Google Japan.

Among the remaining 11 studies, 2 (3%) were categorised as ‘submitted’, 2 (3%) were ‘not submitted’ and 7 (9%) were categorised as ‘no plans to publish’ (figure 2). The seven studies categorised as ‘no plans to publish’ were all terminated before meeting their enrolment goal due to recruitment challenges (see online supplementary appendix 3).

Figure 2.

Study publication status after at least 52 months after study completion.

The median time from study completion to publication was 31 months (range 3–63 months). By 12 months after study completion, 5 (7%) studies were published, and at 24, 36, 48 and 60 months, the numbers of published studies were 18 (24%), 41 (54%), 58 (76%) and 64 (84%), respectively (figure 3).

Figure 3.

Cumulative percent of studies published in a peer-reviewed biomedical journal.

Manuscript development history

More than 50% of the manuscripts required submission to more than one journal to achieve publication. Among the 66 published manuscripts, 31 (47%) were published in the first journal selected by the authors, 19 (29%) were published in the second choice journal, 13 (20%) were published in the third choice journal and 1 (1%) was published in the fourth choice journal. Information was not available for 2 (3%) of the manuscripts. The 66 manuscripts were published in 47 different journals. The time from study completion date to receipt, acceptance and publication of the manuscript by the journal increased with the number of consecutive submissions required (figure 4A). Once accepted, the median time to publication by the journal was consistent for all groups (figure 4B).

Figure 4.

Manuscript development timing steps for published studies. (A) Median times from study completion to receipt, acceptance and publication by the journal for 66 manuscripts published in first, second and third choice journals. (B) Median times to publication from receipt by journal, acceptance by journal and completion of study for first, second and third choice journals. Footnote: A total of three manuscripts were not included in the analysis: one was published at the fourth choice journal and two manuscripts did not have information on choice of journal available.

Discussion

Principal findings

Eighty-five per cent (65/76) of the interventional studies in patients or vaccine studies in healthy participants for Pfizer-approved products that completed in 2010 were published within 52 months of study completion. The median time from study completion to publication was 31 months. Allowing for differences in study methodology and duration of follow-up, our results are comparable to studies that evaluated publication rates in National Institutes of Health (NIH)-funded studies,¹⁵ large randomised clinical trials¹⁴ and vaccines studies.¹⁶ In our analysis, there were 11 studies where the primary outcome(s) was not published (all 11 unpublished studies have results posted on ClinicalTrials.gov). Of these, seven studies were classified as ‘no plans to publish’ because they terminated prior to the planned enrolment goal being met due to recruitment challenges, and so a publication would be of limited value for healthcare decisions. Among the remaining four, only two had not yet been submitted at least once. More than half of all manuscripts were rejected from the authors' first choice journal and sometimes multiple submissions were required before acceptance. This was a likely contributor to delays in publications, as manuscripts published at the third choice journal took 12 months longer to be published than manuscripts accepted at the first choice journal. To the best of our knowledge, this is the first study to provide an analysis of the publication history in terms of its component parts and associated timelines.

Implications

A target publication rate of 100% may not be realistic because recruitment challenges, a long-standing problem in clinical trials, can result in study termination with too few participants to allow reliable analyses.^{17 18} For example, an analysis of a cohort of 122 trials between 1994 and 2002 found that only 31% achieved their recruitment target, and enrolment was halted due to poor recruitment in 11% of the studies.¹⁹ For studies that are not published in the medical literature, sponsors can meet their obligation to disseminate trial results by using other avenues, for example, posting the results on a public registry such as ClinicalTrials.gov or on a company website.

The median time to publication of 2.5 years is not satisfactory, and ideally all studies should have been submitted at least once within the follow-up period. Streamlining of the processes from study completion through manuscript submission must be part of any solution. Those processes include verifying and analysing the data, production of the tables and figures, and development of the manuscript. Manuscript development typically takes months as a result of challenges such as obtaining input from multiple authors who are often located in different time zones and have competing priorities. Delays may be introduced by the need to gain consensus about interpretation of results among authors and requests by authors for additional analyses. To reduce the time to submission for manuscripts, some pharmaceutical companies have implemented policies requiring a manuscript reporting the results of the primary outcome(s) of a clinical trial to be submitted within a predefined timeframe (eg, within 12–18 months of study completion).^{20 21}

Assuming an 18-month submission timeline is achieved, journal selection and policies will still play an important role in the overall time to publication. Authors frequently prefer submission to top tier journals with high visibility and impact factors, but the acceptance rates of such journals are typically quite low. Even when an appropriate candidate journal has been identified, the manuscript still may not be accepted because journal editors must make decisions based not only on scientific merit of a particular manuscript but also take into consideration competing editorial priorities. In our study, manuscript rejection was common (over 50% of the manuscripts were rejected at least once) and resulted in many months delay in publication. In a separate analysis of this group of studies, we reviewed the journal decision letters from manuscripts that were rejected from the authors' first choice journal.²² For those with reviewers' comments, we identified factors that, if addressed, could reduce the frequency of rejection in the future. However, 50% of the rejected manuscripts were not peer reviewed, which suggests inappropriate journal selection. Standard journal policies that prohibit simultaneous submission to other journals also contribute to publication delays.²³

Reducing the time from study completion to publication will take the continued and enhanced commitment of all stakeholders including industry, authors and journals. Pharmaceutical companies could, if they have not already, implement a policy requiring submission of a manuscript within a certain time after study completion (eg, 12–18 months); and publish data on their compliance with the policy. Authors (academic and company) could potentially shorten the manuscript development timeline by agreeing to limit the number of individuals contributing to a manuscript, thereby reducing the challenges a large group of individuals encounter when collaborating. The frequency for multiple journal submissions, and the associated delay in publication, might be decreased by lessening the emphasis on publications in high tier journals in the academic tenure process. Inclusion of ‘citable and accessible research products’ beyond publications has been reported in the grant process,²⁴ and if adopted for the tenure process, could have a positive impact on timelines. Journals have improved the dissemination of data by leveraging electronic publishing and addressing the need for the publication of negative and neutral results. Moving forward journals might consider: allowing authors to submit a manuscript simultaneously to more than one journal,²⁵ instituting a centralised submission and peer-review process, and implementing a fast-track process for primary manuscripts from clinical trials. To optimise the publication process, it would be beneficial to form a working group with representation from all the stakeholders.

Strengths and limitations

The main strength of this analysis is that we were able to identify and track the publication history of all the studies using information from Pfizer's publication database as well as the information in the public domain from published manuscripts. This allowed us to collate all studies with their corresponding publications; it also provided a unique opportunity to analyse the major publication steps and to estimate the time associated with each step.

Other studies reporting publication rates for clinical trials have relied on searching a limited number of databases (eg, PubMed, Google Scholar, Scopus and/or EMBASE) to identify published results.^{15 16 23} Using that strategy, the apparent publication rate may be underestimated because studies may not be indexed in the selected databases or might be missed in manual searches. Although the ICMJE recommends clinical trial identifiers to be included in abstracts in PubMed to facilitate identification and enhance transparency,²⁶ that has not been uniformly adopted by authors and journals.^{27 28}

Another strength of the current study is the inclusion of data regarding the time to submission and publication. Previous studies were not designed to provide information on the timing of milestones within the publication process. However, information about prior submissions of published studies and publication milestones for studies that have not yet been published cannot readily be verified by independent researchers, and so that aspect might be considered a limitation.

Other limitations included the relatively small number of studies in the analysis when compared with similar studies in the literature. Our analysis only included studies that were completed in 2010, so it is not clear that the findings can be generalised to those published in previous years or to those published subsequently. Thus, the smaller numbers of studies in subgroup analyses (eg, manuscripts with >2 submissions) may limit the extent to which the timing of the associated steps can be generalised due to the variability of the estimates. However, our analysis does provide an estimate of the time delay incurred by multiple submissions, which has not been available to date in the literature. This analysis also did not address other issues raised in the literature such as biased or selective publication. To appropriately evaluate this important issue, one would require a large cohort of hypothesis testing studies (eg, phase 3 clinical trials). Our cohort included a relatively small sample of hypothesis-testing and non-hypothesis-testing studies (eg, pharmacokinetic studies, open-label extension studies), which does not lend itself to this type of assessment. However, even without a study designed for this purpose, we did encounter examples of publications reporting negative study results (ie, studies of an approved product for an unapproved indication).^29–32

Conclusion

In our analysis, 85% of Pfizer-sponsored clinical trials were published in a peer-reviewed journal with a median time to publication of 31 months. Opportunities exist for authors, journals and sponsors to explore ideas that would result in more timely publication for clinical trial results. However, unless the entire process undergoes a transformation, any improvements are likely to be incremental.