Abstract
Purpose
Trustworthy reporting of quadrivalent human papillomavirus (HPV) vaccine trials is the foundation for assessing the vaccine’s risks and benefits. However, several pivotal trial publications incompletely reported important methodological details and inaccurately described the formulation that the control arms received. Under the Restoring Invisible and Abandoned Trials initiative (RIAT), we aim to restore the public record regarding the content and rationale of the controls used in the trials.
Methods
We assembled a cohort (five randomised controlled trials) described as placebo-controlled using clinical study reports (CSRs) obtained from the European Medicines Agency. We extracted the content and rationale for the choice of control used in each trial across six data sources: trial publications, register records, CSR synopses, CSR main bodies, protocols and informed consent forms.
Results
Across data sources, the control was inconsistently reported as ‘placebo’-containing aluminium adjuvant (sometimes with dose information). Amorphous aluminium hydroxyphosphate sulfate (AAHS) was not mentioned in any trial registry entry, but was mentioned in all publications and CSRs. In three of five trials, consent forms described the control as an ‘inactive’ substance. No rationale for the selection of the control was reported in any trial publication, register, consent form, CSR synopsis or protocol. Three trials reported the rationale for choice of control in CSRs: to preserve blinding and assess the safety of HPV virus-like particles as the ‘safety profile of (AAHS) is well characterised’.
Conclusions
The stated rationale of using AAHS control—to characterise the safety of the HPV virus-like particles—lacks clinical relevance. A non-placebo control may have obscured an accurate assessment of safety and the participant consent process of some trials raises ethical concerns.
Trial registration numbers
NCT00092482, NCT00092521, NCT00092534, NCT00090220, NCT00090285.
Keywords: paediatric infectious disease & immunisation
INTRODUCTION
Randomised, placebo-controlled clinical trials are considered the gold standard for evaluating new vaccines. To evaluate the vaccine’s efficacy and safety, the manufacturer of quadrivalent human papillomavirus (qHPV) vaccine conducted multiple clinical trials including around 30 000 females and males. The vaccine was approved in 2006. Today, citing the vaccine’s efficacy and extensive safety profile, increasing HPV vaccine coverage is a high priority for many national governments and the WHO.1–3
Multiple trials of qHPV vaccine are reported as ‘placebo-controlled.’4–8 However, participants in the ‘placebo’ arms of these pivotal trials received an injection-containing amorphous aluminium hydroxyphosphate sulfate (AAHS), a proprietary adjuvant. AAHS is used in qHPV vaccine to boost immune response, but the rationale for adding AAHS to the ‘placebo’ is not reported in publications of these trials4–8 and is contrary to the advice of the public health bodies and regulators. The WHO, for example, recommends control recipients in trials testing an unlicensed, experimental vaccine receive either an inert substance or an approved efficacious vaccine.9 Similarly, the European Medicines Agency (EMA), which provides specific guidance on the clinical evaluation of vaccines, recommends the use of a licensed vaccine without an effect on the target disease if a placebo cannot be used.10
In addition, the efficacy and safety analyses of these five qHPV vaccine trials were conducted as if the trials were controlled with inert placebo when they were not. None of the key publications for these trials, which have been used to inform regulatory and health decision making, appear to discuss how AAHS-containing control could affect the interpretation of results.4–8
We consider the omission in journal articles, of any rationale for the selection of AAHS-containing control, to be a form of incomplete reporting of important methodological details, and believe the rationale must be reported. We also consider that the use of the term ‘placebo’ to describe an active comparator like AAHS inaccurately describes the formulation that the control arm participants received, and constitutes an important error that requires correction. If trial participants were told they could receive ‘placebo’ (widely defined as referring to an ‘inactive’11 12 or ‘inert’9 substance) without being informed of all non-inert contents of the control arm injection, this raises ethical questions about trial conduct as well.
In January and February 2019, we publicly declared our intention to systematically correct the record for these trials13 14 in accordance with the principles of the Restoring Invisible and Abandoned Trials (RIAT) initiative, of which some of us are founders.15 RIAT was founded with the intent of restoring the text of published biomedical articles where there is clear evidence of distortion or omission (as in the case of reporting bias or non-publication of clinical trials) and the original trial authors and/or sponsors fail to take any corrective action (thereby abandoning their responsibilities). While RIAT projects to date have focused on restoring trials through a reanalysis of all study data for a given trial,16–19 this project differs in its scope in two ways: first, we are focused on multiple trials in a single manuscript; second, this restoration is restricted to a specific aspect of the trials’ methodology (the choice of control). We believe this ‘focused’ RIAT is important given the considerable implications that the choice of control has on the interpretation of trial results, and the fact that the problem is not confined to a single trial but rather spans a trial programme. Other criticisms of the reporting in qHPV trial publications (e.g., selective outcome reporting) have been made,20 but we do not address those concerns here. Consistent with the RIAT approach, we wrote to the corresponding authors of all five publications as well as the sponsor requesting that they correct their publications. None contested our concerns nor indicated any intention to correct the record. As such, we consider these trials abandoned and hereby restore the record using data obtained under freedom of information requests to the EMA according to our prespecified protocol.21
Subjects and methods
Research design
Review of text describing the content and rationale for choice of control across published and unpublished data sources.
Data sources, searches and restoration sample
Since 2010, transparency policies at the EMA have allowed independent investigators to obtain access to clinical study reports (CSRs) and other clinical data once protected as commercially confidential.22 CSRs are reports structured according to international guidelines,23 written by sponsors and submitted to regulators, generally as part of marketing authorisation applications, and provide far more detail than journal publications.24 In 2015, the EMA expanded its efforts to proactively publish CSRs to its website following regulatory decisions,25–27 a practice Health Canada also adopted as of 2019.28
The cohort of trials potentially eligible for restoration was all clinical trials of qHPV vaccine and nonavalent HPV vaccine for which we obtained CSRs from the EMA. These were received in response to a May 2014 freedom of information request. The process of obtaining CSRs has been previously described in an Index study29 and Analysis article.30 All trials evaluating qHPV vaccine and nonavalent HPV vaccine for which CSRs were obtained by 1 November 2018 were eligible for inclusion in this restoration.
Each CSR was reviewed by two independent assessors (with conflicts resolved by discussion or third party) for possible inclusion against the following criterion: trial randomised control group participants to aluminium-containing adjuvant (as a control). We excluded trials which used an approved vaccine as the control.
Identification of matching trial publications
We aimed to identify the most impactful, company-authored trial publication for all trials in our sample using the following approach:
We identified the NCT number for each trial by searching for the trial ID, obtained from the CSR, on ClinicalTrials.gov. We then compared trial characteristics (study population, intervention, comparator, name of primary outcome measure(s) and number of participants) with the CSR and a published index of HPV vaccine trial programme29 to confirm that the correct NCT record was identified.
From the ClinicalTrials.gov registry entry for each trial, we recorded all publications (in all categories) listed under the ‘Study Details’ tab.
Using the DOI of these publications, we identified the most cited publication of each trial using the Scopus database and obtained the article’s full-text PDF and supplementary material.
This process was carried out by two independent reviewers with discrepancies resolved by discussion or third-party consultation.
Determination of under-reporting and misreporting
For each trial using a concurrent aluminium-containing adjuvant as a control, we reviewed the publication to determine the potential presence of under-reporting of the rationale for using such a control and misreporting of the control intervention (i.e., describing the control as ‘placebo’).
Data extraction
For each trial using six data sources (CSR synopsis, CSR main body, trial protocol, informed consent form, trial publication and trial register entry), we recorded (1) the phrases used to describe the comparator to qHPV vaccine; (2) the rationale for using aluminum-containing adjuvant as a control, if present and (3) all listed contents (ingredients) of formulation received by intervention and control arms.
To enable an exhaustive list of all descriptions of the comparator across all document types, especially CSRs which were often thousands of pages long, we searched for the terms placebo, control, comparator, alum, AAHS, adjuvant and MAA (Merck Aluminum Adjuvant). We categorised the verbatim phrases into three broad categories: phrases that described a placebo (with no mention of adjuvant), phrases that described a placebo containing adjuvant and phrases describing a placebo containing adjuvant as well as adjuvant dose information. We recorded the frequency of terms used to describe the formulation of the control arm and its ingredients, by data source.
Extractions were independently carried out by two assessors. There were no discrepancies, and a consolidated analysis data set was created.
Results
We obtained 14 trials from the EMA (eight qHPV vaccine, six nonavalent HPV vaccine), of which five met our prespecified inclusion criteria (V501-012, V501-013, V501-015, V501-019 and V501-020; figure 1). All included trials compared qHPV vaccine to aluminium-containing adjuvant. Corresponding register and publication information for each trial are listed in table 1.
Table 1.
Trial ID/name | ClinicalTrials.gov registration # | Most cited trial publication | Participants enrolled* | Number of citations† |
V501-012 ‘FUTURE 1’ | NCT00092482 | Garland et al 7 | 3882 women, aged 16–24 | 32 |
V501-013 ‘FUTURE 1’ | NCT00092521 | Garland et al 8 | 5455 women, aged 16–24 | 1312 |
V501-015 ‘FUTURE 2’ | NCT00092534 | FUTURE II Study Group4 | 12167 women, aged 15–26 | 1440 |
V501-019 ‘FUTURE 3’ | NCT00090220 | Muñoz et al 5 | 3819 women, aged 21–46 | 318 |
V501-020 | NCT00090285 | Giuliano et al 6 | 4065 men, aged 15–27 | 593 |
*Details taken from the clinical study report.
†Citation count from SCOPUS database as on 14 December 2018.
Describing the control arm
There was heterogeneity in the verbatim phrases used to describe the control arm across all trials and types of documents (table 2).
Table 2.
Phrases used | ICF | CSR | Register | Publication | |||
Synopsis | Main body | Protocol | Abstract | Full text | |||
Placebo | 5/5 | 5/5 | 5/5 | 5/5 | 4/5 | 4/5 | 5/5 |
‘Placebo’ or ‘matching placebo’ | 5/5 | 5/5 | 5/5 | 5/5 | 4/5 | 4/5 | 5/5 |
‘Placebo (which is a dose that contains no active ingredients)’ | 4/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 |
‘Placebo … an inactive substance’ or ‘placebo … an inactive solution’ | 3/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 |
‘Placebo HPV Vaccine’ | 3/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 |
Placebo with adjuvant | 0/5 | 1/5 | 5/5 | 4/5 | 0/5 | 0/5 | 5/5 |
‘Placebo with adjuvant’ | 0/5 | 0/5 | 2/5 | 0/5 | 0/5 | 0/5 | 0/5 |
‘Aluminum adjuvant placebo’ or ‘Alum-placebo’ or ‘aluminum-placebo’ or ‘aluminum-containing placebo’ or ‘visually indistinguishable aluminum-(or ‘AAHS’)containing placebo’ or ‘matched aluminum-placebo’ or ‘placebo (aluminum adjuvant)’ | 0/5 | 1/5 | 4/5 | 4/5 | 0/5 | 0/5 | 5/5 |
Placebo with adjuvant and dose | 3/5 | 3/5 | 3/5 | 5/5 | 0/5 | 0/5 | 0/5 |
‘Placebo’-containing 225 ug of aluminium adjuvant | 0/5 | 2/5 | 1/5 | 4/5 | 0/5 | 0/5 | 0/5 |
‘Placebo’ or ‘matched placebo’-containing Merck standard aluminium diluent (225 mcg alum), with or without mention of normal saline | 0/5 | 1/5 | 0/5 | 2/5 | 0/5 | 0/5 | 0/5 |
‘Placebo’-containing 225 mcg of aluminium as AAHS, with or without mention of normal saline | 0/5 | 1/5 | 2/5 | 0/5 | 0/5 | 0/5 | 0/5 |
‘Inactive solution (containing aluminium 225 mcg/dose)’ | 3/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 |
Normal saline = USP (NaCl 0.9%).
AAHS, amorphous aluminum hydroxyphosphate sulfate; CSR, clinical study report; ICF, informed consent form.
The publication abstracts and register entries for four trials only referred to the control arm as ‘placebo’ or ‘matching placebo’ (Garland et al 8/NCT00092521, FUTURE II Study Group4/NCT00092534, Muñoz et al 5/NCT00090220, Giuliano et al 6/NCT00090285). The publication abstract and register entry for one trial (Garland et al 7/NCT00092482) did not mention the existence of a control arm.
The informed consent forms for two trials (V501-019 and V501-020) did not provide participants with any information regarding the contents of the control arm. The informed consent forms for three trials (V501-012, V501-013 and V501-013) informed participants that they may be exposed to an ‘inactive solution (containing aluminium 225 mcg/dose)’.
The CSRs of all trials described the control arm as a ‘placebo’-containing 225 mcg of aluminium. While all CSRs stated the type of aluminium adjuvant (AAHS), the information was not reported in a consistent location (i.e., CSR synopsis, main body and protocol). A summary of reported ingredients, by data source, is presented in table 3.
Table 3.
Trial | Clinical study report | Informed consent form | Register | Publication |
V501-012 | 225 mcg of aluminium as AAHS in normal saline | Inactive solution (containing aluminium 225 mcg/dose) | Control intervention not listed | Aluminium-containing placebo |
V501-013 | 225 mcg of aluminium as AAHS in normal saline | Inactive solution (containing aluminium 225 mcg/dose) | No ingredients listed; only states ‘placebo’ | Aluminium-containing placebo |
V501-015 | 225 mcg of aluminium as AAHS in normal saline | Inactive solution (containing aluminium 225 mcg/dose) | No ingredients listed; only states ‘placebo’ | Aluminium-containing placebo |
V501-019 | Merck standard aluminium diluent (225 mcg alum) in normal saline | No ingredients listed. Only states: ‘The HPV vaccine placebo contains no active vaccine’ | No ingredients listed; only states ‘placebo’ | Aluminium-containing placebo |
V501-020 | 225 mcg of aluminium as AAHS in normal saline | No ingredients listed. Only states: ‘placebo, an injection that looks the same as the vaccine but has no active ingredient’ | No ingredients listed; only states ‘placebo’ | AAHS-containing placebo |
Normal saline = USP (NaCl 0.9%).
AAHS, amorphous aluminum hydroxyphosphate sulfate.
We assessed the frequency of terms used to describe the control for the trial with the most highly cited publication in our study (V501-015). In the publication, the control was referred to as a placebo (with no mention of adjuvant) 67 times and one time as a placebo with adjuvant.4 In the CSR, the control was referred to as a placebo 1450 times versus 326 times as placebo with adjuvant, and across six informed consent forms (US and international versions of the main consent form, child assent form and pregnancy consent addendum), 54 versus 2, respectively. Finally, in the register entry, the control was referred to as a placebo 50 times versus 0 times as a placebo with adjuvant.
Rationale for aluminium-containing control
The rationale for the selection of an AAHS-containing control was reported in the CSR of three trials (V501-012, V501-013 and V501-015). No rationale was provided in any document for two trials (V501-019 and V501-020). In all five trials, the protocol, publication, register entry, informed consent form and CSR synopsis did not contain a rationale for the selection of the control (table 4).
Table 4.
Trial | Publications, registers, study protocols, ICFs, and CSR synopses | Clinical study reports (CSR main body) |
V501-012 | None provided | ‘Aluminum adjuvant was chosen as the appropriate control for the qHPV vaccine for the following reasons:
|
V501-013 | None provided | |
V501-015 | None provided | |
V501-019 | None provided | None provided |
V501-020 | None provided | None provided |
Normal saline = USP (NaCl 0.9%).
CSR, clinical study report; ICF, informed consent form; qHPV, quadrivalent human papillomavirus; VLP, virus-like particle.
The three studies that reported a rationale for the selection of an AAHS-containing control provided identical reasons: (1) to preserve blinding and (2) to assess the safety of HPV virus-like particles given that the ‘safety profile of (AAHS) is well characterised’.
Discussion
On its website, qHPV vaccine’s sponsor defines a placebo as ‘an inactive pill, liquid, or powder that has no treatment value’.11 This definition is consistent with the decades old notion of placebos as pharmacologically inert substances used to obtain unbiased assessments in experimental research.31
The five pivotal trials of qHPV vaccine that were the subject of this restoration used a control with aluminum-containing adjuvant. This was stated in the journal publications of all trials. But contrary to what the original investigators reported, the trials were not placebo-controlled. The inclusion of AAHS in the control arm means that the control should not be reported as a placebo since AAHS is not inactive. While the original FDA-approved prescribing information in 2006 reported the AAHS-containing control as ‘placebo’, this was updated in September 2008, and all such references were removed and replaced with ‘AAHS control’.32–34
Inaccurate use of the term placebo is not confined to qHPV vaccine trials. As no regulations govern placebo composition, researchers have documented a diversity in what gets labelled a placebo in research, and that depending on the experimental arm, some so-called placebos may have effects that influence study outcomes.35
With respect to adjuvants in vaccines, the FDA has noted that ‘adjuvants have their own pharmacologic activity, which may affect both the immunogenicity and the safety of vaccines. Adverse reactions may include local reactions such as pain, swelling, injection site necrosis, and granulomas. Systemic reactions may include nausea, fever, arthritis, as well as potential immunotoxic reactions. Unexpected, rare events may also occur’.36
The FDA has also stated that ‘the evaluation of safety of an adjuvanted vaccine needs to include special safety considerations’, which may include data ‘derived by comparing the adjuvanted vaccine to a placebo or the unadjuvanted vaccine antigen, if feasible’.36 The extent to which the evaluation of qHPV vaccine included such special safety considerations is debatable given the presence of adjuvant in the control.
The implications of AAHS controlled trials on an accurate assessment of the safety and efficacy of qHPV are important. It is possible that the presence of AAHS in both arms of the trials masked AAHS’s reactogenic potential thereby obscuring any differential rates in harms, leading to an underestimate in harms. The qHPV vaccine trial results are consistent with these statements. Across the five trials using AAHS in the control, the absolute risk of injection-site adverse events was only 6%–13% higher in qHPV vaccine patients. By contrast, in study V501-018, which used a control lacking AAHS, the absolute risk increase was 25% (table 5). It is also possible that AAHS’s stimulating properties (which to our knowledge have never been studied on their own) provoked an immune response which may have been protective against endpoints studied in the trial, thereby leading to an underestimation of the effectiveness of qHPV vaccine. Because all the five trials lacked a true inert comparator arm, the trial data cannot resolve these questions.
Table 5.
Trial | Gardasil | Control | Difference | Source |
V501-012 | 87.9% | 78.6% (with AAHS) | 9.3% | CSR p.33 |
V501-013 | 88.0% | 79.8% (with AAHS) | 8.2% | CSR p.13 |
V501-015* | 84.4% | 77.9% (with AAHS) | 6.5% | Publication table 4 |
V501-019 | 76.7% | 64.2% (with AAHS) | 12.5% | CSR p.566 |
V501-020 | 60.1% | 53.7% (with AAHS) | 6.4% | CSR p.348 |
V501-018 | 75.3% | 50.0% (without AAHS) | 25.3% | CSR p.33 |
*The CSR for V501-015 reports injection-site adverse experiences as 8.8% (qHPV) versus 7.6% (control), but these figures are likely an underestimate as injection-site adverse experiences were only collected for participants in the Non-Serious Adverse Experience Substudy, and data for this subpopulation were reported in the publication but not CSR.
AAHS, amorphous aluminium hydroxyphosphate sulfate; CSR, clinical study reports.
It is not clear whether other trials can establish qHPV vaccine’s true safety and efficacy profile. The FDA instructed the sponsor to conduct a safety study with a control not containing aluminium, according to the company’s submission to Japanese regulators,37 and the CSR for trial V501-018 states that it was the only qHPV vaccine trial to do this. In contrast to other studies, the FDA medical officer who reviewed the sponsor’s marketing application referred to this trial’s control as a ‘true placebo’.38 However, V501-018’s short follow-up of control arm participants (30 months), small sample (597 receiving control) and younger age relative to other trials makes it inadequate to resolve safety questions.
Our study raises serious questions about the ethics of consent in the trials. The informed consent forms for the trials described the placebo as an ‘inactive solution’ and ‘a dose that contains no active ingredients’. This likely misled many trial participants who would have reasonably expected the control did not contain a pharmacologically active adjuvant.
Our restoration makes public, for the first time, the sponsor’s rationale for using AAHS as a control. The manufacturer gave two reasons: first, to ensure that the control was not visually distinguishable, and second, to assess the safety profile of the HPV virus-like particles as the ‘safety profile of Merck’s aluminium adjuvant is well characterised’.
The fact that the qHPV vaccine and nonavalent HPV vaccine clinical trial programme included two randomised, double-blind trials that did not use an AAHS-containing control demonstrates that there was no necessity to use AAHS to maintain blinding. In these trials (V501-018 and V503-006), unblinded study personnel who were otherwise not involved in the study prepared and administered injections.
Second, the safety profile of AAHS is not well characterised. We are unaware of any randomised trial directly evaluating AAHS against placebo; a network meta-analysis is underway that aims to use extant randomised controlled trial (RCT) data to determine the safety profile of the AAHS component of qHPV vaccine.39 Moreover, while other vaccines contain AAHS (e.g., Recombivax and VAQTA), such vaccines cannot be used to characterise AAHS safety as ‘it cannot be assumed that an adjuvant that is safe in one vaccine with a given antigen will be safe when added to another vaccine’.40 The unknown consequences of aluminum-containing adjuvants, more generally, is concerning. A Cochrane review is currently underway to determine whether any RCTs exist which evaluated the effects of aluminium adjuvants compared with placebo or no intervention.41
Finally, the stated rationale of using AAHS control, to characterise the safety of the HPV virus-like particles, lacks clinical relevance. The clinically relevant question is what are the effects (benefits and harms) of qHPV vaccine—the whole product, not one of its components.
As crucial documents written before trials commence, it is concerning that study protocols did not include a rationale for the use of AAHS control. Protocols for V501-015, V501-019 and V501-020 stated: ‘To provide an appropriate control for the Quadrivalent HPV (Types 6, 11, 16, 18) L1 VLP Vaccine, the placebo used in this study will be Merck standard aluminum diluent (225 µg alum) in normal saline, USP (NaCl 0.9%)’. But the documents did not explain what made the adjuvant ‘appropriate’. We were also concerned that ClinicalTrials.gov entries described the control with a single word (‘placebo’, not even mentioning AAHS), and in one case did not even list the control.
The Consolidated Standards of Reporting Trials (CONSORT) guidelines for reporting RCTs call for study interventions for each group to be described ‘with sufficient details to allow replication, including how and when they were actually administered’.42 We think CONSORT should also ask investigators to document their rationale for selecting these interventions, particularly the choice of control. Such reporting would allow for more transparency and a more informed discussion around the appropriateness of chosen controls, and is consistent with regulatory advice to industry that ‘the choice of control group is always a critical decision in designing a clinical trial’.43
Limitations
Across all study documents, the only ingredients we found listed in the control were AAHS and normal saline (NaCl 0.9%). However, there may have been other ingredients. According to qHPV vaccine’s prescribing information, each dose of vaccine contains ‘9.56 mg of sodium chloride, 0.78 mg of L-histidine, 50 mcg of polysorbate 80, 35 mcg of sodium borate, <7 mcg yeast protein/dose and water’, in addition to AAHS and HPV virus-like particles.32–34 To test HPV virus-like particles, as the manufacturer stated was its intention in using an AAHS control, the control would logically have also included these other ingredients in addition to AAHS. We recommend that reports of trials (across publications, protocols, CSRs and registers) should contain a complete list of ingredients of all trial arms (if necessary, in supplementary materials when space limitations apply).
Our study may not represent a complete list of qHPV vaccine trial publications that under-report and misreport the control. In our screening, we identified a trial (V501-005) with identical problems to the trials included in this restoration; however, this trial was excluded from our analysis as it reported on the use of a monovalent experimental HPV vaccine which was never licensed.44
Conclusions
The sponsor’s stated rationale of using AAHS control in qHPV vaccine trials—to characterise the safety of the HPV virus-like particles—lacks clinical relevance, as the clinically relevant issue is to characterise the safety profile of the whole qHPV vaccine. The non-placebo control used in qHPV vaccine trials may have obscured an accurate assessment of safety of qHPV vaccine and the participant consent process of some trials raises ethical concerns.
Footnotes
Contributors: PD: conceptualisation; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; software; supervision; validation; visualisation; roles/writing: original draft; PD had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. LS: conceptualisation; data curation; methodology; data curation; writing: review and editing. TJ: conceptualisation; formal analysis; funding acquisition; investigation; methodology; roles/writing: original draft. KH: investigation; writing: review and editing. HL: investigation; writing: review and editing. FB: investigation; writing: review and editing. OS: data curation; writing: review and editing. MJ: investigation; writing: review and editing.
Funding: Laura and John Arnold Foundation
Competing interests: The Laura and John Arnold Foundation funds the RIAT Support Center which supports the salaries of Doshi, Bourgeois, Hong, Jefferson, Jones, Shamseer (until 2018) and Spence. In addition, PD has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the Laura and John Arnold Foundation (2017–2021), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014–2016), Cochrane Methods Innovations Fund (2016–2018) and UK National Institute for Health Research (2011–2014); and is an editor at the BMJ and unpaid member of the Reagan-Udall Foundation for the FDA. TJ was a recipient of a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza. In addition, TJ receives royalties from his books published by Il Pensiero Scientifico Editore, Rome and Blackwells. TJ is occasionally interviewed by market research companies about phase I or II pharmaceutical products. In 2011–2013, TJ acted as an expert witness in litigation related to the antiviral oseltamivir, in two litigation cases on potential vaccine-related damage (including the vaccine Pandemrix (2015–2017)) and in a labour case on influenza vaccines in healthcare workers in Canada. He has acted as a consultant for Roche (1997–1999), GSK (2001–2002), Sanofi-Synthelabo (2003) and IMS Health (2013). In 2014, he was retained as a scientific adviser to a legal team acting on oseltamivir. TJ has a potential financial conflict of interest in the drug oseltamivir. In 2014–2016, TJ was a member of three advisory boards for Boehringer Ingelheim. TJ was holder of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ was a member of an independent data monitoring committee for a Sanofi Pasteur clinical trial on an influenza vaccine. Between 1994 and 2013, TJ was the coordinator of the Cochrane Vaccines Field. TJ was a cosignatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of HPV vaccines and consequent complaints to the European Ombudsman. TJ is coholder of a John and Laura Arnold Foundation grant for development of a RIAT Support Center (2017–2020) and Jean Monnet Network Grant, 2017–2020 for the Jean Monnet Health Law and Policy Network. TJ is an unpaid collaborator to the project Beyond Transparency in Pharmaceutical Research and Regulation led by Dalhousie University and funded by the Canadian Institutes of Health Research (2018–2022). TJ is a consultant to Illumina LLC (2019-current). MJ was a coinvestigator on a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza; was a corecipient of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews; and is a paid consultant on a John and Laura Arnold Foundation grant for development of an RIAT Support Center (2017–2020). LS, HL, FB and KH: no competing interests to declare. OS received the Maryland CERSI Scholar award from the Food and Drug Administration (grant #1U01FD005946) and the PhRMA Foundation’s Predoctoral Fellowship in Health Outcomes.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: Our completed extraction sheets are publicly available on the Open Science Framework (http://doi.org/10.17605/OSF.IO/KPE2T) and CSRs are available upon request.
References
- 1. CDC HPV vaccination is safe and effective [Internet], 2019. Available: https://www.cdc.gov/hpv/parents/vaccinesafety.html [Accessed 12 Dec 2019].
- 2. World Health Organization Human papillomavirus vaccines: WHO position paper, May 2017. Wkly Epidemiol Rec 2017;92:241–68. [PubMed] [Google Scholar]
- 3. World Health Organization Ten threats to global health in 2019 [Internet], 2019. Available: https://www.who.int/emergencies/ten-threats-to-global-health-in-2019 [Accessed 12 Dec 2019].
- 4. FUTURE II Study Group Quadrivalent vaccine against human papillomavirus to prevent high-grade cervical lesions. N Engl J Med 2007;356:1915–27. 10.1056/NEJMoa061741 [DOI] [PubMed] [Google Scholar]
- 5. Muñoz N, Manalastas R, Pitisuttithum P, et al. . Safety, immunogenicity, and efficacy of quadrivalent human papillomavirus (types 6, 11, 16, 18) recombinant vaccine in women aged 24-45 years: a randomised, double-blind trial. Lancet 2009;373:1949–57. 10.1016/S0140-6736(09)60691-7 [DOI] [PubMed] [Google Scholar]
- 6. Giuliano AR, Palefsky JM, Goldstone S, et al. . Efficacy of quadrivalent HPV vaccine against HPV infection and disease in males. N Engl J Med 2011;364:401–11. 10.1056/NEJMoa0909537 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Garland SM, Steben M, Hernandez-Avila M, et al. . Noninferiority of antibody response to human papillomavirus type 16 in subjects vaccinated with monovalent and quadrivalent L1 virus-like particle vaccines. Clin Vaccine Immunol 2007;14:792–5. 10.1128/CVI.00478-06 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Garland SM, Hernandez-Avila M, Wheeler CM, et al. . Quadrivalent vaccine against human papillomavirus to prevent anogenital diseases. N Engl J Med 2007;356:1928–43. 10.1056/NEJMoa061760 [DOI] [PubMed] [Google Scholar]
- 9. Rid A, Saxena A, Baqui AH, et al. . Placebo use in vaccine trials: recommendations of a WHO expert panel. Vaccine 2014;32:4708–12. 10.1016/j.vaccine.2014.04.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. European Medicines Agency Guideline on clinical evaluation of vaccines (2 EMEA/CHMP/VWP/164653/05 Rev. 1) [Internet], 2018. Available: https://www.ema.europa.eu/en/documents/scientific-guideline/draft-guideline-clinical-evaluation-vaccines-revision-1_en.pdf [Accessed 24 Aug 2019].
- 11. Merck Frequently asked questions [Internet]. Available: http://www.merck.com/clinical-trials/frequently-asked-questions.html [Accessed 24 Aug 2017].
- 12. National Library of Medicine, ClinicalTrials.gov . Glossary of Common Site Terms [Internet], 2018. Available: https://clinicaltrials.gov/ct2/about-studies/glossary [Accessed 29 Nov 2018].
- 13. Doshi P, Jefferson T, Jones M, et al. . Call to action: RIAT restoration of a previously unpublished methodology in Gardasil vaccine trials, 2019. Available: https://www.bmj.com/content/346/bmj.f2865/rr-7
- 14. Doshi P, Bourgeois F, Hong K, et al. . Additional trials within scope: a follow-up to our “Call to action: RIAT restoration of a previously unpublished methodology in Gardasil vaccine trials”, 2019. Available: https://www.bmj.com/content/346/bmj.f2865/rr-9
- 15. Doshi P, Dickersin K, Healy D, et al. . Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865 10.1136/bmj.f2865 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Treasure T, Monson K, Fiorentino F, et al. . The CEA second-look trial: a randomised controlled trial of carcinoembryonic antigen prompted reoperation for recurrent colorectal cancer. BMJ Open 2014;4:e004385 10.1136/bmjopen-2013-004385 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Le Noury J, Nardo JM, Healy D, et al. . Restoring study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ 2015;351:h4320 10.1136/bmj.h4320 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Le Noury J, Nardo JM, Healy D, et al. . Study 329 continuation phase: safety and efficacy of paroxetine and imipramine in extended treatment of adolescent major depression. Int J Risk Saf Med 2016;28:143–61. 10.3233/JRS-160728 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Zhang R, Persaud N. 8-Way randomized controlled trial of Doxylamine, pyridoxine and dicyclomine for nausea and vomiting during pregnancy: restoration of unpublished information. PLoS One 2017;12:e0167609 10.1371/journal.pone.0167609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Riva C, Tinari S, Spinosa J-P. Cochrane review on HPV vaccines: Concerns over methodological flaws in the [Internet], 2018. Available: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009069.pub3/detailed-comment/en?messageId=154255807 [Accessed 10 Dec 2019].
- 21. Shamseer L, Jefferson T, Jones M. Protocol [Internet]. Open Science Framework, 2019. Available: 10.17605/OSF.IO/234KW [DOI]
- 22. European Medicines Agency European Medicines Agency policy on access to documents (related to medicinal products for human and veterinary use) POLICY/0043 [Internet], 2018. Available: https://www.ema.europa.eu/en/documents/other/policy/0043-european-medicines-agency-policy-access-documents_en.pdf [Accessed 23 Jun 2019].
- 23. ICH International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Structure and Content of Clinical Study Reports (ICH E3) [Internet], 1995. Available: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E3/E3_Guideline.pdf
- 24. Doshi P, Jefferson T. Clinical study reports of randomised controlled trials: an exploratory review of previously Confidential industry reports. BMJ Open 2013;3 10.1136/bmjopen-2012-002496. [Epub ahead of print: 26 Feb 2013]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. European Medicines Agency European medicines Agency policy on publication of clinical data for medicinal products for human use (POLICY/0070; EMA/240810/2013) 2014.
- 26. European Medicines Agency Online access to clinical data for medicinal products for human use [Internet]. European Medicines Agency Clinical Data, 2017. Available: https://clinicaldata.ema.europa.eu/ [Accessed 6 Jul 2017].
- 27. European Medicines Agency Background to clinical data publication policy [Internet], 2018. Available: http://www.ema.europa.eu/ema/index.jsp?curl=pages/special_topics/general/general_content_000556.jsp&mid=WC0b01ac0580614159 [Accessed 4 Feb 2018].
- 28. Lexchin J, Herder M, Doshi P. Canada finally opens up data on new drugs and devices. BMJ 2019;365:l1825 10.1136/bmj.l1825 [DOI] [PubMed] [Google Scholar]
- 29. Jørgensen L, Gøtzsche PC, Jefferson T. Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review. Syst Rev 2018;7:8 10.1186/s13643-018-0675-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Jørgensen L, Doshi P, Gøtzsche P, et al. . Challenges of independent assessment of potential harms of HPV vaccines. BMJ 2018;362:k3694 10.1136/bmj.k3694 [DOI] [PubMed] [Google Scholar]
- 31. Beecher HK. The powerful placebo. J Am Med Assoc 1955;159:1602–6. 10.1001/jama.1955.02960340022006 [DOI] [PubMed] [Google Scholar]
- 32. Gardasil [package insert]. Whitehouse Station, NJ: Merck & Co [Internet], 2006. Available: http://web.archive.org/web/20060614111131/http://www.merck.com/product/usa/pi_circulars/g/gardasil/gardasil_pi.pdf [Accessed 19 Jun 2019].
- 33. Gardasil [package insert]. Whitehouse Station, NJ: Merck & Co [Internet], 2007. Available: http://web.archive.org/web/20080910144620/http://www.merck.com/product/usa/pi_circulars/g/gardasil/gardasil_pi.pdf [Accessed 19 Jun 2019].
- 34. Gardasil [package insert]. Whitehouse Station, NJ: Merck & Co [Internet], 2008. Available: http://web.archive.org/web/20080916044911/http://www.merck.com/product/usa/pi_circulars/g/gardasil/gardasil_pi.pdf [Accessed 19 Jun 2019].
- 35. Golomb BA, Erickson LC, Koperski S, et al. . What's in placebos: who knows? analysis of randomized, controlled trials. Ann Intern Med 2010;153:532–5. 10.7326/0003-4819-153-8-201010190-00010 [DOI] [PubMed] [Google Scholar]
- 36. Gruber MF, Marshall VB. 79. Regulation and Testing of Vaccines : Plotkin S, Orenstein W, Offit P, Plotkin’s Vaccines. Elsevier, 2017: 1547–65. [Google Scholar]
- 37. MSD. 2.5 臨床に関する概括評価 [2.5 Clinical Summary] [Internet]. Available: http://www.pmda.go.jp/drugs/2011/P201100122/170050000_22300AMX00600000_G100_1.pdf [Accessed 29 Nov 2018].
- 38. Miller N. Clinical Review of Biologics License Application for Human Papillomavirus 6, 11, 16, 18 L1 Virus Like Particle Vaccine (S. cerevisiae) (STN 125126 GARDASIL), manufactured by Merck, Inc [Internet], 2006. Available: http://wayback.archive-it.org/7993/20170723091811/https://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM111287.pdf [Accessed 29 Nov 2018].
- 39. Takeuchi J, Noma H, Sakanishi Y, et al. . Adverse events associated with human papillomavirus vaccines: a protocol for systematic review with network meta-analysis incorporating all randomised controlled trials comparing with placebo, adjuvants and other vaccines. BMJ Open 2019;9:e026924 10.1136/bmjopen-2018-026924 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Garçon N, Friede M. 6. Evolution of Adjuvants Across the Centuries : Plotkin S, Orenstein W, Offit P, Plotkin’s Vaccines. Elsevier, 2017: 61–74. [Google Scholar]
- 41. Djurisic S, Jakobsen JC, Petersen SB, et al. . Aluminium adjuvants used in vaccines versus placebo or no intervention. Cochrane Database Syst Rev 2017;347:f5906 10.1002/14651858.CD012805 [DOI] [Google Scholar]
- 42. Moher D, Hopewell S, Schulz KF, et al. . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869 10.1136/bmj.c869 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Food and Drug Administration Guidance for Industry: E10 Choice of Control Group and Related Issues in Clinical Trials [Internet], 2001. Available: https://www.fda.gov/media/71349/download
- 44. Koutsky LA, Ault KA, Wheeler CM, et al. . A controlled trial of a human papillomavirus type 16 vaccine. N Engl J Med 2002;347:1645–51. 10.1056/NEJMoa020586 [DOI] [PubMed] [Google Scholar]