Aluminium adjuvants used in vaccines versus placebo or no intervention

Snezana Djurisic; Janus C Jakobsen; Sesilje B Petersen; Mette Kenfelt; Christian Gluud

doi:10.1002/14651858.CD012805

. 2017 Sep 24;2017(9):CD012805. doi: 10.1002/14651858.CD012805

Aluminium adjuvants used in vaccines versus placebo or no intervention

Snezana Djurisic ¹, Janus C Jakobsen ², Sesilje B Petersen ³, Mette Kenfelt ⁴, Christian Gluud ^2,^✉

PMCID: PMC6483624

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To assess the benefits and harms of aluminium adjuvants used in vaccines versus placebo or no intervention, taking into consideration the type of the vaccine, and the type, size, and concentration of the aluminium adjuvant.

Background

Description of the condition

The effectiveness of vaccinations has been proven repeatedly since the first introduction of the cowpox vaccine in the 18^th century (Delany 2014; Whitney 2014). In fact, vaccination is considered one of the major triumphs of modern medicine (Delany 2014; Whitney 2014). Vaccination prevents infectious diseases (Delany 2014; Whitney 2014), and the worldwide eradication of the highly contagious and deadly smallpox and the restriction of diseases such as polio, measles, and tetanus can largely be ascribed to the numerous successful mass vaccination programmes launched since the 1960s (Delany 2014; Whitney 2014). In the late 1940s and 1950s, prior to public vaccinations, the poliovirus caused infantile paralysis associated with high mortality in hundreds of thousands of children (Global Polio Eradication Initiative 2016; WHO 2016a). The measles virus was responsible for post‐infectious encephalomyelitis in 1 per 1000 infected individuals, leaving most with permanent impairment of the central nervous system (Miller 1964; CDC 2016). Today, polio is nearly eradicated (WHO 2016a), and global measles death has decreased by 79%, with an estimated 17.1 million deaths prevented from 2000 to 2014 (WHO 2016b).

Current routine vaccine programmes recommended by the World Health Organization (WHO) include vaccines against Bacillus Calmette–Guérin (BCG); hepatitis; polio; diphtheria, tetanus, and pertussis (DTP); haemophilus influenza type B; pneumococcal bacteria; rotavirus; measles; rubella; and human papilloma virus (HPV) (WHO 2017). Additional programmes are proposed for certain regions, high‐risk populations, and for fighting pathogens with certain characteristics (e.g. Bacillus anthracis causing anthrax).

Vaccination mimics infection in the body leading to activation of a potent immune response (Coffman 2010; Kool 2012; O'Hagan 2012; Oleszycka 2014). The vaccines are two types: traditional vaccines which contain vaccine antigens and excipients (substance formulated alongside the active vaccine ingredient) and do not use adjuvants (substance that aid the immune response to an antigen); and vaccines of a newer generation which typically require the addition of adjuvants to the vaccine antigens and the excipient (Coffman 2010; O'Hagan 2012). Vaccine antigens may comprise whole attenuated pathogens, pathogen components, virus‐like particles, or genetic material of the pathogen (Strugnell 2011).

Like other medicinal products, vaccines undergo preclinical testing for safety, for the ability to induce cancer, for the ability to induce immune response, and for overall efficacy before they are licensed. However, rare adverse events or adverse events with delayed onset are not easily detected during the relatively short duration of most preclinical and clinical phase studies, and as proven over the years, safety surveillance in the general population post marketing is essential (Ward 2000; Chen 2005). As an example, the childhood measles‐mumps‐rubella (MMR) vaccine was introduced in the late 1960s as a mixture of three live attenuated viruses, administered via injection (Offit 2007). Over time, doubts about its safety were raised when serious cases of fever seizures, meningitis, and allergic reactions were reported (Kimura 1996; Dourado 2000; Ward 2000). In Japan, a nationwide surveillance programme launched in the early 1990s screened more than 38,000 children vaccinated with four different Urabe‐containing MMR vaccines (Kimura 1996). Serious adverse events included convulsions and aseptic meningitis, and the incidence was shown to be linked to different vaccine strains of mumps virus (Kimura 1996). During the same time period, Brazil experienced a mass outbreak of aseptic meningitis following a Urabe‐containing MMR vaccine with an estimated risk of 1 in 14,000 doses (Dourado 2000). As another example, the DTP vaccine was licensed in the late 1940s as a preparation of three different antigen components (trivalent vaccine) adsorbed to aluminium salt. It was suspected to cause acute encephalopathy and chronic nervous system dysfunction (Cowan 1993). Reports, prepared by the Institute of Medicine (IOM) in the US, concluded that the evidence was insufficient to indicate a causal relationship between the DTP vaccine and acute or permanent neurologic damage (Cowan 1993). In 2000, the WHO published a critical bulletin on vaccine safety, including an overview of serious vaccine‐associated adverse events for which causality had been established or was highly likely (Ward 2000). Among several vaccine‐specific serious adverse events, they found a causal relationship between vaccines against DTP, hepatitis, MMR, and polio and disease dissemination, severe allergic reactions, or death (Ward 2000).

Vaccine toxicity may originate from a plethora of factors, including the vaccine components (e.g. the antigen itself, the adjuvant, or the excipients), interaction between different vaccine components, vaccine manufacture, overall vaccine composition, route of administration, dose, and number of vaccinations (Kocourkova 2016).

One of the latest programmes added to the mass vaccination portfolio is the public HPV vaccination programme launched in the US a decade ago (WHO 2014a). Human papilloma virus causes cervical cancer; the second most common cancer form in women (WHO 2014a), and thus, the aim of the HPV vaccination is to prevent development of cervical cancer. More than 60 countries included the HPV vaccine in their routine vaccinations after its market approval (Bruni 2016). In recent years, concerns were raised about adverse events possibly related to the HPV vaccines. Since the US approval of Gardasil® in 2006 and up until 2012, a total of 21,265 adverse events was reported to the national Vaccine Adverse Event Reporting System (VAERS) (Tomljenovic 2012). HPV vaccines have been blamed for giving rise to more reported adverse events than other types of vaccines (Tomljenovic 2012). In the European Union, the European Medical Agency (EMA) has received similar reports, but found no scientific evidence for an association (EMA 2015). Several observational studies also failed to identify associations with clinical diagnoses (Klein 2012; Arnheim‐Dahlstrom 2013; Donegan 2013; Grimaldi‐Bensouda 2014; Scheller 2015), but reasons to oppose these findings have been put forward (Brinth 2015a; Brinth 2015b; Dyer 2015; Gøtzsche 2016; Gøtzsche 2016a). The symptoms reported following HPV vaccination are varied and include headache, orthostatic intolerance, fatigue, cognitive dysfunction, blurred vision, feeling bloated, abdominal pain, light sensitivity, and involuntary muscle activity (Brinth 2015a; Brinth 2015b). Despite the consistency in reported symptoms, they do not fit into a well‐defined category of diseases or diagnoses, but rather present themselves as a constellation of nonspecific symptoms (Brinth 2015a; Brinth 2015b). Consequently, the observational studies, which based their results on registered diagnoses, may have excluded an important fraction of eligible participants with unclear adverse symptoms, as most young girls that claim to suffer from adverse events following HPV vaccination receive no clinical diagnosis, and are therefore unlikely to appear in medical registers. Moreover, the randomised clinical trials on HPV vaccines, which formed the basis for the safety assessment, have been blamed for not using true placebo (e.g. placebo not containing adjuvants) as the control intervention (Exley 2011).

Description of the intervention

Adjuvants are added to vaccines to enhance the ability to provoke an immune response of weak antigens and improve the overall potency of the vaccine (O'Hagan 2009; Coffman 2010). Adjuvants may also pose other benefits, such as reducing the frequency of vaccination and the dose of antigen per vaccine, and some may provide cross‐clade immunity (i.e. immunity against different clades of viruses or different clades of bacteria descending from different ancestors) or improve the stability of the final vaccine formulation (Carter 2010; Reed 2013). Currently, five adjuvants with completely different mechanism of action are approved for use in vaccine production. These include: aluminium salts (EU, US), MF59 (EU), AS03 (EU), AS04 (EU, US), and virosomes (EU) (Rambe 2015). Virosomes consist of a lipid membrane incorporating virus‐derived proteins, while MF59 and AS03 are squalene‐based adjuvants and AS04 combines aluminium hydroxide with monophosphoryl lipid A. Aluminium salts, also referred to as alum, are most commonly used and encompass aluminium potassium sulphate, aluminium hydroxide, aluminium phosphate, and amorphous aluminium hydroxyphosphate sulfate (Carter 2010).

Different insoluble aluminium salts have been used as vaccine adjuvants since 1926 (Glenny 1926). Aluminium potassium sulphate was the first used. However, because of poor reproducibility, it has been almost completely replaced by aluminium hydroxide and aluminium phosphate, as they can be prepared in a more standardised way, and capture antigens by direct adsorption (Marrack 2009). Aluminium has been the standard adjuvant in vaccines such as those against diphtheria, tetanus, and pertussis, haemophilus influenza type B, pneumococcus conjugates, hepatitis A, and hepatitis B (Tritto 2009). More recently, it was co‐formulated with vaccines against HPV in the form of AS04 (containing aluminium hydroxide) and amorphous aluminium hydroxyphosphate sulfate. Amorphous aluminium hydroxyphosphate sulfate is commercially produced in nanometre‐scale, and represents one of the latest marketed aluminium adjuvants.

The mechanism of action of aluminium, like for most adjuvants, is poorly understood, and widespread beliefs change according to continuously new insights into immunology and physiochemical properties of aluminium (see How the intervention might work) (Carter 2010; Tomljenovic 2011). Despite our incomplete understanding of its effects, the repeated use of aluminium in vaccines is justified by its apparent safety profile, ease of preparation, stability, potent immunostimulatory ability (O'Hagan 2009; Tritto 2009; Mbow 2010), and importantly, due to the lack of suitable alternatives.

How the intervention might work

Aluminium is the third most abundant element in the earth’s crust, but the metal has no known biological or physiological role (Reinke 2003). It is absorbed into the blood through the gastrointestinal tract, and rapidly eliminated by the kidneys and bile (Reinke 2003). While aluminium is considered safe and regularly ingested with food and water, it is toxic in high concentrations (Kisnieriené 2015). The toxicity, however, not only depends on the concentration, but on the chemical form and the environment as well (Kisnieriené 2015). In the blood, aluminium is bound to transferrin with high affinity, where it competes with iron at the binding site (Kisnieriené 2015). Aluminium also affects cellular processes and physiological functions (Kisnieriené 2015). For instance, aluminium competes with magnesium for membrane transporters; disturbs calcium metabolism; increases oxidative stress; binds to the phosphate groups of nucleoside di‐ and triphosphates; and also binds to metal‐binding organic compounds (amino acids) and membrane lipids (Kisnieriené 2015). In high concentrations, aluminium is predominantly accumulated in bone and brain tissue (Yokel 2000; Malluche 2002). In animal and human studies, it has been shown to act as a powerful neurological toxicant and provoke toxic effects in foetuses and embryos if exposed during pregnancy (Reinke 2003). This is supported by recent data indicating that aluminium is able to cross the blood‐brain barrier by directly affecting the cerebral blood vessels (Chen 2008; Sharma 2010).

Despite its unclear biological role, aluminium seems to have an impact on the immune system, which has rendered it useful as a vaccine adjuvant (Tritto 2009; Kool 2012). Aluminium binds antigens with high affinity (antigen adsorption) and was originally thought to exert its function by forming a depot, which allows for a high antigen concentration at the site of injection, and a continuous desorption and dispersion of antigens from the aluminium particles (Kool 2012). Nowadays, aluminium is believed to exert its adjuvantic effects by stimulating Th2‐type responses and antibody production through B cells activation (Grun 1989; Awate 2013), by activating the complement system, and by recruiting immune cells to the site of injection (Ramanathan 1979; Goto 1997; Awate 2013). At the injection site, aluminium promotes antigen uptake by specialised antigen‐presenting immune cells, termed dendritic cells, and dendritic cell maturation (Mannhalter 1985; Morefield 2005; Kool 2008).

One important aspect of adjuvants is the size of the particles, which seems to have a considerable influence on the immune response. Aluminium hydroxide adjuvant is comprised of particles with a calculated dimension of 100 nm, while aluminium phosphate particles are around 50 nm (Hem 2007). In an aqueous (water) solution, particles of both aluminium salts aggregate to form 1 to 20 µm sized particulates (Hem 2007). This size is also known as microscale‐size. Aluminium hydroxide and aluminium phosphate can be produced in nanoscale‐size ≤ 200 nm, but so far, only amorphous (non‐crystalline solid) aluminium hydroxyphosphate sulfate is produced in nanoscale for use in vaccine preparations (Issa 2014; Li 2014). The particle size is directly linked to the adsorption efficiency of antigens (Oyewumi 2010). In line with this, recent evidence shows that nanoscale aluminium particles can adsorb more antigens compared to traditional aluminium‐based adjuvants because of the higher surface‐area‐to‐volume ratio, and that they are more potent than traditional micro‐particles (Caulfield 2007; Salvador 2011; Li 2014). Moreover, the efficacy of particle uptake by the specialised antigen presenting dendritic cells in vitro and in vivo is inversely proportional to the size of the particle with maximum efficiency for nanoscale particles < 100 nm (Foged 2005; Shima 2013). Dendritic cells scavenger and engulf particles of < 10 µm, having evolved to recognise pathogens of this size (Gupta 1995; Foged 2005). Other factors like structure, shape, and surface charge have also been demonstrated to greatly affect uptake by dendritic cells (Thiele 2001; Foged 2005; Bartneck 2012; Son 2013).

Why it is important to do this review

One previous attempt to assess the potential toxic effects of aluminium adjuvant with a systematic review was undertaken in 2004 by Jefferson and colleagues (Jefferson 2004). The systematic review covered existing evidence of adverse events after exposure to the aluminium‐containing DTP vaccine, but it did not assess benefits (Jefferson 2004). The authors included three randomised trials, four semi‐randomised trials, and one cohort study, and they were unable to demonstrate that aluminium adjuvant was responsible for any serious or long‐lasting adverse events (Jefferson 2004). The authors advised the ending of future research despite concluding that their finding was based on poor‐quality evidence (Jefferson 2004).

More than 10 years has passed since the systematic review by Jefferson and colleagues, new adjuvants are being introduced continuously, and FDA and WHO do not require genotoxicity or cardiotoxicity studies of new aluminium adjuvants (WHO 2014b; FDA 2015). Lately, symptoms following HPV vaccination have been suspected of being caused by the addition of aluminium adjuvant (Tomljenovic 2011; Lee 2012; Poddighe 2014; Brinth 2015b; Gruber 2015; Martinez‐Lavin 2015). A recent animal study by Inbar and colleagues managed to spark further controversy by demonstrating behavioral abnormalities in mice administered the aluminium‐containing HPV vaccine Gardasil (Inbar 2016a). Compared to previous animal studies on HPV vaccines, the authors included two control groups: one where mice were administered aluminium adjuvant alone and another with placebo without adjuvant (Inbar 2016a). Inbar and colleagues concluded that Gardasil via both its aluminium adjuvant and HPV antigens can trigger neuro‐inflammation and autoimmune reactions, leading to behavioural changes in mice (Inbar 2016a). Upon submission to a peer‐reviewed journal, the paper was accepted with revisions, and published. However, it was soon withdrawn by the editor (Inbar 2016), only to be published in a competing journal shortly thereafter (Inbar 2016a). The initial withdrawal was allegedly due to "unsound scientific results"; an assertion which was not supported by the final publisher. The theory that aluminium adjuvant is responsible for symptoms following HPV vaccination is impossible to refute or prove based on the current data. Aluminium adjuvant has been administered to both experimental and control group in the vast majority of randomised clinical trials on HPV vaccines, thus masking its potentially harmful effects (Exley 2011). Clinical trials designed to administer vaccine adjuvants to the experimental group as well as the placebo group do, de facto, not compare an intervention against a true placebo, and therefore, do not adequately assess safety (Exley 2011). Indeed, aluminium adjuvants, new or old, should be evaluated for benefits and harms on their own merits.

Aluminium is the most frequently used adjuvant, introduced in vaccination programmes worldwide (Tritto 2009). While the consequences of adding aluminium to vaccines have been discussed broadly, no systematic review has been conducted to assess the effects of aluminium adjuvants across vaccines. The effects of aluminium adjuvants remain to be properly assessed using Cochrane methodology to determine whether they are beneficial, or causally linked to the numerous adverse events reported following immunisation.

Objectives

Methods

Criteria for considering studies for this review

Types of studies

We will search for randomised clinical trials irrespective of publication type, publication status, and language of publication. We will not specifically search for observational studies (quasi‐randomised studies; cohort studies; and patient series), but we may provide a narrative account of such data if we identify valid observational studies during our literature search. We are aware that this approach may be a weakness of our review, making us focus more on short‐term benefits and harms in randomised clinical trials with the risk of overlooking late and very rare adverse effects in observational studies.

Types of participants

We will include all trial participants regardless of sex, age, ethnicity, diagnosis, comorbidity, and country of residence.

Types of interventions

We plan to include trials comparing:

any type of vaccine including any type of aluminium adjuvant (including, but not limited to, aluminium potassium sulphate; aluminium hydroxide; aluminium phosphate; or aluminium hydroxyphosphate sulfate) versus the same vaccine but without the aluminium adjuvant;

any aluminium adjuvant versus placebo or no intervention.

We will accept any co‐intervention if planned to be delivered equally to both intervention groups.

Types of outcome measures

Primary outcomes

Proportion of participants with one or more serious adverse events. We will define a serious adverse event as any untoward medical occurrence that results in death, is life‐threatening, requires hospitalisation or prolongation of existing hospitalisation, or results in persistent or significant disability or incapacity (ICH‐GCP 1997).
All‐cause mortality (as reported by trialists or measured by administrative data)
Proportion of participants with disease (as defined per individual trial)

We will use the trial results reported at maximum follow‐up. If the trialists report results at multiple time points, we will also assess the results reported at the time point closest to three years.

Secondary outcomes

Health‐related quality of life (as measured by interviews or self‐report using any standardised continuous scale)
Non‐serious adverse events (defined as any adverse event not classified as a serious adverse event). We will analyse each adverse event separately.

Exploratory outcomes

Serological response (as defined by trialists, e.g. measured with ELISA, agglutination, precipitation, complement‐fixation, fluorescent antibodies, chemiluminescence, or similar)

We will use the trial results reported at maximum follow‐up to achieve maximum precision and power.

Search methods for identification of studies

Electronic searches

We will search the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, MEDLINE Ovid, Embase Ovid, BIOSIS (Web of Science), LILACS (Bireme), Science Citation Index Expanded (Web of Science), and Conference Proceedings Citation Index – Science (Web of Science) (Royle 2003). Appendix 1 gives the preliminary search strategies with the expected time spans of the searches.

In addition, we will search the Chinese Biomedical Literature Database (CBM), China Network Knowledge Information (CNKI), Chinese Science Journal Database (VIP), and Wanfang Database.

Searching other resources

We will also search Google Scholar, The Turning Research into Practice (TRIP) Database, ClinicalTrial.gov (www.clinicaltrials.gov/), European Medicines Agency (EMA) (www.ema.europa.eu/ema/), WHO International Clinical Trial Registry Platform (www.who.int/ictrp), The Food and Drug Administration (FDA) (www.fda.gov), and pharmaceutical company sources for ongoing or unpublished trials.

We will review bibliographic references of identified randomised clinical trials and review articles to identify randomised clinical trials missed during the electronic searches.

We will consider unpublished and grey literature trials, if identified.

Data collection and analysis

We will perform the review following the recommendations of Cochrane (Higgins 2011a). We will perform the analyses using Review Manager 5 (RevMan 2014), STATA 14 (STATA 14), and Trial Sequential Analysis version 0.9.5.10 Beta (Thorlund 2011; TSA 2011). We will present a table describing the types of adverse events (serious or non‐serious) reported in each trial.

Selection of studies

Two review authors (SD and SBP) will independently screen titles and abstracts for inclusion of potentially eligible trials. We will code included studies as 'retrieve' (eligible or potentially eligible/unclear) or 'do not retrieve'. Following any disagreements, we will ask a third author to arbitrate (JCJ or CG). The selected review author pair will collect full‐text trial reports/publications, and independently screen the full‐texts and identify trials for inclusion. We will report reasons for exclusion of the ineligible studies. We will solve any disagreement through discussion, or, if required, by consulting a third person (JCJ or CG). We will identify and exclude duplicates, and collate multiple reports of the same trial. We will record the selection process in sufficient detail to complete a PRISMA flow diagram and 'Characteristics of excluded studies' table.

Data extraction and management

The review authors, working in pairs, will independently extract and validate data using data extraction forms designed for the purpose. If a trial is identified as relevant by one author but not by the other, the authors will discuss the reasoning behind their assessment. If an agreement is not reached between the two authors, JCJ or CG will serve as arbitrators.

Assessment of risk of bias in included studies

Methodological studies indicate that trials with unclear or inadequate methodological quality may be associated with risk of bias (systematic error) when compared to trials using adequate methodology (Schulz 1995; Moher 1998; Kjaergard 2001; Gluud 2006; Wood 2008; Higgins 2011a; Hrobjartsson 2012; Savović 2012; Savović 2012a; Hrobjartsson 2013; Hrobjartsson 2014; Lundh 2017). Such bias may lead to overestimation of intervention benefits and underestimation of harms.

The selected review author pair (SD and SBP) will independently assess the risk of bias of each included trial according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We will use the below definitions in the assessment of bias risk (Schulz 1995; Moher 1998; Kjaergard 2001; Gluud 2006; Wood 2008; Higgins 2011a; Hrobjartsson 2012; Savović 2012; Savović 2012a; Hrobjartsson 2013; Hrobjartsson 2014; Lundh 2017).

Allocation sequence generation

Low risk of bias: sequence generation was achieved using computer random number generation or a random number table. Drawing lots, tossing a coin, shuffling cards, and throwing dice were adequate if performed by an independent person not otherwise involved in the trial.
Unclear risk of bias: the method of sequence generation was not specified.
High risk of bias: the sequence generation method was not random or only quasi‐randomised. We will only use these studies for the assessment of harms and not of benefits.

Allocation concealment

Low risk of bias: the allocation sequence was described as unknown to the investigators. Hence, the participants' allocations could not have been foreseen in advance of, or during, enrolment. Allocation was controlled by a central and independent randomisation unit, an on‐site locked computer, identical looking numbered sealed opaque envelopes, drug bottles or containers prepared by an independent pharmacist, or an independent investigator.
Unclear risk of bias: it was unclear if the allocation was hidden or if the block size was relatively small and fixed so that intervention allocations may have been foreseen in advance of, or during, enrolment.
High risk of bias: the allocation sequence was likely to be known to the investigators who assigned the participants.

Blinding of participants and treatment providers

Low risk of bias: it was described that both participants and treatment providers were blinded to treatment allocation.
Unclear risk of bias: it was unclear if participants and treatment providers were blinded, or the extent of blinding was insufficiently described.
High risk of bias: no blinding or incomplete blinding of participants and treatment providers was performed.

Blinding of outcome assessment

Low risk of bias: it was mentioned that outcome assessors were blinded and this was described.
Unclear risk of bias: it was not mentioned if the outcome assessors were blinded, or the extent of blinding was insufficiently described.
High risk of bias: no blinding or incomplete blinding of outcome assessors was performed.

Incomplete outcome data

Low risk of bias: missing data were unlikely to make intervention effects depart from plausible values. This could either be: 1) there were no dropouts or withdrawals; or 2) the numbers and reasons for the withdrawals and dropouts for all outcomes were clearly stated and could be described as being similar in both groups, and the trial handled missing data appropriately in an intention‐to‐treat analysis using proper methods (e.g. multiple imputations). Generally, the trial is judged to be at a low risk of bias due to incomplete outcome data if dropouts are less than 5%. However, the 5% cut‐off is not definitive.
Unclear risk of bias: there was insufficient information to assess whether missing data were likely to induce bias on the results.
High risk of bias: the results were likely to be biased due to missing data either because the pattern of dropouts could be described as being different in the two intervention groups or the trial used improper methods in dealing with the missing data (e.g. last observation carried forward).

Selective outcome reporting

Low risk of bias: a protocol was published before randomisation began and all outcome results were reported adequately.
Unclear risk of bias: no protocol was published.
High risk of bias: the outcomes in the protocol were not reported on.

Vested interest bias

Low risk of bias: it was described that the trial was not sponsored by any pharmaceutical company, any person, or any group with a financial or other interest in a certain result of the trial.
Unclear risk of bias: it was unclear how the trial was sponsored.
High risk of bias: the trial was sponsored by a pharmaceutical company, a person, or a group with a certain financial or other interest in a given result of the trial.

Other bias

Low risk of bias: the trial appeared to be free of other bias domains that could put it at risk of bias.
Unclear risk of bias: the trial may or may not have been free of other domains that could put it at risk of bias.
High risk of bias: there were other factors in the trial that could put it at risk of bias.

Overall risk of bias

Low risk of bias: the outcome result will be classified as at overall 'low risk of bias' only if all of the bias domains described in the above paragraphs are classified as at low risk of bias.
High risk of bias: the outcome result will be classified as at 'high risk of bias' if any of the bias risk domains described above are classified as at 'unclear' or 'high risk of bias'.

We will assess the domains 'Blinding of outcome assessment', 'Incomplete outcome data', and 'Selective outcome reporting' for each outcome. This will enable us to assess the bias risk for each outcome result in addition to each trial. We will base our primary conclusions as well as our presentation in the 'Summary of Findings' table on the results of our primary outcomes with low risk of bias.

Measures of treatment effect

Dichotomous outcomes

We will calculate risk ratios (RR) with 95% confidence interval (CI) for dichotomous outcomes, as well as the Trial Sequential Analysis‐adjusted CI (see below).

Continuous outcomes

We will calculate the mean difference (MD) with 95% CI and Trial Sequential Analysis‐adjusted CI for continuous outcomes. If various scales assessing comparable symptoms have been used, we will calculate the standardised mean difference (SMD) with 95% CI for continuous outcomes. Such data can then be calculated back to MD for a preferred scale, if needed.

Unit of analysis issues

We will include data from studies where participants are individually randomised to one of two or more intervention groups. We will collect and analyse single measurements for each outcome from each participant.

Dealing with missing data

We will contact investigators or study sponsors to obtain any missing data.

If standard deviations (SD) are not reported, we will calculate them using data from the trial, if possible. We will not impute missing values for any outcomes in our primary analysis. In our sensitivity analysis for dichotomous and continuous outcomes, we will impute data (see Sensitivity analysis).

Assessment of heterogeneity

We will first visually investigate forest plots to assess the risk of statistical heterogeneity. We will also assess the presence of statistical heterogeneity using the Chi² test (threshold P < 0.1) and measure the quantities of heterogeneity using the I² statistic (Higgins 2002; Higgins 2003).

Assessment of reporting biases

We will assess reporting bias using funnel plots where ten or more trials per comparison are included. Symmetry or asymmetry of each funnel plot will enable assessment of the risk of bias. For dichotomous outcomes, we will assess asymmetry using the Harbord test (Harbord 2006). For continuous outcomes, we will apply the regression asymmetry test (Egger 1997).

Data synthesis

Meta‐analysis

We will conduct this systematic review according to the following recommendations stated in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a), according to Keus and colleagues (Keus 2010), and according to the eight‐step procedure for validation of meta‐analytic results in systematic reviews as suggested by Jakobsen and colleagues (Jakobsen 2014). We will meta‐analyse data using the statistical software Review Manager 5.3 (RevMan 2014).

Trial Sequential Analysis

Cumulative meta‐analyses are at risk of producing random errors due to sparse data and multiple testing of accumulating data (Pogue 1997; Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009; Higgins 2011b; Wetterslev 2017); therefore, Trial Sequential Analysis (TSA 2011) can be applied to control this risk (Thorlund 2011). The required information size (that is the number of participants and number of trials needed in a meta‐analysis to detect or reject a certain intervention effect) can be calculated in order to control random errors (Wetterslev 2008; Wetterslev 2009; Wetterslev 2017). The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk (RR) reduction, and the heterogeneity of the meta‐analysis (Wetterslev 2008; Wetterslev 2009; Turner 2013; Wetterslev 2017). Trial Sequential Analysis enables testing for significance to be conducted each time a new trial is included in the meta‐analysis. On the basis of the required information size, trial sequential monitoring boundaries can be constructed. This enables one to determine the statistical inference concerning cumulative meta‐analysis that has not yet reached the required information size (Wetterslev 2008; Wetterslev 2017).

If the trial sequential monitoring boundary is crossed before reaching the calculated information size, we may conclude that sufficient evidence is collected to validly assess benefit or harm, and that inclusion of additional trial data may be redundant. In contrast, if the boundaries for benefit or harm are not crossed, we may conclude that further trials are necessary before a certain intervention effect can be evaluated. Trial Sequential Analysis also allows for assessment of the sufficiency of evidence for a postulated intervention effect. A lack of effect is evident if the cumulative Z‐score crosses the trial sequential monitoring boundaries for "futility" (the ability of a systematic review of clinical trials to reject a certain postulated intervention effect).

We will make relatively conservative estimations of the anticipated intervention effect to control the risks of random error (Jakobsen 2014). Large anticipated intervention effects lead to small required information sizes, and the thresholds for significance will be less strict after the information size has been reached (Jakobsen 2014).

We will analyse all primary and secondary outcomes using Trial Sequential Analysis. These analyses will allow us to calculate the Trial Sequential Analysis‐adjusted CIs based on the following assumptions:

Primary outcomes

We will estimate the diversity‐adjusted required information size (Wetterslev 2009; Wetterslev 2017) based on the proportion of participants with an outcome in the control group. We will use an alpha of 2.5%, a beta of 10%, and the diversity suggested by the trials in the meta‐analysis (Jakobsen 2014).

As anticipated intervention effects for the primary outcomes in the Trial Sequential Analysis, we will use the following:

Serious adverse events: a relative risk reduction of 20% and the observed proportion of serious adverse events in the control group.
All‐cause mortality: a relative risk reduction of 20% and the observed incidence of mortality in the control group.
Disease: a relative risk reduction of 20% and the observed proportion of participants with disease in the control group.

Secondary outcomes

We will estimate the diversity‐adjusted required information size (Wetterslev 2009; Wetterslev 2017) based on the proportion of participants with an outcome in the control group when analysing dichotomous outcomes, and we will use the observed SD when analysing continuous outcomes. We will use an alpha of 2.5%, a beta of 10%, and the diversity suggested by the trials in the meta‐analysis (Jakobsen 2014).

As anticipated intervention effects for the secondary outcomes in the Trial Sequential Analysis we will use the following relative risk reductions or increases:

Quality of life: observed SD divided by 2.
Non‐serious adverse events: a relative risk reduction of 20%.

Exploratory outcomes

As anticipated intervention effects for the exploratory outcomes in the Trial Sequential Analysis, we will use the following relative risk reductions or increases:

Serological response: a relative risk reduction of 20% and the observed proportion of participants with no serological response in the control group.

We will include particle size (nano‐size or micro‐size as described by trialist or manufacturer) as a covariate in meta‐regression to assess whether particle size influences the effect of aluminium adjuvant administration on outcomes.

Assessment of significance

Intervention effects will be assessed with both random‐effects (DerSimonian 1986) and fixed‐effect meta‐analyses (DeMets 1987). The more conservative point estimate of the two, comprised by the estimate closest to zero effect, will be chosen for assessment of significance (Jakobsen 2014). If the two estimates are comparable, the estimate with the widest confidence interval will be used. For analysis of three primary outcomes, a P value less than P < 0.025 will be considered significant (Jakobsen 2014) because this will secure a family‐wise error rate (FWER) below 0.05. An eight‐step procedure will be applied to assess if the results from the meta‐analyses have passed the thresholds for significance (Jakobsen 2014).

A table describing all types of serious adverse events will be presented for each trial.

Subgroup analysis and investigation of heterogeneity

We will perform the following subgroup analyses:

A: The effect of aluminium adjuvant administration in trials with high risk of bias compared to low risk of bias B: The effect of aluminium adjuvant administration between trials where the intervention groups were treated with different aluminium salts (aluminium hydroxide, aluminium phosphate, or amorphous aluminium hydroxyphosphate sulfate) C: The effect of aluminium adjuvant administration between trials where the intervention groups were treated with different vaccines (including, but not limited to, vaccines against BCG, HPV, hepatitis A and B, HIV, anthrax, DTP, polio, influenza, rotavirus, and MMR) D: Comparison of the effect of aluminium adjuvant administration between trials on newborns, children, adolescents, adults or elderly (or similar terms) as described by trialists E: The effects nanoparticle aluminium adjuvant compared to microparticle aluminium adjuvant regardless of type of aluminium salt F: The effects of aluminium adjuvant administration between trials with different maximal follow‐up periods:

short‐term (1‐30 days after last administration);
medium‐term (1‐12 months after last administration); and
long term (more than 1 year after last administration)

G: The effect of total administration of aluminium between trials H: The effect of aluminium adjuvant administration between trials including healthy participants and trials including participants with any specific diagnosis

Sensitivity analysis

A: To assess the potential impact of the missing data for dichotomous outcomes, we will perform the following analyses:

’Best‐worst‐case’ scenario: it will be assumed that all participants lost to follow‐up in the experimental group survived and did not have a serious adverse event; and all those with missing outcomes in the control group had a serious adverse event and did not survive.
’Worst‐best‐case’ scenario: it will be assumed that all participants lost to follow‐up in the experimental group did not survive and had a serious adverse event; and all those with missing outcomes in the control group survived and had no serious adverse event.

We will present results from both scenarios.

B: We will address missing data for continuous outcomes by calculating a 'beneficial' and a 'harmful' outcome. We will base the 'beneficial' outcome on the group mean plus 2 standard deviations (SDs) (and 1 SD), and the ‘harmful’ outcome on the group mean minus 2 SDs (and 1 SD) (Jakobsen 2014).

C: Potential impact of missing SDs for continuous outcomes will be assessed with the following sensitivity analyses:

Where SDs are missing and not possible to calculate, we will be impute SDs from trials with similar populations and low risk of bias. If no such trials can be found, we will impute SDs from trials with a similar population. As a final option, we will impute SDs from all trials.

Summary of findings

We will use the GRADE system (Guyatt 2008) to assess the quality of the body of evidence associated with each of the primary outcomes. We will construct the 'Summary of findings' (SoF) tables using the GRADEpro software (tech.cochrane.org/revman/other‐resources/gradepro/download). This GRADE system appraises the quality of a body of evidence based on the extent to which one can be confident that an estimate of effect or association reflects the item being assessed. The quality measure of a body of evidence considers within‐trial risk of bias, the directness of the evidence, heterogeneity of the data, precision of effect estimates (Jakobsen 2014), and risk of publication bias. Our primary SoF tables and conclusions will be based on the results of trials with a low risk of bias in all bias risk domains (Schulz 1995; Moher 1998; Kjaergard 2001; Gluud 2006; Wood 2008; Savović 2012; Savović 2012a; Lundh 2017).

Feedback

Will this review investigate/consider the impact of the unnaturally high antibodies induced by HPV vaccination?, 15 January 2018

Summary

In a review paper published in 2010, Ian Frazer, a co‐inventor of the technology enabling the HPV vaccines, states: "HPV immunization induces peak geometric mean antibody titers that are 80‐ to 100‐fold higher than those observed following natural infection [19]. Furthermore, after 18 months, mean vaccine‐induced antibody titers remain 10‐ to 16‐fold higher than those recorded with natural infection [19], and these levels appear to be preserved over time, suggesting that immunization may provide long‐term protection against infection..." (See page S9.) HPV 'immunization' inducing antibody titres that are 80‐ to 100‐fold higher than those observed following natural infection seems to be a very unnatural response. Is this a good thing? Does anybody know?

Frazer's review paper entitled "Measuring serum antibody to human papillomavirus following infection or vaccination", is published in Gynecologic Oncology 118 (2010) S8‐S11 and funded by Merck & Co. Inc. His reference for his high antibody titre comment is a paper by Diane M Harper et al "Efficacy of a bivalent L1 virus‐like particle vaccine in prevention of infection with human papillomavirus types 16 and 18 in young women: a randomised controlled trial", published in The Lancet, Vol 364 November 13, 2004, and funded, and co‐ordinated by GlaxoSmithKline Biologicals. In their paper Harper et al state: "Geometric mean titres for vaccine‐induced antibodies to HPV antibodies were over 80 and 100 times greater than those seen in natural infections with HPV‐18 and HPV‐16, respectively. Vaccine‐induced titres remained substantially raised at 18 months, and were still 10‐16 times higher than those seen in women with natural HPV‐16 or HPV‐18 infections, respectively." (See page 1763.) and on page 1764: "We have shown that the HPV‐16/18 virus‐like particle vaccine adjuvanted with AS04 induces a level of antibody production against HPV‐16/18 that is much higher than that induced by natural infection. Previous work has shown that combinations of the adjuvants MPL and aluminium salts induce an enhanced immune response compared with antigen alone or adjuvanted with only aluminium, at both the humoral and cellular level. These findings suggest that the immune responses induced in vaccinated women may provide a longer duration of protection than the protective effects induced by natural HPV infection; however, a protective antibody level has not been established nor is there sufficient data currently available to estimate the duration of vaccine‐induced protection." Should we be concerned that HPV vaccines produce antibodies over 80 and 100 times greater than those seen in natural infections with HPV‐18 and HPV‐16 respectively, and which remain substantially raised months after vaccination?

Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment.

Date of Submission: 15.01.2018

Contributor: Elizabeth Hart, Australia Affiliation: Independent Role: Independent researcher

Reply

Dear Elizabeth Hart,

Thank you for your interest in our planned research and your questions related to the protocol, which we will try to answer to the best of our ability.

The number of antibodies is a non‐validated surrogate outcome with unknown clinical relevance (see Discussion on surrogate outcomes in Jakobsen, J. C., et al.¹). In Cochrane reviews, ‘hard outcomes’ are typically preferred, and thus our planned review is focused on these; benefits and harms of aluminium adjuvants will be assessed as defined in our protocol. Vaccine efficacy measured by antibody titre is a surrogate measure of the effectiveness of a vaccine. Our protocol is focusing on the potential benefits and harms of aluminium adjuvants. If we succeed to demonstrate such benefits and harms, the next research question could be to examine if antibody titres are involved. This will likely require individual patient data meta‐analysis. Currently, we cannot provide any useful answers to these questions. We would like to emphasise that our chosen (hard) outcomes seek to answer questions that we believe are most relevant to patients and healthcare. Further studies could be employed to investigate possible mechanisms for observed benefits and harms.

Response References 1. Jakobsen JC, Nielsen EE, Feinberg J, Katakam KK, Fobian K, Hauser G, Poropat G, Djurisic S, Weiss KH, Bjelakovic M, Bjelakovic G, Klingenberg SL, Liu JP, Nikolova D, Koretz RL, Gluud C. Direct‐acting antivirals for chronic hepatitis C. Cochrane Database of Systematic Reviews 2017, Issue 9. Art. No.: CD012143. DOI: 10.1002/14651858.CD012143.pub3.

Contributors: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Contributors

Comments made by: Elizabeth Hart Comments addressed by: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Aluminium adjuvants used in vaccines versus placebo or no intervention, 5 February 2018

Summary

In my recent comments I expressed concern about the unnaturally high titres induced by the novel VLP HPV vaccines Gardasil and Cervarix. Millions of children around the world have been given three doses of these vaccine products. In regards to the three dose HPV vaccine regimen, in December 2016 I contacted Professor Diane Harper, an author of the study re the bivalent HPV vaccine (i.e. Cervarix), published in The Lancet in 2004^[1], to ask her if titres were measured after individual doses or after all three doses in that study. I was surprised when Professor Harper responded that "The titers were measured one month after the third dose."^[2] Professor Harper's response indicates that titres were not measured after each individual dose. So, it appears it was not proven that three doses of Cervarix HPV vaccine were required. In her email response to me, Professor Harper said: "The need for long‐term protection drove the fear that three doses would be needed. As we learned one dose of cervarix provides high titers as well and has proven efficacy. It is unfortunate that the WHO would not recommend one dose of cervarix worldwide." In regards to Professor Harper's statement "As we learned one dose of cervarix provides high titers...", another study re Cervarix, published in 2013^[3] states: "Antibody levels following one‐dose remained stable from month 6 through month 48. Results raise the possibility that even a single dose of HPV VLPs will induce long‐term protection." This study was followed up with further analysis in 2015[4] which also indicates there is no evidence to support the three dose Cervarix HPV vaccine regimen. It is shocking to discover there was no evidence to support the three dose HPV vaccine regimen.In regards to unnaturally high titres after HPV vaccination, what are the possibly deleterious effects of revaccinating with multiple doses of novel, turbo‐charged aluminium‐adjuvanted VLP HPV vaccines, i.e. do repeat shots increase the risk of an adverse reaction? Is there any reliable data, including post‐marketing surveillance, to check on this?

HPV vaccination has been fast‐tracked around the world. Children have been given three doses of novel, turbo‐charged aluminium‐adjuvanted VLP HPV vaccines which produce unnaturally high titres. Scientists such as Professor Harper admit "the mechanism of immunogenicity from a scientific perspective is poorly understood".^[5]Children are being used as guinea pigs in a massive international experiment ‐ is this ethical? What are the implications here in regards to informed consent? While the studies I have referred to are about the Cervarix HPV vaccine, this leads to questions about the Gardasil HPV vaccine ‐ what is the evidence supporting vaccination with three doses of the Gardasil HPV vaccine product? Were three doses of HPV vaccines suggested to justify the cost of these vaccine products?

As for Professor Harper's suggestion that Cervarix "has proven efficacy", as far as I am aware, there is as yet no independent and objective systematic review of the efficacy of HPV vaccination in preventing cervical cancer, i.e. untainted by pharma influence or bias. I suggest the public is being misled about the promoted 'efficacy' of globally fast‐tracked HPV vaccination. At this time we have no idea of the long‐term effects of this very questionable medical intervention, particularly if the risks will outweigh the touted benefits. In my opinion the benefits of HPV vaccination are being over‐hyped, and children and their parents are being grossly misinformed about HPV vaccination. At this time there is no independent and objective analysis validating HPV vaccination, and no scientific basis for the three dose regimen. I request your urgent consideration of the points I have raised.

References 1. Diane M Harper et al. Efficacy of a bivalent L1 virus‐like particle vaccine in prevention of infection with human papillomavirus types 16 and 18 in young women: a randomised controlled trial. Lancet. Vol. 364. November 13 2004. 2. Email response from Diane Harper, 11 December 2016. 3. Mahboobeh Safaeian et al. Durable Antibody Responses Following One Dose of the Bivalent Human Papillomavirus L1 Virus‐Like Particle Vaccine in the Costa Rica Vaccine Trial. Cancer Prev Res; 6(11) November 2013. 4. Aimee R Kriemer et al. Efficacy of fewer than three doses of an HPV‐16/18 AS04‐adjuvanted vaccine: combined analysis of data from the Costa Rica Vaccine and PATRICIA trials. The Lancet Oncology Vol 16, July 2015. 5. Diane M Harper. Prophylactic human papillomavirus vaccines to prevent cervical cancer: review of the Phase II and III trials. Therapy (2008) 5(3), 313‐324.

Date of Submission: 05.02.2018

Contributor: Elizabeth Hart, Australia https://over‐vaccination.net/ Affiliation: Independent Role: Independent researcher

Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment.

Reply

Dear Elizabeth Hart,

Thank you once again for your interest in our planned research and your questions related to our protocol.

You raise a number of interesting points regarding HPV vaccines. As the trials involving HPV vaccines that you describe are not based on a comparison between the vaccine including aluminium adjuvant and a true placebo (saline placebo) or no intervention, the results from these trials are unfortunately not eligible for our systematic review and its analyses. The interesting points you raise need to be addressed in other systematic reviews.

We, the authors of the planned systematic review, are focusing on aluminium adjuvants versus placebo or no intervention, and are not at present planning to do systematic reviews on the effects of HPV vaccines. We urge others to do such studies.

Contributors: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Contributors

Comments made by: Elizabeth Hart Comments addressed by: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Aluminium adjuvants used in vaccines versus placebo or no intervention, 5 February 2018

Summary

Further to my previous comment regarding high antibody titres after HPV vaccination (15 January 2018), and my query whether the much higher antibody titre after HPV vaccination, as compared to natural HPV infection, was a 'good thing'. As mentioned in my previous comment, in regards to HPV 'immunisation' inducing antibody titres that are 80‐ to 100‐ fold higher than those observed following natural infection, Ian Frazer (2010)^[1] cites a paper by Diane Harper et al regarding the bivalent HPV vaccine (2004)^[2], i.e. presumably Cervarix. (This study was funded and coordinated by GlaxoSmithKline Bioglogicals.) It appears Frazer generalises about high antibody titres after HPV vaccination, i.e. Gardasil and Cervarix, from Harper et al's paper about Cervarix.^[2]In a later review paper (2008)^[3], Diane Harper refers to high antibody titres after both vaccines, i.e. "the peak response to vaccination was robustly 100‐200‐fold higher than natural infection titers for both vaccines in neutralizing type‐specific antibody titers for both HPV 16 and 18", although in a later paper (2009)^[4] Harper says peak titre after Gardasil vaccination is 104‐fold higher than natural infection for HPV 16, and 27‐fold higher than natural infection titres for HPV 18. In essence though, it appears HPV vaccination with both vaccines creates a much higher antibody response than natural infection, and from my layperson's perspective I wonder if there is any downside to this unnatural response? In her 2008^[3] review paper, Harper also states: "Despite both vaccines having a 100% seroconversion 1 month after three doses of vaccine, the mechanism of immunogenicity from a scientific perspective is poorly understood. The measure of antibody induction by geometric mean titers (GMTs) is dependent on the assay system used, and is not comparable between HPV types within one manufacturer or for identical HPV types between manufacturers." It is concerning that the novel virus‐like particle (VLP) vaccine products Gardasil and Cervarix have been fast‐tracked globally, when "the mechanism of immunogenicity from a scientific perspective is poorly understood". In her 2008^[3] review paper, Harper states: "...both vaccines contain a proprietary adjuvant system to improve the immunologic response to the VLP antigens. The adjuvant system, AS04, in Cervarix contains both an aluminium salt and a toll‐like receptor‐4 agonist (monophosphoryl lipid A); the adjuvant system in Gardasil contains an aluminium salt called aluminium hydroxyphosphate sulfate. Clinical trials in humans show that the HPV 16/18 VLPs adjuvanted with AS04 induce a significantly greater initial antibody response than do the HPV16/18 VLPs adjuvanted with aluminium hydroxide alone, and this superior response continues for at least 4 years...Experiments in mice show that the Merck proprietary amorphous aluminium hydroxyphosphate sulfate used in Gardasil induces a greater initial antibody response to HPV16 VLPs than does the aluminium hydroxide adjuvant alone..." A VacZine Analytics press release titled "GSK and Cervarix ‐ is AS04 a double edged sword?" (2007)^[5] says the novel adjuvant AS04 contained in Cervarix "is a combination of standard aluminium hydroxide and the new component, monophospholipid A (MPL). MPL is a derivative of the lipid A molecule found in gram‐negative bacteria and is considered one of the most potent immune system stimulants known". Merck's proprietary amorphous aluminium hydroxyphosphate sulfate used in Gardasil also appears to be more potent than aluminium hydroxide adjuvant alone.^[3] Harper says the purpose of the adjuvant "is to prolong the immune response for as long as possible with the smallest amount of antigen (VLP) possible".^[4] Again, I register my concern that the novel Gardasil and Cervarix VLP HPV vaccine products have been fast‐tracked around the world, particularly as "the mechanism of immunogenicity from a scientific perspective is poorly understood".If children and their parents were properly informed of the unnaturally high antibody titre induced by both the novel aluminium adjuvanted Gardasil and Cervarix vaccine products, and that scientists such as Diane Harper admit the mechanism of immunogenicity of these products is poorly understood from a scientific perspective, I wonder if they would consent to this still experimental medical intervention?

References 1. Ian H Frazer. Measuring serum antibody to human papillomavirus following infection or vaccination. Gynecologic Oncology 118 (2010) S8‐S11. 2. Diane M Harper et al. Efficacy of a bivalent L1 virus‐like particle vaccine in prevention of infection with human papillomavirus types 16 and 18 in young women: a randomised controlled trial. Lancet, 2004; 364: 1757‐65. 3. Diane M Harper. Prophylactic human papillomavirus vaccines to prevent cervical cancer: review of the Phase II and III trials. Therapy (2008) 5(3), 313‐324. 4. Diane M Harper. Currently Approved Prophylactic HPV Vaccines. Expert Rev Vaccines. 2009; 8 (12): 1663‐1679). 5. GSK and Cervarix ‐ is AS04 a double edged sword? Press Release. VacZine Analytics. Posted on 19 Dec 2007: http://www.pharmiweb.com/pressreleases/pressrel.asp?ROW_ID=2994#.WEo0srJ97IV

Date of Submission: 05.02.2018

Contributor: Elizabeth Hart, Australia https://over‐vaccination.net/ Affiliation: Independent Role: Independent researcher

Reply

Dear Elizabeth Hart, We thank you for the additional comments. We have registered your concern. Referring to our two previous replies, we must stress yet again that your comments and questions cannot be answered from our planned systematic reviews focusing on aluminium versus placebo or no intervention, or versus other aluminium types in connection with vaccines^1,2. We hope that your efforts will be met with success by others assessing the raised issues.

Contributors: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Contributors

Comments made by: Elizabeth Hart Comments addressed by: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Aluminium adjuvants used in vaccines versus placebo or no intervention, 14 March 2018

Summary

Attention to detail ... You wrote: “vaccines undergo preclinical testing for safety, for the ability to induce cancer,” No. Vaccines are NOT evaluated for carcinogenicity.^[1–4] The package inserts usually state “ … has not been evaluated for carcinogenic or mutagenic potential, or for impairment of fertility.”

With new adjuvants, expect new diseases

"Consequently, the observational studies, which based their results on registered diagnoses, may have excluded an important fraction of eligible participants with unclear adverse symptoms, as most young girls that claim to suffer from adverse events following HPV vaccination receive no clinical diagnosis, and are therefore unlikely to appear in medical registers."

With new adjuvants such as amorphous aluminum hydroxyphosphate sulfate (AAHS) in the HPV vaccine and with their new interactions with target proteins (HPV) and non‐target proteins (such as yeast proteins) in the vaccine, one should expect patients to develop completely new diseases that result in "no clinical diagnosis".

Cannot ignore established animal disease induction process using aluminum

Aluminum salts are used as adjuvants to reliably induce allergy, asthma and autoimmunity (such as Experimental Autoimmune Encephalomyelitis (EAE)) in lab animals.^[5,6]

Why perform animal studies if you are going to ignore the fact that you sicken animals the same way you sicken humans?

Details matter ... again Many studies do not account for differences in the same type of vaccine produced by different manufacturers. Engerix B^[1] has 5% yeast proteins vs. Recombivax HB^[4] having 1% yeast proteins.

Aluminum adjuvanted vaccines with 1% yeast proteins will not produce the same rate of adverse events as vaccines with 5% yeast proteins. So such details matter.

Aluminum immunotoxicity could lead to aluminum neurotoxicity

The aluminum elimination process you describe applies to a healthy human body. In the case of Cerebral Folate Deficiency (CFD), folate deficiency impacts aluminum elimination and results in accumulation. CFD itself could have been induced by aluminum adjuvanted cow's milk contaminated vaccines.^[7,8]

Cannot focus only on specific types of studies, use all available evidence in the literature

Mascart et al.^[9] make the mistaken assumption that beta‐lactoglobulin (BLG) is unrelated to vaccines.

BLG is a major cow’s milk allergen and the authors used vaccines that contain residual cow’s milk proteins from the growth media.[10] The study subjects were all bottle‐fed with cow’s milk (thus BLG). Such early oral cow’s milk introduction protects against cow’s milk allergy.^[11,12] However, even in this protected population, the authors report a Th2 response including IgE against BLG . Thus inadvertently, they provide strong evidence of a Th2 and allergic response to target and non‐target proteins contained in aluminum adjuvanted vaccines.

If the subjects had been exclusively breast‐fed (as is recommended), without the benefit afforded by early oral cow’s milk introduction, a lot more of them would have developed a stronger allergic response to cow’s milk. Normally exclusive breast feeding is best. But with aluminum adjuvanted cow’s milk contaminated vaccines, this recommendation could increase the risk of developing cow’s milk allergy.^[13] Who is looking into such consequences?

To really evaluate the safety of aluminum salts in vaccines, one would have to account for all known/potential immunological mechanisms involved with aluminum adjuvants. What are the potential negative outcomes due to that mechanism? What tests are needed to check for those outcomes? Would the outcomes be overt disease or will they be sub‐clinical effects for years? This would determine follow‐up times and decision on serological examination. For example: to assess if aluminum may be increasing the risk of sensitization to cow's milk proteins contaminating the vaccine, one would not only have to wait for 4 weeks after vaccination, but also challenge the patient with cow's milk, pre and post vaccination, to assess the impact. Similarly, to check if aluminum induced an autoimmune disease that may only show up years later, one would have to perform autoimmune serology pre and post‐vaccination checking for changes in autoantibody levels, as suggested by Wraith et al.^[14] Nobody performs such studies.^[8]

Aluminum adjuvanted vaccine induced maternal autism related antibodies can affect the fetal brain.^[15,16] Do you expect to find clinical trials that cover that?

So, how will you be able to perform a review of aluminum safety in vaccines?

In other words, you have to ask if the right type of studies have even been done?

May be you have to investigate some these issues yourself, from scratch, if you really want to answer the question of aluminum adjuvant safety in vaccines. Otherwise, like the IOM 2012^[17] report on vaccine adverse events, you will end up with a useless conclusion: “However, for the majority of cases (135 vaccine‐adverse event pairs), the evidence was inadequate to accept or reject a causal relationship.”. Or worse, you could end up repeating the Jefferson et al.^[18] and Mitkus et al.^[19] fiasco^[8] and set back vaccine safety for another 15 years.

References 1. Engerix B Package Insert [Internet]. [cited 2016 May 8]. Available from: http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM224503.pdf 2. Pasteur S. Menactra vaccine package insert [Internet]. Available from: https://www.fda.gov/downloads/biologicsbloodvaccines/vaccines/approvedproducts/ucm131170.pdf 3. MMR II Vaccine Package Insert [Internet]. [cited 2016 May 3]. Available from: http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM123789.pdf 4. Recombivax HB Package Insert [Internet]. [cited 2016 May 8]. Available from: http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM110114.pdf 5. Arumugham V. Evidence that Food Proteins in Vaccines Cause the Development of Food Allergies and Its Implications for Vaccine Policy. J Dev Drugs. 2015;4(137):2. 6. Libbey JE, Fujinami RS. Experimental Autoimmune Encephalomyelitis as a Testing Paradigm for Adjuvants and Vaccines. Vaccine. 2011 Apr 12;29(17):3356–62. 7. Arumugham V. Epidemiological studies that ignore mechanism of disease causation are flawed and mechanistic evidence demonstrates that vaccines cause autism [Internet]. 2017. Available from: https://doi.org/10.5281/zenodo.1041905 8. Arumugham V. Safety studies of aluminum in vaccines lack immunotoxicity analysis of this immunological adjuvant: Ignorance or deception? [Internet]. 2017. Available from: https://doi.org/10.5281/zenodo.1117241 9. Mascart F, Hainaut M, Peltier A, Verscheure V, Levy J, Locht C. Modulation of the infant immune responses by the first pertussis vaccine administrations. Vaccine. 2007 Jan 4;25(2):391–8. 10. Kattan JD, Cox AL, Nowak‐Wegrzyn A, Gimenez G, Bardina L, Sampson HA, et al. Allergic reactions to diphtheria, tetanus, and acellular pertussis vaccines among children with milk allergy. J Allergy Clin Immunol. 2011;Conference(var.pagings):AB238. 11. Du Toit G, Roberts G, Sayre PH, Bahnson HT, Radulovic S, Santos AF, et al. Randomized trial of peanut consumption in infants at risk for peanut allergy. N Engl J Med. 2015;372(9):803–13. 12. Wells HG, Osborne TB. The Biological Reactions of the Vegetable Proteins I. Anaphylaxis. J Infect Dis. 1911;8(1):66–124. 13. Arumugham V. Professional Misconduct by NAM Committee on Food Allergy [Internet]. 2016. Available from: https://www.zenodo.org/record/1034559 14. Wraith DC, Goldman M, Lambert P‐H. Vaccination and autoimmune disease: what is the evidence? Lancet (London, England). England; 2003 Nov;362(9396):1659–66. 15. Fox‐Edmiston E, de Water J Van. Maternal anti‐fetal brain IgG autoantibodies and autism spectrum disorders: current knowledge and its implications for potential therapeutics. CNS Drugs. 2015 Sep;29(9):715–24. 16. Arumugham V. Strong protein sequence alignment between autoantigens involved in maternal autoantibody related autism and vaccine antigens [Internet]. 2017. Available from: https://www.zenodo.org/record/1034571 17. Stratton K, Ford A, Rusch E, Clayton EW. Adverse Effects of Vaccines: Evidence and Causality. Injury. 2011. 0‐24 p. 18. Jefferson T, Rudin M, Di Pietrantonj C. Adverse events after immunisation with aluminium‐containing DTP vaccines: systematic review of the evidence. Lancet Infect Dis. United States; 2004 Feb;4(2):84–90. 19. Mitkus RJ, King DB, Hess MA, Forshee RA, Walderhaug MO. Updated aluminum pharmacokinetics following infant exposures through diet and vaccination. Vaccine. 2011 Nov 28;29(51):9538–43.

Date of Submission: 14.03.2018

Contributor: Vinu Arumugham Email Address: vinucubeacc@gmail.com Role: Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment.

Reply

Dear Vinu Arumugham,

We thank you for the interest in our peer reviewed Cochrane systematic review protocol and subsequent peer reviewed Cochrane systematic review. Below, we will answer the raised questions to the best of our ability.

It is a general rule, that vaccines, just like other drugs, should undergo different types of pre‐clinical and clinical testing; however, this is not what we do. What we do is collecting the global evidence on a predefined PICOT (participants, interventions, outcomes, time of data collection), mainly from randomised clinical trials, assessing the risk of bias of the trials for inclusion in the review, and meta‐analysing relevant data as part of the systematic review preparation. Thus, the focus of our systematic review is not to determine a safe dose or the general safety profile of one or more vaccines. This should have been established during the pre‐clinical and clinical testing of the vaccines on the market. Our aim is to assess the beneficial and harmful effects of aluminium adjuvants in vaccines.

We are working on two systematic reviews in which we are including trials with human participants. In our first review, we are assessing aluminium adjuvants versus placebo or no adjuvants¹. In our second review, we will be assessing the different forms of aluminium adjuvants versus each other². In addition to the inclusion of randomised clinical trials that have assessed benefits or harms of the aluminium vaccine adjuvants in humans or both, we will also look at quasi‐randomised studies and other observational studies in humans for their report on harms. As you see, our focus of work is not on carcinogenicity. Individual vaccines, following the imposed Regulatory Requirements issued by different, usually, governmental agencies, may or may not include carcinogenic or mutagenic testing. The latter may depend on, for e.g., previous usage; some new vaccines are combinations of several previous vaccines, and, therefore, such tests may not be recommended, and this could also apply for aluminium adjuvants, being used in different formulations and concentrations for nearly a century. This discussion should be taken with the regulatory authorities responsible for the approval of the specific vaccines and the specific adjuvants.

To mix together study data from any source available, without strict study selection criteria and rigorous methods for study assessment cannot be a reliable approach as it would lead to strongly biased and unreliable clinical results. We encourage reading the publication on the ‘hierarchy of evidence’^3,4. According to the ‘hierarchy of evidence’, systematic reviews of randomised clinical trials provide more solid evidence than single randomised clinical trials irrespective of their size. The scientific strength of Cochrane systematic reviews is that they produce integrated knowledge that combines benefits and harms from randomised clinical trials as well as knowledge on harms from observational studies^3,4. Therefore, it would threaten the validity of our research if we were to include “all available evidence in the literature”.

Pre‐clinical testing is essential before launching randomised clinical trials in humans. We want to stress that the planned outcomes before randomised clinical trials are initiated should always be patient‐relevant; that is, what are the benefits and what are the harms, without any pre‐selection based on potential immunological mechanisms from pre‐clinical research, as this may overlook yet uncovered immunological mechanisms and effects. In addition to our systematic reviews on humans, we plan to perform two systematic reviews on animals to investigate the level of evidence for pre‐clinical testing of aluminium adjuvants. The first will assess aluminium adjuvants versus placebo or no adjuvants⁵, and the second will assess the different forms of aluminium adjuvants versus each other⁶. Likewise, in these systematic reviews, any formulations of potential mechanistic explanations may end as obstacles than gaining more insight into the matter.

We agree that follow‐up should be adequate to account for long‐term effects.

At this point, we have little expectations as to what we will find. This is the nature of conducting systematic reviews. Once conducted, our systematic reviews will summarise the available evidence for benefits and harms of aluminium adjuvants from randomised clinical trials in humans and animals in separate. Properly designed and performed randomised clinical trials remain the reference standard for assessing benefits and harms of an intervention. Other types of clinical research may be valid under different scenarios; however, through their design and methodology, they will all be at risk of overestimating benefits and underestimating harms^3,7.

The IOM conclusions from eight years ago referred to by Vinu Arumugham are based on a consensus study report, not on a systematic review. Again, we encourage everyone to read about the ‘hierarchy of evidence’^3,4. Further, aligned in the opinion of many other researchers, a conclusion based on thorough clinical research (properly designed and performed randomised clinical trials) that states that ‘the evidence was inadequate to accept or reject a causal relationship’ is not ideal, but it is not useless either. It simply tells the reader that there is not enough evidence to come with a conclusion. Strong conclusions should preferably be based on systematic reviews of randomised clinical trials. Only when our results are published, then we can determine what kind of success we have had.

Response References 1. Djurisic S, Jakobsen JC, Petersen SB, Kenfelt M, Gluud C. Aluminium adjuvants used in vaccines versus placebo or no intervention (Protocol). Cochrane Database of Systematic Reviews 2017, Issue 9. Art. No.: CD012805. DOI: 10.1002/14651858.CD012805. 2. Djurisic S, Jakobsen JC, Petersen SB, Kenfelt M, Klingenberg SL, Gluud C. Different aluminium adjuvants in vaccines. Cochrane Database of Systematic Reviews 2018 (submitted). 3. Garattini S, Jakobsen JC, Wetterslev J, Bertele V, Banzi R, Rath A, et al. Evidence‐based clinical practice: Overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016; 32:13–21. doi: 10.1016/j.ejim.2016.03.020. 4. Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration, 2011. Available from handbook.cochrane.org. 5. Katakam KK, Jakobsen JC, Petersen SB, Kenfelt M, Djurisic S, Gluud C. Aluminium adjuvants versus placebo or no intervention in pre‐clinical vaccine studies. 2018a (work in progress). 6. Katakam KK, Jakobsen JC, Petersen SB, Kenfelt M, Djurisic S, Gluud C. Aluminium adjuvants versus other aluminium adjuvants in pre‐clinical vaccine studies. 2018b (work in progress). 7. Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non‐randomised intervention studies. Health Technology Assessment 2003;7(27):iii–x. doi: 10.3310/hta7270.

Contributors: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Contributors

Comments made by: Vinu Arumugham Comments addressed by: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Aluminium adjuvants used in vaccines versus placebo or no intervention, 16 March 2018

Summary

Are you setting yourself up for failure?

Responding to Elizabeth Hart, you wrote:

"As the trials involving HPV vaccines that you describe are not based on a comparison between the vaccine including aluminium adjuvant and a true placebo (saline placebo) or no intervention, the results from these trials are unfortunately not eligible for our systematic review and its analyses." As you know, most current aluminum adjuvanted vaccines are subunit vaccines which will be ineffective without the adjuvant. It is considered "unethical" to conduct a trial where some groups will be administered such known ineffective vaccines, placebo or no intervention. So it is unlikely that you will find many studies meeting your requirement below: "any type of vaccine including any type of aluminium adjuvant (including, but not limited to, aluminium potassium sulphate; aluminium hydroxide; aluminium phosphate; or aluminium hydroxyphosphate sulfate) versus the same vaccine but without the aluminium adjuvant; any aluminium adjuvant versus placebo or no intervention." The claim that such studies are unethical involves circular reasoning as the safety of the vaccine has not yet been proven. But that is a separate matter ...

Reply

Thank you again for your interest in our work.

We believe that we have already addressed your additional comment in our previous response. You might be correct that we may not identify many trials, but we cannot know this before we start working on the review and running systematic literature searches to identify relevant trials, no matter the language, format, and year of publication. We would like to pay your attention to the Cochrane synthesis of what a Cochrane systematic review of interventions is:

" What is a systematic review?

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre‐specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions.)"

In other words, whether we shall find evidence or not will depend on what is out there and what the quality of this information is. If we do not find evidence, then this will not only be our failure. If it turns out as you surmise, then our review will hopefully trigger further research that may answer one day the concerns of many people, and not just yours.

Response Reference Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

Contributors: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Contributors

Comments made by: Vinu Arumugham Comments addressed by: Snezana Djurisic, Janus C Jakobsen, and Christian Gluud

Aluminium adjuvants used in vaccines versus placebo or no intervention, 1 June 2018

Summary

The paper starts with this blank statement: "The effectiveness of vaccinations has been proven repeatedly since the first introduction of the cowpox vaccine in the 18th century".

From a group dedicated to Evidence Based Medicine, it is quite shocking and unexpected to read such praise of crude and dangerously septic vaccines of historic times that were never tested against controls or placebos, and therefore never fulfilled the minimum requirements of EBM. It looks like the authors feel a stronger need to pay lipservice to a tradition engrained in medical authorities, for fear of being ostracized, than the need of revising traditional practices that are not evidence based. Very sad to see Cochrane abandoning its own founding principles.

Contributor: Oscar Gonzalez Email Address: deftlook@yahoo.com Role: Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment.

Aluminium adjuvants used in vaccines versus placebo or no intervention, 1 June 2018

Summary

When will the results of this study be released?

Contributor: Madeline Email Address: mcgspiehs@yahoo.com Role: Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment.

Acknowledgements

Cochrane Review Group funding acknowledgement: The Danish State is the largest single funder of The Cochrane Hepato‐Biliary Group through its investment in the Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen University Hospital, Denmark. Disclaimer: The views and opinions expressed in this review are those of the authors and do not necessarily reflect those of the Danish State or the Copenhagen Trial Unit.

Peer reviewers: Jagdish K. Zade, India, and Sachin Thakkar, US Contact Editor: Kurinchi Gurusamy, UK Sign‐off Editor: Kurinchi S Gurusamy, UK

Appendices

Appendix 1. Preliminary search strategies

Database	Time span	Search strategy
Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library	Latest issue	#1 MeSH descriptor: [Adjuvants, Immunologic] explode all trees #2 (immunologic* or alum* or vaccine) and adjuvan #3 #1 or #2 #4 MeSH descriptor: [Vaccines] explode all trees #5 vaccin* #6 #4 or #5 #7 #3 and #6
MEDLINE Ovid	1946 to the date of search	1. exp Adjuvants, Immunologic/ 2. ((immunologic* or alum* or vaccine) and adjuvan).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 3. 1 or 2 4. exp Vaccines/ 5. vaccin.mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 6. 4 or 5 7. 3 and 6 8. (random or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 9. 7 and 8
Embase Ovid	1974 to the date of search	1. exp immunological adjuvant/ 2. ((immunologic* or alum* or vaccine) and adjuvan).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word] 3. 1 or 2 4. exp vaccine/ 5. vaccin.mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word] 6. 4 or 5 7. 3 and 6 8. (random or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word] 9. 7 and 8
BIOSIS (Web of Science)	1969 to the date of search	#5 #4 AND #3 #4 TS=(random* or blind* or placebo* or meta‐analys) #3 #2 AND #1 #2 TS=vaccin #1 TS=((immunologic* or alum* or vaccine) and adjuvan)
LILACS (Bireme)	1982 to the date of search	(immunologic$ or alum$ or vaccine$) and adjuvan$ [Words] and vaccin$ [Words]
Science Citation Index Expanded (Web of Science)	1900 to the date of search	#5 #4 AND #3 #4 TS=(random* or blind* or placebo* or meta‐analys) #3 #2 AND #1 #2 TS=vaccin #1 TS=((immunologic* or alum* or vaccine) and adjuvan)
Conference Proceedings Citation Index – Science (Web of Science)	1990 to the date of search	#5 #4 AND #3 #4 TS=(random* or blind* or placebo* or meta‐analys) #3 #2 AND #1 #2 TS=vaccin #1 TS=((immunologic* or alum* or vaccine) and adjuvan)

Open in a new tab

What's new

Date	Event	Description
16 May 2018	Feedback has been incorporated	Authors' reply to the second feedback submitted on 16 of March 2018
16 March 2018	Feedback has been incorporated	Feedback submitted 16 of March 2018

Open in a new tab

History

Protocol first published: Issue 9, 2017

Date	Event	Description
4 May 2018	Feedback has been incorporated	Authors' reply to the first feedback submitted on 14 of March 2018
15 March 2018	Feedback has been incorporated	Authors' reply to the second submitted feedback on 5 of February 2018
15 March 2018	Feedback has been incorporated	Feedback submitted 14 of March 2018
13 March 2018	Feedback has been incorporated	Authors' reply to the first submitted feedback on 5 of February 2018
13 February 2018	Feedback has been incorporated	Authors' reply to the submitted feedback on 15 of January 2018
5 February 2018	Feedback has been incorporated	Feedback submitted (second comment on 5 of February 2018)
5 February 2018	Feedback has been incorporated	Feedback submitted (first comment on 5 of February 2018)
15 January 2018	Feedback has been incorporated	Feedback submitted
16 October 2017	Amended	We increased the clarity of a few sentences in the protocol text.

Open in a new tab

Contributions of authors

MK and SBP presented and drafted the idea for the systematic review. SD, JCJ, and CG drafted the protocol.

Sources of support

Internal sources

Copenhagen Trial Unit, Denmark.

External sources

No sources of support supplied

Declarations of interest

MK is co‐founder of HPV_update.dk.

Edited (no change to conclusions), comment added to review

References

Additional references

Arnheim‐Dahlstrom L, Pasternak B, Svanstrom H, Sparen P, Hviid A. Autoimmune, neurological, and venous thromboembolic adverse events after immunisation of adolescent girls with quadrivalent human papillomavirus vaccine in Denmark and Sweden: cohort study. BMJ (Clinical Research Ed.) 2013;347:f5906. [DOI] [PMC free article] [PubMed] [Google Scholar]
Awate S, Babiuk LA, Mutwiri G. Mechanisms of action of adjuvants. Frontiers in Immunology 2013;4:114. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bartneck M, Keul HA, Wambach M, Bornemann J, Gbureck U, Chatain N, et al. Effects of nanoparticle surface‐coupled peptides, functional endgroups, and charge on intracellular distribution and functionality of human primary reticuloendothelial cells. Nanomedicine 2012;8:1282‐92. [DOI] [PubMed] [Google Scholar]
Brinth L, Theibel AC, Pors K, Mehlsen J. Suspected side effects to the quadrivalent human papilloma vaccine. Danish Medical Journal 2015;62:A5064. [PubMed] [Google Scholar]
Brinth LS, Pors K, Theibel AC, Mehlsen J. Orthostatic intolerance and postural tachycardia syndrome as suspected adverse effects of vaccination against human papilloma virus. Vaccine 2015;33:2602‐5. [DOI] [PubMed] [Google Scholar]
Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61:763‐9. [DOI] [PubMed] [Google Scholar]
Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38:287‐98. [DOI] [PubMed] [Google Scholar]
Bruni L, Diaz M, Barrionuevo‐Rosas L, Herrero R, Bray F, Bosch FX, et al. Global estimates of human papillomavirus vaccination coverage by region and income level: a pooled analysis. Lancet Global Health 2016;4:e453‐63. [DOI] [PubMed] [Google Scholar]
Carter D, Reed SG. Role of adjuvants in modelling the immune response. Current Opinion in HIV and AIDS 2010;5:409‐13. [DOI] [PMC free article] [PubMed] [Google Scholar]
Caulfield MJ, Shi L, Wang S, Wang B, Tobery TW, Mach H, et al. Effect of alternative aluminum adjuvants on the absorption and immunogenicity of HPV16 L1 VLPs in mice. Human Vaccines 2007;3:139‐45. [DOI] [PubMed] [Google Scholar]
Centers for Disease Control and Prevention. Measles (Rubeola). For Healthcare Professionals. www.cdc.gov/measles/hcp/ (accessed 1 August 2017).
Chen RT, Davis RL, Rhodes PH. Special methodological issues in pharmacoepidemiology studies of vaccine safety. In: Strom BL editor(s). Pharmacoepidemiology. Fourth. Chichester, UK: John Wiley & Sons, Ltd, 2006:455‐85. [DOI: 10.1002/9780470059876.ch30] [DOI] [Google Scholar]
Chen L, Yokel RA, Hennig B, Toborek M. Manufactured aluminum oxide nanoparticles decrease expression of tight junction proteins in brain vasculature. Journal of Neuroimmune Pharmacology 2008;3:286‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]
Coffman RL, Sher A, Seder RA. Vaccine adjuvants: putting innate immunity to work. Immunity 2010;33:492‐503. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cowan LD, Griffin MR, Howson CP, Katz M, Johnston RB Jr, Shaywitz BA, et al. Acute encephalopathy and chronic neurological damage after pertussis vaccine. Vaccine 1993;11:1371‐9. [DOI] [PubMed] [Google Scholar]
Delany I, Rappuoli R, Gregorio E. Vaccines for the 21st century. EMBO Molecular Medicine 2014;6:708‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6:341‐50. [DOI] [PubMed] [Google Scholar]
DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials 1986;7:177‐88. [DOI] [PubMed] [Google Scholar]
Donegan K, Beau‐Lejdstrom R, King B, Seabroke S, Thomson A, Bryan P. Bivalent human papillomavirus vaccine and the risk of fatigue syndromes in girls in the UK. Vaccine 2013;31:4961‐7. [DOI] [PubMed] [Google Scholar]
Dourado I, Cunha S, Teixeira MG, Farrington CP, Melo A, Lucena R, et al. Outbreak of aseptic meningitis associated with mass vaccination with a urabe‐containing measles‐mumps‐rubella vaccine: implications for immunization programs. American Journal of Epidemiology 2000;151:524‐30. [DOI] [PubMed] [Google Scholar]
Dyer O. Canadian academic's call for moratorium on HPV vaccine sparks controversy. BMJ (Clinical Research Ed.) 2015;351:h5692. [DOI] [PubMed] [Google Scholar]
Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315:629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
European Medicines Agency. Assessment report. Review under Article 20 of regulation (EC) No 726/2004 human papillomavirus (HPV) vaccines. www.ema.europa.eu/docs/en_GB/document_library/Referrals_document/HPV_vaccines_20/Opinion_provided_by_Committee_for_Medicinal_Products_for_Human_Use/WC500197129.pdf (accessed 1 August 2017).
Exley C. Aluminium‐based adjuvants should not be used as placebos in clinical trials. Vaccine 2011;29:9289. [DOI] [PubMed] [Google Scholar]
US Food, Drug Administration. Study reports aluminum in vaccines poses extremely low risk to infants. www.fda.gov/BiologicsBloodVaccines/ScienceResearch/ucm284520.htm (accessed 1 August 2017).
Foged C, Brodin B, Frokjaer S, Sundblad A. Particle size and surface charge affect particle uptake by human dendritic cells in an in vitro model. International Journal of Pharmaceutics 2005;298:315‐22. [DOI] [PubMed] [Google Scholar]
Glenny AT, Pope CG, Waddington H, Wallace U. Immunological notes. XVII–XXIV. Journal of Pathology and Bacteriology 1926;29:31‐40. [Google Scholar]
World Health Organization. Global Polio Eradication Initiative. www.polioeradication.org/ (accessed 13 January 2017).
Gluud LL. Bias in clinical intervention research. American Journal of Epidemiology 2006;163:493‐501. [DOI] [PubMed] [Google Scholar]
Goto N, Kato H, Maeyama J, Shibano M, Saito T, Yamaguchi J, et al. Local tissue irritating effects and adjuvant activities of calcium phosphate and aluminium hydroxide with different physical properties. Vaccine 1997;15:1364‐71. [DOI] [PubMed] [Google Scholar]
Grimaldi‐Bensouda L, Guillemot D, Godeau B, Benichou J, Lebrun‐Frenay C, Papeix C, et al. Autoimmune disorders and quadrivalent human papillomavirus vaccination of young female subjects. Journal of Internal Medicine 2014;275:398‐408. [DOI] [PubMed] [Google Scholar]
Gruber N, Shoenfeld Y. A link between human papilloma virus vaccination and primary ovarian insufficiency: current analysis. Current Opinion in Obstetrics & Gynecology 2015;27:265‐70. [DOI] [PubMed] [Google Scholar]
Grun JL, Maurer PH. Different T helper cell subsets elicited in mice utilizing two different adjuvant vehicles: the role of endogenous interleukin 1 in proliferative responses. Cellular Immunology 1989;121:134‐45. [DOI] [PubMed] [Google Scholar]
Gupta RK, Rost BE, Relyveld E, Siber GR. Adjuvant properties of aluminum and calcium compounds. Pharmaceutical Biotechnology 1995;6:229‐48. [DOI] [PubMed] [Google Scholar]
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck‐Ytter Y, Alonso‐Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.) 2008;336:924‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gøtzsche P. Complaint to the European Medicines Agency (EMA) over maladministration at the EMA. Nordic Cochrane Centre, Denmark May 26, 2016.
Gøtzsche P. Complaint to the European ombudsman over maladministration at the European Medicines Agency (EMA) in relation to the safety of the HPV vaccines. Nordic Cochrane Centre, Denmark October 10, 2016.
Harbord RM, Egger M, Sterne JA. A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints. Statistics in Medicine 2006;25:3443‐57. [DOI] [PubMed] [Google Scholar]
Hem SL, Hogenesch H. Relationship between physical and chemical properties of aluminum‐containing adjuvants and immunopotentiation. Expert Review of Vaccines 2007;6:685‐98. [DOI] [PubMed] [Google Scholar]
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21:1539‐58. [DOI] [PubMed] [Google Scholar]
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ (Clinical Research Ed.) 2003;327:557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]
Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.
Higgins JP, Whitehead A, Simmonds M. Sequential methods for random‐effects meta‐analysis. Statistics in Medicine 2011;30:903‐21. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non‐blinded outcome assessors. BMJ (Clinical Research Ed.) 2012;344:e1119. [DOI] [PubMed] [Google Scholar]
Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. Canadian Medical Association Journal 2013;185:E201‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub‐studies. International Journal of Epidemiology 2014;43:1272‐83. [DOI] [PMC free article] [PubMed] [Google Scholar]
International Conference on Harmonisation Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, Philadelphia (PA): Barnett International/PAREXEL, 1997. [Google Scholar]
Inbar R, Weiss R, Tomljenovic L, Arango MT, Deri Y, Shaw CA, et al. WITHDRAWN: Behavioral abnormalities in young female mice following administration of aluminum adjuvants and the human papillomavirus (HPV) vaccine Gardasil. Vaccine 2016;pii:S0264‐410X(16)00016‐5. [DOI: 10.1016/j.vaccine.2015.12.067] [DOI] [PubMed] [Google Scholar]
Inbar R, Weiss R, Tomljenovic L, Arango MT, Deri Y, Shaw CA, et al. Behavioral abnormalities in female mice following administration of aluminum adjuvants and the human papillomavirus (HPV) vaccine Gardasil. Immunologic Research 2016;65(1):136‐49. [DOI] [PubMed] [Google Scholar]
Issa AM, Salim MS, Zidan H, Mohamed AF, Farrag A‐RH. Evaluation of the effects of aluminum phosphate and calcium phosphate nanoparticles as adjuvants in vaccinated mice. International Journal of Chemical Engineering and Applications 2014;5(5):367‐73. [Google Scholar]
Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta‐analytic methods. BMC Medical Research Methodology 2014;14:120. [DOI] [PMC free article] [PubMed] [Google Scholar]
Jefferson T, Rudin M, Pietrantonj C. Adverse events after immunisation with aluminium‐containing DTP vaccines: systematic review of the evidence. Lancet Infectious Diseases 2004;4:84‐90. [DOI] [PubMed] [Google Scholar]
Keus F, Wetterslev J, Gluud C, Laarhoven CJ. Evidence at a glance: error matrix approach for overviewing available evidence. BMC Medical Research Methodology 2010;10:90. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kimura M, Kuno‐Sakai H, Yamazaki S, Yamada A, Hishiyama M, Kamiya H, et al. Adverse events associated with MMR vaccines in Japan. Acta Paediatrica Japonica; Overseas Edition 1996;38:205‐11. [DOI] [PubMed] [Google Scholar]
Kisnieriené V, Lapeikaité I. When chemistry meets biology: the case of aluminium – a review. Chemija 2015;26:148‐58. [Google Scholar]
Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta‐analyses. Annals of Internal Medicine 2001;135:982‐9. [DOI] [PubMed] [Google Scholar]
Klein NP, Hansen J, Chao C, Velicer C, Emery M, Slezak J, et al. Safety of quadrivalent human papillomavirus vaccine administered routinely to females. Archives of Pediatrics & Adolescent Medicine 2012;166:1140‐8. [DOI] [PubMed] [Google Scholar]
Kocourkova A, Honegr J, Kuca K, Danova J. Vaccine ingredients: components that influence vaccine efficacy. Mini Reviews in Medicinal Chemistry 2017;17(5):451‐66. [DOI] [PubMed] [Google Scholar]
Kool M, Soullie T, Nimwegen M, Willart MA, Muskens F, Jung S, et al. Alum adjuvant boosts adaptive immunity by inducing uric acid and activating inflammatory dendritic cells. Journal of Experimental Medicine 2008;205:869‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kool M, Fierens K, Lambrecht BN. Alum adjuvant: some of the tricks of the oldest adjuvant. Journal of Medical Microbiology 2012;61:927‐34. [DOI] [PubMed] [Google Scholar]
Lee SH. Detection of human papillomavirus (HPV) L1 gene DNA possibly bound to particulate aluminum adjuvant in the HPV vaccine Gardasil. Journal of Inorganic Biochemistry 2012;117:85‐92. [DOI] [PubMed] [Google Scholar]
Li X, Aldayel AM, Cui Z. Aluminum hydroxide nanoparticles show a stronger vaccine adjuvant activity than traditional aluminum hydroxide microparticles. Journal of Controlled Release 2014;173:148‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2017, Issue 2. [DOI: 10.1002/14651858.MR000033.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Malluche HH. Aluminium and bone disease in chronic renal failure. Nephrology, Dialysis, Transplantation 2002;17(Suppl 2):21‐4. [DOI] [PubMed] [Google Scholar]
Mannhalter JW, Neychev HO, Zlabinger GJ, Ahmad R, Eibl MM. Modulation of the human immune response by the non‐toxic and non‐pyrogenic adjuvant aluminium hydroxide: effect on antigen uptake and antigen presentation. Clinical and Experimental Immunology 1985;61:143‐51. [PMC free article] [PubMed] [Google Scholar]
Marrack P, McKee AS, Munks MW. Towards an understanding of the adjuvant action of aluminium. Nature Reviews. Immunology 2009;9:287‐93. [DOI] [PMC free article] [PubMed] [Google Scholar]
Martinez‐Lavin M. Hypothesis: human papillomavirus vaccination syndrome ‐ small fiber neuropathy and dysautonomia could be its underlying pathogenesis. Clinical Rheumatology 2015;34:1165‐9. [DOI] [PubMed] [Google Scholar]
Mbow ML, Gregorio E, Valiante NM, Rappuoli R. New adjuvants for human vaccines. Current Opinion in Immunology 2010;22:411‐6. [DOI] [PubMed] [Google Scholar]
Miller DL. Frequency of complications of measles, 1963. Report on a national inquiry by the public health laboratory service in collaboration with the Society of Medical Officers of Health. British Medical Journal 1964;2:75‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352:609‐13. [DOI] [PubMed] [Google Scholar]
Morefield GL, Sokolovska A, Jiang D, HogenEsch H, Robinson JP, Hem SL. Role of aluminum‐containing adjuvants in antigen internalization by dendritic cells in vitro. Vaccine 2005;23:1588‐95. [DOI] [PubMed] [Google Scholar]
O'Hagan DT, Gregorio E. The path to a successful vaccine adjuvant ‐ 'the long and winding road'. Drug Discovery Today 2009;14:541‐51. [DOI] [PubMed] [Google Scholar]
O'Hagan DT, Wack A. Adjuvants: from serendipity to rational discovery. Vaccinology. Wiley‐Blackwell, 2012:348‐60. [Google Scholar]
Offit PA. Vaccinated: One Man's Quest to Defeat the World's Deadliest Diseases. Washington DC (USA): Smithsonian. ISBN 0‐06‐122796‐X., 2007. [Google Scholar]
Oleszycka E, Lavelle EC. Immunomodulatory properties of the vaccine adjuvant alum. Current Opinion in Immunology 2014;28:1‐5. [DOI] [PubMed] [Google Scholar]
Oyewumi MO, Kumar A, Cui Z. Nano‐microparticles as immune adjuvants: correlating particle sizes and the resultant immune responses. Expert Review of Vaccines 2010;9:1095‐107. [DOI] [PMC free article] [PubMed] [Google Scholar]
Poddighe D, Castelli L, Marseglia GL, Bruni P. A sudden onset of a pseudo‐neurological syndrome after HPV‐16/18 AS04‐adjuvated vaccine: might it be an autoimmune/inflammatory syndrome induced by adjuvants (ASIA) presenting as a somatoform disorder?. Immunologic Research 2014;60:236‐46. [DOI] [PubMed] [Google Scholar]
Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta‐analysis. Controlled Clinical Trials 1997;18:580‐93; discussion 661‐6. [DOI] [PubMed] [Google Scholar]
Ramanathan VD, Badenoch‐Jones P, Turk JL. Complement activation by aluminium and zirconium compounds. Immunology 1979;37:881‐8. [PMC free article] [PubMed] [Google Scholar]
Rambe DS, Giudice G, Rossi S, Sanicas M. Safety and mechanism of action of licensed vaccine adjuvants. International Current Pharmaceutical Journal 2015;4:420‐31. [Google Scholar]
Reed SG, Orr MT, Fox CB. Key roles of adjuvants in modern vaccines. Nature Medicine 2013;19:1597‐608. [DOI] [PubMed] [Google Scholar]
Reinke CM, Breitkreutz J, Leuenberger H. Aluminium in over‐the‐counter drugs: risks outweigh benefits?. Drug Safety 2003;26:1011‐25. [DOI] [PubMed] [Google Scholar]
Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager 5 (RevMan 5). Version 5.3. Copenhagen: Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]
Salvador A, Igartua M, Hernandez RM, Pedraz JL. An overview on the field of micro‐ and nanotechnologies for synthetic peptide‐based vaccines. Journal of Drug Delivery 2011;2011:181646. [DOI] [PMC free article] [PubMed] [Google Scholar]
Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta‐epidemiological studies. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]
Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]
Scheller NM, Svanstrom H, Pasternak B, Arnheim‐Dahlstrom L, Sundstrom K, Fink K, et al. Quadrivalent HPV vaccination and risk of multiple sclerosis and other demyelinating diseases of the central nervous system. JAMA 2015;313:54‐61. [DOI] [PubMed] [Google Scholar]
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408‐12. [DOI] [PubMed] [Google Scholar]
Sharma HS, Hussain S, Schlager J, Ali SF, Sharma A. Influence of nanoparticles on blood‐brain barrier permeability and brain edema formation in rats. Acta Neurochirurgica 2010;106(Suppl):359‐64. [DOI] [PubMed] [Google Scholar]
Shima F, Uto T, Akagi T, Baba M, Akashi M. Size effect of amphiphilic poly(gamma‐glutamic acid) nanoparticles on cellular uptake and maturation of dendritic cells in vivo. Acta Biomaterialia 2013;9:8894‐901. [DOI] [PubMed] [Google Scholar]
Son YJ, Kim H, Leong KW, Yoo HS. Multifunctional nanorods serving as nanobridges to modulate T cell‐mediated immunity. ACS Nano 2013;7:9771‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
StataCorp LCC. STATA Data Analysis and Statistical Software. Texas, USA: StataCorp LCC, 2014.
Strugnell R, Zepp F, Cunningham A, Tantawichien T. Vaccine antigens. Perspectives in Vaccinology 2011;1:61‐88. [Google Scholar]
Thiele L, Rothen‐Rutishauser B, Jilek S, Wunderli‐Allenspach H, Merkle HP, Walter E. Evaluation of particle uptake in human blood monocyte‐derived cells in vitro. Does phagocytosis activity of dendritic cells measure up with macrophages?. Journal of Controlled Release 2001;76:59‐71. [DOI] [PubMed] [Google Scholar]
Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses?. International Journal of Epidemiology 2009;38:276‐86. [DOI] [PubMed] [Google Scholar]
Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). www.ctu.dk/tsa/files/tsa_manual.pdf (accessed 13 January 2017).
Tomljenovic L, Shaw CA. Aluminum vaccine adjuvants: are they safe?. Current Medicinal Chemistry 2011;18:2630‐7. [DOI] [PubMed] [Google Scholar]
Tomljenovic L, Shaw CA. Too fast or not too fast: the FDA's approval of Merck's HPV vaccine Gardasil. Journal of Law, Medicine & Ethics 2012;40:673‐81. [DOI] [PubMed] [Google Scholar]
Tritto E, Mosca F, Gregorio E. Mechanism of action of licensed vaccine adjuvants. Vaccine 2009;27:3331‐4. [DOI] [PubMed] [Google Scholar]
Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. Version 0.9.5.10 Beta. Copenhagen: Copenhagen Trial Unit, 2011.
Turner RM, Bird SM, Higgins JP. The impact of study size on meta‐analyses: examination of underpowered studies in Cochrane reviews. PLOS One 2013;8:e59202. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ward BJ. Vaccine adverse events in the new millennium: is there reason for concern?. Bulletin of the World Health Organization 2000;78:205‐15. [PMC free article] [PubMed] [Google Scholar]
Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61:64‐75. [DOI] [PubMed] [Google Scholar]
Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random‐effects model meta‐analyses. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wetterslev J, Jakobsen JC, Gluud C. Trial Sequential Analysis in systematic reviews with meta‐analysis. BMC Medical Research Methodology 2017;17:39. [DOI] [PMC free article] [PubMed] [Google Scholar]
Whitney CG, Zhou F, Singleton J, Schuchat A. Benefits from immunization during the vaccines for children program era ‐ United States, 1994‐2013. Morbidity and Mortality Weekly Report 2014;63:352‐5. [PMC free article] [PubMed] [Google Scholar]
World Health Organization. Human papillomavirus vaccines: WHO position paper. Weekly Epidemiological Record 2014;43(89):462‐92. [Google Scholar]
World Health Organization Expert Committee. Annex 2. Guidelines on the nonclinical evaluation of vaccine adjuvants and adjuvanted vaccines. www.who.int/biologicals/areas/vaccines/TRS_987_Annex2.pdf?ua=1 (accessed 13 January 2017).
World Health Organization. Polio vaccines: WHO position paper. Weekly Epidemiological Record 2016;12(91):145–68. [Google Scholar]
World Health Organization. Measles. Fact Sheet. www.who.int/wer/2014/wer8943/en/ (accessed 1 August 2017).
World Health Organization. WHO recommendations for routine immunization ‐ summary tables. www.who.int/immunization/policy/immunization_tables/en/ (accessed 1 August 2017).
Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ (Clinical Research Ed.) 2008;336:601‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yokel RA. The toxicology of aluminum in the brain: a review. Neurotoxicology 2000;21:813‐28. [PubMed] [Google Scholar]

[CD012805-bbs2-0001] Arnheim‐Dahlstrom L, Pasternak B, Svanstrom H, Sparen P, Hviid A. Autoimmune, neurological, and venous thromboembolic adverse events after immunisation of adolescent girls with quadrivalent human papillomavirus vaccine in Denmark and Sweden: cohort study. BMJ (Clinical Research Ed.) 2013;347:f5906. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0002] Awate S, Babiuk LA, Mutwiri G. Mechanisms of action of adjuvants. Frontiers in Immunology 2013;4:114. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0003] Bartneck M, Keul HA, Wambach M, Bornemann J, Gbureck U, Chatain N, et al. Effects of nanoparticle surface‐coupled peptides, functional endgroups, and charge on intracellular distribution and functionality of human primary reticuloendothelial cells. Nanomedicine 2012;8:1282‐92. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0004] Brinth L, Theibel AC, Pors K, Mehlsen J. Suspected side effects to the quadrivalent human papilloma vaccine. Danish Medical Journal 2015;62:A5064. [PubMed] [Google Scholar]

[CD012805-bbs2-0005] Brinth LS, Pors K, Theibel AC, Mehlsen J. Orthostatic intolerance and postural tachycardia syndrome as suspected adverse effects of vaccination against human papilloma virus. Vaccine 2015;33:2602‐5. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0006] Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61:763‐9. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0007] Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38:287‐98. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0008] Bruni L, Diaz M, Barrionuevo‐Rosas L, Herrero R, Bray F, Bosch FX, et al. Global estimates of human papillomavirus vaccination coverage by region and income level: a pooled analysis. Lancet Global Health 2016;4:e453‐63. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0009] Carter D, Reed SG. Role of adjuvants in modelling the immune response. Current Opinion in HIV and AIDS 2010;5:409‐13. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0010] Caulfield MJ, Shi L, Wang S, Wang B, Tobery TW, Mach H, et al. Effect of alternative aluminum adjuvants on the absorption and immunogenicity of HPV16 L1 VLPs in mice. Human Vaccines 2007;3:139‐45. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0011] Centers for Disease Control and Prevention. Measles (Rubeola). For Healthcare Professionals. www.cdc.gov/measles/hcp/ (accessed 1 August 2017).

[CD012805-bbs2-0012] Chen RT, Davis RL, Rhodes PH. Special methodological issues in pharmacoepidemiology studies of vaccine safety. In: Strom BL editor(s). Pharmacoepidemiology. Fourth. Chichester, UK: John Wiley & Sons, Ltd, 2006:455‐85. [DOI: 10.1002/9780470059876.ch30] [DOI] [Google Scholar]

[CD012805-bbs2-0013] Chen L, Yokel RA, Hennig B, Toborek M. Manufactured aluminum oxide nanoparticles decrease expression of tight junction proteins in brain vasculature. Journal of Neuroimmune Pharmacology 2008;3:286‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0014] Coffman RL, Sher A, Seder RA. Vaccine adjuvants: putting innate immunity to work. Immunity 2010;33:492‐503. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0015] Cowan LD, Griffin MR, Howson CP, Katz M, Johnston RB Jr, Shaywitz BA, et al. Acute encephalopathy and chronic neurological damage after pertussis vaccine. Vaccine 1993;11:1371‐9. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0016] Delany I, Rappuoli R, Gregorio E. Vaccines for the 21st century. EMBO Molecular Medicine 2014;6:708‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0017] DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6:341‐50. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0018] DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials 1986;7:177‐88. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0019] Donegan K, Beau‐Lejdstrom R, King B, Seabroke S, Thomson A, Bryan P. Bivalent human papillomavirus vaccine and the risk of fatigue syndromes in girls in the UK. Vaccine 2013;31:4961‐7. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0020] Dourado I, Cunha S, Teixeira MG, Farrington CP, Melo A, Lucena R, et al. Outbreak of aseptic meningitis associated with mass vaccination with a urabe‐containing measles‐mumps‐rubella vaccine: implications for immunization programs. American Journal of Epidemiology 2000;151:524‐30. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0021] Dyer O. Canadian academic's call for moratorium on HPV vaccine sparks controversy. BMJ (Clinical Research Ed.) 2015;351:h5692. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0022] Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315:629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0023] European Medicines Agency. Assessment report. Review under Article 20 of regulation (EC) No 726/2004 human papillomavirus (HPV) vaccines. www.ema.europa.eu/docs/en_GB/document_library/Referrals_document/HPV_vaccines_20/Opinion_provided_by_Committee_for_Medicinal_Products_for_Human_Use/WC500197129.pdf (accessed 1 August 2017).

[CD012805-bbs2-0024] Exley C. Aluminium‐based adjuvants should not be used as placebos in clinical trials. Vaccine 2011;29:9289. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0025] US Food, Drug Administration. Study reports aluminum in vaccines poses extremely low risk to infants. www.fda.gov/BiologicsBloodVaccines/ScienceResearch/ucm284520.htm (accessed 1 August 2017).

[CD012805-bbs2-0026] Foged C, Brodin B, Frokjaer S, Sundblad A. Particle size and surface charge affect particle uptake by human dendritic cells in an in vitro model. International Journal of Pharmaceutics 2005;298:315‐22. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0027] Glenny AT, Pope CG, Waddington H, Wallace U. Immunological notes. XVII–XXIV. Journal of Pathology and Bacteriology 1926;29:31‐40. [Google Scholar]

[CD012805-bbs2-0028] World Health Organization. Global Polio Eradication Initiative. www.polioeradication.org/ (accessed 13 January 2017).

[CD012805-bbs2-0029] Gluud LL. Bias in clinical intervention research. American Journal of Epidemiology 2006;163:493‐501. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0030] Goto N, Kato H, Maeyama J, Shibano M, Saito T, Yamaguchi J, et al. Local tissue irritating effects and adjuvant activities of calcium phosphate and aluminium hydroxide with different physical properties. Vaccine 1997;15:1364‐71. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0031] Grimaldi‐Bensouda L, Guillemot D, Godeau B, Benichou J, Lebrun‐Frenay C, Papeix C, et al. Autoimmune disorders and quadrivalent human papillomavirus vaccination of young female subjects. Journal of Internal Medicine 2014;275:398‐408. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0032] Gruber N, Shoenfeld Y. A link between human papilloma virus vaccination and primary ovarian insufficiency: current analysis. Current Opinion in Obstetrics & Gynecology 2015;27:265‐70. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0033] Grun JL, Maurer PH. Different T helper cell subsets elicited in mice utilizing two different adjuvant vehicles: the role of endogenous interleukin 1 in proliferative responses. Cellular Immunology 1989;121:134‐45. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0034] Gupta RK, Rost BE, Relyveld E, Siber GR. Adjuvant properties of aluminum and calcium compounds. Pharmaceutical Biotechnology 1995;6:229‐48. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0035] Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck‐Ytter Y, Alonso‐Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.) 2008;336:924‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0036] Gøtzsche P. Complaint to the European Medicines Agency (EMA) over maladministration at the EMA. Nordic Cochrane Centre, Denmark May 26, 2016.

[CD012805-bbs2-0037] Gøtzsche P. Complaint to the European ombudsman over maladministration at the European Medicines Agency (EMA) in relation to the safety of the HPV vaccines. Nordic Cochrane Centre, Denmark October 10, 2016.

[CD012805-bbs2-0038] Harbord RM, Egger M, Sterne JA. A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints. Statistics in Medicine 2006;25:3443‐57. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0039] Hem SL, Hogenesch H. Relationship between physical and chemical properties of aluminum‐containing adjuvants and immunopotentiation. Expert Review of Vaccines 2007;6:685‐98. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0040] Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21:1539‐58. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0041] Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ (Clinical Research Ed.) 2003;327:557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0042] Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

[CD012805-bbs2-0043] Higgins JP, Whitehead A, Simmonds M. Sequential methods for random‐effects meta‐analysis. Statistics in Medicine 2011;30:903‐21. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0044] Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non‐blinded outcome assessors. BMJ (Clinical Research Ed.) 2012;344:e1119. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0045] Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. Canadian Medical Association Journal 2013;185:E201‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0046] Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub‐studies. International Journal of Epidemiology 2014;43:1272‐83. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0047] International Conference on Harmonisation Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, Philadelphia (PA): Barnett International/PAREXEL, 1997. [Google Scholar]

[CD012805-bbs2-0048] Inbar R, Weiss R, Tomljenovic L, Arango MT, Deri Y, Shaw CA, et al. WITHDRAWN: Behavioral abnormalities in young female mice following administration of aluminum adjuvants and the human papillomavirus (HPV) vaccine Gardasil. Vaccine 2016;pii:S0264‐410X(16)00016‐5. [DOI: 10.1016/j.vaccine.2015.12.067] [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0049] Inbar R, Weiss R, Tomljenovic L, Arango MT, Deri Y, Shaw CA, et al. Behavioral abnormalities in female mice following administration of aluminum adjuvants and the human papillomavirus (HPV) vaccine Gardasil. Immunologic Research 2016;65(1):136‐49. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0050] Issa AM, Salim MS, Zidan H, Mohamed AF, Farrag A‐RH. Evaluation of the effects of aluminum phosphate and calcium phosphate nanoparticles as adjuvants in vaccinated mice. International Journal of Chemical Engineering and Applications 2014;5(5):367‐73. [Google Scholar]

[CD012805-bbs2-0051] Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta‐analytic methods. BMC Medical Research Methodology 2014;14:120. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0052] Jefferson T, Rudin M, Pietrantonj C. Adverse events after immunisation with aluminium‐containing DTP vaccines: systematic review of the evidence. Lancet Infectious Diseases 2004;4:84‐90. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0053] Keus F, Wetterslev J, Gluud C, Laarhoven CJ. Evidence at a glance: error matrix approach for overviewing available evidence. BMC Medical Research Methodology 2010;10:90. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0054] Kimura M, Kuno‐Sakai H, Yamazaki S, Yamada A, Hishiyama M, Kamiya H, et al. Adverse events associated with MMR vaccines in Japan. Acta Paediatrica Japonica; Overseas Edition 1996;38:205‐11. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0055] Kisnieriené V, Lapeikaité I. When chemistry meets biology: the case of aluminium – a review. Chemija 2015;26:148‐58. [Google Scholar]

[CD012805-bbs2-0056] Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta‐analyses. Annals of Internal Medicine 2001;135:982‐9. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0057] Klein NP, Hansen J, Chao C, Velicer C, Emery M, Slezak J, et al. Safety of quadrivalent human papillomavirus vaccine administered routinely to females. Archives of Pediatrics & Adolescent Medicine 2012;166:1140‐8. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0058] Kocourkova A, Honegr J, Kuca K, Danova J. Vaccine ingredients: components that influence vaccine efficacy. Mini Reviews in Medicinal Chemistry 2017;17(5):451‐66. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0059] Kool M, Soullie T, Nimwegen M, Willart MA, Muskens F, Jung S, et al. Alum adjuvant boosts adaptive immunity by inducing uric acid and activating inflammatory dendritic cells. Journal of Experimental Medicine 2008;205:869‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0060] Kool M, Fierens K, Lambrecht BN. Alum adjuvant: some of the tricks of the oldest adjuvant. Journal of Medical Microbiology 2012;61:927‐34. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0061] Lee SH. Detection of human papillomavirus (HPV) L1 gene DNA possibly bound to particulate aluminum adjuvant in the HPV vaccine Gardasil. Journal of Inorganic Biochemistry 2012;117:85‐92. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0062] Li X, Aldayel AM, Cui Z. Aluminum hydroxide nanoparticles show a stronger vaccine adjuvant activity than traditional aluminum hydroxide microparticles. Journal of Controlled Release 2014;173:148‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0063] Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2017, Issue 2. [DOI: 10.1002/14651858.MR000033.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0064] Malluche HH. Aluminium and bone disease in chronic renal failure. Nephrology, Dialysis, Transplantation 2002;17(Suppl 2):21‐4. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0065] Mannhalter JW, Neychev HO, Zlabinger GJ, Ahmad R, Eibl MM. Modulation of the human immune response by the non‐toxic and non‐pyrogenic adjuvant aluminium hydroxide: effect on antigen uptake and antigen presentation. Clinical and Experimental Immunology 1985;61:143‐51. [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0066] Marrack P, McKee AS, Munks MW. Towards an understanding of the adjuvant action of aluminium. Nature Reviews. Immunology 2009;9:287‐93. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0067] Martinez‐Lavin M. Hypothesis: human papillomavirus vaccination syndrome ‐ small fiber neuropathy and dysautonomia could be its underlying pathogenesis. Clinical Rheumatology 2015;34:1165‐9. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0068] Mbow ML, Gregorio E, Valiante NM, Rappuoli R. New adjuvants for human vaccines. Current Opinion in Immunology 2010;22:411‐6. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0069] Miller DL. Frequency of complications of measles, 1963. Report on a national inquiry by the public health laboratory service in collaboration with the Society of Medical Officers of Health. British Medical Journal 1964;2:75‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0070] Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352:609‐13. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0071] Morefield GL, Sokolovska A, Jiang D, HogenEsch H, Robinson JP, Hem SL. Role of aluminum‐containing adjuvants in antigen internalization by dendritic cells in vitro. Vaccine 2005;23:1588‐95. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0072] O'Hagan DT, Gregorio E. The path to a successful vaccine adjuvant ‐ 'the long and winding road'. Drug Discovery Today 2009;14:541‐51. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0073] O'Hagan DT, Wack A. Adjuvants: from serendipity to rational discovery. Vaccinology. Wiley‐Blackwell, 2012:348‐60. [Google Scholar]

[CD012805-bbs2-0074] Offit PA. Vaccinated: One Man's Quest to Defeat the World's Deadliest Diseases. Washington DC (USA): Smithsonian. ISBN 0‐06‐122796‐X., 2007. [Google Scholar]

[CD012805-bbs2-0075] Oleszycka E, Lavelle EC. Immunomodulatory properties of the vaccine adjuvant alum. Current Opinion in Immunology 2014;28:1‐5. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0076] Oyewumi MO, Kumar A, Cui Z. Nano‐microparticles as immune adjuvants: correlating particle sizes and the resultant immune responses. Expert Review of Vaccines 2010;9:1095‐107. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0077] Poddighe D, Castelli L, Marseglia GL, Bruni P. A sudden onset of a pseudo‐neurological syndrome after HPV‐16/18 AS04‐adjuvated vaccine: might it be an autoimmune/inflammatory syndrome induced by adjuvants (ASIA) presenting as a somatoform disorder?. Immunologic Research 2014;60:236‐46. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0078] Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta‐analysis. Controlled Clinical Trials 1997;18:580‐93; discussion 661‐6. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0079] Ramanathan VD, Badenoch‐Jones P, Turk JL. Complement activation by aluminium and zirconium compounds. Immunology 1979;37:881‐8. [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0080] Rambe DS, Giudice G, Rossi S, Sanicas M. Safety and mechanism of action of licensed vaccine adjuvants. International Current Pharmaceutical Journal 2015;4:420‐31. [Google Scholar]

[CD012805-bbs2-0081] Reed SG, Orr MT, Fox CB. Key roles of adjuvants in modern vaccines. Nature Medicine 2013;19:1597‐608. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0082] Reinke CM, Breitkreutz J, Leuenberger H. Aluminium in over‐the‐counter drugs: risks outweigh benefits?. Drug Safety 2003;26:1011‐25. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0083] Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager 5 (RevMan 5). Version 5.3. Copenhagen: Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

[CD012805-bbs2-0084] Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0085] Salvador A, Igartua M, Hernandez RM, Pedraz JL. An overview on the field of micro‐ and nanotechnologies for synthetic peptide‐based vaccines. Journal of Drug Delivery 2011;2011:181646. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0086] Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta‐epidemiological studies. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0087] Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0088] Scheller NM, Svanstrom H, Pasternak B, Arnheim‐Dahlstrom L, Sundstrom K, Fink K, et al. Quadrivalent HPV vaccination and risk of multiple sclerosis and other demyelinating diseases of the central nervous system. JAMA 2015;313:54‐61. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0089] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408‐12. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0090] Sharma HS, Hussain S, Schlager J, Ali SF, Sharma A. Influence of nanoparticles on blood‐brain barrier permeability and brain edema formation in rats. Acta Neurochirurgica 2010;106(Suppl):359‐64. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0091] Shima F, Uto T, Akagi T, Baba M, Akashi M. Size effect of amphiphilic poly(gamma‐glutamic acid) nanoparticles on cellular uptake and maturation of dendritic cells in vivo. Acta Biomaterialia 2013;9:8894‐901. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0092] Son YJ, Kim H, Leong KW, Yoo HS. Multifunctional nanorods serving as nanobridges to modulate T cell‐mediated immunity. ACS Nano 2013;7:9771‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0093] StataCorp LCC. STATA Data Analysis and Statistical Software. Texas, USA: StataCorp LCC, 2014.

[CD012805-bbs2-0094] Strugnell R, Zepp F, Cunningham A, Tantawichien T. Vaccine antigens. Perspectives in Vaccinology 2011;1:61‐88. [Google Scholar]

[CD012805-bbs2-0095] Thiele L, Rothen‐Rutishauser B, Jilek S, Wunderli‐Allenspach H, Merkle HP, Walter E. Evaluation of particle uptake in human blood monocyte‐derived cells in vitro. Does phagocytosis activity of dendritic cells measure up with macrophages?. Journal of Controlled Release 2001;76:59‐71. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0096] Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses?. International Journal of Epidemiology 2009;38:276‐86. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0097] Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). www.ctu.dk/tsa/files/tsa_manual.pdf (accessed 13 January 2017).

[CD012805-bbs2-0098] Tomljenovic L, Shaw CA. Aluminum vaccine adjuvants: are they safe?. Current Medicinal Chemistry 2011;18:2630‐7. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0099] Tomljenovic L, Shaw CA. Too fast or not too fast: the FDA's approval of Merck's HPV vaccine Gardasil. Journal of Law, Medicine & Ethics 2012;40:673‐81. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0100] Tritto E, Mosca F, Gregorio E. Mechanism of action of licensed vaccine adjuvants. Vaccine 2009;27:3331‐4. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0101] Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. Version 0.9.5.10 Beta. Copenhagen: Copenhagen Trial Unit, 2011.

[CD012805-bbs2-0102] Turner RM, Bird SM, Higgins JP. The impact of study size on meta‐analyses: examination of underpowered studies in Cochrane reviews. PLOS One 2013;8:e59202. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0103] Ward BJ. Vaccine adverse events in the new millennium: is there reason for concern?. Bulletin of the World Health Organization 2000;78:205‐15. [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0104] Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61:64‐75. [DOI] [PubMed] [Google Scholar]

[CD012805-bbs2-0105] Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random‐effects model meta‐analyses. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0106] Wetterslev J, Jakobsen JC, Gluud C. Trial Sequential Analysis in systematic reviews with meta‐analysis. BMC Medical Research Methodology 2017;17:39. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0107] Whitney CG, Zhou F, Singleton J, Schuchat A. Benefits from immunization during the vaccines for children program era ‐ United States, 1994‐2013. Morbidity and Mortality Weekly Report 2014;63:352‐5. [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0108] World Health Organization. Human papillomavirus vaccines: WHO position paper. Weekly Epidemiological Record 2014;43(89):462‐92. [Google Scholar]

[CD012805-bbs2-0109] World Health Organization Expert Committee. Annex 2. Guidelines on the nonclinical evaluation of vaccine adjuvants and adjuvanted vaccines. www.who.int/biologicals/areas/vaccines/TRS_987_Annex2.pdf?ua=1 (accessed 13 January 2017).

[CD012805-bbs2-0110] World Health Organization. Polio vaccines: WHO position paper. Weekly Epidemiological Record 2016;12(91):145–68. [Google Scholar]

[CD012805-bbs2-0111] World Health Organization. Measles. Fact Sheet. www.who.int/wer/2014/wer8943/en/ (accessed 1 August 2017).

[CD012805-bbs2-0112] World Health Organization. WHO recommendations for routine immunization ‐ summary tables. www.who.int/immunization/policy/immunization_tables/en/ (accessed 1 August 2017).

[CD012805-bbs2-0113] Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ (Clinical Research Ed.) 2008;336:601‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD012805-bbs2-0114] Yokel RA. The toxicology of aluminum in the brain: a review. Neurotoxicology 2000;21:813‐28. [PubMed] [Google Scholar]

PERMALINK

Aluminium adjuvants used in vaccines versus placebo or no intervention

Snezana Djurisic

Janus C Jakobsen

Sesilje B Petersen

Mette Kenfelt

Christian Gluud

Abstract

Background

Description of the condition

Description of the intervention

How the intervention might work

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Types of outcome measures

Primary outcomes

Secondary outcomes

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of treatment effect

Dichotomous outcomes

Continuous outcomes

Unit of analysis issues

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Meta‐analysis

Trial Sequential Analysis

Primary outcomes

Secondary outcomes

Assessment of significance

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Feedback

Will this review investigate/consider the impact of the unnaturally high antibodies induced by HPV vaccination?, 15 January 2018

Summary

Reply

Contributors

Aluminium adjuvants used in vaccines versus placebo or no intervention, 5 February 2018

Summary

Reply

Contributors

Aluminium adjuvants used in vaccines versus placebo or no intervention, 5 February 2018

Summary

Reply

Contributors

Aluminium adjuvants used in vaccines versus placebo or no intervention, 14 March 2018

Summary

Reply

Contributors

Aluminium adjuvants used in vaccines versus placebo or no intervention, 16 March 2018

Summary

Reply

Contributors

Aluminium adjuvants used in vaccines versus placebo or no intervention, 1 June 2018

Summary

Aluminium adjuvants used in vaccines versus placebo or no intervention, 1 June 2018

Summary

Acknowledgements

Appendices

Appendix 1. Preliminary search strategies

What's new

History

Contributions of authors

Sources of support

Internal sources

External sources

Declarations of interest

References