Skip to main content
PMC home page
The Cochrane Database of Systematic Reviews logoLink to The Cochrane Database of Systematic Reviews
. 2018 Jul 2;2018(7):CD013086. doi: 10.1002/14651858.CD013086

Aluminium adjuvants used in vaccines

Snezana Djurisic 1, Janus C Jakobsen 2, Sesilje B Petersen 3, Mette Kenfelt 4, Sarah Louise Klingenberg 5, Christian Gluud 2,✉
PMCID: PMC6373706

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To assess the benefits and harms of aluminium adjuvants used in a vaccine or an excipient versus the same vaccine or excipient, but having a different type of aluminium adjuvant formulation, or a different concentration, or with a different particle size.

Background

Description of the condition

The spread of diseases such as smallpox, polio, measles, and tetanus has been restricted through the numerous mass vaccination programmes since the 1960s (Delany 2014; Whitney 2014). In the late 1940s and 1950s, prior to public vaccinations, the poliovirus caused infantile paralysis associated with high mortality in hundreds of thousands of children (Global Polio Eradication Initiative 2016; WHO 2016a). The measles virus was responsible for post‐infectious encephalomyelitis in 1 per 1000 infected individuals, leaving most with permanent impairment of the central nervous system (Miller 1964; CDC 2017). Today, polio is nearly eradicated (WHO 2016a), and global measles death has decreased by 79%, with an estimated 17.1 million deaths prevented from 2000 to 2014 (WHO 2016b). The effectiveness of vaccinations has been proven repeatedly since the first introduction of the cowpox vaccine in the 18th century (Delany 2014; Whitney 2014). In fact, vaccination is considered one of the major triumphs of modern medicine (Delany 2014; Whitney 2014). The aim of vaccination is to prevent infectious diseases (Delany 2014; Whitney 2014), and to eradicate highly contagious and deadly virus diseases worldwide.

Current routine vaccine programmes recommended by the World Health Organization (WHO) include bacillus Calmette–Guérin (BCG) vaccine against tuberculosis; vaccines against hepatitis; polio; diphtheria, tetanus, and pertussis (DTP); haemophilus influenza type B; pneumococcal bacteria; rotavirus; measles; mumps; rubella; and human papilloma virus (HPV) (WHO 2017). Additional programmes are proposed for certain regions, high‐risk populations, and for fighting pathogens with certain characteristics (e.g. Bacillus anthracis causing anthrax).

One of the latest programmes added to the mass vaccination portfolio is the HPV vaccination programme launched in the USA in 2006 (WHO 2014). HPV causes cervical cancer, the second most common cancer in women (WHO 2014), and HPV vaccination aims to prevent it. Over 60 countries included the HPV vaccine in their routine vaccinations after its market approval (Bruni 2016). The HPV vaccines have been assessed for efficacy (immunogenicity) in clinical trials, and approved on the basis of their ability to raise a potent immune response against HPV, or their ability to prevent persistent HPV infections (Harper 2017). However, the time length by which vaccination can sustain a sufficient amount and repertoire of protective antibodies, and the actual effectiveness of HPV vaccines regarding its ability to reduce the incidence of cervical cancer, remains to be ultimately demonstrated (Kjaer 2018). Early results show benefits in HPV‐vaccinated individuals on cervical pathology (Harper 2017; Kjaer 2018). However, concerns have been raised about adverse events possibly related to the HPV vaccines. Since the USA approval of Gardasil® in 2006 and up until 2012, a total of 21,265 adverse events was reported to the national Vaccine Adverse Event Reporting System (VAERS) (Tomljenovic 2012). HPV vaccines have been associated with more reported adverse events than other types of vaccines (Tomljenovic 2012). In the European Union, the European Medical Agency (EMA) has received similar reports, but found no scientific evidence for an association (EMA 2015). Several observational studies also failed to identify associations with clinical diagnoses (Klein 2012; Arnheim‐Dahlström 2013; Donegan 2013; Grimaldi‐Bensouda 2014; Scheller 2015), but reasons to oppose these findings have been proposed (Brinth 2015a; Brinth 2015b; Dyer 2015; Gøtzsche 2016a; Gøtzsche 2016b). The symptoms reported following HPV vaccination are varied and include headache, orthostatic intolerance, fatigue, cognitive dysfunction, blurred vision, feeling bloated, abdominal pain, light sensitivity, and involuntary muscle activity (Brinth 2015a; Brinth 2015b). Despite the consistency in reported symptoms, they do not fit into a well‐defined category of diseases or diagnoses, but rather present themselves as a constellation of nonspecific symptoms (Brinth 2015a; Brinth 2015b). Consequently, the observational studies, which based their results on registered diagnoses, may have excluded an important fraction of eligible participants with unclear adverse symptoms. Most young girls that claim to suffer from adverse events following HPV vaccination receive no clinical diagnosis, and are therefore unlikely to appear in medical registers. Moreover, the randomised clinical trials on HPV vaccines, which formed the basis for the safety assessment, have been blamed for not using true placebo (e.g. placebo not containing any vaccine or adjuvants) as the control intervention (Exley 2011). Furthermore, HPV vaccine trials offered HPV vaccination to the groups receiving control interventions after the trial had reached the scheduled follow‐up, which effectively precluded any meaningful long‐term follow‐up comparing vaccinated groups versus control groups (FUTURE II Study Group 2007; Paavonen 2009).

Description of the intervention

Vaccination mimics infection in the body and causes activation of a potent immune response (Coffman 2010; Kool 2012; O'Hagan 2012; Oleszycka 2014). The purpose of vaccination is to induce long‐lasting protection against a given pathogen while avoiding unwanted adverse effects. There are two types of vaccines: traditional vaccines that contain vaccine antigens and excipients (substance formulated alongside the active vaccine ingredient) and do not use adjuvants (substance that aid the immune response to an antigen); and a newer generation of vaccines, which typically require the addition of adjuvants (Coffman 2010; O'Hagan 2012). Vaccine antigens may comprise whole attenuated pathogens, pathogen components, virus‐like particles, or genetic material of the pathogen (Strugnell 2011).

Like other medicinal products, vaccines undergo preclinical testing for safety, the risk of inducing cancer, the ability to induce immune response, and for overall efficacy before they are licensed and marketed. However, rare adverse events or adverse events with delayed onset are not easily detected during the relatively short duration of most preclinical and clinical phase studies. As proven over the years, safety surveillance in the general population post‐marketing is essential (Ward 2000; Chen 2006). As an example, the childhood measles‐mumps‐rubella (MMR) vaccine was introduced in the late 1960s as a mixture of three live attenuated viruses, administered via injection (Offit 2007). Over time, doubts about its safety were raised when serious fever seizures, meningitis, and allergic reactions were reported among vaccinated people (Kimura 1996; Dourado 2000; Ward 2000). In Japan, a nationwide surveillance programme launched in the early 1990s screened more than 38,000 children vaccinated with four different Urabe‐containing MMR vaccines (Kimura 1996). Serious adverse events included convulsions and aseptic meningitis, and the incidence was shown to be linked to different vaccine strains of mumps virus (Kimura 1996). During the same time period, Brazil experienced a mass outbreak of aseptic meningitis following a Urabe‐containing MMR vaccine with an estimated risk of 1 in 14,000 doses (Dourado 2000). As another example, the DTP vaccine was licensed in the late 1940s as a preparation of three different antigen components (trivalent vaccine) adsorbed to aluminium salt. It was suspected to cause acute encephalopathy and chronic nervous system dysfunction (Cowan 1993). Reports, prepared by the Institute of Medicine (IOM) in the USA, concluded that the evidence was insufficient to indicate a causal relationship between the DTP vaccine and acute or permanent neurological damage (Cowan 1993). In 2000, the World Health Organization (WHO) published a critical bulletin on vaccine safety, including an overview of serious vaccine‐associated adverse events for which causality had been established or was highly likely (Ward 2000). Among several vaccine‐specific serious adverse events, they found a causal relationship between disease dissemination, death, or severe allergic reactions on one hand and vaccines against DTP, hepatitis, MMR, and polio on the other (Ward 2000).

The benefit of a vaccine is measured according to the degree of protection it confers against a given pathogen. Two means of measuring protection exist: efficacy and effectiveness. While efficacy studies determine if a vaccine works under controlled conditions (degree of immunogenicity or ability to provoke an immune response in clinical trials), effectiveness studies are designed to determine if vaccination helps people (fewer diseased individuals in a long‐term follow‐up clinical trial or real‐life scenario) (Fedson 1998). In addition to its intended effect, a vaccine may by accompanied by one or more harmful effects upon administration. Harms may be considered non‐serious (e.g. mild, transient headache) or serious (e.g. causing hospitalisation or death) and they may appear shortly after vaccine administration (e.g. pain at the injection site) or some time after (e.g. autoimmune responses). Vaccine toxicity, efficacy, and effectiveness may originate from, or depend on, a plethora of factors, including the vaccine components (e.g. the antigen itself, the excipient, or the adjuvant); interaction between different vaccine components; vaccine manufacture; overall vaccine composition; route of administration; dose; and number of booster vaccinations (Kocourkova 2017).

Adjuvants are added to vaccines that employ weak antigens to enhance the ability to provoke an immune response (recombinant subunit antigens, protein toxins, or inactivated viruses) and improve the overall potency of the vaccine (O'Hagan 2009; Coffman 2010). Adjuvants may also pose other benefits, such as reducing the frequency of vaccination and the dose of antigen per vaccine, and some may provide ‘cross‐clade immunity' (i.e. immunity against different clades of viruses or of bacteria descending from different ancestors) or improve the stability of the final vaccine formulation (Carter 2010; Reed 2013). Currently, five adjuvants with completely different mechanism of action are approved for use in vaccine production. These include: aluminium salts (EU, USA), MF59 (EU), AS03 (EU), AS04 (EU, USA), and virosomes (EU) (Rambe 2015). Aluminium salts, also referred to as ‘alum', are most widely used and encompass aluminium potassium sulphate, aluminium hydroxide, aluminium phosphate, and amorphous aluminium hydroxyphosphate sulfate (Carter 2010). Virosomes consist of a lipid membrane incorporating virus‐derived proteins, while MF59 and AS03 are squalene‐based adjuvants and AS04 combines aluminium hydroxide with monophosphoryl lipid A.

Different insoluble aluminium salts have been used as vaccine adjuvants since 1926 (Glenny 1926). Aluminium potassium sulphate was the first used. However, because of poor reproducibility, it has been almost completely replaced by aluminium hydroxide and aluminium phosphate, as they can be prepared in a more standardised way and capture antigens by direct adsorption (Marrack 2009). Aluminium has been the standard adjuvant in vaccines such as those against diphtheria, tetanus, and pertussis, haemophilus influenza type B, pneumococcus conjugates, hepatitis A, and hepatitis B (Tritto 2009). More recently, aluminium was co‐formulated with vaccines against HPV in the form of AS04, containing aluminium hydroxide, and amorphous aluminium hydroxyphosphate sulfate. Amorphous aluminium hydroxyphosphate sulfate is commercially produced in nanometre scale, and represents one of the latest marketed aluminium adjuvants.

The mechanism of action of aluminium, like for most adjuvants, is poorly understood, and widespread beliefs change according to continuous new insights into immunology and physiochemical properties of aluminium (see ‘How the intervention might work') (Carter 2010; Tomljenovic 2011). Despite our incomplete understanding of its effects, the repeated use of aluminium in vaccines is justified by its apparent safety profile, ease of preparation, stability, potent immunostimulatory ability (O'Hagan 2009; Tritto 2009; Mbow 2010), and importantly, due to the lack of suitable alternatives.

How the intervention might work

Aluminium has no known biological or physiological role (Reinke 2003). It is absorbed into the blood through the gastrointestinal tract, and rapidly eliminated by the kidneys and the liver (Reinke 2003). While aluminium is generally considered safe and is regularly ingested in food and water, it can be toxic based on the concentration, chemical form, and the environment (Kisnieriené 2015). In the blood, aluminium is bound to transferrin with high affinity, where it competes with iron at the binding site (Kisnieriené 2015). Aluminium affects cellular processes and physiological functions (Kisnieriené 2015). For instance, aluminium competes with magnesium for membrane transporters; disturbs calcium metabolism; increases oxidative stress; binds to the phosphate groups of nucleoside di‐ and triphosphates; and binds to metal‐binding organic compounds (amino acids) and membrane lipids (Kisnieriené 2015). At high concentrations, aluminium predominantly accumulates in bone and brain tissue (Yokel 2000; Malluche 2002). Based on findings from animal and human studies, it is known to act as a powerful neurological toxicant and provoke toxic effects in foetuses and embryos if exposed during pregnancy (Reinke 2003). This is supported by recent data indicating that aluminium is able to cross the blood‐brain barrier by directly affecting the cerebral blood vessels (Chen 2008; Sharma 2010). Very high aluminium concentrations have been observed in histological brain samples from children with autism, potentially implicating aluminium in the pathogenesis of this disease (Mold 2017).

Despite its unclear biological role, aluminium seems to have an impact on the immune system, which has rendered it useful as a vaccine adjuvant (Tritto 2009; Kool 2012). Aluminium binds antigens with high affinity (antigen adsorption) and was originally thought to exert its function by forming a depot, which allows for a high antigen concentration at the site of injection, and a continuous desorption and dispersion of antigens from the aluminium particles (Kool 2012). Nowadays, aluminium is believed to exert its adjuvant effects by stimulating Th2‐type responses and antibody production through B cells activation (Grun 1989; Awate 2013), by activating the complement system, and by recruiting immune cells to the site of injection (Ramanathan 1979; Goto 1997; Awate 2013). At the injection site, aluminium promotes antigen uptake by specialised antigen‐presenting immune cells, termed dendritic cells, and dendritic cell maturation (Mannhalter 1985; Morefield 2005; Kool 2008).

One important aspect of adjuvants is the particle size, which seems to influence the immune response. Aluminium hydroxide adjuvant is comprised of particles with a dimension of 100 nm, while aluminium phosphate particles are around 50 nm (Hem 2007). In an aqueous (water) solution, particles of both aluminium salts aggregate to form 1 to 20 µm sized particulates (Hem 2007). This size is also known as microscale size. Aluminium hydroxide and aluminium phosphate can be produced in nanoscale size ≤ 200 nm (Issa 2014; Li 2014); so far, only amorphous (non‐crystalline solid) aluminium hydroxyphosphate sulfate is produced in nanoscale for use in vaccine preparations (Caulfield 2007; Lee 2012). The particle size is directly linked to the adsorption efficiency of antigens (Oyewumi 2010). Nanoscale aluminium particles can adsorb more antigens compared to traditional aluminium‐based adjuvants because of the higher surface‐area‐to‐volume ratio, and that they are more potent than traditional microparticles (Caulfield 2007; Salvador 2011; Li 2014). Moreover, the efficacy of particle uptake by the specialised antigen presenting dendritic cells in vitro and in vivo is inversely proportional to the particle size, with maximum efficiency for nanoscale particles < 100 nm (Foged 2005; Shima 2013). Dendritic cells scavenge and engulf particles less than 10 µm in diameter, having evolved to recognise pathogens of this size (Gupta 1995; Foged 2005). Other factors like structure, shape, and surface charge have also been demonstrated to greatly affect uptake by dendritic cells (Thiele 2001; Foged 2005; Bartneck 2012; Son 2013).

Why it is important to do this review

Jefferson 2004 attempted to assess the effects of aluminium adjuvants against placebo in a systematic review of studies available up to 2003. The authors examined "evidence of adverse events after exposure to aluminium‐containing DTP vaccines, alone or in combination, compared with identical vaccines that either did not contain aluminium salts or contained them in different concentrations". Based on three randomised clinical trials, four semi‐randomised trials, and one cohort study, which were all at high risk of bias, they were unable to demonstrate that aluminium adjuvant was responsible for serious or long‐lasting adverse events and surprisingly advised the ending of future research (Jefferson 2004). Despite this recommendation, we have embarked on a Cochrane Review of randomised clinical trials to assess the effects of aluminium adjuvants versus placebo or no intervention (Djurisic 2017). To our knowledge, no systematic review to date has assessed the benefits and harms of different aluminium adjuvants versus other aluminium adjuvants used in vaccines or excipients. This Cochrane Review aims to assess the benefits and harms of aluminium adjuvants used in a vaccine or their excipient versus the same vaccine or excipient but having a different formulation, concentration, or particle size of the aluminium adjuvant. Our aim is not to analyse the benefits and harms of any type of vaccine formulations for prevention of a specific disease. The results of our systematic reviews could influence future vaccine formulation and bring upon changes among policymakers and vaccine manufacturers to secure safe and efficient vaccines to people.

Objectives

To assess the benefits and harms of aluminium adjuvants used in a vaccine or an excipient versus the same vaccine or excipient, but having a different type of aluminium adjuvant formulation, or a different concentration, or with a different particle size.

Methods

Criteria for considering studies for this review

Types of studies

We will include randomised clinical trials irrespective of publication type, publication status, language. and year of publication. We will not specifically search for observational studies (quasi‐randomised studies; cohort studies; and patient series), as through their design they are subject to a number of potential problems that may bias their results regarding benefits of interventions. We will, however, provide a narrative account of harms in such studies if we identify valid observational studies during the literature search. This approach may be a weakness of our review, making us focus more on short‐term benefits and harms in randomised clinical trials with the risk of overlooking late and very rare adverse effects in observational studies (Storebø 2018).

Types of participants

We will include all trial participants regardless of sex, age, ethnicity, diagnosis, comorbidity, and country of residence.

Types of interventions

Experimental

A vaccine or excipient including one type of an aluminium adjuvant formulation (for example, aluminium potassium sulphate, aluminium hydroxide, aluminium phosphate, aluminium hydroxyphosphate sulfate, and others) at a certain concentration or of a certain particle size.

Control

The same vaccine or excipient including a different type of an aluminium adjuvant formulation, or at a different concentration, or with a different particle size.

We will accept co‐interventions if delivered equally to the trial comparison groups.

Types of outcome measures

Primary outcomes

  • All‐cause mortality (as reported by trialists or measured by administrative data).

  • Proportion of participants with disease going to be prevented by the vaccine (as defined per individual trial).

  • Proportion of participants with one or more serious adverse events. We will define a serious adverse event as any untoward medical occurrence that results in death, is life‐threatening, requires hospitalisation or prolongation of existing hospitalisation, or results in persistent or significant disability or incapacity (ICH‐GCP 1997). We will also analyse each serious adverse event separately in exploratory analysis.

We will use the trial results reported at maximum follow‐up.

Secondary outcomes

  • Health‐related quality of life (as measured by interviews or self‐report using any standardised continuous scale).

  • Proportion of participants with one or more non‐serious adverse events (defined as any adverse event not classified as a serious adverse event). We will also analyse each non‐serious adverse event separately in exploratory analysis (see below).

Exploratory outcomes

  • Serological response (as defined by trialists, e.g. measured with ELISA, agglutination, precipitation, complement‐fixation, fluorescent antibodies, chemiluminescence, or similar).

  • Serious and non‐serious adverse events analysed separately (see above).

We will use the trial results reported at maximum follow‐up to achieve maximum precision and power, and will collect data at the end of treatment and end of follow‐up.

Search methods for identification of studies

Electronic searches

We will search the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid, Embase Ovid, BIOSIS (Web of Science), LILACS (Bireme), Science Citation Expanded (Web of Science) and Conference Proceedings Citation Index – Science (Web of Science) (Royle 2003). Appendix 1 gives the preliminary search strategies with the expected time spans of the searches.

In addition, we will search the Chinese Biomedical Literature Database (CBM), China Network Knowledge Information (CNKI), Chinese Science Journal Database (VIP), and Wanfang Database.

Searching other resources

To identify grey literature and ongoing or unpublished trials, we will also search Google Scholar, The Turning Research into Practice (TRIP) Database, ClinicalTrial.gov (www.clinicaltrials.gov/), the European Medicines Agency (EMA) (www.ema.europa.eu/ema/), the WHO International Clinical Trial Registry Platform (www.who.int/ictrp), the US Food and Drug Administration (FDA) (www.fda.gov), and pharmaceutical company trial registries.

We will review bibliographic references of identified randomised clinical trials and review articles to identify randomised clinical trials missed during the electronic searches.

Data collection and analysis

We will perform the review following Cochrane recommendations (Higgins 2011a). We will perform the analyses using Review Manager 5 (RevMan 5) (RevMan 2014), Stata (Stata 2014), and Trial Sequential Analysis version 0.9.5.6 beta (Thorlund 2011; TSA 2011). We will present a table describing the types of adverse events (serious or non‐serious) reported for each trial.

Selection of studies

Two review authors (SD and SLK) will independently screen titles and abstracts for inclusion of potentially eligible trials in Covidence. We will code included studies as either ‘retrieve' (eligible or potentially eligible) or ‘do not retrieve'. A third review author (JCJ or CG) will arbitrate any unresolved disagreements. The selected review author pair will collect full‐text trial reports/publications, and independently screen the full‐texts and identify trials for inclusion. We will report reasons for exclusion of the ineligible studies. We will solve any disagreement through discussion, or, if required, by consulting a third review author (JCJ or CG). We will identify and exclude duplicates, and collate multiple reports of the same trial. We will record the selection process in sufficient detail to complete a PRISMA flow diagram and ‘Characteristics of excluded studies' table.

Data extraction and management

Two review authors will independently extract and validate data using data extraction forms designed for the purpose. If a trial is identified as relevant by one review author but not by the other, the authors will discuss the reasoning behind their assessment. If the two review authors do not reach an agreement, a third review author (JCJ or CG) will arbitrate.

Assessment of risk of bias in included studies

Methodological studies indicate that trials with unclear or inadequate methodological quality may be associated with risk of bias (systematic error) when compared to trials using adequate methodology (Schulz 1995; Moher 1998; Kjaergard 2001; Gluud 2006; Wood 2008; Higgins 2011a; Hróbjartsson 2012; Savović 2012a; Savović 2012b; Hróbjartsson 2013; Hróbjartsson 2014; Lundh 2017). Such bias may lead to overestimation of intervention benefits and underestimation of harms.

Two review authors (SD and SBP) will independently assess the risk of bias of each included trial in accordance with the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We will use the below definitions in the ‘Risk of bias' assessments (Schulz 1995; Moher 1998; Kjaergard 2001; Wood 2008; Gluud 2006; Higgins 2011a; Hróbjartsson 2012; Savović 2012a; Savović 2012b; Hróbjartsson 2013; Hróbjartsson 2014; Lundh 2017).

Allocation sequence generation

  • Low risk of bias: sequence generation was achieved using computer random number generation or a random number table. Drawing lots, tossing a coin, shuffling cards, and throwing dice were adequate if performed by an independent person not otherwise involved in the trial.

  • Unclear risk of bias: the method of sequence generation was not specified.

  • High risk of bias: the sequence generation method was not random or only quasi‐randomised. We will only use these studies for the assessment of harms and not of benefits.

Allocation concealment

  • Low risk of bias: the allocation sequence was described as unknown to the investigators. Hence, the participants' allocations could not have been foreseen in advance of, or during, enrolment. Allocation was controlled by a central and independent randomisation unit, an onsite locked computer, identical‐looking numbered sealed opaque envelopes, drug bottles or containers prepared by an independent pharmacist, or an independent investigator.

  • Unclear risk of bias: it was unclear if the allocation was hidden or if the block size was relatively small and fixed so that intervention allocations may have been foreseen in advance of, or during, enrolment.

  • High risk of bias: the allocation sequence was likely to be known to the investigators who assigned the participants.

Blinding of participants and treatment providers

  • Low risk of bias: it was described that both participants and treatment providers were blinded to treatment allocation.

  • Unclear risk of bias: it was unclear if participants and treatment providers were blinded, or the extent of blinding was insufficiently described.

  • High risk of bias: no blinding or incomplete blinding of participants and treatment providers was performed.

Blinding of outcome assessment

  • Low risk of bias: it was mentioned that outcome assessors were blinded and this was described.

  • Unclear risk of bias: it was not mentioned if the outcome assessors were blinded, or the extent of blinding was insufficiently described.

  • High risk of bias: no blinding or incomplete blinding of outcome assessors was performed.

Incomplete outcome data

  • Low risk of bias: missing data were unlikely to make intervention effects depart from plausible values. This could either be: 1) there were no dropouts or withdrawals; or 2) the numbers and reasons for the withdrawals and dropouts for all outcomes were clearly stated and could be described as being similar in both groups, and the trial handled missing data appropriately in an intention‐to‐treat analysis using proper methods (e.g. multiple imputations). Generally, the trial is judged to be at a low risk of bias due to incomplete outcome data if dropouts are less than 5%. However, the 5% cut‐off is not definitive.

  • Unclear risk of bias: there was insufficient information to assess whether missing data were likely to induce bias on the results.

  • High risk of bias: the results were likely to be biased due to missing data either because the pattern of dropouts could be described as being different in the two intervention groups or the trial used improper methods in dealing with the missing data (e.g. last observation carried forward).

Selective outcome reporting

  • Low risk of bias: a protocol was published (trial registries or similar) before randomisation began and all outcome results were reported adequately.

  • Unclear risk of bias: no protocol was published.

  • High risk of bias: the outcomes in the protocol were not reported on.

Vested interest bias

  • Low risk of bias: it was described that the trial was not sponsored by any pharmaceutical company, any person, or any group with a financial or other interest in a certain result of the trial.

  • Unclear risk of bias: it was unclear how the trial was sponsored.

  • High risk of bias: the trial was sponsored by a pharmaceutical company, a person, or a group with a certain financial or other interest in a given result of the trial.

Other bias

  • Low risk of bias: the trial appeared to be free of other bias domains that could put it at risk of bias.

  • Unclear risk of bias: the trial may or may not have been free of other domains that could put it at risk of bias.

  • High risk of bias: there were other factors in the trial that could put it at risk of bias.

Overall risk of bias

  • Low risk of bias: we will classify the outcome result as at overall ‘low risk of bias' only if all of the bias domains described in the above paragraphs are classified as at low risk of bias.

  • High risk of bias: we will classify the outcome result as at ‘high risk of bias' if any of the bias risk domains described above are classified as at ‘unclear' or ‘high risk of bias'.

Measures of treatment effect

Dichotomous outcomes

We will calculate risk ratios (RR) with 95% confidence interval (CI) for dichotomous outcomes, as well as the Trial Sequential Analysis‐adjusted CI (see below).

Continuous outcomes

We will calculate the mean difference (MD) with 95% CI and Trial Sequential Analysis‐adjusted CI for continuous outcomes. If the included studies used various scales assessing comparable symptoms, we will calculate the standardised mean difference (SMD) with 95% CI. We can then calculate such data back to MD for a preferred scale, if needed.

Unit of analysis issues

We will include data from randomised clinical trials where participants are individually randomised to one of two or more intervention groups. We will collect and analyse single measurements for each outcome from each participant.

In cross‐over trials, we will only include the period before the cross‐over event.

We will include both individual and cluster‐randomised clinical trials and analyse the results separately (Higgins 2011b). Methods have been developed so that results of individual and cluster‐randomised clinical trials may be meta‐analysed together. However, as participants from different clusters will, almost always, have different baseline characteristics and because it has repeatedly been shown that it is not possible with certainty to adjust the statistical analysis for such baseline differences, we will first analyse results of individual and cluster‐randomised clinical trials separately (Deeks 2003). If the intervention effects do not differ, we will try to collapse them.

Dealing with missing data

We will contact investigators or study sponsors to obtain any missing data.

If the included studies do not report standard deviations (SD), we will calculate them using data from the trial, if possible. We will not impute missing values for any outcomes in our primary analysis. In our sensitivity analysis for dichotomous and continuous outcomes, we will impute data (see ‘Sensitivity analysis').

Assessment of heterogeneity

We will first visually investigate Forest plots to assess the risk of statistical heterogeneity. We will also assess the presence of statistical heterogeneity using the Chi2 test (threshold P < 0.1) and measure the quantities of heterogeneity using the I2 statistic (Higgins 2002; Higgins 2003). Regarding heterogeneity, we will interpret the I2 statistic value thresholds according to the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b) as follows: 0% to 40%: might not be important; 30% to 60%: may represent moderate heterogeneity; 50% to 90%: may represent substantial heterogeneity; 75% to 100%: may represent considerable heterogeneity.

Assessment of reporting biases

We will assess reporting bias using funnel plots where 10 or more trials per comparison are included. Symmetry or asymmetry of each funnel plot will enable assessment of the risk of bias. For dichotomous outcomes, we will assess asymmetry using the Harbord test (Harbord 2006). For continuous outcomes, we will apply the regression asymmetry test (Egger 1997).

Data synthesis

Meta‐analysis

We will conduct this systematic review according to the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b), in accordance with Keus 2010, and according to the eight‐step procedure for validation of meta‐analytic results in systematic reviews as suggested by Jakobsen 2014. We will meta‐analyse data using the statistical software in RevMan 5 (RevMan 2014) and Trial Sequential Analysis (TSA 2011).

Trial Sequential Analysis

Cumulative meta‐analyses are at risk of producing random errors due to sparse data and multiple testing of accumulating data (Pogue 1997; Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009; Higgins 2011a; Wetterslev 2017). Therefore Trial Sequential Analysis, TSA 2011, can be applied to control these risks (Thorlund 2011). The required information size (that is the number of participants and number of trials needed in a meta‐analysis to detect or reject a certain intervention effect) can be calculated in order to control random errors (Wetterslev 2008; Wetterslev 2009; Wetterslev 2017). The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk (RR) reduction, and the heterogeneity of the meta‐analysis (Wetterslev 2008; Wetterslev 2009; Turner 2013; Jackson 2017; Wetterslev 2017). Trial Sequential Analysis enables testing for significance to be conducted each time a new trial is included in the meta‐analysis. On the basis of the required information size, trial sequential monitoring boundaries can be constructed. This enables one to determine the statistical inference concerning cumulative meta‐analysis that has not yet reached the required information size (Wetterslev 2008; Wetterslev 2017).

If the trial sequential monitoring boundary for benefit or harm is crossed before reaching the calculated required information size, we may conclude that sufficient evidence is collected to validly assess benefit or harm, and that inclusion of additional trial data may be redundant. In contrast, if the boundaries for benefit or harm are not crossed, we may conclude that further trials are necessary before a certain intervention effect can be evaluated. Trial Sequential Analysis also allows for assessment of the sufficiency of evidence for rejecting a postulated intervention effect. A lack of effect is evident if the cumulative Z‐score crosses the trial sequential monitoring boundaries for ‘futility' (the ability of a systematic review of clinical trials to reject a certain postulated intervention effect).

We will make relatively conservative estimations of the anticipated intervention effect to control the risks of random error (Jakobsen 2014). Large anticipated intervention effects lead to small required information sizes, and the thresholds for significance will be less strict after the information size has been reached (Jakobsen 2014).

We will analyse all primary and secondary outcomes using Trial Sequential Analysis. These analyses will allow us to calculate the Trial Sequential Analysis‐adjusted CIs based on the assumptions described below.

Primary outcomes

We will estimate the diversity‐adjusted required information size (Wetterslev 2009; Wetterslev 2017), based on the proportion of participants with an outcome in the control group of our meta‐analysis. We will use an alpha of 2.5%, a beta of 10%, and the diversity suggested by the trials in the meta‐analysis (Jakobsen 2014).

As anticipated intervention effects for the primary outcomes in the Trial Sequential Analysis, we will use the following.

  • All‐cause mortality: a relative risk reduction of 20% and the observed incidence of mortality in the control group.

  • Disease: a relative risk reduction of 20% and the observed proportion of participants with disease in the control group.

  • Serious adverse events: a relative risk reduction of 20% and the observed proportion of serious adverse events in the control group.

Secondary outcomes

We will estimate the diversity‐adjusted required information size (Wetterslev 2009; Wetterslev 2017), based on the proportion of participants with an outcome in the control group when analysing dichotomous outcomes, and we will use the observed SD in the meta‐analysis when analysing continuous outcomes. We will use an alpha of 3.3%, a beta of 10%, and the diversity suggested by the trials in the meta‐analysis (Jakobsen 2014).

As anticipated intervention effects for the secondary outcomes in the Trial Sequential Analysis we will use the following relative risk reductions or increases.

  • Quality of life: observed SD divided by 2.

  • Non‐serious adverse events: a relative risk reduction of 20%.

Exploratory outcomes

As anticipated intervention effects for the exploratory outcomes in the Trial Sequential Analysis, we will use the following relative risk reductions or increases.

  • Serological response: a relative risk reduction of 20% and the observed proportion of participants with no serological response in the control group.

We will use an alpha of 5.0%, a beta of 10%, and the diversity suggested by the trials in the meta‐analysis (Jakobsen 2014). We will include particle size (nanosize or microsize as described by trialist or manufacturer) as a covariate in meta‐regression to assess whether particle size influences the effect of aluminium adjuvant administration on outcomes.

Assessment of significance

We will assess intervention effects using both random‐effects (DerSimonian 1986) and fixed‐effect meta‐analyses (DeMets 1987). We will choose the more conservative point estimate of the two, comprised by the estimate closest to zero effect, for assessment of significance (Jakobsen 2014). If the two estimates are comparable, we will use the estimate with the widest CI. For analysis of three primary outcomes, we will consider a P value of less than 0.025 to be significant (Jakobsen 2014), as this will secure a family‐wise error rate (FWER) below 0.05. We will apply an eight‐step procedure to assess if the results from the meta‐analyses have passed the thresholds for significance (Jakobsen 2014).

We will present a table describing all types of serious adverse events for each trial.

Subgroup analysis and investigation of heterogeneity

We will perform the following subgroup analyses on a trial level.

  • Trials at high risk of bias compared to trials at low risk of bias. We will also assess the impact of bias according to the ‘Blinding of outcome assessment', ‘Incomplete outcome data', and ‘Selective outcome reporting' domains for each outcome. This will enable us to assess the bias risk for each outcome result in addition to each trial.

  • Trials grouped according to one formulation of aluminium adjuvant compared to different formulations of aluminium adjuvants.

  • Trials grouped according to one concentration of aluminium adjuvant compared to other concentrations of the aluminium adjuvants.

  • Trials grouped according to one particle size of aluminium adjuvant compared to other particle sizes of the aluminium adjuvants.

  • Trials grouped according to age of participants (newborns, children, adolescents, adults, or elderly) as described by trialists.

  • Trials grouped according to types of participants: healthy participants compared to participants with co‐morbidities.

  • Trials grouped according to types of vaccine.

  • Trials grouped according to trials with different duration of follow‐up: short‐term (1 to 30 days after last administration); medium‐term (1 to 12 months after last administration); and long‐term (more than one year).

We will base subgroup analyses on pair‐wise comparisons between each type of aluminium adjuvant against the most commonly used form of aluminium adjuvant being the aluminium hydroxide. If aluminium hydroxide is not the choice of adjuvant for given a trial, then we will perform a pair‐wise comparison against the next most commonly‐used form.

Sensitivity analysis

A: to assess the potential impact of the missing data for dichotomous outcomes, we will perform the following analyses.

  • ‘Best‐worst‐case’ scenario: we will assume that all participants lost to follow‐up in the experimental group survived and did not have a serious adverse event; and all those with missing outcomes in the control group had a serious adverse event and did not survive.

  • ‘Worst‐best‐case’ scenario: we will assume that all participants lost to follow‐up in the experimental group did not survive and had a serious adverse event; and all those with missing outcomes in the control group survived and had no serious adverse event.

We will present results from both scenarios.

B: we will address missing data for continuous outcomes by calculating a ‘beneficial' and a ‘harmful' outcome. We will base the ‘beneficial' outcome on the group mean plus two SDs (and one SD), and the ‘harmful’ outcome on the group mean minus two SDs (and one SD) (Jakobsen 2014).

C: we will assess potential impact of missing SDs for continuous outcomes with the following sensitivity analyses. Where SDs are missing and not possible to calculate, we will be impute SDs from trials with similar populations and low risk of bias. If we cannot find such trials, we will impute SDs from trials with a similar population.

‘Summary of findings' table

We will construct the ‘Summary of findings' tables using GRADEpro software (GRADEpro 2015). We will use the GRADE system to assess the quality of all the primary and secondary outcomes (Primary outcomes; Secondary outcomes; Guyatt 2008). This system appraises the quality of a body of evidence based on the extent to which one can be confident that an estimate of effect or association reflects the item being assessed. The quality measure of a body of evidence considers within‐trial risk of bias, the indirectness of the evidence, heterogeneity of the data, imprecision of the effect estimates, and risk of publication bias. We will base our primary ‘Summary of findings' tables and conclusions on the results from pair‐wise comparisons (one aluminium type versus another aluminium type) of trials at a low risk of bias in all bias risk domains if bias seems to have an effect on the estimates (Schulz 1995; Moher 1998; Kjaergard 2001; Gluud 2006; Wood 2008; Savović 2012a; Savović 2012b; Lundh 2017).

Feedback

Aluminium adjuvants used in vaccines, 31 December 2018

Summary

Article: Aluminium adjuvants used in vaccines Article type: Protocol Review Group: Hepato‐Biliary DOI: 10.1002/14651858.CD013086

Comment : "Vaccination mimics infection in the body"

That is only partially true even for live attenuated vaccines.(1) "Very high aluminium concentrations have been observed in histological brain samples from children with autism, potentially implication aluminium in the pathogenesis of this disease (Mold 2017)." As you point out, aluminum adjuvants stimulate a Th2 (allergic) response. Vaccines not only contain target antigens (derived from viral/bacterial organisms against which protection is desired), they also contain numerous non‐target proteins used to manufacture the vaccine. These non‐target proteins include food proteins (bovine milk, peanut, soy, sesame, wheat, fish, gelatin, egg etc.(2–4)) used as growth media or as excipients. All these proteins are adsorbed on the surface of these aluminum adjuvant particles. The Th2 response thus applies to ALL of these antigens. One of the bovine milk proteins is the bovine folate receptor alpha (FRA) protein. A Th2 response against FRA results in the synthesis of IgE antibodies directed against FRA. Subsequent dietary milk consumption results in the synthesis of IgG4 antibodies directed against FRA. These IgG4 antibodies cross‐react with and bind to human FRA in the choroid plexus. They thus block folate uptake to the brain. The result is autism. Such cerebral folate deficiency impacts elimination of aluminum from the brain, resulting in the very high concentrations you refer above. The source of the aluminum can be the diet as well as vaccine adjuvants.(5) And you have a typo "implication" should read "implicating".

References

1. Arumugham V. Flawed assumptions fuel autoimmune disease: The sorry state of vaccine safety science [Internet]. 2017. Available from: https://www.zenodo.org/record/1034760 2. National Academies of Sciences and Medicine E. Finding a Path to Safety in Food Allergy: Assessment of the Global Burden, Causes, Prevention, Management, and Public Policy. Stallings VA, Oria MP, editors. Washington, DC: The National Academies Press; 2017. 3. Arumugham V. Role of NMDA receptor autoimmunity induced by food protein containing vaccines, in the etiology of autism, type 1 diabetes, neuropsychiatric and neurodegenerative disorders [Internet]. 2018. Available from: https://doi.org/10.5281/zenodo.1463600 4. Vaccine Excipient & Media Summary [Internet]. 2015 [cited 2016 Jan 16]. Available from: http://www.cdc.gov/vaccines/pubs/pinkbook/downloads/appendices/B/excipient‐table‐2.pdf 5. Arumugham V, Trushin M V. Autism pathogenesis: Piecing it all together, from end to beginning …. J Pharm Sci Res. 2018;10(11):2787–9.

Do you have any affiliation with or involvement in any organisation with a financial interest in the subject matter of your comment? NOTE: not provided by the contributor of the comment

Date of Submission: 31.12.2018

Contributor: Vinu Arumugham Email Address: vinucubeacc@gmail.com Affiliation: not provided Role: not provided

Reply

We thank Vinu Arumugham for his interest and comments related to the published protocol for our planned systematic review (Djurisic 2018)(1). We also thank Vinu Arumugham for identifying a typo in our protocol, which we have now corrected.

Our objective with the planned systematic review of randomised clinical trials is to examine the benefits and harms of adding one type of aluminium adjuvant versus another type of aluminium adjuvant to vaccines. This review will be conducted as soon as we have finished our work on the systematic review assessing aluminium adjuvants versus no adjuvants or placebo (Djurisic 2017) (2). We encourage others to perform basic research with vaccine components like: antigens, adjuvants, excipients, preservatives, stabilizers, buffers, surfactants, solvents, residuals, and diluents. We also encourage others to perform relevant and well‐conducted animal studies that may translate into human studies. Moreover, we encourage others to perform relevant and well‐conducted randomised clinical trials with different vaccine components based on strong indications from basic research and animal studies, which may be scrutinised with systematic reviews.

Response References:

1. Djurisic S, Jakobsen JC, Petersen SB, Kenfelt M, Klingenberg SL, Gluud C. Aluminium adjuvants used in vaccines. Cochrane Database of Systematic Reviews 2018, Issue 7. Art. No.: CD013086. DOI: 10.1002/14651858.CD013086. 2. Djurisic S, Jakobsen JC, Petersen SB, Kenfelt M, Gluud C. Aluminium adjuvants used in vaccines versus placebo or no intervention (Protocol). Cochrane Database of Systematic Reviews 2017, Issue 9. Art. No.: CD012805. DOI: 10.1002/14651858.CD012805.

Contributors: Snezana Djurisic; Sara Russo; Sesilje B. Petersen; Mette Kenfeldt; Sarah Loiuse Klingenber; Janus C. Jakobsen; Christian Gluud.

Contributors

Comments made by: Vinu Arumugham. Comments addressed by: Snezana Djurisic; Sara Russo; Sesilje B. Petersen; Mette Kenfeldt; Sarah Loiuse Klingenber; Janus C. Jakobsen; Christian Gluud.

Acknowledgements

Cochrane Review Group funding acknowledgement: the Danish State is the largest single funder of the Cochrane Hepato‐Biliary Group through its investment in the Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen University Hospital, Denmark.

Disclaimer: the views and opinions expressed in this review are those of the review authors and do not necessarily reflect those of the Danish State or the Copenhagen Trial Unit.

Peer reviewers: Gebran W Khneizer, USA; Benjamin M Kagina, South Africa. Contact Editor: Joshua Feinberg, Denmark. Sign‐off Editor: Agostino Colli, Italy.

Appendices

Appendix 1. Search strategies

Database Time span Search strategy
Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library Latest issue #1 MeSH descriptor: [Adjuvants, Immunologic] explode all trees #2 (immunologic* or alum* or vaccine*) and adjuvan* #3 #1 or #2 #4 MeSH descriptor: [Vaccines] explode all trees #5 vaccin* #6 #4 or #5 #7 MeSH descriptor: [Excipients] explode all trees #8 excipient* or bulk* agent* or filler* or diluent* #9 #7 or #8 #10 #3 and (#6 or #9)
MEDLINE Ovid 1946 to the date of search 1. exp Adjuvants, Immunologic/ 2. ((immunologic* or alum* or vaccine*) and adjuvan*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 3. 1 or 2 4. exp Vaccines/ 5. vaccin*.mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 6. 4 or 5 7. exp EXCIPIENTS/ 8. (excipient* or bulk* agent* or filler* or diluent*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 9. 7 or 8 10. 3 and (6 or 9) 11. (random* or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms] 12. 10 and 11
Embase (Ovid) 1974 to the date of search 1. exp immunological adjuvant/ 2. ((immunologic* or alum* or vaccine*) and adjuvan*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word] 3. 1 or 2 4. exp vaccine/ 5. vaccin*.mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word] 6. 4 or 5 7. exp excipient/ 8. (excipient* or bulk* agent* or filler* or diluent*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word] 9. 7 or 8 10. 3 and (6 or 9) 11. (random* or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word] 12. 10 and 11
BIOSIS (Web of Science) 1969 to the date of search #6 #5 AND #4 #5 TS=(random* or blind* or placebo* or meta‐analys*) #4 #1 and (#2 or #3) #3 TS=(excipient* or bulk* agent* or filler* or diluent*) #2 TS=vaccin* #1 TS=((immunologic* or alum* or vaccine*) and adjuvan*)
LILACS (Bireme) 1982 to the date of search (immunologic$ or alum$ or vaccine$) and adjuvan$ [Words] and vaccin$ or (excipient$ or bulk$ agent$ or filler$ or diluent$) [Words]
Science Citation Index Expanded (Web of Science) 1900 to the date of search #6 #5 AND #4 #5 TS=(random* or blind* or placebo* or meta‐analys*) #4 #1 and (#2 or #3) #3 TS=(excipient* or bulk* agent* or filler* or diluent*) #2 TS=vaccin* #1 TS=((immunologic* or alum* or vaccine*) and adjuvan*)
Conference Proceedings Citation Index – Science (Web of Science) 1990 to the date of search #6 #5 AND #4 #5 TS=(random* or blind* or placebo* or meta‐analys*) #4 #1 and (#2 or #3) #3 TS=(excipient* or bulk* agent* or filler* or diluent*) #2 TS=vaccin* #1 TS=((immunologic* or alum* or vaccine*) and adjuvan*)
Open in a new tab

What's new

Date Event Description
23 January 2019 Feedback has been incorporated Reply submitted 11 of January 2019
31 December 2018 Feedback has been incorporated Feedback submitted 31 of December 2018
Open in a new tab

Contributions of authors

SD drafted the protocol. JCJ drafted the protocol. SBP presented and drafted the idea for the systematic review. MK presented and drafted the idea for the systematic review. SLK developed the search strategies and drafted the protocol. CG drafted the protocol. All authors agreed on the final version of the protocol for publication.

Sources of support

Internal sources

  • Copenhagen Trial Unit, Denmark.

    Salaries for the people employed by The Copenhagen Trial Unit.

External sources

  • No sources of support supplied

Declarations of interest

SD: none known JCJ: none known SBP: none known MK: co‐founder of HPV_update.dk SLK: none known CG: none known

Notes

The content of this protocol is similar to another recently published protocol (Djurisic 2017). The main difference is this protocol is a head‐to‐head comparison of aluminium adjuvants versus other aluminium adjuvants, whereas Djurisic 2017 is a comparison of aluminium adjuvants versus placebo or no intervention. Due to the similarities in the scientific content and in the research question, the two protocols greatly overlap in their background and methods sections.

Edited (no change to conclusions), comment added to review

References

Additional references

  1. Arnheim‐Dahlström L, Pasternak B, Svanström H, Sparén P, Hviid A. Autoimmune, neurological, and venous thromboembolic adverse events after immunisation of adolescent girls with quadrivalent human papillomavirus vaccine in Denmark and Sweden: cohort study. BMJ 2013;347:f5906. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Awate S, Babiuk LA, Mutwiri G. Mechanisms of action of adjuvants. Frontiers in Immunology 2013;4:114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bartneck M, Keul HA, Wambach M, Bornemann J, Gbureck U, Chatain N, et al. Effects of nanoparticle surface‐coupled peptides, functional endgroups, and charge on intracellular distribution and functionality of human primary reticuloendothelial cells. Nanomedicine 2012;8(8):1282‐92. [DOI] [PubMed] [Google Scholar]
  4. Brinth LS, Pors K, Theibel AC, Mehlsen J. Orthostatic intolerance and postural tachycardia syndrome as suspected adverse effects of vaccination against human papilloma virus. Vaccine 2015;33(22):2602‐5. [DOI] [PubMed] [Google Scholar]
  5. Brinth L, Theibel AC, Pors K, Mehlsen J. Suspected side effects to the quadrivalent human papilloma vaccine. Danish Medical Journal 2015;62(4):A5064. [PubMed] [Google Scholar]
  6. Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61(8):763‐9. [DOI] [PubMed] [Google Scholar]
  7. Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38(1):287‐98. [DOI] [PubMed] [Google Scholar]
  8. Bruni L, Diaz M, Barrionuevo‐Rosas L, Herrero R, Bray F, Bosch FX, et al. Global estimates of human papillomavirus vaccination coverage by region and income level: a pooled analysis. Lancet Global Health 2016;4(7):e453‐63. [DOI] [PubMed] [Google Scholar]
  9. Carter D, Reed SG. Role of adjuvants in modelling the immune response. Current Opinion in HIV and AIDS 2010;5(5):409‐13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Caulfield MJ, Shi L, Wang S, Wang B, Tobery TW, Mach H, et al. Effect of alternative aluminum adjuvants on the absorption and immunogenicity of HPV16 L1 VLPs in mice. Human Vaccines 2007;3(4):139‐45. [DOI] [PubMed] [Google Scholar]
  11. Centers for Disease Control and Prevention. Measles (Rubeola). For healthcare professionals. www.cdc.gov/measles/hcp/ (accessed 1 August 2017).
  12. Chen RT, Davis RL, Rhodes PH. Special methodological issues in pharmacoepidemiology studies of vaccine safety. In: Strom BL editor(s). Pharmacoepidemiology. 4th Edition. Chichester, UK: John Wiley & Sons, Ltd, 2006:455‐85. [DOI: 10.1002/9780470059876.ch30] [DOI] [Google Scholar]
  13. Chen L, Yokel RA, Hennig B, Toborek M. Manufactured aluminum oxide nanoparticles decrease expression of tight junction proteins in brain vasculature. Journal of Neuroimmune Pharmacology 2008;3(4):286‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Coffman RL, Sher A, Seder RA. Vaccine adjuvants: putting innate immunity to work. Immunity 2010;33(4):492‐503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cowan LD, Griffin MR, Howson CP, Katz M, Johnston RBJ, Shaywitz BA, et al. Acute encephalopathy and chronic neurological damage after pertussis vaccine. Vaccine 1993;11(14):1371‐9. [DOI] [PubMed] [Google Scholar]
  16. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non‐randomised intervention studies. Health Technology Assessment 2003;7(27):1‐173. [DOI] [PubMed] [Google Scholar]
  17. Delany I, Rappuoli R, Gregorio E. Vaccines for the 21st century. EMBO Molecular Medicine 2014;6(6):708‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6(3):341‐50. [DOI] [PubMed] [Google Scholar]
  19. DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials 1986;7(3):177‐88. [DOI] [PubMed] [Google Scholar]
  20. Djurisic S, Jakobsen JC, Petersen SB, Kenfelt M, Gluud C. Aluminium adjuvants used in vaccines versus placebo or no intervention. Cochrane Database of Systematic Reviews 2017, Issue 9. [DOI: 10.1002/14651858.CD012805] [DOI] [Google Scholar]
  21. Donegan K, Beau‐Lejdstrom R, King B, Seabroke S, Thomson A, Bryan P. Bivalent human papillomavirus vaccine and the risk of fatigue syndromes in girls in the UK. Vaccine 2013;31(43):4961‐7. [DOI] [PubMed] [Google Scholar]
  22. Dourado I, Cunha S, Teixeira MG, Farrington CP, Melo A, Lucena R, et al. Outbreak of aseptic meningitis associated with mass vaccination with a urabe‐containing measles‐mumps‐rubella vaccine: implications for immunization programs. American Journal of Epidemiology 2000;151(5):524‐30. [DOI] [PubMed] [Google Scholar]
  23. Dyer O. Canadian academic's call for moratorium on HPV vaccine sparks controversy. BMJ 2015;351:h5692. [DOI] [PubMed] [Google Scholar]
  24. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ 1997;315(7109):629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. European Medicines Agency. Assessment report. Review under Article 20 of regulation (EC) No 726/2004 human papillomavirus (HPV) vaccines. www.ema.europa.eu/docs/en_GB/document_library/Referrals_document/HPV_vaccines_20/Opinion_provided_by_Committee_for_Medicinal_Products_for_Human_Use/WC500197129.pdf (accessed 1 August 2017).
  26. Exley C. Aluminium‐based adjuvants should not be used as placebos in clinical trials. Vaccine 2011;29(50):9289. [DOI] [PubMed] [Google Scholar]
  27. Fedson DS. Measuring protection: efficacy versus effectiveness. Developments in Biological Standardization 1998;95:195‐201. [PubMed] [Google Scholar]
  28. Foged C, Brodin B, Frokjaer S, Sundblad A. Particle size and surface charge affect particle uptake by human dendritic cells in an in vitro model. International Journal of Pharmaceutics 2005;298(2):315‐22. [DOI] [PubMed] [Google Scholar]
  29. Future II Study Group. Quadrivalent vaccine against human papillomavirus to prevent high‐grade cervical lesions. New England Journal of Medicine 2007;356(19):1915‐27. [DOI] [PubMed] [Google Scholar]
  30. Glenny AT, Pope CG, Waddington H, Wallace U. Immunological notes. XVII–XXIV. Journal of Pathology and Bacteriology 1926;29:31‐40. [Google Scholar]
  31. Global Polio Eradication Initiative, World Health Organization. Global Polio Eradication Initiative. www.polioeradication.org/ (accessed 13 January 2017).
  32. Gluud LL. Bias in clinical intervention research. American Journal of Epidemiology 2006;163(6):493‐501. [DOI] [PubMed] [Google Scholar]
  33. Goto N, Kato H, Maeyama J, Shibano M, Saito T, Yamaguchi J, et al. Local tissue irritating effects and adjuvant activities of calcium phosphate and aluminium hydroxide with different physical properties. Vaccine 1997;15(12‐3):1364‐71. [DOI] [PubMed] [Google Scholar]
  34. McMaster University (developed by Evidence Prime). GRADEpro GDT. Version accessed dd Month yyyy. Hamilton (ON): McMaster University (developed by Evidence Prime), 2015.
  35. Grimaldi‐Bensouda L, Guillemot D, Godeau B, Bénichou J, Lebrun‐Frenay C, Papeix C, et al. Autoimmune disorders and quadrivalent human papillomavirus vaccination of young female subjects. Journal of Internal Medicine 2014;275(4):398‐408. [DOI] [PubMed] [Google Scholar]
  36. Grun JL, Maurer PH. Different T helper cell subsets elicited in mice utilizing two different adjuvant vehicles: the role of endogenous interleukin 1 in proliferative responses. Cellular Immunology 1989;121(1):134‐45. [DOI] [PubMed] [Google Scholar]
  37. Gupta RK, Rost BE, Relyveld E, Siber GR. Adjuvant properties of aluminum and calcium compounds. Pharmaceutical Biotechnology 1995;6:229‐48. [DOI] [PubMed] [Google Scholar]
  38. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck‐Ytter Y, Alonso‐Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336(7650):924‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Gøtzsche P. Complaint to the European Medicines Agency (EMA) over maladministration at the EMA [personal communication]. Email to S Djurisic 26 May 2016.
  40. Gøtzsche P. Complaint to the European ombudsman over maladministration at the European Medicines Agency (EMA) in relation to the safety of the HPV vaccines [personal communication]. Email to S Djurisic 10 October 2016.
  41. Harbord RM, Egger M, Sterne JA. A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints. Statistics in Medicine 2006;25(20):3443‐57. [DOI] [PubMed] [Google Scholar]
  42. Harper DM, DeMars LR. HPV vaccines – a review of the first decade. Gynecologic Oncology 2017;146(1):196‐204. [DOI] [PubMed] [Google Scholar]
  43. Hem SL, Hogenesch H. Relationship between physical and chemical properties of aluminum‐containing adjuvants and immunopotentiation. Expert Review of Vaccines 2007;6(5):685‐98. [DOI] [PubMed] [Google Scholar]
  44. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21(11):1539‐58. [DOI] [PubMed] [Google Scholar]
  45. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ 2003;327(7414):557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Higgins JP, Whitehead A, Simmonds M. Sequential methods for random‐effects meta‐analysis. Statistics in Medicine 2011;30(9):903‐21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.
  48. Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non‐blinded outcome assessors. BMJ 2012;344:e1119. [DOI] [PubMed] [Google Scholar]
  49. Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. Canadian Medical Association Journal 2013;185(4):E201‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Hróbjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub‐studies. International Journal of Epidemiology 2014;43(4):1272‐83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. International Conference on Harmonization Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, Philadelphia (PA): Barnett International/PAREXEL, 1997. [Google Scholar]
  52. Issa AM, Salim MS, Zidan H, Mohamed AF, Farrag A‐RH. Evaluation of the effects of aluminum phosphate and calcium phosphate nanoparticles as adjuvants in vaccinated mice. International Journal of Chemical Engineering and Applications 2014;5(5):367‐73. [Google Scholar]
  53. Jackson D, Bujkiewicz S, Law M, Riley RD, White IR. A matrix‐based method of moments for fitting multivariate network meta‐analysis models with multiple outcomes and random inconsistency effects. Biometrics 2017;74(2):548‐56. [DOI: 10.1111/biom.12762] [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta‐analytic methods. BMC Medical Research Methodology 2014;14:120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Jefferson T, Rudin M, Pietrantonj C. Adverse events after immunisation with aluminium‐containing DTP vaccines: systematic review of the evidence. Lancet Infectious Diseases 2004;4(2):84‐90. [DOI] [PubMed] [Google Scholar]
  56. Keus F, Wetterslev J, Gluud C, Laarhoven CJ. Evidence at a glance: error matrix approach for overviewing available evidence. BMC Medical Research Methodology 2010;10:90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Kimura M, Kuno‐Sakai H, Yamazaki S, Yamada A, Hishiyama M, Kamiya H, et al. Adverse events associated with MMR vaccines in Japan. Acta Paediatrica Japonica; Overseas Edition 1996;38(3):205‐11. [DOI] [PubMed] [Google Scholar]
  58. Kisnieriené V, Lapeikaité I. When chemistry meets biology: the case of aluminium – a review. Chemija 2015;26:148‐58. [Google Scholar]
  59. Kjaer SK, Nygård M, Dillner J, Marshall JB, Radley D, Li M, et al. A 12‐year follow‐up on the long‐term effectiveness of the quadrivalent human papillomavirus vaccine in 4 Nordic countries. Clinical Infectious Diseases 2017;66(3):339‐45. [DOI: 10.1093/cid/cix797] [DOI] [PubMed] [Google Scholar]
  60. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta‐analyses. Annals of Internal Medicine 2001;135(11):982‐9. [DOI] [PubMed] [Google Scholar]
  61. Klein NP, Hansen J, Chao C, Velicer C, Emery M, Slezak J, et al. Safety of quadrivalent human papillomavirus vaccine administered routinely to females. Archives of Pediatrics & Adolescent Medicine 2012;166(12):1140‐8. [DOI] [PubMed] [Google Scholar]
  62. Kocourkova A, Honegr J, Kuca K, Danova J. Vaccine ingredients: components that influence vaccine efficacy. Mini Reviews in Medicinal Chemistry 2017;17(5):451‐66. [DOI] [PubMed] [Google Scholar]
  63. Kool M, Soullié T, Nimwegen M, Willart MA, Muskens F, Jung S, et al. Alum adjuvant boosts adaptive immunity by inducing uric acid and activating inflammatory dendritic cells. Journal of Experimental Medicine 2008;205(4):869‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Kool M, Fierens K, Lambrecht BN. Alum adjuvant: some of the tricks of the oldest adjuvant. Journal of Medical Microbiology 2012;61(Pt 7):927‐34. [DOI] [PubMed] [Google Scholar]
  65. Lee SH. Detection of human papillomavirus (HPV) L1 gene DNA possibly bound to particulate aluminum adjuvant in the HPV vaccine Gardasil. Journal of Inorganic Biochemistry 2012;117:85‐92. [DOI] [PubMed] [Google Scholar]
  66. Li X, Aldayel AM, Cui Z. Aluminum hydroxide nanoparticles show a stronger vaccine adjuvant activity than traditional aluminum hydroxide microparticles. Journal of Controlled Release 2014;173:148‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2017, Issue 2. [DOI: 10.1002/14651858.MR000033.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Malluche HH. Aluminium and bone disease in chronic renal failure. Nephrology, Dialysis, Transplantation 2002;17(Suppl 2):21‐4. [DOI] [PubMed] [Google Scholar]
  69. Mannhalter JW, Neychev HO, Zlabinger GJ, Ahmad R, Eibl MM. Modulation of the human immune response by the non‐toxic and non‐pyrogenic adjuvant aluminium hydroxide: effect on antigen uptake and antigen presentation. Clinical and Experimental Immunology 1985;61(1):143‐51. [PMC free article] [PubMed] [Google Scholar]
  70. Marrack P, McKee AS, Munks MW. Towards an understanding of the adjuvant action of aluminium. Nature Reviews. Immunology 2009;9(4):287‐93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Mbow ML, Gregorio E, Valiante NM, Rappuoli R. New adjuvants for human vaccines. Current Opinion in Immunology 2010;22(3):411‐6. [DOI] [PubMed] [Google Scholar]
  72. Miller DL. Frequency of complications of measles, 1963. Report on a national inquiry by the public health laboratory service in collaboration with the Society of Medical Officers of Health. British Medical Journal 1964;2(5401):75‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352(9128):609‐13. [DOI] [PubMed] [Google Scholar]
  74. Mold M, Umar D, King A, Exley C. Aluminium in brain tissue in autism. Journal of Trace Elements in Medicine and Biology 2017;46:76‐82. [DOI] [PubMed] [Google Scholar]
  75. Morefield GL, Sokolovska A, Jiang D, HogenEsch H, Robinson JP, Hem SL. Role of aluminum‐containing adjuvants in antigen internalization by dendritic cells in vitro. Vaccine 2005;23(13):1588‐95. [DOI] [PubMed] [Google Scholar]
  76. O'Hagan DT, Gregorio E. The path to a successful vaccine adjuvant ‐ 'the long and winding road'. Drug Discovery Today 2009;14(11‐2):541‐51. [DOI] [PubMed] [Google Scholar]
  77. O'Hagan DT, Wack A. Adjuvants: from serendipity to rational discovery. Vaccinology. Wiley‐Blackwell, 2012:348‐60. [Google Scholar]
  78. Offit PA. Vaccinated: One Man's Quest to Defeat the World's Deadliest Diseases. Washington DC: Smithsonian. ISBN 0‐06‐122796‐X, 2007. [Google Scholar]
  79. Oleszycka E, Lavelle EC. Immunomodulatory properties of the vaccine adjuvant alum. Current Opinion in Immunology 2014;28:1‐5. [DOI] [PubMed] [Google Scholar]
  80. Oyewumi MO, Kumar A, Cui Z. Nano‐microparticles as immune adjuvants: correlating particle sizes and the resultant immune responses. Expert Review of Vaccines 2010;9(9):1095‐107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Paavonen J, Naud P, Salmerón J, Wheeler CM, Chow SN, Apter D, et al. HPV PATRICIA Study Group. Efficacy of human papillomavirus (HPV)‐16/18 AS04‐adjuvanted vaccine against cervical infection and precancer caused by oncogenic HPV types (PATRICIA): final analysis of a double‐blind, randomised study in young women. Lancet 2009;374(9686):301‐14. [DOI] [PubMed] [Google Scholar]
  82. Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta‐analysis. Controlled Clinical Trials 1997;18(6):580‐93; discussion 661‐6. [DOI] [PubMed] [Google Scholar]
  83. Ramanathan VD, Badenoch‐Jones P, Turk JL. Complement activation by aluminium and zirconium compounds. Immunology 1979;37(4):881‐8. [PMC free article] [PubMed] [Google Scholar]
  84. Rambe DS, Giudice G, Rossi S, Sanicas M. Safety and mechanism of action of licensed vaccine adjuvants. International Current Pharmaceutical Journal 2015;4:420‐31. [Google Scholar]
  85. Reed SG, Orr MT, Fox CB. Key roles of adjuvants in modern vaccines. Nature Medicine 2013;19(12):1597‐608. [DOI] [PubMed] [Google Scholar]
  86. Reinke CM, Breitkreutz J, Leuenberger H. Aluminium in over‐the‐counter drugs: risks outweigh benefits?. Drug Safety 2003;26(14):1011‐25. [DOI] [PubMed] [Google Scholar]
  87. Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager 5 (RevMan 5). Version 5.3. Copenhagen: Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
  88. Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]
  89. Salvador A, Igartua M, Hernández RM, Pedraz JL. An overview on the field of micro‐ and nanotechnologies for synthetic peptide‐based vaccines. Journal of Drug Delivery 2011;2011:181646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]
  91. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta‐epidemiological studies. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]
  92. Scheller NM, Svanström H, Pasternak B, Arnheim‐Dahlström L, Sundström K, Fink K, et al. Quadrivalent HPV vaccination and risk of multiple sclerosis and other demyelinating diseases of the central nervous system. JAMA 2015;313(1):54‐61. [DOI] [PubMed] [Google Scholar]
  93. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408‐12. [DOI] [PubMed] [Google Scholar]
  94. Sharma HS, Hussain S, Schlager J, Ali SF, Sharma A. Influence of nanoparticles on blood‐brain barrier permeability and brain edema formation in rats. Acta Neurochirurgica 2010;106:359‐64. [DOI] [PubMed] [Google Scholar]
  95. Shima F, Uto T, Akagi T, Baba M, Akashi M. Size effect of amphiphilic poly(gamma‐glutamic acid) nanoparticles on cellular uptake and maturation of dendritic cells in vivo. Acta Biomaterialia 2013;9(11):8894‐901. [DOI] [PubMed] [Google Scholar]
  96. Son YJ, Kim H, Leong KW, Yoo HS. Multifunctional nanorods serving as nanobridges to modulate T cell‐mediated immunity. ACS Nano 2013;7(11):9771‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  97. StataCorp. Stata. Version 14. College Station, Texas, USA: StataCorp, 2014.
  98. Storebø OJ, Pedersen N, Ramstad E, Kielsholm ML, Nielsen SS, Krogh HB, et al. Methylphenidate for attention deficit hyperactivity disorder (ADHD) in children and adolescents – assessment of adverse events in non‐randomised studies. Cochrane Database of Systematic Reviews 2018, Issue 5. [DOI: 10.1002/14651858.CD012069.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
  99. Strugnell R, Zepp F, Cunningham A, Tantawichien T. Vaccine antigens. Perspectives in Vaccinology 2011;1:61‐88. [Google Scholar]
  100. Thiele L, Rothen‐Rutishauser B, Jilek S, Wunderli‐Allenspach H, Merkle HP, Walter E. Evaluation of particle uptake in human blood monocyte‐derived cells in vitro. Does phagocytosis activity of dendritic cells measure up with macrophages?. Journal of Controlled Release 2001;76(1‐2):59‐71. [DOI] [PubMed] [Google Scholar]
  101. Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses?. International Journal of Epidemiology 2009;38(1):276‐86. [DOI] [PubMed] [Google Scholar]
  102. Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). www.ctu.dk/tsa/files/tsa_manual.pdf (accessed 13 January 2017).
  103. Tomljenovic L, Shaw CA. Aluminum vaccine adjuvants: are they safe?. Current Medicinal Chemistry 2011;18(17):2630‐7. [DOI] [PubMed] [Google Scholar]
  104. Tomljenovic L, Shaw CA. Too fast or not too fast: the FDA's approval of Merck's HPV vaccine Gardasil. Journal of Law, Medicine & Ethics 2012;40(3):673‐81. [DOI] [PubMed] [Google Scholar]
  105. Tritto E, Mosca F, Gregorio E. Mechanism of action of licensed vaccine adjuvants. Vaccine 2009;27(25‐6):3331‐4. [DOI] [PubMed] [Google Scholar]
  106. Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. Version 0.9 Beta. Copenhagen: Copenhagen Trial Unit, 2011.
  107. Turner RM, Bird SM, Higgins JP. The impact of study size on meta‐analyses: examination of underpowered studies in Cochrane reviews. PLOS One 2013;8(3):e59202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  108. Ward BJ. Vaccine adverse events in the new millennium: is there reason for concern?. Bulletin of the World Health Organization 2000;78(2):205‐15. [PMC free article] [PubMed] [Google Scholar]
  109. Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61(1):64‐75. [DOI] [PubMed] [Google Scholar]
  110. Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random‐effects model meta‐analyses. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  111. Wetterslev J, Jakobsen JC, Gluud C. Trial Sequential Analysis in systematic reviews with meta‐analysis. BMC Medical Research Methodology 2017;17(1):39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  112. Whitney CG, Zhou F, Singleton J, Schuchat A, Centers for Disease Control and Prevention (CDC). Benefits from immunization during the vaccines for children program era ‐ United States, 1994‐2013. Morbidity and Mortality Weekly Report 2014;63(16):352‐5. [PMC free article] [PubMed] [Google Scholar]
  113. World Health Organization. Human papillomavirus vaccines: WHO position paper. Weekly Epidemiological Record 2014;43(89):462‐92. [Google Scholar]
  114. World Health Organization. Polio vaccines: WHO position paper. Weekly Epidemiological Record 2016;12(91):145–68. [Google Scholar]
  115. World Health Organization. Measles. Fact Sheet. www.who.int/wer/2014/wer8943/en/ (accessed 1 August 2017).
  116. World Health Organization. WHO recommendations for routine immunization ‐ summary tables. www.who.int/immunization/policy/immunization_tables/en/ (accessed 1 August 2017).
  117. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ 2008;336(7644):601‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  118. Yokel RA. The toxicology of aluminum in the brain: a review. Neurotoxicology 2000;21(5):813‐28. [PubMed] [Google Scholar]

Articles from The Cochrane Database of Systematic Reviews are provided here courtesy of Wiley

RESOURCES