Skip to main content
. 2018 Feb 1;2018(2):CD001269. doi: 10.1002/14651858.CD001269.pub6
Study Reason for exclusion
ab Wacheck 2010 Experimental vaccine; dose escalation study
ab López‐Macías 2011a Experimental vaccine; no outcomes of interest
ab López‐Macías 2011b Experimental vaccine; no outcomes of interest
ab Mallory 2010 No outcomes of interest
ab Plennevaux 2010 No outcomes of interest
ab Precioso 2011 No outcomes of interest
ab Treanor 2010 Experimental vaccine
ab Turley 2011 Experimental vaccine; no outcomes of interest
Al‐Dabbagh 2013 No outcomes of interest, differences in cytokine levels between ORS cases and controls after vaccination
Ambrosch 1976 Data tables and figure missing
Ambrose 2012 No original data
Andersson 2015 Comment on cb Persson 2014 study
Aoki 1986 Randomised controlled trial, single‐blind. Outcomes were clinical cases and adverse effects. Follow‐up data were not reported by arm.
Arnou 2010 Intradermal administration (3 different lots of the same vaccine) versus intramuscular administration. Serologic response and AE at day 21. No adequate placebo/no intervention control
Atmar 1995 No outcomes of interest
Atmar 2011 Absence of an adequate control
Atsmon 2012 Experimental vaccine; no outcomes of interest
Ausseil 1999 No design (average days of sick leave in vaccinated and non‐vaccinated participants during 1996 and 1997 from staff of an international banking institution)
Banzhoff 2001 No design (cohort), no safety outcomes
Baxter 2010 No design: cohort study for effectiveness
Baxter 2011 A 'head‐to‐head' trial: "FluBlok (purified HA proteins manufactured in expresSF+® insect cells under serum free conditions using a baculovirus expression system (BEVS). Uncleaved HA produced by this method is referred to as rHA0. Vaccine formulation consisted of 135g total HA protein (45g each) as determined by single radial immunodiffusion assay (SRID) and included rHA0 derived from the following influenza strains A/Solomon Islands/03/2006 (H1N1), A/Wisconsin/67/2005 (H3N2), and B/Malaysia/2506/2004 VS. The same CDC‐derived vaccine seed viruses were used for the licensed trivalent inactivated vaccine (TIV; Fluzone [2007–2008 formulation; Sanofi Pasteur, Swiftwater, PA), which contained 15g of each HA [45g total])"
Baxter 2012 No design: controlled case series
Baxter 2013 Self controlled time series study
Belongia 2009 Case‐control study, no harm assessment
Belshe 2001 No original data
Benke 2004 Questionnaire survey; non‐comparative analysis
Beran 2013 Absence of an adequate control group (quadrivalent versus trivalent inactivated vaccine; low versus normal adjuvant content)
Betts 1977b Trial with swine vaccine (Hsw1N1, A/New Jersey/76)
Beyer 1996 Review
Carlson 1979 No adequate control, no outcome of interest
Cate 1977 Trial with swine vaccine (Hsw1N1, A/New Jersey/76)
Chavant 2013 Absence of a control group; study population consists of vaccinated pregnant women only
Chichester 2012 Experimental vaccine; no outcomes of interest
Chlibek 2002 Not a randomised controlled trial
Choe 2011a No design: cross‐sectional study
Choe 2011b No design: case series
Choe 2011c No design: case series
Chou 2007 Case report
Clover 1991 Randomised controlled trial. More than 75% of the study population was out of the age range stated in the protocol.
Confavreux 2001 Participants are MS cases.
Conlin 2013 Inadequate comparison and study design: cohort study with pandemic versus seasonal (not exposed) vaccines in women and newborns
Couch 2012 Experimental vaccine; no outcomes of interest
Das Gupta 2002 Does not contain effectiveness data
Davidson 2011 Inadequate comparison: all enrolled participants received LAIV, then were randomised to either placebo or Lactobacillus rhamnosus GG
Davies 1972 Cohort with efficacy outcomes. Experimental and control group were selected separately.
Davies 1973 Not randomised. Participants volunteered for immunisation, and comparison was made with a randomly selected non‐immunised control group.
De Serres 2003a No comparison, absence of adequate control group
De Serres 2003b No control
De Serres 2004 Population at risk of further ORS episodes.
De Wals 2012 No design: self controlled case series for association between H1N1 and GBS
Dolin 1977 Trial with swine vaccine (Hsw1N1, A/New Jersey/76)
Dominguez 2012 No design: case‐control study assessing effectiveness in general population
Duffy 2014 Case‐centred study
Eames 2012 No design: effectiveness cohort study in general population
Edmonson 1970 Influenza B vaccine was used as control.
Eick‐Cost 2012 No design: case‐control study assessing effectiveness in general population
El'shina 1998 Major inconsistencies in the study text
Englund 1993 Inadequate comparison (tetanus toxoid vaccine)
Finklea 1969 Randomised controlled trial, double‐blind. 2 bivalent inactivated influenza vaccines with the same viral composition, differing in purification procedures, were compared.
 Outcomes were clinical cases and adverse effects.
 Raw data about clinical cases were not reported by arm.
 Circulating virus showed significant antigenic differences from the A2 vaccine strain.
Fisher 2012 No outcomes of interest (antibody titres only)
Foy 1981 Absence of adequate control
Frank 1981 No usable safety data (scores)
Freestone 1976 Conference proceedings
Gerstoft 2001 Not a randomised controlled trial
Greenbaum 2002 No outcome of interest
Greene 2013 Case‐centred study
Gross 1999 Outcome measures outside inclusion criteria.
Grotto 1998 Not a randomised controlled trial
Gruber 1994 Randomised controlled trial conducted in the USA on 41 cystic fibrosis (CF) patients and 89 family members, recruited through a clinic. Participants were randomly assigned in a double‐blinded fashion by family to receive either intranasal, live, cold‐adapted influenza A vaccine or the recommended intramuscular trivalent inactivated influenza vaccine.
The study lasted 3 years (from 1989 to 1991). Participants were immunised each fall, staying in the same assigned vaccine group. The live vaccine arm counted 20 CF and 33 family members; the trivalent vaccine arm 21 and 56, respectively.
69 participants (17 CF patients and 52 family members) dropped out. The reasons were stated in the article.
The live vaccine was the same throughout the period: A/Kawasaki/9/86 (H1N1) 107.3 pfu, A/Los Angeles/2/87 107.3 pfu.
The viral strains used in the inactivated vaccines were:
  • 1989 to 1990: A/Taiwan/1/86 (H1N1), A/Shanghai/11/87 (H3N2), B/Yagamata/16/88, 15 mg/dose of each

  • 1990 to 1991: A/Taiwan/1/86 (H1N1), A/Shanghai/16/89 (H3N2), B/Yagamata/16/88, 15 mg/dose of each

  • 1991 to 1992: A/Taiwan/1/86 (H1N1), A/Beijing/353/89 (H3N2), B/Panama/45/90, 15 mg/dose of each


Live vaccine recipients also received monovalent inactivated influenza B vaccine (identical to that contained in the trivalent vaccine) as an intramuscular placebo. Allantoic fluid was the placebo for aerosol administration.
Data were extracted and loaded for family members only.
Outcomes were clinical and laboratory‐confirmed cases, working days lost, admissions, deaths, and adverse effects.
Clinical cases were classified as "respiratory illness" or "febrile respiratory illness". Laboratory‐confirmed cases were defined by an influenza virus isolation from a throat swab.
Adverse effects were defined as temperature > 38 °C, rhinorrhoea, sore throat, cough, increasing sputum, redness, swelling, chills. Results are expressed as % of participant‐days with symptoms.
Participants were followed throughout the period. Owing to the dropouts, the vaccinated were counted as participant‐years: 54 in the live vaccine arm; 56 in the trivalent vaccine arm.
The influenza illness surveillance period for study participants was defined as the interval from the date of the first influenza isolate from the population under routine surveillance to 2 weeks after the last isolate for each year.
Viral strains circulating during the outbreaks were:
  • 1989 to 1990: A/Shanghai/11/87 (H3N2)

  • 1990 to 1991: A/Beijing/353/89 (H3N2), B/Panama/45/90‐like

  • 1991 to 1992: A/Beijing/353/89 (H3N2)


We excluded this trial because it was not placebo controlled, and the authors did not specify if the strains used to develop cold‐adapted and inactivated vaccines were antigenically comparable or not.
Gwini 2011 No design: self controlled case series
Haber 2004 Analysis of temporal trends of GBS 1990 to 2003, comparison with temporal trends of non‐GBS adverse event reports from the Vaccine Adverse Event Reporting System (VAERS)
Haigh 1973 Not randomised: all the volunteers were immunised on a single day, and the intention to allocate participants randomly was not strictly adhered to
Halperin 2002 Outcome measures outside inclusion criteria.
Hambidge 2011 Participants affected by sickle cell crisis.
Heinonen 1973 Control consists of another vaccine.
Hellenbrand 2012 No design: case‐control study assessing effectiveness in general population
Hobson 1970 Polyvalent influenza vaccine was used as control.
Hobson 1973 Randomised controlled trial. Clinical outcomes were side effects only.
Hoskins 1973 Influenza B vaccine was used as control.
Hoskins 1976 Not placebo or 'do nothing' controlled
Hoskins 1979 No control group
Howell 1967 Not prospective: appears to be an historical cohort
Huang 2011 Comparison is not adequate (vaccine versus vaccine).
Hurwitz 1983 Report of GBS surveillance 1978 to 1979, non‐comparative study
Jackson 2011 No adequate control (the same vaccine prepared with different antigenic concentrations was administered to each group)
Janjua 2012 No design: case‐control study assessing effectiveness in general population
Jianping 1999 Not a randomised controlled trial
Jimenez‐Jorge 2012 No design: case‐control study assessing effectiveness in general population
Keitel 2001 Efficacy outcome measures outside inclusion criteria. The safety data are presented in a non‐analysable way.
Kelly 2012 No design: case‐control study assessing effectiveness in general population
Khazeni 2009 Review and cost‐effectiveness analysis
Kiderman 2001 Tables and text show inconsistencies that do not allow data extraction.
Kim 2012 Surveillance for adverse events
Kissling 2012 No design: case‐control study assessing effectiveness in general population
Kunz 1977 No adequate control
Langley 2004 Review
Lavallee 2014 Review about stroke and vaccination in elderly people
Lee 2011 No design: self controlled case series
Leeb 2011 No design: case series
Leroux‐Roels 2010a Absence of an adequate control, serological outcomes only
Leroux‐Roels 2010b Absence of an adequate control, serological outcomes only
Liem 1973 Reported the results of 9 placebo‐controlled clinical trials and 2 field studies, involving a total of about 10,000 participants, carried out in several countries to assess the efficacy of killed influenza spray vaccines. Studies were conducted during the years 1969 to 1971.
 Allocation of the participants to the arms of the trials was done according to a predetermined randomisation scheme. 8 of the studies were double‐blind. The field studies were not randomised. The attack rate for influenza among the population study was very low, and in 2 of the trials the vaccination procedure started too late, when the outbreak was ongoing. The attack rates, based exclusively on the serologically confirmed cases, are only reported by a graph and deriving the crude data is impossible.
Lind 2014 Surrogate exposure assessment (antibody level)
Liu 2012 Study to identify variables associated with uptake of influenza vaccination during pregnancy
Louik 2013 Methods for assessing flu vaccine exposure during pregnancy
Mackenzie 1975 No design: allocation is arbitrary, and groups with different characteristics were formed
Mackenzie 2012 Non‐comparative design
Mair 1974 Influenza B vaccine was used as control.
Maynard 1968 Influenza B vaccine was used as control.
McCarthy 2004 Review
Mendelman 2001 Does not report original results
Merelli 2000 Review
Meyers 2003a Review
Meyers 2003b Review
Micheletti 2011 Total number of AEs observed after administration of each vaccine type
Monto 2000 Not a randomised controlled trial
Montplaisir 2014 Study population outside age range.
Moro 2011 Non‐comparative study
Morris 1975 Design is unclear: no standard random allocation. Only 25 out of 30 participants seem to have been immunised, but in the method description 30 were considered for exposure to natural influenza A/Scotland/840/74. 1 of these was excluded prior due to tonsillitis.
Mostow 1977 Outcomes were safety only. Absence of adequate control
Muennig 2001 Not a randomised controlled trial
Murray 1979 Not adequate comparison (pregnant versus non‐pregnant women)
Nazareth 2013 Absence of control group, non‐comparative
Nichol 1996 Same data as Nichol 1995 (included)
Nichol 1999b Review
Nichol 2001 Not a randomised controlled trial
Nichol 2003 Contains data from previous studies
Nichol 2004 Re‐analysis of Nichol 1999 (included)
Omon 2011 Non‐comparative study
Petrie 2011 No new data: reports data from already published and included studies (aa Ohmit 2006, aa Ohmit 2008, aa Monto 2009)
Phillips 2013 Absence of adequate control group
Phonrat 2013 No outcomes of interest
Pleguezuelos 2012 Experimental vaccine; no outcomes of interest
Puig‐Barbera 2012 No design: case‐control study assessing effectiveness in general population (also children and elderly)
Puleston 2010 Not outcomes of interest
Pyhala 2001 Not a randomised controlled trial
Reynales 2012 Safety survey after Celtura (H1N1) administration. Absence of control group
Rimmelzwaan 2000 Outcome measures outside inclusion criteria.
Rocchi 1979c Very poor reporting, unclear definition, no description of methods
Rowhani‐Rahbar 2012 Participants are children
Ruben 1972 Absence of adequate control
Ruben 1973 Both arms contained the same vaccine strains.
Safranek 1991 Reassessment of Schonberger 1979 (included)
Sarateanu 1980 Absence of adequate control
Scheifele 2013 No outcomes of interest
Schonberger 1981 Review of the evidence of the aetiology of GBS, no original data presented
Schwartz 1996 Report about Nichol 1995 (included)
Simpson 2012 No design: cohort and case‐control study assessing effectiveness in general population
Sipilä 2015 Ecological study
Skowronski 2002 Non‐comparative (survey)
Skowronski 2003 Population at risk of further ORS episodes
Smith 1977a Reports a small part of the Hoskins trial. It compared illness occurring among a group of vaccinated boys against non‐vaccinated controls that had no part in the trial.
Smith 1977b Trial with swine vaccine (Hsw1N1, A/New Jersey/76)
Song 2011 1 trial is a 'head‐to‐head' trial (Gc501 versus Fluarix) with serological outcomes only; the other trial (safety) has no control.
Souayah 2011 Compares the incidence of GBS cases after tetravalent human papillomavirus vaccine with that observed after pneumococcal and flu vaccine administration
Spencer 1975 Authors did not report crude data on the clinical outcomes.
Spencer 1979 Reporting does not make clear the methods used to allocate participants and to conceal allocation. Clinical outcome data are not reported.
Steinhoff 2012 Inadequate control (23v pneumococcal vaccine administered to the control group). Re‐analysis of Zaman 2008 data (excluded)
Sumaya 1979 No outcomes of interest
Talaat 2010 Data on AEs are not provided in a useful form (bar graphs or cumulatively in the text).
Tavares 2011 Non‐comparative
Taylor 1969 No outcomes of interest, rhinovirus vaccine as control
Taylor 2012 Experimental vaccine; no outcomes of interest
Thompson 2014 Test‐positive case‐control study
Tokars 2012 No design: controlled case series
Treanor 2001 Outcome measures outside inclusion criteria.
Treanor 2002 Outcome measures outside inclusion criteria.
Treanor 2012 No design: case‐control study
Tsai 2010 Non‐comparative
Tsatsaris 2011 Same vaccine administered in different pregnancy weeks (inadequate comparison).
Tyrrell 1970 We were unable to include the 3 studies reported in this paper for the following reasons.
  1. No design, no comparison, no outcomes.

  2. Probable controlled clinical trial, but participants' ages likely out of range (schools).

  3. No design, even if an unvaccinated control group for school 3 and for the employees of the Imperial Chemical Industries is present.

Vesikari 2012 Safety data after dose I (seasonal versus placebo) are not extracted (bar graph).
Warren‐Gash 2013 Outside target age; all participants were older than 60 years
Warshauer 1976 Not randomised. Data reporting was not complete.
Wilde 1999 Pneumococcal vaccine was used as control.
Williams 1973 No placebo or 'do nothing' control
Williams 2011 No design: case series
Wise 2012 No design
Wood 1999 Not a randomised controlled trial
Wood 2000 Not a randomised controlled trial
Xu 2012 No original data presented
Yang 2012 No safety data
Yeager 1999 Non‐comparative study: absence of a control arm
Yih 2012 No design: controlled case series
Zaman 2008 Inadequate control (23v pneumococcal vaccine administered to the control group)

AE = adverse event
 GBS = Guillain‐Barré syndrome
 LAIV = live attenuated influenza vaccine
 MS = multiple sclerosis
 ORS = oculo‐respiratory syndrome
 pfu = plaque‐forming units