Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review

Abstract

Objective To explore the relation between study concordance, take home message, funding, and dissemination of comparative studies assessing the effects of influenza vaccines.

Design Systematic review without meta-analysis.

Data extraction Search of the Cochrane Library, PubMed, Embase, and the web, without language restriction, for any studies comparing the effects of influenza vaccines against placebo or no intervention. Abstraction and assessment of quality of methods were carried out.

Data synthesis We identified 259 primary studies (274 datasets). Higher quality studies were significantly more likely to show concordance between data presented and conclusions (odds ratio 16.35, 95% confidence interval 4.24 to 63.04) and less likely to favour effectiveness of vaccines (0.04, 0.02 to 0.09). Government funded studies were less likely to have conclusions favouring the vaccines (0.45, 0.26 to 0.90). A higher mean journal impact factor was associated with complete or partial industry funding compared with government or private funding and no funding (differences between means 5.04). Study size was not associated with concordance, content of take home message, funding, and study quality. Higher citation index factor was associated with partial or complete industry funding. This was sensitive to the exclusion from the analysis of studies with undeclared funding.

Conclusion Publication in prestigious journals is associated with partial or total industry funding, and this association is not explained by study quality or size.

Introduction

Healthcare workers wanting to keep up to date with recent advances in their specialty must deal with the quantity and quality of information sources.^{1 2} One study estimated that every month journals publish 7287 items (studies, letters, and editorials) relevant to primary care. These would take physicians trained in epidemiology 627 hours to read and appraise. Few healthcare workers have the time and skills to carry out in depth critical appraisal of published articles.³ Given the pressures of everyday work, the time available for reading a scientific article might be as little as 22 minutes.⁴

To inform their conduct and update their knowledge physicians might rely on a brief (two minute) scan of material.^{5 6} This probably includes either browsing the abstract (the “shop window” of an article) or the conclusions paragraph (the “take home message”), especially those published in prestigious journals (that is, those with the highest journal impact factor) that are most readily available or of their digests. The impact factor was specifically developed to facilitate prioritisation of subscription resources, allowing targeting on the most cited journals, considered to be most read and the ones that publish articles of the best quality. These journals are thus more likely to be accessible and their content widely disseminated.^{7 8}

Methodological quality of studies is the other important aspect. Several items, all affecting the reliability of what presents to healthcare workers, are related in a complex fashion. Such quality is associated with the size of the effects reported in a study. The lower the quality, the greater the effect of the intervention seems.^{9 10}

Funding of meta-analyses of antihypertensive drugs by individual pharmaceutical companies has been shown to be inversely associated with whether data presented supported the authors’ conclusions (concordance)¹¹ and directly associated with the probability that study conclusions did or did not support the drugs being evaluated (take home message).¹² The relation between the items of quality, concordance, take home message, and funding of single studies, however, is thought to be specific to the context, drugs evaluated, and outcomes assessed. This implies that generalisation of quality across study designs, interventions, and outcomes cannot be made on current evidence. In addition we found no studies looking at the relation of all these variables with that of study dissemination and possible impact.^{13 14}

We explored the relation between study concordance, take home message, funding, and citation (as a proxy for dissemination) in studies on influenza vaccines by reviewing all published comparative studies. Our choice was motivated by the global nature of the intervention, the almost exclusive analysis by analytical studies of quality items in randomised controlled trials or their meta-analyses (in influenza vaccines there is an abundance of non-randomised evidence), and by the recent publication of studies suggesting poor quality and contradictory evidence from field studies supporting vaccination.^{15 16 17 18 19 20}

Methods

Evidence searches

We systematically reviewed comparative studies on influenza vaccines. We used the bibliography of available up to date systematic reviews of influenza vaccines to identify studies comparing the effects of influenza vaccines with placebo or no intervention (“primary studies”) and carried out supplementary searches for all other comparative primary studies published by the end of 2006 (see appendix on bmj.com for search strategy and procedure).

Data extraction and quality assessment

We extracted primary study data in duplicate according to a list of variables (box 1).

Box 1 Variables extracted from primary studies

• Study identification

• Year(s) of execution of the study (the end of the past influenza season cited in the study)

• Year of publication

• Study design and location

• Number of arms

• Comparison and type of outcome assessed within the same study (papers divided into separate studies or datasets if they contained more than one study or comparison)

• Study population

• Types of influenza vaccine used

• Content of influenza vaccines used

• Match between content and circulating viruses in the community

• Types of outcome assessed

• Intervention arm size

• Number of events observed in the intervention arms

• Types of control

• Size of control arms

• Number of events observed in the control arms

• Type of funder

• Impact factor assigned to the journal for the year after the study was published, number of times the articles have been cited by other sources (citation index factor), conclusions of article (classified according to the categories reported in box 2)

Two reviewers, independently unblinded to the authors and institutions of the article, extracted data from the studies in three phases. Firstly, they read each study and compiled an abstract describing study content. Data presented in the study were summarised on a computerised database based on the Cochrane vaccines field study register. This process involved understanding the design and intent of the authors as well as reading and reconciling each part of the study. We then compared our abstract with the authors’ presentation and summaries of results. To classify concordance and take home message definitions we asked whether data presented in the study supported the authors’ conclusions and then whether the conclusions did or did not support the vaccine(s) being evaluated. We classified take home message of all studies as supporting, neutral, or critical of the effects of influenza vaccines (box 2). The definitions were based on work carried out on study concordance and take home message.^{11 12} We grouped funding sources into government/private/unfunded or industry/mixed, concordance into yes or no/partially/unclear, and take home message into favourable or mixed/unfavourable/unclear.

Box 2 Classification of study take home message

Supportive

• The authors state the dominance of the vaccines even without supporting statistical conclusions

• The study reports statistically significant vaccine effects compared with control intervention(s)

• The authors report the main study end points as statistically significantly dominant (P<0.05) compared with the control arm(s)

• The authors state that vaccination reduces disease burden and/or treatment costs and/or resource consumption

• The authors stress the efficacy, effectiveness, and safety of the vaccines compared with control intervention(s)

• The authors conclude that vaccination should be undertaken

• The authors criticise dissenters on the effects of vaccination

• The authors support at least one of the producers’ or programme sponsors’ statements on the vaccines

Neutral

• The authors conclude that there are insufficient data to assess one or all of the effects of the vaccines

• The authors make no recommendations but seem fairly to weigh up pros and cons of each course of action

• The authors conclude that vaccination is cost neutral

• The authors conclude that the data presented are insufficient to answer the study question

• The authors state that after vaccination the evidence for reduction of disease burden and/or treatment costs and/or resource consumption is slight or non-existent

• The authors dissent from at least one of the producers’ or programme sponsors’ statements on the vaccines but do not reach a definite conclusion

• The authors conclude that the evidence shows efficacy, effectiveness, and safety of the vaccines compared with control intervention(s) for some outcomes but not for others

• The authors conclude that the evidence shows efficacy, effectiveness, and safety of one type of vaccine compared with another and with control intervention(s)

Critical/not supportive

• The authors report no statistically significant efficacy or effectiveness results and/or significant harms

• The authors clearly state the vaccines’ inferiority and present data showing significant vaccine inferiority compared with control(s)

• The authors state that vaccination does not reduce disease burden and/or treatment costs and/or resource consumption

• The authors emphasise concerns over efficacy and/or effectiveness and/or safety and/or recommend alternative interventions to control the burden of disease

• The authors criticise those supporting influenza vaccination or state that vaccination should not be introduced or continued

• The authors are critical of all the producers’ or programme sponsors’ statements on the vaccines

Finally, we assessed quality of randomised and non-randomised studies separately. Randomised studies were assessed according to randomisation method, generation of the allocation sequence, allocation concealment, blinding, and follow-up. Non-randomised studies were assessed for the presence of potential confounders with the appropriate Newcastle-Ottawa scales²⁹ for case-control and cohort studies. We assigned risk of bias (low, moderate, and high risk of bias) as described by the Cochrane Handbook³⁰ by design category (randomised and non-randomised).

Statistical methods

We used χ² tests to assess the association between items such as quality of methods, funding sources, coherence, and take home message.

To assess the level of dissemination we identified the impact factor of the journals publishing each of the studies in the review. A journal’s impact factor is calculated from the number of citations in the current year to items published in the previous two years (numerator), divided by the number of substantive articles and reviews published in the same two years (the denominator). This is proposed as a proxy for journal quality and, by implication, for quality of the research articles in that journal.^{7 8} The impact factor was designed to identify journals worthy of subscription, and its rankings fit well with the perceived prestige of journals⁸ and their circulation.

We identified the citation index of each article (the cumulative total of the number of times the article has been cited in journals indexed in the ISI Web of Science database). The latest impact factor and citation indices were taken from the ISI website http://isiwebofknowledge.com/.

We also tested the association between delay in publication (difference between the publication date and the date the study was completed), the journal impact factor, or the citation index factor against the items described above by Wilcoxon and Kruskal-Wallis non-parametric tests (SAS/STAT 9.1, SAS, Cary, NC). Other studies have described an association between type of take home message and citation and type of funder and citation.^{12 21 22 23 24 25 26 27 28}

Results

Characteristics of primary studies

The figure shows the flow of the included studies.

Flow of primary studies in review (RCT=randomised controlled trial; CCT=clinical controlled trial, semi-randomised trial)

We included 274 comparative studies/datasets (from 259 papers) in our population. These were mostly randomised or semi-randomised (that is, the methods of allocation were alternation or birth date) studies (138/274, 50%) or prospective cohort studies (69/274, 25%). Most studies were carried out on healthy or general populations (203/274, 74%) and were government financed (48%). Seventy per cent of studies reported conclusions favourable to the vaccines but only 18% showed complete concordance between data reported and study conclusions. Over half (56%) of studies were at high risk of bias, with only 4% being at low risk. We did not include any vaccine registration trials.

Association between study quality and concordance, take home message, and funding

Table 1 summarises our main findings. All comparisons assessing the association with study quality were carried out after we excluded studies for which we had insufficient data (4/274). We calculated χ² on the basis of the three methodological quality categories of high, moderate, and low risk of bias. We calculated the χ² for all comparisons of funding sources for three categories (government, industry, missing). Because a high proportion of studies had missing funding information (64/274, 23%) we carried out a full sensitivity analysis to assess robustness of observed associations by systematically assigning studies with unclear funding source to the other known funding categories, thus covering all possible scenarios.

Table 1.

 Summary of main findings of assessment of possible association between methodological quality, concordance between data presented and conclusions reported, take home message of article conclusions, and funding source in comparative studies of influenza vaccines

Is there a relation between:	Odds ratio (95% CI)	Interpretation	Sensitivity analysis carried out? If yes what were the results?
Methodological quality and concordance between data presented and conclusions reported?	Aggregate moderate and high risk of bias: 16.35 (4.24 to 63.04)	Positive association between low risk of bias and concordance	No
Methodological quality and funding source?	Aggregate moderate and high risk of bias, excluding studies with missing funding source: 0.74 (0.19 to 2.84)	No evidence of negative association between governmental funding source and low risk of bias	Yes. Sensitivity analysis carried out on 240 possible scenarios: 1.64% of scenarios with OR <1 significant, 65.4% with OR <1 non-significant; 0.83% with OR=1; and 32.1% with OR >1 non-significant at 5% level
Methodological quality and take home message?	Aggregate moderate and high risk of bias: 0.19 (0.05 to 0.64)	Negative association between low risk of bias and favourable conclusion	No
Concordance between data presented and conclusions reported and take home message?	0.04 (0.02 to 0.09)	Negative association between presence of concordance and favourable conclusion	No
Concordance between data presented and conclusions reported and funding source?	Excluding studies with missing funding source: 1.47 (0.72 to 3.07)	No evidence of positive association between concordance and government funding source at 5% significance level	Yes. Sensitivity analysis carried out on 413 possible scenarios; 16.5% of scenarios with OR <1 and non-significant, 57.9% with OR >1 non-significant, and 25.7% with OR >1 significant at 5% level
Funding source and take home message?	Excluding studies with missing funding source: 0.45 (0.26 to 0.90)	Evidence of negative association between favourable conclusion and government funding	Yes. Sensitivity analysis carried out on 989 possible scenarios; 47.0% of scenarios with OR <1 significant, 38.5% with OR <1 non-significant, 14.4% with OR >1 non-significant, and only 0.1% (one scenario) with OR >1 significant at 5% level

Our analysis shows the presence of a strong positive association between methodological quality and concordance between results presented and study conclusions (when we aggregated studies with moderate and high risk of bias: odds ratio 16.35, 95% confidence interval 4.24 to 63.04). In other words, the higher the study quality (and the lower the risk of bias), the higher the probability of concordance. In addition, the higher the probability of concordance, the lower the probability that a study’s conclusions were in favour of vaccines’ effectiveness (0.04, 0.02 to 0.09); table 2 shows examples of discordance between results and conclusions and take home message. We found no association between funding source and methodological quality (excluding studies with unknown funding: 0.74, 0.19 to 2.84), but there was an inverse association between conclusions in favour of the vaccines’ effectiveness and government funding (0.45, 0.26 to 0.90). This finding confirms the established association between funding source and type of study conclusions.^{21 22 23 24 25 28}

Table 2.

 Examples of discrepancy between results and conclusions

Study	Summary of results	Author’s conclusions and reviewer comments
Wongsurakiat 200431 (randomised placebo controlled trial in 125 adults and older people with COPD)	Significant reduction shown for total acute respiratory infections (ARI) with laboratory confirmation of influenza (RR 0.24, 95% CI 0.06 to 0.7; P=0.005) and influenza like illness (ILI) (0.34, 0.1 to 0.99, P=0.03). For acute exacerbations, difference not significant (0.92, 0.67 to 1.3, P=0.6). In this category 13/21 confirmed cases of influenza isolated, so, as for total ARI episodes (124 in vaccinated and 145 in placebo group), there was no difference between two intervention groups. This applies also to “probability of not being admitted to hospital related to ARI (P=0.2 by log rank test) and probability of not receiving mechanical ventilation related to ARI (P=0.4 by log rank test)”	Study was conducted over one year. Conclusions support recommendation of annual vaccination (one dose is sufficient in adults, as strong response has been observed). Authors note that vaccine effectiveness has been shown, even if it was possibly administered too late (in region where study was carried out peak incidence of influenza occurs usually in May). Comment: though authors state that effectiveness is shown for influenza related ARI only, and not influenza, they recommend vaccination for patients with COPD. This means recommending vaccine though it is not effective against influenza and acute exacerbations. In addition, lack of comment on community viral circulation and vaccine content and matching make verification of effectiveness against ARI impossible
Wilde 199932 (randomised trials on 264 healthy healthcare workers, three consecutive seasons)	Influenza infection (fourfold increase in haemagglutination inhibiting antibody against virus A or B between serum sample after immunisation and after epidemic). Efficacy against A (H3N2) virus estimated as 88% (47% to 97%, P=0.001); against B virus as 89% (14% to 99%, P=0.02). Authors’ conclusions about efficacy derived from cumulative data only. They apply effect measure and significance test (χ2) to cumulative data only. When applied to single comparisons, significance reached only for influenza A in 1992-3 (P=0.026). Days with respiratory illness (52 v 73 days; P=0.57; mean 0.29 (SD 0.68) days v 0.41 (SD 1.0) days, and absence due to illness (18 v 38 days; P=0.41, mean 0.1 (SD 0.35) days v 0.21 (SD 0.75) days no different among vaccinated and control group (three seasons’ cumulative data)	“In conclusion, influenza vaccine is effective in preventing serologically proven influenza infection in young, healthy hospital-based healthcare professionals and may reduce cumulative days of illness and absence. These data suggest that a policy of annual immunization with influenza vaccine in healthcare professionals will reduce influenza infections and can reduce associated illness.” Comment: influenza vaccination is recommended though outcomes are exclusively serological (surrogates) calculated in aggregate over three years (180 in intervention arms and 179 in three different control arms). Clinical outcomes are not significantly affected by vaccination
Carman 200033 (block randomised trial, 20 long term care hospitals)	All cause mortality less common in long term care where vaccination was offered (vaccination rate 13.6%; 102/749) than in those where it was not (vaccination rate 22.4%; 154/688). Crude OR 0.58, 0.40 to 0.84, P=0.014). Significance disappears after adjustment for degree of disability by Barthel scale, age, sex, vaccination of patients (0.61, 0.36 to 1.04, P=0.092). Virological surveillance (routine, from some symptomatic subjects, from some samples taken at death) did not show different frequency of viruses A or B isolates between two groups (culture and PCR)	“Vaccination of health-care workers was associated with a substantial decrease in mortality [for all causes] among patients. However, virological surveillance showed no associated decrease in non-fatal influenza infection in patients.” Comment: implausible conclusion with use of all cause mortality an outcome lacking specificity. Long list of confounders: biased reporting of autopsy sampling, trenchant conclusion despite apparent lack of effect on viral circulation, brief description of vaccine content or matching (in discussion), attrition in serology follow-up, possible selection bias of healthcare workers and patients, higher Barthel score in vaccinated arm. Once data were adjusted for Barthel score, age, and sex no effect was observed

RCT=randomised controlled trial, COPD=chronic obstructive pulmonary disease, RR=relative risk, OR=odds ratio, PCR=polymerase chain reaction.

Association between study quality items, funding, and dissemination

Statistical tests show no evidence of association between the mean time lapsed from the end of the study to the date of publication (publication delay), methodological quality, type of study design, concordance, coherence, and funder. There was no evidence of association between either journal impact factor or citation index factor and study design, methodological quality, and concordance.

The mean journal impact factor of 92 government funded studies was 3.74, and their mean citation index factor was 33.75. We calculated the mean impact factor on 114 studies, as the oldest studies were published in journals for which no impact factor was available. For the 52 studies wholly or partly funded by industry, the mean impact factor was 8.78 and the mean citation index factor was 58.39. We calculated the mean citation index for 74 studies as the oldest studies were published in journals for which no citation index was available.

Non-parametric testing shows that the impact factor score of studies with complete or partial industry funding was significantly higher than that of studies funded by government and other funders (table 3).

Table 3.

 Analysis of relation between journal impact factor, citation index factor, study sample size, publication delay, methodological quality, take home message content, concordance, and source of funding

Is there a relation between:	Statistical tests*	Interpretation	Sensitivity analysis carried out?	If yes, what were results?
Methodological quality and JIF (with aggregate moderate and high risk of bias studies)?	z=1.3, P=0.184	No evidence of difference in mean JIF between studies with low risk of bias and studies with (high or moderate) risk of bias	No	—
Methodological quality and CIF (with aggregate moderate and high risk of bias studies)?	z=−0.19, P=0.851	No evidence of difference in mean CIF between studies with low risk of bias and studies with (high or moderate) risk of bias	No	—
Methodological quality and CSS (with aggregate moderate and high risk of bias studies)?	z=−0.96, P=0.338	No evidence of difference in mean CSS between studies with low risk of bias and studies with (high or moderate) risk of bias	Yes (only with RCT)	No change in interpretation
Methodological quality and PD (with aggregate moderate and high risk of bias studies)?	z=−0.38, P=0.707	No evidence of difference in mean PD between studies with low risk of bias and studies with (high or moderate) risk of bias	No	—
Take home message and JIF?	z=−1.51, P=0.131	No evidence of difference in mean JIF between studies with favourable take home message and studies with mixed/unfavourable take home message	No	—
Take home message and CIF?	z=−1.84, P=0.065	No evidence of difference in mean CIF between studies with favourable take home message and studies with mixed/unfavourable take home message	No	—
Take home message and CSS?	z=−0.41, P=0.682	No evidence of difference in mean CSS between studies with favourable take home message and studies with mixed/unfavourable take home message	Yes (only with RCT)	No change in interpretation
Take home message and PD?	z=−0.89 P=0.375	No evidence of difference in mean PD between studies with favourable take home message and studies with mixed/unfavourable take home message	No	—
Concordance between data presented and conclusions reported and JIF?	z=1.1, P=0.273	No evidence of difference in mean JIF between studies with concordance (yes) and studies with concordance (no/part/unclear)	No	—
Concordance between data presented and conclusions reported and CIF?	z=0.35 P=0.729	No evidence of difference in mean CIF between studies with concordance (yes) and studies with concordance (no/part/unclear)	No	—
Concordance between data presented and conclusions reported and CSS?	z=0.84, P=0.404	No evidence of difference in mean CSS between studies with concordance (yes) and studies with concordance (no/part/unclear)	Yes (only with RCT)	No change in interpretation
Concordance between data presented and conclusions reported and PD?	z=−0.58, P=0.563	No evidence of difference in mean PD between studies with concordance (yes) and studies with concordance (no/part/unclear)	—	—
Funding source and JIF?	χ2=27.4, df=2, P<0.001	Evidence of difference in mean JIF between studies with industry funding source and other funding source. Mean JIF significantly greater in industry funded studies than studies with other funding source	Excluding studies with undeclared funding	No change in interpretation
Funding source and CIF?	χ2=13.5, df=2, P<0.001	Evidence of difference in mean CIF between studies with industry funding source and other funding source. Mean CIF significantly greater in industry funded studies than studies with other funding source	Excluding studies with undeclared funding	Change in interpretation: “no evidence”
Funding source and CSS?	χ2=0.06, df=2, P=0.997	No evidence of difference in mean CSS between industry funded studies and government funded studies	Excluding studies with undeclared funding and only with RCT	No change in interpretation
Funding source and PD?	χ2=0.97, df=2, P=0.616	No evidence of difference in mean PD between industry funded studies and government funded studies	Excluding studies with undeclared funding	No change in interpretation

JIF=journal impact factor; CIF=citation index factor; RCT=randomised controlled trial; CSS=comparator sample size; PD=publication delay (difference between publication year and end of study).

*Kruskal Wallis (χ²) or Wilcoxon (z).

Non-parametric testing for article citation index showed a similar significant difference, which was, however, sensitive to the exclusion of studies with no declared funding source. In this case we found no difference in citation between studies with complete or partial funding from industry and studies funded by government and other funders (table 3).

There was no evidence of association between sample size of comparator and study design, methodological quality, concordance, and take home message.

We conclude that study size is an unlikely explanation for the association between industry sponsorship and publication in journals with higher impact factors and that industry funding is associated with a higher probability of publication in journals with a higher impact factor and possibly a higher citation index.

The differences in impact factor means between studies funded by government and those funded by industry or mixed sponsors were significant and sufficiently robust. Whole or part industrial funding was associated with publication in journals with higher impact factors.

Discussion

In the studies we included, poor methodological quality was associated with a discrepancy between results and conclusions, and this in turn was associated with optimistic conclusions in non-government sponsored studies. We found no direct association between these findings and industry funding, but this might have been affected by the sizeable number of studies with undeclared sponsors (23%).

Studies partly or completely sponsored by industry, however, were published in more prestigious journals and are probably cited more, although their methodological quality and size were similar. Some of these findings might help to explain the continuation of a near global policy, despite growing doubts as to its scientific basis.

We reasoned that the combined impact factor and citation index would give us a good idea of the circulation and dissemination of the study (that is, the interest its publication generated). Most of our studies (70%) were of poor quality with overoptimistic conclusions—that is, not supported by the data presented. Those sponsored by industry had greater visibility as they were more likely to be published by high impact factor journals and were likely to be given higher prominence by the international scientific and lay media, despite their apparent equivalent methodological quality and size compared with studies with other funders. Although differences in citation index by study funding are sensitive to the inclusion of the large number of studies with undeclared sources of funding, the higher impact factor and citation index are probably a reflection of a higher profile of industry sponsored studies and a more thorough dissemination of their content.

There are two possible mechanisms involved. Firstly, the same studies are cited more than the others, possibly because of the systematic nature of the dissemination of their results by industry. We have personally observed this on three recent occasions in which industry representatives presented abstracts or reprints of industry sponsored influenza vaccine studies to decision makers, their advisors, and local researchers in an effort to influence their decisions. Symposiums, conferences, and other types of publication further enhance the dissemination process. Often the abstracts were expensively bound and translated into the local language, a tangible sign of their importance to industry.

We cannot say for certain why industry sponsored studies are more attractive to more prestigious journals, but such journals are preferentially targeted by all studies because of their prominence and prestige, so industry sponsored studies might have a higher probability of acceptance. The two mechanisms might be linked, but further research, especially in other specialties, is required. As a measure of transparency for readers and authors, however, we recommend that once a year editors and publishers should post all sources of income related to the running of the journal.

Our finding of lack of concordance between results and conclusions is similar to those of Yank and colleagues in an industry sponsored meta-analysis of antihypertensive drugs.¹¹ In our studies, however, we found a lack of concordance between results and conclusions associated with poor quality rather than any specific sponsorship. Given our findings of lack of concordance in primary studies, the content of current policy might reflect the gap between results and conclusions—that is, synthesis of evidence for policy making might be carried out at two independent levels: that of results and that of conclusions. In most cases, what you see is not necessarily what you get.

What is already known on this topic

• Study sponsorship is associated with optimistic results

• Influenza vaccination continues to be recommended globally, despite growing doubts about the validity of the scientific evidence underpinning policy recommendations

What this study adds

• Evidence is of poor quality, and studies with conclusions in favour of vaccines are of significantly lower methodological quality

• Influenza vaccines studies sponsored by industry are published in journals with higher impact factors and are cited more but are of similar size and quality to the others

Cite this as: BMJ 2009;338:b354