We have been reading with interest the recent reports of interim estimates of influenza vaccine effectiveness (VE) for the 2014/2015 northern hemisphere season.1–4 The test-negative design has now become established as the standard observational design for timely assessment of influenza VE.5 Studies following this design often report both a crude and adjusted VE estimate. However, it is important to consider our ultimate goal in reporting VE estimates and whether an unadjusted estimate is useful. The term “vaccine effectiveness” implies a causal effect rather than a mere correlation, and estimation of a causal effect requires confounding to be addressed.6
In a typical test-negative study, which is similar to a case-control study, patients with influenza-like illness are enrolled in a clinical setting and tested for influenza. The crude odds ratio is obtained by dividing the odds of vaccination among influenza-positive patients by the odds of vaccination among influenza-negative patients. This measure indicates the correlation of vaccination with influenza, but may not be an accurate estimate of the causal effect of vaccination on the risk of influenza because that association may be confounded. Confounding variables are associated with, but not the result of, both the exposure and the outcome, conditional on all other variables.7 In observational studies, statistical adjustment of estimates (e.g. regression or stratification) is usually necessary to overcome confounding and ensure exchangeability between groups. This adjusted estimate will approximate the causal effect, such as the effectiveness of a vaccine. In observational studies of vaccine effectiveness, including the test-negative study, VE is commonly calculated as 1−ORadj×100%8.
Absent other biases, the difference between the crude odds ratio and the adjusted odds ratio should show the degree of bias caused by confounding. It can therefore be worthwhile to present the crude odds ratio and the adjusted odds ratio. However, we propose here that it is improper to present a “crude VE”, since this value has no causal interpretation and should not be presented as an estimate of VE. Similarly, it should be unnecessary to use the word “adjusted” when reporting VE because adjustment should have been performed in order to present an estimate of a causal effect. We recognise that the differences in crude and adjusted odds ratios may be caused by other biases, including sparse data bias, measurement error or residual confounding. Nevertheless, when reporting an estimate with an implicit causal effect, it can be misleading to report a crude estimate.
Acknowledgments
Financial support
This work has received financial support from the Area of Excellence Scheme of the Hong Kong University Grants Committee (grant no. AoE/M-12/06) and the Harvard Center for Communicable Disease Dynamics from the National Institute of General Medical Sciences (grant no. U54 GM088558). The WHO Collaborating Centre for Reference and Research on Influenza is funded by the Australian Government Department of Health. The funding bodies were not involved in the collection, analysis, and interpretation of data, the writing of the article, or the decision to submit it for publication.
BJC has received research funding from MedImmune Inc. and Sanofi Pasteur, and consults for Crucell NV.
Footnotes
Potential conflicts of interest
The authors report no other potential conflicts of interest.
REFERENCES
- 1.Skowronski D, Chambers C, Sabaiduc S, De Serres G, Dickinson J, Winter A, Drews S, Fonseca K, Charest H, Gubbay J, Petric M, Krajden M, Kwindt T, Martineau C, Eshaghi A, Bastien N, Li Y. Interim estimates of 2014/15 vaccine effectiveness against influenza A(H3N2) from Canada s Sentinel Physician Surveillance Network, January 2015. Euro Surveill. 2015;20(4) doi: 10.2807/1560-7917.es2015.20.4.21022. [DOI] [PubMed] [Google Scholar]
- 2.Pebody R, Warburton F, Ellis J, Andrews N, Thompson C, von Wissmann B, Green H, Cottrell S, Johnston J, de Lusignan S, Moore C, Gunson R, Robertson C, McMenamin J, Zambon M. Low effectiveness of seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: 2014/15 mid-season results. Euro Surveill. 2015;20(5) [PubMed] [Google Scholar]
- 3.Flannery B, Clippard J, Zimmerman RK, Nowalk MP, Jackson ML, Jackson LA, Monto AS, Petrie JG, McLean HQ, Belongia EA, Gaglani M, Berman L, Foust A, Sessions W, Thaker SN, Spencer S, Fry AM. Early estimates of seasonal influenza vaccine effectiveness - United States, january 2015. MMWR Morb Mortal Wkly Rep. 2015;64(1):10–15. [PMC free article] [PubMed] [Google Scholar]
- 4.McNeil S, Andrew M, Ye L, Haguinet F, Hatchette T, ElSherif M, LeBlanc J, Ambrose A, McGeer A, McElhaney J, Loeb M, MacKinnon-Cameron D, Sharma R, Dos Santos G, Shinde V Investigators of the Serious Outcomes Surveillance Network of the Canadian Immunization Research N. Interim estimates of 2014/15 influenza vaccine effectiveness in preventing laboratory-confirmed influenza-related hospitalisation from the Serious Outcomes Surveillance Network of the Canadian Immunization Research Network, January 2015. Euro Surveill. 2015;20(5) doi: 10.2807/1560-7917.es2015.20.5.21024. [DOI] [PubMed] [Google Scholar]
- 5.Sullivan SG, Feng S, Cowling BJ. Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review. Expert Rev Vaccines. 2014;13(12):1571–1591. doi: 10.1586/14760584.2014.966695. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Greenland S, Rothman KJ, Lash TL. Measures of Effect and Measures of Association. In: Rothman KJ, Greenland S, Lash TL, editors. Modern Epidemiology. 3rd ed. Philadelphia: Lippincott, Williams & Wilkins; 2008. pp. 51–70. [Google Scholar]
- 7.Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999;10(1):37–48. [PubMed] [Google Scholar]
- 8.Halloran ME, Longini IM, Struchiner CJ. Statistics for Biology and Health. New York: Springer; 2010. Design and Analysis of Vaccine Studies. [Google Scholar]