Skip to main content
PMC home page
Journal of Epidemiology and Community Health logoLink to Journal of Epidemiology and Community Health
. 2002 Dec;56(12):951–955. doi: 10.1136/jech.56.12.951

Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications

E Hak 1, T Verheij 1, D Grobbee 1, K Nichol 1, A Hoes 1
PMCID: PMC1756997  PMID: 12461118

Abstract

Randomised allocation of vaccine or placebo is the preferred method to assess the effects of the vaccine on clinical outcomes relevant to the individual patient. In the absence of phase 3 trials using clinical end points, notably post-influenza complications, alternative non-experimental designs to evaluate vaccine effects or safety are often used. The application of these designs may, however, lead to invalid estimates of vaccine effectiveness or safety. As patients with poor prognosis are more likely to be immunised, selection for vaccination is confounded by patient factors that are also related to clinical end points. This paper describes several design and analytical methods aimed at limiting or preventing this confounding by indication in non-experimental studies. In short, comparison of study groups with similar prognosis, restriction of the study population, and statistical adjustment for dissimilarities in prognosis are important tools and should be considered. Only if the investigator is able to show that confounding by indication is sufficiently controlled for, results of a non-experimental study may be of use to direct an evidence based vaccine policy.

Full Text

The Full Text of this article is available as a PDF (104.6 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Ahmed A. E., Nicholson K. G., Nguyen-Van-Tam J. S. Reduction in mortality associated with influenza vaccine during 1989-90 epidemic. Lancet. 1995 Sep 2;346(8975):591–595. doi: 10.1016/s0140-6736(95)91434-x. [DOI] [PubMed] [Google Scholar]
  2. Barker W. H., Borisute H., Cox C. A study of the impact of influenza on the functional status of frail older people. Arch Intern Med. 1998 Mar 23;158(6):645–650. doi: 10.1001/archinte.158.6.645. [DOI] [PubMed] [Google Scholar]
  3. Clemens J., Brenner R., Rao M., Tafari N., Lowe C. Evaluating new vaccines for developing countries. Efficacy or effectiveness? JAMA. 1996 Feb 7;275(5):390–397. [PubMed] [Google Scholar]
  4. Connors A. F., Jr, Speroff T., Dawson N. V., Thomas C., Harrell F. E., Jr, Wagner D., Desbiens N., Goldman L., Wu A. W., Califf R. M. The effectiveness of right heart catheterization in the initial care of critically ill patients. SUPPORT Investigators. JAMA. 1996 Sep 18;276(11):889–897. doi: 10.1001/jama.276.11.889. [DOI] [PubMed] [Google Scholar]
  5. Fedson D. S., Hirota Y., Shin H. K., Cambillard P. E., Kiely J., Ambrosch F., Hannoun C., Leese J., Sprenger M. J., Hampson A. W. Influenza vaccination in 22 developed countries: an update to 1995. Vaccine. 1997 Oct;15(14):1506–1511. doi: 10.1016/s0264-410x(97)00091-1. [DOI] [PubMed] [Google Scholar]
  6. Fedson D. S., Wajda A., Nicol J. P., Hammond G. W., Kaiser D. L., Roos L. L. Clinical effectiveness of influenza vaccination in Manitoba. JAMA. 1993 Oct 27;270(16):1956–1961. [PubMed] [Google Scholar]
  7. Fleming T. R., DeMets D. L. Surrogate end points in clinical trials: are we being misled? Ann Intern Med. 1996 Oct 1;125(7):605–613. doi: 10.7326/0003-4819-125-7-199610010-00011. [DOI] [PubMed] [Google Scholar]
  8. Frank J. W., Henderson M., McMurray L. Influenza vaccination in the elderly: 1. Determinants of acceptance. Can Med Assoc J. 1985 Feb 15;132(4):371–375. [PMC free article] [PubMed] [Google Scholar]
  9. Glezen W. P. Serious morbidity and mortality associated with influenza epidemics. Epidemiol Rev. 1982;4:25–44. doi: 10.1093/oxfordjournals.epirev.a036250. [DOI] [PubMed] [Google Scholar]
  10. Greenland S., Neutra R. Control of confounding in the assessment of medical technology. Int J Epidemiol. 1980 Dec;9(4):361–367. doi: 10.1093/ije/9.4.361. [DOI] [PubMed] [Google Scholar]
  11. Grobbee D. E., Hoes A. W. Confounding and indication for treatment in evaluation of drug treatment for hypertension. BMJ. 1997 Nov 1;315(7116):1151–1154. doi: 10.1136/bmj.315.7116.1151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gross P. A., Hermogenes A. W., Sacks H. S., Lau J., Levandowski R. A. The efficacy of influenza vaccine in elderly persons. A meta-analysis and review of the literature. Ann Intern Med. 1995 Oct 1;123(7):518–527. doi: 10.7326/0003-4819-123-7-199510010-00008. [DOI] [PubMed] [Google Scholar]
  13. Hak E., van Essen G. A., Buskens E., Stalman W., de Melker R. A. Is immunising all patients with chronic lung disease in the community against influenza cost effective? Evidence from a general practice based clinical prospective cohort study in Utrecht, The Netherlands. J Epidemiol Community Health. 1998 Feb;52(2):120–125. doi: 10.1136/jech.52.2.120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Hak E., van Essen G. A., Stalman W. A., de Melker R. A. Improving influenza vaccination coverage among high-risk patients: a role for computer-supported prevention strategy? Fam Pract. 1998 Apr;15(2):138–143. doi: 10.1093/fampra/15.2.138. [DOI] [PubMed] [Google Scholar]
  15. Joffe M. M., Rosenbaum P. R. Invited commentary: propensity scores. Am J Epidemiol. 1999 Aug 15;150(4):327–333. doi: 10.1093/oxfordjournals.aje.a010011. [DOI] [PubMed] [Google Scholar]
  16. Kramarz P., DeStefano F., Gargiullo P. M., Davis R. L., Chen R. T., Mullooly J. P., Black S. B., Bohlke K., Ward J. I., Marcy M. S. Influenza vaccination in children with asthma in health maintenance organizations. Vaccine Safety Datalink Team. Vaccine. 2000 Apr 28;18(21):2288–2294. doi: 10.1016/s0264-410x(99)00551-4. [DOI] [PubMed] [Google Scholar]
  17. McClellan M., McNeil B. J., Newhouse J. P. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994 Sep 21;272(11):859–866. [PubMed] [Google Scholar]
  18. Miettinen O. S. Stratification by a multivariate confounder score. Am J Epidemiol. 1976 Dec;104(6):609–620. doi: 10.1093/oxfordjournals.aje.a112339. [DOI] [PubMed] [Google Scholar]
  19. Neuzil K. M., Reed G. W., Mitchel E. F., Jr, Griffin M. R. Influenza-associated morbidity and mortality in young and middle-aged women. JAMA. 1999 Mar 10;281(10):901–907. doi: 10.1001/jama.281.10.901. [DOI] [PubMed] [Google Scholar]
  20. Nichol K. L., Baken L., Nelson A. Relation between influenza vaccination and outpatient visits, hospitalization, and mortality in elderly persons with chronic lung disease. Ann Intern Med. 1999 Mar 2;130(5):397–403. doi: 10.7326/0003-4819-130-5-199903020-00003. [DOI] [PubMed] [Google Scholar]
  21. Nichol K. L., Baken L., Nelson A. Relation between influenza vaccination and outpatient visits, hospitalization, and mortality in elderly persons with chronic lung disease. Ann Intern Med. 1999 Mar 2;130(5):397–403. doi: 10.7326/0003-4819-130-5-199903020-00003. [DOI] [PubMed] [Google Scholar]
  22. Nichol K. L., Margolis K. L., Wuorenma J., Von Sternberg T. The efficacy and cost effectiveness of vaccination against influenza among elderly persons living in the community. N Engl J Med. 1994 Sep 22;331(12):778–784. doi: 10.1056/NEJM199409223311206. [DOI] [PubMed] [Google Scholar]
  23. Nichol K. L., Wuorenma J., von Sternberg T. Benefits of influenza vaccination for low-, intermediate-, and high-risk senior citizens. Arch Intern Med. 1998 Sep 14;158(16):1769–1776. doi: 10.1001/archinte.158.16.1769. [DOI] [PubMed] [Google Scholar]
  24. Ohmit S. E., Monto A. S. Influenza vaccine effectiveness in preventing hospitalization among the elderly during influenza type A and type B seasons. Int J Epidemiol. 1995 Dec;24(6):1240–1248. doi: 10.1093/ije/24.6.1240. [DOI] [PubMed] [Google Scholar]
  25. Rodrigues L. C., Smith P. G. Use of the case-control approach in vaccine evaluation: efficacy and adverse effects. Epidemiol Rev. 1999;21(1):56–72. doi: 10.1093/oxfordjournals.epirev.a017988. [DOI] [PubMed] [Google Scholar]
  26. Rothbarth P. H., Kempen B. M., Sprenger M. J. Sense and nonsense of influenza vaccination in asthma and chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 1995 May;151(5):1682–1686. doi: 10.1164/ajrccm/151.5_Pt_1.1682. [DOI] [PubMed] [Google Scholar]
  27. Rubin D. B. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997 Oct 15;127(8 Pt 2):757–763. doi: 10.7326/0003-4819-127-8_part_2-199710151-00064. [DOI] [PubMed] [Google Scholar]
  28. Schilder A. G., Hak E., Straatman H., Zielhuis G. A., van Bon W. H., van den Broek P. Long-term effects of ventilation tubes for persistent otitis media with effusion in children. Clin Otolaryngol Allied Sci. 1997 Oct;22(5):423–429. doi: 10.1046/j.1365-2273.1997.00047.x. [DOI] [PubMed] [Google Scholar]
  29. Watkins J. Effectiveness of influenza vaccination policy at targeting patients at high risk of complications during winter 1994-5: cross sectional survey. BMJ. 1997 Oct 25;315(7115):1069–1070. doi: 10.1136/bmj.315.7115.1069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. van Essen G. A., Kuyvenhoven M. M., de Melker R. A. Compliance with influenza vaccination. Its relation with epidemiologic and sociopsychological factors. Arch Fam Med. 1997 Mar-Apr;6(2):157–163. doi: 10.1001/archfami.6.2.157. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Epidemiology and Community Health are provided here courtesy of BMJ Publishing Group

RESOURCES