Skip to main content
PMC home page
The BMJ logoLink to The BMJ
. 1994 Nov 19;309(6965):1351–1355. doi: 10.1136/bmj.309.6965.1351

Why sources of heterogeneity in meta-analysis should be investigated.

S G Thompson 1
PMCID: PMC2541868  PMID: 7866085

Abstract

Although meta-analysis is now well established as a method of reviewing evidence, an uncritical use of the technique can be very misleading. One common problem is the failure to investigate appropriately the sources of heterogeneity, in particular the clinical differences between the studies included. This paper distinguishes between the concepts of clinical and statistical heterogeneity and exemplifies the importance of investigating heterogeneity by using published meta-analyses of epidemiological studies of serum cholesterol concentration and clinical trials of its reduction. Although not without some dangers of speculative conclusions, prompted by overzealous inspection of the data to hand, a sensible investigation of sources of heterogeneity should increase both the scientific and the clinical relevance of the results of meta-analyses.

Full text

PDF

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Bailey K. R. Inter-study differences: how should they influence the interpretation and analysis of results? Stat Med. 1987 Apr-May;6(3):351–360. doi: 10.1002/sim.4780060327. [DOI] [PubMed] [Google Scholar]
  2. Brand R., Kragt H. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Stat Med. 1992 Dec;11(16):2077–2082. doi: 10.1002/sim.4780111605. [DOI] [PubMed] [Google Scholar]
  3. DerSimonian R., Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986 Sep;7(3):177–188. doi: 10.1016/0197-2456(86)90046-2. [DOI] [PubMed] [Google Scholar]
  4. Dickersin K., Berlin J. A. Meta-analysis: state-of-the-science. Epidemiol Rev. 1992;14:154–176. doi: 10.1093/oxfordjournals.epirev.a036084. [DOI] [PubMed] [Google Scholar]
  5. Easterbrook P. J., Berlin J. A., Gopalan R., Matthews D. R. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867–872. doi: 10.1016/0140-6736(91)90201-y. [DOI] [PubMed] [Google Scholar]
  6. Galbraith R. F. A note on graphical presentation of estimated odds ratios from several clinical trials. Stat Med. 1988 Aug;7(8):889–894. doi: 10.1002/sim.4780070807. [DOI] [PubMed] [Google Scholar]
  7. Grantham-McGregor S. M., Powell C. A., Walker S. P., Himes J. H. Nutritional supplementation, psychosocial stimulation, and mental development of stunted children: the Jamaican Study. Lancet. 1991 Jul 6;338(8758):1–5. doi: 10.1016/0140-6736(91)90001-6. [DOI] [PubMed] [Google Scholar]
  8. Hughes M. D., Freedman L. S., Pocock S. J. The impact of stopping rules on heterogeneity of results in overviews of clinical trials. Biometrics. 1992 Mar;48(1):41–53. [PubMed] [Google Scholar]
  9. Law M. R., Thompson S. G. Low serum cholesterol and the risk of cancer: an analysis of the published prospective studies. Cancer Causes Control. 1991 Jul;2(4):253–261. doi: 10.1007/BF00052142. [DOI] [PubMed] [Google Scholar]
  10. Law M. R., Wald N. J., Thompson S. G. By how much and how quickly does reduction in serum cholesterol concentration lower risk of ischaemic heart disease? BMJ. 1994 Feb 5;308(6925):367–372. doi: 10.1136/bmj.308.6925.367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. MacMahon S., Peto R., Cutler J., Collins R., Sorlie P., Neaton J., Abbott R., Godwin J., Dyer A., Stamler J. Blood pressure, stroke, and coronary heart disease. Part 1, Prolonged differences in blood pressure: prospective observational studies corrected for the regression dilution bias. Lancet. 1990 Mar 31;335(8692):765–774. doi: 10.1016/0140-6736(90)90878-9. [DOI] [PubMed] [Google Scholar]
  12. Manolio T. A., Pearson T. A., Wenger N. K., Barrett-Connor E., Payne G. H., Harlan W. R. Cholesterol and heart disease in older persons and women. Review of an NHLBI workshop. Ann Epidemiol. 1992 Jan-Mar;2(1-2):161–176. doi: 10.1016/1047-2797(92)90051-q. [DOI] [PubMed] [Google Scholar]
  13. Oxman A. D., Guyatt G. H. A consumer's guide to subgroup analyses. Ann Intern Med. 1992 Jan 1;116(1):78–84. doi: 10.7326/0003-4819-116-1-78. [DOI] [PubMed] [Google Scholar]
  14. Pagliaro L., D'Amico G., Sörensen T. I., Lebrec D., Burroughs A. K., Morabito A., Tiné F., Politi F., Traina M. Prevention of first bleeding in cirrhosis. A meta-analysis of randomized trials of nonsurgical treatment. Ann Intern Med. 1992 Jul 1;117(1):59–70. doi: 10.7326/0003-4819-117-1-59. [DOI] [PubMed] [Google Scholar]
  15. Peto R. Why do we need systematic overviews of randomized trials? Stat Med. 1987 Apr-May;6(3):233–244. doi: 10.1002/sim.4780060306. [DOI] [PubMed] [Google Scholar]
  16. Ravnskov U. Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome. BMJ. 1992 Jul 4;305(6844):15–19. doi: 10.1136/bmj.305.6844.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Senn S. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Stat Med. 1994 Feb 15;13(3):293–296. doi: 10.1002/sim.4780130310. [DOI] [PubMed] [Google Scholar]
  18. Shipley M. J., Pocock S. J., Marmot M. G. Does plasma cholesterol concentration predict mortality from coronary heart disease in elderly people? 18 year follow up in Whitehall study. BMJ. 1991 Jul 13;303(6794):89–92. doi: 10.1136/bmj.303.6794.89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Smith G. D., Song F., Sheldon T. A. Cholesterol lowering and mortality: the importance of considering initial level of risk. BMJ. 1993 May 22;306(6889):1367–1373. doi: 10.1136/bmj.306.6889.1367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Stewart L. A., Parmar M. K. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993 Feb 13;341(8842):418–422. doi: 10.1016/0140-6736(93)93004-k. [DOI] [PubMed] [Google Scholar]
  21. Thompson S. G. Controversies in meta-analysis: the case of the trials of serum cholesterol reduction. Stat Methods Med Res. 1993;2(2):173–192. doi: 10.1177/096228029300200205. [DOI] [PubMed] [Google Scholar]
  22. Whitehead A., Whitehead J. A general parametric approach to the meta-analysis of randomized clinical trials. Stat Med. 1991 Nov;10(11):1665–1677. doi: 10.1002/sim.4780101105. [DOI] [PubMed] [Google Scholar]
  23. Yusuf S., Wittes J., Probstfield J., Tyroler H. A. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA. 1991 Jul 3;266(1):93–98. [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES