Skip to main content
PMC home page
The BMJ logoLink to The BMJ
. 1996 Sep 21;313(7059):735–738. doi: 10.1136/bmj.313.7059.735

The relation between treatment benefit and underlying risk in meta-analysis.

S J Sharp 1, S G Thompson 1, D G Altman 1
PMCID: PMC2352108  PMID: 8819447

Abstract

In meta-analyses of clinical trials comparing a treated group with a control group it has been common to ask whether the treatment benefit varies according to the underlying risk of the patients in the different trials, with the hope of defining which patients would benefit most and which least from medical interventions. The usual analysis used to investigate this issue, however, which uses the observed proportions of events in the control groups of the trials as a measure of the underlying risk, is flawed and produces seriously misleading results. This arises through a bias due to regression to the mean and will be particularly acute in meta-analyses which include some small trials or in which the variability in the true underlying risks across trials is small. Approaches which previously have been thought to be more appropriate are to substitute the average proportion of events in the control and treated groups as the measure of underlying risk or to plot the proportion of events in the treated group against that in the control group (L'Abbé plot). However, these are still subject to bias in most circumstances. Because of the potentially seriously flawed conclusions that can result from such analyses, they should be replaced either by statistically appropriate (but more complex) approaches or, preferably, by analyses which investigate the dependence of the treatment effect on measured baseline characteristics of the patients in each trial.

Full text

PDF
735

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Antman E. M., Lau J., Kupelnick B., Mosteller F., Chalmers T. C. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992 Jul 8;268(2):240–248. [PubMed] [Google Scholar]
  2. Appel L. J., Miller E. R., 3rd, Seidler A. J., Whelton P. K. Does supplementation of diet with 'fish oil' reduce blood pressure? A meta-analysis of controlled clinical trials. Arch Intern Med. 1993 Jun 28;153(12):1429–1438. [PubMed] [Google Scholar]
  3. Berkey C. S., Hoaglin D. C., Mosteller F., Colditz G. A. A random-effects regression model for meta-analysis. Stat Med. 1995 Feb 28;14(4):395–411. doi: 10.1002/sim.4780140406. [DOI] [PubMed] [Google Scholar]
  4. Bland J. M., Altman D. G. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995 Oct 21;346(8982):1085–1087. doi: 10.1016/s0140-6736(95)91748-9. [DOI] [PubMed] [Google Scholar]
  5. Bland J. M., Altman D. G. Regression towards the mean. BMJ. 1994 Jun 4;308(6942):1499–1499. doi: 10.1136/bmj.308.6942.1499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bland J. M., Altman D. G. Some examples of regression towards the mean. BMJ. 1994 Sep 24;309(6957):780–780. doi: 10.1136/bmj.309.6957.780. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Boissel J. P., Collet J. P., Lievre M., Girard P. An effect model for the assessment of drug benefit: example of antiarrhythmic drugs in postmyocardial infarction patients. J Cardiovasc Pharmacol. 1993 Sep;22(3):356–363. doi: 10.1097/00005344-199309000-00003. [DOI] [PubMed] [Google Scholar]
  8. Brand R., Kragt H. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Stat Med. 1992 Dec;11(16):2077–2082. doi: 10.1002/sim.4780111605. [DOI] [PubMed] [Google Scholar]
  9. Cattaneo M., Mannucci P. M. Desmopressin and blood loss after cardiac surgery. Lancet. 1993 Sep 25;342(8874):812–812. doi: 10.1016/0140-6736(93)91581-6. [DOI] [PubMed] [Google Scholar]
  10. Chalmers T. C. Problems induced by meta-analyses. Stat Med. 1991 Jun;10(6):971–980. doi: 10.1002/sim.4780100618. [DOI] [PubMed] [Google Scholar]
  11. Egger M., Smith G. D. Risks and benefits of treating mild hypertension: a misleading meta-analysis? J Hypertens. 1995 Jul;13(7):813–815. doi: 10.1097/00004872-199507000-00014. [DOI] [PubMed] [Google Scholar]
  12. Glasziou P. P., Irwig L. M. An evidence based approach to individualising treatment. BMJ. 1995 Nov 18;311(7016):1356–1359. doi: 10.1136/bmj.311.7016.1356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hayes R. J. Methods for assessing whether change depends on initial value. Stat Med. 1988 Sep;7(9):915–927. doi: 10.1002/sim.4780070903. [DOI] [PubMed] [Google Scholar]
  14. Hoes A. W., Grobbee D. E., Lubsen J. Does drug treatment improve survival? Reconciling the trials in mild-to-moderate hypertension. J Hypertens. 1995 Jul;13(7):805–811. [PubMed] [Google Scholar]
  15. Horwitz R. I. "Large-scale randomized evidence: large, simple trials and overviews of trials": discussion. A clinician's perspective on meta-analyses. J Clin Epidemiol. 1995 Jan;48(1):41–44. doi: 10.1016/0895-4356(94)00171-l. [DOI] [PubMed] [Google Scholar]
  16. L'Abbé K. A., Detsky A. S., O'Rourke K. Meta-analysis in clinical research. Ann Intern Med. 1987 Aug;107(2):224–233. doi: 10.7326/0003-4819-107-2-224. [DOI] [PubMed] [Google Scholar]
  17. Pagliaro L., D'Amico G., Sörensen T. I., Lebrec D., Burroughs A. K., Morabito A., Tiné F., Politi F., Traina M. Prevention of first bleeding in cirrhosis. A meta-analysis of randomized trials of nonsurgical treatment. Ann Intern Med. 1992 Jul 1;117(1):59–70. doi: 10.7326/0003-4819-117-1-59. [DOI] [PubMed] [Google Scholar]
  18. Rothwell P. M. Can overall results of clinical trials be applied to all patients? Lancet. 1995 Jun 24;345(8965):1616–1619. doi: 10.1016/s0140-6736(95)90120-5. [DOI] [PubMed] [Google Scholar]
  19. Senn S. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Stat Med. 1994 Feb 15;13(3):293–296. doi: 10.1002/sim.4780130310. [DOI] [PubMed] [Google Scholar]
  20. Smith G. D., Egger M. Who benefits from medical interventions? BMJ. 1994 Jan 8;308(6921):72–74. doi: 10.1136/bmj.308.6921.72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Smith G. D., Song F., Sheldon T. A. Cholesterol lowering and mortality: the importance of considering initial level of risk. BMJ. 1993 May 22;306(6889):1367–1373. doi: 10.1136/bmj.306.6889.1367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Thompson S. G. Why sources of heterogeneity in meta-analysis should be investigated. BMJ. 1994 Nov 19;309(6965):1351–1355. doi: 10.1136/bmj.309.6965.1351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Van Houwelingen H. C., Zwinderman K. H., Stijnen T. A bivariate approach to meta-analysis. Stat Med. 1993 Dec 30;12(24):2273–2284. doi: 10.1002/sim.4780122405. [DOI] [PubMed] [Google Scholar]
  24. Victor N. "The challenge of meta-analysis": discussion. Indications and contra-indications for meta-analysis. J Clin Epidemiol. 1995 Jan;48(1):5–8. doi: 10.1016/0895-4356(94)00107-2. [DOI] [PubMed] [Google Scholar]
  25. Yusuf S., Zucker D., Peduzzi P., Fisher L. D., Takaro T., Kennedy J. W., Davis K., Killip T., Passamani E., Norris R. Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration. Lancet. 1994 Aug 27;344(8922):563–570. doi: 10.1016/s0140-6736(94)91963-1. [DOI] [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES