Freemantle and Irs are wrong to say that only properly randomised trials can provide truly reliable evidence on adverse events, just as these are the only source of convincing data on drug efficacy.1 Harms due to drugs differ from benefits in several ways: they are multifarious and affect fewer individuals, some of whom may have particular susceptibilities. Harms often cannot be identified in advance. In some cases these features militate against the practicable use of randomised trials.
If observational studies show no evidence of harms, randomised trials are certainly necessary. They are always desirable, and some adverse effects can be elicited reliably only in this way. However, there are examples of anecdotal reports that provide definitive evidence of both harms and benefits, making randomised trials unnecessary.2 There are also examples of adverse effects that have only emerged from observational studies, having failed to be elicited by randomised studies.3
If an observational study suggests a serious adverse effect, it would be hard to justify studying it in a randomised trial. If the benefit to harm balance is unknown, and may be unfavourable, erring on the side of caution is justified.4 Should we ignore evidence of an adverse effect, rare but with a high mortality, in preference to the evidence from trials, which even in combination were not powerful enough to detect it?
Two proponents of randomised controlled trials wrote: “Our main wish, from which all others stem, is that RCTs be taken off their pedestal, their exalted position at the top of an artificial evidence hierarchy; that all forms of evidence be appreciated for what they can offer.”5 We should take note.
Competing interests: JKA is editor of Meyler’s Side Effects of Drugs: The International Encyclopedia of Adverse Drug Reactions and Interactions, and of its annual updates, the Side Effects of Drugs Annuals.
References
- 1.Freemantle N, Irs A. Observational evidence for determining drug safety. BMJ 2008;336:627-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ 2007;334:349-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hylek EM, Heiman H, Skates SJ, Sheehan MA, Singer DE. Acetaminophen and other risk factors for excessive warfarin anticoagulation. JAMA 1998;279:657-62. [DOI] [PubMed] [Google Scholar]
- 4.Salpeter S, Greyber E, Pasternak G, Salpeter E. Risk of fatal and nonfatal lactic acidosis with metformin use in type 2 diabetes mellitus. Cochrane Database Syst Rev 2006;(1):CD002967. [DOI] [PubMed] [Google Scholar]
- 5.Jadad AR, Enkin MW. Randomized controlled trials: questions, answers, and musings 2nd ed. Oxford: Blackwell Publishing/BMJ Books, 2007:128.