Skip to main content
The BMJ logoLink to The BMJ
. 2001 Sep 8;323(7312):546. doi: 10.1136/bmj.323.7312.546

Assessment of therapeutic safety in systematic reviews: literature review

Edzard Ernst 1, Max H Pittler 1
PMCID: PMC48159  PMID: 11546700

Evidence based medicine aims to objectively and continually evaluate all medical interventions with a view to incorporating the knowledge gained into routine healthcare practice. The ultimate goal is to enhance patient care. The most powerful tools used in evidence based medicine are systematic reviews and meta-analyses.1 Generally, these studies focus on the efficacy or effectiveness of therapeutic interventions; indisputably, however, information on safety is equally important for making informed, evidence based decisions on the value of a given treatment. We evaluated two major medical databases to assess the extent to which therapeutic safety is addressed in systematic reviews and meta-analyses.

Methods and results

We performed systematic literature searches to identify all systematic reviews and meta-analyses on the efficacy or effectiveness and the safety of any type of therapeutic intervention. Data sources were Medline (1966-December 2000) and the Cochrane Library (2000, issue 4). The title, abstract, and keywords of papers were searched using the terms efficacy, effectiveness, systematic review, meta-analysis, safety, adverse effect, adverse event, and adverse reaction and their derivatives. No language restrictions were imposed. The title and abstract of all studies were read and data were according to the criteria defined. Studies were included only if they stated that they were systematic reviews or meta-analyses or if they indicated that the available data were systematically searched for, appraised, and summarised. Studies that included no abstract or that provided no data on humans were excluded. Data were assessed according to the year of the study, and the studies were categorised into three categories: systematic reviews on efficacy or effectiveness, systematic reviews that included safety aspects as a secondary outcome measure (safety category A), and systematic reviews with a primary focus on the safety of the therapy (safety category B). Study selection, data extraction, and evaluation were performed independently by the two investigators, and discrepancies were resolved through discussion.

The results (table) showed a continuous increase in the number of systematic reviews and meta-analyses published between 1966 and December 2000. The number of studies that included safety aspects as a secondary outcome measure (category A) amounted to just over one quarter of the total number of studies on efficacy or effectiveness. The number of systematic reviews that assessed safety aspects as the primary focus of the investigation (category B) amounted to only 3-5% (table).

Comment

Systematic reviews and meta-analyses provide little information on the safety aspects of therapeutic interventions. Most systematic reviews include data on safety only in as much as the primary studies, usually randomised clinical trials, report on adverse events. This seems inadequate. Randomised clinical trials typically assess only a small number of patients and thus the chances of detecting rare adverse events are small. Randomised clinical trials are usually of short duration and thus cannot identify delayed adverse events.2 The assessment of safety has to go far beyond randomised clinical trials and should use various methods, including post-marketing surveillance studies, spontaneous reporting schemes, and epidemiological investigations.3 Systematic reviews on the safety of therapeutic interventions should take these considerations into account and should combine data from various types of studies.4 Safety aspects have so far not been adequately investigated by systematic reviews. One challenge of evidence based medicine therefore is to address safety issues more systematically than has been done before.

Table.

Systematic reviews and meta-analyses of therapeutic interventions in Medline and the Cochrane Library. Values are numbers (percentages) of trials

Time period Source Efficacy or effectiveness Safety
Safety Category A Safety Category B
1966-70 Medline    0 0 0
Cochrane Library    1 0 0
1971-75 Medline    0 0 0
Cochrane Library    0 0 0
1976-80 Medline    1 0   1 (100)
Cochrane Library    2 0 0
1981-85 Medline   11 0 0
Cochrane Library   12 0 0
1986-90 Medline  103  10 (10) 5 (5)
Cochrane Library   71   8 (11) 3 (4)
1991-95 Medline  431  60 (14) 19 (4)
Cochrane Library  406  61 (15) 12 (3)
1996-December 2000 Medline 1005 259 (26) 41 (4)
Cochrane Library 1462 398 (27) 54 (4) 

Category A=study included safety as a secondary outcome measure. Category B=study included safety as a primary focus. 

Footnotes

Funding: None.

Competing interests: None declared.

References

  • 1.Mulrow C. Rationale of systematic reviews. BMJ. 1994;309:597–599. doi: 10.1136/bmj.309.6954.597. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Rawlins MD. Pharmacovigilance: paradise lost, regained or postponed? J R Coll Physicians Lond. 1995;29:41–49. [PMC free article] [PubMed] [Google Scholar]
  • 3.Ernst E, Barnes J. Methodological approaches to investigating the safety of complementary medicine. Complement Ther Med. 1998;6:115–121. [Google Scholar]
  • 4.Stevinson C, Ernst E. Safety of hypericum in patients with depression. CNS Drugs. 1999;11:125–132. [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES