Skip to main content
PMC home page
The BMJ logoLink to The BMJ
. 1994 Nov 12;309(6964):1286–1291. doi: 10.1136/bmj.309.6964.1286

Identifying relevant studies for systematic reviews.

K Dickersin 1, R Scherer 1, C Lefebvre 1
PMCID: PMC2541778  PMID: 7718048

Abstract

OBJECTIVE--To examine the sensitivity and precision of Medline searching for randomised clinical trials. DESIGN--Comparison of results of Medline searches to a "gold standard" of known randomised clinical trials in ophthalmology published in 1988; systematic review (meta-analysis) of results of similar, but separate, studies from many fields of medicine. POPULATIONS--Randomised clinical trials published in 1988 in journals indexed in Medline, and those not indexed in Medline and identified by hand search, comprised the gold standard. Gold standards for the other studies combined in the meta-analysis were based on: randomised clinical trials published in any journal, whether indexed in Medline or not; those published in any journal indexed in Medline; or those published in a selected group of journals indexed in Medline. MAIN OUTCOME MEASURE--Sensitivity (proportion of the total number of known randomised clinical trials identified by the search) and precision (proportion of publications retrieved by Medline that were actually randomised clinical trials) were calculated for each study and combined to obtain weighted means. Searches producing the "best" sensitivity were used for sensitivity and precision estimates when multiple searches were performed. RESULTS--The sensitivity of searching for ophthalmology randomised clinical trials published in 1988 was 82%, when the gold standard was for any journal, 87% for any journal indexed in Medline, and 88% for selected journals indexed in Medline. Weighted means for sensitivity across all studies were 51%, 77%, and 63%, respectively. The weighted mean for precision was 8% (median 32.5%). Most searchers seemed not to use freetext subject terms and truncation of those terms. CONCLUSION--Although the indexing terms available for searching Medline for randomised clinical trials have improved, sensitivity still remains unsatisfactory. A mechanism is needed to "'register" known trials, preferably by retrospective tagging of Medline entries, and incorporating trials published before 1966 and in journals not indexed by Medline into the system.

Full text

PDF
1291

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Bernstein F. The retrieval of randomized clinical trials in liver diseases from the medical literature: manual versus MEDLARS searches. Control Clin Trials. 1988 Mar;9(1):23–31. doi: 10.1016/0197-2456(88)90006-2. [DOI] [PubMed] [Google Scholar]
  2. Chalmers I., Dickersin K., Chalmers T. C. Getting to grips with Archie Cochrane's agenda. BMJ. 1992 Oct 3;305(6857):786–788. doi: 10.1136/bmj.305.6857.786. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Chalmers T. C., Berrier J., Sacks H. S., Levin H., Reitman D., Nagalingam R. Meta-analysis of clinical trials as a scientific discipline. II: Replicate variability and comparison of studies that agree and disagree. Stat Med. 1987 Oct-Nov;6(7):733–744. doi: 10.1002/sim.4780060704. [DOI] [PubMed] [Google Scholar]
  4. Daya S. Efficacy of progesterone support for pregnancy in women with recurrent miscarriage. A meta-analysis of controlled trials. Br J Obstet Gynaecol. 1989 Mar;96(3):275–280. doi: 10.1111/j.1471-0528.1989.tb02386.x. [DOI] [PubMed] [Google Scholar]
  5. Dickersin K., Berlin J. A. Meta-analysis: state-of-the-science. Epidemiol Rev. 1992;14:154–176. doi: 10.1093/oxfordjournals.epirev.a036084. [DOI] [PubMed] [Google Scholar]
  6. Dickersin K., Chan S., Chalmers T. C., Sacks H. S., Smith H., Jr Publication bias and clinical trials. Control Clin Trials. 1987 Dec;8(4):343–353. doi: 10.1016/0197-2456(87)90155-3. [DOI] [PubMed] [Google Scholar]
  7. Dickersin K., Hewitt P., Mutch L., Chalmers I., Chalmers T. C. Perusing the literature: comparison of MEDLINE searching with a perinatal trials database. Control Clin Trials. 1985 Dec;6(4):306–317. doi: 10.1016/0197-2456(85)90106-0. [DOI] [PubMed] [Google Scholar]
  8. Dickersin K., Min Y. I., Meinert C. L. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992 Jan 15;267(3):374–378. [PubMed] [Google Scholar]
  9. Dickersin K., Min Y. I. Publication bias: the problem that won't go away. Ann N Y Acad Sci. 1993 Dec 31;703:135–148. doi: 10.1111/j.1749-6632.1993.tb26343.x. [DOI] [PubMed] [Google Scholar]
  10. Easterbrook P. J., Berlin J. A., Gopalan R., Matthews D. R. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867–872. doi: 10.1016/0140-6736(91)90201-y. [DOI] [PubMed] [Google Scholar]
  11. Easterbrook P. J. Directory of registries of clinical trials. Stat Med. 1992 Feb 15;11(3):363–423. [PubMed] [Google Scholar]
  12. Freedman B. Suspended judgment. AIDS and the ethics of clinical trials: learning the right lessons. Control Clin Trials. 1992 Feb;13(1):1–5. doi: 10.1016/0197-2456(92)90025-u. [DOI] [PubMed] [Google Scholar]
  13. Gøtzsche P. C., Lange B. Comparison of search strategies for recalling double-blind trials from MEDLINE. Dan Med Bull. 1991 Dec;38(6):476–478. [PubMed] [Google Scholar]
  14. Jadad A. R., McQuay H. J. A high-yield strategy to identify randomized controlled trials for systematic reviews. Online J Curr Clin Trials. 1993 Feb 27;DOC:[3973 words–paragraphs]. [PubMed] [Google Scholar]
  15. Kirpalani H., Schmidt B., McKibbon K. A., Haynes R. B., Sinclair J. C. Searching MEDLINE for randomized clinical trials involving care of the newborn. Pediatrics. 1989 Apr;83(4):543–546. [PubMed] [Google Scholar]
  16. Kleijnen J., Knipschild P. The comprehensiveness of Medline and Embase computer searches. Searches for controlled trials of homoeopathy, ascorbic acid for common cold and ginkgo biloba for cerebral insufficiency and intermittent claudication. Pharm Weekbl Sci. 1992 Oct 16;14(5):316–320. doi: 10.1007/BF01977620. [DOI] [PubMed] [Google Scholar]
  17. Ohlsson A. Treatments of preterm premature rupture of the membranes: a meta-analysis. Am J Obstet Gynecol. 1989 Apr;160(4):890–906. doi: 10.1016/0002-9378(89)90306-2. [DOI] [PubMed] [Google Scholar]
  18. Poynard T., Conn H. O. The retrieval of randomized clinical trials in liver disease from the medical literature. A comparison of MEDLARS and manual methods. Control Clin Trials. 1985 Dec;6(4):271–279. doi: 10.1016/0197-2456(85)90103-5. [DOI] [PubMed] [Google Scholar]
  19. Scherer R. W., Dickersin K., Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994 Jul 13;272(2):158–162. [PubMed] [Google Scholar]
  20. Silagy C. Developing a register of randomised controlled trials in primary care. BMJ. 1993 Apr 3;306(6882):897–900. doi: 10.1136/bmj.306.6882.897. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Solomon M. J., Laxamana A., Devore L., McLeod R. S. Randomized controlled trials in surgery. Surgery. 1994 Jun;115(6):707–712. [PubMed] [Google Scholar]
  22. Stewart L. A., Parmar M. K. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993 Feb 13;341(8842):418–422. doi: 10.1016/0140-6736(93)93004-k. [DOI] [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES