Skip to main content
NCBI home page
Journal of Epidemiology and Community Health logoLink to Journal of Epidemiology and Community Health
. 1998 Feb;52(2):75–81. doi: 10.1136/jech.52.2.75

Evidence into practice, experimentation and quasi experimentation: are the methods up to the task?

N Freemantle, J Wood, F Crawford
PMCID: PMC1756676  PMID: 9578853

Abstract

OBJECTIVE: Methodological review of evaluations of interventions intended to help health professionals provide more effective and efficient health care, motivated by the current experience of NHS Research and Development in England. Emphasis upon the forms of research appropriate to different stages in the development and evaluation of interventions, the use of experimental and quasi experimental designs, the methods used in systematic reviews and meta analyses. METHOD: A proposed development process is derived from that used in the development of drugs. The strengths and weaknesses of different experimental and quasi experimental designs are derived from published methodological literature and first principles. Examples are drawn from the literature. RESULTS: Like pharmaceuticals, implementation interventions need to go through several stages of development before they are evaluated in designed experiments. Where there are practical reasons that make random allocation impossible in quantitative evaluations, quasi experimental methods may provide useful information, although these studies are open to bias. It is rare for a single study to provide a complete answer to important questions, and systematic reviews of all available studies should be undertaken. Meta analytic techniques go some way towards countering the low power of many existing studies, reduce the risk of bias, and avoid the subjective approaches that may be found in narrative reviews. CONCLUSIONS: The initiative taken by NHS Research and Development in examining methods to promote the uptake of research findings is welcome, but will only prove helpful if careful attention is paid to the different stages of the development process, and different research approaches are used appropriately at different stages.

 

Full Text

The Full Text of this article is available as a PDF (124.0 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Angell M. Patients' preferences in randomized clinical trials. N Engl J Med. 1984 May 24;310(21):1385–1387. doi: 10.1056/NEJM198405243102111. [DOI] [PubMed] [Google Scholar]
  2. Antman E. M., Lau J., Kupelnick B., Mosteller F., Chalmers T. C. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992 Jul 8;268(2):240–248. [PubMed] [Google Scholar]
  3. Cappelleri J. C., Ioannidis J. P., Schmid C. H., de Ferranti S. D., Aubert M., Chalmers T. C., Lau J. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA. 1996 Oct 23;276(16):1332–1338. [PubMed] [Google Scholar]
  4. Cornfield J. Randomization by group: a formal analysis. Am J Epidemiol. 1978 Aug;108(2):100–102. doi: 10.1093/oxfordjournals.aje.a112592. [DOI] [PubMed] [Google Scholar]
  5. Covell D. G., Uman G. C., Manning P. R. Information needs in office practice: are they being met? Ann Intern Med. 1985 Oct;103(4):596–599. doi: 10.7326/0003-4819-103-4-596. [DOI] [PubMed] [Google Scholar]
  6. Davis D. A., Thomson M. A., Oxman A. D., Haynes R. B. Evidence for the effectiveness of CME. A review of 50 randomized controlled trials. JAMA. 1992 Sep 2;268(9):1111–1117. [PubMed] [Google Scholar]
  7. DerSimonian R., Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986 Sep;7(3):177–188. doi: 10.1016/0197-2456(86)90046-2. [DOI] [PubMed] [Google Scholar]
  8. Evans C. E., Haynes R. B., Birkett N. J., Gilbert J. R., Taylor D. W., Sackett D. L., Johnston M. E., Hewson S. A. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986 Jan 24;255(4):501–504. [PubMed] [Google Scholar]
  9. Evans C. E., Haynes R. B., Gilbert J. R., Taylor D. W., Sackett D. L., Johnston M. Educational package on hypertension for primary care physicians. Can Med Assoc J. 1984 Mar 15;130(6):719–722. [PMC free article] [PubMed] [Google Scholar]
  10. Garg R., Yusuf S. Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure. Collaborative Group on ACE Inhibitor Trials. JAMA. 1995 May 10;273(18):1450–1456. [PubMed] [Google Scholar]
  11. Juul-Möller S., Edvardsson N., Jahnmatz B., Rosén A., Sørensen S., Omblus R. Double-blind trial of aspirin in primary prevention of myocardial infarction in patients with stable chronic angina pectoris. The Swedish Angina Pectoris Aspirin Trial (SAPAT) Group. Lancet. 1992 Dec 12;340(8833):1421–1425. doi: 10.1016/0140-6736(92)92619-q. [DOI] [PubMed] [Google Scholar]
  12. Lomas J., Enkin M., Anderson G. M., Hannah W. J., Vayda E., Singer J. Opinion leaders vs audit and feedback to implement practice guidelines. Delivery after previous cesarean section. JAMA. 1991 May 1;265(17):2202–2207. [PubMed] [Google Scholar]
  13. Mosteller F., Colditz G. A. Understanding research synthesis (meta-analysis). Annu Rev Public Health. 1996;17:1–23. doi: 10.1146/annurev.pu.17.050196.000245. [DOI] [PubMed] [Google Scholar]
  14. Norton P. G., Dempsey L. J. Self-audit: its effect on quality of care. J Fam Pract. 1985 Oct;21(4):289–291. [PubMed] [Google Scholar]
  15. Peckham M. Research and development for the National Health Service. Lancet. 1991 Aug 10;338(8763):367–371. doi: 10.1016/0140-6736(91)90494-a. [DOI] [PubMed] [Google Scholar]
  16. Schwartz D., Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967 Aug;20(8):637–648. doi: 10.1016/0021-9681(67)90041-0. [DOI] [PubMed] [Google Scholar]
  17. Soumerai S. B., Avorn J. Principles of educational outreach ('academic detailing') to improve clinical decision making. JAMA. 1990 Jan 26;263(4):549–556. [PubMed] [Google Scholar]
  18. Stross J. K., Bole G. G. Evaluation of an educational program for primary care practitioners, on the management of osteoarthritis. Arthritis Rheum. 1985 Jan;28(1):108–111. doi: 10.1002/art.1780280119. [DOI] [PubMed] [Google Scholar]
  19. Van de Werf F., Topol E. J., Lee K. L., Woodlief L. H., Granger C. B., Armstrong P. W., Barbash G. I., Hampton J. R., Guerci A., Simes R. J. Variations in patient management and outcomes for acute myocardial infarction in the United States and other countries. Results from the GUSTO trial. Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries. JAMA. 1995 May 24;273(20):1586–1591. doi: 10.1001/jama.273.20.1586. [DOI] [PubMed] [Google Scholar]
  20. Wagner E. H., Barrett P., Barry M. J., Barlow W., Fowler F. J., Jr The effect of a shared decisionmaking program on rates of surgery for benign prostatic hyperplasia. Pilot results. Med Care. 1995 Aug;33(8):765–770. doi: 10.1097/00005650-199508000-00002. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Epidemiology and Community Health are provided here courtesy of BMJ Publishing Group

RESOURCES