Skip to main content
PMC home page
Quality & Safety in Health Care logoLink to Quality & Safety in Health Care
. 2003 Feb;12(1):47–52. doi: 10.1136/qhc.12.1.47

Research designs for studies evaluating the effectiveness of change and improvement strategies

M Eccles 1, J Grimshaw 1, M Campbell 1, C Ramsay 1
PMCID: PMC1743658  PMID: 12571345

Abstract



 The methods of evaluating change and improvement strategies are not well described. The design and conduct of a range of experimental and non-experimental quantitative designs are considered. Such study designs should usually be used in a context where they build on appropriate theoretical, qualitative and modelling work, particularly in the development of appropriate interventions. A range of experimental designs are discussed including single and multiple arm randomised controlled trials and the use of more complex factorial and block designs. The impact of randomisation at both group and individual levels and three non-experimental designs (uncontrolled before and after, controlled before and after, and time series analysis) are also considered. The design chosen will reflect both the needs (and resources) in any particular circumstances and also the purpose of the evaluation. The general principle underlying the choice of evaluative design is, however, simple—those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability.

Full Text

The Full Text of this article is available as a PDF (101.1 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Campbell M. K., Mollison J., Steen N., Grimshaw J. M., Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000 Apr;17(2):192–196. doi: 10.1093/fampra/17.2.192. [DOI] [PubMed] [Google Scholar]
  2. Campbell M., Fitzpatrick R., Haines A., Kinmonth A. L., Sandercock P., Spiegelhalter D., Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000 Sep 16;321(7262):694–696. doi: 10.1136/bmj.321.7262.694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Diwan V. K., Eriksson B., Sterky G., Tomson G. Randomization by group in studying the effect of drug information in primary care. Int J Epidemiol. 1992 Feb;21(1):124–130. doi: 10.1093/ije/21.1.124. [DOI] [PubMed] [Google Scholar]
  4. Donner A., Koval J. J. The estimation of intraclass correlation in the analysis of family data. Biometrics. 1980 Mar;36(1):19–25. [PubMed] [Google Scholar]
  5. Eccles M., Steen N., Grimshaw J., Thomas L., McNamee P., Soutter J., Wilsdon J., Matowe L., Needham G., Gilbert F. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001 May 5;357(9266):1406–1409. doi: 10.1016/S0140-6736(00)04564-5. [DOI] [PubMed] [Google Scholar]
  6. Eccles Martin, McColl Elaine, Steen Nick, Rousseau Nikki, Grimshaw Jeremy, Parkin David, Purves Ian. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002 Oct 26;325(7370):941–941. doi: 10.1136/bmj.325.7370.941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Flynn Terry N., Whitley Elise, Peters Tim J. Recruitment strategies in a cluster randomized trial--cost implications. Stat Med. 2002 Feb 15;21(3):397–405. doi: 10.1002/sim.1025. [DOI] [PubMed] [Google Scholar]
  8. Lipsey M. W., Wilson D. B. The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. Am Psychol. 1993 Dec;48(12):1181–1209. doi: 10.1037//0003-066x.48.12.1181. [DOI] [PubMed] [Google Scholar]
  9. Matowe L., Ramsay C. R., Grimshaw J. M., Gilbert F. J., Macleod M-J, Needham G. Effects of mailed dissemination of the Royal College of Radiologists' guidelines on general practitioner referrals for radiography: a time series analysis. Clin Radiol. 2002 Jul;57(7):575–578. doi: 10.1053/crad.2001.0894. [DOI] [PubMed] [Google Scholar]
  10. Mollison J., Simpson J. A., Campbell M. K., Grimshaw J. M. Comparison of analytical methods for cluster randomised trials: an example from a primary care setting. J Epidemiol Biostat. 2000;5(6):339–348. [PubMed] [Google Scholar]
  11. Morgan M., Studney D. R., Barnett G. O., Winickoff R. N. Computerized concurrent review of prenatal care. QRB Qual Rev Bull. 1978 Sep;4(9):33–36. [PubMed] [Google Scholar]
  12. Schwartz D., Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967 Aug;20(8):637–648. doi: 10.1016/0021-9681(67)90041-0. [DOI] [PubMed] [Google Scholar]
  13. Turner M. J., Flannelly G. M., Wingfield M., Rasmussen M. J., Ryan R., Cullen S., Maguire R., Stronge J. M. The miscarriage clinic: an audit of the first year. Br J Obstet Gynaecol. 1991 Mar;98(3):306–308. doi: 10.1111/j.1471-0528.1991.tb13398.x. [DOI] [PubMed] [Google Scholar]

Articles from Quality & safety in health care are provided here courtesy of BMJ Publishing Group

RESOURCES