Skip to main content
PMC home page
CMAJ : Canadian Medical Association Journal logoLink to CMAJ : Canadian Medical Association Journal
. 1995 Nov 15;153(10):1423–1431.

No magic bullets: a systematic review of 102 trials of interventions to improve professional practice.

A D Oxman 1, M A Thomson 1, D A Davis 1, R B Haynes 1
PMCID: PMC1487455  PMID: 7585368

Abstract

OBJECTIVE: To determine the effectiveness of different types of interventions in improving health professional performance and health outcomes. DATA SOURCES: MEDLINE, SCISEARCH, CINAHL and the Research and Development Resource Base in CME were searched for trials of educational interventions in the health care professions published between 1970 and 1993 inclusive. STUDY SELECTION: Studies were selected if they provided objective measurements of health professional performance or health outcomes and employed random or quasi-random allocation methods in their study designs to assign individual subjects or groups. Interventions included such activities as conferences, outreach visits, the use of local opinion leaders, audit and feedback, and reminder systems. DATA EXTRACTION: Details extracted from the studies included the study design; the unit of allocation (e.g., patient, provider, practice, hospital); the characteristics of the targeted health care professionals, educational interventions and patients (when appropriate); and the main outcome measure. DATA SYNTHESIS: The inclusion criteria were met by 102 trials. Areas of behaviour change included general patient management, preventive services, prescribing practices, treatment of specific conditions such as hypertension or diabetes, and diagnostic service or hospital utilization. Dissemination-only strategies, such as conferences or the mailing of unsolicited materials, demonstrated little or no changes in health professional behaviour or health outcome when used alone. More complex interventions, such as the use of outreach visits or local opinion leaders, ranged from ineffective to highly effective but were most often moderately effective (resulting in reductions of 20% to 50% in the incidence of inappropriate performance). CONCLUSION: There are no "magic bullets" for improving the quality of health care, but there are a wide range of interventions available that, if used appropriately, could lead to important improvements in professional practice and patient outcomes.

Full text

PDF
1423

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Adams I. D., Chan M., Clifford P. C., Cooke W. M., Dallos V., de Dombal F. T., Edwards M. H., Hancock D. M., Hewett D. J., McIntyre N. Computer aided diagnosis of acute abdominal pain: a multicentre study. Br Med J (Clin Res Ed) 1986 Sep 27;293(6550):800–804. doi: 10.1136/bmj.293.6550.800. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Antman E. M., Lau J., Kupelnick B., Mosteller F., Chalmers T. C. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992 Jul 8;268(2):240–248. [PubMed] [Google Scholar]
  3. Avorn J., Soumerai S. B., Everitt D. E., Ross-Degnan D., Beers M. H., Sherman D., Salem-Schatz S. R., Fields D. A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. N Engl J Med. 1992 Jul 16;327(3):168–173. doi: 10.1056/NEJM199207163270306. [DOI] [PubMed] [Google Scholar]
  4. Avorn J., Soumerai S. B. Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based "detailing". N Engl J Med. 1983 Jun 16;308(24):1457–1463. doi: 10.1056/NEJM198306163082406. [DOI] [PubMed] [Google Scholar]
  5. Barnett G. O., Winickoff R. N., Morgan M. M., Zielstorff R. D. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983 Apr;21(4):400–409. doi: 10.1097/00005650-198304000-00003. [DOI] [PubMed] [Google Scholar]
  6. Bass M. J., Elford R. W. Preventive practice patterns of Canadian primary care physicians. Am J Prev Med. 1988;4(4 Suppl):17–26. [PubMed] [Google Scholar]
  7. Bass M. J., McWhinney I. R., Donner A. Do family physicians need medical assistants to detect and manage hypertension? CMAJ. 1986 Jun 1;134(11):1247–1255. [PMC free article] [PubMed] [Google Scholar]
  8. Berwick D. M., Coltin K. L. Feedback reduces test use in a health maintenance organization. JAMA. 1986 Mar 21;255(11):1450–1454. [PubMed] [Google Scholar]
  9. Billi J. E., Hejna G. F., Wolf F. M., Shapiro L. R., Stross J. K. The effects of a cost-education program on hospital charges. J Gen Intern Med. 1987 Sep-Oct;2(5):306–311. doi: 10.1007/BF02596163. [DOI] [PubMed] [Google Scholar]
  10. Britton M. L., Lurvey P. L. Impact of medication profile review on prescribing in a general medicine clinic. Am J Hosp Pharm. 1991 Feb;48(2):265–270. [PubMed] [Google Scholar]
  11. Bulpitt C. J., Beilin L. J., Coles E. C., Dollery C. T., Johnson B. F., Munro-Faure A. D., Turner S. C. Randomised controlled trial of computer-held medical records in hypertensive patients. Br Med J. 1976 Mar 20;1(6011):677–679. doi: 10.1136/bmj.1.6011.677. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Buntinx F., Knottnerus J. A., Crebolder H. F., Essed G. G. Reactions of doctors to various forms of feedback designed to improve the sampling quality of cervical smears. Qual Assur Health Care. 1992 Jun;4(2):161–166. [PubMed] [Google Scholar]
  13. Buntinx F., Knottnerus J. A., Crebolder H. F., Seegers T., Essed G. G., Schouten H. Does feedback improve the quality of cervical smears? A randomized controlled trial. Br J Gen Pract. 1993 May;43(370):194–198. [PMC free article] [PubMed] [Google Scholar]
  14. Chalmers T. C. The impact of controlled trials on the practice of medicine. Mt Sinai J Med. 1974 Nov-Dec;41(6):753–759. [PubMed] [Google Scholar]
  15. Chambers C. V., Balaban D. J., Carlson B. L., Ungemack J. A., Grasberger D. M. Microcomputer-generated reminders. Improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989 Sep;29(3):273–280. [PubMed] [Google Scholar]
  16. Chase C. R., Vacek P. M., Shinozaki T., Giard A. M., Ashikaga T. Medical information management: improving the transfer of research results to presurgical evaluation. Med Care. 1983 Apr;21(4):410–424. doi: 10.1097/00005650-198304000-00004. [DOI] [PubMed] [Google Scholar]
  17. Chassin M. R., McCue S. M. A randomized trial of medical quality assurance. Improving physicians' use of pelvimetry. JAMA. 1986 Aug 22;256(8):1012–1016. [PubMed] [Google Scholar]
  18. Cheney C., Ramsdell J. W. Effect of medical records' checklists on implementation of periodic health measures. Am J Med. 1987 Jul;83(1):129–136. doi: 10.1016/0002-9343(87)90507-9. [DOI] [PubMed] [Google Scholar]
  19. Cohen D. I., Jones P., Littenberg B., Neuhauser D. Does cost information availability reduce physician test usage? A randomized clinical trial with unexpected findings. Med Care. 1982 Mar;20(3):286–292. doi: 10.1097/00005650-198203000-00005. [DOI] [PubMed] [Google Scholar]
  20. Cohen S. J., Christen A. G., Katz B. P., Drook C. A., Davis B. J., Smith D. M., Stookey G. K. Counseling medical and dental patients about cigarette smoking: the impact of nicotine gum and chart reminders. Am J Public Health. 1987 Mar;77(3):313–316. doi: 10.2105/ajph.77.3.313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Cohen S. J., Stookey G. K., Katz B. P., Drook C. A., Smith D. M. Encouraging primary care physicians to help smokers quit. A randomized, controlled trial. Ann Intern Med. 1989 Apr 15;110(8):648–652. doi: 10.7326/0003-4819-110-8-648. [DOI] [PubMed] [Google Scholar]
  22. Cohen S. J., Weinberger M., Hui S. L., Tierney W. M., McDonald C. J. The impact of reading on physicians' nonadherence to recommended standards of medical care. Soc Sci Med. 1985;21(8):909–914. doi: 10.1016/0277-9536(85)90147-9. [DOI] [PubMed] [Google Scholar]
  23. Cowan J. A., Heckerling P. S., Parker J. B. Effect of a fact sheet reminder on performance of the periodic health examination: a randomized controlled trial. Am J Prev Med. 1992 Mar-Apr;8(2):104–109. [PubMed] [Google Scholar]
  24. Cummings S. R., Coates T. J., Richard R. J., Hansen B., Zahnd E. G., VanderMartin R., Duncan C., Gerbert B., Martin A., Stein M. J. Training physicians in counseling about smoking cessation. A randomized trial of the "Quit for Life" program. Ann Intern Med. 1989 Apr 15;110(8):640–647. doi: 10.7326/0003-4819-110-8-640. [DOI] [PubMed] [Google Scholar]
  25. Cummings S. R., Richard R. J., Duncan C. L., Hansen B., Vander Martin R., Gerbert B., Coates T. J. Training physicians about smoking cessation: a controlled trial in private practice. J Gen Intern Med. 1989 Nov-Dec;4(6):482–489. doi: 10.1007/BF02599545. [DOI] [PubMed] [Google Scholar]
  26. Davidoff F., Goodspeed R., Clive J. Changing test ordering behavior. A randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care. 1989 Jan;27(1):45–58. [PubMed] [Google Scholar]
  27. Davis D. A., Thomson M. A., Oxman A. D., Haynes R. B. Evidence for the effectiveness of CME. A review of 50 randomized controlled trials. JAMA. 1992 Sep 2;268(9):1111–1117. [PubMed] [Google Scholar]
  28. Dickinson J. C., Warshaw G. A., Gehlbach S. H., Bobula J. A., Muhlbaier L. H., Parkerson G. R., Jr Improving hypertension control: impact of computer feedback and physician education. Med Care. 1981 Aug;19(8):843–854. doi: 10.1097/00005650-198108000-00004. [DOI] [PubMed] [Google Scholar]
  29. Dietrich A. J., O'Connor G. T., Keller A., Carney P. A., Levy D., Whaley F. S. Cancer: improving early detection and prevention. A community practice randomised trial. BMJ. 1992 Mar 14;304(6828):687–691. doi: 10.1136/bmj.304.6828.687. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Eddy D. M. Clinical policies and the quality of clinical practice. N Engl J Med. 1982 Aug 5;307(6):343–347. doi: 10.1056/NEJM198208053070604. [DOI] [PubMed] [Google Scholar]
  31. Evans C. E., Haynes R. B., Birkett N. J., Gilbert J. R., Taylor D. W., Sackett D. L., Johnston M. E., Hewson S. A. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986 Jan 24;255(4):501–504. [PubMed] [Google Scholar]
  32. Everett G. D., deBlois C. S., Chang P. F., Holets T. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med. 1983 May;143(5):942–944. [PubMed] [Google Scholar]
  33. Freemantle N., Grilli R., Grimshaw J., Oxman A. Implementing findings of medical research: the Cochrane Collaboration on Effective Professional Practice. Qual Health Care. 1995 Mar;4(1):45–47. doi: 10.1136/qshc.4.1.45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Friedman L., Wenger N. K., Knatterud G. L. Impact of the Coronary Drug Project findings on clinical practice. Control Clin Trials. 1983 Dec;4(4):513–522. doi: 10.1016/0197-2456(83)90032-6. [DOI] [PubMed] [Google Scholar]
  35. Garnier H. S., Flamant R., Fohanno C. Assessment of the role of randomized clinical trials in establishing treatment policies. Control Clin Trials. 1982 Sep;3(3):227–234. doi: 10.1016/0197-2456(82)90008-3. [DOI] [PubMed] [Google Scholar]
  36. Gehlbach S. H., Wilkinson W. E., Hammond W. E., Clapp N. E., Finn A. L., Taylor W. J., Rodell M. S. Improving drug prescribing in a primary care practice. Med Care. 1984 Mar;22(3):193–201. doi: 10.1097/00005650-198403000-00002. [DOI] [PubMed] [Google Scholar]
  37. Goodwin J. S., Goodwin J. M. Failure to recognize efficacious treatments: a history of salicylate therapy in rheumatoid arthritis. Perspect Biol Med. 1981 Autumn;25(1):78–92. doi: 10.1353/pbm.1981.0053. [DOI] [PubMed] [Google Scholar]
  38. Headrick L. A., Speroff T., Pelecanos H. I., Cebul R. D. Efforts to improve compliance with the National Cholesterol Education Program guidelines. Results of a randomized controlled trial. Arch Intern Med. 1992 Dec;152(12):2490–2496. [PubMed] [Google Scholar]
  39. Heale J., Davis D., Norman G., Woodward C., Neufeld V., Dodd P. A randomized controlled trial assessing the impact of problem-based versus didactic teaching methods in CME. Res Med Educ. 1988;27:72–77. [PubMed] [Google Scholar]
  40. Hershey C. O., Goldberg H. I., Cohen D. I. The effect of computerized feedback coupled with a newsletter upon outpatient prescribing charges. A randomized controlled trial. Med Care. 1988 Jan;26(1):88–94. doi: 10.1097/00005650-198801000-00010. [DOI] [PubMed] [Google Scholar]
  41. Hershey C. O., Porter D. K., Breslau D., Cohen D. I. Influence of simple computerized feedback on prescription charges in an ambulatory clinic. A randomized clinical trial. Med Care. 1986 Jun;24(6):472–481. doi: 10.1097/00005650-198606000-00002. [DOI] [PubMed] [Google Scholar]
  42. Hong S. W., Ching T. Y., Fung J. P., Seto W. L. The employment of ward opinion leaders for continuing education in the hospital. Med Teach. 1990;12(2):209–217. doi: 10.3109/01421599009006698. [DOI] [PubMed] [Google Scholar]
  43. Jennett P. A., Laxdal O. E., Hayton R. C., Klaassen D. J., Swanson R. W., Wilson T. W., Spooner H. J., Mainprize G. W., Wickett R. E. The effects of continuing medical education on family doctor performance in office practice: a randomized control study. Med Educ. 1988 Mar;22(2):139–145. doi: 10.1111/j.1365-2923.1988.tb00424.x. [DOI] [PubMed] [Google Scholar]
  44. Johnson R. E., Azevedo D. J., Campbell W. H., Christensen D. B. Examining physicians' drug order recording behavior. Med Care. 1978 May;16(5):408–416. doi: 10.1097/00005650-197805000-00005. [DOI] [PubMed] [Google Scholar]
  45. Johnson R. E., Campbell W. H., Azevedo D. J., Christensen D. B. Studying the impact of patient drug profiles in an HMO. Med Care. 1976 Oct;14(10):799–807. doi: 10.1097/00005650-197610000-00001. [DOI] [PubMed] [Google Scholar]
  46. Ketley D., Woods K. L. Impact of clinical trials on clinical practice: example of thrombolysis for acute myocardial infarction. Lancet. 1993 Oct 9;342(8876):891–894. doi: 10.1016/0140-6736(93)91945-i. [DOI] [PubMed] [Google Scholar]
  47. Kottke T. E., Brekke M. L., Solberg L. I., Hughes J. R. A randomized trial to increase smoking intervention by physicians. Doctors Helping Smokers, Round I. JAMA. 1989 Apr 14;261(14):2101–2106. [PubMed] [Google Scholar]
  48. Lau J., Antman E. M., Jimenez-Silva J., Kupelnick B., Mosteller F., Chalmers T. C. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992 Jul 23;327(4):248–254. doi: 10.1056/NEJM199207233270406. [DOI] [PubMed] [Google Scholar]
  49. Levinson W., Roter D. The effects of two continuing medical education programs on communication skills of practicing primary care physicians. J Gen Intern Med. 1993 Jun;8(6):318–324. doi: 10.1007/BF02600146. [DOI] [PubMed] [Google Scholar]
  50. Lewis C. E. Disease prevention and health promotion practices of primary care physicians in the United States. Am J Prev Med. 1988;4(4 Suppl):9–16. [PubMed] [Google Scholar]
  51. Linn B. S. Continuing medical education. Impact on emergency room burn care. JAMA. 1980 Aug 8;244(6):565–570. doi: 10.1001/jama.244.6.565. [DOI] [PubMed] [Google Scholar]
  52. Litzelman D. K., Dittus R. S., Miller M. E., Tierney W. M. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993 Jun;8(6):311–317. doi: 10.1007/BF02600144. [DOI] [PubMed] [Google Scholar]
  53. Lomas J., Enkin M., Anderson G. M., Hannah W. J., Vayda E., Singer J. Opinion leaders vs audit and feedback to implement practice guidelines. Delivery after previous cesarean section. JAMA. 1991 May 1;265(17):2202–2207. [PubMed] [Google Scholar]
  54. Magruder-Habib K., Zung W. W., Feussner J. R. Improving physicians' recognition and treatment of depression in general medical care. Results from a randomized clinical trial. Med Care. 1990 Mar;28(3):239–250. doi: 10.1097/00005650-199003000-00004. [DOI] [PubMed] [Google Scholar]
  55. Maiman L. A., Becker M. H., Liptak G. S., Nazarian L. F., Rounds K. A. Improving pediatricians' compliance-enhancing practices. A randomized trial. Am J Dis Child. 1988 Jul;142(7):773–779. doi: 10.1001/archpedi.1988.02150070087033. [DOI] [PubMed] [Google Scholar]
  56. Manheim L. M., Feinglass J., Hughes R., Martin G. J., Conrad K., Hughes E. F. Training house officers to be cost conscious. Effects of an educational intervention on charges and length of stay. Med Care. 1990 Jan;28(1):29–42. doi: 10.1097/00005650-199001000-00005. [DOI] [PubMed] [Google Scholar]
  57. Martin A. R., Wolf M. A., Thibodeau L. A., Dzau V., Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med. 1980 Dec 4;303(23):1330–1336. doi: 10.1056/NEJM198012043032304. [DOI] [PubMed] [Google Scholar]
  58. Marton K. I., Tul V., Sox H. C., Jr Modifying test-ordering behavior in the outpatient medical clinic. A controlled trial of two educational interventions. Arch Intern Med. 1985 May;145(5):816–821. [PubMed] [Google Scholar]
  59. Mayefsky J. H., Foye H. R. Use of a chart audit: teaching well child care to paediatric house officers. Med Educ. 1993 Mar;27(2):170–174. doi: 10.1111/j.1365-2923.1993.tb00248.x. [DOI] [PubMed] [Google Scholar]
  60. Mc Donald C. J. Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976 Feb;84(2):162–167. doi: 10.7326/0003-4819-84-2-162. [DOI] [PubMed] [Google Scholar]
  61. McAlister N. H., Covvey H. D., Tong C., Lee A., Wigle E. D. Randomised controlled trial of computer assisted management of hypertension in primary care. Br Med J (Clin Res Ed) 1986 Sep 13;293(6548):670–674. doi: 10.1136/bmj.293.6548.670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. McConnell T. S., Cushing A. H., Bankhurst A. D., Healy J. L., McIlvenna P. A., Skipper B. J. Physician behavior modification using claims data: tetracycline for upper respiratory infection. West J Med. 1982 Nov;137(5):448–450. [PMC free article] [PubMed] [Google Scholar]
  63. McDonald C. J., Hui S. L., Smith D. M., Tierney W. M., Cohen S. J., Weinberger M., McCabe G. P. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984 Jan;100(1):130–138. doi: 10.7326/0003-4819-100-1-130. [DOI] [PubMed] [Google Scholar]
  64. McDonald C. J., Hui S. L., Tierney W. M. Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992 Sep-Oct;9(5):304–312. [PubMed] [Google Scholar]
  65. McDonald C. J. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976 Dec 9;295(24):1351–1355. doi: 10.1056/NEJM197612092952405. [DOI] [PubMed] [Google Scholar]
  66. McDonald C. J., Wilson G. A., McCabe G. P., Jr Physician response to computer reminders. JAMA. 1980 Oct 3;244(14):1579–1581. [PubMed] [Google Scholar]
  67. McPhee S. J., Bird J. A., Fordham D., Rodnick J. E., Osborn E. H. Promoting cancer prevention activities by primary care physicians. Results of a randomized, controlled trial. JAMA. 1991 Jul 24;266(4):538–544. [PubMed] [Google Scholar]
  68. McPhee S. J., Bird J. A., Jenkins C. N., Fordham D. Promoting cancer screening. A randomized, controlled trial of three interventions. Arch Intern Med. 1989 Aug;149(8):1866–1872. doi: 10.1001/archinte.149.8.1866. [DOI] [PubMed] [Google Scholar]
  69. Meyer T. J., Van Kooten D., Marsh S., Prochazka A. V. Reduction of polypharmacy by feedback to clinicians. J Gen Intern Med. 1991 Mar-Apr;6(2):133–136. doi: 10.1007/BF02598309. [DOI] [PubMed] [Google Scholar]
  70. Morgan M., Studney D. R., Barnett G. O., Winickoff R. N. Computerized concurrent review of prenatal care. QRB Qual Rev Bull. 1978 Sep;4(9):33–36. [PubMed] [Google Scholar]
  71. Mugford M., Banfield P., O'Hanlon M. Effects of feedback of information on clinical practice: a review. BMJ. 1991 Aug 17;303(6799):398–402. doi: 10.1136/bmj.303.6799.398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Murrey K. O., Gottlieb L. K., Schoenbaum S. C. Implementing clinical guidelines: a quality management approach to reminder systems. QRB Qual Rev Bull. 1992 Dec;18(12):423–433. doi: 10.1016/s0097-5990(16)30568-1. [DOI] [PubMed] [Google Scholar]
  73. Nattinger A. B., Panzer R. J., Janus J. Improving the utilization of screening mammography in primary care practices. Arch Intern Med. 1989 Sep;149(9):2087–2092. [PubMed] [Google Scholar]
  74. Norton P. G., Dempsey L. J. Self-audit: its effect on quality of care. J Fam Pract. 1985 Oct;21(4):289–291. [PubMed] [Google Scholar]
  75. Palmer R. H., Louis T. A., Hsu L. N., Peterson H. F., Rothrock J. K., Strain R., Thompson M. S., Wright E. A. A randomized controlled trial of quality assurance in sixteen ambulatory care practices. Med Care. 1985 Jun;23(6):751–770. doi: 10.1097/00005650-198506000-00001. [DOI] [PubMed] [Google Scholar]
  76. Perera D. R., LoGerfo J. P., Shulenberger E., Ylvisaker J. T., Kirz H. L. Teaching sigmoidoscopy to primary care physicians: a controlled study of continuing medical education. J Fam Pract. 1983 Apr;16(4):785–788. [PubMed] [Google Scholar]
  77. Pollack M. M., Getson P. R. Pediatric critical care cost containment: combined actuarial and clinical program. Crit Care Med. 1991 Jan;19(1):12–20. doi: 10.1097/00003246-199101000-00008. [DOI] [PubMed] [Google Scholar]
  78. Pozen M. W., D'Agostino R. B., Mitchell J. B., Rosenfeld D. M., Guglielmino J. T., Schwartz M. L., Teebagy N., Valentine J. M., Hood W. B., Jr The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Ann Intern Med. 1980 Feb;92(2 Pt 1):238–242. doi: 10.7326/0003-4819-92-2-238. [DOI] [PubMed] [Google Scholar]
  79. Pozen M. W., D'Agostino R. B., Selker H. P., Sytkowski P. A., Hood W. B., Jr A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease. A prospective multicenter clinical trial. N Engl J Med. 1984 May 17;310(20):1273–1278. doi: 10.1056/NEJM198405173102001. [DOI] [PubMed] [Google Scholar]
  80. Putnam R. W., Curry L. Impact of patient care appraisal on physician behaviour in the office setting. Can Med Assoc J. 1985 May 1;132(9):1025–1029. [PMC free article] [PubMed] [Google Scholar]
  81. Putnam R. W., Curry L. Physicians' participation in establishing criteria for hypertension management in the office: will patient outcomes be improved? CMAJ. 1989 Apr 1;140(7):806–809. [PMC free article] [PubMed] [Google Scholar]
  82. Raisch D. W., Bootman J. L., Larson L. N., McGhan W. F. Improving antiulcer agent prescribing in a health maintenance organization. Am J Hosp Pharm. 1990 Aug;47(8):1766–1773. [PubMed] [Google Scholar]
  83. Restuccia J. D. The effect of concurrent feedback in reducing inappropriate hospital utilization. Med Care. 1982 Jan;20(1):46–62. doi: 10.1097/00005650-198201000-00004. [DOI] [PubMed] [Google Scholar]
  84. Rockette H. E., Redmond C. K., Fisher B. Impact of randomized clinical trials on therapy of primary breast cancer: the NSABP overview. Control Clin Trials. 1982 Sep;3(3):209–225. doi: 10.1016/0197-2456(82)90007-1. [DOI] [PubMed] [Google Scholar]
  85. Rodman J. H., Jelliffe R. W., Kolb E., Tuey D. B., de Guzman M. F., Wagers P. W., Haywood L. J. Clinical studies with computer-assisted initial lidocaine therapy. Arch Intern Med. 1984 Apr;144(4):703–709. [PubMed] [Google Scholar]
  86. Rogers J. L., Haring O. M. The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979 Jun;17(6):618–630. doi: 10.1097/00005650-197906000-00006. [DOI] [PubMed] [Google Scholar]
  87. Rogers J. L., Haring O. M., Wortman P. M., Watson R. A., Goetz J. P. Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982 Jan;20(1):63–74. doi: 10.1097/00005650-198201000-00005. [DOI] [PubMed] [Google Scholar]
  88. Rosser W. W., McDowell I., Newell C. Use of reminders for preventive procedures in family medicine. CMAJ. 1991 Oct 1;145(7):807–814. [PMC free article] [PubMed] [Google Scholar]
  89. Rubenstein L. V., Calkins D. R., Young R. T., Cleary P. D., Fink A., Kosecoff J., Jette A. M., Davies A. R., Delbanco T. L., Brook R. H. Improving patient function: a randomized trial of functional disability screening. Ann Intern Med. 1989 Nov 15;111(10):836–842. doi: 10.7326/0003-4819-111-10-836. [DOI] [PubMed] [Google Scholar]
  90. Sanazaro P. J., Worth R. M. Concurrent quality assurance in hospital care. Report of a study by Private Initiative in PSRO. N Engl J Med. 1978 May 25;298(21):1171–1177. doi: 10.1056/NEJM197805252982104. [DOI] [PubMed] [Google Scholar]
  91. Schade R. R., Donaldson R. M., Jr How physicians use cimetidine: a survey of hospitalized patients and published cases. N Engl J Med. 1981 May 21;304(21):1281–1284. doi: 10.1056/NEJM198105213042109. [DOI] [PubMed] [Google Scholar]
  92. Sibley J. C., Sackett D. L., Neufeld V., Gerrard B., Rudnick K. V., Fraser W. A randomized trial of continuing medical education. N Engl J Med. 1982 Mar 4;306(9):511–515. doi: 10.1056/NEJM198203043060904. [DOI] [PubMed] [Google Scholar]
  93. Steele M. A., Bess D. T., Franse V. L., Graber S. E. Cost effectiveness of two interventions for reducing outpatient prescribing costs. DICP. 1989 Jun;23(6):497–500. doi: 10.1177/106002808902300612. [DOI] [PubMed] [Google Scholar]
  94. Stross J. K., Banwell B. F., Wolf F. M., Becker M. C. Evaluation of an education program on the management of rheumatic diseases for physical therapists. J Rheumatol. 1986 Apr;13(2):374–378. [PubMed] [Google Scholar]
  95. Stross J. K., Bole G. G. Evaluation of a continuing education program in rheumatoid arthritis. Arthritis Rheum. 1980 Jul;23(7):846–849. doi: 10.1002/art.1780230711. [DOI] [PubMed] [Google Scholar]
  96. Stross J. K., Bole G. G. Evaluation of an educational program for primary care practitioners, on the management of osteoarthritis. Arthritis Rheum. 1985 Jan;28(1):108–111. doi: 10.1002/art.1780280119. [DOI] [PubMed] [Google Scholar]
  97. Stross J. K., Harlan W. R. The dissemination of new medical information. JAMA. 1979 Jun 15;241(24):2622–2624. [PubMed] [Google Scholar]
  98. Stross J. K., Hiss R. G., Watts C. M., Davis W. K., Macdonald R. Continuing education in pulmonary disease for primary-care physicians. Am Rev Respir Dis. 1983 Jun;127(6):739–746. doi: 10.1164/arrd.1983.127.6.739. [DOI] [PubMed] [Google Scholar]
  99. Stross J. K. Relationships between knowledge and experience in the use of disease-modifying antirheumatic agents. A study of primary care practitioners. JAMA. 1989 Nov 17;262(19):2721–2723. [PubMed] [Google Scholar]
  100. Tannenbaum T. N., Gyorkos T. W., Abrahamowicz M., Bédard L., Carsley J., Franco E. D., Delage G., Miller M. A., Lamping D. L., Grover S. A. Immunization delivery methods: practice recommendations. Can J Public Health. 1994 Jul-Aug;85 (Suppl 1):S37–S40. [PubMed] [Google Scholar]
  101. Tape T. G., Campbell J. R. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993 Jun;94(6):619–625. doi: 10.1016/0002-9343(93)90214-a. [DOI] [PubMed] [Google Scholar]
  102. Tierney W. M., Hui S. L., McDonald C. J. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care. 1986 Aug;24(8):659–666. doi: 10.1097/00005650-198608000-00001. [DOI] [PubMed] [Google Scholar]
  103. Tierney W. M., McDonald C. J., Hui S. L., Martin D. K. Computer predictions of abnormal test results. Effects on outpatient testing. JAMA. 1988 Feb 26;259(8):1194–1198. [PubMed] [Google Scholar]
  104. Tierney W. M., Miller M. E., McDonald C. J. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990 May 24;322(21):1499–1504. doi: 10.1056/NEJM199005243222105. [DOI] [PubMed] [Google Scholar]
  105. Turner R. C., Waivers L. E., O'Brien K. The effect of patient-carried reminder cards on the performance of health maintenance measures. Arch Intern Med. 1990 Mar;150(3):645–647. [PubMed] [Google Scholar]
  106. Vinicor F., Cohen S. J., Mazzuca S. A., Moorman N., Wheeler M., Kuebler T., Swanson S., Ours P., Fineberg S. E., Gordon E. E. DIABEDS: a randomized trial of the effects of physician and/or patient education on diabetes patient outcomes. J Chronic Dis. 1987;40(4):345–356. doi: 10.1016/0021-9681(87)90050-6. [DOI] [PubMed] [Google Scholar]
  107. Waddell D. L. The effects of continuing education on nursing practice: a meta-analysis. J Contin Educ Nurs. 1991 May-Jun;22(3):113–118. doi: 10.3928/0022-0124-19910501-09. [DOI] [PubMed] [Google Scholar]
  108. Wexler J. R., Swender P. T., Tunnessen W. W., Jr, Oski F. A. Impact of a system of computer-assisted diagnosis. Initial evaluation of the hospitalized patient. Am J Dis Child. 1975 Feb;129(2):203–205. doi: 10.1001/archpedi.1975.02120390037008. [DOI] [PubMed] [Google Scholar]
  109. White C. W., Albanese M. A., Brown D. D., Caplan R. M. The effectiveness of continuing medical education in changing the behavior of physicians caring for patients with acute myocardial infarction. A controlled randomized trial. Ann Intern Med. 1985 May;102(5):686–692. doi: 10.7326/0003-4819-102-5-686. [DOI] [PubMed] [Google Scholar]
  110. White P. T., Pharoah C. A., Anderson H. R., Freeling P. Randomized controlled trial of small group education on the outcome of chronic asthma in general practice. J R Coll Gen Pract. 1989 May;39(322):182–186. [PMC free article] [PubMed] [Google Scholar]
  111. White R. H., Hong R., Venook A. P., Daschbach M. M., Murray W., Mungall D. R., Coleman R. W. Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing. J Gen Intern Med. 1987 May-Jun;2(3):141–148. doi: 10.1007/BF02596140. [DOI] [PubMed] [Google Scholar]
  112. Wigton R. S., Zimmer J. L., Wigton J. H., Patil K. D. Chart reminders in the diagnosis of anemia. JAMA. 1981 May 1;245(17):1745–1747. [PubMed] [Google Scholar]
  113. Williamson J. W., German P. S., Weiss R., Skinner E. A., Bowes F., 3rd Health science information management and continuing education of physicians. A survey of U.S. primary care practitioners and their opinion leaders. Ann Intern Med. 1989 Jan 15;110(2):151–160. doi: 10.7326/0003-4819-110-2-151. [DOI] [PubMed] [Google Scholar]
  114. Wilson D. M., Taylor D. W., Gilbert J. R., Best J. A., Lindsay E. A., Willms D. G., Singer J. A randomized trial of a family physician intervention for smoking cessation. JAMA. 1988 Sep 16;260(11):1570–1574. [PubMed] [Google Scholar]
  115. Winickoff R. N., Coltin K. L., Morgan M. M., Buxbaum R. C., Barnett G. O. Improving physician performance through peer comparison feedback. Med Care. 1984 Jun;22(6):527–534. doi: 10.1097/00005650-198406000-00003. [DOI] [PubMed] [Google Scholar]
  116. Wones R. G. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care. 1987 Jan;25(1):78–82. doi: 10.1097/00005650-198701000-00009. [DOI] [PubMed] [Google Scholar]

Articles from CMAJ: Canadian Medical Association Journal are provided here courtesy of Canadian Medical Association

RESOURCES