Skip to main content
PMC home page
CMAJ : Canadian Medical Association Journal logoLink to CMAJ : Canadian Medical Association Journal
. 1999 Oct 19;161(8):965–970.

Evaluating the effectiveness of 2 educational interventions in family practice

A E Borgiel 1, J I Williams 1, D A Davis 1, E V Dunn 1, N Hobbs 1, B Hutchison 1, C R Wilson 1, J Jensen 1, J J O'Neil 1, M J Bass 1
PMCID: PMC1230705  PMID: 10551192

Abstract

BACKGROUND: Structured feedback of information can produce change in physician behaviour. The objective of this study was to assess the effectiveness of 2 educational interventions for improving the quality of care provided by family physicians in Ontario: the Practice Assessment Report (PAR) and the Continuing Medical Education Plan (CMEP) with a follow-up visit by a mentor. METHODS: The study was a randomized controlled trial. Physicians in the control group received only the PAR, whereas those in the experimental group received the PAR, CMEP and mentor interventions. The participants were 56 family physicians and general practitioners (27 in the PAR group and 29 in the CMEP group) in southern Ontario who agreed to participate in the interventions and provide data. A total of 2395 patients randomly sampled from the practices returned questionnaires and consented to have their medical records abstracted. The outcome measures were global scores in 4 areas--quality of care, charting, prevention and overall use of medications--and patient ratings of satisfaction with care and preventive practices. The measures were applied at the beginning (phase 1) and end (phase 2) of the study. RESULTS: The mean global scores at the end of the study for the PAR group were 70.1% for quality of care, 84.7% for prevention, 77.7% for charting and 82.2% for overall use of medications. The corresponding scores for the CMEP group were 68.3%, 82.1%, 76.4% and 83.2%. In the patient satisfaction component, the personal care scores at phase 2 were 93.6% for the PAR group and 94.6% for the CMEP group. Examples of the scores for prevention for the PAR group were 98.3% for children's current immunization, 96.6% for blood pressure measured within the previous 5 years, 79.4% for referral of women of the appropriate age for mammography within the previous 2 years, and 58.4% for discussion about alcohol use. The corresponding scores for the CMEP group were 95.8%, 97.6%, 77.6% and 64.6%. The changes in mean scores between phase 1 and phase 2 ranged from -1.9 to 2.3 points. There were no significant differences between the 2 groups in phase 1 or phase 2 scores or in change in scores. A total of 64.3% of the physicians rated the PAR as useful, 26.5% found the CMEP to be useful, and 41.0% considered the mentor strategy to be a useful form of continuing medical education. Although changes in practice related to the PAR, CMEP or mentor were reported by some physicians, they were not related to chart audit or patient scores. INTERPRETATION: Educational interventions based on quality-of-care assessments and directed to global improvements in quality of care did not result in improvements in the outcome measures. Educational interventions may have to be targeted to specific areas of the practice, with physicians being monitored and receiving ongoing feedback on their performance.

Full Text

The Full Text of this article is available as a PDF (258.2 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Attarian L., Fleming M., Barron P., Strecher V. A comparison of health promotion practices of general practitioners and residency trained family physicians. J Community Health. 1987 Spring;12(1):31–39. doi: 10.1007/BF01321395. [DOI] [PubMed] [Google Scholar]
  2. Avorn J., Soumerai S. B. Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based "detailing". N Engl J Med. 1983 Jun 16;308(24):1457–1463. doi: 10.1056/NEJM198306163082406. [DOI] [PubMed] [Google Scholar]
  3. Battista R. N., Williams J. I., MacFarlane L. A. Determinants of primary medical practice in adult cancer prevention. Med Care. 1986 Mar;24(3):216–224. doi: 10.1097/00005650-198603000-00004. [DOI] [PubMed] [Google Scholar]
  4. Berwick D. M., Coltin K. L. Feedback reduces test use in a health maintenance organization. JAMA. 1986 Mar 21;255(11):1450–1454. [PubMed] [Google Scholar]
  5. Borgiel A. E., Williams J. I., Bass M. J., Dunn E. V., Evensen M. K., Lamont C. T., MacDonald P. J., McCoy J. M., Spasoff R. A. Quality of care in family practice: does residency training make a difference? CMAJ. 1989 May 1;140(9):1035–1043. [PMC free article] [PubMed] [Google Scholar]
  6. Buntinx F., Winkens R., Grol R., Knottnerus J. A. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract. 1993 Jun;10(2):219–228. doi: 10.1093/fampra/10.2.219. [DOI] [PubMed] [Google Scholar]
  7. Davis D. A., Thomson M. A., Oxman A. D., Haynes R. B. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995 Sep 6;274(9):700–705. doi: 10.1001/jama.274.9.700. [DOI] [PubMed] [Google Scholar]
  8. Davis D. A., Thomson M. A., Oxman A. D., Haynes R. B. Evidence for the effectiveness of CME. A review of 50 randomized controlled trials. JAMA. 1992 Sep 2;268(9):1111–1117. [PubMed] [Google Scholar]
  9. Horder J., Bosanquet N., Stocking B. Ways of influencing the behaviour of general practitioners. J R Coll Gen Pract. 1986 Nov;36(292):517–521. [PMC free article] [PubMed] [Google Scholar]
  10. Kessner D. M., Kalk C. E., Singer J. Assessing health quality--the case for tracers. N Engl J Med. 1973 Jan 25;288(4):189–194. doi: 10.1056/NEJM197301252880406. [DOI] [PubMed] [Google Scholar]
  11. Kincaid W. H. Changing physician behavior: the peer data method. QRB Qual Rev Bull. 1984 Aug;10(8):238–242. [PubMed] [Google Scholar]
  12. Maheux B., Dufort F., Lambert J., Berthiaume M. Do female general practitioners have a distinctive type of medical practice? CMAJ. 1988 Oct 15;139(8):737–740. [PMC free article] [PubMed] [Google Scholar]
  13. Norman G. R., Neufeld V. R., Walsh A., Woodward C. A., McConvey G. A. Measuring physicians' performances by using simulated patients. J Med Educ. 1985 Dec;60(12):925–934. doi: 10.1097/00001888-198512000-00004. [DOI] [PubMed] [Google Scholar]
  14. Norton P. G., Dunn E. V., Soberman L. What factors affect quality of care? Using the Peer Assessment Program in Ontario family practices. Can Fam Physician. 1997 Oct;43:1739–1744. [PMC free article] [PubMed] [Google Scholar]
  15. Oxman A. D., Thomson M. A., Davis D. A., Haynes R. B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995 Nov 15;153(10):1423–1431. [PMC free article] [PubMed] [Google Scholar]
  16. Palmer R. H., Hargraves J. L. The Ambulatory Care Medical Audit Demonstration Project. Research design. Med Care. 1996 Sep;34(9 Suppl):SS12–SS28. doi: 10.1097/00005650-199609002-00003. [DOI] [PubMed] [Google Scholar]
  17. Sibley J. C., Spitzer W. O., Rudnick K. V., Bell J. D., Bethune R. D., Sackett D. L., Wright K. Quality-of-care appraisal in primary care: a quantitative method. Ann Intern Med. 1975 Jul;83(1):46–52. doi: 10.7326/0003-4819-83-1-46. [DOI] [PubMed] [Google Scholar]

Articles from CMAJ: Canadian Medical Association Journal are provided here courtesy of Canadian Medical Association

RESOURCES