Evaluation of procedural skills in family medicine training

Procedures are regarded as an integral part of family medicine. There are many advantages to doing procedures in the office: patients are more satisfied if procedures are done by their family physicians^1,2; physicians are able to provide continuity of care; procedures cost less than they would if performed by specialists; wait times are shorter; and physician satisfaction is greater.¹ Studies show that family physicians are more likely to perform procedures in their practices if they received training for those procedures during residency.^3,4

Currently, the College of Family Physicians of Canada (CFPC) does not evaluate procedural skills on the Certification Examination in Family Medicine. This does not indicate a lack of interest in promoting or evaluating procedures but, perhaps, a realization of the difficulty of assessing procedures. Several experts have identified the evaluation of procedural skills as the most challenging aspect of developing procedural skills curricula.^5-7

The curriculum for teaching procedural skills is not standardized, and, therefore, training varies widely among the 16 family medicine programs.⁸ This lack of standardization makes evaluation difficult. Yet the authors of an American national survey of procedural skill requirements in family practice residency have challenged educators: “If one accepts that procedural skills are a fundamental part of family practice training, then our specialty must assure core competency for a residency graduate.”⁹

Because evaluation drives the curriculum, the CFPC has an essential role and responsibility in the evaluation of procedural skills. There is a precedent with the patient-centred method: when this became the focus of the Certification oral examinations (or simulated office orals), training centres ensured that the patient-centred method was taught and learned in all the training programs.

Two important steps

There are 2 important initial steps the CFPC should take to ensure that all residents learn core procedures during their training. First, the CFPC should include standards for curricula on procedures in the Red Book of Standards for Accreditation of Residency Training Programs: Family Medicine; Emergency Medicine; Enhanced Skills; Palliative Medicine. Second, the CFPC should include evaluation of procedures on the national Certification examination.

The CFPC’s Working Group on Procedural Skills has developed a core group of procedural skills that all residents should learn during their training; this is described in an article by Wetmore et al in the October 2005 issue of Canadian Family Physician.¹⁰ A core list accomplishes several goals: it allows residents to be aware of exactly which skills they need to master; it clarifies the scope of family medicine for hospital medical advisory committees; and it creates a guide for family medicine educators to develop explicit criteria for assessing competence.⁹

There are several ways to include procedures on the Certification examination. Procedures can be evaluated using short-answer management problems, which could be created to review the steps of a procedure or the indications, contraindications, and precautions. This would be particularly useful with such procedures as intrauterine device insertion or toenail removal that have many steps but few real-life opportunities for practice.

National Objective-Structured Clinical Examinations (OSCEs) would be expensive but could be considered, as OSCEs are currently being offered by the Medical Council of Canada for all residents after 18 months of training and by the Quebec College of Family Physicians for their combined Certification examination and provincial licensing examination. The administration fees for these examinations are more than $1000 per resident. Using OSCEs in family medicine departments might be less expensive and more feasible. Realistic models, such as the breast cyst aspiration model, could produce valid evaluation but are expensive. Realistic models for other procedures could be developed by curriculum-development groups to facilitate the transfer of skills and could be used for resident training and evaluation. Many simple, creative models are inexpensive and effective for training. One example of a breast cyst model is a balloon filled with flour and bath-oil beads, which readily simulates the technical aspects of the procedure.¹¹ Practising suturing using pigs’ feet is a time-honoured method of training and could be used for evaluation. The use of checklists during OSCEs is important to ensure reliability of performance.

Evaluation of procedural skills in a national examination would allow assessment of only specific skills for a few procedures, but it would also allow evaluation of such aspects as comfort with technical procedures and facility with instruments. This should provide some reassurance for certification bodies about skills training in general. Evaluation of procedural skills during examinations should supplement, but not replace, observation of procedural skills by a clinical preceptor. Such in-training evaluations of residents are useful for real-life assessments of performance for common procedures, such as cryotherapy, skin biopsies, and incision and drainage of abscesses. Real-life observation has many strengths. Because it focuses on actual problems in clinical practice, residents are motivated by its relevance. It permits more than the evaluation of procedural skills: it is the only setting that allows evaluation of the integration of history taking, doctor-patient relationship, decision making, and empathy.¹² Clinical settings, however, do have drawbacks. The procedure experience provided varies from setting to setting and depends on clinical opportunities, preceptor interest, and availability. The training experience can be maximized by improving on validity and reliability with the use of checklists. Without structured criteria, 2 evaluators often disagree with each other when assessing a learner.¹³

The Working Group on Procedural Skills and other groups under the guidance of the CFPC’s Committee on Examinations could create step-by-step checklists of each procedure for family medicine training programs. Individual family medicine programs could train residents to assess their colleagues. This would improve the knowledge and skills of those being assessed and those performing assessments.

Earlier introduction

The more common procedures, such as incision and drainage of abscesses, could be taught to medical students during family medicine rotations and electives by family medicine faculty and residents. Students entering family medicine residency would then have these skills, and it would increase the presence and contribution of family medicine at the undergraduate level. Harper and colleagues found a positive correlation between the number of procedures taught in family medicine programs and successful residency recruiting.¹⁴ This might offer a needed boost to the recruitment of medical students to family medicine residency programs.

Training programs

The CFPC should increase the number of continuing medical education events on procedural skills and link them to Mainpro® credits to increase the ability of academic family physicians to teach procedural skills.

Similarly, training programs need to give higher priority to procedural skills training. This should be reflected in faculty recruitment and curriculum development. Such strategies as procedural skills clinics and referrals from family physicians to family medicine training sites could also be used to increase the opportunities for residents to learn procedures.

We need a combination of methods to evaluate resident performance of procedures effectively. To increase performance of procedures in practice, we need to highlight office procedures at all levels of medical education: medical school, residency programs, and continuing medical education.

It is time for action in teaching and evaluating procedural skills. The CFPC can provide leadership by focusing on the following steps:

• promote adoption of a core list of procedures by family medicine training programs;

• encourage curriculum development for procedural skills training, including development of practical models to facilitate skills transfer, especially for less common clinical procedures;

• encourage in-training evaluation involving procedural skills assessment; and

• evaluate core skills training on the national Certification examination.

Biography

Dr Rivet is an Associate Professor in the Department of Family Medicine at the University of Ottawa in Ontario, and Dr Wetmore is an Associate Professor in the Department of Family Medicine at the University of Western Ontario in London.