Defining core procedure skills for Canadian family medicine training

Abstract

OBJECTIVE

To create a list of core and enhanced procedures suitable for family medicine training.

DESIGN

Mailed or e-mailed survey using a Delphi technique.

SETTING

Randomly selected family physician practices across Canada.

PARTICIPANTS

Family physicians from urban, small-town, and rural practice locations and academic family physicians. All were experienced family physicians with from 3 to 36 years in practice.

INTERVENTIONS

Participant physicians were asked to rate each of 158 procedures as to whether they would expect a graduate from a Canadian family practice training program to have learned and be capable of performing that procedure in their own community. In a second survey, participants were asked to verify the core and enhanced procedures lists produced from the first survey.

MAIN OUTCOME MEASURES

Physicians’ opinions about a comprehensive list of skills.

RESULTS

Twenty-two physicians responded to the first survey (92% response rate) and 14 to the second (58% response rate). Sixty-five core procedures and 15 enhanced procedures were identified in the surveys. More procedures were ranked on the core list and were performed by rural and small-town physicians than by urban physicians. Physicians’ agreement with placement of procedures on the core list ranged from 55% to 100% and of procedures on the enhanced list from 50% to 64%. Fifty-five of the procedures on the core list had agreement from more than 70% of participants.

CONCLUSION

Procedure lists represent the opinions of Canadian family physicians about the importance of specific procedure skills for new family physicians in their communities. Procedure lists will be helpful for family medicine training programs to evaluate and refine their teaching of procedure skills.

EDITOR’S KEY POINTS.

• The College of Family Physicians of Canada commissioned a Working Group on Procedural Skills to create a core and enhanced list of skills that would guide residents’ training. This is the first list generated from suggestions of family physicians in practice.

• A two-round Delphi technique was used to create a consensus view. Family doctors from across Canada, in a variety of settings (academic, community, urban, rural) participated.

• Sixty-five procedures were finally agreed on for the core list; an additional 15 were on the enhanced list.

• All physicians indicated more procedures in the core list than they performed themselves in practice. Rural physicians rated more procedures as core than urban physicians and performed more procedures in practice.

POINTS DE REPÈRE DU RÉDACTEUR.

• Le Collège des médecins de famille du Canada a donné au Groupe de travail sur les habiletés techniques le mandat d’établir des listes d’habiletés techniques fondamentales et supplémentaires devant guider la formation des résidents. Il s’agit de la première liste du genre créée à partir des suggestions de médecins de famille en pratique.

• Une technique Delphi en deux étapes a été utilisée pour en arriver à un accord consensuel. Les participants étaient des médecins de famille exerçant dans des contextes divers (universitaire, communautaire, urbain ou rural), un peu partout au Canada.

• On s’est finalement entendu pour inscrire 65 techniques sur la liste de base et 15 autres sur la liste supplémentaire.

• Tous les médecins ont inscrit sur la liste de base plus de techniques qu’ils n’en effectuaient dans leur pratique. Les médecins ruraux ont inscrit plus de techniques dans la liste de base et effectuaient plus de techniques dans leur pratique que les médecins urbains.

Performing clinical procedures is an important part of family medicine practice. Despite its importance, crucial questions about training in family practice procedures remain unanswered. Which procedures should family physicians learn and how should they be learned? These questions are pertinent to family medicine training programs.

A survey of all family medicine program directors in Canada resulted in 24 unique lists of procedure skills. The lists contained from 10 to 75 skills. Only 30 skills appeared on more than 50% of the lists.¹ Similar findings were reported in a US study where 63 separate lists of procedures were obtained with the number of skills ranging from 3 to 117. Twenty-five procedures were common to more than 50% of the lists.² Several studies have reported that performance of procedures varies by practice location and is more common and more varied in rural locations.^3,4

Against this background, defining a core list of skills for family medicine training programs has been attempted both in the United States and in Canada.^1,2,4 Previous attempts to define skills have started with lists of procedures taught in training programs or lists of procedures performed in practice. Wide variability in lists was evident, and it was difficult to find common ground. This study started with the experience of family physicians in a variety of settings and built the skill set required for practice in these settings through consensus.

The College of Family Physicians of Canada (CFPC), through the Section of Teachers of Family Medicine, commissioned the Working Group on Procedural Skills in 2003. This paper describes the work of this group toward development of a list of core and enhanced skills for Canadian family physicians’ training.

METHODS

This study was conducted in family physicians’ practices in a variety of settings across Canada.

Developing comprehensive procedure lists

A comprehensive list of procedures was developed starting with all procedures listed as mandatory by Canadian family medicine training programs in the survey by van der Goes et al.¹ Procedures were added to this list by members of the Working Group and by comparison with the Core Family Medicine Procedures list adapted by the Society of Rural Physicians of Canada (personal communication from Dr James Rourke, 2003). The list was checked against other published lists of procedures in family medicine^2-10 and against results from the 2004 National Physician Survey (personal communication from Ms Sarah Scott, Janus Project Coordinator, College of Family Physicians of Canada 2004).

We aimed to make the list as comprehensive as possible; the Working Group did modify the list, however, according to our adopted definition of a procedure as being “the mental and motor activities required to execute a manual task” and involving patient contact.¹¹ Physical examination skills and purely interpretive skills, such as electrocardiographic reading or stress tests, were excluded. The final list contained 158 procedures.

Selecting experts for survey

This study employed the Delphi technique,¹² in which a group of participants give their opinions regarding specific questions, and these opinions are progressively refined in light of the responses of the entire group. Rounds of data collection allow for development of participants’ responses to reach a consensus of views. In this study we used a two-round modified Delphi technique.

Family physicians were chosen randomly from membership lists of the Section of Teachers of Family Medicine (academic group) and the CFPC, stratified by postal code (urban, small-town and rural groups). Physicians on each random list were contacted and asked to participate in the study. Six physicians were recruited from each group: academic (full-time faculty in a family medicine teaching capacity), urban (practising in a large city), small town (practising in a town of approximately 20 000 to 50 000 population), and rural (practising in a town of 10 000 population or less). For each group, the six participating family physicians were identifiedfrom within the first 10 contacts, suggesting that family physicians were highly motivated to participate in the study. The group size of six was chosen arbitrarily but was within the range of group size for the Delphi technique.¹²

First survey

Family physicians in all groups were given the comprehensive list of 158 procedures and asked to rate each procedure according to the following two statements.

Statement 1. “I would expect a graduate of a 2-year family medicine program in Canada to have learned and be capable of performing this procedure in my community.”

Statement 2. “I perform this procedure in my own practice.”

Members of all groups were asked to add any skills that they thought should be on the comprehensive procedures list.

Preparing core and enhanced lists

A core procedure was defined as one most physicians in at least three of the practice settings would acknowledge as suitable. These procedures would be more likely to reflect the “core” of the skill set (ie, be practised in a variety of settings). An enhanced procedure would be one judged as suitable by most physicians in only two settings. This would be more likely to be a procedure practised more often in only certain settings (eg, rural practice).

The core list and enhanced list for the second survey were developed from the results of the first survey. Procedures were added to the core list if at least 80% of physicians responded positively to statement 1 in at least three of the four groups. Procedures were added to the enhanced list if at least 80% in two of the four groups answered statement 1 positively. Procedures that did not meet either of these criteria were dropped from the list.

Second survey

All participating physicians were given the core and enhanced lists developed from the first survey. They were asked to determine whether each procedure on the list should be a core procedure, enhanced procedure, or should be removed from the list entirely. Once again, participating physicians were allowed to add any procedures that they thought should be on the list.

This study was not submitted for ethics approval, given its context. Participation in this study was considered proof of consent. Physicians’ names were retained confidentially, and only cumulative data were reported. The study was designed in consultation with the Research Department of the CFPC.

RESULTS

Six physicians in each of the four groups initially agreed to take part in the survey. One physician in each of the rural and small-town groups subsequently dropped out of the study. Characteristics of participating physicians are shown in Table 1. All family physicians were Certificants of the CFPC. Twenty-two of the 24 physicians responded to the first survey for a response rate of 92%.

Table 1.

Characteristics of family physicians in Delphi groups

Table 2 shows the number of procedures meeting the criterion for the core list and performance of procedures reported by various groups. In all groups, physicians indicated more procedures on the core list than they actually performed in practice. More procedures were ranked on the core list and were performed by rural and small-town physicians than by urban physicians.

Table 2.

Comparison of Delphi group responses to first survey

*Statement 1: “I would expect a graduate of a 2-year family medicine program in Canada to have learned and be capable of performing this procedure in my community.”

Fourteen physicians responded to the second survey for a response rate of 58%. The final core and enhanced lists were then developed. A procedure remained on the core list or was placed there if more than 50% of respondents agreed with its placement. A procedure remained on the enhanced list or was placed there if more than 50% of respondents agreed with its placement. Using these criteria, three procedures were moved from the initial enhanced list to the final core list, namely, endometrial biopsy, application of scaphoid cast, and peripheral venous access for infants. No procedures were moved from the core to enhanced list, and no procedures were dropped from either the initial core or initial enhanced lists.

The final core list and enhanced list are shown in Tables 3 and 4, respectively, along with the percentage agreement for their placement on these lists and also the average percentage performance of each procedure as obtained from the initial survey.

Table 3.

Core procedures list

Table 4.

Enhanced procedures list

Sixty-five procedures are listed on the final core list and 15 on the final enhanced list. There was, in general, greater agreement among physicians for procedures placed on the core list, and these procedures were more likely to be performed in practice.

DISCUSSION

The question of which procedures should be selected for inclusion in family practice residency training can be approached in several ways, including looking at the most frequently performed procedures; assessing community needs; examining common problems in routine practice and the procedures needed for diagnosis or management; examining current screening recommendations; and using an economic approach.¹³ In our study, physicians’ opinions would reflect procedures performed in practice, common problems in practice, and the needs of their own communities. This is the first study to define a set of core procedure skills for family physician training in Canada using opinions of physicians from a variety of settings.

The data clearly show that family physicians often overestimate the number of skills required relative to their actual performance in practice. This seems most pronounced for academic and urban family physicians and less pronounced for small-town and rural family physicians. Academic family physicians could be reflecting their need to train residents in a variety of procedures for a variety of practice settings. This finding is consistent with other studies.¹⁴ While academic family physicians acknowledge the need for training in a certain number of skills, their actual performance of these skills in their own practices is about 50% of this estimate. This poses an interesting dilemma, given this group’s responsibility to train most of the family physicians in Canada. Family medicine training programs will need to develop innovative methods of teaching those procedures that are less frequently performed in academic practices.

These lists will be a helpful guide to the skills teachers should focus on. This study should be further encouragement for family medicine training programs to evaluate their training in procedure skills thoroughly and adopt strategies to improve procedure skills training for their residents, as has previously been suggested.^3,15

While these lists do represent a solid basis for skills training in family medicine, they will not please all groups of family physicians. As have other studies, this study has shown that rural and small-town family physicians perform more procedures in their practices and are more likely to believe that the core list is not as inclusive as it should be for their practices. These lists do not include all procedures that family physicians could perform, depending on their practice locations, skills, and interests. Nevertheless, the core list does seem to achieve agreement from all the family physician groups surveyed for a high percentage of the procedures listed.

It is appropriate that the lists include clusters of procedures that have similar motor skill requirements. Training programs can train physicians in types of skills that facilitate learning other related skills. As an example, inclusion of endometrial biopsy training along with intrauterine device training is appropriate because the technical aspects of this training are similar for the two procedures.

Limitations

The fact that the Delphi groups were relatively small, particularly in the second round, can be seen as a limitation of this study; however, the ultimate results were based on the cumulative number of participants. The core list was unlikely to have been influenced because there was a high degree of agreement here. It is possible that more skills might have been identified for the enhanced list if more physicians had participated in the second survey. Further studies using these lists would help to define the core list more accurately. A further limitation of this study is the self-report nature of the data. This should not have influenced the results greatly because the outcome of interest was physicians’ opinions of a comprehensive list of skills.

On the positive side, the initial list of procedures accumulated for this survey was quite comprehensive. Attempts were made to base the list on Canadian studies and to include all the various family physician groups. Participants in the study were all experienced family physicians and were encouraged at both survey stages to add procedures that they thought should be present. They also had an opportunity to vote procedures off the lists during the second survey. No other procedures were added or removed from the lists, which suggests that the lists ring true with family physicians from all groups. This reflects the validity of these lists and makes their generalizability to Canadian family physicians more likely.

Conclusion

The core and enhanced lists presented here have been built from the actual practice experience and expectations of family physicians in a variety of practice settings. The objective of the Working Group on Procedural Skills for the core list to receive broad consensus has been met. This list should be relevant for most family physicians across the country. At the very least the core list of procedures should lead to discussion and debate and further study of procedures in family medicine. It is a solid basis for family medicine training programs as they evaluate and refine procedure skills training for their residents.

Biographies

Dr Wetmore is an Associate Professor in the Department of Family Medicine at The University of Western Ontario in London.

Dr Rivet is an Associate Professor in the Department of Family Medicine at the University of Ottawa in Ontario.

Dr Tepper is a rural locum tenens and is on the Council of the Society of Rural Physicians of Canada.

Dr Tatemichi is an Assistant Professor in the Department of Family Medicine at Dalhousie University in Halifax, NS, and practises in an aboriginal community in New Brunswick.

Dr Donoff is a Professor in the Department of Family Medicine at the University of Alberta in Edmonton.

Dr Rainsberry is Director of Education for the College of Family Physicians of Canada in Mississauga, Ont. All are members of the Working Group on Procedural Skills commissioned by the Section of Teachers of Family Medicine at the College of Family Physicians of Canada.