Teaching and evaluation of ethics and professionalism

Abstract

Objective

To document the scope of the teaching and evaluation of ethics and professionalism in Canadian family medicine postgraduate training programs, and to identify barriers to the teaching and evaluation of ethics and professionalism.

Design

A survey was developed in collaboration with the Committee on Ethics of the College of Family Physicians of Canada. The data are reported descriptively and in aggregate.

Setting

Canadian postgraduate family medicine training programs.

Participants

Between June and December of 2008, all 17 Canadian postgraduate family medicine training programs were invited to participate.

Main outcome measures

The first part of the survey explored the structure, resources, methods, scheduled hours, and barriers to teaching ethics and professionalism. The second section focused on end-of-rotation evaluations, other evaluation strategies, and barriers related to the evaluation of ethics and professionalism.

Results

Eighty-eight percent of programs completed the survey. Most respondents (87%) had learning objectives specifically for ethics and professionalism, and 87% had family doctors with training or interest in the area leading their efforts. Two-thirds of responding programs had less than 10 hours of scheduled instruction per year, and the most common barriers to effective teaching were the need for faculty development, competing learning needs, and lack of resident interest. Ninety-three percent of respondents assessed ethics and professionalism on their end-of-rotation evaluations, with 86% assessing specific domains. The most common barriers to evaluation were a lack of suitable tools and a lack of faculty comfort and interest.

Conclusion

By far most Canadian family medicine postgraduate training programs had learning objectives and designated faculty leads in ethics and professionalism, yet there was little curricular time dedicated to these areas and a perceived lack of resident interest and faculty expertise. Most programs evaluated ethics and professionalism as part of their end-of-rotation evaluations, but only a small number used novel means of evaluation, and most cited a lack of suitable assessment tools as an important barrier.

The teaching of ethics and professionalism at all levels of medical education has received broad support.^1–4 The teaching and evaluation of ethics is an accreditation standard for undergraduate and postgraduate medical training in Canada.^5,6 Accreditation standards for family medicine postgraduate training programs in Canada state that each program must “provide a curriculum in family medicine ethics” and must carry out a “formal evaluation of the attitudes, knowledge, and skills pertinent to the ethics of family medicine.”⁷ The College of Family Physicians of Canada (CFPC) provides material to assist postgraduate programs with the teaching of ethics,⁸ and other Canadian, American, and international resources are available.^9–11 In the area of evaluation, there is published material describing strategies for evaluation of ethics and professionalism in the postgraduate population,^12,13 but only a few articles are dedicated to the family medicine context.^14,15 Most of these simply describe different strategies and do not discuss how widely they are being used. The extent and nature of ethics and professionalism teaching occurring in Canadian family medicine postgraduate training programs are unknown.

The primary goal of this survey was to document the scope of ethics and professionalism teaching and evaluation in Canadian family medicine postgraduate training programs. A secondary goal was to identify the barriers to teaching and evaluating ethics and professionalism in these residency programs.

METHODS

A survey methodology was used. There were 17 family medicine postgraduate training programs in Canada in 2008. In June 2008 the program director from each program was asked to take part in the survey. The survey was available in both English and French.

The survey was designed by the author in collaboration with the Committee on Ethics of the CFPC. The survey was based on a literature search and on a workshop on ethics teaching and evaluation presented at Family Medicine Forum in 2006. The survey was further refined and piloted with family medicine teachers across Canada.

The first section of the survey focused on teaching and asked program directors about learning objectives, curricular plans, human resources, use of teaching resources, and barriers to teaching ethics and professionalism. The second part dealt with evaluation and requested information related to the responsibility for, the nature of, and barriers to evaluation. The survey instrument is available upon request.

The initial focus of the survey was ethics; however, piloting the survey and consultation with program directors revealed that the term professionalism was now used by some programs to describe areas previously considered to be ethics, there was considerable overlap between ethics and professionalism, and many programs defined these terms differently. Therefore, respondents were asked to report all teaching and evaluation they included under the broad headings of ethics and professionalism in order to obtain the most complete picture of what was being taught and evaluated in these areas.

The data were collected between June and December of 2008. In order to encourage full disclosure, the study data are reported in aggregate and no individual program is identified in the analysis. The data are reported in a descriptive manner. The study was approved by the Health Research Ethics Board of the University of Manitoba.

RESULTS

In total, 15 of the 17 postgraduate programs returned completed surveys, for a response rate of 88%.

Teaching

Eighty-seven percent of the programs that responded had learning objectives in the area of ethics and professionalism, while 67% had curriculums or teaching plans for ethics and professionalism, and 67% had specific individuals who were assigned the role of coordinating or providing teaching in this area.

About three-quarters of programs (73%) used the CFPC website (Family Medicine Ethics section) as a teaching resource, while 67% used journal articles on ethics topics. About half of the programs used articles from the CMAJ Bioethics for Clinicians series (53%) and guidelines and statements from their provincial College of Physicians and Surgeons (47%).

All programs that responded used small group discussion and clinical case analysis as teaching methods. Ninety-three percent used bedside teaching, while 86% used lectures. Twenty-nine percent used written assignments or journaling.

Programs were also asked about the qualifications of the individuals who taught ethics and professionalism at their sites (Table 1), how many hours of instruction were scheduled for ethics and professionalism teaching (Table 2), and the main barriers to effective teaching of ethics and professionalism (Table 3). In addition to the barriers identified by multiple programs, listed in Table 3, individual programs also cited a lack of teaching tools, generational differences, and ethics and professionalism not being a high enough priority.

Table 1.

Individuals used to deliver ethics and professionalism teaching

TEACHERS	PROPORTION OF SITES
Family physician with interest in the area	80
Family physician with training in the area	73
Clinical ethicist	47
Other physician with training or interest	33
Lawyer	27
Other clinician (eg, nurse) with training or interest	13
Theologian	13
Philosopher	7

Table 2.

Hours of scheduled instruction in ethics and professionalism

HOURS PER YEAR	PROPORTION OF SITES
< 6	40
6–9	27
10–13	27
> 13	6

Table 3.

Barriers to effective teaching of ethics and professionalism, as identified by program directors

BARRIERS	PROPORTION OF SITES
Need for faculty development (awareness and confidence)	47
Competing learning needs or not enough time	47
Residents not interested	47
Lack of evaluation tools or difficult to evaluate	27
Faculty not interested or overworked	27

Evaluation

Ninety-three percent of the programs that responded evaluated some aspect of ethics and professionalism on their end-of-rotation evaluation forms for their core family medicine rotations. For programs that evaluated ethics and professionalism on their end-of-rotation forms, 86% asked evaluators to assess specific domains of ethics and professionalism, while 64% asked evaluators to make global assessments of residents.

Forty percent of programs stated that if a resident failed the ethics and professionalism component of the evaluation, he or she failed the entire rotation. Another 33% of programs stated that a resident who failed the ethics and professionalism component of the evaluation might or might not fail the rotation depending upon the nature and severity of the problem.

The survey offered a list of evaluation strategies, and programs were asked to choose which ones they used to evaluate ethics and professionalism (Table 4).

Table 4.

Strategies used to evaluate ethics and professionalism

EVALUATION STRATEGIES	PROPORTION OF SITES
End-of-rotation evaluation	93
Clinical case analysis	87
Direct clinical observation	80
Simulated office oral or other OSCE	73
Feedback from nurses or other health care professionals	67
Short-answer management problem	20
Reflective diaries or journals	20
Feedback from patients	7
Essay questions	7
Feedback from other residents	7

OSCE—objective structured clinical examination.

Some sites offered additional means of evaluation. Two sites described daily written feedback (called field notes in one program) that residents collected from clinical preceptors that specifically assessed aspects of ethics and professionalism.

Another program developed an evaluation form to assess resident knowledge, skills, and attitudes in ethics and professionalism over the entire 2 years of the program, which was completed by a clinical supervisor near the end of the resident’s training.

Programs were also asked to identify barriers to evaluating ethics and professionalism. The most frequently identified barriers are reported in Table 5. Additional barriers that were reported by individual programs included preceptors who were reluctant to fail residents on professionalism if they were otherwise competent, faculty complaining only informally about a resident’s professionalism, inconsistency across sites, and a lack of commitment.

Table 5.

Barriers to evaluation of ethics and professionalism, as identified by program directors

BARRIERS	PROPORTION OF SITES
Lack of suitable or objective tools	60
Lack of faculty development	53
Lack of faculty time or lack of faculty	33
Remediation challenges	13
Lack of resources	13

DISCUSSION

There was a good response rate to the survey. Despite the fact that ethics teaching is mandated by accreditation standards, 2 programs did not have learning objectives in this area, and 33% of programs did not have teaching plans or curriculums for ethics and professionalism. Most programs reported having family physicians with either interest or training in the area to lead their educational efforts. Given the many learning needs programs must address in addition to ethics and professionalism, it is not surprising that the hours of scheduled teaching were generally low (40% of programs reported less than 6 hours of scheduled instruction per year). Many program directors believed there was a fair amount of unscheduled, informal instruction in the area of ethics and professionalism that occurred in their programs. The most common barriers to teaching were not particularly surprising—lack of faculty engagement, competing learning needs, and lack of resident interest—but it was notable that more programs did not identify these as issues.

Regarding evaluation, by far most programs assessed ethics and professionalism on their end-of-rotation evaluation forms, with most asking evaluators to assess specific domains. A total of 73% of programs stated that a resident who failed the ethics and professionalism portion of his or her evaluation would either automatically fail the rotation, or could potentially be failed depending upon the nature and severity of the concerns. It appears these programs treat concerns about ethics and professionalism seriously and are willing to act upon the information obtained from their evaluations. It also raises questions about how effectively the other 27% of programs are identifying and responding to ethical and professional issues.

Most programs relied heavily on end-of-rotation evaluations, clinical case analysis, and direct clinical observation to evaluate their residents. It was interesting to note that about three-quarters of programs used simulated office orals, but only 20% used short-answer management problems to evaluate ethics and professionalism. Because these are the formats used for the national Certification examination, all programs are presumably using these evaluation tools for other curricular areas, and could be applying them to the evaluation of ethics and professionalism. There is good support in the literature for the use of simulated office orals in the area of ethics and professionalism,^16,17 but it is difficult to find literature on the use of short-answer management problems or other similar forms of evaluation for ethics and professionalism. A national data bank of simulated office oral cases and short-answer management problems specific to ethics and professionalism in family medicine might prove extremely helpful to those programs not currently using these strategies.

A large number of programs used feedback from other health care professionals in their evaluations, but only 1 program collected data from patients and 1 collected data from other residents. While there are challenges to obtaining reliable and valid feedback from patients and peers, these might be particularly valuable sources of information for evaluation in this particular area.^18,19 A small number of programs described innovative evaluation tools such as daily notes focused on ethics and professionalism or a dedicated evaluation form for ethics and professionalism.

The barriers to evaluation identified by programs were varied, but most programs cited a lack of suitable evaluation tools. In fact, the literature describes many tools that seem to be quite suitable,^{12,13,16–18} and so the problem might be more a lack of awareness than a lack of tools.

Even though most programs had faculty leaders with interest or expertise in ethics and professionalism, many programs still identified a lack of faculty awareness and confidence as an important barrier to effective evaluation. This suggests that broader engagement and faculty development with all teachers of family medicine is necessary, in addition to the establishment of program leaders. The CFPC has recently published a new resource on its website, the Ethics in Family Medicine: Faculty Handbook.²⁰ This might be very helpful for postgraduate programs, as most of them reported they were familiar with the previous ethics resources available on the CFPC website.

Limitations

There are a number of limitations to this study. Ethics and professionalism are distinct areas (although there is substantial overlap), and so including both terms in the survey might have been confusing for some respondents. The introduction of the survey acknowledged that these were distinct areas and explained to participants why both were included: to capture the widest possible range of activities that related to the teaching and evaluation of ethics and professionalism. There were a small number of responses overall, but the response rate was good. In order to allow the program directors to feel free to disclose potentially difficult information (eg, if they were not meeting accreditation standards in this area), the information was collected confidentially and was only reported in aggregate. This means potential links between the characteristics of programs and their success (or lack thereof) in specific areas cannot be explored.

This study identifies many issues that warrant further examination. With the adoption of the CanMEDS– Family Medicine roles,²¹ both ethics and professionalism teaching should be included under the professional role. Future studies could explore how different programs are conceptualizing this material, and how they are organizing the relevant curricular content. While many programs identified faculty leads, the dedicated time, resources, and support these individuals received for ethics and professionalism teaching was not explored in this study. These might be better predictors of high-quality ethics and professionalism teaching than simply the presence of a faculty lead. Another issue worth exploring is faculty-wide knowledge and comfort levels with ethics and professionalism concepts and with teaching these concepts. Many programs stated that much of their ethics and professionalism teaching occurred at the bedside, so most ethics and professionalism teaching was delivered by front-line clinical supervisors, not designated faculty leads. This means the quality and consistency of ethics and professionalism teaching will only be as good as relevant faculty development programs that are available and used—something not explored in this study. There was also great variability between programs, and innovative ideas for teaching and evaluation were not widely used. Barriers to the publishing and dissemination of best practices in this area should be further explored.

Conclusion

This is the first published survey of ethics and professionalism teaching and evaluation taking place in Canadian family medicine postgraduate training programs. It demonstrates that most programs have learning objectives and interested family doctor leads to help drive the educational effort. Despite this there are small numbers of scheduled hours, and few innovative methods are used for teaching in this area. Barriers to teaching are wide ranging and include a lack of faculty confidence, a lack of resident interest, and competing learning needs. The end-of-rotation evaluation form is the primary means of evaluation for almost all sites, with many programs evaluating specific domains. Some innovative evaluation strategies were described, including daily feedback strategies, such as field notes, and ethics-specific evaluation forms. Barriers to evaluation included a perceived lack of evaluation tools and a lack of faculty interest and time.

EDITOR’S KEY POINTS.

• The teaching and evaluation of ethics is an accreditation standard for undergraduate and postgraduate medical training in Canada. Despite this requirement, 2 of the postgraduate family medicine training programs surveyed in this study did not have learning objectives in this area, and 33% of programs did not have teaching plans or curriculums for ethics and professionalism.

• This survey found that while most programs had learning objectives and interested family doctor leads to help drive the educational effort, there were small numbers of scheduled hours dedicated to ethics and professionalism and few innovative methods used for teaching in this area.

• Some innovative evaluation strategies were described, including daily feedback strategies, such as field notes, and ethics-specific evaluation forms. The barriers to evaluation identified by programs were varied, but most cited a lack of suitable evaluation tools. In fact, the literature describes many tools that seem to be quite suitable, and so the problem might be more a lack of awareness than a lack of tools.

POINTS DE REPÈRE DU RÉDACTEUR.

• L’enseignement et l’évaluation de l’éthique est un élément requis pour l’agrément des programmes de deuxième et de troisième cycle en médecine au Canada. Malgré cette obligation, 2 des programmes de résidence en médecine familiale examinés dans cette enquête n’avaient pas d’objectifs d’apprentissage dans cette matière et 33 % d’entre eux n’avaient pas de projet pédagogique en éthique et professionnalisme ou de cursus pour ces matières.

• Cette enquête a montré que même si la plupart des programmes avaient des objectifs d’apprentissage et intéressaient les médecins de famille à poursuivre leurs efforts de formation, on consacrait peu d’heures à l’éthique et au professionnalisme et on utilisait peu de méthodes innovatrices pour l’enseignement de ces matières.

• Certaines méthodes d’évaluation innovatrices ont été décrites, incluant un feedback quotidien, par exemple, à l’aide de mémos et de formulaires d’évaluation spécifiques à l’éthique. Les programmes ont identifié plusieurs obstacles à l’évaluation, mais la plupart mentionnaient l’absence d’outils d’évaluation convenables. En réalité, toutefois, plusieurs outils apparemment adéquats sont décrits dans la littérature, de sorte que le problème pourrait être un manque de connaissances plutôt qu’un manque d’outils.