Performance of Residents Serving as Clinical Teachers: A Student-Based Assessment

Abstract

Background

Little is known about the competences of residents as clinical teachers in African health care institutions.

Objective

We evaluated the clinical teaching skills of internal medicine residents from the perspective of medical students in a tertiary teaching institution in Africa.

Methods

We used the augmented Stanford Faculty Development Program Questionnaire, which has evidence of validity and reliability. To avoid a Hawthorne effect, students completed the questionnaire anonymously and confidentially after clinical teaching sessions by residents. A minimum score of 4 on a scale of 1 to 5 was defined a priori as possession of good clinical teaching skills.

Results

Sixty-four medical students assessed all 20 internal medicine residents in the Department of Medicine, University of Ibadan. Mean performance scores for the domains ranged from 3.07 to 3.66. Residents performed best in creating a good learning climate and worst in the promotion of understanding and retention. Sex of the resident, duration of residency, and rank had no significant impact (.09 < P < .94) on their teaching skills.

Conclusions

Consistent with other observations in the literature, residents' clinical teaching skills were suboptimal, particularly in their ability to promote understanding and retention. To enhance these skills, we recommend the integration of appropriately tailored programs to teach pedagogic skills programs in residency training.

Editor's Note: The online version ^{(125.5KB, doc)} of this article contains the Clinical Teaching Assessment Instrument used in this study.

Introduction

Development of clinical acumen through good clinical teaching is a key component of medical education. It is particularly critical in resource-limited settings where technology-assisted diagnosis is not readily accessible. Clinical teaching gives the opportunity to simultaneously impart, formatively assess, and actively improve the full range of skills required of a physician, including cognitive, psychomotor, decision-making, self-learning, and communication skills.

Residents play an important role in imparting these skills to junior learners.^1–4 In the United States, residents provide up to 70% of clinical teaching of medical students.^1–8 They are also playing a significant role in Nigeria as in other African countries, where there are too few physician-educators to cope with the expanding student population.^9,10

Little is known about the clinical teaching skills of residents globally and in Africa.^9–11 Studies are needed to identify areas of deficiencies that can be targeted by interventions.

The objective of our study was to evaluate the clinical teaching skills of medical residents from the perspective of medical students in a Nigerian teaching hospital.

Methods

We conducted this study in all units of the Department of Medicine, University College Hospital, University of Ibadan. All medical students who had their internal medicine rotation during the study period (January 2011–June 2011) and gave consent were enrolled. Respondents were fifth-year medical students undergoing their second 8-week clinical rotation. Our study excluded residents in other departments and dental students rotating through the department to avoid heterogeneous responses. The curriculum for the rotation included history taking, examination, and clinical reasoning skills spanning the various aspects of internal medicine.

Ethical approval was obtained from the University of Ibadan/University College Hospital Human Research Ethics Committee.

Instrument

Methods for evaluating the effectiveness of residents' clinical teaching skills include different areas of assessment (knowledge, attitudes, and skills); various types of evaluators (self, learner, or faculty); and different ways of conducting the assessment (questionnaires, observation, and narrative feedback).^{1–4,12–14} Written feedback may be more valuable than narrative feedback for improving resident performance and increasing resident satisfaction and interest in teaching.¹ An example is the Clinical Teaching Perception Inventory, which is clinically oriented and uses self-reporting by teachers.¹⁴

We conducted a student-based evaluation by using the Stanford Faculty Development Program Questionnaire (SFDPQ), a psychometrically robust instrument for evaluation of clinical teachers. It consists of 58 items in 8 domains and uses a closed Likert-type format (1  =  strongly disagree to 5  =  strongly agree). A score of 4 indicates agreement by the students that the residents possessed the respective skills, while a score of 5 indicates excellent demonstration of the corresponding teaching skill.

The SFDPQ assesses the ability of the teacher to establish a positive learning climate; effectively manage, focus, and pace the teaching encounter; communicate teaching goals; promote effective understanding and retention; evaluate learner's achievements and give feedback; and promote self-directed learning.¹² The SFDPQ is considered the most comprehensive, well-structured, learner-centered, valid, and reliable assessment of teaching skills.¹³ However, it does not assess the teacher's knowledge and subject mastery, necessitating the addition of 5 items to measure teachers' knowledge.¹³ In addition, the SFDPQ does not assess teachers' enthusiasm, warmth, accessibility, optimism, and empathic skills. We included a ninth domain on teacher's attitude, comprising 7 items to produce an augmented SFDPQ (provided as online supplemental material). The additional teacher's attitude domain demonstrated internal consistency reliability (Cronbach α  =  0.89) and good construct validity in relation to the original SFDPQ (ρ  =  0.647, P < .001).

Assessment

We administered the questionnaire (provided as online supplemental material) to medical students after bedside teaching sessions organized by residents. This was performed after the fourth week of the rotation following several encounters with the residents. They were required to complete the questionnaire confidentially and anonymously. To eliminate a Hawthorne effect, residents were not informed that they were being assessed.⁸

Data Analysis

We generated and summarized domain and total scores by using measures of central tendency and dispersion. On the SFDPQ, a score of 4 indicates agreement that the teacher possesses the respective skill, while a score of 5 indicates excellent demonstration of the corresponding skill. We a priori selected a score of 4 as indicating that a resident possessed good clinical teaching skills for the respective item or domain.¹²

We explored possible associations between sex, duration of residency training, rank of the residents, and teaching skill scores by using Mann-Whitney U and Spearman rank correlation statistics as appropriate. We set the level of significance at P < .05.

Results

Twenty residents (15 men and 5 women; 9 registrars and 11 senior registrars) were assessed by 64 medical students (49 men, 15 women) on medical posting in the 7 subspecialties in the Department of Medicine, University College Hospital, Ibadan. Sixty-four of 68 questionnaires were returned for a response rate of 94%.

Female medical students significantly rated the residents better in the “evaluation domain” than male medical students. (Mann-Whitney U  =  109.5, P  =  .03). There was no significant difference in rating between female and male medical students in other domains and on the global scale (31.0 < Mann-Whitney U < 188.0; .14 < P < .80).

Residents' Clinical Teaching Skills

Residents' mean performance scores were suboptimal, ranging from 3.07 to 3.66 (table), with a score of 4 (the a priori determined score signifying the resident possessed the respective skills) not reached in any domain. Residents performed best in creating a good learning climate and least well in promoting understanding and retention (figure).

TABLE.

Evaluation of Clinical Teaching Skills of Residents

FIGURE .

Clinical Teaching Skills (Mean Performance Scores) of Residents, Arranged in Descending Order

The questionnaire used a closed, Likert-type format, with 5 ordinally scaled options per item (1  =  strongly disagree, 2  =  disagree, 3  =  neither agree nor disagree, 4  =  agree, and 5  =  strongly agree). A score of 4 indicates satisfactory demonstration of the corresponding teaching skill.

Determinants of Clinical Teaching Skills

The mean total SFDPQ scores were 3.28 ± 0.13 for male residents and 3.11 ± 0.21 for female residents. The sex of the residents had no significant impact on the SFDPQ domain and total scores (54.0 < Mann-Whitney U < 287.5; .09 < P < .94).

The mean total SFDPQ scores were 3.15 ± 0.15 for senior residents and 3.42 ± 0.16 for junior residents. The level of the resident (junior versus senior) had no significant effect on SFDPQ domain and total scores (99.0 < Mann-Whitney U < 343.5; .16 < P < .54). In addition, the duration of residency did not correlate significantly with the clinical teaching skills (−0.022 < ρ < −0.197; .13 < P < .87).

Discussion

This is a novel study that evaluates the clinical teaching skills of residents in an African institution from the perspective of medical students, using a comprehensive formal assessment with an instrument with established validity. Our survey response rate of 94% is considered satisfactory.

In a Dutch study, in which residents assessed their clinical teachers by using a modified 26-item version of the SFDPQ, communication of goals had the lowest score (mean  =  3.41), and the highest score achieved was 4.07 for “professional attitude toward the residents.”¹⁵ The higher-scoring domains (with mean score of > 3.5) in our assessment were learning climate, teacher's attitude, and evaluation. The remaining domains (including the communication of goals domain) were worse, with the domain assessing the promotion of understanding and retention being the worst (table).

The lowest scoring attribute, the ability of the resident to promote understanding and retention, is a crucial learning goal. It depends on the teacher's ability to present well-organized material, explain relationships in the material, answer learners' questions clearly, emphasize what he or she wanted the learners to remember, have learners reformulate materials, and apply teaching materials to the learners' own experiences.¹² Our observed performance gaps suggest specific pedagogic learning needs for residents, which are similar to the needs identified in a study of residents in Mexico.¹¹ We sought to identify possible factors influencing these performance gaps.

Among the residents, duration of residency did not correlate significantly to clinical teaching skills, and sex and level of training also had no significant influence on clinical teaching skills. Although these negative findings may be due to our relatively small sample size, similar observations have been made elsewhere.^11,15

Limitations of our study include the fact that it was conducted in a single site, reducing the generalizability of our findings. It is also possible that our a priori determined score for competence in clinical teaching may have been set too high for residents as teachers. The Dutch study¹⁵ found a number of scores below 4 for faculty serving as teachers. It may well be that clinical teaching skills need to be improved in both residents and faculty.

Intervention programs to address the limitations and performance gaps in clinical teaching by residents are warranted. There is mounting evidence that such tailored interventions to address pedagogic skills are effective and produce satisfaction and positive change in satisfaction, knowledge, attitude, behavior, and performance.^{1,12,13,16–18}

Conclusion

We found suboptimal clinical teaching skills in residents, as perceived by medical students, particularly in the ability to promote understanding and retention. To enhance clinical teaching skills, formal pedagogic programs should be integrated into residency training. Larger multi-institutional studies are needed to identify factors influencing the clinical teaching skills of resident physicians on the African continent.