Abstract
Context
Medical educational reform includes enhancing role modelling of clinical teachers. This requires faculty being aware of their role model status and performance. We developed the System for Evaluation of Teaching Qualities (SETQ) to generate individualized feedback on previously defined teaching qualities and role model status for faculty in (non) academic hospitals.
Objectives
(i) To examine whether teaching qualities of faculty were associated with their being seen as a specialist role model by residents, and (ii) to investigate whether those associations differed across residency years and specialties.
Methods & Materials
Cross-sectional questionnaire survey amongst 549 Residents of 36 teaching programs in 15 hospitals in the Netherlands. The main outcome measure was faculty being seen as specialist role models by residents. Statistical analyses included (i) Pearson's correlation coefficients and (ii) multivariable logistic generalized estimating equations to assess the (adjusted) associations between each of five teaching qualities and ‘being seen as a role model’.
Results
407 residents completed a total of 4123 evaluations of 662 faculty. All teaching qualities were positively correlated with ‘being seen as a role model’ with correlation coefficients ranging from 0.49 for ‘evaluation of residents’ to 0.64 for ‘learning climate’ (P<0.001). Faculty most likely to be seen as good role models were those rated highly on ‘feedback’ (odds ratio 2.91, 95% CI: 2.41–3.51), ‘a professional attitude towards residents’ (OR 2.70, 95% CI: 2.34–3.10) and ‘creating a positive learning climate’ (OR 2.45, 95% CI: 1.97–3.04). Results did not seem to vary much across residency years. The relative strength of associations between teaching qualities and being seen as a role model were more distinct when comparing specialties.
Conclusions
Good clinical educators are more likely to be seen as specialist role models for most residents.
Introduction
Medical education, and in particular, graduate medical education continues to face many challenges. One of the major challenges is the issue of role modelling by clinician educators in delivering high-quality clinical training to residents on their journey to becoming high-performing physicians. Role modelling is at the heart of building physicians who exhibit the knowledge, attitudes, behaviors and identity of a ‘good professional’ [1]–[3]. Role models not only help shape our future physicians, they also influence their career choices and predict residents' satisfaction with their training [4], [5]. Given the widespread interest for enhancing the effectiveness of medical education the scanty attention for role modelling is remarkable. This evidently has to do with the rather elusive character of role modelling. Learning from role models occurs through observation and reflection [1], [5] and is not a straight forward process. Rather it is a complex mix of conscious and unconscious learning in which explicit, theoretical knowing and implicit or tacit knowing are being transferred from the clinical teacher – the ‘master’ - to the trainee [6]–[9]. Although many clinical teachers are involved in today's residency training residents do not choose to pattern the activities or behaviors of all of them. Not all faculty are perceived by residents as good specialist role models, defined as a person whose skills and behavior residents desire to emulate. In fact various studies report that many faculty - up to more than 50% - are not perceived as role models or are seen as negative role models by residents [1], [2], [10]. This situation should arouse at least some concern about the quality of medical education. As laid down in the Compact Between Resident Physicians and Their Teachers, a declaration of the fundamental principles of graduate medical education, residents deserve faculty to be role models and faculty commit themselves to act like them [11]. Clearly, in today's attention to educational reform enhancing role modelling as a teaching strategy is appropriate if not necessary [2]. Even experienced faculty may have a limited insight in their strengths and weaknesses as teachers, in residents' perception of their behaviour and in the potential for improving their teaching skills and possibly their effectiveness as role models. Improving role modelling requires - at an individual level - that faculty are aware of their role model status and performance, reflect upon their experiences and participate in staff development when deemed necessary [1], [2]. To support this process we developed a system (named System for Evaluation of Teaching Qualities or SETQ) for generating individualized feedback for faculty in academic and other teaching medical institutions [12]–[14]. From the literature and our own conversations with residents we know they do recognize the multiple roles that faculty embody and display – often simultaneously - in performing their daily activities: as teachers and as care givers [15]–[19]. These roles should be distinguished from the person [2], [16]–[18]. There is a wealth of literature describing the characteristics of professionals effectively fulfilling these roles [18]–[20]. What is lacking is empirical evidence demonstrating if and how these roles are related. This study explores the association between faculty being seen as specialist role models and their teaching performance as clinical educators. The first objective of this study was to examine whether the teaching qualities of faculty as evaluated by residents were related to the faculty being considered role models by the residents who themselves are future specialists. The second objective was to investigate if and how any relations between teaching qualities and being seen as role models differed across specialties and residency years.
Methods
The System for Evaluation of Teaching Qualities (SETQ)
We developed the System for Evaluation of Teaching Qualities (or SETQ) to assess, monitor, feedback and possibly improve the teaching performance of clinician educators or faculty in residency programs in academic medical settings [12]–[14]. The SETQ is an integral and cyclical system of (i) two internet-based specialty-specific instruments for evaluating faculty's teaching qualities—one instrument completed by residents and another by faculty themselves, (ii) individualized quantitative and qualitative feedback reporting to each faculty, and (iii) follow-up discussion with the respective program director for individualized maintenance or improvement support. A core purpose of the SETQ study is to assist faculty in increasing their self insight into their teaching. This is aided by residents' evaluations in order to improve faculty's teaching performance within the Dutch graduate medical education system. SETQ was successfully introduced at one academic medical center in the Netherlands and has been adopted by many other residency programs across the Netherlands. The SETQ study aims to provide detailed longitudinal investigation of faculty development in terms of their teaching performance. Over time, the study hopes to report on the personal and contextual determinants of faculty's teaching performance. It will also research the educational and quality of care consequences of the teaching qualities of clinician educators in academic medicine.
The SETQ instruments were initially modeled on the Stanford Faculty Development Program (SFDP26) instrument developed in the United States [21]–[23]. We have described elsewhere the initial development, translation, back-translation, discussion, and specialty-specific adaption of the instruments [12]–[14]. We developed two instruments per specialty: one resident-completed instrument and one faculty self-evaluation instrument. For each specialty, both the residents' and faculty instruments were almost identical except that the resident-completed instrument additionally contained open-ended questions asking the residents to list the core strengths of each faculty they evaluate as well as some improvement points for the faculty. Although the instruments were specialty-specific, they still shared 22 core items aimed at measuring the faculty's performances on: creating conducive learning climate (7 items), their professional attitude toward residents (3 items), communicating learning goals (4 items), evaluating residents' knowledge and skills (4 items), and giving feedback to residents (4 items). Each item had a 5-point response: strongly disagree, disagree, neutral, agree and strongly agree. The 22 items could be grouped into five stable composite-scales for the resident-completed instruments across all specialties. The results of psychometric analysis have been reported elsewhere [12]–[14]. In this study, we used data from the resident-completed instrument.
Setting and Study Population
This multisite study was conducted at 36 teaching programs, of which 18 are being offered by a large academic medical center and 18 by 14 other teaching hospitals in the Netherlands. All 549 residents from 18 (sub-)specialties were invited by email to participate in the SETQ and evaluate all 743 faculty. Each resident received personal login details to access the relevant instruments via a dedicated, secure and password protected web-based portal. The formative purpose of the exercise and anonymity of use and eventual faculty feedback and reporting were emphasized. Residents were free to choose which faculty members to evaluate in their respective specialty and could evaluate many faculty members. Under Dutch law this study did not require ethical approval by the institutional review committee.
Outcome: Considered Role Models for Residents as Future Specialists
The study outcome was each resident's response to the singular global evaluation question “…this faculty is a role model for me as a future [specialist]”. Just like the core items on the instruments, the response here was measured on 5-point scale ranging from strongly disagree to strongly agree. This outcome was dichotomized as follows: 0 being the reference for responses “strongly disagree or disagree or neutral” versus 1 for “agree or strongly agree”.
Main Predictors: Teaching Qualities of Faculty
The main predictors consisted of the 5 composite-scales of teaching qualities of faculty, namely learning climate, professional attitude toward residents, communication of goals, evaluation of residents' knowledge and skills, and feedback. The teaching quality ‘learning climate’ refers to an environment in which self directed learning of residents is promoted, faculty adequately manage teaching encounters and are well prepared for teaching activities. Displaying a ‘professional attitude towards residents’ encompasses establishing a positive respectful ambiance of the faculty-resident relationship. ‘Communication of learning goals’ refers to the educator's ability to express expectations and establish and prioritize goals regarding the residents' skills, knowledge and/or attitudes resulting from a teaching interaction. ‘Evaluation’ describes the assessment of residents' achievements, and lastly, ‘feedback’ focuses on providing a resident with information for the purpose of improving his or her performance. Each composite-scale was the averaged score of the items that loaded primarily on it as shown in Table S1. This averaging ensured that the value of each composite-scale, like the loading items, ranged from 1 to 5 with 5 capturing the best possible teaching quality on that composite-scale.
Covariates
Several covariates were used in the analysis. Specialty — with internal medicine as reference — was adjusted for only in models involving the pooled analysis of all specialties. Sex and residency year (in all models except the residency-specific models for second aim of this paper) of the participating residents were the other covariates included in the statistical analysis descried below.
Statistical Analysis
First, we described the study sample using standard descriptive statistics. Second, for all specialties combined per residency year, and all residency years combined per specialty separately, we used Pearson's correlation coefficient to quantify the correlations between each composite-scale of teaching qualities and the outcome variable. These correlations gave a first insight into the unadjusted relationships between teaching qualities and faculty being seen as role models. Additionally, for easy visualization, the associations between each teaching composite-scale and being seen as a role model are displayed as scatterplots. Third, for a more definitive analysis of the first and second study objectives, for all specialties combined and each specialty separately, we used logistic generalized estimating equations to relate all five composite-scales of teaching qualities to the outcome, adjusted for the abovementioned covariates. These regression equations allowed for appropriate adjustment for cross-clustering because, like many healthcare surveys [24]–[27], evaluation scores from this study were nested within residents and faculty jointly, with some crossing among residents and faculty. That is, some faculty members were evaluated by some or all of the same residents in their residency program. This nested crossing implied that scores of any particular faculty member would correlate better within that faculty than across faculty members but at the same time there would be some correlations among scores given by the same resident to different faculty. Due to smaller sample sizes for some specialties such as neurosurgery, plastic surgery, orthopedic surgery and ophthalmology, we present combined specialty results grouping medical ones separately from the surgical specialties. To see how the associations varied across residency years, the analysis was repeated for all specialties stratified by residency years ranging from the first year to the sixth year. Interns (registered physicians who have not entered residency training yet) were classified separately. We note that not all specialties have six-year residencies in the Netherlands. Estimated odds ratios and their 95% confidence limits were reported.
In sensitivity analysis, we analyzed the outcome and main predictors as continuous scales and also log-transformed them to check the impact of choice of functional form and variable definition on our results. Further sensitivity analysis looked at the impact of different cut-points used in dichotomizing the outcome, for instance. Since there were no material differences in the findings of the different approaches, we report the results of the analysis where the outcome was a dichotomy (0 = strongly disagree or disagree or neutral” versus 1 = “agree or strongly agree”) and main predictors were assumed to be continuous scales. This facilitated interpretation where a 1-point change in each predictor variable would be associated with the reported odds of the outcome. All statistical analyses were conducted in PASW Statistics 18.0 release 18.0.0 for Mac operating system (SPSS Inc, Chicago, IL, 2009).
Results
Of the 549 invited residents 407 ultimately participated in the SETQ study, yielding a response rate of 74.1%. 56,3% of the residents were female. Residents evaluated 662 (89.1%) faculty members by completing a total of 4123 evaluations. There were about 10 evaluations per resident and approximately 6 evaluations per faculty member. These numbers varied between the 19 participating specialties. The mean number of evaluations per resident did not differ significantly between male and female residents nor between residents from different residency years, with the exception of interns who evaluated significantly less (on average 8) faculty members. Table 1 gives an overview of the numbers of participants and the residents' response rates per specialty.
Table 1. Number of Study Participants and Number of Residents' Evaluations.
| Number of residents' evaluations (%) | Number of residents (% female) | Mean number of evaluations per resident(range) | Mean number of resident evaluations per faculty | Number of faculty evaluated by residents | |
| All specialties | 4123 (100) | 407 (57.3) | 10.1 (45) | 6.2 | 662 |
| Internal medicine | 512 (12.4) | 64 (31.3) | 8.0 (24) | 6.2 | 83 |
| Chest medicine | 34 (0.8) | 5 (40) | 6.8 (6) | 4.3 | 8 |
| Cardiology | 177 (4.3) | 16 (31.2) | 11.1 (22) | 7.7 | 23 |
| Gastroenterology | 83 (2.0) | 8 (25) | 10.4 (13) | 5.2 | 16 |
| Neurology | 157 (3.8) | 17 (29.4) | 9.2 (14) | 10.5 | 15 |
| Radiology | 341 (8.3) | 18 (44.4) | 18.9 (22) | 13.6 | 25 |
| Radiotherapy | 135 (3.3) | 15 (60) | 9.0 (19) | 4.2 | 32 |
| Pediatrics | 772 (18.7) | 93 (74.9) | 8.3 (45) | 3.7 | 207 |
| General surgery | 297 (7.2) | 25 (40) | 11.9 (22) | 8.5 | 35 |
| Anesthesiology | 698 (16.9) | 38 (57.9) | 18.4 (38) | 13.7 | 51 |
| Neurosurgery | 18 (0.4) | 2 (50) | 9.0 (0) | 2.0 | 9 |
| Plastic surgery | 33 (0.8) | 7 (57.1) | 4.7 (2) | 6.6 | 5 |
| Ophthalmology | 17 (0.4) | 6 (83.3) | 2.8 (5) | 1.5 | 11 |
| Obstetrics & gynecology | 515 (12.5) | 55 (69.1) | 9.4 (23) | 5.7 | 90 |
| Physical medicine and rehabilitation | 22 (0.5) | 4 (100) | 5.5 (2) | 3.1 | 7 |
| Clinical genetics | 42 (1.0) | 5 (100) | 8.4 (6) | 4.2 | 10 |
| Pathology | 97 (2.4) | 9 (55.6) | 10.8 (13) | 6.9 | 14 |
| Orthopedics | 21 (0.5) | 5 (0) | 4.2 (5) | 3.5 | 6 |
| Ear, nose and throat surgery (ENT) | 152 (3.7) | 15 (18.8) | 10.1 (14) | 10.1 | 15 |
| Year of residency training | |||||
| Interns | 304 (7.4) | 38 (84.2) | 8.2 (10) | ||
| 1st year | 471 (11.4) | 46 (58.7) | 10.0 (23) | ||
| 2nd year | 711 (17.3) | 76 (53.9) | 9.6 (24) | ||
| 3rd year | 647 (15.7) | 67 (62.7) | 9.5 (33) | ||
| 4th year | 858 (20.8) | 70 (50.0) | 12.5 (38) | ||
| 5th year | 770 (18.7) | 70 (45.7) | 10.9 (45) | ||
| 6th year | 360 (8.7) | 39 (66.7) | 9.2 (24) |
Figure 1 shows the scatterplots for the association of each composite-scale of teaching skills with faculty being seen as role models. Table 2 displays the correlations between composite-scales of teaching qualities of faculty and their being seen as role models. For all specialties combined, all five composite scales were correlated with faculty being seen as role models by residents. The correlation coefficients ranged from 0.493 for ‘evaluation of residents’ toward 0.634 for ‘professional attitude toward residents’ and 0.637 for ‘learning climate’ (P<0.001). Among first thru third year residents from all specialties, the highest correlations were observed between ‘professional attitude towards residents’ and the outcome. For the fourth thru sixth year residents the teaching aspects ‘learning climate’ and ‘feedback’ showed the strongest correlations. ‘Evaluation of residents’ and ‘communication of goals’ displayed the smallest correlation with faculty being seen as role models. The specialty-specific correlations between each composite-scale of teaching qualities and faculty being seen as role models pointed to ‘learning climate’, ‘professional attitude toward residents’ and ‘feedback’ having the largest correlations with faculty being seen as role models in many specialties such as internal medicine, pediatrics, cardiology, radiology, general surgery and anesthesiology.
Figure 1. Scatterplots of each of five teaching qualities and being seen as role model specialist.
Table 2. Correlations between Faculty's Teaching Qualities and Their Being Seen as Specialist Role Models.
| Learning climate | Professional attitude to residents | Communication of goals | Evaluation of residents | Feedback | |
| All specialties | 0.637** | 0.634** | 0.516** | 0.493** | 0.599** |
| - First year residents | 0.605** | 0.637** | 0.452** | 0.455** | 0.606** |
| - Second year residents | 0.584** | 0.591** | 0.412** | 0.344** | 0.501** |
| - Third year residents | 0.562** | 0.586** | 0.504** | 0.414** | 0.546** |
| - Fourth year residents | 0.700** | 0.650** | 0.606** | 0.589** | 0.662** |
| - Fifth year residents | 0.655** | 0.657** | 0.495** | 0.536** | 0.619** |
| - Sixth year residents | 0.666** | 0.636** | 0.544** | 0.561** | 0.666** |
| Specific specialties | |||||
| Internal medicine | 0.747** | 0.722** | 0.610** | 0.584** | 0.676** |
| Chest medicine | 0.584** | 0.650** | 0.412* | 0.509** | 0.296 |
| Cardiology | 0.698** | 0.627** | 0.515** | 0.450** | 0.639** |
| Gastroenterology | 0.490** | 0.555** | 0.326* | 0.445** | 0.393** |
| Neurology | 0.633** | 0.379** | 0.494** | 0.553** | 0.318** |
| Radiology | 0.725** | 0.677** | 0.625** | 0.647** | 0.687** |
| Radiotherapy | 0.614** | 0.592** | 0.490** | 0.531** | 0.471** |
| Pediatrics | 0.626** | 0.668** | 0.524** | 0.475** | 0.661** |
| General surgery | 0.710** | 0.674** | 0.639** | 0.559** | 0.722** |
| Anesthesiology | 0.554** | 0.594** | 0.499** | 0.457** | 0.591** |
| Neurosurgery | 0.372 | 0.050 | -0.294 | 0.305 | 0.118 |
| Plastic surgery | 0.576** | 0.502** | 0.542** | 0.667** | 0.534** |
| Ophthalmology | 0.717** | 0.595* | 0.467 | 0.538* | 0.867** |
| Obstetrics & gynecology | 0.509** | 0.566** | 0.478** | 0.357** | 0.462** |
| Physical medicine and rehabilitation | 0.253 | 0.290 | -0.868* | 0.169 | 0.306 |
| Clinical genetics | 0.585** | 0.676** | 0.655** | 0.646** | 0.720** |
| Pathology | 0.464** | 0.403** | -0.179 | -0.020 | 0.241* |
| Orthopedic surgery | 0.591** | 0.761** | 0.888** | 0.276 | 0.247 |
| Ear, Nose and Throat Surgery (ENT) | 0.456** | 0.545** | 0.335** | 0.314** | 0.502** |
Correlation is significant at
P<0.05,
P<0.01.
Table 3 shows the mutually adjusted odds ratios for associations between teaching qualities and faculty being seen as role models for all specialties combined and each specialty separate. Table 4 summarizes these results in terms of the ranking of the relative importance of the teaching qualities. ‘Learning climate’, ‘professional attitude toward residents’ and ‘feedback’ had the highest odds ratios in the analysis combining all specialties. The same three teaching qualities were the most important predictors of faculty being seen as role models in internal medicine, anesthesiology, radiology, pediatrics, ENT, general surgery, and obstetrics and gynecology. In cardiology, neurology and general surgery ‘evaluation of residents’ knowledge and skills' was found to be a more important predictor than ‘ feedback’ (for neurology and surgery) or ‘professional attitude’ (cardiology). When comparing the combined medical and surgical specialties, ‘learning climate’ and ‘feedback’ were the best predicting teaching qualities for the medical specialties (odds ratios being 2.91 and 2.59 respectively) and for the surgical specialties the most important teaching qualities were ‘feedback’ and ‘professional attitude’ (odds ratios 3.55 and 2.92 respectively).
Table 3. Adjusted Odds Ratios (95% Confidence Interval) for Associations between Faculty's Training Qualities and their Being Seen as Specialist Role Models.
| Mutually Adjusted Odds Ratio | 95% Confidence Interval | ||
| All specialties | Learning climate | 2.45 | 1.97–3.04 |
| Professional attitude toward residents | 2.70 | 2.34–3.10 | |
| Communication of goals | 1.10 | 0.93–1.31 | |
| Evaluation of residents' knowledge and skills | 1.50 | 1.25–1.80 | |
| Feedback | 2.91 | 2.41–3.51 | |
| Internal medicine | Learning climate | 3.13 | 1.54–6.36 |
| Professional attitude toward residents | 2.69 | 1.79–4.03 | |
| Communication of goals | 1.57 | 0.87–2.82 | |
| Evaluation of residents' knowledge and skills | 1.82 | 1.06–3.13 | |
| Feedback | 2.39 | 1.29–4.24 | |
| Cardiology | Learning climate | 3.40 | 1.18–9.82 |
| Professional attitude toward residents | 2.29 | 0.91–5.73 | |
| Communication of goals | 1.17 | 0.43–3.22 | |
| Evaluation of residents' knowledge and skills | 3.99 | 0.88–18.04 | |
| Feedback | 3.74 | 0.37–37.72 | |
| Neurology | Learning climate | 7.79 | 1.65–36.76 |
| Professional attitude toward residents | 3.11 | 1.31–7.36 | |
| Communication of goals | 1.15 | 0.52–2.56 | |
| Evaluation of residents' knowledge and skills | 2.16 | 0.97–4.83 | |
| Feedback | 2.03 | 0.77–5.36 | |
| Pediatrics | Learning climate | 2.63 | 1.48–4.67 |
| Professional attitude toward residents | 2.47 | 1.77–3.44 | |
| Communication of goals | 1.16 | 0.81–1.66 | |
| Evaluation of residents' knowledge and skills | 2.16 | 1.37–3.39 | |
| Feedback | 3.63 | 2.25–5.86 | |
| Radiology & | Learning climate | 3.86 | 1.88–7.95 |
| Radiotherapy | Professional attitude toward residents | 2.45 | 1.59–3.78 |
| combined | Communication of goals | 0.78 | 0.43–1.41 |
| Evaluation of residents' knowledge and skills | 1.63 | 1.02–2.61 | |
| Feedback | 3.44 | 2.02–5.86 | |
| Other medicine | Learning climate | 4.74 | 2.04–12.09 |
| specialties * # | Professional attitude toward residents | 1.57 | 0.73–2.23 |
| Communication of goals | 0.52 | 0.23–0.86 | |
| Evaluation of residents' knowledge and skills | 1.45 | 0.67–2.62 | |
| Feedback | 1.84 | 1.29–4.52 | |
| General surgery | Learning climate | 3.12 | 0.81–12.07 |
| Professional attitude toward residents | 4.72 | 1.79–12.47 | |
| Communication of goals | 1.42 | 0.51–3.96 | |
| Evaluation of residents' knowledge and skills | 3.10 | 1.04–9.28 | |
| Feedback | 2.62 | 0.82–8.40 | |
| Otorhinolaryngology | Learning climate | 1.58 | 0.65–3.83 |
| Professional attitude toward residents | 2.59 | 1.15–5.86 | |
| Communication of goals | 1.04 | 0.46–2.36 | |
| Evaluation of residents' knowledge and skills | 0.85 | 0.40–1.80 | |
| Feedback | 3.50 | 1.17–10.56 | |
| Anesthesiology | Learning climate | 1.47 | 0.97–2.25 |
| Professional attitude toward residents | 2.47 | 1.84–3.31 | |
| Communication of goals | 1.40 | 0.91–2.17 | |
| Evaluation of residents' knowledge and skills | 1.18 | 0.76–1.83 | |
| Feedback | 4.03 | 2.58–6.13 | |
| Obstetrics & | Learning climate | 4.50 | 2.38–8.52 |
| gynecology | Professional attitude toward residents | 4.79 | 2.92–7.87 |
| Communication of goals | 1.22 | 0.73–2.04 | |
| Evaluation of residents' knowledge and skills | 0.78 | 0.46–1.32 | |
| Feedback | 3.04 | 1.67–5.51 | |
| Medical Specialties | Learning climate | 3.09 | 2.29–4.17 |
| Combined # | Professional attitude toward residents | 2.33 | 1.96–2.77 |
| Communication of goals | 0.98 | 0.79–1.22 | |
| Evaluation of residents' knowledge and skills | 1.82 | 1.45–2.28 | |
| Feedback | 2.49 | 1.96–3.15 | |
| Surgical Specialties | Learning climate | 2.06 | 1.52–2.79 |
| Combined ## | Professional attitude toward residents | 2.79 | 2.27–3.43 |
| Communication of goals | 1.31 | 1.00–1.71 | |
| Evaluation of residents' knowledge and skills | 1.15 | 0.87–1.53 | |
| Feedback | 3.25 | 2.43–4.36 |
*Gastroenterology, clinical genetics, chest medicine, physical and rehabilitation medicine, pathology;
Adjusted for each specialty in the full model, with internal medicine as reference;
Adjusted for each specialty in the full model, with general surgery as reference.
Table 4. Order of the Importance of Teaching Scales in Predicting Faculty Seen as Role Model Specialists; Results per Specialty (Summarizing Results of Table 3).
| Descending Order of Relative Importance | 1(most relevant) | 2 | 3 | 4 | 5(least relevant) |
| All specialties | Feedback | Professional attitude | Learning climate | Evaluation of residents | Communication of learning goals |
| Internal medicine | Learning climate | Professional attitude towards residents | Feedback | Evaluation of residents | Communication of learning goals |
| Cardiology | Evaluation of residents | Feedback | Learning climate | Professional attitude towards residents | Communication of learning goals |
| Neurology | Learning climate | Professional attitude towards residents | Evaluation of residents | Feedback | Communication of learning goals |
| Pediatrics | Feedback | Learning climate | Professional attitude towards residents | Evaluation of residents | Communication of learning goals |
| Radiology & Radiotherapy combined | Learning climate | Feedback | Professional attitude towards residents | Evaluation of residents | Communication of learning goals |
| Other medicine specialties: GE, clinical genetics, chest medicine, physical and rehabilitation medicine, pathology # | Learning climate | Feedback | Professional attitude towards residents | Evaluation of residents | Communication of learning goals |
| General surgery | Professional attitude towards residents | Learning climate | Evaluation of residents | Feedback | Communication of learning goals |
| Otorhinolaryngology | Feedback | Professional attitude towards residents | Learning climate | Communication of learning goals | Evaluation of residents |
| Anesthesiology | Feedback | Professional attitude towards residents | Learning climate | Communication of learning goals5 | Evaluation of residents |
| Obstetrics & gynecology | Professional attitude towards residents | Learning climate | Feedback | Communication of learning goals | Evaluation of residents |
| Medical Specialties Combined # | Learning climate | Feedback | Professional attitude towards residents | Evaluation of residents | Communication of learning goals |
| Surgical Specialties Combined ## | Feedback | Professional attitude towards residents | Learning climate | Communication of learning goals | Evaluation of residents |
Adjusted for each specialty in the full model, with internal medicine as reference.
Adjusted for each specialty in the full model, with general surgery as reference.
The results of the per residency year analysis shown in table 5 and summarized in table 6 indicate that the teaching qualities ranked somewhat differently in their influence on whether residents considered faculty as role models for different residency years. Yet, there were three noticeable patterns across all residency years. The first is that ‘feedback’ was consistently found to be the most important predictor for all residency years, except the third year. The second is that this was for all years followed by either ‘learning climate’ (second, fourth, fifth and sixth year residents) or ‘professional attitude towards residents’ (interns and first year residents). The third is that, consequently, ‘evaluation of residents’ and ‘communication of goals’ tended to have the least predictive power with the outcome among most residency years except the third year.
Table 5. Odds Ratios (OR) for the Adjusted Associations Between Teaching Qualities and Faculty Seen as Specialist Role Models across Different Residency Years.
| Interns:OR (95% C.I.) | First Year Residents:OR (95% C.I.) | Second Year Residents:OR (95% C.I.) | Third Year Residents:OR (95% C.I.) | Fourth Year Residents:OR (95% C.I.) | Fifth Year Residents:OR (95% C.I.) | Sixth Year Residents:OR (95% C.I.) | |
| Learning climate | 5.50 (1.70–17.82) | 3.77 (1.50–9.48) | 2.94 (1.63–5.30) | 1.26 (0.84–1.87) | 3.16 (1.88–5.34) | 4.32 (2.21–8.45) | 2.76 (1.40–7.53) |
| Professional attitude toward residents | 4.86 (2.60–9.09) | 3.94 (2.19–7.09) | 2.31 (1.68–3.17) | 2.67 (1.92–3.72) | 2.84 (2.00–4.03) | 4.30 (2.79–6.62) | 2.07 (1.18–3.61) |
| Communication of goals | 1.14 (0.57–2.26) | 0.84 (0.45–1.56) | 0.98 (0.64–1.52) | 1.18 (0.78–1.79) | 1.64 (1.10–2.45) | 0.84 (0.52–1.37) | 1.68 (0.82–2.47) |
| Evaluation of residents' knowledge and skills | 1.61 (0.75–3.48) | 1.44 (0.79–2.63) | 1.45 (0.92–2.29) | 1.54 (1.01–2.35) | 1.39 (0.92–2.12) | 2.00 (1.11–3.62) | 1.38 (0.72–2.66) |
| Feedback | 2.19 (0.96–5.01) | 4.19 (1.86–9.43) | 3.73 (2.21–6.31) | 2.33 (1.54–3.54) | 3.57 (2.18–5.84) | 4.79 (2.43–8.37) | 4.38 (2.16–8.90) |
For each residency year, there was simultaneous adjustment for all composite-scales, specialty, and resident's sex.
Table 6. Descending Order of the Importance of Teaching Scales in Predicting Faculty Seen as Role Model Specialists; Results per Year of Training (Summarizing Results of Table 5).
| Interns | First Year Residents | Second Year Residents | Third Year Residents | Fourth Year Residents | Fifth Year Residents | Sixth Year Residents | All residents |
| Learning climate | Feedback | Feedback | Professional attitude toward residents | Feedback | Feedback | Feedback | Feedback |
| Professional attitude toward residents | Professional attitude toward residents | Learning climate | Feedback | Learning climate | Learning climate | Learning climate | Professional attitude toward residents |
| Feedback | Learning climate | Professional attitude toward residents | Evaluation of residents' knowledge and skills | Professional attitude toward residents | Professional attitude toward residents | Professional attitude toward residents | Learning climate |
| Evaluation of residents' knowledge and skills | Evaluation of residents' knowledge and skills | Evaluation of residents' knowledge and skills | Learning climate | Communication of goals | Evaluation of residents' knowledge and skills | Communication of goals | Evaluation of residents' knowledge and skills |
| Communication of goals | Communication of goals | Communication of goals | Communication of goals | Evaluation of residents' knowledge and skills | Communication of goals | Evaluation of residents' knowledge and skills | Communication of goals |
P<0.05.
Discussion
Summary of Main Findings
This study provides strong empirical evidence of good clinical teachers being perceived as specialist role models by residents. Faculty most likely to be seen as good specialist role models are those rated highly on the teaching qualities ‘giving residents feedback’, ‘creating a positive learning climate’ and ‘a professional attitude towards residents’. Residents' views do not seem to vary much across residency years with regards to the relative importance assigned to one or more of these three teaching qualities and they even fully agree on the finding that clinical educators who perform well on the teaching quality ‘feedback’ are most likely seen as specialist role models. The different relative rankings in teaching qualities are more distinct when comparing specialties.
Strengths and Weaknesses
Findings should be considered in the light of potential study strengths and limitations. This study adds to the literature on role models and clinical teaching skills by moving beyond descriptive to empirical analysis of associations. We note, however, that our findings may be limited to the Netherlands, thus necessitating specific evaluations in other countries and health systems. This study involved faculty form 15 teaching hospitals whereby 55% were attending physicians at one academic medical center, thus potentially limiting the findings to this study population. Furthermore, as in many western health care systems, post-graduate medical education is also being modernized in the Netherlands. Although the introduction of competency-based residency training programs accelerates the development of clinician-educators, residents may not yet be exposed to new teaching requirements like the communication of learning goals. This may account for the higher level of “don't know responses” for the items loading on the communication of learning goals scale. Excluding this scale from the teaching qualities did not alter the substantive findings but could have equally introduced a bias by excluding the necessary adjustment for interscale correlations. Hence, we chose to maintain the scale for the reported data analysis in this study. Finally, it could be argued that faculty seen as role model specialists were perceived as better teachers rather than the other way around. Nonetheless, we point out that theoretically the opposite is more plausible since residents were students who were first exposed to the teaching skills of their faculty while their perception of faculty as their role model developed over time. So, even when those residents recall their role models as exhibiting superior teaching skills, our speculation that excellent clinical teachers were seen as role models will still be largely valid. Furthermore, in administering the instruments, residents were first asked to report on the faculty's actual teaching skills before stating the extent to which the evaluated faculty were seen as role models.
Explanation and Interpretation
Role modelling is the teaching method most preferred by residents [3], [28]. In exploring the phenomenon of successful role models research consistently finds that teaching qualities of physicians are of critical value in being seen as a role model by residents [16]–[19]. Most studies reach this conclusion based on qualitative research approaches, typically consulting residents to describe their role models and/or faculty to reflect on being a role model [16]–[18]. Our study goes further than most studies in providing strong empirical evidence for the previously reported qualitative finding that good clinical educators are more likely to be regarded as specialist role models. We hereto statistically explored the phenomenon of specialist role models in two ways. In the first step we looked at the bivariate relationship between being seen as a specialist role model and each of five teaching qualities defined previously in the context of the SETQ system for evaluating faculty as clinical teachers. Most of the five teaching qualities were positively and significantly related to being seen as a role model, with correlation coefficients falling within the range of 0.40 to 0.75 for all specialties combined and for the various residency years. Some of the specialty specific analyses did not yield strong correlations, i.e. for neurosurgery, ophthalmology, pathology and rehabilitation medicine, most likely due to the small sample sizes. The correlation coefficients reported indicate that the association between the teaching qualities and being a role model is moderate to strong lending some support to our informed assumption that teaching performance of faculty is part of the phenomenon of being a specialist role model. In the second step, we used the technique of linear regression to also describe the relation between teaching performance and being a role model. The listed adjusted odds ratios represent the odds of scoring highly on one or more teaching qualities and the outcome measure of being perceived as a role model. They indicate that faculty most likely to be seen as good specialist role models are those rated highly on the teaching qualities ‘giving residents feedback’, ‘creating a positive learning climate’ and ‘a professional attitude towards residents’. On the other hand, communication of learning goals' and ‘evaluation of residents’ are the least distinct predictors for being seen as a specialist role model for residents as future specialists. We may argue that these findings suggest that the latter two skills are more or less restricted to the clinical teaching situation: once in practice there is neither a need nor the required culture to set learning goals or to peer assess each other's knowledge or skills. In contrast, giving feedback, displaying a professional attitude and creating a positive learning climate are all qualities crucial for the high performance of every practicing physician and so need to be transferred from the teaching environment into future practice. Residents in our study seem to be able to distinguish between faculty's teaching qualities relevant for training them now and skills they desire to emulate for future practice.
Residents' views do not seem to vary much across residency years with regards to the relative importance assigned to one or more of these three teaching qualities; they agree on the finding that clinical educators who perform well on the teaching quality ‘feedback’ are most likely seen as specialist role models. Many studies stress that feedback is crucial for delivering effective teaching [29]. This study now finds that it is also important for being seen as a role model by residents across residency years. The different relative rankings in teaching qualities are more distinct when comparing specialties. Scoring well on ‘giving feedback’ clearly stands out as a strong predictor for being seen as a role model for surgical residents, which was also reported by Maker et al [30]. This may reflect their need for concrete guidance or and their tenacity to analyze feedback – be it confirmative or corrective – to improve their performance. For medical residents ‘creating a positive learning climate’ is considered the most important predictor for being perceived as a role model. As the core items of this teaching quality are i.e. participation in discussions, bringing up problems and keeping up to date with the literature, this finding could reflect residents' need to be trained in acquiring knowledge and critical clinical reasoning, typically seen as qualities needed as an internal medicine specialist.
Implications for Medical Education, Research and Policy
Explicit strategies are needed to support all faculty to become role models for residents. This study suggests that an effective and more active way to improve role modelling may be to enhance teaching skills. This requires that faculty are aware of their teaching performance. We found that systems such as SETQ can be instrumental in increasing faculty's self-insight by systematically providing them feedback on their teaching qualities and their role model status. Receiving feedback should be followed up by – preferably guided – reflection [31]–[33] and participation in staff development [1]. Future assessments need to demonstrate actual improvements in teaching and role modelling effectiveness.
Conclusions
Role modelling is an integral aspect of medical education and residents rightfully deserve good role models. Our study shows that good clinical educators are more likely to be seen as specialist role models for most residents. This opens up opportunities for improving role modelling as a teaching strategy since many of the predictors of being seen as a role model are teaching behaviors and skills that can be acquired.
Supporting Information
Psychometric Properties of the Five Composite-Scales of the SETQ. Instruments from 19 Medical and Surgical Specialties.
(DOC)
Acknowledgments
We thank all faculty and residents who participated in this study. We also thank Medox.nl for the development of the web application.
Footnotes
Competing Interests: For the SETQ study the AMC is collaborating with the provider of the electronic feedback reports, Medox.nl. AMC and Medox.nl received a fee per delivered feedback report from the participating teaching hospitals. KL and MJH are employed by the AMC. For his contributions OAA received honoraria from the Department of Quality and Process Innovation of the Academic Medical Center (AMC), Amsterdam. These stated interests do not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Funding: The study was funded by the Academic Medical Center (AMC) of the University of Amsterdam. Dr. Arah is also supported by a grant (Veni number 916.96.059) from the Netherlands Organization for Scientific Research (NWO). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
References
- 1.Cruess SR, Cruess RL, Steinert Y. Role modelling - making the most of a powerful teaching strategy. BMJ. 2008;336:718–721. doi: 10.1136/bmj.39503.757847.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kenny NP, Mann KVP, MacLeod HM. Role modelling in physicians' professional formation: reconsidering an essential but untapped educational strategy. Acad Med. 2003;78:1203–1210. doi: 10.1097/00001888-200312000-00002. [DOI] [PubMed] [Google Scholar]
- 3.Park J, Woodrow SI, Reznick RK, Beales J, MacRae HM. Observation, reflection, and reinforcement: surgery faculty members' and residents' perceptions of how they learned professionalism. Acad Med. 2010;85:134–139. doi: 10.1097/ACM.0b013e3181c47b25. [DOI] [PubMed] [Google Scholar]
- 4.Roberts LJ, Khursandi DC. Career choice influence in Australian anesthesists. Anaesth Intensive Care. 2002;30:355–359. doi: 10.1177/0310057X0203000315. [DOI] [PubMed] [Google Scholar]
- 5.Balmer D, Serwint JR, Ruzek SB, Ludwig S, Giardino AP. Learning behind the scenes: perceptions and observations of role modelling in pediatric residents' continuity experience. Ambul Pediatr. 2007;7:176–181. doi: 10.1016/j.ambp.2006.11.005. [DOI] [PubMed] [Google Scholar]
- 6.Stegeman JH. Dissertation University of Amsterdam. The Hague: Pasmans Drukkerij; 2008. Pupils of modern masters.197 [Google Scholar]
- 7.Polanyi M. Chicago: the University of Chicago Press; 1962. Personal knowledge: towards a post-critical philosophy. [Google Scholar]
- 8.Schon DA. San Francisco: Jossey Bass; 1987. Educating the reflective practitioner. [Google Scholar]
- 9.Henry SG. Recognizing tacit knowledge in medical epistemology. Theor Med Bioeth. 2006;27:187–213. doi: 10.1007/s11017-006-9005-x. [DOI] [PubMed] [Google Scholar]
- 10.Wright SM, Kern DE, Kolodner K, Howard DM, Brancati FL. Attributes of excellent attending-physician role models. N Engl J Med. 1998;339:1986–1993. doi: 10.1056/NEJM199812313392706. [DOI] [PubMed] [Google Scholar]
- 11.Association of American Medical Colleges. Compact between resident physicians and their teachers. http://www.aamc.org/residentcompact. Accessed 2010, February 20. [Google Scholar]
- 12.Lombarts MJ, Bucx MJ, Rupp I, Keijzers PJ, Kokke SI, et al. An instrument for the assessment of the training qualities of clinician-educators. Ned Tijdschr Geneeskd. 2007;151:2004–2008. [in Dutch]. [PubMed] [Google Scholar]
- 13.Lombarts MJ, Bucx MJ, Arah OA. Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology. 2009;111:706–719. doi: 10.1097/ALN.0b013e3181b76516. [DOI] [PubMed] [Google Scholar]
- 14.Lombarts MJMH, Busch ORC, Heineman MJ, Arah O. Ned Tijdschr Geneeskd. 154 A1222. [in Dutch] [Google Scholar]
- 15.Elzubeir MA, Rizk DE. Identifying characteristics that students, interns and residents look for in their role models. Med Educ. 2001;35:272–277. doi: 10.1046/j.1365-2923.2001.00870.x. [DOI] [PubMed] [Google Scholar]
- 16.Boor K, Teunissen PW, Scherpbier AJ, van der VleutenC, van der LandeJ, et al. Residents' perceptions of the ideal teacher – a qualitative study. Eur J Obstet Gynec Reprod Biol. 2008;140:152–157. doi: 10.1016/j.ejogrb.2008.03.010. [DOI] [PubMed] [Google Scholar]
- 17.Ullian JA, Bland CJ, Simpson DE. An alternative approach to defining the role of clinical teacher. Acad Med. 1994;69:832–838. doi: 10.1097/00001888-199410000-00013. [DOI] [PubMed] [Google Scholar]
- 18.Paukert JL, Richards BF. How medical students and residents describe the roles and charateristics of their influential clinical teachers. Acad Med. 2000;75:843–845. doi: 10.1097/00001888-200008000-00019. [DOI] [PubMed] [Google Scholar]
- 19.Sutkin G, Wagner E, Harris I, Schiffer R. What makes a good clinical teacher in medicine? A review of the literature. Acad Med. 2008;83:452–466. doi: 10.1097/ACM.0b013e31816bee61. [DOI] [PubMed] [Google Scholar]
- 20.Wright SM, Carrese JA. Excellence in role modelling: insight and perspectives from the pros. CMAJ. 2002;167:638–643. [PMC free article] [PubMed] [Google Scholar]
- 21.Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156–1161. doi: 10.1001/archinte.152.6.1156. [DOI] [PubMed] [Google Scholar]
- 22.Litzelman DK, Stratos GA, Marriott DJ, Skeff KM. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med. 1998;73:688–695. doi: 10.1097/00001888-199806000-00016. [DOI] [PubMed] [Google Scholar]
- 23.Williams BC, Litzelman DK, Babbott SF, Lubitz RM, Hofer TP. Validation of a global measure of faculty's clinical teaching performance. Acad Med. 2002;77:177–180. doi: 10.1097/00001888-200202000-00020. [DOI] [PubMed] [Google Scholar]
- 24.Streiner DL, Norman GR. Oxford: Oxford University Press; 2008. Health measurement scales: a practical guide to their development and use. 4th ed.431 [Google Scholar]
- 25.Arah OA, ten Asbroek AH, Delnoij DM de Koning JS, Stam P, et al. Psychometric properties of the Dutch version of the Hospital-level Consumer Assessment of Health Plans Survey instrument. Health Serv Res. 2006;41:284–301. doi: 10.1111/j.1475-6773.2005.00462.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Delnoij DM, ten Asbroek AG, Arah OA, de Koning JS, Stam P, et al. Made in the USA: the import of American Consumer Assessment of Health Plan Surveys (CAHPS) into the Dutch social insurance system. Eur J Public Health. 2006;16:652–659. doi: 10.1093/eurpub/ckl023. [DOI] [PubMed] [Google Scholar]
- 27.Damman OC, Stubbe JH, Hendriks M, Arah OA, Spreeuwenberg P, et al. Using multilevel modeling to assess case-mix adjusters in consumer experience surveys in health care. Med Care. 2009;47:496–503. doi: 10.1097/MLR.0b013e31818afa05. [DOI] [PubMed] [Google Scholar]
- 28.Ratanawongsa N, Bolen S, Howell EE, Kern DE, Sisson SD, et al. Residents' Perceptions of Professionalism in Training and Practice: Barriers, Promoters, and Duty Hour Requirements. J Gen Intern Med. 2006;21:758–763. doi: 10.1111/j.1525-1497.2006.00496.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kilminster S, Cottrell D, Grant J, Jolly B. AMEE guide no. 27: effective educational and clinical supervision. Med Teach. 2007;29:2–19. doi: 10.1080/01421590701210907. [DOI] [PubMed] [Google Scholar]
- 30.Maker VK, Curtis KD, Donnelly MB. Are you a surgical role model? Curr Surg. 2004;61:111–115. doi: 10.1016/j.cursur.2003.09.003. [DOI] [PubMed] [Google Scholar]
- 31.Seifert CF, Yukl G, McDonald RA. Effects of multisource feedback and a feedback facilitator on the influence behavior of managers toward subordinates. J Appl Psychol. 2003;88:561–569. doi: 10.1037/0021-9010.88.3.561. [DOI] [PubMed] [Google Scholar]
- 32.Marvel MK. Improving clinical teaching skills using the parallel process model. Fam Med. 1991;23:279–284. [PubMed] [Google Scholar]
- 33.Metheny WP, Espey EL, Bienstock J, Cox SM, Erickson SS, et al. To the point: medical education reviews evaluation in context: assessing learners, teachers, and training programs. Am J Obstet Gynecol. 2005;192:34–37. doi: 10.1016/j.ajog.2004.07.036. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Psychometric Properties of the Five Composite-Scales of the SETQ. Instruments from 19 Medical and Surgical Specialties.
(DOC)