Abstract
BACKGROUND
Awareness of the need for ambulatory care teaching skills training for clinician-educators is increasing. A recent Health Resources and Services Administration (HRSA)-funded national initiative trained 110 teams from U.S. teaching hospitals to implement local faculty development (FD) in teaching skills.
OBJECTIVE
To assess the rate of successful implementation of local FD initiatives by these teams.
METHODS
A prospective observational study followed the 110 teams for up to 24 months. Self-reported implementation, our outcome, was defined as the time from the training conference until the team reported that implementation of their FD project was completely accomplished. Factors associated with success were assessed using Kaplan-Meier analysis.
RESULTS
The median follow-up was 18 months. Fifty-nine of the teams (54%) implemented their local FD project and subsequently trained over 1,400 faculty, of whom over 500 were community based. Teams that implemented their FD projects were more likely than those that did not to have the following attributes: met more frequently (P=.001), had less turnover (P=.01), had protected time (P=.01), rated their likelihood of success high (P=.03), had some project or institutional funding for FD (P=.03), and came from institutions with more than 75 department of medicine faculty (P=.03). The cost to the HRSA wwas $22,033 per successful team and $533 per faculty member trained.
CONCLUSIONS
This national initiative was able to disseminate teaching skills training to large numbers of faculty at modest cost. Smaller teaching hospitals may have limited success without additional support or targeted funding.
Keywords: continuing education, medical education, medical faculty, prospective study
Education is a major mission of U.S. teaching hospitals. To be effective teachers, faculty require diverse skills such as the ability to create a supportive learning environment, assess learners, provide feedback, teach in small groups, lecture, mentor, and develop and evaluate curricula. Such skills can be taught effectively.1–9
Accordingly, the need for teaching skills training for clinician-educators is increasingly recognized.10–19 The Health Resources and Services Administration (HRSA) and other organizations fund local, regional,20 and now national15,21 faculty development activities in teaching skills for internal medicine. Funded by the HRSA (1997–2002), the General Internal Medicine Faculty Development Project: Generalist Faculty Teaching in Ambulatory Settings, led by the General Internal Medicine Generalist Education Leadership Group (GIMGEL), a collaborative group with sponsorship from all major organizations within internal medicine (see acknowledgements), recently completed a national program for faculty development in ambulatory care teaching skills.15,21 This program provided a forum to teach faculty teams recruited from U.S. teaching hospitals how to train and support their community- and hospital-based ambulatory preceptors.
To evaluate the success of these teams, we conducted a prospective follow-up study. Our purpose was to: 1) assess whether these teams implemented local faculty development activities; 2) identify factors associated with successful implementation; 3) describe the characteristics of implemented faculty development projects including the numbers and types of faculty trained; and 4) assess the costs to the HRSA per project implemented and faculty member trained.
METHODS
Skills Development Conference Intervention
Teams of clinician-educators were invited to attend three separate 3-day conferences focused on faculty development in ambulatory teaching skills.15 The GIMGEL program conferences were designed to deliver important content and skills training that would support the goal of having each team implement local faculty development activities to train additional local clinician-educators. Each conference included: 1) plenary presentations by experts in teaching skills; 2) protected time for teams to meet and plan their future faculty development project at their individual institutions; and 3) workshops. The workshops focused on four areas: specific teaching skills; teaching others to teach; recruiting and maintaining community-based preceptors; and designing, implementing, and evaluating faculty development activities. A more detailed description of the content of the conferences has been published previously.21 Conferences were held in Tampa (December 1999), Denver (June 2000), and San Diego (December 2000). Teams were given the opportunity to return to a wrap-up conference in Dallas (December 2001) 12 to 24 months after their initial conference, with the opportunity to present their accomplishments as an oral or poster presentation or a workshop.
Participant Teams
Based on a review of the faculty development and continuing medical education literature,15 a team approach was utilized. We believed training teams would have a greater and more lasting impact on local faculty development efforts. Applicants were required to submit a description of their team and a plan for a faculty development project at their institution. To promote successful implementation of projects and a focus on improving ambulatory and community-based teaching, each faculty development team was asked to include an institutional leader, an educational program leader, and a community-based teacher. Members of the GIMGEL Group and faculty with expertise in teaching skills and faculty development peer reviewed applications to confirm appropriate team composition and that the goal of promoting ambulatory and community-based teaching was met. Participating teams were encouraged to conduct a needs assessment prior to attending the conference. A total of 110 teams of 3 to 5 physicians (478 participants) were funded to attend the conferences.
Study Design
We conducted a prospective observational cohort study to assess the accomplishments of the 110 participating faculty development teams. Each team member completed a questionnaire during the conference (baseline) and these data were aggregated at the team level. The team leader completed follow-up surveys at 6-month intervals from 6 up to 24 months. Teams attending the first conference (Tampa) were followed up for 24 months. All other teams had 18 months of follow-up. The study was exempted from review by the Institutional Review Board at Johns Hopkins Bayview Medical Center.
Baseline and Follow-up Survey Content
Baseline data from team members included: protected time or salary support for the faculty development project, funding for their teaching skills faculty development project, whether a needs assessment had been performed, percentage of time spent in teaching and clinical activities, self-rated likelihood of team success (Likert scale), and prior teaching skills training (assessed using an instrument that listed 24 discrete teaching skills, focused on ambulatory teaching).
Survey data were linked to information about individual teaching hospitals obtained from the Association of Professors of Medicine (APM), the American Hospital Association (AHA), the HRSA, and a previous national survey of the prevalence of faculty development.22 These data identified whether the team's teaching hospital had a primary affiliation with one of the 126 U.S. medical schools (university hospitals) as defined by the APM, rural or urban location, current institutional HRSA funding for teaching skills faculty development in internal medicine, and whether the institution had ongoing faculty development at baseline. Data were available for 387 teaching hospitals, allowing for comparison of the characteristics of those who did and did not participate in the GIMGEL program.
Follow-up surveys collected information on team process issues including team membership turnover, number of team meetings, protected time for faculty working on their faculty development project, receipt of institutional or external funding, salary support, support by supervisors/institutional leaders, identification of dedicated support staff, and instructors with appropriate expertise. Teams reported their progress in implementing and evaluating their faculty development project.
Baseline and follow-up surveys were developed by the evaluation team (JMC, TKH, DEK). Surveys were pilot tested with members of the GIMGEL Group.
Outcomes
Our primary outcome was whether or not the teams reported implementation of their faculty development projects. At each follow-up period, teams responded to the following question: “Please rate the degree to which the following tasks have been accomplished,” with responses on a Likert scale from 1 =not at all accomplished to 5 =completely accomplished. The individual tasks included performed needs assessment, developed objectives, determined intervention strategy/educational methods, developed implementation plans, pilot tested intervention, and, most importantly, “implementation of at least one cycle of your project is complete,” our primary measure of success. This outcome, reported on each follow-up survey, was recoded as a dichotomous variable, positive when the team leader reported that implementation had been completely accomplished. Data from each survey were combined to create a time-dependent dichotomous outcome variable. In addition to the face validity of this measure, implementation was also highly associated with other measures of success including “Please rate the degree to which your group has accomplished the original or revised objectives of your plan” (χ2= 39; P < .001) asked at the end of follow-up, and also with accomplishing the subtasks of developing objectives and pilot testing of the intervention.
Secondary outcomes were assessed for teams that were successful in implementing their faculty development project. For these teams, we assessed the teaching skills topics covered, teaching methods used, and the evaluation techniques used. Other secondary outcomes included the number and type of faculty members trained and any resulting scholarly activities including conference presentations or publications.
Analysis
We used Kaplan-Meier analyses to account for differing amounts of time-to-implementation and the different times of follow-up. We used logrank tests to assess the association of institutional factors (university hospital, number of faculty in the department of medicine, and rural or urban location) and team member factors (number of team members, percent effort in clinical and teaching activities of team members, amount of prior training in teaching skills, and self-rating of the likelihood of success, having dedicated support staff, and instructors with appropriate expertise) with time to successful implementation.
We also used logrank tests to assess whether team process factors within the first 6 months of follow-up were associated with successful implementation. Because process variables (e.g., number of team meetings) achieved in the first 6 months after training and reported on the first outcomes assessment survey at 6 months were strongly correlated with those reported later in follow-up, we used these process variables for all analyses. For example, number of team meetings by 6 months was strongly associated with number of team meetings between 6 and 12 months (χ2= 43; P < .001). In addition to the Kaplan-Meier analyses, sensitivity analyses with χ2 and Pearson's χ2 tests for trend were also used to assess associations as appropriate, with similar results (data not shown). Because many of the factors associated with implementation were highly correlated and had complex interactions with the outcome, we were not able to construct multivariable models.
Open-ended Questions
Final surveys (18 and 24 months) included open-ended questions that asked participants to identify barriers to implementation of their team's faculty development project, benefits gained from involvement in the GIMGEL faculty development project, and how the GIMGEL Faculty Development Project could have better supported their team in accomplishing their project. Three researchers analyzed the open-ended responses (RBL, TKH, DEK). Each independently reviewed all comments to identify themes. Consensus was reached by comparison and discussion. The entire research team checked the analysis for relevance and consistency.
Cost to HRSA
We assessed the cost-effectiveness of the national faculty development conferences from the perspective of the funding agency, HRSA, for this specific project. Total costs were defined as total cost to plan, implement, and evaluate the national faculty development conferences. Effectiveness was defined as cost per successfully implemented project, and also as cost per faculty trained at longest follow-up available. Because we took the perspective of the HRSA, our cost analysis specifically includes the cost of the conferences and the evaluation, but does not include costs to participating institutions nor any cost shifting from other institutional resources or external funded programs. Specifically, the analysis does not include costs that might have been incurred by participating teams’ use of funds from concurrent HRSA institutional grants.
Loss to Follow-up, Missing Data, and Follow-up Times
Multiple attempts were made to contact teams that failed to return follow-up surveys at the defined 6-month intervals and encourage them to respond. At the end of the study, we were able to contact 6 of the 7 teams who had not returned any follow-up surveys. Five of these teams had completely disbanded and had not accomplished their objectives, and 1 team had only partially implemented its project. Thus, for teams with no follow-up, we assumed that they had not implemented at least one cycle of their faculty development project. To assess for bias introduced by this assumption, the analyses were conducted excluding these teams, and the results were similar to those presented in this article.
RESULTS
The overall response rate (at least one survey returned during follow-up) was 94% (103/110), and averaged 80% for each 6-month interval. The average follow-up time was 18 months, with a cumulative total of 1,986 months of follow-up collected for the 110 teams.
Differences Between Participating and Nonparticipating Hospitals
Compared with the 277 teaching hospitals with medicine residency programs that did not participate in the GIMGEL program, the 110 teaching hospitals of the participating faculty development teams were more frequently university hospitals (52% vs 23%; P < .05). Participating hospitals were more likely to have ongoing faculty development (48% vs 34%; P = .03). Also, participating hospitals were more likely to have ever had HRSA funding for faculty development in primary care compared to nonparticipating hospitals (32% vs 23%; P < .05).
Participant and Participating Team Characteristics and Processes
Most teams were from urban teaching hospitals with larger medicine faculty (Table 1). The team members’ mean percent effort in clinical activities was 43% and mean percent effort in teaching activities was 26%. Almost all teams reported some prior training in teaching skills. At the time of their initial national teaching skills conference, 56% percent of the teams believed that they were “highly likely to succeed” (5 on a 5-point Likert scale) in their teaching skills faculty development project.
Table 1.
Baseline Characteristics of the 110 Participating Faculty Development Teams and Their Institutions
| N | % | |
|---|---|---|
| University hospital* | 57 | 52 |
| Nonuniversity hospital | 53 | 48 |
| Larger medicine faculty (> 75) | 66 | 60 |
| Smaller medicine faculty (≤ 75) | 44 | 40 |
| Urban setting (> 500,000 population) | 79 | 77 |
| Suburban/rural setting (< 500,000 population) | 24 | 23 |
| Institution reported ongoing faculty development† | 45 | 48 |
| No ongoing faculty development | 49 | 42 |
| ≤3 team members | 33 | 30 |
| 4+ team members | 77 | 70 |
| Mean time spent in clinical activities by team members | ||
| Less than 40% | 30 | 27 |
| 40% to 55% | 49 | 45 |
| Over 55% | 31 | 28 |
| Mean time team members spent teaching | ||
| Less than 25% | 48 | 44 |
| 25% to 30% | 28 | 26 |
| Over 30% | 34 | 31 |
| Ever had HRSA funding for faculty development in internal medicine at the institution (in past 10 years) | 34 | 32 |
| Never had HRSA funding for faculty development (in past 10 years)‡ | 72 | 68 |
| Concurrent HRSA institutional funding for faculty development (fiscal year 2000) | 6 | 6 |
| No concurrent HRSA funding‡ | 100 | 94 |
| Prior training in teaching skills§ | ||
| High (≥10 skill areas) | 33 | 30 |
| Medium (7 to 9 skill areas) | 45 | 41 |
| Low (≤6 skill areas) | 32 | 29 |
| Team-rated likelihood of success | ||
| Rated high// | 62 | 56 |
| Rated low | 48 | 44 |
University hospital: a teaching hospital that had a primary affiliation with a medical school as defined by the Association of Professors of Medicine.
Data from prior national survey of faculty development, available only for 94 teams.
HRSA funding data available, for 106 teams.
Mean score on 24-item survey of general and specific teaching skills training.
Majority of team rated likelihood of success “extremely likely” (5 on a 5-point scale).
HRSA, Health Resources and Services Administration.
Process Factors During First Six Months
By 6 months of follow-up, about half the teams had conducted a needs assessment (52%). Most (63%) had met more than once to plan their project and two thirds of teams had retained their team membership. A minority of teams reported receiving funding (internal, 29%; or external, 10%), salary support (31%), or protected time (37%) for their faculty development project by 6 months.
Implementation Rate and Factors Associated with Implementation of Faculty Development Projects
By the end of follow-up, 59 (54%) of the 110 teams reported successfully implementing their faculty development project. Cumulative success rates in Kaplan-Meier analysis at 6,12,18, and 24 months were 22%, 36%, 50%, and 54%, respectively.
Factors associated with completely accomplishing implementation included being from a teaching hospital with a large number of faculty (>75), baseline team rating of their likelihood of success high, protected time for team members, more frequent team meetings, and stability of the team (Table 2). In general, the presence of funding (ever having HRSA funding, concurrent HRSA funding, internal or external funding for the project) increased the rate of implementation, but no single factor reached statistical significance. A summary variable combining these sources of funding was significant (logrank, 4.4; P = .04). The greatest difference in success during follow-up was seen between teams that had held two or more meetings by 6 months, compared with those that had held less than two team meeting by 6 months (73% vs 26%; logrank, 15; P < .001).
Table 2.
Factors Associated with Time to Implementation of Faculty Development Projects: Kaplan-Meier Analysis
| Cumulative Kaplan-Meier Survival Function Through Median Follow-up (% Success) | |||||
|---|---|---|---|---|---|
| Follow-up Months | |||||
| N* | 6 | 12 | 18 | Logrank Test χ2 (P Value) | |
| Overall | 110 | 22 | 36 | 50 | |
| Factors significantly (P < .05) associated with implementation | |||||
| 2+ team meetings first 6 months† | 59 | 36 | 53 | 73 | 15 (.001) |
| <2 team meetings | 35 | 1 | 23 | 26 | |
| Team stable† | 64 | 30 | 48 | 64 | 6.5 (.01) |
| Team turnover occurred | 31 | 16 | 26 | 36 | |
| Protected time for faculty to work on teaching skills project† | 35 | 31 | 51 | 71 | 4.8 (.01) |
| No protected time for project | 59 | 20 | 34 | 44 | |
| Team-rated likelihood of success‡ | |||||
| High§ | 62 | 26 | 44 | 61 | 5.0 (.03) |
| Low | 48 | 17 | 27 | 35 | |
| Larger medicine faculty (>75)‡ | 66 | 27 | 39 | 59 | 4.4 (.03) |
| Smaller medicine faculty (≤75) | 44 | 13 | 31 | 36 | |
| Any funding for faculty development†// | 78 | 24 | 41 | 57 | |
| No funding | 32 | 16 | 25 | 38 | 4.4 (.04) |
| Factors not significantly (P≥ .05) associated with implementation | |||||
| Concurrent HRSA institutional funding for faculty development (fiscal year 2000)† | 6 | 50 | 67 | 83 | 3.6 (.06) |
| No concurrent HRSA funding | 100 | 21 | 36 | 48 | |
| HRSA institutional funding for faculty development (past 10 years)† | 34 | 26 | 47 | 59 | 2.4 (.12) |
| No HRSA funding (past 10 years) | 72 | 20 | 33 | 46 | |
| Internal institutional funding† | 71 | 23 | 35 | 48 | 2.2 (.13) |
| No internal funding reported | 39 | 20 | 28 | 38 | |
| External funding for project† | 34 | 21 | 35 | 59 | 0.8 (.4) |
| No external funding reported | |||||
| Target audience needs assessment | 46 | 33 | 43 | 59 | 1.9 (.2) |
| No target audience needs assessment† | 49 | 18 | 39 | 51 | |
| Dedicated support staff obtained† | 66 | 25 | 42 | 64 | 0.6 (.4) |
| No dedicated support staff | 28 | 24 | 39 | 50 | |
| Number of team members over 3‡ | 77 | 22 | 34 | 52 | 0.6 (.4) |
| Number of team members 2 to 3 | 33 | 21 | 42 | 46 | |
| Department of medicine reported ongoing faculty developmentঠ| 45 | 26 | 36 | 51 | 0.36 (.5) |
| No ongoing faculty development | 49 | 22 | 41 | 51 | |
| University hospital‡ | 57 | 23 | 39 | 51 | 0.3 (.6) |
| Nonuniversity hospital | 53 | 21 | 34 | 49 | |
| Salary support for some faculty to work on teaching skills project† | |||||
| Yes | 29 | 23 | 43 | 57 | 0.11 (.7) |
| No | 65 | 28 | 35 | 48 | |
| Support of supervisors/institutional leaders† | 56 | 33 | 44 | 53 | 0.04 (.8) |
| No support of supervisors/institutional leaders | 36 | 20 | 36 | 54 | |
| Urban setting‡ | 79 | 23 | 35 | 49 | 0.02 (.9) |
| Rural setting | 24 | 25 | 46 | 50 | |
| Instructors with appropriate expertise identified† | 80 | 24 | 40 | 54 | 0.01 (.9) |
| Instructors with appropriate expertise not identified | 14 | 28 | 43 | 57 | |
Total N varies slightly due to number of missing values at 6 months.
Reported at 6 months.
Reported or determined at baseline.
Majority of team rated likelihood of success “extremely likely” (5 on a 5-point scale).
Summary of funding: concurrent HRSA funding for faculty development in teaching skills or team's self-report of obtaining internal or external funding.
Data from prior national survey of faculty development,22 available only for 94 teams.
Because teams from smaller departments of medicine (N = 33) were less likely to have implemented their faculty development project in teaching skills, we conducted a secondary analysis to assess factors more prevalent among these teams that may have mediated this difference. We noted that teams from smaller teaching hospital departments of medicine less often reported internal funding for faculty development (50% vs 74%; χ2= 6.8; P = .009) and were less likely to report protected time for faculty (15% vs 49%; χ2= 9.0; P = .003). Perceived likelihood of success was also lower (39% vs 64%; χ2= 5.4; P = .02).
Accomplishments of the 59 Teams that Reported Implementation of Faculty Development Projects
Teams reported the number of participants in their faculty development projects within defined ranges. The median range of participants trained by the 59 teams that completely accomplished implementation of their faculty development project was 10 to 19 at the longest follow-up available. Nine teams (15%) reported training over 50 individuals. By taking the midpoint of each team's reported range of participants trained, we estimate that approximately 1,415 people were trained. Teams reported that approximately 38% (N = 537) of these trained faculty were community-based teachers.
The median number of teaching skills taught was 4, and the most common teaching skills among the 59 successful teams were outpatient precepting skills (85%), general teaching principles (71%), giving feedback (71%), evaluating learners (64%), teaching in the presence of the patient (44%), and teaching evidence-based medicine (31%).
Over one third (N = 22) of the implemented faculty development projects included only lectures or small group discussions. However, 35 projects included experiential learning methods such as role-play, standardized patients, or standardized learners. Performance assessment by audio/video review or observation and feedback of real teaching encounters was included in 14 projects. Evaluation of the impact of the 59 implemented projects was limited, with 32 projects using participant self-assessments only, 2 projects giving knowledge tests, and 25 projects (42%) using ratings by learners, colleagues, instructors, or independent raters to evaluate the teaching skills of the trained participants. Twenty-five projects had a pre-post assessment, and, of these, 6 reported using a control group.
Academic Products of the 59 Successful Projects
Of the 59 successful projects, 24 teams (41%) had presented their project at their institution. Fifteen (25%) had presented at a professional conference and/or had a published abstract. Four teams had published manuscripts, two in peer-reviewed journals.3,23
Open-ended Responses
Major themes derived from analysis of open-ended survey questions are summarized in Table 3. Two major barrier themes were identified that mirrored the quantitative results: competing time demands (45 comments) and lack of adequate funding (21 comments). A unique theme was change as a barrier to project planning and implementation (18 comments). In addition to problems of turnover in their own team membership, team leaders noted major changes at the institutional level (“major institutional upheaval”) and departmental and residency program level (“major program reorganization”) that made completion of their project more difficult. Many of the benefits of the GIMGEL conferences related to the ability to have protected time for team building, meeting with experts, and networking.
Table 3.
Qualitative Themes Related to Benefits of GIMGEL Conferences and Ways GIMGEL Could Have Better Supported the Team Faculty Development Efforts
| Major Theme (Number of Comments |
|---|
| I. What barriers did you encounter while developing your faculty development project? |
| Competing time demands (examples: clinical and administrative responsibilities; lack of protected time for planning; limited ability of community-based faculty to participate) (45) |
| Lack of adequate funding (examples: funding for the project team; lack of specific funding for administrative support, travel; failure to obtain Title VII funding) (21) |
| Change as a barrier to project planning and implementation (examples: in addition to problems of turnover in their own team membership, team leaders noted major changes at the institutional level (“major institutional upheaval”) and departmental and residency program level (“major program reorganization”) (18) |
| Lack of motivation or interest in faculty development (examples: poor motivation or interest in improving teaching skills among 1) faculty development team members, 2) institutional leadership, and 3) the targeted faculty teachers) (13) |
| II. What is the most important benefit you, your team, or your institution has obtained from participating in the GIM faculty development project? |
| Access to resources/materials for faculty development (25) |
| Creation of a faculty development project by team (23) |
| Opportunity to network (20) |
| Increased awareness of the importance of faculty development and improving teaching skills (15) |
| Protected time for team building (14) |
| Increased enthusiasm for faculty development (13) |
| III. How could the GIM Faculty Development Project have better supported you in accomplishing your project? |
| Establishing better follow-up with a facilitator for continued coaching over time (14) |
| Providing additional funding through grants (12) |
| Creation of a national clearinghouse of faculty development resources (examples: website, email listserve) (9) |
GIM, general internal medicine.
Cost to HRSA
The total cost invested to plan, implement, and evaluate the GIMGEL teaching skills faculty development project was $1,300,000. Thus, the cost per team trained was $11,818 ($1,300,000 ÷ 110) and the cost per completely implemented faculty development project was $22,033 ($1,300,000 ÷ 59). This is a conservative estimate assuming that teams that did not completely implement at least one cycle of their project within the follow-up period would never be successful or train any faculty. In comparison, the HRSA, in fiscal year 2000, supported faculty development efforts in general medicine, general pediatrics, or primary care at 45 separate teaching hospitals at a mean cost per institution of $221,254 (data provided by the HRSA).
Based on our estimate of 1,415 participants trained by the successful teams, the cost to the HRSA through the GIMGEL program per individual trained by the successful teams was $918. Again, this is a conservative estimate. In fact, an additional 21 faculty development teams that had partially, but not completely, accomplished implementation of their teaching skills faculty development project reported that they had trained participants. Based on the reports of these 21 teams, approximately 580 additional faculty were trained. If these faculty are included, the cost per trained faculty was $652 ($1.3 million/1,995 faculty). If the 443 team members who participated in the national skills development conferences are included, the cost per trained faculty becomes $533. The estimated costs are also conservative in that we assumed that the responding teams ceased (rather than continued) training faculty at the end of the follow-up period.
DISCUSSION
A recent national survey reported that only 39% of teaching hospitals with internal medicine residencies have ongoing faculty development activities for teaching skills.22 A prior study of a national dissemination model prospectively followed 12 individual facilitators who participated in the Stanford Faculty Development Program, and demonstrated that the trained facilitators were able to train additional faculty at their own teaching hospitals.10 Our study adds a better understanding of the factors associated with dissemination and suggests that the GIMGEL program model is low cost when compared with current HRSA expenditures for faculty development initiatives funded through individual institutions. These data should be of interest to clinician-educators, faculty development coordinators, and funding agencies.
Just over half of the teams attending these faculty development training conferences implemented a local project to increase teaching skills at their hospital within an average of 18 months of follow-up. Our findings not only quantify the success of the teams in implementing activities at their home institutions, but also suggest that future efforts to increase teaching skills using dissemination models should emphasize factors we found to be strongly associated with implementation, such as team meetings, team stability, protected time for team members, and some sort of funding directed toward faculty development. Unfortunately, as many of the factors associated with implementation were highly correlated and had complex interactions with the outcome, we were unable to construct multivariable models to clarify the independent effect of each factor. Also, as with any observational study, the associations we identified cannot be assumed to be directly causal. In fact, many of these factors associated with success may simply be surrogates for a general commitment to the importance of the teaching program by the department or institution, expressed both by the leadership and the selection and support of faculty members to participate.
The GIMGEL project specifically targeted community-based teachers. These teachers may have faculty development needs that are even greater than their teaching hospital-based peers.24 Of note, the 110 teams reported training over 500 community-based teachers through their projects. The program focus on community-based teaching and requirement that participating faculty development teams include a community-based teacher likely facilitated the inclusion of community-based teachers in the resulting local faculty development projects.
Teams from smaller departments of medicine were less likely to have participated in the GIMGEL Faculty Development project and, if they participated, were less likely to have been successful in their faculty development efforts. Because many of these smaller departments are from nonuniversity teaching hospitals, the motivation of faculty to initiate and participate in faculty development initiatives may be different. The unique needs of these smaller departments should be explored, and additional strategies for increasing faculty development in teaching skills that target these smaller departments may need to be developed.
Successful dissemination of innovative teaching practices follows an expected pattern: new teaching behaviors are introduced by prominent opinion leaders, the new behaviors are adopted at an accelerating rate, and they either stabilize or decline depending upon their functional value.25 We have only followed these teams through completion of one cycle of their faculty development projects. Thus, the long-term sustainability of these faculty development efforts is unknown. Additional infrastructure changes at the institutional level that empower faculty development leaders over time may be necessary to sustain faculty development.19 Specifically, supporting protected time for more frequent team meetings, a factor correlating with our teams’ success, may help enhance local faculty development initiatives.
Our study has several limitations. First, our measure of successful implementation was by self-report. This measure was associated with other measures of success (e.g., accomplishing project objectives), as indicated in the methods above. Second, although we were able to describe the common content, methods, intensity, and numbers trained through the faculty development activities, we have no data on the actual quality of teaching or learner outcomes of the teams’ interventions, and no data that allow us to compare these teams’ accomplishments to the accomplishment of other faculty development programs, such as those conducted by institutions individually funded by the HRSA. The processes and accomplishments may have been more superficial and less educationally sound, and therefore less successful in terms of teacher and learner outcomes, as compared with individually funded, more intense interventions. Third, we were unable to demonstrate the independent association of factors with the outcome because of the small sample of teams and the complex interactions of the individual factors. Fourth, we assessed costs exclusively from the perspective of the funding agency, and did not assess costs to the participants’ institutions in terms of direct support, provision of protected time for faculty, or other forms of cost shifting of institutional resources. Strengths include a high response rate and relatively long duration of prospective follow-up.
The lack of stability of many teams, and institutional changes, identified in our open-ended questions at the end of follow-up, created serious challenges for many faculty development teams. Several teams noted upheavals at every level of the institutional infrastructure, suggesting an underlying instability in academic medicine.
Continued external longitudinal support and networking of teams, which were not accomplished in this national program, might enhance the subsequent success of local teams. For example, online “collaboratories” are currently being used to support research and could potentially be used to assist teams with problem solving during planning, implementation, and maintenance of faculty development initiatives.26
Additional research and development is needed to further assess 1) team member, team, and external factors that contribute to success; 2) the relative efficacy of different models of faculty development; 3) the quality of local faculty development initiatives; and 4) the impact of faculty development programs on long- as well as short-term improvements in teacher skills, learner knowledge and performance, and patient outcomes.
Acknowledgments
This work was supported in part by funding from HRSA contract 240-97-0044 “Faculty Development for General Internal Medicine: Generalist Faculty Teaching in Community-based Ambulatory Settings” made to the Association of Professors of Medicine, and by the following federal (DHHS) fellowship training grants: NHLBI 2T32 HL07180 (JMC), HRSA 5T32 PE 10025 (TKH), and HRSA 6D14HP00049-02 (RBL).
The General Internal Medicine Generalist Education Leadership (GIMGEL) Group, which designed and implemented the faculty development program, included representatives from the American College of Physicians-American Society of Internal Medicine, the Association of Professors of Medicine, the Association of Program Directors in Internal Medicine, the Association of Subspecialty Professors, the Clerkship Directors in Internal Medicine, and the Society of General Internal Medicine. (Individual Members of the GIMGEL Group are listed in Appendix A, available online at http://www.jgim.org).
REFERENCES
- 1.Skeff KM. Evaluation of a method for improving the teaching performance of attending physicians. Am J Med. 1983;75:465–70. doi: 10.1016/0002-9343(83)90351-0. [DOI] [PubMed] [Google Scholar]
- 2.Rost K, Gordon GH. The teacher simulation exercise: changes in physician teaching emphasis and strategy. The SGIM Task Force on the Medical Interview. J Gen Intern Med. 1989;4:121–6. doi: 10.1007/BF02602351. [DOI] [PubMed] [Google Scholar]
- 3.Furney LS, Orsini AN, Orsette KE, Stern DT. Teaching the one-minute preceptor. A randomized controlled trial. J Gen Intern Med. 2001;16:620–4. doi: 10.1046/j.1525-1497.2001.016009620.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Steinert Y, Nasmith L, McLeod PJ, Conochie L. A teaching scholars program to develop leaders in medical education. Acad Med. 2003;78:142–9. doi: 10.1097/00001888-200302000-00008. [DOI] [PubMed] [Google Scholar]
- 5.Hewson MG, Copeland HL, Fishleder AJ. What's the use of faculty development? Program evaluation using retrospective self-assessments and independent performance ratings. Teach Learn Med. 2001;13:153–60. doi: 10.1207/S15328015TLM1303_4. [DOI] [PubMed] [Google Scholar]
- 6.Teherani A, Hitchcock MA, Nyquist JG. Longitudinal outcomes of an executive-model program for faculty development. Acad Med. 2001;76(10 suppl:):S68–S70. doi: 10.1097/00001888-200110001-00023. [DOI] [PubMed] [Google Scholar]
- 7.Gruppen LD, Frohna AZ, Anderson RM, Lowe KD. Faculty development for educational leadership and scholarship. Acad Med. 2003;78:137–41. doi: 10.1097/00001888-200302000-00007. [DOI] [PubMed] [Google Scholar]
- 8.Hewson MG, Copeland HL. Outcomes assessment of a faculty development program in medicine and pediatrics. Acad Med. 1999;74(10 suppl):S68–S71. doi: 10.1097/00001888-199910000-00043. [DOI] [PubMed] [Google Scholar]
- 9.Hewson MG. A theory-based faculty development program for clinician-educators. Acad Med. 2000;75:498–501. doi: 10.1097/00001888-200005000-00024. [DOI] [PubMed] [Google Scholar]
- 10.Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156–61. doi: 10.1001/archinte.152.6.1156. [DOI] [PubMed] [Google Scholar]
- 11.Litzelman DK, Stratos GA, Marriott DJ, Skeff KM. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med. 1998;73:688–95. doi: 10.1097/00001888-199806000-00016. [DOI] [PubMed] [Google Scholar]
- 12.Litzelman DK, Westmoreland GR, Skeff KM, Stratos GA. Factorial validation of an educational framework using residents' evaluations of clinician-educators. Acad Med. 1999;74(10 suppl):S25–S27. doi: 10.1097/00001888-199910000-00030. [DOI] [PubMed] [Google Scholar]
- 13.Litzelman DK, Stratos GA, Skeff KM. The effect of a clinical teaching retreat on residents' teaching skills. Acad Med. 1994;69:433–4. doi: 10.1097/00001888-199405000-00060. [DOI] [PubMed] [Google Scholar]
- 14.Skeff KM, Stratos GA, Bergen MR, Sampson K, Deutsch SL. Regional teaching improvement programs for community-based teachers. Am J Med. 1999;106:76–80. doi: 10.1016/s0002-9343(98)00360-x. [DOI] [PubMed] [Google Scholar]
- 15.Bowen JL, Alguire P, Tran LK, et al. Meeting the challenges of teaching in ambulatory settings: a national, collaborative approach for internal medicine. Am J Med. 1999;107:193–7. doi: 10.1016/s0002-9343(99)00227-2. [DOI] [PubMed] [Google Scholar]
- 16.Anderson WA, Carline JD, Ambrozy DM, Irby DM. Faculty development for ambulatory care education. Acad Med. 1997;72:1072–5. doi: 10.1097/00001888-199712000-00017. [DOI] [PubMed] [Google Scholar]
- 17.DeWitt TG. Faculty development for community practitioners. Pediatrics. 1996;98(6 pt 2):1273–6. [PubMed] [Google Scholar]
- 18.Irby DM, Ramsey PG, Gillmore GM, Schaad D. Characteristics of effective clinical teachers of ambulatory care medicine. Acad Med. 1991;66:54–5. doi: 10.1097/00001888-199101000-00017. [DOI] [PubMed] [Google Scholar]
- 19.Wilkerson L, Irby DM. Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med. 1998;73:387–96. doi: 10.1097/00001888-199804000-00011. [DOI] [PubMed] [Google Scholar]
- 20.Fogarty J, First LR, Levine M, Reardon M, Magrane D. The Interdisciplinary Generalist Curriculum Project at the University of Vermont College of Medicine: The Vermont Generalist Curriculum (VGC) Experience. Acad Med. 2001;76(4 suppl):S127–S130. doi: 10.1097/00001888-200104001-00025. [DOI] [PubMed] [Google Scholar]
- 21.Crist TB, Clayton CP. Generalist faculty teaching in community-based settings: an interim report on the General Internal Medicine Faculty Development Project. Am J Med. 2001;111:588–92. doi: 10.1016/s0002-9343(01)00975-5. [DOI] [PubMed] [Google Scholar]
- 22.Clark JM, Houston TK, Kolodner K, Branch WT, Levine RB, Kern DE. Teaching the teachers: national survey of faculty development in departments of medicine of U.S. teaching hospitals. J Gen Intern Med. 2004;19:205–14. doi: 10.1111/j.1525-1497.2004.30334.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Green ML, Gross CP, Kernan WN, Wong JG, Holmboe ES. Integrating teaching skills and clinical content in a faculty development workshop. J Gen Intern Med. 2003;18:468–74. doi: 10.1046/j.1525-1497.2003.20933.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Houston TK, Ferenchick GS, Clark JM, et al. Faculty development needs: comparing community-based and hospital-based internal medicine teachers. J Gen Intern Med. 2004;19:375–9. doi: 10.1111/j.1525-1497.2004.30619.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bandura A. Englewood Cliffs, NJ: Prentice-Hall; Social Learning Theory. [Google Scholar]
- 26.Schleyer TK. Collaboratories: leveraging information technology for cooperative research. J Dent Res. 2001;80:1508–12. doi: 10.1177/00220345010800060601. [DOI] [PubMed] [Google Scholar]