Impact and Application of Material Learned in a Pharmacy Residency Teaching Certificate Program

Abstract

Objective. To describe the impact and application of material learned in a pharmacy resident teaching certificate program on the career experiences of alumni 1 to 11 years after completion of the program.

Design. A teaching certificate program was established in 2001 that brought together residents from various training programs throughout Wisconsin to discuss essential educational skills in a dynamic learning environment. The purpose of the program was to teach participants the fundamental skills to continue to develop as a pharmacy educator throughout their career.

Assessment. An electronic survey instrument was sent to alumni of the program. Greater than 70% of respondents agreed that the teaching certificate program reinforced their desire to teach in practice and that the program helped qualify them for their current or previous practice position. Alumni in academic positions more strongly agreed that the program changed their career interest to include academia and qualified them for their position in academia.

Conclusions. A teaching certificate program can reinforce or stimulate interest among pharmacy residents in pursuing an academic career and prepare them for this role. Completion of the program led to a high level of confidence among the majority of alumni in their ability to precept students and residents and influenced some alumni involved in the hiring of pharmacists.

INTRODUCTION

Teaching certificate programs for pharmacy residents were first described by Romanelli and colleagues in 2001 at the University of Kentucky, and numerous programs developed across the country in subsequent years.¹ These programs stemmed from a need to introduce residents to the fundamental elements of effective teaching, to provide them with opportunities to develop teaching-related skills, and to more adequately prepare them for future careers in academia.^2,3 Teaching certificate programs provide pharmacy residents with more teaching and precepting experiences, increase residents’ confidence in their teaching abilities, and aid in obtaining employment.^4-7 Furthermore, exposing residents to various activities within academia may stimulate their interest in pursuing this pharmacy career path.^8,9 Teaching certificate programs may also serve as a recruitment tool to help fill the estimated 27% of vacant faculty positions that remain unfilled because of a lack of qualified applicants.¹⁰

While TCPs exist across the country, they lack standardization in content and structure. While previous assessments have provided insight into the perceived benefits of TCPs for pharmacy residents, little is known about how professionals actually incorporate their teaching education into their professional functions. Thus, the purpose of our study was to describe the impact and application of a teaching certificate program on the career paths of pharmacy residents up to 11 years after completing the program, and to assess the impact of the components of the program.

DESIGN

The University of Wisconsin-Madison Teaching Certificate Program (UW TCP) was established in 2001 as a formal elective experience designed to expose pharmacy residents to a wide variety of activities related to pharmacy education and the external forces that impact the educational process. The program introduced pharmacy residents to multiple aspects involved in teaching, including both classroom and clinical instruction.

Pharmacy residents in the greater Madison and Milwaukee geographic regions were invited to participate in the program. Priority was given to those residents with teaching responsibilities in the University of Wisconsin-Madison School of Pharmacy (UWSOP) pharmacotherapy laboratory and to residents in programs that have affiliation agreements with the UWSOP. The program, funded by monetary contributions from program participants, costs $100 per participant. The UW TCP has expanded from its initial enrollment of 19 participants (with 8 sessions and 3 workshops) in 2001 to 29 participants in 2013 (with 15 sessions). As of 2013, there were 3 program coordinators, 1 UW faculty member, and 2 previous teaching certificate program participants who served as clinical instructors for pharmacy practice sites in the region in addition to teaching as a significant responsibility of their positions. Other UW faculty members were involved in the teaching certificate program as guest lecturers or by serving as preceptors or mentors for other teaching-related experiences.

The UW TCP incorporated background readings into activities and discussions, which were covered in multiple sessions throughout the residency year. The sessions were more heavily concentrated in the fall than in the spring semester to build a foundation of core educational topics prior to the residents’ engagement in teaching experiences. This “front loading” allowed participants opportunities to transfer their learning to teaching practice and to accommodate their increasing residency workload as the year progressed. Sessions were attended by residents, invited guests, and the program coordinators. Participants earned a certificate of completion by actively participating in 80% of the sessions, undertaking a variety of teaching experiences, and successfully preparing a teaching portfolio.

Each session of the UW TCP was designed, implemented, and led by a pair of pharmacy residents serving as the discussants. The session time was structured into thirds with an emphasis on topic content delivery, an active-learning component, and a guest speaker. Most of the UW TCP sessions were 1½ hours. Discussants prepared a session lesson plan and developed necessary teaching materials with mentorship from an assigned program coordinator.

Additionally, each resident was required to create a teaching portfolio (electronic or hard copy). The portfolios were reviewed by a minimum of 4 individuals (fellow participant, site preceptor, and 2 assigned TCP coordinators). The teaching portfolio provided the resident with a professional, documented record of their education and teaching-related experiences. It also served as documentation of the resident’s experiences throughout the year to determine certificate eligibility. Participants used the portfolio to reflect, examine, and improve upon their work. Their portfolios also illustrated the essence of their work, values, and beliefs in constructing curriculum, instruction, and assessment. Portfolio requirements included a teaching philosophy, short and long-term teaching goals, at least 7 reflections on teaching-related experiences, and 2 peer reviews. The peer-review process created an opportunity for participants to critically evaluate other instructors’ teaching styles and methods to either adopt or avoid their teaching techniques. One of the UW TCP topics, “Assessing Your Teaching Performance,” guided participants in understanding and developing their peer-review process.

EVALUATION AND ASSESSMENT

An anonymous electronic survey instrument was distributed to pharmacy resident alumni who had completed the TCP. This cross-sectional observational study served as a quality measure for the TCP, and was granted exemption from the University of Wisconsin Health Sciences Institutional Review Board.

To obtain current e-mail contact information for TCP alumni, an e-mail query was sent to pharmacy residency directors. Alumni of the program were then surveyed using a Web-based survey instrument (Qualtrics, Provo, UT) in January 2013. Informed consent was embedded in the e-mail request sent to all potential participants. Following the initial invitation to participate in the survey, subjects were e-mailed again 2 and 4 weeks after the initial invitation was sent. The survey items, based on core programmatic skills, were developed by 2 of the study investigators and then reviewed by co-investigators to establish content validity. The survey instrument was developed to: (1) collect demographic and current employment data on 11 years of past participants, (2) determine the type of teaching experiences encountered by participants in current and past career positions, (3) assess overall confidence in teaching-related skills (9 items, 5-point scale), (4) evaluate the application and use of material covered in the teaching certificate program to participants’ current institution, (5) determine differences in applicability and use of the covered material between various pharmacy careers, (6) assess the perceived influence of completion of a teaching certificate program on career procurement from both the employee and employer perspective, and (7) evaluate the necessity of TCP aspects to teaching development.

Participants were eligible for inclusion if they had earned a certificate of completion from the University of Wisconsin-Madison. Alumni had to have finished residency training in order to assess the application of the program to their careers. They also needed a valid e-mail address for the survey instrument distribution method.

Data were analyzed using descriptive statistics generated by the Qualtrics software. The Mann-Whitney U test was applied to examine the relationship between those with an academic commitment to perceptions of the TCP. The Kruskall-Wallace test was used to examine the relationship between the number of hours spent per week on teaching-related activities and teaching-related confidence. Data analysis was completed with the statistical software R, version 2.15.1 (The R Foundation for Statistical Computing, Vienna, Austria).

From 2002 to 2012, 231 participants completed the TCP. Eight alumni who were continuing their education in specialty residencies were excluded from the analyses as they were unable to assess the impact the teaching certificate program had on their career. Thirty-eight alumni without valid e-mail addresses in the database also were excluded. Of the 185 participants eligible for inclusion in the study, 95 alumni completed the survey instrument in its entirety, for a cumulative response of 51%. The majority (75%) of respondents who completed the survey instrument were female, which was representative of TCP participants. At the time of survey instrument completion, 36% and 39% of respondents reported a primary or secondary career commitment as ambulatory care or clinical hospital pharmacists, respectively. The majority of the alumni practiced in either of these 2 settings, similar to the pharmacy residency programs affiliated with the TCP annual enrollments. Those with a primary or secondary academic position, defined as having part of their salary paid by the college or university, represented 15.8% of the study population. Other demographic data are provided in Table 1.

Table 1.

Demographics of Alumni of the University of Wisconsin-Madison Teaching Certificate Program Who Responded to a Survey on the Value of the Program (N=95)

The majority of respondents had a current role in precepting pharmacy students (72%) or pharmacy residents (63%), while relatively few (9%) precepted other healthcare disciplines. Approximately 41% were active in leading small group discussions or laboratories, while slightly fewer participated in classroom lectures (30%), or continuing education sessions or workshops (32%). Only 3% were not active in any teaching activities.

Respondents’ confidence in teaching abilities 1 to 11 years after completion of the TCP was also assessed (Table 2). Approximately 70% of respondents felt more than moderately confident to highly confident in their overall teaching abilities to precept, write learning objectives, lecture to small groups, and provide effective feedback. Respondents were slightly less confident in their abilities to prepare a lesson plan, lecture to large groups, and write effective examination questions; however, the majority still felt moderate to moderately high confidence in these abilities.

Table 2.

Confidence in Teaching Abilities of Participants One to Eleven Years After Completion of the University of Wisconsin-Madison Teaching Certificate Program^a (N=95)

Respondents were asked to consider how the UW TCP affected their careers by reporting their level of agreement with survey items (Table 3). The majority of respondents (88%) reported a low to moderate level of agreement that participating in the TCP changed their career interests to include a teaching or precepting component. This low level of agreement was likely because of participants having a desire to teach or precept upon entering the TCP, which was reflected by the majority of respondents (80%) who reported a moderate to high level of agreement that the TCP reinforced their desire to teach in practice. Alumni reported a moderate to high level of agreement that the program helped qualify them for their current or previous practice positions (72%), and that they placed a higher value on the UW TCP since being out in practice (84%). Respondents most strongly agreed with the item concerning their ability to apply knowledge and skills learned in the program to their workplace. When comparing the level of agreement with how the UW TCP influenced career positions, there were significant differences among those alumni who went into academia as compared with those alumni who did not pursue academic careers. Alumni holding an academic position had a higher level of agreement that the TCP changed their career interest to include academia (p<0.001) and qualified them for their position in academia (p<0.001). Those in academic careers also had a higher level of agreement that they applied the knowledge and skills learned in the TCP in their faculty position compared to their colleagues who did not enter academia (p<0.001). Given the need to fund the program through participant fees, past participants were asked to consider the monetary value they associated with the UW TCP. Greater than one third (33.7%) of respondents felt $100 was an appropriate monetary value and greater than 86% of respondents reported the value of the program was between $100 and $250 (Table 4).

Table 3.

Level of Agreement of Participants with Career Impact of the University of Wisconsin-Madison Teaching Certificate Program^a (N=95)

Table 4.

Monetary Value Associated with the University of Wisconsin-Madison (UW) School of Pharmacy Teaching Certificate Program (N=95)

The majority of respondents felt that learning to teach through the hands-on component of creating and delivering the teaching session as the discussant (77.9%), developing a teaching philosophy (63.2%) and teaching portfolio (67.4%), and discussing teaching topics with fellow residents (87.3%) were probably or definitely necessary to engagement in teaching development (Table 5). There were no differences in the perceived necessity of creating and delivering the teaching session as the discussant (p=0.554), developing a teaching philosophy (p=0.067), developing a teaching portfolio (p=0.195), and teaching with fellow residents (p=0.910) among those alumni who pursued academic careers compared to those who did not.

Table 5.

Necessity of Teaching Certificate Program Components in Teaching Development (N=95)

While the majority of alumni felt a teaching portfolio was necessary for engagement in teaching development, only 13% maintained a portfolio, with most participants citing lack of time and lack of need as barriers to updating. However, the majority of alumni (86%) have continued to reflect upon their teaching despite not maintaining a current portfolio. This may reflect having developed and refined self-assessment skills through the creation of the original portfolio. The teaching portfolio was used for interview purposes by approximately 27% of alumni and 41% shared it as an example for others.

Respondents were asked to rank the top 5 most valuable teaching-related topics covered by the UW TCP; unweighted results are reported. Alumni most valued the following topics since being out in practice: providing effective feedback (n=78), designing and implementing an effective practice experience (n=64), writing objectives and outcomes (n=52), facilitating classroom learning/active learning (n=51), and assessing student learning using performance-based assessments (n=37) (Table 6).

Table 6.

Top-Ranked Topics for the University of Wisconsin Teaching Certificate Program^a (N=95)

Approximately 46% of alumni reported having input into the hiring process at their facility. Of those respondents, 57% (n=25) agreed that completion of a teaching certificate program positively influenced their decision during the hiring process for new employees at their facility. Thirty-four percent (n=15) were neutral on the topic, while only 3 either disagreed or strongly disagreed that teaching programs influenced the hiring process.

Regarding current involvement in teaching certificate programs, 26% of respondents reported current or previous participation as a guest speaker (52%), program coordinator (24%), and/or portfolio reviewer (72%). Six percent of alumni established a teaching certificate program elsewhere after completing the UW program. Approximately 54% of respondents were somewhat or strongly interested in pursuing future involvement with a teaching certificate program. Only 12% of alumni reported being somewhat or strongly disinterested in future involvement in a teaching certificate program.

DISCUSSION

Teaching certificate programs increase participants’ confidence in their teaching abilities and may stimulate participants’ interest in a career in academia. The intent of this study was to assess the impact of a pharmacy resident teaching certificate program and application of material learned on the career experiences of alumni up to 11 years after completion of the program. We found our alumni to be highly engaged with teaching activities across multiple fields of pharmacy. Similar to previous observational studies, our alumni were highly confident in many teaching aspects, with the highest confidence levels in precepting, writing learning objectives, leading small group discussions, and providing effective feedback. This is likely because of the wide applicability of these teaching skills across pharmacy settings. Lower confidence levels were found regarding lecturing to large groups and writing examination questions, which matches annual preprogram and postprogram analysis of UW TCP participants. Overall, the confidence data regarding this teaching certificate program are highly encouraging as colleges and schools of pharmacy move toward more heavily relying on practice sites for education to complement classroom teaching.

This study showed that teaching certificate programs reinforce the desire to teach across pharmacy practice sites and influence career interests to include a teaching component for those entering academia. There was a high level of agreement that the skills and knowledge learned in the teaching certificate program were applicable in their workplace, with a significantly higher level of agreement among those in academia. This shows the potential of teaching certificate programs as a recruitment tool by colleges and schools of pharmacy to help fill vacant faculty positions with qualified candidates.

Similar to previous literature, this study found that alumni of the UW TCP felt completion of the program aided in the procurement of their positions. However, no previous studies have assessed the influence of TCPs on employment from the perspective of the employer. Approximately half of the TCP alumni with hiring input at their facilities felt completion of a TCP positively influenced their decision-making during the hiring process.

Considering the role of teaching certificate programs for developing teaching-related skills and their potential influence on participants’ academic careers, it is important to reflect upon the content and structure of the UW TCP as there is a current lack of standardization across teaching certificate programs.¹¹ While the American College of Clinical Pharmacy (ACCP) Task Force on Residencies set forth guidelines describing the minimum components necessary for a teaching certificate program, not all teaching certificate programs have adopted these criteria into their programs.¹² The UW TCP structure closely mirrors the ACCP Task Force recommended criteria with regards to creation of a teaching philosophy and portfolio, active participation in interactive seminars, and exposure to a wide variety of teaching experiences. While the UW TCP did not specifically assign teaching mentors through the teaching certificate program itself, the majority of participants had academic mentorship through UW SOP pharmacotherapy laboratories. Additionally, the 3 program coordinators of the program serve as teaching mentors for the residents while also serving their role as discussants. The majority of our alumni felt that all program components were probably or definitely necessary to helping engage them in the teaching development process. By allowing residents to serve in the discussant role, they were given a first-hand opportunity to learn through teaching. This direct approach engaged the participants and increased confidence and abilities, as seen in several years of postsurvey data for respective classes. While creation of a teaching portfolio was felt necessary, few alumni maintained a current portfolio because of time constraints and more pressing responsibilities. Despite this, alumni continue to use the reflective skills learned through the process of portfolio development to examine and improve their teaching skills. The most highly regarded topics discussed in the UW TCP centered around precepting skills, which may be attributed to the high applicability of precepting across settings.

Because of participants’ time constraints and distance from campus, at least 1 program has restructured their teaching certificate program to a condensed 2-day conference.¹³ This study did not evaluate alumni’s perception of conference formatting and this may be an area for future evaluation. Given the proximity of residencies to the UW TCP seminar site, attendance remains high with the current format, which also allows residents to serve as discussants and develop session materials with mentorship from a program coordinator. This would not be feasible with a 2-day conference format.

The limitations of this study should be considered when evaluating the results. There was potential for self-selection bias; alumni who had either a positive or negative experience may have been more likely to respond than those with a neutral experience. Also, there was no comparator group to evaluate career choices and teaching perceptions between residents who had participated in a teaching certificate program and those who had not; this is a continued area of interest and future direction. Additionally, survey questions were not validated prior to use.

SUMMARY

This study, which is the largest study done on a teaching certificate program, with inclusion of 11 years of participants and an appropriate survey response rate, showed that the skills taught during a teaching certificate program could benefit not only those considering a career in academia but also those who go on to precept both students and residents. Teaching certificate programs may help stimulate or reinforce an interest in pursuing academic careers and influence the hiring process from the perspective of both the employee and the employer.