The Status and Adequacy of Preceptor Orientation and Development Programs in US Pharmacy Schools

Abstract

Objective. To identify current preceptor orientation and development programs at US colleges and schools of pharmacy and propose future initiatives for preceptor programs.

Methods. An anonymous 28-item survey was administered in January 2017 to 128 experiential education personnel at accredited US schools and colleges of pharmacy. Data from completed survey instruments were tabulated and qualitative responses to open-ended questions were examined using thematic analysis.

Results. Eighty-five experiential education administrators participated in the survey (response rate=67%). Most preceptor orientation programs met the majority of requirements as outlined within the Accreditation Council for Pharmacy Education’s Standard 20.3, although only 42% of programs mandated preceptor orientation prior to student placement. Two-thirds of respondents offered annual, live preceptor development, and 75% of programs used commercially available online products. Nearly 40% of respondents collaborated with other schools or professional organizations to offer preceptor training. Only 29% of programs had specific requirements for pharmacists to maintain their active preceptor status. Seventy percent of respondents reported spending over $2500 and 39% over $5000 annually on preceptor development. Programs with the highest monetary investment (>$10,000/year) in preceptor development offered multiple venues (live and online) for preceptor training. Programs with significant personnel commitment (≥0.5 FTE devoted to preceptor development) frequently had dedicated site visitors.

Conclusion. Preceptor orientation programs at US schools of pharmacy are generally similar, but development programs vary significantly across the Academy. Highly invested programs featured live and online training or site visitors who provided individualized feedback or training. Future studies should explore the cost-effectiveness of program options and their impact on preceptor learning and behaviors.

INTRODUCTION

Standard 20 of the Accreditation Council for Pharmacy Education (ACPE) 2016 Standards (Standards 2016) describes expectations regarding preceptors.¹ Standard 20.1 states that the college or school must make available and apply quality criteria for preceptor recruitment, orientation, performance, and evaluation. Standard 20.3 specifies orientation components of the program’s mission, specific learning expectations, and effective assessment techniques, and requires orientation to occur prior to student placement at a site. Section 20f from the ACPE Standrds 2016 Guidance Document (Guidance 2016) further states that preceptors should be oriented to the school’s mission, goals, values, curriculum, teaching methods, grading and assessment process, objectives for each experience, methods to assess students’ prior knowledge and experience, and how to address student issues.² It would be valuable to identify strategies used within experiential education programs to ensure these requirements are met.

Standard 20.3 requires that schools foster the professional development of their preceptors, but neither Standards 2016 nor Guidance 2016 specifies preceptor development requirements. In national surveys, experiential education administrators have expressed concern about preceptor development issues such as measuring site quality and consistency in student performance assessment.^3-5

The 2011-2012 American Association of Colleges of Pharmacy (AACP) Professional Affairs Committee proposed that schools cultivate a variety of preceptor development opportunities using diverse delivery mechanisms and collaborating with other schools in preceptor development activities.⁶ Innovative preceptor development initiatives that have been reported include include preceptor advisory boards, menus of development options delivered through different media, sharing costs and efforts across schools within a consortium, and creating television-like mini-series or short training videos.^7-10 It is unknown to what extent these innovations have been adopted. A 2013 report of experiential education web sites found that many described preceptor development but the only listed option on 45% of the web sites was a link to a company offering free preceptor training, a finding inconsistent with cultivating a variety of opportunities using diverse delivery mechanisms.¹¹

The only information about preceptor development provided in Standards 2016 is that it is required,¹ and little information exists in the literature to provide a baseline for preceptor development. The 2016-2017 AACP Professional Affairs Committee informally surveyed experiential education administrators, but the goal was to identify resources that AACP could provide directly to preceptors, rather than to identify training needs within programs.¹² Findings from published surveys assessing the development of residency preceptors do not translate easily to pre-graduate level training across a variety of practice settings.^13,14 Given the ACPE mandate for preceptor training and the importance of preceptor orientation and development for quality assurance, the lack of publications characterizing preceptor orientation and development efforts across institutions is surprising.

This study had four objectives. First, to identify strategies used in experiential programs to meet ACPE Standards for preceptor orientation. Second, to establish a baseline for preceptor development programs. Third, to identify unpublished innovations in preceptor development for potential adoption by other schools. Finally, the study sought to identify future areas of scholarly work related to preceptor orientation and development.

METHODS

This study was a cross-sectional analysis of data obtained from a survey of experiential education administrators conducted in January and February of 2017. The investigators were members of the AACP Experiential Education Section who indicated an interest in investigating this topic. Using an iterative approach, a 28-item survey was developed that contained both closed- and open-ended questions pertaining to four general areas: participant demographics, description of preceptor orientation program, description of preceptor development program, and ideal program elements assuming adequate resources. Wording of the survey questions was approached from the perspective of appreciative inquiry.^15,16 Prior to survey release, colleagues of the investigators tested the instrument for clarity and response time. The instrument was delivered via WebQ (Catalyst Web Tools, University of Washington), a web-based survey program.

A list of experiential education administrators was compiled from public websites across the country and from information available on the AACP membership list. Individuals on the list were contacted by investigators to confirm the identity of the single best person on their experiential education team who was knowledgeable about preceptor orientation and development efforts. Identified individuals were notified they would receive an announcement about the survey one week prior to receiving the survey invitation and link. Invited individuals received email reminders at one and three weeks after the original invitation was issued. All survey respondents were anonymous. A subcommittee of the University of Washington Human Subjects Division examined the survey and protocol and determined that the study qualified for exemption.

Descriptive statistics were used to quantify responses to close-ended survey questions. Answers to three open-ended questions were thematically analyzed using inductive coding strategies.^17-19 A study team member who had qualitative research experience, and a student independently read and began the initial coding of the open-ended responses. These two individuals met repeatedly to review themes and coding descriptions, resolving discrepancies until consensus was reached. All other authors acted as verifier coders for one of the open-ended questions and their coding responses were used to calculate percent agreement. All authors participated in identification of notable practices, which were those deemed unusual or innovative. An additional measure, Cohen’s kappa, was used to assess agreement resulting from chance alone, and a value above 0.6 was established as indicating satisfactory agreement.^20,21 Because the results from Cohen’s kappa are overly conservative when the coding tasks being assessed are complex, as was the case with the answers in this study, an additional measure, Gwet’s first agreement coefficient, was used. Although less widely-known, the results from testing with Gwet’s first agreement coefficient would be more reliable in this situation.^22,23 Pearson correlation coefficients were calculated between data derived from demographic questions, quantitative survey answers, and qualitative analysis coding, with application of the Bonferroni-Holm correction for multiple comparisons indicating that a p value <.005 was significant for the pairwise comparisons. All statistical testing was performed using R, version 3.5.0 (The R Project, Vienna, Austria).²⁴

RESULTS

Eighty-five of 128 invited experiential education administrators responded to the survey, resulting in a response rate of 66%. School demographic data obtained from survey respondents are outlined in Table 1. Of the respondents, 14% were from programs that had opened within the five years prior to taking the survey; 15%, within six to 10 years; 26%, within 11 to 19 years, and 45% from schools open for 20 or more years. Fifteen percent of respondents lived in states that legally mandated training for pharmacy preceptors. Overall, 93% of preceptors were considered volunteers, with 24% receiving some kind of stipend. Schools with higher numbers of preceptors were highly correlated with higher numbers of students (r=.53; p<.001) and moderately correlated with years since opening (r=.34; p=.002). There was also a moderate correlation between public schools and years since opening (r=.38; p<.001).

Table 1.

Demographics of Institutions That Participated in a Survey to Assess Preceptor Training Programs Compared to Demographics of all Institutions

Preceptor Orientation

Schools used a variety of methods to orient preceptors to their programs and the type(s) of pharmacy practice experiences offered. At the time of the survey, 65% of the schools represented had met the ACPE requirements outlined in Standard 20.3 for preceptor orientation and 19% had met the orientation suggestions outlined in 20f of the guidance document. Notable topics covered in orientation programs included cultural sensitivity, sexual harassment, and adult learning styles. Only 28% of respondents indicated that their schools provided guidance on assessing a student’s prior knowledge and experience relative to course objectives as part of preceptor orientation. Forty-two percent of respondents’ schools required completion of orientation activities prior to preceptors receiving their first student assignment, while 53% of schools did not specify when orientation activities should be completed.

The majority of respondents indicated orientation programs lasted between one and six hours, with 24% indicating their school held a one- to two-hour orientation; 36%, a two- to three-hour orientation; and 12%, a three- to six-hour orientation. Delivery modalities varied, but the majority of orientation programs were live. Face-to-face orientation programs at the practice site were offered by more than 80% of respondents, live presentation at the school by 40%, downloadable written materials by 65%, slides with or without a voiceover by 42%, and webinars by 20%.

Sixty respondents provided answers to the open-ended question asking what worked well for preceptor orientation at their institution. Four themes emerged during qualitative analysis of the answers, and those themes, along with illustrative quotes and practices deemed notable by the authors are presented in Table 2.

Table 2.

Elements of Successful Preceptor Orientation and Development Programs Identified in a Survey of US Schools and Colleges of Pharmacy

Preceptor Development

Respondents outlined the strategies used to provide ongoing preceptor development at their schools. At 89% of the schools, experiential education faculty members were responsible for preceptor development. Sixty-seven percent of the schools offered live preceptor development annually. Of these, 35% of the programs were multiple-hour and 32% were of shorter duration. Nearly one-third of the schools offered live programming in conjunction with a state association meeting. On average, 25% of the school’s preceptors attended the live events. The most popular content included working with challenging student situations (41%) and providing feedback to students (39%). Other common topics included student assessment and evaluation (24%), generational differences in learners (18%), and programmatic and/or curricular updates based on accreditation standards, Center for the Advancement of Pharmacy Education (CAPE) outcomes, and/or the Pharmacists Patient Care Process (15%).

Over one-third of responding schools worked in collaboration with other schools of pharmacy to create preceptor development programs, and 15% partnered with their state professional pharmacy organization to design events. In addition to live programs, 75% of responding schools offered commercially available online programs and 42% provided self-created slide-based content. To track completion of preceptor development activities, 38% of respondents required manual data entry into their experiential education management system, while 19% had a method for automatic capture of program completion.

Sixty respondents provided answers to the open-ended question asking what worked well for preceptor development at their institution. Four themes emerged during qualitative analysis of the answers. Themes, theme descriptions, illustrative quotes, and notable practices are provided in Table 2.

Seventy percent of respondents reported that their school spent over $2500, and 39% reported that their school spent over $5000 annually on preceptor development. Programs with significant investment (> $10,000/year) in preceptor training usually provided multiple platforms (live and online). Programs with high personnel investment (≥0.5 full-time equivalent [FTE], 19% of programs) in preceptor development more frequently visited sites and provided group or individualized development activities. Fifty percent of respondents’ schools had a minimal number of personnel invested (<0.2 FTE) and 31% had a moderate number of personnel invested (0.2-0.4 FTE) in preceptor development.

Seventy-four respondents (87%) provided an answer to the survey item, “Describe aspects of your ideal preceptor development program if you had unlimited resources (money, time, staff) and creative license.” Qualitative analysis of the answers revealed three themes, which are outlined in Table 3. The theme of retaining preceptors was correlated with schools that had higher numbers of students (r=.30; p=.0047) and preceptors (r=.35; p=.001).

Table 3.

Elements of Ideal Preceptor Development Programs Identified in a Survey of US Schools and Colleges of Pharmacy

DISCUSSION

At the time of the survey, it appeared that schools were making progress in meeting the requirements as outlined in Standards and Guidance 2016 by offering a one- to three-hour orientation. Given that respondents expressed an interest in wider use of technology, it was interesting that the majority still preferred to orient their preceptors face to face.

We were able to establish a baseline for preceptor development programs, which was our second study objective. Although most respondents indicated that they used online programs, either self-made or commercial, two-thirds offered live programming for preceptors annually or more frequently, and many respondents stated that they traveled to preceptor sites to deliver programs requested by those sites. A few programs with preceptors in rural areas or in locations far from the school were delivering the live programming through video conferencing. Thus, live programming in some form, in addition to online options, appeared to be the baseline for preceptor development indicated by program respondents at the time of this survey. We also were able to characterize innovations in preceptor development (notable practices) in this report, which was our third objective.

An unexpected finding was the low percentage of schools requiring their preceptors to receive orientation and development training, given the ACPE mandate. Only 42% of reporting institutions required a formal preceptor orientation to the institution’s program prior to assigning students to a preceptor. Likewise, only 29% of institutions required formal, ongoing preceptor development. Reasons for the low numbers may include a lack of personnel or other resources, lack of adequate incentives, and a high reliance (93%) on volunteer preceptors, who cannot be required to complete development programs. The challenge of developing volunteer preceptors cannot be understated. These individuals are constantly trying to carve time out of their busy schedule to teach, coach, and mentor students. Preceptors may have little to no formal training in teaching and may not be fully supported in this role by their managers. Further complicating the issue is that many volunteer preceptors must complete development activities outside of work hours, which may limit their engagement. Schools need to devote adequate resources and incentives to support the efforts of volunteer preceptors.

One important resource is adequate funding. Fifty-seven percent of institutions reported spending $5,000 or less on preceptor development, which seems insufficient considering that approximately one-third of pharmacy curricula is delivered by preceptors. In addition, most survey respondents noted less than a half of an FTE is devoted to preceptor training, which at some institutions is inadequate. These findings highlight the need to recognize the importance of experiential education resources by academic administrators. A committee of experiential education administrators were tasked by ACPE in October 2018 to create a workload analysis process that could assist institutions with defining and documenting their office activities. The final instrument and process were presented at the July 2019 AACP Annual Meeting and subsequently made available to members through AACP Connect.

Incentives, the most frequently observed theme in the qualitative preceptor development data, may increase preceptor participation in development activities. Incentives may include payment for precepting responsibilities, resident payment, continuing education programming, library access, adjunct faculty title, and conference registration, among others.²⁵ Payment for precepting as an incentive is controversial, but provision of an adjunct faculty appointment and access to electronic information sources are incentives that are valued by some preceptors and may increase engagement.²⁶ An additional incentive could come from pharmacy regulatory bodies. Some state boards of pharmacy recognize preceptors with special credentialing for which completion of a preceptor development program is required. More widespread adoption of this requirement might increase preceptor participation in development activities offered by schools.

Some respondents reported challenges in getting preceptors to participate in development activities despite investing money and personnel into preceptor development. Live preceptor training, used by two-thirds of the survey respondents, reached an average of 25% of preceptors. Only a fifth of respondents reported that their school’s program reached over 35% of their preceptors. We did not collect data on costs specifically associated with live programs, but they generally require significant financial and staff resources. It would be useful in future studies to measure the value of live versus online precepting programs.

Many respondents expressed a desire for individualized preceptor development. Ideally, programs should be personalized to each preceptor and be offered through a diversity of mediums, yet most schools lack the resources to connect preceptors with individualized training resources. A method of streamlining or automating such a connection would be desirable. Important quality measures for individualized preceptor development include determining how preceptors incorporate training into their practice and whether incorporation improves student perceptions of preceptor effectiveness. Our survey results indicate that program completion is generally tracked, but there is little follow-up with or feedback obtained from preceptors, which would be needed to measure program quality.

Approximately 75% of respondents reported contracting with a commercial vendor (eg, Pharmacist’s Letter, CEImpact) to provide preceptor training. Programs offered by commercial vendors, while providing useful information, need to be general in content rather than tailored to an institution’s specific needs or issues. Institutions should consider taking advantage of technology that can promote interactive learning or individualized training. Only 20% and 11% of schools that responded used webinars or interactive online modules for orientation programs and development programs, respectively, and no program reported using any type of podcast technology for training. Other technology platforms (eg, videos) exist, but are often only a small component of training programs. Modern-day learners (soon to be preceptors) may prefer contemporary technology such as podcasts over the traditional lecture model usually used in live programs and in most online voice-over slide presentations. Indeed, many of the participants who responded to the question pertaining to the “ideal precepting program” named short, high-quality videos as a desirable component.

This study had several limitations including a low response rate. While nonresponders would have contributed additional insights, the demographics of the schools represented by the respondents were similar to the demographics of US pharmacy schools as a whole (Table 1), therefore we believe the results of this study are fairly representative of most US pharmacy programs. A second limitation was survey timing. The survey was conducted just six months after Standards 2016 went into effect. As programs undergo re-accreditation, they will likely make changes to their preceptor orientation and development programs based in part on compliance with new guidelines in Standards 2016.

This study had several strengths. We believe this was the first study to examine preceptor orientation and development programs across the Academy. The mixed methods approach provided a variety of data and revealed insights regarding respondents’ feelings and concerns about preceptor development. We believe the appreciative inquiry approach encouraged respondents to identify what they were doing well rather than focus on areas needing improvement.

Future initiatives for preceptor orientation, our final study objective, could include development of components for a national preceptor training program for new IPPE and APPE preceptors, along with a standardized list of elements that could be tailored for each institution. Preceptor development programs should consider using a continuing professional development approach like that outlined by Tofade and colleagues and advocated by the 2011-2012, 2015-2016, and 2017-2018 AACP Professional Affairs Committees.^6,27-29 In such a process, preceptors would identify their individual training needs and then institutions would develop programming to meet those needs. Institutions would need to make available core competency statements and supporting performance indicators to assist preceptors in the process, similar to materials included in a proposed national preceptor development program.^30-32 Performance indicators should arise from assessments made of or by students, sites, preceptors, and experiential education personnel and ideally include self- and peer assessments. Sufficient programming elements for competency areas using a variety of delivery mediums are the goal. Institutions are strongly encouraged to collaborate within consortia, with state professional associations, and with other health profession schools to accomplish these goals. A quality assurance program must routinely assess whether training initiatives are meeting preceptors’ needs. Finally, it is critical for academic institutional administrators to support preceptor orientation and development through provision of adequate resources for programs and personnel.

CONCLUSION

Schools of pharmacy rely on preceptors to support over 30% of the PharmD curriculum. Accreditation standards require preceptor orientation and development plans. Results of this study reveal persistent challenges within experiential programs because of limited resources. The academy would do well to provide additional resources to internal and external teams to address the multitude of requirements and the need for additional personnel.

Minimum expectations for preceptor orientation were not being met by all of the programs that participated in this study. There was wide variation across development plans in terms of content, delivery modes, and resource allocation for preceptor training. Themes for preceptor development included reaching, retaining, and developing preceptors. Results of this study showed that the size or age of the program did not predict the robustness of the preceptor development program, so preceptor development activities may be more affected by the perceptions of each school’s administration regarding the importance of experiential education and by their resulting provision of adequate resources. Future studies should focus on the best ways to individualize preceptor development, assess the impact of preceptor orientation and development efforts, and increase preceptor engagement.

Appendix 1. Survey Questions

1. Which of the following describes your University? (public or private)

2. What is the length of your PharmD program? (4-option list)

3. How many students were in the new class that started your professional program in 2015? (5-option list)

4. How many years ago did your school or college admit its first class of PharmD students? (4-option list)

5. Does your state Board of Pharmacy have any requirement for preceptor training?

6. How many different preceptors hosted one or more students (IPPE and APPE) in the last two years?

7. Please indicate the percentage of your preceptors that best fit each description. The numbers should total 100% (list includes School-based faculty, Volunteer preceptor who receives stipend, Volunteer preceptor without stipend, Other)

8. Indicate which of the following incentives are provided to your preceptors. (10-item list)

9. Identify the components covered in your current general school orientation. (7-item list)

10. Describe all the ways you offer this general school orientation. (7-item list)

11. Identify the components covered in your current course-specific orientation. (9-item list)

12. Describe all the ways you offer this course-specific orientation. (7-item list)

13. Describe any additional training you require preceptors to complete as part of your orientation process. If you don’t require additional orientation activities, please indicate this in the text box.

14. What is the approximate length of time in minutes it takes for an average preceptor to complete all required elements of your orientation program?

15. How far in advance of taking the first student is a preceptor required to complete the orientation? (3-item list)

16. What are elements of your preceptor orientation program that make it particularly effective? Please explain why. (open-ended question)

17. How often do you offer live preceptor development programs? (6-item list)

18. In a typical year, how many preceptors (IPPE and APPE) attend your live preceptor development program?

19. What program topics for your live preceptor development programs have proved popular or been well-received by your preceptors? (thematically-analyzed open-ended question)

20. What types of online preceptor development programs do you offer? (9-item list)

21. How do you provide individualized/one-on-one/targeted preceptor development?

22. Do you have a minimum requirement for completion of preceptor development in order for a preceptor to remain active in your experiential program? (yes or no)

23. Do you collaborate with other schools or professional organizations in creation of any of your preceptor development programs? Please describe.

24. We are interested in learning what works for preceptor development at your institution so that we can compile a list of successful practices. What are elements of your preceptor development program that make it particularly effective? (thematically-analyzed open-ended question)

25. How do you capture data about preceptor participation in development activities in your experiential education database? (3-item list)

26. Please estimate the annual cost of preceptor development to your institution. Do not include salaries for Experiential Education staff time. Only consider “real costs” such as subscriptions to commercial online programs (eg, CEI, APhA), stipends for speakers, costs of meals and/or room rental, costs of online site maintenance, etc. (6-item list)

27. Who in your school has responsibilities for preceptor development? Identify job title (eg, CE Director, EE Program Coordinator), whether the individual is faculty or staff, the nature of that person’s responsibilities, and the percent full-time equivalent (FTE) spent by that individual on those responsibilities.

28. Describe aspects of your ideal preceptor development program if you had unlimited resources (money, time, staff) and creative license. (thematically-analyzed open-ended question)

29. Please indicate any additional comments or questions.