Development of a Postgraduate Community Pharmacist Specialization Program Using CanMEDS Competencies, and Entrustable Professional Activities

Abstract

Objectives. To develop and implement a postgraduate, workplace-based curriculum for community pharmacy specialists in the Netherlands, conduct a thorough evaluation of the program, and revise any deficiencies found.

Methods. The experiences of the Dutch Advisory Board for Postgraduate Curriculum Development for Medical Specialists were used as a guideline for the development of a competency-based postgraduate education program for community pharmacists. To ensure that community pharmacists achieved competence in 10 task areas and seven roles defined by the Canadian Medical Education Directions for Specialists (CanMEDS), a two-year workplace-based curriculum was built. A development path along four milestones was constructed using 40 entrustable professional activities (EPAs). The assessment program consisted of 155 workplace-based assessments, with the supervisor serving as the main assessor. Also, 360-degree feedback and 22 days of classroom courses were included in the curriculum. In 2014, the curriculum was evaluated by two focus groups and a review committee.

Results. Eighty-two first-year trainees enrolled in the community pharmacy specialist program in 2012. That number increased to 130 trainees by 2016 (a 59% increase). In 2015, based on feedback from pharmacy supervisors, trainees, and other stakeholders, 22.5% of the EPAs were changed and the number of workplace-based assessments was reduced by 48.5%.

Conclusion. Using design approaches from the medical field in the development of postgraduate workplace-based pharmacy education programs proved to be feasible and successful. How to address the concerns and challenges encountered in developing and maintaining competency-based postgraduate pharmacy education programs merits further research.

INTRODUCTION

Pharmacy practice has changed rapidly in the past 25 years, guided by the concept of pharmaceutical care.¹ The role of pharmacists has evolved from being mainly product centered toward being more patient centered, and the delivery of clinical pharmacy services has led to new specializations for pharmacists.² To prove their professional competence and to ensure the connection of the education system with the health care system, the achievements and shortfalls of pharmacist trainees need to be measured.³ As seen in other health care professions, competency-based educational models are becoming dominant in pharmacy education.⁴ Standards 2016 of the Accreditation Council for Pharmacy Education (ACPE), for instance, uses competencies to frame the outcomes for pharmacy education programs.⁵ Seeing that the workplace is an important context for the development of competencies, postgraduate educational programs for health care specializations often focus on workplace-based learning.^6,7 In a workplace setting, trainees can demonstrate relevant knowledge, skills, and behaviors in a contextually rich but relatively safe environment and competencies can be assessed in a real world context. Entrustable professional activities (EPAs) can be used to measure competence. In 2017, the AACP list of core entrustable professional activities for pharmacy graduates was published.⁸

Community pharmacists have to various degrees transformed into health care providers managing chronic diseases in countries such as Australia, Canada, England, Scotland, and the United States.⁹ In the Netherlands, the transformation of the community pharmacist into a clinical primary care specialist started in the 1980s with the introduction of medication surveillance for drug-drug interactions and drug-disease interactions.^10,11 Dutch pharmacies use electronic patient records to document use of prescription and nonprescription drugs.¹² In the last decade, community pharmacy practice services have expanded, for example, with the provision of clinical medication reviews for elderly patients.^13,14 Also, in 2016, the Netherlands’ Minister of Health, Welfare and Sport acknowledged the community pharmacist as a specialist.¹⁵

In the Netherlands, a six-year program that involves earning a three-year bachelor’s degree and then a three-year master’s degree must be completed to become registered as a pharmacist in accordance with European law. The Bachelor of Pharmacy degree program provides a broad science education with a strong emphasis on pharmaceutical science. The Master of Pharmacy (MPharm) degree program prepares students for positions in the three main professional domains: community pharmacy, hospital pharmacy, and industrial pharmacy. A six-month internship in a community pharmacy and/or a hospital pharmacy is mandatory to complete the master of pharmacy program.¹⁶ The entire six-year pharmacy program is taught at three Dutch institutions: the Universities of Groningen, Leiden, and Utrecht. Hospital and community pharmacy specializations in the Netherlands require MPharm graduates to complete four and two years of postgraduate training, respectively.

The curriculum for community pharmacist specialization was introduced in 1995. Until 2012, outcomes of the curriculum were not defined and specific competencies of community pharmacists were not assessed explicitly. In that year, a new competency-based curriculum with a focus on workplace-based learning was launched for professionals who wanted to become registered as a community pharmacist. More than 90% of pharmacists working in a Dutch community pharmacy have specialized as community pharmacists. Although it is not obligatory, there are both professional and financial reasons for pharmacists to do so. In 2014, two years after the new curriculum for community pharmacists was launched, it was evaluated using input from all stakeholders, and in 2015, it was revised.

The aim of this study was to describe the development, evaluation, and revision of the two-year postgraduate curriculum for community pharmacists in the Netherlands. This paper discusses our experiences so that other postgraduate pharmacy programs can learn from them. In the program design, several approaches from the medical field were used to develop and assess competencies, including the Canadian Medical Education Directions for Specialists (CanMEDS) competency framework, entrustable professional activities (EPAs), and programmatic assessment.^6,17,18

METHODS

For the development of the competency-based postgraduate education program, the experiences of the Dutch Advisory Board for Postgraduate Curriculum Development for Medical Specialists were used as a guideline.⁷ In the present study, six steps were used to describe the design of the curriculum. These steps were previously described for curricular development of competency-based medical and pharmaceutical education programs (Table 1).^4,19

Table 1.

Steps in Building Competency-based Education Programs^4,19

To identify the professional requirements for community pharmacists, a series of discussion groups was organized with stakeholders within the community pharmacy domain, such as community pharmacists, teachers, and policy advisors.²⁰ A proposal for defining task areas was made by the program director. This proposal was discussed in each group, and written notes of the comments were made. In addition, both individual stakeholders and stakeholder organizations such as the Young Pharmacists’ Association, were encouraged to give feedback on the matter by email. In total, more than 100 of the 2,200 community pharmacists registered in the Netherlands shared their vision of the tasks that should be performed by community pharmacists. Next, a list of 10 “task areas” associated with the role of the community pharmacist was compiled by the program director (Table 2).²¹

Table 2.

Ten Task Areas of Community Pharmacists Working in the Netherlands²¹

The CanMEDS model was used to define the required competencies of community pharmacists. In 2006, medical specialists in the Netherlands chose the 2005 CanMEDS competency framework, which was developed by Canadian medical specialists, to be used in the educational programs of all Dutch medical specialists.^7,17 Although the CanMEDS model was developed specifically for the medical profession, it coincided time wise with the new role of the community pharmacist as a clinical primary care specialist in the Netherlands. The CanMEDS model uses seven roles to describe the functions of the medical specialist. In the adaptation to the Dutch community pharmacist, the medical expert role was replaced by that of the pharmaceutical expert because the community pharmacist is an expert in the fields of pharmacotherapy, product care, and patient care. Product care is defined as “procuring (availability), storage and dispensing, including instruction of the patient of medicinal products as well as with preparation if necessary”.²² The other six CanMEDS roles were adopted with minor changes (Table 3). Each of the seven roles was linked to four competencies as shown in Appendix 1.

Table 3.

Seven Roles of the Dutch Community Pharmacist Based on the CanMEDS Model¹⁷

During the two-year program, trainees need to develop their roles and competencies to a level at which they are competent to act without supervision in all task areas. Four milestones were placed within the development path at six months, 12 months, 18 months, and 24 months of training to guide competency development. At each milestone, a desirable level of competency was defined using the construct of EPAs as proposed by Ten Cate and colleagues.²³ An EPA is an activity or unit of professional practice that can be entrusted to a sufficiently competent trainee. Because EPAs are tied to daily practice, they are suited for training and assessment purposes. To get a complete view of the level of competency required of a trainee, a sufficiently large number of EPAs was needed. Within the program, 40 EPAs were defined to connect every CanMEDS role and every task area through multiple EPAs.²⁴ A scale of entrustability was defined that ranged from level 1 (“the trainee is present and observes”) to level 4 (“the trainee can act without supervision readily available, but with distant supervision and oversight”).^21,23

At the workplace, the supervisor is responsible for giving feedback and assessing the progress of the trainee. Entrustable professional activities need to be assessed as they are completed or shortly thereafter. Depending on the EPA, the activity may need to be assessed only once or possibly several times during the program. In total, 155 workplace-based assessments of EPAs were placed within the curriculum (76 in the first year and 79 in the second year). Within each year, the trainee and supervisor can schedule the assessments as they consider suitable.

Four types of assessment instruments were selected to assess the EPAs (Table 4).^24,25 For each EPA, the best type of instrument was selected based on the content and function of the instrument and the competencies that it measures. Prior experiences from postgraduate medical education programs in the Netherlands were used to inform our assessment.^7,26 Next, the instruments were adapted to fit the context of the EPAs. For example, a short practice observation (SPO) was used to assess patient counseling, and a product evaluation (PE) was used to assess performing a medication review for an at-risk patient, including writing a pharmacotherapy treatment plan. In addition to measuring individual EPAs, a 360-degree feedback instrument was developed to incorporate prior observations of trainees’ competencies by other pharmacists in the working environment, pharmacy assistants, general practitioners, and patients.²⁷ The instrument consisted of an online questionnaire that was to be used once each year and included a self-assessment by the trainee.

Table 4.

Types of Assessment Instruments Used in a Community Pharmacist Curriculum for the Assessment of Entrustable Professional Activities

Instruments were developed for the trainee to maintain a personal development plan to guide his own competency development and assessment in the pharmacy workplace. An online digital portfolio containing all the necessary assessment forms was developed to structure and document the assessment results, scores, and narrative comments. The supervisors were instructed to use this information to monitor the trainees’ competency development. Every three months the supervisor needs to complete a formative performance evaluation by scoring the trainees’ level of competency relative to the required level for a community pharmacist on each of the seven CanMEDS roles.²⁸ At two of the four defined milestones (12 months and 24 months), the supervisor carries out a summative performance evaluation to determine whether the trainee is fit to continue his training and finish the educational program. These scores are also documented in the trainees’ portfolio.

The majority of the educational program is carried out in the workplace. The community pharmacy needs to be officially approved by the Specialist Registration Committee (SRC) before it is accepted as a workplace for trainees. Requirements were developed concerning, for example, the work climate, the attendance of the supervisor, and the facilities of the pharmacy. Pharmacies needed to have a certificate that was ISO 9001 (International Organisation for Standardisation) compatible to ensure that they provided an overall high quality of pharmaceutical care. Also, a competence profile for supervisors was developed consisting of didactic, individual, and knowledge competencies. Based upon this profile, a mandatory, two-day introductory course for pharmacy supervisors was developed. The course included conducting a “360-degree” feedback session and self-assessment providing the supervisors with instruction on how to give feedback to trainees regarding their execution of EPAs.

In addition to the workplace learning, 22 days of classroom courses were developed in which trainees need to participate. These classroom courses are used to support trainees in their development of knowledge and skills on specific subjects, eg, important aspects of pharmacotherapy and product care, leadership and management skills, and ethics. Various written assignments were developed to assess these specific subjects. For example, to test pharmacotherapeutic knowledge, a multiple-choice test containing 100 questions was developed. Finally, the trainees would be required to participate in some nationally organized activities, such as managing an implementation project in the pharmacy, writing a plea based on ethical arguments, and conducting 10 medication reviews with elderly polypharmacy patients.

The whole curriculum was described in a document entitled Education Plan: Advanced Community Pharmacist Education Program.²¹ In December 2011, this education plan was accepted by the Dutch Scientific Community Pharmacists’ Section and approved by the Dutch Board of Pharmacy Specialisms. The education program started in January 2012, with a Director of Education responsible for the program as a whole and for reviewing the pharmacists’ portfolios prior to certification.

In fall 2014, an evaluation of the workplace-based learning and assessment was initiated. Based on input from surveys held earlier with trainees and supervisors, three themes were defined to evaluate in depth: trainees and supervisor workload, learning opportunities in the pharmacy workplace, and utility of the assessment system. Two focus group discussions were held to discuss these themes, one with seven trainees and one with seven supervisors. The participants were asked to identify bottlenecks they experienced and possible ways of improving the curriculum. The outcomes of both focus groups were discussed with a review committee representing relevant stakeholders: pharmacists in training, supervisors, teachers of the classroom courses, the SRC, and the director of community pharmacy education.

RESULTS

Eighty-two first-year trainees enrolled in the program in 2012. Enrollment increased to 130 first-year trainees in 2016 (a 59% increase). Commonly, pharmacists begin specializing within one or two years after graduation. Over the same four-year period, pharmacy degree programs in the Netherlands had a comparable increase in graduates from 172 pharmacists in 2012 to 221 pharmacists in 2016 (29% increase). More than two-thirds of the trainees were female, which is comparable to the male to female student ratio in pharmacy degree programs. Of the 271 trainees who started the program in 2012, 2013, or 2014, 242 (89%) completed the program successfully.

The participants in the focus groups acknowledged the importance of the competence-based pharmacy education program, stating that it improved the capabilities of community pharmacists working as high-standard health care professionals. However, they did raise concerns over each of the three themes that were discussed, experiencing: a heavy workload, insufficient insight into what is expected of both trainees and supervisors within the curriculum, and difficulties in using the assessment instruments and portfolio. Based on these outcomes, the review committee made the following recommendations: decrease the number of assessments, clarify the assessment instruments used, improve the look and feel of the portfolio, make supervisor training more instructive, and hold an additional evaluation of the impact and relevance of the classroom courses in the near future.

The program director used the input from focus groups to adjust the assessment program. A draft version was discussed with a working group consisting of three trainees and three supervisors. With their consent, changes were made to nine (22.5%) of the 40 EPAs that were introduced in 2012. Three EPAs were combined on a higher level of abstraction, two EPAs were combined and replaced by one new EPA, one EPA was reformulated, one EPA was split into two separate EPAs, and two EPAs were removed. Overall, the optimization led to a 10% reduction in the number of EPAs from 40 to 36. The changes were made partly because of changes in pharmacy practice, for example, compounding in community pharmacies was already in a gradual decline, but because of changes in remuneration, compounding became obsolete in three-fourths of community pharmacies within a few years. Other changes were made from a didactic point of view, for example, splitting the EPA on “giving written or verbal information to a group of patients” into two separate EPAs, one on written information and one on verbal information. Each assessment within the program was reweighed for its value and effectiveness. This led to a reduction from 155 to 80 assessments (48.5% reduction). For example, the number of product evaluations (PEs) was halved because the evaluation of patient records was also assessed through short practice observations (SPOs). Another example was reduction of the number of SPOs to assess compounding skills as a result of changes to the corresponding EPA. The user-friendliness of the portfolio was improved through numerous tweaks. For example, the assessment of workplace-related tasks and classroom-related tasks was more clearly distinguished.

The two-day course for supervisors originally focused on the competences needed as a supervisor and performance as a role model. In the adjusted training, every assessment instrument was discussed in more detail. Additionally, the SPOs and the progress reviews were practiced more intensely, and supervisors got a preview of the online portfolio and the management reports it produces. Furthermore, plans were made to introduce an advanced course for supervisors in the future.

In October 2015, the revised curriculum was approved by the Dutch Scientific Community Pharmacists’ Section and by the Dutch Board of Pharmacy Specialisms. The revised assessment program connecting task areas and competencies using EPAs, milestones, and assessment tools is shown in Appendix 2.

DISCUSSION

The advantages of competency-based education have previously been described: a focus on curricular outcomes, emphasis on the abilities that health care professionals require instead of the time spent on training and promotion of a greater learner engagement during training.¹⁹ The implementation of competency-based education, on the other hand, comes with many concerns and challenges.²⁹

In the Netherlands, the implementation of the postgraduate workplace-based curriculum for community pharmacists meant getting more control over the structure and outcomes of the educational program, but it left less freedom for both trainees and supervisors to find their own way. It has taken a few years for all stakeholders to accept and become accustomed to these new circumstances. The evaluation and revision in 2014-2015 and the accompanying communication contributed substantially to the acceptance of the educational program. Meanwhile, the popularity of the specialization did not diminish. Instead, the number of participants increased because of the greater number of pharmacists graduating from university and a growing recognition of the community pharmacists’ role in primary health care.¹⁵

The curriculum for the community pharmacist specialization program was largely built on theories and experiences coming from the field of postgraduate medical education. The CanMEDS competency framework was used, despite that other competency models for pharmacists are available.^4,30-33 The major reason for using the CanMEDS model was to support collaboration with other health care professionals within the Dutch health care system.⁷ In recent years, the CanMEDS model has been adopted by most other health care professionals in the Netherlands, for example nurses and physiotherapists. In 2016 the master’s degree program for pharmacy education also adopted the CanMEDS model, thereby creating an education continuum for pharmacists.¹⁶ The adoption of a medical model necessitated adapting the description of the physician’s roles in the pharmacy context. The changes made to the model were found to be acceptable by both trainees and supervisors in the 2014 evaluation.

Entrustable professional activities are designed to bridge the gap between theory and clinical practice.²³ Indeed, EPAs are found to function as such in competency-based medical education.²⁹ Nevertheless, the challenge remains of finding the delicate balance between assessment in a holistic way vs detailed assessment of specific activities.^24,34 The EPAs translate the abstract concepts of competencies and task areas into measurable outcomes. Pittenger and colleagues reported the successful use of EPAs as an assessment framework for pharmacy education in the United States.³⁵ This led to a plea for using EPAs in postgraduate pharmacy curricula, too.³⁶ The 2014 evaluation of the community pharmacist curriculum showed that it is possible to integrate EPAs in a postgraduate pharmacy curriculum. Additionally, in the 2015 revision, EPAs appeared to be adaptable to changes in pharmacy practice without having to adjust the underlying competency framework.

Assessment of EPAs can be used for both assessment of learning and assessment for learning. A key principle is that individual low-stake (formative) assessment moments are maximized for learning and feedback value, whereas high-stake (summative) decisions are based on the aggregation of the underlying low-stake assessments. This principle is referred to as programmatic assessment.¹⁸ The program evaluation in 2014 revealed that both supervisors and trainees complained of a high administrative burden caused by having to do 155 workplace-based assessments. Trainees within the education program felt highly externally controlled in their motivation to learn as shown by Tjin a Tsoi and colleagues.^37,38 This may negatively impact the learning and performance of trainees, as has been described earlier for medical students.³⁹ To reduce the administrative burden and avoid the accompanying risk of trivialization, all assessments of EPAs were reconsidered, resulting in a 48.5% reduction (from 155 to 80) in assessments in the two-year program.¹⁸ The overall reliability to judge the assessment of learning was felt to remain intact because every CanMEDS role and task area was still assessed through multiple EPAs and a combination of assessment instruments (Appendix 2). Moonen-van Loon and colleagues showed how the composite reliability of the assessment process improves when a combination of assessment tools is used.²⁶

A cause for concern is the dependency on the supervisor as the main assessor of workplace learning. With almost 70% of pharmacies in the Netherlands being independent or franchise, often the supervisor is also the pharmacy owner and therefore the employer of the trainee, paying both the salary of the trainee and the required program fee.⁴⁰ This can lead to bias in the assessment process in various ways. Although the supervisors’ role is somewhat balanced by the addition of 360-degree feedback and the grades on assignments from classroom courses, ideally, the dependency on the supervisor should be lessened. This can be achieved by appointing an additional supervisor, for example, by having trainees do their training in two community pharmacies instead of one. Another possible solution is further integrating the views of expert instructors from the classroom courses in the assessment program.¹⁸

Another concern is the quality of the feedback given to students. Supervisors are faced with a double role, being both the educator providing feedback and the judge reviewing the trainees’ progress. Van der Vleuten and colleagues state that this conflict of interest can decrease the reliability of the judgements.¹⁸ The 2014 program evaluation showed that both supervisors and trainees in the educational program felt the feedback skills of supervisors were insufficient, despite the supervisors participating in two days of training. The 2015 revision of the supervisor training was meant to improve the quality of the feedback, but further research is needed.¹⁸ A different approach to improve the quality of the feedback would be to appoint mentors, eg, highly qualified supervisors from another pharmacy, to support the feedback process and feedback use within the workplace. These mentors would need to be trained to review the progress of competency development based on the trainees’ portfolio.⁴¹ In current practice, however, these approaches are not feasible. Also, pharmacists must combine the roles of educator and judge on a daily basis when leading their pharmacy team.

Although the supervisor’s role is very important in workplace-based learning, research has shown that other aspects of the educational environment of the workplace are also important, such as support from other employees, the organization of the health care setting, and the pharmacy having a focus on learning as an organizational goal.⁴² Research is needed to evaluate whether the educational environment in teaching pharmacies is sufficient to stimulate the learning and performance of trainees.

Finally, our findings regarding postgraduate community pharmacist education are based on a single curriculum in a specific context and therefore are limited. Additional studies reporting the experiences that other institutions have had in developing postgraduate workplace-based curricula for pharmacists may strengthen our findings. Furthermore, in-depth research on student portfolio data, the motivations of community pharmacist trainees and supervisors, and the educational environment of the community pharmacy could reveal new opportunities for improvement of the curriculum.

CONCLUSION

This article describes the stepwise design of a postgraduate competency-based pharmacy curriculum. Design approaches were adopted from the field of medical education such as the CanMEDS competency framework, Entrustable Professional Activities, and programmatic assessment. The 2014 evaluation of the curriculum showed that the CanMEDS framework was well accepted, but the identified EPAs and the assessment program needed revising. Though there is still work to be done in researching and improving the curriculum described here, the authors believe that similar workplace-based curricula can be developed for other postgraduate pharmacy specializations around the globe.

Appendix 1.

Competencies Related to the Seven CanMEDS-roles of Dutch Community Pharmacists

Appendix 2.

Connection Between Task Areas, EPAs, CanMEDS Roles, and Assessment Instruments in the Revised Program