Design and Evaluation of a New National Pharmacy Internship Program in Ireland

Abstract

Objective. To design, deliver, and evaluate a National Pharmacy Internship Program that met the educational requirements of pharmacy graduates to register as competent pharmacists and earned graduates a master’s level degree.

Methods. The National Pharmacy Internship Program was designed as a 12-month, full-time, blended-learning, competency-based program leading to a master’s degree. Intern performance was assessed academically and by pharmacy preceptor (tutor) appraisals. Interns who demonstrated competency were invited to sit for the Professional Registration Examination (PRE). Feasibility and performance were evaluated and a longitudinal approach allowed intern and preceptor views to be compared to the former preregistration year.

Results. Overall performance in the PRE was good and relatively consistent with almost all interns proceeding to register as pharmacists. Interns believed that the program had enabled them to develop the knowledge, skills, and overall competencies required for future independent practice as a pharmacist. Preceptors considered the program to have built on prior learning and provided a sufficiently rounded experience for professional practice. Preceptors also stated that the program was an improved educational experience over the former, less structured, preregistration training.

Conclusion. The National Pharmacy Internship Program was perceived to be an improvement on the previous preregistration year. The program quality assured pharmacy education outcomes at the entry-to-practice level on a national basis, and uniquely recognized the students’ accomplishment by awarding them a master’s degree.

INTRODUCTION

The education of pharmacists in Ireland has undergone significant change over the past decade. Prior to 2009, pharmacy education and training comprised a four-year Honours Bachelor Degree followed by a fifth year of preregistration training. The fifth year was overseen by the Pharmaceutical Society of Ireland (PSI), the National Competent Authority for Pharmacy. The preregistration year involved pharmacy graduates working in community pharmacy, hospital pharmacy, industry, or academia (with a minimum block of six months in either community or hospital practice) under the supervision of a preceptor pharmacist (also called pharmacy preceptor). Preceptors were required to confirm that the preregistration graduate had completed the year, but there was no requirement to confirm competency for practice. The preregistration examination was a multiple-choice examination, mainly pertaining to medicines-related legislation, which was set by the PSI. There was no defined structure, learning outcomes, quality assurance of experiential learning, or recognition of the preregistration training by way of an educational award. Similarly, pharmacy graduates in the United Kingdom, Australia, and New Zealand were completing separate preregistration training prior to registration. These programs varied in their academic requirements and education provision, but none recognized the learning by awarding graduates an educational qualification.¹

The introduction of the Pharmacy Act 2007 served as a catalyst for change in pharmacy education.² The PSI was given enhanced responsibilities for overseeing the education and training of pharmacists. In 2008, the PSI commissioned the Pharmacy Education and Accreditation Reviews (PEARs) project, which was a “root and branch” review of pharmacy education in Ireland.³ The aims of the PEARs project were to identify strengths and weaknesses and make recommendations on a future strategy for the education and training of student pharmacists.³ The PEARs project revealed that there was considerable variability in the experience within the preregistration year. Concern was expressed that the preregistration year was educationally distinct from the undergraduate degree, lacked clear educational objectives, and had poor central quality control and inadequate assessment.³ New secondary legislation, also introduced in 2008, required pharmacy graduates to undertake a year of in-service practical training, followed by preceptor sign-off of professional competency, and successful performance on the Professional Registration Examination (PRE).⁴ The PSI subsequently sought applications for the provision of a competency-based program of in-service practical training and the PRE to replace the preregistration year. The new program would be delivered on behalf of and overseen by the PSI. The Royal College of Surgeons in Ireland (RCSI) was appointed by the PSI in 2009 to design and deliver the program and the PRE. This was designed by the School of Pharmacy as the National Pharmacy Internship Program National Pharmacy Internship Program (NPIP). The term “intern” was deliberately chosen to differentiate the educational experience through the NPIP from those that the previous preregistration training provided. The NPIP was designed as a 12-month, full-time, blended-learning, competency-based program leading to an educational award at the master’s degree level. The limitations within which the NPIP was designed and delivered were to be addressed in the longer term with the introduction of a new integrated five-year Master of Pharmacy (MPharm) program, which was the major recommendation of the PEARs report.³

The NPIP was a novel instructional design by international comparison and recognized as “a major change in the educational landscape in Ireland.”³ Evaluation was designed from the outset to incorporate items from the PEARs project to determine the interns’ and preceptors’ perspectives. The evaluation allowed exploration of the hypothesis that the education and training provided by the NPIP was an improvement on the previous preregistration training and would produce competent pharmacists. The purpose of this paper is to describe the first stage in the journey in the reform of pharmacy education in Ireland; namely, the design of the NPIP, the limitations within which it was designed and delivered, and the findings of the evaluation of the program over the first iteration from 2009-2014.

METHODS

The NPIP was, out of necessity, introduced over a very short timeframe with limited resources. The PSI issued the tender to provide the training program in June 2009, and RCSI designed and developed the program for commencement in October of the same year. The design of the NPIP was undertaken by core staff members from the School of Pharmacy, who later also took on module leadership roles. The involvement of several departments within RCSI, including the Department of Medicine, Department of Surgical Affairs, the Institute of Leadership, and the Quality Enhancement Office, harnessed the skills of those who had designed similar competency-based programs and the skills of medical educators and content experts. Faculty members from the Department of Medicine and the Institute of Leadership also undertook module leadership roles. Access to a well-developed information technology infrastructure also made the rapid development of the program feasible. Several other professors collaborated with the Quality Enhancement Office in the evaluation of the NPIP, providing an opportunity for longitudinal evaluation of the program, which aligned with the objectives of the PEARs project. The PEARs project involved conducting interviews with key staff members at the PSI, conducting focus groups and interviews with preregistration students in 2007-2008, and distributing self-completion questionnaires to all former preregistration students and preceptors involved in the pre-registration training during the five-year period up to 2007-2008.³ This provided broad input about pre-registration training from all major stakeholders. Consultation with other stakeholders and international pharmacy education experts was also undertaken on a less formal basis.

The primary aim of the NPIP was to develop the knowledge, skills, and attitudes acquired at undergraduate level in the practice environment, while ensuring that the emerging practitioners were competent and capable of providing pharmacy services professionally, safely, and effectively. The programmatic learning outcomes defined that graduates would be able to ensure the safe supply of all medicines to patients; contribute to improving prescribing within the health care team; practice pharmacy competently in the primary or secondary health care setting; relate pharmacy law and ethics to practice; implement a safe, high-quality service in all health care settings within a clinical governance framework; apply information and mastery skills to the provision of health-related information, and implement change within their organization or complete a clinical audit.

These programmatic outcomes were based on a NPIP competency framework. The faculty member who led on the development of the competency framework had previously been commissioned by the PSI in 2006 to investigate competency-based education. The competency framework was developed with reference to general-level elements of key international pharmacy competency lists and frameworks that were available at that time.^5-14 Consultation with international experts from the EVOLVE Intern Program New Zealand, and the Competency Development and Evaluation Group (CoDEG), proved particularly valuable. All the information was distilled to formulate six core standards. Behavioral descriptors were developed to assist with interpretations of whether intern performance met expected standards. The resulting competence standards were piloted in diverse training establishments and subsequently amended based on feedback. The NPIP competency framework is available on request.

The program design was influenced by Fink in creating opportunities for significant learning that prepares the learner for the working environment.¹⁵ Immersion in the practice environment provided opportunities for application of knowledge and skills and gave interns the vision of becoming a pharmacist. Experiential learning, therefore, formed the core of the program, with particular attention to integration with academic components.¹⁵ Interns undertook between 35 and 40 hours of experiential learning per week in a training establishment approved by the PSI, in accordance with the PSI (Education and Training Rules) 2008.⁴ Interns spent a minimum of six months working in either a community pharmacy or hospital pharmacy, although there were a very limited number of hospital internship placements available at that time in Ireland. Some interns elected to undertake six months in a nonclinical placement such as the pharmaceutical industry or an academic placement in one of the three schools of pharmacy. A preceptor pharmacist was defined as a registered pharmacist who had practiced for a minimum of three years with a minimum of one year of experience in the field of pharmacy practice in which he or she intended to act as a preceptor pharmacist, and who had completed the requisite program of education and training. The provision of standardized training to preceptor pharmacists was obligated by the PSI (Education and Training) rules 2008,⁴ so a preceptor training program was developed by RCSI. The first iteration of the preceptor training program was an online modular course, with 14 eLearning packages, summatively assessed via multiple-choice questions (MCQs). A Tutor Training and Accreditation Programme (TTAP) Project Steering Group, which included stakeholder representation from across the profession, was set up in 2010 to further develop the preceptor training program. The TTAP was designed to provide preceptor pharmacists with the knowledge and skills to more effectively train and coach pharmacy interns. The required competencies for preceptor pharmacists, consistent with international best practice, were identified. A novel blended learning program was designed using advanced multimedia techniques. Preceptor network meetings were provided for preceptors to learn skills, such as coaching and providing feedback, in an interactive format and to facilitate networking opportunities. The workload allocated to the TTAP was 10 hours. Successful completion of the e-learning program and summative assessment resulted in accreditation for the training establishment in which the preceptor pharmacist operated, and preceptors were recognized as associated teaching faculty members of the college. Preceptors were required to attend a preceptor network meeting every two years, after initial accreditation, to refresh their skills and knowledge. This frequency was a recommendation supported by preceptors.

Interns were geographically dispersed throughout the country in a wide variety of training establishments for their experiential learning. Personal difficulties were identified as a concern in preregistration training in the PEARs report, and so it was important to identify ways to ensure interns were supported.³ A Memorandum of Understanding between RCSI and the PSI enabled the development of a suite of support services including access to advice, assistance with change of placement, counseling, and specialist mental health services. Interns and preceptors were encouraged to contact RCSI in confidence to access these services, which were provided through RCSI and funded by the PSI.

The experiential learning was encompassed within an academic framework, integrated through alignment to the NPIP competency framework, which mapped to the programmatic outcomes (Table 1). The curricular design of the academic component was based on the principles of andragogy, fostering learner-focused enquiry and critical thinking. Interns learned by experience and had flexibility in how they engaged with the content, influenced by self-appraisal and their own learning needs assessment. There was an emphasis in the coursework on problem-solving, related to authentic practice, structured around the competency standard framework. The academic framework of the program was modular and compatible with the award of European Credit Transfer System (ECTS) credits.¹⁶ The program offered six taught modules and a research module. Each ECTS credit represents 25 hours of learning, and so the taught modules each consisted of 250 hours of learning, encompassing direct contact, coursework preparation, independent learning, and formative and summative assessment. The workload of each module only comprised a small proportion of the hours spent in the workplace in the first iteration of the NPIP. The learning outcomes were developed to Masters Level 9 on the Irish National Framework of Qualifications (NFQ), equivalent to Level 7 on the European Qualifications Framework (EQF), and consistent with the second cycle of the Bologna Framework.¹⁷ There was an elective option for the research component. Interns chose between doing a dissertation on organizational development or completing a clinical audit. The module interprofessional prescribing science provided an opportunity to introduce interprofessional education at entry to practice, addressing common and collaborative IPE competencies.¹⁸ Prescribing competencies and collaboration between doctors and pharmacists are important. Thus, this module built on clinical pharmacy taught at the undergraduate level for the pharmacy interns and addressed training gaps in prescribing competence for the medical students and medical interns also enrolled on the module.

Table 1.

National Pharmacy Internship Program - Academic Modules and Aligned Program Learning Outcomes, Integrating Experiential Learning Core Competencies and ECTS Credits

The Virtual Learning Environment (VLE) provided the core vehicle for delivery of the academic content, harnessing the interactive capacity of online learning and overall permitting a blended-learning approach to the program. Learning communities of interns were established via the VLE as interns were able to connect online without having to be in the same place at the same time. This facilitated networking, problem solving, synchronous and asynchronous online discussion, and learning through the social domain.^19-21 Block activities in college were designed to supplement the online component and address topics that could not be effectively delivered online, such as orientation, first aid skills, communication skills, team dynamics, and project support sessions. Block activities comprised one full-time week in college, scheduled at the midpoint of the program.

The assessment strategy for the NPIP used a variety of modalities, mindful of validity and reliability, and the need to assess knowledge skills and performance, encompassing all levels of Miller’s pyramid.²² There were several assignments, including participation in case-moderated studies, evidence of compounding of extemporaneous preparations, medicine usage reviews, drug evaluations, and medicines information queries. Online MCQs were used for summative assessment of calculations and interprofessional prescribing science. All assessments were graded and contributed to the module grades (marks) for the academic award. Formative quizzes were also delivered via the VLE, with the facility for immediate feedback on quiz completion. A formative competence assessment and performance appraisal (CAPA) was conducted midway through the program. This consisted of an interview between the intern and a member of the academic staff to assess academic progress and competency development, with follow-up as necessary. The CAPA also afforded the opportunity to identify interns with difficulties and refer them appropriately for support.

The performance element in the assessment strategy was enabled by the development of the NPIP competency framework, and the behavioral descriptors which assisted with interpretation of whether intern performance met expected standards. Interns were appraised against the competence standards on three occasions in accordance with a five-point scale ranging from 0 to 4 (0=not encountered/applicable, 1=rarely, 2=sometimes, 3=usually, and 4=consistently; Table 2). The competency management feature of a bespoke e-portfolio allowed the interns to record self-reflection and preceptor appraisal of the competencies. On the final appraisal, interns were required to have obtained a level 4 rating on all applicable competencies to be invited to sit for the PRE. This was to ensure that graduates of the program were able to practice competently and fit for purpose, as required by the Education and Training Rules.⁴ The PRE was the high-stakes licensure examination designed to provide a holistic measurement of competency and was conducted by RCSI on behalf of the PSI. The legislative framework for the PRE is set out in part 5 of the PSI (Education and Training Rules 2008).⁴ The PRE consisted of MCQs on pharmaceutical calculations and interprofessional prescribing science, and a 12-station OSCE. The MCQs were standard set using a modified Angoff method, and the OSCE was standard set using Borderline Regression. The OSCE is a recognized, valid, and reliable method for assessing a student’s integration of knowledge and skills, which heretofore had only been employed at entry-to-practice in Canada.^23-24

Table 2.

Competence Standards Assessment Ratings for the National Pharmacy Internship Program Used by Preceptors to Assess the Competency of Pharmacy Interns Both Formatively and Summatively During the Program^a

The evaluation of the program was planned from the outset, with the authors of the PEARs report, to permit a longitudinal approach to program evaluation and comparison with the previous preregistration year. All graduates from the three schools of pharmacy in the Republic of Ireland (approximately 150 per year) were required to take the NPIP from 2009. The outcome measures for evaluation were an estimation of the feasibility of the PRE, with a focus on the costs of running the OSCE, the performance of candidates in the PRE, and the intern and preceptor reactions to the NPIP longitudinally.

The feasibility of the PRE was evaluated in 2009 with data pertaining to the costs of 146 interns undertaking the PRE. The OSCE was identified as being one of the most resource-intensive components of the PRE, and the costs estimated in accordance with an article by Reznick and colleagues entitled “Guidelines for Estimating the Real Cost of an Objective Structured Clinical Examination.”²⁵ The costs were estimated in the four phases of implementation of an OSCE, namely, examination development, examination production, examination administration, and post-examination analysis. The true costs of the actors, assessors, data input, catering, hire of room dividers, and consumables were recorded. The cost of services provided by RCSI, including venue hire and examination development, administration, and analysis were estimated in accordance with the time taken and hourly staff rates.

An in-depth analysis of the validity and reliability of the PRE was determined in 2009 with data from 146 candidates. Item analysis of the MCQs was conducted and the Cronbach alpha coefficient was calculated for the OSCE. Candidate demographics were retrospectively inserted into the master OSCE data set. Analysis was conducted by parametric and nonparametric ANOVA, as appropriate, to determine significant relationships. The overall performance in the PRE was recorded longitudinally from academic year 2009-2010 to academic year 2012-2013.

The interns’ and preceptors’ views of the NPIP and the PRE were obtained by survey on completion of the entire program on an annual basis to evaluate the program and inform continuous quality improvement. All interns were surveyed from academic year 2009-2010 to academic year 2012-2013, inclusive. All preceptors were surveyed from academic year 2009-2010 to academic year 2010-2011 and again in 2012-2013. Data are missing for preceptors from academic year 2011-2012 as in this year, the questionnaire focused on how preceptors could be better supported. Permission was given by the authors of the PEARs Report to use items from the PEARs instruments to allow comparison between the previous preregistration year and the internship program.³ The questionnaire comprised items eliciting yes or no responses and those requiring Likert-scale use, and also allowed for capture of free-text comments. The questionnaires were reviewed and transposed for electronic delivery using the online software SurveyMonkey (San Mateo, CA). Cross-tabular analysis of the responses to the questionnaires in academic year 2009-2010 was achieved using SPSS, v18 (Chicago, IL) and statistical evaluation was undertaken using the chi-square statistical test because of the ordinal nature of the data. A Mann-Whitney test was used to compare the data from the PEARs study with data obtained from this study. Significance was defined as p≤.05.

Evaluation of the program was exempt from investigational review board approval, but had to be conducted through the Evaluation Working Group, a subcommittee of the Quality Enhancement Office. This was required to uphold ethical considerations, including safeguards for the protection of anonymity and appropriate quality standards.

RESULTS

The feasibility of the PRE was investigated. The total (real) cost of conducting the mock and first PRE was estimated to be €105,451 (US $117,670), and the cost per candidate was estimated to be €722 ($806). The cost including faculty time donated and RCSI contributing services was €43,360 ($48,392), with a cost per candidate of €297 ($331).

The overall performance in the PRE was generally good and relatively consistent (Table 3). This is particularly impressive as candidates had to pass all three parts, the calculations MCQ, clinical MCQ, and the OSCE, independently. Over the period from academic year 2009-2010 to academic year 2012-2013, only five interns failed to achieve level 4 on all the competencies and were therefore ineligible to take the PRE. These candidates were offered additional support and opportunities for extended placement to achieve the requisite performance level. A further seven interns required additional time to complete their training for various reasons. Almost all interns who passed the PRE proceeded to register with the PSI to practice as pharmacists. Data analysis after the first year of the program indicated that there was no correlation between the MCQ and OSCE components overall, which was not surprising given that they were assessing different constructs. A Cronbach alpha coefficient of 0.6 indicated moderate reliability of the OSCE component. There was no evidence to suggest a difference in scores in accordance with interns’ age, gender, or whether their clinical placement was in a hospital or a community pharmacy.

Table 3.

Pharmacy Intern Performance on the Professional Registration Examination Taken as Part of the Royal College of Surgeons in Ireland Master of Pharmacy Degree Program

All interns who had completed the program for the first time were surveyed at the end of the program. The response rates were 59.6% (87/146) in 2009-2010, 62.3% (103/162) in 2010-2011, 63.6% (94/153) in 2011-2012, and 54.7% (87/159) in 2012-2013. The overall response rate for academic years 2009-2010 to 2012-2013, inclusive, was 59.8%. A majority of the respondents were female (74.4%, 258/347), and 90.4% (322/356) were less than 30 years of age. The majority (73.6%, 273/371) of the respondents undertook their clinical placement entirely within a community pharmacy (Table 4). The respondents were, therefore, reflective of the composition of the intern group.

Table 4.

National Pharmacy Intern Respondent Demographics as per Sector of Pharmacy Practice

Intern respondents were very positive about their overall internship experience. The vast majority (92.5%, 320/346) either strongly agreed (33.8%, n=117) or agreed (58.7%, n=203) that the internship had enabled them to develop the knowledge, skills, and overall competencies required for future independent practice as a pharmacist. Only 2.6% (n=9) disagreed and 0.3% (n=1) strongly disagreed with this statement (Figure 1). This is a significant improvement on the findings from the PEARs report whereby 78% (212/272) of respondents either strongly agreed or agreed, and 10% (n=27) disagreed and 2% (n=6) strongly disagreed that the preregistration training had enabled them to develop the knowledge, skills, and overall competencies required for future independent practice as a pharmacist.³ A Mann-Whitney test indicated the distribution of responses from the NPIP respondents was different from the PEARs responses (W=36880, p<.001, W=Wilcoxon rank-sum test).

Figure 1.

Intern respondents’ overall opinion of the National Pharmacy Internship Program 2009-2010 and 2012-2013 reflecting on the statement “taking everything into consideration I feel that my internship enabled me to develop my knowledge, skills and overall competencies required for future independent practice as a pharmacist.”

A substantial majority (78.7%, 273/347) of respondents also stated that they enjoyed their internship. This is a similar finding to PEARs where 81% (220/272) enjoyed the former preregistration year.³ This finding is set in the context of a consistent number of interns encountering personal difficulties, such as illness and difficult workplace relationships. Personal difficulties were encountered by 28% (24/86) in 2009-2010, 40% (40/100) in 2010-2011, 39% (36/93) in 2011-2012, and 36% (31/87) in 2012/13, with 35.7% (131/367) of the respondents reporting that they encountered personal difficulties overall. The results of the survey and in-house data showed that interns sought assistance from a range of identified sources.

Interns were asked to identify aspects of their internship that they would deem the “one best thing.” This was an open question. The majority of interns identified their placement experience as the best thing about their internship. Other themes that emerged as being the “best thing” for the interns included the academic program, structure to the year, the preceptor’s support, and patient interaction. The most frequent aspects identified as the “worst thing” were related to the workload and difficulty in achieving a work/life balance. Interns described “pressure” and “stress” associated with the 35 to 40 hours placement time per week as well as the additional academic commitments. One intern expressed the worst thing as “The workload from the masters on top of working a 40-hour week. I felt that I had a constant weight and burden with me.” Interns had mixed opinions regarding the benefits of having a Level 9 qualification. The majority recognized that there may be advantages in having a qualification that would be recognized overseas, but the view that it made little difference regarding job opportunities in Ireland was also frequently articulated.

Seventy-eight responses were received from the cohort of 182 preceptor pharmacists in 2009-2010, giving a response rate of 42.8%. Overall, pharmacy preceptors were very positive, with 87.7% (64/74) either strongly agreeing (25.7%, n=19) or agreeing (60.8%, n=45) that the internship program provides a sufficiently rounded experience to prepare them for a future as a pharmacist (Figure 2). This is again a shift in opinion from the preceptor opinion on the preregistration program where 70% (99/142) either strongly agreed (9%, n=13) or agreed (61%, n=86) that the preregistration year provides a sufficiently rounded experience as a foundation for the future.³ There was a similar pattern in academic year 2010-2011 (response rate 32.9%, 54/164) and again for academic year 2012-2013 (response rate 52.3%, 104/199) when 91.5% (43/47) and 87.3% (83/95) of respondents either agreed or strongly agreed that the internship program provides a sufficiently rounded experience as a foundation for a future as a pharmacist.

Figure 2.

Respondent preceptor pharmacist overall view of the National Pharmacy Internship program 2009-2010, 2010-2011 and 2012/13 reflecting on the statement “taking everything into consideration, I feel that the internship program provides a sufficiently rounded experience as a foundation for a future as a pharmacist.”

Preceptors were asked how strongly they agreed or disagreed with the statement: “Overall, I think the National Pharmacy Internship Program is an improvement on the pre-registration year.” The majority of respondents (76%, 54/71) in 2009-2010 either strongly agreed (33.3%, n=24) or agreed (42.2%, n=30) that it was an improvement. A minority of respondents (9.8%, n=7) disagreed that it was an improvement, but none of the preceptors strongly disagreed. Cross-tabulation with sector did not reveal any significant difference (n=70, p=.347). Essentially the same overall percentage (76.7%, 33/43) of respondents to the survey in 2010-2011 either strongly agreed (25.6%, n=11) or agreed (51.2%, n=22) that it was an improvement. This question was not asked thereafter.

Preceptors considered that the main strengths of the NPIP were that it was structured, standardized, challenging, and supportive, and that it rewarded interns with a Master’s qualification. Comments included: “It formalises the learning process and sets a standard for all throughout the country. Previously students were very dependent on the preceptor and their setting to have a good pre-reg year,” and “It gives the student a Masters. It demands all round ability, skills, and knowledge. The exam is broader, the competencies are useful for illustrating strengths and weaknesses.” Preceptors also identified areas for improvement. The primary concern was the workload for the intern. One comment was that there was a “huge workload and pressure on the interns, to the extent that they spent the entire year stressed out about checking off all the various things that needed to be done and viewing their work experience entirely through the prism of the RCSI course. This left them with little time to spend relaxing and becoming at ease with the role of a pharmacist and developing work experience naturally.”

DISCUSSION

The hypothesis was that the education and training provided by the NPIP was an improvement on the previous preregistration training and would produce competent pharmacists. The results show that the NPIP compares favourably with the preregistration year, from both the interns’ and preceptors’ perspectives, when compared across the metrics employed in the PEARs report.³

The NPIP has a number of key strengths. The program was designed to permit students to undertake a program of education and learning that would enable them to be competent pharmacists. The NPIP incorporated a national licensure examination in the form of the PRE, which gave assurance to the PSI with respect to an appropriate national standard for application for registration and entry-to-practice. Interns are not required to take a separate regulator-run national licensure examination after graduation and before first registration. There were significant costs associated with the OSCE component of the PRE. These were comparable to those reported in the literature and deemed justifiable for the outcomes achieved.²⁵

The NPIP was also designed to fulfil the requirements of Level 9 of the National Framework of Qualifications (NFQ), building on prior learning from the four-year bachelor programs (Level 8 of the NFQ). The structure is fully compatible with the Bologna agreement, in that the learning occurs during a second cycle after the first bachelor’s level degree, which is important for consistency and transparency in educational provision across Europe. The requirement for overseas graduates to have completed five years of education and training before being recognized as a pharmacist in the United States is fulfilled, thus allowing graduates qualifications to be recognized in that jurisdiction. The formal recognition of learning, through an educational award, is an advantage over some other programs provided internationally.

The provision of support services as part of the program addressed the concerns expressed in the PEARs report regarding personal difficulties and the “lack of any national backup scheme to the preceptor for student support.”³ The program was perceived by preceptors to have begun the journey in standardizing the experience for students from all educational institutions and across all sectors. The academic component not only addressed gaps in knowledge and skills, but advanced the educational provision for entry-to-practice pharmacists. The Baseline Survey of Community Pharmacies, commissioned by the PSI in 2009, reported that pharmacists in Ireland were willing to adopt new roles.²⁶ The education and training provided by the NPIP is an important first step in enabling an enhanced scope of practice for pharmacists, while maintaining quality assurance for regulators and the public.

Performance in the PRE was consistent with expectations. The PRE itself was evaluated and found to be moderately reliable. The data analysis was presented to the PSI with the recommendation to add one standard error of measurement to the cut score of the OSCE to reduce the risk of false positives. This was adopted and further ensures the competency of candidates who pass the PRE and are eligible to registration as pharmacists.

The program was, out of necessity, introduced over a very short timeframe with limited resources. The design and implementation of the program was undertaken with an increase of just 1.5 full-time equivalent staff in the School of Pharmacy at RCSI. There were challenges with the rapid development. Students and preceptors were expecting to be involved in a preregistration year and the new arrangements, compounded by perceived communication deficits, inevitably resulted in some initial intern and preceptor dissatisfaction. Community practice preceptors were, at that time, practicing through a restructuring of the contractual basis of services as part of a government response to adverse economic circumstances. There was no available funding to employ practice educators. These would have been appointees of the academic institutions whose role would have been to support interns and preceptors, and assist in enhancing quality assurance of training and training establishments. Ideally, planning for the design and delivery of such a new program of education needs to be timed so that all external stakeholders can be involved in the process.

The program underwent continuous quality improvement, based on intern and preceptor feedback, which informed an annual quality improvement plan. Attempts were made to incrementally reduce the workload. A major revision of the program was subsequently undertaken in academic year 2014-2015, which corresponded with RCSI being selected to run the program for additional years. This provided an opportunity for the program to be redesigned around a new national Core Competency Framework for pharmacists,²⁷ rebalance the academic workload, give due recognition to the hours associated with work-place based learning, make better use of technology enhanced learning, and pay more attention to the “learning-to-learn” element in developing a culture of self-directed learning, and increase the number of OSCE stations in the PRE to enhance reliability. The provision of a national program by one educational provider was, however, not regarded as a long-term solution. It was merely an interim measure until the establishment of a 5-year integrated program, as recommended by the PEARs report.³

CONCLUSION

The introduction of the National Pharmacy Internship Program provided a unique opportunity to move beyond student perception, evaluate outcomes at entry-to-practice and benchmark the quality of education. The program was perceived to be an improvement on the previous pre-registration year. The program quality assured pharmacy education outcomes at the entry-to-practice level on a national basis, and the award of a master’s degree provided recognition of the learning.

The findings from this evaluation are that the delivery of an academic component alongside experiential learning is valuable for learning. Account needs to be taken of the support required for both students and their preceptors, particularly during orientation and developing communities of practice. Attention should also be paid to the workload for both students and preceptors, as excessive workload is a source of stress. Developing student-support services, which meet the needs of students dispersed geographically, is essential. The feasibility of any proposal requires due consideration. The OSCEs, in particular, are expensive, but costs can be contained. Implementation of a comprehensive quality framework for experiential learning is even more resource intensive, but necessary for the provision of a quality assured, standardized, educational experience.