Use of SMART Learning Objectives to Introduce Continuing Professional Development Into the Pharmacy Curriculum

Abstract

Objective. To determine whether a 2-year continuing professional development (CPD) training program improved first-year (P1) and second-year (P2) pharmacy students’ ability to write SMART (specific, measurable, achievable, relevant, and timed) learning objectives.

Design. First-year students completed live or online CPD training, including creating portfolios and writing SMART objectives prior to their summer introductory pharmacy practice experience (IPPE). In year 2, P1 and P2 students were included. SMART learning objectives were graded and analyzed.

Assessment. On several objectives, the 2011 P1 students (n = 130) scored higher than did the P2 cohort (n = 105). In 2011, P2 students outscored their own performance in 2010. In 2011, P1 students who had been trained in online modules performed the same as did live-session trainees with respect to SMART objectives.

Conclusion. With focused online or live training, students are capable of incorporating principles of CPD by writing SMART learning objectives.

INTRODUCTION

As new technologies and therapies are continually introduced into practice, demonstration of competency is essential to the professional growth of healthcare professionals.¹ Traditionally, completion of prespecified hours of continuing education (CE) activities has been required as a way for pharmacists to demonstrate competency.² However, healthcare professionals often apply a nondirectional approach in obtaining CE hours to reach a prespecified state hour requirement.^1,3 Continuing professional development (CPD) is a supplement to traditional CE, providing a more reflective and directed approach to professional growth.⁴ The need for and benefits of CPD for pharmacists have been clearly demonstrated over the past few years. As a result, focus has shifted to introduction of CPD early in student-pharmacist education.

The CPD process involves a cycle of 4 steps (reflect, plan, act, and evaluate), with documentation undertaken at each step.⁴ By following this cycle, individual practitioners are ultimately responsible for assessing their learning needs, making plans to meet their objectives, executing their plans, and evaluating their actions. These steps are continually recorded in an electronic CPD portfolio as a way to document practitioners’ progress.⁵ The CPD cycle used in this study was adapted with permission from the Ontario College of Pharmacists and follows the same approach.^6,7

Positive results have been reported with pharmacists trained in the CPD approach. In a 10-month study of licensed pharmacists employed at a health maintenance organization (HMO), a higher percentage of those randomized to CPD vs. CE reported that they had better interactions with other healthcare providers and initiated practice changes as a result of their educational activities.^8,9 Additionally, compared with CE participants, more CPD participants reported that their educational activities improved patient-care changes, professional knowledge, skills, and attitudes/values. In a study involving 120 community pharmacists using a condensed CPD training program, a majority of participants responded that the educational activity enhanced their knowledge and skill levels and that they would implement CPD at their practice sites.⁶ Further, a 5-state CPD pilot program for practicing pharmacists in Indiana, Iowa, North Carolina, Washington, and Wisconsin concluded that “with appropriate training and support, pharmacists can use a CPD approach to their lifelong learning and professional development.¹⁰

The North Carolina Board of Pharmacy became the first pharmacy board in the United States to approve CPD for pharmacists. After completion of required education and training programs on the method, pharmacists can use CPD as a way to renew their licenses in North Carolina. There are now more than 200 North Carolina pharmacists who apply CPD in practice and use this in place of solely reporting CE hours for licensure renewal.¹¹

While positive results for CPD have been demonstrated among practitioners, early exposure to self-reflection training may be beneficial to promoting wide-scale implementation. In a qualitative study of Canadian pharmacists’ attitudes, behaviors, and preferences, participants were concerned about their lack of ability to self-identify learning needs and were worried about time constraints. Participants also agreed that peer support is vital in adopting a CPD learning model.¹² By introducing CPD early in their training, student pharmacists can develop skills for self-appraisal and obtain peer support through interaction with classmates throughout the process.

Of the few studies that have evaluated CPD among student pharmacists, several have shown the benefits of implementing self-reflection exercises in student course work. In 2010, Briceland and colleagues studied the use of electronic portfolios among students during advanced pharmacy practice experiences (APPEs), concluding that the reflective essay component “proved to be a useful vehicle to demonstrate achievement of ability-based outcomes.”¹³ Through the self-reflective essays, students also recognized the importance of lifelong learning. Motycka and colleagues state that in order to advance and use self-assessment skills, it is essential to validate the appropriate models, provide educators with theoretical background, and “embrace the culture in our educational programs where self-assessment is an essential element to successful professional practice.”¹⁴ McMillan and colleagues also argue that “when students set goals that aid their improved understanding, and then identify criteria, self-evaluate their progress toward learning, reflect on their learning, and generate strategies for more learning, they will show improved performance with meaningful motivation.”¹⁵ Self-reflection in pharmacy school curriculum is clearly a step in the right direction and aligns with ACPE’s 2007 Accreditation Standards and Guidelines, which recommends self-directed learning for students.¹⁶

Based on this demonstrated need for self-assessment early in the curriculum, introducing CPD training into the pharmacy school curriculum warrants additional research as an appropriate next step. Incorporating CPD training into pharmacy school curriculum may further advance student pharmacist development and encourage the practice of self-reflection and lifelong learning. In a 2010 commentary, Janke stressed the importance of shifting focus to training students on CPD: “Students can become more versed in self-assessment, reflection, and planning and documentation strategies.”¹⁷ Janke also discussed the importance of coaching and support throughout the process.¹⁷ Implementation of CPD will require appropriate training of student pharmacists and coaching of educators to provide students any assistance they may need during the CPD training process.

A study conducted at the School of Pharmacy and Pharmaceutical Sciences, University of Central Lancashire, United Kingdom, attempted to engage students enrolled in a master of pharmacy degree program with a CPD activity similar to that for pharmacists.¹⁸ While few students fully grasped the process, the authors concluded that “there is a need for students to be encouraged to take ownership of their undergraduate learning, to gain confidence in self-assessment, and to increase the value they place on reflection.”¹⁸ Advanced students, such as doctor of pharmacy (PharmD) students in the United States, may find the CPD process easier to grasp.

There are no US studies that have evaluated the effectiveness of a training program in helping pharmacy students write learning objectives and implement a CPD approach early in the pharmacy curriculum. This study fills this research gap by examining the effectiveness/utility of a CPD training program (online and live) in helping first- and second-year pharmacy students write SMART (specific, measurable, achievable, relevant and timed) learning objectives and implement a CPD process as they progress through experiential training.

Our earlier study of first-year (class of 2013) PharmD students at the University of North Carolina Eshelman School of Pharmacy comparing live vs online CPD training found that, with focused training, PharmD students are capable of implementing principles of CPD.⁷ The second year of this study incorporated a revised method of educating students on how to write SMART goals and evaluated how changes in training strategy affect students’ abilities to write these goals.⁷ The hypothesis of this study was that PharmD students would show improvement writing SMART goals and that CPD would be incorporated throughout the PharmD curriculum.

DESIGN

The purpose of the study was to evaluate a live and online training program for pharmacy students on the implementation of CPD principles (reflect, plan, act, and evaluate) by having them write SMART learning objectives prior to their hospital and community introductory pharmacy practice experiences (IPPEs). The investigators hypothesized that if students used the CPD process to prepare their own learning goals and objectives prior to each summer experience, they would develop habits that would lead to lifelong learning after graduation.

To achieve SMART goals, students must be precise about desired outcomes, quantify objectives using Bloom’s taxonomy, ensure realistic expectations, align with practice and/or organizational goals, and state when the goal will be achieved. In this study, which included PharmD students from the classes of 2013 and 2014, student achievement of SMART goals was evaluated using a standardized rubric.⁷ In preparation for the study, 2 instructors from the school’s Division of Pharmacy Practice and Experiential Education who had been trained during the North Carolina ACPE CPD pilot programs were chosen to serve as CPD mentors.¹⁰

The school’s hospital and community IPPEs are month-long experiences that take place in the summer following the first and second years, respectively. The primary intent of the IPPEs is to facilitate students’ continuing professional development in the context of the hospital and community pharmacy practice settings. Through structured activities and assignments, students build on knowledge and skills developed in the first and second years of the pharmacy curriculum. Students continue to explore the concepts of professionalism and shared accountabilities for healthcare outcomes; formulate a personal philosophy of and approach to professional practice; expand drug and disease knowledge; and develop practical, critical thinking and lifelong learning skills. The hospital IPPE is the first practice experience that students encounter in the curriculum; therefore, it is considered a good time to introduce CPD into the pharmacy curriculum.

Year 1 of Implementation

The class of 2013 was introduced to the same concepts of CPD during the spring 2010 semester using live training sessions or online training modules. All students (n = 154) were instructed to complete a CPD 101 Webcast developed by ACPE and then attend a 1-hour live classroom presentation.⁸ The CPD 101 Webcast, which is available online, describes concepts and components of CPD, the need for a new approach to continuing education, and compares traditional CE with CPD. During the presentation, students were educated on the definition of CPD, the CPD process, the North Carolina CPD story, lessons learned so far, strategy for dissemination and adoption, and advantages of using CPD to maintain competence in pharmacy school. Students were then offered the opportunity to volunteer to participate in a live CPD training module or continue with online training.

Live training consisted of a 3-hour lecture with discussion and active-learning opportunities. Sixteen of the 154 students completed the live training. Following live training, students were expected to: (1) review the CPD process and learning plan; (2) refine their learning objectives, learning plan, and overall CPD process; (3) discuss an effective documentation plan for learning activities; and (4) list tips for successfully implementing CPD in their practice site. Additionally, 2 faculty mentors were available to provide students with feedback on their initial attempts at writing SMART learning objectives.

The online training involved 3 hours of Webcasts available on the ACPE Web site. Online training was completed by 138 students. Additional Webcasts included (1) Inventory of Learning Styles, (2) Using Reflection to Create a Learning Plan, and (3) Act, Evaluate and Record your CPD. Additional details have been previously described.⁷

While the live CPD training module has been effective for pharmacists, it was offered to students on a voluntary basis because of the challenge of training more than 100 students in a short timeframe prior to the start of summer practice experiences.⁶ CPD Webcasts on the ACPE Web site were available to students who opted not to participate in live training. After completing training and prior to their summer hospital IPPE, students submitted SMART goals on Blackboard (Blackboard Inc., Washington, DC), the institution’s online course-management program. After completing their practice experience, students submitted learning-activity worksheets on Blackboard for each SMART objective.

Year 2 of Implementation

After the success of the training modality in the first year, the investigators explored whether it would be reproducible or improved upon during the second year of the study. In the second year, a 1-hour presentation was given to the class of 2013 students during the spring 2011 semester. The goal of this presentation was to provide results of the CPD intervention from the previous summer and to discuss how to effectively write SMART goals. A major finding from the previous year was that students struggled with writing measurable objectives. As a result, the presentation given to students in the second year focused more extensively on using Bloom’s taxonomy to write measurable objectives.¹⁹ Students were shown the taxonomy pyramid and verbs associated with each level. They were also trained on verbs to avoid and given examples of SMART goals. An example of a SMART goal presented to the students was to “Implement USP 797 in designing a clean room and how its regulations are enforced during normal operations by July 31, 2011.” At the end of spring 2011, students from the class of 2013 (n = 157) were instructed to write and submit SMART goals for their community IPPE and learning-activity worksheets after completion of the practice experience.

Students from the class of 2014 (n = 152) were introduced to CPD in year 2 of the project. During the fall of 2010, a 1-hour introductory presentation was given to the class, with the objective of defining CPD and discussing the advantages of using CPD to maintain competence in pharmacy school. In the spring of 2011, another 1-hour presentation was given to briefly discuss results of CPD intervention from the previous summer with the class of 2013 cohort and explain how to effectively write measurable goals using Bloom’s taxonomy. Students were advised but not required to complete the ACPE modules on CPD. Students from the class of 2014 were then instructed to submit their SMART goals on Blackboard prior to the start of summer hospital IPPEs and their learning-activity worksheets upon completion of their practice experience.

Documentation

Students were given a portfolio, consisting of the education-action plan and learning-activity worksheets. The portfolio also included a section for reflection on past classroom lectures and clinical practice and a final section to document recommendations for future classroom training and pharmaceutical-care laboratory activities. The education-action plan was a tool to help students write SMART goals in preparation for their IPPE, while the learning-activity worksheets allowed students to document progress with each SMART goal after completing their practice experience. This portfolio was adapted from the North Carolina CPD portfolio for student purposes.^11,20

First-year (P1) and second-year (P2) students submitted their CPD portfolios and SMART objectives to Blackboard before their hospital and community IPPEs, respectively. The portfolio included a portion for reflection on past lectures and practice, an education-action plan (student-learning plans), complete with SMART learning objectives. Each student’s information from Blackboard was downloaded and all data were de-identified prior to analysis and summary.

The following cohorts were used in the comparisons: the P1 2010 group trained in live sessions (n = 16), the P1 2010 group trained in online modules (n = 138), the P1 2011 group trained in online modules (n = 152), and the P2 2011 group trained in online modules (n = 157). Pairwise comparisons of SMART objectives were conducted among the cohorts. All analyses were performed using SAS statistical software, version 9.1 (Cary, NC). This project was approved under exempt status and a waiver for informed consent was granted from the UNC Institutional Review Board, as the data under study were already a component of students’ curricular requirements.

EVALUATION AND ASSESSMENT

Study methods included the evaluation of student-learning objectives. An example of a SMART learning objective is: “By May 2013, compare and contrast anticonvulsant drugs used to treat status epileptics.” A grading rubric (modified from a published rubric²⁰ ) was used to assess students’ education-action plans. The rubric uses performance ratings of satisfactory, work in progress, and unacceptable. For consistency, pre-identified student SMART learning objectives were scored using the modified rubric described by Tofade and colleagues.^7,20

Points for each of the 5 SMART sections were summed for a grade out of 100 possible points. These grades were then tabulated and analyzed using descriptive statistics (mean, standard deviation, and variances). Comparisons between the online cohorts were of sufficient size and characteristics for the use of student t tests. F tests for the equality of variances of these comparison groups were performed prior to conducting the t tests. Where the groups had significantly different variances, the Satterhwaite method was used to adjust the student t tests and account for the unequal variances. Comparisons between any cohort and the cohort of students trained in live sessions were made using a Wilcoxon 2-sample exact. Detailed results obtained from the learning-activity worksheets from the first year of the study have been described in a prior publication. They are not presented here because they exist in qualitative form only.⁷

Of the year-1 study cohort (n = 154), 136 (88.3%) submitted an education-action plan by the deadline: 14 (87.5%) of the 16 live-session trainees and 122 (88%) of the online-module trainees. In year 2 of the study, 105 (67%) P2 students and 130 (86%) P1 students submitted an education-action plan for evaluation. The mean scores and standard deviations for each section of the SMART learning objectives are presented in Table 1. The distribution of average scores and the scoring rubric are available from the corresponding authors. The p values for the cohort comparisons are presented in Table 2. P1 2011 students differed from the P2 students in every aspect except for “Relevant” and “Timed” at the 0.05 significance level. In each case, except for number of objectives, the P1 2011 students scored higher than did their P2 cohorts. Similarly, the P2 students’ scores differed from their 2010 scores in all aspects except “Timed” and number of objectives. P2 students outscored their own 2010 P1 data in each case except for number of objectives.

Table 1.

Pharmacy Students’ Scores^a on SMART Learning Objectives After Completing Either Online or Live Training, Mean (SD)

Table 2.

Comparisons of First- and Second-Year Pharmacy Students’ Achievement of SMART Learning Objectives

Wilcoxon sum rank tests showed significant differences between the P2 students and the 2010 P1 cohort trained in live sessions in the areas of “Specific” and number of objectives chosen. This finding was consistent with the results from 2010.⁷ In contrast, the 2011 P1 cohort differed significantly from the live-session trainees only in the “Achievable” category. The 2011 P1 online trainees’ ability to write SMART learning objectives was the same as that of the live-session trainees but they scored significantly higher in achievability of their objectives.

Several students stated that they found the SMART-learning objective exercise useful as they planned for their practice experiences. Combined submitted data from the 2010 and 2011 cohorts highlighting the student’s perception and evaluation of the course were reviewed. Only 7.8% (n = 12) of the study cohort completed the post-CPD training survey tool. Because of the poor response rate and low power to generalize, we choose not to report it here. Written feedback on the submitted survey instruments was generally positive.

Feedback on the CPD process from the class in the first year of the study was mostly positive. In the second year, representatives of the class of 2013 cohort expressed some concern regarding the exclusive use of Bloom’s taxonomy verbs in developing their plans. They suggested that the lecture be presented earlier in the fall semester to allow more buy-in during the preparative session on SMART objectives in the spring. Faculty comments about the CPD sessions were as follows: (1) excellent use of active-learning strategies by instructor; (2) instructor is easily able to communicate enthusiasm for CPD; (3) quiz exercise was effective in holding students accountable for their own learning; (4) consider adding additional real-life examples of how to create a CPD portfolio to add clarity; (5) consider sequencing instruction so that presentation on how to write SMART objectives precedes assignment to self-reflect/write SMART objectives; (6) end with a small group session to review students’ objectives; (7) sensed a level of discomfort with self-reflection among P1s and P2s because self-reflection is not routinely required in other curriculum courses; and (8) a majority of the P3s did not see the benefit or value of CPD; thus, acknowledging this up front with students may be useful. Other than time, minimal resources were needed to execute the teaching innovation. Two professors provided 3 hours of training for the live-session trainees during the first year. A 1-hour lecture was presented to explain the details and purpose of the CPD expectations to the 2010 cohort. Only 1 faculty member was needed for the second year. Thirty minutes of class time were allotted for both 2011 cohorts to explain the results of the first-year study and teach how to better write SMART learning objectives. Other time was required to create the initial PowerPoint slides and revise them in the second year (1 hour each year was the total estimated time commitment). For easy access, all materials for the students were provided in electronic format on Blackboard. The completed portfolios (totaling 750 pages) were printed by the institution at no direct cost to the school.

DISCUSSION

The purpose of the intervention in year 1 was to introduce CPD into the curriculum in accordance with ACPE standards. At the end of year 1, a major gap in the students' ability to write SMART learning objectives emerged. The adjustment to the teaching strategy of providing the online CPD Webcasts, incorporating specific examples of SMART learning objectives in class, and working through each example using the Bloom's taxonomy of verbs to make them measurable increased student scores on how to write SMART learning objectives. The use of online training modules and electronic portfolio submissions made the CPD program much more convenient to administer and averted the need to schedule multiple small-group sessions for each cohort. The investigators believe this translated to significant savings in class time and faculty resources. Our original expectation was that the live-session trainees would outperform all the other cohorts. However, the 2011 P1 cohort outperformed the live-session trainee group in several aspects of the SMART learning objectives, though only the achievable component was significant (Tables 1 and 2). The P2 2011 cohort improved their original scores compared with those of the previous year, suggesting the success of the adjusted teaching strategy (Figure 2).

The lower performance of the P2 2011 cohort compared with that of the other cohorts is intriguing. The poor performance of the P2 2011 cohort in the previous year may have demotivated them in the second year of the study, leading them to put forth a minimal effort. This also may explain the lower number of submitted portfolios from this cohort. What was consistent with the previously published manuscript by Tofade and colleagues⁶ was the performance of pharmacists using CPD and the reported struggle on their ability to write SMART learning objectives.^6,21 The current findings show that, with proper direction, students are capable of writing SMART learning objectives. The practice of writing SMART learning objectives can lead students to form a habit of reflecting and planning prior to attending lectures or engaging in experiential learning, which may eventually evolve into practicing CPD throughout the pharmacy curriculum and after graduation.

The main limitation of this study was our inability to determine how many of the students actually viewed the Webcasts on the ACPE Web site. At the time of the study, there was no tracking in place to provide that information. It would be useful to know whether the Webcasts were a strong contributor to the success of the program or if the live classes alone would have been sufficient for student understanding of the CPD process.

At this point, we can integrate CPD into the curriculum by repeating the online modules and training classes for future P1 and P2 cohorts and continuing to request that students submit a CPD plan and portfolio prior to beginning their experiential learning. The plan will be modified in the final year to request students to submit a CPD plan and updated portfolio prior to each practice experience to foster the habit of writing SMART learning objectives. The main advantage of this process is to develop a habit of lifelong learning in all students while maintaining compliance with ACPE standards using a feasible process at reasonable expense. We believe this study provides some progress in CPD research and supports the investigators' suggestions to incorporate CPD early in the curriculum.^17,22 It will be important to concurrently emphasize the CPD process to preceptors so they can also serve as mentors and models to the students during their practice experiences. Our plan is to conduct an assessment to see whether the CPD process is actually becoming easier for learners as they progress through the pharmacy curriculum and to evaluate the depth of learning and value to learners after each practice experience. We will also assess the perceived impact on the overall curriculum. Based on the results of the study and benefits subscribed to CPD, we hope to make the CPD plan and portfolio mandatory instead of optional. CPD lecture and training will continue in the curriculum and courses will be submitted to the school’s curricular strategic-planning group to ensure proper placement in a more influential location in the curriculum to allow for wider application to learning throughout the 4 years of pharmacy school.

Information on how to implement CPD training is being shared with colleagues at other colleges and schools of pharmacy nationally and internationally. Printing costs should be factored into the overall cost of implementing CPD training where applicable. Based on an estimated 750 pages at 9 cents per page, printing cost to institutions could be as much as $37.50 per class if hard copies are used. With efficient training of students, CPD is anticipated to become a regular component of the pharmacy school curriculum in the United States, thus achieving ACPE’s goal of having student self-reflection and maintenance of performance portfolios incorporated into pharmacy education. With this process, we hope that student pharmacists will gain an appreciation for CPD and use this program throughout their careers.

SUMMARY

The P1 2011 online trainees outperformed the live-session trainees in writing SMART learning objectives and significantly outperformed them in the achievability of their objectives. The P2 2011 cohort significantly improved in their ability to write SMART learning objectives after changes to the training were made in year 2 of the study. Despite its limitations, this study showed that focused training (live or online) enables students to incorporate principles of CPD by writing SMART learning objectives during their IPPEs. The results of this study will be shared with students and faculty members, and the CPD process will be implemented throughout the pharmacy curriculum as directed by the curriculum committee and subsequently among preceptors.