Preceptor Perceptions of Pharmacy Student Performance Before and After a Curriculum Transformation

Abstract

Objective. To explore preceptors’ perceptions about the performance of undergraduate pharmacy students during experiential placements in Australia, before and after curricular transformation.

Methods. Using a semi-structured approach, we interviewed 26 preceptors who had recently supervised students who took part in the transformed curriculum and students from the previous curriculum. A directed content analysis approach was used to analyze the transcripts.

Results. Preceptors described students from the transformed curriculum as having improved professional skills, behaviors, and attitudes and as having an increased ability to perform clinical activities compared to students of the previous curriculum. Preceptors also perceived that students in the transformed curriculum had improved clinical knowledge and knowledge application. They less frequently expressed that students in the transformed curriculum had lower-than-expected knowledge levels.

Conclusion. The results of this study suggest that curricular transformation with a focus on skill-based and active learning can improve the performance of pharmacy students in terms of their professional behaviors and attitudes, skills, knowledge, and clinical abilities, as perceived by preceptors.

INTRODUCTION

Health professions education seeks to equip students to improve health outcomes for individuals and populations in a changing world.¹ To better prepare future health professionals, educators must transform how we teach² and ensure that what we teach encompasses the skills, knowledge, and professional behaviors needed to achieve contemporary and forward-facing practice competencies.^3,4 Recognizing this charge, the Monash University Faculty of Pharmacy and Pharmaceutical Sciences undertook curricular transformation to create a new pharmacy degree program with a greater emphasis on active learning, experiential education, and development of key skills, including oral communication, problem-solving, reflective practice, and teamwork.^5-11 Although the admission process remains the same as for the previous curriculum, in the new program students begin with the Bachelor of Pharmacy (Honors) degree program and then transition into the Master of Pharmacy program. For this reason, we call the new degree a vertical integrated masters (VIM). Table 1 outlines some key differences between the two curricula.^12,13

Table 1.

Key Differences Between the Vertical Integrated Master’s Degree Program and the Previous Curriculum

Theoretically, students in the VIM and similarly transformed competency-based curricula should have improved professional skills, behaviors, and abilities to perform clinical activities.¹⁴ As many schools, faculties, and colleges of pharmacy have been, are, and will be reforming their curriculum,^15,16 it is crucial for educators to share their evaluations and insights across the Academy. Evaluating curriculum changes like these can allow other health professions educators to learn what works and, potentially, to explore how and why.

To comprehensively evaluate a curriculum, educators should use data from a range of sources including assessment performance, graduate outcomes and workplace performance, and student perceptions.^14,17-19 This study represents one part of a larger curriculum evaluation that used multiple sources of evidence. Previously, researchers have evaluated health professions curricula in a variety of ways, mainly by comparing student perceptions of learning^20,21 and preparedness for practice^22-25 and by interviewing graduates about the effectiveness of their program.^26,27 For example, graduates of reformed skills-focused medicine and pharmacy curricula had higher self-assessed preparedness for practice than graduates of the previous curricula at the conclusion of the degree^22,23,25 and six years after graduation.²⁴ However, graduates from traditional medical programs felt more prepared for basic science topics.²⁴ Heiman and colleagues²² also compared academic outcomes, finding no difference between current and former curriculum students for medical licensing examination scores. Less commonly, researchers have evaluated health professional curriculums by collecting perceptions of preceptors. A rare example is Watmough and colleagues’²⁸ study of medical preceptors, who believed that graduates of a reformed curriculum were better prepared for practice.

We evaluated the Monash Pharmacy curriculum from the perspective of recent preceptors, using interviews to explore their perceptions about undergraduate students from the VIM and the previous curriculum based on their performance during experiential placements. Our research objectives were to determine what performance differences exist between the two cohorts in terms of their oral communication skills, professional behaviors, knowledge levels, and abilities to perform clinical activities, and to identify in what ways VIM students could improve their clinical and professional skills. Placements are an ideal environment to capture student performance in real practice settings, and preceptors are well placed to observe and judge student performance, as they are familiar with practice-relevant competencies and supervise many students. Compared to previous literature that had evaluated a curriculum transformation by interviewing medical preceptors,²⁸ this study included pharmacy preceptors and evaluated different curriculum changes (ie, earlier placements, flipped-classroom curriculum, skill development). This approach to evaluating the impact of curriculum changes on student performance in real practice settings may be useful to other universities undertaking curricular transformations, and our findings may be of interest to those seeking to improve similar curricular elements.²⁹

METHODS

This study was part of a larger, ongoing programmatic evaluation of the curriculum transformation.^6,18 Overall, we employed a pragmatic evaluation paradigm by using mixed methods and focusing on data sources useful to current and future stakeholders,¹⁷ and we matched methods to the specific questions, outcomes, and purposes of each study. This study represents one qualitative in-depth investigation of how Monash Pharmacy students performed on placements before and after the curriculum transformation.

This study was approved by the Monash University Human Research Subjects Ethics Committee. The study population comprised registered pharmacists who had precepted Monash undergraduate pharmacy students during experiential placements in Australia between 2017 and 2019. Recent preceptors were selected using purposive sampling^30,31 and were emailed an invitation, explanation of the study, and consent form. All study participants had either already supervised students from the VIM or had, so far, only supervised students from the previous curriculum but were expecting to have supervised VIM students by the time of the planned second-round interviews six months later. Preceptors who had not yet supervised VIM students and were not expecting to supervise VIM students in the next six months were excluded. In general, 20 or more interview participants is a sufficient sample size to achieve saturation.³² A total of 29 eligible preceptors consented to the study and self-reported the number of students they had supervised as precise or best-estimate figures (Table 2).

Table 2.

Study Participants,^a Practice Sites, and Number and Year Levels of Students Precepted

We conducted semistructured interviews³³ to allow preceptors to express themselves in their own words and so that we could compare their experiences and perceptions about students across the interviews.^34,35 Open-ended, nonleading questions (Table 3) were used to allow preceptors to freely express their observations about students in terms of not only their skills, knowledge, and professional behaviors, which is what we intended to study, but also any other attributes that preceptors considered relevant and that could enrich our findings.³⁵

Table 3.

Guide Used to Interview Study Participants (Preceptors)

Two investigators conducted one-on-one telephone interviews with the participants. One of the investigators piloted the use of the interview guide with the first participant, then both interviewers listened to the recording and agreed on a common approach to using the guide before conducting further interviews. First-round interviews were conducted in October and November 2019, when the first cohort of VIM students was in the third year of the program. In the first-round interviews, many preceptors compared their experience with fourth-year students from the previous curriculum to the third-year VIM students. We chose to capture interviews at this time to limit recall bias³⁶ about their experience with previous curriculum students. Some participants (n = 5) participated in second-round interviews in April 2020, by which time participants were expected to have supervised a greater number of VIM students, including some VIM students who would, by then, be in the fourth year. The remaining preceptors (n = 24) declined a second-round interview due to changes in health system priorities and workload related to the COVID-19 pandemic.³⁷ An electronic gift card for A$30 was sent to each study participant after all interviews were completed.

Overall, 29 first-round and five second-round interviews were conducted, each lasting approximately 45 minutes. Audio recordings of 25 first-round interviews and five second-round interviews were stored securely. Due to a smartphone recording app malfunction, four first-round interviews were not recorded and, therefore, not included in the study. Audio recordings were transcribed and anonymized using Rev (Rev.com Inc) and Otter.ai (Otter.ai Inc). We uploaded the interview transcripts to a qualitative data analysis software program, NVivo 12 (QSR International LLC).

Our analysis followed a directed content analysis approach.³⁸ Two investigators created an initial list of coding categories covering ideas relevant to our research questions: student competencies of interest (oral communication skills, professional behaviors, knowledge level, ability to perform clinical activities), preceptor response type (judgments of previous curriculum and VIM students, and prompted or unprompted comparisons between the two), preceptor response valence (below expectations, at expectations, or above expectations), and how VIM students could improve (improvement for VIM). The names of codes representing expected subcategories (eg, knowledge retention or application, ability to take medication histories, global assessment of communication skills) were agreed upon and compiled in a codebook.

During a coding calibration phase, two investigators established the codebook. The two investigators met frequently after independent coding sessions to resolve interrater discrepancies, propose new codes based on the data, and further clarify the meanings of the predetermined codes until the entire codebook could be used reliably. Coding calibration was then completed by five investigators (CF, KML, DL, EH, KYL) with a subset of the data set. After calibration, the data set used for calibration and the rest of the data set were treated in the same way (ie, each transcript was coded by at least two investigators independently). After independent coding was completed, coders met to calculate simple interrater agreement before resolving discrepancies.

After coding, matrices were used to synthesize the data according to our research questions. We used the matrix coding function of NVivo to organize the text data into relevant segments. For example, we used the codes “VIM students,” “above expectations,” and “oral communication” to synthesize all quotes related to when a preceptor thought VIM students had performed oral communication above their expectations. Most of our synthesis was accomplished by making three intermediate representations³⁹: large matrices for preceptor perceptions of previous curriculum students, VIM students, and comparisons between the two programs. The matrices were organized as competency type (eg, knowledge) by preceptor response valence (eg, below expectations). Matrices also included descriptive text summaries and example and representative quotes for each specific combination of competency and response valence codes, copied and pasted directly from the coded transcript data in NVivo.

To capture the prevalence of these perceptions, we counted the number of preceptors (p) who gave each type of response (eg, “students had difficulty applying their knowledge to patients in the hospital setting”) and the total number of segments or quotes (q) representing each type of response across all participants. Because all preceptors were asked to discuss each competency, and each preceptor could give multiple response types for each competency (eg, mentioning a single student as “above expectations” for knowledge and the rest as “at expectations”), one preceptor could be included in the p count for multiple response types. However, for each individual response type, one preceptor was only ever included once in the p count, even if they gave the same response type in both first- and second-round interviews. Meanwhile, an individual preceptor could mention a particular response type multiple times (eg, spontaneously discussing oral communication skills that were above expectations when asked about knowledge) and contribute multiple instances of q for the same response type. Therefore, we considered p and q together, alongside the transcript data. We chose to report these frequencies in our results for descriptive purposes.

In our analysis, the frequency of appearance of any particular response category alone did not determine its importance⁴⁰ and cannot be generalized beyond the study sample.⁴¹ However, conducting a frequency analysis of the qualitative data by counting p and q enabled us to recognize patterns across participants, verify the meanings we arrived at in our analysis, and ensure that we did not discount or overemphasize any data.^42,43 We also used these counts to make the basis of our interpretations clearer^30,41 by assigning the p and q counts frequency labels,^42-44 namely “generally” (all or all but one), “typically” (more than half, up to the cutoff for “generally”), “occasionally” (three or more, up to the cutoff for “typically”), and “rarely” (one or two). For emergent themes that were only discussed by some participants, we avoided using defined frequency labels, which could misleadingly suggest the generalizability of findings to the whole study population,⁴¹ and instead used actual numbers of respondents or more general terms (eg, “some,” “other”).

RESULTS

In total, we analyzed the transcripts of 30 interviews with 26 preceptors: 14 community pharmacists and 12 hospital pharmacists, representing 14 community and 11 hospital placement sites (Table 2). Of these, 25 were first-round interviews (reflecting preceptor experiences with previous curriculum students and first-, second-, and third-year VIM students but no fourth-year VIM students), and five were second-round interviews. All five second-round interviews captured preceptor experiences with fourth-year VIM students, and one of these interviews was with a preceptor who had not supervised any VIM students at the time of their first-round interview but had done so since.

Two preceptors (Participants 9 and 21) who were only interviewed in the first round had only supervised students from the previous curriculum, and one preceptor (Participant 10) had only supervised VIM students. Although these participants could not draw direct comparisons between the cohorts, we determined that their perceptions of each cohort were still relevant to our research questions and analyzed their interview data accordingly. The interrater simple agreement on coding was 84%.

All participants were registered pharmacists practicing in metropolitan or regional locations in Victoria, Australia. Of the 12 hospital-based preceptors, five had worked with Monash students in a one-on-one clinical supervision capacity only, two had only been site placement coordinators and, therefore, supervised multiple small groups of students who also worked with other pharmacists at the same hospital, and five had supervised students in both of these capacities. We considered that this study population represented an adequate heterogeneity in terms of practice setting and extent of precepting experience.

Figures 1 and 2 present the p and q counts for competency types that all participants discussed: oral communication skills and professional behavior and attitudes (Figure 1) and knowledge level and ability to perform clinical activities (Figure 2). Figures 3 and 4 present p and q counts for themes that emerged during interviews with only some participants: therapeutic clinical reasoning and inquiry skills (Figure 3) and confidence (Figure 4). These numbers alone are not results and cannot be interpreted without the qualitative data, because a q of 20, for example, may represent 20 quotes from one participant or one quote each from 20 participants. Rather, these counts are presented for transparency and to complement our analysis by illustrating the patterns we examined, as outlined in the methods.⁴³

Figure 1.

Number of preceptors (p) and quotes (q) expressing preceptor perceptions of the oral communication skills and professional behavior and attitudes of pharmacy students who were in the previous curriculum or VIM as being below, at, or above preceptor expectations.

Figure 2.

Number of preceptors (p) and quotes (q) expressing preceptor perceptions of the knowledge level and ability to perform clinical activities of pharmacy students who were in the previous curriculum or VIM as being below, at, or above preceptor expectations.

Figure 3.

Number of preceptors (p) and quotes (q) expressing preceptor perceptions of the therapeutic clinical reasoning and inquiry skills of pharmacy students who were in the previous curriculum or VIM as being below, at, or above preceptor expectations.

Figure 4.

Number of preceptors (p) and quotes (q) expressing preceptor perceptions of the confidence of pharmacy students who were in the previous curriculum or VIM as being below, at, or above preceptor expectations.

We explored preceptor responses relating to each aspect of our first research question: What differences exist between the two cohorts in terms of their oral communication skills, professional behaviors, knowledge levels, and abilities to perform clinical activities? Preceptors described these differences not only in response to the direct question about how the two cohorts compared and differed (Table 3), but also spontaneously when answering questions about each cohort individually.

Preceptors more frequently described being impressed with the oral communication skills of students in the VIM than those of students in the previous curriculum. Although preceptors generally reported satisfaction with the oral communication skills of some previous curriculum students, they also typically gave multiple descriptions of their expectations not being met; one preceptor stated, “so they might not link ideas very well, or highlight particular points and do so in a way that seems to make the patient concerned about their health” (Participant 9, hospital). Preceptors who were dissatisfied with the communication skills of previous curriculum students generally reported passivity, inflexibility, or lack of flow in their interactions with patients. Compared to previous curriculum students, preceptors perceived VIM students to be more confident and well-spoken when communicating with patients and other health professionals; Participant 26 (hospital) mentioned, “and, the second year, their confidence level is definitely the same level as the fourth year [previous curriculum] students that I've got. They don't shy away from patient counseling.”

Occasionally, preceptors said VIM students still had difficulty responding to unexpected questions. For example, Participant 12 (hospital) stated, “they would have their list of questions and their little spiel that they would, um, say to the patient. But if the patient questioned something outside of their little, um, template, they wouldn't be able to, I suppose um, they weren't flexible, and adaptive.” However, preceptors also occasionally described VIM students as being able to communicate well with patients in unpredictable situations, including early in the pandemic; Participant 23 (community) said, “she was able to speak calmly, even though it was a stressful, stressful, stressful week.” None of the preceptors who directly compared both cohorts said that previous curriculum students had better communication skills than VIM students.

When asked directly, preceptors typically said that students in the previous curriculum demonstrated adequate professional behaviors and attitudes; Participant 14 (community) said, “I haven’t really had any problems with any of them.” However, preceptors also occasionally described students in both cohorts exhibiting undesirable behaviors and attitudes. For example, one preceptor said, “we've also had a [VIM] student who has left early and signed on the piece of paper that they were still here” (Participant 27, hospital). Typically, preceptors who were impressed by the professional behaviors and attitudes of VIM students reported that they demonstrated maturity when dealing with clinical situations; Participant 12 (hospital) mentioned, “and so yeah, nah, they were really great, really just really mature, very good with the communication, really diligent and organized.” Preceptors who compared the two cohorts typically reported that VIM students were more proactive in learning and performing professional activities, and one stated, “very professional. They’re very interested in their placements when they come out. Great learning attitude” (Participant 29, hospital). Occasionally, preceptors described VIM students as more reflective and willing to take responsibility for their learning; Participant 19 (community) mentioned, “they don’t act like kids… they act like they’re going to be the pharmacist.”

Preceptors who directly compared them to previous curriculum students typically felt that third-year VIM students were better able to apply their knowledge to patient care, as indicated by Participant 11 (hospital), who said, “there are pharmacists who commented that the [VIM students] were actually much better at clinical knowledge and better practical knowledge than the fourth-years [previous curriculum students].” Typically, preceptors who were dissatisfied with the knowledge of previous curriculum students said they had difficulty in retaining or applying their clinical and drug knowledge. Preceptors comparing the two cohorts rarely believed that the overall knowledge level of VIM students was lower than that of previous curriculum students. Students in both cohorts had specific gaps in knowledge described differently by preceptors in different practice settings. For VIM students, community preceptors occasionally reported a lack of familiarity with over-the-counter products, and rarely with wound care. Although hospital preceptors typically appraised VIM students as having exceptional clinical knowledge overall, occasionally specific knowledge areas (eg, aseptic technique) were below their expectations.

Preceptors were typically satisfied with or impressed by the ability of VIM students to perform clinical activities such as taking histories, counseling, and completing medication management plans; Participant 13 (hospital) said, “they knew what to do … if they had incomplete information, what to do next.” When community preceptors described the clinical abilities of students as below their expectations, they typically linked this to insufficient over-the-counter product knowledge in both cohorts and inadequate oral communication skills in previous curriculum students. In direct comparisons to previous curriculum students, preceptors generally described VIM students as more independent and better prepared, as indicated by Participant 17 (community), who said, “the newer versions are kind of more confident, they just go out themselves and obviously we supervise them at a distance, but yes, a lot less work with them. But they're at that level where maybe they don't require as much supervision.”

Three additional aspects of performance emerged in preceptor responses: therapeutic clinical reasoning, inquiry skills, and confidence. Interviewers did not explicitly ask participants about these, and only some participants discussed them. Therefore, we describe these emergent categories without reference to their frequency of appearance in the overall study population.^41-43

Preceptors who compared the therapeutic clinical reasoning skills of VIM and previous curriculum students felt that VIM students were either more competent or at the same level; one preceptor mentioned, “in general, comparing fourth-years [previous curriculum students] to two second-year [VIM] students, they are pretty similar level in that” (Participant 26, hospital). Some hospital preceptors were impressed that VIM students could solve clinical problems and complete medication management plans independently. Community preceptors also described individual VIM students as being able to identify and resolve clinical problems with prescriptions; participant 4 (community) mentioned, “he was able to pick up the important points and what other recommendations were required, so yeah, he was very good.” Others said that VIM students still needed help identifying problems or potential solutions.

Of the 10 preceptors who mentioned the inquiry skills of VIM students, nine were satisfied or impressed; one preceptor stated, “even simple things like they're able to really use their resources properly. They know where to go looking for things” (Participant 12, hospital). Some preceptors who mentioned inquiry skills felt that previous curriculum students lacked the ability to navigate resources efficiently or to identify evidence to support their recommendations; Participant 3 (community) said, “if it wasn’t in MIMS [common Australian reference similar to Lexicomp(R)], they didn’t know where to go.”

All four preceptors who directly compared the confidence of the two cohorts described VIM students as more confident than previous curriculum students; for example, Participant 11 (hospital) stated, “they seem to be ready and able and confident to, um, to want to go and engage with the patients rather than having to be pushed.” Some said this may be a result of VIM students having earlier exposure to placement sites or improved skills, as mentioned by Participant 29 (hospital) who said, “I think now we're starting to get a little bit more of that confidence so they'll have the appropriate reasoning and the communication skills to then talk to other allied health professionals.” Some community preceptors described being satisfied or impressed with the confidence of previous curriculum students who had worked in community pharmacies before their placements, but Participant 17 (community) noted that others “needed more help and more guidance, kind of you have to go out with them.” Meanwhile, some preceptors felt second-year VIM students lacked confidence during their first hospital placements.

In response to the direct question asking preceptors to compare the two cohorts, they occasionally said it was difficult to do so because, as Participant 1 (hospital) mentioned, “everyone has got their own strengths and weaknesses.” However, preceptors who did compare the cohorts typically described VIM students as outperforming previous curriculum students in terms of their professional behaviors, ability to perform clinical activities, oral communication skills, and knowledge. Three preceptors perceived previous curriculum students as having better knowledge than VIM students. Similarly, two preceptors described VIM students as being less able to perform clinical activities due to those students having insufficient knowledge to provide counseling. None of the preceptors perceived the previous curriculum cohort as outperforming the VIM cohort in terms of professional behaviors or oral communication skills. Preceptors also gave both prompted and unprompted responses to our second research question: In what ways could students in the VIM improve their clinical and professional skills?

Notwithstanding the extended placement program in the VIM, the most prevalent recommendation was to start placements earlier and increase their duration. One preceptor proposed adopting a medical school placement model with semester- or year-long placements. Preceptors emphasized the importance of experience and opportunities to practice skills in real settings; Participant 5 (hospital) mentioned, “the more experience, the better they're going to be. So, practicing, basically. So, having more placements and getting more opportunities to see and speak to patients and also having the initiative to be doing.” Another common recommendation was for students to apply for a pharmacy job to, as Participant 6 (community) said, “just to get exposure to the environment, on a repeated basis.” Community preceptors felt that this would help students improve their over-the-counter product knowledge.

One preceptor proposed that learning how to engage patients in problem-solving could help VIM students become more flexible and adaptable in their communication. For example, as Participant 11 (hospital) indicated, this approach involves “not just telling them, it's kind of asking a lot of questions … ‘You've already had some constipation, we’ve given you [a laxative], how’s that working? Great, here's some to go home with.’ So, it's a totally different type of counseling than what you do in the OSCEs [objective structured clinical examinations].” Some preceptors made the related suggestion that students could perform clinical activities more effectively if they better understood how the health care system works.

DISCUSSION

This study compared students within a transformed, competency-based curriculum (the VIM) and students from the previous curriculum at the same institution, in terms of their professional skills and behaviors as perceived by placement preceptors. According to preceptors, the VIM students, compared to previous curriculum students, exceeded their expectations in their development of all competencies of interest. Preceptor perceptions were especially promising considering that most of the VIM students supervised by participants in this study were not yet in the final year of the program.

Of all the domains, preceptors most frequently described VIM students as exceeding their expectations and outperforming previous curriculum students in oral communication skills, ability to perform clinical activities, and professional behaviors and attitudes. These improvements could be attributed to several aspects of curricular transformation: The VIM curriculum has integrated case-based active learning,⁴⁵ weekly workshops in each course, in-class role plays, more rigorous OSCEs, a skills coaching program, and earlier and more active experiential placements. Placement experience is likely to have been a significant driver of these and other observed improvements, as time in real practice settings enhances skill development⁴⁶ and allows students to focus on acquiring, applying, and receiving feedback on their development of core pharmacy competencies.

Although we did not set out to ask participants specifically about students’ therapeutic clinical reasoning skills, preceptors highlighted this as an area in which VIM students were more capable. This may reflect the way the VIM teaches students a systematic approach to patient care, as adapted from the Pharmacists’ Patient Care Process created by the Joint Commission of Pharmacy Practitioners,⁴⁷ and scaffolds their skills in identifying and solving medication-related problems using patient cases during interactive lectures, workshops, and preclass online activities.⁶

Preceptors had varied responses about students’ knowledge levels. When deciding what to teach and what not to teach in the VIM to allow for the increased focus on skill development, educators focused on national and international health priorities, with the expectation that students encountering unfamiliar conditions and medicines in practice would be able to apply their improved inquiry and therapeutic reasoning skills to make safe and appropriate clinical decisions. Although many preceptors reported that VIM students had higher levels of knowledge when compared directly with previous curriculum students, many also described their knowledge as below their expectations in specific areas. These specific preceptor comments can be used to improve our curriculum, supporting materials, and preceptor education. Many preceptors described students with both overall impressive knowledge and some specific knowledge gaps. Meanwhile, some preceptors had students showing what they perceived to be exceptional knowledge, while others had students showing what they perceived to be lower-than-expected knowledge of the same type. This may be due to variability among the student cohort, differing expectations held by preceptors, or the fact that many VIM students were in lower year levels and, therefore, had not yet covered as many therapeutic areas at university.

Watmough and colleagues²⁸ found similarly mixed views among preceptors about the knowledge levels of students who graduated from a transformed medical curriculum and from a previous curriculum. To explain this, the authors concluded that the preceptors disagreed about what level of knowledge was necessary for practice. The same may be true for participants in the current study. Because past curricula have focused on knowledge acquisition, some preceptors may overemphasize knowledge; further research could investigate this. At the same time, it is possible that enhanced professional skills and abilities to apply knowledge have been gained at the expense of knowledge acquisition. Health professions educators and the profession more broadly need to continue to evaluate whether the right balance is being achieved.

Despite our hypothesis that VIM students would have improved professional behaviors and attitudes, initial preceptor responses to questions about professionalism appear to suggest that they did not observe this effect, as the professional behaviors of both cohorts of students were described as similar overall. This may be explained by preceptor interpretations of the terminology. When asked about professional behavior and attitude, most preceptors spoke about attributes that would also be expected in professions unrelated to health, such as punctuality, appropriate attire, and respectfulness. When preceptors specifically mentioned characteristics that are more likely to indicate that a student has assumed the mindset of a health professional, eg, maturity in dealing with clinical situations, proactiveness, motivation to learn, and response to feedback, VIM students more often exceeded their expectations. These mixed results may reflect a lack of consensus among pharmacists on the definition of professional behavior^48,49 and a range of preceptor expectations about student behaviors and attitudes, which may be based on their past experiences. Further, because it explored preceptor perceptions, this study had limited ability to measure the professional attitudes of students, which are largely internal. Future research with students could investigate the impact of curriculum transformation on their professional attitudes and professional identity. For example, Noble and colleagues⁵⁰ and Quinn and colleagues⁵¹ found that earlier practice-based experiences during undergraduate studies enhanced the development of professional identity in pharmacy students.

Our findings build on the emerging evidence that curriculum transformation can improve the performance of health professions students^20-28,52 by adding evidence from observations made by placement preceptors, whose judgments may be considered objective, because preceptors are not involved in other parts of the curriculum, and accurate, because preceptors understand the competencies required for their practice settings. Although this is the first study to investigate the impact of curriculum transformation on the performance of pharmacy students from placement preceptors’ perspectives, similar studies have been published in medicine. In Watmough and colleagues’ study,²⁸ preceptors agreed that recent graduates from a transformed, more competency-focused curriculum were better prepared to practice as junior doctors given their improved attitudes, clinical skills, and capabilities as communicators and team workers. Owino⁵² compared recent medical graduates from two universities, one with a problem-based learning approach and the other with a traditional, more didactic curriculum, and found that preceptors perceived graduates from the latter to be more practice ready across four broad competency areas. However, preceptor perceptions of graduates can be influenced by what graduates had learned during their internship or another form of practice after completing their undergraduate studies. By evaluating preceptor perceptions of undergraduate students, the current study can better gauge the contribution of a transformed curriculum to practice readiness.

One limitation of this study is the possibility of selection bias. Although our use of purposive sampling ensured that participants represented a variety of practice settings and roles,³⁰ preceptors with strong opinions about student performance or curricular transformation may have been more likely to respond to the invitation to participate. Social desirability bias may also have influenced participant’s responses during telephone interviews.⁵³ To limit researcher bias so that our interpretations of the data would not be influenced by any desire to favor VIM students, researchers who had worked on redesigning the curriculum were not involved in interviewing or coding, and all data categorizations were the result of at least two researchers coding independently.

Other significant limitations are that participants were asked to compare students of different year levels and that most participants had only supervised a limited number of VIM students who were mostly from lower year levels but had supervised relatively larger numbers (including fourth-year students) from the previous curriculum. Observations about a single student or a small group of students may not fairly or accurately represent the abilities of a whole cohort. Despite this, most preceptors commented that VIM students from lower year levels performed better than previous curriculum students overall. However, the objectivity of preceptors assessing student performance may be unintentionally limited by their recent experiences with other students⁵⁴ and may not necessarily align with program objectives, graduation requirements, or professional competency standards.

While this study provides an in-depth understanding of how preceptors perceive the placement performance of undergraduate students from a transformed curriculum, the performance of these students as graduates is yet to be observed. Hence, future research could investigate preceptor perceptions about the abilities of interns from transformed and previous pharmacy curricula and explore whether those perceptions align with the results of this study. Another future research objective could be to explore whether and how noncurricular factors (eg, work experience) affect student performance during placements.

CONCLUSION

To fulfill its societal mandate, health professions education should foster the knowledge, skills, and professional behaviors needed to optimize health outcomes. During major curricular transformation, it is important to capture and aggregate results from a variety of sources to generate successive approximations toward this goal. This study connects curricular change to undergraduate student performance during their “last mile,” ie, on their experiential placements.

The results of this study demonstrate that curricular transformation with a focus on skill development and active learning can improve pharmacy students’ professional skills and behaviors in real practice settings, as perceived by preceptors. It also shows that taking the extra step to verify preliminary curricular outcomes⁶ by studying preceptor experiences can contribute substantially to understanding overall curricular effectiveness and determining whether the desired balance between skills and knowledge is being achieved. If institutions make similar curricular changes and monitor these for further improvements via comprehensive curriculum evaluation methods, future pharmacists may be better equipped to improve health care.