Pharmacy Resident Perspectives on the Layered Learning Practice Model

Abstract

Objective: To describe pharmacy resident perspectives on the layered learning practice model (LLPM) at large academic medical centers in the United States and identify the types of training residents receive to prepare for the LLPM. Methods: This was a cross-sectional, mixed methods study that surveyed pharmacy residents completing training at large, academic medical centers on their perspectives of the LLPM. Residents with at least 4 weeks of precepting experience were eligible to complete an online survey. Descriptive statistics were generated for demographic and Likert data while themes were identified from narrative free responses. Results: Twenty-seven resident responses from 10 institutions were included. Likert data showed generally positive perspectives toward overall experience with the LLPM and perceived LLPM impact on clinical knowledge and professional practice, with 100.0% of respondents agreeing they enjoyed precepting under the LLPM. However, 44.4% also agreed that precepting was stressful. LLPM impact on resident behavior received positive to neutral responses; 63.0% neither agreed nor disagreed that the LLPM changed their approach to patient care. Written comments revealed themes such as sources of stress within the LLPM, increased depth of learning, improved comprehensive patient care, and improved professional growth. The most common forms of LLPM training occurred through preceptor and student evaluations (73.1% and 61.5%, respectively) and orientation (50.0%). Conclusion: These findings identified a variety of resident perspectives on the LLPM ranging from positive to negative and revealed possible areas of improvement related to LLPM implementation. Residency programs may consider exploring stressors on the resident role in the LLPM to better structure resident precepting training and maximize their clinical learning experience.

Introduction

The concept of a hierarchical training model in which senior learners teach junior learners, often referred to as near-peer teaching, is well-documented in medical education and has become more common in experiential pharmacy training over the past decade.^{1 -3} The layered learning practice model (LLPM), a pharmacy variation of near-peer teaching, was introduced by the University of North Carolina Medical Center and Eshelman School of Pharmacy in 2011. ⁴ It was developed in response to recommendations of the 2010 American Society of Health-System Pharmacists Pharmacy Practice Model Initiative (now known as the Practice Advancement Initiative) aimed at advancing the pharmacy profession.^5,6 In particular, the Pharmacy Practice Model Initiative called for the development of “optimal pharmacy practice models” that most effectively utilize pharmacists and pharmacy resources to deliver comprehensive patient care.^6,7 As such, the LLPM consists of 3 layers of learners: a “seasoned” attending pharmacist, a postgraduate year 1 (PGY1) and/or 2 (PGY2) pharmacy resident, and a pharmacy student.^6,8 The LLPM incorporates pharmacy learners into the workflow as “pharmacist extenders,” allowing them to perform patient care activities while also creating opportunities for mentorship between learners.^{7 -9}

The LLPM has demonstrated benefit on multiple fronts: reduced medication costs, improved patient satisfaction, and improved clinical outcomes.^7,10,11 The experiences of pharmacy personnel involved have also been described. Perspectives of attending pharmacists have been documented in the literature and include discussions of beneficial, challenging, and detracting aspects of the LLPM.^1,3,6,12 Perceptions of introductory and advanced pharmacy practice experience (APPE) students also exist and suggest positive perceptions of the LLPM.^1,3,4,9,13 Pharmacy residents are the least represented, but existing studies conducted at individual institutions have generally shown positive perceptions of the LLPM.^1,3,4 Resident perspectives are also limited by small sample sizes, with 1 published study including only a single resident perspective and the largest study to date including only 8.^1,3,4,9

While the positive documented experiences of pharmacy residents are encouraging, generalizability is difficult given the small number and scope of residents sampled. Thus, the purpose of this study was to describe pharmacy resident perceptions of how practice within the LLPM contributes to or detracts from a learning experience at large academic medical centers across the United States. Further exploration of resident learners’ perceptions, those highly ingrained in the LLPM, may shed important light on the effectiveness, implementation, and training required for successful integration of LLPM precepting.

Methods

This study received exempt status approval from the institutional review board at the University of California, Davis.

Using Qualtrics^® software, an online survey instrument was developed consisting of demographic questions and Likert items about the pharmacy resident’s LLPM experience. Free text fields were also included to allow residents to elaborate on their experiences. The Likert items were adapted from interview questions developed by McIntyre et al ⁹ in their study on pharmacy student experiences in non-traditional student-preceptor models. Their questions were organized and based on a modified version of Kirkpatrick’s 4-level training evaluation framework, which has been used to evaluate medical education.^14,15 The present study adapted this framework into 4 domains: Domain 1: Overall Experience with the LLPM, Domain 2: Perceived LLPM Impact on Clinical Knowledge, Domain 3a: Perceived LLPM Impact on Behavior (Approach to Patient Care), Domain 3b: Perceived LLPM Impact on Behavior (Workplace), and Domain 4: Perceived LLPM Impact on Professional Practice as a Pharmacist.

In October 2020, participation was solicited through a group message board for large academic medical centers in Vizient^®, a national health system consortium, under the assumption that the residency and LLPM experiences would be comparable. Responding institutions were included if they met the following criteria: offered both PGY1 and PGY2 pharmacy residency programs, utilized the LLPM for pharmacy residents and students, had at least 400 beds, and supported at least 100 APPE student-months per year. These criteria were selected to increase the likelihood that participating institutions had the resources, training, and resident precepting opportunities available that may result in meaningful experiences that residents could grow, develop, and comment on. Residency program directors (RPDs) from these institutions were asked in November 2020 to disseminate the online survey instrument to their residents for completion. Email reminders were sent at 2-to-3-week intervals to RPDs for their residents to complete the survey until its closure in January 2021. Participation was voluntary and anonymous. No identifying information about the resident was collected.

For a resident response to be included in the data analysis, the resident must have provided written informed consent and served as an APPE student preceptor for at least 4 rotation weeks during their current residency program. Responses were excluded if the subject did not precept an APPE student for at least 4 rotation weeks in their current program, if the subject failed to respond to the question asking about this, or if the subject submitted an incomplete survey. Descriptive statistics were used to assess demographic information and Likert answers. Free response content was analyzed qualitatively. One researcher identified individual units of meaning (UOM) from each free response; some responses contained several discrete thoughts and therefore could be split into multiple UOM. These were then summarized into condensed UOM and developed into themes. The results of this analysis were independently validated by the other 2 researchers. Selected quotes were chosen to represent themes found in each domain.

Results

A total of 11 institutions distributed across the western, midwestern, northeastern, and southern United States responded to the study invitation, with 10 meeting the inclusion criteria for participation. From these institutions, 54 residents started the survey and 35 (64.8%) completed it, 27 of whom (50.0%) met the inclusion criteria for analysis. The 27 excluded responses consisted of 19 incomplete surveys (none of which progressed further than the first third of the survey) and 8 residents who lacked the minimum 4 weeks of precepting experience. The majority of survey respondents were first year residents (70.4%), completing an acute care residency program (70.4%), and had either some or a strong interest in teaching (81.4%) prior to starting their current program. Additional demographic information of survey respondents can be found in Table 1. Regarding strategies for LLPM preparation, evaluation/feedback from preceptors (73.1%) and students (61.5%) were the most common forms of LLPM training reported, followed by training during orientation (50.0%) (Figure 1).

Table 1.

Demographics of Pharmacy Residents Included in a Study to Identify Pharmacy Resident Perspectives on the Layered Learning Practice Model (n = 27).

Characteristic	No. (%)
Year of residency and practice setting
First	19 (70.4)
Acute care	13 (48.1)
Ambulatory care	5 (18.5)
Community	0 (0.0)
Other a	1 (3.7)
Second	8 (29.6)
Acute care	6 (22.2)
Ambulatory care	1 (3.7)
Community	0 (0.0)
Other a	1 (3.7)
Marital status
Single	17 (63.0)
Married	3 (11.1)
Partnered	7 (25.9)
Average hours worked per week
40-60	7 (25.9)
60-80	19 (70.4)
>80	1 (3.7)
Prior LLPM experience b
Yes	21 (77.8)
No	6 (22.2)
Weeks precepting in current program (n = 26) c
4-5	8 (30.8)
6-7	5 (19.2)
8 or more	13 (50.0)
Students precepted
1	3 (11.1)
2	6 (22.2)
3	6 (22.2)
4 or more	12 (44.4)
Hours per week precepting
2	1 (3.7)
4	10 (37.0)
6	6 (22.2)
8 or more	10 (37.0)
Interest in teaching prior to current program
No interest	5 (18.5)
Some interest	11 (40.7)
Strong interest	11 (40.7)

Note. LLPM = layered learning practice model.

^aResidents indicating “Other” specified “general” (PGY1) and “Pediatrics” (PGY2) in the provided text field.

^bPrior LLPM experience may include APPE student experience in an LLPM.

^cOne respondent answered yes to precepting at least 4 weeks but did not specify how many.

Figure 1.

Results of pharmacy resident survey responses on strategies for LLPM training received (n = 26). Question was formatted as “Select all that apply.” Residents indicating “Other” specified teaching certificate programs in the provided text field.

Note. LLPM = layered learning practice model.

Likert data for Domain 1: Overall Experience with the LLPM revealed generally positive views toward the LLPM (Figure 2), with 100.0% of respondents agreeing that they enjoyed precepting students under the LLPM and 85.2% agreeing they felt confident addressing student questions about clinical knowledge and application. One notable finding, however, was that while 66.6% of respondents disagreed that the LLPM detracted from their learning experience, 44.4% still felt that precepting students was stressful. Qualitative analysis of 21 UOM revealed 4 themes: sources of stress within the LLPM (47.6%) (Table 2), favorable LLPM experiences (28.6%), varying levels of resident involvement (14.3%), and promotion of teaching (9.5%). Specific sources of stress included student expectations of the resident’s baseline knowledge, difficult student personalities, quantity and depth of topic discussions, resident staffing and workload, lack of LLPM training, and learner rotation schedule alignment.

Figure 2.

Results of pharmacy resident survey responses on level of agreement with statements related to Domain 1: Overall Experience with the LLPM (n = 27). Responses that do not total 100.0% reflect items for which not all subjects selected an answer.

Note. LLPM = layered learning practice model; APPE = advanced pharmacy practice experience.

Table 2.

Summary of Most Common Themes Elicited From Pharmacy Resident Free Text Comments in Each Kirkpatrick-Based Domain with Selected Sample Responses (n = 27).

Domain (No. UOM)	Most common theme (% UOM)	Sample responses
Domain 1: Overall Experience with the LLPM (21)	Sources of stress within the LLPM (47.6)	“Having to precept a student with a difficult personality while trying to get the most out of a residency rotation was something I found stressful.” “I feel like I was not that prepared enough to precept students under this learning model; I wish I received more training and guidance.” “When starting a new rotation and expected day 1 that you are the point person is extremely stressful because you are learning from that aspect and do not know what to do with that disease state.”
Domain 2: Perceived LLPM Impact on Clinical Knowledge (11)	Increased depth of learning (54.5)	“Precepting students helped me question the ‘why?’ By asking the student questions on their recommendations it helped me solidify why a patient is [on] a certain medication. . .” “Precepting through the LLPM helped me share pharmacy concepts with other students and also challenged me to be prepared to think ahead and answer [any] questions that might have been asked of me.”
Domain 3a: Perceived LLPM Impact on Behavior (Approach to Patient Care) (10)	Improved comprehensive patient care (70.0)	“LLPM was beneficial to patient care because the student and I spent more time discussing pharmacotherapy options - heavily referencing guidelines and primary literature to support and defend [our] decisions with the medical team.” “I think that having two people look into primary literature may provide more comprehensive and efficient care. For example, one particularly complex patient with HIT required very high amounts of argatroban to achieve his goal aPTT. Having myself and a student look into primary literature certainly helped with providing the team with a more accurate recommendation more quickly.”
Domain 3b: Perceived LLPM Impact on Behavior (Workplace) (8)	Improved communication (75.0)	“Precepting has helped me communicate a lot of handoffs to my preceptors. . .I have been able to relay some of the things we talked about during rounds as a small group.” “After seeing several styles of case presentation, I have started to. . .take pieces of each style and blend them together into my own style.”
Domain 4: Perceived LLPM Impact on Professional Practice as a Pharmacist (8)	Improved professional growth (50.0)	“. . .I have. . .learned to be very careful and acknowledge that I have gaps in knowledge that I could ask for help with.” “. . .[T]hese skills are something I hope to build upon during the remainder of my residency years.”

Note. UOM = units of meaning; LLPM = layered learning practice model; HIT = heparin-induced thrombocytopenia; aPTT = activated partial thromboplastin time.

Domain 2: Perceived LLPM Impact on Clinical Knowledge also received favorable responses (Figure 3). The majority of residents (96.3%) agreed that precepting students was beneficial to building their clinical knowledge, and 77.8% agreed that the LLPM made them more confident in their clinical knowledge. Analysis of 11 UOM revealed 2 themes: increased depth of learning (54.5%) (Table 2) and varying learning opportunities (45.6%). Residents explained that precepting motivated them to research topics more deeply and challenged them to think ahead to better explain concepts to students.

Figure 3.

Results of pharmacy resident survey responses on level of agreement with statements related to Domain 2: Perceived LLPM Impact on Clinical Knowledge (n = 27).

Note. LLPM = layered learning practice model.

Domain 3a: Perceived LLPM Impact on Behavior (Approach to Patient Care) received positive and neutral responses (Figure 4). Although 63.0% of residents agreed that the LLPM was beneficial to patient care, 63.0% also felt neutral on whether the LLPM changed their approach to patient care. The majority were also neutral toward whether the LLPM improved their communication with patients (51.9%) or their empathy and patience when interacting with patients (63.0%). Qualitative analysis of 10 UOM revealed 2 themes: improved comprehensive patient care (70.0%) (Table 2) and modeling patient care (30.0%). For example, residents described working with students to discuss treatment options, searching primary literature, and referencing guidelines to defend recommendations to the medical team.

Figure 4.

Results of pharmacy resident survey responses on level of agreement with statements related to Domain 3a: Perceived LLPM Impact on Behavior (Approach to Patient Care) (n = 27).

Note. LLPM = layered learning practice model.

Domain 3b: Perceived LLPM Impact on Behavior (Workplace) similarly received positive and neutral responses (Figure 5). For instance, residents agreed that precepting students improved communication with their own preceptors (77.8%) and that it helped them become a more confident member of the healthcare team (62.9%), but 48.1% were also neutral on whether the LLPM changed the way they presented patient cases to the healthcare team. From the 8 UOM in this domain, 2 themes were revealed about the LLPM: improved communication (75.0%) (Table 2) and fostering a positive learning environment (25.0%). Residents described how observing student case presentations helped them to reflect on their own skills and adapt different presentation styles into their own. They also described precepting students as helping to communicate handoffs to their own preceptors and creating a “collaborative and fun environment.”

Figure 5.

Results of pharmacy resident survey responses on level of agreement with statements related to Domain 3b: Perceived LLPM Impact on Behavior (Workplace) (n = 27). Responses that do not total 100.0% reflect items for which not all subjects selected an answer.

Note. LLPM = layered learning practice model; APPE = advanced pharmacy practice experience.

In Domain 4: Perceived LLPM Impact on Professional Practice as a Pharmacist, Likert data again skewed positively (Figure 6), with 85.1% of residents agreeing that the LLPM impacted their practice as a pharmacist in a positive way. 77.8% of residents also agreed that the LLPM prepared them to become a more confident future higher-level preceptor, and 66.6% agreed they gained a stronger sense of identity as a pharmacist because of the LLPM. Qualitative analysis of 8 UOM uncovered 3 themes: improved professional growth (50.0%) (Table 2), increased confidence (37.5%), and strengthened professional identity (12.5%). Residents expressed pride in precepting students and described the teaching experience as showing them their potential to be a preceptor/educator. One resident stated that the LLPM “[solidified] my feeling that I am in the correct profession.”

Figure 6.

Results of pharmacy resident survey responses on level of agreement with statements related to Domain 4: Perceived LLPM Impact on Professional Practice as a Pharmacist (n = 27).

Note. LLPM = layered learning practice model.

Discussion

This is the first nationwide study that seeks to provide a comprehensive analysis of the positive, negative, and neutral experiences of pharmacy residents within the LLPM. Use of the Kirkpatrick training evaluation framework, modeled after McIntyre et al, ⁹ also provided an evidence-based approach toward the data collection and analysis.

All residents agreed that they enjoyed precepting students, a perspective consistent with previous research.^3,4 Positive resident perceptions were found across all domains assessed in this study, with Domain 1: Overall Experience with the LLPM, Domain 2: Perceived LLPM Impact on Clinical Knowledge, and Domain 4: Perceived LLPM Impact on Professional Practice as a Pharmacist receiving the most positive responses; these echo the overall favorable attitudes and perceived knowledge attainment described by Bates et al ⁴ and the perceived development of valuable teaching skills described by Tsang et al. ³ Bates et al ⁴ also reported that residents perceived themselves to provide responsible contributions to patient care, which aligns with the theme of improved comprehensive patient care found in this study. These results suggest that residents do indeed find value in several aspects of the LLPM, particularly those related to precepting students, increasing their own depth of learning, providing quality patient care, and professional development.

The present study is notable for its non-positive findings as well. While results in Domain 3a: Perceived LLPM Impact on Behavior (Approach to Patient Care) showed some positive perceptions on how patient care was improved in the LLPM, changes in the residents’ approach to patient care (eg, communication, empathy, patience) received mostly neutral responses, suggesting that the development of precepting skills may not overlap with patient care skills, or that a resident’s approach to patient care may have been developed elsewhere (eg, during their APPE rotations). In Domain 3b: Perceived LLPM Impact on Behavior (Workplace), residents reported having increased confidence as a healthcare team member as a result of the LLPM, but they also had a neutral response toward the LLPM impacting how they presented patient cases to the interprofessional team. This again suggests that certain skillsets, such as patient presentations, may have already been developed through other experiences. If in fact residents did hone their approach to patient care and patient presentation skills prior to their precepting experience, the LLPM could be utilized to have residents specifically mentor as well as seek feedback from pharmacy students in these areas, helping both levels of learners strengthen these skillsets during their clinical rotations.

Perhaps the most prominent findings, however, were the negative resident perceptions of the LLPM, which directly contradicted the findings in Tsang et al’s ³ study on hierarchical teaching in acute care pharmacy teams at a tertiary, university-affiliated teaching hospital. There, 100% of mid-level learners (who consisted of 2 residents and 9 senior pharmacy students) disagreed that mentoring a student significantly impaired their own learning, and 54% disagreed that workload had significantly increased. In the present study, despite all residents agreeing that they enjoyed precepting and most feeling that it was beneficial to their learning, many still agreed that precepting students was stressful, with some saying it detracted from their learning. While balancing teaching with clinical duties is a part of the growth and development process of a pharmacy preceptor, residency programs may benefit from assisting residents in coping with these stressors. Needs may vary between institutions, but a needs assessment could be collected from current residents and preceptors to highlight specific sources of stress and training areas of interest. For instance, programs may find it useful to provide faculty mentorship to residents, training on precepting students with difficult personalities, setting expectations with student learners, and building educational material such as topic discussions. Furthermore, programs may consider developing training materials for residents at the beginning of their program, as only 50.0% of respondents stated they received training for the LLPM during orientation. As institutions implement such changes to their LLPM, it will also be important to evaluate the impact of these changes to determine if they did indeed improve the resident precepting experience.

The presence of negative resident perspectives in this study may be explained by the study’s design. Previous research on resident perspectives were conducted with small sample sizes at single institutions via formal evaluations/surveys^3,4 or direct observation by a pharmacist. ¹ Taken together, these factors could potentially bias or influence reported perspectives of the LLPM. The present study was anonymous and spanned multiple institutions across the United States, which may have provided reassurance for respondents to speak more candidly about their experiences. This study is also the second to assess pharmacy resident perspectives on the LLPM through both quantitative and qualitative means ³ ; other studies have used only qualitative methods to assess resident perceptions.^1,4 The Likert items were useful for quantifying resident attitudes toward aspects of the LLPM that they may or may not have otherwise considered describing, and free text fields were essential to providing insight into these experiences.

This study was not without limitations. The authors contacted institutions through a group message board in the Vizient® national health system consortium anticipating that such an approach would help streamline the institution screening process and more efficiently reach the intended study population. RPDs were also asked if the LLPM was practiced at their institution, but they were not asked to describe the exact structure of their LLPM. This process may have masked differences in LLPM implementation, introduced selection bias, and potentially depressed the response rate from RPDs. The authors were unable to obtain a total number of qualifying institutions to calculate a response rate due to the nature of the message board used to solicit participation: Approximately 800 individuals were in the large academic medical center Vizient^® group, but this did not represent the actual number of institutions, as multiple individuals affiliated with the same institution could participate in the group. The authors thus acknowledge that the sample size was very small considering a nationwide invitation for participation screening. Despite this, it is still the largest and widest resident sample size to date, thus lending itself toward more generalizable results relative to existing studies. It should still be noted that a majority of the respondents were PGY1s (70.4%) and the most common pharmacy practice setting among respondents was acute care (70.4%). Having a more balanced sample of PGY1s and PGY2s as well as practice setting would have provided greater generalizability to the resident population.

In addition, while evidence-based, the survey instrument was not validated or pilot tested prior to dissemination and may have limited the scope of written comments only to topics mentioned in the Likert questions. The inclusion criteria also did not define minimum precepting duties for a pharmacy resident. Instead, this data was collected voluntarily through written comments, and based on these responses, it appears residents were involved in topic discussion facilitation, guiding students in patient workups, researching the literature with students to make clinical recommendations, and observing the primary preceptor teaching the student. Considering that 70.4% of residents reported working 60 to 80 hours per week and 1 resident reported working >80 hours per week, more specific data on job duties performed would have been helpful in understanding their potential impact on workload and stress levels. The survey was also disseminated from November 2020 to January 2021 so that residents would have sufficient time to gain precepting experience from the start of their program. However, this time period is typically a very busy one for residents (eg, residency program recruitment, national conference attendance) and may have depressed the response rate. Participating institutions were not surveyed regarding at what point in their program residents begin their precepting responsibilities; at sites where precepting begins later in the year to afford residents more time to train, the survey timeline may have excluded these residents from participating and potentially bias results to be more negative. However, by recognizing areas of improvement in LLPM training and implementation, this study may open opportunities to engage residents in precepting students sooner in their program rather than later. Lastly, the COVID-19 pandemic, with a winter surge during the survey timeframe, may have further depressed the response rate as well as added additional stress to the resident precepting experience and affected their traditional opportunities to interact with students on rotation.

It is also worth noting that the negative resident perspectives found in this study could potentially be attributed to differences in LLPM implementation, and the survey instrument used in this study did not specifically explore this area or the structure of individual teaching certificate programs in detail. Studies that have described best practices for LLPM implementation exist,^4,6 but it is unclear if resident perspectives were factored into these recommendations. This study did attempt to control for differences in residency precepting experiences by defining inclusion criteria for the academic medical center setting such as minimum numbers of hospital beds, pharmacy residents, and APPE student-months per year.

Conclusion

This exploratory study examined pharmacy resident perspectives of layered learning through a mixed methods approach. Results showed generally positive attitudes toward the LLPM but also identified neutral and negative perspectives not previously documented, as well as the types of training residents received for the LLPM. Future studies may further explore stressors on the resident role in the LLPM and should seek strategies and resources to support pharmacy resident precepting as well as maximize their clinical learning experience. Further investigation is also needed on which training methods are best in preparing for the LLPM, as this will aid in creating a more formal framework or best practices for LLPM implementation.