A Multipreceptor Approach to Ambulatory Care Topic Discussions

Abstract

Objective. To assess the impact of a multipreceptor approach to facilitating topic discussions on students' knowledge and confidence in clinical decision-making during an ambulatory care advanced pharmacy practice experiences (APPEs).

Design. Faculty members with relevant expertise and experience facilitated discussions with fourth-year doctor of pharmacy (PharmD) students regarding 7 ambulatory care topics. A student self-assessment survey and knowledge-assessment instrument was administered before and after discussions.

Assessment. Students' examination scores increased significantly from 59.1% ± 13.9% at baseline to 76.5% ± 12.6% at the end of the 5-week experience (p<0.001). The majority of participants were comfortable making therapeutic decisions regarding medication use as it related to all discussion topics except heart failure.

Conclusions. Participation in topic discussions led by faculty members with expertise and experience for each ambulatory care topic was associated with a significant improvement in knowledge-assessment scores.

INTRODUCTION

The 2013 educational outcomes and domains, as stated in the Center for the Advancement of Pharmacy Education, were designed to describe the capabilities of a graduate from a PharmD program.¹ The first 2 domains on developing foundational knowledge and providing the essentials for practice and care, state that a student should develop and apply knowledge within clinical sciences to solve therapeutic problems and provide patient-centered care as the medication expert. The Accreditation Council for Pharmacy Education (ACPE) is specific regarding curriculum during the fourth year of the PharmD program. The ACPE recommends that high-quality, direct patient-care experiences should be delivered during this final year of the educational program.² In the Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the PharmD Degree, Standard 14 states that direct interaction with diverse patient populations in a variety of practice settings must be included.³

In line with Standard 15, each college or school of pharmacy also must develop and assess activities to evaluate student learning and curricular effectiveness.³ Activities should be consistently and reliably assessed within and among faculty members, practice sites, and preceptors. General guidelines in Standard 15 state that a plan should be followed regarding documentation of the learning experiences, including instructional methods and learning materials used. This documentation should include faculty member and preceptor assessments of student development of competencies with knowledge-based and performance-based assessments.

With respect to assessment in ambulatory care settings, 1 study determined students’ ability to effectively manage ambulatory care disease states with knowledge-based assessment administered prior to and after the practice experience, using topic discussions as the intervention. Student knowledge regarding these core topics improved. However, the assessment instrument used identical questions, which could have resulted in student memorization (recall bias) and attaining the knowledge through a means other than the topic discussions. The short-answer questions also could have been subject to grading bias, as preceptors graded their own students’ tests.⁴

Other literature has assessed the effectiveness of integrating disease-state topics into APPEs. Studies implemented in performing medication therapy management services in a geriatric population, for example, fostered a deeper knowledge base in cultural competency and public health.^5,6 In the first study, students participating in APPEs that included diverse cultural experiences completed a structured reflective exercise at the end of the experiences.⁵ In the latter study, the instrument used for evaluating the discussions was a preexperience and postexperience discussion consisting of a written definition regarding public health.⁶ A final study showed enrichment in preceptor training of postgraduate pharmacy residents when they served as facilitators of disease-state discussions⁷; the purpose of this study was to train residents as academicians rather than centering on learning enhancement of APPE students.

Case-based, small-group discussions also have been implemented in the clerkships of medical schools.^8,9 In 1 study, medical students were guided during the case discussions to answer an assigned clinical question integrating evidence-based medicine into their developing decision-making processes.⁸ Another descriptive study focused on developing an online clerkship in family medicine.⁹ Participants were led by faculty members through threaded discussion groups and completed classroom modules on the treatment of diabetes. Neither study evaluated outcomes through a formal knowledge or skills-based questionnaire or examination.^8,9

Using small group-based discussions can increase the acquisition of knowledge and skills. A self-care course was redesigned from a traditional lecture-based format to focus on case-based delivery of knowledge in a small-group format.¹⁰ Students’ grades and satisfaction ratings were compared over a 2-year period for both class formats. Grades and satisfaction ratings both increased in the small-group format.

During their final year of the PharmD program at the University of Mississippi School of Pharmacy, students complete at least 1 ambulatory care APPE. Coupled with direct patient-care experiences that occur at APPE practice-experience sites, adding high-quality and focused small-group discussions for a variety of disease-state topics was hypothesized to increase the students’ overall experiences. Ambulatory care preceptors with extensive clinical experience in a particular disease state pool their APPE students and offer disease-state topic discussions. Although preceptors have been using a multipreceptor approach to discussions for several years, previously no formal assessment of the process had been done. The purpose of this study was to assess the impact of a multipreceptor approach to knowledge attainment and confidence in clinical decision-making during ambulatory care practice experiences. The results were evaluated, and the potential for implementing an expanded multipreceptor approach to topic discussions throughout the APPE component of the curriculum is described.

DESIGN

During the fourth year of the PharmD program, students at The University of Mississippi School of Pharmacy completed 8 APPEs, with 1 being a required ambulatory-care experience. Although this APPE was labeled as ambulatory care, the majority of faculty members offering this experience had a practice site specializing in 1 or 2 specific disease states. Despite the potential lack of patient contact or exposure in all areas of ambulatory care, the preceptors typically discussed a wide variety of disease states related to ambulatory care with their practice-experience students. The discussion facilitators developed a series of 7 disease-state discussions that were led by an ambulatory care APPE preceptor with expertise and clinical experience in that specific disease state. Topics were either a primary area of current or past clinical practice for preceptors or an area of focus in the preceptor’s other teaching responsibilities at the institution. Occasionally, postgraduate pharmacy residents or practice-experience students facilitated the topic discussions under the direct guidance of their preceptors who provided additional insights, if needed. Any student on an ambulatory care, pedagogy, or research APPE with these preceptors participated in the 7 discussions covering the following topics related to ambulatory care: diabetes, hypertension, dyslipidemia, heart failure, anticoagulation, devices/physical assessment, and asthma. Prior to the discussions, students were provided with a recommended reading list consisting of disease-state guidelines and other literature and were encouraged to use any reference that they felt would help them prepare for each discussion.

All preceptors approached the discussion format in slightly different ways, but most discussions are 60-90 minutes in length. The 7 discussions were scheduled over a 5-week practice experience, with no discussions scheduled on the first 2 days or on the final day of the experience. Generally, 2 to 3 discussions were scheduled weekly, but they were intentionally not scheduled on back-to-back days so students would have adequate preparation time. However, because of schedule limitations, there were rare instances when 2 discussions had to be scheduled on the same date. Given the large number of preceptors and their differing clinic schedules, the exact day of the week and time for each discussion may have been different from 1 practice experience to the next. Each preceptor also had a different style in leading his/her discussion. Although some may have provided a more traditional lecture format with audience participation, most had a student-driven discussion of the topic in which students were asked questions and preceptors filled in missing or incorrect information as needed. Additionally, every preceptor used patient cases or clinical vignettes to highlight information.

The study was approved by The University of Mississippi Institutional Review Board. During the 2012-2013 academic year, fourth-year PharmD students (P4s) on practice experiences with discussion facilitators as their preceptors were administered a self-assessment survey and knowledge assessment instrument before the first disease-state discussion (prediscussion test). A second survey and knowledge-assessment instrument was administered after the final discussion (postdiscussion test). All ambulatory care APPE students on a practice experience with any of the discussion facilitators were eligible for inclusion, but they could participate in the assessment component only once during the academic year. Participation was voluntary. Students were incentivized with a 1% bonus on a required practice experience activity, such as journal club, case presentations, or final examination, if they had completed all study components or an alternate activity. A cover letter was attached to ensure students’ understanding of the study requirements and expectations. After consenting, students were randomly assigned a number by nonstudy personnel to allow for blinded correlation of prediscussion and postdiscussion assessment results. The prediscussion survey instrument asked students which practice experience they were currently completing (ie, first APPE, second APPE), if they had completed a required inpatient internal medicine APPE, and the extent of their past ambulatory care experiences in other practice experiences. The prediscussion knowledge assessment consisted of 14 multiple-choice, application-based items (2 items per discussion topic). Application-based items were used because students were expected to be able to apply content in practice. At the end of the 5-week practice experience following the last discussion, students were administered a postdiscussion survey instrument and knowledge-assessment instrument. This postdiscussion test could have been administered at any time after the last discussion, ie, it may have been administered immediately after the final discussion or a few days later at the preceptor’s convenience. The postdiscussion survey instrument was a self-assessment of students’ level of exposure and experience with the discussion topics during a practice experience, how much time they spent preparing for each discussion, the number of sessions attended, and their confidence level with making clinical decisions related to topics of discussions. The postdiscussion knowledge assessment included the same 14 items as on the prediscussion assessment, as well as 14 additional similar items, which were included in an attempt to minimize the effect of recall bias. Mean scores (percentage of correct responses) on the prediscussion and postdiscussion assessments were compared to determine if knowledge gain occurred as a result of the focused discussions.

Prediscussion and postdiscussion assessment means were compared using a Student t test for paired, 2-tailed data. Subgroups were determined ad hoc, based on preliminary data analysis. Descriptive statistics were used when comparative analysis did not apply. A Microsoft Excel spreadsheet was used for data entry and management, and statistical software (StatsDirect, Version 2.7.9, Cheshire, United Kingdom) was used for statistical analyses.

EVALUATION AND ASSESSMENT

The project was conducted continuously over the 2012-2013 academic year, which was divided into eight 5-week practice experiences. Fifty-four P4 students participated in the study. One participant was excluded for lack of a corresponding postdiscussion test, and 1 matched pair of prediscussion and postdiscussion tests was excluded because the student had previously completed them during a prior APPE. These exclusions resulted in a final sample size of 52 P4 students who attended discussions, completed both the pre- and post-discussion tests, and completed both the prediscussion and postdiscussion survey instruments for demographics and self-assessment. These 52 students accounted for 58% of the P4 class.

Table 1 describes the distribution and demographics of students participating by practice experiences 1 through 8. Distribution of students’ participation was fairly even throughout the year, with the exception of practice experiences 1 and 7. Attendance by students for each discussion topic was ≥90%. Prior to participating in this study, 6% of students had already completed a required ambulatory care APPE, 48% had already completed a required internal medicine APPE, and 19% had already completed a pedagogy APPE. Only 6% of subjects had previously completed an APPE that had a significant ambulatory care component or emphasis. At the time of this study, 90% of participants were on their required ambulatory care APPE, 4% were on an elective ambulatory care APPE, and 6% were on another type of APPE, such as pedagogy or research.

Table 1.

Demographic Characteristics of Pharmacy Students Participating in an Assessment of a Multipreceptor Approach to Ambulatory Care Topic Discussions (N=52)

Of the 7 different discussion topics, emphasis during the practice experience varied for each student depending on specific site and preceptor. Figure 1 displays the students’ self-reported level of exposure to the various topics during their APPE. Most students reported moderate to significant levels of exposure to devices/physical assessment, diabetes, dyslipidemia, and hypertension during the practice experience they were on at the time of the study. Conversely, most students reported minimal or no exposure to anticoagulation, asthma, and heart failure during the same timeframe. Most students reported spending between 1 and 59 minutes preparing for discussions, with the exception of devices/physical assessment, for which 56% of the participants reported spending no time preparing.

Figure 1.

Students’ Self-Assessed Level of Exposure to Ambulatory Care Topic Discussions During Current Advanced Pharmacy Practice Experience

Figure 2 displays the students’ self-reported confidence levels for making therapeutic decisions regarding medication use for each of the topics, as assessed by the postdiscussion survey instrument at the end of the APPE. A majority of students indicated some level of agreement (either agree or strongly agree) with the statement, “I am confident making therapeutic decisions…” for all of the topics except heart failure (46%). The highest level of agreement was regarding hypertension (83%).

Figure 2.

Student Self-Assessed Confidence in Making Therapeutic Decisions Regarding Medication Use for Each Discussion Topic. (Note: For each topic, students rated their level of confidence by responding to the statement “I am confident that I can make therapeutic decisions regarding medication use for…”)

The overall score on the prediscussion knowledge assessment ranged from 19% to 98%, with individual discrimination indices (DI) ranging from 0.00 to 0.57. More than 75% of questions had a DI >0.20. The percentage of correct responses on the postdiscussion knowledge assessment ranged from 35% to 98%, with a DI ranging from -0.07 to 0.57. Nearly 67% of these questions had a DI >0.20. Distributions of the prediscussion and postdiscussion assessment scores are presented in Figure 3.

Figure 3.

Distribution of Prediscussion and Postdiscussion Test Scores

Students' mean examination scores increased significantly from 59.1% ± 13.9% at baseline to 76.0% ± 12.4% and 77.1 ± 12.8% on the 14 original and the 14 additional similar assessment items, respectively. Compared with baseline, scores on the postdiscussion assessment represent a 29.4% relative increase in mean percentage of correct responses (p<0.001). Similarly, a significant increase was observed for prediscussion vs postdiscussion assessment scores for all 28 items. There was no difference in mean final scores on the postdiscussion test item sets (ie, original and similar; p=0.60).

When stratified by disease topic, all scores increased except for the topics of asthma and heart failure. Students reported the highest rates of “no exposure” on APPE for asthma (46%) and heart failure (29%), as described in Figure 1. There was not a significant association between exposure and score (Table 2) for either asthma or heart failure. All other disease topics had only 4% to 17% of students reporting “no exposure” on the APPE, so further post hoc analyses were not warranted. There was no significant predictor of performance when comparing the upper 15% to the lower 15% on the prediscussion test in terms of current or previous APPE. Additionally, there was no temporal influence of the student’s APPE throughout the academic year on the prediscussion test means (R²=0.02).

Table 2.

PreDiscussion and Postdiscussion Mean Test Scores, Stratified by Disease Topic

DISCUSSION

Students’ scores on knowledge-based and clinical decision-making questions significantly increased from baseline to postdiscussion assessments following a series of topic discussions related to ambulatory care, which were led by multiple preceptors with extensive clinical experience in their respective topic areas. Performance on the prediscussion knowledge assessment did not differ regardless of prior practice experiences. Postdiscussion test scores increased for all disease states covered in the discussions except for asthma and heart failure. Students reported the highest rates of “no exposure” on APPEs for asthma and heart failure, although there did not appear to be a statistical association between exposure and score for either disease state. The largest net change in test scores was for diabetes; correspondingly, students reported the highest level of exposure to diabetes on APPEs.

Although the vast majority of students reported that they felt confident making recommendations about diabetes and the largest net change in test scores was seen for this disease state, diabetes had the lowest scores overall at baseline and postdiscussions. Additionally, the highest individual score for students was on the asthma prediscussion knowledge assessment. These findings could indicate varying degrees of difficulty in test questions. The postdiscussion knowledge assessment consisted of 14 items identical to those on the prediscussion test along with 14 additional items that were different but assessed the same concept. This design was chosen to decrease recall bias for identical items. However, the inclusion of 14 different items on the postdiscussion knowledge assessment could also be viewed as a limitation when making direct comparisons. Unfortunately, it was not logistically possible to assess another class or students whose ambulatory care practice experience was with preceptors who did not participate in the study. It is possible that students’ performance may improve similarly based on knowledge gained from clinical experience alone. Because we also did not assess students after the practice experience was complete, the effect of this model on knowledge retention is unknown. Given the small sample size and small number of questions per disease topic, it was not possible to correlate the level of exposure to a particular disease state during an APPE with student performance on that topic domain.

When comparing this study to earlier research, there was a similar difference found in scores from prediscussion to postdiscussion tests in ambulatory APPE discussions, with mean scores increasing from 59.1% ±13.9% to 76.0% ±12.4% vs 50.5% ±15.0% to 74.1% ±13.7%, respectively.⁴ Ambulatory care disease states assessed in the earlier research were limited to hypertension, dyslipidemia, diabetes, and anticoagulation, and discussions were based on patient cases, whereas discussions in the current study included the additional topics of heart failure, devices/physical assessment, and asthma. The previous study used identical short-answer questions for both pre- and post-discussion tests, whereas ours used a mix of identical and similar-concept multiple-choice items. Our standard deviation for test scores is similar to the earlier study’s results. Although using multiple-choice items might be expected to lead to a greater variance among scores compared with that associated with short-answer questions, but this does not seem to have been the case for our results. One earlier study demonstrated that using well-constructed multiple-choice questions was a cost-effective, feasible, reliable, and valid tool of assessment that could challenge students as much as short-answer questions.¹¹ This finding may explain the similar variance noted in both the multiple-choice and short-answer tests.

The findings of our study indicate that topic discussions are an excellent supplement to APPEs for enriching learning experiences, but they are not to be used as a replacement for exposure. Our results also suggest that our students need more clinical experience on APPEs related to asthma and heart failure, given that the majority of our ambulatory care faculty members are focused in diabetes management. In the future, in addition to topic discussions, a multipreceptor approach to APPEs within our ambulatory care faculty will be considered for students’ clinical experiences. This same model may prove effective in other APPEs as well (eg, internal medicine). Considering that not all students are exposed to the same patient populations on practice experiences, additional patient cases also will be added to the current discussions. This model also may be an option for teams of preceptors who are geographically separated. Video-conferencing and other technologies could be used for these discussions.

The discussion facilitators appreciated the decrease in the time needed to lead discussions with students. While having shared discussions takes some faculty coordination, the net decrease in time is beneficial. Given that most discussions are 60-90 minutes in length, and discussion-leading responsibilities for the 7 topics are shared among 11 preceptors, preceptors can expect to save several hours on each practice experience. Shared preceptorship also allows for faculty members with expertise in specific areas to lead the corresponding topic discussions.

SUMMARY

After participating in 7 ambulatory care-related topic discussions led by The University of Mississippi School of Pharmacy faculty members with expertise and clinical experience in specific disease states, P4 students’ performance on a postdiscussion assessment instrument improved significantly compared with their prediscussion assessment scores. After participating in the discussions, students also indicated they were confident making therapeutic decisions regarding medication use for each of the ambulatory care topics, with the exception of heart failure.

This multi-preceptor approach was successful with respect to student performance. It could potentially be applied to other categories of pharmacy practice, such as internal medicine or infectious diseases, in which multiple preceptors within the same institution or geographical region could collaborate to have interactive discussions based on their particular area of expertise in order to provide a more beneficial and enriching experience for student pharmacists.