Evaluation of Pharmaceutical Care Opportunities Within an Advanced Pharmacy Practice Experience

Abstract

Objective

To determine the nature and extent of learning opportunities presented to students during a community advanced pharmacy practice experience.

Methods

Learning opportunities were determined by reviewing students' portfolios. The numbers of unique learning opportunities in which students were engaged were calculated.

Results

While all students had ample opportunities to participate in consultations involving the assessment of new and refill prescriptions and nonprescription drugs, only 78% of students had the opportunity to engage in even 1 comprehensive pharmaceutical care consult over an 8-week period. Students' experiences in providing follow-up care, participating in a patient care project, and shadowing another healthcare professional were also limited.

Conclusion

Analysis of the student portfolios suggested that the learning environment did not provide students with adequate opportunities to develop pharmaceutical care competencies.

INTRODUCTION

In 1999, a new Structured Practice Education Program (SPEP) was introduced within the baccalaureate of pharmacy program at the Faculty of Pharmaceutical Sciences (the Faculty), University of British Columbia (UBC), Vancouver, to meet the expectations set by the Canadian provincial and federal professional and licensure bodies.^1-3 SPEP consisted of 3 different mandatory practice education experiences. The first of the series was an introductory 4-week distribution-focused experience in the community pharmacy setting, scheduled to occur in the summer semester between the third and fourth year of the program. This experience was followed by 2 advanced pharmacy practice experiences (APPEs) in the winter semester (January to April) of the final year of the program. The APPE consisted of an 8-week community pharmacy pharmaceutical care experience completed at 2 different pharmacies (4 weeks at each site) and one 4-week experience completed at an institutional site. The specific experience evaluated in this paper is the community pharmaceutical care APPE completed in the final year.

A planning and evaluation process was undertaken by the SPEP faculty to ensure students completing the 6-credit community APPE were provided with adequate opportunities to meet the pharmaceutical care-related learning outcomes for this experience. The planning process consisted of clearly articulating the learning outcomes and objectives of the experience and determining the competencies students needed to develop. A structured focus group process, involving practicing pharmacists who had previously served as preceptors with UBC and were interested in the patient-centered care aspect of pharmacy practice, was utilized to develop the APPE syllabus. The focus group participants were asked to anticipate the future role of pharmacists within the community setting and consider the Canadian Council for Accreditation of Pharmacy Programs' mandate when developing the syllabus.³ The basis for using such an iterative process was to gain acceptance from preceptors, thus ensuring sustainability. The group identified learning activities that could foster these skills and a minimum number of each learning activity that students should undertake. The intent of introducing a structured syllabus was to ensure that all students received an acceptable level of learning opportunities to practice the desired competencies. The students and preceptors were, however, encouraged to go beyond the minimum number. In addition, a proposed 4-week schedule was developed to assist preceptors in structuring their students' experiences and a minimum weekly student-preceptor contact time was proposed. The APPE development process and syllabus have been discussed in more detail in a previous publication.⁴

An evaluation plan was then developed to assess the effectiveness of the community APPE in meeting the intended goal. The literature on program evaluation proposes that to fully appreciate the outcomes achieved by any program, it is essential for evaluators to first demonstrate the program's ability to execute the intended activities (both in terms of the frequency and nature of the experience).^5-9 In fact, it is suggested that establishing a program's ability to accomplish the desired activities before investing in protocols to measure the outcomes is the most efficient approach.^10,11 Beck, in one of her articles, drew parallels between learning to provide pharmaceutical care and climbing Mount Everest. She suggested that, in both instances, it was not sufficient to provide students with the opportunity to complete the activity, but that it was equally important for students to practice and develop proficiency in the skills, knowledge, and attitudes relevant to achieving the desired performance. In the case of pharmaceutical care, these competencies include the ability to develop relationships with patients that can facilitate discussions about drug-related problems; engage in acquisition and assessment of the patient's drug, disease, and other relevant information to identify actual or potential drug-related problems; engage in informed shared decision-making with patients and other health professionals to prevent and resolve drug-related problems through development of pharmacy care plans, and provide continuity of care by monitoring patients' progress through follow-up care.¹² Based on this premise, a literature search was conducted to ascertain evaluation methods employed by other schools of pharmacy to evaluate their patient-centered (pharmaceutical care) community APPE. The search identified several papers that measured the outcomes of such experiences on community pharmacies, patients, and students; however, the authors of these papers did not discuss to what extent the learning environment at these sites made it possible for students to engage in pharmaceutical care.^5-9 We found no examples in the literature of evaluation processes designed for advance practice experiences that utilized such an approach.

To provide a framework for thinking through the evaluation plan in a systematic fashion, the SPEP faculty used a program logic model approach.^10,11 A logic model is structured around 1 or more questions you wish to answer, and offers a systematic way of presenting the relationships among the program inputs, the planned activities, and the changes or results you hope to achieve. Components of the logic model can be defined as follows: (1) program inputs include the resources or the environment available within the program to carry out the intended activities; (2) results include both program outputs and outcomes; (3) outputs are the direct result of the project activities that can be quantified immediately after completing an activity and are usually described in terms of quantity or scope of the services and products delivered or produced by the program; and (4) outcomes reflect the project goals and assess the degree to which the desired goals are met, and are dependent on the project achieving the stated outputs from the program's activities. Outcomes can vary by timeframe. For example, immediate outcomes, which are most attributable to the project, can be attained within 1-2 years; intermediate outcomes, which reflect results over which the project has a relatively high degree of control but may be influenced by external factors, may take 3-4 years to attain; and final outcomes, which are most likely to be influenced by external factors, can take more than 5 years to attain.^10,11 This project focused on the output aspect of the results.

The objective of this project was to develop and conduct an evaluation process to determine the effectiveness of a community pharmaceutical care APPE in enabling students to execute the learning activities. To fully answer this question, the project (1) measured the extent to which learning opportunities were presented to the students, and (2) described the nature of the students' experiences related to direct pharmaceutical care activities (types of patients encountered and extent of care provided).

METHODS

Design

As part of the Faculty's ongoing quality assurance program, a program evaluation approach was used to assess the effectiveness of the learning environment provided by the participating community pharmacies in meeting APPE's goal. Evaluation of the experience was conducted in the second year after the program's inception. Ethics approval was obtained through the UBC Office of Research Services.

Participants

All 122 senior (fourth-year) students registered in the mandatory community pharmaceutical care APPE in the winter semester of the 2000-2001 academic year were included in this quality assurance project. Prior to the student placement process, students were asked to submit a placement form to the SPEP office with their contact information, home address, and a list of 4 preferred rural placement locations in British Columbia (in addition to placement in the greater Vancouver area) for the SPEP faculty to consider. Only community pharmacies that had agreed to provide students with the opportunities to participate in the designated learning activities (as determined by telephone by the SPEP Director) were used as APPE sites. Once these sites were identified, students were randomly placed in one of their preferred locations. The students' 8-week community pharmaceutical care APPE was completed in 2 different pharmacies (4 weeks at each site). While some students completed their 2 community experiences consecutively, others had an institutional experience scheduled in between. Students did not receive remuneration for their experience. To preserve student anonymity, all student identifiers were removed prior to collating and analyzing the data for this project.

Intervention

The community pharmaceutical care APPE syllabus was designed to provide students with the opportunity to hone pharmaceutical care-related competencies, by engaging in various direct and indirect pharmaceutical care activities. These activities are described in Table 1. Students were expected to engage in all pharmaceutical care activities using the framework defined by Strand and Hepler.^13,14

Table 1.

Pharmaceutical Care Activities for a Community Advanced Pharmacy Practice Experience

Prior to the start of the APPE, all students were required to purchase and review the APPE manual, attend a mandatory face-to-face orientation session, and complete an open manual, online quiz reinforcing the APPE expectations and requirements (this included a discussion on the type and extent of the documentation required). Preceptors were also provided with a copy of the APPE manual; however, no face-to-face education was offered. Although both the students and the preceptors were expected to achieve the APPE expectations by providing students with opportunities to engage in the required activities, there was no negative consequence to the student or the preceptor if the expectations were not met. However, the SPEP office called all sites where students did not achieve the learning expectations to obtain feedback and reiterate the APPE expectations.

Program Evaluation Process

The first step in developing the program logic model was to establish a clear question that needed answering. For this project, the question of interest was: “How effective were the participating community pharmacies in creating a learning environment that enabled students to execute the learning activities in accordance with the APPE's objectives”? To assess effectiveness, this project focused primarily on outputs from the activities and not on final outcomes.

The learning environment created at the APPE sites was seen as the primary input, as it was believed that the environment would either facilitate or hinder learning opportunities and strategies. It was also the expectation of the Faculty that students be given the chance during their APPE to practice their pharmaceutical care competencies with diverse groups of patients. Thus, both the extent of the learning opportunity and the nature of the learning were examined to determine the effectiveness of the learning environment.¹⁵ The activities of interest included both direct and indirect pharmaceutical care activities. A summative method was utilized to obtain the relevant student data, using a snapshot approach of students' work at the end of the clerkship experience to quantify the outputs.

Data Collection

Evidence for evaluating the students' learning environment was collected by reviewing portfolios from all 122 students. Students were required to submit learning portfolios at the end of each 4-week community experience, for a total of 2 portfolios per student over an 8-week period. The student portfolios served 3 purposes. First, for the SPEP faculty, the portfolios served as a record of direct and non-direct pharmaceutical care activities undertaken by the students during each of the clerkships and as an assessment tool demonstrating students' clinical reasoning skills and the type and extent of pharmaceutical care provided by students. Second, for the students, the portfolios were meant to facilitate learning through reflection. The items to be included in the portfolios were identified by the SPEP faculty and included documentation to support the students' involvement in the required learning activities. Third, students utilized a uniform and systematic process for documenting their experiences. These documentation tools were designed to facilitate the process of pharmaceutical care and promote self-directed learning. Thus, these documents were, in essence, artifacts of activities performed by the students that stemmed from their work. Table 1 outlines all the APPE activities along with the corresponding documentation that students were required to complete. Students were provided a sufficient quantity (in hard copy) of documentation tools prior to the start of their experience, and were familiar with using the tools from prior exposure during their undergraduate program.

All non-direct pharmaceutical care activities were captured on the student's 1-page reflective learning log, completed at the end of the activity. For their pharmaceutical care activities, students were provided with documentation tools to facilitate the provision of pharmaceutical care, continuity of care, and critical thinking. For the comprehensive pharmaceutical care activity, these tools served to encapsulate all relevant patient information acquired by the student during the interviewing process and summarized all the drug-related problems identified during the assessment of these patients.¹⁶ Students also completed pharmacy care plans for all initial and follow-up care that they provided when resolving or preventing drug-related problems during all direct pharmaceutical care activities (assessment of new and refill prescription, non-prescription consultations, follow-up care, and comprehensive pharmaceutical care). The care plan also captured the contact person for the care plan (physician, patient, other) and whether the students' recommendation had been accepted by the relevant stakeholder(s). These tools have been described in detail in the literature.¹⁶

Analysis

Data from portfolios submitted by all 122 students were included in the analysis. Once the APPE Director had reviewed the student portfolios and allocated a course grade, relevant information from the portfolios was extracted and entered in a database program (Microsoft Access 2000) designed specifically for this purpose. The data were then transferred onto a spreadsheet (Microsoft Excel 2000) and quantitatively analyzed. Frequencies and standard deviations were calculated from the number of unique learning opportunities students engaged in during their two 4-week community experiences, and reported as the number of events per student over an 8-week period.

RESULTS

A total of 244 learning portfolios (2 per student) were reviewed. The results of the project have been summarized in 2 categories: (1) the extent of learning opportunities provided for all activities, and (2) the nature of the students' experiences related to pharmaceutical care activities.

Extent of Learning Opportunities

Evaluation of students' portfolios indicated that all students exceeded the minimum expectations for assessing patients requiring new and refill prescriptions and nonprescription medications. However, only 78% of the students had the opportunity to engage in 1 comprehensive pharmaceutical care consult over the 8-week period, and notably, 22% never had the opportunity to participate in this activity. With respect to non-direct pharmaceutical care activities, all students were given adequate opportunity to answer drug information questions (at least 2 per week), and all had the opportunity to discuss therapeutic topics with their preceptors on a weekly basis. However, only 32% of the students had the opportunity to work on a site project, and an equivalent percentage (32%) shadowed a nonpharmacist healthcare professional. Of the professionals shadowed, 16 students were with a physician, 15 with a community nurse, 2 with a dietician, and 6 with other health care professionals. In addition, 9 other students shadowed a pharmaceutical company representative and 1 student shadowed a clinical pharmacist working in a specialty outpatient clinic.

Nature of Pharmaceutical Care Experience

Types of patients seen

The mean age of patients who were provided comprehensive pharmaceutical care by the students was 64 ± 18 years. Thirty-seven percent of these patients were male and 63% were female. The 10 most common medical conditions/symptoms encountered by students during these comprehensive pharmaceutical care consults included: hypertension (23%), osteoporosis (14%), musculoskeletal pain (13%), hyperlipidemia (11%), gastrointestinal distress (10%), type 2 diabetes (9%), asthma/chronic obstructive pulmonary disease (8%), insomnia (7%) and bronchitis/pneumonia (5%). Eighty percent of the medications assessed during comprehensive consults were prescription related and 20% were nonprescription related. For new and refill prescriptions, the 10 most common therapeutic drug classes involved with a drug-related problem included: oral antibiotics (35%), inhaled corticosteroids/ beta-agonists (10%), nonsteroidal anti-inflammatory drugs (9%), warfarin (9%), ACE-inhibitors (8%), oral contraceptives (8%), statins (7%), diuretics (5%), selective-serotonin receptor inhibitors (5%), and benzodiazepines (4%). For the nonprescription drugs, the 10 most common ailments students provided consults for included: cough and cold/ sinusitis/allergic rhinitis (41%), pain (11%), cough (10%), dermatological (10%), dermatological infections (6%), ears/eyes/nose (5%), constipation (5%), gastrointestinal distress (5%), osteoporosis (4%) and diarrhea (3%).

Extent of Care Provided

Eighty-seven drug-related problems were identified for the 96 patients who received comprehensive pharmaceutical care, approximately 1 drug-related problem per student (number based on the 95 students who participated in this activity). Figure 1 shows the distribution of drug-related problems identified by students during their comprehensive pharmaceutical care assessment. About half of these patients (46%) received a follow-up consultation from a student, with an average of 1 follow-up per drug-related problem per patient. Although the students made an intervention for 76% of the drug-related problems they identified, by the end of their clerkship only 22% of these drug-related problems were resolved. During assessment of new and refill prescriptions and nonprescription consults, students identified and addressed a total of 6,874 drug-related problems over an 8-week period. Seventy-two percent of these drug-related problems were for nonprescription medications and 28% for prescription medications. Figures 2 and 3 show the distribution of drug-related problems identified for the prescription and nonprescription medications, respectively.

Figure 1.

Distribution of drug-related problems (DRPS) identified by students during comprehensive pharmaceutical care consultations (n = 87 DRPs)

Figure 2.

Distribution of drug-related problems (DRPS) identified by students during assessment of new and refill prescriptions (n = 1,897 DRPs)

Figure 3.

Distribution of drug-related problems (DRPS) identified by students during consultation of non-prescription products (n = 4764)

For new and refill prescriptions and nonprescription medications, the most common intervention made by the students was for initiating a new drug (53%), followed by counseling (28%), discontinuing and starting a new drug (5%), dose or dosing interval change (5%), referral to a physician (4%), introducing measures to enhance adherence (2%), discontinuation of a drug and new drug not started (1%), requisition of laboratory tests (1%), and other interventions (1%). Students followed up on about half of the interventions made (3,282 follow-ups), with a maximum of 1 follow-up per intervention. The patient was the most common point of intervention (for 83% of the interventions), physicians were contacted for 13% of the interventions, and the remaining 4% included contacting caregivers such as parents and family of elderly patients.

DISCUSSION

The objective of this project was to determine if the learning environment created by the newly developed community pharmaceutical care APPE was effective in enabling students to execute the learning activities as intended. The data collated from the students' portfolios suggests that the learning environment at the APPE sites fell short of providing sufficient learning opportunities for students to achieve this objective. While students had the opportunity to interact with patients who offered a wide range of learning possibilities through their diverse drug and disease profiles, students' inability to fully carry out the pharmaceutical care process with these patients restricted their learning. In addition, the students' learning was limited by the lack of opportunity to engage in the proposed nondirect pharmaceutical care activities introduced to complement the pharmaceutical care activities; for example, participating in interprofessional collaboration, population health through disease prevention and health promotion activities.

Both the American Council on Pharmaceutical Education (ACPE) and the Association of Faculties of Pharmacy of Canada (AFPC) have charged schools of pharmacy across North America with the task of ensuring students are adequately skilled in pharmaceutical care so they are able to create opportunities and expand professional boundaries upon graduation.^3,17 However, for pharmaceutical care to be fully appreciated by students, the current understanding of adult learning principles suggest that the pharmaceutical care concepts and principles taught in the classroom must also be experienced in a real-life setting.^12,18-20 It was based on this premise that the community pharmaceutical care APPE was introduced into the UBC curriculum.

The documentation from student portfolios suggests that the community pharmacy sites had little difficulty providing students with the opportunity to engage in learning activities that the pharmacy staff regularly encountered in their practices. These included activities related to dispensing of new and refill prescriptions, providing consultation on nonprescription products, and answering drug information questions. During these activities, students' assessment resulted in identifying over 6,000 drug-related problems, where 72% of the drug-related problems were for nonprescription medications and 28% for prescription medications. The students encountered a variety of therapeutic drug classes during these activities, and preceptors set aside adequate time to review and discuss these therapeutic areas with their students.

In contrast, the community pharmacy sites had difficulty creating learning opportunities involving activities that the pharmacy staff did not commonly engage in, such as comprehensive pharmaceutical care and follow-up care. Only 78% of the students were given the opportunity to participate in the provision of comprehensive pharmaceutical care and, at most, each of these students had no more than one such interaction over an 8-week period. This experience was clearly less than the expected “1 comprehensive pharmaceutical care experience per week” proposed by the focus group. This limited exposure to comprehensive pharmaceutical care assessment clearly minimized the students' opportunity to learn about a variety of diseases and patients' experiences with these diseases in a real-life context.

Students' experiences with providing continuity of care through follow-up were also limited. For both comprehensive pharmaceutical care and assessment of new and refill prescriptions activities, students provided follow-up care to only half of the patients they intervened with. Only 46% of patients for whom comprehensive pharmaceutical care was initiated received any follow-up care from their students. In addition, only 22% of the drug-related problems that students identified had been resolved by the end of their clerkship period. There was no indication on the students' care plans of who would be taking responsibility for the unresolved drug-related problems. Finally, none of the students were able to meet the minimum quota established for providing follow-up care, which was a minimum of 80 patients over an 8-week period (2 per day). On average, each student provided follow-up care to 27 patients over an 8-week period.

Interestingly, results from a recently conducted survey by the ACPE to assess the current state of core advance pharmacy practice experiences in US colleges and schools of pharmacy suggested that large percentages of students did not get sufficient opportunities to participate in and carry out activities, tasks, and processes essential to pharmaceutical care.²¹ Activities in which students were most frequently engaged tended to involve dispensing medications. Although students devoted some time to other activities expected to foster the development of professional care competencies, they were less involved with activities such as designing and implementing drug plans for a specific patient. As highlighted by the paper's author, the greater concern with the results of the study was the low response rate, with only 56% of directors of experiential education and 20.5% of preceptors responding to the survey. If the respondents represented the directors and preceptors who were most engaged, then the actual number of APPEs not meeting ACPE guidelines may be even higher.

The results from the ACPE survey suggest that pharmacy students may be graduating without achieving the desired competencies.²¹ The current literature proposes that activities related to assessment of new and refill prescriptions and nonprescription consultation are essential for developing competencies for initiating pharmaceutical care and triaging patients to different levels of pharmacy care, but offer limited opportunity to develop the more sophisticated pharmaceutical care skills associated with longitudinal care.²² Sophisticated skills such as maintaining effective pharmacist-patient relationships, assuming responsibility for the management of drug-related problems, and evaluating the patients' drug therapy through follow-up can only be developed in a model where care is provided over time.²² Thus, our results showing the limited experience students had with the comprehensive PC and follow-up activities, was important information for the Structured Practice Education Program faculty and support the findings of the ACPE survey and observations made in the current literature.

Studies involving pharmaceutical care have also suggested that the distribution of drug-related problems identified during filling of new and refill prescriptions are different than those identified during the provision of comprehensive pharmaceutical care. In the former situation, where the problem-solving process is often linked with screening of the prescription and the patient profile at the time of dispensing, the drug-related problems identified are often related to the safety, correctness, and completeness of the prescription. In contrast, in comprehensive pharmaceutical care where the emphasis is on evaluating the patient's overall drug-related needs, the distribution of drug-related problems will also include situations where patients require a drug therapy but are not receiving it, and identification problems related to wrong drug, adherence problems, and actual adverse drug reaction.^23-25 The distribution of drug-related problems identified by the students in this project are consistent with what has been observed in the literature. Thus, the limited exposure to comprehensive pharmaceutical care within this APPE restricted the breadth of drug-related problems for which students could take responsibility.

In addition, the learning environment was deficient in providing students with the opportunity to participate in patient care projects. There is a growing appreciation of the importance of disease prevention and health promotion activities in today's health care system and pharmacists need to prepare themselves for taking on a greater role in preventative care.²⁶ The intent of the patient care project was to expose students to health care activities that could serve populations of patients rather than just individual patients. Examples of patient care projects that UBC students have completed include disease management and health promotion clinics, brown-bag forums, presenting seminars on selected health topics, writing on a health issue in the community newsletter, and developing protocols for carrying out similar projects in the future. Only 30% of the sites provided their students with the opportunity to work on such patient care projects.

Students were also provided with few opportunities for collaborating with and shadowing other health care providers. When making interventions for drug-related problems related to new and refill medications, physicians were contacted for only 13% of the interventions. In addition, only 34% of the sites were able to create opportunities for students to spend time with other health care providers through shadowing. When designing the clerkship syllabus, the focus group believed that students needed to develop interaction skills in social diplomacy and tact if others were going to listen to their consultation, and that such skills should be acquired during APPE experiences. This notion has also been expressed by others such as Broadhead, who so eloquently stated: “pharmacists' apparent worth to physicians is created and sustained by virtue of their skills in interprofessional diplomacy and deference, not because physicians relinquish authority over drug therapy in any formal or permanent sense.”¹⁹ One explanation for the low percentage of interprofessional collaboration opportunities provided by the clerkship sites may be related to limited collaborative relationships between most community pharmacists and other health care professionals.^19,24

Future Implications

The SPEP faculty was aware that most of the recruited APPE sites had not incorporated pharmaceutical care into their practice, and was cognizant that learning opportunities to develop competencies related to pharmaceutical care would not be abundant in pharmacies that were modeled after traditional practice, where the majority of interactions occurred in response to filling a prescription or consulting for a nonprescription product.²² In spite of this, the SPEP faculty believed that if they recruited only those pharmacies whose owners, managers, and/or pharmacists were receptive to the concept of pharmaceutical care and committed to providing students with the necessary opportunities to practice pharmaceutical care, it would enhance the likelihood of attaining the objectives of the APPE. However, our project demonstrated that simply recruiting community pharmacy sites that are enthusiastic about pharmaceutical care and student learning is not sufficient.

Data from this project helped the SPEP faculty to identify the specific areas of strengths and weakness with the current community pharmacy pharmaceutical care APPE. The next step was to understand factors that influenced the students' and preceptors' ability to execute the clerkship experience as intended. Students' and preceptors' survey instruments were designed for this purpose, and feedback obtained from these surveys helped to develop interventions that could enhance the clerkship's learning environment. Briefly, the common themes that emerged from the students' and preceptors' feedback included the need to introduce a preceptor educational program to familiarize preceptors with the APPE expectations and evaluation process, and to discuss the pharmaceutical care model and process and how learning opportunities related to pharmaceutical care could be created when their practices did not currently lend themselves to such opportunities.

This project helped the SPEP faculty to secure funding from several community pharmacy chains within British Columbia to develop and evaluate an enhanced model of community pharmacy pharmaceutical care APPE. The enhancement was undertaken in a stepwise fashion and we are currently in the fifth year of this process. The enhancement was initiated at a pilot level involving a handful of sites and students, and gradually expanded to include all sites involved with APPE. Once again, a program evaluation approach was undertaken to develop and evaluate the effectiveness of various interventions that could enhance the students' and preceptors' experiences. These interventions have included introducing a preceptor education program, increasing the length of the experience from 4 to 8 weeks, and providing students with a more comprehensive orientation session prior to starting their clerkship. Experiences from these interventions will be disseminated once the evaluation process is completed. As the APPE has become more standardized, both students and preceptors have been held accountable for achieving the expectations. If a student does not meet the minimum expectations, the student is not permitted to pass the APPE and the preceptor is required to complete a preceptor education program before being reinstated in the program.

Limitations

This project did not focus on final outcomes, but on analysis of the experiential learning environment. Although outcomes are important when assessing the impact of the experience, the SPEP faculty felt it was important to first ensure the APPE was able to produce desirable outputs (ie, the sites offered an acceptable level of learning opportunities), before establishing measures to assess the impact of that experience.

CONCLUSION

This article illustrates how one school of pharmacy incorporated a systematic evaluation process to determine whether the learning environment provided by the community pharmaceutical care APPE was of adequate intensity to provide students with the opportunity to achieve the desired competencies. This was accomplished by examining students' experiences during the community APPE. The data evaluated indicated that while students had the opportunity to interact with patients who offered a wide range of learning possibilities through their diverse drug and disease profiles, students' inability to fully carry out the pharmaceutical care process with these patients restricted their learning.