Abstract
Objectives
To evaluate graduating pharmacy students' perceptions of their preparedness for advanced pharmacy practice experiences and the effectiveness of problem-based learning in their preparation.
Methods
A survey instrument was administered anonymously in May 2004 and May 2005 to graduating pharmacy students of the University of Mississippi School of Pharmacy.
Results
Students reported that the areas in which problem-based learning prepared them most effectively for advanced pharmacy practice experiences were retrieval of medical information (80%), discussion of disease states and drug therapies at the basic science level (56%), and evaluation of the appropriateness of a medication regimen based on patient specific information (50%). Areas in which students reported being inadequately prepared included identifying and utilizing drug assistance programs (42%) and processing prescriptions/hospital orders (40%).
Conclusion
Data from 2 consecutive graduating classes supports that problem-based learning is an effective format for preparing pharmacy students for advanced pharmacy practice experiences in a variety of areas.
Keywords: problem-based learning, curriculum, advanced pharmacy practice experiences
INTRODUCTION
Problem-based learning was first utilized in the 1960s by McMaster University in the instruction of medical students. Since that time, this educational tool has been successfully utilized in nursing, dentistry, pharmacy, veterinary medicine, and public health professional programs.1 Knowles defined self-directed learning in 1975 as “a process in which individuals take the initiative, with or without the help of others, in diagnosing their needs, formulating learning goals, identifying human and material resources for learning, choosing, and implementing appropriate learning strategies and evaluating learning outcomes.”2 Margetson described the link between problem-based learning and self-directed learning as problem identification, followed by students engaging in self-directed learning to solve the problem.3 The use of problem-based learning in the training of healthcare professionals incorporates goals for students that are much broader than the acquisition and application of content. The learner is expected to be an active participant who engages in in-depth learning activities based on the analysis of the problem at hand.4 Problem-based learning has been described as both a curriculum and a process that demands from the learner “acquisition of critical knowledge, problem-solving proficiency, self-directed learning strategies, and team participation skills.”2 Problem-based learning should provide the students with the skills necessary to seek and analyze information and then utilize it effectively.5 Consistent features of problem-based learning include the presentation of a problem without providing the information necessary to solve it, students work in small groups, and students are given guidance and provided with feedback by a facilitator (tutor).6
Currently, the University of Mississippi School of Pharmacy utilizes problem-based learning as the sole teaching method during the third-professional year; therefore, all academic disciplines are included. This problem-based learning process consists of completing 3 distinct pharmaceutical care courses with an evaluation of the student in each: (1) Group Participation, (2) Knowledge and Comprehension, and (3) Problem-Solving. Groups of 6 to 8 students are facilitated by a pharmacy school faculty member in 3 sessions per week. In these sessions, students are presented with a progressive disclosure case that emphasizes a particular disease state. In this process, pages of patient information are distributed sequentially so that the student acquires knowledge about the patient progressively. The students also discuss topics related to smaller “mini-cases” in which topics are not as expansive as those illustrated in progressive disclosure cases. The patient information for these cases is provided to the student several days prior to the group meeting so that he/she may research and be prepared to discuss the topic in 1 session. Both types of cases provide the motivating force behind student learning in problem-based learning and are used to engage students actively in their own learning. Problems in problem-based learning are used to stimulate students to actively construct new knowledge that is strongly linked to previous knowledge.7 Students are evaluated by faculty members on their participation in the group process in regard to knowledge acquisition, self-directed learning, and clinical reasoning. Evaluation of group participation validates the importance of acquiring the skills to work effectively in a group and refine communication skills, as well as the acquisition of content knowledge.4
In Knowledge and Comprehension, the students are tested on knowledge and recall. The students are administered a content knowledge examination every 4 weeks which tests the concepts, disease states, and medications that were covered in the problem-based learning group sessions. For Problem Solving, the students are evaluated by examination according to Bloom's taxonomy of application, analysis, synthesis, and evaluation.8 This is accomplished through problem-solving examinations administered every 8 weeks. The focus of the examinations progresses through Bloom's taxonomy. The first examination emphasizes application, for example, pharmaceutical calculations; while the last examination emphasizes the evaluation of a problem, such as providing a comprehensive treatment plan which includes recommendations for drug regimen, patient monitoring, and patient education.
The problem-based learning process at the University of Mississippi School of Pharmacy has evolved over the past 9 years. The Department of Pharmacy Practice on the Medical Center campus provides instruction in the third-professional year and most of the faculty members have now had several years of experience with this teaching format. Changes have been implemented as the faculty members have become more familiar with the mechanics of the program. For example, the testing process has changed from 1 inclusive examination every 8 weeks to the present policy of a Knowledge and Comprehension examination every 4 weeks and a separate Problem-Solving examination every 8 weeks. This was restructured to allow the students to be more focused in their preparation and therefore retain more of the knowledge. It was the intent also that this opportunity for improved knowledge retention would improve the students' preparedness for performance in the advanced pharmacy practice experience settings. However, no data have been collected to evaluate the outcomes of this and other changes. It was the intent of this project to collect data to evaluate students' perceptions of their preparedness for advanced pharmacy practice experiences. The goal of this program evaluation was to identify those areas of instruction that are adequately preparing students, as measured by their perceptions, as well as those areas in need of modification based on these perceptions.
METHODS
The problem-based learning assessment committee, which is composed of School of Pharmacy faculty members, developed the survey instrument using the problem-based learning evaluation benchmarks for group performance during the third-professional year as well as expected competencies at the completion of advanced pharmacy practice experiences. The benchmarks relate to the adequacy of students' preparedness in knowledge acquisition, self-directed learning, and clinical reasoning. Face validity of the instrument was assessed by the committee. The 17-item survey instrument was then transferred to a Scantron-readable format for manual completion. Participants were to rate the adequacy of their preparedness for advanced pharmacy practice experiences on a 1 to 5 Likert scale with 1 = very well prepared and 5 = very poorly prepared.
The anonymous survey instrument was administered to graduating pharmacy students in May 2004 and May 2005. All graduates were asked to complete the survey instrument but participation was voluntary. The students were instructed to recall, as much as possible, only those experiences of the third-professional year as they related to advanced pharmacy practice experiences, when answering the survey items. The survey results were compiled and analyzed by a biostatistician in the Learning Resource Department of the University of Mississippi Medical Center using descriptive frequency analysis.
RESULTS
Data collected from each of the assessments (2004 and 2005) were combined for the conclusions in this report. The percentages reported are a compilation of all responses for both years and are a percentage of students who responded with each choice, not a mean or median. In assessing the data acquired, the authors considered responses of “well” and “very well” as indicating above average preparation, a response of “somewhat” as indicating average preparation, and responses of “poor” and “very poor” as indicating below average preparation. The percentages given are the combined percentages for the 2 data categories (very well + well = above average; poor + very poor = below average). The survey items were organized into 4 main categories: knowledge acquisition, self-directed learning, clinical reasoning, and communication. The individual results by year for each item are grouped according to these heading and are reported as actual percentages in table format (Tables 1, 2, 3, 4, 5).
Table 1.
Senior Pharmacy Students' Responses to Survey Questions 2, 3, 11, and 12 Regarding How Well Problem-based Learning Courses Prepared Them for Advanced Pharmacy Practice Experiences in the Area of Knowledge Acquisition, %
Table 2.
Senior Pharmacy Students' Responses to Survey Questions 4 and 16 Regarding How Well Problem-based Learning Courses Prepared Them for Advanced Pharmacy Practice Experiences in the Area of Self-Directed Learning, %
Table 3.
Senior Pharmacy Students' Responses to Specific Survey Questions Regarding How Well Problem-based Learning Courses Prepared Them for Advanced Pharmacy Practice Experiences in the Area of Clinical Reasoning, %
Table 4.
Senior Pharmacy Students' Responses to Survey Questions 1, 5, and 10 Regarding How Well Problem-based Learning Courses Prepared Them for Advanced Pharmacy Practice Experiences in the Area of Communication, %
Table 5.
Senior Pharmacy Students' Responses to Survey Question 7 Regarding How Well Problem-based Learning Courses Prepared Them for Advanced Pharmacy Practice Experiences in Miscellaneous Areas, %
When asked to discuss disease states and drug therapies at the basic science level, more than half the students (56%) felt they had above average preparation due to the problem-based learning process. Preparation to incorporate knowledge from all academic disciplines when discussing a patient problem was rated by more than a third (35%) of students as above average. Nearly one half (47%) of students felt they had average preparation to accurately perform pharmaceutical calculations, but 26% felt their preparation was below average. Sixty-three percent of students indicated that problem-based learning provided below average preparation to accurately and completely process a prescription/hospital order to dispense medications.
When questioned about their preparation to efficiently retrieve appropriate current medical information, 80% of students responded that problem-based learning provided above average preparation; 15% felt they had average preparation and only 6% found their preparation to be below average. More than one third (34%) of students considered they had achieved above average preparation to improve their writing skills and avoid plagiarism in written assignments; 43% felt they had average preparation in this skill and 23% felt their preparation was below average.
Students reported responses of 40% or greater indicating above average preparation for all survey items related to clinical reasoning. Three of these items related to appropriateness of a drug therapy regimen. Fifty percent of students reported having above average preparation to competently evaluate the appropriateness of a drug therapy regimen based on patient specific information; 45% when based on the patient problem; and 40% when based on the specific characteristics of agents within a drug class. When asked whether the problem-based learning requirement to include specific patient-monitoring parameters for therapeutic outcomes and adverse events in treatment plans prepared them to determine the appropriate monitoring of patients assigned during advance pharmacy practice experiences, 48% of students indicated above average preparation to include therapeutic outcome monitoring and 40% indicated above average preparation to include monitoring for adverse events. Forty-five percent of students rated their preparation by problem-based learning to identify clinically significant drug interactions as above average. When questioned about their ability to evaluate the information given, form hypotheses, and reason through real-life scenarios, 42% of students identified their preparation as above average, 40% as average, and 18% as below average.
Forty-seven percent of students reported above average preparation to communicate with patients about their medications and disease states and 18% reported less than desirable preparation. When this skill was extended to include communication with other healthcare professionals, 41% felt they had above average preparation and over one-third (38%) felt they had below average preparation. Thirty-eight percent of students felt including patient-specific medication education in treatment plans provided them with above average preparation to assist them with counseling patients and 23% felt it provided below average preparation.
When asked whether the problem-based learning process prepared them to identify and utilize state and national drug-assistance programs, only 26% of students reported receiving average or above average preparation, while nearly three fourths (74%) reported receiving below average preparation.
DISCUSSION
Previous studies have used licensure examinations, rating scores, or a variety of types of examinations to obtain data to determine the effectiveness of problem-based learning as a teaching method. These traditional methods of measurement may not be the best methods of identifying advantages of problem-based learning; such as the ability to acquire and retrieve information and then apply it in real-life situations.6 Therefore, this study used the measurement of this ability as the students applied their knowledge in advanced pharmacy practice experiences. Students sometimes overestimate their skills while they are still in the didactic phase of their education and exposure to real pharmacy practice gives them a sense of their true abilities and skills.9 Therefore, the survey instrument was administered just prior to graduation, at which point all advanced pharmacy practice experiences were complete and most students were on campus for other commencement preparations.
The overall results compiled from the 2 years of survey data confirm the use of problem-based learning as an appropriate tool to prepare pharmacy students to perform above average during advanced pharmacy practice experiences based on student perceptions. With the excess of medical information resources available today, students must acquire the skill of identifying appropriate sources. Searching the literature has been identified as one of the most difficult skills to achieve as it requires students to learn about databases, develop search strategies relevant to learning objectives, and use critical appraisal strategies to select literature.10 Self-directed learning within the problem-based learning process requires the student to review appropriate resources of current medical information and 80% of the students reported that this process provided above average preparation for this task during their advanced pharmacy practice experience. Discussion of disease states and drug therapies at the basic science level is an important benchmark in evaluation of the student's participation in Group. More than half the students (56%) responded that this group participation afforded them above average preparation to utilize this skill in their advanced pharmacy practice experiences. Evaluating the appropriateness of a drug therapy regimen based on patient specific information is facilitated in problem-based learning by the format of progressive disclosure cases. This allows the student to build a thorough understanding of the patient as additional information is disclosed. Half of the students (50%) indicated that this learning process supplied them with above average preparation to apply this knowledge to patients encountered in their advanced pharmacy practice experiences. Identifying drug interactions and determining appropriate patient monitoring parameters for therapeutic outcomes were also identified as areas in which nearly half the students (45% and 48%, respectively) stated that they had above average preparation.
Pharmaceutical calculations skills are not taught in the problem-based learning process; this skill is taught earlier in the curricula. Nearly half the students (47%) reported only average preparation in this area; this may not reflect their overall preparation to perform pharmaceutical calculations, but instead, the fact that they did not receive this skill in the problem-based learning process, as students were requested to consider only the instruction provided during the third-professional year. Preceptors were surveyed during this timeframe also and they rated the students as having above average preparation (65%) in performing pharmaceutical calculations.11 The Problem-Solving test peer-review committee has also reviewed these findings and is including more calculations in the early Problem-Solving examination to encourage the students to review this specific skill.
Incorporating knowledge from all academic disciplines when discussing a patient problem was an area in which nearly half of the students reported having only average preparation (47%). Evaluation of the appropriateness of a drug therapy regimen based on the specific characteristics of agents within a drug class was also identified as an area in which average preparation was reported by a large percentage of students (44%). Changes have been made to the Group Participation assessment instrument, putting greater emphasis on knowledge acquisition in each of the academic disciplines (physiology, pathology, pharmacotherapy, and pharmacokinetics). Findings concerning student preparation of appropriateness of a drug regimen based on characteristics of agents within a class will be evaluated by the faculty for modifications in the problem-based learning format to enhance students' skills. Changes made in the format or evaluation will then be monitored to determine whether improvement was accomplished.
Almost two thirds of the students (74%) reported below average preparation in identifying and utilizing state and national drug assistance programs. This has been addressed through the addition of seminar/workshop segments focusing on medication assistance programs. The students' response of very poor preparation (40%) to accurately and completely process a prescription/hospital order to dispense medications is also being evaluated by the faculty for modifications. This skill is also one that is not emphasized by the problem-based learning format, but may need increased focus as a review of previous learning activities.
The focus of this study was student preparation for advanced pharmacy practice experiences, and due to the anonymous nature of the survey instrument we are unable to link individual student performance, but the ultimate standard of student preparation is their performance on the North American Pharmacist Licensure Examination (NAPLEX). The scaled scores for the University of Mississippi were above the national scaled score in all reporting periods from 2001-2005, except one. The passing rate was 100% for these same reporting periods except one (not the same period in which the scaled score was less than national), and even then, the passing rate for University of Mississippi students was higher than the national passing rate.
A limitation of the study is that the demographic characteristics of the students were not collected. Demographic data would have allowed comparisons based on age, gender, previous work history, etc. Also, since the survey was anonymous, test scores from the third-professional year and advanced pharmacy practice experience grades could not be compared with student perceptions. The authors acknowledge that these results are based on students' perceptions, but recognize these are upper-level pharmacy students and their perceptions of how they learn and what experiences were most valuable for them should have merit.
CONCLUSION
Problem-based learning is the instructional format used during the third-professional year at the University of Mississippi School of Pharmacy. A survey tool was developed and administered to 2 consecutive classes of senior pharmacy students to assess the effectiveness of this format in preparing pharmacy students to perform well during advanced pharmacy practice experiences. The problem-based learning assessment committee concluded from the data collected that graduating students report that problem-based learning effectively prepares them for advanced pharmacy practice experiences in a variety of areas, such as retrieval of medical information, discussion of disease states and drug therapies at the basic science level, and evaluation of the appropriateness of a medication regimen based on patient specific information. However, areas in need of further evaluation were identified and included identifying and utilizing drug assistance programs and processing a prescription/hospital order. Modifications based on these findings have been implemented and these changes will be evaluated in future studies.
ACKNOWLEDGEMENTS
The authors wish to acknowledge the assistance of the other members of the problem-based learning assessment committee: Billy Brown, PharmD; Carter Haines, PharmD; and Holly Rogers Moore, PharmD.
Funding for this project was made available through the University of Mississippi School of Pharmacy, Department of Pharmacy Practice.
REFERENCES
- 1. Camp G. Problem based learning a paradigm shift or a passing fad? Med Educ Online. 1996;1:2. Available at: http://www.utmb.edu/meo/f0000003.htm Accessed on October 15, 2005.
- 2.Knowles MS. Self-directed learning: A Guide for Learners and Teachers. Chicago: Follet Publishing; 1975. [Google Scholar]
- 3.Margetson D. Current educational reform and the significance of problem based learning. Stud Higher Educ. 1994;19:5–19. [Google Scholar]
- 4.Charlin B, Mann K, Hansen P. The many faces of problem-based learning: a framework for understanding and comparison. Med Teach. 1998;20:323–30. [Google Scholar]
- 5.O'Shea E. Self-directed learning in nurse education: a review of the literature. J Adv Nurs. 2003;43:62–70. doi: 10.1046/j.1365-2648.2003.02673.x. [DOI] [PubMed] [Google Scholar]
- 6.Beers GW. The effect of teaching method on objective test scores: problem-based learning versus lecture. J Nurs Educ. 2005;44:305–9. doi: 10.3928/01484834-20050701-03. [DOI] [PubMed] [Google Scholar]
- 7.Dolmans D, De Grave W, Wolfhagen HAP, van der Vleuten C. Problem-based learning: future challenges for educational practice and research. Med Educ. 2005;39:732–41. doi: 10.1111/j.1365-2929.2005.02205.x. [DOI] [PubMed] [Google Scholar]
- 8.Bloom BS. Handbook l, Cognitive Domain. New York: Longmans, Green; 1956. Taxonomy of educational objectives: the classification of educational goals. [Google Scholar]
- 9.Ried LD, Brazeau GA, Kimberlin C, Meldrum M, McKenzie M. Students' perceptions of their preparation to provide pharmaceutical care. Am J Pharm Educ. 2002;66:347–56. [Google Scholar]
- 10.Patterson C, Crooks D, Lunyk-Child A new perspective on competencies for self-directed learning. J Nurs Educ. 2002;41:25–31. doi: 10.3928/0148-4834-20020101-06. [DOI] [PubMed] [Google Scholar]
- 11.Lundquist LM, Hogan S. Student preparedness for advanced practice experiences: preceptors' perceptions [abstract] Pharmacotherapy. 2005;25:1498. [Google Scholar]