An Outcomes-based Approach to Curriculum Development in Pharmacy

Abstract

Objective

To implement an outcomes-based approach to pharmacy curriculum development.

Design

Desired learning outcomes were identified; course content, learning activities, and assessment instruments were designed; and evaluation was conducted and feedback obtained to refine the curriculum.

Assessment

All professional skills, 4 generic capabilities, and the coherence of the curriculum scales showed significant improvement compared to graduates' performance under both the old and transitional curriculum.

Conclusion

An outcomes-based approach to pharmacy curriculum development provided convincing evidence of enhancement to the curriculum. Such an approach should be considered when implementing or revising pharmacy curriculum.

INTRODUCTION

The School of Pharmacy at the Chinese University of Hong Kong (CUHK) was established in 1992 and is the only pharmacy school in Hong Kong until 2009. The School offers a 3-year Bachelor of Pharmacy degree (BPharm) as its undergraduate degree. Students begin the BPharm program after completing 7 years of secondary school under the British secondary education system; thus, the mean age of students entering the program is19 years. The completion rate for the program is 100% due to the high academic standards for students admitted to the program. The average number of students admitted to the program is about 30 per year. The graduates from this program are required to complete a 1-year internship before licensure.

The initial BPharm program of the School was modelled after the former Australian/British pharmacy curricula and there was no clinical pharmacy experience component in the curriculum. In 2001, the School of Pharmacy Curriculum Committee revised the BPharm curriculum to incorporate not only an integrated curriculum (eg, organ system-based instruction of combined anatomy/physiology, combined pharmacology and therapeutics) but also various pharmacy clerkships and projects in clinical pharmacy, community pharmacy, industry, and research (Table 1).

Table 1.

Bachelor of Pharmacy (BPharm) Degree Program Reflecting an Outcomes-based Approach to Curriculum Development

^bStudents are required to take and pass 63 units of required courses (including 12 units of Pharmacy Clerkship/Project I & II) and take 12 units of Major Elective/Prerequisite courses in which 5 units must be passed for graduation. In total, a minimum of 68 units of courses must be passed before graduation

^aMajor elective or prerequisite courses

Along with these revised courses, the curriculum committee utilized an outcome-based approach to curriculum development.¹ Outcomes-based education implies that when programs are planned, desirable learning outcomes are identified and considered in the formulation of the plans. Course content, learning activities, and assessment are designed to be consistent with the achievement of the desired learning outcomes. Evidence is then gathered to determine whether the desired outcomes are being achieved. This evaluation provides feedback to ensure that elements in the teaching and learning environment are acting in concert to facilitate the nurturing of the desired outcomes.

The aspects of curriculum planning can be incorporated into a model (Figure 1), which shows how they relate and influence each other. The starting point of the model is identification of student learning needs. These lead to 5 elements of the curriculum which need to be incorporated into curriculum plans – desired learning outcomes, fundamental concepts, learning activities, assessment, and feedback. These are shown as related to student learning needs by a feedback loop to complete the cycle. The central part of the model might be compared to the interaction model of curriculum development, one of those commonly cited in curriculum development texts.² This consists of the 4 elements of objectives, content, activities, and evaluation, with each element permitted to interact with the others.

Figure 1.

Curriculum Planning Model Illustrating an Outcomes-based Approach to Curriculum Development.

The 5 central elements are linked and interrelated. In the interactive model of curriculum development, the linking implies that the order of dealing with each element is not predetermined. In our model, the linking is intended to emphasize the importance of achieving consistency between the curriculum elements in the model. For a curriculum to succeed in achieving the intended learning outcomes, each element needs to be related to the other elements in a coherent manner.

The main objective of this work is to provide an example of an outcome-based approach in pharmacy curriculum development.

DESIGN

Identifying Desired Learning Outcomes

We first considered students' learning needs, which led to specification of desired learning outcomes. The desired learning outcomes were identified through the collaborative efforts of the School's curriculum committee, which consisted of the program director, faculty members with experience in both pharmaceutical science and pharmacy practice, and a consultant with extensive experience in teaching and learning in higher education. The committee examined the pharmacy curricula from various parts of the world, including the United States, United Kingdom, Canada, and Australia. The International Pharmaceutical Federation's Statement of Policy on Good Pharmacy Education Practice was also consulted.³ Brainstorming sessions were held to determine the students' learning needs. In the professional course, students' learning corresponded to the knowledge, capabilities, skills, and qualities required by practicing pharmacists.

The desired learning outcomes were then formulated under the following main areas: professional knowledge, professional attitude, work manner, use of information, and counselling skills.

In addition to these professionally related capabilities, there was also recognition that graduates would need a broader range of capabilities, such as critical thinking, self-managed learning, adaptability, problem-solving skills, communication skills, interpersonal skills and group work, and computer literacy. These capabilities were initially identified by the University's Centre for Learning Enhancement and Research for the University-wide Student Engagement Questionnaire and adopted by the pharmacy program.

Revisions to Curriculum

The next task was to design content, learning activities, and assessment. (Our pharmacy curriculum and a brief description of each course is available on our Web site: www.pharmacy.cuhk.edu.hk.) The curriculum committee closely examined the existing curriculum and concluded that 3 major areas of revision were needed: (1) introduce students to the professional practice of pharmacy at the onset of their pharmacy education, (2) provide opportunities for students to practice what they learn in real-life environments, and (3) integrate related courses to reduce duplication of materials and increase relevance between subjects.

The curriculum committee established the basic framework for the new courses and then presented the proposed changes at the annual curriculum retreat and solicited faculty members' comments and suggestions. After obtaining faculty support for the curriculum revision, individual faculty members were appointed as course coordinators and charged with the responsibility of planning and implementing these courses.

Three new courses were added during the revision process. The first course was Introduction to Pharmacy, placed in first term of year 1. This course provided an overview of the role of pharmacists, what they do, and the skills they need to fulfil their professional responsibilities. Both academic teaching staff members and practicing pharmacists were involved in giving lectures and seminars in their area of expertise. Student learning took place through seminars, student presentations, and practice site visits. This was a foundation course that motivated students by showing them what their future role would be in different practice environments and why they needed the knowledge and skills taught in other courses.

Another new course was Pharmaceutical Research Methods & Techniques, placed in the second year. The curriculum committee recognized the increasing role played by pharmacists in actively acquiring new knowledge to improve patient care, and added this course to provide students with an understanding of the principles and methods of pharmaceutical research in pharmaceutical sciences and pharmacy practice. Biostatistics was incorporated in this course. The course is taught with lectures, tutorials, and workshops. This course also serves as a foundation for literature evaluation and provides students with the research skills they will need to complete their final year project.

The third new course was Pharmacy Clerkship/Project, which was taught in both terms of the third year. The clerkship consisted of rotations in hospital, community, and industrial pharmacy settings so that all students would gain experience in all 3 practice areas. The students also had to complete a research-oriented or practice-based project to fulfil the course requirements. A pharmacist tutor was appointed at each clerkship site. The tutors monitored the students, involved them in site-specific professional activities, and served as role models. The students learned the skills and knowledge needed to achieve a specific competency level of practice at the respective site as these were important in developing the desired professional learning outcomes. The instructional activities included project proposals, implementation and reports, oral presentations, workshops, short written assignments, site visits, and ward rounds. To ensure consistency in the assessment of students from different tutors, standardized assessment criteria were developed and reviewed by the practice-based faculty members and then implemented. The criteria used corresponded to the desired professional and generic outcomes as demonstrated by students' knowledge, attitudes, and skills at the clerkship sites. Other than the traditional written examination, students' oral/written communication and analytical skills were also assessed through case presentations, written project proposals, and final reports. Both formative and summative assessments were incorporated into the courses.

Apart from these new courses, an integrated approach was proposed for several existing courses. One example was Pharmacology and Therapeutics, which consisted of a series of 4 courses in years 2 and 3. Originally, Pharmacology courses were taught in year 2, followed by Therapeutics courses in year 3. With the integrated approach, the course has been changed to a modular basis. The course content is arranged into various modules according to organ systems. Within each module, the pharmacology of drugs affecting an organ system is taught, followed by the therapeutics. Such an approach allowed students to better appreciate the interrelation between the mechanism and site of drug action and its application in disease management. Another course example was Anatomy and Physiology. In a similar manner, this course was also structured based on organ systems where the teaching of the anatomic structure was followed by a discussion of its function within the body.

Because the curriculum changes were implemented in stages, one class of students completed all year 1 and some year 2 courses from the old curriculum and the remainder of year 2 and all of year 3 courses from the new curriculum. Hence, their curriculum was a hybrid of the old and new system (ie, a transitional curriculum). Although this class experienced some minor problems and obstacles associated with the early stage of implementation of the new curriculum, this class indicated in a student survey that the new pharmacy clerkship/project courses taken in their final year was the most valuable aspect of their curriculum.

Evaluation and Refinement

Our curriculum-planning model used feedback as the central element. It also featured a feedback loop and interaction between the components. This implies that evaluation and curriculum refinement are ongoing processes. Feedback needs to be sought from a variety of sources to inform regular improvements to the curriculum. Therefore, the curriculum committee met regularly to review feedback and to suggest improvements.

An important aspect of evaluation used by the School of Pharmacy was the annual graduate survey conducted about 6 months after graduation (Appendix 1). The survey was designed specifically to garner feedback from graduates on their attainment of the desired learning outcomes by stating their level of agreement or disagreement with the statements. Thus, the survey instrument had scales for each of the 5 professional areas listed above, with items relating to facets of competence in each area. The instrument also included scales for the more general capabilities from the Student Engagement Questionnaire, a questionnaire used for quality assurance purposes at the Chinese University of Hong Kong.^4-7 The graduate survey instrument, therefore, provided measures of students' perceptions of the achievement of the desired learning outcomes.

Administration of the graduate survey started with the last group of graduates from the old curriculum and continued annually. It was possible to track the effect of changes to the curriculum by monitoring student perceptions of the achievement of the desired outcomes.

Apart from the graduate survey, feedbacks on the revised curriculum were also collected from the undergraduate pharmacy students through student forums or focus group reviews. Written comments collected by the class representatives were reviewed by the curriculum committee, which then formulated plans for further curriculum refinement. Based on the students' feedback, the pharmaceutical chemistry courses were further streamlined and the timetabling of the courses in Dosage Form Science and Basic Dispensing Techniques was synchronized so the courses complimented each other. Feedback from faculty members was also collected. For example, the overall course coordinator of the pharmacy clerkship/project courses and the section coordinators for the clinical, community, industrial, and research tracks met on a regular basis to discuss the results of the tutors' assessments of students' performance and students' evaluations of the clerkship sites and tutors. Responses to the students' and tutors' feedback and the action plans were all well documented by the curriculum committee. Assignments and examinations were utilized to prove the content and skills learned, when applicable. The results of course examinations were discussed by faculty members in the Exam Panel Meeting held at the end of every semester.

Data Analysis

The questionnaire consisted of 40 items of which 5 scales referred to professional knowledge or skills, 7 to generic competencies, and 1 to the coherence of the curriculum. The 5 scales on professional knowledge or skills were initially developed by 2 experienced faculty members with pharmacy practice degrees, while the other scales on generic competencies and curriculum coherence were adopted from the Student Engagement Questionnaire, which had been validated with students in the same University,^4-7 each measured by a set of items ranging from 2 to 7. These 40 items were then reviewed by all teaching staff members at the School, with final revisions made by the curriculum committee and adopted by the School. Data analyses proceeded through a series of steps. First, exploratory factor analyses using the overall sample were applied to each of the 7 scales measured by more than 2 items for data reduction separately to examine for unidimensionality of the scales. We used 3 criteria for factor retention: (1) a Scree plot shows a sharp drop, (2) the interpretability of the factors, and (3) factor loadings > 0.3. Second, we checked for multicollinearity among the 13 competencies by examining conditioning indices and variance-decomposition proportions associated with each competence based on their composite scores.⁸ Multicollinearity is closely related to construct reliability and discriminant validity,⁹ in that if items are too highly correlated across competencies, the instrument lacks construct reliability and discriminant validity. Too large a conditioning index (> 30) and at least two-variance-decomposition proportions greater than 0.5 in the case of only one near dependency, or the sum of the variance-decomposition proportions greater than 0.5 indicating multicollinearity.¹⁰ Third, reliabilities of the scales were then further assessed by computing the Cronbach alpha values for the overall sample.¹¹ Since a value of 0.5 is considered not to seriously attenuate validity,¹² this value was chosen as the cut-off point for our data analysis. Given low levels of multicollinearity and acceptable reliability, the corresponding items of each scale were combined to form scales by computing the mean score. Fourth, MANOVA comparing the mean levels of the 13 outcomes among data from 4 years (from 2003 to 2006) were carried out. Finally, post hoc Tukey procedures for pairwise comparisons were performed to identify source of differences in mean levels, if any. For all tests, a p value < 0.05 was considered significant. All the analyses were performed using SPSS (SPSS Inc, Chicago, Illinois).¹³

ASSESSMENT

The survey results on the achievement of learning outcomes collected from the graduates of the BPharm program during 4 years, beginning with the old curriculum to 2 years following completion of the new curriculum, are shown in Figures 2 and 3. Twenty-eight (100%) of the graduates responded to the survey in 2003; 29 (100%) in 2004; 29 (93.5%) in 2005; and 25 (86.2%) in 2006. For each of the 7 exploratory factor analyses, 1 factor was retained based on the 3 criteria, and each of them explained the percentage of variances of items in the scale ranging from 50.9% to 80.6% (Table 2). The largest conditioning index was 5.8, which indicated low levels of multicollinearity among scores of the 13 competencies. The internal consistencies of all 13 scales for the overall sample were above the cutting point of 0.5 and 8 of them were above 0.7 (Table 2). We then constructed a composite score based on the mean scores for items in each of the 13 scales. Significant differences among the mean scores of the majority of the outcome variables were obtained in the MANOVAs. Of the 13 individual composite scores, only 3 scores (self-managed learning, problem solving, and computer literacy) failed to achieve a significant difference among different years (Table 3). Nonetheless, the self-managed learning and problem-solving scales were comparatively higher than the other generic learning outcomes. Since the longitudinal data for the problem-solving scale was showing an upward trend, no specific action was taken by the curriculum committee. In 2005, when the committee noticed a drop in the self-managed learning outcome in the new curriculum (Figure 3), a review of the learning activities of all the BPharm courses was undertaken. Following the review, the committee made a recommendation to some of the course coordinators to devote a minimum percentage of the course time to self-learning activities. The mean score of this outcome improved in the following year.

Figure 2.

Longitudinal data for professional learning outcomes.

Figure 3.

Longitudinal data for generic learning outcomes.

Table 2.

Measures of 5 Professional and 8 Generic Learning Outcomes in Curriculum Development Among Pharmacy Graduates (n = 111)

Table 3.

Pharmacy Graduates' Scores on 5 Professional and 8 Generic Competencies Before and After Implementation of an Outcomes-based Approach to Curriculum Development

There was no significant change in computer literacy scale over the years; thus, this item was not included in Figure 3. The reason for lack of change is that The Chinese University of Hong Kong provides every student with an information technology (IT) training course and every student must successfully pass an IT competency test before graduation.

DISCUSSION

The results obtained over the 4 years indicate that the curriculum development process has been effective in improving the development of both professional and generic capabilities. It has taken time for the effect to be manifested. It is the intention of the School to collect this longitudinal data for a few more years to determine if there is a continuous upward trend resulting from further curriculum refinement or a plateau has been reached.

The processes of cyclical refinement based upon feedback, which are inherent in the curriculum development model, are important steps to achieving a curriculum that is consistent with achieving the desired learning outcomes. The results of the graduate survey presented in this paper are only one form of feedback. Other sources of feedback are also important and include the Student Engagement Questionnaire administered by the University, undergraduate student forums organized by the class, external examiner's reports, tutor/teacher's assessments, course coordinator's comments, and course evaluation results.

Another example of our curriculum refinement process is the reorganization of the series of 4 pharmacology and therapeutics courses in 2005 based upon the teachers' and the students' feedback. The individual topics and the time allotted were systematically reviewed and amended. Not surprisingly, the course evaluation results improved in the year following the changes.

The refinement, as described above, is an important aspect of success. However, cooperation from all the course coordinators and teachers, as well as strong direction from the curriculum committee are vital. Another important aspect in achieving the positive results of the present study is the availability of an education expert to serve as a member of the curriculum committee. One of the co-authors is an education expert who has served as a member of the School of Pharmacy Curriculum Committee since the planning of the revised curriculum in 2001.

With the recent change in the secondary school education system in Hong Kong, all the universities in Hong Kong will begin to offer 4-year degrees starting from year 2012 when the secondary school will be reduced from 7 years to 6 years. Accordingly, the BPharm program will be extended to 4 years, with the first year being a foundation year incorporating general education courses. This presents a great opportunity for the School to examine the need for another curriculum revision. Recent evidence points towards a dire need to address the heavy workload perceived by the students and to incorporate some experiential learning in the earlier years of the curriculum.

CONCLUSION

This example of an outcome-based approach in pharmacy curriculum development is from a small pharmacy education program (with about 30 students per class) over a 4-year longitudinal period in Hong Kong. While this may not be the best representative number and length of time, the results of this program do appear to be consistent during the 2-year post-implementation of the outcome-based approach. We believe the results obtained are strong indications of improvement of learning outcomes achieved and can serve as a useful model for curriculum development for other programs. An outcomes-based approach to curriculum development is essential in structuring and achieving desirable learning outcomes of a professional education program. Our example provided strong support of this approach for curriculum development of educational programs.

Appendix 1. Questionnaire for the Annual Graduate Survey