Assessing the Alignment of Pharmacotherapeutics Course Outcomes With Topic Outcomes

Abstract

Objective. To determine the level of alignment between course and topic outcomes of a third- and fourth-year pharmacotherapy curriculum.

Methods. Each topic taught in each course was categorized into subtopics and each subtopic had a set of learning outcomes. The learning outcomes for the courses and subtopics were matched to examine alignment. Cognitive levels were categorized using the Structure of the Observed Learning Outcome (SOLO) taxonomy of prestructural, unistructural, multistructural, relational, or extended abstract. The types of knowledge that the curriculum was intended to instill also were categorized as declarative or functioning.

Results. There were eight explicit course outcomes for each course and 140 subtopic outcomes for course 1 and 136 for course 2. Wide variations in alignment between the courses and subtopic outcomes were found. Issues identified included the use of redundant verbs and insufficient alignment and balance of cognitive levels and types of knowledge embedded in each outcome.

Conclusion. Alignment between the course and subtopic outcomes was not optimal. Using a constructive alignment analysis, learning outcome issues that need improvement for course alignment were identified. The methodology used here can be applied to other courses to promote effective curriculum design.

INTRODUCTION

Medical and pharmacy schools around the world are focusing increasingly on students attaining educational outcomes required for competent clinical practice.^1-5 One measure of educational outcomes is the constructive alignment of the curriculum, which is commonly defined as alignment between the learning outcomes, teaching and learning activities, and assessment.^6,7 In an aligned curriculum, all parts of the curriculum work together to guide teaching and learning activities to facilitate deep approaches for student learning.^6,8,9 To promote and support high-quality student learning, instructional outcomes and assessments must be aligned.^4-6,8,10,11

Choosing the ideal alignment approach depends on the purpose for conducting the alignment and feasibility of undertaking the task.^4,10 Biggs and Tang⁶ recommend an ordered sequence of alignment, starting with program outcomes and ending with course and topic outcomes. Wittstrom and colleagues⁵ used expert review and Webb’s model of alignment to investigate the alignment of two pharmacotherapy courses. However, these models require item-by-item review by content experts trained to judge alignment, which can be time consuming and pose financial constraints in institutions that are resource poor. FitzPatrick and colleagues also reviewed two therapeutics courses but used student focus groups and a document analysis approach.⁴ They matched course outcomes and assessment for specific therapeutic content and identified the cognitive processes according to the six cognitive levels of the revised Bloom’s taxonomy.¹² One critique of the revised Bloom’s taxonomy is that the understanding level is not clearly defined, having verbs such as identify, discuss, and explain, which overlap one another and represent three of the Structure of the Observed Learning Outcome (SOLO) levels.⁶ Bloom’s taxonomies were not developed specifically with university teaching in mind, nor for the purpose of formulating intended learning outcomes.¹³ Because of this, they are not accompanied by criteria for judging the outcomes of the activity levels.¹⁴ Thus, while Bloom’s taxonomies are well recognized, the SOLO taxonomy is more suited to a constructive alignment analysis.

The SOLO taxonomy describes a hierarchy where each level becomes the foundation on which further learning is built (Table 1). Cognition is categorized into five cognitive levels, according to their increasing level of structural complexity from prestructural, to unistructural, to multistructural, to relational, and finally extended abstract.^6,13 The SOLO taxonomy can be used to define learning outcomes, which describe the expectations of student learning and for evaluating learning outcomes^6,14,15 to determine students’ cognitive level of performance. The verbs used in the outcomes are further categorized according to two types of knowledge: declarative or functioning knowledge that students are expected to acquire. Declarative knowledge is content knowledge expressed in spoken and written word, and its overuse in the curriculum may encourage surface approaches to learning. Functioning knowledge is knowledge that informs action, where the learner’s performance of a task is underpinned by understanding. It stimulates deep approaches to learning, which is described as getting the task done appropriately and meaningfully using the most appropriate cognitive activities.^6,16,17 Professional practice is more concerned with functioning knowledge, as declarative knowledge is used to inform professional decision-making during practice.⁶ In this study, we matched the learning outcomes for the courses and subtopics according to SOLO taxonomy levels and the types of knowledge taught to determine alignment.

Table 1.

SOLO Taxonomy^6,13

METHODS

The two pharmacotherapeutics courses examined, which were delivered in years three and four of a four-year bachelor of pharmacy program, are referred to here as course 1 and course 2, respectively. The courses combined knowledge from the basic sciences disciplines of anatomy, physiology, biochemistry, pathology, and pharmacology, and the core pharmacy courses of pharmaceutics and pharmacy practice. Each course had two hours of lectures and two hours for tutorials, where students were divided into groups not exceeding six students to discuss case studies based on the lectures. The courses were theory based and had no workplace attachment. However, students were expected to apply knowledge and skills gained from these courses in the pharmacy practice course delivered in year 4. Furthermore, after graduating from the program, students were expected to apply the knowledge and skills in a one-year internship program coordinated by the Fiji Ministry of Health and Medical Services. Upon successful completion of the internship program, students could register as pharmacists with the Fiji Pharmacy Profession Board, and practice without supervision.

Course outlines contained the details of the learning outcomes and assessments. Course 1 had 12 broad topic areas, which were broken into 45 subtopics for delivery. Similarly, course 2 had 11 broad topic areas and 46 subtopics. Delivery of each subtopic was guided by a teaching guide that detailed the subtopic outcomes and gave a summary of content that had to be covered during the course. The teaching guide was part of the course syllabus. Each course was delivered in one year over two 14-week semesters. The weekly schedule included two hours of lectures and two hours of tutorials. Assessments for each course included two in-class tests per semester, an oral presentation, an assignment, and two end-of-semester written examinations. The study did not involve participants, so ethics approval was not required.

Course documents including the course outlines and teaching guides were reviewed and assessed using the SOLO taxonomy to determine the constructive alignment between the courses and subtopic outcomes. The cognitive levels and type of knowledge (declarative or functioning) taught were also investigated. The course outlines contained the objectives, while teaching guides contained subtopic objectives. Although the term “course/subtopic objectives” was used in the course outlines and teaching guides, the structure of the objectives was in line with that of intended learning outcomes¹³ as shown in the example of course objective 3: student (subject) discuss (verb) pharmacotherapy and nonpharmacological therapy (direct object).

Therefore, the objectives were interpreted as intended learning outcomes and analyzed using SOLO taxonomy following Biggs and Tang’s⁶ approach.

Constructive alignment was evaluated by matching subtopic outcomes to respective course outcomes. Subtopic outcomes that did not match the course outcomes were considered not aligned. Subtopics that did not have teaching guides were categorized as having no outcomes. To determine the cognitive level, the verbs used in the learning outcomes were matched using the SOLO taxonomy and categorized according to the SOLO levels of prestructural, unistructural, multistructural, relational, or extended abstract. In addition, the average SOLO levels were calculated to match the cognitive level between the courses and subtopic outcomes.

Brabrand and Dahl’s¹³ approach was used to calculate an average SOLO level to assess overall cognitive levels at the course and subtopic levels. Each verb used in the outcome statement was classified according to its level on the SOLO scale and given a score from 1 for prestructural to 5 for extended abstract. These scores were calculated to determine the total score. To determine the average SOLO level, the total score was divided by the total number of outcomes. For the subtopic outcomes that are syntactically structured, the SOLO level was calculated as shown in the example: “Define (SOLO level 1) hemorrhoids and discuss (SOLO level 3) the different types of hemorrhoids.” The average SOLO level for this outcome would be calculated as

Three verbs used in the subtopic outcomes (appreciate, understand, and undertake) categorized as nonoperational competencies¹³ were excluded from the calculation of average SOLO level because these verbs are not considered in the SOLO Taxonomy. In addition, to determine the type of knowledge, verbs were categorized as declarative or functioning according to the SOLO taxonomy.⁶ The nonoperational competencies also were excluded from this analysis.

RESULTS

There were eight course outcomes for each of the courses. The curriculum development committee intended for the course outcomes to be covered upon completion of the 12 topics in course 1 and 11 topics in course 2. Course 1 topics included: introduction to pharmacotherapeutics/clinical pharmacy, drug therapy monitoring, adverse drug reactions, special patient groups, cardiovascular disorders, hematology, respiratory disorders, endocrinology, gastrointestinal disorders, infectious diseases, nephrology, and eye disorders. Course 2 topics included: neurology, mental health, women’s health, men’s health, rheumatology, sports medicine, dermatology, wound management, oncology, medication management review, and applied therapeutics. These broad areas were divided into 45 subtopics in course 1 and 46 subtopics in course 2. Constructive alignment analysis was carried out on each of the subtopics and learning outcome issues that need improvement for course alignment were identified.

Almost half (42%) of the 19 subtopics covered in course 1 and more than a third (37%) of the 17 subtopics covered in course 2 did not have teaching guides and therefore lacked learning outcomes. The subtopics for course 1 that did not have teaching guides were: anemia, drug-induced hematologic disorders, asthma, diabetes, nausea, vomiting and diarrhea, acute and chronic renal failure, and all the subtopics in the infectious diseases topic. The subtopics for course 2 that did not have teaching guides were: drug dependence, attention deficit hyperactive disorders, menstrual disorders, osteoarthritis, rheumatoid arthritis, gout, bone disorders, sports injury and first aid treatment, drugs in sports, pharmacist’s roles in sports medicine, nail infections, drug-induced skin disorders, hair disorders, pigmentation disorders, and selected case studies.

More than half (58%) of the 45 subtopics in course 1 and almost two thirds (63%) of the 46 subtopics in course 2 had learning outcomes, giving a total of 140 and 136 subtopic outcomes, respectively. For course 1, the total outcomes for each subtopic ranged from 1 to 12, with an average of 8 outcomes for each subtopic. For course 2, the total number of outcomes for each topic ranged from 1 to 14, with an average of 5 outcomes for each subtopic. Five and 10 nonoperational competencies were identified in course 1 and course 2 subtopic outcomes, respectively. Alignment of the total number of subtopics to course 1 outcomes varied from 4% to 81% (Figure 1, Course 1). As for course 2, alignment ranged from 0% to 93% (Figure 1, Course 2).

Figure 1.

Alignment of Course and Subtopic Learning Outcomes

The hierarchy of verbs used in the course outcomes of both courses included those within the multistructural and relational levels of the SOLO taxonomy (Table 2). Similar verbs were used in the two courses. Compared to the course outcomes, the subtopic outcomes for both courses were categorized in the unistructural, multistructural and relational levels (Table 2). The calculated average of the SOLO levels for the course outcomes for both courses were at the multistructural level. While the frequencies of verbs used in the subtopic outcomes varied widely, the calculated average of the SOLO level for both courses were at the unistructural level (Table 2). One nonoperational competency for course 2 outcomes and 5 and 10 nonoperational competencies for course 1 and 2 subtopic outcomes were excluded from this analysis.

Table 2.

Level Averages for the Structure of the Observed Learning Outcome (SOLO)^a

The verbs used were distributed within both the declarative and functioning knowledge categories. Three out of the five verbs used in the course 1 outcomes and two out of the eight verbs used in the course 2 outcomes were from the functioning knowledge category. Most of the verbs used in the subtopic outcomes in course 1 (85%) and course 2 (94%) were from the declarative knowledge category (Figure 2).

Figure 2.

Type of Knowledge

DISCUSSION

Findings from this study suggest inadequate alignment between the course and subtopic outcomes of the two pharmacotherapeutics courses. We also observed that the hierarchy of verbs used in the subtopic outcomes did not appropriately match the hierarchies used in the course outcomes. Furthermore, an imbalance was found between declarative and functional knowledge taught in the courses.

In a curriculum, selected topics and subtopics are designed to achieve the course outcomes, therefore the topic and subtopic outcomes should be adequately aligned with the course outcomes. Poorly aligned topics will result in failure to accomplish the course learning outcomes.⁶ Almost half (42%) of the subtopics in course 1 and more than a third (37%) in course 2 lacked subtopic outcomes. This is cause for concern because teaching should be guided by learning outcomes. In light of the increasing prevalence of diabetes, kidney disease, asthma, and anemia in Fiji,¹⁸ these subtopics must be adequately addressed in the curriculum. The subtopics should be categorized according to disease frequency, burden, and pharmacist involvement in medication therapy, similar to categorization used in the American College of Clinical Pharmacy Pharmacotherapy Toolkit.¹⁹ According to this toolkit, students should receive adequate education and training in diabetes, kidney disease, asthma, and anemia to prepare them for clinical practice. Similarly, cardiovascular disorders, which currently have a high disease burden in Fiji, are poorly aligned with course outcomes. For consistency in the delivery of the courses, appropriate teaching guides must be developed that have subtopic outcomes that are aligned with the course outcomes.

Learning outcomes were found for 58% of course 1 subtopics and 63% of course 2 subtopics. In course 1, the total number of outcomes for each subtopic ranged from one to12, with an average of eight. Therefore, both the total (in most cases) and the average subtopic outcomes exceed the recommended number of five or six.⁶ For course 2, the range was from 1 to 14 with an average number of 5 outcomes. While the average is within the recommended 5 outcomes, some subtopics still exceeded the recommended number. This mismatch commonly occurs when course design occurs when the process of deciding on the topics and that for writing the learning outcomes occurs separately. When the number of learning outcomes for each topic exceeds the recommended five or six outcomes, it is very difficult to apply the concepts of constructive alignment.⁶ As an initial step in course alignment, the subtopic outcomes for both courses should be reviewed for an appropriate number of learning outcomes that are adequately structured for optimal course alignment. Writing integrated learning outcomes that address several topics would solve the issue of having more than the recommended number of outcomes.⁶

Alignment of course 1 subtopics to course outcomes varied widely, while course 2 generally showed poor alignment (Figure 1). In both courses, more alignment was observed for the course outcomes on the pathophysiology and pharmacotherapy of diseases (course 1, objectives 2 and 3; course 2, objective 2). These are positive observations, as the courses are designed to prepare students to be clinical experts in drug use, and they need to understand the pathophysiological mechanisms of disease causation to make appropriate decisions regarding disease management. However, fewer subtopics were aligned with course outcomes one, four, five, six, seven, and eight. This may imply to students that these outcomes are less important to their learning and the profession. FitzPatrick and colleagues had similar observations for an outcome which was similar to learning outcome one in both courses in this study.⁴ They decided the outcome was not part of the intent of the course and deleted it. Similar suggestions may be made from this study. However, course outcomes six and seven are core to the intent of the course and promote higher cognitive levels of learning that must be retained. Moreover, the program faculty members should weigh the importance of each of the course outcomes, and proportions of subtopic alignment should be distributed accordingly. This can further inform the teaching and learning activities and development of an assessment blueprint. Course and subtopic outcomes deemed less important should be reviewed for appropriate placement within the curriculum or deleted.⁵

Given that the aim of the program is to “provide professionally trained pharmacists for Fiji and the region,” this analysis has highlighted that medical ethics was not included, and should be considered for incorporation into the courses. Medical ethics is not delivered as a generic course in the program and may be covered only minimally in some courses. Because of continuous developments in medicine and clinical practice, there is international consensus that medical ethics should be an important part of any health professional curriculum including that for the pharmacy profession and such a course should be as academically rigorous as any other subject.^20,21 Because pharmacotherapeutics is a core course that prepares students for clinical pharmacy practice, medical ethics should be integrated into the topics delivered.²¹ Moreover, pharmacotherapy of comorbidity requires the use of two or more different drugs that may have complex drug-drug interactions and drug-patient interactions and cause various side effects.^22,23 The curriculum should enforce ethical decision-making in such cases, taking into account the patient's welfare and preference, weighing the risks and benefits, and in some cases, cost-benefit analysis for the patient. Content should include the principles of bioethics and their application in ethical decision-making in clinical practice.^20,21 This is a very important concept in clinical pharmacy as students should be prepared to make optimal ethical decisions when choosing between treatment options or when deciding on patients’ choices of treatment, or distribution of the minimal resources available, especially drugs in resource poor settings like Fiji and other Pacific island nations.^20-22 The Ministries of Health often encourage the use of generic drugs as recommended by the World Health Organization. Generic drugs are cheaper compared to the branded ones and pharmacists may substitute the prescribed generic form for the branded form. However, if the patient is required to pay for the treatment, decisions to make such substitutions should also include patient’s choice and ability to pay for the treatment.

This analysis found that the alignment of cognitive levels between the course and subtopics was insufficient. Because the course outcomes only fall into the multistructural and relational levels, having subtopic outcomes in the unistructural level (Table 2) is a clear indication of insufficient alignment between the subtopic and course outcomes. Despite this observation, there may be sufficient alignment since the varieties of verbs are widely distributed. If this were true, however, there should be a match in the average SOLO levels for both course and subtopic outcomes. However, the mismatch between the average SOLO level, being multistructural at the course level and unistructural at the subtopic level (Table 2) further confirms insufficient alignment. This also suggests that although the course outcomes are aimed at the multistructural level where a student is expected to understand several relevant concepts, at the subtopic level, understanding may only be achieved at the unistructural level where a student may be able to learn only one relevant concept as a result of only surface approaches to learning being used in the curriculum. Generally, this further suggests that learning is a quantitative accumulation of content knowledge that lacks application to clinical situations.^6,21

More varieties of verbs are used in the subtopic outcomes compared with the course outcomes. Ten different verbs were used in the subtopic outcomes compared with five in the course outcomes (Table 2). This further indicates inadequate alignment between topic and course outcomes. The most commonly used verb was “discuss,” which appeared 72 times. “Discuss” is a knowledge verb and results in attainment of minimal practical skills. Similar critiques can be made for the other common verbs used such as outline, define, and list.

Additionally, redundant verbs^6,24 such as “understand” were used in the subtopic outcomes. Biggs and Tang argue that if students can “understand” something, they can also “identify, discuss or explain” it.⁶ The intended activity of “understand” is not very clear and can cause confusion among students and teachers. Brabrand and Dahl¹³ used the term “nonoperational competencies” to describe “understand” and other similar verbs such as “appreciate” and “undertake” found in the subtopic objectives. These verbs are not captured and addressed by the SOLO taxonomy. However, use of these verbs in the subtopic outcomes suggests that a review of the outcomes for quality improvement in the course alignment and design is justified. Learning outcomes should be very specific and easily understood, clearly stating the performance intended.^6,25 Similarly, the common di-transitive verbs used “define and discuss” seem redundant as when a student can define something she can also discuss it.⁶

In pharmacotherapeutic courses, students are expected to apply their knowledge about drugs to manage certain diseases. To accomplish this, the learning outcomes should be designed to ascertain both the student’s declarative and functional knowledge. The design of the courses encouraged more declarative than functional knowledge as from the eight course outcomes for each course, only two verbs, “apply” and “utilize,” enhance functioning knowledge (Figure 2). There should be a balance between declarative and functioning knowledge.¹⁷ However, because the courses are delivered in the higher program level, increasing emphasis should be on functioning knowledge.⁶ As students enrolled in these courses are expected to apply knowledge in practice, these results suggest the need for incorporating more functioning knowledge verbs in the learning outcomes.

Verbs such as reflect, generate, or solve can be used in the learning outcomes to allow the student to functionalize the theoretical knowledge they have acquired. Appropriate teaching activities such as role playing and problem-based learning²⁶ to complete case studies in groups may provide optimal student learning experiences. For example, role-play activities require students to gather a patient’s drug history relevant to the case (knows), and use this to identify the drug that may have caused the patient’s presenting complaint (knows how) and provide solutions through a management plan (shows how).^24,27 Through the role-playing exercise, students will be required to demonstrate their ability to identify and manage drug-related issues in patients.

The limitation of this study is that it included only two courses within a four-year bachelor of pharmacy program. Therefore, the findings of this study cannot be generalized to the entire curriculum or to other programs. However, through document analysis of course and subtopic outcomes, we identified areas of improvement in the instructional design of the courses. Thus, the methodology might be able to be applied to other courses for quality improvement of the entire curriculum. Similarly, the methodology may be used by other pharmacy educators for course design and quality improvement. The areas of improvement identified in this study have been addressed in the revised curriculum, and comparative analysis can be carried out after implementation of the revised curriculum.

CONCLUSION

This study assessing the alignment of pharmacotherapeutics course outcomes with topic outcomes found there is a lack of consideration of program aims in the instructional design of these courses. Alignment between courses and subtopic outcomes varied widely, which suggests there is room for improvement in course alignment. There was also a mismatch in cognitive levels. Moreover, because the design of the courses focuses more on declarative knowledge, care should be taken to avoid surface approaches to learning. This paper has demonstrated that the methodology of constructive alignment analysis is effective in identifying issues for improvement in curriculum design for two pharmacy courses. The methodology can be applied to other courses to both check and encourage effective course design.