Using Curriculum Mapping to Engage Faculty Members in the Analysis of a Pharmacy Program

Abstract

Objective. To develop a curriculum mapping process that supports continuous analysis and evidence-based decisions in a pharmacy program.

Design. A curriculum map based on the national educational outcomes for pharmacy programs was created using conceptual frameworks grounded in cognitive learning and skill acquisition.

Assessment. The curriculum map was used to align the intended curriculum with the national educational outcomes and licensing examination blueprint. The leveling and sequencing of content showed longitudinal progression of student learning and performance. There was good concordance between the intended and learned curricula as validated by survey responses from employers and graduating students.

Conclusion. The curriculum mapping process was efficient and effective in providing an evidence-based approach to the continuous quality improvement of a pharmacy program.

INTRODUCTION

As professional education programs, pharmacy curricula are subject to comprehensive reviews as required by internal or external (eg, accreditation) processes. Given the specialized knowledge and skills required for pharmacy practice, it is important to demonstrate that the content and structure of a curriculum are appropriate. The essential educational outcomes for pharmacy programs in Canada are outlined by The Association of Faculties of Pharmacy of Canada (AFPC) as: care provider, communicator, collaborator, manager, advocate, scholar, and professional.¹ Each AFPC educational outcome is composed of elements and subelements such as: “1.2 Elicit and complete an assessment of required information to determine the patient’s medication-related and other relevant health needs.” The essential educational outcomes for pharmacy programs in the United States are described by the Center for the Advancement of Pharmacy Education (CAPE).² Educational outcomes guide pharmacy curricula in providing the learning opportunities and setting the expectations that prepare students for entry to practice. This is referred to as the intended curriculum. However, the question remains of how to assess whether students have been given adequate learning opportunities and achieved the appropriate level of expertise. Faculty members at the University of Manitoba developed an effective process for planning and developing a pharmacy curriculum map that can be used to analyze how well an intended curriculum aligns with essential educational outcomes, and how effectively it is delivered or learned by students. The value of curriculum mapping in engaging faculty members in the continuous analysis and evidence-based decisions in a pharmacy program is also demonstrated.

Curriculum mapping is generally used to evaluate an existing or to create a new curriculum.³ It documents what is taught (learning objectives and course content), how it is taught (teaching methods and learning opportunities), when it is taught (timetabling and sequencing), and what is learned (learning outcomes and student assessment). It also demonstrates the connections among the components (eg, courses, discipline streams, years) of a program. Curriculum mapping can be used to demonstrate alignments with essential educational outcomes or accreditation standards.⁴ Specifically, it can identify gaps, redundancies, or inconsistencies in content, learning opportunities, or student assessments.^4-6 It can also guide and support faculty members in making informed decisions regarding the required course or curricular change.⁷ Finally, curriculum mapping can be used to analyze alignments between an intended and “learned” curriculum.^4,8

As described by Harden, curriculum mapping can have numerous functions.⁹ For teaching faculty members, a curriculum map can demonstrate how their course relates to other courses and the overall curriculum. It can assist in course design and decisions related to content and sequencing, teaching methods, and student assessment. For administrators, a curriculum map can inform program planning in areas related to course evaluation, faculty development, teaching assignments, and resource allocation. Finally, for program evaluation, a curriculum map can provide structure for informative inquiry, analysis, and continuous quality improvement.^3,4,9 It can measure how well the intended curriculum aligns with the learned one, and by extension, its impact on student learning and performance, and ultimately, their professional practice.^4,8,10

Curriculum mapping can be a complex process. Effective collaboration is essential, with pharmacy experts providing the content and context and education experts guiding and validating the process. Faculty engagement encourages participation and reduces concerns related to course ownership, teaching evaluations, or potential requirements for changes.³ Curriculum mapping needs to be practical and productive, with data collection that is purposeful and reliable. The end product needs to be informative and functional in a way that maintains faculty interest and ownership of the process.¹¹ Finally, the process requires support from school administration, effective leadership, and adequate resources.¹²

The primary goal of this study was to implement an inclusive and comprehensive curriculum mapping process that supported continuous analysis and evidence-based decisions in a pharmacy program. The key objectives were: to (1) determine to what degree our intended curriculum covered each of the AFPC educational outcomes, and (2) how our intended curriculum aligned with what students actually learned.

DESIGN

Curriculum mapping was initiated by the Curriculum Management Committee at the University of Manitoba’s Faculty of Pharmacy as an ongoing process to facilitate curriculum documentation, analysis, and continuous quality improvement. The school’s 4-year degree program was composed of 36 courses (140 credit hours) categorized into general discipline streams of: pharmaceutical sciences, clinical and applied sciences, and performance-based courses and pharmacy practice (ie, experiential). Key threats to successful curriculum mapping were identified through consultations with local and national experts as: (1) adopting an overly theoretical approach that does not engage and maintain faculty support; (2) collecting inappropriate, inconsistent, or excessive data; and (3) using processes without sufficient focus on results and action.

The Curriculum Management Committee partnered with University Teaching Services (later renamed the Centre for the Advancement of Teaching and Learning), which provided leadership and support for individuals and programs at the University of Manitoba. The 1-year process began with initial faculty discussions followed by a series of seminars on the purpose, components, construction, and design of curriculum maps. Faculty members agreed on an approach based on mapping course objectives to the AFPC educational outcome elements. Faculty development sessions were conducted to review the key components, appropriate language, and effective format for course objectives. Further instruction was provided on linking course objectives to the relevant AFPC educational outcome element(s).

The curriculum map was developed using conceptual frameworks grounded in the theories of cognitive learning and skill acquisition. The former was incorporated using an educational framework called ICE (ideas, connections, and extensions) that defined levels of learning as: ideas (student has basic or fundamental knowledge), connections (student uses basic concepts to make connections or form relationships), extensions (student synthesizes knowledge and uses concepts to extrapolate and make informed decisions).¹³ As described by the developers, “the framework… helps to clarify the characteristics and markers that indicate where learners are along the learning continuum and in doing so, enables teachers to make instructional decisions that maximize learning.”¹³ Teaching faculty members used the ICE framework to assign their expectation of student learning for each AFPC educational outcome element linked to a course objective.

Next, the process of acquiring skills and professional attitudes was captured in a framework on the development of expertise. Early research by Dreyfus and Dreyfus described the acquisition of skill through a series of stages,¹⁴ whereas later work by Ericsson stressed the importance of deliberate practice in the process.¹⁵ Similar to the ICE levels for student learning, we developed levels for student performance based on an expertise framework of: novice (student has cognitive abilities limited to facts or does not connect knowledge with pharmacy practice; student requires extensive coaching or supervision in pharmacy practice or simulation environments), functional (student demonstrates ability to connect knowledge with pharmacy practice; student requires minimal coaching or supervision in pharmacy practice or simulation environments), competent (student demonstrates ability to independently and consistently synthesize and extend knowledge to pharmacy practice; student does not require supervision in pharmacy practice or simulation environments). Teaching faculty members used the NFC (novice, functional, competent) framework to assign their expectation of student performance for each AFPC educational outcome element linked to a course objective. The systematic approach in using conceptual frameworks for cognitive learning and skill acquisition ensured that the learning objectives not only aligned with the curriculum but also with the expectations that teaching faculty members had for students. Importantly, the use of conceptual frameworks framed the acquisition of knowledge, skills, and professional attitudes in an integrated process rather than in the mastery of separate content areas or disciplines.

Teaching faculty members submitted information for their individual courses, while course coordinators collected and collated data for multi-instructor courses. A data collection template (Table 1) was provided to link each course objective to the relevant AFPC educational outcome element(s) and to document the expected levels of student learning (ICE) and performance (NFC). All information was reviewed by the curriculum management committee and an education expert.

Table 1.

Course Objectives Linked to Principal AFPC Educational Outcomes Along With Expected Learning and Performance Levels

Data management and analysis were conducted using Excel, whereas final versions of the curriculum map were available in PowerPoint files that contained hyperlinks to more detailed information. The curriculum map was available in 2 versions: an outcome-centered map that followed the AFPC educational outcomes and a student-centered map that followed progression through each year of the program. Importantly, the curriculum map captured data for every course at the level of each AFPC educational outcome (and element). It provided robust analytical capabilities using accessible software where the weighting, learning levels, and performance levels of each AFPC educational outcome (and element) could be assessed within courses, discipline streams, years, or the overall program. Although electronic and Web-based curriculum management systems were available, we recognized important limitations related to inflexible platforms, specialized operating skills and costs.^11,12 Thus, we made the decision to use Microsoft Office software based on its widespread access,familiarity, and cost effectiveness.

EVALUATION AND ASSESSMENT

Assessing the Intended Curriculum

First, the curriculum map was used to characterize the relative number of course objectives (weightings) linked to each AFPC educational outcome. As summarized in Table 2, the alignment of the curriculum with the AFPC educational outcomes showed the highest weighting of course objectives for care provider (35.0%), followed by scholar (27.0%), and the lowest number for manager (4.3%). Next, the curriculum map and weighting of course objectives were compared with the national licensing examination constructed by the Pharmacy Examining Board of Canada (PEBC). In this case, the weighting of course objectives were compared with the weighting of professional competencies outlined in the examination blueprint (Table 2).¹⁶ The results showed reasonable concordance between the curriculum map and the national licensing examination, with weightings of 35.0% vs 38% for care provider, 27.0% vs 22% for scholar, and 15.3% vs 20% for communicator, respectively.

Table 2.

Alignment of the Intended Curriculum With the Principal AFPC^a Educational Outcomes and Weighting of Professional Competencies in the PEBC^b National Licensing Examination

The curriculum map was also used to characterize the sequencing and leveling of student learning and performance through the program. An analysis of learning levels by program year showed objectives at an introductory level (ideas) in 42% of cases in the first year, with progression to an advanced level (extensions) in 77% of cases in the final year (Figure 1). The most common expected learning level was ideas in year 1 (42.0%), connections in year 2 (78.0%), and extensions in years 3 and 4 (53.0% and 76.5%, respectively). Similar progressions were observed for the expected performance levels as students gained experience and expertise over time (Figure 2). The most common expected performance level was novice in year 1 (98.5%), functional in years 2 and 3 (48.0% and 67.8%, respectively) and competent in the final year (67.8%).

Figure 1.

Expected student learning level based on course objectives by program year

Figure 2.

Expected student performance level based on course objectives by program year

Validating the Learned Curriculum

The learned curriculum was assessed and validated using employer surveys where participants rated their satisfaction with the performance and skills of recent graduates (within 5 years) in areas related to the AFPC educational outcomes. Overall, 71.0% of respondents ranked the school’s pharmacy graduates as very good or excellent, 12.9% as good, and 16.1% as satisfactory. No respondents rated the graduates as unsatisfactory. Employer feedback on specific educational outcomes is detailed in Table 3.

Table 3.

Validation of the Learned Curriculum as Demonstrated by the Results of Employer and Graduating Student Exit Surveys

In addition, graduating student exit surveys and focus groups were conducted where respondents rated their abilities. As seen in Table 3, there was high concordance between the employer ratings and the self-perceptions of graduating students.

DISCUSSION

The curriculum mapping process was an efficient and effective method of deconstructing and analyzing the intended curriculum relative to the national educational outcomes for pharmacy programs and competencies for pharmacy licensure. The map showed an emphasis on care provider and scholar, which supported the focus on pharmacy knowledge and professional practice. The map also demonstrated progression in the leveling and sequencing of student learning and performance through the program. The process identified some important areas for improvement. For example, gaps in educational outcomes related to the role of manager were triangulated with the light weighting of course objectives on the curriculum map and low scores on both the employer and graduating student exit surveys. The issue was investigated further in student focus groups that specified content gaps related to business processes, human resources, and drug plan procedures. This information was shared with the Curriculum Management Committee, which investigated further to determine where and at what level learning objectives should be added to the curriculum.

Our curriculum mapping process resulted in a number of positive outcomes for the school. It reinforced key educational principles and introduced conceptual frameworks that provided a systematic approach and common language for discussing, analyzing, and modifying the curriculum. The inclusive approach resulted in participation from all teaching faculty members in providing the necessary data to create a relevant and reliable curriculum map. Information collected during the process was also valuable for teaching faculty members to make course-specific decisions related to the content, teaching methods, and learning opportunities. For example, a survey of teaching faculty members identified concerns related to teaching critical-thinking skills. A faculty development session was conducted during which we defined critical thinking and outlined the important skills expected of students. These included the ability to define questions, search and appraise relevant information, make informed decisions, and deal with ambiguity. An analysis of the curriculum map revealed that these skills were covered with the exception of managing ambiguity. This observation was validated in focus groups where students described being comfortable conducting literature searches and appraisals but less confident dealing with ambiguous or inconsistent information. As such, teaching faculty members were given guidance on teaching methods for developing critical-thinking and decision-making skills.

The curriculum map was used to facilitate curriculum presentation and analysis in an accreditation review. The curriculum mapping process also instilled an approach of inquiry and scholarship, which led to the creation of a formal program evaluation committee. This extended the mandate to developing a formal process for evaluating performance outcomes of the pharmacy program including admissions, the curriculum, teaching and learning, and graduate performance. This work has fostered close collaborations with committees and administrators of other schools. Finally, the curriculum mapping experience yielded broader institutional benefits. As the educational principles and conceptual frameworks were relevant and transferable, other professional programs adopted our approach to constructing curriculum maps. Our institution also embarked on an interprofessional education and competency mapping process to align with entry-to-practice standards of health care professionals in the local region.

Our experience confirmed some essentials for successful curriculum mapping. Close collaboration of experts from pharmacy and education ensured a process that was relevant and grounded in solid principles. Faculty engagement and development improved the consistency and reliability of data used to construct the curriculum map. Data collection and management was facilitated by using common templates and familiar software programs. Finally, involvement and support from administration reinforced the importance and value of curriculum mapping.

One limitation of our original curriculum map was that it lacked information related to teaching methods and student assessment. Although these data were added to an updated version, the information was only descriptive. As such, we have embarked on a pilot project that uses the curriculum map to determine how well student assessments (eg, examinations) align with the content, level, and weighting of course objectives.

Another limitation was the need for more objective external validation of alignment between the intended and learned curricula. Although the overall performance of graduates on the PEBC licensing examination is very good, it is only a crude indicator of the effectiveness of the learned curriculum. As such, we have engaged in collaborative research with the PEBC to allow more detailed analyses of our graduates’ performance on each professional competency in the licensing examination blueprint.

SUMMARY

A curriculum map embedded in solid educational principles demonstrated how courses were integrated to achieve an intended curriculum. It provided ongoing documentation and analysis as opposed to a “snapshot” description for periodic accreditation or other reviews. It also demonstrated the longitudinal progression and scaffolding of student learning and performance through a program. The curriculum map promoted and developed shared responsibility for the curriculum and its mandate of preparing students for pharmacy practice. This curriculum mapping process was efficient and effective in providing an evidence-based approach to continuous quality improvement of a pharmacy program.