A Curriculum Review and Mapping Process Supported by an Electronic Database System

Abstract

Curriculum mapping and review is now an expected continuous quality improvement initiative of pharmacy professional programs. Effectively implementing and sustaining this expectation can be a challenge to institutions of higher education and requires dedicated faculty members, a systematic approach, creativity, and—perhaps most importantly—demonstrated leadership at all levels of the institution. To address its specific situation and needs, the University of Oklahoma College of Pharmacy implemented a peer review process of ongoing curriculum mapping and evaluation. An electronic Pharmacy Curriculum Management System (PCMS) was developed to support faculty efforts to manage curricular data, monitor program outcomes, and improve communications to its stakeholders on 2 campuses and across the state.

INTRODUCTION

In response to society's expectations for greater accountability in higher education, academic pharmacy has been challenged over the past decade to define efficient and effective management strategies to support efforts to optimize outcomes of the professional curriculum. Curriculum mapping and review with subsequent responsive modifications are expected continuous quality improvement initiatives of all academic institutions as outlined in the accreditation standards and guidelines for professional programs of the Accreditation Council for Pharmacy Education (ACPE).¹ With the rise of educational technologies and communities of pharmacy educators and students separated by distances, efforts to communicate the curriculum and the expectations of the professional program to all the key stakeholders can also be challenging.

This paper shares the experiences of the University of Oklahoma College of Pharmacy with curriculum review and the methods of communication it developed to facilitate more effective professional education in the current academic environment. Specifically, this paper reviews the historical context serving as the institution's stimulus for curriculum mapping and review, provides an overview of the peer review processes used to accomplish this work, and describes the electronic database system (Pharmacy Curriculum Management System) developed as a support and communication tool. Finally, the results of this work and what we learned about the process are presented.

Historical Context

The College initiated its first professional degree doctor of pharmacy program in fall 1998 while completing the 3-year baccalaureate and 2-year postbaccalaureate doctor of pharmacy degree programs for students already enrolled. In fall 1999, the college offered these 3 programs plus a transition doctor of pharmacy program for students completing the first 2 years of the baccalaureate program, and an alternate pathway doctor of pharmacy degree in partnership with Southwestern Oklahoma State University School of Pharmacy for pharmacists in the state and region desiring this additional professional education. In addition to managing these academic programs, the College expanded its professional program to the University of Oklahoma Schusterman Center in Tulsa in fall 2002 using distance education technologies, a substantial change. The concurrent delivery of several academic programs under differing circumstances made it more difficult for college faculty members to understand and come to grips with the new professional program and its outcomes.

To face the usual challenges associated with the implementation and evaluation of a new professional degree program and in preparation for its next accreditation site visit in November 2005, the College needed broader and improved understanding of the curriculum and improved communication at all levels. ACPE standards effective in 1997 expected professional programs to demonstrate that the new degree outcomes were achieved using a variety of delivery and assessment methods of the recommended curricular content.² Specific additional challenges to our college included: (1) a doubling of student enrollment from approximately 70 students per class on 1 campus to 140 students on 2 campuses; (2) increased number of faculty members, many of whom were younger and unfamiliar with academic processes; (3) lack of faculty understanding or agreement about reasonable expectations for outcomes of a first-professional degree PharmD program; and (4) unfamiliarity with how best to use distance education technologies. Adding to these challenges was a traditional academic culture of individual or departmental ownership of courses as opposed to college ownership of the curriculum. The College needed to find ways to facilitate an environment of openness, mutual understanding, and support required to achieve the new professional program outcomes based on the CAPE standards.³ The College's first step was a thorough review of each course in the curriculum to increase common understanding of the curriculum and to facilitate the development of our first comprehensive curriculum map.

PLANNING FOR CURRICULAR REVIEW

Reviewing and mapping a professional curriculum is a large undertaking. Some institutions do this work solely to be in compliance with accreditation standards; however, this external motivation generally only promotes episodic data collection on how classes are taught and often assumes that students have learned because faculty members have taught the content.⁴ However, because of the program changes occurring at our institution, the faculty and administration had a natural curiosity about how effective our curriculum could be in graduating pharmacists prepared to enter practice. Initial curriculum review efforts were designed to meet 2 primary objectives: (1) making the curriculum transparent for all stakeholders, and (2) linking elements of the curriculum. This information was needed to support more informed and sophisticated discussions about curriculum effectiveness and possible changes in curriculum content and program assessment methods.

To meet these 2 objectives and to make the implicit curriculum explicit and transparent to all stakeholders including students, faculty members, and practitioners in the field, faculty members were asked to define more specifically the learning outcomes in their courses. This request challenged faculty members' more common approach of focusing on course content rather than on what students should be able to do upon successful completion of a course. Faculty members were also asked to define how their coursework contributed to student performance of the professional program's final outcomes, based on those provided by CAPE and unanimously supported by the faculty. Faculty members were asked to present evidence of what students actually learned in their courses rather than assumptions of what was learned based on content delivered. This transparency was needed to identify gaps or unnecessary redundancies in course content, as well as to increase communication among faculty members and other stakeholders. This information served faculty members' efforts to link elements of the curriculum together within a course, a semester, a professional year, and the entire program.

To address these objectives of curriculum transparency and linkage, individual course reviews were explored through the lens of 4 main questions:

• (1) What is taught? Is course content current, relevant, and taught at the level allowing students to achieve expected proficiencies with program outcomes?

• (2) How is it taught? What teaching methods are used? What is the balance between acquiring and applying knowledge? What learning resources and opportunities are available? What is the level of integration into the curriculum? Is the syllabus comprehensive? Is student workload appropriate?

• (3) When is it taught? Are course prerequisites appropriate? Is the course offered in the best semester or professional year?

• (4) What measures are used to determine if students achieved desired learning outcomes? How are students assessed? Are these assessments aligned with course objectives and program outcomes?

After defining the primary objectives of the initial curriculum review, the college curriculum committee defined 4 steps for the curriculum review process: (1) defining participant roles in the course review process, (2) determining the sequence for course review, (3) designing a standardized data collection instrument for course reviews, and (4) reporting and documenting course reviews.

The curriculum committee decided to gain 2 reviews of each course, 1 completed by the course coordinator and 1 by a peer review team. The curriculum committee, with input and advice from department chairs and the academic deans, constructed the peer-review teams. Teams were generally composed of 5 members including approximately 3 full-time faculty members from either campus who had expertise in course content, 1 of whom was a member of the curriculum committee if possible. One of the faculty members on the team was named by the curriculum committee as chair. The remaining 2 team members were adjunct faculty members—usually preceptors with content-related expertise—to provide a practitioner's view of course relevance and preparation for contemporary practice. Student input was gained by each team with some conducting focus groups with randomly selected or volunteer students from both campuses either live or via video conferencing. Course coordinators provided the curriculum committee with all their course materials including syllabi, course and lecture objectives, lecture handouts, PowerPoint slides, course group and individual activities, quizzes, and examinations. The curriculum committee's administrative support gathered and distributed all of these materials to the peer-review teams. All peer-review teams also had access to the current and historical student evaluations of the course.

The curriculum committee decided to begin the review process with courses in the first semester of the first-professional year and proceeded to review the curriculum in order of its delivery to students. This plan allowed the curriculum committee to evaluate logically the construction and assessment of the curriculum including how effectively the curriculum incorporated and integrated prerequisite knowledge and skills into each subsequent professional course.

The curriculum committee developed a standardized data collection instrument to assess 8 areas of each course. Table 1 shows the general areas of questioning pursued by each team. The CAPE outcomes were reduced to 15 broad statements to facilitate the mapping of the content of each course. To ensure more meaningful results and consistency of data collection, all course coordinators and peer review teams were expected to organize their reviews to address all questions in these 8 areas.

Table 1.

Questions Included on a Standardized Data Collection Instrument

Using the data collection instrument, course coordinators and peer review teams evaluated the course materials and generated a report of findings and recommendations, which was then presented in writing and verbally to the curriculum committee. During these presentations, the peer-review team chairs were given 30 minutes to present their reports and respond to questions from the curriculum committee. These reviewers were also asked to provide feedback about the course review process and data collection instrument. All course reports for the semester under review were presented during a special half-day session coordinated by the curriculum committee.

Following this reporting session, the curriculum committee met for an additional half day to finalize a report for the academic dean outlining commendations and recommendations for each course and summatively for the semester under review. This report, using a standardized format, included 2 major sections. The first section included an evaluation of each course and included specific comments in the 5 areas of course organization, content, delivery, assessment, and other findings including course efforts to support professionalization of students. The second section included an evaluation and mapping of courses within the reviewed semester and offered comments about course integration in the curriculum, sequencing of courses, adequacy of prerequisite coursework, course effectiveness in holding students accountable for prior knowledge and skills, assessment methods, and collective professionalization efforts during the semester. The entire process for each semester's courses, including defining teams, completing course evaluations, and preparing and delivering reports, was accomplished over a 3-month period. This time interval allowed team members discretion on how and when to integrate this task into their schedules, as well as to define times when the majority of team members could meet. Most groups met a minimum of 2 times: once to define tasks and divide work, and a second time to review the results and consolidate them into a report.

The academic dean presented the findings of these reports to the dean's advisory group, which consisted of deans, chairs, program directors, students, and alumni practitioners. After this presentation and discussion with these stakeholders and administrators, the academic dean discussed recommendations for change with course coordinators as needed. During these discussions, the recommendations for course modifications were prioritized and course changes requiring additional support were identified and coordinated with the department chairs. At the conclusion of the next offering of the course, course coordinators who had been asked to make changes provided updates to the curriculum committee and the academic dean.

RATIONALE FOR AN ELECTRONIC CURRICULUM MANAGEMENT SYSTEM

The first iteration of this curriculum review process was completed using a combination of electronic or printed files and photocopies of course materials, course coordinator and peer review team reports, and the report of the curriculum committee; all these materials were compiled into voluminous notebooks. At the completion of this process for the first semester of the curriculum, the limitations of a static, essentially paper-based system were apparent; therefore, the college identified the need to make curriculum materials and reports readily accessible to and searchable by faculty members on both campuses to facilitate and support a dynamic and sustainable curriculum mapping and peer review process. Because no adequate system was found in the marketplace, the college invested in the development of an electronic system that would provide a mechanism for updating and storing course materials, viewing course reviews, and facilitating access to these materials by all stakeholders in all locations. The resulting Pharmacy Curriculum Management System (PCMS) supports curriculum mapping and review and communication with all faculty members about course content and skill development, and when the system is completed, it will also track student performance in program outcomes.

The PCMS is a Web-based relational database management system that stores and displays the content of the College's curriculum and peer reviews of various courses. The PCMS was developed primarily using Microsoft technologies (eg, active server pages, Internet information services, and SQL server) over a secured socket layer. The 3 main components of the PCMS are the curriculum module, administration module, and the student performance module, which is now under development. To prevent unauthorized access and security breaches, the PCMS is password-protected with users gaining system access, as authorized, to the various modules through secured password authentication via their employee login accounts. The system also includes various tools to support its functions. A searching and browsing tool allows users to find desired curricular content by keywords, phrases, course names, learning objectives, and program outcomes. One can also locate a course by semester when taught as well as by course coordinator or instructors. Another tool allows the assigned system administrator to update course materials at the end of each semester.

The PCMS curriculum module contains course materials and displays several reports, including course content overview reports, course peer review reports, curriculum committee recommendations, and course update reports. The information in the course content overview report is compiled on a yearly basis by course coordinators and includes a detailed outline of topics covered in the course as well as the syllabus, lecture notes, handouts, slides, examinations, and quizzes. The system allows course coordinators to modify material from a previous year when significant changes have not occurred with included data. In addition, student evaluations of the course are included for review by those authorized. The peer review report is created by the assigned course peer-review team using data stored in the system. The final report is filed into the system by the chairman of this team and may be viewed by the curriculum committee. The final recommendations of the curriculum committee for each course reviews and the annual course update reports reflecting course changes prioritized by the academic dean are included for viewing. Only the course coordinators and course faculty members can access the course content overview reports, peer evaluation reports, curriculum committee recommendations, course update reports, and student course evaluation reports. All college faculty members may access all course overview reports and search all course content included in the PCMS. Currently, the college has reviewed all didactic courses, and almost all course content from fall 2002 to spring 2008 is included in the PCMS.

The PCMS administrative module is primarily used by the PCMS administrator to perform the managerial tasks of setting up courses for updates by the course coordinator, uploading student course evaluations, adding curriculum committee recommendations, accessing any system information requested by the curriculum committee or faculty members, and monitoring logs of system use.

The PCMS student performance module, currently under development, is being designed to provide an evaluation of student performances from matriculation to graduation. Data elements to be included in this module include Pharmacy College Admission Test PCAT scores, grades in prepharmacy coursework, interview scores, grades in professional coursework and experiential assignments, and scores assessing student performance on program outcomes using standardized performance-based and knowledge-based examinations developed by the faculty. Student portfolios, including reflective writing assignments, will also be accessible in the system. The college intends to use this data to analyze student performance, to develop predictive tools based on performance in earlier coursework and on standardized tests, and to provide a more objective basis for change in curriculum content and assessment.

RESULTS AND LESSONS FROM THE MAPPING PROCESS

The College's work in curricular mapping helped faculty members identify 3 main areas for improvement. First, the process successfully identified those professional courses that needed content revision and renewed alignment with program outcomes. Second, the mapping process identified and evaluated “curricular streams,” sequences of courses that were related in terms of content and accountable to each other to build knowledge and skills. It became apparent that some courses in these “streams” required re-sequencing in the curriculum in order to build students' knowledge and skills more intentionally and effectively. For example, the clinical communications course and laboratory was moved from the fall semester of the third year to the fall semester of the second year and more introductory communication methods have been added to the first-year pharmacy practice courses to support professional development in introductory pharmacy practice experiences. The faculty members also moved the drug information course from the fall semester of the second year to the spring semester of the first year to facilitate students' abilities to retrieve and communicate appropriate drug information to patients and health care providers. Third, faculty members learned that assessment methods in the College needed continual attention if the professional program was to be more successful in developing the competencies that faculty expected to observe in students in the final year of the curriculum. For example, the faculty members identified the need to introduce basic practice skills (eg, clinical communication and drug information retrieval) and knowledge to students earlier in the curriculum and then hold them accountable for this foundation knowledge and these skills (eg, drug product information and pharmaceutical calculations) by conducting subsequent and repetitive assessments in other courses later in the curriculum.

There are several administrative challenges that must be addressed to assure an effective curriculum review and mapping process. First, there must be administrative support at all levels to initiate and sustain the process. There is naturally inertia among faculty members to undertake this process; appropriate leadership is needed to help all parties move beyond personal self-interests and insecurities common in any institution of higher education when matters of curriculum are raised. The administration of the institution must be committed to providing personnel support to facilitate faculty efforts. Ongoing educational efforts are also required to keep faculty members up to date on changes and improvements in the electronic system and to inform them of how best to use the stored data to enhance and refine their courses by reviewing and evaluating the information that may be gained about courses prerequisite and subsequent to the courses they teach. Finally, the leadership of the educational institution and the curriculum committee must continually communicate the spirit of the process, which always seeks to meet the needs of the profession rather than individuals. And even if all of these administrative issues are continually addressed, some tension will still remain during the process – this is the nature of academic dialogue.

A curricular mapping process is a large undertaking requiring the involvement of many faculty members. This work was generally perceived as an additional burden by faculty members but one that was accepted as necessary to gain an understanding of the curriculum and to facilitate efforts to improve the professional program. Dispersing the beneficial mapping process into many different teams of faculty members and practitioners rather than limiting the reviews to only a few people facilitated the goal of making the curriculum more transparent to the stakeholders. Subsequent reviews and studies of curriculum components are now much easier to organize and conduct, and with the added benefit that the work may be viewed by all faculty members for a variety of purposes to support their educational efforts.

CONCLUSION

An ongoing curriculum evaluation and mapping process involving peer review is a key component in developing and sustaining an effective professional program. If conducted in a positive and constructive way, it can alter the culture in an academic institution. By promoting the spirit of inquiry, which is the essence of academic institutions, this process allows for an open, objective dialogue about the educational program. The process increases communication and collaborative efforts regarding instructional strategies, course content, assessment methods, and expected program outcomes among faculty members and other stakeholders. Finally, this process ensures that the curriculum reflects the goals not only of the academic institution but also of the profession, making the endpoints of the professional program visible to all involved.