Writing PharmD Program-Level, Ability-Based Outcomes: Key Elements for Success

Abstract

A set of PharmD program curricular outcomes form the foundation of a doctor of pharmacy (PharmD) curriculum and are critical to the development of both the structure/courses of the curriculum and the assessment plan for the program. A goal for developing these outcomes is to craft a set of clear, concise, assessable statements that accurately reflect competencies of the generalist entry-level pharmacist or graduate of the first-professional doctor of pharmacy degree. This article will provide a review of one specific type of outcome, ability-based outcomes, and present a case study of how one college revised their PharmD program-level outcomes. A discussion of key elements for the successful adoption of these outcomes is also presented.

INTRODUCTION

Producing graduates who are capable generalist entry-level pharmacists is a fundamental goal of pharmacy education. Huba and Freed¹ put forth the concept of the backward design of a curriculum to achieve this goal. This approach advocates creating a set of learning outcomes that explicitly define what students should be able to do as a result of completing a program of study, and then designing educational experiences to achieve those outcomes. So we design backwards and deliver forwards.¹ In addition to their role in the design of a curriculum, outcomes can also be used in assessing the adequacy of an existing program.

Zlatic described the role of ability-based outcomes (ABOs) in the context of pharmacy curricula and their assessment.² ABOs are defined as explicit statements describing what students can do as a result of instruction, ie, their abilities. These abilities require the student to integrate and apply knowledge, skills, and attitudes learned in the curriculum to specific pharmacy-related situations. ABOs therefore focus on the results of instructional experiences rather than on discrete knowledge, skills, or attitudes.²

Writing ability-based outcomes for professional pharmacy education requires specifying the abilities of the generalist entry-level pharmacist (ie PharmD graduate). The process must consider the influence of the external environments including both the health care environment and professional and regulatory bodies on the mission of the educational program.³ In fact, regulatory requirements are set forth in the Accreditation Council for Pharmacy Education's (ACPE) Standards 2007: “The curriculum must define the expected outcomes and be developed with attention to sequencing and integration of content and the selection of teaching and learning methods and assessments.”⁴

Well-defined program-level outcomes have a key role in curricular design and assessment as conceptualized and depicted elsewhere.⁵ Doctor of Pharmacy degree program-level ability-based outcomes can be defined as explicit statements describing what students will be able to do as a result of the integration of knowledge, skills, and attitudes gained by completion of the professional curriculum as a whole.² Course-level outcomes are similar except that they focus on what students will be able to do upon completion of a specified PharmD course. Table 1 provides examples of course and PharmD program-level ability-based outcomes. PharmD Program-level outcomes, as influenced by the external environment, can guide the development of educational experiences that constitute the curriculum as well as the assessment methods used to show the extent to which the outcomes are being achieved.

Table 1.

Doctor of Pharmacy Course and Program-Level Ability-based Outcomes

In 2004, the Ohio State University College of Pharmacy completed its self-study for ACPE accreditation. One of the action items from the self-study was to revise the PharmD program-level outcomes. This task was assigned to the College's Curriculum Committee. This paper describes the process used to develop the set of ABOs at this college and presents a set of 8 key points that may be helpful to others revising or writing program outcomes. This manuscript complements the work of Draugalis et al.⁶

WRITING PROGRAM-LEVEL OUTCOMES

The College Curriculum Committee was responsible for revising the PharmD program-level outcomes. The Curriculum Committee membership consisted of a faculty representative from each of the College's 4 divisions, 2 students selected by their peers, the Director of Assessment, and the Associate Dean for Professional Programs. Collectively these committee members embodied expertise in pharmacy practice, research, and education as well as a vast knowledge of institutional history or a knowledge of other programs. The Committee's goal was to define a minimal set of clear, concise, assessable, program-level outcomes that accurately reflect the abilities of students completing the PharmD program at The Ohio State University in preparation for employment or further study. This goal had been defined as part of the College's self-study for accreditation.

The College's PharmD program mission served as an important starting point for this revision process. This mission was articulated in a 1993 document, Report of the Curriculum Committee: Missions, Outcomes, Competencies, Practice Functions and Goals Associated with an Entry-Level PharmD Curriculum, which is still relevant to the current program design. This document also provided the outcomes, competencies, practice functions, and goals that served as the Committee's starting point for constructing the new PharmD program-level outcomes. Several other documents served as source material to help the Committee incorporate the external environment^4,7-9 into the new outcomes document (as discussed above and presented elsewhere).⁵ Several pharmacy specific journal articles were also used to inform the process.^2,3,6

The Curriculum Committee formulated an iterative process for constructing the new outcomes. The chair of the Committee was given responsibility for writing draft outcome statements. The draft statements were then brought to the Committee for discussion. The chair revised the ABOs based on the Committee discussions, and then brought the revised outcomes back to the Committee for further discussion/revision. This iterative process of writing and discussing the outcome statements continued until all members supported the draft.

During the outcomes construction process, an active effort was made to create a set of ABOs that the entire faculty could embrace. To this end, the chair consulted with individual or small groups of Committee members who had expertise in an area when writing outcomes related to that area. If no Committee member had the required expertise, the chair sought input from non-Committee faculty members who did. In addition, free and open discussion was encouraged during Committee meetings. This often resulted in lively debates. Importantly, these debates were conducted in a respectful tone, with each member striving to understand differing points of view.

The Committee occasionally found it necessary to step back from the draft revisions to address various topics that were standing in the way of progress. Three examples particularly stand out. The first was the challenge of reaching a common understanding of what a program-level, ability-based outcome is. In the end, our ABOs were conceptually and practically based on the work of Zlatic² as described above. This resulted in an important “test” that the Committee applied to the draft outcomes. We continually asked whether all 3 parts of the definition of an ABO were met. That is, could we easily see a knowledge component, a skill component, and an attitude component for the drafted ABO. If the answer was no, then the outcome was revised until it was stated in such a fashion as to include all 3 aspects of the definition or discarded if the original was not editable.

Another example of a stumbling block to progress was the lack of a shared understanding of the terminology being used in Committee discussions. Even though we were all pharmacy educators or students, the Committee frequently found basic terms did not have the same meaning to all of its members. We therefore made a conscious effort to define terms we were using or to ask for a definition if the meaning was uncertain. We gradually created a glossary of terms for the Committee's use, and this glossary was included with the ABOs document that was distributed to the faculty for discussion and approval.

A third challenge that the committee faced was in determining the level of detail of the outcomes statements, which was what ultimately led to the total number of ABOs in our final draft. We set out with the intention of minimizing our existing set of 76 outcomes. We discussed the intuitive appeal of a smaller number of outcomes. Ultimately the deciding factor was seen as whether an individual ABO was assessable. As stated in our goal for the process, we wanted to write a “minimal set of clear concise assessable statements.” In order to be able to assess student achievement of individual ABOs, we felt that they needed to express specific abilities rather than broad areas. Thus, we also subjected each ABO to a test of “assessability.” We wanted to reasonably expect that we could generate data to demonstrate student achievement of the ABO. Ultimately, our stated goal of a minimal set of clear, concise, assessable statements actually ended up as 100 ABOs.

Throughout the process of outcomes revision, the Committee (with support from the Dean of the College) updated the rest of the faculty on the revision process using portions of several faculty meetings and a faculty retreat to educate faculty members about ABOs and the assessment of professional education. Additionally, the College's Experiential Advisory Committee, made up of preceptors and faculty members, was involved with the process. This Committee reviewed drafts of the document and provided feedback to the Curriculum Committee. Once the Curriculum Committee approved the new outcomes, they were presented to the entire faculty for discussion and approved unanimously.

The ABOs were implemented immediately as the process of mapping the curriculum to the outcomes unfolded. The outcomes are modified or amended as necessary by the Curriculum Committee. To date these changes have been minor. The full document will be reviewed during the next self-study for accreditation.

After the approval of the new outcomes by the College faculty, a volunteer subgroup of 6 participants of the Curriculum Committee analyzed the 12-month process to determine the key elements for a successful outcome. We defined success for this revision process as the unanimous approval of the document by the faculty. This subgroup noted that a great deal of Committee time and learning was invested in the process and wished to characterize elements or areas that could be reproduced by others undertaking similar revision processes. In addition, we were encouraged by a human resources consultant who was working with the College at the time on our strategic plan to share this work for the benefit of others. He had seen other schools and Colleges on our campus struggle with the process of writing program outcomes. The group met multiple times to reflect on and discuss how we had accomplished the revision process, culminating in approval by the entire faculty.

ABILITY-BASED OUTCOMES

The complete set of faculty-approved outcome statements consisted of 100 outcomes divided into 3 areas: provide population-based and patient-specific pharmaceutical care (76 outcomes); manage and use resources of the health-care system (22 outcomes); and promote health improvement, wellness, and disease prevention (2 outcomes).¹⁰ An excerpt from section 1 of the revised outcomes is presented in Table 2 along with the corresponding outcomes from the original version (2003).

Table 2.

PharmD Program-Level Ability-Based Outcomes Excerpt and Original Outcomes Excerpt

Because these outcomes are directly tied to both Standards 2007⁴ and the CAPE document,⁷ they provide a foundation for assessment purposes. In other words, as we develop methods for demonstrating student achievement of these outcomes, we can easily tie this evidence to Standards 2007. The final document also contained the following sections: an executive summary; a preamble defining ABOs, PharmD program-level outcomes and the generalist, entry-level pharmacist; the College Mission of Pharmacy Practice; the College Mission of Doctor of Pharmacy Education; and a glossary of key terminology used in the document.

DISCUSSION

The subgroup of the Curriculum Committee determined that 8 reproducible key elements contributed to the successful outcomes revision process.

Shared vision of pharmacy practice.

In order to create a set of truly useful ABOs, the leadership at the institution needs to have a vision of not only what pharmacists need to be able to do, but also how well-written ABOs can be used to revise/design a curriculum. The leadership must also take responsibility for creating a curricular assessment process that examines the extent to which the institution is implementing that vision and achieving the stated outcomes.

Resources to support the process.

One or more people within the organization should have or develop expertise in the areas of assessment, curriculum development and revision, and pharmacy practice. Keep the endpoint in mind. The goal of the process of writing a set of program-level outcomes is to describe the abilities of a generalist entry-level practitioner as opposed to the abilities of an experienced practitioner. Additionally, the outcomes document should not attempt to define every conceivable outcome associated with the PharmD curriculum.

Committee composition.

The diversity and experience of the Committee members are key to representing the broad viewpoints and expertise of the faculty. In addition, members who can contribute their knowledge of institutional history also can add depth to the discussion. The Committee chair should manage the knowledge and diversity of the Committee to ensure that differing viewpoints enrich the final product rather than degenerate to turf wars or irreconcilable arguments. Committee consistency is also important; this can be achieved by multiyear or staggered appointments.

Defining terminology.

The areas of assessment and educational literature are rife with jargon and discipline-specific language. Sometimes even educational practitioners disagree on the meaning of their own terminology. Addressing these discrepant meanings and coming to a shared knowledge of terminology is a key factor in developing a working knowledge of important educational concepts, particularly among faculty members who may regard educational processes with suspicion. Members should be reminded that ABOs are the sum of knowledge, skills, and attitudes.

Keep all faculty members informed about the process.

Use routine college/school meetings to deliver brief educational messages and update the non-committee members on the status of the revision process. If all members of the faculty are learning along the way, the approval process should go smoother. Ability-based outcomes are not the same as content. Ability-based outcomes are complex integrations of knowledge, skills, and attitudes.² Taken as a group, ABOs describe what the entry-level practitioner is able to do; they do not explicitly state the content of the PharmD coursework. Therefore, ABOs tend to be threatening to faculty members who are looking for what they teach to be represented directly in the outcome statements. Advanced training of faculty members about the nature and purpose of ABOs is critical to their acceptance of the paradigm shift in their view of the curriculum – from teaching students content to developing their abilities to deliver pharmaceutical care.

Be sensitive to and aware of the issues and concerns of your stakeholders.

Taking the time to understand the needs and concerns of the stakeholders in the educational process at a college/school is important and may ultimately smooth the way for subsequent approval and implementation of the outcomes. An environment within the college that values pharmacy practice and the PharmD program supports efforts to enhance the educational process used to achieve program outcomes related to patient care are very important. This environment should also embrace the contributions of all pharmacy disciplines toward achieving the program outcomes. As mentioned above, educational jargon and the global nature of ABOs can be threatening to faculty members. Educational efforts about ABOs should not be limited to the faculty members: students need to understand how the outcomes will affect them. Moreover, preceptors and employers must see the relevance of the outcomes to their practice settings.

Once ABOs have been drafted and approved, they can be used for curricular mapping. A curricular map links course content to the outcomes using a visual format. This tool can then be used to identify gaps and overlaps in the curriculum. Kelley et al present a strategy for mapping pharmacy curricula.⁵ The outcomes should serve as the basis for assessment plans and should link to assessment tools that determine to what extent students are achieving the outcomes as a result of experiencing the curriculum. Outcomes should also be linked to ACPE Standards 2007 so that programs can show that they are meeting the curricular outcomes requirement of Standard 12.⁴

SUMMARY

The 100 ability-based outcomes created by the Curriculum Committee were important to the continued development and refinement of the PharmD curriculum at The Ohio State University. By reflecting on what was done and how it was done, after approval of the outcomes, a subgroup of faculty members recognized that the process that evolved over the roughly yearlong timeframe was equally, if not more important, than the outcomes themselves. This group developed the 8 reproducible elements that are shared herein for others to consider when undertaking the process of revising or writing program-level outcomes for their curricula.