Evaluation, Assessment, and Outcomes in Pharmacy Education: The 2007 AACP Institute

Institutions measure what is important to them.

-Robert Bruininks, President, University of Minnesota

In the summer of 2007, the American Association of Colleges of Pharmacy (AACP) offered its 11th Institute to enhance institutional members' understanding of issues impacting pharmacy education. The theme for the Institute was understanding and responding to the changes in evaluation, assessment, and outcomes for doctor of pharmacy programs described in the Accreditation Council for Pharmacy Education (ACPE) Standards and Guidelines that became effective July 2007.¹ The 2007 AACP Institute focused on: (1) the new ACPE Standards and Guidelines; (2) the implementation of a new ACPE evaluation rubric for the accreditation self-study process for colleges and schools of pharmacy; and (3) how US Department of Education (USDE) policies would impact higher education in general and pharmacy in particular with respect to educational accountability and outcomes.

The focus/theme of the 2007 AACP Institute was in response to a recommendation of the 2003-2004 Academic Affairs Committee report²:

AACP, through its Institute for promoting leadership and continuous improvement of curricular and pedagogical activities and other programs, products, and services, should continue to provide member education and resource materials to support the implementation of program assessment processes by member institutions.

Pharmacy education in the United States has undergone a major transformation over the past decade with the adoption of the doctor of pharmacy (PharmD) degree as the first professional degree. Concurrent with this transformation has been a revision of the Standards and Guidelines for the PharmD degree program designed to assist colleges and schools of pharmacy in developing and maintaining academically strong, effective programs that are responsive to the changing health care needs of society. Furthermore, accreditation standards promulgated by regional accreditation agencies and specialty accreditation organizations (eg, ACPE), as well as by governmental agencies (eg, USDE), have resulted in a demand for greater accountability and transparency within higher education for all stakeholders.

Colleges and schools of pharmacy must employ a variety of valid and reliable assessment measures to collect information about the attainment of desired student learning outcomes (Standard No. 15).¹ Furthermore, such reliable and valid assessments must be employed systematically and sequentially throughout the professional degree program, be of a formative and summative nature, and be employed in both didactic and experiential education within the curriculum. The ACPE Guidelines for Standard 15 suggest that a variety of assessments be used to evaluate student learning in both didactic and experiential education.

College- or school-based teams responsible for planning, implementing, and evaluating their doctor of pharmacy curriculum were encouraged to participate in the 2007 AACP Institute. The format of the Institute allowed for blocks of time devoted to team meetings during which teams were able to prepare a plan specific to their own institution and to continue to work on this plan at their respective campuses. Activities were designed to enable teams to envision their college or school's evaluation, assessment, and outcome activities from a tactical and a strategic perspective. Approximately 280 faculty members and administrators from 66 colleges and schools of pharmacy attended the Institute in 2007.

If the opening message is true, then the opposite may also be true in that organizations may not measure what is not important to them. A pre-Institute assessment survey (administered approximately 1 month prior to the venue) was returned by 62 teams. Seventy-nine percent of respondents reported that their college or school had a formal written assessment plan in place and 63% indicated that their university had an assessment plan as well. The top priorities described for their respective programs were: (1) designing instruments and tools for assessment activities; (2) curricular mapping strategies, and (3) developing and implementing an assessment plan. The mean number of full-time equivalent (FTE) faculty positions dedicated to assessment at an institutional level was 0.65. Approximately 84% of respondents indicated that the college or school of pharmacy had a separate committee to oversee assessment activities. Assessment committees for the most part were comprised of faculty members (63%), administrators (18%), and students (15%), though relatively few indicated inclusion of alumni, preceptors, and staff members. Based on this convenience sample, it does appear that academic pharmacy does place importance on educational outcomes and accountability as fostered through various assessment activities occurring throughout campuses.

Challenges in educational assessment include the use of efficient, valid, and sound methods for data collection and analysis as well as the development of comprehensive assessment plans of which actionable plans are articulated to stakeholders in the educational system. Several case studies from the 2007 AACP Institute were selected for inclusion in this series authored by Boyce, Britton et al, Kelley et al, Piascik and Bird, and Szilagyi. Each case study provides insight into key areas of assessment from creating a culture of assessment, using readily available sources of assessment data, writing ability-based outcomes, and curricular mapping, to the use of curricular progress assessments in pharmacy education. It is hoped that these papers will foster dialogue among faculty members and administrators within academic pharmacy as it relates to educational and programmatic assessment for the continuous improvement of curricular and pedagogical activities in pharmacy education. For those individuals experienced as well as those new to educational assessment, these papers are a rich collection of exemplars, from varied institutions that should serve as excellent models of how academic pharmacy programs approached this vital topic. Furthermore, the importance of sharing resources including financial, intellectual, and technological can not be underestimated in this area of limited expertise yet growing demand.