Scholarly and Best Practices in Assessment

The Accreditation Council for Pharmacy Education (ACPE) 2016 Accreditation Standards advocates for innovative instructional methods and enhanced assessment of learning outcomes.¹ Assessment of outcomes related to interprofessional education (IPE) including teamwork and collaboration, respect and values, and communication, and the affective domains in Standard 4 (including self-awareness, leadership, innovation and entrepreneurship, and professionalism) present a challenge. Even if schools developed an integrated program of assessment as advocated by Fielding and colleagues, adherence to the “best practices” in assessment is needed.² The question is what are best practices?

Fjortoft highlighted the challenges in assessing achievement of the affective domain.³ There is limited empiric data to assess and document learning of the emphasized learning outcomes.³ Lack of empiric data using robust assessments was also noted when evaluating outcomes of health humanities education, which are intended to develop the affective domains of being a health professional.⁴ Shortcomings in how IPE interventions are assessed have also been previously reported.^5-8 Even with the availability of quantitative methods such as the various IPE assessment tools we are challenged with how to select those that are validated for our teaching and learning environment.⁹

Fjortoft highlighted how assessing Standard 4 could be done by using reflections and self-assessment.³ Tsingos suggested a model using reflective writings to assess affective domain outcomes.¹⁰ While qualitative methods, such as reflections, have been used to evaluate IPE outcomes, there are many weaknesses in using these methods. Reeves advocated for more robust evaluations using mixed methods to evaluate IPE outcomes.¹¹ As science scholars, we are more attuned to quantitative research methods and are not as familiar in qualitative methods. While qualitative research is often criticized for lack of rigor, when completed using a standardized approach, it can be used together with quantitative data to add richness and insight. It can also be used to investigate research questions not readily addressed by quantitative methods. Anderson reviewed the strengths and limitations of qualitative research, and provided examples of how qualitative data from an interview, direct observation, and focus groups can be presented in a research article.¹² In addition, Anderson provided guidance for authors and peer reviewers when evaluating qualitative research manuscripts for publication. There are other qualitative methods that could be explored as scholars including the use of narrative analysis.¹³ Personal narratives can be a source of data. However, this method is contrary to methods used in the physical, biological, and social sciences, and lacks validation and reproducibility. The pharmacy profession trained more in the physical and biological sciences will struggle with evaluating the robustness of data generated by qualitative methods as few faculty are well-versed in the methods.

Use of methods that are consistent with rigorous assessments and best practices in assessment should be used to evaluate the new learning outcomes desired in our curriculum. In conducting a literature search, the closest best practices were a series of nine principles of good practice for assessing student learning advocated by the American Association for Higher Education (AAHE) in 1996.¹⁴ This document in combination with the award criteria used to evaluate excellence in programmatic assessment from the American Association of Colleges of Pharmacy (AACP), led us to generate seven best practices.¹⁵ These best practices are modeled after the continuous quality improvement (CQI) process of PDCA: plan, do, check and act (Table 1).¹⁶

Table 1.

Continuous Quality Improvement Process of PDCA

McLaughlin and colleagues discussed how adherence to sound principles of educational research should guide implementation of curricular innovations.¹⁷Adherence to a scholarly approach to assess learning of challenging educational outcomes and the use of rigorous assessments, including best practices in assessment, should result in enhancing the scholarship of teaching and learning. Schools need to approach the assessment of these challenging learning outcomes in a scholarly manner. Banta outlined the characteristics of effective outcomes assessment which should lead to building a culture of scholarship of assessment.¹⁸ Using such an approach will allow schools to make significant progress in demonstrating the value of the various educational endeavors. One may recognize that these best practices adhere to the six principles for assessment of scholarship.¹⁹ These include clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and reflective critique.

At the Southern Illinois University Edwardsville School of Pharmacy, we have used a scholarly approach to develop tools and a process to assess IPE team performance in error disclosure.²⁰ Work done in assessing IPE learning includes using best practices in assessment and following a CQI process. As we implement innovative instructional methods to address the ACPE accreditation standards, we need to move from approaching assessment as a tedious process for mainly checking the boxes for meeting accreditation standards to using a scholarly approach to assessment that illustrates best practices in assessment.