Analysis of Expert Feedback to Determine Intellectual Standards for Student Pharmacist Clinical Reasoning Development

Abstract

Objectives. Clinical reasoning (CR) is one of the most important skills for pharmacy learners. Feedback has been proposed as a pedagogy to improve CR skills; however, essential components of CR have yet to be determined within pharmacy education. This study sought to streamline feedback to align with the Pharmacists’ Patient Care Process (PPCP).

Methods. The investigators used deidentified clinical reasoning “Keep,” “Start,” or “Stop” (KSS) feedback comments from student-written CR “think-aloud” sessions with pharmacy students in their third professional year. Sections were mapped to the PPCP and were coded by 2 independent investigators according to proposed essential components of CR, using an adapted grounded-theory approach. Investigators could inductively add codes after conferring with the other. Coded feedback was analyzed using a summative content approach. Intercoder reliability was calculated via Holsti index.

Results. Five essential components of CR were identified after analysis of 635 KSS comments. The 5 essential components of CR were coded 1178 times. “Accurate,” “Concise,” “Specific,” and “Thorough” were identified a priori, while “Connected” was discovered during feedback comment review. Literature analysis added supporting data to these results through the Paul-Elder Critical Thinking Framework. To maintain consistency in language, these essential components will be referred to as “intellectual standards” moving forward.

Conclusion. This novel study successfully identified 5 key intellectual standards of CR. These intellectual standards provide a framework for pharmacy educators to focus feedback to improve student CR. Future research of other intellectual standards pertinent to experiential education is imperative.

INTRODUCTION

Clinical reasoning (CR) is said to be one of the most important attributes a healthcare professional can possess, and has been likened to the scientific method for clinical practitioners.^1,2 It is a highly complex cognitive and noncognitive skill set that shapes how healthcare professionals interact with patients as well as with the environment.^1,3 Pharmacists use this skill set in nearly every setting, from community care to ambulatory care to acute management of hospitalized patients. It is through the CR process that pharmacists collect and interpret patient data and medication data, weigh the benefits and risks of actions, and understand patient preferences to optimize pharmacotherapy treatment plans.³

The terms “critical thinking,” “clinical reasoning,” and “clinical decision-making” are interrelated concepts that are often used interchangeably. However, they are not one and the same, and the subtle differences among them are important for pharmacy educators to understand. Critical thinking, as defined by the National Council for Excellence in Critical Thinking, is “the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action…that transcend[s] subject matter divisions.”⁴ CR comprises the cognitive and metacognitive processes used for analyzing knowledge relative to a clinical situation or specific patient.⁵ From the perspective of a pharmacist, CR informs recommendations through data interpretation and analysis; and evaluation of evidence relevant to a specific patient, patient population, or medication therapy problem. Clinical decision-making is an intricate, multifaceted skill that combines evidence-based knowledge and intuition and contextual factors.⁵ Critical thinking is the key component of clinical decision-making; it involves some skills and attitudes necessary for the development of CR and is used by pharmacists in the process of identifying and managing medication therapy problems.⁶ All 3 of these concepts are important for competent pharmacy practice, as the Accreditation Council for Pharmacy Education (ACPE) “Standards 2016” Specific Key Element 25.7 highlights the importance of developing and assessing CR skills within pharmacy education. This element states, “Evidence-based clinical reasoning skills, the ability to apply these skills across the patient’s lifespan, and the retention of knowledge that underpins these skills, are regularly assessed throughout the curriculum.”⁷ The main aim of developing critical thinking and CR in pharmacy education is to improve independent clinical decision-making ability. Additionally, critical thinking and CR are the skills underpinning the Pharmacists’ Patient Care Process (PPCP) that inform clinical decision-making; they are essential for effective communication and documentation of the pharmacist’s contribution to the healthcare team.⁸ However, best practices for teaching and evaluating these essential skills have yet to be determined.

One proposed method for teaching and evaluating critical thinking and CR skills is through the provision of feedback to learners. Feedback is an important part of clinical training programs for healthcare professionals and is a key element and Advanced Pharmacy Practice Experience curriculum assessment expectation.^7,9-11 Thus, feedback is frequently provided to pharmacy learners from a multitude of sources to advance a learner’s knowledge and skills closer to that of practicing pharmacists.¹⁰ Specifically, feedback regarding patient workup, where CR skills are essential, is among the most common feedback pharmacy learners will receive.¹² Feedback has many potential benefits and is essential for the correction of student knowledge, reinforcement of correct learning, and documentation of progression toward student goals.¹² In order for feedback to be effective, it must be directly related to the content knowledge, learning, or goals.¹³ However, essential components of CR have not been adequately defined in pharmacy education literature. Therefore, the aim of this study was to identify possible essential components of CR contextualized in pharmacy practice to help learners to achieve the desired PPCP outcomes through the provision of CR-focused feedback given by pharmacy educators.

METHODS

This study utilized an adapted grounded theory research methodology to identify essential components of CR needed for successful execution of the PPCP. The adapted grounded theory methodology comprised 4 components: purposive sampling, data collection, coding, and literature analysis.¹⁴ This study was deemed exempt by the institutional review board at the University of North Carolina.

The purposive sample comprised written CR think-aloud sessions (wCRTA) regarding patient cases. These cases encompassed various medication therapy problems and medical conditions submitted within an integrated pharmacotherapy (iPHTH) 3-course sequence. The wCRTA consists of 6 sections aligned with the PPCP⁸: findings, assessment and goals of therapy, recommendations and monitoring, medication education, rationale, and references (Table 1). As a primary pedagogical approach to teach clinical reasoning, faculty content experts provided written feedback on student wCRTAs to make expert thinking visible.¹⁵ In addition to section-specific feedback, faculty also provided global “Keep, Start, Stop” (KSS) feedback. This approach is an easy method for students to categorize and look for patterns. The purpose of KSS feedback is to provide focused and actionable next steps for students to apply to the following wCRTA submission.¹⁶ Feedback providers were not specifically trained how to focus their feedback for the purposes of this study to reduce the risk of confirmation bias. They had been previously trained to provide comments on student clinical reasoning based on how they would approach patients in practice as expert clinicians in their respective fields. To further minimize the risk of bias, neither section-specific feedback nor KSS feedback was shared with subsequent feedback providers. KSS feedback was shared with small-group facilitators who coordinated with the students to develop plans for addressing future wCRTA feedback; however, no facilitator also served as a feedback provider.

Table 1.

Alignment of Written CR Think-Alouds (wCRTA) With Phases of the Pharmacists’ Patient Care Process (PPCP)

Given the more focused and higher emphasis placed on KSS feedback, deidentified KSS feedback comments from all (n=5) wCRTAs submitted by third-year professional students enrolled in iPHTH in Fall 2019 were collected for analysis. Feedback comments were first uploaded to Atlas.ti. version 8.4.4 (Atlas.ti. Scientific Development GmbH, Berlin, Germany). Each feedback comment was reduced to a unique quotation and categorized according to “Keep,” “Start,” or “Stop.” Quotations were then coded according to a priori–identified essential components of clinical reasoning identified by 2 independent investigators, including “Accurate” (correct information pertinent to the drug therapy or medical condition), “Concise” (succinct, free from extraneous information), “Specific” (details are unique/pertinent to a particular drug, medical condition, or patient) and “Thorough” (includes all information that is necessary in general, ie, not necessarily related to any particular drug or medical condition). These were developed from the collective clinical experience of the investigators. Investigators were able to add or split quotations (eg, split a single “Keep” comment into 2 quotations where one sentence related to “Specific” while the following sentence related to “Concise”) and insert additional codes during data analysis when appropriate. When an additional code was identified, investigators met to discuss the new code and agree upon a code definition for future coding. Coded data were then synthesized for thematic analysis using a summative content approach to identify essential components of clinical reasoning; intercoder reliability was evaluated on all feedback comments for each respective essential component within Atlas.ti using the Holsti Index. Intercoder reliability <60% was considered inadequate for this study, while 60-79% reliability, 80-90% reliability, and >90% reliability were deemed moderate, strong, and almost perfect, respectively.¹⁷ A literature analysis was then conducted for existing critical thinking or CR frameworks for further data analysis.

RESULTS

A total of 635 KSS feedback comments from 155 students within 27 small groups were included for analysis. A breakdown of each proposed essential component is described in Table 2. Importantly, an additional essential component, “Connected” (interrelated wCRTA/PPCP sections, drug therapy, and/or comorbid medical conditions), emerged through inductive coding. Figure 1 highlights the application of this clinical reasoning framework to pharmacy education and practice. A total of 560/635 (88.2%) of KSS feedback comments were coded to essential components of clinical reasoning. Each essential component was coded with moderate to strong inter-coder reliability.¹⁷ (Table 1) “Thorough” was the most coded essential component (32.9%); however, students had most room to improve on “Specific” and “Connected” based on the percentage of “Start” or “Stop” comments (78.5% and 78.3%, respectively). (Table 1) Students performed similarly on “Accurate” and “Thorough” with 33.6% and 31.1% “Keep” comments, respectively. “Concise” had the highest percentage of “Keep” comments (53.2%), but only accounted for 9.8% of all coded essential components.

Table 2.

Summary of Keep, Start, Stop Feedback Comments and Interrater Reliability of Coded Intellectual Standards

Figure 1.

The phases of the Pharmacists’ Patient Care Process (PPCP) are how pharmacists approach caring for patients. For pharmacists to successfully execute the PPCP, they must utilize appropriate clinical reasoning skills, which include appropriate application of the essential components of clinical reasoning (CR). As a result of the application of essential components of CR to the PPCP will help pharmacists realize the vision outlined by Joint Commission of Pharmacy Practitioners.

The majority of KSS comments related to more than 1 essential component of CR as the 5 essential components were coded a total of 1178 times among the 635 KSS comments. “Specific” and “Thorough” were more-often coded together than with other essential components, followed by “Connected” and “Thorough.” “Accurate” and “Connected” were least commonly coded together, and “Accurate” was more often coded as a singular code while “Connected” was most often coded with other essential components within a single comment.

Table 3 shows the frequency at which each essential component of clinical reasoning was coded to each wCRTA section and corresponding PPCP phase. “Accurate” was most commonly commented on with respect to Recommendations and monitoring and Rationale sections. “Connected” was co-coded to two or more wCRTA sections, revealing linkages from Findings to Assessment and goals of therapy, which help inform student recommendations and monitoring, with goals of therapy especially informing monitoring parameters. The Rationale section explains these “Connected” relationships. “Specific” and “Thorough” feedback was also prevalent in each section, most notably in Findings, Recommendations and monitoring, and Rationale. Similarly, “Concise” comments were most frequent within Findings.

Table 3.

Intellectual Standards Coded According to Written Clinical Reasoning Think-Aloud Section*

Literature analysis of critical thinking or clinical reasoning frameworks revealed similar components to develop critical thinking skills, including a well-accepted critical thinking framework: the Paul-Elder Critical Thinking Framework.^18,19 The framework has three components to helps students improve their reasoning, including elements of thought, intellectual standards, and intellectual traits.¹⁹ Specifically, as related to this study, the seven of the nine intellectual standards of the Paul-Elder Critical Thinking Framework were associated with coded essential components of clinical reasoning. Table 4 shows the Paul-Elder Framework intellectual standards of critical thinking mapped to the coded essential components of clinical reasoning within this study.

Table 4.

Mapping of Identified Essential Components of Clinical Reasoning to Paul-Elder Intellectual Standards of Critical Thinking

DISCUSSION

To our knowledge, this is the first study that attempted to identify essential components of clinical reasoning from which pharmacy educators could target their feedback on clinical reasoning in the context of the PPCP. The identification of such essential components of clinical reasoning serves as a key initial step to facilitate the development of student clinical reasoning skills. The high degree of feedback comments coded to the 4 proposed a priori (independent from any framework) essential components of clinical reasoning and fifth inductively determined essential component of clinical reasoning, in addition to these data aligning with a well-accepted framework for critical thinking, indicate a good starting place for pharmacy educators to focus their feedback with the aim of improving student clinical reasoning skills. Because of the close alignment of the Paul-Elder Critical Thinking Framework intellectual standards with the resulting essential components of this study and importance of maintaining consistency within educational language, moving forward the identified essential components of clinical reasoning will be referred to as intellectual standards of clinical reasoning.¹⁹

Consistently utilizing these identified intellectual standards of clinical reasoning can provide pharmacy educators with a standardized approach to ensure the evolution of critical thinking, clinical reasoning, and ultimately, sound clinical decision-making. Since critical thinking is a facilitating component to developing future pharmacists’ clinical reasoning skills, this study sought to identify a framework that could be applied to the context of clinical reasoning necessary within the PPCP. Based upon the results of this study, seven of the nine Paul-Elder Critical Thinking Framework intellectual standards for critical thinking were aligned to coded clinical reasoning feedback.¹⁹ There may be additional intellectual standards of clinical reasoning, however, that could not be captured in this one semester, single course data but deserves further identification and validation. Other possible intellectual standards which may have a context within clinical reasoning include “Significance” (problem prioritization), “Fair” (consideration of context, minimization of distortions), or “Feasible” (ability of the recommended intervention to succeed).¹⁹

It is likely that some of these uncoded intellectual standards are more applicable earlier in the curriculum or during experiential education, as this study took place during the fifth semester of the students’ curriculum and learners were not in a direct patient care setting. This highlights the significance of applying this model throughout the spectrum of educational development of critical thinking, clinical reasoning, and clinical decision-making in pharmacy education. While considering the application of this model across a pharmacy curriculum, it is important to consider the sequence of introducing these intellectual standards of clinical reasoning. Future research should evaluate the unbundling of these intellectual standards of clinical reasoning throughout pharmacy education.

There are notable limitations to the present study. First, section specific feedback was not assessed in this study. Global KSS feedback was chosen to be analyzed to focus on feedback of key issues deemed most important to feedback providers for improving student clinical reasoning skills. Additionally, these results may not be generalizable to all levels of pharmacy education as this study only assessed students who were in the third professional year of their curriculum as stated previously. Further research into the best sequence by which to introduce these intellectual standards to students throughout the curriculum will be beneficial moving forward.

CONCLUSION

While clinical reasoning skills are vital to the success of pharmacists to provide quality patient care, optimal teaching and assessment strategies are lacking. This study also proposes a contextualized framework for providing focused and consistent clinical reasoning feedback to student pharmacists using multiple intellectual standards of clinical reasoning, including “Accurate,” “Concise,” “Connected,” “Specific,” and “Thorough.” Further investigation into additional intellectual standards of clinical reasoning and optimized sequence of introduction within pharmacy curricula is needed. Moving forward it is important that pharmacy educators incorporate and focus feedback related to these intellectual standards of clinical reasoning and develop learning experiences with systematic questioning around each of the intellectual standards.