Use of verbal protocol analysis for identification of ADE signals

Abstract

We describe the use of verbal protocol analysis for evaluating the textual signals used by pharmacists for detection of adverse drug events (ADEs). “Think aloud” technique was used to gain insight into how pharmacists reason about ADE occurrence, when reading patient progress notes. We used case-scenarios for five ADEs consisting of information regarding patient history, medications, laboratory results, vital signs and patient progress notes. Pharmacists extensively used information present in the progress notes to make inferences about ADEs.

BACKGROUND

Protocol analysis has been used extensively for eliciting verbal reports of thought sequences that occur when performing a particular task. Newell and Simon propagated the use of verbal protocol analysis for testing information processing models of human reasoning.¹ Additionally, Ericsson and Simon demonstrated that thoughts verbalized during the process of task completion represented the most proximal connection between thinking and verbal reports.² Previous studies have used “think aloud” methodology for understanding the decision-making processes of experts.^{3, 4} We used this analytic technique to gain insight into how pharmacists reason about ADE occurrence when processing patient progress notes.

METHODOLOGY

We developed five case scenarios representative of the ADEs of somnolence, hypokalemia, hyperkalemia, constipation and hypotension, which consisted of information regarding patient history, medications, laboratory results, vital signs and patient progress notes. The information was derived from real patient cases and structured in a manner identical to what the pharmacists would see if they were interacting with the computer interface. Using “think aloud” techniques for data collection, five pharmacists described their thought on detection of ADE, in two focus group sessions. Pharmacists were asked to verbalize their thoughts as they read the case scenarios and highlight the textual signals that led them to think that an ADE had occurred. The responses were recorded and transcribed for analysis.

RESULTS

We extracted 186 phrases from the transcripts and categorized them into five themes of information: medication orders (42%), laboratory reports (12.4%), vital signs (9%), and textual signals (36.5%, consisting of progress notes, 23.1%, and patient history 13.4%), that triggered detection of an ADE across the five scenarios. Results from the verbal protocol analysis provided empirical evidence that pharmacists used specific textual signals extensively to inform their ADE detection process. Pharmacists’ decision-making strategies were inferred from the generated protocols which included looking for drug-drug interactions, dosing, matching the outpatient and inpatient medications, looking at time of administration from the bar-code medication administration (BCMA) record, and matching medications with the admission diagnosis and patient history. Additionally the inadequacy of the progress notes and the reliance on the patient interview process for deriving conclusions about ADEs for the pharmacists was found.

CONCLUSIONS

Use of “think aloud” techniques for data gathering and analyses enhanced our understanding of information processing by pharmacists for ADE detection. Pharmacists relied heavily upon specific textual signals to deduce the occurrence of an ADE. Finally, implications for accurate documentation of the patient’s therapeutic condition to enable timely detection of ADEs are considered.