Understanding pharmacist decision-making for Adverse Drug Event (ADE) detection

Abstract

Rationale, aims and objective

Manual chart-review is an effective but expensive method for ADE detection. Building an expert system capable of mimicking the human expert’s decision pathway, to deduce the occurrence of an ADE, can improve efficiency and lower cost. As a first step to build such an expert system, this study explores pharmacist’s decision making processes for ADE detection.

Methods

Think-aloud procedures were used to elicit verbalizations as pharmacists read through ADE case-scenarios. Two types of information were extracted, firstly pharmacists’ decision-making strategies regarding ADEs and secondly information regarding pharmacists’ unmet information needs for ADE detection. Verbal protocols were recorded and analyzed qualitatively to extract ADE information signals. Inter-reviewer agreement for classification of ADE information signals was calculated using Cohen’s kappa.

Results

We extracted a total of 110 information signals, of these 73% consisted of information that was interpreted by the pharmacists from the case-scenario and only about half (53%, n=32) of the information signals were considered relevant for the detection of the ADEs. Excellent reliability was demonstrated between the reviewers for classifying signals. Fifty information signals regarding unmet information needs were extracted and grouped into themes based on the type of missing information.

Conclusions

Pharmacists used a forward reasoning approach to make implicit deductions and validate hypotheses about possible ADEs. Verbal protocols also indicated that pharmacists’ unmet information needs occurred frequently. Developing alerting systems that meet pharmacists’ needs adequately will enhance their ability to reduce preventable ADEs, thus improving patient safety.

Introduction

Previous studies have described the integral role that pharmacists play in adverse drug event (ADE) detection. In a systematic review by Kaboli et al. [1] pharmacist’s participation in clinical care, such as health care team rounds [2, 3] medication reconciliation services [4, 5], or patient discharge counseling [6, 7] improved patient safety. Additionally, pharmacists have participated in several ADE research studies as chart-reviewers [8–11]. Pharmacists’ training in therapeutics and comprehensive drug knowledge makes them an obvious choice for ADE surveillance [12, 13]. A meta-analysis of ADE incidence rates detected using chart-review showed that pharmacists are capable of detecting higher incidence rates than other clinical specialties, such as nurses or physicians and non-clinical personnel, engaged in ADE surveillance [14]. Participation of clinical pharmacists in ADE detection although valuable, is expensive, laborious and takes pharmacists’ time away from participation in patient care activities. While several studies have shown the important role that pharmacists play in ADE detection, no study thus far has evaluated pharmacists’ decision-making processes and information needs for ADE detection.

Background

Several methods of ADE surveillance, such as spontaneous reporting, and computerized surveillance techniques, are used in the clinical setting to detect ADEs. Previous studies have shown spontaneous reporting rates to be as low as 1.5% [15]. Computerized surveillance systems employ several mechanisms such as the use of ICD-9 codes [16] or rule based-criteria [12] for the generation of alerts. The performance of these systems is heavily contingent upon the rules that drive their signal detection. These rules perform well with integrated clinical information systems that offer links between laboratory, pharmacy, and administrative data [17–19]. A major disadvantage of these systems lies in their inability to search through progress notes for textual signals, leaving a considerable amount of data unexplored.

While alerting mechanisms use laboratory values, medication orders, and administrative data to uncover ADEs, chart review uses text as its primary corpus. Manual chart review is used as the gold standard in many ADE detection studies [8, 20, 21] owing to its ability to identify a greater number of ADEs than other methods [22]. Previous studies comparing chart review with spontaneous reporting and computer-generated alerts have shown a low overlap between the ADEs detected [23, 24]. Jha et al. found that chart review detected events primarily shown in the medical record as symptoms (e.g., change in mental status, nausea, and vomiting) and the computer more reliably identified events associated with changes in laboratory values (e.g., renal failure, hypoglycemia, hyperkalemia) [24]. This is a strong argument for using methods that combine use of text along with laboratory values and medication orders for increasing signal detection. Owing to the large resource utilization and expense underlying the chart review method, it is generally reserved for research studies [24, 25].

Building an expert system capable of mimicking the human expert’s decision pathway would increase the efficiency of ADE detection. According to Endsley, automation of human processes in complex and dynamic environments requires design of systems that represent closely the mental models of the user, which are central to effective decision-making [26]. By understanding the cognitive processes involved in ADE detection one can get an indication of what kinds of information signals are used by pharmacists during chart-review. Automating these signals would capture the advantages of the chart-review method while doing away with the associated labor and expense.

Previous studies have illustrated the use of theories and methods of cognitive science in the development and evaluation of clinical information system [27–29]. Patel et al have described the use of cognitive science methods to provide a framework for the analysis and modeling of complex human performance [29, 30]. The cognitive approaches of task analysis, specifically think-aloud protocol analysis, are commonly used in the domain of psychology to elicit knowledge from experts [27, 31–35]. Think-aloud protocols were developed by Ericsson and Simon to identify the sequence and verbalization of thoughts. The underlying assumption to conducting think-aloud analyses was that they depicted to a high degree of reliability the cognitive processes involved in performing the task[36]. Use of these methods in medical informatics has furthered understanding of the mental processes and knowledge based reasoning strategies used in clinical problem solving [27, 30].

Elstein et al. proposed the hypothetico-deductive model for understanding expert diagnostic reasoning [37]. The underlying theory of this model is that early generation of hypotheses allows physicians to take a more directed approach towards finding an accurate diagnostic solution. Therefore, an information system incapable of providing the necessary information for the generation of these hypotheses is impeding the clinician from finding the accurate diagnostic solution efficiently. We examine empirically the source of information for hypothesis generation and the unmet information needs of pharmacists for ADE detection. Such an assessment allows us to identify the sources of information in the EMR that can enhance the generation of critical hypotheses for ADE detection.

The focus of this study is to use think-aloud analysis to identify what types of signals the human expert looks for while using the method of chart-review in the process of ADE detection. Insight into the reasoning process of pharmacists will enable us to design automated systems capable of emulating their cognitive framework. Such systems can help bridge the gaps between pharmacists’ information needs and existing clinical information systems and serve the longer term goal of allowing improved medical care with lower cost.

Previous Research

An ADE surveillance study was previously conducted at the Veterans Administration (VA) Medical Center in Salt Lake City, a 110-bed tertiary care, teaching facility. Over a 20-week period (August 13 through December 31, 2000) newly admitted patients to all wards were randomly assigned to the study. Two research pharmacists carried out a prospective chart review using a well-defined list of inclusion and exclusion criteria. These criteria and the details of this study are published elsewhere [8]. Out of the 1,050 events that were identified, we chose two descriptive ADEs, namely constipation (117 events, 11.1%) and somnolence (36 events, 3.4%), and two ADEs based on laboratory values, namely hypokalemia (67 events, 6.4%) and hyperkalemia (35 events, 3.3%). All four of these ADEs were among the top ten most frequently occurring ADEs in our hospital system. Numbers previously published [8] eliminated ADEs that were mild in severity, took place prior to admission and did not meet certain criteria for causality and severity. For this study, the numbers reported above constitute total number of ADEs reported by the pharmacists, following manual chart-review.

Purpose

The purpose of this study was to use a cognitive approach, specifically think-aloud analysis, to assess the decision-making processes used by pharmacists for ADE detection, during chart-review. Additionally, information regarding the portion of the electronic medical record (EMR) from which the signals were extracted and pharmacists’ unmet information needs, are described.

Methods

Think-aloud protocol analysis was used to determine pharmacists’ information needs and reasoning strategies for ADE detection. The development of case-scenarios, conducting the focus-group sessions and the data analysis is described below.

Development of case-scenarios

Case-scenarios for the four ADEs of constipation, hypokalemia, somnolence, and hyperkalemia were developed. The results of the previously conducted study, which we have described above, were used to identify the four frequently occurring ADEs. An underlying criterion was to include a combination of ADEs: those that could be identified using traditional rule-based logic and those that needed to be identified through descriptions recorded in the patient’s clinical notes. Hypokalemia and Hyperkalemia represented the first category of the laboratory based ADEs. Somnolence and constipation belonged to the latter category of descriptive ADEs. We then chose actual patient cases representing each of the ADEs, to develop the case-scenarios.

Each case-scenario consisted of sections of the patient’s electronic medical record (EMR). These sections included information regarding the patient’s medications, laboratory values, vital signs and various types of clinical notes, such as, History and Physical notes, progress notes, pharmacy and social worker assessments.

The case-scenarios were designed to be as similar as possible to actual patient records that the pharmacists would encounter during their routine practice. However, some patient records were too large to read in a two hour focus group session. Thus, we presented excerpts of the clinical information to make it feasible to read and discuss them within the allotted time. The excerpts chosen drew heavily from the clinical information that was found relevant by the pharmacists, during the manual chart-review done in the previous ADE study. The case scenarios were de-identified, in keeping with HIPAA regulations. An excerpt from the case scenario of somnolence is presented in Figure 1.

Figure 1.

An excerpt from the case scenario for the ADE of somnolence

Focus group Sessions

One physician and two pharmacists used standard moderation techniques in conducting the focus group sessions [38–40]. Discussions were taped and later transcribed. The Institutional Review Board (IRB) approved the study and all participants provided written informed consent.

Five pharmacists from the specialties of intensive care, general medicine, and rehabilitation agreed to participate in the study. There were a total of 6 clinical pharmacists at the facility at the time of this study and we considered this sample as representative of the clinical pharmacists at this institution.

At the beginning of the focus group session, one moderator described the purpose of the study and the method of think-aloud analysis. The subjects were then asked to read through the case-scenarios and describe their thoughts. During the course of the discussion moderators also asked pharmacists about what they would do to verify their hypotheses about whether an ADE had occurred and whether the clinical information system (CIS) would be able to provide them with the necessary information for this verification.

Data Analysis

Qualitative assessment of the verbal protocols from focus groups allowed extraction of ADE information signals. Joseph and Patel [35] have described a methodology for further analysis of the protocols retrieved from think-aloud analysis. Categorization of the information signals was performed using two criteria: whether it was a ‘repetition’ or an ‘interpretation’, and it’s relevance for ADE detection [35]. A ‘repetition’ was defined as an information signal where the clinicians reproduced the stimulus text verbatim and did not explicitly associate the cue(s) with any aspect of the correct ADE. An ‘interpretation’ was defined as an information signal where the clinicians explicitly linked information in the text segment to any aspect(s) of the ADE. Patel and colleagues [35, 41, 42], describe further classification on the basis of relevance of the information signal for detecting the accurate diagnosis. While Patel’s methodology described a classification into critical, relevant and irrelevant, we modified this classification and determined only the relevance and irrelevance of cues while retaining the definition for these categories [35]. A ‘relevant’ cue was defined as one that provided potentially important information for detecting the ADE. An irrelevant cue was one that was not considered directly relevant to the ADE in question. Additionally, ADE information signals were mapped to the portion of the electronic medical record (EMR) from which pharmacists extracted their information. Two reviewers classified the information signals; disagreement in classification was resolved by discussion until consensus was reached. Consensus was reached on all of the signals that were identified. The categories presented here reflect the final reasoning strategy utilized. Cohen’s kappa was calculated to determine inter-rater agreement. Analysis was conducted using SAS, Version 9.1.

Pharmacists’ unmet information needs were also extracted from the verbal protocols. These information signals were grouped together into themes based on their semantic relatedness and the type of missing information.

Results

Two focus group sessions were conducted for a total of six hours. Figure 2 shows an excerpt from the focus group session where pharmacists discussed the case scenario of somnolence. Extracted phrases, related to ADE detection are italicized in the excerpt. We extracted a total of 110 information signals from the pharmacists’ verbal protocols. Of these, 60 were cues for ADE detection across the four ADE case-scenarios and 50 were related to unmet ADE information needs of pharmacists.

Figure 2.

An excerpt from the focus group session where pharmacists discussed the case scenario of somnolence. Extracted phrases, related to ADE detection are demarcated in italics in the excerpt.

Distribution of ADE information signals across the four ADEs were somnolence 35% (n=21), hyperkalemia 30% (n=18), hypokalemia 20% (n=12), and constipation 15% (n=9). Information signals were further classified into the categories of repetition/interpretation and relevant/irrelevant. Excellent reliability was demonstrated between the reviewers for classifying signals in the categories of repetition/interpretation (kappa=0.80, CI: 0.63–0.96, p<0.0001) and relevant/irrelevant (kappa=0.78, CI: 0.61–0.95, p<0.0001). An example of the categorization of some information signals for the ADE of somnolence, along with the portion of the EMR from which the signals were extracted, is presented in Table 1. Most ADE information signals were extracted from the clinical notes section (n=29, 48.33%), followed by medications (n=18, 30%), implicit information (n=9, 15%) and equal numbers from laboratory results (n=2, 3.33%) and vital signs (n=2, 3.33%). Distribution of signals for the four ADEs, across the categories of Notes, Laboratory results, Medications, Vital signs and Implicit information, is shown in Figure 3.

Table 1.

Examples of the classification of information signals for the ADE of somnolence. The source of information indicates the portion of the EMR from which the signals were extracted.

	Information Signal	Source of Information	Classification of Information Signals
			Repetition	Interpretation	Relevant	Irrelevant
1.	there is of course the mention of melanotic stools	Progress Notes	1	0	0	1
2.	(shows) some mental status changes even after he was transfused	Progress Notes	0	1	1	0
3.	Hematocrit seemed to respond nicely	Laboratory Values	0	1	0	1
4.	he is 80 yrs old even though that is a relatively low dose, I’m not sure what effect that is having also with his dementia or his mental status changes.	Medication List	0	1	1	0
5.	there was this history of you know, hospital multi infarct dementia vs Alzheimer’s and whatnot.	Progress Notes	1	0	1	0
6.	(…) he is on different kinds of medications which can also interact (…) causing accumulation of increased levels of Haloperidol.	Medication List	0	1	1	0

Figure 3.

Distribution of signals for the four ADEs, across the categories of Notes, Laboratory results, Medications, Vital signs and Implicit information.

Fifty information signals regarding unmet information needs were extracted. Some unmet information needs (n=8, 16%) were only relevant to the case-scenario that was presented, and were classified separately. Forty-two of the remaining information needs were grouped into 7 themes based on the type of information. These themes were: Inability to access outpatient medications, Incompleteness of information entered in the notes, Institutional change, Needs related to the clinical information system (CIS), Reliance on paper resources, Sources of Information other than the CIS and Trust regarding certain note-types. The themes along with the specific underlying needs are presented in Table 2.

Table 2.

Unmet Information needs of pharmacists for ADE detection and specific examples under each category.

Missing Information and Information Needs	Theme
What were his medications on admission? I would like to know the dose of the aspirin. It could also be over the counter. I don’t know. The first thing that I would do is look at his outpatient meds.	Inability to access outpatient medications
Just tell me on the H&P what is the IR pacemaker placement? What kind of a pacemaker placement (is it?) I just wanted to know what’s different in her blood, you know what is different in her blood pre-op/post –op, I guess. When did her Afibs start? It says the patient remains in Afib. Is she controlled, is she rate controlled Afib? Tele (telemedicine) seems to keep more or less everything at the bedside. They do a template. The template tells you nothing. Yeah the templates are really bad.	Incompleteness of information entered in the notes
When they started the templates, it was just a bad idea. That made the nursing notes utterly useless. I actually did that (looked at the notations next to the BCMA records), because I was the one who brought BCMA up on line. And I still can’t get the nurses to do that. So on finger sticks they’re supposed to put the blood sugar and how much insulin they gave and everyday I’ll have some patient who just has nothing.	Institutional change
I have to look in three different places (for the blood sugar level and amount of insulin received). You can look in BCMA and look under the medication log is one place you can look and see if they have added comments but you have to check what is it? View order or view audits and comments. You have to click on that to see that but you have to look at the individual day or you can go under the lab and type in finger stick and then up will come finger sticks. Then see you’ve got results in there, then you’ve got results in BCMA (it’s really easy, P1, all they have to do under dose is put 352 = 10) and that is exactly what nurses are doing now. So if someone is in tele and they are going to convert him to oral dose, I want to know what IV dose they were on. Like how much it was running per hour and sometimes I can’t find that information anywhere. It’s just not there. I don’t know how many milligrams for an hour. Sometimes they write, sometimes it’s there and sometimes it’s not and then you can’t make a great conversion to oral therapy if you don’t know where they were. Like you will look at the BCMA stuff and it is actually does not list the missed doses yet.	Need related to CIS
Or when the rates were changed and you can’t even find it on the bedside. There is no, it is an incomplete chart in my mind cause I don’t know when you increase the rate, what the rate is now, unless you look at the flow charts.	Reliance on Paper Resource
I think you would have to interview the patient rather than anything because it (reasons for noncompliance) is usually not documented. So I probably get a lot more information from nurses in person. Um, and I have a lot smaller unit so I tend to get a lot more one on one from social workers and stuff. I have noticed that it is helpful, when some patients come in like this, is to interview the family. They will actually talk about this is baseline or this is not baseline and this is different or this is the same as what he is usually like.	Sources of Information other than the CIS

Discussion

The Institute of Medicine (IOM) report, To Err is Human, provided a detailed analysis of the alarming frequency with which medical errors occur in our healthcare system[43]. Developing a culture of safety requires paying attention to human factors, such as building collaborative multi-disciplinary clinical teams, encouraging communication about errors, and developing clinical decision-support systems capable of providing information in an effective and timely manner. Building an expert system capable of automating the human expert’s decision pathway can improve the efficiency of ADE detection. Pharmacists frequently participate in the clinical, operational and research activities encompassing ADE surveillance in a healthcare organization. This is the first study to use a cognitive approach for understanding the decision-making processes used by pharmacists for ADE detection.

Assessment of ADE Information Signals

All the pharmacists that participated in this study had considerable clinical experience. As shown by Patel and colleagues [44], in the domain of medicine, these experts used a forward reasoning approach, that is they analyzed the data available in the case-scenarios to generate hypotheses about possible ADEs. Only 27% of the signals consisted of information that was explicitly repeated from the case-scenario. The remaining 73% of the information was implicitly derived by the pharmacists. Pharmacists used the data available in the patient’s EMR to make implicit deductions and validate hypotheses about possible ADEs that might have occurred.

Previous research also suggests that experts have the ability to discriminate between pertinent and non-pertinent information, as opposed to novices who are unable to sift out the less relevant information [44, 45] in the process of problem-solving. Also, previous research has classified the need for additional diagnostic aids as a characteristic of problem-solving by novices [31]. Contrary to these conclusions, only about half (53%, n=32) of the information signals were considered relevant for the detection of the ADEs discussed in the focus groups. Moreover expert pharmacists in this study needed additional information such as laboratory tests, outpatient medications, and clarification regarding clinical notes, in order to generate their hypotheses. This might be a result of the limited information that could be presented in the case-scenarios. Pharmacists often asked for information that could have been present in the patient’s medical record but was not included in the case-scenario owing to constraints of document length. However, this finding might provide support to our previous claim about pharmacists’ hypothetico-deductive approach to problem-solving in which the available data are assessed and hypotheses are generated, but there is need for additional information to verify whether their implicit derivation regarding the occurrence of an ADE is accurate.

Of the 60 ADE information signals 15% were classified as implicit information. Examples of this category include signals like, “Why no one says he has Afib anywhere?. It doesn’t say anywhere he has coronary artery disease that I would think a diabetic would.” or “(..)why is somebody who has a variety of these things like he is not on Spironalactone. That seems kind of odd too.” These signals consisted of information where the pharmacists made implicit judgments about the patient’s data and could not be directly mapped to a specific portion of the patient’s medical record. These judgments could have been driven by previous experiences of having seen similar patients, with an expectation to see certain diagnoses or medications in the patient’s EMR.

In a review of problem solving methodologies used in clinical reasoning, Elstein and Schwarz described four possible strategies: hypothesis testing, pattern recognition, problem-solving by specific instances or prototypes [46]. Two competing models have been proposed to better understand pattern recognition by experts. One model is “instance based”, in which experts categorize a new case to a specific instance that they have previously seen [47, 48]. The other model attempts to understand pattern matching on the basis of a more abstract prototype approach, and proposes that clinical experience facilitates the construction of mental models, abstractions, or prototypes [49]. The prototype model is supported by several characteristics of experts, such as their ability to ask for additional information to complete a clinical picture and relate these findings to the overall concept of the case [46]. Pharmacists’ need for additional information and their ability to make implicit judgments in order to complete the clinical picture beyond the data presented in the case-scenario, fits well into the prototype model described above. Pharmacists’ vast knowledge about drugs allows them to construct diverse links between medication information and the clinical information that they see in the notes. This is in keeping with Lemieux’s findings [50] concerning the ability of experts to create abstract sets of semantic relations between clinical features and diagnostic categories.

Almost half of the ADE information signals were extracted from the clinical notes section of the EMR. A national impetus towards implementation of EMRs in hospitals calls for an assessment of the organization of information in the EMR for ADE detection. The focus of ADE surveillance systems thus far has been on developing alerts largely based on medication and laboratory data [17, 18]. This study validates pharmacists’ extensive utilization of text-based signals present in the clinical notes, to deduce the occurrence of an ADE. It was apparent that pharmacists relied on information in the free text format for the more descriptive ADEs, such as somnolence (62%) and constipation (56%). For these ADEs, specific laboratory and medication values usually fail as direct indicators. Contrary to our expectation, for the more lab-based ADEs, such as, hyper/hypokalemia, pharmacists did not rely only on medication and laboratory data. The most important sources of information, for both hyper and hypokalemia were clinical notes and medications. Thus, this study provides further empirical support that text-based triggers can complement more traditional computerized ADE surveillance technologies to broaden their scope of detection.

Identifying missing information needs

Assessment of the verbal protocols also indicated that unmet information needs occurred frequently. We categorized these missing information needs into a total of 8 themes. One theme represented the information that was missing in the specific case-scenario, as presented. Examples of specific information needs included in this theme can be found in Table 2. The remaining 7 themes represented a more global assessment of the CIS for gaps in information for ADE detection. A discussion of the gaps identified from the focus group sessions and their correlation with the missing information in the EMR is presented below:

Inability to access outpatient medications

This theme represents a lack of information regarding outpatient medications the patient was receiving. Medication reconciliation is important for preventing medication errors and is mandated by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)[51]. Pharmacists rely primarily on patient and family interviews and medication administration records (MAR) from other facilities to derive this information. At the VA in Salt Lake City a local solution has been developed and integrated with the clinical information system namely, CPRS (Computerized Patient Record System). This local module enables the pharmacists to check against the outpatient medications that the patient is receiving and produces a list of non-reconciled medications for the pharmacist to review.

Documenting herbal medications poses another problem. The CPRS system has the capability to record medications that are outside of the VA formulary. However, this list of medications is small and while it includes some herbal medications, pharmacists find it difficult to choose specific medications from the list. Also, the list forces the pharmacist to choose specific dosage, strength and brand-information that is often not available and pharmacists are forced to record inaccurate information. Extending this list of non-VA formulary medications might allow greater flexibility and accuracy in documenting these medications.

Incompleteness of information entered in the notes

Clinical notes often lack the level of detail that is necessary to derive useful information about the occurrence of ADEs. Clinical notes form an important mechanism for various members of the clinical team to assess the patient’s condition and document changes in a timely manner. In the focus group session pharmacists mentioned that they would much rather have verbal conversations with the nurse to receive the most recent update rather than rely on the clinical notes. Detection of descriptive ADEs, such as somnolence and constipation, relies heavily on the presence of textual cues. Lack of patient-specific and detailed information in the clinical notes will inhibit detection of ADEs in a timely manner. Institutional change: The lack of detail in clinical notes and a need for more structured documentation led to the development of templates within clinical notes, in the CPRS system. While templates are useful for novice clinicians in structuring their documentation, expert clinicians usually create work-arounds and avoid entering details in these notes. This resulted in a lack of information in the templated notes and the recording of most of the information in free-text format in the addenda following the note. Pharmacists felt that the templated notes provided generic information that was not helpful in making clinical decisions specific to the patient parameters that needed attention.

Other issues suggesting institutional change consisted of the nursing work-arounds with the use of Bar Code Medication Administration (BCMA) system. The BCMA system is a safety net in ensuring the five rights: right patient, right drug, right time, right dose and right route, for patient safety. Nursing work-arounds allowed charting of medications that were not actually administered to the patient or charting medications by entering the patient identification number instead of actually scanning the patient’s bar coded wrist-band. An institutional level mandate on the inability to over-ride BCMA alerts would curtail such work-arounds.

Need related to the clinical information system (CIS)

Pharmacists were satisfied with the alerting provided for drug-drug interactions during order entry and thought that the system did not over or under alert in most cases. However, they did not find the drug-disease interaction alerts to be adequate. Pharmacists were generally satisfied with the ability to track varied pieces of the patient’s information through the integrated CIS. The CPRS system allowed pharmacists to read clinical notes. Pharmacists can also order drugs and laboratory tests and view changes in drug orders and dosage changes. To retrieve information about previous admissions and visits, pharmacists sometimes have to refer to the patient history through the Veterans Health Information System and Technology Architecture (VistA) system. This is specifically important when they look for drug allergies. Documentation of drug allergies might be incomplete, and pharmacists often search manually for the date that drugs that were discontinued, and then read the notes around that date to detect descriptions of previous harm related to the drug. Documentation of allergies is essential for informing future use of the drug and for triggering future drug allergy warnings. Encouraging appropriate documentation in the CPRS Adverse Reporting Tracking (ART) system will protect patients and enhance patient safety efforts.

Reliance on paper resources

Despite the existence of a fully integrated CIS, pharmacists’ reliance on paper resources was disappointing. Real-time information regarding vital signs was difficult to procure from the CIS because only a few random values were entered into the system and the information was entered once per nursing shift. This prevented deliberations of trends over time for which pharmacists relied on bedside charts. Also, pharmacists’ felt that the CIS was designed with oral medications in mind and did not do a good job of recording IV medications and drips. Although the CPRS provides the facility for recording ‘ins and outs’ or I/O s, information regarding changes in the rates of administration of medications was usually obtained from flowcharts or sometimes from physically looking at the pump, rather than the CIS. Pharmacists’ noted that this was especially important for preventing ADEs related to antibiotics, such as, vancomycin or high-risk drugs such as warfarin where flow rates needed to be closely monitored.

Sources of Information other than the CIS

For information regarding medication history, compliance, allergies, use of outpatient and herbal medications, and tolerating certain medications in the past, such as antibiotics, pharmacists relied on patient and family interviews. As discussed previously pharmacists often did not find this information documented in a reliable manner. Pharmacists found Interviews with patient’s family to be important in extracting information regarding patient symptoms, how long the patient had these symptoms and their perceptions about possible medications that could be related to the certain indications. These pieces of information were important in determining drug-drug interactions, interactions with herbal remedies and possible allergies that might not have been previously recorded in the CIS. Pharmacists also relied on verbal communication with nurses for most recent updates on the patient’s condition, actual administration of medications recorded in BCMA and general assessment of patient status such as urine outputs and bowel movements.

Trust regarding certain note-types

Medical student notes were usually very long and pharmacists found them to be most helpful in determining patient status. Pharmacists had mixed reactions regarding nursing notes and found the templated nursing shift assessment to be particularly unhelpful. Often the physician notes would contain information regarding changes in the patient’s medication profile which the pharmacists found helpful.

The assessment of unmet information needs can serve as a framework for the organizational and functional content of CIS designed to aid detection of ADEs. In this light, it is important to note the existence of such information gaps in one of the most well-integrated CIS that exist today. The results have several potential implications for design of future information systems for ADE surveillance. The CPRS incorporates several functionalities and checks for the prevention of ADEs; however not all of these functionalities are put to use as intended.

Recommendations for upgrading the CIS

Based on the information needs identified above we make specific recommendations for upgrading the current VA clinical information system. Medication Reconciliation is one of National Patient Safety goals of the JCAHO making it mandatory for receiving hospital accreditation [51, 52]. While standard procedures such as reviewing admission medication histories and discharge medication counseling are carried out improved documentation in CPRS for medication reconciliation is needed. Medication reconciliation is a collaborative effort between pharmacists and nurses that requires a redesign of clinical processes. Improved documentation could consist of a home medication profile report, home medication reconciliation report, discharge medication reconciliation report, and patient discharge medication report [53] for reconciling medications electronically. While there is an attempt to implement local solutions, efforts to leverage the national EMR and CPOE system to recognize non-reconciled medications are needed.

The VA has an excellent clinical documentation system however the pharmacists reported the lack of detailed clinical information in the notes. Lack of detail might be an issue of training clinicians to record detailed information in clinical notes. Previous studies have shown that educational interventions can improve the clinical documentation quality, in the ambulatory setting [54, 55]. Additionally, clinical documentation could be structured in a manner that allows the nurses to record information regarding patient status, I/O medications and current allergy information, in an efficient manner. Participation of nurses and pharmacists in documentation development is essential for identifying the structure of progress notes to reflect the thought process of nurses and the data elements required by pharmacists to derive essential information from the clinical notes. This study does not indicate that the semi-structured template based notes, such as those that are currently used at the VA, should be discarded. As has previously been studied [56] the type of clinical documentation varies depending on the workflow practices of an institution. Fully structured, coded notes, such as those offered by vendor systems, for example Logician by General Electric, EpicCare by Epic Systems, facilitate real-time surveillance and data collection for research. However, a major disadvantage is the time required to fill out the entire note during patient encounters, additionally these notes may lack the flexibility and expressivity required for general medical practices [56]. Thus, future work should identify mechanisms to bridge the information gaps of end-users, through improved clinical documentation.

Use of cognitive methods of analysis, such as “natural data collection” and “task analysis”, inform the iterative development of information systems that reflect medical decision-making as it occurs in routine practice [27, 57]. Systems that are designed to keep the user in mind should have the capability to be adopted into the user’s workflow and better represent the mental concepts that are utilized by the user for problem solving [58]. Thus, understanding the pharmacist’s cognitive framework for ADE detection and unmet information needs would allow us to develop surveillance systems that can begin to “think” like a pharmacist.

Limitations

This work represents a pilot study for inferring pharmacists’ needs for ADE detection and has limitations. Even though five out of a total of six of our clinical pharmacists participated in this study, the cognitive evaluation is limited by its small sample size and the evaluation of pharmacists from only one institution. Discussion of unmet information needs is relevant to the clinical information system used at our hospital. Although the CPRS system exists at all VA facilities there are modifications that can be made at a local level. Further evaluation of generalizability of these information needs and possible solutions that exist in other VA facilities or integrated CIS is needed.

Conclusion

This is the first study to attempt to understand the decision-making processes and information needs of pharmacists for ADE detection. This evaluation enhanced our understanding of pharmacists’ information needs. Developing alerting systems that meet pharmacists’ information needs will enhance their ability to reduce preventable ADEs, thus improving patient safety.