Medication safety and knowledge-based functions: a stepwise approach against information overload

Abstract

Aims

The aim was to improve medication safety in an emergency department (ED) by enhancing the integration and presentation of safety information for drug therapy.

Methods

Based on an evaluation of safety of drug therapy issues in the ED and a review of computer-assisted intervention technologies we redesigned an electronic case sheet and implemented computer-assisted interventions into the routine work flow. We devised a four step system of alerts, and facilitated access to different levels of drug information. System use was analyzed over a period of 6 months. In addition, physicians answered a survey based on the technology acceptance model TAM2.

Results

The new application was implemented in an informal manner to avoid work flow disruption. Log files demonstrated that step I, ‘valid indication’ was utilized for 3% of the recorded drugs and step II ‘tooltip for well-known drug risks’ for 48% of the drugs. In the questionnaire, the computer-assisted interventions were rated better than previous paper based measures (checklists, posters) with regard to usefulness, support of work and information quality.

Conclusion

A stepwise assisting intervention received positive user acceptance. Some intervention steps have been seldom used, others quite often. We think that we were able to avoid over-alerting and work flow intrusion in a critical ED environment.

WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT

• Clinical decision support systems (CDSS) for medication safety are considered helpful.

• Frequently, CDSS are not used as intended and problems such as over-alerting and over-riding are often reported.

• There have been publications advising how to improve the situation.

WHAT THIS STUDY ADDS

• We designed a new CDSS for an emergency department (ED) which distinguishes between four levels of clinical decision support to avoid over-alerting and work flow disruption.

• The approach prioritizes physician information requirements and observed typical medication safety problems in the ED to generate either immediate alerts or medication safety information of lower urgency and permits in depth search for detailed drug information.

• Within the ED we measured that active interventions were used for a small number of drugs only.

• In contrast to comparable paper-based measures the computerized interventions were rated better regarding perceived usefulness, compatibility to work-flow and intention to use.

Introduction

Medication errors (MEs) and adverse drug reactions (ADRs) are common and cause up to 10% of hospital admissions [1]. Estimates of the Institute of Medicine from 1999 indicate between 44 000 and 100 000 deaths per year in the US which may be attributable to MEs and ADRs. Pirmohamed et al. report a prevalence of 6.5% of hospital admissions which are related to ADRs [2]. About half of the ADRs and (by definition) all MEs should be preventable [3].

Clinical decision support systems (CDSSs) have been repeatedly advocated to prevent drug therapy problems [4, 5]. A multitude of commercial drug information systems such as Black-BoxRx, Drugdex, Epocrates, Facts and Comparisons, Lexicomp, PDF.NET are available internationally [6] as well as from German vendors (e.g. ID Diacos Pharma, ifap Medical Decision Support System, Pharmindex Plus, RpDoc Klinik) [7]. They can be invoked on demand or linked as a CDSS to the hospital information system.

We faced the task to integrate CDSS functions in the working environment of a busy emergency department (ED). For an ED, similar to intensive care environments, it is absolutely essential to avoid work flow disruption and time delay in treatment which could result in a catastrophic patient outcome [8]. Thus we had to give special attention to critical reports such as [8–19], where other researchers reported documentation problems, faulty dosing [9–12], duplicate medications [13], wrong dose suggestion [14], but most essential over-alerting and alarm over-riding [15–18].

This led early on to the decision for voluntarily activated, mostly non-automatically triggered decision support. On the other hand we intended that the CDSS functions would be used. A recent editorial by Kulikowski [18] illustrates the contrast between todays poor acceptance and routine use of CDSS and the high expectations and postulated CDSS demand which has been described by Spreckelsen et al. [19]. This gap could potentially be diminished with a design according to the ‘five rights approach’ of medication administration [20, 21], which has been adapted for CDSSs by Sirajuddin et al. [22]: ‘the interventions must deliver the right information, to the right person, in the right format, through the right channel, at the right point in workflow’.

Therefore we intended to answer the following questions:

1. Which are the implications of the ‘five rights’ approach for the design of computer-assisted medication safety measures?

2. Which major design factors must be considered to avoid over-alerting and disruption of the clinical work flow?

3. Can we identify success factors and barriers for the acceptance and routine usage of such IT-based decision support measures in drug therapy?

Methods

The project described here has been pursued at the Klinikum Fürth (Germany), a 749 bed adult tertiary care/level III and teaching hospital with an annual ED census of 40 000 patients. A prospective three phase study design aimed at assessing and improving medication safety for ED patients. In the first study phase 752 patients have been screened by an independent expert panel for risk situations associated with drug therapy. The second phase comprised paper based and oral educational methods, checklists and teaching to address these safety issues of drug therapy which were identified in the first phase. In the third study phase we augmented this with electronic tools for documentation and assessment of drug therapy including the described computer-assisted interventions. Study infrastructure and data collection have been described in detail elsewhere [23, 24]. The study was approved by the ethics committee of the University Erlangen-Nürnberg as well as the local data safety officer.

Analysis and design of an application according the five rights principle

The ED physicians were identified as the appropriate stakeholders, since they are directly responsible for ordering and/or continuing the drug therapy. Workflows in the ED were analyzed to detect the right time to provide medication safety advice. Results of the first study phase showed that the appropriate information concerned especially those drugs which the patient took regularly at home and which produced the majority of risk situations. For the computer-assisted interventions in phase 3 the clinical workstation was identified as the appropriate communication channel in this phase, whereas in the previous study phase 2 we relied on paper based information and teaching. For the right format we decided to test different formats of information presentation to find the right one (Figure 1).

Figure 1.

The new Orbis electronic case sheet is based on the 4SDRI principle [22]. Seven drugs have been entered. Step I: Tick boxes indicating whether or not the current indication of the drug is known. Step II: A tooltip information appears for drugs previously associated with problems in the ED. Step III: ‘MedikationsCheck’ button invokes CDSS (OntoDrug®) and transfers patient drugs, diagnoses, age, gender and creatinine value. New pop-up window displays contraindications, warnings and interactions, here metformin is contraindicated with diagnosis of chronic renal insufficiency. Step IV: warning extract is linked to SPC in Pharmindex Plus® and activated by clicking on highlighted link. The risk situation found in step III is immediately highlighted. Targeted drug information is displayed

Implementation of computer-assisted interventions to improve medication safety

The following activities were accomplished to improve safety of drug therapy at the ED of Klinikum Fürth:

1. An existing electronic case sheet for ED patients within the commercial hospital information system (HIS) in Fürth (Orbis, AGFA HealthCare, Germany) [25] was redesigned and adapted in order to facilitate acquisition of the previous drug history and to invoke a stepwise medication check.

2. A graphical end user interface for the server-based commercial drug information system (MMI Pharmindex Plus, Medizinische Medien Informations GmbH, UBM Medica, Germany) has been installed on each ED workstation.

3. The first medication warning has been realized in the electronic case sheet as a tooltip which is based on a list of drugs most often causing medication safety problems in the first study phase.

4. A medication check button (Medicheck) has been implemented in the case sheet invoking a locally installed CDSS (OntoDrug®, Medizinische Medien Informations GmbH, UBM Medica, Germany) used for on-demand checking of contraindications and severe drug problems. For this purpose the electronic case sheet was enabled to compose an xml file containing the patients diagnoses list, gender, age, creatinine level and all documented drugs. A background process scanned this file and invoked the CDSS with the data transfer. The CDSS was modified to display only crucial contraindications, drug–renal impairment problems and clinically relevant drug–drug interactions. Dispensable decision support functions such as doping information were switched off. For a period of 26 weeks the CDSS was controlled automatically when a new patient case sheet was filled, and the physician had to confirm at least once that he or she had noticed the information.

Two additional mobile workstations on a cart were added to the three existing in the ED department to support bedside electronic documentation.

Evaluation of use and acceptance of implemented approach

To monitor the use of the different computer-assisted interventions log functions have been implemented. Logging was continuous, we present data of a 6 month period centred around the third study phase. In the HIS electronic case sheet a counter clocks the activation of the tooltip. Items of checkbox ‘indication still valid for drug’ have been analyzed directly.

To evaluate the emergency physician's use and acceptance (or rejection) of measures in phase 2 (education, checklist, poster) and phase 3 (computer-assisted intervention) an explorative survey was conducted using a three-part questionnaire. The first part contained six questions assessing the usage behaviour, e.g. ‘how often did you use the xyz medication safety measure?’, and three open ended questions to capture perceptions, e.g. ‘in which situation did you use the xyz measure?’. The second part was based on a validated questionnaire of [26] using the technology acceptance model 2 (TAM2) [27, 28]. Whereas Chismar & Wiley-Patton [26] examined the acceptance factors perceived usefulness, perceived ease of use, subjective norm, image, job relevance, output quality, demonstrability of results and use intention we added items from [29, 30] and [28] to examine also physicians' resistance to change, compatibility to work practice and voluntariness of use. All items were measured on a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. A pre-test of the questionnaire resulted in pruning four items. The resulting 26 items indicated a reliability level (Cronbach's alpha) of 0.88 to 1.00.

The last part of the questionnaire comprised four questions regarding job level, work experience, experience with electronic health record and self-assessed computer skills.

Data were collected once on a sample of 12 physicians working permanently in the Fürth ED. Ten questionnaires (83.3%) were returned, one incomplete, resulting in a valid response rate of nine (75%). Descriptive statistics were used to describe the sample population and the self-reported usage data. For the acceptance measurement only respondents who used the medication safety measures were counted. The Mann–Whitney test was used when appropriate to compare data between different measures (significance level of 0.05).

Results

Electronic information on drug risks: the four step drug risk indicator

Following the five rights approach drug risks were presented to the user in a four step drug risk indicator (4SDRI) within the electronic case sheet for ED patients. The electronic case sheet was complemented with a list of fields to record home medication using structured data from a drug list containing more than 850 drugs with generic and brand name. Figure 1 demonstrates the 4SDRI implementation in screenshots.

Step I had the goal to prompt ED physicians to consider if each single drug taken at home was still justified for the patient. Before intervention we had noticed that in up to a third (33%) of all drug risk situations it was not documented or traceable why the drug should have been taken. This motivated the insertion of a checkbox for each drug to confirm that the drug indication was known (and presumably valid).

Measures of step II were also directly based on real drug-related risk situations identified in the first study phase. An information button (infobutton) was implemented for the top 160 drugs involved in the majority of drug problems identified in phase 1 (in phase 2 this information was provided on a pocket checklist). The infobutton served as an immediate visual indicator producing a tooltip message when moving the mouse over it. The tooltip contained a catalogued short message referencing the typical risk situation(s) observed in phase 1. Typical risk situations included, e.g. drugs such as acetylsalicylic acid causing a bleeding or drugs prolonging the QT interval in the ECG such as citalopram. An example tooltip (2) is shown in Figure 1 (it presents a warning suggesting to check the patient's renal function).

Step III was designed as a digital ‘Medicheck’ button to be used for actively checking contraindications, renal impairment and, if necessary, drug–drug interactions. In our example metformin is contraindicated due to patient's diagnoses.

Step IV enables the involved clinician to access more detailed drug information. Case specific links in the CDSS point directly to the server based drug information system containing enhanced summary of product characteristics (SPC) for more than 11 000 pharmaceutical products of the German market. In our example the contextual contraindication of a drug due to patient's diagnosis is directly linked to the SPC description of metformin-ratiopharm® describing this risk situation.

System use

In a period of 32 weeks 13 257 drugs have been documented for 9480 cases. In 3009 cases the electronic case sheet was not used or no drug was documented. The results in Table 1 and Figure 2 concern only those electronic case sheets where at least one drug has been documented for the patient. There we recorded an average of 4.12 drugs per patient. We have normalized the data in Table 1 and Figure 2 using either number per patient case sheet or number per drug as the common denominator. Data are displayed as weekly average.

Table 1.

Number of cases, number of completed electronic case sheets (ECS) and drugs per ECS cumulated through calendar weeks and logs of system use based on two steps: confirmed indication per drug and invoked tooltips per drug

Figure 2.

Frequencies of cases and electronic case sheets plus means of usage of four steps approach per calendar week. , cases; , electronic case sheets; , electronic case sheets with ≥drug; , drugs/electronic case sheet (mean); , indications confirmed/drug (mean); , tooltips/drug (mean)

In step I only in 3% of the documented drugs the checkbox ‘indication still valid for drug’ was used. In contrast the tooltip of phase II was used on average for 48% of all documented drugs. For technical reasons steps III and IV were not logged.

Acceptance of the conventional and digital medication safety measures

Among the nine respondents were two senior physicians, one specialist in internal medicine and six junior doctors. Two-thirds had less than 5 years working experience. About half rated their experience with electronic data processing ‘less than 3 years’. Seven participants ranked their computer ability as average, one as poor. Seven had used the paper based drug safety measures in the past, eight physicians had used the electronic case sheet and one had no experiences with both.

Self reported system use for paper based poster, paper based checklist and electronic case sheet was measured on a five-point scale ranging from ‘don't use it at all’ to ‘several times a day’ and results are given in Figure 3.

Figure 3.

Frequency of medication safety measure usage in daily routine, n = 9 emergency physicians. , no answer; , use several times per day; , use once a day; , use weekly; , use monthly; , don't use at all

The electronic case sheet was adopted best and used daily by five persons. Next came the pocket checklist and the poster information was least used. In the free text comments the users complained that posters were not visible from each bedside and checklists were too specific and unfamiliar compared with, for example, ‘Arzneimittel Pocket’ (pocketbook for drugs referenced in Germany). Lack of time to record the drugs manually in the system was given as reason to neglect the electronic case sheet.

Then we asked how often electronic support had been used in patients with ‘critical’ drug orders (many drugs, diagnoses etc). There the majority of our physicians answered using measures in ‘up to 10% of orders’ on a scale ‘don't use at all’, ‘up to 10%’, ‘up to 25%’, ‘up to 50%’, ‘up to 100%’. A quarter of the ED physicians indicated none or insufficient training for the medication safety measures. In accordance with the log data, the tooltip inside the electronic case was used most (eight of eight), followed by the Medicheck button invoking the CDSS OntoDrug® (five of eight) and the SPC source PharmIndex Plus® (five of eight). Four of the eight physicians reported the use of the field ‘UAE’ (adverse drug event). The checkbox ‘indication still valid for drug’ was reportedly used least (two of eight).

Technology acceptance has been rated in Table 2, again considering n = 8 physicians for the electronic case sheet and n = 7 physicians for the non-digital measures education, paper poster and paper checklist. Although ratings for paper based measures are mostly somewhat lower, differences between computer and paper based measures are, due to small sample size, not significant (Wilcoxon test, P = 0.05).

Table 2.

Mean scores of physicians' perceptions of the different measures (five point scale: 1 = strongly disagree; 5 = strongly agree)

Both paper-based and computer-based measures were rated useful (PU, mean = 3.61/4.43), important to support the work (JR, mean = 4.36/4.56) to have a high level of quality of information (OQ, mean = 3.80/4.33). The results of the medication safety measures were ‘tangible’, observable and communicable (RD, mean = 3.61/4.16). The influence of colleagues (SN, mean = 3.60/4.00) on the usage behaviour was considered high. The perceived ease of use (EOU, mean = 3.36/3.38) was rated moderate. The measures were used by choice mostly (VO, mean = 4.57/4.25). The respondents did not confirm that the use of such measures would improve one's status (IM, mean = 2.83/3.07). Although the usage of the measures only partially fitted into their current workflow (COM, mean = 2.57/3.50), physicians were willing to adapt their way of working (RTC, mean = 4.58/4.36). The intention to use the measures was high if barriers could be overcome (ITU, mean = 3.50/4.42).

Discussion

Alerts and alert over-riding

In EDs a response to sudden events is part of the daily routine. Many events are medical equipment alerts, e.g. monitoring or infusion pumps and have to be acknowledged by medical staff. In this environment, CDSSs potentially add an additional annoying source of alerts. CDSSs supporting medication safety, for example, are triggered by computerized drug order entry and indicate risk situations such as drug–drug interactions, contraindications, required drug dose adjustment, duplicate orders, allergic reactions or inappropriateness of certain drugs in the elderly [31].

However, to establish appropriate CDSSs for medication safety in an ED environment remains difficult. Most importantly, decisions must be made fast, often based on incomplete information such as accompanying diseases (possible contraindications) and unknown drug therapy (current medication in a comatose patient). Therefore, available drug databases may be insufficient to consider the complex ED situation, thus generating medication safety alerts of low relevance, low priority or even inappropriate alerts, e.g. concerning systemic drug reactions when a topical drug is applied [15]. In addition, it has been reported that CDSS and CPOE systems may produce, e.g. inaccurate drug dose suggestions [32]. Overloading a busy ED environment with such additional alerts results not only in low acceptance and alert over-riding, but may endanger patients when the treatment process is slowed down or when other essential alarms will be missed. Van der Sijs et al. report over-riding of, for example, drug–drug interaction alerts in 49% to 96% of cases [15]. Seidling et al. proposed forced acknowledgement of alerts as a corrective, which in turn may result not only in deterioration of user satisfaction, but also in deactivating alarm functions [33]. Therefore, Riedmann et al. proposed a context model for mandatory alerting in order to prioritize medication safety alerts [34, 35]. As an alternative, authors such as Cimino et al. implemented non-mandatory information sources of low intrusiveness in the shape of infobuttons for context-specific access to health knowledge and were able to report over 70% user satisfaction [36].

Design criteria to improve alerting

Improved medication safety has continuously been in the focus for CDSS functions [4, 37, 38]. Technologies such as infobuttons [36], computerized alerting [39, 40] and evidence-based electronic recommendations [5] have been proposed. Several publications reported improved process quality. Vardi et al., for example were able to eliminate some medication errors and to reduce time for medication documentation significantly [41]. Galanter et al. developed drug alerts where the user was asked to add an appropriate diagnosis to the problem list and were able to measure improved user compliance [42]. Tamblyn et al. even presented improved patient outcome when implementing an approach to identify clinically meaningful alerts and quantify the risk associated with an alert [43]. Such success is mandatorily based on high user acceptance and user satisfaction with the CDSS. In this context Phansalkar et al. defined recommendations for alert design based on the human factor principles and emphasized the importance of alert modality, alert presentation, design of accompanying alerts and the setting in which the alert is implemented [44].

Based on these recommendations Seidling et al. extracted measurable factors and evaluated their impact upon users' acceptance of drug–drug interaction alerts, thus identifying knowledge quality and textual information as the two significant factors [33]. In their context, knowledge quality refers to correctness of alert text and textual information refers to good textual representation. For the implementation of those significant factors the work of Shah et al. provides some solutions. They propose a knowledge base to filter highly severe drug alerts with the target to minimize work flow interrupts in the ambulatory care setting [45].

Other approaches vary from the design of event monitoring [46], contextualization of alerts [47], patient-specific risk estimates for the likelihood of harm [43], to shutdown of frequently over-ridden drug alerts [48]. Recently, Sirajuddin et al. [22] proposed a five rights framework for best practices in interventional CDSSs for medication management safety: ‘the interventions must deliver the right information, to the right person, in the right format, through the right channel, at the right point in workflow’.

The 4SDRI as a potential approach

We implemented our interventions based on the premise that user acceptance is higher when all computerized interventions are optional and can be ignored. Generally, any disrupting interference with a busy ED environment should be omitted, taking the view that mandatory CPOE and CDSS acknowledgement could potentially even harm patients due to waste of time [8, 49, 50]. To improve the knowledge quality of system alerts we focused on drugs identified as frequently causing medication safety problems in this ED environment. Similar to a selective knowledge base approach of significant alerts by Shah et al. [45] we replenished all textual information according to the presumed alert priority. Additional information is available on demand only, resulting in a step-by-step approach for in depth information seeking called the 4SDRI. Figure 4 illustrates the 4SDRI and the mapping to the five rights framework of Sirajuddin, Osheroff et al. [22]).

Figure 4.

The 4SDRI model and the five rights principle by Sirajuddin et al. [22]

In our case the key information about the patient's status, cardinal symptoms, age and gender were present in an electronic case sheet (ECS) for order entry at admission stage. Laboratory information to assess renal impairment was available from the electronic patient record. Patient admission was defined as the right time in workflow and the ECS directed towards the physician seeing the ED patient was deemed the appropriate channel. A multi-tier application with the option to retrieve additional information if required was considered the optimal intervention format. Sequential arrangement of steps starts with an active plausibility check if the drug(s) is still justified and needed. In step II known drug risks of the ED are displayed in tooltips to inform about harmful contraindications. Next the physician may start a medication check CDSS based on patient information in step III and retrieve detailed drug information from the SPC in step IV.

The 4SDRI model is transferable to all clinical environments where computerized prescribing is established and valid data on previous medication risks are available to adjust step II. The second prerequisite is the ability to invoke a third party drug information system or CDSS and transmit patient data to check medication safety.

Which amount of system support is suitable?

As mentioned, use of the 4SDRI interventions is optional. At first, we were astonished by the low usage of some intervention steps. The tooltips in step II were used in 48% of the ordered drugs. This indicates that not all drugs known to have previously caused problems in this ED have been checked consistently. Considering a previous study phase where the same physicians had already been trained with education and checklists to watch for drug risk situations we can assume some training effects which may account for this fact. This study does not permit to confirm or deny if this was an appropriate rate of use, because we do not have a gold standard which amount of decision support may be required. However, we do know from literature, that other CDSSs produced overalerting effects. Previous enhanced teaching for known risk situations may render digital information seeking less essential. A decreasing use of deeper levels of detailed information in step III and IV is to be expected and is consistent with the subjective self reporting of physicians who put most weight on step II which has been regularly used by all physicians. It is certainly remarkable, that a confirmation that the indication for a drug is still appropriate (step I) was used so rarely (3% of recorded drugs). On the other hand, this checkbox requires user activity for any ordered drug and it may be that physicians acted cautiously when they were not sure about the valid indication. Documented valid indications for all ordered activities will gain more and more relevance not only for drug application (which can, from a legal viewpoint, be regarded as bodily impairment subject to penalty if not appropriate) but for all other invasive activities performed on a patient. Therefore we think that we will see in future even more detailed documentation requirements as to why a drug was administered. Further research about the appropriate amount of alerts and interventions is certainly necessary. We hope, at a later stage, to be able to contrast our results with patient outcome data in terms of observed drug risk situations.

Technology acceptance and system use do not match

In contrast to the logged use of 4SDRI components, the computer-assisted interventions received positive acceptance in the TAM2 model with regard to usefulness, support of work and information quality and for delivering tangible results which were all rated above four in a five point Likert scale. It is interesting to see that for most questions, the 4SDRI interventions tend to be rated better than previous paper based interventions (checklist and poster). The 4SDRI fit to work flow was 3.5 (mean would have been 3). Intention to use (when overcoming barriers) was scored with 4.4 on average which is exceptional considering the limited use observed in the logs. These results indicate that physicians appreciate the 4SDRI approach and maybe use it as often as they deem necessary.

‘Disallowance of order entry (hard stops) is unacceptable in the opinion of the authors because decision support cannot replace the physicians' responsibility for the treatment of the patient …’ [15]

This citation reflects the nature of clinical decision support which is never meant to replace, but to support the physician in his/her tasks. The 4SDRI adopts this approach by offering stepwise support as needed and received positive ratings regarding user acceptance. Some intervention steps were seldom used, others quite often. We think that using 4SDRI we have been able to avoid over-alerting and workflow intrusion in a critical ED environment. Nevertheless we need more research to assess when and how often decision support or context specific knowledge on demand is required in a given environment. This research must consider all other parallel supportive measures such as training and education on the job. The success of CDSS implementation depends on many facets including staff training, appropriate workflow adaption, reliability of medication safety alerts and team leadership [51]. We have been able to transfer the 4SDRI approach to the new ED information system eCare® which was introduced to the Fürth ED in 2012 and we are confident that transfer of the 4SDRI model to other environments can also be achieved.

Competing Interests

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare TB, HD and RM had support from German Ministry of Health for the submitted work; For the duration of the study, the applications Ontodrug® and Pharmindex Plus® were provided free of charge from UBM Medica. There are no other relationships or activities that could appear to have influenced the submitted paper.

This research project was supported by The German Federal Ministry of Health within the ‘German Coalition for Patient Safety’ (http://www.aktionsbuendnis-patientensicherheit.de/) by a BMG grant II A 5 - 2509 ATS 003 to HD, RM and TB.