Computerized Pharmaceutical Algorithm Reduces Medication Administration Errors During Simulated Resuscitations

Abstract

OBJECTIVE

Medication errors involving intravenous medications continue to be a significant problem, particularly in the pediatric population due to the high rate of point-of-care and weight-adjusted dosing. The pharmaceutical algorithm computerized calculator (pac2) assists in converting physician medication orders to correct volumes and rates of administration for intravenous medications. This study was designed to assess the efficacy of the pac2 in simulated clinical scenarios of point-of-care dosing.

Methods

The study design was a within-subject controlled study in which 33 nurses from pediatrics, pediatric critical care, or critical care (mean nursing experience of 10.9 years) carried out various point-of-care medication-dosing scenarios with and without the aid of the pac2.

RESULTS

Use of the pac2 resulted in a significantly higher percentage (mean [95% CI]) of medication volumes calculated and drawn accurately (91% [87–95%] versus 61% [52–70%], p<0.0001), a higher percentage of correct recall of essential medication information (97% [95–99%] versus 45% [36–53%], p<0.0001), and better recognition of unsafe doses (93% [87–99%] versus 19% [12–27%], p<0.0001) as compared to usual practice. The pac2 also significantly reduced average medication calculation times (1.5 minutes [1.3–1.7 minutes] versus 1.9 minutes [1.6–2.2 minutes], p=0.0028) as compared to usual practice.

CONCLUSIONS

The pac2 significantly improved the performance of drug calculations by pediatric and critical care nurses during simulated clinical scenarios designed to mimic point-of-care dosing. These results suggest that the pac2 addresses an area of safety vulnerability for point-of-care dosing practices and could be a useful addition to a hospital's overall program to minimize medication errors.

INTRODUCTION

Medication errors are a serious safety problem in hospitalized patients. The American Academy of Pediatrics Committee on Drugs and Committee on Hospital Care defined a medication error as any preventable event that occurs in the process of ordering or delivering a medication, regardless of whether an injury occurred or the potential for injury was present.¹ The reported incidence of medication errors in hospitalized adults ranges from less than 1% to as high as 56% of all medications administered with the estimation that medication errors occur in 6% of all adult patients.² The reported rates of medication errors in pediatric patients are often higher with error rates as high as 1 per 6.8 (14.7%) of children admitted to a Neonatal or Pediatric In tensive Care Unit (PICU).³ Takata et al. reported an adverse medication event rate of 11.1 per 100 patients in 12 children's hospitals in the United States.⁴ The potential impact of such medication errors in the pediatric population is especially worrisome since medication errors in pediatric patients are three times more likely to result in a clinically important adverse event.⁵ Incorrect dosing is the most commonly reported medication error in pediatrics including errors in the computation of dosage and dosing interval.^{4, 6–10} Pediatric studies have noted that the majority of medication errors occur either during the prescribing^11,12 or administration stage.^3,9,13,14

The unstable infant or child in an emergency department or ICU requires immediate pointof-care medications. Figure 1 is an outline of a medication administration sequence, with interdisciplinary responsibilities, for point-of-care medication dosing in hospitalized patients. The figure demonstrates the critical and complex role of the bedside nurse in this process. In addition to medication administration, the nurse is often responsible for traditional pharmacy activities including preparation, dispensing, and recognition of incorrect doses.

Figure 1.

Typical Process and Individual Responsibilities for Point-of-Care Medication Dosing of Hospitalized Patients

An example of a point-of-care administration sequence involves the acute resuscitation of child with shock. The physician orders a medication in terms of dose/kg/time (dopamine 10 mcg/kg/ min by continuous infusion or epinephrine 0.01 mg/kg IV push). The bedside nurse must then quickly and correctly translate the physician's order into an intravenous flow rate (volume/ time) taking into account the patient's weight, the concentration of the available medication in the solution, and the ordered dose. It is estimated that nearly 20% of all hospital medications are dispensed at the point-of-care without first being measured in the hospital pharmacy.¹⁵ Nursing staff must be proficient in performing these mathematical calculations. There have been multiple studies over the past three decades which have documented an unacceptably high calculation error rate by practicing nurses.^16–19

The Pharmaceutical Algorithm Computerized Calculator (pac2) was conceptualized to assist with calculations for point-of-care dosing. The pac2 automatically performs all unit conversions (e.g., lb to kg or mg to mcg) while calculating the correct infusion rate (volume per time) based upon the concentration of the medication solution, the child's weight, and the physician's prescribed dose (Figure 2). This handheld computer captures common entry errors such as misplaced decimal points and improper expression factors. The pac2 can be custom programmed to provide an immediate visual warning if the input medication order falls outside the hospital's established limits for the prescribed medication, a given age range (e.g., adult versus pediatric), or care unit. The current study prospectively evaluates the efficacy of the pac2 in reducing medication-dosing errors when practicing pediatric and critical care nurses prepared and administered intravenous medications during simulated resuscitations of critically ill children.

Figure 2.

. Screen of the Pharmaceutical Algorithm Computerized Calculator.

METHODS

Study Participants

Nurses from three different institutions volunteered to participate in the study. Participants needed to be a registered nurse, involved in direct patient care, and currently working in a pediatric, pediatric critical care, or critical care setting. Those nurses participating were willing to comply with the requirements of the protocol. The study was approved by the local institutional review board.

Two separate groups of nurses were enrolled for participation in the study: the full study group and the learning effect group. The full study group included all nurses that were enrolled into the study with the intent of performing both sets of scenarios, first without and then with the pac2. The learning effect group consisted of a subset of different nurses who were selected to reflect the demographic/experience profile of the full study group. This group only performed one set of scenarios using the pac2. The intent of this group was to assess the potential learning bias in the full study group from the consistent usage of the pac2 in the second set of scenarios. It was reasonable to expect the full study group's pac2 results to outperform the learning effect group's pac2 results if a learning bias existed. The possibility that prior experience using the pac2 might impact performance existed since some nurses in the full study group had evaluated the pac2 prior to this study and had some limited experience operating the device. Analyses of the full study group minus the nurses with prior experience (i.e., naïve subgroup) were therefore performed.

Study Design

The study design was a within-subject controlled study. The nurses were randomized to perform one of two simulation scenario sets (see below) without the pac2 during the first phase of the study. The calculation of the medications was performed using their usual practice. Usual practice allowed the use of handheld calculators, pre-calculated pediatric emergency drug cards, and other supportive materials typically available at point-of-care dosing. Usual practice was assessed first to control for a learning effect from using the pac2 initially. The nurses then watched a 20-minute instructional tutorial describing the function and operation of the pac2 after completing the first set of scenarios without the pac2. The nurses were allowed to ask questions during and after the tutorial. The nurses were then crossed over to perform the alternative scenario set using the pac2 during the second phase of the study after watching the tutorial. Those nurses who performed scenario set A during the first phase of the study performed scenario set B during the second phase of the study and vice-versa. The performance of each nurse during the two sets of simulated dosing scenarios was evaluated.

Clinical Scenarios

Scenario set A included the case of a pediatric patient with anaphylaxis and impending respiratory arrest and a second pediatric patient with seizures who required a series of three medication doses including one continuous intravenous medication infusion. Scenario set B included a different emergency situation with a pediatric patient with status epilepticus who progressed to respiratory arrest requiring rapid sequence intubation with resulting hypotension and a second pediatric patient in severe pain who required a series of three medication doses including one continuous intravenous medication infusion. Each set of scenarios required the nurses to perform 12 dose volume calculations (i.e., make eight mcg to mg conversions, compute three continuous IV rates and one non-weight based order), recognize five unsafe doses (ten or more times above or below acceptable ranges), and recall seven critical pieces of medication administration information. The scenarios were designed to be of equal degrees of difficulty. Three trained nursing school faculty observers followed a written script to oversee and score the nurse's performance that was recorded on a case report form and measure the time required to complete the calculation task.

Study Endpoints

The primary efficacy endpoint was predefined as the percent of medication volumes calculated and drawn accurately (i.e., ± 10% correct medication volume) into syringes. Some results are presented as the percent error rate (e.g., percent error rate equals 100% minus the percent of medication volumes calculated and drawn accurately). The secondary efficacy endpoints included: percent of medication volumes calculated and stated accurately, percent of unsafe medication doses identified, percent of essential medication administration information recalled, average medication calculation time, and average time to complete the scenario. A composite score was calculated as the unweighted average of percent of volumes calculated and drawn accurately, percent of unsafe doses identified and percent of essential information recalled.

Statistical Analysis

Results are reported as mean (95% CI) percentages. The percentage of accurately drawn volumes with and without the pac2 was determined for each nurse. A paired t-test was used to compare the average percentage of correct volumes without the pac2 to the average percentage of correct volumes with the pac2. A paired t-test was used to compare results with the pac2 (average percentage of accurately stated volumes, percentage of unsafe doses recognized, percentage of essential information recalled, average medication calculation time, average overall scenario time and a composite score) to results without the pac2. The significance of the average paired difference was tested at a=0.05 (SAS 9.1.3, SAS Institute, Inc, Cary, NC).

RESULTS

The study cohort included 39 nurses, 33 of whom participated in the full study group and six in the learning effect group. The two groups were comparable with respect to age, sex, total nursing experience, and specialty nursing experience (Table).

Table 1.

Summary of Subject Demographics

Full Study Group

There was no difference in any of the efficacy endpoints for the naïve subgroup relative to the full study group and therefore the following data includes the 33 nurses who comprised the full study group. Use of the pac2 resulted in a significantly higher rate of medication volumes that were drawn up accurately (91% [87–95%] versus 61% [52–70%]; p < 0.0001), a significantly higher percentage of essential information recalled (97% [95–99%] versus 45% [36–53%]; p < 0.0001), and a significantly higher percentage of recognition of unsafe doses (93% [87–99%] versus 19% [12–27%]; p < 0.0001) as compared to usual practice (Figure 3). These findings were consistent irrespective of the order of the scenario sets (A or B) and were consistent across all nursing subgroups (pediatric, critical care or pediatric critical care). Similar results were observed for the endpoint of medication volumes stated accurately.

Figure 3.

Comparison of Outcome Variables for Simulated Scenario Sets with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, ▪ With PAC2, Percent correct response with 95% confidence intervals, ^* = Significance of paired difference, p<0.0001

Use of the pac2 resulted in a significantly higher composite score (94% [91–97%] versus 42% [37–46%]; p < 0.0001) as compared to usual practice (Figure 3). The average composite error rate without the pac2 was 58% compared to 6% with the pac2. All three of the setting subgroups (pediatric, critical care, or pediatric critical care) demonstrated a significant increase in the correct composite score with the pac2.

Use of the pac2 resulted in faster medication calculation time (1.5 minutes [1.3–1.7 minutes] versus 1.9 minutes [1.6–2.2 minutes], p = 0.0028) as compared to usual practice (Figure 4). Although a faster calculation time was observed in all nursing subgroups, it was not consistent across both randomized groups. Those that had the A scenarios first showed no difference while those that had the B scenarios first did. This finding may be reflective of the difference in the three medications that were selected for timing from each scenario set.

Figure 4.

Average Drug Calculation Times (minutes) with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, ▪ With PAC2, Average drug calculation time (minutes) with 95% confidence intervals, * = Significance of paired difference

The pediatric critical care subgroup of nurses performed the medication calculations most quickly with no significant difference between using the pac2 or not. However, both of the other nursing subgroups performed comparably and demonstrated a significant decrease in medication calculation time for the scenarios with pac2. All three nursing subgroups were comparable in medication calculation time with the pac2, but the pediatric critical care subgroup tended to be quicker without the pac2. The nurses also completed the scenario sets significantly faster with the pac2 as compared to usual practice (19.4 minutes [17.9–21.0 minutes] versus 22.9 minutes [20.8–25.0 minutes]; p = 0.0012). This finding was consistent irrespective of the order of scenario sets (A or B) and was consistent across all nursing subgroups.

Learning Effect Group

There was no evidence that participation in the simulations alone during the first phase of the study resulted in a learning effect that would carry over to improved performance when using the pac2. The learning effect group, which only used the pac2 without prior completion of a simulation, demonstrated higher percentages of accurately drawn volumes (93% [86–100%]), accurately stated volumes (93% [86–100%]), essential information recalled (100% recalled; no variation among subgroup), and recognition of unsafe doses (97% [88–100%]) as compared to the nurses evaluated without and with the pac2 in the full study group. The learning effect group also completed the scenarios more quickly (15.0 minutes [12.2–17.8 minutes]) than the full study group (19.4 minutes [17.9–21.0 minutes]).

DISCUSSION

Many process changes and newer technologies have been reported to reduce medication errors rates. Ward-based clinical pharmacists,⁶ individual unit-dosing, computerized physician order entry,^20,21 preprinted order sheets,²² standard infusion concentrations plus “smart-pump” technology,²³ and a computer-based medication and equipment calculator²⁴ have been reported to reduce medication errors. However, the computer-based medication and equipment calculator was the only safety measure/device that attempted to address administration errors that occur during point-of-care dosing during resuscitations. Shannon and colleagues²⁴ tested a computerized calculator for accuracy and quickness compared to conventional paper-based calculation methods. Subjects made significantly fewer errors and were three times faster in performing theoretical resuscitation calculations for pediatric and adult patients with this resuscitation calculator. Similarly, the pac2 also improved accuracy and reduced the average medication calculation time and overall time to complete the scenario as compared to usual practice. Some of the potential advantages of the pac2 versus the computerized calculator include the portability of the handheld pac2, the large number of medications stored (285 vs. 21) including non-resuscitation medication infusions, the availability of essential medication information, and customized warnings for doses outside the institution's typical ranges. Advantages of the computerized resuscitation calculator include the feature that all 21 medication doses are calculated simultaneously, with increased ease of use, and likely a lower cost.

The only other process/device demonstrated to reduce medication administration errors at pointof-care was a combination of standard medication concentrations, “smart-pump” technology, and user-friendly labels. Larsen and colleagues²³ demonstrated that the combination of these features significantly reduced continuous medication infusion errors. The pac2 is complementary to the “smart-pump” because it has distinctive features not shared with this technology, including dose calculation of manual intravenous, intramuscular, and subcutaneous injections and dose volume calculations for point-of-care medication liquids, elixirs, and suspensions. The error rate in medication volumes observed in our study was seemingly high at 39%, but is supported by other published investigations. Bayne and Bindler tested 67 practicing nurses with a 20-item medication calculation test and found that the average correct score was 75.5%.¹⁸ In a study of 546 nurses working at a university hospital in Finland, only 56% answered the “volume of solution” dosage calculation correctly.¹⁹ Bindler and Bayne documented a 48% error rate for intravenous calculation questions among a sample of 110 nurses.²⁵ Kozer and colleagues found a 20% error rate in 16% (9/58) of syringes examined after eight mock resuscitations.²⁶ They concluded that many administration errors could only be detected by analyzing the syringe content. Combining the added “stress” of a simulated resuscitation plus a predefined error rate of ± 10% likely contributed to our seemingly high medication volume error rate of 39%.

The major limitation of this study is that it was conducted under simulated conditions rather than actual clinical conditions where other factors may influence performance and outcomes. The increased mortality reported by Han and colleagues after the implementation of a computerized physician order entry system serves as a good example.²⁷ Some factors attributed to this negative outcome included significant changes in clinical workflow, communication among providers, and distribution of responsibilities.²⁸ Effectiveness of the pac2 may not be susceptible to these factors in actual clinical practice but other factors that may influence the impact of the pac2 include greater time pressure and stress during acute resuscitations, increased susceptibility of severely ill children to medication errors,^6,11 and unpredictable logistical problems that arise during emergency situations. Simulation studies are commonly used in medicine for educational and training purposes,²⁹ and the controlled environment of a simulation laboratory was appropriate for this initial study to assess the efficacy of the pac2 by nursing staff. Our simulation study specifically did not include the presence of a clinical pharmacist as we wished to mimic emergency resuscitations in which nurses need to function without this benefit. Another limitation of the study is that it was not possible to blind the observers regarding use of the pac2. However, the only outcome measure directly controlled by the observers was timing of the exercise. There are certain types of medication administration errors that were not and cannot be addressed by the pac2. These errors include missed doses, prescribing the wrong medication, and administration at the wrong time. Finally, calculating medication doses based on a patient's weight is not usual practice for adult critical care nurses. Interestingly, the medication calculation times were not significantly different across the three nursing subgroups.

This study provides the first controlled evaluation of a novel device that addresses an area of safety vulnerability in point-of-care dosing. Our results demonstrated the reduction in medication calculation errors that can be obtained by nurses using a pac2 in a simulated pediatric patient care setting. Use of the pac2 device reduced the volume calculation error rate by nurses from 39% to 9%, allowed the nurses to recognize unsafe doses more frequently, and recall essential medication information more effectively than during performance without the pac2. These improvements were consistently observed from nurses across pediatric, critical care and pediatric critical care settings (with the exception of medication calculation time). Future research to collect information under conditions of actual use in clinical practice would be helpful to define the full range and precise role of this device in improving the accuracy of dosing within the hospital.

ABBREVIATIONS

pac2

Pharmaceutical Algorithm Computerized Calculator

PICU

Pediatric Intensive Care Unit