The impact of paediatric dose range checking software

Abstract

Objective

Dosing errors can cause significant harm in paediatric healthcare settings. Our objective was to investigate the effects of paediatric dose range checking (DRC) clinical decision support (CDS) software on overdosing-related outcomes.

Methods

A before-after study and a semistructured survey of prescribers was conducted across inpatient wards (excluding intensive care) in a regional children’s hospital. DRC CDS software linked to a paediatric drug formulary was integrated into an existing electronic prescribing system. The main outcome measures were; the proportion of prescriptions with overdosing errors; overdosing-related clinical incidents; severity of clinical incidents; and acceptability of the intervention.

Results

The prescription overdosing error rate did not change significantly following the introduction of DRC CDS software: in the preintervention period 12/847 (1.4%) prescriptions resulted in prescription errors and in the postintervention period there were 9/684 (1.3%) prescription overdosing errors (n=21, Pearson χ² value=0.028, p=0.868). However, there was a significant trend towards a reduction in the severity of harm associated with reported overdosing incidents (n=60, Mann-Whitney U value=301.0, p=0.012). Prescribers reported that the intervention was beneficial and they were also able to identify factors that may have contributed to the persistence of overdosing errors.

Conclusion

DRC CDS software did not reduce the incidence of prescription overdosing errors in a paediatric hospital setting but the level of harm associated with the overdosing errors may have been reduced. Use of the software seemed to be safe and it was perceived to be beneficial by prescribers.

Introduction

Medication dosing errors cause significant harm in paediatric healthcare settings.^1–4 An estimated 21.8% of children experience a dosing error at some point during an inpatient admission⁵ and these errors can cause serious morbidity and mortality.^{6 7}

Health information technologies have been proposed as a means of minimising the risks associated with paediatric prescribing. One potential advantage of using electronic prescribing technologies is that these tools can incorporate clinical decision support (CDS) systems. These technologies use the automated analysis of databases to match individual patient characteristics with relevant advice from treatment guidelines.⁸ This advice is presented to users in the form of a notification, message or alert. Dose range checking (DRC) CDS systems may reduce harm by alerting healthcare professionals to potential dosing errors before medications are administered.

In adults, DRC CDS systems can improve prescribing outcomes. A Cochrane review found that using DRC CDS software improved adherence to dosing guidelines for certain antibiotics and anticoagulant medications, thus reducing the risks of nephrotoxicity, thrombotic episodes and bleeding events.⁹

Previous paediatric studies of DRC CDS interventions have either been conducted in limited populations (for example in neonatal or critical care units) or have only been implemented for a limited range of medications. These studies have delivered inconsistent results and it is uncertain if DRC CDS software has any effect on dosing errors in paediatric settings.^10–13

Given the limited evidence, our aim was to investigate the effects of using a more comprehensive DRC CDS software intervention, across a mixed range of wards, in a paediatric hospital. Our objectives were to investigate the effects of using this software using a before-after study design. Specifically we sought to investigate the effects of using DRC CDS on the rate of prescriptions that included overdosing errors and the number of clinical incidents (adverse events) associated with medication overdosing errors. We also aimed to investigate whether prescribers found the software acceptable to use, to identify any unintended consequences of using the DRC CDS intervention, and to gather qualitative data about users’ perceptions of factors that may have contributed to its effects.

Methods

We followed existing recommendations for evaluating the effects of health information technologies by using a mixed-methods approach.^14–16 The effects of the intervention were evaluated using a before-after design and the evaluation of acceptability was completed using a cross-sectional survey. An evaluation protocol was approved by the Alder Hey Children’s NHS Foundation Trust Quality and Governance department in December 2018 and is available within the online supplementary data file.

Setting and participants

The evaluation was conducted within a regional specialist children’s hospital in the UK. At the time of conducting the evaluation, medication errors were the most frequently reported form of clinical incident in the hospital. Previously identified overdosing incidents included 10 times errors of potentially harmful medications including opioids, heparin and phenobarbitone.

Previous audits had identified an overall prescription error rate of 10.3% (including omissions, formulation and timing errors) and a prescription overdosing error rate of 0.89%. Antibacterial and analgesic drugs were identified as being most frequently associated with medication errors.

The before-after study used prescribing and incident reporting data from every inpatient ward in the hospital (excluding the paediatric intensive care unit but including general paediatric, surgical, neonatal, oncology and paediatric subspeciality wards) between March 2018 and March 2019.

The acceptability survey was carried out with a mixed sample of prescribers from the hospital in June 2019.

Intervention

In September 2018 the DRC CDS software was integrated into an existing electronic health record (EHR) electronic prescribing system (MEDITECH V.6.08, Massachusetts, USA). The DRC alerts cross-referenced patient-specific parameters recorded within the EHR (age, weight, body surface area) with the recommended dosing parameters for 1227/2003 (61.3% of total formulary items) paediatric medication formulations. The dosing recommendations were either based on those detailed in the British National Formulary for Children (BNFc)¹⁷ or were configured to reflect local dosing guidelines. The dosing database and DRC software was provided by First Databank Europe.

Alerts were generated when healthcare providers attempted to prescribe a dose of a drug that was above the recommended limits. The alerts were configured with a 5% margin for rounding. This margin was agreed as a safe limit that would allow health professionals to round up doses in order to prescribe measurable volumes of medications (alerts were provided for maximum single doses and maximum daily doses) (see figure 1.). The alerts could be overridden by providing a clinical justification.

Figure 1.

Schematic diagram demonstrating a dose range checking alert and options for manually overriding the notice within the MEDITECH V.6.08 order entry module.

The DRC CDS intervention was implemented alongside existing and continuing methods for preventing overdosing errors. These included prescription reviews led by the ward-based clinical pharmacists and a policy of secondary dose checking prior to administration of newly prescribed medications.

Outcomes

The primary study outcome was the change in the proportion of prescriptions that included medication overdoses following the introduction of the DRC CDS software.

Secondary outcomes included:

• Changes in the severity of overdosing prescription errors.

• Changes in the numbers of overdosing error-related incident reports.

• Changes in the number of overdosing error-related incident reports resulting in episodes of harm.

• The acceptability of the DRC software to prescribers.

• Factors perceived to have contributed to the failure of the DRC to prevent dosing prescription errors.

• Adverse events associated with the introduction of the DRC software.

Data sources

Overdosing error rate

Routinely collected hospital data were used to identify overdosing errors. These data were collected on a monthly basis by the ward-based clinical pharmacists, using a published method for identifying prescription errors.¹⁸ At the end of the evaluation period, data from March 2018 to August 2018 (preintervention period) and September 2018 to February 2019 (postintervention period) were reviewed by an investigator who was blinded to the initial data collection date. The blinded investigator validated the prescription overdosing errors that had been identified by the pharmacy team and graded each error according to a published Severity Classification Scheme (identifying error as either minor, significant, serious or potentially lethal errors).¹⁸ Prescription overdoses were validated by comparing the prescribed dose to the maximum recommended dosing value in the BNFc or local prescribing protocols (based on the available age/weight/body surface area data for that patient).

Overdosing-related clinical incidents

Data from the hospital’s voluntary incident reporting system were analysed to identify any incidents (including ‘near-miss’ incidents) that were categorised as relating to dosing prescription errors during the evaluation period. The level of harm caused by each incident was recorded by the professional who initially reported the incident.

Acceptability

A semistructured questionnaire (see online supplementary data file) and purposive sampling were used to gather data relating to the acceptability and usability of the DRC CDS system.

Adverse events

Adverse events associated with the introduction of the DRC software were identified by reviewing clinical incident reports, through active monitoring of the system by the hospital’s lead informatics pharmacist and by including questions relating to adverse events in the questionnaire provided to prescribers.

Statistical methods

We used descriptive statistics to compare the dosing error rates, number of dosing incidents and severity of harm caused by dosing errors. The significance of differences in nominal data were tested using Pearson χ² and Fisher’s exact tests.

In the study protocol we planned to use an interrupted time series model to determine the significance of any changes in dosing error rates following the introduction of the DRC CDS software. This approach was modified due to the low overall rate of dosing errors and due to missing data at some time points (due to incomplete collection of routine data). No patterns of missing data were identified using a visual inspection method.

Ethics

Use of the Health Research Authority Decision Tool determined that the before-after investigation qualified as a service evaluation and did not require NHS Research Ethics Service approval.

Results

During the course of the evaluation 268 415 electronic prescriptions were completed (131 612 during the 6-month preintervention period and 136 803 during the 6-month intervention period).

Overdosing error rate

There was no significant change in the prescription overdosing error rate between the preintervention and postintervention periods. A total of 1552 medication orders were reviewed to identify prescription errors. There were overdosing errors in 12/847 (1.4%) prescriptions in the preintervention period and 9/684 (1.3%) prescriptions in the postintervention period (n=21, Pearson χ² value=0.028, p=0.868).

There was no significant change in the severity of prescription overdosing errors that were identified by the pharmacy team. In the preintervention period 8/12 (66.7%) errors were minor and 4/12 (33.3%) were significant. In the postintervention period 7/9 (77.8%) errors were minor and 2/9 (22.2%) were significant. These differences were not statistically significant (Fisher’s exact test n=21, p=0.659).

Overdosing-related clinical incidents

During the evaluation period, 60 clinical incidents involving electronic prescriptions and overdosing errors were reported by clinical staff. Twenty-eight incidents were reported in the preintervention period and 32 were reported in the postintervention period.

Following the introduction of the DRC CDS software there was a significant trend towards a reduction in the severity of reported overdosing incidents (n=60, Mann-Whitney U value=301.0, p=0.012). In the preintervention period 1/28 (3.6%) incident resulted in minor harm, 20/28 (71.4%) incidents were classified as ‘near misses’ and 7/28 (25.0%) incidents were categorised as resulting in ‘no harm’. In the postintervention period 15/32 (46.9%) incidents were classified as ‘near misses’ and 17/32 (53.1%) were categorised as resulting in ‘no harm’ (results summarised in table 1).

Table 1.

Summary of prescription overdosing error data and severity of reported overdosing-related clinical incidents

	Preintervention	Postintervention
Number of electronic prescriptions completed	131612	136 803
Incidence of overdosing errors	12/847 (1.4%)	9/684 (1.3%)
Proportion of overdosing errors classified as minor errors	8/12 (66.7%)	7/9 (77.8%)
Proportion of overdosing errors classified as significant errors	4/12 (33.3%)	2/9 (22.2%)
Total overdosing-related clinical incidents	28	32
Proportion of overdosing incidents resulting in ‘No Harm’	7/28 (25.0%)	17/32 (53.1%)
Proportion of overdosing incidents resulting in ‘Near Misses’	20/28 (71.4%)	15/32 (46.9%)
Proportion of overdosing incidents resulting in ‘Minor Harm’	1/28 (3.6%)	0/32 (0.0 %)

Acceptability and user perceptions

Acceptability and user perception questionnaires were completed by 44 prescribers. The respondents included trainee doctors (n=23), consultants (n=11), specialist nurse prescribers (n=9) and a prescribing pharmacist (n=1) (see table 2). Participants had a median of 7 years of paediatric prescribing experience (range 0.17–26 years).

Table 2.

Roles of respondents to acceptability survey

Prescriber role	Number of respondents (%)
Nurse specialist	9 (20.5)
Trainee doctor	23 (52.3)
Consultant (attending)	11 (25)
Pharmacist	1 (2.3)

Since the introduction of the DRC CDS software, the majority of prescribers (n=34/44, 77.3%) reported having experienced a DRC alert. A significant minority of prescribers (n=14/44, 31.8%) also reported that they had made a dosing error (involving a completed prescription) following implementation of the DRC CDS software. However, all of these prescribers reported being unaware (n=11/14 (78.6%)) or ‘not sure’ (n=3/14 (21.4%)) if they had overridden an overdosing alert while completing the prescription.

Using Likert Scale items, the majority of respondents indicated that the DRC CDS software had prevented them from making overdosing errors (n=32/44, 72.7%) and that the software had improved the quality of their prescribing (n=28/44, 63.6%). Some respondents provided reports that the software had prevented overdosing errors (see figure 2).

Figure 2.

Examples of responses from prescribers that indicate their experiences of using the DRC CDS software. BNFc, British National Formulary for Children; CDS, clinical decision support; DRC, dose range checking.

Respondents also provided reflections on factors that they thought they may have contributed to overdosing errors that occurred despite the introduction of the DRC CDS software. These included ‘alert fatigue’, failure to account for an infant’s corrected gestational age and alert messages that were ‘wordy’ and difficult to interpret (see figure 2.). Prescribers also highlighted that the software was not effective when a patient’s weight had not been entered into the electronic patient record.

Adverse events

One survey respondent reported that less experienced prescribers had become more cautious about intentionally prescribing doses of medications that were above the recommended limits. They were concerned that this may have delayed the prescribing of some medications while the doses were checked with supervising consultants. Following the implementation of the DRC CDS system the hospital pharmacy team also identified discrepancies that resulted in appropriate dosing alerts failing to be generated for three medications. None of these discrepancies were associated with overdosing incidents and were all reported to the suppliers for correction.

Discussion

To our knowledge, this is the first study to investigate the effects of using a DRC CDS intervention, covering the majority of paediatric formulary items, across a range of specialised and general paediatric wards. There was no evidence that the use of DRC CDS software reduced the frequency of overdosing errors, or reduced the likelihood of clinical incidents involving overdosing errors. However, the introduction of the software was associated with a statistically significant trend towards a reduction in the severity of harm associated with reported overdosing incidents.

Prescribers who used the DRC CDS software reported that it may have prevented them from making overdosing errors. They also tended to perceive that it was beneficial to their prescribing practice. In addition, prescribers reported that clearer alerts and a reduction in ‘false positive’ alerts may have helped to improve the effectiveness of the software.

In keeping with recommended approaches for conducting evaluations of health information technologies we used a mixed-methods design to evaluate the software.^14–16 We believe that this approach has enabled us to identify important contextual factors that help to explain the observed effects and will help to inform future iterations and evaluations of the DRC software.

One limitation of this evaluation is that we used an uncontrolled before-after study design. This was due to a lower than anticipated baseline of overdosing errors and some missing data points, which prevented us from completing the planned interrupted time series study. However these findings do suggest that future studies of the effects of the DRC CDS software should be powered to detect small differences in overdosing error rates. An additional limitation is that incident reporting data were provided by hospital staff on a voluntary basis and so may not accurately reflect the total number of overdosing-related clinical incidents.

We suggest that this study provides a useful resource for decision makers considering the introduction of DRC CDS software into paediatric hospital settings. It provides evidence that use of this software does not reliably prevent overdosing errors and that appropriate training and prescription checking processes should always continue alongside these interventions. It also provides evidence that use of DRC CDS systems seems to be safe and may be perceived to be beneficial by prescribers. Implementations of DRC CDS systems should therefore be accompanied by a careful consideration of the associated costs, including capital expenditure and maintenance and configuration of the system.

Conclusion

The DRC CDS software did not reduce the incidence of prescription overdosing errors in a paediatric hospital setting, but the level of harm associated with overdosing errors may have been reduced. A number of factors including alert fatigue and the clarity of overdosing alerts may have limited the effectiveness of the intervention. Despite these limitations, use of the software seemed to be safe and it was perceived to be beneficial by prescribers.

What this paper adds.

What is already known on this subject

• Dose range checking (DRC) software has improved prescribing outcomes in some adult settings.

• Previous paediatric studies of DRC software have demonstrated inconsistent results.

What this study adds

• Use of paediatric DRC software did not prevent or significantly reduce the incidence of overdosing errors.

• Use of paediatric DRC software may have reduced the severity of harm associated with overdosing errors but overdosing alerts need to be clear and well targeted to be most effective.

Data availability statement

Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information. All attempts have been made to include data relevant to the study in the supplementary data file. Any reasonable requests for access to additional data will be considered. Please contact Dr MN; ORCID ID https://orcid.org/0000-0001-6069-2940.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

Ethics approval for the acceptability evaluation was provided by the University of Liverpool Research Ethics Committee (Reference Number 4694, approved 15/05/2019).