Effect of a voice recognition system on pediatric outpatient medication errors at a tertiary healthcare facility in Kenya

Abstract

Background:

Medication-related errors account for one out of every 131 outpatient deaths, and one out of 854 inpatient deaths. The risk is threefold greater in the pediatric population. In sub-Saharan Africa, research on medication-related errors has been obscured by other health priorities and poor recognition of harm attributable to such errors.

Our primary objective was to assess the effect of introduction of a voice recognition system (VRS) on the prevalence of medication errors. The secondary objective was to describe characteristics of observed medication errors and determine acceptability of VRS by clinical service providers.

Methods:

This was a before–after intervention study carried out in a Pediatric Accident and Emergency Department of a private not-for-profit tertiary referral hospital in Kenya.

Results:

A total of 1196 handwritten prescription records were examined in the pre-VRS phase and 501 in the VRS phase. In the pre-VRS phase, 74.3% of the prescriptions (889 of 1196) had identifiable errors compared with 65.7% in the VRS phase (329 of 501).

More than half (58%) of participating clinical service providers expressed preference for VRS prescriptions compared with handwritten prescriptions.

Conclusions:

VRS reduces medication prescription errors with the greatest effect noted in reduction of incorrect medication dosages. More studies are needed to explore whether more training, user experience and software enhancement would minimize medication errors further. VRS technology is acceptable to physicians and pharmacists at a tertiary care hospital in Kenya.

Introduction

The first Quality Chasm report of the Institute of Medicine’s (IOM), ‘To Err Is Human’ stated that medication-related errors accounted for 1 out of every 131 outpatient deaths and 1 out of 854 inpatient deaths.^1–3 Provision of medication entails prescribing, verifying, dispensing, administering, monitoring and reporting.⁴ Studies estimate 68–75% of adverse drug events occur as a result of mistakes that occur during the prescription stage with illegible handwriting contributing significantly to medication errors.^5,6 Errors may arise from any of the several stages of the prescription process, with the risk being threefold greater for pediatric medications.⁵ Dosage calculation in children is based on age, weight, body surface area or severity of the clinical condition. In addition, children have a narrow therapeutic window and a lower physiological threshold for buffering overdose errors compared with adults. Children also lack capacity to participate effectively in the medication process, thus placing them at greater risk for harm.² The risk of medication errors is higher in outpatient settings where prescribers operating in a stressful work environment are less familiar with patients.

Interventions that have been used to reduce medication errors include enhancing knowledge on medication safety, development of reporting systems and implementation of safety systems in healthcare organizations at the delivery level.^2,7,8 Other interventions include use of tutorials, computerized physician order forms and workbooks.^9–11 Electronic prescription systems have immediate benefits of improving legibility and completeness while eliminating transcription errors.^12–14 In VRS, voice is used to input data into the computer through a microphone. A computer software then converts voice data into text and stores it in a database, facilitating retrieval for subsequent use. Though readily available, VRS use in the medication process is largely unexplored.^15,16 Kang and colleagues used Dragon^® voice application to prepare pathology reports and demonstrated an 81% decrease in average turnaround time and a 48% decrease in the number of errors identified before signing out the report.¹⁵ Proper training followed by practice in VRS use is considered crucial for the success of voice recognition in reducing errors.¹⁷

Medication errors among the pediatric population in sub-Saharan Africa remain largely understudied, hence there are limited data on effectiveness of various strategies to reduce medication errors in this region. The primary aim of this study was to determine if introduction of a voice recognition system (VRS) into the medication process would reduce the occurrence of prescription errors in a pediatric Accident and Emergency (A&E) department at a tertiary care hospital in Kenya. Our secondary objectives were to describe the pattern of medication errors, factors associated with their occurrence and to determine acceptability of VRS by prescribing doctors and dispensing pharmacists.

Methods

Study design and setting

We conducted a before–after observational study at the Aga Khan University Hospital (AKUH) pediatric A&E department and main pharmacy. AKUH is a private ‘not-for-profit’ tertiary care hospital in Nairobi that mainly caters to middle- and high-income earners.

Study procedures

Retrospective chart review was used to ascertain prescription and dispensing errors among senior house officers (SHOs), resident trainees and clinical instructors attending to clients at the pediatric A&E department. SHOs are recently licensed but non-specialized doctors; residents are pediatric trainees. Instructors are recently qualified pediatricians undergoing apprenticeship prior to licensing by the Kenya Medical Practitioners and Dentists Board. Doctors work in 6–12 hour shifts under supervision of the instructors but they all assess patients and prescribe independently.

A prescription error was defined as an omission or incorrect documentation of patient’s name, age, gender. Other errors included incorrect drug name, dose, route, frequency or duration as ascertained using the British National Formulary (BNF). A dispensing error was defined as a discrepancy between a correct prescription and the actual medication instructions that the dispensing pharmacist issued to the patient.

We conducted a retrospective chart review between May 2012 and April 2013 to ascertain prescription and dispensing errors among SHOs, resident trainees and clinical instructors attending to clients at the pediatric A&E department.

Intervention

In consultation with the AKUH department of information technology, a VRS was installed at the pediatric A&E and the main pharmacy prior to commencement of this study in May 2013. Installation entailed connecting a microphone to a computer based at the pediatric A&E department and linking it to the main hospital pharmacy through a central server. A medical dictionary consisting of common medical terms obtained from medication records obtained in the pre-VRS phase was set up and stored in the computer database. Doctors and pharmacists consenting to the use of VRS were then trained to enhance proficiency in its use. Voice profiles of participating doctors were also installed in order to enhance voice recognition.

The same team of doctors enrolled patients at the time of medical consultation with written informed consent from accompanying guardians. Patients then physically presented the VRS prescription at the main pharmacy to obtain their medications. Dispensing pharmacists verified biodata and medication particulars of patients from a label affixed to the medication package before dispensing with instructions on usage. We then analyzed medication particulars on VRS prescriptions and their corresponding dispensing records to determine the proportion of prescription errors and dispensing errors. At the end of the study, the doctors and pharmacists were requested to indicate whether they found the electronic VRS to be acceptable for medical prescription.

Data abstraction

Information extracted from prescription and dispensing records in the period before and after introduction of VRS included drug name, dose, route, frequency and duration of treatment. The corresponding biodata of patients, diagnosis, professional category of prescriber (SHO, resident or instructor) and time of day the prescription was made were retrieved from medical records and documented in a standard study tool.

Ethical considerations

Ethical consent procedures were reviewed and approved by the AKUH ethical review committee, Ref 2012/26 (V3). Written informed consent was obtained from parents of all participating study children. Consent was also obtained from prescribing doctors and pharmacists. Investigators gave non-disclosure guarantee on identities of participants unless with prior approval.

Errors identified at the prescription stage were noted and immediately rectified in consultation with the attending physician. Where errors were identified at the dispensing stage, the principal investigator recorded the same and informed the pharmacist to amend the dispensing label prior to dispensing the medication.

Sample size

In the absence of local information, sample size estimation for the number of prescriptions needed for analysis was based on a prior study by Kang and colleagues that reported 14% prevalence of medication errors and an error reduction of 48% following introduction of VRS.¹⁶ A minimum number of 496 in each arm was needed to provide 90% power with a 95% confidence interval.

Data management and analysis

Information was then entered into excel tables secured in a password-protected computer. Each entry was then reassessed to ensure all variables were entered correctly. In the case of missing information, medical, prescription and dispensing records were reanalyzed to capture the missing information.

Data was analyzed using STATA version 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP. It is manufactured by StataCorp in Texas USA). Demographic characteristics for both phases of the study were summarized as frequencies and proportions. The proportion of prescription or dispensing errors was compared between handwritten prescriptions and VRS-generated prescriptions using chi-square tests. Two sample tests for proportions were used to assess for significant differences in prescription errors for periods before and after intervention with stratification by work shift and doctor’s designation. The Mantel-Haenszel method was used to estimate odds ratios (ORs) of prescription errors between the pre-VRS and VRS phase. Acceptability of VRS by healthcare providers was estimated using simple proportions.

Results

A total of 1196 handwritten prescription records were examined in the pre-VRS phase and 501 in the VRS phase. Table 1 illustrates the biodemographic characteristics of study participants.

Table 1.

Biodemographic characteristics of study participants.

	Pre-VRS		During VRS		Total
	n	%	n	%	n	%
Sex of child
Male	696	58.2	226	45.1	922	54.3
Female	459	38.4	275	54.9	734	43.3
Missing	41	3.4	0	0	41	2.4
Shift worked
Morning	377	31.5	99	19.8	476	28
Afternoon	391	32.7	246	49.1	637	37.5
Evening	394	32.9	156	31.1	550	32.4
Missing	34	2.8	0	0	34	2
Designation
Instructor	550	46	229	45.7	779	45.9
Consultant	13	1.1	0	0	13	0.8
Resident	167	14	126	25.2	293	17.3
SHO	393	32.9	146	29.1	539	31.8
Intern	44	3.7	0	0	44	2.6
Unknown/missing	29	2.4	0	0	29	1.7
Age categories (years)
0–1	281	23.5	79	15.8	360	21.2
1–5	518	43.3	261	52.1	779	45.9
5–10	269	22.5	133	26.6	402	23.7
10–15	95	7.9	28	5.6	123	7.2
Missing	33	2.8	0	0	33	1.9
Total	1196	100	501	100	1697	100

SHO, senior house officer; VRS, voice recognition system.

NOTE: The bold figures demonstrates the totals i.e if one is to add all the entries Sex of child it should add up to 1196.

In the pre-VRS phase, 74.3% of the prescriptions (889 of 1196) had identifiable errors compared with 65.7% in the VRS phase (329 of 501).

Among prescription errors analyzed, the greatest impact of introduction of VRS was in reduction of incorrect medication dose. Incorrect medication doses were reduced by 13.4% with use of VRS (p < 0.0001). In the pre-VRS phase, 21.4% (256/1196) of the prescriptions had incorrect doses compared with only 8% (40/501) in the VRS phase (p < 0.0001). Unfortunately, as shown in Table 2, use of VRS was associated with an increase in errors related to documentation of drug name, drug frequency and duration of treatment.

Table 2.

Prescription errors before and after introduction of voice recognition system.

	Pre-VRS		During VRS		Total
	n	%	n	%	n	%	p-value
Drug name
Correct	1163	97.2	468	93.4	1631	96.1	0.000
Incorrect	33	2.8	33	6.6	66	3.9
Drug dose
Correct	739	61.8	377	75.2	1116	65.8	0.000
Incorrect	256	21.4	40	8	296	17.4
Omitted	201	16.8	84	16.8	285	16.8
Drug route
Correct	699	58.4	316	63.1	1015	59.8	0.206
Incorrect	10	0.8	4	0.8	14	0.8
Omitted	487	40.7	181	36.1	668	39.4
Drug frequency
Correct	886	74.1	361	72.1	1247	73.5	0.000
Incorrect	253	21.2	84	16.8	337	19.9
Omitted	57	4.8	56	11.2	113	6.7
Drug duration
Correct	975	81.5	409	81.6	1384	81.6	0.000
Incorrect	83	6.9	11	2.2	94	5.5
Omitted	138	11.5	81	16.2	219	12.9
Omission in prescription1
No	505	42.2	237	47.3	742	43.7	0.054
Yes	691	57.8	264	52.7	955	56.3
Incorrect prescription2
No	684	57.2	346	69.1	1030	60.7	0.000
Yes	512	42.8	155	30.9	667	39.3
Any prescription errors
Correct	307	25.7	172	34.3	479	28.2	0.000
Incorrect	889	74.3	329	65.7	1218	71.8

¹Omission of indicating either drug name, dose, route frequency or duration.

²Incorrect prescription regarding either drug name, dose, route frequency or duration, excluding omissions.

VRS, voice recognition system.

There were no significant differences noted in dispensing errors with use of VRS. There were 92.7% errors noted in the pre-VRS phase (1030 of 1111) compared with 93.3% in the VRS phase (332 of 356). The use of VRS was associated with an increase in omission of duration of medication prescription (p = 0.006). The effect of introduction of VRS on dispensing errors is shown in Table 3.

Table 3.

Dispensing errors before and after introduction of voice recognition system.

	Pre-VRS		During VRS		Total
	n	%	n	%	n	%	p-value
Drug name
Correct	1074	96.7	334	93.8	1408	96	0.017
Incorrect	37	3.3	22	6.2	59	4
Drug dose
Correct	765	68.9	264	74.2	1029	70.1	0.006
Incorrect	208	18.7	41	11.5	249	17
Omitted	138	12.4	51	14.3	189	12.9
Drug route
Correct	516	46.4	156	43.8	672	45.8	0.688
Incorrect	9	0.8	3	0.8	12	0.8
Omitted	586	52.7	197	55.3	783	53.4
Drug frequency
Correct	803	72.3	242	68	1045	71.2	0.246
Incorrect	196	17.6	69	19.4	265	18.1
Omitted	112	10.1	45	12.6	157	10.7
Drug duration
Correct	232	20.9	56	15.7	288	19.6	0.006
Incorrect	27	2.4	2	0.6	29	2
Omitted	852	76.7	298	83.7	1150	78.4
Dispensing omission1
No	115	10.4	27	7.6	142	9.7	0.124
Yes	996	89.6	329	92.4	1325	90.3
Incorrect dispensing2
No	708	63.7	235	66	943	64.3	0.434
Yes	403	36.3	121	34	524	35.7
Dispensing errors
Correct	81	7.3	24	6.7	105	7.2	0.726
Incorrect	1030	92.7	332	93.3	1362	92.8
Total (n)	1111		356		1467

¹Omission of indicating either drug name, dose, route frequency or duration when dispensing.

²Incorrect dispensing regarding either drug name, dose, route frequency or duration, excluding omissions.

VRS, voice recognition system.

Introduction of VRS was associated with a reduction in the proportion of errors noted in the afternoon and evening shifts by 8.6% (p = 0.023) and 13.3 % (p = 0.002), respectively. Similarly, residents and SHOs demonstrated a reduction in prescription errors by 19.9% (p < 0.001) and 14% (p = 0.001), respectively, as shown in Table 4.Use of VRS was associated with a reduction in proportion of prescription errors by 33% and 46% in the afternoon and evening shift, respectively. Among the various cadres of healthcare workers, the greatest risk reduction in errors with the use of VRS was noted among residents with a 61% reduction in the odds of committing prescription errors. Table 5 shows the effect of shift and designation on the occurrence of errors.

Table 4.

Proportion of prescription errors by shift and designation of doctors.

	Pre-VRS		During VRS		Total		p-value
	n	%	n	%	n	%
Shift
Morning	284	75.3	76	76.8	360	75.6	0.767
Afternoon	286	73.2	159	64.6	445	69.9	0.023
Evening	290	73.6	94	60.3	384	69.8	0.002
Designation
Instructor	402	73.1	166	72.5	568	72.9	0.863
Resident	130	77.8	73	57.9	203	69.3	0.000
SHO	297	75.6	90	61.6	387	71.8	0.001

SHO, senior house officer; VRS, voice recognition system.

Table 5.

Overall risk of prescription errors by shift and designation of doctors.

	Odds ratio	95% CI
Crude	0.66	0.53	0.83
Shift worked
Morning	1.08	0.64	1.82
Afternoon	0.67	0.48	0.95
Evening	0.54	0.37	0.80
M-H combined	0.69	0.55	0.87
Designation
Instructor	0.97	0.69	1.37
Resident	0.39	0.24	0.65
SHO	0.52	0.35	0.78
M-H combined	0.65	0.52	0.82

CI, confidence interval; M-H, Mantel-Haenszel; SHO, senior house officer.

A total of 58% of the healthcare providers interviewed (7/12) expressed desire for future use of VRS in the medication process. Some of the advantages noted by healthcare providers in using VRS were that it saved time during the prescription process and reduced errors associated with illegibility. Pharmacists observed that diction was a challenge in the utilization of VRS as they often had to log on to a separate online system to access VRS-generated prescriptions, which was perceived to increase workload.

Five out of twelve of the healthcare providers proposed incorporation of VRS into the hospital online information system, while one recommended expansion of the medical dictionary within the VRS to allow for comprehensive identification of medical terms. Other proposals made were to incorporate a prescription template and an online drug list within the software to automate dosages, route and frequency.

Discussion

This study was conducted in a pediatric A&E department of a private, not-for-profit, tertiary healthcare facility to primarily establish if introduction of a VRS would reduce medication errors. We demonstrated that overall, 74.3% (889 of 1196) of prescriptions in the pre-VRS phase had errors compared with 65.7% (329 of 501) in the VRS phase. These are unacceptably high error rates which may be attributed to how medication error was defined. Our study did not, for instance, include missing or wrong patient weight and errors in prescription dates in the list of errors. Kang and colleagues, using a similar VRS system on pathology reporting observed 48% reduction in errors.¹⁵ Further studies are required to ascertain how best to incorporate VRS within already existing health systems to maximize on its benefits.

In this study, the greatest impact of VRS introduction was in reduction of incorrect drug dosage. This advantage was, however, countered by an increase in wrong documentation of drug name, and failure to indicate frequency and duration of treatment. Our study, however, was not powered for a detailed subanalysis of the many variables in this study. These serious prescription omissions and commissions may be attributed to inadequacy in staff training and experience in using the new system. Antiles and colleagues indeed emphasized how crucial training and experience are for the successful use of the technology.¹⁷ Errors may also have been minimized if the VRS had been programmed to make it mandatory for all critical information to be entered before the user could proceed with the prescription process. Automation of dosages computed from weight or age entries could reduce errors even further. Fortunately, we did not find any incident of administering medications to the wrong patient during the study period.

The greatest impact of reduction in prescription errors was noted among residents and SHOs. This may be accounted for by the fact that junior doctors are yet to establish their prescription habits and patterns. Consequently, they readily take up new practices to improve prescription habits. Implementation of VRS reduced the prescription errors occurring in the afternoon and evening shifts. We postulate that this was due to the fact that most of these shifts were done by residents and SHOs.

The majority of our study respondents preferred VRS prescriptions compared with handwritten prescriptions. They were of the view that VRS reduced prescription illegibility, was user friendly and saved time. Diction was, however, of concern to dispensing pharmacists and will require further studies to provide software solutions.

In hospitals without online prescribing systems, patient information is typically dispersed in a collection of paper records that are poorly organized, illegible, and not easy to retrieve.⁷ Thus, information technology holds untapped potential for improving efficiency in healthcare delivery systems with positive impact on service quality and client satisfaction. Since VRS software can be downloaded from the internet at no cost, incorporation of VRS into hospital systems should be considered when facilities undertake digitalization of patient data. Equipment procurement, setup and maintenance may still be cost effective, given the high cost of manual record keeping and risk of serious harm to patients. The effect of more intensive training and close prescription monitoring with feedback warrants further study in efforts to lower error rates associated with use of the electronic system. Our study was not designed to assess cost effectiveness of VRS but we recommend that future studies ascertain the cost effectiveness of the free VRS software in the medication process.

Before–after trials have the inherent drawback of inability to control for changes that may take place over time and which would be beyond the control of investigators. A randomized trial would be preferable in assessment of effectiveness of VRS. However, in a relatively small A&E unit, with 11 full-time staff doing different shifts, it would present challenges, as blinding would not be possible. Implementation of VRS poses various logistical challenges too. During the study, it was not possible to integrate the existing pharmacy online system with the VRS due to software incompatibility. Despite creating a medical dictionary within the VRS, repeated trials at dictation and editing of medical terms were required during its use. Consequently, the greatest disadvantage cited by dispensing pharmacists during this study was diction that may have contributed to increased incidence in incorrect drug name entry in the VRS phase. Local customization of software that takes diction into account requires further exploration.

While acknowledging VRS prescription performed well below our expectation, our findings demonstrate the potential of using modern information technology to improve patient safety, provided areas of concern that we have highlighted are systematically addressed. As the Chair of the Committee on Quality of Health Care in America states, limited awareness on the magnitude of medication errors and extent to which healthcare professionals have fallen short of making optimal use of technology to improve healthcare safety explains in part the slow uptake of technology in the medication process.⁷ We hope that similar larger studies will lead to improvement in quality of medical prescription electronic software.

Conclusion

Implementation of VRS in the medication process has the potential to reduce medication errors, with the greatest impact noted particularly in reduction of incorrect dosages. Further research is recommended to determine if more user training, experience, close monitoring of prescriptions and software improvement will minimize drawbacks associated with VRS. Since VRS can be downloaded from the internet at no cost and appears to be acceptable to prescribing physicians and dispensing pharmacists, further studies should be undertaken to improve performance and assess cost effectiveness.