Classification of medication incidents associated with information technology

Abstract

Introduction

Information technology (IT) plays a pivotal role in improving patient safety, but can also cause new problems for patient safety. This study analyzed the nature and consequences of a large sample of IT-related medication incidents, as reported by healthcare professionals in community pharmacies and hospitals.

Methods

The medication incidents submitted to the Dutch central medication incidents registration (CMR) reporting system were analyzed from the perspective of the healthcare professional with the Magrabi classification. During classification new terms were added, if necessary.

Main measures

The principal source of the IT-related problem, nature of error. Additional measures: consequences of incidents, IT systems, phases of the medication process.

Results

From March 2010 to February 2011 the CMR received 4161 incidents: 1643 (39.5%) from community pharmacies and 2518 (60.5%) from hospitals. Eventually one of six incidents (16.1%, n=668) were related to IT; in community pharmacies more incidents (21.5%, n=351) were related to IT than in hospitals (12.6%, n=317). In community pharmacies 41.0% (n=150) of the incidents were about choosing the wrong medicine. Most of the erroneous exchanges were associated with confusion of medicine names and poor design of screens. In hospitals 55.3% (n=187) of incidents concerned human–machine interaction-related input during the use of computerized prescriber order entry. These use problems were also a major problem in pharmacy information systems outside the hospital.

Conclusions

A large sample of incidents shows that many of the incidents are related to IT, both in community pharmacies and hospitals. The interaction between human and machine plays a pivotal role in IT incidents in both settings.

Introduction

In 2001 the Institute of Medicine Committee on the Quality of Health System for the 21st Century predicted that information technology (IT) would play a pivotal role in improving patient safety.¹ IT can facilitate access to medical and medication information, assist with calculations, perform checks (in real time or afterwards), assist with monitoring, and support communication between healthcare professionals.^2–5 In particular, the introduction of computerized prescriber order entry (CPOE) systems created high expectations for enhancing patient safety in drug treatment. Not surprisingly, early studies of the introduction of IT in the healthcare sector focused only on the benefits of IT tools. For example, several studies investigated the implementation of CPOE in hospitals and its effects. Most of these studies showed a decrease in prescribing error rates (ranging from 29% to 96%) after implementation of CPOE.⁶ It was also found, however, that IT can cause new problems for patient safety.^6–10 An example of an IT-related incident is the juxtaposition error in CPOE. In a juxtaposition error CPOE users may unintentionally select a wrong item or patient because the items are close to each other on the screen.¹¹ Problems may also arise from the use of other technology such as health information systems, bar code scanning systems, automated dispensing cabinets, printers, and infusion pumps.

To get an insight into such IT-related incidents an instrument for measurement and analysis is needed. In a qualitative and quantitative study in a hospital Koppel et al¹² divided the incidents into two groups: human–machine interaction-related problems and information errors generated by fragmentation of data. With interviews, focus groups, shadowing and observations they identified 22 situations in which CPOE increased the probability of prescribing errors. Magrabi et al¹³ proposed a classification of IT-related incidents based on an analysis of patient safety incidents associated with computer use. They analyzed 111 incidents from hospitals that were derived from a voluntary reporting system in Australia to explore the unintended consequences related to IT. In a second study Magrabi et al¹⁴ expanded their original classification after analyzing 436 IT manufacturer incidents, which had been submitted to the US Food and Drug Administration manufacturer and user facility device experience (MAUDE) database. Manufacturers in the USA are required to report medical device malfunction to MAUDE and manufacturers voluntarily report IT-related incidents to MAUDE. The usefulness of the resulting classification across different healthcare settings has yet to be tested. This study therefore aimed at the analysis of the nature and consequences of a large sample of IT-related medication incidents, as reported by Dutch healthcare professionals in community pharmacies and hospitals, using the most recently adapted version of the classification of Magrabi et al.¹⁴

Methods

Setting

In The Netherlands, there were 93 hospitals and 1997 community pharmacies in 2012.^{15 16} Hospitals and community pharmacies have a long history of implementing IT tools and both have started in 2006 and 2010, respectively, to report their medication incidents to a nationwide Dutch reporting system: central medication incidents registration (CMR).^{17 18} The general picture is as follows, all hospital pharmacies and community pharmacies now have a computer system for entering prescriptions. CPOE is not yet fully implemented in all hospitals. In a recent study using questionnaires, CPOE was used or was being implemented by 64 of the 72 responding hospitals. In those hospitals 10 different CPOE systems were used.¹⁶ All primary care physicians use CPOE and electronic medical records. Despite the use of CPOE by primary care physicians not all prescriptions can be transmitted electronically to the pharmacy, because of a lack of system connectivity. Both hospitals and community pharmacies have integrated clinical decision support systems in their IT systems. The pharmacy staff generally use barcode scanning during dispensing. Compounding is generally supported by electronic protocols and in process controls (eg, checking of batch numbers, monitoring the correct type and amount of ingredients with barcode scanning and linked weighing balances).

Data source

For this study we used a subset of the reported medication incidents that were sent by hospitals and community pharmacies to the Dutch CMR database from March 2010 to February 2011. These incidents had already been analyzed for a general study about the CMR. The collection and analysis of the incidents is exempt from medical ethical approval by Dutch clinical trial law as it does not compromise the integrity of patients. All data were handled according to the privacy legislation in The Netherlands.¹⁸

Identification of relevant incidents: development of a search tool

A string of search terms referring to IT was developed for identifying text fragments in the free text description. An initial set of terms was derived from the literature and adapted on the basis of the experiences of members of the research team (KCC, PDS) with the weekly screening of incidents to the CMR.¹⁸ This initial set of terms was applied to a set of 100 incidents that had been randomly selected from the database. The same set of 100 incidents was also analyzed manually by researcher KCC. The researcher read the free text description and decided for each report whether the incident was related to IT (see supplementary appendix A, available online only, for the chapters and items on the CMR reporting form). Both selections (after applying the set of search terms and after manual analysis) were compared, on the basis of which set of terms was adapted. Eventually this process was iterated five times until no new terms emerged. To check this set of terms, a second researcher (WVN) followed the same iterative method and if necessary the set of terms was expanded with new terms. Researcher WVN applied the set of terms once to a different set of 100 incidents and manually checked these for comparison. The final set of search terms consisted of unique 121 items and some words were repeated in misspellings or in a part of the word (see supplementary appendix B, available online only, for the list of 121 search terms).

Identification of relevant incidents: application of the search tool

The final set of search terms was applied to the CMR incidents that had been reported in the period of March 2010 up to February 2011. The incidents thus identified were independently reviewed by two researchers (KCC and WVN). They selected incidents if they perceived that technology had somehow contributed to the incident. The resulting incidents were subsequently divided into three groups:

• both researchers assessed that the incident was suitable for inclusion

• both researchers assessed that the incident was not suitable for inclusion (exclusion)

• one or both researchers had doubts about the suitability of the incident.

The latter category of incidents was reviewed by a third researcher (PDS) to make a final decision on inclusion or exclusion.

After reviewing duplicate incidents were removed (seven incidents from community pharmacies and one incident from hospitals). During analysis our insight into IT incidents deepened and eventually we removed six incidents because they had been mistakenly selected initially (one incident from a community pharmacy and five incidents from hospitals).

Classification of relevant incidents

The two researchers (KCC, WVN) analyzed and classified 200 incidents together to become accustomed with the analyzing method and with the axes of the most recent Magrabi classification, which was published in 2012.¹⁴ The remaining incidents were then independently analyzed and classified by the two researchers. They subsequently came together to compare their results and to reach consensus on the classification of the incidents. If an incident described more than one IT-related incident, the researchers classified all the problems separately. For the incidents that were independently analyzed, the percentages of agreement were calculated. The percentages of agreement were calculated for the two axes (the principal source of the IT-related problem and the nature of the error) and the additional category IT system and phases of the medication process. Within the incidents from community pharmacies the percentages of agreement ranged from 85.8% to 93.3% and within hospital incidents ranged from 52.7% to 80.0%. For both the community pharmacies and the hospitals the percentages of agreement were lowest for the axis of the nature of the error.

This classification consists of two axes: the principal source of the IT-related problem (‘machine-related error’ or ‘human–machine interaction-related error’) and the nature of the error (problem). Magrabi et al¹⁴ subdivide the latter axis (the nature of the error) into incidents related to input (data entry), to output (data retrieval), and to transfer (transfer of data between systems). In addition, Magrabi et al¹⁴ had two separate items in the classification that were not linked to input, transfer or output (contributing factors and general technical). The contributing factors were not strictly related to IT and we did not find examples in our analysis. The general technical terms were rearranged during our classification and linked to input, transfers or output. In total the Magrabi classification consists of 32 preferred terms; for example, wrong input, (machine) not alerted, data loss, etc. During classification of the CMR incidents new preferred terms were added, if the Magrabi classification could not cover the incident adequately. For the axis ‘nature of the error’, the two researchers maintained the subdivisions input, transfer and output. The preferred term ‘wrong input’ was elaborated by adding nine new preferred subterms: wrong patient; wrong medicine; wrong dose; wrong duration of therapy; wrong time of administration; wrong pump speed; wrong prescriber; duplicate input; and other wrong input. An extra subdivision of five preferred terms for wrong medicine was considered necessary to classify the incidents in sufficient detail. For the preferred term ‘not done’ two new preferred subterms were added. The researchers also added five new preferred terms in the subdivision output (data retrieval) and two new terms in the subdivision transfer (data of transfer) (see table 1 and figure 1).

Table 1.

Nature of the error

	Problems in community pharmacies N (%)	Problems in hospitals N (%)	Examples of incidents in community pharmacies and hospitals
Data entry and record manipulation
No input
Not done*
Not done by human*	9 (2.5)	85 (25.1)	▸The pharmacist received an e-mail with a prescription; due to an unknown reason the pharmacist assistant did not enter the prescription into the system. (CP) ▸After the ward round the physician forgot to enter the prescriptions into the CPOE. (H) ▸The physician was not familiar with CPOE and could not order the medicine with the CPOE. (H)
Not possible to import record†	–	8 (2.4)	▸Rifampicin was not listed in the CPOE. The consequence was that the physician could not order rifampicin in the CPOE. (H)
Not possible to change predefined record†	–	2 (0.6)	▸The physician could not change the infusion rate of a predefined antibiotic order in the CPOE. (H)
Wrong input*
Wrong medicine†
Wrong identity medicine†	49 (13.4)	12 (3.6)	▸The pharmacist assistant entered ‘CHLOO25’ in the system and accidentally chose chlortalidone 25 mg instead of chlordiazepoxide 25 mg on the screen. (CP)
Wrong dosage form†	26 (7.1)	6 (1.8)	▸An erroneous exchange between immediate release tablet and slow release tablet. The pharmacist assistant chose the wrong medicine from the list, which was presented by the pharmacy information system. (CP)
Wrong route of administration†	1 (0.3)	1 (0.3)	▸For eye drops the right eye was entered in the pharmacy information system instead of the left eye. (CP)
Wrong strength of product†	72 (19.7)	17 (5.0)	▸The pharmacy dispensed sifrol 3.75 mg instead of 0.375 mg. (CP)
Selected medicine not available†	2 (0.5)	–	▸The general practitioners repeated a prescription and the original identification record was canceled. In the community pharmacy this repeat record cannot be recognized by the pharmacy information system. (CP)
Wrong patient†	54 (14.8)	18 (5.3)	▸Pharmacist assistant used date of birth to find a patient in the system. After entering the date of birth a list of patient names with the same day of birth was shown on the screen. A wrong patient was selected due to a poor design of screens. (CP) ▸At the ward there were two patients with the same family name. The physician selected the wrong patient on the screen of the CPOE and entered a prescription for the wrong patient. (H) ▸The physician entered a prescription into CPOE for a one-day-old newborn. During dispensing the pharmacist assistant noticed the birth day and called the ward. During the call they discovered the medicine should have been prescribed to the mother. (H)
Wrong dose/frequency†	47 (12.9)	23 (6.8)	▸A pharmacist duplicated a record in the system and accidentally repeated an outdated dose in this process. (CP)
Wrong duration of therapy/quantity of the medicine†	13 (3.6)	3 (0.9)	▸The pharmacist assistant entered 10 tablets of ondansetron 8 mg instead of 30 tablets of ondansetron. (CP)
Wrong time of administration†	2 (0.5)	23 (6.8)	▸A wrong time of administration was entered into the CPOE. The patient needed the medicine around 12 : 00 h and the time of administration in the CPOE was 14 : 00 h. (H)
Wrong infusion pump rate†	–	21 (6.2)	▸The rate of an infusion pump was accidentally set wrongly. Due to the low infusion pump rate the patient received only half of the dose. (H)
Wrong prescriber†	5 (1.4)	1 (0.3)	▸The pharmacist assistant entered the wrong code of the prescriber into the pharmacy information system. (CP)
Duplicate input†	8 (2.2)	10 (3.0)	▸The pharmacist assistant entered the prescription two times in the pharmacy information system. (CP) ▸The physician entered the same medicine twice into the CPOE. (H)
Other wrong input†	6 (1.6)	12 (3.6)	▸The physician entered diclofenac into the CPOE for a patient for whom diclofenac was contraindicated. (H)
Failure to communicate after input*	–	5 (1.5)	▸The physician entered the medication order into the CPOE but he forgot to brief the nurses about the new medication. (H)
Data retrieval
No output
System slow/down*	–	14 (4.1)	▸Physicians and nurses could not reach the CPOE because there was a large-scale IT malfunction. (H) ▸The nurse did not administer the antibiotic because the printer was down and she could not print out the administration list. (H)
Not done by human (did not look)*	14 (3.8)	11 (3.3)	▸The pharmacist assistant did not look into the notes of the patient file and missed the information that the patient needed a home delivery of the medicine. (CP) ▸Nurses did not realize the physician had entered a note in the electronic patient file and thereby missed the administration of an antibiotic. (H)
Not alerted/no output*	9 (2.5)	7 (2.1)	▸A cardiologist accidentally prescribed a high dose of flecainide for a patient in primary care and the pharmacy computer system did not alert the community pharmacist about it. There was no alert because formally it was not an overdose, but according to the cardiologist the dose was too high for the patient in primary care. There should have been an alert. (CP)
Wrong output
Output error*	5 (1.4)	9 (2.7)	▸The infusion pump alerted the nurses too late about an obstruction in the tube. (H)
Unclear output
Different output online and printed†	1 (0.3)	2 (0.6)	▸In the CPOE the nurse read that the aspirin needed to be administered with a high loading dose, but on the paper medication list the information about the high loading dose was missing. (H)
Differences between two files†	–	3 (0.9)	▸In the CPOE the nurse read from the medication list that the patient needed tolbutamid. In a separate memo field in the CPOE the nurse read that tolbutamid should not be administered to the patient. (H)
Other unclear output†	6 (1.6)	35 (10.4)	▸A community pharmacist printed out a medication list for a patient going to hospital. The printout was unclear and the consequence was that a physician in the hospital misinterpreted this medication list. He thought the patient only used 50 mg losartan per day instead of 2 times 50 mg. (CP) ▸A nurse administered 5 times more bisoprolol than prescribed. On the medication list she read that the patient needed bisoprolol and on the list the number 5 was printed without unit (mg or tablet). Eventually she administered 5 tablets of isoprolol 5 mg to the patient. (H) ▸The nurse missed a new prescription order because the printer had printed out all the orders at once with the new prescriptions at the bottom of the pile of paper (even after orders that had already been stopped). (H)
Failure to react on signal†	29 (7.4)	5 (1.5)	▸Due to alert fatigue a pharmacist assistant overruled the signal from the pharmacy barcode scanning system that the wrong medicine had been chosen. (CP) ▸The general practitioner ignored a drug–drug interaction signal. (CP) ▸The infusion pump made an alarm sound. The nurse could not identify the problem and eventually switched off the alarm of the infusion pump. (H) ▸A pharmacist assistant did not respond correctly to alerts of the pharmaceutical clinical decision support system, such as allergy warnings or drug–drug interaction warnings. For example, an order for a cephalosporin was executed despite an alert for an allergy. (H)
Other output†	2 (0.5)	1 (0.3)	▸For dispensing the pharmacist assistant printed out a list, which was not up to date anymore. (H)
Data transfer
Mistranslation of data between 2 systems†	4 (1.1)	–	▸An incomplete transfer of an e-prescription between the computers of the general practitioner and the community pharmacist. The information of the brand of the medicine was missing. (CP)
No data transfer between 2 systems†	3 (0.8)	4 (1.2)	▸A physician could not use the CPOE because of a technical malfunction in the connection between the CPOE and the medical record system in the hospital. (H)

*This preferred term was also available in the Magrabi classification.

†This preferred term is new.

CP, community pharmacies; CPOE, computerized physician order entry; H, hospitals.

Figure 1.

Adapted diagram of Magrabi et al¹⁴ 2012 classification and added terms from central medication incidents registration incidents.

After categorizing the IT incidents using the Magrabi classification as described above, further characterization of the incidents was performed by designating the IT-related problem to the IT system involved (Otable 2) and the specific phase of the medication process into which the medication incident had occurred (Otable 3). Information about the consequences of the incidents was collected directly from the incident report forms (see supplementary appendix A, available online only, for the chapters and items on the CMR reporting form).

Table 2.

Overview of IT systems involved

IT systems	Involved in the problems in
	Community pharmacies N (%)	Hospitals N (%)
Automated dispensing cabinets	2 (0.5)	–
Computerized physician order entry	21 (5.8)	250 (74.0)
Order system website*	1 (0.3)	–
Electronic health record	–	21 (6.2)
Fax	–	1 (0.3)
Infusion pump	–	27 (8.0)
Laboratory diagnostic analyser†	–	1 (0.3)
Medication administration registration	–	5 (1.5)
Pharmacy bar code scanning system	13 (3.6)	–
Pharmacy information system	326 (89.3)	28 (8.3)
Prescription scanner‡	1 (0.3)	–
Printer	1 (0.3)	5 (1.5)

*Website used by pharmacies to purchase medicine.

†Automatic devices used by diagnostic laboratories to analyze blood, urine, etc.

‡Community pharmacies scan the prescriptions after dispensing to digitally archive the prescriptions.

Table 3.

IT problems in the different phases of the medication process

Phase in medication process	Problems in community pharmacies N (%)	Problems in hospitals N (%)
Prescribing	23 (6.3)	225 (66.6)
Transcription	–	2 (0.6)
Entering of prescriptions into the pharmacy information system*	322 (88.2)	22 (6.5)
Compounding	–	–
Dispensing	16 (4.4)	4 (1.2)
Administration	–	82 (24.3)
Patient monitoring	–	3 (0.9)
Storage and logistics	4 (1.1)	–

*This is including pharmaceutical clinical decision support.

Results

Identification of relevant incidents

In the period of March 2010 up to February 2011, the CMR received 4161 incidents. Healthcare providers working in community pharmacies submitted 1643 (39.5%) incidents and those in hospitals submitted 2518 (60.5%) incidents. The set of IT-related search terms yielded 624 incidents from community pharmacies and 877 incidents from hospitals. After reviewing by two researchers (KCC, WVN), 16.1% (668/4161) of all CMR incidents were somehow related to technology. In the batch of incidents from the community pharmacies, 21.5% (351/1636) of the incidents were related to technology and in the batch from the hospitals this percentage was 12.6% (317/2517). The researchers (KCC, WVN) extracted 365 problems from the 351 community pharmacy incidents and 338 problems from the 317 hospital incidents (see supplementary appendix C, available online only, for the flow chart of this process).

Consequences of incidents

Community pharmacies reported 167 (47.6%) incidents, which had reached the patient. Most of these incidents (82.0%, n=137) were harmless to the patient; 12.0% (n=20) of incidents caused minimal harm, 2.4% (n=4) caused serious temporary harm, and for six (3.6%) incidents the outcome for the patient remained unknown. In the hospitals 193 (60.9%) incidents reached the patient; 46.6% (n=90) of these 193 incidents were harmless to the patient, 23.8% (n=46) of incidents caused minimal harm, 8.3% (n=16) of incidents caused serious temporary harm, two (1.0%) incidents were associated with the death of a patient, and for 20.2% (n=39) of the incidents the outcome was unknown.

Classification of relevant incidents

Table 4 shows a combination of two axes, ie, the principal source of the IT-related problem and the nature of the error (only subdivided as input, transfer and output). Most of the incidents were classified as human–machine interaction-related incidents.

Table 4.

Principal source of IT-related problem and nature of the error

	Problems in community pharmacies N (%)	Problems in hospitals N (%)
Human–machine interaction-related input	291 (79.7)	234 (69.2)
Human–machine interaction-related output	48 (13.2)	56 (16.6)
Machine-related input	3 (0.8)	15 (4.4)
Machine-related output	16 (4.4)	29 (8.6)
Machine-related transfer	7 (1.9)	4 (1.2)

In the community pharmacies 92.9% (n=339) of all the incidents concerned interactions between humans and the IT system. Table 4 shows that most problems (79.7%, n=291) were classified as human–machine interaction-related input (data entry). A relatively common problem was a healthcare provider choosing the wrong patient when entering the prescription into the pharmacy computer system.

Fewer problems (85.8%, n=290) reported from hospitals belonged to an interaction between a human and a machine. Within this group data entry (input) was the most classified problem and 16.6% (n=56) of the incidents were classified as human–machine interaction-related output. Most of these incidents were about unclear printouts.

Nature of the errors

The axis of the nature of the errors ultimately comprised 28 preferred terms (see table 1 and figure 1).

In community pharmacies 41.0% (n=150) of the incidents were about choosing the wrong medicine. Most of the erroneous exchanges were caused by confusion of medicine names and poor design of screens. The second most frequent problem was choosing the wrong patient. In community pharmacies incidents related to output (data retrieval) were not common.

A quarter (25.1%, n=85) of the incidents in hospitals dealt with healthcare providers who did not enter (‘not done by human’) data in the systems (eg, CPOE). It was not always clear why the physicians did not enter the prescription(s) into the CPOE. The incidents classified to ‘output unclear’ concerned problems with printouts of medication lists for administration. The machine-related output incidents in hospitals were about printers with technical malfunction so that nurses could not print out medicine lists any more.

IT system

The IT system category consisted of 12 different IT systems (see Table 2). Most IT systems were used in hospitals and community pharmacies, but some IT systems (infusion pumps) were only mentioned in the incidents from hospitals. Sometimes systems were linked to each other, for example, a printer connected to a computer with a software program (CPOE or pharmacy information system). In the hospital the CPOE was generally linked to the pharmacy information system so that physicians, pharmacists and nurses could use the same system for prescribing, dispensing and administration. In the community pharmacies, the pharmacy information system and the pharmacy bar code scanning system were linked to each other. Clinical decision support systems are always incorporated in CPOE systems or pharmacy computer systems. In this study we classified all incidents concerning clinical decision support as CPOE or pharmacy information system.

In community pharmacies 74.0% (n=270) of the incidents were related to the pharmacy information system and concerned human–machine interaction-related input. Other incidents with the pharmacy information systems were related to human–machine interaction-related output (9.9%, n=36) and machine-related output (3.6%, n=13). In the machine-related output a pharmacy information system gave incorrect and confusing advice to the pharmacy assistant.

In hospitals the CPOE was the most frequently implicated IT system and 55.3% (n=187) of the incidents concerned human–machine interaction-related input in combination with CPOE; 9.2% (n=31) of the incidents concerned CPOE and human–machine interaction-related output. One example was a large-scale malfunction of the CPOE, during which physicians and nurses could not reach the system any more. Physicians and nurses could not prescribe or administer. Incidents with pharmacy information systems were not so common but when they occurred most of them concerned human–machine interaction-related input (5.6%, n=19).

Phases of the medication process

Table 3 shows the classification of problems into the different phases of the medication process.

In community pharmacies 88.2% (n=322) of the incidents occurred during the entering of prescriptions into the pharmacy information system. Obviously all incidents in this phase were related to the pharmacy information systems.

In hospitals 66.6% (n=225) of the incidents occurred during the prescribing process, the second place was taken by the administration phase (24.3%, n=82). In the prescribing phase the CPOE had a prominent position (63.6% (n=215) of all incidents). The CPOE also played a role in the administration phase (10.1% (n=34) of all incidents). Most of these latter incidents related to the printing of medication lists, for example, physicians forgot to print the list after entering prescriptions into the CPOE. Incidents in the transcription phase, patient monitoring phase and storages and logistics were hardly reported from hospitals and community pharmacies.

Discussion

This is the first study on the nature and frequency of medication incidents related to IT in a large sample of IT-related incidents reported by healthcare providers in community pharmacies and hospitals. We found that one of six reported incidents (16.1%, n=668) were related to IT and that more incidents were related to IT in the community pharmacies (21.5%, n=351) than in hospitals (12.6%, n=317). As far as we know, this is also the first study analyzing medication incidents related to all kinds of IT systems, thereby showing the pivotal role of CPOE and pharmacy information systems in medication incidents.

Within the Magrabi classification we expanded the ‘input’ group with a subdivision to make the incidents more specific and concrete. Magrabi et al¹⁴ primarily chose an IT perspective, which seems especially important for IT professionals who develop healthcare-related IT systems. Our angle was guided by the proposal of Sittig and Singh¹⁹ to define IT incidents not only from the technical viewpoint of manufacturers, developers, and vendors but also from the social technical viewpoint of end users. The underlying principle is that healthcare providers wish to learn about IT-related risks by considering when and what they can do wrong with what type of IT system. We analyzed the incidents from a healthcare provider's perspective and we combined it with the technical items. Eventually, we related the technical items to input or output problems. Magrabi et al¹⁴ also had ‘contributing factors’, which consisted of organizational or individual causes of incidents. We were focussed on the nature of the incident and we did not use these items.

Interestingly, our study showed that the input problems occurring with CPOE also occurred with pharmacy information systems outside the hospital. Most studies that we found were about the impact of CPOE and there were no studies about the impact of pharmacy information systems.^6–10 Despite the use of CPOE in primary care many of the community pharmacists still need to enter the prescriptions manually into their pharmacy information systems. One of the reasons is that generally not all prescriptions can be electronically transmitted from the CPOE system to the pharmacy information system.

Although frequencies have to be interpreted carefully in this study, it is interesting to compare our results with those of other studies. In their first study, Magrabi et al¹,³ identified 111 incidents from a database with 42 616 incidents (0.3%, n=111) and in the second study 678 incidents were selected from a database with 899 768 incidents (0.1%, n=678).¹⁴ IT was much more frequently involved in our sample of incidents. One reason may be that the latter consisted entirely of medication incidents. Another contributory factor could be the long history of implementing IT tools in Dutch healthcare. Since the 1970s, community pharmacists have applied IT in their daily practice (followed later by primary care physicians).¹⁷ In hospitals the shift from a paper-based to a computerized system began 10 years later.

In the first study of Magrabi et al,¹,³ 45% (n=53) of the incidents were human–machine interaction-related problems.¹³ In their second study this number was lower and only 4% (n=30) were human–machine interaction-related problems.¹⁴ The MAUDE database contains incidents from manufacturers in the USA and probably these incidents were more focussed on pure IT aspects (only machine-related problems), such as software problems. In contrast, our study showed that a majority of the incidents were human–machine interaction related. Healthcare providers reported directly to CMR and although it may be difficult for them to identify the underlying technical causes of IT-related incidents, they can readily recognize the nature and clinical consequences of such incidents. The predominance of incidents concerning data entry and record manipulation (input) is in line with the results of Magrabi et al,¹,³ who classified 31% (n=36) of the incidents as information input problems. A USA national voluntary medication error-reporting database showed comparable CPOE input problems. Half of the incidents involved dosing errors such as wrong doses.²⁰ Zhan et al²⁰ concluded that CPOE-related medication errors are not only caused by faulty computer interfaces but also by common use errors such as typing errors. Most studies about CPOE have shown comparable input problems.^{6 8 12 21–23}

Our low proportion of transfer problems was in contrast with Magrabi et al,¹³ who classified 20% (n=23) of all incidents as transfer problems in their first study. Magrabi et al¹³ classified incidents related to computer network, systems integration issues and inaccessibility of systems, from as little as 15 min to as long as 8 h, as information transfer problems. In their second study, however, only a small proportion of problems (2%, n=13) was related to transfer of information.¹⁴ With the healthcare provider's perspective we focused on how the problems affected the work processes, and eventually incidents were classified as input or output problems. This could explain our low proportion of transfer problems. We only assigned two types of transfer problems: ‘mistranslation of data’ and ‘no data transfer’. These kinds of problems were also mentioned in a literature study about the transferring and displaying of pathology data in electronic health records.²⁴

Strengths and limitations

The main strengths of this study were the comparison between the different healthcare settings and the high number of incidents, as well as the use of a classification system that is in accordance with the healthcare provider's perspective. This study proved that one classification could be used for both settings.

For this study we analyzed a large number (668) of incidents from community pharmacies and hospitals. Lewis²⁵ states in an article about post-marketing surveillance that the number of studied drug users must be three times as high as the frequency of an adverse drug reaction to have a 95% chance that the reaction will actually occur in the study population. For instance, 300 subjects have to be studied to have 95% confidence to detect an adverse drug reaction with an incidence of one in 100.²⁶ This means that the number of analyzed incidents in our study was more than sufficient to get an insight into the most frequent unintended consequences associated with IT incidents.

Despite the rigorous validation process a potential limitation of this study is that the adapted classification was only applied to one set of incidents. A logical next step would be its validation in a new set of data. Another limitation was the variable quality of the descriptions of the incidents. Not all the incidents were described well and some of them hardly contained enough information for further analysis. To minimize the risk that the researcher would infer some details of the incident that were not actually reported, the two researchers analyzed the incidents independently and met afterwards to reach consensus. A third limitation was the difficulty to classify the incidents in the axis of the nature of the error. The IT systems were easier to classify because they were more concrete.

Last but not least, the incidents came from a voluntary reporting system and it could be possible that healthcare providers primarily focused on incidents that they considered important or out of the ordinary. Especially after the introduction of a new IT system healthcare providers might focus more on the use of this new IT system.²⁷ On the other hand, incidents that were not recognized by healthcare providers will thereby have remained unreported. So the real number of unintended consequences with IT could also be higher.

Implications for practice

Considering the percentage of incidents related to IT, it is necessary to pay attention to this new field of incidents in healthcare. IT was introduced with the idea of preventing incidents and healthcare providers may trust IT too much in supporting their daily practice. This study helps healthcare providers to become more aware of the unintended consequences related to IT.

Our study identified all kinds of IT problems and healthcare workers need to be aware that such problems can occur. Healthcare providers must know how to intercept or respond to these IT incidents to prevent patient harm. Interceptions may be performed from the human perspective (eg, training of individuals) or from the technical/organizational perspective (eg, system design and workflow changes). In general, the latter are preferred because they form a system solution instead of an individual solution.²⁸

This study suggests a few interceptions. An accessible back-up of patient records is required when a large-scale malfunction of the CPOE prevents physicians and nurses from reaching the regular system. When printers are not able to print any more nurses should be aware that they have to access patient information by other means. The input problems that were caused by poor design of screens need to be discussed with the software vendors. The implementation of complex CPOE or any IT system should be accompanied by adequate training in the use and possibilities of such IT systems. Healthcare organizations should consider the relevant work processes when installing a new IT system. The problem ‘not done by human’ could sometimes be related to the introduction of a new IT system, which does not fit well into an existing work process. Finally, the classification system used in this study may help to increase the information value of incidents.

Implications for research

Future research should be carried out in collaboration with users, vendors and incident-analysis experts to get a more intensive insight into IT-related incidents. The classification of Magrabi et al¹⁴ was useful after we had added some preferred terms, but for more information about the incidents we believe that subsequent analysis of underlying causes, harm to the patient and which healthcare profession was involved, might be helpful. This should be the subject of further study and the final classification system should be validated using different sets of incidents.

Technology is changing fast and every day new IT system can be introduced that will entail their own unintended consequences. The introduction of new IT system should be accompanied by prospective risk analysis.¹⁶ Research on the performance and effect of such risk analyses is necessary.

Information transfer problems are an important new area for research. At this moment these problems are not yet common but more and more computers will be linked to each other. Thus a malfunction in one setting can rapidly spread to other departments or healthcare organizations.^{29 30}

This study was focussed on the determination of IT-related incidents and compared these in community pharmacies and hospitals. Some interceptions were suggested to prevent reoccurrence of the incidents. Research is needed to investigate the interceptions on human perspectives and technical/organization perspectives. Probably a combination of both sorts of interception is necessary to prevent IT-related incidents.

Conclusion

This is the first study that shows how many of the incidents in the CMR database are related to IT in both community pharmacies and hospitals. The interaction between human and machine plays a pivotal role in the IT incidents. In community pharmacies the pharmacy information system was most frequently involved while in hospitals the CPOE was most frequently involved. The classification of Magrabi et al¹⁴ was a very useful starting point but we added some new preferred terms during analysis. In subsequent analysis we introduced the IT system category in this study and phases of the medication process. The slightly adapted Magrabi classification will help healthcare providers in picturing the incidents, as these axes help to put the incidents in the context of healthcare practice. This classification system seems useful for reporting and analyzing IT incidents in healthcare in general, but further research will have to prove this.