Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Jul 1.
Published in final edited form as: Int J Ind Ergon. 2011 Jul 1;41(4):370–379. doi: 10.1016/j.ergon.2011.02.007

That’s nice, but what does IT do? Evaluating the impact of bar coded medication administration by measuring changes in the process of care

Richard J Holden 1,2,*, Roger L Brown 3, Samuel J Alper 4,†, Matthew C Scanlon 5, Neal R Patel 6, Ben-Tzion Karsh 7
PMCID: PMC3113497  NIHMSID: NIHMS275786  PMID: 21686318

Abstract

Health information technology (IT) is widely endorsed as a way to improve key health care outcomes, particularly patient safety. Applying a human factors approach, this paper models more explicitly how health IT might improve or worsen outcomes. The human factors model specifies that health IT transforms the work system, which transforms the process of care, which in turn transforms the outcome of care. This study reports on transformations of the medication administration process that resulted from the implementation of one type of IT: bar coded medication administration (BCMA). Registered nurses at two large pediatric hospitals in the US participated in a survey administered before and after one of the hospitals implemented BCMA. Nurses’ perceptions of the administration process changed at the hospital that implemented BCMA, whereas perceptions of nurses at the control hospital did not. BCMA appeared to improve the safety of the processes of matching medications to the medication administration record and checking patient identification. The accuracy, usefulness, and consistency of checking patient identification improved as well. In contrast, nurses’ perceptions of the usefulness, time efficiency, and ease of the documentation process decreased post-BCMA. Discussion of survey findings is supplemented by observations and interviews at the hospital that implemented BCMA. By considering the way that IT transforms the work system and the work process a practitioner can better predict the kind of outcomes that the IT might produce. More importantly, the practitioner can achieve or prevent outcomes of interest by using design and redesign aimed at controlling work system and process transformations.

Keywords: health information technology, bar coded medication administration, process change, patient safety, human factors engineering

1. Introduction

Health information technology (IT) is commonly regarded as a key patient safety solution, and there exists evidence to support that it should be: “Data now show that information technology can reduce the frequency of [medical] errors of different types and probably the frequency of associated adverse events” [1]. From a human factors engineering point of view, the quote begs the question, what is it that health IT does that improves (or, in some cases, worsens) safety outcomes? A human factors approach would suggest that changes in outcomes are a function of how IT transforms the nature of work [2–5]. In Section 1.1, we elaborate on that notion, presenting a human factors model of health IT’s transformational effects. Afterwards, findings are reported from a study of how one particular type of health IT, bar coded medication administration (BCMA) technology [6], transformed a critical piece of acute care nursing work: medication administration. An intent of this paper is to move beyond the simple idea that health IT has safety benefits toward a fuller understanding of the impact of IT, or “what IT does.”

1.1. Human factors model of health IT impact

The human factors model of health IT change (Fig. 1) builds directly on the conceptual frameworks of Carayon et al [7] and Holden and Karsh [2, 8, 9, 4] and indirectly on “structure-process-outcome” [10] models and evaluation frameworks in the health IT literature [11–16]. Briefly, Carayon and colleagues’ Systems Engineering Initiative for Patient Safety (SEIPS) framework depicts a complex sociotechnical system—henceforth, “work system”—composed of five interacting elements: person, task, tools/technologies, organization, and environment. As a function of its dynamic interactions, “the work system can influence processes, in particular care processes, which in turn are key contributors to the quality and safety of care; in addition, the work system can influence employee and organizational outcomes, such as job satisfaction, burnout and worker safety” [17]. Holden and Karsh similarly posit a complex work system, but one that is hierarchically arranged (e.g., person-, unit-, and organization-level) [8, 9, 4]. Those authors stress that the nature and design of the work system will determine the nature of performance processes, which they define as the “mental, physical, social, and behavioral activities carried out by HCPs [health care providers] toward some (usually patient related) goal” [4]. Performance processes result in patient and employee outcomes similar to those in the SEIPS model [3, 18, 4].

Figure 1.

Figure 1

Open in a new tab

A human factors model of health information technology (IT) impact.

Health IT is depicted in the human factors model in Fig. 1 as a new element being integrated into a pre-existing work system, becoming “intimately woven into the system itself” rather than acting merely as “an appendage to ordinary work” [19]. An implication of this is that the work system following the implementation of new IT (or any other major change, for that matter) will be a transformed work system [20, 9].

It follows from the human factors model of health IT in Fig. 1 that a work system transformed by health IT will produce changes in the processes of care. Many studies show how the implementation of health IT creates or eliminates care-related activities or process steps [21–23] and how health IT alters patterns of communication and cooperation [24–26], process timing and sequence [27, 28], and the mental demands or interruptions associated with particular care processes [29, 21]. These changes are not passively accepted; often, clinicians respond to IT-driven change with their own changes to the system [20], including the development of work-arounds [30–32] and other adaptive strategies [33–35].

Health IT can have an indirect, yet, at times, considerable, effect on various outcomes of the care process such as medication safety and cost savings [36–39, 16]. In Fig. 1, it is clear that the addition of health IT only changes outcomes for the better (or worse) by first changing the work system, and consequently, the process of care; to think that health IT will automatically and directly improve safety or save money is considered by some to be “magical thinking” [40] because it ignores the mediating roles of system and process transformations [2, 4]. Similarly, with increasing pressure and funding for health IT implementation worldwide, we must keep in mind that the real test of success is not whether the health IT is physically or digitally present, because health IT does not deterministically render good results [41]. Instead, health IT success is contingent on the way in which it fits with [42, 9, 43] and transforms the work system, the way that the new work system supports the processes of care and the extent to which those processes yield favorable or unfavorable outcomes—in other words, is care better delivered?

Finally, following Karsh et al [4], the model proposes a central role for design. Design/redesign is applied when outcomes are not acceptable, and design constitutes a change that, like health IT, has the potential to transform the work system, process, and outcomes.

1.2. A study of transformational effects of bar coded medication administration (BCMA)

Measures can be made to separately assess health IT transformations of the work system, care process, and outcomes. Our five-year human factors study of BCMA technology in a pediatric hospital yielded measures of all three. Here we present findings pertaining to changes to the care process, particularly the process of medication administration. Other good examples of measures and studies of the impact of health IT— including BCMA—on care processes appear in other recent work [27, 44, 2, 45–47].

We were particularly interested in the effect of BCMA on accuracy, usefulness, consistency, time-efficiency and ease of performance of the medication process, as perceived by nurses involved in that process. Given the safety functions and purposes of BCMA [48–50], we also sought to measure if nurses perceived that the process became less prone to errors and that errors in the process could be more likely detected. To assess change, we surveyed nurses before and after the implementation of BCMA and concomitantly surveyed nurses in a comparison hospital, where BCMA was not implemented. Nurses evaluated three distinct processes of medication administration on the seven dimensions mentioned above (i.e., accuracy, usefulness, consistency, time-efficiency, ease of performance, error likelihood, error detection likelihood). The three processes were (1) matching medications to the medication administration record (MAR), (2) checking patient identification (ID), and (3) documenting medication administration. These processes are described in Table 1 along with related processes that were not assessed in surveys. Based on interviews and observations of the medication administration process before and after BCMA, we expected that BCMA would change some processes for the better, some for the worse, and others not at all, and this would be reflected in nurses’ perceptions.

Table 1.

General description of the process of medication administration pre- and post-BCMA at Hospital 1.a

Pre-BCMA Post-BCMA
1) Pre-administration Medications are ordered using computerized provider order entry (CPOE) and transcribed into pharmacy information system (IS) by pharmacist. Paper medication administration record (MAR) is printed from pharmacy IS once per day and updated by nurses with labels generated by pharmacy. Medications are ordered using computerized provider order entry (CPOE) and and transcribed into pharmacy information system (IS) by pharmacist. This updates or creates the nurses’ electronic MAR (eMAR). eMAR is updated by pharmacy throughout the day.
2) Identifying self/logging in n/a Nurse scans bar code on badge or logs in manually to identify self in system and access patient profile.
3) Matching medication to MAR Upon retrieving a medication, nurse compares the medication to what is printed in the MAR. Medications can also be checked to the order in CPOE. Upon retrieving a medication, nurse is required to scan medication bar code using scanning device (nurse can also manually select medication) to ensure it is the right medication for the selected patient. Medications can also be checked to the order in CPOE.
4) Checking patient identification (ID) Nurse checks patient ID by visually inspecting or by asking patient/patient family for identifying information. Nurse required to scan bar code on patient’s ID bracelet (nurse can also manually select patient).
5) Administering medication Nurse administers medication or leaves it with patient/family. Nurse administers medication or leaves it with patient/family.
6) Documenting medication administration Nurse initials scheduled time slot on MAR. If medication unscheduled or administration time differs from schedule, nurse writes time of administration. Nurse clicks “Submit” in BCMA, adds any notes, and provides mandatory explanation for late or missed medications.
7) Monitoring patient condition Nurse monitors patient condition/reactions post- administration. Nurse monitors patient condition/reactions post-administration.
Open in a new tab

BCMA = Bar coded medication administration

a

Ordering of steps is based on a common linear progression of events. Steps can be added, skipped, and carried out in various sequences.

2. Methods

The overall study used a repeated measures, nonequivalent groups quasi-experimental design [51] with measures taken once pre-BCMA and twice post-BCMA. Two hospitals participated. Hospital 1, a large pediatric hospital in the Midwest US underwent unit-by-unit BCMA implementation starting in December 2006, without any experimental intervention. Hospital 2, a large pediatric hospital in the South US served as the comparison site because BCMA was not implemented until after all measures were taken. Both hospitals were large, academic, free-standing, pediatric hospitals providing tertiary care in an urban setting. Both hospitals used computerized provider order entry (CPOE) and pharmacy checking of orders.

Data collection centered on the work of pharmacy and nursing staff at both hospitals and consisted of pre- and post-BCMA work system and process observations, interviews, observational medication error analyses, and surveys. Data presented here are only from two rounds of nurses’ surveys, one shortly before BCMA roll-out (November 2006, “PRE”) and one about one month after the last study unit was completely on BCMA (August 2007, “POST”). In addition, data from over 200 hours of observations and 68 short interviews with pharmacy workers and nurses at Hospital 1 are used in Section 4 to help interpret survey findings.

Surveys were paper based and were hand-delivered to all registered nurses with at least 24 hours per week of primarily direct care responsibilities in three units that were similar between hospitals: pediatric intensive care unit (PICU); hematology-oncology, and medical/surgical. Completed surveys were returned to and processed by an independent survey center to assure confidentiality of responses and respondents’ identities. More complete details of survey development and administration are available elsewhere [41].

2.1. Survey items

Twenty-one process perception items were analyzed for this study. Questions pertained to nurses’ perceptions of each of three medication processes, described in Table 1:

  • “matching medications to the MAR/medication profile” (Match to MAR)

  • “checking patient identification in order to administer medications” (Check ID)

  • “documenting administered medications” (Document)

Nurses were asked the following questions for each of the three medication processes:

  1. In actual practice, how accurate is the process of _____? (Accuracy)

  2. In actual practice, how useful is the process of _____? (Usefulness)

  3. In actual practice, to what extent is the process of _____ done the same way every time? (Consistency)

  4. In actual practice, how time-efficient is the process of _____? (Time efficiency)

  5. In actual practice, how easy is it to perform the process of _____? (Ease of performance)

  6. In actual practice, how likely is an error to occur through the process of _____? (Error likelihood)

  7. In actual practice, if an error occurs through the process of _____, how likely is it to be detected before it could lead to an adverse event? (Error detection likelihood)

Nurses responded on a 7-point Likert-like scale, from 0 = “Not at all” to 6 = “A great deal.” Nurses were prompted several times to “respond thinking about … the past 30 days.” Identical questions were asked in all versions of the survey: PRE and POST, at both hospitals. This allowed the analysis of change in nurses’ perceptions of the seven process characteristics.

2.2. Analysis

A general linear mixed model with repeated measures was carried out testing the effects on process perceptions of the following: HOSPITAL (Hospital 1 vs. 2), TIME (PRE vs. POST), COHORT (“True” cohort member vs. one-time participant), and HOSPITAL X TIME. All effects were modeled as fixed and between-subjects. Twenty-one models were constructed, one each for the seven process dimensions across the three medication processes. The alpha level was set to 0.05 and was not adjusted for multiple tests.

Estimating parameters for the full model, particularly for the HOSPITAL X TIME interaction, allowed a formal comparison of changes in perception between the two hospitals. However, there were concerns about the interaction effect not providing useful or interpretable information. One concern was that non-secular changes (e.g., in policy or training) at the comparison hospital might parallel changes in the BCMA hospital, eliminating an interaction effect. Similarly, non-secular changes at the comparison hospital might be in the opposite direction of those in the BCMA hospital, producing an uninterpretable interaction effect. Thus, whether or not the interaction was significant, the simple main effect of TIME was calculated for Hospital 1, to assess differences in process perception from PRE to POST, and the same was done for Hospital 2. For the simple main effects, Cohen’s d was calculated as a measure of effect size, using the pooled standard deviation calculated as the root mean square of the pre- and post-BCMA standard deviations [52]. Effect sizes can be interpreted in several ways. One option is to use Cohen’s [52] labels of “small” (d = .2), “medium” (d = .5) and “large” (d = .8), although those were developed based on studies of psychology students, and may not be applicable here. A second option is to consider the effect size as a description of overlap between pre- and post-BCMA distributions. This is possible because an effect size is exactly equivalent to a z-score of a standard normal distribution. Thus, for example, an effect size of 0.80 means that the score of the average nurse pre-BCMA exceeds the scores of 79% of nurses post-BCMA.

3. Results

Information on survey response rates and characteristics of respondents is presented in Table 2. Process perception changes are described in Figures 2 through 5 and in the text below.

Table 2.

Response rates and respondent characteristics.

Response rates
Hospital 1 Hospital 2
PRE POST PRE POST
121/203 (59.6%) 94/202 (46.5%) 78/144 (54.2%) 85/172 (49.4%)
Number of nurses who responded both PRE- and POST-BCMA (% of PRE-BCMA samplea) 56 (46.3%) 28 (35.9%)
Respondent characteristicsb
Gender, % female 95.7% 98.8%
Race, % white, not Hispanic 95.7% 90.6%
Education, % completing college 90.3% 97.6%
Age (%)c
 18–29 40.9% 62.4%
 30–39 28.0% 17.6%
 40–49 19.4% 15.3%
 50+ 11.9% 4.7%
Shift (%)c
 Day 47.3% 35.3%
 Evening/afternoon 9.7% 3.5%
 Night 25.8% 49.4%
 Other (e.g. weekends) 17.2% 11.8%
Hours/week, Mean (SD)d 31.67 (7.61) 35.33 (5.99)
Years in job, Mean (SD)d 8.22 (8.29) 4.04 (5.96)
Years on unit, Mean (SD)d 7.97 (8.03) 3.57 (5.29)
Years with employer, Mean (SD)d 8.77 (8.51) 4.26 (5.96)
Years in occupation, Mean (SD)d 10.84 (10.22) 6.58 (7.52)
Open in a new tab

BCMA = Bar coded medication administration

a

This percentage reflects attrition from the study due to non-response, drop-out from the sample (e.g., due to turnover), and unavailability POST-BCMA (e.g., due to maternity leave)

b

Calculated from POST-BCMA Time 2, calculated from nurses who provided responses

c

Hospital 1 differed from Hospital 2, p ≤ 0.05 (Pearson chi-square)

d

Hospital 1 differed from Hospital 2, p ≤ 0.05 (t-test, equal variances not assumed)

Figure 2.

Figure 2

Open in a new tab

Pre- and post-BCMA process perceptions and full model results for perceived (a) accuracy and (b) usefulness.

Figure 5.

Figure 5

Open in a new tab

Pre- and post-BCMA process perceptions and full model results for perceived (a) error likelihood and (b) error detection likelihood.

Figures 2 through 5 show the results of tests for changes from PRE- to POST-BCMA in nurses’ perceptions of the accuracy, usefulness, consistency, time efficiency, ease of performance, error likelihood, and error detection likely. At Hospital 1, where BCMA was implemented, significant changes in the process of matching medications to the MAR were: decreased error likelihood (F(1,362) = 5.32, p ≤ .05) and increased likelihood that an error occurring during that process would be detected (F(1,361) = 11.58, p ≤ .05). There were also significant changes in the perceptions of the process of checking patient ID at Hospital 1: increased accuracy (F(1,370) = 24.83, p ≤ .05); increased usefulness (F(1,369) = 4.77, p ≤ .05); increase consistency (F(1,371) = 27.70, p ≤ .05); decreased error likelihood (F(1,361) = 3.71, p ≤ .05); and increased error detection likelihood (F(1,358) = 4.41, p ≤ .05). Significant changes in the perceptions of the process of documenting medication administration at Hospital 1 were: decreased usefulness (F(1,368) = 5.67, p ≤ .05); decreased time efficiency (F(1,370) = 34.53, p ≤ .05); and decreased ease of performance (F(1,369) = 34.36, p ≤ .05).

At Hospital 2, the control hospital, perceptions of the process did not change from PRE- to POST-BCMA. Significant HOSPITAL X TIME effects (see Figures 2 through 5) confirmed that changes in process perceptions were greater at Hospital 1 than at Hospital 2.

In 9 of 21 models, the main effect of HOSPITAL was significant, indicating that there were, sometimes sizeable, differences in process perceptions between hospitals. In only 2 of 21 models, the main effect of COHORT was significant, indicating few differences in process perceptions between nurses who participated in only the PRE- or only the POST-BCMA survey versus nurses who participated in both. The cohort effect was generally negligible: two-thirds of the 21 models yielded an F ≤ 1 (mean F = 1.29).

4. Discussion

According to the human factors model of health IT impact presented herein (Fig. 1), the benefits of BCMA are contingent on the way in which BCMA transforms the work system and the work process [2]. Thus, rather than investigate the impact of BCMA on medication error rates or other “distal” outcomes, this study inspected a more immediate but theoretically important effect—the impact of BCMA on the medication administration process. Although a few studies exist reporting the desirable and undesirable effects of BCMA on nurses’ work processes (for review, see [6]), the present study is only the second to report quantitative changes in the medication administration process, along with Hurley’s [45]. The present study had much in common with Hurley’s, particularly the fact that both studies investigated changes in nurses’ perceptions of medication administration PRE- and POST-BCMA, although in the present study nurses recruited from a control hospital provided a needed comparison group. Both studies accord great importance to the perceptions of nurses, who are the primary actors in the administration process and the primary users of BCMA systems. However, a key difference between the two studies is that whereas Hurley found uniformly desirable changes in process perceptions, this study showed that when considering different medication administration processes separately, the effects of BCMA on work process are more complex and sometimes undesirable. Findings of those effects are discussed next. To put the findings in context and thus to help interpret them, we introduce some supplemental data obtained through over 200 hours of observation of medication management at Hospital 1 PRE- and POST-BCMA and 68 interviews with pharmacists, pharmacy technicians, and study unit nurses POST-BCMA.

4.1. Changes in medication administration processes

The desirable impact of BCMA on work process—improvement in perceived accuracy, usefulness, consistency, error likelihood, and error detection likelihood—was seen mostly for the process of checking patient ID. Accuracy improvements can be attributed to (1) the BCMA’s use of a multidigit patient ID number rather than of more confusable/less unique identifiers (e.g., patient name, birthdate, telephone number), and (2) the IT’s superiority in comparing those ID numbers to the expected database value [53]. Computerizing the checking ID process, not surprisingly, also improved perceived consistency. Consistency and accuracy improvements were probably responsible for the decrease in perceived error likelihood and increase in perceived error detection likelihood, producing a process less susceptible to confirmation bias, perceptual degradation, memory failures, the effects of time pressure, and other problems associated with human verification of visual information [54–56]. As a result, it is not surprising that the perceived usefulness of the checking ID process, whose aim is to prevent wrong patient errors, increased at Hospital 1. Improvements in process ease and time efficiency were much smaller and not statistically significant; still, that means that using bar code scanning of patient ID need not be “tedious and time-consuming” [57]. Of course, the benefits of BCMA to both usefulness and ease/efficiency may have been more pronounced if not for observed challenges such as computer breakdowns and usability problems, short tethers on desktop bar code scanners, and the inaccessibility of patients’ ID bands (e.g., when bundled in blankets), which prevented the patient ID from being (easily) scanned. Confronted with such challenges, nurses were found in this study [58–60, 33] and in others [30, 46] to skip or work around the process for checking patient ID.

The undesirable changes—reductions in perceived usefulness, time efficiency, and ease of performance—were seen only for the documentation process. One might imagine that with BCMA, the documentation process would be straightforward and easy, a matter of scanning and perhaps a single mouse click for confirmation, which would automatically populate the patient’s electronic MAR (eMAR) with information about the time of administration, the medication name, dose, and type/route, and who was the administering nurse. Observations revealed that such a process indeed occurred for simple and uncomplicated cases of medication administration. However, documentation became potentially difficult any time that the administration of a medication became non-routine, such as when a patient was off the floor [58, 33] or when there was a lack of accessible intravenous (IV) lines. As a result, nurses had to do extra documentation during high-workload periods (e.g., post-procedure) using an unfamiliar software interface that appeared to have usability problems (e.g, lack of feedback). There was also the problem of documenting administrations from prior shifts, an intrusive and sometimes impossible task, not to mention one that yielded potentially inaccurate and thus less useful information. Finally, during observations and interviews nurses expressed uncertainty about whether anyone was reading their documentation. The above problems and several others [33] help to explain the drastic reductions in the perceived ease of performance and time efficiency of the documentation process at Hospital 1 and the smaller reductions in its perceived accuracy and usefulness.

For the most part, nurses’ perceptions of the process of matching (i.e., checking) the medication to the MAR did not change from PRE- to POST-BCMA. From observations, the major qualitative change in that process was the use of an eMAR rather than a paper MAR. An obvious advantage of an eMAR is that it can be updated more quickly and more permanently by pharmacists, compared to the pre-BCMA paper version that was printed once per day and updated mostly by pasting in new stickers delivered by pharmacy. The eMAR also may have been more legible than the printed, but sometimes hand-annotated, paper MAR. Those differences might account for the decreased perceived likelihood of error and the increased perceived likelihood of error detection for the matching to MAR process. The small, non-significant reduction in time efficiency for that process may be explained by the possibility that the paper MAR was easily accessible, often placed inside or just outside the patient’s room and its access was not delayed by computer slowness or log-in time.

As Fig. 1 implies, observed undesirable changes are not merely nuisance “side effects” but rather the very changes that determine key outcomes such as patient safety. A similar relationship can be drawn between desirable changes and desirable outcomes. Indeed, we might hypothesize based on research on human behavior [61–63], health care process acceptance [64, 65], and BCMA [45, 31] that favorable process perceptions resulted in greater acceptance of the process by nurses, in improved job satisfaction, and in more effective patient care, whereas less favorable perceptions resulted in lower acceptance and in greater likelihood of procedural violations during the process. Another important interpretation of the findings based on the human factors model in Fig. 1 is that observed (perceived) process changes resulted from specific changes in the work system created by the implementation of BCMA. Identifying those work system changes, both good and bad, can help to produce design and redesign guidelines.

4.2. Limitations

The subjects in the PRE and POST data were not all the same subjects. We took steps to handle this methodologically and analytically. All subjects were tracked with unique identification numbers so we knew exactly which subjects were in both samples and which were only in one. We also introduced a COHORT variable in the models to determine any effects of membership in both groups and to adjust all estimates for variability due to COHORT. The COHORT variable had little impact on our results. This study analyzed data that were self-reports, which could be perceived as a limitation to the extent that individual perceptions are not reflective of reality. However, it is perceptions that drive behavior [66]. Three medication administration processes were chosen, those that were expected to change with BCMA, but other processes may have been affected as well (see Table 1). Analytic limitations included the testing of multiple models without adjusting for family-wise increases in the probability of Type I error. Because we only studied one hospital’s implementation of BCMA, it remains to be seen whether the theory and findings presented here would transfer to other hospitals or health ITs.

5. Conclusion

In answer to the question, “what does health IT do?” a human factors approach suggests that IT changes work, transforming the work system and work processes and indirectly transforming the outcomes. The present study showed evidence of the process transformations that came about from a hospital work system that was changed considerably by the introduction of BCMA. Process changes were not uniformly desirable, as reported elsewhere [45]; some mirrored the BCMA process problems described in qualitative studies [30, 46, 31]. Both process problems and improvements can be linked to BCMA-related transformations of the work system (and, of course, those are shaped by the design and implementation of the IT [67]). Fortunately, such work system transformations are measurable and also controllable through design. The knowledge and tools amassed in the field of human factors engineering/psychology will greatly inform and facilitate both the measurement and design of those transformations.

Figure 3.

Figure 3

Open in a new tab

Pre- and post-BCMA process perceptions and full model results for perceived consistency.

Figure 4.

Figure 4

Open in a new tab

Pre- and post-BCMA process perceptions and full model results for perceived (a) time efficiency and (b) ease of performance.

Acknowledgments

The authors wish to thank the study participants and colleagues involved in the Bar Coding study and the anonymous reviewers for their comments. This work was supported by grants from the US Agency for Healthcare Research and Quality (AHRQ) (1 R01 HS013610) to author BK and the US National Institutes of Health (1 R01 LM008923) to author BK. RJH is supported by a post-doctoral training grant from AHRQ (5 T32 HS000083-11). The granting agencies had no role in the study or publication except for providing funding for the study.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Bates DW, Gawande AA. Patient safety: Improving safety with information technology. N Engl J Med. 2003;348:2526–34. doi: 10.1056/NEJMsa020847. [DOI] [PubMed] [Google Scholar]
  • 2.Holden RJ. Cognitive performance-altering effects of electronic medical records: An application of the human factors paradigm for patient safety. Cogn Technol Work. 2010 doi: 10.1007/s10111-010-0141-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Holden RJ, Scanlon MC, Brown RL, Karsh B. What is IT? New conceptualizations and measures of pediatric nurses’ acceptance of bar-coded medication administration information technology. Annual Meeting of the Human Factors and Ergonomics Society. 2008;52:768–72. [Google Scholar]
  • 4.Karsh B, Holden RJ, Alper SJ, Or CKL. A human factors engineering paradigm for patient safety – designing to support the performance of the health care professional. Qual Saf Health Care. 2006;15:i59–i65. doi: 10.1136/qshc.2005.015974. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Woods DD, Dekker SWA. Anticipating the effects of technological change: A new era of dynamics for human factors. Theor Issues Ergon Sci. 2000;1:272–82. [Google Scholar]
  • 6.Karsh B, Wetterneck T, Holden RJ, Rivera AJ, Faye H, Scanlon M, et al. Bar coding in medication administration. In: Yih Y, editor. Handbook of Healthcare Delivery Systems. Boca Raton, FL: CRC Press; in press. [Google Scholar]
  • 7.Carayon P, Schoofs Hundt A, Karsh B, Gurses AP, Alvarado CJ, Smith M, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15:i50–i8. doi: 10.1136/qshc.2005.015842. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Holden RJ, Karsh B. A review of medical error reporting system design considerations and a proposed cross-level system research framework. Hum Factors. 2007;49:257–76. doi: 10.1518/001872007X312487. [DOI] [PubMed] [Google Scholar]
  • 9.Holden RJ, Karsh B. A theoretical model of health information technology usage behaviour with implications for patient safety. Behav Inf Technol. 2009;28:21–38. [Google Scholar]
  • 10.Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260:1743–8. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
  • 11.Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform. 2007;76S:S21–S7. doi: 10.1016/j.ijmedinf.2006.05.017. [DOI] [PubMed] [Google Scholar]
  • 12.Barber N, Cornford T, Klecun E. Qualitative evaluation of an electronic prescribing and administration system. Qual Saf Health Care. 2007;16:271–8. doi: 10.1136/qshc.2006.019505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Battles JB. Patient safety and technology, a two-edged sword. In: Carayon P, editor. Handbook of Human Factors and Ergonomics in Patient Safety. Mahwah, NJ: Lawrence Erlbaum; 2007. pp. 383–91. [Google Scholar]
  • 14.Cornford T, Doukidis GI, Forster D. Experience with a structure, process and outcome framework for evaluating an information system. International Journal of Management Science. 1994;22:491–504. [Google Scholar]
  • 15.Hebert M. Telehealth success: Evaluation framework development. MEDINFO. 2001;84:1145–9. [PubMed] [Google Scholar]
  • 16.Shekelle PG, Morton SC, Keeler EB. Costs and Benefits of Health Information Technology. Evidence Report/Technology Assessment No. 132. Rockville, MD: Agency for Healthcare Research and Quality; 2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Carayon P, Alvarado CJ, Hundt AS. Work design and patient safety. Theor Issues Ergon Sci. 2007;8:395–428. [Google Scholar]
  • 18.Karsh B, Escoto KH, Beasley JW, Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon. 2006;37:283–95. doi: 10.1016/j.apergo.2005.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Perry SJ, Wears RL, Cook RI. The role of automation in complex system failures. J Patient Saf. 2005;1:56–61. [Google Scholar]
  • 20.Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--An interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14:542–9. doi: 10.1197/jamia.M2384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Hertzum MSJ. Positive effects of electronic patient records on three clinical activities. Int J Med Inform. 2008;77:809–17. doi: 10.1016/j.ijmedinf.2008.03.006. [DOI] [PubMed] [Google Scholar]
  • 22.Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197–203. doi: 10.1001/jama.293.10.1197. [DOI] [PubMed] [Google Scholar]
  • 23.Young D. CPOE reduces number of steps in medication-use process. 2001 doi: 10.1093/ajhp/58.13.1170. [cited 2009 Aug 1]; Available from: http://www.ashp.org/import/news/HealthSystemPharmacyNews/newsarticle.aspx?id=612. [DOI] [PubMed]
  • 24.Beuscart-Zephir MC, Pelayo S, Anceaux F, Meaux J-J, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int J Med Inform. 2005;74:629–41. doi: 10.1016/j.ijmedinf.2005.01.004. [DOI] [PubMed] [Google Scholar]
  • 25.Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13:547–56. doi: 10.1197/jamia.M2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Cheng CH, Goldstein MK, Geller E, Levitt RE. The effects of CPOE on ICU workflow: An observational study. Proceedings of the Symposium of the American Medical Informatics Association; 2003. pp. 150–4. [PMC free article] [PubMed] [Google Scholar]
  • 27.Carayon P, Wetterneck TB, Hundt AS, Ozkaynak M, DeSilvey J, Ludwig B, et al. Evaluation of nurse interaction with bar code medication administration technology in the work environment. J Patient Saf. 2007;3:34–42. [Google Scholar]
  • 28.Han YY, Carcillo JA, Venkataraman ST, Clark RSB, Watson RS, Nguyen TC, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–12. doi: 10.1542/peds.2005-1287. [DOI] [PubMed] [Google Scholar]
  • 29.France DJ, Levin S, Hemphill R, Chen K, Rickard D, Makowski R, et al. Emergency physicians’ behaviors and workload in the presence of an electronic whiteboard. Int J Med Inform. 2005;74:827–37. doi: 10.1016/j.ijmedinf.2005.03.015. [DOI] [PubMed] [Google Scholar]
  • 30.Koppel R, Wetterneck TB, Telles JL, Karsh B. Workarounds to barcode medication administration systems: Their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc. 2008;15:408–23. doi: 10.1197/jamia.M2616. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Patterson ES, Rogers ML, Chapman RJ, Render ML. Compliance with intended use of bar code medication administration in acute and long-term care: An observational study. Hum Factors. 2006;48:15–22. doi: 10.1518/001872006776412234. [DOI] [PubMed] [Google Scholar]
  • 32.Vogelsmeier AA, Halbesleben JRB, Scott-Cawiezzel JR. Technology implementation and workarounds in the nursing home. J Am Med Inform Assoc. 2008;15:114–9. doi: 10.1197/jamia.M2378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Holden RJ, Alper SJ, Scanlon MC, Murkowski K, Rivera AJ, Karsh B. Challenges and problem-solving strategies during medication management: A study of a pediatric hospital before and after bar-coding. Proceedings of the 2nd International Conference on Healthcare Systems Ergonomics and Patient Safety; 2008; Strasbourg, France. 2008. [Google Scholar]
  • 34.Wears RL, Cook RI, Perry SJ. Automation, interaction, complexity, and failure: A case study. Reliability Engineering and System Safety. 2006;91:1494–501. [Google Scholar]
  • 35.Wears RL, Perry SJ, Wilson SJ, Galliers J, Fone J. Emergency department status boards: user-evolved artefacts for inter- and intra-group coordination. Cogn Technol Work. 2007;9:163–70. [Google Scholar]
  • 36.Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: A systematic review. J Am Med Inform Assoc. 2008;15:585–6. doi: 10.1197/jamia.M2667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Chaudhry B, Wu S, Maglione M, Mojica W, Roth E, Morton SC, et al. Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:E12–E22. doi: 10.7326/0003-4819-144-10-200605160-00125. [DOI] [PubMed] [Google Scholar]
  • 38.Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, et al. Return on investment for a computerized physician order entry system. J Am Med Inform Assoc. 2006;13:261–6. doi: 10.1197/jamia.M1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Poon EG, Cina JL, Churchill W, Patel N, Featherstone E, Rothschild JM, et al. Medication dispensing errors and potential adverse drug events before and after implementing bar code technology in the pharmacy. Annals of Internal Medicine. 2006;145:426–34. doi: 10.7326/0003-4819-145-6-200609190-00006. [DOI] [PubMed] [Google Scholar]
  • 40.Diamond CC, Shirky C. Health information technology: A few years of magical thinking? Health Aff. 2008;27:w383–w90. doi: 10.1377/hlthaff.27.5.w383. [DOI] [PubMed] [Google Scholar]
  • 41.Karsh B, Holden RJ, Escoto KH, Alper SJ, Scanlon MC, Arnold J, et al. Do beliefs about hospital technologies predict nurses’ perceptions of quality of care? A study of task-technology fit in two pediatric hospitals. Int J Hum Comput Interact. 2009;25:374–89. [Google Scholar]
  • 42.Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: A fit framework and a case study. BMC Med Inform Decis Mak. 2006:6. doi: 10.1186/1472-6947-6-3. http://www.biomedcentral.com/1472-6947/6/3. [DOI] [PMC free article] [PubMed]
  • 43.Kaplan B. Evaluating informatics applications—some alternative approaches: Theory, social interactionism, and call for methodological pluralism. Int J Med Inform. 2001;64:39–56. doi: 10.1016/s1386-5056(01)00184-8. [DOI] [PubMed] [Google Scholar]
  • 44.Dykes PC, Hurley A, Cashen M, Bakken S, Duffy ME. Development and psychometric evaluation of the Impact of Health Information Technology (I-HIT) Scale. J Am Med Inform Assoc. 2007;14:507–14. doi: 10.1197/jamia.M2367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Hurley AC, Bane A, Fotakis S, Duffy ME, Sevigny A, Poon EG, et al. Nurses’ satisfaction with medication administration point-of-care technology. Journal of Nursing Administration. 2007;37:343–9. doi: 10.1097/01.nna.0000285114.60689.02. [DOI] [PubMed] [Google Scholar]
  • 46.Patterson ES, Cook RI, Render ML. Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc. 2002;9:540–53. doi: 10.1197/jamia.M1061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Poon EG, Keohane CA, Bane A, Featherstone E, Hays BS, Dervan A, et al. Impact of barcode medication administration technology on how nurses spend their time providing patient care. Journal of Nursing Administration. 2008;38:541–9. [PMC free article] [PubMed] [Google Scholar]
  • 48.Cescon DW, Etchells E. Barcoded medication administration - A last line of defense. JAMA. 2008;299:2200–2. doi: 10.1001/jama.299.18.2200. [DOI] [PubMed] [Google Scholar]
  • 49.Engelbright JD, Franklin M. Managing a new medication administration process. Journal of Nursing Administration. 2005;35:410–3. doi: 10.1097/00005110-200509000-00011. [DOI] [PubMed] [Google Scholar]
  • 50.Institute of Medicine. Preventing Medication Errors. Washington, DC: National Academies Press; 2007. [Google Scholar]
  • 51.Cook TD, Campbell DT. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton Mifflin; 1979. [Google Scholar]
  • 52.Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
  • 53.The Joint Commission. Two Patient Identifiers. 2008 [cited 2009 October 4]; Available from: http://www.jointcommission.org/AccreditationPrograms/LongTermCare/Standards/09_FAQs/NPSG/Patient_ID/NPSG.01.01.0/Two_Patient_Identifiers.htm.
  • 54.Reason J. Human Error. New York: Cambridge University Press; 1990. [Google Scholar]
  • 55.Schachter DL. The Seven Sins of Memory: How the Mind Forgets and Remembers. New York: Haughton Mifflin; 2002. [Google Scholar]
  • 56.Wickens CD, Lee JD, Liu Y, Gordon-Becker S. An Introduction to Human Factors Engineering. 2. Englewood Cliffs, NJ: Prentice Hall; 2003. [Google Scholar]
  • 57.Ketchum B. INFORMATION TECHNOLOGY: IT equals better patient care? Nursing Management. 2008;39:21–3. doi: 10.1097/01.NUMA.0000320634.89059.9a. [DOI] [PubMed] [Google Scholar]
  • 58.Alper SJ, Holden RJ, Scanlon MC, Patel N, Murkowski K, Shalaby TM, et al. Violation Prevalence After Introduction of a Bar Coded Medication Administration System. 2nd International Conference on Healthcare Systems Ergonomics and Patient Safety; 2008; Strasbourg, France. 2008. [Google Scholar]
  • 59.Alper SJ, Karsh B, Holden RJ, Scanlon M, Patel N, Kaushal R. Protocol violations during medication administration in pediatrics. Human Factors and Ergonomics Society 50th Annual Meeting; 2006; San Francisco: Human Factors and Ergonomics Society; 2006. [Google Scholar]
  • 60.Alper SJ, Scanlon MC, Murkowski K, Patel N, Kaushal R, Karsh B. Routine and situational violations during medication administration. In: Sznelwar L, Mascia F, Montedo U, editors. 9th International Symposium on Human Factors in Organizational Design and Management 2008. Guarujá, São Paulo; Brazil: 2008. [Google Scholar]
  • 61.Ajzen I. The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes. 1991;50:179–211. [Google Scholar]
  • 62.Eagly AH, Chaiken S. The Psychology of Attitudes. Belmont, CA: Wadsworth; 1993. [Google Scholar]
  • 63.Fishbein M, Ajzen I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley; 1975. [Google Scholar]
  • 64.Ammenwerth E, Mansmann U, Iller C, Eichstädter R. Factors affecting and affected by user acceptance of computer-based nursing documentation: Results of a two-year study. J Am Med Inform Assoc. 2003;10:69–84. doi: 10.1197/jamia.M1118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Campbell-Heider N, Knapp TR. Nurses’ attitudes toward conventional and automated vital signs measurement methods. Medical Instrumentation. 1988;22:257–62. [PubMed] [Google Scholar]
  • 66.Ajzen I. Attitudes, Personality, and Behavior. Chicago: Dorsey Press; 1988. [Google Scholar]
  • 67.Karsh B, Holden RJ. New technology implementation in health care. In: Carayon P, editor. Handbook of Human Factors and Ergonomics in Patient Safety. Mahwah, NJ: Lawrence Erlbaum; 2007. pp. 393–410. [Google Scholar]

RESOURCES