Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Dec 1.
Published in final edited form as: Int J Med Inform. 2011 Oct 21;80(12):863–871. doi: 10.1016/j.ijmedinf.2011.10.003

Barriers and Facilitators to the Use of Computer-based Intensive Insulin Therapy

Thomas R Campion Jr 1,2, Lemuel R Waitman 3, Nancy M Lorenzi 1, Addison K May 4, Cynthia S Gadd 1
PMCID: PMC3226863  NIHMSID: NIHMS333951  PMID: 22019280

Abstract

Purpose

Computerized clinical decision support systems (CDSSs) for intensive insulin therapy (IIT) are increasingly common. However, recent studies question IIT’s safety and mortality benefit. Researchers have identified factors influencing IIT performance, but little is known about how workflow affects computer-based IIT. We used ethnographic methods to evaluate IIT CDSS with respect to other clinical information systems and care processes.

Methods

We conducted direct observation of and unstructured interviews with nurses using IIT CDSS in the surgical and trauma intensive care units at an academic medical center. We observed 49 hours of intensive care unit workflow including 49 instances of nurses using IIT CDSS embedded in a provider order entry system. Observations focused on the interaction of people, process, and technology. By analyzing qualitative field note data through an inductive approach, we identified barriers and facilitators to IIT CDSS use.

Results

Barriers included (1) workload tradeoffs between computer system use and direct patient care, especially related to electronic nursing documentation, (2) lack of IIT CDSS protocol reminders, (3) inaccurate user interface design assumptions, and (4) potential for error in operating medical devices. Facilitators included (1) nurse trust in IIT CDSS combined with clinical judgment, (2) nurse resilience, and (3) paper serving as an intermediary between patient bedside and IIT CDSS.

Conclusion

This analysis revealed sociotechnical interactions affecting IIT CDSS that previous studies have not addressed. These issues may influence protocol performance at other institutions. Findings have implications for IIT CDSS user interface design and alerts, and may contribute to nascent general CDSS theory.

Keywords: blood glucose, clinical decision support systems, critical care, electronic medical records, ethnography, hyperglycemia, hypoglycemia, insulin

Introduction

Intensive insulin therapy (IIT) is the standard of critical care, but recent studies raise questions about the therapy’s mortality benefit and risk to patients [1-4]. To maintain tight blood glucose (BG) control, IIT requires nurses to perform frequent BG measurements, usually using handheld testing devices [5], and adjust insulin infusion pumps according to protocol logic [6]. Increasingly nurses deliver IIT using computerized clinical decision support systems (CDSSs) [7], which studies have deemed effective for controlling hyperglycemia and safe for achieving low rates of hypoglycemia [8]. Researchers have identified sources of variability affecting IIT performance—patient populations, BG target ranges, nutrition sources, nurse staffing, and genetic factors [5]—and the impact of paper-based IIT protocols on nurse workflow [9-10]. However, few investigations have formally assessed the workflow complexity introduced by IIT CDSS [8] or how nurses perceive the technology’s role in patient care [11-12].

Previous studies quantified the effect of nurse workflow on IIT CDSS performance. Researchers identified relationships between late timing of BG measurements and hyper- [13] and hypoglycemic episodes [13-14] as well as between blood glucose variability and mortality [15]. In a prior investigation, we showed that 5.3% of BG values manually transcribed to IIT CDSS did not match corresponding source values from handheld testing devices, which affected CDSS alerts and dosing recommendations as well as BG variability [16]. Investigators reported rates of nurse IIT CDSS override, or deviation from system recommendations, ranging from 2% to 23% [11, 14, 17-19]. Previously we demonstrated that 83.4% of IIT CDSS override doses were less than recommended by the system [17], quantifying nurse “fear of hypoglycemia” [20]. Additionally, categorization of IIT CDSS free text comments at one institution showed that nurses chose to override system recommendations due to fear of hypoglycemia [11], dose disagreement [11, 18], and workflow factors [11]. Timing, mismatched data, and overrides affect IIT CDSS performance.

Although prior studies described IIT CDSS use, they relied primarily on system log data that may not fully capture the “assumptions, norms, values, choices, and interactions” involved in decision making [21]. Few studies of IIT CDSS [8], and CDSS in general [22], have evaluated interventions with respect to social, organizational, and contextual characteristics of use including interactions with other clinical information systems and care processes. A descriptive, exploratory approach is appropriate for investigating such issues [23]. Using naturalistic methods [23], the goal of this study was to illuminate barriers and facilitators to use of IIT CDSS.

Methods

We conducted a qualitative study of IIT CDSS by directly observing and interviewing clinical personnel in the 21-bed surgical intensive care unit (SICU) and 31-bed trauma intensive care unit (TICU) at Vanderbilt University Hospital, an urban tertiary care academic facility in Nashville, Tennessee. For nearly two decades, researchers and staff at Vanderbilt University Hospital have developed and implemented clinical information systems to facilitate quality improvement (Figure 1).

Figure 1.

Figure 1

Open in a new tab

Architecture of clinical information systems developed and implemented in SICU and TICU at Vanderbilt University Hospital. All systems were in use at the time of study. Arrows indicate bidirectional data transfer. Users accessed clinical applications and specific application modules embedded within them from workstations connected to the hospital network located on mobile carts, at nursing stations, and in patient rooms at the bedside. The clinical data repository stored data generated by clinical applications and their modules. For implementation details, see [8] for IIT CDSS and [24] for dashboard. MARS was originally developed at the University of Pittsburgh in the mid-1980s.

As a result, use of clinical information systems has become an established part of clinician culture. Although other intensive care units at the institution treated patients using IIT CDSS, we focused our study on SICU and TICU due to common management by Division of Trauma and Surgical Critical Care faculty. 35.1% (n=1,883) of SICU patients and 26.3% (n=2,152) of TICU patients received IIT CDSS between November 2004 and February 2009 [16]. Compared to a previous paper protocol, IIT CDSS improved glycemia control in SICU and TICU [25-26].

IIT CDSS description

After a patient’s blood glucose level exceeded 110 mg/dL and a physician initiated the intensive insulin therapy protocol [25], a nurse used IIT CDSS embedded in the provider order entry system to maintain tight BG control between 80 and 110 mg/dL by default (Figure 2). To calculate a recommendation, IIT CDSS first required a nurse to enter a BG value obtained from a handheld testing device via keyboard, select a dextrose source, and specify the site and method of blood draw. IIT CDSS then calculated insulin rates using a linear equation [27-28]: insulin dose (units/hour) = (BG in mg/dL − 60) * multiplier [25]. The multiplier, an estimate of a patient’s insulin resistance based on current and previous BG measurements, started at 0.03, increased by 0.01 when BG levels indicated hyperglycemia, and decreased by 0.01 or 0.02 depending on degree of hypoglycemia; it could not be less than zero [25]. To maintain BG control, nurses performed IIT CDSS every two hours, although timing varied in practice [13]. BG values less than 80 mg/dL triggered an order for 50% dextrose (D50W) and checking of BG in thirty minutes [25]. Additionally, BG values less than 60 mg/dL generated instructions for nurses to suspend insulin administration for thirty minutes [25]. A nurse could view IIT CDSS’s rationale by clicking the “Explain Recommendations” button. If a nurse wished to override an insulin recommendation, he or she replaced the numerical figure next to “New Insulin Infusion Rate” with a different value [17]. IIT CDSS logged all on screen input including designation of override doses. Using the values recorded by IIT CDSS, a nurse adjusted an Alaris® infusion pump and/or administered D50W via injection. Infusion pumps were capable of exporting data electronically but the feature was not enabled during the study. IIT CDSS did not remind nurses to perform BG tests or administer insulin.

Figure 2.

Figure 2

Open in a new tab

IIT CDSS interface used by nurses. Similar to interfaces of other decision support modules in the provider order entry system, IIT CDSS displayed colorful numbers to guide users through a sequence of operations.

Under certain circumstances the care team wished to temporarily regulate a patient’s BG through other means (e.g. during an operating room procedure according to surgery protocol) but not discontinue orders for IIT CDSS. When a patient returned to BG control through IIT CDSS, a nurse clicked the “Patient Has Been Off Protocol” checkbox and provided a reason as well as the previous BG and current insulin infusion rate used in the alternate BG regulation strategy. IIT CDSS then used these figures and a current BG value to calculate recommendations.

Data collection

A researcher trained in ethnographic methods (TRC) collected data by observing nurse workflow and conducting unstructured interviews with nurses and other clinicians between February 16, 2010 and March 18, 2010 [17]. In conducting observations, the researcher paid attention to all activities encountered by a nurse including direct patient care, computer use, and interactions with clinicians, staff, and patients’ families. The researcher performed unstructured interviews to clarify observations and in response to comments volunteered by clinicians. Prior to observations and interviews, the researcher briefly explained the project and obtained verbal assent from clinicians, patients, and families as necessary. Participation was voluntary and responses were confidential. The Vanderbilt University Institutional Review Board approved this study.

The researcher recorded de-identified notes using a pen and paper before transcribing them electronically for further analysis. Notes consisted of narrative text describing observations as well as direct quotes from clinical personnel. Data collection started in SICU because it was the original IIT CDSS implementation site, moved to TICU, and then shifted between the two as needed. Informed by pilot study experience [8], a typical observation period lasted two-to-three hours to account for nurses performing IIT CDSS at two hour intervals by default. The goal of each session was to follow one nurse who performed at least two IIT CDSS iterations. In some sessions the researcher followed multiple nurses. The researcher observed nurses on weekdays between 9:00am and 7:00pm as well as on one weeknight and one weekend day. In total the researcher observed 49 hours of SICU and TICU workflow in which nurses used IIT CDSS 49 times (Table I). As described elsewhere [16], we also had access to a database of 52 months of retrospectively collected IIT CDSS records for quantitatively examining trends seen in observations.

Table I.

Characteristics of study observations

SICU TICU
Time 25 hours 25 minutes 23 hours 35 minutes
Sessions 11 9
Nurses observed (unique) 16 (13) 14 (12)
Patient: nurse ratio 2:1 (1:1) 11 (1) 8 (1)
Nurses interviewed but not observed 2 5
Patients observed (on IIT CDSS) 25 (16) 21 (16)
IIT CDSS iterations observed 22 27
Open in a new tab

Data analysis

Informed by grounded theory [29], the researcher inductively examined observation and interview notes using the constant comparative method to allow themes to emerge from data. Using NVivo 8, a software package for qualitative analysis, the researcher coded notes line-by-line by labeling concepts and iteratively refined associations between concepts. As part of the analysis process, the researcher recorded memos to reflect on observations, the coding process, and emergent themes. Coding and memoing occurred immediately following each data collection session. Concepts that emerged from data analysis informed the direction of future data collection sessions, a process called theoretical sampling [29]. Data collection and analysis ceased when no new data emerged, a point called data saturation [29].

This study ensured analytical rigor by employing standard techniques of naturalistic inquiry [30]. Throughout the study the researcher confirmed observations and emerging concepts through opportunistic interviews with nurses in the study sites, a process called member checking [30]. Additionally, the researcher met with a peer debriefer [30], an informatician trained in ethnography who was not involved in the study, to confer on data emergence and alternative approaches to data collection and analysis. The researcher also triangulated [30] data sources—the multiple nurses observed across two ICUs; critical care personnel not observed including a nurse manager, an attending, a fellow, and a pharmacist; and retrospective database records—and methods—direct observation, unstructured interviews, and database queries—in order to verify IIT CDSS findings from multiple perspectives. Finally, the researcher maintained reflexivity [30], or awareness of self and bias in collecting data, through regular journaling exercises in order to limit interference with data collection and analysis.

Results

We identified four barriers and three facilitators to IIT CDSS use from analysis of observation and interview data. Examination of retrospective database records confirmed observations when needed.

Barrier 1: workload tradeoffs between computer use and direct patient care

Time and effort demands of electronic nursing documentation

Nurses expressed concern about the time required to use HED, the electronic nursing documentation system, and cited its use as an occasional impediment to patient care. Depending on patient acuity, nurses prioritized patient care ahead of computer system usage. As a result, nurses spent considerable time “backcharting” care they provided in the past. HED hindered efficiency by requiring double documentation of data points unrelated to IIT, a source of complaints from nurses. Nurses recognized HED use as a necessary activity for regulatory and legal purposes but questioned the utility of perfunctory documentation. Although infusion pumps and pulmonary artery catheter monitors were capable of electronic data output, the institution had not yet pursued automatic data integration from these specific sources, so nurses manually entered values from the devices into HED, which increased time spent documenting. Nurses appreciated the automatic data transfer of vital signs from bedside monitors to HED; the safety provided by Admin-Rx, the barcode medication administration system, despite additional time required; and the display of hour-by-hour medications in Care Organizer, the medication scheduling tool.

Nurse suspicion of dashboard

The dashboard was a desktop screensaver that displayed color-coded protocol compliance indicators based on nursing documentation system and electronic medical record data. Initially well-received by nurses, the dashboard became viewed as a managerial surveillance tool promoting documentation compliance at the occasional expense of patient care. Skepticism of the dashboard increased after the addition of falls prevention parameters to the display. Complaining of alert fatigue, one nurse said, “We’re near the breaking point where people stop paying attention to [the dashboard] and it loses its functionality.” Nurses stated the benefit of the dashboard accrued to unit and hospital management rather than nurses and patients. Instead of the system providing beneficial reminders, nurses said the dashboard prompted charge nurses to constantly monitor compliance and urge general care nurses to complete tasks to avoid display of non-compliance indicators on the dashboard. General care nurses believed the emphasis on achieving dashboard measure compliance detracted from their ability to appropriately deliver patient care, especially when caring for multiple complex patients whose needs may not be represented fully by dashboard indicators. Over time nurse perception of the dashboard as a platform for care reminders changed from helpful to intrusive.

Time and effort demands of IIT CDSS

Occurring at two hour intervals or less, computer-based intensive insulin therapy occupied a significant portion of nurses’ time. Treating hypoglycemia required more time from nurses to administer D50W and monitor patients [16]. While preparing to administer D50W and recheck BG in thirty minutes according to IIT CDSS recommendations following a BG level of 79 mg/dL, a nurse remarked “I’ll soon enough be chasing my tail the other way” when the protocol would resume its regular schedule and focus on hyperglycemia control. With care tasks adding up, nurses often remarked, “I wish someone would do my sugars!” On several occasions we observed an idle nurse offer to help another nurse with BG testing for IIT CDSS. Additionally, when nurses deemed other care needs more pressing than BG regulation (e.g. for patients with stable BG levels), IIT CDSS usage became less of a priority [16]. Given time pressures, nurses continually balanced direct patient care with electronic nursing documentation and IIT CDSS usage.

Double documentation in electronic nursing documentation and IIT CDSS

IIT CDSS stored BG, insulin, D50W, and dextrose source data in the electronic medical record for each therapy iteration based on nurse input, but nurses also manually recorded these parameters in HED, a form of double documentation [16]. Although IIT CDSS eliminated double documentation of values inherent to the previous paper-based protocol, HED, which was installed after IIT CDSS, re-introduced double documentation by requiring input of the same data already entered by nurses into IIT CDSS. Furthermore, nurses recorded insulin data twice in HED—once for rate and once for volume.

Barrier 2: lack of IIT CDSS protocol reminders

Insulin and D50W doses generated by IIT CDSS appeared in Care Organizer, the medication scheduling tool, after their administration because Care Organizer used the most recently processed medication orders to populate the list. At the time of study, Care Organizer displayed medications scanned using the Admin-Rx barcode medication administration system. Nurses scanned intravenous infusions, like the insulin infusions used for IIT, only when they were first administered.

Box 1 presents a case where Care Organizer did not accurately display IIT CDSS orders. In this example, nurse resilience maintained IIT compliance despite clinical information system rigidity. However, the researcher also observed a nurse almost forget to recheck a patient’s BG 30 minutes after administering D50W and another nurse nearly fail to check BG after three hours, the definition of overdue for the standard two hour interval [13]. Both times the researcher inquired about IIT CDSS to jog the nurses’ memory, which led the nurse to carry out the therapy.

Box 1. Example demonstrating lack of reminders through Care Organizer configuration.

After using IIT CDSS at 10:02am, an order for insulin for the amount recorded by IIT CDSS appeared in Care Organizer. When the nurse checked Care Organizer at 10:59am for 11:00am medications to administer, the nurse acknowledged the 10:02am IIT CDSS order because it had already been given, removing it from the list. At 11:55am the nurse accessed Care Organizer to check the list of medications to administer at 12:00pm. An order for heparin injection appeared but not insulin. Knowing the two hour interval for intensive insulin therapy required a 12:00pm dose, the nurse used IIT CDSS at 12:05pm and generated insulin and D50W orders due to the patient’s blood glucose falling slightly below 80 mg/dL. The nurse immediately administered D50W and planned to perform IIT CDSS in thirty minutes per protocol for hypoglycemia risk. At 12:37pm the nurse performed IIT using CDSS and the patient’s BG had increased above 80 mg/dL. At 1:00pm the nurse accessed Care Organizer and acknowledged insulin orders from 12:05pm and 12:37pm as well as the 12:05 D50W dose because they had already been given. IIT CDSS use at two hour intervals resumed at 2:00pm.

Barrier 3: inaccurate user interface design assumptions

Unintended use of “off protocol” functionality

In one instance, the researcher observed unintended and potentially harmful use of IIT CDSS’s “Patient Has Been Off Protocol” functionality (Box 2). Review of IIT CDSS database records from November 2004 through February 2009 indicated that other nurses may have similarly used IIT CDSS to incorrectly administer overrides in less than one percent of cases.

Box 2. Example demonstrating improper use of patient off protocol functionality.

Uncomfortable with a current high insulin rate for a patient treated with IIT, a nurse clicked the “Patient Has Been Off Protocol” checkbox and entered the current BG result and an insulin rate of his choosing less than that currently being administered. Asked to explain this behavior, the nurse described his intent to override, which led the researcher to describe the override procedure he had seen in previous observations of replacing the recommended rate text. The nurse stated he was unaware that administering an override via the “Patient Has Been Off Protocol” functionality was incorrect and continued to administer the dose in this fashion.

Dextrose source selection

Although IIT CDSS required nurses to select a dextrose source in order to calculate recommendations, patients frequently received dextrose from multiple sources simultaneously. This led nurses to select the greatest of dextrose sources to satisfy IIT CDSS requirements and then fully document multiple dextrose sources in HED. Despite partial double documentation, IIT CDSS log data did not fully reflect patient nutrition.

Blood draw source selection

Most of the blood tests we observed used blood drawn from capillary sources (e.g. finger stick) rather than arterial sources that yield more accurate results [5]. “Arterial” is the default blood source selection in IIT CDSS, and we observed instances where nurses performed a test using capillary blood but did not select “capillary” in the system. Review of retrospectively collected IIT CDSS data indicated that nurses selected “capillary” for 39% of blood sources, and the percentage might underestimate the true occurrence of tests performed using capillary blood. Use of capillary blood may prevent IIT CDSS from using accurate measurements to calculate recommendations, and the IIT CDSS interface may facilitate incorrect documentation of blood sources.

Barrier 4: potential for error in operating medical devices

Insulin infusion pump adjustment

IIT CDSS recommends insulin infusion rates and allows nurses to override recommendations, but the protocol relies on a nurse to manually adjust a patient’s infusion pump. Although infusion pumps can transfer insulin adjustment data to other systems, this feature was not enabled during the study. In one instance we observed a nurse enter a rate of 2.6 units per hour on the pump after accepting IIT CDSS’s recommendation of 2.7 units per hour. We are unsure whether this occurred deliberately or by chance. In another instance a nurse failed to adjust an infusion pump after accepting IIT CDSS recommendations due to an emergent patient care situation at another bedside.

Handheld testing device use and physical layout with respect to isolation precautions

When treating patients under isolation precautions, nurses did not bring handheld testing devices into patient rooms in order to prevent device contamination according to hospital policy. Instead nurses leaned through doorways to conduct tests on devices just outside isolation rooms. In one instance, a nurse conducted the test outside the patient room before returning to a computer workstation at the isolation patient’s bedside to enter the value into IIT CDSS and administer care. She did not use a paper sheet to keep notes and instead relied on memory. The value entered by the nurse, 224, did not equal the value from the testing device, 226. In another instance, we observed a nurse conducting a BG test outside of an isolation room for a non-isolation patient situated in it although the device could have been used at the bedside. One nurse stated a preference for conducting BG testing and computer usage in the patient room in order to avoid making mistakes.

Facilitator 1: trust in IIT CDSS combined with clinical judgment

Nurses reported trusting IIT CDSS because the system’s underlying protocol was evidence-based and generated appropriate recommendations [17]. Nurses recognized the cumulative impact of BG and insulin data on IIT CDSS output, and one respondent said he needed to trust the previous shift nurse’s use of IIT CDSS to feel comfortable using the system. Typical IIT CDSS use involved nurses transcribing BG values, selecting additional required parameters on screen, clicking the calculate button, and accepting recommendations quickly and without argument [17]. Nurses accepted 94.9% of recommended insulin doses between November 2004 and February 2009 [17], and BG values following override doses were within acceptable ranges in most cases [17]. When asked to explain decisions to override, nurses frequently referred to patients’ BG trends and sources of nutrition as well as their own general intuition and interest in preventing hypoglycemia [17]. One nurse said, “I would never give more [insulin]; I would only give less. I wouldn’t feel comfortable giving more. I don’t want [patients] becoming hypoglycemic.” Nurses appeared to have appropriately exercised clinical judgment by compensating for glycemia-influencing factors beyond IIT CDSS algorithm parameters.

Despite trust in recommendations, nurse understanding of IIT CDSS’s dosing algorithm varied. Several nurses described the system as a “black box” that “learns the patient,” and nurse explanations of IIT CDSS algorithm inputs—current BG and dextrose source; the average of two BG values; and a combination of patient’s height, weight, BG, previous insulin, and dextrose source—were not accurate. One nurse stated that he clicked the “Explain Recommendations” once but found the description confusing and closed the window. Regardless of their reported IIT CDSS algorithm understanding, nurses’ “feel” for IIT CDSS recommendations appeared appropriate.

Facilitator 2: nurse resilience

Nurses adapted to myriad changes in order to perform IIT CDSS. Without any systematic reminders, nurses remembered to perform IIT CDSS at two hour intervals. In the event of hypoglycemic events, nurses successfully altered their work schedules to obtain D50W doses and recheck BG measurements in thirty minutes before reverting to the default two hour interval. Despite competing care tasks and workplace interruptions, nurses diligently adjusted insulin pumps after using IIT CDSS. Additionally, nurses regularly dismissed insulin and D50W doses that no longer applied from Care Organizer in order to clarify medication scheduling.

Facilitator 3: paper documentation as an intermediary

For most nurses paper served as the conduit between bedside readings and HED and IIT CDSS. Although this practice amounted to triple documentation, the majority of nurses would have relied solely on memory in a disruptive work environment and potentially forgotten or misremembered data points critical to patient care. Nurses used blank paper sheets, pre-specified forms of their or a colleague’s creation, or improvised scraps and bandage wrappers to record parameters from devices, physical assessments, and computer systems. Nurses frequently recorded BG measurements on paper sheets and referred to these notes when using IIT CDSS.

Discussion

This investigation is the first field study of IIT CDSS to the best of our knowledge, and reveals sociotechnical interactions affecting protocol performance not addressed by previous research. The relationship between IIT CDSS and subsequently-implemented software shows how the whole of clinical information systems, greater than the sum of its parts, affects computer-based intensive insulin therapy. Factors affecting IIT CDSS use in this study—nursing documentation and reminder systems, inaccurate user interface design assumptions, medical device operation, and nurse attitudes—may similarly influence IIT CDSS at other institutions and indicate opportunities for improvement.

A tradeoff in critical care nurse workload exists between direct patient care and computer system use, and attitudes toward IIT CDSS and HED underscore the importance of adding value to clinical documentation. Both HED and IIT CDSS required significant time commitments. However, nurses felt that nursing documentation system use occasionally hindered their ability to administer patient care, but valued IIT CDSS, which recommended drug doses based on nurse data input, as a trusted clinical process. Researchers have recognized the opportunity to provide decision support through electronic documentation systems, and provision of such “Smart Forms” [31] for ICU documentation may improve nurse perception, data quality, workflow, and patient outcomes. Figure 3 depicts an incremental workflow improvement leveraging streamlined documentation for dosing recommendations and drug safety. While not available in most hospital environments, ideally IIT CDSS would transmit updated insulin rates directly to infusion pumps for nurses to confirm, eliminating an opportunity for human error in setting pumps.

Figure 3.

Figure 3

Open in a new tab

Current and future IIT CDSS documentation workflow. Nurses recorded data for each protocol iteration in IIT CDSS and HED, an electronic nursing documentation system. In the future, nurses could record these data in IIT CDSS, which could then transfer the data to HED to eliminate double documentation and save nurses time. Automatic transfer of infusion pump data to HED could also save time and help nurses double check that administered insulin rates match orders to improve safety.

Capturing more parameters in IIT CDSS could also facilitate use of more sophisticated dosing algorithms that incorporate dextrose sources [32]. Due to the varying degrees of algorithm understanding expressed by nurses in this study, changes to the underlying dosing algorithm may go unnoticed, which can be of potential value in a blinded study. However, changing the dosing algorithm without notifying nurses could also have deleterious effects on nurses’ mental models of IIT CDSS and thus their clinical judgment and patients’ safety. Simulation studies may assist with evaluating the appropriateness of different dosing algorithms’ insulin recommendations prior to implementation.

Timing of IIT CDSS affects protocol performance [13-14], and some implementations actively remind nurses to perform IIT CDSS using popup alerts [19, 33]. Alert fatigue is a concern in clinical informatics [34-35], and the passive alerts observed in this study, provided through the nurse medication scheduling tool and dashboard, appeared effective to varying degrees for non-IIT nursing tasks. Opportunities exist to improve display of IIT CDSS dosing in both systems, but staff attitudes toward the dashboard may pose a barrier to use. Institutions should consider user perceptions of clinical systems when implementing new or modified functionalities supporting IIT CDSS and other care protocols. In the sites we studied, dashboard-based interventions were associated with reductions in nosocomial infection rates [24, 36]. Improved feedback to nurses regarding the impact of their dashboard-based efforts may ameliorate perceptions of the system. Regardless, the absence of reminders for IIT and presence of reminders for all non-IIT tasks may adversely affect nurses’ perception of IIT compliance necessity, which in turn can worsen protocol performance and outcomes.

Previously researchers identified barriers and facilitators to use of computerized reminders in outpatient settings [34, 37-38], and we have extended this line of inquiry to intensive care units for IIT CDSS. Scholars have noted the paucity of theory in biomedical informatics research [39-40], but the common themes identified by previous studies [34, 37-38] and this investigation—limiting the number of reminders, reducing the perception that management benefits more from system use than end users, eliminating double documentation, and integrating CDSS into workflow—indicate the beginning of an empirically derived theory for computerized clinical decision support systems to promote high performance of clinical protocols.

This study has limitations. In 49 hours of observation we did not observe protocol initiation; patient transport to the operating room or imaging suites; or IIT CDSS use in hospital units beyond two intensive care units. Additionally, we conducted the study at a single academic medical center with high informatics commitment that may not be widespread. As a result, findings may not generalize at or beyond the study institution. However, other institutions may find the study’s methods and findings of sociotechnical interactions affecting IIT CDSS, which we believe to be the first reported in the literature, useful as a starting point for IIT CDSS and general CDSS improvements. Due to resource constraints, a single researcher conducted observations, interviews, and data analysis. However, ethnographic studies conducted by single researchers have previously contributed to informatics theory and practice [41-44]. The researcher’s presence during observations may have affected clinicians’ actions [45], but prolonged engagement [30] in the study sites may have mitigated this behavior.

By evaluating a clinical decision support system for intensive insulin therapy with respect to other clinical information systems and care processes, we identified nurse attitudes, data processing gaps, design assumptions, and nurse resilience that affected system use. Findings suggest opportunities for improvement involving documentation-oriented user interface changes and reminders. Software developers and evaluators of IIT CDSS and other clinical information systems can use the results of this study to plan new projects and improve existing implementations. Research addressing the context of clinical decision support system use can contribute to a theory of effective system use and assist practitioners with system design and implementation.

Research Highlights.

  • This study identified novel sociotechnical interactions affecting IIT CDSS performance.

  • Findings have implications for IIT CDSS alerts and user interface design.

  • Themes from this and other studies contribute toward theory for high performance CDSS.

Summary table.

What was already known on this topic

  • Factors affecting intensive insulin therapy protocol performance include patient populations, blood glucose target ranges, nutrition sources, genetic characteristics, nurse staffing, blood glucose measurement timing, data entry, and nurse overrides.

  • Few studies of computer-based intensive insulin therapy, and clinical decision support systems in general, have evaluated interventions with respect to other clinical information systems and care processes.

What this study added to our knowledge

  • This investigation is the first field study of IIT CDSS and reveals sociotechnical interactions affecting protocol performance not addressed by previous research.

  • Common themes identified in this and other studies of computerized reminders may indicate the beginning of an empirically derived theory for clinical decision support systems to promote high performance of clinical protocols.

Acknowledgements

Dr. Campion received support from National Library of Medicine Training Grant NLM T15 007450-07. The authors thank Lillian McGehee, MSN, RN and an anonymous study participant nurse for manuscript feedback. Dr. Campion thanks Asli Ozdas, Ph.D., M.S. for project involvement; Kim M. Unertl, Ph.D. for analytical assistance; and SICU and TICU nurses, patients, and families for their cooperation in making this study possible.

Footnotes

Authors’ contributions Dr. Campion developed the research question, collected data, analyzed data, and generated the manuscript. Drs. Waitman, Lorenzi, May, and Gadd provided guidance throughout the research process as well as critical manuscript revisions. Dr. May also provided access to the study sites.

Statement on conflicts of interest The authors have no conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Finfer S, Chittock DR, Su SY, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med. 2009 Mar 26;360(13):1283–97. doi: 10.1056/NEJMoa0810625. [DOI] [PubMed] [Google Scholar]
  • 2.Wiener RS, Wiener DC, Larson RJ. Benefits and risks of tight glucose control in critically ill adults: a meta-analysis. Jama. 2008 Aug 27;300(8):933–44. doi: 10.1001/jama.300.8.933. [DOI] [PubMed] [Google Scholar]
  • 3.Griesdale DE, de Souza RJ, van Dam RM, et al. Intensive insulin therapy and mortality among critically ill patients: a meta-analysis including NICE-SUGAR study data. CMAJ. 2009 Apr 14;180(8):821–7. doi: 10.1503/cmaj.090206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Preiser JC, Devos P, Ruiz-Santana S, et al. A prospective randomised multi-centre controlled trial on tight glucose control by intensive insulin therapy in adult intensive care units: the Glucontrol study. Intensive Care Med. 2009 Oct;35(10):1738–48. doi: 10.1007/s00134-009-1585-2. [DOI] [PubMed] [Google Scholar]
  • 5.Fahy BG, Sheehy AM, Coursin DB. Glucose control in the intensive care unit. Crit Care Med. 2009 May;37(5):1769–76. doi: 10.1097/CCM.0b013e3181a19ceb. [DOI] [PubMed] [Google Scholar]
  • 6.van den Berghe G, Wouters P, Weekers F, et al. Intensive insulin therapy in the critically ill patients. N Engl J Med. 2001 Nov 8;345(19):1359–67. doi: 10.1056/NEJMoa011300. [DOI] [PubMed] [Google Scholar]
  • 7.Eslami S, Abu-Hanna A, de Jonge E, de Keizer NF. Tight glycemic control and computerized decision-support systems: a systematic review. Intensive Care Med. 2009 Sep;35(9):1505–17. doi: 10.1007/s00134-009-1542-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Campion TR, Jr., Waitman LR, May AK, Ozdas A, Lorenzi NM, Gadd CS. Social, organizational, and contextual characteristics of clinical decision support systems for intensive insulin therapy: a literature review and case study. Int J Med Inform. 2010 Jan;79(1):31–43. doi: 10.1016/j.ijmedinf.2009.09.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Aragon D. Evaluation of nursing work effort and perceptions about blood glucose testing in tight glycemic control. Am J Crit Care. 2006 Jul;15(4):370–7. [PubMed] [Google Scholar]
  • 10.Malesker MA, Foral PA, McPhillips AC, Christensen KJ, Chang JA, Hilleman DE. An efficiency evaluation of protocols for tight glycemic control in intensive care units. Am J Crit Care. 2007 Nov;16(6):589–98. [PubMed] [Google Scholar]
  • 11.Sward K, Orme J, Jr., Sorenson D, Baumann L, Morris AH. Reasons for declining computerized insulin protocol recommendations: application of a framework. J Biomed Inform. 2008 Jun;41(3):488–97. doi: 10.1016/j.jbi.2008.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Vogelzang M, Zijlstra F, Nijsten MW. Design and implementation of GRIP: a computerized glucose control system at a surgical intensive care unit. BMC Med Inform Decis Mak. 2005;5:38. doi: 10.1186/1472-6947-5-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Dossett LA, Collier B, Donahue R, et al. Intensive insulin therapy in practice: can we do it? JPEN J Parenter Enteral Nutr. 2008 Jan-Feb;33(1):14–20. doi: 10.1177/0148607108321703. [DOI] [PubMed] [Google Scholar]
  • 14.Vogelzang M, Loef BG, Regtien JG, et al. Computer-assisted glucose control in critically ill patients. Intensive Care Med. 2008 Aug;34(8):1421–7. doi: 10.1007/s00134-008-1091-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Dossett LA, Cao H, Mowery NT, Dortch MJ, Morris JM, Jr, May AK. Blood glucose variability is associated with mortality in the surgical ICU. Am Surg. 2008 Aug;74(8):679–85. doi: 10.1177/000313480807400802. discussion 85. [DOI] [PubMed] [Google Scholar]
  • 16.Campion TR, Jr., May AK, Waitman LR, Ozdas A, Gadd CS. Effects of blood glucose transcription mismatches on a computer-based intensive insulin therapy protocol. Intensive Care Med. 2010 Sep;36(9):1566–70. doi: 10.1007/s00134-010-1868-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Campion TR, Jr., May AK, Waitman LR, Ozdas A, Lorenzi NM, Gadd CS. Characteristics and effects of nurse dosing over-rides on computer-based intensive insulin therapy protocol performance. J Am Med Inform Assoc. 2011 May 1;18(3):251–8. doi: 10.1136/amiajnl-2011-000129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Morris AH, Orme J, Jr., Truwit JD, et al. A replicable method for blood glucose control in critically Ill patients. Crit Care Med. 2008 Jun;36(6):1787–95. doi: 10.1097/CCM.0b013e3181743a5a. [DOI] [PubMed] [Google Scholar]
  • 19.Rood E, Bosman RJ, van der Spoel JI, Taylor P, Zandstra DF. Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation. J Am Med Inform Assoc. 2005 Mar-Apr;12(2):172–80. doi: 10.1197/jamia.M1598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Anger KE, Szumita PM. Barriers to glucose control in the intensive care unit. Pharmacotherapy. 2006 Feb;26(2):214–28. doi: 10.1592/phco.26.2.214. [DOI] [PubMed] [Google Scholar]
  • 21.Orlikowski WJ, Barley SR. Technology and Institutions: What Can Research on Information Technology and Research on Organizations Learn from Each Other? Management Information Systems Research Center, University of Minnesota; 2001. pp. 145–65. [Google Scholar]
  • 22.Kaplan B. Evaluating informatics applications--clinical decision support systems literature review. International journal of medical informatics. 2001 Nov;64(1):15–37. doi: 10.1016/s1386-5056(01)00183-6. [DOI] [PubMed] [Google Scholar]
  • 23.Friedman CP, Wyatt JC. Evaluation methods in biomedical informatics. Springer-Verlag; New York: 2005. [Google Scholar]
  • 24.Zaydfudim V, Dossett LA, Starmer JM, et al. Implementation of a real-time compliance dashboard to help reduce SICU ventilator-associated pneumonia with the ventilator bundle. Arch Surg. 2009 Jul;144(7):656–62. doi: 10.1001/archsurg.2009.117. [DOI] [PubMed] [Google Scholar]
  • 25.Boord JB, Sharifi M, Greevy RA, et al. Computer-based insulin infusion protocol improves glycemia control over manual protocol. J Am Med Inform Assoc. 2007 May-Jun;14(3):278–87. doi: 10.1197/jamia.M2292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Dortch MJ, Mowery NT, Ozdas A, et al. A computerized insulin infusion titration protocol improves glucose control with less hypoglycemia compared to a manual titration protocol in a trauma intensive care unit. JPEN J Parenter Enteral Nutr. 2008 Jan-Feb;32(1):18–27. doi: 10.1177/014860710803200118. [DOI] [PubMed] [Google Scholar]
  • 27.Bode BW, Braithwaite SS, Steed RD, Davidson PC. Intravenous insulin infusion therapy: indications, methods, and transition to subcutaneous insulin therapy. Endocr Pract. 2004 Mar-Apr;10(Suppl 2):71–80. doi: 10.4158/EP.10.S2.71. [DOI] [PubMed] [Google Scholar]
  • 28.White NH, Skor D, Santiago JV. Practical closed-loop insulin delivery. A system for the maintenance of overnight euglycemia and the calculation of basal insulin requirements in insulin-dependent diabetics. Ann Intern Med. 1982 Aug;97(2):210–3. doi: 10.7326/0003-4819-97-2-210. [DOI] [PubMed] [Google Scholar]
  • 29.Corbin JM, Strauss AL. Basics of qualitative research: techniques and procedures for developing grounded theory. SAGE Publications; Los Angeles: 2008. [Google Scholar]
  • 30.Lincoln YS, Guba EG. Naturalistic inquiry. Sage Publications; Beverly Hills, Calif.: 1985. [Google Scholar]
  • 31.Schnipper JL, Linder JA, Palchuk MB, et al. “Smart Forms” in an Electronic Medical Record: documentation-based clinical decision support to improve disease management. J Am Med Inform Assoc. 2008 Jul-Aug;15(4):513–23. doi: 10.1197/jamia.M2501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Pachler C, Plank J, Weinhandl H, et al. Tight glycaemic control by an automated algorithm with time-variant sampling in medical ICU patients. Intensive Care Med. 2008 Jul;34(7):1224–30. doi: 10.1007/s00134-008-1033-8. [DOI] [PubMed] [Google Scholar]
  • 33.Juneja R, Roudebush C, Kumar N, et al. Utilization of a computerized intravenous insulin infusion program to control blood glucose in the intensive care unit. Diabetes Technol Ther. 2007 Jun;9(3):232–40. doi: 10.1089/dia.2006.0015. [DOI] [PubMed] [Google Scholar]
  • 34.Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005 Jul-Aug;12(4):438–47. doi: 10.1197/jamia.M1777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007 Jan-Feb;14(1):29–40. doi: 10.1197/jamia.M2170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Miller RS, Norris PR, Jenkins JM, et al. Systems initiatives reduce healthcare-associated infections: a study of 22,928 device days in a single trauma unit. J Trauma. 2010 Jan;68(1):23–31. doi: 10.1097/TA.0b013e3181c82678. [DOI] [PubMed] [Google Scholar]
  • 37.Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc. 2004 Jan-Feb;11(1):50–9. doi: 10.1197/jamia.M1364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Militello L, Patterson ES, Tripp-Reimer T, et al. Clinical Reminders: Why don’t they use them. Human Factors and Ergonomics Society Annual Meeting Proceedings. 2004;48:1651–5. [Google Scholar]
  • 39.Kaplan B. Evaluating informatics applications--some alternative approaches: theory, social interactionism, and call for methodological pluralism. International journal of medical informatics. 2001 Nov;64(1):39–56. doi: 10.1016/s1386-5056(01)00184-8. [DOI] [PubMed] [Google Scholar]
  • 40.Chiasson M, Reddy M, Kaplan B, Davidson E. Expanding multi-disciplinary approaches to healthcare information technologies: what does information systems offer medical informatics? Int J Med Inform. 2007 Jun;76(Suppl 1):S89–97. doi: 10.1016/j.ijmedinf.2006.05.010. [DOI] [PubMed] [Google Scholar]
  • 41.Patterson ES, Cook RI, Render ML. Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc. 2002 Sep-Oct;9(5):540–53. doi: 10.1197/jamia.M1061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Reddy MC, Pratt W, Dourish P, Shabot M. Asking questions: information needs in a surgical intensive care unit. Proc AMIA Symp. 2002:647–51. [PMC free article] [PubMed] [Google Scholar]
  • 43.Richardson JE, Shah-Hosseini S, Fiadjoe JE, Ash JS, Rehman MA. The effects of a hands-free communication device system in a surgical suite. J Am Med Inform Assoc. 2011 Jan 1;18(1):70–2. doi: 10.1136/jamia.2009.001461. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Unertl KM, Weinger MB, Johnson KB, Lorenzi NM. Describing and modeling workflow and information flow in chronic disease care. J Am Med Inform Assoc. 2009 Nov-Dec;16(6):826–36. doi: 10.1197/jamia.M3000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Wickstrom G, Bendix T. The “Hawthorne effect”--what did the original Hawthorne studies actually show? Scand J Work Environ Health. 2000 Aug;26(4):363–7. [PubMed] [Google Scholar]

RESOURCES