Characteristics and effects of nurse dosing over-rides on computer-based intensive insulin therapy protocol performance

Abstract

Objective

To determine characteristics and effects of nurse dosing over-rides of a clinical decision support system (CDSS) for intensive insulin therapy (IIT) in critical care units.

Design

Retrospective analysis of patient database records and ethnographic study of nurses using IIT CDSS.

Measurements

The authors determined the frequency, direction—greater than recommended (GTR) and less than recommended (LTR)— and magnitude of over-rides, and then compared recommended and over-ride doses' blood glucose (BG) variability and insulin resistance, two measures of IIT CDSS associated with mortality. The authors hypothesized that rates of hypoglycemia and hyperglycemia would be greater for recommended than over-ride doses. Finally, the authors observed and interviewed nurse users.

Results

5.1% (9075) of 179 452 IIT CDSS doses were over-rides. 83.4% of over-ride doses were LTR, and 45.5% of these were ≥50% lower than recommended. In contrast, 78.9% of GTR doses were ≤25% higher than recommended. When recommended doses were administered, the rate of hypoglycemia was higher than the rate for GTR (p=0.257) and LTR (p=0.033) doses. When recommended doses were administered, the rate of hyperglycemia was lower than the rate for GTR (p=0.003) and LTR (p<0.001) doses. Estimates of patients' insulin requirements were higher for LTR doses than recommended and GTR doses. Nurses reported trusting IIT CDSS overall but appeared concerned about recommendations when administering LTR doses.

Conclusion

When over-riding IIT CDSS recommendations, nurses overwhelmingly administered LTR doses, which emphasized prevention of hypoglycemia but interfered with hyperglycemia control, especially when BG was >150 mg/dl. Nurses appeared to consider the amount of a recommended insulin dose, not a patient's trend of insulin resistance, when administering LTR doses overall. Over-rides affected IIT CDSS protocol performance.

Introduction

Intensive insulin therapy (IIT) is the standard of critical care, but recent studies questioned the treatment's mortality benefit and safety.^1–4 Intensive care units commonly employ IIT, a nurse-driven treatment combining frequent blood glucose (BG) measurements and insulin titrations, to achieve tight control of patients' BG levels (eg, 80–110 mg/dl). Factors affecting IIT performance include patient populations, BG monitoring techniques, BG target ranges, nutrition provisions, nurse staffing ratios, and molecular and cellular features.⁵ Institutions increasingly use computerized clinical decision support systems (CDSS) to deliver protocol-based IIT,^{6 7} and computer-based workflow has been identified as another source of variability affecting IIT.^{6 8–10} Researchers have shown a relationship between timing of BG measurements and hypo- and hyperglycemia.^{9 10} Previously, we demonstrated the effect of data-entry error on IIT CDSS recommendations and BG variability.⁸ In the present study, we examined nurse insulin-dosing over-rides, or deviations from IIT CDSS protocol recommendations, to determine if and how the behavior affected IIT performance.

Existing studies have measured the frequency and rationale of nurse over-rides, but little is known about the quantitative characteristics of IIT CDSS over-rides and their effect on BG variability and insulin resistance, two measures of IIT CDSS protocol performance associated with mortality.^{11 12} Compliance with IIT CDSS recommendations at other institutions varies from 77%¹³ to up to 98%.^{10 14 15} Nurses' reasons for over-rides include concerns about hypoglycemia due to data trends, administration of glucose-affecting medications, and comorbidities¹⁵; disagreement with dose recommendations^{14 15}; and workflow issues.^{14 15} The objective of this study was to determine the conditions leading to and resulting from nurse over-ride of IIT CDSS recommendations. We compared BG variability and insulin resistance when nurses administered recommended and over-ride doses. We hypothesized that rates of hypoglycemia and hyperglycemia would be greater for recommended than over-ride doses. Additionally, we used ethnographic methods to understand nurse perceptions of IIT CDSS and the manner in which nurses over-rode system recommendations.

Methods

The Vanderbilt University Institutional Review Board approved this study.

Setting

This study examined IIT CDSS usage in the 21-bed surgical and 31-bed trauma intensive care units (SICU and TICU) at Vanderbilt University Hospital, a 501-bed academic urban tertiary care facility in Nashville, Tennessee. Critical-care attending physicians from the Division of Trauma and Surgical Critical Care oversaw unit management and patient-care decisions using evidence-based protocols intended to standardize care and reduce practice variability.^16–18 Although clinicians in other intensive care units at the institution treated patients using IIT CDSS, we focused our investigation on SICU and TICU because of the units' common management and care processes. On average, the ratio of patients to nurses was 2:1 overall and 1:1 for complex patients. All patients admitted to the SICU and TICU received regular human insulin and standardized nutrition.

SICU piloted IIT CDSS in November 2004 and required nurses to treat all eligible patients using IIT CDSS starting in December 2005.¹⁹ In June 2005, IIT CDSS was modified to require nurses to identify a patient's primary dextrose source and automatically trigger an order for intravenous 10% dextrose if a patient had no primary dextrose source and a current BG less than 80 mg/dl. The intent of the change was to minimize hypoglycemia. TICU implemented IIT CDSS as the standard of care in October 2005.²⁰ The IIT CDSS recommendation algorithm has remained unchanged since November 2004, and SICU and TICU researchers have demonstrated effective hyperglycemia control with limited hypoglycemia.^{19 20} We investigated all SICU and TICU IIT CDSS recommended and over-ride doses over the period of this study.

IIT CDSS description

Critically ill or injured²⁰ mechanically ventilated patients with a BG value above 110 mg/dl received IIT according to a protocol embedded in the institution's computerized order entry system. Clinicians accessed the order entry system using desktop computers located near the bedside. Figure 1 depicts the IIT CDSS workflow.

Figure 1.

Intensive insulin therapy (IIT) clinical decision support system (CDSS) workflow. The dashed line represents an optional process contingent on system notification. CDR, clinical data repository; MD, medical doctor; NA, nursing assistant; RN, registered nurse. After administering a dose, a nurse documented a patient's blood glucose value, insulin dose, and primary dextrose source in a separate electronic nurse charting system that did not interface with IIT CDSS (not pictured). A nurse repeated IIT CDSS workflow at 2 h intervals (Q2H) or 1 h intervals (Q1H) based on system recommendations. Reproduced with permission of Intensive Care Medicine from Campion et al.⁸

As described elsewhere,^{6 8 19 20} a physician confirmed the initial protocol order, which directed a nurse to measure a patient's BG and administer insulin according to CDSS recommendations at 1 or 2 h intervals to maintain BG between 80 and 110 mg/dl. IIT CDSS instructed a nurse to administer an intravenous 50% dextrose dose in 5 ml increments¹⁹ and recheck the patient's BG in 1 h if BG was less than 80 mg/dl. For BG less than 60 mg/dl, the protocol additionally instructed a nurse to stop insulin administration for 1 h. All other IIT CDSS protocol iterations occurred at 2 h intervals. IIT CDSS did not remind nurses to measure BG. Nurse adherence to the protocol's timing of BG measurements was 86%; 8% of measurements were taken late.⁹

Based on Bode²¹ and White's²² dosing equation, IIT CDSS adjusted a coefficient ‘multiplier,’ an estimate of a patient's insulin resistance according to current and previous BG measurements, for use in this formula: insulin dose in units/h=(current BG in mg/dl–60)×multiplier.¹⁹ The multiplier was initially set to 0.03, but increased by 0.01 when BG levels indicated hyperglycemia, decreased by 0.01 if BG was less than 80 mg/dl, decreased by 0.02 if BG was less than 60 mg/dl, and could not fall below zero.¹⁹ A greater multiplier value reflected increased insulin resistance. After the initial order set the multiplier, IIT CDSS obtained the previous multiplier not by retrieving it from the system database but by solving the dosing formula using the previous BG and insulin rate stored by the order entry system as inputs (ie, multiplier=insulin dose/(previous BG–60)).

IIT CDSS calculated recommendations using the formula and logic above after a nurse manually transcribed, or entered via keyboard, a BG value obtained from a handheld testing device, selected a patient's primary dextrose source from a list of radiobuttons, and clicked the ‘calculate recommendations’ button.^{6 8 19 20} If a nurse agreed with the recommended dose displayed on screen, he or she clicked the ‘submit’ button to accept the recommendation. If a nurse disagreed with the recommended dose, he or she replaced the recommended dose value via keyboard and then clicked the ‘submit’ button to over-ride the recommendation. After a nurse clicked the ‘submit’ button, the order entry system updated the existing order and logged the insulin rate, multiplier, over-ride status, BG value, primary dextrose source, and timestamp along with patient and nurse identifiers. The nurse then adjusted the intravenous insulin pump to use the IIT CDSS insulin rate. There was no electronic interface between intravenous insulin pumps and clinical information systems. Handheld testing devices stored each BG value with a timestamp as well as a nurse identifier and patient medical record number; nurses recorded these identifiers by scanning barcodes or entering them manually. At the beginning of each 12 h nursing shift, a nursing assistant collected all handheld BG testing devices and placed them in docking stations for transfer to the clinical data repository.⁶

Retrospective data collection and analysis

We retrospectively collected order entry and handheld BG testing device data from the institution's clinical data repository for all SICU and TICU patients with more than five IIT CDSS values between 1 November 2004 and 1 February 2009. We stored study data in a secure, password-protected database and deidentified data prior to analysis and reporting.

Because IIT CDSS logged both recommended and over-ride doses that nurses administered but not calculated doses nurses elected to over-ride (eg, IIT CDSS calculated an insulin dose of 4.2 U/h that the nurse over-rode with 2.2 U/h; the system logged 2.2 U/h but not the calculated 4.2 U/h), we recreated the conditions for each insulin administration to determine calculated doses. For each patient, we processed BG values and insulin rates to determine multiplier and recommendations per the IIT CDSS dosing algorithm. If the care team discontinued and later reinitiated the protocol for a patient, we treated these as separate runs of IIT CDSS to assure correct calculation of recommendations and comparison of BG values. We identified instances that did not recreate multipliers and/or recommended insulin doses in the log data. To control for the effect of keystroke error of BG values contributing to over-ride decisions, we linked each IIT CDSS BG value with a corresponding device BG value and identified pairs of mismatched values as well as IIT CDSS values lacking a device value.⁸ To assure quality, we examined only recommended and over-ride instances with successfully recreated output and matching BG values⁸ so that we could reliably determine calculated doses in the event of over-ride.

We determined the frequency, BG variability, and insulin resistance associated with recommended and over-ride doses. We divided over-rides into greater than recommended (GTR) and less than recommended (LTR) doses. Additionally, for each over-ride dose, we computed the degree of deviation of an actual dose from a calculated dose by determining the absolute value of the difference of the actual dose and the calculated dose divided by the actual dose.²³ We identified three types of deviations: ‘small’ as ≤25%, ‘medium’ as 26% to 49%, and ‘large’ as ≥50%.²³

Previous studies have associated BG variability and insulin resistance with mortality in SICU and TICU patients treated with IIT CDSS.^{11 12} Several measures of BG variability have been correlated with mortality in critically ill patients.^{24 25} To assess BG variability, we examined BG values before (BG_n−1), during (BG_n), and after (BG_n+1) each insulin administration (n) for both recommended and over-ride doses. Additional measures of BG variability included successive BG change, which reflects both regular and abrupt fluctuations in the distribution of BG values,¹² as well as hypoglycemia and hyperglycemia, which measure IIT safety and effectiveness, respectively. We defined hypoglycemia^{i 26} as BG_n ≥ 60 mg/dl at the time of dose followed by subsequent BG_n+1 <60 mg/dl and hyperglycemiaⁱⁱ as BG_n<200 mg/dl at the time of dose followed by subsequent BG_n+1≥200 mg/dl. Thresholds of 60 and 200 mg/dl for hypoglycemia and hyperglycemia, respectively, are common in the IIT literature, although definitions vary across studies.²⁷ We assumed nurses performed IIT CDSS iterations independently; thus, we considered BG input to be correct and examined each dose instance BG value along with immediately preceding and succeeding dose BG values. Although the timing of IIT CDSS iterations varied,⁹ we assumed nurses made a good-faith effort to adhere to the protocol, and we evaluated records chronologically for each patient. To assess insulin resistance,¹¹ a proxy for the body's stress response and BG metabolism related to illness severity and mortality risk,^{11 12} we compared insulin dose and multiplier between recommended, GTR, and LTR doses. For over-ride instances, we compared actual versus calculated insulin doses. Greater insulin dose and multiplier levels indicated greater insulin resistance.¹¹

We summarized and compared normally distributed continuous variables using mean±SD and two-sample t tests for independent samples. For non-normally distributed continuous variables, we summarized and compared data using median and IQR and used the Wilcoxon rank-sum test for unpaired data and Wilcoxon signed rank test for paired data. To compare differences in proportions, we used a χ² test. Data represent grand summaries of IIT data and do not address repeated measures within patients. A two-sided p value of <0.05 indicated statistical significance. We used STATA version 10.1 to perform calculations.

Ethnographic data collection and analysis

To understand over-ride behavior in the context of actual system use, a researcher trained in ethnographic methods (TRC) conducted 49 h of direct observation and unstructured interviews with 25 nurses using IIT CDSS in SICU and TICU between 16 February 2010 and 18 March 2010 as part of a larger investigation of IIT CDSS with respect to other clinical information systems and care processes. Having completed a pilot study⁶ and quantitative analysis of data entry⁸ and over-ride behavior, the researcher was familiar with use of the system studied. In sessions lasting 2 to 3 h, the researcher followed one or more nurses through clinical shift work. Observations focused on the interaction of people, process, and technology in IIT CDSS delivery while unstructured interviews clarified observations. While conducting observations and interviews, the researcher used pen and paper to record narrative text and direct quotes describing nurse use of and attitudes toward IIT CDSS. Notes were then transcribed electronically for subsequent analysis. Verbal assent was obtained from clinicians, patients, and families (if present) before data collection. Data were analyzed inductively using a grounded theory²⁸ approach to allow themes to emerge. Data collection concluded after reaching a point of saturation when observations no longer yielded new concepts.²⁸ As in other qualitative informatics studies²⁹ using ethnographic methods,^{30 31} we ensured analytical rigor by using standard techniques of naturalistic inquiry including peer debriefing, member checking, and prolonged engagement.³²

Results

Retrospective data analysis

Table 1 presents patient demographics. 203 188 IIT CDSS instances and 423 463 BG testing device values were available. Figure 2 shows the IIT CDSS instances included and excluded from dose analysis. Manual process error by nurses and nursing assistants explains the presence of missing and mismatched data excluded from this analysis (appendix 1, available as an online data supplement at www.jamia.org). During the process of data cleaning and determining descriptive statistics, we discovered through a manual review that IIT CDSS misidentified 27 instances as over-rides that had actual doses equal to calculated doses, which may have resulted from users modifying existing insulin orders. We analyzed dosing for the remaining 179 452 IIT CDSS instances.

Table 1.

Study patient demographics

	Surgical intensive care unit	Trauma intensive care unit
Study period	1 November 2004–1 February 2009	5 October 2005–1 February 2009
Patients admitted, no	5364	8178
Patients treated with intensive insulin therapy clinical decision support system, no (%)	1883 (35.1%)	2152 (26.3%)
Age, years	58.9±14.6	41.5±18.8
Male sex	1130 (60%)	1576 (73%)
Body mass index, kg/m2	29.6±11.8	26.9±6.9
Admission service
Trauma	71 (3.8%)	2044 (95%)
Liver transplant	342 (18.2%)
Emergency general surgery	338 (18.0%)
Vascular surgery	194 (10.3%)
General surgery	154 (8.2%)	62 (2.9%)
Cardiac/thoracic surgery	119 (6.3%)	1 (<0.1%)
Oncology/endocrine surgery	97 (5.2%)	1 (<0.1%)
Urology	83 (4.4%)
Orthopedics	76 (4.0%)	20 (0.9%)
Other	409 (21.7%)	24 (1.1%)
Acute Physiology and Chronic Health Evaluation II (surgical intensive care unit)	18.9±6.5
Injury Severity Score (trauma intensive care unit)		27.7±11.9
History of diabetes	206 (10.9%)	71 (3.3%)
Hospital length of stay, days	17.8±16.1	14.1±13.9
Intensive-care-unit length of stay	9.1±10.6	9.6±10.6
Hospital mortality	284 (15.1%)	323 (15.0%)

Data represent mean±SD unless noted.

Figure 2.

Intensive insulin therapy (IIT) clinical decision support system (CDSS) instances excluded and included for analysis: data sources (black), instances excluded from dose analysis (gray), and instances included for dose analysis (white). To reliably determine doses that nurses elected not to administer, we excluded 23 736 (11.7%) of 203 188 IIT CDSS doses.

Over-rides accounted for 5.1% of IIT CDSS activity, of which 83.4% of over-ride doses were less than recommended and the remainder greater than recommended (table 2). The majority of greater than recommended doses differed from calculated doses by a small deviation whereas the plurality of less than recommended doses differed from calculated doses by a large deviation.

Table 2.

Frequency and direction of nurse over-rides

IIT CDSS doses, n (%) (N =179 452)	Degree of over-ride deviation, n (%)
	Small (≤25%)	Medium (26–49%)	Large (≥50%)
Recommended doses: 170 377 (94.9%)
Over-ride doses: 9075 (5.1%)	3966 (43.7%)	1549 (17.1%)	3560 (39.2%)
Greater than recommended: 1505 (16.6%)	1188 (78.9%)	202 (13.4%)	115 (7.6%)
Less than recommended: 7570 (83.4%)	2778 (36.7%)	1347 (17.8%)	3445 (45.5%)

Nurses chose to over-ride 5.1% of intensive insulin therapy (IIT) clinical decision support system (CDSS) recommended insulin doses, and more than four out of five over-ride doses were amounts less than those recommended by IIT CDSS.

When examining insulin administration by BG band (table 3), nurses administered the highest proportion of recommended doses when BG was <60 mg/dl (recommended dose was zero) and the lowest proportion when BG was 60–80 mg/dl. The number of less than recommended doses exceeded the number of greater than recommended doses in every BG band, including when BG exceeded 110 mg/dl, except when BG was less than 60 mg/dl and the protocol instructed nurses not to administer insulin.

Table 3.

Blood glucose (BG) values by band at the time of insulin administration

	BG <60	BG 60–80	BG 80–110	BG 110–150	BG 150–200	BG >200
Recommended, n (%)	2113 (99.8%)	9262 (87.0%)	73 045 (96.0%)	64 543 (95.5%)	16 521 (93.2%)	4893 (92.4%)
Greater than recommended	4 (0.2%)	66 (0.6%)	411 (0.5%)	489 (0.7%)	336 (1.9%)	199 (3.8%)
Less than recommended	0 (0%)	1321 (12.4%)	2639 (3.5%)	2531 (3.7%)	873 (4.9%)	206 (3.9%)

Percentages are based on column total, and BG values ranges are presented in mg/dl. The protocol specifies that nurses should not administer insulin when BG <60 mg/dl.

Nurses administered the highest proportion of less than recommended doses when BG was 60–80 mg/dl and greater than recommended doses when BG was >200 mg/dl. However, the number of greater than recommended doses did not exceed the number of less than recommended doses when BG was >200 mg/dl.

BG values differed before, during, and after each insulin administration for recommended greater than recommended and less than recommended doses (table 4).

Table 4.

Blood glucose (BG) variability for recommended (R), greater than recommended (GTR), and less than recommended (LTR) insulin doses

	R	GTR	LTR	R versus GTR	R versus LTR
BG (mg/dl), mean±SD before (BGn−1)	118±37	150±55	103±32	<0.001	<0.001
BG mean±SD during (BGn)	117±36	143±54	115±38	<0.001	<0.001
BG mean ± SD after (BGn+1)	115±33	132±46	122±35	<0.001	<0.001
BG change before (BGn−1–BGn)	−2.43±32.92	−9.91±36.05	11.09±42.24	<0.001	<0.001
BG change after (BGn–BGn+1)	−2.17±33.19	−10.96±37.47	7.19±38.53	<0.001	<0.001
Hypoglycemia, n (%)	1872 (1.1%)	12 (0.8%)	64 (0.8%)	0.257	0.033
Hyperglycemia	1411 (0.8%)	23 (1.5%)	118 (1.6%)	0.003	<0.001

As shown in figure 3, recommended doses showed a gradual trend of BG values toward the protocol target range as evidenced by downward trends for mean BG and high BG (+1 SD) and a slight upward trend in low BG (−1 SD). The BG value following a recommended dose was lower than those following greater than recommended and less than recommended doses. Greater than recommended doses showed a pronounced downward BG trend with hyperglycemic BG levels preceding and at the time of dose followed by a continued downward trend.

Figure 3.

Blood glucose (BG) levels (mean±SD) before, during and after insulin administration for recommended, greater than recommended, and less than recommended doses. These differed across all comparisons (p<0.001). The shaded area indicates the BG target range of 80–110 mg/dl. The figure depicts mean BG values, high BG values (+1 SD), and low BG values (−1 SD).

Greater than recommended insulin administration appears to have reduced elevated BG levels on average and for high BG and maintained levels in the target range for low BG (figure 3). BG values following administration of a greater than recommended dose were mostly higher than those following recommended and less than recommended doses. Less than recommended doses showed a pronounced upward trend with lower BG values preceding and at the time of dose compared to recommended and greater than recommended doses. A continued upward trend followed and resulted in BG values out of range for mean and high BG and in range for low BG (figure 3). BG values following less than recommended doses were higher than those following recommended doses.

Most measures of successive BG change, hypoglycemia, and hyperglycemia differed between recommended, less than recommended and greater than recommended doses (table 4). Successive BG change before and after each insulin administration differed for recommended (p<0.001) and less than recommended doses (p<0.001) but not for greater than recommended doses (p=0.621). The proportion of hypoglycemia was significantly greater for recommended doses than for less than recommended doses (p=0.033); however, the proportion of hypoglycemia was not significantly greater for recommended doses than for greater than recommended doses (p=0.257). The proportion of hyperglycemia was significantly lower for recommended doses compared to greater than recommended doses (p=0.003) and less than recommended doses (p<0.001).

Table 5 summarizes the insulin-resistance parameters. Comparing overall calculated insulin dose, recommended differed significantly from greater than recommended (p<0.001) and less than recommended (p=0.002) doses. Hypoglycemia and hyperglycemia occurred infrequently, and some insulin parameters differed for these conditions between recommended, greater than recommended and less than recommended doses. When nurses administered less than recommended doses overall and in the event of hyperglycemia, the multiplier was dramatically elevated compared to recommended doses.

Table 5.

Insulin-resistance parameters for recommended (R), greater than recommended (GTR), and less than recommended (LTR) doses overall and in the event of subsequent hypoglycemia and hyperglycemia

	R	GTR	LTR	R versus GTR	R versus LTR
Overall, n (%)	170 377 (94.94%)	1505 (0.84%)	7570 (4.22%)
Actual dose, median (IQR)	3.9 (1.7 to 6.1)	6.0 (3.1 to 9.0)	2.5 (0.4 to 4.6)	<0.001	<0.001
Calculated dose		4.8 (2.3 to 7.4)	3.7 (0.8 to 6.7)	<0.001	0.002
Multiplier	0.068 (0.042 to 0.110)	0.066 (0.029 to 0.102)	0.080 (0.043 to 0.118)	<0.001	<0.001
Hypoglycemia, n (%)	1872 (1.04%)	12 (0.01%)	64 (0.04%)
Actual dose	5.1 (1.8 to 6.8)	5.1 (2.4 to 7.8)	2.0 (0 to 4.5)	0.696	<0.001
Calculated dose		3.3 (1.1 to 5.5)	2.45 (0 to 5.0)	0.164	0.001
Multiplier	0.080 (0.045 to 0.115)	0.065 (0.024 to 0.106)	0.057 (0.030 to 0.084)	0.461	0.004
Hyperglycemia, n (%)	1411 (0.79%)	23 (0.01%)	118 (0.07%)
Actual dose	4.1 (1.8 to 6.5)	5.0 (2.8 to 7.3)	2.25 (0 to 5.3)	0.741	<0.001
Calculated dose		3.0 (1.1 to 4.9)	4.55 (0 to 8.9)	0.024	0.625
Multiplier	0.060 (0.026 to 0.095)	0.030 (0.019 to 0.041)	0.085 (0.053 to 0.116)	<0.001	<0.001

Ethnographic data analysis

Nurses said they trusted IIT CDSS because it was evidence-based and made appropriate recommendations. Typically nurses transcribed BG values, selected other required parameters, clicked the calculate button, and accepted recommendations within 2 s and without objection. However, when electing to over-ride, nurses appeared concerned about on-screen insulin recommendations (eg, wincing facial expressions upon seeing doses), which nurses said were higher than they felt comfortable dispensing. Asked to explain over-ride decisions, nurses cited BG trends, primary dextrose sources, general intuition, and a desire to prevent patients' BG levels from ‘bottoming out’ in hypoglycemia. Nearly all over-rides observed were to administer less than recommended doses. Nurses said they found IIT CDSS to be valuable but cited the time required to administer the therapy, especially during episodes of hypoglycemia or emergent care situations, along with additional documentation of BG, insulin, and primary dextrose source data in a separate nurse charting system as impediments to workflow.

Discussion

Nurses expressed favorable attitudes toward a CDSS for IIT and administered 94.9% of recommended insulin doses. Although studies from other institutions have reported IIT CDSS over-ride rates,^{10 13–15} this investigation is the first to our knowledge to determine the direction and magnitude of over-rides. Nurses elected to over-ride 5.1% of dosing recommendations generated using a commonly adopted algorithm, and 83.4% of over-ride doses were less than recommended by CDSS; 45.5% of these doses were ≥50% less than recommended. For over-ride doses, the rate of hypoglycemia was lower and the rate of hyperglycemia higher compared to recommended doses. Nurse over-rides emphasized prevention of hypoglycemia, reflecting the ‘fear of hypoglycemia’ in intensive care units,³³ and occasionally interfered with hyperglycemia control.

Recommended, less than recommended and greater than recommended insulin doses exhibited significantly different BG variability and insulin-resistance characteristics, and may represent different populations. When nurses administered less than recommended doses overall, patients required more insulin as estimated by the multiplier than when nurses administered recommended and greater than recommended doses. This behavior suggests that nurses considered the amount of a recommended dose, not the trend of past insulin resistance, when over-riding IIT CDSS with less than recommended doses. Results of the ethnography help us understand this observation, as nurses appeared concerned about on-screen insulin recommendations prior to administering less than recommended doses.

Based on BG and insulin trends observed in this study, we have identified opportunities to improve IIT CDSS protocol compliance and BG control through nurse education, interface changes, and algorithm modifications. Although mean BG following administration of less than recommended doses increased to clinically acceptable levels below 140 mg/dl⁵ (figure 3), BG nonetheless moved away from the target range. Current IIT CDSS recommendations may be sufficient and require user training to encourage nurses to administer recommended doses more frequently instead of less than recommended doses. The system studied displayed BG trends on screen, and showing the multiplier value on screen might provide decision support regarding patients' insulin resistance to prevent nurses from administering less than recommended doses as frequently. However, displaying the multiplier may also create visual clutter and confusion. For high BG, BG levels following less than recommended doses continued above 150 mg/dl after increasing prior to the dose (figure 3). An alert triggered when BG increases from below to above 150 mg/dl might encourage users to administer a recommended insulin dose instead of an less than recommended dose. When BG was below the target range, nurses administered less than recommended doses that were followed by BG values in the target range (figure 3). Changing the protocol logic to reduce the multiplier by 0.02 rather than 0.01 when BG is <80 mg/dl will generate lower recommended insulin doses that nurses might be more likely to accept, increasing protocol adherence and potentially BG target range achievement.

The 94.9% IIT CDSS protocol compliance rate observed in this study compares to 77%,¹³ 91%,¹⁴ 94%,^{14 15} 95%,¹⁴ 97%,¹⁰ and 98%¹⁴ at other institutions. Variation in compliance across sites may be explained by differences in case mix and other concurrently administered therapies as well as IIT CDSS protocol and workflow differences.²⁷ In this study, nurses accessed IIT CDSS through a computerized provider order entry system, a central part of the institution's workflow that facilitated improved IIT protocol compliance.¹⁹ However, nurses also documented IIT CDSS data in a separate nurse charting system, reducing the time nurses could spend on direct patient care including IIT. Clinical information systems developers should try to reduce double documentation of IIT CDSS data to maximize care process efficiency. This investigation identified sociotechnical interactions involved in nurse IIT CDSS over-rides, and we recently completed a broad ethnographic investigation aimed at understanding how IIT CDSS affects and is affected by other care processes, clinical information systems, and personnel. Explicit disclosure of computer-based workflow factors will facilitate comparison of IIT CDSS studies.^{7 27}

The strengths of our study include the use of a large data set derived from clinical practice in an institution with cultural acceptance of clinical information systems. Several institutions^34–37 use a similar dosing equation^{21 22} to that studied, so results may be generalizable to other critical care unit settings. Limitations include conclusions not being generalizable to other settings due to high clinical informatics commitment^38–42 at the study institution; the unit of analysis being the data point rather than the patient; the observational study design; and measurement of glucose control instead of achievement of therapy goals including reduced rates of infection and cognitive impairment.⁵ Additionally, we excluded almost 12% of IIT CDSS instances from dose analysis to ensure data quality and recognize the over-ride rate could be greater than reported. Missing data, failure to reproduce log data, and incorrectly marked system over-rides can occur due to device malfunction, data-transfer failure, undocumented code changes, and other process errors inherent to the ecology of clinical information systems. Mismatched BG data affect IIT CDSS recommendations,⁸ and we encourage investigators to similarly control for data discrepancies when conducting clinical research. This study used qualitative methods drawn from the field of ethnography to illuminate user behavior, which decision support^{43 44} and over-ride researchers⁴⁵ have advocated. Although only one researcher conducted the ethnographic observations, several useful studies in the informatics literature have also been conducted by single researchers.^{31 46–50}

By examining insulin resistance and BG variability for recommended, greater than recommended and less than recommended IIT CDSS doses, we identified potential modifications to IIT CDSS that might improve protocol compliance and BG control. This investigation focused on IIT CDSS using a linear equation,^{21 22} and more sophisticated quadratic or model-based approaches might lead to better performance.^{51 52} Examining how nurse over-ride behavior changes over time may also provide insight to improve IIT CDSS performance. Researchers and practitioners at other institutions may find our approach useful for understanding and improving IIT CDSS.

Conclusion

Nurse over-ride of CDSS dosing recommendations is a source of variability in IIT in critical care unit settings. Nurses accepted almost 95% of IIT CDSS recommendations, but when electing to over-ride, nurses overwhelmingly administered insulin doses that were less than recommended. Nearly all measures of BG variability and insulin resistance differed between recommended, less than recommended, and greater than recommended IIT CDSS doses. When over-riding IIT CDSS with less than recommended doses, nurses appeared to consider the amount of a recommended insulin dose, not the patient's trend of insulin resistance. For the system studied, nurse education, interface changes, and algorithm modifications might improve BG control while minimizing hypoglycemia. Examining the frequency, direction, and magnitude of IIT CDSS over-rides and associated BG variability and insulin resistance might be useful to researchers and practitioners at other institutions seeking to improve IIT CDSS protocol performance.