Effects of blood glucose transcription mismatches on a computer-based intensive insulin therapy protocol

Abstract

Purpose

Computerized clinical decision support systems (CDSS) for intensive insulin therapy (IIT) generate recommendations using blood glucose (BG) values manually transcribed from testing devices to computers, a potential source of error. We quantified the frequency and effect of blood glucose transcription mismatches on IIT protocol performance.

Methods

We examined 38 months of retrospective data for patients treated with CDSS IIT in two intensive care units at one teaching hospital. A manually transcribed BG value not equal to a corresponding device value was deemed mismatched. For mismatches we recalculated CDSS recommendations using device BG values. We compared matched and mismatched data in terms of CDSS alerts, blood glucose variability, and dosing.

Results

Of 189,499 CDSS IIT instances, 5.3% contained mismatched BG values. Mismatched data triggered 93 false alerts and failed to issue 170 alerts for nurses to notify physicians. Four of six BG variability measures differed between matched and mismatched data. Overall insulin dose was greater for matched than mismatched [matched 3.8 (1.6–6.0), median (interquartile range, IQR), versus 3.6 (1.6–5.7); p < 0.001], but recalculated and actual dose were similar. In mismatches preceding hypoglycemia, recalculated insulin dose was significantly lower than actual dose [recalculated 2.7 (0.4–5.0), median (IQR), versus 3.5 (1.4–5.6)]. In mismatches preceding hyperglycemia, recalculated insulin dose was significantly greater than actual dose [recalculated 4.7 (3.3–6.2), median (IQR), versus 3.3 (2.4–4.3); p < 0.001]. Administration of recalculated doses might have prevented blood glucose excursions.

Conclusions

Mismatched blood glucose values can influence CDSS IIT protocol performance.

Introduction

Intensive insulin therapy (IIT) is the standard of critical care, but concerns exist regarding its effectiveness and safety [1]. In surgical and trauma intensive care unit patients treated with a computerized intensive insulin therapy protocol [2, 3], blood glucose variability [4] and insulin resistance [5] were associated with mortality, and delayed blood glucose measurements were associated with severe hyper- and hypoglycemia [6, 7]. These findings suggest that workflow may influence computer-based IIT performance and patient outcomes [6]. Clinical decision support systems (CDSS) for IIT are commonplace [8, 9], and many implementations rely on manual transcription of blood glucose values to generate recommendations, a practice that could yield unintended consequences including error [8, 9]. CDSS IIT approaches that ignore transcription mismatches—when manually entered values do not equal corresponding automatically captured device values—implicitly assume that differences between matched and mismatched data are nonsignificant and do not affect IIT protocol performance. The purpose of this study is to measure the frequency of blood glucose transcription mismatches and their effect on intensive insulin therapy protocol performance.

Materials and methods

The cohort included all critically ill or injured mechanically ventilated patients treated with a computer-based intensive insulin therapy protocol in the surgical and trauma intensive care units at Vanderbilt University Hospital, an urban tertiary care facility. We examined only patients with five or more CDSS IIT values. Online Resource 1 presents patient characteristics.

Online Resource 2 describes the CDSS IIT in detail.¹ The CDSS [2, 3, 5, 9] uses a linear equation [10, 11] that adjusts a “multiplier” according to current and previous blood glucose (BG) values. Slight (e.g., 138 versus 139) and large differences (e.g., 138 versus 238) in BG value transcription can have major, minor, or no effects on dosing recommendations, depending on clinical scenario.

For each patient, we retrospectively linked each manually transcribed BG value with the device BG value closest to the manually transcribed value’s timestamp within a 1-h window, i.e., 1 h before or 1 h after, to accommodate time variations between computers and devices. Manually transcribed and device-captured BG value pairs that were equal were designated matched and those that were unequal were designated mismatched. For mismatched instances, we recalculated CDSS recommendations using corresponding device BG values. We assumed that nurses transcribed BG values independently; thus, we considered previous input to be correct and examined the effect of each manual transcription on CDSS output.

Measurements

The study objective is to determine the frequency of blood glucose mismatches and their effect on IIT protocol performance. We hypothesized that: (1) manually transcribed blood glucose values do not always equal device values, (2) matched and mismatched data differ in terms of alerts generated by CDSS, (3) matched and mismatched data differ in terms of blood glucose variability, (4) matched and mismatched data differ in terms of dosing, and (5) recalculated doses differ from corresponding actual doses but are similar to recommended doses generated with matched data.

To assess alerts, we compared historical CDSS output versus output generated from corresponding automatically captured device values. To measure BG variability, we focused on blood glucose excursions in terms of successive device BG change [4], hypoglycemia (i.e., current manually transcribed BG ≥60 mg/dL and next manually transcribed BG <60 mg/dL), and hyperglycemia (i.e., current manually transcribed BG <200 mg/dL and next manually transcribed BG ≥200 mg/dL). Successive BG change reflects both steady and rapid fluctuations in the distribution of BG values [4]. Hypoglycemia² and hyperglycemia measure safety and effectiveness, respectively.

To evaluate dosing, we examined insulin dose (unit/h), equation multiplier, and intravenous 50% dextrose (D50) dose (mL). Insulin dose and multiplier have been identified as markers for insulin resistance [5]. To control the effect of nurse overrides, we excluded override doses and examined recommended doses. We then compared matched doses, mismatched actual doses generated with manually transcribed values, and mismatched recalculated doses generated with corresponding device values. Matched recommended doses, free of override bias and mismatched data, served as the reference standard of IIT protocol adherence.

Statistical analysis

To summarize and compare normally distributed continuous variables, we determined mean and standard deviation (SD) and used two-sample t tests for independent samples. To summarize and compare non-normally distributed continuous variables, we determined median and interquartile range (IQR) and used the Wilcoxon rank-sum test for unpaired data and Wilcoxon signed-rank test for paired data. We used a χ² test to compare differences in proportions. A two-sided p value less than 0.05 indicated statistical significance. To perform calculations we used STATA 10.1 (STATA Corp., College Station, TX, USA). All patient data were stored in a secure, password-protected database and deidentified prior to analysis and reporting. The Vanderbilt University Institutional Review Board approved this study.

Results

Mismatched pairs accounted for 5.3% of IIT data (Fig. 1). Matched and mismatched pairs differed in most respects. Mismatched data caused the CDSS to trigger 93 false alerts and to fail to issue 170 alerts for nurses to notify physicians. Blood glucose variability differed between matched and mismatched data in four of six measures. Online Resource 3 presents detailed findings.

Fig. 1.

Categorization of IIT manually transcribed and device-captured pairs and subsequent insulin doses generated using blood glucose (BG) values. Gray areas include recommended and override doses, whereas black areas include only recommended doses to control for effect of overrides. A total of 203,188 IIT instances were available for analysis. Of manually transcribed blood glucose values, 11,901 had no corresponding device value identified and were excluded. An additional 1,788 IIT instances (<1%) produced output unequal to historical CDSS recommendations and were excluded. Of manually transcribed and device-captured blood glucose values, 189,499 pairs remained for analysis

Compared with matched data, mismatched data generated lower dosing parameters overall, with some exceptions (Table 1). Hypoglycemia occurred in 1.1% (n = 2,073) of total CDSS IIT activity. Of these instances, 6% (n = 125) contained mismatched data, and actual dose was 22.9% higher than recalculated (p < 0.001). Additionally, matched recommended dose was 37.2% higher than recalculated (p < 0.001). Hyperglycemia occurred in 0.9% (n = 1,780) of total CDSS IIT activity. Of these instances, 12.8% (n = 228) contained mismatched data, and actual dose was 42.4% lower than recalculated (p < 0.001). Additionally, matched recommended was 14.6% lower than recalculated (p = 0.006).

Table 1.

Dosing parameters for matched and mismatched data including recalculated doses

	Total (recommended and override)			Recommended only
	Matched	Mismatched	Matched versus mismatched	Matched recommended	Mismatched actual	Mismatched recalculated	Matched recommended versus mismatched actual	Matched recommended versus mismatched recalculated	Mismatched actual versus mismatched recalculated
Overall, n (%)	179,479 (94.7%)	10,020 (5.3%)		170,377 (94.7%)	9,474 (5.3%)
Insulin (U/h)	3.8 (2.1–6.5)	3.6 (2.0–6.1)	<0.001	3.9 (2.1–6.5)	3.6 (2.0–6.1)	3.7 (2.0–6.2)	<0.001	<0.001	0.6991
Multiplier	0.076 (0.048–0.117)	0.066 (0.038–0.109)	<0.001	0.760 (0.048–0.117)	0.0650 (0.038–0.108)	0.0647 (0.037–0.107)	<0.001	<0.001	<0.001
D50 (mL)	15 (5–10)	15 (5–10)	0.283	15 (5–10)	15 (5–10)	15 (5–10)	0.251	<0.001	<0.001
Hypoglycemia, n (%)	1,948 (1.0%)	125 (0.1%)		1,872 (1.0%)	107 (0.1%)
Insulin (U/h)	4.2 (2.2–7.3)	3.4 (1.7–6.0)	0.012	4.3 (2.25–7.3)	3.5 (1.8–6.0)	2.7 (1.1–5.7)	0.022	<0.001	<0.001
Multiplier	0.079 (0.050–0.120)	0.060 (0.033–0.116)	0.01	0.080 (0.051–0.120)	0.060 (0.033–0.119)	0.060 (0.030–0.120)	0.008	0.002	<0.001
Hyperglycemia, n (%)	1,552 (0.8%)	228 (0.1%)		1,411 (0.8%)	214 (0.1%)
Insulin (U/h)	4.1 (2.2–7.0)	3.3 (2.15–4.2)	<0.001	4.1 (2.4–7.1)	3.3 (2.3–4.2)	4.7 (3.8–6.7)	<0.001	0.006	<0.001
Multiplier	0.061 (0.032–0.100)	0.030 (0.030–0.059)	<0.001	0.060 (0.031–0.099)	0.030 (0.030–0.056)	0.030 (0.030–0.056)	<0.001	<0.001	0.317

“Recommended” only controls for effect of override doses. Matched and mismatched data generated significantly different insulin dosing parameters, especially for episodes of hypoglycemia and hyperglycemia. Of patients, 27.5% (n = 1,110) experienced at least one episode of hypoglycemia (current BG ≥60 mg/dL and next BG <60 mg/dL). Severe hypoglycemia (current BG ≥40 mg/dL and next BG <40 mg/dL) occurred in 172 instances (0.09% overall), and six contained mismatched data. Of patients, 3.9% (n = 157) experienced at least one episode of severe hypoglycemia. Recalculated D50 dose [mean ± standard deviation (SD): 14 ± 9] was significantly lower than mismatched actual (mean ± SD: 17 ± 8, p < 0.001) and matched recommended (mean ± SD: 16 ± 5, p < 0.001). D50 doses in table are skewed because of 5 mL dose increments

The significance for bold values is stated as p < 0.05

Discussion

Manual transcription of blood glucose values is a source of intensive insulin therapy process variability associated with patient care changes and affecting clinical data and provider workflow. Manual transcription may affect paper-based IIT, but computer-based approaches provide detection of mismatches more readily. In a review of CDSS IIT approaches, Eslami et al. [8] identified manual transcription of blood glucose values as a “critical safety issue. ” In our study, overall insulin doses generated with matched and mismatched data differed, but recalculated doses were similar to actual doses, suggesting that the effect of mismatched data is insignificant overall. However, in cases of hypoglycemia and hyperglycemia, dosing differences caused by mismatched data may be clinically significant: the administration of recalculated doses that were less than and greater than actual recommendations, respectively, might have prevented “never events” [12] from occurring. Other patient-level factors may contribute to blood glucose variability, but this study quantifies mismatched data as a parameter to consider for IIT improvement.

Matched and mismatched data exhibited significantly different characteristics and may represent distinct populations. Compared with matched data, mismatched data showed a significantly greater mean BG value, lower multiplier, greater magnitude of successive BG change, and greater occurrence of hyperglycemia. Additionally, mismatched data generated fewer “notify physician” alerts than would have occurred had transcribed values matched device values. Mismatches occurred about as frequently as three other system events: D50 administrations and nurse-initiated insulin overrides (Online Resource 3), which were expected, and manual transcriptions missing device values (Fig. 1), which were unanticipated. Online Resource 4 lists potential reasons for mismatched and missing data [13, 14]. Manual BG transcription affects protocol performance, and future work will involve direct observation and nurse interviews to understand CDSS IIT usage.

Results of this study will fuel the debate regarding replacement of handheld glucometers with continuous monitoring technology [15, 16]. The rate of insulin doses generated with mismatched data exceeds the 3.4 per million event defect rate in industrial Six-Sigma quality efforts [17] but compares favorably with studies of data entry error and CDSS [18, 19]. Although automatic capture of device data can improve CDSS recommendations [20], implementing new monitors that automatically measure and record BG data poses challenges related to cost, system integration, and unintended consequences. Researchers and practitioners can explore incremental variability reductions through CDSS interface design, nursing staff education, and electronic surveillance.

This study has strengths and weaknesses. The large dataset reflects standardized care for two intensive care units, and high compliance with computerized recommendations shows that the protocol is well received by clinicians. Findings may not generalize to sites using different dosing equations or with less informatics development and/or organizational commitment. However, several institutions [8, 9] use the equation studied [10, 11]. Because we calculated the multiplier using mismatched current but not mismatched previous device BG values, this analysis does not show impact over time of improper multiplier adjustments. This study examined IIT protocol performance, not individual patient effects, and showed correlation, not causation, between data mismatches and BG changes.

Mismatched data occurred relatively infrequently, influenced IIT performance, appear to have varying clinical impact and etiology, may compromise patient safety, and may represent a different population than matched data. Researchers and practitioners should pay greater attention to frequency and effect of mismatched blood glucose data on IIT performance.