The Impact of an Intraoperative Clinical Decision Support Tool to Optimize Perioperative Glycemic Management

Abstract

With the transition from Vanderbilt’s Perioperative Information Management System (VPIMS) to Epic’s Best Practice Advisory (BPA) framework, a replacement intraoperative glucose clinical decision support (CDS) system was designed. We examined changes in the frequency of intraoperative glucose monitoring, hyper- and hypoglycemia rates in the post-anesthesia care unit (PACU), to determine the impact of the changes on glucose management. Data were collected into three phases: 1) VPIMS CDS, 2) No CDS, and 3) BPA CDS. One-way ANOVA was conducted to test the significance of changes in the frequency of glucose monitoring and abnormal glucose across phases. Interrupted time series segmented analysis was performed to assess the autocorrelation and trend over times. A total of 3706 cases were analyzed. The monitoring rate fell from 84.5% in VPIMS CDS to 67.6% in No CDS (p < .001) and increased to 83.1% in BPA CDS (p < .001). The PACU hyperglycemia rate increased from VPIMS CDS to No CDS (5.2% to 10.4%, p < .001) and decreased from No CDS to BPA CDS (10.4% to 7.2%, p = 0.031). The segmented analysis demonstrated immediate changes in the intraoperative monitoring frequency (p< .001) and postoperative hyperglycemia rate (p = 0.002) with the replacement of CDS. The temporary removal of CDS was associated with a significant reduction in intraoperative glucose monitoring and increased hyperglycemia in the PACU. Implementation of the BPA CDS led to a significant improvement in the intraoperative glucose monitoring and glucose management in the PACU.

Introduction

With the increased sophistication of the modern electronic health record (EHR), the implementation of electronic clinical decision support systems (CDS) makes it possible to address deficiencies in caregiving in novel ways. These tools have been widely-used in the perioperative setting to promote the quality of health care provided to surgical patients [1–6]. Nevertheless, many obstacles could constrain the effectiveness of CDS, including low utilization, alert fatigue, and poor usability [7–9], which may result in a reduction in the impact of CDS on clinic outcomes, healthcare quality, and cost reduction [10–12].

Although there was no consensus as to what range an optimal glucose management should be, the perioperative glycemic management avoiding abnormal glucose levels has been reported to positively impact the surgical outcomes [13–15]. A previous study conducted at Vanderbilt University Medical Center (VUMC) demonstrated the effectiveness of using a Vanderbilt Perioperative Information Management System (VPIMS) CDS tool to improve the perioperative glucose management [16]. Specifically, the implementation of VPIMS CDS improved the reliability of intraoperative glucose management, reduced the frequency of recovery room hyperglycemia and reduced the postoperative surgical site infection rate. With the transition to a new EHR (Epic, Verona, WI) in November 2017, it became necessary to design a replacement tool using the new system.

In this study, we sought to assess the impact of replacing the glucose CDS, with interval downtime, followed by the implementation of a new CDS, designed using Epic’s Best Practice Advisory (BPA) framework. We hypothesize that the frequency of intraoperative glucose monitoring would decrease with removal of the CDS and increase with its reimplementation. Since the system specifically targeted patients with impaired glucose management, we secondarily hypothesized that renewed glucose monitoring would also decrease the incidence of postoperative hyper- and hypoglycemia.

Methods

Human subjects protection

The Vanderbilt Human Subjects Research Protection Program determined that this study was exempt from Institutional Review Board (IRB) review and granted a waiver of the requirement for informed consent.

Clinical decision support tool

VPIMS CDS was designed to provide in-room pop-up prompts to the providers to measure the glucose for patients who had impaired glucose management [16].

Based on the similar principle, the new intraoperative glucose alert was designed to facilitate the maintenance of normoglycemia in at-risk patients using Epic’s integrated clinical decision support framework, BPA. The tool was designed to target patients at risk for abnormal perioperative glucose levels. This group includes patients with diabetes, as well as patients for whom impaired glucose management has been established, as evidenced by a history of recent insulin administration. The BPA was displayed to anesthesia providers when the patient’s preoperative documentation met the following criteria: 1) a documented diagnosis of diabetes, without a recorded measurement of blood glucose within the last two hours, or 2) the patient had an insulin administration within the last twelve hours, and did not have a recorded measurement of blood glucose within the last hour. Insulin administration was determined by looking for bolus, infusion or subcutaneous administration records in the medication administration record. For those cases that met these criteria, the BPA would prompt the anesthesia provider to measure the patient’s blood glucose.

Once the BPA triggered, a provider was prompted to respond by selecting one of three acknowledgement reasons as follows: 1) “Deferred - case completion within 30 min”; 2) “Will measure in 15 minutes”; or 3) “Will measure now”. If the provider indicated that they “will measure now”, another BPA would be triggered in the case of no glucose measurement was entered within fifteen minutes. If the provider selected either the “deferred” option, or the “measure in 15 minutes” option, the BPA would alert again if no glucose measurement was entered within 30 min. A provider was not required to choose an acknowledgement reason and could dismiss the BPA without further action.

Data collection

Data were collected from our perioperative and enterprise data warehouses, spanning May 2,2017 to August 6, 2018, including throughout the transition to Epic on November 2, 2017. These data were divided into three distinct time periods as follows:

• VPIMS CDS (May 2, 2017 to November 1, 2017): This constitutes the six-month period before the transition to Epic and the data collected during this phase include health information and intraoperative alert notification data that were generated in the previous EHR. The data collected during this period serves as a baseline.

• No CDS (November 2 to April 11, 2018): During this phase immediately following Epic go-live, BPA triggers were documented in a log for testing and validation, but end-users did not see the BPA alerts, following advise to remove unnecessary alerts during the immediate post-go-live stabilization phase.

• BPA CDS (April 12, 2018 to August 6,2018): This period began with the BPA being enabled for end-users through the end of data collection.

Only the cases that had received an intraoperative alert to monitor glucose (and thus had an indication to monitor glucose, according to the alert criteria) were queried and included in the final analysis. The data collected during these time periods includes a log of all intraoperative alert firings and acknowledgment reasons (what follow-up action the user indicated in response to the alert), as well as the patients’ relevant perioperative data. For the cases with missing data that were considered missing at random were imputed with median values.

Statistical analysis

Data collected during each phase were analyzed for central tendencies, and specific counts were generated in preparation for subsequent comparison between the phases. The data in all three phases were divided into weekly subsets, to adjust for variation in case volume. Counts of cases that received intraoperative glucose monitoring, cases that were hyperglycemic upon first PACU measurement, and cases that were hypoglycemic upon first PACU measurement were obtained for each subset. The counts were then divided by the total number of cases that triggered an intraoperative alert during the respective week to generate a percentage. Hyperglycemia was defined as a blood glucose value ≥250, and hypoglycemia was defined as a blood glucose level of ≤75, consistent with previous studies [16]. All weekly counts between the phases were entered into a contingency table and an unadjusted one-way ANOVA test was performed.

To better understand the impact of the BPA, an interrupted time series segmented analysis was performed. The weekly average percentages described above were analyzed using interrupted time series segmented analysis to evaluate the autocorrelation and secular tendency over time among those three phases.

A two-sided hypothesis testing with p value of less than 0.05 was deemed to indicate statistical significance. Statistical analyses were performed in SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

Results

A total of 3706 cases met inclusion criteria. VPIMS CDS included 1841 cases; No CDS included 1045 cases; and BPA CDS included 820 cases. From these, 68(1.8%) patients with missing BMI values were imputed with median values. Patients had a mean age of 59.9 (SD = 13.4) years and 1595 (43.0%) were female. Detailed patient demographics and clinical characteristics are listed in Table 1. Among three phases, there were a total of 762 (20.6%) patients whose providers didn’t measure the glucose during the surgery. Specifically, 287 (15.6%) patients were in VPIMS CDS, 339 (32.4%) No CDS, and 136 (16.6%) BPA CDS (Appendix Table 4).

Table 1.

Demographic Information of Patients Stratified by Phases

	VPIMS CDS	No CDS	BPA CDS
Cases (N)	1841	1045	820
Age in Years, mean (SD)	59.9 (13.4)	60.0 (13.3)	59.9 (13.6)
BMI in kg/m2, mean (SD)	32.5 (8.9)	32.5 (8.4)	32.2 (8.6)
Gender (%)
Female	768 (41.7%)	470 (45.0%)	357 (43.5%)
Race(%)
Caucasian	1453 (78.9%)	878 (84.0%)	671 (81.8%)
African American	274 (14.8%)	110 (10.4%)	105 (12.8%)
Asian	12 (0.7%)	10 (1.0%)	13 (1.6%)
American Indian	3 (0.2%)	3 (0.3%)	0 (0.0%)
Other	11 (0.6%)	10 (1.0%)	4 (0.5%)
Unknown	88 (4.8%)	34 (3.3%)	27 (3.3%)
ASA Classification (%)
I	1 (0.0%)	1 (0.1%)	0 (0.0%)
II	151 (8.2%)	99 (9.5%)	67 (8.2%)
III	1167 (63.4%)	651 (62.3%)	521 (63.5%)
IV	506 (27.5%)	284 (27.1%)	228 (27.8%)
V	16 (0.9%)	10 (1.0%)	4 (0.5%)
Anesthesia Type (%)
General	1699 (92.3%)	990 (95.9%)	748 (91.7%)

Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory; ASA, American Society of Anesthesiologists; SD, Standard Deviation

The average intraoperative monitoring rate was 84.5% in VPIMS CDS, 67.6% in No CDS, and 83.1% in BPA CDS (p < .001). The incidence of hyperglycemia at first PACU measurement was 5.2% in VPIMS CDS, 10.4% in No CDS, and 7.2% in BPA CDS; there was a statistically significant increase from VPIMS CDS to No CDS (p < .001) and decrease from No CDS to BPA CDS (p = 0.031). We did not detect a difference in the incidence of hypoglycemia at first PACU measurement between phases (VPIMS CDS, 1.2%; No CDS, 1.1% [change from VPIMS CDS, p = 0.754]; BPA CDS, 1.4% [change from No CDS, p = 0.504]) (Table 2).

Table 2.

ANOVA test of Glucose Management in each phase

	Contrast	Contrast Value	p value
Monitoring Rate	VPIMS CDS vs. No CDS	84.5% vs. 67.6%	< .001*
	No CDS vs. BPA CDS	67.6% vs. 83.1%	< .001*
Hyperglycemia Rate	VPIMS CDS vs. No CDS	5.2% vs. 10.4%	< .001*
	No CDS vs. BPA CDS	10.4% vs. 7.2%	0.031*
Hypoglycemia Rate	VPIMS CDS vs. No CDS	1.2% vs. 1.1%	0.754
	No CDS vs. BPA CDS	1.1% vs. 1.4%	0.504

Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory

^*Level of significance p = 0.05

An interrupted time series segmented analysis was conducted to evaluate the quality of intraoperative glucose management between phases. In terms of the intraoperative monitoring rates, an immediate decrease was observed (p < .001) with the transition to Epic (85.6% to 67.6%). Monitoring improved after implementation of the BPA, from 67.6% (offset) to 84.2% (p = 0.001). The trend of the rate was unchanged between VPIMS CDS (slope: 0.08) and No CDS (slope: −0.002) (p = 0.698). Meanwhile, no significant trend change was observed after the BPA implementation in BPA CDS (slope: −0.12, p = 0.711) (Table 3).

Table 3.

Interrupted Time Series Analysis of Glucose Management

Monitoring Rate (%)	Intercept	VPIMS CDS	No CDS	p value
		85.6	67.6	<.001*
		No CDS (offset)	BPA CDS
		67.6	84.2	0.001*
	Slope	VPIMS CDS	No CDS
		0.08	−0.002	0.696
		No CDS	BPA CDS
		−0.002	−0.12	0.711
Hyperglycemia Rate (%)	Intercept	VPIMS CDS	No CDS
		4.4	12.8	0.002*
		No CDS (offset)	BPA CDS
		8.0	9.5	0.685
	Slope	VPIMS CDS	No CDS
		−0.06	−0.19	0.44
		No CDS	BPA CDS
		−0.19	−0.24	0.875
Hypoglycemia Rate (%)	Intercept	VPIMS CDS	No CDS
		1.3	0.01	0.085
		No CDS (offset)	BPA CDS
		2.2	1.4	0.362
	Slope	VPIMS CDS	No CDS
		0.004	0.09	0.081
		No CDS	BPA CDS
		0.09	−0.008	0.178

Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory

^*Level of significance p = 0.05

An immediate impact was only detected in hyperglycemia rate between VPIMS CDS (4.4%) and No CDS (12.8%, p = 0.002), and no difference was found in tendency between each phase (Table 3). Based on the results from interrupted time series segmented analysis, visualizations of the impacts of the interventions and the tendencies over times were shown for intraoperative monitoring and hyperglycemia rate in Fig. 1 and Fig. 2, and hypoglycemia rate in Appendix Fig. 3.

Fig. 1.

Visualization of the intraoperative monitoring rate change among phases. (Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory)

Fig. 2.

Visualization of the hyperglycemia rate change in PACU among phases. (Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory)

Discussion

In this paper, we report the impact of the replacement of a perioperative glucose management clinical decision support tool on providers’ behavior and patient outcomes. The decommission of the existing clinical decision support tool resulted in a significant reduction in the frequency of intraoperative glucose monitoring and increased PACU hyperglycemia. With the implementation of the newly designed BPA system, intraoperative glucose surveillance rates improved, as well as decreased rates of postoperative hyperglycemia in at-risk patients.

We observed changes in the intraoperative glucose monitoring rate between each phase, confirming what we reported in a previous study, demonstrating the sustained impact of a perioperative glucose management clinical decision support tool [16]. Our findings reinforce the importance of maintaining and sustaining key clinical decision support tools, particularly at our university-affiliated tertiary-care medical center, with relatively sustained turnover of in-room providers. This turnover does not explain the change between phases, however, as changes were nearly immediate with decommission and reimplementation of the alert and did not correspond to changes in the academic year. We contend that the effect of this clinical decision support tool is dependent on the ongoing presence of the tool itself and may be less dependent on learned behavior. This provides a strong rationale for implementation and ongoing maintenance of these types of clinical decision support tools.

Other studies have reported that the adoption of clinical decision support tool could significant positively impact the performance of providers [17]. However, it is controversial to use practitioners’ behaviors as a proxy for patient outcomes and others have suggested that studies of clinical decision support tools should assess the impact of the decision support system on both providers’ behavior and clinical endpoints [18]. In this study, we demonstrate that the clinical decision support tool impacts both and that the decommission of this tool results in a nearly-immediate impact in both, which elucidates the ability of clinical decision support tool in maintaining the providers’ glucose management pattern. It has been reported that in spite of the providers acknowledge the importance of regular and frequent glucose monitoring in intraoperative glycemic management, they did not proportionally adhere to the insulin dosage recommendations [19]. Thus, we believe that a live intraoperative decision support tool would be an efficient way to facilitate the consistent monitoring of the blood glucose levels of patients, especially for those who had impaired glucose management. Another strength of this study is to use the interrupted time series analysis, which is the most powerful quasi-experimental approach in assessing the longitudinal effects of interventions for the research design with a temporal component [20, 21].

There are several limitations to this study. First, the results of this study are unadjusted and may have been impacted by potential confounding factors. Data are limited, however, on what variables could plausibly introduce confounding in this situation. In the absence of plausible and evidence-based confounding, we have presented unadjusted data. An additional limitation is our use of cutoffs to define hyperglycemia and hypoglycemia. While we agree that there is no consensus for these terms, we attempted to use thresholds that are widely used and that have been previously used in studies of clinical decision support for glucose management. However, given the relatively low event rate, especially for the hypoglycemia rate analysis, we must point out that an inadequate power may lead to the higher probability of committing a type II error. Larger, and potentially multicenter, studies could explore this important outcome in greater detail. Finally, the duration of the three phases was not identical. While both were sufficiently long that we would contend that behaviors likely reached a steady state, we cannot definitively state that the results, particularly those presented for BPA CDS, will be sustained.

Conclusion

In this study, we observed a reduction in the average frequency of intraoperative glucose monitoring and increased incidence of hyperglycemia in the PACU with the decommission of our original alert system. The implementation of new BPA led to a nearly immediate increase in the rate of intraoperative glucose surveillance, as well as the decreased rates of hyperglycemia in the patients with impaired glucose management. In light of these findings, we believe that the adoption and ongoing use of a clinical decision support tool is important to optimize perioperative glucose management in at-risk patients.

Appendix

Table 4.

Demographics and Clinical Characteristics of Patients with No Intraoperative Glucose Measurement

	VPIMS CDS	No CDS	BPA CDS
Cases (N)	287	339	136
Age in Years, mean (SD)	59.6 (13.4)	60.9 (13.1)	58.7 (14.7)
BMI in kg/m2, mean (SD)	33.6 (9.2)	34.0 (9.0)	34.0 (11.3)
Gender (%)
Female	144 (50.2%)	185 (54.6%)	74 (54.4%)
Race (%)
Caucasian	232 (80.8%)	290 (85.6%)	115 (84.6%)
African American	39 (13.6%)	34 (10.0%)	12 (8.8%)
Asian	3 (1.1%)	4 (1.2%)	1 (0.7%)
Other	1 (0.3%)	3 (0.8%)	1 (0.7%)
Unknown	12 (4.2%)	8 (2.4%)	7 (5.2%)
ASA Classification (%)
I	1 (0.4%)	0 (0.1%)	0 (0.0%)
II	51 (17.8%)	49 (14.4%)	22 (16.2%)
III	207 (72.1%)	271 (79.9%)	97 (71.3%)
IV	28 (9.7%)	19 (5.6%)	15 (11.0%)
V	0 (0.0%)	0 (0.0%)	2 (1.5%)
Anesthesia Type (%)
General	248 (86.4%)	298 (90.0%)	101 (87.5%)
Postoperative Hyperglycemia (%)	10 (4.1%)	16 (6.4%)	2 (2.1%)
Postoperative Hypoglycemia (%)	4 (1.6%)	2 (0.8%)	2 (2.1%)

Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory; ASA, American Society of Anesthesiologists; SD, Standard Deviation

Fig. 3.

Visualization of the hypoglycemia rate change in PACU among phases (Abbreviations: VPIMS, Vanderbilt Perioperative Information Management System; CDS, Clinical Decision Support; BPA, Best Practice Advisory)