Using the Glucometrics Website to Benchmark ICU Glucose Control Before and After the NICE-SUGAR Study

Abstract

Prior to 2009, intensive glycemic control was the standard in main intensive care units (ICUs). Glucose targets have been recalibrated after publication of the NICE-SUGAR study in that year, followed by updated guidelines that endorsed more moderated control. We sought to determine if the prevalence of hyperglycemia in US ICUs had increased after the NICE-SUGAR study’s results were reported. We used data from hospitals submitted to the Yale Glucometrics™ website to assess mean blood glucose values, percentage of blood glucose within various ranges, and the prevalence of hypo- and hyperglycemic excursions, based on the patient-day method, comparing the pre- to post-NICE-SUGAR time period. Among more than a total of 2 million blood glucose determinations, comprising 408 790 patient-days, median patient-day blood glucose decreased from 144 mg/dL to 141 mg/dL (P < .001) in the pre- versus post-NICE-SUGAR time period. The percentage of patient days with a mean blood glucose of 110-179 mg/dl increased from 58.3 to 63.6%. The percentage of patient-days with either hypoglycemia (<70 mg/dl) or severe hyperglycemia (≥300 mg/dl) decreased during this time. Our results suggest that glycemic control in US ICUs has improved when comparing time periods before versus after publication of the NICE-SUGAR study. We found no evidence that fewer hypoglycemic events were achieved at the expense of more hyperglycemia.

As a result of increased counterregulatory hormone concentrations, hyperglycemia is common in acutely ill patients, including those without a prior history of diabetes. Elevated blood glucose concentrations in both critical- and non-critical-care settings are also associated with increased morbidity and mortality and higher health care expenditures.^1,2 Epidemiological data support and some clinical trials have suggested a mortality benefit from tight glycemic control in the hospital, with most studies having been conducted in the intensive care unit (ICU). Predominately based on observational data and a single-center surgical ICU trial from Belgium published in 2001,³ by 2004, many professional groups in the United States had endorsed a stringent blood glucose target of <110 mg/dl.^4-6 In response, institutions around the country established protocols and strategies to meet or approach this new quality measure, with aggressive use of intravenous insulin infusions in many hyperglycemic patients, even those without a history of diabetes.

In subsequent years, however, the findings of the Belgian investigators could not be replicated by others, including 2 multicenter trials^7,8 and a separate study conducted in the medical ICU from the same research group.⁹ Finally, in 2009, the largest study of its type, the landmark NICE-SUGAR study,¹⁰ which included more than 6100 critically ill patients from 42 hospitals in Australia, New Zealand, and Canada, found no benefit to the use of an insulin infusion protocol targeting 81-108 mg/dl versus “conventional” therapy (insulin infusion if necessary to maintain blood glucose 140-180 mg/dl). In fact, an actual 14% relative increase in 90-day mortality was detected in the group randomized to more intensive therapy. Although these patients did experience 6-fold greater incidence of severe hypoglycemia (defined as BG < 40 mg/dl), and a subsequent publication from these collaborators identified hypoglycemia as a predictor of mortality, the actual relationship between hypoglycemia and excess mortality remains incompletely understood.

The NICE-SUGAR findings, however, which became widely disseminated, led to calls to increase the optimal glycemic range for patients admitted to the ICU. Within 2 months, a consensus statement from the American Association of Clinical Endocrinologists (AACE) and the American Diabetes Association (ADA) endorsed a more conservative target of 140-180 mg/dL for critically ill patients,¹¹ with admonitions to avoid hypoglycemia in this setting. Since then, no large study has been able to refute the findings of the NICE-SUGAR investigators. Adherence to the AACE-ADA recommendations has been variable across the nation, with some centers readjusting their ICU glucose targets, and some continuing their prior, more intensive, approaches. There also remain unanswered questions about whether certain patient types (for example, post–cardiothoracic surgery) may attain some benefit from more stringent control.

Valid concerns have subsequently been raised that the higher national blood glucose targets may result in physicians managing the critically ill reverting back to permissive hyperglycemia as the standard of care in the ICU, in an overabundance of caution to avoid hypoglycemia.¹² It is not known, however, whether on a national basis, any reassessment of ICU glucose targets based on the NICE-SUGAR findings has had a deleterious effect on mean blood glucose control in this setting.

Glucometrics™ is a free, open-access, academic website, based at the Center for Medical Informatics at Yale University School of Medicine.¹³ It allows hospitals to upload patient-level deidentified blood glucose data to obtain aggregated reports to assess the quality of blood glucose control. Data were collected for 3 years prior to the publication of NICE-SUGAR. We used this unique opportunity to assess the impact on national ICU glucose trends both before and after the study results became available and were interpreted by major national organizations. Our specific hypothesis was that blood glucose values submitted to the website would show significantly more hyperglycemia in the time period after publication of the NICE-SUGAR results (March 2009) and AACE-ADA Consensus Statement (May 2009), as compared to before.

Methods

Website and Analytical Methods

The Glucometrics website, accessible at http://metrics.med.yale.edu, has been providing service in an open-access fashion to institutions interested in a regimented, uniform analysis of inpatient blood glucose data.¹³ While institutions remain anonymous, users create a hospital or individual unit/ward profile, reporting simple hospital demographics (region, size, etc). Data files must also be labeled as collected from ICU or non-ICU patient care areas. Submitted data files must include a deidentified unique patient identifier, time and date stamp, and the blood glucose value in mg/dl, typically obtained from point-of-care capillary blood glucose meter. The website’s software program transforms these data into aggregate metrics for each ward or unit submitted. As previously described,^13,14 our analytical method involves 3 separate assessments. The first is based on the “patient sample” as the analytical unit. Here, each blood glucose sample contributes equally to the average for that ward or unit, with hyperglycemic and hypoglycemic events calculated as the percentage of samples above or below certain predefined thresholds. The second is based on the “patient-stay.” Here each patient’s average blood glucose during the entire hospitalization is calculated, and composite data for the ward/unit are subsequently derived. Hyperglycemic and hypoglycemic events during the course of the hospitalization will thereby characterize the entire hospital stay. Third is the “patient-day” model, which we consider to be the most equitable assessment, since it allows for comparisons of glucose control over equal time periods, while also guarding against oversampling during hyperglycemic and hypoglycemic periods. In this model, the mean blood glucose is calculated for each patient day, and these values are then utilized to derive the ward/unit composite data. In the patient-day model, any day with a hyperglycemic and hypoglycemic event is so identified but that event does not necessarily characterize the entire hospitalization. The net results are hyperglycemic and hypoglycemic percentages intermediate between the relatively lower percentages as calculated using the patient sample, and the relatively higher percentages using the patient stay. Such an approach has been adopted by other institutions and investigators, following our initial publication.

Study Methods

By the end of November 2010, there had been 15 735 661 glucose readings uploaded to our website. Exclusion criteria were applied to screen out the following files or glucose values:

1. Data from non-US institutions

2. Data from these specialties: anesthesiology, emergency medicine, physical medicine, psychiatry, obstetrics/gynecology, pediatrics, and ophthalmology

3. Data from step-down wards, which typically function at an intermediate level between ICU and general wards

4. Files of test data made available to users or tagged as “test files” by users

5. Files with <50 glucoses (often test data)

6. Individual files with <5 glucoses (likely 1-day admissions in the hospital)

7. Glucose readings >999 and <10 due to concerns about validity. An exception was a −1 value, which some glucometers may record as a code for values >500. As was previously done in our original report,¹⁴ these were replaced with a value of 501 mg/dl.

8. Data with erroneous date-time values

9. Duplicate data

From the 15 735 661 BG readings uploaded to the website since 2006 (which included data back to January 2002), a total of 7 361 752 glucoses were excluded, leaving 8 373 909 glucose readings, 2 315 546 of which were from ICUs.

To analyze the impact of the NICE-SUGAR study, which was published in March 2009, the data gathered was divided into pre-NICE-SUGAR (time stamp before March 1, 2009, midnight) and post-NICE (time stamp after July 1, 2009, midnight). The post-NICE date was chosen to be 3 months after the study’s publication to allow some time for the results to affect practice. After excluding glucose values with a time stamp between March 1, 2009, and July 1, 2009, there were 7 301 351 glucose readings, 2 005 245 coming from the ICU. Of these, 688 176 were glucoses pre-NICE-SUGAR and 1 317 069 were post-NICE-SUGAR. From these submitted glucoses, glucometrics were calculated. This study focused on patient-day, specifically mean patient day, for each group analyzed. Comparison of median patient-day values pre- versus post-NICE-SUGAR were then performed to determine if there was any worsening of glycemic control (ie, increase in hyperglycemia) after guidelines recommending higher blood glucose targets were set forth. We used the Mann–Whitney U test to determine significance at a level of P < .05 for medians and Pearson’s chi-square test for contingency tables of glucose ranges.

Follow-up analysis was performed using data submitted from ICUs from January 1, 2011, to December 31, 2013. The same exclusion criteria were applied.

Results

The data were from blood glucose values submitted by 64 ICUs. There were 144 104 patient-days with blood glucose values submitted prior to the publication of the NICE-SUGAR study, and 264 686 patient-days beyond 3 months postpublication. A modest decrease in mean patient-day blood glucose from 158.2 ± 59.2 mg/dL to 152.7 ± 52.2 mg/dL and of medians from 144 mg/dL to 141 mg/dL (P < .001) in the post-NICE-SUGAR time frame was noted (Table 1). Upon segmenting the patient-day blood glucose values into glycemic ranges, we detected an increase from 58.3% (pre) to 63.6% (post) of total patient-days with mean blood glucose in the range of 110-179 mg/dL (P < .001). This consisted of small increases in both the 110-139 mg/dl and the 140-179 mg/dl ranges. Simultaneously, there was also a decrease in patient-days with mean blood glucose > 180 mg/dL from 25.8% (pre) to 21.3% (post), as well as a small decrease in patient-days with mean blood glucose between 70-109 mg/dL (15.4% [pre] to 14.0% [post]) (Table 2). As regards patient-days with adverse glycemic episodes, we observed a decrease in both hypoglycemia (7.2% [pre] to 5.9% [post] of patient-days with any blood glucose < 70 mg/dL) as well as in severe hyperglycemia (10.4% [pre] to 7.9% [post] of patient-days with at least 1 blood glucose > 300 mg/dL) (Table 3). In addition, there is also a narrower glycemic spread of patient-samples during the latter time frame (212 vs 186 mg/dl).

Table 1.

Overall ICU Glycemic Comparisons Based on Patient-Day Blood Glucose, Pre- Versus Post-NICE-SUGAR.

	Pre	Post
Patient-days	144 104	264 686
Mean ± SD (mg/dl)	158.2 ± 59.2	152.7 ± 52.2
Median (IQR) (mg/dl)	144 (119-182)	141 (119-172)
5th-95th percentile (mg/dl)	93-271	95-252
Glycemic spread (mg/dl)	178	158

P < .001 for difference in medians. IQR, interquartile range; SD, standard deviation.

Table 2.

ICU Patient-Day Blood Glucose Ranges, Pre- Versus Post-NICE-SUGAR.

	Percentage patient-day glycemic ranges
Blood glucose (mg/dL)	Pre	Post
70-109	15.4	14.8
110-139	30.3	34.0
140-179	28.0	29.6
≥180	25.8	21.3

P < .001 for each comparison.

Table 3.

Glycemic Excursions: ICU Patient-Days With at Least 1 Hypoglycemic or Severely Hypoglycemic Blood Glucose Pre- Versus Post-NICE-SUGAR.

	Percentage patient-days with ≥1 glycemic excursion
Blood glucose (mg/dL)	Pre	Post
<70	7.2	5.9
<40	0.9	0.7
≥300	10.4	7.9

P < .001 for each comparison.

In the follow-up analysis of ICU glucoses from January 1, 2011, to December 31, 2013, that included 2 727 671 glucoses and 597 268 patient-days, the mean patient-day glucose was 152 mg/dL and the median was 140 mg/dl. The proportion within the range of 70 to 109 mg/dL increased slightly to 17.3%, while that in the range of 110 to 179 mg/dL decreased slightly to 61.2 %.

Discussion

Our results may allay concerns that the NICE-SUGAR study¹⁰ and the subsequent AACE-ADA Consensus Statement¹¹ has lead to any retreat in controlling hyperglycemia in ICUs in the United States. Indeed, our data would seem to suggest that there has been modest, but statistically significant, improvements in overall glycemic control during this time frame, with a decrease in mean patient-day blood glucose of approximately 5-6 mg/dl, an absolute 5% increase in the number of patient-days whose mean blood glucose was between 110-180 mg/dL range, with a concomitant single digit absolute decreases in the percentage of patient days containing any hypoglycemic or severely hyperglycemic readings. Importantly, however, our results show that despite the higher BG target set forth in the AACE-ADA Consensus Statement on inpatient hyperglycemia as a reflection of the results of NICE-SUGAR, there appears to be no evidence that hospitals have begun to tolerate hyperglycemia in an effort to avoid hypoglycemia. Moreover, the narrower spread of blood glucose for patient-sample, patient-stay, and patient-day in the later time frame may be evidence of reduced glycemic fluctuations (although we did not specifically assess glycemic variability in this analysis).

Although our initial study design was to analyze data submitted to the website until the end of November 2010, these trends have persisted in a follow-up examination of submitted data over the 3 subsequent years. Since 2011, the mean blood glucose levels submitted from ICUs remains approximately 152 mg/dl. The percentage of patient days within the 110-179 mg/dL range has been approximately 61% and the percentage of patient days with any hypoglycemic or severely hyperglycemic episode have been 5% and 7.5%, respectively.

There are limited benchmarking data in the medical literature on inpatient glucose control from US hospitals and none that have assessed the impact of the NICE-SUGAR study or the ADA-AACE Consensus Statement in 2009. Bersoux et al found no clinically meaningful change in mean blood glucose levels from 126 US hospitals from 2007 to 2009 (167 vs 166 mg/dL) and no changes in patient-day weighted mean blood glucose > 180 mg/dL.¹⁵ Wexler and colleagues outlined high rates of hypoglycemia and hyperglycemia in a report that merged 2 surveys involving a total of 44 US hospitals, but they did not specify ICU versus non-ICU patients, nor did they assess for temporal trends.¹⁶ Kaukonen et al recently reported glycemic data from 49 hospitals in Australia and New Zealand before and after NICE-SUGAR.¹⁷ In contrast to the United States, these investigators found that intensive insulin management was not standard therapy either before or after 2009. Mean first day blood glucose was essentially unchanged during this time period (143.4 to 144.5 mg/dl). There were also similar rates of moderate and severe hypoglycemia. These data, however, provide little insight into the effect of this landmark clinical trial in the United States, where, by 2009, intensive insulin infusions had already become the standard of care in most ICUs.

There are several limitations to our study. Our results are based solely on data voluntarily and anonymously submitted to the website by various hospitals and institutions. The data are mainly restricted to capillary blood glucose readings. While they are very convenient to obtain and widely used, these are neither as accurate nor precise in the assessment of glycemic status as are laboratory plasma glucose measurements. Furthermore, the individual meters used in the different institutions vary in accuracy and precision, as does the skill with which they were used by patient support staff. Our study is also a cross-sectional analysis of data over 2 time periods, with different hospitals were represented during each. We also cannot tell if submitting hospitals were conducting any quality assurance projects before or after submitting their data. Accordingly, while our goal was to determine if there were any overall trends in glycemic management during these time periods, we are well aware that specific measures and actions taken by individual hospitals to address their inpatients’ glycemic control would likely have a more significant impact than the external forces we were attempting to assess. So, while we have not found any overall deterioration (ie, more hyperglycemia) in ICU glucose control during the assessed periods, our data cannot obviously comment on any changes in glycemic control at an intrainstitutional level. Our follow-up data over the subsequent 3 years shows stability since 2010 and argues against any major secular trends in ICU glucose control. Finally, because of the cross-sectional nature of the study, clinical outcomes as a result of glycemic control could not be assessed. Moreover, since our website does not collect any information on severity of illness, which could influence glycemic (and other) outcomes, this parameter cannot be compared between the 2 time periods.

In summary, we found no evidence of more hyperglycemia in the ICUs following the NICE- SUGAR study, based on data submitted to the Glucometrics website, comparing before versus after publication of this landmark clinical trial that called into question the benefit of intensive glucose management in the critically ill. Instead, our analysis reveals modest reductions in both hypoglycemic and hyperglycemic episodes, with more patient-days within a desirable glycemic range. Indeed, one might conclude that acknowledgement of the NICE-SUGAR findings and the cautions expressed in the AACE-ADA Statement may have had desirable effects in promoting more reasonable and safer glycemic control strategies in hospitalized patients.

Conclusions

Our results from the Glucometrics website suggest that glycemic control in US ICUs has improved modestly when comparing time periods before versus after publication of the NICE-SUGAR study and the 2009 AACE-ADA Consensus Statement, the latter of which advised more conservative blood glucose targets in the ICU. We found no evidence that less hypoglycemia was achieved at the expense of more hyperglycemia.

With the caveats noted above, this website can also continue to serve as a benchmarking tool to assess national trends in the quality of hospital glucose management.