Use and Accuracy of Inpatient CGM During the COVID-19 Pandemic: An Observational Study of General Medicine and ICU Patients

Abstract

Background:

Continuous glucose monitoring (CGM) is widely used in the outpatient setting for people with diabetes and has been limited to investigational use only for the inpatient population. In April 2020, the US FDA exercised enforcement discretion for the temporary use of inpatient CGM during the pandemic, thus hospitals were presented the opportunity to implement this technology.

Methods:

We sought to investigate the accuracy of CGM in hospitalized patients on general care floors and the intensive care unit (ICU) in attempts to decrease healthcare professional exposure to COVID-19 and ultimately improve glycemic management of patients affected by COVID-19. Point of care (POC) and laboratory (Lab) glucose values were matched with simultaneous CGM glucose values and measures of accuracy were performed to evaluate the safety and usability of CGM in this population. Our data are presented drawing a distinction between POC and Lab as reference glucose sources.

Results:

In 808 paired samples obtained from 28 patients (10 ICU, 18 general floor), overall mean absolute relative difference (MARD) for all patients using either POC or Lab as reference was 13.2%. When using POC as the reference glucose MARD was 13.9% and using Lab glucose as reference 10.9%. Using both POC and Lab reference glucose pairs the overall MARD for critical care patients was 12.1% and for general floor patients 14%.

Conclusion:

We determined, with proper protocols and safeguards in place, use of CGM in the hospitalized patient is a reasonable alternative to standard of care to achieve the goal of reducing healthcare professional exposure. Further study is necessary to validate safety, accuracy, and efficacy of this technology. Investigation and analysis are necessary for the development of protocols to utilize CGM trend arrows, alerts, and alarms.

Introduction

The Coronavirus Disease 2019 (COVID-19 caused by SARS-CoV-2 infection) pandemic affected our institution significantly, leading to challenges in managing hospitalized patients with diabetes. In addition to limited protective personal equipment (PPE) and repetitive exposure of nursing staff to patients with COVID-19, we were faced with managing the hyperglycemia found in these patients due to underlying proinflammatory metabolic state. ¹

Blood glucose (BG) measurements using point of care (POC) and whole blood lab analysis have been the only means of assessing glycemic outcomes in the hospital setting. The evolution of self-monitoring of BG was further revolutionized with the introduction of continuous glucose monitoring (CGM) and an abundance of glucose data. Prior to the COVID-19 pandemic, CGM was not approved for use in the inpatient setting. In April 2020, the United States Food and Drug Administration expanded the availability and capability of noninvasive patient monitoring devices during the COVID-19 pandemic by exercising enforcement discretion so that CGM can be used temporarily in the inpatient setting. ²

CGM technology is well suited to address many of the challenges of this unique circumstance by reducing nursing exposure, preserving PPE, and providing additional glucose data. ³ As such, we were prompted to assess the safety and accuracy of CGM in the inpatient setting.

CGM offers a promising alternative to periodic BG measurements with advantages that include automatic measurements at 5-minute intervals, automatic transmission of estimated glucose values to physically distant display devices, and programmable alerts to warn of existing or impending dysglycemia. Hypoglycemia and hyperglycemia are associated with longer hospitalizations, increased risk of hospital-acquired infection, morbidity, and mortality.^4,5

Prior to the COVID-19 pandemic, CGM has been evaluated in hospital settings, where a CGM system incorporated into a glucose telemetry system has been shown to reduce the risk of inpatient hypoglycemia ⁶ and hyperglycemia. ⁷ Since the beginning of the COVID-19 pandemic, CGM has been investigated and implemented in many institutions.^3,8-10

We sought to investigate the use and accuracy and of these devices in the inpatient setting. Our goal was to collect glucose data from the CGM device and compare them to POC and/or lab-drawn blood samples in order to validate that CGM is an acceptable alternative to the current standard of care of POC testing.

Methods

Our team at Lahey Hospital and Medical Center obtained CGM devices for inpatient use during the COVID-19 pandemic. Our data collection started in the spring of 2020 and then continued from December 2020 to February 2021. We collected data on adult hospitalized patients infected with COVID-19 infection (n = 28) who were located on medical general care floors (n = 18) and in the medical intensive care unit (ICU; n = 10). Two patients initially on the general floor were transferred to the ICU and we divided data by location. We collected sensor glucose data from the CGM and compared them with the POC glucose using Accu-chek Inform II System (Roche Diagnostics Corporation, Indianapolis, IN, USA) or to whole blood analyzed in the lab using Beckman DxC 800 Analyzer (Brea, California, USA).

Patients were considered for CGM if positive for COVID-19 and ordered for POC BG monitoring. Patients were excluded if they had gross anasarca or if imminent hospital discharge, transition to comfort measures only or death was predicted by medical staff within 48 hours. Patients were assessed by the CGM team for appropriateness of therapy.

The majority of patients were receiving or recently received steroids (dexamethasone 6-10 mg daily). This dose was given at various times during the day. The patients in the critical care units were all mechanically ventilated and treated with a variety of therapies including continuous veno-venous hemofiltration (CVVH), vasopressors, paralytics, steroids, and enteral feedings. Capturing the specifics of which therapies patients were receiving and for how long was out of scope for this study. The majority of the patients followed were male (20 of the 30 patients). All of the patients had documented type 2 diabetes or steroid-induced hyperglycemia except 1 patient with latent autoimmune diabetes in adults.

Instead of placing the sensor on the abdomen which is the FDA-approved location for the brand of CGM used (Dexcom G6, Dexcom, San Diego, CA, USA), the sensor was placed in the subcutaneous tissue of the upper arm to allow for easier access for staff and prone positioning which is frequently used to treat the hypoxia associated with COVID-19 infection. ¹¹ Arm placement of CGM sensors has been shown to have similar accuracy when compared to abdominal placement using an earlier model of the same device (G4 Platinum). ¹² An Android device was hung outside each patient’s room within 20 feet of the patient to allow for transmission of data to the device while reducing staff exposure.

The endocrinology team placed CGM sensors and was responsible for Android device and alarm setup. The medical team including nurses, physicians, nurse practitioners and physician assistants (NP/Pas) were able to view the data on the phones hanging outside the patient room or at the phones docked at the nurses’ station (loaded with the Follow application). The Follow app allowed us to enable hypoglycemia and hyperglycemia alarms to be easily audible to our nursing staff.

After reviewing pilot data from spring 2020, we devised a protocol with our nursing colleagues to meet the nursing workflow and safety demands of the pandemic. In December 2020, CGMs were applied and standard POC glucose checks were continued for the first 24 hours. After 24 hours, if POC and CGM glucose values were within 35 mg/dL, the CGM was deemed acceptable for glucose monitoring. Daily am Lab glucose and pm POC glucose was performed. If the difference between reference and CGM value was >35 mg/dL, CGM accuracy was reviewed by the CGM team. A difference of 35 mg/dL was chosen as the nursing staff felt a set reference was more feasible than calculating a 20% difference with the current nursing demands. Any sensor glucose <80 mg/dL or >400 mg/dL was confirmed with a POC glucose.

In a review of the literature addressing accuracy measures of CGM, we calculated the following measures of accuracy: mean absolute relative difference (MARD), the Bias, the coefficient of variation (CV; a measure of glucose variability), and the lower and upper 95% limit of agreement. MARD is the most commonly used measure to assess CGM performance and represents the average of the absolute error between CGM glucose and matched reference glucose values. ¹³ Bias incorporates the directionality of the difference (positive or negative) compared to the reference sample ¹⁴ whereas CV is the standard deviation divided by the mean and is used to compare the overall precision of the two reference glucose data sets. ¹⁵

In addition we constructed Bland-Altman Plots, computed surveillance error grid (SEG) analysis using SEG software ¹⁶ and graphed our data using the International Standardization Organization (ISO) 15197:2013 standards. ¹⁷ To compare the accuracy of pairs of POC vs CGM glucose data and pairs of Lab vs CGM data, we performed a 2-sample t-test comparing the absolute relative differences between the reference glucose values.

Results

We collected a total of 808 paired samples. The CGM glucose was recorded within 5 minutes of the reference glucose. Each paired data set compares either POC vs CGM or Lab vs CGM. POC measurements accounted for 607 of the paired samples and 201 were obtained from Lab glucose measurements. Overall MARD for all patients using either POC or Lab as reference was 13.2%. When utilizing POC as the reference, glucose MARD was 13.9% and Lab glucose as reference was 10.9% (Tables 1 and 2).

Table 1.

MARD for All Patients by Reference Glucose.

	All patients CGM vs POC and Lab	All patients CGM vs Lab only	All patients CGM vs POC only
MARD all days	13.20%	10.90%	13.90%
MARD excluding day 1	12.36%	10.19%	13.07%

CGM, continuous glucose monitoring; MARD, mean absolute relative difference; POC, point of care.

Table 2.

Accuracy Data by Patient Location and Reference Glucose.

	Floor patients			Critical care patients			All patients
	CGM vs POC and Lab	CGM vs Lab only	CGM vs POC only	CGM vs POC and Lab	CGM vs Lab only	CGM vs POC only	All days CGM vs POC and Lab	All days CGM vs Lab only	All days CGM vs POC only
Total POC or Lab vs CGM	461	117	344	347	84	263	808	201	607
Bias	2.10%	4.70%	1.00%	2.70%	0.40%	3.5	2.40%	2.90%	2.20%
MARD	14.00%	11.30%	14.80%	12.10%	10.40%	12.70%	13.20%	10.90%	13.90%
CV	22.30%	16.30%	24.00%	16.30%	13.10%	17.20%	20.00%	15.10%	21.30%
Lower 95% limit of agreement	−41.70%	−27.20%	−46.00%	−29.30%	−25.20%	−30.20%	−36.80%	−26.70%	−39.60%
Upper 95% limit of agreement	45.80%	36.6	48.10%	34.70%	26.00%	37.20%	41.50%	32.60%	43.90%
Mean glucose of reference (POC, Lab) (mg/dL)	171.60	149.90	18 0.30	169.3	162.71	168.11	170.6	155.26	175.3

CGM, continuous glucose monitoring; CV, coefficient of variation; MARD, mean absolute relative difference; POC, point of care.

We also compared MARD excluding the first day of sensor wear to MARD of all days of sensor wear (Table 1) because the first day of sensor wear has been demonstrated to be less accurate. ¹⁸ We designed our protocol with this in mind by continuing regular POC monitoring for the first 24 hours of sensor wear.

We further divided data comparing critical care patients vs general floor patients (Table 2). Of note, the overall MARD for critical care patients was 12.1% (10.7% in the Lab glucose reference pairs). For general floor patients, using both POC and Lab reference glucose pairs, the overall MARD was 14%, with 14.8% for the POC reference group and 11.3% with Lab as reference glucose. By comparison, the MARD for adults using the G6 CGM in an outpatient setting was reported at 9.8%. ¹⁸

Figure 1 graphs the percent of measurements of the absolute relative difference (%) of both POC as reference glucose and Lab as reference glucose. Given the unequal sample sizes (Lab n = 201, POC n = 607), we chose to display as percent of measurement to equalize the comparison.

Figure 1.

% ARDs in range.

Height of bar represents percentage of samples that fall in each segment of % of absolute relative difference (reference glucose – CGM glucose/reference glucose). ARD, absolute relative difference; CGM, Continuous glucose monitoring.

Figure 2 is a graph of our data using ISO 15197:2013 standards with limits drawn at 15 mg/dL when reference BG ≤100 mg/dL and ±15% when reference >100 mg/dL. ISO 15197:2013 standards expect 95% of results to fall within these limits. (For comparison, the FDA requirements for hospital BG POC include 95% of readings must fall within 12% of comparator for BG >75 mg/dL and 12 mg/dL for BG <75 mg/dL; 98% of readings must fall within 15% of comparator for BG >75 mg/dL and 15 mg/dL for BG <75 mg/dL.) ¹⁹ When divided by reference glucose type, 93% of Lab glucose pairs fell within 15%/15 mg/dL ISO analytical accuracy standards vs 87.6% of POC pairs. For general floor patients, 88.5% of all measurements met ISO criteria vs 89.3% of critical care patients.

Figure 2.

ISO 15197:2013 limits chart. ISO, International Standardization Organization.

Bland-Altman analysis compares the difference of the measured glucose minus the reference value with the recommendation that 95% of the data points should lie within ±2 standard deviations of the mean difference ²⁰ (see Figure 3). These graphical representations also reflect the relatively low bias we found.

Figure 3.

Bland-Altman relative difference CGM vs POC% and CGM vs Lab%. CGM, continuous glucose monitoring; POC, point of care.

There were a limited number of glucose values in the hypoglycemic range (<70 mg/dL). Of the 808 measurements, 15 values were 70 mg/dL or less. Given the clinical concern with the accuracy of hypoglycemia detection and the elevated MARD in hypoglycemic ranges, we included matched pairs for all glucose readings less than 70 mg/dL (see Supplemental Table S1).

The SEG error grid (Figures 4 and 5) for clinical accuracy with delineated risk zones (Table 3) provides a visual as well as numerical representation of our data. When compared to ISO 15197:2013 standards, it is recommended that >97% of data points fall in the no-risk zone of the SEG to be classified as accurate, according to similar requirements as current FDA standards.^21,22 The percentage of matched pairs in the no-risk zone using lab values as the reference glucose is 86.1% and 82.5% using POC as the reference glucose.

Figure 4.

All patients. CGM, continuous glucose monitoring; POC, point of care.

Figure 5.

All patients. CGM, continuous glucose monitoring.

Table 3.

Surveillance Error Grid—Percent of Pairs Within Risk Zones.

(Percent of pairs within each risk zone)	Floor patients			Critical care patients			All patients
SEG risk	CGM vs POC + Lab	CGM vs Lab	CGM vs POC	CGM vs POC + Lab	CGM vs Lab	CGM vs POC	CGM vs POC + Lab	CGM vs Lab	CGM vs POC
0 None	82.90%	84.60%	82.50%	84.10%	88.10%	82.90%	83.40%	86.10%	82.50%
1 Slight, lower	13.60%	12.00%	14.00%	13.30%	10.70%	14.10%	13.50%	11.40%	14.20%
2 Slight, higher	2.20%	2.60%	2%	2.30%	1.20%	2.70%	2.20%	2.00%	4.00%
3 Moderate, lower	1.10%	0.90%	1.20%	0.30%	0	0.40%	0.60%	0.50%	0.70%
4 Moderate higher	0.20%	0	0.30%	0.00%	0	0.00%	0.20%	0	0.30%
5 Severe, lower	0.00%	0	0.00%	0.00%	0	0.00%	0.00%	0	0.00%
6 Severe, upper	0.00%	0	0.00%	0.00%	0	0.00%	0.00%	0	0.00%
7 Extreme	0.00%	0	0.00%	0.00%	0	0.00%	0.00%	0	0.00%

CGM, continuous glucose monitoring; POC, point of care; SEG, surveillance error grid.

It became evident that the accuracy measures (MARD, 95% agreement, SEG) when comparing CGM to Lab tended toward better accuracy values than when compared to POC glucose as reference. We performed a 2-sample t-test comparing the absolute relative differences between POC glucose as the reference (M = .139, SD = .162) and Lab glucose as the reference (m = .109, SD = .108), t(808) = 2.39, P = .0169.

Discussion

This observational study demonstrated the use of CGM in the inpatient setting during the COVID-19 pandemic. The CGM vs whole blood Lab glucose MARD calculated in this smaller data set was higher than the MARD of 9% reported in the outpatient setting using whole blood Lab as reference glucose. ¹⁸ The MARD demonstrated more agreement in the critical care setting when using Lab glucose as reference against CGM and less agreement in the general floor setting when using POC glucose as reference against CGM.

Overall the MARD calculated from Lab glucose as a reference was significantly lower than the MARD using POC as a reference glucose. As the Lab glucose is the gold standard of reference and in-hospital POC glucose meters may have up to ±15% deviation from the Lab glucose per the ISO standards, the POC device may be expected to have greater noise than the lab instrument, resulting in greater dispersion of ARD as seen in Figure 1.

This was also demonstrated in Figure 1 as the percent of relative difference values (absolute value of reference glucose subtracted from CGM glucose and divided by reference glucose) using Lab glucose as reference were more concentrated around lower percentages. The relative differences using POC as reference had a wider dispersion compared to Lab as reference values. This finding is worth noting for future protocols for confirmatory testing.

The percentage of matched pairs in the no-risk zone of the SEG error grid did not meet ISO 15197:2013 standards. However, it is worth nothing that in a study of 90% of the commercially available BG monitors approved by the US Food and Drug Administration, only 6 of the 18 meters tested met these accuracy standards. ²³

Our results are relatively similar when compared to other studies of inpatient CGM accuracy. In a small pilot study of 10 patients using G6 undergoing surgery before the COVID-19 pandemic, the MARD was reported to be 9.4%, correlation coefficient 0.76, mean bias −0.37 mg/dL, and 95% limits of agreement −42.4 mg/dL and 41.7 mg/dL with 89% of paired glucose values within the no-risk SEG error grid. ²⁴

In another trial, 11 critically ill patients using CGM, either G6 or Medtronic Guardian Connect (Medtronic, Minneapolis, MN, USA), sensors were analyzed against POC glucose monitoring. The MARD for the Guardian Connect was reported at 13.1% with 100% of readings in zones A and B on Clarke EGA and mean bias of −17.76 mg/dL. The MARD for G6 was 11.1% with 98% of readings in zones A and B of the Bland-Altman plots and a mean bias of −1.94 mg/dL with reported “wide” 95% limits of agreement. ¹⁰

A pre-COVID-19 prospective study comparing Libre Pro CGM (Abbott Care Diabetes, Alameda, CA, USA) to the hospital POC BG meter at an academic medical center in the United States yielded a MARD of 14.8%. The cohort consisted of 134 non-critically ill inpatients with type 2 diabetes on basal-bolus insulin therapy. ²⁵

An early COVID-19 pandemic study was conducted on 9 general ward patients with COVID 19 using G6 and POC as reference glucose. The MARD was calculated at 9.77% with 84% of values within Clarke zone A and 100% of values within zones A and B, correlation of coefficient 0.927, and mean bias 2.45 mg/dL. ⁹

Given the known lag time between capillary glucose (measured by POC) and interstitial glucose (measured by CGM), a difference is expected between CGM and reference glucose. ²⁶ The lag between interstitial glucose and capillary glucose has been assumed to be 5-10 minutes; however, this can vary, particularly in the setting of rapidly rising or falling BG ²⁷ such as seen in patients receiving IV steroids or bolus enteral nutrition. Despite the expected limitation of lag time in this population, the correlation between CGM and reference instrument remained clinically acceptable for glucose monitoring as long as periodic safety comparisons were made per institutional protocol.

These raw data do not include the presence of trend arrows, alerts, or alarms and therefore do not account for action taken by nursing staff in response to these alerts and alarms to prevent dysglycemia. Protocol development and staff education must take into account the given physiologic lag time between BG and interstitial glucose ²⁶ as well as sensor instrument lag and data smoothing lag and need to be proactive for hypoglycemia prevention. Our data also do not account for hypoglycemia that may have occurred between POC and Lab glucose measurements for which CGM has been shown to prevent. ²⁸ Further studies with larger sample sizes and trials designed to evaluate the role of CGM in hypoglycemia frequency and prevention strategies in the inpatient setting are necessary.

One of the limitations of our study was the lack of simultaneous glucose levels obtained from the whole blood analysis and capillary blood using the POC system and then compared against the CGM glucose. We felt that it was not practical to ask the nursing staff to perform additional tasks given the complexities of the pandemic. No serious adverse events were reported during the study period.

We reviewed possible influences on the differences between POC and Lab accuracy and ICU vs general floor patients’ glucose values. Lab measurements were more likely to be collected overnight or in the morning which were less likely affected by the patient’s food consumption, because lag is more of a problem in the fed state than the fasted state. The majority of critical care patients were on continuous tube feeding or NPO. Less prandial fluctuation could result in less rapid change in glucose at the time of collection hence a lesser degree of error between reference and CGM glucose in the critical care population.

Steroid use was frequent in our patient population; however, dosing occurred at varying times of the day. We did not identify any obvious influence of CVVH or vasopressor use. The hyperglycemic effect of steroids was quite noticeable on CGM and may have resulted in more rapidly changing glucose levels in this patient population potentially influencing accuracy data because of interstitial lag. It is important to note that collection technique and interfering substances could also influence the difference in MARD between CGM and the Lab or POC values. ²⁹ This particular CGM reports hydroxyurea and acetaminophen as interfering substances. ²⁹ In our protocol, CGM is not used on patients receiving hydroxyurea. Our prescribing practices for acetaminophen are within the manufacturer’s recommended dosage for safe usage of 1 g every 6 hours. ²⁹

We did not directly observe PPE use and nursing exposure. Based on our current protocol, less patient interaction is required for POC glucose monitoring for example, twice per day vs 5 times per day under usual care. The anecdotal feedback from our nursing colleagues was overwhelmingly positive. They also reported improved confidence in insulin dosing with use of real-time glucose data and hypoglycemia alarms.

Conclusion

Our results indicate CGM systems with remote monitoring capability can help manage patients with COVID-19 infection in general care units and select critical care patients. The calculated MARD varied between 10.9% and 14%, with the lowest MARD calculated when using Lab values as reference glucose. CGM systems offer hospital staff the opportunity to monitor patients remotely and be alerted to abnormal values, providing the opportunity to take preventative action.

Future studies are most certainly required to investigate the accuracy and safety of inpatient CGM in a variety of patient populations, in particular with simultaneously collected CGM, POC, and Lab glucose values. The development of standardized protocols for use of inpatient CGM is also necessary. Expanded protocols that utilize trend arrows and predictive alerts could be beneficial in improving glycemic outcomes, especially hypoglycemia prevention. The integration of CGM measurement into the electronic health record and labeled as sensor glucose (CGM) is important for long-term feasibility, efficient use of data, and safety. Finally, ongoing validation to ensure that use of this technology results in better outcomes for our hospitalized patients will be required along with cost-benefit analysis.