Continuous Glucose Monitoring Accuracy Results Vary between Assessment at Home and Assessment at the Clinical Research Center

Abstract

Background

Continuous glucose monitoring system (CGMS) accuracy is of critical importance both in delivering therapeutic value and as a component of a closed-loop system. This study aims at assessing the differences between accuracy assessments of CGMS at home and at the clinical research center (CRC).

Methods

Twelve patients with type 1 diabetes used the Dexcom® SEVEN® PLUS (DexCom, Inc.) CGMS for 7 days. Patients performed ≥6 finger pricks [self-measurement of blood glucose (SMBG)] per day while at home. Reference blood glucose measurements were taken during a 24 h CRC admission (YSI 2300 STAT Plus™). Continuous glucose monitoring system data were compared with YSI and SMBG values. Outcome measures included mean absolute relative difference (MARD) and Clarke error grid analysis (CEGA).

Results

During CRC admission, the MARD of CGMS vs YSI glucose values was 19.2% (n = 509)—significantly higher than 16.8% at home (n = 611) (p = .004). In the hypoglycemic range, MARD was 23.9% at CRC (n = 26)—not significantly different from 41.6% at home (n = 39) (p = .269). In the hyperglycemic range, CRC MARD at 20.3% (n = 115) was significantly higher than home MARD at 11.2% (n = 118) (p = .001). Clarke error grid analysis showed no significant difference in distribution of data pairs (overall p = .317).

Conclusions

This study illustrates the importance of the setting used when assessing CGMS accuracy. Continuous glucose monitoring system accuracy at home appeared better than at the CRC. This is probably due to the higher sampling rate of reference measurements, feasible only in the CRC. Testing CGMS accuracy in the CRC provides valuable information over and above home testing.

Introduction

With continued development of continuous glucose monitoring systems (CGMSs), the promise of real-time glucose measurements in patients with diabetes is materializing.¹ Still, the optimal assessment of CGMS performance is under discussion.² Accuracy of CGMS is most often assessed by Clarke error grid analysis (CEGA) and determination of the mean absolute relative difference (MARD) between the CGMS reported glucose value and the concurrent reference blood glucose value.³ Currently, data for such analyses are often gathered by conducting clinical trials with CGMS in the clinical research center (CRC); these trials are conducted under standardized conditions and use a rigid sampling schedule.⁴ However, trials in the CRC are costly, in part because they are labor-intensive. Therefore, alternatively, accuracy of CGMSs might be assessed while patients are using them at home. A benefit of the home setting is that it is often perceived as being more real-life. However, in the home setting, questions arise about the validity of gathered data. Some studies have assessed CGMS accuracy while patients were using the device in the home setting, either using SMBG as a reference⁵ or relying on a laboratory method as a reference to determine accuracy and using the home phase for the study of device reliability and usability only.^6,7

The aim of this study was to investigate whether CGMS accuracy as determined at home differs from accuracy as determined at the CRC, thereby investigating the effect of different settings and methodologies of acquiring reference samples when assessing CGMS accuracy.

Methods

The study was conducted in four clinical centers of the AP@home consortium (Amsterdam, The Netherlands; Graz, Austria; Neuss, Germany; and Padua, Italy) and was registered at the ISRCTN trial registry (ISRCTN18000305). The study was designed to collect CGMS data for a period of 1 week to improve the usability of CGMS in a closed-loop system and was carried out in accordance with the Declaration of Helsinki as revised in 2000. Twelve patients with type 1 diabetes mellitus gave informed consent and used the Dexcom® SEVEN® PLUS CGMS device (Dexcom, Inc., San Diego, CA) for a period of 7 days. Main inclusion criteria were an age of ≥18 years,diagnosed with type 1 diabetes for at least 6 months, BMI ≤35 kg/m², and a hemoglobin A1c ≤86 mmol/mol (10%). The sensor of the CGMS was inserted on day 1, and patients received training in the use and calibration of the device. Patients were able to contact study personnel at all times during the study in case of problems or questions. A blood-plasma-calibrated study blood glucose meter (ACCU-CHEK® Aviva Nano, Roche Diagnostics, Basel, Switzerland) was used to perform all SMBG and calibrations both during home and CRC phases. The CGMS was calibrated twice daily according to the manufacturer’s specification. Patients received a diary in which they noted the dates and times of all SMBG values. Patients performed a minimum of 6 SMBGs per day. Whenever possible, SMBG was performed directly before and 90 min after breakfast, lunch, and dinner.

Patients were admitted to the CRC on either day 3, 4, or 5 of CGMS use. During this 24 h admission, patients received standardized meals containing 60 g of carbo-hydrates at breakfast and lunch and 80 g of carbohydrates at dinner. All meals had to be fully ingested within 20 min. Blood was sampled on average every 30 min throughout admission for plasma glucose measurements with a laboratory method (YSI 2300 STAT Plus™, YSI Inc., Yellow Springs, OH).

At the end of the 7 days, CGMS data were downloaded, and YSI (CRC) or blood glucose meter values (at home) were paired with CGMS glucose values at the concomitant times of reference measurements. Outcome measures included mean absolute relative difference (MARD)—bothoverall and separately for hypoglycemic (≤70 mg/dl), euglycemic (70–200 mg/dl), and hyperglycemic (≥200 mg/dl)ranges and CEGA. Mean absolute relative differences and distribution of data pairs in the zones of CEGA were analyzed between CRC and at-home phases using the nonparametric Kruskal-Wallis test.

Results

All patients included in this trial completed the 7-day study period. There were no adverse events reported during this trial. The study included 7 (58%) males and 5 (42%) females, mean age was 42 (range 24–56) years, and duration of diabetes was 21 (range 11–33) years.

Total mean registration time of CGMS was 6.4 (range 5.9–6.9) days. Out of raw data, 509 data pairs pertaining to the CRC phase could be used for analysis versus 611 data pairs pertaining to the home phase. During CRC admission, overall MARD of CGMS-reported glucose values versus reference (YSI) values was 19.2% (n = 509); this was significantly higher than the MARD of 16.8% during the home phase when CGMS-reported glucose values were compared with SMBG values (n = 611) (p = .004). In the hypoglycemic range, MARD was 23.9% during the CRC phase (n = 26), which was not different from a MARD of 41.6% during the home phase (n = 39) (p = .269). Also in the euglycemic range, the MARD of 18.4% during the CRC phase (n = 368) and the MARD of 14.7% during the home phase (n = 454) were not different (p = .197). In the hyperglycemic range, the MARD of 20.3% during the CRC phase (n = 115) was higher than the MARD of 11.2% during the home phase (n = 118) (p = .001). Clarke error grid analysis showed no significant difference in distribution of data pairs per zone [CRC vs home: 67.0% and 71.5% in zone A; 28.7% and 26.4% in zone B; 1.2% and 0.3% in zone C; 2.4% and 1.5% in zone D; and 0.8% and 0.3% in zone E (overall p = .317)(see Figure 1)]. Rate of change according to CGM data was not significantly different between the home [0.063 (range -5.9–9.4) mg/dl/min] and CRC phase [-0.064 (range -23.8–14.8) mg/dl/min (p = .569)]. The MARD of the sensor, calculated using SMBG as reference, decreased over time, dropping from 19.8% on day 1 of use to 18.1% on day 2, 19.4% on day 3, 17.6% on day 4, 16.6% on day 5, 13.7% on day 6, and 13.1% on day 7 of sensor wear. This change was significant (p = .009).

Figure 1.

Clarke Error Grid Analysis of data pairs of CGMS and reference samples. Reference samples used at home were SMBG samples, and reference samples in the CRC were YSI samples. The distribution of data pairs was not significantly different between at-home and CRC assessment (overall p value = .317).

Discussion

During this trial, CGMS accuracy expressed as MARD using SMBG reference measurements assessed at home appeared better than when accuracy was assessed in the CRC using YSI reference measurements. This difference was not significant in all different glycemic regions and in CEGA, most likely because of lack of power. However, the trend was always towards seemingly lower accuracy at the CRC. To our knowledge, the influence of different setting and reference methods has never been studied. Patients and caregivers could be misled by seemingly high CGMS accuracy, and therefore CGMS assessment studies should be designed in such a way that accuracy is assessed comprehensively. This should include enough values in all glycemic regions and a combined assessment at home and at the CRC.

A limitation of all accuracy studies performed in real-life is that the timing of patient-performed SMBG reference measurements is dependent on patient behavior. Not onlyis SMBG often performed inaccurately,⁸ but it is also more likely that samples taken at home will be more aggregated around daytime and extreme events (perceived hyper- and hypoglycemia), whereas in the CRC, samples are usually more evenly distributed throughout the day. It is likely that the rate of change in blood glucose affects the MARD, because sensors that measure glucose in the interstitial fluid experience a lag in glucose changes in the interstitial fluid compared with changes in blood glucose. However, rate of change was not different between the CRC and home phases of this trial, making it unlikely that rate of change could account for the significant differences in MARD. It is likely that the observed difference in CGMS accuracy is mainly due to the higher sampling rate of paired values feasible only in the CRC. A higher sampling rate in the home setting would be practically unattainable due to added burden to study participants.

Continued glucose monitoring system accuracy studies also have to take into account the method of calibration. In our study, CGMS was calibrated using the same blood glucose meter both during the CRC and home phases. This allowed us to compare CRC and home phases without the need to adjust for the confounding effect of using different calibration methods. Other research has shown that the method of calibration accounts for major differences in accuracy in the Dexcom SEVEN CGMS. In one study, MARD decreased from 16.0% when calibrated with SMBG values to 8.5% when calibrated with YSI values.⁹ However, calibration of CGMS with YSI values is impossible outside the CRC, and the increase in sensor accuracy is therefore artificial and of little relevance to the patient in normal daily use of CGMS. We used the YSI as reference in the CRC and SMBG as reference for the home phase. Using different methods of obtaining reference samples could lead to variance in measuring error between study phases. However, factory calibration of the study blood glucose meter involves calibration against YSI values, so this effect should be minimized. Of course, in studies that use many different blood glucose meters, this variance may increase even further, and in all real-life studies, accuracy of the blood glucose meter measurements may be compromised for a variety of reasons.

During this trial, we found that the MARD diminished over time. This increase in accuracy over time has also been shown to exist in the predecessor of the Dexcom SEVEN PLUS, the Dexcom SEVEN system.¹⁰

In studies that aim to assess sensor accuracy, paucity of measurements in the hypoglycemic range is a common problem that could occur both at home and at the CRC. This problem also occurred in our study. Therefore, an intervention is needed, which induces (mild) hypoglycemia, ensuring an adequate number of samples throughout CGMS wear time at euglycemic, hypoglycemic, and hyperglycemic levels.³

Conclusions

In our study, CGMS accuracy was worse when assessed at the CRC using YSI reference measurements than when assessed in the home situation with SMBG reference measurements. Testing CGMS accuracy under standardized conditions in the CRC provides the most optimal and balanced assessment; however, real-life studies that incorporate a home phase are also needed to assess duration of use and to confirm that accuracy of CGMS at home is in the same range as accuracy assessed in the CRC.

Glossary

(CEGA)

Clarke error grid analysis

(CGMS)

continuous glucose monitoring system

(CRC)

clinical research center

(MARD)

mean absolute relative difference

(SMBG)

self-measurement of blood glucose

Disclosures

All authors confirm that they have no relevant duality of interest to declare relevant to this work.