Abstract
Background:
To be able to compare continuous glucose monitoring (CGM) systems, they have to be worn in parallel by the same subjects. This study evaluated the performance and usability of three different CGM systems in direct comparison.
Method:
In this open, prospective study at two sites, 54 patients with diabetes wore three CGM systems each (Dexcom G5™ Mobile CGM system [DG5], Guardian™ Connect system [GC], and a Roche CGM system [RCGM]) in parallel for 6 or 7 days in a mixed inpatient and outpatient setting. Capillary comparison measurements were performed using a self-monitoring of blood glucose (SMBG) system. During study site visits, glucose excursions were induced. Performance of the systems was evaluated by calculating mean absolute relative differences (MARD, calculated as absolute differences for glucose concentrations <100 mg/dL and as relative differences for glucose concentrations ≥100 mg/dL), and mean relative differences (MRD, bias) between CGM and SMBG results. In addition, usability of the systems was assessed.
Results:
Overall MARD was 10.1 ± 2.1 for DG5, 11.5 ± 4.2 for GC, and 11.9 ± 5.6 for RCGM. Performance improved in all systems after the first day of use. All systems showed >99% of values within zones A and B of the consensus error grid. Overall, all CGM systems showed a small negative bias compared to SMBG. Usability of the systems differed regarding patch adhesion rate, failure rate, and patient rating. Most patients preferred GC, but in general all systems were rated positively.
Conclusion:
All three CGM systems showed similar overall accuracy in this direct comparison, but small differences were observed with regard to specific glucose ranges and usability aspects.
Keywords: continuous glucose monitoring, CGM system, MARD, performance, comparison study, usability
Glucose monitoring is essential for all people with diabetes, especially when insulin-treated. To manage their therapy and to avoid hypoglycemia or hyperglycemia, patients generally perform self-monitoring of blood glucose (SMBG) measurements multiple times per day using capillary finger sticks. Achieving a complete picture including glucose dynamics would require very frequent measurements, also during nights. This is not feasible for patients. Instead, continuous glucose monitoring (CGM) systems have become an option for many patients in recent years. Current needle-type real-time CGM systems capture continuous subcutaneous tissue glucose values for up to seven days, when calibrated with SMBG measurements twice a day. Such systems therefore provide much more information to evaluate glycemic control compared to occasional measurements. Improvements in glycemic control using CGM have been shown for various patient groups.1,2 While the performance of early CGM systems lay far below that of most SMBG systems, accuracy of current CGM generations has improved considerably.3 Since some CGM systems are already intended for nonadjunctive use,4 that is, as a replacement for SMBG measurements, and may therefore be used for therapeutic decisions, accuracy and reliability of CGM values have also become important in terms of patient safety.5
The mean absolute relative difference (MARD) is a parameter often used to describe CGM system accuracy. However, as MARD is dependent on the exact test setting and as there are no mandatory specifications for testing procedures, MARD values given in the literature for different systems may not be directly comparable.6 In addition, as the calculation of relative differences at lower glucose concentrations may be unsuitable for accuracy assessments,7 MARD is often reported as a combination of absolute and relative differences, but the specific cutoff values vary.8-11 The best way to compare different CGM systems is therefore to perform a study in which the systems are worn in parallel in the same subject. The Clinical and Laboratory Standards Institute guideline POCT05-A12 gives recommendations for study procedures. These were applied to the presented study, in which three CGM systems were tested in parallel.
Methods
This open, prospective study was performed according to Good Clinical Practice and the declaration of Helsinki (ICH-GCP) at two investigational sites, one in Austria (Privatklinik Wehrle-Diakonissen, Salzburg) and one in Germany (Institut für Diabetes-Technologie, Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm). The protocol was approved by the responsible ethics committees and all participants gave informed consent prior to the study procedures.
The primary objective was to determine the MARD of three CGM systems. Secondary objectives were the evaluation of further performance measures and usability aspects of the systems.
CGM Devices
Three CE-marked CGM systems were used in this study: Dexcom G5™ Mobile CGM system (Dexcom Inc, San Diego, CA, USA) (DG5), Guardian™ Connect system (Medtronic MiniMed Inc, Northridge, CA, USA) (GC), and a Roche CGM system (Roche Diabetes Care GmbH, Mannheim, Germany) (RCGM) (see Table 1). The systems were used according to the respective manufacturer’s instructions, but not as a replacement for SMBG. The intended wearing time was 7 days for DG5 and RCGM, and 6 days for GC. Thresholds for warnings about low and high glucose levels were set to 70 mg/dL and 280 mg/dL, respectively, in all devices. Device data were downloaded using the respective associated software.
Table 1.
Characteristics of the Three Continuous Glucose Monitoring (CGM) Systems as Used in the Study.
| Name | Manufacturer | Sensor life time | Calibration pattern | Measuring range | Data display | Sensor lot (expiry date) |
|---|---|---|---|---|---|---|
| Dexcom G5 Mobile CGM system (DG5) | Dexcom Inc, San Diego, CA, USA | 7 days | 2 h after sensor insertion; then every 12 ha | 40-400 mg/dL | Dexcom G5 Mobile receiverb | 5222281 (2017-12-07) |
| Guardian Connect System (GC) | Medtronic MiniMed Inc, Northridge, CA, USA | 6 days | 2 h after sensor insertion, 5 h after sensor insertion; then every 12 ha | 40-400 mg/dL | Smart device (iPod Touch) | D107P (2017-10-07), D177P (2017-10-14) |
| Roche CGM Systemc (RCGM) | Roche Diabetes Care GmbH, Mannheim Germany | 7 days | 2 h after sensor insertion, within 5 h after sensor insertion; then after every 12 hd | 40-400 mg/dL | Smartphone (HTC One M7) | 30000430 (2017-12-31) |
And if requested by system.
Smart device possible according to manual.
Not available on the market.
No later than every 14 h according to manual.
Comparison Measurements
The Accu-Chek® Guide SMBG system (Roche Diabetes Care GmbH, Mannheim, Germany) was used to calibrate the CGM systems and for capillary comparison measurements.
This system’s accuracy is well-defined and fulfills the ISO 15197:2013 criteria.7 Control measurements were performed with every test strip vial. During study procedures, accuracy was verified by comparison with capillary hexokinase-based laboratory measurements (Cobas 6000 c501, Roche Diabetes Care GmbH, Mannheim, Germany).
Additional venous comparison measurements using a hexokinase method calibrated with higher order reference material were performed during dynamic phases (see below). Venous blood samples were drawn, deproteinized, and then centrally measured at Roche, Mannheim, Germany.
Participants
Insulin treated people with type 1 or type 2 diabetes mellitus, diagnosed at least 12 months prior to the study and without further risk factors for cardiovascular disease were eligible for this study. All participants followed their current therapy during the study, except for dynamic phases where insulin doses were determined by a study physician. All treatment decisions were based on SMBG values.
Study Procedures
The study was performed in a mixed inpatient and outpatient setting for 7 days (distributed over 8 study days) per participant. On study day 1, a sensor of each CGM system was applied on the abdomen of the participants in randomized positions after disinfection of the application sites. The study period included 4 home use days representing the participants’ daily routine and 2 days with study site visits. Days with study site visits were randomized among the participants.
On home use days, participants were instructed to perform SMBG measurements before and 2 hours after each meal, before insulin administration, before bedtime, and for CGM calibration. The systems were calibrated at the same time entering the same SMBG value in all three devices. Approximately 9-14 SMBG measurements, including those for calibration, were to be obtained in order to have at least 6 paired data points per day available for performance evaluation.
At study site visits, dynamic phases were induced in participants having a high glycemic index breakfast and an increased and delayed insulin bolus. Participants performed SMBG measurements every 15 minutes during the dynamic phases (approx. 5.5 hours) and every hour thereafter until the evening. In addition, venous blood samples for analysis with the hexokinase method were taken every 15 minutes during the dynamic phases.
All sensors were removed after their regular wearing period, that is, on study day 7 for GC and on study day 8 for DG5 and RCGM. Insertion sites and patch adhesion were evaluated on a daily basis. If a low or high glucose warning occurred, it was recorded and an SMBG measurement was to be performed for confirmation. All system failures and adverse events were documented.
After all sensors had been removed, participants completed a questionnaire about handling and usability of the devices.
Statistical Evaluation
The primary variable was the MARD across all experiments (all successfully calibrated devices) based on paired points of CGM values and SMBG values in the range of 40 mg/dL to 400 mg/dL (specified measurement range of all three systems). Except for the evaluation of dynamic phases of provoked glucose rise and fall during glucose excursions with SMBG values every 15 minutes, only SMBG values taken at intervals of at least 50 minutes were included in the analysis. Calibration measurements were excluded. MARD was calculated for all measurements of an experiment (ie, one participant), excluding days with less than 6 paired data points, and then averaged across all experiments. Aggregated MARD was calculated using all paired points from all experiments. In addition, for each paired point, the mean relative difference (MRD, including the positive or negative sign) between the CGM value and the SMBG value was calculated. For both MARD and MRD, absolute differences between CGM and SMBG glucose measurement results were calculated at glucose concentrations <100 mg/dL and relative differences were calculated at glucose concentrations ≥100 mg/dL. The cutoff of 100 mg/dL was chosen because it is also specified in ISO 15197:2013 for the evaluation of the performance of SMBG devices.7 Because of the mixed character of these variables, with results in different units (mg/dL or %), MARD and MRD results are given in this article without a unit.
Consensus error grid (CEG) analyses were performed to assess deviations between CGM and SMBG values from a clinical perspective.13
Usability was evaluated based on detection rates and warning rates for low and high glucose values, patch adhesion, sensor failure rates, and a user questionnaire. The evaluation of detection rates for low and high glucose values was based on thresholds of 70 mg/dL and 280 mg/dL, respectively. A low glucose value was considered detected if at least one CGM value was below or equal to the low threshold within ±15 minutes of the SMBG value. The low warning rate was calculated based on how often a low glucose warning of the CGM system could be confirmed by an SMBG value below or equal to the low threshold within ±15 minutes of the warning. Correspondingly, high glucose detection and warning rates were calculated based on glucose values above or equal to the high threshold.
The number and nature of adverse events and device deficiencies were evaluated for each system.
All analyses were performed using the software package SAS®, version 9.3 (SAS Institute Inc, Cary, NC, USA). No significance testing was performed.
Results
Study Participants
The study was performed between May and August 2017. A total of 71 participants were screened at the two study sites, 17 of whom did not meet the eligibility criteria. All 54 participants that started the experimental phase completed the study. The participants were 45 ± 13 (mean ± standard deviation) years of age, 52% female, with a BMI at screening of 26.2 ± 3.8 kg/m2. All participants had type 1 diabetes. HbA1c at screening was 7.0 ± 0.7%. Diabetes duration of the participants was 23 ± 13 years. Most participants (85%) were already experienced with CGM.
Performance
For RCGM, 96% of successfully calibrated systems completed the experiment, as compared with 100% for DG5 and 89% for GC.
Overall MARD across all experiments was 10.1 ± 2.1 for DG5, 11.5 ± 4.2 for GC, and 11.9 ± 5.6 for RCGM (Table 2); aggregated MARD results were similar. All systems showed the highest MARD on day 1 and an improved performance on the following days (Table 2). RCGM showed its lowest MARD in the hypoglycemic range, whereas DG5 and GC performed best in the hyperglycemic range. The same was true for MARD during dynamic phases. Compared to venous comparison samples, overall MARD was higher for DG5 and GC, but not for RCGM.
Table 2.
Mean Absolute Relative Difference (MARD) With Standard Deviation Per Continuous Glucose Monitoring (CGM) System.
| DG5 | GC | RGCM | ||||
|---|---|---|---|---|---|---|
| Na,b | Mean ± SD | Na,b | Mean ± SD | Na,b | Mean ± SD | |
| Compared to SMBG routine measurements | ||||||
| Overall | 53 | 10.1 ± 2.1 | 52 | 11.5 ± 4.2 | 54 | 11.9 ± 5.6 |
| Day 1 | 40 | 12.9 ± 4.5 | 38 | 16.4 ± 7.4 | 41 | 15.3 ± 10.5 |
| Day 2 | 48 | 10.5 ± 3.8 | 48 | 10.6 ± 6.2 | 49 | 10.1 ± 3.8 |
| Day 3 | 50 | 9.7 ± 3.3 | 49 | 9.6 ± 4.1 | 50 | 13.7 ± 5.6 |
| Day 4 | 47 | 9.5 ± 3.4 | 44 | 9.5 ± 3.9 | 43 | 9.3 ± 4.3 |
| Day 5 | 47 | 9.0 ± 3.1 | 45 | 10.2 ± 4.0 | 47 | 8.8 ± 5.6 |
| Day 6 | 47 | 9.0 ± 3.0 | 45 | 10.9 ± 6.6 | 44 | 9.4 ± 6.9 |
| Day 7 | 46 | 9.8 ± 5.8 | − | − | 40 | 10.3 ± 5.6 |
| Aggregated MARD | ||||||
| Overall | 3537 | 10.1 ± 9.4 | 3021 | 11.2 ± 11.1 | 3452 | 11.3 ± 12.6 |
| Hypoglycemiac | 168 | 10.1 ± 8.6 | 145 | 12.1 ± 10.4 | 164 | 8.4 ± 8.4 |
| Euglycemiad | 2339 | 10.4 ± 9.8 | 1983 | 11.7 ± 11.6 | 2271 | 11.0 ± 13.3 |
| Hyperglycemiae | 1030 | 9.5 ± 8.4 | 893 | 9.9 ± 9.7 | 1017 | 12.5 ± 11.4 |
| Compared to SMBG during dynamic phases | ||||||
| Aggregated MARD | ||||||
| Overall | 2389 | 10.6 ± 9.5 | 2236 | 11.6 ± 11.6 | 2331 | 10.7 ± 9.2 |
| Hypoglycemiac | 140 | 11.0 ± 7.9 | 130 | 13.0 ± 13.8 | 128 | 7.9 ± 7.7 |
| Euglycemiad | 1369 | 11.0 ± 10.1 | 1291 | 12.4 ± 12.6 | 1337 | 9.6 ± 8.7 |
| Hyperglycemiae | 880 | 9.9 ± 8.5 | 815 | 10.2 ± 9.2 | 866 | 12.6 ± 9.8 |
| Compared to venous hexokinase measurements during dynamic phases | ||||||
| Aggregated MARD | ||||||
| Overall | 2212 | 12.1 ± 11.6 | 2080 | 13.5 ± 13.6 | 2168 | 10.8 ± 9.8 |
| Hypoglycemiac | 174 | 15.0 ± 13.5 | 167 | 16.8 ± 16.2 | 171 | 10.7 ± 10.5 |
| Euglycemiad | 1233 | 12.7 ± 12.2 | 1160 | 14.5 ± 15.0 | 1202 | 10.2 ± 9.8 |
| Hyperglycemiae | 805 | 10.5 ± 9.7 | 753 | 11.3 ± 10.0 | 795 | 11.8 ± 9.7 |
MARD is shown for the whole experiment and for individual days. Aggregated MARD is shown for different glycemic ranges and for dynamic phases compared to self-monitoring of blood glucose (SMBG) and to venous samples. Because of the mixed character of MARD with different units (mg/dL or %), numbers are given without a unit.
Bold values show overall results.
Number of experiments (participants) for MARD across all experiments (numbers differ between days because days with less than 6 paired data points were not included).
Number of paired data points for aggregated MARD.
SMBG ≤70 mg/dL.
70 mg/dL < SMBG ≤180 mg/dL.
SMBG >180 mg/dL.
All CGM systems had a small negative bias (MRD) compared to SMBG (Figure 1). In total, the MRD across all experiments ranged from −1.1 ± 6.4 to −4.9 ± 5.6 (Table 3). In the dynamic phases of the experiment, aggregated MRD ranged from 0.5 ± 16.4 to −5.6 ± 12.9 compared to SMBG, with RCGM showing the highest bias. When comparing to venous measurements, however, RCGM showed the lowest bias.
Figure 1.
Box plots for mean relative differences (MRD) across all experiments per CGM day compared to capillary SMBG measurements. Displayed are mean (diamonds), median (horizontal line within boxes), 25th and 75th percentiles (lower and upper border of the boxes), and minimum and maximum values (whiskers). The red horizontal line shows the total bias of the respective system over the whole study.
Table 3.
Mean Relative Difference (MRD) With Standard Deviation per Continuous Glucose Monitoring (CGM) System Compared to Self-Monitoring of Blood Glucose (SMBG) and to Venous Samples.
| DG5 | GC | RCGM | ||||
|---|---|---|---|---|---|---|
| Na,b | Mean ± SD | Na,b | Mean ± SD | Na,b | Mean ± SD | |
| Compared to SMBG routine measurements | ||||||
| Overall MRD | 53 | −2.7 ± 3.3 | 52 | −1.1 ± 6.4 | 54 | −4.9 ± 5.6 |
| Aggregated MRD | 3537 | −2.6 ± 13.5 | 3021 | −0.4 ± 15.8 | 3452 | −4.7 ± 16.3 |
| Compared to SMBG during dynamic phases | ||||||
| Aggregated MRD | 2389 | −0.5 ± 14.2 | 2236 | 0.5 ± 16.4 | 2331 | −5.6 ± 12.9 |
| Compared to venous hexokinase measurements during dynamic phases | ||||||
| Aggregated MRD | 2212 | 4.1 ± 16.2 | 2080 | 5.7 ± 18.3 | 2168 | −1.1 ± 14.6 |
Because of the mixed character of MRD with different units (mg/dL or %), numbers are given without a unit. In contrast to the mean absolute relative difference (MARD) shown in Table 2, MRD values also consider the direction (positive or negative) of the deviation in relation to the comparison method.
Number of experiments (participants) for MRD across all experiments (days with less than 6 paired data points were not included).
Number of paired data points for aggregated MRD.
In the CEG analysis, 99.8% (DG5), 99.6% (GC), and 99.2% (RCGM) of the CGM systems’ results were in zones A and B (Figure 2).
Figure 2.
Consensus error grid analyses per continuous glucose monitoring (CGM) system.
Accuracy of SMBG Measurements
For the SMBG system used as comparison and for calibration, 99.6% of the results were within ±15 mg/dL of the hexokinase measurements at glucose concentrations <100 mg/dL or within ±15% at glucose concentrations ≥100 mg/dL. A negligible bias of 0.3% was observed.
Usability
Detection Rate and Warning Rate for High and Low Glucose Values
Detection rates were 79% for DG5, 84% for GC, and 61% for RCGM for high glucose values and 88% for DG5, 82% for GC, and 83% for RCGM for low glucose values (Table 4). The rate of confirmed warnings in the high glucose range was similar for all three systems (62% to 64%). For low glucose values, the rate of confirmed warnings was lower than in the high glucose range (41% to 48%; Table 4). Analysis of the actual deviation in case of unconfirmed warnings showed that between 70% and 90% of CGM values at the time of the warning were less than 20% below the high glucose threshold for all systems and approximately 70-80% of CGM values with unconfirmed low glucose warnings were less than 20 mg/dL above the threshold.
Table 4.
Detection and Warning Rates per Continuous Glucose Monitoring (CGM) System for High (≥280 mg/dL) and Low (≤70 mg/dL) Glucose Values.
| System | Detection of high glucose values | Detection of low glucose values | Confirmed high glucose warnings | Confirmed low glucose warnings |
|---|---|---|---|---|
| DG5 | 211/268 (79%) | 264/301 (88%) | 103/163 (63%) | 211/436 (48%) |
| GC | 196/234 (84%) | 222/272 (82%) | 97/157 (62%) | 176/380 (46%) |
| RCGM | 166/271 (61%) | 252/302 (83%) | 111/173 (64%) | 205/497 (41%) |
Patch Adhesion Rate
The RCGM patch had the highest adhesion rate for the duration of wear (92.6%). Patch adhesion rates for the other systems were markedly lower (77.4% for DG5, 68.5% for GC).
User Evaluation
In general, the majority of the participants gave positive ratings for all three systems. Differences were found between systems in the rating of the receiver or respective smartphone app: for GC and RCGM, participants found that readability and functions offered were superior compared to DG5. The wearing comfort of GC was preferred by most participants due to the flat transmitter. Patch adherence was rated best for RCGM, followed by DG5 and GC. For GC, patch reactions like itching or redness were reported more frequently than for the other systems. In total, 8 participants reported problems with DG5, 20 participants with GC, and 25 participants with RCGM, most of which were connection problems between transmitter and receiver. Overall, the majority of participants (26 participants) preferred GC due to the user-friendly app and the flat format of the transmitter.
Safety
Adverse Events
During the study, 111 adverse events were reported by 50 participants. None of the adverse events were classified as serious. Eighty-four adverse events were considered related to the investigational devices (DG5 17, GC 42, RCGM 25; see Table 5). All device-related adverse events were classified as either mild (79) or moderate (5) by the study physicians.
Table 5.
Device-Related Adverse Events per Continuous Glucose Monitoring (CGM) System.
| Description | Number of reports | ||
|---|---|---|---|
| DG5 (57 systems) | GC (61 systems) | RCGM (57 systems) | |
| Total reports | 17 | 42 | 25 |
| Erythema at sensor application site | 2 | 18 | 6 |
| Erythema and itching at sensor application site | 0 | 1 | 0 |
| Hematoma at sensor insertion site | 4 | 6 | 5 |
| Inflammation at sensor insertion site | 3 | 0 | 3 |
| Itching at sensor application site | 4 | 8 | 5 |
| Pain at sensor insertion site | 3 | 4 | 2 |
| Pressure mark at sensor application site | 0 | 0 | 1 |
| Pustulae at sensor application site | 0 | 4 | 2 |
| Skin lesion at sensor application site | 1 | 1 | 1 |
Device Deficiencies
Thirty-three CGM-related device deficiencies were reported, 4 for DG5, 13 for GC, and 16 for RCGM. The most frequent deficiencies were transient connection failures between the transmitter and the receiver (39.4%) and system failures (36.4%). Other device deficiencies occurred rarely and included calibration failures, sensor application failures and damage of packaged systems of unknown cause. One RCGM sensor and 6 GC sensors ended the experimental phase prematurely after calibration, mainly due to sensor malfunctions and denied recalibration attempts. A further RCGM sensor was removed early without technical failure of the device. No DG5 sensors ended the experimental phase prematurely.
Overall, RCGM and DG5 had a system failure rate of 7% each, whereas 24% deficient systems were documented for GC (including 2 systems which were not applied because defects were noticed already after removing the packaging). The majority of system failures occurred before initial calibration (DG5: 4 systems, 100%; GC: 9 systems, 60%; RCGM: 3 systems, 60%).
Discussion
In this study, three different CGM systems were worn in parallel by 54 participants in a mixed inpatient and home setting in order to gain information about their performance in daily life. It is one of only a few studies in which different CGM systems have been tested in parallel, thus allowing a direct comparison of MARD values.11,14-17 Further development of RCGM was stopped shortly after the study; therefore this system is not available on the market.
In the current study, all CGM systems were used adjunctively to an SMBG system; however, DG5 is available and intended as a nonadjunctive device. In this analysis, it showed a marginally lower overall MARD compared to the other two systems. In the low glucose range and dynamic phases, however, RCGM showed the lowest MARD.
As expected, all three systems had their highest MARD on day 1. This can be explained by the flow rate of glucose into the sensor that requires several hours to stabilize after insertion.18 On all subsequent days, the CGM systems showed MARD values of about 10, with the exception of RCGM on day 3, where the MARD value was 13.7. This latter finding might be explained by an overcompensation of the drift by the algorithm during the first two days.
Since the MARD does not account for the direction of the deviation, that is, over- or underestimation of glucose values, the MRD between CGM and SMBG was also calculated in this study. Overall, all systems had a slight negative bias compared to SMBG. While DG5 and GC had the largest negative bias on day 1, RCGM initially showed a positive bias. For all three systems, at least 99% of values were within zones A and B of the CEG, indicating no risk from a clinical perspective.
Of note is that the study design provides only information about CGM performance relative to the SMBG system used. Different study settings and procedures limit the comparability of the presented results to other published studies. Nevertheless, similar results were found by other investigators. DG5 achieved a MARD of 13.3%19 and 11.3%10, and a MARD of 12.38%20 was shown for the sensor of GC. For a predecessor developmental model of RCGM, a MARD of 9.2% was reported a few years ago.21
In this study, venous samples were used as an additional comparison during dynamic phases. Comparison of the MARD and MRD results shows the impact of choice of the reference sample.22 In general, MRD during dynamic phases was approximately 5% higher when compared to venous samples because postprandial glucose concentrations are higher in capillary blood than in venous blood.23 Consequently, RCGM which had the largest negative bias of the three systems compared to SMBG in the hyperglycemic range, showed the smallest absolute bias compared to venous samples. The BGMS used for calibration likely was only a minor influencing factor as a comparison of capillary samples measured by the BGMS and by the laboratory analyzer showed only negligible bias. Because venous comparison samples are often used during dynamic phases for practical reasons, the impact on accuracy results has to be taken into account.
Usability aspects and the perceived accuracy of a CGM system can contribute to adherence to CGM and thus to the success of therapy.24 The three systems used in this study were similar with regard to detection rates and warnings for high and low glucose values, reflecting performance in the different glucose ranges. The rate of confirmed low glucose warnings was less than 50%, but most of these false-positive warnings occurred at glucose values close to the actual threshold and might therefore have had a preventive rather than a bothersome character.
Participants rated adherence of the patch best for RCGM, and this was confirmed by the calculated patch adhesion rate. In general, all three systems were rated positively, and the majority of participants stated that altogether they liked GC most because it has a flat transmitter and offers a user-friendly app with a pleasant display of data. As DG5 was used with the receiver in this study, and the other devices with a smartphone, comparability is limited. The lowest numbers of device-related adverse events, that is, application site issues, and device deficiencies were reported for DG5; however, no serious or severe events occurred with any of the systems.
Conclusion
The three CGM systems tested in this study showed similar overall performance with differences depending on the specific glucose range, study phase (ie, home use or study site visits with dynamic phases), and sample material used for comparison. Generally, all systems were rated positively by most users, but with differences in individual preferences based on usability aspects.
Acknowledgments
The authors would like to thank Delia Waldenmaier, Stefan Pleus, and Jochen Mende (all IDT) and Chris Priestley (Chris Priestley Ltd) for medical writing and editing of the manuscript.
Footnotes
Abbreviations: CEG, consensus error grid; CGM, continuous glucose monitoring; DG5, Dexcom G5; GC, Guardian Connect; MARD, mean absolute relative difference; MRD, mean relative difference; RCGM, Roche CGM; SMBG, self-monitoring of blood glucose
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: GF is general manager of the IDT (Institut für Diabetes-Technologie, Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies on the evaluation of BG meters and medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IDT have received speakers’ honoraria or consulting fees from Abbott, Ascensia, Bayer, LifeScan, Menarini Diagnostics, Novo Nordisk, Roche, Sanofi, Sensile, and Ypsomed. ML, UK, and CH are employees of the IDT. BB is employee of the Wehrle-Diakonissen Hospital, Salzburg, Austria, and has no financial interests concerning this study. RW is head of diabetology at the Department of Internal Medicine, Wehrle-Diakonissen Hospital, Salzburg, Austria, which takes care of people with diabetes and carries out various clinical pharmacological and medical device studies. RW has received speaker’s honoraria or consulting fees from Abbott Diabetes Care, Astra Zeneca, Boehringer-Ingelheim, Dexcom, Eli Lilly, MSD, Novo Nordisk, Roche Diabetes Care, Sanofi, Servier, Spar, and Takeda.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study and writing of the manuscript were funded by Roche Diabetes Care GmbH, Germany.
ORCID iD: Guido Freckmann
https://orcid.org/0000-0002-0406-9529
References
- 1. Rodbard D. Continuous glucose monitoring: a review of recent studies demonstrating improved glycemic outcomes. Diabetes Technol Ther. 2017;19:S25-S37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Heinemann L, Freckmann G, Ehrmann D. et al. Real-time continuous glucose monitoring in adults with type 1 diabetes and impaired hypoglycaemia awareness or severe hypoglycaemia treated with multiple daily insulin injections (HypoDE): a multicentre, randomised controlled trial. Lancet. 2018;391:1367-1377. [DOI] [PubMed] [Google Scholar]
- 3. Rodbard D. Continuous glucose monitoring: a review of successes, challenges, and opportunities. Diabetes Technol Ther. 2016;18(suppl 2):S3-S13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. FDA advisory panel votes to recommend non-adjunctive use of Dexcom G5 mobile CGM. Diabetes Technol Ther. 2016;18:512-516. [DOI] [PubMed] [Google Scholar]
- 5. Shapiro AR. Nonadjunctive use of continuous glucose monitors for insulin dosing: is it safe? J Diabetes Sci Technol. 2017;11:833-838. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kirchsteiger H, Heinemann L, Freckmann G. et al. Performance comparison of CGM systems: MARD values are not always a reliable indicator of CGM system accuracy. J Diabetes Sci Technol. 2015;9:1030-1040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Brazg RL, Klaff LJ, Sussman AM. New generation blood glucose monitoring system exceeds international accuracy standards. J Diabetes Sci Technol. 2016;10:1414-1415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Garcia A, Rack-Gomer AL, Bhavaraju NC. et al. Dexcom G4AP: an advanced continuous glucose monitor for the artificial pancreas. J Diabetes Sci Technol. 2013;7:1436-1445. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Bailey T, Bode BW, Christiansen MP, Klaff LJ, Alva S. The performance and usability of a factory-calibrated flash glucose monitoring system. Diabetes Technol Ther. 2015;17:787-794. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Bailey TS, Chang A, Christiansen M. Clinical accuracy of a continuous glucose monitoring system with an advanced algorithm. J Diabetes Sci Technol. 2015;9:209-214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kropff J, Bruttomesso D, Doll W. et al. Accuracy of two continuous glucose monitoring systems: a head-to-head comparison under clinical research centre and daily life conditions. Diabetes Obes Metab. 2015;17:343-349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Clinical and Laboratory Standards Institute (CLSI). POCT05-A Performance Metrics for Continuous Interstitial Glucose Monitoring. Approved Guideline. Vol. 28, No. 33 Wayne, PA: CLSI. [Google Scholar]
- 13. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care. 2000;23:1143-1148. [DOI] [PubMed] [Google Scholar]
- 14. Kovatchev B, Anderson S, Heinemann L, Clarke W. Comparison of the numerical and clinical accuracy of four continuous glucose monitors. Diabetes Care. 2008;31:1160-1164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Freckmann G, Pleus S, Link M, Zschornack E, Klotzer HM, Haug C. Performance evaluation of three continuous glucose monitoring systems: comparison of six sensors per subject in parallel. J Diabetes Sci Technol. 2013;7:842-853. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Damiano ER, McKeon K, El-Khatib FH, Zheng H, Nathan DM, Russell SJ. A comparative effectiveness analysis of three continuous glucose monitors: the Navigator, G4 Platinum, and Enlite. J Diabetes Sci Technol. 2014;8:699-708. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Matuleviciene V, Joseph JI, Andelin M. et al. A clinical trial of the accuracy and treatment experience of the Dexcom G4 sensor (Dexcom G4 System) and Enlite sensor (Guardian REAL-Time System) tested simultaneously in ambulatory patients with type 1 diabetes. Diabetes Technol Ther. 2014;16:759-767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Castle JR, Ward WK. Amperometric glucose sensors: sources of error and potential benefit of redundancy. J Diabetes Sci Technol. 2010;4:221-225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Faccioli S, Del Favero S, Visentin R. et al. Accuracy of a CGM sensor in pediatric subjects with type 1 diabetes. Comparison of three insertion sites: arm, abdomen, and gluteus. J Diabetes Sci Technol. 2017;11:1147-1154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Taleb N, Emami A, Suppere C. et al. Comparison of two continuous glucose monitoring systems, Dexcom G4 Platinum and Medtronic Paradigm Veo Enlite system, at rest and during exercise. Diabetes Technol Ther. 2016;18:561-567. [DOI] [PubMed] [Google Scholar]
- 21. Zschornack E, Schmid C, Pleus S. et al. Evaluation of the performance of a novel system for continuous glucose monitoring. J Diabetes Sci Technol. 2013;7:815-823. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Kropff J, van Steen SC, deGraaff P. et al. Venous, arterialized-venous, or capillary glucose reference measurements for the accuracy assessment of a continuous glucose monitoring system. Diabetes Technol Ther. 2017;19:609-617. [DOI] [PubMed] [Google Scholar]
- 23. Thennadil SN, Rennert JL, Wenzel BJ, Hazen KH, Ruchti TL, Block MB. Comparison of glucose concentration in interstitial fluid, and capillary and venous blood during rapid changes in blood glucose levels. Diabetes Technol Ther. 2001;3:357-365. [DOI] [PubMed] [Google Scholar]
- 24. Polonsky WH, Hessler D. Perceived accuracy in continuous glucose monitoring: understanding the impact on patients. J Diabetes Sci Technol. 2015;9:339-341. [DOI] [PMC free article] [PubMed] [Google Scholar]


