Performance of Three Continuous Glucose Monitoring Systems in Adults With Type 1 Diabetes

Abstract

Background:

The performance of continuous glucose monitoring (CGM) systems is difficult to compare due to different study designs and a lack of head-to-head studies. This study evaluated the performance of FreeStyle Libre 3 (FL3), Dexcom G7 (DG7), and Medtronic Simplera (MSP) against different comparator methods and during clinically relevant glycemic scenarios.

Method:

Twenty-four adult participants with type 1 diabetes mellitus wore one sensor of each CGM system in parallel for up to 15 days. Sensors of DG7 and MSP were exchanged on days 5 and 8, respectively. Three 7-hour sessions with 15-minute comparator blood glucose–level measurements using YSI 2300 (YSI, venous), Cobas Integra (INT, venous), and Contour Next (CNX, capillary) were conducted on days 2, 5, and 15. Simultaneously, glucose-level excursions with transient hyperglycemia and hypoglycemia were induced according to a recently published testing procedure. The accuracy was evaluated using various metrics, including mean absolute relative differences (MARDs).

Results:

Compared with YSI data, the MARDs of FL3, DG7, and MSP were 11.6%, 12.0%, and 11.6%, respectively. Relative to the INT data, the corresponding MARDs were 9.5%, 9.9%, and 13.9%, respectively, and compared with CNX data, MARDs were 9.7%, 10.1%, and 16.6%, respectively. Both FL3 and DG7 showed better accuracy in the normoglycemic and hyperglycemic range, while MSP performed better in the hypoglycemic range.

Conclusions:

Performance results of all CGM systems varied depending on the comparator method. However, across comparators FL3 and DG7 tended to be more accurate compared with MSP. All CGM systems showed a lower accuracy compared with previous studies, emphasizing the need for comprehensive study design guidelines.

Introduction

The availability of accurate and reliable systems for continuous glucose monitoring (CGM) is crucial for the safe and effective application of most modern diabetes therapy approaches. ¹ In particular, CGM-based glucose management reports are now used in routine clinical practice to determine the status of glycemic control. ² However, several studies have shown that at least for older-generation CGM systems, glycemic control metrics such as time in range or the glucose management indicator can differ substantially between systems,^3-6 which is most likely due to differing device characteristics. ⁷ It is, however, challenging to compare the performance of different CGM systems, and thus to evaluate the reliability of provided therapy metrics, due to a lack of comprehensive and detailed guidelines describing procedures for the standardized performance testing of CGM systems, in particular regarding the collection and characteristics of comparator data.^8,9

To address this issue, a group of experts recently proposed recommendations for the distribution of comparator data and an associated procedure for manipulating the participants’ glucose levels, which involves the recreation of clinically relevant glycemic situations where CGM accuracy is of particular importance. ¹⁰ We have, therefore, designed a study incorporating these procedures to assess the clinical performance of current-generation CGM systems from the three principal manufacturers: Abbott, Dexcom, and Medtronic. Due to the lack of a consensus on the optimal comparator measurement approach for CGM performance studies, specifically regarding the choice of comparator device and sample origin, CGM performance was evaluated using three different comparator methods to highlight the effects of these varying approaches.

Methods

The study was conducted between April and July 2024 as a prospective, interventional study in adult participants with type 1 diabetes mellitus by the Institute for Diabetes Technology Ulm, Germany. The main exclusion criteria were severe hypoglycemia in the six months prior to enrollment, hypoglycemia unawareness, HbA1c >10%, and intake of substances known to affect the performance of the examined CGM systems. A complete list of inclusion and exclusion criteria and additional information on the methods are provided in the supplemental materials. Ethical approval was granted by the responsible ethics committee and the study was notified to the competent authority and registered with the German Clinical Trials Register (DRKS00033697).

Study Devices

This study examined the performance of the FreeStyle Libre 3 (FL3; Abbott Diabetes Care Inc., Alameda, California), Dexcom G7 (DG7; Dexcom Inc., San Diego, California), and Medtronic Simplera (MSP; Medtronic Minimed, Northridge, California) CGM systems, with sensor lifetimes of 14, 10 (plus a grace period of 0.5 days), and 7 days, respectively. These CGM systems are factory-calibrated, but DG7 and MSP allow optional calibrations that were not performed during the study. Each participant received a single, android-based smart device on which the three software applications of the CGM systems were installed. The sensors of FL3 were ordered from regular distribution channels of Abbott Diabetes Care Germany without explicitly declaring their use in a clinical performance study. This was not possible for the sensors of DG7 and MSP, which were ordered from the manufacturers after disclosing the purpose of their use.

Venous comparator blood glucose (BG) concentration measurements were obtained with the YSI 2300 STAT PLUS laboratory analyzer (YSI; YSI Inc., Yellow Springs, Ohio) utilizing a glucose oxidase–based method and the COBAS INTEGRA 400 plus Analyzer (INT; Roche Diagnostics GmbH, Mannheim, Germany) using a hexokinase-based method. Capillary comparator measurements were obtained with the handheld Contour Next blood glucose monitoring system (CNX, Ascensia Diabetes Care Holdings AG, Basel, Switzerland) using a glucose hydrogenase–based approach. All comparator measurements were carried out in duplicate.

Study Procedures

The study duration, during which the participants wore the CGM sensors, was 15 calendar days, with most time spent in a free-living setting. Three frequent sampling periods (FSPs) were scheduled on study days 2, 5, and 15. An overview of the study protocol is provided in Figure 1 of the supplemental materials. One sensor of each CGM system was inserted on day 1 by the participants in their upper arms and sensors could be replaced within the first 12 hours in case of failure. Insertion sites were equally distributed between the right and left arms for sensors of the same CGM system. The sensors of FL3 were worn until the end of their lifetime (14 days) on day 15. The sensors of MSP were replaced on day 8 at the end of their lifetime (seven days). The sensors of DG7 were replaced before the end of their lifetime (10.5 days) on study day 5 after the FSP to ensure that the second set of DG7 sensors reached the end of their lifetime on study day 15. Sensors could be affixed with additional tape if necessary. On study days 1 and 15, the participants were asked to complete a questionnaire for user satisfaction standardized for CGM performance studies currently under development. ¹¹

On study day 1, CNX measurements were collected every 30 minutes for approximately five hours after insertion of the sensors (not part of the FSPs) and twice during the night. In the free-living setting, participants followed their regular daily routine but were asked to perform at least seven capillary BG measurements per day (immediately before and two hours after breakfast, lunch and dinner, and before going to bed).

During the seven-hour FSPs, comparator measurements were scheduled every 15 minutes with YSI and INT (venous plasma) and CNX (capillary whole blood). Simultaneously, participants underwent a previously proposed glucose manipulation procedure, designed to produce a sufficient percentage of comparator data indicative of both high and low as well as rapidly rising and falling BG levels. ¹⁰ For this procedure, participants consumed a carbohydrate-rich breakfast followed by a delayed insulin bolus to induce initial hyperglycemia, then hypoglycemia accompanied by rapid changes in BG levels, and finally stable BG levels in the normoglycemic range. Individual excursions were managed based on capillary measurements by an experienced physician using fast-absorbed carbohydrates, additional insulin boluses, and mild exercise to ensure that the targeted comparator data distribution is achieved and participant safety is maintained.

Data Analysis

Comparator measurement duplicates were averaged. The distribution of comparator measurements collected during the FSPs was assessed by classifying the combination of BG level and rate of change (RoC) in the previously proposed dynamic glucose regions (DGR). ¹⁰

As the retrospectively available data from FL3 and MSP do not allow a distinction between readings at and outside the measuring range limits (40-500 mg/dL for FL3, 40-400 mg/dL for DG7, 50-400 mg/dL for MSP), CGM readings of all systems at or outside these limits were excluded for the analysis of analytical point accuracy. Here, comparator measurements were paired with CGM readings recorded closest in time with a maximum time difference of ±5 minutes. For that, the CGM data automatically stored every five minutes by all CGM systems were used.

Aggregate differences between paired CGM and comparator measurements were characterized using mean absolute relative difference (MARD), relative bias (also known as mean relative difference), and agreement rate (AR), indicating the percentage of CGM measurements within ±20 mg/dL (for comparator values <100 mg/dL) or ±20% (for comparator values ≥100 mg/dL). For the bias, the standard deviation (SD) of individual relative differences was calculated to indicate CGM system imprecision. The Continuous Glucose Deviation Interval and Variability Analysis (CG-DIVA) was also carried out. ¹² Clinical point accuracy was assessed using the newly introduced Diabetes Technology Society Error Grid. ¹³ The CGM and comparator glucose profiles collected during FSPs were averaged to allow a qualitative but accessible evaluation of CGM accuracy, which was made possible by the harmonized glucose manipulation procedure which provided sufficient consistency in the glucose profiles. Hypoglycemia and hyperglycemia alert reliability was examined by calculating true alert rate (percentage of CGM readings <70 mg/dL or >250 mg/dL concurrent with a comparator measurement below/above the same threshold within ±15 minutes) and the true detection rate (percentage of comparator measurements <70 mg/dL or >250 mg/dL concurrent with a CGM reading below/above the same threshold within ±15 minutes). The stability of sensors was characterized by calculating MARD and bias for every 24-hour period after sensor insertion with respect to free-living capillary comparator data. This data set included all CNX measurements collected outside the FSPs as well as four measurements collected during the FSPs at 08:00, 10:00, 13:00, and 15:00.

Technical reliability was assessed using a Kaplan-Meier sensor survival analysis, ¹⁴ including documentation of failure modes. Safety was examined by documenting device-related adverse events.

Results

Comparator Data

A total of 24 adult participants with type 1 diabetes were included and completed the study. Their baseline characteristics are shown in Table 1.

Table 1.

Baseline Characteristics of the Participant Population.

Characteristic	Value
Gender, N (%)
Male	17 (70.8)
Female	7 (29.2)
Age (years) a	51.5 (23, 71)
Body mass index (kg/m2) a	26.1 (21, 32)
Diabetes type, type 1, N (%)	24 (100)
Diabetes duration (years) a	25.9 (3, 46)
HbA1c (%) a	6.6 (5.3, 8.2)
Therapy regimen, N (%)
MDI	5 (20.8)
CSII	19 (79.2)

Abbreviations: MDI, multiple daily injections; CSII, continuous subcutaneous insulin infusion.

^aResults are given as mean (minimum, maximum).

Of the 2088 measurements scheduled during the FSPs, 2060, 2059, and 2063 data points were obtained with YSI, INT, and CNX, respectively. The bias between INT and YSI (with respect to YSI) was +5.2%, the bias between CNX and YSI (with respect to YSI) was +9.9%, and the bias between CNX and INT (with respect to INT) was +4.5%. The distribution of comparator BG-RoC pairs, in terms of the DGR and the mean absolute RoC (MARoC), which is an indicator of the overall speed of BG level changes, are shown in Table 2. Here, the total number of BG-RoC pairs is reduced, for example, due to missing consecutive values and because no RoC can be calculated for the first measurement of the FSP. The corresponding DGR plots are shown in the supplemental materials.

Table 2.

Characteristics and Distribution of Comparator BG-RoC Pairs According to the DGR.

DGR	Definition a	YSI, N (%)	INT, N (%)	CNX, N (%) b
BG low	BG < 70 mg/dL, any RoC	282 (14.2)	237 (12.0)	217 (10.9)
BG high	BG > 300 mg/dL, any RoC	139 (7.0)	206 (10.4)	269 (13.6)
Alert low	BG ≥ 70 mg/dL, RoC < −1 mg/dL/min BG < 70 mg/dL within 30 min at current RoC	166 (8.4)	177 (8.9)	194 (9.8)
Alert high	BG ≤ 300 mg/dL, RoC > +1.5 mg/dL/min BG > 250 mg/dL within 30 minutes at current RoC	175 (8.8)	168 (8.5)	148 (7.5)
Neutral	All other RoC-BG pairs	1223 (61.6)	1195 (60.3)	1157 (58.3)
Total	–	1985	1983	1985
MARoC	Mean absolute RoC in mg/dL/min	1.27	1.34	1.44

Abbreviations: BG, blood glucose; DGR, Dynamic Glucose Regions; RoC, rate of change; YSI, YSI 2300 STAT PLUS laboratory analyzer; INT, COBAS INTEGRA 400 plus Analyzer; CNX, Contour Next blood glucose monitoring system.

^aBG levels were paired with RoC values calculated from the current and preceding BG level spaced ≤20 minutes apart.

^bOnly CNX data collected during frequent sampling periods.

CGM Performance

In total, 25 FL3, 48 DG7, and 49 MSP sensors were inserted (on study day 1, one FL3 sensor was replaced due to wearing discomfort and one MSP sensor was replaced due to failure). Results of the aggregate analysis of point accuracy for all three CGM systems with respect to different comparator data sets and collection timepoints are shown in Table 3. Here, results with respect to capillary CNX measurements were calculated separately for data collected during the FSPs, data collected outside the FSPs (including four measurements taken during FSPs at 08:00, 10:00, 13:00 and 15:00), and data from the first approximately 12 hours after sensor insertion on study day 1. Beyond these aggregate point accuracy metrics, only results with respect to YSI measurements are shown in the main text. The corresponding results for INT and CNX (collected during the FSPs) are provided in the supplemental materials.

Table 3.

Aggregate Point Accuracy Results for All CGM Systems and Comparator Devices.

	FL3	DG7	MSP
YSI (venous)
N Datapoints / N Sensors  20/20 AR a  MARD  Bias b	2002 / 24 88.9% 11.6% +8.4% ± 13.4%	2047 / 48 88.0% 12.0% +7.1% ± 13.9%	1932 / 46 87.0% 11.6% −6.0% ± 13.3%
INT (venous)
N Datapoints / N Sensors  20/20 AR a  MARD  Bias b	2001 / 24 93.2% 9.5% +3.1% ± 12.8%	2046 / 48 91.0% 9.9% +1.8% ± 13.4%	1931 / 46 80.2% 13.9% −10.6% ± 12.9%
CNX (capillary, FSPs only)
N Datapoints / N Sensors  20/20 AR a  MARD  Bias b	2005 / 24 92.4% 9.7% −1.1% ± 12.7%	2049 / 48 90.1% 10.1% −2.5% ± 13.1%	1933 / 46 68.6% 16.6% −14.5% ± 12.7%
CNX (capillary, free-living)
N Datapoints / N Sensors  20/20 AR a  MARD  Bias b	2796 / 25 93.4% 8.7% −1.7% ± 11.6%	2816 / 48 88.9% 10.7% −4.8% ± 12.9%	2661 / 48 72.3% 15.7% −13.7% ± 12.1%
CNX (capillary, first 12 hours)
N Datapoints / N Sensors  20/20 AR a  MARD  Bias b	303 / 25 84.5% 10.9% −5.9% ± 12.9%	307 / 24 81.4% 12.8% −7.8% ± 14.9%	262 / 24 53.8% 20.0% −16.9% ± 16.2%

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; AR, agreement rate; MARD, mean absolute relative difference; FSP, frequent sampling period; YSI, YSI 2300 STAT PLUS laboratory analyzer; INT, COBAS INTEGRA 400 plus Analyzer; CNX, Contour Next blood glucose monitoring system.

^aThreshold for switching between absolute (±20 mg/dL) and relative (±20%) differences at 100 mg/dL.

^bValues given as mean ± standard deviation of relative differences.

The results of the CG-DIVA are shown in Figure 1 (numerical results are provided in the supplemental materials) and the analysis of clinical point accuracy is displayed in Figure 2. The stratification of MARD and bias with respect to YSI RoCs is provided in Table 4.

Figure 1.

Panel a: Deviation intervals with respect to YSI of the continuous glucose deviation and variability analysis (CG-DIVA) in different glucose ranges covering the US food and drug administration iCGM criteria: the dark gray boxes contain 85%, 70%, 80%, and 87% of expected deviations, respectively, and the light gray boxes contain 98%, 99%, and 99% of expected deviations. Colored dashes show median deviation. Panel b: Sensor-specific median (dot) and 90% range of relative differences to YSI. For DG7 and MSP solid and dashed lines indicate sensors tested before and after sensor replacement, respectively.

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; YSI, YSI 2300 STAT PLUS laboratory analyzer.

Figure 2.

Diabetes technology society error grid analysis of FL3 (panel a), DG7 (panel b) and MSP (panel c) against YSI data.

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; YSI, YSI 2300 STAT PLUS laboratory analyzer; CGM, continuous glucose monitoring.

Table 4.

Point Accuracy With Respect to Comparator (YSI) Measurement Rate of Change.

YSI RoC (mg/dL/min)	N			MARD			Bias
	FL3	DG7	MSP	FL3	DG7	MSP	FL3	DG7	MSP
<−2	158	164	158	16.2%	16.3%	9.2%	+15.5%	+13.9%	+1.0%
≥−2 to <−1	355	367	340	13.1%	14.8%	9.7%	+11.7%	+12.3%	−0.5%
≥−1 to ≤+1	992	1005	946	12.1%	11.8%	10.4%	+9.9%	+6.8%	−4.5%
>+1 to ≤+2	187	194	186	7.9%	9.3%	13.3%	+2.7%	+2.3%	−12.1%
>+2	237	242	229	8.1%	8.7%	20.1%	−2.2%	−0.6%	−19.9%

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; RoC, rate of change; MARD, mean absolute relative difference; YSI, YSI 2300 STAT PLUS laboratory analyzer.

A graphical summary of the CGM accuracy complementing the previous results is provided in Figure 3. In panel a, the mean CGM profiles during the first half of the FSP, when hyperglycemia was dominant, are presented. This was made feasible by the synchronized consumption of breakfast. The BG profiles became more discordant between participants in the second half of the FSP when hypoglycemia was induced. Therefore, panel b shows the average comparator and CGM profiles synchronized to the first YSI measurement <70 mg/dL, thus demonstrating the average CGM behavior prior to, during, and following hypoglycemia. Note that this analysis does not eliminate CGM system time lag. The results of the alert reliability analysis are shown in Table 5.

Figure 3.

Mean time course of comparator and CGM glucose data during the first half of the frequent sampling period (FSP, panel a) and surrounding hypoglycemia (panel b). In panel b, all measurements were synchronized to the time point of the first YSI measurement <70 mg/dL after the initial glucose level peak.

Abbreviations: CGM, continuous glucose monitoring; FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; YSI, YSI 2300 STAT PLUS laboratory analyzer; INT, COBAS INTEGRA 400 plus Analyzer; CNX, Contour Next blood glucose monitoring system.

Table 5.

Alert Reliability With Respect to YSI Data.

	FL3	DG7	MSP
≤70 mg/dL
N Alerts  True Alert Rate  N Detections  True Detection Rate	582 93.6% 278 73.0%	672 87.5% 285 79.6%	956 84.7% 276 93.1%
≥250 mg/dL
N Alerts  True Alert Rate  N Detections  True Detection Rate	1251 88.9% 355 99.7%	1311 88.0% 361 99.7%	714 95.5% 345 84.6%

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; YSI, YSI 2300 STAT PLUS laboratory analyzer.

The stability of the CGM systems, that is, accuracy compared with free-living CNX data with respect to the sensor wear time, is shown in Figure 4. Here, it should be reiterated that half of DG7 sensors were removed per protocol after approximately four days, thus halving the amount of available data pairs from day 5 onward.

Figure 4.

Stability of CGM sensors in terms of MARD (panel a) and bias (panel b) with respect to free-living CNX measurements stratified according to the time after sensor insertion. The dashed lines indicate aggregate results for this data set over the entire study duration.

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system; MARD, mean absolute relative difference; CNX, Contour Next blood glucose monitoring system.

Regarding the technical reliability, the results of the Kaplan-Meier survival analysis are shown in Figure 5. The estimated survival probability to the end of the respective sensor lifetimes and 95% confidence intervals (CIs) were 91.7% (CI: 70.6%-97.8%, N = 25, two failures) for FL3, 100% (CI not possible, N = 48, no failures) for DG7, and 81.5% (CI: 67.5%-89.9%, N = 49, nine failures) for MSP. For FL3, one failure was due to a sensor error and one failure due to adhesive weakness. For MSP, all failures were due to sensor errors.

Figure 5.

Kaplan-Meier survival plot. The vertical lines indicate sensors removed prior to the end of their lifetime as part of the protocol (censoring).

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system.

A selection of results from the user satisfaction questionnaire is shown in Figure 6. It consisted of several items of the CGM user experience that were rated on a scale of 1 to 5.

Figure 6.

Summary results of selected items of the user satisfaction questionnaire.

Abbreviations: FL3, FreeStyle Libre 3 CGM system; DG7, Dexcom G7 CGM system; MSP, Medtronic Simplera CGM system.

Safety analysis identified five, three, and three adverse events related to FL3, DG7, and MSP, respectively. All events were localized to the sensor insertion sites and all but one event included hematoma or erythema and were classified as mild. The remaining event was classified as moderate, where pain after insertion of an FL3 sensor led to the replacement of the sensor on day 1, as mentioned above. As this sensor was functioning at the time of removal, it was censored in the survival analysis and not regarded as a failure.

Discussion

For the first time, the present study evaluated the performance of the current-generation systems of the three principal CGM system manufacturers Abbott, Dexcom, and Medtronic in a head-to-head design. For this assessment, a study protocol was developed that accounted for the differing sensor lifetimes while ensuring that comparator data were collected at the beginning, middle, and end of each sensor’s lifetime. A limitation of this protocol is that, due to the differing sensor lifetimes between products, twice as many DG7 and MSP sensors were tested compared with FL3 sensors. This discrepancy slightly impairs the comparability of results between CGM systems. Another relevant difference between the tested CGM systems is that the values displayed on the FL3 app to the user can differ from the continuously stored data; this was not the case for DG7 and MSP. However, we used the continuously stored data from all CGM systems for the performance analysis to facilitate the study procedures and harmonize the pairing between CGM and comparator data. Furthermore, we opted not to use the optional calibration feature of DG7 and MSP, which might have improved their accuracy, as this is impossible for FL3.

This study collected comparator measurements from three combinations of comparator sample origin and measurement device during in-clinic sessions. While differences of physiological origin between capillary and venous BG levels are expected, ¹⁵ we also found a considerable bias between YSI and INT measurements (+5.2% with respect to YSI) despite being carried out on the same venous blood samples. These differences in comparator data are reflected in CGM accuracy results and highlight the need to standardize the comparator measurement approach in CGM performance studies. ¹⁶

The study also utilized a recently suggested procedure for manipulating BG levels to reproduce clinically relevant scenarios in which CGM accuracy is essential. This was achieved by targeting rapidly increasing and decreasing BG levels toward hyperglycemia and hypoglycemia occurring in real life (Figure 3). In addition, the procedure was kept sufficiently consistent for all participants and FSPs to allow the clear visualization in Figure 3. The recommended distribution characteristics, that is, at least 7.5% of BG-RoC pairs in each critical DGR (BG high, BG low, Alert high and Alert low), were met by CNX and INT. For YSI, there was an insufficient number of pairs in the BG high region, which can be explained by the fact that BG level manipulation was carried out based on CNX measurement results and the positive bias of CNX (+9.9%) with respect to YSI. A separate article will discuss the feasibility of the glucose manipulation procedure and possible adaptations in more detail.

The CGM performance results indicate similar MARDs and ARs in FL3 and DG7 and larger disparities to MSP for all comparators. The CGM readings of FL3 and DG7 generally agreed, with MSP showing lower readings in comparison. In terms of bias to the comparators during the FSPs, FL3 readings matched closest to CNX measurements (−1.1%), DG7 readings showed best agreement to INT measurements (+1.8%), and MSP readings were closest to YSI measurements (−6.0%). Furthermore, it can be concluded that FL3 and DG7 perform well during fast increasing, normal, and high BG levels, whereas MSP appears to adequately capture the decrease in BG levels before and the nadir of hypoglycemia. The alert reliability results also reflect this impression. All CGM systems showed reduced accuracy during the first 12 hours after insertion and a more negative bias, compared with the aggregate accuracy over the entire lifetime. The stability analysis over the rest of the sensor lifetime in FL3 and DG7 showed a typical pattern with lowest accuracy at the beginning and end of the sensor lifetime (Figure 4). In contrast, accuracy of MSP increased almost consistently over the sensor lifetime with the best accuracy found on the last day. The sensor-to-sensor variability results are difficult to compare due to the differing number of sensors, but appear slightly better for FL3 (Figure 1b). In terms of the sensor survival, most sensor failures were found in MSP. The evaluation of user satisfaction indicates similar ratings in FL3 and DG7, with MSP showing more negative impressions for several items, for example, the reliability of CGM readings. The analysis of adverse device effects only revealed expected events that mainly were classified as mild.

The data availability, which assesses the ability of a CGM system to provide a continuous stream of measurements to the user, was not examined due to the idiosyncrasies in the data display and recording of FL3 and DG7. While FL3 showed a nearly complete data set in the retrospectively downloaded data, there were numerous instances during the FSPs where no readings were displayed by the app, especially during high RoCs. In contrast, DG7 occasionally displayed values that could not be retrieved by the retrospective data download, preventing a fair assessment of data availability.

Previous performance reports of FL3, DG7, and MSP for adults with type 1 diabetes are relatively sparse.^17-20 Studies that were likely used to gain regulatory approval found consistently higher accuracies compared to this study with MARDs in adults relative to venous YSI data of 7.5% for FL3, ¹⁷ 8.2% for DG7, ¹⁸ and 10.2% for MSP, ¹⁹ which is likely due to different study procedures. This suggests that none of the tested CGM systems was favored by the design of the presented study. For DG7, a relevant difference between this study and the study reported by Garg et al ¹⁸ was the use of “arterialization” of venous samples, which could have increased venous BG levels, thus reducing the positive bias of DG7 observed in this study. Furthermore, Garg et al ¹⁸ likely collected comparator data with lower RoCs (the MARoC was likely <1 mg/dL/min, this study had an MARoC of ~1.3 mg/dL/min) and paired comparator measurements with CGM data recorded afterward, instead of closest in time, both of which can lead to better accuracy results. For FL3 and MSP, the published reports lack the necessary information to assess the differences in study procedures in more detail. These observations underscore the importance of adopting standardized testing protocols for CGM systems.

Hanson et al ²⁰ recently published a head-to-head comparison between FL3 and DG7 that found MARDs of 8.9% and 13.6% relative to venous YSI measurements, respectively. These results could also not be replicated in the presented study. In particular, the results of Hanson et al indicated a sizable systematic bias between FL3 and DG7 of around 9%, whereas the bias found in the present study was only 1% to 2% (see Table 3 by comparing the bias of FL3 and DG7). This discrepancy is unlikely to result from different study procedures but rather caused by the tested sensors themselves.

Conclusion

This study implemented a recently published testing protocol to recreate clinically relevant glycemic scenarios with varying glucose dynamics. In addition, it represents the first manufacturer-independent performance evaluation of the three current-generation CGM systems from Abbott, Dexcom, and Medtronic relative to different comparator measurement approaches. Despite the rigorous testing conditions, our results show adequate performance across all tested CGM systems while revealing differences in accuracy between FL3 and DG7 on one hand, and MSP on the other. We thus argue that our study procedures can effectively identify weaknesses in performance, which, if implemented in a future standard, can help ensure the continued safety and reliability of CGM systems as an increasing number of manufacturers enter the market. In addition, a direct comparison of CGM system performance allows users to understand the strengths and weaknesses of each system.

We also found different results with different comparator measurement approaches as well as differences in accuracy compared with previously published studies. This observation again demonstrates that the results of a CGM performance evaluation highly depend on the chosen study procedures. This finding must be considered when defining any minimum performance criteria, such as the US Food and Drug Administration’s “integrated” CGM requirements, ²¹ or the notion that an MARD <10% indicates the suitability of a CGM system for non-adjunctive use in diabetes therapy. ²² We, therefore, conclude that minimum CGM performance criteria cannot be established without comprehensive guidelines for study procedures, in particular the collection of comparator data, which is the main objective of the Working Group on CGM established by the International Federation of Clinical Chemistry and Laboratory Medicine. ¹⁶