Performance Assessment of Three Continuous Glucose Monitoring Systems in Adults With Type 1 Diabetes

Abstract

Background:

FIND, the global alliance for diagnostics, identified the nonmarket-approved continuous glucose monitoring (CGM) system, FiberSense system (FBS), as a potential device for use in low- and middle-income countries. Together with two market-approved, factory-calibrated CGM systems, namely, the FreeStyle Libre 2 (FL2) and the GlucoRx AiDEX (ADX), the FBS was subjected to a clinical performance evaluation.

Methods:

Thirty adult participants with type 1 diabetes were enrolled. The study was mainly conducted at home, with three in-clinic sessions conducted over the study period of 28 days. Comparator measurements were collected from capillary samples, using a high-quality blood glucose monitoring system.

Results:

Data from 31, 70, and 78 sensors of FBS, FL2, and ADX, respectively, were included in the performance analysis. The mean absolute relative differences between CGM and comparator data for FBS, FL2, and ADX were 14.7%, 9.2%, and 21.9%, and relative biases were −2.1%, −2.5%, and −18.5%, respectively. Analysis of individual sensor accuracy revealed low, moderate, and high sensor-to-sensor variability for FBS, FL2, and ADX, respectively. Sensor survival probabilities until the end of sensor life were 47.2% for FBS (28 days), 71.3% for FL2 (14 days), and 48.4% for ADX (14 days).

Conclusions:

The results of FBS were encouraging enough to conduct further performance and usability evaluations in a low- and middle-income country. The results of FL2 mainly agreed with existing studies, whereas ADX showed substantial deviations from previously reported results.

Introduction

The use of continuous glucose monitoring (CGM) technology in diabetes management has drastically increased in recent years due to the technology’s positive impact on glycemic control. ¹ However, to date, mostly people with diabetes living in high-income countries benefit from the technology, and implementation of CGM in low- and middle-income countries (LMICs) is lacking because of its relatively high costs. ² To accelerate the availability of CGM technology in LMICs, FIND, the global alliance for diagnostics, launched a project to identify promising new devices in development to conduct an independent performance evaluation. ³ For that, FIND published a request for proposal through its website, social media, and shared it with companies developing potential products. ⁴ Requirements for those products included the previous demonstration of basic performance and safety, a planned product launch in the next two years, and a lack of regulatory approval. The first phase of the FIND project was a clinical CGM performance study, evaluating the accuracy, reliability, and safety of the products in adults with type 1 diabetes in a high-income country setting, the results of which are reported in this article.

The proposals submitted by manufacturers went through the FIND review process, conducted by a team of four reviewers (two external and two internal reviewers). The proposals were assessed and partner manufacturers were selected in a process that was designed to be objective, independent, and transparent. At the end of this process, one device, the FiberSense CGM system (FBS; EyeSense GmbH, Großostheim, Germany),^5,6 was chosen for the initial performance study. As the results of such studies can be highly dependent on study design, and to facilitate the interpretability of results, two market-approved devices were tested in parallel. In particular, the widely adopted FreeStyle Libre 2 CGM system (FL2; Abbott Diabetes Care, Alameda, CA, United States) and the novel GlucoRx AiDEX CGM system (ADX; MicroTech Medical Co., Ltd., Hangzhou, China) were chosen because of their comparatively low costs in high-income settings and a lack of manufacturer-independent performance assessment in the scientific literature.^{7 -9}

Methods

Study Design

This prospective, explorative, open-label, monocenter clinical study was performed following the Declaration of Helsinki, Good Clinical Practice, and local laws and regulations at the Institut für Diabetes-Technologie, Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm in Ulm, Germany, between July and October 2022. The study protocol was approved by the responsible ethics committee at the Landesärztekammer Baden-Württemberg, and all participants gave written informed consent prior to any study procedures.

Participants aged ≥18 years with clinically diagnosed type 1 diabetes for at least one year and an HbA1c <10% were included if none of the exclusion criteria were met. The main exclusion criteria were hypoglycemia unawareness or severe hypoglycemic events within three months prior to study start, severe illnesses or infectious diseases, pregnancy or breast feeding, and skin issues.

In accordance with the explorative nature of this study and the examination of an unapproved device, a study population of 30 participants was deemed to be an adequate sample size.

Role of Funding Source

FIND, through funding support from Bundesministerium für Bildung und Forschung (BMBF) and Kreditanstalt für Wiederaufbau (KfW), supported the selection and procurement of study devices; the study design, evaluation, and reporting were done in concordance with the named FIND authors of this study.

Study Devices

This clinical performance evaluation was conducted on the FiberSense CGM system (FBS; EyeSense GmbH, Großostheim, Germany) as primary investigational device, as well as the FreeStyle Libre 2 CGM system (FL2; Abbott Diabetes Care, Alameda, CA, United States) and the GlucoRx AiDEX CGM system (ADX; MicroTech Medical Co., Ltd., Hangzhou, China) as secondary investigational devices. The three systems were used by all participants simultaneously.

The FBS has a specified sensor runtime of up to 28 days and makes use of a photometric glucose measurement principle. ⁵ The sensor is connected to a separate and reusable transmitter, which had to be disconnected and recharged multiple times during use. During the study, the transmitter transferred the raw sensor data to a smartphone running a dedicated application for data recording, with the real-time display of glucose levels switched off. After the study had ended, the glucose levels were generated from the recorded data by the manufacturer. For that, the manufacturer was provided with the raw data and specific glucose measurements for calibrations: one measurement per day in the morning, as well as one and two hours after sensor insertion, and one after each recharging of the transmitter. According to the manufacturer, the algorithm was applied prospectively, thus simulating user-entered, real-time calibrations. In accordance with the requirements of FIND, the system was not Conformité Européenne (CE)-marked at the time of study conduct and all devices used in the study were provided by the manufacturer through FIND.

Both FL2 and ADX have a specified sensor runtime of up to 14 days, are CE-marked, based on an enzymatic measurement principle, and employ factory-calibration. In case of ADX, the transmitter is reusable and user-entered calibrations are possible, but this feature was not used by the participants during the study. The FL2 were acquired from the manufacturer. The ADX were procured from its European distributor GlucoRx (GlucoRx Ltd., Guildford, UK).

All systems were used according to their respective manufacturer’s instructions and no therapeutic decisions were based on their measurements. While the sensors of all three systems were worn on the upper arm, FBS and ADX sensors can also be worn on the abdomen. Furthermore, the study staff received specific training on device use by the manufacturers personally or by online training.

Comparator Measurements

For the assessment of CGM performance, capillary blood comparator measurements were performed with the glucose dehydrogenase-based blood glucose monitoring system (BGMS) Contour Next One (CNO; Ascensia Diabetes Care GmbH, Basel, Switzerland). The BGMS measurements were performed in duplicate based on a specific schedule (see “Study Procedures” section) and the characteristics of their distribution were analyzed. To assess the compliance of the BGMS with recently proposed analytical performance specifications for glucose comparator measurements in CGM performance studies, ¹⁰ up to five capillary reference measurements with a hexokinase-based laboratory analyzer (Cobas Integra 400 plus, Roche Diagnostics GmbH, Mannheim, Germany) were performed in every participant throughout the study period. The accuracy of the laboratory analyzer was validated through regular external and internal quality checks as well as multiple measurements of glucose reference material of higher order at four concentrations (SRM 965b, National Institute of Standards and Technology, Gaithersburg, MD, USA) throughout the study period.

The participants were instructed to scan the FL2 sensor with every comparator blood glucose (BG) measurement to obtain a scanned measurement value.

Study Procedures

The experimental part of the study had a duration of 28 days, divided into 23 home-use days and five in-clinic days, spread over three distinct sessions, where the majority of time was spent under supervision at the study site. At the beginning of the study, participants were physically examined and demographic data were recorded. Furthermore, participants were trained on device use by the study staff and, throughout the study, participants followed their current therapy.

On the first study day, the participants applied one sensor of each device to their upper arms (side at their preference) under the supervision of study staff. On study days, 2, 11, and 28, which were spent in-clinic, a 5-hour frequent sampling period (08:00-13:00 hours) was scheduled, which involved capillary BG measurements every 15 ± 5 minutes after consumption of a standardized breakfast. After that, BG measurements were performed every 30 minutes until 17:00 hours. During the remaining two in-clinic days, which were partially spent at the study site, measurements were performed every 30 minutes (day 12, 08:00-12:00 hours) or every hour (day 1, 10:00-22:00 hours). On home-use days, participants were instructed to perform BGMS measurements at least eight times a day or more: after waking up, before and one hour after each main meal (breakfast, lunch, and dinner), and before going to bed. The FBS sensors could be replaced if they failed within the first two study days. To cover the complete study duration of 28 days and maximize the time during which all three systems were used simultaneously, sensors of FL2 and ADX could be replaced after their runtime had ended or if a sensor failure occurred. The participants were allowed to replace the failed sensors of FL2 and ADX themselves at the physician’s discretion. The safety of the CGM systems was assessed by documenting adverse events related to the examined CGM systems.

Data Analysis

The CGM data affected by deviations from the study protocol or device deficiencies were excluded from the performance analysis. Furthermore, comparator BGMS measurements were excluded if the second value in a duplicate differed by more than ±10 mg/dL or more than ±10% from the first one, whichever was larger.

Accuracy evaluation was based on paired CGM-comparator data points. In accordance with the data recording intervals of FBS and ADX of two and five minutes, respectively, only valid CGM values recorded simultaneously or within the following one and four minutes, respectively, of a valid BGMS measurement were paired for subsequent performance analyses. In accordance with a previous publication assessing FL2 performance, ⁸ only results from scanned measurements were used for the analysis. Here, values were paired if they were obtained within ±3 minutes of a valid BGMS measurement.

CGM point accuracy was assessed using the recently proposed Continuous Glucose Deviation Interval and Variability Analysis (CG-DIVA), which characterizes the deviations of CGM measurements in different BG ranges as well as the variability in accuracy between sensors of the same CGM system. ¹¹ Furthermore, the mean absolute relative differences (MARDs) and agreement rates (ARs) between CGM and comparator pairs were calculated. In particular, the ARs were calculated as the percentage of CGM values within ±15%/20%/40% of the comparator BG value for BG levels ≥100 mg/dL or within ±15/20/40 mg/dL for BG levels <100 mg/dL. ¹² Two-sided 95% confidence intervals (CIs) of MARDs and ARs were determined using a recently proposed clustered bootstrap method. ¹³

The sensor stability was assessed by calculating MARD values on each study day, limited to sensors inserted on the first study day. This was done to ensure that the same conditions, in particular duration and order of home and in-clinic use, were the same within each of the CGM systems.

Trend accuracy, that is, the ability of a CGM system to correctly indicate the rate of change (RoC) of BG levels, was assessed using the rate error grid analysis (R-EGA). ¹⁴ The R-EGA assigns every pair of comparator and CGM RoCs to risk zones A to E, depending on their difference. Very similar to the error grid analysis for point accuracy, zones A and B of the R-EGA indicate very small or benign errors. Zone C indicates rapid CGM RoCs but slow BG RoCs, which could lead to overcorrection in insulin dosing or food intake; zone D indicates rapid BG RoCs that are not detected by the CGM system; and zone E indicates CGM RoCs that are opposite to the comparator RoC. The comparator RoCs were calculated from subsequent BGMS measurements during the frequent sampling periods (time difference ≤20 min) and the corresponding CGM RoCs were determined using a linear regression approach recommended in the POCT05 guideline. ¹²

The CGM sensor survival was analyzed using the Kaplan–Meier approach, ¹⁵ whereas CGM system failures were classified into device malfunctions, adhesive failures, and user errors (not counted as failures).

Results

Participant Population and Data Exclusions

In this study, 30 adult participants with type 1 diabetes were included and all participants completed the study. One sensor of FBS was replaced on study day 2; for FL2 and ADX, a maximum of four sensors were worn consecutively by the same participant. Detailed demographic information on the participant population is provided in Table 1.

Table 1.

Baseline Participant Characteristics (n = 30).

Demographic	Value
Sex, n (%)
Men	12 (40)
Women	18 (60)
Diagnosis, n (%)
T1DM	30 (100)
Therapy, n (%)
MDI	13 (43)
CSII	17 (57)
Age (years)
Mean (SD)	54.8 (12.6)
BMI (kg/m²)
Mean (SD)	27.3 (3.0)
HbA1c (%)
Mean (SD)	6.7 (0.6)
Duration of diabetes (years)
Mean (SD)	28.8 (14.9)

Abbreviations: T1DM, Type 1 diabetes mellitus; MDI, multiple daily injections; CSII, continuous subcutaneous insulin infusion; BMI, body mass index.

Of more than 10 000 comparator measurements, a total of 199 were excluded as they contained only single measurements or the duplicates deviated by more than ±10% or ±10 mg/dL. In addition, comparator measurements used for calibration of FBS were excluded from the accuracy analysis of FBS. Protocol deviations led to the exclusion of data from one complete ADX sensor (wrong insertion site) and approximately 18 days of data from one FBS sensor (participant went swimming). Furthermore, data were excluded from one FBS sensor after it had detached and from one ADX sensor after a receiver malfunction.

Comparator Measurements

The BGMS measurements with Contour Next One (CNO) showed a mean relative bias of +2.2% with respect to the laboratory analyzer (n = 142). Based on the averaged results from all duplicate measurements with CNO (n = 10 278), its precision could be estimated with a coefficient of variation (CV) of 1.8%. ¹⁶ The CNO thus fulfills the “desirable” analytical performance specifications (bias <2.4%, imprecision (CV) <2.5%). ¹⁰

On five days throughout the entire study duration, measurements with material of higher order were also performed with the laboratory analyzer (concordant with the days of capillary blood sample assessment on the laboratory analyzer). Here, the mean relative bias of the laboratory analyzer with respect to the four nominal glucose levels of the higher order material was −0.4% (n = 20) and CVs for each level ranged from 1.1% to 2.0%, thus fulfilling “desirable” analytical performance specifications.

As the accuracy of a CGM system can be dependent on BG levels and their RoC, it is important to characterize the distribution of comparator measurements, as provided in Figure 1a. In addition, it was possible to estimate the true BG level RoC from comparator measurements during the frequent sampling periods. The respective RoC distribution is displayed in Figure 1b. The mean absolute BG RoC was 1.06 mg/dL/min.

Figure 1.

(a) Distribution of comparator blood glucose (BG) levels collected from all participants over the entire study period. (b) Distribution of comparator glucose rate of change (RoC) calculated for BG measurements spaced less than 20 minutes during frequent sampling periods on study days 2, 11, and 28.

Point Accuracy

The results of the CG-DIVA are provided in Figure 2. The plots in Figure 2d-f, indicating sensor-to-sensor variability, only display a selection of 20 representative sensors, including the 10 sensors with the most positive and negative median deviation in the total BG range. Overall median biases were −2.1%, −2.5%, and −18.5% for FBS, FL2, and ADX, respectively. More detailed numerical results of the CG-DIVA are provided in the supplemental material (Supplemental Tables S1 and S2). The results of ARs and MARDs are provided in Table 2.

Figure 2.

Results of the Continuous Glucose Deviation Interval and Variability Analysis (CG-DIVA). Absolute deviations are provided for comparator glucose levels <70 mg/dL and relative deviations for all other levels. Panels a-c show the deviation intervals as light/dark gray boxes, which indicate the range in which a certain share of deviations are expected to lie. These shares (printed above the plot) and the colored background were set according to the requirements defined by the U.S. Food and Drug Administration for “integrated” CGM systems. If the second deviation interval (light gray box) extends beyond the y-axis, its upper/lower limit is printed. The median deviations are shown by the black dashes. Panels d-f show the characterization of the sensor-to-sensor variability on a representative selection of 20 sensors. Each sensor is described by its median and 90%-range of deviations and sensors are ordered according to the median deviation in the total glucose range. If less than 10 data points within a range are available for a sensor, the full range of deviations is displayed and indicated with caps at the end of the antennae. Note that some antennae extend beyond the y-axis limits.

Abbreviations: FBS, FiberSense CGM system; FL2, FreeStyle Libre 2 CGM system; ADX, GlucoRx AiDEX CGM system.

Table 2.

Accuracy Parameters.

CGM system	n (data points)	n (sensors)	AR %15/15a,b [%]	AR %20/20a,b [%]	AR %40/40a,b [%]	MARDa,c [%]
FBS	6437	30	65.4 (62.6 - 68.1)	77.2 (74.7 - 79.4)	96.0 (94.9 - 96.9)	14.7 (13.9 - 15.5)
FL2	8577	68	84.9 (82.6 - 87.2)	92.3 (90.3 - 94.1)	98.6 (97.2 - 99.7)	9.2 (8.4 - 10.1)
ADX	8013	76	37.2 (30.9 - 43.4)	51.2 (44.6 - 57.8)	90.9 (86.0 - 95.4)	21.9 (19.5 - 24.4)

Abbreviations: FBS, FiberSenseGM system; FL2, FreeStyle Libre 2 CGM system; ADX, GlucoRx AiDEX CGM system.

^aThe numbers in brackets indicate the two-sided 95% confidence intervals.

^bAgreement rates (ARs) are calculated as the percentage of CGM values within ±15%/20%/40% of comparator glucose measurements when glucose levels are ≥100 mg/dL or within ±15/20/40 mg/dL when glucose levels are <100 mg/dL.

^cMean absolute relative difference (MARD) between CGM and comparator measurements.

Sensor Stability

The results of the sensor stability analysis, considering only data from sensors that were inserted on study day 1, are provided in Figure 3.

Figure 3.

Sensor stability as characterized through mean absolute differences (MARD) on every study day including only sensors inserted on study day 1. The shaded areas indicate the 95% confidence intervals. The circled points indicate days that were at least partially spent at the study side; red circles indicate days with frequent sampling periods.

Abbreviations: FBS, FiberSense CGM system; FL2, FreeStyle Libre 2 CGM system; ADX, GlucoRx AiDEX CGM system.

Trend Accuracy

The results of the trend accuracy analysis using R-EGA are provided in Table 3. The corresponding figures are provided in the supplemental material (Supplemental Figure S1).

Table 3.

Results of the Rate Error Grid Analysis.

Zone	FBS	FL2	ADX
A	76.3% (941)	77.4% (1127)	75.7% (1084)
B	19.4% (239)	18.5% (270)	18.6% (267)
C	1.5% (19)	2.5% (36)	0.8% (11)
D	2.3% (28)	1.4% (21)	4.7% (68)
E	0.5% (6)	0.3% (4)	0.1% (2)
A + B	95.7% (1180)	95.9% (1397)	94.3% (1351)
Total	1233	1457	1432

Zones A + B: very small or benign errors. Zone C: rapid CGM RoCs, slow BG RoCs. Zone D: rapid BG RoCs, slow CGM RoCs. Zone E: CGM RoCs of opposite direction.

Abbreviations: FBS, FiberSense CGM system; FL2, FreeStyle Libre 2 CGM system; ADX; GlucoRx AiDEX CGM system; RoC: rate of change.

Sensor Survival and Failure

In total, 31, 70, and 78 sensors of FBS, FL2, and ADX, respectively, were included in the Kaplan–Meier survival analysis and the results are depicted in Figure 4. The estimated survival probabilities (95% CI) were 47.2% (28.9% - 63.5%) for FBS (28 days), 71.3% (57.0% - 81.6%) for FL2 (14 days), and 48.4% (35.9% - 59.9%) for ADX (14 days). More detailed information on sensor survival can be found in the supplemental material (Supplemental Figure S2).

Figure 4.

Results of the survival analysis. The shaded areas indicate the 95% confidence intervals. The vertical dashes indicate sensors that were removed from participants prior to the end of their lifetime for reasons not related to the device.

Abbreviations: FBS, FiberSense CGM system; FL2, FreeStyle Libre 2 CGM system; ADX, GlucoRx AiDEX CGM system.

In terms of failures, a total of 16 FBS sensors failed: five due to technical malfunction of the device and 11 due to a failure of the adhesive leading to sensor loss. For FL2, there were a total of 17 sensor failures, nine due to technical malfunction and eight due to adhesive failure. For ADX, there were a total of 37 sensor failures: 24 due to technical malfunction and 13 due to adhesive failure.

Safety Assessment

For FBS, there were a total of four adverse events related to the device: two skin reactions (eg, redness or itching), one hematoma, and one short bleeding. The FL2 was related to two adverse events: one strong bleeding after sensor insertion that led to sensor removal and one hematoma. The ADX was related to nine adverse events: four skin reactions including minor swelling, four hematomas of which one was preceded by short bleeding after sensor insertion that led to sensor removal, and one pain sensation. All hematomas were small and of similar size and all adverse events were rated as mild.

Discussion

In this study, the performance of the nonmarket-approved FBS CGM system and the two market-approved CGM systems, FL2 and ADX, was assessed. Despite the fact that all three systems were tested in parallel, a direct comparison of their performance is not carried out and the results of each system are discussed separately. However, the following points regarding the impact of the study protocol on the results of all systems should be mentioned. First, the protocol did not involve deliberate glucose manipulations to induce extreme BG levels and fast glucose changes. This could have affected the results as the accuracy of CGM systems tends to decrease during phases of rapid BG-level change and hypo- and hyperglycemic episodes. ¹⁷ Second, the study was conducted partially during warm temperatures in the summer months, which could have facilitated sensor losses in all devices due to adhesive failure. Furthermore, the fact that participants spent the vast majority of the study unsupervised at home impaired the precise documentation of device deficiencies, in particular sensor failures and their causes. Finally, the validation of the comparator device demonstrates that a high-quality BGMS is suitable to provide comparator measurements of adequate accuracy in CGM performance studies. Another general aspect of this study is the utilization of the recently introduced CG-DIVA because it can provide a comprehensive characterization of CGM point accuracy, including BG range-specific bias, imprecision, and sensor-to-sensor variability.

In terms of point accuracy, the CG-DIVA of the FBS data (Figure 2a) shows a low overall systematic bias. However, if the bias is examined separately in each glucose range, it becomes apparent that FBS generally overestimated BG levels in the hypoglycemic range (<70 mg/dL) and underestimated BG levels in the hyperglycemic range (>180 mg/dL). In accordance with the manual calibrations of the device, the sensor-to-sensor variability is low, with a rather high within-sensor variability (Figure 2d), causing a generally high imprecision as indicated by the extent of deviation intervals in Figure 2a. This imprecision could be at least partially caused by the comparatively high data recording rate of two minutes. The overall MARD of 14.7% in Table 2 is considerably higher than the value of 8.3% reported in 2013 from a study with an earlier generation FBS system and a different experimental protocol. ⁵ However, a more recently published abstract reported a MARD of 15.0%, ⁶ which is in agreement with this study. The accuracy is therefore comparable to earlier generation CGM systems, such as Dexcom G4 and FreeStyle Navigator II. ¹⁷ However, the FBS has a considerably longer lifetime of 28 days and shows no deterioration in accuracy with time, although the device demonstrates below average accuracy in the first five days (Figure 3), indicating a longer stabilization period. Excluding the first five days of sensor wear time reduces the MARD to 13.1% (more details in Supplemental Table S3). The survival rate of FBS sensors stabilized after day 12 until a drop in the survival probability occurred between days 18 and 22, which was mainly caused by adhesive failures. Given the fact that the FBS was still in development at the time of testing, it can be expected that its performance will be improved.

For FL2, overall point accuracy, sensor survival, and sensor stability results agree with previously published results (MARD 9.2% in adults) obtained with a different study design featuring venous comparator measurements and only data from in-clinic sessions with deliberate glucose manipulations. ⁸ The CG-DIVA demonstrates low overall and BG range-specific bias and imprecision (Figure 2b). Typical for factory-calibrated devices, the sensor-to-sensor variability is fairly pronounced and there is low within-sensor variability (Figure 2e). The CG-DIVA also revealed that there were three sensors (4.4%) with an overall median deviation larger than approximately −20% and up to −60%, which causes the 99% deviation intervals for BG levels above 70 mg/dL to be asymmetric toward negative deviations. This is a new finding as the previous publication did not include an examination of sensor-specific accuracy ⁸ and highlights the importance of such an analysis, in particular for factory-calibrated systems. Trend accuracy was also not reported previously and the results in Table 3 indicate a slightly increased number of data points in zone C, indicating falsely high CGM rates of change that could lead to overcorrection. Despite the comparatively long market-availability of FL2, and to the best of our knowledge, this is the first manufacturer-independent performance evaluation of this widely used CGM system.

For ADX, a previous study reported an overall MARD of 10.1% using capillary comparator measurements, ⁷ whereas our study found a MARD of 21.9%. The CG-DIVA results (Figure 2c) indicate a considerable overall negative median bias of −18.5% and similar negative biases for comparator values between 70 and 180 mg/dL and above 180 mg/dL. In contrast, there is a positive bias of +7.9 mg/dL for BG levels below 70 mg/dL. There is also a considerable imprecision, which is mainly caused by a high sensor-to-sensor variability, with individual sensors showing an overall median bias ranging between −56.5% and +47.2%. This high sensor-to-sensor variability also causes a comparatively large uncertainty (large CIs) of MARD and AR point estimates as shown in Table 2 as well as the stability analysis as seen in Figure 3, impairing its interpretability. The estimated survival probability of 48.4% was considerably lower than the previously reported value above 90%. ⁷ In this study, the failures occurred regularly throughout the 14-day lifetime, and the majority of early failures could be attributed to a device malfunction. Regarding trend accuracy, there was a noticeable accumulation of data points in zone D (Table 3), indicating falsely low CGM rates of change.

The reason for the differences in point accuracy and survival in ADX between this study and the previously reported results is unclear; however, the previous study included a majority of participants (88.7%) with type 2 diabetes, which could have led to slower BG changes and thus better accuracy. ⁷ It should also be emphasized that the ADX can be calibrated by the users but this feature was apparently not used in the previous study. Furthermore, the ADX is only approved for adjunctive use, meaning that users are required to rely on other means of glucose concentration measurement for therapy decisions, and the user manual suggests calibration in case of larger deviations to BGMS measurements. ¹⁸ The fact that this suggestion was not followed by participants is a limitation of this study and the accuracy improved when a single manual calibration at the beginning of sensor wear time was simulated retrospectively (MARD of 15.5%, more details in section 3 of the supplemental material). In addition, it should be mentioned that 21 out of 30 participants used transmitters with an updated firmware, which led to an improved accuracy in comparison with the remaining participants (MARD of 21.0% vs 24.3%; more details in section 4 of the supplemental material). It can therefore be expected that future updates to the system may lead to further improvements in accuracy.

Conclusion

This study shows that independent performance evaluations of CGM systems are critical to gain additional insights into device performance at the time of development, as well as after market approval. In LMICs, cost is often a key decision factor as CGM systems are largely financed through out-of-pocket spendings. Independent performance data are thus critical to support informed decision-making for investment of scarce resources for accurate and reliable devices.

Our study shows that the in-development FBS demonstrated promising accuracy results, comparable to earlier generation CGM, ¹⁷ with the added advantage of a 28-day sensor lifetime. Results of the established, market-approved FL2 largely agreed with the previous study ⁸ ; however, the importance of assessing individual sensor accuracy was highlighted. For the newer, market-approved ADX, the results were markedly different in comparison with a previous study ⁷ and indicated lower performance. However, the comparability of performance results between different studies is generally limited because of a lack in standardization of CGM performance studies, especially with regard to study design.

This study highlights that people with diabetes, clinicians, and purchase decision-makers in LMICs intending to introduce CGM systems in local diabetes management should consider consulting data from independent performance analyses prior to making decisions about implementation.