Performance of a new real‐time continuous glucose monitoring system: A multicenter pilot study

Abstract

Aims/Introduction

The present study aimed to investigate the performance of a new real‐time continuous glucose monitoring system.

Materials and Methods

Interstitial glucose levels were monitored for 7 days in 63 patients with type 1 or type 2 diabetes using the Medtrum A6 TouchCare^® CGM System. Venous blood was collected on a randomized day of the wear period. Plasma glucose levels were measured as reference values.

Results

Among 1,678 paired sensor–reference values, 90.5% (95% confidence interval 89.1–91.9%) were within ±20%/20 mg/dL of the reference values, with a mean absolute relative difference of 9.1 ± 8.7% (95% CI: 8.9–9.2%). The percentages of paired sensor–reference values falling within zone A and B of the Clarke error grid analysis (EGA) and the type 1 diabetes consensus EGA were 99.1 and 99.8%. Continuous EGA showed that the percentages of accurate readings, benign errors, and erroneous readings were 89.9, 6.3 and 3.8%, respectively. Surveillance EGA showed that 90.6, 9.2, and 0.2% of sensor–reference values with no, slight and lower moderate risk, respectively. The mean absolute relative difference was 16.6, and 96.0% of the sensor values fell within zones A and B of the consensus EGA for hypoglycemia. More than 85% of sensor values were within ±20%/20 mg/dL of reference values, the mean absolute relative difference was <11, and >99.5% of the sensor values fell in zones A and B of the consensus EGA.

Conclusions

The Medtrum real‐time continuous glucose monitoring system was numerically and clinically accurate over a large glucose range across 7 days of wear.

Introduction

An effective method for monitoring blood glucose levels is critical to successful management of diabetes. Traditional methods, such as self‐monitoring of capillary blood glucose, only provide intermittent blood glucose levels, which might not show all the hyperglycemic and hypoglycemic episodes or glucose variability. To overcome such limitations, continuous glucose monitoring (CGM) has been developed as a novel glucose monitoring method that provides 24 h continuous glucose data. The CGM technology is divided into two categories: retrospective and real‐time (RT). A retrospective CGM system can provide retrospective data of glycemic patterns, but does not provide glucose data for immediate adjustment in treatment under various situations in daily use. A RT‐CGM system not only provides instant information about the patient's glucose levels, but also sends high/low glucose alerts as well as rate of change alerts. Previous clinical studies showed that CGM contributed to the control of blood glucose levels, a reduced occurrence of hypoglycemia, lower levels of glycated hemoglobin A_1c (HbA_1c), as well as a decreased risk of diabetes complications1, 2, 3.

The accuracy of sensor readings is a critical factor for the clinical application of CGM, and the current gold standard assessment method in terms of accuracy is measurement of the glucose levels in venous blood using the Yellow Springs Instrument (YSI; YSI Life Sciences, Yellow Springs, Ohio, USA)4. The commonly used methods for evaluating the accuracy of a CGM system include bias analysis, agreement analysis and error grid analysis (EGA)5, 6, 7, 8. There are three major producers of commercial CGM systems: Medtronic MiniMed (Northridge, California, USA), Dexcom (San Diego, California, USA) and Abbott Diabetes Care (Alameda, California, USA), and numerous clinical trials were carried out to investigate the performance of these CGM systems9, 10, 11, 12. Furthermore, there were articles that compared the performance of these CGM systems head‐to‐head13, 14, 15. Recently, Medtrum Technologies, Inc. developed a new RT‐CGM system, Medtrum A6 TouchCare^® CGM System, which uses a small, soft and transcutaneous glucose oxidase‐based electrochemical glucose sensor (MD‐JY‐006) to detect glucose levels in the interstitial fluid every 2 min over 7 days. The present multicenter study was carried out to investigate the performance of the new RT‐CGM system.

Methods

Patients

The present study was a multicenter, prospective, randomized study. A total of 63 participants who were treated between March 2015 and May 2016 at three sites in Shanghai were included. The sample size was equal at each site. The inclusion criteria included: (i) aged 18–70 years; (ii) having a clinical diagnosis of type 1 or type 2 diabetes for ≥3 months; and (iii) having not participated in any clinical study for the last 3 months. The exclusion criteria included: (i) pregnancy, psychosis, immunosuppressive disorders or systemic neurological disease; (ii) diabetic ketoacidosis or non‐ketotic hyperosmolar syndrome and other acute complications; (iii) other diseases with a life expectancy of no more than 1 year; (iv) severe allergies; (v) severe circulatory disturbance; and (vi) a history of adhesive tape allergy.

Ethics

The present study was carried out in accordance with the Declaration of Helsinki, and independently approved by the ethics committee of each participating hospital (Shanghai Jiao Tong University Affiliated Sixth People's Hospital; Huashan Hospital, Shanghai Medical College, Fudan University; and Shanghai Tenth People's Hospital, School of Medicine, Tongji University). All participants provided written informed consent before study initiation.

The Medtrum RT‐CGM system was Conformité Européene marked in 2016 with the notification body as TUV Rheinland, and it has been launched on the European market. The Medtrum RT‐CGM system has not received Food and Drug Administration approval yet.

This clinical study was carried out in order to obtain Chinese Food and Drug Administration approval for this new CGM system, and therefore all the requirements of a multicenter clinical study were strictly followed, whereas the current study does not have a registration number.

Measurement procedure

All the patients wore the MD‐JY‐006 glucose sensor of the RT‐CGM system of Medtrum Technologies, Inc. (Figure S1, www.medtrum.com) for seven successive days. The A6 touchcare CGM system uses glucose oxidase based an electrochemical glucose sensor and one‐point calibration algorithm. This CGM system needs to be calibrated at an interval of 12 h. The Medtrum RT‐CGM system has a small flexible sensor, automatic sensor insertion and a small‐sized introducer needle (26‐G). During the monitoring period, fingertip blood glucose levels were tested using the One Touch UltraEasy blood glucose meter (LifeScan Inc., Milpitas, California, USA) at least four times daily, and two of the blood glucose measurements were inputted into the CGM system for calibration at intervals of 12 h. On a randomized day of the 7‐day wear period, patients participated in a venous blood glucose test in the clinic that lasted for 7 h with venous blood collected every 15 min, and the plasma glucose levels were measured by the YSI 2300 STAT PLUS analyzer (YSI Life Sciences) to serve as the reference values. Any adverse events were noted during the period of blood glucose monitoring.

Data collection

All the data from the study were collected using Oracle Remote Data Capture (Oracle Corp., Redwood, California, USA). The electronic data stored in the CGM system were exported.

Effectiveness analysis

The plasma glucose levels measured by the YSI system were used as reference values, and each reference value was paired with the corresponding CGM system sensor reading (sensor value). The primary analysis determined the agreement between the sensor values and reference values at a deviation of ±20% or ±20 mg/dL, which was the percentage of sensor values that fell within either ±20 mg/dL of the reference values for glucose concentrations <100 mg/dL or within ±20% for glucose concentrations ≥100 mg/dL. The glucose concentration limit of 100 mg/dL was chosen according to the newest standard for in vitro blood glucose monitors, ISO 15197:201316. The statistically critical value for agreement at the deviation of ±20%/20 mg/dL was 85%, according to clinical results of the CGM system in China17.

The secondary analysis included the following statistical methods. (i) EGA, such as Clarke EGA5, consensus EGA6, continuous EGA7, and the recently developed surveillance EGA8. (ii) Bias analysis, such as mean, median, standard deviation, standard error, and 95% confidence interval (95% CI) of difference, relative difference, absolute difference and absolute relative difference (ARD). Referring to the new standard of ISO 15197:201316, the ARD is the absolute difference between the sensor value and the reference value at glucose concentrations (reference value) <100 mg/dL or the ratio of the absolute difference between the sensor value and the reference value to the reference value at glucose concentrations ≥100 mg/dL. (iii) Correlation analysis, a descriptive parameter, was used to assess the correlation between the sensor value and reference value. (iv) Linear models were used to assess the bias between the sensor and reference values. (v) Bland–Altman analysis was carried out to obtain a scatter plot based on the reference value (horizontal coordinate), and the difference between paired sensor–reference values (vertical coordinate) was used to assess the relationship between the bias and blood glucose level.

The sample size selected was based on the primary effectiveness analysis. A sample of 63 adult patients provided a total of 252 paired measures for each of the seven testing days, producing a total of 1,764 paired measurements.

Safety analysis

Descriptive statistics were used to describe the safety events. Adverse events were monitored each day. All moderate‐to‐serious adverse events associated with the device or operation were reported to the sponsors through an electronic case report form. All serious adverse events and unexpected device‐related adverse events were also reported to the sponsors through the electronic case report form. In addition, skin samples of the patients who wore the sensors were evaluated.

Statistical analysis

All data were entered into Excel (Microsoft Corp., Redmond, Washington, USA), and all statistical analyses were carried out using the SAS version 9.2 (SAS Institute, Cary, North Carolina, USA).

Results

Clinical characteristics of study participants

In total, 63 adult participants (35 men and 28 women; median age 59 years, age range 24–70 years) were enrolled in the present clinical study, including 10 patients with type 1 diabetes and 53 patients with type 2 diabetes. The median diabetes duration was 10 years, ranging 1–32 years. At baseline, participants had an HbA_1c of 8.2 ± 1.7%. The body mass index (BMI) was 24.7 ± 3.0 kg/m², and waist circumference was 85 ± 9 cm.

Agreement analysis

In total, 1,678 paired sensor–reference values were collected from 60 participants, whereas no paired sensor–reference values were obtained from the other three patients. The results of agreement analysis are shown in Table 1. Among the 1,678 sensor values, 90.5% (95% CI: 89.1–91.9%) were within ±20%/20 mg/dL of the reference values, which met the expected accuracy. The percentages of sensor values that met the ±10%/10 mg/dL, ±15%/15 mg/dL, ±30%/30 mg/dL, and ±40%/40 mg/dL deviation criteria were 65.7, 81.5, 96.9 and 98.9%, respectively.

Table 1.

Agreement between paired sensor–reference values in the range of reference glucose levels

Agreement level	±10%/10 mg/dL	±15%/15 mg/dL	±20%/20 mg/dL	±30%/30 mg/dL	±40%/40 mg/dL
Total	65.7	81.5	90.5	96.9	98.9
≤70	24.0	36.0	72.0	96.0	100.0
71–180	61.2	78.1	88.2	96.3	99.0
>180	70.9	85.9	93.0	97.5	98.9

Bias analysis

The results of bias analysis are shown in Table 2. For the 1,678 sensor values, the ARD was 9.1 ± 8.7% (95% CI: 8.9–9.2%), and the median ARD was 6.7%. For the calculation of the ARD, the range of blood glucose levels was stratified.

Table 2.

Bias analysis between paired sensor–reference values

Bias	D (mg/dL)	RD (%)	AD (mg/dL)	ARD (%)
Mean	−4.6	−1.5	16.8	9.1
Median	−2.0	−1.3	13.0	6.7
SD	23.1	13.0	16.4	8.7
SE	0.1	0.1	0.1	0.1
95% CI	− 4.9 to − 4.4	− 1.7 to − 1.3	16.6 to 17.0	8.9 to 9.2

AD, absolute difference; ARD, absolute relative difference; CI, confidence interval; D, difference; RD, relative difference; SD, standard deviation; SE, standard error.

Clarke EGA

Clarke EGA was carried out on 1,678 paired sensor–reference values. The scatter plot was created in the Clarke error grid with the reference value as the horizontal coordinate and the sensor value as the vertical coordinate (Figure 1a). The results showed that 99.1% of the paired sensor–reference values fell within zones A and B (89.7% within zone A and 9.4% within zone B). Just 15 (0.9%) paired sensor–reference values fell within clinical risk zone D, which represents ‘dangerous failure to detect and treat’ errors. No paired sensor–reference values fell within zone C or E.

Figure 1.

(a) Clarke error grid analysis and the (b) type 1 diabetes consensus error grid analysis of the paired sensor–reference values. Error grid analysis was divided into five zones of varying degrees of accuracy and inaccuracy of glucose estimations. The risk categories, in order of increasing severity, were defined as follows: A, no effect on clinical action; B, altered clinical action, or little or no effect on clinical outcome; C, altered clinical action – likely to effect clinical outcome; D, altered clinical action – could have significant medical risk; and E: altered clinical action – could have dangerous consequences. l, lower; u, upper.

Continuous EGA

Continuous EGA was carried out on 1,616 paired sensor–reference values excluding 62 paired values without corresponding rates of change. The continuous EGA was carried out in three steps: (i) rate‐EGA showed that the percentage of sensor–reference values in zones A and B was 90.8% (71.8% in zone A and 19.0% in zone B); (ii) point‐EGA showed that the percentage of sensor–reference values in zones A and B was 98.9% (91.8% in zone A and 7.1% in zone B); and (iii) the combined continuous error grid matrix of the rate‐EGA and the point‐EGA showed that the percentages of sensor–reference values in the zones of accurate readings, benign errors, and erroneous readings were 89.9, 6.3 and 3.8%, respectively.

Consensus EGA

On the type 1 diabetes consensus EGA, 99.8% of the paired sensor–reference values fell within zones A and B (89.6% within zone A and 10.3% within zone B; Figure 1b), which showed that <0.5% of the values produced by the Medtrum CGM system had risk likely to affect clinical outcome. Just three (0.2%) paired sensor–reference values fell within the altered clinical action zone C, and no paired sensor–reference values fell within zone D or E. According to the type 2 diabetes consensus EGA, 99.8% of the paired sensor–reference values fell within zones A and B (94.3% within zone A and 5.5% within zone B).

Surveillance EGA

As shown in Figure 2, 99.8% of the paired sensor–reference values had no or slight clinical risk (90.6% with no risk, 7.3% with lower slight risk and 1.9% with higher slight risk). Just three paired sensor–reference values (0.2%) had lower moderate risk (absolute risk value >1.5–2.0). No paired sensor–reference values had a higher clinical risk than lower moderate risk.

Figure 2.

Surveillance error grid analysis with risk scores.

Correlation analysis and linear regression

The correlation analysis showed significant positive correlations between the sensor values and reference values (r = 0.944, P < 0.001), and a linear regression equation fitted as y = 11.63 + 0.915x (P < 0.001). If the intercept was fixed at zero, the slope rose to 0.968, which meant that the sensor values were likely to be slightly underestimated. This characteristic was also revealed in surveillance EGA. As shown in Figure 2, there are slightly more values with a potential risk of causing hyperglycemia than those with a risk of causing hypoglycemia. The bias analysis also found a negative difference (−4.6 mg/dL) and relative difference (−1.5%) between the sensor values and reference values.

Bland–Altman analysis

The Bland–Altman analysis showed that the mean difference between the paired sensor–reference values was −4.6 mg/dL (95% CI: −4.9 to −4.4 mg/dL, Figure S2), and there was no obvious variation of the differences between the paired values at different blood glucose levels. A total of 81.5% of the sensor values were within ±15%/15 mg/dL of reference values.

Accuracy over a range of glucose concentrations

The results of the accuracy analysis for the paired sensor–reference values over a range of glucose levels are shown in Table 3. In the hyperglycemia range, the sensor values had the lowest ARD, and there were the most percentages of sensor values within the ±20%/20 mg/dL criterion, in zone A of the type 1 diabetes consensus EGA, and with no risk in the surveillance EGA. In the range of hypoglycemia, the mean absolute relative difference (MARD) was 16.6, and 72% of sensor values within the ±20%/20 mg/dL criterion. A total of 96% of the sensor values in the hypoglycemia range fell within the ±30%/30 mg/dL criterion, in zones A and B of type 1 diabetes consensus EGA, and in the no or slight clinical risk zones of surveillance EGA.

Table 3.

Numerical and clinical accuracy analysis of the paired sensor–reference values in the range of reference glucose levels

Range of reference values (YSI)	Hypoglycemia (YSI ≤ 70)	Euglycemia (70 < YSI ≤ 180)	Hyperglycemia (YSI > 180)	Total
Pairs	25	781	872	1,678
Bias
MD (mg/dL)	12.7	−0.9	−7.1	−4.6
Mean ARD (%)	16.6	9.9	8.1	9.1
Median ARD (%)	18.0	7.4	5.9	6.7
Agreement
Within ±20%/20 mg/dL	72.0	88.2	93.0	90.5
Consensus EGA
Zone A	76.0	86.2	93.0	89.6
Surveillance EGA
No risk	24.0	88.7	94.3	90.6
Agreement
Within ±40%/40 mg/dL	96.0†	99.0	98.9	98.9
Consensus EGA
Zones A and B	96.0	100.0	99.8	99.8
Surveillance EGA
None and slight risk	96.0	99.7	100.0	99.8

^†For glucose in hypoglycemia range, the agreement criterion of ±30%/30 mg/dL was chosen for clinical risk limitation. ARD, absolute relative difference; EGA, error grid analysis; MD, mean difference; YSI, Yellow Springs Instrument.

Stability analysis

The results of the agreement analysis, type 1 diabetes consensus EGA and MARD analysis between paired sensor–reference values across the wear duration are shown in Figure 3. On every day during the 7‐day wear duration, >85% of sensor values were within ±20%/20 mg/dL of reference values, the MARD was <11%, and more than 99.5% of the sensor values fell in zones A and B of the type 1 diabetes consensus EGA. The MARD was within 10%, except the first day and the last day. However, there was no significant difference among the MARDs for the 7 days tested by the analysis of variance.

Figure 3.

(a) Percentage of agreement, (b) percentage in consensus error grid analysis (EGA) zone, (c) mean absolute relative difference (MARD) (d) between paired sensor–reference values and the number of paired values across the wear duration. Agreement criteria were within ±20%/20 mg/dL, ±20%/20 to ±30%/30 mg/dL, ±30%/30 to ±40%/40 mg/dL and beyond ±40%/40 mg/dL. The consensus EGA zones were zone A (green), zone B (yellow) and zone C (brown).

Flexibility analysis

The results of the agreement analysis and the type 1 diabetes consensus EGA between paired sensor–reference values as a function of various factors or patient characteristics are shown in Figure S3. Sensor accuracy was not affected by factors such as type of diabetes, sex, therapy method, age, HbA_1c, diabetes duration, waist circumference or BMI. For all situations, the percentages of sensor values within ±20%/20 mg/dL of reference values were >87.3%, and the percentages of sensor values in zone A were >86.5%. The clinical risk was not affected by these factors or patient characteristics, and the percentages of sensor values within zones A and B and within ±40%/40 mg/dL of reference values were all greater than 99 and 97.5%, respectively. These percentages were slightly lower for patients with a BMI >28 kg/m², whereas the percentage of sensor values within zone A of the consensus EGA was the greatest for patients with a BMI >28 kg/m².

Consistency analysis

In total, 63 participants wore 126 sensors. A total of 71.4% of these sensors worked on day 7, and the median number of readings provided by each sensor was 5,034. The histogram and cumulative plot of the MARD per sensor are shown in Figure S4. The mean and median ARD per sensor were 9.1 and 9.0%, respectively. In addition, 57.6, 85.0, and 96.7% of the sensors had a MARD <10, <12 and <14%, respectively.

Safety

Few adverse events related to the device or operations occurred during the clinical study. Only one patient experienced mild erythema at the implant area. No other skin‐associated adverse event, such as bruising, phyma, exudation or scleroma, was found. One patient experienced ketosis at the start of the clinical study, which was not related to the study or device.

Discussion

Since the first CGM system was approved by US Food and Drug Administration in 1999, several pertinent statistical methods have been used to evaluate CGM system data, which are both complex and voluminous18. In the present study, the accuracy of the Medtrum RT‐CGM system was evaluated using multiple statistical methods, including agreement analysis, multiple EGAs, bias analysis, correlation analysis, linear regression and Bland–Altman analysis. The agreement analysis, bias analysis, correlation analysis and Bland–Altman analysis reflect point numerical accuracy; the Clarke EGA, consensus EGA and surveillance EGA reflect point clinical accuracy; and the continuous EGA reflects trend clinical accuracy. The accuracy of the Medtrum RT‐CGM system was mainly discussed with the agreement analysis, the consensus EGA and the bias analysis.

As shown above, the Medtrum RT‐CGM system had a MARD of 9.1%. Dexcom developed an advanced algorithm and improved the MARD of the G4^® PLATINUM CGM system from 13.2 to 11.7%19. Bailey et al.20, 21 reported a 9% MARD for the modified G4 Platinum CGM system with the advanced algorithm that would be integrated in the new G5 Mobile CGM system. The Enlite^® 3 sensor used in the Medtronic 670G system had a MARD of 9.64%, with four calibrations per day22. FDA approved a Flash Glucose Monitoring System provided by Abbott in September 201623. This device had a MARD of 12%23, 24, and could function for 14 days with factory calibration; however, it operated in a flash model, and did not belong to the RT‐CGM system category. The interval of glucose values of the Abbott Flash Glucose Monitoring System was 15 min. The Medtrum RT‐CGM system provides glucose values at an interval of 2 min, and both of the CGM systems of Medtronic and Dexcom provide glucose values every 5 min.

In the present study, 90% of the sensor values could be considered as accurate readings. By comparison, the percentages of sensor values within ±20%/20 mg/dL of reference values were 93, 90.7, and 83.8% for the Dexcom G4 Platinum CGM system with advanced algorithm, Medtronic 670G system using the Enlite^® 3 sensor and Abbott Flash Glucose Monitoring System, respectively21, 22, 23. Additionally, 86.7% of sensor results of the Abbott Flash Glucose Monitoring System fell in zone A of the consensus EGA24, and the percentage in the clinically accurate zone A of Clarke EGA was 92.4% for the Dexcom G4 Platinum CGM system with advanced algorithm20. As for the clinical risk limitations, in the present study, approximately 99% of the sensor values were accurate or benign with no or slight clinical risk, which meant that approximately 1% of the sensor values might pose the risk of affecting the clinical outcome adversely. For the Dexcom G4 Platinum CGM system with advanced algorithm, the percentage in zone A and zone B of Clarke EGA was 99.5%20.

The Medtrum RT‐CGM system monitored glucose levels with accuracy continuously for 7 days. Usually, the performance of a CGM system on the first day of wear was poor. On day 1 of the clinical trial, the Medtrum RT‐CGM system had a MARD of 10.5%, and the percentage within ±20%/20 mg/dL of reference values was 89.7%. By comparison, the MARDs on day 1 were 10.7, 11.7 and 13.7, whereas the percentages of sensor values within ±20%/20 mg/dL of reference values were 84, 85.3, and 77.7% for the Dexcom G4 Platinum CGM system with advanced algorithm, Medtronic 670G system using the Enlite^® 3 sensor and Abbott Flash Glucose Monitoring System, respectively21, 22, 23.

Compared with the glucose sensors used in the Medtronic MiniMed and Dexcom, the MD‐JY‐006 glucose sensor has a shorter length and a smaller cross‐sectional area. The MD‐JY‐006 sensor can be inserted quickly using an automatic insertion mechanism with a small‐sized introducer needle (26‐G), which contributes to a smaller wound, less bleeding and therefore, weaker sensation of pain.

There were some limitations to the present study. First, the number of young patients with type 1 diabetes was small. The present findings should be generalized in a larger sample of patients with type 1 diabetes and type 2 diabetes, separately. The duration of frequent venous blood glucose testing for each participant was just 7 h, which did not fully cover the 12‐h calibration interval. Furthermore, only a limited number of paired sensor–reference values were in the range of hypoglycemia, and the study was carried out in only one country. Further research is required to enroll more young patients, contain a 12‐h frequent blood glucose test, record more hypoglycemic glucose values and make the direct comparison with other sensors. Meanwhile, prolonged research is required to evaluate whether the CGM system could provide additional benefits, such as improving glycemic control, reducing hypoglycemia episodes and decreasing the HbA_1c level.

In conclusion, the present study investigated the performance of the Medtrum RT‐CGM system including accuracy, stability, flexibility, consistency and safety. The results showed that the Medtrum RT‐CGM system had excellent accuracy and limited clinical risk compared with venous blood glucose in the range of 40–400 mg/dL over 7 days, and was not affected by factors such as type of diabetes, sex, therapy method, age, HbA_1c, diabetes duration, waist circumference or BMI.

Disclosure

The authors declare no conflict of interest.

Supporting information

Figure S1 | Medtrum A6 TouchCare^® CGM System.

Figure S2 | The Bland–Altman analysis of the mean difference between the paired sensor–reference values.

Figure S3 | Agreement analysis and type 1 diabetes consensus error grid analysis between paired sensor–reference values.

Figure S4 | The histogram and cumulative plot of the mean absolute relative difference (MARD) per sensor.

J Diabetes Investig 2018;9: 286–293