Accuracy of the SEVEN® Continuous Glucose Monitoring System: Comparison with Frequently Sampled Venous Glucose Measurements

Abstract

Background

The purpose of this study was to compare the accuracy of measurements obtained from the DexCom™ SEVEN® system with Yellow Springs Instrument (YSI) laboratory measurements of venous blood glucose.

Methods

Seventy-two subjects with insulin-requiring diabetes, aged 18–71, were enrolled in a multicenter, pro-spective single-arm study. All participants wore the SEVEN continuous glucose monitoring (CGM) system for one, 7-daywear period. Calibration with capillary finger stick measurements was performed 2 hours after sensor insertion and once every 12 hours thereafter. A subset of subjects (28) wore two systems simultaneously to assess precision. All subjects participated in one, 10-hour in-clinic session on day 1, 4, or 7 of the study to compare CGM measurements against a laboratory method (YSI analyzer) using venous measurements taken once every 20 minutes. Carbohydrate consumption and insu-lin dosing were adjusted in order to obtain a broad range of glucose values.

Results

Comparison of CGM measurements with the laboratory reference method (n = 2318) gave mean and median absolute relative differences (ARDs) of 16.7 and 13.2%, respectively. The percentage was 70.4% in the clinically ac-curate Clarke error grid A zone and 27.5% in the benign error B zone. Performance of the SEVEN system was con-sistent over time with mean and median ARD lowest on day 7 as compared to YSI (13.3 and 10.2%, respectively). Average sensor time lag was 5 minutes.

Conclusions

Measurements of the DexCom SEVEN system were found to be consistent and accurate compared with venous measurements made using a laboratory reference method over 7 days of wear.

Introduction

The recent introduction of continuous glucose monitoring (CGM) technology has given patients the ability to utilize real-time glucose data to help them achieve tighter glycemic control.^1–3 Preliminary clinical evidence also suggests that active use of CGM can reduce glycemic variability,^2,4–13 which an increas-ing body of evidence suggests can be an independent risk factor for diabetes complications.^14–21 More recently, results of a Juvenile Diabetes Research Foundation-funded randomized, controlled trial of CGM dem-onstrated a statistically significant 0.53% hemoglobin A1c (HbA1c) improvement for adults (25+ years old) with type 1 diabetes mellitus (T1DM), as compared to the control group that performed conventional self-monitoring of blood glucose (SMBG).¹³

Despite cited accuracy and reliability limitations of CGM,^22–24 the initial clinical benefits of continuous glucose monitoring mentioned earlier have been demonstrated with early generation CGM systems approved by the U.S. Food and Drug Administration (FDA).^25–27 As use of CGM expands, it will be essential for each new generation of technology to demonstrate improved accuracy, reliability, and usability to ensure that health care professionals and patients are able to gain the full benefit from this technology.

The purpose of this study was to evaluate the accuracy of a second-generation, 7-day CGM device, the DexCom™ SEVEN® system (DexCom, Inc., San Diego, CA), comparedto frequent venous blood samples analyzed with a laboratory standard.

Subjects and Methods

Study Population

Seventy-two subjects enrolled in this multicenter U.S. study. Individuals less than 18 years old and those pregnant, lactating, or with a contraindication to using the CGM (known allergy to medical adhesives) were excluded. The protocol was approved by the institutional review boards of all centers, and all subjects provided witnessed, written informed consent prior to enrollment. Subjects were 49 ± 14 [mean ± standard deviation (SD)] years old, and 37 (51.4%) were women. Subjects had a diagnosis of diabetes for 24 ± 14 years, 54 (75%) had T1DM, and 18 (25%) had type 2 diabetes mellitus (T2DM). Thirty-eight (53%) were on continuous subcutaneous insulin infusion pumps, and 34 (47%) delivered insulin via multiple daily injections. The average body mass index was 27.3 ± 6.36 kg/m², and the mean HbA1c was 7.6 ± 1.5%.

Sensor and Transmitter

The SEVEN sensor consists of an applicator, sensor probe, and transmitter housing as described previously.^5,28 The transmitter housing was adhered to the patient's abdomen, and the needle (containing the sensor probe) was inserted into the subcutaneous tissue. The needle was retracted and the applicator was removed, leaving the sensor probe within the subcutaneous tissue. After thetransmitter was installed, an averaged glucose signal transmitted wirelessly to the receiver at 5-minute intervals.

Receiver

The SEVEN receiver is an externally worn pager-sized device. For this study, the receiver used cable-uploaded SMBG meter values for calibration (i.e., to convert glucose signal measured by the sensor into user-viewable glucose concentrations). Two hours after the sensor was inserted, two SMBG values were uploaded for calibration. Thereafter, patients were instructed to upload at least one SMBG value every 12 hours. Once calibrated, the receiver displayed glucose values updated at 5-minute intervals, showed glucose trend graphs of the preceding 1, 3, or 9 hours, and generated high and low glucose alerts and alarms. In this study, the high glucose alert was set at 200 mg/dl and the low glucose alert was set at 80 mg/dl. A nonmodifiable hypoglycemia alarm was triggered at glucose levels ≤55 mg/dl.

Study Design

This was a prospective, single-arm study conducted at five research sites in the United States. On study day 1, participants received the CGM device and underwent training on proper use. Subjects were instructed to use CGM data as an adjunct to, and not as a replacement for, SMBG finger sticks when making diabetes-related treatment decisions. Insertion of the sensor in the abdomen was performed at the clinic. A subset of subjects (n = 28) wore two systems simultaneously to assess precision; one of these systems was blinded (real-time glucose values, trend graphs, and alerts/alarms not provided).

During home use, subjects were asked to use two blood glucose meters provided to them for calibration and comparative means. Subjects used a calibration meter for minimum calibration requirements (two per day) and a comparative meter for all other diabetes management finger sticks. All subjects participated in one, 10-hour in-clinic session on day 1, 4, or 7 of the study to assess accuracy across the duration of sensor life; irrespective of the in clinic session day, subjects wore the CGM system for 7 days. During the in-clinic session, all subjects underwent peripheral intravenous catheterization of the dorsal hand, lower arm, or antecubital region. Venous (serum) measurements were taken once every 20 minutes and analyzed using an accepted laboratory method [Yellow Springs Instrument (YSI) 2300 STAT Plus glucose and lactate analyzer, YSI Life Sciences, Yellow Springs, OH] to compare with SEVEN system values. YSI values were not used to calibrate the CGM; calibration was performed using a SMBG meter in accordance with instructions for use. Site staff adjusted subject carbohydrate intake and insulin dosing to obtain a full range of glucose values; this was done at the discretion of qualified medical personnel who supervised each in-clinic session. Formal insulin-dosing algorithms and/or glucose clamps were not used in this study.

Statistical Analysis

Analyses of system accuracy were performed using SAS® software, version 9.1.3 or later (SAS Institute, Inc., Cary, NC). Summary statistics for continuous variables include mean, standard deviation, median, and range. Categorical variables are presented as counts and percentages.

The prespecified primary efficacy end point was bias of paired sensor and YSI values during in-clinic use as compared to the SEVEN system using Deming regression. The Deming method takes into account the error in comparative meter measurements by using a variance ratio between the sensor and the YSI. The variance ratio was estimated to be 0.001. Deming regression takes into consideration the fact that measurements by both methods are subject to random errors. The Deming regression provided estimates for both y intercept and slope, which the sum of the squares of both the x residual and the y residual is minimized and the estimates for intercept and slope result from choosing the line that minimizes the sum of the squares of the perpendicular distances from the data points to the line.²⁹ The Deming method was chosen over simple (ordinary least squares) regression at the recommendation of the FDA and provided more realistic estimates for the intercept and slope. Using a two-sided alternative hypothesis, any formal hypothesis tests were carried out at the 5% level of significance. All confidence intervals provided were calculated with a 95% confidence level.

Precision in the subgroup of subjects that wore two systems simultaneously was assessed using percent absolute relative difference (PARD) and percent coefficient of variation (PCV). Percent absolute relative difference was defined as the blinded system minus the unblinded system divided by the average of the two systems. Percent coefficient of variation was defined as the standard deviation of the average of the two systems divided by the average of the two systems.

A post-hoc analysis of sensor vs YSI time lag was performed by determining the optimal correlation coefficient (R) between raw sensor data and YSI data when sensor data were shifted in 5 minute increments (reporting interval of the device), from −5 to +25 minutes.³⁰

Results

Sensor Accuracy and Stability

Of the 72 subjects enrolled, 69 subjects contributed to the 2318 sensor–YSI matched pair points between 40 and 400 mg/dl (range of CGM used in this study). Three subjects were excluded from efficacy evaluation for the following reasons: two subjects received sensor failures before the scheduled in-clinic day and one subject's receiver data were lost. All analyses were prospective, i.e., by using sensor glucose values as displayed to (or blinded from) subjects in real time.

In keeping previous reports of stable home use accuracy across 7 days relative to SMBG,⁵ sensor performance in this study relative to YSI was stable across all 7 days of sensor wear (Table 1). The overall median absolute relative difference (ARD) was 13.2%, and the mean ARD was 16.7%. There was no appreciable difference in the overall accuracy results, and median and mean relative differences were actually slightly better on day 7 of wear (10.2 and 13.3%, respectively). Mean and median absolute differences across glucose ranges are also reported in Table 1. The system demonstrated an overall slight negative bias (−7.3% relative difference), which is consistent with the −8% described in device labeling.²⁶

Table 1.

Sensor Accuracy as Derived by Relative Difference and Absolute Relative Difference from the YSI Overall and for Each Day of Use^a

Sensor day of wear	Subjects N (%)b	Matched pairs n (%)c	(%) relative difference				(%) absolute relative difference
			Mean	STD	Median	Min, max	Mean	SD	Median	Min, max
Day 1	27 (39%)	818 (38%)	−10.7	25.1	−13.5	−62.4, 186.4	19.3	19.3	15.6	0, 186.4
Day 4	21 (30%)	625 (29%)	−4.0	21.6	−6.9	−46.1, 117.5	17.0	14.0	13.7	0, 117.5
Day 7	21 (30%)	705 (33%)	−4.9	17.1	−6.4	−44.4, 103.7	13.3	11.8	10.2	0, 103.7
Overall	69	2148c	−6.8	21.9	−9.4	−62.4, 186.4	16.7	15.8	13.2	0, 186.4

YSI glucose range	Subjects N (%)b	Matched pairs n (%)c	Mean	SD	Median	Min, max	Mean	SD	Median	Min, max
40−80	39	206	5.3	37.8	−1.1	−42.6, 186.4	24.3	29.4	17.8	0.1, 186.4
81−120	57	444	−4.5	24.2	−6.0	−62.4, 131.2	17.3	17.5	13.0	0, 131.2
121−240	69	1154	−9.0	18.4	−10.3	−57.1, 131.3	16.1	12.7	13.0	0, 131.2
241−300	50	339	−11.1	13.6	−12.3	−42.0, 53.1	14.6	9.8	13.2	0, 53.1
301−400	27	175	−10.6	12.6	−10.5	−37.2, 31.8	13.7	9.1	11.5	0.2, 37.2
Overall	69	2318	−7.3	21.6	−9.7	−62.4, 186.4	16.6	15.6	13.2	0, 186.4

^aYSI in-clinic days occurred on days 1, 4, and 7 of the study. Day 1 sensor elapsed time evaluated: 2−30 hours; day 4 sensor elapsed time evaluated: 62−99 hours; and day 7 elapsed time evaluated: 139−168 hours. Glucose measurements are between 40 and 400 mg/dl.

^bN, number of subjects contributing data; n, number of matched pairs.

^cOf the matched pairs, 170 fell outside of day 1, 4, or 7 time windows.

Table 2 provides traditional Clarke error grid results^31,32 across the entire reportable glucose range of the SEVEN system (40–400 mg/dl) and by low (40–75 mg/dl), mid (76–180 mg/dl), and high glucose (181–300 and 301–400 mg/dl) ranges. For all ranges, 70.4% of points fell in the clinically accurate A zone, with 97.9% of all points falling in the A and B zones. On the Clarke error grid, 1.5% of points were in the D zone and 0.04% was in the E zone. Figure 1 shows the Clarke error grid plot, indicating that the sensor has the tendency to read slightly lower than the YSI.

Table 2.

Percentage (and Number) of DexCom SEVEN System Results Falling within Various Zones of the Clarke Error Grid, Stratified by YSI Glucose Concentrations^a

YSI glucose range	Total	A + BN (%)	AN (%)	BN (%)	CN (%)	DN (%)	EN (%)
40–80 mg/dl	206	180 (87.4)	139 (67.5)	41 (19.9)	2 (1.0)	23 (11.2)	1 (0.5)
81–120 mg/dl	444	440 (99.1)	305 (68.7)	135 (30.4)	4 (0.9)	0 (0.0)	0 (0.0)
121–180 mg/dl	643	641 (99.7)	443 (68.9)	198 (30.8)	2 (0.3)	0 (0.0)	0 (0.0)
181–240 mg/dl	511	508 (99.4)	377 (73.8)	131 (25.6)	3 (0.6)	0 (0.0)	0 (0.0)
241–300 mg/dl	339	324 (95.6)	248 (73.2)	76 (22.4)	3 (0.9)	12 (3.5)	0 (0.0)
301–400 mg/dl	175	175 (100)	124 (70.9)	51 (29.1)	0 (0.0)	0 (0.0)	0 (0.0)
Overall	2318	2268 (97.8)	1631 (70.4)	637 (27.5)	14 (0.6)	35 (1.5)	1 (0.0)

^aGlucose measurements between 40 and 400 mg/dl, inclusive, are included.

Figure 1.

Clark error grid plot of 7-day sensor (STS-7; DexCom, San Diego, CA) versus YSI. There are 70.4% of points in the clinically accurate A zone, 27.5% in the benign error B zone, and only 1.5% in the D zone and 0.04% in the E zone.

Results of continuous glucose error grid analysis (CG-EGA) are included in Table 3. The CG-EGA zones have similar clinical significance as the standard Clarke error grid, but zone demarcations are modified to account for the rate of change observed in CGM. The CG-EGA is designed to evaluate the clinical accuracy of CGM in terms of both precision of blood glucose readings and precision of blood glucose rate of change. Unlike standard error grid analysis, the CG-EGA examines temporal characteristics of continuous sensing data.^33,34 Continuous error grid analysis demonstrated 94.7% of points falling in the clinically accurate regions with 2.3% benign errors and 3.0% inaccurate readings overall. Table 3 indicates that the SEVEN system reads best in the euglycemic range but still provides 76.0% clinical accurate readings at hypoglycemic levels and 93.8% in the hyperglycemic range. When reviewing the CG-EGA, it should be noted that the rate of change measurement used to generate this CG-EGA was derived by YSI samples taken every 20–25 minutes; literature references recommend samples be taken every 15 minutes.³³

Table 3.

Continuous Error Grid Point and Rating Tables

Twenty-eight subjects wore two SEVEN systems simultaneously to assess precision. Overall precision of the SEVEN system as derived by PARD was approximately 16.0%, indicating that sensor-to-sensor agreement on the same individual is similar to the sensor-to-YSI agreement as derived by the mean ARD of 16.7% shown in Table 1. The mean PCV showed that the error from the average glucose value of two sensor values was approximately 11.3%. Both mean PARD (range of per day values: 14.4–16.9%) and mean PCV (range of per day values: 9.7–11.9%) were stable across all 7 days of sensor wear, not limited to the in-clinic period (Table 4).

Table 4.

Precision Analysis by Sensor Day of Wear^a

Sensor wear day	No. of matched pairs	Percent ARD mean	Percent ARD (SD)	PCV mean	PCV (SD)
Overall	32,088	15.6	15.73	11.03	11.13
Sensor day 1	5,250	15.98	14.93	11.3	10.56
Sensor day 2	5,250	14.35	14.23	10.15	10.06
Sensor day 3	5,125	15.8	15.83	11.17	11.2
Sensor day 4	4,729	16.61	15.04	11.74	10.63
Sensor day 5	4,599	13.77	13.49	9.74	9.54
Sensor day 6	4,018	16.41	18.52	11.6	13.09
Sensor day 7	3,116	16.88	18.83	11.94	13.31

^aOne matched pair was outside of the 7-day time window.

No severe adverse events or infection at the sensor pod were reported related to use of the sensor inserted for up to 168 hours. Overall, 89% of subjects reported no symptoms of device-related irritation, 10% reported mild erythema, and 1% reported mild ecchymosis.

Seventy-five percent of sensors lasted until study day 7; of these sensors, seven fell off the subject's skin (adhesive failure). During a 7 day session, a maximum of 1992 glucose readings are expected; 69% of the devices used in this study provided 1537–1992 readings (top quartile of potential values). The average amount of time a subject went without getting CGM readings was 15 minutes (three sequential values).

To explore the association between interstitial fluid (ISF) time lag and possible effect on sensor accuracy, a post-hoc, exploratory analysis of retrospective time-lag analysis was performed on 88 of the sensors used in this study that had at least 8 hours of corresponding YSI data. The time lag of the ISF sensor was found to average 5 minutes with a rate of change within −2 to 2 mg/dl/minute.³⁰

Conclusions

This study showed that median ARD by day of wear ranged from 10.2 to 15.6%, with an average of 13.2%. The lowest median ARD occurred on day 7 of wear (10.2%), indicating that there is no late decline in sensor accuracy as it is worn over a 7-day period. Figure 2 shows a glucose profile plot from the present study for one subject's in-clinic session with data from the inserted sensor and 20-minute venous samples. Additionally, nonparametric Kruskal–Wallis tests on SEVEN system bias were performed (measuring relative difference and ARD between different in-clinic days) and showed a p value <0.001. This concludes that accuracy improvements (lower relative difference and ARD) on in-clinic day 7 compared with days 1 and 4 were statistically significant.

Figure 2.

Sample case. YSI on day 7: The subject was a 69-year-old man with a history of type 1 diabetes on continuous subcutaneous insulin infusion with a baseline HbA1c of 7.0% and a body mass index of 27.6 kg/m². Subject wore two sensors and had separate Ultra meters for calibration (CalMeter) and compara-tive (CompMeter) purposes. During the in-clinic day on day 7 of wear, reference YSI and CompMeter (Ultra) SMBG values were measured every 20 minutes, and a wide-range of glucose values was obtained by manipulating insulin dosing and carbohydrate ingestion. Note: for illustrative purposes, a good example of CGM tracking is shown; aver-age and poor examples are present in FDA-approved device labeling.²⁶

The scope of this article was to describe performance of this CGM system as compared to YSI glucose measurements. However, the accuracy of this 7-day device during home use has been reported previously in a separate 86 subject, 3-week study.⁵ Accuracy versus SMBG in the outpatient setting was also stable over 7 days of sensor life, with an overall mean ARD of 15.7% (slightly better than the 16.7% versus YSI reported in the present study). Median ARD was 11.4% versus home use SMBG as compared to 13.2% versus YSI. Performance in the home use setting appears to be slightly better than what was shown at the clinic.

In addition, 97.9% of measurements collected across all glucose ranges were clinically acceptable (Clarke error grid zones A and B), and all glucose ranges reported that the majority of these values (70.4%) were considered clinically accurate (within Clarke error grid zone A). CG-EGA results were best in the euglycemic range, with 97.5% accurate readings; performance during hyperglycemia was similar with 96.0% accurate readings. Together, this accounted for 96% of data collected during this study (52% during euglycemia and 44% during hyperglycemia). Given the 40-mg/dl lower reporting limit of this device, it is more difficult to collect data in the hypoglycemic range of the CG-EGA (≤70 mg/dl); for the 4% of data collected in this range, 76% were accurate and 24% were categorized as inaccurate according to the CG-EGA. These results compare favorably with the 59.5% accurate, 0.8% benign, and 39.7% inaccurate readings for another FDA-approved subcutaneous CGM system.²⁷

Results from CG-EGA showed better accuracy in the hypoglycemia region (blood glucose less than 70 mg/dl) among type 1 subjects (77.6%) compared with 62.5% among type 2 subjects (Table 5). This post-hoc analysis may suggest differences in accuracy between T1DM and T2DM subjects. Surprisingly, the CG-EGA tables presented suggest better accuracy in subjects with T1DM. This finding should be evaluated further in a larger study enrolling similar numbers of T1DM and T2DM subjects. The present study had fewer T2DM subjects and numbers of paired samples within the hypoglycemia region (ratio of 75%:25% for T1DM:T2DM). A recent study evaluating the aforementioned devices corroborated these findings in the hypoglycemic zone with 81.0% accuracy for the SEVEN system and 57.1% accuracy for the other device.³⁵ Labeling for the third FDA-approved CGM system²⁵ does not include CG-EGA performance.

Table 5.

Continuous Error Grid Point and Rating Tables by Diabetes Type

Diabetes type	CG-EGA	HypoglycemiaBG ≤ 70 mg/dl	Euglycemia70 < BG ≤ 180 mg/dl	HyperglycemiaBG > 180 mg/dl
T1DM	Accurate readings	77.63%	98.42%	96.25%
(n = 54, 75% of subjects)	Benign errors	0.00%	1.21%	1.31%
	Erroneous readings	22.37%	0.36%	2.45%
T2DM	Accurate readings	62.50%	94.95%	95.66%
(n = 18, 25% of subjects)	Benign errors	0.00%	2.84%	2.02%
	Erroneous readings	37.50%	2.21%	2.31%

Sensitivity and specificity analyses are not available for the present study due to the fact that the in-clinic sessions were not structured to collect sufficient data to make the results meaningful. For instance, subjects were likely to cross the <72-mg/dl threshold once during a given in-clinic session, if at all. In future studies, defining an appropriate window about actual threshold crossing will be vital. Also, in order to accrue sufficient numbers of events, it will be useful to perform such analyses on data collected during home use studies.

Tangible evidence of promising technology improvement can be demonstrated in accuracy measures as CGM evolves to subsequent generations. For example, the first-generation DexCom 3-day CGM system showed a median ARD of 23% with Clarke error grid zones A and B of 49 and 41%, respectively.²⁶ The increase of accuracy in the DexCom 7-day system to an overall median ARD of 13.2% as compared to the DexCom 3-day system median ARD of 23%²⁶ is largely attributable to the modified algorithm parameters to better handle rates of change. Unlike other CGM devices, the DexCom SEVEN system does not introduce a long calibration delay to increase overall accuracy measures³⁴ and achieves comparable median ARD on both its first (15.6%) and last (10.2%) days of wear with only a 2-hour break-in period.

A follow-up study on the DexCom 7-day system was conducted and showed no significant difference in sensor accuracy when patients calibrated the system with a new manual entry for a blood glucose feature as compared to auto-upload of calibration values from a one-touch meter.³⁶ This feature allows patients to use any FDA-cleared meter to calibrate rather than a specific brand, providing more patient choice in second-generation technology. Because post-hoc analysis showed that the time lag of the ISF sensor was found to average 5 minutes with a rate of change within −2 to 2 mg/dl/minute, the effects of time lag and glucose rate of change on sensor accuracy versus YSI should not be substantial.³⁴

As demonstrated by these marked changes from first-generation to second-generation technology, the future of CGM is likely to continually improve to meet health care and patient needs. The current level of accuracy and sta-bility over 7 days of the DexCom SEVEN system reported earlier (e.g., median ARD 13.2%) and tolerability of 7 days of wear may provide a starting point for significant clinical benefits for patients with diabetes.¹³ The demonstration of rapid improvement from early CGM technology platforms forms a basis for future clinical studies aimed to improve the user friendliness of CGM technology while increasing accuracy and length of wear.

Abbreviations

ADA

American Diabetes Association

ARD

absolute relative difference

CG-EGA

continuous glucose error grid analysis

CGM

continuous glucose monitoring

DCCT

Diabetes Control and Complications Trial

FDA

Food and Drug Administration

HbA1c

hemoglobin A1c

ISF

interstitial fluid

PARD

percent absolute relative difference

PCV

percent coefficient of variation

SD

standard deviation

SMBG

self-monitored blood glucose

T1DM

type 1 diabetes mellitus

T2DM

type 2 diabetes mellitus

YSI

Yellow Springs Instrument