Accuracy of the Modified Continuous Glucose Monitoring System (CGMS®) Sensor in an Outpatient Setting: Results from a Diabetes Research in Children Network (DirecNet) Study

Abstract

Objective

We previously reported the results of an inpatient accuracy study in children with type 1 diabetes using the Continuous Glucose Monitoring System (“CGMS”; Medtronic MiniMed, Northridge, CA).(1) During the course of that study, a new process was implemented for manufacturing the CGMS sensor. Accuracy from the resulting modified sensor used by only 14 children was significantly better than the original version (median relative absolute difference [RAD] 11% vs. 19%; P<0.001). Baseline data from a subsequent outpatient study provide an opportunity to further assess the accuracy of the modified sensor in a much larger sample of children with type 1 diabetes.

Research Design and Methods

As part of a randomized trial to assess the utility of the GlucoWatch G2® Biographer (“GW2B”; Cygnus Inc., Redwood City, CA), 200 children with type 1 diabetes were instructed to wear a CGMS for 48–72 hours in an outpatient setting at baseline. Glucose measurements from a OneTouch® UltraSmart® (Lifescan, Inc., Milpitas, CA) home glucose meter were downloaded and used as reference values to calculate accuracy measures.

Results

The overall median RAD was 12%. Accuracy was better during hyperglycemia than hypoglycemia (median RAD 10% vs. 20%; P<0.001) and on optimal versus non-optimal days but did not vary significantly by the number of calibrations entered.

Conclusions

These data confirm the improved accuracy previously reported for the modified version of the CGMS sensor.

The Continuous Glucose Monitoring System (“CGMS”; Medtronic MiniMed, Northridge, CA) was the first glucose sensor approved by the Food and Drug Administration, and has been used extensively in the management of individuals with diabetes. The DirecNet Study Group evaluated the accuracy of the original CGMS sensor in 78 children with type 1 diabetes who were hospitalized in a clinical research center for 24 hours.(1) In this study, 5,658 original CGMS sensor glucose values were compared with reference plasma glucose measurements performed in a central laboratory. During the course of the study, a modified CGMS sensor became available and was tested in 14 subjects. For the 1,120 sensor-reference glucose pairs with the modified sensor, accuracy was considerably better than with the original sensor (median RAD 11% versus 19%; P<0.001).

The purpose of this analysis was to determine whether the improved accuracy with the modified sensor could be confirmed with additional data in a much larger group of diabetic children and whether this accuracy could be maintained in an outpatient setting.

Methods

The Diabetes Research in Children Network (DirecNet) is an NIH-sponsored collaborative group whose objective is to evaluate the clinical usefulness of new near-continuous glucose sensor systems in children and adolescents with T1DM. DirecNet conducted a randomized clinical trial to assess the merits of the GlucoWatch G2® Biographer (“GW2B”; Cygnus Inc., Redwood City, CA) in improving glycemic control in children with type 1 diabetes. From July 2003 through January 2004 as part of the study’s baseline testing prior to randomization, each subject used a CGMS in an outpatient setting to provide a measure of glycemic control and frequency of hypoglycemia. A secondary pre-planned objective of this study was to assess the accuracy of the modified CGMS in an outpatient setting.

The modified CGMS sensor was inserted by study personnel and subjects were instructed to use it for 48 to 72 hours. The subject and caregiver were instructed in the use of the CGMS and were requested to insert four calibration glucose values a day obtained using a new OneTouch® UltraSmart® home glucose meter (“Ultra”) supplied by the study. On two of the days of CGMS use, the subject was instructed to check his/her blood glucose with the Ultra meter at 8 preset times (before and after each meal, before bed, and between 12 midnight and 4 a.m.). Following the use of the CGMS, the subject returned to the clinical center where the CGMS and Ultra data were downloaded.

As part of the inpatient sensor accuracy study mentioned earlier, we also evaluated accuracy of the OneTouch® Ultra® home glucose meter and reported a median RAD of 6% based on 2,068 Ultra-laboratory glucose pairs.(2) The OneTouch Ultra meter was therefore considered sufficiently accurate for its values to serve as the reference glucose measurements for this analysis.

Of the enrolled 200 subjects in the randomized trial, six were excluded from this analysis because all Ultra meter values were used to calibrate the CGMS (leaving none for analysis) and three were excluded because they averaged fewer than 3 calibrations of the CGMS per 24 hours. The remaining 191 subjects were included in analysis: 46% female; mean age 12.5 ± 2.8 years; and ethnicity/racial distribution of 84% White, 7% African American, 3% Hispanic or Latino, and 6% mixed or other races. Mean hemoglobin A1c was 8.0 ± 0.9%; 46% of the subjects were pump users.

The DirecNet Data and Safety Monitoring Board and the Institutional Review Boards at each of the DirecNet centers approved the study protocol, consent form and assent form. A parent or guardian gave written consent and patients gave written assent prior to the performance of any study procedures.

The CGMS glucose measurements were matched to reference measurements from the Ultra meter performed within 2.5 minutes, after adjusting the CGMS time by 2.5 minutes (to account for the averaging of glucose measurements made over the prior 5 minutes). Subjects averaging fewer than 3 calibrations of the CGMS per 24 hours of use were excluded from analysis. Ultra meter glucose values used to calibrate the CGMS were not included in the analysis.

For each matched pair, the following were computed: difference (sensor value minus reference value), absolute difference (absolute value of difference), relative difference (difference divided by reference value, multiplied by 100 to convert proportion to percentage), and relative absolute difference (absolute difference divided by reference value, multiplied by 100 to convert proportion to percentage, referred to as “RAD”). Each pair was also evaluated to determine whether it met the proposed International Organisation for Standardisation (ISO) criteria (for reference glucose value ≤75 mg/dL, CGMS value within ± 15 mg/dL and for reference glucose value >75 mg/dL, CGMS value within ± 20%, hereafter referred to as the “ISO criteria”).(3)

Summary statistics (e.g., mean and median) were calculated by pooling all paired values. The bootstrap technique(4) (resampling subjects with replacement) was used to account for the within subject correlation in the statistical comparisons and calculation of confidence intervals

In addition to the overall analyses, we performed a separate set of accuracy analyses to compare CGMS values from “optimal days” and “non-optimal” days. The CGMS software analyzes the relationship between sensor outputs and blood glucose calibration values and classifies each calendar day of sensor function as optimal or non-optimal. If the range (minimum to maximum) of meter values is at least 100 mg/dL, then a day is considered optimal when the mean relative absolute difference between at least three paired sensor glucose and meter glucose values is ≤28% and the correlation coefficient between paired sensor glucose and meter glucose values is ≥0.79 whereas if the range is less than 100 mg/dL, then the day is considered optimal when the relative mean absolute difference is ≤18%.

Results

The analysis included 1,899 CGMS-reference pairs. The mean number of pairs per subject was 9.9 (median 10; range 1–31). The CGMS glucose values tended to be about 3% lower than the reference glucose values (median relative difference = −3%; 25^th, 75^th percentiles −15%, +9%; P<0.001). The median relative absolute difference (RAD) was 12% and the ISO criteria were met by 72% of pairs (Table 1).

Table 1. CGMS Accuracy Summary Statistics.

(N=1,899 sensor-reference pairs)

	Mean (95% confidence interval)	Median (25th, 75th percentiles)
Difference* mg/dL	−9.1 (−12.1, −6.2)	−4 (−24, 13)
Absolute Difference† mg/dL	29.3 (26.9, 32.0)	18 (8, 38)
Relative Difference‡	−2% (−3%, 0)	−3% (−15%, 9%)
Relative Absolute Difference§	17% (16%, 18%)	12% (6%, 23%)
ISO Criteria Met||	72% (69%, 74%)	N/A

^*Difference is the sensor glucose value minus the reference glucose value.

^†Absolute difference is the absolute value of the difference.

^‡Relative Difference is the difference divided by the reference glucose value (expressed as percentage).

^§Relative absolute difference is the absolute value of the relative difference (expressed as percentage).

^||ISO criteria: for reference glucose value ≤75 mg/dL, CGMS value ±15 mg/dL and for reference glucose value >75 mg/dL, CGMS value within ±20%.

Accuracy varied with the glucose level, being greater at higher glucose levels than lower glucose levels (for RAD, P < 0.001; Figure 1/Table 2). For the 556 pairs where the reference value was >240 mg/dL, the median RAD was 10% and 77% of pairs met the ISO criteria whereas for the 176 pairs where the reference value was ≤70 mg/dL, the median RAD was 20% and 66% of pairs met the ISO criteria.

Figure 1. Comparison of Accuracy for Original vs. Modified CGMS Sensors.

Median 255 relative absolute difference (RAD) is plotted against the reference glucose value for each 256 sensor type.

Table 2. Accuracy Summary Statistics According to Glucose Level.

	# Paired Data Points	Relative Absolute Difference (median)	ISO criteria met* (percentage)	P-values†
Overall	1,899	12%	72%	-
Reference Glucose Level (mg/dL)				< 0.001/0.002
≤ 70	176	20%	66%
71–120	404	14%	66%
121–180	408	10%	73%
181–240	355	12%	70%
> 240	556	10%	77%

^*ISO criteria: for reference glucose value ≤75 mg/dL, CGMS value ±15 mg/dL and for reference glucose value >75 mg/dL, CGMS value within ±20%.

^†The first p-value is for RAD and the second p-value is for ISO criteria met.

Since the CGMS is retrospectively calibrated, real-time sensitivity for detection of hypoglycemia cannot be assessed. However, we evaluated these data to give an upper bound on what the accuracy would be with prospective calibration. For an alarm setting of 60 mg/dL, the sensor sensitivity for detection of a glucose level ≤60 mg/dL (based on the reference glucose value) would be 54% and 62% of alarms would be false. The reference glucose was above 80 mg/dL for 25% of the pairs where the CGMS value was ≤60 mg/dL.

Of the 611 total subject-days in this data set, 396 (65%) met the definition of an “optimal” day as defined in the Methods. Accuracy was better on optimal than on non-optimal days (median RAD 11% vs. 16%, P<0.001; percentage meeting the ISO criteria 75% vs.62%, P<0.001).

The mean number of calibrations per 24 hours of CGMS use was 5.3 (median 4.9, range 3.2 to 11.6). Comparison of the 24 subjects who averaged ≤4 daily calibrations vs. the 167 subjects who averaged >4 daily calibrations showed no differences in accuracy (median RAD 12% vs. 12%, P=0.34 and percentage meeting the ISO criteria 71% vs. 72%, P=0.86).

Discussion

Our results confirm that the modified CGMS sensor that became available in November 2002 is more accurate than the original sensor that had been in use prior to that date. In a previous inpatient study we reported a median RAD of 19% with the original sensor and 11% with the modified version.(1) Our accuracy data in this outpatient setting (12% median RAD and 72% of values meeting ISO criteria) are very similar to those observed in an inpatient setting (11% median RAD and 72% of values meeting ISO criteria)(1) even though the current study used Ultra meter values as the reference and the inpatient study used central laboratory measurements. As we found in the inpatient study, accuracy was lower for hypoglycemia compared with hyperglycemia. However, some degree of lower accuracy on a relative scale is expected of all measurement methods for glucose values that are in the hypoglycemic range. Our results also confirm the finding from the inpatient study that accuracy is greater on ‘optimal’ CGMS days than on ‘non-optimal’ days.

The CGMS glucose values tended to be slightly lower (median 3%) than the Ultra values used as the reference. Prior studies using capillary blood have reported values from the OneTouch Ultra that were higher than venous reference measurements.(2,5,6) However, any bias from using the Ultra as the reference glucose in this study would be expected to cancel out since capillary Ultra measurements were also used to calibrate the CGMSs. In a prior study, we found that the modified CGMS sensors did not show bias compared with venous serum glucose values.(1)

In summary, our results confirm the earlier observations of the DirecNet inpatient accuracy study. The modified CGMS sensor is considerably more accurate than the original sensor over the full range of blood glucose levels, though accuracy remains better for hyperglycemia than hypoglycemia. In clinical practice, CGMS data are used retrospectively to assess for trends in glucose levels and alter subsequent diabetes management. A moderate degree of inaccuracy could be present for individual glucose determinations and the sensor could still be providing useful information about trends. Hopefully as occurred with home glucose meters, accuracy of continuous glucose monitors will continue to improve with further technologic advances.

Appendix

Writing Committee

Michael J. Tansey, MD; Roy W. Beck, MD, PhD; Bruce A. Buckingham, MD; Nelly Mauras, MD; Rosanna Fiallo-Scharer, MD; Dongyuan Xing, MPH; Craig Kollman, PhD; William V. Tamborlane, MD; Katrina J. Ruedy, MSPH.

The DirecNet Study Group

Clinical Centers

Listed in alphabetical order with clinical center name, city, and state. Personnel are listed as (PI) for Principal Investigator, (I) for co-Investigator and (C) for Coordinators.

1. Barbara Davis Center for Childhood Diabetes, University of Colorado, Denver, CO

   H. Peter Chase, MD (PI); Rosanna Fiallo-Scharer, MD (I); Jennifer H. Fisher, ND, RN (C); Barbara Tallant, RN (C)

2. Department of Pediatrics, University of Iowa Carver College of Medicine, Iowa City, IA

   Eva Tsalikian, MD (PI); Michael J. Tansey, MD (I); Linda F. Larson, RN (C); Julie Coffey, MSN (C)

3. Nemours Children’s Clinic, Jacksonville, FL

   Tim Wysocki, PhD, ABPP (PI); Nelly Mauras, MD (I); Larry A. Fox, MD (I); Keisha Bird, MSN (C); Kelly L. Lofton, RN (C)

4. Division of Pediatric Endocrinology and Diabetes, Stanford University, Stanford, CA

   Bruce A. Buckingham, MD (PI); Darrell M. Wilson, MD (I); Jennifer M. Block, RN, CDE (C); Paula Clinton, RD, CDE (C)

5. Department of Pediatrics, Yale University School of Medicine, New Haven, CT

   Stuart A. Weinzimer, MD (PI); William V. Tamborlane, MD (I); Elizabeth A. Doyle, MSN (C); Kristin Sikes, MSN (C)

Coordinating Center

Jaeb Center for Health Research, Tampa, FL

Roy W. Beck, MD, PhD; Katrina J. Ruedy, MSPH; Craig Kollman, PhD; Dongyuan Xing, MPH; Cynthia R. Silvester

Data and Safety Monitoring Board

Dorothy M. Becker, MBBCh; Christopher Cox, PhD; Christopher M. Ryan, PhD; Neil H. White, MD, CDE; Perrin C. White, MD

University of Minnesota Central Laboratory

Michael W. Steffes, MD, PhD; Jean M. Bucksa, CLS; Maren L. Nowicki, CLS; Carol A. Van Hale, CLS

National Institutes of Health

Gilman D. Grave, MD; Barbara Linder MD, PhD; Karen K. Winer, MD