Accuracy of Newer Generation Home Blood Glucose Meters in a Diabetes Research in Children Network (DirecNet) Inpatient Exercise Study

Abstract

Objective

Assess how the accuracy of the FreeStyle^® Flash™ (Abbott Diabetes Care, Alameda, CA) meter compares with the One Touch^® Ultra^® (Lifescan, Milpitas, CA) home glucose meter (HGM).

Research Design and Methods

Fifty children with type 1 diabetes (T1D), 10–17 years old, were admitted for two separate 24 hour periods to assess the effect of exercise on subsequent nocturnal hypoglycemia. Resulting data were used in a pre-planned analysis of the accuracy of the Ultra and FreeStyle HGMs. Glucose levels were measured throughout the day and night and every 15–20 min during a standardized exercise protocol. Reference samples were assayed in a central laboratory using a hexokinase enzymatic method. These reference glucose measurements were paired with HGM values from venous blood obtained within ± 5 minutes.

Results

The median relative absolute difference (RAD) was 5% for both the Ultra and FreeStyle HGMs and the percentage of pairs meeting the ISO criteria were 99% and 98% respectively. The FreeStyle tended to read slightly higher than the reference method (median difference = +3 mg/dL; p<0.001) and there was trend in this direction for the Ultra (median difference = +2 mg/dL, p=0.15). Sensitivities for detection of hypoglycemia (reference ≤60 and HGM ≤70 mg/dL) were 96% and 100% for the Ultra and FreeStyle, respectively, and corresponding false positive rates were both 5%.

Conclusions

In a controlled clinical setting using venous blood samples, both the Ultra and FreeStyle meters demonstrated a high degree of accuracy compared with the laboratory reference over a broad range of glucose concentrations in children with T1D

Introduction

In order to reduce the incidence of microvascular complications, people with type 1 diabetes (T1D) are encouraged to achieve near-normal blood glucose concentrations.¹ These goals would not have been possible without the development of home glucose meter (HGM) technology, which has enabled people with T1D to monitor glycemic status frequently and accurately. Multiple studies have shown the beneficial effects of self monitoring of blood glucose (SMBG) on achievement of tight glycemic control in adults and youth with diabetes,^{2, 3} and the American Diabetes Association recommends frequent SMBG for all patients with T1D.^{4, 5}

The technology of HGM devices has improved dramatically since their introduction over 20 years ago; newer devices are faster, use smaller volumes of blood, and are more accurate than their older counterparts, and also incorporate memory functions to store blood glucose data. The FreeStyle^® Flash™ (“FreeStyle”; Abbott Diabetes Care, Alameda, CA) is one of the most recent generation of HGM devices that possess the capability of transmitting blood glucose data directly to an insulin pump, which in turn calculates suggested insulin bolus doses based on preprogrammed sensitivity and correction factors. However, the accuracy of the FreeStyle meter has not been independently evaluated in children.

The Diabetes Research in Children Network (DirecNet) is an NIH-sponsored multi-center collaborative group whose objective is to evaluate the clinical usefulness of continuous glucose sensors and other new technologies in children and adolescents with T1D. We previously reported favorable performance of the One Touch^® Ultra^® (“Ultra”; Lifescan, Milpitas, CA) HGM in an inpatient accuracy study.⁶ As a pre-planned secondary objective in a study conducted to assess the effect of exercise on subsequent nocturnal hypoglycemia in children with T1D, we evaluated the accuracy of the Ultra meter and the FreeStyle meter compared with a laboratory reference.

Methods

The DirecNet Data and Safety Monitoring Board and the Institutional Review Boards at each of the DirecNet centers approved the study protocol, consent form and assent form. A parent or guardian and each subject gave written consent and assent, respectively. Subject eligibility criteria included age between 10 and 18 years, type 1 diabetes for at least 18 months, and HbA1c ≤10.0%. Each subject was hospitalized in a clinical research center for approximately 24 hours on two occasions separated by one to four weeks. During one of the two days, the subject had a 75-minute exercise session in the late afternoon.

Blood samples for central laboratory determination of serum glucose levels were obtained from an intravenous catheter hourly on both days from 10 p.m. to 6 a.m. and during the exercise period on one day. Glucose determinations were made at the DirecNet Central Laboratory at the University of Minnesota using a hexokinase enzymatic method, which has been proposed as the reference method for measuring glucose.^{7, 8} Each time a blood draw was obtained for a laboratory measurement, concurrent measurements were performed using both HGM devices. The HGM samples were taken directly from the intravenous catheter after the line was cleared. The Ultra meter strips were from multiple lots and the Freestyle strips were from the same lot. Ninety-eight percent of the HGM measurements at the time of a blood draw were made with venous blood and 2% with capillary blood from a finger stick.

Statistical analysis

Analysis included glucose determinations at the time points for which there was a laboratory-measured reference value and values derived from venous blood within ±5 minutes for both HGM values. The small number of HGM values from capillary blood pairing to a reference value were summarized separately.

For each reference-HGM matched pair, the following were computed: difference (HGM value minus reference value), absolute difference (absolute value of difference), relative difference (difference divided by reference value, multiplied by 100 to convert proportion to percentage), and relative absolute difference (absolute difference divided by reference value, multiplied by 100 to convert proportion to percentage, referred to as “RAD”). Each pair was also evaluated to determine whether it met the International Organisation for Standardisation (ISO) criteria (for reference glucose value ≤75 mg/dL, HGM value within ±15 mg/dL and for reference glucose value >75 mg/dL, HGM value within ±20%, hereafter referred to as the “ISO criteria”).⁹

Summary statistics (e.g., mean and median) were calculated by pooling all paired values. The bootstrap technique (resampling subjects with replacement)¹⁰ was used to account for the within subject correlation in the statistical comparisons and calculation of confidence intervals. Accuracy measures were also stratified by reference glucose level.

Results

Overall accuracy

Fifty subjects participated in the study between June 2004 and November 2004. Their average age was 14.8 ± 1.7 years; 44% were female. Ninety percent were Caucasian, 4% African-American, 4% Asian and 2% Hispanic. The mean duration of diabetes was 7.0 ± 3.7 years. Mean HbA1c was 7.8 ± 0.8 %.

The 50 subjects had a total of 1,103 reference measurements for which concurrent measurements using venous blood from both HGM devices were available. The number of HGM-reference glucose pairs per subject ranged from 10 to 27 (median= 23; 25^th–75^th percentiles = 21–24).

The FreeStyle meter tended to read slightly higher than the reference method (median difference = +3 mg/dL; p<0.001; Table 1) and there was a trend in this direction for the Ultra meter (Ultra: median difference = +2 mg/dL, p=0.15). The Ultra and FreeStyle meters had similar accuracy results (median values compared with reference glucose values: absolute difference 6 vs. 6 mg/dL, relative difference +1% vs. +2%, RAD 5% vs. 5% and ISO criteria met 99% vs. 98%). Both the Ultra and FreeStyle were within ±10% of the reference for 81% of the pairs.

Table 1.

Accuracy Summary Statistics for Venous Measurements According to Glucose Level Comparing Both HGMs with the Laboratory-measured Glucose Values (data from 50 subjects).

	Overall (N=1,103 Pairs)		By Reference Glucose Level(mg/dL)
	Mean(95% confidence interval)	Median(25th–75thpercentiles)	≤ 70 (N=162)	71–120 (N=410)	121–180 (N=262)	181–240 (N=150)	> 240 (N=119)
Difference*					median
Ultra	+0.6 (−1.4, +2.6)	+2 (−5, +7)	+4	+2	−1	+1	+1
Freestyle	+1.4 (+0.1, +2.7)	+3 (−2, +8)	+4	+5	+2	+1	−8
Absolute Difference†
Ultra	7.8 (6.9, 8.9)	6 (3, 10)	5	5	6	7	13
Freestyle	7.4 (6.6, 8.2)	6 (3, 10)	5	6	6	5	10
Relative Absolute Difference‡
Ultra	6% (6%, 7%)	5% (2%, 9%)	8%	6%	4%	4%	5%
Freestyle	6% (6%, 7%)	5% (2%, 9%)	9%	6%	4%	3%	4%
ISO Criteria met§					percentage
Ultra	99% (98%, 99%)	N/A	96%	99%	100%	99%	100%
Freestyle	98% (97%, 99%)	99%	97%	97%	100%	100%

^*Difference is the meter glucose minus the reference glucose value.

^†Absolute difference is the absolute value of the difference.

^‡Relative absolute difference is the absolute difference divided by the reference value (expressed as a percentage).

^§ISO criteria: for reference glucose value ≤75 mg/dL meter value within ±15 mg/dL and for reference glucose value >75 mg/dL meter value within ±20%.

Accuracy during hypoglycemia

During hypoglycemia (162 cases with reference glucose ≤70 mg/dL) the median absolute difference was 5 mg/dL for both HGM devices and ISO criteria were met by 96% and 99% of Ultra and FreeStyle values, respectively. For the 70 pairs where the reference was ≤60 mg/dL, 67 (96%) of Ultra and 70 (100%) of FreeStyle values were ≤70 mg/dL. False positive rates (HGM ≤60 mg/dL, but reference >70 mg/dL) were 5% (3/61) for the Ultra and 5% (3/62) for the FreeStyle.

Accuracy using capillary blood

There were 18 Ultra glucose measurements and 20 FreeStyle glucose measurements from capillary blood, mostly made during hypoglycemia. All of the Ultra and 90% (18/20) of the FreeStyle measurements met ISO criteria when compared with the reference glucose values obtained from venous blood.

Discussion

In the present study, we examined the accuracy of over 1,000 HGM-reference glucose pairs over a broad range of blood glucose levels in children and adolescents with T1D. These results confirmed our previous report of the accuracy of the Ultra HGM, which demonstrated median RAD of 5%, similar to the 6% in the prior report.⁶ We also demonstrated comparable accuracy of the FreeStyle HGM, with an overall median RAD of 5%. Overall, the Ultra and FreeStyle met ISO criteria for accuracy for 99% and 98% of HGM-reference pairs, respectively.

The present study was performed under optimal conditions, in that all testing was performed by trained study staff in an inpatient Clinical Research Center setting. It is unknown whether similar accuracy would be obtained under less controlled circumstances, such as in the home environment, where HGM devices are used by children and families and may be subject to greater temperature and humidity fluctuations, typical trauma of daily use, and variations in technique. It has previously been demonstrated that the accuracy and precision of HGM devices are operator dependent; coefficients of variation in older meters may be greater when used by patients compared with trained technical personnel.¹¹ However, newer meters are easier to use and may not demonstrate the same magnitude of operator dependence. It should also be noted that the current study did not employ alternate site testing, which has been shown to increase error, particularly under rapidly changing glucose conditions.^{12, 13}

It is also important to recognize that venous blood rather than capillary blood was used for almost all of the HGM measurements. In a prior study of the Ultra meter, we found that accuracy was slightly lower using capillary compared with venous samples, principally due to a tendency for the capillary sample values to be slightly higher than the reference values, which was not seen with venous sample values.⁶ In the current study, 36 of 38 of HGM measurements from capillary blood met ISO criteria.

Despite these limitations in extrapolating the present results to general HGM use, the Ultra and FreeStyle meters demonstrated sufficient accuracy over a broad range of subject characteristics and glucose concentrations to warrant their use in children and adolescents with T1D. While HGM accuracy is obviously important for diabetes self-management in the current era of intensive diabetes control, it is even more critical when considering the recent linking of HGM data to insulin delivery through continuous subcutaneous insulin infusion systems. The FreeStyle meter is presently used in a system in which its blood glucose data is communicated directly to an insulin pump. The Deltec Cozmo^® (Smiths Medical, London, UK) pump now incorporates a meter, equivalent to the FreeStyle, into its physical design (the CoZmonitor^TM). The glycemic data may be displayed on the pump screen, stored for future analysis, and utilized in the calculation of bolus doses of insulin for the user, along with the personalized insulin-carbohydrate ratio and insulin sensitivity factor. With these features, children or caregivers may now just perform a HGM test and enter the carbohydrate content of the meal, and the pumps will determine the optimal bolus dose. Further studies are needed to ascertain whether these innovations are associated with improvements in glycemic control and quality of life with children with T1D.

Figure 1.

Accuracy of HGM venous measurements compared to the laboratory-measured reference by glucose level

Appendix

Writing Committee

Stuart A. Weinzimer, MD; Roy W. Beck, MD, PhD; H. Peter Chase, MD; Larry A. Fox, MD; Bruce A. Buckingham, MD; William V. Tamborlane, MD; Craig Kollman, PhD; Julie Coffey, MSN; Dongyuan Xing, MPH; Katrina J. Ruedy, MSPH

The DirecNet Study Group:

Clinical Centers

Listed in alphabetical order with clinical center name, city, and state. Personnel are listed as (PI) for Principal Investigator, (I) for co-Investigator and (C) for Coordinators.

1. Barbara Davis Center for Childhood Diabetes, University of Colorado, Denver, CO

   H. Peter Chase, MD (PI); Rosanna Fiallo-Scharer, MD (I); Jennifer H. Fisher, ND, RN (C); Barbara Tallant, RN, MA (C)

2. Department of Pediatrics, University of Iowa Carver College of Medicine, Iowa City, IA

   Eva Tsalikian, MD (PI); Michael J. Tansey, MD (I); Linda F. Larson, RN (C); Julie Coffey, MSN (C)

3. Nemours Children’s Clinic, Jacksonville, FL

   Tim Wysocki, PhD, ABPP (PI); Nelly Mauras, MD (I); Larry A. Fox, MD (I); Keisha Bird, MSN (C); Kelly L. Lofton, RN (C)

4. Division of Pediatric Endocrinology and Diabetes, Stanford University, Stanford, CA

   Bruce A. Buckingham, MD (PI); Darrell M. Wilson, MD (I); Jennifer M. Block, RN, CDE (C); Paula Clinton, RD, CDE (C)

5. Department of Pediatrics, Yale University School of Medicine, New Haven, CT

   Stuart A. Weinzimer, MD (PI); William V. Tamborlane, MD (I); Elizabeth A. Doyle, MSN (C); Kristin Sikes, MSN (C)

Coordinating Center

Jaeb Center for Health Research, Tampa, FL

Roy W. Beck, MD, PhD; Katrina J. Ruedy, MSPH; Craig Kollman, PhD; Dongyuan Xing, MPH; Andrea Kalajian, MS; Cynthia R. Silvester

University of Minnesota Central Laboratory

Michael W. Steffes, MD, PhD; Jean M. Bucksa, CLS; Maren L. Nowicki, CLS; Carol A. Van Hale, CLS; Vicky Makky, CLS

National Institutes of Health

Gilman D. Grave, MD; Barbara Linder MD, PhD; Karen K. Winer, MD

Data and Safety Monitoring Board

Dorothy M. Becker, MBBCh; Christopher Cox, PhD; Christopher M. Ryan, PhD; Neil H. White, MD, CDE; Perrin C. White, MD