Analysis of “Accuracy Evaluation of Five Blood Glucose Monitoring Systems: The North American Comparator Trial”

Abstract

In an article in Journal of Diabetes Science and Technology, Halldorsdottir and coauthors examined the accuracy of five blood glucose monitoring systems (BGMSs) in a study sponsored by the manufacturer of the BGMS CONTOUR NEXT EZ (EZ) and found that this BGMS was the most accurate one. However, their findings must be viewed critically given that one of the BGMSs (ACCU-CHEK Aviva) was not compared against the reference measurement specified by its manufacturer, thus making it likely that it performed suboptimally. Also, the accuracy of the glucose-oxidase-based ONE TOUCH Ultra2 and TRUEtrack BGMS is likely to have been underestimated because of the expected low oxygen level in the glycolysed blood samples used to test the performance of these BGMSs under hypoglycemic conditions. In conclusion, although this study shows that EZ is an accurate BGMS, comparisons between this and other BGMSs should be interpreted with caution.

The use of blood glucose monitoring systems (BGMSs) plays an important role in the management of blood glucose level in diabetes. For this reason, many studies have examined the accuracy of many BGMSs currently available on the market to assess and compare their performances.^1–5 The purpose of the study published by Halldorsdottir and coauthors¹ was to compare the accuracy of the CONTOUR NEXT EZ (EZ; CONTOUR XT outside the United States) BGMS to that of four other BGMSs [ACCU-CHEK Aviva (ACAP), FreeStyle Freedom Lite, ONE TOUCH Ultra2 (OTU2), and TRUEtrack (TT)]. This study found that, across a broad range of blood glucose levels, EZ had the greatest accuracy (EZ had the lowest mean absolute relative difference) compared with the other four BGMSs. It must be stressed, however, that, although this study was, in general, well designed, it faces some limitations that have the potential to bring into question some of its findings.

The study of Halldorsdottir and coauthors¹ is to be commended for highlighting most of its limitations. For instance, the authors made it clear that their study was not designed to follow the DIN EN ISO 15197:2003/2013 standards. Also, they recognized that the ACAP BGMS was not compared against the reference measurement specified by its manufacturer (hexokinase-based glucose reference method), thus raising the possibility that this might have led to an underestimation of its accuracy. Finally, the authors acknowledged that, since oxygen level is likely to have been way below saturation in the glycolysed blood samples used to test the performance of BGMSs under hypoglycemic conditions, this might have contributed to the poor performance of the glucose-oxidase-based OTU2 and TT compared with what has been reported in the literature.^2–5 Although the aforementioned limitations could be addressed in future studies, it is important to stress that there are other limitations affecting this and most comparative studies on BGMSs that are more difficult to overcome as described in the following lines.

One such important limitation is that the research on BGMS performance has typically been funded and often performed by device manufacturers rather than by independent laboratories. Ideally, as recommended by oversight agencies such as the Food and Drug Administration of the United States, trials should be independent of the manufacturer and performed by an independent organization to remove the possibility of perceived, or real, bias. Short of meeting this requirement, it is important for manufacturer-funded studies to provide evidence of impartiality by highlighting the limitations of their findings and by thoroughly comparing their results with those in the literature, as is the case in the study of Halldorsdottir and coauthors.¹ Performing such comparisons between studies also provides an objective means to assess the performance of different BGMSs, particularly when they are the object of consistent reports of good or poor performance across studies.

Another limitation, also difficult to avoid, is the issue of whether the BGMS and test strip lots used for testing are representative of the overall population of BGMSs and test strips being compared. This factor together with the fact that the sample distribution in the study of Halldorsdottir and coauthors¹ was skewed toward low glucose levels compared with past studies has been proposed to explain, at least in part, why the performance of the FreeStyle Freedom Lite and OTU2 is reported by others^2–5 to be far better than in the study of Halldorsdottir and coauthors.¹

Finally, one potentially important but generally overlooked issue likely to become relevant in the near future as the accuracy of BGMSs improves further is the limitation associated with using instruments such as the YSI analyzer as the “reference method” against which BGMSs are compared. Ideally, the performance of BGMSs should be compared against a golden standard analytical method such as isotope dilution mass spectrometry rather than to an instrument such as the YSI analyzer. Since there are reported limitations with the accuracy of the YSI 2300 instrument,⁶ it follows that, as the accuracy of BGMSs improves in the future, it is possible that their performances will eventually approach or exceed that of reference analyzers, thus invalidating the use of these analyzers to compare BGMSs. However, one may argue that, by the time the accuracy of BGMSs matches that of reference blood glucose analyzers, there will be little need to improve further the high level of accuracy then attained by such BGMSs. What this interpretation assumes, however, is that the glucose readings provided, for instance, by different YSI analyzers differ little between laboratories. Unfortunately, to the best of our knowledge, no study has performed a multicenter study to compare the interlaboratory accuracy of YSI analyzers (and other analyzers) calibrated against the same standard. Also, no study has evaluated the performance of current reference glucose analyzers (e.g., YSI analyzer) over a range of blood glucose levels compared with mass spectrometry. It is important to assess the level interlaboratory variability in the accuracy of reference analyzers, particularly considering that differences in the accuracy of different BGMSs may in the future reflect to a large extent differences in the accuracy of the reference analyzers used for comparisons and calibrations rather than that of the BGMSs being investigated.

Overall, although this study shows that the EZ has a greater accuracy compared with other BGMSs, these comparisons should be interpreted with caution, as the experimental conditions adopted in this study were suboptimal for the ACAP, TT, and OTU2. However, the authors make a strong case that EZ is a very accurate instrument.

Glossary

(ACAP)

ACCU-CHEK Aviva

(BGMS)

blood glucose monitoring system

(EZ)

CONTOUR NEXT EZ

(OTU2)

ONE TOUCH Ultra2

(TT)

TRUEtrack