Abstract
Cleared blood glucose monitor (BGM) systems do not always perform as accurately for users as they did to become cleared. We performed a literature review of recent publications between 2010 and 2014 that present data about the frequency of inaccurate performance using ISO 15197 2003 and ISO 15197 2013 as target standards. We performed an additional literature review of publications that present data about the clinical and economic risks of inaccurate BGMs for making treatment decisions or calibrating continuous glucose monitors (CGMs). We found 11 publications describing performance of 98 unique BGM systems. 53 of these 98 (54%) systems met ISO 15197 2003 and 31 of the 98 (32%) tested systems met ISO 15197 2013 analytical accuracy standards in all studies in which they were evaluated. Of the tested systems, 33 were identified by us as FDA-cleared. Among these FDA-cleared BGM systems, 24 out of 32 (75%) met ISO 15197 2003 and 15 out of 31 (48.3%) met ISO 15197 2013 in all studies in which they were evaluated. Among the non-FDA-cleared BGM systems, 29 of 65 (45%) met ISO 15197 2003 and 15 out of 65 (23%) met ISO 15197 2013 in all studies in which they were evaluated. It is more likely that an FDA-cleared BGM system, compared to a non-FDA-cleared BGM system, will perform according to ISO 15197 2003 (χ2 = 6.2, df = 3, P = 0.04) and ISO 15197 2013 (χ2 = 11.4, df = 3, P = 0.003). We identified 7 articles about clinical risks and 3 articles about economic risks of inaccurate BGMs. We conclude that a significant proportion of cleared BGMs do not perform at the level for which they were cleared or according to international standards of accuracy. Such poor performance leads to adverse clinical and economic consequences.
Keywords: accuracy, BG monitor, FDA, glucose, ISO, performance
Cleared blood glucose monitors (BGMs) do not always function as well as they did to become cleared. Poorly performing BGMs are risky to make treatment decisions and calibrate CGMs (continuous glucose monitors). Patients and health care professionals are now demanding accurate BGMs.1,2
BGMs were cleared according to an international standard, ISO 15197 2003, in both the United States and Europe until ISO 1597 2013 was developed 2 years ago. Now Europe is preparing to use ISO 15197 2013, which will take effect in 2016.3 After the new ISO standard was released 2 years ago, FDA declined to adopt this standard, and instead elected to specify more rigorous requirements for personal BGMs based on their analysis of the need for accuracy and comments provided to this agency at a public meeting on BGM accuracy in 2010.4 In January 2014 FDA released a draft guidance for these devices which is currently being reviewed based on public comments.5 It is expected that FDA will release a final guidance on these products this year. FDA also released a draft guidance for professional BGMs in 2014 for devices used in the hospital and by health care professionals on their patients. FDA ISO 15197 2013 in its scope section states that this standard does not apply to glucose meters intended for use in medical applications other than self-testing for the management of diabetes mellitus. This article is only about personal BGM systems.
According to ISO 15197 2003 the minimum accuracy criteria for BGMs are that 95% of glucose levels must be (1) for glucose < 75 mg/dl—within 15 mg/dl of reference; and (2) for glucose ≥ 75 mg/dl—within 20% of reference.6 According to ISO 15197 2013, 95% of glucose levels must be (1) for glucose < 100 mg/dl—within 15 mg/dl of reference; and (2) for glucose ≥ 100 mg/dl—within 15 % of reference. Furthermore 99% of glucose results must be within the Parkes (consensus) error grid zone A or B.7 According to FDA 2014 draft guidance, the minimum accuracy requirements for personal BGMs are (1) 95% of glucose results must be within 15% of reference; and (2) 99% of glucose results must be within 20% of reference.5
After a BGM is cleared by a regulatory agency with adequate performance, investigators have found in some cases that the performance of a cleared BGM does not match the level of performance required by regulatory agencies for initial clearance. BGMs are regulated for accuracy to minimize errors. This is because information from these devices is used for making treatment decisions and erroneous readings can lead to incorrect treatments which in turn can lead to excessive lowering of blood glucose values and hypoglycemic episodes or else to inadequate lowering of blood glucose levels.8 Furthermore BGM readings are used to calibrate CGMs so there is a risk that inaccurate BGMs could lead to inaccurate calibration of CGMs and render these products less accurate.
The significance of BGM performance not matching the level of performance for which they were cleared can be assessed by reviewing the literature of 2 types of studies: (1) reports published in PubMed-indexed journals over the past 5 years between 2010 and 2014, which assess whether particular BGMs function up to the standards for which they were cleared or similar well defined standards; and (2) modeling studies that present simulated performance of inaccurate BGMs and the adverse clinical or economic outcomes attributed to inaccuracy.
Methods
Performance of BGM Systems
We performed a literature review through PubMed of recent publications (published between 2010 and 2014) that present data about the frequency of inaccurate performance using ISO 15197 2003, ISO 15197 2013, or FDA draft 2014 as target standards for cleared BGMs. Articles were also identified through searching the reference list of selected articles and by using the Google Scholar database to check whether articles that were identified met inclusion criteria. Articles were in selected if they were published in PubMed-indexed journals, written in English and if they described the performance against ISO 15197 2003, 15197 2013, or FDA draft 2014. Articles were selected if they described studies of at least 2 BGM systems from different manufacturers on human subjects or at least 3 different BGM systems with at least 3 strip lots per system for BGM systems from a single manufacturer. We selected the most recent 5-year time frame to study the performance of products that are likely to be currently on the market. We defined ISO 15197 2003 positive and ISO 15197 2013 positive as meeting the analytical targets specified by these standards, and we did not address the clinical accuracy of any BGM systems in our analysis.
Risks of Inaccurate BGM Systems
We performed 2 additional literature reviews through PubMed of publications that present data about the clinical risks of inaccurate BGMs for making treatment decisions or calibrating CGMs and about the economic risks of using such products. Clinical and economic risks can be estimated from either empirically collected data from poorly performing inaccurate BGMs or from simulated modeled data. However, the use of an inaccurate BGM to make treatment decisions and see what types of complications ensue would be unethical. Furthermore, it would be very difficult to construct a BGM with an exact target level of inaccuracy to be part of a trial. Such an empiric study of the clinical outcomes or costs of inaccurate BG monitoring, to our knowledge, has never been reported. We therefore searched for modeled simulated patient data because no empiric data was available to address the topic of clinical risks or economic risks of poorly performing BGM systems. We did not restrict the time frame because the consequences of poor BGM performance are not affected by whether a product is or is not currently available.
Results
Performance of BGM Systems
We found 10 articles and 1 letter.9-19 All the BGMs tested were available in either Europe or in the United States or both. Not all BGMs tested were necessarily cleared by FDA. To check which BGMs were cleared in the United States by FDA, we went to the FDA 510(k) database, which lists and provides information on all cleared BGMs.20 We defined a BGM as FDA-cleared if it was listed in this database on the date we performed the search, which was February 15, 2015. That same day we also checked for a new name for each cleared BGM on the “CLIA Currently Waived Analytes” list of waived laboratory test systems.21
No publications were found that specifically addressed performance against FDA draft 2014. In every publication except 1, no conclusion could be made of the percentage of the selected cleared monitors that could meet this standard. The requirements for clearance of a BGM data set for data points of 100 mg/dl or greater are the same for both ISO 15197 2013 and FDA draft 2014, but for data points below 100 mg/dl the requirements are stricter for FDA draft 2014. Therefore, a product failing to meet ISO 15197 2013 would not meet FDA draft 2014, however a product meeting ISO 15197 2013 might still not necessarily meet FDA draft 2014. The only study where performance against FDA 2014 draft guidance could be estimated was one where all BGMs failed to meet ISO 15197 2003,6 so therefore these BGMs would all fail to meet the more stringent ISO 15197 2013,7 and the 2014 draft FDA guidance.5 ISO 15197 2013 is more strict than ISO 15197 2003. If a system failed to meet ISO 15197 2003 but it was not tested against ISO 15197 2013,9 then the system was defined as not meeting ISO 15197 2013. If a system met ISO 15197 2013 but it was not tested against ISO 15197 2003,18 then the system was defined as meeting ISO 15197 2013. If a system was tested with multiple lots of strips at 1 site and not all lots met a particular standard,13 then that system was defined as not meeting the standard.
If a system was tested by multiple investigators and failed to meet a particular standard in at least 1 study, then that system was defined as not meeting the standard. Given our binary standard of passing or not passing, a system that sometimes failed was defined as not meeting the standard. Only named BGM systems were included in the analysis.
The performance of BGMs in these 11 recent publications is presented in Table 1. The performance of each tested BGM system by ISO 15197 2003 and/or ISO 15197 2013 standards as reported in each publication is presented in the appendix.
Table 1.
Performance of BGM Systems in 11 Studies Using ISO 15197 2003 and ISO 15197 2013 as Standards.
| Study | Reference | Year | First Author | BGMs (n) | ISO 2003+ (%) | ISO 2013+ (%) |
|---|---|---|---|---|---|---|
| 1 | 9 | 2010 | Freckmann | 27 | 59 | ? |
| 2 | 10 | 2010 | Sonmez | 5 | 0 | 0 |
| 3 | 11 | 2012 | Tack | 5 | 60 | 40 |
| 4 | 12 | 2012 | Freckmann | 43 | 67 | 49 |
| 5 | 13 | 2012 | Baumstark | 5 | 40 | 20a |
| 6 | 14 | 2013 | Brazg | 7 | 43 | 14 |
| 7 | 15 | 2014 | Freckmannb | 10 | 100 | 80 |
| 8 | 16 | 2014 | Pfützner | 6 | 100 | 83 |
| 9 | 17 | 2014 | Link | 3 | 100 | 100 |
| 10 | 18 | 2014 | Hasslacher | 27 | ? | 41 |
| 11 | 19 | 2014 | Huang | 2 | 100 | 100 |
Question mark indicates BGM systems were not evaluated against this standard.
Includes a BGM system that did not meet ISO 15197 2013 criteria in 1 of 4 lots of strips.
This study tested 12 BGM systems and identified 10 of them. Only named BGM systems were included in the analysis.
Among the 11 studies, a total of 98 different systems were reported in at least 1 article; however there were some systems tested more than once by various investigators. Table 2 presents the performance of all 98 of the BGM systems that were tested in any of the 11 reviewed studies.
Table 2.
Performance of BGM Systems in 11 Studies Using ISO 15197 2003 and ISO 15197 2013.
| Baumstark et al 2012 |
Brazg et al 2013 |
Freckmann et al 2010 |
Freckmann et al 2012 |
Freckmann et al 2014 |
Hasslacher et al 2014 |
Huang et al 2014 |
Link et al 2014 |
Pfützner et al 2014 |
Sonmez et al 2010 |
Tack et al 2012 |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BGM system | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003a | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003b | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 | ISO 15197 2003 | ISO 15197 2013 |
| Accu-Chek Active | Yes | Yes | Yes | |||||||||||||||||||
| Accu-Chek Aviva | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | |||||||||||
| Accu-Chek Aviva Nano | Yes | Yes | No | |||||||||||||||||||
| Accu-Chek Aviva Plus | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Compact | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Compact Plus | Yes | Yes | ||||||||||||||||||||
| Accu-Chek GO | Yes | Yes | No | No | ||||||||||||||||||
| Accu-Chek Mobile (maltose dependent) | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Mobile (maltose independent) | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Mobile (maltose dependence not specified) | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Performa (maltose dependent) | Yes | Yes | ||||||||||||||||||||
| Accu-Chek Performa (maltose independent) | Yes | Yes | Yes | Yes | ||||||||||||||||||
| Accu-Chek Performa Nano | Yes | Yes | ||||||||||||||||||||
| Advocate Redi-Code | No | No | ||||||||||||||||||||
| alphacheck professional | Yes | Yes | ||||||||||||||||||||
| Ascensia Contour | Yes | |||||||||||||||||||||
| Beurer GL 30 | No | No | ||||||||||||||||||||
| Beurer GL32 | No | No | ||||||||||||||||||||
| Beurer GL40 | Yes | Yes | ||||||||||||||||||||
| BG Star | Yes | Yes | Yes | Yes | No | Yes | Yes | |||||||||||||||
| Biocheck TD-4225 | No | No | ||||||||||||||||||||
| Bionime Rightest GM101 | Yes | |||||||||||||||||||||
| Bionime Rightest GM300 | Yes | |||||||||||||||||||||
| Breeze | No | |||||||||||||||||||||
| CareSens N | Yes | Yes | ||||||||||||||||||||
| CareSens N POP | Yes | Yes | ||||||||||||||||||||
| Clever Chek TD-4222 | No | No | ||||||||||||||||||||
| Contour | Yes | No | No | No | ||||||||||||||||||
| Contour Plasma | No | |||||||||||||||||||||
| Contour TS | No | No | No | No | ||||||||||||||||||
| ContourUSB | Yes | No | Yes | Yes | ||||||||||||||||||
| Contour USB Next | Yes | Yes | ||||||||||||||||||||
| Contour XT | Yes | Yes | Yes | Yes | ||||||||||||||||||
| Element | Yes | No | Yes | No | ||||||||||||||||||
| Embrace | No | No | ||||||||||||||||||||
| EZ Smart | No | No | ||||||||||||||||||||
| Finetest | Yes | |||||||||||||||||||||
| Finetest Auto-coding | No | No | ||||||||||||||||||||
| FineTouch | No | No | No | |||||||||||||||||||
| Fora TD-4227 | No | No | ||||||||||||||||||||
| FreeStyle Freedom | Yes | |||||||||||||||||||||
| FreeStyle Freedom Lite | Yes | Yes | Yes | Yes | Yes | Yes | ||||||||||||||||
| FreeStyle Lite | Yes | Noc | Yes | Yes | Yes | Yes | Yes | Yes | Yes | |||||||||||||
| Futura Monometer | No | No | ||||||||||||||||||||
| GE 200 | Yes | Yes | ||||||||||||||||||||
| GE 100 | Yes | Yes | ||||||||||||||||||||
| GL 40 | Yes | No | No | |||||||||||||||||||
| GL 50 | No | |||||||||||||||||||||
| Gluco-test Plus+ TD 4230 | Yes | Yes | ||||||||||||||||||||
| Gluco-Test TD-4209 | No | No | ||||||||||||||||||||
| GlucoCard-X-Meter | Yes | |||||||||||||||||||||
| GlucoCheck Classic | No | No | ||||||||||||||||||||
| GlucoCheck Comfort | Yes | Yes | ||||||||||||||||||||
| GlucoCheck XL | No | No | Yes | No | ||||||||||||||||||
| Glucofix mio | No | No | ||||||||||||||||||||
| GlucoHexal | No | No | ||||||||||||||||||||
| GlucoHexal II | No | No | ||||||||||||||||||||
| Glucomen LX | No | |||||||||||||||||||||
| Glucomen LX plus | No | |||||||||||||||||||||
| GlucoRx (TD-4230) | No | No | ||||||||||||||||||||
| GlucoSmart Swing | Yes | No | No | |||||||||||||||||||
| GlucoTel | No | No | ||||||||||||||||||||
| GM700 | Yes | Yes | ||||||||||||||||||||
| iBG Star | Yes | No | Yes | Yes | Yes | Yes | ||||||||||||||||
| (continued) | ||||||||||||||||||||||
| iDia | No | No | ||||||||||||||||||||
| IME-DC BG meter | No | No | ||||||||||||||||||||
| IME-DC Fidelity | No | No | ||||||||||||||||||||
| iXell | Yes | No | ||||||||||||||||||||
| iXell OLED | Yes | Yes | ||||||||||||||||||||
| microdot | Yes | No | ||||||||||||||||||||
| my glucohealth | No | |||||||||||||||||||||
| myLife Pura | Yes | Yes | Yes | Yes | ||||||||||||||||||
| myLife Unio | Yes | Yes | ||||||||||||||||||||
| Omnitest 3 | Yes | No | Yes | No | No | |||||||||||||||||
| OneTouch Select | No | No | ||||||||||||||||||||
| OneTouch Ultra 2 | Yes | Yes | Yes | |||||||||||||||||||
| OneTouch Ultra Easy | Yes | Yes | Yes | No | No | |||||||||||||||||
| OneTouch Verio | Yes | Yes | ||||||||||||||||||||
| OneTouch Verio IQ | Yes | Yes | ||||||||||||||||||||
| OneTouch Verio Pro | No | No | No | No | Yes | Yes | No | |||||||||||||||
| OneTouch VITA | Yes | Yes | Yes | Yes | ||||||||||||||||||
| Optium Xceed | Yes | No | No | |||||||||||||||||||
| Prodigy Voice | No | No | ||||||||||||||||||||
| Pura | Yes | Yes | ||||||||||||||||||||
| Pura/mylife Pura | No | No | ||||||||||||||||||||
| SeniorLine GM210 | No | No | ||||||||||||||||||||
| SensoCardPlus | Yes | |||||||||||||||||||||
| Smart Lab Mini | No | |||||||||||||||||||||
| Smart Lab Sprint | Yes | No | ||||||||||||||||||||
| smartLAB genie | No | No | ||||||||||||||||||||
| smartLAB global | No | No | ||||||||||||||||||||
| Stada Glucocheck | No | No | ||||||||||||||||||||
| TRUEbalance | No | No | ||||||||||||||||||||
| WaveSense Jazz | Yes | Yes | ||||||||||||||||||||
| WaveSense Presto | Yes | No | ||||||||||||||||||||
| Wellion Calla | No | |||||||||||||||||||||
| Wellion CALLA light | No | No | ||||||||||||||||||||
| Wellion Linus | Yes | |||||||||||||||||||||
Indicates that in this study BGM systems were tested only against ISO 15197 2003 and if a system failed to meet this standard, then it was defined as not meeting ISO 15197 2013.
Indicates that in this study BGM systems were tested only against ISO 15197 2013 and if a system met this standard, then it was defined as meeting ISO 15197 2003.
Indicates that this BGM system did not meet ISO 15197 2013 in 1 of 4 lots of strips.
The data in Table 2 indicate that 53 of the 98 (54%) tested BGM systems met ISO 15197 2003 and 31 of the 98 (32%) tested BGM systems met ISO 15197 2013. An additional 7 of the 98 (7.1%) passed ISO 15197 2003 and 10 of the 98 (10.2%) passed ISO 15197 2013 in some but not all of the studies which evaluated their accuracy. Of the 98 BGM systems tested, 33 systems were identified by us as FDA-cleared. Of those 33 systems, 32 were evaluated for ISO 15197 2003 and 31 were evaluated for ISO 15197 2013.
The performance of each of the FDA-cleared BGM systems is presented in Table 3. Among FDA-cleared BGM systems that were tested in these 11 publications, 24 out of 32 (75%) met ISO 15197 2003 and 15 out of 31 (48%) met ISO 15197 2013. In addition, 2 out of 33 (6.1%) met ISO 15197 2003 and 6 out of 31 (19%) met ISO 15197 2013 criteria in some but not all of the studies which evaluated their accuracy. Of the 98 BGM systems tested, 65 were not FDA-cleared. Among these systems 29 of 65 (44%) met ISO 15197 2003 and 15 out of 65 (23%) met ISO 15197 2013. Another 5 out of 65 (7.7%) met ISO 15197 2003 and 4 out of 65 (6.2%) met ISO 15197 2013 criteria in some but not all of the studies which evaluated their accuracy. It is more likely that an FDA-cleared BGM system, compared to a non-FDA-cleared BGM system, will perform according to ISO 15197 2003 (χ2 = 6.2, df = 3, p = 0.04) and ISO 15197 2013 (χ2 = 11.4, df = 3. None of these studies evaluated BGM systems based on the performance targets described in the draft 2014 FDA guidance. Given that this guidance is more stringent than ISO 2013, we assume that any BGM system that did not meet ISO 2013 standards will also fail to meet 2014 draft FDA standards.
Table 3.
FDA-Cleared BGM Systems Meeting ISO 15197 2003 and ISO 15197 2013 Standards.
| Meter Name | Company | ISO 15197 2003 (32/33 evaluated) | ISO 15197 2013 (31/33 evaluated) |
|---|---|---|---|
| Accu-Chek Active | Roche Diagnostics | Yes | Yes |
| Accu-Chek Aviva | Roche Diagnostics | Yes | Noa |
| Accu-Chek Aviva Plus | Roche Diagnostics | Yes | Yes |
| Accu-Chek Compact | Roche Diagnostics | Yes | Yes |
| Accu-Chek Compact Plus | Roche Diagnostics | Yes | Yes |
| Accu-Chek Go | Roche Diagnostics | Nob | Nob |
| Accu-Chek Performa (maltose dependent) | Roche Diagnostics | Yes | Yes |
| Accu-Chek Performa (maltose independent) | Roche Diagnostics | Yes | Yes |
| Advocate Redi-Code | TaiDoc Technology Corp | No | No |
| alphacheck professional | i-SENS inc | Yes | Yes |
| Ascensia Contour | Bayer HealthCare LLC | Yes | ? |
| BG Star | AgaMatrix | Yes | Noc |
| Breeze | Bayer Health Care | ? | No |
| CareSens N | i-SENS inc | Yes | Yes |
| CareSens N POP | i-SENS inc | Yes | Yes |
| Clever Chek TD-4222 | Taidoc Technology Corp. | No | No |
| Contour | Bayer Consumer Care | Nob | No |
| Contour USB | Bayer Consumer Care AG | Yes | Nob |
| Contour USB Next | Bayer Health Care | Yes | Yes |
| Element | Infopia Co | Yes | No |
| EZ Smart | Tyson Bioresearch | No | No |
| FreeStyle Freedom | Abbott Diabetes Care Inc | Yes | ? |
| FreeStyle Lite | Abbott Diabetes Care Inc | Yes | Nod |
| GM700 | Bionime | Yes | Yes |
| iBG Star | AgaMatrix | Yes | Noe |
| microdot | Cambridge Sensors Limited | Yes | No |
| OneTouch Select | Lifescan | No | No |
| OneTouch Ultra 2 | LifeScan Inc | Yes | Yes |
| OneTouch Verio | Lifescan Inc | Yes | Yes |
| OneTouch VITA | Lifescan Inc | Yes | Yes |
| Prodigy Voice | Diagnostic Devices Inc | No | No |
| TRUEbalance | Nipro Diagnostics | No | No |
| WaveSense Jazz | AgaMatrix | Yes | Yes |
| FDA-cleared BGM systems meeting standards when tested for ISO standards | 24/32 (75%) | 15/31 (48%) | |
Question mark indicates the BGM system was not evaluated against this standard. Of the 98 BGM systems, 33 were FDA-cleared. Of those, 32 were evaluated for ISO 15197 2003 and 31 were evaluated for ISO 15197 2013.
Did not meet criteria in 1 of 5 studies.
Did not meet criteria in 1 of 2 studies.
Did not meet criteria in 1 of 4 studies.
Did not meet criteria in 1 of 4 lots of strips in a single study.
Did not meet criteria in 1 of 3 studies.
A significant proportion of BGM systems that are available in the United States and in Europe do not meet the so-called old performance standards specified by ISO 15197 2003 for which many currently cleared products were held to at the time of their clearance. It is not possible to check whether any given product in the United States was cleared according to that standard or to an even earlier standard, but most products have come to FDA for modifications in the past 12 years and most of them have had to then verify their performance to at least the 2003 standard. Whereas neither ISO 15197 2013 nor draft 2014 are currently used in the United States, the figures for the proportion of BGM systems that meet the so-called new standards can be estimated by the percentage adhering to ISO 15197 2013. FDA draft 2014 is more rigorous than ISO 15197 2013, so the percentage of BGM systems adhering to current FDA standards (assuming that the performance requirements do not change between issuance of the draft 2014 guidance and the upcoming final guidance) would be no higher and possibly lower.
Risks of Inaccurate BGM Systems
We identified 7 articles about clinical risks of inaccurate BGMs. They are presented in Table 4. We also identified 3 articles about economic risks of inaccurate BGMs. They are presented in Table 5.
Table 4.
Clinical Risks of BGM Inaccuracy—Literature Review.
| Authors | Year | Reference | Patient type | Modeled outcome due to BGM error |
|---|---|---|---|---|
| Boyd, Bruns | 2001 | 22 | Any | Insulin dosing errors |
| Boyd, Bruns | 2009 | 23 | Inpatients | Intensive intravenous insulin dose errors |
| Breton, Kovatchev | 2010 | 24 | Any adults | Hypoglycemia detection, risk of hypoglycemia, GV, and mean glycemia |
| Karon et al | 2010 | 26 | Inpatients | Insulin dosing errors during tight glycemic control |
| Virdi, Mahoney | 2012 | 27 | Any at mealtime | Insulin dosing errors due to errors of BGMs and carbohydrate estimation |
| Karon et al | 2013 | 28 | Inpatients | Insulin dosing errors during moderate glycemic control |
| Thomas et al | 2014 | 29 | Inpatient babies | CGM miscalibration, inaccurate hypoglycemia and hyperglycemia detection |
Table 5.
Economic Risks of BGM Inaccuracy—Literature Review.
| Authors | Year | Reference | Patient type | Modeled intervention leading to savings |
|---|---|---|---|---|
| Budiman et al | 2013 | 30 | Insulin users in the United States | Use of the least likely instead of the most likely BGM to cause hypoglycemia |
| Schnell et al | 2013 | 31 | Insulin users in Germany | Improvement in BGM accuracy from error of 20% down to 5% |
| Schnell, Erbach | 2014 | 32 | Insulin users in Germany | Improvement in BGM accuracy from error of 20% down to 15% and 10% |
Clinical Risks
The first article to link analytical accuracy of a BGM with glycemic control was written by Boyd and Bruns in 2001.22 Errors by BGMs were modeled to be 5% or 10% away from reference. Total Error was defined as bias + imprecision. A published sliding scale to dose insulin was used for BG <60 mg/dl up to >250 mg/dl. For BGMs with total error 5%, insulin dose errors occurred with 8-23% of doses. For BGMs with total error 10%, insulin dose errors occurred with 16-45% of doses.
In 2009 Boyd and Bruns conducted a proof-of-principle computer simulation of hospitalized patients on intensive insulin therapy.23 They modeled the effects of BGM inaccuracy and imprecision (expressed as the coefficient of variation, which is the standard deviation divided by the mean and then multiplied by 100 percent) on 2 regimens for intensive insulin therapy: 1 from the University of Washington and the other from Yale University. They found that simulation of the clinical effects of measurement error was an attractive approach for assessing BGM performance. They saw how the accuracy of BGMs contributed to various improved adverse glycemic outcomes. They concluded that the performance of glucose measurement is a critical but overlooked factor in the success of tight glycemic control programs.
In 2010 Breton and Kovatchev, supported by a grant from Diabetes Technology Society, reported the impact of modeled BGM errors in type 1 diabetes on (1) detection of hypoglycemia, (2) risk for hypoglycemia, (3) glucose variability, and (4) average control.24 They reported that for 5 magnitudes of error, when the BGM error increased from 0% to 5% to 10% to 15% (the level specified by ISO15197 2013 for most data points) to 20% (the level specified by ISO15197 2003 for most data points) 4 outcomes were observed. (1) The probability of failing to detect a hypoglycemic BG levels of 60 mg/dl or lower as a hypoglycemic level (defined as a BGM reading of less than 70 mg/dl) will increase for these 5 magnitudes of error from 0% to essentially 0% to 1% to 3.5% to 10%. (2) The incidence of a correction bolus (administered at a reference BG level of 200 mg/dl intended to bring the BG down to 100 mg/dl if there were no BGM error) resulting in overshoot hypoglycemia (BG < 70 mg/dl) increased from 0% to 0% to 0% to 0.1% to 10%. (3) The variability of preprandial and peak postprandial glucose levels increased as the magnitude of BGM error increased. Using a tool for quantifying minimum/maximum variability known as control variability grid analysis (CVGA),25 when the permitted error increased from 5% to 20%, then the percentage of points within the desired CVGA A+B zones decreased from 97 to 85% and the percentage of points in the dangerous C, D, and E zones increased 5-fold from 3% to 15%. (4) The greater the BGM error, the more the target mean BG had to be scaled back to maintain the incidence of hypoglycemia at the baseline level of an episode on 15% of days. The incidence of hypoglycemia during a day increased with increasing magnitude of permitted BGM error from 15% at 0% error to 15.2% at 5% error to 18.8% at 10% error to 22% at 15% error to 25.6% at 20% error. To scale down the risk of hypoglycemia to the base case incidence of 15%, the simulated patients had to decrease their insulin doses, which then resulted in a progressive rise in mean glycemia as evidenced by the A1C level. As the permitted error rose from 0% to 5% to 10% to 15% to 20% the A1C increased from 7.00% to 7.01% to 7.12% to 7.26% to 7.40%.
In 2010 Karon et al performed simulation modeling of insulin dosing to achieve tight glycemic control based on glucose monitor performance.26 They worked with 29,920 glucose values from inpatients receiving tight glycemic control along with the insulin dosing regimen that was in effect at the hospital at the time of BG testing. Simulation models were used to relate BGM analytical errors of 10%, 15%, or 20% errors to insulin dosing errors. The purpose of the study was to estimate the amount of BGM error that was tolerable for safe management of patients on tight glycemic control.
Table 6 presents results of the estimated frequency of insulin dosing errors according to the magnitude of BGM error according to simulation by Karon et al.26
Table 6.
Frequency of Insulin Dosing Errors as a Function of Error Condition for 29,920,000 Simulated Glucose Values Using the Gaussian Error Model.
| Error condition | 10% error (%) | 15% error (%) | 20% error (%) |
|---|---|---|---|
| No change | 71.4 | 58.7 | 48.8 |
| 1 category | 28.4 | 39.3 | 44.8 |
| 2 category | 0.2 | 2.0 | 6.1 |
| ≥3 category | 0.0 | 0.02 | 0.3 |
In 2012 Virdi and Mahoney simulated the likelihood of insulin dosing errors based on of various levels of inaccurate measurements by BGM systems and various errors in carbohydrate estimation.27 They modeled the performance of 1 BGM system that had 95% of its results within 10% of reference, 2 BGM systems that had 95% of their results within 15% of reference (similar to ISO 15197 2013 except for BG levels below 75 mg/dl), and 2 BGM systems that had 95% of their results within 20% of reference (similar to ISO 15197 2003 except for BG levels below 75 mg/dl). The simulation study was performed with 3 different ranges of preprandial glycemia. When carbohydrate estimation was accurate, then insulin was correctly dosed 50.2-98.5% of the time (see Table 7), but when there was a 20% error in carbohydrate estimation, the likelihood of a correct insulin dose dropped to 27.2-80.1%. The range of likelihood of correct dosing depended on the amount of error in the BGM system and the range of preprandial glycemia. In the presence of carbohydrate estimation errors, the likelihood of an insulin dosing error increased, but the influence of BGM system error was blunted. The authors pointed out that an insulin requiring patient who tests SMBG (self-monitoring of blood glucose) with each meal checks approximately 1000 glucose readings per year and an increase of 1% in the frequency of incorrect dosing might result in 10 hypoglycemic or hyperglycemic episodes per year.
Table 7.
Percentage Likelihood of On-Target Insulin Dosages Based on Blood Glucose Meter Error but No Carbohydrate Estimation Error.
| BGM | Glucose (mg/dl) |
|||
|---|---|---|---|---|
| 90-150 | 150-270 | 270-450 | 90-450 | |
| BGM 1 ±10% | 98.5 | 89.8 | 71.0 | 81.8 |
| BGM 2 ±15% | 96.2 | 83.1 | 60.9 | 74.1 |
| BGM 3 ±15% | 96.3 | 82.9 | 61.0 | 74.1 |
| BGM 4 ±20% | 91.9 | 73.4 | 50.1 | 64.6 |
| BGM 5 ±20% | 92.2 | 73.6 | 50.2 | 64.3 |
In 2013, Karon et al compared the predicted distribution of errors in glucose measurement during moderate glycemic control for a simulated population of hospitalized ICU patients with a set of actually hospitalized patients.28 They analyzed 4017 paired data points of reference/BGM glucose values and found that their model of BGM inaccuracy predicted a total error of their hospital’s POC BGM as 15-20%. They generated with this prediction because the distribution of 1-, 2-, and 3-step errors in insulin dosing for BGM errors in the 2 empiric population was similar to a modeled population with a BGM total error of this magnitude. This simulation method can estimate the performance of a hospital’s BGM if their model is correct. They did not go out and empirically test their hospital’s BGM, however, to test the accuracy of their model. Their model also demonstrated that BGMs that limit total error to 15% or less are not generally associated with large insulin dosing errors.
In 2014 Thomas and colleagues noted that CGMs use BGM measurements for calibration and their performance could be affected by the accuracy of the BGMs.29 They created a model of CGM performance based on published accuracy data for each of 3 identified BGMs. They also included timing errors along with glucose concentration errors in the simulations of incorrect calibrations. Timing errors alone had little effect on CGM performance. Measurement errors had a significant adverse effect on CGM performance. They found that a BGM with a high bias when calibrating a CGM will result in underreporting hypoglycemia. High bias causes the readings to be pulled upward and hypoglycemic episodes will be reported as shorter duration. A BGM that has a low bias when calibrating a CGM will result in overreporting hypoglycemia. Low bias causes the CGM reading to be pulled downward and hypoglycemic episodes will be reported as longer duration. The authors pointed out that if one compares outcomes data from separate studies of various interventions that affect the incidence of hypoglycemia, then the results could be affected by the type of BGM used for CGM calibration in each study.
Economic Risks
The first of 3 studies to model the economic impact of inaccurate BGM systems was published by Budiman and colleagues in 2013.30 They estimated the number of insulin users in the United States is 958 thousand type 1 and 1.35 million type 2 patients. They assumed that patients will choose from 1 of 5 specific BGMs which are manufactured by 4 leading diagnostics companies, including Abbott Diabetes Care (Alameda, CA), Bayer Vital GmbH (Leverkusen, Germany), LifeScan, Inc (Chesterbrook, PA), and Roche Diagnostics (Mannheim, Germany). They modeled the performance of these BGMs from data in the medical literature and then left the 5 levels of performance blinded in their article. Savings will accrue from an improved outcome, which is avoidance of 296,000 hypoglycemic episodes annually by using the BGM associated with the lowest incidence of hypoglycemia instead of the BGM with the highest incidence. The annual savings was estimated to be approximately $339 million for type 1 and $121 million for type 2.
In 2013 Schnell and colleagues modeled the potential cost savings related to greater accuracy of BGMs based on outcomes and costs in Germany.31 They based the cost savings on the number of insulin users and the costs of BGM testing in Germany, as well as the impact of BGM testing on mean glycemia and the incidence of hypoglycemia and myocardial infarctions. A reduction of BGM error from 20% to 5% was associated with reductions of 10% in severe hypoglycemia, 0.39% in A1C, and 0.5% in myocardial infarctions. Based on estimated numbers of 390, 000 type 1 and 2.3 million type 2 insulin users in Germany, these improved outcomes could result in decreased costs of more than €9.4 million and €55.5, respectively. In 2014 this team calculated the cost savings for type 1 and type 2, respectively, due to the intermediate benefits of reducing BGM error from 20% to 15% (€1.02 million and €6.03 million) and of reducing BGM error from 20% to 10% (€3.41 million and €20.13 million).32 These improvements in performance were, respectively, associated with reductions of 1% in severe hypoglycemia, 0.14% in A1C, and 0.18% in myocardial infarctions (20% to 15%) and reductions of 3.5% in severe hypoglycemia, 0.28% in A1C, and 0.5% in myocardial infarctions (20% to 10%).
Discussion
According our review of 11 publications in the medical literature, it is evident that a significant proportion of cleared BGMs do not perform at the level for which they were cleared, which is ISO 2003 in many cases. An even higher percentage fail to perform according to the current international standard, which is ISO 2013. It is likely that even fewer products on the market perform according to FDA draft 2014 and it is highly likely that even if it is modified, the final FDA Guidance will be no less stringent than ISO 15197 2013.
One possible limitation of this analysis is that 4 of the 11 studies10,11,14,18 did not follow the ISO-specified distribution of glucose concentrations for their data sets and had they instead followed this distribution, then their performance might have been better or worse. Some of the tested systems in the reviewed studies were obtained directly from the manufacturer, while others were bought on the market. A manufacturer of a product whose distribution they exclusively control might respond to an investigator’s request for BGMs by selecting only atypically high-quality batches of supplies shipped by atypically high-quality temperature-controlled methods. In that case, the performance of these BGM systems might exceed that achieved by products sourced through the usual supply chain. There is therefore a possibility that for some BGM systems discussed in this article overestimates the likelihood of adequate performance against standards when testing is performed by a patient.
The medical literature also clearly demonstrates that adverse clinical outcomes are associated with the use of inaccurate BGMs. This finding as has been demonstrated by 6 modeling studies that we reviewed. The economic costs to individuals and society of the adverse clinical outcomes associated with inaccurate BGMs are very high according to the 2 articles in the literature that we reviewed. While it might be interesting to see additional studies published in this field, the problem has now been clearly defined by many investigators, clinicians, and health economists.
A key mission of FDA is to monitor medical devices for continued safety and effectiveness after they are in use.33 In 2008 the agency launched the Sentinel Initiative, which will be a national electronic system to track reports of adverse events linked to the use of its regulated products.34 The agency released a report in 201235 and an update to that report in 2013.36 These 2 documents discussed FDA’s plans for postmarket surveillance of regulated medical devices. In 2014 under a cooperative agreement FDA assigned the Engelberg Center for Health Care Reform at the Brookings Institution to convene the National Medical Device Postmarket Surveillance Planning Board. In 2015, this board issued a report titled “Strengthening Patient Care: Building a National Postmarket Medical Device Surveillance System.” The report presented a plan for creating a surveillance program for regulating medical devices and it concluded that congressional support will be needed to create and sustain the needed infrastructure for medical device surveillance in the United States.37 In line with this plan, a post market BGM surveillance program for cleared BGM systems is currently being developed by Diabetes Technology Society (DTS).38
Blood glucose monitoring has been shown to improve outcomes in diabetes. If the technology is not delivering accurate information, however, then its benefit will be eroded. The medical literature indicates that an FDA-cleared BGM system, compared to a non-FDA-cleared BGM system, is more likely to perform according to international standards. Nevertheless, it is noteworthy that a significant proportion of cleared BGM systems do not perform at the level for which they were cleared or at a level mandated by international standards of accuracy.
Acknowledgments
The authors would like to acknowledge Annamarie Sucher for her assistance in assembling the data.
Appendix
Baumstark et al. Lot-tolot Variability of Test Strips & Accuracy Assessment of Systems for SMBG According to ISO 15197. JDST. Vol 6, Iss 5, Sept 2012.
| Year: | 2012 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 5 | |||
| % ISO 2003 | 40% | |||
| % ISO 2013 | 20% | |||
| % FDA Cleared: | 40% | |||
| % FDA Cleared Meeting ISO 2013: | 100% | |||
| % FDA Cleared Meeting ISO 2003 | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva | Roche Diagnostics GmBH | Yes | Yes | Yes |
| FreeStyle Lite | Abbott Diabetes Care Inc | Yes | Yes | No |
| GlucoCheck XL | aktivmed GmBH | No | No | No |
| Pura/mylife Pura | Bionime Corporation | No | No | No |
| OneTouch Verio Pro | Lifescan Europe | No | No | No |
Brazg et al. Performance Variability of 7 Commonly Used SMBG Systems: Clinical Considerations for Patients and Providers. JDST. Vol 7, Iss 1, Jan 2013.
| Year: | 2013 | |||
|---|---|---|---|---|
| Author’s Country: | USA | |||
| # BGMs: | 7 | |||
| % ISO 2003 | 43% | |||
| % ISO 2013 | 14% | |||
| % FDA Cleared: | 71% | |||
| % FDA Cleared Meeting ISO 2013: | 20% | |||
| % FDA Cleared Meeting ISO 2003: | 60% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva Plus | Roche Diagnostics | Yes | Yes | Yes |
| Advocate Redo-Code | TaiDoc Technology Corp | Yes | No | No |
| Element | Infopia Co | Yes | Yes | No |
| Embrace | Apex Biotechnology Corp | No | No | No |
| Prodigy Voice | Diagnostic Devices Inc | Yes | No | No |
| TRUEbalance | Nipro Diagnostics | Yes | No | No |
| WaveSense Presto | AgaMatrix | No | Yes | No |
Freckmann et al. System Accuracy Evaluation of 27 BGM Systems According to DIN EN ISO 15197. DTT. Vol 12, Number 3, 2010.
| Year: | 2010 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 27 | |||
| % ISO 2003 | 59% | |||
| % ISO 2013 | Not Evaluated | |||
| % FDA Cleared: | 30% | |||
| % FDA Cleared Meeting ISO 2013: | Not Evaluated | |||
| % FDA Cleared Meeting ISO 2003: | 88% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Active | Roche Diagnostics | Yes | Yes | ? |
| Accu-Chek Aviva | Roche Diagnostics | Yes | Yes | ? |
| Ascensia Contour | Bayer HealthCare LLC | Yes | Yes | ? |
| Bayer Contour TS | Bayer Consumer Care AG | No | No | No |
| Beurer GL 30 | Beurer GmbH & Co. | No | No | No |
| Bionime Rightest GM101 | Bionime Corp., | No | Yes | ? |
| Bionime Rightest GM300 | Bionime Corp. | No | Yes | ? |
| Clever Chek TD-4222 | Taidoc Technology Corp. | Yes | No | No |
| Finetest | Infopia Co., Ltd. | No | Yes | ? |
| Finetest Auto-coding | Infopia Co., Ltd. | No | No | No |
| FineTouch | Terumo Corp. | No | No | No |
| Fora TD-4227 | Taidoc Technology Corp. | No | No | No |
| FreeStyle Freedom | Abbott Diabetes Care Inc | Yes | Yes | ? |
| FreeStyle Lite | Abbott Diabetes Care Inc. | Yes | Yes | ? |
| GlucoCard-X-Meter | Arkray, Inc. | Yes | Yes | ? |
| Glucofix mio | Menarini Diagnostics S.r.l. | No | No | No |
| GlucoHexal | Allmedicus Co., Ltd. | No | No | No |
| Gluco-Test TD-4209 | Taidoc Technology Corp. | No | No | No |
| IME-DC BG meter | IME-DC | No | No | No |
| OneTouch Ultra 2 | LifeScan Inc | Yes | Yes | ? |
| OneTouch Ultra Easy | LifeScan Inc. | No | Yes | ? |
| Optium Xceed (E) | MediSense | No | Yes | ? |
| Optium Xceed (F) | Abbott Diabetes Care Ltd | No | Yes | ? |
| SensoCardPlus | 77 Elektronika Kft | No | Yes | ? |
| SmartLAB sprint | HMM Diagnostics GmbH | No | Yes | ? |
| Stada Glucocheck | Home Diagnostics, Inc. | No | No | No |
| Wellion Linus | AgaMatrix | No | Yes | ? |
Freckmann et al. System Accuracy Evaluation of 43 BGM Systems for Self-Monitoring of Blood Glucose according to ISO 15197 (2003). JDST. Vol 6, Issue 5, Sept 2012.
| Year: | 2012 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 43 | |||
| % ISO 2003 | 67% | |||
| % ISO 2013 | 49% | |||
| % FDA Cleared: | 35% | |||
| % FDA Cleared Meeting ISO 2013: | 73% | |||
| % FDA Cleared Meeting ISO 2003: | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Check Aviva | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Check Compact Plus | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Active | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Aviva Nano | Roche Diagnostics | No | Yes | Yes |
| Accu-Chek GO | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Mobile maltose-dependent | Roche Diagnostics | No | Yes | Yes |
| Accu-Chek Mobile maltose-independent | Roche Diagnostics | No | Yes | Yes |
| Accu-Chek Performa maltose dependent | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Performa maltose independent | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Performa Nano | Roche Diagnostics | No | Yes | Yes |
| Bayer Contour usb | Bayer Consumer Care AG | Yes | Yes | No |
| Beurer GL32 | Beurer GmBH | No | No | No |
| Beurer GL40 | Beurer GmBH | No | Yes | Yes |
| BGStar | AgaMatrix | Yes | Yes | Yes |
| Biocheck TD-4225 | TaiDoc Technology Corp | No | No | No |
| Element | Infopia Co. Ltd. | Yes | Yes | No |
| FreeStyle Freedom Lite | Abbott Diabetes Care Inc | No | Yes | Yes |
| Freestyle Lite | Abbott Diabetes Care Inc | Yes | Yes | Yes |
| Futura Monometer | TaiDoc Technology Corp | No | No | No |
| Gluco-test Plus+ TD 4230 | TaiDoc Technology Corp | No | Yes | Yes |
| GlucoCheck Classic | TaiDoc Technology Corp | No | No | No |
| GlucoCheck Comfort | aktivmed GmBH | No | Yes | Yes |
| GlucoCheck XL | aktivmed GmBH | No | Yes | No |
| GlucoHexal II | Med-WatchDoc GmBH & Co | No | No | No |
| GlucoRx (TD-4230) | TaiDoc Technology Corp | No | No | No |
| GlucoSmart Swing | MSP bodmann GmBH | No | Yes | No |
| GlucoTel | BodyTel Europe GmBH | No | No | No |
| iBGStar | AgaMatrix | Yes | Yes | No |
| iDia | IME-DC GmBH | No | No | No |
| IME-DC Fidelity | IME-DC GmBH | No | No | No |
| iXell | Genexo Sp | No | Yes | No |
| iXell OLED | Genexo Sp | No | Yes | Yes |
| microdot | Cambridge Sensors Limited | Yes | Yes | No |
| Omnitest 3 | B. Braun Meisungen AH | No | Yes | No |
| OneTouch Verio | Lifescan Inc. | Yes | Yes | Yes |
| OneTouch Verio Pro | Lifescan Europe | No | No | No |
| OneTouch VITA | Lifescan Inc. | Yes | Yes | Yes |
| Pura | Bionime Corporation | No | Yes | Yes |
| SeniorLine GM210 | Bionime Corporation | No | No | No |
| smartLAB genie | HMM Diagnostics GmBH | No | No | No |
| smartLAB global | HMM Diagnostics GmBH | No | No | No |
| WaveSense Jazz | AgaMatrix | Yes | Yes | Yes |
| Wellion CALLA light | MED TRUST Handelsges m.b.h. | No | No | No |
Freckmann et al. Evaluation of 12 BGM Systems for Self-Testing: System Accuracy & Measurement Reproducibility. DTT. Vol 16, No 2, 2014.
| Year: | 2014 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 10 | |||
| % ISO 2003 | 100% | |||
| % ISO 2013 | 80% | |||
| % FDA Cleared: | 20% | |||
| % FDA Cleared Meeting ISO 2013: | 100% | |||
| % FDA Cleared Meeting ISO 2003: | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva | Roche Diagnostics | Yes | Yes | Yes |
| BGStar | AgaMatrix Inc | Yes | Yes | Yes |
| Contour XT | Bayer Consumer Care | No | Yes | Yes |
| GE 100 | Bionime Corp | No | Yes | Yes |
| GE 200 | Bionime Corp | No | Yes | Yes |
| GL 40 | Beurer GmBH | No | Yes | No |
| myLife Pura | Bionime Corp | No | Yes | Yes |
| mylife Unio | Bionime Corp | No | Yes | Yes |
| Omnitest 3 | B. Braun Melsungen | No | Yes | No |
| OneTouch Verio Pro | LifeScan Europe | No | Yes | Yes |
Hasslacher et al. Analytical Performance of Glucose Monitoring Systems at Different BG Ranges and Analysis of Outliers in a Clinical Setting. JDST. Vol 8, Iss 3, 2014.
| Year: | 2014 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 27 | |||
| % ISO 2003 | Not Evaluated | |||
| % ISO 2013 | 41% | |||
| % FDA Cleared: | 30% | |||
| % FDA Cleared Meeting ISO 2013: | 75% | |||
| % FDA Cleared Meeting ISO 2003: | N/A | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva Nano | Roche Diagnostics | No | ? | No |
| Accu-Chek Compact | Roche Diagnostics | Yes | Yes | Yes |
| Accu-Chek Mobile | Roche | No | Yes | Yes |
| BG Star | AgaMatrix | Yes | ? | No |
| Breeze | Bayer Health Care | Yes | ? | No |
| Contour Plasma | Bayer Health Care | No | ? | No |
| Contour USB | Bayer Health Care | Yes | Yes | Yes |
| Contour USB Next | Bayer Health Care | Yes | Yes | Yes |
| Contour XT | Bayer Health Care | No | Yes | Yes |
| FineTouch | Terumo Corp | No | ? | No |
| FreeStyle Lite | Abbott Diabetes Care | Yes | Yes | Yes |
| GL 40 | Beurer Medical | No | ? | No |
| GL 50 | Beurer Medical | No | ? | No |
| (continued) | ||||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Glucomen LX | Menarini Diagnostics | No | ? | No |
| Glucomen LX plus | Menarini Diagnostics | No | ? | No |
| GlucoSmart Swing | MSP Bodmann | No | ? | No |
| iBG Star | AgaMatrix | Yes | Yes | Yes |
| my glucohealth | Entra Health Systems | No | ? | No |
| mylife Pura | Ypsomed AG | No | Yes | Yes |
| Omnitest 3 | B. Braun | No | ? | No |
| One Touch Ultra Easy | LifeScan Inc | No | Yes | Yes |
| One Touch Verio IQ | LifeScan Inc | No | Yes | Yes |
| One Touch Verio Pro | LifeScan Inc | No | ? | No |
| One Touch Vita | LifeScan Inc | Yes | Yes | Yes |
| Smart Lab Mini | HMM Diagnostics | No | ? | No |
| Smart Lab Sprint | HMM Diagnostics | No | ? | No |
| Wellion Calla | MedTrust | No | ? | No |
Huang et al. Evaluation of accuracy of FAD-GDH- and mutatant Q-GDH-based blood glucose monitors in multi-patient populations. Clinica Chimica Acta. Vol 433, 2014.
| Year: | 2014 | |||
|---|---|---|---|---|
| Author’s Country: | Taiwan | |||
| # BGMs: | 2 | |||
| % ISO 2003 | 100% | |||
| % ISO 2013 | 100% | |||
| % FDA Cleared: | 100% | |||
| % FDA Cleared Meeting ISO 2013: | 100% | |||
| % FDA Cleared Meeting ISO 2003: | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| GM700 | Bionime | Yes | Yes | Yes |
| Accu-Chek Performa | Roche Diagnostics | Yes | Yes | Yes |
Link et al. Accuracy Evaluation of 3 Sysems for SMBG with 3 Different Test Strip Lots Following ISO 15197. JDST. Vol 8, Iss 2, 2014.
| Year: | 2014 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 3 | |||
| % ISO 2003 | 100% | |||
| % ISO 2013 | 100% | |||
| % FDA Cleared: | 100% | |||
| % FDA Cleared Meeting ISO 2013: | 100% | |||
| % FDA Cleared Meeting ISO 2003: | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| CareSens N | i-SENS inc | Yes | Yes | Yes |
| CareSens N POP | i-SENS inc | Yes | Yes | Yes |
| alphacheck professional | i-SENS inc | Yes | Yes | Yes |
Pfutzner et al. Performance of Blood Glucose Meters in Compliance with Current and Future Clinical ISO15197 Accuracy Criteria. Curr Med Res Opin. 30(2) 2014.
| Year: | 2014 | |||
|---|---|---|---|---|
| Author’s Country: | Germany | |||
| # BGMs: | 6 | |||
| % ISO 2003 | 100% | |||
| % ISO 2013 | 83% | |||
| % FDA Cleared: | 83% | |||
| % FDA Cleared Meeting ISO 2013: | 80% | |||
| % FDA Cleared Meeting ISO 2003: | 100% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva | Roche Diagnostics | Yes | Yes | Yes |
| BG*Star | AgaMatrix | Yes | Yes | Yes |
| iBG*Star | AgaMatrix | Yes | Yes | Yes |
| Contour | Bayer Consumer Care | Yes | Yes | No |
| Freestyle Freedom Lite | Abbott Diagnostics | No | Yes | Yes |
| OneTouch Ultra 2 | LifeScan Inc. | Yes | Yes | Yes |
Sonmez, et al. The Accuracy of Home Glucose Meters in Hypoglycemia. DTT. Volume 12, Number 8, 2010.
| Year: | 2010 | |||
|---|---|---|---|---|
| Author’s Country: | Turkey | |||
| # BGMs: | 5 | |||
| % ISO 2003 | 0% | |||
| % ISO 2013 | 0% | |||
| % FDA Cleared: | 60% | |||
| % FDA Cleared Meeting ISO 2013: | 0% | |||
| % FDA Cleared Meeting ISO 2003: | 0% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Optium Xceed | Abbott Diabetes Care | No | No | No |
| Contour TS | Bayer Diabetes Care | No | No | No |
| Accu-Check Go | Roche Ltd. | Yes | No | No |
| OneTouch Select | Lifescan | Yes | No | No |
| EZ Smart | Tyson Bioresearch | Yes | No | No |
Tack et al. Accuracy Evaluation of 5 BGM Systems Obtained from the Pharmacy: A European Multicenter Study with 453 Subjects. DTT. Vol 14, No 4, 2012.
| Year: | 2012 | |||
|---|---|---|---|---|
| Author’s Country: | Netherlands | |||
| # BGMs: | 5 | |||
| % ISO 2003 | 60% | |||
| % ISO 2013 | 40% | |||
| % FDA Cleared: | 60% | |||
| % FDA Cleared Meeting ISO 2013: | 33% | |||
| % FDA Cleared Meeting ISO 2003: | 67% | |||
| Meter Name | Company | Cleared? | ISO 2003 | ISO 2013 |
| Accu-Chek Aviva | Roche Diagnostics GmBH | Yes | Yes | No |
| Contour | Bayer Consumer Care AG | Yes | No | No |
| FreeStyle Freedom Lite | Abbott Diabetes Care Inc | No | No | No |
| FreeStyle Lite | Abbott Diabetes Care Inc | Yes | Yes | Yes |
| OneTouch UltraEasy | Lifescan Inc | No | No | No |
Footnotes
Abbreviations: BGM, blood glucose monitor; CGM, continuous glucose monitor; CVGA, control variability grid analysis; DTS, Diabetes Technology Society; SMBG, self-monitoring of blood glucose.
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: DCK is a consultant for Google, Insuline, Lifecare, Novartis, Roche, Sanofi, Tempramed, and Voluntis. He is also a stockholder in Tempramed. PP has no disclosures.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
- 1. Endocrine Today. Poor performance of blood glucose monitors prompts call to action. October 1, 2013. http://www.healio.com/endocrinology/diabetes/news/print/endocrine-today/%7B9119eae5-4781-4c64-a2ac-7476ca45aa98%7D/poor-performance-of-blood-glucose-monitors-prompts-call-to-action. Accessed April 8, 2015.
- 2. PR Newswire. Diabetes Hands Foundation commends the Diabetes Technology Society (DTS), Food and Drug Administration (FDA) for first steps to protect diabetes patients. http://www.prnewswire.com/news-releases/diabetes-hands-foundation-commends-the-diabetes-technology-society-dts-food-and-drug-administration-fda-for-first-steps-to-protect-diabetes-patients-223155911.html. Accessed April 8, 2015.
- 3. Schnell O, Hinzmann R, Kulzer B, et al. Assessing the analytical performance of systems for self-monitoring of blood glucose: concepts of performance evaluation and definition of metrological key terms. J Diabetes Sci Technol. 2013;7(6):1585-1594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Klonoff DC. The Food and Drug Administration is now preparing to establish tighter performance requirements for blood glucose monitors. J Diabetes Sci Technol. 2010;4(3):499-504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. US Food and Drug Administration. Self-monitoring blood glucose test systems for over-the-counter use: draft guidance for industry and Food and Drug Administration staff. http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM380327.pdf. Accessed April 8, 2014.
- 6. International Organization for Standardization. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. EN ISO 15197:2003 (E). [Google Scholar]
- 7. International Organization for Standardization. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. ISO 15197:2013 (E). [Google Scholar]
- 8. Lias C. Setting the bar for blood glucose meter performance. FDA Voice; January 9, 2014. http://blogs.fda.gov/fdavoice/?s=Setting+the+Bar+for+Blood+Glucose+Meter+Performance. Accessed April 8, 2015
- 9. Freckmann G, Baumstark A, Jendrike N, et al. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther. 2010;12(3):221-231. [DOI] [PubMed] [Google Scholar]
- 10. Sonmez A, Yilmaz Z, Uckaya G, et al. The accuracy of home glucose meters in hypoglycemia. Diabetes Technol Ther. 2010;12(8):619-626. [DOI] [PubMed] [Google Scholar]
- 11. Tack C, Pohlmeier H, Behnke T, et al. Accuracy evaluation of five blood glucose monitoring systems obtained from the pharmacy: a European multicenter study with 453 subjects. Diabetes Technol Ther. 2012;14(4):330-337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Freckmann G, Schmid C, Baumstark A, Pleus S, Link M, Haug C. System accuracy evaluation of 43 blood glucose monitoring systems for self-monitoring of blood glucose according to DIN EN ISO 15197. J Diabetes Sci Technol. 2012;6(5):1060-1075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Baumstark A, Pleus S, Schmid C, Link M, Haug C, Freckmann G. Lot-to-lot variability of test strips and accuracy assessment of systems for self-monitoring of blood glucose according to ISO 15197. J Diabetes Sci Technol. 2012;6(5):1076-1086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Brazg RL, Klaff LJ, Parkin CG. Performance variability of seven commonly used self-monitoring of blood glucose systems: clinical considerations for patients and providers. J Diabetes Sci Technol. 2013;7(1):144-152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Freckmann G, Baumstark A, Schmid C, Pleus S, Link M, Haug C. Evaluation of 12 blood glucose monitoring systems for self-testing: system accuracy and measurement reproducibility. Diabetes Technol Ther. 2014;16(2):113-122. [DOI] [PubMed] [Google Scholar]
- 16. Pfützner A, Hengesbach C, Demircik F, Schipper C, Forst T, Musholt PB. Performance of blood glucose meters in compliance with current and future clinical ISO15197 accuracy criteria. Curr Med Res Opin. 2014;30(2):185-190. [DOI] [PubMed] [Google Scholar]
- 17. Link M, Pleus S, Schmid C, et al. Accuracy evaluation of three systems for self-monitoring of blood glucose with three different test strip lots following ISO 15197. J Diabetes Sci Technol. 2014;8(2):422-424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Hasslacher C, Kulozik F, Platten I. Analytical performance of glucose monitoring systems at different blood glucose ranges and analysis of outliers in a clinical setting. J Diabetes Sci Technol. 2014;8(3):466-472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Huang TY, Chang HW, Tsao MF, et al. Evaluation of accuracy of FAD-GDH- and mutant Q-GDH-based blood glucose monitors in multi-patient populations. Clin Chim Acta. 2014;433:28-33. [DOI] [PubMed] [Google Scholar]
- 20. US Food and Drug Administration. 510(k) premarket notification. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm. Accessed February 15, 2015.
- 21. U.S. Food and Drug Administration. “CLIA - Clinical Laboratory Improvement Amendments - Currently Waived Analytes.” Accessed February 15, 2015 http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfClia/analyteswaived.cfm?start_search=G.
- 22. Boyd JC, Bruns DE. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem. 2001;47(2):209-214. [PubMed] [Google Scholar]
- 23. Boyd JC, Bruns DE. Monte Carlo simulation in establishing analytical quality requirements for clinical laboratory tests meeting clinical needs. Methods Enzymol. 2009;467:411-433. [DOI] [PubMed] [Google Scholar]
- 24. Breton MD, Kovatchev BP. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol. 2010;4(3):562-570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Magni L, Raimondo DM, Man CD, et al. Evaluating the efficacy of closed-loop glucose regulation via control-variability grid analysis. J Diabetes Sci Technol. 2008;2(4):630-635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Karon BS, Boyd JC, Klee GG. Glucose meter performance criteria for tight glycemic control estimated by simulation modeling. Clin Chem. 2010;56(7):1091-1097. [DOI] [PubMed] [Google Scholar]
- 27. Virdi NS, Mahoney JJ. Importance of blood glucose meter and carbohydrate estimation accuracy. J Diabetes Sci Technol. 2012;6(4):921-926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Karon BS, Boyd JC, Klee GG. Empiric validation of simulation models for estimating glucose meter performance criteria for moderate levels of glycemic control. Diabetes Technol Ther. 2013;15(12):996-1003. [DOI] [PubMed] [Google Scholar]
- 29. Thomas F, Signal M, Harris DL, et al. Continuous glucose monitoring in newborn infants: how do errors in calibration measurements affect detected hypoglycemia? J Diabetes Sci Technol. 2014;8(3):543-550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Budiman ES, Samant N, Resch A. Clinical implications and economic impact of accuracy differences among commercially available blood glucose monitoring systems. J Diabetes Sci Technol. 2013;7(2):365-380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Schnell O, Hinzmann R, Kulzer B, et al. Assessing the analytical performance of systems for self-monitoring of blood glucose: concepts of performance evaluation and definition of metrological key terms. J Diabetes Sci Technol. 2013;7(6):1585-1594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Schnell O, Erbach M. Impact of a reduced error range of SMBG in insulin-treated patients in Germany. J Diabetes Sci Technol. 2014;8(3):479-482. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. US Food and Drug Administration. National medical device postmarket surveillance plan. http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/ucm301912.htm. Accessed April 8, 2015.
- 34. US Food and Drug Administration. FDA’s sentinel initiative. http://www.fda.gov/Safety/FDAsSentinelInitiative/default.htm. Accessed April 8, 2015.
- 35. US Food and Drug Administration. Strengthening our national system for medical device postmarket surveillance. September 2012. http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/UCM301924.pdf. Accessed April 8, 2015.
- 36. US Food and Drug Administration. Strengthening our national system for medical device postmarket surveillance: update and next steps. April 2013. http://www.fda.gov/downloads/MedicalDevices/Safety/CDRHPostmarketSurveillance/UCM348845.pdf. Accessed April 8, 2015.
- 37. Brookings Institution. Strengthening patient care: building a national postmarket medical device surveillance system. February 2015. http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/UCM435112.pdf. Accessed April 8, 2015.
- 38. Klonoff DC, Reyes JS. Do currently available blood glucose monitors meet regulatory standards? 1-day public meeting in Arlington, Virginia. J Diabetes Sci Technol. 2013;7(4):1071-1083. [DOI] [PMC free article] [PubMed] [Google Scholar]