Abstract
Regulatory interest has focused on the accuracy of blood glucose monitoring systems. Currently, almost all systems meet the International Organization for Standardization (ISO) 15197 clinical standard (≥95% of the values within 20% of the reference for values above 75 mg/dl and within 15 mg/dl below that level). Should the systems have to meet one of the extended ISO standards of 15%, 10%, or even 5%? There is a wide variety of people with diabetes doing glucose monitoring, and the majority do not need better accuracy. Indeed, when selecting an insulin dose, the inaccuracy of the glucose reading has little effect compared with the inaccuracy in counting carbohydrates and the variability in insulin absorption. It might be far better to evaluate the accuracy in a standard method and provide the accuracy values on a standard label. Patients and health care providers could then select the monitoring system that best meets their needs.
Keywords: accuracy, blood glucose monitoring, consumer product labeling, regulation
Introduction
Blood glucose monitoring (BGM) with home monitoring systems is widely used for self-monitoring of blood glucose (SMBG) and for clinical settings such as doctors' offices, hospital wards, and intensive care settings, but there has been some controversy about the accuracy of the meters and the role of U.S. Food and Drug Administration (FDA) regulation.1–3 I plan only to discuss SMBG since the intended use of most monitors is for this purpose. Self-monitoring of blood glucose is used by three major distinct groups of people with diabetes: (A) patients with type 2 diabetes on diet and/or medications that do not cause hypoglycemia, (B) patients with type 2 diabetes using insulin or medications that can cause hypoglycemia, and (C) patients with type 1 diabetes. The accuracy needs of each of these groups is different.
We measure accuracy by a number of methods, but two of the most useful are mean absolute error (MAE)4 and the extended International Organization for Standardization (ISO) 15197 standard.5 The MAE is a single number that reflects both accuracy and precision, and ISO 15197 is a worldwide standard for glucose monitors, including laboratory and clinical testing that evaluates accuracy at a series of levels. The least demanding of the ISO standards is that 95% of values above 75 mg/dl be within a 20% deviation from the reference and values below 75 mg/dl within a 15 mg/dl deviation. This is used for approval of BGM systems in Europe and with minor variation in the United States. The extended ISO standard also allows determination of the percentage of values within smaller deviations. Shown in Figure 1, they are 15%, 10%, and 5% deviation from the reference for values above 75 mg/dl with corresponding limits below 75 mg/dl. Should we require manufacturers of BGM systems to meet one of these stricter limits. Is it necessary, and can they do it?
How Much Accuracy Do We Need?
Glucose monitoring is used differently for each of the clinical groups. Patients with type 2 diabetes not subject to hypoglycemia test occasionally to see how they are doing, to determine the effect of foods on their diabetes, and sometimes to determine if they need to increase their medication. For this purpose, meters meeting the current standard of 95% of values falling within a deviation of 20% (approximately an 8% MAE) is appropriate. If the true glucose is 150, roughly 2/3 of the values will fall from 135 to 165, and this range will have little clinical effect.
Patients with type 2 diabetes on insulin or hypoglycemic agents need more accuracy. They sometimes use the glucose value to determine insulin dose (see later comments on total accuracy), to determine if they are hypoglycemic or heading towars hypoglycemia, and to make therapeutic adjustments. For these patients, 95% of the values should fall within 15% of the reference for values above 75 mg/dl and within 12 mg/dl below that.
Finally, patients with type 1 diabetes need the greatest accuracy.6,7 They use glucose monitoring routinely to make therapeutic decisions and to inform themselves about hypoglycemia. In addition, these devices are now used to calibrate continuous glucose monitors and need to be very accurate. Although routinely used to make insulin decisions at meals, glucose monitors account for only a small portion of the total error in the absorbed insulin dose. Errors in carbohydrate counting are routinely 15–25%, errors in the constants used (carbohydrate-to-insulin ratio and insulin sensitivity factor) are routinely 10–25%, and insulin absorption from the injection site varies by 20–30%. Assuming these errors are independent, the sum of these errors is 27–46%. An error in glucose monitoring of 6% adds only about 0.5%, and doubling the error to 12% adds only 1.5–2.5% to that range.
In contrast, the ability of a monitoring system to detect hypoglycemia is highly dependent on the accuracy of the system. Breton,8 using an in silico simulation, determined that systems with an ISO inaccuracy (95% limit) of 5% found all of 100 hypoglycemia events correctly, that those with a limit of 10% missed 1%, but that systems with a limit of 15% missed 5% and current systems with a limit of 20% missed 10%. Thus, for patients with type 1 diabetes, monitoring systems should have an extended ISO limit of 10%8
One area of accuracy that is overlooked is that of outliers. The ISO standards describe 95% of values that make up the standard, but ignore the other 5%. In practice, most of the values in this 5% are very close to the required accuracy standard, but some have large deviations from the reference.. As seen in Figure 2, representative of actual clinical data of 5000 samples, approximately 1 in 1000 values have a serious error that could potentially harm a patient who used that data point. Most manufacturers have this data, at least for laboratory data, but it has not yet been a point of discussion. I think that needs to change.
How Accurate Are Current Systems?
In clinical practice, monitoring systems have errors from multiple sources, which have been described in a previous article.4. Errors arise from: (1) manufacturing variation (well size, enzyme coverage, mediator oxidation, and age of strip), (2) physical location (temperature and altitude) and interfering substances (endogenous and exogenous), and (3) patient-related errors (hematocrit, lack of proper coding, and hand-washing). Laboratory testing and most clinical trials eliminate all but the manufacturing variation from their testing. In one study, Freckmann and coworkers9 tested 27 monitoring systems in the laboratory with 200 values each. Of the 10 most commonly used in the United States, all passed the ISO 20% standard, 6 passed the 15% standard, and 1 passed the 10% standard. Data from the package inserts of 9 common systems in the United States is shown in Table 1. Six meet the 15% criteria, but none meet the 10% criteria.
Table 1.
System | Percentage meeting standard (above 75 mg/dl; absolute value in parenthesis below 75 mg/dl) | |||
---|---|---|---|---|
20% (15 mg/dl) | 15% (12 mg/dl) | 10% (8 mg/dl) | 5% (4 mg/dl) | |
A | 99 | 91 | 68 | 35 |
B | 96 | 91 | 82 | 52 |
C | 99 | 95 | 84 | 53 |
D | 100 | 98 | 88 | 52 |
E | 99 | 98 | 92 | 63 |
F | 98 | 97 | 89 | 61 |
G | 98 | 95 | 79 | 41 |
H | 99 | 98 | 92 | 65 |
I | 99 | 98 | 93 | 71 |
Currently, each manufacturer does their own testing, so the protocols are not identical, the methods vary, and the results may not be comparable from one system to another. In Europe, all testing is done by companies that are certified by the European Union, called Notified Bodies.10 All but one use the same protocol and same analysis, so the testing should be comparable. We should adopt a similar system in the United States.
How Do We Get Better Accuracy?
Over the next few years, accuracy is likely to improve. Many meters no longer require coding, and dynamic electrochemistry11 minimizes the effect of manufacturing variation, hematocrit, altitude, and interferants, as do alternating current impedance, thermistors, multiple sensors, and data on the cartridge.
The easiest way to ensure better accuracy would be to tighten the accuracy standards for approval by the FDA, but I do not think this is the wisest choice. A substantial portion of the current users do not need greater accuracy and should not have to pay extra for an unwanted feature. Tightening the standard to 15% would still not inform the patient with type 1 diabetes, who needs the greatest accuracy, which is the best system for them.
Glucose monitors are consumer products. Manufacturers now compete on size, color, display, time, and even accuracy (without showing data). They should be able to compete on accuracy using data that are concise, understandable, and comparable. Data about the MAE and the extended ISO values should be clearly displayed on the strip container and the outside of the box. Similar to a nutrition label, it should be easy to understand. An example is shown in Table 2. For such a label to be meaningful, testing of the monitoring systems would have to be done by independent laboratories, using random lot, with a common protocol and common analytical methods and statistical analysis.
Table 2.
Errors in measurement | |
---|---|
Average error (smaller is better) | 6.1% |
Tests with <20% errora (larger is better) | 99% |
Tests with <15% errora (larger is better) | 91% |
Tests with <10% errora (larger is better) | 68% |
Tests with <5% errora (larger is better) | 35% |
For values below 75 mg/dl, error allowed is the same as the error in mg/dl allowed at 75 mg/dl.
Recommendation
I would recommend the following:
Keep the current standard of ISO 20% but have the clinical testing done independently and with a sufficient number of samples and a variety of patients to ensure the accuracy as determined by both the ISO 20% standard and the extended standard.
Label systems with accuracy data using a standard label such as shown in Table 2. Additional education of patients will be needed, and it is likely that the diabetes professional and patient advocacy associations will suggest accuracy standards for specific groups of patients. Manufacturers may use this in advertising.
Perform clinical and laboratory testing using random lots by certified organizations using common protocols, similar patient groups, common analytic methods, and statistical analysis. This testing should be done for initial approval of the device and periodically (every 6–12 months) to ensure continued accuracy. Failure to meet the same accuracy on a periodic test would require correction of the deficit or, if not possible, changing the label.
Develop a system for better analysis of outliers and reporting and tracking them.
I believe that better education and market forces will be better at producing monitoring systems appropriate for individual patients than harsh regulatory steps.
Abbreviations
- BGM
blood glucose monitoring
- FDA
U.S. Food and Drug Administration
- ISO
International Organization for Standardization
- MAE
mean absolute error
References
- 1.Hirsch IB, Bode BW, Childs BP, Close KL, Fisher WA, Gavin JR, Ginsberg BH, Raine CH, Verderese CA. Self-monitoring of blood glucose (SMBG) in insulin- and non-insulin-using adults with diabetes: consensus recommendations for improving SMBG accuracy, utilization, and research. Diabetes Technol Ther. 2008;10(6):419–439. doi: 10.1089/dia.2008.0104. [DOI] [PubMed] [Google Scholar]
- 2.Klonoff DC. Regulatory controversies surround blood glucose monitoring devices. J Diabetes Sci Technol. 2010;4(2):231–235. doi: 10.1177/193229681000400201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Klonoff DC, Bergenstal R, Blonde L, Boren SA, Church TS, Gaffaney J, Jovanovic L, Kendall DM, Kollman C, Kovatchev BP, Leippert C, Owens DR, Polonsky WH, Reach G, Renard E, Riddell MC, Rubin RR, Schnell O, Siminiero LM, Vigersky RA, Wilson DM, Wollitzer AO. Consensus report of the coalition for clinical research—self-monitoring of blood glucose. J Diabetes Sci Technol. 2008;2(6):1030–1053. doi: 10.1177/193229680800200612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ginsberg BH. Factors affecting blood glucose monitoring: sources of errors in measurement. J Diabetes Sci Technol. 2009;3(4):903–913. doi: 10.1177/193229680900300438. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.International Organization for Standardization In vitro diagnostic test systems. Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. ISO 15197:2003. http://www.iso.org/iso/catalogue_detail.htm?csnumber=26309
- 6.Boyd JC, Bruns DE. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem. 2001;47(2):209–214. [PubMed] [Google Scholar]
- 7.Krouwer JS, Cembrowski GS. A review of standards and statistics used to describe blood glucose monitor performance. J Diabetes Sci Technol. 2010;4(1):75–83. doi: 10.1177/193229681000400110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Breton MD, Kovatchev BP. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol. 2010;4(3):562–570. doi: 10.1177/193229681000400309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Freckmann G, Baumstark A, Jendrike N, Zschornack E, Kocher S, Tshiananga J, Heister F, Haug C. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther. 2010;12(3):221–231. doi: 10.1089/dia.2009.0128. [DOI] [PubMed] [Google Scholar]
- 10.The European Parliament and the Council of the European Union. Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on in vitro diagnostic medical devices. Official Journal of the European Communities. http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:1998:331:0001:0037:en:PDF
- 11.Anderson JL, Coury LA, Jr, Leddy J. Dynamic electrochemistry: methodology and application. Anal Chem. 2000;72(18):4497–4520. doi: 10.1021/ac0007837. [DOI] [PubMed] [Google Scholar]