The Food and Drug Administration Is Now Preparing to Establish Tighter Performance Requirements for Blood Glucose Monitors

Abstract

On March 16 and 17, 2010, the Food and Drug Administration (FDA) presented a public meeting about blood glucose monitoring at the Gaithersberg Hilton Hotel. The meeting was intended to present expert opinions and solicit input from the public about whether to develop new regulatory policies for blood glucose monitors. The meeting was divided into three sections: (1) Clinical Accuracy Requirements for Blood Glucose Monitors, (2) Interferences and Limitations of Blood Glucose Monitors, and (3) Tight Glycemic Control. Many officials from the Center for Devices and Radiologic Health and the Office of In Vitro Diagnostic Devices, which are the parts of FDA that regulate approval of blood glucose monitors, either spoke on the agenda or attended in the audience. Approximately 300 people attended; they were mostly clinicians (such as adult endocrinologists, pediatric endocrinologists, internists, clinical chemists, intensivists, surgeons, nurses, and diabetes educators) or industry officials from companies involved in glucose monitoring, pharmaceutical products, data analysis, or regulatory consulting.

Introduction

The introductory remarks at the Food and Drug Administration (FDA) Blood Glucose Monitor Meeting were made by the highest-ranking FDA official at the meeting, Jeffrey Shuren, M.D., J.D., the director of the Center for Device and Radiologic Health. Dr. Shuren stated that the importance of self-monitoring of blood glucose in diabetes is unquestioned. He questioned whether there was evidence of a current need for higher standards for blood glucose monitors. He presented two questions that are critical to the FDA's role in regulating blood glucose monitors. First, how should the FDA balance the needs of patients and developers of technology? Second, what are the FDA's responsibilities to ensure that blood glucose monitors are safe? He concluded that the FDA is currently weighing new industry guidelines for blood glucose monitors.

Over the course of two days, the meeting featured presentations by FDA officials from the Office of In Vitro Diagnostic Devices, adult and pediatric endocrinologists, clinical chemists, an intensivist, a systems engineer, a representative from the Centers for Medicare and Medicaid Services, industry regulatory and medical experts, two attorneys, and a diabetes patient advocate. Both formal presentations and panel discussions raised numerous topics related to recommendations for both more restrictive regulatory standards (which are defined as achievable goals given today's amount of technology and financial resources) and clinical standards (which are defined as ideal goals irrespective as to whether they are currently achievable).

Analytical and Clinical Performance Accuracy

The meeting attendees heard presentations about the current levels and possible future levels of analytical and clinical performance for blood glucose monitors. Currently, for all blood glucose monitors, the FDA has adopted the International Organization for Standardization (ISO) 15197 standard that specifies that, for a blood glucose level of below 75 mg/dl, the monitor reading must be within 15 mg/dl of the reference reading and, for readings of 75 mg/dl or higher, the monitor reading must be within 20% of the reference reading in 95% of tests.¹ Over time, according to Courtney Harper, Ph.D., the director of the Division of Clinical Chemistry Devices at the Office of In Vitro Diagnostic Devices, outpatient blood glucose monitors have migrated to wider use in hospital intensive care units, even though they are not approved for critical care uses.

The FDA currently uses the same minimum accuracy criteria from ISO 15197 for both lay use and health care professional use. The manufacturers of some over-the-counter blood glucose monitors for home use have neither applied for nor received FDA approval for use in hospitals or long-term facilities, but these devices are brought into hospitals or long-term facilities and used anyway. It is possible that, in the future, the FDA will develop a different standard for meters used in hospitals and long-term facilities compared to those intended for use at home.

Blood glucose monitors intended for hospitals and long-term facilities currently tend to be large and have a capability to transmit data. They also tend to contain docking stations, barcode-reading capabilities, quality-control lockouts, and large memory capacity for storing data. Over-the-counter monitors intended for use at home generally do not share these features.

There are three principal reasons why hospitals use over-the-counter blood glucose monitors off label, even though the accuracy of such devices is less than that for other methods that are approved for use in a hospital setting. First, an over-the-counter monitor approved for use at home can provide a capillary blood glucose result in seconds, but many approved reference methods require arterial or venous blood samples and can require an hour to test and report. There are some point-of-care instruments that provide a result in only minutes, but they are a bit more expensive and less convenient to use. Second, the amount of costly labor and supplies required for a point-of-care monitor is often far less than what is needed for an approved reference method test of blood glucose. Third, many intensive care unit patients are anemic, and a blood glucose monitor will almost always require less blood for a measurement than an approved reference method test of blood glucose, which will minimize further blood loss. Thus a blood glucose monitor that is approved for use at home will generally save on time, money, and blood compared to a monitor that is approved for use in a hospital or long-term facility.

The intention of all the participants, both those of the speakers' panel and those in the audience who provided comments from the public, was to see high-quality blood glucose monitors on the market. Thus everyone's goal was in alignment, but there were many ideas presented as to what would be the best path to achieving that goal. Specifically, the meeting participants and attendees presented a wide range of opinions as to what the FDA should consider to be the best metrics to measure blood glucose monitor performance, the optimal levels of blood glucose monitor analytical and clinical performance, and the appropriate type of product labeling to report the metrics of performance. A call to mandate greater analytical accuracy of all glucose monitors was raised repeatedly by experts on the agenda and by audience members. An alternate point of view was expressed that more accurate results can be more readily achieved by encouraging or mandating better adherence to human factors practices. This point of view specified that patients and health care professionals need to use monitoring equipment more in line with manufacturers' recommendations. In the end, it was left to the FDA regulators to sort out the best goals and the best paths to achieve those goals.

Themes at the Meeting

At the meeting, ten themes emerged from comments made by many participants about how to modify current regulatory standards for blood glucose monitors. These themes are summarized in Table 1 and discussed in the following ten paragraphs:

Table 1.

Ten Factors Needed to Improve the Performance of Blood Glucose Monitors

1	Separate analytical performance standards for different populations
2	Separate clinical performance standards for different populations
3	Specific standards for outlier and no-reading data points
4	New labeling standards for analytical and clinical performance
5	New protocols for testing interfering substances
6	New protocols for reporting interfering substances
7	Tradeoffs of current features in exchange for improved accuracy
8	Data to quantitatively link improved accuracy with improvedoutcomes
9	Improved performance of human factors which also affect accuracy
10	Consensus targets for glycemic control in acutely ill hospital patients

1. Separate analytical performance standards are needed for different populations of patients with diabetes. While there was widespread agreement that noninsulin-treated outpatients with type 2 diabetes as well as most insulin-treated outpatients with type 2 diabetes need greater accuracy for their monitors than is currently mandated, there are a few special populations, such as hospitalized patients or women with gestational diabetes who might require even greater accuracy of their monitors to achieve tight glycemic control within a narrow range and who are at high risk of hypoglycemia in case of accidental insulin overdosing. This approach was compared with driving laws that establish a speed limit for residential neighborhoods and an even more restrictive speed limit for streets adjacent to a school. Another analogy was laws in California that require residential and office buildings to withstand a moderate earthquake, but require hospitals to withstand a severe earthquake.² In these two analogous situations to regulation of blood glucose monitors, safety is mandated for all situations, but only settings where much greater safety is required are mandated to expend the additional burdensome resources to achieve an even higher level of safety. The establishment of the more restrictive of the two sets of standards would be excessively burdensome to society, and so two levels of safety are mandated. Regarding the analogy between blood glucose monitor performance standards and seismic construction standards, over time, as better technology for seismically safe building construction has evolved and the costs of safer construction technology have decreased, the mandated standards for building construction seismic safety have become progressively stricter.

2. Separate clinical performance standards are needed for different populations of patients with diabetes. The reasoning is the same as for the first theme. In the case of clinical performance, accuracy means the extent to which any blood glucose reading will lead to an appropriate decision. There was support for using an error grid to qualitatively identify the performance of each data point in a clinical trial of a blood glucose monitor and for the FDA to then establish a quantitative scoring system that would mandate what percentage of data points must or must not fall into zones A, B, C, D, or E. A specific error grid could be established for each type of setting where unique performance standards are to be mandated; for example, one grid could be developed for standard populations and one could be developed for special populations who require tighter control. The advantage of an error grid over specifying the percentage of data points that fall within a specified percentage of the reference method is that, for high blood glucose levels, there is greater tolerance for percentage error and, for low blood glucose levels, there is less tolerance for percentage error.³ The current system for describing how close all data points come to being within a specified percentage range, however, treats high and low glucose readings alike.

3. Analytical performance standards should focus more closely on the data points that are currently permitted to fall outside the currently mandated limits of accuracy. According to ISO 15197, 95% of the data points must be within range and the other 5% need not be within the specified range. The problem was felt to be that these 5% of data points can be extremely far from accurate. There is no limit for their degree of inaccuracy. There was sentiment to either require 100% of data points to fall within specified ranges or else to set up some type of mandated distribution range for these other 5% of data points. Likewise, current standards do not, but could, count nonreading data points that could be counted as part of the denominator in quantifying performance.⁴

4. New labeling specifications for analytical performance and clinical performance are needed. These could incorporate analytical measures of mean average relative error or clinical measures of error grid performance or even the percentage of data points that fall within a specified percentage of the reference level.

5. New protocols for measuring interfering substances are needed. These could be developed by a Clinical and Laboratory Standards Institute (CLSI) committee on interfering substances. The most recent CLSI guideline on interference testing in clinical (document EP7-A2) was approved in 2005.⁵ The testing of both interfering substances as well as analytical accuracy might be best performed by a notified body instead of the monitor's manufacturer.

6. New protocols for reporting results of testing for interfering substances are needed. These could be developed by a new CLSI committee. Perhaps monitors that do not meet updated performance standards will be phased out from the marketplace under well-defined conditions.

7. The diabetes community must begin thinking about the tradeoffs that will probably be needed to achieve better performance of blood glucose monitors. Everything has a cost, including analytical accuracy. The cost could be money that will be needed to pay for development and production of more accurate monitors. The cost might, however, turn out to be scaling back on ease-of-use features that could result in greater time for making a measurement, a greater size for the drop of blood, a greater number of steps to perform a reading, greater size of the monitor, a greater amount of reagent instability, or a greater amount of environmental restrictions on strips. Many companies have told me that consumers will not want to switch to monitors with more of any of these types of costs. Nevertheless, if greater performance is mandated for hospitals and long-term facilities for these settings, the products will not typically be selected or operated by consumers. Therefore, any increased monetary or nonmonetary costs necessary to attain greater accuracy in hospitals or long-term facilities will probably be tolerated by the consumers.

8. Additional data are needed to address the extent to which improved analytical or clinical accuracy leads to improved outcomes. The basis of many recent calls for better performance by blood glucose monitors is an assumption that the resources that will need to be expended on developing monitors with greater accuracy will be a worthwhile allocation. Unfortunately, no empiric, randomized data exist that link improved accuracy with improved outcomes. Improved human factor performance associated with eliminating coding errors in blood glucose monitors has been reported to be associated with improved clinical outcomes.⁶ The first modeling study ever to describe a link between improved analytical accuracy and improved glycemic outcomes was performed by Marc Breton and Boris Kovatchev⁷ and is reported in this issue of Journal of Diabetes Science and Technology. These investigators are to be commended for writing a landmark article on this topic, which reports the extent to which improved analytical accuracy can impact clinical outcomes. As in any modeling study, they made assumptions. There is a need for additional modeling studies on the effect of improved performance on glycemia using different assumptions to even further establish the relationship between better accuracy and better outcomes. It may turn out that, in different patient populations and with different assumptions, the use of more accurate blood glucose monitors will lead to different predicted levels of improved outcomes.

9. A process is needed to improve the performance of other aspects of blood glucose monitoring that can affect results, besides monitor accuracy. This is because these factors can also impact the accuracy of measurements. Whereas inherent monitor accuracy contributes to inaccurate readings and therefore to inappropriate dosing of medications, food, or exercise, monitor accuracy is not the only factor that can degrade accuracy if not properly controlled. So-called human factor errors can also affect the accuracy of blood glucose readings no matter how accurate a monitor might be.⁸ Improper techniques can lead to monitor error of a magnitude comparable to that of current monitor error alone in the absence of human factor errors. Proper technique is needed for such user-controlled steps as hand washing, filling of strips with blood, selection of a blood specimen source (arterial, capillary, or venous) for testing, and meter coding (where applicable).⁹ Improper storage of strips is also a significant and commonly occurring source of error with blood glucose meters.¹⁰ There is a wide range of potential adherence to proper techniques by subjects or technicians testing blood glucose monitors in clinical trials. Advances in technology, such as the development of blood glucose testing meters that can withstand frequent disinfection and noninvasive glucose monitoring methods, will likely prove useful in improving patient safety.¹¹ Standardization of testing by a notified body designated by the FDA might be the best method to ensure standardized and accurate efficacy testing of blood glucose monitors under optimal conditions.

10. Optimal targets for glycemic control in acutely ill patients must be defined to facilitate development of glucose monitor performance standards for these types of patients. The current debate as to what level of tight glycemic control is optimal for hospitalized patients is not settled.¹² Part of the problem in achieving tight glycemic control of acutely ill hospitalized patients is that, when blood glucose monitors that are approved for outpatient use are then used in the hospital, their inaccuracy can lead to inadvertent insulin overdosages and hypoglycemia.¹³ Blood glucose monitors with greater accuracy might allow tighter control by triggering fewer instances of inadvertent insulin overdosages.

Conclusions

Comments of participants at the FDA Blood Glucose Monitor Meeting generated five conclusions:

1. Better analytical performance is needed.

2. Better clinical performance is needed.

3. Better adherence to human factor behaviors that can affect the accuracy of blood glucose monitoring, in addition to inherent monitor accuracy, is needed.

4. Better labeling of interfering substances is needed.

5. Better agreement on glycemic targets of hospitalized patients is needed.

The FDA Blood Glucose Monitor Meeting was characterized by a collegial atmosphere between the medical, industry, patient, and regulatory communities. Multiple problems remain to be solved. Among them are concerns that (1) achieving improved monitor analytical accuracy will be limited by inaccuracy of currently used reference methods; (2) optimal goals for various populations need to be defined, such as by development of multiple population-specific error grids with mandated performance distributions for each error grid; (3) patient education and training for using blood glucose monitors needs to be improved; (4) ongoing vigilance about interfering substances will be needed, because future drugs might be introduced that could interfere with the performance of currently approved monitors; and (5) much more empiric and modeled data are needed to determine target levels of glycemic control in hospitalized patients by optimizing tradeoffs between tighter mean glycemic control, increased hypoglycemia risk, and hyperglycemic complication risk when stratified against varying levels of blood glucose monitor analytical and clinical performance.

The participants at the FDA meeting arrived with a common goal of wanting to see self-monitoring of blood glucose be a useful tool to help people with diabetes. It is anticipated that the regulatory, industry, academic, and patient communities will continue to work together to develop reasonable performance standards for blood glucose monitors. As technology for glucose monitoring improves over time, it is expected that the FDA will progressively tighten the performance requirements for blood glucose monitors. The FDA's blood glucose monitoring meeting on March 16-17, 2010 in Gaithersberg will be regarded as an important milestone in this evolving regulatory process.

Abbreviations

CLSI

Clinical and Laboratory Standards Institute

FDA

Food and Drug Administration

ISO

International Organization for Standardization