Analysis of the Performance of the CONTOUR^® TS Blood Glucose Monitoring System: When Regulatory Performance Criteria Are Met, Should We Have Confidence to Use A Medical Device with All Patients?

Abstract

The article entitled, Performance of the CONTOUR^® TS Blood Glucose Monitoring System, by Frank and colleagues in this issue of Journal of Diabetes Science and Technology, demonstrates that the CONTOUR^® TS glucose meter exceeds current regulatory expectations for glucose meter performance. However, the appropriateness of current regulatory expectations, such as International Organization for Standardization (ISO) 15197:2003, is being reevaluated because of increasing concern regarding the reliability of glucose meters in ambulatory and hospitalized environments. Between 2004 and 2008, 12,673 serious adverse events with glucose meters that met the ISO 15197 expectations were reported in the Food and Drug Administration–Manufacturer and User Facility Device Experience surveillance database. Should different glucose meter performance criteria be applied to ambulatory versus critical care patients?

In March 2010, the U.S. Food and Drug Administration (FDA) held a public forum to address issues of whole blood glucose meter adequacy in meeting clinical expectations.¹ Discontent is growing in both public and health care communities regarding the reliability of glucose meters. Evidence fueling this concern can be found in the Manufacturer and User Facility Device Experience (MAUDE) surveillance system, which has recorded 12,673 serious adverse events with whole blood glucose meters between 2004 and 2008. The investigation by Frank and colleagues, reported in Performance of the CONTOUR^® TS Blood Glucose Monitoring System, in this issue of Journal of Diabetes Science and Technology clearly indicates that this blood glucose monitoring system meets current approved glucose meter performance guidelines developed by the Clinical Laboratory Standards Institute (CLSI) and the International Organization for Standardization (ISO 15197).^2–4 Error grid analysis graphically depicts when glucose meter results fall within the accuracy goals and has been applied successfully by the authors. Realizing that thousands of serious adverse events have been reported with approved devices, one ponders whether current regulatory expectations for glucose meter performance are adequate for all patients. Are the goal posts used to gauge expected accuracy too wide (Figure 1 )?Should devices for self-monitoring of blood glucose and assisted-monitoring of blood glucose in ambulatory or critical care settings have different performance criteria?

Figure 1.

Achieving our goals! (but are the goal posts too wide?)

MAUDE is the database established by the FDA in June 1993 to collect voluntary reports of adverse events involving medical devices. This database has open access and can be queried online to offer reports on patient outcomes associated with the use of glucose monitoring devices.⁵ MAUDE’s outcome data have the potential to indicate if the existing regulatory performance expectations ensure patient safety. An analysis of 4,629 glucose meter reports in the MAUDE database for a 6-month period from August 2006 to January 2007 revealed that 70.8% of events were categorized as “malfunction,” and 47.7% of the identified device problems were related to test results being inaccurate.⁶Eleven deaths (0.2%) and 1,259 injuries (27.2%) were also documented. In preparation for this analysis response to the article by Frank and coworkers, we conducted a search of the MAUDE database (January 1, 2010, to September 30, 2010) for the CONTOUR^® TS meter (Bayer Healthcare, Tarrytown, NY) and found 3 “injury” and 36 “malfunction” reports. Recognizing that approximately 6.2 billion glucose meter measurements are conducted each year in the United States, it is not yet clear if the small proportion of serious adverse events in the MAUDE database indicates the success of current regulatory standards, or if more stringent standards could prevent or reduce the number of adverse outcomes reported overall or in patient subgroups.

What criteria need to be met to ensure sufficiently accurate glucose meter performance for patients? Both CLSI and ISO continue to develop and refine their guidelines for assessing glucose meter performance; the CLSI Point-of-Care Testing (POCT) revision, POCT12-A3, is in preparation as of 2010. In ISO 15197, the minimum acceptable performance expectations for glucose meter performance are that 95% of the individual glucose results shall fall within ±15 mg/dl (0.83 mmol/liter) at low glucose concentrations (<75 mg/dl, <4.2 mmol/liter) and within ±20% when glucose concentrations are higher. The performance goals used to evaluate glucose meters need to be sufficiently narrow to enable consistent medical decisions. At this time, there is a single standard applied to glucose meters used on all patients. As Frank and colleagues have shown in their evaluation, when ambulatory adults are assessed, devices such as the CONTOUR TS have performance that clearly surpasses the ±20% criteria. The percent concordance of glucose results for capillary blood and the Yellow Springs Instrument Co. (Yellow Springs, OH) plasma reference method was between 97.9 to 98.6% at the ±20% expectation and 91.7 to 93.3% at a ±15% expectation. When venous blood was analyzed, the percent concordance increased to 99.8% at a ±20% expectation and 97.9% at a ±15% expectation. The patient population examined in this evaluation represents an ambulatory diabetic community. Relative to the current analytical goals outlined in the error grid figures, we note the excellent analytical performance with the CONTOUR TS and that this performance has also been observed with many other commercial glucose meters when evaluation is performed on ambulatory patients without anemia.

When regulatory performance criteria are met, should we have confidence to use a medical device with all patients? The report by Frank and colleagues fulfils regulatory performance expectations for use of the CONTOUR TS; however, it did not evaluate performance over the full range of glycemia and hematocrits observed in hospitalized and critical care adults, pediatrics, and neonates. These heterogeneous patient groups require stringent glucose monitoring, and the composition of their blood is more variable than in healthier populations. Whole blood glucose monitoring devices are often prone to matrix effects that can modify analytical performance, and the extent of interference may depend on glucose concentration.^7,8 To achieve performance targets with these heterogeneous hospitalized patients implies that the methods need to be precise and relatively free from matrix influences. In the future, it is possible that regulatory agencies could require assessment of matrix effects on whole blood glucose testing to ensure confidence in the analytical performance of these devices with all patients. Alternatively, the possibility exists that distinct performance criteria could be established for devices used in hospitalized and critical care patients. As the future unfolds, we expect the MAUDE database will continue to provide information to manufacturers and consumers on adverse outcomes, and we hope that regulatory agencies will aim to develop performance criteria to ensure reliable whole blood glucose testing in communities, hospitals, and critical care units.

Abbreviations

CLSI

Clinical Laboratory Standards Institute

FDA

Food and Drug Administration

ISO

International Organization for Standardization

MAUDE

Manufacturer and User Facility Device Experience