(Analytical) Accuracy of Blood Glucose Meters and Patients: How Do They Come Together?

Having been ignored for the most part, analytical accuracy of blood glucose (BG) meters is now the subject of a considerable series of studies (manuscripts). While the results of these studies show considerable differences between meters, they also show that many meters have good analytical quality, i.e., in the hands of highly trained technicians and optimally controlled environmental conditions, these meters are able to measure BG in blood samples with such precision (as compared with a laboratory method) that a good percentage of them would also fulfill the requirements of the new International Organization for Standardization (ISO) standard (±15%).

Such standards are supposed to represent what is achievable; however, it is not clear whether the anticipated Food and Drug Administration (FDA) guidance, which is expected to be more stringent than the proposed new ISO 15197 standard, can be met by any current BG meter at all points. Assuming that the FDA will go for a requirement of ±10% accuracy at all points within the claimed ranges for interferences and environmental conditions and knowing that the current BG meters have a total error (coefficient of variation) of approximately 4.0% to 4.5%, then then average bias of the system must be within ±2.6% to meet this goal (the center of the distribution in a normal distribution must be close to 0, otherwise too much of the distribution would fall outside the accuracy limits). The ±10% accuracy expected at non-nominal conditions is more stringent than the ±15% accuracy expected at nominal conditions; the nominal condition is normal hematocrit (approximately 42%) and “normal” temperature, relative humidity, altitude, and ranges for interferences. Typically, system performance is optimized at the nominal condition and allowed to degrade slightly moving farther from it. Since most systems barely achieve ±10% accuracy at nominal conditions, it is unrealistic to expect it at the extremes of the operating range.

We might please all the clinical chemists who work for the regulatory authorities and the notified bodies that provide a Communauté Européenne (CE) mark for meters in Europe if we expect high analytical accuracy and raise the requirements in the standards; however, after the pendulum has swung from one extreme to the other, is our current focus on analytical accuracy a bit overdone? In other words, for patients with diabetes, should we ignore, to a given extent, what type of accuracy--and how much accuracy--they achieve and actually need in daily life? It is probably worth considering, at first, what factor(s) would make a relevant difference. Is it:

• 1.Statistical significance (this is a mathematical calculation),
• 2.Clinical significance (a difference likely to result in improved/worsened clinical outcomes), or
• 3.Cost effectiveness (a calculation to determine the cost-to-benefit ratio of an action).

With respect to clinical significance of accuracy, we have to admit that, unfortunately, we have no well-designed long-term randomized controlled trials (with a head-to-head comparison) demonstrating that BG meters with different analytical accuracy result in meaningful clinical differences in patients with diabetes. This lack of evidence does not mean that I do not believe that improved accuracy leads to better clinical outcomes; currently, I fully believe that improved accuracy might be better and should not be worse. The outcome of such studies might clearly depend on the patient group we are focusing on, e.g., pregnant women with type 1 diabetes might be much more sensitive to insufficient analytical measurement quality than elderly patients on oral antidiabetic drugs. However, also under such circumstances, it might very well be that it is not the analytical measurement quality of the meters per se that drives the result but the “total system performance” (TSP). What do we mean by this?

We have to acknowledge that the analytical accuracy of a given BG meter is only one component in the overall TSP of the meter in the hands of patients. In practice, there is a chain of additional steps/aspects that determine the quality with which patients measure their capillary BG levels in daily life. Meters and testing supplies are also affected by environmental (e.g., temperature, humidity, and air pressure) and physiological variables (e.g., hematocrit). There are also other factors that many patients do not think about that can cause incorrect readings, e.g., improper storage of test strips or expired strips. Technical safeguards might help prevent common user mistakes by providing warnings in case of underdosing (not enough blood on the test strip) or if test strip expiration is automatically detected. In the end, all steps and factors have to be taken into account in evaluating the TSP of a BG meter: system limitations, system safeguards, labeling, quality assurance, and support and education. It would be good to have standards for these factors as well (the recent revision of the ISO standard is headed in this direction); however, systematic studies investigating these factors are almost nonexistent.

It is also possible that a BG meter that has a high analytical accuracy, but is difficult to handle or has other shortcomings, could fail under such circumstances. Many meters are easy to use for young people with good eyesight and high dexterity; however, for elderly people, it can be a highly frustrating procedure to insert a small electrode into a small slit in the meter and to place a small blood drop—they can hardly see—toward the tip of the test strip. Considering the “stress” a BG meter will undergo in the daily life of an active patient, it could be that such a patient would place more value on durability rather than analytical accuracy. Likewise, for an athlete, what might be more important than anything else is to be able to measure his BG with one hand while riding a bicycle.

One measure to evaluate the measurement quality (i.e., TSP) in patients’ hands is to perform a parallel measurement while they are visiting their treating physician. In principle, comparison with a laboratory method can provide relevant information; in practice, great care must be taken to measure the same blood sample (capillary blood) using both methods. If it is not practical for the laboratory to measure capillary blood, it is essential that the glucose concentration be in steady state. Both measurements must be performed instantaneously (e.g., to avoid glycolysis or the degradation of the sample as time passes). I have no knowledge as to how often in practice such parallel measurements are performed; probably not very often. Even if there is a bias to the true glucose value (hopefully measured by the laboratory method) caused by patients measuring their glycemia over and over again (intraindividual), for them, the absolute accuracy of the measurement would be less relevant than reproducibility and precision. Another limitation of this approach is that the agreement between the BG meter and the laboratory methods might vary in different ranges of glycemia, i.e., many BG meters have limited accuracy in the lower BG range.

I would like to propose a round table discussion in which patients with diabetes (especially) and diabetes nurses (along with diabetologists, scientists, representatives from manufacturers, and regulatory bodies) examine which aspects are important for handling BG meters in daily life and how such “human factors” can be evaluated appropriately (this event can be organized by an independent organization such as Diabetes Technology Society). My emphasis toward practical use is clearly in response to the human factor approach initiated by the FDA; however, one wonders how this is implemented in practice and to what extent the groups mentioned are involved. As with many others aspects of BG meter evaluation, it might be best that it is performed by an independent institution to avoid a certain bias. This independent institution should also perform random “off-the-shelf” testing of all approved products on a regular basis to make certain that the quality of the product is maintained post-market.

We should also think about the “price” we are willing to pay for analytical accuracy. If it were possible to manufacture a BG meter that measures BG with an accuracy of ±10%, for example, but the cost would be three times higher, would this be a good balance? From a clinical perspective, what does this added accuracy mean and what difference would it make? In other words, how much better must accuracy be to observe a “clinically meaningful difference”? Better accuracy is always desirable, but how much better must the accuracy be to afford a meaningful clinical difference? What does this mean exactly when it comes to the frequency of hypoglycemic events, metabolic control, and insulin dosing?

In summary, a move toward testing the accuracy of BG meters in the hands of patients (probably separated for different patients groups/patient needs) might be the next step. It would simply be good to see more studies evaluating the TSP of modern BG meters. It is also clear that it can be assumed that BG meters in which all handling steps required are done automatically reduce handling errors and further improve measurement reliability in patients hands.

Glossary

(FDA)

Food and Drug Administration

(ISO)

International Organization for Standardization

(TSP)

total system performance

Disclosures

Lutz Heinemann advises various companies in the development of new diagnostic and therapeutic approaches to diabetes therapy. He is a shareholder and consultant at Profil Institute for Metabolic Research, Neuss, Germany, and Profil Institute for Clinical Research, San Diego, CA.