Comparison of Accuracy Guidelines for Hospital Glucose Meters

Abstract

When used in hospital settings, glucose meter performance issues involve analytic comparability to lab-based testing, patient and sample variables, and clinical affects such as insulin treatment protocol outcomes and morbidity or outcome risk factors. Different tools are available to assess these issues, including accuracy and precision statistics along with clinical risk measures such as error grids or simulation testing. Regulatory, guidance, and professional bodies have advocated a number of varying recommendations for glucose meter performance in different situations and under different patient conditions. These are summarized and compared, but reconciling these guidelines can be confusing or difficult for providers. Blood glucose meters are useful in the management of patients in acute or assisted care facilities, but users must appreciate the variables that affect measurements and provide for oversight that can manage risk factors and maintain meter performance expectations.

Urine and blood glucose tests were available in the 19th and early 20th centuries, but advances in tablet or dry chemistry allowed rapid testing, leading to glucose meters for professional use and patient self-monitoring of blood glucose (SMBG). While these developments advanced diabetes care, they also presented issues associated with the clinical effects of accuracy and errors in glucose measurements.¹

Early rapid glucose measurements raised concerns about operator technique, ability to differentiate color development related to glucose levels, and patient variability issues. As methods were improved, concerns about technique persisted, but variables associated with the physical device, testing methodology, and the level of accuracy and precision needed for different populations and under different testing conditions became important. Even with specimen handling and information management improvements, concerns about glucose meter performance with certain specimens and in key patient populations have continued, especially with critically ill patients under dynamic clinical states in acute care settings.

In a hospital setting, the analytical performance of glucose meters is paramount to the reliability of test results. Insulin dosage and clinical management of the patient is frequently based on the glucose meter result at the bedside, because a test from the central laboratory requires considerably longer turnaround time. By the time a result is available from the central laboratory, often an hour or more after sample collection, the patient’s condition may have changed, particularly for fragile patients with diabetes or critically ill patients. The rapid availability of results from a glucose meter becomes a convenient means of making timely clinical decisions. Yet, glucose is also routinely reported off chemistry panels that are ordered on hospitalized patients periodically during their stay. Both the glucose meter and the central laboratory results must agree, or there can be clinical confusion regarding patient management.

What is acceptable for analytical agreement between a glucose meter and the central laboratory in a hospitalized patient? Paired results between two different methodologies are rarely identical, even from samples collected at the same time in a patient. This review will describe different tools used to judge the acceptable differences between blood glucose meter systems (BGMS) and central laboratory results, and how those tools have been used for specific professional BGMS performance guidelines.

Clarke error grid analysis (EGA) was used formally in 1987 to compare different instruments used for SMBG.² By grouping glucose values with potential clinical effect of over- or undertreatment into risk zones, EGA was supposed to help make meter performance assessment more clinically relevant and not just rely on analytical statistical parameters that may represent mathematical but not clinical variation between BGMS and central laboratory results. It recognizes that actual value differences may be more important than percentage differences when evaluating the significance of measurement variations.

Koshinsky et al introduced a modification to better combine analytical performance factors with potential clinical effects.³ Analytical statistical and clinical tools should be complementary when evaluating BGMS performance.^4-6 Parkes et al updated and simplified the error grid tool by getting a larger consensus on how to classify variances into risk zones and defining slightly different zones for people with type 1 or type 2 diabetes.⁷ The original Clarke EGA zones may have been arbitrary, set by a few contributors, and were to be evaluated or modified by users. For the Parkes approach, hypothetical clinical cases were stratified into 5 risk zones by 100 endocrinologists experienced in diabetes care. Some of the zones were modified from the original Clarke’s grid, but the concept of classifying analytical glucose variance into clinical risk zones was retained.

To allow for the clinical and technologic changes that had evolved, in 2014 the Surveillance Error Grid, with contributions from academia, industry, and regulatory agencies, was introduced. Over 200 panelists made suggestions for blood glucose values that would define significant differences in several clinical scenarios. This input was merged into a new error grid that has continuous color zones ranging from red to green, indicating relative risks of differences between a meter and reference glucose result.⁸ The new grid is felt to provide more granular data than the prior grids and to allow post–market assessment of glucose monitoring devices.

The different error grids continue to be used in peer publications assessing the risks of using BGMS. Clinicians should be aware of their strengths and weaknesses when reviewing data about the acceptability of using specific BGMS in different settings. The original error grids were developed for assessing the use of SMBG and not for evaluating BGMS use in critical care settings with different types of insulin protocols. However, the concept of using clinically significant error zones associated with analytical variation has been a useful one.

In 2001, the first of several articles addressing “tight glycemic control” or the benefits of “intensive insulin therapy” in certain acutely ill hospitalized patient groups was published.⁹ Subsequent publications raised doubts about this approach,^10,11 and there was increasing discussion about the risks of using BGMS for assessing these populations, the risks of hypoglycemia and deaths with insulin therapy, and the different performance levels needed for BGMS used in SMBG versus acute or chronic care facilities. A key factor involved the variability in capillary samples under certain patient conditions and with certain therapies, but concerns about the accuracy of other sample types (arterial, venous, and line samples) were also raised. There was also the increasing awareness of the complexity of dysglycemia and not just regulation of diabetic control with acutely ill patients. People without diabetes could also exhibit hyperglycemia, hypoglycemia, and glycemic variability in critically ill conditions and be at risk from these states and insulin treatment.¹²

The United States Food and Drug Administration (FDA) began to seriously address concerns about BGMS performance in 2010,¹³ and in 2014 it released a draft guidance followed by a final guidance in 2016, “Blood Glucose Monitoring Test Systems for Prescription Point-of-Care Use”.¹⁴ Each document recommended conditions which manufacturers should test and meet for clearance of BGMS used with critically ill and assisted care patients. Importantly, target performance levels at different glucose levels were proposed and these differed from SMBG for personal use and what the FDA called “prescription use” for assisted care, acutely, or critically ill patients.

The FDA documents highlighted the importance of only using BGMS that are cleared for an “intended use” patient population in specific care settings and by approved types of trained operators. BGMS cleared for SMBG are not automatically cleared for use in a critically ill or acute care population where more stringent performance levels are needed because of more serious patient conditions and risks associated with treatment protocols. In 2013, the Clinical and Laboratory Standards Institute (CLSI) released the guideline point-of-care testing (POCT) 12-A3, “Point-of-Care Glucose Testing in Acute and Chronic Care Facilities,” describing a system laboratories and healthcare facilities should use for implementation and to verify BGM performance targets for use in key patient populations and at different glucose concentrations.¹⁵

Modeling systems allow for computer simulation of test and treatment variables in predicting risk and possible outcomes when assessing situations that have multiple potential combinations of elements. Calculations and graphical demonstrations can be made when a model is constructed and potential possibilities of inputs are considered.¹⁶ Thousands to millions of events can be considered in short periods that otherwise might take years and thousands of real patient events to accumulate, and might be unethical or too risky to evaluate in real patients. Simulation studies are especially valuable when a test value can drive a particular action, such as the decisions made in managing glucose levels of acute care patients. An effective model of complex interactions such as glucose physiology, insulin response, patient condition or treatment variables, different sample types, and other variables can be difficult to construct, but designs to look for risk or influence of key variables such as test quality (e.g. bias and imprecision) can be constructed.

Monte Carlo simulation has been used to estimate the percent of insulin dosing errors dependent on meter precision and accuracy,¹⁷ showing that coefficients of variation (CVs) and bias of glucose meters needed to be <2% to select the correct insulin dose 95% of the time. It has shown bias and accuracy affects insulin treatment protocols for tight glycemic control.^18,19 Simulation modeling has also been used to compare performance characteristics needed for intermittent versus continuous glucose monitors (CGMs) in intensive care settings, indicating that CGM systems can perform well with less stringent bias and precision requirements because they measure results more frequently.^20-23 The results of simulation studies influenced the selection of performance targets for the CLSI document POCT 12-A3.

Questions were raised as to how good performance levels needed to be, especially if care-givers could identify the clinical situations where meter results might not be reliable and if glucose or insulin control protocols were relaxed to minimize the potential for adverse outcomes. Several recent publications have promoted the clinical acceptability of using BGMS in hospital settings. Two publications showed single-institution experiences where retrospective BGMS comparisons to central laboratory testing were deemed to be clinically acceptable across patient and sample types.^24,25 Another study used multicenter paired results from a spectrum of critically ill patients for arterial and venous samples only and applied an algorithm of four comparison tools including analytical, Parkes error grid, simulation modeling, and stratified clinical sensitivity and specificity analyses for one meter type to show clinical acceptability of results.²⁶

In 2018, an FDA Advisory Board reviewed data from three large unpublished studies that showed while two BGMS in acute care settings could meet target performance for arterial and venous samples, they fell short of those standards for capillary samples. However, the Advisory Board believed that the benefits of using BGMS for capillary samples in these settings outweighed the risks, and one BGMS was FDA cleared for use with capillary samples in “prescription” settings.²⁷ Other manufacturers of BGMS were encouraged to submit data and apply for FDA clearance.

The following summarizes the different professional body proposals for performance targets and conditions for acceptable BGMS use. Most of these are based on technical or analytic comparisons of BGMS to a comparative method, usually the central laboratory analyzer. SMBG is included because recommendations have evolved as the different performance requirements for personal and acute care BGMS have been appreciated.

The American Diabetes Association (ADA) first recommended in 1987 that for SMBG, the goal of future systems should achieve variation (system plus user) of <10% at glucose concentrations of 30-400 mg/dL, 100% of the time.²⁸ The ADA recognized that a major source of variability for glucose meter results was the user. In contrast to the accuracy and precision of glucose meters under controlled laboratory conditions where CVs were usually 2%-5%, up to 50% of the SMBG results could vary more than 20% from a reference value for meters in general use. Some of the variables included the size and placement of the blood sample, timing of the test, and removal of blood from the strips with some systems being more skill dependent than others. Additional variability arises from instrument malfunction, use of reagent strips that are outdated or improperly stored, and from physiologic extremes of hematocrit in the patient, as well as imprecision in the hypoglycemic and hyperglycemic range.²⁸

For the then current generation of glucose meters, the ADA recommended that test results should be within 15% of a reference method (Table 1). Individuals who could not consistently meet this criterion should undergo further training until this goal was met. Similar goals were recommended if glucose meters were to be used for patient management in other settings, such as camp, school, or nursing home.²⁸

Table 1.

Meter Performance Criteria.

Meter performance criteria
ADA ’87 (100% of data)	30-400 mg/dL	±15%
ADA ’94 (100% of data)	All levels	±5%
FDA (95% of data)	<75 mg/dL	±12 mg/dL
(Draft 2018 guidance prescription use)	≥75 mg/dL	±12%
And 98% of data	<75 mg/dL	±15 mg/dL
	≥75 mg/dL	±15%
ISO 15197:2013 (95% of data)	<100 mg/dL	±15 mg/dL
	≥100 mg/dL	±15%
And 98% of individual values fall in zone A, B, or C of Consensus Error Grid
CLSI (POCT 12-A3) (95% of data)	< 100 mg/dL	± 12.5%
	≥ 100 mg/dL
And the sum of (1) errors >15 mg/dL for values <75 mg/dL and
(2) errors >20% at ≥75 mg/dL should be <2%
Agence du Médicament	<100 mg/dL	±20 mg/dL
(95% of data)	≥100 mg/dL	±20% (CV <7.5%)
CSA (Z316.4-94, 1999)	<45 mg/dL	±25% (CV<12.5%)
	≥90 mg/dL	±15% (CV <7.5%)
IMSS	<60 mg/dL	±25%
	≥60 mg/dL	±20%
TNO	<117 mg/dL	±20 mg/dL
	<117 mg/dL	±15 mg/dL (CV <10%)

Abbreviations: ADA, American Diabetes Association; CLSI, Clinical and Laboratory Standards Institute; CSA, Canadian Standards Association; CV, coefficient of variation; FDA, United States Food and Drug Administration; IMSS, Instituto Mexicano del Seguro Social; ISO, International Organization for Standardization; POCT, point-of-care testing; TNO, Netherlands Organization for Applied Scientific Research.

The ADA realized in 1994 that manufacturers of SMBG systems could not meet the <10% criteria. The College of American Pathologists found variability of glucose meters ranged from 4% to 33% CV. While some of this variability may be due to matrix effects of the proficiency testing samples, uniform standards for determining the accuracy of SMBG were needed. At this time, the ADA recommended that manufacturers of future SMBG systems set a goal for analytical error of <5%.²⁹

The Canadian Standards Association (CSA) also published performance specifications around the same time for glucose meters for use in diabetes management.³⁰ Each sample was recommended to be tested by both methods in duplicate with no more than 4% drift between replicates for the reference method and using at least four glucose meters. The CSA criteria for glucose meter performance recommended that CV should not exceed 12.5% at blood glucose levels of 45 mg/dL and should not exceed 7.5% CV for the remaining sample concentrations. In addition, the total error (2 × SD + bias) should not exceed 25% at the blood glucose concentration of 45 mg/dL and should not exceed 15% for the remaining sample concentrations.³⁰

The Netherlands Organization for Applied Scientific Research, Technology in Health Care Division, made similar glucose meter recommendations in 1991 with revisions in 2001.^31,32 Using a reference method with traceability to the hexokinase method, 95% of glucose values (ranging from the lowest measurable value to the highest measurable value specified by the manufacturer) should be within ±15% of the laboratory reference method for a value ≥ 117 mg/dL and ±15 mg/dL for values <117 mg/dL. The overall correlation coefficient should be >0.95 over the whole range when using capillary whole blood. The CV of 10 independent observations should be <5% for values ≥90 mg/dL or a maximum SD of <9 mg/dL for glucose values <90 mg/dL. One in 10 results can be outside the requirement for precision.

In 1998, the Instituto Mexicano del Seguro Social recommended criteria for agreement of glucose meters with a reference laboratory method of ± 25% for glucose values <60 mg/dL or ±20% for glucose values ≥60 mg/dL.³³ Also, in 1998, the French Agence du Médicament allowed for 95% of values to be within ±20 mg/dL for values <100 mg/dL and ±20% for glucose values ≥100 mg/dL with a CV of <7.5%.³⁴

More recently, the CLSI has published consensus guidelines recommending that laboratories validate the performance of their model of glucose meter against the central laboratory using at least 40 fresh whole-blood samples that span the entire measuring interval of the glucose meter. Duplicate laboratory results should match the certified reference material within 4% or ± 4 mg/dL whichever is greater. Meter performance is acceptable for use in hospitals when 95% of the individual results from the glucose meter agree within ±12 mg/dL of the laboratory analyzer values at glucose concentrations below 100 mg/dL and within ± 12.5% of the laboratory analyzer values at glucose concentrations at or above 100 mg/dL. In addition, the sum of the number of individual results with (1) errors that exceed 15 mg/dL at glucose concentrations below 75 mg/dL and (2) errors that exceed 20% at glucose concentrations at or above 75 mg/dL should not exceed 2% of all results.

While CLSI directs their recommendations to healthcare professionals using glucose meters in the hospital, the International Organization for Standardization (ISO) directs their recommendations to manufacturers of glucose meters.³⁵ A reference method that conforms to traceability requirements of ISO 17511 should be used to assign glucose reference values. Duplicate measurements using an IVD medical device (in a medical laboratory) shown to have adequate performance characteristics may be used to assign reference values with traceability and performance information obtained from the manufacturer. ISO recommends that glucose meters meet the following minimum acceptable system accuracy: (1) 95% of measured glucose values shall fall within either ±15 mg/dL of the average measure values of the reference measurement procedure at glucose concentrations <100 mg/dL or within ±15% at glucose concentrations ≥100 mg/dL, and (2) 99% of measured individual glucose values shall fall within zones A and B of the Parkes Error Grid for type 1 diabetes. The first criterion applies to each reagent lot individually, while the second criterion applies to three reagent lots taken together.³⁵

The FDA also made recommendations targeted to manufacturers as well as FDA staff. The FDA stresses the evaluation of glucose meter system accuracy to support professional use of the devices in the intended use population.¹⁴ Criteria for demonstrating sufficient accuracy for use by healthcare professionals require the manufacturer to demonstrate that 95% of all values are within ±12% of a comparator method for glucose concentrations ≥75 mg/dL and within ±12 mg/dL at glucose concentrations <75 mg/dL. In addition, 98% of values should be within ±15% of the comparator method for glucose concentrations ≥75 mg/dL and within ±15 mg/dL at glucose concentration <75 mg/dL.¹⁴

Clinicians should realize that there is no universal consensus for comparison of glucose results between different methods. When implementing a new glucose device or comparing glucose results between different methods, they should pick the agreement criteria in advance of performing the study and utilize those specifications which most appropriately fit their region of the world and the clinical protocols that are interpreting the glucose results as some applications may allow for more variability (such as adjusting diet, exercise, and insulin dose for outpatients) than others (like tight glycemic control on intensive care patients). While some standards are directed at manufacturers, these same criteria are equally applicable for use by clinicians in comparing glucose results between a meter and the laboratory in a hospital.

All of the peer guidelines for glucose meter performance recommend comparison of the meter result with a reference result from a laboratory instrument using the “same” sample. Although clinicians are faced with results from multiple samples and methods in the patient’s medical record (capillary fingerstick glucose meter results side-by-side venous blood results from a central laboratory and sometimes blood gas analyzers using whole blood), there are no guidelines that address performance specifications comparing two different sample types. Physiologically, glucose concentrations in capillary blood are only 2-5 mg/dL higher than venous blood in a fasting patient. However, postprandially, glucose levels can be 20-70 mg/dL (equivalent to 20%-25%) higher in capillary blood than a concurrently drawn venous sample.^36-38 Glycolysis will decrease the glucose concentration by 5%-7% an hour in whole blood at room temperature. Centrifugation and separation of serum or cell-free plasma off the cells will stabilize glucose concentrations for several hours at room temperature and for several days when refrigerated.³⁸ So, analysis must occur rapidly after collection of blood or the sample must be centrifuged, and serum/plasma separated for laboratory analysis.

Comparing glucose meter results to a laboratory instrument is not straightforward and clinicians are warned to consider the many factors that may contribute to differences between a glucose meter result and a laboratory glucose result in the patient’s medical record, beyond the actual meter analytical performance. There are other patient conditions and technical factors that may cause variation between BGMS and central laboratory or blood gas analyzer results. These are further discussed in detail in a recent IFCC document addressing many technical and analytic factors affecting blood glucose meter analysis for critically ill patients³⁹ and summarized in Table 2.

Table 2.

A Partial List of Preanalytic Variables to Consider When Comparing Glucose Methods (from the IFCC Guideline).³⁹

Pre-analytical variables for glucose comparisons
Quality assurance program
Training and competency of staff
Analytical and clinical goals for intended patient populations (variability tolerance)
Glycolysis and delays in processing plasma/serum for laboratory comparison
Anticoagulant preservatives
Poor peripheral circulation—collection of fingerstick capillary blood
Extremes of hematocrit—low or high hematocrit
Hypoxia and acid/base disturbances
Drug interferences (acetaminophen, triglycerides, uric acid, vitamin C and icodextrin)
Altered protein or lipid levels
Sedation or abnormal mental status that can mask hypoglycemia

BGMS are a useful and effective tool for management of patients with dysglycemia, but the factors affecting analytic comparability or clinical acceptability between BGMS and central laboratory or blood gas analyzer glucose results are complex, especially with acute care or critically ill patients. There is yet no universally accepted single method to judge comparability and acceptability, but there are tools to make these assessments. The different systems for performance targets can be confusing and difficult to reconcile. A common theme is that the better the analytic agreement between methods, the better the clinical acceptability will be. However, while BGMS and central laboratory methods may be comparable under tightly controlled circumstances, the variables associated with point-of-care BGMS, including patient conditions and sample type, pose a definite degree of uncertainty. To minimize this uncertainty and risk, stakeholders need to be aware of, respect, and control the known variables as conscientiously as possible. One of the best ways to do this should be to first educate operators and stakeholders about the concepts of variables, intended use, and associated risks. This should then be reinforced with comprehensive quality systems that monitor not only technical performance but also clinical events and outcomes.