Intraoperative Accuracy of a Point-of-Care Glucose Meter Compared with Simultaneous Central Laboratory Measurements

Abstract

Background

Concerns have been raised about the use of point-of-care (POC) glucose meters in the hospital setting.

Accuracy has been questioned especially in critically ill patients. Although commonly used in intensive care units and operating rooms, POC meters were not approved by the Food and Drug Administration for such use.

Data on POC glucose meter performance during anesthesia are lacking. We evaluated accuracy of a POC meter in the intraoperative setting.

Methods

We retrospectively reviewed 4,333 intraoperative records in which at least one intraoperative glucose was measured using electronic medical records at a large academic hospital. We evaluated the accuracy of a POC glucose meter (ACCU-CHEK® Inform, Roche Pharmaceuticals) based on the 176 simultaneous central laboratory (CL) blood glucose (BG) measurements that were found (i.e., documented collection times within 5 minutes). Point-of-care and central lab BG differences were analyzed by Bland-Altman and revised error grid analysis (rEGA).

Results

Mean POC BG was 163.4 ± 64.7 mg/dl [minimum (min) 48 mg/dl, maximum (max) 537 mg/dl] and mean CL BG was 162.6 ± 65.1 mg/dl (min 44 mg/dl, max 502 mg/dl). Mean absolute difference between POC and CL BG was

24.3 mg/dl. Mean absolute relative difference was 16.5% with standard deviation 26.4%. Point-of-care measurements showed a bias of 0.8 relative to the corresponding CL value, with a precision of 39.0 mg/dl. Forty (23%) POC BG values fell outside the Clinical and Laboratory Standards Institute guideline and 3.4% POC measurements fell in zones C and D of the rEGA plot.

Conclusions

The tested POC glucose meter performed poorly compared to a CL analyzer intraoperatively. Perioperative clinicians should be aware of limitations of specific POC glucose meters, and routine use of POC glucose meters as sole measurement devices in the intraoperative period should be carefully considered.

Introduction

Although point-of-care (POC) glucose meters are used in many hospitals for analysis of samples from critically ill patients, performance of such devices in this patient population has not been evaluated as part of the Food and Drug Administration (FDA) 510(k) clearance process. Because of concerns about accuracy in critically ill patients, the FDA held a public meeting about clinical accuracy requirements for POC glucose meters and tight glycemic control.^1,2 A need for improved performance of POC glucose meters and greater attention to human factors affecting device accuracy were articulated at the meeting. Moreover, it was suggested that different populations should have separate analytical standards as well as separate clinical performance standards. This is true especially in critically ill patients on tight glycemic control.² Indeed, several studies reported poor performance of POC glucose meters in critically ill patients in the intensive care unit (ICU) environment.^3–6

Patients with diabetes frequently need anesthesia for various surgical procedures. Accuracy of blood glucose (BG) determinations might be even more important during anesthesia because patients cannot report symptoms of hypoglycemia during general anesthesia, and their capacity to inform may be impaired during procedures performed under sedation. Furthermore, autonomic responses may be masked by anesthetic agents and other drugs (e.g., beta blockers, opioids), making it even harder to recognize hypoglycemia.⁷ Concerns have been raised about the safety and accuracy of peri-operative use of POC meters.^8,9 However, data on POC glucose meters performance in the operating room (OR) are lacking. Point-of-care glucose meter accuracy perioperatively and in the ICU has been reviewed.^8–10 Rice and colleagues reviewed possible errors using POC glucose meters in the OR.⁸ They identified a number of factors, including various medications (e.g., ascorbic acid, mannitol, acetaminophen, dopamine), hematocrit, oxygen concentration, pH, hypothermia, and hypotension, that could influence the accuracy of POC measurements. Rice and colleagues also commented on the lack of studies in the OR environment, that the spectrum of POC glucose meter accuracy is unknown, and that accuracy would also be affected by the training of the particular operator.⁸ They concluded that many health care professionals do not appreciate that POC glucose meters results cannot simply be substituted for central laboratory (CL) measurements.

This study addresses this knowledge gap by comparing the accuracy of a POC glucose meter used intraoperatively during routine clinical care with simultaneous samples assayed by a CL analyzer.

Methods

Study Design and Data Collection

After obtaining Institutional Review Board approval with waiver of informed consent, we performed a retrospective review of all surgical cases performed in our university hospital (Thomas Jefferson University Hospital, Philadelphia, PA) from October 25, 2005, to November 23, 2010, in which at least one intraoperative glucose was measured (n = 4,333). Using data from our anesthesia information management system (AIMS), we obtained all BG and hemoglobin values, sample times, and measurement methods (CL or POC) entered in the AIMS during the intraoperative period. Providers were unaware that the accuracy of their POC measurements was being measured. There isno official perioperative BG control target or protocol in our institution. However, most of the clinicians use our institution's computerized insulin calculator, which is designed for ICU patients with a BG target between 140 and 180 mg/dl.

Blood Glucose Measurements

Glucose values were determined using a Beckman LX20 chemistry analyzer (Beckman Coulter, Brea, CA) in the hospital's central laboratory or ACCU-CHEK® Inform glucose meters (Roche Pharmaceuticals, Basel, Switzerland) in the OR. Point-of-care devices were maintained according to standards established by the institution's department of pathology, in accordance with the Clinical Laboratory Improvement Amendments requirements for POC glucose meters. Control checks were performed at least every 24 hours, and provider training, including yearly satisfactory performance of test measurements using a high and low control sample, was ensured. Values measured by the CL were automatically transmitted via an interface into the AIMS and could not be edited, while POC values were manually entered by providers. All BG measurements (POC glucose meter and CL analyzer) were done on whole blood samples. The AIMS does not record the site of the blood sample. Central laboratory specimens were determined to be arterial if they were sent as part of a blood gas panel.

Comparison of Simultaneous Point-of-Care and Central Laboratory Blood Glucose Values

Blood glucose determinations from the POC device and CL were considered to be simultaneous if the specimen collection time on the lab slip was within 5 minutes of the timestamp of the POC determination in the AIMS. A 5-minute window was selected to account for potential clock differences between the AIMS workstation clocks and the different time source used by the circulating nurse who filled out the lab slips.

Statistical Analysis

Differences between simultaneous POC and CL values were analyzed using the methods of Bland and Altman¹¹ and the revised error grid analysis (rEGA).¹² The rEGA is based on the Clarke EGA¹³ and divides the grid in a manner that avoids the discontinuities among zones that are present in the original graph. We plotted values on the noninsulin-dependent rEGA grid because a large majority of our patients have type 2 diabetes and we could not determine retrospectively which patients had type 1 or 2 diabetes. The influence of the hemoglobin on the bias between simultaneous POC glucose meter and CL BG from arterial samples was assessed by a general linear model (SYSTAT version 12.0, Chicago, IL). Values are reported as mean ± standard deviation (SD).

Results

Point-of-Care Glucose Meter vs Central Laboratory Values

A total of 21,028 BG determinations were recorded in the AIMS, 61.2% of which were performed by the central laboratory. There were 176 simultaneous POC and CL glucose values suitable for analysis, of which 164 (93.2%) were arterial and 12 (6.8%) were venous specimens. The paired samples represented 160 different cases (145 with one comparison, 14 patients with two comparisons, and 1 with three comparisons). Mean POC BG was 163.4 ± 64.7 mg/dl [minimum (min) 48 mg/dl, maximum (max) 537 mg/dl] and mean CL BG was 162.6 ± 65.1 mg/dl (min 44 mg/dl, max 502 mg/dl). Mean absolute difference (MAD) between POC and CL BG was 24.3 ± 30.5 mg/dl. Mean absolute relative difference was 16.5% ± 26.4%. Because CL glucose values are considered to be a gold standard, the CL value was selected for the x-axis on the Bland-Altman plot rather than the average of the paired values (Figure 1).¹⁴ There was no difference in the bias comparing the arterial and venous samples (p = .46), so all samples were combined on the plot. Point-of-care measurements showed a bias of 0.8 mg/dl relative to the corresponding CL value, with a precision of 39.0 mg/dl. There was considerable deviation outside the Bland-Altman 95% limit lines of agreement (bias ± 1.96 SD). There was also wide deviation outside the agreement limit of 75 mg/dl above (±20%) and below (±15 mg/dl) the value as specified by the Clinical and Laboratory Standards Institute (CLSI)-approved guideline EP7-A2.¹⁵ Forty (23%) POC BG values fell outside the CLSI guideline, with 20 above and 20 below (Figure 1). There were only 10 BG values below 75 mg/dl. This number was too low to perform a separate analysis in the hypoglycemic range.

Figure 1.

Bland-Altman plot of simultaneous point-of-care and central laboratory blood glucose values. The MAD was 24.3 mg/dl. The bias (dotted purple line) was 0.8. The dotted red lines represent the 95% limits of agreement and the splayed green lines represent limits of agreement of ± the maximum of 15 mg/dl and 20% of the reference lab value. The ACCU-CHEK Inform glucose meter was used for all POC measurements.

The rEGA plot (Figure 2) of the simultaneous POC and CL BG values demonstrated that most patients (96.6%) in the analysis fell into zones A or B (no or minimal effect on clinical outcome). However, 5 patients fell into zone C (altered clinical action that is likely to affect clinical outcome) and 1 patient was in zone D (altered clinical action that could have significant medical risk). There were no patients in zone E (altered clinical action that could have dangerous consequences).

Figure 2.

Revised error grid analysis for 176 simultaneous point-of-care and central laboratory blood glucose values. The rEGA for type 2 diabetes patients was used. Point-of-care BG values are plotted on the y axis versus CL BG values on the x axis for simultaneous samples taken from the same site. The grid is divided into zones that represent the degree of risk created by each error: zone A represents no effect on clinical action; zone B represents altered clinical action with minimal effect on clinical outcome (170 patients together); zone C (5 patients) represents altered clinical action that is likely to affect clinical outcome; zone D (1 patient) represents altered clinical action that could result in significant medical risk; and zone E (no patients) represents altered clinical action that could have dangerous consequences. The ACCU-CHEK Inform glucose meter was used for all POC measurements.

There was no influence of the hemoglobin concentration (9.7 ± 1.9 mg/dl, range 3.3 to 15.5 mg/dl) on the bias between POC glucose meter and CL BG values (r² = 0.018, p = .09), using data from the 164 arterial blood gas specimens.

Discussion

The result of our study showed that the accuracy of BG determinations using the POC device, measured under clinical conditions in the OR, was poor (precision = 39.0 mg/dl and MAD = 24.3 ± 30.5 mg/dl). Since studies about POC glucose meters accuracy during anesthesia are lacking, we compared our study to similar evaluations in ICU patients. Our data confirmed poor performance of POC glucose meters in the OR, as reported earlier in the ICU.^3–6 The bias in our study (0.8 mg/dl) was significantly lower and precision was worse (39.0 mg/dl) than in the retrospective Finkielman and colleagues study (7.9 mg/dl, precision = 17.6 mg/dl, p = .02)⁶ and the prospective Critchell and colleagues study (8.6 mg/dl, precision = 18.6 mg/dl, p = .01).¹⁶ Other ICU studies had biases similar to ours. Desachy and colleagues, in a prospective study using capillary and whole blood (arterial or venous) samples, had a bias of 1.4 mg/dl and a precision of 20.5 mg/dl (p > .05).¹⁷ Kulkarni and colleagues, in another prospective study using capillary blood and arterial blood samples, had a bias of 2.6 and a precision of 6.9 mg/dl (p > .05).¹⁸ Ray and colleagues used only arterial blood samples and had a bias of 0.7 and a precision of 20.7 mg/dl,¹⁹ and Vlasselaers and colleagues also used arterial blood samples and had a bias of -6.3 and a precision of 10.1 mg/dl.⁵ Although the biases in these studies were not statistically different than in our study, precision was approximately twice as high in our study. Some ICU studies used MAD as opposed to the standard deviation of the bias to assess accuracy of POC glucose meters. Our study had a much higher MAD (24.3 mg/dl) than Hoedemaekers and colleagues (5.8 mg/dl)³ and Maser and colleagues. (10 mg/dl).²⁰ Taken all together, these data suggest that performance of a POC blood glucose meter in the OR environment could be worse than in ICUs.

Recognizing hypoglycemia in diabetes patients and in those at risk for hypoglycemia is one of the most important tasks in the perioperative period. Fear of hypoglycemia even precludes some clinicians from using insulin. In our study, we had too few BG values in the hypoglycemic range to make any conclusion. Of interest, however, is that 2 of the 10 patients with a CL BG less than 75 mg/dl had a corresponding POC measurement that was more than 100 mg/dl higher (44 vs 155 and 74 vs 196 mg/dl, respectively). This could be particularly dangerous if an anesthetized patient with a BG >150 mg/dl were treated with insulin when the real BG was <50 mg/dl.

The retrospective nature of our study precluded analysis of potential reasons for poor accuracy in our patient population. Dynamic changes in patients' physiologic status (e.g. rapid changes in hematocrit, blood volume, acid-base status, body temperature, anesthetic depth, sympathetic discharge, and peripheral vasoconstriction) may have an even higher impact on POC performance during surgery than in the ICU because of the more acute fluctuations in the OR. In addition, the multitude of competing tasks in the OR may distract anesthesia providers from complete attention to details of the analytical technique and contribute to the poor accuracy found in our study. Although anesthesia providers at our institution receive yearly training on the POC devices and pass quality control checks, there is a wide variability in the frequency of POC device use by individual providers that may influence the overall precision of the results.

Our data support the concern that caution is required in interpreting POC glucose meter results measured in the OR, as there were large, unpredictable errors in both directions from the reference BG value. The number of BG values outside CLSI guideline limits was high (23%) and evenly split above and below the CLSI limits. We did not detect a significant trend with the ACCU-CHEK Inform POC glucose meter (Figure 1). In contrast, lower BG values in the lower range and higher BG values in the higher range were found in ICU patients using the same model glucose meter.⁵ However, such a trend was not evident with the HemoCue® Glucose 201 (HemoCue, Derbyshire, UK) POC glucose.⁵ These data highlight the importance of understanding the wide limits of variation between the value displayed by a POC glucose meter and the actual BG. Different POC glucose meters have different biases and precisions and may have varying performance in different environments (e.g., OR vs ICU) or with varying patients' conditions. Stress on clinicians dealing with rapid physiologic changes in the OR and urgency in obtaining BG samples might also influence the performance of POC glucose meters by inducing preanalytical error. Accordingly, our data showed that 3.4% of POC measurements fell in zones C and D of the rEGA plot (Figure 2) and could adversely affect clinical outcome.

The POC glucose meter used in our study (ACCU-CHEK Inform) has been reported to be affected by hemoglobin levels by showing a negative bias at high hemoglobin levels.²¹ In our study, hemoglobin did not have a significant bias (r² = 0.018, p = .09) on BG values despite a wide range of hemoglobin values (3.3 to 15.5 mg/dl).

Our study has several limitations. Because POC BG values were entered manually into the AIMS system, these were subject to data entry errors. However, when we checked a sample of POC values entered into the AIMS with the central database to which the POC devices transmit their data when placed in their charging stations, there was >98% concordance between the manually entered and electronically transmitted data. Our AIMS is configured such that laboratory values automatically sent from the laboratory cannot be edited, so this was not an issue for the CL measurements. We were not able to determine precisely the time interval from specimen collection in the OR to performance of the CL assay. Prolonged delays can result in false decreases in the lab measurements, as glycolysis continues after the specimen is drawn. However, 96.6% of the specimens sent to the CL were heparinized and sent “stat” as part of a blood gas panel, and appeared to have been processed promptly. Results were called back by phone prior to being entered into the lab database, and then subsequently sent to the AIMS via an interface. Because 90% of results (n = 4724) were transmitted to the AIMS within 31 minutes of specimen collection, this implies that nearly all determinations were completed within 30 minutes of specimen collection.

Conclusions

This study showed considerable discrepancy between simultaneous POC and CL BG measurement measured during routine intraoperative clinical care. Performance of POC glucose meters in patients during anesthesia may be even less accurate than in ICU patients, in whom such concern already exists. Perioperative clinicians should be aware of the limitations of specific POC glucose meters, and routine use of POC glucose meters as sole measurement devices in the intraoperative period should be carefully considered.

Glossary

Abbreviations

(AIMS)

anesthesia information management system

(BG)

blood glucose

(CL)

central laboratory

(CLSI)

Clinical and Laboratory Standards Institute

(FDA)

Food and Drug Administration

(ICU)

intensive care unit

(MAD)

mean absolute difference

(max)

maximum

(min)

minimum

(OR)

operating room

(POC)

point-of-care

(rEGA)

revised error grid analysis

(SD)

standard deviation