Skip to main content
NCBI home page
Indian Journal of Clinical Biochemistry logoLink to Indian Journal of Clinical Biochemistry
. 2021 Mar 25;37(2):199–205. doi: 10.1007/s12291-021-00971-4

Clarke Error Grid Analysis for Performance Evaluation of Glucometers in a Tertiary Care Referral Hospital

Sohini Sengupta 1,✉, Anil Handoo 2, Inaamul Haq 3, Karamvir Dahiya 4, Sanjay Mehta 5, Mradul Kaushik 6
PMCID: PMC8993970  PMID: 35463101

Abstract

Glucometer is the most commonly used POCT device and guides monitoring of blood glucose level in both clinical settings and outside. Inaccurate glucometer readings resulting in erroneous therapeutic intervention has critical consequences on patient care. Regulatory guidelines for performance evaluation of glucometers are not available in many countries. A robust program implemented by the hospital is essential to ensure accuracy and precision of glucometers to produce optimal results. The objective of this study was to design a quality assurance program for the evaluation of glucometers in a high volume tertiary care referral hospital and evaluate the results from July’18 to July’19. Seventy three glucometers used across the hospital were subjected to Internal Quality Control checks and Proficiency Testing performed once a month and every 3 months respectively. The results were reviewed and plotted on a Bland Altman Graph. Clarke Error Grid Analysis was done to evaluate the clinical significance of inaccuracies in the measurement of blood glucose concentration as per ISO 15197: 2013. Eight devices were identified as unacceptable by ISO standards and replaced subsequently. 96.83% and 3.17% of the values were in Zone A and B of Clarke Error Grid Analysis. The study complied with the standard which requires that 99% of the values fall within zones A and B. The review of the program after one year and its ability to identify defective glucometers has validated the efficacy of the model. The method used may be suggested as a prototype for quality management of glucometers in a clinical setting.

Keywords: Glucometer, Internal quality control (IQC), Point-of-care-testing (POCT), Proficiency testing (PT)

Introduction

Point-of-care-testing (POCT) has become increasingly important in the monitoring of blood glucose level and management of diabetes across the globe [1]. The blood glucometer is the most commonly used POCT equipment. Based on the results of the DCCT (Diabetes Control and Complications Trial), the American Diabetes Association (ADA) has recommended the use of SMBG for achieving therapeutic targets in patients with diabetes [2]. In addition, glucose monitors are extensively used in the hospital setting, both for inpatient and outpatient services. Accurate measurement of glucose by POCT devices is crucial because hyperglycemia, hypoglycemia and glucose variability may cause increased mortality in critical care units [3, 4]. Innovations in glucometer technology have enabled better accuracy in recent times. However, handling by different testing personnel, lack of awareness, non- adherence to manufacturer’s instructions, presence in varied locations across the hospital and absence of defined quality assurance practices influence the performance of glucometers and may produce deviant results in clinical settings [5]. There are several guidelines for determining the accuracy of glucometers, but the two most commonly used ones are the International Organization for Standardization (ISO 15,197:2013) and the FDA. The FDA requires that the devices meet or exceed the requirements of ISO15197. System accuracy criteria for verification of analytical performance capability was established in the first edition of International Standards, ISO 15,197:2003, which states that the acceptable system accuracy should be ± 0.83 mmol/L at glucose concentrations of < 4.2 mmol/L, and ± 20% at glucose concentrations of ≥ 4.2 mmol/L [6]. However, the second edition, ISO 15,197:2013, includes stricter criteria, stipulating that 95% of measured glucose values fall within either ± 0.83 mmol/L of the average measured values of the reference measurement procedure at glucose concentrations of < 5.55 mmol/L, or within ± 15% at glucose concentrations of ≥ 5.55 mmol/L [7]. However, the ADA recommends that these devices should produce measurements within 5% of reference values. It also states that because the accuracy of SMBG is instrument and user dependent, it is important for health care providers to evaluate the operator’s monitoring technique, both initially and at regular intervals thereafter [8]. Use of control solutions and comparison with reference methods on a regular basis helps ensure technical and clinical accuracy of test results [9].

Seventy three glucometers are in use across BLK Super Speciality Hospital, Delhi, India, including wards, intensive care units (Medical, Surgical, Neonatal, Neuro, Paediatric ICUs), out-patient departments, emergency and ambulances. In a tertiary care referral hospital like ours, with multiple super speciality departments and intensive care units, the reliability of glucometer readings and their impact on therapeutic interventions assume immense clinical significance.

Like several other countries, in India too, quality assurance of POCT devices is not routinely practiced at the user end. Most of the studies so far done in this area have either reviewed the accuracy of glucometers before being introduced for clinical use or compared glucose monitors from different manufacturers to evaluate their reliability in the clinical setting. Few studies have also compared the glucose value of glucometers to the reference method being used in the laboratory. However, to our knowledge, there is no study in the literature, which has reviewed the efficacy of a quality assurance program for glucometers in a high volume clinical setting for both accuracy and precision. The novelty of this study lies in the design of a program for glucometers and its statistical evaluation as per ISO 15,197:2013 guidelines, thereby facilitating early identification of faulty devices.

Methods

Seventy three glucometers are currently in use in BLK Super Speciality Hospital, Delhi, India, a high volume tertiary care referral hospital. The No Coding One Plus Blood Glucose Monitoring System (i-sens Biosensors India Pvt Ltd.) used is based on amperometric detection technique and glucose oxidase method. It requires 0.5 µL of capillary whole blood and gives a plasma equivalent of glucose value.

The present study was done to evaluate the performance of seventy three glucometers used in the hospital, over a period of 1 year. The study was conducted in the Department of Laboratory Diagnostic Services. The manufacturer recommended calibration and use of control solutions once a month. Calibration was performed once every month by the Application Specialist of the manufacturing company. All testing personnel were adequately trained in the handling and processing of glucometers. The instruments had acceptable sensitivity and specificity, and were not influenced by Maltose, Lactose, Galactose or Xylose. Based on ISO 15,197:2013 guidelines, the acceptable Hematocrit was between 15 and 65%. As per the defined Quality Assurance policy of the hospital on POCT, Internal Quality Control (IQC) was performed once a month and Proficiency Testing (PT) once in 3 months. The glucometers were brought to the laboratory in two batches on the designated dates. This was done to ensure smooth functioning of the wards during the process. For IQC, two levels of control solution (high and low) provided by the manufacturer were run. The glucometers reflecting values outside the manufacturer’s range were considered as outliers. For PT, a patient’s venous sample was collected in a Sodium Fluoride- Potassium Oxalate vial, and analyzed by the enzymatic reference method (Hexokinase) in the Cobas C- 6000 (Roche) fully automated modular system in the Clinical Biochemistry laboratory. Two levels of IQC material were run in the analyzer before processing the patient’s sample. The accuracy of the Hexokinase method was ensured by reviewing the BioRad EQAS results. The same sample was then analyzed in all the glucometers, which were brought to the laboratory from across the hospital. This ensured comparison of each glucometer value to the reference method, and also provided inter-instrument comparisons. Mean, SD and z-scores were calculated. A z-score of < 2 was considered to be acceptable, 2–3 as warning and > 3 as rejection. % variation of individual glucometers from the reference value was calculated and the ISO 15,197:2013 criteria were used as cut-offs. The distribution of IQC values for one year was evaluated and presented graphically. The PT values were plotted on Bland Altman Graphs. The ISO 15,197:2013 recommends the use of Clarke Error Grid Analysis to evaluate the clinical significance of inaccuracies in the measurement of blood glucose concentration. The values obtained in the glucometer were therefore plotted on a Clarke Error Grid to evaluate the clinical implication of difference in glucometer values from the reference method.

Statistical Analysis

Statistical analysis was done using STATA version 15. The reference limits for low and high level IQC material provided by the manufacturer were 41–71 mg/dl and 208–282 mg/dl respectively (to convert to millimoles per liter, multiply by 0.0555). The mean, Standard Deviation (SD) were calculated and Coefficient of Variance (CV %) derived as (SD/Mean) × 100. The values were plotted graphically to evaluate the distribution of individual control values of all glucometers over a period of 1 year. For Proficiency Testing the peer group mean was calculated for each patient’s sample run. Z-scores and percent variation were calculated for individual glucometers. Bland Altman graphs were plotted for values obtained during proficiency testing every 3 months for comparison with the reference method. Bland Altman graph is a difference plot, recommended for determining system accuracy because of minimal statistical assumption. It reflects the difference between two methods plotted on the y-axis, and the mean (or average) of the two methods on the x-axis. A Clarke Grid Analysis was done to evaluate the clinical significance of inaccuracies in the measurements of blood glucose concentration. It compared the bias between the glucometers and laboratory reference results. The data points were assigned to five zones (A–E) on the error grid. The results falling in zones A and B were considered to be clinically acceptable, meaning that the observed bias from laboratory results would not lead to treatment decisions that could put the patient at risk. As the bias increased (zones C, D, E) there was a greater risk of over or under treating the patient based on the glucometer results.

Results

The reference range for i-Sens glucose control solution was 41–71 and 208–282 mg/dL for low and high levels of control respectively. Mean glucose concentration of the low and high level control solutions calculated for all glucometers over a year along with their SD and CV % are presented in Table 1. The mean value for the low level control obtained was 59.98 mg/dl with a SD of 4.77. The mean value for the high level control was 236.77 mg/dl with a SD of 14.9. Figure 1a and b present the results of both levels of control in all glucometers over a period of 1 year.

Table 1.

Internal quality control for glucometers from july’18 to july’19

Level of IQC N Mean SD CV
Low control 680 59.98 4.773 7.96
Hogh control 680 236.77 14.993 6.33
Open in a new tab

Range Low level: 41–71 mg/dl, High level: 208–282 mg/dl

N Total number of values from glucometers; SD standard Deviation; CV Coefficicent of variance

Fig. 1.

Fig. 1

Open in a new tab

a Low level control (IQC) measured in glucometers from July’18 to July’19 (Range: 41–71 mg/dL) b High level control (IQC) measured in glucometers from July’18 to July’19 (Range: 208–282 mg/dl)

For PT, the patient samples evaluated and compared in the months of July’18, October’18, January’19, April’19 and July’19 are presented in Table 2. Figure 2a, b, c, d and e present the Bland Altman graphs for the corresponding months. In the graphs, the solid line indicates zero, the dotted line indicates mean of the difference (between glucometer and auto-analyzer value), the dash line (uppermost line) indicates mean + 1.96xSD, and the dash-dot line (lowermost line) indicates mean − 1.96xSD. To evaluate the performance of the analyzer on the day of sample run for PT, the internal quality control for glucose was reviewed in Cobas C- 6000 and found to be within acceptable limits (Multi control1: 88–118 mg/dL, Multi control 2: 207–279 mg/dL). The within-run precision and total precision for the Hexokinase method was less than 2.5% in each of these months. External Quality Assurance Scheme (BioRad EQAS) samples for glucose by Hexokinase had a z-score < 2 every month during the period of study. The plasma glucose levels of patient samples by Hexokinase method in July’18, October’18, January’19, April’19 and July’19 were 95, 90, 122, 74 and 248 mg/dL respectively. The hematocrit values of the patient samples were 45%, 52%, 44%, 49% and 53%. All the comparisons were within acceptable limits in July’18, April’19 and July’19. There were six outliers in October’18 and two in January’19. All eight were unacceptable according to ISO 15,197:2013.

Table 2.

Proficiency testing for glucometers from July’18 to July’19

July 2019 April 2019 January 2019 October 2018 July 2018
Mean 234.34 66.18 111.68 91.79 101.78
SD 18.748 4.099 9.930 9.344 7.078
CV% 8.00 6.19 8.89 10.18 6.95
Open in a new tab

SD Standard Deviation; CV Coefficient of Variance

Fig. 2.

Fig. 2

Open in a new tab

a Bland Altman graphs (July’18) b Bland Altman graphs (October’18) c Bland Altman graphs (January’19) d Bland Altman graphs (April’19) e Bland Altman graphs (July’19)

The Clarke Error Grid Analysis was used for accuracy quantification of POCT data against plasma glucose values obtained from Cobas C-6000. The grid has five zones:

  1. Zone A No effect on clinical action

  2. Zone B Altered clinical action with little or no effect on clinical outcome

  3. Zone C Altered clinical action, which is likely to affect clinical outcome

  4. Zone D Altered clinical action, which could have significant medical risk

  5. Zone E Altered clinical action, which could have dangerous consequences.

Figure 3 presents the Clarke Error Grid Analysis for the results of patient samples in each glucometer versus the value obtained by the Hexokinase method. There were a total of 252 measurements over a 1 year period. Out of 252 measurements 244 (96.83%) were in zone A and 8 (3.17%) were in Zone B. This implied that for 96.83% of the values (Zone A), the difference in value with the reference method was clinically insignificant. For 3.17% of the results (Zone B), the difference could alter clinical action, with or without any effect on patient outcome. This corroborated with the results of our PT program as well.

Fig. 3.

Fig. 3

Open in a new tab

Clarke error grid analysis for patient results obtained in each glucometer versus the Hexokinase (Reference) method

Discussion

Glucometers are the most commonly used POCT device and guide monitoring of glucose level in intensive care units (ICU) [10]. Although generally ‘assumed’ to be accurate, there are several reports of morbidity and mortality resulting from inaccuracies in glucometers. Inaccurate glucometer results causing errors in Insulin dosage calculation have critical consequences on patient care [11]. The best way to circumvent these discrepancies is the development of a quality assurance model, which reviews accuracy and precision of glucometers at regular intervals. POCT laboratory accreditation under ISO22870 in conjunction with ISO 15,189 has been implemented in many countries across Europe. Some countries have formulated national policies for the supervision of POCT activities [12]. In India however, there is no regulatory guideline for overseeing the performance of POCT equipments in hospital settings and outside. In this scenario, we attempted to design a sustainable program for evaluation of glucometers being used in our hospital. This program provides a robust model for the establishment of a quality protocol (both internal quality control and proficiency testing) for glucometers in a clinical setting.

In our study, most values of control material analyzed every month in the glucometers were within the defined reference range. Any outlier (as also reflected on the distribution graph for IQC) detected was re-evaluated on the following day to rule out any random error (user or instrument). If the problem persisted, the glucometer was sent to the Department of Biomedical Engineering for further checks. The comparison with the reference method in the central laboratory (for Proficiency Testing) every 3 months (as depicted in Bland Altman graphs) showed no outliers in July’18, April’19 and July’19. There were six outliers in October’18 and two in January’19. All eight were unacceptable according to ISO 15,197:2013 [7]. Comparative analysis was repeated for these glucometers on the following day to rule out a random error. This program helped identify defective glucometers, which could be sent to the Department of Biomedical Engineering for necessary action. The new glucometers which were issued were also verified in the Central Laboratory before being put to use. The Clarke Error Grid was used as per the recommendation of ISO 15,197:2013. 96.8% of the values were in Zone A, meaning that the difference in value between the glucometer and the Central Laboratory was clinically insignificant. 3.2% of the values were in Zone B indicating that the difference could alter clinical action, with or without any effect on clinical outcome. The values in Zone B indicated those glucometers which were deemed unfit in Proficiency Testing, and for which corrective action had been already initiated. Our study complies with the the standard which requires that 99% of the values fall within zones A and B of the consensus grid [7].

Wehmeier et al. [13] highlighted the importance of regular monitoring of the analytical performance of glucometers. This could involve daily internal quality control processes by control material provided by the manufacturer to check precision along with frequent comparisons with the traceable method. In a systematic review, the proportion of non-agreement between capillary blood glucose measured in glucometers and plasma glucose varied between 1.4 and 27.1% [14]. In another study, the day-to-day variation for glucometers was between 1.5 and 7.9% with control sera. Method comparison of Hexokinase or Glucose Oxidase with glucometers also showed a good agreement [15]. Ullal et al. [16] also compared glucometers in the indoor and outpatient departments to the laboratory reference method in a hospital setting. A pilot study in 2000 highlighted the need for an external quality control program to enable regular checks on glucometer performance [17]. Aykal et al. [18] emphasized on this need and proposed a model for monitoring glucometers in a high volume clinical setting. The internal quality checks performed in glucometers in our hospital increased the confidence of clinicians on its results. Considering the high cost of registering in External Quality Assurance programs for glucometers, the in-house proficiency testing program we designed also appeared to be useful. A major challenge to the implementation of this program was the initial lack of compliance of the testing personnel. However, regular training sessions and mock audits helped motivate them and overcome this hindrance.

The study has a few limitations. Using a wider range of blood glucose values for Proficiency Testing to cover the entire analytical range of glucometers 20–600 mg/dL shall be initiated shortly. Though not recommended by the manufacturer, the feasibility of running IQC material at more frequent intervals shall also be explored.

Conclusion

The principle challenge in achieving an ideal QC coverage for POCT equipment lies in avoiding an increase in the operating cost, or complexity of maintaining the device. The quality assurance scheme designed for glucometers in our hospital is cost effective and sustainable. The review of the program after one year as per ISO 15,197:2013 guidelines and its ability to identify erroneous glucometers also validated the efficacy of the model. The program used may be suggested as a prototype for optimum quality management of glucometers in high volume super speciality hospitals.

Acknowledgements

The authors would like to thank Ms Rosamma (Nursing Superintendent), Ms Anumol Joseph (Deputy Nursing Superintendent) and Ms Sheeba Jose (Nursing Supervisor), BLK Super Speciality Hospital, for facilitating the implementation of quality practices in POCT in the hospital.

Author contributions

All authors whose names appear on the submission made substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data.

Funding

No funds, grants, or other support was received.

Declarations

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethics approval

Ethical approval was waived by the Institutional Ethics Committee of BLK super Speciality Hospital, Delhi, in view of the retrospective nature of the study and all the procedures being performed were part of the routine care.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Freckmann YG, Schmid C, Pleus S, Baumstark A, Link B, Stolberg E, et al. System accuracy evaluation of systems for point-of-care testing of blood glucose: a comparison of a patient-use system with six professional-use systems. Clin Chem Diabetes Med. 2014;52:1079–1086. doi: 10.1515/cclm-2013-0976. [DOI] [PubMed] [Google Scholar]
  • 2.American Diabetes Association Tests of glycemia in diabetes. Diabetes Care. 2003;26(S1):S106–S108. doi: 10.2337/diacare.26.2007.S106. [DOI] [PubMed] [Google Scholar]
  • 3.Krinsley JS, Egi M, Kiss A, Devendra AN, Schuetz P, Maurer PM, et al. Diabetic status and the relation of the three domains of glycemic control to mortality in critically ill patients: an international multicenter cohort study. Crit Care. 2013;17(2):R37. doi: 10.1186/cc12547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Egi M, Bellomo R, Stachowski E, French CJ, Hart G. Variability of blood glucose concentration and short-term mortality in critically ill patients. Anesthesiology. 2006;105:244–252. doi: 10.1097/00000542-200608000-00006. [DOI] [PubMed] [Google Scholar]
  • 5.Sengupta S, Handoo A. Optimizing quality practices in POCT: a journey just begun! Point Care J Near-Patient Test Technol. 2020;19(1):4–11. doi: 10.1097/POC.0000000000000198,IssnPrint:1533-029X. [DOI] [Google Scholar]
  • 6.In vitro Diagnostic Test Systems. Requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus. TS EN ISO 15197:2003.
  • 7.In vitro Diagnostic Test Systems. Requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus. TS EN ISO 15197:2013.
  • 8.American Diabetes Association Self-monitoring of blood glucose (consensus statement) Diabetes Care. 1994;17:81–86. doi: 10.2337/diacare.17.1.81. [DOI] [PubMed] [Google Scholar]
  • 9.Tonyushkina K, Nichols JH. Glucose meters: a review of technical challenges to obtaining accurate results. J Diabetes Sci Technol. 2009;3(4):971–980. doi: 10.1177/193229680900300446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Estock JL, Pham IT, Curinga HK, Sprague BJ, Boudreaux-KellyMY AJ, et al. Reducing treatment errors through point-of-care glucometer configuration. Joint Comm J Qual Patient Saf. 2018;11:683–694. doi: 10.1016/j.jcjq.2018.03.014. [DOI] [PubMed] [Google Scholar]
  • 11.Finfer S, Wernerman J, Preiser JC, Cass T, Desaive T, Hovorka R, et al. Clinical review: consensus recommendations on measurement of blood glucose and reporting glycemic control in critically ill adults. Crit Care. 2013;17(3):229. doi: 10.1186/cc12537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Policies, procedures and guidelines for point-ofcare testing. RCPA Quality Assurance Programs Pty Ltd. Available at: http://www.aacb.asn.au/documents/item/635.
  • 13.Wehmeier M, Arndt BT, Schumann G, Kulpmann WR. Evaluation and quality assessment of glucose concentration measurement in blood by point-of-care testing devices. Clin Chem Lab Med. 2006;44:888–893. doi: 10.1515/CCLM.2006.141. [DOI] [PubMed] [Google Scholar]
  • 14.Inoue S, Egi M, Kotani J, Morita K. Accuracy of blood-glucose measurements using glucose meters and arterial blood gas analyzers in critically ill adult patients: systematic review. Crit Care. 2013;17(2):R48. doi: 10.1186/cc12567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Rosak C, Lotz N, Bachmann W, Hürter P, Kerner W, Koschinsky T, et al. Clinical testing of a new blood glucose measuring system: a cooperative study at 8 centers. Dtsch Med Wochenschr. 1987;112(12):466–470. doi: 10.1055/s-2008-1068077. [DOI] [PubMed] [Google Scholar]
  • 16.Ullal A, Parmar GM, Chauhan PH. Comparison of glucometers used in hospitals and in outpatient settings with the laboratory reference method in a tertiary care hospital in Mumbai. Indian J Endocrinol Metab. 2013;17(Suppl 3):S688–S693. doi: 10.4103/2230-8210.123569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Suhasini D. Glucometers-need for external quality control. IJCB. 2000;15(2):128–133. doi: 10.1007/BF02883741. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Aykal G, Yegin A, Tekeli O, Yilmaz N. A model for managing and monitoring the quality of glucometers used in a high-volume clinical setting. Biochem Med. 2016;26(2):202–209. doi: 10.11613/BM.2016.022. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Indian Journal of Clinical Biochemistry are provided here courtesy of Springer

RESOURCES