Abstract
Background:
System accuracy of current blood glucose monitors (BGMs) in the market has already been evaluated extensively, yet mostly focused on European and North American manufacturers. Data on BGMs manufactured in the Asia-Pacific region remain to be established. In this study, we sought to assess the accuracy performance of 19 BGMs manufactured in the Asia-pacific region.
Methods:
A total of 19 BGMs were obtained from local pharmacies in China. The study was conducted at three hospitals located in the Asia-Pacific region. Measurement results of each system were compared with results of the reference instrument (YSI 2300 PLUS Glucose Analyzer), and accuracy evaluation was performed in accordance to the ISO 15197:2003 and updated 2015 guidelines. Radar plots, which is a new method, are described herein to visualize the analytical performance of the 19 BGMs evaluated. Consensus error grid is a tool for evaluating the clinical significance of the results.
Results:
The 19 BGMs resulted in a satisfaction rate between 83.5% and 100.0% within ISO 15197:2003 error limits, and between 71.3% and 100.0% within EN ISO 15197:2015 (ISO 15197:2013) error limits.
Conclusions:
Of the 19 BGMs evaluated, 12 met the minimal accuracy requirement of the ISO 15197:2003 standard, whereas only 4 met the tighter EN ISO 15197:2015 (ISO 15197:2013) requirements. Accuracy evaluation of BGMs should be performed regularly to maximize patient safety.
Keywords: accuracy, Asia-Pacific region, blood glucose monitoring system, EN ISO 15197:2015 (ISO 15197:2013)
Diabetes mellitus is a chronic illness affecting 387 million people worldwide.1 Thus, tight glycemic control is crucial in delaying the onset or progression of diabetes-associated complications.2 With the advancement of technology nowadays, blood glucose can be easily monitored and managed via self-monitoring of blood glucose, utilizing a piece of electrochemical biosensor.3-7 Blood glucose monitors (BGMs) serve as an effective tool for caregivers and patients to make appropriate clinical decisions. In 1962, Clark and Lyons proposed the concept of enzyme-based electrode systems for blood glucose (BG) quantification.8 Ever since the discovery of such system, development of glucose biosensors became a common goal worldwide for patients with diabetes to manage. In 2003, the International Organization for Standardization (ISO) published the 1st set of guidelines, ISO 15197, in regulating BGMs and the booming market.9 The ISO 15197:2003 guideline has been widely used for evaluating the accuracy of BGMs, in which at least 95% of meter readings should fall within ±15 mg/dl or ±20% of the reference at glucose concentrations <75 mg/dl and ≥75 mg/dl, respectively.10 Recently, a more rigorous EN ISO 15197:2015 (ISO 15197:2013) has been implemented (at least 95% of meter readings should fall within ±15 mg/dl or ±15% of the reference at glucose concentrations <100 mg/dl and ≥100 mg/dl, respectively).11 The tighter regulations warrant BGM manufacturers to improve on accuracy performance. To assure quality and provide valuable references for choosing reliable BG meters in the current market, accuracy evaluation of BGMs has been performed extensively by both manufactures and third-party institutions (ie, Institute for Diabetes Technology).12-14 In the past 5 years, there were 21 and 15 large evaluations on the performance of BGMs manufactured in the regions of Europe and North America, respectively,12-20 whereas only 5 studies mentioned a few BGMs manufactured in the Asia-Pacific region.12-14,21,22 Not one single large data was published on BGMs manufactured in different regions of Asia. It is obvious that accuracy reports on BGMs manufactured in the Asia-Pacific region, especially China, are lacking thus remains to be established.
The study sought to evaluate 19 BGM manufactured in the Asia-Pacific region. Accuracy performances were analyzed in accordance to the ISO 15197:2003 and updated 2015 guidelines.
Materials and Methods
The study was conducted at Chung Shan Medical University Hospital, Min-Sheng General Hospital and Shanghai General Hospital. Signed Informed consent forms were obtained from all participants at Chung Shan Medical University Hospital and Min-Sheng General Hospital, Fingertip capillary blood draws are routinely tested for patients with diabetes, while residual blood was measured without additional skin puncturing at Shanghai General Hospital. The study was completed in April 2014.
Study Participants
A total of 400 samples were collected for each of the systems tested. Participants were adults (age: 21-86 years old; gender: 30.2% of males, 69.8% of females) with and without type 1 and 2 diabetes. Exclusion criteria were (1) pregnancy for female subjects; (2) patients suffering from severe complications in recently 6 months, such as apoplexy, myocardial infarction, serious trauma, and major surgery; (3) the health care professional evaluates the patient’s health condition as unsuitable for this study.
BGM Devices
Table 1 illustrates the details of the 19 BGMs tested, manufactured in the Asia-Pacific region. All systems were obtained from local pharmacies with proper storage and maintenance. Quality checks were performed in accordance to manufacture’s instruction prior to experimental procedures to ensure optimal functioning of the device.
Table 1.
Nineteen BGMs Manufactured in the Asia-Pacific Region.
| BGM model | Manufacturer | Test strip enzyme | Strip lot number | Expiry date | Accuracy criteria version |
|---|---|---|---|---|---|
| BeneCheck PLUS JET | General Life Biotechnology, Taiwan | GOx | G1300563 | 04/2015 | ISO 15197:2003 |
| CareSens II | i-SENS, South Korea | GOx | KY06CP18B | 06/2015 | ISO 15197:2003 |
| Fora Comfort Pro GD40b | TaiDoc Technology, Taiwan | GDH | TD13D301-BGE | 01/2015 | EN ISO 15197:2015 |
| Fora G31 | TaiDoc Technology, Taiwan | GOx | TD13E129-B0C | 05/2015 | ISO 15197:2003 |
| Glucocard G+meter GT-1820 | ARKRAY Factory, Japan | GDH | 3B5F51 | 08/2014 | ISO 15197:2003 |
| GlucoSure ADVANCE | Apex Biotechnology, Taiwan | GDH | M1S1307001 | 07/2015 | ISO 15197:2003 |
| GlucoSure Plus | Apex Biotechnology, Taiwan | GOx | BS431D | 04/2015 | ISO 15197:2003 |
| Medisafe Mini | Terumo, Japan | GOx | 120726M | 06/2014 | ISO 15197:2003 |
| OKmeter Match | OK Biotech, Taiwan | GOx | S130516-1 | 05/2015 | ISO 15197:2003 |
| Omron HEA-230 | DELBio, Taiwan | GOx | 1212A1911 | 04/2014 | ISO 15197:2003 |
| On Call Advanced | ACON Biotech, China | GDH | 201301060 | 12/2014 | ISO 15197:2003 |
| On Call Plus | ACON Biotech, China | GOx | 201208003 | 06/2014 | ISO 15197:2003 |
| Rightest GM300 | Bionime Corporation, Taiwan | GOx | 1135301 | 04/2015 | EN ISO 15197:2015 |
| Rightest GM550 | Bionime Corporation, Taiwan | GOx | 113610K | 05/2015 | EN ISO 15197:2015 |
| Rightest GM700 | Bionime Corporation, Taiwan | GDH | 1134295 | 03/2015 | EN ISO 15197:2015 |
| SAUNNO SXT Glucometer | Sinocare, China | GOx | 2317EZ | 01/2016 | ISO 15197:2003 |
| Super Glucocard II GT-1640 | ARKRAY Factory, Japan | GOx | 3D5B4S | 10/2014 | ISO 15197:2003 |
| Yicheng 5D-1 | Beijing Yicheng Bioelectronics Technology, China | GOx | 20130808 | 02/2015 | ISO 15197:2003 |
| Yuwell Accusure 710 | Jiangsu Yuyue medical equipment & supply, China | GOx | 130307 | 02/2015 | ISO 15197:2003 |
Reference method, strip lot number, and expiry date are listed according to the manufacturer’s labeling.
Reference Measurements
YSI 2300 PLUS Glucose Analyzer (YSI Incorporated, Yellow Springs, OH) was used as reference instrument to minimize variance associated with different testing locations. Quality control for YSI 2300 PLUS was performed in accordance to manufacturer’s procedures for every 5 measurements.
Hematocrit Measurements
Puncturing the side of a fingertip, a Micro Hematocrit capillary tube (Kimble Chase Life Science, Heparinized, red coded) was placed near the incision site and the blood was allowed to flow into the tube until it was two-third to three-fourths full. The bottom of the tube is sealed with clay. The tube is then placed in a hematocrit centrifuge (Hsiang Tai Industry) and spun for 3 minutes at 12 000 RPM. Using a ruler to measure the length of the column of the packed red cells and divided by the length of the column of the whole blood. To obtain the hematocrit, multiply the number by 100%.23
Experimental Procedures
All experimental procedures were conducted by trained technicians in accordance to study protocol. All participants were asked to thoroughly wash and dry their hands prior to the experimental procedures.
The experimental procedures were summarized as follows: (1) fingertip capillary blood samples were obtained from participants via skin puncturing; (2) BG concentration was measured by reference instrument prior to first BGM measurement; (3) BG concentration was measured twice for each BGM device; (4) BG concentration was measured again by reference instrument after last BGM measurement. Procedures (1) to (4) were repeated until sufficient measurements for each device were acquired.
Distribution of BG concentration should follow the criteria described in EN ISO 15197:2015 (ISO 15197:2013) for system accuracy evaluation: 5% of samples should be ≤50 mg/dl, 15% > 50 and ≤80 mg/dl, 20% > 80 and ≤120 mg/dl, 30% > 120 and ≤200 mg/dl, 15% > 200 and ≤300 mg/dl, 10% > 300 and ≤400 mg/dl, 5% > 400 mg/dl. If the study population does not provide a sufficient number of fresh capillary blood samples with very low and very high glucose concentrations, the capillary blood samples are modified by incubation to lower the glucose concentration or supplementation with glucose (1 molarity stock glucose solution) is done to raise the glucose concentration, in which the glucose concentration may be substituted to achieve the required distribution. Several samples were prepared to reflect extremely high (>400 mg/dl) and low (≤50 mg/dl) BG concentration. However, a number of BG specimens distributed in a defined concentration range was slightly different (1% deviation) from the ISO description due to difficulty in obtaining sufficient specimens,
Statistical Analysis
Measurement results were excluded in the following circumstances: (1) hematocrit values were outside the manufacturers’ specifications; (2) user or technical errors were observed; (3) patients had taken medicine with potential interference to BGM measurement but failed to inform technicians during experimental period.
BGM readings were compared to corresponding reference values immediately before and after each measurement. Radar plots, a new method described in recent studies,24,25 were used to visualize analytical performance. Differences between BGMs and reference instrument measurements were plotted using polar. While innermost circle represented BGM readings within ±5 mg/dl and ±5% of reference measurement, outermost boundaries were defined as BGM readings within ±20 mg/dl and ±20%. In accordance to ISO 15197:2003, a number of measurements should fall within ±15, ±10, and ±5 mg/dl of reference method at BG concentrations <75 mg/dl and within ±20%, ±15%, ±10%, and ±5% at BG concentrations ≥75 mg/dl. The updated EN ISO 15197:2015 (ISO 15197:2013) criteria were also applied, in which BGM readings should fall within ±15, ±10, and ±5 mg/dl of reference method at BG concentrations <100 mg/dl and within ±15%, ±10%, and ±5% at BG concentrations ≥100 mg/dl.
Consensus error grid is used for evaluating the clinical significance of the result between the glucose measurement by BGMs and the capillary blood reference measurements. The graphics can be divided into five zones, glucose falling in zones A and B, which means little or no effect on clinical outcome, whereas results that fall in zone C to E mean increasing risk of an adverse outcome.26
To evaluate accuracy of BGMs at different BG concentrations, four BG intervals were defined as described in the ISO 15197:2003 and 2015 guidelines: <70 mg/dl, 70-109 mg/dl, 110-126 mg/dl, and ≥200 mg/dl. BG intervals were defined by the World Health Organization (WHO) and American Diabetes Association (ADA) as the diagnostic criteria for diabetes mellitus,27,28 in which the differentiation of BG values referred to the WHO and ADA criteria for diagnosing diabetes mellitus.27,28 BG concentration <70 mg/dl was defined as hypoglycemia, 70-109 mg/dl was regarded as low risks region or euglycemia, 100-126 mg/dl was classified as impaired fasting glucose (IFG), and ≥200 mg/dl was defined as diabetes.
Results
The hematocrit range in the study is from 22.7% to 50.5%. Accuracy performance of 19 BGMs evaluated was visualized using radar plots (Figure 1). Percentage of BGM readings within specific accuracy limits of the ISO 15197 guidelines was illustrated in Table 2. Evaluation of system accuracy in accordance to ISO 15197: 2003 and 2015 criteria resulted in an 83.5–100.0% and 71.3–100.0% satisfaction rate, respectively (Table 2). Among the 19 BGMs evaluated, 12 BGMs (CareSens II, Fora Comfort Pro GD40b, Fora G31, Glucocard G+meter GT-1820, GlucoSure Plus, OKmeter Match, Omron HEA-230, On Call Advanced, On Call Plus, Rightest GM300, Rightest GM550, and Rightest GM700) met the minimum accuracy requirement of the ISO 15197:2003 standard, whereas only 4 BGMs (Fora Comfort Pro GD40b, Rightest GM300, Rightest GM550, and Rightest GM700) met the minimum accuracy requirement of the EN ISO 15197:2015 (ISO 15197:2013) standard (Figure 1 and Table 2). When accuracy limits were tightened to ±10 mg/dl at BG concentration <100 mg/dl and ±10% at BG concentration ≥100 mg/dl, only 1 BGM (Rightest GM700) satisfied this requirement.
Figure 1.
Radar plots of 19 BGMs. Blue dots indicate the BG readings within ±15 mg/dl or ±15% of the reference method, and the red dots indicate the readings without ±15 mg/dl or ±15%. The solid straight lines represent the BG concentrations by the reference method while the curved lines represent the accuracy limits, that is, ±5 mg/dl and ±5%, ±10 mg/dl and ±10%, etc. The upper half of the target plot means positive deviations and biases, and the lower half of the plot means negative deviations and biases. The thicker lines that link the radar center and the 100 mg/dl represent the threshold described in EN ISO 15197:2015 (ISO 15197:2013). The YSI 2300 Plus Glucose was the reference instrument.
Table 2.
System Accuracy of 19 BGMs.
| BGM model | ISO15197:2003 |
EN ISO 15197:2015 |
|||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Within accuracy limits (±15 mg/dl and ±20%) | BG concentration |
Within accuracy limits (±15 mg/dl and ±15%) | BG concentration |
||||||||||||
| <75 mg/dl |
≥75 mg/dl |
<100 mg/dl |
≥100 mg/dl |
||||||||||||
| ±15 mg/dl | ±10 mg/dl | ±5 mg/dl | ±20% | ±15% | ±10% | ±5% | ±15 mg/dl | ±10 mg/dl | ±5 mg/dl | ±15% | ±10% | ±5% | |||
| BeneCheck PLUS JET | 93.5 (374/400) |
83.8 |
50.0 | 27.9 | 95.5 | 89.8 | 70.8 | 37.3 | 89.5 (358/400) |
85.6 |
54.2 | 27.1 | 91.1 | 72.7 | 39.7 |
| CareSens II | 98.3 (393/400) | 94.1 |
76.5 |
39.7 | 99.1 | 87.7 | 67.8 | 37.0 | 89.8 (359/400) | 92.0 |
69.6 |
39.3 | 88.9 | 69.4 | 36.8 |
| Fora Comfort Pro GD40b | 98.0 (392/400) | 100.0 | 93.6 | 65.4 | 97.5 | 93.5 | 71.1 | 38.2 | 95.3 (381/400) | 99.2 |
88.1 |
59.5 |
93.4 |
70.8 |
36.9 |
| Fora G31 | 98.3 (393/400) | 97.3 | 77.0 | 40.5 | 98.5 | 90.5 | 74.5 | 37.1 | 92.5 (370/400) | 94.2 | 75.0 |
43.3 |
91.8 | 76.1 | 36.4 |
| Glucocard G+meter GT-1820 | 95.5 (382/400) | 98.6 | 80.0 |
38.6 |
94.8 | 74.2 | 42.4 | 19.7 | 79.0 (316/400) | 94.0 | 73.3 | 31.9 | 72.9 | 41.2 | 19.4 |
| GlucoSure ADVANCE | 94.8 (379/400) | 100.0 | 74.3 | 33.8 | 93.6 | 84.4 | 71.2 | 42.6 | 88.8 (355/400) | 96.8 | 72.6 |
35.5 |
85.1 | 73.2 | 43.5 |
| GlucoSure Plus | 98.3 (393/400) | 98.6 | 77.8 | 48.6 | 98.2 | 89.3 | 72.3 | 39.0 | 92.3 (369/400) | 98.4 | 79.4 | 44.4 | 89.4 | 72.6 | 39.4 |
| Medisafe Mini | 84.8 (339/400) | 62.1 | 40.9 | 4.5 | 89.2 | 73.1 | 51.8 | 28.1 | 72.0 (288/400) | 62.5 | 40.2 | 13.4 | 75.7 | 53.8 | 28.8 |
| OKmeter Match | 98.0 (392/400) | 97.4 | 89.5 | 47.4 | 98.1 | 88.9 | 70.7 | 38.9 | 91.8 (367/400) | 93.7 | 81.0 |
48.4 |
90.9 | 71.5 |
37.6 |
| Omron HEA-230 | 97.3 (389/400) | 94.3 | 67.1 | 34.3 | 97.9 | 90.9 | 79.1 |
46.4 |
92.0 (368/400) | 91.8 | 72.1 | 43.4 | 92.1 | 79.9 |
44.6 |
| On Call Advanced | 99.5 (398/400) | 100.0 |
90.8 |
57.9 |
99.4 | 85.8 | 68.2 | 36.7 | 90.8 (363/400 | 96.7 |
80.8 |
51.7 |
88.2 | 69.6 | 37.1 |
| On Call Plus | 96.0 (384/400) | 93.2 |
74.3 |
51.4 | 96.6 | 87.4 | 67.2 | 40.5 | 88.5 (354/400) | 88.3 |
75.0 |
49.2 | 88.6 | 68.9 | 40.0 |
| Rightest GM300 | 100.0 (400/400) | 100.0 | 94.6 | 67.6 | 100.0 | 96.6 | 84.0 | 43.9 | 97.8 (391/400) | 99.2 | 89.3 | 59.8 | 97.1 | 84.5 | 43.9 |
| Rightest GM550 | 99.8 (399/400) | 100.0 | 95.6 | 44.1 | 99.7 | 97.3 | 81.3 | 48.2 | 98.3 (393/400) | 100.0 | 95.1 | 54.9 | 97.5 | 80.2 | 45.0 |
| Rightest GM700 | 100.0 (400/400) | 100.0 | 100.0 | 71.6 | 100.0 | 100.0 | 95.7 | 72.4 | 100.0 (400/400) | 100.0 | 100.0 | 74.2 | 100.0 |
94.9 |
71.7 |
| SAUNNO SXT Glucometer | 91.8 (367/400) | 87.5 |
70.8 |
27.8 | 92.7 | 84.8 | 70.1 | 39.0 | 86.0 (344/400) | 90.0 |
71.7 |
31.7 | 84.3 | 70.0 | 39.6 |
| Super Glucocard II GT-1640 | 93.3 (373/400) |
86.7 |
31.7 |
10.0 | 94.4 | 76.2 | 56.2 | 29.7 | 81.0 (324/400) | 80.9 |
28.2 |
7.3 | 81.0 | 62.4 | 34.5 |
| Yicheng 5D-1 | 84.5 (338/400) | 92.1 | 73.7 | 38.2 | 82.7 | 66.7 | 44.1 | 22.8 | 73.0 (292/400) | 81.0 | 53.4 | 27.6 | 69.7 |
48.2 |
25.0 |
| Yuwell Accusure 710 | 83.5 (334/400) | 85.7 | 58.6 | 15.7 | 83.0 | 67.6 | 47.6 |
23.9 |
71.3 (285/400) | 67.3 | 42.7 | 10.9 | 72.8 | 52.1 |
26.9 |
Values are percentages. Numbers are greater than 95% are in bold.
According to the EN ISO 15197:2015 (ISO 15197:2013) criteria, 99% of individual glucose measured values should fall within zones A and B of the consensus error grid. Among the 19 BGMs, 100% of individual glucose measured values fall within zones A and B, meaning all BGMs met the requirements of EN ISO 15197:2015 (ISO 15197:2013)(Figure 2).
Figure 2.
Consensus error grid of 19 BGMs. The horizontal lines represent the BG concentrations by the YSI analyzer while the vertical lines represent the glucose concentration measured by different glucose meters. The consensus error grid is divided into five zone signifying the estimated risk to the patient if a result falls in a given zone. Zone A: no effect on clinical action; Zone B: altered clinical action—little or no effect on clinical outcome; Zone C: altered clinical action—likely to affect clinical outcome; Zone D: altered clinical action—could have significant medical risk; Zone E: altered clinical action—could have dangerous consequences. (a) BeneCheck PLUS JET; (b) CareSens II; (c) Fora Comfort Pro GD40b; (d) Fora G31; (e) Glucocard G+meter GT-1820; (f) GlucoSure ADVANCE; (g) GlucoSure Plus; (h) Medisafe Mini; (i) OKmeter Match; (j) Omron HEA-230; (k) On Call Advanced; (l) On Call Plus; (m) Rightest GM300; (n) Rightest GM550; (o) Rightest GM700; (p) SAUNNO SXT Glucometer; (q) Super Glucocard II GT-1640; (r) Yicheng 5D-1; (s) Yuwell Accusure 710.
Accuracy evaluation of 19 BGMs within four BG intervals was shown in Table 3. In accordance to the ISO 15197:2003 criteria, 10, 7, 13, and 13 devices met the requirement at BG concentrations <70 mg/dl, 70-109 mg/dl, 110-126 mg/dl, and ≥200 mg/dl, respectively (Table 3A).16 devices evaluated met the requirement in at least one BG interval, whereas only 6 devices complied with the criteria throughout all four BG intervals (Table 3A). In accordance to the EN ISO 15197:2015 (ISO 15197:2013) criteria, 10, 6, 4, and 4 devices met the requirement at BG concentrations <70 mg/dl, 70-109 mg/dl, 110-126 mg/dl, and ≥200 mg/dl, respectively (Table 3B). 12 evaluated devices met the requirement in at least one BG interval, whereas only 3 devices met the requirements throughout all four BG intervals (Table 3B).
Table 3.
Accuracy Evaluation of the 19 BGMs in Four BG Intervals.
| (A) BGM models | Hypoglycemia |
Euglycemia |
IFGa |
Diabetes |
|---|---|---|---|---|
| <70 mg/dl | 70-109 mg/dl | 110-126 mg/dl | 2-hour glucoseb ≥200 mg/dl | |
| BeneCheck PLUS JET | ✓ | |||
| CareSens II | ✓ | ✓ | ||
| Fora Comfort Pro GD40b | ✓ | ✓ | ✓ | ✓ |
| Fora G31 | ✓ | ✓ | ✓ | |
| Glucocard G+meter GT-1820 | ✓ | |||
| GlucoSure ADVANCE | ✓ | ✓ | ||
| GlucoSure Plus | ✓ | ✓ | ✓ | ✓ |
| Medisafe Mini | ||||
| OKmeter Match | ✓ | ✓ | ✓ | |
| Omron HEA-230 | ✓ | ✓ | ||
| On Call Advanced | ✓ | ✓ | ✓ | ✓ |
| On Call Plus | ✓ | ✓ | ||
| Rightest GM300 | ✓ | ✓ | ✓ | ✓ |
| Rightest GM550 | ✓ | ✓ | ✓ | ✓ |
| Rightest GM700 | ✓ | ✓ | ✓ | ✓ |
| SAUNNO SXT Glucometer | ✓ | |||
| Super Glucocard II GT-1640 | ✓ | ✓ | ||
| Yicheng 5D-1 | ||||
| Yuwell Accusure 710 | ||||
| (B) BGM models | Hypoglycemia | Euglycemia | IFGa | Diabetes |
| <70 mg/dl | 70-109 mg/dl | 110-126 mg/dl | 2-hour glucoseb ≥200 mg/dl | |
| BeneCheck PLUS JET | ||||
| CareSens II | ✓ | |||
| Fora Comfort Pro GD40b | ✓ | ✓ | ||
| Fora G31 | ✓ | |||
| Glucocard G+meter GT-1820 | ✓ | |||
| GlucoSure ADVANCE | ✓ | ✓ | ||
| GlucoSure Plus | ✓ | ✓ | ||
| Medisafe Mini | ||||
| OKmeter Match | ✓ | |||
| Omron HEA-230 | ||||
| On Call Advanced | ✓ | |||
| On Call Plus | ||||
| Rightest GM300 | ✓ | ✓ | ✓ | ✓ |
| Rightest GM550 | ✓ | ✓ | ✓ | ✓ |
| Rightest GM700 | ✓ | ✓ | ✓ | ✓ |
| SAUNNO SXT Glucometer | ✓ | |||
| Super Glucocard II GT-1640 | ||||
| Yicheng 5D-1 | ||||
| Yuwell Accusure 710 |
If a BGM fulfilled the accuracy requirement in a defined interval, the interval is ticked. (A) Evaluation based on the ISO 15197:2003 criteria (±15 mg/dl and ±20%). (B) Evaluation based on the EN ISO 15197:2015 (ISO 15197::2013) criteria (±15 mg/dl and ±15%).
Impaired fasting hyperglycemia.
Venous plasma glucose 2 hours after ingestion of 75 g oral glucose load.
Discussion
To address the current knowledge gap, we aimed to investigate the system accuracy of 19 BGMs manufactured in the Asia-Pacific region in comparison to ISO guidelines. Radar plots analysis, a method recently introduced by Bayer Health Care,25 was used to visualize analytical performance of BGMs. Such method facilitated visualization of accuracy and precision in a single graphic, simplifying the presentation of meter system performance data for effective clinical decision making. Radar plots also allow visual comparison of multiple meter systems in a single analysis, providing useful insights on choosing a reliable system. As shown in Figure 1, readings of 4 BGMs assessed by the EN ISO 15197:2015 (ISO 15197:2013) criteria concentrated around the radar center, indicating a higher precision and accuracy performance than the rest of the BGMs. Consensus error grid is another tool for evaluating the accuracy of BGMs, and based on the result, 100% of individual glucose measured values fall within zones A and B, while only Rightest GM700 has 100% of all readings falling within zones A (Figure 2).
Accuracy of BGMs is crucial for glycemic control in patients with diabetes, especially for those with insulin dependency. Significant errors in insulin dosage and undetected hypoglycemic events have been reported from glucometers with a large error range.29,30 Hasslacher et al investigated accuracy performance of BGMs within a narrowly defined BG range.31 They demonstrated that BGMs satisfying a wider range of accuracy requirement reflected a better overall performance. Similar outcomes were shown in our study (Table 2), except with a slight difference in BG intervals defined. As shown in Table 3, BG intervals defined herein were in accordance to risk factors for diabetes mellitus suggested by WHO and ADA, in which fasting plasma glucose between 70 and 109 mg/dl is regarded as euglycemia, while 110 to 126 mg/dl is considered as intermediate hyperglycemia or so-called prediabetes.27 Oral glucose tolerance test is usually given to prediabetic patients to identify those with impaired glucose tolerance (IGT) or impaired fasting glucose (IFG). A prediabetic patient can be IGT if the patient’s 2-hour plasma glucose is lower than 200 mg/dl, or can be IFG if the patient’s 2-hour plasma glucose is lower than 140 mg/dl. Therefore, BGMs should provide accurate measurement for tight glycemic control within these specific BG intervals. While 12 of the 19 devices tested met the minimal accuracy requirements of ISO 15197:2003, only 6 devices (Fora Comfort Pro GD40b, GlucoSure Plus, On Call Advanced, Rightest GM300, Right GM550, and Right GM700) satisfied the accuracy requirement at all BG intervals (Table 3A). When the tighter 2015 ISO standard was applied, 4 devices met the minimal accuracy requirements yet only 3 devices (Rightest GM300, Right GM550, and Right GM700) satisfied accuracy requirements at all BG intervals (Table 3B). It is interesting to note that For a Comfort Pro GD40b, which met the 2015 ISO standard, failed to perform well at higher BG concentrations (Table 3B). The poor accuracy performance reported was believed to be attributed to test strips being manufactured to meet the ISO 15197:2003 criteria. Therefore, it remains a major task for BGM manufacturers to improve and optimize their accuracy performance to meet the newly implemented 2015 ISO standard.
According to EN ISO 15197:2015 (ISO 15197:2013), the minimum study design to evaluate system accuracy should make use of 3 different strip lots. From our study, only one strip lot was used with each BGM due to difficulty in obtaining 3 different strip lots for19 BGMs. For further evaluation of the accuracy, we will choose some of the 19 BGMs which met the minimum acceptable accuracy to full compliance with the EN ISO 15197:2015 (ISO 15197:2013).
Conclusions
In this study, we established data on system accuracy of 19 BGMs manufactured in the Asia-Pacific region. While 12 of the 19 BGMs evaluated satisfied accuracy criteria described in ISO 15197:2003, only 4 BGMs met the new EN ISO 15197:2015 (ISO 15197:2013) requirements. As the EN ISO 15197:2015 (ISO 15197:2013) guidelines will soon be widely implemented, it remains a great challenge for BGM manufacturers to meet the tighter criteria yet demonstrate consistent performance throughout all BG intervals. It is particularly important to evaluate performance of BGMs on a regular basis to provide accurate and safe glucometer for routine users.
Acknowledgments
Bionime Corporation is appreciated for their technical assistance.
Footnotes
Abbreviations: BG, blood glucose; BGMS, blood glucose monitoring system; IGT, impaired glucose tolerance; IFG, impaired fasting glucose; ISO, International Organization for Standardization.
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was sponsored by Bionime Corporation.
References
- 1. International Diabetes Federation. IDF Diabetes Atlas. 6th ed. Available at: http://www.idf.org/sites/default/files/Atlas-poster-2014_EN.pdf. Accessed January 26, 2015.
- 2. Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329(14):977-986. [DOI] [PubMed] [Google Scholar]
- 3. Consensus statement on self-monitoring of blood glucose. Diabetes Care. 1987;10(1):95-99. [PubMed] [Google Scholar]
- 4. Blonde L, Karter AJ. Current evidence regarding the value of self-monitored blood glucose testing. Am J Med. 2005;118(suppl 9A):20S-26S. [DOI] [PubMed] [Google Scholar]
- 5. Sacks DB, Arnold M, Bakris GL, et al. Position statement executive summary: guidelines and recommendations for laboratory analysis in the diagnosis and management of diabetes mellitus. Diabetes Care. 2011;34(6):1419-1423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Polonsky WH, Fisher L, Schikman CH, et al. Structured self-monitoring of blood glucose significantly reduces A1C levels in poorly controlled, noninsulin-treated type 2 diabetes: results from the Structured Testing Program study. Diabetes Care. 2011;34(2):262-267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Miller KM, Beck RW, Bergenstal RM, et al. Evidence of a strong association between frequency of self-monitoring of blood glucose and hemoglobin A1c levels in T1D exchange clinic registry participants. Diabetes Care. 2013;36(7):2009-2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Clark LC, Jr, Lyons C. Electrode systems for continuous monitoring in cardiovascular surgery. Ann NY Acad Sci. 1962;102:29-45. [DOI] [PubMed] [Google Scholar]
- 9. Wang J. Glucose biosensors: 40 years of advances and challenges. Electroanalysis. 2001;13(12):983-988. [Google Scholar]
- 10. International Organization for Standardization. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. ISO 15197:2003. 2003. [Google Scholar]
- 11. International Organization for Standardization. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. ISO 15197:2013. 2013. [Google Scholar]
- 12. Freckmann G, Baumstark A, Jendrike N, et al. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther. 2010;12(3):221-231. [DOI] [PubMed] [Google Scholar]
- 13. Freckmann G, Schmid C, Baumstark A, et al. System accuracy evaluation of 43 blood glucose monitoring systems for self-monitoring of blood glucose according to DIN EN ISO 15197. J Diabetes Sci Technol. 2012;6(5):1060-1075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Freckmann G, Baumstark A, Schmid C, et al. Evaluation of 12 blood glucose monitoring systems for self-testing: system accuracy and measurement reproducibility. Diabetes Technol Ther. 2014;16(2):113-122. [DOI] [PubMed] [Google Scholar]
- 15. Kristensen GB, Monsen G, Skeie S, et al. Standardized evaluation of nine instruments for self-monitoring of blood glucose. Diabetes Technol Ther. 2008;10(6):467-477. [DOI] [PubMed] [Google Scholar]
- 16. Kuo CY, Hsu CT, Ho CS, et al. Accuracy and precision evaluation of seven self-monitoring blood glucose systems. Diabetes Technol Ther. 2011;13(5):596-600. [DOI] [PubMed] [Google Scholar]
- 17. Tack C, Pohlmeier H, Behnke T, et al. Accuracy evaluation of five blood glucose monitoring systems obtained from the pharmacy: a European multicenter study with 453 subjects. Diabetes Technol Ther. 2012;14(4):330-337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Baumstark A, Pleus S, Schmid C, et al. Lot-to-lot variability of test strips and accuracy assessment of systems for self-monitoring of blood glucose according to ISO 15197. J Diabetes Sci Technol. 2012;6(5):1076-1086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Brazg RL, Klaff LJ, Parkin CG. Performance variability of seven commonly used self-monitoring of blood glucose systems: clinical considerations for patients and providers. J Diabetes Sci Technol. 2013;7(1):144-152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Hasslacher C, Kulozik F, Platten I. Accuracy of self monitoring blood glucose systems in a clinical setting: application of new planned ISO-standards. Clin Lab. 2013;59(7-8):727-733. [DOI] [PubMed] [Google Scholar]
- 21. Brazg RL, Klaff LJ, Parkin CG. Performance variability of seven commonly used self-monitoring of blood glucose systems: clinical considerations for patients and providers. J Diabetes Sci Technol. 2013;7(1):144-152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Kuo CY, Hsu CT, Ho CS, Su TE, Wu MH, Wang CJ. Accuracy and precision evaluation of seven self-monitoring blood glucose systems. Diabetes Technol Ther. 2011;13(5):596-600. [DOI] [PubMed] [Google Scholar]
- 23. Billett HH. Hemoglobin and hematocrit. In: Walker HK, Hall WD, Hurst JW, eds. Clinical Methods: The History, Physical, and Laboratory Examinations. 3rd ed. Boston: Butterworth: 1990:718-719. [PubMed] [Google Scholar]
- 24. Pardo S, Pflug B, Dunne N, et al. Evaluating the accuracy and precision of 6 blood glucose monitoring systems using radar plots. Poster presented at: Diabetes Technology Society; November 6-8, 2014; Bethesda, MD. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Simmons DA. How should blood glucose meter system analytical performance be assessed? J Diabetes Sci Technol. 2016;10(1):178-184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care. 2000;23(8):1143-1148. [DOI] [PubMed] [Google Scholar]
- 27. World Health Organization, International Diabetes Federation. Definition and diagnosis of diabetes mellitus and intermediate hyperglycemia. Available at: http://whqlibdoc.who.int/publications/2006/9241594934_eng.pdf?ua=1.
- 28. American Diabetes Association Workgroup on Hypoglycemia. Defining and reporting hypoglycemia in diabetes: a report from the American Diabetes Association Workgroup on Hypoglycemia. Diabetes Care. 2005;28(5):1245-1249. [DOI] [PubMed] [Google Scholar]
- 29. Boyd JC, Bruns DE. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem. 2001;47(2):209-214. [PubMed] [Google Scholar]
- 30. Breton MD, Kovatchev BP. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol. 2010;4(3):562-570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Hasslacher C, Kulozik F, Platten I. Analytical performance of glucose monitoring systems at different blood glucose ranges and analysis of outliers in a clinical setting. J Diabetes Sci Technol. 2014;8(3):466-472. [DOI] [PMC free article] [PubMed] [Google Scholar]