The standard ISO (International Organization for Standardization) 151971 is widely accepted for the accuracy assessment of systems for self-monitoring of blood glucose (SMBG). In the European Union, manufacturers have to provide evidence of conformity with ISO 15197 to obtain the Conformité Européene (CE) mark for their meter. However, application of the CE mark is a 1-time procedure before the market approval; regular and independent accuracy evaluations of market-released test strip lots of a SMBG system are not mandatory. In fact, studies have repeatedly shown that individual test strip lots of available systems do not comply with ISO 15197 accuracy criteria.2-5
In this study, 3 CE-marked SMBG systems (CareSens N, CareSens N POP, alphacheck professional) with 3 different test strip lots each were procured by the manufacturer (i-SENS, Inc, Korea) to assess compliance to accuracy criteria stipulated in ISO 15197:2003. In addition, more stringent criteria of its revision, ISO 15197:2013,6 were applied.
The study was performed at the Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany in compliance with the German Medical Devices Act. The study protocol was approved by the Ethics Committee and the Federal Institute for Drugs and Medical Devices.
Each system was tested on 100 capillary blood samples from different subjects (≥18 years, diabetes type 1 or type 2 or no diabetes) following procedures described in ISO 15197:2003. Comparison measurements were performed on capillary plasma with a glucose oxidase laboratory method (YSI 2300 STAT Plus™ glucose analyser, YSI Incorporated, Yellow Springs, OH, USA). To confirm trueness and precision of the comparison method, regular internal and external quality control measures were performed.
Each of the tested systems fulfilled with all 3 tested lots accuracy criteria of ISO 15197:2003 with 99.5% to 100% of measurement results within ±15 mg/dl of the comparison measurement at glucose concentrations <75 mg/dl and within ±20% at glucose concentrations ≥75 mg/dl (Table 1). Applying the tighter criteria of the revision ISO 15197:2013, the systems showed 98% to 100% of measurement results within ±15 mg/dl of the comparison measurement at glucose concentrations <100 mg/dl and within ±15% at glucose concentrations ≥100 mg/dl. Consensus error grid analysis as required by ISO 15197:2013 showed for each system 100% of measurement results within zones A and B. The relative bias according to Bland and Altman, that is, average across all differences between SMBG results and comparison method measurement results divided by their mean, ranged from 1.3% to 2.3% for CareSens N, from −0.7% to 1.6% for CareSens N POP and from −0.3% to 0.9% for alphacheck professional.
Table 1.
System Accuracy Results (Number and Percentage) Within Limits of ISO 15197:2003 and ISO 15197:2013.
| Results within |
|||||||
|---|---|---|---|---|---|---|---|
| SMBG system | ISO 15197:2003 criteriaa | ISO 15197:2013 criteriab | Consensus error grid zones A and Bc | Relative bias (Bland and Altman) | |||
| Meter | Test strip lot | ±15 mg/dl and ±20% | ±10 mg/dl and ±15% | ±15 mg/dl and ±15% | ±10 mg/dl and ±10% | ||
| CareSens N | JD22LN47A | 199/200 (99.5%) | 196/200 (98%) | 199/200 (99.5%) | 186/200 (93%) | 600/600 (100%) | 2.3% |
| CareSens N | JD26LN51A | 200/200 (100%) | 198/200 (99%) | 200/200 (100%) | 181/200 (90.5%) | 1.3% | |
| CareSens N | JD01LN17C | 200/200 (100%) | 192/200 (96.0%) | 196/200 (98%) | 183/200 (91.5%) | 2.2% | |
| CareSens N POP | JD22LN47A | 200/200 (100%) | 200/200 (100%) | 200/200 (100%) | 191/200 (95.5%) | 599/599 (100%)d | 0.5% |
| CareSens N POP | JD26LN51A | 200/200 (100%) | 199/200 (99.5%) | 199/200 (99.5%) | 190/200 (95%) | –0.7% | |
| CareSens N POP | JD01LN17C | 200/200 (100%) | 196/200 (98%) | 198/200 (99%) | 187/200 (93.5%) | 1.6% | |
| alphacheck professional | JD07XA15D | 200/200 (100%) | 196/200 (98%) | 199/200 (99.5%) | 188/200 (94%) | 600/600 (100%) | 0.9% |
| alphacheck professional | JD08XA16A | 200/200 (100%) | 198/200 (99%) | 198/200 (99%) | 188/200 (94%) | –0.1% | |
| alphacheck professional | JD08XA17B | 200/200 (100%) | 195/200 (97.5%) | 197/200 (98.5%) | 187/200 (93.5%) | –0.3% | |
Number and percentage of the results within zones A and B of the consensus error grid (across all 3 tested lots). Bias between the systems’ measurement results and the comparison measurement results according to Bland and Altman.
95% of the individual glucose measurement results shall fall within 15 mg/dl of the comparison measurement procedure at BG concentrations <75 mg/dl and within 20% at BG concentrations ≥75 mg/dl.
95% of the individual glucose measurement results shall fall within 15 mg/dl of the comparison measurement procedure at BG concentrations <100 mg/dl and within 15% at BG concentrations ≥100 mg/dl.
ISO 15197:2013 stipulates that 99% of individual measurement results shall fall within zones A and B of the consensus error grid.
The consensus error grid counts only individual BG results with comparison measurement results <550 mg/dl and BG system measurement results <550 mg/dl. Results that did not fall within these limits were not assessed.
All 3 systems fulfilled with all 3 investigated test strip lots accuracy requirements of the international standard ISO 15197:2003 and its revision ISO 15197:2013, for which mandatory compliance is recommended after a 36-month transition period. In bias analysis for all 3 systems minimal lot-to-lot variability was observed. Regular and independent evaluations after the market approval of a SMBG system is helpful to ensure constant measurement quality and adherence of market-released test strips to accuracy requirements stipulated in ISO 15197.
Footnotes
Abbreviations: CE, Conformité Européene; ISO, International Organization for Standardization; SMBG, self-monitoring of blood glucose.
Declaration of Conflicting Interests: All authors are employees of the Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm (IDT), Ulm, Germany. GF is general manager of the IDT, which carries out studies evaluating SMBG systems and medical devices for diabetes therapy on behalf of various companies. GF and IDT have received speakers’ honoraria or consulting fees from Abbott, Bayer, Becton Dickinson, Berlin Chemie, Menarini Diagnostics, Novo Nordisk, Roche Diagnostics, Sanofi, and Ypsomed.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by a grant from i-SENS, Inc, Korea.
References
- 1. In Vitro Diagnostic Test Systems—Requirements for Blood-glucose Monitoring Systems for Self-testing in Managing Diabetes Mellitus. DIN EN ISO 15197. 2003. [Google Scholar]
- 2. Freckmann G, Baumstark A, Schmid C, Pleus S, Link M, Haug C. Evaluation of 12 blood glucose monitoring systems for self-testing: system accuracy and measurement reproducibility [published online ahead of print November 8, 2013]. Diabetes Technol Ther. [DOI] [PubMed]
- 3. Brazg RL, Klaff LJ, Parkin CG. Performance variability of seven commonly used self-monitoring of blood glucose systems: clinical considerations for patients and providers. J Diabetes Sci Technol. 2013;7(1):144-152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Tack C, Pohlmeier H, Behnke T, et al. Accuracy evaluation of five blood glucose monitoring systems obtained from the pharmacy: a European multicenter study with 453 subjects. Diabetes Technol Ther. 2011;14(4):1-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Freckmann G, Baumstark A, Jendrike N, et al. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther. 2010;12(3):221-231. [DOI] [PubMed] [Google Scholar]
- 6. In Vitro Diagnostic Test Systems—Requirements for Blood-glucose Monitoring Systems for Self-testing in Managing Diabetes Mellitus. ISO 15197. 2013.