Monitoring of the Analytical Performance of Four Different Blood Glucose Monitoring Systems: A Post-market Performance Follow-Up Study

Abstract

Introduction

A sizeable minority of commercially available blood glucose monitoring (BGM) systems fail to satisfy regulatory accuracy requirements, such as ISO 15197:2013, after approval. This study assessed whether the BGMs tested could consistently meet these ISO requirements by investigating their accuracy in a non-standardized setting.

Methods

In this 18-month post-market performance study, using the ISO criteria, healthcare professionals tested the accuracy of four CE-marked BGM systems (Roche Diabetes Care, Mannheim, Germany) on European adults with diabetes mellitus. ISO criteria included 95% of blood glucose (BG) values being within ± 15 mg/dl of a reference measurement for BG < 100 mg/dl or ± 15% for BG ≥ 100 mg/dl and, in the Parkes Consensus Error grid for type 1 diabetes comparing capillary BGM measurements versus reference method, 99% of BG values falling within zone A (no effect on clinical action or outcome) and zone B (altered clinical action with little or no effect on clinical outcome).

Results

BGM readings were obtained from 1650 participants, and the number of readings per BGM system was between 1712 and 2376. The percentage of BGM readings that fell within ISO 15197:2013 limits ranged from 99.4 to 99.9%. For all meter types, 100% of data points fell within zone A or zone B, and most data points for each meter (≥ 99.9%) were in zone A.

Conclusion

All four CE-marked BGM models showed results within the accuracy limits defined by ISO 15197 in a non-standardized setting and thus consistently met regulatory accuracy requirements.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13300-024-01665-9.

Key Summary Points

Why carry out this study?

After regulatory approval, some commercially available blood glucose monitoring (BGM) systems do not meet regulatory accuracy requirements, such as ISO 15197:2013

This study assessed whether four different CE (Conformité Européenne)-marked BGM systems would consistently meet these International Organization for Standardization (ISO) requirements by investigating their accuracy in a non-standardized setting

What was learned from the study?

The four BGMs tested met ISO regulatory requirements for accuracy in a non-standardized setting. The accuracy criteria met included ISO 15/15 and the stricter 10/10 criteria

In meeting these regulatory accuracy requirements, the accuracy of these BGM models was demonstrated to be robust and consistent

Patients with diabetes and healthcare professionals benefit from routine post-market surveillance of BGM systems, which ensures the accuracy of these systems is routinely verified after regulatory approval

Introduction

People with diabetes mellitus use blood glucose monitoring (BGM) systems to adjust their therapy appropriately, helping them to achieve recommended blood glucose (BG) targets and to avoid acute and chronic diabetes complications [1, 2]. An essential requirement for both the safety and clinical value of BGM systems is that they should measure BG levels accurately. Accuracy is crucial because people with diabetes use measured values to guide their therapy. Healthcare professionals rely on reported BGM readings to make the right decisions about diabetes treatment so accuracy is equally important for them. However, after market approval, not all commercially available BGMs consistently satisfy regulatory accuracy requirements [3–5], such as those of the International Organization for Standardization (ISO 15197:2013) [6].

More than one in five modern CE (Conformité Européenne)-labeled BGM systems fail to reliably meet ISO 15197 accuracy criteria post launch with their tested strip system lot [3]. False readings from inaccurate BGMs may lead to treatment errors and severe hypoglycemic episodes [7, 8]. For example, accurate BGMs are essential for calculating the correct insulin dose for people with diabetes, especially those who need multiple daily injections. Failure to calculate an adequate insulin dose can result in hypoglycemic or hyperglycemic episodes that harm people with diabetes [9].

A contemporary surveillance study of BGM system accuracy that examined 18 CE-marked BGMs with capillary blood from 100 people with diabetes showed that a sizeable minority of BGM systems (N = 4/18, 22%) did not achieve ≥ 95% of results within the ISO 15197:2013 limits of ± 15 mg/dl for BG concentrations < 100 mg/dl or ± 15% for BG concentrations ≥ 100 mg/dl (ISO 15/15) [3, 6]. The same study also showed that, when stricter limits than ISO 15/15 were applied, few of the tested BGM systems (N = 3/18, 17%) achieved ≥ 95% of results within ± 10 mg/dl for BG concentrations < 100 mg/dl or ± 10% for BG concentrations ≥ 100 mg/dl [3].

Different accuracy requirements for BGM systems exist, and these vary, for example, with the BG testing setting. The US Food and Drug Administration (FDA) has issued two BGM guidelines, one for over-the-counter (OTC) testing and another for point-of-care (POC) testing for healthcare professionals [10, 11]. CE-marked BGM systems that can consistently meet regulatory accuracy requirements are uncommon [5]. The investigated devices have been shown to satisfy ISO 15/15 in CE-mark studies performed under standardized, controlled conditions, but little is reported in the literature about the consistency of their accuracy when used under non-standardized conditions.

The objective of this study was to determine whether the accuracy of contemporary CE-marked BGMs is consistent by evaluating their ability to meet ISO and FDA regulatory accuracy requirements in a non-standardized setting.

Methods

Study Design

An 18-month post-market performance study was conducted over 3 years, from August 2020 to January 2023, in three 6-month installments in the second halves of 2020, 2021, and 2022. Participants were adults ≥ 18 years old with type 1 or type 2 diabetes. The study took place in private clinics, hospitals, and clinical trial research centers with longstanding experience (≈20 years) in conducting post-market performance follow-up studies in Austria, Germany, and The Netherlands.

Accuracy testing was undertaken by study nurses, technicians, medical students, and healthcare providers. The BGM systems tested were the Accu-Chek Guide (BGM1), Accu-Chek GuideMe (BGM2), Accu-Chek Instant (BGM3), and Accu-Chek Instant S (BGM4) (Roche Diabetes Care, Mannheim, Germany). All four meter types are CE-marked BG self-monitoring devices that measure electrochemical reactions to quantitatively estimate the BG concentration in fresh capillary blood.

After blood has been put on a test strip, algorithmic analysis of these measurements generates an estimate of BG concentration in < 4 s. During each 6-month period, each BGM was tested on two to four strip lots. Selected strip lots were manufactured at approximately 6-week intervals. All strip lots originated from ongoing production and were placed on the market for sale. Meters and test strip lots were provided to the testing sites by the manufacturer. The accuracy of the BGMs was evaluated by lot according to the thresholds specified in ISO 15197:2013. The ISO standard states that BGM accuracy testing requires a minimum of three or more strip lots and duplicate measurements from 100 or more participants [6].

Technical Approach

BG was measured using the BGM and reference method by obtaining capillary blood from participants’ fingertips. Each blood sample was examined using test strips from two to four different lots during each 6-month period. Blood samples for reference testing were taken before and after the blood samples for strip testing. The difference between the two reference method measurements was calculated to determine whether there was unacceptable glucose instability in the sample, known as drift. Drift was defined as a difference > 4 mg/dl for BG concentrations < 100 mg/dl or > 4% for BG concentrations ≥ 100 mg/dl. If drift was noted, data were removed from the analysis.

Capillary BG measurements obtained with the CE-marked meters were compared with the mean value of the BG measurements obtained using the reference method. For the reference method, 20 µl of capillary blood was diluted in 400 µl of perchloric acid, frozen, and shipped to a laboratory (Roche Diabetes Care GmbH, Mannheim, Germany) for analysis. BG was measured using a Cobas 6000 c501 biochemical analyzer (Roche Diagnostics) and a glucose hexokinase-based reagent system. The hexokinase value was mathematically converted to a plasma-like reference. The reference measurement procedure used was directly linked to a higher-order measurement procedure (isotope dilution gas chromatography mass spectrometry (ID/GC/MS)) by a non-interrupted traceability chain [12]. Requirements for external factors, such as temperature, humidity, and interfering substances (e.g., dirt, alcohol, water, medication), were not generally regulated and varied between sites, so the study setting resembled a non-standardized, real-world setting.

Criteria Used to Measure BGM Accuracy and Safety

Accuracy

Primary Analyses

ISO 15197:2013 specifies minimum BGM system accuracy performance criteria [6]. First, 95% of measured glucose values should fall within either ± 15 mg/dl of the average measured values of the reference measurement procedure for BG concentrations < 100 mg/dl or within ± 15% for BG concentrations ≥ 100 mg/dl for each strip lot. Mean bias was calculated as the absolute difference in concentration (meter results – reference) for reference values < 100 mg/dl or as the relative difference in concentration ([(meter results—plasma like reference)/plasma-like reference] × 100) for reference values ≥ 100 mg/dl. Second, 99% of individual measured glucose values should fall within zones A and B of the Parkes Consensus Error Grid for type 1 diabetes [13]. Results in zone A indicate no effect on clinical action or outcome, while results in zone B indicate altered clinical action with little or no effect on clinical outcome results. Results in zone C indicate altered clinical action that is likely to affect clinical outcome. Results in zone D indicate altered clinical action that could have a significant medical risk, and results in zone E indicate altered clinical action that could have dangerous consequences.

Secondary Analyses

As part of the secondary analyses, criteria stricter than ISO 15197:2013 were used. The stricter 10/10 performance criteria specify that 95% of measured glucose values should fall within either ± 10 mg/dl of the average measured values of the reference measurement procedure for BG concentrations < 100 mg/dl or within ± 10% for BG concentrations ≥ 100 mg/dl. The accuracy of the BGM systems at extreme BG values was also evaluated using breakpoints for low and high BG. Low BG was defined as a value < 70 mg/dl, and high BG was defined as a value > 180 mg/dl.

Lastly, the accuracy of the BGM systems was evaluated against FDA accuracy requirements for BGM systems. For self-monitoring BGM systems for OTC use, ≥ 95% of readings need to fall within ± 15% and ≥ 99% of readings within ± 20% of the reference value [10]. For BGM systems for prescription POC use, ≥ 95% of readings should fall within either ± 12 mg/dl of the average measured values of the reference measurement procedure at glucose concentrations < 75 mg/dl or within ± 12% at glucose concentrations ≥ 75 mg/dl. In addition, ≥ 98% of readings should fall within either ± 15 mg/dl of the average measured values of the reference measurement procedure at glucose concentrations < 75 mg/dl or within ± 15% at glucose concentrations ≥ 75 mg/dl [11].

It should, however, be noted that the study was only partially compliant with FDA requirements. Requirements for glucose concentrations were different, and measurements were not taken by the end user, as directed by the guideline, but by trained study staff instead.

Safety

Safety was determined by monitoring the occurrence of safety incidents and serious safety incidents related to the use of the BGMs during each of the three 6-month periods.

Ethics

The study protocol was approved by the appropriate health authorities according to local guidelines and by an Institutional Review Board/Independent Ethics Committee. The Ethics Committees included: Ethikkommission of the State Medical Association of Baden-Wuerttemberg, Stuttgart (AZ MP-2018–10) for the center in Ulm, Germany and the Ethikkommission of the Medical Association of Lower Saxony, Hanover (AZ Bo/05/2018) for the center in Cloppenburg, Germany. Approval for conducting the study at the centers in Vienna and Graz in Austria was given by the Austrian Agency for Health and Food Safety (AGES). Study approval for the center in Zwolle, The Netherlands, was given by the local authority. The study was conducted in accordance with the Declaration of Helsinki of 1964 and its later amendments. Study participants provided written informed consent.

Statistical Methods

To ensure the meaningful evaluation of BGM system accuracy, the sample size was based on the requirements specified in ISO 15197:2013 for the evaluation of system accuracy (see Study design section). Two to four strips lots were tested during each 6-month period for each meter type (Tables S1–S4). Thus, data representing 8 to 12 strip lots per meter type were pooled from eight sites and collected during the three 6-month periods over 3 years. The pooled data were analyzed to determine whether the acceptance criteria were met.

The minimum accuracy acceptance criteria for quantitative testing in the primary analyses were that: (1) 95% of the measured glucose values fell within either ± 15 mg/dl of the perchloric acid hexokinase (PCA-HK) adjusted reference results for concentrations < 100 mg/dl or within ± 15% of the PCA-HK adjusted reference results at glucose concentrations ≥ 100 mg/dl for each system tested for each lot tested; (2) 99% of individual measured glucose values fell within zones A and B of the Parkes consensus error grid for type 1 diabetes for each system with all lots combined. Similar requirements were applied to the more stringent 10/10 performance criteria and to both the FDA OTC and POC criteria.

Results

Population

The study included 1650 participants and tested 8 to 12 different strip lots per meter type. Minimum requirements were thus met regarding the number of participants and the number of reagent system lots for BGM accuracy testing of ISO 15197:2013 and FDA guidance on the accuracy of BGM systems for prescription POC use and OTC use. The largest number of valid BGM readings was obtained for BGM1 (N = 2376) followed by BGM2 (N = 1949) and BGM4 (N = 1797). The smallest number of BGM readings was obtained for BGM3 (N = 1712) (Supplementary Tables S1-S4, Appendix 1). Participants were generally tested with four strip lots per meter, but the number of valid BGM readings ultimately depended on drift and missing data.

Accuracy

All BGM systems in the study met ISO 15/15 accuracy criteria. Depending on the meter type, 99.4–99.9% of readings were within the ISO 15197:2013 limits of ± 15 mg/dl for BG concentrations < 100 mg/dl or ± 15% for BG concentrations ≥ 100 mg/dl (Table 1). These criteria were met when the strip lots were pooled and when lots were analyzed individually (data not shown). In addition, the stricter 10/10 accuracy criteria were also met: 96.6–97.5% of readings were within ± 10 mg/dl for BG concentrations < 100 mg/dl or ± 10% for BG concentrations ≥ 100 mg/dl (Table 1).

Table 1.

Accuracy of four CE-marked blood glucose meters versus a reference method based on the breakpoint of 100 mg/dl in ISO 15197:2013

Meter	N (pooled across years)	Mean bias (mg/dl or %)	SD of mean bias	Percentage of readings within ± 10 mg/dl or within ± 10%	Percentage of readings within ± 15 mg/dl or within ± 15%
BGM1	2376	− 0.6	4.6	96.7	99.9
BGM2	1949	− 0.3	4.5	97.3	99.6
BGM3	1712	− 0.3	4.5	97.5	99.7
BGM4	1797	− 0.7	5.0	96.6	99.4

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne, ISO International Organization for Standardization, N number of meter readings, SD standard deviation (of mean bias)

The BGM systems tested were also accurate at measuring BG values at the extremes of the BG range. For low BG concentrations < 70 mg/dl, 96.7–100.0% of readings were within ± 10 mg/dl and 96.7–100.0% of readings were within ± 15 mg/dl (Table 2). For high BG concentrations ≥ 180 mg/dl, 95.6–97.8% of readings were within ± 10% and 98.9–100.0% of readings were within ± 15% (Table 3).

Table 2.

Accuracy of four CE-marked blood glucose meters versus a reference method in participants with low blood glucose (< 70 mg/dl)

Meter	N (pooled across years)	Mean bias (mg/dl)	SD of mean bias	Percentage of readings within ± 10 mg/dl	Percentage of readings within ± 15 mg/dl
BGM1	28	− 0.3	2.5	100.0	100.0
BGM2	30	0.3	4.3	96.7	96.7
BGM3	20	− 1.7	3.9	100.0	100.0
BGM4	16	− 0.4	3.1	100.0	100.0

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne, N number of meter readings, SD standard deviation (of mean bias)

Table 3.

Accuracy of four CE-marked blood glucose meters versus a reference method in participants with high blood glucose (> 180 mg/dl)

Meter	N (pooled across years)	Mean bias (%)	SD of mean bias	Percentage of readings within ± 10%	Percentage of readings within ± 15%
BGM1	904	− 1.7	4.2	97.7	100.0
BGM2	684	− 1.3	4.4	97.2	99.4
BGM3	548	− 1.5	4.1	97.8	99.8
BGM4	637	− 2.3	4.9	95.6	98.9

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne, N number of meter readings, SD standard deviation (of mean bias)

Regardless of the meter type examined, readings from the BGMs were sufficiently accurate to have little or no effect on clinical outcome, based on Parke consensus error grid definitions (Table 4) [12]. For all meter types, 100% of data points comparing capillary BGM measurements versus reference method fell within zone A or zone B of the Parkes consensus error grid (Table 4, Supplementary Figs. S1–S4). Most data points (≥ 99.9%) for each meter were in zone A, indicating the BGM measurement would have no effect on clinical outcome.

Table 4.

Parkes consensus error grid evaluation of the accuracy-related impact on clinical outcome for four CE-marked blood glucose meters

Meter	No effect on clinical action n (%) (Zone A)	Little or no effect on clinical outcomea n (%) (Zone B)	Likely to affect clinical outcomea n (%) (Zone C)	Could have significant medical riska n (%) (Zone D)	Could have dangerous consequencesa n (%) (Zone E)
BGM1	2376 (100.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
BGM2	1949 (100.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
BGM3	1711 (99.9)	1 (0.1)	0 (0.0)	0 (0.0)	0 (0.0)
BGM4	1795 (99.9)	2 (0.1)	0 (0.0)	0 (0.0)	0 (0.0)

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne

^aAltered clinical action in addition to the stated effects on clinical outcome

The different BGM meter types also met accuracy thresholds specified in FDA guidance. Regarding the accuracy of self-monitoring BGM systems for OTC use, the percentage of BGM readings within ± 15% of the reference value for the four different meter types tested ranged from 99.3 to 99.7% (Table 5). The percentage of BGM readings within ± 20% of the reference value for the four different meter types tested ranged from 99.7 to 100.0%. Regarding the accuracy of BGM systems for prescription POC use, the percentage of BGM readings within ± 12 mg/dl for reference BG concentrations < 75 mg/dl or ± 12% for reference BG concentrations ≥ 75 mg/dl ranged from 98.4 to 98.9% (Table 6). The percentage of BGM readings within ± 15 mg/dl for reference BG concentrations < 75 mg/dl or ± 15% for reference BG concentrations ≥ 75 mg/dl ranged from 99.3 to 99.7%.

Table 5.

Accuracy of four CE-marked blood glucose meters versus a reference method based on FDA thresholds for over-the-counter use

Meter	N (pooled across years)	Mean bias (%)	SD of mean bias	Percentage of readings within ± 15%a	Percentage of readings within ± 20%a
BGM1	2376	− 0.6	4.7	99.7	100.0
BGM2	1949	− 0.3	4.6	99.6	99.9
BGM3	1712	− 0.3	4.6	99.7	100.0
BGM4	1797	− 0.7	5.1	99.3	99.7

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne, FDA Food and Drug Administration, N number of meter readings, SD standard deviation (of mean bias)

^aFDA thresholds for over-the-counter use specify that 95% of results should be within 15% of the reference and 99% of results within 20% of the reference. These results did not use a breakpoint

Table 6.

Accuracy of four CE-marked blood glucose meters versus a reference method based on FDA thresholds for point-of-care use

Meter	N (pooled across years)	Mean bias (mg/dl or %)	SD of mean bias	Percentage of readings within ± 12 mg/dl or within ± 12%a	Percentage of readings within ± 15 mg/dl or within ± 15%a
BGM1	2376	− 0.6	4.6	98.9	99.7
BGM2	1949	− 0.3	4.5	98.9	99.6
BGM3	1712	− 0.3	4.5	98.9	99.7
BGM4	1797	− 0.7	5.0	98.4	99.3

BGM1 Accu-Chek Guide, BGM2 Accu-Chek GuideMe, BGM3 Accu-Chek Instant, BGM4 Accu-Chek Instant S, CE Conformité Européenne, FDA Food and Drug Administration, N number of meter readings, SD standard deviation (of mean bias)

^aFDA thresholds for point-of-care use specify that 95% of all values should be within ± 12% of the comparator method for glucose concentrations ≥ 75 mg/dl and within ± 12 mg/dl for glucose concentrations < 75 mg/dl, and that 98% of all values should be within ± 15% of the comparator method for glucose concentrations ≥ 75 mg/dl and within ± 15 mg/dl for glucose concentrations < 75 mg/dl

Safety

No safety incidents or serious safety incidents were reported.

Discussion

The four BGMs tested met ISO 15/15 regulatory accuracy requirements (ISO 15197:2013). Although not required, tighter 10/10 thresholds were also met, and the BGMs also met more stringent FDA OTC and POC requirements under non-standardized testing conditions. The performance of the BGMs in a clinical setting is an indication of the BGM system performance under real-world conditions, as the clinical setting includes several sources of potential inaccuracy, including system-related factors, subject-specific factors, and environmental conditions [14]. System-related factors affecting the accuracy of BGM systems include variations in the manufacturing of test strips, strip storage, and strip aging. Subject-specific factors include interfering substances, such as medications and metabolites, dirt, alcohol, or water from poor drying after hand washing. Environmental conditions include temperature and humidity [14].

The In Vitro Diagnostic Regulation (IVDR) of the European Union today requires manufacturers to actively collect data during the post-market phase of a product’s lifecycle [15]. Post-market performance follow-up is an integral part of post-market surveillance (PMS), which ensures that diagnostic devices meet their intended purpose and are safe and effective. PMS is defined as the systematic collection of device information after market approval to permit the updating of technical documentation and to allow competent authorities to conduct device vigilance and market surveillance [15]. PMS is a proactive process to maintain the quality, performance, and safety of a diagnostic device, whereby device manufacturers actively seek out post-market data rather than passively rely on the feedback and market information they receive [16].

The protocol for the collection of PMS data, the PMS plan, is developed before a device appears on the market and is updated throughout a device’s lifetime. The PMS plan addresses how information related to quality, performance, and safety is collected and used. Although manufacturers routinely conduct high-quality BGM accuracy studies to obtain a CE mark, independent testing and regular PMS of BGM systems could help confirm the findings from these studies and provide users with additional reassurance that the BGM systems they choose have the best accuracy performance [17].

In this study, system accuracy was also measured against stricter 10/10 criteria across the full BG range as, for people living with diabetes, accurate BGM systems ensure that diabetes is properly managed, treatments are adjusted correctly, and episodes of hypoglycemia or hyperglycemia are avoided. Inaccurate BGM systems are a potential source of patient harm [9]. False normal results can obscure hypoglycemia or lead a person with diabetes to inject an inadequately high dose of insulin. Conversely, false low results might lead to suboptimal diabetes therapy and cause a person with diabetes to unnecessarily consume fast-absorbed carbohydrates, which, over the long term, can lead to hyperglycemia, high HbA1c, and an increased risk of diabetic complications [18, 19].

Accurate readings allow people with diabetes to assess the impact of medication, diet, or lifestyle choices on BG levels and, if changes are required, ensure that medication dosages, insulin regimens, or dietary recommendations are adjusted to optimize BG control [1]. Precise and timely BG measurement can either prevent serious illness or prevent it from becoming worse. For example, it allows people with diabetes to avert potentially life-threatening hypoglycemia or hyperglycemia while, in patients who are already severely ill, it guarantees proper management, which is vital as inaccurate readings can lead to incorrect treatment decisions with dangerous adverse outcomes. Lastly, accurate BG monitoring gives people with diabetes confidence in managing their condition and a sense of control over problems associated with poorly controlled BG.

The accuracy requirements of BGM systems used by healthcare professionals are generally more stringent than those of BGM systems used by people with diabetes. BGM systems used by healthcare professionals require high accuracy performance and reliability to conform with the specific accuracy requirements of POC guidelines, such as those of the FDA [11]. BGM systems for POC use by trained personnel in professional healthcare settings, such as hospitals, clinics, and physician practices, often include additional features and functions and provide more comprehensive data for clinical decision-making.

These systems are frequently more complex and have higher performance requirements compared to BGM systems for OTC use [10, 11]. BGM systems for OTC use are designed for use by the general population, for example, individuals with diabetes who monitor their BG levels during ambient living conditions. As such, these systems are usually simpler and require less technical expertise. By separating POC testing from OTC testing, the FDA has provided healthcare professionals and the general population with access to accurate and reliable BGM systems that meet their specific needs based on appropriate regulatory requirements and guidance for each category [10, 11].

Strengths and Limitations

One strength is that the results of the study represent the performance of 8 to 12 test strip lots per meter type, which is more than the minimum requirement of 3 specified by ISO and the FDA. A further strength is that the study was not controlled for interferants. Care should be taken when interpreting the accuracy findings in patients with hypoglycemia (< 70 mg/dl) as the sample sizes were small. The measurements from the BGMs were taken by trained healthcare personnel in a clinical setting, so the readings may be more accurate than those taken by people with diabetes, their family, or carers [20, 21]. However, the downside to this is that our results may not be fully generalizable for these other end users. The study focuses on the technical performance of these BGM systems in an optimized setting that excludes the effect of naïve end users as much as possible. As such, the study is unlikely to be markedly influenced by these end users and, therefore, provides a better reflection of the stability of the BGM systems and glucose strips, which is an important part of post-market surveillance. In addition, trained healthcare personnel are likely to know about subject-specific factors affecting accuracy and how to mitigate their impact. While the study followed participants for 18 months, a longer, multi-year analysis would have permitted a more comprehensive evaluation of BGM system reliability, including long-term trends and device degradation. Lastly, this study cannot be compared directly to studies performed according to ISO 15197 or FDA guidance, as our study procedures did not consider all their requirements. Nevertheless, our study shows that the BGM systems tested function accurately based on testing by healthcare professionals.

Conclusions

The four CE-marked BGM systems tested met ISO and FDA regulatory requirements for accuracy in a non-standardized setting. The accuracy criteria met included ISO 15/15 and the stricter 10/10 criteria as well as those in the FDA POC and OTC guidance for industry. In meeting these varied regulatory accuracy requirements, the accuracy of these BGM systems was demonstrated to be robust and consistent.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contributions

Annette Baumstark, Johannes Tüting, Günter Sokol, Robbert J. Slingerland, and Ruth Schuebel were involved in study design and conceptualization. Julia K. Mader, Johannes Tüting, Günter Sokol, and Robbert J. Slingerland validated and verified the data. Yuhong Tong, Julia Roetschke, and all the other authors were involved in study execution, data interpretation, visualization and data presentation, and writing, editing, and reviewing the manuscript.

Funding

This study was conducted and funded by Roche Diabetes Care, including the journal’s Rapid Service Fee.

Data Availability

Additional data are available in the supplementary material, including the assessment of evaluable data for individual BGM systems and Parkes Consensus Error Grids for each BGM system.

Declarations

Conflict of interest

Julia K. Mader is a member of advisory boards of Abbott Diabetes Care, Becton-Dickinson, Boehringer Ingelheim, Eli Lilly, Embecta, Medtronic, NovoNordisk A/S, Roche Diabetes Care, Sanofi-Aventis, and Viatris. She has received speaker honoraria from A. Menarini Diagnostics, Abbott Diabetes Care, AstraZeneca, Boehringer Ingelheim, Dexcom, Eli Lilly, Medtrust, MSD, NovoNordisk A/S, Roche Diabetes Care, Sanofi, Servier, Viatris, and Ypsomed. Julia K. Mader is a shareholder of decide Clinical Software GmbH and elyte Diagnostics and serves as CMO of elyte Diagnostics. Annette Baumstark is an employee of the Institut für Diabetes-Technologie, Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm (IfDT), which carries out clinical studies, for example, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. Johannes Tüting and Günter Sokol have no conflicts of interest to declare. Robbert J. Slingerland is chair of the Clinical Chemistry Department of Isala, which carries out clinical studies, for example, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. Robbert J. Slingerland has received speakers’ honoraria or consulting fees in the last three years from Roche and Menarini. Ruth Schuebel, Yuhong Tong, and Julia Roetschke are employees of Roche.

Ethical Approval

The study protocol was approved by the appropriate health authorities according to local guidelines and by an Institutional Review Board/Independent Ethics Committee. The Ethics Committees included: Ethikkommission of the State Medical Association of Baden-Wuerttemberg, Stuttgart (AZ MP-2018-10) for the center in Ulm, Germany; and the Ethikkommission of the Medical Association of Lower Saxony, Hanover (AZ Bo/05/2018) for the center in Cloppenburg, Germany. Approval for conducting the study at the centers in Vienna and Graz in Austria was given by the Austrian Agency for Health and Food Safety (AGES). Study approval for the center in Zwolle, The Netherlands, was given by the local authority. The study was conducted in accordance with the Declaration of Helsinki of 1964 and its later amendments. Study participants provided written informed consent.