Applying Parkes Grid Method to Evaluate Impact of Variation in Blood Glucose Monitoring (BGM) Strip Accuracy Performance in Type 1 Diabetes Highlights the Potential for Amplification of Imprecision With Less Accurate BGM Strips

Abstract

Background:

The National Health Service spends £170 million on blood glucose monitoring (BGM) strips each year and there are pressures to use cheaper less accurate strips. Technology is also being used to increase test frequency with less focus on accuracy.

Previous modeling/real-world data analysis highlighted that actual blood glucose variability can be more than twice blood glucose meter reported variability (BGMV). We applied those results to the Parkes error grid to highlight potential clinical impact.

Method:

BGMV is defined as the percent of deviation from reference that contains 95% of results. Four categories were modeled: laboratory (<5%), high accuracy strips (<10%), ISO 2013 (<15%), and ISO 2003 (<20%) (includes some strips still used).

The Parkes error grid model with its associated category of risk including “alter clinical decision” and “affect clinical outcomes” was used, with the profile of frequency of expected results fitted into each BGM accuracy category.

Results:

Applying to single readings, almost all strip accuracy ranges derived in a controlled setting fell within the category: clinically accurate/no effect on outcomes areas.

However modeling the possible blood glucose distribution in more detail, 30.6% of longer term results of the strips with current ISO accuracy would fall into the “alter clinical action” category. For previous ISO strips, this rose to 44.1%, and for the latest higher accuracy strips, this fell to 12.8%.

Conclusion:

There is a minimum standard of accuracy needed to ensure that clinical outcomes are not put at risk. This study highlights the potential for amplification of imprecision with less accurate BGM strips.

What Is Already Known About This Subject?

Previous modeling work/real-world data analysis highlighted that actual blood glucose variability can be more than twice as much as the reported variability of blood glucose monitoring (BGM) strips that are used in everyday practice.

What Is the Key Question?

We applied those results to other studies where error grids have been developed. We wanted to determine the actual clinical impact of less vs more accurate BGM strips.

What Are the New Findings?

Those meters with variability of readings between 10% and 20% vs the laboratory standard fall into the category of potentially affecting clinical outcomes. Thus, we have highlighted the previously possible unknown risks associated with using less accurate BGM strips.

Introduction

Accessible blood glucose monitoring (BGM) has been part of the management of diabetes mellitus, since 1981 with the launch of the Glucometer.¹ The technology was initially applied to diabetes patients treated with insulin and more recently used by people with type 2 diabetes on oral hypoglycemic agents, particularly the insulin secretagogues.

The National Health Service (NHS) spends £170 million on BGM strips each year and there are pressures to use cheaper less accurate strips.² Technology is also being applied to increase test frequency with less focus on accuracy.

Previous modeling work/real-world data analysis³ highlighted that actual blood glucose variability (BGV) can be more than twice as much as the blood glucose monitor variability (BGMV) of the meters being used. We applied those results to methods from other studies where error grids have been developed to highlight the potential clinical impact of this.

We recently showed⁴ that use of more variable/less accurate BGM strips in Type 1 diabetes mellitus (T1DM) is associated both theoretically and in practice with a larger variability in measured blood glucose and HbA1c, with implications for patient confidence in their day-to-day monitoring experience. This was based on in-silico analysis by Breton and Kovatchev in 2010.³ We have also shown that decisions taken in general practices (GPs) have a profound influence on glycemic outcomes in T1DM.⁵

Our analysis showed both in modeling and in real-world data that the effect of longer term multiple use of higher variability/less accurate BGM strips resulted in an increase in longer term variability of actual blood glucose levels in both models and as measured by HbA1c.⁴ We found that the increase in BGV was over twice the change in variability in blood glucose strips.

One possible reason for this amplification may be due to the effect less accurate measurements have on the amount of insulin chosen to be injected, which then amplifies the impact of inaccuracy. Table 1 is a worked hypothetical example, for a patient with a personal correction factor of 3, using ISO (15197:2013)⁶ standard accuracy BGM strips (where 95% of meter results fall within ±15% of actual value), starting with an actual blood glucose measured value of 14 mmol/L and calculating a bolus dose to achieve a target of 5 mmol/L, the actual final blood glucose could be between 3.5 and 8 mmol/L. This gives a blood glucose variance of −30% to +60% which is two to five times the original variance in the blood glucose measurement.

Table 1.

Worked Example: Impact BGM Variance on Bolus Insulin Dosing and Blood Glucose Result Variance.

	Actual	95% results fall within
		Low reading	High reading
BG meter display value (mmol/L)	14	11.9	16.1
BG meter ISO variance % from actual value		−15%	15%
Subtract target BG (5 mmol/L) from display value	9	6.9	11.1
Apply personal correction factor (3 mmol/L = 1 Unit) to calculate insulin units	3	2.3	3.7
Round insulin amount to nearest 0.5 unit and inject	3	2	3.5
Insulin impact reduction in actual BG level based on Personal Correction Factor (PCF)	9	6	10.5
Actual blood glucose achieved (actual minus insulin impact)	5	8	3.5
Actual BG variance as % of expected actual BG value		60%	−30%
Actual BG variance relative to BG meter variance		×4	×2

BG, blood glucose; BGM, blood glucose monitoring.

In that case one time in 20 results, this patient would see a reading of either above 13.8 mmol/L or less than 10.2 mmol/L. Subtracting their target would give them a difference either below 5.2 mmol/L or above 8.8 mmol/L. Having applied their personal correction factor, they would then proceed to inject either less than 1.5 units or more than 3 units of insulin and that would then see their actual blood glucose value fall from the actual 12 mmol/L to either above 7.5 mmol/L or below 3 mmol/L, which is greater than ±40% of their target 5 mmol/L.

This exemplifies how the percentage level of variation in actual resulting blood glucose level can be more than double that of the percentage variation in measurement strip accuracy. Given the short-term consequences of hypoglycemia, many users with lower confidence in their measurement accuracy adjust their target blood glucose to higher values to avoid this effect and so carry the longer term consequences.

The Parkes error grid was published in 2000.⁷ This described how the clinical accuracy of a single blood glucose value measured by a blood glucose monitor can be expressed as a description of the potential clinical outcome associated with basing a treatment decision on this value. The methodology was further developed by Pfützner et al in 2013.⁸

We here report a relevant further analysis of our data for BGM strip use in England/Wales utilizing the Parkes grid methodology but applying the variability to the actual blood glucose values.

Methods

Amplification of BGV as percent comes through the differential between measurement uncertainty and target blood glucose and then selection of insulin dose. Previous work, specifically Bretton³ in silico modelling and Heald⁴ real-world data analysis both independently showed over multiple readings the amplification factor between meter variance and actual blood glucose variance to be around 2.5 as highlighted in Table 1 worked example.

Error grids are used to link the difference in device readings to reference values to possible clinical outcomes. We here applied the Parkes error grid model^7,8 to the different levels of accuracy for BGM strips including current and previous ISO standard (some of these lower accuracy strips are still used).

There are various bands of analytical performance levels, which are assessed by percentage of the nominal value to contain 95% of actual results. These include:

• ±5%—value being exceeded by clinical laboratory blood tests.

• ±10%—value being exceeded by higher accuracy strips.

• ±15%—value required for current ISO 2013 certification.

• ±20%—value required for previous ISO 2003 certification and still delivered in some lower accuracy strips.

These single measurement limits were overlaid on to the Parkes error grid,⁷ with scales converted to the UK blood glucose unit of mmol/L. These figures were then increased by the amplification factor 2.5 to simulate the above increased variability in blood glucose levels due to multiple repetitions as estimated in previous modeling studies.^3,4

In order to identify the percent of blood glucose values that would fall into the “affect clinical decision” category, the distribution of blood glucose values (assumed to be Gaussian) with the blood glucose measurement variance ±10%, ±15%, and ±20% around a reference blood glucose level of 8 mmol/L was plotted. The percent of actual blood glucose values falling outside these limits was shown as percent of total results.

Result

Single Measure of Blood

Applying to single readings, almost all strip accuracy ranges derived in a controlled setting fall within Category A: clinically accurate/no effect on outcomes areas—see Figure 1.

Figure 1.

Parkes error grid reference glucose value vs meter readings for variety of accuracy (single reading).

Longer Term Impact (on Actual Blood Glucose)

Both Breton et al in an in-silico study³ looking at blood glucose level variability and Heald et al⁴ considering HbA1c variability showed that in cumulative readings over three months that variability in actual blood glucose was between two and three times higher than the meter variability. If a 2.5 factor is applied to increase the meter variability ranges, then a significant proportion of outcomes from <15% to <20% meters now fall into Category B: alter clinical action and are approaching Category C: affect outcomes (Figures 2 and 3).

Figure 2.

Parkes error grid vs impact of meter accuracy on longer term blood glucose variability.

Figure 3.

Impact of variability in device measurement of blood glucose on proportion of values exceeding the Parkes Grid “Alter Clinical Action” limits based on reference sample at 8 mmol/L.

Any further influences on variability incurred by patients’ own use (including coding, strip storage, and shelf life, using correct procedures) have not been included and so could potentially further increase the impact on risk and hence clinical outcomes—Figure 2.

If we model the possible distribution of blood glucose results in more detail and display as a distribution, 30.6% of longer term results of the strips with current ISO accuracy (within 15%) would fall into the “alter clinical action” and if previous ISO strips (accuracy within 20%) are evaluated, this rises to 44.1%, and if the latest higher accuracy strips (within 10%), this falls to 12.8% (Figure 3).

Discussion

We have shown a link between analytical blood glucose meter accuracy and an established qualitative error grid, highlighting the potential impact of accuracy on clinical decision and outcomes.

There is a minimum standard of accuracy needed to ensure that clinical outcomes are not put at risk. This study highlights the previously possible unknown risks associated with using less accurate strips and the potential for “amplification” of imprecision, as people with diabetes requiring insulin treatment frequently make decisions about insulin dose based on the last blood glucose reading.

Increased variability in blood glucose values is viewed with increasing importance in relation to clinical decisions and outcomes.^4,5 One source of this variability is the accuracy of the BGM strips, the results from which patients use to subtract their target and select their insulin dose.⁴ The growing use of faster insulins and of insulin pumps increases this effect. Uncertainty around BGM accuracy also might cause patients to aim for a higher blood glucose target to reduce the risk of hypoglycemia long-term complications in terms of diabetes tissue complications.

In T1DM, the FreeStyle Libre flash blood glucose monitor^9,10 allows users retrospectively to review the preceding eight hours of continuous glucose data, along with a contemporary estimated blood glucose value and trend line. This technology is increasingly being utilized by people with insulin treated diabetes. Nevertheless traditional blood glucose monitoring will continue to be the way that most people who take insulin to treat diabetes monitor their diabetes.

This study supports the view that conventional analytical single stage accuracy does not reflect all the short- and long-term impact on blood glucose associated with BGM.

Strengths: This analysis has built on our previous work using nationally available data resources on prescribing and outcomes in T1DM and the available BGM strip accuracy data.

Limitations: Our conclusions are based on real-world modeling rather than a formal prospective study of outcomes. This implies that there may be unidentified confounding factors influencing the results. However, the face validity of the results and the methodological approach indicates likely clinical relevance.

Our results suggest there are clear advantages to utilizing best in class accuracy BGM strips in terms of precision of monitoring, quality of patient experience, and patient confidence in the technology applied to monitor their blood glucose.

Conclusion

Frequent, accurate and convenient testing of glycaemic control is the gold standard for the management of diabetes. Blood glucose monitoring for patients become extensively used more than 30 years ago.¹ The full potential benefits are now being held back by the continued use of older lower accuracy strips in some contexts. BGM strips of the highest possible accuracy should be considered for those individuals who titrate their insulin dose.