Interference Testing: Why Following Standards Is Not Always the Right Thing to Do

Abstract

Lucarelli and colleagues in this issue of Journal of Diabetes Science and Technology describe the results of interference testing for a continuous glucose monitoring system. The authors follow the Clinical Laboratory Standards Institute guideline EP7-A2, including their conclusions, in which the concepts of a statistically significant interfering substance and a clinically important interference have been combined in a way whereby information from the experiment has been lost and could be misleading. A better way to treat the data is presented, including a simulation method to evaluate the effects of interferences.

Lucarelli and colleagues¹ describe the results of interference testing for a continuous glucose monitoring (CGM) system. Although the authors follow the Clinical Laboratory Standards Institute (CLSI) guideline about interference testing, EP7-A2,² the CGM system presents some challenges to an otherwise simple interference experiment. The purpose of this article is to highlight some problems in their study and to present ways to represent the effect of interferences more clearly.

The authors screen their list of candidate interferences by testing them in a modified CGM setup, which excludes the microdialysis system. They then test compounds that have failed the screen in the complete CGM system (with microdialysis). This is a biased approach. While micro-dialysis has apparently reduced the interference effects of some of the compounds that failed the screen, it is unknown whether the interference of other compounds, which were not tested because they passed the screen, might actually be worse when tested with the complete CGM system.

That aside, how can one best report the results of an interference study? The basic experiment is to test with sufficient replicates a test solution with a candidate interfering substance and an otherwise identical solution without the candidate interfering substance (e.g., the control). The average difference and a 95% confidence interval for this difference are calculated. Figure 1 shows possible outcomes for several candidate compounds.

Figure 1.

Average percentage difference (middle point) and 95% upper and lower confidence interval for test minus control for various hypothetical candidate interfering substances. The goal for clinical acceptability of this difference in this example is ±10%. Note, a similar figure appeared in the CLSI guideline EP7-P but is not in later versions of the guideline.

The correct way to state the results for compound A is to say that no interference was detected. It is incorrect to say that there was no interference. While this may seem to be nitpicking, the distinction is important because one can never prove the null hypothesis, and the actual interference could be as large as the ends of the confidence interval chosen. The 95% confidence interval gives an indication of the reliability of the experiment. For example, the result for compound B is also that no interference was detected, but here, the interference could be as great as +8% or -8%. In this case, probably due to imprecision, a larger sample size would be indicated.

Perhaps the biggest problem is with compound C. If the authors have set an acceptability limit of ±10%, it would be incorrect to say that compound C does not interfere because it is within the clinical acceptability goal. Compound C does interfere. Compound D also interferes, but its point estimate is not above 10%. Finally, compound E does interfere and has also failed the acceptance limit of ±10%.

Returning to compound C, one can ask, what if there are several compounds, all of which interfere but many at a level less than the 10% acceptance limit? In the case of the authors’ work for serum, besides dopamine, xylose interferes. For whole blood, besides dopamine, acetaminophen and glutathione (and possibly maltose and ascorbic acid) interfere. To determine if interferences cause too much error, one can perform a Monte Carlo simulation. In a full simulation, one would need the glucose response of every error source and each source’s distribution. For example, reagent lot is a common source of bias. If a manufacturer had reagent release specifications of ±3%, then ±3% would presumably be the largest effect from reagent lot bias. But one also has to randomly sample from reagent lot bias, so one would have to know the distribution of actual reagent lot biases. Imprecision is always present as an error source. For interferences, one needs to know the response of the interference, which is determined by experiments as the authors performed, and one also needs to know the distribution of each interfering substance’s concentration expected in the population of patients of interest. The Monte Carlo simulation would start with a known glucose concentration and simultaneously randomly sample from each error source’s distribution and apply the glucose response to the concentration of error source sampled such that, across all error sources, one would obtain a glucose result to be compared with the starting glucose value. One would repeat this process thousands of times across the glucose range. If one performed the simulation with and without the interfering substances, one could evaluate the difference in error due to interferences.

To be fair to the authors, EP7-A2 is misleading, if not outright wrong. In the two examples in EP7-A2 on interference claims (section 9.1.1), for aspartate amino-transferase, it is stated that, “A bias exceeding 10% is considered a significant interference,” and in 9.1.2, “The following substances, when tested in serum in [aspartate aminotransferase] activities of 25 and 200 U/literaccording to this CLSI protocol, were found not to interfere at the concentrations indicated. A bias of less than 10% (upper limit of 95% confidence interval) is not considered a significant interference.” If one examines package insert sheets, one will commonly see statements that follow this model. Thus, if a compound interfered at 9%, it would be put in the same category as a compound for which no interference was detected, but the 9% interference would be unreported and hence unknownto users. In the EP7-A2 bilirubin example, it is stated, “Bias exceeding 0.2 mg/dl is considered interference.” Again, this implies that a compound with statistically significant bias less than 0.2 does not interfere, which is incorrect (the bilirubin reference interval is approximately 0.1 to 1.2 mg/dl).

An interference experiment such as EP7-A2 describes a protocol to get data and an analysis method to turn data (facts and figures) into information (knowledge gained from the data). The problem is that two concepts (analytical interference, whether a candidate interfering substance’s effect is statistically different from zero, and clinical importance, whether an analytical interference will adversely affect a treatment decision) have been combined in a way that discards the analytical interference information if it is less than a clinical importance limit. This is an undesirable and unnecessary loss of information. A more cynical view is that this manufacturer-dominated standards group has chosen this practice to present a smaller list of interfering substances. A simple alternative would be to present two columns for each interfering substance—the level of analytical interference found (if detected) and whether this level is of clinical importance.

On a more positive note, the authors’ Figures 2 and 3 clearly show what is going on and should be in every publication that contains an evaluation of interferences.

Glossary

(CGM)

continuous glucose monitoring

(CLSI)

Clinical Laboratory Standards Institute