Reply to “Sensitivity and Specificity of a Competitive Enzyme-Linked Immunosorbent Assay Relative to Quantitative Reverse Transcriptase PCR for Detection of Anaplasma marginale and A. phagocytophilum”

REPLY

We thank Pascale Aubrey and Julie Paré for their thoughtful consideration (1) of the information presented in our article (2). Many points of consideration have been presented, as well as the identification of the need for authors' corrections. Therefore, we will respond to each as they are addressed in their letter to the editor.

It was suggested that in Fig. 2a of our article, the legend and the label on the x axis are incorrectly identified as a “True negative rate”. We agree that this was a mistake and have addressed this as part of our authors' correction (3).

It was suggested that in Fig. 2b, the legend incorrectly identified the two curves. We agree that this was a mistake and have addressed this as part of our authors' correction (3).

We agree that an optimal cutoff is one that maximizes the sum of the sensitivity and specificity. The 15.3% cutoff identified in Fig. 2b does this. Furthermore, the Results report the basic approach in how this optimal negative cutoff was selected by us, specifically “by intersection of the plots for epidemiologic sensitivity and specificity”. The 20% cutoff suggested by Pascale Aubrey and Julie Paré does not agree with the method of selection that was reported. Their suggestion offers an alternative cutoff selection based on their perceived relative seriousness of either a false-negative or a false-positive test result. Furthermore, Fig. 2b was provided so that readers could evaluate different cutoffs based on their own perceived relative seriousness of either a false-negative or a false-positive test result, similarly to the suggestion made by Pascale Aubrey and Julie Paré. The reader also needs to consider the limitations of the data set because of the potential for the plots to change based on the population being sampled. Therefore, a cutoff for a two-way ROC curve (or any ROC curve, for that matter) will change on the basis of the prevalence of a disease in a sample population. Last, the presentation of a different cutoff for this cELISA was only offered as a point of comparison of the assay performance of the RT-PCR assay and the cELISA due to the cutoff used for the cELISA (manufactured by VMRD Inc.) presently being 30% inhibition, even though the Canadians use a negative cutoff of 42% inhibition (see references 2 and 37 in the report). Results of the cELISA are only reported as “positive” or “negative” to a client unless a request is also made for the percent inhibition results. The self-determination of a “positive” or a “negative” result by a client by the use of a cutoff other than 30% would be arbitrary unless a similar data set (as presented in the article) is available for the selection of an alternate cutoff. Therefore, we respectfully disagree with the arguments presented. The emphasis of this paper is for the development and validation of a duplex molecular biological assay.

We regret that the intended meaning of the discussion in our paper regarding sensitivity and specificity was misinterpreted by Pascale Aubrey and Julie Paré. We confirm that the article does indeed support the discussion as written. It is suggested by Pascale Aubrey and Julie Paré that an error was made in Results for the reporting of sensitivity and specificity of the cELISA if 15.3% inhibition is used as the negative cutoff value. It should be evident that there is a mistaken focus of attention on the information in Fig. 2b, rather than the information contained in Results. Specifically, the epidemiologic sensitivity and specificity of the cELISA was reported for 30% inhibition in Results and the effects of the implementation of 20% or 15.3% inhibition are reported for comparison in the Discussion. Therefore, we respectfully disagree with the concern raised.

We have addressed the concerns regarding the sensitivity and specificity information presented in the Discussion as part of our authors' correction (3). Further, as to the other issues surrounding this portion of the Discussion, the selection of 40% inhibition to achieve 95% sensitivity, similar to that of the licensure study, was determined by recalculation of sensitivity for the data set while incrementally increasing the percent inhibition cutoff until this target sensitivity was achieved. Thus, a 40% inhibition cutoff would have been necessary to achieve sensitivity results similar to those of the licensure study, in which a 28% inhibition cutoff was selected. Last, the reader is then asked to consider the effect of the selection of a 40% inhibition cutoff on the sensitivity and specificity of the cELISA with respect to our data set. This helps to reemphasize the previous discussion of the selection of alternative cutoffs on the basis of the perceived relative seriousness of either a false-negative or a false-positive test result. Therefore, we respectfully disagree with this argument and did not make modifications to the article.

In conclusion, we regret the errors that have been made and applaud both Pascale Aubrey and Julie Paré for bringing attention to the errors, as well as reaffirmation of the remaining results and interpretations presented in our article.