Reply to “Interlaboratory and Interstudy Reproducibility of a Novel Lateral-Flow Device: a Statistical Issue”

REPLY

We welcome the letter from Dr. Sabour concerning our manuscript on the interlaboratory and interstudy reproducibility of the results of the lateral-flow device (LFD) as well as on the influence of antifungal therapy on the sensitivity of this device for the detection of invasive pulmonary aspergillosis (1). As Dr. Sabour suggests in his letter, we have revisited the data and evaluated its interlaboratory reproducibility by kappa analysis (2). For this purpose, we considered two categories, positive and negative results, as these are the most relevant and the ones most likely to be used clinically. For serum, the kappa value was 0.933, indicating very good agreement between the laboratories for the LFD. For BAL fluid, the kappa value for the interlaboratory LFD comparison was 0.687, indicating good agreement for this assay between the two laboratories. Of note, with uninfected animals, the two laboratories were in 100% agreement, as shown in Tables 1 and 2 of our article. We also compared the results for the LFD from separate studies conducted at different time points. This was also done for the clinically available diagnostic assays for galactomannan and (1,3)-β-d-glucan. As shown in Table 3 of our article, variability was observed for each assay between the different studies. Because the samples were from different animals in different experiments, a kappa analysis was not performed. Instead, as described in the methods section, we used Fisher's exact test to identify statistically significant differences for each surrogate marker assay at the specified time points. No significant differences were found for the LFD at any time point between the studies that were conducted in different periods.

While we appreciate Dr. Sabour's comments, we are a bit perplexed by his statement regarding the inappropriate use of sensitivity as a measure of reliability in our study. While one of our objectives was to determine if the in vivo sensitivity of the LFD was affected by antifungal exposure, this was completely separate from the work that was undertaken to determine the interlaboratory and interstudy reproducibility of the results of this assay. Furthermore, and contrary to Dr. Sabour's inference, we did not use sensitivity as a measure of the reliability of the LFD. In fact, a separate animal experiment was conducted for this purpose. This question is of clinical importance, as the reduced sensitivity of surrogate marker assays may result in delays in the diagnosis of breakthrough infections, and previous studies have reported reduced sensitivity for the galactomannan and (1,3)-β-d-glucan assays in patients exposed to antifungals (3, 4). As we reported in Fig. 1 of our article, the sensitivity of the LFD within serum was reduced in animals exposed to antifungals. Because this also occurred with the galactomannan and (1,3)-β-d-glucan assays, these could not be used as the gold standards to compare against the LFD. However, numbers of CFU within lung homogenates can be used for this purpose, and as we show in our article, these values remained elevated in all animals treated with antifungals in that study. These results demonstrate, unequivocally, that the sensitivity of each surrogate marker assay within serum was reduced in animals treated with antifungals.

In summary, we stand by our statement that LFD testing is a reproducible assay. Moreover, additional scrutiny of our results using Dr. Sabour's recommendation serves to further strengthen our original conclusions.