Ranking Filter Methods for Concentrating Pathogens in Lake Water

LETTER

Accurately comparing filtration methods for concentrating waterborne pathogens is difficult because of two important water matrix effects on recovery measurements, the effect on PCR quantification and the effect on filter performance. Francy et al. (1) did not account for either effect, calling into question the validity of their filter comparisons. To account for the first effect, we quantify experimental seed concentrations in a negative final concentrate created by filtering unseeded water of the same source and volume as the recovery test, followed by all elution and secondary concentration steps, to create a matrix identical to the seeded test water (2, 3). This is necessary because matrix constituents ending up in the quantitative PCR (qPCR) can lower or raise quantification cycle (Cq) values (4, 5, 6, 7), biasing the absolute measure of target concentration. If concentrations of the recovered pathogen (the dividend of the percent recovery calculation) and pathogen seed (the divisor) are not measured in identical matrices, biases in measurement error are not identical and recovery is under- or overestimated (for example, see Fig. 1 in reference 15). Francy et al. did not account for this matrix effect for four test microorganisms. Moreover, without identical measurement matrices, qPCR inhibition levels could differ between recovered pathogen and seed measurements. For instance, compared to qPCR measurement of recovered pathogen levels in the final concentrate, measurement of pathogen seed in beef extract eluent was inhibited, and with an underestimated seed concentration as the divisor in the recovery calculation, this made filter performance appear better than the actual performance (2). Francy et al. allowed inhibition to shift Cq values for up to two cycles, which for a 100% efficient reaction means that seed or recovery quantities could be off by a factor of 4. Ten percent recovery could actually be 40% or vice versa.

Second, the water matrix can affect filter performance for both VIRADEL (adsorption/elution) and ultrafiltration methods for some microorganisms (2, 8, 9, 10, 11, 12, 13). This effect was not controlled statistically by Francy et al., and it appears that the effect was not removed experimentally by simultaneous side-by-side recovery trials of the five filters using the same water matrix. The numbers of recovery trials differed by filter type, replicate trials took several days to complete under differing lake conditions, and some recovery data represent a mix of seeded and naturally occurring microorganisms. Absent these controls, filter type and water matrix were confounded variables, making it impossible to determine whether differences in pathogen recoveries attributed to filters were, in fact, from differences in water matrices and making filter comparisons uninterpretable.

Lastly, Francy et al. developed the quantity RCV (rank × coefficient of variation) to identify the best filtration method. This is an ad hoc measure whose properties are not well described. Let CV_M and R_M be the coefficient of variation and recovery-based rank for filtration method M. The relationship in equation 1 must hold for two hypothetical filtration methods A and B to have the same RCV.

$$C{V}_A=\frac{{\mathit{R}}_B}{{\mathit{R}}_A}C{V}_B$$ (1)

The following can be inferred from equation 1:

1. The better the recovery-based rank, the more difficult it is to have a less-favorable RCV-based rank. For example, for filtration methods with recovery-based ranks of 1 and 2 to switch ranks to 2 and 1 based on RCV, the CV of the method with the highest recovery must be more than twice as large (R_B/R_A = 2/1) as the other method's CV, whereas for methods with recovery-based ranks of 4 and 5, only a 25% difference in CV (R_B/R_A = 5/4) is necessary to switch interpretations.

2. The relative ranking of two filtration methods as best or worst using the RCV quantity depends on their position in recovery-based rank (Table 1).

We believe interpretation of both the raw and ranked RCV values is problematic and that more established statistical methods should be used to evaluate filter performance.

Table 1.

Four comparisons of hypothetical filtration methods A and B with fixed median recovery and coefficient of variation and varying rank based on median recovery

Filtration method (fixed)	Median recovery (fixed)	CVa (fixed)	Rank based on median recovery (varying)	RCV	Best method (i.e., lower RCV)
A	20	0.30	2	0.60
B	68	0.45	1	0.45	B
A	20	0.30	3	0.90
B	68	0.45	2	0.90
A	20	0.30	4	1.20	A
B	68	0.45	3	1.35
A	20	0.30	5	1.50	A
B	68	0.45	4	1.80

^aCV, coefficient of variation.

The weight of evidence from numerous studies using VIRADEL and ultrafiltration methods suggests that all work reasonably well (14). Recoveries for a given method range widely and vary with sampling conditions, undermining any claim for “best method.” Cost, ease of use, requirements for recovery controls, and the capability to achieve study objectives are more appropriate considerations for method selection.