Measuring Antimicrobial Efficacy against Biofilms: a Meta-analysis

Through a statistical meta-analysis of published data on antimicrobial efficacy against biofilms formed by two common bacterial species, it was concluded that the particular experimental method used is the most important factor determining the outcome of the test. An expected dose-response relationship (greater killing with higher doses or longer treatment times) was observed for data sets derived from a single method but was not observed when data from multiple studies using diverse methods were pooled.

ABSTRACT

Through a statistical meta-analysis of published data on antimicrobial efficacy against biofilms formed by two common bacterial species, it was concluded that the particular experimental method used is the most important factor determining the outcome of the test. An expected dose-response relationship (greater killing with higher doses or longer treatment times) was observed for data sets derived from a single method but was not observed when data from multiple studies using diverse methods were pooled. Method-specific properties such as the surface area/volume ratio, areal biofilm cell density, and microbial species were shown to influence quantitative measurements of biofilm killing. A better appreciation of the method characteristics that affect antibiofilm efficacy tests could aid decision-making related to investment in research and development and regulatory approvals for biofilm control strategies. The following recommendations are offered to those working in research and development related to biofilm control: (i) report the log reduction, surface area/volume ratio, and biofilm areal cell density; (ii) include data for a benchmark agent, making sure that this agent performs competitively at the dose tested; (iii) measure the dose-response relationship, i.e., make measurements at multiple treatment concentrations or dose durations; and (iv) use a standardized method in addition to research methods.

INTRODUCTION

Over the past several years, there has been a flood of literature reports of new antimicrobial chemistries and strategies with efficacy against biofilms. These are important contributions, because established biofilms are highly tolerant to conventional antimicrobial agents (1, 2) and there is pressing demand for better approaches to controlling detrimental biofilms in medicine, dentistry, industry, and consumer environments. Molecules with antibiofilm activity span the chemical and biochemical spectrum and include novel synthetics, plant-derived agents, antimicrobial peptides, and a wide variety of nanotechnologies, photoactivated compounds, quorum-sensing inhibitors, and other creative strategies. The examples are too numerous to cite completely; here we reference a few recent reviews (3–7) and a few primary research articles (8–12) to convey the possibilities and excitement generated by this work.

Which of these approaches should be invested in? How can claims of antimicrobial activity against biofilms be evaluated and compared? Reliable and well-characterized in vitro methods are a critical enabling technology for answering these questions (13–16). There are few standardized methods for evaluating biofilm susceptibility to antimicrobials, and it is not clear how the different methods in the literature compare or which factors govern outcomes. The purpose of the work reported in this article is to analyze literature data on antimicrobial efficacy against biofilms for patterns, to determine the characteristics of the methods being used that are most important for the test outcome, and to make suggestions and recommendations for researchers and developers turning to a biofilm antimicrobial test method.

The analysis pursued here was organized around the concept of an antimicrobial dose-response relationship (17, 18). In this simple version, an increase in biofilm killing (measured as a log reduction [LR] in viable cell counts) is anticipated when the dose of antimicrobial agent (calculated as the product of dose concentration and dose duration [Ct_d]) is increased.

RESULTS AND DISCUSSION

Antimicrobial dose-response relationships with biofilms are method dependent.

When biofilm killing (measured as a LR in viable cell counts) was plotted as a function of Ct_d, no coherent overall dose-response relationship was observed (Fig. 1A). The coefficient of determination (R²) for a least-squares linear regression of the data set graphed in Fig. 1A (both microorganisms and all antimicrobial agents) was 0.106. One way to interpret this value is that just 10.6% of the variation in LR could be attributed to the linear effect of the antimicrobial dose. Separating the data for the two microorganisms changed the correlations only slightly (Pseudomonas aeruginosa, R² = 0.0378; Staphylococcus aureus, R² = 0.2561) (Fig. 1B and C).

FIG 1.

Lack of overall dose-response relationship for biofilm killing by antimicrobial agents. (A) Combined data for P. aeruginosa and S. aureus. (B) Data for P. aeruginosa. (C) Data for S. aureus. Dashed lines connect data points from a single study.

The dose-response relationship is expected to be strongly influenced by the choice of antimicrobial agent, and it is reasonable to begin by inspecting the data for specific antimicrobial agents. Consider the data for quaternary ammonium compounds (QAC) for the P. aeruginosa data set (Fig. 1B, blue symbols). No overall trend is evident in the 9 data points from 4 studies (R² = 0.2957 [P = 0.1745]). Only when the data points from within an individual study (employing exactly the same method) are connected (Fig. 1B, dotted lines) does the expected dose-response relationship (larger LR with increasing dose) become visually evident. The two highlighted studies differ quantitatively, however; at an exposure of approximately 1,000 mg · min/liter, the study by Campanac et al. (19) found a reduction about 17 times larger than that in the study by Buckingham-Meyer et al. (20). Similar findings and a similar conclusion are arrived at by examining the S. aureus QAC data set over 5 studies (R² = 0.4197) (Fig. 1C, blue symbols). When data sets from a single study are inspected, they always display the expected dose-response behavior (for example, see the data for thyme oil against S. aureus biofilm in Fig. 1C [green symbols]). Unfortunately, when independent studies of the same agent are compared, quantitative results can vary widely. To underscore this discrepancy, consider two investigations of the action of peracetic acid against P. aeruginosa biofilms (Fig. 1B, magenta symbols). The two studies have internally consistent dose-response behaviors; they differ dramatically, however, in their quantitative determinations of efficacy. To achieve a 3-log-unit reduction, the study by Blanchard et al. (21) indicates an exposure of 16.5 mg · min/liter, whereas an exposure of 84,000 mg · min/liter is suggested by the data of Spoering and Lewis (22), which is a 5,000-fold difference (3 orders of magnitude).

What this qualitative analysis by inspection suggests is that the overwhelming factor determining a dose-response relationship for an antimicrobial biofilm test is the particular method used to make the measurements. This interpretation is borne out by statistical analyses of data for individual antimicrobial agents (Table 1). Within an individual study, LR correlates positively and strongly with Ct_d. When data from ≥2 studies are combined, the correlation usually vanishes. The loss of dose-response correlation when data from studies using different methods are combined is apparent in Fig. 2. We conclude that antimicrobial dose-response relationships with biofilms are highly method dependent.

TABLE 1.

Dose-response relationship statistics for antimicrobial agents tested against biofilms^a

Species and agent	Reference	Mb	Nc	Slope (LR/mg · min)	P	R2	AIC	SD(LR)d
P. aeruginosa
QAC	19	1	3	3.946	0.237	0.8676	11.9	1.12
Bismuth thiol	46	1	3	3.316	0.0599	0.9912	0.04	0.16
Peracetic acid	22	1	5	12.0	0.0009	0.9839	5	0.29
Peracetic acid	21	1	4	3.263	0.0692	0.8664	11.3	0.66
Chlorine dioxide	56	1	11	5.525	<0.0001	0.8712	33.9	0.95
Monochloramine	58	1	3	13.2	0.305	0.7881	13.5	1.46
Peracetic acid	22, 46	2	9	4.3	0.1969	0.000	37.7	11.4
Hydrogen peroxide	32, 34, 35, 54, 55	4	4	−0.3125	0.643	0.1278	9	0.51
Chlorine	20, 46, 55	4	7	1.9377	0.0049	0.8213	27	1.06
QAC	19, 20, 52	4	9	1.72	0.1745	0.2957	36	1.99
Fluoroquinolone	30–32, 46, 53	5	5	−0.375	0.718	0.0498	23	1.51
Aminoglycoside	30–32, 37, 46, 53	6	6	0.013	0.994	1.5 × 10−5	31	2.23
All	19–22, 30–32, 34, 35, 37, 46, 52–58	18	63	0.195	0.4867	0.0378	284	2.31
S. aureus
Thyme oil	42	1	8	1.7163	<0.0001	0.9873	−5.9	0.13
Bismuth thiol	46	1	3	3.897	0.117	0.9667	7.8	0.57
Biguanide/pyridine	40, 44, 45	3	3	0.1609	0.937	0.0098	15.8	2.15
Vancomycin	25, 33, 49	3	4	2.4771	0.1356	0.9237	13	0.94
Chlorine	20, 40, 41	4	6	0.0686	0.8765	0.000	13	0.55
QAC	19, 20, 36, 39, 40	5	8	2.280	0.0263	0.4197	23	1.85
All	19, 20, 25, 26, 33, 36, 38–51	21	55	0.7825	<0.0001	0.2561	200	1.48

^aThe statistics show strong correlations for data from a single study and mostly poor correlations when data from multiples studies are aggregated. This analysis used data sets when ≥3 data points from a single study, ≥5 points combined for 2 studies, or ≥3 points for ≥3 studies were available. Bold type indicates a negative slope for the regressed line, contrary to the expected positive dose-response relationship.

^bM indicates the number of studies; the study by Buckingham-Meyer et al. (20) counts as 2 (1 for the CDC biofilm reactor system and 1 for the drip flow biofilm reactor system).

^cN indicates the total number of data points.

^dSD(LR) indicates the study reproducibility SD of LRs for M values of >1; for single studies (M = 1), the column indicates only the within-study SD of the LR.

FIG 2.

Dose-response correlations for biofilm killing by specific antimicrobial agents diminish as data from multiple studies are aggregated.

Predictors influencing outcomes of antimicrobial biofilm tests.

We can gain some insight into the method features that are most important by briefly digressing to consider the methodological differences between the systems used by Blanchard et al. (21) and by Spoering and Lewis (22). Again, these studies both investigated the effects of peracetic acid on P. aeruginosa biofilms but arrived at quite different results. Blanchard et al. (21) used a modified Robbins device under continuous flow. Biofilm-covered coupons grown in this device were transferred to a clean device and treated with a once-through flow, which resulted in a surface area/volume ratio of approximately 0.002 cm⁻¹. Spoering and Lewis (22) employed the Calgary biofilm device (now marketed as the Innovotech minimum biofilm eradication concentration [MBEC] assay), in which biofilm grows on pegs immersed in the wells of a 96-well plate (23). This is a static system with a much higher surface area/volume ratio of approximately 2 cm⁻¹. The surface density of bacteria was approximately 25 times greater in the system used by Spoering and Lewis (22) (10⁸ CFU/cm²) than in the apparatus used by Blanchard et al. (21) (4 × 10⁶ CFU/cm²). The biofilms in these two systems were challenged at similar biofilm ages. This comparison suggests that flow conditions, biofilm areal density, or surface area/volume ratio might be critical parameters for an antimicrobial biofilm test.

We identified the following five model-dependent parameters that could influence the outcome of a biofilm antimicrobial efficacy test: microbial species, surface area/volume ratio, biofilm areal cell density, static or flow conditions, and biofilm age. The choice of antimicrobial agent also influences the outcome, as discussed above. Linear regression analyses were performed to assess the relative importance of these factors individually in determining the killing rate log₁₀(LR/Ct_d); results are tabulated in Table 2, with additional statistical results shown in Table S2 in the supplemental material.

TABLE 2.

Dependence of rates of biofilm killing [log₁₀(LR/Ct_d)] on surface area/volume ratio, microbial species, and biofilm areal cell density^a

Biofilm parameter	Combined			P. aeruginosa			S. aureus
	R	P	Slope	R	P	Slope	R	P	Slope
Species	0.568	0.0995	0.304
log10(A/V)	0.611	0.0002	−0.825	0.673	0.0081	−0.882	0.329	0.016	−0.754
log10(Xo)	0.560	0.4681	−0.091	0.627	0.4536	−0.186	0.304	0.053	−0.299
Flow vs static	0.569	0.0935	0.648	0.637	0.1503	0.966	0.246	0.951	0.026
Age	0.559	0.612	−0.001	0.624	0.7294	0.0011	0.253	0.462	−0.002
log10(XoA/V)	0.582	0.011	−0.277	0.672	0.0052	−0.562	0.335	0.016	−0.291

^aSee Table S2 in the supplemental material for additional statistical results of this analysis. For the categorical factor species, the difference in means between P. aeruginosa and S. aureus is reported. For the categorical factor flow versus static, the difference in means between flow and static conditions is reported.

The method parameter that most consistently and strongly influenced the rate of antimicrobial killing in a biofilm [log₁₀(LR/Ct_d)] was the surface area/volume ratio of the test system (for the data set with data for both species, R² = 0.611, Akaike information criterion [AIC] = 253 [P = 0.0002]) (Table 2). For the combined data set and also for the two species individually (P. aeruginosa, R² = 0.673, AIC = 127 [P = 0.0081]; S. aureus, R² = 0.329, AIC = 132 [P = 0.016]), the slopes of the regressed lines were close to −1, indicating a reciprocal dependence. In other words, doubling the surface area/volume ratio approximately halved the killing rate. The effects of flow rate and species were the next most important factors for the combined data set. Age and initial biofilm areal cell density did not have large effects on the killing rate for either species.

Although biofilm age did not emerge in this analysis as an influential predictor, multiple groups have reported clear effects of biofilm age within a given study (1). The absence of correlation with other method-dependent parameters noted does not prove that these parameters are not important. The inherent variability associated with diverse experimental methods makes it difficult to detect the contributions of individual method features. Further research to understand the additional factors that influence the antimicrobial susceptibility of biofilms is warranted. Some of the potentially important factors that come to mind include biofilm age, biofilm thickness (as a proxy for diffusive path length), growth rate, extracellular polymeric substances content, and temperature. The number of factors involved is a reminder to scientists working in the biofilm field to report as many of these factors as possible when describing their work (24).

For each of the P. aeruginosa and S. aureus data sets, we discovered that a combination of surface area/volume ratio and biofilm areal cell density, X_oA/V, which was termed biomass density, yielded the strongest effect. This relationship for P. aeruginosa is graphed in Fig. 3A (R² = 0.672, AIC = 132 [P = 0.005]); the relationship for S. aureus is graphed in Fig. 3B (R² = 0.335, AIC = 129 [P = 0.017]).

FIG 3.

(A) Rates of killing of P. aeruginosa biofilms depend on the biomass density, X_oA/V, in the test system. (B) Rates of killing of S. aureus biofilms depend on the surface area/volume ratio, A/V, in the test system.

Identifying superior antibiofilm agents.

We return now to the question of how to compare agents to identify the most effective antimicrobial for a biofilm application. It will be helpful to consider some specific papers to illuminate the issues.

Consider first the synthetic antistaphylococcal pyrrolomycins reported by Yang et al. (8). Biofilm killing was measured in a microtiter plate assay and compared to that observed with vancomycin, a well-known antibiotic. Killing was reported in the article as percent reduction in viable cell numbers. Because it is preferable to work in terms of LR, we made this conversion. A 48-h treatment of S. aureus biofilm with 32 μg/ml vancomycin (22 μM) produced a 0.2-log-unit reduction, whereas a 48-h exposure to 4 μg/ml pyrrolomycin 4 (11 μM) produced a 2.0-log-unit reduction. Thus, the new compound outperforms the benchmark antibiotic. Examining the dose-response relationship for vancomycin (Fig. 4), it can be seen that vancomycin is consistently ineffective (LR against biofilm cells of <0.5 log units) when relatively short exposures are used. Vancomycin works well when prolonged exposures are evaluated (25). In other words, outperforming vancomycin in the Ct_d range of 10⁴ to 10⁵ mg · min/liter is not a particularly impressive outcome. In a comparison of the performance of pyrrolomycin 4 to that of other antimicrobials, its efficacy was respectable but not outstanding (Fig. 4). This leads to the suggestion to choose a benchmark agent that exhibits competitive efficacy in the dose range of interest.

FIG 4.

Appropriate benchmark agents facilitate comparisons of antimicrobial activity against biofilms.

Another novel antibiofilm drug was reported by Morrow et al. (26). There the benchmark agent was the antibiotic ciprofloxacin, which falls in the middle of the group in the overall dose-response plot for S. aureus (Fig. 4), with a LR of 1.1 log units at a Ct_d value of 8,640 mg · min/liter. The new fluoroquinolone JNJ-Q2 clearly outperforms the benchmark at the same dose (Fig. 4), with a LR of 4.1 log units. The model used by Morrow et al. (26) was robust, in that it involved a 4-day-old biofilm, a high biofilm areal cell density, and a surface area/volume ratio above the median.

Next, we undertook to rank the potency of oxidizing biocides against P. aeruginosa biofilm. We took biomass density into account because we know this parameter is important. As displayed graphically in Fig. 5, biofilm killing rates were ranked in the following order: monochloramine > chlorine and chlorine dioxide > peracetic acid > hydrogen peroxide. Such a ranking of biofilm potency based on measurements from multiple investigations has not been reported previously.

FIG 5.

Relative potencies of oxidizing biocides against P. aeruginosa biofilms become apparent when biomass density is taken into account.

It is worth remembering that relative potency is by no means the only or most important property of an antimicrobial agent. In practical applications, cost, environmental impact, material compatibility, cytotoxicity, and safety are all important considerations in the selection of an antimicrobial agent.

One solution to the problem of comparative testing of antibiofilm agents would be for the research community to adopt a common method. Ideally, this common method would have undergone a rigorous standardization process, such as that afforded by the American Society for Testing Materials (ASTM). There are two candidate standard methods for measuring biofilm killing. In 2017, the U.S. Environmental Protection Agency (EPA) issued guidance for testing the efficacy of antimicrobial products against Pseudomonas aeruginosa or Staphylococcus aureus biofilms grown in the CDC biofilm reactor (27, 28). Mature biofilms grown in this high-shear system are used to test antimicrobials using the single-tube method (ASTM method E2871). By design, the single-tube method measures only the LR in viable biofilm bacteria; in other words, removal is not assessed. The second candidate method is the MBEC biofilm disinfection efficacy assay. Formerly known as the Calgary biofilm device, the MBEC system grows biofilm in a low-shear environment, on pegs molded to the top of a microtiter plate lid (23). The mature biofilm is then exposed simultaneously to multiple concentrations of biocide by placing the lid on a new microtiter plate bottom. The MBEC biofilm disinfection test has been standardized and validated as ASTM method E2799 (29). Both of these methods have been validated in interlaboratory studies.

In summary, we performed a meta-analysis of published data on antimicrobial efficacy against biofilms formed by two common bacterial species. We conclude that the particular experimental method used is the most important factor determining the outcome of the test and different methods can produce extremely different results even for the same antimicrobial agent. Method-specific properties such as the surface area/volume ratio, areal biofilm cell density, and microbial species can influence quantitative measurements of biofilm killing. The following recommendations are offered to those working in research and development related to biofilm control: (i) report the LR, surface area/volume ratio, and biofilm areal cell density; (ii) include data for a benchmark agent, making sure that this agent performs competitively at the dose tested; (iii) measure the dose-response relationship, i.e., make measurements at multiple treatment concentrations or dose durations; and (iv) use a standardized method in addition to research methods.

MATERIALS AND METHODS

Meta-analysis.

(i) Data compilation and exclusions. Quantitative data on biofilm killing by antimicrobial agents were gathered from 35 published reports (19–22, 25, 26, 30–58). These papers were identified by searching in the PubMed and Web of Science databases using keywords such as biofilm, antimicrobial, disinfectant, biocide, and log reduction. Approximately one-third of the hits initially identified as containing quantitative data on killing also contained sufficient documentation of other parameters. The following criteria were required for a study to be included in the data set: (i) single-species biofilm of either Pseudomonas aeruginosa or Staphylococcus aureus; (ii) biofilm killing reported as a quantitative LR in viable cell numbers (or LR was readily calculable); (iii) antimicrobial chemistry defined; (iv) quantitative dose concentration (C) and dose duration (t_d) reported; (v) biofilm areal cell density (X_o) reported for untreated control in CFU per square centimeter (or density was readily calculable); (vi) surface area/volume ratio (A/V) of the biofilm treatment system reported or calculable (specifically, the biofilm-covered area divided by the total volume of antimicrobial treatment solution applied); and (vii) biofilm age at the time of treatment reported. Data for any wild-type strain of P. aeruginosa or S. aureus were used. The data represent 13 different strains for P. aeruginosa and 16 different strains for S. aureus. Data from studies published until early 2017 were included. The resulting collection of data might not be comprehensive for the applied criteria. The data sets are tabulated in Table S1 in the supplemental material.

(ii) Statistical analyses. Least-squares linear regression was used to examine the data from each study separately for the presence of a dose-response relationship and for the influence of individual method-specific parameters on the rate of biofilm killing. This analysis assumed that the rate of decrease of viable cells was proportional to both the antimicrobial agent concentration and the concentration of viable cells, dX/dt = −kCX, where X denotes the biofilm areal cell density, t is time, k is a biofilm killing rate coefficient, and C is the antimicrobial concentration. The solution to this equation, when C is constant, is −ln(X/X_o) = 2.303LR = kCt_d, where X_o is the initial biofilm areal cell density and LR = log₁₀(X/X_o). This result predicts that LR should increase linearly with Ct_d. To analyze for this dependence, LR values were regressed against Ct_d values on a linear scale and the coefficient of determination (R²) was calculated.

To evaluate the dependence of biofilm killing on features of the method used, the specific rate of killing, calculated as log₁₀(LR/Ct_d), was regressed against the common logarithm of identified methodological parameters (e.g., surface area/volume ratio), which yielded the coefficient of determination (R²) and the slope. The slope of the log-log plot corresponds to the exponent in a power law dependence. For species and flow versus static conditions, an analysis of variance was fit.

Data across multiple studies were analyzed by linear mixed-effects models (LMMs). These LMMs included a single predictor (as outlined above) and a random effect for study, to estimate the variability of results across studies. All LMMs were fit using the software R (59), package lme4 (60). Model fits were assessed by residual plots. Conditional AIC values were used to compare different LMMs (61), with smaller values indicating better models. Also reported were the study reproducibility standard deviation (SD) (calculated as the square root of the sum of the among-study variance and the residual variance [16]) and the conditional R², which is a measure of the variability of the response explained by both the predictor and the random effect in the model (e.g., Ct_d) (62, 63).