ABSTRACT
This study describes the results of a randomized clinical trial investigating the effect of oxytetracycline treatment dose and mode of administration on the selection of antibiotic-resistant coliform bacteria in fecal samples from nursery pigs. Nursery pigs (pigs of 4 to 7 weeks of age) in five pig herds were treated with oxytetracycline for Lawsonia intracellularis-induced diarrhea. Each group was randomly allocated to one of five treatment groups: oral flock treatment with a (i) high (20 mg/kg of body weight), (ii) medium (10 mg/kg), or (iii) low (5 mg/kg) dose, (iv) oral pen-wise (small-group) treatment (10 mg/kg), and (v) individual intramuscular injection treatment (10 mg/kg). All groups were treated once a day for 5 days. In all groups, treatment caused a rise in the numbers and proportions of tetracycline-resistant coliform bacteria right after treatment, followed by a significant drop by the time that the pigs left the nursery unit. The counts and proportions of tetracycline-resistant coliforms did not vary significantly between treatment groups, except immediately after treatment, when the highest treatment dose resulted in the highest number of resistant coliforms. A control group treated with tiamulin did not show significant changes in the numbers or proportions of tetracycline-resistant coliforms. Selection for tetracycline-resistant coliforms was significantly correlated to selection for ampicillin- and sulfonamide-resistant strains but not to selection for cefotaxime-resistant strains. In conclusion, the difference in the dose of oxytetracycline and the way in which the drug was applied did not cause significantly different levels of selection of tetracycline-resistant coliform bacteria under the conditions tested.
IMPORTANCE Antimicrobial resistance is a global threat to human health. Treatment of livestock with antimicrobials has a direct impact on this problem, and there is a need to improve the ways that we use antimicrobials in livestock production. We hypothesized that antibiotic resistance development following treatment of diarrhea in nursery pigs could be reduced either by lowering the dose of oxytetracycline or by replacing the commonly used practice of flock treatment with individual or small-group treatments, since this would reduce the number of pigs treated. However, the study showed no significant difference between treatment groups with respect to the number or proportion of tetracycline-resistant coliforms selected. The most important conclusion is that under practical field conditions, there will be no added value, in terms of lowering resistance development, by exchanging flock treatment for individual or small-group treatment of nursery pigs. The reason for the lack of an effect of single-animal treatment is probably that such animals share the environment with treated animals and take up resistant bacteria from the environment.
KEYWORDS: dose, flock treatment, nursery pigs, tetracyclines
INTRODUCTION
Antibiotic-resistant bacteria are a recognized threat to public health. They cause increased mortality from infectious diseases (1) and a higher cost of treatment due to the prolonged recovery time and the need for the use of more expensive antibiotics, and they increase the need for and the cost of biosecurity in hospitals (2). The same is true in veterinary medicine, where resistant bacteria increase the cost of treatment and may lead to animal welfare problems due to unsuccessful treatments (3, 4). For these reasons, it is important to reduce the factors that result in the selection of antibiotic-resistant bacteria as much as possible.
Antibiotic resistance in the animal sector can reach humans through the food chain, through the environment, and by direct and indirect contact with animals and animal products (5, 6). While antibiotic-resistant pathogenic bacteria are the immediate threat, antibiotic resistance in the commensal bacteria of food animals is considered a reservoir of antibiotic resistance genes that may aggravate the problem (7). For example, surveillance results show rates of tetracycline resistance of 36% in commensal Escherichia coli strains from pigs in Denmark (8). Thus, the minimization of resistance in the commensal flora of food animals may be important in order to reduce the risk to human health from the use of antibiotics in the livestock industry.
Enteric disease is very common in industrial pig production, especially during the nursery period (9). As a consequence, the highest single indication for treatment with antibiotics in the Danish livestock industry is diarrhea in pigs during the nursery period, and 42% of total antibiotic use for pigs in Denmark is for this indication, with tetracyclines being the drug class used the most often (8). In order to reduce the total amount of antibiotics used in the pig industry, it is important to find more intelligent ways to treat enteric diseases during the nursery period.
Treatment of diarrhea in nursery pigs is often carried out using oral flock treatment (also referred to here as a batch treatment), where a full section of pigs is treated with antibiotic in the feed or water when disease is seen in a prefixed proportion of the population. The justification for this approach is that apparently healthy animals in close proximity to diseased individuals are likely to be subclinically infected and will progress to develop clinical disease (10, 11). This batch treatment regimen exposes the commensal intestinal flora of all pigs to a selective pressure, which is presumed to increase the total amount of resistant bacteria on farms significantly compared to the amount obtained by treatment of individual pigs (12, 13). However, to the authors' knowledge, this has not been investigated under field conditions.
Apart from the treatment regimen (flock treatment versus individual treatment), the selection of antibiotic-resistant bacteria is influenced by factors such as the treatment dose (14, 15), the number of animals housed together (16), and other management factors (17–21). Among these factors, mathematical modeling suggests that dose may play a particularly important role in the selection of resistant coliform bacteria following tetracycline treatment (22). Such modeling predicts that the consumption of high doses of antibiotics is positively correlated with a subsequent high proportion of resistant fecal coliforms and with a longer time for the proportion of resistant bacteria to nonresistant bacteria to return to the pretreatment equilibrium.
The aim of the present study was to determine the effect of five different oxytetracycline (OTC) treatment regimens with various doses and various modes of treatment on the occurrence of antibiotic-resistant coliform bacteria in nursery pigs in a randomized clinical field trial.
RESULTS
Effect of treatment dose and treatment regimens with OTC on selection of tetracycline-resistant coliform bacteria.
In total, 224 pigs received a high dose (HD) as a flock treatment, 241 pigs received a normal dose (ND) as a flock treatment, and 224 pigs received a low dose (LD) as a flock treatment. A total of 241 pigs belonged to the pen-wise (PW) treatment group, and 221 pigs belonged to the individual intramuscular (i.m.) treatment group. In total, samples from 1,167 animals were analyzed (Table 1).
TABLE 1.
Farm | No. of pigs receiving the following treatment: |
|||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Oral batch treatment |
Oral PW treatment |
Individual i.m. injection treatment |
||||||||||||
HD |
ND |
LD |
Treated pigs |
Untreated pigs |
Treated pigs |
Untreated pigs |
||||||||
T1 to T3 | T4 | T1 to T3 | T4 | T1 to T3 | T4 | T1 to T3 | T4 | T1 to T3 | T4 | T1 to T3 | T4 | T1 to T3 | T4 | |
A | 45 | 19 | 46 | 17 | 45 | 20 | 40 | 20 | 21 | 6 | 24 | 10 | 21 | 6 |
B | 60 | 33 | 30 | 23 | 45 | 29 | 37 | 23 | 8 | 5 | 26 | 11 | 19 | 17 |
C | 44 | 18 | 59 | 30 | 44 | 12 | 43 | 20 | 1 | 1 | 22 | 11 | 18 | 11 |
D | 45 | 5 | 45 | 13 | 45 | 0 | 46 | 6 | 0 | 0 | 24 | 1 | 21 | 1 |
E | 46 | 0 | 61 | 1 | 45 | 0 | 38 | 0 | 7 | 0 | 28 | 0 | 18 | 0 |
Total | 240 | 75 | 241 | 84 | 224 | 61 | 204 | 69 | 37 | 12 | 124 | 33 | 97 | 35 |
T1 to T4 refer to the time points when samples were obtained. T1 was immediately before treatment, T2 was 2 days after the end of treatment, T3 corresponded to the time when the pigs left the nursery unit, and T4 was 1 to 7 days before slaughter.
The method used to count the bacteria consisted of culture on MacConkey agar with added tetracycline. In order to validate that this method distinguished between tetracycline-resistant and tetracycline-sensitive coliform bacteria, 49 coliform strains were plated on agar with and without antimicrobials. The number of CFU of cultures of sensitive strains was 7.0 ± 0.5 log10 units lower on plates containing tetracycline than on plates not containing antibiotic, and only one strain showed colonies. The corresponding value for resistant strains was a difference of 0.3 ± 0.5 log10 units, and all strains showed colonies (see Fig. S1 in the supplemental material).
Effect of OTC dose on selection of tetracycline-resistant coliform bacteria.
As can be seen from Fig. 1, the variation in the log10 number of CFU per gram of tetracycline-resistant coliforms between pigs was large in all groups at all time points. The average number of coliform bacteria and tetracycline-resistant coliform bacteria did not differ significantly between groups before the initiation of treatment (time point 1 [T1]) (Fig. 1 and S2). On average, pigs carried 6.0 ± 0.8 log10 CFU/g total coliform bacteria and 5.5 ± 0.9 log10 CFU/g tetracycline-resistant coliform bacteria at T1. Treatment irrespective of dose caused a significant rise in the number of tetracycline-resistant coliforms 2 days after the end of treatment (time point 2 [T2]), followed by a significant drop toward the time when pigs left the nursery unit (time point 3 [T3]) (paired one-sided t test, P < 0.0005). The rise from T1 to T2 was the highest in the HD group. In all three dose groups, the average log10 number of CFU per gram of tetracycline-resistant coliform bacteria at slaughter was significantly below the value at T1 (paired one-sided t test, P < 0.05). The proportions of tetracycline-resistant coliforms also increased significantly in all groups following treatment (paired one-sided t test, P < 0.005) but dropped to below the starting point at 1 to 7 days before slaughter (time point 4 [T4]) (Fig. 2). The difference between the proportions at T1 and T4, however, was not significant.
We analyzed the overall effect of the treatment dose on the change in the proportion of tetracycline-resistant coliforms between T1 and T3 using a mixed linear model. In this analysis, farm was included as a fixed effect and batch was included as a random effect. We found no significant effect of treatment dose. The only significant effect in the model was the random effect of batch.
Effect of treatment mode on selection of tetracycline-resistant coliform bacteria.
The use of the PW or i.m. treatment strategy with the aim to treat fewer pigs than the number treated during flock treatment did not significantly affect the number of tetracycline-resistant coliform bacteria selected or the proportion of resistant coliforms at different time points. As for oral batch treatment, the number and proportion of resistant bacteria at 1 to 7 days before slaughter (T4) were lower than those before treatment (Fig. 1 and 2). The only significant effect in the logistic model here, too, was the batch effect.
In both the PW and the i.m. treatment groups, some pigs did not receive treatment (n = 26 and n = 79, respectively) (Table 1). The mean log10 number of CFU per gram of tetracycline-resistant coliforms in these groups at T3 (5.0 log10 CFU/g and 5.2 log10 CFU/g, respectively) was lower than the mean log10 number of CFU per gram of tetracycline-resistant coliforms in the treated pigs (5.4 log10 CFU/g and 5.3 log10 CFU/g, respectively). The difference was significant in the PW treatment group but not the i.m. treatment group (two-sided t test, P = 0.01 and P = 0.39, respectively) (Fig. S2). At T4, there were no significant differences between treated and untreated pigs in the PW treatment group (P = 0.06).
Control treatment with tiamulin.
For animal welfare reasons, the clinical trial did not contain a nontreated control group. Instead, a control experiment in which pigs suffering from Lawsonia intracellularis-induced diarrhea were treated with an unrelated antibiotic, tiamulin, was conducted. As shown in Fig. 3, treatment with this drug did not result in a significant increase in the number of tetracycline-resistant coliforms. Similarly, the proportion of tetracycline-resistant coliform bacteria did not change as a result of treatment. This showed that the effects seen after OTC treatment were specifically related to the use of this drug and did not represent normal development in the coliform flora of nursery pigs.
Coselection for other antibiotics.
In all treatment groups, there were no significant differences in the number of ampicillin-, sulfonamide-, and cefotaxime-resistant coliforms before the initiation of treatment (data not shown). The counts showed a close, highly significant correlation between the changes in the proportion of tetracycline-resistant coliforms between T1 and T2 and the changes in the proportion of ampicillin- and sulfonamide-resistant coliforms between the same time points (Pearson's product moment correlation coefficient, P < 0.0001), indicating that resistance to these antibiotics was selected together. On the contrary, no significant correlation between tetracycline- and cefotaxime-resistant coliforms was observed (data not shown). Nevertheless, 282 out of the 1,167 pigs analyzed were found to carry cefotaxime-resistant coliforms at T1 (average for positive pigs, 3.2 log10 CFU/g; range, 2.7 [detection limit] to 7.0 log10 CFU/g), and at least one pig on all farms was positive for cefotaxime-resistant coliforms.
DISCUSSION
The purpose of this study was to estimate the effect of the OTC treatment dose and the treatment regimens on the selection of tetracycline-resistant coliforms in nursery pigs under field conditions. We used an easy agar dilution counting method, based on the inclusion of breakpoint concentrations of OTC in MacConkey agar plates. This method has previously been validated for use with MacConkey agar and added tetracycline (23), with 8 μg/ml being the concentration of tetracycline added. We performed our own validation of the method with 16 μg/ml tetracycline added to the plates and found that this, too, gave a 100% ability to distinguish between tetracycline-sensitive and -resistant coliforms.
In accordance with the findings of a previous study (14), we observed a significantly higher number of tetracycline-resistant E. coli bacteria right after treatment in the group receiving the highest dose, but in contrast to the findings described in the previous publication, the concentration and proportion returned to the starting levels within 3 to 4 weeks. The proportions of resistant coliforms at T4, corresponding to the time shortly before slaughter and, thus, the time when the pigs entered the food chain, was significantly below the level before treatment. Thus, pigs receiving a high dose of tetracycline may shortly show a higher level of resistant bacteria, but according to our results, they do not possess a higher risk of transfer of resistant bacteria to consumers.
Reports on the proportion of tetracycline-resistant bacteria among E. coli isolates randomly collected from pigs in Denmark have been published since the 1970s (24). Comparison of the findings of these old studies and the results of the current surveillance program in Denmark (25) shows that the mean proportion of tetracycline-resistant commensal E. coli bacteria has varied over the years; however, it never seems to climb above approximately 40%. A possible reason for the minimal selective effect of dose in our study may have been the very high starting concentrations of resistant bacteria. While this is representative of the proportions of tetracycline-resistant commensal E. coli bacteria in pigs in Denmark (25), it is much higher than the 1 to 10% chlortetracycline-resistant E. coli strains detected by Delsol et al. (14) prior to their experiment. Our results may thus not be representative of those for farms with an initial lower concentration of tetracycline-resistant bacteria. Compared to the number of pigs included in the previously published studies, a high number of pigs was included in the present study, and the conclusions must be considered strong. Still, the trials were conducted in only five different herds for which management practices were quite similar. We cannot rule out the possibility that with very different management practices the results would have been different.
Previous studies on the effect of dose on the selection of resistance have generally been concerned with differences between the therapeutic and subtherapeutic concentrations of antibiotics (see the results of a meta-analysis in reference 26). In contrast, we considered therapeutic doses. By putting the results together and including the results of studies with poultry as well, there seems to be a minimal effect of treatment dose on the selection of resistant indicator bacteria (26, 27). This indicates that within quite broad ranges, veterinarians might change the dose to achieve a better treatment efficacy, without significantly changing the level of selection of resistant bacteria. It should be noted that while a dose of 5 mg/kg of body weight, corresponding to the low dose used in the current study, is sufficient to reduce the amount of L. intracellularis bacteria to below the threshold for pathological changes in the intestine of pigs, it takes 10 mg/kg to eliminate the bacterium to nondetectable levels (28).
On a population level, there is a direct association between the intensity of use of antibiotics and the proportion of bacteria resistant to such antibiotics. This has been demonstrated for clinical as well as indicator bacteria and for both humans (29, 30) and farm animals (31), though the relation is not always straightforward (32). As a consequence, there is a tendency to argue against flock treatment of farm animals. A large proportion of the reduction in the amount of antimicrobials used to treat farm animals in the Netherlands has been reported to be due to the restricted use of flock treatment (33), and legal restrictions specifying certain preconditions on the use of flock medication have been gradually introduced in Denmark. Although the phasing out of oral flock treatment leads to less antibiotic usage, it has never been thoroughly investigated whether this also leads to less resistance under field conditions, where untreated animals are housed in close proximity to treated animals, and we tried to answer this question in the current study.
Surprisingly, we did not observe any significant differences in the selection of tetracycline-resistant coliform bacteria when we compared oral flock treatment to oral pen-wise (small-group) and single-animal i.m. treatments. This is difficult to explain, given that the overall levels of use of OTC were 15% and 44% lower in the PW and i.m. treatment groups, respectively, than in the oral flock treatment group. A detailed analysis of our results showed that untreated pigs in the PW treatment group but not untreated pigs in the i.m. treatment group had significantly lower counts of tetracycline-resistant coliforms than the treated pigs in the same groups. The most likely explanation for the lack of a difference in the individual treatment group is that they shared the environment (the pen) with treated pigs and thus were exposed to a high number of tetracycline-resistant coliforms that were excreted from treated pigs. Contrary to this, untreated pigs in the PW group always shared the pen with untreated pigs.
The lack of an overall difference between the PW and ND treatment groups, we believe, is simply a matter of numbers. The vast majority of pigs in the PW treatment group were treated, because the pen fulfilled the criterion for treatment of diarrhea. In that respect, our study confirms previous observations that once diarrhea is observed in a fraction of the nursery pigs, there is a high risk that the remaining pigs are subclinically infected (10). Taken together, our results nevertheless indicated that a form of treatment where treated pigs are separated from untreated pigs might be a better strategy for reducing antimicrobial resistance than individual treatment when treated and untreated pigs share the same pen. PW and i.m. treatments with OTC have been shown to be ineffective compared to flock treatment for the treatment of L. intracellularis diarrhea (11). When this observation is combined with our results, continued use of oral flock treatment seems justified, at least when the conditions are similar to those investigated in the current study. In the study of treatment efficacy (11), the authors argued that oral flock treatment may be needed as long as there are no good, rapid, and precise diagnostic methods for the detection of individual pigs with an intestinal disturbance, since pigs with an intestinal disturbance may go unnoticed with current diagnostic procedures. This puts an emphasis on improved diagnostics corresponding well to the WHO action plan against antimicrobial resistance, which emphasizes the need for the development of improved diagnostic tests in the fight against antibiotic resistance (34). The results of the current study might also indicate that measurement of antibiotic consumption is not always a good surrogate for measurement of antimicrobial resistance, even though this is currently one of the cornerstones in national surveillance programs on antibiotic resistance.
For animal welfare reasons, we could not leave pigs untreated when outbreaks of diarrhea were present. To be able to control for natural development in the coliform flora, we chose instead to treat a batch with tiamulin, a drug belonging to the groups of pleuromutilins and used exclusively in veterinary medicine. As this drug does not select for tetracycline-resistant coliforms, this group could be used to create a baseline for the natural fluctuation in the numbers and proportions of tetracycline-resistant coliforms in nursery pigs. The results showed that the fluctuations that we observed in tetracycline-treated pigs in the clinical trials were associated with the OCT treatment and were different from the fluctuations in pigs treated with tiamulin.
Langlois et al. (35) showed that pigs in herds with a history of routine use of antibiotics developed higher numbers of tetracycline-resistant coliforms following chlortetracycline treatment than pigs from another herd without such a history. During and before the current clinical trial, farmers were allowed to treat pigs for other diseases, when needed. Treatments between birth and T1 may very well influence selection between T1 and T2 by having preselected for tetracycline-resistant coliforms. However, we systematically collected data on the consumption of antibiotics and analyzed for the effect of treatment with other antibiotics before and after our treatment on the selection of tetracycline-resistant coliforms. The results showed no significant effect of the three most common additional treatments, and we ruled out the possibility that additional treatments were a confounding factor. After time point T3, pigs were distributed to different fattening units, and only a fraction of the pigs were resampled at T4. No records on antibiotic use covering the time periods from birth to T1 and between T3 and T4 were available to us. We cannot rule out the possibility that treatment between T3 and T4 may be the reason for the lack of differences between the groups at T4. However, in general, the number of treatments during the fattening period is far below the number during the nursery period in Danish pig production (8), making this less critical for the current study.
The fact that flock and pen-wise (small-group) treatments were carried out through the administration of the medication in the drinking water introduced an uncertainty with regard to the dose obtained by individual pigs. We ensured that the dose given to the flock and the pen was consumed (in total), but we could not ensure that all pigs received equal treatments. This means that the dose in the groups receiving flock and pen-wise treatments was an average for the pigs and there was variation between pigs. Similarly, treatments were initiated (T1) when the clinical inclusion criterion was fulfilled, while T3 (the end of the nursery period) was a fixed date for each pig. This introduced variation in the duration of the period between T1 and T3, and this, too, may be a factor in the lack of significant differences between treatments. On the other hand, this is the situation in real life, and our results represent the naturally occurring variation in dosing and treatment time under field conditions.
Besides being tetracycline resistant, commensal E. coli bacteria from food animals in Denmark are commonly resistant to ampicillin and sulfonamides (8), indicating coselection, and a study from the United States indicated that tetracycline treatment of calves could lead to coselection for resistance genes encoding 3rd- and 4th-generation cephalosporin resistance (36). It has been reported that commensal tetracycline-resistant E. coli bacteria are often resistant to ampicillin and, further, that they may carry class 1 integrons encoding sulfonamide resistance genes (37). To test whether tetracycline treatment resulted in a specific increase in the amounts of coliforms with other resistance markers, all samples were also cultured on MacConkey agar containing ampicillin, sulfonamide, or cefotaxime. Cefotaxime was included to investigate the possible selection of extended-spectrum beta-lactamase (ESBL)-producing bacteria, which constitute a growing health concern (38).
In the current study, selection of tetracycline-resistant coliforms from T1 to T2 was significantly associated with selection for ampicillin- and sulfonamide-resistant coliforms. Since we have not characterized the bacteria counted in the current study, we cannot prove that this coselection is caused by colocalization of the resistance genes, but the observation is hard to explain by any other mechanism. One of the most prominent antibiotic resistance-related threats to human health is the growing prevalence of ESBL-producing Gram-negative bacteria (39). In the current study, we found that ESBL-producing coliforms could be identified on all farms, and on average, approximately 20% of the pigs were shown to be carriers. However, there is currently no indication that pigs are an important reservoir for ESBL infection in humans in Denmark (8), and on the basis of our results, the use of tetracycline as a factor for the (co)selection of such bacteria can be ruled out. The Danish pig industry does not use cephalosporin drugs, and due to this, the prevalence of ESBL-producing coliforms has decreased rapidly in recent years (31).
Several studies modeling the development of tetracycline resistance in pigs following treatment under different scenarios have recently been published (22, 40–42). Such models have been fed with data on the growth responses of E. coli to different concentrations of tetracycline. In relation to our study, the multistrain, multipig model by Græsbøll et al. (22) is the most relevant. This model predicts that a high dose will result in a higher proportion of tetracycline-resistant bacteria than a low dose. In that sense, the results of our field study are in agreement with the results of the model. However, the modeling also predicts that the proportion will return to the pretreatment level in a dose-dependent manner. This prediction was not confirmed by our field study. At T3 there was no significant difference between treatment groups.
The measurement of resistance in coliform bacteria is a widely used method in studies of the development of antibiotic resistance in bacterial populations, both in society in general and in intervention studies (43), but it is a narrow approach. Therefore, follow-up studies where one looks at the changes in the microbiome in general are indicated, since coliform bacteria are not the only risk factor for the transfer of resistance genes to human-pathogenic bacteria through the food chain. Such studies should preferably be carried out using culture-independent techniques.
In conclusion, the current study showed that the dose of oxytetracycline used during flock treatment and the mode of application do not have a significant influence on the selection of coliform bacteria in the intestines of nursery pigs under the conditions tested. This means that doses can be set by emphasizing efficacy and the price of treatment and that, from an antibiotic resistance point of view, there appear to be no benefits from the use of treatment of single animals, unless the treated animals are separated from nontreated pen mates.
MATERIALS AND METHODS
Clinical field trial.
The setup for the randomized clinical field trial has previously been described in two studies measuring the efficacy of various OTC treatment doses and treatment regimens (administration routes) for Lawsonia intracellularis diarrhea (11, 28), and the reader is referred to those two studies for a comprehensive description and for calculation of the sample size. In brief, five herds with a history of L. intracellularis-induced diarrhea were preselected. Each herd had between 2,300 and 3,600 pen places and an all-in all-out method of batch production in sectioned compartments. The flooring consisted of 1/3 solid floor and 2/3 slatted floor. In each herd, 15 batches were included in the study after the pigs were weaned. When clinical signs of diarrhea appeared, the pigs were treated as described below and followed until at least the end of the 7-week nursery period. Where possible, pigs were also resampled in the week prior to slaughter. A batch was defined as a group of nursery pigs weaned at the same time and housed in a number of pens within one stable. In each batch, 15 animals, randomly distributed over pens, were selected as trial pigs. The allocated treatment regimen, however, was applied to all pigs in the section as previously described (28). All trial pigs were ear tagged with a unique identifier.
When a new batch was weaned, it was monitored once a week for an outbreak of diarrhea. When an outbreak, defined as at least 25% of pigs showing clinical signs of enteritis (watery feces, scouring of the back, and/or a poor body score), was detected, pigs were subjected to one of five treatment regimens: oral flock treatment with a standard dose of 10 mg/kg OTC (Terramycin vet, 20%; Orion Pharma) in water for 5 days (normal dose [ND]), oral flock treatment with 20 mg/kg OTC in water for 5 days (high dose [HD]), oral flock treatment with 5 mg/kg OTC in water for 5 days (low dose [LD]), oral pen-wise (small-group) treatment with a standard dose of 10 mg/kg OTC for 5 days in water (PW), or individual intramuscular (i.m.) treatment of pigs with diarrhea with a standard dose of 10 mg/kg OTC for 5 days. Pen-wise treatment was initiated when more than 25% of the pigs in a pen had clinical signs of enteritis, while intramuscular treatment was initiated in animals showing clinical signs of enteritis. Flock treatment was administered to the pigs through the common water supply, whereas pen-wise treatment was administered to the pigs through the water in troughs, but the pigs also had access to medicine-free water through the common water supply. Each treatment was repeated three times in each herd, and the order of the treatments was chosen at random. The number of pigs from each farm included in the different groups can be seen in Table 1. Outbreaks of diarrhea and, thus, initiation of treatment occurred from 2 to 6 weeks after weaning.
In order to be able to estimate the selection of tetracycline-resistant coliform bacteria in pigs not exposed to tetracycline treatment, 25 pigs in one additional batch from the herd on farm A suffering from an outbreak of diarrhea were treated by oral flock treatment with a standard dose (8 mg/kg) of tiamulin (Denagard vet; Novartis, Copenhagen, Denmark) for 3 days.
All pigs in the trial received 2,500 ppm zinc oxide supplement in the feed for the first 14 days after weaning. Farmers were asked to keep a record of all antibiotic treatments carried out in the herd before and during the field trial. This allowed the investigators to control for confounding due to additional antibiotic treatments. A total of 889 pigs received antibiotic treatment before T1, and 402 pigs received treatment during the trial period between T2 and T3 (see Table S1 in the supplemental material). The treatments were farm specific: at one farm, pigs very rarely received additional treatments either before or after the study treatment protocol. On three of the farms, the farmer regularly treated the pigs with colistin shortly after they entered the nursery unit, i.e., shortly before the trial period. On two of these three farms, treatments other than this were rare, while the remaining farmer additionally treated some pigs with doxycycline between T2 and T3. Finally, on one farm, pigs were often treated with amoxicillin before T1 but received no other treatments between T2 and T3. Antibiotic treatments between T3 and T4 were not consistently recorded and were thus not taken into account in the analyses. When analyzing for the effect of treatments with antibiotics before and after our treatments, there were no significant effects of the three additional treatments administered to the largest number of pigs (colistin treatment before T1, amoxicillin treatment before T1, and doxycycline treatment after T2) on the absolute number of tetracycline-resistant coliform bacteria, the proportion of tetracycline-resistant coliforms, or the change in the proportion of tetracycline-resistant coliforms, and we concluded that these treatments were not confounders in our study.
Sampling.
Fecal samples were collected from all trial pigs between October 2011 and April 2013 either at defecation or per rectum. Samples were collected from all pigs at three time points: time point 1 (T1) was the first day of treatment, immediately before antibiotic administration, time point 2 (T2) was 2 days after the end of the 5-day treatment, and time point 3 (T3) was the time when pigs were moved from the nursery stables to finisher stables either in the same herd or in other herds. When possible (n = 296), a fourth sample was collected from the rectum 1 to 7 days before slaughter (time point 4 [T4]). Samples were stored in 40-ml containers and shipped to the laboratory in cooled boxes.
Bacterial quantification.
Suspensions of 10−1 (wt/vol) from approximately 1 g of fecal sample were made in phosphate-buffered saline (PBS), and 1 ml of this suspension was used for the preparation of 10-fold serial dilutions from 10−2 to 10−4. Twenty microliters of each dilution was plated on four MacConkey agar plates (Oxoid Ltd., Thermo Scientific, Roskilde, Denmark) containing different antibiotics (16 mg/liter tetracycline, 16 mg/liter ampicillin, 256 mg/liter sulfamethizole, or 2 mg/liter cefotaxime) and on a MacConkey agar plate without added antibiotics, using the principle of the drop plate method (44) with a 4-by-4 grid. Antibiotics were purchased from Sigma (Sigma-Aldrich, Copenhagen, Denmark). Antibiotic concentrations were based on EUCAST epidemiological cutoffs for E. coli, as recommended previously (45).
The plates were incubated overnight at 37°C, followed by enumeration of dark red colonies with a size of >0.5 mm. To confirm that the colonies counted were coliforms, 100 colonies were randomly picked and subjected to species identification using matrix-assisted laser desorption ionization–time of flight mass spectrometry (MALDI-TOF MS; Vitek MS RUO; bioMérieux, France). All colonies were shown to belong to the species E. coli (data not shown). For each plate, a count, expressed as the number of CFU per gram, was determined using a weighed arithmetic mean based on the two highest dilutions showing the separation between colonies, and finally, the number of CFU per gram was log10 transformed. The detection limit for the method used was 500 CFU per gram of feces, corresponding to a log10 value of 2.70.
In order to validate that this method distinguished between tetracycline-resistant and -susceptible isolates, a representative collection of commensal E. coli isolates from Danish pigs previously used to model the growth response of E. coli to antimicrobials (46) was tested. The collection consisted of 32 isolates with MICs of between 0.24 μg/ml and 2.0 μg/ml (sensitive isolates) and 16 isolates with MICs of between 16 and 512 μg/ml (resistant isolates). They were grown in LB broth (Oxoid Ltd., Thermo Scientific, Roskilde, Denmark) at 37°C overnight. Tenfold dilutions were made in PBS, and dilutions were plated on MacConkey agar without tetracycline and MacConkey agar containing 16 μg/ml tetracycline. The number of CFU was counted after 20 h of incubation at 37°C, and the difference between the estimates of the number of CFU on the two plates was determined for each strain.
Statistics.
The clinical trial was set up as a five-treatment trial, and statistical analysis of the differences between groups with respect to selection for resistant coliform bacteria was therefore carried out with all groups in one analysis. The effects of the different treatment protocols on the number of antimicrobial-resistant bacteria were analyzed either by using log10-transformed counts of resistant bacteria or by testing for significant changes in the square root of the proportion or change of proportion of resistant bacteria, i.e., (RTx/CTx) or (RTx/CTx)/(RTy/CTy), where R is the count (in number of CFU per gram) on the antibiotic plate at time Tx or Ty and C is the total coliforms (in number of CFU) at time Tx or Ty. Due to the uncertainty over the CFU counts, proportions could be greater than 1; however, proportions greater than 2 were considered outliers and excluded. The square root transformation was selected to improve the normality of the residuals of the tests. Pigs with dropout data (data missing at any of the time points from T1 to T3) were removed from the study, while the dropout of data for T4 had to be accepted because only a small fraction of pigs was available for sampling at T4.
Analyses were performed by the use of linear mixed-effects models to determine significant differences in resistant coliform bacteria and the fraction of resistant bacteria from T1 to T3 using lmer from the package lme4 in R, version 3.2.2 (47). When testing for the effect of treatment, farm identifier and the interaction between farms and treatment were included as fixed effects, while the batch of pigs was included as a random effect. To identify the significant effects, back-wise elimination was performed using the step function and the Akaike information criterion (AIC). Confidence intervals (CI) were found by bootstrapping using bootMer from the lmer test package. A test for differences between multiple groups at single time points was done using the Kruskal-Wallis rank sum test (kruskal.test), while the test for differences in the numbers or proportions of resistant bacteria between different time points within a group was done using Student's t test (t.test), and correlations were tested using Pearson's product moment correlation coefficient (cor.test), all of which were done in R (47).
Ethical statement.
The clinical trial was approved by the Danish Medicines Agency (license no. 2011090862/2012053751), and the participating herd owners signed a written owner informed consent explaining the scope of the field trial.
Supplementary Material
ACKNOWLEDGMENTS
This study would not have been possible without the constructive collaboration from the owners of and workers with the five participating pig herds. We acknowledge this valuable contribution. Pia Rønnow Mortensen and Tony Bønnelycke are thanked for skillful technical assistance.
The study was supported by a grant from the Danish Innovation Fund (grant number 0603-00385B).
Footnotes
Supplemental material for this article may be found at https://doi.org/10.1128/AEM.00538-17.
REFERENCES
- 1.Ferri M, Ranucci E, Romagnoli P, Giaccone V. 13 October 2015. Antimicrobial resistance: a global emerging threat to public health systems. Crit Rev Food Sci Nutr doi: 10.1080/10408398.2015.1077192. [DOI] [PubMed] [Google Scholar]
- 2.Smith R, Coast J. 2013. The true cost of antimicrobial resistance. BMJ 346:f1493. doi: 10.1136/bmj.f1493. [DOI] [PubMed] [Google Scholar]
- 3.Vallat B. 2012. Antimicrobial resistance in animal and public health. Preface. Rev Sci Tech 31:9–14. [DOI] [PubMed] [Google Scholar]
- 4.Bengtsson B, Greko C. 2014. Antibiotic resistance—consequences for animal health, welfare, and food production. Ups J Med Sci 119:96–102. doi: 10.3109/03009734.2014.901445. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gilchrist MJ, Greko C, Wallinga DB, Beran GW, Riley DG, Thorne PS. 2007. The potential role of concentrated animal feeding operations in infectious disease epidemics and antibiotic resistance. Environ Health Perspect 115:313–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Spellberg B, Guidos R, Gilbert D, Bradley J, Boucher HW, Scheld WM, Bartlett JG, Edwards J Jr, Infectious Diseases Society of America. 2008. The epidemic of antibiotic-resistant infections: a call to action for the medical community from the Infectious Diseases Society of America. Clin Infect Dis 46:155–164. doi: 10.1086/524891. [DOI] [PubMed] [Google Scholar]
- 7.Smith DL, Harris AD, Johnson JA, Silbergeld EK, Morris JG Jr. 2002. Animal antibiotic use has an early but important impact on the emergence of antibiotic resistance in human commensal bacteria. Proc Natl Acad Sci U S A 99:6434–6439. doi: 10.1073/pnas.082188899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.The Danish Integrated Antimicrobial Resistance Monitoring and Research Programme. 2015. DANMAP 2014—use of antimicrobial agents and occurrence of antimicrobial resistance in bacteria from food animals, food and humans in Denmark. The Danish Integrated Antimicrobial Resistance Monitoring and Research Programme, Copenhagen, Denmark. [Google Scholar]
- 9.Laine TM, Lyytikainen T, Yliaho M, Anttila M. 2008. Risk factors for post-weaning diarrhoea on piglet producing farms in Finland. Acta Vet Scand 50:21. doi: 10.1186/1751-0147-50-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Weber N, Nielsen JP, Jakobsen AS, Pedersen LL, Hansen CF, Pedersen KS. 2015. Occurrence of diarrhoea and intestinal pathogens in non-medicated nursery pigs. Acta Vet Scand 57:64. doi: 10.1186/s13028-015-0156-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Larsen I, Nielsen SS, Olsen JE, Nielsen JP. 2016. The efficacy of oxytetracycline treatment at batch, pen and individual level on Lawsonia intracellularis infection in nursery pigs in a randomised clinical trial. Prev Vet Med 124:25–33. doi: 10.1016/j.prevetmed.2015.12.018. [DOI] [PubMed] [Google Scholar]
- 12.Salyers AA, Amabile-Cuevas CF. 1997. Why are antibiotic resistance genes so resistant to elimination? Antimicrob Agents Chemother 41:2321–2325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stanton TB, Humphrey SB. 2011. Persistence of antibiotic resistance: evaluation of a probiotic approach using antibiotic-sensitive Megasphaera elsdenii strains to prevent colonization of swine by antibiotic-resistant strains. Appl Environ Microbiol 77:7158–7166. doi: 10.1128/AEM.00647-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Delsol AA, Anjum M, Woodward MJ, Sunderland J, Roe JM. 2003. The effect of chlortetracycline treatment and its subsequent withdrawal on multi-resistant Salmonella enterica serovar Typhimurium DT104 and commensal Escherichia coli in the pig. J Appl Microbiol 95:1226–1234. doi: 10.1046/j.1365-2672.2003.02088.x. [DOI] [PubMed] [Google Scholar]
- 15.Wagner BA, Straw BE, Fedorka-Cray PJ, Dargatz DA. 2008. Effect of antimicrobial dosage regimen on Salmonella and Escherichia coli isolates from feeder swine. Appl Environ Microbiol 74:1731–1739. doi: 10.1128/AEM.01132-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rosengren LB, Waldner CL, Reid-Smith RJ, Dowling PM, Harding JC. 2007. Associations between feed and water antimicrobial use in farrow-to-finish swine herds and antimicrobial resistance of fecal Escherichia coli from grow-finish pigs. Microb Drug Resist 13:261–269. doi: 10.1089/mdr.2007.781. [DOI] [PubMed] [Google Scholar]
- 17.Dunlop RH, McEwen SA, Meek AH, Black WD, Clarke RC, Friendship RM. 1998. Individual and group antimicrobial usage rates on 34 farrow-to-finish swine farms in Ontario, Canada. Prev Vet Med 34:247–264. doi: 10.1016/S0167-5877(97)00093-7. [DOI] [PubMed] [Google Scholar]
- 18.Dunlop RH, McEwen SA, Meek AH, Black WD, Friendship RM, Clarke RC. 1998. Prevalences of resistance to seven antimicrobials among fecal Escherichia coli of swine on thirty-four farrow-to-finish farms in Ontario, Canada. Prev Vet Med 34:265–282. doi: 10.1016/S0167-5877(97)00094-9. [DOI] [PubMed] [Google Scholar]
- 19.Akwar HT, Poppe C, Wilson J, Reid-Smith RJ, Dyck M, Waddington J, Shang D, McEwen SA. 2008. Associations of antimicrobial uses with antimicrobial resistance of fecal Escherichia coli from pigs on 47 farrow-to-finish farms in Ontario and British Columbia. Can J Vet Res 72:202–210. [PMC free article] [PubMed] [Google Scholar]
- 20.Akwar HT, Poppe C, Wilson J, Reid-Smith RJ, Dyck M, Waddington J, Shang D, McEwen SA. 2008. Prevalence and patterns of antimicrobial resistance of fecal Escherichia coil among pigs on 47 farrow-to-finish farms with different in-feed medication policies in Ontario and British Columbia. Can J Vet Res 72:195–201. [PMC free article] [PubMed] [Google Scholar]
- 21.Taylor NM, Clifton-Hadley FA, Wales AD, Ridley A, Davies RH. 2009. Farm-level risk factors for fluoroquinolone resistance in E. coli and thermophilic Campylobacter spp. on finisher pig farms. Epidemiol Infect 137:1121–1134. doi: 10.1017/S0950268808001854. [DOI] [PubMed] [Google Scholar]
- 22.Græsbøll K, Nielsen SS, Toft N, Christiansen LE. 2014. How fitness reduced, antimicrobial resistant bacteria survive and spread: a multiple pig-multiple bacterial strain model. PLoS One 9:e100458. doi: 10.1371/journal.pone.0100458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Vieira AR, Wu S, Jensen LB, Dalsgaard A, Houe H, Wegener HC, Lo Fo Wong DM, Emborg HD. 2008. Using data on resistance prevalence per sample in the surveillance of antimicrobial resistance. J Antimicrob Chemother 62:535–538. doi: 10.1093/jac/dkn210. [DOI] [PubMed] [Google Scholar]
- 24.Aalbaek B, Rasmussen J, Nielsen B, Olsen JE. 1991. Prevalence of antibiotic-resistant Escherichia coli in Danish pigs and cattle. APMIS 99:1103–1110. doi: 10.1111/j.1699-0463.1991.tb01307.x. [DOI] [PubMed] [Google Scholar]
- 25.The Danish Integrated Antimicrobial Resistance Monitoring and Research Programme. 2010. DANMAP 2010—consumption of antimicrobial agents and antimicrobial resistance in bacteria from food animals, food and humans in Denmark. The Danish Integrated Antimicrobial Resistance Monitoring and Research Programme, Copenhagen, Denmark. [Google Scholar]
- 26.Burow E, Simoneit C, Tenhagen BA, Kasbohrer A. 2014. Oral antimicrobials increase antimicrobial resistance in porcine E. coli—a systematic review. Prev Vet Med 113:364–375. doi: 10.1016/j.prevetmed.2013.12.007. [DOI] [PubMed] [Google Scholar]
- 27.Simoneit C, Burow E, Tenhagen BA, Kasbohrer A. 2015. Oral administration of antimicrobials increase antimicrobial resistance in E. coli from chicken—a systematic review. Prev Vet Med 118:1–7. doi: 10.1016/j.prevetmed.2014.11.010. [DOI] [PubMed] [Google Scholar]
- 28.Larsen I, Hjulsager CK, Holm A, Olsen JE, Nielsen SS, Nielsen JP. 2016. A randomised clinical trial on the efficacy of oxytetracycline dose through water medication of nursery pigs on diarrhoea, faecal shedding of Lawsonia intracellularis and average daily weight gain. Prev Vet Med 123:52–59. doi: 10.1016/j.prevetmed.2015.12.004. [DOI] [PubMed] [Google Scholar]
- 29.Joseph NM, Bhanupriya B, Shewade DG, Harish BN. 2015. Relationship between antimicrobial consumption and the incidence of antimicrobial resistance in Escherichia coli and Klebsiella pneumoniae isolates. J Clin Diagn Res 9:DC08–DC12. doi: 10.7860/JCDR/2015/11029.5537. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Agodi A, Auxilia F, Barchitta M, Brusaferro S, D'Errico MM, Montagna MT, Pasquarella C, Tardivo S, Mura I, SPIN-UTI Network of the GISIO Working Group of the Italian Society of Hygiene, Preventive Medicine and Public Health (SItI). 2015. Antibiotic consumption and resistance: results of the SPIN-UTI project of the GISIO-SItI. Epidemiol Prev 39:94–98. [PubMed] [Google Scholar]
- 31.Agerso Y, Aarestrup FM. 2013. Voluntary ban on cephalosporin use in Danish pig production has effectively reduced extended-spectrum cephalosporinase-producing Escherichia coli in slaughter pigs. J Antimicrob Chemother 68:569–572. doi: 10.1093/jac/dks427. [DOI] [PubMed] [Google Scholar]
- 32.Garcia-Migura L, Hendriksen RS, Fraile L, Aarestrup FM. 2014. Antimicrobial resistance of zoonotic and commensal bacteria in Europe: the missing link between consumption and resistance in veterinary medicine. Vet Microbiol 170:1–9. doi: 10.1016/j.vetmic.2014.01.013. [DOI] [PubMed] [Google Scholar]
- 33.Speksnijder DC, Mevius DJ, Bruschke CJ, Wagenaar JA. 2015. Reduction of veterinary antimicrobial use in the Netherlands. The Dutch success model. Zoonoses Public Health 62(Suppl 1):79–87. doi: 10.1111/zph.12167. [DOI] [PubMed] [Google Scholar]
- 34.World Health Organization. 2015. Global action plan against antimicrobial resistance. World Health Organization, Geneva, Switzerland. [Google Scholar]
- 35.Langlois BE, Dawson KA, Stahly TS, Cromwell GL. 1984. Antibiotic resistance of fecal coliforms from swine fed subtherapeutic and therapeutic levels of chlortetracycline. J Anim Sci 58:666–674. doi: 10.2527/jas1984.583666x. [DOI] [PubMed] [Google Scholar]
- 36.Kanwar N, Scott HM, Norby B, Loneragan GH, Vinasco J, Cottell JL, Chalmers G, Chengappa MM, Bai J, Boerlin P. 2014. Impact of treatment strategies on cephalosporin and tetracycline resistance gene quantities in the bovine fecal metagenome. Sci Rep 4:5100. doi: 10.1038/srep05100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lay KK, Koowattananukul C, Chansong N, Chuanchuen R. 2012. Antimicrobial resistance, virulence, and phylogenetic characteristics of Escherichia coli isolates from clinically healthy swine. Foodborne Pathog Dis 9:992–1001. doi: 10.1089/fpd.2012.1175. [DOI] [PubMed] [Google Scholar]
- 38.Day MJ, Rodriguez I, van Essen-Zandbergen A, Dierikx C, Kadlec K, Schink AK, Wu G, Chattaway MA, DoNascimento V, Wain J, Helmuth R, Guerra B, Schwarz S, Threlfall J, Woodward MJ, Coldham N, Mevius D, Woodford N. 2016. Diversity of STs, plasmids and ESBL genes among Escherichia coli from humans, animals and food in Germany, the Netherlands and the UK. J Antimicrob Chemother 71:1178–1182. doi: 10.1093/jac/dkv485. [DOI] [PubMed] [Google Scholar]
- 39.Adler A, Katz DE, Marchaim D. 2016. The continuing plague of extended-spectrum beta-lactamase-producing Enterobacteriaceae infections. Infect Dis Clin North Am 30:347–375. doi: 10.1016/j.idc.2016.02.003. [DOI] [PubMed] [Google Scholar]
- 40.Ahmad A, Græsbøll K, Christiansen LE, Toft N, Matthews L, Nielsen SS. 2015. Pharmacokinetic-pharmacodynamic model to evaluate intramuscular tetracycline treatment protocols to prevent antimicrobial resistance in pigs. Antimicrob Agents Chemother 59:1634–1642. doi: 10.1128/AAC.03919-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Ahmad A, Zachariasen C, Christiansen LE, Græsbøll K, Toft N, Matthews L, Damborg P, Agerso Y, Olsen JE, Nielsen SS. 2015. Pharmacodynamic modelling of in vitro activity of tetracycline against a representative, naturally occurring population of porcine Escherichia coli. Acta Vet Scand 57:79. doi: 10.1186/s13028-015-0169-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Ahmad A, Zachariasen C, Christiansen LE, Græsbøll K, Toft N, Matthews L, Olsen JE, Nielsen SS. 2016. Multistrain models predict sequential multidrug treatment strategies to result in less antimicrobial resistance than combination treatment. BMC Microbiol 16:118. doi: 10.1186/s12866-016-0724-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bager F, Aarestrup FM, Jensen NE, Madsen M, Meyling A, Wegener HC. 1999. Design of a system for monitoring antimicrobial resistance in pathogenic, zoonotic and indicator bacteria from food animals. Acta Vet Scand Suppl 92:77–86. [PubMed] [Google Scholar]
- 44.Chen CY, Nace GW, Irwin PL. 2003. A 6 × 6 drop plate method for simultaneous colony counting and MPN enumeration of Campylobacter jejuni, Listeria monocytogenes, and Escherichia coli. J Microbiol Methods 55:475–479. doi: 10.1016/S0167-7012(03)00194-5. [DOI] [PubMed] [Google Scholar]
- 45.Katakweba AA, Moller KS, Muumba J, Muhairwa AP, Damborg P, Rosenkrantz JT, Minga UM, Mtambo MM, Olsen JE. 2015. Antimicrobial resistance in faecal samples from buffalo, wildebeest and zebra grazing together with and without cattle in Tanzania. J Appl Microbiol 118:966–975. doi: 10.1111/jam.12738. [DOI] [PubMed] [Google Scholar]
- 46.Ahmad A, Zachariasen C, Christiansen LE, Græsbøll K, Toft N, Matthews L, Nielsen SS, Olsen JE. 2016. Modeling the growth dynamics of multiple Escherichia coli strains in the pig intestine following intramuscular ampicillin treatment. BMC Microbiol 16:205. doi: 10.1186/s12866-016-0823-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.R Core Team. 2015. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria: https://www.R-project.org/. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.