Abstract
As part of a European research project, the performance of a PCR assay to detect food-borne thermotolerant campylobacters (Campylobacter jejuni, C. coli, and C. lari) was evaluated through an international collaborative trial involving 12 participating laboratories. DNA from 10 target and 8 nontarget strains was tested, and the results were reported as the presence of a positive signal after gel electrophoresis. The overall inclusivity (sensitivity) was 93.7%, and the exclusivity (specificity) was 100%. The results indicate that the assay can become an international standard and can be confidently applied in microbiological laboratories.
There are a great number of PCR assays for food-borne campylobacters that have been developed and reported in the scientific literature, but none have been validated for use by a full-scale interlaboratory collaborative trial (5, 7). Proper validation based on consensus criteria is an absolute prerequisite for successful adoption of a PCR-based diagnostic methodology (3). Due to lack of international validation and standardized protocols, as well as the quality of reagents and equipment, the transfer of the assays from expert laboratories to end-use laboratories has met with great difficulties. As a step toward the development of a standard PCR-based method to detect thermotolerant campylobacters (C. jejuni, C. coli, and C. lari) in foods, the performance characteristics of the assay itself, especially its ability to distinguish between a range of target and nontarget strains, should be evaluated (3). In order to do so, 12 European laboratories (from Austria, Czech Republic, Denmark, France, Germany, Greece, The Netherlands, Slovakia, Spain, Sweden, and the United Kingdom) participated in a collaborative validation trial that was performed to assess the reproducibility of the thermotolerant Campylobacter PCR assay developed as part of the European FOOD-PCR project by Lübeck et al., as described in the companion publication (6). The assay had been validated in-house against an extensive list of thermotolerant and nonthermotolerant campylobacters and other bacterial isolates and was shown to be highly accurate according to MicroVal criteria (1). Each laboratory received 20 coded “blind” identical DNA samples, including DNA extracted from 10 thermotolerant Campylobacter strains (C. jejuni, C. coli, and C. lari) and 8 other Campylobacter spp. and non-Campylobacter species (Table 1). Each participant received a detailed trial chronology, a standard operating procedure (SOP; available at http://www.pcr.dk), and a test report on which to record the results to return to the trial leader for analysis. The SOP was based on the method reported in the companion paper (6). Each participant received sufficient reagents to perform PCRs in triplicate for each sample. The strains were maintained and cultured as described in reference 6. DNA was extracted from a loop of colonies from blood plates by using a DNeasy tissue kit (69504; Qiagen, Hilden, Germany) and was quantified with a TD-360 Mini-Fluorometer (Turner Designs, Sunnyvale, Calif.). Five-microliter samples containing 100 pg of DNA μl−1 were sent on dry ice to each participant by courier.
TABLE 1.
Bacterial strains used in the collaborative trial
Strain | Serotype | Description |
---|---|---|
C. jejuni | ||
CCUG 11284 | Type strain | |
CCUG 10936 | Penner 2 | Common serotype |
CCUG 10938 | Penner 4 | Common serotype |
CCUG 10950 | Penner 19 | Group B streptococcus |
CCUG 12795 | Penner 55 | |
C. coli | ||
CCUG 11283 | Type strain | |
CCUG 10939 | Penner 5 | Common serotype |
CCUG 10960 | Penner 30 | Common serotype |
C. lari | ||
CCUG 23947 | Type strain | Represents one taxonomic group of C. lari |
CCUG 20707 | Represents another taxonomic group of C. lari | |
C. upsaliensis | ||
CCUG 14913 | Type strain | Nontarget |
CCUG 20818 | Nontarget | |
C. helveticus | ||
CCUG 30682 | Type strain | Nontarget |
CCUG 30566 | Nontarget | |
C. hyointestinalis CCUG 14169 | Type strain | Nontarget (found in food animals) |
C. fetus CCUG 6823 | Type strain | Nontarget (found in food animals) |
Arcobacter butzleri CCUG 30483 | Type strain | Nontarget (found in food animals) |
Salmonella enterica CCUG 31969 | Type strain | Nontarget |
The PCR mixture contained 24 μl of a master mixture containing the following: 10× PCR buffer for Tth DNA polymerase (1480022; Roche Applied Science, Hvidovre, Denmark), 25 mM MgCl2 (N808-0010; Applied Biosystems, Nærum, Denmark), 10 mM each deoxynucleoside triphosphate (dNTP; 27-2035-03; Amersham Pharmacia Biotech, Buckinghamshire, United Kingdom), 0.22 μM primer OT1559, 0.24 μM primer 18-1, 1 U of Tth DNA polymerase (14800322; Roche Applied Science), 20 mg of bovine serum albumin per ml (711454; Roche Applied Science), and 103 copies of internal control plasmid (6). One hundred picograms of DNA solution was used as target. The thermocycling program was as follows: 94°C for 2 min; followed by 35 cycles of 94°C for 30 s, 58°C for 15 s, and 72°C for 30 s; and a final extension step at 72°C for 4 min. After cycling, the PCR amplicons were detected by electrophoresis in a 1.8% agarose gel, stained with ethidium bromide, and visualized under UV light. Table 2 shows the results from each participant in the collaborative trial. Only laboratory 1 reported positive signals from nontarget DNA samples. Participants 2, 4, and 7 performed the assay only in duplicate.
TABLE 2.
Participants' results in the collaborative trial of the thermotolerant Camplylobacter PCR assay
Strain | No. of positive signals expecteda | No. of positive signals obtained by participant laboratory:
|
|||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 2b | 3 | 4b | 5 | 6 | 7b | 8 | 9 | 10 | 11 | 12 | ||
C. jejuni | |||||||||||||
CCUG 11284 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 0 | 3 | 3 | 3 | 3 | 3 |
CCUG 10936 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
CCUG 10938 | 3 | 3 | 1 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
CCUG 10950 | 3 | 3 | 2 | 3 | 2 | 3 | 2 | 2 | 3 | 3 | 3 | 3 | 3 |
CCUG 12795 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
C. coli | |||||||||||||
CCUG 11283 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
CCUG 10939 | 3 | 3 | 2 | 3 | 2 | 0 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
CCUG 10969 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
C. lari | |||||||||||||
CCUG 23947 | 3 | 3 | 2 | 3 | 2 | 3 | 3 | 2 | 3 | 0 | 3 | 3 | 0 |
CCUG 20707 | 3 | 2 | 2 | 3 | 1 | 1 | 3 | 2 | 3 | 0 | 3 | 3 | 0 |
C. upsaliensis | |||||||||||||
CCUG 14913 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
CCUG 20818 | 0 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
C. helveticus | |||||||||||||
CCUG 30682 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
CCUG 30566 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
C. hyointestinalis CCUG 14169 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
C. fetus CCUG 6823 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Arcobacter butzleri CCUG 30485 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Salmonella enterica CCUG 31969 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Number of positive PCR signals expected from triplicate reactions.
Only duplicate reactions were performed by these participants.
The results of the trial were evaluated according to the methods of Langton et al. (4) (Table 3). These methods are useful for analyzing collaborative trial data regarding qualitative microbiological methods (8). In this study, the accuracy parameters—sensitivity and specificity—of the assay are termed “inclusivity” and “exclusivity,” respectively, as suggested by the new International Organization for Standardization (ISO) standard (3). Inclusivity is defined as the percentage of target DNA samples that gave a correct positive signal. Exclusivity is defined as the percentage of nontarget DNA samples that gave a correct negative signal (i.e., only the internal amplification control [IAC] signal appeared). Confidence intervals for the accuracy parameters were calculated by the method of Wilson et al. (9). Repeatability and reproducibility were determined by calculating the accordance and concordance values (4, 8). Accordance is defined as the percentage chance of finding the same result (i.e., either positive or negative whether correct or not) from two identical DNA samples analyzed in the same laboratory under standard repeatability conditions. Concordance is defined as the percentage chance of finding the same result from two identical samples analyzed in different laboratories under standard repeatability conditions. The calculations take into account differences in replication in different laboratories by weighting results appropriately. In the present trial, all of the results were combined for this determination, and identical samples were therefore defined as containing either target or nontarget DNA. The concordance odds ratio (COR) (4) was calculated in order to assess the degree of between-laboratory variation in results. Confidence intervals for accordance and concordance were calculated by the bootstrap method of Davison and Hinckley (2). The statistical evaluation of the data from this set of tests is as follows (with lower and upper 95% confidence intervals in parentheses): inclusivity, 93.7% (90.3%, 95.9%); exclusivity, 100%; accordance, 93.4% (88.4%, 97.7%); concordance, 93.2% (88.5%, 97.2%); and COR, 1.04 (0.97, 1.19). Note that the results from laboratory 1 were excluded from the analysis. Although there was no evidence for protocol violation, the high number of false positives from this laboratory indicated a strong possibility of sample cross-contamination during analysis, justifying the exclusion of their results from the analysis under the MicroVal recommendations (1).
TABLE 3.
Statistical evaluation of the collaborative trial of the thermotolerant campylobacter PCR assaya
Inclusivity (%) | Exclusivity (%) | Accordance (%) | Concordance (%) | COR |
---|---|---|---|---|
93.7 (90.3, 95.9) | 100 | 93.4 (88.4, 97.7) | 93.2 (88.5, 97.2) | 1.04 (0.97, 1.19) |
Numbers in parentheses are the lower and upper 95% confidence intervals.
In the collaborative trial, accuracy values were high. We propose that, with regard to the outcome of collaborative trials of PCR assays, inclusivity and exclusivity values higher than 90% should signify that the assay is acceptable for implementation in end-use laboratories. Accordance and concordance parameters were designed to be analogous to the repeatability and reproducibility values used in validation of quantitative methods (4). In this collaborative trial, these values were high, indicating that the method may be confidently reproduced and applied in other laboratories. The COR reflects the relative magnitude of the accordance and concordance values (4). A COR of 1.00 or less indicates that two samples sent to different laboratories will probably produce the same result as the two samples analyzed by the same laboratory. A COR significantly greater than 1.00 indicates that variability between laboratories is greater than intralaboratory variation. In this collaborative trial, the CORs for the results of the analysis of both target and nontarget DNA samples were not significantly greater than 1.00. This shows that the PCR assay was just as reproducible between laboratories as it was repeatable within a laboratory.
It is intended that the PCR assay be used for the detection of these pathogens in foods and materials used in primary food production. Sample pretreatment methods, based on enrichment culture, have been developed that facilitate PCR-based detection of thermotolerant campylobacters in pig carcass swab and poultry carcass rinse samples (Josefsen et al., unpublished data). In a subsequent collaborative trial, the assay has been performed successfully with chicken rinse samples (Josefsen et al., unpublished).
It is hoped that the validated methods will be suitable for ultimate adoption as standards. This should encourage the implementation of these PCR-based methods and their full acceptance alongside traditional diagnostic procedures in routine microbiological laboratories.
Acknowledgments
The validation of the thermotolerant Campylobacter PCR assay has been performed as part of the European project FOOD-PCR (QLK1-CT-1999-00226). N.C. acknowledges the support of the United Kingdom Food Standards Agency.
We thank the following laboratories for their participation and cooperation in this collaborative trial: M. Kuhn, Congen GmbH, Berlin, Germany; M. D'Agostino, Central Science Laboratory, Sand Hutton, United Kingdom; P. Tassios, Department of Microbiology at University of Athens, Athens, Greece; F. Hansen, Danish Meat Research Institute, Roskilde, Denmark; V. Kmet, Institute of Animal Physiology, Slovak Academy of Sciences, Kosice, Slovakia; K. Demnerova, Institute of Chemical Technology, Prague, Czech Republic; A. Heuvelink, Inspectorate for Health Protection, Commodities and Veterinary Public Health, Zutphen, The Netherlands; A. Abdulmawjood, Institute of Veterinarian Food Science, Justus Liebig University, Giessen, Germany; P. Radstrom, Department of Applied Microbiology, University of Lund, Lund, Sweden; and J. Vazquez-Boland, University of Leon, Leon, Spain. We thank Alistair Murray for advice on the statistical analysis. We also acknowledge the technical assistance of all of the laboratory staff with the preparation and analysis of the samples used in the study.
REFERENCES
- 1.Anonymous. 2002. Microbiology of food and animal feeding stuffs—protocol for the validation of alternative methods (EN ISO/FIDS 16140). European Committee for Standardization, AFNOR, Paris, France.
- 2.Davison, A. C., and D. V. Hinckley. 1997. Bootstrap methods and their application. Cambridge University Press, Cambridge, United Kingdom.
- 3.Hoorfar, J., and N. Cook. 2002. Critical aspects of standardisation. Methods Mol. Biol. 216:51-64. [DOI] [PubMed] [Google Scholar]
- 4.Langton, S. D., R. Chevennement, N. Nagelkerke, and B. Lombard. 2002. Analysing collaborative trials for qualitative microbiological methods: accordance and concordance. Int. J. Food Microbiol. 79:175-181. [DOI] [PubMed] [Google Scholar]
- 5.Lübeck, P. S., and J. Hoorfar. 2002. PCR technology for detection of foodborne pathogens. Methods Mol. Biol. 216:65-84. [DOI] [PubMed] [Google Scholar]
- 6.Lübeck, P. S., P. Wolffs, S. L. W. On, P. Ahrens, P. Rådström, and J. Hoorfar. 2003. Toward an international standard for PCR-based detection of food-borne thermotolerant campylobacters: assay development and analytical validation. Appl. Environ. Microbiol. 69:5664-5669. [DOI] [PMC free article] [PubMed]
- 7.Olsen, J. E., S. Aabo, W. Hill, S. Notermans, K. Wernars, P. E. Granum, T. Popovic, H. N. Rasmussen, and Ø. Olsvik. 1995. Probes and polymerase chain reaction for detection of food-borne bacterial pathogens. Int. J. Food Microbiol. 28:1-78. [DOI] [PubMed] [Google Scholar]
- 8.Scotter, S. L., S. Langton, B. Lombard, N. Schulten, N. Nagelkerke, P. H. In't Veld, P. Rollier, and C. Lahellec. 2001. Validation of ISO method 11290. Part 1. Detection of Listeria monocytogenes in foods. Int. J. Food Microbiol. 64:295-306. [DOI] [PubMed] [Google Scholar]
- 9.Wilson, E. B. 1927. Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 22:209-212. [Google Scholar]