Abstract
With the support of a ministerial program for innovative and expensive technologies, dedicated to the economic evaluation of laboratory diagnosis of pertussis by real-time PCR, external quality assessment for real-time IS481 PCR was carried out. Coordinated by the National Centre of Reference of Pertussis and other Bordetelloses (NCR), this study aimed to harmonize and to assess the performances of eight participating microbiology hospital laboratories throughout the French territory. Between January 2006 and February 2007, 10 proficiency panels were sent by the NCR (ascending proficiency program), representing a total of 49 samples and including eight panels to analyze and evaluate the global sensitivity and specificity of real-time PCR, one to assess the limit of detection, and one to evaluate nucleic acid extraction methods. As part of the descending proficiency program, extracted DNA from clinical samples was sent by the eight participating laboratories in different panels and analyzed by the NCR. In the ascending proficiency analysis, the sensitivity and specificity of the real-time PCR methods were 92.2% and 94.3%, respectively. The limit of detection of the different methods ranged between 0.1 and 1 fg/μl (0.2 to 2 CFU/μl). The nucleic acid extraction methods showed similar performances. During the descending proficiency analysis, performed with 126 samples, the result of the NCR for 15 samples (11.9%) was discordant with the result obtained by the source laboratory. Despite several initial differences, harmonization was easy and performances were homogeneous. However, the risk of false-positive results remains quite high, and we strongly recommend establishment of uniform quality control procedures performed regularly.
Bordetella pertussis is the etiological agent of whooping cough, a highly contagious respiratory disease with a long course occurring exclusively in humans (9, 24). Despite the widespread vaccination of children in France since 1966, whooping cough remains a major public health problem, with adults and adolescents now transmitting the infection to vulnerable infants. Whooping cough is the leading cause of death from community-acquired bacterial infections in infants between the ages of 10 days and 2 months in developed countries and is the third most frequent cause of death from bacterial infection in children of all ages (8). In addition, nosocomial infections with the bacterium, leading to outbreaks of whooping cough in hospitals, are now also being reported (4, 6, 10, 23).
Unvaccinated children present the typical clinical symptoms of whooping cough, whereas the clinical symptoms are much more variable and milder in neonates and previously vaccinated adults. Clinical diagnosis must therefore be confirmed by biological tests. Rapid and sensitive diagnostic methods are required to guide treatment and to prevent transmission. The specific tests available are culture, real-time PCR, and the detection of anti-pertussis toxin (PT) antibodies. All these techniques have limitations in terms of sensitivity, specificity, and practicability. B. pertussis culture remains the gold standard method for diagnosis but has low sensitivity, takes a long time, and may be difficult in practical terms. The detection of anti-PT antibodies by reference enzyme-linked immunosorbent assay is highly specific (1), but there is a lack of validated commercial tests (22). Furthermore, the increasing use of adult vaccination renders this approach to diagnosis much less useful (2).
Real-time PCR assays were developed to overcome these limitations and have been shown to be sensitive and rapid but technically difficult to perform. The insertion sequence element IS481, found in several hundred copies in the B. pertussis genome, is frequently used as a target for B. pertussis detection and has a much greater analytical sensitivity than assays with single-copy target sequences, such as that of the pertussis toxin promoter (1). The European consensus group EUPertStrain has issued several recommendations for the standardization of real-time PCR diagnosis to increase the quality of the results (20). However, many laboratories using nucleic acid amplification tests (NAATs) do not follow these recommendations, and a multitude of in-house NAATs are currently used. Several kits with CE approval are now available, but they are expensive, making the continued use of in-house PCR likely. Real-time PCR technology is very flexible, and many alternative machines and fluorescent probe systems are currently available. Consequently, NAATs are highly variable in terms of nucleic acid extraction methods, equipment, probes, extraction, and amplification controls.
Under a French Ministry of Health program providing support for innovative and expensive technologies (STIC no. IC050870, “Diamocoq”), a collaboration was established between eight hospital microbiology laboratories located throughout France and the French National Reference Center for Whooping Cough and other Bordetelloses (NRC) for a medical and economic evaluation of the use of real-time PCR for whooping cough diagnosis. The objectives of this project were to assess the economic impact of laboratory diagnosis of whooping cough by real-time PCR, which takes less than 24 h, by comparison with standard practices yielding results within 3 to 5 days. As a first step toward achieving this objective, we needed to make uniform the in-house real-time PCR techniques used. This process included the establishment of quality control for all the laboratories involved in the study.
We report here the results of the first proficiency program for the diagnosis of whooping cough by real-time PCR for eight French hospital laboratories and the NRC. This program included the design and analysis of results obtained with 10 independent proficiency panels.
MATERIALS AND METHODS
Participating laboratories.
Eight French hospital bacteriological laboratories dispersed throughout France and the NRC were involved in Ministry of Health STIC program number IC050870. The aim of this project was to carry out a medical and economic evaluation of the use of real-time PCR to diagnose whooping cough in infants and children, mostly under the age of 6 months (no adults were included in this study). At the start of the study, all the participating laboratories had at least 1 year's experience of the use of real-time PCR for whooping cough diagnosis but none had undergone regulatory external quality controls.
Methods used by participating laboratories.
Before starting the global proficiency program, an initial meeting of all the laboratories participating in the Diamocoq study and the NRC was held to establish standardization for several crucial steps in whooping cough diagnosis by real-time PCR, to increase the efficiency of this quality control program, and to optimize the procedure. This standardization was carried out in line with the recommendations of the European pertussis PCR consensus group (20). Nasopharyngeal aspirate was adopted as the optimal sample for all patients. The aspirate was systematically treated using the solubilization protocol available at the EUPertStrain website (www.eupertstrain.org) before testing. The target used for real-time PCR was the insertion element IS481, with sequence-specific detection carried out by hybridization (fluorescent resonance energy transfer [FRET]), TaqMan PCR, or with Molecular Beacon probes. Non-sequence-specific formats using SYBR green were discarded. Run and extraction controls were introduced into all PCR protocols, and physically separate areas were used to prevent crossover contamination.
Each participant chose the extraction method to be used, and five different methods (High Pure PCR template preparation kit [Roche], MagNA Pure compact nucleic acid isolation kit ′Roche], Genomic KF [Promega], QIAamp DNA mini kit [Qiagen], and EZ1 robot [Qiagen]) were used in this program (Table 1). Sequence parameters were also defined as a function of local methods of whooping cough diagnosis, with three laboratories using in-house PCR, four using a method adopted from that of Reischl et al. (18), and two laboratories using methods adapted from those of Kosters et al. (11) and Templeton et al. (21).
TABLE 1.
Real-time PCR methodology used by NRC and participating laboratories
| Laboratory | Extraction kit | Target genes | Real-time apparatus type | No. of cycles | Probe type | Origin of reaction mixture | Reaction vol (μl) | Eq. sample used for PCRd | No. of separate areas | UNG usee | Extraction controls | Amplification controls | Primer/probe supplier | Primer/probe origin or reference
|
|||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IS481 | IS1001 | ptxA-Pr | Porin | ||||||||||||||
| 1 (NRC) | High Purea | IS481, ptxA-Pr, IS1001 | LC 1.0b | 45 | FRET | FastStart Master+c | 20 | 5 | 4 | Yes | Yes | Yes | TibMolBiol | 18 | 18 | 1 | NAh |
| 2 | High Pure | IS481 | LC 2.0 | 45 | FRET | FastStart Master+ | 20 | 10 | 3 | Yes | Yes | Yes | TibMolBiol | 18 | NA | NA | NA |
| 3 | High Pure | IS481, IS1001 | Rotorgene | 40 | TaqMan | Platinum Taq DNA Polj | 25 | 10 | 4 | No | Yes | Yes | Eurogentec | In-house | In-house | NA | NA |
| 4 | Genomic KF (Promega) | IS481, IS1001, porin | LC 1.0, SmartCycler 2.0f | 45 | TaqMan | Multiplex PCR Qiagen | 25 | 10 | 4 | Yes | Yes | Yes | Sigma | 11 | 11 | NA | In-house |
| 5 | MagNA Pureg | IS481 | SmartCycler 2.0 | 40 | TaqMan | Premix Ex TaqTM | 25 | 5 | 3 | No | Yes | Yes | Eurogentec | In-house | NA | NA | NA |
| 6 | QiAamp DNA mini kit | IS481 | LC 2.0 | 50 | FRET | FastStart Master+ | 20 | 2.5 | 3 | Yes | No | Yes | GenProbe | 18 | NA | NA | NA |
| 7 | EZ1 robot (Qiagen) | IS481 | iQ cycler | 48 | Molecular Beacon | iQ Supermix (Bio-Rad) | 50 | 5 | 3 | No | Yes | Yes | Sigma | 21 | 21 | NA | NA |
| 8 | MagNAPure | IS481 | Applied 7300 | 45 | TaqMan | Applied | 50 | 5 | 4 | Yes | Yes | Yes | Genset Oligos | In-house | NA | NA | NA |
| 9 | QIAamp DNA mini kit | IS481 | SmartCycler 2.0 | 40 | TaqMan | Eurogentec | 25 | 5 | 3 | Yes | Yes | Yes | Eurogentec | 18i | NA | NA | NA |
High Pure PCR template preparation kit (Roche).
LC, LightCycler.
FastStart Master +, LightCycler FastStart DNA master plus hybridization probes (Roche).
Equivalent sample used, determined as the starting volume (if fluidified, dilution factor included) used for concentration and extraction × fraction of extracted volume added to the amplification mix.
UNG, uracyl-N-glycosylase.
For QC8 and QC9, SmartCycler 2.0 apparatus used with qPCR mastermix noRox probes (Eurogentec).
MagNA Pure Compact nucleic acid isolation kit (Roche) with use of the robot.
NA, not applicable.
Only for the primers; in-house probe design.
Pol, polymerase.
The PCR equipment used was also very diverse, as were the types of probes used (FRET, TaqMan, or Molecular Beacon) (Table 1).
After global standardization according to the recommendations of the European consensus group, each laboratory followed its own in-house procedures for the detection of B. pertussis by real-time PCR (see Table S6 in the supplemental material).
Design, preparation, and distribution of proficiency panels.
All the proficiency panels were prepared by the NRC and sent to the various participating laboratories on dry ice. Four Bordetella species were selected for the various proficiency panels. The three reference strains, the genomes of which have been sequenced—B. pertussis Tohama I, Bordetella parapertussis 12822, and Bordetella bronchiseptica RB50—were chosen. Four isolates of B. bronchiseptica of human origin (R1, BIS, ROY, and FR2045) and one isolate of Bordetella holmesii (Bho1) were also included. We chose to include the Bho1, BIS, ROY, and FR2045 isolates because these isolates also have genomes harboring IS481. Genomic DNA was extracted from each Bordetella strain or isolate with the DNeasy kit (Qiagen), used according to the manufacturer's recommendations, without the RNase step. The DNA concentration was determined by spectrophotometric quantification. Each proficiency panel consisted of four to six specimens, including either DNA from Bordetella species, a mixture of DNA samples from different Bordetella strains, or molecular biology-quality water. Each specimen was prepared in sufficient amounts to be sent out to each participating laboratory (including two tests performed by the NRC): at least 25 μl of sample, with a DNA concentration of 0.01 to 30,000 pg per μl. During the proficiency program, each laboratory was asked to perform real-time PCR on a 5-μl test sample from each specimen. Sample quality and homogeneity were assessed by systematic testing of the specimens by the NRC before their distribution. Each laboratory was informed of the date of transport and provided with instructions and a reply form to be sent by electronic mail. The laboratories were asked to report on the electronic reply form whether the samples were positive or negative for B. pertussis, with the crossover threshold (CT) value. Laboratories detecting another Bordetella species were also able to enter this result on the form. The results were compiled and analyzed by the NRC. Each proficiency test result was transmitted to all the participants, and meetings were regularly organized.
A total of 10 proficiency panels were sent by the NRC to the 8 laboratories (ascending proficiency) between January 2006 and February 2007. Laboratories were asked to evaluate one panel per month for 6 months and then one panel per quarter (Tables 2, 3, and 4). Eight proficiency panels (quality control panel 1 [QC1] to QC7 and QC9) were based on DNA extracted from Bordetella strains or isolates. After 1 year of tests, one proficiency panel (QC8), including various concentrations of B. pertussis DNA (0.01 fg/μl to 1000 fg/μl), was used to evaluate the analytical sensitivity of the real-time PCR diagnosis of B. pertussis in each laboratory. Finally, a proficiency panel (QC10) prepared with nasopharyngeal samples from infants was included in the proficiency program to assess the performance of the nucleic acid extraction step.
TABLE 2.
Results for ascending proficiency program
| Sample code | Reference strain or clinical isolate useda | Amt of DNA (pg/μl) | Result for IS481 detectionb
|
Mean CT | Interval CT | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Lab. 1a | Lab. 2 | Lab. 3 | Lab. 4 | Lab. 5 | Lab. 6 | Lab. 7 | Lab. 8 | Lab. 9 | |||||
| QC1-1 | Bp Tohama I | 0.010 | + | + | + | + | + | + | + | + | NDd | 31 | 24-35 |
| QC1-2 | Bb RB50 | 10 | − | +D | − | − | − | +D | +D | +D | ND | ||
| QC1-3 | Negative | NAc | − | − | − | − | − | +D | − | − | ND | ||
| QC1-4 | Bp Tohama I | 10 | + | + | + | + | + | + | + | + | ND | 22 | 15-26 |
| QC1-5 | Bpp 12822 | 10 | − | − | − | − | − | − | − | − | ND | ||
| QC2-1 | Negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC2-2 | Bp Tohama I | 0.01 | + | + | + | + | + | + | + | + | + | 30 | 26-35 |
| QC2-3 | Bpp 12822 | 10 | − | − | − | − | − | − | − | − | − | ||
| QC2-4 | Negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC2-5 | Bho isolate (Bho1) | 10 | + | + | + | + | + | + | + | + | + | 22 | 17-27 |
| QC3-1 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC3-2 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC3-3 | Bp Tohama I | 1 | + | + | + | + | + | + | + | + | + | 23 | 19-29 |
| QC3-4 | Bb RB50 | 10 | − | − | − | − | − | − | − | − | − | ||
| QC3-5 | Bpp 12822 | 1 | − | − | − | − | − | − | − | − | − | ||
| QC4-1 | Bp Tohama I + Bpp 12822 | 10 | + | −D | + | + | −D | + | −D | + | −D | 34 | 32-37 |
| QC4-2 | Bb isolate (RemI) | 10 | − | − | − | − | − | − | − | − | − | ||
| QC4-3 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC4-4 | Bp Tohama I | 5 | + | −D | + | + | −D | + | −D | + | −D | 35 | 32-36 |
| QC4-5 | negative | NA | − | − | − | − | − | − | − | +D | − | ||
| QC5-1 | negative | NA | − | − | − | − | − | − | +D | − | − | ||
| QC5-2 | negative | NA | − | − | − | +D | +D | − | − | − | − | ||
| QC5-3 | Bp Tohama I + Bpp 12822 | 10 | + | + | + | + | + | + | + | + | + | 22 | 18-24 |
| QC5-4 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC5-5 | Bp Tohama I | 10 | + | + | + | + | + | + | + | + | + | 22 | 18-24 |
| QC6-1 | Bp Tohama I | 10 | + | + | + | + | + | + | + | + | + | 22 | 21-24 |
| QC6-2 | Bp Tohama I | 5 | + | + | + | + | + | + | + | + | + | 23 | 22-24 |
| QC6-3 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC6-4 | negative | NA | − | − | − | − | +D | − | − | − | − | ||
| QC6-5 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC7-1 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC7-2 | Bp Tohama I | 50 | + | + | + | + | + | + | + | + | + | 22 | 18-35 |
| QC7-3 | negative | NA | − | − | − | − | − | − | − | − | − | ||
| QC7-4 | Bp Tohama I | 5 | + | + | + | + | + | + | + | + | + | 24 | 22-31 |
| QC9-1 | Bb (Bis) | 30,000 | + | + | + | + | + | ND | + | + | + | 33 | 26-37 |
| QC9-2 | Bp Tohama I | 10 | + | + | + | + | + | ND | + | + | + | 21 | 15-25 |
| QC9-3 | negative | NA | − | − | − | − | − | ND | − | − | − | ||
| QC9-4 | Bb (FR2045) | 10 | + | + | + | + | + | ND | + | + | + | 28 | 22-31 |
| QC9-5 | Bb (Roy) | 30,000 | + | + | + | + | + | ND | + | + | + | 34 | 27-38 |
Bp, B. pertussis; Bpp, B. parapertussis; Bb, B. bronchiseptica; Bho, B. holmesii. “Negative” indicates molecular-biology-quality water only.
Lab., laboratory. A superscript uppercase “D” indicates discordant results between the NRC and the laboratories; rows with boldface characters indicate cross-reaction between species.
NA, not applicable.
ND, not done.
TABLE 3.
Results for real-time PCR sensitivity proficiency program
| Sample code | Strain | DNA (fg/μl) | Result for IS481 detectiona
|
Mean CT | Interval CT | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Lab. 1 (NRC) | Lab. 2 | Lab. 3 | Lab. 4 | Lab. 5 | Lab. 6 | Lab. 7 | Lab. 8 | Lab. 9 | |||||
| QC8-1 | CIP8132 | 1,000 | + | + | + | + | + | + | + | + | + | 24 | 17-29 |
| QC8-2 | CIP8132 | 100 | + | + | + | + | + | + | + | + | + | 28 | 21-33 |
| QC8-3 | CIP8132 | 10 | + | + | + | + | + | + | + | + | + | 32 | 24-37 |
| QC8-4 | CIP8132 | 1 | + | + | + | + | + | + | + | + | + | 35 | 27-42 |
| QC8-5 | CIP8132 | 0.1 | + | − | − | + | − | − | + | + | + | 38 | 31-44 |
| QC8-6 | CIP8132 | 0.01 | − | − | − | − | − | − | − | + | − | ||
Lab., laboratory.
TABLE 4.
Results for respiratory sample extraction methodology
| Sample code | Result for Lab.a:
|
Mean CT | Interval CT | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 (NRC) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | |||
| QC10-1 SE-YE | + | + | + | + | + | + | + | + | + | 34 | 32-38 |
| QC10-2 SE-KA | − | − | − | − | − | + | − | − | − | ||
| QC10-3 SE-VO | + | + | + | + | + | + | + | + | + | 14 | 11-16 |
| QC10-4 SE-CQ | + | + | + | + | + | + | + | + | + | 14 | 12-16 |
Lab., laboratory. Bold highlighting indicates a discordant result between the NRC and the laboratory.
In parallel, each participating laboratory was also asked to send the NRC DNA extracted from nasopharyngeal samples (descending proficiency program). The eight participating laboratories sent 126 samples to the NCR for blind testing. Each laboratory sent three to four individual panels, each containing five to seven extracted DNA samples.
RESULTS
A total of 10 ascending proficiency panels were prepared and analyzed, corresponding to a total of 49 specimens. The results obtained for the 10 proficiency panels are summarized in Tables 2, 3, and 4. All the participating laboratories sent their data for each panel to the NRC, with the exception of two laboratories, which failed to send one panel each (Table 2).
The eight proficiency panels (QC1 to QC7 and QC9) based on DNA extracted from Bordetella strains or isolates were used to assess both analytical sensitivity and analytical specificity, focusing particularly on the effect of the considerable diversity of the approaches used. The results obtained with these proficiency data sets from all the laboratories and the NRC are shown in Table 2. False-positive results (as determined with negative specimens and without results obtained by the NRC) were reported in 10 cases over 175 negative results (specificity, 165/175 [94.3%]). Depending on the panel, the proportion of false positives varied between 0% (QC2, -3, and -7) and 18.7% (QC5), with no real change over time. Six of the eight laboratories obtained false-positive results despite all using physically separate areas for DNA isolation and PCR testing, and four of these six laboratories using dUTP and uracil glycosylase. It is important to be aware of the possibility of false-positive results with sensitive real-time PCR tests of this type. Four laboratories also reported eight false-negative results (sensitivity, 95/103 [92.2%]), and this finding is of much greater importance for whooping cough diagnosis. These false-negative results were obtained only in QC4 for four of the eight laboratories. Furthermore, the positive results obtained by the other four laboratories and the NRC were incorrect in terms of the amount of DNA distributed (10 pg/μl), with a mean CT of 34 and 35 instead of 21 and 22. We concluded that an incident had probably occurred during the preparation or transport of this quality control panel.
All laboratories identified the specimens spiked with B. pertussis Tohama I reference strain DNA other than QC4. These specimens contained concentrations of 0.010 to 10 pg/μl, including mixtures of B. pertussis DNA with B. parapertussis 12822 reference strain DNA. The two laboratories (other than the NRC) using PCR targeting IS1001 for B. parapertussis diagnosis efficiently identified the specimens spiked with B. parapertussis DNA, even in cases of mixtures (samples QC1-5, QC2-3, QC3-5, QC4-1, and QC5-3). All eight laboratories and the NRC identified the sample spiked with B. holmesii DNA (sample QC2-5) at a concentration of 10 pg/μl as positive, confirming that the B. holmesii genome contains the insertion sequence IS481. The detection limit for B. bronchiseptica DNA depended on the number of IS elements present in the B. bronchiseptica DNA.
The QC8 panel was used to determine the detection limit for B. pertussis isolates with the various PCR methods used. The results are summarized in Table 3. The DNA concentrations ranged from 0.01 fg/μl to 1,000 fg/μl. Whooping cough detection was found to be highly sensitive, regardless of the features of the real-time PCR test used, since all laboratories detected 1 fg/μl (0.2 CFU/μl) of DNA and half of the laboratories even detected below this threshold, right down to 0.1 fg/μl (0.02 CFU/μl).
The last proficiency panel (QC10), including three positive nasal aspirate samples and one negative one, was used to determine the sensitivity and quality of the extraction methods used by the participating laboratories. We avoided bias in the PCR and compared the various extraction approaches by testing all the extracted materials with the same real-time PCR procedure at the NRC. The results are presented in Table 4. Regardless of the nucleic acid extraction protocol used, the sensitivity of detection was excellent, with a minimal CT interval.
As part of the descending proficiency program, the NRC analyzed 126 specimens sent by the eight participating laboratories in different panels. For 111 of the 126 samples analyzed, the results obtained by the NRC were consistent with those obtained by the laboratory sending the sample. Overall, 53% of these 111 samples (n = 59) were negative and 47% (n = 52) were positive. However, discordant results between the NRC and the original laboratory were obtained for 15 samples (11.9%) (3 samples found positive by the NRC but negative by the source laboratory and 12 found negative by the NRC but positive by the source laboratory), 93% of which (n = 14) had very high CT values (>35), corresponding to the detection limit.
DISCUSSION
Due to its sensitivity, specificity, and speed, real-time PCR is accepted worldwide as a suitable method for the diagnosis of Bordetella infection, even in cases in which notification is mandatory. Bidet et al. recently reported the persistence of B. pertussis DNA in serial nasopharyngeal aspirates from 22 children treated for whooping cough, as determined by real-time PCR tests (5). They demonstrated that real-time PCR could be used to diagnose whooping cough in young children for up to three weeks after the start of treatment. A study comparing current widely used techniques for whooping cough diagnosis in exposed populations showed a combination of real-time PCR for IS481 and single serological tests to be the most sensitive diagnostic approach (1).
The clinical value of real-time PCR for whooping cough diagnosis is well established, but until recently, no validated commercial kits were available and each microbiological laboratory therefore had to develop its own NAAT. This resulted in a large number of different NAATs being used, and the rapid changes in these tests have made it difficult to apply existing guidelines. Moreover, no guidelines for real-time PCR have been developed by regulatory agencies. Recommendations for the use of real-time PCR for the diagnosis of B. pertussis infections were published in 2005 by the EUPertStrain network (20). These recommendations were followed by the French NRC and all the participating bacteriological laboratories during this proficiency program. Unfortunately, too many hospitals and private laboratories do not respect these guidelines and are not subject to regular quality controls.
A previous external quality assessment (EQA) program for the molecular detection of B. pertussis in European countries was organized by the Belgian Centres for Molecular Diagnostics and was reported in 2005 (14). This program was set up to evaluate block-based and real-time PCR, and one of the French laboratories participating in the study reported here joined this EQA on a voluntary basis. The Belgian program included two independent proficiency panels, each consisting of 35 samples. Not all the participating laboratories (six for the first EQA and nine for the second) used the same target or the same real-time PCR test. Moreover, the sizes of the samples used for PCR varied widely (1 to 30 μl). This study concluded that the lack of homogeneity between PCR protocols and performance highlights the need for external quality controls.
Our study provided a unique opportunity for evaluating and comparing the various protocols used for the diagnosis of whooping cough by real-time PCR in France. All these protocols targeted the IS481 element. This proficiency program is the first French EQA based solely on real-time PCR to be described since the publication of the European recommendations.
All the laboratories used in-house NAAT methods. Very few discrepant results were observed if QC4 was excluded. This consistency was not expected on the basis of previous reports and the differences in the equipment used by the nine laboratories. False-positive results with water specimens or with the DNA of a Bordetella species devoid of IS481 were observed in both the first and last quality control panels, highlighting the ever-present risk of contamination in such sensitive assays. The B. pertussis reference strain contains a large number of copies of IS481 copies, rendering the real-time PCR highly sensitive, but the number of copies of this sequence is increasing in the isolates currently circulating in regions in which vaccination is carried out (7). False-negative results with B. pertussis DNA samples were observed, but only for QC4, probably due to the quality control panel itself. Indeed, we hypothesized that an incident had occurred during the preparation or transport of the samples, because even when the DNA was detected, the CT values were not correct for the amount indicated for any of the laboratories. These data again highlight the need for such QC but also the difficulties involved in its organization.
The frequency of false-positive results with B. holmesii DNA specimens has already been highlighted in other studies (12, 18). One large study revealed that B. holmesii was not detected in respiratory samples from patients with suspected whooping cough (3). However, careful surveillance of this Bordetella species in respiratory samples is required, based on both specific B. holmesii PCR and culture, because B. holmesii has been reported to cause respiratory infections (13). Although the genome sequence of B. bronchiseptica strain RB50 (15) was shown not to harbor IS481, some participating laboratories reported a positive result with RB50-spiked specimens during the first QC. Since this was the first step in the proficiency program, the NRC provided the same types of specimens (with the same concentration of DNA) during QC3: no false-positive results were reported by any of the laboratories. However, IS481-based molecular assays have been reported to display cross-reactivity with B. bronchiseptica isolates. In a study published in 2006, the prevalence of the IS481 repetitive element in B. bronchiseptica isolates was reported to be about 5% (17). During our proficiency program, the NRC included three human B. bronchiseptica isolates, all harboring the IS481 sequence and all detected by all eight laboratories (one laboratory did not provide results for QC9). These data confirm the results obtained in a previous study of the preparation of B. pertussis DNA from respiratory samples for real-time PCR (19).
The descending proficiency program was also very informative, since all samples (11%) with discordant results had a high CT value (>35), highlighting the importance of concerns about the interpretation of results with molecular diagnosis tests of such high sensitivity. Cycle number may also be important, as shown by the results in Table 4 for laboratory 6, which used the highest cycle number.
The use of IS481 as a target in real-time PCR should make the reporting of the diagnosis more rigorous. Using this technique, a positive detection indicates the detection of “Bordetella” DNA and not the detection of “B. pertussis” DNA unless another specific target, such as the pertussis toxin promoter, is used to confirm the detection of “B. pertussis” DNA. In fact, some laboratories used other targets in addition to IS481, such as the repetitive element IS1001, the pertussis toxin promoter region ptxA-Pr, or the porin gene, and these laboratories confirmed the identification of B. pertussis or B. parapertussis (data not shown).
Promising alternative targets for PCR should therefore continue to be evaluated (16), and the clinical sensitivity and specificity of IS481-based real-time PCR for whooping cough diagnosis should be explored further.
This proficiency program led to a request from the participating laboratories for the establishment of permanent ongoing external quality control for molecular diagnosis in collaboration with the NRC. The challenges faced by clinical laboratories include determining the type of verification experiments required for a real-time PCR assay and the appropriate number of specimens to be evaluated. Guidelines for real-time PCR, including all the necessary verification and validation by accreditation agencies, would be of great benefit to laboratories acquiring this technology. There is a real need for a well-defined quality control program for real-time PCR assays, particularly for the diagnosis of whooping cough. Quality controls allow the laboratory to minimize the reporting of inaccurate results, to report results with a high degree of confidence, and to decrease costs by detecting errors before the reporting of results.
Supplementary Material
Acknowledgments
We thank Valerie Bouchez for assistance in the preparation and validation of the various quality control panels.
This proficiency program was funded by the STIC program, grant no. IC IC050870, the Institut Pasteur Foundation, and the National Reference Center for Whooping Cough and other Bordetelloses.
Footnotes
Published ahead of print on 19 August 2009.
Supplemental material for this article may be found at http://jcm.asm.org/.
REFERENCES
- 1.Andre, P., V. Caro, E. Njamkepo, A. M. Wendelboe, A. Van Rie, and N. Guiso. 2008. Comparison of serological and real-time PCR assays to diagnose Bordetella pertussis infection in 2007. J. Clin. Microbiol. 46:1672-1677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Anonymous. 2009. Calendrier vaccinal 2009—Avis du Haut conseil de la santé publique. Bull. Epidemiol. Hebdomadaire. 16-17:147. [Google Scholar]
- 3.Antila, M., Q. He, C. de Jong, I. Aarts, H. Verbakel, S. Bruisten, S. Keller, M. Haanpera, J. Makinen, E. Eerola, M. K. Viljanen, J. Mertsola, and A. van der Zee. 2006. Bordetella holmesii DNA is not detected in nasopharyngeal swabs from Finnish and Dutch patients with suspected pertussis. J. Med. Microbiol. 55:1043-1051. [DOI] [PubMed] [Google Scholar]
- 4.Bassinet, L., M. Matrat, E. Njamkepo, S. Aberrane, B. Housset, and N. Guiso. 2004. Nosocomial pertussis outbreak among adult patients and healthcare workers. Infect. Control Hosp. Epidemiol. 25:995-997. [DOI] [PubMed] [Google Scholar]
- 5.Bidet, P., S. Liguori, A. De Lauzanne, V. Caro, M. Lorrot, A. Carol, A. Faye, N. Guiso, E. Bingen, and S. Bonacorsi. 2008. Real-time PCR measurement of persistence of Bordetella pertussis DNA in nasopharyngeal secretions during antibiotic treatment of young children with pertussis. J. Clin. Microbiol. 46:3636-3638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bonmarin, I., D. Levy-Bruhl, S. Baron, N. Guiso, E. Njamkepo, and V. Caro. 2007. Pertussis surveillance in French hospitals: results from a 10 year period. Eurosurveillance, Stockholm, Sweden. 12:http://www.eurosurveillance.org/em/v12n01/1201-226.asp.
- 7.Bouchez, V., V. Caro, E. Levillain, G. Guigon, and N. Guiso. 2008. Genomic content of Bordetella pertussis clinical isolates circulating in areas of intensive children vaccination. PLoS ONE 3:e2437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Floret, D., Group de Pathologie Infectieuse Pédiatrique, and Groupe Francophone de Réanimation et d'Urgence Pédiatrique. 2001. Les décès par infection bactérienne communautaire. Enquête dans les services de réanimation pédiatrique français. Arch. Pédiatr. 8:705-711. [DOI] [PubMed] [Google Scholar]
- 9.Forsyth, K., M. Nagai, A. Lepetic, and E. Trindade. 2005. Pertussis immunization in the global pertussis initiative international region: recommended strategies and implementation considerations. Pediatr. Infect. Dis. J. 24:S93-S97. [DOI] [PubMed] [Google Scholar]
- 10.Guiso, N. 2008. Coqueluche nosocomiale: actualités, méthodes diagnostiques et prévention. Hygiènes 16:351-355. [Google Scholar]
- 11.Kosters, K., M. Riffelmann, and C. H. Wirsing von Konig. 2001. Evaluation of a real-time PCR assay for detection of Bordetella pertussis and B. parapertussis in clinical samples. J. Med. Microbiol. 50:436-440. [DOI] [PubMed] [Google Scholar]
- 12.Loeffelholz, M. J., C. J. Thompson, K. S. Long, and M. J. R. Gilchrist. 2000. Detection of Bordetella holmesii using Bordetella pertussis IS481PCR assay. J. Clin. Microbiol. 38:467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mazengia, E., E. A. Silva, J. A. Peppe, R. Timperi, and H. George. 2000. Recovery of Bordetella holmesii from patients with pertussis-like symptoms: use of pulsed-field gel electrophoresis to characterize circulating strains. J. Clin. Microbiol. 38:2330-2333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Muyldermans, G., O. Soetens, M. Antoine, S. Bruisten, B. Vincart, F. Doucet-Populaire, N. K. Fry, P. Olcen, J. M. Scheftel, J. M. Senterre, A. van der Zee, M. Riffelmann, D. Pierard, and S. Lauwers. 2005. External quality assessment for molecular detection of Bordetella pertussis in European laboratories. J. Clin. Microbiol. 43:30-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Parkhill, J., M. Sebaihia, A. Preston, L. D. Murphy, N. Thomson, D. E. Harris, M. T. Holden, C. M. Churcher, S. D. Bentley, K. L. Mungall, A. M. Cerdeno-Tarraga, L. Temple, K. James, B. Harris, M. A. Quail, M. Achtman, R. Atkin, S. Baker, D. Basham, N. Bason, I. Cherevach, T. Chillingworth, M. Collins, A. Cronin, P. Davis, J. Doggett, T. Feltwell, A. Goble, N. Hamlin, H. Hauser, S. Holroyd, K. Jagels, S. Leather, S. Moule, H. Norberczak, S. O'Neil, D. Ormond, C. Price, E. Rabbinowitsch, S. Rutter, M. Sanders, D. Saunders, K. Seeger, S. Sharp, M. Simmonds, J. Skelton, R. Squares, S. Squares, K. Stevens, L. Unwin, S. Whitehead, B. G. Barrell, and D. J. Maskell. 2003. Comparative analysis of the genome sequences of Bordetella pertussis, Bordetella parapertussis and Bordetella bronchiseptica. Nat. Genet. 35:32-40. [DOI] [PubMed] [Google Scholar]
- 16.Probert, W. S., J. Ely, K. Schrader, J. Atwell, A. Nossoff, and S. Kwan. 2008. Identification and evaluation of new target sequences for specific detection of Bordetella pertussis by real-time PCR. J. Clin. Microbiol. 46:3228-3231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Register, K. B., and G. N. Sanden. 2006. Prevalence and sequence variants of IS481 in Bordetella bronchiseptica: implications for IS481-based detection of Bordetella pertussis. J. Clin. Microbiol. 44:4577-4583. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Reischl, U., N. Lehn, G. N. Sanden, and M. J. Loeffelholz. 2001. Real-time PCR assay targeting IS481 of Bordetella pertussis and molecular basis for detecting Bordetella holmesii. J. Clin. Microbiol. 39:1963-1966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Riffelmann, M., J. Schmetz, S. Bock, and C. H. Wirsing von Koenig. 2008. Preparation of Bordetella pertussis DNA from respiratory samples for real-time PCR by commercial kits. Eur. J. Clin. Microbiol. Infect. Dis. 27:145-148. [DOI] [PubMed] [Google Scholar]
- 20.Riffelmann, M., C. H. Wirsing von Konig, V. Caro, and N. Guiso. 2005. Nucleic acid amplification tests for diagnosis of Bordetella infections. J. Clin. Microbiol. 43:4925-4929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Templeton, K. E., S. A. Scheltinga, A. van der Zee, B. M. Diederen, A. M. van Kruijssen, H. Goossens, E. Kuijper, and E. C. Claas. 2003. Evaluation of real-time PCR for detection of and discrimination between Bordetella pertussis, Bordetella parapertussis, and Bordetella holmesii for clinical diagnosis. J. Clin. Microbiol. 41:4121-4126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Tondella, M. L., G. M. Carlone, N. Messonnier, C. P. Quinn, B. D. Meade, D. L. Burns, J. D. Cherry, N. Guiso, E. L. Hewlett, K. M. Edwards, D. Xing, A. Giammanco, C. H. Wirsing von Konig, L. Han, L. Hueston, J. B. Robbins, M. Powell, C. M. Mink, J. T. Poolman, S. W. Hildreth, F. Lynn, and A. Morris. 2009. International Bordetella pertussis assay standardization and harmonization meeting report. Centers for Disease Control and Prevention, Atlanta, Georgia, United States, 19-20 July 2007. Vaccine 27:803-814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ward, A., J. Caro, L. Bassinet, B. Housset, J. A. O'Brien, and N. Guiso. 2005. Health and economic consequences of an outbreak of pertussis among healthcare workers in a hospital in France. Infect. Control Hosp. Epidemiol. 26:288-292. [DOI] [PubMed] [Google Scholar]
- 24.Wirsing von Konig, C. H., S. Halperin, M. Riffelmann, and N. Guiso. 2002. Pertussis of adults and infants. Lancet Infect. Dis. 2:744-750. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.