Follow-Up of External Quality Controls for PCR-Based Diagnosis of Whooping Cough in a Hospital Laboratory Network (Renacoq) and in Other Hospital and Private Laboratories in France

Abstract

The French National Reference Centre (NRC) for Whooping Cough carried out an external quality control (QC) analysis in 2010 for the PCR diagnosis of whooping cough. The main objective of the study was to assess the impact of this QC in the participating laboratories through a repeat analysis in 2012.

TEXT

Whooping cough, a highly contagious respiratory disease affecting humans (1), can be diagnosed by culture, PCR, or serology (2). PCR, and particularly real-time PCR (RT-PCR) methods, overcomes some of the limitations of culture and serology and has become the most widely used method for diagnosing Bordetella infection. Insertion sequences IS481 and IS1001 are the most frequently used targets for the detection of B. pertussis and B. parapertussis, respectively. However, the IS481 sequence is also present in the B. holmesii genome (3). Moreover, B. bronchiseptica is causing problems for current consensus molecular diagnostic tests for pertussis, because some isolates harbor IS481 (4) and/or IS1001 (5). Following on the initial proficiency program (6), the French National Reference Centre (NRC) offered quality controls (QCs) to the pediatric hospital laboratory network (7) and some other hospital and private medical laboratories in 2010 (QC-2010) and 2012 (QC-2012). The main objectives were (i) to provide an overview of the PCR methods used by the laboratories and of the interpretation of the data obtained and (ii) to assess the impact of QC-2010 on the participating laboratories by repeat evaluation in 2012. Participation was voluntary, with 33 laboratories participating in QC-2010 and 35 in QC-2012. The participating laboratories were asked to complete a technical questionnaire (Table 1) to provide information about their equipment, reagents, and methods. The DNA samples of Bordetella isolates harboring the IS481 or IS1001 sequence in their genome were prepared by NRC and sent to the participating laboratories (Table 2). Each proficiency panel consisted of six specimens, each containing DNA at a concentration of 0.01 to 100.00 pg/μl or molecular biology-quality water. The participants were asked to report the positive or negative result obtained for DNA detection for each tube, together with the crossover threshold (Ct) value and the conclusive sentence routinely used for such situations in their analysis reports. The correct interpretation for a sample that tests positive for IS481 or IS1001 should be “detection of Bordetella species DNA.” The conclusive sentence was considered incorrect if the laboratory interpreted a positive test for IS481 as indicating that the sample corresponded to B. pertussis or a positive test for IS1001 as indicating that the sample corresponded to B. parapertussis. The results were compiled and analyzed by NRC and then transmitted to all participants.

TABLE 1.

Summary of participating laboratories responses to the technical questionnaire

Analysis yr	Extraction method (%)	Target combination (%)	Real-time apparatus type (%)b	No. of cycles (%)	Probe type (%)	Amplification mixture (%)	Use of extraction (%)/amplification control(s) (%)
2010 (n = 32a)
No. of answers	28	31	30	29	30	30	29
	Commercial kit (50)	IS481/IS1001 (47)	LightCycler (Roche) (37)	40 (55)	Hydrolysis (73)	Commercial kit (33)	76/100
	Automated system (36)	IS481 (44)	SmartCycler (Cepheid) (26)	45 (41)	Hybridization (17)	PCR master mix completed with oligosc (67)
	In-house method (14)	IS481/IS1001/ptxA or ptxA-Pr (6)	ABI Prism (Applied Biosystems) (20)	35 (4)	SYBR Green (7)
		IS481/ptxA (3)	Rotor-Gene (10)		Beacon (3)
2012 (n = 34a)
No. of answers	34	34	34	34	34	34	34
	Commercial kit (52)	IS481/IS1001 (71)	LightCycler (Roche) (40)	45 (59)	Hydrolysis (79)	Commercial kit (62)	84/97
	Automated system (38)	IS481 (23)	SmartCycler (Cepheid) (35)	40 (26)	Hybridization (15)	PCR master mix completed with oligosc (38)
	In-house method (13)	IS481/ptxA (3)	ABI Prism (Applied Biosystems) (12)	50 (15)	SYBR Green (3)
		IS481/IS1001/ptxA-Pr/recA-h (3)	Rotor-Gene (9)		Beacon (3)

^aParticipants using RT-PCR methodology are presented. The only laboratory using endpoint PCR method (Genoquick Bordetella Hain LifeScience kit; IS481-IS1001; Perkin Elmer apparatus) is not included in the table.

^bLightCycler 1/2/480; SmartCycler I/II; ABI Prism 7000/7500/7900; Rotor-Gene 6000.

^cParticipants used RT-PCR mixture from different suppliers (Applied Biosystems, Roche, Eurogentec) completed with primers and probes from different suppliers.

TABLE 2.

Proficiency panel results of the QC-2010 and QC-2012 samples

Yr of analysis and code	NRC proficiency panelsa		Results for IS481 or IS1001 detection by laboratoriesb
	Sample	Ctc	No. of positive/total answersd	Mean Cte (no. of samples)	Ct interval	Ct CV (%)
2010
QC10-1	Water	Undet	1/33	34 (1)	NA	NA
QC10-2	B. pertussis 10 pg/μl	22	33/33	21 (30)	14–32	18
QC10-3	B. parapertussis 1pg/μl	25	18/18	25 (16)	19–36	15
QC10-4	Water	Undet	1/33	40 (1)	NA	NA
QC10-5	B. pertussis 0.1 pg/μl	28	33/33	27 (30)	21–38	13
QC10-6	B. holmesii 10 pg/μl	23	33/33	22 (30)	15–32	18
2012
QC12-1	B. holmesii 10 pg/μl	23	34/35	22 (33)	20–28	8
QC12-2	B. parapertussis 0.1 pg/μl	29	24/27	31 (23)	29-41	9
QC12-3	B. bronchiseptica 100 pg/μl	26	32/35	26 (31)	23–36	11
QC12-4	B. pertussis 0.01 pg/μl	31	32/35	31 (31)	28–37	6
QC12-5	Water	Undet	2/35	39 (2)	NA	NA
QC12-6	B. pertussis 1 pg/μl	23	35/35	24 (34)	20–28	8

^aCt, cycle threshold; undet, undetermined.

^bCV, coefficient of variation (ratio of the standard deviation to the mean of the Ct values); NA, not applicable.

^cAverage Ct of at least two replicates.

^dThe laboratory using endpoint PCR method (n = 1) is only included in the positive/total answers.

^eMean and interval of Ct calculated with the values reported by the participating laboratories.

A summary of the technical questionnaire completed by the laboratories is presented in Table 1. In 2010 and 2012, half of the laboratories routinely used commercial kits, a third used an automated system for DNA extraction from respiratory samples, and the others were still using a manual in-house method. In 2012, two manufacturers (Roche and Cepheid) accounted for 75% of the PCR instruments used by the participating laboratories. The frequency of use of IS1001 and IS481 targets in combination increased from 53% in 2010 to 74% in 2012. The increase in the number of laboratories using the IS1001 target improved the surveillance of B. parapertussis or, in some cases, B. bronchiseptica circulation in France. Only a few laboratories used additional targets and had the capacity to specifically test the presence of B. pertussis or B. holmesii DNA (Table 1). All except one laboratory used RT-PCR in 2010 and 2012. The proportion of laboratories that used commercial kits for the amplification step increased from 33% in 2010 to 62% in 2012. Commercial kits systematically include an internal control for validation of the extraction step and/or amplification, increasing the reproducibility of testing (8).

We analyzed the results for PCR sensitivity of the two QC exercises (Table 2). In 2010, the external QC included two negative and four positive samples. All of the participating laboratories detected the DNA in the four positive samples, regardless of the Bordetella species concerned. Three laboratories reported at least one false-positive result (two in a negative sample). To overcome the risk of a false-positive result, it is crucial to use a clean space for the PCR assay. The interval of the Ct values was between 17 and 18, regardless of the concentration of DNA and the targets used, reflecting the heterogeneity of the techniques. In 2012, the QC included one negative and five positive samples. A majority of the participants gave correct results. However, three laboratories reported at least one false-negative sample, and one laboratory reported both a false-positive and a false-negative sample. Finally, a laboratory disarrayed the tubes from the panel and reported several incorrect results. As in 2010, we found that the laboratories were using diverse amplification methods, but 91% were able to detect the lowest concentration of B. pertussis DNA (0.01 pg/μl).

The participants were asked to report the conclusive sentence they routinely used in their analysis reports. In 2010, among the laboratories obtaining correct results (n = 30), only 4 laboratories gave the correct interpretation, regardless of the target used, and 4 others gave the correct interpretation for IS481 but an incorrect conclusion for the IS1001 target. In 2012, the number of laboratories giving correct interpretations was higher than in 2010, contributing to an improvement in reporting: among the laboratories obtaining correct results (n = 29), 11 laboratories gave the correct conclusion, regardless of the target used, and 7 laboratories gave the correct interpretation for IS481 but an incorrect conclusion for the IS1001 target.

In conclusion, the major problem identified in this study was the incorrect interpretation of results obtained by the participating laboratories, as already reported by the European surveillance network for vaccine-preventable disease (EUVAC.Net) (9). A positive result in the IS481 assay does not constitute proof of the presence of B. pertussis DNA. Instead, it should be interpreted as demonstrating the presence of genetic material from Bordetella species. Likewise, a positive result in the IS1001 assay does not constitute proof of the presence of B. parapertussis DNA, as it may, in some cases, be due to the presence of DNA from B. bronchiseptica. It should therefore be indicated in the analysis report, even if infrequent. This is particularly important for adolescent and adult populations. As previously shown, B. holmesii carriage is mostly found in teenagers and adults, at least in Europe and North America (10, 11), although it has been detected in some newborns in Romania, Chile, and Argentina (12–14). In these populations, it is important to identify the Bordetella species present in the samples, particularly when vaccine effectiveness or the duration of vaccine-induced immunity is determined.

External QC campaigns for the diagnosis of whooping cough by PCR should be organized regularly (ideally every 2 years) by the NRC to assess the effects of proficiency panels on laboratory performances, particularly for reporting of results and interpretation.

Funding Statement

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.