Multicentric Comparative Assessment of the Bio-Evolution Toxoplasma gondii Detection Kit with Eight Laboratory-Developed PCR Assays for Molecular Diagnosis of Congenital Toxoplasmosis

Abstract

The detection of Toxoplasma gondii in amniotic fluid is an essential tool for the prenatal diagnosis of congenital toxoplasmosis and is currently essentially based on the use of PCR. Although some consensus is emerging, this molecular diagnosis suffers from a lack of standardization and an extreme diversity of laboratory-developed methods. Commercial kits for the detection of T. gondii by PCR were recently developed and offer certain advantages; however, they must be assessed in comparison with optimized reference PCR assays. The present multicentric study aimed to compare the performances of the Bio-Evolution T. gondii detection kit and laboratory-developed PCR assays set up in eight proficient centers in France. The study compared 157 amniotic fluid samples and found concordances of 99% and 100% using 76 T. gondii-infected samples and 81 uninfected samples, respectively. Moreover, taking into account the classification of the European Research Network on Congenital Toxoplasmosis, the overall diagnostic sensitivity of all assays was identical and calculated to be 86% (54/63); specificity was 100% for all assays. Finally, the relative quantification results were in good agreement between the kit and the laboratory-developed assays. The good performances of this commercial kit are probably in part linked to the use of a number of good practices: detection in multiplicate, amplification of the repetitive DNA target rep529, and the use of an internal control for the detection of PCR inhibitors. The only drawbacks noted at the time of the study were the absence of uracil-N-glycosylase and small defects in the reliability of the production of different reagents.

INTRODUCTION

Toxoplasmosis is a worldwide infectious disease that is usually asymptomatic and not severe in humans, except in certain circumstances. Thus, when primarily acquired during pregnancy, Toxoplasma gondii infection in the mother can lead to fetal infection, i.e., congenital toxoplasmosis. The diagnosis of congenital toxoplasmosis may prove a difficult task, as it requires a combination of clinical criteria and results from a battery of serologic and molecular tests in the prenatal, neonatal, and postnatal periods (1). In France, the prenatal diagnosis of congenital toxoplasmosis was based on Toxoplasma isolation in fetal blood and amniotic fluid (AF) by mouse inoculation and the detection of specific antibodies in fetal blood until the 1990s, when these methods were superseded by PCR using amniotic fluid (2–4). In France, amniocentesis is performed ≥4 weeks after Toxoplasma infection of the mother but not before the 18th week of amenorrhea (see http://cnrtoxoplasmose.chu-reims.fr); it is followed by PCR-based molecular diagnosis. A positive Toxoplasma PCR result affirms congenital toxoplasmosis; a combination treatment using pyrimethamine and sulfadiazine-sulfadoxine is then used in order to limit the presence of sequelae in the fetus, thus increasing the frequency of asymptomatic infection at birth. When a Toxoplasma PCR result is negative, congenital toxoplasmosis cannot be ruled out due to a rate of false-negative results, which, thanks to the constant progress of molecular methods (5), has been reduced to 10 to 20% (1, 6, 7). Using a high-quality molecular diagnostic method, and in spite of the persistence of false negatives, posttest risk curves using both negative and positive results can now prove to be highly informative, allowing a good assessment of the actual risk for congenital toxoplasmosis (1). A national program for the screening of acute Toxoplasma infection has been effective in pregnant women in France for decades (1, 6–8). Within this framework, only authorized proficient centers are able to realize this molecular diagnosis. In 2012, 186 cases of congenital toxoplasmosis were diagnosed in France, indicating a prevalence of 0.226 cases per 1,000 births. In 72 cases of these, amniocentesis was performed, and the Toxoplasma PCR was found positive in 60 cases and negative in 12. Thus, the overall sensitivity of the Toxoplasma PCR in France was 83.3%, and the rate of false-negative results was 16.7% (see http://cnrtoxoplasmose.chu-reims.fr). The molecular detection of T. gondii has therefore become an essential diagnostic tool in this clinical context; yet, its efficiency is hampered by a lack of standardization due to the fact that almost all PCR assays used are laboratory-developed assays, i.e., set up independently in each laboratory. This in turn leads to important variations in the protocols between laboratories (particularly in DNA extraction, the choice of DNA target, design of primers, PCR conditions, and amplicon detection) and hence in their performances (9, 10). One of the major objectives of the French National Reference Centre for Toxoplasmosis, created in 2006, was to improve and standardize the molecular diagnosis of congenital toxoplasmosis at the national level. This included improving laboratory-developed PCR assays, establishing recommendations, and eventually, testing new methods. With this aim in mind, and also with the aim of fitting with quality management policies, we wished to compare the technical performance of a commercially available PCR assay to that of laboratory-developed PCR assays (i) routinely used in eight proficient laboratories from academic hospitals and (ii) that were representative of the different methods used in France. Although nucleic acid extraction methods have been commercialized for several years, only a few turnkey systems for the molecular detection of T. gondii in humans have been marketed over the past few years. Their use appears to be an attractive alternative, as they offer a chance for standardization and they respond to an increasing demand from quality management systems. However, (i) there is no report in the literature of a comparative study in which a Toxoplasma PCR kit proved to be better than finely optimized laboratory-developed assays (11) and (ii) a few in vitro diagnostic (IVD)-labeled Toxoplasma PCR kits are currently available. The aim of this multicentric study was to compare a new commercial kit used for the detection of T. gondii by PCR to optimized reference PCR assays using a panel of amniotic fluid samples from pregnant women acutely infected with T. gondii.

MATERIALS AND METHODS

Study scheme.

The departments of parasitology and mycology of eight French Academic Hospital Centers that constitute the Molecular Biology Pole of the French National Reference Center for Toxoplasmosis (http://cnrtoxoplasmose.chu-reims.fr) participated in the study. Cryopreserved extracted DNA from AF samples drawn for the prenatal diagnosis of congenital toxoplasmosis during routine practice in each participating center was tested again with both the commercial kit and the local laboratory-developed assay. These clinical samples were negative or naturally infected with T. gondii. The study was performed in accordance with the regulations of the local medical ethics committee of each participating center, in line with the revised Helsinki Declaration. In that respect, written consent was obtained before any AF sampling.

Clinical cohort.

The AF samples were drawn by amniocentesis in pregnant women acutely infected with T. gondii during gestation. AF collection was done in line with the routine practice of each participating laboratory (in particular with respect to the volume of fluid used; see Table 1 for details). A determination of the date of maternal infection was done either after serologic conversion (i.e., the shift from a negative to a positive Toxoplasma serology with specific IgMs and IgGs) or after studying the kinetics of the specific IgG titer and taking into account the result of the test for the avidity of Toxoplasma-specific immunoglobulin G. The delay between infection and amniocentesis was calculated in weeks of amenorrhea (WA). Of note, cases of early seroconversion in which amniocentesis was delayed until the 18th WA were excluded for this calculation. In each center, the criteria used in the present study to confirm or rule out the diagnosis of congenital toxoplasmosis (i.e., reference diagnosis) were those defined by the European Research Network on Congenital Toxoplasmosis (12). These criteria allow a classification for the likelihood of T. gondii infection into five mutually exclusive categories: definite, probable, possible, unlikely, and not infected.

TABLE 1.

Main features of the PCR assays used in each center in this study

Center	Extraction method (manufacturer or study)	Mean ± SD vol of AF tested (ml)	Vol of elution (μl)	Vol of eluate/vol of PCR (μl)	PCR method	Amplicon revelation method	Detection of PCR inhibitorsa	Apparatus (manufacturer)
A	QIAamp DNA minikit (Qiagen)	1.5 ± 0	200, kit buffer	5/20 triplicate	Reischl et al. (15), Sterkers et al. (9)	FRETe	Plasmid internal control	LightCycler II (Roche)
B		13.5 ± 5	200, kit buffer	7/20 triplicate	Reischl et al. (15), Brenier-Pinchart et al. (21), Sterkers et al. (9)		β-Globin gene	LightCycler II (Roche)
C		14 ± 7	200, NATc water	5/50 triplicate	Reischl et al. (15), Yera et al. (20), Sterkers et al. (9)		β-Globin gene	LightCycler II (Roche)
D	Tween-Nonidet-NaOH lysis method (Hohlfeld et al. [13])	4 ± 0	100, TNNd lysis buffer	5/20 triplicate	Reischl et al. (15), Sterkers et al. (1), Morelle et al. (11)		T. gondii DNA internal control	LightCycler 480 (Roche)
E	QIAamp DNA blood minikit (Qiagen)	4.6 ± 3	100, kit buffer	5/20 triplicate	Reischl et al. (15), Yera et al. (20), Sterkers et al. (9)		T. gondii DNA internal control	LightCycler I (Roche)
F	High Pure template kit (Roche)	7 ± 0	200, kit buffer	5/20 triplicate	Cassaing et al. (17), Sterkers et al. (9)		β-Globin gene	LightCycler II (Roche)
G	QIAmp DNA minikit (Qiagen)	10 ± 4	100, NAT water	5/25 triplicate	Talabani et al. (16), Sterkers et al. (9)	TaqMan	Plasmid internal control	ABI Prism 7000 (Applied Biosystems)
H		NAb	NA	5/25 tetraplicate	Fekkar et al. (18), Sterkers et al. (9)		T. gondii DNA internal control	ABI Prism 7500 (Applied Biosystems)
All	All the methods above	NA	NA	Bio-Evolution no. BE-A997-AB, 5/20; or Bio-Evolution no. BE-A997-CD, 5/25	Bio-Evolution (#BE-A997)	TaqMan	Internal control	ABI Prism 7000 and 7500 (Applied Biosystems) (Bio-Evolution no. BE-A997-AB) or LightCycler 480 (Roche), LightCycler I and II (Roche) (Bio-Evolution #BE-A997-CD)

^aThe absence of reaction inhibition was verified by amplifying a positive internal control concurrently and in the same reaction tube as the test DNA after the addition of a control sequence of target DNA (internal control); this control DNA was either highly diluted T. gondii genomic DNA (equivalent to 1 or 0.5 tachyzoite genomes), an artificial plasmid DNA construct containing the primer sequences (amplified by the test primers), or a defined sequence of DNA amplified by a second primer pair, e.g., β-globin or albumin amplified under stringent conditions (to increase the PCR sensitivity to the presence of inhibitors in the sample). Center A also systematically performed one PCR with the matrix DNA diluted.

^bNA, not available.

^cNAT, nucleic acid tested.

^dTNN, Tween-Nonidet-NaOH.

^eFRET, fluorescence resonance energy transfer.

DNA extraction and cryopreservation.

DNA extraction of the AF samples was done <48 to 72 h after sampling and performed using the protocol used in routine practice in each participating laboratory. Indeed, as the Bio-Evolution kit does not include an extraction step, each center used its proper DNA extraction method. This consisted of either the QIAamp DNA minikit (Qiagen, Courtaboeuf, France) (five laboratories), the QIAamp DNA micro kit (Qiagen) (one laboratory), the High Pure PCR template (Roche, Meylan, France) (one laboratory), or the Tween-Nonidet-NaOH method (13) (one laboratory). DNA was eluted in 200 μl (for the QIAamp DNA minikit, QIAamp DNA micro kit, and High Pure template) or in 100 μl (for the QIAamp DNA minikit, and Tween-Nonidet-NaOH method). Next, DNA-extracted AF samples were frozen at −80°C (7 laboratories) or −20°C (1 laboratory) for a period of 5 years to ≤1 year. When the samples were tested prospectively, the extracted DNA was not frozen.

PCR assays.

Each participating center was asked to test thawed DNA extracts from AF samples with (i) its own laboratory-developed PCR assay (used in routine practice) and (ii) the Bio-Evolution kit for T. gondii detection by real-time PCR (Bio-Evolution reference no. BE-A997; Bussy-Saint-Martin, France). All PCR assays targeted the repetitive noncoding “cryptic” DNA element (14) we termed rep529. The primers and probes used were Tox-9/Tox-11 and HP1/HP2 (15) in five laboratories, those described by Talabani et al. (16) in one laboratory, those described by Cassaing et al. (17) in one laboratory, and those described by Fekkar et al. (18) in one laboratory (Table 1). The laboratory-developed PCR assays were performed using a LightCycler 1.0 (Roche, Meylan, France) in three laboratories, a LightCycler 2.0 (Roche) in two laboratories, and a LightCycler 480 (Roche), an ABI Prism 7000, and an ABI 7500 (Applied Biosystems, Villebon-sur-Yvette, France) in one laboratory each. Real-time PCR amplification with the Bio-Evolution kit was performed as recommended by the manufacturer and was done using the same real-time PCR apparatus as the one used for the laboratory-developed PCR assay in each center. The volume of DNA extract added to the PCR mix was 5 μl (in seven laboratories) or 7 μl (in one laboratory) in the laboratory-developed methods and 5 μl for the commercial kit (as recommended by the manufacturer). The presence/absence of PCR inhibitors was tested using positive controls for each DNA-extracted sample with the laboratory-developed PCR assays and with the commercial kit (see Table 1 for details). Negative controls were included in each PCR run.

Data analysis.

All DNA extracts were tested in triplicate with the commercial kit and also in triplicate when possible with the laboratory-developed PCR assays. The detection of T. gondii was considered positive when at least one reaction tube was positive. Detection was considered negative when all three reactions were negative in the confirmed absence of PCR inhibitors. When the two types of detection (laboratory-developed PCR assays or the commercial kit) were performed at the same time in the same center and found positive in triplicate (all centers but center E, which performed only one replicate), the PCR crossing points (Cp) were compared in a Bland-Altman plot. For this, the mean Cp value obtained for each sample by each method (laboratory-developed methods and the commercial kit) was calculated, and the mean of these two values, as well as the difference between these two values, were plotted with the mean of the two methods in abscissa and the difference between the two methods in ordinate.

RESULTS AND DISCUSSION

Proficiency of the participating laboratories.

The eight participating laboratories are proficient in the molecular detection of T. gondii at the regional and national levels in France and also at an international level. Indeed, all of them are authorized by the French national authority Agence de la Biomédecine to perform prenatal diagnostics for congenital toxoplasmosis. They are also all members of the Molecular Biology Pole of the French National Reference Center for Toxoplasmosis (http://cnrtoxoplasmose.chu-reims.fr), and each of them has developed a high-performing laboratory-developed PCR assay (1, 6, 9–11, 16–21) that targets the repetitive DNA element rep529, shown to be the most efficient target to date for this diagnosis (10, 15, 17, 22–24) (for details, see Table 1; see also Table S1 in the supplemental material). This same working group earlier recommended that laboratories work toward a sensitivity threshold of 0.75 to 2.5 tachyzoites/ml of AF (10) and 100% specificity. All participating centers in this study were able to detect ≥5 tachyzoites (T) per ml of AF, as checked, for example, by yearly external quality assessments (25).

Description of the cohort.

One hundred fifty-seven DNA-extracted AF samples were included in the study; 140 were retrospective samples kept in biobanks, and 17 were prospectively enrolled (see Table S1 in the supplemental material). The storage conditions and the absence of effect of long-term conservation have been described elsewhere (26). The dates of maternal infection, according to gestational age, were established using serologic tests for 137/157 (87%) patients. According to the classification system proposed by the European Research Network on Congenital Toxoplasmosis (12), primary maternal infection during pregnancy was definite for 112/157 (71%) patients; 26%, 60%, and 14% of them were infected during the first, second, and third trimesters of pregnancy, respectively. The mean date of amniocentesis was 29 WA (19 WA to 42 WA), and the mean time interval between infection and amniocentesis was 8 weeks (0 to 28 weeks). Eleven and five samples collected by amniocentesis performed at 18 WA and during delivery, respectively, were excluded from these calculations (see Materials and Methods). Of note, the characteristics of the cohort should not be compared to those of previously published cohorts, because the AF samples were retrospectively selected in the biobanks (see Materials and Methods); therefore, these figures do not represent a natural situation. A definite final diagnosis of congenital toxoplasmosis in the fetus/infant was asserted in 84/157 (54%) cases. Further details about the cohort can be seen in Table S1 in the supplemental material.

Comparison of T. gondii detection with the commercial kit versus with the laboratory-developed PCR assays.

As the Bio-Evolution kit does not include an extraction step, each center used its own DNA extraction method. Among the 157 AF DNA extracts analyzed, 76 (48%) were found to be T. gondii positive using the laboratory-developed PCR assay of the corresponding laboratory at the time of the initial diagnosis, and all but nine that could not be tested again were confirmed in the second test performed in this study. Among these 76 samples, 75 were also found to be positive using the commercial kit (Table 2). The concordance between the laboratory-developed PCR assays and the commercial kit was 99%. Only one discrepant result was found; it was positive in all triplicates (Cp, 38.4 ≥ 0.6) at the time of initial diagnosis and positive in one reaction tube out of two (Cp, >45) in the confirming test performed here, but it was negative when tested by the commercial kit. It should be stressed that (i) only 40 cycles were performed with the commercial kit, and (ii) no PCR inhibitors were detected by the kit. Although some DNA degradation during preservation cannot be ruled out, the laboratory-developed method still detected T. gondii DNA at the time of the study. The 81 remaining samples were all classified as T. gondii negative using the laboratory-developed PCR assay and the commercial kit. All DNA extracts were found to be free of PCR inhibitors, using the positive controls of both the laboratory-developed PCR assays and the commercial kit. Taking patient follow-up into account, we used the classifications and definitions developed by the European Research Network on Congenital Toxoplasmosis (12) to determine the diagnostic performances of the assays used; the overall sensitivity of the eight laboratory-developed PCR assays and of the commercial kit with this cohort was estimated to be 86% (54/63) (Table 2). The specificity was 100% for all assays, and no false-positive results were detected by any method.

TABLE 2.

Comparison of the detection of T. gondii using laboratory-developed PCR assays and commercial kit: overall performances of the methods

Assay used	Detection of T. gondii		Performance (no. detected/total no. [% {95% CI}])a
	Positive	Negative	Sensitivity	Specificity
Laboratory-developed PCR assays	76	81	54/63 (86 [74–93])	29/29 (100 [85–100])
Commercial kit	75	82	54/63 (86 [74–93])	29/29 (100 [85–100])

^aTo calculate sensitivity and specificity, cases with a loss of follow-up were excluded; more PCR-negative than PCR-positive infants were lost during follow-up. See the supplemental data for follow-up and for the final diagnosis according to the Lebech et al. classification (12). CI, confidence interval.

Comparison of qualitative results and relative quantification between the commercial kit and the laboratory-developed PCR assays.

At and around the sensitivity threshold of a given PCR method, only a proportion of the reaction tubes appears positive, which implies that for very low concentrations of the pathogen, several PCRs have to be carried out for each experiment (to increase the probability of amplifying the pathogen DNA) (10, 20, 27–29). In the current study, inconsistently positive results were found in 7 occurrences by both the reference methods and the commercial kit.

To evaluate further the concordance between the laboratory-developed PCR assays and the commercial kit, the Cp values obtained using both methods were compared for each of the 63 positive (out of 76) samples and plotted in a Bland-Altman graph (Fig. 1). In all cases but two, the values were within the ±1.96 standard deviation interval, allowing us to conclude that the relative quantification using the commercial kit was in good agreement with all the laboratory-developed methods.

FIG 1.

Concordance between Cp values obtained using laboratory-developed methods and the Bio-Evolution commercial kit for the molecular detection of T. gondii. The Cp values obtained for each sample by each method (laboratory-developed method and commercial kit) were calculated for each center independently (A to H). For each sample, the means of the Cps obtained with both methods and the differences between the Cps obtained with both methods were calculated. To obtain the Bland-Altman plots, these values were plotted, with the mean of both methods in abscissa and the difference between both methods in ordinate for each center. The pink lines represent the means and ±1.96 standard deviations.

Evaluation of handiness and good laboratory practices.

The Bio-Evolution kit was felt as an easy-to-handle turnkey kit. Detailed and well-written instructions were included for its users. More importantly, with respect to good laboratory practices, the manufacturer (i) pointed out that the analysis should be performed in multiplicate and (ii) included positive and negative controls corresponding to 200 T. gondii genome equivalents/μl and distilled water (dH₂O), respectively. However, the commercial kit did not include the use of uracil-N-glycosylase (UNG) to limit carryover contaminations from previously amplified PCR products. In addition, all the components of the kit, i.e., positive controls, negative controls, and mixture vials, are stored in the same box. Six of the eight laboratories reported problems with the volumes of the reagents of the commercial kit, since the volume found was below the volume stated on the vials. Finally, the manufacturer's recommendation was to set the threshold manually in the real-time PCR analysis software (so-called “fit-point” method); however, the LightCycler software (Roche) offers the possibility of using an automated analysis (so-called “second derivative” method) that is unbiased and hence more reproducible than the fit-point method. This point has been amended in the newest version of the kit's instruction manual.

Conclusion.

In total, the commercial kit tested here for T. gondii detection by real-time PCR (Bio-Evolution reference no. BE-A997) is a well-designed and useful kit that leaves some room for improvement, but in our view, it represents an excellent tool for the molecular detection of T. gondii. The use of the recommended DNA target rep529 is likely an important factor of the efficiency of this kit, although that in itself is not sufficient to guarantee good results (11). Although, at the time of testing, it suffered from some defects in the reliability of the production of different reagents, this commercial method showed equivalent performances to those of eight finely optimized laboratory-developed PCR assays used in proficient French laboratories.