Abstract
We compared 14 molecular assays for their ability to detect the Mycobacterium tuberculosis complex in bronchoalveolar lavage fluid samples. Three approaches were followed. First, by using DNA from Mycobacterium bovis BCG, we determined the detection limits of the assays using routine molecular methods. Second, in order to determine the analytical sensitivities of the assays, we added one of four M. tuberculosis isolates with various numbers of the insertion sequence IS6110 to N-acetyl-l-cysteine (NALC)-NaOH-treated bronchoalveolar lavage fluid samples in dilutions of 1:10 to 1:10,000,000. Third, intertest variabilities were measured and defined by the standard deviations for the quantitation cycle (Cq) values of three positive test results per dilution per assay. The 14 assays tested had similar analytical sensitivities, except for GeneXpert, which had an analytical sensitivity that was 10- to 100-fold lower than that of the other assays. The MP MTB/NTM test and the in-house TaqMan-10 revealed the best performances for the detection limit and had the highest analytical sensitivities. Most of the tests performed well regarding detection limit and analytical sensitivity for the detection of the M. tuberculosis complex in serial dilutions, and the differences were small. The MP MTB/NTM and the in-house TaqMan-10 assays revealed the best, and GeneXpert the worst, overall performances.
INTRODUCTION
In 2011, 6.2 million patients were diagnosed with tuberculosis (TB) worldwide and reported this to the national tuberculosis control programs (NTP) (1). Control of this high-burden disease heavily depends on improved rapid diagnosis and optimal treatment. Real-time (RT)-PCR is the standard DNA amplification technique currently used in TB laboratories. In a short time span, a substantial number of commercial nucleic acid amplification tests for Mycobacterium tuberculosis complex (MTC) infections have become available. However, these assays have not been compared in a systematic manner. The clinical sensitivity of a part of the assays has been reported in only a few studies, using culture positivity and/or clinical diagnosis as the “gold standard” (2–6).
The insertion sequence IS6110 has long been appreciated as a target in the molecular detection of the M. tuberculosis complex in clinical material, and it is the most abundant IS element in the genome of M. tuberculosis. However, among clinical isolates worldwide, the number of IS6110 copies in the genome of M. tuberculosis varies from 0 to 25, potentially influencing the detection limits of related assays (7–10).
In this study, we compared the analytical performances of 14 assays for the molecular detection of MTC in three different approaches. We hypothesized that not all assays would have equally good analytical sensitivities and that the analytical sensitivity of assays targeting IS6110 would depend on the number of target copies in the genomes of M. tuberculosis strains studied. We therefore compared IS6110-targeting tests with those not targeting this element or that explored unknown or undisclosed targets.
MATERIALS AND METHODS
Assays.
Fourteen assays were compared, 9 of which are commercially available (Table 1). The assays were performed according to the instructions of each manufacturer. The cutoff values of the assays applied were those provided by the manufacturers.
Table 1.
Molecular assays tested in order of their target
| Target | Assay name and description (short name used in article) | Assay manufacturer (reference) |
|---|---|---|
| IS6110 | Exptl RT-PCR: 5 μl DNA input and Roche master mix (in-house Roche-5) | Roche Diagnostics Nederland BV, Almere, the Netherlands (21) |
| IS6110 | In-house RT-PCR IS6110: 5 μl DNA input and TaqMan universal PCR master mix, +AmpErase UNG (in-house TaqMan-5) | Applied Biosystems, Nieuwerkerk aan de Ijssel, the Netherlands (21) |
| IS6110 | In-house RT-PCR IS6110: 10 μl DNA input and TaqMan universal PCR master mix, +AmpErase UNG (in-house TaqMan-10) | Applied Biosystems (21) |
| IS6110 | MTB Q-PCR Alert kit detection of Mycobacterium tuberculosis DNA (Lucron) | Lucron ELITechGroup, Dieren, the Netherlands |
| IS6110 | RealAccurate Mycobacterium tuberculosis PCR kit (PathoFinder) | PathoFinder BV, Maastricht, the Netherlands |
| IS6110 | MTB real-time kit for detection of Mycobacterium tuberculosis complex (MP MTB kit) | Sacace Biotechnologies, Como, Italy, provided by MP Products, the Netherlands |
| IS6110 | MTB/NTM kit: multiplex MTC real-time PCR kit: AmpliSens MTC-FRT PCR kit (MP MTB/NTM) | InterLabService (ILS), Moscow, Russia, provided by MP Products, the Netherlands |
| IS6110 | Myco Direct 1.7, DNA based identification of Mycobacterium tuberculosis complex (MTUB) and other Mycobacteria (MOT) LCD array kit (Chipron)a | Chipron, GmbH, Berlin, Germany |
| 16S-23S rRNA gene | Internal transcribed sequence high-resolution melting (ITS-HRM) and Roche HRM master mix (in-house ITS-HRM) | Roche Diagnostics Nederland BV (11) |
| 16S rRNA of bp 584 | Cobas TaqMan MTB test CTM MTB (Cobas) | Roche Diagnostics Nederland BV |
| Direct repeat region of M. tuberculosis | Spoligotyping (spoligotyping)b | Homemade (22) |
| rpoB gene | GeneXpert MTB/RIF assay (GeneXpert) | Cepheid Benelux, Apeldoorn, the Netherlands (23) |
| Not disclosed by the manufacturer | AccuPower MTB real-time PCR kit (Goffin MTB) | Bioneer, South Korea, provided by Goffin, the Netherlands |
| Not disclosed by the manufacturer | AccuPower MTB&NTM real-time PCR kit (Goffin MTB/NTM) | Bioneer, South Korea, provided by Goffin |
The target-specific capture probes and the capture probes for the external control sequence Ctrl-EX1 together form a cross or a “+” sign.
Recorded positive if ≥3 spacers are present.
Internal transcribed sequence high-resolution melting.
Amplification of M. tuberculosis complex DNA was done using a real-time PCR with a high-resolution melting (HRM) mix (HRM master mix; Roche Diagnostics Nederland BV, Almere, the Netherlands). The amplification of a part of the internal transcribed spacer (ITS) spacer of 200 to 330 bp, which is located between the 16S and 23S gene spacer, was performed with the primers tb-ITS-fw and tb-ITS-re-rev1. For this amplification, no probe is used, so the sensitivity of this test solely depends on the primers. The melting curve results of the M. tuberculosis complex control were recorded as positive if the DNA denatured between 89 and 90°C. Therefore, when testing for M. tuberculosis, the results were considered positive when melting curves were between 89 and 90°C. Subsequently, the quantitation cycle (Cq) values were used for further calculation (11).
M. tuberculosis complex control.
The Mycobacterium bovis BCG strain is normally used as an internal control for the in-house PCR assay, which is used in our laboratory and contains a single copy of IS6110 (12, 13). One colony of this M. bovis BCG strain was added to PrepMan solution (PrepMan Ultra sample preparation reagent; Applied Biosystems, Nieuwerkerk aan de IJssel, the Netherlands) to prepare a turbid suspension. This suspension was incubated for 10 min at a temperature of 100°C to release DNA. Thereafter, a 1:10 dilution was prepared in Tris-EDTA (TE) buffer. This extracted DNA was used in all the PCRs for determining the detection limits (DLs).
Isolates.
Four M. tuberculosis isolates with a known number of IS6110 copies were selected from the M. tuberculosis isolate collection at the National Tuberculosis Reference Laboratory, each containing 1, 5, 10, or 20 copies of the IS6110 element. The four isolates were cultured to prepare a large set of identical bronchoalveolar lavage fluid (BALF) samples with quantified numbers of mycobacteria. Cultures were done in the MGIT Bactec 960 system (Becton, Dickinson, Sparks, MD).
Specimen processing.
We pooled redundant BALF specimens obtained from routine bronchoscopy procedures performed at several hospitals on various patients not suffering from any Mycobacterium infection. The BALF specimens were combined to a total volume of 800 ml and mechanically liquefied and homogenized with N-acetyl l-cysteine (NALC)-sodium hydroxide (NaOH) (14). After centrifugation, the treated BALF combination was concentrated, and this resulted in a total volume of 80 ml. The concentrated NALC-NaOH-treated BALF mixture was checked to be negative for MTC using the in-house IS6110 RT-PCR.
Strain processing.
Cultures were prepared and heat killed (40 min at 115°C). Heat killing of the strains was performed before and after mixing the bacteria with the NALC-NaOH-treated BALF mixture. The difference in the quantitation cycle (Cq) values in the in-house IS6110 RT-PCR for one strain between the two processes was never higher than 0.57 (1:10 dilution from an M. tuberculosis isolate with one IS6110 copy).
Serial dilution and DNA isolation.
A fixed number of heat-killed M. tuberculosis organisms was added to the NALC-NaOH-treated BALF mixture. Dilutions of 1:10 to 1:10,000,000 were made of this suspension, and the dilutions were frozen at −80°C in portions of 250 μl. From these diluted M. tuberculosis suspensions, 200 μl was extracted with NucliSens easyMAG (bioMérieux, Boxtel, the Netherlands). The extracted DNA was tested in the in-house IS6110 RT-PCR for reproducibility and was further used in a comparison of all the assays.
The reproducibilities of the DNA extraction and the in-house IS6110 RT-PCR were tested on the frozen material at three different time points, in duplicate. In this way, the means and coefficients of variance (CVs) for the Cq values of the eight different dilutions from the four isolates in duplicate were calculated. The difference in the mean Cq value was never higher than 1.08, and the CV was never higher than 2%.
Assay comparison.
The analytical sensitivities of the assays were compared in two different ways. First, we determined the detection limit (DL). For this purpose, we used a dilution series of the DNA of the M. bovis BCG strain that is used as a control in routine diagnostics. The DNA was diluted in TE buffer. The assay was tested in triplicate for each dilution. A higher dilution of the suspension was recorded as positive only if it tested positive three times in the respective assay. After a first negative result, the dilutions between the first negative dilution and the last positive dilution were tested as well.
Second, we determined the analytical sensitivity that captures both the process of DNA extraction and the use of the extracted DNA from the M. tuberculosis serial dilutions in the BALF mixture in RT-PCR. Again, the test of a higher dilution was recorded as positive only if the three results were all positive.
Additionally, as a third method of comparison, the intertest variability of the assays was assessed. The intertest variability of each assay was defined using the standard deviation (SD). The SD was calculated for the Cq values of the three positive test results per dilution per assay, using the results from the M. tuberculosis serial dilutions in BALF samples and not the results using the M. bovis BCG strain diluted in TE buffer for detection limit determination.
Statistics.
Descriptive statistical methods are provided in the methodological sections above.
RESULTS
Detection limit.
We used serial dilutions of M. bovis BCG DNA that is used routinely as a positive control in molecular assays and that contains one copy of IS6110 per genome. When comparing the detection limits, we considered the assay with a positive result at the greatest dilution to have the lowest detection limit. The results for assays targeting IS6110 and for assays using other genetic targets, or for tests with a target not disclosed by the manufacturer, are shown in Fig. 1. Four tests had the same low detection limit: these assays positively identified MTC at up to a 1:100 DNA dilution. The detection limit, independent of the type of target, was highest in the Lucron, PathoFinder, and in-house ITS-HRM assays. These assays amplified up to a 1:10 DNA dilution.
Fig 1.
Comparison of detection limits using dilution series of the DNA of M. bovis BCG that is used as positive control in routine diagnostics and contains one copy of IS6110. The x axis shows the dilution, which was recorded as positive if the assay tested positive three times; the longer the black bar, the better the test performance. The y axis shows the assays tested. The first eight assays target IS6110. Three other assays do not target this insertion sequence, but target another known target (see Table 1). For the two Goffin assays, the manufacturer does not disclose the target.
The detection limit of GeneXpert assay could not be calculated because this assay can be performed only with clinical intact material and not with purified DNA.
Analytical sensitivity in serial dilutions with different numbers of IS6110 elements.
The second approach for comparing the assays was to analyze the DNA from the different M. tuberculosis strains with various numbers of IS6110 copies, serially diluted in a BALF mixture. The results are shown in Fig. 2.
Fig 2.
Comparison of analytical sensitivities of the assays tested with dilutions of DNA isolated from strains with various numbers of IS6110 copies in BALF samples. The x axis shows the dilution, which was recorded as positive if an assay tested positive three times for the specific assay. The y axis shows the assays tested. The first eight assays target IS6110. The next four assays are known to target IS6110. For the two Goffin assays, the manufacturer does not disclose the target.
Of the assays targeting IS6110 in the dilutions with the strain carrying one copy of IS6110, the in-house Roche-5 assay was able to detect M. tuberculosis DNA to a 1:10,000 dilution. All other assays were able to detect M. tuberculosis DNA to a 1:100,000 dilution, and the MP MTB was even able to detect M. tuberculosis DNA to a 1:1,000,000 dilution.
The highest analytical sensitivity with the dilution of the five-IS6110-copy strain was found for the PathoFinder assay, which detected M. tuberculosis DNA to a 1:1,000,000 dilution. All other assays detected M. tuberculosis DNA to a 1:100,000 dilution.
With dilutions of the 10-IS6110-copy strain, both the in-house Roche-5 and the in-house TaqMan-5 assays detected M. tuberculosis DNA to a 1:100,000 dilution of. All other assays were able to detect M. tuberculosis DNA to a 1:1,000,000 dilution.
With dilutions of the 20-IS6110-copy strain, the MP MTB/NTM and in-house TaqMan-10 assays were able to detect M. tuberculosis DNA up to a 1:10,000,000 dilution. All other assays were able to detect M. tuberculosis DNA to a 1:1,000,000 dilution.
All assays except the MP MTB had increased analytical sensitivity for multicopy IS6110 strains.
The analytical sensitivities of the assays not targeting IS6110 yielded comparable results for the assays and were, as expected, not dependent on the number of IS6110 copies per genome in the strains. An exception was the GeneXpert assay, which had an analytical sensitivity that was 10-fold lower than that of the other assays. The analytical sensitivity of the Goffin MTB/NTM assay was higher if 10 IS6110 copies were present in the M. tuberculosis strain tested. For dilutions of the strain with 20 copies of IS6110, the Cobas, Goffin MTB, and spoligotyping tests were able to detect M. tuberculosis DNA at a one-log-higher dilution than for strains containing fewer copies of IS6110.
Intertest variability of the molecular assays.
The intertest variability of each assay was defined using the standard deviation (SD). The SD was calculated for the Cq values of the three positive test results per dilution per assay, using results from the M. tuberculosis serial dilutions in the BALF mixture.
The results of three different dilutions are depicted in Tables 2 to 5. The first results are for the undiluted series. The second results are for a dilution of 1:10,000, the dilution in which the first assay (in-house Roche-5) reached its analytical sensitivity (Fig. 2). The last results are for the dilutions at which the separate assays reached the analytical sensitivity (AS) (which was a different level depending on the assay).
Table 2.
Intertest variability of assays targeting IS6110, calculated by standard deviation of the Cq values
| Assay | Dilution | Result with IS6110 copy no. of: |
|||
|---|---|---|---|---|---|
| 1 | 5 | 10 | 20 | ||
| Expt 5a | Undiluted | 0.19 | 0.29 | 0.06 | 0.07 |
| 1:10,000 | 0.26 | 0.41 | 0.06 | 0.16 | |
| ASb (1:10,000) | 0.26 | 1.53 | 0.46 | 1.08 | |
| In-house TaqMan-5 | Undiluted | 0.04 | 0.39 | 0.04 | 0.01 |
| 1:10,000 | 0.12 | 0.08 | 0.07 | 0.05 | |
| AS | 0.77 | 0.48 | 0.18 | 0.25 | |
| In-house TaqMan-10 | Undiluted | 0.07 | 0.05 | 0.02 | 0.13 |
| 1:10,000 | 0.10 | 0.02 | 0.07 | 0.03 | |
| AS | 0.60 | 0.29 | 0.36 | 0.4 | |
| Lucron | Undiluted | 0.05 | 0.03 | 0.04 | 0.26 |
| 1:10,000 | 0.5 | 0.33 | 0.07 | 0.2 | |
| AS | 1.21 | 0.14 | 0.44 | 0.61 | |
| PathoFinder | Undiluted | 0.03 | 0.01 | 0.08 | 0.02 |
| 1:10,000 | 0.43 | 0.10 | 0.05 | 0.04 | |
| AS | 0.47 | 0.39 | 0.40 | 0.23 | |
| MP MTB | Undiluted | 0.4 | 0.30 | 0.07 | 0.07 |
| 1:10,000 | 0.09 | 0.22 | 0.16 | 0.10 | |
| AS | 0 | 0.06 | 1.03 | 0.23 | |
| MP MTB/NTMa | Undiluted | 0.03 | 0.03 | 0.06 | 0.04 |
| 1:10,000 | 3.2 | 1.00 | 0.07 | 0.04 | |
| AS | 0.57 | 0.27 | 0.69 | 0.39 | |
Assays with highest intertest variability according to the standard deviation (SD).
AS, analytical sensitivity.
Table 5.
Intertest variability of the probes of GeneXpert, calculated by standard deviation of the Cq values
| GeneXpert probe and laboratory testing location | Dilution | Result with IS6110 copy no. of: |
|||
|---|---|---|---|---|---|
| 1 | 5 | 10 | 20 | ||
| A | Undiluted | 1.2 | 2.0 | 2.8 | 1.1 |
| Lab 1 | 1:1,000 | 1.0 | 0.7 | 1.0 | 1.4 |
| Lab 2 | 1:1,000 | NCa | 2.7 | 1.5 | 8.2 |
| Lab 1b | 1:10,000 | 0.9 | 0.8 | 2.8 | 1.8 |
| Lab 2 | 1:10,000 | 1.3 | 1.3 | NDc | 1.1 |
| B | Undiluted | 1.4 | 2.1 | 2.9 | 1.6 |
| Lab 1 | 1:1,000 | 1.0 | 0.7 | 1.1 | 1.3 |
| Lab 2 | 1:1,000 | NC | 2.5 | 1.3 | 7.3 |
| Lab 1 | 1:10,000 | 1.0 | 0.7 | 2.3 | 1.9 |
| Lab 2 | 1:10,000 | 1.0 | 1.8 | ND | 1.0 |
| C | Undiluted | 1.1 | 2.1 | 3.0 | 1.0 |
| Lab 1 | 1:1,000 | 1.0 | 0.7 | 1.1 | 1.4 |
| Lab 2 | 1:1,000 | NC | 2.7 | 1.4 | 8.3 |
| Lab 1 | 1:10,000 | 0.9 | 0.8 | 2.3 | 1.7 |
| Lab 2 | 1:10,000 | 1.3 | 1.6 | ND | 0.9 |
| D | Undiluted | 1.1 | 1.7 | 2.8 | 1.0 |
| Lab 1 | 1:1,000 | 1.1 | 0.6 | 1.0 | 1.3 |
| Lab 2 | 1:1,000 | NC | 2.5 | 1.4 | 8.0 |
| Lab 1 | 1:10,000 | 0.9 | 0.8 | 2.0 | 1.7 |
| Lab 2 | 1:10,000 | 1.1 | 1.5 | ND | 1.1 |
| E | Undiluted | 1.3 | 2.3 | 3.0 | 0.8 |
| Lab 1 | 1:1,000 | 1.1 | 0.7 | 1.0 | 1.3 |
| Lab 2 | 1:1,000 | NC | 2.8 | 1.6 | ND |
| Lab 1 | 1:10,000 | 0.9 | 0.9 | 2.3 | 1.8 |
| Lab 2 | 1:10,000 | 1.6 | 1.4 | ND | 1.2 |
NC, one of the three runs gave an error and thus the interest variability could not be calculated.
GeneXpert reached its analytical sensitivity with a dilution of 1:10,000 in lab 1.
ND, at least one run could not be detected; thus, the analytical sensitivity was not reached at this dilution.
Table 3.
Intertest variability of assays not targeting IS6110, calculated by standard deviation of the Cq values
| Assay | Dilution | Result with IS6110 copy no. of: |
|||
|---|---|---|---|---|---|
| 1 | 5 | 10 | 20 | ||
| In-house ITS-HRM | Undiluted | 0.04 | 0.06 | 0.06 | 0.06 |
| 1:10,000 | 0.25 | 0.61 | 0.64 | 0.15 | |
| AS | 1.36 | 0.53 | 0.73 | 0.20 | |
| Cobas | Undiluted | 0.1 | 0.25 | 0.1 | 0.06 |
| 1:10,000 | 0.25 | 0.2 | 0.31 | 0.21 | |
| AS | 0.46 | 1.06 | 0.72 | 1.93 | |
Table 4.
Intertest variability of assays with a target not disclosed by the manufacturer, calculated by standard deviation of the Cq values
| Assay | Dilution | Result with IS6110 copy no. of: |
|||
|---|---|---|---|---|---|
| 1 | 5 | 10 | 20 | ||
| Goffin MTB | Undiluted | 0.03 | 0.02 | 0.06 | 0.03 |
| 1:10,000 | 0.18 | 0.04 | 0.08 | 0.11 | |
| AS | 0.97 | 0.43 | 0.23 | 0.20 | |
| Goffin MTB/NTM | Undiluted | 0.05 | 0.05 | 0.09 | 0.42 |
| 1:10,000 | 0.29 | 0.14 | 0.14 | 0.15 | |
| AS | 1.15 | 0.40 | 0.82 | 0.47 | |
For all assays, the intertest variability was within a standard deviation of 2. Only a disproportionately high intertest variability (SD, 3.2) was found for the MP MTB/NTM assay for the 1:10,000 dilutions containing one copy of IS6110.
Calculating the intertest variability was not possible for the GeneXpert assay as a sole test, due to its five probes. All probes had their own Cq values and had to give a positive result to score the overall result as positive. After the results for the GeneXpert assay in this laboratory became known, we retested GeneXpert in another laboratory. We did this to calculate the intertest variability of GeneXpert with isolates at dilutions of 1:1,000 and 1:10,000 in triplicate. The results of the intertest variability of the probes themselves from both laboratories are shown in Table 5.
DISCUSSION
This is the first study to describe a direct comparison of 14 different PCR assays for the molecular detection of M. tuberculosis complex in clinical material. We assessed the sensitivities of the assays using two different approaches, as well as studied intertest variability. All assays performed equally well regarding detection limit and analytical sensitivity for the detection of serial dilutions of M. tuberculosis suspensions with various numbers of IS6110 elements; the positive exceptions were the in-house TaqMan-10 and the MP MTB/NTM assays. A more clear discrepancy between the results in both approaches was observed for the Lucron and PathoFinder tests. The intertest variability of all assays tested was within 2 standard deviations, and their performances were thus comparable.
For the first comparison approach, assays targeting IS6110 showed the lowest detection limits. In a comparison of the overall detection limits for all assays with the analytical sensitivities, all assays performed similarly, except for the Lucron and PathoFinder assays. An explanation for this might be the lower DNA input in these two assays (5 μl DNA) compared to that of the best-performing assays (10 μl DNA). Another possible explanation is that the DNA extraction method (PrepMan for the DL and easyMAG for the analytical sensitivity) yields differences in the purity of the DNA, and less-pure DNA might yield a higher detection limit (15). To our knowledge, no direct comparison of the PrepMan and easyMAG extraction procedures for M. tuberculosis has been performed so far.
All assays targeting IS6110 had increasing analytical sensitivity with an increasing number of IS6110 copies in the strains tested, which is in accordance with our hypothesis. Only the MP MTB assay yielded unexpected results. This kit had an analytical sensitivity with the dilution of the one-copy IS6110 strain, which was better compared to that of the five-copy IS6110 strain. Both in-house TaqMan-10 and MP MTB/NTM had the lowest detection limits as well as the highest analytical sensitivities with the dilution of the 20-copy IS6110 strain. These were the only assays yielding fully concordant results between the detection limit analysis and the analytical sensitivity. Assays not targeting IS6110 yielded comparable results with both approaches. All assays had a better analytical sensitivity at the dilutions of the 20-IS6110-copy strain, except for the in-house ITS-HRM and GeneXpert assays. Since the analytical sensitivity of the Goffin MTB/NTM assay, which has an undisclosed molecular target, performed better if more IS6110 copies were present in the test strain, this assay probably also targets IS6110.
Intertest variability was the third study subject and was tested on the basis of triplicate results from the serial dilutions in a BALF mixture. Overall, no assay was found to under- or outperform another. Nevertheless, the highest intertest variability of the assays, independent of their target and the tested dilution, was found with the MP MTB/NTM assay at the 1:10,000 dilutions. One of the three runs of this assay had a Cq value at this dilution, which is used as cutoff value by the manufacturer. The result was still positive but had a difference of 5 Cq values compared to those of the other two runs. After the results were analyzed, this was the only divergent run of all the runs tested for each PCR assay included, for which we found no obvious laboratory-related explanation. Tests that have greater intertest variability obviously are more prone to produce more divergent results.
We checked our methodology in this study regarding the even distribution of M. tuberculosis in the BALF samples, the procedure of heat killing of the strains, and the efficacy of DNA isolation. The homogenized and concentrated NALC-NaOH-treated BALF mixture was checked to be negative for MTC using the in-house IS6110 RT-PCR. Comparing the amplification of DNA before and after heat killing tested the influence of this crucial step on DNA isolation and the even distribution of M. tuberculosis in BALF samples. Furthermore, the reproducibility of the DNA isolation was tested at three different time points. All differences in the Cq values of the heating procedure and the reproducibility were below the 0.5-log10 difference. A difference above this value is considered to be clinically relevant (16, 17). We therefore conclude that our methodology of DNA isolation of the strains by using heat-killed bacilli, the even distribution of MTC, and the reproducibility was appropriate.
The focus of this study was on the analytical sensitivity of the assays. A comparison of clinical sensitivities was not performed on routine clinical samples, such as sputum, because the quantities of these samples are normally low, especially after NALC-NaOH treatment and homogenization of this material. Also, the distribution of M. tuberculosis DNA in sputum is generally not homogeneous. Therefore, a comparison of 14 PCR assays using sputum samples is virtually impossible and would have yielded irreproducible sensitivity results for each assay. A benefit of our comparison using one pool of BALF samples is that this is highly comparable for each assay. This material was used to add cultured bacteria in serial dilutions in order to mimic as much as possible the clinical context for patients with difficult-to-diagnose TB. For future evaluation of novel molecular assays to be compared under the same conditions, this material has been stored at −80°C.
In our current analysis comparing 14 assays, the GeneXpert assay showed a lower analytical sensitivity than all other assays. However, it is likely that after the suspension was heated, frozen, and thawed, some of the bacterial cells were already lysed. As this is the only assay purifying whole bacterial cells from clinical material, this may have negatively influenced its performance in comparison to the methods that directly isolate and purify DNA. This hypothesis is a subject for further research, which is planned for the near future. As the individualized probes of the GeneXpert assay sometimes revealed intertest variability exceeding 2 standard deviations, we retested the GeneXpert intertest variability in another laboratory with isolates that were kept at −80°C. In this second laboratory, the intertest variability for the individualized probes was even greater than in the first laboratory.
Our study also evaluated the dependence of the assays on the number of IS6110 copies in the genome of the MTC species detected in the BALF samples. It is known that especially in Asia, IS6110 is absent in a considerable number of M. tuberculosis isolates (18). In the Netherlands, over a 5-year period, the IS6110 element was absent in only 3 isolates of a total of 3,884 isolates. As expected, our study showed that the assays targeting IS6110 indeed have a higher sensitivity with larger numbers of IS6110 copies. In any case, the results in PCR targeting IS6110 in MTC will vary according to the number of targets present in the detected bacteria, as now has been proven in this study.
Clinical sensitivity is widely used as a measure for evaluating the performance of a PCR assay. The analytical sensitivity as used in our study has, in contrast to clinical sensitivity, hardly been used for comparing the performance of PCR assays (2–6). However, a comparison of data regarding clinical sensitivity must be interpreted with caution because clinical sensitivity is calculated by comparing results with culture results or with the clinical diagnosis. Comparing the clinical sensitivities of assays is important in smear-negative culture-positive samples, as smear-positive but culture-positive samples already are known to have high clinical sensitivity due to a higher bacterial load (19, 20). Research focused on comparing the clinical sensitivities of selected assays in smear-negative culture-positive samples, using the same clinical specimens, is therefore needed.
In conclusion, comparing the sensitivities of PCR assays is most reliable by analyzing the analytical sensitivities and the detection limits. The detection limits were nearly identical compared to the results of the analytical sensitivities in this study, except for the results for the Lucron and PathoFinder assays. As expected, the analytical sensitivity was better for strains with more copies of IS6110 elements. GeneXpert had the lowest analytical sensitivity and was at least 10-fold less sensitive than the other assays. In our study, the in-house TaqMan-10 and MP MTB/NTM assays had the lowest detection limits and the highest analytical sensitivities while testing dilutions of the 20-IS6110-copy strain. Overall, intertest variability was within 2 standard deviations and was comparable for all assays, except for the individualized probes of the GeneXpert assay, which sometimes had intertest variability exceeding 2 standard deviations.
ACKNOWLEDGMENTS
We thank Vera Tuchter for her contribution to testing the assays. We also thank Roche, Bioneer, Chipron, MP Products, Lucron ELITechGroup, and PathoFinder BV for providing free kits for this study.
Footnotes
Published ahead of print 21 August 2013
REFERENCES
- 1.World Health Organization 2012. Global tuberculosis report 2012. World Health Organization, Geneva, Switzerland: http://www.who.int/tb/publications/global_report/en/ [Google Scholar]
- 2.Cho SY, Kim MJ, Suh JT, Lee HJ. 2011. Comparison of diagnostic performance of three real-time PCR kits for detecting Mycobacterium species. Yonsei Med. J. 52:301–306 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hur M, Moon HW, Yun YM, Kang TY, Kim HS, Kim HS, Lee KM, Kang SH, Lee EH. 2011. Detection of tuberculosis using artus M. tuberculosis PCR kit and Cobas Amplicor Mycobacterium tuberculosis test. Int. J. Tuberc. Lung Dis. 15:795–798 [DOI] [PubMed] [Google Scholar]
- 4.Greco S, Girardi E, Navarra A, Saltini C. 2006. Current evidence on diagnostic accuracy of commercially based nucleic acid amplification tests for the diagnosis of pulmonary tuberculosis. Thorax 61:783–790 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Goessens WH, de Man P, Koeleman JG, Luijendijk A, te Witt R, Endtz HP, van Belkum A. 2005. Comparison of the Cobas Amplicor MTB and BDProbeTec ET assays for detection of Mycobacterium tuberculosis in respiratory specimens. J. Clin. Microbiol. 43:2563–2566 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Scarparo C, Piccoli P, Rigon A, Ruggiero G, Scagnelli M, Piersimoni C. 2000. Comparison of enhanced Mycobacterium tuberculosis amplified direct test with Cobas Amplicor Mycobacterium tuberculosis assay for direct detection of Mycobacterium tuberculosis complex in respiratory and extrapulmonary specimens. J. Clin. Microbiol. 38:1559–1562 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Thierry D, Cave MD, Eisenach KD, Crawford JT, Bates JH, Gicquel B, Guesdon JL. 1990. IS6110, an IS-like element of Mycobacterium tuberculosis complex. Nucleic Acids Res. 18:188. 10.1093/nar/18.1.188 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Thierry D, Brisson-Noël A, Vincent-Lévy-Frébault V, Nguyen S, Guesdon JL, Gicquel B. 1990. Characterization of a Mycobacterium tuberculosis insertion sequence, IS6110, and its application in diagnosis. J. Clin. Microbiol. 28:2668–2673 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Eisenach KD, Crawford JT, Bates JH. 1988. Repetitive DNA sequences as probes for Mycobacterium tuberculosis. J. Clin. Microbiol. 26:2240–2245 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cave MD, Eisenach KD, McDermott PF, Bates JH, Crawford JT. 1991. IS6110: conservation of sequence in the Mycobacterium tuberculosis complex and its utilization in DNA fingerprinting. Mol. Cell. Probes 5:73–80 [DOI] [PubMed] [Google Scholar]
- 11.Roth A, Reischl U, Streubel A, Naumann L, Kroppenstedt RM, Habicht M, Fischer M, Mauch H. 2000. Novel diagnostic algorithm for identification of mycobacteria using genus-specific amplification of the 16S-23S rRNA gene spacer and restriction endonucleases. J. Clin. Microbiol. 38:1094–1104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.van Soolingen D, Hermans PW, de Haas PE, Soll DR, van Embden JD. 1991. Occurrence and stability of insertion sequences in Mycobacterium tuberculosis complex strains: evaluation of an insertion sequence-dependent DNA polymorphism as a tool in the epidemiology of tuberculosis. J. Clin. Microbiol. 29:2578–2586 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Collins DM, Erasmuson SK, Stephens DM, Yates GF, De Lisle GW. 1993. DNA fingerprinting of Mycobacterium bovis strains by restriction fragment analysis and hybridization with insertion elements IS1081 and IS6110. J. Clin. Microbiol. 31:1143–1147 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kubica GP, Dye WE, Cohn ML, Middlebrook G. 1963. Sputum digestion and decontamination with N-acetyl-l-cysteine-sodium hydroxide for culture of mycobacteria. Am. Rev. Respir. Dis. 87:775–779 [DOI] [PubMed] [Google Scholar]
- 15.Aldous WK, Pounder JI, Cloud JL, Woods GL. 2005. Comparison of six methods of extracting Mycobacterium tuberculosis DNA from processed sputum for testing by quantitative real-time PCR. J. Clin. Microbiol. 43:2471–2473 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gale H. 2000. Evaluation of the quantiplex human immunodeficiency virus type 1 RNA 3.0 assay in a tertiary-care center. Clin. Diagn. Lab. Immunol. 7:122–124 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jennings C, Harty B, Granger S, Wager C, Crump JA, Fiscus SA, Bremer JW. 2012. Cross-platform analysis of HIV-1 RNA data generated by a multicenter assay validation study with wide geographic representation. J. Clin. Microbiol. 50:2737–2747 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Radhakrishnan I, K MY, Kumar RA, Mundayoor S. 2001. Implications of low frequency of IS6110 in fingerprinting field isolates of Mycobacterium tuberculosis from Kerala, India. J. Clin. Microbiol. 39:1683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Allen JL. 1992. A modified Ziehl-Neelsen stain for mycobacteria. Med. Lab Sci. 49:99–102 [PubMed] [Google Scholar]
- 20.Sepkowitz KA, Raffalli J, Riley L, Kiehn TE, Armstrong D. 1995. Tuberculosis in the AIDS era. Clin. Microbiol. Rev. 8:180–199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Savelkoul PH, Catsburg A, Mulder S, Oostendorp L, Schirm J, Wilke H, van der Zanden AG, Noordhoek GT. 2006. Detection of Mycobacterium tuberculosis complex with real time PCR: comparison of different primer-probe sets based on the IS6110 element. J. Microbiol. Methods 66:177–180 [DOI] [PubMed] [Google Scholar]
- 22.Kamerbeek J, Schouls L, Kolk A, van Agterveld M, van Soolingen D, Kuijper S, Bunschoten A, Molhuizen H, Shaw R, Goyal M, van Embden J. 1997. Simultaneous detection and strain differentiation of Mycobacterium tuberculosis for diagnosis and epidemiology. J. Clin. Microbiol. 35:907–914 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Helb D, Jones M, Story E, Boehme C, Wallace E, Ho K, Kop J, Owens MR, Rodgers R, Banada P, Safi H, Blakemore R, Lan NTN, Jones-López EC, Levi M, Burday M, Ayakaka I, Mugerwa RD, McMillan B, Winn-Deen E, Christel L, Dailey P, Perkins MD, Persing DH, Alland D. 2010. Rapid detection of Mycobacterium tuberculosis and rifampin resistance by use of on-demand, near-patient technology. J. Clin. Microbiol. 48:229–237 [DOI] [PMC free article] [PubMed] [Google Scholar]