Evaluation of the BDProbeTec ET System for Direct Detection of Mycobacterium tuberculosis in Pulmonary and Extrapulmonary Samples: a Multicenter Study

Abstract

We evaluated the BDProbeTec ET system (Becton Dickinson, Sparks, Md.), a strand displacement amplification-based technique, for direct detection of Mycobacterium tuberculosis in 867 clinical samples. Of 294 extrapulmonary specimens, 52 had positive results by both BDProbeTec ET and culture and 209 had negative results by both methods; sensitivity and specificity were 76.5 and 95.9%, respectively. After resolution of discrepancies, the sensitivity rose to 77.8%.

Genetic amplification techniques for the detection of Mycobacterium tuberculosis first appeared as commercial diagnostic tools about 10 years ago. Although the sensitivities of methods detecting M. tuberculosis are much lower than those achieved for other microorganisms, they are widespread in diagnostic laboratories. The need for standardization and automation has become therefore a major issue, which has been addressed by the leading diagnostic firms. Becton Dickinson (Sparks, Md.) has recently introduced worldwide (except for the United States, where it is waiting for Food and Drug Administration approval) the new BDProbeTec ET system. The new BDProbeTec ET system allows amplification and detection of M. tuberculosis complex (MTBC) DNA in as little as 1 h and simultaneously detects the presence of inhibitors as well. The target of the BDProbeTec ET system is a 95-bp region of IS6110, a highly specific insertion element in the MTBC DNA where it is present in multiple copies. Nucleic acid amplification is isothermal and is based on homogeneous strand displacement amplification (SDA) (19, 24), while detection is based on real-time fluorescent energy transfer (10; J. M. Harris, T. Brink, Jr., S. Buston, D. W. Copertino, G. Franklin, K. Kelly, O. Llorin, and K. Yanson, Abstr. 99th Gen. Meet. Am. Soc. Microbiol., abstr. U25, 1999). An internal amplification control (IAC) is run with each sample to confirm the validity of the amplification reaction and to identify potential inhibitory factors from the processed specimen.

The aim of this study was to assess the reliability of the new amplification assay by comparing it with culture test results from respiratory samples and from specimens having an extrapulmonary origin.

The trial was performed in three different laboratories. Laboratory A tested all consecutive samples, while laboratory B tested the samples from patients with high clinical suspicion of tuberculosis and laboratory C mainly tested nonrespiratory samples. Clinical samples (867 samples from 632 patients) included 573 respiratory and 294 nonrespiratory specimens. The major nonrespiratory samples were urine (n = 88), biopsy (n = 64), pleural fluid (n = 46), pus (n = 37), and cerebrospinal fluid (n = 28) samples. Upon arrival at the laboratory, cerebrospinal fluid samples were concentrated by centrifugation, while all other samples were digested and concentrated by the N-acetyl-l-cysteine-NaOH method (2% NaOH). The sediment was stained by auramine-rhodamine for the microscopic evaluation of acid-fast bacilli and cultured on Lowenstein-Jensen medium and in a liquid medium.

The isolates were identified by hybridization with commercial DNA probes (AccuProbe [Gen-Probe, San Diego, Calif.] and INNO LiPA Mycobacteria [Innogenetics, Ghent, Belgium]) (20, 23). Species not identified with these probes were identified by high-performance liquid chromatography (21) assisted, in some cases, by biochemical tests. BDProbeTec ET processing was performed the day after or, whenever impossible (i.e., in about 50% of cases), on the digested material stored at −20°C for no more than 50 days.

The SDA assay was performed according to the manufacturer's directions (package insert; Becton Dickinson). When the amplification signal, converted to MOTA (metric other than acceleration) units by the instrument, was greater than 3,400, the tests were considered positive regardless of the IAC values. Values lower than 3,400 were considered negative when the IAC value was greater than 5,000 and indeterminate when the IAC value was less than 5,000. In the latter case, samples were retested, and the results of the repeated tests were reported. In those cases where the results of DNA amplification and culture disagreed, tests were repeated, provided enough sample was available; for the determination of specificity, sensitivity, and likelihood ratios (LR) (see below), however, only the results achieved before repetition were used. Since the BDProbeTec ET system is approved for testing respiratory specimens only, two separate statistical evaluations of the results were made for pulmonary and extrapulmonary samples. Statistical analysis included the determination of specificity, sensitivity, and positive and negative LR. The latter test was preferred to positive and negative predictive values which, being strongly dependent on prevalence (4), are not appropriate in studies like ours and in most of the studies aimed at the validation of diagnostic tests, which are not designed to obtain valid estimates of prevalence. The significance of the difference in sensitivity between respiratory and nonrespiratory samples was assessed by the χ² test.

Acid-fast bacilli were microscopically detected in 204 samples (one sample was negative by culture), while isolates grew in 313 cultures. A large majority of the strains (n = 252) belonged to the MTBC. The remaining isolates belonged to the following species: Mycobacterium chelonae (n = 20), members of the Mycobacterium avium complex (n = 15), Mycobacterium fortuitum (n = 7), Mycobacterium kansasii (n = 7), Mycobacterium xenopi (n = 6), Mycobacterium gordonae (n = 3), Mycobacterium lentiflavum (n = 1), Mycobacterium mucogenicum (n = 1), and Mycobacterium terrae (n = 1) (Table 1). Sixteen specimens with negative results by BDProbeTec ET and three specimens with positive results by BDProbeTec ET yielded contaminated cultures and were eliminated from the study. Seven samples initially failed the IAC amplification, but while six of these samples gave negative results when the test was repeated, the seventh could not be retested because the sample was insufficient and was therefore excluded from the study.

TABLE 1.

Positive results achieved with microscopy, culture, and the BDProbeTec ET system in respiratory and nonrespiratory samples which yielded mycobacteria in culture

Species	No. of specimens with positive results by:
	Microscopy		Culture		BDProbeTec ET
	Respiratory specimens	Nonrespiratory specimens	Respiratory specimens	Nonrespiratory specimens	Respiratory specimens	Nonrespiratory specimens
M. tuberculosis complex	135	28	184	68	168	52
M. chelonae	11	5	15	5	1
M. avium complex	9	1	13	2
M. fortuitum	2	1	5	2
M. kansasii	4	2	5	2
M. xenopi	2	2	4	2		1
M. gordonae		1	2	1
M. mucogenicum			1
M. terrae			1
M. lentiflavum			1
Total	163	40	231	82	169	53

Of 252 MTBC-positive cultures, 163 of which were from specimens with positive results by microscopic evaluation, 220 gave positive results and 32 gave negative results with the BDProbeTec ET system. Of the samples yielding MTBC-negative cultures, 579 gave negative results and 16 (2 of which grew M. chelonae and M. xenopi) gave positive results with BDProbeTec ET. Of 561 respiratory samples, 96% gave concordant results by both the BDProbeTec ET and culture, while 4% (n = 23) yielded discordant results. For all these samples, the sensitivity and specificity of BDProbeTec ET were 91.3 and 98.1%, respectively, and positive and negative LRs were 49.2 and 0.1. As expected, sensitivity was clearly higher among the smear microscopy-positive samples (99.2%) than among the microscopy-negative ones (70.6%).

The sensitivity (76.5%) was significantly lower (P < 0.01) for the 286 extrapulmonary specimens than for the pulmonary specimens (Table 2), with 91% concordance between amplification and culture results, but the specificities (95.9 and 98.1%) were similar. Again, sensitivity (90%) was significantly higher for smear-positive samples than for smear-negative samples (65.8%). Separate analysis of the most frequent types of extrapulmonary specimens revealed sensitivity values unexpectedly high for urine and low for cerebrospinal fluid samples; however, the latter data may be biased due to the limited number of positive samples (Table 3).

TABLE 2.

Sensitivity, specificity, and LRs of the BDProbeTec ET system on nonrespiratory specimens, in comparison with culture, before and after resolution of discrepancies

Specimen type	No. of specimens with the following results:				Sensitivity (%)	Specificity (%)	Positive LR	Negative LR
	Culture positive		Culture negative
	BDa positive	BD negative	BD positive	BD negative
Before resolution of discrepancies
Total	52	16	9	209	76.5	95.9	18.5	0.2
Smear positive	27	3	1	11	90	91.7	10.8	0.1
Smear negative	25	13	8	198	65.8	96.1	16.9	0.3
After resolution of discrepancies
Total	56	16	5	209	77.8	97.7	33.3	0.2
Smear positive	27	3	1	11	90	91.7	10.8	0.1
Smear negative	29	13	4	198	69	98	34.9	0.3

^aBD, BDProbeTec ET system.

TABLE 3.

Sensitivity and specificity of the BDProbeTec ET system in comparison with culture when major extrapulmonary specimen types are considered separately

Specimen type	No. of specimens with the following results:				Sensitivity (%)	Specificity (%)
	Culture positive		Culture negative
	BDa positive	BD negative	BD positive	BD negative
Urine	11	1	2	68	91.7	97.1
Biopsy	14	4	1	45	77.8	97.8
Pleural fluid	2	1	1	42	66.7	97.7
Pus	12	6	3	16	66.7	84.2
Cerebrospinal fluid	1	2	1	24	33.3	96

^aBD, BDProbeTec ET system.

A total of 48 discrepant results were noted; 16 were false-positive results, and 32 were false-negative results. The results of the repeated tests, performed when there was sufficient sample, again gave discrepant results with the culture results in 13 cases, while in 3 cases, the repeated test results agreed. Of the patients whose samples gave false-positive BDProbeTec ET results, one had a further positive culture and five (seven cultures) had been previously diagnosed with tuberculosis and were under treatment; all these samples were therefore considered true-positive results. Because of the resolution of the above discrepancies, the overall sensitivity and specificity rose for both respiratory and nonrespiratory specimens. The increase in sensitivity, which was negligible for respiratory samples (from 91.3 to 91.5%), was more pronounced for nonrespiratory specimens (from 76.5 to 77.8%) and particularly for the smear-negative specimens (from 65.8 to 69%).

Of the published studies on the BDProbeTec system, some studies were of a previous version of the BDProbeTec system (2, 8, 13); only two studied the BDProbeTec ET system (1, 9). The results of these two studies are similar to ours, with sensitivities ranging from 82.7% (9) to 100% (2) for respiratory samples and with the specificities ranging from 96.5% (13) to 99.8% (8). A major goal of our investigation was to test the reliability of the BDProbeTec ET assay with extrapulmonary specimens. Indeed, extrapulmonary specimens represent a major diagnostic problem, mainly as they are often paucibacillary and at times contain inhibitors. With such samples, the resolved sensitivity of the BDProbeTec ET system was, as expected, lower (77.8%) than with pulmonary specimens (91.5%), but nevertheless higher than with microscopy (63.1%). Very recently, an investigation of extrapulmonary specimens using the BDProbeTec ET system has been reported (9) from which a sensitivity lower than ours emerges (60.7%). The data concerning other automated amplification systems does not substantially differ from ours (3, 5-7, 11, 12, 14-18, 22, 25, 26).

In our survey, the overall number of cases where the IAC revealed failure to amplify was below 1%, a value that redefines the putative role of inhibitors, probably overestimated due to suboptimal sensitivities of various techniques for MTBC amplification. Failure to amplify occurred at identical frequencies in respiratory and nonrespiratory samples.

Of the false-positive results, the ones confirmed by repetition of the test appear to be accounted for by contamination of the sample before analysis. Unconfirmed false-positive results, on the other hand, appear most likely to have been contaminated, probably by aerosolization, during the amplification procedure. The possibility of carryover between consecutive microwells was excluded due to the lack of positive samples in the wells surrounding each well with a false-positive result. Two false-positive results were obtained from samples which grew M. chelonae and M. xenopi. These species grew from a number of other samples in this study, and none of these samples gave positive results with the BDProbeTec ET system; therefore, there is no evidence suggesting that the system possesses cross-reactivity. False-negative results may be attributable to sampling errors or to insufficient sensitivity of the system.

The BDProbeTec ET system is licensed only for use with respiratory samples; however, the results reported in this study, based on a large number of extrapulmonary specimens, seem encouraging. The lower sensitivity, which is expected for extrapulmonary samples, is probably affected by the poor bacterial load these samples usually harbor, as suggested by the fact that the percentages of smear-positive results among samples yielding positive cultures were 70.6 for specimens originating from the lung and 48.8 for samples from extrapulmonary sources.

In conclusion, the sensitivity of the BDProbeTec ET system is good for smear-positive samples. Among smear-negative samples, however, the number of false-negative results is still elevated, thus confirming the universally accepted role of amplification techniques as an adjunct to culture and not as a replacement for it. The sensitivity and specificity values of the BDProbeTec ET system fall within the most satisfactory area of the range considered at present acceptable for commercially available MTBC amplification systems. The performance, the presence of an internal control, the minimization of contamination due to the use of sealed microwells, and the possibility of reagent storage at room temperature make the BDProbeTec ET system a reliable and convenient tool for the rapid diagnosis of tuberculosis.