Microscopic Observation Drug Susceptibility Assay, a Rapid, Reliable Diagnostic Test for Multidrug-Resistant Tuberculosis Suitable for Use in Resource-Poor Settings

Abstract

There is an urgent need for new tools to improve our ability to diagnose tuberculosis (TB) and multidrug-resistant TB (MDR-TB) in resource-poor settings. In a retrospective analysis undertaken in a region with a high incidence of TB, we evaluated the performance of the microscopic observation drug susceptibility assay (MODS), a novel assay developed in Perú which uses an inverted light microscope and culture in Middlebrook 7H9 broth to detect mycobacterial growth. MODS detected 94.0% of 1,908 positive sputum cultures, whereas Löwenstein-Jensen (LJ) culture detected only 86.9% (P < 0.001). The median time to culture positivity was 8 days (compared to 16 days for the same 208 samples by LJ culture; P < 0.001, Wilcoxon signed rank test). The results obtained by direct susceptibility testing using MODS demonstrated excellent concordance for isoniazid and rifampin and the detection of multidrug resistance with those obtained by indirect colorimetric methods: the microplate Alamar Blue assay (MABA) and the tetrazolium microplate assay (TEMA) (agreement, 95, 98, and 94%; kappa values, 0.8, 0.7, and 0.7, respectively). The concordance of the susceptibility testing results for ethambutol and streptomycin was poor. MODS is a novel assay which can detect the organisms responsible for TB and MDR-TB directly from sputum inexpensively, rapidly, and effectively. A comprehensive prospective evaluation of MODS is under way in Perú, and independent validation in nonresearch laboratories should be undertaken at the earliest opportunity.

Every single day at least 6,000 people die of tuberculosis (TB), a curable respiratory disease. The diagnosis of TB by sputum smear microscopy is an integral feature of the World Health Organization DOTS (direct observation of treatment-short-course chemotherapy) strategy for global TB control (25). Low cost, simplicity, and inherent detection of the most infectious cases are the three principal advantages of microscopy for acid-fast bacilli. However, the sensitivity of microscopy for the detection of all cases is low, even when the optimum sensitivity of microscopy is achieved (approximately half of all culture-positive cases are smear negative), and the performance of microscopy is highly variable. Furthermore, the contribution of transmission of infection by smear-negative culture-positive patients (which, by definition, pass undetected when the sole mode of diagnosis is sputum smear) is not inconsiderable (2), and the potential impact of the detection and treatment of these patients is significant (19). Moreover, in this era of emerging drug resistance (9), the lack of information on drug susceptibility threatens the continuing role of the sputum smear as the sole tool for the diagnosis of the majority of cases of TB worldwide. The development of new, low-cost diagnostic tools offers the possibility of future TB control on the basis of culture-based diagnosis and more widespread, targeted susceptibility testing.

The simple microscopic observation drug susceptibility assay (MODS) (5), developed in our laboratory in Lima, Perú, uses two well-known properties of Mycobacterium tuberculosis: (i) the rate of growth in liquid medium is considerably quicker than that on solid medium, and (ii) the morphology of M. tuberculosis in liquid culture is characteristic and recognizable, consisting of tangles or cords of organisms. By use of an inverted light microscope to examine 24-well plates inoculated with Middlebrook 7H9 broth and decontaminated sputum, mycobacterial growth can thus be detected long before it would be visible to the naked eye. Incorporation of anti-TB drugs at the outset enables direct susceptibility testing with clinical specimens. The ability to detect mycobacterial growth by identification of the color changes accompanying the reduction of nitrate to nitrite has been widely used previously for secondary drug susceptibility testing (1, 20, 21) and is the basis for the microplate Alamar Blue assay (MABA) and the tetrazolium microplate assay (TEMA) (4, 6, 11, 12, 26). MICs are determined by using serial dilutions of anti-TB drugs in 96-well microtiter plates containing the indicator, 7H9GC broth, and M. tuberculosis isolates. An operational evaluation of these tools, which have been in use in our laboratory for more than 5 years, is under way. Here we report on an analysis of their performances in epidemiological studies conducted over the last 4 years.

The analysis had three distinct objectives: (i) comparison of the sensitivity of detection of mycobacterial growth by MODS with that by conventional culture on Löwenstein-Jensen (LJ) medium; (ii) comparison of the time to culture positivity of sputum specimens culture positive by both MODS and LJ culture; and (iii) evaluation of the concordance of the susceptibilities obtained by direct susceptibility testing of clinical samples by MODS and (for the same isolates of M. tuberculosis) by indirect susceptibility testing by MABA or TEMA for rifampin, isoniazid, ethambutol, and streptomycin.

MATERIALS AND METHODS

Sputum decontamination.

All sputum specimens were digested and decontaminated of other bacteria by the standard N-acetyl-L-cysteine-NaOH-sodium citrate method (16).

MODS.

MODS was performed as described previously (5, 23). Wells containing decontaminated sample, Middlebrook 7H9 broth, oleic acid-albumin-dextrose-catalase, and polymyxin B-nalidixic acid-trimethoprim-azlocillin, with or without drug, were examined daily (weekdays only) from days 5 to 15, on alternate days from days 16 to 25, and twice weekly from days 26 to 40 under an inverted light microscope at ×40 magnification. The organism in a sample was considered susceptible if growth was observed in the drug-free control wells but not in wells containing antimicrobial agents. Drug-sensitive positive control strains were included daily.

MABA and TEMA.

MABA and TEMA were performed as described previously (4, 12). The MICs of each antibiotic (defined as the lowest drug concentration which prevents a color change) were determined for each isolate.

LJ culture.

Two hundred microliters of each decontaminated sample was used to inoculate one LJ slant (Difco), which was incubated at 37°C and examined twice weekly from weeks 2 to 8.

Data sources.

The data for this analysis were drawn from two previous epidemiological studies conducted in Lima: (i) a study of the impact of human immunodeficiency virus infection and multidrug-resistant (MDR) TB (MDR-TB) upon infectivity and mortality in two hospital-based populations (inpatients and outpatients at the Hospital Maria Auxilliadora and the Hospital Nacional Dos de Mayo) (V. Kawai et al., 50th Ann. Meet. Am. Soc. Trop. Med. Hyg., abstr. 44, 2001) and (ii) a study of the prevalence and predictors of active TB among subjects with respiratory symptoms presenting to the Hospital Nacional Dos de Mayo. Samples in these studies were cultured by MODS and on LJ slants, and susceptibility testing was performed by MODS and either MABA or TEMA.

Inclusion and exclusion criteria.

The final data sets for analysis for the three objectives were nonidentical, as the inclusion criteria differed. For analysis of the time to culture positivity and the sensitivity of detection, all specimens collected in the course of each study with a recorded date of sputum collection and a recorded date of positive culture by MODS and/or in LJ medium were included, regardless of the availability of susceptibility testing results. For each drug in the susceptibility concordance analysis, all samples with a result by MODS and a corresponding result by MABA and TEMA were included, regardless of whether the susceptibility data were complete for all four drugs and regardless of the availability of time-to-positivity data.

Analysis.

The denominator for the sensitivity of detection by each method (MODS and LJ culture) was culture positivity by either method. The comparison of the time to detection of culture positivity was performed by the Wilcoxon signed rank test. In order to adjust for the potential bias of different frequencies of reading of the microtiter plates and LJ slopes, the data were reanalyzed after subtraction of 3.5 days (the maximum possible discrepancy) from each time to culture positivity by LJ culture. For the susceptibility testing concordance analysis, two-by-two tables of the paired data were constructed; and by using MABA or TEMA as the “gold standard,” the performance of MODS with respect to its sensitivity, specificity, and positive and negative predictive values was determined. The concordance of the susceptibility testing results was analyzed by McNemar's χ² paired test for discordant pairs, determination of the difference in the proportions of isolates identified as resistant, and determination of the actual proportion of agreement and also agreement beyond chance by calculation of a kappa statistic (3). Each drug and the entity of MDR (resistance to isoniazid and rifampin) were analyzed separately. Data were grouped and analyzed separately for samples obtained pretreatment (defined as prior to treatment or within the first week of treatment) and while on treatment (defined as having received at least 8 days of treatment for TB). Patients could contribute one sample to each group; thus, each analysis is for one sample per patient, although some patients contributed samples to both groups. Data were collected in a Microsoft Excel 2002 database and analyzed by using Stata software (version 7.0; Stata Corporation, College Station, Tex.).

RESULTS

The attrition of sample numbers from the overall data set of 5,771 samples and the reasons for exclusion are illustrated in the flowcharts in Fig. 1a and b.

FIG. 1.

Flowcharts of attrition of specimens for detection of sensitivity of detection and analysis of time to culture positivity (a) and concordance of results of susceptibility testing stratified by treatment (b).

Sensitivity of detection.

There was a significant difference between the proportions of all samples (n = 4,213) culture positive for M. tuberculosis by MODS and LJ culture: 0.43 (95% confidence interval [CI], 0.41 to 0.44) versus 0.39 (95% CI, 0.38 to 0.41), respectively (z = 3.12; P < 0.01). Of the total of 1,908 culture-positive specimens (599 [31%] of which were smear negative), 1,799 were culture positive by MODS (of which 592 were smear negative) and 1,659 were positive on LJ slopes (of which 424 were smear negative), representing overall sensitivities of detection of 94 and 87%, respectively (95% CIs, 93 to 95% versus 85 to 88%, respectively; z = 7.8; P < 0.001).

Time to culture positivity.

Of the 1,908 specimens positive by LJ culture or MODS, paired data were available for 272 specimens, which were evaluable in the analysis of the time to culture positivity; 208 were classified as pretreatment specimens and 64 were classified as on-treatment specimens. Samples were excluded owing to a lack of growth by at least one of the methods or a lack of data for the time to culture positivity or because they represented multiple samples from the same patient. The cumulative frequency chart (Fig. 2a) illustrates the distribution of the times to culture positivity for pretreatment and on-treatment samples by MODS (means, 8.2 days [95% CIs, 7.9 to 8.4 days] and 10.0 days [95% CIs, 9.0 to 11.0 days], respectively) and LJ culture (means, 17.3 days [95% CIs, 16.5 to 18.1 days] and 23.6 days [95% CIs, 20.8 to 26.5 days], respectively. The time to culture positivity was significantly shorter by MODS for both pretreatment samples (medians, 8 versus 16 days, respectively; z = 12.5; P < 0.001 [Wilcoxon signed rank test]) and on-treatment samples (medians, 9 versus 21 days, respectively; z = 6.8; P < 0.001). More than 98% of MODS cultures of samples from patients who had not yet received TB treatment were positive by day 12, whereas only 18% of LJ cultures were positive by day 12. This analysis could theoretically be biased by the practice of viewing LJ cultures only twice weekly, whereas MODS cultures were viewed on 5 days (weekdays) of 7 days (which was the highest viewing frequency, which occurred in the first 15 days); thus, the data were reanalyzed after 3.5 days (the maximum possible discrepancy for each sample) was subtracted from each of the LJ culture results. Not surprisingly, given the large difference described above, this did not alter the findings in any meaningful way (P < 0.001 [Wilcoxon signed rank test]). Smear status significantly altered the time to culture positivity by both MODS (median times for smear-positive and smear-negative specimens, 7 and 10 days, respectively) and LJ culture (median times for smear-positive and smear-negative specimens, 16 and 28 days, respectively). The difference in the time to a positive culture result between smear-negative and smear-positive samples was more marked for LJ cultures than for MODS cultures (Fig. 2b); the effect of smear status on the time to a positive culture was the same for pre- and on-treatment samples in this analysis (data not shown).

FIG. 2.

Time to culture positivity by MODS and LJ culture by effect of TB treatment (a) and effect of smear status (b).

Susceptibility testing results.

Two hundred seventy-six specimens from 276 patients were potentially evaluable in the susceptibility comparison analysis. Samples were analyzed separately according to whether they were obtained prior to (n = 207) or during (n = 69) TB treatment. The performance (sensitivity, specificity, and positive and negative predictive values) of MODS compared to the susceptibility testing results obtained by MABA or TEMA is illustrated in Table 1. Only five of the samples in this part of the analysis were smear negative; thus, the data are presented for smear-negative and smear-positive samples combined. Stratification of pre- and on-treatment groups by quantitative smear status (+, ++, and +++) resulted in small subgroups but, nevertheless, revealed no effect on any of the analyses of the concordance of the susceptibility testing results (data not shown).

TABLE 1.

Criteria for resistance and performance characteristics of MODS for direct detection of drug resistance compared to those of indirect susceptibility testing by MABA-TEMA (gold standard)

Drug	MIC (μg/ml)		Sensitivity (%)		Specificity (%)		Positive predictive value (%)		Negative predictive value (%)
	MABA- TEMA	MODSa	Pretreatment (n = 207)	On treatment (n = 69)	Pretreatment (n = 207)	On treatment (n = 69)	Pretreatment (n = 207)	On treatment (n = 69)	Pretreatment (n = 207)	On treatment (n = 69)
Isoniazid	≥0.5	0.4	81.1	90.3	96.9	91.2	85.7	90.3	95.7	91.2
Rifampin	≥2.0	1.0	100	92.9	98.3	89.7	57.1	86.7	100	94.6
MDRb			100	90.9	98.2	93.3	57.1	87.0	100	95.5
Ethambutol	≥4	2.5	42.3	64.7	91.9	91.1	45.8	73.3	90.7	87.2
Streptomycin	≥2	6.0	44.4	50.0	98.3	98.0	57.1	87.5	97.2	87.3

^aGrowth in the presence of the indicated concentration was designated resistant.

^bMDR was defined by convention as combined isoniazid and rifampin resistance.

Isoniazid.

Overall, the organisms in 19% of pretreatment samples and 48% of on-treatment samples were designated isoniazid resistant by MABA and TEMA. The proportions of agreement between MODS and MABA-TEMA for the detection of isoniazid resistance for pre- and on-treatment samples were 95 and 91%, respectively (kappa values = 0.80 and 0.82, respectively), with no statistically significant difference (discordance) between the two methods (McNemar's test pretreatment value = 0.33 [P = 0.6]; McNemar’s test posttreatment value = 0.00 [P = 1.0]). The MABA-TEMA MICs for the eight isolates with discordant results designated resistant by MODS were 0.125 μg/ml (n = 7) and 0.25 μg/ml (n = 1); all isolates grew in MODS cultures at concentrations of 0.1 and 0.4 μg/ml. For the 10 isolates with discordant results designated susceptible by MODS and resistant by MABA-TEMA, the corresponding MICs by MABA-TEMA were only 0.5 μg/ml (n = 1), 2.0 μg/ml (n = 6), and 4.0 μg/ml (n = 3); none of the isolates classified as susceptible by MODS were highly resistant by MABA-TEMA.

Rifampin.

Overall, the organisms in 2% of pretreatment samples and 42% of on-treatment samples were designated rifampin resistant by MABA and TEMA. The proportions of agreement between MODS and MABA-TEMA for the detection of rifampin resistance for pre- and on-treatment samples were 98 and 91%, respectively (kappa values = 0.72 and 0.82, respectively), with no statistically significant difference (discordance) between the two methods (McNemar's test pretreatment value = 3.00 [P = 0.08]; McNemar’s test posttreatment value = 0.67 [P = 0.41]). The MABA and TEMA MICs obtained for the seven isolates designated resistant by MODS but susceptible by MABA-TEMA were 0.063 μg/ml (n = 3), 0.125 μg/ml (n = 2), 0.25 μg/ml (n = 1), and 0.5 μg/ml (n = 1). The MICs obtained by MABA-TEMA were >16 μg/ml for the two isolates susceptible by MODS and resistant by MABA-TEMA. In contrast to the results for isoniazid, these are large discrepancies, but importantly, they occurred very rarely, in only 0.7% (2 of 276) of specimens.

MDR.

Overall, the organisms in 2.3% of pretreatment samples and 33% of on-treatment samples were designated MDR by MABA-TEMA. The proportion of agreement between MODS and MABA-TEMA for the detection of MDR was 94% for both pre- and on-treatment samples (kappa values = 0.72 and 0.83, respectively), with no statistically significant difference (discordance) between the two methods for pretreatment and on-treatment samples (McNemar’s test pretreatment value = 3.00 [P = 0.08]; McNemar’s test posttreatment value = 0.66 [P = 0.20]). The overall negative and positive predictive values for this population were 97.7 and 81.8%, respectively.

The discordant results encountered in this analysis reflect the discrepancies described above for isoniazid and rifampin alone. Of the six isolates with discordant results by MODS (MDR) and MABA-TEMA (non-MDR), three isolates were resistant to isoniazid alone and one isolate was resistant to rifampin alone by MABA-TEMA; only two isolates were fully susceptible to rifampin and isoniazid by MABA-TEMA. Of the two isolates with discordant results by MODS (non-MDR) and MABA-TEMA (MDR), both were identified as susceptible to isoniazid and rifampin by MODS. In contrast, the results of MABA-TEMA were unequivocal, with rifampin MICs ≥16 μg/ml for both isolates and isoniazid MICs of 2 and 4 μg/ml for the two isolates, respectively.

The analyses for ethambutol and streptomycin were hampered by the lack of clear cutoff points for MABA-TEMA (as described previously [12]); thus, multiple analyses were performed for the various possible combinations of cutoff points. The concordance between the two methods was poor for both ethambutol and streptomycin susceptibility testing, but the most favorable (still poor) results are reported here.

Ethambutol.

Overall, the organisms in 14% of pretreatment samples and 27% of on-treatment samples were designated ethambutol resistant by MABA-TEMA. The concordance between MODS and MABA-TEMA was poor for ethambutol, regardless of the cutoff points used. The best results were obtained when resistance by MABA-TEMA was defined as an MIC ≥4 μg/ml and resistance by MODS was defined as growth in the presence of 2.5 μg/ml. The proportions of agreement between MODS and MABA-TEMA for the detection of ethambutol resistance for pre- and on-treatment samples were 85 and 83%, respectively (kappa values = 0.35 and 0.58, respectively), with no statistically significant difference (discordance) between the two methods (McNemar’s test pretreatment value = 0.14 [P = 0.71]; McNemar’s test posttreatment value = 0.40 [P = 0.53]).

Streptomycin.

The concordance between MODS and MABA-TEMA for streptomycin was also poor. The best results were obtained when resistance by MABA-TEMA was defined as an MIC ≥2 μg/ml and resistance by MODS was defined as growth in the presence of 6 μg/ml. Overall, the organisms in 5% of pretreatment samples and 22% of on-treatment samples were designated streptomycin resistant by MABA-TEMA. The proportions of agreement between MODS and MABA-TEMA for the detection of streptomycin resistance for pre- and on-treatment samples were 96 and 87%, respectively (kappa values = 0.48 and 0.57, respectively), with statistically significant differences (discordance) between the two methods for on-treatment samples (McNemar’s test pretreatment value = 0.50 [P = 0.48]; McNemar’s test posttreatment value = 4.50 [P = 0.03]).

Effect of treatment on concordance of susceptibility testing results.

For isoniazid, rifampin, and streptomycin, discordant results were observed more commonly for on-treatment samples (8.7, 8.6, and 11.6%, respectively) than pretreatment samples (5.8, 1.4, and 3.9%, respectively). Treatment had no effect on the high proportion of discordance for ethambutol susceptibility testing (14.5 and 13.5%, respectively, for pretreatment samples).

DISCUSSION

The two key findings of this analysis are the superior speed and sensitivity of MODS over those of LJ culture and the high concordance of the susceptibility results for MDR M. tuberculosis obtained by MODS directly from clinical samples with those of MABA-TEMA performed on the same clinical isolates.

Our results for direct susceptibility testing from sputum concur with those of the important comparison reported by Park et al. (23), in which there was 100% agreement between MODS and the reference gold standard proportion method for indirect testing of the susceptibilities of M. tuberculosis strains to isoniazid and rifampin.

All the susceptibility data generated by MODS were not perfect. Indeed, the results for ethambutol and streptomycin were disappointing (pyrazinamide is not discussed here, as testing is problematic for the same pH-related reasons encountered with other methodologies, but modifications to accommodate this are under investigation). Given that the reproducibility of the results of ethambutol susceptibility testing, even when gold standard methods are used by the most experienced hands, has always been problematic (17, 18), the poor correlation for ethambutol in this analysis is not entirely unexpected (23).

However, the excellent performance of Alamar Blue, particularly for isoniazid and rifampin susceptibility testing, has consistently been demonstrated by several groups, including the detection of a concordance rate of 94% with the BACTEC method by some investigators (6, 12, 26), validating the use of colorimetric methods as comparators. In our laboratory, the use of low-cost Alamar Blue ($7.54 for four-drug susceptibility testing per strain) has been superseded by the use of the less expensive tetrazolium bromide ($5.04 for four-drug susceptibility testing per strain by TEMA), as the results of the two methods demonstrate excellent concordance (kappa values = 1.0 for rifampin and isoniazid) (4).

In contrast to the data for ethambutol and streptomycin, the results for isoniazid alone, rifampin alone, or isoniazid and rifampin combined (the MDR test) show excellent concordance. This is indeed fortuitous, as in a programmatic setting, the impact of information about ethambutol, streptomycin, or even isolated isoniazid resistance is likely to be small. The design of drug regimens is such that the capacity to achieve a cure in the presence of resistance to one drug is built in. In reality, the only susceptibility data that are likely to be usefully translated into clinical action (in terms of a change in the drug regimen) in a programmatic setting would be identification of MDR-TB. The overall negative predictive value of 97.7% and the overall positive predictive value of 81.8% for this population indicate that MODS tends to overdiagnose disease caused by MDR organisms. However, four of the six isolates with discordant results by MODS (MDR) and MABA-TEMA (non-MDR) were monoresistant to either isoniazid or rifampin. Even though MODS may marginally overdiagnose MDR, it rarely does so when an isolate is fully susceptible.

The treatment failure rates among subjects whose initial isolates are MDR are high (8, 13), and detection of the likelihood of treatment failure at the outset of therapy would be worthwhile, as most TB physicians would probably advise a regimen switch (10, 14, 22). Failure to identify these cases leads to increased rates of morbidity, mortality, and, most importantly, continuing ongoing transmission during inappropriate therapy (15, 24). For patients failing to respond to TB therapy, the availability of a rapid, reliable diagnostic method for the detection of MDR is also critical. This analysis suggests that MODS offers excellent performance for the early, inexpensive detection of MDR-TB both in patients prior to therapy and in those already receiving treatment.

The vast majority of TB diagnosed in the world occurs in developing countries (7), where resources are scarce. Interruption of the transmission of TB through the early detection of cases depends as much on the availability of a sensitive and inexpensive diagnostic tool as on health-seeking behavior interventions. The advantages of culture-based diagnosis over sputum smear include an increased sensitivity of diagnosis and the availability of susceptibility data. Culture, however, can be useful in many settings only if it is relatively rapid to minimize loss to follow-up and to meaningfully inform clinical decision making. In this analysis, 99.5% of MODS cultures of pretreatment samples were found to be positive within 2 weeks, whereas only 34% of LJ cultures of the same samples were detectably positive within the same time period. Aside from the clinical advantage that such early information provides, there are two further important benefits: first, if a MODS culture is negative at 3 weeks, it can be regarded as truly negative (which has important clinical implications), and second, as a result of this, cultures need to be kept for only 3 weeks rather than 6 weeks, which results in a marked reduction in the amount of culture material (and, thus, reductions in the amount of space required and potential biological hazard) in the laboratory.

The data presented here extend our experience with MODS as a tool for the detection of TB and MDR-TB by direct susceptibility testing and is the largest evaluation to date of this exciting tool, which is very well suited to implementation in resource-poor settings where TB is endemic. A further prospective, operational evaluation of MODS is under way in Lima, Perú, in which the performance of MODS is being compared with those of semiautomated mycobacterial culture with susceptibility testing (MB BacT) and conventional LJ culture with susceptibility testing by the proportion method. This work is being conducted with samples from both community- and hospital-based populations and should provide critical information about the utility and performance of MODS for specific target groups and should define the optimum format for the methodology (e.g., elimination of the ethambutol and the streptomycin wells would reduce both the workload and the cost without a loss of clinically useful information). MODS appears to have the potential to become a tool which may improve our approach to TB diagnosis in resource-poor settings.