Diagnostic accuracy of the microscopic observation drug susceptibility assay: a pilot study from India

SUMMARY

SETTING

The microscopic observation drug susceptibility (MODS) assay is a rapid, sensitive, low-cost liquid culture technique.

OBJECTIVE

To establish the accuracy of MODS for the detection of active pulmonary tuberculosis (TB), and to document the costs and challenges of setting up this assay in a low-income setting.

DESIGN

Prospective blinded pilot study of 200 adult TB suspects at a tertiary referral hospital in India. Reference standard included culture (Löwenstein-Jensen and automated liquid culture) and clinical diagnosis.

RESULTS

Patients were mostly male (n = 122, 61.1%) and out-patients (n = 184, 92.0%), with a mean age of 40.4 years (standard deviation 16.2). Seventeen (8.5%) were human immunodeficiency virus infected and 47 (23.5%) were reference culture-positive. Compared to reference culture, MODS was 78.9% sensitive (95%CI 62.2–90.0) and 96.7% specific (95%CI 92.0–98.8). Clinical assessment suggested that MODS was false-negative in 3/8 reference culture-positive MODS-negatives and true-positive in 4/6 reference culture-negative MODS-positives. MODS was faster than solid (P < 0.001) and liquid culture (P = 0.088), and cheaper than both.

CONCLUSION

MODS may be a good alternative to automated liquid culture, but there were several challenges in setting up the assay. Prior training and validation, setup costs and inability to rule out cross-contamination need to be taken into account before the test can be established.

India has the greatest burden of tuberculosis (TB) disease in the world.¹ Although the World Health Organization (WHO) has recommended the scale-up of liquid culture capacity in TB-endemic countries,² commercial liquid culture techniques are expensive and require laboratories with advanced infrastructure and external quality assurance. India has few laboratories that meet these standards, and the Revised National Tuberculosis Control Programme (RNTCP) has committed to improving this.³ Several alternative diagnostic strategies have emerged to improve conventional TB culture.⁴

The microscopic observation drug susceptibility (MODS) assay has been described and well validated in Peru.^5–7 Mycobacteria grow faster in liquid culture, and members of the Mycobacterium tuberculosis complex produce characteristic, microscopically visible, tangled ‘cords’, allowing a rapid, specific low-cost detection system.⁸ MODS may therefore provide a more cost-effective alternative to commercial liquid culture systems.

Our objective was to establish the MODS technique in India and perform a pilot, prospective, blinded evaluation of MODS for TB detection only among consecutive pulmonary TB suspects, using conventional culture methods as a reference standard.

METHODS

Setting

The present study was conducted at a large tertiary referral centre in Vellore, India (annual TB case detection rate 144 per 100 000 population).⁹ It was approved by the institutional review boards at the Christian Medical College (CMC), Vellore and McGill University, Canada. Informed written consent was obtained from all subjects using the local language. The mycobacteriology laboratory is externally accredited by the RNTCP, Ministry of Health, Government of India.

Consecutive patients with suspected TB were recruited from a central DOTS clinic and medical wards. Patients were re-assessed 8 weeks later to determine if they had received empirical treatment, and if so, if they had responded to treatment, according to predetermined definitions.

Recruitment

Patients were recruited between October 2007 and July 2008, independently of any results of previous laboratory testing for TB. Inclusion criteria were as follows: age >18 years; assessed by a physician who thought a diagnosis of TB was possible; consent to study protocol, including human immunodeficiency virus (HIV) testing, specimen submission and follow-up visit; and at least two of the following: cough for ≥2 weeks, weight loss of at least 10% of healthy body weight, symptoms of fever for ≥2 weeks or one temperature measured at >38.5°C.

Patients who had received any anti-tuberculosis treatment within the last 6 months were excluded from the study.

Specimen collection

Patients submitted one good quality spot sputum specimen of at least 4 ml. The three sputum smears used for routine testing for TB were also collected.

Decontamination

An equal volume of fresh 4% sodium hydroxide-N-acetyl-L-cysteine (NaOH-NALC) solution was added to each specimen in a sterile 50 ml conical screw-top centrifuge tube.^10,11 The tube was tightly closed, inverted and agitated on the vortex, then incubated at room temperature for exactly 15 min, followed by dilution up to 45 ml with phosphate buffer saline (PBS). After centrifugation at 3000 g for 15 min, the supernatant was decanted.

Blinding

The pellet was resuspended in 3 ml of PBS and divided into four Eppendorf tubes labelled with preassigned random four-digit numbers. Each tube containing 750 μl of decontaminant was used for one MODS or reference culture immediately after decontamination. A table connecting the random numbers with patient identification was kept in a locked file by the laboratory investigator.

Solid reference culture

Five drops of decontaminant were inoculated onto Löwenstein-Jensen (LJ) slants (in-house), and incubated on a slant in the dark at 37°C for 7 weeks. Cultures were examined weekly, and contaminated slants were re-inoculated with stored decontaminant. Positives were confirmed with Ziehl-Neelsen (ZN) smear. Every new batch of LJ media was controlled for sterility and for growth using M. tuberculosis American Type Culture Collection (ATCC) H37Ra 21577.

Liquid reference culture

Automated liquid culture was performed using MB BacT (bioMerieux India, New Delhi, India) for the first 68 specimens, then BACTEC 460 MB (BD India, Haryana, India) thereafter, as BACTEC reagent supplies were not available at the time the study started. BacT/ALERT^® mycobacteria process bottles contained 10 ml Middlebrook 7H9 broth with 0.5 ml antibiotic supplement. BACTEC 12B bottles contained 4 ml Middlebrook 7H12 broth with 0.1 ml antibiotic supplement. The decontaminant (0.5 ml) was added to each bottle and incubated at 37°C for 6 weeks. Non-inoculated bottles served as negative controls. Positive bottles were confirmed by acid-fast bacilli (AFB) smear. If no AFB were seen, a Gram stain was performed and blood agar inoculated to detect bacterial contamination. In case of contamination, the decontaminant was reinoculated. The p-nitro-α-acetylamino-β-hydroxy propiophenone (NAP) test was used for identification,¹² and was controlled using ATCC H37Ra (25177). BACTEC performance test standards (BD) were used twice monthly and needles were autoclaved weekly.

MODS technique

All assays were performed by the same technician, who had 6 years of experience in mycobacteriology but had not received any special training in the MODS assay. The investigators developed the MODS technique based on previous publications,^5–8 the Peru MODS website^* and input from D Moore (Imperial College, London).

The MODS technique was performed using 24-well tissue culture plates.^13* Middlebrook 7H9 broth containing 5.9 g of 7H9 broth base (BD), 0.31% glycerol (Sigma, St Louis, MO, USA) and 1.25 g of Bacto Casitone per litre, with growth supplement OADC (oleic acid, albumin, dextrose and catalase) and PANTA antibiotics (polymyxin B, amphotericin B, nalidixic acid, trimethoprim, azlocillin) (BD) was mixed and 900 μl was added to each of the four test wells. Initially, 100 μl of the sample pellet was inoculated into each well. Following the availability of the online MODS SOP, the procedure was changed slightly: for the last 40 specimens in the study, 1000 μl of sample pellet was added to 5.1 ml of 7H9 broth, which was distributed into four wells. In this way, the ratio of sample to broth was doubled from 1:10 to 1:5. Isoniazid (INH) and rifampicin (RMP) (100 μl) were added to two of the four wells (final concentration INH 0.4 μg/ml, RMP 1.0 μg/ml). The procedure was performed once daily. The broth media were stored at 4°C in sterile tubes and supplements were added to the media daily.

Every plate contained four wells of negative control (no specimen) and four wells of positive control containing M. tuberculosis H37Ra (ATCC 25177). Every fifth specimen was duplicated to assess reproducibility. Plates were permanently sealed using cellophane tape and incubated at 37°C for 35 days. Each well was observed daily between days 4 and 14, then weekly until day 35 for cording, using an inverted microscope. The presence of cording was the only identification test used. If both test wells contained more than two colony-forming units, the test was reported as positive, and wells containing no cording at 35 days were reported as negative. Contaminated specimens were re-inoculated using stored decontaminant. Observation was performed at days 1, 2 and 3 for cloudiness indicating bacterial contamination.

Analysis

Two hundred TB suspects were recruited as part of a larger pilot diagnostics evaluation project. For this evaluation, the small number of extra-pulmonary cases was excluded. Data were entered by two operators and manually rechecked after entry. Detection performance was reported as sensitivity and specificity of MODS as compared to a combination reference standard of positivity in either solid or liquid reference culture. Detailed clinical information was collected for all patients, and a final clinical diagnosis was defined based on reference culture, smear, histology and response to empirical TB treatment at 8 weeks (presence of two or more predefined clinical and laboratory response criteria, modified from Wilson et al.¹⁴).

Patients with any one positive reference culture were considered ‘definite TB’, smear- or histology-positives with response to treatment were considered ‘probable TB’, patients with negative investigations who responded to treatment were considered ‘possible TB’, those who did not receive treatment or who had no alternative diagnosis were considered ‘undetermined’ and those with an alternative diagnosis with microbiological evidence were considered ‘TB ruled out’. None of the reference standards contained MODS results.

Analyses were performed using SPSS 15.0 (LEAD Technologies Inc, Charlotte, NC, USA) and the Clinical Calculator package (VassarStats, Vassar College, Poughkeepsie, NY, USA, from http://faculty.vassar.edu).

Reagent costs for detection by the different culture techniques were compared. These costs did not include setup, labour or overhead costs.

RESULTS

Figure 1 shows the recruitment process and Table 1 provides a description of the study participants. No study-related adverse events were observed.

Figure 1.

Recruitment of TB suspects. TB = tuberculosis.

Table 1.

Demographics of study participants

	n	%
Total TB suspects included	189	100
Male	116	61.4
Mean age, years (SD)	40.4 (16.0)
Mean BMI (SD) (n = 179)	19.6 (4.0)
Out-patients	178	94.2
Living in Tamil Nadu State	175	92.6
Monthly family income <US$115	173	91.5
Experiencing cough	188	99.5
Experiencing chest pain	102	54.5
Experiencing fever	162	85.7
Mean number of baseline symptoms (SD)	3.9 (1.1)
HIV-infected	15	7.9
Given treatment for TB after recruitment (n = 132)	34	25.8
Reference standard definition
Definite TB (any L J or BACTEC culture positive)	44	23.3
Probable TB	2	1.1
Possible TB	5	2.6
TB ruled out	27	14.3
Cannot be determined	111	58.7

TB = tuberculosis; SD = standard deviation; BMI = body mass index; HIV = human immunodeficiency virus; LJ = Löwenstein-Jensen.

Among 200 patients recruited, 189 were pulmonary and 11 were extra-pulmonary TB suspects. The 189 pulmonary suspects provided 189 sputum samples. All specimens provided evaluable MODS results (cording seen or not seen), with no indeterminate or invalid MODS results. All 40 duplicate MODS demonstrated expected results (100% reproducibility). All positive control wells showed cording, and no negative control wells did so. Figure 2 describes overlapping results of liquid and solid reference culture and MODS.

Figure 2.

Venn diagram of overlapping positive results. MODS = microscopic observation drug susceptibility.

During a period of 4 days, between 15 and 19 October 2007, mycobacterial cross-contamination was suspected because 14 of 16 specimens inoculated into MODS between these dates were MODS-positive, an unexpectedly high positive rate. None of the negative control wells for these specimens were positive. All specimens during this period were re-inoculated, and eight of the positive specimens became negative, indirectly suggesting mycobacterial contamination. The repeat results were included in the final analysis. These specimens were not included in the reproducibility analysis.

Nine MODS cultures were contaminated with bacteria (4.8%); each was re-inoculated, and all nine provided evaluable results and were included in the final analysis. Three solid reference cultures (1.6%) and 12 liquid reference cultures (6.3%) were contaminated with bacteria; all were redecontaminated, and repeat results were included in the final analysis. Nine BACTEC bottles (4.8%) could not be read during a period of BACTEC machine failure, so they were re-inoculated from frozen decontaminant and repeat results were included in the final analysis.

Among the 189 pulmonary TB suspects, 38 were reference culture-positive on either solid or liquid culture, and MODS detected 30 of these cases (sensitivity 78.9%, 95% confidence interval [95%CI] 62.2–90.0; see Table 2); 151 specimens were reference culture-negative, and MODS was negative in 146 (specificity 96.7%, 95%CI 92.0–98.8).

Table 2.

Diagnostic accuracy of MODS compared to combined culture^*

	Positivity on solid or liquid culture		Total not TB or not treated
	Positive	Negative
MODS
Positive	30	5	35
Negative	8	146	154
Total	38	151	189

^*Sensitivity: 78.9% (95%CI 62.2–89.9). Specificity: 96.7% (95%CI 92.0–98.8). Positive predictive value: 85.7% (95%CI 70.0–94.6). Negative predictive value: 94.8% (95%CI 89.7–97.6).

MODS = microscopic observation drug susceptibility; CI = confidence interval.

Discrepant results were analysed using clinical information (see Table 3). Among eight reference culture-positive MODS negatives (‘Definite TB’), one patient was smear-positive with ill-defined opacities and responded to TB treatment, confirming false-negative MODS. The other seven were direct ZN and fluorescent sputum smear-negative. Three of these had data available on response to treatment, and all three responded well to empirical treatment. One had HIV, with a CD4 count of 210 cells/μl. These three cases would also suggest false-negative MODS. If the remaining four were reclassified as true-negatives, MODS sensitivity would increase to 89.5%.

Table 3.

Discrepant analysis

	Specimen	ZN smear	Fluorescent smear	Solid culture	Liquid culture	Response to treatment	HIV	CXR	Tuberculin skin test
Reference culture-positive, MODS-negative
3	Sputum	Negative	Negative	Negative	Positive	Missing	Positive	Right middle lobe consolidation	Negative
28	Sputum	Negative	Negative	Negative	Positive	Missing	Negative	Signs of consolidation	Positive
30	Sputum	Negative	Negative	Positive	Negative	Missing	Negative	Bilateral reticulonodular	Missing
69	Sputum	Negative	Negative	Negative	Positive	Yes	Negative	Small pleural effusion	Negative
79	Sputum	Negative	Sample not received	Negative	Positive	Missing	Negative	Apical pleural thickening	Positive
87	Sputum	Negative	Negative	Positive	Positive	Yes	Negative	Normal study	Negative
88	Sputum	Positive	Positive	Positive	Positive	Yes	Negative	Ill-defined opacities	Positive
104	Sputum	Negative	Negative	Negative	Positive	Yes	Negative	Normal study	Positive
Reference culture-negative, MODS-positive
6	Sputum	Negative	Negative	Negative	Negative	No		Normal study	Positive
21	Sputum	Negative	Negative	Negative	Negative	Missing	Negative	Normal study	Negative
41	Sputum	Negative	Negative	Negative	Negative	Yes	Negative	Normal study	Negative
45	Sputum	Negative	Negative	Negative	Negative	Yes	Negative	Normal study	Positive
67	Sputum	Positive	Positive	Negative	Negative	Missing	Negative	Consolidation with breakdown in right upper lobe	Negative

ZN = Ziehl-Neelsen; HIV = human immunodeficiency virus; CXR = chest radiograph; MODS = microscopic observation drug susceptibility.

Among five reference culture-negative MODS-positives, one was smear-positive with consolidation in the right upper lobe, and three others responded to empirical treatment (‘probable TB’), suggesting a false-negative reference culture in these four cases. If these four were reclassified as true-positives, MODS specificity would increase to 99.3%.

Time to detection (from inoculation to reporting of a positive result) was compared between all methods (see Table 4). Mean time to detection among 35 positive MODS from sputum specimens was 10.2 days (standard deviation 3.7). This was significantly faster than solid reference culture (30.2 days, P < 0.001) and slightly faster than liquid reference culture (12.0 days, P = 0.088).

Table 4.

Time to positivity^*

	n	Positives n	Mean time to positive, days	SD days	Mean time to positive, including repeats due to contamination, days	P value vs. solid culture†	P value vs. liquid culture
Solid culture sputum	191	30	30.2	8.1	30.6
Liquid culture sputum	191	35	12.0	7.7	12.8	<0.001
MODS sputum	191	35	10.2	3.7	10.7	<0.001	0.088

^*Time between inoculation and reporting of positive result.

^†Matched pairs t-test.

SD = standard deviation; MODS = microscopic observation drug susceptibility.

Reagent-only costs for detection were compared. MODS cost Rs 100 (US$2.06) per specimen as compared to Rs 250 (US$5.14) for solid culture, Rs 500 (US$10.29) for BACTEC liquid culture, and Rs 1000 (US$20.58) for MB BacT liquid culture.

DISCUSSION

We report our pilot experience with the MODS technique at a tertiary centre in India. Although MODS has been introduced in the Indian literature,¹⁵ it has not been evaluated in India to date. We faced several challenges in setting up this assay, and our sensitivity was not as high as that reported by established MODS centres, suggesting that we could optimise our technique further. We hope our experience will assist other centres who are establishing MODS.

This pilot study showed us that more extensive training and standardisation should have been done before collecting our data. We worked with ATCC strains and a small number of clinical specimens prior to beginning the study, but did not complete a full validation. We found that time to detection decreased during the study as the technician became more experienced. Our lack of sensitivity may have been related to a smaller sample:broth ratio early in the study. We had no way to test reliably for mycobacterial cross-contamination, as molecular fingerprinting is not available in our laboratory. We suspected cross-contamination by closely observing trends in the data, but this method is subject to interpretation bias. Because we plated positive controls beside each specimen instead of in a separate plate as suggested, we increased our risk of cross-contamination, although negative controls were consistently negative.

We faced availability and reliability problems with the BACTEC 460 system. Although other centres have evaluated MODS without external training or standardisation,^16,17 we would recommend validation with at least 120 specimens, using a supervising laboratory.

In established MODS centres, there is no difference in mycobacterial cross-contamination rates between MODS and other culture techniques.^5,18 Bacterial contamination is controlled using chemical pretreatment of sputum. In our study, decontamination with 4% NaOH was appropriate as our bacterial contamination rate was 4.8%.

There is excellent evidence from TB-endemic countries that MODS is an accurate diagnostic tool.^{6–8,19–21} However, our experience suggests that not all countries may achieve high accuracy without extensive training and standardisation.

Liquid culture techniques cost up to US$70 for detection and four-drug first-line drug susceptibility testing (DST), while MODS offers detection and two DST results for US$2.00 in Latin America.⁶ In our study, reagent costs for MODS detection were lower than for solid or liquid culture. MODS labour costs may be higher, depending on frequency of examination, and set-up costs would include an inverted microscope. Further work is necessary to perform more extensive cost and cost-effectiveness analyses.

The benefits of MODS are the lack of any proprietary ingredient and its rapid growth and high screening efficacy. An experienced technician can search one well in less than 1 min.¹⁸ The technique does not require the manipulation of positive cultures for inoculum density adjustment, subculture or identification, and the MODS plate is permanently sealed at the time of inoculation, when organism density is low, and disposed of by autoclave after reading. In this way, MODS may be equally as biosafe as conventional culture techniques, but this requires further confirmation.¹⁸

As certain species of non-tuberculous mycobacteria may also demonstrate cording in liquid culture, this finding is not specific only to the M. tuberculosis complex.^22,23

MODS may provide a lower cost alternative to conventional liquid culture if adequate validation is performed before initiating the technique.