Sensitivity and Specificity of Fluorescence Microscopy for Diagnosing Pulmonary Tuberculosis in a High HIV Prevalence Setting

Abstract

Setting

Mulago Hospital, Kampala, Uganda

Objective

To evaluate the diagnostic performance of fluorescence microscopy (FM) for diagnosing pulmonary tuberculosis in a high HIV prevalence setting.

Design

Consecutive inpatients with cough > 2 weeks submitted two sputum specimens for smear microscopy. Smears were examined by conventional light microscopy (CM) and FM. Performance of the two methods was compared using mycobacterial culture as a reference standard.

Results

426 patients (82% HIV-infected) were evaluated. FM identified 11% more smear positive patients than CM (49% vs. 38%, p<0.001). However, positive FM results were less likely than positive CM results to be confirmed by culture when smears were read as either “scanty” (54% vs. 90%, p<0.001) or 1+ (82% vs. 91%, p=0.02). Compared to CM, FM sensitivity was higher (72% vs. 64%, p=0.005) and specificity lower (81% vs. 96%, p<0.001). In receiver operating characteristic analysis, maximum area under the curve for FM was obtained at a threshold of > 4 AFB/100 fields (sensitivity 68%, specificity 90%).

Conclusion

Although FM increases the sensitivity of sputum smear microscopy, additional data on FM specificity and on the clinical consequences associated with false positive FM results are needed to guide implementation of this technology in high HIV prevalence settings.

BACKGROUND

Rapid identification and treatment of new cases is the keystone of tuberculosis control worldwide. Yet, case detection rates in many high burden countries are below the 70% target adopted by the World Health Organization (WHO).(1) In most of these countries, case detection relies primarily on identification of acid-fast bacilli (AFB) in unconcentrated sputum smears using a conventional light microscope. Conventional microscopy (CM) is inexpensive, rapid, and highly specific but has poor sensitivity, particularly in patients co-infected with HIV.(2, 3)

Fluorescence microscopy (FM) has been proposed as a technique to increase the sensitivity of smear examination. A recent meta-analysis concluded that FM was approximately 10% more sensitive than CM and had similar specificity.(4) However, only two of the included studies reported data from high HIV prevalence areas, and only one of these compared FM results to a reference standard.(5, 6) Moreover, many of the comparisons were not blinded and few studies collected data under field conditions. Additional data collected in reference and peripheral laboratories in high tuberculosis burden settings, especially those where HIV infection is also prevalent, are needed to confirm the performance characteristics of FM and to define optimal thresholds for a positive FM smear examination.(7–10)

To address these issues, we conducted a prospective, blinded comparison of CM and FM for the evaluation of patients suspected of having pulmonary tuberculosis admitted to Mulago Hospital in Kampala, Uganda. We report the diagnostic performance of each test compared to the reference standard of mycobacterial culture and determine the impact of different thresholds for defining a positive smear on test sensitivity and specificity.

STUDY POPULATION AND METHODS

Study Population

Between September 2007 and January 2008, we screened all patients within 24 hours of admission to the medical wards of Mulago Hospital, an acute care general hospital in Kampala, Uganda, to identify those suspected of having tuberculosis. All patients with cough greater than 2 weeks duration were eligible for the study if they resided within 30 kilometers of the hospital, were not on anti-tuberculosis treatment, did not have clinical evidence of heart failure, and provided at least 2 sputum specimens for smear microscopy. Patients were excluded if any smear result was unavailable or tuberculosis status could not be established due to mycobacterial culture contamination (at least two negative cultures were required to exclude tuberculosis). The study protocol was approved by institutional review boards at Makerere University, Mulago Hospital, the Uganda National Council for Science and Technology, and the University of California, San Francisco.

Patient Evaluation

HIV testing was performed according to a sequential testing algorithm incorporating three rapid enzyme immunoassay kits. CD4+-T lymphocyte counts were measured in all HIV-infected patients. Sputum specimens were collected at enrollment (on the morning after hospital admission) and on the subsequent morning. In addition, HIV-infected patients with negative CM results underwent bronchoscopy with bronchoalveolar lavage (BAL) if referred by the treating ward physician. BAL samples were examined for the presence of mycobacteria, Pneumocystis jirovecii, and other fungal pathogens.

Smear microscopy and mycobacterial culture

Smear microscopy was performed at the Uganda National Tuberculosis Reference Laboratory (NTRL). For each sputum specimen, a direct smear was made using single frosted glass slides and stained using the hot Ziehl-Neelsen method (1% carbol-fuchsin dye).(11) The remaining specimen was then decontaminated (1% NALC-2% sodium hydroxide-1.5% sodium citrate) and concentrated by centrifugation at 3000 X g for 10 minutes.(12) A second smear was made using the concentrated specimen, labeled with a random identification number by a research study officer, and returned to the NTRL staff. The randomly labeled smear was stained with auramine-O for 15 minutes, decolorized with 3% acid alcohol, counter-stained with potassium permanganate for 60 seconds, and read within 24 hours.(12) For both CM and FM, NTRL technicians read a single two centimeter length of the smear. Ziehl-Neelsen stained slides were read using a standard light microscope (magnification 1000X, 100 fields) and auramine-O stained slides using a mercury vapor lamp fluorescent microscope (magnification 200X, approximately 30 fields). The presence or absence of AFB was reported using the 1998 WHO guidelines.(13) As recommended, the number of AFB observed with FM was divided by a factor of 10 to yield an approximate number that would have been observed if 100 fields were examined under 1000X magnification.(13) A patient was considered to be smear-positive if any AFB were detected in at least one sputum specimen.

Mycobacterial cultures were performed on Lowenstein-Jensen medium using a portion of the concentrated sputum or BAL fluid sediment.(13) Cultures were considered positive when mycobacterial growth > 0 colony forming units (CFUs) was observed within 8 weeks of incubation. Positive cultures were confirmed by staining with the Ziehl-Neelsen method.

Outcome Classification

The reference standard outcome of tuberculosis was defined as present if there was a positive mycobacterial culture result on any sputum or BAL fluid specimen (culture-positive TB).

Quality Assurance

Only sterile, distilled water was used to prepare all reagents. FM has been performed on most specimens submitted to the NTRL since 1988. All smears were read by one of five full-time laboratory technicians who have a median experience of 10 years (range 5–20 years) with FM. Since 2005, the Uganda NTRL has participated in a biannual external quality assurance (EQA) program for smear microscopy administered by the WHO. Re-checking of randomly selected slides was not routinely performed at the Uganda NTRL at the time of this study. However, known positive and negative control slides were included with each batch of slides read. For mycobacterial culture, contamination rates per slope and per individual technician were recorded on a monthly basis. “False-negative” culture rates were not systematically monitored (e.g., smears reported as positive were not re-read when corresponding culture results were negative).

Statistical Analysis

Bivariate analyses were performed using the chi-squared test for dichotomous variables, and the Mann-Whitney rank-sum test for continuous variables. Concordance was measured using unweighted kappa. The proportion of positive cultures within each direct and concentrated smear result category was compared using a generalized estimating equation logistic regression model to account for repeated measures. Sensitivity and specificity of CM and FM were calculated in reference to culture results and compared using McNemar’s test. Receiver operating characteristic (ROC) analysis was performed using the non-parametric method. All analyses were performed using STATA 9.0 (Stata Corporation, College Station, Texas), with the level of significance specified in reference to a two-tailed, type I error (p-value) less than 0.05.

RESULTS

Study Population

During the study period, there were 446 eligible patients and 426 (94%) were included in the analysis (Figure 1). Of the 20 excluded patients, smear results were unavailable in 10 (2%) and mycobacterial cultures contaminated in 10 (2%) patients. Baseline characteristics (gender, age, HIV status, anti-retroviral use) were not significantly different between excluded and included patients (Table 1). However, excluded patients had lower median CD4+ T-lymphocyte counts (23 vs. 48, p=0.04) and higher in-hospital (20% vs. 7%, p<0.04) mortality.

Figure 1. Study Population.

Of 446 patients eligible for the study, 426 (96%) were included. Pulmonary tuberculosis, defined as ≥ 1 positive culture result, was diagnosed in 240 (56%) patients.

Table 1.

Demographic and Clinical Characteristics

Characteristic	Study Cohort (N=426)	Excluded Patients (N=20)	p-value
Male, N (%)	197 (46)	12 (60)	0.23
Median age (IQR)	34 (28–42)	34 (26–39)	0.52
HIV-infection*, N (%)	350 (82)	18 (94)	0.16
Median CD4+ T-lymphocyte count (IQR)	48 (14–160)	23 (6–54)	0.04
Anti-retroviral use, N (%)	53 (15)	3 (17)	0.86
In-hospital mortality, N (%)	31 (7)	4 (20)	0.04

^*HIV status was unknown in one excluded patient

Of the 426 patients included in the analysis, 350 (82%) were HIV-infected. The median CD4+ T-lymphocyte count was 48 (Interquartile Range (IQR) 14–160) in this subset. Overall, 240 (56%) patients were found to have culture-positive tuberculosis. The diagnosis was confirmed by sputum culture in 228 (95%) and BAL fluid culture in 12 (5%) patients.

Specimen Quality

Of 852 specimens included in the study, 15% were salivary (contained no mucoid elements), 78% muco-purulent, and 7% contained blood. Excluding salivary sputum specimens from the analysis did not alter the estimates of diagnostic performance reported below.

Smear Reproducibility

The reproducibility of CM and FM results was evaluated in a subset of 247 consecutive sputum specimens. For these specimens, four separate smears were prepared: 2 for CM and 2 for FM examination. The two CM readings were concordant in 93% of cases (unweighted kappa 0.82), while the two FM readings were concordant in 74% of cases (unweighted kappa 0.41). In the majority (46/65, 66%) of cases in which FM results were discordant, one slide was classified as “scanty” (i.e., reported as 1–10 AFB/100 fields) while the other slide was classified as negative.

Smear Positive Proportion

At each smear-positive threshold evaluated, a significantly higher proportion of patients had positive FM compared to CM results (p<0.001 for all comparisons) (Table 2). For CM, a threshold of ≥ 1 AFB/100 fields increased the smear positive proportion by 4% (95% Confidence Interval (CI) 2–6%, p<0.001) compared to a threshold of > 4 AFB/100 fields and 7% (95% CI 4–10%, p<0.001) compared to a threshold of > 10 AFB/100 fields. For FM, a threshold of ≥ AFB/100 fields increased the smear positive proportion by 6% (95% CI 4–9%, p<0.001) compared to a threshold of > 4 AFB/100 fields and 12% (95% CI 9–16%, p<0.001) compared to a threshold of > 10 AFB/100 fields. Similar results were observed when the analysis was stratified by culture status.

Table 2.

Proportion of Smear Positive Patients, by Tuberculosis Culture Status

Threshold	Conventional Microscopy	Fluorescence Microscopy	p-value
All patients (N=426)
Any AFB	38	49	<0.001
> 4 AFB/100 fields*	34	42	<0.001
> 10 AFB/100 fields †	31	37	<0.001
Culture-positive (N=240)
Any AFB	64	72	0.005
> 4 AFB/100 fields*	58	68	<0.001
> 10 AFB/100 fields†	53	61	0.003
Culture-negative (N=186)
Any AFB	4	19	<0.001
> 4 AFB/l00 fields*	3	10	0.002
> 10 AFB/100 fields†	3	5	0.29

Abbreviations: AFB, acid-fast bacilli; HPFs, high power fields

^*Corresponds to observing > 40 AFB/length for fluorescence microscopy

^†Corresponds to observing > 100 AFB/length for fluorescence microscopy

For both CM and FM, there was an increasing proportion of positive culture results with increasing smear grade (p<0.001 for both trends). However, the proportion of positive cultures was significantly higher for CM compared to FM when the smear result was reported as scanty (90% vs. 54%, p<0.001) or 1+ (91% vs. 82%, p=0.02) (Table 3). No significant difference in the proportion of culture-positive CM and FM results was seen at higher smear grades.

Table 3.

Correlation between smear result category and positive culture result

	Conventional Microscopy		Fluorescence Microscopy
Smear Result*	Total	Culture-positive (N, %)	Total	Culture-positive (N, %)	p-value†
Negative	485	131 (27%)	446	124 (28%)	0.58
Scanty (1–9 AFB/100 fields)	48	43 (90%)	67	36 (54%)	<0.001
1+ (10–99 AFB/100 fields)	70	64 (91%)	57	47 (82%)	0.02
2+ (1–10 AFB/field)	41	40 (98%)	45	43 (96%)	0.62
3+ (> 10 AFB/field)	72	72 (100%)	101	100 (99%)	N/A††

Abbreviations: AFB, acid-fast bacilli

^*To compare conventional and fluorescence microscopy on the same scale, the number of AFB observed with fluorescence microscopy was divided by a correction factor of 10 to adjust for the lower level of magnification. The actual number of AFB observed with fluorescence microscopy was: Negative, 0 AFB/length; Scanty, 1–99 AFB/length; 1+, 100–999 AFB/length; 2+, 10–100 AFB/field; and 3+, > 100 AFB/field.

^†Compares proportion of positive cultures within each smear result category for conventional and fluorescence microscopy.

^††3+ smear result category dropped from the model due to co-linearity.

Diagnostic Performance

In per patient analysis, FM sensitivity was significantly higher than CM sensitivity (72% vs. 64%, difference 8%, 95% CI 2–14%, p=0.005) but FM specificity was significantly lower (81% vs. 96%, difference 15%, 95% CI 8–21%, p<0.001) (Table 4). When the analysis was stratified by HIV status, FM sensitivity remained significantly higher (71% vs. 63%, difference 8%, 95% CI 2–15%, p=0.005) and specificity significantly lower (76% vs. 96%, difference 20%, 95% CI 12–28%, p<0.001) in HIV-infected patients. However, in the 76 HIV-uninfected patients, there was no significant difference between FM and CM with respect to sensitivity (77% vs. 73%, difference 4%, 95% CI −21% to +15%, p=0.65) or specificity (98% vs. 96%, difference 2%, 95% CI −7 to +12%, p=0.56).

Table 4.

Sensitivity and Specificity of Smear Microscopy in HIV-infected and uninfected Patients

	Conventional Microscopy*	Fluorescence Microscopy*	p-value†
Overall (N=426)
% Sensitivity (95% CI)	64 (58–70)	72 (66–78)	0.005
% Specificity (95% CI)	96 (92–98)	81 (75–87)	<0.001
HIV-infected (N=350)
% Sensitivity (95% CI)	63 (56–69)	71 (65–77)	0.005
% Specificity (95% CI)	96 (91–98)	76 (68–83)	<0.001
HIV-uninfected (N=76)
% Sensitivity (95% CI)	73 (55–88)	77 (58–90)	0.65
% Specificity (95% CI)	96 (85–100)	98 (89–100)	0.56

^*Both conventional and fluorescence microscopy results were considered positive if ≥ 1 acid-fast bacilli per length were detected

^†McNemar’s test

The impact of “scanty” FM results (those classified as less than 1–29 AFB/length) on sensitivity and specificity estimates was explored using receiver operating characteristic analysis (Figure 2). For this analysis, patients were classified according to the highest number of AFB seen on examination of either sputum specimen. The maximum area under the curve (AUC) was obtained at a threshold of > 4 AFB/100 fields (sensitivity 68%, specificity 90%, AUC 0.82, 95% CI 0.78–0.85) (Figure 2). However, at this threshold, sensitivity of FM was not significantly higher than CM interpreted at a threshold of ≥ 1 AFB/length (68% vs. 64%, difference 4%, 95% CI −2% to +9%, p=0.19). FM specificity remained significantly lower (90% vs. 96%, difference 6%, 95% CI 1% to 11%, p=0.01).

Figure 2. Receiver Operating Characteristic (ROC) curve for Fluorescence Microscopy (FM).

The sensitivity and specificity of FM at different smear positive thresholds was assessed using ROC analysis. The points on the graph from left to right represent thresholds of 1 through 9 acid-fast bacilli (AFB) per length followed by 1+, 2+ and 3+ FM smear results. Maximum area under the curve (0.82, 95% CI 0.78– 0.85) was obtained at a threshold of > 4 AFB/100 fields and resulted in a sensitivity of 68% and specificity of 90%.

DISCUSSION

In this study, we found that FM was more sensitive but less specific than CM for the diagnosis of pulmonary tuberculosis in HIV-infected patients. To our knowledge, low FM specificity in HIV-infected patients has not been previously reported.(4) We found that decreased specificity was most common with “scanty” FM results. Sputum smears that were reported as “scanty” had less reproducible readings and were less likely to be associated with positive cultures. While FM increased the proportion of smear positive patients, our findings from a national reference laboratory suggest that FM may also lead to overtreatment of tuberculosis in high HIV prevalence settings.

Only one previous study evaluated FM performed on sputum concentrated using the NALC-NaOH method in HIV-infected patients, and reported that specificity was 100% in an outpatient cohort of Kenyan tuberculosis suspects.(5) The lower specificity observed in our study likely reflects differences between test performance in research versus field settings.(14) Research infrastructure allows for tight quality control, increased oversight, and strict adherence to standardized protocols. The Uganda NTRL has a high level of technician experience and participates in recommended EQA programs but still may not function at the same standard as laboratories utilized in clinical research studies.(15) Moreover, with expansion of FM to more peripheral laboratories, diagnostic results are likely to be less accurate than those obtained in national reference laboratories.

The potential for decreased specificity necessitates further consideration of the impact of false positive FM results. The current WHO/IUATLD recommendation to consider a smear positive when any number of AFB are detected maximizes case detection.(8) However, we found that 13–25% of tuberculosis cases diagnosed by FM based on this recommendation may be false positive. While current guidelines focus on increasing sensitivity, further studies are needed to define costs and outcomes associated with false positive and false negative test results prior to determining the optimal threshold for smear microscopy. In particular, unnecessary treatment of tuberculosis and failure to pursue alternate diagnoses may have adverse consequences in HIV-infected patients.

Several features of our study support the internal validity of our sensitivity and specificity estimates. First, unlike many previous studies, we implemented many of the recommendations of the STARD (Standards for the Reporting of Diagnostic Accuracy) guidelines for evaluating diagnostic tests.(16, 17) Previous methodologic studies have shown that failure to incorporate these design features – clear reporting of inclusion and exclusion criteria, blinded assessment of the test under investigation, use of a well-defined gold standard – is associated with biased estimates of diagnostic test performance.(18, 19) Second, chance is an unlikely explanation for our findings, since the upper bound of the 95% CI for our specificity estimate is 87%, which is still lower than previously reported.

Our study also has several potential limitations. First, false positive results were most likely when FM results were “scanty”. The false positive proportion may have decreased had we used liquid culture, a more sensitive culture technique, as the reference standard.(20) Second, we were not able to identify factors associated with false positive FM results in HIV-infected patients because there were only a few of these in absolute terms (N=34). Third, blinded re-checking of randomly selected slides was not performed at the Uganda NTRL during the study period. In contrast to our results, Shea et al recently reported both high sensitivity and specificity when FM was performed under study conditions in a research laboratory at the National Institutes of Health Clinical Center.(21) Though such stringent conditions are unlikely to apply, even in national reference laboratories, quality control procedures should be strengthened when implementing FM in low-income settings. Finally, our study population consisted of hospitalized patients, most of whom had advanced HIV infection. Though the point estimates of sensitivity and specificity reported here may not generalize to ambulatory or predominantly HIV-uninfected populations, these factors are less likely to affect the relative differences between CM and FM observed in our study.

In summary, our prospective, blinded study confirms that FM is more sensitive than CM for TB diagnosis in HIV-infected patients but may be less specific. The poor reproducibility and high false positive proportion observed with FM performed at the Uganda NTRL pose significant challenges to implementing this technique in more peripheral laboratories in low-income countries. Though there has been significant interest in expanding the use of FM, additional field data are needed to ensure effective implementation of this technology in low-income countries. Future studies should include outpatients, use a more sensitive gold standard, and evaluate the costs and benefits of false positive and false negative FM results.