Accuracy of bronchoalveolar lavage enzyme-linked immunospot assay to diagnose smear-negative tuberculosis: a meta-analysis

Abstract

Purpose: While the bronchoalveolar lavage enzyme-linked immunospot assay (BAL-ELISPOT) shows promise for diagnosing smear-negative tuberculosis, its accuracy remains controversial. We meta-analyzed the available evidence to obtain a clearer understanding of the diagnostic accuracy. Methods: Studies of the diagnostic performance of ELI-SPOT on smear-negative tuberculosis were identified through systematic searches of the PubMed and EMBASE databases. Pooled data on sensitivity, specificity and other measures of accuracy were meta-analyzed using a random-effects model. Summary receiver operating characteristic curves were used to assess overall test performance. Results: A total of 7 studies were included in the meta-analysis. Diagnostic performance was as follows: sensitivity, 0.89 (95% CI 0.84 to 0.93); specificity, 0.78 (95% CI 0.74 to 0.81); positive likelihood ratio, 4.2 (95% CI 2.42 to 7.28); negative likelihood ratio, 0.14 (95% CI 0.06 to 0.33); diagnostic odds ratio, 36.16 (95% CI 9.70 to 134.73); and area under the curve, 0.9605 (SEM 0.0247). Conclusions: Available evidence suggests that BAL-ELISPOT may perform better than blood-ELISPOT for both screening and confirming a diagnosis of smear-negative tuberculosis. Nevertheless, BAL-ELISPOT should be not used alone but rather in parallel with clinical manifestations and conventional tests to ensure reliable diagnosis.

Introduction

Although the global incidence of tuberculosis (TB) has declined recently, it remains a leading cause of morbidity and mortality [1], affecting nearly a third of the world’s population [2]. Eliminating the disease remains the goal of the World Health Organization’s “Stop TB Strategy” [3]. A major challenge to eliminating TB is detecting it efficiently. Fewer than half of TB patients give positive results with acid-fast bacilli in sputum smear microscopy [1]. Culture and biopsy are much more reliable diagnostic tools, but sputum culture takes 6-8 weeks and biopsy is invasive [4].

In an effort to improve methods to diagnose smear-negative TB, bronchoalveolar lavage enzyme-linked immunospot (BAL-ELISPOT) is increasingly popular. Some studies suggest that its sensitivity and specificity is 100% in subjects with smear-negative TB [5], but others suggest much lower sensitivity of 73% and specificity of 48% [6]. This controversy around the diagnostic accuracy of BAL-ELISPOT should be addressed to help inform and guide its implementation for more efficient TB detection. Therefore we meta-analyzed the literature to estimate the diagnostic performance of BAL-ELISPOT for smear-negative TB.

Methods

Search strategy and selection criteria

We searched PubMed and EMBASE for studies of the diagnostic accuracy of BAL-ELISPOT to detect smear-negative TB that were published in English by June 1, 2015. The following search terms were used: “bronchoalveolar fluid” OR “bronchoalveolar lavage” AND “enzyme-linked immunospot” OR “t-spot” AND “sensitivity” AND “specificity” AND “diagnosis”. The literature was also searched for systematic reviews and meta-analyses, and reference lists in relevant articles were manually searched.

To be included in our meta-analysis, studies had to (1) use a case-control design; (2) clearly report diagnostic criteria for TB; and (3) report sensitivity and specificity data, or at least sufficient data to allow these parameters to be calculated. Two reviewers (Z.L. and W.Q.) independently assessed the eligibility of studies. Disagreements were resolved by discussion between the two reviewers.

Data extraction and quality assessment

Two reviewers (Z.L. and W.Q.) independently extracted the following data from the included studies: author, publication year, the country of origin, numbers and characteristics of participants, test methods, cut-off value, sensitivity and specificity data, data for assessing methodological quality, and the numbers of true positives, false positives, true negatives and false negatives for each study.

The methodological quality of each study was assessed using the quality assessment for studies of diagnostic accuracy (QUADAS-2), which has a maximum score of 11 [7]. Quality scores were computed based on the title, introduction, methods, results and discussion. A score of 1 was given if a criterion was fulfilled, -1 if it was not and 0 if the study did not provide enough information to decide whether a criterion was fulfilled.

Statistical analyses

Recommended methods for meta-analyzing diagnostic test evaluations [8] were performed using Stata 12 (Stata Corporation, College Station, TX, USA) and Meta-Disc 1.4 for Windows (XI Cochrane Colloquium, Barcelona, Spain). The following measures of test accuracy were computed: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and the area under the curve (AUC). A random-effects model was used to meta-analyze the data and provide pooled values of sensitivity, specificity and other accuracy measures [9,10]. I² values were used to assess whether significant heterogeneity was present across the meta-analyzed studies. Risk of publication bias was assessed using Deeks’ funnel plots [11]. All statistical tests were two-sided, and the threshold of significance was defined as P < 0.05.

Results

Systematic searching of the literature turned up 22 potentially eligible studies, of which 15 were excluded based on review of titles and abstracts. The remaining 7 studies were read in full and included in the meta-analysis [5,6,12-16]. The clinical characteristics of each study, along with QUADAS-2 scores, are summarized in Table 1.

Table 1.

Summary of included studies

Study	Country	No. cases/controls	Method*	Diagnosis	Cut-off	TP	FP	FN	TN	QUADUS-2
C. Jafari	Germany (2013)	19/77	BAL T-SPOT	culture M.t or clinical symptoms	five/double SFCs	17	29	2	48	10
C. Jafari	Germany (2009)	66/249	BAL T-SPOT	culture M.t or clinical symptoms	five/double SFCs	60	50	6	199	10
H. Li	China (2012)	36/66	BAL T-SPOT	culture M.t or histology biopsies	five/double SFCs	34	14	2	52	9
C. Jafari	Germany (2011)	24/87	BAL T-SPOT	culture M.t or clinical symptoms	five/double SFCs	22	11	2	76	9
C. Jafari	Germany (2006)	12/25	BAL T-SPOT	convential test or clinical symptoms	five/double SFCs	12	0	0	25	8
Cattamanchi A	USA (2012)	36/38	BAL T-SPOT	culture M.t	five/double SFCs	26	20	10	18	9
K Dheda	South Africa (2009)	18/32	BAL T-SPOT	culture M.t or histology biopsies	unclear	16	2	2	30	8

Abbreviations: TP: True positive; FP: False positive; FN: False negative; TN: True negative;

^*T-SPOT.TBH, Oxford Immunotec, Oxford, UK.

Study characteristics and reporting quality

The total sample size across all 7 studies was 785, including 211 subjects with TB and 574 subjects without it. In all studies, BAL-ELISPOT was performed on mononuclear cells from BAL that were stimulated with M. tuberculosis-specific antigens including early secretory antigenic target 6 (ESAT-6) and culture filtrate protein 10 (CFP-10). All but one study [16] used a detection cut-off of either five spot-forming cells (SFCs) in the well after subtracting the number in the negative control well, or a total number of SFCs in the well that was at least double the number of SFCs in the negative control well. All the studies had high QUADAS-2 scores.

Diagnostic accuracy

Figure 1 shows a Forest plot of sensitivity and specificity for BAL-ELISPOT in the diagnosis of smear-negative TB. The pooled sensitivity was 0.89 [95% confidence interval (95% CI) 0.84 to 0.93], while pooled specificity was 0.78 (95% CI 0.74 to 0.81) (Figure 1). PLR was 4.2 (95% CI 2.42 to 7.28) (Figure 2), NLR was 0.14 (95% CI 0.06 to 0.33) (Figure 2), and DOR was 36.16 (95% CI 9.70 to 134.73) (Figure 3). Heterogeneity was significant for all these parameters across studies, as indicated by I² values of 50.8% for sensitivity, 88.1% for specificity, 88.7% for PLR, 73.7% for NLR, and 82.0% for DOR.

Figure 1.

Forest plot of estimates of sensitivity/specificity for BAL-ELISPOT in the diagnosis of smear-negative tuberculosis. The point estimates of sensitivity/specificity from each of the studies are shown as solid circles, the sizes of which reflect the size of the study. Error bars show 95% confidence intervals.

Figure 2.

Forest plot of estimates of positive/negative likelihood ratio for BAL-ELISPOT in the diagnosis of smear-negative tuberculosis. Point estimates from each of the studies are shown as solid circles, the sizes of which reflect the size of the study. Error bars show 95% confidence intervals.

Figure 3.

Forest plot of estimates of diagnostic odds ratio for BAL-ELISPOT in the diagnosis of smear-negative tuberculosis. Point estimates from each of the studies are shown as solid circles, the sizes of which reflect the size of the study. Error bars show 95% confidence intervals. Reference numbers of cited studies are given.

Summary receiver operating characteristic (SROC) curves for BAL-ELISPOT were generated by plotting sensitivity against (1-specificity) for each study (Figure 4). The SROC curve for BAL-ELISPOT was located near the desirable upper left corner, and the maximum joint sensitivity and specificity was 0.96, with an AUC of 0.9605 (SEM 0.0247). At the same time, DOR was 36.16, indicating relatively high overall accuracy.

Figure 4.

Summary receiver operating characteristic curves for BAL-ELISPOT for diagnosing smear-negative TB. Each solid circle represents each study in the meta-analysis, and the size of the circle represents the size of each study.

Publication bias

Asymmetry in funnel plots was visible, suggesting risk of publication bias in our meta-analyses (Figure 5). However, the Deeks’ test gave a P = 0.58, suggesting no significant risk of publication bias.

Figure 5.

Funnel graph for evaluation of publication bias in the 7 included studies. The funnel plot shows the log of the diagnostic odds ratio (DOR) as a function of the standard error of the log of the DOR (an indicator of sample size). Each solid circle represents each study in the meta-analysis. The regression line is indicated.

Discussion

Here we show that the BAL-ELISPOT, which is much faster than the culture test for TB and much less invasive than biopsy, shows potential for screening and diagnosing smear-negative infection. However, it does not perform sufficiently well to stand alone and should not serve by itself as the basis for initiating therapy. Instead, it should be used in combination with clinical signs and conventional tests.

The BAL-ELISPOT assay is based on the first-line immune response to M. tuberculosis inhalation in the airway lumen, which appears to be more substantial than the response in lung interstitial tissue. For instance, studies have found elevated Th1 immune response in the airway lumen, but not the peripheral blood, in patients with active pulmonary TB [12-14]. This may help explain why BAL-ELISPOT appears to perform better than interferon-γ release assays (IGRAs) of peripheral blood, which show low sensitivity (81%) and specificity (59%) when used to diagnose active TB [18]. IGRAs measure the immune response with T cells from peripheral blood or the site of infection are manually stimulated with M. tuberculosis-specific antigens such as ESAT-6 and CFP-10 [17]. IGRAs with peripheral blood are poor at differentiating latent and active TB.

The DOR combines the data of sensitivity and specificity into a single number and is a single indicator of test accuracy [19]. It refers to the ratio of the odds of positive test results in individuals with disease relative to the odds of positive test results in individuals without disease [20]. Higher DOR values indicate better discriminatory test performance. Our meta-analysis found the DOR of BAL-ELISPOT to be 36.16, indicating relatively high overall accuracy.

The SROC curve and its AUC display the trade-off between sensitivity and specificity, and present an overall summary of test performance [20]. BAL-ELISPOT showed relatively high accuracy by these metrics, with pooled sensitivity of 0.89, pooled specificity of 0.78 and an AUC of 0.9605. Maximum joint sensitivity and specificity was 0.96.

DOR and SROC curves are not easy to interpret for clinical practice [21], and likelihood ratios are more clinically meaningful for measuring diagnostic accuracy. The meta-analyzed PLR of 4.2 suggests that patients with TB had an approximately 4-fold higher chance of giving a positive BAL-ELISPOT result than patients without TB. This is not high enough for clinical reliability. At the same time, NLR was 0.14 in the meta-analysis. In other words, a patient giving a negative BAL-ELISPOT result still has a 14% chance of having TB. This is not low enough to reliably rule out infection. These findings suggest that BAL-ELISPOT should not be used alone to decide whether to initiate anti-tuberculosis therapy.

One challenge with using BAL-ELISPOT is that key procedural aspects of the assay have yet to be standardized, such as the numbers of cells to use and the ratio of BAL to serum. This may help explain why the data from included studies showed significant heterogeneity for all diagnostic parameters tested. Future studies should seek to refine the BAL-ELISPOT platform and standardize it for greater reliability. These studies should also explore how to minimize risk of mycobacterial dissemination after lavage, and assess how the assay performs in TB patients coinfected with HIV.

This meta-analysis involved subjects from areas with high and low TB burden, which may make it more representative. At the same time, the meta-analysis has several limitations. The numbers of included studies and patients were relatively small. This means that our results cannot be regarded as definitive. It also prevented us from meta-analyzing results by subgroups based on methodological quality, though all the included studies had high QUADAS-2 scores. Our included studies showed significant risk of publication bias, which may reflect the fact that we excluded conference abstracts, letters to the editor, publications not in English and unpublished studies.

The present meta-analysis suggests a potential role for BAL-ELISPOT in screening and confirming a diagnosis of smear-negative TB. Since no diagnostic test for TB is specific for M. tuberculosis, BAL-ELISPOT should be interpreted in parallel with clinical manifestations or the results of conventional tests. The diagnostic accuracy of BAL-ELISPOT requires confirmation in further studies.

Disclosure of conflict of interest

None.