Cerebrospinal fluid AFB smear in adults with tuberculous meningitis: a systematic review and diagnostic test accuracy meta-analysis

Abstract

Background:

Cerebrospinal fluid (CSF) Ziehl–Neelsen acid-fast bacilli (AFB) smear is a rapid, cheap, widely available test for tuberculous meningitis (TBM). Yet, reported test sensitivity is highly variable. We performed a systematic review and meta-analysis for CSF AFB smear vs. other mycobacterial tests to diagnose TBM.

Methods:

We searched MEDLINE and Embase for studies reporting sensitivity and specificity of AFB smear against mycobacterial tests (reference standard) in adults (≥15 years) with suspected TBM. We used the QUADAS-2 tool to assess risk of bias. We estimated pooled sensitivity and specificity of AFB smear versus the reference standard using random-effects bivariate modeling. We used the I² statistic to assess heterogeneity between studies.

Results:

Of 981 articles identified, 11 were eligible for inclusion with a total of 1,713 participants. Seven studies were from high-TB burden settings and 4 from low-TB burden settings. The pooled sensitivity and specificity of CSF AFB smear were 8% (95%CI 3–21) and 100% (95%CI 90–100), with substantial heterogeneity in diagnostic performance (I² >95% for both) and reference standards.

Conclusion:

CSF AFB smear has poor sensitivity in most settings. If other more sensitive tests are available, those should be used preferentially rather than CSF AFB smear.

Background

Tuberculous meningitis is the most a severe form of tuberculosis (TB) with an estimated mortality of 24% overall, and up to 57% for HIV-positive persons, even when treated (1, 2). One of the persistent challenges with identifying and treating tuberculous meningitis is difficulty in confirming the diagnosis due to poor diagnostic test accuracy. There is currently no single ‘gold standard’ for the diagnosis of tuberculous meningitis. Inadequate tests cause delay in diagnosis and subsequent treatment leading to poor outcomes while also inhibiting the ascertainment of the true burden of tuberculous meningitis (3, 4).

Current diagnostic strategies for tuberculous meningitis are multifaceted and heterogeneous. In 2010, a committee of 41 tuberculous meningitis experts developed the Uniform Tuberculous Meningitis Case Definition (Marais Criteria) for use in clinical research (5). While this case definition has helped to standardize research, it is not appropriate for rapid diagnosis in routine clinical care as it depends on tests such as CSF culture, which can take up to 6 weeks to show a result, and may include tests that are not widely available in low-resource settings. More recently, nucleic acid amplification tests (NAATs), which may be rapid and have relatively good, pooled sensitivity (~70%) have been added to tuberculous meningitis diagnostics (6). However, many NAATs are not readily available in some resource-limited settings where tuberculous meningitis is most prevalent (4, 6). A cartridge based, rapid NAAT, Xpert MTB/Rif Ultra (Cepheid, Sunnyvale, CA, USA), is currently recommended as the first test to be utilized by the World Health Organization but does not have adequate negative predictive value to ‘rule-out’ TB meningitis (7). As a result, clinicians may choose to conduct multiple tests in an attempt to quickly identify and treat individuals with suspected tuberculous meningitis.

Cerebrospinal fluid (CSF) acid fast bacilli (AFB) smear is the most widely accessible and affordable rapid diagnostic test for tuberculous meningitis (8). CSF AFB smear is a microscopic evaluation of CSF that has been stained with the Ziehl-Neelsen stain to detect Mycobacterium tuberculosis bacilli. Preparation of AFB smear is a straightforward technique, however performing the microscopy requires expertise, and thus studies evaluating its performance on CSF vary widely. Many studies have demonstrated low (10–15%) sensitivity for CSF AFB smear (8). In contrast, a study in Vietnam comparing diagnostic methods for tuberculous meningitis reported a 78.6% sensitivity compared to the Marais criteria (9). The Vietnam/Oxford tuberculous meningitis research collaboration consistently has excellent results with CSF AFB smear, at least in part due to their expertise, experience, and diligence in securing relatively high volumes of CSF (at least 6 mL) with a relatively long duration of microscopy (at least 30 minutes) (10). Unfortunately, many other groups have not been able to replicate their success.

Despite the widespread use of CSF AFB smear in clinical and research settings for diagnosing tuberculous meningitis, no systematic review and meta-analysis has focused on its diagnostic performance. The objective of this systematic review was to assess existing evidence regarding the diagnostic accuracy (sensitivity and specificity) of CSF AFB smear compared with other mycobacteriology or molecular tests for the diagnosis of tuberculous meningitis.

Methods

Literature search strategy

A systematic electronic search was conducted across 8 databases: Medline and Embase via Ovid, Cochrane Library via Wiley, Web of Science Core Collection, Scopus, ClinicalTrials.gov, WHO ICTRP, and Global Index Medicus. We sought to identify all studies with diagnostic test accuracy results for CSF AFB smear among adults with tuberculous meningitis from 1988 to 2019. This time period corresponds to the WHO recommendation of standard quadruple therapy for the treatment of tuberculosis (11). Controlled vocabulary and natural language terms identified key search concepts such as: “tuberculosis”, “meningitis”, “acid-fast”, “sensitivity” and “specificity.” Full search strategies are presented in Appendix A. Searches were conducted on February 17, 2020.

The protocol for this systematic review, including detailed methods, is published at the International Prospective Register of Systematic Reviews (PROSPERO), registration number CRD42020171799. This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement for the reporting of diagnostic test accuracy studies (12).

Study selection

A two-stage screening process was employed: (1) at title and abstract; and (2) at full text level according to eligibility criteria as detailed below. This process was facilitated by Rayyan, a web-based systematic review screening tool (13). Screening was performed in duplicate by two independent reviewers and any unresolved disagreements were resolved by a third, independent reviewer. Reference and citation checking were conducted for included articles. Studies were eligible for inclusion if they (i) included adults (aged ≥ 15 years) with confirmed or suspected TB-meningitis; (ii) systematically utilized diagnostic criteria for diagnosing confirmed / suspected TB-meningitis including AFB smear data OR systematically compared AFB smear to culture and/or nucleic acid amplification testing; (iii) reported on at least one of the following outcome measures of interest: sensitivity, specificity, negative predictive value, positive predictive value of AFB compared to culture and/or NAATs or diagnostic criteria; (iv) employed any of the following study designs: consecutive case series, cross sectional study, case control study, cohort study, randomized controlled study, systematic review, or meta-analysis (retrospective and prospective); (v) included a minimum of 10 study participants.

The following exclusion criteria were applied: (i) studies not written in English; (ii) No comparison of AFB smear to another diagnostic test or reference standard (e.g. culture, NAAT, or Marais criteria); (iii) Full text unable to be located; (iv) No disaggregated TBM data; (v) No disaggregated adult data; (vi) Any study not in humans; (vii) Narrative reviews not adding new data or new analysis of data to existing knowledge; (viii) Commentaries and mathematical modelling studies; (ix) Any systematic review superseded by an updated systematic review.

Two independent reviewers assessed risk of bias using QUADAS-2 (University of Bristol, Bristol, UK), the recommended tool for evaluating primary studies for inclusion in systematic reviews involving assessment of diagnostic accuracy (14). Any discrepancy in bias assessment was resolved by a third, independent reviewer. Poor QUADAS-2 scores did not impact inclusion or exclusion of eligible articles.

Data analysis

There is no single, diagnostic reference standard for tuberculous meningitis. As such, we defined “reference standard” as any mycobacterial test used to identify tuberculous meningitis in the CSF as described by the included studies.

The sensitivity and specificity for each study were obtained and subsequently pooled. We used DerSimonian-Laird bivariate random-effects model to take into account correlation between sensitivity and specificity. We assessed heterogeneity across studies using the I² statistic and presented pooled sensitivity and specificity in a paired forest plot.

To provide an inference of diagnostic performance over different thresholds, we plotted a hierarchical summary receiver operating characteristic (HSROC) curve, in which the diagnostic accuracy of AFB smear was estimated by the area under the curve (AUC) and the summary operating point. We examined clinical utility of AFB smear using a Fagan plot (15).

We performed one subgroup analysis and two post-hoc sensitivity analyses to evaluate heterogeneity. First, we used a univariate meta-regression to compare diagnostic performance by criteria used to diagnose tuberculous meningitis (Marais Criteria vs other diagnostic criteria). Second, we restricted the meta-analysis to high-quality studies (showing no high risk of bias in any QUADAS-2 domains). Second, we examined the spread of the observed data and identified outliers using a bagplot. Then we restricted the meta-analysis to studies that fell in the 95% confidence bounds of the median distribution.

All analyses were conducted in Stata version 15·1 (StataCorp, College Station, TX, USA) with the “midas” command (16). The “metandi” command was used to construct HSROC curves (17).

Role of the funding source

The funders had no role in the study design, data collection, data analysis, data interpretation, or in the writing of the manuscript. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Search results, studies, and participants included

Our searches yielded 981 reports. After removal of duplicates (n=421), 560 studies underwent title and abstract screening, and 109 full texts were reviewed (Figure 1). Eleven studies met our eligibility criteria for inclusion and analysis with a total of 1,713 participants (Table 1) (10, 18–27). These 11 studies were published between 1995 and 2017 of which: 10 (92%) were cohort studies, and one (8%) was a case-control study. Studies arose from seven countries including a range of epidemiological settings; seven were from high-TB burden settings and four were from low-TB burden settings. Our meta-analysis includes diagnostic test accuracy of AFB smear for 761 adults with tuberculous meningitis and 954 adults with non-TB meningitis. Two studies included a total of 119 HIV-positive patients.

Figure 1:

PRISMA Flow Diagram of Study Selection Process

Table 1.

Characteristics of Studies Meeting Inclusion Criteria

Author	Year	Country	TB Burden	Study Design	N	HIV	Positive AFB	TBM by Ref	Reference standard1	Technique2	Volume3	Time/Fields4	Sensitivity %	Specificity %
Lin(18)	1995	Taiwan	Low	Cohort	45*	0	0	20	CSF Profile, Symptoms	ND	ND	ND	0	100
Desai(19)	2002	India	High	Cohort	105	0	2	33	NAAT	Concentration (details ND), ZN and auramine smears	ND	ND	6	99
Thwaites(10)	2004	Vietnam	High	Cohort	330	14	73	132	Culture, AFB	Centrifuged 3000g ×15 min.	1.5mL HIV−, 4.0mL HIV+	Up to 50 min.	55	100
Dil-Afroze(20)	2008	India	High	Case Control	47	0	0	27	CSF Profile, Symptoms	ND	ND**	ND	0	50
Mendes(21)	2010	Portugal	Low	Cohort	107	0	1	17	Culture, AFB	ND	ND	ND	6	100
Narotam(22)	2012	India	High	Cohort	64	0	2	32	Culture	ND	ND	ND	6	100
Feng(23)***	2014	China	High	Cohort	280	0	8	240	Definite, Probable, or Possible TBM	Centrifuged 3000g ×10 min.	2mL	300 fields	3	100
Neelakantan(24)	2014	India	High	Cohort	281	0	23	32	Culture	Centrifuged 3000g × 10 min., ZN and auramine smears	ND	ND	72	86
Bahr(25)	2015	Uganda	Low	Cohort	107	105	4	18	Definite TBM	Centrifuged 3000g × 15min.	0.1mL of 4mL re-suspended sample	ND	22	100
Park(26)	2016	South Korea	Low	Cohort	147****	0	3	38	Definite or Probable TBM	ND	ND	ND	8	100
Sun(27)	2017	China	High	Cohort	200	0	5	172	Definite TBM	ND	1mL	ND	3	100

¹Composite reference standard separated by comma

²Technique=any technique used to potentially increase sensitivity

³Volume/fields=median volume in mL used per smear/fields examined per slide

⁴Time=time in minutes on average spent examining each specimen

^*45 participants but with 47 samples (two participants with multiple samples from different time points), all four samples from these two participants were negative.

^**Not specifically discussed, 2–3mL used for NAAT, culture, and AFB smear

^***A modified ZN using 0.5mL, 100g ×5 min. in a cytospin chamber with poly-L lysine slides and examined 300 fields was completed in this study but not included in the meta-analysis as our focus was to quantify conventional ZN staining.

^****Though 276 samples were included in the study, only 147 had AFB smear completed

N=total cohort number; HIV=number people living with HIV among the cohort; AFB = acid fast bacilli; Positive AFB: number of cases positive by AFB smear among those positive by the reference standard; TBM=TB meningitis, TBM by Ref.=cases that qualifying as TBM per the reference standard in that study; CSF=cerebrospinal fluid;; CSF Profile = Combination of CSF protein, white blood cell count, white blood cell differential, and/or glucose; Culture = M. tuberculosis culture growth in CSF;; NAAT = Nucleic acid amplification test (e.g. PCR); Symptoms = B symptom evaluation; Definite, probable, and possible TBM refer to Marais criteria categories

ND=not discussed; min.= minutes

Reference standards varied for diagnosing tuberculous meningitis. Four more recent studies utilized the Marais criteria with two using definite TBM, one definite or probable TBM and another definite, probable or possible TBM (5). Four studies used culture (n=2), culture or AFB smear (n=2), or NAAT (n=1) as a reference standard – all published in 2014 or earlier. Two additional studies used a combination of patient symptoms and CSF evaluation (variable by study, included a combination of total protein, white blood cell count (+/− differential), and/or glucose)(Table 1).

Diagnostic accuracy of AFB smear

All 11 studies had disaggregated diagnostic data of CSF AFB smear compared with an eligible reference standard in the CSF (culture, NAAT, or both), which allowed for pooled sensitivity and specificity to be calculated. Individual study estimates for both specificity and sensitivity were not consistent across the studies (Figure 2). Point estimates from the 11 studies ranged from 0% to 72% for sensitivity and 50% to 100% for specificity. Compared with a reference standard, the pooled sensitivity of CSF AFB smear was 8% (95% Confidence Interval (CI); 3–21, I²=98%) and the pooled specificity was 100% (95%CI; 90–100; I²=95%). The area under the hierarchical summary receiver operating characteristic curve was 0·74 (95%CI; 0·70–0·77) (Figure 3).

Figure 2:

Diagnostic sensitivity and specificity of AFB Smear versus reference standard

TP=true positive, FN=false negative, FP=false positive, TN=true negative

Meta-analysis of the diagnosis of tuberculous mdeningitis in the 11 studies included. Reference standards for each of the studies included some combination of AFB smear, CSF profile, M. tuberculosis culture, NAAT, B symptom evaluation, and/or the Uniform Tuberculous Meningitis Case Definition (Marais criteria). Dashed vertical lines show the pooled estimates. I² shows inter-study heterogeneity.

Figure 3:

Hierarchical summary receiver operating characteristic (ROC) curve meta-analysis of diagnostic performance of AFB smear versus reference standard

HSROC=hierarchical summary receiver operating characteristic curve

Area under the hierarchical summary ROC curve C-statistic is 0·74 (95% CI 0·70–0·77). The confidence region shows the range that is likely to contain the population summary operating point and the prediction region is the range that is likely to contain where study data that are not yet observed would fall. Circle size corresponds to sample size in the included studies, where larger circles indicate larger sample sizes relative to other included studies.

With the estimated prevalence of tuberculous meningitis set at 20%, Fagan’s nomogram showed that CSF AFB smear would increase the probability of correctly detecting mycobacteriology-confirmed tuberculous meningitis in the study population by an absolute value of 23% (from a 19% pre-test probability to a 42% post-test probability) (Appendix C). When CSF AFB smear is negative, the probability that a patient could have mycobacteriology-confirmed tuberculous meningitis was 19% (Appendix C).

Pooled estimates of sensitivity and specificity by reference standard (Marais Criteria vs other) showed substantial heterogeneity (Appendix D). Sensitivity and specificity did not change significantly, but pooled estimates for studies that used the Marais Criteria as the reference standard had a narrower confidence intervals. However, these need to be interpreted with caution considering the small numbers.

Only one study had a potential risk of bias in the patient selection domain of QUADAS-2 (Appendix B) (20). A sensitivity analysis conducting the meta-analysis without the study with a high risk of bias yielded sensitivity, specificity, and I² estimates similar to the full analysis (Appendix E). There were three outliers – Dil-Afroze (2008)(20), Neelakantan (2014)(24), and Park (2016)(26) – identified from the bagplot that had values outside the 95% confidence bounds of the median distribution (Appendix F). After excluding these studies from the meta-analysis, heterogeneity in sensitivity remained the same while heterogeneity in specificity was completely accounted for (Appendix E).

Discussion

This systematic review and meta-analysis provides evidence that outside of centers with significant expertise in performing high quality CSF AFB smear – this method has limited utility for the diagnosis of TB meningitis. Our meta-analysis found that among 11 eligible studies with 1,713 participants, CSF AFB smear showed only 8% pooled sensitivity (95% CI 3–21%), implying that over 90% of TBM cases could be missed.

This study provides the most robust evidence to date confirming the low sensitivity of CSF AFB smear to diagnose TB meningitis in most settings. Our meta-analysis included data from seven countries with two thirds of included studies coming from settings with high TB burdens. The diversity of locations provides a more inclusive study population and broadens generalizability, although, of course, the related heterogeneity affects the analysis as well. Further, only one study was conducted in sub-Saharan Africa. That said, our strict inclusion criteria and study protocol as well as our thorough analysis of outlier data and data heterogeneity with subsequent sensitivity analyses further strengthens the overall conclusions of the data.

One limitation of our study was the variation in reference standards used. Heterogeneous reference standard selection is common in studies of TB meningitis diagnostic techniques. The Marais criteria were created to allow the possibility of a uniform research diagnostic reference standard. Yet, in our study we found four of six studies published after the availability of the Marais criteria made use of them and of those that used them, the Marais criteria utilized varied. Of the two additional studies one used culture only while the other used patient symptoms and basic CSF testing. This heterogeneity underscores the need for uniform study design when evaluating new diagnostic techniques. Ideally, both strict microbiologic reference standards and the Marais criteria should be utilized. Importantly, sensitivity was not significantly affected by the reference standard used (Marais Criteria versus other) while those studies who used the Marais criteria generally found better specificity. Although these smaller sub-analyses should be interpreted with caution, they are generally reassuring when considering the effect of the heterogeneity of reference standard use on the study results.

Another limitation is the heterogeneity of test performance between studies (I² >95% for both pooled sensitivity and specificity). Figure 2 shows outlier studies for both sensitivity and specificity. Sensitivities were low outside of two studies, one in Vietnam that found 55% sensitivity among 330 subjects, and another in India that found 72% sensitivity among 281 subjects (10, 24). Most specificities in individual studies were quite high, with one small study (n=47) standing out with a specificity of only 50% (20). Thus, the high pooled specificity in spite of this one study’s findings. Removal of outliers from the analysis (Appendix F) completely accounted for the heterogeneity in specificity although heterogeneity for sensitivity remained unchanged.

These findings drive home that differences in operator expertise in CSF AFB smear, time spent on each sample, and volume of CSF utilized can vary widely and significantly affect results. Thus, for sites with the capacity to develop significant expertise in reading AFB smears, who are able to allow significant time (>30 minutes) to read the smear, and who are able to routinely dedicate 6 mL of CSF to AFB smear – the test may be an option (10). However, in more routine scenarios where expertise in reading AFB smears is difficult to obtain and takes years to develop, the test has limited utility if other, more sensitive and less user-dependent tests are available. At present, the combination of GeneXpert MTB/Rif Ultra and culture present the best chance to diagnose TB meningitis in most settings, assuming availability and the ability to manage each test’s limitations. GeneXpert Ultra is rapid (~2 hours) and has sensitivity of approximately 60–80% but requires a stable electrical supply (7). Culture has sensitivity of approximately 50–60% but is slow, with results in 2–6 weeks (7). Both tests are expensive and require significant infrastructure investments and each test can discover cases that the other test misses (7). Thus, in order to utilize these technologies, significant investments are required.

Despite the required investments, CSF should be devoted to these tests when possible in most situations to maximize their sensitivities. If these investments are truly not possible, then AFB smear is a reasonable test to perform. However, decision makers should be aware that significant numbers of cases will be missed in TB endemic areas if such a choice is made and that missed cases are nearly uniformly fatal. Clinical expertise remains critically important in diagnosing TB meningitis and can inform interpretation of these and all diagnostic tests for TB meningitis.

In summary, we found poor pooled sensitivity and excellent pooled specificity for CSF AFB smear for the diagnosis of TB meningitis with high study heterogeneity. Those who utilize AFB smear for TB meningitis should be aware of its limitations in most settings and if possible, utilize more sensitive tests to maximize the likelihood of a correct diagnosis.

Research in Context.

Evidence before Study

Cerebrospinal fluid (CSF) Ziehl–Neelsen acid-fast bacilli (AFB) smear microscopy is a rapid, cheap, widely available test for tuberculous (TB) meningitis. This test is available in nearly every location in which a lumbar puncture can be performed. Most, but not all studies reporting sensitivity have reported low values for AFB smear, but a meta-analysis has not been conducted on this topic to summarize the evidence.

Added value of this study

This systematic review and meta-analysis contributes a high-quality quantification and summary of the available data to inform clinicians as well as local health systems and country-level health programs as to the sensitivity of CSF AFB smear. The pooled sensitivity of 8% (95%CI 3–21) shows that across settings, CSF AFB smear performed poorly, albeit with significant heterogeneity between studies. This study also reaffirmed the generally high specificity of CSF AFB smear for TB meningitis diagnosis. This study summarizes the highest quality data in a way that had not been previously done.

Implications of all the available evidence

Though there are settings in which high performance of AFB smear has been reported, these are the minority and have developed impressive expertise over many years to achieve their proficiency with AFB smear. Those without the ability to dedicate significant time and resources to AFB smear quality should not assume they can replicate adequate performance of AFB smear for TB meningitis. Clinicians, health systems, and national health programs should invest in rapid technology with higher sensitivities such as GeneXpert MTB/Rif Ultra (recommended as the first test for TB meningitis by WHO) and less rapid technologies such as mycobacterial culture to diagnose TB meningitis. If these investments are truly not possible, then AFB smear is a reasonable test to perform. However, decision makers should be aware that significant numbers of cases will be missed in TB endemic areas without investments in sensitive technologies and that missed cases are nearly uniformly fatal. Further, utilizing AFB smear in addition to more sensitive technologies is not advisable in most settings as AFB is unlikely to have a positive result that was ‘missed’ by other technologies. Rather than performing AFB smear in these settings, a better use of the CSF would be to devote all possible CSF volume to the more sensitive tests, increasing the likelihood of diagnosing TB meningitis when it is present.