Underestimation of the true specificity of the urine lipoarabinomannan (LAM) point-of-care diagnostic assay for HIV-associated tuberculosis

We read with interest the article by Drain and colleagues reporting on the diagnostic accuracy of a urine lipoarabinomannan (LAM) lateral-flow point-of-care diagnostic assay for HIV-associated tuberculosis (TB).¹ The ease of obtaining urine samples and the simplicity of this assay format make this a very attractive prospective tool.² With a growing number of evaluations of this assay having been published, the evidence base concerning the use of this assay is to be reviewed by a WHO expert panel in 2015.

Although studies agree that the sensitivity of the assay is only moderate at best, a number of features potentially compensate for this.² First, as a point-of-care assay, treatment decisions can be made at a single clinical encounter, greatly increasing the chances of treatment being quickly started following a positive test result. Second, sensitivity is highest among those patients who are sickest and have the highest mortality risk, thereby benefitting those with the greatest clinical need.³ Third, it provides useful incremental sensitivity to that provided by existing diagnostic tests, such as sputum smear microscopy.⁴

For this assay to be useful in practice, however, specificity must be very high such that positive results can be acted on with confidence. It has been suggested that the specificity of any new TB diagnostic assay should be at least 95% in comparison to culture.^5,6 It is therefore of concern that the assay evaluation by Drain and colleagues found an assay specificity of 92%,¹ similar to a specificity of 90% reported by the same authors in a previous evaluation.⁷ Several published studies have reported specificities of between 97% and 99%,^8-10 although some have also reported substantially lower specificities when using non-optimised cut-offs on the reference reading card.^9-11

There are two key methodological determinants of the assessed specificity of the urine LAM assay. The first is which of the reference card cut-offs is used and how the test strips are interpreted – issues which Drain and colleagues explored in their study. Suggested guidelines on reading the test strips were published in 2013¹² and have since been widely accepted and also adopted by the assay manufacturer, leading to modification of the reference card since January 2014. The second critical issue affecting assessed specificity is the use of an appropriate microbiological reference standard.

Specificity can only be reliably assessed if all participants included in a study are correctly classified as having TB or being TB-free following microbiological investigation. To accurately determine the TB status of HIV-infected patients can be difficult. HIV-associated TB is frequently extrapulmonary and is often challenging to either detect or exclude, especially in patients with advanced immunodeficiency.¹³ In their studies, Drain and colleagues used culture of a single sputum sample as the reference standard. As they themselves suggest, reliance on just one sputum sample may be insufficient. This could easily have given rise some false-negative reference standard tests, resulting in some true TB cases being misclassified as TB-free. This, in turn, could have given rise to some true-positive LAM results being classified as false-positives, directly resulting in underestimation of the specificity of the urine LAM assay. It was notable that assay specificity reported by Drain and colleagues was only 80% among those with CD4 cell counts <100 cells/μl; we suggest that this is because these are the very patients in whom extrapulmonary TB is most likely and in whom a single sputum reference standard is not likely to be adequate. Thus, we believe that differences in the robustness of the reference standard may well contribute to the substantial variation in the specificities for urine LAM assays reported by studies published to date.

In our previous studies of the use of urine LAM assays among ambulatory out-patients, we found that liquid culture of two carefully obtained sputum samples (with the assistance of a respiratory nurse and at least one of the samples being obtained by sputum induction) provided a very adequate reference standard. In both of these studies, the LAM assays were found to have excellent specificity (both ≥99%).^8,14

However, in our more recent study of HIV-infected patients requiring acute hospital admission, we used a much more comprehensive reference standard as we suspected that sputum alone would likely provide an inadequate assessment in these very sick patients with very advanced immunodeficiency.¹⁵ We used a comprehensive sampling strategy as part of a larger study, obtaining 1,745 respiratory and non-respiratory samples from the 427 study patients (mean, 4.1 samples per patient). The samples represented a median of three anatomic compartments per patient, such as sputum, blood, urine, pleural fluid, etc. TB was defined by at least one positive culture or Xpert test on any clinical sample.

In an exploratory analysis, we assessed the specificity of the LAM assay in one of two ways. First, using the data generated from the comprehensive reference standard (i.e., results from all respiratory and non-respiratory specimens), specificity was found to be 98.9% (95%CI, 96.9-99.8%). However, we then calculated what the revised specificity would have been if only respiratory samples were taken into account; this resulted in an assessed specificity of 89.6% (95%CI, 86.0-92.5%). Thus, we found that reliance on respiratory samples alone for the reference standard would have resulted in a substantial underestimation of the urine LAM assay specificity (by 9.3%) and that this then fell below the suggested acceptable threshold for a new diagnostic test of >95%.^5,6

It can be very challenging to obtain good quality sputum specimens from acutely ill HIV-positive hospital in-patients despite the assistance of a respiratory nurse and use of sputum induction in those in whom there is no contra-indication. Drain and colleagues studied out-patients, in whom it can also be difficult to obtain good samples in the typically overcrowded clinics in sub-Saharan Africa where space and facilities are all too often limited. In contrast, good quality non-respiratory samples such as urine and mycobacterial blood cultures can be readily obtained. Thus, we strongly advise that evaluations of the diagnostic accuracy of non-sputum-based diagnostic assays for TB (especially for HIV-associated TB) should use a more comprehensive reference standard that includes non-respiratory samples in addition to sputum.