The real-world performance and inter-observer agreement of urine lipoarabinomannan in diagnosing HIV-associated tuberculosis in an emergency center

Abstract

Background

The urine lipoarabinomannan (LAM) lateral flow assay is a point-of-care test to diagnose HIV-associated tuberculosis (TB). We assessed the performance of urine LAM in HIV-positive patients presenting to the emergency center and evaluated the inter-observer agreement between emergency center physicians and laboratory technologists.

Setting

A cross-sectional diagnostic study was performed at the emergency center of a district hospital in a high HIV-prevalence community in South Africa.

Methods

Consecutive HIV-positive adults presenting with ≥1 WHO TB symptom were enrolled over a 16-month period. A urine LAM test was done at point-of-care by an emergency physician, and interpreted independently by two physicians. A second test was done in the laboratory, and interpreted independently by two laboratory technologists. The reference standard was a positive TB culture or Xpert MTB/RIF test on sputum, or appropriate extra-pulmonary samples. We compared diagnostic accuracy and reproducibility of urine LAM between point-of-care readers and laboratory readers.

Results

1388 samples (median, 3 samples/participant) were sent for TB microbiology tests in 411 participants; 170 had confirmed TB (41.4%). Point-of-care and laboratory-performed urine LAM had similar sensitivity (41.8% vs 42.0%, p=1.0) and specificity (90.5% vs 87.5%, p=0.23). Moderate agreement was found between point-of-care and laboratory testing (κ=0.62), but there was strong agreement between point-of-care readers (κ=0.95) and between laboratory readers (κ=0.94). Positive percent agreement between point-of-care and laboratory readers was 68%, and negative percent agreement 92%.

Conclusion

There is no diagnostic accuracy advantage in laboratory-performed versus point-of-care performed urine LAM tests in emergency care centers in high burden settings.

Introduction

Diagnosing active TB in HIV-positive patients with advanced immunosuppression is challenging.¹ Patients with HIV-associated TB needing emergency care are at high risk for mortality, exacerbated by delay in starting TB treatment. Autopsy studies of HIV-positive adults reported that 46% TB cases were undiagnosed antemortem.² Rapid, simple, inexpensive, and accurate diagnostic tests for TB in HIV-positive patients are thus needed.

Assays that detect lipoarabinomannan (LAM) in urine are recommended by the WHO for use in HIV-positive adults who are seriously ill or have advanced immunosuppression (CD4 cell count ≤ 100 cells/μL).³ Although the sensitivity of the current assay is sub-optimal, urinary LAM performs better in patients at high risk of mortality,^4,5 where its use can decrease mortality.^6,7

The development of a low cost, rapid, lateral flow assay has made urine LAM a useful point-of-care test.⁸ However, most studies to date performed the LAM lateral flow assay in a laboratory,⁴ and none in an emergency care setting. We assessed the performance of urine LAM in HIV-positive patients presenting to the emergency center and evaluated the inter-observer agreement between emergency center physicians and laboratory technologists.

Methods

Design and Setting

A cross-sectional diagnostic study was performed at the emergency center of Khayelitsha Hospital in Cape Town, South Africa, which is a 300-bed district level hospital with an emergency center that manages about 30 000 patients per annum and has a 30% admission rate. A recent study found the prevalence of HIV infection was 23% in severely ill patients managed in the emergency center.⁹ The enrolment period was June 2016 through October 2017.

Population

All HIV-positive adults with a WHO TB symptom who were evaluated in the emergency center of Khayelitsha Hospital were eligible. Inclusion criteria were: age ≥ 18 years; HIV-positive (clinical records or laboratory confirmation); having at least one TB symptom (cough of any duration, fever, drenching night sweats, or weight loss); and informed consent. Exclusion criteria were: having taken anti-TB treatment within 3 months prior to enrolment; presenting to the emergency center more than 24 hours before screening; pregnancy; trauma, gynecological or psychiatric-related presentation; and main clinical presentation of meningitis syndrome or new focal neurology.

All participants provided written informed consent and the study was approved by the Human Research Ethics Committee of the University of Cape Town and the Western Cape Provincial Health Research Committee.

Procedures and samples

Consecutive patients admitted to the emergency center were screened for eligibility from Monday to Thursday. Demographic and clinical information were recorded on a standardized data collection form. Microbiologic tests to confirm TB included Xpert MTB/RIF (urine and sputum) and liquid mycobacterial culture of sputum and blood. Urine, blood and sputum samples were obtained from all patients whenever possible. Urine was collected in sterile single-use disposable containers. Patients who could not produce a spontaneously voided urine sample were catheterized. Urine samples were tested for the presence of LAM on site, then the residual urine was transported on ice to reach the laboratory within six hours of collection. Two sputum samples were collected; sputum induction with hypertonic saline in a ultrasound nebulizer was performed when a spontaneous sputum could not be produced. On-site urine LAM test results were made available to managing clinicians as soon as they became available. Samples collected as part of the routine standard of care by hospital clinicians (non-study) were not duplicated and appropriate extra-pulmonary samples were included as part of the reference standard.

Fresh urine samples were tested for the presence of LAM (Alere Determine™ TB LAM Ag test, Alere Inc., Waltham, MA, USA). A drop of urine (±60 μL) was applied to the sample pad on the test strip. The test was done once and independently read at different times by two clinicians between 25 min and 120 min later, under ambient hospital lighting conditions. One clinician (DJvH) was the primary reader for all the tests while the most senior clinician on duty in the emergency center on the day of enrollment performed the second read. Strict adherence to the suggested time frame for reading the strips was impossible since both clinicians still had clinical duties that sometimes delayed the reading of the test. However, reading was never done before the recommended minimum time. The second reader was blinded to the primary reader’s interpretation and to clinical data and other results. The lowest reading of the two readers was taken as the final point-of-care result. A separate urine LAM test was also performed in the laboratory according to the manufacturer’s recommendations.⁸ Two trained laboratory technologists independently recorded the result and the lowest result was taken as the final laboratory result. The laboratory readers were blinded to each other’s results, and to clinical status and microbiological test results. The final LAM result was deemed positive at the grade 1 cut-off.

CD4 counts were not repeated if performed within 3 months before enrollment. Study microbiology tests were performed by the research microbiology laboratory at the University of Cape Town. Microbiology samples were processed with standardized protocols in an accredited laboratory. Sputum and concentrated urine samples were tested using the Xpert MTB/RIF assay (GX4) (Cepheid Inc., Sunnyvale, CA, USA)¹⁰ and sputum was also cultured in liquid media (MGIT; Becton Dickson, Sparks, MD, USA). BACTEC MYCO/F Lytic blood culture bottles (Becton Dickson, Sparks, MD, USA) were filled with at least 5ml of blood and incubated for up to 6 weeks. The MTBDRplus assay (Hain Lifescience, Nehren, Germany) was used to identify cultured isolates as M. tuberculosis complex.

Case definition

A confirmed TB case was defined as the detection of M. tuberculosis complex by Xpert MTB/RIF and/or culture on any specimen from any anatomical site obtained during hospital admission (including any sample collected by clinical staff) and up to 6 weeks after hospital discharge. Tests done after discharge from hospital were identified using the National Health laboratory Service (NHLS) TrakCare Lab Webview – a web viewer providing access to all NHLS results within South Africa.

Statistical Methods

Statistical analysis

Summary statistics were used to describe the variables. Comparisons were done using the t-test or Mann-Whitney test when comparing means or medians, or the χ²-test when comparing proportions. Diagnostic test characteristics (with 95% confidence intervals) were calculated using standard formulas. Sensitivity and specificity were compared with the McNemar test, which was applied separately to participants with confirmed TB (sensitivity comparison) and those without (specificity comparison).¹¹ Reproducibility was measured using κ statistics with 95% confidence intervals, and was interpreted as previously described.¹² Statistical analyses were performed using MedCalc for Windows, version 18.5 (MedCalc Software, Ostend, Belgium; https://www.medcalc.org; 2018) and SPSS Statistics for Windows, Version 25.0 (IBM Corp. Released 2017. Armonk, NY: IBM Corp.).

Results

A total of 556 patients were screened, 427 (76.8%) of whom were enrolled; 16 patients were excluded from the analysis (see Figure, Supplemental Digital Content 1, which illustrates the flow of study participants). In total 1388 samples from the 411 evaluated participants (median per participant 3; range 1 to 9) were sent for microbiological testing. Fifteen participants (3.6%) had only one Xpert MTB/RIF, 11 (2.7%) only one culture, and 385 (93.7%) had at least one Xpert MTB/RIF plus at least one culture done. TB was confirmed in 170 (41.4%) participants (See Table, Supplemental Digital Content 2, for the results of confirmatory tests). The characteristics of study participants with and without confirmed TB are summarized in supplementary material (see Table, Supplemental Digital Content 3, for demographic and clinical characteristics of the study population). Sixty-three (26.1%) participants without confirmed TB were started on TB treatment of which seven were based on a positive point-of-care LAM test alone (see Table, Supplemental Digital Content 4, for reasons why TB treatment was initiated in relation to LAM results).

Two laboratory results were excluded (no test performed n=1 (TB confirmed); invalid result n=1 (TB not confirmed)). The result of the second reader of the laboratory tests was not completed for one participant and the reading of the single reader was taken as the final result. The point-of-care urine LAM test was positive in 94/411 (22.9%) participants, compared with 101/409 (24.6%) of the laboratory-performed tests (p=0.54). No point-of-care result was inconclusive, while 10 of the laboratory results were; these results were regarded as negative. Point-of-care and laboratory-performed urine LAM had similar sensitivity (p=1.0) and specificity (p=0.23) (Table 1).

Table 1. Comparison of diagnostic accuracy of lipoarabinomannan (LAM) between point-of-care and laboratory testing on urine samples.

	Point-of-Care*	Laboratory†
Sensitivity, % (95%CI‡)
Overall	41.8 (34.3 to 49. 6)	42.0 (34.5 to 50.0)
CD4 ≤ 100 cells/μL	55.1 (45.2 to 64.8)	57.6 (47.6 to 67.1)
Specificity, % (95%CI)
Overall	90.5 (86.0 to 93.9)	87.5 (82.6 to 91.4)
CD4 ≤ 100 cells/μL	86.5 (78.7 to 92.2)	83.8 (75.6 to 90.1)
Positive Predictive Value, % (95%CI)
Overall	75.5 (66.8 to 82.6)	70.3 (61.8 to 77.6)
CD4 ≤ 100 cells/μL	79.7 (70.5 to 86.7)	77.2 (68.3 to 84.2)
Negative Predictive Value, % (95%CI)
Overall	68.8 (65.8 to 71.6)	68.2 (65.1 to 71.1)
CD4 ≤ 100 cells/μL	66.7 (61.6 to 71.4)	67.4 (62.0 to 72.4)
Positive Likelihood Ratio, (95%CI)
Overall	4.4 (2.9 to 6.7)	3.4 (2.3 to 4.9)
CD4 ≤ 100 cells/μL	4.1 (2.8 to 6.7)	3.6 (2.3 to 5.6)
Negative Likelihood Ratio, (95%CI)
Overall	0.6 (0.6 to 0.7)	0.7 (0.6 to 0.8)
CD4 ≤ 100 cells/μL	0.5 (0.4 to 0.7)	0.5 (0.4 to 0.6)

^*Overall n=411, CD4≤100 cells/μL n=218;

^†Overall n=409 (missing (n=1) and invalid (n=1) results excluded), CD4≤100 cells/μL n=218;

^‡Confidence interval;

Inconclusive tests (n=10) regarded as negative (see Table, Supplemental Digital Content 5, where inconclusive results were excluded)

Strong agreement was found between the two point-of-care readers (κ = 0.95 (95%CI 0.92 to 0.99) and between the two laboratory readers (κ = 0.94 (95%CI 0.91 to 0.98) for both groups), but only moderate agreement between point-of-care and laboratory testing (κ = 0.62 (95%CI 0.53 to 0.71)) (Table 2). The positive agreement between point-of-care and laboratory readings was 69/101 (68.3%) and the negative agreement 283/308 (91.9%) (Table 2). Eight of the 11 participants with inconclusive laboratory readings had confirmed TB; six of these 8 had a positive point-of-care reading. In cases with differences in grade reading, most of the disagreements (55/90, 61.1%) were between negative and grade 1 cut-offs (see Tables, Supplemental Digital Content 7, for the level of agreement relating to difference in grades).

Table 2. Level of agreement of lipoarabinomannan (LAM) readings (positive or negative) between point-of-care reading and laboratory reading.

	Laboratory reading, n(%)
Point-of-care reading, n(%)	Negative*	Positive†
Negative	283 (69.2)	32 (7.8)	315 (77)
Positive†	25 (6.1)	69 (16.9)	94 (23)
	308 (75.3)	101 (24.7)	409 (100)

^*Includes inconclusive (n=10) results;

^†Urine lipoarabinomannan (LAM) grade-1 cut off or higher;

Urine not done by laboratory (n=1) and invalid laboratory result (n=1) excluded (see Table, Supplemental Digital Content 6, where inconclusive results were excluded);

Shaded areas represent agreement

Discussion

Our findings suggest that the urine LAM test can provide results in busy emergency care settings that are similar to those provided with laboratory testing. The modest sensitivity (42% overall) of urine LAM amongst HIV-positive patients presenting to the emergency center is similar to that described previously,⁴ and indicates that it should not be used as a rule-out test. High inter-observer agreements were found both between readers at the point-of-care (κ = 0.95) and the research laboratory (κ = 0.94), but only a moderate level of agreement (κ = 0.62) was found between point-of-care readers and laboratory readers. However, despite the lower level of agreement between point-of-care and laboratory readers, the diagnostic accuracy of urine LAM was similar between clinicians and laboratory technologists. Most (61%) of the disagreements between the laboratory and the point-of-care readings occurred at the grade 1 cut-off point, which may have significant clinical implications since this cut-off is used for making decisions on whether to treat for TB.

The diagnostic performance of urine LAM in our busy emergency care setting is comparable to that in other settings.⁴ Unlike most other studies, clinicians performing and reading the test strips were not dedicated study staff and had to fit the study-related processes in between their normal clinical duties. Despite there often being delays in reading the test strips, the diagnostic accuracy of point-of-care readings was similar to laboratory-performed readings.

The high inter-observer agreement between laboratory personnel and between point-of-care clinicians was similar to previous studies.^13–16 There are two possible explanations for the lower inter-observer agreement between the laboratory and point-of-care readings. First, urine samples should be tested immediately as there might be a decrease in LAM concentrations after 2 hours.¹⁷ Urine was transported on ice to reach the laboratory within 6 hours; 43 (11%) of laboratory readings had a lower score than point-of-care reading while 47 (12%) point-of-care readings had a lower score than the respective laboratory reading, which makes this explanation unlikely. Second, the interpretation of the reference card requires a subjective discrimination between the intensity of the color bands.¹⁸ Subjectivity might explain differences between individual readers but would not explain differences between groups of readers. Instead, there may have been a shared approach between point-of-care readers and a shared approach between laboratory readers, but the approach itself might have differed since the point-of-care group trained together but separately from the laboratory group. Lastly, the discrepancy could just relate to errors made by both groups, but in different patients.

Uninterpretable readings were more common than previously described. Up to 2% of tests have been reported as uninterpretable, although some studies had no invalid or uninterpretable results.⁴ Only the laboratory readers recorded uninterpretable readings and all by the same individual reader. It is thus possible that more strict criteria for defining uninterpretable readings were applied.

The diagnostic performance of urine LAM as a point-of-care test in a real-world emergency care setting was similar to that of laboratory reading. Urinary LAM is thus truly a point-of-care test.