Evaluation of the implementation of the Xpert® MTB/RIF assay in Fiji

Abstract

Setting: All Xpert® MTB/RIF tests performed in the three TB (tuberculosis) treatment centres in Fiji from June 2012 to February 2013.

Objectives: To determine 1) the number of Xpert tests performed in each centre, 2) the association between sputum quality and Xpert results, 3) the agreement of Xpert with acid-fast bacilli (AFB) smear microscopy and TB culture and 4) error rates.

Design: Retrospective review of records.

Results: A total of 415 Xpert tests were performed in the study period. Mycobacterium tuberculosis was detected in 69 (16.6%) samples. No rifampicin resistance was detected. M. tuberculosis was detected from 60 (18.7%) good-quality sputum samples. A total of 43 (10.4%) errors occurred during this period. M. tuberculosis was detected in 10 (2.9%) smear-negative specimens. There was a substantial and an almost perfect agreement between Xpert and AFB microscopy (κ = 0.793) and culture results (κ = 0.818), respectively.

Conclusion: Although a good correlation between Xpert and the two tests were shown in the study, Xpert should not replace the routine first-line TB diagnostic tests used in Fiji for reasons related to logistics and sustainability. A further evaluation of the assay's performance is required over a longer time period to gauge its diagnostic value in detecting smear-negative, Xpert-positive cases in Fiji.

Smear microscopy for acid-fast bacilli (AFB) is the standard tool for the diagnosis of pulmonary tuberculosis (TB) caused by Mycobacterium tuberculosis in Fiji. Sputum culture using Ogawa medium is used to complement AFB smear microscopy, especially among smear-negative specimens. Diagnosis of TB in Fiji is constrained by the low sensitivity of AFB smear microscopy, the inability to determine drug susceptibility and the variability in operator performance. In addition, solid culture takes about 6–8 weeks to obtain a result, and significant numbers of cultures become contaminated.

In June 2012, the Fiji National TB Programme introduced the Xpert® MTB/RIF assay (Cepheid, Sunny-vale, CA, USA) at the three anti-tuberculosis treatment centres in Fiji to increase case detection rates and investigate rifampicin (RMP) resistance. Xpert is a cartridge-based, fully automated, real-time nucleic acid amplification test for rapid detection of M. tuberculosis and RMP resistance, which is a good marker for multi-drug-resistant TB (MDR-TB). Results are obtained within 2 h, which helps reduce the length of hospitalisation and the number of deaths due to TB.¹ The results from a recent study showed that the mean time to TB detection was <1 day for Xpert, whereas smear microscopy requires a day.²

A number of issues have been identified while implementing Xpert in Fiji, and this requires further investigation. These included a significant number of errors during sample testing and issues related to the short shelf-life of the kits. Similar issues were identified by the World Health Organization in 2011.² Fiji now has three diagnostic tools for TB, with a degree of overlap in their functionality. There is therefore a need to understand the diagnostic value of Xpert and its level of agreement with AFB smear microscopy and TB culture in Fiji. It was also important to compare the performance of the three TB treatment centres and the quality of sputum to determine if these had contributed to the errors. We therefore conducted this study to provide answers to some of these questions and to prevent the previously mentioned constraints from recurring.

METHODS

This was a retrospective study using data from all samples tested with Xpert in the three anti-tuberculosis treatment centres in Fiji from June 2012 to February 2013.

Setting

Fiji is an archipelago of more than 322 islands which lies at the heart of the Pacific Ocean. The population of Fiji is approximately 837 271,³ 87% of whom reside in the two major islands, Viti Levu and Vanua Levu. There are three anti-tuberculosis treatment centres and four AFB smear microscopy laboratories in Fiji. Tuberculosis culture on Ogawa medium is only performed by the Daulako Mycobacterium Reference Laboratory (DMRL), located in P J Twomey Hospital in the capital, Suva. Xpert testing is performed at the DMRL, Lautoka and Labasa Hospitals. The National Tuberculosis Programme (NTP) criteria for the use of Xpert include patients who are symptomatic but AFB smear-negative (on three specimens), AFB smear-positive but failed to convert after 3 months of the intensive phase of treatment, relapses after apparently successful treatment, human immunodeficiency virus (HIV) positive patients and cases from high multi-drug-resistant TB burden countries.

Data variables, data collection instrument

Data for Xpert testing performed between June 2012 and February 2013 were extracted from the AFB smear microscopy register in the laboratories in each of the three anti-tuberculosis treatment centres. Data for each variable were manually collected into a structured proforma. Exposure variables for each sputum specimen included quality of sputum (poor or good), age, sex and testing laboratory. Quality of sputum was classified as follows: good = blood-stained, mucoid or purulent; poor = thin, watery or mainly composed of bubbles and saliva. Outcome variables included Xpert, smear and culture results. AFB smear microscopy results were classified according to the International Union for Tuberculosis and Lung Disease (The Union), Paris, France.⁴

Analysis and statistics

Data were collected manually and double-entered into an electronic database (EpiData, version 3.1, EpiData Association, Odense, Denmark). A uniform data entry file was developed using EpiData Entry 3.1. Errors in data entry were detected by comparing the duplicate EpiData files for any discordance. Corrections were made by cross-checking with the proformas to produce a final data set.

Data were first analysed using descriptive exploratory statistics. The statistical significance of observed differences between the proportions of determinants and outcome variables was determined using the χ² test at a 95% confidence level using EpiAnalysis (EpiData Association). The level of agreement between tests (κ), sensitivity and specificity and their binomial 95% confidence intervals (CIs) for Xpert using culture and AFB smear microscopy, were calculated using WinEpi.⁵

Ethics approval

Ethics approval was obtained from the Ethics Advisory Group (EAG) of The Union, the National Health Research Committee, Suva, and the Fiji National Research Ethics Review Committee, Suva, Fiji.

RESULTS

The results from testing 415 sputum specimens using Xpert are presented in Table 1. The majority of the samples were tested at the DMRL (n = 164, 39.5%), followed by the Western (n = 114, 27.5%) and Northern Divisions (n = 137, 33.0%).

TABLE 1.

Characteristics of sputum specimens tested using Xpert® MTB/RIF at three anti-tuberculosis treatment centres, Fiji, June 2012–February 2013

Sputum quality

A total of 94 (22.7%) sputum specimens were of poor quality. M. tuberculosis was detected in 60 (18.7%) good quality and 9 (9.6%) poor quality sputum specimens using Xpert. The overall error rate was similar in good (10.3%, 95%CI 7.3–14.3) and poor quality (9.6%, 95%CI 4.7–17.8) sputum specimens. There was a significantly higher positivity rate in good than in poor quality specimens (20.9%, 95%CI 16.4–26.2 vs. 10.6%, 95%CI 5.3–19.6, P < 0.05).

AFB smear microscopy

M. tuberculosis was detected in 10 (2.9%) AFB smear-negative specimens. Three (27.8%) of the specimens were smear-positive but with no M. tuberculosis detected using Xpert. Xpert errors were detected in 34 (10.6%) AFB smear-negative specimens and in 9 (9.6%) AFB smear-positive specimens.

The sensitivity and specificity of Xpert using microscopy as the gold standard was respectively 80.8% (95%CI 71.8–89.9) and 96.8% (95%CI 94.8–98.7). There was substantial agreement between microscopy and Xpert results (κ = 0.793).

TB culture

Of the 217 (52.3%) sputum specimens tested using both Xpert and culture, 10 (2.4%) were contaminated; 3 (5.5%) specimens were Xpert-negative but culture-positive. The sensitivity and specificity using culture as the gold standard were respectively 93.2% (95%CI 85.7–100.6) and 93.1% (95%CI 89.0–97.2). There was almost perfect agreement between culture and Xpert results (κ = 0.818).

Errors

We detected a total of 43 (10.4%) errors. There were no significant differences in error rates in testing conducted in the three divisions. The results of different error types are presented in Table 2. Error ‘5007’ (n = 10) and ‘no result’ (n = 11) were the most common errors recorded. DMRL recorded a high number of errors due to the large number of samples tested. The number of errors detected at the Lautoka and Labasa Hospitals was the same (n = 12).

TABLE 2.

Number and type of error recorded after testing sputum specimens for the presence of Mycobacterium tuberculosis using Xpert® MTB/RIF in three laboratories, Fiji, June 2012–February 2013

DISCUSSION

This is the first study in Fiji to evaluate the implementation of Xpert for TB diagnosis. Fiji does not currently perform routine drug susceptibility testing (DST) for M. tuberculosis isolates. In cases where DST is needed, culture isolates on Ogawa are sent to a reference laboratory in Australia for testing. To date, RMP resistance has not been detected in any M. tuberculosis culture isolates from Fiji,⁶ and no RMP resistance was detected using Xpert during the study period.

One of the most significant logistical issues experienced during implementation was the overestimation of Xpert cartridge requirements in the implementation phase, and the Xpert cartridges were obtained in a single shipment. In addition, only a small number of samples met the algorithm developed by the NTP for Xpert testing. Furthermore, as the purpose of the new test and the structure of the implementation phase were not communicated fully to the clinical staff in the hospitals in Fiji, except for those in the TB clinics, only a small number of requests came from doctors in the divisional hospitals, which have a large daily turnover of patients. Sputum samples were only submitted if the microscopy results were indeterminate. The end result was that about 30% of the 600 cartridges were discarded due to their short shelf-life. This was followed by a lapse of 7 months before the second batch of test cartridges arrived. This suggests that the Fiji NTP needs both to revise its procurement guidelines to improve stock management for newly introduced tests and also to initiate training in Xpert and its value for medical officers, not only at the TB clinics but at all the other facilities in Fiji. In a similar study by Creswell et al. evaluating data from nine countries, cartridges were shipped in different batches to avoid waste.⁷ In addition, NTP guidelines for Xpert in the nine countries were more focused on detecting drug resistance than on active case finding.⁷

It is interesting to note that a high number of errors occurred in good quality sputum. We also documented high rates of M. tuberculosis in good quality sputum specimens. Use of good quality sputum for Xpert testing was also recommended in a study in Zimbabwe.⁸ Rectifying this error was therefore a priority for Fiji. A similar problem was observed in India after Xpert was decentralised to a lower level of the health system.⁹ The majority of the failures in this case were caused by the high viscosity of good quality sputum samples. It was therefore recommended that sputum specimens with high viscosity be incubated for an additional 10 min before Xpert testing.⁹

Xpert correctly identified M. tuberculosis in 80% of the AFB smear-positive specimens, indicating that it reliably identifies the presence of M. tuberculosis in patients with a high bacterial load. In Fiji's low-prevalence setting, the number of new Xpert-positive, smear-negative specimens was low (n = 3, 0.9%). In such a low-burden setting, the NTP needs to re-evaluate its stringent algorithm for Xpert testing. In South Africa, the time to diagnosis increased by almost two-fold when Xpert (82%) was used for diagnosis followed by smear microscopy, culture and clinical observations (44%).¹⁰ In the multicentre study by Creswell et al., Xpert was used as a second test for smear-negative cases in the nine countries studied.⁷ It is important to note that Xpert can also be performed for highly suspected clinical cases.

Of the 10 cases that were AFB smear-negative but Xpert-positive for M. tuberculosis, 3 were new cases and 7 were follow-up patients who were on anti-tuberculosis medication at the time. The seven follow-up specimens were tested using Xpert only to check for any RMP resistance, as Xpert is not used to monitor treatment.

The almost perfect agreement between Xpert and culture is reassuring, as it suggests that culture is being performed relatively efficiently. Surprisingly, Xpert was unable to detect two AFB-positive smears as positive, although these were confirmed as positive on culture. The two cases were follow-up patients. This result was similar to a study in Uganda, where it was documented that the assay was unable to detect M. tuberculosis in 20.6% of children with culture-confirmed pulmonary TB.¹¹ Similar results were found in a study performed in two hospitals in Cape Town, South Africa, where nearly a quarter of the children were M. tuberculosis-negative using Xpert but positive using culture.¹² A thorough clinical assessment is therefore important in introducing anti-tuberculosis treatment in Xpert-negative patients. Culture thus remains the gold standard for the diagnosis of TB, as it has the ability to grow low concentrations of bacteria from sputum isolates, which can be used to perform DST and phenotyping.

Using Xpert, 43 (10.4%) errors were detected. The error code ‘no result’ (n = 11, 26%) indicates a power failure; however, in this case no power failure was recorded in the three centres and all were using an uninterrupted power supply. Furthermore, although Cepheid recommends that Xpert machines be calibrated after 2000 cycles or after 1 year,⁵ whichever comes first, calibration had still not been performed 1 year after installation. In the early programmatic implementation of Xpert in nine countries, the assay was repeated if it failed in the first run.⁷

As data from all Xpert tests were acquired from all the three centres in Fiji, this study is therefore representative of the nationwide implementation of Xpert. AFB microscopy and culture results were also extracted for comparison purposes, and the data collected were validated by cross-checking with the electronic data stored in Xpert.

The age of the samples could not be analysed due to unavailability of the date of specimen collection. Although the dates recorded in the laboratory registers indicated when the sample had been received by the laboratory, they could not be taken as date of collection, as the samples were referred from different health facilities. We could not therefore determine whether the age of the sample affected Xpert results, and particularly the errors. However, in India, regional distribution, demographic profile and level of decentralisation were not related to test failures.⁹ Nevertheless, it is still recommended that all health facilities follow national guidelines and correctly identify the date of collection both in the request form and on the specimen container before sending them to the test laboratory, both for Xpert testing and for culture, where the risk of contamination is high.

As Fiji will soon be introducing liquid culture on MGIT™ (BD, Sparks, MD, USA), further research with larger samples and comparisons of the four tests can be carried out.

CONCLUSION

This study shows a good correlation between Xpert and the two tests routinely used for TB diagnosis in Fiji. However, it also outlines a number of manageable obstacles to implementation. Further evaluations of Xpert are required to re-evaluate the performance of the test in detecting smear-negative, Xpert-positive cases over a longer time period. Because of logistical and sustainability issues, Xpert cannot replace routine first-line diagnostic tests in Fiji.