Countrywide roll-out of Xpert^® MTB/RIF in Swaziland: the first three years of implementation

Abstract

Setting: All 19 public health laboratories in Swaziland that had Xpert^® MTB/RIF machines installed as part of a countrywide roll-out between June 2011 and June 2014.

Objective: To evaluate the utilisation and functionality of Xpert from 2011 to mid-2014.

Design: Descriptive study of Xpert implementation using routinely collected data.

Results: Of 48 829 Xpert tests conducted, 93% were successful: 14% detected Mycobacterium tuberculosis and 12% showed rifampicin resistance. The most common cause of unsuccessful tests was an ‘Error’ result (62%). Similar findings were obtained in government-supported and partner-supported laboratories. Annual utilisation of Xpert improved from 51% of maximum capacity in 2011 and 2012 to 74% in 2013 and 2014. A monitoring and supervision exercise of all Xpert testing sites in 2014 showed a generally good performance, with over 50% of laboratories achieving a ⩾80% score on most components. However, poor scores were obtained with equipment use and maintenance (6% achieving a score of ⩾80%), internal audit (19% achieving a score of ⩾80%) and process control (25% achieving a score of ⩾80%).

Conclusion: Countrywide roll-out of Xpert in Swaziland has been successful, although operational issues have been identified and need to be resolved.

Sputum smear microscopy remains the most widely used technology for the diagnosis of pulmonary tuberculosis (TB) in low- and middle-income countries. Although inexpensive to perform, smear microscopy is cumbersome, costly for patients, diagnostically insensitive and does not detect drug-resistant TB. New diagnostic tools are therefore urgently needed in the fight against TB.¹

The most important and revolutionary diagnostic development is a sensitive, specific, fully automated and commercially available nucleic acid amplification test, the Xpert^® MTB/RIF assay (Cepheid Inc, Sunnyvale, CA, USA), for use with sputum and other specimens.¹ Specimen processing prior to Xpert is greatly simplified compared with processing prior to culture, minimal laboratory expertise is required, results are provided in less than 2 h, and sensitivity and specificity for the diagnosis of TB and susceptibility to rifampicin (RMP) are high.¹ In 2010, the World Health Organization (WHO) strongly recommended the widespread use of Xpert, especially for individuals presumed to have multidrug-resistant (MDR) TB and human immunodeficiency virus (HIV) associated TB.^2,3

Work is ongoing to assess the feasibility, accuracy and effectiveness of Xpert at district and subdistrict health facilities.^4,5 The instrument's functionality in these settings depends on various operational factors, including cost, environmental temperatures, shelf-life of cartridges, electricity supplies, maintenance and the need for annual calibration of the machine.⁶

Swaziland is a lower middle-income country with a high burden of TB and HIV.⁷ In 2011, Xpert machines were introduced into the country with partner support, and although data on the tests performed and the number of patients diagnosed with TB and/or MDR-TB are available, there is little or no published information on the operational components of the technology, such as workload analysis, machine utilisation and operational challenges. Unpublished studies have been conducted in Lesotho and in Tanzania by the Foundation for Innovative New Diagnostics (FIND) to assess the implementation of Xpert, and a similar evaluation was performed by TB REACH in nine countries.⁵ This assessment, based on a convenience sample of projects, identified that electrical power supply and inconsistent recording and reporting were the main operational challenges.

This first national study of the operational use of Xpert conducted in Swaziland used a standardised laboratory monitoring and supervision checklist to assess operational and technical requirements of successful Xpert implementation. The country National TB Control Programme (NTCP) was also interested in finding out whether there were differences between government- and partner-supported sites and between urban and rural areas. This is a priority area for operational research in global TB control.⁸

The aim of this study was to evaluate the current utilisation and functionality of Xpert in Swaziland from 2011 to mid-2014. Specific objectives were to determine 1) the number of Xpert tests performed and the results, stratified by type of health facility laboratory; 2) site-specific monthly utilisation of Xpert per year in relation to the defined maximum monthly capacity for each machine; and 3) operational aspects and functionality of the Xpert machines, determined by the monitoring and supervision checklist scores.

METHODS

Study design

This was a descriptive study of Xpert implementation in Swaziland using routinely collected data and a standardised laboratory monitoring and supervision checklist.

Setting

General setting and the National Tuberculosis Programme

Swaziland, a small mountainous kingdom in southern Africa with a total population of 1.2 million, is classified as a lower middle-income country.^9,10 The country has the highest burden of TB per capita in the world, with estimates of 1380 per 100 000 population per annum.¹¹ This is combined with high HIV prevalence rates in the adult population: 26% among those aged 15–49 years, and over 40% in pregnant women.¹⁰ National TB control efforts have been complicated by the associated HIV-TB epidemic, with co-infection rates estimated at 80% and increasing MDR-TB rates of 7.7% in new cases and 33.8% in retreatment cases.^11,12 Swaziland declared TB a national emergency on 24 March 2011.¹¹ The focus of the NTCP is on expanding access to directly observed treatment (DOT), diagnostic services, treatment especially for drug-resistant TB and strengthening health systems. The National TB Reference Laboratory (NTRL), which has the overall responsibility for the TB laboratory network, has overseen the rollout of GeneXpert^® equipment for Xpert in all TB diagnostic laboratories and has adopted it as an initial diagnostic test for all presumptive TB cases in the country, regardless of HIV status or history of previous treatment.

Scale-up of Xpert machines in Swaziland

By June 2014, a total of 23 Xpert machines (22 4-module machines and one 16-module machine) had been installed in 19 TB diagnostic laboratories, with future plans to install 13 more machines.¹³ To increase access to TB laboratory services, an effective sample transportation system, the National Sample Transportation System (NSTS), has been provided, covering 78% of the country (the Lubombo, Manzini and Hhohho regions), with Médecins Sans Frontières (MSF) covering the remaining Shiselweni region. Samples for culture and drug susceptibility testing (DST) are sent to the NTRL through the NSTS and via DHL couriers.

Monitoring of Xpert at each site

A standardised laboratory monitoring and supervision checklist developed by FIND was used to assess the operational and functional aspects of Xpert in the TB diagnostic laboratories.^* For each laboratory, the checklist was completed with a numeric score assigned for each of the 12 quality system essentials. Each component had a maximum score possible, and laboratories were assessed as to whether they achieved ⩾80% of this score. If the score was <80% then the corrective action related to that component was carried out, including intensified supportive supervision, with the aim of reassessing the laboratory at the next quarterly supervisory visit.

Study sites

The study sites included all public health laboratories in Swaziland that had one or more Xpert machines installed between June 2011 and June 2014.

Data variables

Data variables were collected in relation to study objectives. For Objective 1, these included the number of machines and the number of Xpert tests conducted since installation, including successful tests with their results and unsuccessful tests with their reasons. For Objective 2, data variables included the number of tests performed for each Xpert machine. Given that a microscopist works 8 h per day, 20 days/month, the estimated maximum capacity for a 4-module machine is 400 tests/month (for the 16-module machine it is 1600 tests/month), excluding chances of error. The data sources for these two objectives were the archive files, retrieved and degenerated into raw data for analysis. For Objective 3, data variables included a score for each component on the standardised laboratory monitoring and supervision checklist. The data source was the standardised laboratory monitoring and supervision checklist used to assess laboratories during the course of the study. These laboratory assessments were conducted by WS and TZ.

Analysis and statistics

The data from the Excel files and the checklists were double-entered into an electronic EpiData file (version 3.1, EpiData Association, Odense, Denmark). A descriptive analysis was performed using frequencies and proportions. Comparisons were made between different types of health facility laboratory using χ² tests, with levels of significance set at 5%.

Ethics approval

Permission for the study was obtained from the Swaziland NTCP, the NTRL and the local Swaziland Scientific and Ethics Committee. The study met the MSF Ethics Review Board approved criteria for studies of routinely collected data, and was approved by the Ethics Advisory Group of the International Union Against Tuberculosis and Lung Disease, Paris, France.

RESULTS

The number of Xpert tests performed and the test results for the period from June 2011 to June 2014 are shown in Table 1. Ninety-three per cent of the tests were successful, and of these, 14% detected Mycobacterium tuberculosis. Most of these were drug-susceptible, with 12% showing resistance to RMP. The most common cause of an unsuccessful test was an ‘Error’ result (Table 2).

TABLE 1.

Number of Xpert^® MTB/RIF machines, tests performed and results, Swaziland, June 2011–June 2014

TABLE 2.

Main causes of ‘Error’ results in the tests performed with Xpert^® MTB/RIF machines, Swaziland, June 2011–June 2014

Government-supported health laboratories had more Xpert machines and performed more tests than partner-supported health laboratories, but the results were otherwise fairly similar between the two types of facility (Table 3). In government-supported health laboratories, the number of tests was similar in urban and rural sites, although urban sites recorded a significantly higher proportion of successful tests (P < 0.001) (Table 4). ‘Error’ results were the most common cause of an unsuccessful test in all settings. However, an unsuccessful test due to ‘No result’ was significantly more common in government-supported sites and in rural settings.

TABLE 3.

Number of Xpert^® MTB/RIF machines, tests performed and results in government- and partner-supported health laboratories, Swaziland, June 2011–June 2014

TABLE 4.

Number of Xpert^® MTB/RIF machines, tests performed and results in urban and rural government-supported health laboratories, Swaziland, June 2011–June 2014

The monthly utilisation of Xpert machines in relation to maximum monthly capacity for each of the 4 years is shown in the Figure (A–D). In 2011 and 2012, annual utilisation was approximately 50% of maximum capacity, and in 2013 and 2014 this increased to 74% for both years.

FIGURE A–D.

Average monthly utilisation of Xpert^® MTB/RIF instruments in relation to the defined maximum monthly capacity for each of the 4 years, Swaziland, 2011–2014. A) 2011; B) 2012; C) 2013 and D) 2014.

The number and proportion of laboratory sites achieving a score of ⩾80% for each component of the Xpert site monitoring and supervision checklist is shown in Table 5. More than half of the laboratories performed well on most components. The three components for which laboratories performed badly were equipment use and maintenance (6% achieved a score of ⩾80%), internal audit (19% achieved a score of ⩾80%) and process control (25% achieved a score of ⩾80%). For equipment use, the main cause of low scores was a lack of recording of maintenance and repairs to the machine by the operator. For internal audit, there was no established external quality assurance (EQA) scheme to validate sputum results from the Xpert tests in Swaziland. For process control, there was poor documentation of samples received and results obtained in the laboratory registers.

TABLE 5.

Number and proportion of laboratory sites achieving an 80% score for each component of the Xpert^® MTB/RIF site monitoring and supervision checklist as assessed in June 2014, Swaziland, stratified by government- and partner-supported sites

The results of the monitoring and supervision exercise were similar when partner-supported laboratories were compared with government-supported laboratories (Table 5). Within government-supported laboratories, almost identical results were found between urban and rural sites (Table 6).

TABLE 6.

Number and proportion of laboratory sites achieving an 80% score for each component of the Xpert^® MTB/RIF site monitoring and supervision checklist as assessed in June 2014, Swaziland, stratified by urban and rural government supported laboratories

DISCUSSION

This study of the national roll-out of Xpert in Swaziland in the last 3 years shows successful early implementation of the new technology. In terms of utilisation, nearly 50 000 tests were conducted. The large majority of tests were successful, and detected TB in one in seven patients. Of these, RMP resistance was identified in just over 10%. The main cause of an unsuccessful test was an ‘Error’ result, with power supply issues being the main reason, despite having an uninterrupted power supply for each Xpert machine. Fairly similar results of implementation were found between partner-supported and government-supported sites and between urban and rural government facilities. In the first 18-month period, utilisation of the Xpert machines was at 50% of maximum capacity, but this increased to about 75% in the second 18-month period, indicating better use of the resource as experience was gained.

The June 2014 monitoring and supervision exercise showed that laboratories generally performed well against a standardised checklist. The cut-off of 80% for each component was arbitrary, and related to what was thought to be a realistic standard in the routine setting for determining that corrective action was needed. With time and experience this cut-off will be set at a higher level. The main problems identified during supervision were related to health care worker performance, either with the maintenance of the machine or with the recording and reporting of sputum specimens and results. These could be corrected with refresher training, mentorship and onsite supportive supervision. With regard to the internal audit, in 2013 Swaziland enrolled all sites performing Xpert in an EQA proficiency testing scheme run by the US Centers for Disease Control and Prevention (Atlanta, GA, USA) International Laboratory Branch. However, during the course of the study, some reports were not available on site as evidence of participation, and suggested lack of feedback of proficiency testing results to sites.

The strengths of this study are that it was conducted nationwide and included government- and partner-supported health facilities. The study was a joint collaboration between the NTCP and the NTRL, and as similar methods were used to assess laboratories, extract and analyse data, there was no methodological bias. Finally, the study adhered to the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guidelines and sound ethical principles.^14,15 The operational challenges encountered may be context-specific, however, and not applicable to other countries.

There are some important lessons to be learnt from this roll-out of Xpert. The WHO Case Definition Framework, which includes Xpert results, was only updated in 2013,¹⁶ and lack of international guidance before this date has hampered the enumeration and proper documentation of results. As new technology is introduced, it is important to ensure that standardised methods of incorporating results into records and reports are updated as early as possible. At the time of roll-out of the Xpert machines, there was no dedicated government budget to support maintenance, annual calibration, modular replacements and purchasing of warranty packages, and we believe that this has contributed to the number of unsuccessful tests.

What are the implications of this study? First, there has been a considerable increase in the scale-up of Xpert. By September 2014, 3553 Xpert machines and over 8.8 million cartridges had been procured in the public sector in 110 of the 145 countries eligible for concessional pricing.¹⁷ The global evidence base needs more national studies, similar to this report from Swaziland, describing the implementation, utilisation and functionality of this new technology. Second, as the machines need to be functional all the time, the costs of maintenance, calibration and uninterrupted power supplies need to be incorporated into country budgets for machine function, maintenance and cartridge procurement. Cepheid has developed an extended warranty package (from 1 to 5 years) to accommodate some of the challenges for low- and middle-income countries. Third, the data need to be backed up centrally to ensure timely retrieval in case of system (computer) failure. Remote monitoring systems need to be integrated into the routine implementation of Xpert. Fourth, the one-off monitoring and supervision exercise that was carried out in June 2014 needs to be assessed to determine its value in identifying and rectifying problems and improving the output of successful tests. Finally, national monitoring should be combined with national programme supervision to record and report the treatment outcomes of TB patients as Xpert is further rolled out. Recent studies from Africa have shown that Xpert increases the number of confirmed TB cases and reduces the time between diagnosis and the start of treatment, but as yet there seems to be little impact on reducing morbidity and mortality.^18,19

In conclusion, the national roll-out of Xpert in Swaziland has gone well. The majority of the tests have been successful, with one in seven patients detected with TB, increasing utilisation of the machines as experience of their use is gained, along with fewer stock-outs of Xpert cartridges, more partner support for laboratory staff, more site supervision and increased confidence in using the technology. Monitoring and supervision have identified some operational issues that need to be resolved.