Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/supranational reference laboratory

Joel Kabugo; Joanita Namutebi; Dennis Mujuni; Andrew Nsawotebba; George William Kasule; Kenneth Musisi; Edgar Kigozi; Abdunoor Nyombi; Pius Lutaaya; Fredrick Kangave; Moses L Joloba

doi:10.1371/journal.pone.0251691

. 2021 May 14;16(5):e0251691. doi: 10.1371/journal.pone.0251691

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/supranational reference laboratory

Joel Kabugo ^1,^*, Joanita Namutebi ¹, Dennis Mujuni ^1,², Andrew Nsawotebba ^1,³, George William Kasule ¹, Kenneth Musisi ¹, Edgar Kigozi ², Abdunoor Nyombi ¹, Pius Lutaaya ¹, Fredrick Kangave ¹, Moses L Joloba ^1,²

Editor: Shampa Anupurba⁴

PMCID: PMC8121318 PMID: 33989348

Abstract

Background

Following the WHO’s endorsement of GeneXpert MTB/RIF assay for tuberculosis diagnosis in 2010, Uganda’s ministry of health introduced the assay in its laboratory network in 2012. However, assessing the quality of the result produced from this technique is one of its major implementation challenges. To bridge this gap, the National tuberculosis reference laboratory (NTRL) introduced the GeneXpert MTB/RIF proficiency testing (PT) Scheme in 2015.

Methods

A descriptive cross-sectional study on the GeneXpert PT scheme in Uganda was conducted between 2015 and 2018. Sets of panels each comprising four 1ml cryovial liquid samples were sent out to enrolled participants at preset testing periods. The laboratories’ testing accuracies were assessed by comparing their reported results to the expected and participants’ consensus results. Percentage scores were assigned and feedback reports were sent back to laboratories. Follow up of sites with unsatisfactory results was done through “on and off-site support”. Concurrently, standardization of standard operating procedures (SOPs) and practices to the requirements of the International Organization for Standardization (ISO) 17043:2010 was pursued.

Results

Participants gradually increased during the program from 56 in the pilot study to 148 in Round 4 (2018). Continual participation of a particular laboratory yielded an odd of 2.5 [95% confidence interval (CI), 1.22 to 4.34] times greater for achieving a score of above 80% with each new round it participated. The “on and off-site” support supervision documented improved performance of failing laboratories. Records of GeneXpert MTB/RIF PT were used to achieve accreditation to ISO 17043:2010 in 2018.

Conclusion

Continued participation in GeneXpert MTB/RIF PT improves testing accuracy of laboratories. Effective implementation of this scheme requires competent human resources, facility and equipment, functional quality management system, and adherence to ISO 17043:2010.

Introduction

The GeneXpert MTB/RIF assay since its recommendation for tuberculosis (TB) diagnosis in 2010 by WHO has been used worldwide. In Uganda, the assay was introduced in the laboratory network in 2012 to universalize access to TB services including notification, treatment, and prevention [1, 2]. The potential benefits of the assay concerning patient management are more noticeable at the lower healthcare facilities of a country’s healthcare system than at the national level and reference testing settings. This is because lower healthcare facilities are more closer to patients which reduces the turnaround time (TAT) to report Mycobacterium tuberculsosis (M.tb) diagnosis results. The challenge comes with verification of the quality of the results produced and since lower healthcare facilities such as health center three (HCIII) have less developed quality management systems, this may be compromised [3, 4].

The most effective and credible way to obtain accurate, reliable, and cost-effective results is through the implementation of a laboratory quality management system (LQMS) [5]. The healthcare system in developing countries has immeasurably recognized the impacts of LQMS when it comes to patient care and is gradually applying it [6]. It has indeed become necessary for all countries to strengthen the capacity of clinical laboratories and this can be achieved through setting up systems that monitor performance such as external quality assurance (EQA) [7, 8]. Recent studies in Africa show that the laboratory fraternity and test quality for all types of clinical laboratories remain poorly established [9, 10]. EQA in TB performing laboratories is implemented through; onsite support supervision, blinded rechecking, and proficiency testing (PT) [11].

With GeneXpert MTB/RIF test, blinded rechecking/retesting is not possible since the tested samples are discarded immediately after testing. This puts PT as the best-suited technique to assess the performance of laboratories with the method involving sending known samples to participants [8]. PT intends to assess all phases of diagnostic testing from pre-analytical, analytical and post-analytical. It involves periodically sending multiple well-characterized strains to participants for analysis. Each laboratory’s result is compared with the expected result and the consensus of other participants. Testing sites with discordant results and errors are identified, timely feedback is sent including recommendations for corrective action, and where necessary on-site supervisory visits are conducted to address nonconformities[12, 13].

National TB programs in most resource-limited countries especially in Sub-Saharan Africa have not been able to establish PT in the GeneXpert MTB/RIF assay [14]. This hardship is mainly attributed to many aspects which include: technical incompetence to prepare panels, lack of infrastructure, lack of human resource, high cost to procure PT preparation equipment, lack of suitable means of transporting panels to laboratories, supply chain management breakages, poor connectivity and Information Technology (IT) [4, 15, 16]. Implementation of EQA through organizing regular PT rounds and identification of training needs is one of the key aspects for efficient tuberculosis (TB) laboratory network [17].

According to a recommendation from the World Health Organization’s Regional Office for Africa (WHO AFRO), National public health reference laboratories have to develop and implement a quality management system (QMS), which encompasses the provision of PT services to all sites under the network [9, 18]. As a result, the Uganda national TB reference laboratory (NTRL) [19] which is also a World Health Organization (WHO) supranational TB reference laboratory (SNRL) designed a GeneXpert MTB/RIF PT Scheme to promote the quality of TB diagnosis when using the GeneXpert MTB/RIF platform. This study was conducted to demonstrate the lessons and attributes of scaling up the implementation of GeneXpert PT in Uganda.

Materials and methods

Uganda NTRL conducted a descriptive cross-sectional study from October 2015 to May 2018 among GeneXpert MTB/RIF testing laboratories. In this study, standard test samples were sent out to enrolled testing laboratories within and out of Uganda to assess the quality of Mycobacteria tuberculosis complex diagnosis using the GeneXpert MTB/RIF® platform.

An implementation road map was drawn and activities generated as shown in “Fig 1”.

GeneXpert MTB/RIF PT scheme concept inception and principle

The Uganda NTRL and a few of the peripheral laboratories it supports started the use of GeneXpert MTB/RIF® technology as a TB diagnostic method in 2012. The laboratory scientists at NTRL received prior technical training as GeneXpert MTB/RIF® super-users from Cepheid®. The team used this knowledge to develop a concept for preparing and producing GeneXpert MTB/RIF PT panels. The GeneXpert MTB/RIF assay cartridges and machines in the Uganda laboratory network were obtained from Cepheid® Inc with catalog number GXMTB/RIF-10. In 2014, the team used the known principle that the lowest detection level of GeneXpert MTB/RIF® technology was 100–1000 bacilli/ml in sputum [20] to conclude the required bacterial load concentration in the liquid panels. With this knowledge, McFarland 1.0 standard suspensions were prepared from Löwenstein Jensen (LJ) positive cultures and diluted to 5.0 x10⁴ CFU/ml that was used to prepare PT panels (liquid PT panels) after inactivation using the autoclaving technique.

Staff training, SOP development, and ISO/IEC 17043 standard alignment

Staff who participated in the concept development and experiment in 2014 validated the procedure at the Uganda NTRL/SNRL in 2015 becoming pioneers and initiators of GeneXpert PT preparation. These were responsible for training, assessing the competence of additional personnel, and developing standard operating procedures. The initial SOP for GeneXpert MTB/RIF PT panel preparation was developed in 2014 and later revised in 2016 and 2017 to conform with ISO 17043 accreditation requirements to ensure quality panels were continually produced. The revision involved a break down of the original single SOP document to separate working documents. These included SOP for proficiency testing plan, instructions to participants, PT preparation and testing, data analysis, packaging, labeling, and distribution. The implementation of ISO/IEC 17043 further took shape with performance of internal audits twice a year to assess adherence to the set standards. The non-conformities identified from these audits were subjected to root cause analysis and subsequent corrective actions were addressed within given a period. Adherence to the clauses of the ISO/IEC 17043 standard lead to the assignment of a coordinator of the GeneXpert PT scheme and creating a PT team. Strict timelines for PT preparation procedures and other processes were drawn that streamlined practice and improved the QMS as regards PT preparation. This process later lead to an application process for accreditation to ISO/IEC 17043 of the South African national accreditation system (SANAS).

PT panel production

Stock strains with known phenotypic and genotypic Polymerase Chain Reaction (PCR) sequencing results were used to produce the panels. A repository of these selected strains was kept at ultra-low freezer temperature (-80C). Negative strains were selected from Mycobacterium fortuitum which is a non-tubercle mycobacterium strain (NTM), internally prepared Phosphate Buffer solution (PBS), and sterile distilled water.

During the panel preparation phase, samples of three varying GeneXpert MTB RIF test results were incorporated. These included: (i) Negative strain; MTB Not Detected, (ii) Positive strain; MTB Detected: Rifampicin Sensitive and (iii) Positive strain; MTB Detected: Rifampicin Resistant.

After selection of these well-characterized Mycobacterium tuberculosis complex (MTBC) and Nontuberculous mycobacteria (NTM) strains, they were inoculated and grown on LJ culture media for 3–4 weeks to achieve the desired confluent growth. These biological samples used were not obtained through a medically prescribed test. They were got from an established bio-bank of the Institute of Tropical Medicine (ITM) WHO-Supranational Reference Laboratory Belgium (https://www.itg.be/E/administrator-of-the-worlds-largest-collection-of-tbc-strains) and each was accompanied with a conformity certificate as shown in “S1–S3 Texts” attached in the appendices.

The strains used were not specifically obtained for this study but also used for internal Quality controls during routine M.tb culture, phenotypic and genotypic drug susceptibility testing. Cultures were grown in quadruplicates and only those with colony growth (+3) according to Global Laboratory Initiative (GLI) guidelines were selected for use in producing strain dilutions [21, 22].

Strain suspensions of 1.0 Mcfarland standard were prepared from freshly grown cultures and heat-inactivated using an autoclave at 121pa for 30minutes to make the initial strain stocks. A volume of 0.5ml of the inactivated suspension was inoculated on Mycobacteria Growth Indicator Tube (MGIT) and Lowenstein-Jensen (LJ) culture medium. The inoculum was monitored for 42 days and 8 weeks on MGIT and LJ respectively to prove the success of inactivation and ensure the safety of the personnel handling and testing the panel. Five pretest runs per sample stock were made to generate preliminary results which were to be compared with the validation results after proof of inactivation run.

On completion of proof of inactivation, aliquots of 0.5 mL of 0.5Mcfarland standard were added to 9.5 ml of sterile water to obtain a bacterial concentration of 1.0 x 10⁶ bacilli/mL. Using the concentration of 1.0 x 10⁶ bacilli/mL, an aliquot of 0.5 mL was added to 9.5 mL of sterile water to obtain a concentration of 5.0 x 10⁴ bacilli/mL as the final dilution for the panels. This concentration when tested on GeneXpert MTB/RIF gives a medium-range quantification of M.tb per ml with a cycle threshold value between 16–22. Five aliquots of 1.0 mL of bacterial concentration for each strain were made into cryovials and validated on GeneXpert MTB/RIF assay to ascertain the quality and consistency with the intended results. All procedures above were conducted in a functional biosafety level III laboratory following developed SOPs and ISO/IEC 17043 standard.

Each panel was composed of four known liquid pretested and validated samples. The samples were completely de-identified from their original strain names to Q1, Q2, Q3, and Q4 labeling before validation and shipping to the participating laboratories. The Colony Forming Unit (CFU) count/ bacilli Load per sample was determined through panel pretest and validation using the GeneXpert MTB/RIF® assay. Although the amounts of detectable DNA varied from sample to sample, the panel composition was made from stocks that only yielded a cycle threshold (Ct) value of 16–22 (medium) during validation and pretesting.

Validation involved testing five-panel items from the stock of each sample that was chosen to be included in the panel. Inclusion of a sample in a panel was only after achieving 100% accuracy agreement among its validation test runs. Panels were freshly made for each round with uniqueness in composition i.e. the number of positive and negative samples, as well as the rifampicin sensitive and resistance, was distinctive for each new round.

Triple packaging was ensured by placing cryovial tubes into zip-lock bags that were wrapped in envelopes for sites within Uganda and transport boxes for those outside Uganda with adherence to the international air transportation association (IATA) regulations [23]. Each package was accompanied with the instructions to participants, confidentiality waiver, delivery acknowledgment, and result feedback form. The PT panels were sent as non-infectious shipments to the various laboratories. After shipment, a panel set was left at room temperature and tested after three weeks to check for stability and consistency. The three weeks were allocated on assumption that the latest panel sets would reach facilities after this period. This panel set was kept at ambient temperature and tested after three weeks from the date of dispatch. An IATA accredited courier company was sub-contracted for laboratories outside Uganda while to Ugandan laboratories, the ministry of health sample transport system was used. The above-described cascade of events was repeated for each round during the study.

A PT preparation and distribution schedule was established to streamline the scheme activities. Timelines were established for panel preparation, dispatch, and submission of feedback reports after closing dates. A period of 120 days (4 months) was allocated from panel preparation to dispatch of the panel sets to the testing sites, 42 days for laboratories to test and submit results back to NTRL, and 42 days to analyze data, prepare feedback, and round reports.

PT panels were sent out once in 2015 and 2016, twice in 2017 in February and August, and once in 2018. This schedule arrangement was put forward to give sufficient time for all the GeneXpert PT preparation and handling processes for each round conducted.

Ethical statement

This was a mandated quality assurance initiative by National tuberculosis and leprosy control program to the NTRL to ensure quality services for all laboratories performing TB diagnostic work in the network. Approval was sought from the National Health Laboratory and Diagnostics services (NHLDS) review board and permission to publish study findings obtained from NTRL management under approval number NTRL/OR-2018001. It is worth noting that all enrolled sites were required to complete a Confidentiality Waiver and Participant Biosafety Compliance Letter Agreement before participation.

Enrollment of participants in the PT scheme

This was dependent on the availability of a functional GeneXpert machine at the testing laboratories. An enrollment log was completed once annually with subsequent enrollment and recruitment of all laboratories housing functional machines within Uganda and outside Uganda. The enrollment was independent of the number of functional modules available for as long as a laboratory had at least a functional module. All laboratories without GeneXpert machines and those with all modules non-functional were excluded from enrollment in that particular round. All PT activities were centrally coordinated as shown in “Fig 2”.

PT pilot

The study started with a pilot that was conducted between October 2015 to May 2016. A sum of fifty-five testing laboratories was enrolled at this phase. These were selected from the different regions in Uganda. The selection was to give a comparison on how the different geographical dynamics and testing conditions would affect the performance of laboratories in GeneXpert PT.

Round 1 (2016) to Round 4 (2018)

After the pilot study, all the TB testing laboratories in Uganda were enrolled in the PT Scheme. The full enrollment was also a mandate of NTRL to provide quality assurance in the national laboratory network and the SRL-network. Enrollment forms sent out to participants captured their laboratory number and name, details of focal persons and location plus the status of QMS accreditation. The safety compliance details of all the laboratories enrolled were assessed during the process of GeneXpert machine installation to satisfy the minimum safety requirements to participate in the scheme.

For each new round of enrolment, the list of participants was updated. Both new and old participants were assessed for fitness to participant in the latest round. Testing was performed and results reported in the same format as in the pilot rounds. Scores were issued by NTRL in form of individual feedback reports for each site, detailing the breakdown of scores per tested sample. Individual performance feedback reports were sent through email and courier companies to the participants. Laboratories with unsatisfactory performances (defined as a panel score of less than 80%) were targeted for onsite supervision and followed up to do corrective actions.

Results compilation and data analysis

Results obtained by participants from GeneXpert machines were manually entered into the reporting form that was sent along with the panels. The reporting form was then sent back through courier companies or electronically emailed to Uganda NTRL for analysis. Results were compiled and manually entered into controlled Microsoft excel version 2010 spreadsheets after submission at NTRL. The “IF command”, “summation” and “Average” functions in excel were applied to award scores for each item tested by an individual participant and average testing turnaround time. This was also applied to calculate the final score. To evaluate trends of performance, graph charts were obtained using extracted summary data exported into a new excel spreadsheet. Laboratory performance was assigned an accuracy score expressed as a percentage. Each of the four samples correctly tested accounted for 2 points, errors/invalids/no result machine readings were scored 1 point, and zero points were given for an incorrect result. The performance threshold for a satisfactory score was set at 7 out of 8 (7/8 points) which translated to a site correctly testing three samples and only scoring an error/invalid/no result for one of the four samples. The total points obtained were converted into a percentage that had to appear on the feedback reports shared with the participants.

To statistically understand the interaction of continuous participation of an individual laboratory and its performance in PT over rounds, we imported the data into SPSS version.24 and applied Ordinal logistic regression to determine the proportional odds. We hypothesized that PT performance would improve with continued participation. The association between PT performance (unsatisfactory or satisfactory) and the number of times a laboratory had participated was examined using the fixed- and mixed-effect proportional odds model [24]. GeneXpert MTB/RIF testing laboratories represented sampling clusters hence no random effects on the results obtained with conclusion done based on the fixed-effect model.

Results and discussion

Laboratory enrollment and participation

Five PT rounds were sent out between August 2015 and May 2018 as part of the GeneXpert MTB/RIF PT program of Uganda NTRL. A gradually increasing number of laboratories were enrolled from 55 sites in the pilot to 148 sites in Round 4(2018). In all the rounds, laboratories tested the panels using the GeneXpert MTB/RIF® assay.

The set target for result submission was 80% of all participants enrolled per round. Overall, this was achieved with the lowest submission at 81.3% (104/127) in Round 3(2017).

In total, 18 sites reported being non-functional throughout the rounds with the highest number 5(4.2%) reported in Round 2(2017). The number of Laboratories not reporting after receiving PT panels varied from round to round, with Round 3(2017) having the highest no response rate of 20 (15.7%). The summary of participation is as shown in “Table 1” below.

Table 1. Participation profile for all the rounds sent out.

Characteristics	Pilot study	Round 1(2016)	Round 2 (2017)	Round 3 (2017)	Round 4 (2018)
Laboratories enrolled	55	110	121	127	148
Return of results
Total results returned n (%)	51(93%)	96 (87.3%)	105(86.7%)	104 (81.9%)	134(90.5%)
No returns n (%)	1(2%)	10(9.1%)	11(9.1%)	20(15.7%)	11(7.4%)
Non-functional machines	3(5%)	4(3.6%)	5(4.2%)	3(2.4%)	3(2.1%)

Open in a new tab

Performance of the participants

Laboratories achieving satisfactory and unsatisfactory scores were individually determined for each round. In the entire study, 490 results were generated with 93.5% (458/490) as satisfactory scores having the highest satisfactory performers 28.1% (129/459) in round 4 (2018). Round 1(2016) saw the greatest proportion of unsatisfactory scores 37.5% (12/32) for all the rounds sent out. The majority of the participating laboratories failing at their first time of enrollment improved their performance in their subsequent participation while those initially scoring above 80% continued to match their original score. The lowest number of errors/invalids/no results was reported in Round 4 (2018) and highest in Round 1 (2016). Table 2 below summarizes the performance across all the rounds.

Table 2. Performance of participating sites throughout the different rounds.

Performance	Pilot Study	Round1(2016)	Round 2 (2017)	Round 3 (2017)	Round 4 (2018)
Labs with Satisfactory score (≥80%), n(%)	48(94%)	84(87.5%)	98(93.33)	99(95.2%)	129(96.3%)
Labs with Unsatisfactory score (≤80%), n(%)	3(6%)	12(12.5%)	07(6.67%)	5(4.8%)	5(3.7%)
Errors/invalid/no result (n)	00	14	06	11	02
Discordant results	03	06	05	10	07

Open in a new tab

The median score improvement for a laboratory’s first-time participation to its latest round of participation was 12 percentage for all laboratories, from 88% to 100%. On excluding laboratories that scored 100% at their first time of participation, the average score for the remaining facilities improved by 23%, from an average of 75% at the first time of participation to 98% in Round 4(2018).

An increasing sum of laboratories obtaining satisfactory scores increased with more times of participation as shown in Table 3 below.

Table 3. Number of laboratories achieving satisfactory scores and their different participation times.

Time(s) of participation	Pilot study	Round 1 (2016)	Round 2 (2017)	Round 3 (2017)	Round 4 (2018)
First	48	36	7	5	23
Second		48	40	6	07
Third			51	40	6
Fourth				48	43
Fifth					50

Open in a new tab

Table 3 above shows laboratories that obtained satisfactory results during their proceeding times of participation. In the pilot study, 48/55 participants scored satisfactorily and this was taken as the baseline for all the laboratories that had been enrolled. In round 1(2016), 84/96 laboratories scored above 80% with 36 sites being their first time of participation and 48 their second time. In round 2 (2017), 98/105 laboratories scored satisfactorily with 51 these participating for the third time, 40 for the second time and 7 for the first time.

In round 3 (2017), of the 99 laboratories with satisfactory results 48 had participated consistently four times, 40 sites for three times, 6 two times and 5 laboratories at their first enrollment. Of the total 129/134 laboratories that had satisfactory results in round 4(2018), 50 were participating for the fifth time, 43 at their fourth time, 6 third time, 7 second time, and 23 were first-timers.

During these rounds of participation some sites didn’t have satisfactory results at the first time of participation and later improved in the subsequent round increasing the number of satisfactory scores with each new round of enrollment which can also be noted in Fig 3 below.

Evaluation of the times of participation and unsatisfactory performance

A total of 32 unsatisfactory scores was generated from 24 laboratories throughout all the rounds. The main cause of unsatisfactory scoring was discordant result reporting at 68% (22/32) with the rest due to a laboratory reporting more than one error/invalids/no results. The error results reported were due to error codes 5006, 5007, 2008, and 2011. The percentage of sites achieving unsatisfactory scores generally decreased per round from 12.5% in round 1 (2016) to 3.7% in round 4 (2018) as shown in Fig 3 above. The rate of scoring unsatisfactorily varied over times of participation as shown in Table 4 below.

Table 4. Laboratories scoring unsatisfactorily and their times of participation into the PT scheme.

Times of participation	Pilot study	Round 1 (2016)	Round 2 (2017)	Round 3 (2017)	Round 4 (2018)
First	3	9	2	2	1
Second		3	5	1	0
Third			0	2	3
Fourth				0	0
Fifth					1

Open in a new tab

Seventeen laboratories failed at their first time of participation, 9 at their second round of participation, and 5 at their third time from the pilot to round 4 (2018). For the fourth time of participation in round 3 (2017) and round 4(2018), no laboratory scored unsatisfactory results. At the fifth time of participation, only one laboratory failed among those that had been enrolled since the pilot study.

Of the three laboratories that scored unsatisfactory in the pilot study, one of these continued to score below 80% in round 1 (2016) and thereafter scored satisfactorily throughout the next rounds. In round 1 (2016), 12 laboratories scored unsatisfactorily with 9 of these participating for the first time and 3 for the second time. It is notable that 4/12 sites continued to score unsatisfactory in round 2 (2017) with one out of these four repeatedly failing in round 3 (2017) and thereafter all obtained satisfactory scores in round 4 (2018).

In round 2 (2017), 7 laboratories scored unsatisfactorily of which 2 were participating for the first time and 5 second time. All laboratories participating for the third time scored satisfactorily. In round 3 (2017), 5 laboratories scored unsatisfactorily with 2 of these being their first time of participation, one second time, and 2 for the third time. All laboratories that had failed in round 3 (2017) got satisfactory scores in round 4(2018).

In round 4(2018) that had 5 participants with unsatisfactory results, one of these was at their first time of enrollment, 3 were participating for their third consecutive time and one was at their fifth time of participation. The laboratory failing at its fifth time had been scoring satisfactorily in all the previous rounds it was enrolled.

In summary, only one laboratory consistently failed thrice, six sites twice and seventeen failed once. Of the seventeen laboratories that scored unsatisfactory once throughout their participation, ten of these scored at their first time of participation, three at their second consistent time of participation, three at their third participation time, and only one at its fifth participation round.

The odds of achieving scores of above 80% were estimated to be 2.51 [95% confidence interval (CI), 1.78 to 4.34] times greater with each new consistent round of participation. From this, we can say that a laboratory’s performance in PT has higher chances of improving the more it participates. Our proportional odds regression model predicts that 91% of sites should be expected to achieve satisfactory scores (≥80%) after two rounds of prior experience with GeneXpert PT participation, and all should be expected to achieve 100% scores after four rounds of prior experience as also shown in Fig 4 below.

Discordant results

The highest recorded discordant result was the false-negative M. tuberculosis 35.4% (11/31) and the least being false rifampicin resistance at 19.4% (6/31) reported in all the rounds. There was no observed pattern between the number of errors/invalids/no results and the PT rounds.

From Table 5, 31 discordant results were recorded by 22 laboratories. One discordant result per laboratory was reported in rounds pilot study, round 1 (2016) and round 2 (2017) among the discordant performers. Round 3 (2017) registered two laboratories with 3 discordant results each with the rest being reported by one laboratory. Round 4 (2018) saw two laboratories reporting two discordant results each and the remaining three getting individually reported by three participants.

Table 5. Discordant results reported from the different rounds.

discordant results	Pilot study	Round 1(2016)	Round 2 (2017)	Round 3 (2017)	Round 4 (2018)	Total
Laboratories reporting discordant results	3	6	5	3	5	22
false M.tb negative	1	2	1	3	4	11
false M.tb detected	0	2	2	2	1	7
false Rifampicin sensitive	0	0	1	4	2	7
false Rifampicin resistant	2	2	1	1	0	6

Open in a new tab

Turnaround time (TAT)

From the 490 results from all the rounds, 78.8% (386/490) were received at NTRL within the set TAT of 42 days. The highest number of sites that submitted results within TAT was in round 4(2018) at 33.4% (129/386) just after round 3(2017) that registered the highest number of laboratories submitting results after established TAT 10.6% (11/104).

The average time to submit results by participants showed an improving trend from 30 days in the pilot to 18.5days in Round 4(2018) as shown in Fig 5. The study had its longest time taken to receive samples being 14 days with the highest contributors of this fate residing in the north-eastern part of Uganda. This can be accounted on for the long-distance these sites scale from NTRL. The time taken to receive samples in this study showed no correlation with obtaining discordant results or scoring unsatisfactorily.

On and off-site Support Follow up (OSF) and corrective actions

Result reports were reviewed by competent personnel in the GeneXpert PT team before their dispatch to the respective participants. It is at this phase where laboratories that had failed (scored unsatisfactorily) were assigned to individuals technical people at NTRL for follow-up. During annual supervision, NTRL personnel were able to follow up 23 laboratories onsite and support them in conducting root cause analysis and documenting corrective actions. The rest of the laboratories not visited due to logistic limitations were contacted and urged to submit their root cause analyses and corrective actions. Of the sites supervised, 17(90%) managed to score satisfactorily in their next round of participation signifying the positive impact of onsite follow-ups. Root cause analysis and corrective actions were not done for three (3) laboratories and these were among those not offered onsite follow-up. The mentioned three (3) above continued to score below 80% for the next round they participated in. The following issues as shown in Table 6 were listed by 29 laboratories to be the causes for their failure.

Table 6. Frequencies of issues identified during root cause analysis for laboratories with unsatisfactory scores.

Issue identified	Frequency
Clerical reporting	24.1% (7/29)
Poor expert machine maintenance	10.3% (3/29)
Overdue instrument calibration	13.8% (4/29)
Power blackout and lack of power backup	20.7% (6/29)
Lack of air conditioners	10.3% (3/29)
Lack of staff training and orientation of new staff	6.9% (2/29)
Sample pouring	3.4% (1/29)
GeneXpert cartridge stock out	3.4% (1/29)
Lack of a printer	3.4% (1/29)
Switching of samples	3.4% (1/29)

Open in a new tab

Among the Issues identified during OSF and root cause analysis, clerical errors ranked highest for the unsatisfactory scores. This matter was cited among laboratories reporting false M.tb detection, false rifampicin sensitive, false rifampicin-resistant, and false-negative results. Submission of corrective actions from laboratories was challenging and a majority of sites had to be followed up over time to submit.

ISO/IEC 17043 implementation

Documents and all records which included participants’ results, preparation worksheets, SOPs, and personnel files generated during the rounds sent out, were used to apply for ISO 17043 accreditation. The application was successful and the scheme was accredited in 2018 SANAS. The adherence to this standard streamlined the follow of activities and promoted the quality of the PT program.

Discussion

The introduction of GeneXpert MTB/RIF® technologies intensified case finding for TB. This is as a result of its improved sensitivity and turnaround time mainly in areas that have for long relied on sputum smear microscopy as the primary TB diagnostic [25]. By 2018 Uganda had a total of over 148 GeneXpert sites with 100% enrollment into the NTRL PT scheme. This number was supplemented by facilities outside Uganda that happen to be supported as a WHO Supra-national Reference Laboratory. The increase in the number of PT enrolled sites every year was because of the installation of more GeneXpert machines in Uganda and more laboratories outside Uganda applying for ISO 15189 accreditation that required them to be participating in a PT scheme.

The satisfactory performance of sites can be attributed to the competency at testing sites, the positive role of the QMS being implemented at several sites [26], proper functionality of the machines that were regularly serviced and well maintained daily.

The causes of unsatisfactory scores are several and these vary from one laboratory to another. These were related to human error, equipment maintenance, staff training and competency, environment and procedures being used. To a less extent, a false result could be attributed to the PT preparation process regardless of the rigorous quality control procedures undertaken as highlighted by other PT studies [27] and not the case in this one. Therefore, to unearth the possible cause of performance failure in a PT, one has to holistically do unbiased root cause analysis and identify the only possible cause if it is not so obvious.

The commonest errors reported in this study were 5006, 5007, 2008, and 2011 that are due to Probe Check failures caused by high sample viscosity and/or low volumes and communication loss between the GeneXpert machine and the computer. These were followed by invalid results which occur due to Internal Control failure instigated by several factors such as improper kit and sample storage and inadequate sample volume [28].

The majority discordant result (false negative) M. tuberculosis complex results can be caused by PT samples having small amounts of detectable M. tuberculosis DNA. This is resultant from sample degradation caused by the addition of insufficient PT sample volume to test cartridges or other procedural errors such as improper sample storage and cross-contamination. Other reasons can be clerical reporting on the result form and switching of samples during testing. These same aspects in this study were also reported in the article “Development, evaluation, and implementation of a new rapid molecular diagnostic for tuberculosis and rifampicin resistance” [29]. This highlights the picture of the possible errors in the analytical cascade for TB clinical specimen vis-à-vi GeneXpert MTB/RIF® assay utilization. The major cause of all false results was clerical reporting caused by manual transcription of results to the provided reporting template. To overcome this challenge, PT participants should employ and strengthen their result review process before reporting to the PT provider. This system should directly be applied to the routine clinical samples to mitigate transcription errors. The performance of laboratories within and outside Uganda had no significant difference and scores were seen not to vary at all in regards to the location of the laboratory.

The Turnaround time (TAT) in PT can be reasonably related to the TAT taken to report patient results. Timely feedback in EQA is crucial to the program’s effectiveness. Delayed result submission not only obstructs the timely implementation of essential corrective actions but also prolongs poor testing accuracy, thereby influencing the quality of patient care and EQA relevance in the end. The mechanism of PT delivery and result sending in this study was found to highly affect the time taken to test and report results.

The main courier company used at the time of the study was not highly effective in reaching every laboratory as quickly as possible causing delays in shipment and testing of samples. The great improvement in round 1 (2018) TAT was due to improved adherence to instructions sent to the participants and increased use of emails to submit results compared to the earlier rounds.

The low response rate and many sites reporting after set TAT in round 3(2017) were as a result of NTRL transitioning from one courier company to another that the participants had not yet fully got accustomed to. The delays were seen more in the Ugandan laboratory network where PT materials never got to reach their intended destinations within ten working days that were always allocated for dispatch and receipt of PT materials. This, therefore, implies the need for a robust but flexible transport system that reaches up to final laboratories directly from the PT provider. The other solution is enrolling out an online result submission system that bypasses the need to submit results through courier. In this study, the emailing system was employed more in the last round to help in receiving feedback results in addition to the courier. This contributed to the improvement in response rate.

Activities performed during on-site supervision ranged from refresher pieces of training and competence assessment of personnel, review of laboratory registers, EQA files, GeneXpert machine maintenance logs, and troubleshooting for errors/invalids/no results generated. The Authorized GeneXpert Servicing contractor by Cepheid in Uganda handled all the responses and matters that were beyond the technical knowledge of NTRL personnel such as module replacement, calibration, and software upgrades. All these actions need to be routinely done to ensure consistency of quality results in a laboratory network. The majority of the problems listed under root cause analysis were instantly fixed. Suggestion and actions taken included the development of temperature monitoring and Xpert machine maintenance logs for sites without them. Laboratory managers were assigned to compile maintenance logs monthly, schedule timely Xpert instrument calibration every year, and improve supply chain management for the supply of cartridges to fix the challenges raised. Actions that required more funding included the installation of power backups such as solar panels and UPS, replacement of failed modules, installation of air conditioners, and procurement of result printers. These were forwarded to the National TB and Leprosy Program(NTLP) to be fixed. Fixing of actions that required many funds was never achieved in short times and affected the performance and routine effectiveness of the laboratories.

To further improve the GeneXpert PT program, some technical personnel were trained by the Centers for Disease Control (CDC), Atlanta in July 2017 to enroll Dry Tube Specimen (DTS) that has better performance characteristics as compared to the liquid panel that was provided in these rounds.

The establishment of a quality management system based on ISO 15189 for medical Laboratories and the 17043 PT program and having their accreditation is fundamental in sustaining a PT program and having a quality output.

Conclusion

The results of this implementation study reveal the gaps and roadmap needed to establish a GeneXpert PT scheme in countries with similar settings and bottlenecks. The results clearly show the positive impact of continued PT participation and how it improves testing accuracy and consistency at the TB testing laboratories. The study also shows that there is a great value of following up sites with unsatisfactory performance as this enables an efficient fix and gives a long-term solution to occurrences at the laboratories with issues regarding their quality management system, with improvement in their performance. The findings highlight the feasibility of implementing a GeneXpert PT Scheme, and the need to reduce the turnaround time for all the steps involved in the same. The lessons learned may be helpful for other countries to replicate successfully in their settings.

Supporting information

S1 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(364.1KB, pdf)}

S2 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(374.1KB, pdf)}

S3 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(527.4KB, pdf)}

S1 File

(PDF)

Click here for additional data file.^{(554.8KB, pdf)}

S2 File

(PDF)

Click here for additional data file.^{(215.9KB, pdf)}

S3 File

(PDF)

Click here for additional data file.^{(452KB, pdf)}

S4 File

(PDF)

Click here for additional data file.^{(549.5KB, pdf)}

S5 File

(PDF)

Click here for additional data file.^{(259.4KB, pdf)}

Acknowledgments

We thank the study participants for their time, interest, and willingness to and the staff of the National TB Reference Laboratory of Uganda who we applaud for the technical and administrative work. We acknowledge the training and advisory support from and Regional Global fund through the East, Central, and Southern Africa (ECSA) Health Community Project.

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

The author(s) received no special funding for this work. The financial support to run all the above activities was part of the routine funding through the Regional Global fund through the East, Central and Southern Africa (ECSA) Health Community Project to the Supra-National Reference Laboratory of Uganda to support Tuberculosis (TB) diagnostic implementation and quality management systems in the Laboratories performing TB testing.

References

1.Scott C. et al. , “Evaluation of automated molecular testing rollout for tuberculosis diagnosis using routinely collected surveillance data—Uganda, 2012–2015,” Morbidity and Mortality Weekly Report, vol. 66, no. 12. Department of Health and Human Services, pp. 339–342, 31-March-2017. 10.15585/mmwr.mm6612a6 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.World Health Organization (2013)., Global tuberculosis report 2013. World Health Organization. https://apps.who.int/iris/handle/10665/91355. [Google Scholar]
3.Denkinger C. M., Nicolau I., Ramsay A., Chedore P., and Pai M., “Are peripheral microscopy centres ready for next generation molecular tuberculosis diagnostics?,” Eur. Respir. J., vol. 42, no. 2, pp. 544 LP– 547, August. 2013. 10.1183/09031936.00081113 [DOI] [PubMed] [Google Scholar]
4.Creswell J. et al. , “Results from early programmatic implementation of Xpert MTB/RIF testing in nine countries,” BMC Infect. Dis., vol. 14, no. 1, p. 2, December. 2014. 10.1186/1471-2334-14-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Audu R. A. et al. , “Proficiency testing for HIV, tuberculosis and malaria diagnosis in clinical laboratories in Nigeria,” Afr. J. Lab. Med., vol. 3, no. 1, 2014. 10.4102/ajlm.v3i1.102 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Barbé B., Yansouni C. P., Affolabi D., and Jacobs J., “Implementation of quality management for clinical bacteriology in low-resource settings,” Clinical Microbiology and Infection, vol. 23, no. 7. Elsevier B.V., pp. 426–433, 01-July-2017. 10.1016/j.cmi.2017.05.007 [DOI] [PubMed] [Google Scholar]
7.Libeer J. C., “External quality assurance programmes in medical laboratories,” Accredit. Qual. Assur., vol. 6, no. 4–5, pp. 151–153, 2001. 10.1016/s0009-8981(01)00518-6 [DOI] [PubMed] [Google Scholar]
8.“Brookman Brian, et al. Selection, Use and Interpretation of PT Schemes: 2nd edition 2011.”. [Google Scholar]
9.Gershy-Damet G. M. et al. , “The world health organization African region laboratory accreditation process: Improving the quality of laboratory systems in the African region,” American Journal of Clinical Pathology, vol. 134, no. 3. pp. 393–400, September-2010. 10.1309/AJCPTUUC2V1WJQBM [DOI] [PubMed] [Google Scholar]
10.Palamountain K. M. et al. , “Perspectives on Introduction and Implementation of New Point-of-Care Diagnostic Tests,” J. Infect. Dis., vol. 205, no. suppl_2, pp. S181–S190, May 2012. 10.1093/infdis/jis203 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.“Gumma V, Bond K, DeGruy K, Alexander H, Nhung NV, Hung NV, et al. 2016. Successful implementation of external quality assessment for GeneXpert and its impact on quality of testing in Vietnam, p 26. Abstr 3rd Int Conf Afr.”. [Google Scholar]
12.Parekh B. S. et al. , “Dried tube specimens: A simple and cost-effective method for preparation of HIV proficiency testing panels and quality control materials for use in resource-limited settings,” J. Virol. Methods, vol. 163, no. 2, pp. 295–300, February. 2010. 10.1016/j.jviromet.2009.10.013 [DOI] [PubMed] [Google Scholar]
13.Garcia A. et al. , “Impact of proficiency testing program for laboratories conducting Early Diagnosis of HIV-1 infection in infants in low- to middle-income countries,” J. Clin. Microbiol., vol. 52, no. 3, pp. 773–780, 2014. 10.1128/JCM.03097-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Scott L., Albert H., Gilpin C., Alexander H., DeGruy K., and Stevens W., “Multicenter feasibility study to assess external quality assessment panels for Xpert MTB/RIF assay in South Africa,” J. Clin. Microbiol., vol. 52, no. 7, pp. 2493–2499, July. 2014. 10.1128/JCM.03533-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Miller W. G., Jones G. R. D., Horowitz G. L., and Weykamp C., “Proficiency testing/external quality assessment: Current challenges and future directions,” Clinical Chemistry, vol. 57, no. 12. pp. 1670–1680, December-2011. 10.1373/clinchem.2011.168641 [DOI] [PubMed] [Google Scholar]
16.Albert H., Nathavitharana R. R., Isaacs C., Pai M., Denkinger C. M., and Boehme C. C., “Development, roll-out and impact of Xpert MTB/RIF for tuberculosis: what lessons have we learnt and how can we do better?,” Eur. Respir. J., vol. 48, no. 2, pp. 516–525, August. 2016. 10.1183/13993003.00543-2016 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Nikolayevskyy V. et al. , “External Quality Assessment for Tuberculosis Diagnosis and Drug Resistance in the European Union: A Five Year Multicentre Implementation Study,” PLoS One, vol. 11, no. 4, p. e0152926, April. 2016. 10.1371/journal.pone.0152926 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.“Quick Guide to Quality Assurance of Xpert MTB/RIF Testing,” 2019. [Google Scholar]
19.“SUPRANATIONAL REFERENCE LABORATORY–Timely Accurate Diagnostics for a TB Free Africa.” https://ntrl.or.ug/.
20.Helb D. et al. , “Rapid detection of Mycobacterium tuberculosis and rifampin resistance by use of on-demand, near-patient technology,” J. Clin. Microbiol., vol. 48, no. 1, pp. 229–237, January. 2010. 10.1128/JCM.01463-09 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.“global laboratory initiative advancing TB diagnosis A publication of the Global Laboratory Initiative a Working Group of the Stop TB Partnership Mycobacteriology Laboratory Manual.”
22.“Stop TB Partnership—Global Laboratory Initiative (GLI).” Available: http://www.stoptb.org/wg/gli/gat.asp.
23.“IATA—Infectious Substances Shipping Guidelines.” Available: https://www.iata.org/en/publications/store/infectious-substances-shipping-guidelines/.
24.Agresti A L. J., “Analysis of Ordinal Categorical Data—Alan Agresti—Google Books,” Biometrika, vol. 80, pp. 527–534, 1993. [Google Scholar]
25.WHO, “WHO monitoring of Xpert MTB/RIF roll-out,” WHO, 2017. [Google Scholar]
26.Manickam T. S. and Ankanagari S., “Evaluation of quality management systems implementation in medical diagnostic laboratories benchmarked for accreditation,” vol. 6, no. 5, pp. 27–35, 2015. [Google Scholar]
27.Gumma V. et al. , “Impact of external quality assurance on the quality of Xpert MTB/RIF testing in Viet Nam,” J. Clin. Microbiol., vol. 57, no. 3, March. 2019. 10.1128/JCM.01669-18 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Hanrahan C. F. et al. , “The patient impact of point-of-care vs. Laboratory placement of XpertW MTB/RIF,” Int. J. Tuberc. Lung Dis., vol. 19, no. 7, pp. 811–816, July. 2015. 10.5588/ijtld.15.0013 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Lawn S. D. and Nicol M. P., “Xpert® MTB/RIF assay: Development, evaluation and implementation of a new rapid molecular diagnostic for tuberculosis and rifampicin resistance,” Future Microbiology, vol. 6, no. 9. Europe PMC Funders, pp. 1067–1082, September-2011. 10.2217/fmb.11.84 [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0251691.r001

Decision Letter 0

Pradeep Kumar

1 Sep 2020

PONE-D-20-20339

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

PLOS ONE

Dear Kabugo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. In addition to reviwers comments, it is noted that the text has similarity to atleast one other study PMID : 30567748. So, the text must also be revised for oroginality.

Please ensure that your decision is justified on PLOS ONE’s publication criteria and not, for example, on novelty or perceived impact.

==============================

Please submit your revised manuscript by November 28, 2020. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Pradeep Kumar, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please upload a copy of Figures 1-5, to which you refer in your text. If the figures are no longer to be included as part of the submission please remove all references to them within the text.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: • What is the reason behind author’s choice for using autoclaving as an inactivation method when SR reagent is a proven method of inactivation for Mtb? Did the authors conduct any studies to verify that autoclaving does not impact the GeneXpert results?

• Further, authors note that samples are stored at 2-8C for 42 days until results from MGIT are verified for inactivation. Does this long storage of samples have any impact on the results? Were any experiments conducted?

• It is noted in the supplementary section that samples are shipped at ambient temperature. Were overnight shipment methods used? What was the longest it took for samples to arrive at a designated laboratory? Was there any correlation with long transit times and discordant/invalid results? In other words, did long transit times lead to sample degradation and hence invalid results?

• Authors are requested to provide an original reference for lines 110-111 for the detection limit of GeneXpert MTB/RIF test.

• Under supporting information, authors listed figures 1 to 5 as S1 Fig 1, S2 Fig 2 etc., Please clarify which figures belong in the main manuscript and supplementary sections respectively.

• There is no information about the process behind obtaining ISO accreditation. Some detail about what changes were made to the SOPS to align with ISO’s requirements and how this aided in study’s success supporting lines 438-439 would be helpful.

• Authors note that the major cause of false results is clerical errors. Did the authors consider using automated GeneXpert test reports instead of manual reporting?

• Were the panel of strains freshly prepared during each around? Please clarify in methods section.

• How are the results reported when a test reports no MTB detected but RIF sensitivity/resistance is detected?

Minor comments:

• Authors are requested to expand the acronyms the first time they appear in the manuscript.

For example, PT in line 25, EQA in line 61, QMS in line 85 etc.,

• Similarly, authors use proficiency testing and PT interchangeably throughout the manuscript. For consistency authors should either stick to the acronym or the expanded version not both.

• Authors are advised to thoroughly revise the manuscript for grammatical errors and sentence formation for clarity in several areas.

• In lines 351-353, machine calibration is mentioned twice as one of the issues.

• Please rephrase lines 27-28 of abstract to “conducted between years 2015 to 2018” for clarity

Reviewer #2: Overall

This paper reports data from a cross-sectional study assessing the testing accuracy of the GeneXpert MTB/RIF assay in laboratories linked to the Uganda National Tuberculosis Reference Laboratory and assessing whether a program of GeneXpert proficiency testing in Uganda led to improved testing accuracy among participating laboratories. As the authors well describe in the introduction, national TB programs in resource-limited countries, especially in sub-Saharan Africa, have not generally established proficiency testing for GeneXpert MTB/RIF. This is a novel and interesting piece of work because it shows that setting up a persistent country-wide proficiency testing program for GeneXpert can lead to increased testing accuracy through root cause analysis and assistance and advice from a reference laboratory. However, a number of important inconsistencies and errors in the results section need to be corrected before the manuscript is suitable for publication, and I have concerns about the 80% threshold used to judge a laboratory as satisfactory. The manuscript would also benefit from a more in-depth discussion of the headline result that average laboratory scores increased the more rounds of the testing program they participating in. The article is intelligible but would nevertheless benefit from independent editorial assistance to improve the style and occasional grammatical errors. Below are my numbered comments and recommendations:

Introduction

1. Line 61: EQA should be defined as it is its first use.

2. Overall, the introduction explains the background to the study well, including the Uganda-specific situation and gives a good definition of proficiency testing.

Methods

I am unable to comment on the methods of the PT panel preparation as this is outside my area of expertise.

3. Line 242-4: I don’t understand why the scoring threshold in set at 80%, or even expressed as a percentage at all, because it gives the impression of greater accuracy than is truly available from your testing system, in which each laboratory tests 4 samples. 6 out of 8 points gives 75% (unsatisfactory) and 7 out of 8 gives 87.5% (satisfactory). So to be judged satisfactory, laboratories have to correctly test all 4 samples (100%), or correctly test 3 samples and have one error/invalid/no result (87.5%), but will automatically be judged unsatisfactory if they give one of more incorrect results. This system is fine, but it would be better would be to express the threshold as 7 out of 8 points for clarity.

4. Currently the names of the study rounds are very confusing and difficult to keep track of, with round 2 followed by round 1 followed by round 2 followed by round 1. Much better would be to make the numbering sequential, along the lines of: Pilot study, Round 1 (2016), Round 2 (2017), Round 3 (2017), Round 4 (2018). Or alternatively: Pilot study, 2016, 2017a, 2017b, 2018.

5. I have no ethical concerns, or concerns about the statistical methods.

Results

6. There are significant discrepancies between the numbers in Table 1 and Table 2. For the pilot study, Table 1 says that 48 results were returned but Table 2 has 48+3=51 results. Round 2 2016 and Round 1 2017 have consistent numbers between the two tables, but Round 2 2017 has 104 results returned in Table 1 but 100+5= 105 results in Table 2 and Round 1 2018 has 134 results returned in Table 1 but 115+4=119 results in Table 2. So 3 out of 5 are inconsistent.

7. Additionally, Line 288 says: “There were a total of 19 unsatisfactory results form 15 laboratories”. However, according to Table 2, there were 3+12+7+5+4=31 unsatisfactory scores. And then line 347-8 says: “3 out of the total 19 sites with unsatisfactory scores…”

8. Lines 308-310: These results are not in fact shown in Table 2 and Figure 3. Table 2 shows 29 discordant results, not 23. Presumably, those additional 6 discordant results came from laboratories which had more than one discordant result. Table 2 would be more useful if updated to show the number of laboratories with at least one discordant result in each round, rather than the total number of discordant results, possibly with the breakdown of false positives/negatives/resistant/sensitive below.

9. Lines 305-7: These lines are not results and should be moved to the discussion section.

10. Line 337-9: These lines are not results and should be moved to the discussion section.

11. Lines 275-6, 292-4, 298-300 and 310-11: These points are very interesting, and the odds ratio of 2.5 is one of the major results you report in the abstract, so I feel they deserve a graph or table of their own, showing the average scores (or number of satisfactory and unsatisfactory results) of laboratories in the first round they engaged in, and the second round, third round and fourth round for those which engaged in multiple rounds. One purpose of the study was to see whether continual participation in the proficiency testing program would improve the quality of TB testing results, which it appears to, and this graph or table would present this data in the clearest way.

12. Line 350-360: This list of issues could be made more useful if the frequency with which each issue arose was displayed.

13. It would be useful to see results broken down by whether the laboratory was inside or outside Uganda. I can see potential for the impact of the proficiency testing to be different for laboratories outside Uganda. Or if there was no significant difference, add a comment to say there was no difference.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 May 14;16(5):e0251691. doi: 10.1371/journal.pone.0251691.r002

Author response to Decision Letter 0

13 Dec 2020

I extend my appreciation to the precious time and dedication given to my manuscript. The reviews and comments raised have been taken positively and as authors of this manuscript, we would like to respond to them as below;

Review Comment: It is noted that the text has similarity to atleast one other study PMID : 30567748. So, the text must also be revised for originality

Response: The manuscript has been exclusively revised to exhibit originality of research and bring out the innovation of using liquid panels for GeneXpert proficiency Testing panels with clear distinction from the research work presented in study PMID : 30567748

Response to Reviewer 1

1. What is the reason behind author’s choice for using autoclaving as an inactivation method when SR reagent is a proven method of inactivation for Mtb? Did the authors conduct any studies to verify that autoclaving does not impact the GeneXpert results?

Yes, we conducted a preliminary experiment by inactivating eight samples using either method and inoculating on both LJ culture and MGIT culture after inactivation. This was done at the initiation of this study. The preliminary results obtained from inactivation of samples for GeneXpert proficiency testing, showed that the method of using sample reagent (SR) buffer to inactivate samples yielded less effectiveness than the autoclaving method. Proof of inactivation results gave two sample strains inactivated using sample reagent (SR) showing growth on both MGIT method and LJ culture method at the 5th week of culturing while all those inactivated using autoclaving method showed no growth.

2. Further, authors note that samples are stored at 2-8C for 42 days until results from MGIT are verified for inactivation. Does this long storage of samples have any impact on the results? Were any experiments conducted?

Yes, experiments were done to ensure stability and consistency. For every round prepared, pretest of five runs on inactivated sample was done before storage at 2-8°C and after the forty two days a validation test was done to prove consistency of M.tb DNA in the samples as highlighted on line 162-164 and 169-171. The results showed no difference on the expected results.

3. It is noted in the supplementary section that samples are shipped at ambient temperature. Were overnight shipment methods used? What was the longest it took for samples to arrive at a designated laboratory? Was there any correlation with long transit times and discordant/invalid results? In other words, did long transit times lead to sample degradation and hence invalid results?

In this study no overnight shipment methods were used. The longest it took samples to reach final destination was fourteen days. The time taken to receive samples in this study showed no correlation with obtaining discordant results as highlighted in line 371-374. To ascertain whether our samples were still stable and consistent even after shipment were performed a stability test on a set of panel for each round. This panel was kept at ambient temperature and tested after three weeks from date of dispatch as shown in line 193-198.

4. Authors are requested to provide an original reference for lines 110-111 for the detection limit of GeneXpert MTB/RIF test

This has been added on line 111 (citation 21; Study done by Danica Helb et, al in 2010

5. Under supporting information, authors listed figures 1 to 5 as S1 Fig 1, S2 Fig 2 etc., Please clarify which figures belong in the main manuscript and supplementary sections respectively.

Figures 1-5 all belong to the main manuscript and have been added.

6. There is no information about the process behind obtaining ISO accreditation. Some detail about what changes were made to the SOPS to align with ISO’s requirements and how this aided in study’s success supporting lines 438-439 would be helpful

Line 120-134 have been added in methods to detail the process behind ISO 17043 accreditation. Line 402-407 have been added also to show the impact of aligning to ISO 17043.

7. Authors note that the major cause of false results is clerical errors. Did the authors consider using automated GeneXpert test reports instead of manual reporting?

During this study, we noticed the need to enroll an electronic system but we were unable to enroll it out during the study period. The system has developed in 2020 and piloted in Uganda but not yet put to full use.

8. Were the panel of strains freshly prepared during each around? Please clarify in methods section.

Panels were freshly prepared for each round as highlighted on line 185-187.

9. How are the results reported when a test reports no MTB detected but RIF sensitivity/resistance is detected?

The score was zero (0) for a site that reported “no MTB detected but RIF sensitivity/resistance is detected”. This being that rifampicin resistance could not be expected without MTB detection.

10. Minor comments

All acronyms have been expanded the first time they appear in the manuscript. For consistency acronym PT has been used in the entire manuscript. The entire manuscript has been revised to address grammatical errors.

Response to Reviewer 2

1) EQA should be defined as it is its first use.

This has been addressed.

2) The scoring system

For the scoring system, we have added the fraction score, e.g. 6 out of 8 points (line 262-266) in the methodology write up as suggested by the reviewer to clarify on the passing threshold. The adoption of a percentage score was to align the participant reports to the requirements of ISO 17043 clauses on data analysis that required percentage-scoring system.

3) Names of the study rounds

The round number have been renamed as; Pilot study retained its naming, round 2 2016 to round 1 (2016), round 1 2017 to round 2 (2017), round 2 2017 to round 3 (2017), and round 1 2018 to round 4 (2018).

4) Result discrepancy

The results have been thoroughly re-analyzed from the raw data to address the discrepancy in the earlier submission as noticed by the reviewer.

6a) The results in table 1 for pilot study were corrected to,

parameter Before review After review

Total results returned 48 51

No returns 4 1

Non-functional sites 3 3

In table 2, the results remained as before for the pilot study as follows

parameter Before review After review

Satisfactory score 48 48

Unsatisfactory score 3 3

Non-functional sites 3 3

In table 1, the results remained as before review for round 3 (2017)

The results in table 2 for round 3 (2017) were corrected as below,

parameter Before review After review

Satisfactory score 100 99

Unsatisfactory score 5 5

In table 1, the results remained as before review for round 4 (2018)

The results in table 2 for round 3 2017 were corrected as below,

parameter Before review After review

Satisfactory score 115(97.5%) 128(95.5%)

Unsatisfactory score 4(2.5%) 6(4.5%)

Line 297 (formerly line 288), number of times with sites scoring unsatisfactory was adjusted to 32 to match results in table 2 after data re-analysis.

Table 3 and Table 4 were added to show the number of laboratories scoring satisfactory and those with unsatisfactory respectively with their continuous times of participation.

Figure 4 graph has been added after re-analysis to show probability of scoring satisfactorily given the number of times a laboratory consistently participates

Table 5 was added showing to breakdown the different discordant results (false positives/negatives/resistant/sensitive)

5) List of issues raised during on and off site follow up

Table 6 has been added to show the frequency of issues raised during on and off site follow up

6) Performance of a laboratory inside or outside Uganda

The performance of laboratories within and outside Uganda had no significant difference and scores were seen not to vary at all in regards to the location of the laboratory as highlighted in line 446-448.

I would like again to appreciate the time awarded for this manuscript.

Thanks PlosOne

Yours faithfully

Kabugo Joel;

Author Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

Attachment

Submitted filename: response to reviewers.docx

Click here for additional data file.^{(20.3KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0251691.r003

Decision Letter 1

Pradeep Kumar

20 Jan 2021

PONE-D-20-20339R1

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

PLOS ONE

Dear Dr. Kabugo,

Please addresss reviewers comments and I would like to draw attention to a thorough review of langauge.

Please submit your revised manuscript by Mar 06 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Pradeep Kumar, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: No

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #1: In this revised version of manuscript titled “Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory” authors addressed most of the comments made by the reviewers. Important corrections were made in the results section and additional tables were included as per the suggestions of one of the reviewers. However, there are few additional corrections and discrepancies noted in the results section that need to be addressed before the manuscript is acceptable for publication. Further, the manuscript requires thorough revision to address typographical, grammatical errors and paraphrasing in certain sections for clarity. Following are my comments:

• In table 2, the percentage for round 2 should be corrected to 93.33 instead of 94.1

• Numbers in tables 3 and 4 do not make sense. For example, in table 3, how did the labs participating for the third time increase to 51 in round 2. Also, numbers 50 and 43 in round 4 also do not add up.

• Similarly, in table 4 how did the laboratory scoring unsatisfactorily for the 5th time appear while there are zero instances of 4th time. The results in the table also does not match the text in lines 335-338. Authors are requested to carefully revise and address.

• Lines 168-172: Authors performed a 20-fold dilution from 1.0 x 10^6 bacilli/mL. This will achieve a concentration of 5 x 10^4 bacilli/mL and not 1.0 x 10^4 bacilli/mL. Authors are requested to clarify. What is the final concentration tested on geneXpert?

Minor comments:

• Lines 51-54 and 210-211 may be rephrased for clarity.

• Please explain the relevance of lines 153-155

• Table 5 needs formatting adjustments for the headers to match the columns. Also, for consistency, authors may follow similar table formatting throughout the manuscript.

• Figure 1 has a typographical error in steps 4 and 5 (contious). The word pursued in misspelled in line 34 of abstract.

• Capitalization of words in the middle of a sentence is noted in several areas of the manuscript. For example – line 27 (Panels); line 29 (Laboratories); line 64 (Support Supervision). Authors are suggested to revise such areas.

Reviewer #2: The manuscript is much improved from the previous version. Figure 4, Table 5 and Table 6 are all good additions to the manuscript. However, there are still some errors and inconsistencies in the tables. My numbered points are below:

Table 2:

Discordant results read: 3, 6, 5, 10, 7

Table 5:

Discordant results read: 3, 6, 5, 3, 5

Table 5: Column values for false negative M. tb add up to 11 (1+2+1+3+4) but total says 10

Table 5: Round 4 (2017) should read Round 4 (2018)

Line 368-9: “The lowest number of errors/invalids/no results was reported in Round

369 4 (2018) and highest in Round 1 (2016) with errors reported as shown in Table 5”

This is misleading because from Table 5 one would say that the lowest number was in the pilot study or Round 3 (2017) and the highest was in Round 1 (2016). I presume you’re talking about the errors/invalids/no results as shown in Table 1. This comment should therefore be moved to Table 1.

Line 64: First use of PT in the text (as opposed to the introduction) so should be “Proficiency Testing (PT)”

Line 97: “Mycobacteria. Tuberculosis complex” should be “Mycobacterium tuberculosis complex”

Line 154: is this supposed to have an error message?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

PLoS One. 2021 May 14;16(5):e0251691. doi: 10.1371/journal.pone.0251691.r004

Author response to Decision Letter 1

10 Feb 2021

Response to Reviewer 1

1. In table 2, the percentage for round 2 should be corrected to 93.33 instead of 94.1 and unsatisfactorily adjusted to 6.67%. This has been done and also fig 3 adjusted

2. Tables 4 and 5 have been given further elaborations on the increasing numbers of participating scoring satisfactory scores as we moved through the rounds from line 323-327 and 352-372.

3. The results in the table also does not match the text in lines 335-338. Authors are requested to carefully revise and address.

More revisions has been made and explanation for this made from line 352-372. Information in the table has been aligned to match with that in the paragraphs (352-362) formerly 335-338.

4. Lines 168-172: Authors performed a 20-fold dilution from 1.0 x 10^6 bacilli/mL. This will achieve a concentration of 5 x 10^4 bacilli/mL and not 1.0

This has been amended to a concentration of 5.0 x 104 bacilli/mL as the final dilution for the panels since this is the actual dilution made. The target bacterial concentration of the panels was between 1.0 x 104 and 7.5 x 104 bacilli/mL.

5. Line 51-54 rephrased starting from line 51-58.

The whole manuscript has been paraphrased

6. Line 153-155 now line 157-159, now line 158-60

Was a clarification request from the Plos one Editorial team to “declare whether the strains used in the study were specifically for this study only or not”.

7. All tables have been reformatted and made consistent

8. Figure 1 has been rephrased at step 4 and 5 correcting typographical errors and capitalization in the middle of sentences.

9. Word Pursued in Line 34 has been corrected.

10. Other Words with capitalization in the middle of the sentence have been corrected with a more thorough manuscript revision by coauthors

Reviewer 2

1) In table 5, the figures 3, 6, 5, 3, 5 corresponds to the number of laboratories that reported the 3, 6, 5, 10, 7 discordant results in table 2 during the different rounds in the study. Elaboration statements have been added below table 5 to bring out this information.

2) The total of false negative M. tb in table 5 is adjusted to 11 corresponding to the figures when added up

3) Renaming of rounds in table 5 has been made.

4) The statement “the lowest number was in the pilot study or Round 3 (2017) and the highest was in Round 1 (2016) has been moved to line 308-309.

5) PT written in full (Proficiency Testing) for its first appearance in text line 68 former line 64

6) Mycobacteria. Tuberculosis complex” corrected to “Mycobacterium tuberculosis complex” in line 101.

7) Line 153-155 now line 157-159, Was a clarification request from the Plos one Editorial team to “declare whether the strains used in the study were specifically for this study only or not”.

Attachment

Submitted filename: response to reviewers.docx

Click here for additional data file.^{(20.3KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0251691.r005

Decision Letter 2

Shampa Anupurba

16 Apr 2021

PONE-D-20-20339R2

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

PLOS ONE

Dear Dr. kabugo,

Please submit your revised manuscript by May 31 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Shampa Anupurba, MD

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Line 55- ' mycobacterium tuberculsosis ' to be corrected to 'Mycobacterium tuberculosis'

Line 168- ' mgit and lj' to be written as MGIT and LJ and also expanded.

Line 421- 'It is at this phase were...'- were may be replaced by 'where'

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #2: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #2: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #2: (No Response)

**********

6. Review Comments to the Author

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

PLoS One. 2021 May 14;16(5):e0251691. doi: 10.1371/journal.pone.0251691.r006

Author response to Decision Letter 2

24 Apr 2021

Content Review

Line 55- ' mycobacterium tuberculsosis ' has been corrected to 'Mycobacterium tuberculosis'

Line 165 and 166- ' mgit and lj' has been re-written as MGIT and LJ and also expanded.

Line 421- were has been replaced by 'where'

References review

Reference (2) has been re-referenced citing it as it is supposed to be written. Earlier in the manuscript, its citation was incomplete.

Reference (11) [Agizew T, Boyd R, Ndwapi N, Auld A, Basotli J, Nyirenda S, et al. Peripheral clinic versus centralized laboratory-based Xpert MTB/RIF performance: Experience gained from a pragmatic, stepped-wedge trial in Botswana. PLoS One. 2017 Aug 1;12(8).] formerly reference (11) in the manuscript has been removed from this manuscript. This is because the information it carries does not strongly connect with the content matter mentioned in line 67-68.

Reference 13 has been re-referenced. Earlier in the manuscript, its citation was incomplete

Reference 18 has been re-referenced. Earlier in the manuscript, its citation was incomplete

Reference 21 has been re-referenced. Earlier in the manuscript, its citation was incomplete.

All the other references have been crosschecked for the correctness and support of content in the manuscript and we feel they are fit to be cited.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(15KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0251691.r007

Decision Letter 3

Shampa Anupurba

3 May 2021

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

PONE-D-20-20339R3

Dear Dr. kabugo,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Shampa Anupurba, MD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

PLoS One. doi: 10.1371/journal.pone.0251691.r008

Acceptance letter

Shampa Anupurba

5 May 2021

PONE-D-20-20339R3

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/Supranational reference laboratory

Dear Dr. Kabugo:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Shampa Anupurba

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(364.1KB, pdf)}

S2 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(374.1KB, pdf)}

S3 Text. Conformity certificate of biological samples used to prepare PT panels.

(PDF)

Click here for additional data file.^{(527.4KB, pdf)}

S1 File

(PDF)

Click here for additional data file.^{(554.8KB, pdf)}

S2 File

(PDF)

Click here for additional data file.^{(215.9KB, pdf)}

S3 File

(PDF)

Click here for additional data file.^{(452KB, pdf)}

S4 File

(PDF)

Click here for additional data file.^{(549.5KB, pdf)}

S5 File

(PDF)

Click here for additional data file.^{(259.4KB, pdf)}

Attachment

Submitted filename: response to reviewers.docx

Click here for additional data file.^{(20.3KB, docx)}

Attachment

Submitted filename: response to reviewers.docx

Click here for additional data file.^{(20.3KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(15KB, docx)}

Data Availability Statement

All relevant data are within the paper and its Supporting Information files.

[pone.0251691.ref001] 1.Scott C. et al. , “Evaluation of automated molecular testing rollout for tuberculosis diagnosis using routinely collected surveillance data—Uganda, 2012–2015,” Morbidity and Mortality Weekly Report, vol. 66, no. 12. Department of Health and Human Services, pp. 339–342, 31-March-2017. 10.15585/mmwr.mm6612a6 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref002] 2.World Health Organization (2013)., Global tuberculosis report 2013. World Health Organization. https://apps.who.int/iris/handle/10665/91355. [Google Scholar]

[pone.0251691.ref003] 3.Denkinger C. M., Nicolau I., Ramsay A., Chedore P., and Pai M., “Are peripheral microscopy centres ready for next generation molecular tuberculosis diagnostics?,” Eur. Respir. J., vol. 42, no. 2, pp. 544 LP– 547, August. 2013. 10.1183/09031936.00081113 [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref004] 4.Creswell J. et al. , “Results from early programmatic implementation of Xpert MTB/RIF testing in nine countries,” BMC Infect. Dis., vol. 14, no. 1, p. 2, December. 2014. 10.1186/1471-2334-14-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref005] 5.Audu R. A. et al. , “Proficiency testing for HIV, tuberculosis and malaria diagnosis in clinical laboratories in Nigeria,” Afr. J. Lab. Med., vol. 3, no. 1, 2014. 10.4102/ajlm.v3i1.102 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref006] 6.Barbé B., Yansouni C. P., Affolabi D., and Jacobs J., “Implementation of quality management for clinical bacteriology in low-resource settings,” Clinical Microbiology and Infection, vol. 23, no. 7. Elsevier B.V., pp. 426–433, 01-July-2017. 10.1016/j.cmi.2017.05.007 [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref007] 7.Libeer J. C., “External quality assurance programmes in medical laboratories,” Accredit. Qual. Assur., vol. 6, no. 4–5, pp. 151–153, 2001. 10.1016/s0009-8981(01)00518-6 [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref008] 8.“Brookman Brian, et al. Selection, Use and Interpretation of PT Schemes: 2nd edition 2011.”. [Google Scholar]

[pone.0251691.ref009] 9.Gershy-Damet G. M. et al. , “The world health organization African region laboratory accreditation process: Improving the quality of laboratory systems in the African region,” American Journal of Clinical Pathology, vol. 134, no. 3. pp. 393–400, September-2010. 10.1309/AJCPTUUC2V1WJQBM [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref010] 10.Palamountain K. M. et al. , “Perspectives on Introduction and Implementation of New Point-of-Care Diagnostic Tests,” J. Infect. Dis., vol. 205, no. suppl_2, pp. S181–S190, May 2012. 10.1093/infdis/jis203 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref011] 11.“Gumma V, Bond K, DeGruy K, Alexander H, Nhung NV, Hung NV, et al. 2016. Successful implementation of external quality assessment for GeneXpert and its impact on quality of testing in Vietnam, p 26. Abstr 3rd Int Conf Afr.”. [Google Scholar]

[pone.0251691.ref012] 12.Parekh B. S. et al. , “Dried tube specimens: A simple and cost-effective method for preparation of HIV proficiency testing panels and quality control materials for use in resource-limited settings,” J. Virol. Methods, vol. 163, no. 2, pp. 295–300, February. 2010. 10.1016/j.jviromet.2009.10.013 [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref013] 13.Garcia A. et al. , “Impact of proficiency testing program for laboratories conducting Early Diagnosis of HIV-1 infection in infants in low- to middle-income countries,” J. Clin. Microbiol., vol. 52, no. 3, pp. 773–780, 2014. 10.1128/JCM.03097-13 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref014] 14.Scott L., Albert H., Gilpin C., Alexander H., DeGruy K., and Stevens W., “Multicenter feasibility study to assess external quality assessment panels for Xpert MTB/RIF assay in South Africa,” J. Clin. Microbiol., vol. 52, no. 7, pp. 2493–2499, July. 2014. 10.1128/JCM.03533-13 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref015] 15.Miller W. G., Jones G. R. D., Horowitz G. L., and Weykamp C., “Proficiency testing/external quality assessment: Current challenges and future directions,” Clinical Chemistry, vol. 57, no. 12. pp. 1670–1680, December-2011. 10.1373/clinchem.2011.168641 [DOI] [PubMed] [Google Scholar]

[pone.0251691.ref016] 16.Albert H., Nathavitharana R. R., Isaacs C., Pai M., Denkinger C. M., and Boehme C. C., “Development, roll-out and impact of Xpert MTB/RIF for tuberculosis: what lessons have we learnt and how can we do better?,” Eur. Respir. J., vol. 48, no. 2, pp. 516–525, August. 2016. 10.1183/13993003.00543-2016 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref017] 17.Nikolayevskyy V. et al. , “External Quality Assessment for Tuberculosis Diagnosis and Drug Resistance in the European Union: A Five Year Multicentre Implementation Study,” PLoS One, vol. 11, no. 4, p. e0152926, April. 2016. 10.1371/journal.pone.0152926 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref018] 18.“Quick Guide to Quality Assurance of Xpert MTB/RIF Testing,” 2019. [Google Scholar]

[pone.0251691.ref019] 19.“SUPRANATIONAL REFERENCE LABORATORY–Timely Accurate Diagnostics for a TB Free Africa.” https://ntrl.or.ug/.

[pone.0251691.ref020] 20.Helb D. et al. , “Rapid detection of Mycobacterium tuberculosis and rifampin resistance by use of on-demand, near-patient technology,” J. Clin. Microbiol., vol. 48, no. 1, pp. 229–237, January. 2010. 10.1128/JCM.01463-09 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref021] 21.“global laboratory initiative advancing TB diagnosis A publication of the Global Laboratory Initiative a Working Group of the Stop TB Partnership Mycobacteriology Laboratory Manual.”

[pone.0251691.ref022] 22.“Stop TB Partnership—Global Laboratory Initiative (GLI).” Available: http://www.stoptb.org/wg/gli/gat.asp.

[pone.0251691.ref023] 23.“IATA—Infectious Substances Shipping Guidelines.” Available: https://www.iata.org/en/publications/store/infectious-substances-shipping-guidelines/.

[pone.0251691.ref024] 24.Agresti A L. J., “Analysis of Ordinal Categorical Data—Alan Agresti—Google Books,” Biometrika, vol. 80, pp. 527–534, 1993. [Google Scholar]

[pone.0251691.ref025] 25.WHO, “WHO monitoring of Xpert MTB/RIF roll-out,” WHO, 2017. [Google Scholar]

[pone.0251691.ref026] 26.Manickam T. S. and Ankanagari S., “Evaluation of quality management systems implementation in medical diagnostic laboratories benchmarked for accreditation,” vol. 6, no. 5, pp. 27–35, 2015. [Google Scholar]

[pone.0251691.ref027] 27.Gumma V. et al. , “Impact of external quality assurance on the quality of Xpert MTB/RIF testing in Viet Nam,” J. Clin. Microbiol., vol. 57, no. 3, March. 2019. 10.1128/JCM.01669-18 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref028] 28.Hanrahan C. F. et al. , “The patient impact of point-of-care vs. Laboratory placement of XpertW MTB/RIF,” Int. J. Tuberc. Lung Dis., vol. 19, no. 7, pp. 811–816, July. 2015. 10.5588/ijtld.15.0013 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0251691.ref029] 29.Lawn S. D. and Nicol M. P., “Xpert® MTB/RIF assay: Development, evaluation and implementation of a new rapid molecular diagnostic for tuberculosis and rifampicin resistance,” Future Microbiology, vol. 6, no. 9. Europe PMC Funders, pp. 1067–1082, September-2011. 10.2217/fmb.11.84 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Implementation of GeneXpert MTB/Rif proficiency testing program: A Case of the Uganda national tuberculosis reference laboratory/supranational reference laboratory

Joel Kabugo

Joanita Namutebi

Dennis Mujuni

Andrew Nsawotebba

George William Kasule

Kenneth Musisi

Edgar Kigozi

Abdunoor Nyombi

Pius Lutaaya

Fredrick Kangave

Moses L Joloba

Roles

Abstract

Background

Methods

Results

Conclusion

Introduction

Materials and methods

Fig 1. Roadmap for establishing GeneXpert PT scheme in Uganda.

GeneXpert MTB/RIF PT scheme concept inception and principle

Staff training, SOP development, and ISO/IEC 17043 standard alignment

PT panel production

Ethical statement

Enrollment of participants in the PT scheme

Fig 2. Procedures during the GeneXpert MTB/RIF PT preparation and testing process.

PT pilot

Round 1 (2016) to Round 4 (2018)

Results compilation and data analysis

Results and discussion

Laboratory enrollment and participation

Table 1. Participation profile for all the rounds sent out.

Performance of the participants

Table 2. Performance of participating sites throughout the different rounds.

Table 3. Number of laboratories achieving satisfactory scores and their different participation times.

Fig 3. Shows the performance of participants during the different round enrolled.

Evaluation of the times of participation and unsatisfactory performance

Table 4. Laboratories scoring unsatisfactorily and their times of participation into the PT scheme.

Fig 4. Probability of laboratories scoring satisfactorily in GeneXpert PT over their continued rounds of participation.

Discordant results

Table 5. Discordant results reported from the different rounds.

Turnaround time (TAT)

Fig 5. Shows average TAT taken for sites to report results from the dispatch date during the different rounds.

On and off-site Support Follow up (OSF) and corrective actions

Table 6. Frequencies of issues identified during root cause analysis for laboratories with unsatisfactory scores.

ISO/IEC 17043 implementation

Discussion

Conclusion

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Pradeep Kumar

Roles

Author response to Decision Letter 0

Decision Letter 1

Pradeep Kumar

Roles

Author response to Decision Letter 1

Decision Letter 2

Shampa Anupurba

Roles

Author response to Decision Letter 2

Decision Letter 3

Shampa Anupurba

Roles

Acceptance letter

Shampa Anupurba

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles