Diagnostic accuracy of paper-based reporting of tuberculosis treatment outcomes in rural eastern Uganda

Highlights

• •Accurate tuberculosis (TB) data is imperative for performance monitoring and strategic decision-making.
• •Data inaccuracies between TB control programmes are largely understudied.
• •A paper-based system for treatment outcome compilation is inaccurate.
• •A computer-generated TB treatment outcome is needed to ensure data accuracy.

Abstract

Inaccurate reporting of tuberculosis (TB) data to the district and national TB control programmes undermines effective TB control, yet this remains understudied. This study assessed the accuracy of the paper-based approach compared with the World Health Organization (WHO) standard TB treatment outcome as the gold standard for the determination of TB treatment outcome. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the paper-based approach, as well as the percentage agreement between the paper-based approach and the WHO standard TB treatment outcome, are reported. Data from 987 participants were used. Ninety-three participants were misclassified as cured and 195 were misclassified as not cured by the paper-based approach, giving 62.7% sensitivity, 80.0% specificity, 77.9% PPV, 65.5% NPV and percentage agreement of 70.8%. Treatment failure had 64.7% sensitivity, 99.9% specificity, 52.4% PPV, 99.4% NPV, and percentage agreement of 98.4%. Treatment success had 98.8% sensitivity, 96.8% specificity, 99.2% PPV and 94.8% NPV. The paper-based approach was found to report treatment success accurately, but did not report cure and treatment failure accurately. Interventions are thus required to improve the accuracy of the paper-based approach.

Background

The World Health Organization (WHO) recommends a cure rate of 85% and a treatment success rate of ≥95% for optimally performing TB control programmes (World Health Organization, 2017). The success of a national TB control programme for effective control of TB largely depends on its ability to report accurate data on TB treatment outcomes at all levels (World Health Organization, 2018). Inaccurate data have a negative impact on patient treatment and programming. Accurate data are imperative for efficient programme management as this guides the evaluation of programme targets, efforts to allocate staff, and result monitoring.

In Uganda, paper-based TB unit registers are used for routine reporting of TB data to the National TB and Leprosy Control Programme (NTLP) from peripheral health facilities, districts and regions. As the data capture process is not electronic, it is referred to as ‘paper-based’. Usually, TB focal persons (healthcare providers identified and designated to provide stewardship for TB care and management) at peripheral health facilities record, count, compute and submit the health facility TB performance data to the district through the District TB and Leprosy Supervisor (DTLS) who, in turn, submits the data to NTLP. TB treatment outcomes are thus computed through a paper-based system as there are no computerized systems. Although this is cheap and pragmatic for a resource-limited setting, using the paper-based approach for the determination of TB treatment outcome has the potential disadvantage of inaccurate data generation, but the magnitude of these inaccuracies has not been studied rigorously in Uganda. The primary objective of this study was to investigate the accuracy of the paper-based approach in determining TB treatment outcomes compared with the WHO standard TB treatment outcomes. Although data for all TB treatment outcomes were available, cure, treatment failure and treatment success were considered in this study as they are determined based on conditional probabilities, and are therefore prone to error.

Materials and methods

This study used data from a previous study on the completion of sputum smear monitoring among drug-susceptible persons with pulmonary bacteriologically confirmed TB, collected across 10 TB treatment units in rural eastern Uganda between January 2015 and June 2018 (Izudi et al., 2020d). These data have been described elsewhere (Izudi et al., 2020a,2020b,2020c).

The parent study received approval from Mbarara University of Science and Technology Research Ethics Committee (Ref. No. 03/11-18) and the Uganda National Council for Science and Technology (HS 2531).

A waiver to analyse the secondary data was granted by the ethics committee. A diagnostic test was conducted using the WHO standard TB treatment outcome computed using a statistical software program as the gold standard, and cross-tabulated with TB treatment outcomes reported in the TB unit register. Paper- and case-based computerized data were from the same register and were collected after routine (quarterly) data validation by the DTLS. The outcomes (i.e. cure, treatment failure and treatment success) were assigned in accordance with the WHO definitions (Table 1). In the diagnostic test/analysis, the paper-based and WHO standard TB treatment outcomes were compared, with the former as the screening test and the latter as the true outcome. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were computed (Table 1). All analyses were performed in Stata Version 15 (Stata Corp., College Station, TX, USA) using the "diagt" command.

Table 1.

List of operational definitions.

Terms	Definitions
Sensitivity	The proportion of participants who have the WHO standard TB treatment outcome, and who have been correctly identified as having the outcome by the paper-based approach.
Specificity	The proportion of subjects who do not have the WHO standard TB treatment outcome, and who have been correctly identified as not having the outcome by the paper-based approach.
Positive predictive value	The probability that a participant who has been identified by the paper-based approach as having a TB treatment outcome has the outcome as per the WHO standard TB treatment outcome.
Negative predictive value (NPV)	The probability that a participant who has been identified by the paper-based approach as not having a TB treatment outcome does not have the outcome as per the WHO standard TB treatment outcome.
Cure	A patient with pulmonary TB with bacteriologically confirmed TB at the beginning of treatment, who is smear or culture negative in the last month of treatment and on at least one previous occasion (World Health Organization, 2013).
Treatment failed	A patient whose sputum smear or culture is positive after ≥5 months of treatment (World Health Organization, 2013).
Treatment success	The sum of cured and treatment completed (World Health Organization, 2013).

WHO, World Health Organization; TB, tuberculosis.

Results

In total, 987 patients were analysed (Table 2). For cure, the paper-based approach showed sensitivity of 62.7% [95% confidence interval (CI) 58.4–66.9], specificity of 80.0% (95% CI 76.0–83.5), PPV of 77.9% (95% CI 73.6–81.8) and NPV of 65.5% (95% CI 61.5–69.5). The percentage agreement between the two approaches was 70.8% (Kappa=0.421; P<0.0001). For treatment failure, sensitivity was 64.7% (95% CI 38.3–85.8), specificity was 99.9% (95% CI 98.1–99.5), PPV was 52.4% (95% CI 29.8–74.3) and NPV was 99.4% (95% CI 98.7–99.8). The percentage agreement was 98.4% (Kappa=0.571, P<0.0001). For treatment success, sensitivity was 98.8% (95% CI 97.7–99.4%), specificity was 96.8% (95% CI 93.1–98.8), PPV was 99.2% (95% CI 98.4–99.7) and NPV was 94.8% (95% CI 90.6–97.5).

Table 2.

Cross-tabulation of tuberculosis (TB) treatment outcomes based on the World Health Organization (WHO) definition with the paper-based system tuberculosis treatment outcome registered in the TB unit register (n=987).

TB treatment outcome recorded in TB unit register	True TB treatment outcome based on WHO definition
		No	Yes	Total
Cure	No	371	195	566
	Yes	93	328	421
	Total	464	523	987
Treatment failure	No	960	6	966
	Yes	10	11	21
	Total	970	17	987
Treatment success	No	181	10	191
	Yes	6	790	796
	Total	186	800	987

Discussion

The data show that the paper-based approach determines treatment success accurately, but does not determine cure and treatment failure accurately. Incorrectly categorizing people who have not achieved cure as ‘cured’ exacerbates TB morbidity and mortality at both individual and population levels. Inconsistences in TB data recording and reporting are not unique, and have been reported previously (Heunis et al., 2011).

Previous research to assess the accuracy and completeness of data in the TB treatment register in two South African urban communities using record linkage also found inadequacies in the accuracy and completeness of data between the TB treatment and central laboratory registers (Dunbar et al., 2011). The present finding is thus consistent with a previous study recommendation calling for training of TB staff on the correct recording and reporting of TB data and improved access to computers (Podewils et al., 2016). The reasons for incorrect computation of TB treatment outcomes in the study setting need to be identified and tackled to enable accurate reporting of TB performance data at national and international levels. The paper-based approach to computing TB treatment outcomes and reporting to NTLP is accurate for treatment success, but not for cure and treatment failure. Revisions that incorporate conditional probabilities are required in the paper-based approach to improve its accuracy along with correct recording. This will enable good strategic planning, effective programme design and implementation, and correct decision-making at both district and national levels.