SUMMARY
BACKGROUND
Confirmation of cure for multidrug-resistant tuberculosis (MDR-TB) patients requires laboratory tests for Mycobacterium tuberculosis growth on culture media. Outcome decisions dictate patient management, and inaccuracies place patients at an increased risk of morbidity and mortality, and may contribute to continued transmission of MDR-TB.
OBJECTIVE
To examine concordance between programmatic and laboratory-based MDR-TB treatment outcomes.
METHODS
The study population included 1658 MDRTB patients in Peru treated between 1996 and 2002 with both program and laboratory-based outcomes. Laboratory-based outcomes were assigned according to international standards requiring at least five consecutive negative cultures in the last 12 months of treatment to confirm cure.
RESULTS
Compared to the global culture-defined standard classification, only 1.1% of treatment successes, but 54.3% of failures, were misclassified programmatically. Overall, 10.4% of patients identified by a clinician as having a successful treatment outcome still had cultures positive for MDR-TB.
CONCLUSION
Most patients with successful treatment outcomes by strict culture definitions were also classified by clinicians as having successful outcomes. However, many culture-confirmed failures were missed. In light of delays and incomplete access to culture in MDR-TB programs, efforts should be made to improve the accuracy of programmatically determined treatment outcomes.
Keywords: drug-resistant TB, program factors, cross-validation, bacteriologic outcomes, clinical judgment
There are an estimated 14 million cases of tuberculosis (TB) worldwide, with multidrug-resistant TB (MDR-TB) making up a growing percentage.1,2 MDR-TB is defined as a strain of Mycobacterium tuberculosis resistant to at least both isoniazid and rifampin, the two most effective first-line drugs. Treatment for MDR-TB is longer and more complex than for drug-susceptible TB, requiring four or more secondline medications.3 These drugs are more expensive, can cause severe adverse events, and must be taken for at least 18–24 months.4,5 Due to these challenges, poor treatment outcomes among patients with MDR-TB are more common than for those with drug-susceptible TB.6,7
Patients with MDR-TB should be placed on an appropriate treatment regimen and treated continuously until a cured or completed outcome is achieved. If treatment ends before a patient is effectively cured, it can result in poor individual outcomes, including worsening disease and death.5 These patients may also contribute to the ongoing spread of MDR-TB in the community, compromising TB control efforts.
Proper and complete treatment for individuals with MDR-TB is therefore extremely important. The standardized method of determining cure is based on bacteriologic laboratory testing for the growth of M. tuberculosis on culture media. An international working group proposed a set of consensus definitions in 2005, with outcome determinations based on culture results in the final 12 months of treatment.8,9 Without laboratory validation research and studies looking at prospective data on disease recurrence, it is unknown whether bacteriologic results accurately reflect true treatment outcomes, but laboratory-based determinations are currently considered best practice. In resource-poor areas, however, laboratory results can be slow and difficult to obtain, due to shortages in equipment and human resources, inadequate infrastructure, and weak transportation and management systems.10 Health care providers therefore often rely on clinical observation to determine treatment outcomes. These programmatically based outcomes have become the convention in many areas, even where laboratory testing services are available.11
Given the practice of basing treatment decisions on provider-determined outcomes and the difficulties of obtaining laboratory results in resource-poor settings, it is important to establish the accuracy of programmatically determined treatment outcomes. Here, we describe the concordance between the two types of outcome assignments and the predictors of concordance among a cohort of MDR-TB patients in Peru.
METHODS
Study population
A retrospective cohort study was conducted to compare programmatically assigned and bacteriologically defined MDR-TB outcomes among adult patients in Peru who initiated treatment between August 1996 and March 2002. Eligible patients were identified from an electronic database with information abstracted from patient medical charts. Individuals who started treatment during the study period were examined for treatment outcomes.
The subset of patients who had both programmatic and laboratory-based outcomes was analyzed to determine concordance between the two methods, and to identify any socio-demographic, social or clinical characteristics associated with concordance.
The study was reviewed and approved by the ethics review boards at the Instituto Nacional de Salud del Perú, the Harvard Medical School, and the US Centers for Disease Control and Prevention.
MDR-TB treatment outcome definitions
Programmatically determined outcomes
Programmatically determined outcomes were final MDR-TB treatment outcomes assigned by the provider and recorded in the patient medical chart. These outcomes were based on clinical judgment, taking into account medical history, treatment adherence, available smear and culture laboratory results and clinical presentation. Programmatically determined treatment outcome categories included cured, completed, defaulted, failed, died, discontinued due to adverse events, treatment suspended, transferred, in treatment, and not available. During the study period, programmatically determined outcome definitions were not uniform across providers or over time.
Laboratory determined outcomes
International consensus definitions were used for laboratory determined treatment outcomes, with three (cured, completed or failed) determined by bacteriologic testing.8 Cure was defined as completion of 18 months of treatment for standardized regimens and 24 months for individualized regimens, plus a minimum of five cultures testing consistently negative for M. tuberculosis in the final 12 months of treatment. Patients with a single positive culture within this time frame were still considered cured, as long as the positive culture was followed by a minimum of three consecutive negative cultures taken at least 30 days apart. MDR-TB patients were considered to have failed treatment if, based on a minimum of five cultures in the final 12 months of treatment, they had more than one positive culture result. Furthermore, patients with one of the final three specimens taken during treatment testing culture-positive, or for whom a clinical decision had been made to terminate treatment due to persistent culture positivity or adverse drug reactions, were also considered treatment failures. The treatment outcome ‘completed’ was assigned to patients who had adequately completed treatment, but had insufficient bacteriologic evidence to conclusively establish cure or failure.8
Patient socio-demographic and clinical characteristics
All medical and social history variables were categorized as ‘Known history of ___’ or ‘No known history of ___’. A count variable of social risk factors tallied the number of the following: homelessness, smoking, alcoholism, drug addiction, commercial sex work, criminal activity, imprisonment, unemployment, institutionalization, and military service. Body mass index (BMI) and age were examined as both continuous and categorical variables. Urban was defined as residing in Lima or Callao, the metropolitan Lima area.
Statistical analyses
A simple κ statistic and 95% confidence interval (CI) was calculated to assess overall agreement between programmatic and laboratory outcomes. As the laboratory outcome variable had only three categories (cured, completed, failed), the programmatic outcome variable was also limited to these three categories.
Next, cured and completed outcomes were combined to form a single successful outcome, and contrasted against failure. For the purposes of this analysis, performance characteristics (sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were calculated using the laboratory outcome as the gold standard.
Finally, logistic regression was used to examine possible factors associated with outcome concordance. Available socio-demographic, clinical, and social history variables were examined for univariate association with outcome agreement, defined as 1 if laboratory and programmatic outcomes matched and 0 if they were different. Any variable with a univariate association P ≤ 0.2 was considered for inclusion in a multivariate model. All models included age and sex as potential confounders. Effect modification by treatment strategy (individualized or standardized) was assessed. A final model predicting concordance was constructed retaining only variables with P < 0.05. Analyses were performed using SAS, version 9.2 (SAS Institute Inc, Cary, NC, USA).
RESULTS
A total of 2961 MDR-TB patients initiated treatment during the study period: 1658 patients were included in the present analysis (Figure). The final study population, comprising patients with both programmatically and bacteriologically defined outcomes, was similar to the original cohort in terms of sex, marital status, and occupation, but had a significantly lower rate of human immunodeficiency virus (HIV) infection and a higher proportion of patients receiving individualized treatment (Table 1). The majority were male (60%), and 49% were aged 20–29 years. Residence in the Lima/Callao metropolitan area was reported for 81% of the study population, and 58% were household contacts of a TB case.
Figure.
Flow chart for inclusion in analysis of outcome concordance for MDR-TB patients in Peru, 1996–2002. MDR-TB = multidrug-resistant tuberculosis.
Table 1.
Socio-demographic and clinical characteristics of multidrug-resistant tuberculosis patients in Peru, 1996–2002
| All study participants (N = 2961) n (%) |
No laboratory outcome (n = 1224) n (%) |
No clinic outcome (n = 269) n (%) |
Analysis population (n = 1658) n (%) |
P value* | |
|---|---|---|---|---|---|
| Age, years | |||||
| 18–19 | 228 (7.7) | 98 (8.0) | 24 (9.0) | 124 (7.5) | <0.01 |
| 20–29 | 1362 (46.1) | 508 (41.6) | 110 (41.0) | 815 (49.2) | |
| 30–39 | 673 (22.8) | 292 (23.9) | 64 (23.9) | 362 (21.9) | |
| 40–49 | 314 (10.6) | 145 (11.9) | 35 (13.1) | 160 (9.7) | |
| 50–59 | 211 (7.1) | 93 (7.6) | 15 (5.6) | 116 (7.0) | |
| ≥60 | 168 (5.7) | 86 (7.1) | 20 (7.5) | 78 (4.7) | |
| Sex | |||||
| Male | 1778 (60.1) | 743 (60.7) | 155 (57.6) | 994 (60.0) | 0.93 |
| Female | 1182 (39.9) | 481 (39.3) | 114 (42.4) | 663 (40.0) | |
| Body mass index, m/kg2† | |||||
| Underweight (<18.5) | 296 (22.5) | 164 (30.7) | 31 (29.0) | 126 (16.8) | <0.01 |
| Normal (18.5–25) | 806 (61.4) | 308 (57.7) | 62 (57.9) | 479 (63.9) | |
| Overweight /obese (>25) | 211 (16.1) | 62 (11.6) | 14 (13.1) | 145 (19.3) | |
| Marital status† | |||||
| Single/divorced/widowed | 1540 (55.8) | 585 (44.5) | 118 (50.0) | 905 (58.0) | 0.24 |
| Married or living together | 1214 (44.2) | 526 (47.3) | 118 (50.0) | 657 (42.1) | |
| Post-secondary/technical education† | 547 (20.2) | 162 (13.2) | 51 (19.0) | 367 (22.1) | 0.05 |
| Urban or rural residence | |||||
| In Lima/Callao metropolitan area | 2305 (77.9) | 902 (73.7) | 175 (65.1) | 1341 (80.9) | <0.01 |
| Number of social risk factors‡ | |||||
| 0 | 1737 (58.7) | 664 (54.3) | 169 (62.8) | 1027 (61.9) | 0.01 |
| 1 | 775 (26.2) | 337 (27.5) | 63 (23.4) | 416 (25.1) | |
| 2+ | 449 (15.2) | 223 (18.2) | 37 (13.8) | 215 (13.0) | |
| Health care worker | 46 (1.6) | 19 (1.6) | 6 (2.2) | 25 (1.5) | 0.76 |
| Household contact of TB case | 1702 (57.5) | 691 (56.5) | 143 (53.2) | 962 (58.0) | 0.67 |
| HIV-positive | 67 (2.3) | 49 (4.0) | 5 (1.9) | 17 (1.0) | <0.01 |
| Treatment strategy | |||||
| Individualized | 399 (13.5) | 113 (9.2) | 34 (12.6) | 271 (16.3) | <0.01 |
| Standardized | 2562 (86.5) | 1111 (90.8) | 235 (87.4) | 1387 (83.7) |
Based on χ2 (categorical variables) or t-test (continuous variables) comparisons of the final analysis population to the original cohort.
In the final analysis population, 908 observations were missing a value for body mass index, 96 were missing marital status information, and 123 were missing education level information.
Included known history of homelessness, smoking, alcoholism, drug addiction, commercial sex work, criminal activity, imprisonment, institutionalization, unemployment, or military service.
TB = tuberculosis; HIV = human immunodeficiency virus.
Concordance between programmatically and laboratory determined outcomes
Of the 1152 patients declared cured by a clinician, 123 (10.7%) were bacteriologically deemed to be treatment failures (Table 2). Similarly, 27 (9.4%) of the 287 individuals clinically categorized as completed were bacteriologic failures. Together, these misclassified categories totaled 150 patients (9.0% of all patients) who were programmatically considered treatment successes, yet had laboratory evidence of persistent MDR-TB.
Table 2.
Concordance between programmatically based and laboratory determined treatment outcomes (all categories) for multidrug-resistant tuberculosis patients in Peru, 1996–2002
| Laboratory outcome | ||||
|---|---|---|---|---|
| Programmatic outcome |
Cured n (%) |
Completed n (%) |
Failed n (%) |
Total n |
| Cured | 864 (75.0)* | 165 (14.3) | 123 (10.7) | 1152 |
| Completed | 188 (65.5) | 72 (25.1) | 27 (9.4) | 287 |
| Failed | 9 (6.4) | 5 (3.6) | 127 (90.1) | 141 |
| Discontinued | 0 | 0 | 4 (100.0) | 4 |
| In treatment | 31 (43.7) | 20 (28.2) | 20 (28.2) | 71 |
| Transfer | 1 (33.3) | 1 (33.3) | 1 (33.3) | 3 |
| Total | 1093 | 263 | 302 | 1658 |
Number of patients (row percentage).
At the same time, 9 (6.4%) patients deemed treatment failures by their health care providers were cured, and an additional 5 (3.6%) completed treatment based on their laboratory results. Of the 71 individuals still in treatment, 31 (43.7%) had sufficient laboratory results to be considered cured, and 20 (28.2%) had laboratory results establishing treatment completion.
The overall κ statistic for percentage agreement was 0.30 (95%CI 0.25–0.34).
For the dichotomized outcome–successful (cured or completed) vs. unsuccessful (failed)–performance characteristics of programmatically determined outcomes were evaluated using the outcome based on bacteriologic laboratory results as the gold standard. The sensitivity of the clinicians’ determinations was 98.9%, but specificity was only 45.7% (Table 3). The PPV and NPV were respectively 89.6% and 90.1%.
Table 3.
Concordance between programmatically based and laboratory determined treatment outcomes in binary categories for multidrug-resistant tuberculosis patients in Peru, 1996–2002
| Programmatic outcome |
Laboratory outcome | ||
|---|---|---|---|
| Cured/completed | Failed | Total | |
| Cured/completed | 1289 (true-positive) | 150 (false-positive) | 1439 |
| Failed | 14 (false-negative) | 127 (true-negative) | 141 |
| Total | 1303 | 277 | 1580 |
| Sensitivity, % | 98.9 | ||
| Specificity, % | 45.8 | ||
| PPV, % | 89.6 | ||
| NPV, % | 90.1 | ||
Modeling for predictors of concordance
In univariate analysis, advanced education, employment as a health care worker, previous contact with a TB case, HIV, and receiving individualized treatment were the only predictors significantly associated with concordance between programmatic and laboratory-based outcomes, based on a cut-off of P < 0.2 (Table 4). Only treatment strategy was retained in the final multivariate model as a significant predictor of the outcome match variable (individualized vs. standardized treatment odds ratio 2.00, 95%CI 1.48– 2.70, P < 0.0001). In sub-analyses using stratified models and interaction terms, there was no detectable effect modification of other covariates by treatment strategy.
Table 4.
Univariate associations between socio-demographic characteristics and outcome match variable
| Variable | OR (95%CI) | P value |
|---|---|---|
| Age category (6 levels, by decade) | 1.00 (0.93–1.09) | 0.30 |
| Sex (male/female) | 1.08 (0.88–1.32) | 0.49 |
| BMI category (4 levels) | 1.07 (0.86–1.33) | 0.57 |
| Marital status | 1.03 (0.89–1.19) | 0.78 |
| Any advanced education (yes/no) | 1.22 (0.95–1.56) | 0.12 |
| Occupational category | 1.00 (0.94–1.07) | 0.61 |
| Lives in metropolitan Lima (yes/no) | 1.11 (0.86–1.43) | 0.42 |
| Count of social risk factors (0/1/2+) | 1.03 (0.89–1.19) | 0.69 |
| Worked as health care worker (yes/no) | 2.98 (1.02–8.72) | 0.04 |
| Contact with TB case (yes/no) | 1.15 (0.94–1.41) | 0.17 |
| Diabetes (yes/no) | 0.92 (0.58–1.45) | 0.71 |
| HIV-positive (yes/no) | 0.49 (0.19–1.29) | 0.14 |
| Treatment strategy (individualized/standardized) | 2.01 (1.49–2.71) | <0.0001 |
OR = odds ratio; CI = confidence interval; BMI = body mass index; HIV = human immunodeficiency virus.
DISCUSSION
In this cohort of 1658 MDR-TB patients in Peru, the majority (64.1%) had the same outcome using bacteriological or clinical criteria. However, the proportion classified differently is of concern. Overall, 10.4% of the patients (150/1439) declared to be cured or completed by their health care providers were defined as treatment failures based on bacteriologic results. Once assigned a successful outcome, these individuals were released from care and given no further treatment, which could have resulted in worsening disease or death and a risk to their community. Misclassification as failure could also be cause for concern. This cohort contained 65 patients (4.0%) whose bacteriologic results indicated cure or treatment completion but who were still receiving treatment, or had been deemed treatment failures and could be referred for additional treatment. This unnecessary treatment could have negative consequences, including increased costs and potentially toxic adverse events from second-line medications.
The κ statistic of 0.30 indicates poor agreement between programmatic and laboratory outcomes. Due to the vital importance of accurate treatment outcomes for MDR-TB and the implications of misclassification, this level of agreement is not adequate.
The sensitivity and specificity illustrate the differences between the two outcome determinations. A high sensitivity of 98.9% shows that most laboratory-based successful outcomes are also being identified as successful outcomes using programmatic methods. However, the specificity of 45.7% indicates that fewer than half of bacteriologically unsuccessful MDR-TB treatment outcomes are being recognized as such by clinicians. This suggests a need to educate health care providers about the importance of continuing treatment until a successful outcome is confirmed bacteriologically. Further, previous research has reported that less than half of health care workers in Peru are aware that inadequate treatment could lead to drugresistant TB, highlighting a need to ensure all clinicians are aware of the causes of drug-resistant TB and the consequences of premature discontinuation of therapy.10 While the individuals surveyed were not necessarily the physicians making outcome determinations, these gaps in TB knowledge among health care workers could be negatively affecting TB management in Peru.
Only one variable was found to be a significant predictor of concordance. MDR-TB patients who received individualized treatment had twice the odds of having concordant programmatic and laboratory outcomes as those who received standardized treatment. Patients receiving individualized treatment regimens had more intensive follow-up (including twice-daily directly observed therapy by community treatment supporters), which could explain their increased odds of concordance.
Study limitations
This study had several limitations due to the constraints of available data. Nearly half of this patient cohort was excluded due to either a missing programmatic outcome or a lack of sufficient bacteriologic evidence to establish a laboratory outcome of cure, completion, or failure. The analysis population was statistically different from the full patient population on a number of characteristics. The necessary exclusion of this subset of patients may therefore have distorted the final results, if those without both outcome determinations were either more or less likely to have concordant outcomes.
The lack of validation of either bacteriologic or programmatic outcomes in this study makes the outcome discordance difficult to interpret. Our laboratory outcome of treatment failure was based on international consensus definitions;8 however, we recognize the constraints in deeming patients as failed based on bacteriologic findings at the conclusion of the treatment course without post-treatment followup.12 Prospective studies that include 6- or 12-month post-treatment cultures and clinical monitoring to reexamine treatment failures and relapse or recurrent disease rates of MDR-TB patients would provide valuable data on the accuracy of each of these outcome determination methods.
Data for this analysis were abstracted from medical charts and program data, which were not intended for research purposes. Many of the social and medical history variables were therefore missing for the majority of patients, and there was no mechanism for verifying the abstracted information. The missing data might be one reason why no social factors were identified as significant predictors of concordance.
Another shortcoming was the lack of information on physician characteristics. A physician’s training, experience, background, or location (e.g., in a high or low TB prevalence area) could conceivably affect his or her ability to correctly judge MDR-TB treatment outcomes. Some providers or facilities may have better track records than others at correctly determining outcomes, and future research could look at provider characteristics that may predict outcome concordance.
CONCLUSIONS
There are many reasons why programmatic outcomes do not always agree with laboratory results. In some cases, bacteriologic results may not be communicated to the physician, even if tests are performed and recorded at the national level. In other cases, the return of results to the clinician may be slow, resulting in the patient’s outcome being established before the laboratory results arrive. Finally, some clinicians will rely on their own clinical judgment over the determination of a laboratory, perhaps because they distrust laboratory methods, or because they trust their own observation above a test result. Whatever the reason for the discordant outcomes, steps must be taken to increase concordance between providers and internationally recognized bacteriologic outcome definitions. Until a rapid test to monitor response to TB treatment becomes readily available, outcome determinations by clinicians will continue to be used; it is therefore necessary to create, evaluate, and implement a mechanism to improve the accuracy of health care providers’ outcome determinations. Additional research is needed to identify patient or physician characteristics that influence accurate clinical determinations. Studies looking at what characteristics physicians rely upon to make these decisions may help guide interventions to improve classifications. It is also imperative to emphasize increased laboratory capacity and quality assurance, as well as training on the importance of bacteriologic results. If clinicians are better informed about the value of laboratory results in making outcome determinations, then, as laboratory services become more readily available, these clinicians may rely on them to a greater extent. Ensuring that outcome determinations are accurate is an essential step towards controlling and reducing the spread of MDR-TB.
Acknowledgements
Sources of financial support for this study include the Bill & Melinda Gates Foundation, the David Rockefeller Center for Latin American Studies at Harvard University, and Thomas J White. The authors also acknowledge the support of career development awards from the National Heart, Lung, and Blood Institute (K01 HL080939, to MCB) and the National Institute of Allergy and Infectious Diseases (K01 A1065836, to CDM).
References
- 1.World Health Organization. Geneva, Switzerland: WHO; 2010. [Accessed December 2011]. Media centre: tuberculosis fact sheet no. 104. http://www.who.int/mediacentre/factsheets/fs104/en/index.html. [Google Scholar]
- 2.World Health Organization. Geneva, Switzerland: WHO; 2010. [Accessed December 2011]. Multidrug and extensively drug-resistant TB (M/XDR-TB): 2010 global report on surveillance and response. WHO/HTM/TB/2010.3. http://whqlibdoc.who.int/publications/2010/9789241599191_eng.pdf. [Google Scholar]
- 3.Mukherjee JS, Rich ML, Socci AR, et al. Programmes and principles in treatment of multidrug-resistant tuberculosis. Lancet. 2004;363:474–481. doi: 10.1016/S0140-6736(04)15496-2. [DOI] [PubMed] [Google Scholar]
- 4.Bloss E, Kukša L, Holtz TH, et al. Adverse events related to multidrug-resistant tuberculosis treatment, Latvia, 2000–2004. Int J Tuberc Lung Dis. 2010;14:275–281. [PubMed] [Google Scholar]
- 5.Johnston JC, Shahidi NC, Sadatsafavi M, Fitzgerald J M. Treatment outcomes of multidrug-resistant tuberculosis: a systematic review and meta-analysis. PLoS One. 2009;4:e6914. doi: 10.1371/journal.pone.0006914. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ahmad S, Mokaddas E. Recent advances in the diagnosis and treatment of multidrug-resistant tuberculosis. Respir Med. 2009;103:1777–1790. doi: 10.1016/j.rmed.2009.07.010. [DOI] [PubMed] [Google Scholar]
- 7.Wright A, Zignol M. Geneva, Switzerland: World Health Organization; 2008. Anti-tuberculosis drug resistance in the world. Report no. 4. [Google Scholar]
- 8.Laserson KF, Thorpe LE, Leimane V, et al. Speaking the same language: treatment outcome defi nitions for multidrug-resistant tuberculosis. Int J Tuberc Lung Dis. 2005;9:640–645. [PubMed] [Google Scholar]
- 9.World Health Organization. Geneva, Switzerland: WHO; 2008. Guidelines for the programmatic management of drug-resistant tuberculosis: emergency update 2008. WHO/HTM/TB/2008.402. [Google Scholar]
- 10.Nkengasong JN, Nsubuga P, Nwanyanwu O, et al. Laboratory systems and services are critical in global health: time to end the neglect? Am J Clin Pathol. 2010;134:368–373. doi: 10.1309/AJCPMPSINQ9BRMU6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Petti CA, Polage CR, Quinn TC, Ronald AR, Sande MA. Laboratory medicine in Africa: a barrier to effective health care. Clin Infect Dis. 2006;42:377–382. doi: 10.1086/499363. [DOI] [PubMed] [Google Scholar]
- 12.Chiang C-Y, Van Deun A, Trébucq A, Heldal E, Caminero JA, Aït-Khaled N. Treatment of multidrug-resistant tuberculosis: defi nition of the outcome ‘failure’. Int J Tuberc Lung Dis. 2011;15:4–5. [PubMed] [Google Scholar]