A quality assessment tool for tuberculosis control activities in resource limited settings

Abstract

Tuberculosis (TB) is a significant problem, infecting nearly 9 million new patients per year and killing about 2 million a year. The primary means with which to affect TB globally are to decrease transmission locally, mainly by effective identification, diagnosis, and treatment of infectious TB patients. Therefore, quality assurance of TB control efforts at the local level is essential. This study describes the creation of a data extraction tool for retrospective chart review based on the International Standards for TB Care, 2009 for the assessment of TB control programs located in resource limited settings. The tool was field tested at a rural mission hospital in central Kenya. Results were used by host site staff to develop a quality improvement plan. The process prompted revision of the tool to clarify questions and answers. This is a tool that can be used in resource limited settings for data collection to assess the quality of TB care and to inform the design, implementation, and further assessment of future quality improvement initiatives.

1. Introduction

Tuberculosis (TB) is a significant global infectious disease, infecting 9.4 million patients in 2009 and nearly 14 million people living with the disease worldwide¹. The World Health Organization (WHO) Stop TB Strategy set targets to half the world prevalence and reverse the incidence of TB by 2015 and to eliminate TB as a public health threat by 2050². Development of serious complications such as extreme poverty, the HIV epidemic, and the emergence of drug resistance have made TB control incredibly difficult in many settings^3,4. Kenya is one of the 22 WHO defined High Burden Countries where 80% of the world's burden for TB exist^1,5. Kenya experienced an increase in detection of TB cases from 51 to 320 per 100,000 people between 1987 and 2004, with an average annual increase in cases of 7% over the past 10 years⁶. This rapid expansion of TB in Kenya is largely due to the HIV epidemic, poverty, and urbanization. In settings such as Kenya, quality TB care is essential for TB control.

Effectively, TB control measures are limited by the quality of care provided at the local level; care such as proper case detection and treatment of infectious cases⁷. However, it is apparent that many providers in resource limited settings often deviate from known quality standards in TB care resulting in delayed diagnosis, treatment with ineffective regimens, and decreased patient adherence to therapy^8,9,10. Guidance for local programs in quality improvement is lacking. Most documents about quality address National TB Program (NTP) performance ^{11,12,13,14,15}. Furthermore, published quality improvement projects in resource limited settings tend to describe large scale projects affecting multiple sites at the national or district level, such as demonstrated in Vietnam, Bolivia and Bangladesh^16,17,18. The current top down approach to improve TB care through national policies, guidelines, and initiatives¹⁹, although important, leaves local programs with little to effectively evaluate their own progress toward quality TB care.

Tools and resources are needed for quality improvement in local TB programs. Retrospective chart review is a time honored method for data collection in quality assessment and improvement work²⁰. Recently, a chart audit tool for retrospective analysis of tuberculosis deaths to assess missed opportunities in diagnosis and care was published for use in resource limited settings²¹. The one page tool was used mainly on deaths of inpatients with TB. However, there is no comprehensive tool for quality assessment of TB control activities. The purpose of this study is to develop a simple tool that assesses program function in diagnosis, treatment, comorbidity care, and public health in resource limited settings that meets appropriate criteria for program evaluation material as usable, feasible, ethical, and accurate²².

2. Methods

2.1 Tool Development

An extensive search of international literature for standards and guidelines for quality tuberculosis care in resource limited settings was conducted and reported elsewhere²³. As a result, the International Standards for TB Care was identified as the most comprehensive source of evidence based measurable quality indicators that could be applied to local programs. The 21 standards identified in the ISTC were used to develop a logic model to describe appropriate TB care. The logic model informed the identification of quality indicators that seemed readily measurable by chart audit given available host site documentation. A questionnaire was designed to gather information about each indicator. Specific answer choices were pre-determined based on common terms reported in the literature²⁴.

2.2 Field Testing

The field site was chosen based on mutual interest and agreement between the institutions. The AIC Kijabe Hospital, a rural mission hospital located above the Rift Valley north of Nairobi, has an active TB control program treating hundreds of patients every year for TB. All eligible patients diagnosed with TB are enrolled in the TB program and recorded in the TB treatment register, maintained by the Kijabe TB officer. Patients are monitored by-weekly for the first two months of treatment and then return monthly until treatment is complete. Kijabe adheres to the national recommended treatment regimen of fixed dose medications provided for by the Kenyan Ministry of Health. Pill counts are done at every visit to assess for adherence. Patients who default are traced by the hospital's HIV program community health workers, especially when sputum positive at diagnosis. Records are maintained both in the patient file and in the TB register.

The inclusion criteria for chart review were as follows: 1) the patient was registered in the TB control program for the treatment of TB from January through April 2009 and 2) documentation (i.e. a patient chart) could be accessed for the patient. The time period was chosen so that patient progress and outcomes could be reviewed up to one year after enrollment. The charts were collected from the Kijabe Main Hospital as well as four outlying clinic locations. The chart review and data collection took place in August of 2010. De-identified information was entered directly into an excel database. The excel database was then decoded and uploaded into SPSS v. 19 for analysis. For simple interpretative purposes, proportions were calculated for each item answer. Treatment success rates were calculated as a proportion of smear negative, smear positive, or retreatment patients who completed treatment successfully to total number of patients in that category²⁵.

To further evaluate the use of this tool in resource limited settings, it was necessary to determine if the collected data could be used to develop a plan to improve the quality of care. The collected data was presented to Kijabe TB and HIV care staff in a focus group setting in June of 2011and was discussed in a roots-cause analysis type format. Priorities problems and identifiable solutions were discussed. Staff were asked to review the tool and data in terms of feasibility, propriety, usability, and accuracy in their setting.

2.3 Tool Revision

After data collection and focus group presentation, the tool was revised and edited to improve questions and answers to better represent the data and objectives of the chart review.

2.4 Ethical Consideration

Ethical approval for this study was sought through both the University of Texas Medical Branch Internal Review Board and the AIC Kijabe Hospital Ethics Board. Information was collected in a de-identified database. Any personal identifying data used to identify study cases was maintained separately and subsequently destroyed after data collection.

3. Results

3.1 Tool Description

Table 1 shows a logic model of quality indicators based on the ISTC that could potentially be assessed on retrospective chart review. Based on this, the data extraction tool was a series of 45 questions assessing baseline characteristics of the patient population, indicators of diagnosis, treatment, management of HIV and comorbidities, and public health measures. The data can be broken down to input data, or cohort characteristics, process data, or operational characteristics, and outcome data, including both patient and program outcomes.

Table 1.

TB care quality indicators based on the ISTC

Inputs	Process	Outcomes
Population characteristics	Program/Operational Characteristics	Patient/Program Outcomes
Gender	Diagnosis:	Patient outcomes:
Age	Sputum smears	One year status
Location	Specimens for extra- pulmonary	Cure
New Cases:	Use of CXR	Program outcomes:
Smear positive	Smear negative diagnosis	Treatment success rates
Smear negative	Diagnosis in children
Extra-pulmonary	Treatment:	Documentation agreement
Retreatment cases:	Treatment regimen
Relapse	Treatment of drug resistant TB
Treatment after failure	Sputum collection and follow up
	Adherence strategy
Treatment after default	Drug resistance assessment
	Maintenance of written record
HIV status	HIV and Co-Morbidities:
Drug resistance	HIV testing and referral
	Evaluation for ART
	Prophylaxis with INH
	Co-trimoxazole prophylaxis
	Co-morbidity assessment and referral
	Public Health:
	Contact investigations
	INH prophylaxis for children
	Infection control

ART: Antiretroviral therapy, CXR: Chest x-ray, INH: Isoniazid

3.2 Data Collection Results

Kijabe Hospital enrolled 1076 patients for TB treatment in 2009. Approximately two hundred patients were enrolled in the AIC Kijabe program during from January through April in 2009. Of those, 106 charts were tracked, identified to meet inclusion criteria, and reviewed using the data extraction tool, representing about 9.9% of the patients enrolled in 2009. The cohort characteristics are shown in Table 2 as input indicators. Most tuberculosis patients were HIV positive (81.1%). Interesting, half of the TB cases in this cohort were smear negative, with only 11.3% (n=12) smear positive cases identified. Most cases were new patients (85.8%), or never previously treated for TB.

Table 2.

Cohort description

Indicator	Kijabe
Female	54.7 %
Male	45.3 %
Adult cases	86.8 %
Pediatric (age less than 15)	13.2 %
Diagnosis:
Smear positive	11.3%
Smear negative	50.0 %
Smear unknown	21.7 %
Extra pulmonary	17.9 %
Treatment Category:
New (Treatment naïve)	85.8%
Relapse	2.8 %
Treatment after failure	1.9 %
Treatment after default	8.5%
HIV positive	81.1 %

Table 3 presents selected process indicators. Most pulmonary TB patients (74.7%) at Kijabe received two sputum smears for diagnosis. However, only 32.1% of those diagnosed with smear negative pulmonary TB met all requirements for smear negative diagnosis; trial of broad spectrum antibiotics, chest radiography consistent with TB, and two negative sputum smears. A little more than half, 58.3%, of patients with smear positive TB had follow up sputum within the recommended time frame. Of those recommended for drug resistance testing, 26.7% were able to have culture or drug susceptibility testing (DST) or both. Most HIV positive patients (87.2%) were on antiretroviral therapy at some point during treatment for TB. Nearly all HIV positive patients received co-trimoxazole prophylaxis (97.7%). Lastly, 29.2% of patients had clear documentation of assessment for other co-morbid conditions such as malnutrition or substance abuse.

Table 3.

Selected process indicators

Indicator	Kijabe
Sputum smears completed for PTB	74.7 %
Smear negative patient meeting total requirements for diagnosis	32.1 %
Smear positive PTB with follow up sputum within recommended time frame	58.3 %
Recommended patients who received drug resistance testing	26.7 %
HIV positive patients on ARV therapy during TB treatment	87.2 %
HIV positive patients on cotrimoxazole prophylaxis	97.7 %
Patients with clear assessment for other comorbid conditions	29.2 %

Table 4 reports patient and program outcomes. Most patients completed treatment within the year (84%), while 4.7% were treatment failures and 9.4% were lost to follow up. A small percentage of patients were transferred out. Kijabe showed a smear positive treatment success rate of 60%. The smear negative success rate was 87.2%, and the success rate among retreatment cases was 80%. Information presented in the patient charts were considered accurate as represented. The program register, used to identify enrolled patients, also collects data on date of diagnosis, follow up sputum, HIV status, and referrals, and is the primary means for reporting information to the National TB program. There was 77.4% agreement between the TB patient register and the patient charts.

Table 4.

Patient and program outcomes

Indicator	Kijabe
Patients completed treatment successfully in one year	84 %
Patients who failed treatment or had recurrent disease in one year	4.7 %
Patients who were lost to follow up or had unknown outcomes	9.4 %
Patients who were transferred out	1.9 %
Treatment success rate: new smear positive	60
Treatment success rate: new smear negative	87.2
Treatment success rate: retreatment	80

3.3 QI Focus Group Results

The Kijabe Hospital staff identified priority areas included smear positive follow up, sputum smears for diagnosis of all pulmonary TB patients, smear negative diagnosis, and co-morbidity assessment. The current data on these priority indicators is shown in Table 5, along with perceived causes and potential interventions discussed by staff and providers.

Table 5.

Focused indicators for quality improvement

Current data	Perceived causation	Potential Interventions
74.7% Sputum smears completed for pulmonary TB	Patients refuse or are unable to produce sputum.	Staff training- patient coaching to obtain proper sputum
	Patients present or bring sputum samples at times when the lab is unable to include them in the daily sputum analysis.	Lab coordination- timing of sputum processing in lab means that patients who come in the afternoon with sputum samples are disregarded.
		Link lab data with patient data: not all results get reported in patient file
58.3% smear positive patients had follow up sputum within recommended time frame	Community follow up is limited to patients co-infected with HIV.	Dedicated TB community health workers.
29.2% of patients had clear documentation of comorbidity assessment	Providers focus on HIV or are unaware of other significant co-morbidities.	Produce check-off sheet for co-morbidities.
		Train staff in OPD and HCC to ask about co-morbidities.
32.1% Smear negative diagnosis meet total requirements based on	Providers do not document all steps taken in diagnosis.	Continued staff training on importance of smear negative diagnostic criteria.
ISTC	Providers are weary of the use of antibiotics when TB is suspected, especially in HIV positive patients.	Produce diagnosis worksheet with check boxes for steps taken to clearly present diagnosis.

ISTC: International Standards for TB care, HCC: HIV comprehensive care clinic, OPD: Out-patient department, PTB: Pulmonary tuberculosis

It was agreed that tracing smear positive patients for follow up sputum samples had high priority due to the public health impact of smear positive infection in the community and that the program should focus on obtaining and training community health workers for that purpose. Providers further agreed to address obtaining sputum smears for all pulmonary TB suspects. Lack of obtaining sputum smears is potentially due to multiple patients denying the ability to produce sputum and to a problem with the timing of sample analysis in the laboratory. Staff training on how to coach patients to produce good sputum samples, emphasizing the importance of obtaining samples for all pulmonary patients, may help increase this number; as well as collaborating with laboratory personnel to address timing issues and results communication. Improving smear negative diagnosis with training and diagnosis check lists may have a significant impact overall since half of all pulmonary patients were considered smear negative. Lastly, it was agreed that co-morbidity assessment could be addressed as well with a simple check list in the charts and staff training.

3.4 Tool Revision

Testing this tool prompted revision to improve the document for further use. The original questions were converted from a multiple choice answer format to a check box format that requires no translation and can be directly entered into a database. The final tool includes more fill in the blank spaces for dates to be used in an analysis on the timing of treatment and follow up sputum results. Answer choices were simplified to reflect internationally accepted terms and to eliminate ambiguous terms such as “unknown”. Redundant questions were eliminated and information was consolidated into more pertinent questions. The result is a 27 question data extraction tool is available in supplementary data for this article.

4. Discussion

This study describes the development and field testing of a data extraction tool for use in quality improvement in tuberculosis care and control activities in resource limited settings. The tool was derived from the evidence based standards defined in the International Standards for TB Care and field tested at a rural TB control program in Kenya. The tool was easily implemented in this setting and results were accepted by staff to develop a quality improvement plan. The tool was considered feasible, ethical, useful, and accurate in terms of quality improvement by site staff.

This study has several limitations. The tool was tested at only one resource limited program. While considered appropriate for the field test site, testing at other locations is necessary to prove the tool can be applied in general to resource limited settings. The use of this tool has not been correlated with improvement in patient or program outcomes, and therefore its effectiveness to improve quality is yet to be known. Also, the sample size of the testing cohort was relatively small. Lastly, the use of retrospective chart review, although common in quality improvement literature, has limitation in itself; namely that recorded data may be inaccurate or incomplete and therefore lead to inappropriate conclusions about what is truly happening²⁰.

Given these limitations, the data extraction tool described here is a good start toward equipping local programs with tools for self-evaluation and quality improvement; a needed step in creating sustainable TB control programs that are effective at treating and eliminating TB from local communities. Compared to what is only recently available²¹, this tool makes a comprehensive assessment of TB control activities (addresses diagnosis, treatment, co-morbidity care, and public health), requires little clinical judgment to complete the data abstraction, has been vetted by TB program staff in a resource limited setting, and revised after field testing. This tool should be tested at other locations and adapted to fit the needs of individual sites. It can be used on its own or as part of a comprehensive quality improvement strategy. For programs with simple technology capability, the questionnaire can be adapted to an electronic format and used in retrospective analysis or even in prospective data collection to follow patient progress and outcomes over time. Valuable insights might be gathered by using this tool in a continuous quality improvement process and correlating patient and program outcomes with intervention implementation.

Attention to quality in patient care is important in every setting, especially when attempting to control devastating infectious diseases such as tuberculosis. The gap in what is done and what should be done can be closed by providers who implement the principles of quality improvement and have good tools and resources to do so.