Quality of TB services assessment: The unique contribution of patient and provider perspectives in identifying and addressing gaps in the quality of TB services

Abstract

Ambitious efforts to detect and treat tuberculosis (TB) are required to reduce the burden of disease in low resource settings, and the provision of high quality TB services is critical to reaching global TB targets. The quality of TB services assessment (QTSA) is a facility-based approach aimed at identifying gaps in TB services and prioritizing interventions to improve care across multiple countries with high TB burden. Randomly sampled facilities are assessed with standardized instruments to collect data on structures, processes, and outcomes of TB care, with adaption for local diagnostic and treatment algorithms. The sampling strategy is modified to ensure representation of all levels of the health system where TB services are provided, as well as inclusion of private sector or other facility types relevant to the context. Instruments include a facility audit, provider and patient interviews, and a review of TB registers. A recent QTSA in the Philippines generated important data on provider and patient perspectives on quality of care, showing that providers are more likely to report that they counseled current TB patients on key aspects of TB diagnosis and treatment than patients are to report having received the information. These comparisons highlight areas where refresher training or interpersonal communication and counseling skills may be needed.

1. Introduction

Tuberculosis (TB) is the world's leading cause of death due to infectious disease. The World Health Organization (WHO) reports that 64% of an estimated 10 million TB cases are detected and treated each year, leaving 3.6 million cases with either no care at all, sub-optimal care for which the quality of services is unknown, or adequate care but not reported to National TB Programs (NTP). [1] Following the first United Nations High-Level Meeting on the Fight to End TB (UN HLM), held in September 2018, there is renewed emphasis on improving TB case detection, TB prevention, and ensuring timely care for all people with active TB disease. High burden countries have committed to ambitious treatment targets intended to reduce morbidity and mortality due to TB and interrupt the chain of transmission; the ultimate goal of the UN HLM commitments is to accelerate progress towards the elimination of TB as a public health challenge. [2] The UN HLM commitments include treatment targets for drug sensitive TB and drug resistant TB (DR-TB), as well as specific targets for pediatric case finding and TB preventive therapy among contacts of confirmed TB cases and people living with HIV/AIDS.

As the largest bilateral donor to TB programs throughout the world, the United States Agency for International Development (USAID) has an important role in supporting NTP in their efforts to achieve the UN HLM targets. Investments in high quality TB diagnosis and treatment services at all levels of the health system are an essential component of USAID's support to high burden countries. The first of four pillars underlying the US Government TB strategy is “improve access to high quality, patient centered TB, TB/HIV and DR-TB services.” [3] In order to ensure this pillar is in place as a foundation for the UN HLM targets, USAID recognized the need for detailed data on quality of TB services.

Thus, there is a need to measure quality of TB services in a systematic way across high burden countries where USAID provides financial and technical support to NTP. The approach to measuring quality of TB services is conceptualized according to Donabedian's framework of structure, process, and outcome. [4] Structure refers to the resources available at a health facility, for example, equipment, reagents, and other materials needed to provide services. Process is defined as the interaction between patients and providers. Finally, outcomes are the consequences of the services provided, for example, a confirmed TB diagnosis or treatment initiation and completion. The quality of TB services assessment (QTSA) was developed to provide detailed data aimed at identifying gaps in quality of care that should be prioritized by NTP and donors such as USAID, so that investments are targeted for maximum effect. Additionally, QTSA includes the perspective of current TB patients and providers, which are critical to ensuring quality of care. The objective of this paper is to describe the QTSA methodology and provide an example of how it can be used to identify and prioritize gaps in the quality of care through a case study in the Philippines.

2. Overview of TB quality of care measurement methods

There are a variety of methods which have been used to measure quality of health services in settings with high TB burden. Most are broad analyses and are not specific to TB. Methods generally include one or a combination of a facility audit, direct observations, provider interviews, and patient exit interviews.

The Service Provision Assessment (SPA), which is part of the USAID-funded Demographic and Health Surveys Program, assesses health facilities to measure healthcare availability and readiness. [5] Provider and patient interviews are also conducted. However, even though providers are asked about their training on and provision of TB services, patient exit interviews are only conducted with those visiting the antenatal care clinic, family planning services, or with a sick child. Therefore, no comparisons can be made between patient and provider perceptions of their interactions regarding TB.

The Service Availability and Readiness Assessment (SARA) [6] was developed jointly by USAID and the WHO to build upon the SPA, and is facilitated by the WHO. The SARA focuses on service availability and service readiness of specific domains, including TB. The measures related to TB are assessments of whether specific services such as diagnostics and drug regimens are available or not and training of healthcare providers. These are assessed using an in-charge interview and direct observations. There are no patient interviews conducted and so a key component, the patient perspective of quality of care, is missing.

A third methodology is the Quality of Care Through the Eyes of the Patient (QUOTE), which was developed by the USAID-funded TB Control Assistance Program to TB. [7] As the name implies, this tool includes a strong focus on the patient's perspective of quality of care. Quality is measured using nine dimensions: communication and information, professional competence, availability of TB services, affordability, patient provider interaction and counseling, support, TB/HIV relationship, infrastructure, and stigma. Patients are involved from the beginning of the process including priority rankings of the dimensions of quality and as interviewees. The strength of the QUOTE method is its focus on the patient perspective, however, without a simultaneous facility audit or provider interview, there is no context available for the patient perspective and no comparisons with provider perspectives can be made.

Finally, standardized patients have been used to assess quality of clinical practices related to TB care in urban settings in India. A validation study by Das et al. describes the process by which 17 individuals were trained to present with specific symptoms and treatment seeking scenarios at private providers with different professional qualifications in Mumbai and Patna [8]. The standardized patients reported data on 250 interactions with providers, with only 21% of reporting correct procedures for the scenario they presented to the clinician, demonstrating a wide gap between training and practice. The authors concluded that use of standardized patients is feasible, successful and that this approach yields useful insights about the quality of TB services and allows for comparisons between different types of providers.

3. Why QTSA? Methods and analysis

The QTSA is a survey of a random sample of health facilities providing TB services using standardized data collection instruments to gather information on key aspects of screening, diagnosis, and treatment services as well as supporting environmental or contextual factors. The multi-stage sampling procedure is adapted to provide a representative sample of facilities from various levels of the health system for the desired geographic area covered by the survey, which depends on the preferences of key stakeholders and the availability of funding. The survey team invites a sample of three to five TB patients per facility who are 15 years and older, who have been on treatment for more than two weeks, and who are present on the day of the survey to participate in an interview. For facilities with a low volume of TB diagnosis and treatment services, this sampling approach may include all patients who present on the day of the survey. Facility in-charges and staff with responsibility for providing TB screening, diagnosis, and treatment services are also included in the survey.

The data collection instruments include a facility audit, register review, and patient and provider interview guides. The QTSA methodology and instruments were initially piloted in a subset of local government areas (LGAs) in Nigeria in collaboration with the USAID mission and NTP. The materials were revised after this pilot in preparation for implementation at national level in the Philippines and there are multiple QTSA surveys planned for high burden countries. The survey is designed and implemented by MEASURE Evaluation, a USAID-funded project, in partnership with a local research organization (LRO) and with the NTP, USAID mission and other local TB stakeholders involved in the planning and analysis stages.

During the planning stage, the generic data collection instruments are adapted to the local context by aligning items related to TB services to NTP algorithms. Additionally, some countries request more detail on specific elements of TB care, for example, detection and treatment of childhood TB, contact investigation practices and clients’ experiences of stigma. While the generic set of QTSA tools includes a standard set of variables collected across countries, with a corresponding data analysis plan, they are flexible enough to accommodate country specific needs. Fig. 1 is a visualization of the QTSA tools with linkage to the key elements of quality of services. Supplementary Tables 2 and 3 provide the questions included in the provider and patient questionnaires, respectively.

Fig. 1.

Overview of assessment tools.

Overview of the four tools used during a QTSA assessment and the information collected by each one.

Table 2.

Patient characteristics.

	Overall 560	Hospital 86 (15%)	Non-hospital 474 (85%)	Urban facilities 219 (39%)	Rural facilities 341 (61%)
Sex
Male	61%	62%	61%	57%	64%
Female	39%	38%	39%	43%	36%
Age
15–24	12%	12%	12%	14%	10%
25–34	17%	17%	16%	23%	13%
35–44	18%	27%	17%	19%	18%
45–54	18%	17%	19%	13%	22%
55–64	19%	17%	20%	21%	18%
65+	16%	9%	17%	10%	20%
Average age in years (range: 15–88)	46	44	47	43	49
Marital Status
Married	54%	53%	54%	43%	62%
Never married	24%	24%	24%	33%	19%
Widowed/divorced/separated	13%	13%	14%	12%	14%
Currently living with a partner (unmarried)	8%	9%	8%	12%	6%
Residence (NR = 1)
Rural	52%	52%	52%	16%	75%
Urban	48%	48%	47%	84%	24%
Highest level of completed education (NR = 1)
Primary/elementary or less	27%	15%	29%	21%	31%
Secondary/high School	44%	38%	45%	42%	45%
Post-secondary/technical/vocational	29%	45%	26%	37%	23%
Employment status (NR = 8)
Unemployed	44%	43%	44%	43%	45%
Employed (full or part time)	26%	27%	26%	32%	22%
Self-employed	21%	20%	21%	12%	27%
Retired	5%	6%	5%	6%	4%
Student	3%	5%	2%	5%	1%
Average monthly household income (PHP) (NR = 35)
0–5000	49%	45%	50%	40%	55%
5001–10,000	27%	27%	27%	21%	30%
10,001 and above	18%	24%	17%	28%	11%
Current smoker	7%	6%	8%	15%	3%
TB diagnosis self-reported (imputed)
Drug susceptible	53% (77%)	76%	77%	60%	88%
Drug resistant	10% (13%)	24%	10%	25%	5%
Unknown	37% (10%)	0%	12%	16%	7%
Phase of treatment
Intensive	40%	34%	41%	41%	40%
Continuation	28%	27%	28%	29%	26%
Unknown	32%	40%	31%	29%	34%

Abbreviations: NR = No response; PHP = Philippine Pisos.

Table 3.

Bivariate statistical analysis of patient responses.

	The patient was more likely to state that information on the following was shared with them by a health worker
	How the disease is spread	Taking medicines regularly	Completing treatment	Side effects	What to do if they have side effects	Treatment duration
The patient talks to the same health provider every time they visit the facility	*			**	*	**
Health providers usually explain things in a way the patient can understand	**	***	***	**	***	***
Health providers give the patient a chance to ask questions about anything that concerns them	*			***	***	***
The health providers listen carefully to the patient	***	***	***	*		***
The patient has enough time to discuss their health needs with the health providers				***	***	*

*p < 0.1, **p<0.01, ***p<0.001.

The multiple facility-based data sources provide a unique opportunity to identify specific gaps in the quality of care, for example, by comparing client and provider perspectives on the quality of care and triangulating with availability of commodities and equipment. Data analysis consists of a standard set of calculations based on agreed-upon indicators to identify gaps in service delivery, including the percentage of each facility type with specific equipment, commodities, and services needed to screen, diagnose, and treat TB, as well as perceptions of providers and clients on the quality of services. Similar to the sampling strategy, the analysis plan can be adapted to the needs of each NTP, depending on existing concerns regarding quality of care or specific questions they have about client and provider perspectives. Overall, QTSA data analysis is aimed at identifying and prioritizing gaps in quality of care; the results are presented at stakeholder workshops to engage all partners in interpretation of the data and prioritization of next steps. Although originally designed to support USAID missions, implementing partners, and host NTPs, the vision for QTSA is for broader adaptation and use through high burden settings or in any setting where quality of TB services is of concern to stakeholders. For example, the tools have been shared with research and implementing partners in non-USAID supported countries to support data collection on quality of TB services.

4. QTSA in Philippines: a case study

According to the WHO, Philippines is a high burden country for TB with an estimated 581,000 incident cases in 2017. It is also considered a high burden country for DR-TB, with 27,000 estimated rifampicin-resistant cases. [1] Historically, the Philippines has been a priority country for the USAID TB Program, receiving approximately $12 million per year for the past 6 years; the range of annual investment during the same time period is $1.5 to 13 million per year. On April 24, 2019, the Philippines Department of Health and the WHO declared an “all out war” on TB, recommitting the country to treating 2.5 million people with active TB by the end of 2022 in line with the UN HLM target set in September 2018. [9] One of the three key strategies for reaching this target is massive screening, diagnosis, and treatment, which will provide access to improved technologies and treatment regimens for all forms of TB.

Currently, the NTP is implementing the Philippine Strategic TB Elimination Plan (PhilSTEP) from 2017 to 2022 [10]. PhilSTEP emphasizes access to quality services, for example, with certification of health facilities for compliance with national standards for TB care, implementation of quality assurance procedures in laboratories, and eliminating stockouts of anti-TB medication and supplies needed for TB services. The plan reiterates the importance of integrated, patient-centered care throughout the health system and generation of support for TB services across all sectors and at the regional and local levels. Given the ambitious scope of PhilSTEP, the NTP supported the implementation of the QTSA to provide baseline data on the availability and quality of TB services, which can be used to identify gaps and prioritize interventions needed to ensure screening, diagnosis, and treatment are in place and support implementation of the plan. Additionally, the private sector is an important provider of TB diagnosis and treatment services in the Philippines, and the USAID TB Division wanted to use QTSA among private sector providers. Finally, the NTP was in agreement with QTSA objectives and wanted to use the data to inform their strategy for meeting HLM targets.

The ethics review for this QTSA was conducted and approved by the John Snow, Incorporated Institutional Review Board (IRB) in the United States and the Asian Eye Ethics Review Board in the Philippines. Additional IRB submissions were required for two hospitals included in the study and the Philippines Statistics Authority cleared the QTSA design. All clients and providers were read an informed consent statement describing their rights as participants and guarantee of confidentiality; no personal identifying information was collected and neither providers nor clients can be linked back to specific facilities.

4.1. Sampling

A total of 202 health facilities (public and private) from the NTP network providing TB and TB-related services such as diagnosis, care, and treatment were randomly selected using a multistage sampling procedure. The first stage involved stratifying the 17 regions into high, medium, and low categories based on the incidence and prevalence of TB and then randomly selecting six regions with two regions selected from each of the high, medium, and low categories. At the second stage, three provinces or highly urbanized cities were selected from each region and lastly, a sample of about 10 facilities per province/highly urbanized city within the network of NTP facilities was selected. Supplemental Table 1 in the appendix provides details on facility types and characteristics.

Table 1 provides summary data on provider characteristics. The assessment included 435 TB service providers of which the majority were female (85%). Most of the providers were attached to non-hospitals (86%) and worked in rural facilities (68%). About two thirds had a Bachelor's degree (68%) and 16% had an educational level lower than a Bachelor's degree. Providers with a higher educational level were found to be working at the hospital while those with a lower level of education work at non-hospital facilities. Almost half of the providers were registered nurses (49%) who predominantly work in hospitals, and a quarter were rural health midwives (25%). Only 9% were medical doctors. Overall, 70% of those interviewed were the TB focal person at the facility.

Table 1.

Provider characteristics.

	Overall 435	Hospitals 59 (14%)	Non-hospitals 376 (86%)	Urban facilities 140 (32%)	Rural facilities 295 (68%)
Sex
Female	85%	76%	86%	83%	85%
Male	15%	24%	14%	17%	15%
Highest Level of Schooling
Diploma/associate degree/other	16%	2%	19%	9%	20%
Bachelor's degree	68%	83%	65%	66%	68%
Master's degree	10%	12%	10%	19%	6%
Doctorate	6%	3%	6%	6%	6%
Occupation
Registered Nurse	49%	73%	46%	62%	43%
Rural Health Midwife	25%	2%	29%	15%	30%
Medical Technologist	12%	15%	11%	6%	14%
Medical Doctor	9%	7%	9%	10%	8%
Barangay Health Worker	2%	0%	2%	2%	1%
Other	3%	3%	3%	4%	3%
TB Focal Person	70%	73%	69%	72%	69%

A total of 560 patients were interviewed as part of the assessment; however, five patients did not complete the survey. On average, slightly less than three patients were interviewed per facility, almost two-thirds of whom were male (61%) while a little more than half of the sample was married (54%); the mean age was 46 years (Table 2). Respondents were almost equally distributed across all age groups and 52% lived in rural areas. Not surprisingly, urban dwellers were more frequently treated at urban facilities and rural dwellers treated at rural facilities. Nonetheless, 16% of the patients treated in urban facilities came from rural areas and almost a quarter of the patients attending rural facilities were urban dwellers.

Almost three fourths of the patients were educated beyond the primary school level and 44% had a secondary school degree. Patients with the highest level of education were more often seen at a hospital while those with the least education were seen at non-hospital facilities. When asked about their employment status, the percentage of patients who responded that they were employed (47%) was roughly the same as the percentage of those who were unemployed (44%). About half of the patients had an average monthly household income below 5000 PHP (equivalent to 95 USD). Those with the highest monthly income were frequently seen at urban facilities.

4.2. Analysis

Although the QTSA includes many variables related to quality of care, the Philippines case study focuses on patient and provider perspectives on quality of care, as this aspect of the methods differentiates the approach from other surveys. Observing the differences in response in these two groups provides unique insights into gaps in the quality of services and can be used to identify training and supervision needs. For example, if providers consistently score themselves much higher on the type of information provided to clients than the clients report, this points to the need for additional training or supervision to ensure correct information on TB treatment, infection control, and other topics is consistently provided to clients.

Fig. 2 shows the percentage of providers and clients who responded that they provided or – in the case of the client, were provided – information on specific topics regarding TB treatment, side effects, infection control, and contact tracing. These are key topics that providers are expected to cover in detail as part of counseling for confirmed TB patients undergoing treatment at their facility. The percentage of providers responding that they covered the topics is displayed on the left side, while the percentage of clients responding (unprompted) that they heard information about the topic is displayed on the right side. The line between the two columns shows the differences in percentage reporting the topics covered in counseling. For example, while 77% of providers reported that they discussed duration of TB treatment, only 33% of clients reported knowing how long treatment would last. Likewise, 76% of providers reported that they discussed the importance of taking medications regularly for the full course of treatment, while only 43% of clients said they had received this information. The analysis shows that providers consistently reported having covered basic TB information more often than patients reported receiving the information during counseling.

Fig. 2.

Comparison of patient vs. provider responses related to patient-provider interactions.

Percent of providers (n = 330) or DS-TB patients (n = 428) who stated that the following topics were shared during patient-provider interaction.

To further explore the reasons why providers and clients may report differences in information covered during counseling, a series of items related to interpersonal counseling and communication (IPCC) skills were analyzed. The hypothesis is that when providers demonstrate use of IPCC skills, the client may be more receptive to the information provided during counseling and more likely to remember details. For example, clients were asked whether or not they talk to the same providers at every follow-up visit and if the providers usually explain things to them in a way they can understand. We compared recall of topics to specific elements of IPCC to determine if there were any statistically significant relationships, using Chi-square analysis. Generally, where clients reported lower levels of IPCC, their recall of key topics covered in counseling was lower. Table 3 provides a summary of findings related to this analysis.

5. Discussion

The QTSA is a comprehensive yet flexible approach to assessing the availability and quality of TB services at different levels of the healthcare system. The methodology yields actionable information that can be used to prioritize interventions needed to ensure that TB screening, diagnosis, and treatment services are available. The value-added of the QTSA is the inclusion of both provider and client perspectives on quality, which can be used to identify the need for specific interventions.

The findings from the Philippines point to a need for improved communication and counseling skills, both in terms of the quality of information provided to clients and the way the information is delivered. Possible interventions to address this gap include pre-service, in-service, or on-the-job training to introduce or refresh providers on IPCC, as well as integration of IPCC in the general curriculum used to train providers on TB diagnosis and treatment, and what information should be provided to patients. Future data collection could include repeat QTSA or a standardized patient approach to identify any persistent information gaps on the part of providers and clients. Additionally, more detailed data on the duration of assignment and day to day availability of the TB service provider in facilities may be required to identify where IPCC or additional supervision is a promising intervention, as staff rotation and turnover affect the availability of the TB focal point on a day to day basis.

The planning and implementation of the QTSA require a high level of buy-in from all stakeholders and commitment of financial and human resources in order to proceed smoothly. The exercise itself can provide opportunities for capacity building of the NTP and its partners and lead to future investments in quality assessments, as planned in the Philippines. One success story from this case study is the engagement of the LRO and their partnership with the NTP; a follow-up QTSA is planned after quality improvement activities have been implemented. Additionally, the NTP is using the results to update standard operating procedures that will be issued to all facilities that provide TB services. To date, QTSAs have been implemented in Nigeria and the Philippines, and planning is underway to conduct the exercise in Uganda, Ethiopia, and Afghanistan. Each country will generate lessons learned and insights that can be used to refine the methodology and tools.

6. Limitations

The QTSA provides detailed data on quality of services in a standardized way; at the same time the approach has several limitations to consider. First, although the flexibility of the approach is a strength in terms of responsiveness to NTP needs, differences in sampling across countries to address the context of health structures limit opportunities for multi-country analysis and comparison. Related, the inclusion of private sector facilities can be challenging due to lack of interest or a complete sampling frame, which limits comparisons between public and private sector facilities. Second, QTSA includes the perspectives of clients who are already on treatment, which biases the conclusions towards those with access to services; the insights of those who are not seeking services at all (perhaps due to perceptions about quality or how they will be treated at a health facility), are not represented. Third, the phrasing of two questions did not allow for direct comparison of provider and client perspectives on issues related to quality of care. For example, providers were asked if they discussed “TB and TB treatment, including duration and dosage” while patients were asked if the provider discussed “how long your treatment would last and how to take your medicines.” Although intended to capture the same concept, it is possible that patients interpreted “how to take your medicines” to refer to taking medicine with food or water or another aspect of administration other than dosage. A second item asked of providers emphasizing the “importance of taking medicine regularly for the full course of treatment” was phrased “importance of taking medicine regularly and completing treatment” in the patient questionnaire. The questionnaire will be revised in future QTSAs to ensure that these questions are consistently phrased to avoid possible misinterpretation.

7. Conclusion

The QTSA is a standardized yet flexible approach for measurement of quality of TB services at different levels of the health system. It provides important information on provider and client perspectives, as well as a broad range of other data, which can be used to inform NTP strategies and prioritization of actions to improve services.