Use of standardized patients to assess quality of tuberculosis care: a pilot, cross-sectional study

SUMMARY

Background

Existing studies on quality of tuberculosis care mostly reflect knowledge, not actual practice.

Methods

We conducted a validation study on the use of standardized patients (SPs) for assessing quality of TB care. Four cases, two for presumed TB and one each for confirmed TB and suspected MDR-TB, were presented by 17 SPs, with 250 SP interactions among 100 consenting providers in Delhi, including qualified (29%), alternative medicine (40%) and informal providers (31%). Validation criteria were: (1) negligible risk and ability to avoid adverse events for providers and SPs; (2) low detection rates of SPs by providers, and (3) data accuracy across SPs and audio verification of SP recall. We used medical vignettes to assess provider knowledge for presumed TB. Correct case management was benchmarked using Standards for TB Care in India (STCI).

Findings

SPs were deployed with low detection rates (4.7% of 232 interactions), high correlation of recall with audio recordings (r=0.63; 95% CI: 0.53 – 0.79), and no safety concerns. Average consultation length was 6 minutes with 6.2 questions/exams completed, representing 35% (95% confidence interval [CI]: 33%–38%) of essential checklist items. Across all cases, only 52 of 250 (21%; 95% CI: 16%–26%) were correctly managed. Correct management was higher among MBBS doctors (adjusted OR=2.41, 95% CI: 1.17–4.93) as compared to all others. Provider knowledge in the vignettes was markedly more consistent with STCI than their practice.

Interpretation

The SP methodology can be successfully implemented to assess TB care. Our data suggest a big gap between provider knowledge and practice.

INTRODUCTION

India accounts for a quarter of the nine million tuberculosis (TB) cases that occur worldwide, and for one million of the three million ‘missing’ cases.¹ India’s TB burden is exacerbated by fragmented healthcare provision through diverse providers, and an unregulated private sector that accounts for half of TB treatment.^{2, 3} The private sector includes qualified allopathic doctors (e.g. Bachelor of Medicine & Surgery [MBBS] degree), practitioners of alternative health systems (e.g. Ayurveda), and informal providers with minimal or no formal qualifications.

Data from India have raised concerns about the poor quality of medical care in general.^4–7 A systematic review has shown that in most Indian studies, less than half of providers knew to order microscopy testing for patients with TB symptoms, and less than a third knew the correct treatment regimen.⁸ Adherence to International Standards for TB Care was significantly lower (P<0.05) in the private compared to the public sector. Studies on patient pathways for TB have shown convoluted paths,⁹ with an average of three healthcare providers seen over 55 days from onset of symptoms to diagnosis and treatment.¹⁰

Studies of quality of TB care in India and elsewhere have relied on recall-based patient surveys, questionnaire surveys of knowledge, and prescription/chart analysis.⁸ These methods may not reflect actual practice.^{4, 6, 11–14} Consequently, standardized patients (SPs) are increasingly used in low-income countries to assess quality of medical care, as shown by recent studies from India and China.^5,13,6 Relative to other methods, data from SPs yield an assessment of provider practice that is free from observation bias, less vulnerable to recall bias than patient exit interviews, and more complete than medical records.^{14, 15} Further, SPs permit estimates of case detection rates since illnesses are fixed by design. Finally, because case presentations are standardized, the SP methodology allows for valid quality comparisons across different types of health care providers. We conducted a pilot validation study of the SP methodology for TB.

METHODS

Objectives

To validate the SP methodology for assessing the quality of TB care in the private sector using three criteria: (1) negligible risk and ability to avoid adverse events for providers and SPs; (2) low detection rates of SPs by providers, and (3) data accuracy across SPs and audio verification of SP recall.

Setting

The study was conducted in Delhi. An urban setting minimizes the risk of SP detection, since many patients are unknown to health care providers.

Study Population, Study Protocols, and Quality of Care Measures

First, TB tracer conditions (cases) were developed together with a group of international and Indian TB experts, using the Standards for TB Care in India (STCI) as the benchmark.¹⁶ The STCI itself is based on the International Standards for TB Care (ISTC).¹⁷ Although there are a large number of standards, the expert group focused on the most critical aspects of TB detection and treatment, and helped put together case-specific checklists of essential and recommended treatments (or expected correct case management). Second, 17 individuals were recruited to be SPs, and the medical representations of the tracer conditions were further enhanced with the psychosocial aspects of TB presentation in the community. These TB “scripts” were developed by social scientists with the help of the recruited SPs. All scripts were translated into Hindi, the vernacular. Third, a sample of providers was recruited and consented for the study from low and middle-income neighborhoods of Delhi. Fourth, SPs were randomly allocated to providers, and within an hour of each SP interaction, a structured questionnaire was used to extract information on what each provider had done. All SPs paid the providers their usual fee. The information obtained from SPs was compared to audio recordings. Fifth, participating providers were visited after the completion of all interactions to assess detection rates. During the same visit, providers were subsequently administered a medical vignette for one of the SP cases to assess provider knowledge.

Tracer Conditions

Four tracer conditions were developed (Table 1, with detailed descriptions in the Supplementary Appendix). These conditions approximate the pathways for TB care in the country. Each condition was scripted to make the diagnosis as obvious and uncomplicated as possible. For instance, in the case of SP1, appropriate questioning would reveal that the patient has had a cough for 2–3 weeks, produces sputum, has fever with night sweats, and has lost appetite and weight. According to STCI, this should lead the provider to suspect TB and order appropriate microbiological tests or refer for TB testing. Script development and SP training methods are described in the Appendix.

Table 1.

Standardized patient (SP) cases and expected correct case management

Case #	Case description	Case was designed to assess quality of care for a person who	Expected correct case management
SP1	Classic case of presumed TB with 2–3 weeks of cough and fever	Presents with presumptive TB, for the first time, to a private healthcare provider	Recommendation for sputum testing or chest X- ray or referral to a public DOTS center/qualified provider
SP2	Classic case of presumed TB who has had 2–3 weeks of cough and fever and a history of 1 week of broad-spectrum antibiotic (amoxicillin) treatment by another provider, with no improvement	Presents after an initial, failed (empiric) treatment with broad-spectrum antibiotics	Recommendation for sputum testing or chest X-ray or referral to a public DOTS center/qualified provider
SP3	Chronic cough with positive sputum smear report for TB from a public health facility	Presents with evidence of microbiologically confirmed TB	Either referral to a public DOTS center, a qualified private provider, or specialist or (in the case of a qualified private provider) initiation of treatment with standard, 4-drug first-line anti-TB therapy (HRZE regimen: isoniazid [INH], rifampicin, pyrazinamide, and ethambutol)
SP4	Chronic cough and a positive sputum smear report from a public health facility, and, if asked, history of previous, incomplete TB treatment, which would raise the suspicion of multidrug-resistant TB (MDR-TB).	Presents as a previously treated TB patient with recurrence of TB (i.e. suspicion of drug-resistance)	Recommendation for any drug-susceptibility test (culture/DST, line probe assay or Xpert MTB/RIF) or referral to a public DOTS center

Choice of Quality Measures

Quality of care was assessed through adherence to case-specific checklists of recommended care, the appropriateness of treatment, and the use of unnecessary and/or contraindicated treatment (e.g. steroids). The checklists for recommended care include only items that can be completed in low-resource settings and are in accord with the STCI.¹⁶ The recommended list of questions, and the subset that was considered ‘essential’ are shown in the Appendix. Expected correct case management, based on STCI, is shown in Table 1. In addition, we collected data on consultation duration and the cost of the consultation and medicines for each interaction. Finally, in 129 randomly selected SP interactions, digital recorders were used to assess accuracy of recall.¹⁸

Study Population

A convenience sample of 106 private healthcare providers practicing in outpatient settings was recruited in low- and middle-income areas of Delhi. High-income areas were excluded. All recruited providers were consented and informed that over the subsequent six months, they may receive someone who is not a real patient, and if they suspected any of their patients to not be genuine, they should write the name and date of the visit. SP cases were randomly assigned across 100 providers for a total of 250 interactions (details in the Appendix).

Post SP Interaction Protocols

Three to four weeks after the completion of SP visits, participating providers were visited and asked whether they suspected any fake patients at their clinics, and if so, what the presenting symptoms, age, and sex of the SP were. These data were used to assess detection rates. In addition, providers were asked to complete a medical vignette depicting exactly the same case as SP1 (2–3 weeks of cough). The vignette started with SP1’s opening statement, and the provider was asked to proceed exactly as he/she may with a real patient. This vignette was used to assess knowledge of diagnosis and treatment for SP1 and to compare knowledge with practice.

Data analysis

Following the compilation of data, the list of labelled medicines and prescriptions received by the SPs were coded by two independent doctors with expertise in TB (SS) and infectious diseases (RS), who then identified drugs as anti-TB drugs, antibiotics (other than TB drugs), or steroids. Among antibiotics, we specifically coded fluoroquinolones as a distinct category, given their ability to mask underlying TB. Other drug classifications were prescribed or dispensed (e.g. bronchodilators, cough suppressants), but not coded. Of all medicines that were dispensed in the clinic 54% were loose, unlabelled pills and therefore could not be coded. We employed two independent pharmacists to identify these pills and based on their assessments we could identify whether the medicines included at least one antibiotic. Because chemical analysis is expensive and could not be done, we report the fraction of interactions where an antibiotic was used among labelled and unlabelled pills separately.

Logistic regression was used to assess associations between provider qualifications and various components of case management for all four SP cases combined. These components were (a) an overall index of correct case management aggregated across all cases, (b) individual components of case management (referrals for all cases, recommendations for CXR or sputum tests for SP1 and SP2 cases, initiation of TB treatment for SP3 and SP4), and (c) the use of antibiotics and fluoroquinolones for all cases. Results are reported as adjusted odds ratios for MBBS versus non-MBBS providers (rationale provided in the Appendix). All regression models included the age and sex of the provider, the number of patients waiting at the time the SP entered the clinic (as an index of patient case load), and SP and case fixed-effects. The Appendix reports additional specifications with SP age, sex, height and weight instead of SP fixed-effects as independent variables. Both T-tests and logistic regression were used to assess the gap between provider knowledge as elicited through medical vignettes for SP1 and provider practice, as measured using SPs.

Ethical Approvals

This study obtained ethical approvals from McGill University in Montreal, and the Institute of Socio-Economic Research on Development and Democracy (ISERDD) in Delhi. Written informed consent was obtained from all recruited providers (including the use of audio recorders), and confidentiality was assured. The study sponsors had no role in the preparation of this manuscript, nor the decision to submit the manuscript for publication.

RESULTS

During the study, 100 providers received SP cases for a total of 250 SP interactions. Nearly all providers were male (97) with a mean age of 47 years (range 22 to 75). Of the 100 providers, 29 were doctors with an MBBS degree (qualified allopathic doctors); 40 held degrees in alternative systems of medicine; and the remainder (31) were informal providers with minimum or no qualifications. On average, 1.27 patients were waiting in the clinic at the time of the SP visit.

Detection Rates

Of the 93 providers who completed the detection survey and received SPs prior to the survey, eight providers reported 11 detections for an overall detection rate of 4.7% from a total of 232 interactions. Of the 11 detections, seven were reported to have occurred during the visit, although in none of the cases did the provider voice suspicion during the interaction with the SP, and four were reported after the completion of the interaction, primarily because the SP did not return for follow-up. In 14 cases, providers mistook real patients to be SPs.

Adverse Events for Providers and SPs

No providers voiced concern that participation in the study adversely affected them. Additionally, no financial losses were incurred by providers as the SPs paid normal fees to receive services, and there were no added inconveniences to the provider as SPs were trained to immediately step aside if there were an emergency in the clinic, demanding the provider’s attention. None of the SPs had any threats to their safety.

Data Accuracy using Audio Taping

Variation across SPs and across providers in quality of care using data from audio recordings versus structured recall questionnaires were highly correlated (the correlation coefficient between audio and recall was 0.63 (95% CI: 0.53 – 0.79).

Quality of Care

Overall Quality and Variation across Cases

An average SP interaction lasted 6 minutes (95% CI: 5.5–6.6), with a payment of 109 Rupees (approximately US$1.70). Providers completed, on average, 6.2 questions and/or examinations, representing 35% (95% CI: 33%–38%) of the essential checklist (Table 2 and Figure 1). SP1 and SP2 were the classic presumptive TB patients, and here, the use of chest X-rays, sputum tests and referral rates uncovered deficits relative to existing standards. Consequently, correct case management rates of 12% (95% CI: 6%–21%) were notably low for SP1 and increased only slightly to 17% (95% CI: 10%–27%) for SP2. Antibiotic use was high: Using labelled medicines, at least 64% of SP1 cases and 55% of SP2 (95% CI: 53%–74% and 43%–65%) cases received antibiotics with the fraction increasing to 88% (95% CI: 79%–94%) and 77% (95%CI 67%–85%), once unlabelled medicines were taken into account.

Table 2.

Main outcomes of standardized patient (SP) interactions for each SP case and all cases combined

	SP1 (N=75)	SP2 (N=75)	SP3 (N=50)	SP4 (N=50)	All (N=250)
Mean Time With Doctor (Minutes)	5.96 (5.19–6.73)	4.27 (3.62–4.91)	8.04 (6.10–9.99)	6.64 (5.49–7.79)	6.00 (5.45–6.56)
Mean Number of Questions and Exams	6.31 (5.56–7.05)	5.93 (5.13–6.73)	5.60 (4.57–6.63)	6.94 (5.68–8.21)	6.18 (5.72–6.64)
Mean Essential History Checklist %	0.36 (0.32–0.40)	0.32 (0.27–0.37)	0.46 (0.40–0.51)	0.30 (0.25–0.36)	0.35 (0.33–0.38)
Mean Cost of Consultation and Medicines in 2014 Indian Rupees	₹133.68 (107–160)	₹130.96 (107–154)	₹150.29 (101–199)	₹156.32 (116–197)	₹140.71 (124–157)
Mean Cost of Consultation and Medicines in 2014 U.S. Dollars	$2.14 (1.71–2.57)	$2.10 (1.72–2.47)	$2.40 (1.62–3.19)	$2.50 (1.86–3.15)	$2.25 (1.99–2.51)
Case Management
Proportion With Correct Case Management	0.12 (0.06–0.21)	0.17 (0.10–0.27)	0.38 (0.26–0.52)	0.22 (0.13–0.35)	0.21 (0.16–0.26)
Proportion Ordering Chest X-Ray	0.09 (0.05–0.18)	0.12 (0.06–0.21)	0.36 (0.24–0.50)	0.44 (0.31–0.58)	0.22 (0.18–0.26)
Proportion Ordering Sputum Test	0.04 (0.01–0.11)	0.07 (0.03–0.15)	0.06 (0.02–0.16)	0.06 (0.02–0.16)	0.06 (0.03–0.09)
Proportion Ordering DST or GeneXpert	0.00 (0–0.05)	0.00 (0–0.05)	0.02 (0.004–0.10)	0.02 (0.004–0.10)	0.01 (0.002–0.03)
Proportion Starting TB Treatment	0.00 (0–0.05)	0.00 (0–0.05)	0.14 (0.07–0.26)	0.08 (0.03–0.19)	0.04 (0.02–0.08)
Proportion Referring Case	0.03 (0.01–0.09)	0.01 (0.002–0.07)	0.26 (0.16–0.40)	0.20 (0.11–0.33)	0.10 (0.07–0.15)
Tests and Medicines
Mean Number of Lab Tests	0.25 (0.08–0.43)	0.35 (0.16–0.53)	0.82 (0.52–1.13)	0.98 (0.63–1.33)	0.54 (0.42–0.66)
Mean # Medicines Given or Prescribed	4.85 (4.52–5.18)	5.05 (4.75–5.36)	3.86 (3.15–4.57)	3.64 (2.95–4.33)	4.47 (4.22–4.72)
Proportion Giving Any Labelled Antibiotic	0.64 (0.53–0.74)	0.55 (0.43–0.65)	0.36 (0.24–0.50)	0.38 (0.26–0.52)	0.50 (0.44–0.57)
Proportion Giving Any Antibiotic (including unlabelled)	0.88 (0.79–0.94)	0.77 (0.67–0.85)	0.52 (0.39–0.65)	0.48 (0.35–0.61)	0.70 (0.64–0.75)
Proportion Giving Arty Fluoroquinolone	0.16 (0.09–0.26)	0.12 (0.06–0.21)	0.10 (0.04–0.21)	0.16 (0.08–0.29)	0.14 (0.10–0.18)

Notes: Correct case management defined as chest x-ray, sputum test or referral (SP1 and SP2) HRZE or referral (SP3) and drug-susceptibility test (DST), GeneXpert or referral (SP4). Labelled antibiotic use is a lower bound as only identified drugs are included. Any antibiotic use also includes cases where an unlabelled medicine was identified as an antibiotic by chemists who were specifically recruited to identify unlabelled medicines. Costs in 2014 Indian rupees (INR) and 2014 U.S. dollars at ($1 = INR62.27). Purchasing-power-parity adjustment would result in an exchange rate of INR16.76 per dollar. Costs reflect the fee paid at the time of the interaction to the provider plus prices for medicines. Costs do not include lab tests or other fees for procedures that SPs did not complete. 95% CIs in parentheses; we use the Wilson interval without continuity correction for binary variables.

Figure 1.

Major outcomes, stratified by standardized patient case

Notes: For each outcome, SP1 value is on the top; moving downwards, SP4 at bottom. Correct case management for these four cases were defined as:

• SP1 & SP2: Recommendation for sputum testing or chest X-ray or referral to a public DOTS center/qualified provider
• SP3: Either referral to a public DOTS center, a qualified private provider or specialist, or (in the case of a qualified private provider) initiation of treatment with standard, 4-drug first-line anti-TB therapy (HRZE regimen)
• SP4: Recommendation for any drug-susceptibility test (culture/DST, line probe assay or GeneXpert MTB/RIF), or referral to a public DOTS center

CXR: chest x-ray

DST: drug-susceptibility testing

GeneXpert: Xpert MTB/RIF test (Cepheid Inc, CA)

In the case of SP3 and SP4, although both were carrying lab reports that microbiologically confirmed a TB diagnosis, a significant fraction were recommended chest X-rays. Correct case management rates were higher for SP3, with 7 of 50 (14%, 95% CI: 7%–26%) started on TB treatment and 13 of 50 (26%, 95% CI: 16%–40%) referred to the DOTS center or qualified providers, but dropped again to 11 of 50 (22%, 95% CI: 13%–35%) for SP4, reflecting both the low use of drug susceptibility testing necessary for the work-up of a patient with recurrent TB, and low referrals to the public sector. Low correct case management rates reflect, in part, the low use of essential and recommended actions (Figure 2 and Appendix). In the case of SP1, for instance, although most providers asked about cough duration, auscultated the chest and measured temperature, few checked weight or blood pressure and a minority asked about blood in the sputum.

Figure 2.

Proportion of providers who completed history and physical examinations for SP1 cases (N=75 interactions)

Notes: SP1 is a standardized patient presenting as a classic case of presumed TB with 2–3 weeks of cough and fever. Each bar in the figure shows the proportion of providers who asked the corresponding question or completed the corresponding examination. For instance, 93% of all providers asked about cough duration and 76% of all providers auscultated the SP.

Individual pathways to diagnosis and treatment were complex. Figures S1, S2, S3 and S4 (Appendix) show the different actions that the four cases encountered. Notable is the high use of broad-spectrum antibiotics for SP1 and SP2, with quinolones used in at least 12–15% of all cases, and the use of steroids in at least 14% of all interactions for SP2. Referrals increased as we moved to SP3 and SP4, with some decline in the use of antibiotics and TB treatment initiation. Among the seven TB drug prescriptions that were given for SP3 cases, six were correct regimens.

Variation in quality across providers

Figure 3 shows that MBBS providers, as compared to non-MBBS, were more likely to correctly manage SP cases although they were less likely to refer but the latter was not statistically significant. For SP1 and SP2, MBBS doctors were more likely to order chest X-rays and sputum tests than non-MBBS practitioners. They were, however, equally likely to prescribe broad-spectrum antibiotics and were more likely to prescribe fluoroquinolones.

Figure 3.

Impact of provider qualifications on main standardized patient outcomes

Notes: Results are reported as adjusted odds ratios for MBBS providers (N=29) relative to non-MBBS (N=71), which includes practitioners of alternative systems of medicine, and informal providers with minimum or no qualifications. Correct case management is defined as a chest x-ray [CXR] or sputum test or referral for SP1 and SP2; as an HRZE regimen or referral for SP3; and a drug-susceptibility test [DST] or GeneXpert or referral for SP4. The antibiotics measure is a lower bound as only identified drugs are included. DST and GeneXpert are excluded from regression because the incidence rate is too low for statistical inference. Regressions are controlled for provider age, provider gender, and caseload on arrival of the SP and SP case and individual fixed effects. *** p<0.01, ** p<0.05, * p<0.1

Disaggregating by SP cases suggests that the MBBS advantage was most marked for SP1, where rates of chest x-rays and sputum tests were negligible for non-MBBS, but the correct treatment advantage also carried over to SP2 and SP3 (data not shown). Notably, for SP4, MBBS providers were less likely to correctly treat the patient as they were more prone to starting TB treatment rather than referring the SP or recommending DST. Finally, provider age, sex, and the number of patients waiting at the time of SP arrival were not significantly associated in a consistent manner with correct case management or the use of antibiotics.

Know-Do Gap

Table 3 compares providers’ knowledge as measured by vignettes for SP1 to actual practice in the SP interactions for 69 providers who both received an SP1 case and completed the vignettes. Provider performance in the vignettes was markedly more consistent with STCI, with significantly higher use of chest x-rays (OR of SP relative to vignettes 0.05, 95% CI: 0.01–0.13) and sputum tests (OR 0.06, 95% CI: 0.02–0.19). Rates of providers questioning the production of sputum or blood in sputum were also higher; however, both in vignettes and practice providers were equally unlikely to ask about past TB or family history of TB. Notably, referrals were significantly lower in SPs than in vignettes (OR 0.07, 95% CI: 0.02–0.33).

Table 3.

Know-do gap: comparison of data from vignettes versus SP1 interactions among the same providers

Notes: For all items, the prefix “E” indicates examinations and laboratory evaluations; the prefix “Q” indicates history questions. Reported percentages reflect the percentage of providers who gave that item during an SP visit or vignette. For “Mean # Medicines Given or Prescribed”, the illustrated bar gap is the percentage change in means. The differences are the result of t-tests comparing the vignette performance percentage with the SP performance percentage for each item. Some differences are inexact because of rounding. Odds ratios as estimated by logistic regression are reported where computable. For all differences and odds ratios the significance levels are indicated by stars, where *** = p<0.01, ** = p<0.05, * = p<0.1.

DISCUSSION

India has the world’s highest TB burden, and MDR-TB is a growing threat.^{19, 20} Recognizing this, the Revised National TB Control Program (RNTCP) has formulated an ambitious National Strategic Plan (NSP, 2012–2017), with the goal of “universal access to quality TB diagnosis and treatment for all TB patients in the community”.²¹ This is consistent with the End TB Strategy by WHO, which emphasizes patient-centered care, and engagement of all providers.²² Since the private sector is a major provider of healthcare^{2, 3, 23}, it is important to know how private providers manage TB in their practice, and about factors associated with suboptimal care.²⁴ Such information is critical for designing public-private mix (PPM) interventions.

Our study, the first assessment of quality of care for TB using the SP methodology, builds on similar research by our team members for other conditions.^{5, 6} Our results suggest that the SP methodology can be successfully implemented for TB – the detection rate in our study compares well with detection rates of 0% to 25% (mean of 15%) in other SP studies, including those from India, and could be further reduced by increasing the study duration.^{5, 6, 25} There were no risks to either providers or SPs during the study, and recall by SPs compared well with audio. Thus, future iterations of this methodology might help TB control programs design monitoring and evaluation indicators for quality of TB care.

However, our study had limitations. As our pilot study was designed to validate the use of SPs in a sample of consenting private providers, a random or comprehensive sample of providers was not the goal. The “ideal” population of providers would be a random sample of providers, would include both private and public sectors, and would cover diverse areas, given the geographical variation in quality of care in India.^{4–6, 11}

A second disadvantage, inherent in the SP approach, is that data must be analyzed in the same manner as it is obtained. This limitation is noticeable in our analysis of drugs dispensed: because 54% of all pills were unlabelled (loose) drugs, we were unable to conclusively assess the rate of antibiotic and steroid use in such cases. SPs were instructed to not ask about the nature of the pills, because this is not consistent with typical patient behavior, and increases detection risk. We attempted to identify these unlabelled drugs through local pharmacists but in the absence of chemical analysis, their identification cannot be confirmed. Chemical analysis of pills is possible, but expensive. This remains a limitation in any context where unidentified pills are given to patients, and this, by itself, suggests poor quality of medical care. Although it is highly unlikely that any of the unlabelled medicines were anti-TB drugs, our estimates of antibiotic and steroid use represent lower bounds on their actual use in the population.

Our SPs did not have physical signs (e.g. crackles) that could be identified by chest auscultation, and providers may have been misled by the lack of physical findings among our SPs. We analyzed data separately for providers who performed auscultation and those who did not for SP1 and SP2 cases and were unable to detect systematic differences in their behavior (data not shown).

The SP methodology works well with one-time and new patient interactions, as opposed to multiple visits to the same provider, or for patients who are already known to the doctor. In 76% of all SP interactions, the provider asked the SPs to come back if they did not get better or to come back with the recommended test results. However, our SPs did not return, as this would have increased the risk of detection, and case scenarios would have become more complex, as well as less ‘standardized’.

Interviews with providers revealed a potential pathway whereby providers “try out” a cocktail of drugs for the first week to 10 days and, if the patient does not improve, move on to a different set of drugs and ultimately chest X-rays and sputum tests. The SP2 case was specifically designed to reflect this pathway as patients may switch providers rather than return to the original doctor.⁹ Although rates of sputum tests and chest x-rays are indeed higher for SP2, they remain low and statistically indistinguishable from SP1. It is possible that a design with repeated SP visits might produce new insights. Such a study, although challenging, may be worth pursuing in the future.

Despite these limitations, our results confirm those from a number of Indian studies (reviewed elsewhere⁸), using tests of knowledge or prescription practices that show low adherence among providers to recommended standards, and highly variable practices among providers.

Our study also raises policy-relevant issues. First, it is widely believed that informal providers contribute to poor diagnosis and treatment.^{9, 26} We did find important differences between MBBS and non-MBBS providers in diagnosis, with the latter less likely to order either sputum tests or chest X-rays for SP1 and SP2. However, overall rates of lab investigations were low for both groups: even among MBBS providers, sputum tests were ordered in only 8 of 71 (11%, 95% CI: 6%–21%) of all cases, and a DST was never ordered for the suspected MDR case. Contrary to concerns about the use of unnecessary laboratory tests in the private sector, our data suggest that persons with TB symptoms in India may be severely under tested, although repeat visits by the SPs will be necessary to better understand this phenomenon. Further, our data suggest that non-MBBS providers do not seem to prescribe TB drugs, even for SP3 and SP4 cases. We also found no evidence on use of the serological tests that were banned in 2012.

Second, our data also cast doubt on the hypothesis that lack of knowledge and capacity constraints are the limiting factors in the accurate diagnosis and treatment of TB. Our data suggest a big ‘know-do gap’. There are key items that doctors did not know (on vignettes), such as asking about past history of TB or about Xpert MTB/RIF testing, but many did know about sputum testing and chest x-rays, yet did not order them in practice. We are conducting additional work to confirm our pilot data, understand the know-do gap in provider behavior, and to identify the best approach to measure and improve quality of TB care in India.

Research in context.

Evidence before this study

A recent systematic review of Indian studies has shown that in most studies, less than half of providers knew to order sputum microscopy testing for patients with TB symptoms, and less than a third knew the correct treatment regimen for TB. Quality of care was lower in the private compared to the public sector. Existing studies of quality of TB care in India and elsewhere have relied on recall-based patient surveys, questionnaire surveys of knowledge, and prescription or medical record analysis. No study has used standardized (simulated) patients [SP] to assess actual clinical practice.

Added value of this study

Our study is the first assessment of quality of care for TB using the SP methodology. The results suggest that the SP methodology can be successfully implemented for TB. Our pilot data suggest low adherence of providers to established standards of TB care in clinical practice despite markedly higher levels of knowledge.

Implications of all the available evidence

Published data and our results demonstrate a big gap between what healthcare providers know about TB, and what they actually do in their clinical practice. Future iterations of the SP methodology might help TB control programs design monitoring and evaluation indicators for quality of TB care.