In Urban And Rural India, A Standardized Patient Study Showed Low Levels Of Provider Training And Huge Quality Gaps

Abstract

This article reports on the quality of care delivered by private and public providers of primary health care services in rural and urban India. To measure quality, the study used standardized patients recruited from the local community and trained to present consistent cases of illness to providers. We found low overall levels of medical training among health care providers; in rural Madhya Pradesh, for example, 67 percent of health care providers who were sampled reported no medical qualifications at all. What’s more, we found only small differences between trained and untrained doctors in such areas as adherence to clinical checklists. Correct diagnoses were rare, incorrect treatments were widely prescribed, and adherence to clinical checklists was higher in private than in public clinics. Our results suggest an urgent need to measure the quality of health care services systematically and to improve the quality of medical education and continuing education programs, among other policy changes.

To improve the quality of health care in resource-poor settings, such as India, national governments, donors, and aid agencies have focused on investments in infrastructure and medical equipment, combined with expansions in the public provision of primary care services and the number of qualified health personnel.^1,2 These investments have been motivated by an implicit assumption that a scarcity of qualified health providers and a lack of physical infrastructure are the primary drivers of low-quality care in resource-poor settings.

Missing from this debate is systematic evidence on the quality of care that patients actually receive when they enter a clinic. There is scant evidence linking improvements in structural aspects of quality, such as the availability of basic medical equipment and medicines, to better diagnoses and treatments for patients. Some recent studies suggest that measures of structural quality are poor proxies for the quality of care.^3–5

There is also little information on the largest sector providing primary care in resource-poor settings—private providers—and no information on private providers without medical qualifications, who often provide the bulk of primary care in the rural areas of many low-income countries.

To address the gap in evidence, this article reports the first estimates of the quality of primary care services in a low-income country, as measured by 926 clinical interactions between 305 medical care providers in rural and urban India and 22 unannounced standardized patients. These patients were people recruited from the local community who were trained to present a consistent case of illness to multiple health care providers.

The use of standardized patients presents a number of advantages described below, relative to other methods of assessing quality, such as direct clinical observations, inspection of medical records where they exist, and patient exit interviews. The use of standardized patients is therefore widely regarded as the “gold standard” in quality measurement.⁶

First, data from standardized patients yield an assessment of provider practice that is free from observation and recall bias. That is, the use of standardized patients is a preferable methodology because the doctor does not change his or her behavior because of awareness of being observed,⁷ it is less vulnerable to recall bias than patient exit interviews,⁸ and it is more complete than what doctors might record themselves in medical records.⁹

Second, standardized patients permit estimates of case detection rates since illnesses are prespecified in the study design. We show below that providers’ diagnoses are often inaccurate, so that methods based on medical records or clinical observations may not yield accurate data on the true illness of the observed patients.

Finally, because all case presentations are standardized, the standardized patient methodology allows for valid quality comparisons across different types of doctors and clinics. Poorer patients or patients with more complicated symptoms might choose particular providers. Thus, data based on real patients could confound true differences in provider quality with differences in patient characteristics.

This study is unique in scale and scope. Our population-based sample of health care providers is representative of primary care facilities that serve the average household in rural Madhya Pradesh, one of India’s poorest states. The sample includes private providers with and without formal medical training. Private providers account for more than 80 percent of primary care visits in India, as is the case in many low-income countries.^10–12

We supplemented our rural data with results from a convenience sample in urban Delhi, one of India’s wealthiest states. Comparisons across the rural and urban sites allowed us to better understand quality deficits and whether they arise only in particular settings.

Health Care In India

India spends 4.2 percent of its gross domestic product on health care, the bulk of which represents private out-of-pocket expenditures. This reflects, in part, the high use of health care services in both urban and rural settings. More than half of the households in rural Madhya Pradesh had sent a household member to a primary care provider in the preceding month. In urban Delhi, the average household visits doctors 2.1 times a month, totaling more than twenty-five visits a year.¹³

For primary care, patients may choose to visit publicly funded sites, ranging from specialized urban hospitals to rural primary health centers staffed with qualified doctors and health assistants. Doctors in the public sector receive a fixed salary, and all consultations should be either free or nominally priced. All public-sector doctors are required to be qualified, and the majority should hold a bachelor of medicine and bachelor of surgery degree—a six-year degree that is the equivalent of a US medical degree.

Most patients, however, seek care in the private sector, where qualifications to practice range from a bachelor of medicine and bachelor of surgery degree to degrees based on traditional systems of medicine such as Ayurveda or Unani to distance education courses with six months of training to no medical training at all. An earlier survey of rural households found that public providers accounted for 8 out of every 100 visits to a health care provider, while unqualified private providers accounted for 70 out of every 100 visits.¹²

Among urban households, 70 percent of all visits were to private-sector providers, and the remaining 30 percent were evenly divided among public primary health care centers and hospitals.¹⁴ Among the visits to urban private-sector providers, 31.5 percent were to providers with minimal or no training, 43.6 percent to providers with some training, and 24.8 percent to providers with a bachelor of medicine and bachelor of surgery degree.

One of the stated aims of the government of India is to improve the delivery of health care, particularly in rural areas. Through the government’s flagship program, the National Rural Health Mission, funding for rural health care services has tripled, from 100 billion rupees (US$1.83 billion) in 2005–06 to 304 billion rupees (US$5.54 billion) in 2011–12, with the goal of strengthening and improving the quality of health care delivery.¹⁵

Nevertheless, large geographic disparities remain in health status. Underscoring the sharp differences between the two sites in our study, infant mortality in Delhi in 2010 was 30 per 1,000 births, compared to 62 per 1,000 births in Madhya Pradesh.¹⁶

Study Data And Methods

There are several components of the standardized patient methodology used to measure the quality of health care. In the description below, we focus on recruitment and training; the choice of medical cases that standardized patients presented; the quality measures for those cases; and the sampling methodology, along with method limitations. The online Appendix¹⁷ provides further methodological details.

RECRUITMENT AND TRAINING

Recruitment and training protocols were designed to ensure that the standardized patients conformed closely to the providers’ regular patient populations. All standardized patients were recruited from local communities and trained for 150 hours by a cross-disciplinary team to consistently portray the emotional, physical, and psychosocial aspects of the cases and to accurately recall interactions with providers. They were also coached to avoid invasive examinations and retain medicines dispensed in the clinic.

This extensive training led to low detection rates of standardized patients by sampled providers. In follow-up visits with private providers in the Delhi sample, doctors reported that they had been visited by a standardized patient in fewer than 1 percent of interactions. In Madhya Pradesh, no provider voiced any suspicions, so the detection rate can thus be considered zero.

CASES PRESENTED BY STANDARDIZED PATIENTS

Each standardized patient presented one of three medical cases: unstable angina, asthma, or dysentery of a child who was not present. These cases satisfy the criteria of relevance, because they represent conditions of high incidence in low-income settings.^18,19 The three cases also pose a low risk to standardized patients from invasive examinations. Finally, the cases represent conditions that have established medical protocols with clear triage, management, and treatment checklists developed by the government’s National Rural Health Mission.^20–22

During training, the standardized patients rehearsed medical case scripts that were designed to make the diagnosis as obvious and uncomplicated as possible. For instance, in the case of a standardized patient with unstable angina, a forty-five-year-old male was coached to complain of chest pain the previous night. The physician or health care provider would be expected to take an appropriate history that would reveal classic signs, such as radiating or crushing pain. The appropriate history would also record risk factors, including smoking and untreated diabetes, as well as whether the patient had a family history of cardiac illness or unstable angina.

For the dysentery case, the standardized patient was a twenty-six-year-old father or mother of a two-year-old child, coached to request medicine for a child with diarrhea. If probed, the standardized patient was to immediately reveal the presence of a fever and the frequency and quality—bloody with mucus—of the child’s stools. The physician or health care provider would be expected to suspect a bacterial infection and ask the parent to bring in the child for further consultation, prescribe appropriate anti-infectives and oral rehydration therapy, or both.

QUALITY MEASURES

We assessed the quality of providers’ medical care by measuring adherence to case-specific checklists of essential and recommended care, the likelihood of correct diagnosis, and the appropriateness of treatment. To collect each of these measures, standardized patients were debriefed with a structured questionnaire within one hour of the interaction with the health care provider. Medicines dispensed in the clinic were saved, and the medicine names were recorded when a label was present.

The checklists for essential and recommended care used here to evaluate the quality of health care included only items that can be completed in low-resource settings, such as asking appropriate questions regarding the patient’s history and performing examinations such as checking the pulse or blood pressure. As a result, the checklists constituted a parsimonious subset of processes that have been recommended in the guidelines of the National Rural Health Mission.

Essential care refers to questions providers must ask and examinations providers must conduct to accomplish a basic diagnosis or protect a patient from serious harm, as determined by a panel of qualified providers. Recommended care refers to essential care plus other required questions and examinations so that providers can develop a reasonable differential diagnosis—for instance, to distinguish between bacterial and viral etiologies for the dysentery presentation.

SAMPLE

In both the rural and urban settings, we sampled among health care providers who accounted for the bulk of primary care visits in the associated communities in both the public and private sectors and regardless of the providers’ qualifications. We excluded community health workers and midwives from the sample.

In 2009 we piloted the standardized patient methodology in urban Delhi among forty-one private providers and twenty-three public clinics offering primary health care in six neighborhoods of the city. This convenience sample of providers was based on previous work in the neighborhoods, and 248 out of a total possible 256 patient-provider interactions were completed.²³

In 2010 standardized patients visited health care providers in sixty villages in Madhya Pradesh, spread across three districts of the state. These providers were randomly selected from among all eligible providers who accounted for 80 percent of all primary care visits from households in the sampled villages. Standardized patients were randomly assigned to providers to ensure that inter-rater differences did not bias the results, and a total of 677 out of a possible 738 interactions were completed with 241 distinct providers offering primary care.

LIMITATIONS

There are limitations to both the standardized patient methodology and our specific sampling strategy in this study. In terms of the standardized patient methodology, to minimize potential harm to the standardized patients, we restricted cases to those that did not require invasive examinations, including the use of thermometers, which are often reused without disinfection. Thus, these estimates of quality might not generalize to communicable diseases.

Ethical concerns also limited the use of standardized patients to the adult population. The only case that related to childhood illness was that of the standardized patients who presented the case of a child at home with symptoms of dysentery.

The narrow standardized patient cases used for this study certainly limited the examinations that the provider could perform, and the scripts presented by the standardized patients may have altered the observed results. For instance, the people who portrayed standardized patients were unknown to the providers they visited, which potentially biased interactions away from continuing care that a doctor may provide for patients with whom he or she has had longer associations.

In terms of sampling, in the rural setting, we could not visit very remote locations, where health care providers would expect to know everyone in a particular village or area. Thus, from a total of seventy-two villages that we could have sampled for the standardized patient fieldwork, we chose fifty-eight.

In the urban sample, we were also limited in our ability to generalize the results as representative of urban populations of providers. Although the urban sample was similar to that of the population in the neighborhoods considered, the urban health care providers were not randomly selected. It is possible that the private-sector providers included in the study were more confident of their abilities, thus biasing the results toward higher-quality care than in a representative sample.

Study Results

RURAL MADHYA PRADESH

First, our study sample reflects the low overall levels of medical training among health care providers in the public and private sectors in this area. In rural Madhya Pradesh, only 11 percent of health care providers that were sampled reported having a medical degree, and just over half reported some education beyond high school (Exhibit 1). Sixty-seven percent reported having no medical qualification at all. Of the remainder, 22.5 percent reported some training in traditional medicine such as Ayurveda or Unani.

Exhibit 1.

Characteristics Of Health Care Providers In Madhya Pradesh And Delhi Samples

Characteristic	Madhya Pradesh (n = 241)	Difference between Madhya Pradesh sample and population (n = 1,039)	Delhi (n = 64)
Age (years)	43.12	1.20	43.60
Male	94%	0.06**	93%
Has MBBS degree	11%	0.01	52%
Has no medical qualification	67%	−0.03	16%
Works in public sector	17%	−0.07**	34%
Average patients per day	17.90	2.57**	14.63
Experience (years)	12.96	0.92	30.78
Educated beyond high school	53%	0.01	— a
Dispenses medicine	84%	−0.04*	— a
Has stethoscope	95%	0.06***	— a
Has sphygmomanometer	77%	0.05	— a
Has thermometer	92%	0.05**	— a
Has electricity	94%	0.09***	— a

source Authors’ analysis. notes For a description of the sample and the population of providers they were selected from, see the Appendix (Note 17 in text). A provider has no medical qualification if he or she has no medical training whatsoever, not even training in short courses or in traditional systems of medicine. High school is the completion of twelve years of schooling. A provider works in the public sector if the standardized patient received attention from the provider in a government clinic. Experience is total years practiced in current location in the Delhi sample and total years practiced in the Madhya Pradesh sample. MBBS degree is a bachelor of medicine and bachelor of surgery degree. Although standardized patients visited 241 unique providers in Madhya Pradesh, we could complete a facility survey with only 226 of them, to obtain the data shown here.

^aProviders in the Delhi sample were not administered a facility survey, but it would be safe to assume that they all have basic medical equipment.

^*p < 0.10

^**p < 0.05

^***p < 0.01

More than 90 percent of the providers in rural Madhya Pradesh were male (Exhibit 1). A large majority of providers owned the medical equipment required for basic examinations. Sampled providers were generally similar in characteristics to the population of providers in the rural setting, and the differences that existed suggest that we sampled among higher-quality providers.

In public health clinics in rural Madhya Pradesh, standardized patients were seen by whoever was providing care at the time of the visit. In 63 percent of the interactions in public clinics, this person turned out to be a provider without medical training, even though the government policy for staffing public clinics calls for a trained public health doctor. Our finding reflects the difficulty India and other countries face in motivating qualified doctors to work in rural areas and the high rate of absenteeism among public health workers.²⁴

Second, study results in rural Madhya Pradesh indicated serious deficits in quality in the performance of health care providers (Exhibit 2). Interactions between health care providers and standardized patients were brief (3.6 minutes), with low levels of history taking and examinations and an emphasis by the provider on giving medication to the patient. On average, about a third of the essential questions and examinations were completed by providers, and each standardized patient received 2.5 medicines.

Exhibit 2.

Mean Quality Outcomes In Standardized Patient-Provider Interactions In Madhya Pradesh

	Standardized cases
	All (n = 677)		Unstable angina (n = 221)		Asthma (n = 228)		Dysentery (n = 228)
Quality outcome	Mean	SE	Mean	SE	Mean	SE	Mean	SE
Visit length (minutes)	3.60	0.12	3.54	0.17	5.31	0.27	1.94	0.11
Recommended questions asked	3.00	0.10	3.05	0.18	3.86	0.17	2.10	0.11
Recommended exams performed	1.49	0.06	1.52	0.09	1.46	0.07	— a	— a
Recommended questions and exams (%)	21.9	0.60	24.1	1.30	24.2	1.00	17.5	0.90
Essential questions asked	1.34	0.03	1.00	0.06	1.44	0.06	1.54	0.06
Essential exams performed	0.40	0.02	0.68	0.04	0.51	0.03	— a	— a
Essential questions and exams (%)	33.7	0.90	29.3	1.70	32.7	1.30	39.1	1.60
Medicines given	2.50	0.06	2.71	0.11	3.21	0.11	1.58	0.08
Medicines given without product label	0.58	0.04	0.75	0.09	0.69	0.09	0.32	0.05
diagnosis (percent)
Gave a diagnosis	32.6	1.80	47.1	3.40	33.8	3.10	17.6	2.50
Diagnosis correct, if given	12.2	0.02	9.6	0.03	18.2	0.04	7.5	0.04
Diagnosis partially correct, if given	41.2	0.03	40.4	0.05	35.1	0.05	55.0	0.08
Diagnosis wrong, if given	46.6	0.03	50.0	0.05	46.8	0.06	37.5	0.08
treatment prescribed or dispensed (percent)
Correct treatment	30.4	0.02	31.2	0.03	47.8	0.03	12.3	0.02
Unnecessary or harmful treatment	41.7	0.02	55.2	0.03	62.7	0.03	7.9	0.02

source Authors’ analysis. notes The unit of observation is the standardized patient-provider interaction. Exhibit 3 lists recommended and essential questions and exams for the unstable angina case, and Exhibits A1 and A2 in the Appendix (see Note 17 in text) list them for the dysentery and asthma cases, respectively. Providers often crush up pills or give loose pills to patients without any labeling; these medicines are coded as medicines given without product label. Exhibit A3 in the Appendix (see Note 17 in text) lists for each case what diagnoses and treatments were considered correct, partially correct, and wrong. SE is standard error.

^aBecause it was a proxy case, exams were not possible for dysentery.

In the case of unstable angina, for instance, providers asked about the location of pain in two-thirds of the interactions, but all other checklist items were completed in less than half of the interactions (Exhibit 3). Only 14 percent of providers asked about pain radiation—a clear symptom of stable or unstable angina. One or more vital sign checks were completed in fewer than a third of all interactions. Checklist completion rates were just as low for the asthma and dysentery cases (see Exhibits A1 and A2 in the Appendix).¹⁷

Exhibit 3. Adherence To Checklist Of Questions And Exams For Unstable Angina In Madhya Pradesh.

source Authors’ analysis. notes Mean values and 95% confidence intervals are shown. All items listed are recommended; those marked “[E]” are essential. A temperature attempt refers to checking temperature either by touch or with a thermometer. EKG refers to either an electrocardiogram performed in the clinic itself or a referral for an electrocardiogram. Beedi-cigarette indicates whether the doctor asked about tobacco use; a beedi is an Indian cigarette consisting of tobacco wrapped in a leaf. “Pain start time” is asked to ascertain a specific time of day.

Brief consultation times and poor adherence to checklists resulted in treatments that were inconsistent with treatment guidelines. Across all cases, the correct treatment protocol was followed 30.4 percent of the time, while an unnecessary or harmful treatment was prescribed or dispensed 41.7 percent of the time (Exhibit 2). For example, only 31.2 percent of standardized patients presenting with unstable angina were correctly treated; in only 12.3 percent of the dysentery interactions were standardized patients advised to give their child oral rehydration solution; and providers in 62.7 percent of the asthma cases provided unnecessary or harmful treatment.

Third, there was little provider-patient communication. Only one-third of providers articulated a diagnosis, correct or incorrect. When a diagnosis was articulated, close to half were wrong, and only 12.2 percent were fully correct (see Exhibit A3 in the Appendix for details).¹⁷ Some of the incorrect diagnoses for unstable angina, for instance, included gastrointestinal or weather-related problems.

The true rate of correct diagnosis for all interactions was probably lower than these results. Empirical patterns in the data suggest that had they been forced to say something to the standardized patient, providers who did not articulate a diagnosis would have been less likely to articulate a correct one than those who did say something to the patient. Those providers who articulated any diagnosis completed more checklist items than providers who said nothing. Among those articulating a diagnosis, greater completion of checklist items was associated with a higher likelihood of correct diagnosis. If we assume that the relationship between adherence to the checklist and the likelihood of articulating a correct diagnosis holds for everyone—both those articulating any diagnosis and those who said nothing—then accounting for the missing diagnosis data would further lower the rates of correct and partially correct diagnoses in these data.

Do these results in rural Madhya Pradesh reflect low rates of training of health care providers, high caseloads of patients, or poor health infrastructure? Exhibit 4 shows results from multiple linear regressions of three measures of quality. The exhibit shows that, first, qualifications of health care providers do matter, but perhaps not as much as expected. After we controlled for provider and clinic characteristics, unqualified providers completed 3.24 percentage points fewer recommended questions and exams but were just as likely to articulate a diagnosis and provide correct treatment, compared to qualified providers.

Exhibit 4. Marginal Effects On Process Quality, Diagnosis, And Treatment Of Provider And Clinic Characteristics In Madhya Pradesh.

source Authors’ analysis. notes This exhibit graphs the regression coefficients of three separate regressions of provider and clinic characteristics (an indicator for working in the private sector, an indicator for having no medical qualifications, a facilities and equipment index, and the patient caseload at the time of the standardized patients’ visits) on the percentage of recommended questions asked and exams performed, the relative risk of giving any diagnosis, and the relative risk of giving a correct treatment. Thus, the effects in regression 1 are percentage points. The effects in regressions 2 and 3 are rates—for example, private providers are 1.76 times as likely as public providers to provide any diagnosis. Bars labeled with an asterisk denote statistical significance at least at the 90 percent level of confidence. The other coefficients cannot be significantly distinguished from 0 (for regression 1) or from 1 (for regressions 2 and 3). Each regression also controls for the provider’s sex and experience and contains indicator variables for each standardized patient and for the village’s being in the bottom 20 percent of a wealth distribution implied by an asset index we created from the average household’s ownership of seventeen assets, such as a television or a floor that is not made out of mud. *p < 0.10

Second, infrastructure and the patient caseload at the time of the visit had little significant association with any quality measure. Third, the biggest difference in adherence to the checklist and the likelihood of articulating a diagnosis was between the public and the private sectors. The mean checklist completion rate for recommended questions and examinations was 6.81 percentage points higher for all private providers (Exhibit 4) and 3.6 percentage points higher among private providers with no training (unadjusted means comparison; data not shown).

We note that this result could arise in part from the small number of qualified doctors within public-sector clinics at the time of the standardized patient visit. Sixty-three percent of the standardized patients were attended to by personnel without medical qualifications in the public sector. Public- and private-sector doctors, however, were equally likely to adhere to the appropriate treatment protocol.

URBAN DELHI

Results from the urban setting showed a higher level of trained health care providers but similarly large quality deficits with equally poor adherence to checklists. However, diagnosis and treatment rates were somewhat better in the urban setting than in the rural setting.

In Delhi 52 percent of providers in the sample working in the public and private sectors had medical degrees, and only 16 percent reported having no medical qualifications. In the public sector, all providers who were visited had medical degrees.

Patterns of variation across different types of providers and clinics in urban Delhi were similar to patterns in the rural setting. Specifically, consultation times were somewhat longer (5.4 minutes) than in the rural setting, and the rates of correct diagnosis (21.8 percent) and treatment (45.6 percent) were higher (data not shown). However, the adherence rate in Delhi to the essential care checklist (31.8 percent) was slightly lower than the rate in Madhya Pradesh (33.7 percent).

Exhibit 5 shows that as in the rural sample, private health care providers showed higher levels of adherence to checklists, although the likelihood of prescribing the correct treatment was significantly lower—a result primarily driven by the asthma case. Again, qualifications did matter to some degree, but there were few if any associations between our quality-of-care measures and structural quality, such as medical equipment or patient caseloads.

Exhibit 5. Marginal Effects On Process Quality, Diagnosis, And Treatment Of Provider And Clinic Characteristics In Delhi.

source Authors’ analysis. notes This figure graphs the regression coefficients of three separate regressions of provider and clinic characteristics (an indicator for working in the private sector, an indicator for having no medical qualifications, and the patient caseload at the time of the standardized patients’ visits) on the percentage of recommended questions asked and exams performed, the relative risk of giving any diagnosis, and the relative risk of giving a correct treatment. Thus, the effects in regression 1 are percentage points. The effects in regressions 2 and 3 are rates—for example, private providers are three times as likely as public providers to provide any diagnosis. Bars labeled with an asterisk denote statistical significance at least at the 90 percent level of confidence. The other coefficients cannot be significantly distinguished from 0 (for regression 1) or from 1 (for regressions 2 and 3). Each regression also contains indicator variables for each standardized patient and for the neighborhood’s being in the bottom 20 percent of a wealth distribution implied by an asset index we created from the average household’s ownership of seventeen assets, such as a television or a floor that is not made out of mud. *p < 0.10

Discussion

The deployment of unannounced standardized patients in a rural and urban setting in India uncovered health care providers practicing without medical training and major practice deviations by providers. These practice deviations consisted of a minimal completion of checklists of recommended care, low diagnosis rates, poor adherence to treatment guidelines, and frequent use of harmful or unnecessary medications.

Structural quality measures such as medical equipment, provider qualifications, and patient caseloads were, at best, weakly associated with the observed quality of care. The persistence of large quality deficits in the urban setting suggests that low income and education levels in patient populations alone cannot explain the observed low quality of medical care.

One concern in interpreting these results is that as in most standardized patient studies, our patients did not display active symptoms of the cases presented at the time of the visit to the health care provider. Therefore, some providers may have discounted the reported illness or symptom history because the patient did not appear to be in real distress.

The government’s medical guidelines, however, stipulate that providers should complete a set of essential questions and examinations, regardless of the patient’s physical condition at the time of presentation. The standardized patient presenting with chest pain, for example, should have been asked about pain radiation—a red flag that should have led to a referral to a hospital or, at least, a referral for an electrocardiogram.

Moreover, if doctors doubted the standardized patient presentations, further questioning and examinations would have led the provider away from making a correct diagnosis as more information about the patient was revealed to the doctor. We found exactly the opposite. Greater adherence to the checklist increased the likelihood of correct diagnosis, which suggests that the case presentations were convincing enough to result in the correct diagnosis and treatment as long as the provider asked the right questions and performed the right examinations.

A second concern is that the methodology of assessing providers using checklists, diagnoses, and treatment protocols is based on a Western model of medicine. This might not be applicable for the 25 percent of providers with some training in traditional medicine, such as Ayurveda or homeopathy. However, previous work demonstrates that such providers are no less likely than Western providers are to prescribe antibiotics and no more likely to prescribe traditional medicines.¹⁴

This result continues to hold in the current sample. For example, health care providers with degrees from traditional systems of medicine prescribed steroids in 9.6 percent of interactions with standardized patients, which was statistically indistinguishable from the corresponding rate—7.8 percent—among providers with medical degrees. Moreover, providers from traditional systems of medicine serve as households’ first point of contact with primary care services and thus should be evaluated based on how well they can triage and manage the conditions that bring patients into their clinics.

Conclusion

For many readers, it may be startling to learn that a large portion of rural health care in India is provided by people without formal medical training. Such providers accounted for 70 out of every 100 primary care visits in our rural setting. Yet simply advocating for an increase in the number of trained medical providers ignores both the scale and the multiple dimensions of the problem. There are fifteen times as many unqualified providers as those with a medical degree in the rural setting. Increasing the number of trained providers fifteenfold while maintaining quality standards of training would be, at the very least, a long-term effort.

Furthermore—and this is at the heart of our article—training in and of itself is not a guarantor of high quality. In both the rural and urban setting, we found only small differences between trained and untrained doctors in adherence to the checklist and no differences in the likelihood of providers’ giving a diagnosis or providing the correct treatment. Nor did we find better care in public relative to private clinics. In fact, the evidence suggests that untrained private-sector providers were better in adhering to the checklist, and no worse in their treatment protocols, than their public-sector counterparts.

At the very least, therefore, our results necessitate an urgent call to better integrate quality measures such as those advanced here into existing health policy in India instead of assuming that quality is higher for trained doctors or in facilities with more equipment. It is unlikely that quality can improve unless it is first measured systematically and in multiple settings.

More substantively, if quality cannot be improved solely by investing in infrastructure or deploying more qualified health personnel, why is this the case? At least in the Indian context, there are several potential explanations.

First, medical education in India is provided by a large number of institutions, both public and private, with vast variation in the quality of instruction.^25–27 Previous work has documented that providers with medical degrees in urban Delhi can range from the very best to among the worst in terms of pure technical competence, depending on where they were trained.²⁸ There are also no national continuing education or recertification requirements. Fundamental reforms of the way that medical degrees are awarded and continued recertification may offer partial solutions.

Second, provider effort may be a key determinant of quality in health service provision.^29,30 A recent systematic review of public and private ambulatory care in low- and middle-income countries concluded that the private sector exerts more effort than the public sector.³¹ This is a potential explanation for why we observed better care in the private sector despite lower qualifications. Although we do not have specific evidence from research in India, recent studies from other low-income countries suggest that improvements in provider effort could come from some combination of better payment mechanisms (for instance, performance-based pay), better monitoring, and providing denser peer networks.^32–34

Business-as-usual expansions with a focus on physical inputs and putting doctors in clinics may be necessary, but they are far from sufficient to improve the quality of health services provided. A much broader debate on getting doctors to put all of their knowledge into actual practice is required.

Biographies

Jishnu Das is a senior economist at the World Bank.

In this month’s Health Affairs, Jishnu Das and coauthors report on a study that used “standardized” patients trained to present consistent cases of illness to providers in order to assess the quality of health care provided in India. The study unearthed widespread quality deficits in both rural and urban settings, with providers exhibiting low adherence to clinical checklists, poorly diagnosing illnesses, and providing a large share of incorrect treatments. The authors say that the results suggest an urgent need to measure quality of health care services systematically in India and to improve the quality of medical education and continuing education programs, among other policy changes.

Das is a senior economist at the World Bank and a visiting fellow at the Center for Policy Research, in New Delhi. His recent research focuses on child learning, the quality of health care, mental health, information, and trust. In 2011 he was part of the core team that prepared the World Development Report on Gender and Development, and in 2006 he received both the George Bereday Award from the Comparative and International Education Society and the Stockholm Challenge Award for public administration for aid coordination through an information and communication technology project after the 2005 earthquake in Pakistan. He holds a doctorate in economics from Harvard University.

Alaka Holla is an economist at the World Bank.

Alaka Holla is an economist in the Chief Economist’s Office of the World Bank’s Human Development Network. Her research focuses on measuring the quality of service delivery; discrimination in access to services; and the relationships among prices, access, and quality in health and education markets.

Holla is currently contributing to analytical work in Argentina, Cambodia, India, Nigeria, Serbia, and Tanzania. She holds a doctorate in economics from Brown University.

Veena Das is the Krieger-Eisenhower Professor of Anthropology at the Johns Hopkins University.

Veena Das is the Krieger-Eisenhower Professor of Anthropology and a professor of humanities at the Johns Hopkins University. She is the author or editor of ten books and several papers in peer-reviewed journals. Her most recent book is Life and Words: Violence and the Descent into the Ordinary (2007).

Das is a fellow of the American Academy of Arts and Sciences and the Academy of Scientists from Developing Countries. She is also the recipient of an honorary doctorate from the University of Chicago and was a Simon Guggenheim Fellow during 2009–10. She received a doctorate in sociology from the University of Delhi.

Manoj Mohanan is an assistant professor at Duke University.

Manoj Mohanan is an assistant professor of public policy and global health at the Sanford School of Public Policy at Duke University. His current research includes the experimental evaluation of financial and nonfinancial incentives in health care, development of new tools to measure quality of care, impact evaluation of large public-sector subsidy programs, evaluation of social franchising models, and estimation of the effect of quality and prices on households’ choice of providers.

Mohanan is a cofounder of the Collaboration for Health Systems Improvement and Impact Evaluation in India. He received a master’s degree in public health and a doctorate in health policy, with a specialization in economics, from Harvard University. He also holds a medical degree from Grant Medical College, in Mumbai.

Diana Tabak is a lecturer in the Department of Family and Community Medicine, University of Toronto.

Diana Tabak is a lecturer in the Department of Family and Community Medicine and associate director of the Standardized Patient Program at the University of Toronto. She has been a standardized patient educator at the University of Toronto for more than twenty years, working closely with faculty in medicine, pharmacy, and other health professions in the design and development of educational and training programs using clinical simulations. She has received teaching and educational development awards and has collaborated in a variety of research projects.

Tabak has consulted internationally on the design and development of standardized patient programs and methodology at universities and research institutes, most recently as part of the Medical Advice, Quality, and Absenteeism in Rural India project and the Institute of Socio-Economic Research on Development and Democracy in Delhi and Serbia. She received a master’s degree in education from the University of London and a diploma in surgical education from Imperial College, London.

Brian Chan is a clinical and research fellow at Massachusetts General Hospital and Brigham and Women’s Hospital.

Brian Chan is a clinical and research fellow in infectious diseases at Massachusetts General Hospital and Brigham and Women’s Hospital. In addition to work in India, Chan has served as a consultant and policy adviser in Liberia, Indonesia, Kenya, and Serbia.

Chan’s policy and research interests include the effective delivery of primary care and care of chronic diseases, including HIV/AIDS, in resource-limited settings. He received a medical degree from Harvard University and completed his training in internal medicine at Massachusetts General Hospital.