Should we have confidence if a physician is accredited? A Study of the Relative Impacts of Accreditation and Insurance Payments on Quality of Care in the Philippines

Abstract

It is unclear whether health provider accreditation ensures or promotes quality of care. Using baseline data from the Quality Improvement Demonstration Study (QIDS) in the Philippines we measured the quality of pediatric care provided by private and public doctors working at the district hospital level in the country’s central region. We found that national level accreditation by a national insurance programme influences quality of care. However, our data also show that insurance payments have a similar, strong impact on quality of care. These results suggest that accreditation alone may not be sufficient to promote high quality of care. Further improvements may be achieved with properly monitored and well-designed payment or incentive schemes.

Introduction

This paper examines the effectiveness of accreditation to ensure or promote the quality of inpatient, pediatric care. We specifically go beyond earlier work to ask whether accreditation by a national health insurance program can support a higher quality standard among public and private providers for selected high prevalence conditions (diarrhea, pneumonia, and dermatitis) in patients under five years old. We also examine the usefulness of an alternative policy handle, insurance payments, in encouraging high quality of care.

Theoretical Framework

An important characteristic of health care markets is the presence of information asymmetry, where health providers have more information than consumers concerning the state of health, the technology needed to address an illness, and the potential outcomes of services provided (Arrow, 1963). Moreover, when health providers act as “less than perfect agents” of consumers (Pauly, 1980), over- and underprovision, as well as variations in quality, becomes a health concern.

Institutional mechanisms have evolved in response to these problems. Licensing laws, for example, are meant to limit the range of alternative quality levels (Arrow, 1963) and reduce quality uncertainty (Akerlof, 1970). While licensing is required by the state, accreditation is typically voluntary, a requirement for participation in certain programs of care, and focuses on quality standards that exceed those of licensing. Moreover, some form of self-selection is expected – health care providers who have the capacity to meet higher-than-minimum quality standards are likely to apply and ultimately qualify for accreditation.

Accreditation is expected to minimize variations in medical practice, eliminate medically inappropriate care, and control costs (Shin, 1995; Viswanathan & Salmon, 2000). By subjecting health care organizations to a formal process that makes them meet predetermined standards, accreditation is also presumed to address the possibility that quality is underprovided (Akerlof, 1970).

Despite this widely held practice, it remains unclear if accreditation is an effective quality screen for patients, payers, or policymakers. Donabedian (2005), citing evidence as far back as 1964 on New York hospitals, writes that accreditation “does not appear, in and of itself, to be related to quality of care.” Some studies have suggested that the impact of accreditation on quality is somewhat weak (Salmon, Heavens, Lombard, & Tavrow, 2003; Donabedian, 2005; Kumaranayke, Mujinja, Hongoro, & Mpembeni, 2000; Chen, Rathore, Radford, & Krumholz, 2003). Accreditation, by its very nature, does not measure the interaction between the provider and the patient and focuses instead on the structural elements of care (Peabody, Gertler, & Leibowitz, 1998). This limitation has prompted accrediting bodies to begin moving beyond structures-based standards and towards the inclusion of process and outcomes-based performance standards. Still, progress in this area has been limited primarily by the challenge of accurately measuring outcomes and resistance from facilities to measure items other than simple structural inputs (Bodenheimer, 1999).

Incentives for Improvement in Quality

Besides “rules” - which apart from licensing and accreditation include treatment protocols, practice guidelines, and utilization reviews - incentives are yet another set of instruments for quality promotion (Hillman, 1991). Typically financial in nature, incentives follow a reward and penalty system for certain clinical decisions. While they are embedded in all types of provider payment, it is managed care incentives that are useful for purposes of quality assurance. When care is unmanaged, as in the traditional fee-for-service system, the incentive is strongest for cost expansion and unlikely for quality improvement (Dudley, Miller, Korenbrot, & Luft, 1998; Miller & Luft 2002; McGlynn, 2004). As care is increasingly managed financially, this raises the issue of how well incentives address patient well-being.

Recently, there has been increased interest in the use of financial rewards for quality (Epstein, Lee, & Hamel, 2004). A basis for this interest is the growing body of evidence that pay-for-performance can be effective in improving quality (Lindenauer, Remus, Roman, Rothberg, Benjamin, Ma, et al. 2007; Petersen, Woodard, Urech, Daw, & Sookanen, 2006; Campbell, Reeves, Kontopantelis, Middleton, Sibbald, & Roland, 2007). As seen most frequently in the U.S, a growing number of pay-for-performance programs are adding payment incentives on top of exiting accreditation requirements, making it all the more important to consider them in conjunction with one another. As seen most frequently in the U.S, a growing number of pay-for-performance programs are adding payment incentives on top of exiting accreditation requirements, making it all the more important to consider them in conjunction with one another.

II. Study Background

The Social Health Insurance System in the Philippines

The national health insurance system in the Philippines, also known as PhilHealth, is the largest insurance program in the country. PhilHealth reimburses on a first-peso basis for inpatient services including room and board, surgical procedures, physician services, selected procedures, and drugs.

Public and private providers are eligible to receive payment from the program after undergoing accreditation procedures. In the Philippines, public sector doctors practice in government-owned health facilities, including rural health units or district hospitals. Private doctors, on the other hand, practice in either free-standing clinics or hospital-based clinics. While public doctors receive fixed monthly salaries, private doctors charge fee-for-service. Thus, while the piece rates faced by private doctors tend to encourage overprovision of services, the fixed and regular lump-sums received by public doctors provide an incentive for underprovision. For both public and private doctors,, PhilHealth payments represent an important portion of physician reimbursement among the sample of 150 doctors we interviewed in our study sites (about 7–12 percent of total physician income).

Quality Assurance through Accreditation and Insurance Payments

Within PhilHealth, there are three dominant mechanisms to intended to promote quality of care. The first is accreditation, defined by the organization as a “verification process of the qualification and capabilities of health care providers prior to granting of privilege of participation in the National Health Insurance Program, to ensure that health care services that they are to render have the desired and expected quality” (Philippine Health Insurance Corporation, 2004). Second, all claims submitted are subjected to a medical review before insurance payments are approved and reimbursed. Claims review is intended primarily to reject claims for fraudulent services and inappropriate care. The third mechanism works through the financial incentives built into regular PhilHealth payments, which represent an additional and possibly steady source of income. Providers can make economic gains by meeting quality standards - whether through accreditation or claims processing. PhilHealth accreditation for health care facilities and professionals are separate processes. To be accredited physicians are required to have passed the national board exam, be members of the Philippine Medical Association, and be registered with the Bureau of Internal Revenue. Facilities need to show proof of compliance of mostly structural measures of quality, such as number of beds, presence of certain medical equipment, and staff qualifications.

PhilHealth payment eligibility differs between public and private physicians. Private doctors are required to undergo physician-level accreditation to be eligible for payments, which they receive directly and entirely. A public doctor, however, needs only to be employed by an accredited hospital headed by a hospital chief who is accredited. In public hospitals, PhilHealth reimburses individual doctors indirectly. Payments are issued to the hospital chief who in turn distributes these to the hospital staff; while the distribution scheme may vary by hospital, the attending physician receives some amount for effort, regardless of individual accreditation status.

The QIDS Research Study

The data for this study come from the Quality Improvement Demonstration Study (QIDS), an experiment designed to evaluate the impact of health policy reforms on patient and provider behavior and health outcomes. Policies were implemented in a randomized controlled fashion to 30 hospitals located in 10 provinces situated in the greater Visayas region of the Philippines. Baseline (pre-intervention) assessment of the 30 hospital districts was conducted in 2003–2004.

III. Methods

Data Collection

The relevant QIDS data for this paper were those collected at baseline from individual physicians at each of the participating hospitals. Three physicians were selected from each hospital. In parallel, two private physicians were randomly selected from a comprehensive roster of private doctors from the catchment areas of each of the 30 hospitals. For inclusion into the study, all physicians had to be graduates of an accredited medical school. While doctors were not required to be pediatricians, children had to account for a significant amount of their practice. Public physicians had to practice full-time in public hospitals while private physicians had to live in the same district as the public hospital and serve the same geographic population. Participation was strictly voluntary.

Vignettes were administered concurrently to all public and private physicians in the study. Vignettes are written case scenarios designed to measure the quality of clinical care by measuring a doctor’s ability to properly diagnose and treat patients (Dresselhaus, Peabody, Lee, Glassman, & Luck, 2000). Studies have validated clinical vignettes as measures of clinical practice and the quality of care provided to the patient (Peabody, Luck, Glassman, Jain, Hansen, et al., 2004).

Vignettes require the doctor to answer open-ended questions about a typical pediatric patient. The physician responses are scored and used to examine the clinical decisions made by physicians in five domains of care: (i) taking a medical history, (ii) performing a physical exam, (iii) ordering tests, (iv) making a diagnosis and (v) prescribing a treatment plan. Each physician answered three vignettes, one for diarrhea, one for pneumonia, and a third one on a common dermatologic condition. A total of 15 vignettes – 5 vignettes for each of the 3 conditions – were prepared for the baseline survey. Assignment of the specific vignettes was done randomly.

The vignettes were independently scored by two trained physician abstractors, who were blinded to the physician’s identity and each other’s scores. Scorers reviewed the physician responses and used a scoring form to indicate items which physicians correctly identified on their answer sheets. Weights were assigned to each item on the scoring form, with a weight of 1.0 assigned to those items that are deemed most critical by experts. An average vignette score for each physician was thus generated. Scores had a possible range of 0 to 100 percent. We concurrently administered a physician survey to collect data on personal demographics, education, specialty, time allocation, practice characteristics, clinic characteristics, income and expenditures.

Analysis: Modeling Vignette Scores

We first compared basic demographics of pubic versus private providers and their overall quality by specific clinical domain (e.g. history taking). We next modeled vignette scores as a function of our main variables of interest, PhilHealth accreditation and payments as well as other physician characteristics including age, sex, specialization, and type of practice. The basic model is summarized by the following equation:

$$S_{ij}={\alpha }_j+\beta A_i+{\displaystyle \sum _{k=1}^K{\phi }_k{P}_{ik}+{\displaystyle \sum _{l=1}^L{\theta }_l{X}_{il}+{\epsilon }_{ij}}}$$

where S_ij is the average vignette score of the i^th physician in the j^th district; A_i is a dummy variable indicating the accreditation status of the i^th physician; P is a vector of K PhilHealth payment variables including an indicator variable for receipt of PhilHealth payments and the peso amount of PhilHealth payments; X is a vector of L physician characteristics; and ε is a random disturbance term. With random effects (RE), α_j = α₀ + u_j, where u_j represents a random error term that is specific to districts.

We used multivariate regressions to estimate models of aggregate vignette scores (the average of all three vignettes taken) to isolate the accreditation and insurance effect estimates on vignette scores. The accreditation variable indicates whether the set of minimum qualifications for physician accreditation is a sufficient mechanism for quality and thus represents a screening threshold for quality. The receipt of payments indicates whether insurance claims have been found to be satisfactory after they have been subjected to medical reviews. Finally, the payment amount reflects the importance of such inflow as an income source to the provider.

We first estimated the basic model using Ordinary Least Squares with both accreditation and payment variables as independent variables. However, in this combined model, both coefficients were insignificant, which is attributable to the high degree of collinearity between these two independent variables (correlation coefficient equal to 0.80), prohibiting us from determining the independent effects of accreditation versus payment in this combined model. We thus elected to model these effects separately. We used the full sample of private physicians – including those who were accredited, not accredited, and those who received and did not receive payment - to model accreditation and payment. We present separate models for accreditation and payment: Model 1 includes the accreditation variable and Model 2 includes the payment variables.

We then used the public physician sample to determine the independent effects of accreditation. The public physician sub-sample was analyzed in 4 separate models: (i) doctors who received insurance payments versus (ii) those who did not and (iii) doctors who are accredited versus (iv) those who are not. Importantly, these four sub-sets of public physicians are not mutually exclusive as a result of the PhilHealth payment rules for public physicians. In short, we used the private sub-sample to establish the presence of the collinearity effects; then used the public sub-sample to further qualify these effects. Thus, any insurance effect found among non-accredited public physicians can be interpreted as a “pure” payment effect. Similarly, we can test the existence of a “pure” accreditation effect from the sub-sample of public physicians who are accredited but did not receive payments.

A Hausman test, performed to determine the correct specification of the models estimated on the public sample, indicated that the correct specification for these sub-sample regression analyses was a random effects rather than a fixed effects model; hence random effects was used in these models. All statistical analyses were performed using STATA 8.0.

III. Results

The Sample of Physicians

The mean age of doctors in the study was 42 years old; doctors as young as 24 years old and as old as 70 years old participated in the survey. In our regression sample, 61 percent of physicians are public and 39 percent are private. A majority of the doctors are PhilHealth accredited (66 percent) and reported receipt of PhilHealth payments in the year prior to the survey (64 percent). There were two significant differences between the samples of public and private doctors. A higher proportion of private doctors were pediatricians while a higher proportion of public doctors reported to be receiving payments at the time of the survey. There were no significant differences across sub-samples for other characteristics examined in this study (see Table 1).

Table 1.

Descriptive Statistics of Private and Public Physicians

	Private	Public	p-valuea
Age (yrs)	42.82	41.43	0.41
Male (proportion)	0.34	0.35	0.91
Pediatrician (proportion)	0.36	0.21	0.06
Accredited (proportion)	0.58	0.71	0.12
Payments received (proportion)	0.48	0.74	0.00
Payment amount (annual, pesos)	19,900	16,636	0.66
Number of observations	56	89

^aT-test for difference in means

Quality of Care among Public versus Private Doctors

Among the sample the average aggregate vignette score is relatively low (54 percent), consistent with assessments of quality in other parts of the world (Peabody & Liu, 2007). Table 2 presents the regression results for the separate models. Across models results indicate that younger doctors provide significantly higher quality of care. There were no statistically significant differences in the quality of care by gender or between pediatricians versus non-pediatricians. Accreditation, in Model 1, is significantly associated with quality of care, as is the receipt of payments, in Model 2. The coefficients suggest that insurance payments potentially affect quality for private doctors more than accreditation. A private doctor receiving PhilHealth payments has an average vignette score 8 percentage points higher, while an accredited doctor’s average vignette score is on the margin 6 percentage points higher than those without PhilHealth accreditation. The actual amount of the payment has no influence on quality in the models.

Table 2.

Regression Models of Average Vignette Scores of Private and Public Physicians

	Private		Public
	Model 1	Model 2	Model 3: Accredited	Model 4: Not Accredited	Model 5: Received Payments	Model 6: No Payments
Independent Variable	Coefficients (p-value)	Coefficients (p-value)	Coefficients (p-value)	Coefficients (p-value)	Coefficients (p-value)	Coefficients (p-value)
Age	−0.0033 (0.025)	−0.0036 (0.017)	−0.0019 (0.200)	−0.0032 (0.060)	−0.0008 (0.590)	−0.0114 (0.000)
Male	0.0047 (0.879)	0.0006 (0.986)	−0.0716 (0.020)	0.0636 (0.120)	−0.0239 (0.360)	−0.0631 (0.120)
Pediatrician	0.0436 (0.157)	0.0365 (0.258)	0.0151 (0.660)	−0.0117 (0.760)	0.0174 (0.530)	0.0477 (0.230)
Accreditation	0.059 (0.051)				−0.0213 (0.470)	0.154 (0.000)
Payments received		0.0804 (0.013)	0.0068 (0.840)	0.0957 (0.020)
Payment amount (annual, pesos)		−6.80E-08 (0.800)	0.000 (0.620)	0.000 (0.540)
Constant	0.6395 (0.000)	0.6609 (0.000)	0.6195 (0.000)	0.5874 (0.000)	0.5878 (0.000)	0.8826 (0.000)
R-squared	0.1569	0.2616	0.1071	0.3334	0.0459	0.301
Number of observations	55	55	63	26	66	23

We next investigated the impact of accreditation and payment among the public doctors (Models 3–6). We observed significant associations between accreditation and vignette scores, or ‘accreditation effects’ on quality, in public doctors who did not receive payments. For the non-accredited sample of public doctors, we found a significant association between payment and vignette scores, or what we call a ‘payment effect’ on quality of care. When we predict the average vignette scores at the means for each sub-sample, payment and accreditation both account for about a 2 percentage point change in quality.

IV. Discussion

Other studies have shown that quality of clinical care is generally low (Institute of Medicine, 2001; Peabody & Liu, 2007). Improving quality is a rapid way to improve the provision of services and even health outcomes in resource constrained environments (Peabody, Tozija, Munoz, Nordyke, & Luck, 2004).

We find evidence that accreditation can be an effective mechanism for quality assurance among both public and private providers in a developing country setting, but that insurance payments to providers are equally associated with higher quality of clinical care. Thus, accreditation alone may not be a sufficient quality assurance mechanism; improvements in quality may further be achieved with a properly designed payment scheme.

The payment effects on quality that we find can be viewed as a type of incentive effect. Insurance claims that are found to be unsatisfactory by the claims review unit of PhilHealth are not reimbursed and thus, an incentive exists for providers to comply with clinical practice guidelines on which the review process is based. From a doctor’s point of view, good quality care—not accreditation per se—is rewarded in the form of insurance payments. The payment effect that we observe for the group of public, non-accredited doctors suggests that a “properly designed” payment scheme is one that links payment with performance.

The existing literature on provider accreditation and quality of care lacks studies from developing countries. A notable exception is a study based on data from hospitals in South Africa (Salmon et al, 2003), which found little or no effect of a randomized accreditation program on quality measures apart from increases in perception of quality among nurses. A thoughtful discussion of the lack of association points to possible methodological problems, including difficulty in adequately measuring quality indicators. Our study advances the literature on accreditation in the developing country setting by using a validated direct measure of quality of care that controls for case-mix, which can confound quality of care analyses (Peabody et al., 2004). Furthermore, by looking solely at high prevalence diseases among the pediatric population, our findings are less likely to be subject to methodological issues relating to differences in case mix. Our study is the first study of its kind on accreditation and payment effects on pediatric quality of care. This area would benefit from additional research, particularly to address the idea that improving quality involves a package of the following key elements: accredition, pay-for-performance, non-financial incentives, and structural reforms.

While our study contributes to a limited body of literature, we note some limitations which future research can address. First, we note the usual potential biases that may arise as a result of using non-experimental data in the analysis. Second, our small sample prohibited us from disentangling the effects of accreditation and payment among private doctors. Although we suggest payment be linked to performance no differently for public and private doctors, with a larger sample size, more statistically robust results could be obtained for the two distinct groups.

Our findings also provide basis for the expectation that the QIDS bonus scheme currently being implemented in 10 public hospitals in Central Philippines can be an effective policy handle in improving quality of care. The bonus is a performance-based payment, where bonus eligibility depends on attaining pre-set quality standards. Our results suggest that even in the pre-experiment regime, payments have built in quality incentives that work well in the private setting where the link between payments, effort, and outcomes is clear and can be influenced by individual doctors. Extending this to the public sector is an important challenge and the subject of ongoing research.