Does Competition Improve Health Care Quality?

Abstract

Objective

To identify the effect of competition on health maintenance organizations' (HMOs) quality measures.

Study Design

Longitudinal analysis of a 5-year panel of the Healthcare Effectiveness Data and Information Set (HEDIS) and Consumer Assessment of Health Plans Survey® (CAHPS) data (calendar years 1998–2002). All plans submitting data to the National Committee for Quality Assurance (NCQA) were included regardless of their decision to allow NCQA to disclose their results publicly.

Data Sources

NCQA, Interstudy, the Area Resource File, and the Bureau of Labor Statistics.

Methods

Fixed-effects models were estimated that relate HMO competition to HMO quality controlling for an unmeasured, time-invariant plan, and market traits. Results are compared with estimates from models reliant on cross-sectional variation.

Principal Findings

Estimates suggest that plan quality does not improve with increased levels of HMO competition (as measured by either the Herfindahl index or the number of HMOs). Similarly, increased HMO penetration is generally not associated with improved quality. Cross-sectional models tend to suggest an inverse relationship between competition and quality.

Conclusions

The strategies that promote competition among HMOs in the current market setting may not lead to improved HMO quality. It is possible that price competition dominates, with purchasers and consumers preferring lower premiums at the expense of improved quality, as measured by HEDIS and CAHPS. It is also possible that the fragmentation associated with competition hinders quality improvement.

Reform of the health care system to promote improved quality of care and lower cost growth has been a policy objective for decades. A strategy frequently discussed as a means to achieve these goals is the pursuit of a competitive health care delivery and financing system, including a system of competing health maintenance organizations (HMOs) (Haas-Wilson and Gaynor 1998; Gaynor and Vogt 2000; Enthoven 2004; Nichols et al. 2004). While competition among HMOs remains an important pillar of policy and cost containment, as evidenced by the Medicare Advantage program (Gold 2005), recently, other plan types such as high deductible and consumer-driven plans have also grown in importance.

Advocates of the competitive approach assert that competition will both reduce costs and improve quality. Because market structure is a central component of price theory (Feldman et al. 1990; Feldman, Wholey, and Christianson 1996; Wholey et al. 1996), it is reasonable to believe that quality should also be influenced by market traits because, like price, quality is an endogenous outcome of the competitive process. While there is some evidence from hospital markets that relates hospital competition to inpatient costs and quality (Kessler and McClellan 2000; Kessler and Geppert 2005), evidence examining the relationship between competition and quality is much more limited in insurance markets. Studies examining the effects of insurance competition on premiums generally find that increased competition results in lower premiums (Wholey, Feldman, and Christianson 1995; Wholey et al. 1997; Pizer and Frakt 2002). Understanding whether competition will drive quality improvement, or whether lower premiums will sacrifice quality, is crucial for assessing whether quality improvement objectives can be achieved by market approaches.

In this paper, we use HMO quality measures from the Healthcare Effectiveness Data and Information Set (HEDIS) and the Consumer Assessment of Healthcare Providers and Systems Survey® (CAHPS) to measure plan quality. HEDIS and CAHPS are the foundation of most health plan report cards and can be examined empirically to assess whether increased HMO competition results in improved quality.

LITERATURE

Empirical evidence relating HMO competition to HMO quality is sparse (Schoenbaum and Coltin 1998; Morrisey 2001; Gaynor 2006). Several studies investigating factors related to HMO quality have focused on specific health plan or enrollee traits but have not incorporated market traits such as HMO competition (Himmelstein et al. 1999; Zaslavsky, Landon et al. 2000; Born and Simon 2001; Landon et al. 2001). Other literature has examined the percentage of variation in HEDIS and CAHPS measures attributable to the region, the market, or the health plan, without relating the variance to specific plan or market attributes (Zaslavsky, Hochheimer et al. 2000).

Scanlon et al. (2005, 2006a) examine whether market and plan characteristics are related to HEDIS and CAHPS performance using a cross-sectional design. Although careful to explore the sensitivity of results to inclusion of important covariates, the results are subject to bias if there are unobserved market traits correlated with both HMO competition and plan quality. The studies find no evidence that competition is associated with quality or improvement in quality over time. The current study extends the previous work by using fixed-effects models to adjust for any time-invariant traits that may mask a relationship between competition and quality.

CONCEPTUAL FRAMEWORK

Many plans attempt to influence and improve the HEDIS measures by using techniques such as provider and patient education, clinical guideline adoption, case and disease management, pay-for-performance and incentive programs, and decision support and electronic information systems (Scanlon et al. 2001). Likewise, plans can influence CAHPS measures by making investments in plan service that are visible to members.

The theoretical relationship between competition and quality is complex, in part because quality is only one of several endogenous variables, such as premiums and benefits packages, that plans choose when competing for enrollees. Gaynor (2006) discusses the relationship between competition and quality, and concludes that when premiums are regulated, as they are in many public sector programs, competition should be positively related to quality because firms will use quality to compete for enrollees. The magnitude of the effect of competition on quality depends on the quality elasticity of demand. When prices are not regulated, however, like in the commercial insurance market, economic theory is ambiguous about how competition and quality are related. Plans may attract enrollees with higher quality, better benefits, or lower premiums, with the total effect of each being a function of the relative price, benefit, and quality elasticities of demand (Gaynor 2006).

There may even be differential effects of competition on the HEDIS and CAHPS measures, and even differences among the HEDIS indicators, if elasticities for those different aspects of performance vary. For example, to the extent that competition drives plans to respond to consumer preferences, CAHPS measures might be more responsive to competition because these measures capture aspects of quality that are more salient to health plan members, while the HEDIS measures reflect what experts think is good quality of care, but may not be highly correlated with what consumers think about plan quality.

In the health insurance sector, there are several additional considerations. First, the role employers play in selecting the menu of plan options and their prices to employees may influence elasticities. The relevant elasticities combine both employee and employer preferences, depending on the extent to which employers act as agents for employees. Similarly, insurers may serve both public and commercial markets, and production processes may impede providing a different quality to each sector, resulting in plans' decisions being driven by their overall purchaser mix.

Second, improvements in quality may lead to lower costs in the future, although there is no solid evidence on this point (McLaughlin and Ginsburg 1998; Leatherman et al. 2003). Profit-maximizing firms should seek to exploit these efficiencies, regardless of competition; however, competition may affect the extent to which they do so because it may be associated with greater plan switching, thereby diminishing the ability of plans to capture these savings.

Third, competition among insurers may be associated with physicians serving fewer patients from any one insurer. This reduces the incentive plans have to undertake quality improvement initiatives because their efforts may spill over to patients from other plans (Beaulieu 2002) and may increase the costs associated with quality improvement initiatives if physicians are less responsive to messages from plans that cover fewer of their patients.

In this paper we do not identify the structural parameters that determine the relationship between competition and plan quality, but instead focus on the reduced form relationship. This is analogous to the literature examining the relationship between competition and premiums, which also focuses on reduced form relationships. Our analyses examine the important policy question concerning competition and quality, addressing the assertion that, despite the ambiguity of theory, competition will improve quality (while lowering premiums).

DATA AND MEASURES

Data

The primary data sources used to derive the analytic sample were National Committee for Quality Assurance's (NCQA) HEDIS data and the Interstudy Corporation's MSA Profiler and Competitive Edge data (calendar years 1998–2002). NCQA data included observations from all plans reporting HEDIS in NCQA's Quality Compass 1999 through Quality Compass 2003 (i.e., the publicly reporting plans), as well as plans that reported data to NCQA but requested that the information not be included in Quality Compass (i.e., the nonpublicly reporting plans). The HEDIS data reflect member health care encounters and survey responses occurring during the prior calendar year (e.g., 1999 for Quality Compass 2000 ).

For our analysis we included any plan reporting in one or more of the 5 years of our panel. Because NCQA and Interstudy do not use common health plan identifiers, we merged these two data sources manually, relying primarily on the health plan name, the state of service, and the Metropolitan Statistical Areas (MSAs) the plan served. We also performed manual checks of the data because there was a high rate of plan mergers, acquisitions, and name changes during this 5-year period of HMO consolidation.

Quality Measures

Our dependent variables are a subset of the HEDIS clinical process, outcome, and utilization measures as well as several CAHPS measures. The HEDIS measures are assessments, based on administrative claims and medical record data, of whether care deemed appropriate for specific populations was provided. While the HEDIS measures are the industry standard in terms of comparable data measuring HMO quality, they fail to capture some important aspects of the quality of care, including diagnostic proficiency, technical quality of care delivered, or patient severity and compliance with treatment recommendations. The CAHPS measures include assessments of quality from the perspective of consumers. For example, there are CAHPS measures for overall plan quality and for more specific aspects of performance such as claims processing and the quality of plans' contracted doctors. Although CAHPS fills gaps omitted by HEDIS, the union of these measures still leaves some dimensions of plan quality unmeasured.

We chose six HEDIS process measures and three CAHPS measures. The HEDIS measures were chosen because they are related to diabetes and cardiovascular disease, two important chronic illnesses that many health plans have focused on improving through “in house” or contracted disease management programs, and have heavily marketed these programs to employers. These were also two of the chronic illnesses for which McGlynn et al. (2003) found that patients received recommended care about 55 percent of the time. The CAHPS measures were chosen because they represent plan enrollees' opinions about plan quality as well as the quality of their health care and of their physician. Tables 1 and 2 list the nine measures included in our analyses, the percentage of plans reporting multiple years of data for each measure, and summary statistics for these measures. The mean value for each measure generally increased over time, although the magnitude of the change varied. In all of the models estimated, we assume that missing data on the outcome are missing at random, because jointly modeling the structure of the attrition process is complicated by the absence of valid exclusion restrictions in our data.

Table 1.

Number (%) of Plans Reporting Multiple Years of Data for each HEDIS and CAHPS Measure

Performance Measure	Number of Plans (%)						Total*
	No Report	1-Year	2-Year	3-Year	4-Year	5-Year
CAHPS health plan rating	26	166	83	72	91	140	552
	4.50	28.72	14.36	12.46	15.74	24.22
CAHPS health care rating	26	166	84	71	91	140	552
	4.50	28.72	14.53	12.28	15.74	24.22
CAHPS physician rating	26	166	84	72	90	140	552
	4.50	28.72	14.53	12.46	15.57	24.22
Comprehensive diabetes care—hemoglobin A1c screening	106	117	75	66	182	32	472
	18.34	20.24	12.98	11.42	31.49	5.54
Comprehensive diabetes care—annual eye exam rate	103	119	75	68	186	27	475
	17.82	20.59	12.98	11.76	32.18	4.67
Comprehensive diabetes care—lipid screening rate	107	117	75	62	190	27	471
	18.51	20.24	12.98	10.73	32.87	4.67
Comprehensive diabetes care—nephropathy monitoring rate	107	120	72	66	187	26	471
	18.51	20.76	12.46	11.42	32.35	4.50
β-blocker medication after heart attack	212	97	56	55	59	99	366
	36.68	16.78	9.69	9.52	10.21	17.13
LDL cholesterol screening rate	147	122	58	58	80	113	431
	25.43	21.11	10.03	10.03	13.84	19.55

^*Total represents number of unique plans reporting data for any HEDIS or CAHPS measure in any of the 5 years. Plans are only included in one category.

CAHPS, Consumer Assessment of Health Plans Survey; HEDIS, Healthcare Effectiveness Data and Information Set; β-blocker, Betablocker; LDL, low-density lipoprotein.

Table 2.

Summary Statistics for HEDIS and CAHPS Measures

HEDIS/CAHPS Measure	1998 (%)		1999 (%)		2000 (%)		2001 (%)		2002 (%)		Average Number of Plans Reporting
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
CAHPS health plan rating	57.60	8.63	57.01	8.63	59.44	8.02	61.89	7.85	61.27	8.11	259
CAHPS health care rating	70.21	6.04	70.21	6.21	71.99	6.09	73.19	5.72	75.17	5.57	322
CAHPS physician rating	71.74	5.06	72.76	4.65	74.25	4.55	74.62	4.50	75.01	3.99	322
Comprehensive diabetes care—hemoglobin A1c screening rate	73.19	13.26	75.04	10.83	78.44	9.83	81.43	7.64	82.67	9.38	271
Comprehensive diabetes care—annual eye exam rate	41.82	15.30	45.30	15.13	48.36	13.87	52.15	13.38	51.85	13.34	270
Comprehensive diabetes care—lipid screening rate	61.49	12.44	69.27	10.93	76.72	9.16	81.49	6.71	85.26	8.37	270
Comprehensive diabetes care—nephropathy monitoring rate	29.87	14.00	36.09	14.28	41.40	13.73	46.10	13.21	51.81	12.09	268
β-blocker medication after heart attack	80.36	11.87	85.21	10.82	89.65	8.70	92.75	6.21	93.76	7.98	221
LDL cholesterol screening rate	59.28	17.62	68.95	11.46	74.16	9.26	77.05	8.06	79.46	7.38	259

CAHPS, Consumer Assessment of Health Plans Survey; HEDIS, Healthcare Effectiveness Data and Information Set; β-blocker, Betablocker; LDL, low-density lipoprotein; SD, standard deviation.

Covariates

We measure insurance market competition by using measures common in the health services and health economics literature, including the Herfindahl–Hirschman Index (HHI) for the commercial HMO market and, alternatively, the number of HMOs competing in the market (Baker 2001; Scanlon et al. 2006b). The HHI is based on the market shares of all competitors in the insurance market; it ranges between zero (perfect competition) and 1 (monopoly).

We estimate the HMO HHI and the number of HMOs from the Interstudy data. For the HHI, we apportion plans' commercial HMO enrollment to the markets they serve. When the same firm offers more than one commercial HMO in a market, our HHI aggregates the enrollment to the firm level.

We also control for the HMO penetration rate reported by Interstudy, which includes the proportion of a market's insured population (e.g., Medicare, Medicaid, and commercial insurance) enrolled in an HMO. We include the penetration measure because higher values reflect greater acceptance of HMOs relative to preferred provider organizations (PPOs) or discounted fee-for-service products. Because HEDIS and CAHPS are measured primarily in HMOs, and because HMOs engage in various programs to improve their HEDIS and CAHPS scores, we hypothesize that markets with greater levels of HMO penetration may have better HEDIS and CAHPS results.

We also control for health plan and market traits (Table 3). Health plan traits allow us to control for differences among plans including whether the plan allows NCQA to publicly report HEDIS data, ownership status (for-profit versus not-for-profit), plan age, and the data collection method used for each HEDIS measure (hybrid versus administrative). The hybrid data reporting method uses both administrative data and patient chart data and is considered more accurate, but can be more expensive.

Table 3.

Summary Statistics for Model Covariates

	2000
Variable	Mean	SD	Minimum	Maximum
MSA commercial HMO HHI (=1 if monopoly, excludes Medicare and Medicaid)*	0.33	0.14	0.12	1.00
MSA number of HMOs*	11.15	4.64	1.0	23.41
MSA HMO penetration*	0.34	0.12	0.01	0.70
MSA hospital HHI* (=1 if monopoly)	0.25	0.14	0.05	0.85
For-profit health plan	0.75	0.43	0.00	1.00
Staff/group model plan (mixed model)	0.03	0.18	0.00	1.00
IPA/network model plan (mixed model)	0.64	0.48	0.00	1.00
Plan age (years)	15.23	8.75	0.30	57.00
Publicly reports HEDIS/CAHPS	0.72	0.45	0.00	1.00
Blue Cross Blue Shield affiliation*,†	0.16	0.36	0.00	1.00
Percentage of employees in MSA in large firms*,†	0.51	0.04	0.36	0.60
Percentage of federal employees in MSA*,†	0.02	0.02	0.01	0.16
Percentage of MSA population under age 5*,†	0.04	0.02	0.01	0.08
Percentage of MSA population age 65 and older*,†	0.06	0.03	0.01	0.15
MSA per capita income (thousands)*,†	28.02	4.22	12.76	51.87
Percentage of MSA population nonwhite*,†	0.18	0.10	0.01	0.68
β-blocker medication after heart attack—hybrid data collection	0.76	0.42	0.00	1.00
Comprehensive diabetes care—annual eye exam rate—hybrid data collection	0.96	0.20	0.00	1.00
Comprehensive diabetes care—hemoglobin A1c screening rate—hybrid data collection	0.96	0.20	0.00	1.00
Comprehensive diabetes care—lipid screening rate—hybrid data collection	0.96	0.20	0.00	1.00
Comprehensive diabetes care—nephropathy monitoring rate—hybrid data collection	0.96	0.20	0.00	1.00
LDL cholesterol screening rate—hybrid data collection	0.81	0.39	0.00	1.00

^*Variables weighted for plans serving multiple MSAs.

^†Variables included only in the cross-sectional models due to collinearity with the fixed effect (models 1 and 2 reported in the Appendix).

CAHPS, Consumer Assessment of Health Plans Survey; HEDIS, Healthcare Effectiveness Data and Information Set; HHI, Herfindahl–Hirschman Index; HMO, health maintenance organization; β-blocker, Betablocker; LDL, low-density lipoprotein; MSA, Metropolitan Statistical Area; SD, standard deviation.

Market-level covariates are included to account for potential demand differences across plans and include variables used in prior studies: the percentage of employees in the MSA working in large firms, the percentage of federal employees in the MSA, the percentage of the MSA population under age 5 and age 65 and older, the MSA per capita income, the percentage of the MSA population that is nonwhite, and the hospital HHI, computed based on hospitals' share of beds in a market. These variables are obtained (or computed) based on data from Interstudy, the Area Resource File, and the Bureau of Labor Statistics. In cases where the measurement of these variables does not vary over the panel, they are excluded from the fixed-effects models.

MSA Weights

The unit of analysis is the health plan. On average, plans operate in 5.06 MSAs (minimum=1, maximum=25 in 1999). We aggregate all MSA-level data to the plan level by taking the weighted average for each plan over the markets it serves. The weights are the plan-specific average total enrollments in each MSA over the study period based on Interstudy data. Because we use average enrollment, the weights are time invariant. For example, if in 1999 a plan had 40 percent enrollment in MSA 1 and 60 percent of its enrollment in MSA 2, and in 2000 it had 50 percent of its enrollment in each MSA, the aggregation of all market variables would be based on 45 percent enrollment in MSA 1 and 55 percent enrollment in MSA 2.

Figure 1 illustrates that there is longitudinal variation in our key measures, plotting the 1998 values against the 2002 values for plans reporting in both years, using constant weights based on average enrollment.

Figure 1.

Change in HMO HHI^*, Number of HMOs^* and HMO Penetration^* 1998–2002 (N=167)

*Computed at the plan level and weighted for MSA level enrollment for plans serving multiple MSAs Monopoly=1 for the HHI measure

METHODS

In our longitudinal data, there are three sources of variation in HMO competition. First, there is cross-sectional variation in competition that arises because different markets (e.g., MSAs) have different levels of competition at any point in time. Second, there is change in competition within MSAs over time. These changes are driven by a combination of plan mergers, plan entry, and plan exit. In models that use the HHI, changes in competition over time are also driven by changing market shares due to enrollment changes.

The third source of variation in competition arises because plans serve multiple MSAs. Even if competition within MSAs remains constant, plan-level competition may still change because of changes in the distribution of a plan's enrollment across MSAs over time. Because we use time-invariant MSA weights in our study, our results exclude this source of variation.

We estimate models that use only variation in competition within markets over time to identify the effects of competition. The general specification can be written as equation (1), where q_it is the quality of HMO i on a specific HEDIS or CAHPS measure at time t:

(1)

HMO HHI (and the number of HMOs) denotes HMO competition and PEN denotes HMO penetration. The X matrix denotes the vector of plan and market-level covariates discussed above. We also include dummy variables for year of observation (t=1999 is the reference).

The plan-level fixed effect is η_i. It absorbs much of the cross-sectional variation in competition, but some of the variation associated with different markets will remain because of changing enrollment patterns across MSAs. This variation is controlled for because we use time-invariant weights to aggregate covariates. Thus, the estimates rely only on longitudinal changes in competition within MSAs to identify the effect of competition on quality. We cannot estimate the effect of any plan-level time-invariant variable (such as HMO profit status) because these and other time-invariant traits are captured in the plan-level fixed effect.

Standard Errors and Multiple Comparison Tests

Because our unbalanced panel includes multiple observations from plans on each measure, we need to adjust the standard errors to account for correlation within plan observations over time on the various HEDIS and CAHPS measures. Standard errors are estimated using the bootstrapping technique shown to result in tighter standard errors relative to asymptotic standard errors (Goncalves and White 2005).

Because we have nine quality measures, our estimation yields multiple parameters measuring the relationship between competition and quality. This increases the likelihood that some coefficients will appear statistically significant by chance (Diaconis 1985). To overcome this problem, we utilize an approach, due to Bonferroni, that adjusts the type-I error rate to account for the multiple comparisons being made (Hochberg and Tamhane 1987).

RESULTS

HMO Competition

The estimates suggest that there is not a consistent relationship between HMO competition and quality (Tables 4 and 5). The estimates are small, generally not statistically significant, and even conflicting in sign. For example, estimates for the CAHPS measures suggest that more competition results in a worse plan rating, but in a better physician rating. Recall that a negative coefficient on HHI suggests that competition improves quality and in the models using the number of HMOs, a positive coefficient indicates that competition improves quality.

Table 4.

Plan Fixed-Effects Estimates (Constant Weight)—CAHPS Measures (Bootstrapped Standard Errors)

Covariate	CAHPS Health Plan Rating		CAHPS Health Care Rating		CAHPS Physician Rating
	HHI	#HMOs	HHI	#HMOs	HHI	#HMOs
HMO HHI	−0.656	NA	−1.185	NA	−2.594*	NA
	(1.913)	NA	(1.622)	NA	(1.324)	NA
Number of HMOs	NA	−0.225***	NA	−0.079	NA	−0.058
	NA	(0.078)	NA	(0.062)	NA	(0.054)
HMO penetration	−0.146	0.986	1.645	1.876	0.772	0.599
	(3.431)	(3.381)	(2.313)	(2.294)	(2.011)	(2.068)
Hospital bed HHI	−2.112	−2.467	1.571	1.035	1.101	0.435
	(3.160)	(3.036)	(2.086)	(2.136)	(2.389)	(2.380)
Allows NCQA to report publicly	1.566***	1.528***	1.991***	1.992***	1.483***	1.501***
	(0.538)	(0.484)	(0.404)	(0.409)	(0.386)	(0.384)
1998†	0.241	0.186	−0.305	−0.293	−1.134***	−1.121***
	(0.389)	(0.391)	(0.304)	(0.307)	(0.282)	(0.292)
2000	2.083***	1.998***	1.270***	1.223***	1.296***	1.230***
	(0.291)	(0.297)	(0.260)	(0.271)	(0.220)	(0.203)
2001	4.069***	4.182***	2.057***	2.086***	1.518***	1.513***
	(0.446)	(0.448)	(0.328)	(0.363)	(0.276)	(0.273)
2002	3.231***	3.040***	3.821***	3.679***	1.748***	1.534***
	(0.595)	(0.577)	(0.437)	(0.466)	(0.349)	(0.342)
Constant	57.701***	59.798***	68.019***	68.629***	72.238***	72.358***
	(2.743)	(2.575)	(1.675)	(1.803)	(1.328)	(1.490)
N	1,608	1,610	1,607	1,609	1,606	1,608
Within R2	0.18	0.18	0.27	0.27	0.18	0.18

^*Significant at 10%

^**Significant at 5%

^***Significant at 1%.

Bootstrapped standard errors based on 500 replications.

^†1999 is the omitted year.

CAHPS, Consumer Assessment of Health Plans Survey; HHI, Herfindahl–Hirschman Index; HMO, health maintenance organization; NCQA, National Committee for Quality Assurance.

Table 5.

Plan Fixed Effects Estimates (Constant Weight)—HEDIS Measures (Bootstrapped Standard Errors

Covariate	Hemoglobin A1c Screening†		Diabetes Eye Exam Process Measure†		Lipid Screening Process Measure†		Nephropathy Monitoring Process Measure†		LDL Cholesterol Screening Process Measure		β-Blocker Medication Rate
	HHI	#HMOs	HHI	#HMOs	HHI	#HMOs	HHI	#HMOs	HHI	#HMOs	HHI	#HMOs
HMO HHI‡	2.148	NA	−1.636	NA	−3.629	NA	−5.543	NA	−2.406	NA	13.267*	NA
	(5.256)	NA	(5.795)	NA	(4.603)	NA	(6.027)	NA	(6.364)	NA	(7.670)	NA
Number of HMOs‡	NA	−0.235	NA	0.063	NA	−0.146	NA	0.235	NA	0.063	NA	−0.828***
	NA	(0.254)	NA	(0.211)	NA	(0.199)	NA	(0.287)	NA	(0.236)	NA	(0.244)
HMO penetration‡	1.722	3.521	2.870	1.819	5.910	6.461	8.111	6.317	8.251	7.506	−6.882	−1.786
	(6.998)	(6.893)	(7.352)	(7.729)	(7.070)	(6.959)	(9.911)	(10.546)	(12.097)	(9.799)	(7.251)	(7.540)
Hospital bed HHI‡	4.340	5.111	−5.817	−5.756	2.767	1.895	7.778	5.588	−25.433**	−25.798***	−13.723*	−8.301
	(11.119)	(11.346)	(9.418)	(9.906)	(11.028)	(10.360)	(14.309)	(15.720)	(11.671)	(9.245)	(7.387)	(7.184)
Allows NCQA to report publicly	3.684**	3.536**	4.371***	4.523***	1.439	1.452	−0.289	−0.161	−0.743	−0.696	4.333***	4.095***
	(1.692)	(1.740)	(1.526)	(1.490)	(1.121)	(1.049)	(1.754)	(1.650)	(1.522)	(1.150)	(1.357)	(1.307)
Hybrid data collection	3.718*	3.826*	2.872	2.810	5.392**	5.484**	5.199**	5.219**	8.511***	8.546***	13.667***	13.910***
	(2.072)	(1.960)	(3.204)	(3.277)	(2.431)	(2.590)	(2.461)	(2.581)	(2.077)	(1.660)	(2.060)	(1.900)
1998§	−3.604***	−3.740***	−4.084**	−4.028**	−9.690***	−9.676***	−6.810***	−6.572**	−5.552***	−5.512***	−1.904**	−2.382**
	(1.153)	(1.239)	(1.636)	(1.591)	(1.554)	(1.726)	(2.492)	(2.592)	(1.226)	(1.057)	(0.925)	(0.936)
2000§	3.148***	3.126***	2.202***	2.180***	7.806***	7.697***	6.124***	6.096***	5.464***	5.438***	NA	NA
	(0.588)	(0.582)	(0.619)	(0.597)	(0.561)	(0.548)	(0.762)	(0.778)	(0.852)	(0.674)	NA	NA
2001§	NA	NA	NA	NA	NA	NA	NA	NA	8.687***	8.624***	5.756***	6.261***
	NA	NA	NA	NA	NA	NA	NA	NA	(0.991)	(0.830)	(0.699)	(0.730)
2002§	5.882***	5.879***	4.181***	4.127***	15.797***	15.473***	16.787***	16.558***	NA	NA	NA	NA
	(1.224)	(1.135)	(1.035)	(1.006)	(0.993)	(0.948)	(1.258)	(1.287)	NA	NA	NA	NA
Constant	66.957***	69.485***	40.019***	39.049***	61.193***	61.739***	33.505***	30.280***	65.867***	64.718***	71.930***	82.350***
	(3.844)	(3.769)	(4.541)	(5.334)	(4.400)	(4.229)	(5.663)	(6.241)	(6.435)	(6.386)	(5.493)	(4.877)
N	1,044	1,046	1,039	1,041	1,040	1,042	1,032	1,034	1,032	1,032	645	645
Within R2	0.25	0.25	0.15	0.15	0.62	0.62	0.43	0.43	0.48	0.47	0.48	0.47

^*Significant at 10%.

^**Significant at 5%.

^***Significant at 1%.

Bootstrapped standard errors based on 500 replications.

^†For Betablocker medication, diabetes hemoglobin A1c screening, diabetes eye exams, diabetes lipid screening, LDL cholesterol screening and diabetes nephropathy monitoring plans were allowed to rotate measures (2001 for diabetes measures, 2000 and 2002 for β-blocker medication, and 2002 for LDL cholesterol screening). All observations in years where plans were permitted to rotate were dropped from the analysis (and thus a “NA” is included for the year covariates).

^‡HMO HHI, the number of HMOs, Hospital bed HHI, and HMO penetration rate were aggregated to plan level using MSA weights averaged over 1998–2002.

^§1999 is the omitted year.

HEDIS, Healthcare Effectiveness Data and Information Set; HHI, Herfindahl–Hirschman Index; HMO, health maintenance organization; NCQA, National Committee for Quality Assurance.

The strongest result is for the CAHPS physician rating when HHI is used (Table 4), although the magnitude of the effect of competition is quite small, with the estimated effect of moving from monopoly (HHI=1) to perfect competition (HHI=0) only resulting in an increase of about 2.6 percentage points. The coefficients for the other CAHPS variables are not statistically significant. Using Bonferroni's inequality to test, we cannot reject the hypothesis that HMOHHI has no effect on the CAHPS measures. When the number of HMOs instead of HHI is used, the estimates suggest the opposite conclusion, an inverse relationship between competition and quality for the CAHPS measures, although only the coefficient in the health plan rating model is statistically significant.

The coefficients for the HEDIS variables vary in sign and are not statistically significant for all HEDIS measures, except the β-blocker (Betablocker) medication rate. For the β-blocker medication rate, both measures of competition suggest that performance decreases with greater competition. For example, using the HHI, the estimated effect of moving from monopoly to perfect competition is a 13.3 percentage point decrease in the β-blocker medication rate.

HMO Penetration

None of the coefficients achieves statistical significance. Using Bonferroni's adjustment and the results in Tables 4 and 5, we cannot reject the null hypothesis that HMO penetration has no effect on the CAHPS measures (critical t-value=2.4) or on the HEDIS measures (critical t-value=2.64).

Robustness

We conducted four sensitivity analyses to assess the robustness of our estimates. These tests examined the impact of (a) dropping plans with outlier values on the HEDIS and CAHPS measures, (b) using robust versus bootstrapped standard errors, (c) examining quality above and below a threshold value rather than continuous levels of performance on HEDIS and CAHPS measures, and (d) using lagged values of HHI or number of HMOS to instrument for current values.

Outliers were defined as plans with scores on specific HEDIS/CAHPS measures that exceeded 10 percentage points annually in absolute value, on the assumption that these changes may be unrealistic and thus may possibly be due to measurement error. The results with outliers dropped do not differ significantly from our base results, which include all plan observations.

In addition to bootstrapping, we also computed the standard errors using the Huber–White (White 1980) robustness computation. While, as expected, the standard errors are smaller when the bootstrapping method is used, the differences are not substantial enough to affect the conclusions on most of the covariates of interest.

To test whether competition is related to a plan being above or below a quality threshold rather than associated with a continuous value, we constructed a dichotomous dependent variable that identifies whether a plan's HEDIS score was in the top quartile of all plans for a given year. Our fundamental conclusions did not change with this new specification. Finally, consistent with our conclusions from the fixed-effects models, the instrumental variables (IV) models, which required dropping a year of data, did not provide evidence that competition improved quality.

Comparisons with Other Models

We also compare our results with three other models that differ in the source of variation to identify the effect of competition on quality. The first is a simple cross-sectional model for 1999, without time dummies. The second is a random-effects model using the 5-year panel of data, identified from a combination of cross-sectional and longitudinal variation. And the third also is a fixed-effects model, similar to our base model, but aggregating market covariates each year using weights based on actual MSA market shares, as opposed to constant market share weights. These estimates thus incorporate some of the variation associated with plans' enrollment changing across markets over time.

None of the models suggests that competition (HHI or number of HMOs) improves quality (results available in an electronic appendix, supporting information Tables S1–S10). The fixed-effects model using actual, as opposed to constant MSA weights generates results similar to those we report, with the weighting scheme mattering in only a few cases. The 1999 cross-sectional results are consistent with prior literature (Scanlon et al. 2005) and suggest that increased levels of competition have deleterious effects on the CAHPS measures. Similarly, the longitudinal random-effects model suggests that if a relationship between competition and quality exists, it is negative.

The two models that rely primarily on cross-sectional variation indicate that higher HMO penetration is associated with better HEDIS performance. This finding is not supported in the plan-level fixed-effects models that we estimate. This pattern of results suggests bias due to omitted variables in study designs that use cross-sectional variation (either partly or exclusively), suggesting the importance of omitted time-invariant plan characteristics for which the plan fixed-effects models attempt to control.

DISCUSSION

We found no evidence of a strong and consistent relationship between HMO competition (measured either by the HHI or the number of HMOs) and plans' scores on the CAHPS and HEDIS measures of health plan performance. While our reduced form analysis prevents us from identifying the specific reason(s) why competition does not positively impact quality, our conceptual framework suggests possible explanations for this result.

First, our estimates are consistent with the hypothesis that the elasticity with respect to quality is small (or at least small for these measures of quality), relative to the premium elasticity. A low elasticity with respect to quality may reflect the lack of salience of the particular quality measures used or the challenges associated with informing the public about quality scores. In this case, health plans would compete primarily on premium rather than on quality. The key policy question is whether the weight given to quality, relative to premiums, is fully informed and thus optimal from the perspective of purchasers, consumers, and society, or whether quality is undervalued because of an information asymmetry.

Second, it may be the case that competition affects the costs of producing quality. If more fragmentation increases the costs of altering physician behavior or decreases plan incentives to alter physician behavior and to invest in quality improvement efforts, we may see no effect of competition on quality.

There are several important limitations with our study. First, the performance measures used are incomplete measures of quality. Because quality is multidimensional, a latent variable approach may be advantageous. Second, the literature has recognized the complexity of measuring competition (Baker 2001; Robinson 2004; Gaynor 2006; Scanlon et al. 2006b) and our competition measures, which follow the literature, suffer from common imperfections. For example, because of lack of data, we do not consider the role of alternative insurance products such as PPOs. Our measures also ignore the role of potential entrants, which contestability theory would suggest may matter (Baumol, Panzar, and Willig 1982). Our competition measures may also be noisy if the MSA is not the appropriate geographic unit to define competition. For example, plans may compete for national contracts more aggressively than for local business. Thus, corporate initiatives driven by competition defined more broadly than at the MSA level may generate improvement in quality; this may cause benefits to spill over into markets that, in themselves, are not very competitive.

Third, our competition measures may be endogenous or our models may suffer from omitted variable bias. For example, if HMO quality influences plan market share, the HHI measure will be endogenous. Even if reverse causality is not a major concern, time-varying omitted variables may bias the results. The robustness of the results to use of IV models, or cross-sectional variation, mitigates but does not eliminate, this concern.

Despite these limitations, our results have potentially important policy implications for those interested in increasing quality generally, and HMO performance specifically. Many policy analysts believe that the combination of increased HMO competition and the availability of standardized performance information should lead to better performance on measurable aspects of quality (e.g., Enthoven 1993, 2004). Our results do not lend support for this hypothesis under the current structure of employer-based commercial insurance.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1

Author Matrix.

Table S1

CAHPS Health Plan Rating (Bootstrapped Standard Errors).

Table S2

CAHPS Health Care Rating (Bootstrapped Standard Errors).

Table S3

CAHPS Physician Rating (Bootstrapped Standard Errors).

Table S4

HEDIS Hemoglobin A1c Screening¹ Process Measure (Bootstrapped Standard Errors).

Table S5

HEDIS Diabetes Eye Exam¹ Process Measure (Bootstrapped Standard Errors).

Table S6

HEDIS Lipid Screening¹ Process Measure (Bootstrapped Standard Errors).

Table S7

HEDIS Diabetes Nephropathy Monitoring¹ Process Measure (Bootstrapped Standard Errors).

Table S8

HEDIS LDL Cholesterol Screening¹ Process Measure (Bootstrapped Standard Errors).

Table S9

HEDIS Beta Blocker¹ Process Measure (Bootstrapped Standard Errors).

Table S10

Tests of Significance for Competition Coefficients (Using the Bonferonni Correction for Multiple Comparisons⁺).

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.