Do Commercial Managed Care Members Rate Their Health Plans Differently than Medicaid Managed Care Members?

Abstract

Objective

To determine if members of commercial managed care and Medicaid managed care rate the experience with their health plans differently.

Data Sources

Data from both commercial and Medicaid Consumer Assessment of Health Plan Surveys (CAHPS) in New York State.

Study Design

Regression models were used to determine the effect of population (commercial or Medicaid) on a member's rating of their health plan, controlling for health status, age, gender, education, race/ethnicity, number of office visits, and place of residence.

Data Collection

Managed care plans are required to submit to the New York State Department of Health (NYSDOH) results of the annual commercial CAHPS survey. The NYSDOH conducted a survey of Medicaid enrollees using Medicaid CAHPS.

Principal Findings

Medicaid managed care members in excellent or very good health rate their health plan higher than commercial members in excellent or very good health. There is no difference in health plan rating for commercial and Medicaid members in good, fair, or poor health. Older, less educated, black, and Hispanic members who live outside New York City are more likely to rate their managed care plan higher.

Conclusions

Medicaid members rating of their health care equals or exceeds ratings by commercial members.

Managed care has been the dominant health service delivery system for commercially insured working adults and their families for many years. It is increasingly becoming the delivery system for states' Medicaid and State Child Health Insurance Program (SCHIP) populations. States are turning to managed care for the same reason as employers—to provide a means to control spiraling costs while providing greater accountability for health outcomes. In addition, managed care holds the promise of improving access to and quality of care for traditionally underserved populations. By expanding provider networks to the Medicaid population there is the potential to improve access to the delivery systems for the publicly insured.

New York State (NYS) is no exception to this trend. Managed care has been an option for Medicaid enrollees since 1984, and by 1995 it had grown to where more than 600,000 Medicaid recipients voluntarily enrolled in a managed care plan. Since 1997, NYS has been enrolling Medicaid recipients under the authority of an 1115 waiver granted by the Center for Medicare and Medicaid Services (CMS) and as of September 2002 approximately 50 percent of all eligible Medicaid recipients were enrolled in a managed care plan. The waiver allows the state to mandate enrollment for most of the recipients eligible for Aid to Families with Dependent Children and exempts those eligible for SSI and those chronically ill, such as with serious and persistent mental illness or HIV disease.

While commercial enrollment in the state has been relatively stable for several years (at 5.5 million or 53 percent of all eligibles) it is expected that Medicaid enrollment in managed care will increase from 1.1 million in September 2002 to over 1.8 million by the end of 2003. The majority of this new growth will be attributed to the final phase-in of the 1115 waiver in the New York City region.

The state currently certifies 29 managed care plans to serve Medicaid recipients. Of these, 18 are Prepaid Health Service Plans (PHSPs) that restrict enrollment to publicly funded programs such as Medicaid and the State Child Health Insurance Program (SCHIP). Many of the PHSPs are provider sponsored and clinic based. The remaining 11 plans are HMOs with Medicaid product lines. These plans are more likely to be located in upstate New York while the provider-sponsored PHSPs are more likely to be located in New York City. Approximately 52 percent of Medicaid managed care enrollees are in a PHSP.

As this transformation in the state's Medicaid program occurs there is heightened interest on the part of policymakers, advocates, and legislators in evaluating the experience of care of Medicaid recipients in managed care. One way to do this is through the administration of standardized consumer satisfaction surveys, such as the Consumer Assessment of Health Plans Survey (CAHPS^®).

The CAHPS was developed by a consortium of researchers from Research Triangle Institute, RAND, Harvard Medical School, and WESTAT with funding from the Agency for Healthcare Research and Quality and CMS (Agency for Health Care Policy and Research 1999). The CAHPS assesses health plan enrollee experiences with access, services, providers, and overall satisfaction with their health plan. There are CAHPS tools available for use with commercial and Medicaid populations, thereby giving researchers a means to compare the experiences between the two populations and to evaluate whether some of the goals related to moving Medicaid recipients into managed care, such as improved access, have been met.

Many factors have been shown to influence health care satisfaction. Studies show that older, less-educated, and healthier patients are more satisfied with their health care (Zahn et al. 2002; Buchner and Probst 1999; Carlson et al. 2000; Zaslavsky et al. 2001; Weisman et al. 2001; Mainous, Griffith, and Love 1999; Hall and Dornan 1990; Cleary and McNeil 1988). Research on gender differences in health care satisfaction shows mixed results. Females are more satisfied then males in many studies (Zahn et al. 2002; Carlson et al. 2000; Cleary, Zaslavsky, and Cioffi 2000; Weisman et al. 2001; Mainous, Griffith, and Love 1999; Cleary and McNeil 1988), but other studies have shown no difference between women and men (Buchner and Probst 1999; Zaslavsky et al. 2001; Hall and Dornan 1990). The association between race/ethnicity and health care satisfaction is also not clear. In some studies, minorities, in particular blacks and Hispanics, tend to rate their health care higher than whites (Zahn et al. 2002; Morales et al. 2001; Carlson et al. 2000; Weisman et al. 2001), yet other studies show no differences (Mainous, Griffith, and Love 1999; Hall and Dornan 1990) among minorities and whites.

A managed care satisfaction survey conducted in Utah has shown that Medicaid enrollees rated their health care higher than commercial care enrollees (Ipsen et al. 2000). In a study in Kentucky, patient satisfaction was found to be no different among three levels of insurance: uninsured, Medicaid, and commercial insurance (Mainous, Griffith, and Love 1999). Other research on the effect of socioeconomic status (SES) and health care satisfaction show no consistent pattern. Lower SES, defined by annual family income, was found to be associated with higher health care satisfaction (Carlson et al. 2000). Hall and Dornan (1990) have shown no consistent association between measures of SES and health care satisfaction.

This study is the first to use CAHPS data to determine whether commercial and Medicaid enrollees rate their managed care plan differently on member satisfaction, while controlling for various demographic variables such as age, gender, race, and health status.

Methods

Data Sources

Information for this study came from two CAHPS data sources: commercial managed care CAHPS and Medicaid managed care CAHPS. The New York State Department of Health (NYSDOH) requires commercial managed care plans to submit CAHPS satisfaction survey data for their adult population each calendar year. Commercial managed care plans were required to use the CAHPS 2.0H instrument and survey protocol as outlined in HEDIS 2001 (National Committee for Quality Assurance 2000a). The managed care plans must hire independent National Committee for Quality Assurance (NCQA) accredited survey firms to administer the CAHPS survey, and upon completion, send the results to NYSDOH. The survey population was composed of current plan members as of December 31, 1999, who were continuously enrolled for 12 months and who were 18 years of age or older. In 1999, there were 23 plans in NYS who were required to conduct this survey. The response rates ranged from 38 percent to 65 percent with an overall response rate of 51 percent.

The NYSDOH contracted with Roper Starch Worldwide Inc. to survey enrollees in the state's Medicaid managed care program. The Medicaid managed care CAHPS survey was conducted statewide in the spring of 2000, using the 2.0H Medicaid CAHPS protocol with some minor modifications. To increase the response rate, a financial incentive of two dollars was used. In addition, seven New York State-specific questions were added, and also a smaller sample size than the recommended CAHPS sample size was used. One thousand enrollees from each of the 30 plans were surveyed, of which two-thirds were parents of children aged 0–12 and one-third were adults aged 18–64. The survey design reflected the distribution of children and adults in the Medicaid Managed Care program. Each member had to be enrolled in a Medicaid managed care plan as of December 31, 1999, and continuously enrolled for at least six months. The response rates ranged from 33 percent to 60 percent for the 30 plans, with an overall response rate of 43 percent.

The information from the two adult CAHPS studies was combined for the analysis in this report, with the following necessary adjustments. Excluded were commercial respondents older than 64 years of age to make it comparable to Medicaid, which also did not include anyone older than 64. A total of 1,272 (1,028 commercial and 244 Medicaid) responses from enrollees who did not answer questions on age, sex, race/ethnicity, education, number of office visits, self-assessed health status, and rating of health plan were deleted. A total of 11,749 commercial and 3,357 Medicaid surveys were used for the analysis.

The CAHPS survey question of interest for this study was the following:

We want to know the rating of all your experience with your health plan. Use any number from 0 to 10 where 0 is the worst health plan possible and 10 is the best health plan possible. How would you rate your health plan now? (Scale of 0 through 10) (Agency for Health Care Policy and Research 1999)

Analysis was performed to evaluate if there are differences between Medicaid and commercial members' rating of their health plans. At the time of the study, there were 14 plans that served only the commercial population, 21 plans that served only the Medicaid population, and 9 plans that served both commercial and Medicaid in New York State. The analysis included all managed care plans to determine if rating of health plans differed between Medicaid and commercial managed care in any setting.

Variables

The independent variables in this study included: population (commercial or Medicaid), self-assessed health status (health status), age, gender, education, race/ethnicity, number of visits to doctor's office, place of residence, and population served. The respondents were asked to rate their overall health as excellent, very good, good, fair, or poor. Respondent's age was categorized as 18–24, 25–34, 35–44, 45–54, and 55–64. Education was categorized as less than eighth grade, some high school or high school graduate, some college or four-year college graduate, and more than four-year college degree. Respondents were assigned to four racial/ethnic categories (Hispanic, white, black, and other), based on the following hierarchy. Members who responded that they were of Hispanic or Latino origin descent were categorized as Hispanic regardless of race. For non-Hispanic members, race/ethnicity is categorized by race alone. For respondents who checked multiple races on the survey, the hierarchy of black/African American, white, and other was used to categorize them into only one race/ethnicity category. Asian, Native Hawaiian, or other Pacific Islander and American Indian or Alaska Native respondents were all placed into the “other” category due to small sample sizes. Four office visit categories of none, one or two visits, three or four visits, and five or more visits were constructed from a survey question that asked the respondent how many times they went to a doctor's office or clinic. Based upon the respondent's address, the place of residence was categorized as New York City or Rest of State. Finally, population served is defined as managed care plans that serve Both Commercial and Medicaid enrollees, and managed care plans that serve Only Commercial or Only Medicaid enrollees.

Statistical Analysis

All analysis was conducted using SAS software (SAS Institute Inc. 1999). Because managed care plans in New York range in size from under two thousand to more than one million members, the results were weighted by enrollment. Analysis of variance was used to determine the relationship between each independent variable and health plan satisfaction. The mean health plan satisfaction results for commercial and Medicaid managed care populations by health status were compared using t-tests. To test the effect of population on member satisfaction, multivariate regression models were used controlling for the other independent variables. Multivariate regression models were developed using SAS SURVEYREG, a procedure that accounts for the complex survey design of this study. The commercial and Medicaid CAHPS surveys are collected with the managed care plan as the primary sampling unit. The SAS SURVEYREG procedure allowed for the regression coefficients to be produced among data that is stratified (by plan), and data that is not equally weighted.

Results

Table 1 shows the percent distribution and means for rating health plans for each of the independent variables. For health status, age, education, race/ethnicity, number of office visits, place of residence, and population served, there was a statistical significance ( p<.0001) of how respondents rate their health plan. The biggest disparity occurred with health status. We see that respondents who rated their health status as poor, gave their health plan an average rating of 6.34 on a scale of 0 to 10, whereas, respondents who rated their health status as excellent, gave their health plan an average rating of 7.72. Other notable differences were found with age, education, race, number of office visits, and place of residence. Older respondents, aged 55–64, had an average plan rating of 7.98 versus those aged 18–24 of 7.40. Respondents with more than a college degree rated their health plan the lowest (7.02) versus respondents with less than an eighth grade education (7.80) or those who graduated high school (7.89). Respondents who were black (7.64) or Hispanic (7.54) rated their plans slightly higher than whites (7.46). Respondents designated as other race/ethnicity rated their health plans the lowest at 7.01. Although statistically significant, there was only a slight difference in plan ratings depending on the number of office visits. Respondents with no visits rated their health plan slightly lower (7.28) than respondents who had 1 or 2 (7.52), 3 or 4 (7.51), or 5 or more (7.44) visits. Respondents living in New York City gave a lower plan rating (7.18) versus respondents living in Rest of State (7.59). Enrollees of managed care plans that serve both commercial and Medicaid rated their health plan higher (7.74) than enrollees of plans that serve only commercial or only Medicaid enrollees (7.23). We found no statistical difference for gender and population in how members rate their health plan.

Table 1.

Rating of Health Plan by Selected Characteristics

	Rating of Health Plan
	Percent	Mean	P-Value
Population
Commercial	95.43	7.46	0.0773
Medicaid	4.57	7.60
Health Status
Excellent	19.91	7.72	<.0001
Very Good	39.70	7.55
Good	30.81	7.38
Fair	8.20	6.99
Poor	1.38	6.34
Age
18–24	6.07	7.40	<.0001
25–34	18.94	7.20
35–44	29.36	7.27
45–55	26.09	7.52
55–64	19.55	7.98
Gender
Male	38.15	7.48	0.6901
Female	61.85	7.46
Education
Less than 8th grade	1.42	7.80	<.0001
Some high school or high school graduate	30.34	7.89
Some college or college graduate	49.09	7.37
Greater than 4-year college	19.15	7.02
Race/Ethnicity
White	74.53	7.46	<.0001
Black/African American	12.27	7.64
Hispanic	8.57	7.54
Other	4.63	7.01
Number of Office Visits
None	12.86	7.28	<.0001
1 or 2	33.73	7.52
3 or 4	29.84	7.51
5 or more	23.58	7.44
Place of Residence
Rest of State	29.78	7.59	<.0001
New York City	70.22	7.18
Population Served
Only Commercial or Medicaid	54.16	7.23	<.0001
Both Commercial and Medicaid	45.84	7.74

Figure 1 shows the mean rating of a health plan by population and health status. Medicaid managed care members in excellent or very good health rated their health plan higher than commercial members in excellent or very good health. For respondents in good, fair, and poor health, there was no statistical difference between commercial and Medicaid members.

Figure 1.

Rating of Health Plan By Health Status and Population

Table 2 displays the results of the multivariate regression model. An interaction between population and health status was added to the model due to the results shown in Figure 1. The interaction of population and health status, age, education, race/ethnicity, place of residence, the number of office visits, and population served were all significant. Members that were in excellent, very good, and good health for both populations, 55–64 years old, have less than a four-year college degree, were black or Hispanic, had one or more office visits in the last year, and live outside New York City, are more likely to rate their managed care plan higher. The parameter estimates of the interaction between population and health status showed that Medicaid members that were in excellent or very good health (parameter estimates 2.73 and 2.03) rated their health plan higher than commercial members (parameter estimates 1.97 and 1.68) did.

Table 2.

Multivariate Analysis of Rating of Health Plan

	Regression Coefficient	Standard Error	P-Value
Intercept	5.96	0.41	<.0001
Population/Health Status
Medicaid—Excellent	2.73	0.40	<.0001
Medicaid—Very Good	2.03	0.40	<.0001
Medicaid—Good	1.54	0.39	<.0001
Medicaid—Fair	1.01	0.40	0.0126
Medicaid—Poor	0.60	0.48	0.2162
Commercial—Excellent	1.97	0.39	<.0001
Commercial—Very Good	1.68	0.38	<.0001
Commercial—Good	1.35	0.38	0.0004
Commercial—Fair	0.82	0.40	0.0378
Commercial—Poor*	0.00	0.00
Age
18–24	−0.73	0.15	<.0001
25–34	−0.77	0.10	<.0001
35–44	−0.75	0.08	<.0001
45–54	−0.44	0.08	<.0001
55–64*	0.00	0.00
Gender
Female	−0.02	0.06	0.7217
Male*	0.00	0.00
Education
Less than or equal to 8th grade	0.72	0.30	0.0162
Some high school or high school graduate	0.83	0.09	<.0001
Some college or 4-year college graduate	0.36	0.08	<.0001
Greater than 4-year college degree*	0.00	0.00
Race/Ethnicity
Black	0.34	0.10	0.0007
Hispanic	0.37	0.14	0.0074
Other	0.02	0.15	0.8803
White*	0.00	0.00
Number of Office Visits
1 or 2 times	0.32	0.10	0.0007
3 or 4 times	0.33	0.10	0.0007
5 or more times	0.38	0.11	0.0004
None*	0.00	0.00
Place of Residence
New York City	−0.33	0.09	0.0001
Rest of State*	0.00	0.00
Population Served
Only Commercial or Medicaid	−0.42	0.06	<.0001
Both Commercial and Medicaid*	0.00	0.00

^*Reference group

A nonresponse analysis was conducted for the Medicaid survey to compare persons that responded to the survey to those that did not. Responders and nonresponders to the survey had similar demographic characteristics including age, gender, and Medicaid aid category. A similar analysis was not possible for the commercial survey.

Discussion

The results of this study show that commercial and Medicaid managed care members in excellent or very good health rate their plan differently, while members in good, fair, or poor health rate their health plan the same, regardless of population. In interpreting this relationship, we suggest that one possible explanation for the difference in health plan satisfaction for members in excellent and very good health is not a difference in the actual health care experience, but in perceived experience or expectation about the health care that is delivered. This relationship holds for all managed care plans, including those that serve both commercial and Medicaid populations, and plans that serve commercial-only or Medicaid-only populations. Previous research has suggested that socioeconomic status, such as population in our study, affect individuals' perception of health care quality (Hall and Dornan 1990). Carlson states that this difference may be due to higher-SES members having higher expectations than their lower-SES counterparts (Carlson et al. 2000). However, why this “expectation” only holds true in our study for persons in very good or excellent health is not clear.

We found that members aged 55–64 were more likely to rate their health plan higher than younger members, which was similar to previously published studies (Buchner and Probst 1999; Carlson et al. 2000; Zaslavsky et al. 2001; Mainous, Griffith, and Love 1999; Hall and Dornan 1990; Cleary and McNeil 1988). Zahn et al. (2002) and Zaslavsky, Zaborski, and Cleary (2000) each found a linear effect of increased satisfaction with increased age. Our study did not find a linear association between age and member satisfaction, but found a dichotomy below and at or above age 55. Gender was not found to be a significant predictor of health plan satisfaction as it was for Buchner and Probst (1999), Zaslavsky et al. (2001), and Hall and Dornan (1990). Enrollees with the highest level of education (greater than a four-year degree) were less satisfied with their health plan than the other education categories were. This is different from previous studies (Zahn et al. 2002; Buchner and Probst 1999; Carlson et al. 2000) that have shown a linear effect of education. Managed care members who are minorities are more likely to be satisfied with their health plan, similar to previous studies (Zahn et al. 2002; Morales et al. 2001; Weisman et al. 2001). Our results show that members who had one or more office visits are more satisfied than those with none, similar to the results found by Carlson et al. (2000). Finally, members in New York City are less satisfied with their health care plan then members in the Rest of State. The cause for this is unclear, however consistent with other findings (New York State Department of Health 2001).

There are many potential reasons why Medicaid members in excellent and very good health rate their health plan higher than their commercial counterparts. Medicaid members may be more satisfied due to the increased number of providers available to them in Medicaid managed care. Medicaid managed care members in New York report higher levels of access and satisfaction compared to the fee-for-service Medicaid program (Sisk et. al. 1996). In New York State, Medicaid managed care members have no copayments, and have access to a very comprehensive benefit package. Commercial members have increasing premiums, copays, and more benefit limitations than Medicaid members.

Our study shows that enrollees in managed care plans that serve both commercial and Medicaid populations rate their plan higher than enrollees in commercial-only or Medicaid-only managed care plans. Whether this is a function of the provider networks these plans engage, or specific health plan characteristics, is not clear. With respect to the plans, it may be that those with multiple products need to provide a greater diversity of member service functions given the different structural demands of the programs and the populations they enroll. This could result in a greater array of services, and improved responsiveness to individual member needs. In terms of the clinical networks, many (though certainly not all) plans engaged in both businesses have created increased access to private practitioners for their public program enrollees, while maintaining access to “traditional” Medicaid providers. This increased choice may result in improved satisfaction for Medicaid enrollees. Characteristics of health plans associated with higher quality, including satisfaction, is an area in need of future research and exploration. The answers could have policy implications as well as practical ones for health plan improvement.

There are limitations to this study. The study is based on a sample of members from 23 commercial and 30 Medicaid managed care plans in New York State, and may not reflect differences in these populations nationwide. The response rates for commercial and Medicaid managed care plans are 51 percent and 43 percent, respectively. Analysis of the nonresponders was not possible for the commercial population due to the method of data collection. The effect of the different continuous enrollment criteria (twelve months or more for commercial enrollees, six months or more for Medicaid enrollees) is not clear. However, in an analysis not shown here, models including only Medicaid enrollees with twelve or more months of continuous enrollment show similar results as those presented. Finally, the data from the survey is all self-reported by managed care members, and may not always be reliable, or reflect objective measures of quality.

Our study used CAHPS, a national standardized survey that has been rigorously tested, has a standard set of survey protocols, and has been administered by over three hundred managed care plans (National Committee for Quality Assurance 2000b). Independent survey firms collected the data for the commercial managed care plans, while a single vendor collected the Medicaid data under the same protocol as outlined by NCQA and CAHPS.This study provides a comprehensive assessment of commercial and Medicaid managed care in New York State. This study uses both commercial and Medicaid CAHPS data to determine population differences in health plan satisfaction. Other studies that are based on CAHPS (Carlson et al. 2000; Morales et al. 2001; Weisman et al. 2001; Zahn et al. 2002; Zaslavsky, Zaborski, and Cleary 2000; Zaslavsky et al. 2001) have been based on commercial CAHPS only, or a combined population that did not analyze population differences. Unlike national CAHPS data, all plans are required to submit CAHPS survey results to the NYSDOH for their commercial members. Survey results from all commercial and Medicaid managed care plans have been publicly reported (New York State Department of Health 2001). Finally, our findings are consistent with previous literature showing that socioeconomic status influences member satisfaction with their health plan.

As New York State transitions from a conventional fee-for-service Medicaid program to managed care, tools such as CAHPS are being used to evaluate access to care and quality of care being delivered by managed care plans. These results show that Medicaid managed care members are at least as satisfied with their health plans as commercial members, and for those in excellent or very good health, are more satisfied than their commercial counterparts. The CAHPS surveys will be used to monitor health plan satisfaction to determine if these findings are consistent over time.