Relationships Among Performance Measures for Medicare Managed Care Plans

Abstract

The Health Care Financing Administration (HCFA) has relied primarily on the Health Employer Data Information Set (HEDIS^®), the Consumer Assessment of Health Plans Study (CAHPS^®), and the Medicare Health Outcomes Survey (HOS) to track health plan performance. However, many relationships among these measures are unknown. We found significant relationships between four HEDIS^® measures and many items in the CAHPS^® measure as well as items in HOS concerning beneficiary general health ratings. Our study suggests that interpretation of performance data is improved by integrating access, effectiveness of care, beneficiary experience, health status, and risk measures into an analytic framework.

Introduction

The development of performance measures for Medicare managed care plans, coupled with internal steps to develop similar measures for the traditional program, have been critical to HCFA's effort in transforming from a payer to a value-based purchaser. HCFA's performance measurement system can support policy development, monitor and enforce contract standards, inform beneficiaries about their choices, and guide targeted quality improvement efforts. Still, there remains substantial work to be done in making performance data optimally useful to both purchasers and consumers.

Comparing organizations in terms of performance or quality is a daunting task, and we have substantial limitations in our ability to do so at present. It is more difficult to compare the quality of health plans than it is to measure the quality of one plan (Schneider et al., 1999). Davies (1998) points out the ambiguity of most health outcomes and their often misleading nature “…there are real concerns that the act of measurement itself has taken on a symbolic significance over and above the power of such information to promote beneficial and worthwhile change. We do not yet know how to make such systems deliver on the promises made for them”. Jencks (2000) indicates that performance data do not automatically improve clinical performance suggesting that health plans, with their substantial purchasing power, can create a market force for better performance in health plans. The literature has generally done a better job defining quality than it has performance, although, at times, these terms appear to be used interchangeably. The Institute of Medicine defines quality as “…the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current knowledge” (Lohr, 1990). The lack of definition of performance prevents the development of models that have a solid mathematical basis and can be operationalized.

Lied and Kazandjian (1999) contend that the scientific study of performance requires a model or paradigm. They propose a performance model consisting of a combination of four primary elements: (1)quality of care, (2)cost of care, (3)access to care, and (4)satisfaction. Satisfaction is defined as a function of perceived and expected outcomes of care and perceived an expected input. The higher the ratio of perceived to expected outcomes and the lower the ratio of perceived to expected input, the greater the satisfaction with the health care provider or plan. Performance is viewed as a multiplicative function of value (quality/cost), access, and satisfaction. This model is synergistic in the sense that increases or decreases in two or more of these elements lead to exponential changes in performance. The authors view their model as a starting point for developing rigorous approaches to performance measurement that take into account the patient's expectations as well as more typical components of performance such as quality, access, and cost of care. This model is well defined, has explicit assumptions, and has an underlying mathematical basis; however, it has not yet been empirically tested.

Although HCFA has not formally used a comprehensive, mathematically-based performance model that systematically combines components of performance, it has been active in the development and testing of individual performance measurement sets over the past few years. For example, HEDIS^®, CAHPS^®, and HOS have been developed or adapted for Medicare managed care plans. HCFA has begun to analyze the initial years of these data in order to determine how useful they will be to the program's objectives. An essential part of this systematic approach to performance measurement involves an indepth examination of the relationship among existing performance indicators.

Such an approach is especially important to HCFA in determining how to use its extensive performance measurement database for formulating and implementing purchasing policies. Many individual measures will be useful for monitoring activities and pursuing specific quality improvement efforts. For other purposes, such as developing proposals for performance-based pricing and for developing beneficiary information frameworks, combining or grouping several measures into a single indicator may be necessary. A comprehensive understanding of the relationships among key performance measures is critical to being able to combine them into useful aggregate measures.

This article examines the correlation between key performance indicators. We first examine the relationships between measures of beneficiary satisfaction and measures of clinical effectiveness. We then examine relationships between health status and satisfaction, as well as whether health status moderates the relationships between satisfaction and clinical effectiveness measures. Finally, we examine whether there is a relationship between average health status of a plan and clinical effectiveness measures.

Methods

In this study, we used the four measures from HEDIS^® that were audited for both the 1997 and 1998 measurement years. We also used a number of transformed measures from CAHPS^® that were believed to reflect beneficiary satisfaction.¹ Only 1998 measurement year data were analyzed here.

The four HEDIS^® measures were as follows:

• Adults' access to preventive/ambulatory health services aged 65 or over—measures the percentage of enrollees age 65 or over who were continuously enrolled during the measurement year and who had an ambulatory or preventive care visit during the measurement year.

• Beta blocker treatment after a heart attack—measures the percentage of enrollees age 35 or over during the measurement year who were hospitalized and discharged alive between January 1 and December 24 of the measurement year with a diagnosis of acute myocardial infarction (AMI) and received an ambulatory prescription for beta blockers upon discharge.

• Breast cancer screening—measures the percentage of women age 52-69 who were continuously enrolled during the measurement year and the preceding year and who had a mammogram during the measurement year or the preceding year. Enrollees may have no more than one gap in enrollment of up to 45 days during each year of continuous enrollment.

• Eye exams for people with diabetes—measures the percentage of Medicare beneficiaries with Type 1 or Type 2 diabetes age 18-75 who were continuously enrolled during the measurement year who had an eye screening for diabetic retinal disease.

HEDIS^® measures were calculated using aggregate data for each health plan. We use the term, plan, in this study in referring to the Medicare reporting entity, this study's unit of analysis. Medicare reporting entities, i.e., plans, prepare a separate Medicare HEDIS^® report for each Managed Care contract. Separate reporting is done within Medicare managed care contracts for market areas that are not geographically contiguous if the contract covered more than one major community in which there were at least 5,000 Medicare enrollees for that organization. As an additional requirement for inclusion, the denominator value of the at-risk population had to be at least 30 for each plan during 1998.² HEDIS^® rates for the four measures were equal to the percentage of plan enrollees in the at-risk population who received the care specified by the measure. For example, a rate of 75 percent for a given plan for the breast cancer screening measure should be interpreted as indicating that 75 percent of the at-risk population of the denominator received a breast cancer screen according to the criteria of the measure. The mean rates (across all plans) were unweighted; that is, they were the sum of all plan rates divided by the number of plans reporting with denominator values of at least 30.

From the CAHPS^® survey, we chose 27 items we believed to be conceptually related to beneficiary satisfaction. Rates were developed for each response to a particular item. The rate was the percentage of the total plan participants in the sample of plan respondents that endorsed a particular response for a given item. This can be mathematically represented as follows:

$$r_i=\mathit{(}{Q}_i/\mathit{\sum }{Q}_i\mathit{)}x\mathit{100}$$

Where: n = number of possible item responses excluding the non-applicable response and i = 1, n. The CAHPS^® rates were then transformed as follows: The rates for the responses for any item, e.g., item a, can be combined in what we term a transformed rate (R_a) according to the mathematical expression:

$$R_a=\mathit{(}+\mathit{0}\mathit{(}{r}_{\mathit{1}})+\mathit{1}(r_{\mathit{2}})+\mathit{2}(r_{\mathit{3}})+\dots +(n-\mathit{1})\mathit{(}{r}_n\mathit{)}\mathit{)}/\mathit{(}\mathit{(}n-\mathit{1}\mathit{)}(r_{\mathit{1}}+r_{\mathit{2}}+r_{\mathit{3}}+\dots +r_n\mathit{)}\mathit{)}.$$

R_a is the transformed rate for item a that has n possible responses on the questionnaire, excluding the “non-applicable” response. Under this transformation, the value of R_a can vary from 0.00 to 1.00. The value of this rate for any given plan is assumed to reflect the plan's position on an underlying continuum of consumer satisfaction for a particular item on CAHPS^®.

Two measures were used to reflect overall satisfaction with the health plan. The first of these was the sum of the 27 individual transformed item rates used in this study. These 27 rates were viewed as reflecting beneficiary satisfaction directly related to the particular experiences represented by the 27 individual questions. We felt that this rate was worth investigating, because it might capture a more global or overall measurement of beneficiary satisfaction than any of the individual items. The measure was computed as follows:

$$R_{\mathit{\text{Total}}}=\mathit{\sum }{R}_a$$

(where a varies from 1 to 27). The second measure was developed directly from Item 56 on the survey which asks: “How would you rate all your experience with your Medicare health plan?” This measure is likely to reflect closely the R_Total as well as experiences and subjective feelings patients have about their plan that are not captured in the individual questions.

For each plan, the merged file contained aggregate data at the contract service area level on the four HEDIS^® measures, the 27 individual items from CAHPS^® and the two global satisfaction measures. Spearman rank-order correlations were computed between each of the HEDIS^® measures and the transformed CAHPS^® 2.0 item rates to determine the relationship between effectiveness measures and consumer satisfaction measures of plan performance.

We attempted to determine if there were significant differences in the overall computed CAHPS^® satisfaction rates (R_Total) when the bottom and top 10 percent of the health plans were compared with the four HEDIS^® measures. T-tests for independent samples were used to test the significance of the difference in means for the two groups, i.e., the bottom and top 10 percent. All four HEDIS^® measures were used for the significance tests, and the results were also graphically depicted.

We also looked at whether health plan mean risk scores played a role in moderating (altering) the relationship between plan performance and overall beneficiary satisfaction. Overall beneficiary satisfaction was measured by the sum of the 27 CAHPS^® items that we believed best reflect beneficiary satisfaction (R_Total). Mean plan risk scores were based on the Principal Inpatient Diagnostic Cost Group (PIP-DCG) risk adjustment model in which the plan's mean risk score equals predicted Medicare cost divided by average Medicare cost. It is derived from fee-for-service data with an average risk score of 1.00 for fee-for-service beneficiaries. Mean beneficiary risk scores by plan were used in this analysis to determine if the relative risk of health services resource utilization (a proxy for the relative health of a plan's enrollees) played a role in the relationship between CAHPS^® scores and HEDIS^® measures. The plans were divided between those in the upper and lower halves (median-split) according to mean risk score of the beneficiaries enrolled in the plan.

Finally, we investigated the relationship between the first two items on the Medicare HOS, mean plan rates on the four HEDIS^® measures, and mean plan risk scores. We also examined the relationship between the two health status items and our overall beneficiary satisfaction measure, R_Total of CAHPS^®. The two HOS items ask respondents to rate their current health status (item 1) and their health status as compared with 1 year ago (item 2). The scores on these two items were transformed in a manner that was similar to the approach used to transform the CAHPS^® items.

Results

There were 306 health plans for which data were available for measurement year 1998, although not all of these plans reported on each measure used in this study. A total of 278 of these plans (90.8 percent) were enrolled in the Medicare managed care program since 1996. We reported the descriptive statistics (means, standard deviations) for the CAHPS^® transformed rates and the rank-order correlation coefficients (Spearman Rho) among the HEDIS^® rates and transformed CAHPS^® rates for measurement year 1998.³ The majority of the values of the transformed CAHPS^® rates were 0.80 or above suggesting that beneficiaries were generally satisfied with their health plans. The standard deviations tended to be small, indicating rather low variability in rates. Table 1 displays the descriptive statistics for the four HEDIS^® measures for measurement year 1998. The HEDIS^® rates varied from a low mean rate of 52.39 percent for eye exams for people with diabetes to a high of 88.95 percent for adult access to preventive ambulatory care, age 65 or over. The mean breast cancer screening rate was 73.24 percent, while the mean rate for beta blocker administration following a heart attack was 80.45 percent.

Table 1. Descriptive Statistics for Audited HEDIS^® Rates in Measures: 1998.

Measure	Number of Plans	Rate
		Mean	Standard Deviation	Minimum	Maximum
		Percent
Adults' Access to Preventive/Ambulatory Care Age 65 or Over	300	88.95	10.28	8.78	100.00
Beta Blocker Administration Following a Heart Attack	169	80.45	17.95	0	100.00
Breast Cancer Screening	264	73.24	9.44	26.79	89.66
Eye Exams for People with Diabetes	306	52.39	15.13	1.28	86.11

NOTES: Rates are unweighted, reflecting the sum of individual plan rates divided by the number of plans. HEDIS^® is Health Employer Data Information Set.

SOURCE: Author's tabulations from the Health Care Financing Administration's HEDIS^® files, 1998.

The two measures of overall satisfaction with the health plan were highly correlated with each other (rho = 0.81) as expected. R_Total, was significantly correlated with all four HEDIS^® measures: R_Total correlated 0.48 with the access measure, 0.29 with beta blocker administration, 0.36 with breast cancer screening, and 0.21 with eye exams. The CAHPS^® transformed score for item 56: “How would you rate all your experience with your health care plan?” was significantly correlated with the access and the breast cancer screening measures only.

An indepth examination of the relationship between individual HEDIS^® measures and individual CAHPS^® transformed rates is revealing. The vast majority of CAHPS^® measures demonstrate statistically significant correlations with the adult access and breast cancer screening measures. Thus, plans that exhibit better performance in terms of providing these prevention services also tend to perform better in a wide range of areas important to patients' satisfaction. A smaller set of CAHPS^® measures correlates significantly with the beta blocker and eye exams for people with diabetes measures—measures reflecting plans' effectiveness at providing needed care to patients whose conditions require such services. It is interesting to note that those CAHPS^® measures that do exhibit statistically significant correlations with these HEDIS^® measures generally reflect patients' perceptions of getting care when needed for illness or injury, or getting access to special providers or services.

Relationships with CAHPS^®

Adults' Access to Preventive/Ambulatory Care

A total of 25 of the 27 correlation coefficients involving the adults' access to preventive/ambulatory care for the age group 65 or over were statistically significant. A number of the correlations, while statistically significant, were low—in the 0.20s and low 0.30s range. However, a handful of the relationships were impressive, given the low variability in the CAHPS^® transformed rates. For example, the transformed rate for the question, “In the last 6 months, how much of a problem, if any, was it to get a referral to a specialist that you needed to see?” correlated 0.44 with adults' access to preventive/ambulatory care for the age group 65 or over. Apparently, health plans that had high rates of access for preventive/ambulatory services also tended to provide good access to specialty care. From the consumer's point of view, plans providing above average adults' access to preventive/ambulatory care also tended to be above average in providing the following:

• The help or advise needed over the telephone during regular office hours (rho=0.40).

• The care believed necessary by doctor or patient (rho=0.39).

• Courteous and respectful treatment (rho=0.38).

• Needed special medical equipment (rho=0.46).

Beta Blocker Administration After a Heart Attack

Of the 27 transformed CAHPS^® rates, 10 were significantly correlated with the HEDIS^® measure, beta blocker administration following a heart attack. The relationships between this measure and the CAHPS^® items were generally lower than with the previous HEDIS^® access measure. One CAHPS^® item that added particular credibility to the HEDIS^® beta blocker measure was the following: “In the last 6 months, how much of a problem, if any, was it to get your prescription medicine from your Medicare health plan?” The correlation (rho) between this item and beta blocker administration rates was 0.40.

Breast Cancer Screening Rates

Of the 27 transformed CAHPS^® rates, 23 were significantly correlated with breast cancer screening rates. Breast cancer screening rates displayed the highest relationships with the following aspects of health care as measured by CAHPS^® items:

• Not having a problem getting a referral to a specialist (rho=034).

• Not having a problem with delays while waiting for an approval (rho=0.39).

• Not having a problem getting special medical equipment (rho=0.41).

• Not having a problem getting special therapy (rho=0.38).

• Not having a problem understanding information on written materials (rho=0.41).

Rates of Eye Exams for People with Diabetes

A total of 10 of the 27 CAHPS^® transformed rates were significantly related to this HEDIS^® rate. Most of the significant relationships were low (less than rho= 0.30). The highest relationships were with the following aspects of health care delivery as rated by CAHPS^®:

• Not having much problem with delays while waiting for an approval (rho=0.29).

• Not having a problem getting special medical equipment (rho=0.29).

• Not having much problem getting prescription medication from health plan (rho=0.35).

• Not having a problem understanding information on written materials (rho=0.38).

There were statistically significant and substantial differences between the bottom 10 percent and top 10 percent of the plans with regard to R_Total and all four HEDIS^® measures (Table 2), supporting our hypothesis. In a subsidiary analysis, we compared mean HEDIS^® scores of the lower and upper 10 percentiles of plans with R_Total and item 56 of CAHPS^®, a global rating of experience with the health plan. It is noteworthy that R_Total was more effective in discriminating between high and low plan performers on the four HEDIS^® measures than item 56. Comparing the use of R_Total with item 56 to define the lower 10 percentile, we found lower mean HEDIS^® scores for all measures for the R_Total group. Conversely, in comparing R_Total with item 56 to define the upper 10 percentile, we found higher mean HEDIS^® scores for all measures for the R_Total group.

Table 2. HEDIS Performance: Bottom and Top 10 Percent of Plans Based on CAHPS^®: 1998.

Measure	Percent Group	Mean	Standard Deviation	t	df
Adults' Access to Preventive/Ambulatory Care Age 65 or Over	Lower 10	76.22	23.5	—	—
	Upper 10	94.08	3.65	*-3.65	39
Beta Blocker Administration Following a Heart Attack	Lower 10	62.42	28.85	—	—
	Upper 10	88.32	10.29	*-3.13	23
Breast Cancer Screening	Lower 10	65.19	8.54	—	—
	Upper 10	79.66	6.62	*-5.31	39
Eye Exams for People with Diabetes	Lower 10	45.88	18.21	—	—
	Upper 10	60.51	14.51	*-2.70	41

^*p < 0.05

NOTES: HEDIS^® is Health Employer Data Information Set. CAHPS^® is Consumer Assessment of Health Plans Study. df is degrees of freedom.

SOURCE: Author's tabulations from the Health Care Financing Administration's HEDIS^® files, 1998.

We investigated whether plans with risk scores exceeding the median risk score (implying their enrollees are of poorer health status on average) display a stronger relationship between clinical effectiveness measures of plan performance and beneficiary satisfaction measures than plans with risk scores below the median. Our hypothesis was that the perceptions of less healthy beneficiaries about their health plan are more closely related to clinical effectiveness measures, because these beneficiaries presumably had more need for and exposure to treatments. The mean plan risk score for 186 plans that had risk scores was 0.849, and the median was 0.848. Mean plan risk scores were found to moderate the relationship between the total CAHPS^® score as measured by R_Total, the sum of the 27 CAHPS^® items used in this study, and the 4 HEDIS^® measures. There was a considerably stronger relationship between the CAHPS^® measure and the HEDIS^® measures for the plans in the upper half of mean plan risk scores (Table 3). In a subsequent linear regression analysis, we found that we were able to predict R_Total by using the four HEDIS^® measures as independent variables with greater precision for plans in the upper half in terms of mean plan risk scores (R² =0.43) than for plans in the lower half (R² =0.32).

Table 3. Risk Scores (PIP-DCG) as a Moderator of the Relationship Between CAHPS^® and HEDIS^®: 1998.

Measure	Rank Order Correlation with RTotal
	Plan Risk Score < Median	Plan Risk Score ≥ Median
Adults' Access to Preventive/Ambulatory Care Age 65 or Over	*0.43	*0.54
Beta Blocker Administration Following a Heart Attack	0.02	*0.43
Breast Cancer Screening	*0.27	*0.43
Eye Exams for People with Diabetes	0.17	0.24

^*p < 0.05.

NOTES: PIP-DCG is Principal Inpatient Diagnostic Group. CAHPS^® is Consumer Assessment of Health Plans Survey. HEDIS^® is Health Employer Data Information Set.

SOURCE: Author's tabulations from the Health Care Financing Administration's HEDIS^® files, 1998.

Table 4 presents the relationship between scores on the four HEDIS^® measures in this study and items 1 and 2 of the Medicare HOS. Items 1 and 2 of HOS asked respondents to rate their general health (item 1) and to compare their general health status now versus 1 year ago (item 2). Beneficiaries responded on a five-point Likert-type scale to each item. High scores were consistent with good health (item 1) and favorable current health status versus the previous year (item 2). We found that mean plan HEDIS^® scores on beta blocker administration following heart attacks, breast cancer screening, and eye exams for people with diabetes were significantly positively related to mean plan scores on item 1 of HOS (rho=0.35, 0.36, and 0.36, respectively). Plans with enrollees who rate their current health as better than average have higher utilization on these preventive measures. The relationship of the adults' access to preventive/ambulatory care for the age group 65 or over with item 1 of HOS, was not statistically significant (rho=0.12). This measure was significantly negatively related (rho=-0.14) to item 2 scores, although this relationship was low. None of the other HEDIS^® measures was significantly related to item 2 scores. The low, but significantly negative relationship between the adults' access to care measure and decline in health status suggests that this HEDIS^® rate could actually be lower for people with declining health. The relationships between health status and overall satisfaction with the health plan (R_Total) were not significant (rho=0.11 and -0.05 for items 1 and 2, respectively).

Table 4. Relationship (Rho) Between Health Outcomes and Risk and HEDIS^® Scores.

Measure	HOS Scores
	Item 1	Item 2
Health Outcomes Survey
Overall Health Rating (Item 1)	1.00	*-0.54
Change in Health Rating (Item 2)	*-0.54	1.00
Risk Score
Principal Inpatient Diagnostic Cost Group	*-0.38	*-0.26
HEDIS® Measures
Adults' Access to Preventive/Ambulatory Care	'0.12	*-0.14
Beta Blocker Administration Following a Heart Attack	*0.35	0.11
Breast Cancer Screening	*0.36	0.02
Eye Exams for People with Diabetes	*0.36	0.09

^*p < 0.05

SOURCES: Author's tabulation from the Health Employer Data Information Set (HEDIS^®) files, Health Outcomes Survey, and the Principal Inpatient Diagnostic Cost Group, 1998.

Current health status and recent change in health status of plan beneficiaries were negatively related to plan risk scores. Items 1 and 2 of HOS were significantly negatively related to mean plan risk scores (-0.38 and -0.26, respectively). Since both lower values of risk scores and higher values of item 1 from HOS imply better health status, the negative correlation suggests that plan health status as measured by the PIP-DCG risk adjustor tracks well with beneficiaries' self-rating of health status.

Summary and Discussion

This article investigated various relationships involving performance, beneficiaries' experience with their health plans, health status, and risk in Medicare managed care plans for measurement year 1998. The variables included 4 audited measures on HEDIS^®, 27 items and an overall satisfaction score that we computed on CAHPS^®, the first 2 items on HOS involving health status, and plan risk scores based on the PIP-DCG risk adjustment model. Three of the HEDIS^® measures involved effectiveness of care: beta blocker administration following a heart attack, breast cancer screening, and eye exams for people with diabetes. The fourth measure involved access to care: adults' access to preventive/ambulatory care for the age group 65 or over.

We found that clinical effectiveness measures of plan performance were positively related to beneficiary experience. There were many significant positive relationships between the HEDIS^® measures and the CAHPS^® items. The HEDIS^® measures with the highest relationship with the CAHPS^® items were adults' access to preventive/ambulatory care for the age group 65 or over and breast cancer screening. Out of 27 CAHPS^® items, 25 were significantly related to adults' access to preventive/ambulatory care for the age group 65 or over and 23 were significantly related to breast cancer screening. Even for the two HEDIS^® measures with less impressive relationships with the CAHPS^® items, beta blocker administration following a heart attack and eye exams for people with diabetes, we found a number of significant relationships. For example, beta blocker rates correlated 0.40 with having little problem getting prescription medicines filled, and diabetes eye exam rates correlated 0.38 with having little problem finding or understanding information in the written materials. Moreover, all four HEDIS^® measures were significantly related to the CAHPS^® overall measure (R_Total). The highest statistical association was with adults' access to preventive/ambulatory care for the age group 65 or over (rho=0.48).

We found that plans with risk scores that are above the overall median display a higher relationship between plan performance and beneficiary satisfaction than plans with risk scores below the median. We also found that HEDIS^® utilization rates are positively related to current beneficiary health status. In general, plans with beneficiaries who rated their health status as above average (compared with other plans) had higher rates on three HEDIS^® measures of effectiveness of care which emphasize prevention: beta blocker administration following heart attacks, breast cancer screening, and eye exams for people with diabetes. Health status ratings, as predicted, were found to be related to plan risk scores—the higher the health status, the lower the risk scores. However, health status measured at the plan level was not significantly related to overall beneficiary satisfaction.

We found that current health status and recent change in health status (in the last year) were negatively related to plan risk scores, consistent with our hypothesis. That is, items 1 and 2 of HOS were significantly negatively related to mean plan risk scores (-0.38 and -0.26, respectively). Plans with enrollees of above average current health and above average health in comparison with the previous year tend to have lower mean plan risk scores. Finally, we found that there are significant differences in HEDIS^® rates between plans in the upper and lower decile in overall beneficiary satisfaction as measured by R_Total, the computed rate we developed specifically for this study. We were able to demonstrate statistically significant and substantial differences in mean HEDIS^® rates between these groups, consistent with our predictions.

These results supported the continued use of HEDIS^®, CAHPS^®, and HOS in assessing health plan performance, consumer experience, and health status. The essential convergence or agreement among these measures confirms their validity in tapping important components of plan performance, beneficiary experience, health status, and risk. Our findings also suggest that HEDIS^®, CAHPS^®, and HOS, when viewed in combination, provide more useful information about health plans, than when any of these measures is viewed separately. Approaches to performance assessment of health plans should emphasize the need for close integration of measures that tap various domains of performance—quality, beneficiary satisfaction, access to care, and health outcomes. It may be useful to transform and combine rates within some of the measures to improve their usefulness. This study found support for this approach when 27 CAHPS^® items were first transformed to an ordinal scale with a maximum range in values from 0 to 1, and then correlated with HEDIS^® measures, both singularly and in total combination (R_Total).

We believe that the next steps in the evolution toward integrated approaches to performance measurement should include more analyses of data from the Medicare HOS in conjunction with HEDIS^® and CAHPS^®. In this study we only looked at the first two items of HOS, admittedly, an incomplete assessment of the utility of this measure to add interpretative value to CAHPS^® and HEDIS^®. In addition, further analyses should include disenrollment data and additional HEDIS^® measures. Additional analysis of these data will help to paint a more complete picture of the relationship between plan performance, beneficiary experience, and health outcomes that could lead to the development of models of performance that systematically combine elements.