Quality measures developed to quantify health care provider-specific performance can be useful tools for informed decision-making by consumers, purchasers, and regulators. Additionally they can stimulate improvements in the quality of care and provide insight in targeting of quality improvement activities. Following up on the spring 2001 issue of the Review on performance measures for health plans, this issue addresses provider-specific measures of quality and refinements to existing plan-specific measures.
The first five articles in this issue of the Health Care Financing Review report on recent initiatives to develop and implement new provider-specific measures of quality in health care delivery, focusing on settings other than health plans. This set of articles provides insights into the processes used in the design, development, and implementation of these initiatives and each offers lessons learned for future work. Aspects addressed in these articles include the crafting of the measures as well as the challenges of presenting, explaining, and disseminating the information to the public. The spectrum of providers covered includes nursing homes, dialysis facilities, hospitals, and primary care case management programs.
These new initiatives have balanced concerns of validity, reliability, timeliness, and value to the user. Despite acknowledged shortcomings, these projects have expanded the information available to the public and to health care providers, often under constraints of aggressive timelines to meet legislative mandates. The authors acknowledge that each of these projects is only a first step, and relay that the importance of advancing the quality process outweighs limitations in the indicators. Use of these new measures affords opportunities for continued improvement and evolution of the measures, their implementation, and the supporting communication strategies.
The next four articles in this theme focus on refinements and new applications of existing quality measurement tools used in health plans. These studies illustrate the further evolution and application of established quality measures. The contrast between the levels of refinement in these two sections of this issue highlights the advancements that have been made in quality measurement for health plans, a health care delivery setting that has a history of systematically using established quality measures.
New Initiatives Mesuring Provider Quality
In the first article, Harris and Clauser report on CMS's Nursing Home Quality Initiative. The initiative uses nursing home performance measurement as a tool to disseminate information on quality of care provided in nursing homes. The article provides background on the evolution of performance measurement by nursing homes and limitations with existing measures. The authors highlight recent efforts by CMS to develop new measures of nursing home quality and the current pilot project to test the feasibility and effectiveness of the new communications program. With an emphasis on beneficiary choice and quality improvement, this initiative has provided more timely, useful, and understandable information to consumers and their families to help them make more informed decisions in selecting a nursing home and to motivate providers to proactively address and improve quality of care.
Complementing the first article, Berg, Mor, Morris, Murphy, Moore, and Harris provide additional detail on the background work for the CMS initiative, summarize their work in identifying and evaluating existing quality indicators for long-term care settings. Despite the content validity and perceived clinical utility of existing measures, they found there was little or no documentation for many of the quality indicators currently in use.
The authors contrast the needs and perspectives of multiple audiences: facilities themselves for internal quality monitoring, facility surveyors or external regulators, and consumers. Measures that meaningfully reflect both the post-acute and long-term resident populations served are proposed. The authors point out that unlike most other health care provider types, the nursing home sector has the advantage of existing standardized assessment data with clinically meaningful resident-level information collected on an ongoing basis for all residents. They note that this Minimum Data Set (MDS) data has greatly facilitated the development of quality measurement for nursing homes.
In developing new measures, Berg et al. propose desired measurement properties of quality indicators for making external comparisons. These features include minimizing assessment measurement errors (detection bias) and using case-mix adjustment to address the selection bias resulting from differential admission practices and differential rates of discharge to assure the validity of interfacility comparisons. The researchers provide preliminary analysis of metrics within four domains of functional status, clinical, complexity, psychosocial, and pharmacotherapy using the national MDS data.
The Balanced Budget Act of 1997 required CMS to develop a system to measure and report on the quality of dialysis services under the Medicare Program. Frederick, Maxey, Clauser, and Sugarman report on the process used to develop and report these performance measures. The response to the legislation included enhancements to the existing End Stage Renal Disease Core Indicators Project and new efforts to provide dialysis facility-specific information to the public that culminated in the Internet-based Dialysis Facility Compare tool.
A time-limited legislative mandate forced the reliance on existing data sources. CMS, as the dominant payer for dialysis care, was able to convene key stakeholders and constituency groups to enhance the development process.
In the development of reports to the public comparing hospital patient satisfaction data, Barr, Boni, Kochurka, Nolan, Petrillo, Sofaer, and Waters describe a collaborative process used in Rhode Island. The first step in implementing the State's mandated public reporting of hospital performance addresses patient satisfaction. The project focused on the technical issues of developing and determining the most appropriate measurements as well as the reporting issues related to determining the most appropriate reporting format.
A pilot survey, conducted prior to the first round for public reporting, was used to test the sample and data collection methodology and provide baseline information to assist in the format of the public report. Decisions regarding the format of the report could be made independent of actual data; this prevented the final report from being influenced by any particular hospital's standing in the data. The pilot survey data were also used by the hospitals to carry out quality improvement efforts between the time of the pilot and the administration of the reportable survey. The authors conclude that hospitals were making improvements all along, but were able to target specific areas with the help of the pilot survey. They highlight the need for extensive efforts at dissemination and assessment of public response to the health care quality performance reports.
The Massachusetts Medicaid Program, MassHealth, has implemented provider profile reports for quality improvement in their Primary Care Case Management Program. Walsh, Osher, Nason, Porell, and Asciutto describe the program, report on provider perspectives, and identify challenges in evaluating the impact. To complement the reporting process, the plan conducts in-person meetings of a regional network manager with the practice manager, medical director, or other representative of each primary care practice in the program following the dissemination of profile reports. Walsh and colleagues highlight the strengths of the program, revisions already made, and suggest factors for others to consider in implementing similar strategies in the primary care case management framework.
Refining Health Plan Performance Measures
The next four articles in this issue focus on refinements and new applications of existing quality measurement tools used in health plans. These articles address novel strategies in the use of two well established health care quality measurement tools: the Consumer Assessment of Health Plan Study (CAHPS®) survey and the Health Plan Employer Data Information System (HEDIS®). Both tools have had extensive use in measuring and reporting on the quality of commercial and Medicare managed care (MMC) plans.
Although the health plan is the typical unit for reporting health care quality measurement in managed care, Solomon, Zaslavsky, Landon, and Cleary explore the variation in quality measures at organizational levels of health care delivery within a plan. They looked at variation in CAHPS® survey scores across health plans and across the service delivery structures within a plan. These sub-units include groups of affiliated medical groups and hospitals (known as regional service organizations), medical groups, and individual practice sites by looking at 30 medical groups providing care in 49 sites in eastern Massachusetts.
Their findings suggest that plan- and group-level performance measures currently being reported may mask substantial sub-unit variation. Health plans account for much less of the variation in measures of patient-reported quality compared with other units of analysis. The authors conclude that medical groups, rather than plans, may be the more appropriate focus of intervention designed to improve care. Not only is this the organizational level found to display variations in quality, but the closer association with individual practitioners may also make medical groups the more salient quality reporting unit for consumers.
The next two articles focus on combining measures in CAHPS® and HEDIS® into fewer summary performance measures. Zaslavsky, Shaul, Zaborski, Cioffi, and Cleary combined both CAHPS® and HEDIS® measures, condensing 20 measures to create 4 scores which summarize most of the variability in these measures. Lied, Malsbury, and Ranck combined 17 HEDIS® measures into 1 composite score that focuses on outcomes-based measurement. Although none of these streamlined metrics have been tested for usefulness or comprehensibility, the innovative analyses reported provide the initial steps in the development of more condensed measures to report plan performance.
Advantages posed by these new measures include ease of use due to their more limited number and the concise information. Potential disadvantages noted by Zaslavsky et al., are that composite indicators might be harder to interpret than a single measure and may be less likely to suggest direct links to quality improvement.
Each of these approaches provides additional unique features. Zaslavsky and colleagues imputed missing performance scores for units which did not report or were unable to generate HEDIS® measures due to small sample size and offers rationale for the importance of this imputation process in providing a complete picture of quality. Lied et al. propose computation of State, regional and national averages or “norms” to provide a scorecard to benchmark plans against their peers.
In the final article addressing the theme of this issue, Langwell and Moser explore strategies of measuring and improving the performance of Medicare+Choice plans in serving racial and ethnic minorities. The authors explore the differences in health plan experiences and patterns of service use by racial and ethnic minorities enrolled in MMC. They conclude that although health plans generally meet the needs of the majority of their members, enrollees who are members of certain racial/ethnic populations may encounter greater difficulties in obtaining and using services from their health plans.
Using MMC CAHPS® data, the authors identify potential areas of concern. Patterns of use suggest that most MMC plans may not be recognizing and responding to the unique needs of their enrollees who are members of minority groups. The authors offer several strategies to further understand and monitor these findings as well as specific actions that could be implemented to address these concerns.
Conclusion
The articles in this issue addressing the theme of provider-based performance measurement illustrate advancements that have been made in tackling the challenges of performance measurement in the health care system. Performance measurement and the public reporting of its results have expanded beyond the health plan setting into other health care settings. Moving the measures and reporting units closer to the point of service delivery enhances the utility of these initiatives for consumers and for providers. Broader availability of performance measures underscores the need to make these tools more user friendly and increasingly meaningful across many audiences. The projects reported in this issue demonstrate that the concept of continuous improvement applies not only to the quality of health care, but also to the metrics and reporting of performance measurement.
Footnotes
The author is with the Centers for Medicare & Medicaid Services (CMS). The views expressed in this article are those of the author and do not reflect the views of CMS.
Reprint Requests: Mary C. Kapp, Centers for Medicare & Medicaid Services, Office of Research, Development, and Information, 7500 Security Boulevard, C3-19-07, Baltimore, MD 21244-1850. E-mail: mkapp@cms.hhs.gov