Performance Measures and Report Card Initiatives

The widening use of performance measures and the report cards that derive from them in rating institutional, practice, and individual physician performance is affecting every aspect of healthcare, from the way physicians are collecting data to the way we are paid and offered incentives and the way insurers, purchasers, hospitals, and even patients are defining quality of care. As the influence and authority of these measures and reports spreads, they will have a profound effect on our responsibilities and challenges in improving care, as well as the economic landscape of the overall healthcare system in the United States.

Current Use of Report Cards: Two Examples

In Minnesota, a report card format using composite measures has been in place for several years. Minnesota Community Measurement (MCM; www.mnhealthcare.org) is a nonprofit entity established in 2002 that more recently has become an AQA Alliance pilot organization. They report at a clinic level on over 20 measures, including diabetes management, asthma management, compliance with basic screening tests, and other common services performed by physicians. MCM utilizes a sampling method based on chart reviews that, although not comprehensive, has been well developed since its inception. They have signed up over 90% of the care base in Minnesota and ask for provider-specific levels with the same composite measures examined on the clinic level. Although the methodology of their analysis appears to be sound, the data sets produced thus far have not been validated. A truly comprehensive report card on overall performance or extremely specific report cards on less common procedures or treatment could be more controversial and require further validation.

In California, the Integrated Healthcare Association (IHA; www.iha.org) issues report cards that are medical group-specific with no information on individual physicians. These reports measure compliance with immunizations, certain cancer screenings including colorectal cancer screening and mammography, and other common measures like diabetes and asthma. The plus of the IHA approach, in terms of validity, is that all practices are being measured in a similar manner, which minimizes methodology errors. However, this method still samples a sample, which can lead to a distorted view of actual practice. Further, while some groups work electronically, others continue to work on paper, and there is a significant cost associated with pulling charts in order to gather the necessary data for reporting.

Utilizing Report Card Information

It is important to recognize that report cards, whether used by physicians, employers, or patients, represent only one part of a continuum of information. A number or grade is meaningless unless it is considered as part of all the other domains and environments of care that are associated with that individual provider. This has significant applicability in a consumer-directed environment, with a high deductible plan. For example, let us suppose that the patient has a $5,000 deductible. An “A” on a report card may be meaningless if Doctor X performs colonoscopy in a hospital outpatient department that charges a $2,000 facility fee (added onto the physician's professional fee), while Doctor Y charges a bundled fee of $1,000 including the professional and facility fee for the same procedure. Is cost the only consideration? Is it important to know how often each doctor reaches the cecum, the physician's colonoscopy withdrawal time, their adenoma detection rate, and/or when they bring patients back for surveillance?

Errors due to subgroup analysis also plague endoscopy “quality” report cards. For example, do we measure adenoma detection rates only on patients who are ageappropriate and present for their first screen? For each physician performing colonoscopy in our Minnesota practice, only approximately 100–200 men and women per year would fit these criteria. This leads to the concern that these numbers might be too small to make meaningful comparisons.

Another important aspect of report card practice that has been demonstrated repeatedly in primary care practice is that when broad goals are measured (eg, diabetes care, asthma care, vaccination rates), the entire structure of the clinic must be changed in order to improve scores. When the structure of the clinic changes to heighten quality in one area, it can spread and become generalized, which makes these measures useful. However, if these measures are used to compare individual providers and say, “This physician is a 98, whereas this one is only a 93,” they generally are not appropriate.

Developing Meaningful Measures

An important question to ask is whether a report card is measuring something arbitrary and trying to apply it from one physician to the next, or is truly a risk-adjusted measure of physician performance. Mark C. Rattray, MD, a gynecologist and one of the methodologist consultants to the AQA Alliance, has designed a method to measure episodes of care, not only in single patients, but as aggregate numbers across a physician's entire practice (www.carevariance.com). Although this is a very time- and labor-intensive method, this methodology does factor in a risk-adjustment variable. This is important, particularly in looking at gastroenterologists and other subspecialty providers who are not necessarily responsible for all domains of care and who work with patients with varying levels and severities of comorbidity.

If the report card uses standards that have been established by trained methodologists at the level of the National Committee for Quality Assurance (NCQA) or Institute for Clinical Systems Improvement (ICSI) and bias has been eliminated, it must still be established whether these standards provide a meaningful measure. Not until then do report cards become truly valid. A dilemma that has become apparent with specialty societies attempting to develop measures for the CMS Physician Quality Reporting Initiative (www.cms.hhs.gov/pqri/) is that some measures are being developed in the name of simply reporting on something. Whether the measures are meaningful and have any real effect on patient care and outcomes is debatable. As a result, employer groups and purchasing coalitions are beginning to question a number of these measures because they do not see them as relevant to improving patient health. If anything, some of these reporting measures have the potential for resulting in higher costs without improving care. The report card system in California assigns 20% of the weight in its scoring to the patient's “environment of care.” Does this mean that a practice can improve its overall score based on whether or not it has valet parking? Thankfully not. A number of health plans have included measures based solely on generic drug compliance. Although these types of measures may save money overall for primary care conditions where there are a plethora of pharmaceutical agents available, in the specialty of gastroenterology, where we are treating patients with inflammatory bowel disease or hepatitis and medications can cost tens of thousands of dollars, emphasizing generic drug compliance as the sole quality measure sends the wrong message.

Part of the problem in deciding which measures are meaningful comes from the presumption that evidence-based medicine is derived from randomized, controlled trials where the populations are well defined and pure. When these findings are applied in clinical practice where patients have comorbidities and complicating factors, they can be misleading. Yet the purchasing/payer community wants validated measures and to attach incentives to those measures. In order to make evidence-based measures work, they need to be applied in the community setting for several years to get a baseline measurement. Then, they need to be fine-tuned, based on the vagaries of regular practice. Hospitals have had over a decade to refine a very limited set of measures and, as has recently been shown, such measures may only be valid in low-performing hospitals. In medium- or high-performing hospitals, following those measures in an internalized system does not improve care.

Inequality in Report Card Incentives

Report card-based incentives tend to disenfranchise small practices. Medical practices have never invested money in what we think of as “doing the right thing,” in terms of infrastructure and electronic systems. In a tax environment that encourages physicians' practices to close their books after Christmas and distribute anything left in the practice before the end of the year so that the practice is not taxed twice, money is not being retained for infrastructure, for quality, for electronic records, or for process improvement. Whereas larger practices may have more than enough resources to invest in infrastructure, solo physicians and smaller groups tend to fall behind. Without investing in infrastructural improvements, data collection and reporting become more difficult, thereby making the achievement of meaningful measures and the acquisition of related financial incentives more difficult as well for the practice.

Inner-city and rural practices often serve patients who face greater challenges in terms of compliance, due to lack of transportation or less work flexibility. Without an equation to factor in these variables, they can have an effect on the ratings of a physician practice. A recent statewide program in Tennessee attempted to improve compliance by putting the onus on patients to follow prescribed treatment regimens or lose their state-subsidized individual benefits. However, this system proved truly unfair to those people who could not take time off from work, did not have access to day care, and/or lacked adequate transportation. As a result, patients were perversely incentivized to avoid care. Further, there is the risk that practices that serve these communities, regardless of their patient demographics, could be at a disadvantage if they do not have access to an electronic medical record system and/or lack the ability to report data in a way that is compliant and efficient.

Further, although the Inspector General has developed safe harbors for hospitals to provide electronic medical records and prescribing modules to physicians, the Internal Revenue Service (as of March 1, 2007) still professes that if a hospital provides an electronic health record to a physician practice, this represents a taxable benefit to the physician. What could be the implications of a nonprofit entity such as a hospital providing goods and services to a physician practice? It essentially constitutes an antikickback violation. Branches of the federal government need to communicate and make rules that allow for the logical sharing of patient information.

The Need for a Common Language

Typical gastroenterologists perform procedures in an ambulatory surgical center (ASC) and in a hospital and also see patients in their own offices. Do any of these records-keeping systems interact? Most practices have systems that are totally walled off from connectivity and communication. However, gastroenterologists need to think about whether their ASC report writer, capturing report card elements, is communicating with their office-based electronic medical record system, which contains data such as pathology results, and therefore incorporates a reminder for future exams. If a practitioner has identified a patient who had a colonoscopy and had a polyp removed, the system needs to identify what the pathology was and send an appropriate reminder to that patient for follow-up at a predetermined time. If the procedure was done in a hospital setting, there are numerous additional issues related to access to and transfer of information.

Breaking down these barriers requires outside-the-box thinking. For example, patients could have their e-Health database on a website similar to a MySpace or Friendster page, where information could be added that is patient-specific and password protected, but easily accessible. Although physician practices might have an electronic health record, it needs to be ensured that the information can be accessed when the patient arrives in the emergency room. Ultimately, there is no language that speaks to all of us. As a result, any accumulation of data or health records breaks down and becomes useless because there is no standard method of reporting across the profession. If there is no way to access information from one institution to another, this makes it nearly impossible to produce any sort of valid judgments or incentive programs. No entity lesser than the federal government can mandate a common system; although the federal government has certainly called for information technology connectivity, it has not yet put enough money into supporting it, especially with the threat of declining reimbursement hanging over our heads.

First Steps

The Health Plan Employer Data and Information Set (HEDIS) is a standardized data set developed by well-trained methodologists and supported by the NCQA. HEDIS is a work in progress that continually moves to raise the bar. For example, one of the HEDIS measures for diabetes has evolved from seeing whether a patient with diabetes had a hemoglobin A1C level to monitoring whether the value is within the acceptable range for patients with diabetes. HEDIS might be seen as one of the first steps in developing a common set of measures that can be used across specialties and venues of care to develop the common language that we need.

The ICSI is a not-for-profit organization in Minnesota, funded by major health plans along with 57 physician member groups, that has developed 94 practice guidelines. As a part of membership, physician groups must share in the burden of developing guidelines. Currently, these guidelines are mostly primary care-based, but as the program grows, that will change. Hospital-based order sets are also in development but present a challenge as each hospital must conform to its own formulary and specific drug usage. What the ICSI guideline measures can do is provide guidelines that allow groups like MCM to utilize something that has been debated, validated, and agreed on, with the end result of valid measure sets and report cards.

It is essential that the specialty societies that address digestive and hepatic disorders, including medical, surgical, radiologic, and pathologic societies that speak for pediatric and adult patients, commit to working together to develop valid and meaningful measures. There is little to be gained by creating report card measures that have not been endorsed by all stakeholders, particularly when well-intentioned measures can be turned by payers and purchasers into programs designed to punish physicians. Physicians must be at the table, working with the various agencies that are developing performance measures, and present at the outset of the discussions, as this is critical to creating guidelines and measures that are rational, meaningful, and can be implemented by practitioners who treat the condition, regardless of their specialty.

Whatever report card measures are ultimately chosen, they should be agreed upon by all gastroenterologists as practical and meaningful. Reporting on something simply because the data are accessible does not make it meaningful. To paraphrase former Speaker of the US House of Representatives Tip O'Neill, given that all healthcare is local, if one method is used in one area and a different one in another, it does not necessarily mean that a given practitioner or group is right or wrong. Thus, the measures that we ultimately use must be flexible as well as meaningful in helping us improve the care and outcomes for the patients we serve.