Understanding Quality Measures in Otolaryngology–Head and Neck Surgery

Abstract

As health care reimbursements based on pay-for-performance models become more common, there is an unprecedented demand for ways to measure health care quality and demonstrate value. Performance measures, a type of quality measure, are unique tools in a health care delivery system that allow objective monitoring of adherence to specific goals and tracking of outcomes. We sought to provide information on the development of quality measures in otolaryngology–head and neck surgery, as well as the goals of performance measurement at a national level and for our specialty. The historical development, various types, and approach to creating effective performance measures are discussed. The primary methods of developing performance measures (using clinical practice guidelines, clinical registries, and alternative methods) are also discussed. Performance measures are an important tool that can aid otolaryngologists in achieving effective, efficient, equitable, timely, safe, and patient-centered care as outlined by the Institute of Medicine.

A significant component of health care reform in the United States has been the pursuit of high-quality and high-value health care. As health care reimbursements increasingly follow pay-for-performance models, there is an unprecedented demand for ways to measure health care quality and demonstrate value. As recently as January 2015, the Department of Health and Human Services mandated that, by 2018, up to 90% of Medicare payments be linked to a quality measure.¹ However, the discipline of otolaryngology–head and neck surgery is in the early stages of defining quality measures, and further work is necessary to perfect these measures.

The Institute of Medicine defines quality care as being effective, efficient, equitable, timely, safe, and patient centered.² Another earlier definition of quality from the Institute of Medicine is “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”^3(p21) It is important that physicians, policymakers, payers, and patients share a common definition of quality regarding the delivery of health care. To define robust quality measures and reduce variation, many agencies, including the Centers for Medicare & Medicaid Services and the Agency for Health Care Research and Quality, have turned to performance measures.

Performance measures are a unique tool to demonstrate value and quality in a health care delivery system by objectively monitoring adherence to specific goals and tracking outcomes. The Institute of Medicine defines performance measures as a “numeric quantification of healthcare quality.” Alternatively, the American College of Cardiology/American Heart Association Task Force on Performance Measures describes performance measures as a subset of quality metrics that are “specifically suitable for public reporting, external comparisons, and possibly pay-for-performance programs.”^4(p2113) The term performance measure is reserved for those quality metrics only with “…attributes rendering them suitable for public reporting and for explicit comparisons of care between institutions and/or healthcare providers.”^4(p2114)

Donabedian⁵ first described performance measurement as applied to health care as away to measure the various domains of care delivery and focused on structural, process, and outcome measures. Birkmeyer et al⁶ later discussed applying this paradigm specifically to surgical care. Performance measurement in surgery has continued to evolve (Figure 1).⁷ Currently, quality measures in use by the Department of Health and Human Services are available. For example, measure HMIS 000608, “timing of antibiotic prophylaxis (prophylactic antibiotic initiated within 1 hour prior to surgical incision) in surgery,”^8(p63) is a measure of the number of patients aged 18 years or older who undergo procedures with indications for prophylactic parenteral antibiotics and are given the antibiotic within an hour prior to incision. The objective of this review is to provide information on quality measures in otolaryngology–head and neck surgery, the goals of performance measurement at a national level and within our specialty, and how quality and performance measures are developed.

Figure 1. Types of Performance Measures.

Examples shown encompass the various domains used to measure the quality of health care delivery.

Goal of Performance Measurement

In general, the purposes of performance measurement are to (1) define the outcome of an intervention, (2) measure an improvement in outcomes caused by a modification of a treatment or care process, and (3) compare the quality of care delivered by various entities, including hospitals, medical groups, or physicians.⁹ However, it is important to consider the alternative side of performance measurement from the payer’s perspective.

In otolaryngology, patient safety and quality improvement are sometimes seen as interchangeable; however, the 2 factors are slightly different in an important way. The patient safety movement is primarily focused on identifying how adverse events occur and subsequently implementing changes to reduce their occurrence. To use the paradigm of the Oxford Center for Evidence-Based Medicine Levels of Evidence¹⁰ that span diagnosis, prognosis, screening, treatment benefits, and harms, only treatment harms and errors of diagnosis are usually addressed by patient safety initiatives. Although this method is fundamentally important for reducing adverse events and should be continued, performance measurement as a method of quality improvement, in contrast, is more broadly focused.

Performance measurement is a way to examine positive outcomes as well as adverse events, and thus incentivize best practices. Rather than focusing on the avoidance of practices associated with a higher risk of adverse events, performance measurement aims to take the best possible characteristics, processes, and outcomes within a discipline and translate them into actionable goals. The Table reports examples of current performance measures in use via the Physician Quality Reporting System in otolaryngology.¹¹

Table.

Existing Performance Measures in Otolaryngology–Head and Neck Surgery in Current Use by the Physician Quality Reporting System^a

Diagnosis	Type	Measure
AOE
Topical therapy	Process	Percentage of patients aged ≥2 y with AOE who received prescriptions for topical preparations
Systemic antimicrobial therapy (avoidance of inappropriate use)	Process	Percentage of patients aged ≥2 y with AOE who did not receive prescriptions for systemic antimicrobial therapy
Adult sinusitis
Antibiotic prescribed for acute sinusitis (appropriate use)	Process	Percentage of patients aged ≥18 y with acute sinusitis who received prescriptions for an antibiotic within 7 d of diagnosis or within 10 d after onset of symptoms
Appropriate choice of antibiotic: amoxicillin prescribed for patients with acute bacterial sinusitis (appropriate use)	Process	Percentage of patients aged ≥18 y with acute bacterial sinusitis who received prescriptions for amoxicillin, with or without clavulanate, as a first-line antibiotic at the time of diagnosis
CT scan for acute sinusitis (overuse)	Outcome	Percentage of patients aged ≥18 y with acute sinusitis who received a CT scan of the paranasal sinuses at the time of diagnosis or within 28 d after date of diagnosis
>1 CT scan within 90 d for chronic sinusitis (overuse)	Outcome	Percentage of patients aged ≥18 y with chronic sinusitis who received >1 CT scan of the paranasal sinuses at the time of diagnosis or within 90 d after the date of diagnosis

Abbreviations: AOE, acute otitis externa; CT, computed tomography.

^aInformation obtained from the Centers for Medicare & Medicaid Services.¹¹

Historical Background

The first national program devoted to the reporting of quality measures in medicine(ORYX Initiative) was launched in 1997 by The Joint Commission. This initiative was driven by “continuous and increasing pressure for cost containment and quality improvement.”^12(p63) For a hospital to be accredited, it was required to report data on 2 of 4 core performance measure sets, including acute myocardial infarction, heart failure, pneumonia, and pregnancy.¹³ Initially, there was no consensus on the kinds of performance measures for reporting, and none of the measures submitted to The Joint Commission were publicly available.

Numerous important changes occurred in 2004. First, The Joint Commission began making the reported data from previous years available to the public, which today can be found online.¹⁴ Second, the Centers for Medicare & Medicaid Services began reducing payments to hospitals that did not report the previously mentioned Joint Commission measures and instituted their own public reporting system the following year. At present, The Joint Commission requires health care facilities to report 6 sets of performance measures to maintain accreditation.¹⁵ The Centers for Medicare & Medicaid Services also requires reporting via the Physician Quality Reporting System to avoid a negative 2% payment adjustment in 2017.¹⁶

Components of a Good Performance Measure

It is important for physicians to not focus narrowly on maximizing scores on quality measures and forget the overall needs of the patient.¹⁷ The use of performance measures to improve quality of care should thus be held to rigorous criteria to avoid unintended adverse consequences. Chassin et al¹⁸ have proposed 4 accountability measures to which process measures should adhere: (1) there is a strong evidence base showing that the care process leads to improved outcomes, (2) the measure accurately captures whether the evidence-based care process has been provided, (3) the measure addresses a process that has few intervening care actions that must occur before the improved outcome is realized, and (4) implementation of the measure has little or no chance of inducing unintended adverse consequences.

Choosing a Topic for Performance Measure Development

The American Academy of Otolaryngology–Head and Neck Surgery¹⁹ has outlined a list of 28 individual Physician Quality Reporting System performance measures and 3 measure groups that may be applicable to an otolaryngology practice. However, if otolaryngologists are to use the full potential of performance measures to improve quality of care, we must continue to carefully develop quality measures. Areas of particular interest are procedures with high morbidity and mortality, such as laryngectomy²⁰; high resource utilization, such as cochlear implantation²¹; and high volume, such as tympanostomy tube insertion in children.²²

Performance Measure Development

Currently, performance measures are primarily developed by committees in subspecialty organizations working with national organizations, such as the Agency for Healthcare Research and Quality and the Physician Consortium for Performance Improvement of the American Medical Association (AMA-PCPI). These 2 organizations represent the first layer of rigorous testing and evaluation beyond the subspecialist expert committee. When a set of performance measures is finalized, the measures can be turned over to the National Quality Forum, which then subjects the measures to a rigorous testing phase and allows for open comments from all stakeholders, including patient advocates. National Quality Forum approval of a measure is generally considered the pinnacle of performance measure quality and validation. In the following sections, we discuss various methods of developing performance measures (Figure 2).

Figure 2. Potential Pathway of Quality and Performance Measure Development.

AAO-HNS indicates American Academy of Otolaryngology–Head and Neck Surgery; AHRQ, Agency for Healthcare Research and Quality; AMA-PCPI, American Medical Association–Physician Consortium for Performance Improvement; CMS, Centers for Medicare & Medicaid Services; and NQF, National Quality Forum.

Clinical Practice Guidelines as Process Measures

Within otolaryngology, past performance measures have come from translating clinical practice guidelines into process measures. When a clinical practice guideline establishes a best practice, the performance measure then becomes determining how often this practice is followed. Similar to a statistical regression analysis of actual vs expected outcomes, a practice guideline is the clinical correlate.²³ Specifically, strong recommendations from clinical practice guidelines can be converted to effective performance measures.²⁴

One example of a performance measure in otolaryngology that has been developed using a clinical practice guideline is the use of tympanometry to diagnose otitis media with effusion in children. The key action statement from this guideline, a “strong recommendation to use tympanometry or pneumatic otoscopy in diagnosis of [otitis media with effusion],” was converted to a process measure (ie, how often this procedure was followed).^25(p598) Using this performance measure, Lannon et al²⁶ were able to show that only 33% of pediatric clinics were following this strongly recommended practice. This finding may be the result, in part, of a failure in documentation since this study was conducted by using a review of medical records. However, a study by Patel et al²⁷ that surveyed otolaryngologists on how they diagnosed otitis media with effusion found that 25 of 29 of the respondents (86.2%) reported using pneumatic otoscopy or tympanometry to make the diagnosis, meaning that at least 1 of 10 otolaryngologists surveyed did not follow the guidelines. This is but one example of how performance measures may highlight areas in which we are not following our own evidence-based guidelines.²⁸

One advantage of using clinical practice guidelines as process measures is that the bulk of the data collection has already been done. Thus, enforcing the adoption of an action carrying a strong recommendation from a guideline is relatively straightforward. A disadvantage of this method is that there are relatively few procedures for which guidelines exist, and guideline development will always lag years behind new procedures, since they require robust evidence for their endorsement. When guidelines do not exist for a procedure, alternative methods of quality measure development must be sought.

Using Clinical Registries for Performance Measures

Clinical registries are an excellent source of data from which to develop performance measures because the data can be of very high quality and prospectively collected. Having a large collection of patients in a focused registry allows for comparison of patients going through similar care pathways. Both process and outcome measures can then be developed from these data and subsequently tested.

Our cardiology colleagues have served as outstanding role models. By encouraging participation in the Get With the Guidelines– Stroke program, Schwamm et al²⁹ were able to show improvement in 8 separate performance measures in a sample of 790 hospitals within the United States. For example, the percentage of patients presenting within 2 hours of stroke symptom onset who received intravenous tissue plasminogen activator within 3 hours of symptom onset increased from 42% at baseline to 73% across the entire sample of 322 847 patients after 5 years of participation in the program. With strong process measures, it may be possible to encourage similar changes in otolaryngology.

An advantage of using clinical registries for performance measure development is that much larger numbers of patients can be studied than possible in single-center or even multicenter studies in academic centers. A disadvantage of this method is that the quality of the data are dependent on the level of detail recorded in the registry. As seen in studies based on administrative data, at times the conclusions may be quite limited, as seen in studies of thyroidectomy from the National Inpatient Sample.³⁰

Other Methods of Developing Performance Measures

We should not preclude developing quality measures for procedures for which there are no existing clinical practice guidelines or registries. Although these quality measures may not be as robust as performance measures (and thus not suitable for public reporting), solo or group practices, academic departments, and hospitals may still benefit from tracking quality measures internally. Furthermore, by starting the process of developing and tracking quality measures, we begin the long process of performance measure development by presenting evidence to organizations such as the AMA-PCPI to conduct more rigorous testing.³¹

There is compelling evidence for provider volume as a quality measure. A study³² of the National Inpatient Sample showed that, for certain procedures (eg, pancreatectomy), the postoperative mortality rate varied from 3.8% in high-volume centers to 16.3% in low-volume centers after adjusting for patient age, sex, race, procedure year, urgency of admission, Charlson score, and socioeconomic status. However, the use of provider volume as a quality measure is controversial. Although differences in mortality across low- vs high-volume hospitals are observed on the aggregate level, provider volume is not a good predictor of individual hospital mortality rates. In addition, not all procedures are associated with a difference in provider experience.⁶ Thus, we must be careful not to overuse this measure by assuming it to be true of all surgical procedures and also not unfairly penalize high-performing hospitals regardless of their volume. However, for selected procedures, including pancreatectomy and esophagectomy,³³ provider volume can be an effective performance measure.³⁴

The development of patient-centered outcome measures should be a priority for otolaryngologists. Although performance measures focused on morbidity and mortality are well suited for high-risk procedures, low-risk procedures require patient-centered outcome measures, especially when the goal of the intervention is to improve quality of life.⁶ An example of such a procedure is cochlear implantation²¹; the risk of mortality is low, but the effect on quality of life from a poor outcome can be tremendous, preventing a child from attending mainstream schools or an adult from continuing to work.

An advantage of alternative forms of performance measure development other than using guidelines or registries is that almost any topic can be targeted within reason. The combination of a systematic review and an expert panel can provide a similar framework to guideline development and result in the creation of high-quality performance measures.³⁵ A disadvantage of this method is that there are added steps in advancing from a quality measure to a publicly reportable performance measure because endorsement by the American Academy of Otolaryngology–Head and Neck Surgery must be obtained prior to submitting to national quality organizations, such as the AMA-PCPI.

Conclusions

Performance measures are an important tool that can aid otolaryngologists in achieving effective, efficient, equitable, timely, safe, and patient-centered care as outlined by the Institute of Medicine. The use of performance measurement, both for quality improvement and cost containment, is here to stay. As experts in our specialty, we must take the lead in creating well-developed quality and performance measures.