Developing Quality Measures for Adult Cochlear Implant Centers: Preliminary Findings

Abstract

The study objective was to develop quality measures for adult cochlear implant centers. A modified Delphi design beginning with focus groups of surgeons and audiologists was used, adapted from the American College of Cardiology/American Heart Association method for creating quality measures. Two academic cochlear implant programs and one private program participated. Qualitative focus group analysis yielded 58 candidate measures. An additional 5 candidate measures were added from a systematic review of the literature. After excluding pediatric measures, structure measures, and process measures, and discussing details and implications of each measure, 8 measures remained as the preliminary Adult Cochlear Implant Outcome (CI-OUTCOME) Measure Set. This study provides a preliminary set of measures for evaluating the quality of adult cochlear implant centers, based on input from implant surgeons and audiologists. The next steps will be to gather feedback from implant patients.

INTRODUCTION

The cochlear implant (CI), approved by the FDA nearly 30 years ago, is a safe and effective technology that partially restores a sense of sound in hearing-impaired and deaf individuals. CI technology has been improved dramatically over time, allowing patients to obtain meaningful employment in the hearing world, for example. These outcomes directly lead to improvement in overall quality of life for these individuals.^1,2

Obtaining these devices is expensive, however. While payors continue to shift towards pay-for-performance models, the Centers for Medicare and Medicaid Services have declared that their goal is to have 90% of all Medicare fee-for-service payments tied to quality by 2018,³ implying sweeping changes to current payment structures. The resulting implication is that quality measures will be fundamentally involved with reimbursement.

Otolaryngologists are beginning to make progress in the development of quality measures. The recent announcement of Regent, the first otolaryngology-specific clinical data registry, is exciting and promises to help accelerate the creation of new quality measures.⁴ In addition, clinical practice guidelines continue to provide targets for quality measurement. Unfortunately, no quality measures currently exist specifically for hearing-related conditions.⁵ Effective quality measures would not only allow patients, referring physicians and audiologists to select a high-performing center, but would also allow surgeons to identify best practices. The goal of this study was to develop quality measures for adult CI centers.

METHODS

This study was deemed exempt by the Washington University in St. Louis IRB. We collected feedback from experts in cochlear implantation. Because children have a much more complicated period of aural rehabilitation and language development, we decided to start with adults. We also set the level of analysis at the center, rather than focus on the individual surgeon or audiologist, as quality improvement should be focused on the system delivering care.^6,7

Using the American College of Cardiology/American Heart Association method for developing performance measures,⁸ we first held focus groups with CI surgeons and audiologists at Washington University in St. Louis, the University of Miami, as well as a neurotologist in solo private practice in St. Louis, MO. Focus groups centered on educating participants about quality measures, then engaging them on what to measure to ensure high quality care in a CI center. Conversations were recorded and reviewed to extract candidate measures.

A systematic review of randomized, controlled trials was also performed to examine the outcome domains reported in the literature.⁹ After including measures from the systematic review, measures were organized using the Donabedian framework¹⁰ into structure, process, and outcome. Measures only applicable to children, structure, and process measures were excluded. A clinical rationale, numerator, denominator, data source, period of care, and exclusions (expected violations of the measure which should not be counted) were specified for each candidate measure.

Two conference calls were held with a group of 11 neurotologists to review each measure in detail, with attention to each of the above mentioned categories. Comments were incorporated, and measures were excluded if they did not satisfy the following criteria: 1) based on scientific evidence, 2) interpretable by practitioners, 3) provides actionable targets, 4) applies to a meaningful group of patients, 5) high face validity, 6) high content validity, 7) high reproducibility across centers, 8) feasible to record based on cost, and 9) feasible to record based on time period specified.¹¹ The emphasis in the conference calls was on including outcome measures, though process measures were included if felt to be sufficiently important. Finally, the measures were presented to a group of 9 adult CI audiologists to discuss the specifics of each measure, and feedback was incorporated into the final version (Figure 1).

Figure 1.

Method for developing quality measures for adult cochlear implant centers used in this study, as adapted from the American College of Cardiology/American Heart Association.⁸

RESULTS

Qualitative analysis of the focus groups yielded 58 candidate measures. An additional 5 candidate outcome measures from the systematic review not brought up in the focus groups were added, and after excluding candidate measures not satisfying inclusion criteria, 13 candidate measures remained. Five additional measures were excluded from feedback collected during the conference calls to discuss the components of each measure in detail, and audiologists provided feedback on each measure, leaving 8 quality measures included as the preliminary Adult Cochlear Implant Outcome (CI-OUTCOME) Measure Set (See Table 1).

Table 1.

The CI-OUTCOME measure set for adult cochlear implant centers.

Measure	Rationale	Numerator	Denominator	Data Source	Period of Care	Exclusions
1. Preoperative Vaccinations	Patients should be vaccinated with 13-valent pneumococcal conjugate vaccine (PCV-13, Prevnar 13) and 23-valent pneumococcal polysaccharide vaccine (PPSV-23, Pneumovax 23) prior to obtaining a cochlear implant, per the Centers for Disease Control (CDC) guidelines.12	Patients undergoing cochlear implantation that have received both vaccinations prior to the implantation date.	Patients undergoing cochlear implantation.	Documentation in the medical chart by the implant surgeon.	Within five years prior to the date of implantation.	None.
2A. Immediate Facial Nerve Preservation	The facial nerve should not be injured during placement of a cochlear implant.	Patients undergoing cochlear implantation with unchanged facial nerve function after implantation as compared to their preoperative status.	Patients undergoing cochlear implantation.	Documentation of facial nerve weakness using the House-Brackmann scale in the medical chart by the implant surgeon.	Within 24 hours after implantation.	None.
2B. Delayed Facial Nerve Preservation	The facial nerve should not be injured during placement of a cochlear implant. In the unfortunate event that the facial nerve is injured during surgery, it should recover to the preoperative level of function.	Patients undergoing cochlear implantation with unchanged facial nerve function after implantation as compared to their preoperative status.	Patients undergoing cochlear implantation.	Documentation of facial nerve weakness using the House-Brackmann scale in the medical chart by the implant surgeon.	Within 6 months from the date of implantation.	None.
3. Postoperative Infection Rate	The postoperative infection rate from cochlear implant surgery should be reported, in order to identify high- and low-level performers.	Patients requiring an inpatient admission for intravenous antibiotics due to a surgical site or wound infection after cochlear implantation.	Patients undergoing cochlear implantation.	Diagnosis code for postoperative infection (ICD-9-CM 998.59, ICD-10-CM T85.79) and an inpatient admission in the medical chart.	Within one month from the date of implantation.	Patients with immune disorders.
4. Reoperation Rate	The rate of return to the operating room from cochlear implant surgery should be reported, in order to identify high- and low-level performers.	Patients returning to the operating room for revision of the electrode insertion, position of the receiver stimulator, or removal of the implant due to infection.	Patients undergoing cochlear implantation.	Operative note from return to operating room in the medical chart.	Within six months from the date of implantation.	Patients undergoing reoperation for device failure, malfunction, or upgrade.
5. Speech Perception	As the most commonly reported outcome measure for cochlear implantation, speech perception should be reported in a manner that allows for ease of tracking, and should be measured postoperatively in every patient.	Patients undergoing cochlear implantation with a documented speech perception score.	Patients undergoing cochlear implantation that are physically and mentally capable of participating in a speech perception test.	Minimum Speech Test Battery for Adult Cochlear Implant Users (MSTB), including the Consonant-Nucleus-Consonant (CNC) Word Test, AzBio sentences, or Bamford-Kowal-Bamford Speech-in-Noise (BKB-SIN) test reported in the medical chart.	Within six months from the date of implantation.	None.
6. Cochlear Implant Usage	Patients using their cochlear implant at least 8 hours a day may be a marker of a successful user, and implant centers should encourage their patients to follow this practice.	Patients with at least 8 hours of average daily use as logged on their cochlear implant.	All patients undergoing cochlear implantation with a data-logging feature on their cochlear implant.	Documentation from the audiologist in the medical chart.	Measured in the previous month, at least six months from the date of implantation.	None.
7. Quality of Life Improvement after Implantation	Quality of life is positively impacted by cochlear implantation, and should be measured and reported.	Patients with an improvement of at least 10% after implantation.13	All patients undergoing cochlear implantation.	Abbreviated Profile of Hearing Aid Benefit (APHAB) results from the medical chart.	Comparing the preoperative score to at least six months from the date of implantation.	Patients with inability to fill out a quality of life questionnaire.

DISCUSSION

In this semi-qualitative study applying a widely used method for developing performance measures, we have developed a preliminary set of quality measures to use for adult CI centers. In the first phase of this study, feedback was collected from both implant surgeons and implant-trained audiologists, and efforts were taken to focus not only on large academic centers. In considering the burden of collecting and reporting this data, some decisions were made that the authors acknowledge may not be followed currently at most CI centers. For example, the APHAB was the recommended quality of life instrument due to ease of administration and limited number of questions. Though other centers may use different instruments, or not track this over time, our hope is that centers will be encouraged to collect these data in the future by widespread reporting.

Future Directions

In the absence of a gold standard for measuring the performance of an implant center, measures must be developed de novo. Though this preliminary set of measures has high face validity due to the method used in this study, we have not yet included feedback from patients, and the measures have not yet been validated. The next step will be to collect feedback from implant patients using a similar methodology, and subsequently test the measures in implant centers around the United States.