Development and Implementation of the Cochlear Implant Quality of Life (CIQOL) Functional Staging System

Abstract

Objectives:

The purpose of this study is to develop and implement a functional staging system using the Cochlear Implant Quality of Life (CIQOL) framework. The CIQOL-35 Profile was developed and validated following a rigorous research design and found to be more comprehensive and psychometrically sound than previous patient-reported outcome measures (PROMs) applied to adult CI users. However, interpreting the CIQOL-35 Profile (and all PROMs) relative to real-world functioning remains difficult for patients and clinicians, which limits the capacity of PROMs to direct clinical care. To address this limitation, a functional staging system based on PROM scores was developed to provide detailed descriptions of patients’ self-reported abilities (clinical vignettes) without sacrificing the inherent value of the psychometrically derived scores. The current study (1) creates an evidence-based CIQOL functional staging system using advanced psychometric techniques, (2) confirms the clarity and meaningfulness of the staging system with patients, and (3) implements the staging system to measure CIQOL stage progression using data from a longitudinal study design.

Methods:

Item response theory (IRT) analyses of CIQOL-35 Profile data from 705 experienced adult CI users and expert opinion were used to determine the cut-scores that separated adjacent stages for the six CIQOL-35 domains (communication, emotional, entertainment, environment, listening effort, and social). The research team then created clinical vignettes based on item response patterns for each stage. Semi-structured key informant interviews were conducted with 10 adult CI users to determine the clarity and meaningfulness of the CIQOL stages and associated clinical vignettes. Finally, we prospectively collected CIQOL-35 Profile scores from 42 CI users prior to cochlear implantation and then at 3- and 6-months post-CI activation to measure CIQOL stage progression.

Results:

Psychometric analyses identified five statistically distinct stages for the communication domain and three stages for all other domains. Using IRT analysis results for guidance, research team members independently identified the cut-scores that represented transitions between the functional stages for each domain with excellent agreement (κ = 0.98 [95% confidence interval 0.96–0.99]). Next, the key informant interviews revealed that CI users found the clinical vignettes to be clear and only minor changes were required. Participants also agreed that stage progression represented meaningful improvements in functional abilities. Finally, 88.1% of 42 patients in the prospective cohort (n = 37) improved from pre-CI functional stage by at least one functional stage in one or more domains. The communication domain had the greatest number of patients improve by one or more stages (59.5%) and the social domain the fewest (25.6%). There was also a trend for less improvement at 3- and 6-months post-CI activation for patients at higher pre-CI functional stages, even though higher stages were achievable.

Conclusion:

The new CIQOL functional staging system provides an evidence-based understanding of the real-world functional abilities of adult CI users from pre-CI to 3- to 6-months post-CI activation across multiple domains. In addition, study results provide the proportion of CI users in each stage at each timepoint. Results can be used during discussions of expectations with potential CI users to provide enhanced insight regarding realistic outcomes and the anticipated timing for improvements. The use of the CIQOL functional staging system also presents an opportunity to develop individualized goal-based rehabilitation strategies that target barriers to stage advancement faced by CI users.

INTRODUCTION

Cochlear implantation is the standard of care treatment for adult with bilateral moderate to profound hearing loss who no longer receive benefit from hearing aids. While cochlear implants (CIs) have demonstrated an impact on patients’ lives,^1–5 outcomes have primarily focused on speech recognition ability improvement measured in controlled, clinical settings.^6,7 Recently, an increased emphasis has been placed on understanding the broader impact of cochlear implantation through the use of patient-reported outcome measures (PROMs).^8–20 This is consistent with evidence of weak associations between speech recognition scores and self-reported real-world functional abilities of CI users, which suggests that PROMs provide unique information not available from speech recognition scores.^{14,17,18,21–23}

The increased emphasis of PROMs is not unique to cochlear implantation. Their importance is highlighted by the Centers for Medicare and Medicaid Services identifying PROMs and quality of life improvement as “meaningful measures,” which designate these instruments as those that “reflect core issues that are most vital to high quality care and better patient outcomes.”²⁴ PROMs are also now required as a primary outcome measure in any trial where investigators are seeking FDA approval for an intervention.²⁵

The real-world communication experiences of adult CI recipients are far more complex than can be simulated using standard clinical speech recognition measures. PROMs allow patients to report functional communication abilities through validated instruments. These experiences include social, emotional, and listening effort abilities that may change following cochlear implantation but are ignored in the standard CI test battery.^6,7 As such, there is a gap between our current clinical outcome measures for adult CI users and their real-world experiences. The Cochlear Implant Quality of Life-35 (CIQOL-35) Profile was developed to address this gap.

The CIQOL-35 Profile was developed and validated following the rigorous Patient-Reported Outcomes Measurement Information System (PROMIS)²⁶ and COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN)²⁷ guidelines. This comprised a sequential mixed methods research design including stakeholder engagement by using CI patient focus groups, cognitive interviews, and extensive psychometric analyses (item-response theory; IRT). These methodologies have rarely been applied to adults with hearing loss and CI users but provide substantial benefit from a measurement perspective. First, the qualitative portion provided the instrument’s face and content validity and ensured the topics and items included in the instrument were meaningful to the populations of interest.²⁰ Moreover, a logical item difficulty hierarchy was established, consistent with CI patient perspective regarding functional ability levels for each domain.²⁰ Second, the application of the item-level analyses provides many potential measurement advantages compared to test-level psychometrics (classical test theory). IRT identifies the fit of each individual item to the hierarchical model, matches individual item difficulty to person ability level, and ensures that the included items cover the ability range of the population of interest.^28–30 With this information, items can be ordered by difficulty to create a hierarchical ability-level model. As such, scores derived from IRT-developed instruments are interval in nature and independent of the sample tested.²⁹ Therefore, changes in individual CI patient CIQOL responses characterize an ordered improvement in self-reported functional ability. This contrasts with the previous standard for PROM development, classical test theory, which provides ordinal scores. Although IRT is now being used to develop other hearing- and CI-specific PROMs,^8,31,32 the CIQOL-35 Profile is the only fully validated instrument available for adult CI users to provide IRT-derived scores that are interval in nature. In addition, the CIQOL-35 Profile has been found to be more comprehensive and psychometrically sound than the most commonly used PROMs.¹⁴ Finally, given that more than 700 adult CI patients from all regions of the United States were included in the development and validation of the CIQOL-35 Profile instrument,^14,16,19 the established hierarchical item structure is generalizable to the adult CI population.

The CIQOL-35 Profile consists of 35 items organized into six domains (communication, emotional, entertainment, environmental, listening effort, social). Patients respond to each item on a Likert scale with response options that range from “never” to “always” being able to endorse the functional ability represented by the item. Based on response patterns, a raw score is calculated and then converted into an IRT-based outcomes score that is interval in nature. Scores for each domain range from lowest (0) to highest (100) functional abilities.^14,16,33 Despite the improved capacity to measure the real-world benefits from cochlear implantation, the CIQOL-35 Profile (and all PROMs scores) have some of the same interpretation barriers as speech recognition scores, specifically the difficulty for patients and clinicians to easily translate numerical scores to real-world experiences. PROM scores are typically presented as integer values that can either be interpreted relative to their location within the range of possible scores (i.e., 0–100) or compared to group outcomes from patients with or without the condition (i.e., T-scores). Neither method is particularly informative for patients or clinicians who are trying to understand the meaning of a PROM score to make treatment decisions. Although changes in scores can be monitored over time, the lack of inherent meaning of the score limits meaningful interpretation and patient discussions.

The development of functional staging systems based on PROM scores directly addresses these limitations and enhances the capacity of PROM scores to provide an evidence-based understanding of functional abilities and improve patient care. Functional staging systems provide detailed descriptions of patients’ self-reported performance (clinical vignettes) without sacrificing the inherent value from the IRT-derived quantitative scores.^34–36 This means that PROM domains can have multiple stages that maintain the hierarchic ability structure established during PROM development. These stages can then be used to monitor individual patient progress and identify the patient-specific barriers within specific domains that prevent further functional improvement.^34–36 Moreover, data regarding stage progression (along with associated clinical vignettes) can provide meaningful insight for patients prior to implantation to provide realistic expectations regarding outcomes and the timing for improvement, both of which have been identified as important to clinicians³⁷ and CI users.³⁸

The current study reports the development and implementation of the CIQOL functional staging system to demonstrate its readiness for clinical and research use. The study is separated into three phases (Fig. 1). First, we applied advanced psychometric techniques to create an evidence-based functional staging system for adult CI users. Second, we performed key-informant interviews with adult CI users to (a) ensure the clinical vignettes associated with each stage are clear to CI users; (b) confirm that stage progression represents meaningful improvement in ability; (c) better understand patients’ perspective of the staging system’s utility in CI care. Third, we then pilot tested CI users’ CIQOL stage progression in a longitudinal design in a patient cohort.

Fig. 1.

Outline of the three phases of the current study.

METHODS

Phase I: Development of the CIQOL Staging System

Participants and data collection.

IRB approval was obtained through our institution for each phase of this study. Data used to develop the staging system were acquired during the development and validation of the CIQOL-35 Profile instrument.^14,16,19 Here, 705 CI users with ≥12 months of CI experience were recruited through the CIQOL Development Consortium, which consists of 30 institutions established to recruit a large sample of CI users who were representative of the broader adult CI population. Participants were recruited through flyers distributed on paper and electronically through the centers. Participants were required to: (1) be between 18 and 89 years of age (as individuals >89 years of age are considered a special population), (2) have used a CI for 1 year or more, (3) have post-lingual moderate-to-profound hearing loss, and (4) not have received a CI for single sided deafness. Participants completed the CIQOL-35 Profile through REDCap (Research Electronic Data Capture), a secure web-based data collection platform. In addition, all participants completed a demographic and hearing/CI history survey. Participants also obtained their most recent best aided speech recognition scores from their audiologist and entered them into REDCap. These scores could include Consonant-Nucleus-Consonant (CNC) word scores and AzBio sentence scores in quiet and in noise at a +10 dB signal-to-noise ratio (SNR), as these are components of the minimum reporting standards.⁶ Participants were not excluded if they could not obtain speech recognition scores.

Identification of cut-scores for each stage.

The number of stages for each domain was based on the number of statistically distinct groupings into which patients can be reliably assigned based on their ability level. Termed strata, this IRT-derived value accounts for the measurement error of the instrument and the person ability levels of the study population³⁹ The number of strata for each domain were previously calculated and reported,¹⁴ which resulted in five stages for the communication domain and three stages for all others. Next, using IRT analysis (Winsteps 3.93.1), we identified the Rasch-half-point threshold (the point on the 0–100 CIQOL outcome scale) that represents the boundary (0.5 probability) of being in one category (e.g., “often”) or the next higher category (“always”) for each item. This identified the range of ability level (outcomes scores) among CI users’ who endorsed each option on the rating scale (“never” to “always”) for every item in each domain. Item thresholds were then graphically displayed to demonstrate the hierarchical nature of the items for each domain and the ability level thresholds (see Fig. 2, horizontal bars). Next, two CI audiologists, a CI surgeon-scientist, a hearing research scientist, and two psychometricians used these threshold data to independently place the cut-scores that separated CI users into distinct functional stages. Each reviewer was provided the same instructions and did not confer with each other. These instructions identified the number of stages that should be created for each domain (based on strata) and asked the individuals to use the differing ability levels and associated response patterns (Fig. 2) to identify these functionally distinct groupings. Cut-score values were then compared for agreement among the reviewers using intraclass correlation coefficients (MedCalc). The median values for each cut-score were calculated and the final cut-score was placed at the nearest item-level rating score threshold (Fig. 2, vertical black bars). These final values were then presented to the reviewer who agreed on their location. Finally, based on item response patterns, the reviewers developed clinical vignettes to describe the functional abilities for each stage. For instances where several response options for an item were included in a stage, broader, encompassing language was used to describe the full range of functional abilities associated with that stage. In accordance with PROMIS standards, the Lexile Analyzer was then used to ensure that all clinical vignettes had a 6^th grade or lower reading level.²⁶

Fig. 2.

Outcome scores (0–100) associated with responses to the items in the CIQOL-35 Profile (n = 705). Items are listed by domain on the left in order of increasing difficulty (bottom to top). Vertical lines represent cut-scores that differentiate functional stages. Values for cut-scores are provided in Table IV. *denotes items that are reversed scored.

Phase II: Key Informant Interviews

Participants and data collection.

Ten CI users from our CI center were recruited for key informant interviews. Given the planned implementation of the CIQOL staging system for longitudinal CI care, participants were enrolled along the range of pre-CI to post-CI. These semi-structured interviews covered the following topics: (1) clarity of clinical vignettes associated with each functional stage (2) whether stage progression for each domain represents meaningful improvement in functional abilities; and (3) what functional stage each participant believed they were in for each domain. In accordance with PROMIS standards, if major changes were required, then an additional five participants would be recruited.²⁶ As discussed later, this was not required. The meaningfulness of stage progression was determined by reviewing with the participants the clinical vignettes for each stage for each domain. Each clinical vignette was presented along with clinical vignettes for adjacent stages. Participants were then asked whether moving to a higher stage would represent a meaningful (1) decrease in functional ability; (2) increase in functional ability; or (3) no change. Participants were not made aware of the order of stages in which they were analyzing. After completion of the interview, participants completed a REDCap survey that asked their preference for how CI outcomes should be discussed by clinicians during the CI evaluation process, during the post-CI activation follow-up visits, and with regard to potential benefits of CI-related interventions (such as increasing CI use or home listening activities). For each scenario, participants were asked to rank-order speech recognition score (0%–100%), CIQOL score (0–100), and CIQOL functional stage based on their preference (first to third choice).

Phase III: CIQOL Stage Progression Pilot Study

Participants and data collection.

We prospectively collected CIQOL-35 Profile data on consecutively implanted adult CI users in our CI center. In addition, we performed a retrospective review of our center’s prospectively maintained adult CI database of these same patients. Among other information, the database includes pre-operative and post-operative speech recognition scores. Given the relatively recently validation and implementation of the CIQOL-35 Profile, data in the current study are limited to pre-CI, and 3- and 6-months post-CI activation follow-up. Inclusion criteria were: documented history of post-lingual onset of hearing loss; age ≥ 18 years old; pre (aided)-and 6-month post-operative speech recognition scores; and pre, 3-month and 6-month CIQOL-35 Profile scores. Speech recognition was measured using Consonant-Nucleus-Consonant (CNC) word and/or AzBio sentences scores in quiet or +10 dB SNR.^40,41 Patients were not excluded if one or more CIQOL-35 Profile domain data were missing or incorrectly completed. Exclusion criteria were: CI surgery performed at another institution; revision cochlear implantation; and cochlear implantation for single-sided deafness.

The following data were extracted from our adult CI patient database: age at implantation, sex, duration of hearing loss prior to CI, and listening modality (CI with or without contralateral hearing aid). Duration of hearing loss prior to CI was defined by self-reported number of years with hearing loss prior to implantation. Contralateral hearing aid use was defined by the patient’s self-reported active hearing aid use. Pre-operative speech recognition was measured with hearing aids (personal or stock aids) fitted to NAL-RL targets¹⁸; post-operative testing was conducted in the CI-only condition. All speech recognition testing was performed in a sound-treated room in the sound field with speech presented at 60 dB SPL (0 degrees azimuth). Cohen’s effect sizes (d with 95% CIs) were calculated to compare pre- to post-CI changes in CIQOL domains and speech recognition scores. Per Cohen’s convention, effect sizes were interpreted as follows: 0.2–0.49 = small effect; 0.5–0.79 = medium effect; and ≥0.8 = large effect.⁴²

RESULTS

Phase I: Development of the CIQOL staging system

As described in Table I, 705 experienced CI users were included in the development and validation studies for the CIQOL-35 Profile. Overall, there were more female than male participants. Most were married with approximately half having children <18 years of age in the household. The majority lived in suburban environments, and similar numbers lived in urban and rural locales. The vast majority of participants were White and not Latinx. Participants were fairly evenly split among the household income categories except in the lowest ($0–$20,000). Most had some education beyond a high school diploma and were employed or were retired. All regions of the United States were represented with individuals from our institution representing only 2.8% of participants. The full range of age at implantation, duration of CI use, speech recognition abilities listening modalities, and CI manufacturers were represented (Tables II and III).

TABLE I.

Demographic and Hearing History of Phase I Participants (n = 705).

Variable	n (%)
Gender
Male	285 (40.4)
Female	420 (59.6)
Marital status
Married/domestic partner	472 (67.0)
Separated/divorced	93 (13.2)
Single, never married	100 (14.2)
Widowed	40 (5.7)
Has children <18 in the home	362 (51.3)
Environment where participant lives
Rural	130 (18.4)
Suburban	408 (57.9)
Urban	167 (23.7)
Race
Asian	6 (0.9)
Black or African American	9(1.3)
Hawaiian or Other Pacific Islander	1 (0.1)
White	663 (94.0)
More than 1 race	9(1.3)
Not reported	17 (2.4)
Ethnicity
Hispanic or Latinx	24 (3.4)
Not Hispanic or Latinx	600 (85.1)
Not reported	81 (11.5)
Combined annual household income
$0–$20,000	40 (5.7)
$20,001–$50,000	129 (18.3)
$50,001–$80,000	166 (23.5)
$80,001–$110,000	125 (17.7)
>$110,000	179 (25.4)
Not reported	66 (9.4)
Highest level of education
Nursery school to 8th grade	1 (0.1)
Some high school, no diploma	2 (0.3)
High school graduate or equivalent	46 (6.5)
Some college	97 (13.8)
Trade/technical/vocational school	28 (4.0)
Associate degree	67 (9.5)
Bachelor’s degree	221 (31.3)
Master’s degree	157 (22.3)
Professional degree	30 (4.3)
Doctorate degree	56 (7.9)
Employment status
Employed, working 40+ h/week	220 (31.2)
Employed, working <40 h/week	91 (12.9)
Not employed, looking for work	20 (2.8)
Not employed, not looking for work	31 (4.4)
Retired	304 (43.1)
Disabled, not able to work	39 (5.5)
Region
Midwest	158(22.4)
Northeast	96 (13.6)
South	262 (37.2)
West	176(24.9)
Not Reported	13(1.9)
Hearing Modality
Bilateral CI	346(49.1)
CI and Hearing Aid	201 (28.5)
CI without Hearing Aid	158(22.4)

TABLE II.

Participant Hearing and CI History. n Indicates the Number of Participants Who Were Able To Provide Speech Recognition Data.

Variable	Mean (SD)
Age	59.5 (15.2)
Duration of hearing loss, years	26.6 (18.1)
Duration of CI use, years	7.6 (6.7)
CNC Word scores (%, n = 371)	68.4 (23.8)
AzBio Sentences in quiet (%, n = 378)	78.7(24.1)
AzBio Sentences in noise at +10 dB SNR (%, n = 252)	64.4 (26.2)

TABLE III.

Participant CI Device Information (n = 705).

Variable	n (%)
CI Company
Advanced Bionics	138 (19.6)
Cochlear	343 (48.7)
MED-EL	223 (31.6)
Not reported	1 (0.1)
Listening Modality
Bilateral CI	346 (49.1)
CI and Hearing Aid	201 (28.5)
CI without Hearing Aid	158 (22.4)
Combined electro-acoustic hearing (hybrid)
Yes	678 (96.3)
No	26 (3.7)
No response	1 (0.1)

The IRT-derived threshold values between adjacent rating scale options for each of the items in the CIQOL-35 Profile are graphically displayed in Figure 2 (multicolored, horizontal bars). Items are listed for each domain in ascending order of difficulty (bottom to top). The range of outcomes scores associated with each rating scale option for each item are based on the psychometrically derived person ability levels. As expected, given the established item hierarchy, responding “always” (light blue) to an easier item (lower rows within a given domain) corresponds to a wider range of lower outcome scores, but responding “always” to the most difficult items (higher rows within a given domain) corresponds to a narrower range of higher outcome scores. In contrast, responding “never” (dark blue) to an easier item is associated with a narrow range of lower scores but associated with a wider range of scores for more difficult items. As seen, use of the full rating scale for all items provides the capacity to precisely identify CI users’ functional ability for each domain.

Based on these transitions and previously established strata for each domain, the research team members (described in methods), independently identified the cut-scores that represented transitions between functional ability populations. Overall, there was excellent agreement between reviewers regarding the location of the cut-scores (κ = 0.98 [95% CI 0.96–0.99]). The vertical lines in Figure 2 separate adjacent functional stages for each domain. This demonstrates how item responses correspond to the functional stages. Stage I describes patients who responded “never” or “rarely” to even the easiest items and higher stages (e.g., stages III-V) describe patients who are most likely to respond “often” and “always” for all items, even the most the difficult. For all domains, stage I represents patients with the lowest functional abilities with higher stages representing incremental increases in functional abilities. Based on the psychometric properties of each domain (i.e., strata), communication domain had five stages (I–V) and the other domains had three stages (I–III).

The CIQOL stage distribution of the 705 CI users included in this study is displayed in Table IV. Given that all participants were experienced CI users, it was anticipated that there would be a fairly low number of users in stage I for each domain. Nevertheless, inclusion of stage I and its associated low CIQOL score range for each domain allows for assessment of patient functional abilities prior to implantation. The distribution patterns for other stages differed based on CIQOL domain. For example, the emotional, environment, and social domains each had >30% of participants in stage III, the highest stage for these domains. In contrast, far fewer users were in the highest stage for the communication (stage V) and listening effort (stage III) domains. Finally, based on the item response patterns, the research team developed clinical vignettes for each functional stage (discussed later; Table IV).

TABLE IV.

The CIQOL Score Ranges, Number and Percentage of Participants (of Total n = 705), and Clinical Vignettes Describing the Functional Abilities Associated With Each CIQOL Stage.

Domain	Stage	Score range	Patients (%)	Clinical Vignette
Communication	I	0–15.1	3 (0.4)	• Unable to have a conversation in any listening environment
	II	15.2–37.0	91 (12.9)	• Can sometimes have a conversation in quiet environments • Other people’s voices may sometimes sound clear and natural • Need people to repeat themselves to understand conversation in quiet environments • Usually unable to have a conversation in noisy environments
	III	37.1–60.1	453 (64.3)	• Sometimes able to have a conversation in small group in quiet environments • Has great difficulty understanding, even with lip reading, in noisy environments • Can sometimes have a conversation without asking people to repeat themselves
	IV	60.2–81.8	148 (20.0)	• Able to have a conversation in small groups in quiet • Rarely needs to ask a lot of questions about what is being said in a conversation • Can sometimes have a conversation in noisy environments without lip reading
	V	81.9–100	10 (1.4)	• Able to have a conversation in all listening environments with essentially no lip-reading • Other people’s voices always sound clear and natural
Emotional	I	0–24.9	8 (1.1)	• Hearing has a large, negative impact on emotional state • Hearing loss always results in irritability and feeling inadequate • Always keeps quiet to avoid saying the wrong thing
	II	25.0–59.4	339 (48.1)	• Hearing sometimes negatively impacts emotional state • Can sometimes feel comfortable being themselves • Hearing loss can result in irritability and feeling inadequate at times
	III	59.5–100	358 (50.9)	• Hearing loss rarely results in irritability and feeling inadequate • Always feels comfortable being themselves
Entertainment	I	0–34.2	125 (17.7)	• Usually unable to enjoy music • Usually unable to recognize melodies in music
	II	34.3–69.0	417 (59.1)	• Hearing loss may prevent them from listening to TV or Radio • Music does not always sound clear and natural
	III	69.1–100	163 (23.1)	• Usually able to enjoy music • Music usually sounds clear and natural • Hearing loss usually does not prevent them from listening to TV or radio
Environment	I	0–31.1	22 (3.1)	• Everyday sounds usually do not sound clear and natural • Usually unable to locate where sounds are coming from
	II	31.2–68.8	457 (64.8)	• Everyday sounds can sometimes sound clear and natural • May occasionally be able to hear someone approaching from behind
	III	68.9–100	226 (32.1)	• Typically able to distinguish sounds in nature • Everyday sounds usually sound clear and natural • Usually able to locate where sounds are coming from
Listening Effort	I	0–27.1	104 (14.8)	• Takes great effort and concentration to follow or participate in a conversation in any listening environment • Unable to ignore competing sounds and focus on person speaking
	II	27.2–67.8	560 (79.4)	• Can sometimes follow a conversation with minimal effort • Amount of concentration needed to participate in a conversation depends on the listening environment
	III	67.9–100	41 (5.8)	• Able to have a conversation in any environment without concentrating • Usually able to focus on the person speaking and ignore competing sounds
Social	I	0–27.2	12 (1.7)	• Typically avoids socializing and social events due to hearing loss • Usually does not have the confidence to socialize
	II	27.3–71.5	385 (54.6)	• Can sometimes join family and friend for social events • Can feel left out when with a group due to hearing loss
	III	71.6–100	308 (43.7)	• Usually socializes and attends social events when interested • Usually has the confidence to socialize • Usually does not feel left out when with a group due to hearing loss

Phase II: Key Informant Interviews

Demographic and hearing history data for key informant interview participants are included in Table V. Overall, the staging system and the clinical vignettes associated with the stages for each domain were found to be clear with only minor changes suggested by CI users. These were all grammatical in nature without any content related changes. Table IV displays the final clinical vignettes associated with each stage.

TABLE V.

Demographic and Hearing History For the Key Informant Interview Participants (n = 10).

Variable	n (%)
Sex
Female	4 (40)
Male	6 (60)
Race
Black or African American	2 (20)
White	8 (80)
Ethnicity
Not Hispanic or Latinx	10 (100)
Hearing Modality
Bilateral CI	2 (20)
CI and Hearing Aid	7 (70)
CI without Hearing Aid	1 (10)
	Mean (SD)
Age	61.8 (14.4)
Duration hearing loss, years	22.9 (12.3)
Duration CI use, year	1.9 (3.2)
CNC word score	56.3 (29.2)
AzBio quiet	62.1 (29.7)
AzBio +10 dB SNR	35.6 (32.1)

During the interview, all participants agreed that domain-specific stage progression represented meaningful improvement in functional ability. All participants completed the CIQOL-35 Profile at least 1 h prior to their interview and during the interview were asked to identify their domain-specific functional stage based on the provided clinical vignettes. When comparing these data, 78.9% of participants had CIQOL domain scores within the range of their self-identified CIQOL stage. Compared to CIQOL domains scores, 13.2% of participants self-identified higher CIQOL stages, and 7.9% lower CIQOL stages. The final portion of this study included a REDCap survey asking interview participants their preference regarding whether CI outcomes should be discussed using speech recognition percentage scores, domain specific CIQOL integer scores, or the CIQOL staging system with associated clinical vignettes. Participants were asked their preference for three scenarios: (1) potential CI outcomes at the time of CI evaluation; (2) monitoring their CI outcome progress during post-CI follow-up visits; and (3) potential benefit of CI-related interventions (e.g., increasing daily hours of CI user or use of CI listening activities). For all three scenarios, there was a fairly even (40.0%–60.0%) share of participants who preferred the CIQOL staging system and speech recognition scores. Interestingly, there was a clear subset of patients (60.0%) who ranked speech recognition scores last in perceived value when discussing the potential benefits of CI-related interventions.

Phase III: CIQOL Stage Progression Pilot Study

A cohort of 42 CI users who completed the CIQOL-35 Profile prior to cochlear implantation and at 3- and 6-months post CI piloted the CIQOL functional staging system. Mean age of this cohort was 63.1 years (±17.1) with a mean duration of hearing loss of 25.1 years (±18.5) prior to implantation. Additional demographics and hearing history are included in Table VI. The mean changes in speech recognition and CIQOL outcome score from pre-CI to 6 months post CI activation for each domain are displayed in Table VII. As seen, the cohort, on average, demonstrated medium to very large significant improvements in speech recognition and CIQOL scores (d range 0.53–2.46). The mean improvement in CIQOL domain scores ranged from (10.5–20.8) with the greatest improvements in the environment and communication domains.

TABLE VI.

Demographic of the Participant Cohort for Phase III (n = 42).

Variable	n (%)
Sex
Female	23 (54.8)
Male	19 (45.2)
Race
Black or African American	5 (11.9)
White	36 (85.7)
Not reported	1 (2.4)
Ethnicity
Not Hispanic or Latinx	41 (97.6)
Not reported	1 (2.4)
Hearing Modality
CI and Hearing Aid	35 (70.5)
CI without Hearing Aid	7 (15.9)

TABLE VII.

Mean Change In Speech Recognition and CIQOL Scores from Pre-CI to 6-Months Post Activation (n = 42).

Variable	Mean Pre-CI (±SD)	Mean 6 months post-CI (±SD)	Change (±SD)	Effect size (95% CI)
Speech recognition score
CNC word	12.1 (13.9)	59.6 (23.5)	47.5 (9.6)	2.46 (1.88;3.01)
AzBio quiet	15.1 (19.1)	67.6 (25.3)	52.5 (6.3)	2.34 (1.77;2.87)
AzBio +10 dB SNR	7.7 (15.2)	49.5 (28.0)	41.8 (12.8)	1.86 (1.33;2.35)
CIQOL outcome score
Communication	28.2 (13.4)	45.1 (9.4)	16.8 (4.0)	1.46 (0.96;1.93)
Emotional	44.3 (12.7)	58.3 (13.9)	14.0 (1.1)	1.05 (0.59;1.80)
Entertainment	38.3 (12.7)	49.9 (20.5)	11.6 (7.7)	0.68 (0.23;1.11)
Environment	32.7 (17.3)	53.5 (15.9)	20.8 (1.4)	1.25 (0.77;1.70)
Listening Effort	21.7 (15.5)	35.7 (12.2)	14.0 (3.3)	1.00 (0.54;1.45)
Social	51.3 (20.9)	61.8 (18.7)	10.5 (2.2)	0.53 (0.09; 0.96)

Compared to pre-CI ability, 88.1% (n = 37) of patients demonstrated improvement by at least one functional stage in one or more domains by 6 months. Patterns of functional stage change for each CIQOL domain are displayed in Table VIII. The communication domain had the greatest number of patients improve by one or more stages followed by listening effort. The social domain demonstrated the smallest number of patients improving from one stage to another and the largest number with decreased functional stage. The emotional, entertainment, and environment domains had similar improvement patterns. For all domains, patients who had no change in functional stage by 6 months experienced small increases in mean CIQOL outcome scores. The mean domain outcome score improvement for this cohort ranged from 4.8 (listening effort) to 9.1 (environment).

TABLE VIII.

Changes in CIQOL Outcome Score and CIQOL Stage By Domain from Pre-CI to 6 Months Post CI-Activation.

Domain	Improvement by ≥1 stage (n;%)	No change (n;%)	Decrease by ≥1 stage (n;%)
Communication	25 (59.5)	17 (40.5)	0 (0)
Emotional	16 (39.0)	24 (58.5)	1 (2.4)
Entertainment	17 (41.5)	22 (53.7)	2 (4.9)
Environment	18 (43.9)	23 (56.1)	0 (0)
Listening Effort	19 (47.5)	21 (52.5)	0 (0)
Social	10 (25.6)	25 (64.1)	4 (10.3)

Note: Some domains have less than 42 patients due to incorrectly completed or missing data.

The number of CI users in each CIQOL stage pre-CI, and 3- and 6-months post-CI activation are displayed in Figure 3. As anticipated, a larger percentage of patients were in stage I prior to implantation for all domains than the experienced CI users used to establish the staging system (Phase I, earlier). In general, there were more patients in higher stages compared to lower stages over time. Importantly, the magnitude of this change differed based on domain. For communication, all patients in pre-CI stage I (n = 7) advanced to either stage II (n = 3) or stage III (n = 4) by 6 months. The percentage of patients in stage III demonstrated the greatest increase over time (26%–69%). Interestingly, of the 11 patients in stage III prior to implantation, only one (9.1%) demonstrated an increase in stage by 6 months even though a large range of scores were available for improvement.

Fig. 3.

CIQOL functional stage progression for each domain from pre-CI to 3- and 6-months post CI. Percentages represent the percentage of patients in each stage at each time point (n = 39–42; see Table VIII). Note, no patient was in communication stage V at 6 months post-CI activation.

The entertainment, environment, and listening effort domains had a substantial proportion of patients in stage I prior to CI. This percentage decreased over time, demonstrating improvement, but several patients remained in stage I for the entertainment (n = 6) and listening effort (n = 9) domain by 6 months. In contrast, only one patient in environment stage I prior to implantation remained in that stage. For the environment domain, the number of patients in stage II demonstrated the greatest increase over time followed by stage III. However, these proportions were fairly stable from 3 to 6 months post-CI. Finally, the social domain showed the least change in stage proportions over time. Here, the greatest change was stage II to stage III membership, which occurred by 3 months and was stable by 6 months.

There were several patients at the highest functional stage (stage III) prior to CI for the emotional (n = 4) and social (n = 6) domains. Closer analysis of these individuals’ outcomes displayed interesting trends. First, all CI users who demonstrated a decrease in CIQOL stage for these two domains (Table VIII) were in stage III prior to implantation. Second, although these individuals were in the highest possible CIQOL stage for these domains prior to CI, a substantial range of CIQOL domain outcome scores were still available to monitor for longitudinal changes. For these patients, only one had an increase in CIQOL outcome score for the emotional domain and two for the social domain by 6 months.

DISCUSSION

Functional staging systems have been used in several rehabilitative fields but have never been available for hearing-related clinical care. The rigorous mixed methods used to create the CIQOL-35 Profile provides a unique opportunity to develop the CIQOL functional staging system. As demonstrated in the results, the staging system has the capacity to stratify patients based on their functional abilities within six CIQOL domains to provide a more comprehensive understanding of the real-world experiences of adult CI users.

The development of the CIQOL functional staging system is central in advancing CI clinical care from sole reliance on traditional speech recognition outcome measures to a more comprehensive, evidence-based patient-centered assessment that includes PROMs. The CIQOL staging system addresses the main limitation with PROM and speech recognition scores—interpreting these values with respect to real-world situations. The application of a functional staging system based on PROM scores advances the use of IRT-based psychometric methods beyond PROM development. As demonstrated in Phase I, the interaction between person ability and item difficulty that is fundamental for PROM development is applied to create a staging hierarchy that is both psychometrically (IRT) based and clinically meaningful to patients. Moreover, the key informant interviews performed in Phase II confirms the face/content validity of the staging system and its associated clinical vignettes. As discussed in the next phase, the application of the staging system has the potential to impact a wide range of areas in cochlear implantation clinical care.

Although the increased abilities associated with CIQOL stage progression are logical, the current study provides the first evidence-based understanding of the sets of abilities that define the full range of CI outcomes. For example, in the communication domain, the results demonstrate the impact of communicating in different listening environments, number of communication partners, and reliance on visual cues (lip-reading) on functional communication ability. Importantly, CI user CIQOL scores can be easily converted into a clinical vignette that not only provides increased understanding of the difficulties faced by individual patients but is also extremely valuable in providing accurate outcomes expectations for potential CI users prior to surgery.

Realistic CI outcome expectations are reported by CI audiologists to be one of the most important aspects to determine candidacy during cochlear implant evaluations.³⁷ Yet, clinicians have limited tools available to portray potential CI outcomes, including speech recognition scores for the average CI patient and clinicians’ personal experiences regarding real-world functional outcomes.³⁷ This is problematic as speech recognition scores demonstrate large variability^43–45 with few pre-operative predictors available to estimate likely outcomes.²¹ Moreover, the weak associations between speech recognition outcomes and patient-reported CI benefit are well-established.^17,18,22,23 Thus, even if accurate predictions of speech recognition outcomes were possible, they would provide minimal patient understanding of their potential real-world experience. In addition, the sole focus on speech recognition ignores the many non-communication benefits of cochlear implantation.^9,20

The current study provides the evidence necessary to begin the development of a patient-centered shared-decision making approach for adult cochlear implantation.⁴⁶ During the CI evaluation, clinicians can present the data in Table IV to demonstrate the percentage of CI users in each functional stage. Then, using the clinical vignettes, patients can be provided real-world examples of the potential improvements in abilities associated with implantation. For example, these data demonstrate that the vast majority of experienced CI users are in communication stage III or higher and are thus able to have conversations in small groups in quiet but continue to have difficulty in noisy environments. Clinicians can also use these data to counsel patients about the potential need for continued reliance on lip-reading as only 21.4% of CI users were able to communicate in noisy environments based on auditory input alone. Clinicians and potential CI candidates can also compare baseline (pre-CI) CIQOL stage to potential post-CI outcomes to better gauge the expected benefit from cochlear implantation. The use of these data in shared decision-making frameworks may have the greatest potential impact for patients who are borderline CI candidates based on speech recognition scores or only qualify for cochlear implantation based on speech recognition in noise scores. As we learn more about CIQOL stage progression and factors associated with distinct patterns of progression, the implementation of this staging system for clinical decision making will become increasingly effective.

An additional benefit of functional staging systems is the capacity to develop individualized rehabilitation strategies for CI users.^{34–36,47–50} Focusing treatment on the immediate barriers that impede progress is the cornerstone of rehabilitation. However, this degree of precision medicine has never previously been available to adult CI users. Using the CIQOL staging system, item response patterns that differentiate functional stages for each domain can be used to identify individualized rehabilitation goals. For example, consider a patient in communication stage III. The response patterns in Figure 2 show this patient communicates well in quiet environments, but still has difficulty in more complex listening situations with a single communication partner. Therefore, improvement in one-on-one conversation with background noise is an appropriate short-term rehabilitative goal for this patient rather than focusing on far more difficult items/situations. The staging system allows clinicians to identify transitions that are just beyond the patient’s ability level, which represent appropriate and potentially obtainable rehabilitation goals that would result in meaningful improvement. The CIQOL stating system represents the first time that these transition barriers can be identified and specifically targeted.

Importantly, this same goal-oriented rehabilitation can be accomplished for any domain, such as listening effort, emotional, or social domains. This unique aspect of the CIQOL-35 profile has the potential to introduce an innovative approach to CI care that treats the whole person, rather than solely focusing on communication. For example, it is possible to identify patients whose social activities and emotional states have not improved after implantation, which may highlight the need for a multidisciplinary team to comprehensively treat patients who may have difficulty reentering the hearing world after years of severe hearing loss and isolation. Here, multiple disciplines (beyond otology, audiology, and speech pathology) may need to be engaged in post-operative rehabilitative care and provide domain-specific support when needed. Taken together, implementation of the staging system provides the opportunity to fundamentally change the way CI outcomes are discussed during pre-CI counseling and monitored with the goal of an increased focus on personalized rehabilitation.

The longitudinal data described in phase III provide preliminary evidence regarding early domain-specific patterns of functional improvement after cochlear implantation. Using these results, we can begin to model post-CI functional improvement and understand the timing and to what degree functional abilities improve. As seen in Figure 3, differences in domain-specific improvement can be used as a guide for potential CI users. For example, substantial changes in entertainment, environment, and listening effort domains were observed by 3 months, but were fairly similar at 6 months. In contrast, improvements in communication continued during each time interval. As additional long-term data are collected we will determine whether CIQOL improvement patterns mirror the early changes and plateaus observed with speech recognition scores^3,51,52 or continue to improve over time as reported for functional outcomes.⁵³ Moreover, we will also be able to develop a greater understanding of the patient and hearing factors associated with CIQOL functional stage improvement.

One notable trend is the decreased likelihood of patients in higher pre-CI CIQOL stages advancing to higher stages after implantation. Moreover, only one of the 11 patients who was in stage III of the communication domain prior to implantation demonstrated an increase in functional stage by 6 months. These results are unlikely to be due to a ceiling effect given that a substantial proportion (21.4%) of CI users in the phase I study were in stage IV or V, which demonstrates that higher stages are achievable. Note also that participants in the phase I study were all >12 months post-CI, which is not the case for the patients in the longitudinal study who were enrolled prior to implantation. Therefore, this cohort has the potential for continued functional improvement over time. However, these findings could have substantial implications for CI counseling if the results are confirmed in an increased sample size and with longer follow-up. Specifically, patients at higher CIQOL stages prior to implantation could receive more guarded counseling about real-world functional benefits after CI.

LIMITATIONS

An additional consideration is that longitudinal data collection took place during the COVID-19 pandemic. This is especially important for the emotional and social domains, which may have been impacted by changes in routine behaviors and increased isolation. In fact, the social domain demonstrated the least change in the current study. This may represent lack of opportunities for socializing and more negative attitudes toward socializing due to the pandemic. The result may also be related to the relatively short follow-up period as several studies have demonstrated increased social abilities after CI.^54–56 Interestingly, the 6 months social domain scores appear to be lower than those observed during the phase I CIQOL development study (Table IV) meaning that more improvement with longer follow up may be seen. This will need to be addressed in future studies after the pandemic recedes.

The greatest limitation of the current study is the inherent measurement constraints of PROMs and specifically the CIQOL. To address this possible limitation, we opted to use strata in developing the CIQOL staging system, which is a conservative approach to determining the number of functional stages that should exist for each domain. The use of strata ensures that the stages represent statistically distinct groupings that consider the instrument’s measurement error. Other methods, such as the use of one-half of the standard deviation, are less precise and likely result in a large number of stages that contain overlapping functional abilities.⁵⁷ In fact, application of this latter method for the CIQOL staging system would result in more than 10 stages for each domain that would contain overlapping item response patterns for all domains (see Fig. 2). Here, patients’ functional abilities would have to improve by multiple stages to represent meaningful gains, which would create interpretation difficulties similar to those for 0–100 scoring systems. Moreover, the associated clinical vignettes would be confusing due to the overlapping abilities associated with these additional stages.

It is possible that the number of functional stages defined for each domain limits the classification of CI user functional abilities, resulting in a reduced ability to monitor progression over time. However, prior psychometric analyses of the full CIQOL item bank (81 items vs. the current 35 items) resulted in only one more strata per domain than the CIQOL-35 Profile.¹⁹ Therefore, completion of the full item bank (an additional 46 items) would not provide a substantial increase in the number of stages but would add considerable burden for patients and clinicians. We believe our methods strike an optimal balance considering the inherent measurement limitations when a continuous variable is converted into categorical variable. Thus, given these potential limitations, we recommend reporting changes in CIQOL domain scores along with the functional stage to provide a more complete understanding of patients’ functional abilities.

One final limitation is related the study population. Although participants were recruited from a consortium of 30 CI centers in the United States to enhance generalizability and were representative of the known demographics of this population, there are many potential disparities regarding CI care that have not been thoroughly studied and remain unknown. As such, it is possible that our sample may misrepresent one of these unknown factors.

SUMMARY AND CONCLUSIONS

This study presents the development and implementation of a novel functional staging system for adult CI users. Using psychometric analyses of data from 705 experienced CI users, the CIQOL staging system stratifies CI users’ abilities into 3–5 stages per domain. The associated clinical vignettes for these stages were confirmed by CI users to be easy to understand and that changes in stages represented meaningful increases in their functional abilities. When implementing the staging system in a pre-CI to 6 months post-CI activation longitudinal study, the majority of patients demonstrated increases in at least one domain stage. The largest early increases were observed in the communication and listening effort domains. In addition, there was a trend for patients at higher pre-CI functional stages demonstrating less improvement over time. Together, these results provide an evidence-based understanding of the changes over time in real-world functional abilities of adult CI users across multiple domains and provide the proportion of CI users in each stage. The functional staging system can be used during discussions of expectations with potential CI users so clinicians can provide evidence-based insight regarding realistic outcomes that are not limited to discussions based on speech recognition scores.⁵⁷ Moreover, the patterns of early changes within each domain provide additional insight for patients regarding the timing of functional improvements after CI activation. Finally, the use of CIQOL functional staging system presents an opportunity to develop goal-based, individualized rehabilitation strategies that target barriers to stage advancement faced by CI users.