Health-Related Quality of Life Instruments for Children with Cochlear Implants: Development of Child and Parent-Proxy Measures

Abstract

Objectives

Severe to profound hearing loss is associated with worse health-related quality of life (HRQoL), reflecting the wide-ranging effects of deafness on spoken language, cognition, and social/behavioral development. However, there are currently no cochlear implant (CI)-specific HRQoL measures that were developed using the FDA Guidance on patient-reported outcomes. This study developed the first HRQoL instruments (CI-QoL) for children with CIs, ages 6 to 12, and a parent-proxy measure for this age group.

Design

Two phases of instrument development were conducted. Phase 1 consisted of a literature review yielding a conceptual framework and discussion guides to elicit information from stakeholder focus groups at CI clinics in Miami and Philadelphia (n=30) (e.g., physicians, speech pathologists). During Phase 2, open-ended interviews were conducted with 21 parent-child dyads (M child age= 9.1 years) recruited from these two clinics. Interviews were transcribed, followed by content analysis in NVivo to identify the most frequent and difficult themes. Items were then derived from these themes to form the initial draft instruments. A multimodal approach was used to create the child-report version (i.e., pictorial representations, audio recording of items, written text above the drawings) to maximize comprehension and ease of responding. Both measures were developed to be administered electronically on a tablet device. In Phase 3, a new set of parent-child dyads (n=20; child age M = 9.2 years) completed a cognitive testing protocol to ensure clarity, ease of use, and comprehensiveness. Cognitive testing led to revisions and finalization of the instruments.

Results

The final self-report measure contained 33 items across eight domains: Noisy Environments, Academic Functioning, Child Acceptance, Oral Communication, Social Functioning, Fatigue, Emotional Functioning, and Device Management. The final parent-proxy measure included 42 items on nine scales: the same eight scales that appear on the child version, with the addition of Behavior Problems. Correlations between child and parent reports on each scale ranged from r=.08–.48.

Conclusions

CI-specific HRQoL instruments have now been developed for school-age children with CIs, with an accompanying parent-proxy version. Following a psychometric validation, these CI-specific measures will enable us to track long-term outcomes, evaluate the efficacy of interventions to improve CI use (e.g., single vs. bilateral implantation, AV therapy, maternal sensitivity training), and provide a profile of the “whole child’s” functioning to facilitate care.

Introduction

Estimates suggest that 5 out of every 1,000 children born in the United States have hearing loss (Boulet et al. 2009). Despite this, there are few standardized measures designed to monitor overall outcomes and provide targets for intervention in pediatric hearing loss. One of the most important domains to assess is health-related quality of life (HRQoL), which provides unique and valuable information about the effects of a disability or medical condition on daily functioning (Palermo et al. 2008; Quittner et al. 2009).

Traditional measures of auditory and communicative abilities in deaf children are essential, but fail to capture the cascading effects of childhood deafness on the individual’s social, emotional, behavioral, and cognitive functioning. Assessing these domains is crucial for deaf children with cochlear implants (CIs), who have deficits in social competence, externalizing behavior, and linguistic skills (Quittner et al. 2007; Barker et al. 2009; Cejas et al. 2014; Hoffman et al. 2015). Furthermore, these deficits are more pronounced in CI children with comorbid conditions, who account for 30–40% of the CI population (Johnson & Wiley 2009; Cruz et al. 2012; Cejas et al. 2015; Inscoe & Bones 2016).

Currently, there are no CI-specific HRQoL measures for children and their parents. Thus, the major purpose of this study was to develop the first CI-specific HRQoL measures for school-age children (6–12 years), accompanied by a parent-proxy version. These measures will facilitate research on the psychosocial outcomes of CIs, the effects of new medical technologies, and the efficacy of audiological, language and behavioral interventions. Furthermore, the Food & Drug Administration now encourages their use as secondary outcomes in clinical trials (FDA 2009).

The FDA Guidance details a specific path for the development of patient-reported or proxy outcome measures. It is an extensive document that draws heavily on classical test theory and specifically recommends beginning with a qualitative process that includes focus groups with all stakeholders and open-ended interviews. Most importantly, it represents the individual’s voice and items are written using transcripts of their actual language. Following identification of key impacts, items are tested for clarity, comprehensiveness and relevance. This is an iterative process, with refinement of the measures at each stage. The final phase of development includes a large-scale psychometric validation. If the measures are developed according to the FDA process, and are context-specific, they can be used as primary or secondary endpoints in clinical trials of new devices, programming strategies, or unilateral versus bilateral implantation. This guidance has been used to develop a number of condition-specific HRQoL instruments that have been used successfully to approve new medications and treatments (Retsch-Bogart et al. 2009). In the area of hearing loss, this may be critically important in obtaining insurance approvals for cochlear implant surgeries and reimbursement of CI processors.

Previous Assessments of HRQoL in Children with CIs

Generic measures of HRQoL, such as the Pediatric Quality of Life Inventory (PedsQL; Varni 2001), contain global items that are relevant to generic samples (e.g., “How often do you feel sad or blue”). The advantage of this approach is that these measures can be completed by healthy children as well as those with chronic conditions. They can also be used to evaluate the cost-effectiveness of interventions (e.g., EuroQoL-5D)(Ware & Sherbourne 1992). In contrast, context-specific HRQoL measures are designed for a specific medical population and include items that assess unique daily experiences (e.g., “How much does it bother you when others ask about your CI?”). Substantial evidence indicates that condition-specific instruments are more sensitive and responsive because they include items that are more relevant and important to patients/parents (Quittner et al. 2009).

Most previous studies assessing HRQoL in children with CIs have used generic measures (e.g., SF-36 or Health Utilities Index) to demonstrate increases in quality of life pre vs. post implantation and compare HRQoL in implanted children with and without comorbidities (Beadle et al. 2000; Cheng et al. 2000; Krabbe et al. 2000; Zaidman-Ziat et al. 2008; Meserole et al. 2014). However, less is known about quality of life in children with CIs vs typically hearing peers (Quittner et al. 2016). The majority of studies addressing this issue have utilized the KINDL (Ravens-Sieberer & Bullinger 1998), a generic measure of quality of life.

Studies using the KINDL have yielded mixed results. Warner-Czyz and colleagues (2009) compared preschool children with CIs (ages 4–7; N=50) to 25 typically hearing peers, and found that both groups reported similar overall HRQoL. Similarly, in an older sample of children with CIs, Loy et al. (2010) compared 86 children (ages 8–16) to the KINDL’s normative sample and found no differences in the total scores. However, among children ages 8–11, those with CIs reported significantly lower scores on the Family subscale compared to the normative sample. In contrast, Huber (2005) compared ratings of children with CIs ages 8–16 (n=44) to the normative data and found that children ages 8–12 rated themselves as having significantly lower overall HRQoL. Analyses of the individual subscales were split based on gender and showed that girls with CIs reported significantly lower scores vs norms on all six subscales, whereas boys only reported significantly lower scores on the Psychological Well-Being and Self-Esteem scales. Conversely, adolescents ages 13–16 reported similar overall scores to the norms, and only the Self-Esteem subscale was significantly lower for adolescent girls.

In sum, the use of the KINDL has produced inconsistent quality of life results in children with CIs. A few of the generic subscales of the KINDL did yield similar scores across children with CIs and typical hearing, however, other subscales demonstrated group difference. One possible explanation for this is that children who completed it varied in age, given that they were at different points in development. Another explanation is the lack of contextual specificity of these items to the challenges experienced by children with hearing loss (e.g., communication skills, listening in noisy environments). The KINDL also does not assess functional status, one of the four core domains of HRQoL (Lin & Niparko 2006; Edwards 2007; Loeffler et al. 2010; Meserole et al. 2014).

Condition-Specific Measures

A comprehensive literature review indicated that only one condition-specific measure exists for school-age children with hearing loss. The Hearing Environments and Reflection on Quality of Life (HEAR-QL-26, for children ages 7–12) was developed using focus groups with adolescents who had mild to profound hearing loss and their parents (Umansky et al. 2011). It contains 26 items and three scales that contribute to the overall score: difficulty hearing in certain environments, impact of hearing loss on social/sports activities, and effects of hearing loss on child’s feelings. The discriminant validity of the HEAR-QL-26 vs the PedsQL was assessed in 80 children with hearing loss and 35 typically hearing siblings. A significant difference between the groups was found on total QoL scores for the HEAR-QL-26, but not the PedsQL, indicating that the HEAR-QL-26 is more sensitive than a generic measure.

However, the HEAR-QL-26 is not appropriate for school-age children with CIs for several reasons. First, the qualitative phase was completed with adolescents and not school-age children, and few participants used CIs. Second, the measure was not developed following the FDA Guidance (2009), which mandates interviews with a variety of health care providers (e.g., speech-language pathologists, surgeons), age-appropriate patients, and parents. Third, it was normed using only eight children with bilateral CIs; none had unilateral CIs and most had mild to moderate hearing loss. Finally, it does not have a parent-proxy version, which captures the broader perspective of caregivers (Varni et al. 2007).

Two other measures using parent-proxy responses were created to assess HRQoL in school-age children with hearing loss: The Parent Views and Experiences with Pediatric Cochlear Implant Questionnaire (PEVCIQ; ages 5–16) and the Children’s Quality of Life Questionnaire (ages 3–20) (Nunes et al. 2005; Schorr et al. 2009). The PEVCIQ utilizes a semi-structured interview containing 74 questions, which requires substantial training and time to administer, making it less feasible for use in clinical settings. The Children’s Quality of Life Questionnaire consists of 13 items that load onto two subscales: Benefits and Problems, which do not measure the core domains of HRQoL and thus, may not be useful in guiding intervention or evaluating child functioning. Furthermore, neither measure is completed by the child.

In conclusion, none of the existing deaf-specific measures are appropriate for children with CIs. The major aim of this study was to develop CI-specific HRQoL measures following the FDA Guidance (2009), for children ages 6–12 years and their parents. These instruments will fill important gaps in the literature and facilitate: 1) evaluation of new devices and programming strategies, 2) assessment of functioning in children with unilateral vs. bilateral CIs, 3) monitoring progress in children with CIs and comorbid disabilities (e.g., autism, epilepsy), and 4) identification of intervention targets (e.g. language, behavior difficulties, auditory skills).

Materials and Methods

The instrument development process consisted of three phases. Phase 1 included a comprehensive literature review, development of a conceptual framework, and completion of focus groups with key stakeholders (e.g., CI surgeons, auditory-verbal therapists, educators). In Phase 2, we conducted qualitative interviews with children with CIs and their parents, using discussion guides developed in Phase 1. Content analysis of these interviews were completed in Phase 3, yielding saturation matrices and item generation for the preliminary instruments (Quality of Life-CI). This was followed by cognitive testing of the draft instruments in a new sample to ensure that the items were clear, comprehensive and easy to rate.

Participants

To ensure demographic and geographical diversity, we recruited stakeholders, children, and parents at two national, pediatric CI centers: The University of Miami Ear Institute (UM) and Children’s Hospital of Philadelphia Cochlear Implant Program (CHOP). We also recruited stakeholders from the Debbie School in Miami, an early intervention preschool for children with hearing loss that is affiliated with the UM CI Center. These efforts facilitated identification of content that was representative and generalizable to the broader CI population. This study and all procedures were approved by the University of Miami Institutional Review Board, protocol #20150127.

Literature Review (Phase 1)

A thorough literature review was conducted to evaluate the effects of childhood deafness on school-age children with cochlear implants. Searches were conducted in the following areas: 1) studies of children with CIs 6–12 years which evaluated their daily functioning, management of their CI, emotional development, and social skills, 2) studies examining the effects of CIs on early childhood development (e.g., joint attention, behavioral regulation, family relationships), 3) outcome studies of HRQoL in children with CIs and childhood deafness, and 4) existing HRQoL measures for this population, including the items and psychometric properties. Results from this review informed the development of the conceptual framework and the discussion guides that were used for the focus groups and open-ended interviews.

Stakeholder Focus Groups (Phase 1)

Stakeholder focus groups included faculty and staff at the UM Ear Institute, Debbie School, and/or CHOP Department of Audiology working directly with children with CIs ages 6–12. Focus group participants were multidisciplinary healthcare providers (n = 30), over half of whom were audiologists and speech and language pathologists (53.33%) (See Table 1 for complete stakeholder demographic data).

Table 1.

Stakeholder Demographics from Focus Group Interviews

Characteristic	Total (N=30) (%)
Gender
Male	5 (16.68%)
Female	25 (83.32%)
Age
21–30	7 (23.24%)
31–40	10 (33.33%)
41–50	6 (20.00%)
51–60	6 (20.00%)
61–70	1 (3.33%)
Race/Ethnicity
Asian	1 (3.33%)
Hispanic/Latino	10 (33.33%)
White	17 (56.66%)
Biracial	2 (6.68%)
Education
High school/GED	3 (10.00%)
Bachelor’s degree	4 (13.33%)
Master's degree	10 (33.34%)
M.D.	4 (13.33%)
Doctorate	9 (30.00%)
Job title
Audiologist	9 (30.00%)
Educator for the Deaf	3 (10.00%)
Physician	4 (13.33%)
Speech/Language Pathologist	7 (23.24%)
Social Worker	4 (13.33%)
Teacher	1 (3.33%)
Other (Student and Senior Administrator)	2 (6.68%)
Institution
The Debbie School	8 (26.67%)
Children’s Hospital of Philadelphia	9 (30.00%)
University of Miami Ear Institute	13 (43.33%)
Experience with CIs (years)
<1	3 (10.00%)
3–5	9 (30.00%)
5–10	7 (23.33%)
10+	11 (36.67%)
Length of current placement (years)	8.40 (SD=7.70)

Child and Parent Open-Ended Interviews (Phase 2)

Inclusion criteria for parent-child dyads were: 1) children with unilateral or bilateral CIs, 2) chronological age between 6 years 0 months – 12 years 11 months, and 3) able to communicate in English and/or sign language. Exclusion criteria included: 1) children with developmental disabilities that severely limited their ability to complete open-ended or cognitive interviews (e.g., severe cerebral palsy or Autism Spectrum Disorder), and 2) children/parents who did not speak English and/or sign language. Flyers about the study were distributed to all families of pediatric CI children in our age range. No one who expressed interest was excluded.

Children who participated in the open-ended interviews (n=21) were, on average, 9.11 years old (SD=1.66) and mostly male (76.19%). The majority of children used bilateral CIs (66.67%), had congenital hearing loss (61.90%), communicated via spoken language (71.44%), and had private insurance (56.10%). In addition, 42.86% of the participants had comorbid medical diagnoses, which is similar to the overall CI population. The most common comorbidities included attention-deficit/hyperactivity disorder (12.19%), auditory neuropathy (9.75%), and additional developmental delays beyond language (7.32%). See Table 2 for additional demographic data on the open-ended and cognitive interview participants. Of the parents who completed the open-ended interviews, most were mothers (76.19%) who had earned a college degree (85.74%). See Table 3 for parent demographic data.

Table 2.

Demographics for Children Enrolled in Open-Ended Interviews and Cognitive Testing

Characteristic	Open-Ended Interviews (N=21) (%)	Cognitive Testing (N=20) (%)
Gender
Male	16 (76.19%)	13 (65.00%)
Female	5 (23.81%)	7 (35.00%)
Age (in years)	9.11 (SD=1.66)	9.17 (SD=1.87)
6–7	8 (38.10%)	6 (30.00%)
8–9	6 (28.57%)	6 (25.00%)
10–11	6 (28.57%)	7 (35.00%)
12	1 (4.76%)	1 (5.00%)
Race/ethnicity
Asian	0 (0.00%)	2 (10.00%)
African American	3 (14.29)	4 (20.00%)
Hispanic/Latino	5 (23.81%)	2 (10.00%)
White	12 (57.14%)	11 (55.00%)
Biracial	1 (4.76%)	0 (0.00%)
Other	0 (0.00%)	1 (5.00%)
Classroom type
Mainstreamed classroom	9 (42.86%)	12 (60.00%)
Mainstreamed with pullout classes	4 (19.05%)	3 (15.00%)
Auditory/oral classroom	5 (23.81%)	4 (20.00%)
Special education classroom	2 (9.52%)	1 (5.00%)
School for the deaf and hard of hearing	1 (4.76%)	0 (0.00%)
Device type
Unilateral	1 (4.76%)	3 (15.00%)
Bimodal	6 (28.57%)	4 (20.00%)
Bilateral	14 (66.67%)	13 (65.00%)
Type of hearing loss*
Congenital	13 (61.90%)	13 (65.00%)
Progressive	9 (42.86%)	5 (25.00%)
Sudden	2 (9.52%)	2 (10.00%)
Onset age (mean in months (SD))	12.50 (23.99)	3.40 (7.16)
Birth	11 (52.38%)	15 (75.00%)
>0	5 (23.81%)	5 (25.00%)
unknown	5 (23.81%)	0 (0.00%)
Etiology
Cytomegalovirus	1 (4.76%)	0 (0.00%)
Genetic	7 (33.33%)	5 (25.00%)
Usher syndrome	1 (4.76%)	0 (0.00%)
Measles	0 (0.00%)	1 (5.00%)
Meningitis	1 (4.76%)	1 (5.00%)
Fetal Alcohol Syndrome	1 (4.76%)	0 (0.00%)
Unknown/Other	10 (47.63%)	13 (65.00%)
Pure tone average in better ear (db HL)	91.9 0 (19.17)	100.25 (13.74)
Additional Comorbidities
No	12 (57.14%)	10 (50.00%)
Yes	9 (42.86%)	10 (50.00%)
Age at implantation (mean in months (SD))	52.05 (32.78)	34.00 (27.39)
Mode of communication
Spoken language	15 (71.44%)	19 (95.00%)
Sign language	1 (4.76%)	1 (5.00%)
Total communication	3 (14.28%)	0 (0.00%)
Sign/speech mixture with speech emphasis	2 (9.52%)	0 (0.00%)
Currently receiving speech/language therapy
No	13 (61.90%)	13 (65.00%)
Yes	8 (38.10%)	7 (35.00%)

Table 3.

Demographics of Parents Enrolled in Open-Ended Interviews and Cognitive Testing

Characteristic	Open-Ended Interviews (N=21) (%)	Cognitive Testing (N=20) (%)
Relationship to child
Mother	16 (76.19%)	18 (90.00%)
Father	3 (14.28%)	2 (10.00%)
Grandparent	2 (9.53%)	0 (0.00%)
Age
20–29	2 (9.53%)	1 (5.00%)
30–39	8 (38.09%)	4 (20.00%)
40–49	7 (33.33%)	15 (75.00%)
50–59	3 (14.28%)	0 (0.00%)
60–69	1 (4.73%)	0 (0.00%)
Race/ethnicity
African American	3 (14.28%)	3 (15.00%)
Asian	0 (0.00%)	1 (5.00%)
Hispanic/Latino	5 (23.83%)	3 (15.00%)
White	10 (47.61%)	13 (65.00%)
Biracial	3 (14.28%)	0 (0.00%)
Education
Completed high school	2 (9.53%)	1 (5.00%)
Some college	1 (4.73%)	2 (10.00%)
Associates degree	0 (0.00%)	1 (5.00%)
Completed college	11 (52.41%)	10 (50.00%)
Graduate school or higher	7 (33.33%)	6 (30.00%)
Income
Less than $15,000	2 (9.53%)	0 (0.00%)
$15,000–29,999	1 (4.73%)	2 (10.00%)
$30,000–49,999	2 (9.53%)	4 (20.00%)
$50,000–74,999	3 (14.28%)	1 (5.00%)
$75,000–$100,000	4 (19.04%)	3 (15.00%)
More than $100,000	6 (28.61%)	9 (45.00%)
Don’t know/declined to answer	3 (14.28%)	1 (5.00%)
Primary language spoken at home
English	19 (90.54%)	17 (85.00%)
Spanish	1 (4.73%)	2 (10.00%)
Other	1 (4.73%)	1 (5.00%)

Child and Parent Cognitive Testing (Phase 3)

Inclusion criteria for parent-child dyads who participated in the cognitive testing (n= 20) were similar to those for the open-ended interviews. Average child age was 9.17 years (SD=1.87) and 65% were male. Most used bilateral cochlear implants (65%), had congenital hearing loss (65%) and communicated via spoken language (95%). Half of these participants had comorbid medical diagnoses. Parents who completed the cognitive interviews were mostly mothers (90%) who had earned a college degree (80%).

Procedures

Phase 1

This phase included a literature review, development of a conceptual model and discussion guides for the focus groups and open-ended interviews. Key concepts related to the effects of hearing loss on daily functioning (e.g., social competence, academic performance) and any prior measures of quality of life were reviewed and integrated into the model. See Figure 1 for the final conceptual framework.

Figure 1.

Note: The conceptual framework was used to guide the open-ended and cognitive interviews, as well as item generation. The framework was iteratively modified throughout the measure development process and includes key concepts related to the effects of hearing loss on daily functioning.

Following creation of this framework, a total of six focus groups (five at UM, one at CHOP) were conducted with a variety of stakeholders (n = 30) at both CI clinics and the Debbie School. These focus groups were led using semi-structured discussion guides, which included an outline of open-ended questions (e.g., tell me how having (a) CI(s) affects your child) and a series of follow-up probes to elicit additional information.

Phase 2

Open-ended, individual interviews were conducted with 21 children with CIs, ages 6–12, and their parents (n=12 at UM, n=9 at CHOP). The interviews followed the discussion guides, as described above. All interviews were audiotaped, transcribed and coded for content using NVivo (Richards, 2005).

Phase 3

Content analyses of the focus groups and interviews yielded preliminary draft instruments that were tested using cognitive, “think aloud” procedures (Schwarz & Sudman 1996; Quittner et al. 2000). These interviews were conducted with a new sample of children and parents (n=20 dyads; 11 at UM, 9 at CHOP) to evaluate whether the items and rating scales were clear, comprehensive and easy to complete (Schwarz & Sudman 1996; Quittner et al. 2000). In a series of probes, lasting approximately 60 minutes, respondents discussed what they thought each item meant and their rating of the item’s relevance and importance. Examples included: “What did you think of when answering this question?” “How did you decide on your rating?” These are well-established techniques for developing patient-reported outcome measures (Quittner et al. 2000; Quittner et al. 2013), which address problems, such as awkward wording, redundancies, and confusing response options.

All cognitive interviews were audio-recorded and transcribed and interviewers took detailed notes. These data were used to modify the initial instruments. A one-week recall period was utilized to ensure that participants rated recent events that were not influenced by poor memory or recall (Quittner et al. 2013). Currently, there is no consensus on the “optimal” recall period, however, a short recall period is recommended to enhance accurate responding (FDA, 2009).

Measures

Demographic & Medical Questionnaire

Stakeholders representing the multidisciplinary team, as well as educators, completed a brief demographic questionnaire assessing age, gender, race/ethnicity, educational background, current position, and experience working with children with CIs. Parents also completed a background questionnaire assessing key demographic and audiological variables, such as parent education, child’s etiology of hearing loss, type of deafness, and school placement. Chart reviews were performed to obtain additional medical, audiological, and speech/language data, such as pure tone average and date of diagnosis/implantation.

Severity and Frequency Rating Scales

All focus group members and parents who participated in the open-ended interviews completed a brief severity and frequency rating scale across domains (e.g., expressive language, behavior, school functioning). Participants rated the impact of deafness and subsequent implantation on a 1–5 Likert rating scale (i.e., with higher ratings indicating worse functioning in that domain). An example of a severity item was: “How difficult is it for you and your child to manage their device/s?” An example of frequency item was: “How often are your child’s peer relationships affected by us of a CI?” See supplemental materials for the complete measure). Results from this questionnaire guided item generation by identifying which areas of development stakeholders and parents felt were most affected by hearing loss/CIs.

QoL-CI

We created a child self-report and parent proxy version of the QoL-CI in Phase 3. The child measure was designed to be administered using a multimodal approach to enhance children’s understanding. Items and response choices were presented in three modalities: written, auditory and pictorial forms. The questionnaires were created using FileMaker Pro version 14, a free software for mobile devices. During the cognitive testing phase, the instruments were administered using an iPad, which provided sound and automatic scoring. Note that all children completed the questionnaire independently, and parents completed their instrument in a separate room.

Data Analysis

Content Analyses of Open-Ended Interviews

Content derived from the focus groups and open-ended interviews were used for item generation. All transcripts were uploaded into NVivo, a software analysis program for qualitative data. To identify common themes in the transcripts and generate initial codebooks for the coding team, 20 transcripts were randomly selected from the child (n=10) and parent interviews (n=10). The first and last authors grouped phrases from the transcripts by theme to create two codebooks—one for children and one for parents. These codebooks were then used to code all 42 open-ended child and parent transcripts. Codes were defined using five steps suggested by Guest and colleagues (2006): 1) a “brief definition” 2) a “full definition” that completely explained the code, 3) a “when to use,” 4) a “when not to use” section (i.e., another code may fit better), and 5) examples. The frequency of these themes were then quantified.

The coding team consisted of four doctoral and post-baccalaureate research assistants. All transcripts were coded in pairs to achieve consensus coding on each segment of text. Coding pairs were rotated to prevent rater bias and drift. Meetings were held throughout the coding process to discuss questions and challenges, and resolve disagreements between coding pairs. Interrater reliability was calculated using Cohen’s kappa. An independent rater who was not part of the coding team coded 14 transcripts (33.3% of the transcripts; 7 child, 7 parent). Level of agreement was calculated using the original ratings from the coding team and the reliability coder. Viera and colleagues suggested a Kappa of .61 or greater for “substantial agreement” (Viera & Garrett 2005).

Severity and Frequency Ratings

Means and standard deviations for severity and frequency ratings were examined separately for stakeholders and parents to determine the domains of functioning most impacted by hearing loss. Furthermore, stakeholder severity ratings were compared across the top four professional specialties (audiologists vs. speech and language pathologists vs. physicians vs. teachers) to examine differences in mean ratings among professions. This was analyzed with a Multivariate Analysis of Variance (MANOVA).

Saturation of Content from Open-Ended Interviews

Saturation of content was calculated by assessing the frequency with which each item was mentioned by children and parents, and the point at which no new content was identified. Although the definition of saturation of content has varied, this study utilized that of Guest and colleagues, who defined saturation as “the point in data collection and analysis when new information produces little or no change to the codebook” (Guest et al. 2006). Saturation of content can also be understood as the point at which adding new participants to the sample will not generate new, meaningful data. NVivo produces saturation grids, which were used to guide the item generation phase.

Development of Draft Measures

Items were generated to reflect the most frequent themes identified in the saturation matrices (occurring across at least 25% of transcripts) and aggregated into scales based on the original conceptual framework. Items were written using the words and phrases in the interviews to improve the clarity, comprehension, and content validity of the measures (Creswell & Miller 2000). Frequent themes were aggregated into scales based on the framework.

Cognitive Testing

Data from responses to the cognitive testing were quantified to examine preliminary floor and ceiling effects and item distributions (Hays & Hayashi 1990; Hays et al. 1993). Responses to each question were grouped together to identify potential problems with the items (e.g., no variability across respondents) or response scales (e.g., floor, ceiling effects). Draft instruments were created using this feedback.

Results

Phase 1: Conceptual Framework and Stakeholder Focus Groups

Eight domains comprised the initial framework: Academic Functioning, Behavioral Regulation, Communication/Auditory Skills, Device Management, Emotional Functioning, Parent/Child Acceptance, Physical Functioning, and Social Functioning.

Based on paired coding of the six stakeholder focus groups, 19 major themes and 72 subthemes were identified. Major themes included: Academic Functioning, Behavior, Benefits of CIs, Clinic Services, Device Management, Device Usage, Diagnosis, Emotional Functioning, Family Relationships, Noisy Environments, Oral Communication, Parent Facilitating, Parental Stress, Parent-Child Acceptance, Physical Functioning, SES, Social Functioning, and Teacher Facilitating Functioning.

Each major theme centered on an area of difficulty. For example, within Academic Functioning, one teacher said, “It’s really tough because you have so many grammatical rules, and you have to learn all of these things. They have a hard time with sounds, just hearing different sounds, producing sounds. So that automatically affects their reading, their fluency.” These results supported the domains identified in the initial conceptual framework and emphasized the important role parents play in their child’s daily functioning.

Severity and Frequency Ratings

Five-point Likert scales evaluating the severity and frequency of these effects were completed by the stakeholders and parents. Based on stakeholders’ ratings, the most impacted domains were: expressive language (M_{severity of impact}=4.07/5 (SD=.78); M_{frequency of impact}=4.23/5 (SD=.68)), receptive language (M_severity=4.27 (SD=.69); M_frequency=4.23 (SD=.68)) and school performance (M_severity =4.07 (SD=.74); M_frequency =3.86 (SD=.73)). Although not as severe, device management was a frequent problem (device management: M_severity =2.32 (SD=.65); M_frequency =4.32 (SD=.85)). Overall, stakeholders rated the effect of deafness and cochlear implantation across all 10 domains as substantial, giving a mean severity rating of 3.23 out of 5 (SD=.73) and mean frequency of 3.15 (SD=.71).

Analyses comparing differences in mean severity ratings across stakeholders revealed a significant difference in receptive language based on profession with a medium effect size, F(3,22) =5.66, p<.01; partial η²=.46. In addition, differences in expressive language (F (3,22) =2.58, p=.08; partial η²=.25) approached significance. For receptive language, post-hoc analyses showed that physicians and educators rated receptive language impairments higher than audiologists (M_physician=4.75 (SD=.50), M_audiologist=3.88 (SD=.60); t(11)=2.49, p=.03; Cohen’s d=1.57; M_teacher=4.85 (SD=.38) t(12)=3.71, p<.01; Cohen’s d=1.92). These findings should be interpreted with caution because of the small sample size.

Similar to the stakeholders, the parents rated expressive language (M_severity=2.62/5 (SD=1.28); M_frequency= 2.43/5 (SD=1.43)), receptive language (M_severity=2.95 (SD=0.92); M_frequency=2.62 (SD=1.07)), and school performance (M_severity=2.71 (SD=1.23); M_frequency=2.76 (SD=1.41)) as the domains most affected by hearing loss/CIs. In contrast, device management (M_severity=1.19 (SD=0.40); M_frequency=0.80) and self-esteem (M_severity=1.57 (SD=0.68); M_frequency=1.47 (SD=0.60)) received the lowest severity and frequency ratings. When compared to the stakeholder ratings, parents rated the impact of deafness lower across all domains; the mean severity score was 2.05 out of 5 (SD=0.56) and the mean frequency score was 1.95 (SD=0.46).

Phase 2

Child Qualitative Interviews

All 21 child transcripts were coded using the codebooks described above. Based on weekly coding team meetings, the final child codebook contained 16 major themes and 34 subthemes. Kappa estimates of reliability were very strong, ranging from .83–.99, with an average kappa of .95 across all child transcripts. Item generation was primarily based on eight themes that were most central to HRQoL: Academic Functioning, Child Acceptance, Device Management, Device Usage, Emotional Functioning, Hearing in Different Environments, Oral Communication, and Social Interactions. See Table 4 for a listing of major themes and sample quotes used to generate the items.

Table 4.

Sample Quotations from Child and Parent Open-Ended Interviews

Topic	Quote
	Child	Parent-Proxy
Noisy Environments	8-year-old boy: “There’s a lot of people and the teacher mutes the transmitter before you go to the cafeteria and I can hear my friends okay but sometimes just the cafeteria that’s really loud.”	Father of 12-year-old boy: “One of the things that I notice, I play music, but he was never interested in music really…he used to complain about the drum being loud.”
Academic Functioning	7-year-old boy: “Reading takes forever”; 10-year-old girl: “My teacher is good because she understands why we are deaf and when I tell her I can’t understand her, and she speak it again, she don’t make the attitudes.”	Mother of 7-year-old girl: “Sometimes she gets stuck, I’ll be like write your own sentence, and she’s like, ‘I don’t know, help me.’…Little things, but everything does take a lot longer.”
Child Acceptance	8-year-old boy: “I’m just completely normal, but I have CIs.”	Mother of 9-year-old boy: “I wasn’t sure if I was ready for that. He said, ‘I want red!’ and I was like, oh God of all the colors. But he did it, I let him do it, and I got over it, because he wanted it.”
Oral Communication	10-year-old girl: “When I’m speaking low – they’ll tell me, ‘What are you saying?’ and ‘huh’?’ So, I have to speak very loud and they understand me.”	Mother of a 9-year-old boy: If the kids are sarcastic, he doesn’t get it, because he doesn’t get that sarcasm. He doesn’t hear it.
Social Functioning	6-year-old girl: “They only make fun of me if they see me with a short haircut [because they can see the device].”	Mother of 8-year-old boy: “We were at Disney World and there was this little girl and she was so super shy and she was just looking at him with this big huge smile. She had an implant too and he was like,’Mom look she has ears like me’…He gets really excited when he sees other kids with ears like him.”
Fatigue	12-year-old boy: “I get up and go to eat, come back and get ready and put {my CI] on…which is about an hour…because when I start to hear it gets loud, it’s loud at first.”	Mother of 8-year-old boy; “I find that when…he comes home he is sometimes exhausted because he has had to give 110% towards trying to focus.”
Emotional Functioning	7-year-old boy: “Sometimes it really makes me upset [when someone asks about my CIs].”	Mother of 8-year-old boy: “he gets mad because he’ll be like “I can’t hear!”…because you know kids are kids and are going to be yelling and screaming and he’s like “what did you say?”
Device Management	9-year-old girl: “But sometimes I wear it for a long time and then [the magnet site] just hurts.”	Mother of 10-year-old-boy: “[sweat damage] is frequent during the summer, I have to have the dryer on all the time, if not it gets damaged from the sweating and I did get the sweat guards already, you know.”
Behavior	N/A	Mother of 8-year-old boy: “I find that when…he comes home [from school he is] sometimes exhausted because he has had to give 110% towards trying to focus”.

Parent Qualitative Interviews

After coding the 21 parent transcripts, the finalized version of the codebook contained 16 major themes and 49 subthemes. Kappa reliability for individual transcripts was strong, ranging from .77–.95, with a mean kappa of .86 across parent transcripts. Item generation for the parent-proxy measure was primarily based on 9 major themes: Academic Functioning, Behavior, Device Management, Device Usage, Emotional Functioning, Hearing in Different Environments, Oral Communication, Parent-Child Acceptance, and Social Interactions.

There was significant overlap in the themes identified across the child and parent transcripts in the following domains: Academic Functioning, Behavior, Benefits of CI(s), Child Acceptance, Cost/Fears Over Losing the Device, Device Management, Device Usage, Emotional Functioning, Family Interactions, Hearing in Different Environments, Oral Communication, Physical Functioning, Relationship with CI Team, and Social Interactions. In contrast, themes related to Child Expectations and Parent Facilitating were unique to the child transcripts, and unique parent interviews included: Comorbidities, Diagnosis, Parental Behaviors, and School Type. The high degree of content overlap indicated that children and parents have similar perceptions of how deafness and CIs affect HRQoL.

Phase 3: Child Saturation of Content, Item Generation, and Cognitive Testing

Saturation matrices indicated that 29 themes occurred at a frequency greater than 25% and saturation was achieved before the last interview, ranging from the 3^rd to 6^th based on the scale, suggesting that no new, significant content emerged beyond that interview. See Table 5 for the final saturation matrix. The initial version of the child measure (version 1.0) contained 32 items on eight scales: Academic Functioning (5 items), Child Acceptance (4), Device Management (3), Emotional Functioning (5), Fatigue (3), Noisy Environments (5), Oral Communication (3), and Social Functioning (4).

Table 5.

Saturation Matrix From Open-Ended Interviews for Children with Cochlear Implants and Their Parents

	Child Participants
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	Total
Social Functioning
Positive relationships	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	1	1	1	20
Self-advocacy/independence	0	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	1	1	1	19
Curiosity/reactions to staring	0	1	1	1	1	1	1	1	0	1	1	0	1	1	1	1	1	0	1	1	1	17
Swimming	1	1	0	1	1	1	1	1	0	0	1	0	1	1	1	1	1	0	1	1	0	15
Bullying/teasing	0	1	1	0	1	0	1	1	1	0	1	1	1	1	0	0	0	1	0	1	0	12
Feeling/looking different	0	0	1	0	1	0	0	1	1	0	1	1	0	0	1	0	1	1	0	1	1	11
Social isolation	0	0	0	0	1	0	0	1	0	1	1	0	0	1	1	1	1	0	0	1	1	10
Parent Participants
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	Total
Behavior
Attention and distractibility	1	0	1	0	1	0	1	1	1	1	1	0	1	0	0	0	0	1	1	1	1	13
Fatigue	0	0	0	0	0	0	1	0	0	0	0	0	1	1	1	1	1	1	1	1	1	10
Externalizing behavior problems	1	1	0	0	0	0	0	1	1	0	0	1	0	0	0	0	1	1	0	0	1	8
Behavioral regulation	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1

Note: Participant endorsement of row item indicated by "1." Bolded "1" indicates the first endorsement of item by any participant. Saturation was reached by end of the fifth interview for social functioning and by the end of the 7th interview for behavior.

Several steps were taken to ensure that the questions were clear, comprehensive and easy to respond to; this was critically important given the common language delays in this population. First, items were written using common phrases and words children used in their interviews. This enhanced face validity and ensured that the language level was appropriate. Second, the electronic questionnaire presented the questions in a multimodal format, which included an auditory reading of the item, appearance of the written item across the top of the screen, and a cartoon picture depicting the scenario and several response options (See Figure 2 for a sample item). Thus, the child could use multiple contextual clues to understand both the question and response options. Third, the instrument began with an “instructions” screen and two practice items. Practice questions enabled the person administering the questionnaire to assess the child’s understanding of the task based on his/her ability to independently answer the items correctly. Finally, response options were kept consistent (e.g., “how hard” was not alternated with “how easy”) and questions in the same scale were presented in a block, rather than randomized across the measure, to minimize cognitive effort. Children were also reminded about the recall period (i.e., “Thinking about the past week…”) and were prompted to respond if an item was skipped.

Figure 2.

Note: Image from the self-report version of the QOL-CI, illustrating the question “How hard is it for you to hear others in the lunchroom?” Respondents can listen to the audio recording, read the written text, and/or look at the pictures to understand the content of the item.

Cognitive Testing

Cognitive interviews were reviewed frequently in an iterative process to revise the questionnaire based on respondent feedback. Modifications included shortening the instructions, collapsing similar items, rewording items to better reflect children’s vocabulary (i.e., changing “FM system” to “mic”), deleting items that lacked variability, and adding items based on respondents’ suggestions (i.e., “How often does your CI feel uncomfortable?”). The final version of the measure contained 33 items. In general, children reported they enjoyed the pictures, found the iPad easy to use, and could complete the measure in about 8 minutes, with no fatigue.

Initial Quantitative Data

Initial psychometric properties, such as means, standard deviations, and ranges for each item and scale were reviewed (See Table 6). Overall, the means for the scales ranged from 72.22–81.50 and standard deviations ranged from 12.77–25.74. Scaled scores ranged from 0–100, which suggested that the full scale was being used. Item-level means ranged from 2.30–3.79, and standard deviations ranged from 0.44–1.22; all response options were used on 20 of 33 items, indicating good variability in responses across children (Taylor 2013).

Table 6.

Self-Report and Parent-Proxy Questionnaire Results by Scale

	Self-Report			Parent Report
	Mean	SD	Range	Mean	SD	Range
Noisy Environments	75.61	20.31	33.33–100	68.59	14.74	33.33–95.83
Academic Functioning	77.67	18.20	41.67–100	76.75	19.53	41.67–100
Oral Communication	77.78	25.74	0.00–100	66.67	20.17	13.33–93.33
Child Acceptance	81.50	17.40	50.00–100	70.25	14.99	33.33–86.67
Fatigue	75.42	17.27	22.22–100	75.00	16.00	33.33–100
Social Functioning	79.00	17.09	40.00–100	78.42	17.83	25.00–100
Emotional Functioning	78.33	12.77	46.67–100	79.17	13.38	58.33– 100
Behavior Problems	N/A	N/A	N/A	74.00	25.12	26.67–100
Device Management	72.22	17.84	33.33–100	73.78	15.77	33.33–100

Note: Scores on each scale ranged from 0–100, with higher scores indicating higher health-related quality of life.

Phase 3: Parent Saturation of Content, Item Generation, and Cognitive Testing

Thirty themes were coded across at least 25% of the transcripts and saturation was achieved after the 4^th to 7^th transcript, based on scale (see Table 5). Following the process described above, saturation matrices were reviewed to generate specific items for the preliminary, draft instrument. The initial parent-proxy questionnaire (version 1.0) contained 47 items on nine scales: Academic Functioning (5 items), Behavioral Problems (5) Child Acceptance (5), Device Management (6), Emotional Functioning (4), Fatigue (4), Noisy Environments (8), Oral Communication (5), and Social Functioning (5). Items were modified to improve clarity and/or increase variability based on the cognitive interviews.

Initial psychometric analyses including means, standard deviations, and ranges were examined. Means ranged from 66.67–79.17, standard deviations ranged from 13.38–25.12, and scaled scores ranged from 13.33–100, indicating good variability in parent responses (See Table 6). The full range of rating options (1–4) were used for 26 of 42 questions.

Exploratory Aims

Prior research has consistently shown that age at implantation is a significant predictor of speech perception, spoken language, visual attention, and child behavior problems (Quittner et al. 2007; Barker et al. 2009; Niparko et al. 2009; Hoffman et al. 2018). Thus, we examined this variable in relation to HRQoL scores. We hypothesized that age at implantation would significantly impact context-specific HRQoL scores; earlier age was expected to correlate with better HRQoL. No significant associations were found for the child measure, however, significant, negative associations were found between age at implantation and parent-proxy scales: Noisy Environments (r=−.60), Oral Communication (r=−.53), Social Functioning (r=−.52), and Parenting Stress (r=.−52). Thus, children implanted at an older age had worse quality of life in these domains as reported by parents.

Correlations were also computed between similar scales on the child and parent-proxy measures. These paired correlations ranged from .08–.48 with a statistically significant association between respondents on the Noisy Environments scale (r = .48, p<.05). Although not statistically significant, we found substantial associations on other scales that indicate possible trends which would require a larger sample to reach statistical significance: Device Management r=.40, Academic Functioning r=.39, and Emotional Functioning r=.33. Paired samples t-tests indicated that children rated the Child Acceptance scale significantly higher than parents, suggesting greater acceptance of the CI (t (19) = 2.32, p < .05). These associations between child and parent proxy provide initial evidence of convergent validity and are consistent with other studies comparing child and parent ratings of HRQoL in pediatric populations (Warner-Czyz et al 2009).

Discussion

To date, HRQoL measures have only been developed for children with varying degrees of hearing loss and are agnostic with respect to device type (HEAR-QL - Umansky et al. 2011: YQOL-DHOH – Patrick et al. 2011). The purpose of this study was to develop the first CI-specific HRQoL measures for children ages 6–12 and with accompanying parent-proxy. We utilized the instrument development phases recommended by the FDA’s Guidance on patient-reported outcomes (2009). Using participants from two CI clinics, we conducted multidisciplinary expert focus groups and open-ended interviews with children with CIs and their parents, followed by item generation and cognitive testing.

A critical component of the instrument development process was the inclusion of patient and parent input at each phase. Furthermore, to help ensure generalizability and validity, we included children with comorbid medical conditions. Notably, children and parents reported on similar issues, including difficulties hearing in noisy environments, fatigue, and academic struggles. These QoL-CI instruments are important because they provide unique and valuable information about the effects of hearing loss and cochlear implantation on an individual’s daily functioning. Furthermore, they expand on traditional measures of auditory and communicative abilities by assessing the cascading effects of childhood deafness on social, emotional, behavioral, and cognitive development. These measures can be used to assess the “whole child” rather than focusing solely on auditory and linguistic performance (Cejas & Quittner in press). In addition, they can be used to evaluate the effects of CIs on daily functioning from the child’s perspective, which facilitates a patient-centered, collaborative model of care.

The rich data generated from the focus groups, interviews and cognitive testing provided insights into the daily functioning of children with CIs. These data clearly showed that the critical content relevant to quality of life are condition-specific (e.g., device management, disclosing the reasons for using a CI). Thus, generic measures that focus on physical, social and emotional functioning at a general level will not be as sensitive or prescriptive for children with CIs.

In-depth analyses of the specific contexts in which quality of life is affected are key to targeting interventions most effectively. These types of data also provide insights into the downstream consequences of these negative effects. For example, the most frequently reported noisy environments in which children struggled were the playground, school cafeteria, and restaurants. Difficulties in these settings affect a child’s ability to engage with peers and family members, which could lead to social isolation and/or behavior problems stemming from frustration. Interestingly, parents “in tune” with their child’s difficulty hearing in a noisy restaurant had generated crafty ways to incorporate their child into the family conversation (e.g., seating the child in the middle of the family, facing the child away from the noisy part of the restaurant). In addition, many children reported feeling fatigued at the end of the school day, which is consistent with previous studies assessing fatigue in school-age children with hearing loss (Bess & Hornsby 2012; Hornsby et al. 2014). It is possible that listening for these children is more effortful, leading to exhaustion and poor attention at school, which in turn can affect academic performance. Children had incorporated small breaks into their time after school or brief naps before they started their homework.

Despite considerable convergence between the self-report and parent-proxy versions, parents often noted more severe difficulties than their children across several domains, such as behavior problems and withdrawal during social situations with peers. Convergence between parent-child dyads was low to moderate across scales, which likely reflects the different perspectives children and parents often have and the fact that different items fell on scales with similar names (Hoffman et al. 2015). Studies comparing child and parent ratings of HRQoL in cystic fibrosis, diabetes, and other pediatric populations have also found low concordance rates (Eiser & Morse 2001; Modi & Quittner 2003; Upton et al. 2008; Kalyva et al. 2011). These discrepancies do not represent measurement error, but instead arise from each informant’s unique perspective and the attributions they make about their experiences (De Los Reyes & Kazdin 2005). These discrepancies also highlight the importance of assessing quality of life from multiple perspectives, something previous measures have failed to do. However, once these measures are validated, it will be important to assess parent-child agreement and its relationship to audiological and linguistic skills.

Limitations and Strengths

Although we made efforts to enroll children using different modes of communication (oral, total communication, or sign), half of our sample used primarily spoken language and were in mainstreamed classrooms. In addition, we collected data at two pediatric CI programs which included a multidisciplinary team and highly specialized services for this population. Thus, these centers may not be representative of all cochlear implant programs.

This study had several strengths. First, the measures were developed following the FDA guidance on patient-reported outcomes (2009), enabling their inclusion as secondary outcomes in clinical trials. The QoL-CI measures could be used to test the effectiveness of new CI processors, software, and accessories. Second, the instruments were created using diverse samples (e.g., Hispanic, Asian, and African-American participants, varied socioeconomic status/parental education), which increased the representativeness of our sample and the generalizability of the findings. Third, the open-ended and cognitive interview samples included a large proportion of children with comorbid medical diagnoses (46.34%), which increases its relevance to the current, national sample of children using CIs (30–40%; Johnson & Wiley 2009). Finally, the measures were designed using pictures and audio-recordings to facilitate children’s comprehension of the items and response options. This multimodal format, designed for implementation on tablets made it both easy and fun for children to complete. Both child and parent QOL-CI versions are administered electronically, can be completed in 10 minutes, and score automatically. This minimizes burden on clinicians and increases feasibility.

Future Directions and Clinical Implications

The next step in this measurement process is to conduct a psychometric validation with multiple centers and a larger sample. In addition, we plan to translate these measures into multiple languages, including Spanish. Finally, the instruments will be disseminated to CI clinics across the United States for free.

These measures can also be utilized in future research. Given the heterogeneity of the CI population and the differing causes of deafness, these measures can be used to compare HRQoL across etiologies (i.e., genetic loss vs. sudden onset), compare CI outcomes across centers, and examine the benefits of simultaneous vs. sequential bilateral implantation. In addition, studies could examine changes in HRQoL over time, predictors of better quality of life (e.g., age at implantation, socioeconomic status) and the effects of medical comorbidities on daily functioning.

The QOL-CI has the potential to be used clinically. CI centers could administer these tools annually to identify those at-risk, target interventions and track patient outcomes. Notably, even children with good oral skills reported struggling with school and other social challenges. These tools could also be used to generate a profile of the strengths and weaknesses of the “whole child.” This is crucial because often children are doing very well in some areas, but need additional support in others (e.g., assistive technology, CI accessories/active wear). In conclusion, the QoL-CI measures are the first condition-specific instruments for children with CIs that evaluate several domains of functioning from both the child and parent perspective. Both the development and utilization of these measures maps onto innovative strategies for patient-centered, collaborative care.