Development of a New Patient-reported Outcome Measure for Ear Conditions: The EAR-Q

Abstract

Background:

Patient-reported outcome measures are widely used to improve health services and patient outcomes. The aim of our study was to describe the development of 2 ear-specific scales designed to measure outcomes important to children and young adults with ear conditions, such as microtia and prominent ears.

Methods:

We used an interpretive description qualitative approach. Semi-structured qualitative and cognitive interviews were performed with participants with any type of ear condition recruited from plastic surgery clinics in Canada, Australia, United States, and United Kingdom. Participants were interviewed to elicit new concepts. Interviews were audio-recorded, transcribed, and coded using the constant comparison approach. Experts in ear reconstruction were invited to provide input via an online Research Electronic Data Capture survey.

Results:

Participants included 25 patients aged 8–21 years with prominent ears (n = 9), microtia (n = 14), or another condition that affected ear appearance (n = 2). Analysis of participant qualitative data, followed by cognitive interviews and expert input, led to the development and refinement of an 18-item ear appearance scale (eg, size, shape, look up close, look in photographs) and a 12-item adverse effects scale (eg, itchy, painful, numb).

Conclusions:

The EAR-Q in currently being field-tested internationally. Once finalized, we anticipate the EAR-Q will be used in clinical practice and research to understand the patient’s perspective of outcomes following ear surgery.

INTRODUCTION

Children and young adults with ear conditions, such as microtia or prominent ears, report teasing and bullying and that they feel self-conscious and/or embarrassed about the appearance of their ears.^1,2 Ear reconstruction to create a normal-looking ear for patients with microtia can be complex and expensive, with different approaches taken to achieve the same goal.^3–7 As surgical techniques continue to evolve, a carefully developed, valid, and reliable patient-reported outcome (PRO) measure is needed.

In plastic surgery, measurement of the patient’s perspective has been hampered by the lack of validated PRO instruments. Systematic reviews of pediatric PRO instruments for conditions associated with a facial difference identified only 1 ear-specific PRO measure,^1,8,9 that is, the Congenital Aural Atresia Questionnaire, which measures hearing and psychosocial concerns.¹⁰ Given the lack of ear-specific PRO instruments, researchers have used generic measures.^11–15 Although generic instruments measure psychosocial concerns, which are important to people with ear anomalies, the use of such tools alone misses the opportunity to measure the key outcome targeted by surgery, that is, satisfaction with ear appearance. Recently, a PRO measure for microtia was published.^16,17 This new scale measures ear appearance alongside satisfaction with care and psychosocial behavior, though details about its development and psychometric performance are absent.

To address the lack of PRO measures for children and young adults with conditions associated with a facial difference, our team commenced research to develop a comprehensive set of scales. We began with the CLEFT-Q for cleft lip and/or palate, which contains 12 scales and a checklist.^18–20 The CLEFT-Q was developed from qualitative data collected from 138 patients from 138 patient with cleft lip and/or palate from 6 countries.¹⁹ The subsequent CLEFT-Q field-test study included 2,434 patients aged 8–29 years from 12 countries.²⁰ To address other craniofacial conditions, including ear anomalies, we conducted interviews with 84 patients aged 8–29 years and developed an additional 16 scales of which 2 are specific to ears. Elsewhere, we described the full set of 26 independently functioning scales, collectively called the FACE-Q Module for Children and Young Adults.²¹

Since the description of the development of the 26 FACE-Q scales are not presented in-depth due to publication space limitations,²¹ the aim of this article was to provide an in-depth account of the development of the 2 ear-specific scales.

METHODS

Our team followed best practice guidelines for PRO measure development.^22–25 Figure 1 shows our multi-phased iterative approach.¹⁸ Elsewhere, we report findings from our systematic review of pediatric PRO instruments developed for conditions associated with a facial difference.⁸

Fig. 1.

Mixed method iterative approach to developing a patient-reported outcome measure.

Qualitative Interviews

For the qualitative phase, we used an applied health services research approach called interpretive description.²⁶ Before patient interviews, we obtained research ethics approval from the Hamilton Integrated Research Ethics Board (Canada), The Hospital for Sick Children Research Ethics Board (Canada), The University of Colorado (United States), The Children’s Hospital at Westmead (Australia), and local and National Health Services permission for Great Ormond Street Hospital National Health Service Foundation Trust (United Kingdom). Written informed assent and/or consent was obtained from participants and guardians.

We conducted qualitative interviews to elicit concepts from patients and to create a comprehensive item pool for scale development. Eligible participants were aged 8–29 years and fluent in English. We included both pre- and post-operative patients for any type of ear condition. Recruitment took place at William Osler Health System (Etobicoke, Canada), Hospital for Sick Children (Toronto, Canada), and Great Ormond Street Hospital (London, United Kingdom). Participants were invited to take part in the study by a member of the health care team. Based on participants’ preference, interviews were conducted in the home or hospital or, for participants aged 12 years and older, by phone. The qualitative phase took place between May 2014 and August 2016. An interview guide, described elsewhere, was used to direct the interviews.²¹ Interviews were recorded digitally, transcribed verbatim, and coded using a line-by-line coding approach. Data collection and analysis occurred concurrently to build on the knowledge gained from each interview.

Quotes (participant words/phrases) that pertained to any aspect of outcome were moved into Excel along with specific participant characteristics (ie, age, sex, and ear condition) and categorized into conceptual top-level domains, categorized into conceptual top-level domains, themes and subthemes.²⁷ For example, a 14-year-old girl with microtia said the following: “If you look at it [ear], if you stare at it, you can notice there’s something wrong with it. But if you’re just walking past on the street normally, then there’s nothing.” As the participant was describing how her ear looked from different distances, this quotation was assigned the top-level domain “appearance,” the major theme “scenario” and the minor theme “up close/far away”. Table 1 shows additional examples of how the data were categorized. Coding was performed by 1 member of the research team and confirmed by a second member. Interviews continued until no new concepts were elicited from additional interviews.²⁸

Table 1.

Examples for How the Qualitative Data Were Categorized into Domains and Themes

Scale Formation

We developed a comprehensive item pool from participant coded quotes. We retained participant-specific wording such that items would be easy to understand and resonate with patients. The item pool was used to form scales in line with a modern psychometric approach called Rasch Measurement Theory.²⁹ In this approach, items on a scale map out a concept of interest in terms of a clinical hierarchy (ie, from a small amount of the concept being measured to a large amount). For each scale, 4 labeled response options were developed and instructions were included to orient the participant to the task required.

Cognitive Interviews

Cognitive interviews were used to refine the scales. We included participants from the initial qualitative interviews and added new participants recruited from the United States and Australia. We used the “think aloud” technique (verbalize thought processes), and probing to identify whether participants understood the instructions, response options, and items.^30,31 For any content that seemed difficult, participants were encouraged to suggest ways to simplify the wording. The interviewer also probed for any missing content. Interviews were recorded digitally, transcribed verbatim, and coded in Excel. Cognitive interviews were conducted in rounds to continually revise the scales based on feedback.

Expert Review

Before the final round of cognitive interviews, a secure web-based Research Electronic Data Capture³² survey was administered to ear reconstruction experts. In July 2016, we invited 36 experts within our team’s professional networks to participate. Experts were asked to provide feedback on all aspects of the scales and to nominate missing content. One reminder e-mail was sent a week later. Findings were used to revise the scales.

RESULTS

Qualitative Interviews

Table 2 shows the sample characteristics. The sample included 13 females and 12 males aged 8–21 years. Most participants had either prominent ears or microtia. Half of the microtia participants also had a craniofacial syndrome. Data analysis of the item pool led to the identification of 3 top-level domains as follows: appearance concerns (n = 199, 29%); physical concerns (n = 61, 9%); and health-related quality of life concerns (n = 434, 63%). These domains were important to both participants with microtia and with prominent ears. Each domain had multiple themes and subthemes, which are described below.

Table 2.

Patient Characteristics

Appearance Concerns

The 199 items in the item pool generated from quotations in this domain covered 12 main themes (Fig. 2). Most items were generated from codes that were negative (n = 156, 78%); positive items mainly related to how the ear(s) looked after surgery. Participants described the appearance of their ears in terms of their position (eg, stuck out, flat, or close against the head), size (eg, too big or small), qualitatively (eg, looked good, nice, pretty, ugly, weird, different), shape (eg, circle, rectangular, folded, oval, pointy), symmetry (eg, not the same), contour (eg, smooth, bumpy), how the ears looked in scenarios (eg, up close, in mirrors, with hat on), and how normal/abnormal the ears looked. The rich descriptive details and language used by participants to describe the appearance of their ears were used to develop a preliminary 22-item scale hypothesized to cover the concerns of patients with microtia and prominent ears. Instructions were added that asked participants to answer based on how their ears looked now. If their ears looked different from each other (ie, don’t match), they were instructed to answer each item thinking about the ear that they were least happy about. The items asked about ears in terms of size, overall shape, how well they match each other, how specific parts look (eg, top part, earlobes), how close they are to the head, and how they look in different scenarios (eg, with wet hair, wearing a hat, glasses or sunglasses, in photographs, a mirror and profile view, from a distance and up close). For each item, instructions asked “How much do you like…” with 4 response options provided: not at all; a little bit; quite a bit; and very much.

Fig. 2.

Number of items for each major appearance theme by ear condition. Note: Excluded 2 codes for “other ear conditions” to maintain anonymity due to small sample.

Physical Concerns

The 61 physical items in the item pool generated from quotations in this domain covered 2 main themes: hearing (n = 14); and adverse effects (n = 43), see Figure 3. Adverse effects covered a range of ear-specific issues related to how the ears felt to the touch (eg, hard, bumpy, squishy), and symptoms (eg, sensitive, swelling, itchy, numb, pain). The item pool was used to develop a preliminary 13-item scale, which we hypothesized would cover adverse effects following any type of ear surgery. Instructions were added that asked participants to answer based on the past week. If their ears felt different from each other, participants were asked to answer each item thinking of the ear that bothers them the most. Adverse effects included the following: itchy, tingling, pain, numbness, discoloration, activity limitations, and sleep interference. For this scale, 4 response options measured frequency: never; sometimes; often; and always.

Fig. 3.

Number of items for each major physical theme by ear condition. Note: Excluded 2 codes for “other ear conditions” to maintain anonymity due to small sample.

HR-QOL Concerns

The 434 HR-QOL items in the item pool generated from quotations in this domain covered psychological (n = 201), social (n = 173), and school-specific (n = 60) concerns. For the psychological items, most were negative in focus (n = 135, 67%). The most common themes included using concealment behaviors to hide the ears by wearing a hat or with long hair (“I would just, leave it down and try to keep it over my ears”), feeling self-conscious of the ears (“I would actually hesitate to answer a question because I wouldn’t want people to look at me”), confidence (“Like taking pictures, I can feel confident where I used not to be very confident taking pictures”), and feeling normal (“In the hospital I feel normal. Like at school I feel normal for the most part, but if I were to go to a club I would not feel normal”). In terms of social function, most items were positive in focus (101, 58%). The most common themes included obtaining instrumental support (“[My] parents have always been, like, taking me to hospital and always being there.”) and emotional support (“My friends have always supported me and they helped me.”) from family and friends. Participants also mentioned negative themes such as social isolation due to being teased, feeling judged, or being stared at by people (“No one pointed at it, but they used to stare at me”). Some participants mentioned that their ear condition affected their ability to participate in certain sporting activities (“Because of my surgeries, for 2 years—last year and this year—I missed out on high jump in track and field”). The school codes were mainly negative in focus (45, 75%). The most common theme was missing school due to the ear condition (“The part where it’s bad is because I miss out on school”). Development and refinement of scales that can be used for patients with ear conditions to measure appearance-related distress and psychological, social, and school function are described elsewhere.²⁶

Cognitive Interviews

Table 2 shows the participant characteristics. The ear-specific scales were shown to 10 females and 7 males aged 8–21 years. Nine of these participants also took part in the initial qualitative sample. For ear appearance, based on feedback from 11 participants over 3 rounds of interviews, 2 items were added, 5 items were revised, and 6 items were dropped. An example of a suggestion for adding an item was as follows: “Well, from the back it’s kind of weird, because you can still see kind of the bumps of the stitches.” We added the following items to capture this concern: “…how your ears look from behind (eg, a photograph that shows the back of your head)?”

For the adverse effects scale, over the course of 3 rounds of interviews, 1 item was added, 8 items were revised, and 2 items were dropped. The least understood item based on feedback was “My ears tingle”. For example, 1 participant clearly misunderstood the item: “It would probably be like if they were ringing, so if there was some really loud siren outside or some really loud noise coming somewhere.” Several participants suggested that tingle was like “pins and needles,” which we used to refine the item to become: “My ears feel tingly (pins and needles feeling).” In addition to this scale, we added 2 items for postoperative patients to comment on how their surgical scars look and feel (ie, smooth or bumpy).

Following the third rounds of interviews, 6 patients from the United States (n = 4) and Australia (n = 2) completed cognitive interviews to confirm that the scales worked in different countries. No further changes were needed based on feedback from these cognitive interviews. The response options were deemed acceptable by all patients.

Expert Input

Feedback on the revised scales was obtained from 13 clinical experts from 6 countries (Australia, Canada, England, Netherlands, Scotland, and United States). Experts included 12 surgeons and 1 clinical psychologist. Almost all (n = 12) experts reported that ear reconstruction was an important focus of their clinical practice. Experts provided detailed feedback that was helpful to refining items. For example, an item in the appearance scale asked “…how your ears look if you put on a hat?” Two experts raised the issue that they thought the item was not specific enough. One said: “… Not sure about the ‘hat’ question—does it need to be clearer whether the hat will cover the ears, for example, a beanie, as they may feel different about a hat like that compared with a cap that reveals part of the ear.” The feedback was useful in revising the item to read: “…how your ears look if you put on a hat that shows your ears (eg, a baseball cap)?”

DISCUSSION

There are many surgical approaches to address different congenital and acquired ear conditions. Though autologous (costochondral grafts) ear reconstruction is the gold standard for microtia other options such as porous polyethylene implants⁴ and prosthetics⁵ are also used. Advancements in 3D printing³³ and ongoing research into tissue engineering will mean that future new clinical applications could become available. Given the range of treatment options for ear surgery, a scientifically sound and clinically meaningful PRO measure that measures ear appearance and adverse effects following surgery is needed. Such a PRO measure would not replace current outcome measures, but could be used in addition to provide a fuller understanding of the impact of different surgical approaches on patient outcomes.

Although prominent ears can affect up to 5% of the population,³⁴ microtia is rare, affecting 1 in every 7,000 live births.³⁵ Developing a PRO measure for a rare condition can be challenging and requires a collaborative network of committed patients, health professionals, and researchers.³⁶ Engaging patients in the identification of important concepts and using their stories to develop highly specific content has resulted in the development of an 18-item ear appearance scale and a 12-item adverse effects scale. We are now field-testing these scales in craniofacial sites in multiple countries. Our overall goal is to refine each scale by identifying the best subset of items that work together to measure a clinical hierarchy. By recruiting patients from multiple countries into the field-test study sample, we can ensure that the final set of items retained in each of scale, and their scoring algorithms, reflects multiple perspectives. Once finalized, this new ear-specific PRO measure could be used in comparative effectiveness research to address variation in practice and outcomes based on different surgical approaches.

This study has certain limitations. First, only 2 participants in the qualitative sample had an ear condition other than prominent ears or microtia and the oldest participant was 21 years of age. It was not possible to know if the concepts generated from our qualitative dataset include the concerns of all ear conditions or older adults. To address these limitations, the international field-test study does not have an upper age limit and is open to patients with any kind of ear condition. Rasch Measurement Theory analysis²⁹ will be used to examine differential item functioning (DIF) to enable us to determine if the scales work the same across all ear conditions as well as by age. To address this limitation, the international field-test study is open to patients with any kind of ear condition, and Rasch Measurement Theory analysis²⁹ will be used to examine differential item functioning (DIF). This approach will enable us to determine if the scales work the same across all ear conditions. If DIF is detected, we can decide to drop the items with DIF or keep the items and adjust for DIF in the scoring of the scales. Another limitation is that we did not develop a scale to measure hearing problems, which was a concern raised by some participants with microtia. There are, however, other pediatric PRO measures designed to measure hearing problems.^37,38

CONCLUSIONS

The aim of this study was to develop 2 ear-specific scales for ear conditions. The 2 ear scales are currently being field-tested internationally as part of our efforts to develop the FACE-Q Module for Children and Young Adults. Once finalized, we anticipate these new scales will be used in clinical practice and in research to understand the patient perspective of outcomes following ear surgery.

ACKNOWLEDGMENTS

The authors are grateful for grant funding provided by the Canadian Institutes for Health Research and the Plastic Surgery Foundation.