CEDRA – a tool to help consumers assess risk for ear disease

Abstract

This paper introduces the Consumer Ear Disease Risk Assessment (CEDRA) tool. CEDRA is a brief questionnaire designed to screen for targeted ear diseases. It offers an opportunity for consumers to self-screen for disease before seeking a hearing device and may be used by clinicians to help their patients decide the appropriate path to follow in hearing healthcare. Here we provide highlights of previously published validation in the context of a more thorough description of CEDRA’s development and implementation. CEDRA’s sensitivity and specificity, using a cut-off score of 4 or higher, was 90% and 72% respectively relative to neurotologist diagnoses in the initial training sample used to create the scoring algorithm (n = 246). On a smaller independent test sample (n = 61), CEDRA’s sensitivity and specificity were 76% and 80%, respectively. CEDRA has readability levels similar to many other patient-oriented questionnaires in hearing healthcare, and informal reports from pilot CEDRA-providers indicate that the majority of patients can complete it in less than 10 minutes. As the hearing healthcare landscape changes and provider intercession is no longer mandated, CEDRA provides a measure of safety without creating a barrier to access.

Introduction

Policies regulating the provision and acquisition of hearing aids are substantially changing. In the United States, government-sponsored committees have recommended restructuring the market to include an over-the-counter (OTC) hearing aid category, and to eliminate the need for a medical evaluation (or waiver of such an evaluation) prior to purchasing a hearing aid (President’s Council of Advisors on Science and Technology 2015; Committee on Accessible and Affordable Hearing Health Care for Adults 2016). In December 2016, the Food and Drug Administration (FDA) announced it would no longer enforce the medical evaluation/waiver provision for acquiring a hearing aid. In August of 2017, the President signed the Warren-Grassley Over-the-Counter Hearing Aid Act into law, which mandates the FDA to create an OTC hearing aid category for individuals, “without the supervision, prescription, or other order, involvement, or intervention of a licensed person” (S.670). One potential benefit of these changes is reduced cost to consumers and the health care system. For example, the removal of a required medical evaluation could have an immediate impact on Medicare costs. Freeman and Lichtman (2005) estimated annual cost-savings of $84–168 million to Medicare when comparing direct access to the audiologist versus physician-referring-to-audiologist pathways. The proposed changes to hearing healthcare have been made to increase the accessibility of hearing aids, and hearing healthcare for the American public.

These modifications have provoked trepidation and outright opposition among some in the hearing healthcare community. The arguments against OTC hearing aids and the elimination of a medical evaluation are similar to those that prompted the initial regulations; that hearing loss is a “medical issue” (Sawalich 2017), and that a lack of provider involvement creates a safety risk for individuals seeking hearing aids. Many conditions and diseases can include hearing difficulty as a symptom (Kleindienst et al. 2016), and some of these diseases have the potential for serious adverse health consequences. Unfortunately, accurate estimates of utilization of the physician clearance and medical waiver options are not available. Similarly, the epidemiological evidence is sparse on health outcomes for those seeking hearing aids with disease other than presbycusis or noise-induced hearing loss, but case reports highlight the potential risk. For example, vestibular schwannoma can have hearing loss as the primary presentation yet have significant negative outcomes, including death (Carlson et al. 2017). The new OTC hearing aid category, which removes any provider involvement, eliminates the additional safety offered by a trained professional evaluating a patient, whether that professional is an audiologist (Zapala et al. 2010) or physician.

Previously published red flags, or potential contraindications to hearing aid use, have been defined by the Food and Drug Administration (21 CFR 801.420, 1977) and further elaborated on by the American Academy of Otolaryngology, Head and Neck Surgery (2014). When red flag conditions are present, a medical evaluation is recommended. The FDA identified eight red flags including medical conditions of the ear (i.e. ear deformity, active drainage, cerumen or foreign body in ear canal), symptoms (i.e. ear pain, acute or chronic dizziness, sudden or rapid progression of hearing loss), and audiometric findings of air-bone gap of greater than 15 dB in the speech frequencies. In 2014, the AAO-HNS expanded upon the existing flags, and removed guidelines that constrained symptoms to particular time periods. The expanded guidelines added history of ear infections and family history of hearing loss, further specific diseases, recurrent episodes of dizziness, certain kinds of hearing loss (e.g., unexplained conductive hearing loss, asymmetric or unilateral hearing loss and mild or greater bilateral hearing loss), unilateral or pulsatile tinnitus, and > 15% speech discrimination differences between ears or < 80% bilaterally. The purpose of the expanded red flags was to ensure accurate medical diagnosis and appropriate medical/surgical treatment for those with hearing loss. Using the red flags, treatment of hearing loss with hearing aids was essentially limited to those with noise- or age-related sensorineural hearing loss, with all others requiring further medical evaluation.

We believe that it is possible to balance the concerns of safety and accessibility in hearing healthcare and that we can achieve this balance without requiring every hearing healthcare-seeking individual to first see a physician. The purpose of this perspectives paper is to introduce a free, easy-to-use tool that provides a measure of safety without creating a barrier to entry into hearing healthcare: the Consumer Ear Disease Risk Assessment (CEDRA). We provide highlights of previous research defining the range of diseases of concern and the initial validation of CEDRA in screening for these diseases. Also, we detail the development of CEDRA, provide new information on its scoring and the score’s relationship to disease status in the initial validation, and new details on its usability and availability.

CEDRA was developed by a multidisciplinary team of researchers with input from patients, clinical audiologists, and otolaryngologists to detect the presence of 104 targeted ear diseases (TEDs; Kleindienst et al. 2016). CEDRA is designed to be completed by an adult consumer considering hearing aids for the first time. The questionnaire can be completed in less than 10 minutes and is available in English and Spanish. It has been tested with 307 patients at the Mayo Clinic Florida, (Kleindienst et al. 2017) and is currently being validated in a larger sample of patients at the Mayo Clinic Florida, Mayo Clinic Arizona, University of Texas, Medical Branch – Galveston, Northwestern University, and several non-tertiary audiology clinics. The Supplemental Digital Content includes a full version of the CEDRA questionnaire. A web-based version is also available at http://cedra.northwestern.edu. CEDRA is a promising new tool for consumers to assess their risk of disease.

Until the FDA’s recent announcement ceasing its enforcement, the federal regulation required a physician evaluation prior to an individual acquiring a hearing aid. Anecdotally, consumers frequently chose an alternative to this evaluation by signing a waiver stating they knew that a medical evaluation was in his or her best health interest (21 C.F.R. § 801.421 2016). If such a waiver was signed, the hearing aid provider was expected to monitor for so-called “red flags” that should prompt the provider to refer the consumer to a physician. Both options, the evaluation and the waiver, require the presence of a provider to assess the consumer’s risk of TEDs or other potential contraindications of hearing aid use and are still in place in most state licensure regulations. The new OTC category of hearing aids specifically forbids the requirement of a licensed provider, so any OTC-related assessment of TED risk must be done without a licensed provider’s intercession. We believe CEDRA could mitigate patient risk by acting as a decision advisor without the cost of using a licensed provider. By completing CEDRA when a hearing aid is purchased in the absence of a physician, CEDRA could contribute significantly to improving consumer safety by alerting the purchaser to their disease risk.

Development of the CEDRA Tool

The initial step in the creation of CEDRA was to define the scope necessary for a disease-detection tool. Kleindienst et al. (2016) reported on this effort to identify and prioritize diseases relevant for hearing healthcare. The authors identified 195 potential diseases and conditions from a review of textbooks on otology, neurotology, and otolaryngology pathologies. These 195 conditions, along with 15 non-otologic conditions (e.g., anxiety), were given to 5 board-certified, practicing neurotologists for rating on three scales: 1) the difficulty of diagnosing the disease, 2) potential consequences of missed diagnosis, and 3) the likelihood that hearing loss would be the primary or initial symptom of the disease/condition. Kleindienst et al. then took the median of each scale for each of the 210 items and reduced the scope to 104 diseases based on further review of each disease’s trajectory and the likelihood of otologic symptoms over that trajectory (Kleindienst et al. 2016, Table 2 and Figure 1). The list of 104 diseases/conditions offers one definition of the range of conditions that should be monitored in hearing healthcare. It also provides expert ratings of the severity of these TEDs, which could be used for future research and policy-making regarding disease detection in hearing healthcare.

Figure 1.

Results from the initial test of CEDRA (n=307), showing A: The probability of a diagnosis of targeted ear disease(s) by a certified neurotologist plotted against the CEDRA score. B: The relationship of the CEDRA scores to the missed diseases (squares) and unnecessary referrals (crosses). BOTH: The vertical dashed line shows the recommended consumer cut-off score of 4 or higher.

Though many aspects of current shifts in hearing healthcare are unprecedented, using a questionnaire to screen for conditions or diseases is not. Many such questionnaires exist in medicine, ranging from depression (Spitzer, Kroenke, & Williams 1999) to pulmonary disease (Martinez et al. 2008) to connective tissue disease (CTD, Karlson, et al. 1995). This last example may be informative because similar to CEDRA the Connective Tissue Disease Screening Questionnaire (CSQ) screens for a variety of diseases rather than a single one. After its initial development and validation, the CSQ has been translated into Spanish to help screening reach additional populations (Potter et al. 2008). It has been used to screen for comorbidities in individuals already diagnosed with a CTD (Farrell et al. 2013), and has been used in large cohorts (n = 218,623) to screen for CTDs to determine risk factors of their development (Lu et al. 2014). A paper-and-pencil questionnaire used as a screener is often a cost-effective method to filter a large population for those most likely to need further diagnosis and treatment (e.g., Jiao et al. 2017, Wilson et al. 2018).

The research team decided that the consumer questionnaire mechanism was likely to accurately screen for TEDs while not impeding access to hearing healthcare. Kleindienst et al. (2017) described the development and validation of CEDRA, a consumer questionnaire designed to assess risk for the 104 TEDs defined in Kleindienst et al. (2016). First, three experts in TED diagnosis and symptomology designed preliminary questions based on the “red flags” from the FDA, the American Academy of Otorhinolaryngology, Head and Neck Surgery, and their clinical expertise with the TEDs. This process produced 40 potential questions. These questions were then evaluated for readability and intelligibility in cognitive interviews, modified, and evaluated again using a second round of cognitive interviews with a new sample of 28 items. After this final round of interviews, we slightly modified CEDRA before it was deemed ready for its initial test with clinic patients. For example, an initial prototype of CEDRA had the question “Have you ever had a sudden or rapidly progressing hearing loss?” which was eventually modified to “Have you ever had a sudden permanent change in your hearing?” Changes such as this one reflected suggestions by participants to improve intelligibility and readability. The research team suspended further revisions when participants indicated no changes were necessary, or suggested changes that were reversions to previous iterations. This indicated that the suggested changes were more likely reflective of individual preference than general issues with readability. Validation of the final version of CEDRA in a sample of 307 patients at the Mayo Clinic in Florida suggested the retention of 15 items would allow for adequate sensitivity and specificity.

The CEDRA Tool

CEDRA is a 15-item questionnaire with yes/no and multiple-choice questions, available in both Spanish and English. The initial statement to the patient describes the purpose of the questionnaire and offers a reminder to seek medical attention if they have health-related questions or concerns. The majority of questions relate to hearing and balance, one question relates to general health, and the remainder query non-otological symptoms, such as vision impairment or recurring fever that co-occur with hearing issues in some TEDs. Throughout the design and validation process, a small number of study participants took the opportunity to comment on their intent or meaning when answering a question or creating new answer categories when using a paper version of CEDRA. In a clinical setting, patients writing-in their own answers could complicate scoring CEDRA by creating response categories not tested in the validation study. However, this characteristic is true of all paper-and-pencil questionnaires, and write-ins could help guide a clinician to ask follow-up questions in their examination to get clarification of answers. Based on post-test interviews and informal timing during its development and validation, most patients find CEDRA relatively easy to complete and can answer all questions in less than 10 minutes.

Access and ease of use are dependent on a variety of factors. CEDRA has a Flesch-Kincaid reading grade level (Kincaid, Fishburned, Rogers, & Chissom 1975) of 5.8, and a Flesch Reading Ease (FRE) score of 74.3, which corresponds to a “fairly easy” reading level (Flesch 1948). Kelly-Campbell and colleagues have determined the readability of a variety of questionnaires and written tools over the past several years and found comparable levels. For example, the Hearing Handicap Inventory for the Elderly has been found to have an FRE of 78 (Kelly-Campbell et al. 2012), and the International Outcome Inventory for Hearing Aids has a Flesch-Kincaid reading level of 7.9 (Donald & Kelly-Campbell 2018). Other hearing healthcare questionnaires’ and patient-reported outcome measures’ readability scores vary, but grade levels were reported in the range of 5.9–15.4 (Donald & Kelly-Campbell 2018). General guidelines for health information indicate that written materials should be around the 4^th to 6^th-grade level to allow for the generally low literacy levels of the American public (Matthews & Sewell 2002; Ang, Miller, Schmitt, & Wen 2013). It is promising that CEDRA falls within this range, but we have not yet conducted formal testing of CEDRA usability beyond the cognitive interviews used in its development. Older adults (aged 65 years and older) have lower health literacy than their younger cohorts (Kutner et al. 2006), whilst simultaneously suffering higher rates of hearing loss (Lin, Niparko, & Ferrucci 2011). Usability will be particularly important if patients are interested in completing and scoring CEDRA themselves, rather than with the assistance of a hearing healthcare provider.

CEDRA Scoring

The third page of CEDRA displays the scoring algorithm. It is a simple summation of “yes” responses to most questions, along with a dichotomization of the few scalar questions. For example, Q10. “Overall, how would you rate your health? Very good, good, poor, very poor,” is scored as one point for an answer of ‘poor’ OR ‘very poor,’ and no points for an answer of ‘very good’ OR ‘good’. For questions 11 and 12, we use a similar process, with the response scales divided into two and the worse half receiving one point. Question 13, regarding tinnitus, does not receive any points by itself, as our analysis showed that the presence of tinnitus was not an adequate predictor of TED status. This is likely because tinnitus is common in the general population, with an estimated prevalence of 8–30% (Sindhusake et al., 2003; Shargorodsky, Curhan, & Farwell 2010; Kochkin et al. 2011; Nondahl et al. 2011, Bhatt et al. 2016). Question 13a, however, does receive a point if tinnitus is present in either the left OR right ear, but no points for both ears or the patient being “unsure” about its laterality.

Consumers can use the third page to calculate their CEDRA score or use the web-based CEDRA for automated scoring. Based on our initial research outlined above, CEDRA makes recommendations for consumers to see a doctor if the score is 4 or greater. This cut-off provided the best specificity (72%) while maintaining an adequately high sensitivity (90%) in our initial training sample.

CEDRA Validation

To test CEDRA, patients from the otorhinolaryngology and audiology departments at the Mayo Clinic, Florida, were recruited (n = 307). TED representation in this sample was exceptionally high; 75% (n = 231) had one or more TED, whereas the remaining 25% (n = 76) had diagnosed age- or noise-related hearing loss. The CEDRA data from the whole sample was then randomly separated into training (80%, n = 246) and test sub-samples (20%, n = 61). The training data were used to develop the simple scoring algorithm discussed above that provided sensitivity of 90% and specificity of 72%, while the test data was held separately to provide a second check on the scoring system. In the test sample, the scoring algorithm provided a sensitivity of 76% and specificity of 80%. Using the same dataset, Klyn et al. (2018) found that the FDA red flags, which require a provider for assessment, had similar test characteristics as CEDRA in the training sample (FDA sensitivity = 91%, FDA specificity = 72%). CEDRA performance was worse in the smaller test sample, which is not surprising given that the algorithm was calibrated based on the training data.

The cut-off score of 4 was chosen based on an a priori decision to find the score at which the probability of ear disease in the test sample was greater than 50%. As is to be expected, there is a trade-off between sensitivity and specificity for different CEDRA scores which clinicians and researchers may find useful. Figure 1 displays the relationships of CEDRA scores to the probability of TEDs (A) and sensitivity and specificity (B) in the initial test of 307 patients. As the CEDRA score increases, the probability that a participant had one or more TEDs similarly increased, reaching asymptote near a CEDRA score of 10 (Fig. 1A). Relatedly, the misses or false negatives went up as the CEDRA score increased, while the false positives decreased (Fig 1B). The more traditional ROC curve may be seen in Kleindienst et al. (2017). Table 1 provides a subset of these data for easier implementation by a clinician. One important consideration for a clinician selecting a cutoff score may be cost. Due to the wide range of potential outcomes of a missed or delayed diagnosis of a TED, we were unable to provide an estimate of the cost due to missed cases. However, it is important to realize that these costs may be quite significant. For the patient, a delay in diagnosis may result in extra days of missed work, increased travel expenses, decreased quality-of-life, and increased risk of mortality and morbidity. For each delayed diagnosis, the healthcare system may have to absorb extra appointments, surgeries, or increased risk of complications in procedures.

Table 1.

Estimated trade-off of sensitivity and specificity

CEDRA score	Sensitivity (%)	Specificity (%)	# of false positives per 100 patients	# of missed TEDs per 100 patients
X	100	0	95	0
2	98	39	58	0
4	90	72	27	1
8	54	95	5	2
12	20	100	0	4

Data based on the initial “training” sample from Kleindienst et al. (2017) and an assumed TED prevalence of 5% for illustrative purposes. The first row represents every patient receiving a medical evaluation, and assumes the physician would catch every targeted ear disease (TED). Subsequent rows show different CEDRA cut-off scores and the estimated outcomes.

Similarly, a false positive may result in unnecessary expenses (i.e., physician evaluations) or even costs related to psychosocial harm (e.g., Brodersen & Siersma 2013). These results provide the opportunity for clinicians to interpret the cut-off score depending on context. For example, in settings where a consumer can easily and inexpensively obtain a physician evaluation, or when patients have very low-risk tolerance, a lower cut-off may be warranted. Conversely, older patients who are also more risk tolerant may desire a higher cut-off, as the consequences of a missed TED diagnosis may be less dire.

CEDRA does not replace a medical evaluation. Instead, it is designed to function as a screening tool for determining the need for a medical evaluation. This screening would be important for people who might forego the medical evaluation altogether during their search for a hearing aid. The scoring system, shown on the final page of the CEDRA tool, is designed to offer a simple yes/no recommendation based on the receiver operating characteristic analysis from the initial validation study. Such a cutoff is common in screening, where the intent is to identify the need for further diagnostic testing, rather than provide a diagnosis.

Limitations

There are some important caveats to note. First, the preliminary sample for this study was not a random representative sample of the United States. Thus, it is important to study diverse subgroups of people with hearing loss in future studies. Our sample was chosen to have a high prevalence of TEDs to provide an adequate test of CEDRA’s sensitivity. It is clear that the initial validation sample is not representative of the broader United States population in its high prevalence of TEDs and its racial and ethnic makeup (91% White and 73% White, Not-Hispanic or Latino). It is also important to report in this context that the sample used for both the red flags and CEDRA validation was unusually educated: 52.4% had an undergraduate degree or pursued graduate study, another 32% had either a trade-school education or had attended some college, 14.3% had a high school diploma, and only 1.3% had not completed high school. Compare this to the national statistics for those 40 years old or older in 2016: 31.8% bachelors or higher, 26.3% trade school or some college, 30.3% high school diploma, and 11.7% without a high school degree (U.S. Census Bureau 2017). It is likely that the sample had higher literacy levels than the general American public (Kutner et al. 2006), and therefore may have found CEDRA more readable and easier to complete.

Further studies should determine differences between our sample and the general hearing-aid-seeking population or specific sub-groups. One such study is currently underway at the Mayo Clinic Florida, Mayo Clinic Arizona, University of Texas Medical Branch, and Northwestern University. Second, CEDRA only screens for the 104 TEDs identified by Kleindienst et al. (2016). We have not validated CEDRA for other diseases or conditions that may contraindicate hearing aid use. One prominent example is cerumen impaction, which affects 2–5% of otherwise healthy adults (Roeser & Ballachanda 1997; Karlsmose et al. 2001; Kozin et al. 2015). It is currently untested whether CEDRA would correctly flag individuals with cerumen impaction, though the current iteration is not specifically designed to do so.

Furthermore, in the initial validation presented in Kleindienst et al. (2017) the severity of the patient’s condition was not considered, only whether a TED was present or not. Finally, when considering using CEDRA, it is worth recalling the low prevalence of TEDs and the relatively high prevalence of age- and noise-related hearing difficulties. As an example, assume that the prevalence of TEDs is 5%. Out of every 100 individuals, CEDRA will suggest 33 for further evaluation, if the test characteristics found in the larger training sample bear out; 4 will have underlying TEDs, and 29 will be false alarms. If the true sensitivity and specificity are closer to that found in the testing sample, CEDRA will suggest 23 for future evaluation; 4 with underlying TEDs and 19 false alarms. This test performance is comparable to estimates of screening for acoustic neuromas using threshold asymmetry and better than some found for acoustic reflex thresholds (Hunter et al. 1999). Nevertheless, improving the specificity of CEDRA is a major goal for future research.

Summary

CEDRA is a tool designed for consumers to easily and inexpensively assess their risk of ear disease and we designed it with the intention of maintaining safety while increasing accessibility of hearing healthcare. CEDRA is a brief questionnaire designed to screen for targeted ear diseases and has been tested in a clinical sample of 307 patients. It offers an opportunity for consumers to self-screen for disease prior to seeking a hearing device or may be used by providers to help their patients decide the appropriate path to follow in hearing healthcare, whether it be medical intercession or the acquisition of hearing aids. As of this writing, many state licensing boards still require medical evaluation or waiver prior to hearing aid sales, even though the FDA no longer enforces this requirement at the federal level. CEDRA cannot replace a medical evaluation in these cases though may serve as a decision-aid for those considering a waiver. The introduction of OTC hearing aids may also necessitate a tool such as CEDRA, as the OTC hearing aids are mandated to be “available over-the-counter, without the supervision, prescription, or other order, involvement, or intervention of a licensed person, to consumers through in-person transactions, by mail, or online” (21 C.F.R. § 360j 2018). With no licensed provider, OTC hearing-aid seekers do not have any assurance that their hearing difficulties are not signs of a disease or condition that warrants medical intervention. CEDRA would provide additional warnings to those considering an OTC device, which would, in turn, direct these hearing-aid seekers to a professional who can provide medical evaluation and treatment.

It is important to note that the absence of a TED does not, by itself, indicate that an individual is an appropriate candidate for hearing aids, nor does the presence of a TED necessarily contraindicate hearing aid use. CEDRA is not designed to diagnose disease, nor to provide an estimate of its severity. That responsibility remains with medical professionals. Importantly, based on its initial study, it appears that CEDRA can offer the consumer some ability to screen for the probable presence of TEDs without the need to see a hearing healthcare provider. Validation in a larger, more representative sample is currently underway in a multi-site study (R33 DC013115 NIH/NIDCD).