Using Benefit-Cost Ratio to Select Universal Newborn Hearing Screening Test Criteria

Abstract

Objectives

Current protocols presumably use criteria that are chosen on the basis of the sensitivity and specificity rates they produce. Such an approach emphasizes test performance, but does not include societal implications of the benefit of early identification. The purpose of the present analysis was to evaluate an approach to selecting criteria for use in UNHS programs that utilizes BCR to demonstrate an alternative method to audiologists, administrators, and others involved in UNHS protocol decisions.

Design

Existing data from over 1200 ears were used to analyze benefit-cost ratio (BCR) as a function of DPOAE level. These data were selected because both audiometric and DPOAE data were available on every ear. Although these data were not obtained in newborns, this compromise was necessary because audiometric outcomes (especially in infants with congenital hearing loss) in neonates are either lacking or limited in number. As such, it is important to note that the characteristics of responses from the group of subjects that formed the bases of the present analyses are different from those for neonates. This limits the extent to which actual criterion levels can be selected but should not affect the general approach of using BCR as a framework for considering UNHS criteria. Estimates of the prevalence of congenital hearing loss identified through UNHS in 37 states and U.S. territories in 2004 were used to calculate BCR. A range of estimates for the lifetime monetary benefits and yearly costs for UNHS were used, based on data available in the literature. Still, exact benefits and costs are difficult to know. Both one-step (DPOAE alone) and two-step (DPOAE followed by AABR) screening paradigms were considered in the calculation of BCR. The influence of middle-ear effusion was simulated by incorporating a range of expected DPOAE level reductions into an additional BCR analyses.

Results

Our calculations indicate that for a range of proposed benefit and cost estimates, the monetary benefits of both one-step (DPOAE alone) and two-step (DPOAE followed by AABR) NHS programs outweigh programmatic costs. Our calculations indicate that BCR is robust in that it can be applied regardless of the values that are assigned to benefit and cost. Maximum BCR was identified and remained stable regardless of these values; however, it was recognized that use of maximum BCR could result in reduced test sensitivity and may not be optimal for use in UNHS programs. The inclusion of secondary AABR screening increases BCR, but does not alter the DPOAE criterion level at which maximum BCR occurs. The model of middle-ear effusion reduces overall DPOAE level, subsequently lowering the DPOAE criterion level at which maximum BCR was obtained.

Conclusion

BCR is one of several alternative methods for choosing UNHS criteria, in which the evaluation of costs and benefits allows clinical and societal considerations to be incorporated into the pass/refer decision a meaningful way. Though some of the benefits of early identification of hearing impairment cannot be estimated through a monetary analysis, such as improved psychosocial development and quality of life, this paper provides an alternative to audiologists and administrators for selecting UNHS protocols which includes a consideration of societal implications of UNHS screening criteria. BCR suggests that UNHS is a worthwhile investment for society as benefits always outweigh costs, at least for the estimations included in this paper. Though the use of screening criteria that maximize BCR results in lower test sensitivity compared to other criteria, BCR may be used to select criteria that result in increased test sensitivity and still provide a high, though not maximal BCR. Using BCR analysis provides a framework in which the societal implications of NHS protocols are considered, and emphasizes the value of UNHS.

INTRODUCTION

Current approaches in universal newborn hearing screening (UNHS) presumably use pass/refer criteria selected because they are expected to result in a desired sensitivity and/or specificity. The purpose of the present study was to evaluate an approach to selecting criteria for use in UNHS programs that utilizes benefit-cost ratio (BCR) to demonstrate an alternative method to audiologists, administrators, and others involved in UNHS protocol decisions. At its core, BCR is one of several metrics commonly used in cost-benefit analyses. Data are described using national statistics highlighting the utility of this method at administrative and policy levels. There are many costs and benefits from UNHS, some of which are easy to estimate (for example, UNHS and follow-up costs) and some of which are more difficult to estimate (such as costs for an undetected hearing loss, which will depend on the magnitude of the loss; lifetime benefits of early identification relative to quality of life). In this paper, we use estimations and speculations of benefits and costs to present a model for selecting criterion values based on BCR. While every effort is made to select values that are consistent with published reports, the specific costs and benefits are perhaps less important than the notion of using benefit and cost to determine criteria for screening protocols.

The Centers for Disease Control (CDC) program on Early Hearing Detection and Intervention (EHDI) estimated that approximately 92% of all infants born each year from 2004 to 2006 in the United States received initial screening for hearing loss as newborns (CDC, 2004c; 2005c; 2006). Of these children, approximately 2–3% did not pass their newborn hearing screening. Approximately 6% of those that did not pass were found to have hearing loss. These findings are consistent with past prevalence estimates indicating an occurrence of hearing loss in approximately 1.27 per 1000 infants screened (Prieve and Stevens, 2000; Finitzo, Albright, and O’Neal, 1998). Since the initiation of UNHS, the average age of identification has decreased from 24–30 months of age to 2–3 months of age (Harrison, Roush, and Wallace, 2003). Thus, in terms of early identification, UNHS apparently is achieving its goal.

The focus on early identification relates to the importance of hearing in the development of speech and language skills, which, in turn, impacts educational, economic, and social abilities. In a recent review, the United States Preventive Services Task Force (USPSTF) recommended screening for hearing loss in all newborns, noting that children with hearing loss have increased difficulties with communication skills, behavior, psychosocial well-being, and educational attainment, compared to children with normal hearing, and that early detection of hearing impairment improves language outcomes (USPSTF, 2008). It has been shown that children with hearing impairment achieve better language skills if the impairment is identified early (Kennedy et al, 2006) and is accompanied by early intervention services (Appuzo and Yoshinaga-Itano, 1995; Moeller, 2000). There are benefits to early identification and intervention regardless of degree of hearing loss (Yoshinaga-Itano, 2003; Appuzo and Yoshinaga-Itano, 1995; Moeller, 2000). Past research has shown that children with hearing loss who were identified later than six months of age had language levels that were correlated with degree of hearing loss; in contrast, the language development of children with hearing loss who were identified prior to six months of age was not predicted by degree of loss (Yoshinaga-Itano, 2003). Keren et al. (2002) modeled similar language benefits and demonstrated cost-effectiveness of UNHS for a hypothetical cohort of 80,000 infants. It may be reasonable to expect that children with age-appropriate language skills will likely incur less educational expense. Schroeder et al. (2006) reported that there are increased costs for health and social services, including education, in children with hearing loss compared to children with normal hearing. As expected, cost was predicted by severity of hearing loss and inversely related to receptive language abilities. The benefits of early identification and intervention occur regardless of the magnitude of the loss, but there may be an interaction between the need for educational support services, language ability, and magnitude of hearing loss for children in whom hearing loss was identified later in life.

It is known that children with sensorineural hearing loss have language and educational delays compared to children with normal hearing (Appuzo and Yoshinaga-Itano, 1995; Bess, Dodd-Murphy, and Parker, 1998; Kelly and Gaustad, 2006; Kennedy et al., 2006; Moeller, 2000; Yoshinaga-Itano, 2003). Educational delays or lack of educational achievement can result in lifelong personal and societal monetary effects. The likelihood that young adults with hearing loss needed Social Security Disability Insurance and Supplemental Security Income decreased as the level of education increased (Clarcq and Walter, 1998). It has also been reported that the dropout rate among deaf students is 44%, compared with a rate of 19% among the general population (Blanchfield et al., 2001). This finding has important implications, as recent U.S. Department of Labor employment projections indicate that unemployment rates are higher and median weekly earnings are lower for those without a high-school diploma (Bureau of Labor and Statistics, Current Population Survey, 2007). In summary, studies suggest that low language and literacy skills (which are, at times, associated with hearing loss) can result in lower educational achievement, higher unemployment rates, and lower earnings. Thus, hearing loss may have both societal and personal costs associated with it.

Much of the work related to the development of screening measures for hearing loss comes from work in patients for whom the pure-tone audiogram is known. Distortion Product Otoacoustic Emission (DPOAE) test protocols, at least for older patients, typically are based on prior laboratory studies in which DPOAE data were compared to the pure-tone audiogram (e.g., Gaskill and Brown, 1993; Gorga et al., 1993a, 1993b, 1996b, 1997). Several analyses of DPOAE data have been used to differentiate between normal-hearing and hearing-impaired ears. In some cases, simple criteria (such as DPOAE level or signal-to-noise ratio, SNR) are selected based on expected sensitivity and specificity rates (e.g., Gorga et al., 1997). Other studies have described both transient evoked OAE and DPOAE test performance for more complex criteria, such as those derived from multivariate analyses (Gorga et al., 1999, 2005; Dorn et al., 1999, Hussain et al., 1998). In other cases, assumptions may be made that a certain SNR (say, 6 dB) indicates that hearing is normal, although test performance data supporting this view may be lacking. While seldom explicitly stated, it is assumed that DPOAE protocols, whether for use in UNHS or in outpatient clinics, are typically based on criteria chosen because of their expected sensitivity and specificity. Limited knowledge about sensitivity and specificity probably is a more common occurrence in UNHS because of the prevalence of congenital hearing loss and difficulty in obtaining gold-standard data that are needed to determine the value of these test-performance variables.

A consideration of societal implications of screening algorithms may be warranted when justifying UNHS protocols. In this light, we propose an alternative approach in which criteria for UNHS protocols are developed in relation to cost and benefit. Many health interventions are analyzed using cost-utility analysis to guide abetment decisions such as cochlear implants, orthopedics and cardiovascular medicine (Barton et al. 2006; Palmer et al., 1999; O’Neil et al., 2000; Brauer, Neumann, and Rosen, 2007; Winklemayer et al., 2003). This study focused on a monetary-based, BCR analysis in determining the criteria for use in UNHS programs. In general, BCR is the ratio of the benefits of a program relative to its costs, with both variables being expressed in monetary terms. It is typically used in cost-benefit analyses to summarize the overall economic value of a program. A limitation of BCR analyses is that they exclude factors that are not defined monetarily. In the case of hearing loss, this would include factors such as the emotional impact on the family and the psychosocial development of the child. These factors would increase “value” of UNHS, although not in a way that can be estimated financially.

We determined the effect that varying the sensitivity and specificity of DPOAE NHS protocols has on BCR in relation to the degree of impairment. Given the paucity of test-performance data on neonates (eg., Norton et al., 2000), we relied on data from an older cohort of subjects on whom both DPOAE and gold-standard (audiometric) data were available. The use of this cohort influences the specific criterion value at which “optimal” performance occurs, but should not influence the demonstration of the usefulness with which an approach based on monetary factors can be used. In the case of UNHS, BCR is expected to vary with prevalence, degree of loss, age of identification, method of screening (one-step versus two-step program), and status of the middle ear. Some of these influences are considered in this paper.

METHODS

A. Estimated Monetary Benefits of Early Identification

Data regarding expected societal costs in relation to severity of hearing impairment are limited, as are data regarding the societal costs relative to age of identification. However, data do exist from which estimates can be derived. In addition to determining that societal costs decreased with increasing receptive language scores, Schroeder et al. (2006) found that children with profound hearing loss generated increased costs for annual health visits and their parents incurred greater personal costs when compared to children with severe and moderate hearing impairment. After adjusting for level of hearing loss, nonverbal cognitive ability, socioeconomic status, and the presence of other health impairments, overall costs were approximately 15% lower in the cohort of children with hearing impairment identified by UNHS. This difference was not statistically significant, although it has been suggested that this may be due to high variability in individual cost data (Grosse and Ross, 2006). For the purposes of a methodological evaluation, we presume that there are lifetime monetary benefits associated with UNHS, and propose values relative to magnitude of hearing loss in order to demonstrate the utility of the model. It should be noted, however, that although the benefit of early identification has been shown for language, literacy, and psychosocial skills, the monetary benefit to society of early intervention remains less defined.

Benefit associated with early identification may depend on the severity of hearing loss. For example, the benefit of early intervention of a severe-to-profound hearing loss is expected to include increased opportunities for language, literacy, education, and workforce opportunities that may otherwise have been inaccessible. Mohr et al. (2000) reported profound hearing loss which occurs prior to the acquisition of language may have an overall lifetime cost that can exceed $1 million including educational services, social services, and reduced work productivity. The benefit of early intervention for a child with mild hearing loss might include increased opportunities for speech and language development and access to early linguistic cues which will assist with later academic and social achievement that may not have occurred if the loss was identified later in the child’s life. Although more research is needed, past research has shown that even children with minimal hearing loss demonstrate lower skills in some academic subjects, compared to peers with normal hearing (Bess, Dodd-Murphy, and Parker, 1998), and that 18% to 35% of children with unilateral hearing loss fail one or more grades (for review, see Tharpe, 2008). These data suggest that even mild hearing loss, left untreated, has costs associated with it. The cost of educational services, social services, and reduced work productivity will be less for children with mild hearing loss, compared to costs associated with greater hearing loss, but they are not zero.

Individual estimates of lifetime cost reductions were applied to reported CDC prevalence data (CDC, 2004a, 2004b) to estimate benefit for the identification of hearing loss. Our initial estimates of benefit are based on data reported by Schroeder et al. (2006) and Mohr et al. (2000). Schroeder et al. (2006) reported an approximate two-fold increase in cost with increasing severity of hearing loss. Profound hearing loss was estimated to cost approximately $11,108 less per year when diagnosed with UNHS, compared to the costs for cases which were identified later. Applying this value to a person with an average lifespan of 79.2 years, the lifetime savings of UNHS totals $879,780. Though we assumed costs to be the same for every year of life to simplify our methodological illustration, costs and benefits would likely differ across the lifespan. Mohr et al. (2000) described annual cost of educational services, noting that these costs increase with level of intervention as follows: resource room ($6,100), self-contained classroom ($14,500), day school ($28,200), and residential school ($53,200). This pattern may provide a framework for assigning costs relative to magnitude of loss, assuming there is a correlation between level of required service and magnitude. Over the educational lifetime of a child, money would be saved if early intervention was to result in placement in a more inclusive educational setting. It is also reasonable to estimate that over the lifespan, money would be saved if early intervention was to result in less social-services support and increased earnings. For the purposes of this study, lifetime monetary benefit of early identification were made based on these assumptions, and are estimated by severity of hearing loss as follows: mild = $250,000, moderate = $500,000, severe = $750,000, and profound = $1,000,000. As will be shown in subsequent sections of this paper, the present approach, in which BCR is used to determine UNHS protocols, can be applied regardless of the specific benefits and costs assumed in the equation, a potentially important observation, given the uncertainties in some of the above estimates.

Weighted values, in which prevalence for hearing-loss categories were taken into account, were combined to provide an estimate of lifetime monetary benefit according to severity of hearing loss (Table 1). A subset of national data were reported which included type and severity of hearing loss for 2,073,969 infants who were screened for hearing loss (CDC, 2004a, 2004b). This number represents approximately 54% of the 3,496,452 infants reported to have been born that year (CDC, 2004c). These data were used in the estimates to follow because information about severity of loss is necessary for calculations of BCR. It is important to note that we remain uncertain regarding the validity of these lifetime cost-savings estimates, which may be too generous. For example, Keren et al. (2002) estimated the difference in lifetime societal costs of approximately $429,000 between children with profound deafness who had delayed language development and those with normal language development. For the purposes of demonstrating methodological utility, additional scenarios are described, for which estimates of lifetime cost-savings are half of the savings proposed above.

Table 1.

Lifetime benefit estimates for 37 states and territories in U.S. in 2004 as reported by the CDC (n=2,073,969).

Estimate of Benefit	Mild	Moderate	Severe	Profound	Total
Prevalence	910	453	244	236	1843
Individual cost reduction	$250,000	$500,000	$750,000	$1,000,000
Total cost reduction (in millions)	$227.5	$226.5	$183.0	$236.0	$873.0

B. Estimates of Cost for a UNHS

The costs for UNHS programs, although not expected to be uniform for all birthing centers, are perhaps better known than the savings incurred as a result of early identification. At the least, it is possible to provide UNHS cost estimates that may be better based in facts. The costs of follow-up diagnostic testing should be considered, even if they are not incurred by the birthing hospital, because they represent a cost to society as well. These costs will depend on the measures that are used at follow up and on conditions that may be idiosyncratic to individual centers; however, regardless of these factors, costs will be affected by the referral rate. If many babies are referred from UNHS programs, the costs for follow-up testing will be large, whereas if few babies are referred, the costs will be lower, regardless of the follow-up protocols used at individual centers.

The cost of a UNHS program that utilizes a one-step DPOAE screening was based on data reported by Gorga et al. (2001) and Gorga and Neely (2003). Costs of a two-step screening including AABR are included in subsequent sections. It was assumed that DPOAE equipment would cost $6500 for every 4000 babies born, disposable tips cost $1 each, staff salary would be $20 per hour, and an average of four babies could be screened per hour. Cumulative cost was calculated for DPOAE criteria yielding referral rates ranging from 0% to 100%. Follow-up diagnostic ABR testing was assumed to cost $400. If criteria were such that no babies referred on the initial DPAOE screening, the cost of a national UNHS program was estimated at approximately $16 million. In contrast, if criteria were such that all babies were referred for additional diagnostic testing, the national cost for UNHS was estimated as $845 million.

For the purposes of this methodological demonstration, an alternate example was chosen in which the one-step DPOAE NHS program was more expensive, compared to the one described above. In the more expensive example, we estimated that one piece of DPOAE equipment would be needed for every 2000 babies born and staff would be paid $40 per hour, essentially doubling the costs from the previous example. The primary purpose of this paper is to illustrate the utility of BCR analyses in determining criteria for UNHS. As such, our benefit and cost estimates are important to the extent that they demonstrate that a variety of estimates can be used in the equation.

Following the identification of hearing loss, the cost of initial examination and diagnosis is similar within a certain time frame. If hearing loss is identified later in life, other techniques may be employed to verify and diagnose hearing loss such as visual reinforcement audiometry (VRA) and conditioned play audiometry (CPA). The costs of these evaluations are different from the cost of diagnostic ABR evaluations. Due to the incidence of secondary disabilities, evaluations by other specialists (for example, otolaryngology, ophthalmology, and genetics) are often recommended once hearing loss has been confirmed. Obviously, these additional evaluations have costs associated with them, but those are not included in our estimates of the cost of a UNHS program.

C. DPOAE Test Performance Data

In general, there is a paucity of information describing test performance for NHS tests, mainly because of the difficulty in obtaining the follow-up, gold-standard audiometric data against which the data obtained in the neonatal period were compared. Although Norton et al. (2000) described neonatal DPOAE data in relation to subsequent audiometric findings; the sample of infants with hearing loss was relatively small, making it difficult to reliably estimate criterion values from these data. On the other hand, differences between the characteristics of the subjects in our data set and those from newborns exist. Research has suggested that DPOAE responses are larger in newborns and infants than older children and adults. Prieve et al. (1997) reported infants younger than one year of age had higher DPOAE levels than older children and adults within the 2 kHz to 6 kHz range. Other data suggest greater differences between newborn and adult DPOAE levels at low- to mid-frequencies, but similar DPOAE levels at higher frequencies (Lafreniere et al., 1991; Lasky et al., 1992). Finally, the responses of neonates and infants are characterized by higher levels of noise (e.g., Prieve et al., 1997; Gorga et al., 2000; Abdala and Oba, 2008). These higher noise levels will limit the DPOAE level that can be measured reliably in neonates. For these reasons, the DPOAE data presented in this paper cannot be directly applied to a newborn population; rather, data specific to the newborn population must be collected and used for such analyses. Unfortunately, a large body of neonatal data from which test performance can be estimated reliably does not currently exist, forcing us to use a dataset from an older group of subjects to illustrate our points.

Given the limitations of the existing neonatal data in the literature, we opted instead to use DPOAE data from a large sample of older subjects for whom audiometric and DPOAE data were available, knowing that this approach is limited in its applicability with neonates. Having said that, these data are suitable for illustrating the general approach, even though they cannot be used to select criteria specific to neonates. To perform the present analyses, we used the DPOAE data, originally reported by Gorga and colleagues (Gorga et al., 1997). In the previous study, DPOAEs were measured in over 1200 ears of 806 subjects ranging in age from 1.3 to 96 years; approximately 1/3 of these ears had normal hearing, with the remaining 2/3 having hearing loss. Audiometric data were available for octave and half-octave frequencies from 750 Hz to 8 kHz, with pure-tone thresholds ranging from 15 to 120 dB HL. All subjects had normal middle-ear function verified by tympanometry, otoscopy, and, in some cases, comparisons of air-and bone-conduction thresholds at the time of the DPOAE test.

The f₂ frequencies used during testing corresponded to audiometric frequencies, and were presented at a fixed f₂/f₁ approximately equal to 1.22. Only data at 4 kHz were used in present analysis. This frequency was chosen, in part, because it results in the best test performance when DPOAEs are used in the clinic to distinguish between normal and impaired ears (e.g., Gorga et al, 1993a, 1997; Kim et al., 1996). Primary tones were presented at levels of L₁=65 dB SPL and L₂=55 dB SPL. The choice of these levels by Gorga et al. (1997) was based on previous data that demonstrated good test performance for these conditions (Stover et al., 1996). The selection of these DPOAE conditions simplified the analyses; however, it is expected that the approach would be equally useful if more complex DPOAE criteria, such as those derived from multivariate analyses, were used.

Data were divided into hearing-loss categories as follows: normal (-5 to 15 dB HL), mild (16 to 40 dB HL), moderate (41 to 60 dB HL), severe (61–90 dB HL) and profound (90 dB HL or greater). These category definitions are the same as those used by the CDC in its estimates of prevalence in relation to magnitude of hearing loss. Unilateral losses were categorized as mild hearing loss regardless of severity based on the assumption that benefits and costs for a unilateral hearing loss would be similar to those associated with a mild hearing loss. Cumulative distributions of DPOAE level (for f₂ = 4 kHz and L₁/L₂ = 65/55 dB SPL) within each of the above categories are shown in Fig. 1. The ordinate in Fig. 1 indicates the probability that the DPOAE level was at or below the value indicated on the abscissa. The probability of identifying an infant with hearing loss increases as DPOAE level increases, as expected.

Figure 1.

Cumulative probability as a function of DPOAE level at 4 kHz. The parameter n this figure is degree of hearing loss, as described in the text.

CDC estimates of the prevalence of congenital hearing loss identified in 2004 were applied to DPOAE data to calculate a weighted estimate of the prevalence of DPOAE responses in the general population. Based on current prevalence estimates, it was assumed that the overall incidence of hearing loss in newborns was approximately 2 per 1000 births. This may underestimate the actual prevalence of hearing impairment in newborns as current screening protocols do not typically identify all cases of mild and unilateral hearing loss (Gravel et al., 2005). Of the subset of reported data available to us, 49%, 25%, 13%, and 13% were diagnosed with mild or unilateral, moderate, severe, and profound hearing loss, respectively.

RESULTS

A. Benefit-Cost Ratio

Total cumulative lifetime monetary benefit was calculated for each category of hearing loss relative to DPOAE level using estimated benefit values as previously discussed. Other benefit estimates may be substituted in this formula. Lifetime monetary benefit for all severities of hearing loss was calculated by multiplying the maximum possible benefit for each hearing-loss category (B_mild, B_mod, B_sev, B_pro) by the proportion of ears that produced a DPOAE level less than a specified level for that category (P_mild, P_mod, P_sev, P_pro). National estimated lifetime monetary benefit relative to prevalence data are shown in Fig. 2. This figure illustrates the effect that prevalence has on estimated benefit. Though we expect less estimated monetary benefit per individual for the identification of a mild or moderate hearing loss compared to severe hearing loss, there is a higher prevalence of mild or moderate hearing loss in our data set compared to severe hearing loss. As indicated in Fig. 2, this results in less monetary benefit for severe hearing loss compared to mild and moderate hearing loss. To obtain an estimate of total benefit, the values for each hearing-loss category were summed.

Figure 2.

Estimated cumulative lifetime monetary benefit as a function of DPOAE level at 4 kHz. The parameter in this figure is degree of hearing loss, as described in the text. These lifetime monetary benefits take into account prevalence estimates for 37 states and territories in U.S. in 2004 as reported by the CDC (n=2,073,969).

$$\text{Total}\text{Benefit}={\text{P}}_{\text{mild}}·{\text{B}}_{\text{mild}}+{\text{P}}_{mod}·{\text{B}}_{mod}+{\text{P}}_{\text{sev}}·{\text{B}}_{\text{sev}}+{\text{P}}_{\text{pro}}·{\text{B}}_{\text{pro}}$$

As the probability of identifying an infant with hearing impairment increases, so does the benefit. As criteria become more stringent, the specificity decreases. This is equivalent to making it more difficult to pass the test. There are negative consequences to be considered with decreased specificity, including family anxiety, decreased confidence in the screening test, and increased costs relative to additional testing. This outcome is inevitable for any test in which the distributions of responses from normal and impaired populations overlap. In the context of the present analyses, follow-up costs vary with referral rate, which depends on the DPOAE criterion used during the NHS test. As a result, lifetime monetary benefit grows in conjunction with increasing probability of identifying hearing loss, as does programmatic cost.

Initial estimated costs for a UNHS program were calculated considering reported population prevalence data relative to estimated equipment costs, costs of disposable ear tips, and the salary and benefits of the staff as previously described. Other cost estimates may be substituted in this formula. In the equation below, N = the total number of infants screened.

$$\text{Initial}\text{Screening}\text{Cost}=(\text{N}/4000)·\$6500+\text{N}·\$1+(\text{N}/4)·\$20$$

The total cost of a UNHS program also includes the cost for follow-up, which depends on the proportion that fails the initial screening test. The proportion that fails the initial NHS test is calculated as a weighted average of the proportion with DPOAE level less than a specified level in each hearing-loss category (P_mild, P_mod, P_sev, P_pro). The categories are weighted by their expected prevalence (E_norm E_mild E_mod, E_sev, E_pro).

$${\text{P}}_{\text{wa}}={\text{P}}_{\text{norm}}·{\text{E}}_{\text{norm}}+{\text{P}}_{\text{mild}}·{\text{E}}_{\text{mild}}+{\text{P}}_{mod}·{\text{E}}_{mod}+{\text{P}}_{\text{sev}}·{\text{E}}_{\text{sev}}+{\text{P}}_{\text{pro}}·{\text{E}}_{\text{pro}}$$

The weighted-average proportion was essentially equal to the proportion of the normal category due to the prevalence of normal hearing. We estimated the cost of follow-up diagnostic ABR testing as $400 per individual. Overall costs, including the cost of follow-up diagnostic ABR testing, can be expressed relative to DPOAE level as follows:

$$\text{Cost}=\text{Initial}\text{Screening}\text{Cost}·\$400·{\text{P}}_{\text{wa}}·\text{N}$$

In the equation above, N = the total number of babies screened. Estimates of national benefit and costs of a UNHS program for congenital hearing loss identified in 2004 is shown in Fig. 3. These results are also shown as a benefit-cost difference or net value (Fig. 4). In addition to illustrating that benefit almost always exceeds cost for estimates used in our calculation, a calculation of net value is yet another measure which can be utilized when analyzing cost and benefit for the purpose of determining UNHS criteria. For the current data set, maximum net value is obtained for a DPOAE level of −7 dB SPL. Recall, however, that this specific DPOAE level would not necessarily be appropriate for neonates, given the differences (noted above) in responses properties between them and the subject group used in this application.

Figure 3.

Cumulative benefit and cumulative cost as a function of DPOAE level at 4 kHz for a high benefit and low cost estimate of UNHS weighted in relation to prevalence estimates for 37 states and territories in U.S. in 2004 as reported by the CDC (n=2,073,969).

Figure 4.

Net value as a function of DPOAE level at 4 kHz for a high benefit and low cost estimate of UNHS weighted in relation to prevalence estimates for 37 states and territories in U.S. in 2004 as reported by the CDC (n=2,073,969).

BCR was calculated for DPOAE criteria from −60 to 30 dB SPL. BCR for a given DPOAE level is calculated by the following formula:

$$\text{BCR}=\text{Total}\text{Benefit}/\text{Total}\text{Cost}.$$

BCR data for several cost and benefit estimates are shown in Fig. 5. Initial estimates of cost and benefit, which were described above, were used to estimate BCR when benefits were high and costs were low. Using these estimates indicate that a DPOAE level of −15 dB SPL¹ yields maximal BCR, which is equal to lifetime benefits that are 25 times greater than the costs of the UNHS program. This value differs from the value at which maximum net value is obtained and these differences may need to be considered when using a quantitative approach to choose screening criteria. To obtain a more descriptive analysis of screening criteria, these methods can be applied concurrently. For example, maximum net value is equal to lifetime benefits that are 11 times greater than costs of the UNHS program. Use of either maximum BCR or maximum net value for this data set would result in missing some ears with hearing loss. These missed hearing losses would have implications on personal, familial, and societal costs and must be considered when determining screening criteria. It may be the case that using test criteria which utilizes maximum BCR (or any other metric that stresses maximum monetary value) is not preferred. It is important to keep in mind that the criterion value maximizing BCR from the present data cannot be applied to a newborn population. Data from Gorga et al. (2000) and Abdala et al. (2008) show the average DPOAE level around 4 kHz to be approximately 5–9 dB SPL. These levels are well above the DPOAE level required for maximum BCR for the present data set. The purpose of this methodological analysis is not to suggest optimal screening criteria, but to demonstrate a method which considers quantitative values associated with various outcomes of screening criteria.

Figure 5.

BCR as a function of DPOAE level at 4 kHz. The parameter in this figure is the estimate used for benefit and for cost.

B. Influence of Adjustments to Costs and Benefits

In addition to our initial estimates, influence of three alternate estimates of cost and benefit are shown in Fig. 5. As a reference point, note that for each estimate, maximum BCR is obtained for a DPOAE level of −15 dB SPL. According to our analysis of the present DPOAE data set obtained on children and adults, the overall referral rate corresponding to this DPOAE level is 1%. This value is lower than expected, but it is important to recognize that it was based on BCR, not test performance. The criterion DPOAE of −15 dB SPL, which maximized BCR, would result in a low referral rate, but also would miss some ears with hearing loss. It is for this reason that selecting criterion values that are not at the maximum BCR may be more appropriate for clinical use.

Estimated referral rates from UNHS programs range between approximately 2–12% (Cox and Toro, 2001; Lin et al., 2003; Owen, Webb, and Evans, 2001; White, 1997). In addition to the effect of maximizing BCR (described above), differences in our calculated referral rate and published estimates of referral rates may be due to differences in the population being tested in each instance. Failure rates for OAE screening in a newborn population are highest during the first 24 hours after birth and decrease as a function of time in days (Kok et al., 1993; Welch et al., 1996). This may be the result of partial obstruction due to vernix, decreased mobility of the tympanic membrane, and/or increased noise levels in the neonatal population. The optimal time to screen a newborn may be 3–4 days after birth (Kok et al, 1993); however, the average hospital stay in the United States following birth is 2.1 days (Kozak et al., 2001). Our result represents a low referral rate because it was based on maximum BCR and the data on which it was based were obtained under different conditions, in a different population. Our calculations also show that the referral rate for mild, moderate, severe, and profound hearing loss at this criterion level is 0.22, 0.81, 0.97, and 0.97, respectively. This illustrates that a significant proportion of mild and moderate hearing loss would be missed by choosing this criteria. Thus, the criterion that results in maximum BCR may not be optimal under clinical conditions; however, the analysis provides information regarding the implications of various criterion levels that may be chosen for use in UNHS programs. It also provides the information that would be needed if one wished to find a compromise between monetary benefit and test performance (sensitivity and specificity) to use clinically. Assuming that current protocols base the selection of test criteria on test performance (something that may not be the case), they still lack the information that would be needed to determine financial benefit associated with specific test performance characteristics.

Returning to BCR when different values are assigned to benefit and cost, expected outcomes are observed. In the case when benefit was estimated to be half of our initial estimate, BCR was approximately half of the original estimate. Even so, benefit was greater than cost by a factor of 12 for initial low-cost program estimates. Even for a high-cost program coupled with low benefit, overall lifetime benefit is eight times greater than cost when maximum BCR is considered. It should be noted that, for at least DPOAE criterion levels at 4 kHz, the point at which maximum BCR is obtained does not vary when alternate estimates of cost and benefit are used. This means that for a given data set, pass/refer criteria based on BCR should be relatively insensitive to the specific values assigned to the underlying costs and benefits. For the reasons described above, the specific value most appropriate for neonates might vary, but the trends should be the same. Unfortunately, we are unaware of test performance data from neonates that would allow us to determine the specific criterion that would maximize BCR in that population.

C. Influence of a Secondary Screening Test

To reduce referral rates from DPOAE screening, many UNHS programs include a secondary screening test such as AABR. The influence of a secondary screening test on BCR was modeled by estimating referral rates for the AABR screening test in each of the 5 hearing sensitivity categories. A new set of 5 reduced cumulative probabilities was calculated by multiplying the cumulative probabilities shown in Fig. 1 by the referral rates in Table 2. These probabilities were used in conjunction with one-step UNHS program costs and the additional costs of AABR equipment and supplies. Costs for AABR screening equipment were approximated at $17,000 per unit (Gorga et al., 2001; Vohr et al., 2001), though higher estimates have been reported ($20,791, Keren et al., 2002). Earmuffs for AABR units typically cost $10 per infant screened. Other cost estimates were the same as those used for a one-step program (e.g. one piece of equipment would be needed for every 4000 infants, staff pay would be $20 per hour, and follow-up diagnostic ABR testing would cost equaled $400). The resulting maximum BCR occurs at −12 dB SPL (Fig. 6) and yields an overall referral rate of 0.001 (1/1000). In addition to previously discussed differences in our data set and the newborn population, the surprisingly low overall referral rate of the two-step screening at the DPOAE level (−12 dB SPL) associated with maximum BCR may reflect unrealistic assumptions about AABR test performance. It is important to remember that these values are based on maximizing the BCR, and not on achieving some pre-determined referral rate. Though this referral rate is influenced by the low prevalence of congenital hearing impairment (1-2/1000), it may have also been influenced by the use of population estimates rather than an exhaustive data set. According to our analysis, the referral rate for mild, moderate, severe, and profound hearing loss at a criterion level of −12 dB SPL is 0.09, 0.79, 0.98, and 0.99 respectively. Again we see that use of maximum BCR might not provide an optimal UNHS criterion, but it provides information about the implication of our choices. Independent of the BCR, it is the case that the likelihood of missing an ear with hearing loss decreases as the magnitude of the loss increases. This is true regardless of the basis on which test criteria are selected. Use of a two-step UNHS program including secondary AABR screening provides greater BCR while influencing DPOAE criterion levels by only 3 dB.

Table 2.

Estimates of AABR referral rates. Other estimations may be applied when calculating BCR.

NH (-5-15)	Mild (16–40)	Moderate (41–60)	Severe (61–90)	Profound (>90)
3%	30%	90%	99%	100%

Figure 6.

BCR as a function of DPOAE level at 4 kHz for two different screening protocols. BCR for the one-step protocol is derived from the scenario in which benefit is high and cost is low (as shown in Fig. 5). BCR for a two-step protocol uses AABR as the second step and is based on the referral rates provided in Table 3.

D. Influence of Otitis Media

Subjects with otitis media were excluded from the sample on which the cumulative probabilities in Fig. 1 were based. In the case of UNHS, however, it is not possible to identify ears with middle-ear dysfunction, at least with technologies in current clinical use. For the purposes of the present demonstration, we modeled the influence of middle-ear effusion on BCR as a distribution of attenuations in DPOAE level that is independent of hearing-loss category. To calculate cumulative probabilities for the reduced DPOAE levels, the frequency distributions for the ears without middle ear effusion were convolved with an attenuation distribution which estimates a greater number of ears to have no attenuation than the number for ears with a maximum attenuation (arbitrarily estimated at 9 dB). The hypothetical distribution of DPOAE attenuations is shown in Fig. 7.

Figure 7.

Hypothetical proportion of ears affected by middle-ear effusion as a function of the attenuation (in dB) provided by the effusion.

The resulting frequency distributions were broader and shifted to the left, toward lower DPOAE levels (Fig. 8). Total benefit and total cost were calculated from these modified frequency distributions using the same procedure described above. As a reference, the resulting maximum BCR occurs at −20 dB SPL (Fig. 9) which is lower and shifted to the left in comparison to initial estimates of maximum BCR. From this analysis, it can be seen that the influence of middle-ear effusion is to decrease the DPOAE level that is needed to obtain maximum BCR.

Figure 8.

Cumulative probability as a function of DPOAE level at 4 kHz for ears with and without middle-ear effusion. The parameter in this figure is status of the middle ear.

Figure 9.

BCR as a function of DPOAE level at 4 kHz for ears with and without middle-ear effusion. BCR for the ears without middle-ear effusion is derived from the scenario in which benefit is high and cost is low (as shown in Fig. 5). BCR for ears with middle-ear effusion uses attenuations (in dB) based on the hypothetical proportions described in Fig. 7.

DISCUSSION

Our calculations indicate that for none of the above scenarios do programmatic costs outweigh projected estimates of benefit, indicating that UNHS is a worthwhile investment for society. The approach we used suggests that benefits outweigh costs by as much as a factor of 25:1 when high benefit and low costs are assumed. Importantly, this model illustrates that a range of cost and benefit estimates does not have an effect on the DPOAE level required to maximize BCR. The addition of AABR screening to the NHS process increased maximum BCR for by a factor of two; however, the point at which maximum BCR is obtained was changed by only 3 dB. Maximum BCR is altered more by an attenuation of DPOAE level, as might be the case in the presence of middle-ear effusion. Middle-ear effusion can attenuate DPOAE level and, as a consequence, plots of cumulative distributions of DPOAE level in ears with middle-ear effusion are lower than the distributions for ears that have no effusion. The resulting BCR distribution is shifted toward lower DPOAE levels as well, thus shifting the point at which maximum BCR is obtained.

BCR takes into account monetary benefits and compares them to cost. In this way, BCR provides weight to each hearing-loss category and may be used to guide decisions for screening criteria in a manner that considers factors other than test sensitivity and specificity. It is one of several potential approaches that can be used to incorporate monetary and test-performance factors in the decision-making process. Calculation of net value is an alternative measure that can be used to analyze cost and benefit for the purpose of determining DPOAE screening criteria for UNHS. According to the present analysis, BCR suggests that a DPOAE level of −15 dB SPL at 4 kHz would be most cost-effective. Maximum net value is obtained at a DPOAE level of −7 dB SPL at 4 kHz. Conclusions regarding criterion levels required to obtain maximum benefit vary based on the chosen method; however, when used together these methods can provide a more descriptive analysis of screening criteria. For example, at the DPOAE criterion level for which maximum net value is obtained, benefits outweigh costs by 11:1. We do not wish to imply that either of these values (−15 dB SPL or −7 dB SPL) be used during neonatal screening for the following reason. While these criteria maximize BCR and net value, they also would reduce the number of ears with hearing loss identified at birth, which would impact personal, familial, and societal costs and should be considered when determining screening criteria. A lower BCR may be acceptable in order to identify more infants with hearing loss, yet still produce overall monetary benefits to society. The strength of methods described in this manuscript is that they provide a framework that would enable society to select criteria that would achieve some desired balance between number of infants identified and financial benefit and cost.

In addition to choosing DPOAE criterion level, this method can also be used to analyze decisions about screening programs including the influence of secondary screening, such as ABR, the influence of middle-ear effusion, and though not analyzed in this paper, BCR can be used to analyze other screening parameters such as DPOAE stimulus conditions and multivariate approaches to NHS. These methods can also be employed for populations in which the prevalence of hearing loss varies, as would be expected for babies in neonatal intensive care units compared to those in well baby nurseries.

One limitation of the present approach is that it does not address psychosocial development. Though it was not considered in our estimates of benefit, psychosocial development in children with hearing impairment may be at risk due to delayed skills in communication, limited access to linguistic discourse, and environmental effects such as noise and reverberation (Moeller, 2007). Like other considerations in this paper, psychosocial effects may vary with degree of hearing loss. Bess, Dodd-Murphy, and Parker (1998) reported a higher incidence of behavioral and self-esteem issues among students with minimal hearing loss who were not using amplification. Hind and Davis (2000) found that families of children with moderate hearing impairment reported less impact on their quality of life, compared to families of children with severe and profound hearing loss. Perhaps unexpectedly, Yoshinaga-Itano and deUzcategui (2001) showed that children who were late-identified with mild hearing loss had poorer social skills than children who were late identified with moderate-to-profound hearing loss. This paradoxical effect may be due to the tendency to underestimate the needs of children with mild hearing loss. Psychological effects can vary with age at identification and intervention. Wake et al. (2004) examined outcomes in children with hearing impairment who did not have access to early identification, and found that they differed significantly from children with normal hearing on a measure of psychosocial health-related quality of life. While it may be difficult to assign monetary value to these psychological issues, there is little doubt that there may be psychological costs when children with hearing loss go un-served or under-served early in life. These factors likely provide additional incentives for UNHS.

In addition, this paper does not consider the expected age at identification of hearing loss in the absence of UNHS. It is likely that severe-to-profound hearing loss will be detected earlier, compared to losses of lesser degrees and/or configurations in which frequency regions of normal hearing exist. Thus, estimating societal costs of late identification of hearing loss, taking only magnitude of hearing loss into account, may be overly simplistic and misleading. A more accurate societal estimate of cost might also need to factor in the likelihood that the timely detection of hearing loss might depend on its magnitude. Unfortunately, we are unaware of data that might be brought to bear on this issue.

Admittedly, the benefit and costs we chose were estimates. A more accurate BCR can be derived if specific costs of programs are tracked and implemented in the calculations. The calculation of the monetary benefit of early identification would be improved if the yearly cost of services for children and adults with hearing loss were gathered across the lifespan. At the very least, however, BCR provides an alternative approach to the one in common use that is presumably based on expected sensitivity and/or specificity, but does not take into account benefits, costs, or prevalence. This methodological demonstration provides a framework which can be used by audiologists, administrators, and others involved in establishing criteria for use in NHS settings. While we emphasize that the approach followed above is based on several speculations, even for our estimates of a low-benefit program that has high costs, BCR is greater than 1. That is, for all of our estimates, benefit outweighs cost. This provides additional support for the societal value of UNHS.

¹ It is useful to note that the present set of data can be used in an outpatient clinic serving patients 1.3 years of age and older. It is unlikely that clinics are using −15 dB SPL even for a patient group similar to the one used here. These results suggest that, even in older subjects, BCR might be greater if a less stringent criterion was used. It may be reasonable to expect a similar trend among neonates, even though the specific criterion value is not known at this time.