The Effects of Meaning-Based Auditory Training on Behavioral Measures of Perceptual Effort in Individuals with Impaired Hearing

Abstract

There has been considerable interest in measuring the perceptual effort required to understand speech, as well as to identify factors that might reduce such effort. In the current study, we investigated whether, in addition to improving speech intelligibility, auditory training also could reduce perceptual or listening effort. Perceptual effort was assessed using a modified version of the n-back memory task in which participants heard lists of words presented without background noise and were asked to continually update their memory of the three most recently presented words. Perceptual effort was indexed by memory for items in the three-back position immediately before, immediately after, and 3 months after participants completed the Computerized Learning Exercises for Aural Rehabilitation (clEAR), a 12-session computerized auditory training program. Immediate posttraining measures of perceptual effort indicated that participants could remember approximately one additional word compared to pretraining. Moreover, some training gains were retained at the 3-month follow-up, as indicated by significantly greater recall for the three-back item at the 3-month measurement than at pretest. There was a small but significant correlation between gains in intelligibility and gains in perceptual effort. The findings are discussed within the framework of a limited-capacity speech perception system.

Learning Outcomes: As a result of this activity, the participant will be able to (1) describe whether auditory training can reduce the effort associated with understanding speech in individuals with impaired hearing, and (2) describe whether improvements in speech intelligibility resulting from auditory training are related to changes in perceptual effort

Adult-onset hearing loss is among the most prevalent and burdensome conditions worldwide, especially for individuals over age 40. By age 65, ∼30% of individuals will have a hearing loss significant enough to qualify for a hearing aid.¹ The World Health Organization estimates that by 2030, hearing loss will be among the top 10 most burdensome conditions, ranking ahead of diabetes in terms of overall burden.² Hearing loss also is associated with negative social outcomes, including increased depression, increased employment difficulties, and overall reductions in quality of life.³

The most common treatment for adult-onset hearing loss is some form of amplification, generally in the form of a hearing aid. However, one of the most frequent complaints of individuals with hearing loss is not that they cannot detect sound, but rather that they cannot understand what is being said. Thus, although well-fit hearing aids have been shown to provide long-term benefits, they rarely return spoken communication to preclinical abilities. Consistent with relatively low satisfaction rates with hearing aids, Humes et al reported that only ∼20% of individuals in the United States who would be candidates for hearing aids actually obtain them and, of these, only ∼50% use them on a regular basis.⁴

One rehabilitative intervention that can be used either in conjunction with sensory aids or as an alternative to such aids is auditory training. Auditory training is designed to improve speech intelligibility by developing perceptual skills such as phoneme discrimination, stream segregation, and word identification that are critical for understanding speech in less than ideal listening environments. Although individual auditory training programs differ considerably in terms of specific activities and training durations, most focus on improving perceptual or listening skills through analytic and synthetic training procedures. As was the case for hearing aids, however, auditory training often produces only small improvements in speech intelligibility, especially in noisy or reverberant environments.⁵ Moreover, clinician-delivered auditory training is extremely time and resource demanding. Recent attempts to reduce this burden by using computer-based auditory training have thus far not produced large generalizable gains in speech-in-noise perception.⁶

Despite the relatively limited gains in speech intelligibility from auditory training, especially for items that were not specifically included in training protocols, compliance rates and overall satisfaction with auditory training programs can be quite high. For example, Tye-Murray et al reported over 95% compliance for 100 participants who were trained on the Computerized Learning Exercises for Aural Rehabilitation (clEAR; formerly I Hear What You Mean), a computerized auditory training program.⁷ Moreover, participants self-reported significant gains in benefit and confidence as a result of completing the program. One possible explanation for the generally positive reports from participants in auditory training programs despite somewhat limited gains in intelligibility is that training reduced the perceptual effort needed to understand spoken language.

The concept of perceptual effort or listening effort is based on a limited-capacity resource model in which current cognitive operations engage a given percentage of total cognitive capacity.⁸ Increasing task demands, such as increasing background noise or reverberation, will require additional resources and will therefore increase the overall perceptual effort required for successful speech perception. Several researchers have reported that, in addition to reduced speech intelligibility, individuals with hearing loss often report increased effort and fatigue, particularly when listening in noise, even when changes in the listening environment do not produce changes in overall performance.^{9 10 11 12 13} Within the framework of perceptual effort, these findings suggest that when listening conditions become more difficult, individuals can maintain a given level of performance by increasing the overall percentage of resources or effort that are engaged to complete a task.

Of particular importance to the current study, the link between perceptual effort and hearing loss also suggests that reductions in perceptual effort may increase the availability of cognitive resources and thereby improve listeners' ability to successfully complete other ongoing activities. Rabbit was one of the first to test what has become known as the effortfulness hypothesis—the idea that differences in the difficulty or effortfulness of initial encoding can have downstream consequences for ongoing cognitive functions.¹⁴ In one study, for example, normal-hearing young adults were presented with lists of digits either in a quiet background or in the presence of noise. Participants in both groups were able to shadow the digits perfectly, but those presented with the stimuli in a background noise showed significantly lower memory for the digits on a subsequent recall test. Rabbit attributed this difference in recall performance to the increased effort needed to encode words presented in noise compared to those heard in quiet. This additional effort, according to Rabbit, reduced available resources for rehearsal and other processing that would support memory of the items and therefore recall in the noise condition was lower despite equivalent and nearly perfect intelligibility.

More recently, McCoy et al reported a similar pattern of results using a modified version of the n-back task.⁹ They tested two groups of older adults differing in the extent of age-related hearing loss on a task in which highly familiar words were presented individually without background noise. List presentation was stopped randomly and participants were asked to recall the last three words that were presented before the list was stopped. Both groups of older adults were able to recall the most recently presented word nearly perfectly, suggesting that both groups were able to encode the items. Group differences were observed, however, in recall of the three-back word (i.e., the one presented least recently), with those in the group having greater amounts of hearing loss demonstrating poorer three-back recall than those with better hearing. McCoy et al argued that this difference could be attributed to greater effort at encoding for the group with more impaired hearing, reducing resources available to update and rehearse the three most recently presented words.⁹

The use of perceptual effort as a complement to more traditional measures of speech-in-noise identification has led other investigators to establish factors that may modulate perceptual effort,^{9 14 15 16 17 18 19 20 21} including signal-to-noise ratio (SNR),^{20 21 22} visual speech information,^{15 18 19} hearing thresholds,^{9 19 23} and type of masker.^{16 17} Although these investigations include a range of methodologies for assessing perceptual effort, the general finding is that factors that function to increase phoneme and word discrimination or to ease lexical access can significantly reduce listening effort even under conditions in which speech intelligibility is unchanged.

As noted, auditory training is one intervention that is specifically targeted at improving lexical discrimination and therefore is an ideal candidate to examine with respect to its effects on perceptual effort. To our knowledge, only one previous study has examined changes in both intelligibility and effort as a consequence of auditory training.²⁴ Participants were trained over the course of 20 sessions lasting ∼90 minutes each to recognize the 400 most frequent words in English.^{25 26 27} Changes in perceptual effort were indexed by pupillometry using an orthographic version of the visual world paradigm.^{20 21} Both word identification and reaction times to the training stimuli were improved significantly at post- compared with pretraining time points. The training also resulted in reduced perceptual effort as indicated by a larger and faster peaking pupillary response compared to a passive control group. Of particular interest with respect to clinical applications, Kuchinsky et al reported that perceptual effort was reduced even for those words that were identified accurately at both pre- and posttest.²⁴ This last finding suggests that changes in perceptual effort can be observed even when intelligibility is maintained.¹⁷

The current study was designed to expand on the findings from Kuchinsky et al in several ways.²⁴ First, we wanted to examine changes in perceptual effort following auditory training that included materials at several different psycholinguistic levels, including phoneme, word, sentence, and discourse. Second, we wanted to measure changes in effort behaviorally, as a complement to the psychophysiological measures used by Kuchinsky et al.²⁴ We therefore adapted a version of the n-back task used by McCoy et al as an index of changes in effort.⁹ Finally, we wanted to take advantage of our relatively large sample size to establish whether individual differences in the benefits of training would correlate with improvements in perceptual effort.

Methods

Participants

A total of 83 (45 male) participants completed training and both the pre- and posttests. All participants were current hearing-aid users. These participants were part of a larger study about the efficacy of auditory training. Our previous reports indicated that auditory training led them to have better speech discrimination and better perceived listening performance.⁷ Forty-two of the participants completed single-talker (ST) training in which they heard the same talker during the entire course of training. Forty participants completed multiple-talker (MT) training in which they heard six different talkers (three male and three female talkers) during each of the training exercises. The specific talker that was used for the ST training was counterbalanced such that approximately equal numbers of participants in the ST condition heard one of the six talkers used in the MT training as the talker during all training exercises. Table 1 displays means and standard deviations for demographic measures obtained on participants in the ST and MT groups. The two groups did not differ significantly on age (p = 0.51) or pure tone average (PTA) (p = 0.91). Participants in the ST group had significantly more years of education than those in the MT group (t[82] = 1.9, p < 0.05). In addition, participants in the ST group had a significantly longer duration of hearing loss than those in the MT group (t[82] =2.2, p < 0.05).

Table 1. ST and MT Training Groups on Demographic Measures.

Variable	ST, mean (SD)	MT, mean (SD)
Age	68.5 (13.8)	66.3 (18.2)
Better PTA	48.8 (16.9)	51.0 (14.1)
Education (y)	16.3 (3.3)	15.1 (2.3)
Years of HL	15.1 (10.3)	22.6 (18.8)

Abbreviations: HL, hearing level; MT, multiple-talker; PTA, pure tone average; ST, single-talker; SD, standard deviation.

Auditory Training Program

All participants completed 12 training sessions on the clEAR training program. Each training session included five activities ranging from primarily analytic (phoneme discrimination) to synthetic (comprehension of short paragraphs). The 12 training sessions each focused on a particular theme (e.g., family, restaurant, vacation) and each lasted approximately 1 hour. The five training activities were:

• Activity 1: Introduction of lesson topic. Participants were asked to identify the position of phonemes in consonant–vowel–consonant words.

• Activity 2: Meaning-based four-choice phoneme discrimination. Participants heard two words presented (e.g., bat, mat) and were then shown four pictures simultaneously (a bat and a bat, a mat and a mat, a mat and a bat, and a bat and a mat).

• Activity 3: Sentence completion. Participants were asked to select among words to complete a meaningful sentence.

• Activity 4: Sentence comprehension. Participants were asked to select a sentence that would most likely follow one they heard.

• Activity 5: Paragraph comprehension. Participants heard a paragraph and answered comprehension questions.

All training was conducted in a sound-attenuated booth in the presence of four-talker babble. SNR was adjusted adaptively to maintain ∼80% correct on each activity.

Measure of Perceptual Effort

Perceptual effort was assessed using a version of the n-back task developed by McCoy et al and was included as part of an extensive battery of testing conducted prior to and immediately following auditory training.⁹ Participants heard eight lists of highly familiar English words presented without background noise. The talker who produced the words for the effort test had not been heard previously by any of the participants. The lists contained varying numbers of words (10, 7, 13, 5, 12, 15, 8, and 14 for lists 1 to 8, respectively). Following the last word in each list, the participant saw “Please repeat the last three words” on the computer screen. Participants were unaware of the number of items in each list and were therefore required to update the last three items in memory after each word was presented.

Results

We first conducted a three-way mixed-design analysis of variance (ANOVA) to determine if recall performance on either the pretest or posttest for any of the three positions (one, two, or three words back) differed for those in the ST versus MT training groups. Training condition (ST versus MT) was a between participants factor and word position (one, two, or three words back) and testing time (pretest, posttest, 3-month follow-up) were repeated-measures variables. There was no main effect of training group (F [1, 80] = 1.2, p > 0.2, η_p ² = 0.01), and the training group did not enter into any significant interactions (all F's< 1). For the remaining results, we therefore combined data from the MT and ST training groups.

Fig. 1 displays the mean number of words recalled for each of the three recall positions. As in McCoy et al, recall of the most recent word (one-back position) was used as a measure of how well participants could hear and encode the words.⁹ Mean recall for the first position back exceeded seven (out of eight) for both the pre- and posttests, suggesting that participants had little difficulty hearing and encoding the words. The data were analyzed using a two-way repeated measures analysis with test time (pre- versus posttest) and position back (one, two, or three words back) as variables. As expected, recall performance decreased across the three positions with the poorest recall performance for the three-back word condition (F [2, 162] = 205.4, p < 0.001, η_p ² = 0.6). Overall recall performance was higher for the posttest than for the pretest (F [1, 81] = 23.7, p < 0.001, η_p ² = 0.23). Of particular importance, however, was that differences between pre- and posttest performance differed across the three positions (F [2, 162] = 9.3, p < 0.001, η_p ² = 0.11). To determine the source of this interaction, we conducted planned pairwise comparisons on differences in word recall (between posttraining and pretraining) across the three word positions. The pre- to posttraining change in word recall was not different for either word position 1 or word position 2 (p > 0.2 for both comparisons). However, recall of words in the three-back position, was significantly higher posttraining than pretraining (p < 0.01).

Figure 1.

Mean number of words recalled for the pretest and posttest as a function of recall position. Error bars represent standard errors of the mean.

Three-month Follow-up

Seventy of the original 83 participants returned for a follow-up testing session 3 months after completing the initial posttest. Participants received the same posttest measures in the follow-up as they did in the immediate posttest, and none of the participants engaged in auditory training exercises between initial posttest and 3-month follow-up. Fig. 2 displays the original pre-and posttest data (as displayed in Fig. 1) along with the results for the 3-month follow-up. To determine whether the gains in effort made immediately following training were maintained in the 3-month follow-up interval, as assessed by word recall in the three-back position, we conducted a one-way repeated-measures ANOVA with testing time (pretest, posttest, 3-month follow-up) as the repeated-measures variable. Recall performance differed significantly across the three time points (F [2, 138] =14.4, p < .001, η_p ² = 0.17). Pairwise comparisons with a Bonferroni correction for multiple comparisons indicated that recall performance in the 3-month follow-up (mean = 4.4) was significantly lower (p < 0.01) than in the immediate posttest (mean = 5.3), but was still significantly better (p < 0.001) than the pretest performance (mean = 3.7). These findings suggest that participants lost some, but not all of the gains in perceptual effort from the immediate to the 3-month posttest and that some benefits of auditory training on perceptual effort remained even after a 3-month interval.

Figure 2.

Mean number of words recalled for the pretest and posttest as a function of recall position. Results from the 3-month follow-up are shown in the gray bars. Error bars represent standard errors of the mean.

Relationship between Training Gains and Changes in Effort

To examine whether there was a relationship between the benefits of auditory training and reductions in perceptual effort (i.e., whether those individuals who had the largest gains in training also exhibited the largest reductions in perceptual effort), we correlated differences between pre- and posttraining scores for the four-alternative forced choice test with pre- and posttraining differences in word recall for the three-back word position. We elected to use the four-alternative forced choice task as an index of training gains because this measure showed the largest and most variable changes pre- to posttraining. Recall that in this test participants heard two words (e.g., mat, bat) and had to choose from among four pictures representing the correct words (in this case a picture of a mat on the left and a bat on the right). The Pearson product-moment correlation between the two measures was significant (r = 0.29, p < 0.002), suggesting a small but significant relationship between gains in speech intelligibility and reductions in perceptual effort.

Predictors of Changes in Perceptual Effort

To examine whether any of the demographic measures that we collected, age, better ear PTA, and duration of hearing loss were related to the changes in perceptual effort, we correlated each of these measures with pre- to posttraining changes in word recall for the three-back position. None of the correlations approached significance (all p > 0.8).

Discussion

The current findings suggest that meaning-based auditory training can not only improve speech recognition in noise but can also reduce the perceptual effort required to understand spoken language. We used changes in recall for the three-back position in a modified n-back task as an index of changes in perceptual effort. Participants who completed ∼12 hours of training on the clEAR program were able to remember slightly more than one additional item in the three-back position following training as compared with before training. A follow-up testing session found that participants maintained some gains in perceptual effort 3 months after the end of training. Finally, there was a small but significant relationship between improvements in speech-in-noise recognition and improvements in perceptual effort, with those showing the greatest improvements in recognition tending to show greater improvements in perceptual effort.

The present results extend the findings of Kuchinsky et al in several ways.²⁴ First, Kuchinsky et al used changes in pupil response as an index of perceptual effort, whereas the current study used a behavioral measure of changes in memory performance. The improved memory performance following auditory training is particularly important because speech perception almost always takes place in the context of other ongoing cognitive activities. Thus, the current findings suggest that auditory training may facilitate performance on everyday cognitive tasks (e.g., remembering a spoken phone number) by reducing the effort associated with initial encoding. Second, in contrast to Kuchinsky et al, who assessed changes in effort using the same words and, in some cases, at the same SNRs that were heard during training, none of the words that were presented during the current memory task served as targets during auditory training. These findings suggest that changes in perceptual effort are not necessarily limited to items that were specifically trained, but can also generalize to novel material.

How then might we account for the reduced perceptual effort observed both in the current study and in Kuchinsky et al following auditory training?²⁴ One model that provides a useful framework for addressing this question is the Ease of Language Understanding (ELU).²⁸ Briefly, in the ELU model when speech is presented under good conditions, such as normal-hearing adults listening under favorable SNRs, matching incoming signals with lexical representations stored in long-term memory is relatively automatic and requires minimal cognitive effort. However, when listening situations become more difficult, for example, when individuals have reduced auditory sensitivity or background noise becomes louder, individuals must engage explicit cognitive abilities such as working memory to match the degraded acoustic signal with the stored representations. Auditory training may function to lessen these additional cognitive demands by improving listeners' ability to match the degraded acoustic signals to stored representations. That is, within the ELU model auditory training may function to move individuals from the more explicit pathway that engages several resource demanding cognitive abilities back to the more automatic pathway in which speech perception takes place automatically and with relatively little effort.

A related explanation for reduced perceptual effort following auditory training is based on the concept of cognitive spare capacity.^{29 30 31} Mishra et al suggested that when working memory or other cognitive resources must be engaged during speech perception as a consequence of either poor listening conditions or impaired hearing, there is reduced capacity for other ongoing operations.^{29 31} Factors that can reduce encoding demands and thereby free resources for working memory or other cognitive abilities will increase cognitive spare capacity and potentially reduce perceptual effort. For example, Mishra et al measured cognitive spare capacity in a group of older adults with mild to moderate hearing loss and found increased spare capacity for auditory-visual, compared with auditory-only presentations.²⁹ Mishra et al interpreted these findings as suggesting that the addition of visual speech information reduced demands on working memory by providing a complementary signal from which to obtain cues to phonetic, phonological, and lexical information.²⁹ If auditory training can similarly improve access to lexical representations, then the observed reductions in perceptual effort may be a consequence of increased spare cognitive capacity. Consistent with this explanation, Picou et al reported a moderate correlation between working memory capacity and subjective measures of perceptual effort.¹⁹

In contrast to the benefits of auditory-visual compared with auditory-only presentations reported by Mishra et al,²⁹ Anderson Gosselin and Gagné,¹⁵ using a dual-task methodology, reported increased perceptual effort for auditory-visual compared with auditory-only presentations in both older and younger adults. Currently, it remains unclear how methodological differences across the studies may have contributed to the discrepant findings.^{15 29} However, the conflicting results highlight the critical need for studies that use multiple measures of perceptual effort to establish the extent to which the different methodologies index the same underlying construct.

Clinical Implications

The ability to measure perceptual effort and how it changes with different audiological interventions could provide a powerful new tool for clinicians and for every day functioning in individuals with impaired hearing. Specifically, the current findings suggest that increased speech understanding alone may not always capture improvements in everyday communication. As noted, individuals with hearing loss can have difficulty maintaining engagement in a conversation even when they can understand everything that is being said because speech perception is fatiguing or effortful. The fatiguing effects are especially noticeable under difficult listening situations such as restaurants or parties where there is not only background noise but the background noise is other speech. If auditory training can reduce perceptual effort, then these individuals may be more likely to continue engagement in social activities and other functions that require accurate speech perception even if there is relatively little change in more traditional measures of speech perception. For example, Surprenant reported that improving SNR increased individuals' ability to remember words despite no changes in overall speech-in-noise recognition.³² Within a clinical setting, changes in effort could be used to adjudicate between different hearing aids or processing strategies even if the devices all have similar effects on speech perception. Consistent with this proposal, Humes reported that hearing aid outcomes are better characterized by a combination of improvements in speech intelligibility and reductions in effort than by any one single dimension.³³

Limitations

The principal limitation of the current study is the absence of a control group that did not receive auditory training. It is possible, for example, that the improvements in recall of words from the three-back position following auditory training was a consequence of participants becoming more familiar with the task and learning new strategies during the course of testing. One finding that argues against a strictly learning or experience-based account of reduced perceptual effort following auditory training is that training produced no significant improvement for words in the two-back position on the immediate posttest, despite scores that were lower than ceiling-level performance. Moreover, recall for the two-back position in the 3-month follow-up was nearly identical to that in the initial pre-and posttests. Nevertheless, it remains possible that learning or experience contributed to the improved performance for the three-back position, and future research should address this concern by including an appropriate control group.

Conclusions

The increased perceptual effort associated with impaired hearing can have profound effects on the social, financial, and emotional well-being of individuals. Interventions such as auditory training that can reduce the perceptual and cognitive demands associated with hearing impairment can therefore have a profound effect on the everyday communication abilities of individuals with impaired hearing. Measures of perceptual effort can therefore provide an important clinical tool in assessing the efficacy of different rehabilitative strategies in improving overall communication abilities of individuals with hearing impairment.