Child–Adult Differences in Using Dual-Task Paradigms to Measure Listening Effort

Abstract

Purpose

The purpose of the project was to investigate the effects modifying the secondary task in a dual-task paradigm to measure objective listening effort. To be specific, the complexity and depth of processing were increased relative to a simple secondary task.

Method

Three dual-task paradigms were developed for school-age children. The primary task was word recognition. The secondary task was a physical response to a visual probe (simple task), a physical response to a complex probe (increased complexity), or word categorization (increased depth of processing). Sixteen adults (22–32 years, M = 25.4) and 22 children (9–17 years, M = 13.2) were tested using the 3 paradigms in quiet and noise.

Results

For both groups, manipulations of the secondary task did not affect word recognition performance. For adults, increasing depth of processing increased the calculated effect of noise; however, for children, results with the deep secondary task were the least stable.

Conclusions

Manipulations of the secondary task differentially affected adults and children. Consistent with previous findings, increased depth of processing enhanced paradigm sensitivity for adults. However, younger participants were more likely to demonstrate the expected effects of noise on listening effort using a secondary task that did not require deep processing.

Listening in complex environments can be problematic for listeners of all ages, not only because listeners may not hear important speech segments but also because these complex environments may increase the mental effort necessary for a listener to understand speech. This additional mental effort, or listening effort, is often described as the cognitive resources necessary to understand speech (e.g., Hicks & Tharpe, 2002; Picou, Ricketts, & Hornsby, 2013). The consequences of increased listening effort for adults could be substantial, including effects in the short term (e.g., impaired recall [Rabbitt, 1968]) and in the long term (e.g., reduced physical well-being [Hua, Karlsson, Widén, Möller, & Lyxell, 2013]; mental fatigue [Hornsby, 2013]). For children who are still learning and developing, the consequences of increased listening effort may be especially important, particularly considering that typical classroom environments are notoriously acoustically disadvantaged (Crandell & Smaldino, 2000; Finitzo-Hieber & Tillman, 1978; C. E. Johnson, 2000; Nelson & Soli, 2000; Neuman, Wróblewski, Hajicek, & Rubinstein, 2010; Palmer, 1997). Perhaps as a result of the potential for severe, negative consequences of sustained increases in listening effort for adults and children, researchers and clinicians have shown interest in understanding, evaluating, and remediating listening effort (e.g., McGarrigle et al., 2014).

The model of Ease of Language Understanding (Rönnberg et al., 2013; Rönnberg, Rudner, Foo, & Lunner, 2008) provides a conceptual framework for listening effort. The model suggests that understanding language involves both implicit and explicit processing. Listeners implicitly, automatically, and rapidly bind language segments, such as phonemes, and compare these bound units to long-term memory stores. When there is an easy match between language input and long-term memory, speech recognition will be achieved with minimal effort exertion. However, in situations of a mismatch, such as when the signal is degraded, a listener must use explicit processing and additional cognitive resources to understand speech. Therefore, situations that degrade the speech signal are expected to increase listening effort (Rönnberg, 2003; Rönnberg et al., 2013, 2008).

In order to fully understand listening effort, it is critical to have valid and sensitive measures of listening effort. Because mental effort is not directly observable, all objective measures rely on inferences and assumptions about human cognition. Response time-based measures capitalize on limited human cognitive capacity (Kahneman, 1973). Responding quickly utilizes available cognitive resources; when response times are slowed, it is an indication that there are fewer cognitive resources available. Thus, increased listening effort is inferred on the basis of slowed responses to a speech stimulus (e.g., Gatehouse & Gordon, 1990; Gustafson, McCreery, Hoover, Kopun, & Stelmachowicz, 2014; Hornsby, 2013) or slowed responses during a secondary task (e.g., Downs, 1982; Hicks & Tharpe, 2002; Picou & Ricketts, 2014; Picou et al., 2013; Sarampalis, Kalluri, Edwards, & Hafter, 2009).

Dual-Task Paradigms

Investigators have used a variety of secondary tasks including simple secondary tasks, such as responding as quickly as possible to a simple visual probe (Downs, 1982; Hicks & Tharpe, 2002; Picou et al., 2013), and more complicated secondary tasks, such as vibrotacticle pattern recognition (Fraser, Gagné, Alepins, & Dubois, 2010; Gosselin & Gagné, 2011) or number recognition (Sarampalis et al., 2009). The secondary task must require cognitive resources in order to be indicative of excess or spare cognitive capacity. If a secondary task is too simple, it may become automatic and thus require minimal cognitive resources (Hasher & Zacks, 1979). Therefore, it might be assumed that tasks that are not complex enough may be less sensitive to changes in effort. However, if a task is too complex, it may negatively affect performance on the primary task, also invalidating it as a measure of effort.

Some investigators have explicitly evaluated the effects of manipulating secondary task complexity. It is important to note that complexity is not equivalent to difficulty. Increases in task complexity are achieved by increasing the number of actions, the number of decisions required, or the number of possible response options. Wu et al. (2014) tested adult listeners with hearing loss using two dual-task paradigms. The primary task was sentence recognition; the secondary task was either number recognition (traditional paradigm) or driving in a simulator (more complex, ecologically valid paradigm). The same listeners were tested with both paradigms and in several hearing aid conditions (unaided, aided with omnidirectional microphones, aided with directional microphones). Results revealed a similar pattern of results using both paradigms, suggesting increasing task complexity did not alter paradigm sensitivity to the conditions evaluated.

Similar findings were reported by Picou and Ricketts (2014), who evaluated the effects of increasing secondary task complexity for young adults with normal hearing and older adults with hearing loss. Picou and Ricketts tested participants using a simple paradigm, during which the secondary task was to press a single button in response to a simple visual probe. In addition, participants were evaluated using a paradigm with a more complex secondary task. However, both the simple and the complex paradigms were equally sensitive to the effects of noise. Thus, although there is theoretical reason to suspect that increasing secondary task complexity should increase paradigm sensitivity, reported manipulations have not affected sensitivity for adult listeners with normal or impaired hearing.

However, Picou and Ricketts (2014) found that paradigm sensitivity could be improved by increasing depth of processing of the secondary task. Depth of processing refers to the level of perceptual analysis required to perform a task (Craik & Lockhart, 1972; Eysenck & Eysenck, 1979). During tasks involving shallow processing, participants respond to a physical feature of a word; meaning extraction is not required. For example, participants could be asked to respond if the word presented contains the /b/ sound. The deepest level of processing requires meaning extraction, for example, asking participants to respond if the word heard is a member of a certain category, such as “something you wear.” Since the secondary task requires a single binary decision (e.g., yes or no, the word is a member of the target category), depth of processing can be manipulated without increasing complexity (e.g., choosing from more than two options).

Previous research suggests that increasing depth of processing improves memory performance (Duchek & Neely, 1989; Eysenck & Eysenck, 1979; Schulman, 1971; Tresselt & Mayzner, 1960), likely because stimulus elaboration allows for better signal encoding and thus longer lasting mental representation (Craik & Lockhart, 1972). In theory, in a dual-task paradigm, the effect of increasing depth of processing would be to increase the competition for language-specific resources and could increase paradigm sensitivity to factors that affect effort. Increasing the depth of processing might also increase a paradigm's ecological validity, because in everyday communication scenarios, listeners are more likely to be considering and elaborating on speech signals, rather than simply repeating speech while responding to random visual probes.

Listening Effort in Children

Despite the important implications for learning and development, fewer investigations have focused on listening effort in school-age children. Further, some of these reports have revealed somewhat surprising or unexpected findings, particularly when using dual-task paradigms. For example, Hicks and Tharpe (2002) used a dual-task paradigm similar to the “simple” paradigm used by Picou and Ricketts (2014). They tested 6- to 11-year-old children with normal and impaired hearing in quiet and at three signal-to-noise ratios (SNRs; +20, +15, +10 dB). Results revealed a significant effect of hearing status, but not SNR, suggesting that children with hearing loss experienced more listening effort than their peers, but adding or changing the level of background noise did not affect effort. Nonsignificant effects of noise have also been reported by Howard, Munro, and Plack (2010). These findings are inconsistent with published studies with adult participants (e.g., Desjardins & Doherty, 2014; Picou & Ricketts, 2014; Sarampalis et al., 2009).

There are several potential reasons for the different trends reported for children, including true adult–child differences in susceptibility to noise, dual-task paradigm complications, and insensitive paradigms. First, it is possible that children and adults are differentially sensitive to changes in background noise. However, there is substantial evidence to suggest that, for speech recognition, children are more susceptible to the negative effects of background noise than adults (e.g., C. E. Johnson, 2000; Markham & Hazan, 2004; Stelmachowicz, Hoover, Lewis, Kortekaas, & Pittman, 2000), and they are more likely to require better SNRs for similar levels of speech recognition performance (e.g., Hall, Grose, Buss, & Dev, 2002; Neuman et al., 2010). On the basis of these findings, one might expect that changing SNRs would lead to larger and more consistent changes in listening effort in children than adult listeners; instead, the opposite has been reported.

A second possible explanation is that the dual-task paradigms used were not optimal for evaluating listening effort. As mentioned previously, a critical assumption underlying the dual-task paradigms is that a secondary task does not interfere with the primary task. If a task is too complicated, a child may focus on the secondary task, letting speech recognition performance decline. Consistent with this limitation, in some published investigations, performance on the primary task was negatively affected by the addition of the secondary task. For example, the results of Howard et al. (2010) demonstrate a negative effect of adding the secondary task on speech recognition performance at the most difficult SNRs (0 and −4 dB). McFadden and Pittman (2008) similarly found that the introduction of a dot-to-dot game as a secondary task interfered with the primary task for children with minimal hearing loss, suggesting the secondary task was too engaging.

Last, it is possible that the sensitivity of the paradigms used previously could be improved. For example, Hicks and Tharpe (2002) used a simple dual-task paradigm and found nonsignificant effects of noise. However, for adults, Picou and Ricketts (2014) found that increasing the depth of processing improved paradigm sensitivity. Therefore, it is possible that modifying the secondary task could alter paradigm sensitivity for children. However, neither the complex nor the deep paradigm used by Picou and Ricketts is appropriate for school-age children.

To be specific, Picou and Ricketts (2014) increased secondary task complexity by presenting a visual number as a probe during word recognition. The adults were tasked with categorizing a number (even/odd) and then choosing from one of two response options (left/right facing arrow) on the basis of the number categorization. Although this task has been used for adults (Picou & Ricketts, 2014; Sarampalis et al., 2009; Wu et al., 2014), it might be too difficult for children. School-age children likely do not have adult-like mastery of numbers (e.g., Ashcraft, 1982; Groen & Parkman, 1972), and the national standards suggest that the concepts of odd and even be included in a second-grade curriculum (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010a). Therefore, children may not be introduced to the concepts of odd/even until the age of 7 or 8 years old.

Furthermore, Picou and Ricketts (2014) manipulated depth of processing for adult listeners by asking participants to judge whether the word presented was a noun. This task might also be difficult for children, particularly for younger children. According to the national curriculum standards, the concepts of nouns, pronouns, verbs, and adjectives are typically taught in third grade (National Governors Association Center for Best Practices, 2010b), or approximately 8 to 9 years of age.

Purpose

The purpose of this project was to evaluate the effect of increasing secondary task complexity and depth of processing in a dual-task paradigm for measuring objective listening effort by using secondary tasks that are appropriate for school-age children. A control group of adult listeners also participated to confirm the previously established effects of increasing complexity and depth of processing (Picou & Ricketts, 2014). Participants with normal hearing were tested in quiet and in noise using the same three dual-task paradigms (simple, increased complexity, increased depth of processing). It was expected that background noise would increase listening effort as measured using all three dual-task paradigms, but the task with increased depth of processing would be the most sensitive to the effects of noise for adults and children.

Method

Participants

Two groups of listeners participated: 16 adults (M = 25.4 years, SD = 3.3, range = 22–32; 15 women) and 22 children (M = 13.2 years, SD = 2.6, range = 9–17; 13 girls). An a priori power analysis on the basis of conservative estimates of effect sizes suggested that a sample size of 16 in each group would be sufficient to detect small effects (e.g., .25). However, additional children were included because it was expected that the performance of children would be more variable than adults. All participants had normal hearing (< 25 dB HL at audiometric octaves) and normal middle ear function at the time of testing as indicated by self-report. No participant exhibited otologic, cognitive, or neurogenic disorders, as evidenced by patient or parental report. Furthermore, children exhibited normal speech, language, and motor development, as indicated by parental report. Testing was completed with the approval of Vanderbilt University's Institutional Review Board. Participants were compensated for their time.

Stimuli

Three dual-task paradigms were used in this experiment. For all paradigms, the primary task was monosyllable word recognition, wherein participants verbally repeated each word presented. The three paradigms mainly varied with regard to either secondary task complexity or depth of processing. The secondary tasks were (a) a simple task wherein the secondary task was a physical response to a simple visual stimulus (simple), (b) a complex task wherein the secondary task was a physical response to a visual stimulus on the basis of a complex decision (complexity), and (c) a deep task wherein the secondary task was a physical response on the basis of a categorical size judgment of the speech presented (depth of processing). The same three dual-task paradigms were used for all participants.

Primary Task

Monosyllable words were adopted from previously published or commercially available word lists, including Central Institute for the Deaf W22 (Hirsh et al., 1952), Northwestern University Auditory Test No. 6 (Tillman & Carhart, 1966), Phonetically Balanced Kindergarten Word Lists (Haskins, 1949), and the Pediatric Speech Intelligibility Test (Jerger, Lewis, Hawkins, & Jerger, 1980). Words were recorded by a female talker in a professional recording studio using 44.1 kHz sampling rate and 16-bit resolution. Recordings were edited to each have approximately the same root-mean-square level. These stimuli have previously been used in measures of listening effort (e.g., Picou, Aspell, & Ricketts, 2014; Picou & Ricketts, 2014; Picou et al., 2013). Unlike previous investigations, the word lists were modified to include only concrete nouns. This subset of 200 words was arranged into eight lists of 25 words each. On the basis of pilot testing, words were arranged to create lists that were approximately equally intelligible. See the Appendix for words used that were arranged into the eight lists.

Speech stimuli were presented at 65 dBA. Noise, when present, consisted of four female talkers reading passages from the Connected Speech Test (Cox, Alexander, & Gilmore, 1987; Cox, Alexander, Gilmore, & Pusakulich, 1988), as previously described by Picou et al. (2014). Adults were tested with −4 dB SNR. For children, the SNR was chosen individually for each child to be −4, −2, or 0 dB in order to approximate 50% word recognition performance in noise. The SNR determination was based on a child's word recognition performance during practice. For most children (17/22), the SNR was −2 dB; four were tested with −4 dB SNR, and only one participant required a 0 dB SNR.

Secondary Tasks

The secondary tasks of the paradigms varied in complexity (complex paradigm) and depth of processing (deep paradigm) relative to a simple secondary task (simple paradigm), similar to the methods used by Picou and Ricketts (2014). However, the paradigms were modified to be more appropriate for school-age children, as described below. All secondary tasks included nonprobe and probe trials. During nonprobe trials, participants were expected to only perform the primary task of word recognition. During probe trials, participants were expected to perform word recognition and also make a timed physical response to a stimulus via 21.5-in. touchscreen monitor (Dell S2240T, Round Rock, TX) located directly in front of a participant. A colored shape appeared on the touchscreen and disappeared as soon as a participant touched the screen. Shapes appeared 500 ms after word onset. The color and shape varied on the basis of secondary task. Figure 1 shows schematic representations of visual stimuli timing for probe (top panels) and nonprobe trials (bottom panels) for all three secondary tasks.

Figure 1.

Example time course of probe trials (top panels) and nonprobe trials (bottom panels) for the simple, complex, and deep paradigms (left, center, and right panels, respectively). Displayed probe trials shapes represent a red square (simple), blue circle (complex), and white square (deep); displayed nonprobe trials shapes represent a white fixation cross (simple), blue triangle (complex), and white square (deep). Alternative probe trials in the complex task included a yellow triangle; alternative nonprobe trials included a yellow circle or a white fixation cross.

Simple Paradigm

During probe trials of the simple paradigm, a red rectangle (6.5 × 6.5 cm) appeared against a black background. During nonprobe trials, a small white fixation cross (1 × 1 cm) was displayed. For each 25-word list, 10 probe trials were included. Probe trial presentation was randomized within the constraints that one probe trial occurred once out of every three trials; only correct responses were included in response time analyses. Specific instructions to the patient were to “listen to the words and repeat every word you hear. Also, as soon as you see a red box on the screen, touch the screen to make the box go away as quickly as possible. If you see a red box, touch the screen before repeating the word.” Therefore, the simple paradigm required only a simple decision (red rectangle present or absent) and did not require deep processing, only word recognition.

Complex Paradigm

During probe trials of the complex paradigm, either a blue circle (6.5 × 6.5 cm) or a yellow triangle (6.5 × 6.5 cm) appeared against a black background. During nonprobe trials, a blue triangle (6.5 × 6.5 cm), a yellow circle (6.5 × 6.5 cm), or a small white fixation cross (1 × 1 cm) was displayed. For each 25-word list, 10 probe trials were included; nine nonprobe trials included the incorrect color/shape combination; six nonprobe trials included only the fixation cross. Probe trial presentation was randomized within the constraints that one probe trial occurred once out of every three trials; only correct responses were included in response time analyses. Specific instructions to the patient were to

“listen to the words and repeat every word you hear. Also, as soon as you see a blue circle or yellow triangle on the screen, touch the screen to make the shape go away as quickly as possible. If you see a blue circle or yellow triangle, touch the screen before repeating the word. Do not touch the screen if you see a blue triangle or a yellow circle, instead only say the word.”

Thus, relative to the simple paradigm, the complex paradigm required a more complicated decision (correct color AND correct shape), but required only word recognition without deep processing.

This task was developed specifically for school-age children. Color mastery occurs relatively early in development; infants as young as 4 months old categorically respond to colors (Ozturk, Shayan, Liszkowski, & Majid, 2013), particularly the four basic colors (red, green, blue, and yellow; Bornstein, Kessen, & Weiskopf, 1976). Moreover, the ability to name these colors reaches maturity around 4 or 5 years of age (e.g., Anyan & Quillian, 1971; Cook, 1931; E. G. Johnson, 1977). Simple shape identification (e.g., square, triangle, circle) similarly matures early in development and may develop even earlier than color mastery (Bornstein, 1985). Children in preschool and kindergarten can identify shapes with a high degree of accuracy (Clements, Swaminathan, Hannibal, & Sarama, 1999).

Deep Paradigm

During probe trials of the deep paradigm, a white square (6.5 × 6.5 cm) appeared against a black background. During nonprobe trials, the white square was also visible. The difference between probe and nonprobe trials was whether the word presented represented an object bigger than a basketball. Ten words in each list represented objects whose typical exemplars were bigger than a basketball (e.g., cab, shore) and an additional five of the words represented objects whose relative size comparison could be uncertain (e.g., dog, vine). The remaining 10 words represented objects that are always smaller than a basketball (e.g., bean, mouse). Probe trial presentation was randomized within the constraints that one probe trial occurred once out of every three trials. Specific instructions to the patient were to

“listen to the words and repeat every word you hear. Also, if the word you heard could be bigger than a basketball, touch the screen to make the shape go away as quickly as possible. For example, if the word were ‘bus,’ you would touch the screen. If the word were ‘mouse,’ you would not touch the screen. Touch the screen even if the word you heard could be bigger than a basketball. For example, a ‘fire’ could be bigger or smaller than a basketball. You would touch the screen if you heard the word ‘fire.’ If you decide the word is bigger than a basketball, touch the screen before repeating the word.”

Thus, similar to the simple paradigm, the decision was simple (bigger yes/no), but meaning extraction was required during word recognition.

This task was developed to increase the depth of processing, while being appropriate for school-age children. Object categorization is acquired early in development, simultaneously with language acquisition (e.g., Markman & Hutchinson, 1984; Waxman & Markow, 1995); children as young as 2 and 3 years old rely on similar features for object categorization as adults (Landau, Smith, & Jones, 1988). Although there are many features that could be used to make categorical object judgments, object size is a reasonable candidate. The stability of perceived object size develops in infancy (McKenzie, Tootell, & Day, 1980; Slater, Mattock, & Brown, 1990). Thus, even young children have considerable experience interacting in a world with objects whose sizes are stable and largely predictable on the basis of previous encounters. Therefore, object categorization on the basis of size is a task that school-age children could accomplish readily, but would still require deeper processing than simple word recognition.

Unlike the simple and complex paradigms, all responses during the deep paradigm were included in data analysis, unless a particular participant exhibited a suspicious response pattern (e.g., responding to every other word or responding much more or much less frequently than average). Two female adult participants were withdrawn from the study as a result of suspicious response activity; they responded to nearly every word in quiet and in noise. Thus the data presented include responses for the 14 participants whose responses were consistent with task instructions. No children demonstrated a suspicious pattern of responses.

All responses were included for several reasons. First, this scoring method was used previously with adults to evaluate the effects of changing the secondary task (Picou & Ricketts, 2014). Second, the purpose of increasing the depth of processing was to increase the competition for cognitive resources but not evaluate a participant's ability to accurately compare the size of imagined objects. Third, participants naturally varied in their abilities to consider all object exemplars. For example, a child with interest in aquatic transportation might readily recognize that the word sub could be larger than a basketball. However, a participant who preferred eating to boating might not consider a sub to be larger than a basketball. By including all responses, the potential negative interference of a participant's limited linguistic flexibility could be avoided.

Procedure

Data collection occurred over the course of three test sessions, which were always separated by at least a half day. The initial visit included consent, assent (for pediatric participants), hearing screening, and practice rounds, followed by testing with one of the three paradigms. The subsequent test sessions only included practice and testing with one of the two remaining paradigms. All testing with a particular paradigm was completed during a single test session. Paradigm test order was counterbalanced.

Participants practiced each task three times prior to data collection. During the first practice, a participant performed only the secondary task. The second and third practice involved a participant performing both primary and secondary tasks in quiet and in noise, respectively. During the third practice, if a participant's word recognition performance deviated substantially from 50% correct (> 75% or < 35%), the SNR was adjusted and the list was repeated. The starting SNR for all pediatric participants was −2 dB. During the fourth list prior to data collection, a participant performed only the secondary task. This served as a baseline response time for each paradigm. The same sequence was repeated prior to testing with each dual-task paradigm, except SNR adjustments were only made on the first day. During data collection, participants were tested in quiet and in noise. Eight 25-word lists were used for testing in each dual-task paradigm (four lists in quiet, four lists in noise). Test condition and word list orders were randomized.

Test Environment

Testing occurred in a sound-attenuating booth (4 × 4.3 × 2.7 m). Speech stimuli were delivered via custom programming of Presentation software (Neurobehavioral Systems 16.1) and were routed through an audiometer (Madsen Orbiter 922 v2, Schaumburg, IL), amplifier (Russound DPA6-12, Newmarket, NH), and then a loudspeaker (Tannoy System 600, Coatbridge, Scotland) positioned 1 m directly in front of the participant. The touchscreen monitor was on a table directly below the loudspeaker. The background noise was routed via sound editing software (Adobe Audition CSS 5.5), to the amplifier, and to loudspeakers (Definitive BP-2X, Definitive Technologies, Owings Mills, MD) positioned at 45°, 135°, 225°, and 315°, 1 m away from the participant.

Data Analysis

Two data transformations were conducted prior to analysis. First, word recognition performance was converted to rationalized arcsine units (RAUs) to normalize variance at the extremes (Studebaker, 1985). Second, after excluding responses < 100 ms or > 3 SDs from the mean, the log of mean responses in each condition was calculated. Transformation was necessary because the distribution of response times measured using the simple and complex paradigms were positively skewed (p < .05) and not normally distributed (Kolmogorov-Smirnov p < .05). In addition, response times in the simple and complex paradigms violated the assumption of kurtosis (p < .05). After transformation, all assumptions underlying parametric analyses were met. Therefore, the transformed response times were used for all analyses. Response times were analyzed with and without baseline corrections, but the pattern of results was identical for both analyses. Only the analysis with transformed, but uncorrected, response times are reported.

Results

Word Recognition Performance

Word recognition scores (RAU) are displayed in Table 1. A mixed-model analysis of variance (ANOVA) was used with two within-subjects factors, noise (absent, present) and task (simple, complex, deep), in addition to one between-subjects factor, age (adult, child). Results revealed a significant effect of noise, F(1, 36) = 1462.89, p < .001, η_p ² = .98. There was no main effect of task and were no significant interactions. These results suggest that noise negatively affected word recognition performance, but the effect was independent of age group and secondary task. It is important to note that the results suggest that increasing complexity or depth of processing did not interfere with the primary task performance.

Table 1.

Mean word recognition performance (rationalized arcsine unit) for each paradigm in quiet and noise for both groups of listeners.

Condition	Simple	Complex	Deep
Adults
Quiet	116.0 (4.0)	116.5 (3.9)	116.5 (3.1)
Noise	55.1 (9.2)	54.5 (11.8)	55.0 (13.6)
Children
Quiet	116.6 (3.0)	117.8 (1.9)	116.4 (3.3)
Noise	59.7 (10.5)	59.5 (8.6)	59.4 (10.5)

Note. Numbers in parentheses represent standard deviations.

Response Times

Response times measured while participants performed only the secondary task are displayed in Table 2. These baseline response times were analyzed, after log transformation as described in the Data Analysis section, using an ANOVA with a single within-subjects factor (task) and a single between-subjects factor (age). Results revealed a significant effect of age, F(1, 34) = 7.72, p < .01, η_p ² = .19, suggesting baseline response times were longer for children than adults. In addition, there was a significant main effect of task, F(2, 33) = 81.38, p < .001, η_p ² = .83, with baseline response times longest in the deep paradigm compared to the complex (p < .001) and simple (p < .001) paradigms; baseline response times were also longer in the complex paradigm relative to the simple paradigm (p < .001).

Table 2.

Response times measured during secondary task alone for each paradigm for adults and children.

Condition	Simple	Complex	Deep
Raw response times (ms)
Adults	502.2 (174.8)	786.2 (191.7)	974.2 (210.6)
Children	636.2 (208.5)	920.2 (187.0)	1153.6 (204.9)
Transformed response times (log[ms])
Adults	2.68 (0.12)	2.88 (0.10)	2.98 (0.10)
Children	2.78 (0.14)	2.96 (0.09)	3.06 (0.08)

Note. Top rows indicate raw response times (in milliseconds) prior to transformation. Bottom rows indicate transformed scores (log[ms]). Numbers in parentheses represent standard deviations.

Mean response times are displayed in Figure 2. Transformed response times (see Table 3) were analyzed using a mixed-model ANOVA with two within-subjects factors, noise and task, and one between-subjects factor, age. Results revealed significant main effects of noise, F(1, 34) = 96.67, p < .001, η_p ² = .74; task, F(2, 33) = 179.46, p < .001, η_p ² = .92; and age, F(1, 34) = 15.51, p < .001, η_p ² = .31. In addition, there were significant interactions of Task × Age, F(2, 33) = 4.16, p < .05, η_p ² = .20; Task × Noise, F(2, 33) = 5.48, p < .01, η_p ² = .2; and Noise × Age, F(1, 34) = 6.92, p < .05, η_p ² = .17. These results suggest that the presence of background noise, increased complexity or depth of processing, and participant youth all increased response times. However, the significant interactions suggest that the results of each of these factors depended on the other factors.

Figure 2.

Mean response times (in milliseconds) for each paradigm for adults (left panel) and children (right panel). Solid bars and hashed bars indicate responses measured in quiet and noise, respectively. Responses measured using the simple, complex, and deep paradigms are represented by white, light gray, and dark gray bars, respectively. Error bars represent ±1 SD.

Table 3.

Transformed response times (log[ms]) measured during each dual-task paradigm for each secondary task for adults and children in quiet and in noise.

Condition	Simple	Complex	Deep
Adults
Quiet	2.73 (0.12)	2.90 (0.09)	3.14 (0.06)
Noise	2.82 (0.14)	2.99 (0.11)	3.20 (0.11)
Children
Quiet	2.92 (0.15)	3.01 (0.11)	3.23 (0.06)
Noise	2.99 (0.13)	3.07 (0.11)	3.25 (0.04)

Note. These scores were used for all analyses. Numbers in parentheses represent standard deviations.

Because the purpose of this article was to evaluate the effect of changing the secondary task on paradigm sensitivity to noise, the decision was made a priori to analyze the effects of noise separately for each paradigm if significant task interactions were present. Therefore, separate follow-up ANOVAs for each secondary task were conducted, each with one within-subjects factor (noise) and one between-subjects factor (age). Analysis of the transformed response times revealed a significant effect of noise using the simple, F(1, 34) = 35.91, p < .001, η_p ² = .51, and the complex paradigms, F(1, 34) = 83.43, p < .001, η_p ² = .71. Analysis of transformed response times with the deep task revealed a main effect of noise, F(1, 34) = 66.40, p < .001, η_p ² = .66, and a significant Age × Noise interaction, F(1, 34) = 22.09, p < .001, η_p ² = .39, indicating the effect of noise was different for adults and children. Follow-up testing revealed that the effect of noise was significant for both groups, but the calculated effect size was larger for adults (η_p ² = .82) than children (η_p ² = .29). In total, these results suggest that noise increased listening effort, but the size of the effect depended on the secondary task and participant age. The results from the simple and complex paradigms were independent of participant age, whereas the calculated effect of noise was larger for adults than children using the deep paradigm.

The potential effects of age were further explored using a partial correlation between age and the calculated effect of noise (log[ms in noise] – log[ms in quiet]) for each paradigm, while statistically controlling for SNR (see Figure 3). Results revealed a significant correlation between age and effect of noise measured with the deep paradigm, r(33) = .65, p < .001, indicating noise had a less detrimental effect on response times for younger listeners. However, there were no significant relationships as measured with the complex or simple paradigms, r(33) = −.03, p = .89 and r(33) = −.11, p = .52, respectively, suggesting that effects of background noise on listening effort were generally independent of age, as measured with these paradigms. In addition, there was a significant relationship between the effect of noise measured using the simple and complex paradigms, r(33) = .52, p < .01, indicating that an individual's susceptibility to noise measured with one task was highly related to susceptibility measured with the other task. However, there was no relationship between susceptibility measured with the deep paradigm and the simple, r(33) = .07, p = .70, or the complex, r(33) = .03, p = .85, paradigms.

Figure 3.

Calculated effect of noise (in milliseconds) as a function of participant age. Squares, triangles, and circles represent effects of noise measured using the simple, complex, and deep paradigms, respectively.

Although children's performance was generally more variable, the average effect of noise was approximately the same for children and adults using the simple (137 ms) and complex (172 ms) paradigms. However, using the deep paradigm, the calculated effects of noise were 63 and 224 ms, on average, for children and adults, respectively. Furthermore, six of the youngest children demonstrated improvements in response times as measured using the deep paradigm, whereas no adults demonstrated such a pattern. These findings suggest that increasing the depth of processing did not improve paradigm sensitivity to the effects of noise. Instead, particularly for the youngest listeners, the deep paradigm may be an inappropriate paradigm to evaluate listening effort.

Number of Responses

The number of secondary task responses recorded during each 25-word list was calculated and converted to RAU (see Figure 4) to normalize the variance and the extremes and to facilitate comparisons across paradigms, since the total number of possible correct responses varied as a function of paradigm (10, 10, and 15, for the simple, complex, and deep paradigms, respectively). The converted number of responses was analyzed using an ANOVA with two within-subjects variables (noise, task) and one between-subjects variable (age). Results revealed no significant main effect of age. However, there was a significant effect of task, F(2, 33) = 21.79, p < .000, η_p ² = .57, and a significant effect of noise, F(1, 34) = 21.59, p < .001, η_p ² = .39. In addition, there were significant interactions of Age × Noise, F(1, 34) = 8.58, p < .001, η_p ² = .20, and Task × Noise, F(2, 33) = 15.16, p < .001, η_p ² = .48. As a result of the multiple interactions and of the findings related to response times, follow-up analyses focused on the effects of noise and age within each task. Results revealed no significant effects of noise or age for the number of responses recorded in simple or complex paradigms (p > .05). However, for the deep paradigm, there was a significant effect of noise, F(1, 34) = 32.84, p < .001, η_p ² = .49, but no Age × Noise interaction. These results suggest that noise did not affect the number of responses recorded during the simple and complex tasks, but did decrease the number of responses recorded during the deep paradigm. These results are expected because the secondary task during the deep paradigm depends on word recognition; when fewer words are heard, there are fewer opportunities for responding. It is important to note that there were no main effects of age (adult vs. child) in the number or pattern of responses, suggesting that both groups approached the tasks similarly.

Figure 4.

Number of responses recorded (RAU) during a 25-word list for each paradigm for adults (left panel) and children (right panel). Solid bars and hashed bars indicate responses measured in quiet and noise, respectively. Responses measured using the simple, complex, and deep paradigms are represented by white, light gray, and dark gray bars, respectively. Error bars represent ±1 SD. RAU = rationalized arcsine unit.

Discussion

The purpose of this study was to investigate the effects of changing the secondary task in a dual-task paradigm designed to measure objective listening effort. Adults and school-age children were tested using three dual-task paradigms in quiet and in noise.

Effects of Background Noise

The Ease of Language Understanding model suggests that factors that degrade speech inputs would increase listening effort because explicit cognitive resources are necessary to achieve understanding (Rönnberg et al., 2013, 2008). Consistent with this model and with previous findings (e.g., Fraser et al., 2010; Picou et al., 2013; Sarampalis et al., 2009), the results of this study suggest that background noise increased listening effort for adults.

Unlike some previous investigations (e.g., Hicks & Tharpe, 2002), the present study also demonstrated increases in listening effort with the addition of background noise for school-age children. Using the simple and complex paradigms, the effects of noise on listening effort were similar for adults and children (see Figure 3, dashed lines). Although previous results might suggest that children would be more susceptible to the effects of noise (e.g., C. E. Johnson, 2000; Markham & Hazan, 2004; Stelmachowicz et al., 2000), recall that the SNR was adjusted for children separately from adults in order to achieve comparable word recognition performance. Most children were tested with a −2 dB SNR, whereas adults were tested with a −4 dB SNR. As a result, children and adults achieved comparable word recognition performance (see Table 1) and were equally affected by background noise on measures of listening effort using the simple and complex paradigms.

These findings demonstrate that dual-task paradigms can be useful for evaluating listening effort in school-age children. In addition, the findings suggest that paradigm sensitivity may not have been solely responsible for previously reported nonsignificant findings; the simple paradigm in the present study was modeled after the one used by Hicks and Tharpe (2002) and was found to be sensitive to the effects of noise. There are several alternative possible reasons why effects of noise on listening effort were found in the present study but not in others (e.g., Hicks & Tharpe, 2002; McFadden & Pittman, 2008). These possible explanations include SNR, task engagement, and participant age.

First, some of the previous nonsignificant findings might be related to SNR and speech recognition performance. In the present study, word recognition performance was significantly degraded by implementing poor SNRs (0 to −2 dB), whereas Hicks and Tharpe (2002) and McFadden and Pittman (2008) used more positive SNRs (+2 to +10dB and 0 to +6 dB, respectively). Indeed, consistent with the present results, Howard et al. (2010) reported significant increases in listening effort comparing secondary task performance in quiet and at a −4 dB SNR. Therefore, future studies should carefully consider the SNR and word recognition performance levels when investigating the effects of noise on listening effort.

Second, paradigms designed to evaluate listening effort need to be carefully constructed to be moderately difficult; they need to be engaging enough to require competition for cognitive resources, but not so engaging that performance on the primary task declines. Previous studies with nonsignificant findings have demonstrated changes in primary task performance with the addition of the secondary task. For example, Howard et al. (2010) used word recognition as a primary task and five-digit number recall as the secondary task; they found that the addition of the secondary task hurt word recognition performance at the most negative SNRs. McFadden and Pittman (2008) similarly reported that, for children with minimal hearing loss, the addition of a dot-to-dot game (secondary task) negatively affected word categorization (primary task). In total, these results suggest that digit recall and dot-to-dot games might be too difficult for secondary tasks, whereas responding to visual probes as in the present study is appropriate for school-age children, as evidenced by stable word recognition performance across secondary tasks and the expected effects of noise.

A final contributing factor to the differences between previous studies and the present studies regarding the effects of background noise on listening effort for children might be differences in participant ages across studies. Participants in the present study were between 9 and 17 years old, whereas previous investigators have included younger participants (e.g., Hicks & Tharpe, 2002: 6 to 11 years; Howard et al., 2010: 9 to 12 years; McFadden & Pittman, 2008: 8 to 12 years). It is possible that the older children in this study behaved more like adults than the younger children in the previous studies, accounting for some of the differences in results.

Secondary Task

On the basis of previous findings from Picou and Ricketts (2014), it was expected that increasing depth of processing would increase paradigm sensitivity. The results of the present study confirm this expectation for adults, but not for children. The deep paradigm resulted in the most variable effects of noise on response times for children; there was a linear relationship between age and calculated effects of noise using the deep paradigm. Furthermore, some of the youngest children responded faster when noise was present than in quiet (see Figure 3, circles and solid line). There are several possible explanations for this somewhat surprising finding, including the development of object naming, inattention, response scoring, and level of processing.

First, although object naming begins developing in preschool, it is not mature until adulthood. Adults and children have different conceptual organizations for object names and descriptions (Landau, Smith, & Jones, 1998). Early in childhood, children have fewer experiences with objects and have more simplistic internal representations. As children develop, they use increasingly elaborate definitions of objects and exemplars; categorization becomes more sophisticated (Keil, 1994). One of the hypothesized benefits of increasing the depth of processing for a listening effort task is to capitalize on the same pool of cognitive resources for the primary and secondary tasks—that is, depth of processing requires participants to think about the language presented and to categorize the word heard. Perhaps because children's mental representations are less sophisticated, the size categorization task was too easy for young children. Because their representations are more simplistic, the task of identifying the size of an object may not be as competitive with primary task resources. Consistent with this hypothesis, six of the 10 youngest children demonstrated unexpected effects of noise on listening effort, whereas only one of the older children and none of the adults exhibited this pattern. Future work is warranted to further explore the underlying reasons for the adult–child differences in dual-task paradigm sensitivity.

Second, previous investigations suggest that the addition of background noise can differentially influence cognitive performance of school-age children, depending on whether or not the child is attentive (Helps, Bamford, Sonuga-Barke, & Söderlund, 2014; Söderlund, Sikström, Loftesnes, & Sonuga-Barke, 2010). For example, Söderlund et al. (2010) evaluated the effect of adding 78 dB background noise on children's verbal recall performance and found that performance improved with background noise for inattentive children, but decreased the performance of attentive children. A similar mechanism may underlie the finding reported by Choi, Lotto, Lewis, Hoover, and Stelmachowicz (2008) where word recognition in noise performance improved in younger children with the addition of a secondary task. It is possible that during the deep paradigm, younger children were inattentive in quiet and the addition of the background noise improved their performance, resulting in faster response times (see circles on Figure 3).

Third, response scoring could have contributed to the difference between adults and children. Unlike the simple and complex paradigms, all responses were accepted during the deep paradigm, unless a participant was not following task instructions (e.g., responding to the secondary task after every word). Accepting all responses was done to avoid the interference of a child's linguistic flexibility on measurements of effort and to be consistent with previous work (Picou & Ricketts, 2014). However, it is possible that the younger children were prone to guessing or responding without deeply processing the words. To investigate the possibility that participants were guessing, the data were reanalyzed using only correct responses in the deep paradigm. Results revealed an identical pattern of results. Calculated effects of background noise were nearly identical when using all responses (η_p ² = .82 and η_p ² = .29) compared to using only correct responses (η_p ² = .81 and η_p ² = .28) for adults and children, respectively. Therefore, it seems unlikely that type of response scoring contributed to the findings.

Last, it is possible that the level of processing required of participants, especially the children, was not deep enough to increase paradigm sensitivity. Picou and Ricketts (2014) manipulated depth of processing by asking adults to categorize whether the word heard was a noun, which requires full meaning extraction. However, the present study only required participants to consider the size of an object, not all of the possible meanings of the word. It is possible that a secondary task that requires a deeper level of processing—for example, whether the object is dangerous—would be more sensitive to factors that affect listening effort.

Conclusions

The results of this study suggest that manipulations of the secondary task are age dependent. For adults, increasing the depth of processing significantly increased paradigm sensitivity, consistent with previous literature. However, for younger children for whom object representations are more simplistic, increasing the depth of processing by requiring a size categorization judgment did not increase paradigm sensitivity. Instead, increasing the depth of processing of the secondary task decreased paradigm sensitivity, particularly for the youngest listeners. The present study design does not allow for determination of whether adult–child differences were due to increasing depth of processing or increasing combined task difficulty (or both). However, on the basis of these results, it seems prudent to consider the nature of the secondary task and the potential interaction with participant age when designing future studies of listening effort. To be specific, when evaluating school-age children, it may be important to avoid increasing the depth of processing and evaluate task difficulty to ensure that performance on the primary or secondary task does not improve during dual task testing.

Appendix

Monosyllable Words Presented During Testing With Dual-Task Paradigms

List 1	List 2	List 3	List 4	List 5	List 6	List 7	List 8
Tire	Me	Path	Lot	Tool	Tray	Door	Ham
Rat	Slip	Bee	Snake	Mouth	Spoon	Soap	Bike
Hall	Mop	Bead	Horse	Loop	Tooth	Lake	Teeth
Niece	Nest	Suit	Road	Tape	Box	Bug	Bird
Moth	Purse	Tongue	Knee	Ship	Man	Girl	Fudge
Pants	Goose	Moon	Pad	Watch	Cup	Base	Crab
School	Knife	Pill	Neck	Jam	Cab	Yam	Cave
Sack	Note	Hand	Bell	Clown	Weed	Paste	Scab
Wheel	Well	Pod	Roof	Home	Head	Phone	Mob
Sink	Sun	Park	Coat	Shack	Light	Deck	Chin
Mouse	Cape	Mess	Bun	Mud	Web	You	Hair
Vine	Dime	Room	Rug	Slice	Pond	Goal	Nick
Lip	King	Coin	Sled	Bear	Doll	Bush	Fan
Plow	Perch	Nail	Peg	Hut	Frog	Toad	Dish
Juice	Match	Kite	House	Page	Back	Rain	Feet
Sail	Calf	Pool	Loaf	Lung	Ring	Limb	Shop
Shore	Safe	Dad	Truck	Rag	Gum	Ball	Bone
Comb	Nuts	Splash	Tick	Vase	Chair	Fair	Wheat
Jug	Soup	Map	Cart	Patch	Dress	Gun	Lawn
Sea	Sub	Cake	Rib	Foot	Loot	Shirt	Cheek
Case	Shoe	Milk	Axe	Food	Hen	Book	Cage
Dog	Whip	Team	Jar	Geese	Bean	Jail	Sheep
Wife	Tar	Train	Hole	Witch	Fish	Gem	Germ
Nose	Lid	Shade	Tree	Boat	Kid	Beet	Chalk
Pole	Clock	Cat	Bed	Cheese	Judge	Duck	Pearl

Funding Statement

This work was funded by Sonova and Maddox Charitable Trust. In addition, a training grant provided some financial support to the second author (NIH NIDCD T35DC008763; PI: Linda Hood).