Older Listeners' Perception of Speech With Strengthened and Weakened Dynamic Pitch Cues in Background Noise

Abstract

Purpose

Dynamic pitch, which is defined as the variation in fundamental frequency, is an acoustic cue that aids speech perception in noise. This study examined the effects of strengthened and weakened dynamic pitch cues on older listeners' speech perception in noise, as well as how these effects were modulated by individual factors including spectral perception ability.

Method

The experiment measured speech reception thresholds in noise in both younger listeners with normal hearing and older listeners whose hearing status ranged from near-normal hearing to mild-to-moderate sensorineural hearing loss. The pitch contours of the target speech were manipulated to create four levels of dynamic pitch strength: weakened, original, mildly strengthened, and strengthened. Listeners' spectral perception ability was measured using tests of spectral ripple and frequency modulation discrimination.

Results

Both younger and older listeners performed worse with manipulated dynamic pitch cues than with original dynamic pitch. The effects of dynamic pitch on older listeners' speech recognition were associated with their age but not with their perception of spectral information. Those older listeners who were relatively younger were more negatively affected by dynamic pitch manipulations.

Conclusions

The findings suggest the current pitch manipulation strategy is detrimental for older listeners to perceive speech in noise, as compared to original dynamic pitch. While the influence of age on the effects of dynamic pitch is likely due to age-related declines in pitch perception, the spectral measures used in this study were not strong predictors for dynamic pitch effects. Taken together, these results indicate next steps in this line of work should be focused on how to manipulate acoustic cues in speech in order to improve speech perception in noise for older listeners.

Dynamic pitch, which is defined as the variation of fundamental frequency (F0) of speech, is one of the acoustic cues that aid speech perception under adverse conditions. Data from younger listeners with normal hearing have shown that dynamic pitch in natural speech plays an important role in speech perception in background noise (e.g., Binns & Culling, 2007; Laures & Bunton, 2003; Miller et al., 2010). When a dynamic pitch contour is inverted or restructured, speech recognition in noise becomes significantly more difficult. For instance, Binns and Culling (2007) manipulated the strength of dynamic pitch cues by inverting or weakening the pitch contours of the target speech, which was then embedded in a static noise that had the spectral characteristics of the same talker. While speech recognition performance was significantly lower with the inverted pitch contour, it was not affected by weakened dynamic pitch as compared to natural pitch. Consistent with this finding, Miller et al. demonstrated that inverted or frequency-modulated pitch contours negatively affected speech recognition in steady-state noise. Interestingly, exaggerated dynamic pitch was found to either harm (Miller et al., 2010) or not affect speech recognition (Clarke et al., 2017) as compared to original (unaltered) dynamic pitch. This is also supported by more recent data from a study using naturally produced speech stimuli (instead of manipulating pitch by signal processing; Calandruccio et al., 2019). In general, the effect of pitch inversion and restructuring on speech perception has been attributed to the role of natural dynamic pitch as one of the prosodic cues in speech perception and serves to facilitate speech processing (e.g., Binns & Culling, 2007). While the previous studies have shown the detrimental or null effects of exaggerated/weakened dynamic pitch cues for younger listeners' speech perception in noise, the effects of dynamic pitch manipulations on older listeners' speech perception in noise are more complex and variable.

The following sections of the introduction are structured as below. The first section reviews the evidence that motivates the first objective of this study: to determine whether mildly strengthened and weakened dynamic pitch cues affect older listeners' speech perception in noise. The second section summarizes the evidence on interlistener variability in the effects of dynamic pitch manipulations. This review leads to the second objective of this study, which is to measure a number of individual factors, including static and dynamic spectral perception abilities, and to test the relationship between these metrics and the effects of dynamic pitch manipulations on older listeners' speech perception in noise.

Effects of Manipulated Dynamic Pitch Cues on Older Listeners' Speech Perception in Noise

Built upon the previous studies on younger listeners, a series of projects has examined the effect of exaggerated dynamic pitch cues on older listeners' speech recognition in different types of noises (Shen & Souza, 2017a, 2017b, 2018). These projects provided evidence concerning when and how much exaggerated dynamic pitch cues are beneficial for older listeners with and without hearing loss. First, older listeners as a group recognize speech better with exaggerated dynamic pitch as compared to flattened pitch. However, their performance with both manipulated conditions is lower than with the original dynamic pitch cues. Second, the effects of dynamic pitch appear to be stronger with temporally modulated noise that does not have pitch cues (Shen & Souza, 2017a, 2018). Last, there is substantial individual variability in older listeners' responses to original versus strengthened dynamic pitch cues (Shen & Souza, 2017b). In other words, some older listeners recognize speech in noise better with original dynamic pitch, while others do better with strengthened dynamic pitch. It is worth noting that the exaggerated dynamic pitch in the previous studies on older listeners was 1.75 times the original pitch strength. It is possible that these much-exaggerated pitch contours deviated too much from the acoustic/linguistic cues that are familiar to the listeners (see also Calandruccio et al., 2019). It thus remains an open question whether mildly exaggerated dynamic pitch strength may work better for older listeners, as compared to the original dynamic pitch.

While detrimental effects of weakened dynamic pitch cues have not been found across a few studies in younger listeners (Binns & Culling, 2007; Calandruccio et al., 2019; Clarke et al., 2017), it is yet to be tested whether weakened dynamic pitch cues can affect speech perception in noise by older listeners with hearing loss. In fact, we know speech signals are redundant in terms of acoustic cues (e.g., Wright, 2004). For younger listeners with good hearing, the evidence so far suggests that they do not rely heavily on dynamic pitch cues for speech recognition in noise. Therefore, weakened dynamic pitch cues do not have a strong impact on their speech perception performance. For older listeners, however, the speech perception in noise task is much more effortful. As listeners are forced to utilize all of the cues available, weakened dynamic pitch may pose an observable detriment to speech in noise performance. In this study, we aimed to better understand the impact of weakened dynamic pitch cues on older listeners' speech perception in noise. In order to generalize the results to a large group of older adults, the experiment included a group of older listeners who varied in hearing status from near-normal to mild-to-moderate hearing loss. Their data were also compared to data from a control group of younger listeners with normal hearing.

Individual Factors and Dynamic Pitch Effects

One of the key findings from previous studies is substantial variability in the effects of dynamic pitch manipulations across older listeners. For example, across individual listeners in a group of older participants with hearing loss, the effect of dynamic pitch (i.e., the difference in speech recognition with dynamic pitch as compared to with flattened pitch) is spread across a wide range of 1.2–5.2 dB SNR, which corresponds to 40% in speech recognition performance on a psychometric curve (Shen & Souza, 2017b). This finding indicates some older listeners understand speech much better in noise with dynamic pitch cues, while others were less affected by these cues.

To use dynamic pitch cues in speech perception in noise, an older listener with hearing loss first has to be able to perceive dynamic pitch in speech. While the negative impact of hearing loss was demonstrated by one study with a nonspeech masker (Shen & Souza, 2017a), hearing as measured by pure-tone thresholds does not consistently predict the variability in the effects of dynamic pitch manipulations. It is possible that a range of individual factors, including perceptual and cognitive abilities, play roles in modulating dynamic pitch effects on older listeners' speech in noise performance. This hypothesis has so far gained support from the finding that working memory capacity contributes to the effect of dynamic pitch strengthening in speech maskers (Shen & Souza, 2018). Built on this rationale, this study included in the analysis several key individual factors that are involved in older listeners' speech perception in noise, such as hearing loss (i.e., pure-tone thresholds), age, and general cognitive ability (i.e., scores on a comprehensive cognitive screening test).

Specifically concerning perceptual abilities, we know pure-tone thresholds do not directly reflect the aspect of hearing ability that is crucial for perceiving (and therefore being affected by) pitch cues. For instance, frequency discrimination data collected from older listeners with sensorineural hearing loss and near-normal hearing suggest the ability to perceive pitch cues varies substantially in older listeners with similar audiograms (Moore & Peters, 1992). While it is likely that multiple perceptual factors play roles in influencing older listeners' pitch perception in background noise, this study was focused on listeners' ability to perceive static and dynamic spectral information.

A number of studies have demonstrated the connection between listeners' static spectral perception (i.e., spectral resolution) and pitch perception performance (Bernstein & Oxenham, 2006; Florentine, 1980; Moore & Peters, 1992). Moore and Peters' (1992) study was one of the earlier studies that examined this relationship. They measured the frequency difference limen and spectral resolution (as indicated by auditory filter width) of older and younger listeners either with hearing loss or normal hearing. While the results were quite variable both within and across groups, it was demonstrated that pitch perception performance was lower for listeners who have poor spectral resolution. Later data from Bernstein and Oxenham (2006) are consistent with this finding in showing poor spectral resolution negatively impacts listeners' ability to resolve harmonics, causing poor pitch perception as a result. Built on this evidence, we hypothesized that dynamic pitch cues do not affect speech perception in noise for some older listeners because they have poor spectral resolution and cannot perceive pitch cues in speech. Discrimination of spectral ripple stimuli is one of the frequently used paradigms for measuring spectral resolution (e.g., Sheft et al., 2011; Won et al., 2007) and has been found to be a significant predictor for pitch perception in music and speech perception in noise (Won et al., 2010, 2007). We chose to use this test in this study because its wideband stimuli encompass the frequency range of speech and its shorter testing time, which make it more suitable than other spectral tests (e.g., frequency selectivity test using notched noise) for the investigation of speech perception in older listeners.

Among those older listeners who can resolve static spectral cues, we know some nevertheless have difficulty perceiving dynamic spectral cues (Sheft et al., 2012; Shen et al., 2016). For example, Sheft et al. (2012) tested younger and older listeners with frequency modulation discrimination tests. The stimuli were generated by modulating a 1-kHz carrier with 5-Hz low-pass noise, and the performance was measured using multiple variables including frequency excursion, duration, and signal-to-noise ratio (SNR) in background noise. As compared to the data from younger listeners, performances of older listeners were quite variable across individuals, and most of the measures were found to correlate with age but not with hearing thresholds. This substantial variability was again observed in dynamic pitch perception with synthesized vowels (Shen et al., 2016). Interestingly, even within a group of older listeners who have near-normal hearing and good ability to perceive static pitch cues (as measured by frequency difference limens), performance on a dynamic pitch perception test varied across individuals. This dynamic pitch perception measure, however, did not directly predict the effect of dynamic pitch strengthening (Shen & Souza, 2017a). To further explore the contribution of dynamic spectral perception on the effects of dynamic pitch manipulations, we hypothesized those older listeners who cannot perceive brief dynamic spectral patterns would not be affected by dynamic pitch manipulations in speech recognition in noise. Specifically, the frequency modulation discrimination test (Sheft et al., 2012) was employed to measure older listeners' ability to perceive changes in dynamic spectral cues.

Method

Participants

The older group consisted of 33 older adults (17 women and 16 men) aged 62–85 years (mean age of 70.48 years). They had various levels of hearing ranging from near-normal hearing to mild-to-moderate hearing loss. Inclusion criteria included symmetrical hearing as defined by a difference in pure-tone average (0.5, 1, 2 kHz) ≤ 10 dB HL across ears and air–bone gaps ≤ 10 dB HL at octave frequencies between 0.5 and 3 kHz. Their audiograms (averaged across two ears) are presented in Figure 1. Seventeen of the participants had hearing loss as defined by pure-tone average (i.e., average of pure-tone thresholds at 0.5, 1, 2 KHz) of 25 dB HL or above. Seven of the participants were hearing aid users.

Figure 1.

Audiograms of the older listeners (n = 33). Dashed lines = individual thresholds; thick line = group average.

The participants were recruited via newspaper advertisement and flyers in the Kalamazoo area. All participants were native English speakers. None of the participants had experience in tonal languages, or more than 12 years of nonprofessional musical experience (musical experience was therefore controlled for in the analysis). All older participants passed screening for cognitive impairment by using Saint Louis University Mental Status Examination (Tariq et al., 2006). Ethical and safety review of the test protocol was conducted and approved by the local institutional review board. Participants were compensated for their time.

Another group of 16 younger adults (11 women, five men) aged 19–29 years (mean age of 22.25 years) participated in this study. All the listeners had normal hearing in both ears as defined by pure-tone thresholds ≤ 20 dB HL (American National Standards Institute, 2004) at all octave frequencies between 250 and 8000 Hz. The participants were recruited via flyers around the Western Michigan University campus. All participants were native speakers of English with no experience in tonal languages and had nonprofessional musical experience that was comparable to the older group (no more than 12 years).

Stimuli

The target speech stimuli were drawn from Harvard Institute of Electrical and Electronics Engineers sentences (Rothauser et al., 1969) that were produced by a female talker. These sentences have low predictability and therefore were ideal for minimizing the use of linguistic context for speech recognition. The sentences were resynthesized to have manipulated dynamic pitch cues using the Praat program (Boersma & Weenink, 2013) with the method described in a previous paper (Shen & Souza, 2017a). Specifically, four dynamic pitch strength conditions were created for each sentence using the following formula:

$$\text{Instant}F\mathit{0}=\text{Sentence\hspace{0.17em}average}F\mathit{0}+\left(\text{Original\hspace{0.17em}instant}F\mathit{0}-\text{Sentence\hspace{0.17em}average}F\mathit{0}\right)\times \text{Pitch\hspace{0.17em}factor}.$$ (1)

The pitch factors in the four conditions were set as: 1 in the original-pitch condition, 1.65 in the strong-pitch condition, 1.3 in the mildly strong-pitch condition, and 0.5 in the weak-pitch condition. It is worth noting the original-pitch condition was generated by setting the pitch factor as 1 to keep pitch contour unaltered, while going through the same resynthesis process as other conditions. Table 1 reports the mean pitch and pitch range measures of sentences (collapsed over all sentences in each condition). Two noise conditions were chosen from the International Collegium for Rehabilitative Audiology noises (Dreschler et al., 2001): nonspeech noise with spectral and temporal characteristics of a six-talker babble (three men, three women) and unmodulated noise that was frequency-shaped by the spectrum of a male talker. The rationale for using these two noises was based on three considerations. First, six-talker noise, which was generated based on multitalker babble with little temporal modulation, was included to balance the needs of simulating a realistic masking scenario and minimizing the influence of temporal modulation in noise on the effect of dynamic pitch strengthening (Shen & Souza, 2017a). Second, an unmodulated noise was chosen to make a comparison to previous data possible, as many previous studies used unmodulated noises (e.g., Binns & Culling, 2007; Laures & Bunton, 2003). Last, considering the strong impact of energetic masking on speech in noise by listeners with hearing loss (Goossens et al., 2017), the noises were chosen to limit energetic masking as they were generated from different gender (male in this study as the target talker is female) and a mix of both genders (six-talker), while the target speech was produced by a female talker.

Table 1.

Average pitch and pitch range measures of sentences (collapsed over all sentences in each pitch condition, standard deviation in parenthesis, in the unit of Hz).

Variable	Original	Weakened	Original (processed)	Mildly strengthened	Strengthened
Average Pitch	242.81 (11.70)	240.18 (9.12)	240.90 (8.96)	241.61 (8.87)	241.71 (10.11)
Pitch Range	163.46 (35.67)	81.65 (16.57)	161.13 (33.28)	223.10 (45.24)	275.50 (53.92)

Procedure

Prior to experimental testing, participants completed an audiometric battery consisting of case history, otoscopy, pure-tone threshold testing, and word recognition in quiet. The audiometric testing was done using a GSI Pello audiometer connected to IP30 insert earphones. All testing was completed in a double-walled soundproof booth. All stimuli were presented binaurally using an RME Fireface UCX soundcard and Sennheiser HD-25 headphones.

Speech in Noise Test

Speech reception thresholds (SRTs) were obtained for all eight conditions (4 dynamic-pitch conditions × 2 background-noise conditions) using a customized MATLAB program with an adaptive procedure (Plomp & Mimpen, 1979). The noise segment on a given trial was randomly selected from a 1-min noise signal. The starting SNR was −4 dB SNR. The initial SNR increment was 6 dB until at least three out of five key words were repeated correctly. For each subsequent sentence, the SNR increased by 2 dB when zero to two key words were correctly repeated or decreased by the same amount for three to five correct key words. The number of trials was fixed at 15. The SNR varied across trials to track a performance level of 50% correct. SRTs for each condition were measured twice. When any testing run provided fewer than three reversals, or when the standard deviation across the final reversals exceeded 4 dB, SRT was measured a third time. Thresholds for each run were computed by taking the mean SNR (dB) across the reversals at the final step size of 2 dB. The order of conditions was counterbalanced across participants following a Latin-square design. Prior to testing, participants were given brief training on all conditions to familiarize them with the different types of speech and noise. Practice consisted of eight trials and started at 0 dB SNR. Participants were instructed to repeat back the sentences aloud for the experimenter, who scored the responses from outside the booth. To make the signal equally audible to all listeners whose hearing status varied across a wide range, the stimuli were amplified using the National Acoustics Laboratories–Revised linear prescriptive formula on the basis of individual thresholds (Byrne & Dillon, 1986) with a baseline presentation level of 65 dB SPL.

Spectral Tests

Older listeners' ability to discern static spectral patterns (i.e., spectral ripples with different starting phases) was measured by a spectral ripple test (Sheft et al., 2011). The 500-ms stimuli were sinusoidal ripples that were derived from the long-term spectrum of wideband noise stimuli. Ripple density was 1.5 cycles per octave with a peak-to-trough difference of 30 dB. The task measured the just-detectable change in the starting phase of the spectral ripple with randomization of ripple phase across trials. All the stimuli were shaped with a 50-ms rise/fall time and passed through a speech-shaped filter that emphasized the frequency range of 200–500 Hz, with a steep roll-off in the low frequencies and a gradual roll-off in the high frequencies (Byrne et al., 1994). Ripple-phase thresholds in radians were logarithmically transformed before statistical analysis.

A dynamic spectral perception test was used to measure listeners' ability to discern dynamic spectral patterns (i.e., frequency modulated tones; Sheft et al., 2012). The stimuli were 1-kHz pure tones that were frequency modulated by 5-Hz low-pass noise, with random modulation patterns and maximum frequency excursions (ΔF) of 400 Hz. The duration started at 450 ms, and the task measured the minimum duration needed for discriminating between the tones with different frequency-modulation patterns.

The paradigm for both static and dynamic spectral tests was a cued two-interval forced-choice procedure. The test protocol was based on a modified descending method of limits (G. R. Kidd et al., 2007). The individual thresholds were derived from performance on two 42-trial blocks, each cycling through six levels of the ripple phase (in Static test) and duration (in Dynamic test). Presentation level was set to 30 dB SL and did not exceed 85 dB SPL.

Results

Effect of Pitch and Noise in Older and Younger Groups

Data were analyzed using mixed-effects linear regression (Baayen, 2008) in R (Version 3.2.1) with the lme4 and lmerTest libraries (Bates et al., 2014; Kuznetsova et al., 2013). The first model included fixed effects for dynamic pitch, noise types, order of measurement, and random intercepts for participant. The dependent variable was SRT; order of measurement was included as a control variable. Simple coding was used to compare the SRT (in dB SNR) in the three manipulated pitch conditions as compared to that in the original pitch condition.

Using t tests with Satterthwaite approximations to degrees of freedom (Kuznetsova et al., 2013), the effects of dynamic pitch and noise were further tested as fixed factors in the linear mixed model. The results showed that speech recognition performance was significantly poorer (with higher numeric values of SRT) in all three manipulated pitch conditions as compared to the original pitch condition (weakened: β = 1.04, SE = 0.19, t = 5.31, p < .01; mildly strengthened: β = 0.80, SE = 0.19, t = 4.08, p < .01; strengthened: β = 1.16, SE = 0.19, t = 5.92, p < .01).

Data from the younger normal hearing group demonstrated a pattern that was consistent with the older group data in terms of the effects of dynamic pitch manipulations. Speech recognition performance was significantly poorer in all three manipulated pitch conditions as compared to the original pitch condition (weakened: β = 0.83, SE = 0.22, t = 3.72, p < .01; mildly strengthened: β = 1.11, SE = 0.22, t = 4.96, p < .01; strengthened: β = 1.12, SE = 0.22, t = 5.01, p < .01).

The effect of noise type was also significant in both older and younger groups, albeit in different directions. Speech recognition performance in the older group was better (i.e., with lower numeric values of SRT) in the unmodulated noise condition than in the six-talker noise (β = −0.43, SE = 0.14, t = −3.09, p < .01). The younger group, however, had better speech recognition performance in the six-talker noise than in the unmodulated noise (β = 1.73, SE = 0.15, t = 10.92, p < .01). SRT data of older and younger groups are presented in Figure 2.

Figure 2.

Speech reception thresholds (SRTs) of the younger and older groups. Error bars indicate ±1 standard error. SNR = signal-to-noise ratio.

Effects of Individual Factors in Older Group

For the older group, it was also of interest to examine the potential influence of individual factors (e.g., age, hearing loss, general cognitive ability, and musical experience) on the effects of dynamic pitch manipulations. This was implemented by a second analysis on whether the effect of manipulated dynamic pitch was affected by these individual factors. Two models were built with same structure, except that one included age, while the other included hearing loss (i.e., pure-tone average) because age and hearing loss were significantly correlated, r = .46, t(31) = 2.85, p < .01. The other fixed factors in these models were pitch conditions, noise conditions, cognitive ability (Saint Louis University Mental Status Examination score), music experience (in number of years), and baseline speech in noise ability (SRT with original pitch in six-talker noise); the model also included random intercepts for participant. The dependent variable was the effect of dynamic pitch manipulations, as defined by SRTs in original pitch condition minus SRTs in manipulated pitch conditions (collapsed across noise conditions). In this analysis, age was the only factor significantly predicting the effect of dynamic pitch (hearing loss: t = 1.66, p > .1; age: t = 2.53, p < .05; cognitive ability: t = −0.99, p > .1). The relationship between age and the effects of dynamic pitch manipulations is shown in Figure 3, and it indicates that those older listeners who were relatively younger were more negatively affected by strengthened dynamic pitch. This result persisted after the listener with 3.9-dB dynamic pitch effect (which was an outlier) was removed from the analysis (t = 2.17, p < .05). Figure 3 plots the effects of dynamic pitch manipulations as functions of listeners' age. In this plot, lower numeric values of the effect of dynamic pitch manipulations indicate speech perception performance was more strongly affected by dynamic pitch manipulations.

Figure 3.

The effects of dynamic pitch manipulations as a function of the older listeners' age. Dynamic pitch effect: speech reception thresholds in original pitch condition minus speech reception threshold in the manipulated pitch condition (a lower numeric value indicates stronger effect from dynamic pitch manipulation). yrs = years.

Effect of Spectral Abilities in Older Group

The purpose of this analysis was to examine whether an older listener's ability to perceive spectral information can serve as an indicator for predicting the effect of manipulated dynamic pitch cues on their speech perception in noise. The ranges of the data from the two spectral perception tests were comparable to those in previous studies (Sheft et al., 2011, 2012), when hearing loss is taken into consideration. Specifically, the range was 0.15–0.83 rads for static spectral perception threshold and 85.71–167.98 ms for dynamic spectral perception. Using two mixed-effects linear regression models, the effects of the two spectral perception tests were tested separately. Each model had fixed factors of pitch and noise conditions, performance on spectral perception test (i.e., thresholds), baseline speech in noise ability (SRT with original pitch in six-talker noise), and random intercepts for participant. The dependent variable was the effect of dynamic pitch manipulations. Neither of the spectral perception measures was significant in predicting the effects of dynamic pitch manipulations (static spectral perception: t = −1.20, p > .1; dynamic spectral perception: t = −1.61, p > .1).

Discussion

Manipulated Dynamic Pitch and Speech Perception in Noise

In this study, we were first interested in speech perception performance with manipulated dynamic pitch cues as compared to the original ones. Specifically, the effects of mildly strengthened and weakened dynamic pitch cues were of interest as there was no previous data on how these manipulations affect older listeners' speech perception in noise. To this end, there were a few findings that have advanced our understanding of the effects of dynamic pitch manipulations of older listeners' speech perception.

In general, the results from this study align with the literature in demonstrating a negative effect from manipulated dynamic pitch cues on speech perception in noise, as compared to the original pitch contours in speech. We show for the first time that weakened dynamic pitch had a detrimental effect on speech perception for younger and older listeners. While several previous studies have used the same pitch manipulation technique and shown a detrimental effect of flattened pitch contour (Binns & Culling, 2007; Miller et al., 2010; Shen & Souza, 2017a), only Binns and Culling (2007) included a weakened pitch condition. In that study, no significant difference was found in speech-in-noise performance by younger listeners for weakened versus original dynamic pitch cues. In contrast, this study demonstrated a detrimental effect of weakened dynamic pitch (as compared to original dynamic pitch) in both younger and older listeners. The discrepancy can potentially be explained by the differences in background noises and the amount of energetic masking between studies. Binns and Culling (2007) used static noise that was speech-shaped by the spectrum of their target talker, which is a masking situation known to be dominated by energetic masking (Brungart et al., 2006). Because this study used nonspeech noises with spectral characteristics modeled on speech produced by a single talker of a different gender from the target talker or by six talkers varying in gender, the amount of energetic masking due to spectral overlap is lower than in the previous study. As illustrated by Figure 4, the two noises used in this study had different long-term spectra as compared to the target speech, which was produced by a female talker. To demonstrate the amount of energetic masking when target and masker are from the same talker (i.e., as in the previous study), Figure 4 also includes a masker condition modeled on speech produced by the talker who produced the target speech. Taken together, the data from this study suggest that manipulated dynamic pitch has a stronger effect when energetic masking is less dominant, which also aligns with previous research findings (e.g., Calandruccio et al., 2019). This difference in the effects of dynamic pitch manipulations in various masking scenarios also agrees with previous data in showing the effect of dynamic pitch is modulated by noise characteristics (Shen & Souza, 2017a).

Figure 4.

Long-term average speech spectra of the target speech, two noise conditions (six-talker and static noises), and a hypothetical noise condition with same female talker (to simulate the same-talker speech-on-speech masking).

Also consistent with previous studies, we found poorer speech perception performance with strengthened dynamic pitch cues for both younger and older listeners. More importantly, the results demonstrated a negative effect of even mildly strengthened dynamic pitch. This finding is along the same line as the previous evidence (Calandruccio et al., 2019; Clarke et al., 2017), in showing strengthened dynamic pitch is not helpful for listeners to understand speech in noise. In order to better understand these results, we should first review a few findings on the effects of dynamic pitch that shed light on the potential mechanisms. First of all, based on the evidence we have, at least one of the mechanisms that underserves speech perception in noise with dynamic pitch cues is enhanced perceptual continuity (e.g., Dannenbring, 1976), which helps listeners to better segregate target speech from background noise. This can potentially explain the detrimental effect of flattened pitch contour found in previous studies (Binns & Culling, 2007; Clarke et al., 2017; Shen & Souza, 2017a). This mechanism, however, likely works in an all or none fashion and does not depend on the strength of dynamic pitch cues. In other words, compared to original dynamic pitch, strengthened dynamic pitch does not provide extra perceptual benefit based on this mechanism. Second, in addition to the perceptual continuity hypothesis, previous research has suggested another mechanism from a linguistic perspective. Because inverted and frequency modulated dynamic pitch cues consistently have a negative effect on speech perception in noise (Binns & Culling, 2007; Miller et al., 2010), it supports the linguistic hypothesis that dynamic pitch contours that deviate from natural ones distort the intonation cues and consequently lead to poorer speech perception.

Unlike flattened or inverted dynamic pitch, stronger pitch cues have been observed in clear speech and Lombard speech (e.g., Picheny et al., 1986; Summers et al., 1988). While it is understandable that the strengthened pitch condition (1.65 times original pitch) likely exaggerates natural pitch range and makes speech sounds unnatural (e.g., Krause, 2001), the new finding of a detrimental effect from the mildly strengthened pitch condition (1.3 times original pitch) raises the question of why the current pitch strengthening strategy is not helpful for speech perception in noise. A detailed examination of the literature can provide a few possible explanations, which should be investigated in future research. First, previous research has consistently demonstrated the roles of multiple acoustic cues in clear speech (e.g., Krause, 2001; Picheny et al., 1986) and Lombard speech (e.g., Castellanos et al., 1996; Letowski et al., 1993; Summers et al., 1988). These cues consist of a wide range of acoustic properties including global measures (e.g., spectral distribution), phonemic characteristics, and prosodic cues (e.g., intensity, duration, and pitch). As exaggerated dynamic pitch (i.e., larger pitch range) is typically one of these cues that are associated with increased intelligibility, it was reasonable to expect an effect of artificially manipulated dynamic pitch on speech intelligibility in noise. Data from multiple studies, however, point to a more complicated story. When implemented independent of other cues, manipulations of dynamic pitch (either digitally or naturally) have not been found effective for speech perception in noise (Calandruccio et al., 2019; Miller et al., 2010; Shen & Souza, 2017a, 2018). One possibility that should be explored is a strengthened dynamic pitch has to work with other cues in order to contribute to increased speech intelligibility in noise. Future work is needed to incorporate manipulations of other acoustic cues besides pitch and examine the relative role of dynamic pitch in speech intelligibility.

Second, the increased pitch range found in clear speech is usually accompanied by higher mean pitch (Bradlow et al., 2003; Krause, 2001; Picheny et al., 1986). It is likely, then, that a pitch enhancement strategy that changes pitch range but keeps mean pitch unchanged is not an effective method for improving speech intelligibility in noise. In fact, previous data from clear speech have shown a larger pitch change is mostly driven by higher maximum pitch values, which inevitably increase the mean pitch (e.g., Krause, 2001). Built on this rationale, a new pitch enhancement strategy of asymmetrical pitch exaggeration (i.e., by only increasing the pitch values higher than mean pitch) should be investigated in future research.

Another possible explanation for the negative impact of even mild pitch strengthening is based on evidence suggesting the alteration of dynamic pitch in Lombard speech is not applied globally (i.e., same formula applied across all words) as in the recent pitch manipulation studies. For instance, using an interactive communication task, Patel and Schell (2008) found the information-bearing words were more acoustically enhanced than function words by talkers in terms of duration and pitch in Lombard speech. In another study, an algorithm that enhanced pitch cues on content words was implemented in a speech synthesis system, producing 7% increase of speech intelligibility reported (Patel et al., 2006). These findings suggest future work on developing acoustic enhancement strategies should take into account the linguistic properties of speech content.

Finally, these explanations still do not explain why naturally produced pitch exaggeration is not helpful for speech perception in noise (e.g., Calandruccio et al., 2019). It is worth pointing out though, like the majority of speech-in-noise research, all the studies on the effects of dynamic pitch manipulations have used read sentences as stimuli. As we know, the prosodic cues in read versus spontaneous version of speech are different in multiple aspects, including pitch variation (e.g., Blaauw, 1991; Remez et al., 1986). In addition, the role of dynamic pitch in connected discourse is stronger than it is in separated sentences (e.g., Ayers, 1994). Therefore, it is possible that any positive psycholinguistic effect of strengthened dynamic pitch can only be evident in connected and/or spontaneous speech material. This possibility is currently being examined in a follow-up study.

One interesting finding of this study was a different pattern of noise effect on overall speech perception performance between the younger and older groups. While the younger group perceived speech better in six-talker modulated noise than in the steady-state noise, the older group showed the opposite pattern, albeit with a much smaller effect size (β = − 0.43 for older group vs. β = 1.73 for younger group). As we know the temporal modulation in six-talker modulated noise can provide more glimpsing opportunity for younger listeners with normal hearing, it is expected that this group perceived speech better in six-talker noise than in steady-state noise (e.g., Rosen et al., 2013). On the other hand, the older group's poorer performance in the six-talker noise can potentially be attributed to two reasons. The first one is the inability to glimpse target speech in this temporally modulated noise due to aging and hearing loss (Takahashi & Bacon, 1992). Another reason is that listeners with hearing loss are more susceptible to energetic masking than those with normal hearing (Goossens et al., 2017; G. Kidd et al., 2019). Due to greater spectral overlap, the six-talker noise introduced more energetic masking as compared to the steady-state noise and was a more difficult condition for the older group.

Individual Factors and Effects of Dynamic Pitch Manipulations

The second goal of the study was to examine whether the effects of dynamic pitch manipulations were associated with several individual factors in a group of older listeners. An interesting finding that emerged from the analyses was that those older listeners who were relatively younger (in their 60s instead of over 70 years old) were more negatively affected by manipulated dynamic pitch for speech perception in noise. Built on the rationale that the current pitch manipulation strategy may not faithfully represent pitch exaggeration in clear speech, it is reasonable to argue those who better perceive these pitch cues can be affected more in speech perception. We know the ability to perceive pitch changes with aging process in both music and speech domains (e.g., Russo et al., 2012; Wang et al., 2017), largely due to age-related declines in temporal processing (see Pichora-Fuller & McDonald, 2008, for a review). It is possible that those older listeners who were younger perceived pitch cues better than those who were older and, therefore, were more negatively affected by the manipulated cues for speech recognition in noise. As this study did not find a specific psychoacoustic measure as predictor for dynamic pitch effects, this possibility still awaits future investigation.

As part of a large project exploring perceptual metrics that could predict the effects of dynamic pitch manipulations on speech perception in noise, two spectral tests were used in this study to measure the static and dynamic aspects of spectral perception in the older group. While the results did not support the author's hypotheses on the relationship between the effects of dynamic pitch manipulations and listeners' spectral perception, two possibilities should be considered. First, because the impact of aging on pitch perception ability is modulated by both temporal and spectral mechanisms, it is possible that temporal mechanism is more dominant than the spectral one in explaining older listeners' ability to perceive pitch cues (see also Russo et al., 2012; Schvartz & Chatterjee, 2012) and consequently the effects of dynamic pitch manipulations for speech perception in noise. A second possibility is that the spectral perception tasks employed in this study may not capture the ability that is important for perceiving (and affected by) dynamic pitch. For instance, a narrow-band measure of spectral perception with a focus on the f0 within the range of dynamic pitch stimuli may better serve the purpose of measuring static spectral perception and should be examined in future research.

In conclusion, the study examined the effects of manipulated dynamic pitch cues on older listeners' speech perception in nonspeech noises. We found older listeners as a group performed more poorly with both weakened and strengthened dynamic pitch cues than the original ones. Furthermore, older listeners' responses to manipulated dynamic pitch cues appeared to be modulated by their age. Those listeners who were relatively younger were more negatively affected by dynamic pitch manipulations. Last, older listeners' ability to perceive spectral cues as measured by two spectral perception tests did not significantly predict the effects of dynamic pitch manipulations. These results suggest, as compared to original dynamic pitch cues, the current pitch manipulation strategy does not benefit older listeners for speech perception in noise. The detriment probably stems from multiple factors including distortions of naturally produced prosodic cues and influences from choice of speech stimuli, which should be investigated by future research.

Funding Statement

This work was supported by NIH Grant R21DC017560 (awarded to J. S.). A portion of the data was presented at the Aging and Speech Communication Conference 2019, Tampa, FL.