Cross-modal Plasticity in Developmental and Age-Related Hearing Loss: Clinical Implications

Abstract

This review explores cross-modal cortical plasticity as a result of auditory deprivation in populations with hearing loss across the age spectrum, from development to adulthood. Cross-modal plasticity refers to the phenomenon when deprivation in one sensory modality (e.g. the auditory modality as in deafness or hearing loss) results in the recruitment of cortical resources of the deprived modality by intact sensory modalities (e.g. visual or somatosensory systems). We discuss recruitment of auditory cortical resources for visual and somatosensory processing in deafness and in lesser degrees of hearing loss. We describe developmental cross-modal reorganization in the context of congenital or pre-lingual deafness in childhood and in the context of adult-onset, age-related hearing loss, with a focus on how cross-modal plasticity relates to clinical outcomes. We provide both single-subject and group-level evidence of cross-modal reorganization by the visual and somatosensory systems in bilateral, congenital deafness, single-sided deafness, adults with early-stage, mild-moderate hearing loss, and individual adult and pediatric patients exhibit excellent and average speech perception with hearing aids and cochlear implants. We discuss a framework in which changes in cortical resource allocation secondary to hearing loss results in decreased intra-modal plasticity in auditory cortex, accompanied by increased cross-modal recruitment of auditory cortices by the other sensory systems, and simultaneous compensatory activation of frontal cortices. The frontal cortices, as we will discuss, play an important role in mediating cognitive compensation in hearing loss. Given the wide range of variability in behavioral performance following audiological intervention, changes in cortical plasticity may play a valuable role in the prediction of clinical outcomes following intervention. Further, the development new technologies and rehabilitation strategies that incorporate brain-based biomarkers may help better serve hearing impaired populations across the lifespan.

1. Introduction

The human cortex demonstrates an exquisite capacity for neuroplasticity over the course of the lifespan, capable of adapting to intrinsic and extrinsic forces during development and adulthood, to alterations in sensory input, insult, injury, and learning. Cross-modal plasticity is one such form of cortical neuroplasticity. Cross-modal plasticity can occur as a result of decreased or abnormal sensory input, whereby cortical regions of the deprived modality become vulnerable to the recruitment by the remaining, intact sensory modalities. Intra-modal plasticity is another form of cortical plasticity, whereby brain changes are induced within a particular cortical area as a result of increased or decreased input to that sensory system. Auditory deprivation, as in hearing loss or deafness, may result in cortical cross-modal plasticity, whereby the auditory cortex is recruited for visual or somatosensory processing (Allman et al., 2009; Buckley and Tobey, 2010; Campbell and Sharma, 2016, 2014; Chen et al., 2016; Doucet et al., 2006; Finney et al., 2003; Finney, 2001; Gilley et al., 2008; Giraud and Lee, 2007; Giraud et al., 2001; Kim et al., 2016; Lee et al., 2007; Levänen and Hamdorf, 2001; Meredith and Lomber, 2011; Sharma et al., 2016, 2015; Stropahl et al., 2015). Similar phenomena are well documented in the visual neuroscience literature, in which blindness results in the recruitment of visual cortex for somatosensory (vibrotactile) and auditory processing (see Lazzouni and Lepore, 2014 for a review).

In this review, we will describe current evidence of cross-modal and intra-modal plasticity in hearing impaired populations across the lifespan, with particular focus on how these brain changes may relate to clinical behavioral outcomes. We will discuss the wide range of variability in speech perception outcomes observed in hearing impaired populations, and how cross-modal and intra-modal changes within the sensory cortices may contribute to this variability. As a field, we are beginning to gain a better understanding of other downstream effects of hearing loss, such as compromises in neurocognitive abilities (e.g. working memory deficits) and changes in social-emotional regulation, in both adults and children (Kral et al., 2016). Recently, untreated hearing loss has been linked to increased risk of cognitive decline among older adults, though the potential causal mechanisms underlying this relationship are poorly understood (Contrera et al., 2016; Lin et al., 2011; Lin et al., 2011; Lin et al., 2014; Mick et al., 2014; Peelle et al., 2011).

2. Developmental Sensitive Periods and Cross-modal Plasticity

In children, auditory deprivation leads to delayed or abnormal development of the central auditory pathways, particularly if deprivation occurs during a sensitive period, or an established time window of approximately 3.5 years during which alterations in sensory input (e.g. deafness or hearing loss) can lead to profound and long-term impacts on the brain (See Kral and Sharma, 2012; Sharma et al., 2009, 2002 for discussion on the sensitive period for cortical maturation in deaf children). Animal studies suggest that auditory deprivation, especially that which is allowed to continue beyond the sensitive period, alters functional connectivity within the auditory system, between sensory systems and between the auditory system and higher-order neuro-cognitive centers resulting in significant deficits in brain and behavior (including sequence processing, working memory, executive functioning and concept formation) (Kral et al., 2016). One form of the afore-mentioned change in cortico-cortico connectivity is cross-modal re-organization between the auditory system and other sensory systems (e.g. vision). Animal studies suggest that sensory repurposing of auditory cortices appears to occur in higher-order sensory cortices as opposed to primary auditory cortices (Kral and Sharma, 2012; Kral et al., 2003). A recent paper (Land et al., 2016) examined visual responsiveness in a higher-order auditory cortical area (dorsal zone or DZ), which has been implicated in cross-modal re-organization (Lomber et al., 2010). A small number of visually responsive neurons were found in DZ in congenitally deaf cats. However, the vast majority of neurons in DZ showed auditory responsivity. Further, the visual and auditory neurons formed distinct populations that did not interact, suggesting that visual cross-modal re-organization does not decrease auditory responsiveness in congenitally deaf cats. Thus, while cross-modal recruitment of higher-order auditory areas is likely involved in closing developmental sensitive periods in deafness (Kral & Sharma, 2012; Kral, 2007), it appears that auditory responsivity is maintained despite cross-modal re-organization by vision (Land et al., 2016).

2.1. Visual and Somatosensory Cross-Modal Plasticity in Deaf Cochlear Implanted Children

Cross-modal re-organization by vision has been observed in developing and adult animals and humans with congenital or pre-lingual onset of deafness (Neville & Lawson, 1987; Buckley and Tobey, 2010; Dewey and Hartley, 2015; Doucet et al., 2006; Finney et al., 2003; Finney, 2001; Lee et al., 2001, 2007; Lomber et al., 2010). In congenitally deaf cats, for example, it appears that enhanced peripheral localization abilities observed in these animals is sub-served by the posterior auditory field (Lomber et al., 2010). That is, while the deaf cats show enhanced peripheral visual abilities compared to normal hearing cats, the temporary deactivation of the posterior auditory cortex leads to a depression in these abilities. Similarly, enhanced visual motion detection appears to be sub-served by dorsal auditory cortex (Lomber et al., 2010). More recently, increased performance in visual motion detection abilities has been shown in humans with pre-lingual hearing loss onset (Hauthal et al., 2013; Shiell et al., 2016).

Like deaf cats, cross-modal cortical re-organization by the visual modality has been documented in congenitally deaf children fitted with CIs. In a recent study by our laboratory, cortical visual evoked potentials (CVEPs) were recorded using 128-channel high-density EEG in a group of CI children (n=14) and an age-matched group of normal hearing children in response to a radially modulated visual grating stimulus giving the effect of apparent motion, (see Campbell and Sharma, 2016 for details and methodology used). In the group of CI children, the average age of first implant was 3.12 years (sd = +/−2.27 years), and the average age of implantation of the second ear was 6.20 years (sd = +/−3.45 years). Thus, while the average age of implantation for the first ear fell towards the end of the sensitive period of approximately 3.5 years, the average age of implant for the second CI was well beyond the sensitive period. The CI children in this study exhibited larger CVEP amplitudes with earlier CVEP latencies compared to the age-matched, normal hearing group. Further, the underlying cortical generators differed between the two groups. As depicted in Fig. 1A, the normal hearing children demonstrated activation of cortical regions associated with typical processing of visual motion stimuli (e.g. cerebellum, striate, extra striate) for all CVEP components (Bertrand et al., 2012; Campbell and Sharma, 2014), while the CI children demonstrated additional activation of right lateral temporal cortex for the higher-order N1 and P2 CVEP components (e.g. right inferior, middle, and superior temporal gyrus. Furthermore, speech perception performance in background noise using a clinical test (BKB SIN) was significantly negatively correlated with CVEP latency for CI children, such that poorer speech perception was associated with earlier CVEP N1 latency. Given that listening in noise is one of the most difficult acoustic environments for CI children, Campbell and Sharma’s results suggest that difficulty processing speech in background noise for CI children is related to compensatory cross-modal re-organization by vision, presumably since attention to visual cues helps to disambiguate the speech signal in noise. These findings are supported by a series of Positron Emission Tomography (PET) studies by Lee and colleagues (Lee et al., 2005, 2007), in which poor speech perception outcomes after CI in pre-lingually deaf children who were implanted at later ages were correlated with higher levels of resting glucose metabolism over temporal, frontal, and visual cortices, supporting the notion that auditory cortices had become repurposed (Lee et al., 2005, 2007). Given that children in the Campbell and Sharma and Lee studies as a whole were not implanted early in life, by the FDA age of 12 months, future studies should examine the effects of age of implantation on possible synesthesia and developmental effects of cross-modal re-organization in deafness.

FIGURE 1. Visual Cross-Modal Plasticity in CI Children.

Panel 1A: Visual cross-modal re-organization in a group of CI children. Visual cortical evoked potential (CVEP) current density source for a group of normal hearing children (n=41) and a group of CI children (n=14) in response to a visual motion stimulus. For the higher-order CVEP components, cortical areas involved in processing of visual motion stimuli are observed for the normal hearing group, whereas the CI children show additional recruitment of temporal cortex. Adapted from Campbell & Sharma (2016).

Panel 1B: Visual cross-modal re-organization in individual CI children. Top Panel: Current density source reconstructions for the P2 cortical visual evoked potential (CVEP) component in a normal hearing child, a cochlear implanted child with excellent speech perception, and a CI child with average speech perception recorded in response to a visual motion stimulus. The normal hearing child and CI child with good speech perception show expected activation of cortical areas involved in processing of visual motion stimuli, while the CI child with average speech perception shows additional recruitment of temporal cortices. Bottom Panel: Current density source reconstructions for the N70 cortical somatosensory evoked potential (CSSEP) component in a normal hearing child, a CI child with excellent speech perception, and a CI child with average speech perception recorded in response to a 250 Hz tone applied to the right index finger. Vibrotactile stimulation in the normal hearing and CI child with excellent speech perception show expected activation of cortical areas involved in somatosensory processing, while the CI child with average speech perception shows additional recruitment of auditory cortex. Adapted from Sharma, Campbell, & Cardon (2015).

Evidence cross-modal plasticity by vision over right temporal cortex in CI children reported by Campbell & Sharma (2016) is consistent with evidence of cross-modal reorganization by vision over the right temporal lobes observed in adult CI users (Sandmann et al., 2012) and in congenitally deaf adults (Finney et al., 2003; Fine et al., 2005). Thus, while it appears that both the right and left temporal cortices may be susceptible to cross-modal plasticity as a result of hearing loss, there is evidence that the right temporal cortex may be more susceptible to the effects of sensory deprivation, whereas left temporal cortex plasticity may be more linguistically driven (Cardin et al., 2013; Cardin et al., 2016).

More recently, cross-modal plasticity between the auditory and somatosensory modalities has been documented in pediatric deafness. In children with long durations of deafness, there is evidence to suggest that the somatosensory cortex may be activated in response to auditory stimulation. For instance, in a study by Gilley, Sharma, & Dorman (2008), cortical auditory evoked potentials (CAEPs) were recorded in a group of normal hearing children (n=9), an age-matched group of children with pre-lingual deafness who were early implanted (n=8, mean age at implant = 2.79 years, sd = +/− 0.78), and a group of children with pre-lingual deafness who were late implanted (n=8, mean age at implant = 11.33 years, sd = +/−1.12) in response to a speech stimulus (Gilley et al., 2008). While the normal hearing and early-implanted children in this study demonstrated expected activation of auditory cortex (superior temporal gyrus, inferior temporal gyrus), the late-implanted group showed significant activation of post-central gyrus in somatosensory cortex. Activation of somatosensory cortices in the late-implanted children suggests abnormal functional connectivity for processing of auditory stimuli and has been associated with poor outcomes with the CI in anecdotal reports.

Other forms of cross-modal plasticity by the somatosensory system have been documented in CI children. Emerging evidence from our laboratory indicates that temporal cortex may be recruited for somatosensory processing in children with CIs (Cardon, 2015). In this study, cortical somatosensory evoked potentials (CSSEPs) were recorded in response to a 250 Hz vibrotactile stimulus applied to the index finger in a group of CI children (n=13, mean age at test=12.38 years, mean age at first implant=3.90 years [sd = +/− 4.03 years], mean age at second implant=7.33 years [sd = +/− 4.47 years]) and an age-matched group of normal hearing children (n=35, mean age at test = 10.54 years, sd = +/− 4.03 years) (Cardon, 2015). Results of this study suggest that regions of contralateral auditory cortex (inferior temporal gyrus, middle temporal gyrus, superior temporal gyrus) and somatosensory cortex (pre/post central gyrus) are activated by vibrotactile stimuli in CI children, whereas the vibrotactile stimulus only elicits activation in contralateral somatosensory cortex (pre/post central gyrus) in the normal hearing group. Somatosensory cross-modal re-organization has been similarly observed in congenitally deaf adults (Levänen and Hamdorf, 2001; Levänen et al., 1998; Schürmann et al., 2006), and cross-modal recruitment by the somatosensory system has been reported in blindness, vision loss, and as a result of adult-onset visual deprivation (Collignon et al., 2015; Merabet et al., 2004; Niechwiej-szwedo et al., 2016; Sadato et al., 2005). As the aforementioned studies suggest, higher-order auditory cortex appears to be recruited by visual and somatosensory modalities in childhood deafness.

2.2. Is There a Functional Correlate to Cross-Modal Plasticity?

It has been proposed that the recruitment of auditory cortex by vision may be functionally correlated to increased reliance on visual cues as a result of auditory deprivation or degradation. In children, Bergeson et al (2005) reported that late-implanted pediatric CI recipients exhibited higher levels of visual-only speech perception (lip-reading abilities) and auditory-visual gains (speech perception performance in auditory-visual task compared to performance in the auditory-only task), in relation to early-implanted children exhibiting higher auditory-only speech perception scores (Bergeson et al., 2005). In fact, it appears that CI recipients continue to show improvement in visual-only and auditory-visual speech perception several years after CI (Bergeson et al., 2005; Tyler et al., 1997). Similarly, in a study by Schorr et al. (2007) which examined audio-visual integration in the pediatric population, children with CIs were more dependent on visual cues for the McGurk effect compared to age-matched, normal hearing children (Schorr et al., 2007). This behavioral evidence is substantiated by electrophysiological evidence. In the aforementioned pediatric study by Campbell & Sharma (2016), for example, the group of CI children demonstrated larger P2 CVEP amplitudes compared to the normal hearing children, suggesting increased visual intra-modal plasticity (Campbell and Sharma, 2016). Given the role of visual cues in naturalistic and/or difficult communication environments, increased reliance on visual cues may result in the cross-modal recruitment of auditory cortical areas for visual processing, and/or increased intra-modal plasticity within the visual system, particularly in early childhood when a child is just beginning the language learning process. While an increased reliance on vibrotactile cues is consistently reported in the blindness literature, a functional correlate of somatosensory cross-modal plasticity in hearing loss may appear less intuitive than cross-modal recruitment by vision (Burton et al., 2004; Sadato et al., 2005, 1998, 1996). It is possible that both the close proximity of auditory and somatosensory cortices and the overlap of neurons responding to auditory and somatosensory inputs sub-cortically may give rise to cross-modal recruitment of auditory cortex by the somatosensory system as a result of hearing impairment (Allman et al., 2009; Dehmel et al., 2008; Kanold and Young, 2001; Meredith and Lomber, 2011; Shore and Zhou, 2006). Finally, future research should focus on examining the functional involvement of vibrotactile (somatosensory) inputs in speech production and perception in the normal hearing population and in the hearing loss population (Tremblay et al., 2003).

2.3. Can Cross-Modal Re-organization Be Used to Predict Clinical Outcomes?

What are the clinical implications of cross-modal plasticity in individual children with hearing loss? There is group-level evidence to suggest that cross-modal plasticity may be related to CI outcomes in children. For example, in a series of studies by Lee and colleagues (2007) and Giraud & Lee (2007), pediatric and adult CI patients exhibiting higher levels of metabolic activity in dorsolateral pre-frontal cortex demonstrated higher speech perception scores, whereas those CI patients exhibiting higher levels in ventral visual processing regions demonstrated lower speech perception scores. These authors have suggested a dorsal/ventral dichotomy of cortical areas for good versus poor clinical outcomes, respectively (Giraud and Lee, 2007; Lee et al., 2007). The afore-described Campbell and Sharma (2016) study showed a negative correlation between visual cross-modal re-organization and speech perception performance in noise with the CI (Campbell and Sharma, 2016). These studies suggest that cross-modal plasticity may be a predictor of behavioral outcomes with the CI in groups of children.

Recently in our laboratory, we have begun to examine cross-modal changes in individual patients to assess whether we can develop biomarkers of clinical performance. In a recent study (Sharma et al., 2015), we reported visual and somatosensory cross-modal plasticity in six individual children with CIs. Fig. 1B shows current density source reconstructions (CDRs) for the P2 CVEP in 3 children: a normal hearing child (age 10 years), a pediatric CI user (age 8 years) exhibiting excellent speech perception (96% on Lexical Neighborhood Test), and a pediatric CI user (age 6 years) exhibiting average speech perception (67% on Multisyllabic Lexical Neighborhood Test). For both the normal hearing child and the good CI performer with excellent speech perception, these children demonstrated expected activation of visual areas associated with visual motion processing for the P2 CVEP component (e.g. occipital gyrus, fusiform gyrus, lingual gyrus), suggesting minimal or no cross-modal recruitment of auditory cortex. In contrast, the average CI performer showed additional activation of regions typically associated with auditory processing (e.g. middle and superior temporal gyrus) in response to the visual motion stimulus, suggestive of cross-modal recruitment by vision. Fig. 1B also shows CDRs for the N70 CSSEP for 3 different children: a normal hearing child (age 7 years), a pediatric CI user (age 13 years) exhibiting excellent speech perception (94% on Consonant Nucleus Consonant (CNC) test), and a pediatric CI user (age 15 years) exhibiting average speech perception performance (76% on the CNC test). Stimulation of the right index finger via a vibrotactile stimulus resulted in activation of classic somatosensory cortices in the normal hearing child and pediatric CI performer with excellent speech perception (e.g. post-central gyrus), whereas additional recruitment of auditory cortical areas (e.g. superior temporal gyrus, transverse temporal gyrus) was observed in the pediatric CI user with average speech perception, suggestive of cross-modal recruitment.

While this evidence stems from case studies and thus should be interpreted cautiously, there exists great potential for the development of biomarkers of cross-modal re-organization to help predict individual outcomes following audiological intervention with a hearing aid or CI. Knowledge of compensatory mechanisms may help interventionists individualize rehabilitation and training programs for deaf children. It is possible that multi-modal approaches to rehabilitation following CI may be more beneficial for certain patients than others (Isaiah & Hartley, 2015; Isaiah et al., 2014). From a clinical perspective, research in this area may lead to the development of targeted rehabilitation programs based on the cortical organization profile of the child, in conjunction with their audiological profile.

2.4. Can Cross-Modal Re-organization Reverse Following Audiological Intervention?

In a previously published study, we have demonstrated the potential for complete or partial reversal of cross-modal plasticity in a pediatric case of progressive single-sided deafness (SSD). This child was identified with a moderate sensorineural hearing loss in her right ear at age 5 years, which progressed to severe-profound by the time she was 9 years old. The child has normal hearing in her left ear. At the age of 9.86 years, she received a CI in her right ear. In this study, we tracked developmental cortical neuroplasticity in this child before and over the course of 2 years after CI. We recorded CAEPs in response to a speech stimulus, CVEPs in response to a visual motion stimulus, and CSSEPs in response to a vibrotactile stimulus delivered to the right index finger using 128-channel high-density EEG (see Sharma et al., 2016 for methodology used). Current density source reconstruction results are depicted in response to the visual and somatosensory stimuli pre-CI and 27 months post-CI in Fig. 2A and 2B, respectively. Whereas pre-CI the SSD child demonstrated recruitment of the left temporal cortex for somatosensory processing (middle temporal gyrus) in addition to expected somatosensory activation (pre-central gyrus, post-central gyrus, inferior and superior parietal lobule), we saw a complete reversal of recruitment of auditory (temporal) cortex by somatosensory processing by 27 months post-CI. Similarly, partial reversal was observed for the visual modality. Pre-CI the SSD child demonstrated recruitment of the left temporal cortex (inferior temporal gyrus, middle temporal gyrus) and frontal cortex (inferior frontal gyrus) for visual processing, whereas post-CI the child showed more typical activation of higher order visual areas including fusiform gyrus, as well as residual activation of superior and middle temporal gyrus. We also observed the restoration of more typical auditory activation patterns in the SSD ear within 14 months post-CI (Sharma et al., 2016). The more typical cortical networks after CI (including reversal or partial reversal of cross-modal re-organization, reduced frontal activation, and decreased auditory dominance of the normal hearing ear) was accompanied by the child’s improved performance with her CI. By 33 months post-CI, the child’s speech perception in noise performance was comparable to other SSD-CI adults and her sound localization ability was comparable to normal hearing adults (Sharma et al., 2016). Given the fact that CIs are not currently approved by the Federal Drug Administration in cases of SSD, these data add to the growing body of evidence supporting potential benefits of CI in SSD, including improvements in speech discrimination, sound localization, tinnitus reduction, and/or quality of life (Arndt et al., 2015, 2011; Cadieux et al., 2013; Dorman et al., 2015; Friedmann et al., 2016; Távora-Vieira et al., 2013). This case study provides a demonstration of how biomarkers of cross-modal plasticity could be used to assess the effects of audiological intervention, in conjunction with conventional behavioral measures. Future studies should systematically examine cross-modal plasticity before and after CI in larger populations of children and adults with SSD, as well as other hearing impaired populations following audiological intervention.

FIGURE 2. Cross-Modal Re-organization in Pediatric Single-Sided Deafness Following Cochlear Implantation.

Panel 2A: Current density source reconstructions for the P2 cortical visual evoked potential (CVEP) pre-CI and at 27 months post-CI. CVEPs were recorded in response to a visual motion stimulus. While pre-CI there is evidence of recruitment of temporal cortex for visual motion processing, post-CI, there is partial reversal of recruitment of temporal cortex by vision.

Panel 2B: Currently density source reconstructions for the N70 cortical somatosensory evoked potential pre-CI and at 27 months post-CI. SSEP responses were recorded in response to a vibrotactile stimulus. While pre-CI there is evidence of recruitment of temporal cortex for somatosensory processing, post-CI there is complete reversal of the recruitment of temporal cortex for somatosensory processing.

Figure adapted from Sharma et al. (2016).

3. Visual and Somatosensory Cross-modal Plasticity in Adults with Hearing Loss

The recruitment of auditory cortical regions for visual processing has been long-documented in both pre-lingually and post-lingually deaf adults, and in adults with profound hearing loss receiving CIs (Buckley and Tobey, 2010; Finney et al., 2003; Finney, 2001; Kim et al., 2016; Sandmann et al., 2012). For example, in a recent study by Sandmann et al (2012), CVEPs were recorded in a group of normal hearing adults, and a group of CI participants with adult-onset hearing loss, many of whom had been deaf as briefly as 1 year prior to CI (Sandmann et al., 2012). In their study, the CI group exhibited increased activation in the auditory cortex compared to the normal hearing group, and significant correlations were observed between amount of auditory activation in right auditory cortex and speech intelligibility (Sandmann et al., 2012). Increased CVEP amplitudes over temporal cortex have also been reported in pre-lingually deaf CI adults (Buckley and Tobey, 2010) and post-lingually deaf CI adults (Doucet et al., 2006; Kim et al., 2016), and are negatively correlated with behavioral speech perception abilities in quiet and in noise, further supporting the idea that cross-modal re-organization may occur in adult-onset hearing loss. Interestingly, Buckley and Tobey (2010) found no significant association between duration of deafness and N1 CVEP amplitude in either pre-lingually or post-lingually deafened CI adults, suggesting that cross-modal re-organization may be induced by auditory deprivation itself, regardless of the duration of auditory deprivation prior to cochlear implantation.

Further, it appears that some populations with hearing loss show more extensive cross-modal and intra-modal re-organization than others. For example, Doucet et al. (2006) reported that pre- and post-lingually deaf CI adults who were good users (demonstrating excellent speech perception abilities) showed cortical activation patterns restricted to visual cortex in response to visual motion stimuli (e.g. intra-modal plasticity), whereas pre- and post-lingually deaf CI adults who demonstrated poorer speech perception abilities with the implant showed more widespread cortical activation, spreading anteriorly into temporal cortices (e.g. cross-modal plasticity) (Doucet et al., 2006). Based on a series of studies by Strelnikov and colleagues (2009, 2013, 2015), it also appears that improvements in speech perception in adults post-implantation may at least in part rely on the initial functional level of the visual cortex (Barone et al., 2013; Strelnikov et al., 2015, 2013, 2009). That is, intra-modal changes within the visual modality in post-lingual deafness may influence auditory behavioral outcomes following CI. Strelnikov et al. (2009) reported increased metabolic activity in occipital cortex and decreased metabolic activity in temporal cortex in CI adults at rest compared to normal hearing controls (Strelnikov et al., 2009). In a later PET study, the Strelnikov group also found that higher levels of metabolic activity in the visual cortex in post-lingually deafened CI adults shortly after implantation was positively correlated with auditory speech perception recovery 6 months after implantation, and this correlation was significant across resting state, visual-only, and auditory-visual conditions (Strelnikov et al., 2013). In contrast, a negative correlation was observed between levels of metabolic activity in superior temporal gyrus shortly after implantation and auditory speech perception recovery 6 months after implantation across conditions. In a recent study (Kim et al., 2016), cross-modal and intra-modal cortical activation was examined in a group of post-lingually deafened CI adults with poor speech perception scores and a group of post-lingually deafened CI adults with good speech perception scores. In this study, poor CI performers exhibited larger P1 CVEP amplitudes in the right temporal cortex compared to good CI performers. Further, P1 CVEP amplitude over right temporal cortex was negatively correlated with speech perception scores (evidence of cross-modal cortical plasticity), whereas P1 CVEP amplitude over occipital cortex was positively correlated with speech perception scores (evidence of intra-modal cortical plasticity) (Kim et al., 2016). Taken together, these studies provide evidence of cross-modal reorganization, marked by decreased activity over temporal cortex, as well as evidence of intramodal plasticity marked by increased activity over occipital cortex, in adult CI users (Lee et al., 2007; Strelnikov et al., 2013; Kim et al., 2016).

3.1. Cross-modal Plasticity in Mild-Moderate Age-Related Hearing Loss

Increasingly, it has become apparent that cross-modal plasticity is not just restricted to severe-profound hearing loss and/or long durations of deafness. Recently, our laboratory examined whether cortical cross-modal plasticity is evident in adults with lesser degrees of hearing loss. In this study by Campbell & Sharma (2014), CVEPs were collected in response to the visual-motion stimulus in a group of adults with bilateral, early-stage, mild-moderate hearing loss, and an age-comparable group of adults with normal hearing. Many of the participants in the hearing loss group were unaware that they had a hearing loss upon time of enrollment in the study. High-density EEG source localization of the underlying CVEP components was performed in order to visualize cortical activation patterns in the two groups (see Campbell and Sharma, 2014 for methodology used). The current density source reconstruction results for the P2 CVEP component is shown in Fig. 3C. While the normal hearing group demonstrated dominant activation in cortical areas associated with visual motion processing (cerebellum, fusiform gyrus, lingual gyrus) for all CVEP components (P1, N1, P2), the hearing loss group showed recruitment temporal cortex for the N1 and P2 CVEP components (inferior temporal gyrus, medial temporal gyrus, superior temporal gyrus) and frontal cortical regions (inferior frontal gyrus, BR 47), suggestive of cross-modal recruitment of auditory cortex for visual processing (Fig. 3C). In addition, the hearing loss group demonstrated significantly higher P1, N1, and P2 CVEP amplitudes in comparison with the normal hearing group, and earlier N1 CVEP latencies in the hearing loss compared to normal hearing group. The N1 CVEP latency was significantly negatively correlated with behavioral speech discrimination in background noise. Based on these data, it appears that even mild auditory deprivation may induce cross-modal plasticity. Recruitment of auditory cortex for visual-motion processing has been observed in deaf adults and adult CI users using both functional near-infrared spectroscopy (fNIRS) techniques (Chen et al., 2016; Dewey and Hartley, 2015), electroencephalography (EEG) (Buckley and Tobey, 2010; Doucet et al., 2006; Karns et al., 2012; Neville and Lawson, 1987), and functional magnetic resonance imaging (fMRI) (Shiell et al., 2016), but the results from Campbell and Sharma (2014) are the first to suggest that cross-modal recruitment by vision appears to occur in age-related, mild-moderate hearing loss.

FIGURE 3. Changes in Cortical Resource Allocation in Adults with Age-related Mild-Moderate Hearing Loss.

Cortical visual evoked potential (CVEP) and cortical auditory evoked potential (CAEP) source reconstructions for the P2 components are shown in a group of normal hearing adults (n=8) and a group of hearing loss adults with bilateral, early-stage, mild-moderate hearing loss (n=9). CVEPs were recorded in response to a visual motion stimulus. CAEPs were recorded in response to an auditory speech stimulus.

Panel 3A: While the normal hearing group shows dominant activation of temporal cortex in response to the auditory stimulus, the hearing loss group shows visible decreased activation of temporal cortex in response to auditory stimulation.

Panel 3B: In addition to temporal activation in response to the auditory stimulus, the hearing loss group shows the recruitment of frontal cortex for auditory processing that is not present in the normal hearing group.

Panel 3C: While the normal hearing group shows dominant activation in cerebellar and visual cortical areas in response to the visual motion stimulus, the hearing loss group shows significant recruitment of temporal cortex and frontal cortex, in addition to visual processing regions.

Panel 3D: A framework for changes in cortical resource allocation in early-stage, mild-moderate hearing loss is described, in which decreased input to auditory cortex (as in deafness or hearing loss) taxes the brain, results in compensatory recruitment of frontal cortices for top-down modulation of sensory processing and cross-modal recruitment of auditory cortex by vision likely associated with a greater reliance on visual cues to help disambiguate the speech signal. Figure adapted from Campbell & Sharma (2013) and Campbell & Sharma (2014).

Recruitment of auditory cortex for somatosensory processing has been reported in congenital and early-onset deafened adults (Karns et al., 2012; Levänen et al., 1998; Sharma et al., 2007), similar to studies in blindness, which demonstrate the recruitment of visual cortices for somatosensory processing (Burton et al., 2004; Sadato et al., 1998, 1996). In the animal model, an increase in auditory cortical neurons responding to somatosensory inputs in both cats and ferrets with adult-onset deafness was reported (Allman et al., 2009; Wong et al., 2015). Preliminary data from our laboratory provides evidence of somatosensory cross-modal cortical plasticity in adults with early stage, mild-moderate hearing loss in adults, and in adult-onset SSD. Future research examining cross-modal re-organization in lesser degrees of hearing loss may allow us to understand the timeframe and mechanisms underlying cross-modal plasticity in early-stage hearing loss.

3.2. How Soon After Hearing Loss Onset Does Cross-modal Plasticity Occur?

Currently, it is not well understood how short-term cross-modal changes take place in humans after the onset of hearing loss or deafness. In animal models, somatosensory cross-modal plasticity was observed over a relatively short time course after hearing loss onset, as early as 16 days after deafness onset (Allman et al., 2009). Recently, we had the opportunity to examine the time frame of deprivation-induced visual cross-modal plasticity in an adult (male, age 62 years) who sustained a sudden mild sloping-to-severe sudden bilateral sensorineural hearing loss following a viral infection. We documented changes in visual cross-modal cortical neuroplasticity by recording CVEPs in response a visual-motion stimulus using high-density 128-channel EEG shortly after hearing loss onset, 3 months, and 1 year after the sudden hearing loss. Current density source reconstruction for the P2 CVEP at each test session is shown in Fig. 4A. Shortly after hearing loss onset, source localization showed very little recruitment of auditory cortex, with dominant activation occurring in cerebellar and visual processing regions, consistent with typical processing of this visual motion stimulus (Campbell and Sharma, 2014). At this first test session, the participant scored 84% correct on a sentence level test of auditoryvisual speech perception in noise. By 3 months after initial onset of the hearing loss, we observed activation in visual areas (fusiform gyrus) as well as activation of temporal auditory processing areas (middle temporal gyrus, inferior temporal gyrus) and frontal activation (inferior frontal gyrus) in response to the same stimulus, and auditory-visual speech perception remained the same (85% correct), suggestive of cross-modal recruitment. Then, by one year after sudden onset of the hearing loss, we observed continued recruitment of visual, temporal, and frontal cortices (middle temporal gyrus, inferior temporal gyrus, superior temporal gyrus, fusiform gyrus, inferior frontal gyrus), and a 15% functional improvement in auditory-visual speech perception scores (100% correct). These findings are from a single case study and they should be interpreted cautiously. However, it is often hard to document the exact time of onset of hearing loss in adult age-related hearing loss. In this patient in whom we were able to document the onset of the hearing loss, cross-modal plasticity was evident at least 3 months after the hearing loss onset, although it is possible it might have occurred prior to the 3-month time point. Given recent evidence that declines in auditory function may induce neural changes not detectible in conventional audiometric testing (Kujawa and Liberman, 2009), biomarkers of cross-modal reorganization could be useful in the identification of brain-based changes in the early stages of hearing loss, and may helpful in determining the timeframe in which cross-modal plasticity occurs following the onset of hearing loss in addition to the clinical course of intervention.

FIGURE 4. Visual Cross-modal Re-organization in Individual Adult Patients with Hearing Loss.

Panel 4A: Timeframe for cross-modal re-organization following hearing loss onset: Current density source reconstructions for the P2 cortical visual evoked potential component in an adult with a sudden-onset mild sloping to severe bilateral sensorineural hearing loss. Results are shown shortly after hearing loss onset, 3 months, and 1 year after the sudden hearing loss. Auditory-visual speech perception scores (percent correct) are also shown at the bottom on Panel A. Whereas shortly after the hearing loss onset the visual motion stimulus elicits activation of cerebellar and visual processing areas, within 3 months there is significant cross-modal recruitment of temporal cortex for visual motion processing. By 1 year there is a functional increase in the patient’s auditory-visual speech perception score in addition to the continued cross-modal recruitment by vision.

Panel 4B: Cross-modal re-organization as a predictor of hearing aid performance: Current density source reconstructions for the P2 cortical visual evoked potential component in an adult with bilateral mild-moderate hearing loss who is a good hearing aid user with excellent speech perception in noise performance (score of 1 dB SNR on the QuickSin, a routinely used clinical assessment of speech perception in noise), and an adult with bilateral mild-moderate hearing loss fitted with hearing aids who has moderate difficulty processing speech in noise (score of 8 dB SNR on the QuickSin). Both patients had used bilateral hearing aids for at least 18 months.

While the good hearing aid user shows expected activation of only cortical regions associated with visual motion processing, the hearing aid user with moderate speech perception processing difficulty shows activation of auditory cortical regions in response to the visual stimulus suggestive of cross-modal recruitment.

3.3. Functional Significance of Cross-modal Plasticity in Adults with Hearing Loss

The recruitment of auditory cortex by vision in adult-onset hearing loss may hold functional significance. Enhanced visual-only speech perception abilities have been observed in post-lingually deafened CI adults compared to normal hearing adults (Stropahl et al., 2015). Further, in a face processing task, these post-lingually deaf CI users demonstrated larger activation of occipito-temporal cortical regions, while the normal hearing adults showed primarily occipital cortical activation, demonstrating recruitment of temporal (auditory) cortices for visual processing (Stropahl et al., 2015). Similarly, enhanced audiovisual integration and visual-only (lip-reading) abilities have been reported in older adults receiving CIs (Hay-McCutcheon et al., 2005), deaf adults using sign language (Mitchell et al., 2013), and adults with mild-moderate hearing loss (Tye-Murray et al., 2007). Cross-modal recruitment of auditory cortex by the somatosensory system may relate to the close proximity of auditory and somatosensory cortices (Allman et al., 2009), the overlap of the auditory and somatosensory pathways sub-cortically (Dehmel et al., 2008; Schürmann et al., 2006; Shore and Zhou, 2006), or increased reliance on vibrotactile (somatosensory) inputs known to play a role in auditory speech perception and production (Gick and Derrick, 2009; Ito et al., 2009; Skipper et al., 2007). Cross-modal re-organization by vision and somatosensation in adult-onset hearing loss may, in result, occur from an unmasking of multisensory (e.g. auditory-visual) or unimodal (e.g. somatosensory) pathways latent in the mature auditory cortex (Allman et al., 2009).

With further research, brain-based markers of cross-modal re-organization in adults with hearing loss may prove useful in predicting individual outcomes following audiological intervention. Our laboratory is currently examining visual cross-modal plasticity in individual cases of adults with hearing loss following intervention with hearing aids. Fig. 4B demonstrates CVEP source localization for the P2 component in response to our visual motion stimulus in two adults with hearing loss who had worn bilateral hearing aids for a duration of at least 18 months. Both adults had a bilateral sensorineural hearing loss that was mild-moderate in nature, but these patients presented with very different speech perception abilities with their hearing aids. The good hearing aid user (female, age 57 years) demonstrated excellent speech perception in background noise (1 dB SNR on the QuickSin, a widely used clinical threshold test evaluating sentence-level speech perception in background noise), while the poor hearing aid user (female, age 64 years) presented with greater difficulty in speech perception in background noise (8 dB SNR on the QuickSin). As seen in Fig. 4B, while the good hearing aid user shows expected activation in visual cortical areas (cerebellum, fusiform gyrus, inferior semi lunar lobule, culmen), the hearing aid user who has greater difficulty in speech perception, shows recruitment of temporal cortices in response to visual stimulation (middle temporal gyrus, superior temporal gyrus, inferior temporal gyrus), suggestive of cross-modal re-organization. While these are case study data and should be interpreted as such, these examples illustrate the possible clinical applications of cross-modal plasticity to help interpret behavioral outcomes following audiological intervention. Future studies should examine the impact of both aging and hearing loss on cross-modal plasticity.

4. Recruitment of Frontal Cortical Networks for Sensory Processing in Hearing Loss

While the previously described studies focused on cross-modal and intra-modal changes in hearing loss, there are often other simultaneous changes in cortical resource allocation that occur as a result of auditory deprivation. In the same group of normal hearing adults and adults with early-stage, mild-moderate hearing loss reported in Campbell & Sharma (2014) who showed cross-modal re-organization by vision, we recorded CAEPs in response to a speech stimulus in these subjects using high-density EEG (see Campbell and Sharma, 2013 for details regarding methodology). Results from this study are depicted in Fig. 3A and 3B. Notably, we observed decreased temporal activation in the hearing loss group (Fig. 3A), consistent with MRI studies showing accelerated atrophy and decreased volume of gray matter over right temporal lobes in the hearing impaired adults (Lin et al., 2014; Peelle et al., 2011; Wingfield et al., 2006). Further, while auditory stimulus elicited activation of auditory processing regions in the normal hearing adults (inferior temporal gyrus, middle temporal gyrus, superior temporal gyrus), consistent with normal processing of these auditory stimuli, the adults with early-stage hearing loss adults showed additional recruitment of frontal cortex (medial frontal gyrus, superior frontal gyrus) for all CAEP components (Fig. 3B). This finding is consistent with other studies which show increased activation of frontal cortices during degraded listening situations in adults with normal hearing and adults hearing loss (Peelle et al., 2011; Wingfield and Peelle, 2015) as well as deficits in cognitive and central auditory processing in CI adults (Henkin et al., 2014; Finke et al., 2015). Central auditory deprivation, decreased audibility, and auditory degradation—as in hearing loss—may contribute to changes in sensory and cognitive cortical resource allocation. Thus, recruitment of frontal cortices in hearing loss may help patients compensate in speech perception and/or higher-order cognitive processing tasks (Cardin et al., 2016).

Results from the Campbell & Sharma (2013) and Campbell & Sharma (2014) studies have allowed us to develop a framework of cortical resource allocation changes in early-stage, age-related hearing loss, as described in Fig. 3D. First, lack of normal auditory input results in decreased intra-modal activation of auditory cortex likely leading to difficulties in speech perception. This is accompanied by compensatory cross-modal re-organization of auditory cortex by vision likely due to increased reliance on visual cues to help disambiguate the degraded speech signal. In tandem, frontal cortices are recruited in an effort to improve sensory perception via top-down modulatory control (Campbell and Sharma, 2013; Peelle et al., 2011; Sharma et al., 2016; Wingfield and Peelle, 2015; Wingfield et al., 2006). Given increasing evidence of the cognitive and neural consequences of untreated hearing loss in adults, including working memory and executive function deficits, and increased risk for all-cause dementia (Contrera et al., 2016; Lin et al., 2011; Lin et al., 2011; Lin et al., 2014; Mick et al., 2014; Peelle et al., 2011), future research should aim to refine this framework by examining the link between cross-modal recruitment by vision and intra-modal plasticity (decreased auditory activation and increased frontal activation) as it relates to neurocognitive function.

5. Summary & Conclusions

As we have described in this review, cross-modal plasticity occurs as a result of auditory deprivation across the lifespan. Animal and human models show evidence of cross-modal plasticity in congenital deafness where it appears to be restricted to higher-order auditory cortex. Cross-modal plasticity by vision in congenital deafness is likely due to compensatory dependence on the visual modality for communication especially in difficulty listening situations. The implication of cross-modal plasticity in deaf children needs to be further investigated. For example, it would be beneficial to know whether cross-modal re-organization is absent in children who receive cochlear implants very early in life and have near-normal auditory processing, or whether cross-modal re-organization is present due to the importance of visual cues needed for multimodal speech processing regardless of age of implantation. The documentation of cross-modal plasticity from the somatosensory modality is new in pediatric deafness and its relationship to speech perception and production needs to be further explored. While there is a high incidence of deafness in childhood (Boulet et al., 2009), age-related hearing loss is also the third most common chronic health condition facing older adults in the United States (Collins, 1997). Our studies are the first to document evidence of cross-modal plasticity in humans with early-stage mild-moderate hearing loss. Based on these studies we have proposed a framework, in which decreased input to auditory cortex (as in deafness or hearing loss) taxes the brain, resulting in compensatory recruitment of frontal cortices for top-down modulation of sensory processing and cross-modal recruitment of auditory cortex by vision likely associated with a greater reliance on visual cues to help disambiguate the speech signal. Future research should focus on examining changes in cortical resource allocation seen in age-related hearing loss as it relates to increased cognitive load.

Finally, we have described how the development of brain-based markers of changes in cortical resource allocation may be applied to individual patients with hearing loss. We have presented evidence of visual and somatosensory cross-modal plasticity in congenitally deaf children receiving CIs, progressive childhood hearing loss and SSD, and adults with early-stage mild-moderate hearing loss, and patients with hearing aids. Future research should focus on understanding the mechanisms underlying cross-modal plasticity in hearing impaired populations, which may be fundamentally different in developmental versus adult-onset hearing loss. Ultimately, audiological intervention relies on the principles of neuroplasticity, and the ability for the brain to adapt to new or restored auditory input. Further, inherently involved in aural rehabilitation are principles of neuroplasticity, the ability for the brain to learn, for a patient to receive training via bottom-up and top-down approaches so that they can optimize their performance after new or restored auditory input following audiological intervention. A better understanding of cortical cross-modal and intra-modal neuroplasticity in the context of hearing loss may allow us to harness neuroplasticity to optimize outcomes in patients with hearing loss.

Highlights.

1. We describe evidence of cross-modal plasticity from vision and somatosensory modalities in children and adults with varying degrees hearing loss.

2. We describe evidence of compensatory activation of frontal cortices in hearing loss.

3. We describe clinical applications of cross-modal activation in patients with cochlear implants, age-related hearing loss, single-sided deafness, and hearing aids.

Abbreviations

CI

Cochlear implant, cochlear implantation

CAEPs

Cortical auditory evoked potentials

CVEPs

Cortical visual evoked potentials

CSSEPs

Cortical somatosensory evoked potentials

SSD

Single-sided deafness