The Cognitive Neuropsychology of Auditory Hallucinations: A Parallel Auditory Pathways Framework

Abstract

Auditory hallucinations are generally defined as false perceptions. Recent developments in auditory neuroscience have rapidly increased our understanding of normal auditory perception revealing (partially) separate pathways for the identification (“what”) and localization (“where”) of auditory objects. The current review offers a reexamination of the nature of auditory hallucinations in schizophrenia using this object-based framework. First, the structural and functional organization of auditory what and where pathways is briefly described. Then, using recent functional neuroimaging data from healthy subjects and patients with schizophrenia, key phenomenological features of hallucinations are linked to abnormal processing both within and between these pathways. Finally, current cognitive explanations of hallucinations, based on intrusive cognitions and impaired source memory, are briefly outlined and set within this framework to provide an integrated cognitive neuropsychological model of auditory hallucinations.

Introduction

Auditory hallucinations (AHs) in schizophrenia typically involve voices. Obviously, voices carry speech, and in schizophrenia, these may range from single words or short phrases to linguistically complex perceptions of multiple voices conversing or commenting on the voice hearer. Not surprisingly many cognitive explanations of AH have assumed abnormal mechanisms of speech. However, the perception of voice normally entails more than just words; it involves the perception of information about speaker identity and vocal affect. Somewhat intriguingly, patients with schizophrenia tend to perceive hallucinated voices more often as male, while the tone of voice is most frequently described as being negative or critical. The human voice may, then, be thought of as an “auditory face”¹ involving high-level (complex) perception processes in much the same way as faces are processed in the visual system.

Importantly, not all AHs involve voices. Phenomenological surveys indicate that almost 60% of patients also report hallucinations involving environmental noises² (such as a door banging, a barking dog) and occasionally music,³ which cannot readily be explained within purely language or speech-based cognitive models. Given this phenomenological diversity, it seems likely that the pathophysiology underlying such phenomena is heterogeneous, involving multiple cognitive and neural mechanisms that are needed to explain both verbal and nonverbal AHs. Alternatively, it is possible that a common abnormality underpins these auditory events because, descriptively, both types of experience involve the perception of complex auditory objects.^4,5 Consequently impairment in high-level (complex) auditory object processing would seem to be a reasonable candidate mechanism. A similar formulation, ie, abnormal object perception, has recently been put forward to account for recurrent complex visual hallucinations,⁶ raising the possibility that impaired object perception provides a more general framework underpinning all forms of complex hallucinations.⁷

Emerging evidence points to the existence of, at least 2, partially distinct, processing streams in human auditory cortex specialized for the identification (what) and localization (where) of auditory objects. These developments provide a clear framework for understanding and integrating current neural and cognitive models of AHs in schizophrenia and in the normal population.^8,9 This review briefly summarizes evidence supporting the existence of parallel auditory pathways and draws links between this functional organization, the phenomenology of hallucinations, and current cognitive accounts of AHs.

Evidence for Parallel Auditory Pathways

The idea that the human visual system comprises parallel, hierarchically organized cortical pathways is now well established. Initially, 2 major pathways were identified for the processing of visual objects, via a ventral pathway, and spatial location, via a dorsal route.¹⁰ This so-called “what-where” model was later refined to form a third pathway¹¹ linked to the visual control of action. More recently, neuroimaging and neuropsychological studies have revealed similar parallel and hierarchically organized pathways in the human auditory system, specialized for processing the identity of sounds (ie, auditory objects) and their spatial attributes, ie, auditory what and where pathways.^12–16 In general, auditory tasks involving spatial computations or representations (eg, responding to moving sounds) engage a dorsal network including posterior superior temporal, inferior parietal, and superior frontal cortical areas (see figure 1). In contrast, a ventral, anterior superior temporal—to—inferior frontal network is preferentially involved with the processing of nonspatial auditory information (eg, identifying particular voices, segregating environmental sounds from background noise). While the visual where pathway provides a rapid and coarse analysis of visual space that assists the slower, more detailed processing in the ventral what pathway, it is currently unclear whether this type of functional interaction also occurs in the auditory system. However, additional differences in sensitivity to the temporal and spectral variation of auditory input in the left and right hemispheres have also been postulated that may underlie hemispheric specialization for speech and pitch perception, respectively.^17,18

Fig. 1.

Schematic Illustration of Parallel Pathways in the Human Auditory Cortex. A. Approximate projections and selected functions of separable “what” and “where” auditory pathways (dashed lines), within which both top-down and bottom-up activation may occur—signified by dual headed arrows. The dorsal auditory where pathway, involved in processing spatial attributes of sounds, originates in posterior, but not anterior, areas of the superior temporal cortex and projects to inferior parietal and superior frontal cortical areas. Activation in planum temporale and inferior parietal cortex contributes to external sound localization. A dorsal stream auditory-motor integration function has also been proposed, implicating area Spt (within planum temporale) and frontal articulatory/action networks. Hearing action-related speech activates relevant motor representations in premotor cortex. In contrast, processing of sound identity in the ventral auditory what pathway is more distributed, within anterior (voice recognition) and posterior (environmental sound recognition) temporal cortex, and projects to inferior frontal cortex.^12,22,28 B. Phenomenological features of auditory hallucinations linked to auditory pathway functions. Abnormal activity in the auditory what pathway may lead to the perception of complex objects (voices, environmental sounds) in the absence of external stimulation (ie, auditory hallucinations). Additionally, less detailed processing of vocal information in the ventral pathway, probably involving basic auditory sensory deficits, results in more male-sounding voices and may lead to the predominance of male “voices” (see text for details). Abnormal activation within the posterior region of the auditory where pathway, including planum temporal and inferior parietal lobe, may produce hallucinations perceived in external space (ie, “outside the head”), while “voices” commenting on patients’ actions/telling patients what to do may engage neural structures in the anterior dorsal stream coding the heard actions.

Some aspects of the auditory what-where framework clearly remain controversial. For example, while auditory spatial processing appears to be restricted to posterior temporal cortex, processing of sound identity is more distributed across anterior and posterior areas of the temporal lobe, suggesting that functional segregation is partial rather than absolute.^12,15 Such observations have led to alternative formulations of the dual pathway model, including the need for further subdivision of auditory processing streams and, in particular, a reconsideration of the functional role of the auditory where pathway.^19,20 In light of these recent formulations, both dorsal and ventral auditory pathways shown in figure 1 are depicted as comprising multiple streams of processing. As in the visual domain, there is increasing recognition that information about actions is intimately related to auditory representations. Consequently, Warren and colleagues, eg, proposed that the dorsal stream serves as an auditory “do” pathway, transforming auditory input (either speech or space) into motor output.¹⁹ While acknowledging these challenges, recent studies support the automatic, intrinsic activation²¹ and attentional modulation of separable auditory pathways,¹⁶ providing strong support for this general framework.

Complex Auditory Objects

Speech perception is a particularly important component of the what pathway entailing the processing of linguistic auditory objects (ie, words) and articulatory gestures together with acoustic information about who is speaking (eg, speaker gender, age, accent) and their affective state.

It has been proposed that each of these separate types of information from voices may also be processed in partially dissociated functional pathways,¹ though the specific neural mechanisms are still under investigation. Such a system can account for clinical case studies exhibiting double dissociations between speech perception and speaker identity and predicts a range of other functional dissociations (eg, intact processing of speaker identity vs abnormal processing of vocal affect).

Hickok and Poeppel²² recently utilized the dual-stream model of auditory processing to integrate a variety of (and often conflicting) empirical findings on speech processing, including research on speech development, neuropsychological data, and neuroimaging studies. In this model, the main function of the ventral what pathway is to process speech sounds for comprehension (mapping speech sounds to conceptual/semantic representations), while the dorsal stream is assumed to provide an auditory-motor integration function, which maps speech sounds to articulatory representations in the posterior frontal lobe. Hickok and Poeppel²² suggest that an area at the boundary between parietal and temporal lobes, area Spt, underpins this sensorimotor interface, which also includes projections to multiple target zones comprising a frontal articulatory network (probably including Broca's area and more dorsal premotor regions, as shown in figure 1) (see also Arnott et al¹²). In contrast to the prevailing view, this model assumes that the relevant structures in the ventral pathway (superior and middle portions of the temporal lobe) are largely bilaterally activated (though other computational differences between hemispheres are proposed) while the dorsal pathway is assumed to be strongly left dominant.

The specific acoustic parameters used to determine “who” is speaking remain to be specified; however, there is increasing evidence that perception of speaker identity (eg, speaker age and gender) and voice recognition engages anterior temporal lobe regions, especially—though not exclusively—in right superior temporal sulcus (STS).^1,23 For example, detecting a general change in speaker voice has been shown to activate posterior temporal regions, while increasingly detailed analysis of individual voices occurs in anterior STS.²⁴

Though traditionally viewed as a product of the right hemisphere, recent functional neuroimaging studies indicate that processing of affective information (emotional prosody) in speech involves the integration of both verbal and vocal information and is mediated by bilateral temporal—inferior frontal networks plus subcortical structures (including the amygdala).^25,26

The processing of environmental sounds is also closely connected to, but at least partially separable from, the processing of speech, as demonstrated by occasional reports of auditory agnosia for environmental sounds with intact speech comprehension.²⁷ Exposure to new environmental sounds typically occurs in a rich multisensory context (hearing a tool, seeing it used, or making the tool movement). Using functional imaging, Lewis et al²⁸ have demonstrated that recognizable sounds from the environment (eg, tools, animals, liquids, and dropped objects) also activate selected parts of the auditory what pathway, including a strong bilateral response in posterior middle temporal gyrus. This area of activation, which is also illustrated in figure 1, partially overlaps cortical regions linked to the processing of biological motion. Further specialization to categories of object sounds (tools vs animals) has also been reported again entailing activation in the right STS.^29,30

Together these findings are consistent with a hierarchical—and increasingly combinatorial—organization within the auditory what pathway underpinning the recognition of particular (ie, individual) auditory objects. The emerging model of parallel auditory pathways for identifying and localizing complex auditory objects highlights a novel, plausible neural mechanism involved in the generation of AH. The dissociable nature of the model¹ predicts that each pathway may have somewhat separable influences on the presence (or absence) of hallucinations.³² Moreover, due to the rapid increase in understanding what and where auditory pathways, it is now possible to integrate the differential response properties of these pathways with the phenomenological diversity of AHs. Indeed, many of the characteristic features of hallucinations appear to reflect abnormal cortical activity, at different hierarchical levels, both within and between parallel auditory pathways.

From Speech and Sound to AHs

Verbal and Nonverbal Hallucinations

Hallucinated voices and environmental sounds may now be understood as an outcome of dysfunctional processing within the auditory what pathway (see figure 1B), which is normally responsible for the perception of a variety of complex auditory objects. Inconsistent evidence of structural abnormalities in the superior temporal gyrus (STG) in early studies of AHs in schizophrenia led Stephane et al³¹ to suggest that for hallucinatory experiences to emerge the relevant neural structures may need to be relatively intact. However, more recent neuroimaging data consistently indicate volume reductions in patients with AH, especially in left STG and in nonsensory areas including prefrontal (and cerebellar) cortices,³³ providing a potential substrate for abnormal neural activity at different hierarchical levels along the ventral what pathway. In addition, there is interesting new evidence suggesting that AHs may be rooted in the tendency of auditory cortex to spontaneously activate (ie, in the absence of external stimulation), even in healthy individuals.³⁴ Individual differences in this endogenous neural activity could provide a functional neural basis for verbal and nonverbal hallucinations in both clinical and nonclinical individuals. Furthermore, because AHs and auditory perceptions must compete for the same neural resources, hallucinated voices/environmental sounds would be expected to interfere with the normal processing of external speech and environmental sounds, respectively. Functional neuroimaging data support this proposal, showing a lower response to external speech in left STG as the severity of hallucinations increases.³⁵ Similar data concerning hallucinations of environmental sounds have not been reported, though patients with schizophrenia do exhibit deficits in the perception and recognition of environmental sounds.³⁶

Predominance of Male Voices

The perception of gender in speech also depends on the integrity of the auditory what pathway. Sokhi et al,³⁷ eg, recently demonstrated that both male and female voices activate distinct anterior temporal lobe regions (in healthy male listeners). In particular, the perception of female voices results in greater activation of voice-selective regions in the right anterior STG, near the STS. Consequently, it is likely that abnormal functioning in the anterior auditory what pathway underpins the predominance of male voices in AHs². Findings of Sokhi et al³⁷could not be explained by simple differences in pitch perception and may relate to the higher processing demand engaged by the increased use of affective information (eg, emotional prosody) in female speech. This is consistent with the view that female voices are relatively more “complex” auditory objects than male voices.

Importantly, significant and consistent deficits in the perception of emotional prosody have been reported in patients with schizophrenia.³⁸ Furthermore, impaired processing of auditory affect information—including both verbal and nonverbal stimuli—is specifically present in patients with auditory-verbal hallucinations.³⁹ The predominance of male “voices” in AHs² may therefore be due, in part, to impaired perception of auditory affect information (which presumably results in less detailed auditory representations of individual speakers that are functionally more similar to a male-sounding voice). This feature of AHs—shown in figure 1B as an outcome of impaired ventral auditory pathway processing—cannot readily be explained by existing cognitive models of hallucinations. There is an extensive body of evidence indicating that impaired prosodic processing in schizophrenia may, in turn, result from disturbances in basic auditory sensory processing,^40,41 including poor frequency discrimination. The neural basis of these sensory impairments, as revealed, eg, by diffusion tensor imaging, includes the primary auditory cortex as well as both ventral and dorsal auditory streams⁴² highlighting the likely importance of lower as well as higher levels of the auditory system in the analysis of hallucinated voices. Further research is, therefore, required to examine differences in processing of specific acoustic parameters of voice gender and vocal affect in patients with and without hallucinations.

Localizing Hallucinations in External Space

Patients with schizophrenia often perceive hallucinated voices/sounds as being located in external auditory space, while others are experienced “inside the head.”⁴³ Phenomenological data suggest that inner-outer space location is a key dimension underlying hallucinatory experiences that may be related to a common neural mechanism.⁴⁴ Within the current framework, hallucinations of complex auditory objects in external space (eg, “there's a train whistling in the garden”) are likely to involve abnormal activation of those auditory regions responsible for spatial representations (ie, the where pathway). Consistent with this notion, recent functional neuroimaging in healthy subjects shows that the perception of externality is subserved by a temporoparietal network including planum temporale (PT), in the posterior superior temporal plane, and the inferior parietal lobe (as shown in figure 1A). In particular, virtual acoustic techniques producing speech with a clear “outside the head” location produced greater activation in left PT than speech perceived as “inside the head.”^45,46 Importantly, in patients with schizophrenia, there is evidence of impairments in localizing sounds in free field situations⁴⁷ and evidence of abnormal activation of relevant structures in the where pathway (eg, PT) linked to the presence of hallucinations,⁴⁸ (see figure 1B) though future studies are needed to correlate this activity with the specific phenomenological report of perceived external location.

Elsewhere, activation of PT has been argued to reflect a more general capacity of the dorsal auditory pathway to disambiguate information concerning sound location from that relating to sound identity, normally arising in complex sounds.¹⁹ Such disambiguation may be achieved by matching incoming auditory information with stored templates, based on prior experience—a process indexed in electrophysiological studies by the mismatch negativity (MMN) response. Matthews et al⁴⁹ have recently shown that patients with schizophrenia exhibit a reduced MMN response when encoding interaural timing and phase (ie, temporal), but not loudness, cues to sound location which might suggest that a selective dysfunction in template matching of temporal cues contributes to the development of hallucinations—perhaps via other auditory processes dependent on temporal resolution, such as speech.^18,22 However, further investigation of this proposal is required because the stimuli in this study were presented via headphones and spatial positions were experienced as inside the head.

Action Representations of Sound

Voices discussing a patient's actions, or telling the patient what to do, continue to have a particular significance in diagnosing schizophrenia.⁵⁰ Such AHs belong to the category of first-rank symptoms of schizophrenia, in which there is the loss of a clear boundary between “self” and others,⁵¹ and generally comprise linguistically complex speech with multiple voices arising from an external source. Stephane et al⁴⁴ have suggested that hallucinations of this type comprise a distinct cluster—possibly reflecting different neural origins than other types of hallucinations. Recent functional imaging shows that the normal perception of action-related speech (such as voices issuing instructions) produces activation of premotor and prefrontal extensions of the dorsal auditory stream⁵²; this suggests that abnormal activation within this pathway provides the relevant neural substrate for complex AHs. Accordingly—as depicted in figure 1B—it is proposed that listening to real or hallucinated action-related sentences (eg, “he cut the meat”) entails activation of motor representations relevant to both speech (ie, mouth/vocal tract) and action (eg, hand).

This approach to understanding AHs builds on recent proposals that assign an auditory-motor integration function to the dorsal auditory stream: linking sounds with actions, such as speech sounds with articulatory gestures.^19,22 Both bottom-up and top-down transfer of information occurring between auditory and motor regions normally provides a bidirectional flow of information within this pathway (signified by bidirectional arrows in figure 1). For example, intentional, self-generated (top-down) speech acts usually decrease auditory cortical responsiveness. However, there is now a growing body of evidence that this form of self-attenuation is deficient in patients with AHs^53,54 consistent with an abnormality in the dorsal auditory network. Individuals with AHs may—as a consequence of this abnormal functioning—endorse an external agency as the source of the experience because action representations of sounds from self and others both engage the dorsal auditory network. As yet, this explanation for the sense of agency in AH has not yet been directly tested and remains a matter of debate. The assumption²² that speech production in the dorsal stream is typically strongly left hemisphere dominant also provides an obvious interface with existing models of schizophrenia based on abnormal lateralization of language.

Alternatively, the alien (nonself) character of AHs may be influenced by the perceptual correlates of auditory activations (see also Larøi and Woodward⁵⁵). For example, subvocalized speech (covert articulation), which frequently accompanies AHs, signifies activation of PT in the dorsal auditory pathway and is likely to bias the perception of a real voice in external space. Similarly, activation of vocal acoustic features (such as those signaling gender) in the auditory what pathway that do not match the patients own voice may result in an hallucination that is attributed correctly or incorrectly to a particular external agent (other). Future studies are required to test these proposals.

In summary, AHs in schizophrenia are a highly individualized experience whose characteristic features may be related, at least in part, to the abnormal activation of auditory what and where pathways. It is important to note that current pharmacological treatments have proven effective in reducing the symptoms of schizophrenia, including AHs, by blocking subcortical dopamine (D₂) transmission and/or enhancing cortical dopamine (D₁₎ and N-methyl-D-aspartate (NMDA) transmission.⁵⁶ The treatment efficacy of antipsychotic medications may encompass direct or indirect influences on auditory processing streams. For example, recent evidence shows that dopamine may play an important role in inhibiting spontaneous and evoked responses in auditory nerve fibers,⁵⁷ while NMDA has a specific role in auditory sensory memory,⁵⁸ as revealed in auditory MMN responses. However, future studies are required to examine the action (if any) of antipsychotic medications on more specific functional characteristics of parallel auditory pathways.

Integrating What, Who, and Where

Ultimately, information conveyed in the auditory what and where pathways must converge in order to produce an integrated representation of objects in context. In healthy individuals, binding content (what) and context (who, when, where) depends on the integrity of the medial temporal lobes (MTLs), in particular the hippocampus. Consequently, understanding extensions of this auditory hierarchy into the MTL “memory system” may also be essential to gaining a complete understanding of AHs in schizophrenia.⁵⁹ For example, superior temporal auditory association cortex has a distinct projection to the posterior regions of MTL, particularly parahippocampal cortex,⁶⁰ which can result in the selective impairment of voice context (ie, speaker identity) memory following damage to this region.⁶¹ Finally, it must be noted that the prefrontal cortex also plays an important role in voluntary context binding⁶²; consequently, future studies are required to disentangle the separate contributions of automatic and voluntary context binding to the experience of AHs.

Cognition: Control and Context

Regulating Object-Level Processes

Both structural and functional brain imaging accounts of verbal and nonverbal AHs outlined above underscore the link between hallucinations and abnormal (eg, involunatry or spontaneous) neural activation. Such formulations dovetail with current cognitive explanations of AHs that emphasize the need to control unintended and/or intrusive mental activity. From a cognitive perspective, “inhibition” is critical to the ability to control and regulate mental representations (thoughts) and behaviors. Not surprisingly, therefore, deficient inhibitory control has been assumed to play a pivotal role in the onset of hallucinations, though initial empirical support was inconsistent. Recent models suggest that inhibition comprises a family of processes⁶³ with separable neural bases, and new findings confirm that hallucinations are significantly associated with a deficit of intentional inhibition, while automatic inhibition remains intact.^64,65 Furthermore, a similar difficulty in inhibitory control is also present in healthy individuals predisposed to hallucinations.^66,67 Functionally, therefore, AHs appear to arise from a failure to control previously relevant mental representations, allowing old memories to intrude into current events and become confused with ongoing reality.⁶⁸ Subjectively, individuals who are actively hallucinating do not feel as if they have direct, voluntary control over the experience; in fact, patients with schizophrenia rate “sense of control” as one of the most salient features used to differentiate their own verbal thoughts from hallucinated voices.⁶⁹

What might be the consequences of impaired cognitive control? First, such difficulties may lead to repeated or increased efforts to inhibit spontaneous and/or unwanted cognitions (hallucinations). Recent evidence indicates that voluntary suppression can induce a sense of involuntariness,⁷⁰ which may, in part, explain why individuals with hallucinations do not feel like they are able to control the experience. Increased attempts to achieve control are also likely to result in both the heightened availability (ie, increased frequency) of hallucinations through well-described paradoxical effects of suppression and further depletion of limited inhibitory control resources⁷¹ in subsequent episodes, potentially contributing to the recurrence of hallucinations. It is also tempting to speculate that cognitive therapies based on “acceptance” and “mindfulness” methods⁷² are effective precisely because they focus on acceptance rather than control of hallucinatory experiences.

Context Memory and Binding

Abnormalities in context (source) memory and integration, ie, binding memories of objects or events into their correct context, figure prominently in current cognitive,^65,73 as well as biological,⁵⁹ explanations of schizophrenia and AHs. Such difficulties are assumed to result in an inability to correctly identify the origin of mental events. For example, in the case of verbal hallucinations, contextual features—including the source (self/other), ie, who produced the speech, the emotional tone of voice, and when (temporal) or where (spatial) it was spoken—may be missing or incomplete. One important line of studies suggests that hallucinations arise from a difficulty in reality monitoring such that hallucinators exhibit a bias toward attributing self-generated speech or thoughts to an external source,^74,75 though some methodological and interpretative limitations of such studies have been noted.⁵⁵ For example, an externalization bias alone does not address the rich diversity of hallucinations including those phenomenological features described in the current review. Others have proposed that hallucinations involve misremembered voices, ie, individuals with hallucinations may be correctly reporting the speech and voices of other people but fail to retrieve critical contextual details, such as time and place.⁷⁶ Thus, abnormalities in context binding may not only explain why hallucinations are perceived as originating from an external source but can also accommodate many other features of the experience. Supportive evidence including deficits in binding either spatial or temporal context, or both, have recently been documented in schizophrenia and linked to current hallucination experiences.^77–79 Furthermore similar impairments have also been reported in adolescents with psychotic symptoms,⁸⁰ suggesting that context memory and binding may play an important role in the emergence of psychotic symptoms.

Concluding Remarks

This review illustrates how an object-based framework, arising from current models of normal auditory perception, may be used as a neural foundation on which to understand the phenomenology of AHs in schizophrenia and in the general population. This formulation can accommodate both bottom-up and top-down influences in the genesis of hallucinations and provides a coherent platform from which to generate questions for future research. Furthermore, cognitive abnormalities assumed to underpin the development of AHs are readily integrated into this framework, providing a bridge between biology and phenomenology.