Acquired auditory-visual synesthesia: A window to early cross-modal sensory interactions

Abstract

Synesthesia is experienced when sensory stimulation of one sensory modality elicits an involuntary sensation in another sensory modality. Auditory-visual synesthesia occurs when auditory stimuli elicit visual sensations. It has developmental, induced and acquired varieties. The acquired variety has been reported in association with deafferentation of the visual system as well as temporal lobe pathology with intact visual pathways. The induced variety has been reported in experimental and post-surgical blindfolding, as well as intake of hallucinogenic or psychedelics. Although in humans there is no known anatomical pathway connecting auditory areas to primary and/or early visual association areas, there is imaging and neurophysiologic evidence to the presence of early cross modal interactions between the auditory and visual sensory pathways. Synesthesia may be a window of opportunity to study these cross modal interactions. Here we review the existing literature in the acquired and induced auditory-visual synesthesias and discuss the possible neural mechanisms.

The term synesthesia is composed of two parts, syn = together and aesthesia = sensation. It denotes that sensory stimulation of one sensory modality (the inducer) will elicit an involuntary or automatic sensation in another sensory modality (the concurrent).¹ For the purpose of clear communication in this paper synesthesia will be addressed by two consecutive words of inducer-concurrent. For example, if auditory stimuli evoke visual sensations, it is called auditory-visual synesthesia. Although there is no standard terminology, this has been the most common trend in the past decade.

Synesthesia has been scientifically reported as early as 1883 by Galton² in an assay in which he referred to published earlier cases including the ones by Bleuler and Lehman in 1881. As a result there is centenarian literature on developmental as well as drug-induced synesthesias.

The developmental synesthesias are not a neurologic disorder but a different way of experiencing one’s environment. They are life-long (ie, the subjects remember them back to their childhood), idiosyncratic, automatic, involuntary and consistent experiences across ones, lifespan.³ The acquired variety is different in that it is emergent and usually happens subsequent to neuropathologic insult to the brain. There is also induced synesthesia that can happen with sensory deprivation as well as intake of hallucinogenic or psychedelics.^4–7

In 1977, Bender brought this topic to neurologic attention by describing a case of acquired auditory-visual synesthesia.⁸ Since, there have been several case reports and case series in the acquired variety, but the topic stays under-reported and under-appreciated in the field of neurology. In this paper we review the existing literature in the most common acquired variety, auditory-visual synesthesia.^8–15 We also mention briefly less common reported acquired varieties as well as the induced synesthesias. Developmental synesthesias are not discussed here and the interested reader is encouraged to learn about this by referring to the already existing and extensive literature on the topic by cognitive neuropsychologist colleagues.

Acquired auditory-visual synesthesia

The majority of the acquired synesthesias have been reported in the setting of sensory deafferentation, when sensory stimuli to another sensory system is experienced ectopically in the deafferented system. There are also few case reports of acquired synesthesia with intact sensory pathways. Here the acquired auditory-visual variety is divided into two groups, depending on the presence or absence of visual deafferentation.

Auditory-visual synesthesia associated with visual deafferentation

In all the reported cases with sensory deafferentation, the neuropathologic abnormality involves the anterior optic pathways (optic nerve, optic chiasm or both). Table 1 summarizes the clinical details of twenty existing cases of acquired auditory visual synesthesia with visual deafferentation across different case series and reports.^8–13 In brief the underlying neuropathology involves the optic nerve and/or chiasm and consists of demyelination, ischemia, or tumors as well as post-traumatic total ocular blindness. The reported visual sensation is not well-formed and rather simple, like light flashes, kaleidoscope, color, etc. The very first case in Table 1 (patient #1) is a case of induced synesthesia due to surgical blindfolding of the external eye after surgery for keratoconus. This case is discussed separately under induced synesthesias. The other 19 cases (#2–20) are discussed below depending on degree of deafferentation, onset of synesthesia, laterality of the concurrent in relation to the inducer, duration, vigilance during synesthesia experience and neurophysiologic testing.

Table 1.

Clinical patient summary of reported cases of acquired auditory-visual synesthesia in the literature

Author et al	No	Age/Sex	Path/Side	Onset of synesthesia	Sen deaf	VA	Inducer	Lat	State	Duration
Lessel	1	37/M	Ker-Con/NR	Im	BF	N/A	S, No	*	D, St	1 wk
Lessel	2	37/F	ON/L	1 wk	P	L: 20/200	S, No	L	D, St	NR
Lessel	3	46/F	NL/R	6 mo	P	R: 6/200	S	R	D, St	NR
Jacobs	4	25/F	ON/L	1–3 d	P	L: 20/70 R: 20/20	S	L	D, Rel	Persist
Jacobs	5	38/F	ON/R	1–3 d	P	R: 20/400–20/40	S	R	D, Rel	Persist
Jacobs	6	26/M	ON/R	1–3 d	NC	R: LP	S	R	D, Rel	Persist
Jacobs	7	44/F	Ch Tu	1–3 d	P	R: 20/50 L: 20/20	S	R	Lt, EO	Persist
Jacobs	8	52/F	Ch Tu	1–3 d	P	B: 20/30	S, No	Ipsi and B	D, Rel	Persist
Jacobs	9	61/F	TA/R	1–3 d	NC	R: 20/800 L: 20/40	S, N, Vo	R	D, Rel	Persist
Jacobs	10	49/M	TA/R	1–3 d	NC	R: LP-20/50	S, N, Vo	R	D, Rel	5 mo
Jacobs	11	71/M	PSA/R	3 mo	C	R: NLP	S	R	Lt, EO	Persist
Jacobs	12	70/F	Vas/R	1–3 d	P	R: 20/50 L: 20/30	S, N, Vo	R	D, Rel	Persist
Bender	13	54/M	ON/NR	NR	P	NR	NR	NR	NR	NR
Kim	14	54/M	MC/L	9 mo	P	L: 20/50	No	NR	Dk	49 mo**
Page	15	55/M	ION/R	After***	P	R: 6/60, N 24	S	NR	Dk and Lt	8 mo
Page	16	38/M	ION/R	2 wk	P	R: 6/6, N5	S, N, Vo	NR	Dk	2 mo
Page	17	68/M	ION/L	Before	P	L: 6/9, N10	No, S	NR	Dk	4 mo
Page	18	66/M	art ION/L	1 mo	P	LUH: MP LLH: LP	No	NR	Dk	4 y+
Page	19	30/M	ON/L	2 wk	P	L: 6/12 + 2, N6	S	NR	Dk	10 mo+
Rao	20	52/M	PT TOB/B	1 yr	C	N/A	No, S	Ipsi and B	Awake	Persist

Abbreviations: No, number of patients as referenced in the text; F, female; M, male; Path, pathology; NR, not reported; Ker-Con, keraotconus; ON, optic neuritis; NL, neurilemma; Ch Tu, chiasmal tumor; TA, temporal arteritis; PSA, post-surgical amauresis; Vas, vascular occlusive disease; MC, melanocytoma; ION, ischemic optic neuropathy; art, arteritic; PT TOB, post-traumatic total ocular blindness; Im, Immediate; wk, week; d, day; mo, month; yr, year; Sens deaf, degree of sensory deafferentation by patient report; BF, blindfolded; P, partial; C, complete; NC, near complete; VA, visual acuity of the abnormal eye; NR, not reported; LP, light perception; NLP, no light perception; MP, movement perception; LUH, left upper half; LLH, left lower half; R, right; L, left; Ind, inducer; S, sound; No, Noise; Vo, voice; Lat, laterality of inducer-concurrent (see text); Ipsi, ipsilateral; B, bilateral; D, drowsy; St, still; Rel, relaxed; Dk, dark; Lt, light; +, persistent after the stated time.

Notes:

^*Flash of light localized approximately to the site at which the sound originated;

^**Total of 49 months including 16 months post-enucleation of the left eye;

^***not reported in days, weeks or months.

Degree of deafferentation

Synesthesia was associated with varying degrees of deafferentation. In two cases complete blindness ie, no light perception was reported by the patient. One (#20) had absent visual evoked potentials (VEPs) and in the other (#11) VEP showed incomplete deafferentation. The rest of the cases (#2–10 and #12–19) had mild to severe partial deafferentaion as determined by visual acuity (see Table 1).

Onset of synesthesia

The reported onset of synesthesia in relation to the clinical onset of the visual symptoms was very variable, ranging from synestheisa preceding any visual symptoms to cases that synesthesia started months after. We divided the reported onset of synethesia in relation to clinical onset of visual symptoms into four groups of: preceding, acute (one to three days), sub-acute (one to four weeks) and chronic (in months). These are summarized in Table 2. With the exception of one case, synesthesia followed onset of visual symptoms and hence a variable period of deafferentation. There seems to be no relationship between the onset of synesthesia and the underlying neuropathology.

Table 2.

Summary of onset of synesthesia in relation to visual deafferentation

Onset of synesthesia	No	Pathology: number of cases
Before	17	ION: 1
Acute (1–3 d)	4, 5, 6, 7, 8, 9, 10, 12	ON: 3, ChTu: 2, TA: 1
Sub-acute (1–4 wk)	2,16,18,19	ION: 1, ON: 2, arteritic ION: 1
Chronic (months)	3, 11, 14, 20	NL: 1, MC: 1, PT TOB: 1

Abbreviations: No, number of patients as referenced in the text; ION, ischemic optic neuritis; ON, optic neuritis; ChTu, chiasmal tumor; TA, temporal arteritis; NL, neurilemoma; MC, melanocytoma; PT TOB, post-traumatic total ocular blindness.

Laterality

Table 1 provides information about the laterality in each patient. Laterality refers to the side of the auditory stimuli (inducer) in relation to the side of the visual perception (concurrent). In 12 of the cases where laterality was reported, the induced visual perception was always ipsilateral to the side of the perceived auditory stimuli. In five of these 12 cases synesthesia was experienced in the area of scotoma itself (#4, #6, #7, #9, and #11). Regardless of laterality of the inducer in relation to the concurrent, the induced visual phenomenon was perceived in the deafferented eye with auditory stimuli in 16 of the 19 cases. In patients #13–14, laterality was not reported. Patient #15 was able to lateralize the auditory stimulus, but the induced phosphenes always happened in front of his eyes.

Vigilance

Most patients experienced their synesthesia when in dark and in a relaxed state. Four of the patients (#7, #11, #15, and #20) experienced their synesthesia in light. These included the two patients who reported no light perception (one with absent VEP and one with VEP showing incomplete deafferentation), and two patients with partial deafferentation.

Duration

In 16 cases the duration of synesthesia was reported. In five of the 16 cases synesthesia was a transient phenomenon of variable duration associated with partial deafferentation. In 11 cases synesthesia was persistent. Interestingly, synesthesia persisted 16 months after enucleation of the left eye in patient #14.

Neurophysiologic evidence

Table 3 summarizes the neurophysiologic studies done across all patients.

Table 3.

Neurophysiological studies in patients with auditory-visual synesthesia with visual deafferentation

Author et al	No	VEP	AEP	Occipital activation
Lessel	1	NR	NR	N/A
Lessel	2	NR	NR	N/A
Lessel	3	NR	NR	N/A
Jacobs	4	B Abn*	WNL	NR
Jacobs	5	U Abn	WNL	NR
Jacobs	6	NR	NR	N/A
Jacobs	7	NR	NR	N/A
Jacobs	8	U Abn	WNL	NR
Jacobs	9	U Abn	WNL	NR
Jacobs	10	U Abn	WNL	NR
Jacobs	11**	U Abn	WNL	NR
Jacobs	12	U Abn	WNL	NR
Bender	13	NR	NR	N/A
Kim	14	NR	NR	N/A
Page	15***	U Abn	NR	N/A
Page	16***	Done	WNL	NR
Page	17	Done	NR	N/A
Page	18	Done	NR	N/A
Page	19	Done	NR	N/A
Rao	20	B absent	WNL	Yes

Abbreviations: No, number of patients as referenced in the text; VEP, visual evoked potential; AEP, auditory evoked potential; NR, not reported; B, bilateral; U, unilateral; Abn, abnormality; Done, exact results not reported; WNL, normal; N/A, not applicable (because AEPs were not done).

Notes:

^*Due to presence of left optic neuritis and past history of right retrobulbar neuritis (four years prior);

^**VEP showed incomplete deafferentation despite amauresis reported by patient;

^***No electroretinogram (ERG) activation by sound.

1. Visual evoked potentials (VEPs). Thirteen of the patients had VEPs and these showed abnormalities of anterior optic pathways as expected.

2. Electroretinogram (ERG). Two of the patients (#15 and #16) had this test done and there was no ERG activation after sound stimuli.

3. Auditory evoked potentials (AEPs). In nine of the cases AEPs were performed showing intact auditory pathways. There was no mention of presence or absence of any occipital activation with the exception of #20, a case report by Rao and colleagues.¹³ In this case of post-traumatic total blindness, there was occipital activation with auditory evoked potentials.

Other (not included in Table 1)

In all patients with the exception of three (#13, #14, and #20), the presence of positive visual phenomenon were addressed. Four of these 16 patients (#4, #5, #7, and #10) had concomitant positive visual phenomenon that was not sound induced. One patient (#16) had phosphenes provoked by eye movements in his initial presentation, but not concomitant with synesthesia. The remaining 11 patients did not experience positive visual phenomenon.

Auditory-visual synesthesia associated with intact visual pathways but neuropathologic affection of CNS

The reported cases are confined to synesthesia experienced with pathologic involvement of temporal lobe.^14,15

Vike and colleagues¹⁴ reported a case of synesthesia with intact visual pathways. This patient had a gliotic mass involving his left mesial temporal area and the adjacent midbrain. Synesthesia in this case was reported to be bilateral but more on the left (ipsilateral side). Patient had normal AEP and VEPs and no comment was made about occipital activation. Synesthesia was aborted after the removal of tumor.

Synesthesia-like symptoms was reported by Jacome and colleagues¹⁵ in a case of left temporal lobe seizures. During his aura the patient experienced what the authors called “internal synesthesia”. After sudden onset of the pain on the right side of his face, he simultaneously heard the word “five” in his both ears and saw the number “5” or “five” on a grey background before his eyes.

Other acquired synesthesias

Acquired auditory-tactile synesthesia

Ro and colleagues¹⁶ reported a case of auditory-tactile synesthesia in a 36-year-old right-handed female with a lacunar infarct involving the ventrolateral (VL) nucleus of thalamus. One year after the stroke, patient exhibited multisensory neglect for vision and touch and tactile and visual antiextinction. Three years after her stroke, the patient was reported to experience auditory-tactile synesthesia contralateral to the thalamic lesion. Magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) done in 1.3, three, and six years showed the stroke in right thalamus. White matter asymmetries between right and left hemisphere were found in MRI/DTI done three and six years after the stroke. Authors suggested that abnormal connectivity from right VL to the cortex in the form of disorganized and fewer fiber bundles originating from right thalamus may be the neural basis synesthesia.

Acquired tactile-visual synesthesia

Armel and colleagues reported a case of acquired tactile-visual synesthesia¹⁷ in a patient with retinitis pigmentosa and progressive visual decline, until complete blindness at age 40. Synesthesia in from of tactile stimuli evoking vivid visual sensations started at age 42. Interestingly the touch threshold was reported to be significantly less when the hand was held behind the patient as opposed to front of him. The phenomenon occurred ipsilateral to the stimulation.

Induced auditory-visual synesthesia

Induced auditory-visual synesthesia with deafferentation (by transient blindfolding)

The first of the three patients reported by Lessell and colleagues⁹ had post-surgical blindfolding due to keratoconus surgery. There was no neuropathologic involvement of the anterior optic pathways. This patient had immediate onset of his synesthesia that lasted for one week despite the continued blindfolding. No laterality or neurophysiologic testing was reported.

In a visual deprivation experiment with blindfolding by Merabet and colleagues,¹⁸ two of the 13 subjects experienced their visual hallucinations acutely and in response to auditory stimuli (one to TV and the other to Mozart’s Requiem).

Induced auditory-visual synesthesia without deafferentation (use of hallucinogens and psychedelics)

Synesthesia has been reported with LSD⁵ as accompaniment to other visual hallucinations. These sound-induced visual perceptions and the visual hallucinations can be well formed and complex. The cause of synesthesia is thought to be chemical deafferentation of lateral geniculate body, which will deprive the occipital cortex from visual stimuli.⁵ LSD is known to selectively activate serotonin 2A receptor. Brang and Ramachandran¹⁹ hypothesized that serotonin 2A receptors are the “synesthesia receptor”. Auditory visual synesthesia also has been reported with other serotonin 2A agonists including mescaline,⁴ and ayahuasca.²⁰ Ayahuasca contains serotonergic psychledic N,N-dimethyltryptamine (DMT) and beta-carboline, a monoamine oxidase inhibitor.²¹ DMT is a partial agonist of serotonin 2A receptors and beta-carboline inhibits the visceral metabolism of DMT, increasing its systemic levels.

Salvia divinorum is a psychotropic mint and unscheduled dietary supplement that also produces visual hallucinations and synesthesia.⁶ Its active ingredient, salvinorin A, is a selective kappa-opioid receptor agonist, and hence points to a different mechanism of action compare to serotonin 2A receptor agonists.

Synesthesia has also been reported with hashish.⁷ The active component, delta-9-tetrahydrocannabinol exerts its central effects through the CB1 cannabinoid receptor.²²

Other induced synesthesias

Ramachandrian and Rogers-Ramachandrian²³ induced visual-tactile synesthesia in patients with phantom limbs (ie, in an already deafferented system). They resurrected the phantom limb visually by using a virtual reality box that contained a mirror. The reflection of patient’s intact hand was superimposed on the place of the phantom limb. When the normal hand was moved, the phantom limb was perceived to move by looking at the mirror. Six of the ten patients had kinesthetic sensation of the involved limb. Interestingly, in four of the five patients this technique relieved the painful spasms of the phantom limb.

Shannon and colleagues²⁰ reported that with ayahuasca, synesthesias are mainly from a nonvisual modality to a visual modality. Although the most common experience is auditory-visual but other synesthesias including olfactory-visual and tactile-visual synesthesias occur. Tactile-visual synesthesias have been reported in mescaline-treated subjects.²⁴

Discussion

The pathophysiologic basis of auditory-visual synesthesia is not understood. It is known that the human neonate is an auditory-visual synesthete.²⁵ During the first month of life evoked responses to spoken language were recorded not only over the temporal cortex but also over the occipital areas. The extension of auditory evoked responses over the occipital areas continues until six months of age. After six months of age the auditory evoked responses over the visual areas gradually decrease until they disappear at 36 months.²⁶ There is evidence of persistence of these auditory-visual connections in the early blinds, when auditory evoked potentials are recorded over the occipital areas or when there is functional evidence of auditory^27,28 or somatosensory processing (Braille reading) by functional MRI (fMRI)²⁹ in the occipital cortex.

In case of acquired auditory-visual synesthesia there is an emergent auditory evoked visual awareness, in which the quality of the reported visual perception is rather simple like light flashes, kaleidoscope, color, etc. There is no known anatomical substrate for low level auditory-visual sensory interactions in humans. As a matter of fact in the classical functional neuroanatomy, the primary visual, auditory and sensorimotor koniocortices and the early association areas are considered unimodal.³⁰ In such model the cross modal interactions happens only in higher heteromodal association cortices. Many recent studies have challenged this classical view. For example in the macaque monkeys, it is well known that lesioning superior temporal polysensory (STP) area will cause auditory and somatosensory neglect. Using tracer injections, both Falchier and colleagues³¹ and Rockland and Ojima³² demonstrated connections between auditory and primary visual areas (V1). The former study was done in macaque monkeys and demonstrated projections from parabelt areas of auditory cortex and STP area to the region of primary visual cortex (area 17) that sub-serves the 10–20 degrees of the peripheral visual field. In the latter study there were direct projections from auditory association areas of monkeys to areas V1 and V2. It is not known if the same connections exist in humans. Martuzzi and colleagues³³ evaluated the potential for existence of such an anatomical substrate in humans by an fMRI study. They demonstrated the presence of bilateral BOLD responses in the primary visual cortices subsequent to auditory stimuli.

Neurophysiologic studies in normal human adult by Shams and colleagues³⁴ introduced the sound-induced illusory flash effect. That is when a single visual flash is accompanied by multiple auditory beeps, the single flash is perceived as multiple flashes. The brain evoked potentials recorded from occipital electrodes (O1, O2, and Oz) were similar for a physical flash and illusory flash. Bhattacharya and colleagues³⁵ compared illusory and nonillusory trials in this experiment and found early gamma band responses for the illusory trials.

In humans, acute visual deprivation (by blindfolding) causes cortical excitability. This is evidenced by decreased phosphene threshold to TMS of occipital regions and enhanced activation of visual cortex to incoming input, as measured by fMRI.³⁶ In synesthesia secondary to blindfolding (surgical⁹ or experimental¹⁸) when there is complete and sudden deafferentation but no neuronal injury, the onset of synesthesia is rather acute. The acuteness of the onset points to the presence of some existing auditory-visual pathways that could be unmasked due to hyperexcitability of the cortex deprived of visual stimuli.

When synesthesia occurs as a result of neuropathologic insult to the anterior optic pathways,^8–13 the region of the primary visual cortex corresponding to the affected axons will be deafferented. The imbalance between the excitation and inhibition will result in increased excitability in and surrounding the deafferented area.³⁷ There will be reorganization and change in weight of synaptic neurons before recovery. It has been shown that recovery from optic neuritis to normal or near normal vision is associated with a change in the distribution of cerebral response to visual stimulation in fMRI.³⁸ There is extensive extra-striate activation that may be evidence of long-term changes in networks involved in neuronal processing of visual stimuli.

It is puzzling why some patients experience synesthesia with mild degrees of deafferentation and some with severe degrees of deafferentation, and some never experience it at all. Also it is strange that some patients develop it preceding or acutely after visual symptoms (in the subclinical or clinical denervation supersensitivity period) while others develop it subacute to chronically during or after recovery (ie, during or after any cortical re-organization or plasticity). This may be due to inter-individual differences between patients and their central sensory synapses and the answer is subject to future research.

The reported acquired auditory-visual synesthesias not associated with deafferentation of the visual systems were all associated with temporal lobe pathology. Temporal lobe harbors the primary auditory cortex and auditory association areas in the Heschl’s and superior temporal gyri as well as higher order association areas in the lateral temporal lobe. These areas could be easily recruited by seizures of temporal lobe origin. Alternatively any temporal lobe space occupying lesion can exert mass effect on them. In the case of the mass lesion, the synesthesia disappeared after the removal of the lesion.

The mechanism of drug induced synesthesia in case of LSD has been attributed to chemical deafferentation of lateral geniculate body,⁵ which will deprive the occipital cortex from visual stimuli. Mechanism involving serotonin 2A receptor activation has been hypothesized by Brang and Ramachandran. Other receptors including kappa opioid receptor and cannabinoid receptors may also be involved in the mechanism of synesthesia. The quality of auditory evoked visual symptoms in drug induced cases can be more complex and well formed. This may point to involvement of higher order association areas in the genesis of the visual symptoms as opposed to lower level (primary or early association) audiovisual connections.

Although synesthesia is a symptom of neurologic disease, currently there is no specific ICD diagnosis for it. The purpose of this paper is to bring this symptom to neurologic attention and suggest that auditory-visual synesthesia as well as other forms of synestheisa may be much more common than encountered in the neurologic literature and they provide a window into understanding the basis of human cognitive sensory function. Their presence as a neurologic symptom warrant neurophysiologic testing as well as functional imaging (EP, EEG, MEG, and fMRI) to investigate the spatio-temporal sequence of human cognitive sensory functions that are otherwise not accessible for testing in normal conditions.