Bridging senses: novel insights from synaesthesia

1. Introduction

Synaesthesia is a neurological phenomenon that can offer an unusually clear window into the factors influencing human sensory perception and the range of normal experience [1,2]. Sometimes described as a blending of the senses, where exposure to a triggering stimulus (e.g. music and letters of the alphabet) causes an immediate additional sensation (e.g. colours and tastes), it has been reported that synaesthesia naturally affects 1–4% of people [2], and has a range of different types [3]. These developmental forms of synaesthesia are thought to involve an intertwining of innate predispositions [4] with environmental factors and so have potential to give insights into how genetics and learning interact to create our views of the world around us [1]. The status of synaesthesia as a topic of shared interest across multiple areas of science (psychology, linguistics, cognitive neuroscience, genetics, etc.) necessitates a high level of interdisciplinary interaction to generate scientific advances. Therefore, in October 2018, international experts from these diverse disciplines came together at the Royal Society in London for a Scientific Discussion Meeting on ‘Bridging senses: new developments in synaesthesia’. Over 2 days, we exchanged views on the nature of this fascinating condition and what it might tell us more broadly about the biology of human sensory experiences, with a series of engaging talks presenting new research findings, as well as lively interactive debates. In the current special issue, comprising 16 different articles, speakers and participants from the meeting have contributed peer-reviewed opinion pieces, reviews and original research capturing both the state of the art and insights into future directions of this dynamic field.

2. Perspectives on synaesthesia

The issue begins with several leading voices laying out their distinctive perspectives on synaesthesia [5–8]. Jamie Ward presents a view of synaesthesia as a special condition with causal underpinnings distinct from neurotypical cognition, but at the same time one that encompasses a considerable range of profiles [5]. For example, people vary in the number of types of synaesthesia that they possess, and this measure correlates with differences in other cognitive traits including sensory sensitivity, attention-to-detail and mental imagery, among others. The article pushes for the field to move beyond focusing on individual forms of synaesthesia (such as grapheme–colour, see §3) to a more holistic framework that could better link the associated cognitive profiles with their neurobiological underpinnings [5].

The broader phenomenology of the condition is similarly stressed in the article by Lalwani & Brang [6]. The authors discuss prominent models of synaesthesia from the prior literature, such as increased connectivity between sensory brain regions and disinhibition of cortical feedback, and argue that these cannot fully account for the differences in (for example) imagery, perceptual sensitivity and cortical excitability that have been documented in the affected people. They put forward a new ‘stochastic resonance’ model [6] involving increased neural noise, which can explain not only synaesthetic experience (because pre-existing multisensory circuits elicit supra-threshold activation), but also the broader cognitive/perceptual correlates of the condition, as well as accounting for developmental and induced forms of synaesthesia.

Expanding on this theme of unified explanations for developmental and induced synaesthesia, Schwartzman et al. [7] review a range of methods that have been claimed to artificially induce synaesthesia in non-synaesthetic individuals, including intense cognitive training on letter–colour associations, psychedelic drugs, hypnosis, sensory deprivation and brain injury. This work demonstrates how perceptual plasticity can persist even into adulthood. The authors highlight phenomenological, behavioural and, in some cases, neural parallels between developmental and induced synaesthesia, while arguing that the variable degree of ‘perceptual presence’ should be a major focus for further investigations in this area [7].

Considering the future of synaesthesia research as a whole, the article by Mankin [8] provides an urgent call to overhaul the field, arguing that work thus far has been artificially constrained by core assumptions about the nature of synaesthetic experiences, limiting not just the scope of enquiry, but also the designs and methods employed. The author discusses how current standards for characterizing and studying grapheme–colour synaesthesia fail to capture levels of variation and detail (such as the quality of letter dominance) that may be fundamental to synaesthetic experience [8]. Mankin stresses the need for large-scale investigations of the ways that people with synaesthesia subjectively interpret their experience of colours for words, as well as studies of systematic mappings between synaesthetic colours and linguistic features.

3. Learning in colour

As highlighted by the above overview papers [5–8], the majority of our knowledge about synaesthesia has so far come from investigations of grapheme–colour and related types, and this forms the primary focus of four research articles of the special issue. Studies of adults suggest that these types of synaesthesia are sometimes associated with higher cognitive skills (see [5] in this issue for discussion). The laboratory of Julia Simner has been investigating this question (among others) from a developmental perspective, building on a longstanding interest in childhood synaesthesia (for example [9]). Smees et al. [10] intensively tested a cohort of greater than 2000 children from 130 randomly selected geographical regions of Scotland and identified 51 with grapheme–colour synaesthesia. They studied the performance of these 51 children on vocabulary and sentence comprehension measures at 10 years of age, as compared to non-synaesthetic peers with either average or high memory performance. After accounting for demographic differences, children in the synaesthesia group showed modestly higher expressive and receptive vocabulary, without significant differences in sentence comprehension, as well as reporting higher academic self-concept for reading, but not for numeracy [10]. Retrospectively comparing expressive vocabulary at 10 years of age with earlier measures at 5 years of age, the authors noted that children in the synaesthesia group had not showed an a priori advantage at the start of schooling, but had subsequently increased their vocabulary at a higher rate than the control groups.

Further exploring the connections between synaesthesia and language/literacy, Asano et al. [11] provide two lines of experimental evidence to support the view that grapheme–colour synaesthesia builds on normal language processing. In a first experiment, the authors focused on Japanese kanji (logographic) characters that represent antonym pairs (with opposite meanings, like ‘up’ and ‘down’). They found complex influences of semantic relations on grapheme–colour associations of Japanese people with synaesthesia, in which the effects were strongest for characters that had been learned earliest during schooling [11]. A second experiment demonstrated that when Japanese synaesthetes learn novel meanings or sounds for kanji characters, this leads to slight but significant reductions in the consistency of test–retest grapheme–colour associations, suggesting that synaesthetic colours can be modulated to reflect updated grapheme knowledge.

The study by Root et al. [12] delves further to ask how specific grapheme–colour associations arise during development, taking advantage of gender–colour stereotypes that are prominent in Western cultures. They report that, both in Dutch- and English-speaking samples, adult females with synaesthesia associate the first initial of their name with the colour pink more often than expected by chance, while those without synaesthesia associate this instead with their current favourite colour. The authors propose a model in which environmental factors evoke associations between graphemes and colours in all people (children and adults; synaesthetes and non-synaesthetes), but that children with a synaesthetic predisposition ‘lock in’ particular associations during development, yielding the stable associations in adulthood that are hallmarks of the condition [12]. In a related paper, Rouw & Root [13] note that while synaesthetes and non-synaesthetes may show similar biases in the specific letter–colour associations that they report, it has not previously been determined whether the colours themselves differ in their properties. The authors find that, compared with colour associations reported by non-synaesthetes, the colour concurrents of synaesthetes have an over-representation of unmixed hues, with the increased presence of warm (yellow-orange/brown) and achromatic (grey, white and black) colours [13]. Based on the existence of this ‘synaesthetic colour palette’, they develop a machine-learning approach for identifying people with synaesthesia without relying on more traditional test–retest consistency methods.

Together, these four articles [10–13] provide a snapshot of how investigations of grapheme–colour (and related forms of) synaesthesia are being used to address deep-seated questions about cognitive development, language acquisition, and the interplay of genes and environment in our learning processes.

4. Mental imagery

An area of growing interest for the field centres on the idea that synaesthesia is associated more broadly with enhanced mental imagery [14]. Indeed, two of the articles of this special issue are specifically concerned with what synaesthesia can tell us about imagery, and vice versa. O'Dowd et al. [15] argue that understanding of these relationships has been limited by a lack of knowledge about the nature of imagery for sensory domains other than vision. In one experiment, they investigated three groups of English-speaking participants (n = 10 for each): mirror-touch synaesthetes (see also §6), individuals with other forms of synaesthesia, and non-synaesthetic controls. For the mirror-touch synaesthesia group, there were higher ratings of imagined touch when viewing regions of the body, as well as enhanced tactile imagery of object properties and materials that were not directly implicated in the synaesthetic experience (for example, on being asked to imagine squeezing a wet sponge, synaesthetes reported higher ratings for the vividness of the experience) [15]. A second experiment, involving 16 English-speaking grapheme–colour synaesthetes and 22 controls, found that induced colours in synaesthesia evoked cross-modal correspondences of weight, considered a tactile property of objects, in a manner that was consistent with that experienced for veridical colours (red/blue perceived as heaviest and yellow as lightest). Thus, the authors provide further evidence of enhanced imagery as one of the hallmarks of synaesthesia and stress the importance of a cross-modal perspective on this topic [15].

Spiller et al. [16] investigated 70 adult individuals who were recruited to their study without reference to either synaesthesia or imagery, to ensure that they were naive to the goals of the research. Regardless of whether or not they were synaesthetic, all participants performed letter and number colour-picking consistency tests (see also [12,13] in this issue), and their mental imagery abilities were evaluated via self-report (Vividness of the Visual Imagery Questionnaire and Spontaneous Use of the Imagery Scale) and behavioural tests (Animal Tails and Mental Rotations). Colour-picking consistency was correlated with performance on the Animal Tails visual imagery test, but not with the Mental Rotations task or either self-report measure [16]. In regression models, there was a significant relationship between consistency in colour-picking for letters (but not for numbers) and scores on the Animal Tails test; higher colour-choice consistency predicted faster and more accurate performance in object imagery. Given the unselected nature of the sample, the authors argue that relationships between synaesthetic experience and imagery extend beyond synaesthesia and into the general population [16]. In general, a shift towards studies of naive participants for experimental studies could represent a valuable step forwards for synaesthesia research.

5. Connections to neuropsychiatric variation

While synaesthesia is a condition that often occurs in healthy individuals, there is considerable interest in how aspects of its aetiology may connect with other major neurodevelopmental and neuropsychiatric traits. In recent years, several studies have reported an increased prevalence of synaesthesia in autism spectrum disorder [17,18], and it has been argued that atypical sensory sensitivity could be a shared feature that connects them [1,19]. van Leeuwen et al. [20] analysed 79 Dutch-speaking synaesthetes of mixed types and 76 controls, recruited using nationally advertised crowd-sourcing and online platforms, and found that synaesthetes had higher scores on attention-to-detail and social-skill subscales of the Autism Spectrum Quotient and on hypersensitivity scales of the Glasgow Sensory Questionnaire [20]. They also showed elevated motion coherence thresholds and higher performance on an embedded figures task, consistent with reduced global and enhanced local perception, respectively. However, in a follow-up analysis of 18 people with sequence-space synaesthesia (where ordinal sequences like numbers, months or letters of the alphabet are perceived to occupy points in space), no significant differences from controls were observed on questionnaires and the embedded figure tasks, and the synaesthetes showed reduced (rather than elevated) motion coherence thresholds [20]. The discrepancies in the latter sample might perhaps relate to the more limited sample size and the fact that the cases were only affected with a single form of synaesthesia.

Tilot et al. [21] approached potential overlaps with other brain-related traits from a different direction, exploiting methods from molecular epidemiology. In the largest molecular genetic study of synaesthesia so far, the team recruited a substantial new cohort (n = 723) of unrelated grapheme–colour synaesthetes, verified by consistency testing, all of whom were then genotyped for common DNA polymorphisms across the genome [21]. Using a matched set of non-synaesthetic controls (n = 2181), the study assessed whether polygenic scores derived from large-scale genome-wide association studies of two relevant neuropsychiatric disorders were associated with synaesthesia status. An association was found between schizophrenia polygenic scores and synaesthesia, although the variance explained was very small, of a similar magnitude to that seen in studies of schizophrenia polygenic scores and creativity [22], while no significant relationship was seen for autism or for body mass index (included as a negative control). The study represents a first step for uncovering shared neurogenetic pathways that might connect synaesthesia to other traits, which promises to become more informative as investigations of common genomic variation continue to increase in their size and scope.

Schreiter et al. [23] note that neuropsychiatric disorders involving delusions and hallucinations show that non-veridical perceptions (i.e. not existing in the outside world) can have effects on a person's behaviour. In their article, synaesthesia provides a model condition for asking whether non-veridical perceptions that are non-pathological can also affect actions in healthy individuals. The researchers studied 19 people with grapheme–colour synaesthesia using a novel behavioural paradigm that combines Simon and Go/NoGo tasks to measure inhibitory control within the context of automatic and controlled action selection, as well as probing the neurophysiological correlates using electroencephalography methods [23]. The authors conclude that non-veridical perceptions can indeed affect behaviour outside pathology, and suggest that the effects are explained not only by early stage perceptual processes, but rather higher-order executive control processes linking perceptions to motor responses.

6. Mirror-sensory synaesthesia

This discussion meeting issue ends with two investigations of vicarious perception: when a conscious percept is elicited on a person's body solely from observing the sensation experienced by another individual. Note that there is a continued debate in the literature over whether and how these particular types of experiences (and associated conditions) fit with canonical views of synaesthesia [24,25]. In their study of the relationship between body awareness and vicarious pain perception, Bowling et al. [26] investigated three groups of participants recruited through online testing: ‘non-responders’ who did not report conscious vicarious experiences (n = 432); ‘sensory-localized responders', reporting localized conscious vicarious experiences for the body part that matches the observed stimulus, with sensory descriptors (n = 106); and ‘affective-generalized responders’, reporting generalized conscious vicarious experiences for the whole body, with more affective descriptors (n = 70). Conscious vicarious pain responders (regardless of type) had higher self-reported depersonalization and interoceptive sensibility as well as more internally oriented thinking than non-responders, without significant differences in anxiety. The authors conclude that a stable sense of bodily self may be important for vicarious perception, potentially enhanced by attention towards internal bodily states [26].

Finally, Ioumpa et al. [27] studied a group of females (total n = 18) with mirror-sensory synaesthesia and found that they described more extreme reactions to arousing images depicting touch or pain, as compared with female-matched controls (total n = 18), albeit in the absence of group-based differences in physiological (heart rate, skin conductance and pupil dilation) or hormonal (cortisol-level) responses. The authors also report that the mirror-sensory group donated greater amounts of money in a game designed to assess altruism, and had higher scores on an index of empathy, suggesting links between this form of synaesthesia and prosocial behaviour [27].

7. Conclusion

It is evident from the many valuable discussions at our Royal Society meeting in London, and from this accompanying collection of articles, that the topic of synaesthesia is not just concerned with the bridging of senses. The theme of bridges permeates this entire area of science: bridges between the disciplines needed to study synaesthesia; bridges between synaesthesia and other cognitive traits (sensory sensitivity, attention to detail, imagery etc.); bridges between developmental and induced forms of synaesthesia; and bridges between neurotypical traits and disorder, to name a few. Advances in theoretical models, experimental methods, analytical approaches and cross-disciplinary syntheses make this a particularly exciting time for the field. Crucially, by investigating synaesthesia in the diverse ways that are showcased in this special issue, we are not simply querying the specific phenomenology of a curious condition, but rather revealing essential properties of the neural systems that underpin human experience.

Data accessibility

This article has no additional data.

Authors' contributions

S.E.F. wrote the first draft of the paper and A.K.T. revised the manuscript. Both authors approved the final publication.

Competing interests

We declare we have no competing interests.

Funding

S.E.F. was supported by the Max Planck Society. A.K.T. was supported by European Commission grant no. H2020-MSCA-IF-2015 (grant no. 704393; SynGenes).