Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Sep 1.
Published in final edited form as: Conscious Cogn. 2013 Jul 14;22(3):944–954. doi: 10.1016/j.concog.2013.06.002

Synesthetic grapheme-color percepts exist for newly encountered Hebrew, Devanagari, Armenian and Cyrillic graphemes

Christopher David Blair 1,*, Marian E Berryhill 1
PMCID: PMC3778145  NIHMSID: NIHMS492313  PMID: 23860303

Abstract

Grapheme-color synesthetes experience color, not physically present, when viewing symbols. Synesthetes cannot remember learning these associations. Must synesthetic percepts be formed during a sensitive period? Can they form later and be consistent? What determines their nature? We tested grapheme-color synesthete, MC2, before, during and after she studied Hindi abroad. We investigated whether novel graphemes elicited synesthetic percepts, changed with familiarity, and/or benefited from phonemic information. MC2 reported color percepts to novel Devanagari and Hebrew graphemes. MC2 monitored these percepts over 6 months in a Hindi-speaking environment. MC2 and synesthete DN, reported synesthetic percepts for Armenian graphemes, or Cyrillic graphemes + phonemes over time. Synesthetes, not controls, reported color percepts for novel graphemes that gained consistency over time. Phonemic information did not enhance consistency. Thus, synesthetes can form and consolidate percepts to novel graphemes as adults. These percepts may depend on pre-existing grapheme-color relationships but they can flexibly shift with familiarity.

Keywords: Synesthesia, Synaesthesia, Learning

1. Introduction

For most of us, the letter ‘T’ is not blue and happy, nor the number ‘7’ shiny, yellow and good. Yet those with synesthesia reliably experience perceptual phenomenon absent in the stimulus. Although it sounds like a holdover from the psychedelic era, reports of synesthesia date back 200 years (reviewed in Jewanski, Simner, Day, & Ward, 2011). The most commonly studied form of synesthesia is grapheme-color synesthesia (Simner et al., 2006), in which a grapheme reliably elicits a color percept (reviewed in Hochel & Milan, 2008; Hubbard, 2007; Hubbard & Ramachandran, 2005; Mattingley, 2009; Rich, Bradshaw, & Mattingley, 2005; Sagiv, Heer, & Robertson, 2006; Ward, 2013; Ward & Mattingley, 2006). Synesthetes do not remember forming grapheme-color associations. Indeed, anecdotal reports describe synesthetes feeling frustration when encountering ‘wrongly’ colored alphabets in preschool (e.g. our participant MC2). Others discover their unique percepts later in life when they find that such percepts are not universal (e.g. our participant DN; see also Mills, Metzger, Foster, Valentine-Gresko, & Ricketts, 2009; Mills et al., 2002). Because synesthesia emerges early in childhood, it is difficult to definitively answer questions related to the development of synesthetic experiences. There are few papers describing the acquisition and stability of grapheme-color synesthesia. Simner et al. demonstrated that grapheme color synesthesia may be identified in children as young as 6 years of age (2009). Furthermore, testing children at the beginning and end of a year revealed significant improvements in the consistency of their synesthetic percepts (Simner, Harrold, Creed, Monro, & Foulkes, 2009). Spector and Maurer have worked with pre-literate and literate children to clarify effects of literacy on the development of synesthesia (2009, 2011). They demonstrated that toddlers made predictable color matches to letters and shapes (“O” and ameboids are white; “Z” and jagged shapes are black), but these were based on letter shape and not letter sounds; whereas they failed to make other associations commonly observed in literate children and adults (“B” is blue, “A” is red, etc.) (Spector & Maurer, 2009, 2011). Other findings show that grapheme-color synesthesia can exact a cost in task performance. Green and Goswami showed that children aged 7–15 years with grapheme-color synesthesia experience interference during numerical tasks if the digits were printed in color incongruent with their synesthetic percepts (2008).

One possibility is that an early sensitive period during development is essential for the formation of grapheme-color percepts. If true, novel graphemes encountered in adulthood might not produce synesthetic percepts. Alternatively, synesthetes may continually and automatically develop new grapheme-color percepts, especially for symbols similar to familiar graphemes. Another possibility is that there is an early sensitive period in which color associations are made, and that these associations may then map onto newly encountered graphemes later in life based on similarities in shape, name, meaning, use, etc. Indeed, support for the latter view comes from a recent finding demonstrating that some synesthetes can directly transfer one pre-existing synesthetic color percept onto a novel symbol. These authors presented words written in English but replacing the Latin letter ‘A’ with a novel Glagolitic grapheme and found that the new grapheme produced the same synesthetic color percept associated with the familiar letter ‘A’ (Mroczko, Metzinger, Singer, & Nikolic, 2009). This finding shows that an explicit transfer of synesthetic color to a novel grapheme serving as a placeholder can occur in as little as 10 min. Yet it remains unclear if or how quickly grapheme-color percepts arise for novel graphemes presented without a pre-existing context. Even if novel graphemes elicit a synesthetic percept, the quality of these adult-acquired synesthetic percepts may be different. Case studies of synesthetes describing their experiences with multiple learned alphabets (Mills et al., 2002, 2009; Rich et al., 2005; Witthoft & Winawer, 2006) show that adult synesthetes may overlay pre-existing percepts onto novel graphemes based on similarities between letter shape (Mills et al., 2002; Witthoft & Winawer, 2006) or phoneme (Mills et al., 2009; Witthoft & Winawer, 2006). In the other direction, Mills and colleagues reported that synesthete MLS associated new synesthetic colors with known graphemes, but these associations were temporary and qualitatively different from the synesthetic experiences associated with her native language (Mills et al., 2002). These findings point towards a flexible relationship between grapheme and synesthetic experience, but they do not clarify whether novel graphemes can produce synesthetic percepts independent of specific training or implied association or whether these percepts remain stable and consistent. Furthermore, these studies examined the color associations and consistency of grapheme sets that have already been learned at some earlier point, and thus cannot characterize the acquisition process of grapheme and color associations.

Here, we investigated the existence and consistency of grapheme-color associations for newly encountered novel grapheme sets in two grapheme-color synesthetes. In one unique case, we tested a synesthete before, during, and after she studied Hindi while abroad in India. Upon her return she and a second synesthete were trained in two new novel alphabets in a training study. In contrast to previous association forming studies (Mroczko et al., 2009) all graphemes were presented without any additional context, with the exception of Cyrillic characters, which were presented with an audio recording of their name. The present findings take advantage of two complementary approaches: one ecological and one laboratory based, to provide converging evidence regarding the consistency and quality of synesthetic percepts for novel graphemes as they become more familiar.

2. Experiment 1: Synesthetic color percepts to novel graphemes

Can novel graphemes elicit a synesthetic color percept? If acquiring color percepts for graphemes without additional context requires that they be encountered during a sensitive period over which grapheme-color links form in early childhood, there should be no synesthetic color percept to entirely novel graphemes. Alternatively, it may be the case that synesthetic percepts are automatically elicited for novel graphemes throughout the lifetime, or that previous color associations may map onto new graphemes. If true, novel graphemes should elicit a synesthetic color percept.

2.1. Method

2.1.1. Participant

MC2 (female, 26 years old) a psychology undergraduate at the University of Nevada, Reno participated. The University’s Institutional Review Board approved this and all protocols. MC2 and all participants (participating in Experiment 3) signed informed consent documents. MC2’s first language is English. MC2 reports taking four semesters of college Spanish, but does not claim fluency in any language other than English.

2.1.2. Synesthesia assessment

MC2 describes her synesthetic experience of graphemes not only in terms of color, but also in texture, personality, and emotional terms; we tested grapheme-color percepts. MC2 further reports that she has experienced these percepts for as long as she can remember, and does not recall a time before she had color associations. She reports that she first recognized that her experiences were not typical for everyone when she encountered an “inappropriately colored alphabet” in school. MC2 experiences color rapidly and automatically. This is confirmed by visual pop-out experiments similar to those reported by Ramachandran and Hubbard (2001) and Ward, Jonas, Dienes, and Seth (2010). She identifies briefly presented (26–1000 ms) shapes defined by the synesthetic color of a letter embedded in letter arrays with different synesthetic colors. She can also use her synesthetic percepts to perform arithmetic. For example, when presented with the equation “7 + 2=” followed by a blue color patch, she can quickly and accurately answer true or false (McCarthy, Barnes & Caplovitz, 2013).

To confirm that MC2 was a grapheme-color synesthete, she completed the grapheme color program in the Texsyn Tool-box running in Matlab (Eagleman, Kagan, Nelson, Sagaram, & Sarma, 2007). Each Latin grapheme (letters A–Z, digits 0–9) appeared with a color palette. Participants selected the color best corresponding to their synesthetic percept; there was also a ‘no color’ option. Next, a square patch of the selected color appeared and the participant adjusted its brightness before final selection. Graphemes were presented three times each in pseudorandomized order per ~30-min session. Three values were derived. First, the within-grapheme consistency score reflects the similarity between the three colors selected per grapheme. Within-grapheme consistency is measured in the form of a “difference score.” Lower difference scores indicate higher consistency. In other words, if the “3” was always electric blue there is high consistency/low difference score whereas selections of electric blue, yellow and pink would have low consistency/high difference score. Second, the within-session consistency reflects overall consistency and is calculated by averaging the within-grapheme consistency scores. Participants scoring <1 are considered grapheme-color synesthetes. MC2’s overall consistency score was 0.34; see Fig. 1. Third, the between-session consistency scores reflect the stability of synesthetic percepts over time (Simner et al., 2009). For example, during session 1, ‘B’ elicited a selection of brown and during the next session ‘B’ remained paired with brown. The average color values (RGB) across sessions were compared to assess the reliability of the synesthetic color percept over time. The absolute differences between the old and new average colors for R, G, B were calculated separately, then summed and divided by 255 (color values in RGB color space may be modulated between a minimum of zero, and a maximum of 255) to yield the between session consistency score for a particular grapheme. Once again, lower values indicate greater consistency.

Fig. 1.

Fig. 1

Open in a new tab

A subset of synesthete color choices for MC2 (left) and DN (right) (see Supplementary Materials for full sets). Each row reflects the averaged colors for each grapheme during a single session. The results display the first and last sessions (when available) for each grapheme set: Latin, Hebrew, Devanagari, Armenian and Cyrillic. Graphemes shown in outline represent those for which the “no color” response was selected. Underlined Cyrillic graphemes represent those included in the non-Latin resembling subset.

2.1.3. Tests of novel graphemes

We modified the synesthesia assessment to measure MC2’s synesthetic percepts to novel graphemes. MC2 first viewed the Latin alphabet. She next viewed 10 Hebrew letters (Bet ב, Dalet ד, Gimel ג, He ה, Lamed ל, Mem מ, Qof ק, Resh ר, Shin ש, Tav ת). The Hebrew letters served as a control condition for Experiment 2 as MC2 received no additional information about the Hebrew characters in the form of their names, sounds, meanings etc. over the course of testing, and was only exposed to them during testing sessions. This subset of Hebrew letters was chosen because these characters were less similar to Latin characters, and to minimize the length of the lengthy testing process MC2 graciously endured. We note that in a later experiment, a control grapheme set of length comparable to that of the test was used. MC2 also viewed 49 characters from the Devanagari alphabet that is used in Hindi. MC2 ran each of these programs twice on different days to obtain baseline values and to permit calculation of initial between-session consistency; see Fig. 1. Finally, we asked MC2 to report any similarities in shape between these graphemes and familiar graphemes. A period of roughly two and a half months passed between these two sessions.

2.2. Results and discussion

The within- and between-session grapheme-color consistency scores were calculated for each alphabet. MC2’s within-session difference scores were: Latin session 1, 2: 0.34, 0.28 (M = .31); Hebrew session 1, 2: 0.71, 0.78 (M = .68), Devanagari session 1, 2: 0.68, 1.00 (M = .84); see Fig. 2a. The between-session consistency scores indicating consistency over time were also very good: Latin = 0.18; Hebrew = 0.48; Devanagari = 0.79; see Fig. 2b. As expected, her most consistent performance was for her native Latin alphabet. However, performance on Devanagari and Hebrew letters was also highly consistent. In short, MC2 has stable initial synesthetic grapheme-color percepts for novel symbols.

Fig. 2.

Fig. 2

Open in a new tab

(A) Within-session grapheme-color consistency for Experiments 1 and 2: MC2’s average difference scores for each grapheme set for each session. The larger symbols of the first two points for each grapheme set represent performance in the sessions described in Experiment 1. The dashed line at 1 represents the cutoff score below which people are considered to qualify as grapheme-color synesthetes. (B) Between-session consistency for Experiments 1 and 2: Consistency in average color choices for the same grapheme between sessions for each grapheme set. The first data point for each grapheme set (drawn larger) represents data described in Experiment 1.

Our first question was to investigate whether adult grapheme-color synesthetes have synesthetic percepts for novel graphemes. Synesthete MC2 reported synesthetic color percepts associated with the presentation of novel Devanagari and Hebrew letters. Consequently, the answer to our first question is a resounding, “Yes”. Synesthetic percepts occurred even when the novel graphemes were not associated with a phoneme or other contextual information and even when she did not think the letters resembled previously encountered graphemes. Thus, at least some synesthetes are able to form novel associations immediately. Given that during these initial sessions, only shape/form information was available, it would appear that color associations may be based entirely on form information. This is somewhat surprising, given that synesthetes do not report color associations for every symbol and grapheme encountered. For example, MC2 reports that “#” elicits no color percept for her.

3. Experiment 2: The effect of exposure on synesthetic experience

Our second question was to investigate if and how synesthetic percepts to novel graphemes change with increased familiarity. MC2 provided us the rare opportunity of studying a grapheme-color synesthete studying a new language in a foreign country. She agreed to test herself in the grapheme-color association tests for Latin, Hebrew and Devanagari letters during a semester in India. This allowed us to evaluate consistency across three alphabets: Latin – her native language, Hebrew – a novel untrained set, and Devanagari – a novel set accompanied by increased familiarity through academic training and cultural immersion. We predicted that synesthetic percepts to Latin graphemes would not be labile after a lifetime of consolidation. We expected that living in a Hindi-intensive environment would enhance the consistency of percepts associated with Devanagari. However, it also seemed possible that some grapheme-color associations might shift with the addition of contextual information such as phoneme. We expected her responses to Hebrew letters to become somewhat more consistent due to familiarity from testing, but to improve less than Devanagari because she was not in a Hebrew-immersive setting.

3.1. Method

In India (August–January, 2011–2012), MC2 studied Hindi in an academic/immersion setting. Anecdotally, MC2 reported that Devanagari graphemes were difficult to learn because they were differentiated by subtle sound and shape differences. She tested herself on her laptop six times in India, in addition to the two previous sessions performed in the laboratory at the University of Nevada, Reno. Thus, lighting, screen size, and monitor color output differed from the initial testing. This change inflates the between-session consistency score for sessions 1–2; Fig. 2b. All three grapheme sets (Latin, Devanagari, & Hebrew) were tested during each testing session. The first testing session abroad was performed on September 2, 2011 and subsequent sessions occurred at 2–3 week intervals. It must be noted that MC2 had some knowledge of the study’s purpose. She was aware that the study focused on the effect of continued exposure on novel grapheme color consistency. Further, she did see a brief visual representation of the relative consistencies for each grapheme tested, as well as her overall difference score for the grapheme set after each testing session. However, given the precision with which the paradigm measures color choices, it seems unlikely that her performance could have been significantly affected.

3.2. Results and discussion

To assess the consistency of MC2’s grapheme-color associations over time, we first examined the within-session consistency scores. Improved consistency was confirmed by good fits to power functions with negative exponents, indicating an overall drop in difference scores over time: Latin: R2 = 0.69, Hebrew: R2 = 0.29, Devanagari: R2 = 0.47; see Fig. 2a. MC2’s consistency improved for Devanagari and Hebrew graphemes. As expected, there was little change in her consistency for Latin letters.

The between-session consistency scores indicating whether MC2 selected the same color for each grapheme across sessions were also informative; see Fig. 2b. As expected, Latin consistency remained stable. Consistency improved over time for Devanagari and Hebrew graphemes. However, various graphemes underwent notable shifts in their synesthetic color associations. In Hebrew, the decrease for session 4–5 between-session consistency was driven by the sudden shift in MC2’s synesthetic color percept for 2 of the 10 Hebrew graphemes (ג, ש). In these cases, MC2 reported seeing elements of two Latin graphemes, which variably biased her synesthetic color percept towards one percept or the other inconsistently. In Devanagari, at various times the graphemes ० , ७, ध, ए, इ, ङ, ण, ष, त, and य underwent significant color shifts. For example, MC2 experienced ध as both the digit “3” and the letter “T,” which led to conflicting synesthetic percepts. Otherwise, MC2 reported that her synesthetic percept shifted from being consistent with a similarly shaped Latin grapheme to the grapheme’s associated phoneme or acquired meaning. For example, the grapheme “० ” was originally identified as the letter “O” and elicited a synesthetic color percept consistent with that letter. However, MC2 reported that as she learned the meaning of this grapheme, which is actually the number “0,” her synesthetic color percept remapped accordingly.

This experiment investigated if and how synesthetic percepts to novel graphemes change with increased familiarity over time. First, MC2’s synesthetic percepts to Latin graphemes were unaffected by increased familiarity with other graphemes. Second, studying Hindi in India increased MC2’s familiarity with the Devanagari alphabet and consistency improved with the associated synesthetic percepts both within- and between- session. This confirms that at least for some adult synesthetes, it is possible to develop new grapheme-color associations for novel grapheme sets that continue to stabilize over time, which may be comparable to the stabilization of color associations for a newly learned native alphabet demonstrated in school age children with synesthesia (Simner et al., 2009). We were surprised to see that she showed the same improved consistency with Hebrew letters that were encountered only during testing sessions. Since no phonemic information or additional context was available for the Hebrew characters, shape familiarity or similarity appears sufficient to improve the consistency of synesthetic color percepts, at least in the case of MC2. This result is particularly important, given that MC2 was partially aware of the aims of the experiment, and an improvement was expected for the Devanagari, but not necessarily for the Hebrew graphemes. Given that both were observed, this allays fears that MC2’s knowledge of the experimental aims may have biased her responses.

4. Experiment 3: Training of novel graphemes, and grapheme + phoneme pairs

If shape familiarity alone stabilizes synesthetic color percepts, phonemic information may not influence how quickly percepts became consistent. However, it was not clear whether phonemic information contributed any benefit or detriment, since colors for the Devanagari and Hebrew grapheme sets appeared to stabilize at the same rate. To test whether grapheme-phoneme associations affected the rate of synesthetic percept consistency, we conducted a laboratory based training experiment. Here, two grapheme-color synesthetes, MC2 and DN and two control participants were exposed to novel graphemes (Armenian) or grapheme-phoneme pairs (Cyrillic) over multiple sessions. The prediction was that if phonemic information affects the consistency of synesthetic percepts there should be a differential rate of improvements in consistency for the grapheme + phoneme pairs. To counter concerns that improved consistency is simply due to associational learning rather than synesthesia, we also included two control participants. If consistency were simply due to training there should be no difference in the consistency between the controls and the synesthetes.

4.1. Methods

4.1.1. Participants

A second grapheme-color synesthete, DN, joined MC2. DN reports synesthetic color percepts for letters, digits, days, and months. Her first language is English and she took 5 years of Italian in high school. Her Latin difference score was 0.30. MC2, but not DN, also has sound-color synesthesia. MC2 reports that spoken words, voices, noises such as typing, and timbres of music all produce color experiences for her. Both synesthetes came forward after attending a lecture on synesthesia. Synesthetic participants were female, ages 21 and 26. Two age-, gender-, and major-(psychology) matched control participants, (female undergraduates, ages 22, 23) were also tested. The controls and MC2 performed 8 sessions on different days. Due to scheduling conflicts and availability, DN first participated in 3 sessions and resumed testing after 103 days for a series of 9 sessions conducted within several weeks; see Table 1.

Table 1.

Between session intervals: days between each session for each participant. The total time between the first and last session is also included for each participant.

Participant
Session MC2 DN C1 C2
1 0 0 0 0
2 2 14 1 2
3 7 7 7 7
4 7 103 2 2
5 7 2 4 3
6 7 4 1 2
7 7 2 6 5
8 7 2 1 2
9 3
10 7
11 2
12 1
First to last 44 145 22 22
Open in a new tab

4.1.2. Stimuli

There were two novel alphabets: 38 Armenian graphemes, and 33 Cyrillic graphemes. During each session, participants performed three tasks: Cyrillic phoneme-grapheme association test, Armenian and Cyrillic synesthetic color assessments. For Cyrillic, separate within- and between-session difference score were calculated for a Cyrillic subset comprised of 16 graphemes previously judged to be non-Latin in appearance (after Mills et al., 2002). The separate analysis allays concerns that performance on Cyrillic is simply a re-mapping of Latin synesthetic percepts onto identically shaped Cyrillic graphemes.

4.1.3. Armenian and Cyrillic grapheme-color assessments

Synesthetic percepts for Armenian and Cyrillic were assessed using the protocol described previously (2.1.2) after replacing Latin graphemes with Armenian graphemes, by themselves, or Cyrillic graphemes + phonemes. Both synesthetes (Initial and Final Difference Scores: MC2 = Armenian (A): 0.81, 0.42, Cyrillic (C): 0.82, 0.31, Cyrillic Subset (CS): 1.03, 0.34, DN = A: 0.92, 0.15, C: 0.31, 0.21, CS: 0.45, 0.18), but neither control (C1 = A: 3.54, 2.01, C: 2.86, 2.78, CS: 3.12, 2.51, C2 = A: 3.31. 3.78, C: 3.63, 3.51, CS: 3.77, 3.48), scored in the synesthetic range of below 1.

4.1.4. Phoneme + grapheme association test

It was assumed that participants learned to associate Cyrillic graphemes with their respective verbal labels as the two were repeatedly paired during the Cyrillic Grapheme-Color Assessment condition. To confirm that participants were learning phoneme + grapheme pairs, they were tested at the beginning of each session. Trials began with the presentation of a Cyrillic grapheme + phoneme pair. Graphemes were displayed 4 times, on 50% of presentations the phoneme + grapheme pair matched. Participants pressed a button indicating whether they matched or not.

4.2. Results and discussion

4.2.1. Phoneme–grapheme association test

The Cyrillic phoneme + grapheme pair test confirmed that all participants demonstrated learning; see Table 2. However, there was a notable difference between the synesthetes’ final performance (M = 98.48%, standard deviation = 0) and the control participants’ final performance levels (M = 78.03%, standard deviation = 6.43). Paired t-tests between synesthetes and controls reach significance when either the first eight (original three sessions and first five of the second period) or last eight values (only sessions from the second period) from DN were included (p’s < .0001). Consequently, the synesthetes acquired the grapheme + phoneme pairs more readily than did the controls. This result is consistent with reports of memory encoding advantages for synesthetes, which may be a result of general learning and memory advantages inherent in the organization of the synesthetic brain, or a result of additional association opportunities due to both color and grapheme shape being available (Gross, Neargarder, Caldwell-Harris, & Cronin-Golomb, 2011; Radvansky, Gibson, & McNerney, 2011). We note that DN showed a substantial improvement in her association scores between sessions 4 and 5. Although speculative, one might argue that while performance on session 4 may have suffered due to the long delay between that test and the previous session, session 4 may have served to refamiliarize or “jog” DN’s memory, the effects becoming apparent in the session 5. It should also be noted the participants showed different learning patterns. MC2 learned letter name and shape associations quickly, while other participants performed near chance. Consequently, if both grapheme meaning and shape play a role in determining synesthetic color associations, we may expect MC2 to show meaning related color associations sooner, or to a greater degree than DN.

Table 2.

Cyrillic letter training accuracy: percent correct for each participant for each session in matching Cyrillic graphemes to their corresponding phonemes (chance = 50%).

Participant
Session MC2 DN C1 C2
1 91.67 51.52 54.55 65.15
2 90.15 59.85 68.18 62.12
3 93.94 69.70 69.70 66.67
4 93.18 65.91 75.00 65.91
5 96.21 84.09 84.00 70.45
6 95.45 85.61 82.58 71.97
7 97.73 93.18 78.03 68.94
8 98.48 96.97 82.58 73.48
9 100
10 98.48
11 98.48
12 98.48
Open in a new tab

4.2.2. Within-session and between-session consistency

Synesthetes performed more consistently within- and between-sessions than controls; see Fig. 3a. For the synesthetes, within-session consistency scores were initially within the synesthetic range (<1) and steadily improved, whereas the controls never achieved the cutoff level. The fitted power functions accounted for the following variance (R2): Synesthetes MC2: Armenian, Cyrillic, Cyrillic subset: R2 = A: 0.56, C: 0.70, CS: 0.65, and DN: R2 = A: 0.83, C: 0.37, CS: 0.44; Controls C1: R2 = A: 0.19, C: 0.07, CS: 0.28, and C2: R2 = A: 0.53, C: 0.02, CS: 0.08. The between-session consistency scores were always lower in synesthetes, indicating greater consistency; see Fig. 3b.

Fig. 3.

Fig. 3

Open in a new tab

(A) Within-session consistency for Experiment 3: MC2’s and DN’s average difference scores for each grapheme set per session. Controls’ data are averaged with error bars representing the standard error of the mean. The dashed line represents the cutoff score used to identify grapheme-color synesthetes. (B) Between-session consistency for Experiment 3: Consistency in average color choices for the same grapheme between subsequent sessions for each grapheme set for MC2, DN, and the Controls’ average. Error bars represent standard error of the mean.

Across participants there was no notable difference between Armenian and Cyrillic in within- or between-session consistency even though Cyrillic was accompanied by phonemic information. The correlation between phoneme + grapheme association test accuracy and Cyrillic subset consistency (R2 = 0.37) was actually weaker than the correlation between phoneme + grapheme association test accuracy and Armenian (R2 = 0.43).

Here, we asked whether the addition of phonemic information facilitated synesthetic percept consistency. There was no apparent benefit or detriment of phonemic information as associations for both alphabets appeared to increase in consistency at a similar rate for both synesthetes. We also replicated the finding that some synesthetes perceive synesthetic colors for novel graphemes using novel alphabets and we also replicated that these percepts gained consistency over time. These findings were not simply due to associational learning because controls were unable to ever perform as well as synesthetes. Yet, this does not appear to be driven by faster learning in synesthetes because the correlation between the phoneme-grapheme association test and within-session consistency was higher for the untrained Armenian, rather than the trained Cyrillic.

5. General discussion

Synesthesia is a captivating but poorly understood phenomenon. We undertook this investigation to address several open questions related to what happens when grapheme-color synesthetes encounter and learn novel graphemes. In particular, we tested whether synesthetic percepts were immediately present and the consistency of these percepts to a variety of novel graphemes. We found that for the synesthetes we tested, novel graphemes elicit color percepts at first glance and the consistency of these percepts improves with familiarity. This holds true for graphemes encountered in laboratory or immersion settings. Specifically, synesthetic percepts to Devanagari graphemes associated with phoneme and meaning in an immersion setting, or Cyrillic graphemes + phonemes trained in a laboratory, are not notably more consistent than synesthetic percepts to unaccompanied Hebrew or Armenian graphemes. Given the reports of synesthetes that grapheme name and shape were sometimes in conflict in determining grapheme color, it may be the case that increased information about graphemes increased association consistency, but this is balanced by the decrease in consistency caused by conflicts. However, if true, there should be lower difference scores for the Cyrillic alphabet after such conflicts had been resolved. While this may be the case for MC2 (Cyrillic: 0.31 Cyrillic Subset: 0.34 Armenian: 0.42), this was not the case for DN (Cyrillic: 0.21 Cyrillic Subset: 0.18 Armenian: 0.15). We also found that synesthetes may have an advantage in forming name associations for novel graphemes, suggesting that learning the novel graphemes of a new alphabet may even be facilitated by the presence and stabilization of grapheme-color percepts.

Importantly, our findings demonstrate that it is not necessary for a grapheme to be encountered during an early sensitive period during development to establish a grapheme-color association. If this were required, synesthetes would not experience synesthetic color percepts when encountering novel symbols. However, such a sensitive period may play a role in establishing color associations for synesthetes’ first learned alphabets and influence color associations formed later for novel alphabets. In fact, synesthetes and controls have reported that shape associations between previously learned graphemes can be quickly noted for many graphemes used in these experiments. For some graphemes presented in this study, synesthetes may remap colors from previously learned graphemes to novel graphemes with similar shapes. Thus, new color associations may still be reliant on a previous sensitive period.

Our results are also consistent with the view that the particular pattern of color associations may be influenced by an early sensitive period of language acquisition (Barnett, Feeney, Gormley, & Newell, 2009; Rich et al., 2005). In other words, if particular shapes, sounds, and/or grapheme meanings (Mroczko et al., 2009) are associated with particular color percepts during an early sensitive period, graphemes encountered later in life may then elicit colors consistent with graphemes with similar shapes and sounds that were encountered during this early sensitive period. Evidence supporting this view comes from several reports that childhood familiarity with commonly available colored refrigerator magnets (Fisher-Price) influences color-grapheme associations in synesthetes (Witthoft & Winawer, 2006, 2013; see also Rich et al., 2005). In contrast, the present studies relied primarily on two synesthetes. MC2 reported that she did not have these refrigerator magnets. DN reviewed the colors of the magnets and noted six matches between the magnet colors and her synesthetic percepts. Other possible mechanisms for the original acquisition of grapheme color pairings were examined by Simner et al. (2005). They considered correlations between factors such as the inherent and presentation order of graphemes, the frequency of grapheme use in the language, the ease of generation of color terms, the lexical frequency of color terms and the order in which color terms enter various languages (Berlin and Kay ordering). Their results showed the greatest correlations between grapheme frequency and color frequency, and between grapheme frequency and Berlin and Kay ordering. While colors acquired in this way for the Latin alphabet may have transferred to the novel graphemes used in this study, given the relatively limited exposure of our participants to these grapheme sets, it seems unlikely that grapheme frequency in the languages associated with these sets had any significant effect on the color associations our participants formed.

One limitation of this paper and synesthesia research in general is that it remains unclear how findings generalize across synesthetes in general. We sought to enhance the relevance of these data by testing two rather different synesthetes; MC2 reports synesthetic percepts for phonemes, DN has pure grapheme-color synesthesia. Regardless, there is notable similarity in their within- and between-session consistency. As noted in the introduction, there are other case studies confirming that multilingual synesthetes have synesthetic color percepts across their languages (Mills et al., 2002, 2009; Rich et al., 2005; Witthoft & Winawer, 2006). It was known that synesthetes can directly transfer one synesthetic percept onto a novel replacement grapheme with explicit training (Mroczko et al., 2009). We extended these findings by tracking the improved consistency of synesthetic percepts to a range of novel graphemes as they gained familiarity.

This expediency raises the question: How do synesthetic percepts emerge instantaneously? One neural theory that may account for these findings is termed the cascaded cross-tuning model. It proposes that synesthetic percepts may arise due to activity in ventral stream region V4 fed by posterior temporal grapheme areas responding to shape features (e.g. line segments, curvature) before feedback regarding grapheme identity modulates the color percept (Brang, Hubbard, Coulson, Huang, & Ramachandran, 2010). Neuroimaging evidence supports this view. Color responsive regions (V4) are activated when synesthetes view graphemes, and elicit the synesthetic color percepts, but not when they view other symbols that do not elicit synesthetic percepts (Hubbard, Brang, & Ramachandran, 2011; Nunn et al., 2002; Rouw, Scholte, & Colizoli, 2011). Furthermore, in a region of interest study examining predefined color and grapheme regions in the brain, differences were observed in V4, but there were no general differences in response to color observed between synesthetes and non-synesthetes and differences in response to graphemes were not observed between synesthetes and non-synesthetes outside of V4 (Hubbard et al., 2011). Thus, familiar physical components elicit synesthetic percepts to novel graphemes. It also addresses why novel graphemes with similar shapes to familiar graphemes elicit similar synesthetic colors. Indeed, grapheme-color synesthetes often have similar percepts for similar letters within the Latin alphabet itself (Brang, Rouw, Ramachandran, & Coulson, 2011).

Much of our data are consistent with predictions of the cascaded cross-tuning model. For example, our synesthetes noted that shape similarities promoted the transfer of one synesthetic percept to another. This stands to reason, given that no other information was available to participants about the graphemes apart from their shape with the exception of Cyrillic characters, which were presented with their name, and the Devanagari characters as MC2 studied them. However, transfer can also create problems. MC2 indicated that the initial similarity-based percepts required reconciliation with the meaning and use of the grapheme within the context of the Hindi language, and in Cyrillic with the sound of the letter’s name. This prompted notable shifts in the grapheme-color association over time. By way of example, the Cyrillic grapheme “H” originally elicited the same synesthetic color as the Latin letter “H.” However, this grapheme is associated with an “N” sound in Russian. By the end of testing, MC2 reported a synesthetic color that is closer to that of the Latin grapheme “N.” Furthermore, some graphemes bore little physical similarity to known graphemes. Indeed, this was one reason why we chose Hebrew, Devanagari and Armenian graphemes. Thus, there appears to be an influence of other associations on synesthetic percepts. Consistent with this view is the finding that Japanese synesthetes’ percepts for Kanji (logographs) appear to be strongly driven by phonological information and meaning rather than orthographic similarity to Hiragana (syllables) (Asano & Yokosawa, 2012). In Japan, Kanji is taught several years after children learn Hiragana, but also during childhood. Thus, learned phonological information and grapheme meaning appear to trump physical similarity in driving the synesthetic percept. Other studies have shown that grapheme meaning may have a greater effect on perceived photisms than grapheme shape. For example, when ambiguous graphemes are presented to synesthetic participants, (e.g. may be seen as an “S” or a “5”), changing the context in which the grapheme is viewed can completely change its perceived synesthetic color (Dixon, Smilek, Duffy, Zanna, & Merikle, 2006). However, it has also been shown that grapheme case and font can affect the saturation of resultant photisms (Witthoft & Winawer, 2006). Furthermore, there have been various situations cited in which the meaning of a word conflicts with its perceived synesthetic color (“Blue” is red) (Gray et al., 2006), suggesting that meaning and usage of graphemes may not always be the only, or even principal determinant of perceived synesthetic color. These results seem to reinforce that grapheme shape, as well as meaning, have an effect on synesthetic percepts, though it is not entirely clear what the relative contribution of each may be, or to what degree one takes precedence over the other.

Although the present data addressed our primary research questions, it raised several others. MC2 reports sound-grapheme synesthetic colors, but the addition of phonemes in Experiment 3 did not further enhance the consistency of her synesthetic percepts beyond the presentation of the grapheme alone. As discussed above, multiple competing processes resulting from the presentation of additional phonemic information may be leading to a null final result in our paradigm. However, despite the lack of effect of phonemic information on color consistency, we did find that when there was a conflict between the sound and shape of a grapheme, MC2’s final colors for Cyrillic appear to be divided equally between being consistent with the grapheme’s name, its shape, and something in between. In the case of DN, who does not report sound-color synesthesia, the majority of such graphemes elicited colors consistent with Latin graphemes of similar shape by the end of testing. While our results provide some additional clues, it still remains unclear exactly how different types of synesthetic percepts are modulated by experience or how they interact with each other. Further, the durability of the synesthetic percepts for novel graphemes has not been tested. It seems likely that the percepts remain constant over time, but lack of continued exposure may produce some degree of extinction. In the future, it would also be interesting to compare the grapheme-color percepts for native language speakers versus later learners and their respective patterns of cortical activations. Synesthesia continues to raise intriguing questions relating perception and conscious experience. What is clear is that color associations for novel graphemes can be formed with only shape information. However, shape is clearly not the only determinant of synesthetic color association, as acquisition of meaning and phonemic information can modulate colors originally associated based solely on shape information. These findings also confirm that the laboratory can shed light on the process by which new synesthetic percepts naturally emerge.

Supplementary Material

1

Acknowledgments

We would like to thank MC2 and DN as well as our two control participants for their patience and cooperation throughout these experiments. Research reported in this publication was generously supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under Grant Number P20GM103650, Grant NEI R15EY022775 and start-up funding from the University of Nevada to MEB.

Footnotes

Appendix A. Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.concog.2013.06.002.

References

  1. Asano M, Yokosawa K. Synesthetic colors for Japanese late acquired graphemes. Consciousness and Cognition. 2012;21(2):983–993. doi: 10.1016/j.concog.2012.02.005. [DOI] [PubMed] [Google Scholar]
  2. Barnett KJ, Feeney J, Gormley M, Newell FN. An exploratory study of linguistic-color associations across languages in multilingual synaesthetes. The Quarterly Journal of Experimental Psychology. 2009;62(7):1343–1355. doi: 10.1080/17470210802483461. http://dx.doi.org/10.1080/17470210802483461. [DOI] [PubMed] [Google Scholar]
  3. Brang D, Hubbard EM, Coulson S, Huang M, Ramachandran VS. Magnetoencephalography reveals early activation of V4 in grapheme-color synesthesia. NeuroImage. 2010;53(1):268–274. doi: 10.1016/j.neuroimage.2010.06.008. [DOI] [PubMed] [Google Scholar]
  4. Brang D, Rouw R, Ramachandran VS, Coulson S. Similarly shaped letters evoke similar colors in grapheme-color synesthesia. Neuropsychologia. 2011;49(5):1355–1358. doi: 10.1016/j.neuropsychologia.2011.01.002. [DOI] [PubMed] [Google Scholar]
  5. Dixon MJ, Smilek D, Duffy PL, Zanna MP, Merikle MM. The role of meaning in grapheme-colour synaesthesia. Cognition. 2006;42:243–252. doi: 10.1016/s0010-9452(08)70349-6. [DOI] [PubMed] [Google Scholar]
  6. Eagleman DM, Kagan AD, Nelson SS, Sagaram D, Sarma AK. A standardized test battery for the study of synesthesia. Journal of Neuroscience Methods. 2007;159(1):139–145. doi: 10.1016/j.jneumeth.2006.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Gray JA, Parslow DM, Brammer MJ, Chopping S, Vythelingum GN, ffytche DH. Evidence against functionalism from neuroimaging of the alien colour effect in synaesthesia. Cortex. 2006;42(2):309–318. doi: 10.1016/s0010-9452(08)70357-5. [DOI] [PubMed] [Google Scholar]
  8. Green JAK, Goswami U. Synesthesia and number cognition in children. Cognition. 2008;106:463–473. doi: 10.1016/j.cognition.2007.01.013. http://dx.doi.org/10.1016/j.cognition.2007.01.013. [DOI] [PubMed] [Google Scholar]
  9. Gross VC, Neargarder S, Caldwell-Harris CL, Cronin-Golomb A. Superior encoding enhances recall in color-graphemic synesthesia. Perception. 2011;40(2):196–208. doi: 10.1068/p6647. [DOI] [PubMed] [Google Scholar]
  10. Hochel M, Milan EG. Synaesthesia: The existing state of affairs. Cognitive Neuropsychology. 2008;25(1):93–117. doi: 10.1080/02643290701822815. http://dx.doi.org/10.1080/02643290701822815. [DOI] [PubMed] [Google Scholar]
  11. Hubbard EM. Neurophysiology of synesthesia. Current Psychiatry Reports. 2007;9(3):193–199. doi: 10.1007/s11920-007-0018-6. [DOI] [PubMed] [Google Scholar]
  12. Hubbard EM, Brang D, Ramachandran VS. The cross-activation theory at 10. Journal of Neuropsychology. 2011;5:152–177. doi: 10.1111/j.1748-6653.2011.02014.x. http://dx.doi.org/10.1111/j.1748-6653.2011.02014.x. [DOI] [PubMed] [Google Scholar]
  13. Hubbard EM, Ramachandran VS. Neurocognitive mechanisms of synesthesia. Neuron. 2005;48(3):509–520. doi: 10.1016/j.neuron.2005.10.012. [DOI] [PubMed] [Google Scholar]
  14. Jewanski J, Simner J, Day SA, Ward J. The development of a scientific understanding of synesthesia from early case studies (1849–1873) Journal of the History of the Neurosciences. 2011;20(4):284–305. doi: 10.1080/0964704X.2010.528240. [DOI] [PubMed] [Google Scholar]
  15. Mattingley JB. Attention, automaticity, and awareness in synesthesia. Annals of the New York Academy of Sciences. 2009;1156:141–167. doi: 10.1111/j.1749-6632.2009.04422.x. [DOI] [PubMed] [Google Scholar]
  16. McCarthy JD, Barnes LN, Caplovitz GP. Two plus blue equals green: The minimal cost of doing math with colors in grapheme-color synesthesia. 2013. Manuscript submitted for publication. [DOI] [PMC free article] [PubMed]
  17. Mills CB, Metzger SR, Foster CA, Valentine-Gresko MN, Ricketts S. Development of color-grapheme synesthesia and its effect on mathematical operations. Perception. 2009;38(4):591–605. doi: 10.1068/p6109. [DOI] [PubMed] [Google Scholar]
  18. Mills CB, Viguers ML, Edelson SK, Thomas AT, Simon-Dack SL, Innis JA. The color of two alphabets for a multilingual synesthete. Perception. 2002;31(11):1371–1394. doi: 10.1068/p3429. [DOI] [PubMed] [Google Scholar]
  19. Mroczko A, Metzinger T, Singer W, Nikolic D. Immediate transfer of synesthesia to a novel inducer. Journal of Vision. 2009;9(12):21–28. 25. doi: 10.1167/9.12.25. [DOI] [PubMed] [Google Scholar]
  20. Nunn JA, Gregory LJ, Brammer M, Williams SC, Parslow DM, Morgan MJ, et al. Functional magnetic resonance imaging of synesthesia: Activation of V4/V8 by spoken words. Nature Neuroscience. 2002;5(4):371–375. doi: 10.1038/nn818. [DOI] [PubMed] [Google Scholar]
  21. Radvansky GA, Gibson BS, McNerney MW. Synesthesia and memory: Color congruency, von Restorff, and false memory effects. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2011;37(1):219–229. doi: 10.1037/a0021329. http://dx.doi.org/10.1037/a0021329. [DOI] [PubMed] [Google Scholar]
  22. Ramachandran VS, Hubbard EM. Synaesthesia – A window into perception, thought and language. Journal of Consciousness Studies. 2001;8(12):3–34. [Google Scholar]
  23. Rich AN, Bradshaw JL, Mattingley JB. A systematic, large-scale study of synaesthesia: Implications for the role of early experience in lexical-colour associations. Cognition. 2005;98(1):53–84. doi: 10.1016/j.cognition.2004.11.003. [DOI] [PubMed] [Google Scholar]
  24. Rouw R, Scholte HS, Colizoli O. Brain areas involved in synaesthesia: A review. Journal of Neuropsychology. 2011;5:214–242. doi: 10.1111/j.1748-6653.2011.02006.x. http://dx.doi.org/10.1111/j.1748-6653.2011.02006.x. [DOI] [PubMed] [Google Scholar]
  25. Sagiv N, Heer J, Robertson L. Does binding of synesthetic color to the evoking grapheme require attention? Cortex. 2006;42(2):232–242. doi: 10.1016/s0010-9452(08)70348-4. [DOI] [PubMed] [Google Scholar]
  26. Simner J, Harrold J, Creed H, Monro L, Foulkes L. Early detection of markers for synaesthesia in childhood populations. Brain. 2009;132:57–64. doi: 10.1093/brain/awn292. http://dx.doi.org/10.1093/brain/awn292. [DOI] [PubMed] [Google Scholar]
  27. Simner J, Mulvenna C, Sagiv N, Tsakanikos E, Witherby SA, Fraser C, et al. Synaesthesia: The prevalence of atypical cross-modal experiences. Perception. 2006;35(8):1024–1033. doi: 10.1068/p5469. [DOI] [PubMed] [Google Scholar]
  28. Simner J, Ward J, Lanz M, Jansari A, Noonan K, Glover L, et al. Non-random associations of graphemes to colours in synaesthetic and non-synaesthetic populations. Cognitive Neuropsychology. 2005;22(8):1069–1085. doi: 10.1080/02643290500200122. http://dx.doi.org/10.1080/02643290500200122. [DOI] [PubMed] [Google Scholar]
  29. Spector F, Maurer D. Synesthesia: A new approach to understanding the development of perception. Developmental Psychology. 2009;45(1):175–189. doi: 10.1037/a0014171. http://dx.doi.org/10.1037/a0014171. [DOI] [PubMed] [Google Scholar]
  30. Spector F, Maurer D. The colors of the alphabet: Naturally-biased associations between shape and color. Journal of Experimental Psychology: Human Perception and Performance. 2011;37(2):484–495. doi: 10.1037/a0021437. http://dx.doi.org/10.1037/a0021437. [DOI] [PubMed] [Google Scholar]
  31. Ward J. Synesthesia. Annual Review of Psychology. 2013;64:49–75. doi: 10.1146/annurev-psych-113011-143840. http://dx.doi.org/10.1146/annurev-psych-113011-143840. [DOI] [PubMed] [Google Scholar]
  32. Ward J, Jonas C, Dienes Z, Seth A. Grapheme-colour synaesthesia improves detection of embedded shapes, but without pre-attentive ‘pop-out’ of synaesthetic colour. Proceedings of the Royal Society B-Biological Sciences. 2010;277(1685):1021–1026. doi: 10.1098/rspb.2009.1765. http://dx.doi.org/10.1098/rspb.2009.1765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ward J, Mattingley JB. Synaesthesia: An overview of contemporary findings and controversies. Cortex. 2006;42:129–136. doi: 10.1016/s0010-9452(08)70336-8. [DOI] [PubMed] [Google Scholar]
  34. Witthoft N, Winawer J. Synesthetic colors determined by having colored refrigerator magnets in childhood. Cortex. 2006;42(2):175–183. doi: 10.1016/s0010-9452(08)70342-3. [DOI] [PubMed] [Google Scholar]
  35. Witthoft N, Winawer J. Learning, memory, and synesthesia. Psychological Science. 2013;X(XX):1–8. doi: 10.1177/0956797612452573. http://dx.doi.org/10.1177/0956797612452573. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS492313-supplement-1.pdf (832.2KB, pdf)

RESOURCES