A standardized test battery for the study of synesthesia

Abstract

Synesthesia is an unusual condition in which stimulation of one modality evokes sensation or experience in another modality. Although discussed in the literature well over a century ago, synesthesia slipped out of the scientific spotlight for decades because of the difficulty in verifying and quantifying private perceptual experiences. In recent years, the study of synesthesia has enjoyed a renaissance due to the introduction of tests that demonstrate the reality of the condition, its automatic and involuntary nature, and its measurable perceptual consequences. However, while several research groups now study synesthesia, there is no single protocol for comparing, contrasting and pooling synesthetic subjects across these groups. There is no standard battery of tests, no quantifiable scoring system, and no standard phrasing of questions. Additionally, the tests that exist offer no means for data comparison. To remedy this deficit we have devised the Synesthesia Battery. This unified collection of tests is freely accessible online (http://www.synesthete.org). It consists of a questionnaire and several online software programs, and test results are immediately available for use by synesthetes and invited researchers. Performance on the tests is quantified with a standard scoring system. We introduce several novel tests here, and offer the software for running the tests. By presenting standardized procedures for testing and comparing subjects, this endeavor hopes to speed scientific progress in synesthesia research.

1. Introduction

For the past century, synesthesia research has fought an uphill battle against the perceived subjectivity of the phenomenon. Examining the reality of these synesthetic experiences has been a goal of many researchers over the past decade (Baron-Cohen and Harrison, 1997; Cytowic, 1998; Hubbard and Ramachandran, 2003). To this end, several tests have been devised to objectively quantify a personal experience. For example, some tests distinguish synesthetes from controls by testing the consistency of synesthetic associations across multiple trials over varying periods of time (Asher et al., 2006; Baron-Cohen et al., 1996). Additionally, questionnaires have been developed to collect detailed information about personal synesthetic experience (Grossenbacher, personal communication; Hubbard and Ramachandran, personal communication; Rich et al., 2005; Ward and Simner, 2003; also see http://www.synaesthesia.uwaterloo.ca/ColorAssessment.htm, described in Smilek et al., 2005).

However, there currently exists no standardized method for presenting tests, quantifying scores and phrasing questions, as well as no ready access to the tests and their results for comparing across synesthetes. In an attempt to remedy this deficit, we have developed an online research toolbox, the Synesthesia Battery (available at http://www.synesthete.org). This battery hopes to provide a standard for the community of synesthetes and researchers. It envisages faster scientific progress by allowing comparison and pooling of data across laboratories. The software tests are offered both online and as a downloadable collection of platform-independent MATLAB programs, both freely available at http://www.synesthete.org.

2. The synesthesia battery

The first of its kind, the synesthesia battery is a web-based, online information and testing portal. On entering http://www.synesthete.org, participants register and then sign in with a password which restricts access to their data. During the process, they are given the option to enter a researcher’s email address, allowing them to privately share their test results with only that researcher. Through this mechanism, researchers worldwide can send their subjects to this website for later access to the quantified results. The results are accessible only by the synesthete or by the (optionally) invited researcher.

3. Questionnaire

The online questionnaire consists of 80 questions and requires an average of 10 min to complete, depending on the forms of synesthesia experienced by the participant. The battery begins with a screen asking whether the person believes they have any of a numbers of variants of synesthesia (20 possible choices, with a textbox for any other variants not listed). Their answers on the first screen automatically direct which parts of the questionnaire they will see and which tests they will take. The questionnaire content borrows heavily from one constructed and generously shared by Drs. Edward Hubbard and Vilayanur Ramachandran, with revisions and additions derived over the past 2 years. Some of the questions are intended to quantify traits across many groups of synesthetes, while others are intended to steer the participant to the appropriate online software tests for their form(s) of synesthesia. Finally, several questions in the survey are meant to gather neuropsychological data (such as relationships to autism, dyslexia, head trauma, tumors and so on). Most of the questions are accompanied by text fields for synesthetes to add optional comments and elaborate on personal experiences. Such free-form questions are meant to navigate future experiments; the hope is that participants will proffer new clues about the phenomenology and ontogenesis of the condition.

Why is a standardized questionnaire useful? A recognized problem in the field is the lack of a standard phrasing of certain questions. For example, it appears that some synesthetes for whom letters trigger colors will experience the color with a spatial location, while others experience the color with no particular location [“projectors” and “associators” in the terminology of some authors (Dixon et al., 2004; Smilek et al., 2001); “localizers” and “non-localizers” in the terminology of others (Cytowic and Eagleman, 2006)]. In the past, to distinguish these two types of grapheme-color synesthetes, a question has often been phrased: “Is the color experienced in your mind’s eye or out in the world?” This is an unclear phrasing of the question, since a synesthete who experiences colors with locations is nonetheless aware that the color is in their mind’s eye (it is not confused with a real world object, as in a hallucination). Thus, they might answer the question either way, and as a result the estimated numbers of synesthetes who experience location with their colors has been difficult to determine. Our battery seeks to standardize the phrasing of such questions, making the questions and answers clear and thereby opening the door to large sample numbers and straightforward comparison across laboratories. In the questionnaire we ask a collection of five different questions to distinguish projectors (localizers) from associators (non-localizers), probing whether their synesthesia is present when they close their eyes, whether their synesthetic color is associated with some object in the world, and whether they ever confuse their synesthetic perceptions with the external world, or instead whether they exist in separate dimensions.

In what follows we will describe some of the online software programs available on the website. More programs are being added as they are developed.

4. Grapheme-color consistency test

One of the most prevalent forms of synesthesia – and the most readily studied using Internet technology – is the triggering of color experience by graphemes (letters and digits; Baron-Cohen et al., 1993; Day, 2005; Simner et al., in press). It has been previously reported that grapheme-color synesthetes show greater matching consistency than non-synesthetes—that is, when asked to match colors to graphemes, and then re-tested some time later, they are largely consistent in their matches (Asher et al., 2006; Baron-Cohen and Harrison, 1997; Cytowic, 2002; Dixon et al., 2000; Mattingley et al., 2001; Odgaard et al., 1999).

Testing for consistency across time has been in use for almost a century: a Science paper in 1917 asked a synesthete to describe the colors associated with his graphemes, and then had the synesthete do this again 5 years later (Jordan, 1917). The consistency was claimed to be high, but the problem was that there was no good way of quantifying the responses: is ‘silver’ in one test the same as ‘grey’ upon later testing, or should it be counted as different? And if so, by how much?

Over the years, researchers and synesthetes have resorted to a standard assortment of colors in a crayon set, color posters mailed to participants, or Kay’s designation of 11 basic irreducible color terms for English (Berlin and Kay, 1969). These methods are limited both in terms of color choices and distribution opportunities. In observations drawn from synesthetes, it has been found that their choices are not always generic colors, but more often are quite specific hues and contrasts that they spend considerable amount of time getting just right (Cytowic, 2002; Cytowic and Eagleman, 2006). And the different colors can be within close range, but unique all the same.

The introduction of computers with their extensive color palettes (256 × 256 × 256 colors) has allowed precise ways of addressing these issues. The method used here is an internal consistency task. In other words, responses can be tested for consistency not only within a single session, but also after a time delay. Participants are presented with a single letter or digit on the computer screen (Fig. 1a). Navigating their mouse over a color palette allows them to choose one of 16.7 million different colors that most closely match their synesthetic experience for a letter. After selecting their color, they are presented with the next letter or digit. In total, a participant is presented with 108 trials (the full set of graphemes, A–Z and 0–9) three times in randomized order (Fig. 1b and c). The data is then analyzed for consistency: did the participant choose the same (or similar) colors each of the three times she saw a particular letter? Using this method of testing, synesthetes can be readily distinguished from control subjects who are asked to use free association and memory to choose their colors.

Fig. 1.

Color match consistency testing. (a) Participants are presented with randomly ordered graphemes a total of three times each (108 trials). Using a color palette, participants choose the color that best matches their synesthetic percept for that grapheme. (b) The colors chosen by participants on each of the three randomly ordered trials are displayed on the graphemes. The blue bars show the calculated distance in color space between their three answers. Controls were asked to fake synesthesia, which allowed for strategies like free association and associative memory. Controls (right) are much less consistent in their matching than self-reported synesthetes (left). (c) On http://www.synesthete.org, each subject’s report is accompanied with a graphical layout of the results and automatic scoring. Shown are screenshots from a representative synesthete and a non-synesthetic control. (d) Summary data from 15 self-reported synesthetes and 15 control subjects. All synesthetes scored below 1.0 in their consistency score, while controls had more variance in their color choices.

Formally, the color variation for each letter (v_j) is measured by

$$v_j=\sum _{c=\{\mathrm{R},\mathrm{G},\mathrm{B}\}}\mid x_1^{\mathrm{C}}-x_2^{\mathrm{C}}\mid +\mid x_2^{\mathrm{C}}-x_3^{\mathrm{C}}\mid +\mid x_3^{\mathrm{C}}-x_1^{\mathrm{C}}\mid$$ (1)

which represents the geometric distance in RGB (red, green, blue) color space. R, G, B values are all normalized to lie between 0 and 1. The total color variation score for a participant is

$$V=\frac{{\sum }_{j=\{A-Z,0-9\}}{v}_j}{N}$$ (2)

where N is the total number of graphemes for which the participant experiences synesthetic colors. It is often observed that participants will not have synesthetic color associations for some letters or numbers (additionally, sometimes participants have grapheme color synesthesia only for letters or only for numbers). To account for this, N only includes the graphemes for which there are synesthetic associations. N equals 36 for a participant who has synesthetic associations for all letters and digits.

To assess the usefulness of this measure, 15 self-reported synesthetes were tested and compared against 15 non-synesthetic control subjects who were asked to associate colors with graphemes by using free association or any strategy of their choice. Fig. 1d indicates that the color consistency score is a clear way to distinguish synesthetes from controls: synesthetes show more consistent color choices than controls. Controls have a difficult time faking synesthesia, even when that is their explicit instruction. As for the scoring, a perfect score of 0.0 would mean that there was no difference in the colors selected on each successive presentation of the same letter. We have chosen as our threshold the score of 1.0: a score below 1.0 is ranked as synesthetic, while controls score much higher than 1.0, typically in the range of 2.0. A modified version of Eqs. (1) and (2) can be used to compare re-test results taken at a later date. Note that although the number 1.0 may appear to have some significance because of its unity, this threshold was entirely driven by the data: it was the threshold that most accurately discriminated the synesthetic scores from the non-synesthetic scores. Note that this threshold should not be considered a fool-proof cut-off for discriminating synesthetes from non-synesthetes, but merely an optimal dividing line between two populations whose scores vary along a distribution.

Note that one criticism of computerized tests has been that it is impossible to control the exact appearance of a color palette on different computer monitors. This concern does not apply to our system of scoring because we are testing the difference in color choices across three separate trials on the same computer (Fig. 1b and c).

We have included in the online software an option for researchers to reinvite synesthetes to re-take the battery at any later date. Though long term consistency of synesthetic colors and associations have been reported variously in literature (Asher et al., 2006; Baron-Cohen and Harrison, 1997; Cytowic, 2002; Jordan, 1917), we found to our surprise while developing the battery that one of our synesthetes, EF, who was invited to retake the test made a change from a consistent choice of green over three trials for the grapheme G to a consistent choice of brown over three trials on the retest. The time period between the two tests was more than 6 months and when EF was shown her old color choice, she expressed surprise and said she could have sworn that it had always been brown. We hope that by carefully quantifying other cases of ‘color-drift’ over time, we can enhance our expectations about how much consistency should be expected over long periods of time.

We have used this color palette method and scoring procedure to phenotype several other variants of synesthesia, including time-unit-color synesthesia (in which synesthetes have color experiences triggered by days of the week or months of the year) and sound-color synesthesia (in which synesthetes experience colors with music or environmental sounds; Ward et al., 2006).

These tests allow researchers to address several different questions, such as: is an individual’s self-reported synesthesia genuine (as measured by consistent color associations)? Are there different degrees of consistency among synesthetes? Do patterns emerge across participants’ color associations (Day, 2005; Rich et al., 2005; Simner et al., 2006)?

5. A new measure: the speeded congruency test

There still exists the possibility that a person could cheat on the color consistency test by constructing an arbitrary code or ‘cheat-sheet’ that translates graphemes into colors (an undergraduate student of the author’s did this one summer, with great success). Especially since there is no time pressure on the responses, such cheating would be difficult to detect.

To preclude this possibility, we have devised a new kind of test that is administered to participants as soon as they have completed the internal consistency test (above). In this task, participants see a grapheme flashed on the screen for 1 s (Fig. 2a and b). The grapheme is colored: in 50% of trials, the color is congruent with the participant’s reported synesthetic color, in the other 50% of trials the color is incongruent, but is presented in another color from the participant’s self-reported synesthetic palette. On each trial, participants are instructed to report as quickly as they can whether the color of the letter is congruent with their synesthetic color or not. When the color is incongruent, the software chooses a color that is at least a minimum distance in RGB space from the congruent color: this ensures a sufficiently large difference that the participant does not have to worry about colors being indistinguishably close, but should instead be able to easily discriminate a congruent from an incongruent color. The background is automatically adjusted for better viewing of the graphemes (i.e., a white grapheme is displayed against a black background, a vice versa). Timing is clocked in milliseconds by JavaScript code embedded in the HTML that calls the client-side system clock. The system clock does not need to be accurate, because the reaction time is taken as the difference between two successive calls to the clock. With any modern computer, the resolution of this measure is expected to be precise within a few milliseconds.

Fig. 2.

The speeded congruency test. (a) After taking the consistency test (Fig. 1), participants are presented a series of trials in which a letter appears on the screen for 1 s. The letter’s color is either congruent with the synesthetic color chosen in the previous test, or incongruent. Participants answer as quickly as they can whether the color ‘matched’ or ‘did not match’ their synesthetic perception. (b) Screenshot of the test on http://www.synesthete.org. (c) Synesthetes perform at a much higher accuracy rate than controls (left panel), and do so with faster reaction times (right panel). In combination with the color consistency test, this test provides a clear distinction between synesthetes and controls, and therefore a second level of verification for synesthetic genuineness.

Fig. 2c shows that synesthetes (classified by self-report and verified by the consistency test in Fig. 1) score very well on this speeded congruency test, scoring an average of 94% correct responses with an average reaction time of 0.64 ± 0.78 s. By contrast, non-synesthetic control subjects score an average of 67% with an average reaction time of 0.91 ± 0.87 s. Because of the brief display time and speeded reaction, this test can sensitively discriminate synesthetes from non-synesthetes based on percentage correct as well as average reaction time. In our experience thus far, synesthetes have no trouble performing above 90% correct rates, whereas non-synesthetes perform well below that rate.

In combination, these two tests offer a rapid method for confidently phenotyping subjects who have grapheme-color synesthesia.

6. The effect of grapheme contrast on synesthetic colors: separating ‘lower’ from ‘higher’ synesthetes?

One case study has previously reported that lowering the contrast of a digit can change or eliminate the synesthetic color association (Hubbard et al., 2006). We have developed a variation of the grapheme-color test which presents letters and digits to subjects at seven different levels of contrast from high (black grapheme against a white background, or vice versa) to low (a gray grapheme just slightly different from a gray background). Subjects use the color palette described above to choose their synesthetic colors that best match with the presented grapheme. Testing 12 synesthetic subjects, we have found that only one subject reported a difference in her synesthetic associations with different contrasts (Fig. 3). Thus, the claim that contrast affects synesthetic perception is not true of all grapheme → color synesthetes, but appears to apply only to a small fraction of them. These may be the synesthetes that Hubbard and Ramachandran refer to as ‘lower’ (Hubbard and Ramachandran, 2005), the term referring to the color association with low-level details of the letter, rather than ‘higher’ level encoding of letters, such as concepts. Therefore, as part of our battery, we include a test that presents graphemes at seven different levels of contrast and scores the result according to whether the synesthetic colors disappear or fade in intensity. This test immediately suggests to researchers whether the synesthete is sensitive to contrast or not, presumably categorizing them as ‘lower’ or ‘higher’ synesthetes in Hubbard and Ramachandran’s terminology. Note that the contrast effect is not the sole feature of the lower/higher distinction, but just one hypothesised characteristic of it.

Fig. 3.

Do synesthetic associations disappear with low contrast graphemes? (a) Participants are asked in the questionnaire to examine whether their synesthetic experiences change when looking at letters and numbers of varying contrasts. Screenshot from questionnaire shown. (b) If their synesthetic experience is found to depend on contrast, they can be presented with three graphemes at varying contrasts and asked to choose the synesthetic color that is most strongly associated with the grapheme. Along with 16.7 million color choices, they can also select ‘no color’. Squares in the figure represent the contrast at which the grapheme was presented. (b) The data from one synesthete (out of 12 tested) who showed a disappearance of synesthetic colors at low contrast. For the remaining synesthetes tested, synesthetic colors were experienced independent of the contrast in which the grapheme was presented. This test allows rapid discrimination of grapheme → color synesthetes into two categories, labeled by Hubbard and Ramachandran as ‘low’ and ‘high’ synesthetes, where ‘low’ synesthetes are identified by test results like those in panel (b).

7. Other tests in the battery

In variations on this theme, we have developed other online tests, including those that match colors to weekdays and months (Fig. 4). Other tests in development include comparing colored weekdays in English, Spanish and foreign alphabets, and the presentation of moving stimuli with the ability to choose matching synesthetic pitches, for those with motion → sounds synesthesia. The battery has been constructed with a flexible framework for continued development. As new tests are added, researchers can invite their subjects to complete the latest tests being offered. Because of technological limitations there are many types of synesthesia for which we will be unable to construct tests–these include, for example, taste-shape synesthesia (methodology from Cytowic, 2002; Simner et al., in press), taste-color synesthesia (Simner et al., in press), smell-color synesthesia (methodology from Simner et al., in press), and colored orgasms. Additionally, we have already begun the process of translating the front end of the battery into different languages for multi-national use.

Fig. 4.

Weekday and month color tests. These components of the battery assay consistency in (a) weekday → color and (b) month → color synesthesia, both within sessions and upon later testing. As in the grapheme → color test, scores are automatically quantified for comparison through time and across synesthetes. Note that for this subject the month of November was synesthetically close to white; the program automatically displays the result against a black background for easy reading.

8. Password-protected access to test results

On completion of all the applicable tests, a comprehensive results page is automatically generated. These results can be viewed while logged in to the site or can be securely accessed at any later date by the participant or the researcher named by the participant. No one else will have access to the results. Researchers whose subjects have taken the battery have an automatically generated, password-protected personalized page that shows a list of all their subjects, complete with completion dates and options for re-inviting subjects. The emailing system of http://www.synesthete.org sends out automated emails to participants and researchers on registration, completion and later re-invitation to the battery. Aggregate test results will be used by our research group in the future for statistical statements about the synesthetic population (e.g., “of N verified grapheme → color synesthetes, X% of them also have verifiable week day → color synesthesia”), but the names and data of individual subjects will always remain strictly confidential.

9. Discussion

To enhance the rigor and global accessibility of synesthesia testing, we have developed the synesthesia battery, an online collection of tests designed to study and phenotype synesthesia. The battery is free and open to the public. It is built to grow with new discoveries and user input. The battery is intended to standardize and speed synesthesia research, as well as to open up testing to laboratories around the globe.

The battery has been designed to be user friendly and intuitive. Researchers can send potential synesthetes to the site, and when the subjects choose to share their data, the researcher is automatically emailed and can log on to examine the results. The data is not shared with anyone except for the synesthete and the researcher. The tests taken by a subject depend on his/her answers to a series of initial questions. The completion of each test automatically leads the subject to the next applicable test or questionnaire.

Our computerized battery falls short of a full test since synesthesia comes in an enormous variety of cross-sensory forms (colors experienced in response to smells, tastes in response to words, shapes in response to tastes, and so on; Cytowic, 2002; Cytowic and Eagleman, 2006; Ward and Simner, 2003; Ward et al., 2005). However, this battery provides an initial bedrock of tests that can be expanded with new tests in the future because of its modular design. The continued improvement of the battery anticipates the input and requests of the wider synesthesia community. It is hoped that the development of this software will be of benefit in speeding research within and among laboratories.