Neural Networks of Colored Sequence Synesthesia

Abstract

Synesthesia is a condition in which normal stimuli can trigger anomalous associations. In this study, we exploit synesthesia to understand how the synesthetic experience can be explained by subtle changes in network properties. Of the many forms of synesthesia, we focus on colored sequence synesthesia, a form in which colors are associated with overlearned sequences, such as numbers and letters (graphemes). Previous studies have characterized synesthesia using resting-state connectivity or stimulus-driven analyses, but it remains unclear how network properties change as synesthetes move from one condition to another. To address this gap, we used functional MRI in humans to identify grapheme-specific brain regions, thereby constructing a functional “synesthetic” network. We then explored functional connectivity of color and grapheme regions during a synesthesia-inducing fMRI paradigm involving rest, auditory grapheme stimulation, and audiovisual grapheme stimulation. Using Markov networks to represent direct relationships between regions, we found that synesthetes had more connections during rest and auditory conditions. We then expanded the network space to include 90 anatomical regions, revealing that synesthetes tightly cluster in visual regions, whereas controls cluster in parietal and frontal regions. Together, these results suggest that synesthetes have increased connectivity between grapheme and color regions, and that synesthetes use visual regions to a greater extent than controls when presented with dynamic grapheme stimulation. These data suggest that synesthesia is better characterized by studying global network dynamics than by individual properties of a single brain region.

Introduction

Synesthesia is a perceptual condition in which ordinary stimuli (such as a spoken letter) elicit automatic, simultaneous percepts (such as a color; Hubbard and Ramachandran, 2005; Cytowic and Eagleman, 2009). There are many synesthetic associations, ranging from taste–touch (Cytowic, 2002) to sound–color (Baron-Cohen, 1996) to grapheme–color (Weiss et al., 2005; Dixon et al., 2006). The prevalence of synesthesia is estimated between 1 and 4% of the population (Simner et al., 2006; Barnett et al., 2008), depending on which form is examined, and some forms co-occur more commonly than others. To determine how likely forms of synesthesia are to co-occur, we previously performed a factor analysis on 12,234 verified synesthetes phenotyped through the Synesthesia Battery (Eagleman et al., 2007) and found that synesthesia forms cluster into five main groups (Novich et al., 2011). The largest cluster revealed that if a participant had letter–color synesthesia, he was also likely to have number–, weekday–, and month–color synesthesia. We labeled this cluster “colored sequence synesthesia” (CSS; Novich et al., 2011; Tomson et al., 2011) and it is the form we study here.

The neural basis of CSS is not well understood. The general hypothesis is that synesthesia arises from hyperconnectivity between brain regions, and that this connectivity could arise from insufficient pruning of connections during youth (Baron-Cohen, 1996; Spector and Maurer, 2009), or incomplete inhibitory circuitry (Cohen Kadosh and Henik, 2007). Early functional MRI (fMRI) studies found that CS synesthetes show significant activity in traditional color regions while viewing achromatic numbers and letters (graphemes; Nunn et al., 2002; Hubbard et al., 2005; but see Hupé et al., 2011). Subsequent studies suggest many functional and architectural differences between synesthete and control brains (Rouw and Scholte, 2007, 2010; Jäncke et al., 2009; Weiss and Fink, 2009; Hänggi et al., 2011; Jäncke and Langer, 2011; van Leeuwen et al., 2011; Dovern et al., 2012), but none have converged on a unifying mechanism or a single region of interest (ROI).

To understand how the synesthetic brain makes these associations, it is critical to evaluate the brain under multiple conditions, allowing a thorough comparison of how brain states change between rest and the synesthetic experience. In this fMRI study, we combined two rest conditions with two synesthesia-inducing segments of children's television that were rich in graphemes. Further, we investigated the possibility that brain areas beyond color- and grapheme-specific regions would be involved by using clustering analyses to characterize the entire synesthetic neural network.

Materials and Methods

Two experiments are described in this study. Experiment 1 localized grapheme-specific brain regions in synesthetes and controls, which we refer to as the grapheme localizer. These regions were then used as ROIs in Experiment 2. Experiment 2 analyzed functional connectivity during rest and free-viewing grapheme stimulation conditions.

Experiment 1.

The goal of this experiment was to determine which regions of the brain respond significantly to graphemes, and to see whether the location of grapheme activity overlapped between synesthetes and controls. Sixteen colored-sequence synesthetes (26.2 years ± 6.2; 12 females) and 15 controls (25.9 years ± 4.2; 9 females) were recruited using flyers approved by our Institutional Review Board (IRB), online advertisement, and the synesthesia battery (Eagleman et al., 2007). All participants gave written consent before scanning, as stipulated by the local ethics board (IRB) of Baylor College of Medicine, and had normal or corrected-to-normal vision. All synesthetes were required to pass two tests on the synesthesia battery: a consistency test, which measures the color variability for each grapheme, and a speed-congruency test, which measures how quickly a person can identify whether a colored grapheme is correct (congruent with the color marked in the consistency test) or incorrect. All included synesthetes scored <1 on the consistency test and had an accuracy of >90% on the speed-congruency test. Controls were screened for lack of associations between any overlearned sequence (numbers, letters, weekdays, or months) and color.

Stimuli were designed using Psychtoolbox (Brainard, 1997) and presented on a projection screen inside the scanner bore which subjects viewed through a mirror mounted to the head coil. We presented subjects with blocks of graphemes and pseudographemes (Fig. 1). Fifteen graphemes were chosen for each synesthete based on the strength of the color association (i.e., graphemes with bright color associations were chosen over muted colors, and grayscale graphemes were left out entirely). Five numbers and 10 letters were chosen for each participant, and each synesthetic stimulus set was presented to a matched control. Pseudographemes were created by manipulating graphemes in Adobe Photoshop CS5 so that they no longer resembled the parent character, but remained isoluminant. All symbols were presented in white on a neutral gray background, were 6° tall, and were presented in the center of the screen. The two stimulus conditions (graphemes and pseudographemes) were presented in block design, with one block containing 15 symbols shown for 2 s each. Each block lasted for 30 s, and each condition contained 10 blocks. Block order was randomized and interblock interval was jittered between 8 and 12 s to preclude stimulus onset prediction.

Figure 1.

Grapheme localizer results for 31 subjects (16 CSS synesthetes) reveal distinct activity patterns for controls and synesthetes that share the most significant activity in the bilateral posterior inferior temporal gyri. Areas in red indicate results after grouping all 31 subjects together, and are the regions we refer to as the grapheme regions. All results shown are significant at FDR p < 0.05 for the contrast pseudographeme > grapheme. Local maximum MNI coordinates in xyz: synesthetes (47.8 −74.6 −2; −44 −78 −6), controls (41 −71.2 −2; −44 −78 −6), all (41 −71.2 −2; −44 −74.6 −6).

MRI data were acquired using a 3 tesla Siemens Trio scanner and a Siemens 12 channel head coil at Baylor College of Medicine's Human Neuroimaging Laboratory. An anatomical MPRAGE scan was acquired for the purpose of aligning the functional data (T1-weighted, TR = 1200 ms, TE = 2.66 ms, 245 mm FOV, 1 mm slice thickness). The EPI-BOLD sequence (37 slices, TR = 2000 ms, TE = 25 ms, flip angle = 90 degrees, 64 × 64 matrix, 3.4 × 3.4 mm voxel size, 4 mm slice thickness) was acquired after the MPRAGE. Eight-hundred ninety images were acquired for each subject. All data were processed using SPM8 (Wellcome Trust Centre for Neuroimaging, www.fil.ion.ucl.ac.uk/spm/software/spm8). BOLD images were realigned, coregistered, and normalized to the MNI template. Preprocessing included a six parameter rigid body transformation for motion correction, smoothing with an 8 mm full-width half-maximum Gaussian kernel, and normalization to the Montreal Neurological Institute (MNI) atlas. A temporal bandpass filter (0.01–0.08 Hz) was used to remove physiological low-frequency effects.

To understand how grapheme activity patterns differed between groups, the data were analyzed in two ways: first by evaluating the results within each group (just synesthetes or just controls), and then by collapsing across synesthetes and controls (Fig. 1). All results were thresholded to correct for multiple comparisons (FDR < 5%). The following contrasts were performed within each group and across all participants: grapheme > rest, pseudographeme > rest, grapheme > pseudographeme, and pseudographeme > grapheme (Fig. 1).

Experiment 2.

The principle behind functional connectivity analysis is to expose subjects to long time periods of stimulation or rest, extract the BOLD activity throughout that time period for each brain ROI, and quantify the relationship between each region's time series to create a network map. Our stimulation paradigm presented subjects with 3 min of rest (Rest 1), 5.3 min of the audio track of children's television clips (Audio), 7.1 min of children's television clips with audio and the accompanying visual movie (Audiovisual), and finally 3 min of rest (Rest 2) (Fig. 2). The aim of this analysis was to examine whether a special relationship existed between color and grapheme areas of the synesthetic brain during the synesthetic experience. We first identified the “CSS network,” named for the regions that we hypothesized to be involved in the synesthetic percept. Because previous synesthesia studies have found areas beyond grapheme- and color-specific regions to be important in the synesthetic percept, we also analyzed connectivity using an “anatomical network” (90 cortical regions taken from the automatic anatomical labeling (AAL) atlas; Tzourio-Mazoyer et al., 2002) that represent regions throughout the entire brain.

Figure 2.

fMRI paradigm allows subjects to experience numbers and letters (graphemes) in a natural way by showing grapheme-rich clips of children's television. A, Subjects experience 7 min of watching Sesame Street with audio (Audiovisual), 5 min of listening to Sesame Street with eyes closed and no visual stimulus (Audio), and two 3 min periods of eyes-closed rest at the beginning and end of the scan. B, C, A still-frame example of one of the movies in the Audiovisual condition (B) and the accompanying audio narration (C).

Having found regions that respond to graphemes in both synesthetes and controls, we hypothesized that a special relationship existed between grapheme and color regions in the synesthetic brain. To accomplish this task, we used functional connectivity to characterize the correlation between time series in each ROI. Twenty synesthetes (25.1 years ± 6.9; 13 females) and 19 controls (26.2 years ± 5.0; 10 females) were recruited and phenotyped using the same methods described in Experiment 1. Five of the synesthetes from Experiment 1 also participated in Experiment 2. For the purposes of this study, we only recruited synesthetes who associate colors with numbers and letters. If a participant experienced colors for numbers, letters, and music, for example, the individual was excluded from the study to ensure that stimulus-induced color associations only resulted from the grapheme, rather than the sound of the television clip.

Stimuli were designed using Psychopy (http://www.psychopy.org). Experiment 2 began and ended with 3 min of eyes-closed rest (Fig. 2). Subjects were then presented with two synesthesia-inducing conditions containing clips from the children's television show Sesame Street. The first set of clips was presented with only sound for the Audio condition, and subjects were instructed to keep their eyes closed. The audio clips were played for 5.3 min through scanner headphones with only a crosshair appearing in the center of the screen. The 5.3 min of stimulation contained four ∼1 min clips that were concatenated into one continuous presentation. During the Audiovisual condition, subjects watched 7.1 min of clips and heard the accompanying sound through scanner headphones. Subjects were allowed to open their eyes and freely view the clips. Approximately 1000 images were collected for each individual. Image numbers vary due to interstimulus instructions, which subjects were allowed to read at their own pace and button-press upon completion.

We hypothesized that color and grapheme regions would be more connected in synesthetes than controls during the experience of synesthesia. Although we did not believe these regions to be the only regions involved in the synesthetic percept, we began with the simplest hypothesis by studying only regions we knew to be involved in processing individual aspects of the percept. Because no published synesthesia study has found an anatomical difference in the location of color regions, we used published data from a laboratory with expertise in color localization to designate the color regions (Brewer et al., 2005). The traditional color region is V4, but newer studies also implicate two ventral occipital areas, VO1 and VO2, in color processing (Brewer et al., 2005; Arcaro et al., 2009). For this reason, we specified three color regions in each hemisphere (V4, VO1, and VO2), and created discrete regions of interest using a 9 mm diameter around the averaged central voxel (coordinates from Brewer et al., 2005).

Next, many groups have found that the visual word form area (VWFA) responds not only to words, but to individual letters (Cohen et al., 2002; Dehaene et al., 2002; James et al., 2005; Reinke et al., 2008). To this end, we created an a priori CSS network consisting of nine nodes: two grapheme regions (Fig. 1; −44 −74.6 −6, 52 voxels; 41 −71 −2, 160 voxels), six color regions (bilateral V4, VO1, VO2) shown by previous literature to be critical for color processing (Brewer et al., 2005), and the VWFA (Cohen et al., 2002). Color and VWFA ROIs were created using a 9 mm sphere surrounding seed coordinates (Cohen et al., 2002; Brewer et al., 2005).

Our goal was to determine whether the CSS network was more connected in synesthetes than in controls. To accomplish this goal, we quantified the relationship between the BOLD activity in the CSS Network using partial correlation coefficients. We estimated networks at the group level to compare single global metrics as well as at the subject level to compute node level graph metrics.

Two-sample global sparsity analysis.

We inferred a single network for each group by concatenating subject time series of controls and synesthetes. We used the Graphical Lasso algorithm (Friedman et al., 2008) to estimate Markov networks and stability selection (Liu et al., 2010; Meinshausen and Buhlmann, 2010) to determine the optimal edge sparsity or number of edges in the network. A critical benefit of this approach is that it only retains the most stable edges in the network and has been successfully used for functional connectivity analysis (Ryali et al., 2012). This approach results in a sparse network whose remaining edges represent direct connections between nodes. To determine whether the CSS network is significantly more sparse (less dense) in controls, we obtained a null distribution of sparsity values by permuting subject labels and re-estimating the group networks for each permutation (Fig. 3). Using two-sample permutation tests (Good, 2011), we compared the sparsity of each group to the permutation null distribution (see Fig. 5).

Figure 3.

Permutation testing reveals significantly more edges in the CSS Network of synesthetes during Rest1 (A) and Audio (B) conditions than in controls. Group labels were permuted 100 times (left) and the difference in edge number between permuted groups was compared with the difference between controls and synesthetes (right). Dotted lines represent the actual number of edges in the CSS network for controls (yellow) and synesthetes (pink).

Figure 5.

CSS network of six color regions (purple) and three grapheme regions (gray). In two conditions, Rest1 and Audio, synesthetes have significantly more edges than controls (p < 0.032 and p < 0.026)). Specifically, synesthetes have significantly more edges connecting grapheme to color regions in the Audio condition (p < 0.024). Total number of edges is shown in the center of the networks.

Two-sample node-level modularity analysis.

In the second part of our analysis, we expanded the network to include ninety anatomical regions from the cortical map of the anatomical AAL atlas (Tzourio-Mazoyer et al., 2002). The previous nine regions from the CSS network, combined with the 90 regions from AAL, resulted in a network of 99 regions that we refer to as the anatomical network for distinction (see Fig. 6). We again performed two sample global sparsity analysis and two sample modularity coefficient analysis of the Anatomical Network.

Figure 6.

Synesthetes and controls have significantly different clustering patterns during Audio presentation of graphemes. A, B, Controls (A) cluster parietal and frontal regions, whereas synesthetes (B) cluster primarily visual and fusiform regions. C, The right grapheme region (large green node) in synesthetes clusters more often with V4, VO1, lingual, and occipital cortex than controls. D, E, Similar patterns persist during the Audiovisual condition for synesthetes (D) and controls (E). Green nodes have significantly different clustering patterns between groups, and blue nodes are relatively consistent across groups. Edges are significant at the group level.

We computed graphs for each subject separately, using the same techniques used for the sparsity analysis. We then resampled each subject's data 100 times, re-estimated the graph using established sparsity levels, and applied Newman's modularity algorithm (Newman, 2006) to cluster all nodes into modules based on their connectivity patterns. A module in a network is a group of nodes that are more connected to one another than they are to other nodes in the network. Current clustering algorithms do not inherently provide unique and consistent partitions. To find biologically meaningful and reproducible distinctions in the modularity assignments of the nodes, we used a cluster stability measurement (Dudoit and Fridlyand, 2003) to score the strength of the module allegiance of each node. Rather than assume that each node belonged to only one module based on a single estimate, we perturbed the data by resampling it, then re-estimated and reclustered the networks, and measured the frequency with which every node fell into each module. We subsequently asked which other nodes co-occurred in the same module as a node of interest (NOI). In this analysis, three possibilities exist for each NOI: (1) the NOI clusters with the same regions in synesthetes and controls, yielding an insignificant result, (2) the NOI clusters significantly more often with  nodes in synesthetes, and (3) the NOI clusters significantly more often with B̄ nodes in controls. For this reason, synesthetes and controls can have varying numbers of frequently occurring nodes in a cluster. This provides a data-driven way to find nodes that follow similar activity patterns, potentially suggesting new regions of interest for future focus.

We aggregated the co-occurrence of each node with respect to the NOI across all resampled estimates to create the co-occurrence frequency (Fig. 4). We tested for differences in this co-occurrence frequency using two-sample score statistics using a permutation null distribution. Before testing for statistical significance, we only included those nodes that clustered with the NOI >50% of the time. Therefore the number of hypotheses tested per NOI is often <98, the maximum number of hypotheses per NOI.

Figure 4.

Illustration of the modularity analysis procedure through a toy example. A, After resampling the data, the network is estimated and clustered. B, Node 4 co-occurs with node 2. C, D, The co-occurrence frequencies are aggregated >100 resamples of the data for each subject; an example for a synesthete (C) and control (D) are shown. These subject-specific co-occurrence frequencies are the basis of two-group inference between synesthetes and controls.

We looked for differences in module-affiliation in the anatomical network with respect to a subset of 15 NOIs: nine regions in the CSS network (Fig. 5), bilateral superior parietal lobes (Weiss et al., 2005; Jäncke et al., 2009; Rouw and Scholte, 2010), bilateral lingual gyri (Nunn et al., 2002; Jäncke et al., 2009), and bilateral middle frontal gyri (Rouw and Scholte, 2007, 2010; Table 1). This analysis can be thought of as similar to a seed-based approach. For each of our 15 “seed” NOIs, we asked which of the remaining 98 brain regions clustered significantly more often with the NOI in one group than another (Fig. 4; Table 1). We used a multiple testing threshold based on the number of hypotheses tested per NOI and reported adjusted p values. We controlled for multiple testing errors using the false discovery rate (FDR) procedure (Benjamini and Hochberg, 1995; Storey, 2002). We also performed a data-driven analysis for all possible NOIs in the complete anatomical network, subjected to increasingly stringent correction for multiple comparisons (Table 2). For the data-driven analysis, we adjusted p values based on the total number of hypotheses tested. The p values reported are FDR adjusted (Wright, 1992) at 10%.

Table 1.

Clustering results from a priori nodes reveal that synesthetes cluster visual regions significantly more than controls during both the Audio and the Audiovisual conditions

NOI	Clusters with:	p value	Group
Audio
Middle frontal gyrus (l)	Precuneus (l)	0.001	C
	Precuneus (r)	0.001	C
	Middle occipital (l)	0.023	C
	Sup. parietal lobule (l)	0.041	C
	Calcarine sulcus (l)	0.048	C
	Inf. orbitofrontal gyrus (r)	0.064	S
	Visual word form area	0.096	C
Middle frontal gyrus (r)	Precuneus (r)	0.005	C
	Precuneus (l)	0.011	C
	Supp. motor area (r)	0.096	C
VO2 (l)	Precuneus (l)	0.026	S
	Precuneus (r)	0.026	S
Grapheme (r)	Inferior occipital (l)	0.004	S
	Sup. occipital (r)	0.029	S
	V4 (l)	0.029	S
	Cuneus (r)	0.029	S
	Sup. occipital (l)	0.036	S
	Middle occipital (l)	0.036	S
	Lingual gyrus (r)	0.043	S
	Middle occipital (r)	0.043	S
	Precuneus (r)	0.05	S
	Cuneus (l)	0.05	S
	VO1 (r)	0.05	S
	V4 (r)	0.051	S
	Grapheme (l)	0.055	S
	Calcarine sulcus (r)	0.055	S
	Lingual gyrus (l)	0.055	S
	VO2 (r)	0.064	S
	Precentral (r)	0.07	C
	VO1 (l)	0.095	S
Audiovisual
Lingual gyrus (r)	Visual word form area	0.015	S
	Middle occipital (l)	0.015	S
	Inf. occipital (l)	0.015	S
	Inf. temporal (r)	0.015	S
	Heschl's gyrus (l)	0.015	C
	Middle occipital (r)	0.018	S
	Sup. occipital (r)	0.018	S
	Sup. occipital (l)	0.024	S
	Inf. occipital (r)	0.025	S
	V4 (r)	0.041	S
	Fusiform gyrus (r)	0.045	S
	VO1 (r)	0.065	S
	Fusiform gyrus (l)	0.076	S
Sup. parietal lobule (l)	Angular (r)	0.057	C
Grapheme (l)	Lingual gyrus (r)	0.048	S
	VO2 (r)	0.048	S
Grapheme (r)	Lingual gyrus (r)	0.057	S
	VO2 (r)	0.08	S
	Calcarine sulcus (r)	0.08	S

Group label indicates whether the result is significant for synesthetes (S) or controls (C). All results shown are corrected for multiple testing within each node of interest (NOI). Only significant results are shown (p < 0.1, adjusted for false discovery rate at 10% using Benjamini-Hochberg).

Table 2.

Clustering results from the complete Anatomical network

NOI	Clusters with:	p value	Group
Audio
Precuneus (r)	Inferior parietal (l)	0.058	C
	Middle frontal gyrus (l)	0.080	C
	Angular (r)	0.080	C
Precuneus (l)	Inferior parietal (l)	0.080	C
	Middle frontal gyrus (l)	0.080	C
	Angular (r)	0.080	C
V4 (l)	Angular (l)	0.092	C
Audiovisual
Rolandic operculum (l)	Lingual (r)	0.097	C

Group label indicates whether the result is significant for synesthetes (S) or controls (C). All results shown are corrected for multiple testing within each node of interest (NOI). Only significant results are shown (p < 0.1, adjusted for false discovery rate at 10% using Benjamini-Hochberg).

Two-sample node-level graph theoretic analysis.

For the same individual subject networks we also computed node degree, betweenness centrality, and local efficiency and conducted two-sample permutation t tests to test for differences. We adjusted p values at 10% FDR.

Results

Experiment 1

The goal of this study was to characterize the neural networks involved in synesthesia. We hypothesized that grapheme- and color-specific brain regions would play an important role in the network profile. Although previous groups have suggested that graphemes are processed in lateral posterior inferior temporal (LPIT) gyri/Brodmanns area (BA) 37 (Puce et al., 1996; Garrett et al., 2000; Dehaene et al., 2002; Mechelli et al., 2003; Flowers et al., 2004), we questioned whether synesthetes processed graphemes in a different brain area than controls. Our first step, therefore, was to find regions in the brain that processed graphemes.

Graphemes and pseudographemes compared with fixation revealed significant activity for both groups in bilateral LPIT/BA 37, supporting previous studies that these regions are involved in processing symbolic characters (Puce et al., 1996; Garrett et al., 2000; Mechelli et al., 2003). We found no significant difference between controls and synesthetes when contrasting graphemes or pseudographemes with rest; both showed the same location and degree of activity in LPIT/BA 37. As expected from stimulus novelty, pseudographemes increased activity relative to both graphemes and fixation in LPIT/BA 37 in the collapsed analysis. Contrasting pseudographemes over graphemes, we found distinct activity patterns for synesthetes and controls separately (Fig. 1; Dehaene et al., 2002; Mechelli et al., 2003; Flowers et al., 2004). Because LPIT/BA37 contained the most significant voxels in all three group contrasts (controls, synesthetes, and collapsed), we chose these as our “grapheme regions” (Fig. 1, red) for ROI consistency in the network analysis.

Experiment 2

Edge sparisty

In the anatomical network, there was no significant difference between the total number of edges (sparsity) in control and synesthete graphs (p = 0.86; data not shown), suggesting no significant difference in the global density of edges between groups. Within the CSS network, however, synesthetes had significantly more edges than controls for the first Rest (p < 0.026) and the Audio condition (p < 0.031; Figs. 3, 5). To explore the edge increase during Rest, we examined the distribution of edges connecting all six color regions (Fig. 5, purple nodes) to all three grapheme regions (Fig. 5, gray nodes). In the Audio condition, synesthetes had significantly more edges specifically connecting grapheme and color nodes than controls (p < 0.024). In the second Rest condition, however, synesthetes and controls had equal distribution of edges connecting the nine nodes. Together, these results suggest that the CSS network in synesthetes is more connected than in controls while listening to spoken graphemes (Audio condition) and during the first Rest. More specifically, the pattern of synesthetic connectivity shifts from an overall greater density of connections in Rest to a greater density of connections between color and grapheme regions in Audio. This suggests that the CSS network reorganizes to facilitate or produce the synesthetic association.

Modularity

There was no significant difference between the modularity coefficient (Q) across groups, meaning that controls and synesthetes organize networks into the same number of modules. Although the overall structure appeared the same, it was unclear whether the identity of the modules was also identical. Rather than assume that each node belonged to only one module, we further evaluated the frequency with which a particular NOI fell into each module (clustering). The 15 a priori nodes revealed remarkable clustering differences between synesthetes and controls in the Audio and Audiovisual conditions, whereas no significant differences emerged during either Rest condition. At a glance, one can see that synesthetes cluster visual regions more tightly than controls (Fig. 6A,B,D,E). Specifically, the right grapheme region clusters with the color region V4, the lingual gyri, and several other visual regions significantly more often in synesthetes than in controls (Fig. 6C; Table 1) during the auditory synesthetic experience. Controls show significant clustering among the precuneus, frontal, and parietal regions; a trend that is confirmed by the Anatomical network (Table 2). The complete Anatomical dataset reveals that synesthetes had significantly less clustering than controls between bilateral precuneus and parietal, frontal, and angular regions (Table 2). Overall, these data suggest that synesthetes and controls process dynamic graphemes using strikingly different networks of regions, pointing to many new regions of interest outside of the CSS network.

Beyond modularity, we also evaluated the network in terms of three other graph theory metrics: degree, betweenness centrality, and local efficiency; these did not yield any significant differences between synesthetes and controls (data not shown).

Discussion

The goal of this study was to understand the neural representation of graphemes as experienced by CSS. To accomplish the task, we performed two experiments to (1) find grapheme regions in controls and synesthetes and (2) study the neural network properties of grapheme and other regions during varying stimulation conditions.

Our first important finding was in the grapheme localizer. Although we found the most significant activity in the LPIT/BA 37 for both groups (Fig. 1), we were intrigued by the substantial differences in other areas. The lack of parietal activity in synesthetes compared with controls was at first surprising, given the plethora of studies implicating parietal lobes in the processing of symbolic characters (Menon et al., 2000; Gruber et al., 2001; Van Opstal et al., 2009; Santens et al., 2010) and in synesthesia (Rouw and Scholte, 2010; Rouw et al., 2011; Specht and Laeng, 2011). After visualizing the clustering results between parietal and frontal regions in controls (Fig. 6), and subsequent absence of any significant parietal clustering in synesthetes during dynamic grapheme presentations, it is possible that the parietal lobes are not as important to the synesthetic experience as previously thought. [That said, note that parietal regions have been found by previous studies in nonsynesthetes to respond to graphemes (Garrett et al., 2000; Mechelli et al., 2003; James et al., 2005), pseudographemes (Puce et al., 1996; Garrett et al., 2000; van Leeuwen et al., 2010), faces (Haxby et al., 1991; Sergent and Signoret, 1992), and objects (Sergent and Signoret, 1992; Moore and Price, 1999)]. The distinct patterns we find for each group suggest that synesthetes employ different networks when encountering novel symbols. By probing with the correct paradigms, it is hoped that these activity differences can lead to an understanding of how synesthetes initially develop grapheme–color associations.

In the second experiment, we aimed to understand the relationship between 99 brain regions, including the CSS network, during varying dynamic conditions of rest, auditory graphemes, and audiovisual graphemes. In accordance with Hänggi et al. (2011) we found no significant difference between the sparsity (total number of edges) of the anatomical network between groups in any condition. Given that both groups are otherwise healthy individuals, we did not expect to find significant differences in fundamental properties of the network such as sparsity. When we focused on just the CSS network, however, we found a significant increase in edge number in synesthetes (Fig. 5). The distribution of edges changed from an overall increased edge number (in Rest) to an increased edge number specifically between the grapheme and color regions (in Audio). This suggests that the onset of spoken graphemes is triggering a network reorganization in synesthetes that is not occurring in controls. This result is further supported by clustering analyses during the Audio condition, which reveal that synesthetes cluster the right grapheme region with the V4 color regions significantly more often than controls (Fig. 6C). Together, these results support the hypothesis of a special relationship between grapheme and color regions that is critical to the synesthetic experience.

A general theme we noticed was that the Audio condition consistently yielded the most significant differences between synesthetes and controls. Although all synesthetes reported the experience of color for both spoken and written graphemes, the synesthetic percept is sometimes reported to be more vivid when a synesthete hears the triggering grapheme than when it is visually presented to her (Cytowic and Eagleman, 2009). When a synesthete sees a printed “T” on screen, it might be akin to seeing a blank outline of a banana; the viewer knows it is yellow but does not need to visualize the color to know that. When the letter is spoken, however, a synesthete is free to visualize the “T” in her own imagination. Based on our present data, we suggest that the synesthetic mechanism changes according to the mode of stimulation chosen by the experimenter.

Going beyond the CSS network, we used a clustering approach to extract significant novel relationships hidden in the anatomical network. To do this, we characterized nodal associations between 15 a priori nodes according to how often clusters of nodes fell into modules together. The 15 a priori nodes included the nine CSS network nodes in addition to six nodes implicated in previous synesthesia studies. We then asked: (1) which nodes clustered with the NOI, and (2) was there was a significant difference between the identity of these clustered nodes in synesthetes and controls? We found an unexpected clustering pattern for synesthetes during the Audio and Audiovisual conditions (Fig. 6), suggesting that synesthetes actually use more visual regions than controls during the synesthetic experience. Controls revealed their own unique clustering pattern, uniting frontal and parietal regions significantly more often than synesthetes. This method stops short of proving that these regions are coactivated, but it does suggest that the biggest difference between synesthetes and controls during stimulation is that synesthetes cluster in local visual cortices, whereas controls cluster in more distant cortical space. This result should be interpreted with appropriate caution, as two grapheme nodes were identified using a functional localizer specific to these populations, whereas the remaining 13 nodes were adapted from the synesthesia literature. Nevertheless, this might suggest that synesthetes are wired to process sequences in a more visual manner, “seeing” the grapheme in the mind's eye. Future studies might further explore the visual cortex in synesthetes during imagination or memory of readily visualized objects.

Finally, we evaluated the network in terms of three graph theory metrics: degree, betweenness centrality, and local efficiency. Many studies have found these metrics useful in distinguishing disease from healthy populations (Stam et al., 2007; Liu et al., 2008; Lynall et al., 2010; Ersche et al., 2012); however, we were unable to find a significant difference between synesthetes and controls in any condition (data not shown). Our findings are in line with a recent EEG connectivity study by Jäncke and Langer (2011) which evaluated several graph theory metrics (edge number, clustering coefficient, and path length) during eyes-closed rest and found no significant difference between sound–color synesthetes and controls. A recent structural study reported trends toward increased average degree, decreased betweenness centrality, and decreased modularity coefficient (Q) in synesthetes, but these results were lost at the group level (Hänggi et al., 2011). As the only two studies to have evaluated graph theory in synesthesia using EEG and structural data (respectively) across several frequency bands and partition schemes, it is possible that graph theoretic measures are simply too crude to reproducibly characterize the synesthetic experience. Our results clearly suggest that the dynamic between small groups of nodes is critical to understanding the neural mechanism of synesthesia. By contrast, graph theory metrics characterize a node as an individual (i.e., how many edges does node x have?) rather than as a group (how many edges exist between these 10 nodes?). Perhaps graph theory is better suited for comparing dramatically different brain states in which the structure and function of an individual node might very well be impaired. Synesthetes, however, are healthy individuals who are otherwise indistinguishable from controls.

We present the first study to compare functional neural networks under varied dynamic conditions across two groups of healthy individuals: controls and synesthetes. The greatest challenge in this type of study is linking the subtle differences in perception to a significant difference between two groups of otherwise healthy brains. We have made a substantial step in this direction by combining rigorous phenotyping, traditional fMRI localizer methods, and modern functional connectivity metrics to converge on a detailed portrait of the synesthetic brain. Our findings reveal that hyperconnectivity between color and grapheme regions is a defining characteristic of the synesthetic experience. Further, synesthetes preferentially cluster visual regions during grapheme stimulation, whereas controls cluster frontal and parietal regions. The relationship between these new local networks demands further study and potentially suggests a novel way to characterize subtle functional properties of a neural network.