Neural signatures of lexical tone reading

Abstract

Research on how lexical tone is neuroanatomically represented in the human brain is central to our understanding of cortical regions subserving language. Past studies have exclusively focused on tone perception of the spoken language, and little is known as to the lexical tone processing in reading visual words and its associated brain mechanisms. In this study, we performed two experiments to identify neural substrates in Chinese tone reading. First, we used a tone judgment paradigm to investigate tone processing of visually presented Chinese characters. We found that, relative to baseline, tone perception of printed Chinese characters were mediated by strong brain activation in bilateral frontal regions, left inferior parietal lobule, left posterior middle/medial temporal gyrus, left inferior temporal region, bilateral visual systems, and cerebellum. Surprisingly, no activation was found in superior temporal regions, brain sites well known for speech tone processing. In activation likelihood estimation (ALE) meta‐analysis to combine results of relevant published studies, we attempted to elucidate whether the left temporal cortex activities identified in Experiment one is consistent with those found in previous studies of auditory lexical tone perception. ALE results showed that only the left superior temporal gyrus and putamen were critical in auditory lexical tone processing. These findings suggest that activation in the superior temporal cortex associated with lexical tone perception is modality‐dependent. Hum Brain Mapp, 36:304–312, 2015. © 2014 Wiley Periodicals, Inc.

INTRODUCTION

In a tonal language, pitch (i.e., tone) is used to distinguish lexical or grammatical meanings. As tonal languages account for more than 70% of world languages [Yip, 2002], research on how lexical tone is neuroanatomically represented is central to our understanding of cortical regions subserving language. Previous neuroimaging studies have yielded important findings on the brain mechanisms dedicated to the processing of lexical tone in spoken language. For example, activations in bilateral superior temporal gyrus (STG) and left anterior insula cortex have been demonstrated during auditory perception of Mandarin pitch patterns [Klein et al., 2001; Wong et al., 2004; Xi et al., 2010]. When native speakers of Thai and Mandarin performed tone discrimination tasks on Chinese words or resynthesized stimuli by superimposing Thai tones onto Chinese syllables, both groups exhibited stronger activity in the left planum temporale (PT) in response to native than non‐native tones, revealing the role of PT in perceiving lexical tones [Xu et al., 2006]. In a word naming (production) task, Liu et al. [2006] compared the production patterns of Chinese tones and vowels in an adaptation functional magnetic resonance imaging (fMRI) paradigm, and they found that lexical tone production elicited stronger activation in left middle temporal gyrus relative to vowel production. Luo et al. [2006] used a passive oddball paradigm to measure early auditory processing of Chinese lexical tones and consonants in Mandarin natives, demonstrating that preattentive perception of Mandarin tones evoked significant brain activation in the STG bilaterally, with stronger activity in right hemisphere. In a brain structure study, Crinion et al. [2009] identified greater cortical volume in the left insula and right anterior temporal lobe in Chinese speakers when compared to those who do not speak Chinese, implicating the role of temporal lobe in processing linguistic pitch. Using a different approach, neuroimaging studies used an auditory paradigm to train native English speakers to acquire Mandarin tones, revealing that the learning of Mandarin tone classification was associated with increased brain activity in left STG and posterior transverse temporal area (BA 42) [Wang et al., 2003]. When Americans were trained to match pictures with monosyllabic pseudowords that were composed of Mandarin tones superimposed onto English syllables, less successful learners exhibited a smaller gray matter volume in the left Heschl's gyrus compared to successful learners, indicating that the primary auditory regions are involved in linguistic pitch processing [Wong et al., 2008].

Prior research has also indicated the contribution of the frontal cortex in speech tone processing. Activity in the left inferior frontal gyrus is assumed to be associated with tone perception experience in a number of brain mapping experiments by Gandour et al. [1998, 2000, 2002, 2003] and Hsieh et al. [2001]. In a phonological recognition task, direct contrasts of Mandarin tones relative to consonants and rhymes identified significantly stronger brain activation in right fronto‐parietal regions [Li et al., 2010]. Right‐lateralized cortical activations in posterior inferior frontal gyrus were also found in native Mandarin subjects during the production of Chinese lexical tone sequences [Chang et al., 2014].

Although we have gained an in‐depth understanding of the neural basis for auditory tone perception, little is known as to whether these previously identified neural correlates are involved in the visual modality, that is, when lexical tones are read visually. For example, in written Chinese, printed characters map onto monosyllables with tonal information, and thus, how the brain reads tones is fundamental to reading comprehension. Here, we report fMRI results with regard to the neural substrates for lexical tone processing in Chinese reading. We used a simple tone reading task in which Mandarin natives made explicit Mandarin tone discrimination for every visually presented characters. During the scan, subjects were asked to respond by making button presses as quickly and accurately as possible. If the neural basis for lexical tone is modality‐general, we expect to observe brain activations in similar regions in the frontal areas and the temporal cortex that mediate audition of speech tone as reviewed above.

EXPERIMENT 1

Materials and Methods

Participants

We scanned eighteen native Mandarin speaking adults (9 males and 9 females; average age = 21 years, SD = 1.18 years) from Beijing Capital Normal University with normal or correct‐to‐normal vision. They were free from neurological and psychiatric diseases. All participants were strongly right‐handed. They gave informed consents before the study was performed and were paid for participation. One participant was excluded from the analysis due to task incompletion. The study was approved by the Institutional Review Board of Beijing MRI Center for Brain Research.

Materials and Procedures

A total of 96 characters were used in the present experiment. Forty‐eight characters had low number of strokes ranging from 8 to 10 (mean = 9.375, SD = 0.733) (“visually simple characters”) and another 48 characters had high number of strokes ranging from 12 to 17 (mean = 14.313, SD =2.033) (“visually complex characters”). All of the selected characters consisted of three components and were of high frequency of occurrences (Fig. 1). Forty‐eight out of 96 characters carried the falling tone (tone 4), and the other half carried a nonfalling tone (i.e., high level, high rising, or low dipping).

Figure 1.

Examples of the Chinese characters used in the present study. A: Visually simple Chinese single‐character with an up‐down configuration with low stroke numbers. The sample character is pronounced /ma/ with the forth tone; B: Visually complex Chinese single‐character with left‐right configuration with high‐stroke numbers. The sample character is pronounced /zui/ with the third tone.

The experimental design consisted of a tone judgment task (on visually complex and visually simple characters, respectively) and an arrow direction judgment task as baseline. In the tone judgment task, participants were required to judge whether or not a visually exposed character was pronounced as the falling tone (i.e., tone 4). In each trial, a character was exposed in the centre of the screen for 1,000 ms, followed by a 1,000 ms blank screen display. Participants responded by pressing buttons with their right hand to indicate whether the exposed characters carried the forth tone. They were asked to perform the task as accurately and quickly as possible. The arrow direction judgment task involved a 1,000 ms arrow display and a 1,000 ms blank screen in each trial, and the arrows either pointed upward or downward. Participants were required to distinguish the direction of the displayed arrow by pressing the corresponding key with their right index finger and thumb. Task instruction was presented for 2 s before each block. Prior to scanning, all participants were given practice trials to familiarize with task procedures and to make button presses with one hand.

The experiment was conducted with one single run. It began with a 6‐s fixation crosshair, followed by a 2‐s task instruction and a block of six baseline trials. Then the tone judgment tasks (24 s) with task instructions (2 s) were alternated with baseline blocks. The experiment contained four blocks for visually simple characters, four blocks for visually complex characters, and eight blocks of the control task. In each experimental block, 12 characters were visually presented in succession.

Image acquisition

Whole‐brain fMRI data was acquired on a 3T Siemens scanner at Beijing 306 hospital using a T2*‐weighted‐single‐shot gradient echo‐planar images sequence (TE = 30 ms, TR = 2 s, flip angle = 90°, field of view = 21 cm, slice thickness = 4 mm, and image matrix = 64 × 64). Thirty axial slices were collected. Visual stimuli were presented through a projector onto a translucent screen. Participants were immobilized to avoid head movement during the scan.

Image Analysis

The fMRI data analysis was performed with MATLAB software (Version 7.10; Mathworks, Natick, MA) and SPM8 (Wellcome Department of Cognitive Neurology University College London, London). After data conversion and reconstruction, participants' data were preprocessed in batch mode one by one. The first three volumes of each participant's scan were discarded, and the remaining functional images were slice‐time corrected (in ascending order, with reference slice 16), realigned and unwarped. The data were then spatially normalized to Montreal Neurological Institute (MNI) stereotaxic template, resampled into 2 × 2 × 2 mm cubic voxels and spatially smoothed with a full‐width half‐maximum (FWHM) isotropic Gaussian kernel of 8 mm. Participants' data were high‐pass filtered at 128 s. Initial analysis was conducted on an individual subject basis, and the activation t‐map for each subject was generated using the general linear model. Group analysis was done by obtaining contrast images at second‐level random‐effects model. Activation patterns were evaluated by the tone decision on complex characters > baseline and the tone decision on simple characters > baseline contrasts with a one‐sample t‐test (P < 0.05 false discovery rate (FDR‐) corrected; extent threshold = 10), respectively. Then the tone judgment on all characters > baseline contrast images were computed using a one‐sample t‐test (P < 0.05 FDR‐corrected; extent threshold = 10). Another one‐sample t‐test was performed (P < 0.001 uncorrected; extent threshold = 10) to examine the difference in brain activation between high‐stroke and low‐stroke character conditions.

Results and Discussion

Behavioral Performance

Participants were highly accurate in all conditions. It took 477 ms (SD = 52) to perform the arrow direction judgment, with an accuracy of 99.8% (SD = 4.5). Mean accuracy was 90% (SD = 0.07) for tone judgment of visually complex characters and 93% (SD = 0.05) for tone judgment for visually simple characters. Paired t‐tests indicated higher accuracy for simpler characters than more complex characters [t(16)=−2.76, P = 0.014]. After excluding trials where the participants pressed the wrong key, the average reaction time was 827 ms (SD = 129) for the high‐stroke condition and 790 ms (SD = 101) for the low‐stroke condition and the difference was significant [t(16) = 3.11, P = 0.007].

Imaging Results

Our experiment aimed to examine the neural correlates mediating lexical tone perception in reading visual characters. Images of group maps (17 subjects) for character tone reading relative to arrow direction judgment baseline are shown in Figure 2. Significant brain activations are listed in Table 1, encompassing brain regions in frontal lobes, parietal lobes, temporal lobes, and the visual systems bilaterally. Chinese tone judgment yielded greater activity in bilateral insula and visual systems. Activation in the inferior frontal gyrus (BA 45; x = −43, y = 31, z = 15; BA46; x = 27, y = 30, z = 19) and the middle frontal gyrus (BA9; x = −43, y = 8, z = 32; BA10; x = −25, y = −1, z = 56; BA46; x = −45, y = 30, z = 26; x = 34, y = 35, z = 23) occurred bilaterally but was much stronger in the left hemisphere. The left medial frontal gyrus (BA 6; x = −4, y = 8, z = 49; x = −15, y = −11, z = 64), left superior frontal gyrus (BA 6; x = −11, y = 3, z = 63; x = −24, y = −1, z = 58), left inferior parietal lobule (BA 40; x = −43, y = −40, z = 38), left inferior temporal gyrus (BA37; x = −36, y = −58, z = −9; BA19; x = −29, y = −67, z = 0), left middle/medial temporal gyrus (BA19; x = −24, y = −67, z = 21), right fusiform gyrus (BA 18; x = 29, y = −78, z = −13), and cerebellum also contributed to lexical tone discrimination. Subcortical activations were observed in hippocampus, caudate nucleus and putamen.

Figure 2.

Brain regions with significant activity in tone reading of all characters. A: Lateral view; B: Axial sections. The significant threshold is P < 0.05 FDR‐corrected. The functional maps (in color) are overlaid on the corresponding T1 images (in gray scale). Planes are axial sections, labeled with the height (mm) relative to the bicommisural line. L = the left hemisphere and R = the right hemisphere.

Table 1.

Coordinates of activation peaks

Regions activated	BA	Z score	x	y	z
Tone reading of all characters—baseline
Occipital
L lingual gyrus	18	6.29	−15	−95	−3
R lingual gyrus	17	5.75	17	−91	−12
R precuneus	19	4.69	7	−65	35
R calcarine fissure	18	6.24	20	−93	0
R fusiform gyrus	18	5.6	29	−79	−13
L superior occipital gyrus	19	5.78	−20	−63	33
Frontal
L inferior frontal gyrus	45	5.56	−43	32	15
R inferior frontal gyrus	46	3.05	27	30	19
L middle frontal gyrus	9	4.47	−43	8	32
	6	4.45	−25	−1	56
	46	4.07	−45	30	26
	6	3.04	−40	3	49
R middle frontal gyrus	46	2.85	34	35	23
	6	2.56	26	4	51
	6	2.54	33	10	53
L medial frontal gyrus	6	5.29	−4	8	49
	6	3.92	−15	−11	64
L superior frontal gyrus	6	5.26	−11	3	63
	6	4.67	−24	−1	58
L insula	–	9.3	−29	20	13
	–	5.68	−32	16	2
R insula	–	3.65	29	12	2
	–	3.53	34	14	0
Parietal
L inferior parietal lobule	40	4.83	−43	−40	38
Temporal
L inferior temporal gyrus	37	8.88	−36	−58	−9
	37	4.24	−29	−67	0
L middle/medial temporal gyrus	19	4.17	−24	−67	21
Other areas
L hippocampus	–	8.75	−31	−34	−4
	–	8.07	−27	−38	1
R hippocampus	–	7.63	27	−40	3
L cerebellum	–	6.22	−29	−83	−20
	–	5.68	−17	−87	−19
R cerebellum	–	4.84	−17	−87	−19
L putamen	–	3.53	−22	1	5
	–	3.34	−25	2	2
R putamen	–	4.07	26	12	−1
	–		20	12	0
L caudate nucleus	–	3.91	−17	−29	14
	–	3.63	−13	−5	19
	–	3.36	−11	18	9
R caudate nucleus	–	5.23	15	−13	22
	–	3.51	13	−3	19
	–	3.38	12	18	9

Stereotaxic coordinates (mm) are derived from the human brain atlas of Talairach and Tournoux [1988] and refer to the peak Z scores for each region (P < 0.05 FDR corrected).

The aforementioned patterns of activation are confirmed both in our analysis of the tone judgments of more complex characters contrasted with the baseline, and in our analysis of the tone judgment of simpler characters contrasted with the baseline (Fig. 3A,B).

Figure 3.

Functional maps: averaged brain activation evoked by tone judgment on Chinese characters. A: Tone reading of visually complex characters relative to baseline; B: Tone reading of visually simple characters relative to baseline. The significant thresholds are P < 0.05 FDR‐corrected for both comparisons. All functional maps (in color) are overlaid on the corresponding T1images (in gray scale). Planes are axial sections, labeled with the height (mm) relative to the bicommisural line. L = left hemisphere and R = right hemisphere.

To determine whether there is a significant difference between the lexical tone processing of visually complex and visually simpler characters, we subtracted brain activation evoked by tone reading of simpler characters from tone reading of more complex characters (P < 0.001 uncorrected). We did not find any significant difference, indicating that brain activations provoked by the orthography‐to‐tone mapping remained stable, though the number of strokes the characters possessed varied.

To further investigate whether or not the activations in the temporal cortex revealed in the above, specifically, the left inferior temporal gyrus (BA37; x = −36, y = −58, z = −9) and the left posterior middle/medial temporal gyrus (BA19; x = −24, y = −67, z = 21), were consistent with those found in previous studies of auditory lexical tone perception, we performed a meta‐analysis to combine results across published experiments using activation likelihood estimation (ALE).

EXPERIMENT 2

Materials and Methods

Literature Selection

There were 14 neuroimaging studies of processing of lexical tones with the spoken language. Among these studies, 11 used an auditory lexical tone perception task [Gandour et al., 1998, 2000, 2002, 2003; Hsieh et al., 2001; Klein et al., 2001; Li et al., 2003, 2010; Wang et al., 2003; Wong et al., 2004; Xu et al., 2006], one used Mandarin tone and vowel production task [Liu et al., 2006], and two measured and contrasted subjects' structural brain volume [Crinion et al., 2009; Wong et al., 2008]. Three selection criteria were included: (1) subjects involved normal and healthy subjects; (2) lexical tone related tasks were used in the studies; (3) significant cortical activation was found in the temporal cortex of the left hemisphere.

According to the mentioned criteria, a set of eight studies with Mandarin and Thai tones was selected for the analysis: four used an explicit tone perception task [Gandour et al., 2003; Klein et al., 2001; Wong et al., 2004; Xu et al., 2006], one used Mandarin tone and vowel production [Liu et al., 2006], two used Mandarin tones training [Wang et al., 2003; Wong et al., 2008], and one measured subjects' structural brain differences [Crinion et al., 2009]. Wong et al. [2008] was excluded from the analysis because three‐dimensional coordinates (x, y, z) were not reported. We entered the data of the rest of the seven mentioned studies (for details, please see Table 2). For these seven studies, four utilized fMRI, two utilized positron emission tomography, and one utilized MRI to acquire brain images. The selected studies for the meta‐analysis had different baseline conditions, however, the experimental tasks or structural images of these studies aided to determine the neural mechanisms subserving auditory lexical tone processing in the temporal cortex of the left hemisphere.

Table 2.

Neuroimaging studies selected for the meta‐analysis

Literature	Language	n	Scanner	Task	Baseline
[Klein et al., 2001]	Mandarin	24	PET	Tonal discrimination	Silent baseline
[Gandour et al., 2003]	Mandarin	20	1.5T	Tonal discrimination	Passive listening
[Wang et al., 2003]	Mandarin	6	1.5T	Mandarin tone identification	visual, auditory and motor control tasks
[Wong et al., 2004]	Mandarin, English	14	PET	Mandarin tone discrimination; English pitch discrimination	Passive Mandarin listening; passive English listening; rest
[Xu et al., 2006]	Mandarin, Thai	20	1.5T	Tonal discrimination	Passive listening
[Liu et al., 2006]	Mandarin	10	2T MRI	Mandarin pinyin‐naming; Mandarin character‐naming	Fixation
[Crinion et al., 2009]	Mandarin, English	59	1.5T MRI	–	–

n = number of subjects.

Activation Likelihood Estimation

We analyzed all studies which met the inclusion criteria. All MNI coordinates were converted into Talairach space using the icbm2tal transform [Lancaster et al., 2007] implemented in GingerALE software package [Eickhoff et al., 2009, 2011]. ALE maps were generated by the ALE method [Turkeltaub et al., 2002; Laird et al., 2005], using a FWHM of 10 mm. The test was corrected for multiple comparisons using the FDR method with a threshold at P < 0.05 corrected. ALE maps were computed for studies with auditory lexical tone processing. For detailed procedures of using ALE, please refer to Laird et al. [2005].

RESULTS AND DISCUSSION

Table 3 and Figure 4 illustrated the results of our ALE meta‐analysis of the selected literature on auditory lexical tone processing. Two clusters of activation likelihood were identified, one in the left STG (BA42; x = −58, y = −26, z = 12) and the other in the left putamen (x = −32, y = −16, z = 2), with cluster sizes of 288 and 224 mm³, respectively.

Table 3.

ALE Meta‐Analysis of auditory lexical tone processing in left temporal cortex

Anatomical region	BA	Coordinate (x,y,z)			ALE (10−2)	Vol. (mm3)
L superior temporal gyrus	42	−58	−26	12	1.04	288
L putamen	–	−32	−16	2	0.98	224

BA, Brodmann area; L, left.

Figure 4.

ALE maps of auditory lexical tone processing (P < 0.05, corrected). A: Axial sections; B: Lateral view. The functional maps (in color) are overlaid on the corresponding T1 images (in gray scale). Planes are axial sections, labeled with the height (mm) relative to the bicommisural line. L = the left hemisphere and R = the right hemisphere.

This meta‐analysis indicates that the left STG at BA 42 is responsible for lexical tone processing in the spoken language. This region, however, is spatially different from the two temporal cortex regions we identified in Experiment 1 (i.e., the left inferior temporal gyrus, BA37, and the left posterior middle/medial temporal gyrus, BA19).

GENERAL DISCUSSION

This is the first fMRI study to examine the neural substrates underlying lexical tone perception during silent reading of printed words. Several important findings have been demonstrated. First, a number of brain areas associated with tone perception in spoken language, such as the fronto‐parietal network and the putamen [Gandour et al., 2000, 2003; Hsieh et al., 2001; Klein et al., 2001; Li et al., 2010; Wong et al., 2004; see Zatorre and Gandour, 2008], are demonstrated to be involved in lexical tone reading. Both the left and the right hemispheric regions are engaged, though the left hemispheric activations are much stronger in processing tones of Chinese single characters.

Most importantly, we found that the left STG, a region critical for speech and nonspeech pitch processing widely demonstrated in auditory research [Gandour and Dardarananda, 1983; Gandour, 1992; Kadyamusuma et al., 2011; Liang and van Heuven, 2004; Binder et al., 2000; Price et al., 2005; Zatorre et al., 2002; Gandour et al., 2004; Klein et al., 2001; Luo et al., 2006; Liu et al., 2006; Wang et al., 2003; Wong et al., 2004, 2008; Xu et al., 2006; Zhang et al., 2011], was not involved in lexical tone reading in our experiment. The left posterior middle/medial temporal gyrus at BA 19 was indeed activated in our study, but it is close to the fusiform gyrus and is spatially different from the STG mediating auditory tone perception. The STG is involved in the initial processing of auditory stimuli [Hickok and Poeppel, 2007; Klein et al., 2001] and thus, its activity is assumed to be input‐driven. This reveals that speech tone perception depends on neural mechanisms that are specific to the speech domain. Our results of the tone processing in the visual domain lend support to this input‐dependent hypothesis [Gandour et al., 2000; Liberman and Mattingly, 1989]. Cortical regions critical for lexical tone processing seem to be modality‐specific.

In this study, we did not find activation in the right STG, which has been repeatedly shown to be critical to auditory pitch perception, vocal pitch error detection, and voice control in previous literature (Johnsrude et al., 2000; Zatorre and Berlin, 2001; Robin et al., 1990; Flagmeier et al., 2014; Parkinson et al., 2013). Thus, although both left and right STG are important to pitch modulation and sensory control of vocalization in humans, neither of them is related to lexical tone processing in reading. In fact, even though the posterior middle/medial temporal cortex is relevant to tone reading, its activation is left‐lateralized in our study.

In summary, our findings suggest that the neural circuitry subserving lexical tone processing is differentially engaged depending on the input modality. Further research is needed to directly compare the neural specialization of lexical pitch in auditory and visual domains.