Optical imaging of phonological processing in two distinct orthographies

Abstract

Recent fMRI studies comparing the processing of alphabetic versus logographic scripts provide evidence for shared and orthography-specific regions of neural activity. The present study used near-infrared spectroscopy to compare (within and across brain regions) the time course of neural activation for these two distinct orthographies. Native readers of English and of Chinese were tested on a homophone judgment task. Differences across groups were obtained in the time course of hemodynamic change for the left middle frontal, left superior temporal, and left supra-marginal gyri. Results thus support previous findings using fMRI and suggest that different neural mappings arise depending on whether an individual has learned to process written language using an alphabetic or logographic script.

Introduction

Writing systems differ in how they represent spoken language in written form. For alphabetic writing systems such as English, orthographic units (i.e., letters) represent phonemes. For morphosyllabic writing systems such as Chinese, orthographic unites (i.e., symbols) represent syllables. These differences have consequence, as studies show that writing systems with different orthography to phonology mapping are processed via different underlying cognitive processes (Coltheart et al. 2001; Perfetti et al. 2005).

Recent progress in the development of techniques for recording brain activity has opened doors for researchers who wish to examine the brain areas that are involved in visual word recognition for different writing systems. A primary concern in these studies has been determining the neural substrates of phonological processing. Two recent meta-analyses of neuroimaging studies (Bolger et al. 2005; Tan et al. 2005) identified several brain areas related to phonological processing. With English writing, brain activation was pronounced in the left dorsal temporoparietal system, including posterior regions of the left superior temporal gyrus (BA22), the angular gyrus (BA39), and the supramarginal gyrus (BA40). These areas are thought to mediate grapheme-to-phoneme conversion and fine-grained phonemic processing in alphabetic writing systems (Bolger et al. 2005; Tan et al. 2005). With Chinese writing, strong activation was found in the left middle frontal gyrus (BA9), which has been suggested to relate to “look-up” processes of addressed phonology (Bolger et al. 2005; Tan et al. 2005). Further, the left inferior prefrontal gyrus (BA44,45,46,47), suggested to relate to subvocal rehearsal functions, is relevant to both Chinese and English readers (Tan et al. 2005).

Although these findings are suggestive, few neuroimaging studies have examined phonological processing in different writing systems in a single study. An exception was an fMRI study by Tan et al. (2003), whose findings suggested activation of the middle frontal region for Chinese and of the superior temporal region for English. Unfortunately, this study did not present the data in a way that allowed for a fine-grained analysis of hemodynamic responses.

In the present study, we compared brain areas involved in phonological processing for English and Chinese using the hemodynamic measure of near-infrared spectroscopy (NIRS). NIRS measures changes in the concentration of oxy-hemoglobin and deoxy-hemoglobin in the brain regions of interest after near-infrared light penetrates the scalp, is absorbed and then scattered. NIRS is a more portable and affordable alternative to fMRI, providing reasonable temporal and spatial resolution (Hochman 2000; Strangman et al. 2002a, b). Previous studies suggest that NIRS data are highly consistent with fMRI data (Strangman et al. 2002a, b), but it is an empirical question whether analysis of phonological processing will converge in this instance, using this alternate method.

To examine the processing of different writing systems in a single study, our task was modified in important ways from prior work. Tan et al.'s (2003) fMRI study compared Chinese and English readers on a rhyme judgment task, which is the typical task used in studies with English (Tan et al. 2005). We suggest, however, that rhyme judgment is a less natural task for Chinese speakers, since it forces them to segment sounds into phonemes, which are not represented in the Chinese writing system. We thus used a homophone judgment task for both Chinese and English readers, while acknowledging that this task may be somewhat more difficult than rhyme judgment for English readers. Our decision to use homophone judgment was based on it being a more natural task for Chinese readers since it calls on whole word phonology (Chen et al. 2007).

Our study also differed from previous work in terms of experimental design. Whereas previous NIRS studies of language have used a block design, our study employed an event-related design. Although a block design may have superior statistical power to detect subtle differences (Friston et al. 1999), event-related designs allow randomized presentation. This is desirable in investigations where carryover effects introduce response artifacts (Chee et al. 2003). Using such a design in the present study also allows us to determine whether event-related designs are in general sensitive to the subtle changes typically observed in word recognition experiments. Because the NIRS method does not permit monitoring of blood flow change of the whole brain, it was important to identify the brain region(s) of interest (ROI) at the outset. Four ROIs in the present study were identified, all in the left hemisphere. We report findings for each of these four regions, as outlined below.

Materials and methods

Participants

Fifteen native Chinese readers and 15 native English readers from a large southwestern US university participated in the experiment. The Chinese readers were late Chinese–English bilinguals from Taiwan who were fluent in Mandarin and were undertaking graduate study in the US (for less than 2 years, on average) at the time of testing. All were right-handed with normal or corrected-to-normal vision. Each was paid $10 or received course credit. All experiments were conducted with the understanding and the written consent of each participant. The study was also approved by a local ethics committee.

Materials and procedures

Four experiments were conducted, two with Chinese readers and two with English readers. In each group, the experimental task was homophone judgment and the control task was font-size judgment. The tasks were administered in a random sequence. Ten pairs of homophones were selected for homophone judgment. Stimuli were all medium to high frequency Chinese characters or English words. Chinese stimuli were all single integral characters which contained only one radical. The two Chinese characters selected for the homophonic or non-homophonic pairs were in different graphic forms and were matched in the number of strokes. The two English words selected for the homophonic or non-homophonic pairs were all four-letter monosyllabic words with similar visual forms. In the font-size judgment task, ten pairs of unrelated stimuli with the same font-size were selected. All pairs in the font task were of compatible frequency to that in the homophone judgment task. In both the experimental and control task, another set of ten pairs of Chinese characters or English words were selected as fillers. These were selected to ensure that they shared neither phonology nor visual form but had comparable frequency as the stimuli of interest.

Homophone judgment

Participants, tested individually, first saw a fixation signal (a cross) presented at the center of the screen and were instructed to press a button upon seeing it. A pair of either Chinese characters or English words was then presented at the center of the screen. Participants were instructed to make a speeded homophone judgment and press an assigned button only if they thought the pair of stimuli were homophonic Chinese characters (e.g., , , both of which are pronounced as/hong2/, where two represents the tone in Chinese) or English words (e.g., flower and flour). This was followed by a blank screen presented for 15 s. If participants did not think the stimuli were homophonic and did not press the button, the stimulus disappeared after 2 s and was followed by a blank screen. Reaction time (RT) in milliseconds was recorded from the onset of stimulus presentation until the participant pressed a button. Participants received ten practice trials to familiarize them with the procedure.

Font-size judgment

The procedure here was the same as for the homophone task, except that participants were asked to decide if the two visually presented stimuli used the same font size. Participants were instructed to press a specific button as rapidly as possible but only if they thought the pair of stimuli had the same font size. Each participant received a different randomized sequence of stimuli for each judgment task.

Apparatus

All experiments were prepared using an E-Prime software package (Schneider et al. 2002). The optical signals were collected by an electronic control box serving both as the source and the receiver of the near-infrared light. Strips were designed with two laser emitters that directed the near-infrared light into the scalp, and four laser detectors that received the returning near-infrared light. These were placed on participants’ heads to record changes in blood flow during the experimental tasks and were positioned over the regions depicted in Fig. 1. Specifically, two emitters were located at F₃ and TP₃ following the International 10/20 system. TP₃ is located midway between P₃ and T₃ which has been hypothesized to be relevant to language function (Homan et al. 1987). Four detectors were located 3 cm away from centrally located emitters, which is the distance suggested to ensure detection of near-infrared light following penetration of the neocortex (Hedden and Delpy 1997). Four channels, formed by different pairs of emitters and detectors, covered the different brain areas related to phonological processing: Channel 1 primarily covered the left inferior prefrontal gyrus (Homan et al. 1987; Okamoto et al. 2003), related to both Chinese and English phonological processing (Tan et al. 2005). Channel 2 primarily covered the left middle frontal gyrus (BA9) (Homan et al. 1987; Okamoto et al. 2003), an area suggested to be important in processing addressed phonology in Chinese (Bolger et al. 2005; Tan et al. 2005). Channel 3 primarily covered the left superior temporal gyrus (BA22) and supramarginal gyrus (BA40), and Channel 4 primarily covered the angular gyrus (BA39) (Homan et al. 1987; Okamoto et al. 2003). These areas were identified as playing an important role in grapheme-to-phoneme processing in English (Bolger et al. 2005; Tan et al. 2005). A chin rest was used to reduce motion artifacts.

Fig. 1.

Positions of the probes used. Open circles denote laser emitters and filled circles denote laser detectors. Two laser emitters were placed at F₃ and TP₃ following International 10/20 system. The distances between laser emitters and detectors were set at 3 cm

Each emitter contained two light sources with a wavelength of 690 and 830 nm, respectively. The former is more sensitive to the deoxy-hemoglobin and the latter is more sensitive to the oxy-hemoglobin. Another laptop was programmed to control and record the signals received by the electronic control box. The data recorded by the control box were then converted to relative concentrations of oxy-hemoglobin and deoxy-hemoglobin using the modified Beer–Lambert law (Strangman et al. 2002a, b).

Results

Behavioral data

For all four tasks, only the main stimuli for which participants pressed the button were analyzed. For RT analyses, only correct responses were calculated. Table 1 shows the accuracy and re-computed means for correct RTs for each task. Two separate analyses of variance with Group (Chinese vs. English readers) as the between-subject factor and Task (homophone vs. font size judgment task) as the within-subject factor were conducted, one for accuracy and the other for RT. No differences were found between Chinese and English readers (Accuracy F(1,28) < 1; RT F(1,28) = 1.88, n.s.) but a significant task effect was found (Accuracy F(1,28) = 6.24, P < 0.05; RT F(1,28) = 22.02, P < 0.001], suggesting faster and more accurate responses in the font size judgment task. A significant Group by Task interaction was found for RT, F(1,28) = 5.29, P < 0.05, but not for accuracy, F(1,28) = 2.04, n.s., suggesting the RT difference between two tasks was larger for Chinese readers than for English readers.

Table 1.

Mean reaction time (ms) and accuracy (%) in homophone and font size judgment

	Chinese data		English data
	RT	Accuracy	RT	Accuracy
Homophone judgment	1,208 (62)	92 (2.2)	987 (62)	95 (2.2)
Font size judgment	840 (65)	99 (2.0)	861 (65)	97 (2.0)

Standard errors are reported in parentheses

RT reaction time

NIRS data

The NIRS data from four channels were digitally recorded at 200 Hz. The data were then converted into optical density units that were digitized and low-pass-filtered at 1 Hz and high-pass-filtered at 0.02 Hz to reduce the noise of systemic physiology. The filtered data were then converted to reflect the concentration of both the oxy-hemoglobin and deoxy-hemoglobin; these served as the data used for advanced analysis. The converted data were analyzed in 17-s epochs including 2 s before and 15 s after the onset of the stimuli. Data conversion was conducted using HomER software (Huppert and Boas 2005).

NIRS data were first down-sampled to 2 Hz and then averaged in further analyses. The data were first calculated by subtracting font size judgment data from homophone judgment data to reflect phonological processing, following Tan et al. (2003), by taking out components related to visual and judgment processes. The results are depicted in Fig. 2. We report only the oxy-hemoglobin data. Black lines represent data from native English readers and gray lines represent data from native Chinese readers. Filled circles on the black lines (English) indicate time points at which the changes in blood flow were significantly different from zero. Unfilled circles on the gray lines (Chinese) indicate time points at which the changes in blood flow were significantly different from zero. The asterisks above the standard error bars indicate time points at which the changes in blood flow were significantly different between English and Chinese readers.

Fig. 2.

Grand average changes of the concentration in oxy-hemoglobin. The data were analyzed in 17-s epochs with 2 s before and 15 s after the onset of the stimuli. The data presented here were diference data calculated by subtracting homophone judgment data from font-size judgment data. Data from native English readers were presented in black lines and those from native Chinese readers were presented in gray lines. Filled circles on black lines were time points in which changes of blood flow were significantly different from zero. Open circles on gray lines were time points in which changes of blood flow were significantly different from zero. The bars on each time point show the standard errors. The asterisks on the edges of the standard error bars are time points in which the changes of the blood flow were significantly different for English and Chinese readers

Chinese readers

For this group, only Channel 2, which covered the left middle frontal gyrus (BA9), revealed a significant increase in the change of oxy-hemoglobin concentration, t(14) = 2.70, P < 0.01. When analyzing data for each sampled time point, the change in oxy-hemoglobin concentration was significantly activated at 9 s after the onset of the stimuli in Channel 1, P < 0.05, which covered mostly the left inferior prefrontal gyrus. The change in oxy-hemoglobin concentration was significantly increased from zero at 4–10 and 12 s after the onset of the stimuli in Channel 2, Ps <0.05. No clear increases in the change of oxy-hemoglobin concentration were found in Channel 3 and Channel 4.

English readers

For English readers, a marginally significant increase in oxy-hemoglobin concentration was found in Channel 1, t(14) = 1.38, P = 0.09. The change in oxy-hemoglobin concentration was also found to be marginally significant in Channel 3, which mainly covered the left superior temporal gyrus (BA22) and supramarginal gyrus (BA40), t(14) = 1.57, P = 0.07. When analyzing data for each sampled time point, the increases in oxy-hemoglobin concentration were significantly different from zero at 8.5 to 9.5, 11 to 12, and 13.5 to 14 s after the onset of the stimuli in Channel 1, Ps < 0.05. Oxy-hemoglobin concentration significantly increased from zero at 8 to 10 s after the onset of the stimuli in Channel 2, Ps < 0.05. Oxy-hemoglobin concentration also significantly increased from zero at 8 to 10.5 s after the onset of the stimuli in Channel 3, Ps < 0.05. No clear increases in oxy-hemoglobin concentration were found in Channel 4.

Chinese versus English readers

As revealed by direct comparison of the Chinese and English data, Chinese readers showed stronger activation then English readers during 5.5 to 6.5 s after the onset of the stimuli in Channel 2, Ps < 0.05. Conversely, English readers showed stronger activation then Chinese readers at 8.5 to 9.5 and 10.5 s after the onset of the stimuli in Channel 3, Ps < 0.05. No meaningful differences were found between the two groups of readers in either Channel 1 or Channel 4.

In both our Chinese and English data presented in Fig. 2, the hemodynamic response functions (HRFs) appear slower than those found in typical fMRI studies. However, in our data analyses presented in Fig. 2, it is important to note that we subtracted homophone judgment data from font-size judgment data; thus, the remaining data presented are actually difference data, instead of original HRFs. It is possible that there are no differences between homophone judgment and font-size tasks in the early stage, but that phonological processes in the homophone judgment task lasted longer compared to the processes in the font-size judgment task, revealing a difference at a later stage.

Discussion

Our results suggest that the NIRS method is, in fact, capable of monitoring subtle hemodynamic changes in complex language experiments, even with the less powerful event-related design used (one that guards against carry-over effects). Applying a phonological judgment task that focuses on homophone judgment (which we view as more appropriate than rhyme judgments for language comparisons), our results generally confirmed what has been found in previous fMRI studies comparing brain regions responsible for phonological processing in different writing systems with distinct visual form to phonology mapping (Bolger et al. 2005; Tan et al. 2003, 2005). After subtracting font size judgment responses from homophone judgment ones, the present results suggest an important role of the left middle frontal gyrus (BA9) for Chinese character processing and a special role of the left superior temporal gyrus (BA22) and supramarginal gyrus (BA40) for English readers. Although these language brain areas (BA9,40,22) were activated for both the native Chinese and the native English readers, different brain areas were relatively more important for the different language groups.

Researchers have suggested that the left middle frontal gyrus is relevant to look-up processes for addressed phonology (Bolger et al. 2005; Tan et al. 2003, 2005). For writing systems like Chinese, lexical access is achieved by directly accessing the lexical entry from the graphic form. Because phonology is hard to assemble from sub-lexical phonology in such a system, it is typically retrieved from a stored representation in the mental lexicon (Chen et al. 2007; Coltheart et al. 2001). The finding of stronger activation in the left middle frontal gyrus in Chinese than English readers for homophone judgments confirms the role of this area in addressed phonological processing. We also found some activation in the left middle frontal gyrus in English readers, although its magnitude was weaker than that in Chinese readers. Based on the dual route model of word recognition, high frequency words may use addressed phonology instead of assembled phonology in word recognition (Coltheart et al. 2001; Seidenberg 1985). We suggest that activation of the left middle frontal gyrus was due to the high to medium frequency stimuli selected in the English tasks.

Another possible explanation is that left middle frontal gyrus may support the memory of orthographic information during phonological processing (Perfetti et al. 2007), reflecting the fact that orthographic processing is relatively more important for Chinese reading. Due to the fact that our English stimuli were simple and of high frequency (and presumably required orthographic information in retrieving phonology), some activation found in left middle frontal gyrus even for English may be expected. Although we obtained some additional activation in the left inferior prefrontal gyrus, no differences were found between Chinese and English readers, supporting the notion that this region operates similarly in both Chinese and English for subvocal rehearsal (Tan et al. 2005).

For alphabetic writing systems, fine-grained phonemic processing is comparatively more important than addressed processing. Assembling phonology requires the use of fine-grained grapheme-to-phoneme conversions. Since logo-graphic writing systems have no graphic forms that can map to phonemes, assembled phonological processing is not the dominant means to access phonological information in Chinese (Wu and Chen 2000). This idea was confirmed by our finding of no significant activation in the left dorsal temporoparietal system for Chinese readers. English readers showed activation in posterior regions of the left superior temporal gyrus and supermarginal gyrus, underscoring the important role of assembled phonology in English. However, this difference may also be due to the different brain shapes between Caucasians and Asians in this area (Zilles et al. 2001). Further studies will be needed to rule out this possibility. We did not obtain meaningful activation in Channel 4, which covered the other dorsal temporoparietal system, i.e., the angular gyrus (BA39). Although this brain region is not as consistently found to be relevant to English processing, we suggest that this may be due to the fact that Channel 4 also covers some other visual areas and may thus not be as sensitive in detecting subtle differences.

Conclusion

The present study directly compared different brain regions involved in phonological processing (as indexed by homophone judgments) using a NIRS-based event-related design. Whereas look-up processes of addressed phonology played a more important role in Chinese processing (as evidenced by stronger activation in the left middle frontal gyrus), fine-grained phonemic processing played a more important role in English processing (as evidenced by clearer activation in left superior temporal and supramarginal gyri). Our results with NIRS thus support a previous finding using fMRI and suggest that phonological processing in the human brain may be tuned by different kinds of mappings between orthography and phonology.