Brain activation during semantic judgment of Chinese sentences: A functional MRI study

Abstract

A functional magnetic resonance imaging (fMRI) study was conducted to investigate whether the anatomic substrates of semantic memory may reflect categorical organization and to determine whether the left middle frontal gyrus (Brodmann area [BA] 9) plays a role in Chinese semantic judgment. Unlike previous studies using a word‐retrieval task (e.g., word generation, naming, and word categorization), we used a typical task of semantic knowledge retrieval in cognitive psychology in which subjects were asked to determine whether a sentence describing an attribute of living things or nonliving things was true or not. The experimental conditions evoked extensive activation over several regions of the brain including a very strong activation in the left middle frontal region (BA9 and BA46). Our data show that there is no unique activation associated with living or nonliving things at the statistical threshold used in our study. The results imply that human semantic system is undifferentiated by category at the neural level. Our findings also corroborate and extend the claim that the left middle frontal gyrus plays an important role in reading Chinese at both the sentence and the word level. Hum Brain Mapping 24:305–312, 2005. © 2005 Wiley‐Liss, Inc.

INTRODUCTION

Since Warrington and Shallice [1984] first reported the existence of selective deficits of semantic knowledge in patients with herpes simplex encephalitis, several investigations have confirmed the phenomenon of category‐specific deficits [Campanella et al.,2003; Farah et al.,1991; Hillis and Caramazza,1991; Kensinger et al.,2003; Warrington and McCarthy,1987], implying the existence of category‐specific neural substrates. Typically, deficits for living things are associated most commonly with bilateral anteromedial and inferior temporal lobe lesions, whereas deficits for artifacts are claimed to be associated with extensive left‐lateralized frontoparietal lesions [Tyler and Moss,2001].

It has therefore been assumed that semantic knowledge is organized by taxonomic category (e.g., living things), i.e., a category‐specific hypothesis of human semantic organization.1 It is the most straightforward interpretation for the existence of category‐specific deficits in the knowledge of either living or nonliving things in patients with brain lesions. According to the hypothesis, the lesion of one brain area would be expected to disrupt only knowledge related to one taxonomic category. Although much data is consistent with this hypothesis, this is not always the case. Some neuropsychological results showed that impairments could occur across categories (e.g., some patients with impairments for living things may also show a deficit for nonliving categories such as musical instruments, mass nouns, and clothing) [Borgo and Shallice,2001; Siri et al.,2003; Warrington and Shallice,1984; Warrington and McCarthy,1987]. In addition, some studies demonstrated the existence of dissociation of impairments within one category [Caramazza and Shelton,1998; Farah et al.,1992; Hart et al.,1985; Hart and Gordon,1992; Kensinger et al.,2003]. Furthermore, Neininger and Pulvermuller [2003] reported that specific word‐category deficits could arise from lesions in the right nondominant hemisphere. This variability of deficits among different patients, largely due to the diversity of locations and extents of focal brain lesions, becomes one of the most fundamental challenges in developing inferences regarding the organization of semantic memory.

Functional imaging techniques, which can explore the neural basis of semantic information in human volunteers directly and noninvasively, have greatly facilitated the study in this field since the last decade. To date, there are several studies that address semantic memory organization in the brain [Chao et al.,1999; Damasio,1996; Devlin et al.,2002; Martin et al.,1996; Mummery et al.,1996,1998; Spitzer et al.,1995,1998; for review, see Bookheimer,2002; Thompson‐Schill,2003]. In the various studies, however, there has been little consistency shown in the cortical regions activated for categories. There have been relatively few imaging studies characterizing typical semantic processing at sentence level in contrast to the numerous studies on single‐word processing (e.g., word generation, word categorization, and picture naming). The extent to which category‐specific activation evoked by single‐word processing stimuli reveals the organization of semantic representations is unknown, for cognitive processes involved in those tasks may be related to lexical (including phonologic and orthographic) representations rather than semantic representations only. To explore further the neural system underlying semantic information is the first aim of the present study.

The second objective of this study is to demonstrate the role of the left middle prefrontal region (Brodmann area [BA] 9) in semantic processing of Chinese sentences. There are several studies implicating distinct cortical areas associated with processing of different languages [Dehaene et al.,1997; Gandour et al.,2000; Kim et al.,1997; Mazoyer et al.,1993; Paulesu et al.,2000; Perani et al.,1996]. Written Chinese is considered to be a logographic system, in which characters as a basic writing unit possess a number of strokes packed into a square shape, which presents a sharp contrast to English and other alphabetic writing systems. Recent studies with Chinese [Chee et al.,1999,2000; Chen et al.,2002; Fu et al.,2002; Tan et al.,2000,2001a,b,2003] indicated that some neurocognitive mechanisms underlying Chinese logographic reading might differ from those underlying alphabetic word reading.

Tan et al. [2000] reported that compared to the fixation baseline, peak activations resulting only from Chinese semantic decisions in which subjects were asked to judge whether a pair of Chinese characters exposed synchronously were related semantically were located in the left middle frontal gyrus (BA9). Their finding was supported subsequently by other studies using similar Chinese word or character retrieval tasks [Chee et al.,2000; Fu et al.,2002; Tan et al.,2001a,b]. A more recent work by Luke et al. [2002], in which bilinguals were asked to decide whether a viewed phrase either in Chinese or English was semantically acceptable, also found that the left prefrontal region mediates in Chinese semantic plausibility judgment.

In our experiment, we used semantic judgment tasks at sentence level rather than at word or phrase level. Each sentence depicted one visual attribute of an entity (either a living or nonliving thing). Subjects were asked to judge whether contents described by the sentence were true or false. The experimental tasks were alternated with a control task in which the subjects were instructed to passively view nonletter character strings. Functional brain activation was measured during each block using functional magnetic resonance imaging (fMRI). Comparisons among the conditions allowed us to identify brain activations related to semantic memory, especially for knowledge presented in Chinese sentences.

SUBJECTS AND METHODS

Subjects

Eight right‐handed, healthy male volunteers (age 18–25 years) participated in this study. No subjects had a history of psychiatric or neurologic disorders, head trauma with loss of consciousness, bleeding disorders, or intake of tranquilizing drugs. All subjects were native Chinese speakers. Each subject voluntarily signed the consent form that was approved by the ethics committee at the Chang Gung Memorial Hospital.

Materials and Design

The subjects were instructed to answer yes–no semantic knowledge questions about visual attributes of 24 living things (animals) and 24 nonliving things (e.g., man‐made). For example, “Do ducks have spiky mouths?” and “Is a bus quadrate?” All questions were presented in Chinese (see Fig. 1). Each subject was asked 24 questions for each condition, for a total of 48 different questions in the experiment. The questions were tested in advance for response latency and accuracy by 20 undergraduate students to ensure that all the questions had only one clear answer and subjects respond within 5 s. The length of each question ranged between 10–12 Chinese characters. The number of yes answers to questions about living things was the same as the number of yes answers to questions about nonliving things. The baseline was derived from the experimental questions by replacing all Chinese characters with nonletter character strings (e.g., &*#$), retaining interword spacing and equating for length. Subjects were instructed to use the up or down key of the response box to choose whether the answer to a question was yes or no. For the baseline, participants were also instructed to press the key after visually scanning the lines to balance the muscle recruitment between experimental tasks and baseline.

Figure 1.

Examples of stimuli used in the fMRI study.

Image Acquisition

The experiment was carried out on a 1.5‐T Magnetom Vision MRI scanner (Siemens, Erlangen, Germany) at Chang Gung Memorial Hospital. The stimuli were shown through a goggle display system (Resonance Technology Inc., Northridge, CA). Before MRI, the subject was visually familiarized with the procedural and experimental conditions to minimize anxiety and enhance task performance. After this familiarization, the subject lay supine on the scanning table and was fitted with plastic ear‐canal molds. The subject's head was immobilized by a tightly fitting, thermally molded plastic facial mask that extended from the hairline to the chin. A single‐shot T₂*‐weighted gradient‐echo echo planar imaging (EPI) sequence was used for the fMRI scans with slice thickness = 5 mm, in‐plane resolution = 3.3 mm × 3.3 mm, and repetition time[TR]/echo time [TE]/flip angle = 3,000 ms/60 ms/90 degrees. The field of view was 211 mm × 211 mm, and the acquisition matrix was 64 × 64. To cover the whole brain, 24 contiguous axial slices were acquired; 120 images were acquired for each slice. The anatomic MRI was acquired using a T₁‐weighted 3‐D gradient‐echo pulse sequence. This sequence provided high‐resolution (1 × 1 × 1 mm³) images of the entire brain.

The experiment was conducted in a single run that included eight cycles of alternating 15‐s control and 30‐s task blocks. There were two task conditions, living and nonliving, and each task block represented one of the conditions. Each task block contained six sentences presented for 3 s each, followed by visual fixation on a crosshair for 2 s. Each task condition was repeated four times and the order of presentation was alternated.

Image Analysis

We used MATLAB (Math Works, Natick, MA) and a spatial clustering analysis technique [Xiong et al.,1995] for image data processing. Motion correction of the fMRI images was carried out with a six‐parameter, rigid‐body algorithm using MEDx (Sensor System, Sterling, VA). The images were spatially smoothed by convolution with a 3‐D two‐voxel (6.6 mm) full width at half maximum (FWHM) Gaussian kernel. Skull stripping of the 3‐D T₁‐weighted MR images was done using Alice (Perceptive Systems, Inc., Boulder, CO). These images were then spatially normalized to the Talairach brain atlas using the Convex Hull algorithm. Images from the first 9 s of each condition were excluded from further functional data processing to minimize the transient effects of hemodynamic responses. Activation maps were calculated by comparing images acquired during the task with the control conditions using Student's group t test. Like the T₁‐weighted anatomic images, the activation maps were also spatially normalized into Talairach space using the Convex Hull algorithm. The averaged activation maps of the eight subjects with a t value threshold of 3.2 and a cluster threshold of 400 mm³ (P < 0.05, corrected) were then overlaid on the corresponding T₁ images. Talairach coordinates and volume (in mm³) of the activation clusters were determined based on the activation maps. Anatomic labels (lobes; gyri) and BA designations were applied automatically using a 3‐D electronic brain atlas.

RESULTS

Table I provides spatial coordinates of peak anatomic recruitment within each cluster and summarizes the anatomic location of each significant cluster according to the normalized Talairach standard brain picture. In comparison to the baseline, activation associated with the retrieval of semantics recruited three left‐hemisphere regions and two right‐hemisphere regions to a significant extent, as illustrated in Figure 2. These regions included the frontal regions involving BA6, 9, 46, 45, left and right occipital cortex (BA18 and 19), and left parietal lobe (BA19).

Table I.

Significant activations during retrievals of different types of semantic knowledge against baseline

Activation locus	Brodmann area	Coordinates			Volume (mm3)	t	P
		x	y	z
Living
L frontal lobe	9	−45	16	28	3,440	4.09	0.00013
	46	−43	36	14	448	3.74	0.00034
L parietal lobe	19	−27	−69	35	840	4.63	0.00003
L occipital lobe	18	−21	−84	−15	3,888	4.00	0.00016
R frontal lobe	6	1	−1	54	648	4.00	0.00016
R occipital lobe	18	21	−85	−13	704	4.02	0.00015
Nonliving
L frontal lobe	6	−44	−3	51	568	4.01	0.00016
	6	−40	−6	37	400	3.81	0.00028
	6	0	1	53	960	3.95	0.00019
	9	−45	16	29	4,368	4.10	0.00012
	46	−44	36	10	880	3.88	0.00023
	45	−48	23	3	856	3.78	0.00030
L parietal lobe	19	−26	−69	34	872	4.31	0.00007
L occipital lobe	19	−25	−82	−15	4,848	4.03	0.00015
R occipital lobe	18	21	−87	−13	1,568	4.33	0.00006

Peaks shown for all clusters significant at corrected P < 0.05. Talairach coordinates (x, y, z) are in millimeters. Multiple peaks within a lobe are shown on subsequent lines. L, left, R, right.

Figure 2.

Normalized brain activation maps averaged over eight subjects at a threshold of P < 0.05 (corrected) when compared to baseline. A: Living vs. baseline. B: Nonliving vs. baseline. Greater activations are centered on the left middle frontal gyrus and left visual cortex; activations across conditions are largely overlapped. Talairach Z coordinates are given below each horizontal section; color bar represents t values. Activated areas are described in Table I. L, left hemisphere; R, right hemisphere.

Comparing living activation against the baseline, six large clusters of activation were found, most of which were located in the visual cortex and anterior middle frontal gyrus, primarily in left hemisphere (LH). The volumes activated in left fusiform gyrus and left middle frontal gyrus were more than 3,000 mm³ (3,880 mm³ and 3,440 mm³, respectively). The right fusiform gyrus and right medial frontal gyrus were also activated but to a smaller extent (1,112 mm³ and 448 mm³, respectively).

When comparing nonliving activation against the baseline, a very similar pattern of activations was observed, including left inferior prefrontal region (BA45 and 46), bilateral fusiform and left secondary motor cortex, with a peak in left fusiform and left middle frontal gyrus. The activation volume of living in fusiform (both for left and right) was larger than that of living (left: 4,848 vs. 3,888 mm³; right: 1,568 vs. 704 mm³).

To test further whether a recruitment pattern was associated uniquely with a category, the activity during the retrieval of knowledge about living and nonliving things was contrasted directly. With the original statistical threshold (t > 3.2; corrected P < 0.05), no differences were observed among the signals elicited by the retrieval of semantic information about living or nonliving things. When the statistical threshold was relaxed (t > 2.4; uncorrected P < 0.01; see Table II), however, it was found that small category‐specific activation occurred (Fig. 3). Left lingual gyrus (BA17) and right anterior cingulate (BA32) were activated uniquely during retrieval of nonliving thing knowledge, whereas no areas were activated specifically during retrieval of living thing knowledge.

Table II.

Significant activations for the contrasts across conditions

Significance	Activation locus	Brodmann area	Coordinates			Volume (mm3)	t	P
			x	y	z
Positive	—	—	—	—	—	—	—	—
Negative	L occipital lobe	17	−10	−94	−17	344	2.74	0.004
	R limbic lobe	32	1	48	6	360	2.84	0.003

Positive significance means activation from “living–nonliving” and negative significance means activation from “nonliving–living.” There is no data for positive significance. Peaks shown for all clusters significant at uncorrected P < 0.01. Talairach coordinates (x, y, z) are expressed in millimeters. L, left; R, right.

Figure 3.

Normalized brain activation maps averaged over eight subjects at a threshold of P < 0.01 (uncorrected) after intercondition subtraction (living vs. nonliving). Positive significance (yellow to red) means activation from “living–nonliving” and negative significance (green to blue) means activation from “nonliving–living.” Talairach z coordinates are given below each horizontal section; color bar represents t values. Activated areas are described in Table I. L, left hemisphere; R, right hemisphere.

DISCUSSION

The present study had two objectives. One was to examine further whether there are localized representations of semantic information in the human brain as a function of category, using a typical task of semantic knowledge retrieval. The other was to explore whether there are brain activations specifically associated with semantic processing of Chinese at sentence and at word level, to provide converging evidence for the claim that differences in surface features (orthography, phonology, and syntax) of different languages affect their cerebral organization.

For the first objective, the hypothesis that semantic information is organized by taxonomic category in the human brain is based largely and originally on neuropsychologic dissociations. The evidence for this hypothesis has thus far been controversial among studies using functional imaging. Some studies [Cappa et al.,1998; Chao et al.,1999; Damasio,1996; Martin et al.,1996; Mummery et al.,1996; Okada et al.,2000; Perani et al.,1995; Spitzer et al.,1995,1998] observed activation uniquely associated with category, whereas other studies [Devlin et al.,2000,2002; Tyler and Moss,2001; Tyler et al.,2003] found little difference in activation as a function of category.

As was discussed in the preceding section, it is assumed that one crucial factor that contributes to these inconsistencies is the difference in the stimuli or cognitive tasks used to elicit semantic processing. Most previous studies used words as their stimuli. It is thought that semantic knowledge embodies both concept and proposition, and that semantic features of one concept depend largely on its sentence context and syntax. Semantic processing should thus be reflected better through sentence comprehension than through word cognition. With respect to cognitive tasks, semantic judgment, well known as a typical task of semantic knowledge retrieval in cognitive psychology, is thought psychologically to help subjects better comprehend tasks. Subjects are then expected to produce purer cognitive processing, which results in a much more concentrated brain activation specifically associated with semantic processing. The naming tasks used frequently in prior semantic processing studies are less likely to involve deep semantics, whereas other tasks, due to their ambiguity, are thought to be likely to evoke additional cognitive processing (such as association) and thus confound results. In this study, the experiment therefore used a semantic judgment task presented visually in the form of a sentence. Moreover, to ensure that the experimental task involved single cognitive processing (retrieval of semantic information), each question described only one visual attribute of one object. This design was expected to give a better and more explicit interpretation for the cognitive process related to activation of specific brain regions.

The results showed that compared to baseline, brain areas activated by semantic information about living things and nonliving things largely overlap. When directly comparing activations elicited by semantic information about living and nonliving things, no difference of recruitment pattern was found at the original statistical threshold (t > 3.2; corrected P < 0.05).

With respect to the very small category‐specific sites (left lingual gyrus and right anterior cingulate activated by nonliving things only) at uncorrected threshold (t > 2.4; uncorrected P < 0.01), we argue that this cannot be regarded as evidence supporting a category‐specific hypothesis. As pointed out by Devlin et al. [2002], adopting a statistical threshold without correcting for multiple comparisons can generate many false positives and the resulting significance will likely contain Type I errors.

The results were very compatible with the existing neuroimaging literature that has found little evidence of consistent specialization for either natural kinds or artifacts. For example, Devlin et al. [2000] found no differences between categories or domains when matching items across categories and domains on the crucial variables of frequency, letter length, and visual complexity. And in their subsequent study with a lexical decision and a semantic categorization task (to place greater demand on the semantic system than lexical decision), they found weak evidence of functional segregation by domain or categories only at an uncorrected level of significance. The authors suggested that either there is no difference in activation across domains or categories or that such difference is small and cannot be detected by the current experiment or others like it. The present finding thus provides more evidence supporting the suggestion that the semantic system may be undifferentiated by category at the neural level.

It may be argued that the null difference between living and nonliving categories might be due to insufficient statistical power or result from the possibility that the visual‐shape feature is not a key dimension of semantic organization. Such explanations seem inapplicable to this experiment. Both the number of subjects recruited and the number of stimuli used in this study are generally adequate for fMRI data analysis. In addition, in one of our experiments studying auditory characteristics, subjects were asked to respond to another type of semantic question about 24 living and 24 nonliving things during scanning; category effects were not observed in this study either. It seems impossible that visual‐shape or auditory feature is not a key dimension of semantic organization, because most of our semantic information is acquired by seeing or hearing.

These results corroborate an important difference in cortical organization of English and Chinese, which was demonstrated originally by previous studies [Tan et al.,2000] with word‐retrieval tasks.

In the present study, we found extensive activation during processing of Chinese semantic information compared to baseline activation, regardless of category. Peak activations were localized in the left middle frontal cortex (BA9 and part of BA46) and the left fusiform (BA19). The strong activation of the left middle prefrontal cortex at BA9 and BA46 observed here in the processing of Chinese semantic knowledge is very different from that in alphabetic language with either word‐retrieval tasks or semantic judgment tasks at sentence level [Thomson‐Schill et al.,1999]. It is common to find left frontal region activation during English sentence processing; however, such activation is located mostly on BA44/45 [Ben‐Shachar et al.,2004; Constable et al.,2004; for review see Sakai et al.,2001]. Few studies thus far have observed the activation of BA9 during English sentence processing. We argue that the contentious results might be due mostly to the surface distinction between these two languages. They are significantly different in the sequence of an adjunct and its headword in terms of sentence structure: an adjunct must be placed before its headword in Chinese, whereas it always follows its headword in English. It is suggested that the left middle frontal area (BA9) coordinates and integrates the intensive visuospatial analysis demanded by the square configuration and semantic analysis of logographs.

Results of the strong activation of the left middle frontal cortex at BA9 and BA46 in semantic processing of Chinese sentences extends previous findings of reading in English and Chinese. The present study demonstrates further that the left middle frontal area plays an important role in the cognitive processing of written Chinese, not only at word or character level but also at sentence level.

In addition to the common finding of strong activation in BA9, different brain activations from previous research with Chinese characters were observed. In this study, unlike previous studies showing that activation in the visual cortex is right lateralized [Tan et al.,2000], the activity in the occipital region showed left‐hemisphere dominance (LH 3,888 vs. RH 704 mm³ in living condition and LH 4,848 vs. RH 1,568 mm³ in nonliving condition). Perhaps this is due to variation in experimental paradigms (e.g., word generation vs. sentence comprehension).

CONCLUSIONS

First, we claim that results from this better controlled cognitive neuroimaging study provide no support for category‐specific hypothesis of semantic memory in human brain. Our results showed little difference in brain activation in both extent and strength across conditions even at a low and uncorrected threshold. Second, our findings further indicate that the left middle frontal region plays an important role in reading Chinese in sentence processing as well as in word processing.