The neural organization of semantic memory: Electrophysiological activity suggests feature-based segregation

Abstract

Despite decades of research, it remains controversial whether semantic knowledge is anatomically segregated in the human brain. To address this question, we recorded event-related potentials (ERPs) while participants viewed pictures of animals and tools. Within the 200–600-ms epoch after stimulus presentation, animals (relative to tools) elicited an increased anterior negativity that, based on previous ERP studies, we interpret as associated with semantic processing of visual object attributes. In contrast, tools (relative to animals) evoked an enhanced posterior left-lateralized negativity that, according to prior research, might reflect accessing knowledge of characteristic motion and/or more general functional properties of objects. These results support the hypothesis of the neuroanatomical knowledge organization at the level of object features: the observed neurophysiological activity was modulated by the features that were most salient for object recognition. The high temporal resolution of ERPs allowed us to demonstrate that differences in processing animals and tools occurred specifically within the time-window encompassing semantic analysis.

1. Introduction

In a lifetime, we acquire knowledge about numerous objects in our environment. This knowledge includes their names, their properties (visual, acoustic, motor, olfactory, etc.), our own bodily movements involved in interactions with them, as well as more abstract general knowledge (e.g., a giraffe is an African herbivore). How these diverse semantic memories are organized in the brain has been of intense interest to cognitive neuroscientists for several decades (Chang, 1996; Thompson-Schill, 2003). However, while much has been learned about the nature of these representations, there are still competing theoretical perspectives. According to one account, different types of knowledge are stored within different brain regions. In contrast, the alternative framework posits that all semantic information is coded within a unitary neural system. Below we outline each of these theoretical approaches in more detail in relation to the neuropsychological literature, and then briefly review existent functional neuroimaging evidence for and against them. Finally, we describe how we used electrophysiological recordings of the brain activity to distinguish between these theories.

2. The accounts of semantic memory organization: neuropsychological evidence

It has long been known that some patients with focal brain damage show selective deficits in knowledge about particular object categories (e.g., animals, plants, tools; Basso et al., 1988; Damasio et al., 1996; De Renzi and Lucchelli, 1994; Farah and McClelland, 1991; Farah et al., 1991; Hart and Gordon, 1992; Humphreys and Forde, 2001; Laiacona et al., 1993, 1997; McCarthy and Warrington, 1988; Pietrini et al., 1988; for review see Saffran and Schwartz, 1994). This finding has been often cited as evidence in favor of the neuroanatomical segregation of semantic representations. For example, Caramazza and Shelton (1998) have suggested that semantic memories are organized in the brain at the level of whole objects. They speculated that different object categories (e.g., animals, plants, tools) might be supported by distinct brain regions because each category has played a different role in our survival in the course of evolution.

Warrington and McCarthy (1987), Warrington and Shallice (1984) were among the first to notice that the patterns of cognitive impairment observed in different patients could be classified according to the type of attributes that are particularly important for the identification of objects from the affected categories. For example, some patients were less proficient with items whose visual features are most salient for object recognition (e.g., animals and plants). In contrast, other patients had more difficulties with objects that are best defined by their functional properties (e.g., manipulatable man-made objects and body parts). Based on these observations, Warrington and colleagues proposed a currently popular feature-based account of semantic memory organization (Chao et al., 1999; Chao and Martin, 1999, 2000; Holcomb et al., 1999; Holcomb and McPherson, 1994; Martin and Chao, 2001; Martin et al., 1995, 1996; McPherson and Holcomb, 1999; Paivio, 1971, 1986, 1991; Sitnikova et al., 2003; Warrington and McCarthy, 1987; Warrington and Shallice, 1984; West and Holcomb, 2002). According to this view, different types of object features (e.g., visual, auditory, motor, olfactory, abstract/verbal) are stored in distinct brain regions, and category-specific deficits are a byproduct of selective damage to these feature-specific neurocognitive mechanisms.

The alternative account of semantic memory representation posits that all types of knowledge are supported by a unitary neural system with no correspondence between locations in the brain and the content of stored semantic information (Anderson and Bower, 1973; Devlin et al., 2002; Gernsbacher, 1985; Kroll and Potter, 1984; Pylyshyn, 1980; Tyler and Moss, 2001; Tyler et al., 2000, 2003b). Proponents of this model have argued that it is not inconsistent with category-specific cognitive deficits observed in neuropsychological patients. For example, Moss and Tyler (2000), Moss et al. (1998), Tyler and Moss (2001), Tyler et al. (2000, 2003b) pointed out that selective recognition impairments for animals frequently occur in less severe brain damage and involve inability to identify individual animals while the knowledge of the broad category to which the objects belong remains spared. In contrast, selective recognition impairments for tools are more common in relatively severe brain damage and are characterized by major difficulties in object recognition with only some knowledge of the animal world being preserved. These authors argued that this pattern of deficits can be explained by such factors as increased number of shared, inter-correlated (i.e., consistently occurring together in individual category members) features and reduced number of distinctive features associated with animals relative to tools. As a result, milder brain damage may be more likely to impair patients’ ability to use distinctive features to discriminate between different animals than between different tools, but patients with severe brain injury may be more likely to correctly classify animals while being unable to comprehend tools at all.

3. Functional neuroimaging evidence

Recently, positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) methods have been applied to the study of the neural basis of semantic knowledge. Many of these studies were designed to test the feature-based organization model by examining brain activity elicited by living things (mainly animals) and man-made manipulable objects (mainly tools) in healthy participants (for a review, see Martin, 2001; Martin and Chao, 2001; Martin et al., 2000). In some of these investigations, living things evoked more activity than man-made objects in the fusiform gyrus (Perani et al., 1995, 1999; Thompson-Schill et al., 1999), an area that is part of the ventral object-processing stream and is thought to mediate access to representations of visual features of objects¹. On the other hand, man-made objects evoked category-specific activity in a separate network of brain regions including the left posterior middle temporal gyrus (Chao et al., 1999, 2002; Martin et al., 1996; Moore and Price, 1999; Mummery et al., 1998; Perani et al., 1999), which has been reported to be activated during the generation of action words (Fiez et al., 1996; Martin et al., 1995; Wise et al., 1991), and to moving images of tools (Beauchamp et al., 2002, 2003). Man-made objects also have been reported to produce increased activity in the left premotor and left posterior parietal cortical regions (Chao and Martin, 2000; Chao et al., 2002; Grabowski et al., 1998; Grafton et al., 1997; Martin et al., 1996) that previously have been linked to motor-control and motor-imagery (Binkofski et al., 1998; Decety et al., 1994; Grafton et al., 1996; Stephan et al., 1995).

The above studies have been interpreted as supporting feature-based organization of semantic memory. However, Devlin et al. (2002), Tyler and Moss (2001) have noted that the precise locations of the category-specific activations are not entirely consistent across studies, and frequently do not converge with the brain-damage deficit data. In addition, their studies have failed to replicate the category-specific effects reported above (Devlin et al., 2002; Pilgrim et al., 2002; Tyler et al., 2003a). Therefore, these authors have argued that a single semantic system account is a more parsimonious explanation for the extant data.

4. Event related potentials and the current study

In the present investigation, we aimed to shed further light on how semantic knowledge is organized in the brain by using event-related potentials (ERPs)—a technique that records electrophysiological brain potentials time-locked to the stimuli of interest. Unlike PET and fMRI, ERPs have a temporal resolution of milliseconds, which can be used to detect processing within the specific time-window that is known to encompass online semantic processing (Barrett and Rugg, 1990; Coles and Rugg, 1995; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999). Moreover, the scalp topography of ERPs can be employed to dissociate neurocognitive processes that overlap in time (Holcomb et al., 1999; Kutas, 1993). As a result of this multidimensional nature, ERP data can help to distinguish subcomponents of semantic analysis (see Barrett and Rugg, 1990; Federmeier and Kutas, 2001; Ganis et al., 1996; Hamm et al., 2002; Holcomb et al., 1999; Holcomb and McPherson, 1994; Kellenbach et al., 2002; Kounios and Holcomb, 1994; McPherson and Holcomb, 1999; Sitnikova, 2003; Sitnikova et al., 2003; West and Holcomb, 2000, 2002).

In the present study, we used ERPs to determine whether non-identical neurocognitive mechanisms mediate the semantic analysis of objects from animal and tool categories. We focused on the subprocesses reflected by the N400 ERP waveform—a negativity evoked between 200 and 600 ms after presentation of a meaningful stimulus. This waveform was initially characterized as being sensitive to semantic variables in the language domain (Bentin et al., 1985; Holcomb, 1988; Kutas and Hillyard, 1980, 1984) but has also been described in association with processing pictorial (Barrett and Rugg, 1990; Federmeier and Kutas, 2001; Ganis et al., 1996; Hamm et al., 2002; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999; Sitnikova et al., 2003; West and Holcomb, 2002) and other types of stimuli (Van Petten and Rheinfelder, 1995). Perhaps the most widely accepted account of the N400 posits that this ERP component reflects mental effort involved in the analysis of meaning (Brown and Hagoort, 1993; Holcomb, 1993).

Previous studies have documented that the N400 evoked by verbal stimuli is characterized by a parietal-occipital scalp topography (Friederici et al., 1993; Hagoort and Brown, 2000; Holcomb et al., 1999; Kutas and Van Petten, 1994; van Berkum et al., 1999), while the negativities elicited by pictures (Barrett and Rugg, 1990; Hamm et al., 2002; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999; West and Holcomb, 2002) and silent videos (Sitnikova, 2003; Sitnikova et al., 2003) are typically distributed over more anterior electrode sites. Moreover, the N400 elicited by concrete words has a more anterior scalp topography than that evoked by abstract words (Holcomb et al., 1999; Kellenbach et al., 2002; Kounios and Holcomb, 1994; West and Holcomb, 2000), whereas words denoting manipulable objects and human actions elicit a posterior, slightly left-lateralized N400 (Kellenbach et al., 2002). Differences in scalp topography between ERP components are generally interpreted as indicating non-identical underlying neural sources (e.g., Holcomb et al., 1999; Kutas, 1993). Therefore, these results suggest that the N400 is comprised of several separable late negativities that may reflect processing within distinct feature-specific semantic neural networks (see Holcomb et al., 1999; Holcomb and McPherson, 1994; Kellenbach et al., 2002; McPherson and Holcomb, 1999; Sitnikova et al., 2003; West and Holcomb, 2002). In particular, the more anterior negativities elicited by concrete words and visual images may reflect activation of semantic representations of objects’ visual attributes². In contrast, the posterior N400 may be associated with activation of brain regions selectively devoted to processing representations of verbal and possibly other types of knowledge.

It is noteworthy that in several studies using picture stimuli, the ERP negativity observed in the earlier semantic-processing epoch (between approximately 200–350 ms) had a relatively focal scalp topography limited to more anterior electrode sites. This ERP component has been labeled the N300 (Barrett and Rugg, 1990; Hamm et al., 2002; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999). One interpretation of the functional significance of the N300 is that it reflects the immediate, direct access to visual-feature semantic representations of objects (e.g., McPherson and Holcomb, 1999).

This ERP literature informs our current hypotheses. If, as predicted by the feature-based organization model, identification of animals (relative to tools) involves increased activity in the visual-feature semantic system (Martin and Chao, 2001; Warrington and McCarthy, 1987), animals might be expected to evoke a relatively greater anterior N400 effect than tools. Furthermore, if identification of tools relies predominantly on the retrieval of their functional representations (e.g., coding their typical associated motion—Martin and Chao, 2001), then they might be expected to evoke greater posterior N400 activity than animals.

In the current study, participants viewed pictures of animals and tools. Therefore, we expected that the anterior category-related ERP differences due to processing of visual object properties, predicted by the feature-based organization model, should occur in the earlier, N300 time-window. On the other hand, the posterior negativity effect due to processing functional properties was hypothesized to develop somewhat later, as pictures presumably do not directly access non-visual, functional semantic representations (at the posterior electrode sites, the late negativity to pictures was previously reported to peak at around 450 ms after stimulus onset, and was labeled the N450, see Barrett and Rugg, 1990; McPherson and Holcomb, 1999; Paivio, 1986).

5. Methods

5.1. Construction of materials and normative studies

Stimuli in the present study were full-color drawings of animals and tools obtained from ‘Art Explosion’ CD-ROM package (Nova Development Corporation). The animal category included mammals, reptiles, marine creatures, and insects. The tool category included a variety of man-made objects used to accomplish a specific task (e.g., a comb, a broom, an axe). Prior to the ERP experiment, we conducted two normative studies (see Table 1 for results) to ensure that object naming task would be of a comparable difficulty both for animals and tools selected for our ERP study.

Table 1.

Parameters that were matched between animal and tool pictures

Parameter	Object category	Mean	Standard deviation	t-value
Naming accuracy (%)	Animals	88.00	14.50	−0.424
	Tools	86.67	16.84
Name-verification accuracy (%)	Animals	98.00	3.20	−1.201
	Tools	97.13	4.03
Name-verification reaction time (ms)	Animals	568.34	43.84	1.009
	Tools	577.99	49.79
Picture familiarity rating (0—unfamiliar, 3—familiar)	Animals	2.72	0.54	0.807
	Tools	2.80	0.45
Name printed word familiarity (1—unfamiliar, 7—familiar)	Animals	5.06	0.57	−0.064
	Tools	5.05	0.36
Name word frequency	Animals	11.03	22.43	0.092
	Tools	11.49	19.01
Number of letters in the name	Animals	6.16	2.11	1.595
	Tools	6.92	2.62
Number of syllables in the name	Animals	2.10	0.91	−0.552
	Tools	2.00	0.90

Note: in all comparisons, degrees of freedom = 98; p > 0.1. Naming accuracy and picture familiarity data were obtained from a different sample of participants than name-verification accuracy and reaction time data. We were able to obtain name printed word familiarity from the online MRC psycholinguistic database (http://www.itd.clrc.ac.uk) only for 25 animals and 25 tools. Similarly, name word frequency norms were obtained from Kuèera and Francis (1967) and the MRC psycholinguistic database only for 35 animals and 35 tools. All ERP and behavioral analyses were repeated for these subsets of stimuli, and the obtained results essentially were not different from the results with the full stimuli set.

5.1.1. Normative study 1: naming accuracy and picture familiarity

Twelve Tufts University undergraduates (six females, six males; all right-handed native English speakers) who did not participate in the ERP experiment named pictures of 90 animals and 90 tools. Presentation procedure was identical to the ERP study (see below). Participants were told to provide a name that they would use for an object in real life. An experimenter recorded on-line each response and whether the name was produced immediately. Off-line, we determined the most frequently given name for each object (e.g., butterfly, dog, mallet, paintbrush), and then scored each trial as accurate if such correct name was produced immediately, and as “a miss” if such name was not produced or if the response was given after hesitation. The naming accuracy for each picture was determined as a percentage of correct responses across all participants. The only difference from the ERP study was that in this pretest, after naming each picture, participants also rated on a 0–3 scale the item’s familiarity (answered the question: ‘How familiar are you with the item?’). The picture familiarity rating for each item was determined as an average of ratings given by all participants.

5.1.2. Normative study 2: name-verification

A new group of sixteen Tufts University undergraduates (eight females, eight males; all right-handed native English speakers) who did not participate in the ERP experiment performed a speeded name-verification task. Pictures of animals and tools were preceded with a SOA of 600 ms either by a matching object name (chosen based on the Normative study 1) or a mismatching name from the same object category (e.g., ‘spider’–‘elephant’, or ‘rake’–‘screwdriver’). Participants were instructed to decide whether each object matched the preceding name, and signal their decision as quickly as possible using a response box. For each picture, the name-verification accuracy was determined as a percentage of correct responses across all participants, and the name-verification reaction time was determined as an average of reaction times across all participants.

5.1.3. Selected materials

Based on these normative studies, we selected 50 animal and 50 tool pictures (see Fig. 1, for examples, Appendix A for a list of object names, and http://neurocog.psy.tufts.edu/anitool1 for a full set of object pictures) that were matched for their naming accuracy, name-verification accuracy, name-verification reaction time, and picture familiarity. Moreover, the selected animals and tools had names of similar word length (number of letters and number of syllables). Table 1 shows means and standard deviations for the matched parameters in this final set of stimuli. Finally, we also considered name word frequency and name word familiarity of the selected items, see the Note below Table 1.

Fig. 1.

Examples of animal (A) and tool (B) pictures.

5.2. Participants

Fifteen (seven female, eight male) right-handed undergraduate students from Tufts University aged 18–23 (mean age 19.5) took part in this study. All participants were native speakers of English and had normal or corrected-to-normal vision.

5.3. Procedure

Each participant sat in a comfortable chair in a room equipped with a video camera and a microphone connected via a close circuit to a TV set placed in the experimenters’ room. The stimuli were presented to participants on a computer monitor and were centered on a white background. Fifty animal pictures and 50 tool pictures were presented in pseudo-random order (strongly associated items, such as a spoon and a fork, were separated by at least 20 trials). Each trial began with a small green circle (subtending ~1° of visual angle) and participants were asked to press a “GO” button to trigger, 900 ms later, the presentation of an animal or tool picture (subtending ~5° of visual angle). After being displayed for 500 ms, the picture was replaced by a blank screen for 700 ms, and then the green circle re-appeared on the screen. Participants were instructed to name each item out loud as quickly as possible after seeing the green circle to reappear on the monitor. In this way, the naming response was delayed until after the ERP recording epoch to avoid muscle-contraction artifacts resulting from speech articulation. Participants were instructed to avoid eye-movements and to keep their eyes on the center of the monitor throughout each trial. Participants proceeded from trial to trial at their own pace. Each person was given 12 practice trials prior to the ERP experiment.

Overt naming was used so that we could subsequently exclude ERPs to items that were not correctly and rapidly named from analyses. This was achieved via on-line coding of subjects’ naming accuracy (a response was scored as accurate if the correct name – chosen based on the Normative study 1 described above – was given without any hesitation, immediately after the re-appearance of the green circle).

5.4. Electrophysiological recording

The electroencephalogram (bandpass, 0.01–40 Hz, 6 dB cutoffs; sampling rate, 200 Hz) was recorded from 61 tin electrodes held in place on the scalp by an elastic cap (Electro-Cap International, Eaton, OH), infra-ocular electrodes located below each eye (IO1/IO2), and an electrode positioned over the right mastoid bone. All of these active electrodes were referenced to an electrode placed on the left mastoid. The scalp sites (see diagram in Fig. 2) included 17 standard International 10–20 System locations: FP1, FP2, FPz, F7, F8, Fz, C3, C4, Cz, T3, T4, T5, T6, Pz, O1, O2, and Oz. Other 24 sites were placed at the extended 10–20 system locations: AF7, AF8, FT7, FT8, TP7, TP8, PO7, PO8, FC5, FC3, FC1, FC2, FC4, FC6, C5, C1, C2, C6, CP5, CP3, CP1, CP2, CP4, and CP6. Finally, 20 additional locations included: AF3, AF1, AF2, and AF4 (placed at increments of 20% of the distance between AF7 and AF8); F5, F1, F2, and F6 (20% increments of F7–F8 distance); P5, P1, P2 and P6 (20% increments of T5–T6 distance); PO3, PO1, PO2, and PO4 (20% increments of PO7–PO8 distance); F9 and F10 (at the outer canthi of eyes); and T9 and T10 (at the upper mastoid bones).

Fig. 2.

Schematic locations of parasagittal columns of scalp electrodes: (1) midline, (2) inner-medial, (3) outer-medial, (4) inner-lateral, and (5) outer-lateral (inferior columns, including IO1/IO2, F9/F10, and T9/T10 sites, are not shown).

5.5. Data analysis

For each participant, mean ERPs (epoch length = 100 ms before picture presentation to 1187 ms after picture presentation) were formed off-line by selectively averaging across trials from each condition. Following this, the mean ERPs for each participant were re-referenced to an average of the left and right mastoids, and the group average ERPs were created.

We were careful to include only the trials free of ocular artifacts (trials with activity exceeding 60 μV below eyes, above eyes, or at the eye canthi were excluded; trials with voltage difference exceeding 40 μV between the channels below and above each eye were excluded; each ERP trial of each participant was also visually inspected to ensure that there were no signs of ocular artifacts, i.e., there was no evidence of reversal in polarity of ERPs between the electrode sites positioned immediately below and above each eye; percent of excluded trials across all participants was 7.47% for animals and 6.27% for tools; t = 0.908, p > 0.1). Furthermore, the trials were included only if the participant was able to name the object immediately after the speaking prompt (the green circle) was shown on the screen. All in all, the group average ERP waveforms were created based on 77.33% of trials for animals and 78.40% of trials for tools (t = −0.491, p > 0.1).

To determine whether there were differences in early sensory processing between animal and tool pictures, we compared early (P1/N1) ERP waveforms. Latencies of these sensory potentials were quantified by measuring the timing at PO7 & PO8 sites of the largest positive peak within 150 ms after picture presentation (P1) and the largest negative peak between 150 and 200 ms after picture presentation (posterior N1); and at F7 & F8 sites of the largest negative peak within 150 ms after picture presentation (anterior N1). These latency data as well as average ERPs at these electrodes and time-windows (measured relative to the 100-ms baseline prior to picture presentation) were entered into two three-way repeated-measures analyses of variance (ANOVAs) examining category-related differences in the early potentials’ time-course and amplitude, respectively. Each ANOVA included factors of Object Category (animals and tools), ERP Potential (P1, posterior N1, and anterior N1), and Hemisphere (left and right).

To examine semantic processing of animal and tool pictures, we calculated the mean ERP amplitudes (relative to the 100-ms baseline prior to picture presentation) within 200–300 ms and 300–600 ms time-windows after picture presentation. These time-windows roughly correspond to the time-windows previously used to quantify the earlier anterior negativity (the N300) and the later more posterior negativity (the N450) evoked by picture stimuli. For each of these time-windows, six ANOVAs for repeated measures were conducted in order to examine parasagittal columns of scalp electrodes along the anterior–posterior axis of the head (see Fig. 2). All analyses had an Object Category factor (animals and tools) and all but midline analyses had a Hemisphere factor (left and right). The midline analysis had five levels of Electrode Site (FPz, Fz, Cz, Pz, Oz). The inner-medial analysis had five levels of Electrode Site (AF1/AF2, FC1/FC2, C1/C1, CP1/CP2, PO1/PO2). The outer-medial analysis had seven levels of Electrode Site (FP1/FP2, F1/F2, FC3/FC4, C3/C4, CP3/CP4, P1/P2, O1/O2). The inner-lateral analysis had seven levels of Electrode Site (AF3/AF4, F5/F6, FC5/FC6, C5/C6, CP5/CP6, P5/P6, PO3/PO4). The outer-lateral analysis had seven levels of Electrode Site (AF7/AF8, F7/F8, FT7/FT8, T3/T4, TP7/TP8, T5/T6, PO7/PO8). The inferior analysis had three levels of Electrode Site (IO1/IO2, F9/F10, T9/T10). The Geisser–Greenhouse correction was applied to all repeated measures with more than one degree of freedom (Geisser and Greenhouse, 1959).

It is controversial whether data normalization improves our ability to distinguish between underlying neural generators of ERPs based on differences in scalp topography (see McCarthy and Wood, 1985; Urbach and Kutas, 2002). Therefore, we report all instances where the raw data showed significant interactions between the Object Category factor and any of the topographic variables (i.e., Hemisphere and/or Electrode Site). In addition, we also report the instances where the latter interactions remained significant after voltage values were normalized (using z-scores) within each level of the Object Category variable.

To determine whether the scalp topography of the ERP effect between animals and tools changed over time (from the N300 to N450 epoch), we conducted six additional repeated-measures ANOVAs. In these ANOVAs, the dependent variable was the difference in voltage between ERPs elicited by animals and tools, averaged across 200–300 ms (for the N300 epoch) and across 400–500 ms (for the N450 epoch)³. The independent variables were as described above, but, in place of the Object Category factor, we included a Time-Window factor (N300 and N450).

6. Results

6.1. Behavioral data

Participants were able to name the pictures rather accurately. The average rate of correct naming responses that were given immediately after the speaking prompt was 83.47% for animals and 83.20% for tools. The accuracy rates were not significantly different between animals and tools (t = 0.111, p > 0.1).

6.2. Event-related potential data

For pictures that were named correctly immediately after the speaking prompt, ERPs averaged across participants are shown in Fig. 3. In addition, Fig. 4A displays enlarged plots of these ERPs at two representative channels. All pictures elicited clear sensory/perceptual components. At more anterior sites, a negative-going potential peaking at around 120 ms (N1) was followed by a positivity that was maximal at approximately 190 ms (P2). At more posterior sites, the configuration was somewhat different, and included a positivity with a peak at about 110 ms (P1), a negativity peaking at around 160 ms (N1), and a positivity with a peak at approximately 220 ms (P2). This series of early ERPs was followed by negative-going late components peaking at approximately 250 and 450 ms (N300 and N450, respectively). Particularly at more posterior regions, these late negative-going potentials overlapped with a prominent late positivity, with a maximum at about 325 ms.

Fig. 3.

ERPs time-locked to the presentation of animal and tool pictures averaged across all participants.

Fig. 4.

ERPs time-locked to the presentation of pictures shown at two representative electrode sites (A), and the corresponding difference waves obtained by subtracting the ERPs to tool pictures from the ERPs to animal pictures (B). Head diagram in the center shows approximate locations of the shown electrode sites.

Within the N300/N450 time-window, the ERPs were more negative to animals than to tools at frontal–central (e.g., FC1 and FC2) and anterior–inferior (e.g., IO1 and IO2) electrode sites. In contrast, the effect of an opposite polarity with the ERPs being more negative to tools than to animals was evident primarily at more posterior electrodes (e.g., PO3 and PO4). Fig. 4B demonstrates these category-related differences at the representative frontal–central and posterior electrode sites. In addition, Fig. 5 illustrates scalp topography of these category-related differences.

Fig. 5.

Voltage maps (created using the EMSE Data Editor software; source signal imaging, San Diego, CA) of the category-related ERP differences within the semantic-processing epoch. These maps were derived from the difference waves obtained by subtracting the ERPs to tool pictures from the ERPs to animal pictures: the maps show the data averaged across four consecutive 100-ms-long time-windows. Black contours demarcate change in voltage of 0.2 μV. Scalp regions where animals evoked more negative ERPs than tools are shown in blue with dotted contours, and scalp regions where tools evoked more negative ERPs than animals are shown in red with solid contours.

6.2.1. 0–150-ms and 150–200-ms (P1/N1) epochs

There were no statistically significant differences between animals and tools in either latency or amplitude of the ERP potentials in these time-windows (in all comparisons, F < 2.000, p > 0.1).

6.2.2. 200–300-ms (N300) epoch

During this epoch, ERPs were more negative to animals than tools over frontal–central and anterior–inferior scalp regions. A reversed pattern, with ERPs to tools being more negative than ERPs to animals, was observed over occipital and posterior temporal–parietal areas, where the effect was strongly left-lateralized (see Figs. 3–5). This was manifest by significant Object Category by Electrode Site interactions in midline, inner-lateral, and inferior analyses and Object Category by Hemisphere by Electrode Site interactions in inner-medial, outer-medial, and outer-lateral analyses (see Table 2). With normalized data, all analyses except that at the inner-lateral electrode column yielded the same findings as with the raw data (see Table 2, last column).

Table 2.

Higher-order interactions obtained in the ANOVAs examining parasagittal columns of scalp electrodes

Analysis	Interaction type	Degrees of freedom	F-value	F-value (z-scores)
200–300 ms (N300)
Midline	C × E	4,60	7.417**	3.695*
Inner-medial	C × H × E	4,60	7.278**	7.541**
Outer-medial	C × H × E	6,90	3.877*	4.438*
Inner-lateral	C × E	6,90	8.045**
Outer-lateral	C × H × E	6,90	7.007**	6.114**
Inferior	C × E	2,30	43.070**	26.601**
300–600 ms (N400)
Inner-medial	C × H × E	4,60	5.321**	4.861**
Outer-medial	C × H × E	6,90	3.465*	3.385*
Inner-lateral	C × H	6,90	5.292*	4.772*
Outer-lateral	C × H × E	6,90	5.852**	4.400*
Inferior	C × H × E	2,30	9.762**	3.500*
200–600 ms (entire semantic epoch)
Midline	C × E	4,60	3.940*
Inner-medial	C × H × E	4,60	6.854**	5.398**
Outer-medial	C × H × E	6,90	4.086*	4.012*
Inner-lateral	C × H × E	6,90	3.398*
Outer-lateral	C × H × E	6,90	6.791**	5.306**
Inferior	C × H × E	2,30	8.771**	3.812*

C: Object Category factor; E: Electrode Site factor; H: Hemisphere factor.

^*p < 0.05.

^**p < 0.01.

Planned comparisons at each electrode site showed that the increase in the anterior negativity to animals relative to tools was significant at frontal (F1, F2, & Fz), frontal–central (FC1, FC2, FC3, FC4, FC5, & FC6), frontal–temporal (FT7 & FT8), central (C1, C2, & Cz), and anterior–inferior sites (F9, IO1, & IO2). The posterior differences in negativity to tools relative to animals were significant at occipital (O1 & O2), parietal–occipital (PO1, PO3, PO4, PO7, & PO8) and temporal (T5) sites.

6.2.3. 300–600-ms (N450) epoch

In this time-window, the increased negativity to animals compared to tools was present only at anterior–inferior electrode sites and became slightly left-lateralized (see Fig. 3) as indicated by a significant Object Category by Hemisphere by Electrode Site interaction at inferior electrode columns (see Table 2). On the contrary, the enhanced negativity to tools relative to animals became more widespread, extending to more anterior parietal sites. This effect peaked over the left hemisphere as well (see Figs. 3 and 5; also see Fig. 4 showing this effect at the representative posterior electrode site). Significant interactions were obtained between Object Category, Hemisphere, and Electrode Site factors in inner-medial, outer-medial, and outer-lateral analyses, and between Object Category and Hemisphere factors in the inner-lateral analysis (see Table 2). All of the interactions found with the raw data were replicated in the analyses on the normalized data (see Table 2, last column).

Planned comparisons at each electrode site demonstrated that the anterior–inferior negativity effect to animals relative to tools was significant at the infra-ocular electrodes (IO1 & IO2). The increase in the posterior negativity to tools compared to animals was significant at occipital (O1 & O2), parietal–occipital (PO1, PO2, PO3, PO4, & PO7), parietal (P1, P2, P5 & P6), central-parietal (CP5), and temporal (T5, T9, & TP7) sites.

6.2.4. Comparison between the N300 and N450 epochs

The scalp topography of the ERP differences between animals and tools changed from the earlier (N300) to later (N450) epoch of semantic processing (see Fig. 5), as was determined by a direct comparison between the category-related ERP differences in these epochs. Significant Time-Window by Electrode Site and/or Time-Window by Hemisphere by Electrode Site interactions were obtained in all analyses that used the raw data (see Table 3). Time-Window by Electrode Site interactions remained significant after the data were normalized in the midline, inner-medial, outer-medial, and inferior analyses (see Table 3, last column).

Table 3.

Higher-order interactions obtained in the ANOVAs examining differences in parasagittal columns of scalp electrodes between 200–300 ms and 400–500 ms time-windows

Analysis	Interaction type	Degrees of freedom	F-value	F-value (z-scores)
Midline	T × E	4,60	53.999**	4.664*
Inner-medial	T × E	4,60	5.359**	5.348*
Inner-medial	T × H × E	4,60	20.651**
Outer-medial	T × E	6,90	55.400**	5.050*
Outer-medial	T × H × E	6,90	21.826**
Inner-lateral	T × H × E	6,90	17.412**
Outer-lateral	T × H × E	6,90	18.856**
Inferior	T × E	2,30	37.492**	4.059**
Inferior	T × H × E	2,30	55.992**

H: Hemisphere factor; E: Electrode Site factor; T: Time-Window factor. Note: these analyses were performed on voltage differences between animals and tools.

^*p < 0.05.

^**p < 0.01.

7. Discussion

The ERPs elicited by pictures of animals and tools had distinct spatial distributions across the scalp within a time-window known to index semantic processing (Barrett and Rugg, 1990; Coles and Rugg, 1995; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999). At the earlier part of this time-window (200–300 ms: the N300 epoch), animals elicited a more negative waveform than tools over frontal–central and anterior–inferior electrode sites. In contrast, tools elicited a more negative waveform than animals over occipital, posterior–temporal and posterior–parietal sites, primarily over the left hemisphere. At the later part of the semantic-processing time-window (300–600 ms: the N450 epoch), ERPs were more negative to animals than to tools only at the anterior–inferior sites and the increased posterior negativity elicited to tools, compared to animals, spread toward more anterior parietal scalp areas.

This combination of category-related topographic and time-course differences suggests that semantic processing of animals and tools does not occur in identical brain regions, providing evidence against a unitary semantic system theory (Anderson and Bower, 1973; Devlin et al., 2002; Gernsbacher, 1985; Kroll and Potter, 1984; Pylyshyn, 1980; Tyler and Moss, 2001; Tyler et al., 2000, 2003b). It seems unlikely that these results could be accounted for by category-related differences in lower-level perceptual features of the stimuli (e.g., visual complexity) because differences such as these would probably have affected early ERP components such as the P1 or N1. We found no significant differences between pictures of animals and tools prior to 200 ms. Below we consider our findings in further detail, showing how they support a feature-based model of neuroanatomical organization of semantic knowledge.

7.1. Topographic differences between ERPs to animals and tools

7.1.1. The anterior negativity to animals (versus tools)

The frontal–central distribution of the N300 to animals (relative to tools) resembled a negativity effect that has previously been documented in association with pictures relative to words (Federmeier and Kutas, 2001; Ganis et al., 1996) and in association with concrete relative to abstract words (Holcomb et al., 1999; Kellenbach et al., 2002; Kounios and Holcomb, 1994; West and Holcomb, 2000). Topographic similarities between ERP effects are usually interpreted as suggesting that their neural generators are either identical or located in close proximity to one another (Holcomb et al., 1999; Kutas, 1993). It seems unlikely that separate brain regions, each specializing in processing of animals, pictures, or concrete words, would be located next to each other simply by chance. Therefore, we take this result as evidence against a neuroanatomical organization at the level of whole objects (as was proposed by Caramazza and Shelton, 1998). A hypothesis that the relative increase in frontal–central negativity reflects processing of visual object characteristics is more plausible, given the particular importance of visual features in the identification of all of these stimuli in comparison with their respective controls: pictures in comparison with words, concrete in comparison with abstract words, and animals in comparison with tolls (see Farah and McClelland, 1991; Holcomb et al., 1999; Martin et al., 2000; Paivio, 1986; Warrington and McCarthy, 1987). Similar logic also discounts the possibility that the frontal–central negativity simply reflects increased difficulty in identifying animals relative to tools due to a larger number of shared, inter-correlated features associated with animals (e.g., Moss et al., 1998; Tyler et al., 2003b). A similar negativity was previously observed to pictures relative to words even when the pictures and words referred to the same identical set of concepts (e.g., Federmeier and Kutas, 2001⁴; Ganis et al., 1996). Thus, the finding that the frontal–central N300 was larger to animals than tools fits well a feature-based model of semantic memory organization in the brain.

It is intriguing that the increased negativity to animals (compared to tools) was also observed between 200 and 600 ms at anterior–inferior scalp regions, peaking over the infra-ocular electrode sites. Notwithstanding their relatively remote location from the brain, these infra-ocular electrodes were sensitive to the electroencephalographic activity, because the data at these electrodes (as at all other active leads) were collected with the mastoid reference. One possibility is that the anterior–inferior effect between animals and tools was generated in anterior temporal and/or ventral prefrontal cortical regions⁵. In neuroimaging literature, these brain areas have been reported to display increased activity to living relative to non-living items (Leube et al., 2001; Moore and Price, 1999; Mummery et al., 1996) and were proposed to be involved in the retrieval of knowledge about affective valence associated with animal concepts (Leube et al., 2001) and/or in the selection of specific object identity (McRae et al., 1997; Moore and Price, 1999; Moss et al., 2005; Tyler et al., 2004; selecting unique object identity is necessary for naming, and this process might be more demanding for animals than tools because animals have more shared, inter-correlated visual/semantic features than tools).

7.1.2. The posterior negativity to tools (versus animals)

Tools (in comparison with animals) elicited a negativity between 200 and 600 ms that had a posterior scalp distribution. Of note, however, this negativity was markedly different in its topography from the posterior N400 evoked by verbal stimuli, abstract words in particular. Theverbal N400 is characterized by a widespread parietal–occipital distribution with a dorsal maximum (Holcomb et al., 1999; Kutas and Van Petten, 1994). In contrast, the present effect to tools was restricted to more posterior parietal–occipital–temporal regions and peaked over left-lateralized inferior electrodes.

One explanation for these topographic dissimilarities could be that the current negativity and the verbal effect have distinct neural sources. The posterior left-lateralized distribution of the negativity to tools (relative to animals) is consistent with a neural generator in the lateral portion of the left posterior temporal cortex, which may mediate representations of object motion (Chao et al., 1999; Chao and Martin, 2000; Martin, 2001; Martin and Chao, 2001; Martin et al., 2000, 1996). Thus, we tentatively propose that this effect may reflect access to knowledge about objects’ typical motion – functional information that is critical for tool identification. This interpretation is further supported by a previous ERP study that reported an enhanced posterior late negativity peaking over the left hemisphere to words referring to manipulable objects or human actions (relative to words not associated with human actions; Kellenbach et al., 2002).

On the other hand, the topography of the present negativity to tools (compared to animals) is not inconsistent with the activation of the same neural generator as in the verbal paradigms. The ERP negativity resulting from such activation could be cancelled out at the parietal electrodes by the overlapping anterior negativity effect in the opposite direction (i.e., larger negativity to animals than tools)⁶. Thus, we cannot exclude the possibility that identifying tools was more taxing on the verbal functional system than identifying animals (see Riddoch and Humphreys, 1987; Tyler and Moss, 1997).

Regardless of whether the posterior effect to tools (versus animals) indexed differences in verbal processing demands or was mediated primarily by accessing the knowledge of object motion, this finding is consistent with a feature-based model of neuroanatomical organization of semantic memory. It suggests that tools, primarily identifiable by their functional properties, activated brain areas storing semantic representations of these object features more than animals.

7.2. Time-course differences in processing animals and tools

The topography of category-based ERP effects changed from the earlier to later stages of semantic processing, providing additional evidence against the “whole object” organization model proposed by Caramazza and Shelton (1998). The latter framework only predicts spatial differences in the processing of animals and tools (but no distinctions in the time-course of these category-specific processes), which should have led to ERP effects with a scalp distribution that is constant over time. On the contrary, a feature-based organization model can explain such temporal changes in topography as arising from differences in accessibility of different feature-specific semantic systems (McPherson and Holcomb, 1999; Paivio, 1986; Thompson-Schill et al., 1999). In previous studies, the relatively early appearance of an anterior negativity to pictures of individual objects (the N300) has been interpreted as reflecting a direct activation of visual-feature semantic representations⁷, while the slightly later appearance of a posterior negativity (the N400 or N450) has been hypothesized to reflect the extra time needed for the activation to spread from visual onto other types of representations (Barrett and Rugg, 1990; Holcomb and McPherson, 1994; McPherson and Holcomb, 1999). In the present study, the increased frontal–central negativity to animals (compared to tools) was observed during the earlier (N300) epoch, suggesting that direct activation of the semantic system storing objects’ visual features was enhanced in response to pictures of animals.

Interestingly, the posterior negativity effect to tools (relative to animals) also started in the N300 epoch. One interpretation of this early posterior effect could be that tools might be different from many other object categories in that their form directly (and quickly) accesses functional semantic representations, possibly due to over-learned associations between these two types of attributes. While animals and certain other objects can be identified exclusively based on an invariant relationship between their visual form and identity, the recognition of tools might rely on an invariant association between their function and identity (as there is no clear correspondence between tools’ visual features and identity; see Farah and McClelland, 1991; Martin et al., 2000). This explanation is supported by our observation that the times to recognize the tool and animal stimuli used in the current study were very similar (see Section 5.1.3). The shift in the scalp distribution of the posterior negativity effect from left-lateralized inferior sites in the N300 epoch toward more dorsal parietal sites in the later (N450) time-window could be due to the sequential activation of different types of functional representations. Tools might have first accessed representations of objects’ typical motion, argued to be stored within the left posterior temporal cortex (Martin, 2001; Martin and Chao, 2001; Martin et al., 2000), and later activated verbal representations, associated with the dorsal parietal–occipital ERP negativity (Holcomb et al., 1999; Kutas and Van Petten, 1994).

Alternatively, the posterior category-related effect in the ERPs between 200 and 300 ms might reflect differences in perceptual processing between animal and tool pictures. A similar effect has been previously reported in a subset of the ERP studies that used picture stimuli (Federmeier and Kutas, 2001; Hamm et al., 2002; McPherson and Holcomb, 1999). This effect has been argued to originate from a modulation of a posterior P2 positivity that is sensitive to factors such as perceptual familiarity but not to changes in semantic context⁸ (Federmeier and Kutas, 2001; McPherson and Holcomb, 1999). In the present study, animals might have evoked a larger posterior P2 than tools because they have more shared, inter-correlated visual features (McRae et al., 1997; Moore and Price, 1999), and therefore, identification of animals might require more intense perceptual analysis than identification of tools.

8. Conclusions

To summarize, our findings suggest that the semantic processing of animal and tool pictures is mediated by nonidentical brain regions, and provide further evidence for a feature-based organization of semantic knowledge in the brain. The high temporal resolution of ERPs allowed us to demonstrate that differences in neural processing between animals and tools occurred within a time-window of semantic analysis and to obtain finer-grained information about the precise time-course of the category-related modulation of this neurocognitive processing.

Appendix A. Names of objects used in the study

Animals	Tools
Alligator	Anvil
Armadillo	Axe
Bear	Binoculars
Beaver	Broom
Bee	Chopsticks
Boar	Clamp
Butterfly	Comb
Camel	Compass
Cat	Corkscrew
Cheetah	Divider
Cow	Drill
Crow	Dropper
Deer	Dustpan
Dog	File
Dolphin	Flashlight
Donkey	Fork
Dragonfly	Gavel
Duck	Hairbrush
Eagle	Hairdryer
Eel	Hammer
Elephant	Hoe
Flamingo	Iron
Fly	Knife
Giraffe	Ladder
Grasshopper	Ladle
Horse	Mallet
Kangaroo	Microscope
Koala	Mixer
Llama	Mower
Lobster	Paintbrush
Millipede	Pencil Sharpener
Moose	Pitchfork
Octopus	Pliers
Ostrich	Plunger
Owl	Pocket knife
Parakeet	Pump
Pelican	Rake
Penguin	Ruler
Platypus	Scales
Raccoon	Scissors
Rhinoceros	Screwdriver
Scorpion	Shovel
Sheep	Spoon
Snake	Stapler
Spider	Stethoscope
Squid	Tweezers
Squirrel	Typewriter
Stingray	Weeder
Tiger	Wheelbarrow
Walrus	Wrench