Abstract
Converging evidence supports the existence of functionally and neuroanatomically distinct taxonomic (similarity-based; e.g., hammer-screwdriver) and thematic (event-based; e.g., hammer-nail) semantic systems. Processing of thematic relations between objects has been shown to selectively recruit the left posterior temporoparietal cortex. Similar posterior regions have been also been shown to be critical for knowledge of relationships between actions and manipulable human-made objects (artifacts). Based on the hypothesis that thematic relationships for artifacts are based, at least in part, on action relationships, we assessed the prediction that the same regions of the left posterior temporoparietal cortex would be critical for conceptual processing of artifact-related actions and thematic relations for artifacts. To test this hypothesis, we evaluated processing of taxonomic and thematic relations for artifact and natural objects as well as artifact action knowledge (gesture recognition) abilities in a large sample of 48 stroke patients with a range of lesion foci in the left hemisphere. Like control participants, patients identified thematic relations faster than taxonomic relations for artifacts, whereas they identified taxonomic relations faster than thematic relations for natural objects. Moreover, response times for identifying thematic relations for artifacts selectively predicted performance in gesture recognition. Whole brain Voxel Based Lesion-Symptom Mapping (VLSM) analyses and Region of Interest (ROI) regression analyses further demonstrated that lesions to the left posterior temporal cortex, overlapping with LTO and visual motion area hMT+, were associated both with relatively slower response times in identifying thematic relations for artifacts and poorer artifact action knowledge in patients. These findings provide novel insights into the functional role of left posterior temporal cortex in thematic knowledge, and suggest that the close association between thematic relations for artifacts and action representations may reflect their common dependence on visual motion and manipulation information.
Keywords: thematic knowledge, action knowledge, artifact concepts, left posterior temporal cortex, Voxel Based Lesion-Symptom Mapping
1. Introduction
Many influential models of the organization of semantic knowledge in memory have stressed the importance of feature commonality in the structure of object concepts (Cree & McRae, 2003; Farah & McClelland, 1991; Tyler & Moss, 2001). According to such feature-based models, taxonomic relations based on the features objects may share with each other are the main organizing principle of object knowledge in semantic memory. For example, dog and bear are semantically related by virtue of their many common perceptual and encyclopedic features (both have eyes, have four legs, can breathe, etc.). However, feature commonality is probably not the only principle determining the organization of object concepts. Thematic relations that are based on object complementarity in events have been shown to be a highly relevant dimension of semantic categories. For instance, dog and leash may not share many perceptual features but are nonetheless semantically related by virtue of their complementary roles in the “walking dog” event (see Estes, Golonka, & Jones, 2011 for review).
There is evidence supporting the existence of functionally and neuroanatomically distinct taxonomic and thematic semantic systems (de Zubicaray, Hansen, & McMahon, 2013; Kalénine et al., 2009; Lewis, Poeppel, & Murphy, 2015; Mirman & Graziano, 2012a; Sass, Sachs, Krach, & Kircher, 2009; Schwartz et al., 2011; but see Jackson, Hoffman, Pobric, & Lambon Ralph, 2015). Schwartz and colleagues (2011) investigated the brain regions responsible for taxonomic and thematic semantic errors during picture naming in a large sample of aphasic patients with left hemisphere regions. Using Voxel-based Lesion-Sympton Mapping analyses, they demonstrated that whereas taxonomic errors (e.g., producing “cat” to a picture of a dog) were associated with lesions of the anterior temporal pole, thematic errors (e.g., producing “leash” to a picture of a dog) were associated with lesions of the temporoparietal junction. In an fMRI study with healthy adults, Kalénine et al. (2009) presented on each trial a triad consisting of a target picture and a choice of two additional pictures, only one of which was related to the target. Depending on the condition, the related object could be taxonomically (spoon-colander) or thematically (spoon-yogurt) related to the target, and subjects had to indicate the related object. Both taxonomic and thematic knowledge (compared to baseline) activated a very similar semantic network involving posterior and anterior areas. However, direct comparison of the brain regions activated in the two conditions showed that identification of taxonomic relations selectively activated bilateral occipital areas while identification of thematic relations selectively recruited the bilateral posterior temporoparietal cortex. For the most part, findings from neuroimaging and patient studies show that taxonomic and thematic semantic knowledge rely on partially distinct cortical networks (but see Jackson et al., 2015). In addition, despite variations in the populations studied and the methodologies used, the left posterior temporoparietal cortex has been consistently associated with access to thematic knowledge, a region that is traditionally not considered as central for semantic object representations (Bright, Moss, & Tyler, 2004). A role for the posterior temporal lobe, specifically, has also been reported. Humphreys, Newling, Jennings, & Gennari (2013) reported that the posterior temporal lobe is critical for the processing of event representations, and Wu, Waller, & Chatterjee (2007) found that damage to the posterior lateral temporal cortex produced deficits in thematic role knowledge. Thus, a better understanding of the functional role of the posterior temporal cortex in object conceptual organization is greatly needed.
Growing evidence indicates that the left posterior temporal cortex is also critical for recognizing object-related actions. Left posterior middle/superior temporal cortices have been shown to be required for the representation of object-related gestures (Kalénine, Buxbaum, & Coslett, 2010; Tarhan, Watson, & Buxbaum, 2015; see Urgesi, Candidi, & Avenanti, 2014 for a recent meta-analysis of patient studies). Interestingly, the posterior temporal region includes unimodal and multimodal areas that are important for action semantics. The human visual motion area (hMT+), has been related to both the perception of moving objects (Beauchamp & Martin, 2007) and conceptual processing of pictures displaying transitive (object-directed) and intransitive actions (Kable, Lease-Spellmeyer, & Chatterjee, 2002; Watson, Cardillo, Bromberger, & Chatterjee, 2014). Moreover, part of the left lateral temporo-occipital cortex (LTO) just anterior to hMT+, has been associated with manipulation knowledge as assessed by word stimuli (Binder, Desai, Graves, & Conant, 2009; Fernandino et al., 2015), conceptual processing of action verbs (Kable et al., 2002; Watson, Cardillo, Ianni, & Chatterjee, 2013), and perception of both tool and hand pictures (Bracci, Cavina-Pratesi, Ietswaart, Caramazza, & Peelen, 2012). In combination with the data relevant to thematic relations described above, these data suggest that there may be overlap in the left posterior temporal cortex between conceptual processing of object-related actions and that of at least some types of thematic relations.
The effect of object category on taxonomic and thematic processing supports the hypothesis of a privileged relationship between thematic relations depicting associations between manipulable manufactured objects (hereafter, manipulable artifacts) and their typical agents and recipients (e.g. axe-wood; notebook-student) and the left posterior temporal cortex. In Kalénine et al.’s (2009) fMRI experiment, brain activations when healthy participants made taxonomic or thematic judgments about objects were strongly influenced by target object category. Four object categories were designed by crossing object domain (natural objects, artifacts) and object manipulability (manipulable objects, non-manipulable objects). Activation of occipital areas during taxonomic judgments was stronger for natural than artifact objects, whereas activation of the left posterior temporoparietal cortex during thematic judgments was stronger for manipulable than non-manipulable artifact objects. Moreover, in a behavioral version of the fMRI experiment, differences in response times for identifying taxonomic and thematic relations also depended on object category. Response times on trials with taxonomic relations were shorter for natural objects than artifacts, but response times on trials with thematic relations were shorter for artifacts than natural objects. Moreover, the relatively faster processing of artifacts on thematic trials was exaggerated when the artifacts were manipulable. This suggests that the recruitment of the left temporoparietal cortex during semantic object processing reflects access to some type of information that is important for the comprehension of object relations in terms of their roles in events, and moreover, is critical for the representation of manipulable artifacts. We hypothesize that such information may correspond to action knowledge; specifically, the motion and postures associated with the typical use of manipulable artifacts (Bracci et al., 2012).
Preliminary evidence supporting this possibility may be found in a recent study conducted in patients with lesions anterior (n = 8) or posterior (n = 9) to the left central sulcus (Tsagkaridis, Watson, Jax, & Buxbaum, 2014), along with healthy participants. In a forced-choice triads task, participants were asked to choose which of two objects was most closely related to a target. Taxonomic relations (wine bottle-water bottle), thematic relations including action (wine bottle-corkscrew), and thematic relations without action (wine bottle-cheese) were contrasted in a pairwise manner. Thus, for example, a wine bottle was displayed with both a corkscrew and water bottle, and participants selected whether corkscrew or water bottle was most closely related to wine bottle. Results showed that healthy participants preferentially chose thematic relations including action over the two other types of semantic relations. This preference was significantly less pronounced in posterior stroke patients compared to healthy controls, but the difference between posterior and anterior stroke patients only reached the level of a trend. Thus, preliminary results in stroke patients suggest that left temporoparietal lesions may affect both thematic artifact processing and artifact action knowledge. Yet the selective and critical role of this region in processing both thematic relations for artifacts and artifact-related actions remains to be established. To this aim, we used a whole-brain approach and evaluated processing of taxonomic and thematic relations as well as gesture recognition in a sample of 48 stroke patients with a range of lesion foci in the left hemisphere.
2. Methods
2.1. Participants
Thirty-five healthy older adults and fifty-one left-hemisphere stroke patients took part in the study. Participants were recruited from the Neuro-Cognitive Rehabilitation Research Registry at the Moss Rehabilitation Research Institute (Schwartz, Brecher, Whyte, & Klein, 2005). Patients were excluded if database records indicated language comprehension deficits of sufficient severity to preclude comprehension of task instructions. Participants over the age of 80 and/or with histories of co-morbid neurologic disorders, alcohol or drug abuse, or psychosis were also excluded. All participants gave informed consent to participate in the behavioral testing in accordance with the guidelines of the IRB of Albert Einstein Healthcare Network. They were paid for their participation and reimbursed for travel expenses. Patients also provided informed consent to participate in an MRI or CT imaging protocol at the University of Pennsylvania School of Medicine. Data from three patients were excluded after they participated in the study. The first patient’s brain scan showed only subcortical lesions, the second patient had additional white matter disease, and the third patient demonstrated outlier performance in the behavioral task with reaction times greater than three standard deviations from the mean of the rest of the patient sample. The final sample included 48 patients whose lesions included cortical tissue (27 males and 21 females, mean age = 57, SD = 11 range = 30–77) and 35 healthy older controls with Mini Mental State Examination (MMSE) scores greater than 27 (13 males and 22 females, mean age = 62, SD = 8, range = 43–79). Mean age was close to 60 in both groups, although patients were slightly younger than controls (p = .03). Note that this age discrepancy would, if anything, favor the performance of the patients. Groups were well matched on education (mean controls = 15.9 years; SD controls = 2.3 years, range controls = 12–20; mean patients = 15.4 years, SD patients = 3.5 years, range patients = 11–29). Patient information is reported in Table 1.
Table 1.
Patient | Gender | Age | Scan Type |
Handedness | Education (years) |
No. of voxels lesioned |
Gesture recognition (% correct) |
Residualized Thematic Artifact RTs (ms) |
---|---|---|---|---|---|---|---|---|
188 | M | 63 | MRI | R | 16 | 9420 | 91 | −567 |
190 | M | 64 | MRI | R | 12 | 205297 | 91 | −172 |
281 | F | 51 | MRI | R | 16 | 151318 | 81 | 468 |
286 | M | 77 | CT | R | 14 | 65376 | 96 | −178 |
419 | F | 43 | CT | R | 13 | 51860 | 91 | 11 |
913 | M | 65 | MRI | R | 16 | 54214 | 80 | 420 |
1088 | F | 48 | CT | R | 16 | 89072 | 91 | −26 |
1229 | M | 36 | MRI | R | 12 | 16878 | 98 | −23 |
1238 | M | 54 | MRI | R | 13 | 172209 | 96 | −557 |
1371 | F | 73 | MRI | R | 14 | 21496 | 86 | 418 |
1392 | M | 69 | MRI | R | 19 | 84923 | 75 | 252 |
1449 | F | 58 | CT | R | 12 | 30014 | 92 | 220 |
1510 | M | 55 | MRI | R | 19 | 219930 | 71 | 748 |
1619 | M | 75 | MRI | R | 21 | 8005 | 70 | −76 |
1626 | M | 74 | CT | R | 11 | 77326 | 87 | 79 |
1687 | M | 74 | MRI | R | 29 | 40953 | 88 | 288 |
1743 | M | 53 | MRI | R | 18 | 82864 | 91 | −106 |
1764 | M | 41 | CT | R | 15 | 41846 | 88 | −38 |
1780 | F | 62 | MRI | R | 16 | 37025 | 83 | 330 |
1846 | F | 52 | MRI | R | 14 | 31430 | 98 | 0 |
1857 | F | 74 | CT | R | 12 | 17856 | 63 | 286 |
1958 | M | 42 | CT | L | 16 | 299767 | 86 | −611 |
2006 | M | 50 | CT | R | 14 | 5376 | 83 | −101 |
2011 | F | 47 | MRI | R | 13 | 80020 | 98 | 110 |
2027 | M | 66 | MRI | R | 16 | 271984 | 67 | 408 |
2044 | M | 56 | CT | R | 12 | 25273 | 91 | 752 |
2048 | F | 58 | CT | R | 16 | 180409 | 92 | −28 |
2079 | M | 56 | MRI | R | 12 | 57638 | 82 | −215 |
2083 | F | 62 | MRI | R | 17 | 51525 | 96 | 232 |
2139 | M | 57 | MRI | R | 12 | 23748 | 100 | 151 |
2172 | F | 61 | MRI | R | 16 | 73100 | 91 | −637 |
2180 | M | 68 | CT | R | 14 | 67164 | 52 | 1093 |
2189 | F | 62 | CT | R | 21 | 92002 | 86 | 49 |
2221 | F | 33 | MRI | R | 19 | 63924 | 82 | −215 |
2226 | F | 61 | MRI | R | 15 | 14349 | 96 | −303 |
2284 | M | 65 | CT | R | 18 | 34759 | 87 | −209 |
2323 | F | 70 | MRI | R | 12 | 4191 | 75 | 231 |
2328 | F | 46 | MRI | R | 12 | 140554 | 71 | −306 |
2340 | F | 68 | MRI | R | 12 | 9489 | 74 | −668 |
2350 | M | 48 | MRI | R | 15 | 55685 | 91 | −263 |
2368 | F | 52 | MRI | R | 18 | 6840 | 75 | 121 |
2370 | M | 53 | CT | R | 12 | 91181 | 80 | −735 |
2378 | F | 55 | MRI | R | 12 | 128897 | 57 | 540 |
2464 | M | 62 | MRI | R | 22 | 46433 | 95 | −372 |
2481 | M | 67 | MRI | R | 19 | 68691 | 81 | −604 |
2493 | F | 51 | MRI | R | 12 | 19837 | 92 | −104 |
2516 | M | 30 | MRI | R | 13 | 88046 | 87 | −216 |
2548 | M | 49 | MRI | R | 10 | 90372 | 93 | 122 |
2.2. Behavioral tasks
2.2.1. Identification of thematic and taxonomic relations
All participants performed the forced-choice matching task used in Kalénine et al. (2009). The task aimed at evaluating performance in identification of thematic and taxonomic relations for the same target objects. Stimuli were triads of black-and-white drawings presented on a computer screen using E-prime software (Psychology Software Tools, Pittsburgh, PA). On each triad, a target picture was presented in the upper part of the display, and two pictures of choice were presented in the lower part of the display. One of the pictures of choice was semantically related to the target, the other was unrelated to the target. Several behavioral results indicate that superordinate taxonomic relations tend to be overall less semantically related than thematic relations (e.g., Osborne & Heath, 2003; Scheuner, Bonthoux, Cannard, & Blaye, 2004), which may cause differences in the processing difficulty of the two types of semantic relations (see also behavioral results in Jackson et al., 2015). Accordingly, taxonomic and thematic relations were selected in order to keep overall semantic relatedness equivalent between conditions. Consequently, some taxonomic relations belonged to the same intermediate category (e.g. farm animals) rather than the same superordinate category (e.g., animals). Kalenine et al. (2009) provided normative ratings collected in 10 young adults on a 10-point scale from 0 “not associated at all” to 10 “very strongly associated” which showed that the strength of semantic relations between the target and related picture of choice was equivalent in the thematic and taxonomic conditions [F(1,88)=1.5, p = .22]. Participants in the present study had to decide as quickly and accurately as possible which picture was semantically related to the target by pressing one of two response keys (left/right) with the middle and index fingers of their left hand. Forty-eight target objects (12 non-manipulable natural objects, 12 manipulable natural objects, 12 non-manipulable artifacts, and 12 manipulable artifacts) were presented twice, once with a thematic associate (axe-wood) and once with a taxonomic associate (axe-drill). An example of thematic and taxonomic triads is presented on Figure 1 and the complete list is provided in the Appendix. The 96 trials were presented in random order and the position of the target and foil picture was also randomized. Accuracy and reaction times were recorded for each trial. Eight additional trials were provided for practice before the 96 experimental trials.
2.2.2. Action recognition
Artifact action knowledge was assessed in the patient group using the forced-choice gesture recognition tasks used in Buxbaum, Kyle, & Menon (2005) and Kalenine et al. (2010). Participants heard an action verb repeated twice (e.g., “Sawing. Sawing.”), and simultaneously viewed the verb presented on a card that remained on the tabletop throughout the trial. After a 2-s pause, they saw two repetitions of a videotaped examiner performing a gesture “A”, and after an additional 2-s pause, two repetitions of a second gesture “B”. One gesture was the correct match to the verb (e.g., sawing), and the other was incorrect. The order of the correct and incorrect gesture videos was randomized. Patients had to select which gesture “A” or “B” (by verbalizing or pointing) correctly matched the action verb. They were allowed to respond at any point while the videos were being shown. Twenty-four action verbs referring to transitive actions were presented twice with the same correct gesture but a different incorrect gesture. For one presentation, the incorrect gesture was a semantic foil, namely a correct gesture for another manipulable artifact (e.g., hammering). For the second presentation, the incorrect gesture was a spatial foil presenting hand posture, arm posture, or amplitude/timing errors. Overall, the gesture recognition tasks involved 48 trials.
Patients also completed a control task to ensure they understood the verb phrases used in the gesture recognition tasks. The control task also involved forced-choice matching between action verbs and visual stimuli. Participants heard the same action verbs as in the gesture recognition tasks and had to choose an artifact picture from an array of three artifacts to match with the action name (e.g., matching a saw to the verb “sawing”). Patients were overall highly accurate on this task (M = 92%, SD = 8%). Thus, actions that patients failed to match to the relevant artifact were excluded from their final gesture recognition score, and average action recognition performance was calculated based on an adjusted total number of trials.
2.3. Scanning and lesion segmentation
All patients underwent a structural, high-resolution, 3D magnetic resonance imaging (MRI) or computed tomography (CT) scan. Research MRI scans (N = 31) included whole-brain T1-weighted images collected on a 3T (Siemens Trio, repetition time = 1,620 ms, echo time = 3.87 ms, field of view = 192 × 256 mm, 1 × 1 × 1 mm voxels) or 1.5 T (Siemens Sonata, repetition time = 3,000 ms, echo time = 3.54 ms, field of view = 24 cm, 1.25 × 1.25 × 1.25 mm voxels) scanner, using a Siemens 8-channel head coil. Patients who were contraindicated for MRI (N = 14) underwent whole-brain research CT scans without contrast (60 axial slices, 3–5 mm slice thickness) on a 64-slice Siemens SOMATOM Sensation scanner. The three last patients declined to participate in a research scan but provided clinical scans that were determined by the project neurologist to be of sufficiently high quality to reliably draw the outlines of the lesion.
Lesions on MRI scans were segmented manually upon the digital MRI image file using MRIcron software (Rorden, Karnath, & Bonilha, 2007), and then registered (Avants, Schoenemann, & Gee, 2006) to a common 1 × 1 × 1 mm template (Montreal Neurological Institute space “Colin27”, Holmes et al., 1998), and inspected by an experienced neurologist who was naïve to the behavioral data. Lesions on CT scans were drawn by the same neurologist directly onto the Colin27 template using MRIcron software, which had been rotated to match the pitch of the patient’s scan. All lesions were then thresholded and quantized on the criteria that if more than 50% of a voxel was lesioned, it was assigned a value of 1, all other partially damaged or spared voxels were assigned a value of 0.
3. Results
3.1. Behavioral results
3.1.1. Processing of thematic and taxonomic relations in stroke patients and older controls
Performance in the forced-choice task was analyzed as a function of Group (Controls, Patients), Type of Semantic Relation presented in the triad (Taxonomic, Thematic), Domain (Natural objects, Artifacts) and Manipulability (Manipulable, Non-manipulable) of the target objects. A 2*2*2*2 Analysis of Variance (ANOVA) was conducted on correct response times (RTs) after logarithmic transformation to ensure normality of distributions and homogeneity of variances. Extreme RTs were trimmed before analysis. Reaction times that were below 500 ms, above 10,000 ms, or greater than 3 standard deviations from that participant’s mean RT in each condition were excluded (1.6 % total trials). As noted earlier, after RT trimming, one outlier patient remained extremely slow (mean RTs > 11 sec; > 3SD from remainder of patient group), and was excluded from further analysis.
The ANOVA showed a main effect of Group [F(1,82) = 88.16, η2 = .52, p = .001] with faster RTs in controls (M = 1887 ms, SD = 396 ms) than patients (M = 3352, SD = 993), and a main effect of Manipulability [F(1,82) = 4.92, η2 = .06, p = .03], with faster RTs for manipulable target objects (M = 2688, SD = 1057) than non-manipulable ones (M = 2794, SD = 1116). Critically, the predicted Semantic Relation × Domain interaction was significant F(1,82) = 37.27, η2 = .31, p = .001], and did not further interact with Group [F(1,82) = 0.85, η2 = .01, p = .35]. For natural concepts, identification of taxonomic relations was faster than identification of thematic relations [t(82)= 7.13, p = .001]. For artifact concepts, identification of thematic relations was faster than identification of taxonomic relations [t(83)= −4.97, p = .001]. Figure 2 highlights this interaction pattern for the two groups. No other significant effects were found.
Accuracy in the forced-choice task was analyzed using logistic regression. Response on each trial (correct vs. incorrect) was used as the dependent variable, and Group (Controls, Patients), Type of Semantic Relation (Taxonomic, Thematic), Domain (Natural objects, Artifacts) and Manipulability (Manipulable, Non-manipulable) as predictors. Only Group was a significant predictor of the correct identification of the semantic relation on a given trial [estimate = −0.59, z = −2.00, p = .04]. No other main effects or interactions were found. Mean accuracy and correct reaction times are reported in Table 2.
Table 2.
Group | Semantic Relation | Domain | Manipulability | mean (SD) accuracy | mean (SD) correct reaction times |
---|---|---|---|---|---|
control | Taxonomic | Artifact | NonManip | 0.96 (0.06) | 1984 (571) |
control | Taxonomic | Artifact | Manip | 0.98 (0.06) | 1854 (412) |
control | Taxonomic | Natural | NonManip | 0.95 (0.07) | 1870 (618) |
control | Taxonomic | Natural | Manip | 0.96 (0.05) | 1818 (458) |
control | Thematic | Artifact | NonManip | 0.97 (0.04) | 1813 (371) |
control | Thematic | Artifact | Manip | 0.97 (0.05) | 1777 (387) |
control | Thematic | Natural | NonManip | 0.93 (0.07) | 2031 (481) |
control | Thematic | Natural | Manip | 0.93 (0.05) | 1946 (431) |
patient | Taxonomic | Artifact | NonManip | 0.93 (0.07) | 3558(1154) |
patient | Taxonomic | Artifact | Manip | 0.93 (0.07) | 3348 (1019) |
patient | Taxonomic | Natural | NonManip | 0.93 (0.09) | 3166 (1080) |
patient | Taxonomic | Natural | Manip | 0.95 (0.06) | 3014 (865) |
patient | Thematic | Artifact | NonManip | 0.94 (0.08) | 3107 (882) |
patient | Thematic | Artifact | Manip | 0.95 (0.07) | 3178 (889) |
patient | Thematic | Natural | NonManip | 0.87 (0.09) | 3563 (974) |
patient | Thematic | Natural | Manip | 0.86 (0.10) | 3376 (919) |
3.1.1. Relationship between thematic knowledge for artifacts and gesture recognition abilities in patients
In the light of previous findings (Kalenine et al. 2009; Tsagkaridis et al. 2014), we originally hypothesized that thematic relations between manipulable artifacts specifically rested upon artifact action knowledge. Following from this, we investigated the selective relationship between performance in the identification of thematic relations for manipulable and non-manipulable artifact concepts and performance in artifact action knowledge (gesture recognition) in patients. A table of simple correlations between patient behavioral scores is provided in Table 3 below. Moreover, in order to control for both overall cognitive processing speed and familiarity with the artifacts selected, we computed two partial correlations. The first was a partial correlation of RTs for thematic relation identification for manipulable artifacts and gesture recognition accuracy scores, partialling out taxonomic relation identification for the same artifacts. The second used the same strategy, but with non-manipulable artifacts. We expected a selective negative relationship between gesture recognition accuracy scores and thematic manipulable artifact RTs. We found that thematic manipulable artifact RTs were negatively correlated with patient gesture recognition scores after controlling for taxonomic manipulable artifact RTs [r = − 0.41, R2 = 0.17, p = .004]. However, a similar relationship was found between gesture recognition accuracy scores and thematic non-manipulable artifact RTs, although at the level of a trend [r = − 0.28, R2 = 0.08, p = .053]. The two partial correlations did not differ statistically [Steiger's z = −0.99, p = 0.16].
Table 3.
Them Art NM |
Them Art M |
Them Nat NM |
Them Nat M |
Taxo Art NM |
Taxo Art M |
Taxo Nat NM |
Taxo Nat M |
|
---|---|---|---|---|---|---|---|---|
Gesture Recognition |
−0.38 | −0.43 | −0.30 | −0.40 | −0.27 | −0.27 | −0.35 | −0.33 |
Behavioral correlation results thus indicate a relationship between thematic artifact processing and gesture recognition scores that is relevant for all artifacts, regardless of their manipulability. Consequently, we computed partial correlations between RTs for thematic relation identification for all artifacts and gesture recognition scores while controlling for RTs for taxonomic relations identification for the same artifacts. Overall, thematic artifact RTs were negatively related to patient gesture recognition scores [r = − 0.38, adjusted R2 = 0.14, p = .008]. Patients with lower gesture recognition scores were relatively slower to identify thematic relations for artifacts. No such relation was found for natural objects: thematic natural RTs were not correlated with gesture recognition scores after controlling for taxonomic natural RTs [r = − 0.12, adjusted R2 = 0.01, p = .39]. The statistical difference between the partial correlations related to thematic artifacts and thematic natural objects reached the level of a trend [Steiger's z = −1.46, p = 0.07].
These data support a selective relationship between thematic processing of artifacts and knowledge about artifact-related actions (gesture recognition). We next assessed whether these two tasks are supported by shared neural substrates in the posterior temporoparietal cortex, as predicted.
3.2. Lesion-Behavior mapping
Information about lesioned voxels in the 48 lesion drawings was used in whole-brain Voxel-based Lesion-Syndrom Mapping (VLSM) analyses. The aim of VLSM was to uncover the cortical regions that are critical for identifying thematic relations for artifacts and for recognizing object-related gestures.
Total lesion volume is an important possible confound to take into account in lesion analyses. Moreover, following the same rationale as in the behavioral analysis, we needed a measure of thematic identification performance for artifacts that controlled for taxonomic identification performance for the same artifacts that could be used in the VLSM. Thus for the VLSM on thematic artifact knowledge, we regressed RTs for thematic relation identification for artifacts on RTs for taxonomic relations identification for the same artifacts and total lesion volume. Then we used the residuals of the regression as a selective measure of thematic identification performance for artifacts. The more negative the residuals, the faster the identification of thematic relations for artifacts relative to taxonomic relations for the same artifacts, after controlling for total lesion volume. Thematic artifact residuals are reported for each patient in Table 1.For the VLSM on gesture recognition, we regressed gesture recognition scores on total lesion volume.
Then we used the NPM toolbox of the MRIcron analysis package (http://www.sph.sc.edu/comd/rorden/mricron/stats.html) to carry out a whole-brain VLSM analysis of the voxels most associated with a) thematic artifact RT residuals, b) gesture recognition residuals. Only voxels in which at last five participants had a lesion were considered for the analysis, corresponding to 264,291 voxels in the left hemisphere. An overlap map of the 48 lesions (Figure 3) shows good coverage of the left frontal, temporal and parietal lobes with maximum overlap (n > 20) in peri-sylvian regions. At each voxel, a pairwise comparison (t-test, converted into Z-scores) was performed to assess for differences between scores of participants with and without damage at that voxel. The false discovery rate correction was used to control for Type I errors resulting from multiple comparisons.
Figure 4 presents statistical maps of the Z-scores of voxels associated with the thematic artifact RT residuals (Figure 4A) and gesture recognition scores (Figure 4B). For both behavioral tasks, a large number of voxels in the posterior temporal cortex exceeded the FDR-corrected statistical threshold (regions in yellow). Thematic artifact RT residuals and gesture recognition residuals were associated with a count of 1281 and 905 significant voxels in the posterior temporal cortex (Brodmann Area 37), respectively.
For both tasks, additional voxels with high Z-scores values were also observed in the middle and superior temporal gyri (5619 voxels for thematic residuals, 5058 voxels for gesture recognition residuals) and in the inferior parietal lobe (2836 voxels for thematic residuals, 3548 voxels for gesture recognition residuals) at uncorrected statistical thresholds (regions in red). Information about maximum Z-values of clusters of significant voxels identified at different statistical thresholds is provided in Table 4.
Table 4.
Task | Threshold | Peak X Y Z coordinates (MNI) | cluster size | ||
---|---|---|---|---|---|
Thematic | FDR corrected | −34 | −68 | 11 | 2316 |
−58 | −65 | 3 | 119 | ||
.01 uncorrected | −37 | −66 | 13 | 6857 | |
−65 | −12 | 8 | 853 | ||
−67 | −38 | 24 | 316 | ||
−30 | −44 | 23 | 510 | ||
−52 | −66 | 39 | 116 | ||
Action | FDR corrected | −42 | −68 | 6 | 1993 |
.01 uncorrected | −45 | −68 | 7 | 4474 | |
−43 | −58 | 25 | 529 |
The conjunction of significant voxels (p < .05 FDR corrected) associated with thematic artifact residuals and gesture recognition residuals largely overlapped with hMT+, and, at more lenient thresholds (p < .01 uncorrected), extended to LTO area, as shown on Figure 4C.
A complementary VLSM analysis indicated that no voxels were significantly associated (at FDR-corrected threshold) with thematic natural RT residuals. Moreover, the only voxels showing a relationship to thematic natural RT residuals at an uncorrected level of p = .01 were in the dorsal premotor cortex. This analysis thus reinforces the idea of a selective association between thematic artifact knowledge and the left posterior temporal cortex.
4. Discussion
4.1. Summary of findings
The ability to identify taxonomic and thematic relations for different object categories was assessed in 48 left-hemisphere stroke patients and 35 healthy older controls with the forced-choice matching task used in Kalenine et al. (2009) in healthy younger adults. As in younger adults, response time data showed an interaction between type of semantic relation and object category. For both healthy controls and stroke patients, thematic relations were identified faster than taxonomic relations for artifacts, while taxonomic relations were identified faster than thematic relations for natural objects. Moreover, response times for identifying thematic relations for artifacts were selectively associated with performance in gesture recognition, our measure of artifact action knowledge. Patients with lower artifact action knowledge scores were relatively slower to identify thematic relations for artifacts, whereas artifact action knowledge had no bearing on identification of thematic relations for natural objects or taxonomic relationships for artifacts.
Whole brain Voxel Based Lesion-Symptom Mapping (VLSM) analyses further demonstrated that lesions to a region of the left posterior temporal cortex, including motion area hMT+ and extending anteriorly, were associated with both relatively slower response times in identifying thematic relations for artifacts and poorer artifact action knowledge in patients.
4.2. The selective advantage of thematic relations for artifact concepts may be generalized over populations and tasks
In both older control and patient groups, faster identification of thematic relations was selective to artifact concepts. The same result was found in younger participants with the same items and task (Kalénine et al., 2009). Potential differences in the associative strength of the different semantic relations cannot account for this pattern, since it was equivalent between conditions. Moreover, results are unlikely related to the explicit demands of the forced-choice task. Recent eye-tracking and neurophysiological studies providing implicit measures of semantic processing showed faster processing of thematic relations compared to taxonomic semantic relations (based on shared features) during identification of manipulable artifact objects (Kalénine, Mirman, Middleton, & Buxbaum, 2012; Wamain, Pluciennicka, & Kalénine, 2015). Thus, the present results confirm the advantage of thematic relations for artifacts and suggest that this advantage remains highly stable over age and tasks.
It is interesting to note that as a group, the pattern observed in left hemisphere stroke patients did not differ from older controls, although patients were slower than controls overall. This reinforces the likelihood that the observed modulation of thematic processing by object category is not related to inherent differences in the difficulty of taxonomic versus thematic processing. General cognitive impairment following brain damage would have likely degraded performance for the most difficult semantic relations. Yet the relative disadvantage of taxonomic relations for artifacts was equivalent in older adults and stroke patients.
In contrast to our previous study in younger adults (Kalénine et al., 2009), and contrary to our initial hypothesis, the advantage of thematic relations for artifacts was equivalent for manipulable and non-manipulable artifacts in both control and patient groups. Reasons underlying the absence of an effect of manipulability in older participants remain unclear. However, identification of semantic relations was overall faster for manipulable than non-manipulable objects, a main effect that was not found in younger adults. The greater resistance of manipulable object semantic processing to aging might have attenuated potential benefits of object manipulability as a function of semantic relation and object category.
4.3. The critical role of the posterior temporal cortex in thematic relations is specific to object categories for which action knowledge is important
Differences in the neural substrates of thematic and taxonomic semantic relations have been investigated in several recent neuroimaging and patient studies (Chen et al., 2014; de Zubicaray et al., 2013; Jackson et al., 2015; Kalénine et al., 2009; G. a. Lewis et al., 2015; Maguire, Brier, & Ferree, 2010; Mirman & Graziano, 2012b; Sachs, Weis, Krings, Huber, & Kircher, 2008; Sachs et al., 2011; Sachs, Weis, Zellagui, et al., 2008; Sass et al., 2009; Schwartz et al., 2011). Some studies report only minor differences in the brain regions involved in the processing of thematic and taxonomic relations (Jackson et al., 2015; Sachs, Weis, Krings, et al., 2008; Sachs et al., 2011; Sachs, Weis, Zellagui, et al., 2008). Most often, the few observed neuroanatomical differences have been interpreted as reflecting the extra processing demands involved in processing taxonomic relations, which are frequently assumed to be more difficult (Jackson et al., 2015; Maguire et al., 2010; Sachs, Weis, Krings, et al., 2008; Sachs, Weis, Zellagui, et al., 2008). Interestingly, several studies failing to find specific brain regions associated with thematic relations did not manipulate object category (Jackson et al., 2015; Maguire et al., 2010) and/or used predominantly natural objects (G. a. Lewis et al., 2015). In contrast, at least three studies that have directly manipulated object category (Kalénine et al., 2009) or used predominantly artifacts (Mirman & Graziano, 2012b; Sass et al., 2009) found a selective involvement of the left posterior temporal cortex in thematic relations. In addition, many of the behavioral and functional neuroimaging studies failing to observe selective posterior temporal involvement in thematic processing used tasks involving linguistic stimuli (e.g., Jackson et al., 2015; Lewis et al., 2015; Noonan Jefferies Corbettt and Lambon Ralph, 2010), unlike the pictorial stimuli used here. It may be the case that pictorial stimuli privilege access to manipulation and or visual motion information, thereby highlighting the role of posterior temporal lobe in thematic artifact processing. In general, variations in the modalities assessed as well as the tasks and methodologies used in different studies may in part explain the discrepancies observed. If the left posterior temporal lobe is critical only for thematic relations concerning artifacts, as the present results suggest, then the absence of overall neural differences between thematic and taxonomic processing when collapsing across object category is not surprising. In rare cases, the left posterior temporo-parietal cortex has been selectively associated with access to thematic knowledge without further distinction between object categories (Schwartz et al., 2011), but it is difficult to rule out the possibility that overall differences are actually driven by a subset of items, as demonstrated in Kalénine et al. (2009).
In the present study, the relationships between left posterior temporal lesions, thematic processing for artifacts, and artifact action knowledge extend previous findings by suggesting a mechanism that may explain why thematic relations may selectively involve left posterior temporal regions and are particularly relevant for artifacts. The left posterior temporal cortex is associated with both manipulable artifact concepts (Beauchamp & Martin, 2007; Campanella, D’Agostini, Skrap, & Shallice, 2010; Chao, Weisberg, & Martin, 2002; Devlin et al., 2002; Martin, Kyle Simmons, Beauchamp, & Gotts, 2014; Noppeney, 2008) and artifact action knowledge (Andres, Pelgrims, & Olivier, 2013; Kalénine et al., 2010; Kellenbach, Brett, & Patterson, 2003; Noppeney, 2008; Perini, Caramazza, & Peelen, 2014; Quandt & Chatterjee, 2015; Schubotz, Wurm, Wittmann, & von Cramon, 2014; Tarhan et al., 2015; Tyler et al., 2003). The present findings suggest that understanding the semantic relation between an artifact (manipulable or not) and other objects and agents participating in the same event necessitates access to representation of the actions that are typically associated with the artifact, an ability that requires the integrity of the left posterior temporal cortex. Given the observation that the common region of the left posterior temporal cortex overlaps with hMT+ and LTO, the next section discusses the probable format of action representations that may underlie thematic relations for artifact concepts.
4.4. Multimodal representations of actions may underlie thematic relations for artifacts
The ability to recognize artifact-related gestures was associated with performance in the identification of thematic relations for artifacts (both manipulable and non-manipulable), and both gesture recognition and thematic relation processing relied on regions of the left posterior temporal cortex typically involved in processing visual motion (hMT+, Watson et al., 2014) and manipulation information (LTO, Fernandino et al., 2015). Similar regions have been recently shown critical for artifact action knowledge using the same gesture recognition task in a large sample of left-hemisphere stroke patients (Tarhan et al., 2015). Importantly, lesions to hMT+ were associated with poorer gesture recognition performance after controlling for the ability to actually perform the same gestures. Moreover, bilateral activations of area hMT+ have been repeatedly reported during processing of action photographs (Kourtzi & Kanwisher, 2000) or pictograms (Assmus, Giessing, Weiss, & Fink, 2007; Tranel, Kemmerer, Adolphs, Damasio, & Damasio, 2003) indicating that processing of implied visual human motion may occur in response to static action pictures. These findings suggest that thematic relations for artifacts convey visual motion information typically associated with artifact-related actions. This interpretation is consistent with a recent view of action representations in terms of complementary implementation and association systems (Quandt & Chatterjee, 2015). Whereas the frontoparietal system may be specialized for the production and simulation of actions from an egocentric perspective, the posterolateral temporal system may process actions from a third-person perspective, which would explain the important involvement of the visual modality during the perception of artifact-related actions and during the reconstruction of action events in which thematically-related objects participate.
The format of the information coded in LTO is most likely multimodal (Lingnau & Downing, 2015), but may differentially weight visuomotor information. LTO is involved in both action observation (Caspers, Zilles, Laird, & Eickhoff, 2010; Tarhan et al., 2015) and action performance such as during pantomimes of tool use (Lewis, 2006). In addition, selective responses to both hand and tool pictures have been found in LTO, in a portion that has close interconnections with the anterior intraparietal sulcus, a region involved in coding hand posture for object grasping (Bracci et al., 2012).This suggests that visuomotor information is an important component of action representations in LTO. Findings from neuroimaging studies investigating the neural substrates of concepts using verbal stimuli have also identified LTO has a region supporting action concepts (Fernandino et al., 2015; Watson et al., 2013), suggesting that its role in action representation may be independent of stimulus format. Together, the literature on action representation and the posterior temporal cortex support the hypothesis of a multimodal coding of actions in terms of visual motion and visuomotor properties in this region. Thus, the selective role of this region in thematic artifact knowledge suggests that thematic relations involving artifacts are supported by representations of artifact visual motion and manipulation.
4.5. Thematic knowledge is likely represented in a distributed semantic system
There is growing consensus that thematic relations, along with taxonomic relations, may serve as an organizing principle in semantic memory (Davidoff & Roberson, 2004; de Zubicaray et al., 2013; Estes et al., 2011; Kalénine et al., 2009; G. a. Lewis et al., 2015; Lin & Murphy, 2001; Merck, Jonin, Laisney, Vichard, & Belliard, 2014; Mirman & Graziano, 2012a; Sass et al., 2009; Schwartz et al., 2011; Tsagkaridis et al., 2014). From a cognitive point of view, we have further proposed that thematic relations differ from taxonomic relations in their relative weighting on complementarity-based versus similarity-based mechanisms (Kalénine, Mirman, & Buxbaum, 2012; Kalénine, Mirman, Middleton, et al., 2012). In this regard, processing of thematic relations would rely more strongly on the complementarity between objects than taxonomic relations, which would be reflected in a greater involvement of posterior than anterior cortical areas in semantic processing. One may argue that processing semantic relations based on similarity between objects induces an additional abstraction step (e.g., saw and axe may not be used together in a given event but alternatively to fulfill the same function in the event), which may cause them to be processed more anteriorly in the temporal lobe (but see Kalenine et al. 2009, where stimulus similarity was largely perceptual). In contrast, thematic relations based on complementarity between objects within multimodal events (e.g., saw and wood are used together in the same event) may remain more strongly grounded in posterior multimodal brain regions.”
The hierarchical organization of the posterior semantic network in different levels of unimodal, bimodal, and multimodal convergence zones (Damasio, 1989; Fernandino et al., 2015) may further account for the important heterogeneity of thematic neural representations. Depending on the specifics of the stimuli and tasks, processing thematic relations may evoke a different combination of sensorimotor modalities and may recruit different convergence zones. The present findings suggest that cortical regions that process and integrate visual motion and motor information are critical for process in the semantic relationship between an artifact and its thematic associates. Other unimodal and bimodal areas (e.g., color, shape, sound) may be more important for other types of thematic relations. As a consequence, it may be difficult to highlight a general neural substrate for thematic knowledge (dissociated from taxonomic knowledge).
Regardless, involvement of the highest-level convergence zones in thematic processing may not capture the diversity of thematic knowledge. In a recent norming study, Jouravlev and McRae (2015) identified five subtypes of thematic relations in adults’ verbal productions in response to concept names: attributive (e.g., baker-apron), argument (e.g., bear-fish), coordinate (beer-chips), locative (e.g. doctor-hospital), and temporal (e.g. church, Sunday). Notably, the distribution of the different subtypes varied as a function of object category. Thematic “argument” relations, which broadly corresponded to the action of one object on the other, were the most frequent subtype for the category of “instruments”, i.e. manipulable artifacts. This reinforces the idea that different subtypes of thematic relations, probably relying on different regions of the posterior semantic system, are differentially important in organizing concepts from different categories.
In the light of this, as well as previous findings indicating that the left temporal cortex is selectively activated in healthy subjects during thematic processing of manipulable artifacts (Kalenine et al., 2009), it is not clear why the critical role of the posterior temporal cortex in thematic knowledge is not selective of manipulable artifacts in the present study. One possibility is that the manipulable artifact stimuli used did not activate motor information as strongly as expected in our older participant sample, either because they have a different degree of motor experience with the objects selected or because there is a decline in the ability to mentally represent actions after 65 (Gabbard, Caçola, & Cordova, 2011). This interpretation is consistent with the behavioural data that did not show the expected thematic processing advantage for manipulable artifacts in both patients and healthy older controls. Alternatively, because the distinction is a subtle one, demonstration of a different critical role of the posterior temporal cortex in thematic processing of manipulable versus non-manipulable artifacts may require a larger sample than we have amassed here.
Limitations of the study
In the present study as in Kalénine et al. (2009), the taxonomic and thematic relations selected were not completely doubly dissociated. Because we wanted to equalize the semantic associative strength of the relations between conditions, in the taxonomic condition, some objects are also thematically related. However, it is not the case that the objects in the thematic condition are taxonomically-related, except (for some items) at the broad level of animacy (living versus non-living things; please see Appendix). This may make it difficult to interpret the relatively weak relationship between taxonomic processing and gesture recognition (see Table 3). That is, it may be the case that this relationship is driven by the fact that the taxonomic items also tended to bear thematic relationships to one another. Nevertheless, this study focuses on thematic processing, and we controlled statistically for the possible influence of taxonomic relationships in our thematic items by considering reaction times for identifying thematic relations for artifacts after partialling out reaction times for identifying taxonomic relations for the same artifacts.
Conclusion
In a large patient study, we demonstrated a selective relationship between semantic relations that are based on object complementary in events (i.e. thematic relations), artifact concepts, action knowledge, and the left posterior temporal lobe. Thematic relations show a processing advantage over taxonomic relations for artifact concepts. The magnitude of this advantage related to individual performance in artifact action knowledge, and both abilities rely on the integrity of the left posterior temporal cortex. The present findings are consistent with previous studies showing that thematic relations rely on the posterior temporal cortex (Kalénine et al., 2009; Mirman & Graziano, 2012b; Sass et al., 2009; Schwartz et al., 2011; Tsagkaridis et al., 2014). They further provide novel insights into the functional role of this region in thematic knowledge, and suggest that the close association between thematic relations for artifacts and action representations may reflect their common dependence on visual motion and manipulation information. Future research may usefully consider the diversity of thematic knowledge in relation to the neural organization of the posterior temporoparietal cortex into different levels of modal and multimodal convergence zones.
Acknowledgments
We appreciate the help of H. Branch Coslett with lesion identification and Alexis Kington with testing patients. This research was funded by NIH grant R01- NS065049 to Laurel Buxbaum.
Appendix : Complete list of stimuli
Semantic Relation |
Target | Related | Unrelated | Domain | Manipulability |
---|---|---|---|---|---|
taxonomic | axe | drill | wallet | artifact | manipulable |
taxonomic | bowl | fork | well | artifact | manipulable |
taxonomic | dress | shorts | crane | artifact | manipulable |
taxonomic | glass | whisk | chimeney | artifact | manipulable |
taxonomic | hammer | pliers | washing machine | artifact | manipulable |
taxonomic | mitten | pyjamas | tulips | artifact | manipulable |
taxonomic | notebook | scissors | buffalo | artifact | manipulable |
taxonomic | paintbrush | ruler | cat | artifact | manipulable |
taxonomic | saw | screwdriver | horse | artifact | manipulable |
taxonomic | shirt | sock | peanut | artifact | manipulable |
taxonomic | sneaker shoe | hat | nut | artifact | manipulable |
taxonomic | spoon | colander | trash can | artifact | manipulable |
taxonomic | bed | chair | sword | artifact | non manipulable |
taxonomic | car | bus | stairs | artifact | non manipulable |
taxonomic | castle | hut | beetle | artifact | non manipulable |
taxonomic | couch | dresser | dice | artifact | non manipulable |
taxonomic | motorbike | helicopter | clothes pin | artifact | non manipulable |
taxonomic | plane | backhoe loader | tiger | artifact | non manipulable |
taxonomic | sink | fridge | pine cone | artifact | non manipulable |
taxonomic | small boat | small truck | plant | artifact | non manipulable |
taxonomic | stove | desk | dog house | artifact | non manipulable |
taxonomic | tent | house | onion | artifact | non manipulable |
taxonomic | train | bike | scarecrow | artifact | non manipulable |
taxonomic | truck | large boat | shell | artifact | non manipulable |
taxonomic | apple | pineapple | volcano | natural | manipulable |
taxonomic | carrot | tomato | pacifier | natural | manipulable |
taxonomic | cherries | pear | baby bottle | natural | manipulable |
taxonomic | daisy | tree | small boat | natural | manipulable |
taxonomic | ear | foot | garlic | natural | manipulable |
taxonomic | hand | eye | grass | natural | manipulable |
taxonomic | ivy | mushroom | comb | natural | manipulable |
taxonomic | nose | mouth | pepper | natural | manipulable |
taxonomic | potato | lettuce | camel | natural | manipulable |
taxonomic | rose | plant | tv | natural | manipulable |
taxonomic | strawberry | grapes | fence | natural | manipulable |
taxonomic | Xmas tree | lily of the valley | lemon | natural | manipulable |
taxonomic | bee | hamster | avocado | natural | non manipulable |
taxonomic | camel | rooster | tree branch | natural | non manipulable |
taxonomic | catepillar | duck | artichoke | natural | non manipulable |
taxonomic | cow | snake | glasses | natural | non manipulable |
taxonomic | dog | bear | orange | natural | non manipulable |
taxonomic | fish | elephant | camper | natural | non manipulable |
taxonomic | lion | frog | sweater | natural | non manipulable |
taxonomic | monkey | swan | star | natural | non manipulable |
taxonomic | mouse | turtle | mountain | natural | non manipulable |
taxonomic | sheep | spider | raft | natural | non manipulable |
taxonomic | sparrow | dolfin | rocket | natural | non manipulable |
taxonomic | squirrel | goat | tire | natural | non manipulable |
thematic | axe | wood | cable car | artifact | manipulable |
thematic | bowl | toast | door | artifact | manipulable |
thematic | dress | little girl | shark | artifact | manipulable |
thematic | glass | fruit juice | antelope | artifact | manipulable |
thematic | hammer | nail | fox | artifact | manipulable |
thematic | mitten | skis | celery | artifact | manipulable |
thematic | notebook | school boy | rhinoceros | artifact | manipulable |
thematic | paintbrush | easel | holly | artifact | manipulable |
thematic | saw | carpenter | green onion | artifact | manipulable |
thematic | shirt | iron | hot air balloon | artifact | manipulable |
thematic | sneaker shoe | football | octopus | artifact | manipulable |
thematic | spoon | yogurt | crocodile | artifact | manipulable |
thematic | bed | baby sleeping | flag | artifact | non manipulable |
thematic | car | traffic light | scarf | artifact | non manipulable |
thematic | castle | knight | pants | artifact | non manipulable |
thematic | couch | cushion | peacock | artifact | non manipulable |
thematic | motorbike | bomber jacket | pencil | artifact | non manipulable |
thematic | plane | sun | kitchen counter | artifact | non manipulable |
thematic | sink | toothbrush | donkey | artifact | non manipulable |
thematic | small boat | oars | light bulb | artifact | non manipulable |
thematic | stove | pan | jump rope | artifact | non manipulable |
thematic | tent | feather | domino | artifact | non manipulable |
thematic | train | business man | radish | artifact | non manipulable |
thematic | truck | road | arm chair | artifact | non manipulable |
thematic | apple | knife | crescent moon | natural | manipulable |
thematic | carrot | rake | toes | natural | manipulable |
thematic | cherries | basket | chest of drawers | natural | manipulable |
thematic | daisy | butterfly | whale | natural | manipulable |
thematic | ear | phone | bat | natural | manipulable |
thematic | hand | ring | cloud | natural | manipulable |
thematic | ivy | pot cooked (smelly) |
letter | natural | manipulable |
thematic | nose | chicken | church | natural | manipulable |
thematic | potato | fries | giraffe | natural | manipulable |
thematic | rose | watering can | igloo | natural | manipulable |
thematic | strawberry | jam | cobweb | natural | manipulable |
thematic | Xmas tree | gift | acorn | natural | manipulable |
thematic | bee | finger with sting | book | natural | non manipulable |
thematic | camel | palmtree | round table | natural | non manipulable |
thematic | catepillar | leaf | faucet | natural | non manipulable |
thematic | cow | grass | toothpaste | natural | non manipulable |
thematic | dog | animal dish | santa claus | natural | non manipulable |
thematic | fish | aquarium plant | corn | natural | non manipulable |
thematic | lion | cage | bib | natural | non manipulable |
thematic | monkey | banana | alarm clock | natural | non manipulable |
thematic | mouse | small wheel | candle | natural | non manipulable |
thematic | sheep | wool ball | peach | natural | non manipulable |
thematic | sparrow | eggs | soap | natural | non manipulable |
thematic | squirrel | hazelnuts | cactus | natural | non manipulable |
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Andres M, Pelgrims B, Olivier E. Distinct contribution of the parietal and temporal cortex to hand configuration and contextual judgements about tools. Cortex; a Journal Devoted to the Study of the Nervous System and Behavior. 2013;49(8):2097–2105. doi: 10.1016/j.cortex.2012.11.013. [DOI] [PubMed] [Google Scholar]
- Assmus A, Giessing C, Weiss PH, Fink GR. Functional interactions during the retrieval of conceptual action knowledge: an fMRI study. Journal of Cognitive Neuroscience. 2007;19(6):1004–1012. doi: 10.1162/jocn.2007.19.6.1004. [DOI] [PubMed] [Google Scholar]
- Avants BB, Schoenemann PT, Gee JC. Lagrangian frame diffeomorphic image registration: Morphometric comparison of human and chimpanzee cortex. Medical Image Analysis. 2006;10:397–412. doi: 10.1016/j.media.2005.03.005. [DOI] [PubMed] [Google Scholar]
- Beauchamp MS, Martin A. Grounding object concepts in perception and action: evidence from fMRI studies of tools. Cortex; a Journal Devoted to the Study of the Nervous System and Behavior. 2007;43(3):461–468. doi: 10.1016/s0010-9452(08)70470-2. [DOI] [PubMed] [Google Scholar]
- Binder JR, Desai RH, Graves WW, Conant LL. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex (New York, N.Y. 1991) 2009;19(12):2767–2796. doi: 10.1093/cercor/bhp055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bracci S, Cavina-Pratesi C, Ietswaart M, Caramazza A, Peelen MV. Closely overlapping responses to tools and hands in left lateral occipitotemporal cortex. Journal of Neurophysiology. 2012;107(5):1443–1456. doi: 10.1152/jn.00619.2011. [DOI] [PubMed] [Google Scholar]
- Bright P, Moss H, Tyler LK. Unitary vs multiple semantics: PET studies of word and picture processing. Brain and Language. 2004;89(3):417–432. doi: 10.1016/j.bandl.2004.01.010. [DOI] [PubMed] [Google Scholar]
- Buxbaum LJ, Kyle KM, Menon R. On beyond mirror neurons: internal representations subserving imitation and recognition of skilled object-related actions in humans. Brain Research. Cognitive Brain Research. 2005;25(1):226–239. doi: 10.1016/j.cogbrainres.2005.05.014. [DOI] [PubMed] [Google Scholar]
- Campanella F, D’Agostini S, Skrap M, Shallice T. Naming manipulable objects: anatomy of a category specific effect in left temporal tumours. Neuropsychologia. 2010;48(6):1583–1597. doi: 10.1016/j.neuropsychologia.2010.02.002. [DOI] [PubMed] [Google Scholar]
- Caspers S, Zilles K, Laird AR, Eickhoff SB. ALE meta-analysis of action observation and imitation in the human brain. NeuroImage. 2010;50(3):1148–1167. doi: 10.1016/j.neuroimage.2009.12.112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chao LL, Weisberg J, Martin A. Experience-dependent modulation of category-related cortical activity. Cerebral Cortex. 2002;12(5):545–551. doi: 10.1093/cercor/12.5.545. [DOI] [PubMed] [Google Scholar]
- Chen Q, Ye C, Liang X, Cao B, Lei Y, Li H. Automatic processing of taxonomic and thematic relations in semantic priming - Differentiation by early N400 and late frontal negativity. Neuropsychologia. 2014;64C:54–62. doi: 10.1016/j.neuropsychologia.2014.09.013. [DOI] [PubMed] [Google Scholar]
- Cree GS, McRae K. Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns) Journal of Experimental Psychology. General. 2003;132(2):163–201. doi: 10.1037/0096-3445.132.2.163. [DOI] [PubMed] [Google Scholar]
- Damasio AR. Time-locked multiregional retroactivation: a systems-level proposal for the neural substrates of recall and recognition. Cognition. 1989;33(1–2):25–62. doi: 10.1016/0010-0277(89)90005-x. [DOI] [PubMed] [Google Scholar]
- Davidoff J, Roberson D. Preserved thematic and impaired taxonomic categorisation: A case study. Language and Cognitive Processes. 2004;19(1):137–174. [Google Scholar]
- de Zubicaray GI, Hansen S, McMahon KL. Differential processing of thematic and categorical conceptual relations in spoken word production. Journal of Experimental Psychology. General. 2013;142(1):131–142. doi: 10.1037/a0028717. [DOI] [PubMed] [Google Scholar]
- Devlin JT, Moore CJ, Mummery CJ, Gorno-Tempini ML, Phillips Ja, Noppeney U, Price CJ. Anatomic constraints on cognitive theories of category specificity. NeuroImage. 2002;15(3):675–685. doi: 10.1006/nimg.2001.1002. [DOI] [PubMed] [Google Scholar]
- Estes Z, Golonka S, Jones LL. Thematic thinking: The apprehension and consequences of thematic relations. Psychology of Learning and Motivation. 2011;54:249–294. [Google Scholar]
- Farah MJ, McClelland JL. A computational model of semantic memory impairment: modality specificity and emergent category specificity. Journal of Experimental Psychology. General. 1991;120(4):339–357. [PubMed] [Google Scholar]
- Fernandino L, Binder JR, Desai RH, Pendl SL, Humphries CJ, Gross WL, Seidenberg MS. Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics. 2015:1–17. doi: 10.1093/cercor/bhv020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gabbard C, Caçola P, Cordova A. Is there an advanced aging effect on the ability to mentally represent action? Archives of Gerontology and Geriatrics. 2011;53(2):206–209. doi: 10.1016/j.archger.2010.10.006. [DOI] [PubMed] [Google Scholar]
- Holmes CJ, Hoge R, Collins L, Woods R, Toga AW, Evans AC. Enhancement of MR images using registration for signal averaging. Journal of Computer Assisted Tomography. 1998;22:324–333. doi: 10.1097/00004728-199803000-00032. [DOI] [PubMed] [Google Scholar]
- Humphreys GF, Newling K, Jennings C, Gennari SP. Motion and actions in language: semantic representations in occipito-temporal cortex. Brain and Language. 2013;125(1):94–105. doi: 10.1016/j.bandl.2013.01.008. [DOI] [PubMed] [Google Scholar]
- Jackson RL, Hoffman P, Pobric G, Lambon Ralph MA. The Nature and Neural Correlates of Semantic Association versus Conceptual Similarity. Cerebral Cortex (New York, N.Y.: 1991) 2015;25(11):4319–4333. doi: 10.1093/cercor/bhv003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jouravlev O, McRae K. Thematic relatedness production norms for 100 object concepts. Behavior Research Methods. 2015 doi: 10.3758/s13428-015-0679-8. [DOI] [PubMed] [Google Scholar]
- Kable JW, Lease-Spellmeyer J, Chatterjee A. Neural substrates of action event knowledge. Journal of Cognitive Neuroscience. 2002;14(5):795–805. doi: 10.1162/08989290260138681. [DOI] [PubMed] [Google Scholar]
- Kalénine S, Buxbaum LJ, Coslett HB. Critical brain regions for action recognition: lesion symptom mapping in left hemisphere stroke. Brain: A Journal of Neurology. 2010;133(11):3269–3280. doi: 10.1093/brain/awq210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kalénine S, Mirman D, Buxbaum LJ. A combination of thematic and similarity-based semantic processes confers resistance to deficit following left hemisphere stroke. Frontiers in Human Neuroscience. 2012;6:106. doi: 10.3389/fnhum.2012.00106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kalénine S, Mirman D, Middleton EL, Buxbaum LJ. Temporal dynamics of activation of thematic and functional knowledge during conceptual processing of manipulable artifacts. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2012;38(5):1274–1295. doi: 10.1037/a0027626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kalénine S, Peyrin C, Pichat C, Segebarth C, Bonthoux F, Baciu M. The sensory-motor specificity of taxonomic and thematic conceptual relations: a behavioral and fMRI study. NeuroImage. 2009;44(3):1152–1162. doi: 10.1016/j.neuroimage.2008.09.043. [DOI] [PubMed] [Google Scholar]
- Kellenbach ML, Brett M, Patterson K. Actions speak louder than functions: the importance of manipulability and action in tool representation. Journal of Cognitive Neuroscience. 2003;15(1):30–46. doi: 10.1162/089892903321107800. [DOI] [PubMed] [Google Scholar]
- Kourtzi Z, Kanwisher N. Activation in human MT/MST by static images with implied motion. Journal of Cognitive Neuroscience. 2000;12(1):48–55. doi: 10.1162/08989290051137594. [DOI] [PubMed] [Google Scholar]
- Lewis Ga, Poeppel D, Murphy GL. The neural bases of taxonomic and thematic conceptual relations: An MEG study. Neuropsychologia. 2015;68:176–189. doi: 10.1016/j.neuropsychologia.2015.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis JW. Cortical networks related to human use of tools. The Neuroscientist: A Review Journal Bringing Neurobiology, Neurology and Psychiatry. 2006;12(3):211–231. doi: 10.1177/1073858406288327. [DOI] [PubMed] [Google Scholar]
- Lin EL, Murphy GL. Thematic relations in adults’ concepts. Journal of Experimental Psychology. General. 2001;130(1):3–28. doi: 10.1037/0096-3445.130.1.3. [DOI] [PubMed] [Google Scholar]
- Lingnau A, Downing PE. The lateral occipitotemporal cortex in action. Trends in Cognitive Science. 2015;19(5):268–277. doi: 10.1016/j.tics.2015.03.006. [DOI] [PubMed] [Google Scholar]
- Maguire MJ, Brier MR, Ferree TC. EEG theta and alpha responses reveal qualitative differences in processing taxonomic versus thematic semantic relationships. Brain and Language. 2010;114(1):16–25. doi: 10.1016/j.bandl.2010.03.005. [DOI] [PubMed] [Google Scholar]
- Martin A, Kyle Simmons W, Beauchamp MS, Gotts SJ. Is a single “hub”, with lots of spokes, an accurate description of the neural architecture of action semantics: Comment on “Action semantics: A unifying conceptual framework for the selective use of multimodal and modality-specific object knowledge” by v. Physics of Life Reviews. 2014;11(2):261–262. doi: 10.1016/j.plrev.2014.01.002. [DOI] [PubMed] [Google Scholar]
- Merck C, Jonin P-Y, Laisney M, Vichard H, Belliard S. When the zebra loses its stripes but is still in the savannah: results from a semantic priming paradigm in semantic dementia. Neuropsychologia. 2014;53:221–232. doi: 10.1016/j.neuropsychologia.2013.11.024. [DOI] [PubMed] [Google Scholar]
- Mirman D, Graziano KM. Individual differences in the strength of taxonomic versus thematic relations. Journal of Experimental Psychology: General. 2012a;141(4):601–609. doi: 10.1037/a0026451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mirman D, Graziano KM. Damage to temporo-parietal cortex decreases incidental activation of thematic relations during spoken word comprehension. Neuropsychologia. 2012b;50(8):1990–1997. doi: 10.1016/j.neuropsychologia.2012.04.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Noonan KA, Jefferies E, Corbett F, Lambon Ralph MA. Elucidating the nature of deregulated semantic cognition in semantic aphasia: evidence for the roles of prefrontal and temporo-parietal cortices. Journal of Cognitive Neuroscience. 2010;22(7):1597–1613. doi: 10.1162/jocn.2009.21289. [DOI] [PubMed] [Google Scholar]
- Noppeney U. The neural systems of tool and action semantics: a perspective from functional imaging. Journal of Physiology, Paris. 2008;102(1–3):40–49. doi: 10.1016/j.jphysparis.2008.03.009. [DOI] [PubMed] [Google Scholar]
- Osborne JG, Heath J. Predicting taxonomic and thematic relational responding. The Analysis of Verbal Behavior. 2003;19:55–89. doi: 10.1007/BF03392982. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perini F, Caramazza A, Peelen MV. Left occipitotemporal cortex contributes to the discrimination of tool-associated hand actions: fMRI and TMS evidence. Frontiers in Human Neuroscience. 2014;8:591. doi: 10.3389/fnhum.2014.00591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Quandt LC, Chatterjee A. Rethinking actions: implementation and association. Wiley Interdisciplinary Reviews. Cognitive Science. 2015;6(6):483–490. doi: 10.1002/wcs.1367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rorden C, Karnath H-O, Bonilha L. Improving lesion-symptom mapping. Journal of Cognitive Neuroscience. 2007;19(7):1081–1088. doi: 10.1162/jocn.2007.19.7.1081. [DOI] [PubMed] [Google Scholar]
- Sachs O, Weis S, Krings T, Huber W, Kircher T. Categorical and thematic knowledge representation in the brain: neural correlates of taxonomic and thematic conceptual relations. Neuropsychologia. 2008;46(2):409–418. doi: 10.1016/j.neuropsychologia.2007.08.015. [DOI] [PubMed] [Google Scholar]
- Sachs O, Weis S, Zellagui N, Huber W, Zvyagintsev M, Mathiak K, Kircher T. Automatic processing of semantic relations in fMRI: neural activation during semantic priming of taxonomic and thematic categories. Brain Research. 2008;1218(2):194–205. doi: 10.1016/j.brainres.2008.03.045. [DOI] [PubMed] [Google Scholar]
- Sachs O, Weis S, Zellagui N, Sass K, Huber W, Zvyagintsev M, Kircher T. How different types of conceptual relations modulate brain activation during semantic priming. Journal of Cognitive Neuroscience. 2011;23(5):1263–1273. doi: 10.1162/jocn.2010.21483. [DOI] [PubMed] [Google Scholar]
- Sass K, Sachs O, Krach S, Kircher T. Taxonomic and thematic categories: Neural correlates of categorization in an auditory-to-visual priming task using fMRI. Brain Research. 2009;1270:78–87. doi: 10.1016/j.brainres.2009.03.013. [DOI] [PubMed] [Google Scholar]
- Scheuner N, Bonthoux F, Cannard C, Blaye A. The role of associative strength and conceptual relations in matching tasks in 4- and 6-year-old children. International Journal of Psychology. 2004;39(4):290–304. [Google Scholar]
- Schubotz RI, Wurm MF, Wittmann MK, von Cramon DY. Objects tell us what action we can expect: dissociating brain areas for retrieval and exploitation of action knowledge during action observation in fMRI. Frontiers in Psychology. 2014;5:636. doi: 10.3389/fpsyg.2014.00636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schwartz MF, Brecher AR, Whyte J, Klein MG. A patient registry for cognitive rehabilitation research: a strategy for balancing patients’ privacy rights with researchers’ need for access. Archives of Physical Medicine and Rehabilitation. 2005;86(9):1807–1814. doi: 10.1016/j.apmr.2005.03.009. [DOI] [PubMed] [Google Scholar]
- Schwartz MF, Kimberg DY, Walker GM, Brecher AR, Faseyitan OK, Dell GS, Coslett HB. Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(20):8520–8524. doi: 10.1073/pnas.1014935108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tarhan LY, Watson CE, Buxbaum LJ. Shared and Distinct Neuroanatomic Regions Critical for Tool-related Action Production and Recognition: Evidence from 131 Left-hemisphere Stroke Patients. Journal of Cognitive Neuroscience. 2015;27(12):2491–2511. doi: 10.1162/jocn_a_00876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tranel D, Kemmerer D, Adolphs R, Damasio H, Damasio AR. Neural correlates of conceptual knowledge for actions. Cognitive Neuropsychology. 2003;20(3):409–432. doi: 10.1080/02643290244000248. [DOI] [PubMed] [Google Scholar]
- Tsagkaridis K, Watson CE, Jax Sa, Buxbaum LJ. The role of action representations in thematic object relations. Frontiers in Human Neuroscience. 2014;8:140. doi: 10.3389/fnhum.2014.00140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tyler LK, Moss HE. Towards a distributed account of conceptual knowledge. Trends in Cognitive Sciences. 2001;5(6):244–252. doi: 10.1016/s1364-6613(00)01651-x. [DOI] [PubMed] [Google Scholar]
- Tyler LK, Stamatakis EA, Dick E, Bright P, Fletcher P, Moss H. Objects and their actions: evidence for a neurally distributed semantic system. NeuroImage. 2003;18(2):542–557. doi: 10.1016/s1053-8119(02)00047-2. [DOI] [PubMed] [Google Scholar]
- Urgesi C, Candidi M, Avenanti A. Neuroanatomical substrates of action perception and understanding: an anatomic likelihood estimation meta-analysis of lesion-symptom mapping studies in brain injured patients. Frontiers in Human Neuroscience. 2014 May;8:1–17. doi: 10.3389/fnhum.2014.00344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wamain Y, Pluciennicka E, Kalénine S. A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations. Neuropsychologia. 2015;71:28–37. doi: 10.1016/j.neuropsychologia.2015.02.034. [DOI] [PubMed] [Google Scholar]
- Watson CE, Cardillo ER, Bromberger B, Chatterjee A. The specificity of action knowledge in sensory and motor systems. Frontiers in Psychology. 2014;5:494. doi: 10.3389/fpsyg.2014.00494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watson CE, Cardillo ER, Ianni GR, Chatterjee A. Action concepts in the brain: an activation likelihood estimation meta-analysis. Journal of Cognitive Neuroscience. 2013;25(8):1191–1205. doi: 10.1162/jocn_a_00401. [DOI] [PubMed] [Google Scholar]
- Wu DH, Waller S, Chatterjee A. The functional neuroanatomy of thematic role and locative relational knowledge. Journal of Cognitive Neuroscience. 2007;19(9):1542–1555. doi: 10.1162/jocn.2007.19.9.1542. [DOI] [PubMed] [Google Scholar]