Motor Experience Influences Object Knowledge

Abstract

An object’s perceived readiness-for-action (e.g., its size, the degree of rotation from its canonical position, or the user’s viewpoint) can influence semantic knowledge retrieval. Yet, the organization of object knowledge may also be affected by body-specific sensorimotor experiences. Here, we investigated whether people’s history of performing motor actions with their hands influences the knowledge they store and retrieve about graspable objects. We compared object representations between healthy right- and left-handers (Experiment 1), and between unilateral stroke patients, whose motor experience was changed by impairment of either their right or left hand (Experiment 2). Participants saw pictures of graspable everyday items with the handles oriented toward either the left or right hand, and they generated the type of grasp they would employ (i.e., clench or pinch) when using each object, responding orally. In both experiments, hand dominance and object orientation interacted to predict response times. In Experiment 1, judgments were fastest when objects were oriented toward the right hand in right-handers, but not in left-handers. In Experiment 2, judgments were fastest when objects were oriented toward the left hand in patients who had lost the use of their right hand, even though these patients were right-handed prior to brain injury. Results suggest that at least some aspects of object knowledge are determined by motor experience, and can be changed by new patterns of motor experience. People with different bodily characteristics, who interact with objects in systematically different ways, form correspondingly different neurocognitive representations of the same common objects.

How does motor experience contribute to our knowledge about objects? Here we investigated whether people with systematically different patterns of motor experience form correspondingly different neurocognitive representations of common manipulable objects. Our knowledge for certain object categories, such as manipulable tools, depends on the integration of visual information, manipulation information, and functional information. The dependence of these sensory, motor, and abstract representations on each other has been the topic of a large body of research (for a review, see Barsalou, 2016). Several behavioral studies have shown that perceptual distractors or action-related primes substantially influence lexical or semantic tasks involving common tools. When asked to click on one of four pictures in response to a heard word, for example, participants’ eye movements drifted toward distractor items that shared either perceptual (color) or abstract (function) features with the target object, but not toward items that did not share these features (Yee, Huffstetler, & Thompson-Schill, 2011). Similarly, participants were faster in a lexical decision task if the target tool word was preceded by a word referring to an object that involved the same, versus different, mode of manipulation (e.g., typewriter-piano, Myung, Blumstein, & Sedivy, 2006). Likewise, subjects were slower in selecting an object from an array of tools in the presence of items involving the same action, compared to objects that did not share these action elements (Lee, Middleton, Mirman, Kalénine, & Buxbaum, 2012).

Early studies on the neural systems supporting object knowledge have further revealed distinct activation patterns in ventral premotor and left posterior parietal cortex during viewing, imagining, or naming pictures of manipulable objects (Chao & Martin, 2000; Chao, Weisberg, & Martin, 2002; Decety et al., 1994; Martin & Weisberg, 2003; Paulus, Elk, & Bekkering, 2012). Even in the absence of task-specific instructions, passively observing tools automatically evokes action-related representations (Grafton, Fadiga, Arbib, & Rizzolatti, 1997; see also Johnson-Frey et al., 2003; Johnson-Frey, Newman-Norlund, & Grafton, 2005). Further evidence from patients with semantic dementia (that selectively affects anterior temporal cortex) and apraxia (that is typically due to left or bilateral parietal lobe damage) suggests that the posterior parietal cortex likely supports affordance-guided motor plans by integrating perceptual object properties within a certain context (e.g., using novel objects to solve a mechanical problem), whereas for tools with an established function due to experience this integration appears to require the contribution of semantic knowledge as supported by inferior temporal structures (Gainotti, 2000; 2006; Hodges, Spatt, & Patterson, 1999). Thus, a substantial body of work has shown that accessing knowledge about the function, manipulation of, and action with tools is associated with the integrated activity of an extensive network of dorsal parietal and ventral temporal brain regions (Bar et al., 2001; Bartolo, Daumüller, Sala, & Goldenberg, 2007; Boronat et al., 2005; Buxbaum & Saffran, 2002; Bub & Masson, 2006; Buxbaum, Kyle, Tang, & Detre, 2006; Buxbaum, Schwartz, & Carew, 1997; Buxbaum, Sirigu, Schwartz, & Klatzky, 2003; Creem-Regehr & Lee, 2005; Ebisch et al., 2007; Goodale & Milner, 1992; Kellenbach, Brett, & Patterson, 2003; Silveri & Ciccarelli, 2009; Watson & Buxbaum, 2014; Weisberg, van Turennout, & Martin, 2006).

Effects of Perceptual Properties on Object Knowledge Retrieval

A possible consequence of the spontaneous engagement of several cortical systems for object knowledge is the presence of congruency effects of perceptual object properties on the recognition of (or action with) graspable tools. Several studies have shown that an object’s perceived readiness-for-action, substantially influences semantic knowledge retrieval. For example, object recognition depends on both the viewer’s viewpoint and the extent of the object’s rotation from its canonical position (Gregory & McCloskey, 2010; Wraga, Creem, & Proffitt, 1999; 2000). Although perceptual exposure to objects automatically evokes their potential for action as discussed earlier, the positioning of an object with respect to the observer can influence access to these action properties, likely due to the dependence of imaginary rotation mechanisms on the same left parietal networks that are engaged in motor planning (Creem et al., 2001). Additionally, graspable objects tend to be categorized according to a canonical-for-grasp viewpoint; as a result, objects in altered orientations are recognized slower and less accurately than objects in the canonical orientation; this recognition process appears to occur by similar mental rotation adjustments of the perceptual coordinate system to reorient the object to the canonical position for object use (Graf, Kaping, & Bülthoff, 2005; Kravitz, Vinson, & Baker, 2008; Petit, Pegna, Harris, & Michel, 2005).

Access to knowledge for manmade objects is further influenced by their size and their orientation for action. It has been shown, for example, that larger and familiar objects elicited faster responses in a categorization task (i.e., deciding whether an object is natural or manufactured) relative to smaller and unfamiliar objects, irrespective of size being extraneous to the task; this effect was further moderated by the congruency between the type of grip required for the response and object affordances (Grèzes, Tucker, Armony, Ellis, & Passingham, 2003). Similarly, when asked to decide whether an object was upright or inverted, participants were faster and more accurate when the object’s orientation for action was preferentially compatible with the hand pressing a key to provide a response (e.g., a pan with its handle in the orientation compatible with a reach-and-grasp movement by the right hand, when the right hand was used to make the response; Tucker & Ellis, 1998), although instructions to consider how each object is being used canonically may have influenced participants’ judgments in this study (see Bub & Masson, 2010; Yu, Abrams, & Zacks, 2014; see also Cho & Proctor, 2010). Moreover, objects with handles were stronger distractors for actions involving the hands (grasping, pointing), predominantly when the orientation of the distractor’s handle was compatible with the acting hand (Pavese & Buxbaum, 2002). Overall, these results indicate that access to semantic knowledge for manmade tools is influenced by contextual properties (e.g., observer viewpoint, object size and orientation for action), particularly under circumstances that highlight action-relevant components of object knowledge, such as mode of manipulation (Yee & Thompson-Schill, 2016).

Access to semantic information is equally necessary for meaningful object use (Arbib, 2010; Bub & Masson, 2006; Humphreys & Riddoch, 2007; Yee, Drucker, & Thompson-Schill, 2010; Yee et al., 2011). For example, performing a concurrent task that imposes demands on semantic memory (but not a non-semantic visuospatial task) interferes with participants’ ability to grasp tools by their handles in a manner appropriate for object use (Creem & Proffitt, 2001). Thus, manual experience is thought to shape the connections between an object’s physical properties (e.g., its form) and the specific motor plans associated with its canonical function.

Indeed, information about action plans appears to be an essential component of object knowledge: A study by Yee and colleagues (Yee, Chrysikou, Hoffman, & Thompson-Schill, 2013) revealed that when participants’ hands were engaged in a task that involved movements that were incompatible with those typically associated with using graspable tools, access to knowledge about these manipulable (relative to non-manipulable) objects was selectively disrupted; what’s more, the amount of manual experience with the objects predicted the degree of interference from the concurrent motor task, thus strongly suggesting that action information is an integral part of (not simply peripheral to) tool representations (see also Bub & Masson, 2010, 2012; Masson, Bub, & Newton-Taylor, 2008a; Masson, Bub, & Warren, 2008b).

Effects of Body-Specific Experience on Object Knowledge Retrieval

Although extensive research has investigated perceptual factors that can affect access to knowledge for tools, the experiential bases of object knowledge have remained incompletely understood. If motor experience contributes to people’s knowledge about tools and other manipulable objects, then some aspects of this knowledge should be ‘body specific’ (Casasanto, 2009, 2011). That is, neurocognitive representations of objects should differ between individuals whose bodies cause them to interact with the objects in predictably different ways (e.g., between individuals with right- vs. left-hand dominance). Action execution with the non-dominant hand in the context of complex tasks has been associated with reliable performance costs in movement speed (Grosskopf & Kuhtz-Buschbeck, 2005; Vaughan, Barany, & Rios, 2012). Additionally, handedness has been shown to influence perception of object size, with graspable items placed in the dominant hand being judged as smaller than items placed in the non-dominant hand, even when apparent hand size is artificially manipulated (Linkenauger, Witt, & Proffitt, 2011). Moreover, when viewing movie clips of a rotating screwdriver, both right- and left-handed participants were slower to recognize whether the movie was depicting a screwing or an unscrewing motion if the orientation of rotation was incongruent with the participants’ handedness, particularly for angles in which the object’s handle was farther away from the participant (de’Sperati & Stucchi, 1997). This effect persisted when participants were asked to imagine using their dominant hand, but disappeared when they were asked to imagine their non-dominant hand, providing further support for the influence of hand dominance on object representations.

Although the effect of body-specific experiences on object representations has not been tested previously, studies show that people who perform actions differently also form correspondingly different neural representations of these actions. In one study, words for uni-manual actions (e.g., throw) preferentially elicited higher left premotor cortex activity relative to non-action verbs for right-handed participants, whereas for left-handed participants preferential activation was observed in right premotor cortex (Willems, Hagoort, & Casasanto, 2010). A similar hemispheric reversal was found when right- and left-handers were asked to create mental images of manual vs. non-manual actions (Willems, Toni, Hagoort, & Casasanto, 2009). Relatedly, hand dominance has been shown to influence the lateralization of action-relevant information in the inferior parietal and ventral temporal cortex (e.g., Garcea, Almeida, & Mahon, 2012). These results suggest that body-specific experience can influence many aspects of cognition including memory (Apel, Cangelosi, Ellis, Goslin, & Fisher, 2012), language comprehension (Willems et al., 2010; Willems, Labruna, D’Esposito, Ivry, & Casasanto, 2011), mental imagery (Willems et al., 2009), and word learning (de Nooijer, van Gog, Paas, & Zwaan, 2013). Hand dominance has been shown to shape the mental representation of even some abstract concepts such as the notions of good and bad (see Casasanto, 2009; Casasanto & Chrysikou, 2011; de la Fuente, Casasanto, & Santiago, 2015), and the neural representation of abstract personality traits like ambition and pride (Brookshire & Casasanto, 2012).

In line with these findings, activity in left ventrolateral premotor cortex during identification of images of tools relative to animals was moderated by participants’ hand dominance, such that the degree of right-handedness predicted the strength of left ventrolateral premotor cortex activation in response to tools (Kan, Kable, Van Scoyoc, Chatterjee, & Thompson-Schill, 2006). Nevertheless, the reverse pattern was not observed for right ventrolateral premotor cortex. Other studies have also reported similar asymmetries as a function of handedness. For example, de Nooijer et al. (2013) have found that right-handers learned novel object-manipulation words using pictures better with right-handed, relative to left-handed and bimanual, first-person perspectives, whereas left-handers did not show the same effect with left-handed, relative to right-handed and bimanual, perspectives. Linkenauger, Witt, Stefanucci, Bakdash, and Proffitt (2009) have also shown that right-handers perceived tools with handles that made them hard to grasp with their dominant hand to be farther away, relative to tools with handles easier to grasp; left-handers did not show this effect. Similarly, right- but not left-handed participants remembered more assembly instructions for objects with handles oriented in a direction congruent with the participants’ handedness (Apel et al., 2012). These findings are likely reflective of the more distributed motor representations (i.e., less laterality) in left- than in right-handers (Humphreys & Praamstra, 2002; Singh et al., 1998), which is possibly attributed to the higher frequency of left-handers observing right-handed actions in daily life or having to use their right hand to manipulate objects in a ‘right-handed world’ (e.g., Rocca, Falini, Comi, Scotti, & Filippi, 2008; Stins, Kadar, & Costall, 2001). Such differences in conceptual representations for objects as a function of hand preference provide support for domain-specific distributed models of conceptual knowledge and suggest that effects of handedness can reveal how body-specific experience influences the neural representations of manipulable objects.

The Present Study

Although past research has shown that an object’s perceived readiness-for-action can influence the retrieval of object knowledge, it is not clear to what extent object representations differ as a function of people’s prior (or potential) motor experience. We explored this question in two experiments by investigating whether the orientation of an object (right/left) interacts with participants’ sensorimotor experience as indicated by their hand dominance, by varying natural dominance (Experiment 1) or brain-injury induced dominance (Experiment 2). Most people are right-handed (see Corballis, 2003 for a review), and the majority of research in psychology and neuroscience is conducted in this majority group. The main goal of comparing right- and left-handers was not to understand how objects are represented by the minority group, but rather to use hand dominance as a testbed for investigating how motor experience contributes to our knowledge about objects. We reasoned that if knowledge about tools is determined, at least in part, by motor experience, then tasks that require this knowledge to be retrieved would engage body-specific neurocognitive simulations of tool-relevant actions. These simulations will reflect the ways in which different individuals would use the same tools differently, according to the particulars of their bodies (in this case, of their hands; see Casasanto, 2009, 2014).

To test this hypothesis, in Experiment 1 we asked left- and right-handed participants to make a manipulation judgment pertaining to the grip required for canonical use of graspable objects, which were optimally oriented for action either by the left or by the right hand. We predicted that, due to the engagement of body-specific motor simulations, grip judgments would be faster when the objects were oriented for use by the participant’s dominant hand (handedness-congruent orientation), and slower when they were oriented for use by the nondominant hand (handedness-incongruent orientation)—even though responses were made orally and did not require any hand actions (e.g., Creem et al., 2001; Grafton et al., 1997; Pelgrims, Olivier, & Andres, 2010). Response times (RTs), therefore, would be predicted by an interaction between object orientation (toward the left, toward the right) and hand dominance (left, right). Based on past studies suggesting a possible asymmetry between right- and left-handers in exhibiting such congruency effects (e.g., Apel et al., 2012; de Nooijer et al., 2013; Kan et al., 2006), we expected that this interaction might be driven primarily by the right-handed group because right-handers are likely to use tools with their dominant hands more consistently than left-handers, who live in a world of artifacts customized for right-handers. In Experiment 2, we investigated whether long-term changes in right-handers’ motor experience due to right- or left-hemisphere unilateral stroke differentially influenced the anticipated interaction between hand dominance and object orientation. If sensorimotor experience contributes to certain aspects of object knowledge (e.g., mode of manipulation), then long-term changes in how right-handers use their bodies to interact with objects should result in corresponding changes in object representations.

Experiment 1

Method

Participants

We tested 18 self-identified right-handed (10 males, mean age = 24.61 years) and 15 self-identified left-handed (6 males, mean age = 22.40 years); handedness was confirmed by the Edinburgh Handedness Inventory (EHI; Oldfield, 1971). All participants were native English speakers with normal or corrected-to-normal vision, and participated in this study for course credit after providing informed consent. The number of participants was determined based on the medium-large effect sizes (Cohen’s d ranging from 1.41 to 2.46) calculated from studies using similar tasks (Ping, Dhillon, & Beilock, 2009). Using these effect sizes and G*Power (version 3.1, Faul, Erdfelder, Buchner, & Lang, 2009) we estimated that a sample size of at least 14 participants per group would be necessary to detect an interaction between the between-subjects factor of handedness and the within-subjects factor of object orientation with a power of at least 95% at α = .05; our samples of 18 and 15 participants per condition, thus, allowed for satisfactory power to detect any effects. Based on participants’ EHI scores, the median handedness score of those self-identified as right-handed was EHI = 55 (interquartile range = 30), whereas the median handedness score of those self-identified as left-handed was EHI = −50 (interquartile range = 40). EHI scores between the groups differed significantly (p < .001), thus confirming participants’ self-reported handedness.

Materials

Images of familiar graspable objects were used as stimuli. Half of the graspable objects were randomly oriented to the right and the other half to the left. A pilot study verified that the graspable right- or left-oriented objects were perceived to be in the intended orientation. Participants in the pilot experiment (n = 21) provided the name of each object and rated each object as either better oriented for a left-handed person or a right-handed person. On the basis of this study, 96 black and white two-dimensional images of everyday objects were chosen as experimental stimuli, each with at least 80% agreement on object naming and at least 70% agreement on object orientation (i.e., whether the object was oriented for a right- or left-handed person). All images were presented against a white background. Forty-eight were images of graspable objects oriented to the right and 48 were images of graspable objects oriented to the left. Example stimuli are presented in Figure 1. A full list of the stimuli is available in the Appendix.

Figure 1.

Examples of stimuli: (a) left-oriented object (b) right-oriented object

Procedure

Following written consent, participants received instructions to perform a manipulation judgment task (adapted from Buxbaum et al., 2003), according to which they identified aloud, as quickly as possible, the type of grasp they would employ (i.e., clench or pinch) when using each object for its typical function (see Figure 2). They were also told to remain silent if they did not know the answer for a given object. Verbal responses were selected over key presses to eliminate any potential influences of manual response mode in conjunction with handedness on reaction times. Participants first completed a training session to familiarize themselves with the experimental procedure, as well as to verify that their responses were loud enough to be recorded. Next, participants completed four 5-minute blocks, each consisting of 24 experimental items. Twelve versions of the experiment were created with a random sequence of experimental objects included in each. Stimuli were presented on a standard computer monitor using E-prime software (Psychology Software Tools, Inc.). Responses were recorded through an E-prime-compatible microphone as well as by means of a Sony^® digital voice recorder. Each stimulus was presented separately against a light gray screen background. Each object appeared on the screen for 6000ms followed by a fixation cross screen for 3000ms. After the last trial, participants were asked to complete the EHI (Oldfield, 1971). The entire experiment lasted approximately one hour.

Figure 2.

Type of grasp options for participant responses in Experiments 1 & 2: (a) clench (b) pinch.

Results and Discussion

Voice onset reaction times (RTs) and participants’ responses were recorded. We analyzed median (not mean) RTs to avoid any disproportional influence of extreme RT values on mean RT data between groups (Ratcliff, 1993). The canonical grasp for each object was determined by an independent group of subjects (n = 19), which confirmed the experimenters’ classification of the objects as requiring either a clench or a pinch grasp during their canonical function. Accordingly, all responses in Experiment 1 were scored for accuracy, which was approximately 100% for both handedness groups. All omissions and incorrect responses (less than 1% of all answers) were excluded. Median RTs were subjected to a repeated measures 2 × 2 ANOVA, with self-reported handedness as the between-subjects factor (right handed or left handed) and object orientation (left-oriented or right-oriented) as the within-subjects factor. According to the results, there was no main effect for handedness (F [1, 31] = 1.77, p = .19, η² = .05; see Figure 3) and no main effect for orientation (F[1,31] = 0.74, p = .40, η² = .02; see Figure 3). Critically, however, the interaction between handedness and object orientation was significant (F[1, 31] = 5.85, p = .02, η² = .16; see Figure 3), showing, as predicted, that right- and left-oriented objects differentially affected reaction times by right- and left-handed subjects. To explore this effect further, pairwise comparisons with a Bonferroni-adjusted critical value at p < .025 showed that, as predicted, right-handers responded significantly faster to right- than left-oriented objects (t[17] = 2.66, p = .017, Cohen’s d = 0.65). Conversely, the tendency of left-handers to respond faster to left- than right-oriented objects was not statistically significant (t[18] = −1.12, p = .28, Cohen’s d = 0.27). We further conducted a repeated-measures ANOVA with handedness scores from the EHI as a continuous covariate and object orientation (left-oriented or right-oriented) as the within-subjects factor. According to the results, there was no main effect for orientation (F[1,31] = 0.56, p = .46, η² = .02); the interaction between handedness and object orientation did not reach statistical significance (F[1, 31] = 2.97, p = .095, η² = .09), but was in a direction consistent with the results of our main analysis.

Figure 3.

Voice onset latencies in milliseconds for the manipulation judgment task by object orientation and self-reported handedness in Experiment 1. Error bars indicate the standard error of the mean.

In Experiment 1 we predicted that making a manipulation decision about graspable objects would activate body-specific sensorimotor representations; thus, this task would lead to an interaction between participants’ handedness and object orientation. The results of the experiment supported this prediction: right-handed participants made the manipulation judgment faster for right-oriented objects than left-oriented objects; consistent with past research, the reverse pattern for left-handed participants was not statistically significant, despite trending in the predicted direction. When we used handedness scores from the EHI as a continuous covariate, the interaction between handedness and object orientation did not reach statistical significance. This result is likely attributed to the fact that the EHI is a general measure of handedness and may not necessarily reflect hand preference for the particular objects presented in this study. Overall, object orientation influenced right- and left-handers differently on this task. These results provide support for the contribution of body-specific representations to the organization of object knowledge: right-handers were faster to make semantic judgments about an object’s mode of manipulation when the objects were oriented such that they could easily use them, given the specifics of their bodies (see also Creem et al., 2001; Grafton et al., 1997; Pelgrims et al., 2010).

Experiment 2

The results of Experiment 1 support the conclusion that sensorimotor experience influences certain aspects of object knowledge (e.g., mode of manipulation). Thus, it follows that changes in how right-handers use their bodies to interact with objects should lead to corresponding changes in the objects’ representation. In Experiment 2, we investigated whether long-term changes in motor fluency following unilateral cerebrovascular accident (CVA) that resulted in right or left hemiparesis (weakness or paralysis on one side of the body) differentially influence the interaction between handedness and object orientation in the context of the manipulation judgment task. Based on the results of Experiment 1, we predicted that patients with right-hemisphere CVAs (left hemiparesis), who continued to use their right hand, would show better performance for right- than left-oriented objects. Patients with left-hemisphere CVAs (right hemiparesis), however, who were only able to use their left hand post-stroke, would show the reverse effect.

Method

Participants

All patients (N = 9) were right-handed prior to brain injury. Left-hemisphere CVA led to right hemiparesis in five patients (2 males, mean age = 57.4 years), rendering them effectively left-handed, post-stroke. Right-hemisphere CVA led to left hemiparesis in the remaining four patients (all females, mean age = 56.5 years), preserving their natural right-handedness. Post-stroke handedness was measured via the EHI (Oldfield, 1971). The median handedness score was EHI = 90 (interquartile range = 15) for the left-hemiparesis group, and EHI = −72 (interquartile range = 50) for the right-hemiparesis group. EHI scores between the groups differed significantly (p < .001). Participants were tested at the Hospital of the University of Pennsylvania after they provided informed consent (for detailed patient demographic and clinical characteristics, see Table 1).

Table 1.

Patient demographic and clinical information for Experiment 2

Patient ID	Gender	Age (years)	Education (years)	EHI	Time post-stroke (years)	Locus of unilateral CVA	Lesion Template
BC236	M	63	18	−90	17	L frontal, parietal cortex and basal ganglia
CD141	F	51	16	−40	11	L insular, perisylvian cortex, and basal ganglia
XK342	F	56	12	−50	10	L frontal, parietal white matter; L occipital cortex and centrum semiovale
QN435	M	55	12	−80	8	L frontal, parietal cortex, and pons	Lesion template unavailable
LT85	F	62	15	−100	14	L insula and putamen
TS474	F	50	11	80	8	R parietal cortex
NH192	F	71	12	90	12	R thalamus
MB101	F	57	18	100	35	R temporal, and basal ganglia
KG593	F	48	12	90	4	R frontal, temporal, parietal cortex, and basal ganglia	Lesion template unavailable

Notes: F = Female, M = Male, R = right, L = left, EHI = Edinburgh Handedness Inventory, CVA = cerebrovascular accident. The location of the CVA was determined by the attending neurologist at the Hospital of the University of Pennsylvania, following visual inspection of post-stroke clinical computerized axial tomography (CAT) or magnetic resonance imaging (MRI) scans; the attending neurologist further provided lesion templates from each patient.

Materials

The materials for Experiment 2 were the same as the materials for Experiment 1. To accommodate a shorter testing session due to patient fatigue, we randomly selected 74 of the original 96 black and white two-dimensional images of everyday objects to use as experimental stimuli, of which 37 were images of graspable objects oriented to the right and 37 were images of graspable objects oriented to the left (see Appendix).

Procedure

The procedure was the same as the procedure for Experiment 1.

Results and Discussion

Voice onset reaction times and participants’ responses were recorded. All responses were scored for accuracy (with the canonical grasp for each object determined by an independent group of subjects, as in Experiment 1). All omissions (less than 1% of all answers) were excluded. Median RTs and proportion accuracy for each patient group are presented in Figures 4 and 5. Individual patient median RTs and accuracy rates are also presented in Figures 6 and 7. Because of the small sample size in Experiment 2, we report non-parametric tests of binary outcomes for this study. In line with our prediction, 3 of the 4 left-hemiparesis patients (75%) were faster to respond to right-oriented objects than left oriented objects, whereas all 5 of the right-hemiparesis patients (100%) were faster to respond to left-oriented than right oriented objects, consistent with the direction of their paresis (8 or 9 patients showing the predicted effect, binomial p = .039, two-tailed). For the critical group of right-hemiparesis patients alone the effect was marginally significant in line with our prediction (binomial p = 0.06, 2-tailed). Regarding accuracy, 2 of the 4 left-hemiparesis patients (50%) showed higher accuracy for right-oriented objects than left oriented objects, whereas 4 of the 5 right-hemiparesis patients (80%) showed higher accuracy for left-oriented than right oriented objects, but this difference was not statistically significant (6 of the 9 patients showing the predicted effect, binomial p = .51, two-tailed). Although our sample was small, it was sufficiently variable with regards to time from stroke occurrence (4–35 years for left hemiparesis patients; 8–17 years for right hemiparesis patients; 4–37 years, overall) to examine whether this variable modulated the RT compatibility effect. We performed a regression analysis to examine this possibility. Time from stroke occurrence did not predict the compatibility effect (orientation compatible RTs – orientation incompatible RTs). The results of the regression indicated that time from stroke explained < 1% of the variance (R²=.003, F[1,8] = 0.02, p = .89). Time from stroke did not significantly predict the RT compatibility effect (β = .05, p = .89).

Figure 4.

Median reaction times by object orientation and patient group in Experiment 2.

Figure 5.

Proportion accuracy by object orientation and patient group in Experiment 2.

Figure 6.

Individual patient median reaction times by object orientation in Experiment 2.

Figure 7.

Individual patient accuracy rates by object orientation in Experiment 2.

These results show that long-term changes in motor experience due to unilateral stroke influence people’s object knowledge. Premorbidly right-handed patients, who became effectively left-handed due to unilateral CVA, responded faster for left- than right-oriented objects in the context of the manipulation judgment task. In contrast, patients whose right-handedness was preserved post-stroke showed the opposite effect, similar to right-handers in Experiment 1. Thus, changes in body-specific sensorimotor experience strongly affect certain aspects of object knowledge (e.g., mode of manipulation) that require detailed processing of action-related information about the objects.

General Discussion

In two experiments we investigated whether object knowledge depends in part on an individual’s sensorimotor experience, as indexed by their hand dominance. The results of the first experiment showed that when right- and left-handers made judgments about an object’s mode of manipulation (i.e., grip judgments), the objects’ orientations differentially influenced the speed of these judgments: right-handers, but not left-handers, responded fastest when graspable objects were depicted as oriented toward their right hand. This was true even though all responses were made orally, without participants using the hands. These results could be interpreted as evidence of experience-based differences between right- and left-handers’ object representations, or alternatively, as evidence of innate differences in the organization of right- and left-handers’ semantic knowledge. Experiment 2 challenged this latter possibility. Patients who had undergone long-term changes in sensorimotor experience due to unilateral stroke performed the same manipulation judgment task used in the first experiment. All patients were right-handed prior to their brain injuries, but about half of them had lost the fluent use of their right arms and hands (i.e., right hemiparesis, n = 5), rendering them effectively left-handed. All of these right-hemiparesis patients were faster to make oral grip judgments when the object was oriented toward their left hand, whereas the left-hemiparesis patients showed the opposite pattern, suggesting that changes in motor experience produce corresponding changes in action-relevant (i.e., mode of manipulation) aspects of object representations.

In Experiment 1, the effect of object orientation was significant in right-handers, but only trended in the predicted direction in left-handers. This finding is in line with past research that has revealed similar asymmetries between right- and left-handed participants on tasks requiring the retrieval of action-relevant properties (e.g., manipulation mode) of common objects (e.g., Apel et al., 2012; de Nooijer et al., 2013; Linkenauger et al., 2009; Kan et al., 2006). Many common objects are designed for right-handers. As a result, left-handers are more familiar with right-handed actions and object perspectives than right-handers are with left-handed actions and object perspectives, and left-handers may show less of a hand preference for interacting with objects (see Stins et al., 2001). The organization of left-handers’ conceptual knowledge about objects might therefore show less hand specificity, reflecting their more extensive experience interacting with objects using the non-dominant hand, in addition to the dominant hand (see also Rocca et al., 2008). Unlike natural left-handers who may frequently use their right hand to interact with objects, the right hemiparesis patients in Experiment 2 were forced to use their left hand preferentially subsequent to their brain injury. Accordingly, they showed a complete reversal of the object orientation effect, compared to the left-hemiparesis patients.

These results are consistent with theories of object knowledge which posit that manmade object concepts are represented in a distributed system according to which different object attributes reflect different patterns of activation across brain networks involved in perceptual, action-based, or abstract thought (e.g., Allport, 1985; Martin, 2006; Thompson-Schill, 2003). The extent of the involvement of each of these networks for object knowledge retrieval is determined by a combination of factors, including the object’s size (e.g., Grèzes et al., 2003), the specific demands of a concurrent task (e.g., Creem & Proffitt, 2001; Yee et al., 2013), and, as we show here, the object’s orientation with regards to action (c.f., Graf et al., 2005).

Also consistent with our findings, past research has shown preferential activation of either left or right premotor cortex during action verb comprehension in right- and left-handed participants, respectively (Willems et al., 2010; 2011); in addition, increased left ventral premotor cortex activation during tool recognition can vary as a function of right-hand laterality (Kan et al., 2006). Our findings suggest that activation of these systems has consequences for the ability to access object knowledge.

Prior studies on the interdependence of multiple systems for tool concepts have shown that access to action-related information for object-specific grasping can be selectively interrupted by a semantic (but not a visuospatial) concurrent task (Creem & Proffitt, 2001). Studies on the effects of context on object recognition have further suggested that hand postures that are not optimal for canonical object use can affect object processing, by delaying decisions pertaining to the validity of functional relationships between object pairs (Borghi, Flumini, Natraj, & Wheaton, 2012), an effect that appears to be modulated by early activity in parietofrontal networks (Natraj et al., 2013). Our results are in line with these findings and suggest that the representation of manmade objects is organized across visual, tactile, functional, action/manipulation, and other properties that are at least partly determined by individual experience. As such, manipulable objects may hold a special status in semantic memory due to the contribution of multiple kinds of information for their representation (see Allport, 1985; Tyler & Moss, 2001; see also Creem-Regehr et al., 2005).

Overall, the results of the present experiments are in line with the body-specificity hypothesis (Casasanto, 2009, 2014; cf. Barsalou, 1999; Willems & Francken, 2012; Wilson, 2002), which proposes that access to action-related information is guided by body-specific mental simulations. Moreover, our findings argue in favor of a partially-distributed account of object knowledge (Garcea & Mahon, 2012; Grafton & Hamilton, 2007; Mahon, Milleville, Negri, Rumiati, Caramazza, & Martin, 2007; Mahon & Caramazza, 2008; Martin, 2007; Thompson-Schill, 2003; Tyler & Moss, 2001), according to which visual, motor, tactile, action-related, and abstract functional properties are part of the representation of object concepts, and can be differentially salient depending on individual experience, experimental manipulations, and task demands (Casasanto & Lupyan, 2015; Yee & Thompson-Schill, 2016). Although our two studies address the question of whether individual differences in sensorimotor experience interact with contextual factors (i.e., object orientation) to influence semantic judgments about an object’s proper mode of manipulation, they do not test whether manipulation knowledge is activated spontaneously whenever people think about objects. One possibility is that manipulation knowledge is only activated in contexts in which it is relevant, or potentially relevant. On this view, manipulation knowledge was activated in our tasks because the judgment concerned manipulation, but this knowledge might not be activated (or activated as strongly) in other contexts that direct attention to other aspects of objects (e.g., their color, the substance they are made of, etc.; Casasanto, Brookshire, & Ivry, 2015; Casasanto & Lupyan, 2015; Chrysikou, 2006, 2008; cf. Barsalou, 1983; Yee & Thompson-Schill, 2016). Alternatively, knowledge pertaining to the object’s proper mode of manipulation could be a central component of the object’s representation that is activated invariably, regardless of the context or of the nature of the task (Barsalou, 1982). If so, even tasks that do not involve manipulation judgments would elicit action-related aspects of the object’s representation, as we have shown in past work (Yee et al., 2013). Our conclusions here are compatible with either of these possibilities: The data suggest that differences in perceptuo-motor experience give rise to body-specific manipulation knowledge, which people activate (at minimum) when this knowledge is task-relevant. Further experiments are needed to determine the range of contexts under which body-specific manipulation knowledge is activated.

These data raise a question about the computational-level function (Marr, 1982) of neurocognitive simulations, and may help to adjudicate between two possible answers. Marr’s computational level of explanation concerns the why of cognition: When the brain creates simulations, what problem is it ‘trying’ to solve, or what process is it trying to facilitate? According to canonical theories of modality-specific simulation (e.g., Barsalou, 1999), simulations are reenactments of past experiences. In the present study, therefore, the observed effects would reflect participants reenacting their prior experiences of interacting with the specific stimulus objects. Alternatively, simulations could be “pre-enactments” of potential future experiences (Willems, Toni, Hagoort, & Casasanto, 2010; see also Adams, Shipp, & Friston, 2013; Casasanto & Gijssels, 2016; Chrysikou, 2014; Zwaan, 2004; Zwaan & Kaschak, 2008). On the pre-enactment view, symbol-driven simulations constitute partial preparation of neural systems that would be needed to engage physically with the symbol’s referent. Pre-enactment is computationally motivated (Marr, 1982) insomuch as symbols often precede their referents, so initiating perceptuomotor neural activity in response to the symbol can facilitate appropriate perceptuomotor responses to the referent. For example, hearing “Catch!” often signals that you will soon need to catch something, so the sooner you prepare neural systems for vision and action, the better. This predictive relationship between symbols and referents was especially strong throughout the majority the human brain’s history when symbolic communication necessarily happened in real time, before written words and pictures became ubiquitous. A similar predictive relationship obtains between objects and actions: Looking at an object often precedes interacting with that object. Partially preparing the relevant neural systems via simulation presumably makes this interaction more efficient, if and when the action is carried out (Willems et al., 2010).

Although our data are potentially compatible with either of these views, an examination of our stimulus list raises doubts about a reenactment-based interpretation of our results, and by extension, of this canonical notion of simulation. Although all of the stimulus objects should have been familiar to our participants, it seems unlikely that all participants had direct experience with all of the objects: How much experience did our young adult subjects in Experiment 1 have with a baster, a curling iron, or a pumice brush? How much postmorbid experience did our hemiparesis patients in Experiment 2 have with objects that require a strong non-dominant hand, like a peeler, a shovel, or a rifle? We do not have an accurate record of each participant’s history of experience with each stimulus object (or enough statistical power to evaluate congruity effects for each individual item), but the fact the participants may have had little or no experience with some objects is potentially problematic for a direct ‘reenactment’ view of simulation.

In the present experiments, motor simulations in response to object pictures may constitute the motor system’s partial preparation to act upon the depicted objects. Pre-enactments may be informed by prior sensorimotor experiences, either with the object itself or, through analogy, with objects that have similar affordances. But importantly, unlike reenactments, pre-enactments do not require direct prior experience with the target object. As such, in Experiment 2, the pre-enactment view would support the prediction that time from stroke should not have an effect on participants’ performance on this task; in contrast, a re-enactment view would suggest that time from stroke should be influential in decreasing latencies for items with compatible to post-stroke handedness orientation. Although our sample was small, it was sufficiently variable with regards to time from stroke occurrence to perform this analysis; the results showed that time from stroke occurrence did not significantly predict the RT compatibility effect, thus offering support for the pre-enactment view, which predicts that object pictures would elicit hand-specific motor simulations, giving rise to the observed congruity effects, whether or not the viewer had any direct prior (or any postmorbid) experience interacting with the depicted object. Experiments using novel objects (e.g., Weisberg et al., 2006) could further differentiate these alternative views of neurocognitive simulation, and advance our understanding of the timecourse over which body-specific motor experiences shape our object representations.

Appendix: List of Stimuli in Experiments 1 and 2

Experiment 1	Experiment 2
baster	beer mug
beer mug	bell
bell	broom
broom	bucket
bucket	chainsaw
chainsaw	chalk
chalk	clippers
clippers	clothespin
clothespin	coffee pot
coffee pot	comb
comb	cup
computer mouse	dustpan
cup	feather duster
curling iron	flashlight
drencher	fork
dustpan	frying pan
feather duster	golf club
fishing rod	gun
flashlight	hairbrush
fork	hairdryer
frying pan	hammer
glue gun	hand mirror
golf club	hand mixer
gun	hanger
hairbrush	hoe
hairdryer	ice-cream scoop
hammer	ice-scraper
hand mirror	iron
hand mixer	kettle
hanger	key
highlighter	kitchen knife
hockey stick	knife
hoe	ladle
ice pick	lighter
ice-cream scoop	lighter
ice-scraper	measuring cup
iron	mop
kettle	mop
key	mug
kitchen knife	oven mitt
knife	paintbrush
ladle	peeler
lighter	pen
lighter	pencil
mallet	pipe
measuring cup	pitcher
megaphone	pliers
mop	plunger
mop	razor
mug	remote control
nail file	rifle
oven mitt	ruler
paintbrush	saw
peeler	scissors
pen	screwdriver
pencil	shovel
perfume bottle	shower brush
pipe	shower head
pitcher	spatula
pizza slicer	spatula
plastic shovel	spoon
pliers	spork
plunger	squirt bottle
pumice brush	suitcase
razor	tennis racket
remote control	toilet brush
rifle	tongs
ruler	toothbrush
saw	travel mug
scissors	tweezers
screwdriver	umbrella
shovel	whisk
shower brush	wooden spoon
shower head	wrench
sifter
snuffer
spatula
spatula
spoon
spork
squeegee
squirt bottle
suitcase
sword
tennis racket
toilet brush
tongs
toothbrush
travel mug
tweezers
umbrella
USB Key
water gun
whisk
wooden spoon
wrench