Differential Tuning of Ventral and Dorsal Streams during the Generation of Common and Uncommon Tool Uses

Abstract

Our use of tools is situated in different contexts. Prior evidence suggests that diverse regions within the ventral and dorsal streams represent information supporting common tool use. However, given the flexibility of object concepts, these regions may be tuned to different types of information when generating novel or uncommon uses of tools. To investigate this, we collected fMRI data from participants who reported common or uncommon tool uses in response to visually presented familiar objects. We performed a pattern dissimilarity analysis in which we correlated cortical patterns with behavioral measures of visual, action, and category information. The results showed that evoked cortical patterns within the dorsal tool use network reflected action and visual information to a greater extent in the uncommon use group, whereas evoked neural patterns within the ventral tool use network reflected categorical information more strongly in the common use group. These results reveal the flexibility of cortical representations of tool use and the situated nature of cortical representations more generally.

INTRODUCTION

Tool use is a defining feature of human behavior (Ambrose, 2001; Wilson, 1999). Typically, humans use tools in common ways (e.g., we use a hammer to hammer a nail), and our tool use reflects conventionalized relationships between actions and objects (i.e., the hammer was designed to hammer nails). This ability has allowed humans to transform our physical environments, and because we are adept at learning the common uses of objects from each other (i.e., we can imitate each other and are skilled observational learners), we have transformed our social environments by building a culture of shared knowledge (see Johnson-Frey, 2004). In this way, common tool use shares features with other quintessentially human characteristics, including the conventional use of language and the use and transmission of moral conventions.

According to some authors, early hominids evolved specialized cognitive systems to create tools to efficiently address our goals (e.g., see Donald, 1991). One of the features of this early evolving system is its generative capacity, our ability to break down or categorize features of objects and combine them to solve problems (see Corballis, 2015, in press). For instance, we can categorize graspable things and hard things and combine them to create a tool that is effective at hammering. In most instances, common tool use reflects these conventionalized associations, allowing us to efficiently identify the features of existing tools that satisfy our goals. Importantly, however, we encounter tools in context, and in some contexts, these conventions can be broken. Given our generative capacity, we can analyze features of existing tools and apply them to problems for which they were not designed. For instance, to hammer a nail, we can identify any object with sufficient hardness and graspability, like a shoe. In this way, uncommon tool use, like language use, allows us to extend our problem-solving abilities in potentially infinite ways (see Goldenberg & Hagmann, 1998) and shows that object representations are not static; rather, information composing the object’s representation is situated in a cognitive context and therefore is activated in ways to meet task demands (Kalénine, Shapiro, Flumini, Borghi, & Buxbaum, 2014; see Yee & Thompson-Schill, 2016; Barsalou, 2015, for a review).

Much research has investigated which regions of the cortex reflect tool-related information during common tool use. The general findings from this research are that tool-related information is represented by the activity of a distributed left hemisphere network (i.e., the tool use network; Gallivan, McLean, Valyear, & Culham, 2013; Lewis, 2006; Johnson-Frey, 2004). This network is distributed across both the ventral and dorsal processing streams (see Milner & Goodale, 2008), and its activity during tool use is characterized by flexibly reorganized task-dependent interactions between them (see Hutchison & Gallivan, in press; also Stevens, Tessler, Peng, & Martin, 2015). Nodes of this network show differences in both functional connectivity (e.g., Garcea & Mahon, 2014) and anatomical connectivity (e.g., Caspers et al., 2011). They also show activation sensitivity. Within the ventral stream are regions showing sensitivity to tools (e.g., medial fusiform gyrus; Chao, Weisberg, & Martin, 2002; Chao, Haxby, & Martin, 1999; see Grill-Spector & Malach, 2004, for a review). Martin and colleagues (e.g., see Martin, 2007) have suggested that tool-preferring regions are attuned to shape and motion attributes.

The dorsal stream can be further subdivided (see Binkofski & Buxbaum, 2013; Johnson & Grafton, 2003; Rizzolatti & Matelli, 2003; see also Borra & Luppino, in press; Almeida, Fintzi, & Mahon, 2013). First, a bilateral dorso-dorsal stream processes information related to aspects of tool orientation (e.g., the lateral occipital parietal junction; Valyear, Culham, Sharif, Westwood, & Goodale, 2006) and the kinematics of the arm and hand during online grasping (e.g., the anterior intraparietal sulcus; Culham et al., 2003; Chao & Martin, 2000; see Culham & Valyear, 2006, for a review). Second, a left-lateralized ventro-dorsal stream processes information both about characteristic visual tool motion (e.g., the posterior middle temporal gyrus; Kalénine & Buxbaum, 2016; Beauchamp, Lee, Haxby, & Martin, 2002, 2003; see Beauchamp & Martin, 2007, for a review) and about learned sensorimotor associations between tools and actions (e.g., supramarginal gyrus; see also Buxbaum & Kalénine, 2010, for a review). The role of these regions is confirmed by neuropsychological studies showing that damage to critical nodes of the ventro-dorsal stream in the left hemisphere produces behavioral impairments in tool use (Salazar-López, Schwaiger, & Hermsdörfer, 2016) and the execution and recognition of pantomimed tool use (apraxia; Goldenberg, Hartmann, & Schlott, 2003; see Buxbaum, Shapiro, & Coslett, 2014, for a review; see also Borra et al., 2008; Zhong & Rockland, 2003, for anatomical evidence in nonhuman primates). Thus, the bilateral dorso-dorsal stream is specialized for online aspects of action control, whereas the left ventro-dorsal stream is critical for the representation of learned tool actions (see also Garcea, Kristensen, Almeida, & Mahon, 2016; Kristensen, Garcea, Mahon, & Almeida, 2016).

Importantly, many of the regions involved in actual tool use are active during cognitive tasks in which there is no action execution. For instance, regions of the posterior parietal lobe are activated when viewing images of tools (with no explicit instruction to think about their use; Chao & Martin, 2000). A recent large-scale neuro-psychological study revealed that damage to the posterior temporal lobe, an area often associated with tool-based motion, results in deficits in both action production and action recognition (Tarhan, Watson, & Buxbaum, 2015). This suggests that part of cortical representation of tool use is not only distributed across the ventral and dorsal streams, but the representations within these networks are at least partially grounded in sensorimotor systems involved in action planning (see Martin, 2007). These findings are consistent with the grounded cognition hypothesis that tool cognition (e.g., identifying tools or describing how to use them) requires activation of representations involved in planning tool use, including those relevant for perceiving tool shape, motion, and, possibly, the arm and hand kinematics associated with their use (see Matheson & Barsalou, in press; Tarhan et al., 2015; Barsalou, 2008; Thompson-Schill, 2003).

The claim that representations within the tool use network are distributed and grounded within ventral and dorsal regions has consequences for understanding not only common tool use but also creative or uncommon tool use and its neural correlates. To use a tool in an uncommon way, a problem solver must consider the properties of the tool, both structural characteristics such as shape and size and physical principles such as leverage; such mechanical problem solving may be disrupted in some patients with apraxia after left parietal lesions (see Goldenberg & Hagmann, 1998). These problem-solving skills may be supported by overt or covert mental imagery (e.g., Pearson, Naselaris, Holmes, & Kosslyn, 2015; Kosslyn, Ganis, & Thompson, 2001), perhaps via simulation of possible object uses.¹ A general hypothesis stemming from this perspective is that the ability to generate uncommon tool uses requires activating representations of sensorimotor features in a search of action possibilities.

Two previous investigations from our laboratory are relevant to these ideas. First, the role of anterior and posterior cortical regions in generating uncommon tool uses was investigated by Chrysikou and Thompson-Schill (2011), who reported a reversal in the relative magnitudes of the BOLD response during common versus uncommon use generation when comparing the inferior frontal gyrus (greater for common uses) and the occipito-temporal cortex (greater for uncommon uses). This interaction indicates that the generation of uncommon uses places greater demands than does generation of common uses on the neural systems that support the analysis and representation of features of the visual appearance of an object, such as its shape (see also Tyler & Moss, 2001). Subsequent research revealed that participants take longer to generate uncommon uses than common ones (Chrysikou et al., 2013) and may suggest that the posterior activation observed during the uncommon use task reflects the activation of grounded representations and/or explicit imagery (see also Chrysikou, Motyka, Nigro, Yang, & Thompson-Schill, 2016).

In this study, we further investigated the way in which information within the tool use network is flexibly activated to generate common versus uncommon tool uses. We sought to extend previous research by adopting pattern similarity² analysis (PSA; e.g., Connolly et al., 2012; see Kriegeskorte, Mur, & Bandettini, 2008), allowing us to emphasize how the multivariate responses within the tool use network change between the two tasks. Specifically, we examined whether neural dissimilarity patterns of different regions of the tool use network (e.g., the neural pattern in response to axe vs. hammer vs. brush vs. …) correlated with different behaviorally derived dissimilarity measures relevant to tool use behavior. We investigated three types of information, all of which influence activity in the tool use network: visual (how visually dissimilar is axe vs. hammer vs. brush vs. etc.), action (how dissimilar are the actions associated with using axe vs. hammer vs. brush vs. etc.), and categorical (how dissimilar is the category that axe belongs to vs. hammer vs. brush vs. etc.). Although many recent theories have posited that object representations are dynamically activated (see Mahon & Hickok, 2016, for a review) and some have explicitly focused on how different neural regions come online to support conceptual performance in a dynamic way (e.g., GRAPES model presented by Martin, 2016), there are no studies, to our knowledge, that have investigated how neural tuning of specific regions of the tool use network changes in different tasks or the relationship of that tuning to different behaviorally relevant information. In this study, we hypothesized that the representation of tool-related information is situated and would therefore be differentially tuned within the tool use network in common versus uncommon use tasks, with trade-offs observed between the ventral and dorsal streams. Specifically, because previous univariate results show a trade-off between anterior and posterior regions in these two tasks (Chrysikou & Thompson-Schill, 2011), we hypothesized that information in more anterior and ventral regions supports common tool use information, whereas information in posterior dorsal regions supports uncommon use responses.

METHODS

Participants

We recruited 30 participants from the University of Pennsylvania community through online ads. We randomly assigned each participant to either the common use or uncommon use generation condition; groups were matched on gender (eight men in each group) and age (means = 25.3 and 25.2 years, respectively; t(28) = 0.05, p = .96). With the exception of two participants in the common use group, all participants were right-handed; none of the findings reported below qualitatively change if we remove the two left-handed participants from analyses. All participants reported English as their first language. Participants were monetarily compensated for their time. All participants provided consent, and the study was approved by the institutional review board at the University of Pennsylvania.

Stimuli

We selected 60 color photographs of everyday household objects from the set described in Watson and Buxbaum (2014). See Table 1 for a list of stimuli used in the present experiment.

Table 1.

List of the 60 Tools Used in the Present Experiment (see Watson & Buxbaum, 2014)

Axe	Key	Safety pin
Bottle opener	Keyboard	Saw
Brush	Knife	Scissors
Bubbles	Light bulb	Screw
Calculator	Lighter	Screwdriver
Comb	Lipstick	Soap
Cookie cutter	Magnifying glass	Soap dispenser
Corkscrew	Match	Sponge
Drill	Measuring cup	Spray bottle
Dropper	Nail clipper	Squeegee
Eraser	Nail polish	Staple remover
Fan	Paint brush	Stapler
Fly swatter	Paint roller	Toaster
Fork	Peeler	Tongs
Garlic press	Pencil	Toothbrush
Glue stick	Ping-pong racket	Tweezers
Hair clip	Pliers	Watering can
Hammer	Potato masher	Weight
Hole puncher	Razor	Whisk
Iron	Remote control	Wrench

We used stimuli from Watson and Buxbaum (2014)—which are a subset of those in the larger Bank of Standardized Stimuli stimulus database (Brodeur, Dionne-Dostie, Montreuil, & Lepage, 2010)—because these stimuli have been well characterized with regard to their visual, categorical, and action similarity.

Procedure

Following from Chrysikou and Thompson-Schill (2011), we used a between-group manipulation of the task—common versus uncommon use generation—to limit the frequency with which participants explicitly thought about both types of uses on any given trial. For instance, we did not want to prime participants in the common use group to think of uncommon uses. We instructed participants in the common use condition to “please describe how you would commonly use the object; describe the use that first pops into your head. For instance, if you see a shoe, you might respond, ‘To wear on my feet as I walk around.’” Participants were encouraged to think of this task as accessing what the most common response might be in the population of people familiar with the item. We instructed participants in the uncommon use condition to “please describe a creative and novel use of the object; describe a use that is not obvious. For instance, if you see a shoe, you might respond, ‘Use the sole to hammer a nail into the wall.’” We assured participants that there were no correct answers. Participants in the uncommon use task were further instructed to avoid repeating prior answers (e.g., if “to use as a hammer” was given in response to one object, it should not be given in response to another object) and to avoid generating common alternative uses (e.g., using a toothbrush to clean shower tile).

On each trial, a fixation cross appeared for 6000 msec, followed by the image of an object for 3000 msec. After this, a blank screen was presented for 9000 msec, in which the participants were instructed to simply think about how they would use the object (according to their instruction). After this cogitate period, a small circle appeared in the center of the screen for 6000 msec, cueing the participants to provide their verbal response. Participants completed 60 trials in random order across two 12-min scanning blocks, with a single break.

A microphone positioned near the participant’s mouth transmitted their response to the stimulus computer for coding offline. Vocal responses were recorded onto the native HD with an OptoActive Active Noise Control system (OptoAcoustics, Israel). Image display was controlled by E-Prime (Psychology Software Tools, Pittsburgh, PA) running Windows XP on a Dell ThinkPad laptop computer. Participants viewed images using a mirror positioned on the head coil. Images were projected onto a screen using a projector positioned in the scanning room.

Behavioral Dissimilarity Matrices

We constructed action, visual, and categorical dissimilarity matrices (DSMs) using the data reported in Watson and Buxbaum (2014). Measures of visual and categorical similarity were obtained from participants’ explicit ratings of these two dimensions for each pair of objects: Visual similarity ratings were based on the instructions to rate how similar pairs of tools “looked,” whereas categorical similarity ratings were based on instructions to rate the extent to which pairs of tools were from the same “category.” We derived visual and categorical dissimilarity values by calculating the Euclidean distance between the ratings for all pairs of objects. Action dissimilarities were derived differently. In Watson and Buxbaum (2014), participants were given images of the objects and asked to sort the tools into groups based on “how the objects are typically used,” emphasizing that the sorting was based on the skilled use of the tools. Each pair was assigned a similarity value based on the number of times it was sorted together. PCA was performed on the raw data, generating two principal components, one corresponding to the amount of arm movement associated with the functional use of the tool and one corresponding to the type of hand posture adopted to use the tool (see Figure 2 in Watson & Buxbaum, 2014). To create the behavioral action DSM used here, we calculated Euclidean distance between tool pairs based on their values for each of the two principal components.

In the analysis below, we treat these DSMs as independent, although it is likely that they are not orthogonal. For instance, because the actions we can perform on an object are partly constrained by its shape, we might expect a strong correlation between action and visual information. However, although the action DSM correlated with both the category DSM (r = .26, p < .001) and the visual DSM (r = .12, p < .001) and the visual and category DSMs correlated with one another as well (r = .22, p < .001), the correlations were small suggesting that these matrices are not simply redundant with one another and reflect conceptually different sources of information.

Image Acquisition and Preprocessing

We collected fMRI data using a 3.0-T Siemens Trio (Malvern, PA) with a 32-channel head coil. The imaging procedure began with T1-weighted localizer image. Echo-planar functional images were collected over 44 axial slices (repetition time [TR] = 3000 msec, echo time = 30 msec, 64 × 64 × 44 pixels in a 19.2-cm field of view and a voxel size of 3 mm³).

Data preprocessing was conducted offline using the fMRIB software library (Jenkinson et al., 2012; fsl.fmrib.ox.ac.uk/fsl/fslwiki). First, for each participant, we used the Brain Extraction Tool (Smith, 2002) to eliminate voxels of noninterest (e.g., the skull). Adequate brain extraction was confirmed visually for each participant. We performed motion correction with a linear registration using the MCFLIRT tool (Jenkinson, Bannister, Brady, & Smith, 2002), and we spatially smoothed data with a 5-mm FWHM filter. In addition, the functional data were temporally filtered with a high-pass filter (100-sec cutoff).

Data Analysis

We conducted two analyses. First, we performed a pattern dissimilarity analysis. Second, we performed an exploratory follow-up univariate analysis.

Pattern (Dis)similarity Analysis

For the PSA, the fMRI Expert Analysis Tool was used to model BOLD responses in each run. For the first level analysis, we created two explanatory variables (EVs) to model (a) the stimulus period and (b) the vocal response period of each run; the fixation period between trials was modeled as the baseline. In addition, we created 30 EVs to model the response (i.e., the period of the trial in which the screen was blank, and participants were preparing their responses) period separately for each stimulus (30 response periods per run). To ensure that the patterns used in the PSA were estimated from equivalent stages of processing in the two tasks, the response period was defined separately for the two groups based on the known RT differences in producing common and uncommon use responses (from RTs reported in Chrysikou et al., 2013)³: For the common use group, because RTs are reported at approximately 2000 msec, only the first TR was modeled during the response period to estimate common use patterns. In contrast, for the uncommon use group, because RTs are reported at approximately 5000 msec, the second TR was used. In both cases, the other TRs during this period were modeled separately as a variable of no interest. All EVs were convolved with a gamma function (6-sec lag, 3-sec SD). This generated parameter estimates (i.e., contrast of parameter estimates generated) images for each stimulus (relative to fixation), which were then used as patterns for the PSA. Finally, functional images were registered using FMRIB’s Linear Image Registration Tool (Jenkinson et al., 2002; Jenkinson & Smith, 2001) by first aligning the functional data with the participant’s high-resolution anatomical brain. We used a linear search with 12 degrees of freedom to align the participant’s anatomical to the standard MNI-152 2-mm atlas.

Our PSA proceeded in two phases, following from Proklova, Kaiser, and Peelen (2016) and Connolly et al. (2012) using functions from the CoSMoMVPA toolbox (Oosterhof & Connolly, retrieved from cosmomvpa.org/) as it is implemented in MATLAB (The MathWorks, Inc., Natick, MA) and additional custom scripting. First, for each participant, we conducted a whole-brain searchlight analysis (radius = 3 voxels): Within each searchlight, correlation distance was calculated for each pair of stimulus–response patterns. Thus, for each spherical searchlight, a 60 × 60 (i.e., each object’s neural pattern during the response period) neural DSM was calculated. These neural DSMs were z scored and then used as the dependent variable in a general linear model (i.e., multiple regression) with our three z-scored behavioral DSMs as predictors (action, categorical, and visual), resulting in beta values for each of the three predictors associated with each searchlight center. The searchlight analysis resulted in a whole-brain map of beta values of the neural dissimilarity versus each of the behavioral DSMs.

Because of the large number of comparisons in analyzing the whole-brain map, we only analyzed beta values within a set of ROIs. To identify ROIs, we used the Talairach coordinates (transformed to MNI using the coordinate utility of the SDM neuroimaging software accessed here: www.sdmproject.com/utilities/?show=Coordinates) from 12 regions presented in Gallivan et al. (2013) as nodes in the tool use network. In addition, we used a selection of left hemisphere coordinates listed in Chrysikou and Thompson-Schill (2011) as nodes in the extended network implicated in uncommon tool use (see Table 2).

Table 2.

List of MNI (in Both Voxel Space and Millimeter Space) Coordinates (Transformed from Talairach) Used as the Center of Spherical (3-mm Radius) ROIs

From Gallivan et al. (2013)		From Chrysikou et al. (2011)
Region	x, y, z (Voxel; mm)	Region	x, y, z (Voxel; mm)
Superior parieto-occipital cortex	48, 25, 53; −6, −76, 34	Inferior frontal gyrus (1)	69, 72, 37; −48, 18, 2
Posterior intraparietal sulcus	56, 29, 62; −22, −68, 52	Inferior frontal gyrus (2)	65, 20, 34; −40, 20, −4
Middle intraparietal sulcus	61, 36, 62; −32, −54, 52	Inferior frontal gyrus (3)	67, 65, 38; −44, 4, 4
Posterior anterior intraparietal sulcus	66, 39, 60; −42, −48, 48	Inferior frontal gyrus (4)	70, 69, 33; −50, 12, −6
Anterior intraparietal sulcus	66, 44, 59; −42, −38, 46	Superior frontal gyrus	46, 66, 70; −2, 6, 68
Supramarginal gyrus	74, 46, 54; −58, −34, 36	Middle frontal gyrus	62, 65, 63; −34, 4, 54
Motor cortex	64, 50, 62; −38, −26, 52	Middle occipital gyrus (1)	63, 20, 38; −36, −86, 4
Dorsal premotor cortex	58, 58, 63; −26, −10, 54	Middle occipital gyrus (2)	58, 20, 37; −26, −86, 2
Ventral premotor cortex	72, 65, 42; −54, 4, 12	Middle occipital gyrus (3)	63, 17, 29; −36, −92, −14
Somatosensory cortex	65, 44, 62; −40, −38, 52	Fusiform gyrus	60, 22, 26; −30, −82, −20
Posterior middle temporal gyrus (functionally defined)	72, 33, 35; −54, −60, −2
Extrastriate body area (functionally defined)	70, 25, 38; −50, −76, 4

To generate our ROI masks, we dilated each voxel (at the specified coordinates) in three dimensions by three voxels to create spherical masks (thus, the ROI masks were the same size and shape as our searchlights). To calculate the z value associated with each beta value (reflecting the fit of the behavioral matrix with the neural matrix for that searchlight), we used the Cosmo Monte Carlo Cluster Stat function with multiple comparison correction within the CoSMo toolbox (see Oosterhof, Connolly, & Haxby, 2016). This function was run separately for each ROI. There are a number of steps to this method: First, at each searchlight center, we performed a t test comparing betas in the common use group versus the uncommon use group, resulting in a t score for each searchlight center. These t scores are immediately converted to z scores, resulting in a single whole-brain map of z values. We then used threshold-free cluster enhancement (TFCE) to convert the z map into a map of TFCE values; here, each TFCE value is the z score adjusted by the magnitude of the z values of those surrounding it—in this case, the other betas in the ROI (see Smith & Nichols, 2009).⁴ Next, null TFCE distributions are generated by randomly flipping the signs of the observed beta values and performing t tests on each of 10,000 permutations. Finally, a final z map is derived by comparing, for each searchlight center, the number of times the observed TFCE was smaller than the maximum TFCE in the null maps and dividing this by the number of iterations (therefore correcting for all comparisons within an ROI; see Oosterhof et al., 2016).

z Scores greater than 1.65 are considered significant at an alpha of .05 (one tailed). This criterion allowed us to determine, within each ROI, which searchlight centers showed group differences (common > uncommon and uncommon > common). For visualization, we projected the results of any ROIs showing at least one significant searchlight center onto the surface of the standard MNI-155 2-mm brain template (using FreeSurfer, v5.3.0).

Univariate Analysis

To further explore how the tool use network responds to task changes in common versus uncommon tool use, we conducted an additional univariate analysis: Here, each participant’s 60 response contrast of parameter estimates generated (generated in the PSA analysis) were averaged, resulting in one whole-brain map per participant (representing the average pattern across all 60 objects for each participant). Then, the Cosmo Monte Carlo Cluster Stat function was used with TFCE to identify any voxels that showed greater activation in either the common use group or the uncommon use group. Again, any ROIs showing greater activation are projected onto the surface for visualization.

RESULTS AND DISCUSSION

Behavioral Results

Vocal responses were transcribed by the first author. Responses that were judged to adhere to the instructions (i.e., providing a common or uncommon response) were coded as “correct”; all other responses, or failures to respond within the allotted window, were coded as “incorrect.” A t test comparing the number of correct responses between groups showed that performance in the common group was marginally better (M = 0.99) than was performance in the uncommon group (M = 0.93), t(28) = 2.06, p = .058, but overall performance was high in both groups.

Pattern (Dis)similarity Analysis

We used PSA to characterize the multivoxel patterns of neural activity during the response period separately for the common and uncommon use groups to determine whether different information (characterized as behavioral dissimilarity), including visual, action, and categorical, predicted neural responses in the tool use network. Table 3 lists the voxel coordinates of significant searchlight centers, the associated z scores (generated from permutation testing within the ROI), and the average beta values for the two groups at that location (Figure 1).

Table 3.

Max Z Scores within ROIs Comparing the Correlation Values within the Common vs. Uncommon Groups (n = 15).

	Action				Category				Visual
	Common	Uncommon	x, y, z	Z (p)	Common	Uncommon	x, y, z	Z (p)	Common	Uncommon	x, y, z	Z (p)
ROIs from Chrysikou et al. (2011)
IFG (1)									.03	−.01	−50, 14, 2	1.86 (.03)
IFG (3)									.02	−.02	−42, 6, 4	2.46 (.007)
MFG
MOG (3)	−.02	.05	−40, −92, −10	−1.71 (.04)
FFG
ROIs from Gallivan et al. (2013)
paIPS	−.02	.03	−42, −46, 52	−1.89 (.03)
aIPS
SMG									−.005	.05	−56, −30, 38	−2.0 (.02)
M1
PMd									−.01	.04	−24, −10, 54	−1.73 (.04)
SS
pMTG					.04	−.005	−50, −58, 0	1.92 (.03)

Note that Z scores > 1.65 are significant at alpha = .05 (one tailed). Only ROIs that showed at least one significant test are reported. All other correlations are z < 1.65. Coordinates of max Z scores are given in MNI space (mm). Mean correlations for both groups at each location are also shown. Comparisons highlighted in orange show reliably stronger correlations in the uncommon group than in the common group; comparisons highlighted in light blue show reliably stronger correlations in the common group than in the uncommon group.

IFG = inferior frontal gyrus; MFG = middle frontal gyrus; MOG = middle occipital gyrus; FFG = fusiform gyrus; paIPS = posterior anterior intraparietal sulcus; aIPS = anterior intra parietal sulcus; SMG = supramarginal gyrus; M1 = primary motor cortex; PMd = dorsal premotor cortex; SS = somatosensory cortex; pMTG = posterior middle temporal gyrus.

Figure 1.

Brain areas showing at least one group difference for each of the three types of behavioral dissimilarity. Left-hemisphere ROIs projected onto the inflated, lateral MNI template. See Table 3 for precise coordinates as well as mean beta values for both the common and uncommon use groups at the max z score location.

Action Dissimilarity

For action dissimilarity, two ROIs of the left-hemisphere tool use network showed at least one significantly stronger correlation in the uncommon use group compared with the common use group. Included were the posterior intraparietal sulcus and the middle occipital gyrus. The intraparietal sulcus is implicated in coordinating reaching and grasping (Vingerhoets, 2014; Culham & Valyear, 2006) as well as reaching and grasping imagery (Filimon, Nelson, Hagler, & Sereno, 2007). In the uncommon use group, the computations of this ROI predicted action information (associated with common uses), suggesting that detailed information about kinematics compose (at least in part) the representation of action in this task (see also Orban & Caruana, 2015). It is possible that the computations of this region reflect the increased competition between the representations of typical functional actions and those of structurally derived actions relevant to alternative uses (see Lee, Middleton, Mirman, Kalénine, & Buxbaum, 2013).

The middle occipital gyrus also showed a stronger relationship with action information in the uncommon use group. This extends previous findings from Chrysikou et al. (2011) about the role of posterior visual regions in generating uncommon uses by showing that shape-sensitive regions also contribute to the representation of action in this task. Previous research has shown that the multivoxel pattern dissimilarity responses of this region also correlate with body extension dissimilarity, specifically the degree to which the objects extend the manipulative powers of the body (see Bracci & Peelen, 2013). Our results compliment this finding, demonstrating that higher-level visual cortical information reflects nonvisual (i.e., motoric) information depending on the task.

These ROIs reflect action dissimilarity more in the uncommon as compared with common use group, although no tool is visually presented during the response period, suggesting that the information about common actions is predictable from the patterns evoked during the planning or imagining of uncommon tool uses. This finding elaborates previous research showing that the regions involved in actual tool use are also active during cognitive tasks in which there is no action execution (e.g., Creem-Regehr & Lee, 2005; Johnson-Frey, Newman-Norlund, & Grafton, 2005; Moll et al., 2000; Grafton, Fadiga, Arbib, & Rizzolatti, 1997). These ideas are consistent with the grounded cognition hypothesis that representations of tool use actions are grounded in the reactivation of regions involved in actually using tools, including those relevant for perceiving tool shape and motion and for specifying the arm and hand kinematics associated with their use (see Barsalou, 2008; Thompson-Schill, 2003).

Categorical Information

The computations of the posterior middle temporal gyrus predicted categorical information more strongly in the common use task than in the uncommon use task. This region is widely implicated in tool-related semantics (see Watson & Buxbaum, 2015). Indeed, the integrity of the posterior middle temporal gyrus is necessary for recognizing tool-related actions and producing them (see Kalénine, Buxbaum, & Coslett, 2010). The fact that this region is clearly involved in both production and recognition suggests that the information in the middle temporal gyrus is implicated in organizing tool knowledge and is a critical node grounding tool cognition in action (see Kable, Kan, Wilson, Thompson-Schill, & Chatterjee, 2005). One possibility is that the posterior middle temporal gyrus is critical for retrieving representations of the visual motion of tool actions, consistent with the well-established finding of activity in this region in response to moving tools (Beauchamp et al., 2002, 2003) and deficits in action recognition after damage here (Tarhan et al., 2015). Our results suggest that such simulations support the processing of tool category.

Visual Information

Two ROIs of the tool use network predicted visual information more strongly in the uncommon use group than the common use group. First, the dorsal premotor cortex is implicated in the online control of action and action execution (see Moll et al., 2000). This finding suggests that generating uncommon tool uses relies on computations that reflect visual shape. Indeed, the visual shape of objects will constrain the exact types of uncommon uses that are executable with a tool.

Second, the supramarginal gyrus predicted visual information more strongly in the uncommon use group. The supramarginal gyrus’ involvement suggests that the computations of this region are highly sensitive to the visual form of objects and the precise motor acts that the shape affords, perhaps driven in part by higher-level action plans in the dorsal premotor cortex.

Interestingly, the response patterns of the left inferior frontal gyrus predicted visual information more in the common use group. This region was implicated by Chrysikou et al. (2011) as a region that biases relevant action representations within posterior regions (see also Watson & Buxbaum, 2015; Badre & Wagner, 2007). We extend these results by showing that the computations of this region reflect visual information during the common use task, suggesting that an important dimension for this biasing process is the visual form of the object. There are consequences of this relationship as well, as damage to the inferior frontal gyrus will result in impaired pantomime (see Bohlhalter et al., 2011) and deficits in both producing and recognizing tool-related actions (Tarhan et al., 2015).

Univariate Results

A summary of the univariate analysis is presented in Table 4 and Figure 2.

Table 4.

Max Z Scores Comparing Common vs. Uncommon Groups (n = 15) within Each Left-Hemisphere ROI

ROI	Common > Uncommon		Uncommon > Common
	x, y, z	Z (p)	x, y, z	Z (p)
lIFG (2)	−38, 20, −8	1.8 (.04)
lMOG (1)	−32, −88, 0	1.75 (.04)
lMOG (2)	−30, −88, 2	2.72 (.003)
pIPS			−18, −64, 54	−2.41 (.008)
midIPS			−32, −54, 58	−1.94 (.03)
paIPS			−48, −48, 48	−1.97 (.02)
M1			−40, −30, 52	−2.54 (005)
PMd			−30, −12, 50	−1.96 (.02)
SS			−40, −32, 52	−1.78 (.04)
pMTG			−56, −62, −6	−1.97 (024)

Note that only Z scores > 1.65, considered significant at alpha = .05, are shown. Coordinates are given in MNI space (mm).

lIFG = left IFG; lMOG = left MOG.

Figure 2.

z Scores within ROIs of the tool use network that showed significant differences in the mean activation across patterns between groups.

We observed greater activity in dorsal stream ROIs during the uncommon use task, demonstrating that the uncommon use task recruits areas involved in the online control of action more than the common use task. Interestingly, the left inferior frontal gyrus⁵ responded more in the common use group, but so did posterior regions of the middle occipital gyrus, a univariate result that only partial replicates Chrysikou and Thompson-Schill (2011). However, in Chrysikou and Thompson-Schill (2011), participants viewed tools and were told to give a verbal response while the tool was on the screen; conversely, our participants gave responses after the stimulus viewing period. Therefore, our task gave participants a large amount of time to think of their desired use in the absence of the visual object. Because the contributions of the inferior frontal gyrus and fusiform area are not static, but unfold in a dynamic fashion, introducing a delay may have obscured any differences that arose through the response period. Our design did not permit an investigation of such timing effects, but behavioral (e.g., Bub & Masson, 2012) and neurophysiological (e.g., Kiefer, Sim, Helbig, & Graf, 2011) results show that the activation of action information is highly sensitive to timing (see also Lee et al., 2013).

Summary and Conclusions

We have shown that different types of behaviorally relevant information predict the computations of the ventral and dorsal regions of the tool use network differently when generating common versus uncommon uses, demonstrating the situatedness and flexibility of the neural systems supporting one of our most quintessential skills. Ventral regions reflected categorical information more in the common use group, whereas posterior dorsal regions reflected action and visual information more in the uncommon use group. Although the univariate results show that the absolute magnitude of the activity within a subset of these ROIs varies across the two tasks, they do not provide insight about how the processing of these ROIs relates to different types of information (see Coutanche, 2013). Our PSA findings suggest that the information represented in these regions shifts under the two conditions, with computations within posterior dorsal regions more relevant for generating uncommon uses and computations within ventral and anterior regions more relevant for generating common uses.

Our findings are consistent with a growing number of imaging results that demonstrate that object representations are dynamically activated across different tasks. For instance, areas of the ventral stream represent identical objects differently in perceptual versus conceptual tasks (Harel, Kravitz, & Baker, 2014). Furthermore, attention to different semantic categories (humans vs. vehicles) during natural viewing shifts the tuning curves of almost the entire cortex toward the attended category (Çukur, Nishimoto, Huth, & Gallant, 2013). However, we extend these previous findings by showing that not only do task demands shape the degree of response in different regions, but the fine-grained structure of the response changes across tasks. We have shown that thinking about the properties of a hammer with the goal of describing an uncommon use compared with a common use recruits different types of information. This suggests strongly that there is no single representation of object information supporting task performance but that the representation’s structure changes. Along with previous findings, our results are consistent with recent theoretical proposals that concepts are not static but are situated instead—that context shapes the way in which object concepts are activated (see Barsalou, 2005).

Our findings have an additional, broader theoretical implication. According to grounded theories of cognition, generating uncommon tool uses should result in simulations in modality-specific cortices, particularly those related to shape and action (see Barsalou, 2008). In this study, we found evidence for involvement of functional regions specifying action and visual properties of tools, in particular, regions associated with action planning and recognition (intraparietal sulcus, dorsal pre-motor cortex, posterior middle temporal gyrus). This suggests that, even in the absence of overt tool use behaviors, the activity of these regions reflects action, visual, and category information and may suggest that overt/covert imagery of possible actions supports the generation of uncommon uses.

Overall, the situatedness of uncommon tool use reflects the generative abilities of humans to creatively break the sociocultural conventions associated with tools. The finding that computations within regions of the tool use network reflect different types of information under these two conditions suggests one way in which the cortex supports both common and uncommon tools use. Specifically, each region of the tool use network performs computations that draw upon different types of information and do so in ways that flexibly meet the task demands. The computations of the two streams, classically described as specializing in visual (i.e., in the ventral stream) and motor (i.e., in the dorsal stream) processes, are actually flexibly activated to support different tasks. Our results help illuminate why, in neurological conditions such as apraxia, deficits in tool use behavior can arise after damage to multiple regions (for instance, either posterior parietal regions or prefrontal regions; see Buxbaum et al., 2014): Action-relevant information is represented in the specific computations of both regions. Although speculative, this may support the notion that complex conceptual information is spread across multiple levels of multimodal neural hierarchies and therefore is actually reflected in the computations of most regions of the cortex (see Clark, 2013), all of which that are coordinated to support the task (see Anderson, 2010). This neural flexibility likely contributes to the generative and creative nature of cognition in other domains, including the creation of new linguistic and moral conventions, and reflects a neural foundation of some of our most quintessential skills.