Shared and distinct routes in speech and gesture imitation: evidence from stroke

Introduction

Imitation of visual actions and repetition of sounds both rely on the mapping of sensory information to a motor plan for effector movement. Successful gesture imitation maps visual actions to limb movements, while sound repetition maps auditory information to speech articulation. When gestures or words are meaningful and familiar, semantic information confers a benefit to the sensory input to motor output mapping process in gesture imitation and word repetition accuracy, and at least for gestures, in terms of speed as well (Buxbaum et al., 2005; Goldenberg & Hagmann, 1997; Hanley et al., 2004; Nozari et al., 2010; Press & Heyes, 2008). According to both dual-route models in the action domain and dual-route models in the language domain (Barron, 1986; Buxbaum, 2001; Hickok & Poeppel, 2007; Humphreys & Evett, 1985; Rauschecker & Tian, 2000; Roy & Square, 1985), the benefit of meaning in word repetition and meaningful gesture imitation occurs as a result of processing in an “indirect” route that transforms sensory input into motor output via semantics. In contrast, unfamiliar or meaningless sounds and gestures must rely on “direct” sensory-motor mapping, bypassing semantics (Tessari & Rumiati, 2004).

Dual-route models of both action and of language also propose broadly similar neural architectures, whereby a ventral stream underlies indirect semantic processing and a dorsal stream underlies direct sensory-motor mapping, although the precise neural regions in the two routes may differ across action and language. Neuroanatomical support for dual-route action models comes primarily from lesion analyses in patients with left-hemisphere cerebrovascular accident (LCVA; Achilles et al., 2019; Buxbaum et al., 2014; Buxbaum, 2017) and neuroimaging studies in neurotypical adults (Peigneux et al., 2004; Rumiati et al., 2005). These findings suggest that distinct brain regions are associated with meaningful versus meaningless gesture imitation. Specifically, imitation of meaningful gestures involves a ventral semantic route, subserved by the left posterior temporal cortex and angular gyrus. In contrast, direct visuo-motor transformation underlying meaningless gesture imitation involves a more dorsal kinematic route, including fronto-parietal cortex (see Buxbaum, 2017 for review).

There are parallel accounts of cortical organization of dual-route language models, which similarly consist of a dorsal and ventral stream architecture (e.g., Hickok & Poeppel, 2004, 2007). The dorsal stream represents a sensory-motor interface whereby sensory input interfaces directly with motor output representations via dorsal projections between temporoparietal and inferior frontal regions. In contrast, the ventral stream is a sensory-conceptual interface in which sensory input interfaces with semantic representations via ventral projections to portions of the temporal lobe.

A prominent computational psycholinguistic account, the semantic-phonological interactive two-step model (Foygel & Dell, 2000), has been developed to characterize the processes that mediate lexical processing. The first step maps between semantic representations and lexical (wordform) units (s-weight; Dell et al., 2013). Impairments in this step can reflect deficits in either pre-lexical semantic integrity or mapping between semantics and lexical units (Dell et al., 2013). S-weight has been associated with the inferior frontal lobe, anterior temporal lobe, temporo-parietal junction, angular gyrus, and middle and inferior temporal cortices (Dell et al., 2013; Hula et al., 2020; Schwartz et al., 2012). The second step maps between lexical and phonological units (p-weight), which has been associated with lesions to the posterior frontal lobe, posterior-dorsal temporal lobe, and anterior parietal cortex (Dell et al., 2013; Hula et al., 2020; Schwartz et al., 2012). Both steps of the semantic-phonological model have been proposed to be mediated by the indirect route (Walker & Hickok, 2016). Processing in the indirect route is also proposed to be bi-directional, supporting both language production and comprehension (Hickok & Poeppel, 2004). In contrast, a separate auditory-motor mapping stage has been proposed to underlie the direct route and to support nonword repetition (e.g., Lacey et al., 2017; Walker & Hickok, 2016).

A predominant hypothesis is that the same semantic repository is accessed in both gesture and language comprehension and production. This suggests in turn that portions of the indirect route may be shared across domains. This hypothesis is supported by findings that gestures benefit language processing (Marangolo et al., 2010; Murteira & Nickels, 2020; Raymer et al., 2006), gestures play a role in conceptualizing information for speaking (Hostetter et al., 2007; Kita et al., 2017; Kita & Davies, 2009), and language deficits are associated with meaningful but not meaningless gesture imitation (Achilles et al., 2016; Mengotti et al., 2013). Furthermore, individuals with more severe aphasia show reduced gesture informativeness compared to controls and moderate aphasic speakers, suggesting possible associations between verbal language and gesture production (Mol & Kita, 2012).

Critically, however, prior studies have been based solely on associations between gesture deficits and language deficits, and have not examined whether the integrity of semantic memory (as assessed by tasks that rely on neither language nor gesture) predicts a benefit from meaning in gesture imitation. That is, if gesture knowledge is a subset of broader semantic knowledge, individuals whose semantic processing is intact ought to be able to benefit from their intact gesture knowledge in imitation. On the other hand, if semantic information is not shared across language and gestures, as some have suggested (Willems et al., 2009), then the status of semantic processing (assessed in non-gesture tasks) should have no bearing on whether gesture imitation benefits from gesture meaning. In line with the former set of accounts, we predicted that there would be a strong association between the integrity of semantic processing, as assessed by a common semantic judgement task (Camel and Cactus Test; Bozeat et al., 2000), on the one hand, and the benefit of semantic information in the gesture domain on the other (i.e., unnamed meaningful vs. meaningless gestures).

Furthermore, if the same semantic system is used for language and gestures, then the semantic repository that supports gestures should be accessible via visual or verbal input. However, it remains unclear whether gesture names (i.e., verbs) also confer an advantage to gesture imitation following LCVA. As many as 50%−77% of people with language deficits (aphasia) following LCVA also have impairments in production and understanding of gestures (limb apraxia); moreover, the vast majority of individuals with limb apraxia also have aphasia (Papagno et al., 1993; Weiss et al., 2016).

As such, some individuals with apraxia have deficits in imitating and executing gestures following verbal commands, while others may benefit from additional semantic activation from the lexical system (Cubelli et al., 2000; Foerster et al., 2020). We assessed the prediction that benefit of gesture names would be associated with the integrity of lexical-semantic mapping (s-weight) in individuals with LCVA (i.e., named vs. unnamed meaningful gesture imitation).

In the direct route, dorsal sensory-motor mappings rely on transformation of distinct and specialized information: word repetition involves transformation from sounds to phonemes to mouth movements, while gesture imitation involves transformation from visual input to hand movements. Thus, despite relatively close neuroanatomic proximity of the dorsal stream in language and action domains, the relationship between the integrity of sensory-motor mapping in language (as assessed by non-word repetition) and action domains (as assessed by meaningless gesture imitation) may not be particularly strong.

In summary, despite the fact that apraxia and aphasia frequently co-occur following LCVA, we have limited understanding of how direct versus indirect processes in aphasia may be associated with limb apraxia. Dual-route models of action and language provide a starting place for generating hypotheses. Accordingly, the first aim of this work was to test the hypothesis that semantic representations mediated by the indirect route are shared across language and gesture domains. Following from this, we focused on characterizing the contributions of semantic and lexical information to gesture production following left hemisphere stroke. The second aim of this work was to test the hypothesis that different sensory-motor mappings mediate processing in distinct dorsal routes in language versus gesture domains. These hypotheses enabled the following specific predictions, with a particular emphasis on the role of semantics in gesture imitation:

1. We predicted that the indirect ventral stream would share semantic processes across language and gesture domains, such that (1a) semantic and (1b) lexical information would both benefit gesture production.

   • 1a)
     We expected to replicate findings that semantics facilitate gesture imitation, such that unnamed meaningful gestures are imitated more accurately than meaningless gestures in neurotypical adults and individuals with LCVA.
   • 1b)
     We predicted that gesture names (verbs) would further facilitate gesture imitation, such that participants would perform more accurately on named meaningful than unnamed meaningful gesture imitation.
   • 1c)
     We predicted that: (i) any observed benefit of meaning on gesture imitation would be associated with the integrity of semantic processing (Camel and Cactus Test; Bozeat et al., 2000); and (ii) both reduced benefit of meaning on gesture and impaired semantic processing would be associated with lesions to the ventral stream, specifically left inferior frontal and posterior temporal cortex; and iii) any observed benefit of naming on gesture imitation would be associated with the integrity of lexical-semantic mapping (s-weight; Roach et al., 1996).

2. Finally, because the dorsal stream of the language system (auditory to speech articulator mapping) and the dorsal stream of the gesture system (visual to limb mapping) reflect distinct sensory-motor mappings, we predicted that there would be no relationship between measures of direct sensory-motor mapping in the language (nonword repetition) and gesture (meaningless gesture imitation) domains.

Materials & Methods

In the interest of study design and analysis transparency, we report how we determined our sample size, all data exclusions, all data inclusion and exclusion criteria, whether inclusion and exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study.

a. Participants

Participants were 40 adults with chronic LCVA (>6 months post-stroke) and 18 neurotypical controls drawn from the Neurocognitive Rehabilitation Research Registry at Moss Rehabilitation Research Institute (Schwartz et al., 2005). Data were collected previously under the purview of separate studies of limb apraxia and aphasia. As such, the participant sample and behavioral and neuroimaging data represent a retrospective convenience sample and thus no sample size determination was conducted for this retrospective analysis. All participants were right-hand dominant (pre-morbidly for participants with LCVA). Of the 40 participants with chronic LCVA, 36 completed a research-quality T1 structural MRI in addition to the full behavioral battery. Participants were not included if they had a history of psychosis, drug or alcohol abuse, comorbid neurologic disorder, or traumatic brain injury. In addition, patients were not included if they had severe language comprehension impairments (score <4 on comprehension subtest of the Western Aphasia Battery; Kertesz, 2006). Patients were not otherwise selected for presence or severity of aphasia, limb apraxia, or semantic impairment. These inclusion and exclusion criteria were established prior to this retrospective data analysis. Demographic information is reported in Table 1. In compliance with the guidelines of the Institutional Review Board of Einstein Healthcare Network, all participants provided informed consent and were compensated for their participation and travel expenses. The Institutional Review Board did not approve public archival of data from the parent project. Deidentified trial-level data may be made available upon request to the corresponding author (hdresang@wisc.edu). The only condition is that data must be stripped of all protected health information, no additional stipulations or approvals are required.

Table 1.

Demographic and stroke information for each participant.

Subject ID	Lesion Volume (1mm3)	Months Post Stroke Onset	Education	Sex	Age
LCVA_01	50394	226	Grade school	M	59
LCVA_02	115118	235	Bachelors	F	59
LCVA_03	258736	221	Masters	M	72
LCVA_04	144857	170	High school	M	83
LCVA_05	89072	158	Bachelors	F	56
LCVA_06	166393	169	Some undergraduate	M	61
LCVA_07	76301	150	Masters	M	75
LCVA_08	82964	137	Some undergraduate	M	61
LCVA_09	16547	127	High school	F	81
LCVA_10	80020	177	Some undergraduate	F	55
LCVA_11	NA	136	High school	M	62
LCVA_12	23141	119	High school	M	65
LCVA_13	NA	107	Bachelors	F	68
LCVA_14	62204	109	Masters	F	40
LCVA_15	14349	114	High school	F	68
LCVA_16	94536	171	High school	F	54
LCVA_17	136576	132	High school	F	54
LCVA_18	52416	115	Some undergraduate	M	55
LCVA_19	128897	100	High school	F	61
LCVA_20	71905	89	Masters	M	74
LCVA_21	88046	98	Some undergraduate	M	39
LCVA_22	92744	80	High school	M	55
LCVA_23	68764	87	Bachelors	F	48
LCVA_24	5953	63	Bachelors	M	66
LCVA_25	93628	52	Bachelors	F	39
LCVA_26	64375	43	Some post bac	M	48
LCVA_27	56281	40	Grade school	F	68
LCVA_28	11961	51	High school	M	61
LCVA_29	NA	35	Some undergraduate	F	64
LCVA_30	NA	44	Masters	M	73
LCVA_31	27840	75	High school	F	40
LCVA_32	48459	94	Some undergraduate	F	55
LCVA_33	62530	188	Some undergraduate	F	61
LCVA_34	96196	165	Some undergraduate	M	60
LCVA_35	26504	19	High school	F	50
LCVA_36	189767	55	Bachelors	M	58
LCVA_37	5509	157	Some post bac	M	64
LCVA_38	13834	138	Some undergraduate	M	58
LCVA_39	39337	14	Associates	F	66
LCVA_40	32105	17	High school	M	63
NT_01	NA	NA	Some undergraduate	F	56
NT_02*	NA	NA	Some undergraduate	M	71
NT_03	NA	NA	Bachelors	M	70
NT_04	NA	NA	Some undergraduate	F	71
NT_05	NA	NA	High school	F	76
NT_06	NA	NA	Some undergraduate	F	76
NT_07	NA	NA	Bachelors	M	69
NT_08	NA	NA	High school	F	64
NT_09	NA	NA	Masters	M	64
NT_10	NA	NA	Some undergraduate	F	65
NT_11	NA	NA	Bachelors	F	36
NT_12	NA	NA	Some undergraduate	M	77
NT_13	NA	NA	PhD	F	71
NT_14	NA	NA	Masters	F	60
NT_15	NA	NA	Some undergraduate	M	78
NT_16	NA	NA	Associates	F	66
NT_17	NA	NA	Masters	M	78
NT_18	NA	NA	Masters	M	86

Abbreviations: LCVA, left-hemisphere cerebrovascular accident; NT, neurotypical.

^*Participants excluded from analysis.

i. Excluded and missing trials

We excluded one control participant from analyses due to the possibility that they had mild apraxia based on performance more than two standard deviations below the group mean on all imitation tasks. Thus, behavioral analyses were conducted in a total of 17 neurotypical participants (9 females, 8 males; ages 36–86, M = 69(11) years) and 40 LVCA patients (19 females, 21 males; ages 39–83, M = 61(11) years). Patients were younger than controls (t = −2.69, p = 0.009), which if anything, would tend to favor the patients. Of these participants, one patient did not have an MRI scan and was thus excluded from the neuroimaging analyses. In addition, three patients did not complete the unnamed meaningful task, which was used in both neuroimaging analysis contrasts. Thus, neuroimaging analyses were performed on a total of 36 stroke patients.

b. Gesture imitation experimental task

Participants completed three gesture imitation tasks using the left hand (which for the patients was their ipsilesional hand): (1) imitation of named meaningful gestures (hereafter referred to as “named”), (2) imitation of unnamed meaningful gestures (hereafter, “unnamed”), and (3) imitation of meaningless gestures (hereafter, “meaningless”). For each task, participants were presented with videos of an experimenter performing 16 gestures with the right hand and were instructed to imitate each gesture as if looking in a mirror. For each trial, the verb phrase corresponding to the upcoming gesture (named) or trial number (unnamed, meaningless) was presented visually and verbally. Each gesture was presented two times. During the first presentation, participants were instructed to watch the gesture in its entirety and not to respond. At the start of the second presentation, a tone cued participants to begin gesture imitation such that the participant gestured along with the model. The duration of each gesture video was 4–5 seconds. Novel gestures for the meaningless imitation task were developed to maintain similar spatiomotor characteristics of the meaningful gestures (e.g., plane of movement; joints moved; hand posture – refer to Buxbaum et al., 2014). Named and unnamed gestures were the same items and were therefore matched on all semantic and spatiomotor characteristics. Named performance was always assessed last, to avoid potential lexicosemantic priming. Named and unnamed gestures were recognized reliably by a separate sample of ten right-handed control participants (M = 97.5% accuracy, SD = 3.8%, range = 91.7 – 100%; Tarhan et al., 2015). We contrasted unnamed versus meaningless performance to examine the benefit of semantic meaning, while named versus unnamed performance examined the benefit of a visual/auditory linguistic label for meaningful gestures. The outcome measure across all tasks was gesture imitation accuracy. The Institutional Review Board did not approve public archival of data from the parent project that conducted this experimental task. A github repository includes details on the experimental stimuli materials: https://github.com/hdresang/Apraxia-Gestures. Any additional procedure details may be made available upon request to the corresponding author (hdresang@wisc.edu). No additional stipulations or approvals are required.

c. Gesture data coding

We used a digital camera to record each participant’s gesture production, which was then scored offline by a trained coder naïve to the study hypotheses who had achieved inter-rater reliability (Cohen’s kappa) of at least 85% with other raters in the lab. Each trial was assigned a score of 0 (incorrect) or 1 (correct) for each of four spatiotemporal dimensions: hand posture, arm posture, amplitude, and timing. This praxis scoring procedure has been established and implemented in numerous previous studies (see Buxbaum et al., 2000 for details). The four dimensions were summed; the summary score will hence be referred to as “accuracy”.

d. Language and semantic assessments

A subset of 30 participants with LCVA completed four additional assessments of language and semantic processing ability. Overall language impairment was assessed using the Western Aphasia Battery-Revised (WAB-R; Kertesz, 2006), which evaluates linguistic skills frequently affected by aphasia: content, fluency, auditory comprehension, repetition and naming, reading, and writing. Participants also completed the Philadelphia Naming Test (PNT; Roach et al., 1996), a 175-item picture naming test designed to measure lexical access. Following standardized procedures, responses were coded as correct, semantically related errors, formally (phonologically) related errors, mixed (semantically and formally related) errors, unrelated real words, and non-words (http://mrri.org/philadelphia-naming-test/). The response counts for each participant were fit to the semantic-phonological interactive two-step (SP) model using a web-based application (Walker & Hickok, 2016) that provides estimates of s-weight and p-weight. Additionally, the same 30 participants completed the picture version of the Camel and Cactus Test (CCT; Bozeat et al., 2000), a 64-item measure of semantic association. In this task, participants must match a reference picture (e.g., camel) to one of four options (e.g., cactus [the target], tree, sunflower, rose). Finally, 28 of the 30 participants completed the Philadelphia Non-Word Repetition Task (NWR), which consists of 60 items that are phonologically legal (Dell et al., 2007). The participants’ task was to repeat back what they heard immediately after hearing it, thus assessing direct input-to-output mapping in the language domain (parallel to meaningless gesture imitation in the action domain).

e. Neuroimaging acquisition

Whole-brain high-resolution T1-weighted structural MRI scans were collected on a 3T (Siemens Trio, Erlangen, Germany; repetition time = 1620 ms, echo time = 3.87 ms, field of view = 192 × 256 mm, 1×1×1 mm voxels) scanner using an eight-channel or sixty-four channel head coil. Lesions were manually segmented on each LCVA participant’s T1-weighted structural scan by a trained research assistant under the supervision of an experienced neurologist, both of whom were naïve to the study hypotheses. Lesioned voxels, consisting of both grey and white matter, were assigned a value of 1 and preserved voxels were assigned a value of 0. Binarized lesion masks were registered to a standard template (Montreal Neurological Institute “Colin27”) using a symmetric diffeomorphic registration algorithm (Avants et al., 2008; www.picsl.upenn.edu/ANTS). Volumes were first registered to an intermediate template comprised of healthy brain images acquired on the same scanner. Volumes were then mapped onto the “Colin27” template to complete the transformation into standardized space. To confirm the accuracy of this transformation process, the supervising neurologist inspected all lesion maps. For increased accuracy, the pitch of the template was rotated to approximate the slice plane of each LCVA participant’s scan. This method achieves high intra- and inter-rater reliability (Schnur et al., 2009). Refer to Figure 1 for lesion overlap among the 36 participants with high resolution MRI anatomical scans.

Figure 1.

Voxelwise lesion overlap among 36 LCVA participants. Only voxels with lesion in at least four participants are included.

f. Data analyses

Data were analyzed using linear mixed-effect regression models in the R statistical programming environment with the lme4 package (Bates et al., 2015). Linear mixed-effect models examined trial-level gesture response accuracy as the dependent variable (0–4 for the accuracy of four parameters on which each trial was coded). Each model included two fixed effects: group (participants with LCVA, neurotypical participants) and task condition (meaningless, unnamed, named gesture imitation). All models also included an interaction between group and task condition. Random intercepts were included for subject and item. In all models, we controlled for overall aphasia severity by adding a fixed effect of WAB-Aphasia Quotient (WAB-AQ). We assessed all mixed-effect models by mapping the log likelihood ratio of full and reduced models using a chi square distribution. In addition, we removed interaction terms and assessed the full model when testing for significant main effects. We used an alpha threshold of 0.05 to determine statistical significance. For planned post hoc tests, we used the emmeans package and the Tukey adjustment to correct for multiple comparisons (Lenth, 2021).

Support vector regression-lesion symptom mapping (SVR-LSM) was performed in MATLAB using the SVR-LSM GUI toolbox (DeMarco & Turkeltaub, 2018). SVR-LSM is a multivariate analysis technique that uses machine learning to determine the association between lesioned voxels and behavior, while considering the lesion status of all voxels. Importantly, this approach overcomes limitations of standard voxel-based lesion symptom mapping (VLSM), including inflated false positives from correlated neighboring voxels (Pustina et al., 2018), Type II error due to correction for multiple comparisons (Bennett et al., 2009), uneven statistical power due to biased lesion frequency as a function of vascular anatomy (Mah et al., 2014; Sperber & Karnath, 2018), and inability to account for multiple regions involved in a single behavior (Herbet et al., 2015; Mah et al., 2014; Mirman et al., 2015). Only voxels lesioned in at least four participants were included. Voxelwise statistical significance was determined using a Monte Carlo style permutation analysis (10,000 iterations) in which the critical behavioral data were randomly assigned to a lesion map, and the resulting map was set to a threshold of p < 0.05 to determine chance-level likelihood of a lesion-symptom relationship. We eliminated any clusters with fewer than 500 contiguous voxels (Lacey et al., 2017; Skipper-Kallal et al., 2017). We used the Automated Anatomical Labeling (AAL) atlas (Rolls et al., 2020) and Johns Hopkins diffusion tensor imaging (DTI)-based probabilistic white matter tractography atlas (Mori et al., 2008) to assess the overlap of significant voxels in the SVR-LSM analyses with cortical regions and major white matter fibers at a probability threshold of 25% (see Watson & Buxbaum, 2015).

Below are the model details for each specific hypothesis, as well as planned subsequent analyses relating gesture performance to semantics (CCT, s-weight) and associated lesions. No part of the study analysis was pre-registered prior to the research being conducted. A github repository includes the R code used in this study: https://github.com/hdresang/Apraxia-Gestures. Any additional analysis details may be made available upon request to the corresponding author (hdresang@wisc.edu). No additional stipulations or approvals are No additional stipulations or approvals are required.

1). Indirect semantic route

Predictions 1a) and 1b)

To test the prediction that unnamed meaningful gestures would be more accurate than meaningless gestures and that named meaningful gestures would be more accurate than unnamed meaningful gestures in both groups, we examined fixed effects of group (LCVA, neurotypicals) and task condition (meaningless, unnamed, named gesture imitation), as well as interactions between these terms.¹ Outcome variables, model structure, and post hoc methods were the same as listed above.

Prediction 1c)

To test the prediction that there should be a strong association between semantic processing integrity (CCT performance) and the benefits of meaning on gesture imitation for participants with LCVA, we examined fixed effects of task condition (meaningless, unnamed, named gesture imitation) and CCT performance, as well as interactions between these terms.² Outcome variables, model structure, and post hoc methods were the same as listed above. We predicted that CCT performance would be associated with unnamed meaningful, but not meaningless, gesture performance.

To test the prediction that reduced benefit of meaning on gesture would be associated with lesion to inferior frontal and posterior temporal regions, we performed an SVR-LSM analysis to identify regions associated with reduced benefit of semantics on gesture accuracy. The dependent measure was performance on unnamed meaningful trials, residualized against performance on meaningless trials. Positive residual values indicate better performance on meaningful trials. In addition, we performed a second SVR-LSM analysis to identify regions associated CCT impairment. The dependent measure was total accuracy on CCT, with higher values indicating better performance. We then identified regions of overlap between the two sets of SVR-LSM results. Due to the exploratory nature of this overlap analysis, we adopted a lower threshold of z = −1.65 rather than number of voxels per cluster (Garcea et al., 2020).

Following from evidence that named meaningful gestures were imitated more accurately than unnamed meaningful gestures in individuals with LCVA, we had also planned to assess follow-up predictions regarding the association between lexical-semantic mapping (s-weight) and this benefit. However, to anticipate our results, we observed no difference between named and unnamed gestures.

2). Direct sensory-motor mapping route

To test the prediction that there would be no relationship between direct sensory-motor mapping in language and gesture domains, we examined fixed effects of task condition (meaningless, unnamed, named gesture imitation) and non-word repetition (NWR) performance, as well as interactions between these terms.³ Outcome variables, model structure, and post hoc methods were the same as described earlier. Because NWR was collected only in LCVA participants, neurotypicals were not included in this analysis. We used the BayesFactor package in R to compute the Bayes factor comparing the linear model described above to the same model without a fixed effect of NWR, thus evaluating the prediction that NWR performance would not be associated with performance on any gesture task (Morey et al., 2021).

Results

Each participant’s performance across tasks is reported in Table 2. Patient data from the language and semantic assessments are summarized in Table 3. Task correlations for the LCVA participants who completed the full gesture, language, and semantic assessments are summarized in Table 4.

Table 2.

Gesture imitation performance across tasks.

Subject ID	Meaningless Total Accuracy (%)	Unnamed Total Accuracy (%)	Named Total Accuracy (%)
LCVA_01	76.79a	87.5	91.67
LCVA_02	67.86a	72.92b	66.67c
LCVA_03	71.43a	50b	60.42c
LCVA_04	62.50a	60.42b	54.17c
LCVA_05	78.57	66.67b	72.92c
LCVA_06	75a	83.33	87.5
LCVA_07	73.21a	83.33	87.5
LCVA_08	76.79a	97.92	95.83
LCVA_09	55.36a	43.75b	66.67c
LCVA_10	78.57	83.33	79.17c
LCVA_11	64.29a	NA	72.92c
LCVA_12	76.79a	83.33	95.83
LCVA_13	51.79a	NA	60.42c
LCVA_14	66.07a	64.58b	68.75c
LCVA_15	92.86	93.75	93.75
LCVA_16	71.43a	72.92b	87.5
LCVA_17	62.50a	62.5b	62.5c
LCVA_18	75a	81.25	87.5
LCVA_19	66.07a	81.25	72.92c
LCVA_20	58.93a	66.67b	72.92c
LCVA_21	67.86a	68.75b	72.92c
LCVA_22	0a	79.17b	89.58
LCVA_23	64.29a	72.92b	72.92c
LCVA_24	57.14a	58.33b	66.67c
LCVA_25	85.71	91.67	89.58
LCVA_26	94.64	93.75	91.67
LCVA_27	62.50a	60.42b	75 c
LCVA_28	71.43a	85.42	89.58
LCVA_29	85.71	89.58	83.33c
LCVA_30	73.21a	85.42	91.67
LCVA_31	76.79a	72.92b	89.58
LCVA_32	82.14	89.58	77.08c
LCVA_33	89.29	93.75	91.67
LCVA_34	58.93a	68.75b	62.5c
LCVA_35	62.50a	66.67b	12.5c
LCVA_36	80.36	85.42	81.25c
LCVA_37	73.21a	85.42	79.17c
LCVA_38	69.64a	70.83b	70.83c
LCVA_39	76.79a	79.17b	81.25c
LCVA_40	69.64a	81.25	87.5
NT_01	82.14	85.42	91.67
NT_02*	75	70.83	83.33
NT_03	83.93	93.75	93.75
NT_04	89.29	87.5	89.58
NT_05	80.36	95.83	93.75
NT_06	89.29	85.42	89.58
NT_07	89.29	97.92	97.92
NT_08	80.36	NA	89.58
NT_09	89.29	87.5	87.5
NT_10	89.29	87.5	97.92
NT_11	82.14	87.5	89.58
NT_12	89.29	85.42	91.67
NT_13	92.86	97.92	89.58
NT_14	96.43	89.58	95.83
NT_15	87.50	91.67	97.92
NT_16	98.21	89.58	91.67
NT_17	85.71	87.5	95.83
NT_18	87.50	85.42	91.67

Abbreviations: LCVA, left-hemisphere cerebrovascular accident; NT, neurotypical.

^*Participant excluded from analysis.

^aParticipants with apraxic performance, determined by >2SD below the NT mean for meaningless gesture accuracy.

^bParticipants with apraxic performance, determined by >2SD below the NT mean for unnamed gesture accuracy.

^cParticipants with apraxic performance, determined by >2SD below the NT mean for named gesture accuracy

Table 3.

Language and semantic performance.

Subject ID	PNT Accuracy (%)	SP Model Parameter Estimates			CCT Accuracy (%)	NWR Accuracy (%)	WAB-AQ
		Semantic Parameter	Phonological Parameter	RMSD
LCVA_01	93	0.050	0.017	0.006	91	90	95.1*
LCVA_02	78	0.036	0.016	0.016	89	35	88.5
LCVA_03	41	0.015	0.023	0.053	78	10	51.6
LCVA_04	5	0.002	0.016	0.022	63	2	34.2
LCVA_05	77	0.028	0.028	0.021	66	82	88.3
LCVA_06	81	0.039	0.016	0.008	88	28	82.4
LCVA_07	85	0.027	0.025	0.014	80	47	91.2
LCVA_08	87	0.027	0.031	0.012	84	73	99.3*
LCVA_09	82	0.025	0.029	0.004	80	73	90.2
LCVA_10	79	0.026	0.026	0.016	72	65	88.1
LCVA_12	73	0.020	0.025	0.026	81	NA	83.3
LCVA_14	93	0.050	0.025	0.005	81	32	86.3
LCVA_16	12	0.022	0.024	0.020	72	22	34.6
LCVA_17	70	0.044	0.011	0.010	73	22	88
LCVA_18	87	0.046	0.016	0.004	77	77	89.7
LCVA_19	79	0.035	0.025	0.005	80	80	96*
LCVA_20	70	0.041	0.013	0.014	78	36.7	69.8
LCVA_21	17	0.013	0.012	0.033	53	10	33.2
LCVA_22	89	0.038	0.023	0.011	80	50	81.3
LCVA_23	86	0.042	0.025	0.006	72	58	90.4
LCVA_25	87	0.031	0.028	0.004	75	48	71.6
LCVA_26	93	0.038	0.030	0.002	61	80	92.6
LCVA_27	69	0.028	0.018	0.018	77	15	68.6
LCVA_28	81	0.030	0.020	0.007	86	48	87.6
LCVA_31	83	0.038	0.020	0.007	83	63.3	93.9*
LCVA_32	79	0.030	0.022	0.010	81	43	89.5
LCVA_35	83	0.028	0.028	0.009	69	77	89.2
LCVA_36	84	0.027	0.025	0.014	91	20	71.9
LCVA_39	66	0.021	0.026	0.024	77	NA	90.1
LCVA_40	64	0.025	0.017	0.019	89	6.6	73.8

Abbreviations: LCVA, left-hemisphere cerebrovascular accident; PNT, Philadelphia Naming Test; RMSD, an index of model fit, Root-Mean-Square Difference between the observed and model-predicted response proportions; CCT, Camels and Cactus Test; NWR, Philadelphia Nonword Repetition Test; WAB-AQ, Western Aphasia Battery-Aphasia Quotient (<93.8 indicates aphasia).

^*Participants without aphasia (>93.8 of WAB-AQ).

Table 4.

Task correlation matrix for N = 30 LCVA participants who completed all tasks.

	Meaningless	Unnamed	Named	PNT	NWR	WAB-AQ	CCT
Meaningless	1
Unnamed	0.736***	1
Named	0.540**	0.611***	1
PNT	0.286	0.416*	0.227	1
NWR	0.285	0.326	0.099	0.647***	1
WAB-AQ	0.205	0.371	0.152	0.920***	0.738***	1
CCT	0.010	0.260	0.337	0.488**	0.028	0.442	1

Abbreviations: PNT, Philadelphia Naming Test; NWR, Philadelphia Nonword Repetition Test; WAB-AQ, Western Aphasia Battery-Aphasia Quotient; CCT, Camels and Cactus Test.

^*Correlation is significant at the 0.05 level (uncorrected, 2-tailed).

^**Correlation is significant at the 0.01 level (uncorrected, 2-tailed).

^***Correlation is significant at the 0.001 level (uncorrected, 2-tailed); these are the only significant results after Bonferroni corrections for multiple comparisons.

1). Indirect semantic route

Refer to Figure 2 for group by gesture condition performance, after controlling for random effects of subjects and items. There was a significant main effect of group on accuracy (χ²⁽³⁾ = 28.52, p < 0.001), such that neurotypical controls performed more accurately than LCVA patients. There was also a main effect of task (χ²⁽⁴⁾ = 45.14, p < 0.001) and a significant group-by-task interaction (χ²⁽³⁾ = 227.74, p < 0.001). Subsections below report post hoc results relevant to each prediction.

Figure 2.

Behavioral means and standard error of group by task condition interactions. Abbreviations: LCVA, left-hemisphere cerebrovascular accident; NT, neurotypical.

1a). Do semantics benefit gesture?

Post hoc analyses revealed that LCVA patients performed better on both named (β = 0.30, SE = 0.05, t = 5.93, p < 0.001) and unnamed meaningful compared to meaningless gestures (β = 0.25, SE = 0.05, t = 4.91, p < 0.001). In contrast, controls showed similarly high accuracy for all gesture tasks: named and unnamed gestures (β = 0.13, SE = 0.08, t = 1.66, p = 0.56), named and meaningless gestures (β = 0.19, SE = 0.08, t = 2.59, p = 0.10), and unnamed and meaningless gestures (β = 0.07, SE = 0.08, t = 0.85, p = 0.96). These findings suggest that LCVA patients showed significant benefits from semantics on gesture performance.

1b). Do lexical labels benefit gesture?

Post hoc analyses revealed that accuracy on named and unnamed gestures did not differ (β = 0.05, SE = 0.05, t = 0.92, p = 0.94) and in fact were correlated (see Table 4). Because there was no benefit of visual/auditory lexical labels for patients with LCVA, we did not perform behavioral or neuroanatomical follow-up analyses of this effect (i.e., analyses 1c).

1c). What predicts gesture benefits?

Following from the results of analysis (1a) indicating a benefit of semantics on gestures for patients with LCVA, we examined potential (i) behavioral and (ii) neuroanatomical predictors of this benefit.

(i). Behavioral predictors:

There was a significant main effect of CCT on gesture accuracy (χ²⁽³⁾ = 17.00, p < 0.001), such that higher knowledge of semantic associations corresponded to higher gesture imitation performance. This effect was robust after controlling for variance attributed to overall aphasia severity, which was not a significant predictor of accuracy (β = 0.01, SE = 0.01, t = 0.95, p = 0.35). The interaction between CCT and task condition was also significant (χ²⁽²⁾ = 16.44, p < 0.001). Post hoc pairwise analyses revealed this effect when comparing both named (β = 0.03, SE = 0.01, t = 3.99, p < 0.001) and unnamed to meaningless gestures (β = 0.02, SE = 0.01, t = 2.52, p = 0.03). Thus, patients with higher CCT scores performed better on both types of meaningful gestures as compared to meaningless gestures; i.e., patients with lower CCT scores showed a reduced benefit of meaning on imitation accuracy. There was no interaction between CCT and named versus unnamed gestures (β = 0.01, SE = 0.01, t = 1.42, p = 0.33).

(ii). Neuroanatomical predictors:

SVR-LSM analyses identified the regions critical for the benefit of semantics on imitation accuracy. Figure 3 shows lesions associated with reduced benefit of semantics (unnamed vs meaningless) on imitation accuracy. Refer to Table 5 for AAL regions and peak MNI coordinates identified in the SVR-LSM analyses. As a post-hoc exploratory analysis, we identified the overlap between regions associated with both CCT impairments and reduced benefits of semantics on gesture performance. Refer to Figure 4 and Table 6. Due to the exploratory nature of this overlap analysis, we adopted a lower threshold of z = −1.65 rather than number of voxels per cluster (Garcea et al., 2020). These results should be interpreted with caution.

Figure 3.

(unnamed meaningful vs. meaningless contrast). A. Clusters at surface level. B. Clusters at all levels.

Table 5.

Peak MNI coordinates identified in SVR-LSM analysis of reduced benefits of semantics on gesture performance. Voxels per region > 20 voxels.

AAL Label	AAL #	Mean Center of Mass			Peak Z-Score	# Voxels in Cluster
		X	Y	Z
Precentral gyrus	1	−24	−33	46	−3.35	170
Precentral gyrus	2	−42	−17	38	−3.35	155
Superior frontal gyrus, dorsolateral	3	6	−7	38	−3.35	73
Superior frontal gyrus, dorsolateral	4	−41	−9	38	−3.35	114
Inferior frontal gyrus, pars opercularis	7	18	−8	39	−2.64	72
Inferior frontal gyrus, pars opercularis	8	−44	−9	38	−3.35	90
Superior frontal gyrus, medial	19	−11	−17	42	−3.35	116
Superior frontal gyrus, medial	20	−18	−8	38	−3.35	135
Gyrus rectus	23	−4	19	32	−2.24	32
Insula	34	−22	−5	15	−2.95	35
Amygdala	45	0	−105	5	−2.21	24
Inferior occipital gyrus	57	6	−44	51	−2.95	115
Inferior occipital gyrus	58	−35	−50	47	−2.50	110
Fusiform gyrus	59	15	−74	37	−3.35	131
Fusiform gyrus	60	−44	−69	37	−3.35	151
Postcentral gyrus	61	38	−69	29	−2.25	33
Inferior parietal, excluding SMG & AG	66	−50	−66	14	−2.95	22
Supramarginal gyrus (SMG)	67	−2	−63	42	−2.55	125
Supramarginal gyrus (SMG)	68	−15	−56	42	−2.55	87
Angular gyrus (AG)	69	−4	−24	39	−3.12	83
Angular gyrus (AG)	70	−22	−34	43	−2.89	28
Temporal gyrus, superior	85	38	−66	−11	−4.30	22
Temporal gyrus, superior	86	−72	−63	−11	−4.30	23

Abbreviations: AAL, Automated Anatomic Labeling.

Figure 4.

Lesion associated with overlap between impaired CCT performance and reduced benefit of semantics on gesture.

Notes: Performed on the subset of Figure 3 participants who also completed CCT assessment. Red: CCT performance, Blue: Benefit of semantics on gesture, Magenta: Overlap.

Table 6.

Voxels identified in SVR-LSM overlap between impaired CCT performance and reduced benefits of semantics on gesture performance. Threshold at z = −1.65.

AAL Label	AAL #	Number of Voxels > 0	Proportion of ROI > 0
Precentral gyrus	1	345	0.012
Inferior frontal gyrus, pars opercularis	7	1479	0.038
Inferior frontal gyrus, pars triangularis	10	1829	0.091
Inferior frontal gyrus, pars orbitalis	12	52	0.006
Precuneus	71	444	0.058
Paracentral lobule	73	240	0.030
Putamen	77	154	0.018
Temporal gyrus, superior	85	1212	0.066
Temporal pole, superior	86	37	0.004
Temporal gyrus, middle	89	1718	0.044

Abbreviations: AAL, Automated Anatomic Labeling.

2). Direct sensory-motor mapping route

There was no effect of Philadelphia Non-Word Repetition Task (NWR) on gesture accuracy (χ²⁽³⁾ = 2.88, p = 0.410). The interaction between NWR and gesture task condition was also not significant (χ²⁽²⁾ = 2.75, p = 0.253). A Bayes Factor of 0.25 indicated moderate evidence in support of the null hypothesis compared to the alterative (Quintana & Williams, 2018). Thus, performance on non-word repetition was not likely to be associated with gesture imitation in any of the task conditions.

Discussion

Many patients with left-hemisphere cerebrovascular accident (LCVA) experience chronic impairments in gestures (limb apraxia), language (aphasia), or both. Although limb apraxia and aphasia frequently co-occur, they also dissociate. Such evidence is often taken to indicate that the mechanisms underlying the two disorders are independent, when the more appropriate inference may be that the overlap in language and action networks is not complete. The purpose of this study was to test behavioral and neuroanatomic predictions about shared and distinct processes in gesture and language enabled by dual-route models in each domain.

First, we tested the hypothesis that the indirect route shares overlapping semantic processes across language and gesture domains. In particular, we expected to find that semantics facilitate gesture imitation (Prediction 1a) and that providing gesture names would further facilitate gesture imitation (Prediction 1b). We replicated previous findings that adults with chronic LCVA perform gesture imitation more accurately with meaningful than meaningless gestures (Bartolo et al., 2001; Buxbaum et al., 2005; Goldenberg & Hagmann, 1997; Tessari et al., 2007). In particular, LCVA patients performed better on both named and unnamed meaningful compared to meaningless gestures. This indicates that semantic information confers a benefit for gesture performance, consistent with the involvement of the indirect route in meaningful gesture imitation. Neurotypical adults performed more accurately than LCVA and showed similarly high performance on all gesture conditions.

In contrast to the observed benefit of meaning in gesture imitation, LCVA patients showed similar performance on named and unnamed meaningful gestures, indicating no additional benefit from lexical input on performance. This finding is inconsistent with prior evidence that individuals with apraxia can benefit from additional semantic activation via lexical input (Cubelli et al., 2000). Although we controlled for overall aphasia severity, it is unclear whether the benefit of lexical input in gesture tasks may be conditioned by the degree to which individuals also have language comprehension deficits. Future research may investigate whether apraxic deficits are analogous to findings from aphasic verb production studies, which suggest that compared to neurotypical controls, individuals with LCVA may adapt to their impairments by relying more on semantic than lexical cues (Dresang et al., 2021). Taken together, our primary set of findings showed that semantic but not lexical information conferred benefits for gesture imitation for patients with chronic LCVA.

Second, we examined behavioral and neuroanatomical predictors of the benefit of semantics on gesture performance in participants with LCVA (Prediction 1c). Regarding behavioral predictors, we predicted that the integrity of semantic processing as assessed by a pictorial task requiring neither language nor gesture access (Camel and Cactus Test [CCT]; Bozeat et al., 2000) would be associated with the degree of semantic benefits on gesture. Our findings supported this prediction, such that higher performance on CCT was associated with higher gesture imitation accuracy on both named and unnamed meaningful as compared to meaningless gestures. Thus, patients with impaired semantic processing showed a reduced benefit of meaning on gesture performance, even when controlling for overall aphasia severity.

Regarding neuroanatomical predictors, we identified brain regions associated with benefits of semantic information on gesture following LCVA. Our analyses found that reduced benefit of semantics was associated with lesion to a left-lateralized cortical network, predominantly including posterior superior temporal gyrus, inferior and superior frontal gyri, precentral gyrus, and to a lesser extent, regions of the inferior parietal lobe. A subset of these regions overlapped with lesion associated with CCT impairment, including superior and posterior middle temporal gyri and inferior frontal regions. The identified frontal regions have previously been implicated in semantic control and retrieval processes (Jackson, 2021; Krieger-Redwood et al., 2015; Moss et al., 2005; Whitney et al., 2011) as well as multimodal gesture-speech task performance (Vigliocco et al., 2020; Zhao et al., 2018, 2021) and combined apraxic and aphasic impairments (Weiss et al., 2016). Semantic processing as assessed with both language and gesture tasks has been associated with left inferior frontal gyrus as well as superior and middle temporal gyri (Andric & Small, 2012; Straube et al., 2012). In addition, the precentral gyrus has been associated not only with voluntary movements (e.g., Jasper & Penfield, 1949) but also with multimodal gesture-speech tasks (Vigliocco et al., 2020) and action semantics (Hauk et al., 2004; Kemmerer et al., 2012). Furthermore, these results are also consistent with previous findings that imitation of meaningful gestures involves a ventral semantic route with a major node in the left posterior temporal cortex (e.g., Buxbaum et al., 2014; Buxbaum, 2017). Taken together with the behavioral predictors of gesture benefit, these findings are largely consistent with accounts that gesture meaning is a subset of broader semantic knowledge (Andric & Small, 2012; Straube et al., 2012) rather than being a distinct type of semantic representation that is accessible only in gesture tasks (Willems et al., 2009).

Finally, we tested the prediction that there would be no association between measures of the direct sensory-motor mapping route across language and gesture domains (Prediction 2). For these tasks, we evaluated nonword repetition and meaningless gesture imitation, both of which cannot rely on semantic information to facilitate processing. Following our predictions, we found there was no main effect of nonword repetition on gesture accuracy and no interaction between nonword repetition and gesture task condition. A Bayes Factor analysis further demonstrated moderate evidence of the null hypothesis - i.e., that there was no association between nonword repetition and gesture performance. These results suggest that direct routes that map auditory-verbal and visual-kinematic input to output transformations rely on distinct systems that can be differentially impaired in patients with LCVA.

There are several limitations to this work, which comes from a convenience sample and retrospective data analysis. Analyses were limited to the available dataset and sample size, potentially limiting the sensitivity of neuroimaging analyses to detect overlapping regions associated with semantic processing and the role of semantics on gestures. Future work may examine additional neuropsychological assessments of language (e.g., real word repetition) and semantic processing of actions in both linguistic (verb) and pictorial formats (e.g., Bak & Hodges, 2003; Dresang et al., 2019). More specific assessments and larger samples for neuroimaging could support stronger conclusions about the nature and locus of shared semantic processes across language and action domains, whereas the current findings are limited to the relationship between semantic processing as assessed by a pictorial task (albeit one that is often used in aphasia assessment) and the role of semantics in gesture. Moving forward, it will be important to further characterize how the dual-route system responds to damaged semantic integrity or access (e.g., via lexical input). That is, while it should theoretically be possible to imitate meaningful gestures with the direct route alone, there is some evidence that a damaged semantic route may interfere with the direct route, depressing performance (Mauri et al., 2021). Improving our mechanistic understanding of the interactivity of the two routes in both language and action will point to neurorehabilitation treatment targets for individuals with LCVA.

Conclusion

The current results support the overarching hypothesis that portions of the indirect semantic route are shared in dual-route models of language and action, while direct sensory-motor mapping routes differ across the two domains. These results are consistent with previous evidence supporting a domain-general semantic system that is accessed during both gesture and language processing. This study informs dual-route models of action and language and provides a foundation from which future work can examine cross-domain benefits of semantic information for LCVA patients.

Highlights:

• Gesture imitation in left hemisphere stroke benefits from gesture meaning

• Gesture imitation does not benefit from naming (labeling) of gestures

• Semantic processing and posterior temporal lesion predict the benefit of meaning

• Meaningless speech repetition and meaningless gesture imitation are not associated

• Portions of indirect route are shared across language and action; direct routes are not shared