Neural structures supporting spontaneous and assisted (entrained) speech fluency

Speech entrainment can reinstitute fluent speech for some individuals with chronic aphasia. Using machine learning applied to lesion symptom mapping, Bonilha et al. demonstrate that individuals with loss of dorsal stream, but preservation of ventral stream, structures are more likely to achieve improved speech fluency with entrainment.

Abstract

Non-fluent speech is one of the most common impairments in post-stroke aphasia. The rehabilitation of non-fluent speech in aphasia is particularly challenging as patients are rarely able to produce and practice fluent speech production. Speech entrainment is a behavioural technique that enables patients with non-fluent aphasia to speak fluently. However, its mechanisms are not well understood and the level of improved fluency with speech entrainment varies among individuals with non-fluent aphasia. In this study, we evaluated the behavioural and neuroanatomical factors associated with better speech fluency with the aid of speech entrainment during the training phase of speech entrainment. We used a lesion-symptom mapping approach to define the relationship between chronic stroke location on MRI and the number of different words per second produced during speech entrainment versus picture description spontaneous speech. The behavioural variable of interest was the speech entrainment/picture description ratio, which, if ≥1, indicated an increase in speech output during speech entrainment compared to picture description. We used machine learning (shallow neural network) to assess the statistical significance and out-of-sample predictive accuracy of the neuroanatomical model, and its regional contributors. We observed that better assisted speech (higher speech entrainment/picture description ratio) was achieved by individuals who had preservation of the posterior middle temporal gyrus, inferior fronto-occipital fasciculus and uncinate fasciculus, while exhibiting lesions in areas typically associated with non-fluent aphasia, such as the superior longitudinal fasciculus, precentral, inferior frontal, supramarginal and insular cortices. Our findings suggest that individuals with dorsal stream damage but preservation of ventral stream structures are more likely to achieve more fluent speech with the aid of speech entrainment compared to spontaneous speech. This observation provides insight into the mechanisms of non-fluent speech in aphasia and has potential implications for future research using speech entrainment for rehabilitation of non-fluent aphasia.

Introduction

Non-fluent aphasia is defined by significantly reduced speech production, ranging from total mutism to utterances composed of no more than three to five words (Geschwind, 1970; Gleason et al., 1975), Individuals with non-fluent aphasia typically present with slow speech, effortful speech, frequent pauses, short phrase length, single words, and poor articulation (Poeck, 1989).

Non-fluent aphasia is relatively common among stroke survivors with chronic language impairments, i.e. 6 months after the stroke. Unfortunately the therapeutic options for restoring fluency are suboptimal for this group of individuals (Kelly et al., 2010; Brady et al., 2012). Recent controlled studies suggest that speech therapy can lead to meaningful improvements and partly improve language production for chronic aphasia in general (Breitenstein et al., 2017). However, it is well known that fluency is one of the most refractory symptoms and most patients with non-fluent aphasia usually persist with long lasting moderate to severe impairments even after therapy (Kelly et al., 2010; Brady et al., 2012). As the inability to produce fluent speech is strongly associated with reduced quality of life (Poeck, 1989), it follows that the lack of a more efficient treatment for non-fluency constitutes an important gap in the care of stroke survivors and in the field of neurorehabilitation in general.

One of the most challenging aspects when treating fluency impairments is that the patient with chronic non-fluent aphasia seldom achieves fluent speech on his/her own. As such, forms of therapy that require the patient with aphasia to try to speak without assistance are often unsuccessful, usually leading to fragmented utterances with many errors (Kelly et al., 2010; Brady et al., 2012). With time, these errors may perpetuate and be re-enforced, leading to worse results (Boyle, 2015). Fortunately, there are forms of speech therapy that can use assistive strategies to achieve fluent speech, including a promising innovative strategy entitled speech entrainment (Fridriksson et al., 2012). Rosenbek and colleagues were among the first to use speech entrainment explicitly as a part of a published approach to treat adult neurogenic speech problems (their approach focused specifically on apraxia of speech) (Rosenbek et al., 1973). Speech entrainment involves engaging the patient with aphasia to mimic in real time the speech presented by a therapist (in person or videotaped). During speech entrainment, the patient with aphasia attempts to speak in unison with the therapist or videotaped speech model.

Even though speech entrainment is a promising technique that induces fluent speech in some patients with non-fluent aphasia, its mechanisms are not well understood. Furthermore, it is not clear which individuals with non-fluent aphasia can have their speech entrained. We have previously demonstrated that audio-visual stimuli (video showing the speaker’s mouth along with the auditory speech signal) are more effective at entraining speech compared with auditory speech by itself (Fridriksson et al., 2012). We also demonstrated that individuals with Broca's aphasia show a better response to speech entrainment than individuals with other forms aphasia (Fridriksson et al., 2015a).

Knowledge on the mechanisms supporting the re-establishment of fluency with entrainment would be important to inform future clinical trials involving speech entrainment therapy. For example improved understanding may aid in predicting who would most likely benefit from speech entrainment therapy and further developing more advanced forms of speech entrainment.

In this study, we aimed to evaluate the critical neuroanatomical patterns related to better speech fluency using speech entrainment. This study did not assess speech entrainment as a therapeutically approach; we focused on the mechanistic aspects of speech entrainment during the training phase. We used a lesion-symptom mapping approach to define the intect neuroanatomical structures required to speak with the aid of speech entrainment. We used a machine learning approach (shallow neural network) to determine if a neuroanatomical model could predict which individuals with chronic aphasia could speak more fluently with the aid of speech entrainment compared to their spontaneous speech ability.

We hypothesized that individuals with better speech fluency during entrainment would exhibit damage in supra-sylvian areas, thus having reduced spontaneous fluency, but also preserved areas in infra-sylvian regions that are associated with speech comprehension.

Materials and methods

Participants

Forty-eight consecutive chronic stroke-survivors (18 female, mean age = 58.4 ± 10.1 years) with a history of a single prior left hemisphere stroke participated in this study. All participants were at least 6 months post-stroke and reported no history of other neurological diseases, psychiatric problems or developmental language impairments. The mean time since the stroke was 43.6 ± 49.5 months.

Informed consent was obtained from all participants, and this study was approved by the Institutional Review Boards at the Medical University of South Carolina and at the University of South Carolina.

Behavioural measures and procedures

All participants were administered the Western Aphasia Battery (Revised) (Kertesz, 2007) (WAB-R) to determine the presence of aphasia and to characterize aphasia based on traditional subtypes. Patients without aphasia were included in this study to serve as a comparison for brain lesions not associated with aphasia.

Speech tasks and procedures

Speech tasks and procedures were adapted from our previous studies (Fridriksson et al., 2012, 2015a; Feenaughty et al., 2017). All participants completed two tasks: (i) spontaneous speech; and (ii) audio-visual speech entrainment.

To assess spontaneous speech abilities, participants were required to describe three pictures including the picnic, cookie theft and, the circus scenes from the WAB-R, the Boston Diagnostic Aphasia Examination - third edition, and Apraxia Battery for Adults - second edition, respectively. Each picture was displayed for 2 min. Participants were instructed to describe what was happening in each picture and to try to talk in complete sentences using a pace they found most comfortable. Picture description was therefore used to measure speech fluency that reflects typical speech output. This was preferred instead of phonological or semantic fluency tests, which are not realistic representations of connected speech.

The measures of spontaneous speech during picture description and the measure of speech entrainment were obtained within the same testing session and no speech therapy was provided as part of the current study.

Speech entrainment abilities were measured using similar procedures we have used in our previous studies (Fridriksson et al., 2012, 2015a; Feenaughty et al., 2017). Each participant was seated in front of a lap-top computer used for stimulus presentation and then fitted with headphones for audio presentation. The stimuli consisted of short videos showing a speaker’s face below the nose and the participant is instructed to mimic the speaker in real time. More specifically, the participant attempts to speak in unison with the speech model seen on the video. The video stimuli consisted of three short scripts varying in topic, length (i.e. between 48 and 58 words), and duration (i.e. ranging from 38 to 44 s, average = 41.6 ± 3.2 s) and were presented using Psychtoolbox (http://psychtoolbox.org) and MATLAB software. Subjects were instructed to mimic the speech in real time, i.e. and match the rate of the script. Prior to the beginning of the speech entrainment session, a sound test determined the listening level that was appropriate for each participant and subsequently each participant practiced the task on a short speech entrainment script.

During spontaneous speech and speech entrainment, participants were video-recorded. Videos were saved to a computer for subsequent scoring and analysis of speech samples. All speech samples were orthographically transcribed by trained research assistants and checked by a certified speech-language pathologist with expertise in aphasia and transcription. Discrepancies were resolved through discussion and forced choice agreement.

This study focused on the relationship between the numbers of different words produced per second during speech entrainment in comparison with picture description. For each category (speech entrainment or picture description) the total number of different words was adjusted based on the session time, i.e. divided by the time permitted for each participant to produce a given speech sample (2 min in the case of picture description and 38 to 44 s during speech entrainment) and then averaged across the three sessions for each behavioural task, yielding a time-adjusted measure of different words.

To determine whether speech fluency improved with the aid of speech entrainment relative to spontaneous speech, we calculated a ratio between speech entrainment and picture description (SE/PD). Using this measure, it is straightforward to see that if a participant speaks more fluently during speech entrainment compared to picture description, their SE/PD ratio would be >1.0. The opposite, SE/PD ratio < 1.0, would be the case for participants whose picture description was more fluent than speech entrainment. The SE/PD incorporated only different words produced during speech entrainment and picture description and we believe it provides a realistic measure of accuracy that is based on careful transcription. Neologisms, perseverations, and unrelated words were excluded and only different words are included.

This is a time-consuming approach because it requires detailed transcription of the speech samples and the assessment by a trained clinician. Nonetheless, we feel that it is accurate for representing a realistic representation of spontaneous speech or speak with the aid of speech entrainment.

Imaging acquisition and preprocessing

All participants underwent high-resolution T₁ and T₂ MRI scanning using a Siemens 3 T Trio System with a 12-channel head-coil located at the University of South Carolina. For each participant, T₁-weighted images were obtained using an MP-RAGE sequence with 1 mm³ isotropic voxels, field of view matrix of 256 × 256 mm, 9° flip angle, and 192 sagittal slice sequence with repetition time = 2250 ms, inversion time = 925 ms, and echo time = 4.15 ms, with parallel imaging (GRAPPA = 2, 80 reference lines). T₂-weighted images were obtained using a sampling perfection with application optimized contrasts by employing differing flip angle evolutions (3D SPACE) sequence for the purpose of demarcating lesions. The parameters for this T₂ scan included a voxel size of 1 mm³, 256 × 256 mm field of view matrix, 160 sagittal slice sequence, variable flip angle, repetition time = 3200 ms, echo time = 212 ms, 5/8 partial Fourier, with no parallel slice acceleration. Slice centre and angulation were similar to the T₁ image sequence.

The post-stroke chronic lesions were manually delineated on the T₂-weighted images by a neurologist (L.B.) who was blinded to the behavioural scores at the time of the lesion drawing. To define the magnitude of damage to specific cortical regions, we used the John’s Hopkins University (JHU) brain Atlas developed by Faria et al. (2010), which is a map of grey and white matter regions in standard stereotaxic space. Using SPM12 (https://www.fil.ion.ucl.ac.uk/spm/) and MATLAB scripts developed in-house (Rorden et al., 2012). The stroke lesions were spatially normalized to standard space through the following steps: (i) the T₂ image was linearly co-registered onto the T₁-weighted scan and this transformation was applied to the lesion mask; (ii) the resliced lesion maps were smoothed with a 3 mm full-width at half-maximum Gaussian kernel to remove uneven edges associated with manual drawing; (iii) an enantiomorphic approach (Nachev et al., 2008) using SPM12's unified segmentation-normalization was applied to normalize the T₁-weighted images onto the standard space, using a chimeric T₁-weighted image where the area corresponding to the stroke lesion was replaced by the mirrored equivalent region in the intact (right side) hemisphere; and (iv) only voxels with a probability >50% on corresponding to the lesion mask were maintained in the final normalized lesion mask.

Once the lesion masks were placed in standard space, they were overlaid onto the anatomical parcellation atlas to determine: (i) overall lesion size; and (ii) the percentage of each cortical region damaged in each individual.

Statistical analyses and machine learning

Neuroanatomical predictors

Descriptive statistics were performed and, as an exploratory first pass, we assessed the correlation coefficient between regional damage and the SE/PD ratio, with regional damage being defined as the percentage of damage to each region of interest included in this study. The level of statistical significance was set at P < 0.05 with Bonferroni correction. This analysis was performed to assess the relationship between variables, but did not guide subsequent analyses.

To determine the most important regions associated with speech entrainment, we constructed a shallow feedforward neural network with one hidden layer, leading to two classification groups defined based on the SE/PD ratio. We used MATLAB (version 2018a including the Deep Learning Toolbox) to implement and run the neural networks. The classification target was an n-sized vector (n = number of subjects) defined based on the SE/PD threshold (1 if above the threshold, or 0 otherwise). The predictors were a n × m matrix where n was the number of subjects and m were 24 different regions of interest, which were selected based on the a priori knowledge about grey and white matter areas associated with language either as specifically related to language processing, or as related to domain general aspects of language; they are summarized in Supplementary Table 1.

A shallow neural network linear approach with one hidden layer was chosen because it is relatively straightforward to implement and interpret. It also permits the assessment of feature importance, which is less intuitive with many hidden layers or not feasible with non-linear approaches. Thus, the neural network was a linear feedforward model whose network architecture (Fig. 1) was defined through a grid search where the number of neurons in the intermediate layer ranged from 2 to 4, and the classification groups had a division threshold ranging from 1 to 1.3 (i.e. each classification group contained individuals whose SE/PD ratio was below or above the threshold). The number of neurons in the intermediate layer was arbitrarily chosen, with care to avoid unnecessarily complex (large) hidden features. The classification thresholds were also arbitrarily chosen, with the intent of capturing a relative benefit of speech entrainment over picture description, excluding very high thresholds with only a few individuals in the high SE/PD group.

Figure 1.

Shallow neural network. A shallow neural network was used in this study, in which the input layer consisted of the percentage of damage to each one of the 24 regions of interest listed in Supplementary Table 2. The input to each node in the hidden layer (HL node i) consisted as the vector multiplication between the values in the input layer (X) and their weights (W), with an added bias (b). The hidden layer provided input to the classification layer, where each individual was classified based on the threshold SE/PD. The weights and biases were defined based on training and validation subsets, and tested on an independent testing group. A grid search was performed to define the optimal number of nodes in the hidden layer, and the optimal SE/PD threshold for classification.

The neural network parameters (weights and biases) were optimized based on randomly chosen training and validation sub-datasets (the training set contained 70% of the individuals in the entire sample, and the validation group, 15%), and tested on an independent sample (the testing group contained the remaining 15% of the entire sample). The neural network parameters were defined based only on the training and validation datasets, and the testing dataset did not influence the model.

The neural network was assessed 250 times for each component of the search grid (i.e. different number of neurons and different thresholds), each time with a different random allocation of subjects to training, validation, and testing (maintaining the same 70%, 15%, 15% training, validation and testing allocation proportions, as described above). Overall, 250 simulations took ∼5 to 10 min to compute using a signal thread computer. The classification accuracy was recorded for each run. We also recorded the back propagation weights from each classification category to the hidden layer, and subsequently to the input layer (i.e. predictor regions) to define the weight of each region towards each classification class.

The same procedure was repeated 250 times with random data, i.e. the target vector was shuffled and the neural network was again trained, validated and tested 250 times, with recorded accuracy and back propagation weights.

We then compared the distribution of accuracies between real and random data. The real data model that achieved mean accuracy higher than 95% of the accuracies from the random data model was considered to be statistically significant. Likewise, the same process was repeated for the regions used as predictors. The regions were considered to be statistically significant if their average weights from real data were 95% higher or lower than the weights from randomized data, towards each classification category.

We also assessed whether the continuous value of SE/PD could be predicted by the degree of damage in the 24 regions of interest. For this analysis, we used a similar machine learning approach, where the predictors were damaged in the 24 regions of interest, but instead of a binary class, the resulting output was the continuous SE/PD ratio, i.e. a correlational machine learning approach. For this step, we did not used a grid search (as the SE/PD threshold did not apply), but rather used a fixed 4-node hidden layer. The training, validation and testing split ratios were the same, and the model was tested 1000 times, each iteration with a different random allocation of subjects into the testing, validation and testing groups. The resulting correlation coefficient was assessed for each run and their distribution was compared with the distribution of correlation coefficients for 1000 runs of random SE/PD ratios. We considered the model to be statistically significant if the average of correlation coefficients was higher than 95% of the correlation coefficients from the random distribution.

Behavioural predictors

To define if behavioural assessments could be used to classify binary speech entrainment and picture description results, the same classification procedure described above was repeated using behavioural measures instead of using neuroanatomical predictors. Specifically, the following 13 categories were used: age, time since the stroke, WAB spontaneous speech subscore, WAB comprehension subscore, WAB repetition subscore, WAB fluency subscore, WAB information content subscore, presence or absence of aphasia, and having aphasia classified as Broca's versus anomic versus Wernicke's versus conduction versus global. Thus, the predictors were composed by n subjects × 13 behavioural variables, and the classification procedures were otherwise exactly the same as for neuroanatomical predictors.

We did not include behavioural and neuroanatomical predictors together in the same model because they are highly interrelated. For example, a participant with lesions involving the inferior and middle frontal gyri is likely to have aphasia classified as Broca’s. As such, if the same model included lesion in the frontal gyri and aphasia classification, the weights to each input node would be fragmented and preclude the identification of the neuroanatomical structures and prevent the neurobiological mechanistic investigation propositioned in this study. While this observation is also valid for lesion load, the correlation between lesion load and behaviour was anticipated to be higher than the correlation between regional lesion load. This was tested through multiple correlations that are discussed and presented in the results section below.

Data availability

The data used in this study are available to researchers upon qualified request to the corresponding author.

Results

Description of demographics and language measures

Among the 48 participants included in this study, the mean WAB Aphasia Quotient was 75.3 ± 24.2. The mean and standard deviations for the WAB-R subscores were as follows: spontaneous speech, 14.6 ± 5.2; comprehension, 8.6 ± 1.7; repetition, 7.2 ± 3.0; fluency, 6.9 ± 2.9; and information content, 7.6 ± 2.5.

Fourteen participants (29.2%) had Broca’s aphasia, 14 (29.2%) anomic aphasia, three (6.3%) conduction aphasia, two (4.1%) Wernicke’s aphasia, one (2.1%) global aphasia, and 13 (27.1%) did not have aphasia. The demographics and behavioural information is presented in Supplementary Table 2.

The average SE/PD ratio was 1.28 ± 1.19 different words. Individuals with Broca’s aphasia had the highest SE/PD ratio (2.53 ± 1.6), followed by anomic aphasia (0.92 ± 0.53), Wernicke’s aphasia (0.84 ± 0.19), no aphasia (0.67 ± 0.19), conduction aphasia (0.50 ± 0.40) and global aphasia (0.33 ± 0) (F = 8.96, P = 6.48 × 10⁻⁸).

Participants with Broca’s aphasia were more likely to have an SE/PD ratio > 1 (13/14 = 92.8%). In contrast, few participants with other types of aphasia had an SE/PD ratio >1: 4/14 with anomic aphasia (28.6%), 1/3 with Wernicke’s aphasia (33.3%), and 1/13 with no aphasia (7.7%) (χ²P = 6.24 × 10⁻⁵). No participants with conduction or global aphasia had a ratio >1. Nonetheless, it is important to highlight that not all individuals with Broca’s aphasia had higher fluency with speech entrainment versus picture description, and some individuals with anomic or even Wernicke’s aphasia had higher fluency with speech entrainment versus picture description, suggesting that type of aphasia alone is not sufficient to fully predict whether assisted fluency is superior to spontaneous fluency.

A comprehensive table demonstrating cross correlations between all regional lesions and behavioural values is presented in Supplementary Table 3.

The raw data related to speech entrainment, picture description and the SE/PD ratios are shown in Fig. 2. As seen in Fig. 2, left, there are several individuals with very poor picture description who can achieve near normal speech entrainment or show large improvements with speech entrainment. As speech entrainment has a ceiling, it is not possible for someone with good picture description to have further improvement with speech entrainment (Fig. 2, right). The variability in SE/PD response beyond a critical threshold is not taken into account in the classification analyses below but are important in the regression analyses. Note that if a participant is able to produce a speech entrainment script perfectly (complete word-by-word entrainment of the script), there is a ceiling in the number of different words per minute that can be produced, which explains why SE/PD was low for non-aphasic participants.

Figure 2.

Behavioural data. Left: The relationship between different words per second for speech entrainment (in yellow) versus picture description (in blue). The data are ordered from left to right based on picture description to illustrate the fact that some individuals with very low picture description can still achieve good entrained fluency. Right: The relationship between aphasia types, the ratio between SE/PD (y-axis) and WAB fluency (x-axis). Note the benefit of speech entrainment for most individuals with but not exclusively Broca’s aphasia.

Neuroanatomical predictors

Controlling for lesion size, we observed significant correlation between damage to specific regions of interest and SE/PD (Table 1). With correction for multiple statistical comparisons using Bonferroni (0.05/24 correlations), there was significant association between damage to the precentral gyrus, middle frontal gyrus and precentral gyrus and higher SE/PD ratio. There were no significant correlations between regions with lower level of damage and higher SE/PD ratio. Nonetheless, there were several trends related to structural preservation and higher SE/PD ratio, particularly related to temporal lobe regions. These became apparent when lesion ensembles were assessed using neural networks, instead of the univariate analyses assessing regions in isolation. There was a strong relationship between lesion volume alone and SE/PD ratio (R = 0.52, P = 0.0002, larger lesion = stronger effect of speech entrainment).

Table 1.

Correlation coefficients between regional damage and SE/PD controlling for lesion size

Region	R	P
Superior temporal gyrus left	−0.3151386	0.030958
Pole of superior temporal gyrus left	−0.3772972	0.008939
Middle temporal gyrus left	−0.4206599	0.003238
Posterior superior temporal gyrus left	−0.2261015	0.126463
Posterior middle temporal gyrus left	−0.4165543	0.003585
Middle frontal gyrus (posterior segment) left	0.53311636	0.000114
Inferior frontal gyrus pars opercularis left	0.35720678	0.013706
Inferior frontal gyrus pars triangularis left	0.10231807	0.493743
Angular gyrus left	0.05102143	0.733413
Fusiform gyrus left	−0.2456714	0.096012
Pole of middle temporal gyrus left	−0.4030584	0.004969
Inferior temporal gyrus left	−0.3486818	0.016306
Superior frontal gyrus (posterior segment) left	0.39096968	0.006582
Middle frontal gyrus (dorsal prefrontal cortex) left	0.15598221	0.295104
Inferior frontal gyrus pars orbitralis left	−0.0270108	0.856978
Precentral gyrus left	0.73157201	0.000000
Superior parietal gyrus left	0.37288263	0.009841
Supramarginal gyrus left	0.47679613	0.000703
Posterior cingulate gyrus left	0.13385968	0.369703
Insular left	−0.0403475	0.787722
Inferior fronto-occipital fasciculus left	−0.0091241	0.951464
Sagittal stratum (include inferior longitidinal fasciculus and inferior fronto-occipital fasciculus) left	−0.2099301	0.156690
Superior longitudinal fasciculus left	0.20996758	0.156614
Uncinate fasciculus left	−0.1314296	0.378533

The P-values are not corrected for multiple comparisons, but those below the Bonferroni threshold (0.05/24) are highlighted in bold.

The grid search for the optimal machine learning neural network architecture revealed that three neurons with the SE/PD threshold set at 1.2 yielded the best classification accuracy (77 ± 16%), which represented a greater accuracy than 96% of the randomly distributed data (Supplementary Fig. 1). This classification threshold divided the overall group of 48 participants into a group of 33 participants with lower SE/PD ratio and 15 participants with higher SE/PD ratio. A classifier choosing all subjects in the low SE/PD group would have yielded 69% accuracy (33/48). It is possible that our classifier achieved best results at the 1.2 SE/PD threshold given the characteristics of our sample and the relatively small sample size (n = 48). Other samples could have yielded different thresholds.

Over 250 simulations, this classifier yielded the following results: (i) average percentage of samples misclassified: 23 ± 16%; (ii) average accuracy: 77 ± 16; (iii) average balanced accuracy: 71.19 ± 19.31%; (iv) simulated random average balanced accuracy: 48.63 ± 12.03%; (v) average sensitivity (towards SE/PD above threshold): 54.76 ± 38.86%; (vi) average specificity (towards SE/PD above threshold): 87.69 ± 18.01%; (vii) average positive predictive value (towards SE/PD above threshold): 69.74 ± 34.42%; and (viii) average negative predictive value (towards SE/PD below threshold): 81.62 ± 17.64%. The average confusion matrix (test sample size = 7) is presented in Supplementary Table 4.

The regions that contributed to the classification were: for higher SE/PD threshold, damage to the following regions had a significant input: middle frontal gyrus (96.4%), inferior frontal gyrus opercularis (99.6%), inferior frontal gyrus triangularis (98.4%), precentral gyrus (99.6%), supramarginal gyrus (96.8%), insula (96%), and superior longitudinal fasciculus (99.6%).

For lower SE/PD ratios, damage to the following regions were significant predictors: posterior middle temporal gyrus (96.8%), inferior fronto-occipital fasciculus (97.2%), and uncinate fasciculus (99.2%). These are summarized in Fig. 3.

Figure 3.

Neuroanatomy associated with spontaneous versus entrained speech. The brain regions are coloured in accordance with their weights in the predictive learning model, and the colour code range from 95 to 100, which represents the frequency with which the regions’ weight surpassed the random model. In red-yellow are regions whose lesions were associated with higher assisted versus spontaneous speech (higher SE/PD ratios = higher lesion load to middle frontal gyrus, inferior frontal gyrus opercularis, inferior frontal gyrus triangularis, precentral gyrus, supramarginal gyrus, insula, and superior longitudinal fasciculus), and regions in blue-green are those associated with worse assisted fluency (higher SE/PD rations = lower lesion load in posterior middle temporal gyrus, inferior fronto-occipital fasciculus, and uncinate fasciculus).

There was a strong relationship between lesion volume and the combined lesion to regions whose damaged were associated with better SE/PD ratios [middle frontal gyrus (96.4%), inferior frontal gyrus opercularis (99.6%), inferior frontal gyrus triangularis (98.4%), precentral gyrus (99.6%), supramarginal gyrus (96.8%), insula (96%), and superior longitudinal fasciculus (99.6%)]: R = 0.63, P < 0.0001.

There was also a significant relationship between total lesion volume and the combined lesion to the regions whose preservation was associated with better SE/PD ratios [posterior middle temporal gyrus (96.8%), inferior fronto-occipital fasciculus (97.2%), and uncinate fasciculus (99.2%)]: R = 0.34, P = 0.02.

Controlling for lesion size, the relationship between lesion to these areas and SE/PD remained statistically significant. Namely, partial correlation controlling for lesion size for areas whose combined damage was associated with better SE/PD scores: R = 0.46, P = 0.0011; and areas whose combined preservation was associated with better SE/PD scores: R = −0.4576, P = 0.001.

Predicting SE/PD ratio as a continuous variable (regression)

Using a shallow neural network to predict SE/PD ratios based on the neuroanatomical data, we observed an average correlation coefficient of R = 0.92 ± 16 (explaining 85% of variance) across 1000 runs (root mean square error was 0.06 ± 0.07 on min-max normalized data). The variance explained by this model was significantly higher compared with random model (Fig. 4).

Figure 4.

Predicting SE/PD ratio as a continuous variable. Left: The distributions of the correlation coefficient using the neuroanatomical predictors (blue = real data, red = random model). Middle and right: Plots show min-max normalized observed (y-axis) versus predicted values (x-axis) real and normalized data (blue and red). All runs were collapsed for visualization purposes.

Behavioural predictors

Simple partial correlations demonstrated a significant relationship between SE/PD ratio and WAB spontaneous speech (R = −0.59, P < 0.0001), WAB fluency (R = −0.62, P < 0.0001) and WAB information content (R = −0.62, P = 0.0002) (Bonferroni corrected: 0.05/8 correlations). Age (R = −0.11), time since the stroke (R = 0.13), WAB AQ (R = −0.38), WAB comprehension (R = −0.12) and WAB repetition (R = −0.24) were not significantly associated with SE/PD ratio using this univariate approach.

A grid search demonstrated that the hidden layer with four nodes and 1.2 SE/PD threshold yielded the highest classification accuracy (76 ± 18%), 99.6% higher than a random model (Supplementary Fig. 2).

Based on this optimal configuration, the predictors significantly associated with classification accuracy were as follows: for higher SE/PD threshold, higher scores in WAB comprehension (97.2%), lower WAB spontaneous speech (99.6%), or classification as Broca’s aphasia (99.6%) or anomic aphasia (99%). For lower SE/PD ratio, higher WAB information content (98%), or classification as no aphasia (96%) or Wernicke’s aphasia (98.8%).

Similar to the neural network based on lesions, these results provide a better overview of the combination of predictors towards speech entrainment and picture description rather their effect in isolation.

Discussion

In this study, we investigated the neuroanatomical structures associated with chronic post-stroke aphasia and better speech fluency during speech entrainment compared with spontaneous speech.

We observed that damage to supra-sylvian structures was associated with impaired speech fluency, but preservation of the middle temporal gyrus and anterior temporal white matter was associated with better speech entrainment compared with spontaneous speech. These findings were mirrored by aphasia classification categories: individuals with Broca’s or anomic aphasia, but not with conduction or Wernicke’s aphasia, were more likely have better fluency with assisted (entrained) speech compared to their spontaneous speech.

Using machine learning, we observed that the neuroanatomical pattern of damage could statistically significantly classify speech entrainment and picture description response and predict the continuous SE/PD ratio. These findings have mechanistic and translational implications, which are discussed below. The use of shallow neural networks as a neuroimaging method is also discussed.

Mechanistic implications

Speech entrainment likely depends on the sensorimotor transformation of speech, i.e. the translation of audiovisual sensory signals into motor plans. However, the underlying mechanisms associated the sensorimotor transformation of speech are not well understood.

In this study, we observed that speech entrainment was associated with preservation of the posterior middle temporal gyrus, inferior fronto-occipital fasciculus and uncinate fasciculus. This may be related to the fact that sensorimotor integration depends on lower levels of assimilation of sensory input. More specifically, if speech sounds are not processed into language specific elements, such as phonemes, words or sentences by ventral stream structures, it is unlikely that sensorimotor transformation can subsequently occur. Interestingly, in addition to lesions in the frontal gyrus, we also observed that lesions in the angular gyrus were associated with a higher likelihood of benefiting from speech entrainment, suggesting preservation of temporal structures may be the crucial determinant for achieving improved fluency with speech entrainment. Speech entrainment may use a combination of audio and visual input to recruit lower hierarchical levels of sensorimotor transformation and motor planning (Fridriksson et al., 2015a; Venezia et al., 2016).

Our observations corroborate the findings from Venezia et al., who used functional MRI in health individuals to the study visuomotor circuit for speech production. In their study, they observed that the left posterior middle temporal gyrus was recruited by the perception and covert rehearsing of nonsense syllable sequences cued by audiovisual input (Venezia et al., 2016), supporting its role in the somatosensory transformation of speech.

The lateral temporal cortex may be associated with lower levels of sentence comprehension, which may include syntactic processing (Tyler et al., 2011; Schell et al., 2017). Moreover, the lateral temporal cortex is strongly linked to frontal structures that are crucial for speech production (Fridriksson et al., 2015b), including in the context of verb processing (Thompson et al., 2007, 2010, 2013; Garbin et al., 2012). Therefore, the importance of the lateral temporal cortex and its white matter pathways for speech entrainment may be due to multiple factors, including early audio-visual integration, word and sentence comprehension (Venezia et al., 2016; Bonilha et al., 2017; Maffei et al., 2017; Fridriksson et al., 2018). Moreover, our results suggest that preservation of white matter temporal pathways may support ventral dorsal stream integration. This observation is in accordance with previous studies suggesting that speech production is supported by distributed processes and white matter integrity is crucial for language comprehension (Catani and Mesulam, 2008; Bonilha et al., 2014, 2017; Forkel et al., 2014; Del Gaizo et al., 2017; Fridriksson et al., 2018). For instance, Forkel et al. (2014) show a strong influence of white matter on the recovery. Our group and Xing et al. demonstrated the importance of anterior temporal lobe pathways in word-level comprehension (Bonilha et al., 2017; Xing et al., 2017). Xing et al. also demonstrated the importance of distributed temporal pathways in sentence-level comprehension (Xing et al., 2017) and naming processing (Xing et al., 2018). The importance of left inferior frontal-occipital fasciculus and inferior longitudinal fasciculus in speech comprehension was demonstrated by Ivanova et al. (2016), and Saur et al. (2008) and Kummerer et al. (2013) also observed that comprehension deficits were associated with more ventral-anterior in the temporoprefrontal lesions. These are a few important examples of evidence from the literature regarding white matter pathways supporting speech comprehension. The results from this study corroborate these previous studies and demonstrate that the integrity of temporal lobe white matter in areas previously associated with speech comprehension is also important to support speech entrainment.

The other important mechanistic observation from this study is the fact that lesions to supra-sylvian structures are associated with a higher probability of benefiting from speech entrainment. This finding is in accordance with earlier reports from our group (Fridriksson et al., 2015a; Feenaughty et al., 2017) and provides the confirmation that, even for subjects with large anterior dorsal stream lesions, assisted fluency can be obtained, provided that speech comprehension and temporal regions remain partly intact. As such, speech entrainment provides an external aid that compensates or replaces the phonological mapping that is likely to occur in dorsal structures.

The specific mechanisms through which speech entrainment repairs phonological mapping were not tested in this study and remain a topic for future investigation. For instance, it is possible that speech entrainment mediated fluent speech is only a form of speech mimicking, when the subject is not really able to perform any phonological mapping if the assistance is removed. This could be elucidated by studies evaluating the results of treatment regimens incorporating speech entrainment. This present study did not evaluate treatment. It only focused on whether assisted fluency could be obtained with speech entrainment and its results may be used to better inform the mechanisms underlying speech entrainment. If therapy involving speech entrainment leads to post-treatment improvement, then it has a long-lasting effect on the reorganization of phonological mapping and it may serve as a viable clinical approach for non-fluent aphasia.

We also did not explore how the severity of picture description could be a complex determinant of speech entrainment. For instance, is there a maximum of speech entrainment that one can achieve based on picture description? This could be best explored by varying the duration or type of stimuli during speech entrainment in dedicated studies.

Another limitation of this study is the use of region of interest-based lesion-symptom mapping. While this approach improves statistical power compared with voxel-based analyses by reducing the number of multiple comparisons, it provides a coarser evaluation of the crucial neuroanatomy related to behavioural measures. As such, the fine-grained statistical parcellation within the regions of interest is not possible.

Finally, it is important to highlight that there was a strong correlation between lesion loads particularly between regions within the same lobe. This can be challenging because of collinearity. While collinearity may not affect the accuracy of the model, it can cause instability in the identification of independent predictors. Nonetheless, lesions in temporal lobe had lower correlation coefficients with lesions in the frontal lobe. The reduced collinearity between these regions suggests their independent contribution and corroborate the importance of preservation of the lateral temporal cortex for entrainment benefit. However, the results reported here were obtained from a relatively small sample size and they require further validation with larger and multisite data.

Implications for clinical translation

The basic premise behind most rehabilitation approaches, regardless of the targeted function or modality, is that repeated practice of a specific behaviour within the therapy session leads to an increased likelihood that this behaviour improves and can be executed outside the rehabilitation setting. This concept is related to the principle of Hebbian learning, which is the notion that synaptic strength and functional neuronal connections can be reinforced due to repeated stimulation (Hebb, 1949). New experiences lead to molecular and cellular events that alter synaptic efficacy and lead to reorganization of neuronal circuits (Lowel and Singer, 1992). This principle should be considered in the context of aphasia therapy as conventional therapies typically induce numerous speech errors (Kelly et al., 2010; Brady et al., 2012). In fact, treatment-induced errors can often exceed correct responses (Boyle, 2015). Based on the Hebbian learning principles of neuroplasticity, it is possible that repeated errors could, in fact, lead to maladaptive changes and contribute to the persistence of post-therapy errors and non-fluent speech. During speech entrainment, subjects with aphasia can practice fluent speech with few errors, thus potentially addressing the issue of reinforced errors. Speech entrainment may also overcome the related issue of ‘learned non-use’, which implies that stroke survivors tend to avoid using affected functions because doing so is inefficient (Taub et al., 2006). In the context of motor impairments, learned non-use is manifested by the preferential use of the spared limb, with little or no use of the paretic limb (Mark and Taub, 2004). Although learned non-use has not been frequently addressed in the aphasia literature, it is clear that non-fluent subjects tend to withdraw from communication situations.

Speech entrainment may provide non-fluent stroke subjects with the opportunity to practice fluent speech and reverse some of the effects of learned non-use in speech. Speech entrainment is unique in this respect because it overcomes a difficult-to-treat deficit and serves as a potentially useful conduit to re-train fluency. If speech entrainment were to be included in therapeutic approaches, it could lead to new, improved levels of speech therapy of non-fluent aphasia. The goal of this study was to elucidate the mechanisms related to speech entrainment and not to evaluate its therapeutic benefits. This study did not test speech entrainment as a means of therapy. It did not evaluate speech entrainment as a therapy regimen and whether speech entrainment is associated with medium to long term improvements. However, we believe that our study is an important first step to understand this phenomenon, which may have a real therapeutic translational potential.

Overcoming non-fluency is an important step to re-initiate a lost function and may be an effective means of repairing fluency if preserved gains after therapy are observed. This is an important topic for future early and late clinical trials, which may be guided by mechanistic observations from this study.

Machine learning applied to lesion-symptom mapping

This study used shallow neural networks applied to conventional lesion-symptom mapping for two important reasons. First, neural networks permit the evaluation of how predictors (in this case, brain regions) combine together in specific nodes in the deep layer that influence a predictor. More specifically, they permit the assessment of how an ensemble of structures (and not structures in isolation) can be important for classification. Second, it permits the assessment of out-of-sample prediction, i.e. testing whether the neuroanatomical model can remain accurate for classification or regression when independent testing data are assessed.

By repeating the training, validation and testing procedures multiple times, we demonstrated the stability of the classification and regression accuracies obtained from the neuroanatomical model. We demonstrated how a group of dorsal stream structures was crucially associated with a higher probability of improved fluency with speech entrainment, provided that separate neuroanatomical regions (the lateral temporal cortex and anterior temporal lobe white matter) were intact. The same model also yielded exquisite accuracy in the regression model to predict SE/PD as a continuous variable.

Finally, we chose a linear shallow neural network because its interpretation is more intuitive, and also because it is possible to readily determine feature importance of individual regions in the predictor variables. This model can be compared with a null model for statistical significance (probability of type I error). As demonstrated in Fig. 3, lesions to the precentral gyrus had a higher influence in the model, but other surrounding structures had a statistically significant contribution, and their combination, pooled with the influence exerted by lesions to the temporal region, yielded high predictive accuracy.

Conclusions

Using a machine learning neural network approach integrated into lesion-symptom mapping, we observed that individuals with chronic post-stroke aphasia and reduced spontaneous speech fluency can achieve improved fluency assisted by speech entrainment if their anatomical pattern of damage included supra-sylvian areas, but preserved lateral temporal cortex and temporal white matter. Individuals with this pattern of dorsal stream damage and ventral stream preservation were more likely to have a more fluent speech during entrainment compared with spontaneous speech.

These observations provide insight into the mechanisms supporting assisted recovery of speech fluency and may be used to inform clinical trials to assess the efficacy of speech entrainment as a therapeutic approach for aphasia.

Funding

This work was supported by NIDCD R01DC014021 (L.B.), NIDCD P50 DC014664 (J.F., G.H., A.H., C.R.), NIDCD R01 DC05375 (A.H.), NIDCD T32 DC014435 (A.B.).

Competing interests

The authors report no competing interests.

Glossary

Abbreviations

SD/PD =

speech entrainment/picture description

WAB =

Western Aphasia Battery