Revealing the dual streams of speech processing

Significance

Relatively recently, the concept of dual route neural architecture, where dorsal and ventral brain regions process information synergistically, has been applied to study of speech processing. Although a large body of work has investigated these streams in relation to human speech processing, there is little consensus regarding specific cortical regions implicated. Relying on extensive behavioral and neuroimaging data from a large sample of stroke survivors, we used a data-driven approach to localize regions crucial for motor–phonological and lexical–semantic aspects of speech processing. Results revealed distinct anatomical boundaries between a dorsal frontoparietal stream supporting a form-to-articulation pathway and a ventral temporal–frontal stream supporting a form-to-meaning pathway. This study shows clear division between two processing routes underlying human speech.

Abstract

Several dual route models of human speech processing have been proposed suggesting a large-scale anatomical division between cortical regions that support motor–phonological aspects vs. lexical–semantic aspects of speech processing. However, to date, there is no complete agreement on what areas subserve each route or the nature of interactions across these routes that enables human speech processing. Relying on an extensive behavioral and neuroimaging assessment of a large sample of stroke survivors, we used a data-driven approach using principal components analysis of lesion-symptom mapping to identify brain regions crucial for performance on clusters of behavioral tasks without a priori separation into task types. Distinct anatomical boundaries were revealed between a dorsal frontoparietal stream and a ventral temporal–frontal stream associated with separate components. Collapsing over the tasks primarily supported by these streams, we characterize the dorsal stream as a form-to-articulation pathway and the ventral stream as a form-to-meaning pathway. This characterization of the division in the data reflects both the overlap between tasks supported by the two streams as well as the observation that there is a bias for phonological production tasks supported by the dorsal stream and lexical–semantic comprehension tasks supported by the ventral stream. As such, our findings show a division between two processing routes that underlie human speech processing and provide an empirical foundation for studying potential computational differences that distinguish between the two routes.

Understanding how and where in the brain speech processing occurs has been the focus of concerted scientific endeavor for over one and a half centuries. The most influential model of the neuroanatomical basis of speech processing was proposed by Wernicke (1) and later refined by Lichtheim (2)—the Wernicke–Lichtheim (W-L) model. The W-L model includes two separate routes from a posterior auditory comprehension center to an anterior motor speech production center: a direct route that enables speech repetition and an indirect route via ideation that mediates comprehension and propositional speech. More recently, dual route processing has been recognized as a central principle in the functional organization of the brain. Ungerleider and Mishkin (3) proposed that visual perception in primates is supported by a ventral “what” stream (involving an occipital–temporal lobe route) and a dorsal “where” stream [or later, a “how” stream mediated by an occipital–parietal route (4)]. Similarly, in the auditory domain (5), Rauschecker and Tian (6) proposed a “dual stream” model to account for the identification of what was being heard and from where the sound originated (5, 6). This model, mostly derived from nonhuman primate data, distinguishes between an anterior/ventral route (“what” stream) involving connections from the left posterior superior temporal gyrus (STG) to the left inferior frontal gyrus (LIFG), including pars opercularis and pars triangularis, and a posterior/dorsal route (“where” stream) that extends from the posterior STG to the intraparietal lobule and the premotor cortex. Although this model makes strong predictions about sound-to-object identification for the purpose of comprehension and spatial processing of sound, it is inherently less specific with regard to the role of sensory feedback in speech production, a central feature that enables fluent articulation of speech (7). A later update to this model (5) posits that the posterior/dorsal stream plays an important role in feedback control during speech production.

A neuroanatomical model of speech processing proposed by Hickok and Poeppel (8) also emphasizes similar dual route processing. Their dual stream model includes a bilateral ventral stream extending from the posterior middle and inferior temporal gyrus to the anterior middle temporal gyrus (MTG) that supports auditory comprehension. It makes somewhat stronger claims regarding speech production being supported by a sensorimotor dorsal stream that is left lateralized and involves an area in the STG at the notch of the Sylvian fissure at the boundary of the parietal and temporal lobes (area Spt), along with the posterior inferior frontal gyrus (including pars opercularis and pars triangularis), and the lateral premotor cortex. Specifically, the dorsal stream is suggested to process auditory speech-to-articulation transformations, including feedback necessary for predicting the spoken output and online error detection and modification of speech output (more detail is in ref. 9).

Dorsal route prominence for motor speech production functions is also supported in other work (10–15). The influence of dual stream models of speech processing in the literature is perhaps best shown by the large number of published papers that reference Rauschecker and Tian (6) (1,018 citations), Rauschecker and Scott (5) (781 citations), or Hickok and Poeppel (8) (2,072 citations). Although it has become common practice to explain results of neuroimaging studies of speech processing in the context of dual stream models (16–21), the specific location and extent of the dorsal and ventral streams involved in speech processing, each of which is a theoretical construct based on either animal data (5) or the combination of lesion and functional neuroimaging data in humans (8), remain elusive.

When it comes to mapping the speech areas of the brain, fMRI has been the most popular method during the past two decades (22–24). Although fMRI has yielded insights into the cortical mapping of speech, a shortcoming is that it cannot pinpoint regions that are crucial for speech—only areas that are correlated with a given speech processing task (25). It could be argued that careful metaanalysis studies that incorporate fMRI data may reveal converging evidence regarding the neural architecture that supports speech (26–28), but metaanalysis studies ultimately suffer the same shortcomings as basic fMRI studies: the inputs reflect task-associated cortical “activation” rather than areas of the cortex that need to be intact for successful speech processing.

Unlike fMRI, lesion studies can reveal cortical areas that are crucial for performing a given task, providing an advantage over other common neuroimaging methods to understand the neural basis of speech. One approach to assessing the spatial extent of dual streams is to relate specific measures of speech production or speech comprehension to lesion location in patients (29). However, a limitation of such an approach is that it assumes that the complex aspects of speech production and comprehension can be appreciated with single measures. For example, it is difficult to see how a task, such as speech repetition, comprehensively reflects all aspects of processing necessary to give rise to fluent speech production. Similarly, a single test of auditory comprehension (e.g., single-word comprehension or matching spoken sentences to pictures) probably does not tap into all of the processes that support speech comprehension in natural communication settings. At the same time, such tasks do rely on additional processes that are not or only indirectly related to speech production or perception. It is challenging, therefore, to isolate the processes that underlie production and perception of speech. Accordingly, mapping single behavioral factors onto the brain using traditional lesion-symptom mapping methods may fall short in revealing the processing streams that underlie human speech processing.

Traditional lesion studies also have other inherent limitations. Most group lesion studies rely on data from stroke patients, often presenting with similar general deficits (such as aphasia), which limit what areas of the brain can be studied. For example, the precuneus, a region that has been implicated in language processing (30), is rarely directly affected by stroke, making it difficult to infer the role of this area in human communication. In acute stroke, concerns are that behavioral testing could potentially reflect initial cortical diaschisis and that some acute stroke patients may exhibit a lowered cognitive status that reflects the overall trauma of the stroke rather than direct damage to localized regions of the brain. Finally, studies of chronic stroke may reflect potential cortical reorganization, because many patients experience early recovery. It could be argued that studies that include chronic stroke data reveal areas that are associated with both a given behavioral impairment and the successful recovery of that impairment. Of course, such information can be of interest and clinical importance. In regard to this study, for example, it could be argued that impaired speech processing reflects not only the direct influence of damage on performance but also, the inability of the brain to recover the necessary processes needed for task execution.

This study includes a dataset of stroke survivors tested on a wide range of behavioral tasks to identify the cortical regions implicated in the dorsal and ventral streams and the tasks that are typically supported by these streams. Our simple assertion is that, if there are, in fact, two streams involved with different aspects of speech and language processing, one should be able to implicitly resolve them by observing the pattern of injury associated with impairments on a wide range of tasks. If some tasks rely more on one stream than another, this approach should reveal the two distinct streams. Unlike other related studies (20), our approach is totally unconstrained by prior models: in theory, we could find more than two regions involved with a task. However, based on prior theoretical work, we expected that this approach would identify a dorsal vs. ventral component that would crucially elucidate the functional–anatomical aspects of these streams. Importantly, also, this study does not associate lesions with individual tasks that have been marked as “production” or “perception” tasks a priori. Instead, we take a data-driven approach, whereby the clustering of tasks that are predictive of consistent lesion patterns is interpreted based on the functional nature of the tasks. This study is inherently a lesion-symptom mapping study and as such, challenged by the same limitations of vascularly constrained lesion sites, potential diaschisis, and cortical reorganization as described above. However, we use an approach to the relation between behavioral and lesion data based on a relatively large sample of stroke survivors, which includes participants without language or speech impairments, with a wide range of (left hemisphere) damage that amply covers all of the regions typically included in dual streams models of speech processing.

Other studies have used a principal components analysis (PCA) step as part of the analysis relating localized brain damage to behavioral impairment (19, 31, 32). However, those studies differed from this approach in that each carried out PCA on the behavioral data and then, included the components as dependent factors in a voxel-based lesion-symptom mapping (VLSM) analysis. Posthoc, then, each PCA component is interpreted based on the behavioral tests that load onto that component. In contrast, our study initially carried out separate univariate VLSM analyses for each behavioral factor and then, applied PCA where the inputs consisted of VLSM maps depicting the relationship between localized brain damage and speech or language impairment. PCA computed on behavioral scores will give us the common sources of variance in the behavioral scores, regardless of the underlying neurobiology. For example, if two hypothetical behavioral tests are highly correlated but recruit distinct and nonoverlapping cortical areas, they will end up having very similar loadings in principal components. Running VLSM on these components would, therefore, obscure the difference in their underlying cortical recruitment. In this study, the primary interest was on the similarity/difference in the VLSM maps computed for our set of scores rather the similarity/difference of the scores themselves; accordingly, VLSM was carried out first and then followed by PCA rather than the other way around.

This study relied on lesion data and neuropsychological testing in 138 stroke survivors with left hemisphere damage to understand whether localized brain damage that predicts different patterns of speech impairment loads onto ventral and dorsal streams. The number of participants who completed each neuropsychological test ranged from 38 to 138. Consistent with previous studies of lesion–behavior relationships, we hypothesized that overall severity on speech and language tests would primarily be explained by lesion size. However, if speech processing, in fact, relies on a dual processing route, the remaining variance in our lesion–behavior dataset that is not related to lesion size should be explained by a clear distinction between brain damage that predicts different aspects of speech and language processing. We expected to see a clear distinction between a dorsal and a ventral processing stream that could be explained in relation to the previously hypothesized dual stream models of speech processing (5, 8). To explore whether our results reflected the vascular distribution of the middle cerebral artery rather than damage that predicts speech and language impairment, an additional PCA was carried out, where the inputs included the lesion maps from individual patients. Results from this latter analysis were compared with the primary PCA that included VLSM maps as inputs.

Results

PCA.

After the lesion data had been preprocessed, two primary data analysis steps were carried out. An initial step used multiple univariate VLSM analyses to relate lesion location to performance on each neuropsychological speech and language test independently. Seventy-one subscores representing performance across the different speech and language tests (Table S1) were used as dependent factors in the VLSM analyses. Subsequently, PCA, a method used to extract sets of uncorrelated sources of variance (33), was used to capture the most important common sources of variance in the unthresholded VLSM statistical maps.

Table S1.

List of speech and language batteries used for participant testing

Test and subtests included in PCA	No. of participants tested
WAB-Revised (41)
Speech Fluency	n = 138 for All subtests, except reading and writing (n = 64)
Naming/Sentence Completion Information Content
Speech Repetition
Comprehension of Sequential Commands
Object Naming
Naming/Responsive Speech,
Writing
Naming/Word Fluency
Auditory Word Recognition
Reading
Comprehension of Yes/No Questions
Apraxia Battery for Adults, second edition (42)
Repeated Trials	n = 76
Utterance Time	n = 78
Diadochokinetic Rate	n = 87
Increasing Word Length 1	n = 79
Increasing Word Length 2	n = 66
AOS Inventory Rating	n = 84
Boston Naming Test (43)
Boston Naming Test Total Score	n = 95
NAVS (44)
SPPT	n = 52 (All subtests)
ASPT
NAVS-SCT
NAVS-nc-Total correct (total correct noncanonical productions)
NAVS-SPPT-nc vs. -co (comparison of errors between noncanonical  and canonical sentences during production)
NAVS-SPPT-nc-A-Correct (same as above but only for valid attempts)
NAVS-SCT-nc vs. -co (comparison of accuracy on sentence comprehension  between canonical and noncanonical sentence types)
NAVS-SCT-Noncanon (proportion of correct responses to SCT for  noncanonical sentence types only)
NAVS-ASPT-A-Op2 (accuracy of optional two-place verbs)
NAVS-ASPT-A-Op3 (same as above for three-place verbs)
ASRS (45)
ASRS 1p1-1p6: Production errors uniquely attributed to AOS	n = 43
ASRS 2p1-2p6: Production errors that can occur because of AOS or dysarthria
ASRS 3p1-3p2: Errors that can occur because of AOS or aphasia
ASRS 4p1: Errors that can occur because of AOS, aphasia, or dysarthria
PNT (46)
PNT Correct	n = 94
PNT Self-Corrections	n = 94
PNT Mixed Paraphasias	All other scores n = 93
PNT Neologisms
PNT Phonemic Paraphasias
PNT Articulation Errors
PNT Perseverations
PNT No Response
PNT Unrelated Response
PNT No Response
SCT (47)
SCT Type 1: Active sentences	Types 1–4: n = 52
SCT Type 2: Passive sentences	Type 5: n = 43
SCT Type 3: Truncated passives
SCT Type 4: Cleft sentences with subject gaps
SCT Type 5: Cleft sentences with object gaps
The Grandfather Passage Reading (48)
R-GF-TRW (total read words)	n = 56
R-GF-TDW (total different words read)
R-GF-Sem (semantic errors)
R-GF-Phon (phonological errors)
R-GF-Unrelated (word produced unrelated to passage)
Discourse
VPD-Phon (phonemic errors)	n = 54
VPD-Sem (semantic errors)
VPD-TRW (total real words)
SE (43, 55)
SE-TDW (total different words produced)	n = 59
SE-TRW (total real words produced)
In-house verbal naming test
University of South Carolina Naming Task	n = 93

Subscores from each test are used as dependent factors. Numbers of individuals are included. AOS, apraxia of speech; ASPT, Argument Structure Production Test; R-GF, Reading-Grandfather Passage; SCT, Sentence Comprehension Test; SPPT, Sentence Production Priming Test—canonical and noncanonical; SE, Speech Entrainment; VPD, Verbal Picture Description.

Component Loadings.

PCA revealed two primary components that explained ∼74% of the total variance (Component 1 = 47%; Component 2 = 27%). The amount of variance explained by each subsequent component did not exceed 5%. Approximately 90% of the remaining components explained less than 0.01% of the total variance. The factor loadings of Components 1 and 2 are represented in Table S2. Factor loadings for Component 1 were almost exclusively positive. In contrast, Component 2 was split into positive and negative loadings (Table S2). Negative loadings mostly involved motor and phonological aspects of speech production, whereas positive loadings primarily reflected speech comprehension and verbal naming. Particularly, the strongest negative loadings in Component 2 were associated with tests measuring apraxia of speech (a motor speech disorder) and articulation errors produced during picture naming as well as measures of total verbal output in connected speech and spoken phonological errors. Conversely, the strongest positive loadings were associated with impaired sentence comprehension as well as general auditory comprehension (e.g., answering simple yes/no questions after verbal directions). Positive loadings also included measures involving functions beyond speech comprehension, such as semantic and phonological errors during naming as well as the ability to verbally repeat sentences. Taken together, we suggest that negative loadings reflect form-to-articulation processing and that positive loadings reflect form-to-meaning processing. Form-to-articulation processing includes phonological–motor aspects of speech production. Form-to-meaning processing involves processes necessary for single word and sentence comprehension but could also be reversed (meaning-to-form processing) to support lexical–semantic aspects of speech production.

Table S2.

Factor loadings for Principal Components 1 and 2

Factors are color-coded based on the corresponding behavioral test battery (Table S1 shows additional detail). Colors are as follows: dark green, SCT; garnet, WAB; red, NAVS; grey, BNT; pink, PNT; light blue, reading; dark blue, VPD; light green, SE; orange, ABA-2; brown, UofSC naming test; black, ASRS. Factors are sorted by loadings in PC2 in descending order. Note the following tests: Sentence Comprehension Test (SCT) (47), WAB (41), NAVS (44), Boston Naming Test (BNT) (43), Verbal Picture Description (VPD; i.e., discourse sample) (Table S1), Apraxia Battery for Adults, second edition (ABA) (42), PNT (46), Reading-Grandfather Passage (R-GF) (48), Speech Entrainment (SE) (49), ASRS (45), and University of South Carolina in-house naming task (UofSC-Naming Task). In addition, note the following NAVS subtests: Argument Structure Production Test (ASPT), Sentence Production Priming Test (SPPT), SCT, noncanonical (nc), canonical (co), correct (C), optional two-place verbs (Op2), optional three-place verbs (Op3), and score was based on all valid attempts at production (A). Additional details are in Table S1. AOS, apraxia of speech; PC1, Principal Component 1; PC2, Principal Component 2.

Cortical Loadings.

Visual inspection of brain maps representing Components 1 and 2 suggested that Component 1 reflected overall lesion volume among the study participants, because the most negative voxel values were located in central structures, such as the insula, but gradually became more positive as the map extended to cortical areas farther away from center (Fig. 1, Left). This spatial distribution is commonly seen for lesion overlap maps among participants with unilateral stroke. To verify this observation, we correlated the statistical map representing Component 1 with a lesion overlay map from all 138 participants included in the study sample. The correlation between the two maps was r = −0.78. Correlation between the second component and the lesion overlap map was much smaller: r = 0.04. Because lesion size is typically a dominant factor in lesion-symptom mapping analyses (34), it was not surprising to find that Component 1 seems to reflect the extent of cortical damage. The statistical map for Component 2 showed that the spatial pattern that expressed the strongest source of variance was uncorrelated with the first component; this source largely represents the contrast between the measures of form to articulation (negative loadings) and form to meaning (positive loadings) (Fig. 1, Center). Negative loadings were primarily associated with damage involving the posterior inferior frontal gyrus, including pars opercularis and pars triangularis, lateral premotor cortex, pre- and postcentral gyri, and anterior aspects of the supramarginal gyrus (SMG) as well as the underlying white matter tracts (Tables S3 and S4). Positive loadings of Component 2 were associated almost exclusively with temporal lobe damage but also, involved areas in the parietal lobe, including posterior portions of the SMG and the angular gyrus as well as anteroventral inferior frontal lobe areas, such as the pars orbitalis. It is worth noting that the voxels with positive loadings composed more than 85% of the uncinate fasciculus, a fiber bundle that connects the anterior temporal lobe and the inferior aspects of the posterior frontal lobe, including pars orbitalis and pars triangularis (35).

Fig. 1.

Anatomical representations of the first two principal components from the PCA. Voxelwise values denote how strongly the corresponding component is expressed in a given voxel. The voxelwise weights are in the range of −0.003–0.003, which is evident from the color scale; however, their absolute magnitude is less important rather than their relative magnitude compared with other voxels. Singular value decomposition was used to compute principal components; therefore, the magnitude of voxelwise values cannot exceed one, and squared voxelwise values have to add up to one. Component 1 is represented in Left, Component 2 is represented in Center (Component 2a), and Component 2 modulated by a lesion component derived from lesion maps is represented in Right (Component 2b).

Table S3.

Brain regions and white matter tracts involved in the dorsal streams identified as parts of Component 2

Brain region	Percentage < 0	Mean < 0
Pars opercularis	95.2	0.0021
Superior corona radiata*	95.1	0.0030
Extreme capsule*	88.5	0.0016
Pars triangularis	74.8	0.0009
Putamen	68.4	0.0017
Posterior corona radiata*	67.1	0.0011
Anterior insula	60.1	0.0010
Posterior limb of internal capsule*	54.1	0.0022
Pars orbitalis	49.8	0.0003
Superior longitudinal fasciculus*	48.8	0.0015
Posterior insula	44.8	0.0010
Precentral gyrus	42.5	0.0024
Anterior corona radiata*	39.6	0.0014
Postcentral gyrus	39.3	0.0015
Retrolenticular part of internal capsule*	36.9	0.0012
Middle frontal gyrus	33.5	0.0022
SMG	33.0	0.0010
Inferior frontooccipital fasciculus*	32.8	0.0003
Anterior limb of internal capsule*	22.7	0.0020
Globus pallidus	20.4	0.0015

Positively loaded voxels were primarily associated with tasks involving lexical–semantic processing, naturally including many comprehension tasks. Negatively loaded voxels were associated with tasks involving phonological–motor processing, including many production tasks. We give the percentage of positive and negative voxels for the regions of interest constituting the dorsal and ventral streams and the mean of positive and (separately) negative voxels. For each stream, the table lists 15 regions of interest with the highest percentages of positive (ventral stream) and negative (dorsal stream) voxels.

^*White matter tracts.

Table S4.

Brain regions and white matter tracts involved in the ventral streams identified as parts of Component 2

Brain region	Percentage > 0	Mean > 0
Posterior MTG	89.2	0.0023
Posterior STG	87.6	0.0021
Uncinate fasciculus*	85.2	0.0010
MTG	76.8	0.0020
STG	75.2	0.0018
Superior temporal pole	64.7	0.0014
Angular gyrus	55.4	0.0021
Posterior insula	55.2	0.0008
Sagittal striatum	54.8	0.0016
Superior longitudinal fasciculus*	50.2	0.0017
Retrolenticular part of internal capsule*	44.0	0.0015
Inferior frontooccipital fasciculus*	43.8	0.0004
Anterior insula	39.2	0.0005
Middle temporal pole	37.2	0.0016
Posterior thalamic radiation*	36.2	0.0015
Pars orbitalis	33.7	0.0003
Inferior temporal gyrus	30.7	0.0018
SMG	30.7	0.0010
Posterior corona radiata*	29.4	0.0007
Middle occipital gyrus	25.7	0.0020

Positively loaded voxels were primarily associated with tasks involving lexical–semantic processing, naturally including many comprehension tasks. Negatively loaded voxels were associated with tasks involving phonological–motor processing, including many production tasks. We give the percentage of positive and negative voxels for the regions of interest constituting the dorsal and ventral streams and the mean of positive and (separately) negative voxels. For each stream, the table lists 15 regions of interest with the highest percentages of positive (ventral stream) and negative (dorsal stream) voxels.

^*White matter tracts.

To verify that the positive and negative loadings of Component 2 did not merely reflect the anterior and posterior distributions of the middle cerebral artery, the brain map representing Component 2 was multiplied by a component derived from a PCA that only included binary lesion maps from the patients (n = 138). This step highlighted brain areas strongly associated with the behavioral scores and did not highlight the vascular distribution of stroke damage (Fig. 1, Right). Fig. 2 shows an axial view of the brain areas represented in the positive and negative loadings of Component 2.

Fig. 2.

Axial view of the statistical maps representing Components 1 and 2. The values for each slice designate the slice order in the Z direction. Area Spt (red arrow) is designated by a black sphere (1 cm) centered over a standard coordinate [Montreal Neurological Institute (MNI) space] reported by Buchsbaum et al. (36).

Discussion

We suggest that negative and positive loadings of Component 2 revealed in the primary PCA reflect the spatial extent of the dorsal and ventral streams involved in processing form to articulation and form to meaning, respectively. Based mostly on data from nonhuman primates, the location of the dual streams proposed by Rauschecker and Scott (5) bears little resemblance to the dual streams revealed here. Rauschecker and Scott (5) proposed an anterior/ventral stream that involves a ventral route from the “auditory belt” in the posterior STG to the LIFG, including Brodmann’s areas 44 and 45 (typically considered to be synonymous with pars opercularis and pars triangularis). Although this ventral stream does involve the STG and extends anteriorly to the LIFG, it does not include pars opercularis or pars triangularis. Rather, this ventral stream is anchored by the posterior MTG and posterior STG (Table S4), the areas showing the greatest expression of Component 2, and extends to the posterior inferior parietal lobule (IPL) as well as via the uncinate fasciculus to the pars orbitalis in the inferior frontal gyrus. The posterodorsal stream by Rauschecker and Scott (5) includes the posterior superior temporal sulcus (STS) and the IPL, both of which are regions implicated in the ventral stream in this study. Also, the sharp divisions between these dorsal and ventral streams along the notch of the Sylvian fissure and extending to the intraparietal sulcus (Fig. 2) are in clear conflict with the posterodorsal stream proposed by Rauschecker and Scott (5), because we found no evidence for a continuous stream extending from the posterior STS to the IPL and onto the frontal speech areas.

These findings are in greater agreement with the dual stream model by Hickok and Poeppel (8), which includes a ventral stream mostly involving temporal lobe regions, including the posterior and anterior MTGs, and a dorsal stream extending from area Spt to the LIFG and lateral premotor cortex. Although Hickok and Poeppel (8) proposed that a ventral route from the anterior temporal lobe to the LIFG was important for speech processing, their model did not include the pars opercularis or pars triangularis as a part of the ventral stream. This model is consistent with our findings, where these areas fall into the dorsal stream. It is pertinent here to distinguish between anatomical connectivity and the effective “streams” that underlie particular types of processing. Whereas there is ample evidence that posterior inferior frontal gyrus connects to temporal cortex ventrally as well as dorsally, these data suggest that pars opercularis and pars triangularis functionally play a more important role as part of a dorsal stream that primarily supports form-to-articulation processing. In regard to the dorsal stream by Hickok and Poeppel (8), its anterior speech production regions are consistent with this dorsal stream. In fact, our cortical regions with the highest expression of negative Component 2 loadings (i.e., the dorsal stream) were the pars opercularis, a region classically considered as the posterior portion of Broca’s area, the premotor cortex, and the middle frontal gyrus (Table S3). Importantly, our analysis placed area Spt, a region proposed by Hickok and Poeppel (8) to be crucial for auditory-to-articulation transformations, at the boundary region between the two streams; the anatomical center of area Spt (36) was located in the ventral stream, but the standard coordinate for area Spt was only a few millimeters from the junction of the dorsal and ventral streams identified here (Fig. 2). As pointed out by Hickok et al. (37), the location of area Spt varies considerably across subjects, making it difficult to make a strong claim regarding the average location of this region.

Although the dual stream models proposed by Rauschecker and Scott (5) and Hickok and Poeppel (8) vary considerably in regard to the distribution of processing streams, it is pertinent to point out that the model by Rauschecker and Scott (5) was discussed primarily in the context of sublexical speech processing. In contrast, the model by Hickok and Poeppel (8) focuses more on lexical processing. The production tasks included here focused on both the sublexical and lexical levels as well as sentence and discourse production. However, the comprehension tasks focused more on sentence-level rather than sublexical- or lexical-level processing. Accordingly, the results shown here, especially with respect to the ventral stream, reflect robust influence of sentence comprehension as well as lexical–semantic processing. This result extends the interpretation of the functional roles of the dual streams in speech processing, which we characterize as form to articulation vs. form to meaning, rather than in terms of a categorical division between production and comprehension.

Modulation of the brain map representing Component 2 by a lesion component that most resembled the anterior and posterior distribution of the middle cerebral artery allowed us to verify that our findings were not driven by vascular boundaries. This finding further shows that Component 2 is robust and most likely reflects the neuroanatomical divisions of two large-scale streams supporting different aspects of speech processing. On a related note, it is imperative to mention that, although lesion studies that rely on stroke data are limited to studying cortical areas where stroke is likely to occur (e.g., middle cerebral artery distribution), this study included lesion coverage and adequate statistical power in each of the cortical areas typically included in dual stream models of speech processing in humans (5, 8).

Division of the dual streams revealed here largely reflects what has been observed in the literature on this issue: ventral regions support lexical–semantic processing, whereas dorsal regions support the phonological–motor aspects of speech production. Our primary data analysis—PCA of lesion maps—is well-suited to reveal independence of individual anatomical components as well as the loadings of individual factors on each component. However, this approach is not ideal for showing the functional division and interactions across the dual streams that support speech processing. In this context, it is appropriate to point out that these data do not dispute the notion that successful speech processing relies on functional interactions between the dual streams. For example, it has been shown amply that damage to posterior regions, such as the posterior STG, results in disordered speech production (38, 39). This fact is, indeed, shown in patients with Wernicke’s aphasia, a disorder caused by damage to the posterior regions of the temporal lobe (40), who suffer from severely impaired speech comprehension as well as impaired speech production marked by neologisms, phonemic paraphasias, and semantic paraphasias. Although these patients present with fluent speech production, the content of their speech is affected because of damage to the ventral stream. Accordingly, it was not surprising that the ventral stream revealed here included cortical damage associated with poor speech comprehension as well semantic and phonemic paraphasias. Another aphasia type, conduction aphasia, also shows the necessary interactions among the dual streams that enable normal speech processing. Patients with conduction aphasia typically present with damage to the temporal–parietal region, especially area Spt, and suffer from inability to repeat speech as well as frequent phonemic paraphasias, although their speech fluency and speech comprehension are relatively spared (36). The underlying impairment in these patients probably reflects inability to integrate processing across the dorsal and ventral streams, further highlighting the crucial interaction between two anatomically distinct processing routes. We argue that this interaction is also reflected in ventral stream loadings of production tasks, for which high scores crucially rely on lexical, syntactic, or semantic access rather than motor–speech planning and control, such as object naming on the Western Aphasia Battery (WAB) and argument structure production on the Northwestern Assessment of Verbs and Sentences (NAVS) (Table S1).

In conclusion, this data-driven study empirically defines the boundaries of the dorsal and ventral streams, regions that are crucial for successful speech processing. The ventral stream involves lateral temporal lobe structures extending to the IPL as well as the inferior frontal lobe via the uncinate fasciculus. The dorsal stream extends from anterior speech areas, including pars opercularis and premotor areas, to posterior regions in the SMG and straddles the edge of area Spt. These findings can be used to narrow the functional–anatomical distinction between areas involved in processing form to articulation and form to meaning in future studies and provide a clear anatomical definition of what constitutes the dual streams critical for human speech processing.

Experimental Procedures

Participants.

The data analyzed here were obtained from an archival database in the Aphasia Laboratory, University of South Carolina. Data from 165 persons with unilateral left hemisphere stroke were considered for analyses. Among these 165 persons, 138 (63 females) had both behavioral testing and MRI data. The extent of cortical damage across participants is shown in Fig. 3. At the time of study inclusion, all participants were at least 6 mo poststroke, with a mean age at the time of stroke of 57.31 y old (SD = 11.49) and a mean time poststroke of 36.3 mo (SD = 43.6). Each participant underwent behavioral testing as part of a study of aphasia. However, the extent of behavioral testing and participant involvement varied across different studies: some participants were enrolled in aphasia treatment studies, whereas others were only tested for the purpose of neuropsychological research. All participants signed an informed consent for study inclusion, and the research was approved by the University of South Carolina Institutional Review Board.

Fig. 3.

Left shows a lesion overlay map including lesions from all 138 participants in the study. The color scale range represents the areas of least lesion coverage (n = 1; black) to the areas with the greatest lesion overlap (n = 86; red) designated by the cross-hairs (Left). Right identifies brain areas where at least five participants had damage. The VLSM analyses excluded areas where fewer than five participants had damage.

Behavioral Testing.

Participants completed a neuropsychological test battery that involved measures of speech and language processing (Table S1). Some of the measures included here are typically viewed as clinical tests: for example, the WAB (41), the Apraxia Battery for Adults, second edition (42), and the Boston Naming Test (43). However, other tests, such as the NAVS (44), the Apraxia of Speech Rating Scale (ASRS) (45), and the Philadelphia Naming Test (PNT) (46), are primarily used in research studies. The PNT was scored for correct naming in addition to failed naming attempts that included phonological and semantic errors. In addition to published tests, this study relied on a sentence comprehension test involving various different sentence types (47) as well as analyses of spoken errors during reading (Grandfather Passage) (48), discourse (picture description), and speech entrainment (49). Finally, a large proportion of the participants completed an “in-house” test that involved verbal naming of pictures depicting 80 mid- to high-frequency nouns (50). The same criteria were used to score performance on this test (designated as University of South Carolina in Table S1) as the PNT.

MRI Data Acquisition.

MRI data were acquired using a Siemens 3T Trio System with a 12-channel head coil. All participants underwent scanning that included two MRI sequences: (i) T1-weighted imaging sequence using an MP-RAGE (magnetization-prepared rapid-gradient echo) [TFE (turbo field echo)] sequence with voxel size = 1 mm³, FOV (field of view) = 256 × 256 mm, 192 sagittal slices, 9° flip angle, TR (repetition time) = 2,250 ms, TI (inversion time) = 925 ms, TE (echo time) = 4.15 ms, GRAPPA (generalized autocalibrating partial parallel acquisition) = 2, and 80 reference lines; and (ii) T2-weighted MRI for the purpose of lesion demarcation with a 3D sampling perfection with application optimized contrasts by using different flip angle evolutions protocol with the following parameters: voxel size = 1 mm³, FOV = 256 × 256 mm, 160 sagittal slices, variable flip angle, TR = 3,200 ms, TE = 352 ms, and no slice acceleration. The same slice center and angulation were used as in the T1 sequence.

Preprocessing of Structural Images.

Images were converted to NIfTI format using dcm2niix (51). Stroke lesions were demarcated by a neurologist (L.B.) in MRIcron (52) on individual T2 MRIs (in native space). Preprocessing began with the coregistration of the T2 MRI to match the T1 MRIs, aligning the lesions to native T1 space. Images were warped to standard space using the enantiomorphic (53) segmentation–normalization (54) custom Matlab script (https://github.com/rordenlab/spmScripts/blob/master/nii_enat_norm.m) to warp the images to an age-appropriate template image included with the Clinical Toolbox. The normalization parameters were used to reslice the lesion into standard space using linear interpolation, with the resulting lesion maps stored at 1 × 1 × 1-mm resolution and binarized using a 50% threshold (because interpolation can lead to fractional probabilities, this step ensures that each voxel is categorically either lesioned or unlesioned without biasing overall lesion volume). All normalized images were visually inspected to verify the quality of preprocessing.

VLSM Analyses.

Seventy-one univariate VLSM analyses were completed to identify localized brain damage associated with speech processing impairments. VLSM results are typically reported as a standard brain map (statistical map or VLSM map) showing the statistical likelihood (as Z scores in image voxels) that a given brain location predicts performance on the behavioral test. A correlation matrix showing the shared variance among 71 unthresholded statistical maps is included in Fig. S1. For each analysis, voxelwise statistical significance was determined by voxel-based permutation thresholding (1,000 permutations) and multiple comparison correction (controlling for familywise error). Only voxels where at least five patients had damage were included in each analysis. To ensure that all of the dependent measures were unidirectional, scales where greater impairment is indicated by a high score and less impairment is indicated by a low score (e.g., on the ASRS) were reversed by subtracting each score from the maximum score for each specific factor. All of the VLSM routines used here are integrated in the NiiStat toolbox for Matlab (www.nitrc.org/projects/niistat). A total of 57 of 71 univariate VLSM analyses yielded statistically significant results, revealing localized brain damage associated with poor subscores on the speech or language tests. However, the VLSM maps that were used as inputs in the PCA were not thresholded based on their statistical significance.

Fig. S1.

A correlation matrix showing the shared variance among the statistical VLSM maps for 71 dependent factors. Brighter yellow indicates higher correlation; darker blue indicates lower correlations. ABA, Apraxia Battery for Adults; ASPT, Argument Structure Production Test; BNT, Boston Naming Test; R-GF, Reading-Grandfather Passage; SCT, Sentence Comprehension Test; SE, Speech Entrainment; SPPT, Sentence Production Priming Test; TDW, total different words; TRW, total read words; UofSC, University of South Carolina; VPD, Verbal Picture Description.

PCAs.

The PCAs of the data matrix (where each column is a VLSM map for a particular behavioral factor) produce a series of principal components, each component being a spatial map obtained from a particular linear combination of columns of the data matrix, where each column corresponds to a factor (i.e., a behavioral measure). Because the number of participants that completed each subtest varied, the VLSM maps were created using slightly different voxel masks and therefore, differed in terms of their spatial extent. To compensate for this difference, we identified a set of voxels that was included in the spatial masks in all VLSM maps, and the values of these common voxels formed the columns of the data matrix. The size of the data matrix was 259,393 (n is the number of common voxels) by 71 (P is the number of behavioral measures). Column data were centered by subtracting the column mean.

By definition, the first principal component is associated with the linear combination that maximizes the variance across voxels (it is often said that the first principal component “explains” the largest amount of variance in the data). The second principal component is orthogonal to the first, and it explains the largest amount of variance in the residual data after the first component has been regressed out. The total number of principal components is equal to P, the number of columns in the data matrix, and the amount of variance explained by each successive component does not exceed the amount of variance explained by the previous component. The most important orthogonal sources of variance are typically captured in the first few components, with the last few components containing noise (that is, the variance in the data that is particular to our data sample and is not likely to generalize).

We computed the principal components of our data matrix using singular value decomposition (we used the Matlab function svd). This decomposition represents the data matrix X as a product of three matrices: X = USV^T (here, superscript T denotes matrix transposition).

• The columns of matrix U are the principal components. These columns are mutually orthogonal (i.e., uncorrelated), and the sum of squared voxel values in each column is equal to one.

• The columns of V are the factor loadings: they specify the weights in the linear combination of factors (VLSM maps computed for given behavioral measures) expressed in the corresponding principal component. They are mutually uncorrelated, and the sum of squared loadings for a particular component is equal to one.

• S is a diagonal matrix; the diagonal entries are called singular values. They are ordered in descending order: s_1,1 ≥ s_2,2 ≥ … > s_P,P. They are associated with the percentage of variance in the data matrix explained by each principal component. For ith principal component (i.e., the ith column of U), the percentage of explained variance is given by

$$\frac{s_{i,i}^2}{{\displaystyle \sum _{j=1}^P{s}_{j,j}^2}}.$$

Because this study relied on lesion data from stroke patients, a PCA that only included the frank (binary) lesions (n = 138) was computed to explore the possibility that our results reflected neurovascular distributions (e.g., anterior vs. posterior distributions of the middle cerebral artery) rather than anatomical boundaries of speech processing areas. This step was important because the extent of the VLSM components (described above) is partially driven by how often a given voxel is damaged. Based on visual inspection, one component (Component 2: total variance explained = 15%) emerged from the PCA of lesions that somewhat resembled the anterior and posterior divisions of the middle cerebral artery (Fig. S2). Values in the brain map representing this lesion component were scaled so that 100% indicated a voxel that was maximally predicted by the lesion component, whereas 0% meant that it had no weighting. Then, the VLSM component in Fig. 1 (Component 2a) was multiplied by this scaled lesion component, and therefore, the resulting image highlighted regions that are strong in the VLSM component but weak in the lesion component (Component 2b; readers can access data by contacting J.F.).

Fig. S2.

Brain map representing a principal component derived from frank lesions that most resembled the anterior and posterior divisions of the middle cerebral artery. A scaled version of this map was multiplied by the brain map representing Component 2 in the middle panel of Fig. 1. The resulting brain map is represented in the right panel of Fig. 1.