Can neuroimaging help aphasia researchers? Addressing generalizability, variability, and interpretability

Abstract

Cognitive neuropsychologists have long sought a method that would allow them to characterize the impaired performance of individuals with brain damage, in order to draw valid inferences about information-processing architectures and inform treatment design. Some have advocated group studies where data are averaged over individuals to ensure generalizability, whereas others have argued that high variability in symptoms across individuals prohibits such grouping and instead warrants single case studies that exhaustively characterize individual deficits. In studies that complement such behavioral investigations with neuroimaging data—for linking brain structure and activity to cognitive impairments, spontaneous recovery, or treatment outcomes—the validity of inferences rests on a critical assumption: that a given anatomical landmark corresponds to the same functional unit(s) across individuals. This assumption, however, is fallacious: the mapping of function onto macroanatomy is not consistent even across neurologically healthy individuals, especially in association cortices. Here, we discuss the severe implications of this issue, and argue for an approach that circumvents it by harnessing the complementary strengths of group-based and individual-subjects approaches: group-level analysis of data from subject-specific regions of interest that are localized functionally using fMRI. This method eschews the requirement for functional-anatomical correspondence across brains, allowing for inter-individual variability in the precise location of functional regions. Once localized, the response profile of these regions can be characterized using an array of other tasks, and the results can then be averaged across individuals even if there is little overlap in the location of a given functional region across the sample. This method thus guarantees generalizable results while taking inter-individual variability into account. Furthermore, it eliminates the need for post-hoc “reverse inference” from anatomical landmarks back to cognitive operations, leading to improved interpretability of the data.

Three desiderata for cognitive neuropsychology

Cognitive neuropsychologists study individuals with brain damage in pursuit of two central aims: one is to analyze the behavior of such individuals in terms of impaired cognitive functions and thus develop or constrain theories of intact (non-damaged) information-processing architectures (Caramazza, 1992); the other is to characterize the pattern of spontaneous and treatment-induced recovery and thus inform or evaluate different interventions. The success of this research program critically depends on meeting three desiderata: (i) generalizability: the analysis of impaired behavior should capture universal aspects of cognitive architecture that are compatible with data across different individuals with brain damage, and treatment effects should be reliable across individuals with similar deficits; (ii) taking into account variability: theories of cognitive architecture, as well as treatments, should account for the wide diversity in symptoms across individuals diagnosed with the same syndrome; and (iii) interpretability: the experimental design should yield data that allow for valid inferences regarding both cognitive theories (which, in turn, should be computationally explicit enough to be informed by such data) and treatments.

Here, we discuss how these desiderata can be met in studies of individuals whose brain damage results in impaired language comprehension and/or production, i.e., persons with aphasia (PWAs). Although the ideas we present are pertinent to the study of any high-level cognitive deficit following brain damage, our focus on aphasia is motivated by historical, pragmatic, and discursive considerations. Historically, early aphasia research is acknowledged as the origin of the cognitive neuropsychological discipline (Broca, 1861/2006; Dax, 1863; Lichtheim, 1885; Wernicke, 1874/1969), and it may be interesting to consider what progress has been made from the time of those first studies towards meeting the desiderata articulated above. Pragmatically, aphasia affects approximately one third of stroke survivors (Berthier, 2005) and more than 30% of these exhibit long-term, severe deficits (e.g., Bakheit et al., 2007). Indeed, research on aphasia has been prominently featured in this journal throughout its 34-year history. Discursively, and most importantly, focusing on aphasia frames this commentary within a longstanding, passionate debate on whether one can rationally assume that studies of PWAs are, in principle, capable of meeting the outlined desiderata.

The critical concern of this debate has been that aphasia researchers are seeking knowledge about the language-processing architecture using data that might be inherently inadequate for providing such knowledge. This debate has challenged cognitive neuropsychologists to reexamine the methodological foundations of their discipline, leading to two conflicting conclusions: some scientists called for rejecting the common approach of “group studies”, in which data are averaged across a sample of PWAs with a shared syndrome, in favor of “single case studies” that comprehensively characterize the deficits of single individuals (Badecker & Caramazza, 1985; Caramazza, 1984, 1986, 1991; Caramazza & Badecker, 1989; Caramazza & McCloskey, 1988; McCloskey & Caramazza, 1988; Shallice, 1979); others, instead, defended group studies and questioned the validity of inferences made in single case studies (Bates, Appelbaum, & Allard, 1991; Bub & Bub, 1988; Caplan, 1988; Grodzinsky, Piñango, Zurif, & Drai, 1999; Newcombe & Marshall, 1988; Robertson, Knight, Rafal, & Shimamura, 1993; Whitaker & Slotnick, 1988; Zurif, Gardner, & Brownell, 1989; Zurif, Swinney, & Fodor, 1991).

Opponents of group studies, led by Alfonso Caramazza, have argued that this approach fails to meet two desiderata: it does not take into account the variability in symptoms across PWAs, and does not yield interpretable data. First, they have criticized the existing groupings of PWAs under diagnostic categories such as “agrammatism” for their reliance on subjective clinical intuitions (e.g., Badecker & Caramazza, 1985). Empirical data, they suggested, did not provide necessary and sufficient conditions for inclusion in such categories given the substantial, unexplained differences observed across individuals with a putatively shared deficit. Data interpretation was thus severely limited, with no principled way of distinguishing meaningless differences within a category (i.e., noise) from meaningful differences across categories (reflecting theoretically distinct impairments). Consequently, these categories appeared to be psychologically arbitrary.

Second, these critics emphasized that such arbitrariness cannot be resolved by further experiments. Rather, categorizations for group studies are arbitrary by construction (Caramazza, 1986): in a group study, a set of PWAs all showing some behavioral signature (the grouping criterion) are hypothesized to have a functional impairment in a certain, common component of the language-processing architecture; this hypothesis is evaluated by testing whether this group exhibits, on average, another behavioral signature (the predicted symptom) that should result from this postulated impairment. However, such averaging is valid only if the group is homogeneous with respect to the predicted symptom (and the underlying impairment), for which there is no a-priori guarantee. Thus, when a statistical test indicates that the predicted symptom is present in this group, it does not specify whether the symptom is (i) present in every individual, supporting the tested hypothesis; or (ii) present in most but not all individuals, arguably licensing the rejection of the hypothesis and the adoption of a new categorization scheme: PWAs exhibiting both the grouping criterion and the predicted symptom belong in one category, whereas those exhibiting only the grouping criterion (but not the predicted symptom) belong in another. These two scenarios can be distinguished only through an analysis of each individual in the sample (i.e., a series of single-case analyses).

In contrast, proponents of group studies have pointed out that inter-individual variability in the predicted symptom is naturally incorporated into statistical inference: significance tests evaluate the size of a group-averaged effect (the signal) against this variability (the noise), which is similar to unexplained noise in data from neurologically healthy individuals. High variability would not yield significant effects, and the resulting readjustment of infelicitous grouping criteria should, over time, converge towards theoretically meaningful syndromes (Zurif et al., 1989).

Moreover, group studies have been advocated for overcoming the limitations of single case studies, which have been criticized for failing to meet the desiderata of generalizability and interpretability. Single case studies give excessive weight to the behavioral symptoms of one individual despite lacking a criterion for distinguishing between behaviors that reflect damage to language mechanisms and those that are idiosyncratic, “accidental”, and irrelevant (e.g., Zurif et al., 1989). For instance, the pattern of impaired and spared language functions of a PWA may reflect some atypical premorbid neurocognitive characteristics, unusual behavioral strategies for solving experimental tasks, or a non-representative and extreme manifestation of a certain deficit (e.g., Rosenbaum, Gilboa, & Moscovitch, 2014; Varley, 1998). Taking such idiosyncrasies as evidence for constraining the language-processing architecture is a potentially precarious practice, especially when those idiosyncrasies are incompatible with a cognitive model that has previously received wide empirical support (Caramazza, 1986). Linking behavioral data to an underlying cognitive architecture based on a single case is further complicated because patterns of behavioral impairment are complex: rather than exhibiting impaired performance in one task and intact performance in another, a patient would often show somewhat impaired performance in one task and disproportionately more impaired performance in another (e.g., Bi, Han, Shu, & Caramazza, 2007; Caramazza & Hillis, 1991; Rapp & Caramazza, 2002; see also Van Orden, Pennington, & Stone, 2001). Even less defensible is generalization from a case study when characterizing patterns of spontaneous or treatment-based recovery. Overall, given that a particular behavioral pattern often admits several cognitive explanations—however a-priori unlikely some of them might be—a cognitive neuropsychologist ought to evaluate the relative support that each explanation receives from the data. Because differences in such relative support across hypotheses increase with the addition of independent observations, studies that examine multiple PWAs are sometimes claimed to confer higher generalizability. Simply put, group studies are assumed to average out the effects of potential idiosyncrasies on task performance. Nonetheless, some proponents of single case studies maintain that this statistical sense of “generalizability” is irrelevant to their paradigm, which is argued to provide “existence proofs” of cognitive deficits.

A second, related threat to generalizability in single case studies is that their findings are potentially non-falsifiable in both practice and principle. In practice, a PWA is often chosen as a case study based not on some clinical syndrome but on their performance of the critical tasks in that study. Such screening renders the inclusion criterion and the hypothesis non-independent, virtually assuring support for the hypothesis. The resulting case reports are biased in favor of PWAs whose behaviors exhibit the effect of interest, even if this effect characterizes very few of the individuals originally screened (Robertson et al., 1993). Furthermore, when a single case study reports that two linguistic functions are dissociable – one can be selectively impaired while the other remains intact – the findings are interpreted as a proof of concept that these functions are cognitively separable. Thus, it is in principle impossible to obtain new data from other PWAs that would falsify this conclusion (Caramazza, 1986). However, if scientific progress consists of updating theories based on observed data, then the amount of neuropsychological evidence in support of a theory should be viewed against the amount of neuropsychological evidence against it.

Finally, some critics of single case studies have asserted that the interpretable unit of analysis in aphasia research is the distribution of effects across individuals rather than individual data (Grodzinsky et al., 1999). Specifically, if a group of PWAs with a given syndrome are impaired on a certain task, they are not expected to all perform at chance but, rather, to be distributed around chance performance. Thus, an individual sampled from this group as a case study might perform above chance, leading to the erroneous conclusion that the syndrome in question does not entail impairment on the tested task. In contrast, a group study would yield a distribution of performance scores that is more consistent with chance performance than with no impairment.

Evidently, the debate over single case and group studies has urged advocates of each approach to articulate their conflicting views on using data from PWAs to obtain knowledge about the language-processing architecture. This reflective process has had a formative influence on cognitive neuropsychology. At a more pragmatic level, both single case and group studies have been well justified through their complementary contributions to cognitive science as well as to treatment development, which have been numerous and profound (Caramazza, 1992). Nonetheless, and despite some attempts at a compromise between these approaches (e.g., Bates, McDonald, MacWhinney, & Appelbaum, 1991), a behavioral methodology that synergistically harnesses their respective strengths in order to meet all three desiderata – generalizability, taking into account inter-individual variability, and interpretability – is yet to be found.

Struggling with irreconcilable desiderata in neuroimaging studies of aphasia

Over the past two decades, cognitive neuropsychologists have expanded their methodological arsenal beyond traditional measures of behavioral task performance as they have increasingly recognized the potential of neuroimaging data to inform their research questions. First, insofar as distinct components of the language-processing architecture engage separable neural circuits, we can expect that damage to a given circuit will consistently and selectively impair a certain component across PWAs. Thus, testing whether only those PWAs who share a putative syndrome have a common locus of brain damage (e.g., Bates et al., 2003) could indicate whether this syndrome is a natural category and, consequently, inform cognitive theories (e.g., Mesulam, Thompson, Weintraub, & Rogalski, 2015). Such research is, in essence, an extension of the cognitive neuroscientific program (Mather, Cacioppo, & Kanwisher, 2013) to the study of PWAs. Second, patterns of brain reorganization following damage may have a prognostic value (e.g., van Hees, McMahon, Angwin, de Zubicaray, & Copland, 2014; van Hees, McMahon, Angwin, de Zubicaray, Read, et al., 2014) and inform treatment design (for a review, see: Meinzer, Harnish, Conway, & Crosson, 2011). Specifically, identifying those brain regions or networks whose functional properties are associated with language rehabilitation would both provide targets for neuro-stimulation treatments (e.g., Geranmayeh, Brownsett, & Wise, 2014; Hamilton, Chrysikou, & Coslett, 2011; Naeser et al., 2005) and motivate behavioral interventions to capitalize on the particular cognitive processes that engage these regions / networks (e.g., van Hees, McMahon, Angwin, de Zubicaray, & Copland, 2014).

Excitement about these potential contributions of neuroimaging techniques has been reflected in an increasing number of studies on aphasia employing such techniques, mostly functional MRI (fMRI; Geranmayeh et al., 2014). However, as fMRI gained popularity among aphasia researchers, they have gradually recognized that adapting neuroimaging methodologies – originally developed for the study of neurologically intact populations – had to confront the same challenges faced by behavioral methods: establishing generalizability, taking into account inter-individual variability, and maintaining interpretability. These issues have prompted collaborative efforts to develop methodological guidelines for the use of fMRI in aphasia research (Crosson et al., 2007; Kiran et al., 2013; Meinzer et al., 2013; Price, Crinion, & Friston, 2006; Rapp, Caplan, Edwards, Visch-Brink, & Thompson, 2013; Veldsman, Cumming, & Brodtmann, 2015; Wilson, Bautista, Yen, Lauderdale, & Eriksson, 2016). Naturally, in striving to meet the three desiderata, a critical focus of these efforts has been to negotiate the relative benefits (and respective costs) of group and single case studies.

This negotiation ought to take into account the limitations of both fMRI and the study of brain-damaged populations. First, the signal measured with fMRI is heavily contaminated with fluctuations that are unexplained by the cognitive processes of interest. Such noise compromises the sensitivity to task-induced signal changes, especially given their small effect size (~1–2% increase relative to baseline). Thus, identifying signal differences across experimental conditions in data from a single participant often mandates that the contrasted conditions be substantially distinct (e.g., sentence comprehension vs. fixation; Crosson et al., 2007), but such gross contrasts cannot isolate a single cognitive process. Subtle contrasts (e.g., listening to sentences that are locally ambiguous vs. unambiguous) require larger amounts of data for sufficient statistical power; and more data are easier to obtain by studying a group of PWAs than a single individual due to, e.g., attrition, practice effects, and constraints on scan duration (especially for older individuals, some of whom find the scanner environment uncomfortable).

Second, lesions vary highly in extent and location across PWAs and typically do not neatly correspond to a single brain region or cognitive function, such that an accurate mapping of components in a language-processing architecture onto the brain requires pooling data across individuals (Bates et al., 2003). Third, establishing that patterns of brain reorganization associated with recovery are generalizable rather than idiosyncratic to a particular individual also requires such pooling. Finally, identifying brain activation patterns that are related to behavior is contingent on observing variability in task performance that can be correlated with variability in fMRI data. Although such variability can be obtained in a longitudinal study of a single PWA throughout recovery, in practice its measurement is more feasible across a group of PWAs.

Perhaps unsurprisingly then, neuroimaging studies of aphasia have overall favored group studies over single case studies. In such studies, summary statistics characterizing a sample of PWAs are obtained by collecting spatially localized measures of brain structure or activity for each individual, and then averaging these data across the group. Critically, this averaging requires a principled way for establishing correspondence across brains, which naturally differ in size, shape, and folding patterns. The common solution is to transform data from all individual brains such that they match a template of a “standard”, or “average”, brain. This normalization into a “common space” establishes anatomical correspondence across brains, i.e., a one-to-one mapping between stereotactic coordinates in this space and anatomical landmarks, rendering data pooling straightforward.

This procedure crucially relies on the implicit assumption that, across individuals, a given anatomical location houses the same functional (i.e., cognitive) unit(s). However, this assumption is demonstrably invalid. The mapping of function onto macroanatomy exhibits high inter-individual variability even in the neurologically intact population (Duffau, 2017), in line with variability in the mapping between micro-anatomical structure and sulci/gyri, as has been long established (e.g., Brodmann, 1909/1994). This structural and functional variability is especially pronounced in associative cortices, which are of highest interest to aphasia researchers, i.e., the temporal lobe (Gloor, 1997; Jones & Powell, 1970; Wise et al., 2001), the frontal lobe (Amunts et al., 1999; Juch, Zimine, Seghier, Lazeyras, & Fasel, 2005; Tomaiuolo et al., 1999), and the parietal lobe (Caspers et al., 2008; Caspers et al., 2006; Scheperjans et al., 2008). In these areas, regions with distinct functional profiles often lie side by side, but their precise anatomical locations are inconsistent across individuals. For instance, in Broca’s area, regions that are selectively engaged in linguistic tasks are adjacent to regions that are engaged in cognitively effortful tasks across many domains (Fedorenko, Duncan, & Kanwisher, 2012); and in posterior superior temporal sulcus, language-selective regions are adjacent to regions that support mental state inference (Deen, Koldewyn, Kanwisher, & Saxe, 2015). In both of these areas, a given stereotactic coordinate may thus exhibit one functional profile in one individual and a different profile in another individual (Figure 1).

Figure 1.

Left: language-selective regions (red) and domain-general regions engaged across many cognitively effortful tasks (blue) in the left hemisphere of three neurologically healthy adults. The former regions, identified with a passive reading task, show a stronger response to sentences than lists of nonwords (Fedorenko et al., 2011); the latter regions, identified with a working-memory task that requires keeping track of several spatial locations, are more engaged during trials that are harder (8 locations) vs. easier (4 locations) (Fedorenko et al., 2013). Activations for both tasks are thresholded at the p < 0.001, uncorrected whole-brain level. Right: the same individual maps highlighting the variability in the precise locations of functional activity. Squares: sample regions that are language-selective in one individual but domain-general in another. Circles: sample regions that are language-selective in one individual but are not recruited during either task in another. Triangles: sample regions that are domain-general in one individual but are not recruited during either task in another.

Because anatomy—at least at the current resolution of MRI—is not a precise predictor of function, group averaging of fMRI data based on anatomical alignment of brains is a precarious practice leading to erroneous inferences (Fischl et al., 2008; Frost & Goebel, 2012; Tahmasebi et al., 2012). First, when a functional region shows little spatial overlap across individuals, its activation might go undetected at the group level (compromised sensitivity). Thus, anatomical alignment is biased to reveal an effect of interest only in those brain regions whose activations are relatively strong, spatially extensive, or happen to be anatomically consistent across the individuals in a particular sample; other regions that show the same effect are often missed. Such Type-II errors might be mistakenly taken as evidence that the effect of interest is spatially restricted to a subset of some functional network or even localized to a single region. Moreover, an effect that appears localized to a certain region in some studies might instead appear localized to a different region in other studies (see, e.g., Blank, Balewski, Mahowald, & Fedorenko, 2016 for a discussion of such errors in the case of syntactic processing). Second, anatomically aligned data might conflate functionally distinct activations when they spatially overlap across individuals such that, at the group-level, they appear to originate from a single region (low functional resolution) (Fedorenko & Kanwisher, 2009). This illusory co-localization of effects (Type-I errors) might be taken as evidence against the functional specialization of a brain region for a particular cognitive process, even though, in each individual, the two effects arise in distinct, albeit adjacent, regions (see, e.g., Fedorenko, Duncan, et al., 2012 for a discussion of language specificity in Broca’s area).

Third, even if one could easily determine which group-based effects are representative of individual brains rather than artifacts of anatomical alignment across participants, the interpretation of such effects in terms of mental functions is severely limited. Namely, when we detect group-level activity in a certain anatomical location (e.g., Broca’s area) during some task (e.g., object naming), we must use post-hoc “reverse inference” from stereotactic coordinates back to the underlying cognitive process (e.g., prior studies have reported that Broca’s area is engaged in linguistic processing). However, such inferences are not deductively valid (e.g., activity in Broca’s area might reflect either a language-specific operation or general attention / task engagement) (Poldrack, 2006). Thus, deciding which reverse inferences happen to be veridical and which are false often amounts to a “gambling game” (but see Yarkoni, Poldrack, Nichols, Van Essen, & Wager, 2011; Yarkoni, Poldrack, Van Essen, & Wager, 2010).

These issues are exacerbated when studying brain-damaged populations, due to several added sources of inter-individual variability: (i) lesion extent and location are highly variable, compromising group-level detection of perilesional activations (Crosson et al., 2007); (ii) PWAs with lesions in a common anatomical location can present with different symptoms (Basso, Bracchi, Capitani, & Laiacona, 1987; Hillis et al., 2004); and (iii) PWAs with similar symptoms vary in the anatomical location of their respective lesions (Bonilha & Fridriksson, 2009; Dronkers, 2000; Mesulam et al., 2015; Newhart, Ken, Kleinman, Heidler-Gary, & Hillis, 2007).

Unfortunately, whereas cognitive neuropsychologists recognize these sources of variability across PWAs, they seldom account for the fundamental variability in functional-to-anatomical mapping in their neuroimaging studies (see also Rorden & Karnath, 2004); even when group studies are rejected in favor of single case studies that can capture individual-level relationships between lesions, activity in spared brain regions, and behavior, the cognitive interpretation of neuroimaging data proceeds through precarious reverse inference. The result is a literature with apparent inconsistencies that do not fit naturally within a unified framework. For instance, whereas spontaneous recovery in PWAs appears to be associated with both left-hemispheric and right-hemispheric recruitment, conflicting evidence suggests that the latter might play either a compensatory/restorative role or a maladaptive/detrimental role (Anglade, Thiel, & Ansaldo, 2014; Hamilton et al., 2011; Heiss & Thiel, 2006; Turkeltaub, Messing, Norise, & Hamilton, 2011), even within the same individual (Turkeltaub et al., 2012). In addition, within each hemisphere, many brain regions that engage in language processing in neurologically healthy individuals (Binder et al., 1997; Fedorenko, Behr, & Kanwisher, 2011; Fedorenko & Thompson-Schill, 2014; Jung-Beeman, 2005; Menenti, Gierhan, Segaert, & Hagoort, 2011) have been irregularly reported to also be engaged during spontaneous recovery, but few could be reliably predicted to do so (Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 2004; Meinzer et al., 2011; Mesulam et al., 2015; Mirman et al., 2015). Moreover, as described above, interpreting activations based on macroanatomical landmarks conflates language-specific mechanisms with domain-general resources related to cognitive effort or with mechanisms that support information processing in domains other than language (Geranmayeh et al., 2014). Similar conundrums are widespread in studies of treatment-induced brain changes (Crinion & Leff, 2007; Meinzer & Breitenstein, 2008; Thompson & den Ouden, 2008).

Neuroimaging studies of aphasia have so far struggled to offer a methodological approach that meets all three desiderata of cognitive neuropsychological research. Namely, attempts at generalization beyond a single case have consistently ignored critical inter-individual variability in the precise anatomical location of functional regions, thus paradoxically hindering the replicability of results across studies. Furthermore, such variability renders the linking of anatomy back to cognitive processes logically flawed and, hence, potentially erroneous. These issues affect virtually all existing neuroimaging investigations of PWAs, including (i) voxel-based, lesion-symptom mapping analyses of anatomical data (Bates et al., 2003; Dronkers et al., 2004; Geva et al., 2011; Mesulam et al., 2015; Mirman et al., 2015; Wilson, 2016); (ii) group-level analyses of functional data in a common space, for identifying stereotactic coordinates that show an effect of interest across the sample (e.g., studies contrasting the recruitment of the two hemispheres during spontaneous recovery, cited above); (iii) group- and individual-level functional characterizations of particular brain regions that are chosen based on an independent, but group-based, criterion such as an independent task (Sharp, Turkheimer, Bose, Scott, & Wise, 2010) or data from neurologically healthy individuals (Bonner & Grossman, 2012; Fridriksson, Bonilha, Baker, Moser, & Rorden, 2010); and (iv) comparisons of fMRI data across a series of single cases on the basis of anatomical alignment. The implications, for anyone who regards neuroimaging as a valid research method in cognitive neuropsychology, are alarming.

A possible reconciliation: group-level analysis of subject-specific functional regions

One might wonder whether the criticisms articulated above are not limited only to fMRI studies of brain-damaged individuals and, thus, might more generally compromise the validity of most neuroimaging studies. Indeed, already two decades ago, cognitive neuroscientists became cognizant of the challenges associated with analyzing functional data from anatomically aligned brains. An elegant methodology for circumventing these issues, originally developed for studying low-level visual processes (e.g., Sereno, Dale, Reppas, & Kwong, 1995; Tootell et al., 1995), was brought by Nancy Kanwisher to the study of higher-level cognition in a seminal study of face perception (Kanwisher, McDermott, & Chun, 1997). In particular, she matched brain regions across individuals based directly on the observable functional profiles of those regions rather than through structural alignment, eschewing the reliance on anatomical landmarks as a proxy for function.

Abandoning the requirement for precise anatomical correspondence across brains in favor of functional correspondence has since become a heralded standard in neuroimaging studies of vision and is effectively required for targeting many visual processes (some critiques of this approach have been summarized in Friston, Rotshtein, Geng, Sterzer, & Henson, 2006; for a rebuttal of these early misunderstandings, see: Saxe, Brett, & Kanwisher, 2006). In contrast, most cognitive neuroscientists studying the language network have not embraced this method, barring few early adopters (Ben-Shachar, Hendler, Kahn, Ben-Bashat, & Grodzinsky, 2003; Ben-Shachar, Palti, & Grodzinsky, 2004; Hickok, Buchsbaum, Humphries, & Muftuler, 2003; January, Trueswell, & Thompson-Schill, 2009; Neville et al., 1998). More recently, however, the method has been adapted in a principled manner to the study of language processing by Fedorenko et al. (2010), who argued for its superiority over existing practices on both theoretical (Nieto-Castañón & Fedorenko, 2012; see also Saxe et al., 2006) and empirical (e.g., Blank et al., 2016) grounds. Consequently, it has been increasingly employed to revisit, refine and, often, challenge traditional views on the language-processing architecture (Axelrod, Bar, Rees, & Yovel, 2015; Basilakos, Smith, Fillmore, Fridriksson, & Fedorenko, 2017; Blank, Kanwisher, & Fedorenko, 2014; Chai, Mattar, Blank, Fedorenko, & Bassett, 2016; Deen et al., 2015; Fedorenko et al., 2011; Fedorenko, Duncan, et al., 2012; Fedorenko, Fillmore, Smith, Bonilha, & Fridriksson, 2015; Fedorenko, McDermott, Norman-Haignere, & Kanwisher, 2012; Fedorenko, Nieto-Castañón, & Kanwisher, 2012a; Fedorenko et al., 2016; Humphreys & Gennari, 2014; Hung et al., 2015; Mahowald & Fedorenko, 2016; Overath, McDermott, Zarate, & Poeppel, 2015; Prado, Mutreja, & Booth, 2013; Redcay, Velnoskey, & Rowe, 2016).

Establishing functional correspondence across brains proceeds as follows (Figure 2): first, fMRI data are collected while participants perform a task designed to identify, in each individual, the location of brain regions that exhibit a specific functional signature. For example, to identify regions engaged in high-level language processing, the “localizer” task can contrast linguistically well-formed materials and linguistically “degraded” materials that are matched on low-level properties (e.g., sentences vs. lists of nonwords or acoustically-degraded speech) (Fedorenko et al., 2010; Scott, Gallée, & Fedorenko, 2016); and to define regions sensitive to general cognitive demands (e.g., Duncan, 2010), the localizer task can contrast a harder and an easier version of a demanding executive task (e.g., Fedorenko, Duncan, & Kanwisher, 2013). For each participant, these functional data are processed to produce an individual map showing the effect size of the localizer contrast for each location in the brain.

Figure 2.

Illustration of steps for defining subject-specific fROIs. Anatomical scans for 3 individual PWAs are shown in native space, with demarcated lesions. fROIs are defined by intersecting (i) individual activations during a localizer task (here, targeting the high-level language system); and (ii) broad masks denoting where such activations are likely to occur, taking into account inter-individual variability in their precise locations (here, masks are derived from the localizer data of 220 neurologically healthy, young individuals, and are projected onto the native space of each PWA). Top: posterior temporal mask. Bottom: inferior frontal mask. For further details on the localizer task and masks, see Fedorenko et al. (2010).

Next, each individual map is intersected with a set of spatial “masks” that denote, grossly, where the localizer task is expected to elicit activations, taking into account the anticipated inter-individual differences in the precise location of these activations. These masks effectively constrain the search for individual activation loci; still, within their borders, those loci are free to vary across participants (desideratum ii: accounting for variability). These masks can be based on macroanatomical landmarks, like a gyrus or a sulcus (or a portion thereof), chosen based on previous studies. However, activations do not always map neatly onto such landmarks. An alternative is thus to derive the set of masks algorithmically in a data-driven way from the localizer activation patterns in an (ideally, independent) set of participants: regions of activation with the highest empirical overlap across participants are identified, and the borders around these local maxima are gradually extended until the resulting masks are large enough to encompass the activations of a sufficient percentage of individuals (desideratum i: generalizability) (for full details, see Fedorenko et al., 2010; Julian, Fedorenko, Webster, & Kanwisher, 2012). In studies of PWAs, masks can be generated in different ways depending on the research goals: first, masks based on data from neurologically healthy participants allow the localization of regions whose functional profile in intact brains is well characterized. As long as these regions still show some response to the localizer task in PWAs, one can examine whether the recruitment of these regions during other tasks changes following brain damage or recovery. Second, masks based on previous aphasia research allow the localization of regions that are consistently recruited during language processing only following brain damage or during recovery and whose functional profile in intact brains is yet to be determined. (How consistent in behavior and lesion location a sample of PWAs should be in order to generate reliable masks remains to be empirically determined). Finally, we strongly recommend always examining whole-brain activation maps instead of merely testing activations within the constraints of pre-determined spatial masks; it is indeed possible (especially in cases of brain damage) that some new brain region would emerge outside the boundaries of these large masks. If that region appears to be present in a substantial proportion of PWAs, one would want to include it in the main analysis. Regardless of the process chosen for generating them, the resulting masks are then tailored to each individual PWA by excluding their particular lesion(s), i.e., limiting the search space for activations to intact tissue, capable of producing BOLD signal.

The intersection of an individual’s activation map with a mask for a particular region defines the participant-specific area within that mask that is responsive to the localizer contrast (based on, e.g., a statistical threshold). Because these areas are localized based on functional data, they are called functional regions of interest (fROIs). Critically, the obtained fROIs correspond to the same functional unit across individuals, insofar as they exhibit gross anatomical consistency and a characteristic response to the localizer task (desideratum iii: interpretability). Therefore, functional responses to the main, critical task(s) of the experiment can be extracted from these subject-specific fROIs and submitted to a group-level analysis, even if their fROI locations do not precisely overlap across the sample (for further discussion, see Saxe et al., 2006).

To establish that data from the localizer task allow for a reliable definition of fROIs, it is critical to confirm that the localizer activations are reproducible; otherwise, localization based on these activations might capture random noise instead of functional units with consistent response profiles (desideratum iii: interpretability). To this end, a cross-validation procedure should be employed: here, a portion of the localizer data (e.g., even runs) can be used to define the fROIs, and the held-out data (odd runs) can then be used to independently estimate responses to the localizer contrast in these fROIs. The estimates can then be tested across the sample to verify that the fROIs show the expected response profile (which served as the criterion for their definition). Importantly, such cross-validation also allows one to retroactively define fROIs in experiments where localizer data have not been originally collected, by splitting the data from the critical task into halves; each half in turn serves as “localizer” data for defining fROIs that respond to the critical task, with the remaining half serving as the “main” data for testing whether the responses in these fROIs are robust and replicable (otherwise, they are false positives) (for further discussion, see Nieto-Castañón & Fedorenko, 2012).

Whereas reliability is a necessary condition for functional localization in individual subjects, it is not sufficient: the localizer task should also be valid. Indeed, localizer validity might be compromised by two problems, but both can be empirically tested in a straightforward manner. To illustrate these problems, consider the following scenario: we hypothesize that in PWAs the right-hemispheric homologue of the language network is engaged in syntactic processing. Our critical task therefore tests whether the regions of this network respond more strongly to syntactically complex sentences than to syntactically simpler sentences in a sentence-picture matching task. Nonetheless, we must first functionally localize this network in each individual subject. What should our localizer task be?

We might choose a localizer task that is highly similar to the critical task, i.e., requires syntactic processing. Perhaps we would search for regions that respond more strongly to sentences with syntactic errors than to well-formed sentences (assuming we have established that the activations for such a localizer are robust enough to be reliably detected in individual subjects given the amount of data collected). Here, a potential concern is that the localizer task might bias the data in favor of our hypothesis: it might identify only those regions within the right-hemispheric language network that are engaged in syntactic processing, but miss other regions within this network. For instance, if linguistic processing in PWAs were divided across regions that were each specialized for either lexical or combinatorial processing,¹ then the syntactic-violation-based localizer would fail to identify the former regions. Our data would thus erroneously indicate that all (rather than some or, even, few) right-hemispheric language regions are engaged in syntactic processing. To mitigate this concern, we could instead design a localizer task that contrasts two conditions differing not only in their combinatorial characteristics but also in their lexical characteristics (see suggestions earlier in this section).

Following the reasoning above, we might be tempted to design a localizer that is as dissimilar as possible to the critical task. For instance, given that our main task contrasts sentences with complex vs. simple syntactic structures, our localizer could avoid sentences altogether and search for brain regions that respond more strongly to words than to reading non-words in an n-back task (again, assuming that the resulting activations are reliably detectable at the individual-subject level). However, a potential concern here is that the localizer might miss any regions that are engaged in only combinatorial linguistic processes but not lexical linguistic processes. If linguistic labor in PWAs followed a lexical-combinatorial division, our localizer would fail to identify the very regions that are critical for testing the syntactic-processing hypothesis (but we note that traditional approaches fare much worse in this respect).

To mitigate this concern, an independent run of the critical task could itself be used as a functional localizer, guaranteeing the identification of all regions that reliably scale their activity with syntactic complexity (note that this procedure guards against under-inclusion, unlike the aforementioned practice of behaviorally screening PWAs for single case studies based on the critical deficit itself, which might contribute to under-inclusion). In this case, at least one additional task is required to demonstrate that the functionally localized regions indeed belong to the language network rather than, e.g., scale their response with difficulty in any cognitive domain. Often, an elegant solution is to run an additional analysis where the critical task serves as the localizer and the localizer task serves as the main task. In the current example, this analysis would test whether all regions that are localized based on their engagement in syntactic processing also respond more strongly to words than to non-words during the n-back task (a response profile that is inconsistent with a general difficulty account, given that non-words are harder to process).

As cognitive theories are refined, the functional localizers that are motivated by these theories evolve. Debates surrounding any particular localizer task are a natural part of this process and, as we have demonstrated above, are subject to empirical investigation. To the extent that scientific controversies are inevitable, we hope that they would focus on designing a set of acceptable localizers for neuroimaging studies of PWAs, not on the value of this indispensible approach (discussions could also focus on developing ways to infer individual functional regions from patterns of anatomical connectivity via diffusion tensor imaging, e.g., Saygin et al., 2012; or from patterns of functional correlations via resting-state fMRI, e.g., Cohen et al., 2008; Eickhoff, Thirion, Varoquaux, & Bzdok, 2015; Fox, Liu, & Pascual-Leone, 2013; Jakobsen et al., 2016; Li, Langley, Li, & Hu, 2015). For those hypotheses that are amenable to investigation via fMRI, this approach maximizes the likelihood of gaining accurate insights. In fact, beyond its well-established superiority over traditional approaches that are based on anatomical alignment across brains of neurologically intact individuals, the group-level analysis of subject-specific fROIs confers benefits that are specifically critical to the study of PWAs:

First, this method can detect an effect of a given magnitude (and estimate that magnitude with higher fidelity; Nieto-Castañón & Fedorenko, 2012) with smaller samples than would otherwise be necessary, owing to its increased power, so the often limited pool of PWAs is less of a concern. At the same time, subject-specific localization obviates the need for excessive smoothing that attempts to compensate for inter-individual variability in data from small samples (Mikl et al., 2008). Second, normalization of functional data into a common space is unnecessary when fROIs are subject-specific, so analyses can proceed in the “native space” of each individual to minimize data transformations. Although such analyses instead require that the masks constraining fROI location be transformed from the common space into the native space, this process is relatively insensitive to imperfect alignment because these masks cover large areas of the brain. Third, activity in subject-specific fROIs has been shown to predict behavior or causally influence it, across several functional networks (Assem, Blank, Mineroff, Ademoglu, & Fedorenko, submitted; Pitcher, Charles, Devlin, Walsh, & Duchaine, 2009; Young, Camprodon, Hauser, Pascual-Leone, & Saxe, 2010), and establishing such brain-behavior relationships is crucial for individualized neuro-stimulation treatments.

Interestingly, this approach also provides a way to estimate the response profile that fROIs in PWAs may have exhibited prior to brain damage. Such estimation is informative for inferring the mechanisms underlying recovery in a damaged brain, e.g., reorganization of impaired functions, compensation by spared regions that are not part of the impaired network, or preexisting functional redundancies. To help adjudicate between these hypotheses, we have recently developed a “virtual lesion” procedure (Blank, Rohter, Kiran, & Fedorenko, 2015): first, an image of the lesion from a given PWA is superimposed on the functional data from a matched control participant (or a sample of participants who are matched on average). Then, fROIs in the control participant(s) are re-defined under the constraint that they cannot fall within the virtual lesion and must instead be identified in the “intact” parts of each mask. These fROIs serve as proxies for the neural architecture surrounding and preceding the lesion, and their responses to a variety of tasks can be compared to those of fROIs defined in PWAs. Admittedly, any insights provided by this analysis maintain considerable uncertainty, but no current neuroimaging method offers even this limited inferential power.

The most important, and unprecedented, advantage of this approach is its synergistic harnessing of the respective strengths of group and single case studies in order to meet all three desiderata of cognitive neuropsychology. Like group studies, it provides generalizable results because it tests for an effect of interest across a sample and, moreover, surpasses traditional analyses in sensitivity and functional resolution (Nieto-Castañón & Fedorenko, 2012). Like single case studies, it takes into account inter-individual variability that either had preceded or was induced by the lesion, because it relies on subject-specific localization rather than on anatomical alignment across brains. In addition, the obtained results are functionally interpretable without recourse to reverse inference from macro-anatomical landmarks back to mental operations (Poldrack, 2006, 2011), because fROIs are defined by their engagement in the cognitive processes targeted by the localizer task. Further, results for a set of fROIs from different studies and labs that employ similar functional localizers can be related to one another in a straightforward manner, avoiding the invalid comparison of stereotactic coordinates across studies (e.g., Fedorenko, Nieto-Castañón, & Kanwisher, 2012b). Such integration of knowledge is required for a cumulative research enterprise.

Conclusion

We have emphasized that meeting the three desiderata of cognitive neuropsychology in neuroimaging studies is a methodologically complicated issue, and described at length the proposed solution: group-level analyses of subject-specific functional regions. This intricate discussion, however, might belie the simplicity of implementing this method. Specifically, the entire analysis pipeline—from data-driven generation of masks, through cross-validated fROI definition to group-level analyses of activations in these fROIs during the critical experimental task—can be performed using a publicly available toolbox (https://www.nitrc.org/projects/spm_ss). Moreover, localizer tasks for defining fROIs that are engaged in high-level language processing, including passive listening tasks that are suitable for PWAs, have been developed and thoroughly validated (Fedorenko et al., 2010; Scott et al., 2016; Stoppelman, Harpaz, & Ben‐Shachar, 2013; Tie et al., 2015; Wilson et al., 2016), and are available for download (https://evlab.mit.edu/funcloc/download-paradigms). In addition, these localizers are now available in 41 languages (https://evlab.mit.edu/alice).

To date, only a handful of studies on aphasia have employed subject-specific functional localization, all for the purpose of identifying individualized targets for neuro-stimulation (Baker, Rorden, & Fridriksson, 2010; Fridriksson, Richardson, Baker, & Rorden, 2011; Kakuda, Abo, Kaito, Watanabe, & Senoo, 2010). Nonetheless, we have argued that this method promises to significantly advance the rigorous use of neuroimaging for the study of PWAs in pursuit of other aims as well. Whether the goal is to understand the neurocognitive mechanisms underlying recovery, to apply such knowledge to the design of behavioral interventions, or to generally elucidate the language-processing architecture of the human mind, our conviction is one: it is imperative that we standardize the use of group-level analyses of subject-specific functional regions.