Ten problems and solutions when predicting individual outcome from lesion site after stroke

Abstract

In this paper, we consider solutions to ten of the challenges faced when trying to predict an individual’s functional outcome after stroke on the basis of lesion site. A primary goal is to find lesion-outcome associations that are consistently observed in large populations of stroke patients because consistent associations maximise confidence in future individualised predictions. To understand and control multiple sources of inter-patient variability, we need to systematically investigate each contributing factor and how each factor depends on other factors. This requires very large cohorts of patients, who differ from one another in typical and measurable ways, including lesion site, lesion size, functional outcome and time post stroke (weeks to decades). These multivariate investigations are complex, particularly when the contributions of different variables interact with one another. Machine learning algorithms can help to identify the most influential variables and indicate dependencies between different factors. Multivariate lesion analyses are needed to understand how the effect of damage to one brain region depends on damage or preservation in other brain regions. Such data-led investigations can reveal predictive relationships between lesion site and outcome. However, to understand and improve predictions we need explanatory models of the neural networks and degenerate pathways that support functions of interest. This will entail integrating the results of lesion analyses with those from functional imaging (fMRI, MEG), transcranial magnetic stimulation (TMS) and diffusor tensor imaging (DTI) studies of healthy participants and patients.

Introduction

In recent decades, it has become increasingly clear that the effect of brain damage, after stroke, largely depends on which parts of the brain have been damaged. Most of the evidence has been gleaned from neuroimaging studies that have searched high resolution magnetic resonance images (MRI) for brain regions that are damaged in patients who have particular symptoms/functional loss. The results of these studies, using increasingly sophisticated techniques to overcome age-old challenges (Boxes 1 and 2), are important for identifying “structure-function relationships” and functional specificity in the human brain (see Karnath and Rorden, 2004 for review). The current paper is concerned with a different set of problems that emerge when we attempt to reverse the inference: i.e. predict functional outcome from a lesion (Inoue et al. 2014) as opposed to find lesion sites associated with a functional impairment.

Box 1. Well known problems with lesion-deficit analyses.

Stroke damage is not constrained by anatomical or functional boundaries

Some vascular territories are more likely to be damaged than others

Damage may not be detected by current imaging techniques

Dysfunction may occur where there isn’t any damage

Structure-function relationships may re-organise after brain damage

Cognitive functions typically require multiple distributed brain regions

Anatomical landmarks vary from one individual to another

Normalising anatomical variability is challenging after brain damage

Box 2. Techniques that find lesion sites associated with a functional impairment.

• MAP-3BrainVox’s templates for comparing patient groups (Frank et al., 1997)
• VBMVoxel based morphometry (Mummery et al., 2000)
• VSLMVoxel based lesion-symptom mapping (Bates et al., 2003)
• VALVoxel-based analysis of lesions (Karnath et al., 2004)
• AnaCOMAnatomico-Clinical Overlapping Maps (Kinkingnéhun et al., 2007)
• PM3Proportional MAP-3 (Rudrauf et al., 2008; Inoue et al., 2014)

For further details and comparisons of these techniques, see: Mehta et al., 2003; Karnath et al., 2004; Rorden et al., 2009; Geva et al., 2012.

The problem with reversing the inference is as follows: the results of neuroimaging studies that find brain sites where damage is significantly greater in those with poorer performance (VBM, VSLM) or for a group of patients with a functional impairment relative to another group without the functional impairment (VAL, AnaCOM, MAP-3, PM3), can be driven by a comparatively small subset of participants (Figure 1). Unless further analyses are conducted, the results summed over groups of individuals do not indicate which of the patients with damage will or will not have the functional impairment.

Figure 1. Significant group differences can be driven by a subset a patients.

In order to provide patients with accurate predictions that they and their carers can be confident in, we need to (1) identify lesion-outcome associations that are highly consistent across individuals (i.e. low inter-patient variability); (2) confirm that the identified lesion-outcome associations predict behaviour in new patients (i.e. cross-validation) and (3) develop procedures for introducing this information into the clinic (i.e. clinical translation). The problems and solutions considered in this paper focus only on the first of these steps; i.e. identifying lesion-outcome associations that are highly consistent across individuals. We use the term “outcome” rather than “deficit” or “symptom” because outcome conveys the possibility that a deficit or symptom can vary across time (i.e. the effect of recovery). The word “outcome” can also be used to describe functional abilities that are normal as well as impaired.

When describing potential solutions, we distinguish between two different goals. Studies searching for lesion sites associated with outcomes typically aim to draw conclusions about the function of the underlying brain regions. For example, the precentral gyrus above the insula is thought to play a role in articulating speech because it is damaged in patients who have speech articulation difficulties (Baldo et al., 2011). We refer to this type of goal as a “model-based”. In contrast, when predicting outcome from lesion sites, the goal is to find reliable predictive relationships, irrespective of whether we understand the underlying functional anatomy. We refer to this as a “data-led” account; for more details see Price et al. (2010).

The first three problems described below apply to model-based accounts but are not so critical for data-led accounts. We then focus on solutions that affect both model-based and data-led accounts. Together both approaches can be used to derive a comprehensive understanding of the multifaceted association between structure and outcome.

Problem 1 (P1). Impairments from undamaged but disconnected regions

It has long been appreciated that damage to one part of the brain might disconnect and cause dysfunction in other parts of the brain (Catani and ffytche, 2005; Mesulam, 1990). A function that is lost could therefore have been the property of the damaged region or undamaged but disconnected regions. It is important for model-based accounts of lesion-outcome associations to establish which of these alternative possibilities applies so that they can make interpretations about the underlying functional anatomy. In contrast, data-led predictive accounts of lesion-outcome associations are primarily concerned with whether damage to a region causes a functional deficit, irrespective of whether that function was previously the property of the damaged region or the disconnected regions. Consequently, we are not concerned with finding a solution to this problem at present as long as it is not subject to any of the other problems listed below. We note, however, that sometimes neurophysiological markers of a disconnection of axonal afferants in an undamaged region can be detected. This is referred to as diaschisis which arises when cerebral blood flow decreases in a disconnected area (Slater et al., 1977) as a consequence of changes in anatomical structure and functional connectivity (Carrera and Tononi, 2014).

Problem 2 (P2) Damage to the vascular system not the neural system

This is a variation of the first problem but refers to cases where a region susceptible to vascular damage consistently results in a functional impairment, even though the lost function was never a property of the damaged region (Mah et al., 2014). For example, if a function is supported by neural activity in Regions A,B and C; and each of these regions receives a blood supply stemming from Region V, then damage to Region V will impair the function of A, B and C even though neural activity in Region V is not involved in computing the function. It is important to answer this question for model-based accounts of lesion-outcome associations that do not want to misinterpret the contribution of Region V. On the other hand, for data-based predictions, it is informative to know that a functional outcome consistently occurs after damage to Region V, even if the neural mechanisms underlying the loss are not fully understood. We are therefore not concerned with finding a solution to this problem at present as long as the lesion-outcome association is observed irrespective of lesion size (the importance of which is described in P5 below).

Problem 3 (P3) Damage to the same region impairs different functions

If we are concerned with understanding the functional contribution of a brain region, it can be challenging to interpret observations that the same region appears to be involved in very different functions. In such cases, do the different functions share an underlying computation that has not previously been appreciated? Alternatively, could the common cause of the different impairments be loss of a blood supply that feeds two different neural systems? These questions can be difficult to answer particularly when functions of interest only emerge from integrated activity across multiple brain regions. A body part analogy here would be the function of a finger that contributes to very diverse behaviours (typing or piano playing) depending on the combination of other fingers, thumbs and body parts it is interacting with, and the precise timing of these interactions. We may be a long way from understanding how the brain sustains so many different functions but in terms of predicting outcome, it is sufficient to know that loss of a brain region consistently impairs a set of functions.

Problem 4 (P4): Seeing past inter-patient variability

The issue here is that damage to the same part of the brain can have different effects in different patients (Hillis and Tippett, 2014; Koh et al., 2015; Watila and Balarabe, 2015). In light of this inter-patient variability, how can we make accurate predictions for new patients?

There are several reasons to be optimistic. First, we know that although outcome varies substantially across patients for some lesion sites, damage to other brain regions can have much more consistent effects. For example, left occipito-temporal damage typically impairs fast efficient reading (Leff et al., 2006). We should therefore not let inter-patient variability in the effect of some lesion sites distract us from identifying consistent effects that would be useful and informative for other patients.

Second, functional imaging studies of healthy participants have shown that variability is not just noise, but can inform us about the different ways our brains can support behaviour. Once some sources of variability have been controlled, it becomes clearer that functional responses from many brain regions are highly consistent across healthy participants, particularly in sensory and motor regions. Although consistent functional responses in healthy participants do not predict how the brain will respond after damage, they do indicate when inter-patient variability in response to damage is unlikely to be due to pre-morbid differences in functional anatomy.

Third, many sources of inter-patient variability have already been identified (see Box 3) and can therefore be controlled when searching for consistent lesion-outcome-associations. Some relate to the patients themselves, including demographic factors and co-morbidities, time post-stroke and therapies that speed up the recovery process.

Box 3. Sources of inter-patient variability.

A) Variables of interest that need to be measured in lesion-deficit analyses

Number, location and size of cerebral infarcts (lobar or lacunar) in both hemispheres

Time post stroke (Hope et al., Neuroimage Clin. 2013;2:424-332013)

B) Variables that need to be controlled/investigated

Age and Gender (Knoflach et al., Neurology 2012, 78:279-285)

Ability to see and hear during cognitive and physical assessments

Neurological & psychiatric co-morbidities

Hand preference (particularly important for language and motor studies)

Mother tongue and languages spoken (particulatly important for language studies)

Premorbid cognitive status and education (Ojala-Oksala et al., Stroke 2012, 43:2931-5)

Severity of intial symptoms (from questionnaires and medical records)

C) Variables that are less easy to control/investigate

Nature and duration of interventions

The effect of post stroke depression on relearning

Pathological or age related cognative decline post stroke (from longitudinal assessments)

Progressgive brain atrophy (Seghier et al., Stroke. 2014;45:877-9)

Neuro-infammation (acute swelling and/or later shrinkage)

Abnormalities due to carotid or microvascular disease / Leukoaraiosis

Cerebral re-perfusion after stroke

Many other pathological factors and post stroke interventions

D) Hereditary factors (true inter-patient variability)

Differences in neuronal and vascular network

Differences in functional anatomy

Other sources of inter-patient variability relate to basic principles of functional organisation and re-organisation. For example, the effect of damage to a region will depend on whether other regions that can potentially compensate for the lost region are also damaged (see Figure 2). We have previously referred to the availability of alternative neural pathways to support the same task as “degeneracy” (Price and Friston, 2002) and found this to be a useful concept in explaining recovery of function following damage to the normal system (Price et al., 2010). For example, take the case of patients with cortical blindness after damage to primary visual cortex. Although these patients have damage to the primary geniculo-striate pathway, they can learn to make a wide range of visual discriminations surprising well in their blind field (Das et al., 2014). This ability to relearn is difficult to explain in the absence of an alternative visual pathway. It has now been shown that a second visual pathway exists, in which visual information travels from the superior colliculi (in the brainstem) to the cortex (e.g. the motion area) via the pulvinar (Berman and Wurtz, 2010). The existence of this second pathway, which bypasses the primary visual cortices, can explain many intriguing observations in blind subjects (e.g. blindsight), even though the recovery capacity through this alternative pathway might be limited.

Figure 2. Lesions that do/do not cause persistent task difficulty.

The existence of alternative pathways is not limited to the visual system. Indeed, recent studies have shown that many abilities can be sustained by different pathways, including language processing (Friederici, 2011; Kümmerer et al., 2013), word reading (Richardson et al., 2011; Seghier et al., 2008; Yvert et al., 2012), action imitation (Mengotti et al., 2013; Tessari et al., 2007), and emotion processing (Pessoa and Adolphs, 2010). The point here is that by taking into account the existence of different alternative pathways, we can improve our understanding of lesion-outcome associations and account for inter-patient variability in relearning and recovery. Most poignantly, we can predict that task performance will be worse when all possible neural support system are lost than when one of the possible set are preserved.

In addition to the above sources of inter-patient variability, there are a number of reasons why lesion-outcome associations might vary from one study to another. These include how the lesion is measured, how the outcome is measured and how the lesion and outcome are associated and modelled. In some studies, for example, lesions are identified by (i) a neurological expert manually drawing around the observable sites of damage (Damasio and Frank, 1992; Bates et al, 2003); (ii) automated lesion identification programs that identify abnormal tissue after comparing the patient brain to normal brains (Seghier et al., 2008); or (iii) quantitatively inferred from image signal in voxel based morphometry (Mehta et al., 2003). Likewise, a functional loss may be described in terms of (i) failure to achieve a particular task goal (e.g. name an object) which might be caused by a breakdown at one of several different computational levels (e.g. failure to see, recognise, retrieve the name or articulate speech) or (ii) relative performance on a range of tasks (e.g. patients who can recognise objects but cannot name them). Methods for relating lesion site to functional loss also vary depending on whether those investigating the lesion-outcome association use an anatomical region of interest, a functional region of interest, mass univariate lesion analyses or multivariate lesion analyses (Chen et al., 2008; Smith et al., 2013; Hope et al., 2014; Zhang et al., 2014). Finally, inter-patient variability may depend on other factors such as atypical performance (e.g. because of fatigue) on the day of the assessment.

P4 Solutions (S)

P4_S1) To understand, control and tease apart the relative influence of all the lesion and non-lesion factors generating variability in outcome and recovery post-stroke, we need to conduct standardised assessments on very large cohorts of patients who have, collectively, incurred a comprehensive range of brain lesions, and differ from one another in measurable ways (see Box 3).

P4_S2) To test the consistency of a lesion-outcome association, we need to include patients who do and do not have impairments. This allows us to check whether lesion sites associated with an impairment are also observed in those who do not have the same impairment. By including patients who do not have impairments, we can also identify lesions that rarely if ever cause an impairment.

P4_S3) To account for functional reorganisation and recovery after stroke, we need to include patients who are tested at multiple time points post-stroke. To explain inter-patient variability in the time course of recovery, we also need to consider how the effect of time post-stroke varies with many other factors such as the patient’s age, prior education, amount and type of treatment, current cognitive state and co-morbidities (see Box 3).

P4_S4) When multiple sources of potential variability have been controlled, consistent lesion-outcome associations may be identified either from (a) post hoc analysis of lesion sites seen in those who do versus do not have an impairment in the function of interest, or (b) lesion overlap maps for patients who have very specific functional impairments. Whether or not a consistent lesion-outcome association is observed, further analyses and cross-validation are required (see below).

Problem 5 (P5): Large lesions dominate lesion-outcome associations

A consistent lesion-outcome association should be independent of lesion size. If the lesion-outcome association is identified from samples that include patients with large lesions, a consistent lesion-outcome association might be misinterpreted. This is illustrated in Figure 3 using the hypothetic neural system introduced in Figure 2.

Figure 3. A problem with large lesions.

All four lesions damage the system described in Figure 2 and impair the same function. The most consistent site of damage (B) falsely implies that region B could be the common cause of the deficit even though selective damage to B would not cause a deficit when A to C is preserved.

In Figure 3, four patients with the same functional impairment all have large lesions that include Region B. This consistent lesion-outcome association, based on patients with these large lesions, would therefore lead to the false inference that the cause of the impairment in these patients was damage to Region B, i.e. the site with 100% overlap, when in fact the impairment was variably caused by damage to A, C or the connections between these regions.

P5 Solutions

P5_S1) When a region is identified where damage is consistently associated with impaired performance, the predictive value of this region needs to be checked by investigating the effect of selective damage to the region in patients with focal rather than large lesions. If focal lesions result in an impairment, then we can predict that damage to the identified region causes an impairment irrespective of lesion size; and test this prediction in future patients. In contrast, if focal lesions do not impair performance then the functional impairment observed in patients with larger lesions might be better predicted by the combination of regions that have been damaged (see Figure 2 above and Problem 6 below). For instance, a recent study of patients with vertigo found an 80% lesion overlap in a core region of the retroinsular vestibular network (Dieterich and Brandt, 2015). However, focal lesions to the region of overlap were not consistently associated with vertigo. Instead, vertigo was better predicted by a combination of regions along the vestibular network and the degree of inter-hemispheric connectivity (Brandt et al., 2014).

Note that including lesion size as a nuisance regressor in a lesion-outcome analysis (Karnath et al., 2004; Schwartz et al., 2012; Kümmerer et al., 2013) may reduce the impact of large lesions on the results but does not directly test whether focal damage to a lesion-deficit association causes an impairment.

Problem 6 (P6): Large lesions have inconsistent outcomes

For some functional impairments, the size of the lesion might be the best predictor of outcome but, when the effect of lesion size is inconsistent, we need to understand why some patients with large lesions have the impairment while others do not. As illustrated in Figure 2 and 3, and discussed above, the effect of large lesions can be explained by the fact that they are more likely to knock out all potential recovery systems. An understanding of the underlying neural networks (i.e. a model-based account) would help to constrain our interpretation of which regions need to be preserved (see Problem 10 below). In the absence of this knowledge, the following data-led solutions may be informative.

P6 Solutions

P6_S1) Large lesions that do and do not cause the impairment of interest can be compared to determine whether those causing an impairment have damaged a different combination of regions.

P6_S2) The sample could be expanded to include patients who have focal damage to all, or different combinations of, parts of the large lesions that cause impairments. In the absence of such data, it remains plausible that untested parts of the large lesion caused the impairment.

P6_S3) It is also important to assess patients with focal lesions in the early stages of their recovery. Evidence that focal lesions cause impairments early post stroke with subsequent recovery would indicate functional reorganisation (i.e. alternative pathways take over) that might not be possible when large lesions damage all potential recovery systems (see Figure 2). By collecting evidence of which brain regions caused a temporary deficit, the critical components of large lesions might be revealed; and predictions as to which large lesions will cause a persistent deficit and which will not may become more apparent.

Problem 7 (P7): Multiple lesion sites cause the same impairment

When a function requires multiple brain regions (i.e. a distributed system), then damage to any one of these regions might cause an impairment resulting in variability, rather than consistency, in lesion-outcome associations. For example, in the hypothetic neural system illustrated in Figure 2, patients with damage to A or C would all have a functional impairment but, when grouped together, there would be low inter-patient consistency in the lesion-outcome association particularly when assessed with univariate models. Low consistency within a group would also limit detectable differences between patients with different symptoms (see Rorden and Karnath, 2004).

P7 Solutions

P7_S1) Patients with the functional impairment of interest need to be categorised according to their lesion site. One possible solution would be to select patients who have the smallest and most distinct lesions, then group all the other patients with the functional impairment according to the degree of damage to each of the identified regions; and look for the most consistent lesion-outcome associations that might be predictive in future patients.

Having grouped the patients with the same functional impairment, according to their lesion site, further analyses of their functional impairment might reveal subtle behavioural differences that were not otherwise observed.

Subsequent studies would then be sensitive to the behavioural differences noted for different regions. Indeed, lesion analyses will be more precise when the functions of interest are closest to the underlying computations of different brain regions. We acknowledge, however, that it can be difficult to define the function of individual regions, especially when they only contribute in combination with many other regions in distributed neural systems (i.e. the function arises from the interactions between multiple brain regions).

P7_S2) A more pragmatic solution is to incorporate information from multiple voxels into machine learning algorithms that generate predictive multivariate models of the data (Hope et al., 2013; Smith et al., 2013; Yang et al., 2014). We consider these approaches further in relation to Problem 9 below.

Problem 8 (P8): Inter-patient variability in the effect of time post stroke

The assumption in the above discussion is that time-post stroke is one of the variables that is carefully controlled when identifying consistent lesion-outcome associations. It is important to control time post-stroke because neural plasticity and functional reorganisation are known to persist for years after stroke (see Hope et al., 2013). On the other hand, it is also essential to understand the effect of time-post stroke so that we can (i) identify lesion sites that cause short term impairments (i.e. lesions that patients recover from), (ii) estimate the speed of expected recovery after stroke which could be used to (iii) guide treatment planning, and (iv) provide baseline measurements from which to assess the effect of a treatment.

Measuring the influence of time post-stroke on residual structure-function relationships can make lesion analyses extremely complicated because the structure-function relationships may evolve in both short term (days) and long term (years) time frames. This requires a multifactorial design to investigate how multiple components (lesion sites, behavioural outcomes and time points post-stroke) are interacting and how these relationships are affected by the degree of therapy and intervention.

In addition, the effect of time post-stroke on lesion-outcome associations will be affected by inter-patient variability in the neural pathways that patients preferred to use pre-stroke. Conceptually, this can be illustrated by considering individual differences in using the left or right hand for writing. A lesion to the motor system controlling the right hand will have a more devastating effect in right-handed than left-handed subjects. Conversely, a lesion affecting the motor system controlling the left hand will have a more devastating effect on left handed than right handed subjects. Moreover, within cohort, the speed of recovery will depend on how quickly the individual can learn to use their non-dominant hand.

A second example relates to how healthy controls vary in their preference for using one of two possible neural pathways for reading aloud. In Seghier and Price (2010), we found that some participants used a pathway that was routed via the left putamen and others used a pathway that was independent of the left putamen. The implications are that left putamen damage will have a different effect on reading aloud in those who do and don’t use the left putamen pathway. In those that don’t use the left putamen pathway, left putamen damage may have minimal effect because their preferred pathway isn’t damaged. In those who do use the left putamen pathway, left putamen damage is likely to have an initial impact on reading aloud (because the neural pathway they usually use is damaged) but reading aloud should resolve when the patient learns to use the alternative pathway (if it is preserved). The speed of relearning may also depend on many things including prior experience using the alternative pathway and the degree of intervention.

In summary, when there are alternative pathways for the same task, there will be inter-patient variability at the time of the stroke; and also in the time course of recovery. We would expect, however, that inter-patient variability will decrease as time post stroke increases because, with time, those that are slow using the alternative pathway may eventually catch up with those that were quick to use the alternative pathway.

P8 Solutions

P8_S1) Time post-stroke should be carefully considered when investigating the consistency of lesion-outcome associations across patients. Consistent impairments are most likely to be observed in patients who are early post-stroke. Consistently unimpaired performance is most likely to be observed in patients who are later post-stroke.

P8_S2) Hypotheses about the time course of recovery in different patients need to be validated by longitudinal assessments of the same individuals.

Problem 9 (P9): Outcome depends on a complex mix of variables

Despite our best efforts, we may not find consistent lesion-outcome associations because there are simply too many factors to control or there are complex linear and nonlinear interactions between different variables that are not easy to formalise, particularly when they additionally interact with time post-stroke (i.e. spatiotemporal interactions).

P9 Solution

P9_S1 Machine learning algorithms can play an invaluable role in helping to resolve what might seem like incomputable complexity. These approaches simultaneously consider data from multiple variables (e.g. tissue integrity in multiple spatially distributed brain regions, time-post stroke, age, gender) to find the combination of variables that best predict an outcome, even if we have very little insight into how the predictions are generated. The accuracy, sensitivity and specificity of the predictions can then be established by cross-validation in new samples of patients (Chen et al., 2008; 2015; Hope et al., 2014; Smith et al., 2013; Zhang et al., 2014).

P9_S2) In future, machine learning algorithms could assess and use information from many other variables that may be indicative of functional outcome. For example, metabolic, demographic and mechanistic markers of recovery potential could be included (Coupar et al., 2012; Lee et al., 2015; Shiban et al., 2015; Watila and Babarabe, 2015), or severity of the initial impairment (Coupar et al., 2012; El Hachioui et al. 2013), or the hours or type of therapy. Predictions of outcome from lesion site could also be combined with measures of functional/effective connectivity across networks of brain regions that have also been shown to predict outcome after brain injury (Warren et al. 2014).

The more variables are included, the more accurate the predictions are likely to be but there is a high price to pay for these advances because they will progressively require larger and larger sample sizes (see P4_S1 above). As an illustration, if we have a model with 10 regions, there are 45 possible pairs of regions and 120 triplets. Understanding and replicating all possible combinations would therefore require thousands of patients, unless the analyses were constrained by model-based hypotheses.

Problem 10 (P10): Understanding and improving the predictions

In the Introduction to this paper, we made a distinction between model-based lesion analyses and data-led lesion analyses, arguing that predictions could be made on the basis of data-led observations without understanding the underlying neurological model. Here we consider how the two approaches could be integrated to provide better predictions.

Solutions

P10_S1) Working backwards, if machine learning algorithms (P9_S1) generate accurate predictions, their outputs and computations can be examined to understand the relative contribution of specific variables and how these variables interact with one another to determine outcome after brain damage. Machine learning algorithms also allow us establish lesion-deficit associations across multiple (cognitive) domains (Corbetta et al., 2015).

P10_S2) To understand the combinations of regions that are critically damaged in patients with large lesions (P7), we need to compare the effect of damage to (i) each part and (ii) the combinations of parts. Once the effects are understood, we can generate hypotheses to test in new patients. For example, if a lesion site had two parts A and B, then the critical part could either be: A-only, B-only, either A or B or A&B together. This leads to four testable hypotheses. First, if only part A is important, then task difficulty will be observed after damage to A-only but not B-only. Second, if only part B is important, then task difficulty will be observed after damage to B-only but not A-only. Third, if A & B are both parts of the same pathway, then task difficulty will be observed irrespective of whether damage is to A-only, B-only or A&B. Fourth, if the full region of interest contains 2 independent pathways (A&B), then damage to A-only or B-only should not affect task performance.

P10_S3) To understand which part of a lesion is contributing to a function, we can cross-validate the results of lesion analyses with the results of functional imaging (fMRI), transcranial magnetic stimulation (TMS) and diffusor tensor imaging (DTI) studies on healthy participants.

Discussion

We have identified a set of ten problems associated with finding and understanding lesion-outcome associations that are consistent and predictive for individual patients. These problems and some possible solutions are listed in Box 4 but note that there isn’t a one to one relationship between the 10 problems and 10 solutions because the same solutions can apply to multiple problems. Moreover, some of the problems (P1-P3) are not essential to solve when the aim is purely to make accurate data-led predictions rather than to understand the neural networks associated with the functions of interest.

Box 4. Summary of problems and solutions.

Ten problems with lesion-outcome analyses

P1 Impairments from undamaged but disconnected regions

P2 Damage to the vascular system not the neural system

P3 Damage to the same region impairs different functions.

P4 Seeing past inter-patient variability

P5 Large lesions dominate lesion-outcome associations

P6 Large lesions have inconsistent outcomes

P7 Multiple lesion sites cause the same impairment.

P8 Inter-patient variability in the effect of time post stroke.

P9 Outcome depends on a complex mix of variables

P10 Understanding and improving the predictions

The most important solutions

1 Very large cohorts of patients, with varying outcomes

2 Standardised assessments for all patients

3 Functions of interest that are physiologically likely.

4 Tight control of as many variables as possible

5 Compare the effects of large and small focal lesions

6 Investigate short and long term effects of time post stroke

7 Longitudinal assessments of the same individuals

8 Multivariate lesion analyses

9 Machine learning algorithms

10 Cross-validation with fMRI, TMS and DTI.

We have argued that to predict how an individual will be affected by a stroke we first need to understand and control multiple factors that might influence whether, and to what degree, a function of interest is affected. Controlling for as many sources of inter-patient variability as possible should reveal consistent lesion-outcome associations for some lesion sites even if other lesion sites have more variable outcomes. Consistent lesion-outcome mappings need to be validated with longitudinally collected data from patients with focal lesions. Patients with large lesions can further indicate whether the effect of damage to multiple regions (within the large stroke) is additive, super-additive or has no additional effect compared to the effect of each of the focal lesions (Figure 2). When a consistent lesion-outcome association is identified, new patients can be grouped according to lesion site with the aim of predicting their outcome and its change with time post stroke.

When it is impossible to model complex relationships between multiple variables contributing to outcome, multivariate lesion analyses and machine learning algorithms can be used to identify the most influential variables and generate the most predictive models of all sources of data (Hope et al., 2013). The results of these data-led analyses can be used to provide a better understanding of the most predictive variables; and to update model-based accounts of the neural systems that can support the task of interest. Indeed, although we have argued that data-led analyses are sufficient to make predictions about future outcomes, predictions will improve greatly in future when we have a better understanding of the neural networks underlying sensorimotor and cognitive functions. This in turn will require the results of lesion studies to be integrated with the results of fMRI, MEG, DTI and TMS studies of healthy and brain damaged participants.

Below we discuss how complementary information provided by model-based and data-led accounts can be used to validate each other and provide richer more informative and predictive models of brain function.

Model based accounts, validated and informed by new data

A clinically useful model-based account must be able to predict new data. For instance, when a model-based account assumes only one necessary processing pathway for a task, then its prediction would be that damage to this pathway will impair function. Conversely, if a model-based account predicts the existence of two alternative processing pathways for a given function, it predicts that task performance will be better when only one pathway is damaged than if all pathways are damaged (see Figure 2).

If new data do not confirm the hypotheses, the model needs to be updated. However, it is not always intuitively clear what changes need to be made or how they should be implemented. For example, if patients can perform a task despite damage to the neural system modelled for that task, then there must be unknown ways of performing the task. Finding these alternative neural pathways can be difficult especially if the mechanisms for recovery are inconsistent across patients with similar lesion sites. We describe below how data-led analyses of inter-subject variability can provide fresh insights into how the explanatory model can be updated. (Airan et al., 2016; Dubois and Adolphs, 2016; Kherif et al., 2009; Seghier and Price, 2009; 2016).

Examples of prior studies that have demonstrated inter-subject variability in the neuronal systems underlying a single cognitive task include repeated demonstrations that there are many ways to read the familiar words and that individual subjects differ in their reading strategies and skills (Jobard et al., 2011; Kherif et al., 2009). This translates into the differential involvement of different reading pathways (Hoffman et al., 2015; Richardson et al., 2011; Seghier et al., 2008; Yvert et al., 2012) depending on individual’s strategy and preference. For example, subjects who rely more on semantic mediation during reading have greater structural connectivity between semantic and phonological nodes in the reading network (Graves et al., 2014) and greater functional involvement of the anterior temporal lobe (Hoffman et al., 2015). The impact of damage to the same region is therefore expected to result in individual differences in outcome and recovery trajectory, depending on the emphasis each patient placed on the damaged areas pre-stroke.

By monitoring the capacity and speed of recovery in patients with damage to the normal system, inferences could perhaps also be made about the preferred (dominant) neural pathway that subjects relied on before their stroke. For example, those that recover fast are likely to have an alternative pathway that is already capable of performing the task, while those that recover slowly may not have prior experience using an alternative pathway. This can be appreciated by analogy to hand-dominance. Right handed individuals with damage to the system controlling their right hand will be quicker to recover the ability to write if they already have the potential to write with their left hand. If we were able to find lesion or demographic clues to indicate how an individual patient was going to recover, we would be able to indicate the type of rehabilitation that could speed up that recovery.

Data-led accounts informed by model-based accounts

Data-led accounts are useful for identifying predictive relationships and the major sources of inter-subject variability (Hope et al., 2013). For example, a data-led, machine learning analysis might reveal that damage to a set of regions consistently impairs performance on a task of interest or that preservation of a set of regions consistently preserves task performance. These predictions are useful for giving clinical predictions but they do not explain how or why damage is affecting task performance. To explain the data-led account, we can make and test hypotheses based on model-based accounts. For example, we can hypothesize and test how the regions identified by the data-led account respond during normal task performance, how they connect to one another and what the potential processing pathways might be. This is a challenging endeavour and we consider some of the possible methodological approaches below.

Methodological approaches for constructing model based accounts

Neural systems, for tasks of interest, are typically identified in healthy subjects in terms of both the set of regions activated (whole brain activation maps) and task related directional functional connectivity measures (neural pathways) that indicate how one brain region is influenced by activity in other brain regions (i.e. how activity is propagated through the system). These inter-regional interactions can be estimated using different connectivity techniques (Li et al., 2009; Sakkalis, 2011). One of the widely used connectivity techniques in functional neuroimaging is dynamic causal modelling (DCM, (Friston et al., 2003)), which provides the opportunity to estimate how the rate of change of activity in one region influences the rate of change in other regions (Friston et al., 2003; Seghier et al., 2010). This in turn leads to information about the direction of the influence one brain region may have on another rather than implying a non-directional correlation. DCM can also indicate whether a region is having an excitatory or inhibitory effect on another, and the excitatory/inhibitory balance and neuromodulatory effects within a region (Bhatt et al., 2016). Such information might be invaluable for understanding the mechanisms of recovery after stroke and how this might be influenced by interventions.

Having established the most likely neural pathways and neuromodulatory effects in healthy controls, and quantified the degree of normal inter-subject variability, DCM can be used in patients to investigate the degree of abnormality induced by the stroke. For example, a patient who has retained or recovered the ability to perform a task following damage to a normal neural pathway would be expected to show decreased activity and connectivity in the damaged pathway but increased activity and connectivity in another pathway that was also capable of supporting the task. This would suggest that one pathway could compensate for another; and this hypothesis could be tested by looking to see whether healthy controls show a trade-off between alternative pathways (i.e. strength of connectivity in one being inversely correlated with the strength of connectivity in another), see (Seghier et al., 2014; Seghier et al., 2012).

Alternative neural pathways (nodes plus connections) identified in patients might be missing from the task model because they were not consistently observed in normal subjects. In this scenario, the patient findings motivate further investigation of healthy participants to test whether the nodes and connections identified in the patient are observed in a subset of the healthy participants (Seghier et al., 2012; 2014). Put another way, DCM in patients may help to identify meaningful variability in the normal population that could reflect the use of different cognitive strategies and different processing pathways (Miller et al., 2012). Models of the neural systems underlying a task can then explicitly include these alternative pathways so that they can make more accurate predictions for new patients (Park and Friston, 2013). This long term goal is likely to require DCM investigations into very large cohorts of healthy participants and patients but could ultimately bring us closer to understanding all the possible neural pathways that can support recovery of task function after stroke.

Systematically increasing the predictive power of existing model-based accounts will benefit from the incorporation of as much prior information as possible. For instance, informed modelling with respect to lesion information (Seghier et al., 2012; Zaghlool and Wyatt, 2014), physiology (Havlicek et al., 2015), anatomical connectivity (Stephan et al., 2009) or functional connectivity (Carter et al., 2012) will help to develop useful and credible models. The recent developments in DCM for inverting larger networks (Seghier and Friston, 2013) and for comparing between models and groups (Friston et al., 2016) will open new possibilities to understand the relationships between structure, function, and outcome and the neural mechanisms that support recovery after stroke (Rigoux and Daunizeau, 2015).