Challenges in the use of treatment to investigate cognition

Abstract

The use of data from people with cognitive impairments to inform theories of cognition is an established methodology, particularly in the field of cognitive neuropsychology. However, it is less well-known that studies that aim to improve cognitive functioning using treatment can also inform our understanding of cognition. This paper discusses a range of challenges researchers face when testing theories of cognition and particularly when using treatment as a tool for doing so. It highlights the strengths of treatment methodology for testing causal relations and additionally discusses how generalisation of treatment effects can shed light on the nature of cognitive representations and processes. These points are illustrated using examples from the Special Issue of Cognitive Neuropsychology entitled Treatment as a tool for investigating cognition.

It is a core aim of cognitive neuropsychology to inform theories of cognitive processes and to use data from individuals with cognitive impairments to do so. Nickels, Kohnen and Biedermann (2010) argued that, in addition to the study of patterns of intact and impaired cognitive processes for testing and extending cognitive theories, treatment¹ can also be a powerful tool to achieve these goals. The basic logic behind this enterprise is that the patterns of improvement seen as a result of treatment will differ according to the way knowledge is represented and processed. Hence, treatment can be used to develop and test theories. When successful, this type of research can be an extremely exciting “win/win” approach, achieving both therapeutic and theoretical goals. This issue of Cognitive Neuropsychology is devoted to papers that attempt to do just that. To be included:

1. Articles must report an intervention (or interventions) with design and analysis that allow the interpretation that the results are attributable to the treatment (rather than to potentially confounding factors like spontaneous recovery or placebo effects); and

2. The results of intervention must inform theoretical debate regarding the nature of cognitive representation and processing. Moreover, this contribution needs to be specific, a general statement of broader implications is not sufficient.

Thus, articles included here were restricted to those in which intervention was used to test and/or extend theories of ‘normal’ cognitive function. This is far from an easy task. In this paper we address some of the challenges that face researchers using this approach to theory development. First we focus on those challenges that are not unique to intervention - challenges encountered in some form by all theory testing, while emphasising the particular characteristics for intervention methodology. Subsequently we discuss two key approaches to using intervention to test theories - examination of patterns of generalisation (across items or across tasks) and testing predicted correlations between impairment and response to intervention across a case series.

Challenges in testing theories of cognition

1. The problem of prediction

In order to test a theory of a cognitive process, one has to be able to understand how that theory predicts the process will respond under different conditions. For example, how the human face processing system responds to faces with and without visible external features (hair, facial contours), or how the language system responds to words that differ in frequency. In addition, for the cognitive neuropsychologist one of these different conditions is the impairment itself (Caramazza, 1986). Furthermore, when intervention is involved, there also needs to be a clear understanding of how intervention will impact the impaired (and spared) cognitive processes. Therefore, in this type of research there is a complex set of conditions placed on the cognitive processes of interest and, as a result, prediction is correspondingly complex.

Specification of theory

As noted by Coltheart, Bates and Castles (1994) there is the possibility of theory under-specification, from the description of the relevant processes and representations through the description of the consequences of impairment as well as the effects and mechanisms of intervention. First, theories may not be explicit and precise about the content of representations and the operation of processes. While computational modelling has increased specificity compared to many ‘verbal' models, computational models (usually by design) do not capture the full complexity of the human language system, which may place limits on prediction. For example, the model of Dell and colleagues (Dell, Schwartz, Martin, Saffran & Gagnon, 1997) has been highly influential and informative in our understanding of spoken word production, yet it remains a model that simulates word production for only 5 or 6 monosyllabic words. It is currently unclear what would be predicted for a larger vocabulary that included, for example, polysyllabic or morphologically complex words.

While Dell (2004) notes that computational models are 'by definition simpler than the theories they represent’ (p28), some computational models are larger in scale. For example, the Dual Route Cascaded (DRC) model of Coltheart and colleagues (Coltheart, Rastle, Perry, Langdon & Ziegler, 2001), incorporates a close approximation to a 'full' reading vocabulary by including every (monosyllabic) word in a frequency database. Nevertheless, DRC has no representation of word meanings, and hence remains unable to simulate word comprehension, spoken picture naming, repetition or writing. Hence the interactions between different modalities of the language system remain underspecified and once we go beyond the single word to other aspects of the language system at the phrase, clause, sentence, or discourse levels under-specification can also be an issue.

This underspecification is problematic when the theories lack the specificity required to interpret the results of the experiments that researchers undertake. This situation underscores the point that experiments are best designed to address issues at a level of granularity/specificity that is commensurate with the theories under examination (Rapp & Goldrick, 2000). Of course, this is a challenge for cognitive psychology in general rather than just studies that employ intervention methodologies.

Specification of impairment

Understanding the predicted consequences of impairments to cognitive processes can be extremely complex. And while this has been successfully carried out in cognitive neuropsychology without computer simulation, as theories become more detailed and processes more interactive it becomes increasing difficult to predict the effects of impairment on performance without computer simulation. At that point there are various challenges. First is the practical problem that few computational models are accessible to those without programming skills (for a notable exception see http://langprod.cogsci.illinois.edu/cgi-bin/webfit.cgi). Second, only some computational models are easily lesioned to simulate impairments, by, for example, increasing levels of noise or rate of decay, while others lack this feature.

Attempts to simulate impairment also illustrate the limitations of computational modelling so far. For example, using lesions to the DRC model, Nickels, Biedermann, Coltheart, Saunders and Tree (2008) successfully simulated the nonword reading accuracy patterns of three individuals with acquired phonological dyslexia. However, these simulations could not replicate the qualitative nature of the errors, nor some patterns of word reading impairment in these same individuals. Furthermore, it is worth noting that authors of some influential computational models explicitly reject the simulation of impairment. For example, regarding the WEAVER++ implementation of Levelt, Roelofs and Meyer's influential theory of spoken word production, Levelt et al. (1999b, p68) state that “we feel it as (sic) a bridge too far to expect a patient’s behaviour to conform to our theory”, continuing “there is little a priori reason to suppose that an impaired system performs according to an intact theory” (although see, Roelofs, 2004).

Being unable to fully specify the impairment within a computational model does not mean that it is not possible to conduct cognitive neuropsychological studies to investigate cognitive processes using impairments or interventions (see, for example, Nickels & Howard, 2000) and simulations that have examined the consequences of lesioning and attempted to replicate patterns of impaired behaviour have been informative in both testing cognitive theories and informing our understanding of cognitive impairment (e.g. Harm & Seidenberg, 2001; Nickels et al., 2008; Plaut & Shallice, 1993; Rapp & Goldrick, 2000; Schwartz, Dell, Martin, Gahl & Sobel, 2006).

Specifying the mechanisms and consequences of intervention

There has long been awareness that there is a need for a 'theory of therapy' (e.g. Howard & Hatfield, 1987; Hillis, 1993). In particular, to be able to use intervention to inform theory, we need to understand the mechanism(s) by which a treatment has its effects: what are the changes to the processes and representations that have been brought about as a result of this intervention? Are normal learning mechanisms and principles engaged in “re-” learning? Can damaged networks be repaired, or must they be recreated? In general, the consensus in the literature is that our knowledge is very limited in this respect.

There have been occasional attempts to look at the effects of treatment and recovery on lesioned computational models (e.g. Martin, Saffran & Dell, 1996; Plaut, 1996). For example, Plaut (1996) investigated the effects of retraining a lesioned model of word reading (activation of semantics from orthography). However, Plaut himself notes that the results of the simulation could not be directly applied to 'real life' treatment, as the task used to examine performance in the computational model was much simpler than the tasks performed by patients (Plaut, 1999). Moreover, there is a danger when attempting to generalise the results of computational modelling in one domain (e.g. accessing meaning from print) to another domain (e.g. picture naming), as different factors may influence behaviour in the two domains. Nonetheless, computational approaches have been used with some success to find principles governing the relearning or re-accessing of information during the course of treatment.

Best et al. (2015, this issue) provide an example of the utility of computational modelling. They developed a simple computational model which aimed to simulate the development of word retrieval in both typically developing children and children with word-finding difficulties. This computational model linked representations between a phonological processing component (with phonological input and phonological output) and a semantic processing component (with semantic input and semantic output). Each component was modelled using a three-layer artificial neural network trained with the backpropagation algorithm (Rumelhart, Hinton, & Williams, 1986) so that the component would reproduce the code applied to its input layer onto its output layer via a layer of hidden units. The semantic and phonological components were then linked so that the combination of mappings between input (semantic or phonological) and output (semantic or phonological) could simulate performance on four tasks: picture naming, conceptual (picture) association, nonword repetition and word-picture matching.

Having first demonstrated that the model could simulate typical development, Best et al. then used different lesions to the computational model to simulate the deficits of the two children with word finding difficulties. Importantly, none of the simulations involving a single type and locus of lesion could successfully simulate the patterns of these two children, two loci of deficit (to the starting state of the model) were required to simulate one child’s (Amy's) performance and it required three lesions to come close to simulating the other child’s (Magda's) performance. Finally, the lesioned models were given two types of simulated behavioural interventions (phonological and semantic). This allowed Best et al. to generate predictions regarding how the model behaved following intervention of each type. The model could then be tested by comparing these predictions with the actual results from intervention with the two children. For the child (Amy) where the deficit was simulated with two lesions, the results were as predicted: the phonological intervention was effective, and the semantic intervention was not. For the second child (Magda), the simulated effects of intervention differed depending on the nature of simulated damage to the model (three different combinations of three lesions were suggested to successfully simulate her impairment), and each predicted different responses to intervention. The behavioural study found an effect of semantic but not phonological intervention, which was indeed the pattern predicted by one of the lesion combinations.

While, as is common, the computational model employed in this example, cannot be taken to be a 'full scale' model of the acquisition process, the computational modelling was valuable in enabling clearer predictions to be made regarding the expected interaction between type of impairment and type of treatment. As Best et al. note, the fact that the model failed to predict the results of intervention for a particular impairment type, means that the model cannot be correct. Consequently, it needs to be determined whether this failure is a result of the simplification of the model as a representation of the theory, or the simulated intervention as a representation of the treatment, or whether, instead, the underlying theory needs to be changed. Despite interpretative uncertainties, the potential benefits make this a worthwhile enterprise – through cyclic model building and testing, intervention research should result in a stronger and more detailed a theory of spoken production that can account for the impaired language system of children with word finding difficulties.

While there have been few direct attempts to develop theories of the mechanisms underlying improvements in performance following cognitive impairment, learning theories developed on the basis of neurotypical research participants have been used when describing treatment related gains. For example, in the treatment of anomia, some authors have claimed that the mechanisms underlying treatment gains are those employed in Hebbian learning (e.g. Lambon Ralph & Fillingham, 2007) or repetition priming (Howard, Hickin, Redmond, Clark & Best, 2006; Laine & Martin, 2006; Nickels, 2002b). This is undoubtedly a fruitful direction for research as the fundamental mechanisms of learning are likely shared in intact and damaged brains. However, there may be some differences that arise even at the level of fundamental mechanisms. Certainly at a higher level of description, caution may need to be exercised in generalising from intact to impaired processes, from mechanisms of learning to those involved in “relearning”. For example, in which specific ways is treatment for anomia similar to new word learning in unimpaired subjects and in which ways is it different?

Summary: The challenges of making testable predictions

Because theory development is a work in progress, we have underspecified theories, leading to difficulties in specifying the effects of impairment on processing and even greater difficulties specifying how treatment may affect the impaired processing. As a result, current theories may not be clear about the ways in which a cognitive system can be expected to behave after an acquired impairment of cognition and even less clear about mechanisms of recovery from such impairments either spontaneously or as a result of rehabilitation (Coltheart, Bates & Castles, 1994). While the goal may be to use treatment to develop theories the enterprise relies heavily on theoretical foundations in order to make specific predictions.

Computational models, where they exist, aid in specification at all three of these aspects (processing/representation, impairment and mechanisms of treatment). Dell (2004) notes that cognitive neuropsychology benefits from a variety of models including both informal information-processing models (verbal/descriptive models) and computational models. He also comments that computational models have much to offer "once we get past both our fears that the models are flawed and our beliefs that they are flawless" (Dell, 2004, p29). However, it is critical, for research using intervention to test theoretical claims regarding cognition, to be clear about current limitations that may derive from theory under-specification and how they impact hypothesis testing, experimental design and interpretation of potential findings.

2. Sound experimental design

First, of course, it is critical to have an experimental design that is appropriate for testing the theoretical predictions to be evaluated.

Second, one has to be sure that the results are reliable and not due to experimental error: Intervention studies have particular requirements in this regard. Given the heterogeneity of cognitive disorders, many in the field believe that single case intervention studies are preferable to group studies (e.g. randomised controlled trials; e.g., Hegde, 2007; Howard, 1986). However, as with group studies, single case intervention studies have stringent design requirements in order to demonstrate that any improvement is due to the effects of treatment rather than any other factor (e.g., developmental changes, spontaneous recovery, practice effects, general improvements in mood). There is no one universally preferred design and which design is chosen will be affected by several factors. These include participant characteristics, the type of treatment and the anticipated response to treatment (Beeson, 2015; Nickels, Best & Howard, 2015) but also the opinions of the researcher regarding the central elements of the design (cf, for example, Howard, Best & Nickels (2015) and Thompson (2006, 2015))². What is agreed is that there needs to be measurement of performance prior to treatment, usually through testing on a number of occasions to enable estimation of the extent of variability of this 'baseline level'. For example, Best et al. (2015, this issue) and Keane and Kiran (2015, this issue) both tested naming on three occasions prior to the onset of treatment.

For treatment to be shown to be effective, following the treatment phase the level of performance needs to be demonstrably higher than at the pre-therapy baseline, and the change clearly attributable to treatment. How the latter is achieved differs, but can include (untreated) control items or control tasks, and/or multiple phases of treatment replicating the effects across sets (see Nickels et al., 2015, for discussion). For example, Best et al. (2015, this issue) in their treatment of two children's word finding deficits, used two sets of untreated control items – one that was tested before and after treatment and one that was named (but not treated) in every treatment session, to control for effects of exposure/repeated testing (Nickels, 2002a; Rapp & Kane, 2002). The fact that these two sets did not improve provided evidence that the improvement of treated items was specific to the effects of treatment. In contrast, Smith-Lock (2015, this issue) aimed to teach knowledge of a rule (past tense formation) and an improvement of all past-tense forms (trained and untrained) was predicted by one of the theories under investigation. Consequently, treatment effects were examined on an untreated set of past-tense forms while experimental control (to exclude the possibility of nonspecific effects of therapy - ‘charm effects’) was gained through testing another untreated morpho-syntactic rule (third person singular).

A challenge often faced in intervention studies occurs when participants make gains during the pre-training baseline period. In this case, the experimenter has to show that treatment leads to more improvement than observed during the no-training baseline period. Multiple studies in this Special Issue had to deal with this scenario (Banales et al., 2015, this issue; Smith-Lock, 2015, this issue) and used statistical techniques to demonstrate that treatment indeed generated greater change than was evident during non-treatment phases (see Howard et al., 2015, for an example of a suitable statistical technique).

Additionally, most authors now agree that data from single case studies of intervention needs to be statistically analysed (e.g. Beeson, 2015; Howard et al., 2015), as was the case in all the studies in this issue. Statistical analysis is the norm in the broader field of experimental cognitive psychology. However, in the past there has been a tradition of using visual analysis alone to interpret the results of intervention, a method which has been strongly criticised as leading to high rates of both type I errors (concluding an effect is present when it does not exist) and type II errors (concluding there is no effect when one is present) (e.g. Matyas & Greenwood, 1990). While ‘pure’ visual analysis is now almost universally disfavoured, there have been more recent proposals suggesting quasi-statistical techniques to assist visual analysis (e.g., Brossart, Vannest, Davis, & Patience, 2014; Lane & Gast, 2014). Similarly, which statistical techniques are most appropriate for the analysis of the effects of intervention also remains under debate (see, for example, Beeson, 2015; Fischer-Baum, 2015; Howard et al., 2015; Laganaro, 2015; Nickels et al., 2015; Willmes, 2015).

Summary: Challenges in the testing of theories of cognitive processes

Intervention studies face the same challenges as experimental cognitive psychology more broadly: the researcher needs to derive clear predictions from theories, design sound experiments and draw appropriate conclusions. However, in nearly all of these areas, intervention studies have greater challenges than experimental studies using unimpaired subjects: theories are less well specified when it comes to impairment and response to intervention, design principles more contentious and more complex, and potential alternative accounts for results are more frequent. Nevertheless, this methodology has an important role to play as it may provide unique opportunities to address certain types of questions. For example, as we discuss below, it is likely the strongest tool for the investigation of causal claims.

Using intervention to test cognitive theories

Nickels et al. (2010) noted that while few studies in the literature have explicitly tested theoretical accounts using intervention as the methodology of choice, many intervention studies have contributed to theoretical advances by discussing the theoretical implications of their findings. For this Special Issue, authors took one of the two standard approaches to theory building. Either they used intervention to distinguish between two different theoretical positions (e.g. Schröder et al., 2015, this issue; Smith-Lock, 2015, this issue; Banales, Kohnen and McArthur, 2015, this issue) or they used a confirmatory approach to determine whether the outcome of intervention was consistent with the predictions of a particular theory (e.g.; Best et al., 2015, this issue; Keane & Kiran, 2015, this issue).

Whether comparing two different theoretical positions or examining the predictions of a single theory, there are two major types of questions that intervention-based studies have most commonly been used to address: causal mechanisms of cognitive processing/development and questions concerning the content and organization of cognitive processes/representations. Papers within this special issue represent examples of each (e.g., causal mechanisms: Banales et al., 2015, this issue; Best et al., 2015, this issue; Keane & Kiran, 2015, this issue; processes/representations: Schröder et al., 2015, this issue; Smith-Lock et al., 2015, this issue).

Treatment used to investigate questions of causality

As is well known, many hypotheses of causality are proposed on the basis of suggestive correlation evidence. Some causal hypotheses concern the causal processing chain in the adult cognitive system, others concern causality in the acquisition/development of a cognitive skill. One example regarding the adult system concerns the relationship between phonological short term memory and sentence processing. Based on the observed association between these two deficits (e.g., Saffran & Marin, 1975) or abilities (e.g. Just & Carpenter, 1992), there has been considerable debate regarding whether or not sentence processing actually relies on phonological working memory (e.g., Butterworth, Campbell & Howard, 1986; Caplan & Waters, 1999; Martin, 1987; Vallar & Baddeley, 1984). The alternative hypothesis is that they are not causally related, but rather that it is an accidental relationship (e.g., in cases of damage, the two skills are supported by nearby brain tissue or, that both are independent of one another but causally dependent on a third skill). In terms of causal hypotheses in the area of development, an example is the claim that phonological short term memory plays a causal role in vocabulary acquisition (e.g. Baddeley, Papagno & Vallar, 1988), such that individuals with poor phonological short term memory will have difficulty with word learning although once words are acquired, phonological short term memory need not play a role in the use of the vocabulary in speaking. In either case, intervention provides one of the few methodologies available for explicitly testing the validity of these types of hypotheses. The logic in using intervention to test causality hypotheses is that if process A is causally related to process B, then improvements in A should lead to improvements in B. Positive findings provide support for the causal hypothesis although a failure to find improvement in B following improvement in A does not necessarily disprove the causal relationship as the failure could be due to a range of reasons that may not discredit the causal hypothesis.

In this special issue, Banales et al. (2015, this issue) used an intervention study to examine the well-documented association between poor verbal working memory and poor reading ability. They specifically tested whether or not there is a causal relationship between the two – whether poor verbal working memory disrupts reading acquisition or if poor reading affects the development of verbal working memory. Banales et al. found no evidence for a causal relationship between these two skills - improvements in reading did not lead to improvements in performance on working memory tasks, nor did improvement on working memory tasks lead to improvements in reading. This paper also raises several important questions which may apply to other intervention studies that follow a similar logic. As indicated earlier, null results in this approach are open to a variety of interpretations. For example, if there is a causal relationship, how large an improvement in one skill might be required to produce a significant improvement in the skill which relies on it? In this case, for the children who improved following working memory training, Banales et al. found improved performance on only one of the two working memory tasks. Does improved performance on this one task constitute sufficient improvement to influence reading performance? A second question concerns how soon after the “causal” skill has been improved should there be significant improvement in the acquisition of the dependent skill? For the children who show improved reading, should one expect an immediate or a delayed effect on their verbal working memory (and vice versa)? Inclusion of a delayed post-test (a ‘washout’ phase, e.g. Banales et al., 2015, this issue) is one way to examine this question.

Best et al. (2015, this issue) also examined causality, in their case examining the cause of word finding difficulties in children. They treated two children with word finding difficulties due to different hypothesised impairments. They administered two different treatments to both children and examined the nature of their responses to each treatment. Critically, as described above, they used computational modelling as the bridge between intervention and theory. The computational modelling enabled more precise theoretical predictions regarding how different impairments should respond to treatment. In turn the results of the treatment allowed these predictions to be tested.

Keane and Kiran (2015, this issue) also considered a question of causality but one that does not refer to acquisition. Rather their question concerned the causal role of non-linguistic cognitive control mechanisms in the flow of activation between the different lexical representations within a language and across different languages in the multilingual individual. Treatment was used in a very different way in this study. In word production in multi-lingual people without language impairment, a non-linguistic control system is hypothesised to be responsible for the activation of related items across languages during processing and also for their inhibition at the time of selection of a specific word for production. In addition, the control of activation within the lexical system has been proposed to play a role in cross-language generalisation effects of treatment. Therefore, the hypothesis that Keane and Kiran tested was that disruption to the control mechanism should cause disruption to generalisation of treatment effects to the untrained languages. They also considered additional predictions regarding consequences of damage to the control mechanism including that it should lead to increased rates of cross language intrusions following treatment. The authors found evidence consistent with this causal hypothesis: not only were there no generalisation effects, but following treatment in one language (e.g. French) there were many errors from that language when naming in the other language (e.g., English).

Treatment used to investigate cognitive organization, representation, and processing

Generalisation

A major tool for investigating theoretical accounts using intervention involves the study of patterns of generalisation: following successful treatment, untreated processes and representations may improve. Generalisation has been observed across modalities and across items and, in fact, treatment generalisation is predicted to be observed in a number of situations. We have already seen that treatment generalisation is expected when there are causal relationships between mechanisms and another important situation is one in which processes or representations are shared across tasks, modalities or items. In those situations, intervention can serve to test claims of shared vs. independent processes and representations.

Two papers in this special issue use this generalisation “logic” for theory testing (Schröder et al., 2015, this issue; Smith-Lock, 2015, this issue). Schröder et al. (2015, this issue) contrasted two hypotheses regarding sentence processing: The hypothesis that production and comprehension share processes or, alternatively, that they are independent. The former predicts generalisation of treatment effects across modalities while the latter does not. The fact that Schröder et al. found no generalisation from improvements in sentence production to comprehension of the same structures provided support for modality-specific cognitive processes for sentence comprehension and sentence production.

Rather than examining generalisation across domains or modalities, Smith-Lock examined generalisation across items in order to test the predictions of two different theories of past-tense formation in a treatment study with children with Specific Language Impairment. On the one hand, some theories assume a rule-based mechanism is used to generate regular past-tense forms, while on the other, there are theories that assume that children learn to generate the past-tense of individual items. The two accounts make different predictions regarding the effects of successful training on individual items, with the former, but not the latter, predicting generalisation to untrained items. Accordingly, Smith-Lock treated regular past tense forms using one set of verbs and tested whether there was generalisation to another set of untreated verbs. The fact that generalisation occurred provided evidence against theories positing that regular past tense forms are only learned for individual lexical items, and supported those proposing a general, rule-based mechanism for regular past tense formation, even in children with Specific Language Impairment.

Not surprisingly, generalisation methodology can also run into the challenges of interpreting null results. For example, Schröder et al.'s results were consistent with a modality-specific mechanisms account, given the absence of cross-modality generalisation of the treatment benefits –affirming the null hypothesis. However, as further support for the modality-specific account, Schröder et al. provided converging evidence from correlations in the patterns of impairment before treatment – where there was also no correlation between level of impairment in sentence production and sentence comprehension.

Correlations in case series

Another approach is to use a case series of treatment studies to test predicted correlations. Nickels et al. (2010) provide the example of using response to treatment for impairment of the phonological buffer/phoneme level to test theories that differ regarding whether or not both nonword reading and nonword repetition share this level of processing. If the level of processing is shared then there should be a correlation across cases between the degree of improvement in nonword reading and improvement in nonword repetition following treatment.

While there are no examples of correlation across case series as a methodology in this special issue, the cases in Best et al. (2015, this issue) are the first two children to be reported from what is expected to be a case series intervention study with children with word-finding difficulties. The two cases reported were sufficient to begin to test the adequacy of the computational model, although a larger case series would provide a stronger test and would allow testing of hypotheses proposed in the current study regarding the relationships between level of impairment and response to treatment.

Other issues in the use of intervention as a tool for investigating cognition

While the sections above demonstrate the possibilities for intervention as a method in testing cognitive theories, there can be ethical concerns with this methodology.

Time-consuming assessments, long baseline testing periods and/or no-treatment control sets, are necessary to ensure that any changes can be clearly attributed to the effects of treatment. Yet these may be hard to justify given that the participants require remediation of their skills. This is particularly problematic in populations where time is of the essence – such as children with developmental impairments (e.g. Best et al., 2015, this issue; Banales et al., 2015, this issue; Smith-Lock, 2015, this issue) or those with progressive neurological disorders (e.g. Croot et al., 2015). Consequently, experimental design may suffer, as certain checks may not be implemented (see Limitations section in Smith-Lock, 2015, this issue) and designs may need to differ across participants or phases of the study. These concerns may be somewhat mitigated if studies are carried out when intervention is not, or is no longer, offered as this would potentially allow participants to receive treatment that they would otherwise not have access to. Researchers often feel these conflicts acutely: those who undertake intervention research are often interested in the outcomes not only for theoretical reasons, but they also seek to improve outcomes for their clients.

Another issue is that it is not always the case that the methods used in the research can be readily applied in clinical settings. For example, the intensity of certain published training studies may not be feasible. However, studies in the current issue also show that theoretically informative treatments can be carried out outside the tightly controlled conditions of the laboratory. For example, in both Best et al. and Smith-Lock, the treatment was carried out by clinicians or teachers, and in schools rather than in a research environment.

Finally, while studies with acquired disorders have long been employed to investigate the nature of normal cognition, in the developmental domain it is it is highly debated whether atypical development can inform our understanding of acquisition processes in typically developing children (Castles, Kohnen, Nickels & Brock, 2014; Karmiloff-Smith, 1998) and the same is true of intervention studies in this domain. For example, Smith-Lock investigated past-tense acquisition in a sample of children with Specific Language Impairment and argued that the results provide insights into language processing in typically developing children. She does, however, discuss the possibility that the children with SLI demonstrated pathological rule-based behaviour that is not observed in typically developing children. In this regard, she notes that while it is possible that children with SLI could lack a process that typically developing children use for the processing of past tense, it seems implausible (without a principled reason or empirical data) that children with SLI would behave in a rule-based fashion if typically developing children do not.

Conclusions

Every experimental methodology brings with it unique possibilities for testing and extending our understanding of human cognition. We believe that intervention is an invaluable tool that has been overlooked too often in the past. Nevertheless, all experimental methodologies have challenges and intervention is no different in this respect. The researcher thinking of employing intervention as a tool should not be put off by the issues we have raised as there are many features of intervention research that enable important and sometimes unique opportunities for theory development. Research undertaken using this approach has brought and, we hope, will continue to bring new insights and converging evidence to advance our understanding of cognitive processes and representations and their acquisition. We hope that intervention will become a more standard tool in the researcher's methodological repertoire, and that, as Riddoch and Humphreys noted over 20 years ago, rehabilitation studies will become of as much relevance to theorists as theories are to those performing interventions (Riddoch and Humphreys, 1994, p13).