Dissociable effects of attention vs working memory training on cognitive performance and everyday functioning following fronto-parietal strokes

ABSTRACT

Difficulties with attention are common following stroke, particularly in patients with frontal and parietal damage, and are associated with poor outcome. Home-based online cognitive training may have the potential to provide an efficient and effective way to improve attentional functions in such patients. Little work has been carried out to assess the efficacy of this approach in stroke patients, and the lack of studies with active control conditions and rigorous evaluations of cognitive functioning pre and post-training means understanding is limited as to whether and how such interventions may be effective. Here, in a feasibility pilot study, we compare the effects of 20 days of cognitive training using either novel Selective Attention Training (SAT) or commercial Working Memory Training (WMT) programme, versus a waitlist control on a range of attentional and working memory tasks. We demonstrate separable effects of each training condition, with SAT leading to improvements in spatial and non-spatial aspects of attention and WMT leading to improvements on closely related working memory tasks. In addition, both training groups reported improvements in everyday functioning, which were associated with improvements in attention, suggesting that improving attention may be of particular importance in maximising functional improvements in this patient group.

Introduction

Stroke is the leading cause of long-term disability in the UK and other developed nations, with high costs for health and care provision (Saka, McGuire, & Wolfe, 2009). It can result in persistent physical, cognitive and mood impairments. Whilst the pattern of cognitive impairment will vary according to factors such as lesion extent and location, some presentations are particularly common.

Impaired attention has been reported in up to 92% of stroke survivors in the acute stage (Stapleton, Ashburn, & Stack, 2001) and to persist in up to 50% in the longer-term (Barker-Collo, Feigin, Parag, & Lawes, 2010; Hyndman & Ashburn, 2003). There are many reasons to think that attention – including our ability to detect errors and to remain focused on activities – would be critical skills in maximising functional recovery. Indeed capacity to sustain attention 2 months after stroke is a stronger predictor of motor recovery over the following two years than the level of physical impairment in the acute stage (Robertson, Ridgeway, Greenfield, & Parr, 1997b). Similarly, attentional functioning has also been linked to the recovery of other functions such as language (Geranmayeh, Brownsett, & Wise, 2014). Moreover, attentional deficits that impact spatial awareness (particularly unilateral neglect) are associated with high levels of disability, poor outcome, and increased reliance on public services (Jehkonen, Laihosalo, & Keetuenen, 2006; Katz, Hartman-Maeir, Ring, & Sorojer, 1999).

Perhaps due to its striking presentation (including failure to eat food from half the plate, or dress one side of the body) and link to poor outcome, much of the focus on rehabilitation of attentional difficulties has been directed on trying to reduce the spatial impairments seen in patients with unilateral neglect. Whilst a number of interventions have been proposed to ameliorate spatial bias, including adaptation to prism lenses (Rossetti et al.,1998), hemifield patching (e.g., Rossi, Kheyfets, & Reding, 1990) and training in visual scanning (Luukainen-Markkula, Tarkka, Pitkänen, Sivenius, & Hämäläinen, 2009), a Cochrane review (Bowen, Hazelton, Pollock, & Lincoln, 2013), concluded that there was insufficient evidence of generalised, persistent gains to currently recommend any intervention.

Given the strong evidence that attentional impairments may be key to maximising functional recovery, and that spatial interventions have not given rise to generalised persistent improvements in patients, could training non-spatial aspects of attention be beneficial? Computational models of normal attention, such as Bundesen’s Theory of Visual Attention (TVA; Bundesen, 1990), propose a number of separable but interacting attentional components. Some of these are non-spatial (for example, attentional selection (α), and visual capacity (K)) whilst others, like attentional weighting to one side of space (spatial bias), are clearly spatial in nature. Patient data from both our group (Duncan et al., 1999; Peers et al., 2005) and others (Habekost & Rostrup, 2006; Corbetta & Shulman, 2002; Robertson et al., 1997a) indicate that pathological spatial biases tend to arise and persist in the context of more general attentional impairment. Interventions that manipulate general attentional resources, for example increasing alertness via stimulants or stimulation (George, Mercer, Walker, & Manly, 2008; Gorgoraptis et al., 2012) or reducing alertness with sleep onset (Bareham, Manly, Pustovaya, Scott, & Bekinschtein, 2014), have been shown to phasically modulate spatial bias, suggesting rather direct interactions between these components. Despite not explicitly targeting spatial bias, therefore, it may be possible to improve spatial functions by focussing on other aspects of attention.

There has been little scientific evaluation of the potential success of training specific cognitive functions following stroke (see Sturm et al., 2003; Westerberg et al., 2007 for exceptions). Westerberg et al., (2007), for example, report positive findings from working memory training (WMT) suggesting improvements following training that appear to generalise to untrained tasks. The absence of a control group, however, makes it impossible to rule out the possibility that these effects reflect the general benefit of being involved in any intervention, participants’ expectations, or passive changes in participants condition over time. Johansson, & Tornmalm (2012), in contrast, detected improvements only on trained tasks. Here we ask whether computerised training, which focusses on improving attentional functions can produce specific, measurable changes in cognitive functioning and reduce disability in everyday life.

To this end, we developed a novel Selective Attention Training (SAT) battery, consisting of five tasks developed to shape participants’ ability to rapidly attentionally sift through onscreen stimuli for goal-relevant information. We intended to compare this with another, well established, cognitively demanding WMT battery, Cogmed™. This training battery is attentionally demanding but is primarily focussed on improving working memory capacity. WMT has been studied extensively, mainly in developmental populations. Some studies show gains which may stem from changes within the attentional control system (Astle et al., 2015; Barnes et al. 2016) whilst others show that these improvements extend only to tasks that are similarly structured to those practised during training (Dunning et al., 2013), suggesting that, in children at least, task-specific strategies, rather than generalised attentional improvements, may account for the behavioural gains made.

Whilst generalisation remains highly questionable within the developmental literature (Melby-Lervåg, & Hulme, 2013, Roberts et al. 2016), there are grounds to believe that the case of stroke patients may be very different. Importantly, whilst school-aged children are exposed to hours of structured mental stimulation and feedback in the classroom each day, stroke patients in the community do not receive such stimulation, or feedback, which may be crucial to learning or relearning cognitive skills.

Taking the lead from the Cogmed WMT battery, we produced our SAT to share many of the features that have shown promise in WMT. Both forms of training are adaptive (becoming progressively more difficult as performance improves, and easier if performance is poor), allowing patients to progress at their own rate. Improvements in performance are rewarded with points, melodic flourishes or spoken feedback. Both forms of training employ varied, relatively brief tasks, using colourful displays, and provide trial-by-trial feedback to assist with learning.

Cognitive training could produce general benefits that are unspecific to the training tasks (e.g., structured daily reinforced cognitive practice, sense of confidence and mastery, general expectation effects and so forth). An advantage to comparing two forms of cognitive training was that it allowed us to examine both specific cognitive effects and general benefits. To this end, we randomly allocated patients with likely difficulties in attention following stroke to a WMT, SAT or a waitlist, WL, condition. Outcome measures of working memory, attention, spatial bias, and self-report everyday function were completed before and after 4-weeks of daily training (or equivalent waitlist period). To increase power, WL participants were then randomised to one or other form of training, with their post-WL assessment acting as the baseline for subsequent post-training reassessment. A further benefit of having these carefully matched training regimes, in combination with a range of theoretically motivated cognitive outcome measures is that it allows us to start to explore the potential mechanisms by which any improvements in cognitive function have occurred.

Logically three potential outcomes could be predicted, which would lead to different conclusions about associated mechanisms. Firstly, neither training regime could be associated with improvements in the outcome measures compared with the WL control. This would question whether any form of training could be effective, or whether this null finding could be due to “dose” or insensitivity of the outcome measures. Secondly, both forms of training could produce equivalent general benefits compared with WL suggesting common mechanisms which could potentially be due to motivational or social influences as opposed to training specific cognitive abilities. Finally, each form of training could produce its own profile of cognitive improvements suggesting mechanisms that are, at least in part, specific to each form of training. For example, one might expect to find the greatest effects of WMT on working memory measures and conversely the greatest effects of SAT on attention measures. Finally, given the importance of attention for outcome in stroke, we have included a measure of everyday function to examine whether specific improvements in cognitive function influence everyday functioning.

Materials and methods

Participants

Twenty-three participants from the Cambridge Cognitive Neuroscience Research Panel (CCNRP) gave informed, written consent for their participation in the study. The CCNRP is a database of volunteers who have suffered a brain lesion from various causes and who have expressed an interest in participating in research. Twenty had right-hemisphere stroke, 2 left hemisphere stroke, and 1 had bilateral damage. All were chronic patients (mean time since injury 8.5 years, SD 4.7 years, range 7 months-17 years), aged under seventy-five years (mean 59 years, SD 10.6 years, range 28–74 years) and had no history of other neurological conditions. Patients were selected without knowledge of their behavioural difficulties, but on the basis that they had large lesions compromising frontal and parietal regions (areas of the brain most strongly associated with attentional functions) (see Figure 1 for lesion overlays for the 10 patients for whom MRI scans were available). All had normal or corrected to normal visual acuity and sufficient language to comprehend and respond appropriately to the task demands and to provide informed consent. Although a number of the patients had substantial motor impairments they were able to make required responses (even if sometimes with their non-dominant hand) where these were required. The study was approved by the Cambridge Psychology Research Ethics Committee. Participants received a small honorarium for their time.

Figure 1.

Lesion overlays for 10 of the 23 patients in the study, for whom scans were available. These show the foci of the lesions in frontal, parietal and temporal cortices in the right hemisphere.

Procedure

Prior to their first assessment session participants were randomised into one of three conditions: WMT, SAT, or WL. At this time, the WL patients were further randomised to WMT or SAT to be completed at the end of the WL period.

After completing their initial assessment, participants in training conditions were shown how to log in to the relevant websites and navigate through the tasks. They were asked to try and complete the training each weekday for the next 4 weeks (20 sessions). Participants were encouraged to get in touch with the project team if they experienced any difficulties accessing the tasks. In addition, a weekly phone call was scheduled with a member of the research team in which participants could discuss any difficulties and their general progress. Because the research team received a log of each participant’s use of the programmes, where repeated sessions were missed this could be brought up in the conversation, enquiring whether there were particular barriers, whether the participant had forgotten about the training and so on. If necessary we allowed longer than 4 weeks for participants to complete 20 sessions. All training sessions were completed in participants’ homes, for all but one on the participant’s home computer. WL patients also had a weekly phone call from the research team during their wait period in which they were asked similar questions about progress (but not about training). After the 2nd assessment, when the Waitlist participants began training they received the same level of support described above.

Pre-training assessment session

Participants received an extensive assessment of their cognitive profile and everyday functioning at each of the assessments. The measures focussed predominantly on attention and memory functioning and included both measures taken from basic science as well as those typically used to assess function in the clinic.

Background assessments

Participants completed a number of standard assessments including the Sloan Letter Near Vision Card (Good-lite Co, IL) to assess visual acuity, and the Tests for Colour-Blindness (Ishihara, 1978). All patients had normal or corrected to normal, visual acuity and all but one was found to have normal colour vision. The National Adult Reading Test (NART) (Nelson 1982) was used to estimate premorbid IQ and the Cattell Culture Fair Test (Institute for Personality and Ability Testing, 1973) was used to estimate current fluid IQ.

Attention measures

Partial and Whole Report TVA paradigm. This test, based on tasks extensively used in many studies (Duncan et al. 1999, Peers et al. 2005 and Habekost, 2015 for review), was used to assess the attention parameters of spatial bias, attentional selection (α) and visual short-term memory capacity (VSTM) operationalised in Bundesen’s (1990) Theory of Visual Attention (TVA). This task required participants to verbally report the identities of as many letters of a pre-specified target colour (either black or white) as they could from arrays of briefly presented targets and non-targets, whilst maintaining central fixation. Each trial followed essentially the same pattern. An initial red fixation cross flashed on and off a grey background at a rate of between 150 and 230 ms four times. An array of letters was then presented along with the fixation cross for 150 ms before being replaced by the fixation cross alone until the experimenter had recorded all the participant’s responses and initiated the next trial. The arrays comprised of letters approximately 2 degrees by 3 degrees arranged in a circle approximately 10° radius about the central fixation cross. Letters were selected at random from the set B,C,D,F,G,H,J,K,L,N,P,Q,R,S,T,U,V,X,Y,Z, and were presented in either black or white. Three basic types of array were presented; 1.3 targets (3 T) Unilateral presentation of three letters (in the target colour) to either the left or the right of fixation. 2. 3 targets 3 non-targets (3T3NT) Presentation of three target letters on one side of the screen with three non-targets (in the opposite colour) appearing on the other side of the fixation cross. 3. 6 targets (6 T) Presentation of six letters in the target colour, three to the left, and three to the right, of fixation. From these conditions, 3 separable attentional parameters (closely related to those defined in TVA, but using simplified formulae) were defined:

• Absolute spatial bias; the relative extent to which performance is preserved on a particular side of space in the presence of competing target information on the other side of space. To examine this, we compared the relative reduction in performance between the 3 and 6 T conditions for items presented on the left versus right sides of space, using the following formula:

  $$\mathrm{A}\mathrm{b}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}\mathrm{b}\mathrm{i}\mathrm{a}\mathrm{s}=\mathrm{A}\mathrm{B}\mathrm{S}0.5-\left(\frac{\left(pcorr6T_{left}/pcorr3T_{left}\right)}{\left(pcorr6T_{left}/pcorr3T_{left}\right)+\left(pcorr6T_{right}/pcorr3T_{right}\right)}\right)$$

Where pcorr is the proportion of targets correctly identified in that condition.

• Top Down Control (α’); the extent to which distracting (non-target) information can be ignored. Here we examine where the performance in the 3T3NT condition lies between the 3 T condition and the 6 T condition using the formula below. If participants have very good selection (lower values of α’), the non-targets should have relatively little impact whilst higher values of α’ indicate poorer attentional control.

  $${\alpha }^{\mathrm{\prime }}=\frac{\left(pcorr3T+pcorr6T\right)}{\left(2\times pcorr3T3NT\right)}$$

• Visual Short-Term Memory Capacity (K’); the maximum number of letters that can be reported from a brief display of letters. Following standard practice, we use probability mixtures of the maximum and 1- maximum performance. In this case, m is the maximum number of letters ever reported (in the 6 target condition), and 6T_m is the number of trials in which the participant correctly reported m letters.

  $$K^{\mathrm{\prime }}=\left(m\times \frac{6T_m}{6T_m+6T_{m-1}}\right)+\left(\left(m-1\right)\times \frac{6T_{m-1}}{6T_m+6T_{m-1}}\right)$$

• Variability. In addition to the three traditionally measured TVA parameters an additional measure of participants’ variability in performance was derived. Variability in performance is thought to be indicative of poor sustained attention, which has also been linked to poor spatial awareness (Robertson et al. 1998, Stuss et al. 1989). This was defined as the coefficient of variation (standard deviation divided by the mean) of correct letter reports from the 6 T condition.

Participants completed 4 blocks of 60 trials, two towards the start of the experimental session and two towards the end of the experimental session.

In addition to the TVA paradigm, participants completed 5 other computerised versions of attention measures that have either been used clinically or which have been shown to be sensitive to spatial bias in experimental studies. These were:

Star Cancellation Task (based on Wilson et al. 1987); a version of this well-recognised measure in which participants were asked to mark, using a stylus, all the small stars on a busy array of small and large stars and letters scattered across the screen as quickly as they could. Patients with spatial neglect have a tendency to miss a disproportionate number of targets from one side of the display.

Line Bisection Test; in which participants were asked estimate and mark the mid-point of seven lines, between 11.5 and 15.2 cm in length, presented either centrally or to the left or right of the screen. The bisections of patients with spatial neglect can deviate markedly from the objective centre suggesting that their awareness of one end of the line is impaired (Harvey et al. 1995).

A Temporal Order Judgment Task; in which two boxes appeared to the left or right of fixation either simultaneously or with a variable delay in their onset. The participants’ task is to judge which of the boxes appeared first. This version comprised 6 trials with simultaneous onsets and 2 trials with at each of 51 ms,102, and 500 ms onset asynchronies respectively. It has been reported that patients with left spatial neglect performing a similar task required the left target to appear up to 500 ms ahead of the right target before accurately reporting the order (Rorden et al. 1997).

A Lateral Reaction Time Task; in which participants pressed a central button as soon as they detected a target that could appear either to the left or right of fixation. Fourteen targets were presented with variable inter-target intervals, equal numbers appearing on the left and right. Absent, disproportionately slow and variable response times to targets in neglected space have been reported (Anderson, Mennemier and Chatterjee, 2000).

Slow and Variable Tone Counting Test; in this variant of a test of sustained attention (Robertson et al., 1996) participants must attend to and count a variable series of tones separated by long and unpredictable intervals. Performance on this test, which has no spatial requirement, has been reported to be particularly poor in patients with persistent spatial neglect (Robertson et al. 1996).

Working memory measures

Two spatial short term memory tests from the Automated Working Memory Assessment (AWMA; Alloway, 2007) were carried out.

• AWMA Dot Matrix Test. In this computerised test, a 4 × 4 grid was presented on the screen. The participant watched as a dot appeared at various locations on the grid and then recreated the sequence by pointing to the locations in the correct order. The test began with 2-location sequences and increased in sequence length until accuracy dropped below 50%.

• AWMA Spatial Span Test. Two abstract shapes were presented side by side on the screen. These could be identical or mirror images of one another, with the rightmost shape being presented in the upright position or rotated 120 or 240 degrees about the centre. With each presentation, the participant had to determine whether the 2 shapes were the same or mirror images of one another. The shape on the right was always presented with a dot at one of three locations. At the end of a series of shape pairs, the participant was asked to recall in order the locations of the dot on each pair. The test began with a single pair and increased the number of pairs until accuracy dropped below 50%.

Whilst both these measures are designed to measure visuo-spatial short-term memory the addition of a manipulation element to the Spatial Span Test allows us to examine whether any transfer of training occurs in working memory tasks which are structurally different from those used in the training battery.

Self-reported everyday function measure

• European Brain Injury Questionnaire (EBIQ; Teasdale et al. 1997). This 63-item self-report questionnaire asks participants to rate their own function/symptoms over the preceding month. The items are grouped into nine broad categories; somatic symptoms, cognitive symptoms, motivation, impulsivity, depression, isolation, physical symptoms, communication issues and core symptoms, the latter being a global measure of disability.

Training

The training batteries were internet-based and completed in participants’ own homes. Following an initial induction, they were completed without assistance from the research team. The batteries shared some essential core features, namely: that they were adaptive and therefore designed to keep patients working at their maximal ability, and that trial by trial feedback was given for both learning and motivational purposes.

Working Memory Training (WMT)

The adaptive version of the commercial Cogmed^™ Working Memory Training (Pearson; for full details see www.cogmed.com/rm) was used. Participants attempted 15 trials of 8 tasks in each session, covering both verbal and visuo-spatial working memory. Following the standard set-up, three of the twelve tasks in the battery were presented in every session, with the rest of each session being made up of five of the remaining nine tasks. Most participants completed a session of training in approximately 30–50 min.

Selective attention training (SAT)

This training was designed by the research team and programmed in Flash using Adobe Flex Builder 3. They were deployed via a custom website (https://www2.cbstrials.com) developed in Ruby on Rails. Screen shots of the tasks can be seen in Supplementary Materials. The training consisted of five time-limited tasks designed to improve selective attention. Each of the tasks required patients to direct their attention to target or goal-relevant material. Whilst the tasks were not designed to tap separable components of attention, the varied nature and look of the tasks are intended to maintain interest and generalise these skills across different contexts. Each of the tasks is described below.

• Aliens. In each trial, an onscreen array of cartoon aliens appeared, one of which was designated as the target. The participant’s task was to decide as quickly as possible whether another of the aliens was an exact match to the target, indicating a match/mismatch response by mouse clicking onscreen buttons. All aliens were comprised of a combination of a head part, a body part and legs selected from four prototype heads, bodies and legs. These could vary along parameters such as the texture and thickness of arms and legs, number of eyes, the presence/absence of tail, hairstyles and clothing. With correct responses, task difficulty was increased by increasing the number of aliens in the array and their similarity to the target (requiring increasing attention to small distinguishing details). As with all of the SAT tasks, auditory and visual feedback was given for correct (a large green tick and a bell) and incorrect (a large red cross and a buzz) responses, progress was indicated by an onscreen thermometer (see Supplementary Materials), and the remaining time for the task indicated with an onscreen digital clock. The duration of the Aliens in each training session was 3 min.

• Visual Search. In each trial, an abstract shape was presented on the screen for a few seconds. It was then replaced by an array of objects and the participant was asked to judge whether any exactly matched the original shape. The difficulty was manipulated by increasing the similarity of the objects to the target along dimensions of shape, size, colour and texture. Duration, 4 min.

• Jigsaw. At the top of the screen two or more red boxes were shown each containing a distinct pattern or object (e.g., one with blue and white stripes, the other with an inverted grey triangle). In the lower part of the screen, four or more white boxes also appeared, each with patterns or shapes. The participants’ task was to decide whether the elements of each red box were present in the white boxes such that the “jigsaw” could be made from these pieces. The difficulty was manipulated via the similarity of the elements in the boxes to the target configurations, the number of red boxes that needed to be matched and whether the elements in the white boxes needed to be mentally rotated to make up the patterns. Duration, 4 min.

• Rotations. This test required participants to take in the spatial relations between a series of shapes and then mentally rotate this image to judge whether it would match a second image. In each trial two large squares were presented, each containing one or more smaller green or red squares (in effect, filled cells of an invisible identical grid). The participant’s task was to indicate whether the rotation of one large square and its elements would make it identical to the other. The difficulty was manipulated by the number of elements within the squares, the degree of rotation required and the similarity of the two squares (e.g., the elements being in very different locations compared to only one of many elements differing). Duration, 3 min.

• Button Sorting. On each trial of this set-shifting task, a shape was presented upon which the participant was asked to make a speeded judgment based on a rule also presented on the screen. If the rule was “shape” the participant had to indicate whether the shape most closely resembled a circle or square by clicking on an arrow pointing to one of two reference shapes (circle and square) that were coloured red and yellow (the colour was irrelevant to the “shape” rule). If the rule changed to “colour” the participant had now to click on the arrow pointing to the correct colour and ignore the shape of the reference. The difficulty was increased by morphing the shapes in the direction of the alternate category (e.g., increasingly rounding off the square) and making the colours increasingly similar. Duration, 4 min.

Post-training assessment

These sessions comprised the same tasks as the pre-test session without the background assessments. Sessions were scheduled to occur within 2 weeks of completing the online training, or the case of the WL, 4–6 weeks from their initial assessment.

Results

Complete data were analysed from twenty of the twenty-three participants. Of those whose were omitted, two were removed having suffered subsequent neurological events between initial assessment and final assessment and one had to drop out owing to family circumstances. After reassignment of the waitlist patients, complete data were available for 10 patients in each of the 3 conditions (WMT, SAT and WL). One-way ANOVAs were carried out to see whether the 3 groups differed on background measures. No significant differences between the groups were observed for age (F(2,29) = 0.57), time since injury (F(2,29) = 1.61), visual acuity (F(2,29) = 1.20), NART (F (2,29) = 1.37) or Cattell (F (2,29) = 1.37) suggesting the groups were well matched.

Training

Compliance

Compliance with the training programme was generally good. All patients who started the training completed the study and on average the WMT group completed 19.8 of the intended 20 sessions (range 18–20 sessions) whilst the SAT group completed 20.2 sessions (range 18–23). Patients were in regular contact with the research team (by phone or occasionally email) over the course of the training period. 86% of patients completed the training in the intended 4–5 weeks the remaining 3 patients (2 SAT and 1 WMT) took between 8 and 13 weeks to complete. Participant feedback regarding the training was generally very positive. Nine participants requested to continue with training following their final assessment.

Improvement on training tasks

Mean performance by session data (collapsed across the three continuous Cogmed tasks, or all SAT tasks) for each of the training batteries are shown in Figure 2. Polynomial equations (y = x² + x + c) were fitted for each participant separately. These provided better fits than logarithmic fits and allow us to determine parameters including δy (the improvement in performance), δx (the number of sessions to maximal performance) and δy/δx (the average rate of improvement to asymptote). Polynomial fits were good, with a mean r² of 0.76 (range 0.56–0.87) for the WMT group and 0.91 (range 0.68–0.99) in the SAT group, for all but three patients, one of whom, it became evident, had been writing down number sequences in the memory training, rather than trying to remember them. Data from these three patients with were therefore excluded from further analyses. Generally, the two training conditions appear to show a similar improvement profile with maximal performance achieved after 15.6 sessions (range 11.8–18.8) and 16.6 sessions (range 13.25–23.54) for the WMT group and the SAT group respectively. Direct comparison of the improvements in the two training conditions were precluded by the different scales used. Nonetheless, the WMT group showed an average improvement of 2.7 items (range 1.2–4.6) whilst the SAT group improved by 11.7 points (range 7.9–16.1) indicating that all participants were able to demonstrate improved performance with training.

Figure 2.

Mean (± S.E) performance on the training tasks over the twenty days of training for (A) WMT and (B) SAT groups respectively

Training transfer

Having demonstrated that participants generally improved on the training tasks, we next established whether these improvements transferred to other cognitive tasks and measures of disability. For all subsequent analyses, a regression approach was used to examine whether post-test performance was influenced by the training group (WL, WMT or SAT) whilst adjusting for pre-test performance. As pre-test performance was likely to influence the outcome (after all, individuals with more extensive lesions are likely to have greater difficulties to start off with maybe most likely have greater residual difficulties at the end of the study), it was important to include this as a predictor. Its inclusion enabled correction for individual differences in pre-test score thus any potential between-group differences at pre-test should not dominate the result. Coding dummy variables allowed us to compare the effects of the interventions (i.e., WMT compared to WL and SAT, and SAT compared to WL and WMT) in a single analysis. This regression approach is a stricter test of training gains than standard ANOVAs because interactions in the ANOVA can be at least partly driven by pre-training differences.¹ A standard enter method was used for the regression due to the exploratory nature of the study, with only a small number of predictors entered into each analysis. In addition to the regression approach, paired sample t-tests were carried out to examine whether post-scores differed significantly from pre-scores for each of the groups. For several transfer measures, Figure 3 shows the post-test score minus pre-test score for each of the three groups.

Figure 3.

Mean (± S.E) change in performance from pre-test to post-test for the experimental measures in each of the groups. Plots show performance for; A. change in TVA absolute bias, B. change in K’, C., K variability, D. change in a number of targets cancelled by side on the star cancellation task and E. change in Dot Matrix performance. An * above the bar denotes that that group is a significant (p < .05) predictor of the post-test score. An * to the left of a bar indicates a significant (p < .05) change in test performance between pre and post-test, as measured by paired sample t-tests.

Measures of attentional functions

Turning first to spatial bias (see Figure 3(a)), the regression indicated that between them, “pre-test score” and “experimental group” predictors explained 75.5% of the variance (R² = 0.76, F(3, 26) = 26.74, p < .001). Whilst as might be expected, “pre-test score” was a significant predictor (β = 0.89, p < .001), SAT (compared to a combination of WL and WMT) was also a significant predictor (β = −0.41, p = .002), whereas no such significant effect was seen for WMT (compared to a combination of WL and SAT) (β = −0.18, p = .15). In addition to this, paired samples t-tests indicated a significant change in bias score between pre- and post-testing in the SAT group (t = −3.03, df = 9, p < .05,), no evidence of significant change was observed in either the WMT group (t = −1.27, df = 9, p = .24,), or WL (t = 1.44, df = 9, p = .19,). Thus, only the SAT could be demonstrated to have a beneficial impact on spatial awareness.

As our SAT was focussed on improving selective attention it may be expected that any attentional effects on awareness may stem from improvements in top-down control (α’). However, the regression indicated that between them, “pre-test score” and “experimental group” predictors explained less variance than we saw with spatial bias (R² = 0.57, F(3, 26) = 11.54, p < .001) and “pre-test score” was the only significant predictor (β = 0.74, p < .001).

The K’ capacity measure might have been expected to have been influenced by both WMT and SAT training. The regression indicated that between them, “pre-test score” and “experimental group” predictors explained 68.7% of the variance (R² = 0.69, F(3, 26) = 19.02, p < .001) (Figure 3(b)). In line with our prediction, it was found that “pre-test score” (β = 0.77, p < .001), SAT (β = −0.38, p < .01), and to a lesser extent WMT (β = −0.28, p < .05) were all significant predictors of the post-test score. Paired samples t-tests indicated that K’ values were significantly improved in the SAT (t = 3.73, df = 9, p < .01,) group post-training, an effect that reached near significance in the WMT (t = 2.11, df = 9, p = .06,) group, but was absent in the WL (t = 0.06, df = 9, p = .95,).

It is worth noting that K’ and spatial bias might not be independent. To take an extreme example, if a participant reports all 6 letters, spatial bias must be zero. To address this potential non-independence we re-ran the spatial bias regression including “pre- post-test K’ change” as a predictor of pre- post-training bias change. This indicated that, between them, “pre-test score,” “change in K’” and “experimental group” predictors explained 76.1% of the variance (R² = .761, F(25,29) = 19.89, p < .001). Whilst “pre-test bias” (β = 0.87, p < .001) was a significant predictor, critically, “change in K’” was not (β=0.09, p = 0.49). Importantly, despite this very stringent test, SAT (β = −.36, p < .05) remained a significant predictor, but there was no evidence of a significant effect of WMT (β = −.16, p = .20). This strongly suggests that the effects of improved spatial bias following SAT were not simply an artefact of improved capacity.

We were also able to use 6 T variability (Variability) as a measure of the consistency with which attention was maintained (Figure 3(c)). Despite our training not being specifically designed to develop this skill, “pre-test score” and “experimental group” predictors still explained 36.8% of the variance (R² = 0.37, F(3, 26) = 5.05, p < .01) in post-test Variability. This is driven by both “pre-test score” (β = 0.51, p < .005), and WMT (β = −0.37, p < .05). Here, no significant effect of SAT was seen (β = −0.20, p = .28). Paired samples t-tests indicated that Variability was significantly reduced in the WMT (t = 2.73, df = 9, p < .05,) group post-training, but such reduction did not reach significance in the SAT (t = 1.76, df = 9, p = .11,) or the WL(t = 0.37, df = 9, p = .72,) groups.

Analysis of the standard clinical measures of attention, star cancellation, line bisection, prior entry, lateral reaction time and line bisection were carried out despite most patients showing no significant clinical impairments on these tasks at pre-test (patients were selected on the basis of lesion location rather than clinical symptoms). As exemplified by the star cancellation data (see Figure 3(d)) an encouraging pattern of results was observed post-training, with increased awareness on left-sided items. However, as expected due to pre-test performance, no significant effects were seen in any of the clinical measures.

Working Memory Measures

Change in performance on a measure of visuo-spatial capacity, the AWMA Dot Matrix task is shown in Figure 3(e). The regression indicated that between them, “pre-test score” and “experimental group” predictors explained 62.8% of the variance (R² = 0.63, F(3, 26) = 14.09, p < .001). In line with our prediction “pre-test score” (β = 0.73, p < .001) and WMT (β = 0.30, p < .05) were significant predictors of post-test score, while SAT did not reach significance(β = −0.09, p = .59). Paired samples t-tests indicated that the number of locations correctly recalled was significantly increased in the WMT group (t = 3.19, df = 9, p < .05,) post-training, an effect that reached near significance in the SAT group (t = 2.24, df = 9, p = .05,), but was absent in WL (t = 0.51, df = 8, p = .63,). Performance on the Spatial Recall task of the AWMA did not vary by training condition in the same way. Although a significant regression (R² = 0.55, F(3, 26) = 9.03, p < .001) was observed, the only significant predictor of post-test performance was “pre-test score” (β = 0.70, p < .001) with neither WMT (β = 0.24, p = .15) nor SAT (β = 0.04, p = .85) acting as significant predictors. Despite this, paired-sample t-tests indicated that the WMT group showed a significant improvement in performance between pre- and post-test (t = 2.49, df = 9, p < .05,) whereas neither SAT (t = 0.48, df = 8, p = .65,) nor WL (t = 0.15, df = 8, p = .88,) showed such effects.

Broader measures of functioning

Changes in self-reported EBIQ rating for each of the domains are shown in Figure 4. To limit the number of statistical tests conducted, the formal analysis was limited to the two most pertinent domains; core symptoms (a global measure of impairment) and cognitive symptoms. Turning first to core symptoms, regression indicated that between them, “pre-test score” and “experimental group” predictors explained 66.4% of the variance (R² = 0.66, F(3, 26) = 19.02, p < .001). In line with our prediction, “pre-test score” (β = 0.65, p < .001), WMT (β = −0.61, p < .001), and SAT (β = −0.58, p < .001) were all significant predictors of the post-test score. Paired samples t-tests indicated that core symptoms were significantly reduced post-training in both the WMT (t = −5.42, df = 9, p < .001,) and the SAT (t = −4.38, df = 9, p < .005,) groups, but no significant change was observed in the WL (t = 0.77, df = 9, p = .46,). In a similar manner, a regression analysis indicated that “pre-test score” and “experimental group” predictors explained 55.7% of the variance (R² = 0.56, F(3, 26) = 10.92, p < .001) in post-test cognitive symptoms. Paired samples t-tests indicated that cognitive symptoms were significantly reduced post training in both the WMT (t = −5.78, df = 9, p < .001,) and the SAT (t = 3.41, df = 9, p < .005,) groups, but not in the WL (t = −1.15, df = 9, p = .28,) group.

Figure 4.

Change in self-reported functioning as measured by the EBIQ from pre-test to post-test. Formal statistical testing was only completed for EBIQ cognitive and EBIQ core symptoms. For these measures, an * above the bar denotes that that group is a significant (p < .05) predictor of the post-test score, whilst an * to the left of a bar indicates a significant (p < .05) change in test performance between pre and post-test, as measured by paired sample t-tests. Please note formal to reduce the number of test carried out, formal statistical analyses were only carried out on EBIQ cognitive and EBIQ core sub-scales, the significance of the other subscales has not been tested.

What predicts improvements in self-reported functioning? A key question is whether self-reported improvements were related to objective changes in cognitive function. A regression using “pre-test core symptoms,” “change in absolute bias,” “change in K’,” “change in Dot Matrix performance” and “change in Variability” as predictors of pre-post test change in core symptoms indicated that these variables explained 63% of the variance (R² = 0.63, F(5, 28) = 7.74, p < .001). As may have been expected, “pre-test core symptom score” was a significant predictor (β = 0.44, p = .006) of change in reported core symptoms. In addition, both “change in absolute bias” (β = 0.37, p = .016) and “change in Variability” (β = 0.33, p = .03) significantly predicted change in core symptoms. This was not true for changes in K’ (β = 0.03, p = .83) or Dot Matrix performance (β = −0.20, p = .15).

How does training improvement influence transfer?

The specificity of some of the improvements in working memory and attention tasks may be indicative of task or domain specific training. If this were the case, we might expect that the extent of the training gain would be predictive of the extent of improvement in closely related outcome tests and less predictive of change on more divergent measures. To compare training rates in the two training groups we standardised the rate parameter δy/δx, (Zδy/δx), and then generated in interaction term based on the product of the de-meaned group and the newly standardised rate parameter. Regressions were carried to examine significant predictors of change in performance between pre- and post-test on the basis of: “pre-test score,” “intervention” (WMT vs. SAT), “rate of improvement on training” (Zδy/δx), or the “interaction between group and improvement rate” (Gp*Zδy/δx). Turning first to change in Dot Matrix performance, the regression indicated that between them, “pre-test Dot Matrix score,” “intervention,” Zδy/δx and Gp*Zδy/δx predictors explained 70.6% of the variance (R² = 0.71, F(4, 12) = 7.21, p < .005). “Training type” (β = 0.35, p < .05), “training improvement” (Zδy/δx;β = −0.61, p < .05), and “Training type x training improvement” (Gp*Zδy/δx; β = 0.58, p < .005) were all significant predictors of change whilst “pre-test Dot Matrix performance” was not.

As might be expected from this finding and shown in Figure 5(a), the patients who showed the biggest WMT training also showed the biggest improvements on the untrained though similar Dot Matrix tasks, whereas the extent of SAT training gain did not influence Dot Matrix improvement. The pattern of results was markedly different for attentional measures. For both “change in absolute bias” (see Figure 5(b)) and “change in K’,” the regressors failed to significantly predict variance in change scores. As Figure 5(b) demonstrates, within the SAT group there was virtually no difference in change in bias scores between those who made small and large training gains. Changes in Variability and EBIQ core symptoms were significantly predicted by “pre-test core symptoms,” “training type,” “training gains” and “training gains x training type interaction” (R² = 0.62, F(4, 12) = 4.91, p < .05 for K variability, and R² = 0.70, F(4, 12) = 6.88, p < .005 for the EBIQ core symptoms). However, in both cases “pre-test score” was the only significant predictor of change (β = 0.74, p < .005 for K variability and β = 0.89, p < .001 for EBIQ core symptoms) and as Figure 5(c) shows, for both the WMT and SAT groups, similar reductions in core symptoms were reported by those with relatively small and large training gains.

Figure 5.

Mean (±S.E.) changes in performance for patients who made small and large training gains (based on a median split) as a function of training type. A. Dot Matrix task. B. Absolute spatial bias. C. EBIQ core symptoms.

Discussion

This study examined whether two forms of training aimed at improving attention in stroke patients would lead to improvements in untrained outcome measures and self-reported functioning. A good level of compliance and approximately equivalent exposure to training between the groups allowed us to realistically assess objective benefits and examine whether the distinct training programmes had distinct and/ or generic effects.

Distinct training effects

Participants who trained on the commercially available WMT showed greater improvements than SAT and WL participants on the Dot Matrix working memory outcome measure. Such “near transfer” of training gains has been observed in other populations (Holmes et al. 2015, Dunning et al. 2013). Indeed improvement in Dot Matrix performance was strongly predicted by the extent of training gains made by the WMT group, pointing to learning of specific task related strategies for the spatial span. Given that improvements beyond a narrow training context are a pre-requisite for any likely transfer to everyday activities, but that previous work (Barrett et al. 2006) has shown that even these specific training activities do not always transfer to even closely related tasks, the current finding is encouraging. It suggests that patients can effectively apply memory related strategies developed in training to very closely related tasks. There was some evidence of improvement on the more complex Spatial Recall task in the WMT group; however, WMT was not found to disproportionately influence post-test Spatial Recall relative to SAT and WL suggesting that transfer effects to more distantly related memory tasks may be relatively small.

Perhaps of greater interest, however, are the improvements seen in the SAT participants across a wider variety of attentional parameters, on paradigms that appear quite different from the trained tasks and which are less dependent upon the extent of the improvements made on the training tasks. SAT participants disproportionately improved, relative to those in WMT and WL conditions, in their ability to take in more information “at a glance” (K’) from brief displays and in the reduction of spatial bias (of which there were also hints in the clinical measures). The first is plausibly linked to differences in the training tasks. In WMT participants typically monitor sequences in which a single event occurs at any one time. In contrast, solving the problems presented in SAT involved taking in an increasing amount of visual information. Encouragingly, improving this capacity in a particular context during training led to attentional improvements that participants were able to effectively utilise in different contexts. It is possible that this practised distribution of attention also underpinned the reduction in spatial bias. However, at least in the context of unilateral spatial neglect, even explicit training of visual scanning per se has often proved of limited generalised efficacy (Manly, 2010). Another possibility, alluded to in the introduction, is that the reduction in spatial bias is a consequence of generally improved attentional “tone” – a relatively alert state in which relevant information from across space is better prioritised. It has previously been shown that fluctuations in alertness from stimulant medication, loud tones, time-on-task, and sleep onset can impact on patients’ and healthy participants’ relative awareness for information on the left and right sides of space (Robertson et al. 1998, George et al. 2008, Bareham et al. 2014). Anecdotally, both SAT and WMT patients appeared more awake and engaged after 4 weeks of regular, monitored cognitive activity with direct feedback. Whilst improvements in our proxy of alertness (TVA performance variability) were actually greater on average for the SAT than WMT groups, the substantial variability across SAT participants meant that this change failed to reach statistical significance (Figure 3(c)). Variability scores and change-in-variability scores tend, by their nature, to be somewhat unreliable as noise from the underlying measures is summed and further work is required in operationalising “alertness” and understanding mechanisms of change.

Generic effects

In addition to improvements that were specific to WMT or SAT, more general positive effects of training were observed, particularly a marked improvement in self-reported functioning across both training groups. Importantly these improvements were significantly influenced by improved spatial bias and reduced variability in performance, suggesting a link of self-perception to measurable changes in attentional functions. Interestingly, improvements in WM span did not significantly influence self-reports in the same way. If this finding is replicated one possibility is that SAT practice indeed produces deeper or faster-generalised changes for everyday cognition than WMT. Various accounts can be proposed for such an effect. Firstly, previous studies have suggested that poor attentional functioning is particularly associated with high levels of disability and poor outcome (Jehkonen, Laihosalo & Kettunen, 2006, Katz et al. 1999). Hence the change in these capacities may also produce more generalised effects. Secondly, gains in WMT may be disproportionately achieved via strategy development (see similar findings in Alzheimer’s Disease, Huntley et al. 2016) rather than underlying capacity, and as our data on transfer to other WM tasks suggest, the strategy may be less easily generalised to different contexts. It should be acknowledged that greater effects of SAT on everyday function may be the result of selecting patients with lesions most likely to compromise attention, whose every day difficulties are most likely to be attributable to problems with attention, or SAT being perceived as more relevant and hence being more influential over self-report. Further work should be carried out in a more heterogeneous sample of patients to see whether improving attention is key to improvements in function across all patients, or whether this type of intervention is only beneficial for those with attentional difficulties.

Somewhat unexpectedly, as discussed, WMT was linked with significant reductions in performance variability on the TVA attention measures. If reliable, such transfer to a seemingly unrelated task is particularly striking given some previous literature suggesting WMT gains are restricted to near transfer to very similar span tasks (Dunning et al., 2013). However, it is not implausible to imagine how repeated practice of monitoring increasing sequences of spatial of verbal material for subsequent recall, during which even a brief lapse could prove disastrous for the entire trial, could progressively shape such consistent engagement. Along these lines, studies in the Behavioural Activation literature (encouraging patients to schedule and participate in rewarding, stimulating activity) suggests that engagement in mentally stimulating activities may help to improve alertness (Thimm et al. 2006). It is possible, therefore, that providing a daily structure within which patients were helped to focus on a cognitively demanding task for a relatively prolonged time may be sufficient to help improve alertness, perhaps irrespective of the precise demands of that training.

Appropriateness of home-based computer training for fronto-parietal stroke survivors

The potential efficacy of cognitive training batteries to improve outcome has been vigorously debated in recent times, in both healthy adults and the developmental literature, with many suggesting that improvements may be short-lived and fail to generalise to meaningful improvements in everyday functioning (Owen et al., 2010, Melby-Lervåg, & Hulme, 2013, Roberts et al. 2016). Our data showing improvements in self-reported functioning related to improvements in attentional functions suggest this may not be the case in these particular patients. As discussed, there are plausible reasons why this population may benefit from training in a way that the developmental population may not. Providing some structure, focus and stimulation, as well as clear feedback to help them learn, may be critical to reductions in disability. Whilst these aspects of training are already in place in a school environment, many patients receive little input from clinical services and lack structure or focus to their day. Along these lines, the positive effects of online training on both cognitive function and activities of daily living have been observed in healthy older adults (Corbett et al. 2015).

The success of any intervention is dependent not only upon the potential for improvement following treatment, but also upon how practical and tolerable it is for patients. Here patients’ ability to cope with navigating to websites, logging in etc. was good and attitudes to both interventions were generally positive, with a good proportion of patients feeling it was worthwhile continuing after the study. In accordance with this, despite the time commitment of the study, drop-out rates were very low. A caveat is that this sample was recruited from a panel of individuals who have already indicated that they are motivated to take part in research. It remains to be seen whether such good compliance would be seen in an unselected population of stroke patients.

Implications and future directions

The results so far indicate some specific effects of the two types of training and some generally positive effects from both compared to WL. The specific training effects are well controlled in terms of exposure to training, interaction with the experimenter and the knowledge of being engaged in training hypothesised to be helpful. However, interpretation of the more general effects, is limited by reduced stimulation in the WL and potential expectancy effects. To a degree this is offset by the finding that reductions in spatial bias and improved K’ variability over the course of the study predicted changes in self-reported disability, suggesting that improvements in attentional functioning could be key to reducing disability. Of course, the reverse causality also remains a possibility. An active and plausible control condition hypothesised not to be beneficial is required to clarify these issues.

It is generally accepted that the majority of spontaneous recovery occurs within the first six months after stroke (Skilbeck et al. 1983, Tilling et al., 2001) and it is therefore perhaps surprising we saw such extensive training effects on average 8 years post-injury. Whether training gains in the chronic phase may be more attributable to strategy development than underlying recovery remains an important topic of investigation.

An additional important consideration in the potential use of remote cognitive training for patients with stroke is the potential impact of the patient-carer relationship on both compliance and assistance with training. Whilst in the current study patients were requested to complete training without assistance, and we assume the positive feedback to correct responding acts as a motivator to encourage engagement, it may be that carers or relatives have an important role to play in helping to provide structure and encouragement to maintain compliance and allow patients to gain the most from their training. Indeed it is noteworthy that none of the three individuals who took more than the intended 4–5 weeks to complete the training had a carer living at home. Further work is required to look at the impact of the extent that the patient-carer interactions influence outcome.

In summary, our study provides evidence that cognitive training is feasible in stroke patients, and can lead to both specific improvements in cognitive functions and more general improvements in self–reported function. Further work is required to examine whether such effects can be replicated in a larger unselected community sample to see whether such training could be of benefit to a broader range of stroke patients.

Funding Statement

This research and Open Access was supported by the Medical Research Council UK [grant number SUAG/003/RG91365].

Acknowledgements

This work could not have been carried out without the willingness and effort of our patients and their families.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Polly V Peers http://orcid.org/0000-0003-4470-9508