Online cognitive training in healthy older adults: a preliminary study on the effects of single versus multi-domain training

Abstract

It has been argued that cognitive training may be effective in improving cognitive performance in healthy older adults. However, inappropriate active control groups often hinder the validity of these claims. Additionally there are relatively few independent empirical studies on popular commercially available cognitive training programs. The current research extends on previous work to explore cognitive training employing a more robust control group. Twenty-eight healthy older adults (age: M = 64.18, SD = 6.9) completed either a multi-faceted online computerised cognitive training program or trained on a simple reaction time task for 20 minutes a day over a 28 day period. Both groups significantly improved performance in multiple measures of processing speed. Only the treatment group displayed improved performance for measures of memory accuracy. These results suggest improvements in processing speed and visual working memory may be obtained over a short period of computerized cognitive training. However, gains over this time appear only to show near transfer. The use of similar active control groups in future research are needed in order to better understand changes in cognition after cognitive training.

Introduction

With increasing age, declines in cognitive functioning become more pronounced [1]. With the proportion of the world’s aged population increasing, age-related cognitive decline in addition to neurodegenerative diseases such as Alzheimer’s have become of significance when considering healthcare costs and quality of life [2]. As such, developing appropriate preventative intervention strategies are important in attempting to lower the incidence of cognitive decline and dementia [3,4]. Cognitive training (CT) is an approach that has increasingly become popular over the past two decades, and provides theoretically driven skills and strategies which involve guided practice on tasks that reflect specific cognitive functions [3].

CT can be implemented through a number of different techniques and formats. Training can be process-based, whereby the intervention involves repetitive, drill-like training on specific tasks. Alternatively, more strategic individualized intervention using techniques such as memory formation strategies (e.g., “Method of loci”) can be implemented. Both forms of training have been shown to be effective in numerous population groups. However, of particular interest in the healthy older adults literature is the potential for at-home computerized training due to its easy facilitation into daily routines, and commercial availability.

CT is based upon the theory of cognitive reserve, which stipulates that cognitively engaging activity can lead to protection against cognitive decline in older age [5]. The principle underlying this theory is neuroplasticity; the process by which repetitive activation of brain regions leads to multiple changes in the brain at both cellular and larger network levels [6,7]. Changes in cortical density and neurophysiological responses have now reliably been shown as a result of CT [e.g., 8–10]. In healthy individuals, CT may act as a protective mechanism, delaying impairment, as a result of increased reserve [3].

The ACTIVE study was a key early project in the field and provided evidence to suggest that in older adults cognitive training may lead to sustained improvements in the cognitive domain in which training was applied [11–13]. Subsequently, a large number of studies have now corroborated this initial finding of improvement in cognitive performance following CT in healthy older adults [for reviews see: 14–16]. The ACTIVE study however showed that improvements were limited to the domain that was trained. This is a fairly consistent finding in the literature, with the majority of studies showing transfer onto related unpracticed tasks (near transfer) but not on tasks representing untrained cognitive domains (far transfer) [17].

In a previous study from our laboratory we utilized the commercially available computer-based CT program MyBrainTrainer (MyBrainTrainer L.L.C., Los Angeles, CA, USA) to investigate the efficacy of twenty-one days of CT in healthy older adults [18]. Participants allocated to the experimental group completed twenty minutes of the online program each day, while an active control group played solitaire for an equivalent time and duration. It was found that training significantly improved speed of processing as measured by the “Simple Reaction Time” task on the Swinburne University Computerised Cognitive Aging Battery (SUCCAB) [19], while no improvements were found in “Complex Reaction Time” or “Spatial Working Memory”. Such a finding was in agreement with Ball et al. [11] who suggested speed of processing to be the most reliable domain of improvement.

Despite relatively consistent findings of cognitive improvements, previous investigations of CT efficacy have more recently been criticized a number of methodological limitations [20]. One such drawback is often their inadequate use of control groups, which may potentially lead to questionable demonstrations of efficacy. While a push towards the use of active controls rather than no-contact or waitlist controls in trials is apparent, often the active element is not sufficient, with participants engaging in activities that cannot reliably be compared [21]. Many active control groups employ tasks such as quizzes, questionnaires or videos in attempt to match for time and effort [22–24].

As such, Brehmer, Westerberg and Backman [25] addressed this problem in their study by using a simple cognitive training program as the active control group. These participants ‘trained’ on a low level non-adaptive working memory task, that was not expected to elicit any legitimate change in cognitive function. As a result, they found both young and old adults to improve in cognitive performance compared to controls, a more reliable finding than many other studies. This was the first study to utilize such a methodology for the control group.

In the current study we aimed to also address this methodological limitation in the literature by utilizing simple reaction time (SRT) training as the active control task. In comparison to our previous study [18] which used solitaire as the active control, SRT represents a more robust control task which will enable a more stringent between-group comparison of potential cognitive improvement beyond simple speed of processing gains [25]. The current study also extended the training period to twenty-eight days. It was hypothesized that the control and treatment groups would show similar improvements post-training in simple reaction time performance. It was hypothesized that only the experimental group, who received a multi-faceted CT training program, would demonstrate improvement in higher cognitive domains (i.e. complex reaction time, spatial working memory and contextual working memory) beyond SRT.

Method

Participants

All participants were screened for previous health conditions using self-report including dementia and other neurodegenerative diseases, depression or psychiatric disorders, epilepsy, and drug and alcohol dependency. Participants were screened for dementia and depression using the Mini Mental State Examination (MMSE) [26] and the second edition of the Beck Depression Inventory (BDI-II) [27], respectively.

Thirty-nine participants were initially recruited, and completed baseline assessment. Eight participants withdrew from the study during the training phase. Participants were required to complete a minimum of 80% of the daily brain training exercises. Participants who did not meet the criteria were excluded from the study. In total, three participants were removed due to lack of compliance.

A sample of twenty-eight participants comprised of seventeen female and eleven males between the ages of fifty-five and seventy-eight (M = 64.18, SD = 6.9) completed the study. All participants scored greater than 25 on the MMSE, indicating no significant cognitive impairment [26]. Scores on the BDI-II showed no signs of depression [27]. Participants were recruited through community posters and advertisement on the Swinburne University research website. Participants gave written informed consent in accordance with the procedures outlined by the Swinburne University Human Ethics Committee.

Treatment conditions and study design

The study utilized a single blind, parallel groups, randomized design. Participants were randomly allocated to one of two conditions; A) simple reaction time training (active control condition) or B) cognitive training (treatment condition).

Materials

Cognitive training program

An internet-based commercially available cognitive training program (www.mybraintrainer.com) was utilized. The program was selected due to previous demonstration of efficacy [18] and ease of access for participants to complete CT in the comfort of their own homes. Both treatment conditions accessed the same program, minimizing anticipation effects.

In the treatment group, participants completed 12 tasks, once daily, comprising approximately twenty minutes of training. These tasks are described in Table 1. The active control group participants repeated the simple reaction time task (Task 1), twelve times daily, for the same time period. Therefore, participants were matched on time spent on the CT website, and both had access to their scores and subsequent improvements.

Table 1.

Description of cognitive training tasks used.

	Task	Task description	Cognitive domain targeted
1	Simple reaction time	A stimulus resembling traffic lights was presented. Participants responded as quickly as possible when the light changed.	Reaction time / Processing speed.
2	Recognition reaction time	A stimulus resembling traffic lights was presented. Participants responded with ‘left’ or ‘right’ if the light changed to red or green respectively.	Choice reaction time / Processing speed
3	Binary choice reaction time	A horizontal set of traffic lights displayed. Participants pressed ‘left’ or ‘right’ depending on whether the light was red or green respectively. Twenty correct responses were required.	Choice reaction time / Processing speed
4	Three-choice reaction time	A horizontal set of lights was displayed. Participants responded with ‘left’, ‘right’ or both arrows depending on whether the light was red, green or yellow respectively.	Complex reaction time / Processing speed
5	Inspection time	Two parallel lines (one longer) connected by a horizontal bar were presented quickly and then masked. Participants responded with ‘right’ or ‘left’ depending on the longest perceived length. Twenty correct responses were required.	Processing speed
6	Short-term memory	A set of 4 words were displayed (cues). An additional 10 words then appeared one at a time (probes). Participants responded with ‘right’ if the probe matched a ‘cue’ or ‘left’ if it did not. 80% accuracy was required.	Memory
7	Executive function	2 colored figures (e.g., circle, star) were presented on a 2-lined display. The word ‘shape’ or ‘color’ was presented next to these lines. Participants matched the two stimuli based on the word rule given by clicking ‘right’ for a match and ‘left’ when not a match. Twenty correct responses were required	Executive functioning
8	Visuospatial acuity	A large diamond shaped figure was presented containing 64 equal sized red and yellow squares. Participants responded either ‘left’ or ‘right’ to whether there were more yellow or red squares respectively. Twenty correct responses were required	Visuospatial ability
9	Information processing	An arithmetic problem was presented with a ‘target’ number in the right corner. Participants responded to whether the problem was greater, less or equal to the target as quickly as possible. Twenty correct responses were required	Processing speed.
10	Visuospatial memory	A set of 4 playing cards (cues) were displayed, to be memorized for suit and position. 10 additional cards (probes) were then displayed. Participants responded to whether the probe matched (suit, rank and position). A minimum of 80 % accuracy was required.	Visuospatial memory / Processing speed
11	Visual scanning/ Discrimination	Five boxes containing shapes were presented. A middle box in red was the ‘target’ box. Participants responded to which of the 4 boxes matched the target by pressing ‘left’, ‘right’, ‘up’ or ‘down’ depending on position. Twenty correct responses were required	Visuospatial ability / Processing speed
12	Working Memory	The N-back task. Participants determined whether a stimulus in a sequence matched the one that appeared ‘1’ item before. Thirty correct responses were required	Working Memory

Outcome measures - Swinburne University Computerised Cognitive Aging Battery (SUCCAB)

The SUCCAB is a computerized cognitive assessment battery for use in older adults [19]. The following tasks were chosen as measures of improved cognitive ability. In addition to reaction time measures, two memory tasks were included. These measures were selected as they pertain to abilities which are known to show declines in older adults [19]. As such, they represent suitable targets for intervention outcomes. All outcome measures were assessments of near transfer in the treatment group as they relate specifically to cognitive domains that were targeted through multi-domain training, however do not match the tasks which were used during the intervention period. For the control group, improvements in complex reaction time and working memory tasks would suggest far transfer as they did not complete training on tasks of this cognitive domain.

Simple Reaction Time (SRT)

A single white square was presented in the middle of the computer screen. Participants were required to respond as quickly as possible by pressing the right button on the response box. Following a short practice period, a total of twenty targets were presented with a randomized inter-stimulus interval to prevent anticipation effects. Simple reaction times were measured in milliseconds, as a measure of processing speed, with lower scores suggesting faster processing.

Complex Reaction Time (CRT)

Either a red square or a blue triangle appeared in the centre of the computer screen. Participants were required to press the right button (red) as quickly as possible upon presentation of a red square, or the left button (blue) as quickly as possible upon presentation of the blue triangle. Following a short practice period, twenty targets represented equally by blue triangles and red squares were provided. As with the SRT task, a randomised inter-stimulus interval was also implemented in order to negate the anticipation effects. Scores were recorded in milliseconds with lower scores representing faster decision-making speed.

Spatial Working Memory (SWM)

A white 4×4 grid appeared on the screen against a black background. Five of the gaps within the grid were filled with white squares. Participants attempted to remember the location of these filled squares. A blank grid was then presented with only one square appearing four times in separate locations on the grid, for two seconds each. Participants were required to determine as quickly as possible if the location of the individual white square matched the location of the square in the original presentation by pressing right (yes) or left (no). Following a short practice period, participants completed twenty trials whereby two of the four locations corresponded to the original presentation and two did not. Scores were recorded as time in milliseconds and percentage of correct responses.

Contextual Working Memory (CWM)

A series of twenty everyday images (e.g., food, tools) were presented at the top, bottom, left or right of the screen, for three seconds each. On completion of the series of images, they were presented again in a randomized order in the centre of the screen for two seconds each. Participants were required to respond by pressing top, bottom, left or right depending on where the images were originally presented. This measure of spatial episodic memory has been shown to be difficult in older adults.

Procedure

Participants were randomised by a researcher not involved in the recruitment, assessment, or training of the study. During the first training session, participants completed a demographics questionnaire, in addition to the BDI. The MMSE was then conducted to screen for any significant cognitive decline. The SUCCAB tasks were then administered to obtain a measure of baseline cognitive performance. At the end of this session, participants were given an access code to their allocated training program. Participants were provided with an information sheet which gave details for logging in and completing their tasks. They had access to the researcher’s phone-lines if they had questions regarding the program. Participants were given a date to return immediately following their training period to complete follow-up assessments. During this assessment, participants completed alternate forms of the SUCCAB tasks again (versions randomized between pre-post). Those allocated to the control were notified and offered an alternative code for full use of the CT program.

Results

Data screening and statistical analyses

Data analyses were conducted using the Statistical Package for the Social Sciences (SPSS Statistics for Windows, Version 21.0, IBM Corp., Armonk, NY, USA). The data was assessed prior to analyses in order to ensure statistical assumptions were met. The data violated a number of statistical assumptions, and due to the limited sample size, nonparametric analyses where employed rather than the removal or transformation of data. One participants’ data from the CRT task was lost at follow up, and thus this participant was removed from all analyses of this test. In order to assess for baseline differences, Wilcoxon Signed Ranks tests were used. Pre and post training data was assessed using Mann-Whitney U tests. The effect size for differences between groups using these non-parametric tests was produced using the following formula (r = Z/✓N). For both values, 0.10 represents a small, 0.30 a medium, and 0.50 a large effect.

Descriptive statistics

Table 2 displays the descriptive statistics between groups. No significant differences in age, education, MMSE, or days training where found. There was however a much higher proportion of males in the treatment group than the control (Treatment: N = 9; Active control: N = 2).

Table 2.

Mean and standard deviation values for all participants included in the study.

	Treatment	Active control	Significance
N	16	12	-
Gender (% male)	56%	17%	p < 0.05
Age (years)	64.00 ± 7.37	64.42 ± 6.6	ns
MMSE	28.69 ± 1.03	28.83 ± 1.1	ns
Education (years)	15.44 ± 2.25	15.17 ± 2.6	ns
Days Training	26.69 ± 1.45	26.75± 2.2	ns

ns = p > 0.05

Reaction time

Mean and standard deviation values for reaction time across tasks for both groups are given in Table 3. There were no significant differences in any measure of baseline reaction time performance between groups. Wilcoxon Signed Ranks test showed that the treatment group improved in reaction time for the SRT (Z = −2.79, p = 0.005, r = 0.70), CRT (Z = −2.02, p = 0.044, r = 0.51), SWM (Z = −3.00, p = 0.003, r = 0.75), but not CWM task (Z = −1.14, p = 0.255). The AC group improved reaction times for SRT (Z = −2.75, p = 0.006, r = 0.79) and SWM (Z = −2.12, p = 0.034, r = 0.58), but not CRT (Z = −1.78, p = 0.075) or CWM (Z = −0.79, p = 0.433). Effect sizes show that these improvements were generally stronger in the treatment group.

Table 3.

Mean and standard deviations for reaction time in milliseconds, across tasks for both groups.

Measure and condition	Baseline	Post-training
Simple Reaction Time
Treatment (N=16)	275.38 ± 46.43	235.57 ± 31.31
Active control (N=12)	258.54 ± 24.2	227.00 ± 22.9
Complex Reaction Time
Treatment (N=16)	423.53 ± 63.6	390.26 ± 34.8
Active control (N=11)	419.35 ± 68.3	384.45 ± 48.9
Spatial Working Memory
Treatment (N=16)	1011.38 ± 98.67	928.76 ± 138.82
Active control (N=12)	999.53 ± 79.5	940.62 ± 117.5
Contextual Working Memory
Treatment (N=16)	1015.80 ± 116.07	985.22 ± 168.82
Active control (N=12)	988.32 ± 90.2	956.19 ± 118.1

Accuracy

Mean and standard deviation values for accuracy across tasks for both groups are given in Table 4. SRT was not assessed for accuracy as all participants scored 95–100%. There was a difference in baseline performance of CWM accuracy between groups trending towards significance (U = 65.5, Z = −1.95, p = 0.052). There were no significant differences in any other measure of baseline performance between groups. Wilcoxon Signed Ranks Test showed that the treatment group improved in CRT (Z = −2.40, p = 0.016, r = 0.60) and SWM (Z = −2.78, p = 0.005, r = 0.70). Participants did not show a significant improvement in CWM (Z = −1.83, p = 0.068), although there appeared to be a trend towards improved performance. The AC group was able to improve performance in the CRT task (Z = −2.05, p = 0.040, r = 0.62). However, there was no significant improvement in either SWM (Z = −0.55, p = 0.582) or CWM (Z = −0.71, p = 0.476).

Table 4.

Mean and standard deviations for accuracy across tasks for both groups.

Measure and condition	Baseline	Post-training
Complex Reaction Time
Treatment (N=16)	63.44 ± 17.3	75.63 ± 16.1
Active control (N=11)	61.67 ± 21.1	75.45 ± 16.3
Spatial Working Memory
Treatment (N=16)	71.09 ± 13.48	79.24 ± 12.32
Active control (N=12)	71.28 ± 13.5	70.54 ± 14.7
Contextual Working Memory
Treatment (N=16)	62.19 ± 19.32	71.56 ± 16.10
Active control (N=12)	76.25 ± 10.7	70.00 ± 15.2

Discussion

This study assessed the effect of 28-day multi-domain CT on un-trained cognitive tasks using a novel SRT active control group design. It was hypothesised that speed and accuracy improvements would be observed across all outcome measures in the CT treatment group, whilst the control group would improve only in reaction time performance on the SRT task. Contrary to our hypotheses, comparable improvements in reaction time were observed across groups in the SWM task, while a trend for improvement was seen in the CRT task. However, improved speed of response did not lead to more accurate responding equally across both groups. Only the CT treatment group demonstrated statistically significant improvements to accuracy in the more complex tasks of SWM and a trend was seen for CWM. Such findings raise interesting and important questions regarding the transferability of CT in addition to suggesting that the use of similar control group designs in future CT studies may be advantageous [25].

Previous research in CT has been criticised for a lack of adequate control group tasks [20,28], relating to broader concerns over the validity of many active control groups generally [21]. Numerous studies have employed a control task which is inadequate for differentiating cognitive gains due to CT from other factors such as quizzes, questionnaires or videos [22–24]. In our recent study using the same CT software, [18], we noted the limitations of using solitaire as the active control task, due to the fact that solitaire utilises a range of cognitive domains including executive functioning and problem solving, therefore limiting comparison between groups. In the current study however, participants in the control group were able to log in to a genuine CT software program where participants could track their reaction time improvements and read about the benefits of their training task to daily functioning, possibly providing further motivation to complete competitively. Additionally, this task was unlikely to transfer onto more executive domains where solitaire may, and thus provides further confidence in the observed improvements of the treatment group.

The finding that participants in the CT treatment group did not perform significantly faster than participants in the active control group across the outcome measures during follow-up, in comparison to baseline, can be interpreted in a number of ways. Firstly, such a finding may simply represent a practice effect equal across both groups, which was independent of the training tasks, whereby all participants habituated to the outcome measures and responded more proficiently at follow-up. However, considering that a passive control group was not included in the current study, the veracity of this interpretation cannot be assessed. What can be determined from the current study design is that none of the CT tasks in addition to SRT (which was common to both groups) resulted in relatively significant improvements to speed of response in followup outcome measures. However, it is most important that improvements in speed did not translate to similar improvements in accuracy for the control group. This is an intriguing finding, and not merely representative of a speed-accuracy trade-off as the treatment group became significantly faster at comparable levels. This may reflect changes to processing within white matter tracts [29] that did not affect the more cortical tasks as required for working memory [30], as neuroplastic changes are known to occur rapidly after intervention implementation [31]. However, neuroimaging would be needed to confirm this hypothesis. This finding is in agreement with the literature that show improvements are most substantial in cognitive domains in which the participant trains on. Therefore, multi-domain cognitive training is most likely to be efficacious in healthy populations, although more targeted training may be needed in clinical samples with specific deficits [e.g., 32].

The finding that spatial working memory showed significant and large improvements in accuracy for the treatment group provides support for the efficacy of computerised CT in healthy older adults over a short period of time. This is particularly important given the specific and exaggerated deficits known to be evident in older adults in this domain [19,33–35]. Therefore, as a task assessing this spatial WM was malleable to change, this provides further support for the efficacy of CT in healthy older adults [14].

The significant difference at baseline in the CWM task must be noted when interpreting the large mean improvement in the treatment group for this task. This difference appeared to be driven by two participants in the treatment group performing particularly poorly at baseline. Both participants showed improvements in the task following training, with one participant’s gains (30% to 80%) being substantially greater than the other (15% to 30%). As we had a small sample we chose not to remove these participants, but the non-parametric analyses can somewhat account for these outliers. Given the difference only trends towards significance, we suggest this improvement requires tentative interpretation.

Overall, we propose the current results support the efficacy of multi-domain training. Simple processing speed training did not result in far transfer onto memory, and we believe that short-term training is unlikely to yield these effects. In relation to real-world applications, for healthy older adults where significant declines may not yet have occurred, multi-domain training may be most efficacious.

Limitations

There are limitations to this study which must be noted. Firstly, follow-up assessments would have been beneficial to assess for the maintenance of effects, had the scope of the project allowed. A relatively small sample size in addition to lessened training intensity (non-adaptive tasks) also may have had an effect on the strength of our “findings, particularly where trending results were not statistically significant.

Conclusions

The findings of this study suggest that 28 days of once daily CT can lead to measurable improvements in non-practiced outcome measures in healthy older adults. These findings further add to the growing body of literature that suggest CT may be a valuable tool for improving or restoring specific cognitive abilities in healthy older adults. However, of key importance and novelty in the current study was that these improvements were in relation to a control group that may be deemed more reliable for comparison than many previous studies. We suggest this work highlights the importance of appropriate control groups, and suggest further studies implementing a basic CT task as an active control similar design are appropriate and warranted on a larger scale.