Toward personalized cognitive training in older adults: A pilot investigation of the effects of baseline performance and age on cognitive training outcomes

Abstract

Background:

Cognitive training holds potential as a non-pharmacological intervention to decrease cognitive symptoms associated with Alzheimer’s disease (AD), but more research is needed to understand individual differences that may predict maximal training benefits.

Objective:

We conducted a pilot study using a six-month training regimen in healthy aging adults with no cognitive decline. We investigated the effects of baseline performance and age on training and transfer improvements.

Methods:

Out of 43 participants aged 65–84 years, 31 successfully completed cognitive training (BrainHQ) in one of three cognitive domains: processing speed (N=13), inhibitory control (N=9), or episodic memory (N=9). We used standardized assessments to measure baseline performance and transfer effects.

Results:

All 31 participants improved on the cognitive training regimen and age was positively associated with training improvement (p=0.039). The processing speed group improved significantly across many near- and far-transfer tasks. In the inhibitory control group, individuals with lower baseline performance improved more on inhibitory control and cognitive flexibility tasks. In the episodic memory group, older individuals improved most on a memory task while younger individuals improved most on an executive function far-transfer task.

Conclusion:

Individual differences are predictive of cognitive training gains, and the impact of individual differences on training improvements is specific to the domain of training. We provide initial insight regarding how non-pharmacological interventions can be optimized to combat the onset of cognitive decline in older adults. With future research this work can inform the design of effective cognitive interventions for delaying cognitive decline in preclinical AD.

Introduction

Alzheimer’s disease (AD) is extremely costly with the 2020 total economic burden estimated at $305 billion [1]. Improving cognition for persons with AD and/or preventing cognitive decline in those at risk could result in greatly increased quality of life for those affected with this debilitating disease. Pharmacological treatments have, thus far, not been successful at restoring brain function in patients with AD [2]. Thus, the development of alternative, non-pharmacological methods such as cognitive training, which holds promise in delaying the clinical onset of AD, is an important focus of research [3].

Cognitive training is a program of guided mental activities designed to maintain or improve cognitive functions that support the accomplishment of everyday tasks and independent living [4]. It relies on functional and structural neural plasticity to build on existing resources and improve cognitive functioning. Interest in cognitive training for older adults has increased following findings that the brain is capable of significant plasticity up until very old age [5–8]. Research suggests that cognitive training can abate cognitive decline in aging individuals and may also improve cognitive performance in individuals with mild cognitive impairment [9].

However, results of cognitive training studies show great variability in terms of outcomes, suggesting the importance of considering individual differences when evaluating which individuals are best suited for cognitive training [10]. Prior research suggests that several factors including an individual’s mental status and compliance in implementing cognitive training strategies contribute to variability in response to training [11]. However, two factors in particular – baseline cognitive ability and age – may be highly predictive of cognitive training gains, especially with respect to memory training [12].

Two opposing theories have been proposed to explain the role of age and baseline cognitive ability on the effectiveness of cognitive training [13,14]. The compensation effect states that high-performing or younger individuals will benefit less from cognitive training because they are functioning at their optimal level and have less room for improvement. By contrast, the magnification effect postulates that high-performing or younger individuals who already perform well in a cognitive domain will benefit most from training in that domain because they may utilize their more efficient existing cognitive resources [13,15]. Both theories assume that younger individuals have undergone less cognitive decline than older adults and have a higher baseline cognitive profile. As there is ongoing debate about which theory is most accurate, more research is needed to delineate which of these theories explains variability in training improvements in each cognitive domain to better understand how training gains can be maximized given specific training conditions [14]. These theories provide keys to personalization of cognitive training in which the specific training regimen is customized to each individual’s unique cognitive profile. Such a personalized training approach may hold unique promise to stave off cognitive decline in persons with preclinical AD given the great heterogeneity seen in this disorder in terms of etiology and symptom profiles [16].

Here we present a pilot study in healthy aging adults with limited cognitive decline to better understand factors that could eventually be used to personalize treatment for those with preclinical AD. We assigned participants to domain-specific cognitive training groups for processing speed, inhibitory control, and episodic memory. Prior research demonstrates that all three of these domains represent promising avenues for cognitive training intervention with associated gains in activities of daily living [17], although processing speed and inhibitory control have not been investigated thoroughly in the context of individual differences. Within these domains, participants completed a training regimen of ~6 months, which contrasts with the majority of previous studies in which training durations were much shorter [14]. The ultimate goal of our study was to better understand ways in which we can personalize cognitive training to maximize benefits attained by each individual. Further, our unique design facilitates the comparison of improvements on post-test batteries across groups to disentangle treatment effects from placebo. We also investigate generalizability of improvements to different tasks within the same cognitive domain (near-transfer) and to tasks outside the cognitive domain (far-transfer). Given that improvements in near- and far-transfer tend to correlate with improvements in training – a correlation that is enhanced with longer cognitive training regimens [18] – we expect that the participants who improve most in training will also improve most in near- and far-transfer tasks.

In the domain of processing speed, given the similarities it may share with working memory in terms of a potential ceiling effect, especially in older adults, we expect that older individuals and those with lower baseline performance will improve most in cognitive training and derive the most near- and far-transfer benefits. We also expect to find similar support for the compensation effect theory in the domain of inhibitory control, as consistent with a recent study [19] that observed enhanced neural efficiency in participants with lower inhibitory control baseline ability. Finally, we hypothesize that in episodic memory training, younger individuals and those with higher baseline performance will improve most in training and in near- and far-transfer tasks, which is consistent with past literature and would support the magnification effect theory [14]. Ultimately, we hope that our research will help to expand the knowledge base regarding personalization of cognitive training to maximize its utility.

Materials and Methods

Participants

The current study enrolled 43 participants aged 65 and up who were free of memory complaints and demonstrated normal memory function (Logical Memory II subscale cutoff score of >=9, >=5 and >=3 for 16, 8–15 and 0–7 years of education, respectively) and cognition (absence of significant impairment in cognitive functions and intact Instrumental Activities of Daily Living). Additionally, participants were required to have a clinical dementia rating of 0 and Mini Mental State Examination (MMSE) score ≥ 24. Exclusion criteria were presence of suicidality, current regular use of psychiatric medications, opiates, or thyroid medications, claustrophobia, non-MRI-compatible materials, current substance abuse, post- traumatic or psychotic disorders, bipolar disorder, any significant neurologic disease, including possible and probable dementia, multi-infarct dementia, Parkinson’s or Huntington’s disease, brain tumor, progressive supranuclear palsy, seizure disorder, subdural hematoma, multiple sclerosis, “uncontrolled” hypertension, history of significant head trauma, history of alcohol or substance abuse or dependence within the past 2 years, or any significant systemic or unstable medical condition which could lead to difficulty complying with the training protocol. All procedures were done in accordance with the ethical standards of the Stanford Human Research Protection Program. Informed consent was provided by each participant as approved by the Institutional Review Board (IRB) of Stanford University. Participants received Amazon gift cards as an honorarium.

Baseline Neuropsychological Assessments

Participants completed The National Institute of Health (NIH) Toolbox Cognition Battery (nihtoolbox.org), an iPad-based series of assessments with good test-retest reliability [20,21]. The battery includes measures for cognitive flexibility, inhibitory control, working memory, processing speed, language, and episodic memory as well as crystalized and fluid cognition composite scores. The entire NIH toolbox battery was completed at baseline and post-training during an in-person laboratory visit and scores were used to determine baseline performance and to quantify improvements in training.

Train-to-the-Task Batteries and the Cognitive Training Regimen

Following completion of the NIH Toolbox baseline assessment, participants were assigned to one of three domains: processing speed, inhibitory control, or episodic memory (Figure 1). Assignment was based on baseline performance on the NIH Toolbox test corresponding with the cognitive domain of interest with the goal of having an equal number of higher (score > 1 SD above normative mean) and lower (score </= 1 SD above normative mean) performing individuals in each group. Participants were blinded with respect to cognitive training domain. Blinding of personnel and outcome assessors was done when possible but, due to limited staff availability, this was not always possible. Given that our primary results were within-domain group comparisons our results were not likely biased by knowledge of the treatment domain. Participants completed a BrainHQ 6-month computerized training regimen on their home computer for their assigned cognitive domain [22,23]. This long training duration was chosen to allow for maximal training benefits while minimizing the effects of confounding factors such as inconsistent training conditions, different training devices (i.e., desktop vs. laptop). Cognitive training using BrainHQ from Posit Science has been demonstrated to positively impact working memory and processing speed in older adults, with some evidence that these improvements may translate to improved everyday functioning [24]. Participants were asked to participate in the cognitive training regimen 2–3 times/week, for 20–30 minutes each training session. The regimen consisted of four unique exercises in the corresponding cognitive domain (see Figure 2b), which were adaptive to individual ability and increased in difficulty as participants progressed through the training. We selected a ‘fixed blocks’ schedule for the training design, in which every subject was administered the same number of levels of training and progressed across those levels equivalently assuming they completed an equal number of training sessions. Participants were not excluded for completing the training at a faster or slower pace than recommended, although completion of a minimum of 300 training exercises was required for inclusion in the study.

Figure 1.

Flow diagram of study assessments and training. T2T = train-to-the-task batteries.

Figure 2.

(a) The five train-to-the-task (T2T) batteries, one for each unique cognitive domain. * indicates non- overlapping exercises that were included in the analysis of improvement. (b) The games performed in the three unique cognitive training regimens from BrainHQ.

All participants also completed train-to-the-task (T2T) batteries across five domains (Figure 2a) to assess immediate pre- to post-training improvements in the domains of interest and in additional domains (working memory, cognitive flexibility). Participants were instructed to complete these batteries on their home computers during one sitting and within two days of the start and completion of the cognitive training regimen. For each domain, the T2T battery was similar in structure and format to the corresponding cognitive training regimen, allowing for assessment of near-transfer to structurally similar tasks, which would be the first indication of whether participants attained benefits of cognitive training extending beyond the training regimen.

Follow-up Neuropsychological Assessments

Following completion of the 6-month cognitive training and post-training T2T batteries, participants completed NIH Toolbox assessments to measure changes in cognitive function. However, due to safety constraints brought on by the coronavirus pandemic, several participants completed the NIH Toolbox assessments virtually at follow-up, which caused missing data for several tasks only possible through in-person assessment. Other reasons for dropout during the study included (1) frustration with the original difficulty of the tasks, (2) technical difficulties running the cognitive training exercises on participants’ home-computers, (3) inability to keep up with demands of the training schedule, and (4) non-training related life events.

Operationalization of Cognitive Training Improvements

To operationalize improvements in training, a z-score improvement metric developed by the BrainHQ data analysis team was utilized. For each training level, a ‘difference score’ was calculated, in which the original z-score attained on that level (the z-score achieved the first time that level was completed) was subtracted from the highest z-score attained on that same level. Thus, each training level for each training exercise had a ‘difference score’, and those difference scores were averaged over all the levels for each exercise. Thus, we calculated four difference scores for each participant (one for each training exercise) and averaged those four difference scores together to calculate one aggregate cognitive training z-score improvement metric.

Operationalization of Train-to-the-Task and NIH Toolbox Improvements

To operationalize improvements in the five T2T batteries from pre- to post-training, we averaged the z-scores attained on the four individual exercises for each T2T battery to calculate an average score for that battery. To calculate improvement, we subtracted the average z-score attained on each battery at pre-training from that at post-training. Given overlapping exercises between the inhibitory control and cognitive flexibility T2T batteries and between the working memory and episodic memory T2T batteries, we included only the unique, non-overlapping exercises for these batteries when calculating average scores to avoid potential practice effects (see Figure 2a).

To operationalize improvements in NIH Toolbox assessments, uncorrected standard scores (USS) were utilized for all tasks for which the USS was available. USSs are automatically calculated by NIH Toolbox software and compare the score of the test-taker to those in the entire NIH Toolbox nationally representative normative sample irrespective of demographic variables, which can be useful when monitoring performance over time [20,21]. For the Rey Auditory Verbal Learning and Oral Symbol Digit tasks, USSs were not available and raw scores were utilized. T2T and NIH Toolbox assessment scores were designated as outliers and excluded from the data if they differed by more than three standard deviations from the sample mean for the corresponding task.

Statistical Analyses

Due to the small sample size of this pilot study, the data did not meet assumptions of normality and we utilized two-sided Wilcoxon signed rank tests to quantify improvements in T2T and NIH Toolbox outcomes between pre- and post-training study timepoints across all participants and within each cognitive domain group. We also calculated Cohen’s d effect sizes to further quantify the magnitude of change. We performed linear regressions to determine whether factors such as age and baseline performance in each task and cognitive domain were predictive of training and transfer improvements for the group as a whole and within each cognitive domain. We included covariates including age, education, and number of training exercises completed when examining the impact of individual differences. To compare training and transfer gains between cognitive domain groups, we performed a repeated-measures ANOVA with two factors: (1) time (pre- and post-training) and (2) training group (three cognitive domains).

Results

Demographics

Out of 43 participants who participated in the study, 31 participants (24 females, 7 males) completed at least the cognitive training portion of the study (mean age = 73.19 years (range 65–84), mean education = 17.16 years (SD=2.91). Descriptive characteristics by training domain groups are presented in Table 1. There were no significant differences in sex distribution between any two groups (p’s>0.10). There was a significant difference in age between inhibitory control and processing speed groups (p=0.049, F(1,28) = 2.14, R²= 0.13). There was also a significant difference in years of education between the inhibitory control and processing speed (p=0.031, F(1,28) = 2.59, R² = 0.16). We found no significant between-group differences in baseline MMSE total score, LM-II average score, or NIH Toolbox task performance with the exception of the List Sort Working Memory task, on which the inhibitory control group scored significantly higher than the episodic memory group at baseline (p=0.046, F(1,24) = 2.82, R2 = 0.26).

Table 1.

Demographic characteristics, baseline cognitive scores and training durations across all participants and within the three cognitive domain groups. All values are mean (SD) with the exception of N. MMSE = Mini Mental State Examination total score. Logical Memory II is the average score. Training duration (exercises) = mean number of exercises completed during the training period. Training duration (days) is the mean number of days elapsed between the first and last training sessions.

	All Participants	Processing speed group	Inhibitory control group	Episodic memory group
Total N (N males)	31 (7)	13 (2)	9 (2)	9 (3)
Age (years)	73.19 (5.67)	71.04 (5.29)	75.89 (6.55)	73.58 (4.46)
Education (years)	17.16 (2.25)	18.0 (1.83)	15.89 (2.57)	17.22 (2.11)
MMSE	29.48 (0.72)	29.39 (0.87)	29.56 (0.73)	29.56 (0.53)
Logical Memory-II	12.16 (2.14)	12.23 (2.91)	12.72 (1.15)	11.50 (1.54)
Training Duration (exercises)	445.10 (74.28)	468.69 (75.08)	403.89 (57.52)	426.22 (61.69)
Traning duration (days)	159.19 (85.19)	162.62 (87.22)	179.22 (87.63)	134.11 (85.10)

Logistical cognitive training data

Across all participants, the mean training duration was 445.10 exercises (SD= 74.28, see Table 1). The mean time spent in training (time elapsed between first and last training session) was 159.19 days (SD=85.19). The number of exercises and time elapsed for each cognitive training group are detailed in Table 1.

All Participants Training Results

All 31 participants who completed the training regimen improved on the training (average z-score improvement = 1.07, SD = 0.412). Overall, we found that older age predicted training improvements (p=0.039, F(1,29)=4.68, R²=0.14) and improvement on the Total Cognition Composite metric following training at a trend level (p=0.069, F(1,25)=3.7, R²=0.23) (See Figure 3). Significant pre- to post-training improvement was found on the Fluid Cognition Composite metric (p<0.001, d = 0.88) and the Total Cognition Composite metric (p=0.001, d = 0.81). Improvements in the T2T batteries, NIH Toolbox tasks and moderators of improvement are presented in Table 2.

Figure 3.

Across all participants, age is positively associated with improvement on the (A) cognitive training regimen (p=0.039, F(1,29)=4.68, R²=0.14) and the (B) Total Cognition Composite metric (p=0.046, F(1,25)=4.41, R²=0.15). USS = uncorrected standardized score. Training improvements are normalized Z-scores.

Table 2.

Improvements on T2T batteries and NIH Toolbox tasks across all participants. Tasks corresponding to the training domains are presented in italics and underlined. Baseline performance (BP) effects are presented relative to the corresponding baseline performance on the task of interest.

Outcome type	Test	Improvement from baseline	Effect of baseline performancea	Effect of agea	Effect of cognitive domain groupa
Train to task
	Processing speed (N=25)	p=0.0054, d=0.63*	Neg, p=0.000012, F(1,19)=35, R^2=0.77	p=0.89, F(1,20)=0.02, R^2=0.37	p=0.037, F(1,20)=3.9, R^2=0.37
	Inhibitory control (N=26)	p=0.018, d=0.51**	Neg, p=0.0000063, F(1,20)=37, R^2=0.7	p=0.57, F(1,21)=0.33, R^2=0.15	p=0.52, F(1,21)=0.68, R^2=0.15
	Episodic memory (N=26)	p=0.17, d=0.35	Neg, p=0.00012, F(1,20)=23, R^2=0.7	p=0.82, F(1,21)=0.054, R^2=0.36	p=0.012, F(1,21)=5.5, R^2=0.36
	Working memory (N=24)	p=0.37, d=−0.25	Neg, p=0.021, F(1,18)=6.4, R^2=0.71	Neg, p=0.033, F(1,19)=5.3, R^2=0.61	p=0.001, F(1,19)=10, R^2=0.61
	Cognitive flexibility (N=22)	p=0.016, d=0.46**	p=0.11, F(1,16)=2.9, R^2=0.18	p=0.91, F(1,17)=0.013, R^2=0.031	p=0.81, F(1,17)=0.21, R^2=0.031
NIH Toolbox
	Fluid cognition composite (N=28)	p=0.00016, d=0.88**	Neg, p=0.0025, F(1,22)=12, R^2=0.4	p=0.64, F(1,23)=0.23, R^2=0.078	p=0.55, F(1,23)=0.61, R^2=0.078
	Total cognition composite (N=27)	p=0.001, d=0.81**	p=0.48, F(1,21)=0.51, R^2=0.25	Pos, p=0.069, F(1,22)=3.7, R^2=0.23	p=0.89, F(1,22)=0.12, R^2=0.23
	Crystalized cognition composite (N=27)	p=0.44, d=0.12	p=0.95, F(1,21)=0.0046, R^2=0.3	Pos, p=0.057, F(1,22)=4, R^2=0.3	p=0.16, F(1,22)=2, R^2=0.3
	Pattern comparison processing speed (N=29)	p=0.00038, d=0.86**	Neg, p=0.0016, F(1,23)=13, R^2=0.39	p=0.37, F(1,24)=0.82, R^2=0.05	p=0.99, F(1,24)=0.013, R^2=0.05
	Flanker inhibition (N=29)	p=0.004, d=0.59**	Neg, p=0.000038, F(1,23)=26, R^2=0.58	p=0.62, F(1,24)=0.26, R^2=0.1	p=0.32, F(1,24)=1.2, R^2=0.1
	Dimensional change card sort (N=29)	p=0.012, d=0.54**	Neg, p=0.00031, F(1,23)=18, R^2=0.45	p=0.88, F(1,24)=0.025, R^2=0.025	p=0.86, F(1,24)=0.16, R^2=0.025
	List sorting working memory (N=28)	p=0.19, d=0.3	Neg, p=0.00011, F(1,22)=22, R^2=0.54	p=0.42, F(1,23)=0.66, R^2=0.083	p=0.59, F(1,23)=0.53, R^2=0.083
	Picture sequence memory (N=29)	p=0.051, d=0.39*	Neg, p=0.15, F(1,23)=2.2, R^2=0.34	p=0.5, F(1,24)=0.48, R^2=0.28	p=0.2, F(1,24)=1.7, R^2=0.28
	Oral symbol digit (N=28)	p=0.044, d=0.36**	Neg, p=0.0065, F(1,22)=9.1, R^2=0.44	p=0.26, F(1,23)=1.3, R^2=0.21	p=0.4, F(1,23)=0.96, R^2=0.21
	Rey auditory learning (N=27)	p=0.011, d=0.57**	Neg, p=0.22, F(1,21)=1.6, R^2=0.4	Pos, p=0.0062, F(1,22)=9.2, R^2=0.35	p=0.049, F(1,22)=3.5, R^2=0.35

Neg = significant negative association; Pos = significant positive association.

^aResults include age and cognitive domain covariates in the model.

^**p<0.05;

^*p<0.10.

Processing Speed Group Training Results

13 out of 13 participants in the processing speed group improved significantly in the cognitive training regimen (mean z-score improvement = 0.73, SD = 0.075). A trend level negative association was found between processing speed baseline performance and training improvements (p=0.096, F(1,8)=3.35, R²=0.56) and age was not independently associated with improvements in training (p=0.49, F(1,11)=0.41, R²=0.075). Training duration (number of exercises) was also not an independent predictor of training improvements among all processing speed participants (p>0.10). The processing speed group improved significantly more in the processing speed T2T than the episodic memory and inhibitory control groups (See Figure 4). Finally, significant pre- to post-training improvement was found on the Fluid Cognition Composite metric (p=0.023, d_s = 0.84). Improvements in T2T batteries, NIH Toolbox tasks, and moderators of improvement are presented in Table 3 and Figure 5.

Figure 4.

Boxplots demonstrating enhanced processing speed train-to-the-task improvements in the processing speed (PS) group relative to episodic memory (EM) and inhibitory control (IC) groups (p=0.017, F(1,22)=4.97, R²=0.31). Central red line indicates the median, the bottom and top box edges indicate the 25th and 75th percentiles, the whiskers extend to the most extreme points not considered outliers, and the outliers of values more than 1.5 times away from the box edges are plotted using the “+” marker symbol.

Table 3.

Training results from the processing speed group. The tasks assessing processing speed are presented in italics and underlined. Baseline performance (BP) effects are first presented for the NIH Toolbox processing speed task: Pattern Comparison Processing Speed. Then, the effects of baseline performance on the task of interest (as indicated in the third column for each row) are presented.

Outcome type	Test of interest	Improvement from baseline	Effect of BP on the PS Toolbox taska	Effect of BP on the task of interest	Effect of age
Train to task
	Processing speed (N=11)	p=0.00098, d=1.9**	Pos, p=0.026, F(1,7)=4.33, R^2=0.55**	Neg, p<0.001, F(1,8)=30.6, R^2=0.88**	p=0.14, F(1,9) = 2.66, R^2=0.23
	Inhibitory control (N=13)	p=0.033, d=0.63**	Neg, p=0.055, F(1,9)=2.45, R^2=0.35*	p=0.12, F(1,10)=1.51, R^2=0.23	p=0.88, F(1,11) = 0.023, R^2=0.0021
	Episodic memory (N=12)	p=0.54, d=0.28	p=0.30, F(1,8)=0.66, R^2=0.14	Neg, p=0.030, F(1,9)=3.3, R^2=0.42**	p=0.99, F(1,10) = 0.00026, R^2<0.0001
	Working memory (N=10)	p=0.86, d=−0.025	p=0.91, F(1,7)=0.78, R^2=0.18	p=0.28, F(1,7)=1.59, R^2=0.31	p=0.22, F(1,8) = 1.76, R^2=0.18
	Cognitive flexibility (N=10)	p=0.22, d=0.26	p=0.62, F(1,6)=0.30, R^2=0.091	p=0.18, F(1,7)=1.12, R^2=0.24	p=0.93, F(1,8)=0.0089, R^2=0.0011
NIH Toolbox
	Fluid cognition composite (N=12)	p=0.023, d=0.84**	p=0.57, F(1,9)=0.82, R^2=0.16	p=0.13, F(1,9)=2.23, R^2=0.33	p=0.27, F(1,12)=1.40, R^2=0.12
	Total cognition composite (N=12)	p=0.057, d=0.69*	p=0.46, F(1,9)= 3.22, R^2=0.42	p=0.35, F(1,9)=3.52, R^2=0.44	Pos, p=0.033, F(1,10)=6.10, R^2=0.38**
	Crystalized cognition composite (N=12)	p=0.21, d=0.32	p=0.52, F(1,9)=0.43, R^2=0.088	p=0.23, F(1,9)=1.06, R^2=0.19	p=0.52, F(1,10)=0.52, R^2=0.042
	Pattern comparison processing speed (N=12)	p=0.0098, d=0.91**	p=0.14, F(1,9)=1.33, R^2=0.23	--	p=0.93, F(1,10)=0.0086, R^2=0.0086
	Flanker inhibition (N=12)	p=0.002, d=1.4**	p=0.35, F(1,9)=0.98, R^2=0.18	p=0.65, F(1,9)=0.57, R^2=0.11	p=0.34, F(1,10)=1.00, R^2=0.091
	Dimensional change card sort (N=12)	p=0.082, d=0.65*	p=0.94, F(1,9)=0.31, R^2=0.065	Neg, p=0.025, F(1,9)=4.19, R^2=0.48**	p=0.41, F(1,10)=0.73, R^2=0.068
	List sorting working memory (N=12)	p=0.32, d=0.36	p=0.49, F(1,9)=1.68, R^2=0.27	p=0.14, F(1,9)=3.01, R^2=0.40	p=0.11, F(1,10)=2.99, R^2=0.23
	Picture sequence memory (N=12)	p=0.69, d=−0.12	Neg, p=0.095, F(1,9)=2.55, R^2=0.36*	Neg, p=0.040, F(1,9)=3.84, R^2=0.46**	p=0.28, F(1,10)=1.30, R^2=0.12
	Oral symbol digit (N=12)	p=0.11, d=0.41	p=0.29, F(1,9)=0.22, R^2=0.28	Neg, p=0.022, F(1,9)=5.72, R^2=0.56**	p=0.16, F(1,10)=2.26, R^2=0.19
	Rey auditory learning (N=12)	p=0.0093, d=1.1**	p=0.27, F(1,9)=0.75, R^2=0.14	p=0.62, F(1,9)=0.19, R^2=0.04	p=0.73, F(1,10)=0.13, R^2=0.013

Neg = significant negative association; Pos = significant positive association. PS= processing speed.

^aResults include age and training duration covariates in the model.

^**p<0.05;

^*p<0.10.

Figure 5.

Boxplots summarizing pre- vs post-training scores for the processing speed group on the NIH Toolbox A) Patten comparison processing speed task (p=0.00982, d_s = 0.91 T=3.15, CI=2.74–15.43), B) Flanker Inhibitory Control tasks (p=<0.002, d_s = 1.40 01, T=4.69, CI=2.35 – 6.49) and C) Fluid CognitionIntelligence Composite score (p=0.02314, d_s = 0.84 T=2.92, CI=1.02 – 7.31). USS = uncorrected standard score.

Inhibitory Control Group Training Results

9 out of 9 participants in the inhibitory control group improved (mean z-score improvement =1.03, SD = 0.15).. Inhibitory control baseline performance was positively associated with training improvements (p=0.030, F(1,5)=3.34, R²=0.67), while age was not independently associated with training improvements (p>0.10). No other predictors of training improvement were significant among inhibitory control participants. Significant pre- to post-training improvement was found on the Fluid Cognition Composite metric (p=0.031, d_s = 0.85) and the Total Cognition Composite metric (p=0.031, d_s = 0.95). Improvements in T2T batteries, NIH toolbox tasks and moderators of improvement are presented in Table 4. The association between baseline inhibitory control performance and improvements in the NIH Toolbox Flanker Inhibitory Control task is shown in Figure 6.

Table 4.

Training results from the Inhibitory Control group. The tasks corresponding to the T2T inhibitory control training domain and NIH Toolbox inhibitory control are presented in italics and underlined. Baseline performance (BP) effects are first presented for the inhibitory control task from the NIH Toolbox: Flanker Inhibition. Then, the effects of baseline performance on the task of interest (as indicated in the second column for each row) are presented.

Outcome type	Task of interest	Improvement from baseline	Effect of BP on the IC Toolbox taska	Effect of BP on the task of interest taska	Effect of age
Train to task
	Processing speed (N=8)	p=0.25, d=0.51	p=0.48, F(1,5)=0.29, R^2=0.11	Neg, p=0.040, F(1,5)=3.79, R^2=0.60**	p=0.98, F(1,6)=0.00059, R^2<0.0001
	Inhibitory control (N=6)	p=0.094, d=1.0*	p=0.16, F(1,3)=1.79, R^2=0.54	Neg, p=0.059, F(1,3)=4.57, R^2=0.75*	p=0.79, F(1,4)=0.083, R^2=0.020
	Episodic memory (N=7)	p=0.16, d=−0.52	p=0.57, F(1,4)=0.42, R^2=0.17	p=0.13, F(1,4)=2.18, R^2=0.52	p=0.51, F(1,5)=0.51, R^2=0.09
	Working memory (N=7)	p=0.3, d=0.47	p=0.77, F(1,4)=1.25, R^2=0.39	p=0.37, F(1,4)=2.0, R^2=0.50	p=0.15, R^2=0.27, F(1,5) = 2.94
	Cognitive flexibility (N=5)	p=0.13, d=0.96	Neg, p=0.0018, F(1,2)=295, R^2=0.997**	p=0.84, F(1,2)=0.072, R^2=0.067	p=0.74, F(1,3)=0.13, R^2=0.042
NIH Toolbox
	Fluid cogntion composite (N=8)	p=0.031, d=0.85**	Neg, p=0.068, F(1,5)=3.28, R^2=0.57*	Neg, p=0.010, F(1,5)=2.56, R^2=0.51**	p=0.44, F(1,6)=0.69, R^2=0.10
	Total cognition composite (N=7)	p=0.031, d=0.95**	p=0.24, F(1,4)=2.37, R^2=0.54	p=0.11, F(1,4)=4.19, R^2=0.68	p=0.18, F(1,5)=2.41, R^2=0.33
	Crystalized cognition composite (N=7)	p=0.19, d=0.64	p=0.40, F(1,4)=0.077, R^2=0.72	Neg, p=0.090, F(1,4)=11.2, R^2=0.85*	Pos, p=0.026, F(1,5)=9.76, R^2=0.66**
	Pattern comparison processing speed (N=9)	p=0.063, d=0.74*	p=0.78, F(1,6)=1.18, R^2=0.28	Neg, p=0.040, F(1,6)=5.85, R^2=0.66**	p=0.15, R^2=0.27, F(1,7)=2.63
	Flanker inhibition (N=9)	p=0.13, d=0.63	Neg, p<0.001, F(1,6)=29.2, R^2=0.91**	--	p=0.22, F(1,7)=1.83, R^2=0.21
	Dimensional change card sort (N=9)	p=0.29, d=0.42	p=0.27, F(1,6)=1.1, R^2=0.27	Neg, p=0.027, F(1,6)=4.99, R^2=0.63**	p=0.43, F(1,7)=0.70, R^2=0.091
	List sorting working memory (N=8)	p=0.92, d=0.11	p=0.19, F(1,5)=1.27, R^2=0.34	Neg, p=0.021, F(1,5)=5.80, R^2=0.70**	p=0.69, F(1,6)=0.17, R^2=0.028
	Picture sequence memory (N=9)	p=0.098, d=0.68*	p=0.14, F(1,6)=3.03, R^2=0.50	p=0.74, F(1,6)=1.12, R^2=0.27	p=0.16, F(1,7)=2.42, R^2=0.26
	Oral symbol digit (N=8)	p=0.055, d=0.62*	p=0.48, F(1,5)=0.29, R^2=0.11	Neg, p=0.054, F(1,5)=3.15, R^2=0.56*	p=0.995, F(1,6)<0.0001, R^2<0.0001
	Rey auditory learning (N=7)	p=0.95, d=0.12	Neg, p=0.092, F(1,4)=8.9, R^2=0.82*	Pos, p=0.056, F(1,4)=11.6, R^2=0.85*	Pos, p=0.043, F(1,5) =7.26, R^2=0.59**

Neg = significant negative association; Pos = significant positive association. IC = inhibitory control.

^aResults include age and cognitive domain covariates in the model.

^**p<0.05;

^*p<0.10.

Figure 6.

Baseline performance in inhibitory control is negatively associated with improvement on the NIH Toolbox Flanker Inhibitory Control task (p<0.001, F(1,6) = 29.2, R² = 0.91). USS = uncorrected standardized score.

Episodic Memory Group Training Results

9 out of 9 participants in the episodic memory group improved in the cognitive training regimen, (mean z-score improvement = 1.62, SD =0.27). The association between episodic memory baseline performance and training improvements was not significant (p=0.10, F(1,4)=1.56, R²=0.54). Age did not independently predict improvement in training (p>0.10). No other predictors of training improvement were significant among episodic memory participants. Significant pre- to post-training improvement was found on the Fluid Cognition Composite metric (p=0.047, d = 0.994). Improvements in T2T batteries, NIH Toolbox tasks, and moderators of improvement are presented in Table 5. Age was positively associated with improvement on the Rey Auditory Verbal Learning Memory task (p=0.019, R²=0.63, F(1,6)=10.3) and negatively associated with improvement on the Dimensional Card Sort Executive Function task (p=0.035, R²=0.55, F(1,6)=7.34), as shown in Figure 7.

Table 5.

Training results from the Episodic Memory group. The tasks corresponding to the T2T episodic memory training domain and NIH Toolbox inhibitory control are presented in italics and underlined. Baseline performance (BP) effects are first presented for the inhibitory control task from the NIH Toolbox: Picture sequence memory. Then, the effects of baseline performance on the task of interest (as indicated in the second column for each row) are presented.

Outcome type	Task of interest	Improvement from baseline	Effect of BP on the EM Toolbox task*	Effect of BP on the task of interest taska	Effect of age
Train to task
	Processing speed (N=6)	p=0.69, d=−0.12	p=0.26, F(1,3) =0.99, R^2=0.40	Neg, p=0.098, F(1,3)=2.91, R^2=0.66*	p=0.80, R^2=0.018, F(1,4) = 0.074
	Inhibitory control (N=7)	p=0.81, d=0.21	p=0.93, F(1,4)=2.10, R^2=0.51	Neg, p<0.001, F(1,4) = 134, R^2=0.99**	Neg, p=0.071, F(1,5) = 5.22, R^2 = 0.51*
	Episodic memory (N=7)	p=0.031, d=1.1**	p=0.58, F(1,4)=3.49, R^2=0.64	Neg, p=0.045, F(1,4)=4.15, R^2=0.68**	p=0.96, R^2=0.00061, F(1,5) = 0.0030
	Working memory (N=7)	p=0.063, d=−1.11*	p=0.62, F(1,4)=0.32, R2=0.14	Neg, p=0.020, F(1,4)=7.86, R^2=0.80**	p=0.56, F(1,5) = 0.39, R^2=0.073
	Cognitive flexibility (N=7)	p=0.16, d=0.6	p=0.84, F(1,4)=0.054, R2=0.026	p=0.54, F(1,4)=0.26, R^2=0.11	p=0.80, F(1,5) = 0.072, R^2=0.014
NIH Toolbox
	Fluid cogntion composite (N=8)	p=0.047, d=0.99**	p=0.84, F(1,5)=0.53, R^2=0.18	Neg, p=0.10, F(1,5)=2.9, R^2=0.54	p=0.31, F(1,6) = 1.21, R^2 = 0.17
	Total cognition composite (N=8)	p=0.094, d=0.77*	p=0.88, F(1,5)=0.14, R^2=0.053	p=0.78, F(1,5)=0.17, R^2=0.064	p=0.60, F(1,6)=0.60, R^2=0.048
	Crystalized cognition composite (N=8)	p=0.27, d=−0.33	p=0.83, F(1,5)=0.23, R^2=0.083	p=0.29, F(1,5)=0.97, R^2=0.28	p=0.52, F(1,6) = 0.52, R^2=0.074
	Pattern comparison processing speed (N=8)	p=0.078, d=0.88*	p=0.74, F(1,5)=0.30, R^2=0.11	p=0.33, F(1,5)=0.86, R^2=0.26	p=0.49, F(1,6) = 0.55, R^2=0.084
	Flanker inhibition (N=8)	p=0.7, d=0.11	p=0.99, F(1,5)=2.50, R^2=0.50	Neg, p=0.016, F(1,5)=15.2, R^2=0.86**	Neg, p=0.050, F(1,6) = 6.01, R^2=0.50*
	Dimensional change card sort (N=8)	p=0.23, d=0.47	p=0.21, R^2=0.68, F(1,5)=5.39	p=0.24, F(1,5)=5.02, R^2=0.67	Neg, p=0.035, F(1,6) = 7.34, R^2=0.55**
	List sorting working memory (N=8)	p=0.13, d=0.55	p=0.81, F(1,5)=0.041, R^2=0.016	Neg, p=0.0074, F(1,5)=9.47, R^2=0.79**	p=0.89, R^2=0.0033, F(1,6) = 0.020
	Picture sequence memory (N=8)	p=0.078, d=0.99*	p=0.13, F(1,5)=1.71, R^2=0.41	--	p=0.87, R^2=0.0046, F(1,6) = 0.028
	Oral symbol digit (N=8)	p=0.97, d=0.028	p=0.52, F(1,5)=0.67, R^2=0.21	p=0.15, F(1,5)=2.04, R^2=0.45	p=0.37, R^2=0.14, F(1,6) = 0.94
	Rey auditory learning (N=8)	p=0.22, d=0.51	p=0.53, F(1,5)=4.89, R^2=0.66	p=0.11, F(1,5)=9.3, R^2=0.79	Pos, p=0.019, R^2=0.63, F(1,6) = 10.3**

Neg = significant negative association; Pos = significant positive association. EM = episodic memory.

^aResults include age and cognitive domain covariates in the model.

^**p<0.05;

^*p<0.10.

Figure 7.

Executive functioning transfer improvements in the episodic memory group are associated with age. A) Age is positively associated with Improvement on the NIH Toolbox Rey Auditory Verbal Learning Memory task (p=0.019, R²=0.63, F(1,6)=10.3). B) Age is negatively associated with improvement on the NIH Toolbox Dimensional Card Sort Executive Function task (p=0.035, R²=0.55, F(1,6)=7.34). USS = uncorrected standardized score.

Discussion

Overall, our pilot study suggests that cognitive training is particularly beneficial to individuals with lower baseline cognitive ability. Among all participants, we found evidence for increased improvements older individuals, generally supporting the compensation effect theory. When analyzing the results of each cognitive domain group individually, our findings suggest that the impact of individual differences is domain specific. In the domain of processing speed, participants improved significantly across a range of near- and far- transfer tasks. Processing speed cognitive training may be particularly beneficial to older individuals, as observed through increased improvements in general cognition. In the domain of inhibitory control, our data also suggests particular benefit to individuals with low baseline performance through increased improvement in an inhibitory control near-transfer task. Finally, in the domain of episodic memory, older individuals seemed to improve most in memory ability, while younger individuals experienced greater improvements in far-transfer tasks assessing executive function.

Training Improvements Across all Participants: Enhanced Benefits in Older Individuals

The association between age and training improvements suggests that cognitive training was particularly beneficial for older individuals, supporting the compensation effect theory. This finding suggests that cognitive training may be especially beneficial for individuals who, as indicated in previous research, might have less capability for cognitive plasticity [6]. Moreover, enhanced improvements in older adults highlights the potential utility of BrainHQ training among individuals experiencing cognitive aging or mild cognitive impairment. Future studies should also examine the impact of baseline intelligence on BrainHQ cognitive training improvements to better understand other individual differences predictive of training gains.

Near- and Far-Transfer Effects Across All Participants: Robust Improvements on a Variety of Transfer Tasks

The NIH Toolbox is useful for assessing near- and far-transfer effects to structurally dissimilar tasks, as participants complete exercises that are different in format and structure to those completed during cognitive training (under the supervision of a trained administrator on an iPad as opposed to on their own computer). In general, we found robust improvements across all participants in a wide variety of NIH Toolbox tasks (see Table 2). These findings are encouraging as they suggest that functional plasticity in older adults can be induced in several cognitive domains. Further investigation is required to separate out potential practice effects, although the six-month long training duration decreases the likelihood of a significant practice effect. Moreover, lack of improvement in the crystallized intelligence composite metric, which is composed of repeated tasks in aspects of cognition not targeted by the cognitive training regimen, suggest that these improvements may reflect true benefits to cognitive function rather than improvements attained due to performing tasks more than once.

We also found that older age predicted improvement on the NIH Toolbox Total Cognition Composite score across all participants, suggesting that older individuals may have attained the most transfer benefits from cognitive training, which are more likely to be indicative of real-life cognitive improvements. While older adults have been observed to attain equivalent near- and far-transfer gains after cognitive training [25], few studies have found a positive association between age and transfer effects (see Table 2). One such study showed enhanced transfer effects in older adults in a digit span forward task and a delayed recall task following working memory training [26]. However, older age has generally not been observed to be positively associated with improvements in transfer tasks, which emphasizes the potential utility of BrainHQ online cognitive training regimens for lower-functioning individuals.

Training and Far-Transfer Improvements in the Processing Speed Group

As expected, individuals in the processing speed group improved on the processing speed T2T battery between pre- and post-training but, more importantly, they improved significantly more than the episodic memory (and a trend for more than the inhibitory control) group. This finding suggests the improvement in the processing speed group may represent a true treatment effect, and not a placebo effect. Moreover, the processing speed group also demonstrated significant improvements in the inhibitory control T2T battery indicating that this group gained benefits that extended beyond the cognitive domain being tested. We also found that the processing speed group improved on the near-transfer Pattern Comparison Processing Speed task, two far-transfer tasks with high inhibitory control demands (Flanker Inhibitory Control and Dimensional Change Card Sort), and the Rey Auditory Verbal Learning Memory task which measures verbal memory, efficiency of learning and the effects of interference. One potential explanation for this far-transfer gain in the Rey Auditory Verbal Learning Memory task is that improvements in processing speed transfers to efficiency of learning.

The far-transfer tasks (NIH Toolbox) for which participants improved most were assessments of fluid intelligence and, accordingly, there was a significant improvement in the Fluid Cognition Composite score. The phenomenon of improved fluid intelligence following training in processing speed may be explained by the processing-speed theory of cognitive aging. The theory states that fluid cognitive performance degrades in old age due to a decrease in processing speed that (1) limits the ability to execute successful operations and (2) reduces the amount of simultaneously available information needed for higher level processing [27]. Thus, improvements in processing speed ability may lead to a corresponding increase in fluid intelligence due to an increased ability to execute successful operations quickly and access the information necessary for higher level processing. The association between processing speed and fluid intelligence ability has been supported by past research [28] and may translate to improvements in activities of daily living [29]. Finally, a positive association between age and improvements in the Total Cognition Composite metric suggests that older individuals with less cognitive reserve derive the most cognitive benefits from processing speed training. This finding indicates a compensation effect in the domain of processing speed and supports the hypothesis that plasticity can be induced in the oldest individuals who may need it most.

Near- and Far-Transfer Improvements in the Inhibitory Control Group: A Potential Compensation Effect

Improvement on the inhibitory control T2T battery only reached trend level significance, likely due to low sample size (N=6 and N=9, respectively). Generally, the results within the inhibitory control group supported the occurrence of a compensation effect: individuals with lower baseline performance improved most on a near-transfer measure of inhibition, the NIH Toolbox Flanker Inhibitory Control task. The positive association between inhibitory control baseline performance and training improvements was surprising and suggests that improvements on the training regimen do not necessarily reflect benefits to cognitive function. Nonetheless, our results generally suggest that inhibitory control training may be beneficial to lower functioning individuals. Improvements in inhibitory control, a key aspect of executive functioning, are highly relevant given past research linking executive functioning abilities and performance on activities of daily living in AD [17,30].

Training Improvements in the Episodic Memory Group: Domain-Specific Benefits

In the episodic memory group, older and low baseline-performing individuals seemed to derive the most benefits in memory, observed through a positive association between age and improvements on the Rey Verbal Auditory Learning Memory task. On the other hand, younger individuals seemed to derive enhanced far-transfer benefits in executive function, as observed through enhanced improvements on the Flanker Inhibitory Control task and the Dimensional Change Card Sort task (with high cognitive flexibility demands). The present study is one of the first to report enhanced memory gains in lower functioning individuals following episodic memory cognitive training. These findings are relevant given past literature highlighting the strong association between memory progression and functional impairment in adults with MCI and AD as well as the promise of episodic memory cognitive training in mitigating these negative outcomes [31]. Previous work reported a strong magnification effect, in which between-person differences in episodic memory ability were magnified following repeated cognitive training [15], consistent with the majority of past studies.

The supply-demand mismatch hypothesis [32] may help explain the memory findings observed following episodic memory training. In this model, a mismatch between available cognitive resources and task demands promotes changes in cognitive performance. When the amount of mismatch is ideal, repeated performance in training tasks induces neural changes that ultimately promote cognitive plasticity. In general, episodic memory training regimens, such as training in the Method of Loci, are more difficult and favor high baseline performing individuals. However, it is possible that for high-functioning individuals the BrainHQ episodic memory regimen was not difficult enough to induce the ideal amount of mismatch in these individuals. Thus, these individuals may have increased their available cognitive resources more quickly than the task increased in difficulty, attaining less improvement in memory ability.

Limitations and Future Directions

The primary goal of our study was to better understand the effects of baseline and age on training and transfer improvements. Thus, we did not include a waitlist control group and were unable to control for potential placebo effects nor could we account for natural performance decline associated with cognitive aging. Additionally, small sample sizes within each cognitive domain group limit the ability to draw definitive conclusions from our data, however, we hope that our preliminary findings help inform future hypotheses. We were also unable to fully account for practice effects, which may arise when performing an exercise for the second time at post-training. However, we expected that the duration between pre- and post-training (mean=159.19 days) would minimize practice effects. Practice effects on the T2T exercises were also a concern, as several tasks from the T2T batteries were repeated across different cognitive domains. Given that completing similar tasks in succession could influence participants’ performance, these repeating tasks were removed when calculating scores on the T2T batteries and the order of completion of T2T batteries was randomized across participants using Excel software. Future studies with active control groups will be important to better quantify training-induced participant gains vs. practice effects and other placebo effects.

Individuals function optimally at different paces, thus we allowed for fluctuations in the training pace and duration, including duration of breaks. Additionally, for some participants, the T2T baseline batteries were completed after starting the training regimen or they were completed more than once. Given the pilot nature of our study and the goal to maximize the amount of data available for analysis, these variations were necessary limitations. Another limitation is that this pilot study was not pre-registered.

It is also important to note that age and education differed significantly between the inhibitory control and processing speed groups. Since age and education are known to impact cognitive function, we included age and education as covariates in analyses of all participants. We also confirmed that all participants were free from memory impairments (MMSE score >/=24) and that groups were matched for memory performance (LM-II scores) at baseline. Thus, our results were not likely driven by age-related cognitive decline.

The sample size for this pilot study was chosen based on our previous cognitive training data in older adults (unpublished) which indicated a medium effect of improvement in fluid cognition for the cognitive training group (Cohen’s d = 0.7, N=20). Thus we planned to enroll a moderate sample size of N=60 participants but we were unable to reach this target due to the coronavirus pandemic. The pandemic further impacted the some participants’ ability to attend post-study visits to complete their follow-up NIH Toolbox assessments, although some of these participants were able to complete an adapted virtual NIH Toolbox assessment`. Although the three cognitive training groups did not differ on sex, our study did include a higher proportion of females, which limits generalizability. Average education level was also high across all participants (average = 17.2 years), which limits the generalizability of these findings to individuals with lower education levels for whom rates of mild cognitive impairment and AD are highest. Finally, the use of z-scores allowed us to compare changes in each cognitive training regimen and T2T tasks but, due to the variability across domains and exercises, z-scores were not able to correct for all potential biases and we interpret the between-domain comparisons with caution.

Conclusions

In conclusion, our study suggests that individual differences are acutely important in determining cognitive training gains, and that the impact of individual differences on training improvements is specific to the domain of training. In the processing speed domain, individuals improved on near- and far-transfer tasks including measures of inhibition and fluid intelligence, indicating cross-domain gains potentially due to increased ability to execute successful operations quickly and access the information necessary for higher level processing. Additionally, older individuals derived the most benefits in general cognition, supporting the compensation effect. In the domain of inhibitory control, the impact of baseline performance was mixed; however, the dominant findings suggest that individuals with low baseline performance derived the most benefits. Finally, in the domain of episodic memory, our findings contrast past research that comprehensively supported enhanced benefits from episodic memory cognitive training in individuals who are younger and have higher baseline performance, as both younger and older individuals showed enhanced improvements in specific far-transfer tasks after the cognitive training regimen. Further research with larger sample sizes will help clarify our findings in the domains of inhibitory control and episodic memory and provide more insight into how the personalization of cognitive training can help to maximize gains in each individual and slow down the cognitive aging process. More generally, our findings suggest that lower functioning individuals benefit most from cognitive training, which is promising and suggests that a similar intervention may be beneficial for individuals with mild cognitive impairment. With further research this work can be utilized toward the development of personalized cognitive training for individuals with preclinical AD and/or mild cognitive impairment.

Data availability

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.