The ACTIVE Cognitive Training Interventions and Trajectories of Performance among Older Adults

Abstract

Objectives

Salthouse (2006) illustrated that among Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) randomized controlled trial participants, the pace of cognitive change over time accelerated for persons who had participated in training. Our goal was to determine if the pace of cognitive aging, net of effects due to practice, training, and loss of training gains, differed for persons who received training.

Methods

We evaluated change in cognitive performance over five years following brief cognitive training among older adults (N=1,659, age 65-94) in ACTIVE using a latent growth curve model.

Results

Reasoning training, but not memory or speed, attenuated aging-related change. But this model modification produced instability and was not statistically significant. Memory gains were maintained throughout follow-up. About half of reasoning and speed gains were lost, however all trained groups performed better than controls at 5 years. Performance differences at the end of the follow-up were equivalent to about 6, 4, and 8 years of aging for memory, reasoning and speed training, respectively.

Discussion

Training can appear to accelerate age-related change, because change over time is coupled with loss of training gains. Of the three training interventions, only reasoning training appeared to attenuate the pace of normative decline. However, our analysis is limited by follow-up that is short for precisely characterizing aging-related change.

Introduction

Boosting cognitive performance has implications for successful cognitive aging and neurodegenerative disease. An Alzheimer's disease prevention program delaying onset by only one year would cut the prevalence by more than a third by 2050 (Brookmeyer, Johnson, Ziegler-Graham, & Arrighi, 2007). In support of the search for such a program, the National Institute of Aging and National Institute of Nursing Research have supported a multi-site randomized controlled trial of cognitive training: the ACTIVE study (Advanced Cognitive Training for Vital and Independent Elderly) since 1998. ACTIVE tests the effectiveness of community-based training in three areas of cognitive functioning (logical reasoning, memory, and speed of visual processing) in improving cognitive performance, performance of everyday activities, health-related quality of life, mobility, and health-service utilization (Jobe et al., 2001).

In this article we attempt to characterize the impact of ACTIVE training interventions by evaluating differences between trained and non-trained participants in the slope associated with aging of performance decrements in trained cognitive abilities. Recently, Salthouse (2006) illustrated that the pace of cognitive change over time following the ACTIVE intervention was accelerated for persons who had participated in training. The goal of the current study is to probe this observation. Specifically, we address what Salthouse' called the critical question:

The critical question in the current context is therefore not the magnitude, nor the durability, of training effects, but rather the influence of the relevant experience on the rate of change in measures of cognitive functioning over time. (p74)

We address this question using multiple group growth curve models that decompose change over time into four distinct sources of change. Two of these are theoretically at work in both the intervention and control groups, include a retest or practice effect and age-related or maturational effect. The practice effect describes gains due to familiarity of the testing situation or content caused by repeat testing. Maturational change describes normal aging-related changes. The remaining three sources of change we characterize are evident in the ACTIVE trained group only. These include ACTIVE's symptomatic effect (initial boosting of performance of trained abilities), the pace at which these initial gains are lost with increasing time from the initial training (a loss of training gains effect) and an age modifying effect (altering the trajectory of performance declines with age). The extent that ACTIVE modifies the aging effect is evident from differences in the maturational effect for control and trained participants. Our analytic approach is an extension of latent growth curve models for evaluating randomized trials (Curran & Muthén, 1999; Muthén & Curran, 1997) and longitudinal cognitive performance data (Ferrer, Salthouse, McArdle, Stewart, & Schwartz, 2005; McArdle & Epstein, 1987). Our modeling is a direct test of Salthouse's challenge.

Most studies of cognitive training are not comparable to ACTIVE for the purpose of evaluating the long-term effect of cognitive training on age trends for cognitive abilities. Smith and colleagues present the results of the IMPACT (Improvement in Memory with Plasticity-based Adaptive Cognitive Training), a large randomized controlled two-arm clinical trial of the Brain Fitness Program (Posit Science, San Francisco, CA) (Smith et al., 2009). They only report pre-and post- training differences over a brief time interval. The same is true for Owen and colleagues report on their ‘Bang Goes the Theory’ training (Owen et al., 2010) and Miller and colleagues' Memory Fitness Program (Miller et al., 2011). McDougall and colleagues report 26 month follow-up from their SeniorWISE memory training program, but do not evaluate training related differences in aging trends (McDougall Jr et al., 2010). Several other recently reported trails suffer the same lack of follow-up and/or evaluation of aging trends (Peretz et al., 2011; Pressler et al., 2011; Schmiedek, Lövdén, & Lindenberger, 2010). Perhaps the most relevant data come from observational studies of the effect of lifestyle and background characteristics on aging-related cognitive trends. The evidence for one powerful cross-sectional predictor of cognitive performance -- educational attainment -- is decidedly pessimistic with regard to benefits on aging-related cognitive trends (Glymour, Tzourio, & Dufouil, 2012). In conclusion, the field is currently without a good answer to the question: can cognitive training delay the natural course of cognitive aging? Observational studies suffer important methodological limitations and biases (Glymour, et al., 2012), and apart from ACTIVE, intervention trials do not have the duration of follow-up to address the question.

Therefore, our goal was to attempt to address the question of how the ACTIVE cognitive training programs influence the pace of cognitive aging. Although the impact of ACTIVE on cognitive performance has been described previously (Willis et al., 2006), previous analyses have restricted attention to total performance over time. Our approach is distinctly different because we attempt to decompose training related effects into those that reflect immediate training gains, the loss of those training gains as time from training increases, and differences in aging-related trends.

Methods

The design of ACTIVE has been described elsewhere (Jobe et al., 2001). Briefly, older adults (aged 65-94) were randomly assigned to one of three cognitive training or no contact control arms. Training lasted five to six weeks, and participants were assessed pre and post-intervention, and at one, two, three, five and 10 years after post-test. This analysis considers outcomes through 5 years, as the 10 year main results are currently under analysis.

Participants

From March 1998 through October 1999, participants were enrolled across six field sites. Each field site had a different recruitment strategy. Participants were drawn from motor vehicle license or identification card rolls, clinical patient rolls, elder services rolls, congregate and senior housing sites, senior and community centers, research volunteer registries, churches and senior citizens organizations (Jobe, et al., 2001). Exclusion criteria included age less than 65 years, substantial functional impairment, Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975) score less than 23, self-reported diagnosis of Alzheimer's disease, sensory impairment, inability to communicate in spoken English, medical conditions with poor prognosis, and previous participation in cognitive training. As reported by Willis and colleagues, (Willis, et al., 2006) 67% of the sample was retained through the fifth annual follow-up, and retention was related to age, sex, education, health problems, and baseline cognitive function. Nevertheless, as pointed out by Willis and colleagues, treatment group did not interact with these covariates and therefore do not affect the assessment of group differences of intervention effects (Willis, et al., 2006). The protocol and informed consent procedure was approved by local review boards and the trial monitored by a single Data Safety and Monitoring Board. Characteristics of persons included in our analysis are summarized in Table 1.

Table 1.

Baseline Characteristics of Participants by ACTIVE Intervention Group, Excluding Participants Randomized to Booster Training (N=1,659).

	Mean (SD) or N (%)
Characteristic	Memory	Reasoning	Speed	Control
Total (N)	296	329	336	698
Age (years) [M (SD)]	74 (6)	74 (6)	73 (6)	74 (6)
Sex [N (%)]
Men	62 (21)	75 (23)	79 (24)	184 (26)
Women	234 (79)	254 (77)	257 (77)	514 (74)
Race/Ethnicity [N (%)]
White, Not Hispanic	227 (77)	227 (69)	245 (73)	501 (72)
Black or African-American	67 (23)	97 (30)	88 (26)	187 (27)
All others	2 (1)	5 (2)	3 (1)	10 (1)
Years of Education [M (SD)]	13 (3)	14 (3)	14 (3)	13 (3)
Functioning
MMSE [M (SD)]	27 (2)	27 (2)	27 (2)	27 (2)
Cognitive Ability [M (SD)]	50 (10)	49 (10)	50 (10)	49 (10)
ADL [M (SD)]	50 (10)	50 (10)	49 (8)	50 (9)
IADL [M (SD)]	50 (10)	49 (10)	50 (10)	49 (10)

Note: None of the characteristics differ significantly (all P>.18) by treatment group, using omnibus chi-square tests or ANOVA F-tests for discrete and continuous factors, respectively. Abbreviations used include MMSE (Mini-Mental State Examination), ADL (activities of daily living) and IADL (instrumental ADL). Cognitive ability, ADLs, and IADLs are calibrated as T-scores (mean 50, SD 10). Percents may not add to 100 due to rounding.

Interventions

Each of the three training interventions was designed to target a specific cognitive ability – memory, reasoning, or speed of processing (Ball et al., 2002; Jobe, et al., 2001). Memory training involved teaching mnemonic strategies for remembering verbal material (Rasmusson, Rebok, Bylsma, & Brandt, 1999; Rebok & Balcerak, 1989). Reasoning training involved strategies for finding the pattern in a letter or word series (Willis, 1987; Willis & Schaie, 1986). Speed of processing training focused on visual search and divided attention (identifying an object on a computer screen at increasingly brief exposures, followed by dividing attention between two search tasks) (Ball & Owsley, 2000; Edwards et al., 2002; Roenker, Cissell, Ball, Wadley, & Edwards, 2003). Each training intervention involved 10 sessions. Two booster training interventions, with four sessions of content and structure similar to the initial training, were conducted at about one year and three years after the initial training. Participants who completed the initial training (having completed eight of 10 sessions) were considered eligible for booster training, and a 60% random sample was identified and invited for boosters. The flow of participants through various stages and randomization points is illustrated in Figure 1.

Figure 1. Participant Flow.

After a screening and eligibility phase, persons were randomized (R) to a no-contact control arm or one of three trained arms (memory, reasoning or speed training). After baseline assessment, persons in the active treatment arms participated in cognitive training. About 12 weeks after baseline there was a retest. Prior to the first annual follow-up, a 60% probability sample of persons who were compliant with baseline training (C; completed 8 of 10 sessions) were randomized (R) to a booster training arm. Participants assigned to the Booster arm are not included in this analysis, as discussed in the text. Booster training occurred prior to the first annual and third annual follow-up. It is important to note that this design enriches the trained but non-boosted sample with respect to persons non-compliant with baseline training, and no person not compliant with baseline training is represented in the booster trained arm, a design effect controlled with inverse probability of compliance weighting.

Restriction and Weighting of Participants

This analysis considers a sub-set of the ACTIVE participants. We exclude persons who were randomized to the booster training condition. Booster training represents an interruption into the presumed processes of forgetting initially presented training and aging related change. Because persons randomized to booster training had to be compliant with their initial training, the sample of persons who were initially randomized to training but not to booster is over-represented by persons who were non-compliant with initial training (see Figure 1). We addressed this issue using propensity score methods. We used logistic regression to model compliance among those trained as a function of with age (linear and quadratic functions), sex, education (less than 12 years vs. higher), race (Black or African-American vs. white and All others vs. white), baseline level of general cognitive ability. General cognitive ability was included as a tau-equivalent factor score (Raykov & Marcoulides, 2000) based on three cognitive abilities: memory, reasoning and speed (described below) and basic and instrumental daily functioning. We combined this propensity score with a propensity score for attrition over follow-up (using the same covariates) to generate a person-level weight based on the inverse probability of being compliant with initial training and not a drop out (Fewell et al., 2004). This weighting variable was used in our multivariate latent variable models, described below.

Outcome Measures

This analysis considers only the measures of the cognitive abilities targeted by the ACTIVE interventions. The ACTIVE proximal outcomes were chosen as basic cognitive abilities that had some evidence of being modifiable with focused intervention (Jobe, et al., 2001).

Memory training outcomes were the Hopkins Verbal Learning Test, Rey Auditory-Verbal Learning Test, and the Rivermead Behavioral Paragraph Recall test (Brandt, 1991; Rey, 1941; Wilson, Cockburn, & Baddeley, 1985). ACTIVE used parallel but nonequivalent forms for the memory assessment at each repeated observation to reduce retest effects. Because the forms are nonequivalent and no within-wave counterbalancing of alternate forms was used, the scale of the outcome varies across wave. We placed the alternate forms on an equivalent metric using an equipercentile equating procedure (Kolen & Brennan, 1995). An important artifact of this procedure is the removal of retest effects. Reasoning training outcomes were Letter Series, Letter Sets, Word Series (Ekstrom, French, Harman, & Derman, 1976; Gonda & Schaie, 1985; Thurstone & Thurstone, 1949). Speed of processing training outcomes were derived from the Useful Field of View (Ball & Owsley, 1993; Owsley et al., 1998; Owsley, Ball, Sloane, Roenker, & Bruni, 1991).

Following Ball et al. (2002), proximal outcomes were computed as composites and standardized by pooling and Blom-transforming (Blom, 1958) scores at all time points. We further rescaled the outcomes by standardizing to a T-score distribution based on the baseline mean and standard deviation (SD) of the referent age group among control participants (mean of 50 and SD of 10).

Age

Participants were assigned to one of four groups according to age at baseline (65-69, 70-74, 75-79, 80+). However, we retain use of individual time points of observation corresponding to exact years of observation (see time, below). Group indicator variables were centered using ANOVA-type centering following Kraemer and Blasey (2004). This results in an interpretation of modeled means and intercepts as averages over all age groups.

Time

We used each participant's exact age at time of re-assessment centered, at the mid-point of their age group, and divided by 5.23 (the per-protocol length of follow-up), as the time metric.

Control Variables

In addition to age group, our analytic models include covariate adjustment for participant sex and race/ethnicity (Black or African American versus all others). Because ACTIVE was a randomized trial, the treatment groups can be assumed to be equivalent at baseline with regard to observed and unobserved factors. Table 1, and previous work (Ball, et al., 2002), supports this assumption.

Analytic Approach

We analyzed each of the three cognitive outcomes separately. We fit a generalization of the growth curve model (McArdle & Epstein, 1987) to multiple groups, where the grouping variable was defined by randomization to treatment arm. Our model includes a random effects growth curve model for performance differences over an age basis, and a latent growth curve on an occasion basis for design effects (treatment, retest). The multiple group approach to the evaluation of intervention studies has been addressed previously (Curran & Muthén, 1999; Muthén & Curran, 1997). A general depiction of the modeling strategy for cognitive outcomes is illustrated in Figure 2. A detailed description of the model parameterization is available as an appendix.

Figure 2. Representation of the analytic model.

Observed variables are illustrated with rectangles, and y₁-y₆ correspond to baseline (y₁), post-test (y₂) and annual follow-ups at 1, 2, 3 and 5 years after post-test (y₃-y₆, respectively). Observed variables with missing values are shown with circles inside rectangles. Latent variables, which in the growth modeling framework are fixed and random effects describing initial level and components of change over time are shown as circles. Unidirectional paths are regression effects, bi-directional paths are variances or covariances. Latent variables without variances or residual variances are modeled as fixed effects, those with variances or residual variances are modeled as random effects. Means in latent variables are captured as regressions on a constant ( ). Regression effects a solid dot denote individually varying times of observation (age basis). The figure separates normative growth factors (i, r, s for initial level, retest, and slope) and treatment group-related growth factors (r_tx and s_tx for added retest gain due to training and additional time-related change due to loss of training gains, respectively). See text for more details.

Missing Data

Our models use all available information for model estimation. Cases with missing data in the outcome variable are included using maximum likelihood estimation under the assumption that the missing data are missing at random (Little & Rubin, 1987). Thus our missing data modeling strategies include both maximum likelihood methods and a weighting adjustment based on the inverse probability of drop-out.

Results

Results are presented in Table 2 and Figure 3. The table summarizes models comparing the ACTIVE participants randomized to the control and training condition for the memory, reasoning and speed proximal outcomes, respectively. Parameters reflect averages over all age groups and balanced for race/ethnicity and sex. Results are illustrated for the referent age group in Figure 3. Many parameter estimates are not presented (i.e., those in Γ, θ) but are available upon request. In this section, we first describe results from the baseline models, models with the age-related (or maturational) change means and variances constrained to be equal between trained and control participants. The results of these models are summarized in columns 2-4 of Table 2. Then we discuss the models with means and variances of maturational change freely estimated across treatment group (Table 2, columns 5-7).

Table 2. Growth model parameter estimates. ACTIVE cognitive intervention study (N=1,659).

	Constrained age-related change (s)			Unconstrained age-related change (s)
	Memory Control (n=630)	Reasoning Control (n=698)	Speed Control (n=698)	Memory Control (n=630)	Reasoning Control (n=698)	Speed Control (n=698)
Model Parameter	Trained (n=296)	Trained (n=329)	Trained (n=336)	Trained (n=296)	Trained (n=329)	Trained (n=336)
Growth factor intercepts
i (performance at baseline)
Control	51.3 *	49.4 *	50.1 *	51.1 *	49.4 *	50.2 *
Trained	50.7 *	49.8 *	50.5 *	51.9 *	49.6 *	50.0 *
s (change per ∼5 years of age)
Control	-2.5 *	-1.4 *	2.2 *	-2.1 *	-1.4 *	1.9 *
Trained	-2.5 *	-1.4 *	2.2 *	-6.1 *	-0.9	3.4 *
r (immediate re-test effect)	0 f	2.8 *	-4.0 *	0 f	2.8 *	-3.9 *
r(tx) (immediate training gain)	2.2 *	2.6 *	-7.8 *	3.4 *	2.5 *	-8.3 *
s(tx) (training loss per ∼5y)	0.6	-1.5 *	4.6 *	2.0 *	-1.6 *	4.1 *
Residual Variances†
i (performance at baseline)
Control	59.6 *	68.6 *	53.8 *	59.8 *	69.1 *	56.3 *
Trained	59.9 *	68.6 *	66.4 *	61.4 *	67.1 *	60.1 *
s (change per ∼5yof age)
Control	8.8 *	8.4 *	3.1	9.5 *	8.6 *	4.5
Trained	8.8 *	8.4 *	3.1	6.2	7.1 *	2.4
r(tx) (immediate training gain)	0.0 f	0.0 f	0.0 f	0.0 f	0.0 f	0.0 f
s(tx) (training loss per ∼5y)	0.0 *	0.3	4.0 *	0.0 *	0.5	4.5 *
Fit statistics
Overall Pseudo r2	0.86	0.93	0.84	0.86	0.93	0.85
aBIC	27154.0	26584.0	30006.4	27145.4	26594.6	30011.2
Log likelihood	-13522.2	-13222.4	-14933.6	-13512.4	-13222.1	-14930.4
Scaling correction factor	1§	1.229	1.156	1§	1.354	1.159
Omnibus Test of Modification (P-value) ‡	na	na	na	0.000	0.921	0.072
Implied at end of observation
Control	48.8	50.9	48.3	49.0	50.9	48.2
Trained	51.0	52.4	45.4	51.2	52.4	45.3
Implied training effect, 5y	2.8	1.2	-3.2	1.4	1.3	-2.7
Gains retained at 5y (%)	125	45	41	41	52	33

^*-P < 0.05 (P-values refer to test that parameter is equal to zero),

^f-parameter fixed to indicated value, na - not applicable,

^§-Correction factor not estimated using MLF estimator (all other models use MLR estimator),

^†-Residual variances of observed dependent variables constrained to be equal over time and across group (results not shown),

^‡-Comparison of model log-likelihood with correction for scaling factors (Satorra & Bentler, 1994).

Note: All effects reflect control for baseline age group (65-69, 70-74, 75-79, 80 and older), sex and race/ethnicity. Covariates are centered so that effects represent expected values for a hypothetical cohort balanced by the overall sample distribution of age, sex and race/ethnicity (Table 1). Reasoning and Speed models include weighting adjustment for the inverse probability of being compliant with initial training and completing follow-up, predicted by age, sex, race/ethnicity, education, general cognitive functioning, and ADL and IADL functioning.

Figure 3. Model Implied Change in Cognitive Performance over time within four age groups by ACTIVE trained and control groups.

Model-implied growth trajectories are plotted as averages over all age groups for each training arm, from the constrained aging effect model (Table 2). Trajectories are shown for controls (dashed line) and those trained (solid line). The lack of a retest effect for memory is an artifact of the equating procedure.

Main Model Results

Memory

Initial attempts to fit the memory model with the robust maximum likelihood (MLR) estimator and complex sampling weights failed to converge and produce standard errors. Convergence was achieved and standard errors were approximated as first-order derivatives of the model fit using the MLF estimator. The MLF estimator does not support complex sampling weights, so the memory models do not incorporate the weighting adjustment for balance of likely compliers and drop-out that the other outcome models do. Therefore the memory models include an over-representation of likely compliers among the controls. The implication of this is that the estimates obtained in this analysis can be considered to be biased towards the null.

The estimated memory model fit well (Table 2, overall pseudo r² = 0.86). The normative age-related change in memory performance was estimated as -.2.5 T-score units over the period of observation (∼5.23 years), an effect significantly different from zero (z = parameter estimate/standard error = -8.4, p <0.001). This 0.25 standard deviation (SD) effect size for age-related change, net of retest and training effects, is assumed to be at work in both groups (control and trained). The retest effect for the memory composite (η_r) is not estimated because it is forced to be zero due to the equating procedure. Table 2 also lists a model-implied expectation of performance level at the end of the period of observation, which for the control group 48.8.

The initial memory training effect was estimated to be 2.2 T-score units (z = 6.0 p < 0.001), and the loss of training-related gains was unexpectedly positive (+0.6 T-score units, z = 1.92, p = 0.54). Thus, memory trained persons continued to improve beyond initial retest period. Persons trained in memory retained 125% of their initial training-related gains at ∼5 years after training. The expected treatment effect at the end of the period of observation (i.e., the difference in the model-implied mean memory composite score at the fifth annual assessment for those memory trained relative to controls) is 2.8, a small effect size in Cohen's (1988) effect size taxonomy (0.28 SD units).

Reasoning

The reasoning model fit very well (overall pseudo r² = 0.93, Table 2). Normative age-related change in reasoning performance was -1.4 T-score units over the period of observation, an effect significantly different from zero (z = -5.5, p <0.001), but of trivial to small magnitude. The retest effect for the reasoning composite is small to moderate in magnitude (2.8 T-score units, p <0.001). The model-implied expectation of performance level at the end of the period of observation for persons not trained is 50.9. Comparing this to the control group intercept (49.4) reveals that persons who were not trained perform better at about 5.23 years follow-up relative to baseline (+1.5 T-score units), a consequence of a relatively large retest effect and little age-related change.

The initial reasoning training effect was estimated to be 2.8 T-score units (z = 17.7, p < 0.001); the loss of training-related gains was about half that magnitude (-1.5, z = -7.0, p < 0.001). Therefore about 45% of the training-related gains due to reasoning training were retained over the period of follow-up. The expected treatment effect at the end of the period of observation is 1.2, a trivial effect size. Both the control group and reasoning trained group perform at a higher (better) level at the end of the follow-up period relative to baseline. In the case of controls this is because the net retest effect exceeds the net performance decrement associated with aging-related change. The balance of these effects also influence the expected performance in the reasoning trained group, which also shows a net preservation of almost half of initial training-related gains.

Speed

The speed models fit well (Table 2), with an overall pseudo r² = 0.84. Note that the speed outcome is timed, so lower scores imply better (faster) performance. Normative age-related change was estimated at 2.2 T-score units over the period of observation, an effect that was statistically significant (z = 6.7, p < 0.001) and of small to moderate magnitude. The retest effect for the speed composite is moderate (-4.0, p < 0.001). The initial speed training effect was estimated to be -7.8 T-score units (p < 0.001) and the loss of training-related gains was about half that magnitude (4.6, z = 14.5, p < 0.001). The expected treatment effect at the end of the period of observation is -3.2 T-score units, a small to moderate effect size. About 41% of initial training-related gains were retained at the end of the observation period.

Differences in Maturational Change

Before considering models that test the difference in the pace of age related change, we can see that the groups that received training are performing at a level that is more favorable than those of controls at the end of the follow-up period. While we have expressed these differences in T-score units in the previous section, we can also express them in terms of units of age by dividing the mean difference at the end of the follow-up period by the normative per-year change in cognitive ability. Doing so suggests that persons receiving the memory, reasoning and speed training perform at a level about 5.8, 4.5 and 7.6 years younger than controls, respectively.¹

To test the hypothesis that training changes normative cognitive development, in each of our trained-condition specific models, we relaxed the assumption that age-related change was the same in the trained and non-trained groups. We simultaneously relaxed assumptions on the equivalence of the mean and variance of age related change, resulting in a two degree of freedom omnibus test of statistical significance, evaluated with change in −2×Loglikelihood with Satorra-Bentler (1994) correction for scaling factors. For only the memory trained outcome do we see that the model modification resulted in a statistically significant improvement in model fit (p <0.001). For reasoning and speed the improvement was not significant and the aBIC favors (i.e., is lower for) the models with normative age factors constrained.

Memory

For memory training, the estimated pace of age-related change in the control group (-2.1 T-score units over 5.23 years) implied slower decline than in the trained group (-6.1). This apparent paradoxical finding is exactly the finding identified by Salthouse (2006). As mentioned above, this model modification was significant (p <0.001).

Reasoning

For reasoning training, the estimated pace of age-related change in the control group (-1.4 T-score units over 5.23 years) implied a faster pace of aging than in those trained (-0.9). This aging-related change parameter was not statistically different from zero (z = -1.3, p = 0.18), implying that the reasoning trained group, on average, experienced no significant age-related slowing over the period of observation. The relaxed model, however, was not better fitting overall than the constrained model (p = 0.921) and the aBIC favors the unconstrained model.

Speed

Speed training models, similar to memory training, revealed an estimated pace of age-related change in the control group (+1.9 T-score units over 5.23 years) implied slower decline than in the trained group (+3.4). We interpret this counter-intuitive result as an indication that strong initial training effects coupled with rapid loss of training-related gains contaminate the estimation of aging related change. Relative to normative aging-related change, the initial boost in the targeted ability and the loss of training gains was much more pronounced for the speed training than for memory and reasoning. The relaxed model was not better fitting than the constrained model on the basis of the improvement in log-likelihood or information criteria.

Discussion

We used growth curve models to characterize cognitive change after training implemented in the ACTIVE study. We found that the reasoning training significantly slowed reasoning ability changes due to aging. We do not show an effect of slowing aging related declines in for speed training, and suspect this is due to the extremely powerful initial training effect and intractable confounding of the loss of training gains and maturation. We also do not show a slowing in aging related change due to memory training.

Our analysis was an attempt to respond directly to Salthouse' challenge to intervention studies of cognitive aging. In so doing, we were able to confirm that only one of the three ACTIVE training programs (reasoning) attenuates the pace of normal cognitive decline. Our findings repeat his observation that intervention studies (and in fields beyond cognitive aging) often find that training boosts performance but can also appear to accelerate age or time related change, because change over time becomes coupled with loss of training gains.

Recently, a ten year follow-up of the ACTIVE participants has been completed. With these data, we will have more information to characterize loss functions, including identifying loss of booster training gains, which will allow approaching this analysis question with all persons randomized to one of the ACTIVE training arms. The model can be improved as well, and opportunities to do so present themselves when the additional assessment point become available. An additional data point will allow more complex functional forms for the loss of training related gains, such as modeling a non-linear function with a lower asymptote. Such a model modification is crucial to our general analytic approach, which relies upon our separating the profile of training-related gains (initial gains and their duration) from aging related changes. Our models may not do this well for speed and memory training, and more flexible and appropriate modeling strategies (e.g., non-linear growth models such as structured latent curves models over the assessment occasion basis (Browne & Du Toit, 1991) coupled with piecewise linear random effect growth curve models over an age basis) may be estimable with the additional occasion data from the 10-year analyses.

Our models suggest other significant results. Although we do not demonstrate that memory training attenuates age-related change in memory performance, we do see that the albeit modest initial gains afforded by memory training are maintained at 5 years. Conversely, more than half of the initial gains in reasoning and speed are lost by 5 years. We also found that all of the training interventions attenuate the variance of age-related change in performance in trained abilities. This finding provides encouragement that the additional follow-up of ACTIVE participants will allow more time to pass and changes to accumulate and allow us to better capture the impact of ACTIVE training on change related to aging.

Limitations of this work are worth mentioning. First, as implied in our results, the five year follow-up period has not been sufficient to adequately characterize the effects of the ACTIVE intervention, both in terms of partialling out different aspects of the training effect (patterns of gains and maintenance for initial and booster training) and in characterizing effects on maturational change. This limitation has two further aspects. One is that as an initially healthy and cognitively intact sample of older adults, few ACTIVE participants are likely to have experienced clinically meaningful cognitive decline over five years. The other is that not enough time has passed to adequately characterize the magnitude and sources of change in cognitive abilities due to ACTIVE training. This is why we are forced to exclude Booster participants from this analysis. To include them, we would need additional follow-up in order to identify the initial gains related to booster training and the subsequent loss of this effect.

Other forms of bias may be at work in the ACTIVE study and in our analysis. ACTIVE can best be described as a sample of convenience. The results observed may not translate to representative samples drawn from community based populations. Characteristics of persons that cause them to be willing to participate in a long-term trial of cognitive training may influence the extent to which they are receptive to the effects of training. The detection of differences in aging-related cognitive change may be particularly challenging in ACTIVE if it is true that increasing age is associated with accelerated cognitive decline given a single eligibility threshold for baseline cognitive performance was used.

Conclusion

Our results suggest that the ACTIVE reasoning training program may slow the pace of cognitive aging, and that memory trains gains are, on average, not lost over about 5 years. Regardless, the level of performance of groups receiving the intervention are more favorable than those of controls: persons receiving the memory, reasoning and speed training perform at a level about 6, 4 and 8 years younger than controls, respectively.

Model building.

Our model parameterization was informed by preliminary exploratory data analysis using only the control group to determine the most appropriate functional form for the age and retest effects. Once a well-fitting model was developed, persons in the intervention groups were added to the model.

The analytic model can be represented with two equations.

$$\mathit{\text{y}}=\mathbf{\Lambda }\mathit{\eta }+\mathit{∊}$$ (1)

$$\mathit{\eta }=\mathit{\alpha }+\mathbf{\Gamma }\mathit{\text{x}}+\mathit{\varsigma }$$ (2)

Where a vector of observed dependent variables (y) is expressed as a function of latent variables (η) and where ∊ are residuals assumed to be distributed normal conditional upon the observed variables and independent of the other observed and unobserved variables. The latent variables in η are defined to describe observed differences in y. The system of equations that relate the latent variables to the observed background variables is given by equation (2) where α are latent variable intercepts, Γ contains regressions of the growth factors on covariates in x (age, sex and race/ethnicity group indicators), and ζ residual variances for the latent variables.

Our models include five latent variables (η, see Figure 2). Three are used to capture normative change effects: initial starting level (η_i) where subscript i indicates intercept factor as labeled in Figure 2, a fixed retest effect (η_r) that captures the punctuated improvement in performance associated with practice (Ferrer, Salthouse, Stewart, & Schwartz, 2004), and age-related slope (η_s) modeled as a random effect. Two treatment-related growth factors are included: the effect of the initial training (η_rtx) and the loss of initial training gains (η_stx).

Our models include five latent variables (η, see Figure 2). Three are used to capture normative change effects: initial starting level (η_i) where subscript i indicates intercept factor as labeled in Figure 2, a fixed retest effect (η_r) that captures the punctuated improvement in performance associated with practice (Ferrer, et al., 2004), and age-related slope (η_s) modeled as a random effect. Two treatment-related growth factors are included: the effect of the initial training (η_rtx) and the loss of initial training gains (η_Stx).

In terms of model parameterization, the intercept factor (η_i) captures individual differences in starting values by loading in each observed y with weight of 1.0. The retest factor (η_r) and training effect factor (η_rtx) capture the immediate retest and training-related gain by loading in y₂ − y₆ with a weight of 1.0. The loss of training related gains (η_Stx) loads in y₁ − y₆ with fixed weights (λ) of 0, -0.888, 0.125, 0.485, 0.709 and 1.00 in the trained group. Each weight for y₂ − y₆ is the natural log of the per-protocol year of assessment (0.23, 1.23, 2.23, 3.23, 5.23) divided by the natural log of the total per-protocol follow-up time (5.23). The effect of this parameterization is to impose a linear (within age group) change assumption, but loss of training gains occurs as a logarithmic function of time since baseline. All latent growth factors are scaled to capture change per 5.23 years, the per-protocol period of observation (5.23 years, hereafter and in our Tables we denote this as approximately 5 years, ∼5) and are therefore on the same scale and can be easily combined to evaluate different effects. The rescaling of age (by dividing by 5.23) results in an interpretation of the mean of the age-related change factor (η_s) as the amount of age-related change expected over the follow-up period.

Models were identified by limiting the number of random growth factors to be estimated (η_i, η_s, η_rtx, η_stx) and imposing across-group equality constraints ( ${\alpha }_{\text{r}}^{\text{C}}={\alpha }_{\text{r}}^{\text{T}};{\alpha }_{\text{s}}^{\text{C}}={\alpha }_{\text{s}}^{\text{T}};{\psi }_{\text{s}}^{\text{C}}={\psi }_{\text{s}}^{\text{T}};{\mathbf{\Gamma }}^{\text{C}}={\mathbf{\Gamma }}^{\text{T}};{\mathbf{\theta }}^{\text{C}}={\mathbf{\theta }}^{\text{T}}$;, where superscripts C and T denote control and trained groups, respectively and θ and Ψ contain variances and covariances for the observed and latent variables). Full model specification and syntax are available upon request.

Detecting Differences in Maturational Change

An initial model assumed that the mean aging related change (α_s) and residual variance (ψ_s) were equal in all groups (control, trained). Subsequent models relaxed the equality constraints on α_s and ψ_s. The purpose of these models was to assess the assumption of constant and invariant age-related change across the control and trained groups. To test the hypothesis that training changes normative cognitive development, in each of our trained-condition specific models we relaxed the assumption that age-related change ( $\text{M}({\eta }_{\text{s}}^{•})=\text{M}({\eta }_{\text{s}}^{\text{C}})=\text{M}({\eta }_{\text{s}}^{\text{T}});\text{V}({\eta }_{\text{s}}^{•})=\text{V}({\eta }_{\text{s}}^{\text{C}})=\text{V}({\eta }_{\text{s}}^{\text{T}})$) by relaxing equality constraints on model parameters in α and Ψ, respectively. A two degree of freedom omnibus test of statistical significance was used to guide inference, based on change in −2×Loglikelihood with Satorra-Bentler (1994) correction for scaling factors. Improvement in model fit was informed by examining the sample-size adjusted Bayesian Information Criterion (aBIC). Across all training outcomes, we see that the aBIC favors the models with normative age factors constrained, as the smaller aBIC values appear under the constrained models. For only the reasoning trained outcome do we see that the model modification resulted in a statistically significant improvement in model fit (p <0.001).

Model Estimation, Model Fit and Hypothesis Testing

We estimated the model parameters using the full information robust maximum likelihood estimator of Mplus v5.2 (Muthén & Muthén, Los Angeles, CA). The full information approach does not produce fit statistics typically found in limited information structural equations modeling applications (e.g., the model chi-square, comparative fit index, root mean square error of approximation). Following Singer and Willett (Singer & Willett, 2003), we estimated pseudo r² statistics (the square of the correlation of the model-implied and observed outcome value for all participants) to gauge overall mode fit. To evaluate individual parameters we use the ratio of the parameter estimate to its standard error as a normally distributed test statistic of the parameter being equal to zero. We use an uncorrected type-I error level of .05. Effect sizes are interpreted using Cohen's (Cohen, 1988) effect size taxonomy. The significance of the model modification relaxing the assumption of equal age-related change across treatment group was tested by comparing the model log-likelihood statistics with correction for scaling factors (Satorra & Bentler, 1994). Change in model fit between constrained and unconstrained model is informed by the sample size adjusted Bayesian Information Criterion (aBIC). The aBIC is a statistic that combines the log-likelihood, number of parameters, and sample size. It is a model fit statistic that rewards model parsimony. Models with lower aBIC values are preferred over models with higher aBIC values (Muthen & Asparouhov, 2006).