Using Cognitive Intraindividual Variability to Measure Intervention Effectiveness: Results from the Baltimore Experience Corps Trial

Abstract

Objectives

Studies investigating the effectiveness of intervention programs on cognitive ability in older adults are inconsistent; however, these studies generally focus on traditional measures of cognition, and therefore may miss some improvements by not utilizing alternate measures. We evaluate the potential for intraindividual variability in cognitive speed (IIV), a demonstrated sensitive indicator of cognitive functioning, to be used as an index of cognitive plasticity from an intervention. The current study evaluated whether older adults in a school volunteering program showed a reduction in IIV, compared to a low-activity control group over 2 years of exposure.

Method

Nondemented older adults (n = 336) participated in the Baltimore Experience Corps Trial, an evaluation of a volunteering program conducted at elementary schools designed to increase older adults’ physical, cognitive, and social engagement. Participants completed a cognitive battery that included a Stroop task at baseline and after 12 and 24 months.

Results

Traditional intent-to-treat analyses did not report significant improvements. Participants who complied at the 80th percentile or above showed a significant reduction in IIV at 24 months, with an additional trend of improved IIV with increased compliance to the treatment protocol, both at 12 months, and at 24 months. Men also showed dose-dependent improvements after 12 months.

Discussion

The Experience Corps program resulted in an improvement in cognitive performance as measured by IIV. Analyzing previously collected data with nontraditional measures of cognition, such as IIV, may be a potentially fruitful and cost-effective method for understanding how interventions impact cognition in aging populations.

There is a plethora of research investigating the possibility of improving cognitive ability in older adulthood through cognitive, physical, and social interventions. These interventions can be divided into three overall themes: (a) physical activity programs assigning regular exercise (Colcombe & Kramer, 2003; Sink et al., 2015); (b) cognitive training on various tasks or teaching strategies to improve specific cognitive abilities (e.g., Ball et al., 2002; Edwards, Fausto, Tetlow, Corona, & Valdés, 2018; Nguyen, Murphy, & Andrews, 2019); or (c) enrichment programs involving sustained complex cognitive and/or social activities such as learning to act (Hertzog, Kramer, Wilson, & Lindenberger, 2009; Leanos et al., 2019) or singing in a community choir (Johnson et al., 2018). Evidence of cognitive improvements from interventions has so far been inconsistent in terms of overall effectiveness (Gavelin, Lampit, Hallock, Sabatés, & Bahar-Fuchs, 2019; Karr, Areshenkoff, Rast, & Garcia-Barrera, 2014; Lampit, Hallock, & Valenzuela, 2014; Simons et al., 2016). As such, there have been substantial efforts to determine the right combinations of intervention characteristics that lead to improved cognition, many of which focus on making an intervention longer, more intensive, or by incorporating greater novelty than previous attempts (Diamond & Ling, 2016). In the present paper, we considered a different perspective: what if existing intervention programs were effective, but the cognitive outcome measures used were not sufficiently sensitive to detect those improvements? Specifically, we evaluate the potential for intraindividual variability (IIV) in cognitive speed, a demonstrated sensitive indicator of cognitive functioning (Hultsch, MacDonald, Hunter, Levy-Bencheton, & Strauss, 2000), to evaluate cognitive change from an intervention.

IIV refers to moment-to-moment variations in an individual’s performance on a reaction time (RT) task, and is commonly considered a manifestation of fluctuations in executive control or attention (Bunce, MacDonald, & Hultsch, 2004). IIV is also emerging as an index of cognitive plasticity and is hypothesized to represent neurological integrity, as higher IIV is linked to poorer brain functioning (MacDonald, Nyberg, & Bäckman, 2006). Significant changes in IIV have been found in older adults following two mindfulness lifestyle interventions: a 3-month meditation retreat (Zanesco, King, MacLean, & Saron, 2018), and in older adults with subjective cognitive decline an 8-week randomized controlled trial (RCT) of mindfulness training (Smart, Segalowitz, Mulligan, Koudys, & Gawryluk, 2016). However, RCTs involving other activities have not shown similar reductions in IIV. Bielak and Brydges (2019) analyzed IIV data from the LIFE study (Sink et al., 2015), a 2-year RCT where sedentary older adults participated in a moderate physical exercise program (in comparison to a successful aging educational program), but did not observe improvements in IIV. Of note, the primary aim of the LIFE study was to reduce the likelihood of mobility disability, and the low-intensity nature of the exercise sessions was also not sufficient to change standard cognitive outcomes. Brydges and Bielak (2019) conducted similar analyses in the Synapse project, a 14-week RCT of cognitive engagement and training (learning quilting and/or digital photography) that showed gains specific to episodic memory (Park et al., 2014). However, no improvements were observed in IIV. Alternatively, it remains possible that a multifaceted intervention that includes various forms of engagement may result in reductions in IIV. As research has found that physical activity, cognitive stimulation, and social engagement are each effective at improving different cognitive abilities (Karr et al., 2014; Lampit et al., 2014), it is possible that an intervention program that combines several intervention components (i.e., multimodal) results in greater improvements in cognitive ability than single intervention activities (i.e., unimodal), and could result in improvements in multiple cognitive domains.

What also makes investigating IIV a potentially fruitful avenue for detecting cognitive improvement is that there is evidence that IIV provides information about cognitive ability that goes beyond typically used measures of central tendency (MacDonald & Stawski, 2015). For example, IIV has been shown to be predictive of everyday functioning and current neurological status, even after accounting for mean level of performance (Hultsch et al., 2000). Holtzer, Mahoney, and Verghese (2013) found that IIV (but not mean RT or accuracy) in a Flanker task was associated with gait speed, and Bauermeister and colleagues (2017) reported that IIV, but not traditional measures of executive function, uniquely predicted falls in older adults in a structural equation model. These studies demonstrate that IIV is distinct from standard measures of executive control and can provide unique information about cognitive ability. Higher levels of IIV are also predictive of later clinical cognitive impairments (Tales et al., 2012) and mortality (Haynes, Bauermeister, & Bunce, 2017). Additionally, increased IIV is associated with a range of detrimental outcomes in older adulthood, including lower cognitive ability (Bielak, Hultsch, Strauss, MacDonald, & Hunter, 2010) and poorer health (Bunce, Tzur, Ramchurn, Gain, & Bond, 2008). Overall, IIV provides more sensitive information compared to traditionally used mean RT: for instance, Batterham, Bunce, Mackinnon, and Christensen (2014) found that IIV, but not mean RT, was strongly associated with increased risk of mortality in adults aged 70 years and older. Therefore, IIV may provide insight into intervention-related cognitive change as a potentially more sensitive metric of cognitive change than standardized cognitive tasks, even possibly a behavioral index of neurological plasticity, because it is believed to represent a distinct phenomenon of neurological integrity.

To summarize, some research has demonstrated that IIV may be improved through experimental interventions (e.g., Smart et al, 2016; Vasquez & Anderson, 2018; Zanesco et al., 2018). However, intervention programs have generally focused on improvements in traditional measures of cognitive ability, such as average RT (Edwards et al., 2018), which often report mixed results (Simons et al., 2016). Given that previous research has suggested that IIV is distinct from and may be more sensitive to individual differences in cognitive ability than these traditional measures (Bauermeister et al., 2017), it is feasible that intervention-related changes in cognition may be more apparent if IIV is used as a metric of cognitive ability.

In the current study, we examined whether IIV improved after participating in the Baltimore Experience Corps Trial (BECT; Fried et al., 2013), a RCT where older adults volunteered at elementary schools to help young children with academic achievement (e.g., reading and/or mathematics skills) for 15 hr/week for two school years (September–June each year). Experience Corps is a longstanding program run by the AARP in over 20 cities in the United States. The program was designed to benefit retired adults through increased physical and social activities and cognitive engagement. Previous BECT studies have reported improvements on a range of outcomes including health (Tan, Xue, Li, Carlson, & Fried, 2006), generativity (Gruenewald et al., 2015), lifestyle activity (Parisi et al., 2015), and cognition (executive function and memory) and related cortical functions and structures (Carlson et al., 2008, 2009; 2015; Varma, Chuang, Harris, Tan, & Carlson, 2015).

Given the multimodal engagement design of the Experience Corps program, encompassing social, physical, and cognitive stimulation that requires sustained attention and executive functions to shift flexibly across volunteer roles, we hypothesize that participants in the Experience Corps program have a strong likelihood of showing changes in IIV. Specifically, we predicted that participants in the intervention would improve (i.e., decrease) in IIV in comparison to participants in the control condition over 2 years of exposure. Additionally, Gruenewald and colleagues (2015) and Carlson and colleagues (2018) found that level of intervention compliance was positively associated with the magnitude of improvements in generativity and processing speed. Therefore, we expected improvements in IIV to be related to the degree of compliance.

Method

BECT was an RCT designed to investigate the effects of Experience Corps on health in older adults (Fried et al., 2013). Hypotheses, methods, and analyses were preregistered prior to data analysis (http://aspredicted.org/blind.php?x=59ya5c). As the current study describes secondary analyses on the BECT data, it was declared exempt from ethics review by Colorado State University’s research ethics committee.

Participants

Older adults were recruited through mailings and radio advertisements, and at community locations and events (e.g., health fairs, churches). Individuals were eligible to participate if they (a) are ≥ 60 years old; (b) spoke English; (c) scored ≥ 24 on the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975); (d) achieved ≥ sixth grade reading level on the Wide Range Achievement Test (Wilkinson & Robertson, 2006); and (e) cleared a criminal background check. The sample tested at baseline consisted of 702 participants (n = 352 intervention; n = 350 control). Many participants were not included in the analysis due to missing covariates or dependent variable data (e.g., missing a testing session, n = 313, 76 of whom were missing at least one covariate), and a smaller number (n = 53) were removed for missing the majority of RT trials for either Stroop condition for any occasion (see “Calculation of Intraindividual Variability.” The final sample (n = 336) was predominantly women (81.8%) and African American (90.5%), with an average age of 66.40 years (SD = 5.63; Table 1). The final sample did not significantly differ from the excluded participants in terms of age, sex proportion, race proportion, years of education, self-reported health, or depressive symptoms.

Table 1.

Demographic Characteristics of the Intervention and Control Groups

Variable	Intervention (n = 181) M (SD)	Control (n = 155) M (SD)	p
Age (years)	66.13 (5.46)	66.72 (5.83)	.345
Sex (% women)	81.2%	82.3%	.747
Race (% African American)	89.0%	92.4%	.679
Education (years)	14.06 (2.69)	14.06 (3.22)	.991
Geriatric Depression Scale (short form total score)	1.05 (1.45)	1.11 (1.73)	.730
Self-rated health (% good or better)	94.5%	88.0%	.387
MMSE total score	28.25 (1.50)	28.10 (1.39)	.359

Note: Significance calculated with independent-samples t tests for continuous variables and with Mann–Whitney U tests for categorical variables. MMSE = Mini-Mental State Examination.

Intervention

Participants were randomly assigned to either the intervention condition (Experience Corps), or the control condition (low-activity volunteering programs). In Experience Corps, participants (n = 181 in the current study) were trained in groups of 10–20 and placed in public elementary schools in Baltimore (kindergarten through third grade) in groups of 7–10. The adults agreed to volunteer for 15 hr/week for two school years. Participants were trained to provide support in literacy, math, and library services, and received instruction on behavior management and violence prevention activities (see Fried et al., 2013, for more details). They could choose to add school attendance support, computing support, and enhancing parental involvement over time. Both initial training and returning volunteer refresher training (prior to the beginning of the second year) were provided, and the team of volunteers also met at least biweekly for discussion and problem solving. Participants received $250/month to cover expenses and for incentive.

Participants allocated to the control condition (n = 155 in the current study) were referred to the Baltimore City Commission on Aging and Retirement Education, where they could choose to join low-activity volunteering programs. These activities included volunteering at health fairs, city festivals, and senior center events (i.e., were of short duration and/or low time demand). There was no minimum volunteer requirement or expectation of participation when referred. Control participants were also invited to participate in Experience Corps after 2 years if they were still interested.

Measures

Stroop Task

IIV was calculated from the trial RTs on a computerized Stroop task (Armstrong et al., 2020) administered at baseline, 12, and 24 months, resulting in two IIV outcomes per testing period per participant. Participants had to identify the font color (red, blue, or green) of the target item by pressing a corresponding key as fast as possible. There were two conditions: (a) neutral, where the target item was neutral (e.g., XXX in red ink) and (b) incongruent, where the target item was a color word in a contrasting font color (e.g., “GREEN” presented in red ink). There were 30 trials in each condition.

Covariates

We controlled for possible effects of age, sex, race, education, self-rated health, and depressive symptoms. Education was assessed as the years of full-time formal education. Self-rated overall health ranged from excellent (1) to poor (5) at baseline. Depressive symptoms were measured using the total score from the Geriatric Depression Scale short form (Yesavage & Sheikh, 1986) at the baseline testing session. These covariates were chosen to be consistent with other BECT analyses (Carlson et al., 2018).

Data Preparation

Calculation of IIV.

The calculation of IIV was in accordance with a well-established protocol (Bielak & Anstey, 2019; Hultsch, Strauss, Hunter, & MacDonald, 2008). The calculation procedures were completed separately for each condition at each time point. Only correct responses were included to account for variability due to accuracy (7.58% of trials were incorrect). These RTs were trimmed for outliers by applying a lower limit of 150 msec. Additionally, each trial automatically timed out at 3000 msec. Next, within-participant means and SDs were calculated across all trials, and any RTs greater than 3 SDs from the mean for that individual were removed (M = 0.37 trials/participant per condition). Participants missing data on more than 50% of the trials were removed from those particular IIV calculations, as imputation accuracy has been shown to be significantly reduced when greater than half the items are missing (Burns et al., 2011). Missing values were imputed using regression substitution where individualized equations of the RTs across all trials are used to predict the missing values (Hultsch et al., 2000). This data cleaning and missing data imputation approach is considered conservative as it decreases within-participant variation. Following the IIV calculation protocol designed by Hultsch and colleagues (2000), in order to remove any systematic within (i.e., practice effects) and between-subject (i.e., age group differences in mean RT) sources of variance, the RT data were regressed onto categorical age (younger than 70 years, or 70 or older; note that age was used as a continuous covariate in the ANCOVAs and CACE models), categorical trial, and their interactions. The resulting residuals therefore were independent of any potential confounds of age and trial that could influence the SD (e.g., higher mean RT is associated with higher SD; Hultsch et al., 2008). The residuals were transformed to T-scores (M = 50, SD = 10) to enable comparisons across conditions. Each individual’s standard deviation (ISD) was then calculated as the SD of their T-scores and used as the measure of IIV. ISD values were computed for each condition at each testing period.

Calculation of compliance

Compliance was calculated in the same manner as Carlson and colleagues (2018) and Gruenewald and colleagues (2015), who varied the cutoff values for compliance to examine the effects of treatment dose on outcome (i.e., improvement in IIV). Specifically, the 20th, 40th, 60th, and 80th percentiles of exposure hour distributions were calculated at 12 and 24 months. The number of compliers in the treatment condition for each cutoff criteria was 145, 107, 72, and 36 at each time point, respectively. These percentiles resulted in cutoff points at 98.5, 421.6, 476.9, and 544.8 hr at 12 months, and 98.5, 622.3, 934.6, and 1,054.7 hr at 24 months (the lowest percentiles include participants who completed zero hours of volunteering). For example, for the 20th percentile analyses, participants who completed at least 98.5 hr of volunteering were considered to be compliers.

Statistical Analyses

The data were analyzed with the intent-to-treat (ITT) principle using mixed-design Repeated-Measures Analysis of Covariance (RM-ANCOVAs) in SPSS version 25, and also with complier average causal effect (CACE) models and sensitivity analyses in Mplus 8 (Muthén & Muthén, 1988–2017). The ITT protocol is often used to evaluate the efficacy of RCTs. However, this approach assumes full treatment compliance from every participant, which results in a suppressed estimate of treatment effect (Jo, 2002). CACE models incorporate treatment compliance by comparing the participants who complied with the treatment regimen to members of the control group who are assumed to be likely to have complied if they had been allocated to the treatment group (Jo, 2002). Mplus produces an unstandardized regression estimate and associated p value as part of the analysis output. In the current study, a negative CACE value was expected to be observed, indicating a reduction in IIV over time. That is, a CACE estimate of −3 would indicate a decrease of 3 T-score units of IIV in the treatment group compared to the members of the control group who would have complied with treatment had they been allocated to that group.

The RM-ANCOVAs had time as a within-subjects factor (baseline vs 12 months vs 24 months), and group (intervention vs control) as a between-subjects factor. All RM-ANCOVAs and CACE models included age, sex, race, years of education, self-rated health, and depressive symptoms as covariates. The CACE models additionally included baseline performance as a covariate. Due to the large number of analyses being conducted, a more conservative alpha level of p = .01 was used to control for Type I error rate. A two-step statistical procedure was used when conducting CACE sensitivity analyses: First, a CACE model including covariates was run with the exclusion restriction, where the effect of the intervention on noncompliers was constrained to zero. Second, the exclusion restriction was removed in order to determine the robustness of this assumption. Additionally, Hedges’ g was calculated as a mean effect size with 95% CIs to quantify the change in the compliers above and beyond the noncompliers in the intervention group (as opposed to the CACE models that compared compliers in the treatment group and control participants who would have complied if they were in the treatment group). This was done by creating difference scores for each participant (e.g., neutral condition IIV at baseline subtracted from neutral condition IIV at 12 months), and then calculating the mean difference between the groups and dividing by the pooled standard deviation. Hedges’ g is similar to Cohen’s d, except it accounts for different sample sizes between groups. Values of 0.20, 0.50, and 0.80 are considered small, medium, and large effects, respectively.

Results

Preregistered Analyses

ITT analyses

The ISD values by experimental group and testing period are presented in Table 2. For the RM-ANCOVA testing, the effects of the neutral condition, the Group × Time interaction was not significant, F(2, 656) = 1.19, p = .304, η _p² = .00. The main effects of Group, F(1, 328) = 0.03, p = .861, η _p² = .00, and Time did not reach significance, F(2, 656) = 3.58, p = .028, η _p² = .01.

Table 2.

Intraindividual Standard Deviations of the Two Conditions Analyzed at Baseline, 12 Months, and 24 Months

Group	Testing period	Task condition
		Neutral M (SD)	Incongruent M (SD)
Experience Corps (n = 181)	Baseline	5.88 (2.62)	7.03 (2.62)
	12 months	4.00 (2.47)	6.83 (2.75)
	24 months	6.12 (2.76)	6.56 (2.81)
Control (n = 155)	Baseline	6.31 (2.93)	6.78 (2.69)
	12 months	3.98 (1.90)	6.84 (2.80)
	24 months	6.05 (2.73)	6.56 (2.81)

Similar results were found for the incongruent condition. The Group × Time interaction was not significant, F(2, 656) = 0.71, p = .495, η _p² = .00. The main effects of Group, F(1, 328) = 0.29, p = .593, η _p² = .00, and Time did not reach significance, F(2, 656) = 3.11, p = .045, η _p² = .01. In summary, the ITT analyses indicated that the intervention program did not result in any significant changes in ISD over time.

CACE estimates

The CACE estimates of the 16 tested models are presented in Table 3. The varying levels of compliance were informative, as only participants who complied at the 80th percentile or above showed a significant reduction in ISD (CACE = −3.08). Further, this change was only significant for the incongruent condition, and only at 24 months. There was a general trend of improved ISD and larger effect sizes on the incongruent condition with increased compliance, both at 12 months, and at 24 months (e.g., −.31, −.54, −1.16, −3.08 for the 20th through 80th percentile respectively), although the majority of CACE estimates were not significant. No such trends existed for the neutral condition. There was also a general trend of effect size increasing with compliance in both conditions and at both time points, although the majority of the 95% CIs overlapped with zero.

Table 3.

CACE Model Estimates and Effect Sizes by Each Condition at 20th, 40th, 60th, and 80th Percentile of Exposure at 12 and 24 Months

CACE comparison	Condition	20th percentile of exposure	40th percentile of exposure	60th percentile of exposure	80th percentile of exposure
Baseline—12 months	Neutral	0.39 (0.18) g = 0.27 (0.10–0.64)	0.47 (0.19) g = 0.23 (−0.08–0.53)	0.40 (0.20) g = 0.37 (−0.07–0.67)	0.48 (0.29) g = 0.35 (−0.02–0.71)
	Incongruent	−0.08 (0.30) g = 0.02 (−0.35–0.39)	−0.10 (0.44) g = 0.14 (−0.16–0.44)	−0.56 (0.69) g = 0.23 (−0.08–0.52)	−1.61 (1.05) g = 0.35 (−0.01–0.72)
Baseline—24 months	Neutral	0.04 (0.36) g = 0.08 (−0.29–0.44)	0.61 (0.32) g = 0.19 (−0.11–0.49)	0.77 (0.39) g = 0.21 (−0.09–0.51)	0.44 (1.84) g = 0.29 (−0.08–0.65)
	Incongruent	−0.31 (0.31) g = −0.19 (−0.55-0.18)	−0.54 (0.44) g = 0.16 (−0.14–0.45)	−1.16 (0.73) g = 0.18 (−0.12–0.48)	−3.08 (1.11)* g = 0.34 (−0.03–0.71)

Note: Top row of values are unstandardized CACE estimates (SE in parentheses). Negative CACE values indicate reductions in ISD (i.e., improvements in behavioral performance). g = Hedges’ g (standardized mean difference between compliers and noncompliers at each percentile level). Values in parentheses represent 95% confidence intervals. Positive effect sizes indicate greater improvement in the compliers.

*p < .01.

Relaxing the exclusion restriction in order to test its tenability resulted in similar CACE estimates for the compliers for the baseline-12 month and baseline-24 month comparisons. Therefore, the results were not sensitive to the exclusion restriction.

Nonpreregistered Analyses

Following significant findings by sex in Carlson and colleagues (2018), ITT and CACE analyses were additionally conducted on the data stratified by sex. For the men, the RM-ANCOVAs showed a significant Time × Group interaction (F(2, 108) = 5.76, p = .004, η _p² = .096), but no significant main effects for Group (F(1, 54) = 6.62, p = .013, η _p² = .11) or Time (F(2, 108) = 0.55, p = .579, η _p² = .01) in the neutral condition. However, the interaction was not a result of the intervention; rather, post hoc t tests showed that the control group (M = 7.42, SD = 3.17) had significantly higher ISD than the intervention group (M = 4.82, SD = 1.81) at baseline, t(59) = 4.03, p < .001, Hedges’ g = 1.04 (95% CI = 0.51–1.58), and no other between-group differences were significant. For the incongruent condition, the interaction (F(2, 108) = 3.71, p = .029, η _p² = .06), and main effects of Group (F(1, 54) = 1.11, p = .298, η _p² = .02) and Time (F(2, 108) = 0.31, p = .736, η _p² = .01) were not significant.

For women, neither condition showed any significant interactions or main effects in the RM-ANCOVAs (neutral: Time × Group, F(2, 536) = 0.00, p = 1.00, η _p² = .00, Group, F(1, 268) = 0.15, p = .704, η _p² = .00, and Time (F(2, 536) = 2.88, p = .057, η _p² = .01; incongruent: Time × Group, F(2, 536) = 0.47, p = .623, η _p² = .00, Group, F(1, 268) = 0.58, p = .449, η _p² = .00, and Time (F(2, 536) = 3.68, p = .026, η _p² = .01).

The CACE estimates revealed significant sex differences in ISD change (Tables 4 and 5). For men, increased compliance was associated with a reduction in ISD in the incongruent condition after 12 months. A significant reduction was apparent at both the 40th percentile (i.e., the top 60% of participants with regards to compliance; CACE = −3.45) and the 60th percentile and above (the top 40% of participants in terms of compliance; CACE = −5.91). However, the significant reductions in ISD were not seen at 24 months for the incongruent condition, in comparison to baseline performance. There was evidence of greater ISD (i.e., worse performance) for the neutral condition at 24 months at the 60th percentile of compliance. It should be noted that sample size in the male group was small (total n = 61), especially at the higher percentiles, and consequently estimates at the 80th percentile were not calculated, in line with Carlson and colleagues (2018). For women, there were no significant changes in ISD in either condition at 12 or 24 months, regardless of percentile of compliance.

Table 4.

CACE Model Estimates and Effect Sizes by each Condition at 20th, 40th, and 60th, Percentile of Exposure at 12 and 24 Months in Men

CACE comparison	Condition	20th percentile of exposure	40th percentile of exposure	60th percentile of exposure
Baseline—12 months	Neutral	0.76 (0.56) g = −0.32 (−1.07–0.42)	0.60 (0.44) g = 0.04 (−0.63–0.72)	0.78 (0.72) g = 0.36 (−0.34–1.05)
	Incongruent	−2.03 (1.76) g = −0.15 (−0.89–0.58)	−3.45 (0.92)* g = 0.13 (−0.55–0.80)	−5.91 (0.74)* g = 0.28 (−0.41–0.98)
Baseline—24 months	Neutral	0.77 (0.73) g = −0.73 (−1.49–0.03)	1.09 (1.25) g = −0.04 (−0.71–0.64)	1.54 (0.44)* g = −0.11 (−0.79–0.56)
	Incongruent	−0.81 (0.65) g = −0.42 (−1.17–0.32)	−1.06 (0.73) g = −0.01 (−0.68–0.66)	−0.44 (0.93) g = −0.17 (−0.84–0.51)

Note: Top row of values are unstandardized CACE estimates (SE in parentheses). Negative CACE values indicate reductions in ISD (i.e., improvements in behavioral performance). g = Hedges’ g (standardized mean difference between compliers and noncompliers at each percentile level). Values in parentheses represent 95% confidence intervals. Positive effect sizes indicate greater improvement in the compliers.

*p < .01.

Table 5.

CACE Model Estimates and Effect Sizes by Each Condition at 20th, 40th, and 60th, Percentile of Exposure at 12 and 24 Months in Women

CACE comparison	Condition	20th percentile of exposure	40th percentile of exposure	60th percentile of exposure
Baseline—12 months	Neutral	0.39 (0.20) g = 0.38 (−0.05–0.81)	0.49 (0.22) g = 0.23 (−0.10–0.57)	0.39 (0.24) g = 0.37 (0.04–0.70)
	Incongruent	0.17 (0.31) g = 0.09 (−0.35–0.51)	0.16 (0.40) g = 0.16 (−0.18–0.49)	−0.13 (0.63) g = 0.21 (−0.12–0.54)
Baseline—24 months	Neutral	−0.21 (0.37) g = 0.27 (−0.16–0.69)	0.40 (0.34) g = 0.22 (−0.11–0.55)	0.52 (0.41) g = 0.24 (−0.09–0.57)
	Incongruent	−0.31 (0.34) g = −0.12 (−0.55−0.30)	−0.53 (0.48) g = 0.19 (−0.14–0.53)	−1.12 (1.55) g = 0.24 (−0.09–0.57)

Note: Top row of values are unstandardized CACE estimates (SE in parentheses). Negative CACE values indicate reductions in ISD (i.e., improvements in behavioral performance). g = Hedges’ g (standardized mean difference between compliers and noncompliers at each percentile level). Values in parentheses represent 95% confidence intervals. Positive effect sizes indicate greater improvement in the compliers.

*p < .01.

Discussion

The current study conducted secondary data analyses of data from the BECT, a 2-year RCT that evaluated the impact of a school-based volunteer program for older adults. Building on the idea that IIV may be a sensitive indicator of cognitive plasticity resulting from interventions, we expected that participation in this cognitively, socially, and physically engaging volunteering program would be associated with improved (i.e., decreased) IIV in comparison to participants in the control condition. There was evidence of reduced IIV, but this varied by dosage, timeframe, RT condition, and sex.

In CACE models, we found that for those with higher levels of exposure (80th percentile), IIV improved on the more difficult incongruent condition after 24 months. The incongruent condition also showed a nonsignificant trend of improving IIV as treatment dose increased, at both 12 and 24 months, but no improvements were observed for the neutral condition after 12 or 24 months. These results are consistent with previous research from the Experience Corps program, which has reported improvements in processing speed (Carlson et al., 2018), only after 24 months, though the number of compliers at the 80th percentile was small.

The present comparisons at different doses, timeframe, and task complexity are valuable in pinpointing the specific criteria that were needed to observe change in IIV: a high dose of engagement, a lengthy period of engagement, and a complex cognitive outcome. These dose-dependent relationships potentially demonstrate that an intense, long-term, cognitively and socially engaging intervention program can result in improved cognitive ability, and that the time a person spends in the intervention is associated with the size of cognitive improvement. The present intervention treatment was multifaceted, providing participants with regular cognitive stimulation, physical activity, and socializing opportunities that required sustained attention and executive functions to shift flexibly across volunteer roles. Further, given the long duration and the multiple domains of stimulation through the intervention program, cognitive benefits from the BECT may accrue over multiple systems over time. Comparatively, analyses examining changes in IIV after a 2-year physical activity protocol (i.e., an intervention training a single domain), and cognitive engagement for a relatively short time period (14 weeks) have not been significant (Bielak & Brydges, 2019; Brydges & Bielak, 2019), further suggesting that the above criteria to observe improvements in IIV need to coalesce in one intervention, and, by extension, that improvements in IIV may only be specific to some forms of interventions. At the same time, other interventions that were successful in showing lower IIV were fairly brief (i.e., 8 weeks, 3 months), but implemented some form of direct training of attention (mindfulness training and meditation; Smart et al., 2016; Zanesco et al., 2018). Therefore, there may be multiple paths to influence IIV via lifestyle engagement, though training of attention (e.g., mindfulness, meditation, sustained attention and executive functions to be able to shift flexibly between volunteer roles) appears to be a key component if improvements in IIV are to be observed.

It is also possible that Experience Corps indirectly trained attention through its multiple domains of stimulation. That is, as the Experience Corps program has aspects of physical activity, social engagement, and cognitive stimulation, and it is possible that some common and/or domain-free attention construct is also being indirectly trained through the intervention. Previous research on data collected from the BECT has reported improvements in both memory and executive functioning (as measured by the Rey-Osterrieth complex figure task and the trail-making task, respectively; Carlson et al., 2008), both of which rely upon sustained and focused attention, potentially providing support for this idea. Although previous research has found that IIV is associated with general cognitive function (Hultsch, MacDonald, & Dixon, 2002), increased IIV has also been shown to be related to inattention (Brydges, Ozolnieks, & Roberts, 2018). With the present evidence, however, it is unclear which factor or combination of factors is necessary to reduce IIV in cognitively healthy older adults, and whether IIV is indeed a more sensitive index of cognitive plasticity than traditional metrics.

Exploratory analyses showed that men displayed a dose-dependent relationship on the incongruent task after 12 months. Carlson and colleagues (2018) reported a similar dose-dependent relationship for males in executive functioning and verbal learning, consistent with the results of the current study. Although there is some evidence that women show greater gains from physical exercise interventions (Colcombe & Kramer, 2003), differences by sex for social and cognitive engagement are unclear. Carlson and colleagues (2018) provided a number of explanations for differential change for men, including that providing care, teaching, and support to children may have been more novel for men compared to women. Alternatively, it could be the case that this dose-dependent relationship is mostly driven by a small proportion of highly compliant males, which would indicate a potential cutoff point for when volunteering has a meaningful effect on IIV, rather than a linear association between volunteering and IIV. Although the improvements observed in the men are encouraging, this sex difference should be interpreted with caution due to the small sample size (male n = 61), and the sex-split analyses not being preregistered.

Although the current study has reported some success in improving IIV, there are limitations. First, the analyses only included the Stroop RT task. The neutral condition task is relatively simple and a lack of improvement in IIV may be due to a ceiling effect. Future research should include a range of tasks of varying difficulty to investigative if reductions in IIV are more apparent on tasks that are increasingly demanding (e.g., N-back). Second, the results may not be generalizable to clinical populations. That being said, the present sample had a higher proportion of African Americans, and lower education and baseline MMSE values compared to other studies on aging (Bielak & Brydges, 2019; Bielak et al., 2010). Individuals with these characteristics have an elevated risk for cognitive impairment (Lyketsos, Chen, & Anthony, 1999), and therefore, interventions with a clinical sample (e.g., mild cognitive impairment), may also be effective at reducing IIV, or even lowering the risk of future cognitive impairment (Smart et al., 2016). Further, as cognitive impairment is associated with greater IIV than typically aging individuals (Tales et al., 2012), this may result in potentially greater room for improvement over the intervention period. Third, the RM-ANCOVA required the use of complete cases only, restricting the sample size. However, additional multilevel models including participants with complete covariate data and IIV for at least one time point (n = 582, neutral; n = 549, incongruent) showed there were no intervention × time effects for either ISD outcome, consistent with the RM-ANCOVA results.

The current study adds to the small but growing number of studies investigating whether IIV improves as a result of various engagement-based intervention programs in older adulthood (Bielak & Brydges, 2019; Brydges & Bielak, 2019; Smart et al., 2016; Zanesco et al., 2018). The Experience Corps program resulted in an improvement in performance (as measured by IIV) after 24 months, albeit only when a high treatment dose had been administered, and a dose-dependent improvement in IIV in men was observed after 12 months. It is possible that IIV can decrease further if the tasks used to measure IIV are sufficiently demanding, if the protocol were followed for an even longer period of time, or if the experimental sample is cognitively impaired. Based on associations between increased IIV and maladaptive outcomes (Batterham et al., 2014; Bielak et al., 2010; Bunce et al., 2008), a long-term reduction in IIV could be associated with improved health and cognitive ability, and decreased risk of mortality. The current study demonstrates that re-analyses and secondary data analyses of existing RCTs are potentially fruitful (and cost-effective) methods for further investigating the effects of interventions on cognition in aging populations, and was also preregistered to increase transparency in the findings. Given the time and cost required to design and implement a brand new intervention program that has no guarantee of working, analyzing previously collected data in new ways may be useful for shedding light on key characteristics of effective protocols.

Funding

This work was supported by the National Institutes of Health (R03AG055748 and R03AG055748-01S1 to A.A.M.B.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The Baltimore Experience Corps Trial was funded by the National Institute on Aging under contract P30-AG02133, and state and federal AmeriCorps grants. R.M.A. was supported by a grant from the National Institute on Aging (T32AG027668).

Conflict of Interest

None reported.