Assessing compliance: Active versus inactive trainees in a memory intervention

Abstract

Extensive research on memory interventions has confirmed their success with older adults, but the individual difference factors that predict successful training outcomes remain relatively unexplored. In the current intervention, trainees were identified as active (compliant with training regimens) or inactive using trainer ratings based on attendance, homework completion, and class participation. The active group showed significantly greater training-related gains than the inactive group and the control group on most measures. Compliance was predicted by health, education, and self-efficacy. Specifically, active trainees were more likely to have advanced degrees and somewhat higher self-efficacy, and to have higher vitality and fewer functional limitations than the inactive trainees. This research may assist future investigators to target interventions to those who will show the most benefit.

Memory training interventions have a long history in the research literature, with early work focusing on whether training would be successful (see Poon et al 1980), and more recent work focusing on co-contributing factors that affect memory performance, such as control beliefs (Lachman et al 1992), anxiety and stress (Valentijn et al 2005), memory education (Troyer 2001), attention (Plude 1992), and self-efficacy (West et al 2007) to name but a few. Meta-analytic reviews of the extant training literature clearly demonstrate that memory training group interventions have a positive impact on both subjective memory and memory test scores, with greater impact on test scores (cf, Verhaeghen et al 1992^; Floyd and Scogin 1997). Given that accumulation of evidence on the potential benefits of training programs, it is surprising that we know relatively little about individual difference factors that influence training outcomes. One factor that may be particularly important is the extent to which an individual invests in the training program and complies with training instructions. Although training compliance has been extensively studied in some other intervention domains, to our knowledge, it has not yet been investigated in the memory training field.

Because measures of compliance are rarely reported in the cognitive intervention literature, the exercise, diet, and medical compliance literature informed this investigation. In reviewing this literature, it is important to note the distinction between outcome measures for an intervention, such as achieving a set amount of weight loss (Bautista-Castano et al 2004) and participant compliance measures such as daily caloric intake (Glanz et al 2006) or number of days of exercise per week (Courneya et al 2004). Only true intervention studies with a control group and a stated measure of compliance or adherence are considered in this review.

Overall, compliance with study objectives is primary to any successful intervention (DeKosky 2006) and ultimately to the participant’s well-being, broadly defined (for a general review, see Roter et al 1998). Compliance is clearly related to positive outcomes in exercise studies (Naslund et al 1996; Perna et al 1999; Morey et al 2003). Compared with their nonadhering counterparts, compliers consumed less dietary fat (Naslund et al 1996), achieved greater weight loss (Jeffery et al 1984), and showed a greater reduction in risk factors for obesity (Bautista-Castano et al 2004). It is also clear that adherence is malleable and can be modified (Kaplan et al 1984).

Measurement of compliance is a significant issue in this literature. Compliance is often measured via attendance records (Grove and Spier 1999) or self-reported activity logs (Stetson et al 1997). Some research has attempted to create higher-order compliance scores that take into account both attendance records and social cognitive factors (Maddison and Prapavessis 2004). In the exercise literature, measures of quantitative adherence are common, including heart rate (Brassington et al 2002), oxygen uptake (Morey et al 2003), and energy expenditure, (eg, tread mill scores Courneya et al 2004), and some researchers have attempted to correlate activity logs with more objective measures, such as heart-rate (Wilbur et al 2001). A more generalized approach to compliance, using global trainee ratings, is employed in this investigation.

What factors affect compliance? The extant research shows that reduced compliance is associated with overly complex routines, as over 74% of the participants in one particular study (Topinkova et al 2006) self-reported a memory lapse as the primary reason for adherence failure. Compliance is also reduced as perceived stress increases (Stetson et al 1997). Improvements in compliance, in contrast, are associated with psychosocial strategies, such as set weekly activity schedules and cognitive-behavioral therapy (Reilly-Harrington and Sachs 2006). Increasingly, investigators are tailoring outcome expectancies to participant-set goals, which may lead to higher compliance rates (Konradi and Lyon 2000; Bautista-Castano et al 2004). For exercise, in particular, compliance is increased via peer-led sessions (Grove and Spier 1999), social support (Fraser and Spink 2002), and improved self-efficacy (Maddison and Prapavessis 2004). Strategies that lead to better record-keeping also aid in compliance (Rosen et al 2004; Glanz et al 2006; O’Hara, Vethanayagam and Mayers 2006). This particular intervention targeted self-efficacy beliefs, provided extensive support for trainees to meet their own goals, and offered many opportunities for trainees to learn from each other. Based on the earlier literature in exercise and medical research, these elements should foster strong compliance.

In the memory training literature, attrition has often been assessed, but levels of adherence have been almost completely neglected. Many formats for memory training have been successful, including didactic group sessions for learning mnemonic strategies (Ball et al 2002; West et al 2007), CD-ROM training (Saczynski et al 2004), videotape training (West and Crook 1992), and audiocassette training (Rebok et al 1997). Of course, improved memory is not always guaranteed (cf, Hill et al 1988^; Lachman 1991^; Schmidt et al 1999^; Valentijn et al 2005), and the successful effects of training are often short-lived and/or do not transfer to other cognitive domains that were not directly trained (West et al 2000; Rebok et al 2007). As a consequence, the focus of future memory interventions should not center on the efficacy of memory-training per se, as most such interventions produce positive results—immediate memory gains on trained tasks (Verhaeghen et al 1992). Rather, the emphasis should be on understanding who benefits most from training. Given that the literature in other health domains shows a relationship between compliance and outcome, the immediate and long-term benefits of memory training may also be influenced by compliance with the training regimen.

To our knowledge, no previous memory training studies have focused on compliance issues. One possible reason for this is that few training programs require trainees to do more than attend regular class sessions. Although quite a few training programs required homework (Schmidt et al 1999; Troyer 2001) the majority of memory interventions focused on the didactic delivery of classroom content and not on participants’ compliance with assigned activities. However, a few other individual difference factors related to training impact have been explored. As early as the late 1970s, Poon and his colleagues (1978) argued for the possible interaction of individual differences and training impact. Since then, a few studies have addressed anxiety, age, cognitive skill or level of impairment, personality, education, and depression as potential individual difference factors that could predict training success (for reviews, see Hill et al 2000). However, investigations related to any one issue are few and far between, making it difficult to conclude anything about those critical individual difference factors that magnify or reduce training effects. Here, rather than target a specific characteristic, we identified a group of trainees who were actively engaged and compliant with training regimens, to address two questions: Do compliant trainees in a memory intervention have better training outcomes? If yes, what individual difference characteristics distinguish compliant and noncompliant trainees?

Methods

Participants

The participants were well-educated (M = 15.5 years of education), middle-aged and older adults in the southeast USA who were seeking memory training. Participants were recruited by offering lifelong learning programs on memory at existing residential communities for seniors, by newspaper and newsletter notices, and by direct mail sent to former research participants. At pretest, there were 180 participants, ranging in age from 54 to 92 (M = 70.5, SD = 7.6). Most participants were assigned to the wait-list control condition (N = 41) or the training condition (N = 115) for the entire 9-week program; some participants (N = 24) were in a partial control condition—they were controls at week 5, but received training before week 9. One-fourth of the trainees (25.2%) and controls (26.8%) were male and three-fourths were female. Nine of the recruited participants were dropped from the study due to recent strokes, difficulty completing the interview, or use of anticholinergic medications.

Overview

All individuals were interviewed in groups, with group size ranging from 7 to 19 adults (M = 14.6, SD = 3.3). Interviews lasted 1.5 to 2 hours. Instructions were presented bimodally, with individuals reading printed instructions as the experimenter read them aloud. Groups were assigned randomly to conditions for the 9-week program. Assessments were carried out at week 1 (pretest), week 5 (posttest-1) and week 9 (posttest-2).¹ Primary comparisons of the groups with respect to training effects examined pretest to posttest-1 change, comparing active and inactive trainees with controls who had not yet received training at posttest-1 (including partial controls and wait-list controls). As a replication, we examined change from pretest to posttest-2, comparing active and inactive trainees with wait-list control participants who had not had training as of posttest-2. Significant training effects (comparing trainees and controls) for this intervention program were established and described in greater detail in previous research (West et al 2007). The focus of this study was to compare active and inactive trainees.

Assessments

A summary description of the assessments is provided here, as a detailed description has been published elsewhere (West et al 2007). The social cognitive questionnaires included the locus, achievement, and anxiety subscales (range = 1 to 5 for each subscale) of the Metamemory in Adulthood measure (MIA; Dixon et al 1988), General Self-Efficacy (Schwarzer 1993; range = 10 to 40), and Need for Cognition (Cacioppo et al 1984; range = 18 to 90) for baseline comparisons across groups. The Memory Self-Efficacy Questionnaire-4 (MSEQ-4; West et al 2007), one of our primary outcome measures, had 20 items assessing self-efficacy on 5 levels of performance for each of 4 memory tasks, including grocery list, story, name, and object location recall. Following standard procedures, scores are averaged across all 20 items, and range from 0 to 100.

Three memory tests were administered for two trials each—shopping list recall (15 items at Level 1 and 35 items at Level 2) and names and faces (12 faces at Level 1 and 24 faces at Level 2).² To develop the Level 2 tests, we added more to-be-remembered items or sentences to those items already presented at Level 1, so that participants could experience improvement on the Level 2 tests (see West et al 2007). Each Level 1 test was administered with one minute for study and five minutes for retrieval. Each self-paced Level 2 test provided up to five minutes for study and five minutes for retrieval.

The Level 2 test for each memory task was followed by a self-report strategy questionnaire (West et al 2007). The self-report strategy measure assessed the individual’s usage of common memory strategies (eg, “concentrated,” “tested myself,” “rehearsed”) in addition to the specific strategies taught during the intervention (eg, “image-name-match method,” “PQRST”). Detailed analyses of usage for individual strategies were reported elsewhere (West et al 2007). Here, the focus is on the total number of strategies reported, as this variable is an indicant of strategic effort and generally correlates well with recall (West et al 2007). Scores on each strategy checklist could range from 0 to 15.

Basic demographic information was collected, including age, education, gender, and health information. The latter included a general self-rating, plus scales assessing mood, vitality, pain, activities of daily living, and limitations in social or other activities due to health problems, from the SF-36 (Ware and Sherbourne 1992). In addition, the digit-symbol subtest (Wechsler 1981) was administered as a general measure of processing speed, because it is known to impact strategy usage (Verhaeghen and Marcoen 1994), and a vocabulary scale was utilized as a general measure of verbal intelligence (Shipley 1940). These measures were used for group comparisons at the pretest. All other tests and questionnaires were administered at the pretest and posttest-2. The posttest-1 interview included only the memory tests and strategy checklists.

Intervention

The purpose of the intervention was to improve memory and increase memory self-efficacy for trainees. The training classes consisted of a two-hour session each week (weeks 2, 3, 4) that targeted specific memory improvement techniques and self-efficacy beliefs. The final (week 6) session was a wrap-up session encouraging trainees to continue to work on memory improvement at home. Assessments were conducted at week 1, week 5, and week 9. All training sessions were scripted to include small-group and large-group discussions of readings and homework assignments, didactic instruction on one or two specific strategies, and practice exercises for the strategies. The content of the training program was consistent with other multifactorial approaches to group training, offering didactic instruction on a number of valuable memory strategies, including: organization, association, the image-name-match method for recalling names (West 1985), attention, and a strategy for remembering written text which involves steps of Preview-Question-Read-Summarize-Test (PQRST; West 1985). Unlike other interventions, the entire program was structured to enhance self-efficacy, for example, easier strategies, simpler homework readings and exercises, and less demanding memory skills were presented first, followed by more difficult ones, to encourage early mastery. In addition, more homework was required than is typical in group training programs. The intervention required about three hours per week, including at least one hour of homework per week, on average, for those who were fully compliant with instructions: 15.5 pages of reading, 3 to 4 homework questions, and 8 practice exercises. Complete training details are available elsewhere (West et al 2007).

Following Nasland and colleagues (1996), participants were classified on a global rating scale with respect to their compliance with the intervention. These ratings, on a 1 to 5 scale, were based on an overall evaluation of these six aspects of participation: attendance, homework completion (reading, answering questions, completing practice exercises), the quality of questions asked in class, active involvement in small-group discussions, ability to answer questions when called upon, and general class attentiveness. Participants scoring a 4 or 5 were judged to be active trainees (AT) and those scoring a 3 or less were judged to be inactive trainees (IT). Two trainers assigned subjective compliance ratings to the first 40 participants, with very high agreement (r = 0.90, p < 0.001); thereafter one person completed the ratings. The few disagreements in the initial set were settled by discussion. For analyses of posttest-1 scores, there were 18 participants designated as inactive trainees, 58 active trainees, and 46 control trainees with completed interviews. For analyses of posttest-2 scores, there were 31 controls, 22 inactive and 63 active trainees identified.

Results

The primary univariate analyses examined the impact of AT-IT group differences (active and inactive trainees and controls) on attendance, self-efficacy, and strategy usage. Name and list recall measures were examined separately using a mixed design multivariate analysis of variance with two trials (Level 1 and Level 2) of name recall and list recall scores at each of two interviews (pretest and posttest). Primary analyses examined changes from pretest to posttest-1 (N = 122), with a replication that examined changes from pretest to posttest-2 (N = 116).³ Main effects for AT-IT group would reveal that one group performed consistently better than the other, across all levels and interviews. Interactions of AT-IT group and level would show that at least one group was more willing to invest additional effort to perform well on the more challenging Level 2 tests, and interactions of AT-IT group and interview would reveal greater training gains for one group over another. Significant differences were interpreted based on 95% confidence intervals for relevant means, and effect sizes are partial η² values, both reported by SPSS (Statistical Package for the Social Sciences, Version 13.0; SPSS Inc., Chicago, IL, USA).

Preliminary analyses

Statistical comparisons of week 1 scores for trainees and wait-list controls revealed no significant pretest differences between trainees and controls in age, education, self-rated health, memory scores, general or memory self-efficacy, metamemory self-ratings, need for cognition, vocabulary, or digit-symbol performance. Furthermore, there were no significant baseline ability differences between the IT and AT groups on baseline ability measures including name and list recall, strategy usage, vocabulary, or digit-symbol performance (all p > 0.30).

Attendance

The AT and IT groups were compared on the traditional compliance measure of attendance. On average, the active trainees missed less than one training session (M = 0.44, SD = 0.57) whereas the inactive trainees missed more than one (M = 1.61, SD = 1.47), F (1, 136) = 37.4, p < 0.001, η² = 0.22.

Pretest to posttest 1

Multivariate recall tests

To examine pretest to posttest-1 changes in performance, a multivariate analysis with all three groups (controls, AT, and IT) included list recall and name recall scores at Level 1 and Level 2 for both interviews. The results were significant for group differences, F (4, 238) = 2.5, p < 0.05, η² = 0.04, and there were significant main effects for interview, F (2, 118) = 17.8, p < 0.001, η² = 0.23, and level, F (2, 118) = 461.9, p < 0.001, η² = 0.89, a significant interaction of interview and level, F (2, 118) = 8.3, p < 0.001, η² = 0.12, and a significant interaction of group and level, F (4, 238) = 4.0, p < 0.005, η² = 0.06. Univariate follow-up analyses, to identify the source for these significant differences, examined list and name recall scores separately in repeated measures analyses with two levels, comparing the results for the two interviews.

Name recall

Overall, posttest-1 scores exceeded pretest scores on name recall, F (1, 119) = 25.3, p < 0.001, η² = 0.18. For name recall, there were significant gains from pretest to posttest-1 (averaged across all scores) for the AT group, but not for the IT or control groups, F (2, 119) = 4.0, p < .05, η² = .06, revealing a training effect for the active group only. Level 2 scores (averaged across the two interviews) exceeded Level 1 scores, F (1, 119) = 278.0, p < 0.001, η² = 0.70, for all three groups. There was a significant interaction of level and interview for names, F (1, 119) = 12.8, p < 0.001, η² = 0.10, due to the fact that posttest-1 scores exceeded pretest scores at both levels, but the gains across test sessions were larger on the Level 2 tests. No other differences were significant.

List recall

Overall posttest-1 scores exceeded pretest scores, F (1, 119) = 3.9, p < 0.05, η² = 0.03. Level 2 scores exceeded Level 1 scores (averaged across the two interviews), F (1, 119) = 927.8, p < 0.001, η² = 0.89. There was a significant interaction of level and group due to the fact that group differences varied on the two trials, F (2, 119) = 7.3, p < 0.001, η² = 0.11. Posthoc comparisons of scores (averaged across the two interview sessions) showed that the control and IT groups were comparable on list recall at Level 1, and the AT group surpassed the IT group. For Level 2 list recall, both the AT and control groups surpassed the IT group. The training effect (an interaction of interview and condition) was not significant for lists. Finally, overall comparisons, averaging across all tests and both interviews, showed a main effect for group for list recall, F (1, 119) = 4.8, p < 0.01, η² = 0.08. The AT group scored significantly higher than the IT group on list recall, with the control group scoring in between. This finding, as well as the Level 2 data, indicated a reduced willingness on the part of the IT group to put effort into the memory tests over time, because their scores had been similar to that of the AT and control groups on the baseline list recall measure, as shown in Table 1.

Table 1.

Baseline means (standard deviations) for trainees and controls

Measure	Control group	Trained group	Type of trainee
Age	70.5 (7.3)	70.4 (7.7)	All
		71.7 (8.6)	IT group
		69.6 (7.0)	AT group
Education	15.6 (2.8)	15.5 (3.1)	All
		14.6 (3.5)a	IT group
		16.1 (2.7)b	AT group
Vocabulary	34.6 (4.6)	33.4 (6.2)	All
		33.9 (3.6)	IT group
		33.0 (7.4)	AT group
Digit-symbol	45.5 (14.1)	49.4 (10.0)	All
		48.8 (9.8)	IT group
		50.0 (10.1)	AT group
Name recall	5.1 (2.7)	5.0 (2.7)	All
		4.8 (3.0)	IT group
		5.1 (2.5)	AT group
List recall	9.4 (2.3)	9.6 (2.3)	All
		9.4 (2.2)	IT group
		9.8 (2.3)	AT group
Need for cognition	63.6 (8.9)	59.0 (12.1)	All
		59.1 (12.8)	IT group
		58.9 (11.9)	AT group
General self-efficacy	31.4 (3.8)	30.8 (3.5)	All
		30.8 (3.8)	IT group
		30.8 (3.3)	AT group
Self-efficacy	48.7 (14.8)	47.0 (17.0)	All
		44.0 (18.5)	IT group
		49.3 (15.6)	AT group
MIA anxiety	3.4 (.73)	3.3 (.68)	All
		3.3 (.72)	IT group
		3.3 (.66)	AT group
MIA locus	3.8 (.51)	3.8 (.47)	All
		3.8 (.59)	IT group
		3.8 (.37)	AT group
MIA achievement	4.1 (.35)	4.0 (.39)	All
		4.0 (.43)	IT group
		4.0 (.37)	AT group
Self-rated health	3.0 (1.6)	3.0 (1.6)	All
		3.5 (1.9)a	IT group
		2.6 (1.2)b	AT group

Note: All measures in this table represent baseline or first trial scores. The right column shows the training group total first, with the results for the two identified types of trainees immediately below that. Means that are significantly different from each other (p < 0.05) are indicated by different superscript letters. No other baseline means were significantly different from each other. Variations between these scores and those reported in Table 2 are due to the fact that Table 1 includes all participants and Table 2 includes only those participants that were not missing any Posttest-1 or Posttest-2 data.

Abbreviations: MIA, Metamemory in Adulthood measure; IT, inactive trainee; AT, active trainee.

Multivariate strategy tests

To examine pretest to posttest-1 changes in strategy usage, strategy scores for list recall and name recall for all groups were entered in a multivariate analysis. The results were significant for interview, F (2, 123) = 10.8, p < 0.001, η² = 0.15, and for the interaction of interview and group, F (4, 248) = 2.6, p < 0.05, η² = 0.04. Univariate follow-up analyses for list recall and name recall (separately) were designed to explore these multivariate results.

Name strategies

For name recall, participants used more strategies overall at posttest-1 than at pretest, F (1, 124) = 18.3, p < 0.001, η² = 0.13. There was a significant interaction between interview and AT-IT group, due to a significant training-related increase in strategy usage from pretest to posttest-1 for the AT group, but no strategic gains for the control group or IT groups, F (2, 124) = 4.8, p < 0.01, η² = 0.07.

List strategies

For lists, most participants used more strategies at posttest-1 than at pretest, F (1, 124) = 4.9, p < 0.05, η² = 0.04, showing a practice effect, but there was no significant interaction of group and interview, indicating no differential gain in strategy usage on this task as a result of training. List recall is clearly a task on which individuals can show practice-related gains in strategy and performance without training because the strategies needed for this task (eg, using food categories) are well-known and easy to implement (see also Camp et al 1983).

Pretest to posttest 2

Multivariate recall tests

In a replication, the list and name recall data for Level 1 and Level 2 for the three groups (AT, IT, controls) were examined in a multivariate analysis exploring pretest to posttest-2 change. Note that the participants included in particular condition groups in this analysis are not the same as those included in those groups in the earlier analysis, due to attrition, and due to the fact that 24 control participants at posttest-1 received training between posttest-1 and posttest-2, so their group assignment changed for these analyses. The multivariate results for the memory performance measures revealed significant differences for interview, F (2, 112) = 16.5, p < 0.001, η² = 0.23, and level, F (2, 112) = 367.7, p < 0.001, η² = 0.87, and a significant interaction of interview and level, F (2, 112) = 7.6, p < 0.001, η² = 0.12. There were no overall group differences, but group interacted significantly with level, F (4, 226) = 2.7, p < 0.05, η² = 0.05, and with interview, F (4, 226) = 3.3, p < 0.025, η² = 0.06, showing a training effect.

Name recall

Univariate analyses to follow up the multivariate findings revealed that posttest-2 scores exceeded pretest scores (averaging across both levels), F (1, 113) = 19.4, p < 0.001, η² = 0.15. For name recall, there was a significant training effect for the AT group, with gains from pretest to posttest-2 (averaging across both levels), but no significant gains across interviews for the IT or control groups, F (2, 113) = 3.3, p < 0.05, η² = 0.06. Level 2 scores exceeded Level 1 scores, F (1, 113) = 237.3, p < 0.001, η² = 0.68. There was also a significant interaction of level and interview for names, F (1, 113) = 7.4, p < 0.01, η² = 0.06, due to the fact that posttest-2 scores exceeded pretest scores at both levels (averaged across groups), but these differences were significantly larger for Level 2 tests (see Table 2). No other differences were significant.

Table 2.

Means (standard errors) for recall, strategies, and self-efficacy

Variable	Pretest	Posttest-1	Posttest-2
Recall Scores
List recall
Level 1
Control	9.1 (0.39)	9.7 (0.34)	10.1 (0.41)
Inactive	9.4 (0.50)	8.9 (0.54)	9.3 (0.49)
Active	9.7 (0.27)	10.1 (0.30)	10.6 (0.29)
Level 2
Control	18.0 (0.90)	20.6 (0.71)	20.8 (1.0)
Inactive	17.3 (1.1)	16.9 (1.1)	17.5 (1.2)
Active	20.2 (.63)	20.9 (0.64)	21.6 (0.68)
Name recall
Level 1
Control	5.1 (0.47)	5.5 (0.40)	5.9 (0.48)
Inactive	5.0 (0.56)	4.8 (0.64)	5.0 (0.57)
Active	5.0 (0.33)	5.9 (0.36)	6.1 (0.34)
Level 2
Control	11.6 (1.1)	13.3 (0.91)	12.7 (1.1)
Inactive	10.2 (1.3)	11.1 (1.5)	11.4 (1.3)
Active	10.5 (0.75)	13.7 (0.81)	13.6 (0.79)
Number of strategies
List strategies
Control	3.5 (0.28)	4.2 (0.25)	4.3 (0.3)
Inactive	4.0 (0.33)	4.0 (0.38)	4.1 (0.36)
Active	3.8 (0.19)	4.4 (0.22)	4.7 (0.20)
Name strategies
Control	4.2 (0.33)	4.9 (0.34)	4.6 (0.40)
Inactive	3.6 (0.39)	4.7 (0.51)	4.6 (0.48)
Active	4.2 (0.22)	5.4 (0.30)	5.8 (0.27)
Beliefs
Self-efficacy
Control	48.8 (3.1)		46.0 (2.9)
Inactive	46.6 (3.6)		49.8 (3.3)
Active	48.8 (2.0)		54.9 (1.9)

List recall

Univariate analyses to follow up the multivariate findings revealed that posttest-2 scores (averaged across groups at both levels) exceeded pretest scores for list recall, F (1, 113) = 16.1, p < 0.001, η² = 0.12. The AT group showed a training effect but the IT group did not, F (2, 113) = 3.3, p < 0.05, η² = 0.06, and the control group significantly increased their shopping list scores. Level 2 scores (across both conditions and both test sessions) exceeded Level 1 scores for list recall, F (1, 113) = 733.2, p < 0.001, η² = 0.87. There was also a significant interaction of level and AT-IT due to the fact that all groups were comparable on the Level 1 test, but the AT and control groups surpassed the IT group on the Level 2 list recall test, F (2, 113) = 5.1, p < 0.01, η² = 0.08. This evidence, along with the lack of a training effect for the inactive trainees, suggests that the IT group did not invest effort during the interviews. Combining across groups, there was a significant interaction of level and interview for lists, F (1, 113) = 4.6, p < 0.05, η² = 0.04. Posthoc comparisons showed that posttest-2 scores exceeded pretest scores at both levels, but these differences were significantly larger for Level 2 tests. Finally, overall comparisons, averaging across all tests and both interviews, showed that the AT group scored significantly higher than the IT group for list recall, with the control group performing comparably to both trainee groups overall, F (2, 113) = 3.8, p < 0.05, η² = 0.06, as seen in Table 2. No other differences were significant.

Multivariate strategy tests

For the strategy measures, multivariate comparisons of the pretest and posttest-2 scores for list recall and name recall showed no overall differences between the groups, but there was a significant main effect for interview, F (2, 116) = 18.5, p < 0.001, η² = 0.24, and group interacted significantly with interview, F (4, 234) = 2.8, p < 0.05, η² = 0.04, to reveal a training effect. Univariate follow-up analyses explored these multivariate findings separately for list and name recall.

Name strategies

For name recall, the number of strategies used on posttest-2 (collapsed across groups) exceeded that for the pretest on name recall, F (1, 117) = 30.6, p < 0.001, η² = 0.21. For name recall, there was a training effect, with significant gains in strategy usage from pretest to posttest-2 for the AT group and the IT group, but not the control group, F (2,117) = 4.0, p < 0.025, η² = 0.06. Although both the IT and AT groups used significantly more name strategies after training than before, the AT group showed significantly higher strategy usage than the IT group at posttest-2, due to much larger training-related gains in strategy usage (see Table 2). The control group showed no change in strategy usage across the two interview sessions.

List strategies

For list recall, the number of strategies used on posttest-2 exceeded that for the pretest for all groups, F (1, 117) = 10.7, p < 0.001, η² = 0.08, reflecting practice-related gains. There was no significant interaction of group and test session, and no other differences were significant.

Self-efficacy

Comparisons of pretest and posttest-2 scores showed that self-efficacy did not change overall, but there was a significant interaction of interview and group, F (2, 115) = 3.8, p < 0.025, η² = 0.06, due to the fact that self-efficacy declined slightly for the control group, increased significantly for the AT group, and showed no change for the IT group from pretest to posttest-2. Posthoc comparisons showed that the AT group exceeded the control group in self-efficacy at posttest-2, with the IT group in the middle and not varying significantly from either group. The active group was feeling more confident as their scores were improving, but the inactive group was not showing any efficacy changes over time. Repeated testing commonly leads to reduced self-efficacy (Berry et al 1989; Dittmann-Kohli et al 1991; West et al 1996). The fact that the IT group maintained their initial level of self-efficacy after three tests suggested that the training had some influence on their beliefs, even though analysis of the performance measures showed that the IT group showed significant gains only for strategy usage at posttest-2.

Summary

A number of indicators varied between the inactive and active trainees, including attendance. The AT group showed consistent training-related gains for name recall, performing significantly better on both posttests than on the pretest and performing significantly better than the control group, but these gains were not evident for the IT group. In a number of ways, the IT group failed to invest effort in their performance. Although they did use more strategies than the control group and showed strategy gains by posttest-2, they used fewer memory strategies than the active trainees. Given that there were no baseline differences between groups, indicating comparable levels of basic ability, the interactions of AT-IT and level reveal that even after training, the IT group was not willing to put forth effort on the more challenging Level 2 tests. Finally, the control group was able to raise their scores on list recall, revealing practice-related gains, but the IT group did not show this benefit from practice either, again suggesting a lack of willingness or an inability to invest effort in improving their test performance. Interestingly, the IT group was able to maintain reasonably high self-efficacy (higher than controls) even though their test scores did not improve, whereas the AT group increased both self-efficacy and performance.

Predicting compliance

Given these significant performance, strategy, and attendance differences between active and inactive trainees, and no significant ability differences between the two groups at baseline, we wanted to identify other individual differences factors that might predict who would be most compliant and thus benefit the most from training. Scores on the demographic (age, years of education, self-rated health) and social cognitive measures (general self-efficacy, memory self-efficacy, three MIA scales) were compared for the AT and IT groups (control participants were not included). These analyses showed that education, F (1, 132) = 8.3, p < 0.005, η² = 0.06, varied between the two groups; although both groups were highly educated, with most participants having a high school degree or some college, the active trainees (M = 16.1, SD = 2.7) had significantly more years of education then the inactive trainees (M = 14.6, SD = 3.5). Similarly, self-rated health was also relatively high, with most participants giving scores above 3 on a scale from 1 to 10. Self-rated health differed significantly between the two groups, F (1, 132) = 11.5, p < 0.001, η² = 0.08, with higher health ratings for the active (M = 2.6, SD = 1.2) than the inactive trainees (M = 3.5, SD = 1.9) trainees. Self-efficacy differences between the active trainees (M = 49.3, SD = 15.6) and the inactive trainees (M = 43.9, SD = 18.5) approached significance, F (1, 120) = 3.0, p < 0.10, η² = 0.02. No other pretest scores distinguished between the two groups.

To better determine whether these factors could predict compliance ahead of time, before the individual completed the intervention, a discriminant analysis was utilized, entering education, health, and self-efficacy as predictors. In applying this analysis, the discriminant procedure was set to assume a 60–40 split for the two groups (because more of the trainees were active than inactive), and then the analysis classified people into AT and IT groups based only on their pretest values for education, health, and self-efficacy. The results showed that 72% of the trainees were correctly classified into their groups (Wilks λ = 0.82, χ² = 23.0, df = 3, p < 0.001) when the appropriate coefficients were applied to the individual scores; more importantly, only 9.5% of those who were active/compliant were misclassified into the inactive group. With these coefficients (health = 0.845, education = −0.473, self-efficacy = 0.036), an investigator could use the pretest scores to calculate an overall discriminant value and identify participants who were most likely to benefit from training.

Further exploration of the predictor variables was carried out to understand more about these group differences. For self-efficacy, we examined the four self-efficacy scales separately, but found no significant differences on individual subscales; it is only the overall score that shows group variations. For education, we examined the educational degree classification of the participants in the two groups. This analysis showed that the AT group members were more likely to have post-graduate degrees whereas those in the IT group were more likely to have only finished high school. For the IT group, 44% had high school diplomas, 40% attended college, and 16% had completed masters or PhD degrees. In contrast, for the AT group, 17% had high school diplomas, 49% had attended college, and 34% had advanced degrees. We also conducted in-depth analyses of potential health differences between the AT and IT groups using the SF-36 subscales of mood, vitality, pain, activities of daily living, and limitations in social or other activities due to health problems. Two of these specific health factors varied between the AT and IT groups. The AT group had higher “vitality” (feel full of pep, have a lot of energy) scores than the IT group, F (2,162) = 5.6, p < 0.005, η² = 0.06, and the IT participants reported more often having to limit their work or daily activities because of health issues, F (2, 163) = 3.9, p < 0.05, η² = 0.05.

Discussion

As shown in past research, compliance was a predictor of intervention success. Because no baseline differences existed between the active and inactive groups on memory tests, vocabulary, speed of processing or number of strategies utilized, initial ability differences on these measures are not the explanation for the observed patterns of intervention-related gain for these two groups. From the first day of memory training, all participants had an equal chance of success. For this intervention, participants who were more invested or expended more effort on the homework and in the class—defined as compliance—gained the most from training. This research revealed, for the first time, that compliant memory trainees were most likely to have higher education, better self-rated health and somewhat higher self-efficacy than less compliant trainees.

Education

In prior research, higher education levels have been a consistent predictor of cognitive change (eg, Albert et al 1995^; Small et al 1995) and education levels have also predicted increased strategy usage in cognitive training studies (Verhaeghen et al 1992; Saczynski et al 2002). Also, strategy use increases as knowledge of aging and memory increases (Reese and Cherry 2006), suggesting that those with more background knowledge (likely those with more education) would tend to learn the trained strategies. Higher education levels are typically associated with more professional occupations, leading to a lifetime of more mentally challenging activities, which is related, in turn, to reduced late life memory impairment and more cognitive reserve (for a review see Scarmeas and Stern 2003). Further, an orientation towards lifelong learning could allow a more open orientation towards learning in a training class. Often in memory classes trainees try out several strategies before they identify those that work best, thus a significant increase in the number of strategies employed reveals that participants were seeking to maximize their scores. By posttest-2, the inactive trainees were showing evidence that they were at least trying more strategies. However, the higher-educated active group showed earlier and higher strategy gains, and significant performance gains not observed in the inactives. The more educated trainees were clearly more willing to expend the extra effort needed to implement and benefit from the trained strategies.

Health

Health differences also predicted the inactive designation. The IT group showed lower SF-36 vitality scores and greater propensity to limit daily activities because of health issues. In past research by Murrell and colleagues, higher education levels predicted better health (2002), higher vitality ratings (2003), and buffered low energy levels (2004). In our study, perhaps less energy and greater fatigue by the IT group created barriers to expending full mental energy on the test measures, especially on the more challenging level 2 memory measures. Other memory interventions have shown that older adults can easily learn strategies to improve memory performance but often fail to implement the techniques later, in part because effective strategies require continued high levels of effort to achieve mastery (eg, Camp 1999).

Self-efficacy

Self-efficacy is a strong predictor of compliance in the medical intervention literature, predicting medication adherence and smoking and alcohol avoidance (Schweitzer et al 2007). Self-efficacy also predicts adherence to exercise goals (Stetson et al 1997). In fact, King and colleagues (1992) found self-efficacy to be the most consistent psychological predictor of exercise maintenance. Here, self-efficacy change was related to the AT-IT designation, with the active trainees as the only group showing significant increases in self-efficacy. Given that this particular intervention was specifically designed to boost self-efficacy, it is possible that without the efficacy boost, the lower performing IT group members might have attended even more sporadically, preventing them from making the modest gains they did achieve. Furthermore, baseline efficacy might have had a stronger impact on compliance had we not focused on self-efficacy during training. Repeated testing tends to lower self-efficacy, as it did in the control group, but the IT group did not show that decline. This maintenance of positive beliefs by the IT group, despite a lower commitment to the training regimen, suggests that the positive self-efficacy elements of the program were absorbed even though the trained strategies were not well learned. One possible implication of this result is that self-efficacy-based interventions, combined with training on compensatory memory techniques, could prove useful for stabilizing the beliefs of mildly demented older adults even as they face increasing memory losses.

Measuring compliance

Although past studies have used attendance records as direct or indirect measures of compliance, attendance was just one of several factors in the global assessment of compliance used here. Because attendance records merely document that a seat was occupied, not that participation or material engagement occurred (Wilbur et al 2001), it is possible that other, less successful, interventions unknowingly experienced poor compliance in spite of satisfactory attendance. The most striking evidence of attendance not guaranteeing the transmittal of knowledge is our finding that the non-trained control group outperformed the trained IT group on Level 2 lists. Given the strong association between compliance and intervention outcomes here, it is apparent that compliance ratings can begin to unravel the individual differences between motivated and unmotivated trainees. More importantly, the rating methodology used in this research can be generalized easily to a wide range of interventions that require participant involvement, in such a way as to identify individual difference factors predicting compliance in other intervention domains.

Future directions

Estimations hold that delaying the onset of Alzheimer’s by six months could reduce dementia incidence by a million cases (Brookmeyer et al 1998), and recent literature suggests a link between memory test scores and mild cognitive impairment (Loewenstein et al 2006), believed to be the intermediary stage between normal aging and Alzheimer’s disease (Gauthier et al 2006). Much research supports the view that memory loss is possibly preventable by engaging in leisure activities (Singh-Manoux et al 2003), mentally challenging activities (Hultsch et al 1999), and physically challenging activities (Verghese et al 2003; Small et al 2006); although, for such behaviors to be beneficial, individuals must first invest effort in these activities. Because all forms of noncompliance are estimated to cost US$100 billion annually (Lewis 1997), it behooves scholars to focus more on compliance in the cognitive training literature. With a burgeoning baby boomer cohort, cognitive training programs are of paramount value, but only if researchers can ensure that trainees are motivated to comply with the intervention requirements.

Future research should be directed at exploring some of the variables that were not considered in our analysis. Unmeasured variables, such as personality factors (Gratzinger et al 1990; Schmidt et al 2001), might mediate the AT-IT group designation, as health, education, and self-efficacy did here. Persistence, conscientiousness, or agreeableness might represent personality traits that would be interesting to explore in this regard. Further, it is possible we inadvertently captured individuals, in our active group, that were more motivated to improve. Had we measured dimensions of motivation directly, considerable predictive variance might have been captured by these variables. Future intervention work should take these limitations into consideration during design and implementation of compliance studies.

As memory intervention work moves forward, two different approaches might have heuristic value. One approach would be to target trainees that will benefit most from training. This project showed that memory gains and self-efficacy increases are possible and that those who show sustained effort can make stronger advances. Our health, education, and self-efficacy coefficients could be used with baseline data to identify individuals who will gain the most. We could then target those individuals and thereby maximize training impact. A second approach would be to develop new training methodologies to encourage better gains in older adults who are not inclined initially to invest effort. In this research, investigators would target less educated or less healthy individuals with lower self-efficacy, who would still benefit greatly from training under the right circumstances. Several possibilities for increasing compliance with such individuals might involve setting goals, collecting simple homework assignments, scheduling reminder calls, using daily or weekly logs, and creating individually tailored training programs that take into account less education (eg, train less demanding strategies, as suggested by Camp 1999) or health limitations (eg, bring training into the home), while continuing to boost self-efficacy. Together, these two approaches would help to take memory intervention research to the next level, to encourage more mental challenge and greater memory success for all seniors.