Investigating Message Framing to Improve Adherence to Technology-Based Cognitive Interventions

Abstract

A cognitive intervention study was conducted with the purpose of exploring methods to improve adherence to a technology-based cognitive intervention and uncover individual differences that predict adherence (N = 120). The study was divided into two phases: Phase 1, in which participants were asked to follow a prescribed schedule of training that involved gamified neuropsychological tests administered via tablet, and Phase 2, in which participants were asked to play as frequently as they wished. Positive and negative framed messages about brain health were delivered via the software program, and measures of cognition, technology proficiency, self-efficacy, technology attitudes, and belief in the benefits of cognitive training were collected. Generalized linear mixed-effects models revealed that positive framed messages encouraged greater adherence over negative framed messages, but this effect was restricted to Phase 2 of the study in the absence of social pressure. Measures of memory and self-efficacy demonstrated some, but limited, ability to predict individual differences in adherence.

Successful methods for preventing or reversing the impact of age-related cognitive decline ultimately depend on two components: 1) the ability to improve the performance of important everyday tasks, and 2) the ability to maintain adherence so that these benefits are realized. While a great deal of research has focused on the first component (for recent reviews and meta-analyses, see Basak et al., 2020; Nguyen et al., 2019; Sala et al., 2019; and Simons et al., 2016), far fewer studies have examined the second issue. This is an important oversight as interventions that older adults are not willing and able to engage with over the long-term may be of little value.

One recent example of the potential benefits of long-term engagement is that cognitive speed of processing training delivered over six weeks and followed by booster sessions at 11 and 35 months was associated with reduced dementia risk across ten years (Edwards et al. 2017). However, lack of adherence, with respect to fewer sessions attended, has been found to be detrimental to the benefits of memory training (Bagwell & West, 2008) and processing speed training (Ball et al., 2013). These findings, combined with literature that cognitive training may improve everyday function (Ball et al., 2010; Smith-Ray et al., 2014) and slow functional decline (Rebok et al., 2014; Wolinksy et al., 2015), underscore the need to uncover successful adherence-promoting methods.

Adherence has been a topic of intense investigation by psychologists given how crucial it is to the success of pharmacological and behavioral interventions. Researchers and public health advocates have sought to use persuasive messages as a key strategy to increase motivation to engage in and adhere to behavioral recommendations aimed at improving health and wellbeing (Gallagher & Updegraff, 2012). The manipulation of message framing has been a popular method explored to promote health behaviors. Positive framed messages emphasize the benefits of engaging in a particular behavior, while negative framed messages emphasize the costs of failing to engage in a particular behavior. Research has demonstrated that positive framed messages are more effective than negative framed messages in shaping behaviors related to skin cancer prevention, smoking cessation, and physical activity engagement (Gallagher & Updegraff, 2012). For example, to investigate the best methods to promote adherence to a walking intervention among younger and older adults, Notthoff and Carstensen (2014) verbally presented participants with positive framed messages related to the benefits of engaging in exercise (e.g., “Walking can help prevent bone density loss, loss of coordination, and loss of balance”) or negative framed messages related to the risks associated with not exercising (e.g., “Not walking can increase bone density loss, loss of coordination, and loss of balance”). In two studies, older adults exposed to positive framed messages engaged in significantly more walking activity. Message framing did not impact younger adults in this study.

The results of Notthoff and Carstensen (2014) were interpreted as being consistent with the age-related positivity-effect (Carstensen & Mikels, 2005, Reed & Carstensen, 2012). Older adults and younger adults tend to process emotion-related information differently, with older adults often attending to and remembering positive details more than negative ones (Lockenhoff & Carstensen, 2007; Mather et al., 2005) and finding positive framed messages more motivating (Reed & Carstensen, 2012). Similarly, Shamaskin et al. (2010) found that older adults’ memory for positive statements was significantly better than negative statements within health pamphlets. Results with older adults in part can be explained by socioemotional selectivity theory (SST; Carstensen, 2006). Due to a more limited time horizon, older adults may choose to regulate emotion by giving processing priority to more positive information. This is consistent with evidence that the persuasive advantage of positive frame messages is mediated by positive affect (van’t Riet et al., 2010). Positive affect, generated by positive framed messages, can increase intention to enroll in a physical fitness program, as well as behavior (Mikels et al., 2021).

As older adults are often the target of and could benefit most from cognitive interventions, and the fact that adherence is crucial to cognitive benefits, the main focus of the current study was to investigate the potential impact of message framing on adherence to a home-based cognitive intervention. The ability to increase adherence to cognitive interventions could have multiple benefits. First, assuming effective interventions are uncovered, increased intervention adherence would benefit the individual by helping to improve their cognition and preserve their independence, which would benefit their family members as well. Across the population of older adults, prolonged functional independence would reduce the societal costs of supporting older adults experiencing impaired functional independence due to cognitive decline. Finally, within the context of cognitive intervention clinical trials, improved participant adherence would greatly benefit in the evaluation of the effectiveness of interventions, aiding in the discovery of interventions with the potential to improve cognition and preserve independence.

A secondary aim of this study was to explore what properties of the individual, in terms of attitudes, abilities, and other characteristics, are associated with adherence behaviors. This understanding might benefit adherence promotion by helping to predict who is at greatest risk for adherence lapses. Existing models of adherence provide clues as to the factors that may be most important in the context of cognitive interventions. For example, the influential Health Belief Model (HBM) posits that engagement in a behavior to prevent a health problem is affected in part by an individual’s assessment of their risk, perceived benefit of engaging in the health behavior, and perceived barriers to taking such actions (Janz & Becker, 1984; Rosenstock, 1974). The Protection Motivation Theory (Rogers, 1983), which includes many of the same factors as the Health Belief Model, also has been used to explain motivation to engage in healthy behaviors. According to Protection Motivation Theory, engagement in health behaviors is shaped by the threat appraisal process or an individual’s judgment that they are at risk and the threat is significant, and the coping appraisal process or the belief that the means of reducing risk are within the individuals’ ability to carry out. Finally, response efficacy, or belief in the effectiveness of a behavior in reducing risk, is also highlighted as promoting action. For cognitive interventions specifically, these theories emphasize the importance of measuring objective and subjective cognition (which may shape one’s assessment of risk for cognitive decline and need for intervention) and measuring beliefs in the efficacy of cognitive training (response efficacy). Coping appraisal in the current study was targeted by various measures of self-efficacy.

Other studies have explicitly linked participants’ willingness to invest time in cognitive intervention to factors such as age, belief in the efficacy of the cognitive training, previous computer use, memory performance, and self-perceived cognitive functioning (Double & Birney 2016; Harrell et. al., 2019; Turunen et al., 2019), shaping our choice of individual difference measures. Further, older adults report less technology comfort and technology self-efficacy compared to younger adults (Lee et al., 2019), and less technology proficiency (Roque & Boot, 2018). Since many cognitive interventions, including the one studied here, are delivered via technology, technology proficiency and technology self-efficacy may also be key factors in determining adherence to cognitive interventions, and thus were measured.

The present study investigated older adults’ adherence to a tablet-based cognitive intervention over an extended period (4.5 months) and potential methods that might boost intervention adherence. Participants were randomly assigned to conditions in which they were exposed to positive framed messages, negative framed messages, or no messages at all. Individual difference predictors of adherence were of secondary interest. The study was divided into two phases, one phase in which participants were instructed to maintain adherence to a training schedule, and another phase in which participants were encouraged to play as much or as little as desired. This allowed for insight to be gained into the effectiveness of the message framing manipulation with and without social pressure. It was hypothesized that: (a) Adults who received positive framed messages would have greater intervention adherence to the cognitive training compared to those who received negative framed messages or no messaging at all; (b) Individual difference predictors of self-efficacy, cognitive functioning (both objective and subjective) and technology proficiency would be positively associated with higher levels of adherence; and (c) The factors that influence adherence would potentially differ between phase one (presence of social influence/expectation) and phase two (absence of social influence/expectation). To examine these hypotheses, generalized linear mixed-effect models (GLMM) were used to examine adherence across the period of study, exploring the effect of study phase and evaluating the influence of individual difference predictors on the level of adherence. This approach allows us to: (a) make use of all available data (i.e., rather than relying on coarse aggregates at the level of a study phase); and (b) model the influence of time in study (i.e., how does adherence change over time?).

Method

Sample

One hundred twenty adults older than 64 years of age were recruited from Leon County, Florida, and the surrounding area and completed written informed consent prior to beginning the study. This sample size was substantially larger than the older adult sample of the most analogous study (Study 2) reported by Notthoff and Carstensen (2014), in which 61 older adults participated, and could detect a minimum effect size of Cohen’s d = .59, a medium-large effect, for a difference between three groups with a power of .80 (even after the loss of two participants described below). During the consent process, participants were advised that the information that they provided would be de-identified and used for research purposes including publication of data.

Two participants dropped from the study before the start of the intervention, leaving a total of 118 participants (M = 72.6 years, SD = 5.54). Phone prescreening excluded individuals with dementia or visual or auditory conditions that would prevent them from engaging in the intervention. Individuals were also excluded if they were non-English speakers or had participated in a previous study involving the same cognitive training software or if they indicated that they could not commit to a 4.5 month study (e.g., had an upcoming surgery or vacation).

Only one participant reported having less than a high school education. Across the sample, 3.3% of participants had a high school education or GED equivalent, 3.3% had vocational training, 25.2% had some college or an associate’s degree, 24.4% had a bachelor’s degree, 33.3% had a master’s degree or other post graduate training and 9.8% had advanced degrees. Of the 118 participants, all but two individuals identified their gender, with a sample of 82 women, and 38 men.

Participants received $20 after completing each respective session: Baseline Day 1, Baseline Day 2, Baseline Tablet Training Day 3, Posttesting Day 1 and Posttesting Day 2. After participants returned for follow up at 4.5 months, they received a payment of $100. If participants completed the entire study, the compensation totaled $200.

Materials and Design

Participants completed a battery of cognitive tests and questionnaires designed to assess a variety of constructs predicted to be important by models of health behaviors and technology use and adoption (Table 1). Major constructs included objective and subjective cognition, self-efficacy, attitudes toward technology, and beliefs about cognitive training.

Table 1.

Composites and Other Measures used in Analyses.

Predictor	Measures	Sources
Demographics	Background Questionnaire	Adapted from a survey developed by the Center for Research and Education on Aging and Technology Enhancement
Composite: Objective Cognition, Processing Speed	Digit Symbol Substitution	Wechsler (1997)
	Useful Field of View	Edwards et al. (2006)
Composite: Objective Cognition, Memory	Hopkins Verbal Learning Test and Hopkins Delayed Recall	Brandt & Benedict (2001)
	Rey Auditory and Delayed Rey Auditory	Schmidt (1996)
Composite: Objective Cognition, Reasoning Ability	Letter Sets	(Ekstrom et al., 1976)
	Raven’s Advanced Progressive Matrices	Raven, Raven & Court (1988) Arthur & Day (1994)
Composite: Self-efficacy	General Self-Efficacy Questionnaire	Schwarzer & Jerusalem (1995)
	Technology Self-Efficacy	adapted from the General Self-Efficacy Scale by Schwarzer & Jerusalem (1995)
Composite: Technology Proficiency	Computer Proficiency Questionnaire-12	Boot et al. (2015)
	Mobile Device Proficiency Questionnaire-16	Roque & Boot (2018)
	Technology Readiness Index 2.0	Parasuraman & Colby (2015)
Composite: Perceived Benefits	Brain Training Belief Scale	Rabipour & Davidson (2015)
	Brain Training and Independence Survey	Harrell, Kmetz, & Boot (2019)
Composite: Subjective Cognition	Instrumental Activities of Daily Living	Lawton & Brody (1969)
	Memory Self-Efficacy Questionnaire	Berry, West, & Dennehey (1989)
	Perceived Deficits Questionnaire	adapted from the Multiple Sclerosis Quality of Life Inventory Sullivan et al. (1990)

The intervention was delivered via 10-inch Lenovo tablets with 2GB of Ram and 1.5GHz Processors. All tablets utilized the Android operating system and were secured so that participants could only access the intervention software. ¹ A Wild West-themed game, the Mind Frontiers application, served as the intervention and consisted of seven mini-games (see Supplemental Material A for description of games; see also Baniqued et al., 2015). Screenshots taken from two of the mini-games within Mind Frontiers can be seen in Figure 1.

Figure 1.

Examples of Games from Mind Frontiers That Exercise Working Memory and Spatial Memory.

Participants were randomly assigned to one of three conditions (positive framed messages, negative framed messages, or no messages). Based on the condition a participant was randomized to, messages similar to those used in Notthoff & Carstensen (2014) would be displayed each time a training session was initiated (see Supplemental Material B for a listing of messages). For example, in the positive framed condition, participants received messages related to the benefits of engaging in cognitive training (e.g., “Frequent mental challenge can have a positive impact on the brain”). In the negative framed condition, participants received messages related to the risks associated with not engaging in cognitive training (e.g., “Infrequent mental challenge can have a negative impact on the brain”). For participants in the message conditions, the message would appear on the screen for 10 s to allow ample time for reading of the message. Participants were not permitted to go past the message screen until the time elapsed and the message screen advanced to the home screen where they could select the game that they would engage in for their cognitive training. While this was done to ensure that participants read the messages, ultimately it was up to the participant to attend to and read the message. Participants in the no-message group did not receive a message when they initiated a training session. As a result, they were taken directly to the home screen to begin cognitive training after logging in. Regardless of condition, each time a participant initiated a training session, training time, training activities (games played), and performance metrics were logged by the Mind Frontiers application.

Participants received a training booklet to take home that was printed in color and included: (a) Mind Frontiers-specific information (i.e., information about each game); (b) tablet-specific information (e.g., how to power on/off the device); and (c) support-specific information (i.e., phone number to contact the lab for more information). This booklet also included a copy of a schedule of recommended games to be played each of the five days training occurred and emphasized that training should last for 45 min – a detail we asked participants to track.

Sampling Procedures

After providing written informed consent, participants were assessed individually each time they visited the lab as they completed the cognitive battery and questionnaires and when they received tablet and game training. This resulted in a total of six sessions, each lasting between 1 and 2 hr. At the start of the study, participants were scheduled to come into the lab for three days in close succession (ideally only one day apart from each other). On the first two days, participants completed the assessment battery (see Table 1). The third-day participants received one-on-one training on how to use the tablet hardware and software that they would use at home. Before leaving the lab, participants were assigned a tablet with the Mind Frontiers software that had been randomly assigned to one of the intervention conditions (positive framed, negative framed, or no message) mentioned previously, provided a take-home packet, and reminded that they should engage in training for five days a week, 45 min each day. They were instructed that they should maintain this training schedule for the next 3 months.

When participants returned to the lab 3 months after completing their cognitive training at home, they were asked to complete a cognitive battery that included all the previously administered measures administered at the study onset (see Table 1). As this paper focuses on prediction, however, only pre-intervention data were used in the reported analyses. After completing two days of post-testing, participants were informed that they would transition to the second phase of the study which would last for 1.5 months. Participants were informed that they would continue to have access to the Mind Frontiers cognitive training program for an additional 6 weeks; but, they were not provided with the adherence schedule that they received during the first phase of the study. Instead participants were instructed to engage in as much or as little training as they liked for the remaining 6 weeks of the study. The motivation for dividing the intervention into two phases was to obtain estimates of adherence when there is strong external motivation to train (when a specific amount of training is expected by experimenters) and when this motivation is absent. This work was carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. The Institutional Review Board (IRB) at Florida State University approved the study protocol described in this manuscript (HSC: 2016.18062) and written informed consent form that participants signed prior to participating in the study.

Data Preparation & Analysis Plan

All data were prepared using R version 4.0.2 (R Core Team, 2020), using the ‘tidyverse’ family (Wickham et al., 2019), including ‘ggplot2 ‘for data visualization (Wickham, 2016) and ‘dplyr’ for data wrangling (Wickham et al., 2020). The presented analyses will reflect an exploratory approach suggested by reviewers using generalized linear mixed-effect modeling (GLMM), serving to complement the pre-registered results disseminated via the Open science Framework (OSF) platform https://osf.io/3n5mg/?view_only=51adb3c1175d43ea855ef452812fd821. Confirmatory and exploratory results were consistent.

Calculation of Adherence

The primary measure of adherence was days on which participants engaged in training. This was derived from log files output from the Mind Frontiers software package. All records prior to the tablet-setup appointment date were excluded. The reported analysis used a repeated measures approach. For each person and day in the study, we calculated whether a person played a game (i.e., one or more games on a given calendar date) as the binary outcome of interest for the subsequent analysis. Dichotomous outcome data (i.e., played or not – a measure of daily engagement) were modeled using generalized linear mixed-effects models (GLMM), and a binomial link function, using the `lme4` package in R with "bobyqa" optimizer, specifying a random intercept for each participant (Bates et al., 2015; R Core Team, 2020).

Constructing Composite Measures for Individual Differences

Composite measures, based on our assessment battery (see Table 1), were constructed by z-scoring and averaging together tests of a similar construct (e.g., Subjective Cognition).

Results

Testing the Manipulation

Study condition was contrast coded where the neutral condition was the reference group, the positive messaging condition group was coded as +1 and the negative messaging condition group coded as −1.

Individual Difference Predictors

To examine the individual difference predictors of adherence, adherence data, age, gender, study condition as well as composite measures from each domain (objective cognition – processing speed; objective cognition – memory; objective cognition – reasoning; subjective cognition; technology proficiency, self-efficacy, and perceived benefits) were entered into GLMMs.

GLMM – Modeling Outcomes

To evaluate the influence of variables of interest, we first fit: (a) a null model, predicting binary-coded adherence on a given day from person intercepts only (Model 1); and (b) a model predicting binary-coded adherence on a given day from day in study and person-level random intercepts, adding a fixed effect for day in study (Model 2).

Over and above Model 1, Model 2 includes an interaction between study phase² (phase 1, coded as referent group, phase 2, coded as 1) and the experimental messaging condition (positive, coded as 1; no-message, coded as referent group; negative, coded as −1). Over and above Model 3, Model 4, the full model, contains individual difference measures of interest, including demographics (age, gender), and composite scores discussed in sections prior (i.e., objective cognition – processing speed; objective cognition – memory; objective cognition – reasoning; subjective cognition; technology proficiency, self-efficacy, and perceived benefits).

GLMM – Comparing Model Fit.

Chi-square difference tests were conducted to compare the nested models described in ‘Modeling Approach’ above. Model 2 was a better fit to the data than the Model 1 (the null model; X² (1, N = 118) = 418.29, p < .001. Model 3 (plus study phase x condition interaction) was a better fit to the data than the Model 2 (X² (3, N = 118) = 24.228, p < .001.

GLMM – Results:

The fixed effects in the full model (model 4), accounted for 11.4% of the variance in daily engagement (i.e., marginal R²); fixed effects and random intercepts for each participant (i.e., random effects) accounted for 28.3% of the variance (i.e., conditional R²). Evaluating the results of the full model (model 4)³, odds of daily engagement significantly decreased with each day in the study (OR=0.99, 95% CI: 0.98, 0.99; p < .001), differed as a function of the interaction between messaging condition and study phase (OR=1.40, 95% CI: 1.20, 1.64; p < .001), and decreased as a function of the self-efficacy composite score (OR=0.59, 95% CI: 0.45, 0.77; p < .001). Probing the interaction between study phase and messaging condition further, we computed predicted probabilities – illustrating the probability of daily engagement as a function of phase and messaging condition (see Figure 2 for interaction plot).

Figure 2. Predicted Probabilities of Daily Engagement.

Note. Interaction between study phase and messaging condition.

There was no effect of messaging condition during Phase 1 (predicted probabilities from model 4: positive: 51.6%, vs negative: 46.8%; see Table 3). However, during Phase 2, those in the positive messaging condition had a higher predicted probability of adherence, as compared with the negative messaging condition (predicted probabilities: positive: 53.3%, vs negative: 32.4%).

Table 3.

Predicted Probabilities from Model 4, Extracted using the ‘Effects’ package in R (Fox & Weisberg, 2019).

	Phase 1	Phase 2
Negative	0.468	0.324
No- message	0.492	0.425
Positive	0.516	0.533

These results are consistent with the preregistered nonrepeated measures analyses reported on OSF. The key difference is that this analysis approach makes better use of available data and helps to unpack the “time in study” effect that simple data aggregation ignored in our pre-registered approach.

To further shed light on the relationships among variables, we explored correlations between adherence and individual predictors (see Table 4 for full correlation matrix; correlations adjusted for multiple comparisons using the Bonferroni correction). Not surprisingly, phase one and phase two adherence measures were strongly correlated (r = .77, p < .001). Individuals who played more during Phase 1 continued to play more in Phase 2. Although speculative, this may suggest the importance of habit formation.

Table 4.

Correlations Matrix Evaluating the Relationship Between Composite Scores and Phase One and Two Adherence.

	Age	Phase 1	Phase 2	C: Processing Speed	C: Reasoning	C: Subjective Cognition	C: Memory	C: Perceived benefits	C: Tech Proficiency	C: Self- efficacy
Age
Phase 1	0.01
Phase 2	−0.01	0.77****
C: Processing Speed	−0.28**	0.15	0.09
C: Reasoning	−0.29**	0.04	−0.06	0.37****
C: Subjective Cognition	0.05	−0.06	−0.03	−0.29**	−0.09
C: Memory	−0.39****	0.22*	0.13	0.30**	0.44****	−0.11
C: Perceived benefits	−0.06	−0.15	0.01	−0.21*	−0.30**	0.08	−0.06
C: Tech Proficiency	−0.35****	−0.12	−0.11	0.35****	0.27**	−0.15	0.16	0.08
C: Self-efficacy	−0.16	−0.29**	−0.28**	0.10	0.14	−0.27**	0.11	0.20*	0.49****

Notes: (1)

^‘****’p < .0001

^‘***’p < .001

^‘**’p < .01

^‘*’p < .05

(2) C = Composite score

(3) N = 118

Intervention Perceptions

Finally, we conducted an analysis on questions we asked participants at the end of the study regarding their experiences. Participants who completed both phases of the training were asked: (a) are there game elements or features that might have caused you to play more than you did over the past six weeks?; (b) I would have played the games more if they…? Participants were encouraged to ‘Select all that apply’. The results from these responses are tabulated in Table 5. These responses may be useful in thinking about planning future interventions that might boost adherence.

Table 5.

Game Intervention Attitudes.

Game Aspects	Endorsed Response
	%	Count
Features That Would Encourage Play
Performance Feedback	41.9	44
Difficulty Adjustment	37.1	39
Social, Play w/others	21.9	23
New Levels	21.9	23
Theme Customization	12.4	13
Played More If…
Less boring	40.0	42
Less challenging	37.1	39
More challenging	7.6	8

Note. The table headings delineate responses from two distinct questions, (a) are there game elements or features that might have caused you to play more than you did over the past six weeks?; (b) I would have played the games more if they…? Participants were encouraged to ‘Select all that apply’.

Participants indicated that they might have played more if the intervention had features that allowed them to: receive feedback on cognitive performance (41.9%), adjust the difficulty level of games (37.1%), play with others or have social interaction (21.9%), reach new levels (21.9%), and customize the game theme (12.4%). Participants also shared that they would have engaged with the intervention more if it were less boring (40.0%) and less challenging (37.1%). Contrastingly, a small proportion of the participants (7.6%), reported that they would have engaged more if they perceived the games as being more challenging.

Discussion

The primary focus of this study was to explore adherence to technology-based cognitive interventions as a health behavior similar to exercise or medication adherence. In following this approach, we based our hypotheses on previous studies (e.g., Notthoff & Carstensen, 2014; Shamaskin et al., 2010) examining health outcomes that found older adults attend to, remember, and adhere to recommendations when messages are framed positively. During Phase 1, in which participants received a set schedule to follow for their gameplay, adherence was comparable whether participants received positive, negative or no message at the beginning of their training session. During Phase 2, however, in which participants could freely engage (or avoid) cognitive training as they preferred, our hypothesis was supported; those who received the positive framed messages played significantly more in Phase 2, compared to those who received the negative framed messages. These findings suggest that in the absence of a structured schedule, positive framed messages may help support adherence to cognitive training, serving as gentle nudges to the user. However, what is clearer is that the social pressure of being asked to maintain a regular schedule may be a primary motivator in trials such as this. Once that pressure was lifted, there was relatively little engagement with the intervention. Adding a “Phase 2” to future studies, such as the one implemented here, may be useful in advancing our understanding of cognitive intervention engagement. Cognitive interventions and products often do not recommend a set schedule of engagement; as such this Phase 2 period may be more sensitive to experimental manipulation and serve as a more ecological measure of adherence.

While the primary focus of this study was the effect of message framing on adherence to cognitive training over an extended period of time, individual difference predictors of adherence were also explored. During Phase 1 (i.e., study period with a structured schedule), the significant predictors of adherence were memory and self-efficacy. Prior findings (van ’t Riet et al., 2008, 2010) provide support that self-efficacy can be an important moderator of message framing effects. However, contrary to most models of health behaviors, in the current study lower self-efficacy was associated with greater adherence. Although speculative, this pattern may be related to the domain under investigation. It is possible that individuals with low self-efficacy may feel incapable of handling day-to-day tasks, motivating them to engage in training they believe could benefit the performance of these activities.

In terms of targets to improve adherence, as discussed previously, a high proportion of participants expressed that they would have adhered more if they would have received feedback on their cognitive performance, and if they were able to modify the difficulty level of the games (difficulty level was automatically adjusted by the Mind Frontiers software based on performance). However, other participants endorsed different changes to the intervention that they predicted would have increased their gameplay (e.g., changing the Wild West theme of the game to something else). It is possible that some changes to the intervention itself could have boosted intervention adherence. Given that positive affect has been found to be a key mechanism for the effectiveness of message framing effects (Liu et al., 2019; Mikels et al., 2021), it is possible that positive affect generated by allowing more customizations could boost adherence even further.

Limitations

Simple regression analyses (see OSF) and GLMM results indicated that about 30% of the variance in adherence is explained by our modeling approach (i.e., what might explain the remaining 70% of the variance?). Although our adopted framework focused on attitudes and abilities, it may be that idiosyncratic life events play a much larger role in determining adherence. Absent from this study was a mechanism to collect systematic information about participants’ daily lives. Based on data collected during debriefing, 12% of participants shared that at some point during their involvement in the study, they struggled with adherence due to problems with their own health, the health of someone close to them, or general caregiving responsibilities (e.g., for grandchildren). Another limitation of the current approach was that messages were presented to participants through the intervention itself (on the tablet). Participants who engaged in the intervention less received less adherence support because they did not see the intended messages as often (or at all once adherence failed entirely). Participants who were adherent were exposed to the health messages frequently, though these participants were the least likely to need them. Repeated exposures may have resulted in habituation, and thus reduced message impact. An unanswered question is why our Phase 1 findings differed from the exercise adherence findings of Notthoff and Carstensen (2014). Different characteristics of their verbal presentation of messages vs. our text-based messages may have made their manipulation more effective, or messages may have differed in efficacy as a function of different intervention requirements or properties of the intervention. One final caveat: although our sample size was larger than previous analogous studies, the study was underpowered to detect small effects.

Future Directions and Recommendations

Future studies should examine the impact that caregiving and health over time have on adherence (e.g., using experience sampling techniques, ecological momentary assessments or daily diaries, Shiffman et al., 2008) and whether methods might be developed to either help individuals maintain adherence while adapting to these challenges, or reengage with the intervention once the challenge has passed.

In future studies, the incorporation of messages might be more effective if presented through a separate electronic reminder system (e.g., a SMS message sent because of a detected lapse in adherence). Given the technology-based nature of these and other interventions, the greatest promise most likely lies in electronic reminder systems that can monitor adherence, send custom messages when adherence has lapsed, and use knowledge about the individual to send the right reminder at the right time to encourage intervention success.

Table 2.

Model Results (Model 1-4).

	Daily Engagement			Daily Engagement			Daily Engagement			Daily Engagement
Predictors	Odds Ratios	CI	p	Odds Ratios	CI	p	Odds Ratios	CI	p	Odds Ratios	CI	p
(Intercept)	0.98	0.82 – 1.18	0.861	2.00	1.63 – 2.47	<0.001	1.89	1.53-2.33	<0.001	1.10	0.07 – 18.29	0.945
Day in Study				0.98	0.98 – 0.98	<0.001	0.99	0.98 – 0.99	<0.001	0.99	0.98 – 0.99	<0.001
Messaging Condition							1.14	0.90 – 1.45	0.278	1.10	0.88 – 1.37	0.389
Study Phase							0.77	0.65 – 0.91	0.002	0.76	0.64 – 0.90	0.002
Messaging Condition * Study Phase							1.37	1.17 – 1.60	<0.001	1.40	1.20 – 1.64	<0.001
Age										1.00	0.97 – 1.04	0.856
Gender										1.01	0.67 – 1.53	0.966
C: Tech Proficiency										0.88	0.65 - 1.18	0.375
C: Self-efficacy										0.59	0.45 – 0.77	<0.001
C: Subjective cognition										0.77	0.57 – 1.03	0.080
C: Perceived benefits										0.89	0.70– 1.15	0.376
C: Objective cognition, Reasoning										0.88	0.68-1.13	0.306
C: Objective cognition, Processing Speed										1.10	0.85-1.43	0.461
C: Objective cognition, Memory										1.13	0.99 – 1.30	0.081
Random Effects
σ2	3.29			3.29			3.29			3.29
τ00	0.86experiment_tag			1.06 experiment_tag			1.03 experiment_tag			0.78 experiment_tag
ICC	0.21			0.24			0.24			0.19
N	118 experiment_tag			118 experiment_tag			118 experiment_tag			116 experiment_tag
Observations	9288			9288			9288			9117
Marginal R2 / Conditional R2	0.000 / 0.208			0.054 / 0.285			0.063 / 0.286			0.114 / 0.283