Skip to main content
PMC home page
Frontiers in Psychology logoLink to Frontiers in Psychology
. 2016 Jun 7;7:834. doi: 10.3389/fpsyg.2016.00834

Combined Cognitive Training vs. Memory Strategy Training in Healthy Older Adults

Bing Li 1,2,†, Xinyi Zhu 1,3,†, Jianhua Hou 1,3,†, Tingji Chen 1,4, Pengyun Wang 1, Juan Li 1,5,*
PMCID: PMC4896109  PMID: 27375521

Abstract

As mnemonic utilization deficit in older adults associates with age-related decline in executive function, we hypothesized that memory strategy training combined with executive function training might induce larger training effect in memory and broader training effects in non-memory outcomes than pure memory training. The present study compared the effects of combined cognitive training (executive function training plus memory strategy training) to pure memory strategy training. Forty healthy older adults were randomly assigned to a combined cognitive training group or a memory strategy training group. A control group receiving no training was also included. Combined cognitive training group received 16 sessions of training (eight sessions of executive function training followed by eight sessions of memory strategy training). Memory training group received 16 sessions of memory strategy training. The results partly supported our hypothesis in that indeed improved performance on executive function was only found in combined training group, whereas memory performance increased less in combined training compared to memory strategy group. Results suggest that combined cognitive training may be less efficient than pure memory training in memory outcomes, though the influences from insufficient training time and less closeness between trained executive function and working memory could not be excluded; however it has broader training effects in non-memory outcomes. Clinical Trial Registration: www.chictr.org.cn, identifier ChiCTR-OON-16007793.

Keywords: cognitive training, memory strategy training, executive function training, older adults, mnemonic utilization

Introduction

Aging is associated with brain changes and cognitive decline that could limit older adults' functional capability (Lustig et al., 2009; Bishop et al., 2010; Mitchell et al., 2010; Talmelli et al., 2010). Encouragingly, accumulating evidence has shown that older adults reserve cognitive and brain plasticity, the capacity of relatively long-lasting neurological changes in response to experience (Lampit et al., 2015). Different formats of cognitive intervention, from standardized cognitive training (e.g., Jackson et al., 2012) to cognitive stimulation (e.g., Tesky et al., 2011), have shown beneficial effects in maintaining or even promoting cognitive functioning for both healthy older adults (for reviews see Kelly et al., 2014; Lampit et al., 2015) and elderly with cognitive impairments (for a review see Stott and Spector, 2011).

Memory deficits is one of the major manifestations of cognitive aging, representing both objective declines (Nyberg et al., 1996; Rabbitt and Lowe, 2000; Salthouse, 2004) and subjective concerns (Jonker et al., 2000; Perrig-Chiello et al., 2000). Memory training, which targets on improving episodic memory performance, has been frequently administrated in the elderly population (e.g., Derwinger et al., 2003; Cavallini and Bottiroli, 2009). Memory training enhanced memory performance by teaching memory strategies which facilitates information encoding and retrieval. Mnemonic strategies mainly include method of loci, face-name mnemonics, categorization, association, visual imagery, rehearsal, and so forth (for a review see Gross et al., 2012). The overall training effects of memory training on both objective and subjective measures of episodic memory have been convincingly proved by various studies, including large sample trials as the Advanced Cognitive Training for Independent and Vital Elderly (Ball et al., 2002). Ball et al. (2002) trained a large number of older adults (n = 711) with multiple mnemonics and found that the participants still benefited from the training in a 10-year follow-up. A recent meta-analytic study shows that memory strategy training for older adults induced a moderate training effect on overall episodic memory function (Cohen's d = 0.31) compared to control groups (Gross et al., 2012).

Nevertheless, memory strategy training has its limitation in two aspects. First, mnemonic utilization is difficult for older adults even after memory training. Several studies have found that age differences in memory performance were magnified rather than reduced after mnemonic training (Kliegl et al., 1989; Baltes and Kliegl, 1992; Nyberg et al., 2003). It has been argued that age-related executive function decline might hinder older adults from utilizing mnemonic techniques efficiently (Jones et al., 2006). For example, Nyberg et al. (2003) investigated brain activation differences between younger and older adults using the method of loci strategy. The results showed that when contrasting post- and pre-test, younger adults improved more in the trained memory task than older adults. Moreover, the younger participants showed increased activity in the left dorsolateral prefrontal cortex (DLPFC) during loci utilization compared to baseline conditions, which was not observed in older participants. Kirchhoff et al. (2014) investigated the role of prefrontal cortex in observed age differences in self-initiated strategy use. The results demonstrated that age differences in prefrontal gray matter volume significantly influence the use of self-initiated elaborative memory strategy use. Prefrontal cortex activation was reported in different forms of mnemonic processing such as association, binding, and mental image (Petrides, 2002; Bunge et al., 2004; Jha et al., 2006; Zeithamova and Preston, 2010). Taken together, these results implied that the mnemonic utilization deficit in the elderly may be due to age-related executive function decline. It is possible that conducting executive function training before memory strategy training would promote mnemonic application and lead to larger training gains in memory for older adults. The second limitation of memory strategy training is limited transfer effects (Singer et al., 2003; Carretti et al., 2007). Transfer effects in other cognitive domains were rarely reported in the mnemonic training literature with healthy older adults. Only a few studies reported improvement in other cognitive functions beyond episodic memory, such as in processing speed (Singer et al., 2003) or working memory (Carretti et al., 2007). Compared to memory strategy training, processed-based cognitive training, such as executive function training and working memory training, has relatively larger transfer effects (Zelinski, 2009). Recent meta-analyses and systematic reviews have shown that executive function induced moderate to medium training effects on overall cognition and moderate transfer effects (Karbach and Verhaeghen, 2014). Transfer effects have been reported in training studies on different types of executive functions (for a review see Strobach et al., 2014). In specific, task switching training may lead to improvements in untrained Stroop task and working memory tasks (Karbach and Kray, 2009; Kray et al., 2012) and updating tasks (Zinke et al., 2014). Updating training may induce near transfer to structurally similar working memory updating tasks (Dahlin et al., 2008; Li et al., 2008) and far transfer to task switching (Salminen et al., 2012) and fluid intelligence (Jaeggi et al., 2008).

Multi-component cognitive trainings include complex cognitive process yielding broader effect on multiple cognitive domains than single domain training (Cheng et al., 2012; Walton et al., 2015). However, to our knowledge no study has experimentally explored the comparative effect between executive function plus memory training and pure memory strategy training. We aimed at combining memory strategy training with executive function training to compose a combined training regimen, which was supposed to facilitate the efficiency of memory strategy training. Because mnemonic utilization deficit in older adults related to executive function decline, executive function training was arranged before mnemonic training in this combined training regimen. We assume that the preceding executive function training would improve executive function, and the hypothetically improved executive function would facilitate the effective utilization of memory strategy in older adults, thus resulting in larger gains from subsequent memory strategy training. In this study, we compared combined training to pure memory strategy training. Although the duration of memory strategy training component in combined training group was only half of that in pure memory training group, we expected similar or even larger improvements in memory in the combined group because we assumed better strategy utilization after executive function training in the combined group. In addition, we also predicted broader effects in the combined group, for executive function training would improve performance on both trained and untrained executive function tasks. Specifically, as executive function training included updating and task switching training, we predicted combined group would show improvements in trained updating and switching tasks, as well as in untrained tasks including the Stroop Test and the Trail Making Test. To summarize, we hypothesized that in a fixed training interval this combined cognitive training regimen would be more efficient than pure memory strategy training. That is, it would induce (1) comparable or even larger improvements in memory, and (2) increased performance in executive function and other cognitive domains.

Methods

Participants

The participants were recruited by advertisements on the community bulletin and flyers in the community service station. A group of 68 older adults from the local community were voluntarily contacted in April, 2010. We excluded 28 older adults who (1) scored less than 24 on Mini-Mental State Examination (MMSE) (n = 9); (2) scored more than 16 on Center for Epidemiological Studies Depression Scale (CES-D) score (n = 2); (3) had visual or hearing impairments which would hinder from training (n = 8); (4) could not guarantee a full participation because of scheduling conflicts (n = 9). Thus, 40 participants (mean age = 68.3, SD = 4.4, range 60–76) were randomly assigned to a memory strategy training group (memory group) or a combined cognitive training group (combined group). One participant in memory group who dropped out before post-test was excluded from analysis. Another group of 18 participants were recruited as a passive control group under the same inclusion criterion in October, 2011. They received no training but only the pre- and post-test. At post-test, four participants dropped out (two participants withdrew from the study, and two could not be contacted). Thus, fourteen participants in the control group were included in analysis. At follow-up, two participants in the memory group and six in combined group dropped out. As seen in Table 1, the three groups of participants did not differ significantly (p > 0.05) in age, gender, years of education, global cognition (MMSE), and depression (CES-D). Participants gave informed consent and were paid 200 Yuan (about 30 dollars) for participation. Figure 1 shows the flow of participants.

Table 1.

Demographic and cognitive characteristics.

Memory (n = 19) M (SD) Combined (n = 20) M (SD) Control M (SD) P-value (n = 14)
Age (years) 67.8 (4.3) 68.7 (4.6) 67.6 (4.2) 0.76
Education (years) 13.2 (2.8) 12.8 (3.3) 13.3 (2.7) 0.84
Female/Male 15/4 11/9 10/4 0.26
MMSE 28.3 (1.1) 28.0 (1.5) 28.5 (1.3) 0.49
CES-D 6.1 (5.4) 7.6 (9.0) 5.0 (5.8) 0.58
Open in a new tab

MMSE, Mini Mental State Examination; CES-D, Center for Epidemiological Studies Depression Scale.

Figure 1.

Figure 1

Open in a new tab

Flow of participants.

Procedure

All three groups of participants received cognitive assessments before and after training. Memory group and combined group were also assessed at 4 month after completion of training. Follow-up data of control group was not collected. Cognitive assessments at each time were administrated in one session by assessors who were blind to training assignment. The assessment session lasted about 90–120 min. The pre-training assessments were conducted within 2 weeks before training, and the post-training assessments were completed within a week after the final session of training.

After baseline test, participants in the memory group and combined group received 16 sessions of training during about 6 weeks. Training was held three times per week. Each session lasted for ~60 min (50 min for training and 10 min for break). Thus, each training group received 16 h of training in total. Frequency, time and format of training were matched in two groups. In memory group, participants received 16 sessions of memory strategy training, including training for method of loci (460 min in total) and face-name mnemonic (340 min in total). In combined group, participants firstly received eight sessions of executive function training (200 min for updating and 200 min for switching) and then eight sessions of memory strategy training (260 min for method of loci and 140 min for face-name mnemonic). Executive function training was arranged before memory strategy training because we assumed that enhanced executive function induced by preceding executive function training would facilitate the effective utilization of memory strategy, making the subsequent memory strategy training more efficient. The protocol of this study was approved by the Ethics Committee of the Institute of Psychology, Chinese Academy of Sciences. The study was registered in the Chinese Clinical Trial Registry (www.chictr.org.cn) ChiCTR-OON-16007793.

Memory strategy training

Method of loci

Participants practiced method of loci (Bower, 1970) in the Wordlist task. They were instructed to establish a well-known route with several landmarks and to associate words with those landmarks serially on a mental map. At recall, participants were instructed to mentally revisit the ordered landmarks to retrieve the words. In this study, the Wordlist task words were names of common objects (e.g., animals, fruits) and were read aloud to participants. The method of loci practice started from eight-word lists, and then gradually increased to 10-, 12-, 14-, and 16-word lists. During the final 9–16 sessions, participants practiced only 16-word lists.

Face-name mnemonic

The face-name mnemonic (Yesavage, 1983) was used in the Face-name task. In this task, black-and-white photos of males or females with two-character Chinese names were visually presented to participants on a computer. They were instructed to identify a prominent facial feature, creating a visual association with the name, and then mentally connect the association with the prominent facial feature. At recall, participants needed to identify the prominent feature first, retrieve the related mental image, and then recall the name. For the first eight memory strategy training sessions, participants progressively practiced the face-name mnemonic on one-, two-face, and three-face tasks. During sessions 9–16, participants kept practicing on five- and seven-face tasks.

Executive function training

Updating training

The trained updating task was the keep-track task (Yntema and Mueser, 1962). In this study, the keep-track task was divided into Word-updating and Picture-updating tasks. Both were involved equally in training sessions. In this task, trials of colored words and pictures from various semantic categories (e.g., animals, clothes, vegetables, and fruits) were presented serially at 2000 ms/per item with an interval of 500 ms in random order. Participants were instructed to continuously update the items of targeted categories indicated by boxes at the bottom of the screen and verbally report the last item of each targeted category at the end of the trial. Each trial included two distracting items beyond the targeted categories. Each task included 12 trials of words or pictures. Difficulty was manipulated by varying the number of categories presented (two or three) and the number of items in each category (two, three, or four). Participants started with the easiest task (two categories with two items per category). When easier tasks were finished, they were instructed to conduct the more difficult tasks along with training sessions.

Switching training

We used the task-switching paradigm (Kray and Lindenberger, 2000) modified to include mixed-task blocks only (two- and three-task switching). In this task, participants were required to switch subtasks (two subtasks or three subtasks) on pseudorandom trials according to cues presented in two boxes at the bottom of the screen in each trial. Two-task switching training consisted of two types of training materials performed equally in the training phase—“food” and “poker.” For the “food” task, subtask A required participants to decide whether a picture showed a fruit or vegetable (fruit/vegetable), and subtask B whether a picture was on the left or the right of the screen (left/right). For the “poker” task, participants were required to decide whether a poker card depicted was red or black (red/black), or whether the number on the poker card was even or odd (even/odd). There were also two sorts of training materials performed equally in three-task switching training —“face” (male/female, elders/youngsters, and white/yellow) and “poker” (red/black, even/odd, and < 5/>5). The same two response keys were used for all tasks. Difficulty was manipulated by varying the number of subtasks (two and three) and the number of items in each subtask (8, 10, and 12). The number of trials in each task varied according task difficulty, ranging from 16 trials (the easiest task) to 36 trials (the hardest task). Participants started with the easiest task (two subtasks with eight items per subtask) and continued to more difficult ones along with training sessions.

Outcome measures

Trained memory tasks

For the Wordlist task and the Face-name task, two versions were used at pre- and post-test. These versions were assessed by a group of 15 healthy elderly adults who did not take part in our training, and the results of free verbal recall within each version did not differ significantly from each other. In follow-up, we used the pre-test version.

Wordlist task

An audio-taped list of 16 two-character words was presented to participants at a rate of 6 s per word. At the end of the presentation participants were required to verbally recall as many words on the list as possible with no order constraint. The words were from four semantic categories (fruits, vegetables, animals, clothes) with four words per category and were chosen from the Directed Memory Test (from the Clinical Memory Scale; Xu and Wu, 1986). The dependent measure was the number of correctly recalled words (maximum score = 16).

Face-name task

Participants were presented with 12 black-and-white photographs of faces balanced for sex and paired with two-character Chinese names at the bottom of the screen. Each photograph was shown on the computer for 30 s. At recall, these faces were presented in a different order, and participants were asked to verbally report the names previously paired with those faces. The dependent measure was the number of correctly recalled names (maximum score = 12).

Non-trained memory tasks

Associative learning test (ALT, from the clinical memory scale, Xu and Wu 1986)

Participants were required to study an audio-taped list of 12 pairs of nouns for 2 s per pair. Half of the word pairs were not associated (e.g., teacher-railway) and half were semantically associated (e.g., sun-moon). The list was presented three times with different orders. After each study phase, participants were asked to recall the second noun when given the first noun as a cue after each trial. The cues were not presented in the same order as during the training phase. The number of correctly recalled nouns was the dependent measure (0.5 point per pair for semantically-associated pairs, one point per pair for non-associated pairs, maximum = 27).

Logical memory test (LMT, from the wechsler memory scale-revised in China; Gong, 1989)

Participants were presented with two audio-taped stories and were asked to verbally recall a story immediately after its presentation. The dependent measure was the mean score of the number of correctly recalled episodes (50 in total) of two stories (maximum score = 25).

Multifactorial memory questionnaire (MMQ, Troyer and Rich, 2002)

This questionnaire was used to assess separate dimensions of subjective memory, including contentment, ability and strategy. It contains 57 items addressing a variety of subjective perception of participants' current memory ability. In this study, only the MMQ-Ability Scale (20 items) was used to assess the memory failures in everyday memory situations, such as remembering names and appointments. Participants reported the frequency of each memory failure on a five-point scale (never, rarely, sometimes, often, all the time). Answers on each item were recoded to zero to four points based on the reported frequency. The sum score indexed subjective memory, with higher scores indicating more memory failures.

Trained executive function tasks

Word/picture updating task

Word- and Picture-updating tasks were structurally similar to the trained updating tasks. There were six two-category trials (half three items/category, half four items/category) and six three-category trials (half two items/category, half three items/category) in this task with two distracting items in each trial. The dependent measure was the number of blocks where the last presented items of each category were correctly recalled (maximum score = 12).

Switching task

Switching task was structurally similar to the trained “poker” switching task. This task was three-task switching with 24 items per subtask. Switching costs, the reaction time difference between non-switch and switch items, were measured as dependent variables. Each trial began with fixation-cross presentation (2000 ms) and did not advance until participant response. Participants were instructed to respond as quickly and accurately as possible.

Non-trained executive function tasks

Trail making test (B–A)

The Trail Making Test (TMT, Reitan, 1955) included two parts. Part A is a neuropsychological test of processing speed and part B is a test of switching. Performance was indexed by the difference value between the reaction time on TMT-B and time on TMT-A (TMT B–A). Larger values in TMT (B–A) indicated poorer performance.

Stroop test (Stroop, 1935)

Participants were asked to name the color of dots/words/color words in three cards. Performance was indexed by the time difference between the card of color words and the card of dots. Larger values indicated poorer performance.

Backward digit span task (from the wechsler adults intelligence scale-revised in China; Gong, 1992)

Participants were asked to reversely repeat lists of digits after each auditory presentation. The digit span was the length of the longest list a participant could repeat.

Processing speed, short-term memory, and language tasks

Performance on TMT-A was used to measure processing speed. The Forward Digit Span Task (from the Wechsler Adults Intelligence Scale-Revised in China) was adopted to measure short-term memory. The Verbal Fluency Test (González et al., 2005) was used to measure language abilities. Participants were asked to verbally report as many animals or foods as possible to assess category fluency with this test.

Analysis

First, one-way analyses of variance (ANOVA) were conducted to compare the baseline performance on cognitive tasks in three groups. Significant differences (p < 0.05) were found on the face-name task, ALT, word updating task and backward digit span, and marginally significant difference (p < 0.1) was found on the picture updating task. Thus, to control the discrepant baseline performance, analyses of covariance (ANCOVA) of post-training-minus-baseline change scores were used to compare training effects among three groups, using baseline scores on these five tasks as covariates and group as a factor. ANCOVA was performed for each outcome measure, including four tasks of objective memory, MMQ, and five tasks of executive function. A composite score of objective memory measures was calculated to index the overall memory performance. Raw scores on four trained and untrained memory tasks (the wordlist task, face-name task, ALT, and LMT) were standardized into z scores, and then summed and converted (equally weighted) into a composite t-score (mean = 50, standard deviation = 10). Similarly, a composite score of executive function was computed, combining performance on word, and picture updating tasks, the Stroop Test, TMT and backward digit span. In addition, effect sizes (Cohen's d; Cohen, 1988) for repeated measures (baseline vs. post-training) of all outcome measures and composite scores within each group were also calculated. Baseline data of switching task from eight participants in the control group was lost due to technique problem. Because of high missing rate of data, switching task was not included in the analysis.

Although follow-up data was not available for control group, maintenance effects were analyzed based on data from two training groups. As no significant difference was found between two training groups at baseline, 2 (group: combined group vs. memory group) × 2 (time: baseline vs. follow-up) ANOVAs were used to compare changes from baseline to 4-month follow-up in two training groups.

Results

Training effects on memory

Baseline cognitive performance, estimated means of change scores, effect sizes (Cohen's d) and significance were shown in Table 2. The ANCOVA results showed significant effects of group on the measure of face-name task and composite score of memory (Figure 2). Further analysis showed that, compared to control group, memory group demonstrated significant improvement on the face-name task (p < 0.05) while combined group did not (p > 0.05). For composite score of memory, significant training effects was found in memory group (memory vs. control, p < 0.01), and marginally significant effect was found in combined group (combined vs. control, p = 0.05). The effect size analyses revealed memory group had larger effect sizes on all memory measures than combined group (see Table 2).

Table 2.

Post-training effects of training in three groups.

Memory (n = 19) Combined (n = 20) Control (n = 14) Pb Partial eta squared
Baseline mean (SD) Change mean (95% CI) Effect size Baseline mean (SD) Change mean (95% CI) Effect size Baseline mean (SD) Change mean (95% CI) Effect size
MEMORY
Memory (composite score) 50.71 (7.46) 9.66 (6.40–12.91) 1.189 47.35 (6.84) 7.92 (4.62–11.22) 0.833 52.82 (5.32) 2.26 (−1.89–6.41) 0.486 0.030 0.15
TRAINED MEMORY TASKS
Face-name 3.5 (1.75) 1.12 (0.35–1.90) 0.504 2.70 (2.04) 0.12 (−0.67–0.90) 0.016 4.54 (2.10) −0.54 (−1.53–0.44) −0.16 0.030 0.14
Wordlist 9.89 (2.49) 2.32 (0.72–3.92) 0.972 9.00 (2.22) 2.09 (0.47–3.71) 0.811 9.21 (1.89) 0.23 (−1.81–2.26) 0.219 0.271 0.06
NON-TRAINED MEMORY TASKS
ALT 10.29 (3.73) 4.20 (2.76–5.64) 1.179 10.08 (2.17) 3.95 (2.50–5.40) 1.164 12.5 (2.69) 2.20 (0.37–4.03) 0.764 0.233 0.06
LMT 8.79 (2.75) 2.24 (1.09–3.40) 0.864 7.60 (2.39) 2.20 (1.02–3.37) 0.672 8.61 (2.04) 0.86 (−0.62–2.33) 0.44 0.316 0.05
MMQ 16.89 (10.73) −2.75 (−6.40–0.90) 0.275 20.25 (10.01) 0.70 (−3.07–4.46) −0.049 16.38 (7.65) 0.57 (−4.33–5.47) −0.073 0.349 0.05
EXECUTIVE FUNCTION
EF (composite score) 51.33 (5.82) 3.69 (1.86–5.53) 0.653 46.12 (7.83) 8.19 (6.33–10.04) 1.22 53.74 (7.62) 4.91 (2.57–7.25) 0.52 0.005 0.21
TRAINED EF TASKS
Word updating 5.37 (1.71) 1.53 (0.83–2.24) 0.971 3.80 (2.35) 3.24 (2.52–3.95) 1.659 6.07 (2.53) 1.86 (0.96–2.76) 0.523 0.005 0.21
Picture updating 7.58 (1.74) 0.82 (0.21–1.44) 0.338 6.05 (2.24) 2.70 (2.08–3.33) 1.449 7.21 (2.12) 1.52 (0.74–2.30) 0.787 <0.001 0.29
NON-TRAINED EF TAKS
TMT (B–A)a 25.89 (17.08) −5.76 (−12.28–0.77) 0.74 29.80 (18.65) −5.84 (−12.44–0.77) 0.869 23.50 (13.70) −2.14 (−10.45–6.18) 0.352 0.766 0.01
Stroop testa 12.89 (6.23) −1.30 (−6.14–3.53) 0.143 10.20 (12.69) 2.37 (−2.52–7.26) 0.053 17.5 (8.42) −5.97 (−12.13–0.19) 0.529 0.150 0.08
Digital span backward 4.9 (1.24) 0.13 (−0.34–0.60) 0.172 4.25 (1.16) 0.37 (−0.10–0.85) 0.376 5.86 (1.41) 0.43 (−0.16–1.03) 0.113 0.661 0.02
OTHERS COGNITIVE TASKS
Digital span forward 7.89 (1.15) −0.20 (−0.48–0.88) 0.228 7.50 (1.15) −0.09 (−0.79–0.60) −0.077 7.93 (1.33) 0.86 (−0.01–1.73) 0.658 0.275 0.06
TMT-Aa 32.53 (6.11) −5.74 (−9.53–1.95) 1.083 36.70 (10.27) −11.45 (−15.29–7.62) 1.304 34.21 (10.67) −5.85 (−10.68–1.01) 0.477 0.087 0.10
Verbal fluency 48.05 (8.35) 6.51 (1.89–11.12) 0.834 45.05 (9.24) 5.78 (1.11–10.46) 0.643 50.36 (12.28) 3.91 (−1.98–9.79) 0.225 0.793 0.01
Open in a new tab

SD, standard deviation; CI, confidence interval; ALT, Associative Learning Test; LMT, Logical Memory Test; EF, executive function; TMT, Trail Making Test; MMQ, Multifactorial Memory Questionnaire.

a

Lower scores indicate better performance.

b

Analyses of covariance of posttraining-minus-baseline change scores, controlled for baseline performance on the face-name task, ALT, word updating task and backward digit span, and picture updating task.

Figure 2.

Figure 2

Open in a new tab

Estimated means of change scores with 95% confidence interval on memory (left) and executive function tasks (right). Memory tasks: (A) Composite score of memory; (B) Face-name task; (C) Wordlist task; (D) Associative Learning Test; (E) Logical Memory Test. Higher scores indicates greater improvements. Executive function tasks: (A) Composite score of executive function; (B) Word updating task; (C) Picture updating task; (D) Trail Making Test (B–A); (E) Trail Making Test-A. In (A), (B), and (C), higher scores indicate greater improvements. In (D) and (E), lower scores indicate greater improvements.

Training effects on executive function and other cognitive abilities

For executive function, the ANCOVA results indicated significant effects of group on the measure of word and picture updating tasks (Figure 2). Significant improvements were found in combined group (combined vs. control, p < 0.01, for word updating task; p < 0.001, for picture updating task) but not memory group (memory vs. control, p > 0.05, for word updating task; p > 0.05, for picture updating task). Improvements on the composite score of executive function also showed significant group differences. Further analysis indicated significant improvement in combined group (combined vs. control, p < 0.05) but not memory group (memory vs. control, p > 0.05).

The ANCOVA results also revealed a marginal significant effect of group on TMT-A (Figure 2). The combined group showed a trend of improvement on TMT-A (combined vs. control, p = 0.09) while memory group did not.

Maintenance effects

Results of maintenance effects based on data from two training groups were shown in Table 3. The ANOVAs showed no significant group × time interactions on all outcome measures. The main effects of time were significant on measures of face-name task, wordlist task, ALT, LMT, word and picture updating tasks, TMT-A, and verbal fluency, indicating improved performance at 4-month follow-up in both training groups.

Table 3.

Cognitive performance of two training groups at the 4-month follow-up.

Memory (n = 17) Combined (n = 14) P-valueb Partial eta squared
Basline Follow-up Basline Follow-up
Mean SD Mean SD Mean SD Mean SD
TRAINED MEMORY TASKS
Face-name 3.65 1.79 6.76 2.62 2.75 1.92 4.89 2.63 0.20 0.06
Wordlist 10.06 2.59 12.59 2.79 9.14 1.56 10.43 2.87 0.17 0.06
NON-TRAINED MEMORY TASKS
ALT 10.59 3.79 15.62 4.65 10.11 2.47 14.39 3.13 0.59 0.01
LMT 9.00 2.55 10.38 2.97 8.14 2.20 9.14 2.67 0.71 0.01
MMQ 15.24 9.66 15.00 8.14 20.79 10.89 24.00 11.5 0.22 0.05
TRAINED EF TASKS
Word updating 5.47 1.70 7.47 1.94 3.57 2.21 7.00 2.83 0.11 0.09
Picture updating 7.65 1.80 8.47 1.66 6.07 1.90 8.43 2.31 0.07 0.11
NON-TRAINED EF TAKS
TMT (B-A)a 27.00 17.51 21.12 15.50 29.07 20.44 27.36 18.30 0.52 0.01
Stroop testa 12.00 5.51 13.76 7.66 12.14 13.83 10.64 7.74 0.34 0.03
Digital span backward 5.00 1.28 5.53 1.74 4.14 1.10 4.43 1.83 0.62 0.01
OTHERS COGNITIVE TASKS
Digital span forward 7.76 1.09 7.71 1.05 7.50 1.09 7.43 1.34 0.98 0.00
TMT-Aa 32.47 6.48 26.29 8.56 36.43 8.43 27.36 6.27 0.37 0.03
Verbal fluency 49.00 8.30 53.12 11.29 44.93 8.68 48.93 9.26 0.97 0.00
Open in a new tab

SD, standard deviation; CI, confidence interval; ALT: Associative Learning Test; LMT: Logical Memory Test; EF, executive function; TMT: Trail Making Test; MMQ, Multifactorial Memory Questionnaire.

a

Lower scores indicate better performance.

b

P-values for 2 (combined group vs. memory group) × 2 (baseline vs. follow-up) ANOVAs.

Discussion

The present study compared the effectiveness between combined cognitive training (executive training plus memory training) and memory strategy training. In respect of composite score of memory outcomes, memory group exerted significant improvements than control group while combined group exerted marginally significant enhancement. In respect of non-memory outcomes, improvements on executive function and processing speed were only found in the combined training group but not memory training group.

We hypothesized that combined cognitive training group would exert comparable or even larger improvements in memory than memory training group. Compared with control group, significant training effect was detected in memory tasks and marginally significant training effect was found in combined cognitive training group. Contrary to our expectation, combined cognitive training did not even provide comparable training gains in memory function than memory strategy training alone; rather, the gains were smaller. As our results demonstrate, this objective is not achieved. One possible explanation for this is that executive function training time may be insufficient to effectively improve mnemonic use. Executive function training is based on extensive practice, and its mechanisms leading to training effects are associated with continuous performance of the trained abilities, so training time affects the practice-based processing changes (Zelinski, 2009). According to previous studies, Dahlin et al. (2008) trained participants for 11.25 h on updating tasks and found episodic memory enhancement exclusively in younger adults; Buschkuehl et al. (2008) trained participants for 17.25 h on similar updating tasks and observed an increase in episodic memory in old-old adults. In our study, there were only 4 h of updating task training and 4 h of switching task training. In addition, we trained subjects on executive function instead of directly on working memory, while previous neuroimaging studies showed that working memory contributed to memory organizational skills (D'Esposito et al., 1999; Petrides, 2002; Blumenfeld and Ranganath, 2006), and we did not detect transfer effect on working memory. Further, because memory training time in combined cognitive training group was only half of that in memory group, intervention gain could possibly be more difficult to detect.

Consistent with our expectation, combined cognitive training group demonstrated broader effect in executive function compared with memory training group alone. The improvements in executive function as revealed by Word- and Picture-updating task were in line with previous findings that have reported considerable gains in trained updating or switching tasks (Dahlin et al., 2008; Minear and Shah, 2008; Karbach and Kray, 2009; Borella et al., 2010). For the processing speed improvement, transfer effect was found in Trail Making Test-A which is typical processing speed test in combined cognitive training group. The transfer to processing speed was reported in Singer's et al. (2003) study, which used a processing speed task, the Digit Symbol Test, different from the Trail Making Test-A used in this study. In short, analyses in this study provided more evidence for the exciting idea that combined cognitive training may be capable of improving multi-domain cognitive functions.

Several limitations of the present study should be noted. First, baseline performances were not matched on face-name task, ALT, word and picture updating tasks, and backward digit span. This may due to non-randomization of three groups. Two training groups were randomized, while data of control group was collected a year after the completion of two training regimens. Although baseline performance on these five tasks was treated as covariate in the analyses, we cannot exclude the confounding effects of unmatched baseline performance. Second, the sample size of our study was rather small, leading to limit statistical power for testing hypothesis (Shieh, 2014). Third, follow-up data of the control group was not collected, so whether training gains maintained over time could not be clearly investigated. The results revealed that both training groups showed improved performance on trained and untrained memory tasks, trained EF tasks, and tests of processing speed and language at follow-up compared to baseline. However, the results must be cautiously interpreted because both maintenance effect and practice effect could contribute to performance at follow-up. Nonetheless, this study can contribute to the gerontological literature in that it experimentally analyzed how different components of combined cognitive training interact with each other.

In sum, our main outcomes contribute to the intervention literature in three ways. First, combined cognitive training provided smaller improvement in memory, but broader effects on other cognitive functions. Therefore, it would be a wise choice for combined cognitive intervention in clinical therapy for patients with multi-domain deficit. Second, memory strategy training uniquely improved memory with regard to immediate training effects. For future research, especially for research in clinical therapy for memory deficient patients (e.g., MCI), it is necessary to further investigate whether memory strategy training would provide better training effects to memory-impaired older adults compared to other combined cognitive training regimens. Third, we experimentally explored how executive function training may influence memory training. Given several limitations in this study, further investigations are needed to test how to effectively arrange different components in interventional studies to meet different purposes.

Author contributions

BL and JL initiated and designed the study. BL and PW developed the detailed protocol. BL administrated the training program. XZ, TC, and PW contributed to data collection. BL and XZ analyzed the data. BL, XZ, JH, and JL wrote the paper. JL supervised the study.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (31070916, 31271108, and 31470998), the CAS/SAFEA International Partnership Program for Creative Research Team (Y2CX131003), National Science and Technology Pillar Program of China (2009BAI77B03), and the Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences (KLMH2014ZG03).

References

  1. Ball K., Berch D. B., Helmers K. F., Jobe J. B., Leveck M. D., Marsiske M., et al. (2002). Effects of cognitive training interventions with older adults - a randomized controlled trial. JAMA 288, 2271–2281. 10.1001/jama.288.18.2271 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Baltes P. B., Kliegl R. (1992). Further testing of limits of cognitive plasticity: negative age differences in a mnemonic skill are robust. Dev. Psychol. 28, 121–125. 10.1037/0012-1649.28.1.121 [DOI] [Google Scholar]
  3. Bishop N. A., Lu T., Yankner B. A. (2010). Neural mechanisms of ageing and cognitive decline. Nature 464, 529–535. 10.1038/nature08983 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Blumenfeld R. S., Ranganath C. (2006). Dorsolateral prefrontal cortex promotes long-term memory formation through its role in working memory organization. J. Neurosci. 26, 916–925. 10.1523/JNEUROSCI.2353-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Borella E., Carretti B., Riboldi F., De Beni R. (2010). Working memory training in older adults evidence of transfer and maintenance effects. Psychol. Aging 25, 767–778. 10.1037/a0020683 [DOI] [PubMed] [Google Scholar]
  6. Bower G. H. (1970). Analysis of a mnemonic device. Am. Sci. 58, 496–510. [Google Scholar]
  7. Bunge S. A., Burrows B., Wagner A. D. (2004). Prefrontal and hippocampal contributions to visual associative recognition: interactions between cognitive control and episodic retrieval. Brain Cogn. 56, 141–152. 10.1016/j.bandc.2003.08.001 [DOI] [PubMed] [Google Scholar]
  8. Buschkuehl M., Jaeggi S. M., Hutchison S., Perrig-Chiello P., Däpp C., Muller M., et al. (2008). Impact of working memory training on memory performance in old-old adults. Psychol. Aging 23, 743–753. 10.1037/a0014342 [DOI] [PubMed] [Google Scholar]
  9. Carretti B., Borella E., De Beni R. (2007). Does strategic memory training improve the working memory performance of younger and older adults? Exp. Psychol. 54, 311–320. 10.1027/1618-3169.54.4.311 [DOI] [PubMed] [Google Scholar]
  10. Cavallini E., Bottiroli S. (2009). Can computer familiarity regulate the benefits of computer-based memory training in normal aging? a study with an italian sample of older adults. Aging Neuropsychol. Cogn. 16, 401–418. 10.1080/13825580802691763 [DOI] [PubMed] [Google Scholar]
  11. Cheng Y., Wu W., Feng W., Wang J., Chen Y., Shen Y., et al. (2012). The effects of multi-domain versus single-domain cognitive training in non-demented older people: a randomized controlled trial. BMC Med. 10:30. 10.1186/1741-7015-10-30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Hillsdale, NJ: Lawrence Earlbaum Associates. [Google Scholar]
  13. Dahlin E., Nyberg L., Bäckman L., Neely A. S. (2008). Plasticity of executive functioning in young and older adults: immediate training gains, transfer, and long-term maintenance. Psychol. Aging 23, 720–730. 10.1037/a0014296 [DOI] [PubMed] [Google Scholar]
  14. Derwinger A., Neely A. S., Persson R. D., Hill M., Bäckman L. (2003). Remembering numbers in old age: mnemonic training versus self-generated strategy training. Aging Neuropsychol. Cogn. 10, 202–214. 10.1076/anec.10.3.202.16452 [DOI] [Google Scholar]
  15. D'Esposito M., Postle B. R., Ballard D., Lease J. (1999). Maintenance versus manipulation of information held in working memory: an event-related fMRI study. Brain Cogn. 41, 66–86. 10.1006/brcg.1999.1096 [DOI] [PubMed] [Google Scholar]
  16. Gong Y. (1989). Wechsler Memory Scale – Revised in China. Changsha: Hunan Medical University Publishing. [Google Scholar]
  17. Gong Y. (1992). Manual of Wechsler Adult Intelligence Scale–Chinese Version. Changsha: Chinese Map. [Google Scholar]
  18. González H. M., Mungas D., Haan M. N. (2005). A semantic verbal fluency test for English- and Spanish-speaking older Mexican-Americans. Arch. Clin. Neuropsychol. 20, 199–208. 10.1016/j.acn.2004.06.001 [DOI] [PubMed] [Google Scholar]
  19. Gross A., Parisi J., Kueider A., Spira A. P., Samus Q., Ko J., et al. (2012). Memory training interventions for older adults: a meta-analysis. Aging Ment. Health 16, 722–734. 10.1080/13607863.2012.667783 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Jackson J. J., Hill P. L., Payne B. R., Roberts B. W., Stine-Morrow E. A. L. (2012). Can an old dog learn (and want to experience) new tricks? cognitive training increases openness to experience in older adults. Psychol. Aging 27, 286–292. 10.1037/a0025918 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Jaeggi S. M., Buschkuehl M., Jonides J., Perrig W. J. (2008). Improving fluid intelligence with training on working memory. Proc. Natl. Acad. Sci. U.S.A. 105, 6829–2833. 10.1073/pnas.0801268105 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Jha A. P., Ranucci M. B., Giuliani N. R. (2006). Organization of mnemonic and response operations within prefrontal cortex. Brain Res. 1097, 133–141. 10.1016/j.brainres.2006.02.039 [DOI] [PubMed] [Google Scholar]
  23. Jones S., Lars N. B., Sandblom J., Neely A. S., Ingvar M., Petersson K. M., et al. (2006). Cognitive and neural plasticity in aging: general and task-specific limitations. Neurosci. Biobehav. Rev. 30, 864–871. 10.1016/j.neubiorev.2006.06.012 [DOI] [PubMed] [Google Scholar]
  24. Jonker C., Geerlings M. I., Schmand B. (2000). Are memory complaints predictive for dementia? a review of clinical and population-based studies. Int. J. Geriatr. Psychiatry 15, 983–991. [DOI] [PubMed] [Google Scholar]
  25. Karbach J., Kray J. (2009). How useful is executive control training? Age differences in near and far transfer of task-switching training. Dev. Sci. 12, 978–990. 10.1111/j.1467-7687.2009.00846.x [DOI] [PubMed] [Google Scholar]
  26. Karbach J., Verhaeghen P. (2014). Making working memory work: a meta-analysis of executive-control and working memory training in older adults. Psychol. Sci. 25, 2027–2037. 10.1177/0956797614548725 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kelly M. E., Loughrey D., Lawlor B. A., Robertson I. H., Walsh C., Brennan S. (2014). The impact of cognitive training and mental stimulation on cognitive and everyday functioning of healthy older adults: a systematic review and meta-analysis. Ageing Res. Rev. 15, 28–43. 10.1016/j.arr.2014.02.004 [DOI] [PubMed] [Google Scholar]
  28. Kirchhoff B. A., Gordon B. A., Head D. (2014). Prefrontal gray matter volume mediates age effects on memory strategies. Neuroimage 90, 326–334. 10.1016/j.neuroimage.2013.12.052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kliegl R., Smith J., Baltes P. B. (1989). Testing-the-limits and the study of adult age differences in cognitive plasticity of a mnemonic skill. Dev. Psychol. 25, 247–256. 10.1037/0012-1649.25.2.247 [DOI] [Google Scholar]
  30. Kray J., Karbach J., Haenig S., Freitag C. (2012). Can taskswitching training enhance executive control functioning in children with attention deficit/-hyperactivity disorder? Front. Hum. Neurosci. 5:180. 10.3389/fnhum.2011.00180 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kray J., Lindenberger U. (2000). Adult age differences in task switching. Psychol. Aging 15, 126–147. 10.1037/0882-7974.15.1.126 [DOI] [PubMed] [Google Scholar]
  32. Lampit A., Hallock H., Suo C., Naismith S. L., Valenzuela M. (2015). Cognitive training-induced short-term functional and long-term structural plastic change is related to gains in global cognition in healthy older adults: a pilot study. Frontiers in Aging Neuroscience 7:14. 10.3389/fnagi.2015.00014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Li S. C., Schmiedek F., Huxhold O., Rocke C., Smith J., Lindenberger U. (2008). Working memory plasticity in old age: practice gain, transfer, and maintenance. Psychol. Aging 23, 731–742. 10.1037/a0014343 [DOI] [PubMed] [Google Scholar]
  34. Lustig C., Shah P., Seidler R., Reuter-Lorenz P. A. (2009). Aging, training, and the brain: a review and future directions. Neuropsychol. Rev. 19, 504–522. 10.1007/s11065-009-9119-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Minear M., Shah P. (2008). Training and transfer effects in task switching. Mem. Cognit. 36, 1470–1483. 10.3758/mc.336.8.1470 [DOI] [PubMed] [Google Scholar]
  36. Mitchell A. J., Kemp S., Benito-Leon J., Reuber M. (2010). The influence of cognitive impairment on health-related quality of life in neurological disease. Acta Neuropsychiatr. 22, 2–13. 10.1111/j.1601-5215.2009.00439.x [DOI] [Google Scholar]
  37. Nyberg L., Bäckman L., Erngrund K., Olofsson U., Nilsson L.-G. (1996). Age differences in episodic memory, semantic memory, and priming: relationships to demographic, intellectual, and biological factors. J. Gerontol. B Psychol. Sci. Soc. Sci. 51B, P234–P240. 10.1093/geronb/51B.4.P234 [DOI] [PubMed] [Google Scholar]
  38. Nyberg L., Sandblom J., Jones S., Neely A. S., Petersson K. M., Ingvar M., et al. (2003). Neural correlates of training-related memory improvement in adulthood and aging. Proc. Natl. Acad. Sci. U.S.A. 100, 13728–13733. 10.1073/pnas.1735487100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Perrig-Chiello P., Perrig W. J., Stahelin H. B. (2000). Differential aspects of memory self-evaluation in old and very old people. Aging Ment. Health 4, 130–135. 10.1080/13607860050008646 [DOI] [Google Scholar]
  40. Petrides M. (2002). The mid-ventrolateral prefrontal cortex and active mnemonic retrieval. Neurobiol. Learn. Mem. 78, 528–538. 10.1006/nlme.2002.4107 [DOI] [PubMed] [Google Scholar]
  41. Rabbitt P., Lowe C. (2000). Patterns of cognitive ageing. Psychol. Res. Psychol. Forsch. 63, 308–316. 10.1007/s004269900009 [DOI] [PubMed] [Google Scholar]
  42. Reitan R. M. (1955). The relation of the trail making test to organic brain damage. J. Consult. Psychol. 19, 393–394. 10.1037/h0044509 [DOI] [PubMed] [Google Scholar]
  43. Salminen T., Strobach T., Schubert T. (2012). On the impacts of working memory training on executive functioning. Front. Hum. Neurosci. 6:166. 10.3389/fnhum.2012.00166 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Salthouse T. A. (2004). What and when of cognitive aging. Curr. Dir. Psychol. Sci. 13, 140–144. 10.1111/j.0963-7214.2004.00293.x26558429 [DOI] [Google Scholar]
  45. Shieh G. (2014). Optimal sample sizes for the design of reliability studies: power consideration. Behav. Res. Methods 46, 772–785. 10.3758/s13428-013-0396-0 [DOI] [PubMed] [Google Scholar]
  46. Singer T., Lindenberger U., Baltes P. B. (2003). Plasticity of memory for new learning in very old age: a story of major loss? Psychol. Aging 18, 306–317. 10.1037/0882-7974.18.2.306 [DOI] [PubMed] [Google Scholar]
  47. Stott J., Spector A. (2011). A review of the effectiveness of memory interventions in mild cognitive impairment (MCI). Int. Psychogeriatr. 23, 526–538. 10.1017/S1041610210001973 [DOI] [PubMed] [Google Scholar]
  48. Strobach T., Salminen T., Karbach J., Schubert T. (2014). Practice-related optimization and transfer of executive functions: a general review and a specific realization of their mechanisms in dual tasks. Psychol. Res. 78, 836–851. 10.1007/s00426-014-0563-7 [DOI] [PubMed] [Google Scholar]
  49. Stroop J. R. (1935). Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643–662. 10.1037/h0054651 [DOI] [Google Scholar]
  50. Talmelli L. F. D., Gratäo A. C. M., Kusumota L., Rodrigues R. A. P. (2010). Functional independence level and cognitive deficit in elderly individuals with Alzheimer's disease. Rev. Esc. Enferm. USP 44, 932–938. 10.1590/S0080-62342010000400011 [DOI] [PubMed] [Google Scholar]
  51. Tesky V. A., Thiel C., Banzer W., Pantel J. (2011). Effects of a group program to increase cognitive performance through cognitively stimulating leisure activities in healthy older subjects: the AKTIVA study. GeroPsych. J. Gerontopsychol. Geriatr. Psychiatry 24, 83–92. 10.1024/1662-9647/a000035 [DOI] [Google Scholar]
  52. Troyer A. K., Rich J. B. (2002). Psychometric properties of a new metamemory questionnaire for older adults. The Journals of Gerontol. B Psychol. Sci. Soc. Sci. 57, P19–P27. 10.1093/geronb/57.1.P19 [DOI] [PubMed] [Google Scholar]
  53. Walton C., Alexandra K., A D. L., Justine L., A C. D., Con S. (2015). Online cognitive training in healthy older adults: a preliminary study on the effects of single versus multi-domain training. Transl. Neurosci. 6, 13–19. 10.1515/tnsci-2015-0003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Xu S., Wu Z. (1986). The construction of “The Clinical Memory Test”. Acta Psychol. Sinica 18, 100–108. [Google Scholar]
  55. Yesavage J. A. (1983). Imagery pretrainng and memory training in the elderly Gerontology 29, 271–275. 10.1159/000213126 [DOI] [PubMed] [Google Scholar]
  56. Yntema D. B., Mueser G. E. (1962). Keeping track of variables that have few or many states. J. Exp. Psychol. 63, 391. 10.1037/h0045706 [DOI] [PubMed] [Google Scholar]
  57. Zeithamova D., Preston A. R. (2010). Flexible memories: differential roles for medial temporal lobe and prefrontal cortex in cross-episode binding. J. Neurosci. 30, 14676–14684. 10.1523/jneurosci.3250-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Zelinski E. M. (2009). Far transfer in cognitive training of older adults. Restor. Neurol. Neurosci. 27, 455–471. 10.3233/rnn-2009-0495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Zinke K., Zeintl M., Rose N. S., Putzmann J., Pydde A., Kliegel M. (2014). Working memory training and transfer in older adults: effects of age, baseline performance, and training gains. Dev. Psychol. 50, 304–315. 10.1037/a0032982 [DOI] [PubMed] [Google Scholar]

Articles from Frontiers in Psychology are provided here courtesy of Frontiers Media SA

RESOURCES