Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 1.
Published in final edited form as: J Behav Ther Exp Psychiatry. 2017 Aug 31;58:51–59. doi: 10.1016/j.jbtep.2017.08.005

Patient Learning of Treatment Contents in Cognitive Therapy

Nicole B Gumport 1, Lu Dong 1, Jason Y Lee 1, Allison G Harvey 1
PMCID: PMC5683909  NIHMSID: NIHMS904016  PMID: 28869825

Abstract

Background and objectives

Research has demonstrated that both memory and learning for treatment contents are poor, and that both are associated with worse treatment outcome. The Memory Support Intervention has been shown to improve memory for treatment, but it has not yet been established if this intervention can also improve learning of treatment contents. This study was designed to document the number of times participants exhibited each of the indices of learning, to examine the indices of learning and their relationship to recall of treatment points, and to investigate the association between the indices of learning and depression outcome.

Methods

Adults diagnosed with major depressive disorder (N=48) were randomly assigned to 14 sessions of cognitive therapy-as-usual (CT-as-usual) or cognitive therapy plus the Memory Support Intervention (CT+Memory Support). Measures of learning, memory, and depressive symptomatology were taken at mid-treatment, post-treatment, and at 6-month follow-up.

Results

Relative to the CT-as-usual group, participants in the CT+Memory Support group reported more accurate thoughts and applications of treatment points at mid-treatment, post-treatment, and 6-month follow-up. Patient recall was significantly correlated with application and cognitive generalization. Thoughts and application at mid-treatment were associated with increased odds of treatment response at post-treatment.

Limitations

The learning measure for this study has not yet been psychometrically validated. The results are based on a small sample.

Conclusions

Learning during treatment is poor, but modifiable via the Memory Support Intervention. These results provide encouraging data that improving learning of treatment contents can reduce symptoms during and following treatment.

Keywords: learning, memory, depression, cognitive therapy

1. Introduction

The present study was devised to examine to what extent remembering and learning is occurring during and following the receipt of cognitive therapy. We define memory as “the record of past experiences acquired through learning” and learning as “the process by which changes in behavior arise as a result of experiences interacting with the world” (Gluck, Mercado, & Myers, 2007, 6/7). Hence, memory and learning are conceptualized for the purpose of the present study as interlinked yet separable processes.

Taking memory first, research on patient memory for treatment is important for three reasons. First, extant research indicates that memory for any treatment is poor. Following a treatment session, patients with bipolar disorder were only able to recall 19.6–36.9% of the recommendations made during treatment (Lee & Harvey, 2015). Insomnia patients forgot about one third of recommendations made during treatment and recall for some types of recommendations was only 13% (Chambers, 1992). Recall is also quite poor following a physician’s visit for health behavior change advice across a variety of domains (Flocke & Stange, 2004). Second, existing research suggests that poor memory for the contents of a treatment session is associated with lower treatment adherence (Pickney & Arnason, 2005). Third, past research indicates that better memory for the contents of treatment is associated with better treatment outcome (Harvey et al., 2016; Lee & Harvey, 2015).

Moving on to learning, research on patient learning of treatment contents is important for two reasons. A recent study has demonstrated that the learning of treatment contents following treatment for depression is also poor and is associated with poorer treatment outcome (Gumport, Williams, & Harvey, 2015). In this study, although more than half the participants reported thinking about or applying the contents of treatment following their session each week, only 50–62.5% of these thoughts were accurate and less than half the applications were accurate. More promisingly, participants were able to generalize the contents of treatment more than half of the time, and the ability to generalize was highly correlated with lower depression symptoms each week. These results highlight the difficulty of learning the contents of treatment and the potential relationship between learning and improvement over the course of treatment. It appears that generalization, like recall, may be more strongly associated with improvement during treatment, as opposed to application or thoughts. The current study was designed to evaluate this relationship between these measures of learning with recall. Second, cognitive psychologists have demonstrated the “transfer of learning” problem. Thorndike (1932) posits that successful transfer of learning to novel situations depends on the number of elements in the novel situation that are identical to those in the situation in which the skills are encoded. People are often able to encode, recall, and recognize information, but there are multiple empirical demonstrations that people largely fail to apply the material that was learned in similar situations that differ only in surface features (Mestre, 2005; Rohrer, Taylor, & Sholar, 2010). Given the empirical demonstrations that transfer is worse when the encoding and testing formats differ, much of the material covered in a treatment session may not be transferred to situations outside the session. Additionally, past research has found better results on a test of learning from cognitive bibliotherapy did not predict outcome (Scogin, Jamison, Floyd, & Chaplin, 1998). More recently, better results on a test of knowledge acquisition did not predict improved outcome in internet-based treatment of eating disorders (Strandskov et al., 2017). Taken together, this accumulating evidence suggests that learning, as well as memory, may be suboptimal during treatment.

The current study examines memory and learning in the context of treatment for depression. Depression is associated with several problems with both memory and learning. First, deficits in memory are common in depression (Behnken et al., 2010), including pervasive impairments in declarative memory (Hertel, 1998; Hertel & Rude, 1991) and working memory (Gotlib & Joormann, 2010). Second, forgetting is common. While patients with depression experience more difficulty in forgetting negative words and disorder-related information (Wingenfeld, Terfehr, Meyer, Löwe, & Spitzer, 2013), they also experience greater difficulty in remembering neutral words (Cottencin et al., 2008). Third, depression is characterized by negative emotion and the experience of negative emotion is associated with attentional biasing and narrowing, which impacts which information is encoded (Easterbrook, 1959; Peckham, McHugh, & Otto, 2010; Watkins, Vache, Verney, & Mathews, 1996). One study (Phelps, 2004) found that individuals are more likely to remember the “gist” rather than specific details of an emotional event. However, treatment sessions are often emotionally arousing, and specific details and nuances are likely important when learned in these contexts. Fourth, depression is often characterized by negatively-biased schema. These schemas facilitate faulty information processing and learning, often negatively-biased (Beck & Haigh, 2014). Therefore, learning and processing new information in individuals receiving cognitive therapy for depression may be impaired. Overall, the accumulating evidence suggests that the odds are stacked against people remembering and learning new information gleaned from a cognitive therapy session.

Researchers have begun addressing the problem of poor memory for the contents of treatment. One approach has been to attempt to improve memory for the contents of therapy, which involves incorporating memory support strategies into treatment-as-usual (Harvey et al., 2014, 2016). These strategies were carefully derived from the education and cognitive psychology literatures (Harvey et al., 2014) and are proactively and strategically incorporated by the therapist without extending the session time or changing the basic content of sessions. Existing research has demonstrated that memory for treatment is modifiable using these strategies. Specifically, Harvey et al. (2016) reported that patients who had received this Memory Support Intervention incorporated into cognitive therapy-as-usual exhibited better memory for the contents of treatment relative to cognitive therapy-as-usual without the Memory Support Intervention. They also found that better performance on a free recall task was associated with improved outcome irrespective of treatment condition. Together, these findings raise the possibility that improving memory for treatment may be a pathway to improving outcomes in cognitive therapy. However, the impact of memory support on learning has yet to be examined.

We propose to further investigate this relatively novel pathway to improving treatment outcome by better understanding learning and memory and their relationship to treatment outcome. Building on the findings assessing the transfer of learning described in Gumport et al. (2015) and the Memory Support Intervention described in Harvey et al. (2016), we seek to explore the relationship between memory of treatment contents, transfer of learning of treatment contents, memory support, and treatment outcome in the context of treating depression symptoms using cognitive therapy.

We included an assessment of three indices of learning: (a) whether the participant thought about the CT treatment points, (b) whether the participant applied the CT treatment points and (c) whether the participant generalized the treatment points. The first aim was to document the number of times participants exhibited each of the three indices of learning at mid-treatment, post-treatment, and at follow-up. The hypothesis tested was that transfer of learning of the CT treatment points would be greater in the CT+Memory Support group than in the CT-as-usual group. The second aim was to examine the three indices of learning and their relationship to recall of treatment points. We predicted that greater learning would be associated with increased recall and that generalization would be more strongly associated with better recall relative to the other two indices of learning. The third aim was to investigate the association between the three indices of learning and depression outcomes. The hypothesis tested was that participants who exhibited greater learning would be more likely to exhibit improvement during treatment and that participants who exhibited greater learning would be less likely to experience another depressive episode.

2. Method

Further details regarding treatment rationale, content, and fidelity is described in Harvey et al. (2016).

2.1. Participants

Participants were 48 adults who met diagnostic criteria for MDD, regardless of chronicity or recurrence, who participated in a National Institute of Mental Health-funded randomized controlled trial comparing cognitive therapy-as-usual (CT-as-usual) to cognitive therapy with an adjunctive memory support intervention (CT+Memory Support) (Harvey et al., 2016). Adults were assigned to either CT-as-usual or CT+Memory Support in a in a 1:1 parallel group design. This study was registered (NCT01790919).

Individuals were eligible if they met the following inclusion criteria: (a) diagnosis of MDD, regardless of chronicity or recurrence, according to DSM-IV-TR criteria (American Psychiatric Association, 2000); (b) a score of 26 or above on the Inventory of Depressive Symptomatology, Self-Report (IDS-SR) (Rush, Gullion, Basco, Jarrett, & Trivedi, 1996), (c) a score of 24 or above on the Inventory of Depressive Symptomatology, Clinician-Rated (IDS-C) (Rush et al., 1996), (d) 18 years of age or older; (e) if taking medications for mood, medications must have been stable for the past four weeks, and (f) able and willing to give informed consent.

Individuals were excluded if they met any of the following criteria: (a) history of bipolar affective disorder; (b) history of psychosis or psychotic features; (c) current non-psychotic Axis I disorder that constituted the principal diagnosis (defined below) that required treatment other than that offered within the study; (d) history of substance dependence in the past six months; (e) IQ below 80; (f) evidence of any medical disorder or condition that could cause depression, or preclude participation in CT or that is associated with memory problems; or (g) current suicide risk sufficient to preclude treatment on an outpatient basis. “Principal” diagnosis was defined as the disorder currently most distressing and disabling, using a widely accepted severity rating scale capturing distress and interference (Di Nardo, Moras, Barlow, Rapee, & Brown, 1993).

A total of 48 participants were recruited and randomized for this study. Three participants dropped out from the CT+Memory Support group during treatment. Of these three, two dropped out after mid-treatment thus were still included in the mid-treatment analyses. Three participants dropped out from the CT-as-usual group after randomization or during treatment. Of these three, one dropped out after mid-treatment thus was still included in the mid-treatment analyses. The remaining 42 participants completed treatment. Hence, the overall sample size for mid-treatment analyses is 45 participants. The question as to whether the use of Memory Support can improve treatment outcome has been published elsewhere (Harvey et al., 2016). The current manuscript is a follow-on paper that examines an additional question; namely, does memory support improve learning? In the previously published paper (Harvey et al., 2016), we used an intent-to-treat model that included data for all 48 randomized participants. The reason the current study is based on 45 participants (rather than 48) is that it relies on the data collected during the mid-treatment assessment. Between randomization and the mid-treatment assessment three participants dropped out. This is represented in the CONSORT flow chart in Figure 1.

Figure 1.

Figure 1

Open in a new tab

CONSORT Diagram Illustrating the Flow of Participants Through the Study

2.2. Treatment

Therapy was delivered by licensed therapists or therapists working towards licensure.

2.2.1. Cognitive Therapy-As-Usual (CT-as-usual)

CT was first described by Aaron T. Beck and colleagues (Beck, 1979) and is based on cognitive theories of depression. It was conducted according to the published manuals.

2.2.2. Cognitive Therapy with Memory Support (CT+Memory Support)

Participants in the CT+Memory Support condition received CT with an additional Memory Support Intervention. The Memory Support Intervention is derived from the cognitive psychology and education literatures based on carefully honed criteria (Harvey et al., 2014). These memory-promoting strategies were designed to be strategically and intensively integrated into treatment-as-usual to promote the encoding of treatment contents. This intervention does not lengthen session time or increase the number of sessions required. More specifically, the Memory Support Intervention is comprised of eight strategies: attention recruitment, categorization, evaluation, application, repetition, practice remembering, cue-based reminder, and praise recall, which have been operationalized in previous work (for operational definitions, see the Appendix) (Lee, Worrell, & Harvey, 2015). These strategies are delivered alongside a treatment point, which was defined as a “main idea, principle, or experience that the treatment provider wants the patient to remember or implement as part of the treatment” (Lee & Harvey, 2015). Therapists in the CT+Memory Support condition received training in the Memory Support Intervention in addition to the standard training in CT-as-usual.

2.3. Procedure

All procedures were approved by the University of California, Berkeley, Committee for the Protection of Human Subjects. All participants provided informed consent.

Eligible participants were randomly assigned to receive cognitive therapy-as-usual (CT-as-usual) (N= 23) or cognitive therapy with an adjunctive memory support intervention (CT+Memory Support) (N=25). Regardless of treatment group, all participants received 14 sessions of therapy. At the start of each treatment session participants were asked to complete a measure of depression symptoms. At mid-treatment, immediately following session 7, participants were asked to complete a free recall task and the learning task. Upon completing treatment, participants completed a post-treatment assessment which included a measure of depression symptoms, the free recall task, and the learning task. At six months following the completion of treatment, participants completed another assessment which included a measure of depression symptoms, the free recall task, and the learning task.

2.4. Mood Measures

2.4.1 Structured Clinical Interview for DSM-IV

The SCID was administered by doctoral students in clinical psychology or by trained research coordinators to determine study eligibility. The SCID is a semi-structured interview designed to assess DSM-IV diagnostic criteria for Axis I disorders. The SCID has demonstrated good reliability for the majority of disorders that it assesses (Lobbestael, Leurgans, & Arntz, 2010).

2.4.2 Inventory of Depressive Symptomatology – Self-Report (IDS-SR)

Clinical outcome was measured using the IDS-SR. The IDS-SR is a 30-item measure of depression symptoms over the past 7 days. It has excellent internal consistency (Cronbach’s alpha = 0.92) (Trivedi et al., 2004). The IDS-SR was administered at pre-treatment, post-treatment, and 6-month follow-up.

2.4.3. Quick Inventory of Depressive Symptomatology – Self-Report (QIDS-SR)

Depression symptoms were tracked at each session of treatment using the QIDS-SR. The QIDS-SR is a 16-item measure of depression symptoms over the past 7 days. It has good internal consistency (Cronbach’s alpha = 0.86) (Rush et al., 2003). Questions on the QIDS-SR are identical to questions on the IDS-SR, although there are fewer, making it much more feasible to deliver at the start of a treatment session. Based on the conversion table described in Rush et al. (2003), we converted the QIDS-SR scores collected at mid-treatment to an IDS-SR total. All QIDS-SR scores were converted to the mid-point of the IDS-SR range (for example, a 13 on the QIDS-SR is equivalent to a range of 31–33 on the IDS-SR, so we set this equal to 32).

2.4.4. Longitudinal Interval Follow-Up Evaluation (LIFE)

The LIFE interview was administered at baseline, post-treatment, and 6-month follow-up following a standard procedure (Leverich & Post, 2002). The LIFE graphically sketches the length and severity of each affective episode across an individual’s lifespan. Severity of each episode is categorized as mild, moderate, or severe based on self-reported mood and vegetative symptoms as well as functional impairment. The LIFE has high inter-rater reliability (Keller et al., 1987).

2.4.5. Mood Outcome

Mood outcomes were assessed at post-treatment and 6-month follow-up. Based on the American College of Neuropsychopharmacology (ACNP) criteria (Rush et al., 2006), “response” was defined as a 50% reduction in IDS-SR from baseline to post-treatment. “remission” was defined as less than or equal to 14 on the IDS-SR at post-treatment, “relapse” was defined as greater than or equal to 26 on the IDS-SR at 6-month follow-up for participants who had remitted, and “recurrence” was defined as a return to moderate or severe depression following recovery, which was defined as recovery that had been sustained for ≥ 4 months. Recovery and recurrence were established with the SCID and the LIFE. Recovery from a depressive episode was assessed via the SCID interview, and time until recurrence, if recovery was not maintained, was established by counting the months without moderate or severe depression on the LIFE.

Although relapse was a primary outcome used in the initial report of this data (Harvey et al., 2016), we were unable to examine the association between the indices of learning and relapse as only 6–10 participants who relapsed completed this measure at each time point, and thus analyses lacked sufficient power to interpret.

2.5. Memory Measure

Memory for the contents of treatment was assessed via the Patient Recall Task (Harvey et al., 2016; Lee & Harvey, 2015). In this free recall task completed at mid-treatment, post-treatment, and 6-month follow-up, patients were asked to take 10 minutes to recall treatment session content for all of the sessions they have completed so far. These points were scored for the number of treatment points accurately based on an existing rubric of points covered in CT sessions.

2.6. Learning Measures

2.6.1. Thoughts

Thoughts about the lesson material were collected via a questionnaire asking the participant, “In the past 24 hours, did the lesson you completed this past week come to mind?”, and if yes, “How many times?” and “What came to mind?” This data was collected mid-treatment, post-treatment, and 6-month follow-up. To determine if the thoughts accurately reflected the therapy content, the data was then coded for the number of “treatment points” that participants reported thinking about. A “treatment point” was defined as “an insight, skill or strategy that you think is important to remember and/or implement as part of your treatment” (Harvey et al., 2014). Participant responses were coded for if their treatment point would be broadly acceptable under the guidelines of commonly used CT manuals (e.g., Beck, 1979).

2.6.2. Application

Application of the contents of therapy was assessed via a questionnaire asking participants, “Did you get to apply anything from the lesson in the past 24 hours” and, if yes, “what did you apply?” These responses were coded for accurate application of the treatment points using the method described in the paragraph above. Application was assessed mid-treatment, post-treatment, and 6-month follow-up. Participant responses were coded for the number of accurate applications of treatment points.

2.6.3. Generalization

Generalization of the lesson material was assessed by presenting participants with two scenarios that typically pose an emotional management problem for individuals diagnosed with depression and for which points from the previous lesson were relevant. The two scenarios were rejection after applying for a job and social rejection at a party. These items were drawn from the Ways of Responding Questionnaire (Barber & DeRubeis, 1992). The responses were then coded to determine if the participants generalized lesson material to these hypothetical situations. Two types of generalization were assessed. Cognitive generalization was determined by the response to, “what would you think?” Behavioral generalization was measured by the response to “how would you respond?” Generalization was scored as generalizing 0, 1 or 2 of the scenarios. Generalization was assessed at mid-treatment, post-treatment, and at 6-month follow-up. Participant responses were coded for the accurate generalization of treatment points.

2.7. Data Coding

2.7.1. Learning Measures

Learning measures were coded by two independent raters. Responses were determined to be correct if they referenced content or an adaptive response to a situation that would be included in published manuals for CBT for depression. Both raters independently scored a random subset of the learning measures (36.59%). There was 89.58% inter-rater agreement for thoughts, 84.44% inter-rater agreement for application, 86.49% inter-rater agreement for cognitive generalization, and 90.12% inter-rater agreement for behavioral generalization. Disagreements were resolved through discussion.

2.7.2. Patient Recall Task

As outlined in more detail in Harvey et al. (2016) and Lee et al. (2015), excellent inter-rater reliability was established between two independent coders (n = 32, r = 0.92, p < 0.0001).

2.8. Missing Data

At mid-treatment, all participants who completed this mid-treatment therapy session completed these indices of learning. At post-treatment, we were unable to reach two participants to complete the assessment and four participants who completed the assessment did not complete the indices of learning due to administrative error. At 6-month follow-up, we lost one additional participant, and of those who did complete the assessment, two did not fill out these indices of learning due to administrative error. All six participants who did not complete this learning measure due to administrative error were different at each time point.

2.9. Data Analysis

Analysis were calculated using R (R Core Team, 2015) and Stata 14 (StataCorp, 2015). A significance level of 0.05 was used throughout. Cohen’s d was calculated to examine the difference between the means of each learning measure in the CT-as-usual group compared to the CT+Memory Support group. Values of 0.20–.49 represent a small effect size, 0.50–0.79 represent a medium effect size, and 0.80 and above represent a large effect size (Cohen, 1992). Two-tailed Pearson’s correlations were calculated to examine the association between participant recall and thoughts and application and two-tailed Spearman’s correlations were calculated to examine the association of patient recall with cognitive generalization and behavioral generalization for all participants and separated by treatment group. Hierarchical linear modeling using restricted maximum likelihood estimation was used to examine the relationship of depression symptoms as measured by the converted QIDS-SR or IDS-SR over time with the different indices of learning. Treatment condition (CT-as-usual=0; CT+Memory Support=1) was included as a covariate in the fixed part of the model. The random part of the model included a random intercept and a random slope of time (in days) since entry into the study, assumed to have a bivariate normal distribution with a mean of zero and an unstructured covariance matrix. Logistic regression was used to calculate the odds ratios for the different indices of learning predicting the ACNP depression outcomes: response, remission, and recurrence.

3. Results

3.1. Participant Characteristics

Baseline characteristics of the participants who completed treatment at least up to the mid-treatment point are described in Table 1. No baseline differences were observed between the CT+Memory Support and CT-as-usual groups. No baseline differences were observed between participants who dropped out and those who completed treatment.

Table 1.

Participant Characteristics

Characteristic CT+Memory
Support
(n = 24)		 CT-as-usual
(n = 21)		 Test Statistic (df) p
Gender, n (% female) 11 (45.83) 15 (71.42) χ2(1) =3.01 .08
Ethnicity χ2(2) = 4.82 .09
    Hispanic or Latino, n (%) 4 (16.67) 3 (14.29)
    Not Hispanic or Latino, n (%) 20 (83.33) 15 (71.43)
    Declined to answer, n (%) 0 (0.00) 3 (14.29)
Race χ2(5) = 4.88 .43
    White, n (%) 20 (83.33) 14 (66.67)
    Black, n (%) 0 (0.00) 1 (4.76)
    Asian, n (%) 1 (4.17) 3 (14.29)
    Native American, n (%) 1 (4.17) 0 (0.00)
    Multiracial, n (%) 0 (0.00) 1 (4.76)
    Not specified, n (%) 2 (8.33) 2 (9.52)
Age (years) 43.83 (10.18) 44.38 (12.38) t(43) = 0.16 .87
IDS-SR Pre, M (SD) 38.79 (7.91) 42.33 (9.36) t(43) = 1.38 .18
Open in a new tab

Note. CT+Memory Support = cognitive therapy plus memory support intervention; CT-as-usual = cognitive therapy as usual; IDS-SR = Inventory of Depressive Symptomatology – Self-Report.

3.2. Indices of Learning During Treatment and Follow-Up

Means and group differences of each of the indices of learning are displayed in Table 2. Regarding thoughts, participants in the CT+Memory Support group relative to the CT-as-usual group reported more accurate thoughts about their CT sessions with small to large effect size differences (d = 0.28–0.95). For application, participants in the CT+Memory Support group relative to the CT-as-usual group reported more accurate application of their CT sessions with small to medium effect size differences (d = 0.25–0.76). For cognitive generalization, participants in the CT+Memory Support group relative to the CT-as-usual group displayed more cognitive generalization with small to medium effect size differences (d = 0.26–0.74). For behavioral generalization, participants in the CT+Memory Support group relative to the CT-as-usual group displayed more behavioral generalization with small to medium effect size differences (d = 0.34–0.68).

Table 2.

Means of Indices of Learning During Treatment, Post-Treatment, and Follow-Up

Indices of Learning CT+Memory Support CT-as-usual
N Mean SD N Mean SD t 95% CI d
Thoughts
    Mid-treatment 24 0.96 0.46 19 0.53 0.61 −2.55 [−0.78, −0.09] 0.80
    Post-treatment 16 1.56 1.21 11 1.27 0.79 −0.70 [−1.15,0.56]] 0.28
    6-month follow-up 9 1.44 0.72 9 0.78 0.67 −2.03 [−1.36,0.03] 0.95
Application
    Mid-treatment 24 1.29 0.75 19 1.16 1.02 −0.5 [−0.68,-0.41] 0.15
    Post-treatment 15 1.80 1.08 11 1.55 0.93 −0.63 [−1.10, −0.58] 0.25
    6-month follow-up 9 2.00 0.87 10 1.40 0.70 −1.67 [−1.36,0.16]] 0.76
Cognitive
Generalization
    Mid-treatment 24 0.92 0.78 21 0.71 0.85 −0.84 [−0.69,0.28] 0.26
    Post-treatment 20 1.00 0.73 18 0.50 0.62 −2.27 [−0.95,-0.05] 0.74
    6-month follow-up 20 1.20 0.70 20 0.80 0.7 −1.82 [−0.85,0.05] 0.57
Behavioral
Generalization
    Mid-treatment 24 1.50 0.59 21 1.24 0.77 −1.29 [−0.67,0.15] 0.38
    Post-treatment 20 1.55 0.51 18 1.11 0.76 −2.11 [−0.86,-0.02] 0.68
    6-month follow-up 20 1.45 0.69 20 1.20 0.76 −1.09 [−0.72,0.22] 0.34
Open in a new tab

Note. CT+Memory Support = cognitive therapy plus memory support; CT-as-usual = cognitive therapy as usual.

3.3. Indices of Learning and Patient Recall

As evident in Table 3, for all participants, patient recall at mid-treatment and at post-treatment were significantly correlated with application at mid-treatment. Patient recall at mid-treatment was significantly correlated with cognitive generalization at post-treatment. Patient recall at 6-month follow-up was significantly associated with thoughts at mid-treatment. Patient recall and the indices of learning were not significantly correlated with each other at any of the other time points for all participants.

Table 3.

Correlations between Indices of Learning and Patient Recall

Thoughts Application Cognitive Generalization Behavioral Generalization
Mid Post FU Mid Post FU Mid Post FU Mid Post FU
Patient Recall Task: All Participants
Mid-treatment 0.25 −0.08 0.01 0.48** 0.06 −0.17 0.22 0.41* 0.25 0.18 0.12 0.08
Post-treatment 0.3 0.07 −0.07 0.42** 0.18 −0.04 0.02 0.31 0.21 0.06 0.15 0.12
6-month follow-up .33* −0.07 0.02 0.23 0.23 0.25 −0.01 0.13 0.31 −0.04 −0.01 0.13
Open in a new tab

Note.

*

p< 0.05,

**

p < 0.01.

Mid = mid-treatment; Post = post-treatment; FU = 6-month follow-up.

3.4. Learning and Depression

3.4.1. Symptoms

As evident in Table 4, behavioral generalization was significantly associated with a decrease in IDS-SR score (β = −5.64, SE = 2.57, p = 0.03) regardless of time points. However, there was no significant interaction between behavioral generalization and time points on IDS-SR scores. None of the other learning measures were significantly associated with a change in depression symptoms from mid-treatment to post-treatment or follow-up.

Table 4.

Learning and Depression: Depressive Symptoms

Effect of learning on IDS-SR at
mid-treatment		 Effect of learning on IDS-SR
change from mid-treatment to
post-treatment		 Effect of learning on IDS-SR
change from mid-treatment to
follow-up
IDS-SR (outcome) N Beta SE p N Beta SE p N Beta SE p
Thoughts 37 0.08 3.26 0.98 37 −3.28 3.92 0.40 37 −1.34 4.64 0.77
Application 37 2.05 1.98 0.30 37 −2.53 2.91 0.38 37 2.32 3.67 0.53
Cognitive Generalization 38 −1.30 2.07 0.53 38 2.56 2.91 0.38 38 −1.12 2.93 0.70
Behavioral Generalization 38 −5.64 2.57 0.03* 38 4.07 3.32 0.22 38 3.49 3.28 0.29
Open in a new tab

Note.

*

p < 0.05.

IDS-SR = Inventory of Depressive Symptomatology – Self-Report. Each row shows the results of separate mixed effects model, with learning (Thought, Application, Cognitive Generalization, or Behavioral Generalization) and time (mid-treatment, post-treatment, and 6-month follow-up) as the predictors, IDS-SR as the outcome, and treatment conditions as a covariate.

3.4.2. ACNP Outcomes

These results are presented in Table 5. For response, thoughts and application at mid-treatment were significantly associated with treatment response at post-treatment. The odds of meeting criteria for response at post-treatment were 3.91 times higher for participants who had thought about their CT sessions in the past 24 hours at mid-treatment compared to those who had not and 2.60 times higher for participants who reported applying their CT session material in the past 24 hours at mid-treatment than those who had not. In other words, participants who had thought about or applied the material from their CT sessions in the past 24 hours were more likely to have seen a 50% or more reduction in IDS-SR scores between baseline and post-treatment. Neither behavioral generalization nor cognitive generalization at mid-treatment was associated with treatment response at post-treatment. None of the learning measures at post-treatment were associated with increased odds of responding to treatment at post-treatment.

Table 5.

Learning and Depression: ACNP Outcomes

Measure Response (at post-treatment) Remission (at post-treatment) Recurrence (at 6-month
follow-up)
N OR SE p N OR SE p N OR SE p
Mid-treatment
    Thoughts 40 3.91 1.97 0.04* 40 0.56 1.90 0.36 27 1.54 1.92 0.51
    Application 40 2.60 1.60 0.04* 40 1.45 1.58 0.41 27 1.53 1.59 0.36
    Cognitive Generalization 42 0.67 1.50 0.31 42 1.47 1.59 0.40 29 2.90 1.71 0.04*
    Behavioral Generalization 42 1.18 1.61 0.74 42 0.85 1.72 0.77 29 1.08 1.72 0.88
Post-treatment
    Thoughts 27 1.42 1.50 0.39 27 0.79 1.49 0.56 22 0.94 1.40 0.89
    Application 26 1.05 1.49 0.90 26 1.11 1.54 0.81 27 0.50 1.64 0.16
    Cognitive Generalization 38 0.85 1.59 0.73 38 1.11 1.67 0.84 26 1.60 1.71 0.38
    Behavioral Generalization 38 2.06 1.69 0.17 38 0.50 1.82 0.23 26 2.12 1.93 0.25
6-month follow-up
    Thoughts − − − − − − − − 15 0.60 2.58 0.59
    Application − − − − − − − − 15 0.29 2.25 0.12
    Cognitive Generalization − − − − − − − − 29 0.52 1.68 0.21
    Behavioral Generalization − − − − − − − − 29 0.71 1.67 0.51
Open in a new tab

Note.

*

p < .05.

ACNP = American College of Neuropsychopharmacology.

For remission, none of the learning measures at mid-treatment, post-treatment, or 6-month follow-up were associated with increased odds of remitting due to treatment at post-treatment.

For recurrence, as evident in Table 5, cognitive generalization at mid-treatment was associated with recurrence. The odds of meeting criteria for recurrence at 6-month follow-up were 2.90 times higher for participants who accurately cognitively generalized their CT session material at mid-treatment compared to those participants who did not. In other words, participants who successfully cognitively generalized material at mid-treatment were more likely to be experiencing depression at 6-month follow-up. None of the other learning measures at mid-treatment, post-treatment, or 6-month follow-up were associated with increased odds of recurrence of depression at 6-month follow-up.

4. Discussion

The first aim of the present study was to examine the number of times participants in both treatment groups exhibited each index of learning – thoughts, application, cognitive generalization, and behavioral generalization – at mid-treatment, post-treatment, and 6-month follow-up. Participants in the CT+Memory Support group reported greater amounts of learning at mid-treatment, post-treatment, and 6-month follow-up than those in the CT-as-usual group. Consistent with our hypothesis, adults with depression in the CT+Memory Support condition displayed, on average, greater learning (as indicated by each of the learning measures) with small to large effect size differences compared to those in the CT-as-usual group. These results are congruent with and supplement existing literature on memory support (e.g., Dong, Lee, & Harvey, 2017; Harvey et al., 2016). In addition to memory support improving memory for the contents of therapy, these techniques also appear to enhance learning during treatment.

The second aim was to examine whether the indices of learning were correlated with recall in both treatment conditions. In our sample of adults with depression, recall was at times correlated with several of the indices of learning. These results provide preliminary evidence that learning and memory are related constructs that may be important for gaining skills in therapy.

The third aim was to examine the relationship between the indices of learning and depression. When examining depression symptoms, behavioral generalization was the only learning outcome associated with depression, and it was not associated with a change in depression symptoms across treatment and follow-up. When examining the ACNP outcomes, thoughts about and application of treatment points at mid-treatment predicted treatment response at post-treatment. These findings suggest that learning in the initial sessions of therapy may be directly related to outcomes upon treatment completion. These results are consistent with other psychotherapy research that change early on in the course of therapy can predict sudden gains and positive outcomes for therapy (Tang & DeRubeis, 1999). The only other significant relationship was cognitive generalization predicting recurrence at 6-month follow-up. As reported elsewhere (Harvey et al., 2016), only 29 participants in the sample met criteria for recurrence, and this small sample size likely contributed to larger standard errors. Further research is necessary in order to better elucidate the relationship between cognitive generalization and a recurrence of a major depressive episode. Notably, when examining the direction of the effect for all of the 6-month follow-up indices of learning and their relationship to recurrence, all of which were measured at the same time, 6-month follow-up, all of the odds ratios are less than one, suggesting that better learning predicted better recurrence outcomes, although none of these odds ratios reached statistical significance, possibly due to the small sample size of this pilot study.

These results are partially consistent with the findings of Gumport et al. (2015). That study found a relationship between learning and depressive symptom level, but this relationship was evident only for cognitive and behavioral generalization, not for thoughts or application. In the present study, significant relationships between learning and depression were only present for the measures of thoughts and application. One possible explanation is that in the present study the outcome period was much longer—months as opposed to weeks. In the present study, we were also able to examine temporality as opposed to only correlations, which may also be related to the inconsistent findings with previous work in this domain. Therefore, these results contribute to the mixed findings of better learning being associated with improved treatment outcome (Gumport et al., 2015; Scogin et al., 1998; Strandskov et al., 2017).

These findings should be interpreted within the confines of several limitations. First, the sample was drawn from a pilot randomized controlled trial. As such, the sample size is small and the study is underpowered. However, the results trend in the direction that indicates that learning and memory are impaired yet modifiable during treatment. Second, the validity of the learning measure has not yet been tested, and was used due to the lack of a psychometrically valid alternative. Future research should focus on establishing the validity and reliability of this measure. Third, the generalization part of the learning measure used the same two scenarios at all three assessment points, thus we cannot rule out practice effects. However, participants did not receive feedback on their responses, limiting the potential for practice effects. Future research should utilize multiple counterbalanced scenarios. Fourth, this study only focused on learning and memory during one treatment, CT, for one disorder, depression. Hence, future research should elaborate on these findings and examine learning and memory across treatments and across diagnoses to maximize generalizability. Overall, these results provide preliminary data suggesting that learning is modifiable during treatment for depression.

Highlights.

  • Memory and learning for treatment contents are poor and associated with outcomes

  • Whether the Memory Support Intervention improves learning has not yet been studied

  • Learning is associated with increased odds of treatment response

  • Learning during treatment is poor but modifiable via memory support

Acknowledgments

This research was supported by the National Institute of Mental Health Grant R34MH094535 and the National Institute of Mental Health Grant R01MH108657. Original trial registration: clinicaltrials.gov. Identifier: NCT01790919. We are grateful to Emily Clark for assistance with data coding.

Appendix. Memory Support Strategies (from Lee et al., 2015

Attention Recruitment

Involves the treatment provider using language that explicitly communicates to the patient that a treatment point is important to remember (e.g., “if there is one thing I would like you to remember in ten years time, it is this” or “this is a key point to remember”), or multimedia/diverse presentation models (e.g., handouts, poems, songs, note taking, role-playing, imagery, using a white board) as a means to recruit the patient’s attention.

Categorization

Involves explicit effort by the treatment provider to work with the patient to discuss treatment points discussed into common themes/principles (e.g., “Let’s create a list of ways we can work on waking up at the same time each morning.”).

Evaluation

Involves the treatment provider working with the patient to (a) discuss the pros/cons of a treatment point (e.g., “What would be some advantages/disadvantages of waking up at the same time each morning?”); or (b) use comparisons to compare a new treatment point to an existing or hypothetical alternative (e.g., “How would this new strategy of exercising more compare to your current habit of lying in bed all day when you are feeling depressed?”).

Application

Involves the treatment provider working with a patient to apply a treatment point to past, present, or future (real or hypothesized) scenarios (e.g., “Can you think of an example in which you might try this new method of coping to deal with your stress at work?”).

Repetition

Involves the treatment provider restating, rephrasing, or revisiting information discussed earlier in treatment (e.g., “in other words,” “as we talked about earlier,” or “in sum”).

Practice Remembering

Involves the treatment provider facilitating the patient to regenerate, restate, rephrase, and/or revisit a treatment point (e.g., “Can you tell me some of the main ideas you’ve taken away from today’s session?”).

Cue-Based Reminder

Involves the treatment provider helping the patient develop new or existing cues (e.g., colored wrist bands, reminder text messages/phone calls/emails, smart phone apps, acronyms, rhymes, and other mnemonics) to facilitate memory for treatment points.

Praise Recall

Involves the treatment provider rewarding the patient for successfully recalling a treatment point (e.g., “It’s really great you remembered that point!”) or remembering to implement a desired treatment point (e.g., “I’m so glad you remembered to step back and look at the evidence.”)

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

The authors have no conflicts of interest to report.

References

  1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (4th ed., text rev.) Washington, D.C: 2000. [Google Scholar]
  2. Barber JP, DeRubeis RJ. The ways of responding: A scale to assess compensatory skills taught in cognitive therapy. Behavioral Assessment. 1992;14:93–115. [Google Scholar]
  3. Beck AT. Cognitive Therapy of Depression. New York: Guilford Press; 1979. [Google Scholar]
  4. Beck AT, Haigh EAP. Advances in cognitive theory and therapy: The generic cognitive model. Annual Review of Clinical Psychology. 2014;10:1–24. doi: 10.1146/annurev-clinpsy-032813-153734. http://doi.org/10.1146/annurev-clinpsy-032813-153734. [DOI] [PubMed] [Google Scholar]
  5. Behnken A, Schöning S, Gerss J, Konrad C, de Jong-Meyer R, Zwanzger P, Kaplan E. Persistent non-verbal memory impairment in remitted major depression - caused by encoding deficits? Journal of Affective Disorders. 2010;122(1–2):144–8. doi: 10.1016/j.jad.2009.07.010. http://doi.org/10.1016/j.jad.2009.07.010. [DOI] [PubMed] [Google Scholar]
  6. Chambers MJ. Therapeutic issues in the behavioral treatment of insomnia. Professional Psychology: Research and Practice. 1992;23(2):131–138. http://doi.org/10.1037/0735-7028.23.2.131. [Google Scholar]
  7. Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155–159. doi: 10.1037//0033-2909.112.1.155. http://doi.org/10.1037/0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
  8. Cottencin O, Gruat G, Thomas P, Devos P, Goudemand M, Consoli SM. Directed forgetting in depression. Journal of the International Neuropsychological Society. 2008;14:895–899. doi: 10.1017/S1355617708081186. [DOI] [PubMed] [Google Scholar]
  9. Di Nardo PA, Moras K, Barlow DH, Rapee RM, Brown TA. Reliability of DSM-III-R Anxiety Disorder Categories Using the Anxiety Disorders. Archives of General Psychiatry. 1993;50:251–256. doi: 10.1001/archpsyc.1993.01820160009001. [DOI] [PubMed] [Google Scholar]
  10. Dong L, Lee JY, Harvey AG. Memory Support Strategies and Bundles: A Pathway to Improving Cognitive Therapy for Depression? Journal of Consulting and Clinical Psychology. 2017;85(3):187–199. doi: 10.1037/ccp0000167. http://doi.org/10.1037/ccp0000167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Easterbrook JA. The effect of emotion on cue utilization and the organization of behavior. Psychological Review. 1959;66(3):183–201. doi: 10.1037/h0047707. http://doi.org/10.1037/h0047707. [DOI] [PubMed] [Google Scholar]
  12. Flocke SA, Stange KC. Direct observation and patient recall of health behavior advice. Preventive Medicine. 2004;38(3):343–9. doi: 10.1016/j.ypmed.2003.11.004. http://doi.org/10.1016/j.ypmed.2003.11.004. [DOI] [PubMed] [Google Scholar]
  13. Gluck MA, Mercado E, Myers CE. Learning and Memory: From Brain to Behavior. New York: Worth Publishers; 2007. [Google Scholar]
  14. Gotlib IH, Joormann J. Cognition and Depression: Current Status and Future Directions. Annual Review of Clinical Psychology. 2010;6:285–312. doi: 10.1146/annurev.clinpsy.121208.131305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gumport NB, Williams JJ, Harvey AG. Learning cognitive behavior therapy. Journal of Behavior Therapy and Experimental Psychiatry. 2015;48:164–9. doi: 10.1016/j.jbtep.2015.03.015. http://doi.org/10.1016/j.jbtep.2015.03.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Harvey AG, Lee J, Smith RL, Gumport NB, Hollon SD, Rabe-Hesketh S, Abrons D. Improving Outcome for Mental Disorders by Enhancing Memory for Treatment. Behaviour Research and Therapy. 2016;81:35–46. doi: 10.1016/j.brat.2016.03.007. http://doi.org/10.1016/j.brat.2016.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Harvey AG, Lee J, Williams J, Hollon SD, Walker MP, Thompson MA, Smith R. Improving Outcome of Psychosocial Treatments by Enhancing Memory and Learning. Perspectives on Psychological Science. 2014;9(2):161–79. doi: 10.1177/1745691614521781. http://doi.org/10.1177/1745691614521781. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hertel PT. Relation between rumination and impaired memory in dysphoric moods. Journal of Abnormal Psychology. 1998;107(1):166–172. doi: 10.1037//0021-843x.107.1.166. http://doi.org/10.1037/0021-843X.107.1.166. [DOI] [PubMed] [Google Scholar]
  19. Hertel PT, Rude SS. Depressive deficits in memory: Focusing attention improves subsequent recall. Journal of Experimental Psychology: General. 1991;120(3):301–309. doi: 10.1037/0096-3445.120.3.301. http://doi.org/10.1037/0096-3445.120.3.301. [DOI] [PubMed] [Google Scholar]
  20. Keller MB, Lavori PW, Friedman B, Endicott J, Mcdonald-Scott P, Andreasen NC. The longitudinal interval follow-up evaluation. Archives of General Psychiatry. 1987;44:540–548. doi: 10.1001/archpsyc.1987.01800180050009. http://doi.org/10.1001/archpsyc.1987.01800180050009. [DOI] [PubMed] [Google Scholar]
  21. Lee JY, Harvey AG. Memory for therapy in bipolar disorder and comorbid insomnia. Journal of Consulting and Clinical Psychology. 2015;83:92–102. doi: 10.1037/a0037911. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lee JY, Worrell FC, Harvey AG. The Development and Validation of the Memory Support Rating Scale. Psychological Assessment. 2015;28(6):715–725. doi: 10.1037/pas0000219. http://doi.org/10.1037/pas0000219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Leverich GS, Post RM. The NIMH Life Chart Manual for Recurrent Affective Illness -- Clinician Retrospective (Unpublished Manuscript) Bethesda, MD: NIMH; 2002. [Google Scholar]
  24. Lobbestael J, Leurgans M, Arntz A. Inter-rater reliability of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID I) and Axis II Disorders (SCID II) Clinical Psychology & Psychotherapy. 2010;18(1):75–9. doi: 10.1002/cpp.693. http://doi.org/10.1002/cpp.693. [DOI] [PubMed] [Google Scholar]
  25. Mestre JP. Transfer of learning from a modern multidisciplinary perspective. Information Age Pub Incorporated; 2005. [Google Scholar]
  26. Peckham AD, McHugh RK, Otto MW. A meta-analysis of the magnitude of biased attention in depression. Depression and Anxiety. 2010;27(12):1135–1142. doi: 10.1002/da.20755. http://doi.org/10.1002/da.20755. [DOI] [PubMed] [Google Scholar]
  27. Phelps EA. Human emotion and memory: interactions of the amygdala and hippocampal complex. Current Opinion in Neurobiology. 2004;14(2):198–202. doi: 10.1016/j.conb.2004.03.015. http://doi.org/10.1016/j.conb.2004.03.015. [DOI] [PubMed] [Google Scholar]
  28. Pickney CS, Arnason JA. Correlation between patient recall of bone densitometry results and subsequent treatment adherence. Osteoporosis International. 2005;16(9):1156–1160. doi: 10.1007/s00198-004-1818-8. http://doi.org/10.1007/s00198-004-1818-8. [DOI] [PubMed] [Google Scholar]
  29. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2015. [Google Scholar]
  30. Rohrer D, Taylor K, Sholar B. Tests enhance the transfer of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2010;36(1):233–239. doi: 10.1037/a0017678. http://doi.org/10.1037/a0017678. [DOI] [PubMed] [Google Scholar]
  31. Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The Inventory of Depressive Symptomatology (IDS): psychometric properties. Psychological Medicine. 1996;26:477–486. doi: 10.1017/s0033291700035558. http://doi.org/10.1017/S0033291700035558. [DOI] [PubMed] [Google Scholar]
  32. Rush AJ, Kraemer HC, Sackeim HA, Fava M, Trivedi MH, Frank E, Schatzberg AF. Report by the ACNP Task Force on Response and Remission in Major Depressive Disorder. Neuropsychopharmacology. 2006;31(9):1841–1853. doi: 10.1038/sj.npp.1301131. http://doi.org/10.1038/sj.npp.1301131. [DOI] [PubMed] [Google Scholar]
  33. Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, Keller MB. The 16-Item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biological Psychiatry. 2003;54(5):573–583. doi: 10.1016/s0006-3223(02)01866-8. http://doi.org/10.1016/S0006-3223(02)01866-8. [DOI] [PubMed] [Google Scholar]
  34. Scogin F, Jamison C, Floyd M, Chaplin WF. Measuring Learning in Depression Treat ent: A Cognitive Bibliotherapy Test. Cognitive Therapy and Research. 1998;22(5):475–482. [Google Scholar]
  35. StataCorp. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP; 2015. [Google Scholar]
  36. Strandskov SW, Ghaderi A, Andersson H, Parmskog N, Hjort E, Wärn AS, Andersson G. Effects of Tailored and ACT-Influenced Internet-Based CBT for Eating Disorders and the Relation Between Knowledge Acquisition and Outcome: A Randomized Controlled Trial. Behavior Therapy. 2017;48(5):624–637. doi: 10.1016/j.beth.2017.02.002. http://doi.org/10.1016/j.beth.2017.02.002. [DOI] [PubMed] [Google Scholar]
  37. Tang TZ, DeRubeis RJ. Sudden gains and critical sessions in cognitive-behavioral therapy for depression. Journal of Consulting and Clinical Psychology. 1999;67(6):894–904. doi: 10.1037//0022-006x.67.6.894. http://doi.org/10.1037/0022-006X.67.6.894. [DOI] [PubMed] [Google Scholar]
  38. Thorndike EL. The Fundamentals of Learning. New York: Teachers College Bureau of Publications; 1932. [Google Scholar]
  39. Trivedi MH, Rush AJ, Ibrahim HM, Carmody TJ, Biggs MM, Suppes T, Kashner TM. The Inventory of Depressive Symptomatology, Clinician Rating (IDS-C) and Self-Report (IDS-SR), and the Quick Inventory of Depressive Symptomatology, Clinician Rating (QIDS-C) and Self-Report (QIDS-SR) in public sector patients with mood disorders: a psych. Psychological Medicine. 2004;34:73–82. doi: 10.1017/s0033291703001107. http://doi.org/10.1017/S0033291703001107. [DOI] [PubMed] [Google Scholar]
  40. Watkins PC, Vache K, Verney SP, Mathews A. Unconscious mood-congruent memory bias in depression. Journal of Abnormal Psychology. 1996;105(1):34–41. doi: 10.1037//0021-843x.105.1.34. http://doi.org/10.1037/0021-843X.105.1.34. [DOI] [PubMed] [Google Scholar]
  41. Wingenfeld K, Terfehr K, Meyer B, Löwe B, Spitzer C. Memory Bias for Emotional and Illness-Related Words in Patients with Depression, Anxiety and Somatization Disorders: An Investigation with the Directed Forgetting Task. Psychopathology. 2013;46(1):22–27. doi: 10.1159/000338609. http://doi.org/10.1159/000338609. [DOI] [PubMed] [Google Scholar]

RESOURCES