Memory Support Strategies and Bundles: A Pathway to Improving Cognitive Therapy for Depression?

Abstract

Objective

Therapist use of memory support (MS) alongside treatment-as-usual, with the goal of enhancing patient recall of treatment contents, has been of recent interest as a novel pathway to improve treatment outcome. The Memory Support Intervention involves treatment providers’ using eight specific MS strategies to promote patient memory for treatment. The present study examines to what extent therapist use of MS strategies and bundles improves patient recall of treatment contents and treatment outcome.

Methods

The data were drawn from a pilot randomized controlled trial reported elsewhere. Participants were 48 adults (mean age = 44.27 years, 29 females) with major depressive disorder (MDD), randomized to receive 14 sessions of either CT+Memory Support (n = 25) or CT-as-usual (n = 23). Therapist use of MS was coded using the Memory Support Rating Scale. Patient memory and treatment outcomes were assessed at baseline, mid-treatment (patient recall only), post-treatment, and 6-month follow-up.

Results

Participants in CT+Memory Support received significantly higher amount of MS relative to CT-as-usual. Though not reaching statistical significance, small-to-medium effects were observed between MS strategies and patient recall in the expected direction. Although MS variables were not significantly associated with changes in continuous depressive symptoms, MS was associated with better global functioning. MS also exhibited small to medium effects on treatment response and recurrence in the expected direction but not on remission, though these effects did not reach statistical significance.

Conclusions

These results provide initial empirical evidence supporting an active method for therapists to implement MS strategies.

Although psychosocial interventions such as cognitive behavioral therapy (CBT) are effective in treating psychiatric disorders, there is room for improvement (Lambert, 2011; Rey, Marin, & Silverman, 2011; Vittengl, Clark, Dunn, & Jarrett, 2007). As such, improving psychosocial treatments for mental disorders is a high priority. One novel and promising pathway to improve treatment outcome is by therapists’ infusing treatment-as-usual with Memory Support (MS) strategies to enhance patient memory for the content of treatment (Harvey et al., 2014). Improving patient memory for treatment has been of interest because: 1) patient memory for treatment is poor; 2) poor memory for treatment is associated with poor treatment adherence and outcomes; and 3) mounting evidence suggests that MS strategies can improve outcome. These three domains of evidence will now be reviewed.

There is robust evidence suggesting that patient memory for diagnostic and treatment information is poor. In the medical literature, patients immediately forgot about 40–80% of the advice given by a physician (Kessels, 2003) and only recalled between 19–33% of the medical advice within a month following a clinic visit (Bober, Hoke, Duda, & Tung, 2007; Jansen et al., 2008; Lewkovich & Haneline, 2005; Pickney & Arnason, 2005). There is also evidence that patient memory is particularly poor for medical advice about health behavior change (Flocke & Stange, 2004). In the mental health literature, patients with insomnia forgot one third of the recommendations made by behavioral therapists and the recall was as low as 13% for certain recommendations (Chambers, 1991). A recent study reported that patients with co-occurring bipolar disorder and insomnia only recalled 20–37% of the treatment points from weekly cognitive therapy sessions (Lee & Harvey, 2015). The poor memory for treatment in those diagnosed with a mental disorder is perhaps not surprising because memory impairment is common across these disorders (e.g., Behnken et al., 2010; Martino, Igoa, Marengo, Scapola, & Strejilevich, 2011). Moreover, negative emotions, another common feature associated with mental disorders, can affect the encoding process by biasing attention (Phelps, 2004), which may be one of the mechanisms by which memory impairment is is so pervasive across mental disorders.

There is also evidence that patients’ poor memory for diagnostic and treatment information is associated with poor treatment adherence and outcome. Evidence in the medical literature suggests that poor memory for medical advice is associated with lower treatment adherence (e.g., Bober et al., 2007; Jansen et al., 2008; Tosteson et al., 2003), which leads to incorrect or incomplete implementation of the medical recommendations (Vermeire, Hearnshaw, Van Royen, & Denekens, 2001) as well as worse treatment outcome (e.g., Simpson et al., 2011). Consistent with the medical literature, a study of a psychosocial treatment also found that better patient recall is correlated with improved sleep outcome among patients with co-occurring bipolar disorder and insomnia following cognitive therapy for insomnia (Lee & Harvey, 2015).

Importantly, there is evidence that MS strategies can be effective. MS strategies can improve memory encoding and retention for various patient populations, including those with dementia and depression (Almkvist, Fratiglioni, Agüero-Torres, Viitanen, & Bäckman, 1999; Bäckman & Forsell, 1994; Taconnat et al., 2010). More recently, a novel adjunctive Memory Support Intervention has been developed to improve patient memory for treatment with the goal of improving treatment outcome (Harvey et al., 2014). The first version was comprised of eight MS strategies designed to be utilized by trained treatment providers as an adjunct to treatment-as-usual. The eight strategies were derived from a thorough review of the basic cognitive science and education literatures on strategies demonstrated to enhance learning (Harvey et al., 2014). A pilot randomized controlled trial (RCT) of this first version of the Memory Support Intervention has yielded encouraging preliminary results (Harvey et al., 2016). In this study, patients with depression were randomly allocated to receive either Cognitive Therapy with the Memory Support Intervention (CT+Memory Support) or traditional Cognitive Therapy (CT-as-usual). Small-to-medium effect sizes were observed for both cumulative (d = .38) and past session recall (d = .38) at post-treatment in the direction of the CT+Memory Support condition recalling more treatment points compared to the CT-as-usual condition, although the group difference did not reach statistical significance (both p’s = .24). While the odds of meeting criteria for ‘response’ (OR = 2.80, 95% CI: [0.78, 9.99], p = .11) and ‘remission’ (OR = 3.24, 95% CI: [0.72, 14.57], p = .13) in CT+Memory Support were not significantly different from CT-as-usual, medium effect sizes favoring CT+Memory Support were observed. Compared to CT-as-usual, CT+Memory Support was associated with functional improvement at post-treatment (d = .56, p = .02) and 6-month follow-up (d = .37, p = .12) but was not associated with changes in depressive symptoms. Finally, a three-way interaction (treatment condition × education × time period) was significant for both depressive symptoms and functional impairment in the direction favoring CT+Memory Support for those with lower education level. Of note, baseline declarative memory skills did not moderate the effects of CT+Memory Support versus CT-as-usual on depression outcomes (Harvey et al., 2016).

Despite these promising results, crucial follow-up questions remain. Importantly, the specific types of MS that are associated with the optimization of improved outcome to guide the development and refinement of the Memory Support Intervention are not known. It is important to note that the only difference between randomized treatment conditions CT+Memory Support and CT-as-usual is the levels of MS therapists provided: CT+Memory Support received high levels of MS strategically and deliberately implemented by the therapists, whereas CT-as-usual received low levels of MS already imbedded in CT. Examining the degree and types of MS used in both conditions is a necessary step because the randomized treatment condition does not differentiate the specific features of MS (e.g., total amount of MS, No. of different MS types, MS bundles) or the eight MS strategies in relations to patient recall and treatment outcome. There is a need to determine the relative effectiveness of specific types of MS and if certain “bundles” of MS, defined as using more than one type of MS strategy at one time, are more efficient or effective than others. From a treatment development perspective, results from the present study provide helpful information for guiding future development, refinement, and simplification of MS strategies and how they should be optimally delivered in treatment.

The overall goal of the present study is to characterize the impact of specific MS strategies, and bundles of MS strategies, on patient recall of the treatment contents as well as on clinical outcome. The first aim is to compare the specific types of MS strategies and bundles used during Cognitive Therapy with Memory Support (CT+Memory Support) versus CT-as-usual. The hypothesis tested is that therapists will use more total MS, more different types of MS, and more bundles of MS during CT+Memory Support relative to CT-as-usual. The second aim is to examine whether the use of MS is associated with patient recall of the treatment contents. The hypothesis is that higher total amount of MS, a greater number of different types of MS used, and more bundles of MS used is associated with better patient recall of the treatment contents. The third aim is to examine whether using MS strategies is associated with clinical outcome. The hypothesis is that higher total amount of MS, a greater number of different types of MS used, and more bundles of MS used will be associated with decreased depressive symptom ratings and improved functioning. The fourth aim is to test whether patient recall of treatment contents mediates the effect of the adjunctive Memory Support Intervention on continuous depression outcome (i.e., depressive symptoms and global functioning). The hypothesis is that patient recall will be associated with both treatment condition (CT+Memory Support vs. CT-as-usual) and depression outcome, and the effect of treatment condition on depression outcome will be reduced when controlling for patient recall. There is no specific hypothesis regarding which MS bundles are more effective than others, as the analyses regarding MS bundles are exploratory with the goal of informing future refinement of the Memory Support Intervention. For aim 2 and 3, we conducted the analyses using the combined sample of participants across treatment conditions to achieve higher power and because we expected that higher levels of MS would be associated with better patient recall and clinical outcome regardless of treatment condition.

Methods

Participants

Data were provided by participants who were recruited to participate in a NIMH funded pilot RCT reported elsewhere (Harvey et al., 2016). The participants were 48 adults with Major Depressive Disorder (MDD) recruited through clinician referrals or advertisements between November 2012 and March, 2014. Participants were first screened for eligibility via a telephone interview, and potentially eligible individuals participated in an in-person assessment session. Table 1 shows the demographic information for the participants in the present study. There was no statistically significant difference by treatment condition on the demographic variables. Three participants (one in CT+Memory Support, two in CT-as-usual) were taking benzodiazepine during treatment. No participant endorsed taking modafinil or other wake/memory agent. In addition, the endorsement of current tobacco use was not significantly different across treatment conditions at baseline (8% vs. 13.64%, χ²(1, N = 47) = 0.39, p = .53), post-treatment (4.55% vs. 15%, χ²(1, N = 42) = 1.32, p = .25), or at 6-month follow-up (5% vs. 20%, χ²(1, N = 40) = 2.06, p = .15).

Table 1.

Demographic Information for the Total Sample and Each Treatment Condition

Characteristic	Total (N = 48)		CT-as-usual (N=23)		CT+Memory Support (N= 25)
	M or N	% or SD	M or N	% or SD	M or N	% or SD
Female	29	60.42	17	73.90	12	48.00
Ethnicity
Hispanic or Latino	8	16.67	3	13.04	5	20.00
Not Hispanic or Latino	37	77.08	17	73.91	20	80.00
Declined to answer	3	6.25	3	13.04	0	0.00
Race
American Indian/Alaska Native	1	2.08	0	0.00	1	4.00
Asian	4	8.33	3	13.04	1	4.00
African American	2	4.17	1	4.35	1	4.00
Caucasian	36	75.00	16	69.57	20	80.00
Bi-racial/Multi-racial	1	2.08	1	4.35	0	0.00
Declined to answer	4	8.33	2	8.70	2	8.00
Marital Status
Single	23	47.92	11	47.83	12	48.00
Married/Partnered	18	37.50	8	34.78	10	40.00
Divorced/Separated/Widow	6	12.50	3	13.05	3	12.00
Declined to answer	1	2.08	1	4.35	0	0.00
Employed
Full-time	13	27.08	6	26.09	7	28.00
Part-time	13	27.08	4	17.39	9	36.00
Unemployed	15	31.25	8	34.78	7	28.00
Retired	3	6.25	2	8.70	1	4.00
Declined to answer	4	8.33	3	13.05	1	4.00
Income
<$20,000	17	35.42	7	30.40	10	40.00
$20,000-$35,000	6	12.50	2	8.70	4	16.00
$35,000-$50,000	11	22.92	7	30.40	4	16.00
$50,000-$60,000	6	12.50	1	4.30	5	20.00
>$60,000	3	6.25	2	8.70	1	4.00
Refused/Did not know	8	10.42	4	16.70	1	4.00
Comorbidity, Medical	23	47.92	11	45.80	12	50.00
Comorbidity, Psychiatric	26	54.17	10	43.50	16	64.00
Mood Medication	20	41.67	11	44.00	9	39.10
Age (years)	44.27	10.97	44.65	12.17	43.92	9.98
Education (years)	16.93	2.86	16.26	2.03	15.40	1.68

Note. Total sample size used in data analysis was 44 due to missing MS data. Group differences were not statistically significant for all the demographic variables listed based on t or χ² tests.

All participants were required to meet the following inclusion criteria: 1) diagnosis of MDD, regardless of chronicity or recurrence, according to DSM-IV-TR criteria¹ (American Psychiatric Association, 2000); 2) minimum scores of 26 or above on the Inventory of Depressive Symptomatology, Self-Report (IDS-SR) (Rush, Gullion, Basco, Jarrett, & Trivedi, 1996); 3) minimum scores of 24 or above on the Inventory of Depressive Symptomatology, Clinician Report (IDS-C) (Rush et al., 1996); 4) 18 years of age or older; 5) stable mood medication regimen (if any) for the past four weeks; and 6) able and willing to give informed consent.

Participants were excluded if they met any of the following criteria: 1) history of bipolar affective disorder; 2) history of psychosis or psychotic features (including schizophrenia, schizophreniform disorder, schizoaffective disorder, delusional disorder, or psychotic organic brain syndrome); 3) current non-psychotic Axis I disorder that constitutes the principal diagnosis (defined below) requiring treatment other than that offered within the study; 4) history of substance dependence in the past six months; 5) IQ below 80; 6) evidence of any medical disorder or condition that could cause depression, or preclude participation in CT or that is associated with memory problems; or 7) current suicide risk sufficient to preclude treatment on an outpatient basis. Participants were included as long as MDD was the principal diagnosis. Co-occurring disorders were not exclusionary.

Procedures

The study was approved by the University Committee for the Protection of Human Subjects (CPHS). Full details are available elsewhere (Harvey et al., 2016). In brief, all participants were randomized to receive either Cognitive Therapy plus Memory Support (CT+Memory Support) or standard Cognitive Therapy (CT-as-usual), stratified by age (<46, ≥ 46 years) and gender. All participants were assessed at baseline, end of treatment, and 6-month follow-up. Treatments were delivered by licensed therapists or therapists working toward licensure. All therapists received weekly supervision for either CT+Memory Support or CT-as-usual by licensed clinical psychologists. Both treatments included 14 weekly, 50-min sessions. CT-as-usual was delivered according to published manuals developed by Beck and colleagues (Beck, 1979). CT+Memory Support was delivered with a Memory Support Intervention being added to CT-as-usual. The Memory Support Intervention is comprised of eight MS strategies that were carefully derived based on cognitive science and education literature (Harvey et al., 2014) and designed to be integrated into treatment-as-usual to enhance patient memory for the treatment. The eight MS strategies are: attention recruitment, application, evaluation, categorization, repetition, practice remembering, cue-based reminder, and praise recall (see Appendix). MS is implemented with each “treatment point,” defined as a “main idea, principle, or experience that the treatment provider wants the patient to remember or implement as part of the treatment” (Lee & Harvey, 2015). The addition of the Memory Support Intervention to CT-as-usual is not intended to lengthen treatment sessions. Indeed, in the present study, session length was not significantly different across treatment conditions (average session length: CT+Memory Support = 63.05 min, CT-as-usual = 59.87 min, t = −1.19, p = 0.24).

Measures

Single-blind assessors were graduate students in clinical psychology and research assistants, who were carefully trained and supervised. All assessors were independent of the therapy team and blind to treatment condition. Except where specified, all measures were delivered at baseline, at the end of treatment, and at 6-month follow-up.

Memory Support Rating Scale (MSRS)

Therapists’ use of MS strategies during treatment was measured using the MSRS (Lee, Worrell, & Harvey, 2015). MSRS coders individually established 80% or higher inter-coder agreement with the expert coder across five consecutive 30-minute segments of treatment recordings. The scale scores have adequate convergent validity (r’s = 0.29–36, p’s = .02–.07), discriminant validity (r’s = .07–.13, p’s = .42–.67), group differentiation ability (d’s = 1.50–1.64; p’s <001), internal consistency (Cronbach’s α = .77), inter-rater reliability (ICC’s = .73–74), and test-retest reliability (ICC’s = 70–72) (Lee et al., 2015).

In the present study, MS in a given session was scored for MS strategies, MS summary scores, and MS bundles. MS strategies (8 in total) indicate the average use of each specific MS strategy per session. There are two MS summary scores: 1) total amount of MS indicates the average total amount of MS used per session; 2) no. of MS types use d indicates the average number of different types of MS used per session (Lee et al., 2015). MS bundles were coded if two or more MS strategies were used at the same time (for examples of MS bundles, see Supplemental Material). There are two MS bundle variables: 1) MS bundles ≥ 2 indicates the average number per session of using two or more MS strategies at the same time, and 2) MS bundles ≥ 3 indicates the average number per session of using three or more MS strategies at the same time.

Inventory of Depressive Symptoms - Self Report (IDS-SR)

The primary mood outcome was participants’ depressive symptoms, which was measured by the IDS-SR (Rush et al., 1996) at baseline, post-treatment, and 6-months follow-up. IDS-SR is a widely-used, 30-item measure with adequate reliability and validity (Rush et al., 1996). All items were rated on a 4-point scale. In the current sample, the Cronbach’s alpha for the IDS-SR items were 0.78, 0.89, and 0.92 at baseline, post-treatment, and 6-month follow-up respectively. IDS-SR summary score was generated (Rush et al., 1996) for each time point for data analysis. The total score ranges from 0 – 84 with higher scores indicating greater depressive severity.

Categorical mood outcomes

Additional mood outcomes include four binary variables: response, remission, relapse, and recurrence. Using American College of Neuro-Psychopharmacology (ACNP) criteria (Rush et al., 2006), response was defined as 50% change in IDS-SR from baseline to post-treatment, remission was defined as less than or equal to 14 on the IDS-SR at post-treatment, relapse was defined as greater than or equal to 26 on the IDS-SR at 6-month follow-up for participants who had remitted at post-treatment, and recurrence was defined as a return to moderate or severe depression following recovery which was defined as remission that has been sustained for ≥ 4 months. Recurrence was determined using the combination of Structured Clinical Interview for DSM-IV (SCID) (First, Spitzer, Gibbon, & Williams, 2002) and Longitudinal Interval Follow-up Evaluation (LIFE) (Keller et al., 1987). SCID was used to determine the presence or absence of current DSM-IV-TR depressive episode, and LIFE was used to determine the number of depressive episodes during the time between post-treatment and 6-month follow-up.

Functional impairment outcomes

The Global Assessment of Functioning (GAF) is an assessor rating from 1 to 100, with lower scores indicating more severe impairment (American Psychiatric Association, 2000). There is evidence supporting the inter-rater reliability between trained clinicians as well as between researchers (Hilsenroth et al., 2000; Startup, Jackson, & Bendix, 2002; Vatnaland, Vatnaland, Friis, & Opjordsmoen, 2007). There is also evidence supporting its validity for indicating global psychopathology/illness severity and change over time (Skodol, Link, Shrout, & Horwath, 1988). In the present study, GAF scores are a treatment outcome for depression.

Patient Recall Task (Lee & Harvey, 2015)

Patient recall of the treatment was measured using this free recall task at the end of Sessions 7 and 14 and at the 6-month follow-up. Patients were given 10 min to write down as many treatment points as they could remember from the start of treatment up to (and including) their most recent session. The instruction for patients was: “think back to all the treatment sessions you’ve had with us so far” and “list as many distinct treatment points as you can recall since the start of your treatment.” Cumulative recall for a given session is the raw number of treatment points accurately recalled from the start of treatment up to the most recent session; past session recall for a given session is the raw number of treatment points accurately recalled for the most recent session. Treatment point, as defined earlier, refers to as an insight, skill, or strategy that is important for the patient to remember and/or implement as part of the treatment (Harvey et al., 2014). An expert coder coded the raw number of treatment points accurately recalled by the patients. Excellent inter-rater reliability between two independent coders (n = 32, r = .92, p < .001) and predictive validity of clinical outcome (n = 30, r ‘s = .34–.69, p’s < .001–.154) were established in a previous study (Lee & Harvey, 2015). In the current sample, the scores demonstrated adequate predictive validity with levels of MS received (r ‘s = .29–.36, p = .022–.073).

Data Analysis

All data analysis was conducted using Stata 14 (StataCorp, 2015). Logistic regression was used to examine the impact of MS variables on binary outcome variables (e.g., response, remission). Odds ratios were used to indicate effect sizes for logistic regression analysis. Multilevel modeling (i.e., linear mixed modeling) was used to examine the impact of MS variables on repeated measures of outcome variables. Maximum likelihood (ML) estimation method was used with missing data assumed Missing At Random (MAR). Contiguous mood outcomes (IDS-SR scores and GAF) were measured at baseline, post-treatment, and 6-month follow-up, while patient recall variables were measured at session 7 and 14 and 6-month follow-up. The fixed part of the model includes MS variable (e.g., Total amount of MS, No. of MS types used, or MS bundles), indictor variables for time periods (e.g., post-treatment and 6-month follow-up, with baseline or session 7 as the reference), and two interaction terms between MS variable and time. The random part included a random intercept and slope of time (in days) since entry into the study, assumed to have a bivariate normal distribution with zero means and unstructured covariance matrix. Normality check based on visual inspection of histogram and normal QQ plot suggests normality assumption of the residuals at each level is not violated in these models. For the mediation test, we used the method proposed by Kraemer et al. (2002) as well as its extension (Stice et al., 2007; Stice et al., 2010). Criteria and model specification are based on Stice et al. (2007). We tested for mediation using Pre to Post data, as this is when most intervention effects occur and when changes in the outcome are typically linear (Stice et al., 2007; Stice et al., 2010). In addition to evaluating the statistical significance at α = .05, a corrected significance level of .013 was used for each specific aim (0.05/4 aims) to address potential issue of multiple testing. Given that this is a pilot study in the context of treatment development, we not only interpret results that achieved statistical significance, but also the effect sizes without corresponding statistical significance (Cumming, 2012; Lee, Whitehead, Jacques, & Julious, 2014). Cohen’s d was used to express effect sizes for group comparisons and was interpreted as .20 = small effect, .50 = medium effect, and .80 = large effect (Cohen, 1988). Odds ratios were interpreted as 1.49 (or .67) = small effect, 3.45 (or .30) = medium effect, and 9.00 (or .11) = large effect (Olivier & Bell, 2011). For MLM results, we reported standardized coefficients, which indicate the mean change in standard deviation (SD) units of y for a one SD change in x. Consistent with the interpretation of Cohen’s d, a mean change of .20 SD units of y = small effect, .50 SD change = medium effect, and .80 SD change = large effect.

Results

Aim 1

As shown in Table 2, repetition and attention recruitment were the two most frequently used MS strategies across the two treatment conditions. Comparisons of means by treatment condition indicated that patients in CT+Memory Support received significantly higher levels of MS compared to those in CT-as-usual on six out of the eight MS strategies (i.e., attention recruitment, application, evaluation, repetition, practice remembering, and praise recall) at α = .05. Out of these six MS strategies, five (all except for application) were significant at the corrected α = .013. Total amount of MS and No. of MS types used were also significantly higher in the CT+Memory Support condition than in the CT-as-usual condition at α = .013. Effect sizes were notably large with several greater than 1.

Table 2.

Descriptive Statistics of the Memory Support (MS) Variables and Mean Comparisons by Treatment Condition

	Total (N=44)			CT+Memory Support (N=23)			CT-as-usual (N=21)
MS variables	N	M	SD	N	M	SD	N	M	SD	t	p	d
MS strategies
Attention recruitment	44	3.35	1.97	23	4.33	1.85	21	2.28	1.51	−3.99	0.0003	1.21
Application	44	1.63	1.30	23	2.03	1.47	21	1.19	0.94	−2.28	0.03	0.68
Evaluation	44	0.71	0.65	23	1.01	0.66	21	0.39	0.48	−3.51	0.001	1.07
Categorization	44	0.28	0.35	23	0.25	0.36	21	0.32	0.33	0.66	0.51	0.20
Repetition	44	4.78	2.90	23	6.34	2.93	21	3.08	1.69	−4.57	0.0001	1.36
Practice remembering	44	2.07	2.22	23	3.35	2.37	21	0.67	0.71	−5.18	< .0001	1.53
Cue-based reminder	44	0.36	0.36	23	0.45	0.34	21	0.26	0.37	−1.80	0.08	0.54
Praise recall	44	0.31	0.55	23	0.56	0.66	21	0.04	0.12	−3.79	0.001	1.12
MS summary scores
Total amount of MS	44	13.50	8.54	23	18.32	8.83	23	8.23	3.87	−4.98	< .0001	1.48
No. of MS types used	44	4.13	1.24	23	4.85	1.16	23	3.34	0.74	−5.18	< .0001	1.55
MS bundles
MS bundles = 2	42	2.46	1.60	22	3.24	1.58	20	1.60	1.12	−3.84	0.0004	1.20
MS bundles = 3	24	0.73	0.46	18	0.86	0.45	6	0.33	0.10	−4.64	0.0001	1.62
MS bundles = 4 or 5	10	0.49	0.31	8	0.54	0.33	2	0.29	0.06	−1.00	0.35	1.03
MS bundles ≥2	42	2.99	2.13	22	4.14	2.20	20	1.73	1.10	−4.56	0.0001	1.39
MS bundles ≥3	27	0.83	0.69	19	1.05	0.72	8	0.32	0.09	−4.31	0.0004	1.41

Note. M = mean. SD = Standard Deviation.

We also examined the patterns in which therapists used more than one MS strategy at the same time (i.e., MS bundles). Overall, patients in CT+Memory Support received significantly more bundles of MS strategies compared to patients in CT-as-usual at α = .013. Specifically, using two, three, more than two, and more than three MS strategies at the same time were significantly higher in CT+Memory Support compared to CT-as-usual at α = .013. As evident in Table 2, the effect sizes were large. Although the group difference in the mean use of four or five MS bundles was not significant, the effect sizes were in the large range.

Of the total instances of MS bundle use, 81.84% of the MS bundle uses were comprised of two MS strategies at the same time. The most frequent use of MS bundles were attention recruitment and repetition (32.40%), attention recruitment and application (11.80%), repetition and practice remembering (10.11%), application and repetition (6.18%), and attention recruitment and practice remembering (4.49%).

Aim 2

Descriptive statistics of the patient recall outcomes are presented in Table 3. As shown in Table 4, multilevel modeling indicated that the mean use of two or more MS strategies at the same time (i.e., MS bundles ≥2) was associated with improvement in patient’s cumulative recall from baseline/session 7 to post-treatment (β = .28, SE = .14, p = 03). Increased number of MS types used was associated with improvement in the patient’s past session recall from baseline/session 7 to post-treatment (β = .38, SE = .18, p = .04). There were trends suggesting that more total amount of MS (β = .25, SE = .13, p = 06) as well as a greater number of MS types (β = .26, SE = .14, p = 06) used were associated with improvement in cumulative recall from baseline/session 7 to post-treatment. None of the estimates reached statistical significance at the corrected α = .013. However, as evident in Table 4, the effect sizes of MS variables on patient recall were generally in the small-to-medium range during the active treatment phase (from session 7 to post-treatment). The effect sizes of the MS variables on cumulative and past session patient recall were in the small range for changes from session 7 or post-treatment to follow-up.

Table 3.

Descriptive Statistics for Continuous Outcome Variables in the Total Sample.

	Total Sample			CT-as-usual			CT+Memory Support
Measures	N	M	SD	N	M	SD	N	M	SD
Cumulative recall
Session 7	40	8.20	3.90	19	7.84	2.52	21	8.52	4.87
post-treatment	40	9.00	5.11	18	7.94	3.37	22	9.86	6.13
Follow-up	40	8.13	4.84	19	7.58	4.59	21	8.62	5.11
Past session recall
Session 7	40	4.65	2.77	19	4.95	3.01	21	4.38	2.58
post-treatment	40	4.43	3.94	18	3.61	3.53	22	5.09	4.21
Follow-up	40	3.28	3.22	19	3.11	3.31	21	3.43	3.20
IDS-SR scores
at baseline	48	41.19	9.23	23	43.00	9.77	25	39.52	8.55
at post-treatment	42	22.29	11.56	20	25.45	10.83	22	19.41	11.69
at follow-up	41	23.12	13.30	20	24.60	13.70	21	21.71	13.07
GAF scores
at baseline	47	59.51	4.59	22	59.45	5.47	25	59.56	3.75
at post-treatment	41	69.24	8.72	19	66.63	8.39	22	71.50	8.55
at follow-up	41	70.44	11.54	20	68.80	11.98	21	72.00	11.17

Note. M = Mean. SD = Standard Deviation.

Table 4.

Multilevel Model Results for Repeated Outcome Measures

Measures	MS effect at baseline			MS effect on change during treatment phase			MS effect on change from baseline to follow-up			Contrast treatment phase vs. follow-up
	Beta	SE	p	Beta	SE	p	Beta	SE	p	Beta	SE	p
Patient Recall Outcomes
Cumulative recall
Total amount of MS	.03	.15	.82	.25	.13	.06	.11	.13	.41	.14	.13	.28
No. of MS types used	.07	.15	.62	.26	.14	.06	.16	.14	.25	.11	.14	.44
MS bundles ≥ 2	−.04	.15	.79	.28	.13	.03	.13	.13	.31	.15	.13	.24
MS bundles ≥ 3	.29	.23	.20	.17	.18	.36	.08	.18	.68	.09	.18	.62
Past session recall
Total amount of MS	−.05	.15	.73	.23	.18	.20	.08	.17	.65	.15	.17	.40
No. of MS types used	−.12	.16	.46	.38	.18	.04	.17	.18	.35	.21	.18	.26
MS bundles ≥ 2	−.07	.15	.65	.22	.18	.21	.04	.18	.84	.18	.18	.30
MS bundles ≥ 3	.12	.19	.52	.13	.23	.58	−.09	.24	.70	.22	.24	.35
Clinical Outcomes
IDS-SR
Total amount of MS	−.05	.12	.70	−.03	.13	.81	−.11	.13	.39	.08	.13	.54
No. of MS types used	−.03	.13	.82	−.07	.13	.62	−.16	.13	.22	.10	.13	.47
MS bundles ≥ 2	−.06	.12	.64	.004	.12	.98	.02	.12	.89	−.01	.12	.91
MS bundles ≥ 3	−.13	.16	.44	−.03	.15	.85	−.07	.15	.65	.04	.15	.79
GAF
Total amount of MS	−.07	.13	.57	.26	.14	.07	.40	.14	.01	−.14	.14	.33
No. of MS types used	−.08	.14	.58	.35	.15	.02	.40	.15	.01	−.05	.15	.73
MS bundles ≥ 2	−.08	.13	.54	.23	.15	.13	.29	.15	.05	−.06	.15	.66
MS bundles ≥ 3	−.07	.15	.66	−.02	.17	.89	.17	.17	.31	−.19	.17	.25

Note. Beta = standardized regression coefficients. Cumulative recall and past session recall were measured at session 7, post-treatment, and 6-month follow-up. IDS-SR and GAF were measured at pre-treatment baseline, post-treatment, and 6-month follow-up.

^*For patient recall variables, baseline indicates session 7 assessment. For mood outcomes (IDS-SR and GAF), baseline indicates pre-treatment baseline. Baseline or session 7 was the reference group.

Aim 3

Descriptive statistics of the continuous depression outcome variables (IDS-SR and GAF scores) are presented in Table 3. As shown in Table 4, multilevel modeling indicated that the total amount of MS, using different types of MS, or using MS bundles were not significantly associated with changes in the IDS-SR scores. A small effect size was observed for the number of MS types used on reduction in IDS-SR scores from baseline to follow-up (β = −.16, SE = .13, p =.22). For GAF scores, increased number of MS types used was associated with improvement in GAF scores from baseline to post-treatment (β = .35, SE = .15, p = .02). More total amount of MS (β = .40, SE = .14, p = .01), number of MS types used (β = .40, SE = .15, p = .01), and MS bundles ≥2 used (β = .29, SE = .15, p = .048) were associated with improvement in GAF scores from baseline to follow-up. At the corrected α = .013 level, more total amount of MS and number of MS types used were still significantly associated with improvement in GAF score from baseline to follow-up. There was also a trend indicating that the total amount of MS was associated with improvement in GAF scores during active treatment phase (β = .26, SE = .14, p = .07). As evident in Table 4, the effect sizes were in the small-to-medium range.

As shown in Table 5, for binary mood outcomes (i.e., response, remission, and recurrence), logistic regression indicated that a one-unit increase in total amount of MS (i.e., receiving one additional MS strategy per session on average), No. of MS types used (i.e., receiving one additional different type of MS strategy per session on average), MS bundles ≥ 2 and MS bundles ≥ 3 (i.e., receiving one additional MS bundle on average) were associated with 3% (OR = .97, p = .38), 25% (OR = .75, p = .29), 10% (OR = .90, p = 47) and 49% (OR = .51, p =.30) reduction in the odds of being a treatment non-responder at post-treatment, respectively. Though these ORs did not reach statistical significance, the effect sizes were in the small-to-medium range for No. of MS types used and MS bundles ≥ 3 on treatment response. Total amount of MS and No. of MS types used were not associated with remission (i.e., OR’s = 1.00); a one-unit increase in MS bundles ≥ 2 was associated with a 6% (OR = .94, p = .71) reduction in the odds of being a non-remitter at post-treatment. Further, a one-unit increase in total amount of MS, No. of MS types used, MS bundles ≥ 2 and ≥ 3 were associated with a 12% (OR = .88, p = .046), 67% (OR = .33, p = .02), 33% (OR = .67, p = .09) and 59% (OR = .41, p = .31) reduction in the odds of having recurrence at 6-month follow-up, respectively. None of the ORs reached the corrected significant level of .013. Effect sizes for No. of MS types used and MS bundles ≥ 3 on recurrence were in the medium range.

Table 5.

Impact of Memory Support (MS) on Binary Mood Outcomes

MS variables	Non-Response at Post				Non-Remission at Post				Recurrence at FU
	OR	SE	p	95% CI	OR	SE	p	95% CI	OR	SE	p	95% CI
Total amount of MS	0.97	0.04	0.38	0.90–1.04	1.00	0.04	0.94	0.92–1.08	0.88	0.05	0.046	0.78–0.998
No. of MS types used	0.75	0.20	0.29	0.44–1.28	1.00	0.29	1.00	0.56–1.78	0.33	0.15	0.02	0.13–0.82
MS bundles ≥2	0.90	0.13	0.47	0.67–1.20	0.94	0.15	0.71	0.68–1.30	0.67	0.16	0.09	0.42–1.08
MS bundles ≥3	0.51	0.33	0.30	0.15–1.83	nd	nd	nd	nd	0.41	0.36	0.31	0.07–2.32

Note. OR = Odds Ratio. SE = Standard Error. CI = Confidence Interval. Post = Post-Treatment. FU = Follow-Up. nd= no result due to too few observations (less than 10 remitters who had received MS bundles ≥3). Relapse was not analyzed due to the small cell size.

Table 6 presents the mean values and effect sizes for specific MS strategies by the binary mood outcomes. Overall, the effect sizes for most of the specific MS strategies were larger for recurrence and relapse outcomes compared to response and remission outcomes. Of note, despite the finding that categorization was the least commonly used MS strategies, large effect sizes in the expected direction were observed with regards to remission, recurrence, and relapse.

Table 6.

Comparing Means of MS Variables across Mood Outcomes

MS variables	Non-Responders (N=24)		Responders (N=18)			Non-Remitters (N=31)		Remitters (N=11)			Recurrence (N=12)		Non-Recurrence (N=17)			Relapsed (N=2)		Non-Relapse (N=8)
	M	SD	M	SD	d	M	SD	M	SD	d	M	SD	M	SD	d	M	SD	M	SD	d
MS strategies
Attention recruitment	3.49	1.81	3.38	2.16	0.06	3.39	1.77	3.59	2.46	0.09	2.83	1.50	4.21	2.20	0.74	2.88	2.65	3.74	2.74	0.32
Application	1.50	0.97	1.88	1.70	0.27	1.71	1.44	1.52	0.96	0.15	1.56	1.05	1.85	1.77	0.20	0.75	1.06	1.66	0.97	0.89
Evaluation	0.68	0.64	0.84	0.66	0.25	0.80	0.69	0.59	0.50	0.35	0.43	0.54	0.90	0.74	0.73	0.38	0.53	0.67	0.54	0.56
Categorization	0.33	0.40	0.25	0.26	0.26	0.36	0.36	0.12	0.24	0.77	0.14	0.23	0.43	0.40	0.88	0.00	0.00	0.17	0.27	0.87
Repetition	4.64	2.65	5.05	3.41	0.14	4.78	3.06	4.91	2.83	0.04	3.56	2.25	6.04	3.51	0.84	2.88	2.65	5.76	2.72	1.07
Practice remember	1.51	1.56	2.81	2.75	0.58	1.90	2.22	2.51	2.28	0.27	0.98	1.40	3.05	2.89	0.91	1.13	1.59	2.93	2.51	0.86
Cue-based reminder	0.38	0.39	0.37	0.32	0.01	0.39	0.39	0.33	0.26	0.18	0.14	0.23	0.58	0.41	1.31	0.25	0.35	0.34	0.28	0.29
Praise recall	0.19	0.34	0.49	0.73	0.53	0.33	0.59	0.30	0.46	0.06	0.04	0.11	0.61	0.76	1.05	0.00	0.00	0.31	0.48	0.91
MS summary scores
Total amount of MS	12.71	6.95	15.06	10.54	0.26	13.66	8.68	13.88	8.94	0.02	9.69	5.48	17.67	10.83	0.93	8.25	8.84	15.57	9.49	0.80
No. of MS types used	4.04	1.10	4.43	1.33	0.32	4.20	1.17	4.21	1.36	0.00	3.48	0.91	4.78	1.24	1.19	2.96	1.83	4.47	1.28	0.96
MS bundles
MS bundles ≥2	2.78	1.85	3.27	2.56	0.22	2.91	2.17	3.21	2.22	0.13	2.24	1.34	3.90	2.72	0.78	nd	nd	3.25	2.51	nd
MS bundles ≥3	0.69	0.64	0.98	0.76	0.41	0.87	0.78	0.76	0.51	0.17	0.66	0.40	1.07	0.89	0.58	nd	nd	0.82	0.58	nd

Note. M = means. SD = Standard Deviation. d = Cohen’s d. Means by binary outcomes were compared using independent t-test. nd = no data due to few observations.

Aim 4

As shown in Table 7, during the active treatment phase (Pre to Post), the Memory Support Intervention (CT+Memory Support vs. CT-as-usual) exerted a small effect on IDS-SR score (β = −.17, SE = .23, p = .46) and a medium effect on GAF (β = .48, SE = .25, p = .05). The Memory Support Intervention exerted medium effects on cumulative (β = .29, SE = 22, p = .20) and past session recall (β = .57, SE = .37, p = .14). Cumulative and past session recall exhibited small effects on IDS-SR (β = −.14, SE = .14, p = .29; β = −.22, SE = 15, p = .14, respectively) and GAF (p= −.05, SE = .15, p = .74; β = −.21, SE = .17, p = .21, respectively). The effect of the Memory Support Intervention on IDS-SR was reduced by 35% and 24% when controlling for cumulative and past session recall respectively. The effect of Memory Support Intervention on GAF, however, was not reduced when controlling for either cumulative or past session recall (11% and 7% increase respectively). None of the relationships tested reached statistical significance.

Table 7.

Test of Hypothesized Mediators of the Memory Support Intervention

	Fixed effects			Random effects
Criteria	Beta	SE	p	Level 2: between- subject variance	Level 1: within- subject variance
1. Effect of the treatment on the outcome (Path c)
CT+MS vs. CT-as-usual --> Δ IDS-SR	−0.17	0.23	0.46	0.18	0.30
CT+MS vs. CT-as-usual --> Δ GAF	0.48	0.25	0.05	0.11	0.33
2. Effect of the treatment on the mediator (Path a)
CT+MS vs. CT-as-usual --> Δ Cumulative recall	0.29	0.22	0.20	0.66	0.23
CT+MS vs. CT-as-usual --> Δ Past session recall	0.57	0.38	0.14	0.26	0.71
3. Relation between change in mediator and outcome (Path b)
Δ Cumulative recall --> Δ IDS-SR	−0.14	0.14	0.29	0.17	0.30
Δ Past session recall --> Δ IDS-SR	−0.22	0.15	0.14	0.16	0.30
Δ Cumulative recall --> Δ GAF	−0.05	0.15	0.74	0.10	0.38
Δ Past session recall --> Δ GAF	−0.21	0.17	0.21	0.14	0.34
4a. Effect of treatment on outcome with control for change inmediator (Path c)
CT+MS --> Δ IDS-SR | Cumulative recall	−0.11	0.25	0.66	0.16	0.29
CT+MS --> Δ IDS-SR | Past session recall	−0.13	0.25	0.60	0.15	0.30
CT+MS --> Δ GAF | Cumulative recall	0.54	0.27	0.05	0.11	0.34
CT+MS --> Δ GAF | Past session recall	0.52	0.26	0.04	0.15	0.30
				t	Δ b (%)
4b. Effect of treatment on outcome, with control for change in mediator (Path c’), significantly reduced or eliminated compared with effect of the treatment on the outcome (Path c)
CT+MS --> Δ IDS-SR | Cumulative recall				−1.54	35%
CT+MS --> Δ IDS-SR | Past session recall				−0.97	24%
CT+MS --> Δ GAF | Cumulative recall				−1.14	−11%
CT+MS --> Δ GAF | Past session recall				−0.72	−7%

Note. Beta = standardized regression coefficient. t < −2.01 indicates statistical significance at two-tailed α = 0.05. | indicates controlling for. Note that we omitted Criteria 5 of Stice et al., 2007 because no IDS-SR was available at session 7 and Criteria 1–4 are more typically conducted in the literature. Freedman & Schatzkin’s (1992) method was used to calculate t-statistic. Paths a, b, c, and c’ are indicated in the diagrams in the Supplemental Material.

Discussion

The present study examined the extent to which therapist use of memory support (MS) strategies and bundles is associated with patient recall and cognitive therapy (CT) outcomes for depression. Given that these findings are based on a pilot RCT underpowered for hypothesis testing, the following results are only tentative and replication in a large-scale, confirmatory trial is warranted. The first aim was to compare the use of MS in CT+Memory Support versus CT-as-usual. Consistent with our hypothesis, relative to those in CT-as-usual, participants in the CT+Memory Support condition received a significantly higher amount of specific MS strategies (for 6 out of 8 MS strategies) and total MS, more different types of MS, as well as more MS bundles. These results serve as a manipulation check and suggest that the Memory Support Intervention effectively increases the amount of MS delivered. Consistent with the recommendations of optimal implementation of CT (Beck, 1979), MS strategies are also used in CT-as-usual. Of note, there was variability in how frequent specific MS strategies are used in both treatment conditions. Repetition and attention recruitment were most frequently used in both treatment conditions, perhaps because they are relatively easy to deliver. Practice remembering had the largest effect size between treatment conditions such that CT+Memory Support had significantly higher uses of practice remembering. There was no treatment group difference for categorization and cue-based reminder, possibly because they were the least frequently utilized MS strategies in both treatment groups despite that they are important for improving outcomes (discussed in more detail below). Finally, though therapists delivering CT+Memory Support were not specifically trained to use more than one MS strategy at the same time, therapists did use bundles of MS strategies and mainly used two MS strategies together (rather than 3, 4, or 5 MS strategies together).

The second aim was to examine the extent to which the level of MS therapists used are associated with patient recall of treatment contents. Consistent with the hypothesis, we observed small-to-medium effect sizes, suggesting that MS might be associated with modest improvement of patient recall in a larger sample. Specifically, based on the estimated effect sizes (note that none of these effect sizes reached statistical significance, perhaps given the small sample size), delivering a higher amount of MS, using more different types of MS, and using two MS strategies simultaneously (MS bundles ≥ 2) were associated with a small (but non-trivial) improvement in the patient’s cumulative recall (i.e., of all treatment points since the beginning of the treatment) and past session recall (i.e., treatment points from the most recent session) during the active treatment phase. Prior literature has established that patient recall of treatment contents is poor and that poor recall is related to poor treatment adherence and outcomes (e.g., Lee & Harvey, 2015). These finding suggests that higher levels of MS during treatment might be related to better patient memory for treatment contents, which has been proposed as a putative key component in the pathway to improving adherence and outcomes (Harvey et al., 2014).

Additionally, the Patient Recall Task, which is a free-recall task, is a conservative measure of patient recall. This task has been validated against MS measures (Lee et al., 2015). However, cognitive experiments show that free-recall tasks result in less information recalled compared to recognition and cued-recall tasks (e.g., Hart, 1967; Tulving & Pearlstone, 1966). In future studies, it is recommended that alternative measures of patient recall of treatment contents be used (e.g., recognition and cued-recall task of treatment contents, assessment of patients’ ability to apply principles/skills learned during treatment in hypothetical scenarios). An interesting possibility to explore in future research is whether patient recall of treatment contents is better among patients who experienced relapse or recurrence of depression than those who remained in remission. The literature on mood-congruent memory bias suggests that memory retrieval is enhanced when the current mood is congruent with the mood during the learning/memory episode (e.g., Barry, Naus, & Rehm, 2004). Therefore, it is possible that patients who experience relapse/recurrence may perform better on patient recall task because their depressed mood at the time of recall matches the mood when memory for treatment was form.

The third aim was to examine the extent to which the level of MS therapists used are associated with better outcome for depression. MS variables were not significantly associated with continuous self-reported depression symptoms; there was only a small effect of no. of MS types used on reduction in depression symptoms from baseline to follow-up. For binary depression outcome, based on the small to medium effect sizes observed (note that none of these effect sizes reached statistical significance, perhaps given the small sample size), greater MS was associated with better treatment response and recurrence, but not remission. Specifically, though not reaching statistical significance, a higher total amount of MS, more different types of MS, and more MS bundles were associated with small to medium reduction in the risk of experiencing non-response at post-treatment and recurrence of depression 6-months after the treatment. A visual inspection of the descriptive statistics in Table 6 suggests that specific MS strategies may differentiate patients who experienced recurrence or relapse versus those who did not with large effect sizes, such that patients who did not have recurrence or relapse received greater amounts of almost all MS strategies. Further, based on both effect sizes and statistical significance, higher total amount of MS, more different types of MS, and more MS bundles were associated with improvement in functioning during the entire study period, from baseline to follow-up. Interestingly, the levels of MS delivered were unrelated to baseline IQ (r’s = −.10 ~ .12, p’s > .05) or GAF scores (r’s = −.07 ~ −.12, p’s > .05), suggesting that baseline cognitive abilities and functioning did not influence the amount of MS therapist delivered.

The fourth aim was to test whether patient recall of treatment contents mediated the effect of Memory Support Intervention on the clinical outcome. Overall, preliminary results based on the effect size estimation provided mixed support for the hypothesis. On the one hand, there is a glimpse of support for the hypothesis that the Memory Support Intervention may partially affect the IDS-SR outcome by improving patient recall (i.e., effect sizes for criteria 1–4 in Table 7 are all in the expected direction), suggesting that this mediation path might be evident in a larger sample. On the other hand, the hypothesis that patient recall mediates the effect of Memory Support Intervention on GAF was not fully supported (only criteria 1–3 in Table 7 are supported based on the estimated effect sizes). Although there was some indication that past session recall exhibited a small effect on GAF, adding patient recall did not reduce but instead slightly increased the effect of the Memory Support Intervention on GAF (criteria 4 not supported). Note that we were not able to test for the temporal sequencing of whether mediator changes before outcome (criteria 5 in Stice et al., 2007) in the current study and none of the relationships tested reached statistical significance, perhaps due to the small sample size of this pilot study. Given these preliminary results, replication and further examination of these constructs in a large-scale, confirmatory trial is needed.

There are several implications. First, therapists can be successfully trained to proactively and strategically deliver MS above and beyond the MS already embedded in CT-as-usual. Therapists appear to be mastering the use of the following MS strategies: attention recruitment, repetition, practice remembering, and application. These are the top four most frequently used MS strategies in both treatment conditions, and the combinations of these specific strategies represent the most frequent use of MS bundles (e.g., attention recruitment + repetition, attention recruitment + application). However, improvements in therapist training and supervision may be needed to increase the use of the four other MS strategies, namely categorization, cue-based reminders, praise recall, and evaluation. These four strategies are used infrequently, even in CT+Memory Support - on average less or equal to one instance of utilization per session. However, all four showed medium-to-large effect sizes comparing patients who experienced recurrence or relapse at 6-month follow-up versus those who did not experience recurrence or relapse. In addition, for categorization, there is prior experimental evidence suggesting that the use of categorization improves patient recall and treatment adherence (e.g., Ley, 1979). Together, these results suggest that future training for the Memory Support Intervention could benefit from incorporating more specific instructions and examples to help therapists use more categorization, cue-based reminder, praise recall, and evaluation.

Second, greater total amount of MS and a greater number of different types of MS delivered appear at least equally important for improving outcomes. Using more different types of MS appears to have even stronger effects on certain aspects of patient recall (e.g., past session recall during treatment phase) and clinical outcome (e.g., response, recurrence) than total amount of MS. Therefore, therapists should be encouraged to increase the amount of MS and diversity of MS strategies delivered above and beyond typical use of MS in treatment-as-usual. Furthermore, though therapists in the pilot study were not specifically trained to use MS bundles, delivering MS bundles appears to be a powerful way to provide MS for patients. Future development of the Memory Support Intervention may include specific training on how to deliver MS bundles and perhaps formally examine the optimal use of MS bundles (e.g., specific combinations, numbers of MS used).

There are several limitations. First, due to the pilot nature of this study, the sample size is small and the study is purposefully underpowered to achieve statistical significance. The rationale behind using a small sample was based on recommendations that testing for logistics/feasibility and an initial signal using pilot studies is the first step in the treatment development phase prior to conducting larger-scale clinical trials (e.g., Anderson & Prentice, 1999; Craig et al., 2008). Nonetheless, caution is recommended when interpreting the results of pilot studies (Arain, Campbell, Cooper, & Lancaster, 2010; Kistin & Silverstein, 2015). Outcomes for which the limited power may be problematic include: 1) the effect sizes for MS variables on treatment response did not reach statistical significance, but were suggestive of possible protective effects against non-response and therefore needs further confirmation; and 2) effect sizes of MS variables on patient recall at during treatment phase also did not reach statistical significance, but were suggestive of positive impact of MS on patient recall.

Second, we recommend caution when interpreting the results related to GAF. While the GAF has been a standard and widely used tool to assess patient’s global functioning (Endicott, Spitzer, Fleiss, & Cohen, 1976; Piersma & Boes, 1997) and its use has been empirically supported (e.g., Pedersen & Karterud, 2012; Startup, Jackson, & Bendix, 2002), there have been concerns about the reliability evidence of the GAF particularly in non-research settings (e.g., Grootenboer et al., 2012). Though the inter-rater reliability on the GAF for the current sample is not available, our assessors were licensed clinicians, graduate students in clinical psychology, and trained research assistants, all under the supervision of licensed clinical psychologists. Future studies using alternative assessment of functioning/disability (e.g., the World Health Organization Disability Assessment Schedule [WHODAS 2.0]) should be conducted to replicate the results from the present study.

Third, the present results are based on one specific therapy (CT) for one specific disorder (MDD). The extent to which these findings are generalizable to other types of treatment or other treatment modalities (e.g., internet-delivered, group) are yet to be established. However, we draw attention to the relevance of the present finding for the emerging literature on the internet-delivered interventions (Andersson, 2016; Andersson, Cuijpers, Carlbring, Riper, & Hedman, 2014; Nordgren et al., 2014). These interventions are comprised of text presented on web pages and often involve on-going email support, which are ideal platforms to provide MS and examine the effects of MS. Additionally, as group-delivered interventions often involves didactics and is less tailored than individual therapy, implementing MS may be a method to bolster memories for group-delivered intervention.

Finally, in this study we were unable to determine which specific MS strategy is most effective for improving patient recall and treatment outcome. A valuable next step would be to conduct experimental manipulations in an appropriate platform (e.g., computerized CT) so that each MS strategy or bundles of MS strategies can be experimentally manipulated, thereby providing evidence as to how each MS strategy or MS bundle influences patient recall and clinical outcome. Additionally, it is important for future studies to examine potential treatment mediators (e.g., patient adherence to treatment) or moderators (e.g., mood medication, cognitive abilities) that were not tested in this study. Future research should also examine whether the Memory Support Intervention may be particularly helpful for individuals with cognitive impairment. Yet another future direction includes long-term effects of MS on depression outcome such as relapse beyond 6 months after treatment completion.

In sum, this study is among the first investigations demonstrating that therapists providing MS strategies and bundles during a psychosocial treatment may be a pathway to improving treatment outcome. Results from this pilot study suggest that delivering MS strategies and bundles may be an inexpensive tool to enhance the effectiveness of one of the most commonly used psychosocial interventions, CT for depression. Further evaluating the effect of MS in a large-scale, confirmatory trial is necessary to support the present study findings. Next steps in this line of research are needed to determine whether the Memory Support Intervention also has potential to be integrated into other treatment types (e.g., other evidenced-based psychosocial treatments, physician visits), and to clarify the optimal ways of delivering MS strategies and bundles.

Public Health Significance.

Preliminary evidence from the pilot RCT suggests that memory support (MS) strategies and bundles can potentially improve patient recall of treatment contents and enhance treatment outcome, though larger-scale studies are needed to confirm the present study findings.

Appendix

Memory Support Strategies

(from Lee et al., 2015)

Attention Recruitment

Involves the treatment provider using expressive language that explicitly communicates to the patient that a treatment point is important to remember (e.g., “if there is one thing I would like you to remember in ten years time, it is this skill” or “this is a key point to remember”), or multimedia/diverse presentation modes (e.g., handouts, poems, songs, note taking, role-playing, imagery, using a white board) as a means to recruit the patient’s attention.

Categorization

Involves explicit effort by the treatment provider to work with the patient to group treatment points discussed into common themes/principles (e.g., “Let’s create a list of ways we can work on waking up at the same time each morning.”).

Evaluation

Involves the treatment provider working with the patient to (a) discuss the pros/cons of a treatment point (e.g., “What would be some advantages/disadvantages of waking up at the same time each morning?”); or (b) use comparisons to compare a new treatment point to an existing or hypothetical alternative (e.g., “How would this new strategy of exercising more compare to your current habit of lying in bed all day when you are feeling depressed?”).

Application

Involves the treatment provider working with the patient to apply a treatment point to past, present, or future (real or hypothesized) scenarios (e.g., “Can you think of an example in which you might try this new method of coping to deal with your stress at work?”).

Repetition

Involves the treatment provider restating, rephrasing, or revisiting information discussed in treatment (e.g., “in other words,” “as we talked about earlier,” or “in sum”).

Practice Remembering

Involves the treatment provider facilitating the patient to regenerate, restate, rephrase, and/or revisit a treatment point (e.g., “Can you tell me some of the main ideas you’ve taken away from today’s session?).

Cue-Based Reminder

Involves the treatment provider helping the patient develop new or existing cues (e.g., colored wrist bands, reminder text messages/phone calls/e-mails, smart phone apps, acronyms, rhymes, and other mnemonics) to facilitate memory for treatment points.

Praise Recall

Involves the treatment provider rewarding the patient for successfully recalling a treatment point (e.g., “It’s really great that you remembered that point!”) or remembering to implement a desired treatment point (e.g., “I’m so glad you remembered to step back and look at the evidence.”).