Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Feb 22.
Published in final edited form as: Behav Res Ther. 2021 Dec 31;149:104029. doi: 10.1016/j.brat.2021.104029

Memory and Learning for Sleep and Circadian Treatment in Serious Mental Illness Treated in a Community Mental Health Setting

Nicole B Gumport 1, Allison G Harvey 1
PMCID: PMC10883147  NIHMSID: NIHMS1965134  PMID: 34995953

Abstract

Objective:

Existing research has demonstrated that patient memory and learning of treatment contents are poor and poorer learning is associated with worse treatment outcome. Most prior studies have included individuals from only a single diagnostic group, offer limited data on possible contributors to poor memory and learning, and have included small samples recruited in university settings. This study sought to describe patient recall of treatment contents, describe patient learning of treatment contents, examine contributors to patient recall and learning of treatment contents, and examine the association of patient recall and learning of treatment contents with treatment outcome.

Methods:

Adults with serious mental illness and sleep and circadian dysfunction (N=99) received the Transdiagnostic Intervention for Sleep and Circadian Dysfunction in a community mental health setting. Measures of recall, learning, age, years of education, symptom severity, and treatment outcome were collected at post-treatment and 6-month follow-up.

Results:

Recall and learning were poor, fewer years of education was associated with worse recall and learning, and recall and learning were not associated with treatment outcome.

Conclusions:

The findings offer evidence that poor patient memory for, and learning of, treatment contents extends to community settings and are transdiagnostic concerns.

Keywords: memory, learning, serious mental illness, sleep, circadian

Memory for the contents of a treatment session is poor. In the physical health literature, patients recall approximately one third of the recommendations of a physician visit (Bober, Hoke, Duda, & Tung, 2007; Jansen et al., 2008; Laws, Lee, Taubin, Rogers, & Wilson, 2018). In the mental health literature, patients with insomnia forget about two thirds of treatment recommendations, with recall as low as 13% for some recommendations (Chambers, 1991). More recently, following the receipt of treatment for insomnia, patients with bipolar disorder recalled 36% of recommendations (Lee & Harvey, 2015). In a study of couples treatment, recall was as low as 3% for some recommendations and 50% of patients could not recall any treatment skills (Hahlweg & Richter, 2010). Taken together, these findings are concerning for two reasons. First, recent research indicates that poor memory for treatment is associated with worse treatment outcome and lower adherence in studies of depression treatment (Dong, Zhao, Ong, & Harvey, 2017; Harvey, Lee, et al., 2016; Lee & Harvey, 2015; Zieve, Dong, & Harvey, 2019). Second, as many evidence-based treatments focus on the presentation of novel skills (Hundt, Mignogna, Underhill, & Cully, 2013), it seems unlikely that patients will use skills presented during treatment if they are unable to remember them.

Learning of treatment contents is also poor. For example, in a study of computer-based treatment for depression, only 50–65% of patient thoughts of treatment content were accurate and less than half of patient applications of treatment content were accurate (Gumport, Williams, & Harvey, 2015). Emerging evidence also indicates that learning is associated with treatment outcome. Using these same measures in a randomized controlled trial of treatment for depression, patients’ accurate thoughts of, and applications of, treatment contents were significantly associated with better treatment outcome at post-treatment (Gumport, Dong, Lee, & Harvey, 2018).

While the field has made progress in understanding memory and learning for psychosocial treatment contents, gaps still remain. First, most prior research has focused on memory and learning for treatment among single diagnostic groups, such as depression (Gumport et al., 2018, 2015), bipolar disorder (Lee & Harvey, 2015), or insomnia (Chambers, 1991). However, problems with declarative, episodic, working, and prospective memory are common across many mental illnesses including depression (Gotlib & Joormann, 2010; Hertel, 1998), bipolar disorder (Torres, Boudreau, & Yatham, 2007), anxiety (Airaksinen, Larsson, & Forsell, 2005), schizophrenia (Boyer, Phillips, Rousseau, & Ilivitsky, 2007; Henry, Rendell, Kliegel, & Altgassen, 2007; Saykin et al., 1991), posttraumatic stress disorder (Isaac, Cushway, & Jones, 2006), and substance use (Rendell, Mazur, & Henry, 2009; Serper et al., 2000). Second, samples were small, ranging from 20–48 participants. Third, there is limited prior research evaluating patient factors that may contribute to worse memory or learning of treatment contents. For example, healthy aging is associated with general declines in memory functioning (Glisky, 2007; Grady & Craik, 2000; Li et al., 2016; Reuter-Lorenz, Festini, & Jantz, 2016), although greater education has been shown to be a protective factor for age-related memory loss (Angel, Fay, Bouazzaoui, Baudouin, & Isingrini, 2010; Cabeza et al., 2018). Additionally, at times, greater symptom severity has been associated with greater memory problems in serious mental illness (SMI; McDermott & Ebmeier, 2009; Reichenberg et al., 2009), although there are non-replications (e.g., Woon, Farrer, Braman, Mabey, & Hedges, 2017). A greater understanding of these contributors to patient memory and learning of treatment contents may allow for the targeted dissemination of interventions known to improve patient memory for treatment (e.g., the Memory Support Intervention, Harvey, Lee, et al., 2016). Fourth, the prior studies that focused on mental health were conducted in university research settings, which limits the generalizability of findings to routine care settings.

The present study focuses on a sample of adults with SMI and sleep and circadian dysfunction in a community mental health setting. SMI was operationalized according to Public Law 102–321 and previous research (Wang, Demler, & Kessler, 2002) as the presence, for at least 12 months, of at least one Diagnostic and Statistical Manual-defined (American Psychiatric Association, 2013) mental disorder that leads to substantial inference with major life activities, such as depression, bipolar disorder, schizophrenia spectrum and other psychotic disorders, posttraumatic stress disorder, and substance use disorders. Sleep and circadian problems such as insomnia, hypersomnia, advanced and delayed phase, and irregular schedules are often are comorbid with SMI (Baglioni et al., 2016). These problems regularly persist after treatment is provided for SMI (López, Lancaster, Gros, & Acierno, 2017) and can predict the onset and worsening of SMI symptoms (Hertenstein et al., 2019). Also, independent of SMI, sleep and circadian problems impair memory and learning processes (Walker & Stickgold, 2006; Yoo, Hu, Gujar, Jolesz, & Walker, 2007). Community mental health settings are major, publicly-funded providers for SMI in the United States. They offer services for the most socioeconomically underserved members of the community (Kim et al., 2020). Within these settings, individuals with a SMI often experience high rates of comorbidity and complexity. Hence, data on how individuals diagnosed with SMI and sleep and circadian dysfunction, who receive care in community mental health settings, recall and learn treatment contents may offer relatively generalizable findings given the focus on a representative, real-world sample compared to previous research. In sum, to the best of our knowledge, no prior studies have examined patient memory and learning in this population or in this setting.

The overall goal of this study is to examine memory and learning for the contents of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C; Harvey & Buysse, 2017) among adults with a SMI in a community mental health setting at post-treatment and 6-month follow-up. Initial results of a clinical trial demonstrated that TranS-C improves sleep and SMI outcomes relative to usual care at post-treatment and 6-month follow-up (Harvey et al., under review). The first aim is to describe the extent of patient recall of treatment contents. We expected that patients would accurately recall approximately one third of treatment contents, based on prior work conducted in university settings (e.g., Lee & Harvey, 2015). The second aim was to describe the extent of patient learning of treatment contents. Three hypotheses were tested based on prior research (Gumport et al., 2018, 2015). We expected that (a) patients will report thinking about treatment contents one to two times in the past week, (b) patients will report applying treatment contents one to two times in the past week, and (c) approximately 60% of thoughts would be accurate and that under 50% of applications would be accurate. The third aim was to examine contributors to patient memory and learning of treatment contents. Based on prior studies (Angel et al., 2010; Harvey et al., 2014; Salthouse, 2009), we expected that greater symptom severity, older age, and fewer years of education would be associated with poorer memory and learning for treatment contents at post-treatment and at 6-month follow-up. The fourth aim was to examine the association of patient recall and learning of treatment content with treatment outcome. The hypothesis tested was that better recall and better learning of treatment contents would be associated with improved treatment outcome at post-treatment and 6-month follow-up (Gumport et al., 2018, 2015; Harvey, Lee, et al., 2016; Lee & Harvey, 2015).

Methods

Participants

The 99 participants included in this study were drawn from a National Institute of Mental Health-funded randomized controlled trial that included adults who met criteria for SMI and sleep and circadian disturbance and who were recruited from multiple sites within Alameda County Behavioral Health Care Services (ACBHCS; Alameda County, CA, USA) (Harvey, Hein, et al., 2016). The primary trial from which the data were drawn included 121 participants. However, 22 participants were excluded from the present study as they either did not complete the post-treatment and 6-month follow-up (n = 19) or the memory and learning measures at post-treatment and at 6-month follow-up were missing from their assessments due to administrative error (n = 3). Participant characteristics are displayed in Table 1.

Table 1.

Participant Characteristics

Characteristic Mean or N SD or %
Age (years) 47.38 12.00
Female 51 51.52
Race
 African-American or Black 41 41.41
 American Indian/Alaskan Native 2 2.02
 Asian 7 7.07
 Caucasian 37 37.37
 Native Hawaiian or Other Pacific Islander 2 2.02
 Mixed Race 6 6.06
 Not specified 4 4.04
Ethnicity
 Hispanic or Latino 14 14.14
 Not Hispanic or Latino 84 84.85
 Not specified 1 1.01
Employment
 Full-time 2 2.02
 Part-time 12 12.12
 Unemployed 79 79.80
 Other 5 5.05
 Missing 1 1.01
Education (years) 13.83 3.68
Highest level of education completed
 High school or below 28 28.28
 Vocational school 10 10.10
 Some college or completed college 56 56.57
 Graduate school 5 5.05
Annual personal income ($) 11254.10 7651.56
Annual household income ($) 23016.52 22682.56
Receiving government assistance 95 95.96
DSM diagnoses at pre treatment1
 Schizophrenia spectrum disorder 51 51.52
 Bipolar disorder2 25 25.25
 Major depressive disorder3 21 21.21
 Any anxiety disorder4 47 47.47
 Obsessive compulsive disorder4 19 19.19
 Post-traumatic stress disorder 13 13.13
 Substance use disorder 30 30.30
 Psychotic symptoms/features5 73 73.74
Sleep and circadian diagnoses at pre treatment1
 Insomnia 80 80.81
 Hypersomnolence (provisional)6 25 25.25
 Delayed sleep phase 4 4.04
 Advanced sleep phase 2 2.02
 Irregular sleep-wake disorder 1 1.01
 Restless leg syndrome 5 5.05
Periodic limb movements (provisional)7 4 4.04
Open in a new tab
1

Participants could meet diagnostic criteria for multiple problems.

2

Bipolar disorder with psychotic features is listed in this category, not in the schizophrenia spectrum or psychotic disorders category.

3

Depression with psychotic features is listed in this category, not in the schizophrenia spectrum or psychotic disorder category.

4

No participants were solely diagnosed with an anxiety disorder or obsessive compulsive disorder – all also received a comorbid schizophrenia spectrum, bipolar disorder, major depressive disorder, post-traumatic stress disorder, and/or substance use disorder diagnosis.

5

Psychotic symptoms/features includes depression with psychotic features, bipolar disorder with psychotic features, a schizophrenia spectrum or psychotic disorder diagnosis.

6

A hypersomnolence diagnosis requires a multiple sleep latency test (American Academy of Sleep Medicine, 2014).

7

A periodic limb movement diagnosis requires a polysomnography assessment (American Academy of Sleep Medicine, 2014).

Individuals were eligible if they met the following inclusion criteria: (a) 18 years of age or older; (b) English language fluency; (c) presence of at least one DSM-5 mental disorder for 12 months; (d) having a guaranteed bed to sleep in for the next three months; (e) receiving care for SMI at ACBHCS and consenting to regular communication between the research team and their ACBHCS psychiatrist and/or case manager; and (f) presence of one or more of the following problems, on three or more nights per week, for three months assessed via the Sleep and Circadian Problems Interview: taking 30 minutes or longer to fall asleep, waking in the middle of the night for 30 minutes or longer, obtaining less than six hours of sleep per night, obtaining nine or more hours of sleep per 24 hour period (i.e., nighttime sleep plus daytime napping), maintaining a bedtime later than 2:00am, or having more than 2.78 hours of variability in sleep-wake schedule across one week.

Individuals were excluded if they met any of the following criteria: (a) presence of an active and progressive physical illness or neurological degenerative disease and/or substance use that would make participation in the study unfeasible; (b) current serious suicide risk or homicide risk (both assessed by study staff and a case manager or psychiatrist); (c) night shift work two or more nights per week in the past three months; (d) pregnancy or breastfeeding; or (e) unable or unwilling to participate in and/or complete the pretreatment assessments.

Treatment

Treatment was delivered by nine therapists hired by the University of California, Berkeley system. The therapists traveled between the ACBHCS clinic sites to deliver treatment. Clinicians attended a one-day workshop, used a treatment manual, and received weekly supervision.

TranS-C (Harvey & Buysse, 2017), which was administered in eight weekly 50-minute sessions, is grounded in basic sleep and circadian science and the sleep health framework (Buysse, 2014). TranS-C is derived from several sources. It draws from cognitive behavioral therapy for insomnia, which is the frontline treatment for insomnia (CBT-I) (Edinger et al., 2021).There is a great deal of literature indicating the efficacy of CBT-I for SMI (Morin et al., 2006; Qaseem et al., 2016; Riemann et al., 2017). TranS-C also incorporates principles from Interpersonal and Social Rhythms Therapy (Ehlers, Frank, & Kupfer, 1988), chronotherapy (Wirz-Justice, Benedetti, & Terman, 2009), and motivational enhancement (Miller & Rollnick, 2002). TranS-C includes four cross-cutting modules featured in every session (functional analysis, education, behavior change and motivation, and goal-setting), four core modules that apply to the vast majority of participants (establishing regular sleep-wake times including learning a wind-down and wake-up routine, improving daytime functioning, correcting unhelpful sleep-related beliefs, and maintaining behavior change), and seven optional modules used less commonly, depending on the needs of each participant (improving sleep efficiency, reducing time in bed, dealing with delayed or advanced phase, reducing sleep-related worry/vigilance, promoting compliance with CPAP/exposure therapy for claustrophobic reactions to CPAP, negotiating sleep in a complicated environment, and reducing nightmares). Core and optional modules can be delivered in any sequence and are customized to the participant based on their presentation and goals for treatment.

Measures

Patient Recall Task.

The Patient Recall Task (Lee & Harvey, 2015) is a free recall task. Participants were asked to “Take a moment to think back to your sleep coaching. Can you tell me everything you have learned? We have 5 minutes for this task so please take your time.” Participants responses were recorded and then transcribed. In the few cases where participants declined audio recording, the trained assessor wrote notes. Trained coders evaluated the transcript of each Patient Recall Task. The transcripts of responses were coded for “treatment points.” A treatment point is defined as a main idea, principle, or experience that the treatment provider wants the patient to remember or implement as part of the treatment (Lee & Harvey, 2015). Each treatment point was scored based on a list of 31 possible treatment points that were drawn from a review of the TranS-C treatment manual (“Correctly Recalled”). In addition, recalled items were categorized within the list of 31 possible treatment points. Coders also coded patients responses for inaccurate items (“Incorrectly Recalled”), or items that were inaccurate yet related to TranS-C content (e.g., adults need 4 hours of sleep/night). For a list of all possible treatment points, see the first column of Table 2.

Table 2.

Accurate recall of each treatment point at post-treatment and 6-month follow-up

Treatment Point Post-Treatment (N=92) 6-Month Follow-Up (N=79)
Frequency % Frequency %
1. Consistent bedtime: going to bed at about the same time each night or same bedtime on weekdays and weekends. 33 35.87 25 31.65
2. Consistent waketime: waking up at about the same time each day or same waketime on weekdays and weekends. 30 32.61 26 32.91
3. Early waketime: waking up early or not sleeping in. 11 11.96 4 5.06
4. Social jetlag: going to bed and waking up about the same time on weekends relative to weekdays 4 4.35 0 0
5. Sufficient sleep: 7–8 hours of sleep per night. Must refer to specific amount. 6 6.52 6 7.59
6. Moving bed or wake time by 20–30 minutes each week. 2 2.17 0 0.00
7. It isn’t possible to compensate for lost sleep (“sleep debt”) by sleeping in. 0 0.00 0 0.00
8. Wind down routine (30–60 minutes) before bedtime. May also refer to bedtime routine or sleep routine Examples of wind down routine: showering before bed, drinking decaf tea, reading a book, not watching TV, drawing, knitting, puzzles 38 41.30 19 26.03
9. Not napping or avoiding naps. If napping does occur, they are best when short (less than 30 minutes) and earlier in the day (late morning or early afternoon). 29 31.52 20 25.32
10. Any mention of the circadian rhythm or internal body clock. May also refer to the suprachiasmatic nucleus (SCN) as the central conductor of sleep. 7 7.61 3 3.80
11. Any mention of sleep homeostasis, sleep appetite, or sleep drive. 2 2.17 1 1.27
12. Reducing light exposure in the evening or importance of darkness. 29 31.52 24 30.38
13. Have an “electronic curfew” such as turning off cell phone or computer or TV at a certain time. 25 27.17 20 25.32
14. Melatonin or any reference to hormones that help you fall asleep 21 22.83 7 8.86
15. RISEUP or wakeup routine. RISEUP acronym: Refrain from snoozing, Increase activity upon awakening, Shower or wash face and hands (with cold water), Expose yourself to sunlight, Upbeat music in the morning, Phone a friend, or any mention of social activity in the morning. 38 41.30 23 29.11
16. Being active or doing activities or “generating energy” when feeling tired. May also be referred to as “energy experiment” or “energy generating experiment” 8 8.70 3 3.80
17. I use techniques to reduce worry or thinking interfering with my sleep via savoring, worry time earlier in the day, journaling, gratitude practice. May also refer to “relax the mind” as the overarching concept 39 42.39 20 25.32
18. I get out of bed if I am not able to sleep (within 20–30 minutes). May also refer to sleep restriction or stimulus control. May also refer to not trying to force self to sleep (“trying to fall asleep”). 12 13.04 10 12.66
19. I keep my bed for sleeping only (I do not work in bed or watch TV in bed). 14 15.22 12 15.19
20. Caffeine is found in coffee, soda, and energy drinks. Some medications like cold medicine also have caffeine. Can also refer to avoiding caffeine in the afternoon or evening. 31 33.70 22 27.85
21. Alcohol and other substances (e.g., tobacco, cocaine) can impact my sleep. I avoid these in the evening. 6 6.52 6 7.59
22. I make a point of trying to eat healthy. Note, can include any mention of diet, appetite, or hunger hormones (e.g., ghrelin and leptin), or eating on a regular schedule. 8 8.69 8 10.13
23. Referring to health or mentioning health and sleep. 2 2.17 1 1.27
24. Getting enough sleep can affect your health. Your immune system is influenced by the amount of sleep you get. Pain levels are influenced by the amount of sleep you get. 0 0.00 1 1.27
25. Not getting enough sleep can make it harder to remember things (sleep as related to cognitive functioning) 2 2.17 0 0.00
26. Sleep can improve your physical appearance (or make you more attractive) 0 0.00 0 0.00
27. Sleep is divided into stages and includes REM (rapid eye movement) and non-REM (NREM) sleep. 3 3.26 1 1.27
28. Avoiding going to sleep with the TV or radio on. 9 9.78 3 3.80
29. Sleep inertia: it is normal to feel groggy for the first hour upon waking up. 9 9.78 2 2.53
30. Changing the details and then repeating/rehearsing my dreams during the daytime can reduce my nightmares. 0 0.00 0 0.00
31. I keep my bedroom comfortable for sleep: cool, dark, quiet. May mention wearing earplugs or using a sound machine to block out sounds. May mention using an eye mask to keep dark. May mention asking roommate not to speak to them in the middle of the night or asking roommate to turn off lights too. 19 20.65 11 13.92
Open in a new tab

Learning measures.

Thoughts.

Thoughts about treatment contents was adapted from prior studies (Gumport et al., 2018, 2015). Thoughts about treatment contents were assessed via a questionnaire that asked the participant, “In the last week, did information discussed with your sleep coach come to mind?,” and “If yes above, how many times?” and “What came to mind?” To determine if thoughts accurately reflected the treatment content, responses to “What came to mind?” were coded for treatment points. This data was collected at post-treatment and 6-month follow-up.

Application.

Application of treatment contents was adapted from prior studies (Gumport et al., 2018, 2015). Application of treatment contents was assessed via a questionnaire that asked the participant “Did you get to apply anything discussed with your sleep coach in the past week?”, and “If yes, what did you apply?” These responses were coded for accuracy using the method described above. This data was collected at post-treatment and 6-month follow-up.

Advice.

Advice about treatment contents was assessed via a questionnaire asking, “If you had a close friend with sleep problems, what advice would you give him/her?” Participant responses were coded for accuracy. This data was collected at post-treatment and 6-month follow-up.

Contributors to treatment outcome.

Demographic characteristics.

A demographics form, which assessed age and years of education, was completed by participants at the baseline assessment.

DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult (DSM-5 Cross-Cutting Measure).

The DSM-5 Cross-Cutting Measure is used as a measure of disorder-focused symptoms. It contains 23 questions that assess symptoms in the most recent two weeks across 13 psychiatric domains: depression, anxiety, mania, psychosis, substance use, anger, somatic symptoms, suicidal ideation, sleep problems, memory, repetitive thoughts and behaviors, dissociation, and personality functioning. Items are rated on a 5-point Likert scale (0=none, 1=slight, 2=mild, 3=moderate, 4=severe). Preliminary psychometric data indicates that this measure is highly correlated with other symptoms measures for each of the 13 psychiatric domains (r = 0.20–0.70) (Bravo, Villarosa-Hurlocker, Pearson, & Protective Strategies Team, 2018). A total score on this measure assessed at baseline were evaluated as contributors to memory and learning for the contents of treatment.

Outcome measures.

Descriptive statistics and change over the course of treatment on each of these outcome measures are presented in Table 3.

Table 3.

Means and change across treatment for primary outcome measures

Outcome Measure Baselinea Post-Treatmenta 6- Month Follow-Upa Baseline-Post Changeb Baseline to 6-Month Follow-Up Changeb
Mean(SD) Mean(SD) Mean(SD) Beta SE p 95% CI Beta SE p 95% CI
PROMIS-SD 28.70 (6.22) 21.08 (8.18) 22.37 (8.02) −0.90 0.09 .00 −1.07, −0.72 −0.79 0.09 .00 −0.96, −0.61
PROMIS-SRI 49.40 (12.86) 35.63 (13.57) 38.52 (14.00) −0.90 0.09 .00 −1.08, −0.71 −0.74 0.09 .00 −0.92, −0.56
DSM-5 Cross Cutting 24.11 (10.48) 16.69 (11.08) 19.63 (11.10) −0.63 0.09 .00 −0.81, −0.46 −0.41 0.09 .00 −0.58, −0.24
SDS 13.24 (7.51) 6.59 (6.72) 7.65 (6.45) −0.86 0.10 .00 −1.05, −0.67 −0.75 0.10 .00 −0.94, −0.57
Open in a new tab
***

p<0.001.

a

Raw scores are presented.

b

All variables were standardized with a mean of 0 and standard deviation of 1.All models presented are hierarchical linear models with restricted maximum likelihood estimation was used. The random part of the model included a random intercept for participant, assumed to have a bivariate normal distribution with a mean of zero and an unstructured covariance matrix.

Patient-Reported Outcomes Measurement Information System – Sleep Disturbance (PROMIS-SD).

The PROMIS-SD was developed as a part of the NIH Roadmap initiative and designed to improve patient-reported outcomes using state-of-the-art psychometric methods. It assesses sleep disturbance. The 8-item measure is scored 1 (not at all; never; very poor) to 5 (very much, always, very good), and the items are summed. Patients rate items for the past 7 days (e.g., “My sleep was restless,” “I had trouble sleeping,” “I got enough sleep.”) The scale has established reliability and validity with other established sleep measures (e.g., r = 0.30–0.83) (Buysse et al., 2010; Yu et al., 2011).

Patient-Reported Outcomes Measurement Information System – Sleep-Related Impairment (PROMIS-SRI).

The PROMIS-SRI was developed as a part of the NIH Roadmap initiative and designed to improve patient-reported outcomes using state-of-the-art psychometric methods. It assesses impairment related to sleep. The 16-item measure is scored 1 (not at all; never) to 5 (very much; always), and the items are summed. Patients rate items for the past 7 days (e.g., “I felt tired,” “I felt irritable because of poor sleep,” “I was sleepy during the daytime.”) The scale has established reliability and validity with other established sleep measures (e.g., r = 0.46–0.68) (Buysse et al., 2010; Yu et al., 2011).

DSM-5 Cross-Cutting Measure.

The DSM-5 Cross-Cutting Measure administered at post-treatment and 6-month follow-up was used as a measure of symptom severity and treatment outcome. This measure is described in more detail in the “contributors to treatment outcome” section above.

Sheehan Disability Scale (Sleep) (SDS).

The SDS assessed functional impairment. The SDS evaluates the extent to which work/school, social life, and home/family responsibilities are impaired on a 0–10 (not at all to extremely) scale. Its psychometric properties are well established (e.g., Cronbach’s alpha = 0.89, test-retest reliability = 0.73, correlations with similar measures = 0.27–0.59) (Arbuckle et al., 2009; Sheehan, Harnett-Sheehan, & Raj, 1996). The three items were averaged to assess global functional impairment (0 [not impaired] to 10 [highly impaired]).

Procedure

All procedures were approved by the University of California, Berkeley, Committee for the Protection of Human Subjects. All participants provided informed consent. Participants completed a baseline assessment in which they completed a demographics form and all outcome measures. Participants were randomly assigned to receive TranS-C immediately plus usual care (TranS-C-UC), or to Usual Care followed by Delayed Treatment with TranS-C (UC-DT). The latter group was on a waitlist for eight months and then received TranS-C. At post-treatment immediately following treatment and again at 6-month follow-up, participants completed outcome measures. At post-treatment and 6-month follow-up, participants completed the Patient Recall Task, the learning measures, and the outcome measures.

Data coding

Two independent raters coded a subset of the data for the Patient Recall Task (22.22% of the data) and each of the learning measures (36.31% of the data). The remainder of the dataset was coded independently. There was 84.21% inter-rater agreement for the Patient Recall Task, 81.97% inter-rater agreement for Thoughts, 88.52% inter-rater agreement for Application, and 83.61% inter-rater agreement for Advice.

Data analysis

All analyses were conducted in Stata15 (StataCorp, 2017). A significance level of 0.05 was used throughout. For the first and second aims, means and standard deviations or frequencies and percentages are presented. For the third aim, linear regression was used. For the fourth aim, hierarchical linear modeling with restricted maximum likelihood estimation was used. The random part of the model included a random intercept for participant, assumed to have a bivariate normal distribution with a mean of zero and an unstructured covariance matrix. Baseline scores of each outcome measure were included in the fixed part of the model. For aims three and four, all variables were standardized with a mean of 0 and standard deviation of 1. Standardized coefficients were calculated, as these are interpretable as effect sizes (Lorah, 2018).

For the third and fourth aims, to maintain a family-wise error rate of .05 across all tests conducted for a single predictor (e.g., age), we applied Holm’s Bonferroni method (Shaffer, 1995). This involves ordering a series of tests according to their associated p values (smallest to largest) and comparing each p value against a sequentially calculated cutoff. For the third aim, because eight tests were conducted for each predictor, the smallest p value must be less than .006 (.05/8), whereas the largest p value must be less than .05 to meet criteria for statistical significance. For the fourth aim, because eight tests were conducted for each predictor, the smallest p value must be less than .001 (.05/4), whereas the largest p value must be less than .05 to meet criteria for statistical significance. Holm’s Bonferroni method controls family-wise error without the marked loss of power associated with the traditional Bonferroni correction (Shaffer, 1995).

Results

Recall of treatment contents

As displayed in Table 4, on average, participants correctly recalled 5.51 treatment points at post-treatment based on the list of 31 treatment points (17.78% of possible treatment points). As evident in Table 2, based on a list of 31 treatment points, the top four treatment points recalled at post-treatment were: “I use techniques to reduce worry or thinking interfering with my sleep” (42.39% of participants), “Wind down routine before bedtime” (41.30% of participants), “RISEUP or wakeup routine” (41.30% of participants), and “Consistent bedtime” (35.87% of participants).

Table 4.

Memory and Learning of Treatment Contents

Indices of Memory and Learning N Mean Standard Deviation
Recall: Correctly Recalled
Post-treatment 92 5.51 3.93
6-month follow-up 79 3.85 3.16
Recall: Incorrectly Recalled
Post-treatment 93 0.26 0.49
6-month follow-up 79 0.20 0.46
Thoughts: average number of times in past 7 days
Post-treatment 70 5.39 5.83
6-month follow-up 47 5.23 3.56
Thoughts: accurate number of treatment points
Post-treatment 74 0.96 1.04
6-month follow-up 51 1.06 1.01
Application: accurate number of treatment points
Post-treatment 69 1.43 1.31
6-month follow-up 45 1.38 1.04
Advice: number of treatment points
Post-treatment 86 1.29 1.61
6-month follow-up 74 1.30 1.58
N Frequency %
Thoughts: number of participants who reported thinking about treatment contents in the past week
Post-treatment 74 73 98.65
6-month follow-up 51 47 92.16
Application: number of participants who reported applying treatment contents in the past week
Post-treatment 83 69 83.13
6-month follow-up 69 45 65.22
Open in a new tab

As displayed in Table 4, on average, participants correctly recalled 3.92 treatment points (12.65% of possible treatment points) at 6-month follow-up. As evident in Table 2, the top four treatment points recalled at 6-month follow-up were: “Consistent waketime” (32.91% of participants), “Consistent bedtime” (31.65% of participants), “Reducing light exposure in the evening or importance of darkness” (30.38% of participants), and “RISEUP or wakeup routine” (29.11% of participants).

As evident in Table 4, on average, at post-treatment participants incorrectly recalled on average 0.26 treatment points. At 6-month follow-up, participants incorrectly recalled on average 0.20 treatment points.

Learning of treatment contents

Results are displayed in Table 4.

Thoughts.

At post-treatment, 98.65% of participants at post-treatment reported thinking about the treatment contents on average 5.39 times in the past week. On average, participants reported thinking about 0.96 treatment points at post-treatment. At 6-month follow-up, 92.16% of participants at 6-month follow-up reported thinking about treatment contents on average 5.23 times in the past week. On average, participants reported thinking about 1.06 treatment points at 6-month follow-up. At post-treatment, 100% of participants who reported thinking of treatment contents (n = 74) accurately thought about at least one treatment point. At 6-month follow-up, of the 69 participants who reported thinking about their treatment contents in the past week, only 51 participants (73.91%) accurately thought about at least one treatment point.

Application.

At post-treatment, 83.13% of participants at post-treatment reported applying the treatment contents in the past week. On average, participants correctly applied 1.43 treatment points at post-treatment. At 6-month follow-up, 65.22% of participants reported applying the treatment contents in the past week. On average, participants applied 1.38 treatment points at 6-month follow-up. At post-treatment, of the 83 participants who reported applying treatment contents in the past week, only 69 of these participants accurately applied at least one treatment point (83.13%). At 6-month follow-up, of the 69 participants who reported applying treatment contents in the past week, only 45 of these participants accurately applied at least one treatment point (65.22%).

Advice.

On average, participants recommended 1.29 treatment points at post-treatment and 1.30 treatment points at 6-month follow-up.

Contributors to recall and learning

Results are presented in Table 5. Years of education significantly predicted participant recall, thoughts, application and advice at post-treatment, and recall and advice at 6-month follow-up, with more education being associated with increased memory and learning. Symptom severity and age were not significantly associated with the recall or the learning measures.

Table 5.

Linear regressions evaluating the contributions of symptom severity, age, and years of education to memory and learning of treatment contents

Memory or Learning Measure Post-Treatment 6-Month Follow-Up
Coeff. SE p 95% CI Coeff. SE p 95% CI
Symptom Severity (DSM-5 Cross-Cutting Measure at baseline)
Recall −0.01 0.01 0.32 −0.03, 0.01 −0.01 0.01 0.51 −0.02, 0.01
Thoughts (# accurate) −0.00 0.01 0.91 −0.02, 0.02 0.01 0.01 0.29 −0.01, 0.04
Applications (# accurate) 0.00 0.01 0.85 −0.02, 0.03 −0.00 0.01 0.93 −0.03, 0.02
Advice −0.00 0.01 0.87 −0.02, 0.02 −0.01 0.01 0.63 −0.03, 0.02
Age
Recall −0.01 0.01 0.14 −0.03, 0.00 −0.01 0.01 0.47 −0.02, 0.01
Thoughts (# accurate) −0.00 0.01 0.99 −0.02, 0.02 −0.01 0.01 0.41 −0.03, 0.01
Applications (# accurate) −0.01 0.01 0.42 −0.03, 0.01 −0.02 0.01 0.05c −0.04, −0.00
Advice −0.00 0.01 0.61 −0.02, 0.01 −0.01 0.01 0.50 −0.02, 0.01
Years of Education
Recall 0.09 0.03 0.00 0.03, 0.15 0.10 0.03 0.00 0.04, 0.16
Thoughts (# accurate) 0.11 0.03 0.00 0.05, 0.17 0.06 0.04 0.18 −0.03, 0.14
Applications (# accurate) 0.09 0.03 0.01 b 0.03, 0.15 0.08 0.04 0.05c 0.00, 0.15
Advice 0.08 0.03 0.01 a 0.02, 0.14 0.09 0.03 0.01 0.02, 0.16
Open in a new tab
a

p value = 0.006.

b

p value = 0.005.

c

p value = 0.048. DSM-5 Cross Cutting Measure = DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult. Table displays standardized values. Bold values indicate statistical significance after Holm’s Bonferroni correction (Shaffer, 1995).

Recall, learning, and treatment outcome

Results are presented in Table 6. Patient recall was not associated with treatment outcome. None of the learning measures were significantly associated with treatment outcome.

Table 6.

Multilevel models examining the relationship between memory and learning measures on treatment outcome

Outcome Measure Effect of learning/memory on outcome measure at post-treatment Effect of learning/memory on change in outcome measure between post-treatment and 6-month follow-up
N Beta SE p 95% CI Beta SE p 95% CI
Recall (correct)
PROMIS-SD 99 −0.04 0.02 0.14 −0.08, 0.01 0.03 0.03 0.33 −0.03, 0.10
PROMIS-SRI 99 −0.01 0.17 0.49 −0.06, 0.03 0.00 0.03 0.89 −0.05, 0.06
DSM-5 Cross Cutting 99 −0.01 0.02 0.50 −0.06, 0.03 0.01 0.03 0.81 −0.05, 0.07
SDS 98 −0.02 0.02 0.33 −0.06, 0.02 −0.01 0.03 0.66 −0.07, 0.04
Thoughts (# accurate)
PROMIS-SD 81 0.04 0.09 0.64 −0.22, 0.14 0.16 0.14 0.26 −0.12, 0.44
PROMIS-SRI 81 0.01 0.08 0.91 −0.15, 0.74 −0.07 0.12 0.54 −0.16, 0.31
DSM-5 Cross Cutting 81 0.01 0.09 0.92 −0.17, 0.19 −0.08 0.14 0.59 −0.36, 0.21
SDS 80 0.01 0.09 0.87 −0.15, 0.18 −0.14 0.14 0.30 −0.41, 0.13
Application (# accurate)
PROMIS-SD 81 −0.03 0.08 0.65 −0.18, 0.12 −0.15 0.13 0.26 −0.41, 0.11
PROMIS-SRI 81 −0.09 0.18 0.18 −0.22, 0.04 0.03 0.11 0.81 −0.12, 0.24
DSM-5 Cross Cutting 81 −0.02 0.08 0.75 −0.17, 0.02 −0.14 0.13 0.29 −0.40, 0.12
SDS 80 −0.09 0.07 0.20 −0.22, 0.05 −0.10 0.12 0.39 −0.33, 0.13
Advice
PROMIS-SD 95 −0.04 0.06 0.46 −0.15, 0.10 0.05 0.07 0.52 −0.10, 0.19
PROMIS-SRI 95 −0.06 0.05 0.23 −0.17, 0.04 0.03 0.07 0.64 −0.10, 0.16
DSM-5 Cross Cutting 95 −0.02 0.05 0.65 −0.12, 0.08 −0.11 0.07 0.10 −0.24, 0.02
SDS 94 −0.05 0.05 0.38 −0.15, 0.05 −0.05 0.07 0.44 −0.18, 0.08
Open in a new tab

Note. PROMIS-SD = Patient-Reported Outcomes Measurement Information System – Sleep Disturbance. PROMIS-SRI = Patient-Reported Outcomes Measurement Information System – Sleep-Related Impairment. DSM-5 Cross Cutting = DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult. SDS = Sheehan Disability Scale (Sleep). Table displays standardized values.

Discussion

The overarching goal of the present study was to examine memory and learning for the contents of TranS-C among adults with a SMI who received treatment in a community mental health setting. The first aim was to describe the extent of patient recall. Prior research across patient groups fairly consistently suggests participants recall about one third of treatment contents (Bober et al., 2007; Lee & Harvey, 2015). In the present study, the recall rates observed were lower: 17.78% at post-treatment and 12.65% at 6-month follow-up. This finding is consistent with prior research showing that recall is as low as 3–13% for some recommendations (Chambers, 1991; Hahlweg & Richter, 2010). Prior research suggests that patients recall incorrect information from a treatment session. For example, in a study of physician visits, 25% of patients recalled recommendations that were not made (Bober et al., 2007). Encouragingly, in the present study, on average, participants incorrectly recalled less than a single treatment point at both post-treatment and 6-month follow-up. In other words, while participants may not recall a majority of treatment recommendations, the information they do recall is usually accurate. In terms of specific treatment points, at both assessments, treatment elements from the module promoting regular sleep schedule (e.g., maintaining consistent bedtimes, maintaining consistent waketimes, RISE UP, Wind Down) were among the most frequently recalled treatment elements. On the one hand, this is unsurprising as promoting a regular sleep schedule is one of the TranS-C core modules (Harvey & Buysse, 2017). As such, most participants received this treatment module and learned about these treatment elements. On the other hand, approximately only one third of participants recalled this content at either time point. This finding raises the possibility that interventions designed to improve memory for treatment, such as the Memory Support Intervention, may be helpful to integrate alongside TranS-C (Harvey, Lee, et al., 2016). Alternatively, perhaps participants previously knew treatment content that was more readily recalled. Future research should include a baseline measure of knowledge to address the possibility.

The second aim was to describe the extent of learning of treatment contents. Prior research in patients with depression suggests that participants think about treatment contents one to two times in a week and only 50–65% of patient thoughts of treatment content are accurate (Gumport et al., 2018, 2015). In the present study, participants reported thinking about treatment contents more frequently than expected, on average five times per week. Thoughts were more accurate than expected, with 100% and 73.91% of participants who reported thinking about treatment contents reporting at least one accurate treatment element at post-treatment and 6-month follow-up, respectively. Prior research in patients with depression also indicates that participants report applying treatment contents one to two times in the past week and that under 50% of applications are accurate (Gumport et al., 2018, 2015). Consistently, in the present study, participants reported applying treatment contents 1.43 and 1.38 times per week at post-treatment and 6-month follow-up, respectively. Promisingly, participants applied treatment contents more accurately than expected, with 83.13% and 65.22% of participants applying at least one accurate treatment element at post-treatment and 6-month follow-up, respectively. Perhaps the frequency of thoughts and applications and accurate thoughts and applications were higher in this study than in prior research because the prior study used a much briefer intervention, was computer-based, and included treatment content that was not personalized (Gumport et al., 2015), whereas the present study included eight 50-minute in-person treatment sessions with a modularized treatment tailored to each patient’s individual needs. The final measure of learning, advice for a friend with sleep problems, followed a similar pattern to the thoughts and application findings. Overall, these results demonstrate that learning was better than expected, perhaps because sleep is such a salient issue for the patient group studied. However, learning for treatment contents across the three indices was not optimal. Ideally, we would hope that patients learn and continue to use the vast majority of treatment contents. This is not surprising when considering that TranS-C covers a large amount of information across the course of treatment, the transfer of learning problem (Thorndike, 1932), and that impairments in memory are common across SMI (e.g., Henry et al., 2007; Isaac et al., 2006; Torres et al., 2007).

The third aim was to examine if symptom severity, older age, and fewer years of education were associated with patient memory and learning for treatment contents. In partial support of our hypothesis, more years of education were significantly associated with greater memory and learning at both post-treatment and 6-month follow-up. These results are in line with prior research demonstrating that more years of education are associated with fewer age-related memory declines (Angel et al., 2010; Cabeza et al., 2018). Inconsistent with our hypothesis, older age was not significantly associated with patient recall or with learning of treatment contents at multiple time points. However, the results were all in the expected direction given previous research (Glisky, 2007; Li et al., 2016), with older age being associated with poorer recall and learning. It is noteworthy that there were not enough older adults in this sample to fully power these analyses (n = 8 were 65+ years old and n = 25 were 55–64 years old). Also inconsistent with our hypothesis, greater symptom severity was not associated with memory and learning at any time point. These results contribute to the mixed findings on the relationship between symptom severity and challenges in memory and learning (e.g., McDermott & Ebmeier, 2009; Woon et al., 2017).

The final aim was to examine if patient recall and learning of treatment content was associated with treatment outcome. Neither patient recall nor the learning measures were associated with treatment outcome. These findings are inconsistent with prior studies that have demonstrated that patient recall is associated with improved treatment outcome for treatment for depression and bipolar disorder (Harvey, Lee, et al., 2016; Lee & Harvey, 2015) and that patient learning is associated with improved depression treatment outcomes (Gumport et al., 2018, 2015). These findings are also surprising given that the receipt of TranS-C resulted in improvement across all primary outcome measures. Several other factors come into play in between the process of remembering the contents of treatment and using these contents effectively in an appropriate situation. For example, while patients may remember the contents of treatment, they may not use the skills due to other habitual behavior, a lack of motivation, or a myriad of other possibilities. Therefore, it may be valuable to evaluate if recall and learning of treatment contents are associated with a more proximate contributor to behavior change, such as intention to use a treatment element (Schwarzer, Lippke, & Luszczynska, 2011) or adherence to treatment recommendations (Dong et al., 2017; Gumport, Dolsen, & Harvey, 2019).

This study had several limitations. First, this study focused only on patient recall and learning for only one treatment, TranS-C. Hence, generalizability of the results to other treatments is not known. Second, although it has been used in prior studies, the learning measure included in this study is not psychometrically validated. Future research should focus on evaluating its psychometric properties. Third, the memory and learning tasks used in this study primarily assess verbal memory and learning for the contents of treatment and do not evaluate primacy or recency effects. Future research may consider focusing on other metrics of memory and learning such as role playing and modeling (Kurtz, 2011). Future studies should consider additional assessments of patient memory and learning in order to examine the association of treatment point presentation during treatment with memory and learning. Fourth, we did not assess baseline knowledge of the treatment contents provided in TranS-C so we cannot account for prior knowledge in the assessments of learning and memory. Fifth, the memory and learning assessments relied on retrospective self-report in the five minutes given for the Patient Recall Task. Future research using an ecological momentary assessment approach may provide a more nuanced picture of patient memory and learning for treatment contents. However, using these methods with a community mental health based sample with SMI will pose challenges as most of the participants included in this sample did not have access to a regular cellphone or other technology. In addition, although anecdotally most participants finished the Patient Recall Task in under five minutes, it is possible that the length of the task put a ceiling on how much participants could recall at each assessment. Finally, this study included a heterogenous sample of adults with SMI and sleep and circadian problems. As both SMI and sleep and circadian problems are associated with cognitive impairment (Moffitt et al., 2009; Wardle-pinkston, Slavish, & Taylor, 2019), it is possible that cognitive impairment contributed to these findings. Further research is needed to see if the results replicate in other samples and mental health systems, as well as in larger samples.

Overall, the present study offers additional evidence that patient recall and learning for treatment contents are poor. This study extends prior research on patient memory and learning beyond university settings to examine treatment delivered within a community mental health setting and beyond single diagnostic groups to patients with a range of SMI. The specific treatment elements that adults with SMI recall from TranS-C are identified. These findings offer additional evidence that poor patient memory for, and learning of, treatment contents are not a problem unique to university treatment settings and are transdiagnostic concerns. Hence, integrating interventions designed to improve memory for treatment, such as the Memory Support Intervention (Harvey, Lee, et al., 2016), may be helpful in community settings treating patients with SMI. This study offers data on the treatment points that stand out the most to patients. Treatment developers and treatment providers may consider further testing and simplifying TranS-C—and other complex interventions—into core elements that patients are most likely to remember and that are associated with change during and following treatment (e.g., Gumport et al., 2019).

Acknowledgements:

This research was supported by National Institute of Mental Health grant R01MH105513. We are grateful to Christopher Blay and Cailin Greenburg for assistance with data coding and to Caitlin Gasperetti and Courtney Armstrong for helpful discussions on the topic of this paper.

References

  1. Airaksinen E, Larsson M, & Forsell Y (2005). Neuropsychological functions in anxiety disorders in population-based samples: Evidence of episodic memory dysfunction. Journal of Psychiatric Research, 39(2), 207–214. 10.1016/j.jpsychires.2004.06.001 [DOI] [PubMed] [Google Scholar]
  2. American Academy of Sleep Medicine. (2014). International Classification of Sleep Disorders – Third Edition (ICSD-3). Darien, IL. [Google Scholar]
  3. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Washington, D.C.: American Psychiatric Association. [Google Scholar]
  4. Angel L, Fay S, Bouazzaoui B, Baudouin A, & Isingrini M (2010). Protective role of educational level on episodic memory aging: An event-related potential study. Brain and Cognition, 74(3), 312–323. 10.1016/j.bandc.2010.08.012 [DOI] [PubMed] [Google Scholar]
  5. Arbuckle R, Frye MA, Brecher M, Paulsson B, Rajagopalan K, Palmer S, & Degl’ Innocenti A (2009). The psychometric validation of the Sheehan Disability Scale (SDS) in patients with bipolar disorder. Psychiatry Research, 165(1–2), 163–174. 10.1016/j.psychres.2007.11.018 [DOI] [PubMed] [Google Scholar]
  6. Baglioni C, Nanovska S, Regen W, Spiegelhalder K, Feige B, Nissen C, … Riemann D (2016). Sleep and Mental Disorders: A Meta-Analysis of Polysomnographic Research. Psychological Bulletin, 142, 969–990. 10.1037/bul0000053 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bober SL, Hoke LA, Duda RB, & Tung NM (2007). Recommendation Recall and Satisfaction After Attending Breast/Ovarian Cancer Risk Counseling. Journal of Genetic Counseling, 16(6), 755–762. 10.1007/s10897-007-9109-0 [DOI] [PubMed] [Google Scholar]
  8. Boyer P, Phillips JL, Rousseau FL, & Ilivitsky S (2007). Hippocampal abnormalities and memory deficits: New evidence of a strong pathophysiological link in schizophrenia. Brain Research Reviews, 54(1), 92–112. 10.1016/j.brainresrev.2006.12.008 [DOI] [PubMed] [Google Scholar]
  9. Bravo AJ, Villarosa-Hurlocker MC, Pearson MR, & Protective Strategies Team. (2018). College Student Mental Health: An Evaluation of the DSM-5 Self- rated Level 1 Cross-Cutting Symptom Measure. Psychological Assessment, 39, 1382–1389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Buysse DJ (2014). Sleep Health: Can We Define It? Does It Matter? Sleep, 37(1), 9–17. 10.5665/sleep.3298 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Buysse DJ, Yu L, Moul DE, Germain A, Stover A, Dodds NE, … Pikonis PA (2010). Development and Validation of Patient-Reported Outcome Measures for Sleep Disturbance and Sleep-Related Impairments. SLEEP, 33(6), 29–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cabeza R, Albert M, Belleville S, Craik FIM, Duarte A, Grady CL, … Rajah MN (2018). Maintenance, reserve and compensation: the cognitive neuroscience of healthy ageing. Nature Reviews Neuroscience, 19(11), 701–710. 10.1038/s41583-018-0068-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chambers MJ (1991). Patient recall of recommendations in the behavioural treatment of insomnia. Sleep Research, 20, 222. [Google Scholar]
  14. Dong L, Zhao X, Ong S, & Harvey AG (2017). Patient Recall of Specific Cognitive Therapy Contents Predicts Adherence and Outcome in Adults with Major Depressive Disorder. Behaviour Research and Therapy, 176(12), 189–1999. 10.1016/j.brat.2017.08.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Edinger JD, Arnedt JT, Bertisch SM, Carney CE, Harrington JJ, Lichstein KL, … Heald JL (2021). Behavioral and psychological treatments for chronic insomnia disorder in adults : an American Academy of Sleep Medicine clinical practice guideline. Journal of Clinical Sleep Medicine, 17(2), 255–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Ehlers CL, Frank E, & Kupfer DJ (1988). Social Zeitgebers and Biological Rhythms. Archives of General Psychiatry, 45(10), 948–952. 10.1001/archpsyc.1988.01800340076012 [DOI] [PubMed] [Google Scholar]
  17. Glisky EL (2007). Changes in Cognitive Function in Human Aging. In Riddle DR(Ed.), Human Aging: Models, Methods, and Mechanisms (pp. 3–20). Boca Raton, Florida: CRC Press/Taylor & Francis. [PubMed] [Google Scholar]
  18. Gotlib IH, & Joormann J (2010). Cognition and Depression: Current Status and Future Directions. Annual Review of Clinical Psychology, 6, 285–312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Grady CL, & Craik FI (2000). Changes in memory processing with age. Current Opinion in Neurobiology, 10(2), 224–231. 10.1016/S0959-4388(00)00073-8 [DOI] [PubMed] [Google Scholar]
  20. Gumport NB, Dolsen MR, & Harvey AG (2019). Usefulness and Utilization of Treatment Elements from the Transdiagnostic Sleep and Circadian Intervention with Adolescents with an Evening Circadian Preference. Behaviour Research and Therapy, 123, 103504. 10.1016/j.brat.2019.103504 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gumport NB, Dong L, Lee JY, & Harvey AG (2018). Patient learning of treatment contents in cognitive therapy. Journal of Behavior Therapy and Experimental Psychiatry, 58, 51–59. 10.1016/j.jbtep.2017.08.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Gumport NB, Williams JJ, & Harvey AG (2015). Learning cognitive behavior therapy. Journal of Behavior Therapy and Experimental Psychiatry, 48, 164–169. 10.1016/j.jbtep.2015.03.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hahlweg K, & Richter D (2010). Prevention of marital instability and distress. Results of an 11-year longitudinal follow-up study. Behaviour Research and Therapy, 48(5), 377–383. 10.1016/j.brat.2009.12.010 [DOI] [PubMed] [Google Scholar]
  24. Harvey AG, & Buysse DJ (2017). Treating Sleep Problems: A Transdiagnostic Approach. New York, NY: Guilford Press. [Google Scholar]
  25. Harvey AG, Hein K, Dong L, Smith FL, Lisman M, Yu S, … Buysse DJ (2016). A transdiagnostic sleep and circadian treatment to improve severe mental illness outcomes in a community setting: study protocol for a randomized controlled trial. Trials, 17(1), 606. 10.1186/s13063-016-1690-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Harvey AG, Lee J, Smith RL, Gumport NB, Hollon SD, Rabe-Hesketh S, … Abrons D (2016). Improving Outcome for Mental Disorders by Enhancing Memory for Treatment. Behaviour Research and Therapy, 81, 35–46. 10.1016/j.brat.2016.03.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Harvey AG, Lee J, Williams J, Hollon SD, Walker MP, Thompson MA, & Smith R (2014). Improving Outcome of Psychosocial Treatments by Enhancing Memory and Learning. Perspectives on Psychological Science, 9(2), 161–179. 10.1177/1745691614521781 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Henry JD, Rendell PG, Kliegel M, & Altgassen M (2007). Prospective memory in schizophrenia: Primary or secondary impairment? Schizophrenia Research, 95(1–3), 179–185. 10.1016/j.schres.2007.06.003 [DOI] [PubMed] [Google Scholar]
  29. Hertel PT (1998). Relation between rumination and impaired memory in dysphoric moods. Journal of Abnormal Psychology, 107(1), 166–172. 10.1037/0021-843X.107.1.166 [DOI] [PubMed] [Google Scholar]
  30. Hertenstein E, Feige B, Gmeiner T, Kienzler C, Spiegelhalder K, Johann A, … Baglioni C (2019). Insomnia as a predictor of mental disorders: A systematic review and meta-analysis. Sleep Medicine Reviews, 43, 96–105. 10.1016/j.smrv.2018.10.006 [DOI] [PubMed] [Google Scholar]
  31. Hundt NE, Mignogna J, Underhill C, & Cully JA (2013). The Relationship Between Use of CBT Skills and Depression Treatment Outcome: A Theoretical and Methodological Review of the Literature. Behavior Therapy, 44(1), 12–26. 10.1016/J.BETH.2012.10.001 [DOI] [PubMed] [Google Scholar]
  32. Isaac CL, Cushway D, & Jones GV (2006). Is posttraumatic stress disorder associated with specific deficits in episodic memory? Clinical Psychology Review, 26(8), 939–955. 10.1016/j.cpr.2005.12.004 [DOI] [PubMed] [Google Scholar]
  33. Jansen J, Butow PN, Van Weert JCM, Van Dulmen S, Devine RJ, Heeren TJ, … Tattersall MHN (2008). Does age really matter? Recall of information presented to newly referred patients with cancer. Journal of Clinical Oncology, 26(33), 5450–5457. 10.1200/JCO.2007.15.2322 [DOI] [PubMed] [Google Scholar]
  34. Kim JJ, Brookman-Frazee L, Barnett ML, Tran M, Kuckertz M, Yu S, & Lau AS (2020). How community therapists describe adapting evidence-based practices in sessions for youth: Augmenting to improve fit and reach. Journal of Community Psychology, (March). 10.1002/jcop.22333 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kurtz MM (2011, June). Neurocognition as a predictor of response to evidence-based psychosocial interventions in schizophrenia: What is the state of the evidence? Clinical Psychology Review. NIH Public Access. 10.1016/j.cpr.2011.02.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Laws MB, Lee Y, Taubin T, Rogers WH, & Wilson IB (2018). Factors associated with patient recall of key information in ambulatory specialty care visits: Results of an innovative methodology. PLoS ONE, 13(2), 1–13. 10.1371/journal.pone.0191940 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lee JY, & Harvey AG (2015). Memory for therapy in bipolar disorder and comorbid insomnia. Journal of Consulting and Clinical Psychology, 83, 92–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Li B, Zhu X, Hou J, Chen T, Wang P, & Li J (2016). Combined cognitive training vs. memory strategy training in healthy older adults. Frontiers in Psychology, 7(JUN), 1–11. 10.3389/fpsyg.2016.00834 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. López CM, Lancaster CL, Gros DF, & Acierno R (2017). Residual Sleep Problems Predict Reduced Response to Prolonged Exposure among Veterans with PTSD. Journal of Psychopathology and Behavioral Assessment, 39(4), 755–763. 10.1007/s10862-017-9618-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Lorah J (2018). Effect size measures for multilevel models: definition, interpretation, and TIMSS example. Large-Scale Assessments in Education, 6, 8. 10.1186/s40536-018-0061-2 [DOI] [Google Scholar]
  41. McDermott LM, & Ebmeier KP (2009). A meta-analysis of depression severity and cognitive function. Journal of Affective Disorders, 119(1–3), 1–8. 10.1016/j.jad.2009.04.022 [DOI] [PubMed] [Google Scholar]
  42. Miller WR, & Rollnick S (2002). Motivational interviewing: Preparing people for change. New York: Guilford Press. [Google Scholar]
  43. Moffitt TE, Ph D, Roberts AL, Ph D, Martin LT, Sc D, … Ph D (2009). Childhood IQ and Adult Mental Disorders : A Test of the Cognitive Reserve Hypothesis, (January), 50–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Morin CM, Bootzin RR, Buysse DJ, Edinger JD, Espie C. a, & Lichstein KL. (2006). Psychological and behavioral treatment of insomnia:update of the recent evidence (1998–2004). Sleep, 29(11), 1398–1414. [DOI] [PubMed] [Google Scholar]
  45. Qaseem A, Kansagara D, Forciea MA, Cooke M, Denberg TD, Barry MJ, … Wilt T (2016). Management of chronic insomnia disorder in adults: A clinical practice guideline from the American college of physicians. Annals of Internal Medicine, 165(2), 125–133. 10.7326/M15-2175 [DOI] [PubMed] [Google Scholar]
  46. Reichenberg A, Harvey PD, Bowie CR, Mojtabai R, Rabinowitz J, Heaton RK, & Bromet E (2009). Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophrenia Bulletin, 35(5), 1022–1029. 10.1093/schbul/sbn044 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Rendell PG, Mazur M, & Henry JD (2009). Prospective memory impairment in former users of methamphetamine. Psychopharmacology, 203(3), 609–616. 10.1007/s00213-008-1408-0 [DOI] [PubMed] [Google Scholar]
  48. Reuter-Lorenz PA, Festini SB, & Jantz TK (2016). Executive Functions and Neurocognitive Aging. In Schaie KW & Willis SL (Eds.), Handbook of the Psychology of Aging: Eighth Edition (8th ed., pp. 245–262). San Diego, CA: Academic Press. 10.1016/B978-0-12-411469-2.00013-3 [DOI] [Google Scholar]
  49. Riemann D, Baglioni C, Bassetti C, Bjorvatn B, Dolenc Groselj L, Ellis JG, … Spiegelhalder K (2017). European guideline for the diagnosis and treatment of insomnia. Journal of Sleep Research, 26(6), 675–700. 10.1111/jsr.12594 [DOI] [PubMed] [Google Scholar]
  50. Salthouse TA (2009). When does age-related cognitive decline begin? Neurobiology of Aging, 30(4), 507–514. 10.1016/j.neurobiolaging.2008.09.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Saykin AJ, Gur RC, Gur RE, Mozley PD, Mozley LH, Resnick SM, … Stafiniak P (1991). Neuropsychological Function in Schizophrenia: Selective Impairment in Memory and Learning. Archives of General Psychiatry, 48(7), 618–624. 10.1001/archpsyc.1991.01810310036007 [DOI] [PubMed] [Google Scholar]
  52. Schwarzer R, Lippke S, & Luszczynska A (2011). Mechanisms of Health Behavior Change in Persons With Chronic Illness or Disability: The Health Action Process Approach (HAPA). Rehabilitation Psychology, 56(3), 161–170. 10.1037/a0024509 [DOI] [PubMed] [Google Scholar]
  53. Serper MR, Bergman A, Copersino ML, Chou JCY, Richarme D, & Cancro R (2000). Learning and memory impairment in cocaine-dependent and comorbid schizophrenic patients. Psychiatry Research, 93(1), 21–32. 10.1016/S0165-1781(99)00122-5 [DOI] [PubMed] [Google Scholar]
  54. Shaffer JP (1995). Multiple hypothesis testing. Annual Review of Psychology, 46, 561–584. [Google Scholar]
  55. Sheehan DV, Harnett-Sheehan K, & Raj BA (1996). The measurement of disability. International Clinical Psychopharmacology, 11(Supplement 3), 89–95. 10.1097/00004850-199606003-00015 [DOI] [PubMed] [Google Scholar]
  56. StataCorp. (2017). Stata Statistical Software: Release 15. College Station, TX: StataCorp LP. [Google Scholar]
  57. Thorndike EL (1932). The Fundamentals of Learning. New York: Teachers College Bureau of Publications. [Google Scholar]
  58. Torres IJ, Boudreau VG, & Yatham LN (2007). Neuropsychological functioning in euthymic bipolar disorder: A meta-analysis. Acta Psychiatrica Scandinavica, 116(SUPPL. 434), 17–26. 10.1111/j.1600-0447.2007.01055.x [DOI] [PubMed] [Google Scholar]
  59. Walker MP, & Stickgold R (2006). Sleep, Memory, and Plasticity. Annual Review of Psychology, 57(1), 139–166. 10.1146/annurev.psych.56.091103.070307 [DOI] [PubMed] [Google Scholar]
  60. Wang PS, Demler O, & Kessler RC (2002). Adequacy of treatment for serious mental illness in the United States. American Journal of Public Health, 92(1), 92–98. 10.2105/AJPH.92.1.92 [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Wardle-pinkston S, Slavish DC, & Taylor DJ (2019). Insomnia and cognitive performance : A systematic review and meta- analysis. Sleep Medicine Reviews, 48, 101205. 10.1016/j.smrv.2019.07.008 [DOI] [PubMed] [Google Scholar]
  62. Wirz-Justice A, Benedetti F, & Terman M (2009). Chronotherapeutics for affective disorders: A clinician’s manual for light and wake therapy. Basel, Switzerland: Karger. 10.1159/isbn.978-3-8055-9121-8 [DOI] [PubMed] [Google Scholar]
  63. Woon FL, Farrer TJ, Braman CR, Mabey JK, & Hedges DW (2017). A meta-analysis of the relationship between symptom severity of Posttraumatic Stress Disorder and executive function. Cognitive Neuropsychiatry, 22(1), 1–16. 10.1080/13546805.2016.1255603 [DOI] [PubMed] [Google Scholar]
  64. Yoo SS, Hu PT, Gujar N, Jolesz FA, & Walker MP (2007). A deficit in the ability to form new human memories without sleep. Nature Neuroscience, 10(3), 385–392. 10.1038/nn1851 [DOI] [PubMed] [Google Scholar]
  65. Yu L, Buysse DJ, Germain A, Moul DE, Stover A, Dodds NE, … Pilkonis PA (2011). Development of Short Forms From the PROMISTM Sleep Disturbance and Sleep-Related Impairment Item Banks. Behavioral Sleep Medicine, 10(1), 6–24. 10.1080/15402002.2012.636266 [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Zieve GG, Dong L, & Harvey AG (2019). Patient Memory for Psychological Treatment Contents: Assessment, Intervention, and Future Directions for a Novel Transdiagnostic Mechanism of Change. Behaviour Change, 1–11. 10.1017/bec.2019.1 [DOI] [Google Scholar]

RESOURCES