The Impact of Skills Training on Cognitive Functioning in Older People with Serious Mental Illness: Results from a Randomized Controlled Trial

Abstract

Objectives

Research on psychosocial interventions for people with serious mental illness (SMI) demonstrates that cognitive functioning is associated with psychosocial functioning. However, cognitive impairment is more pronounced and progressive in older adults with SMI and is associated with poorer functioning. Little is known about the long-term impact of psychosocial skills training on cognitive functioning in this rapidly growing group.

Design

We conducted post-hoc analyses on a previously reported randomized control trial to evaluate the relationship between cognitive and psychosocial functioning and the potential impact of psychosocial skills training on cognitive functioning over time.

Setting and Participants

The current study was conducted using a community-dwelling sample of 183 people older than age 50 with SMI.

Intervention

Half of the study sample received a psychosocial skills training and health management program, Helping Older People Experience Success (HOPES), and were compared to a Treatment as Usual (TAU) group.

Measurements

Cognitive and psychosocial functioning were assessed at baseline and at 1-, 2-, and 3-year follow-up.

Results

Psychosocial functioning was strongly related to cognitive functioning at baseline. Participation in HOPES was not associated with greater improvements in overall cognitive functioning; however, exploratory analyses suggested a modest improvement in executive functioning.

Conclusions

Given the lack of long-term data on interventions associated with sustaining or improving executive functioning in older adults with SMI, these exploratory findings suggest that future research is warranted to establish the potential benefit of psychosocial skills training as a component of treatment aimed at enhancing long-term psychosocial and cognitive functioning.

Objective

Impaired psychosocial functioning is a hallmark of schizophrenia and serious mental illness (SMI), which is defined by the presence of chronic symptoms of a major mental illness (e.g., schizophrenia, bipolar disorder, or major depression) together with persistent impairments in several domains of life functioning. Long-term studies underscore the heterogeneity of outcomes for older adults with SMI. For example, a subset of individuals with schizophrenia experience attenuation of core symptoms, improved subjective quality of life, and increased social involvement as they age. (1) However, many older adults with SMI (including people with schizophrenia, bipolar disorder and major depression) have impairments in functioning that are persistent and require ongoing mental health services and functional supports. (2, 3) Also, older people with SMI tend to have greater functional impairments compared to younger people with SMI, including greater need for assistance with basic and instrumental activities of daily living and transportation. (4) Finally, older people with SMI face age-related cognitive declines and increased medical comorbidity that can hamper functioning. When these impairments are present, they contribute to worse outcomes, including more severe symptoms, relapses and hospitalizations, as well as early nursing home placement. (5–7)

Although people with SMI have a life expectancy that is 25 to 30 years less than the general population, (8) the U.S. Census Bureau (9) projects that the number of Americans aged 65 and older will more than double (from 34 million in 1990 to 70 million in 2030) in the coming decades. Therefore, it is important to understand more about the determinants of impaired psychosocial functioning and interventions that specifically target this rapidly growing segment of the population.

It is widely accepted that most individuals with schizophrenia experience some degree of cognitive dysfunction. (10) A similar pattern of cognitive impairments are also often present in people with bipolar disorder or major depression who have SMI, albeit at a lesser magnitude than in schizophrenia. (11–13) An extensive body of research in SMI documents the substantial cross-sectional relationship between impaired cognitive functioning and psychosocial functioning (14–17). Research on pharmacological interventions for people with schizophrenia has demonstrated only modest improvements in specific cognitive functions, though when obtained, better cognitive functioning has been correlated with improvements in quality of life. (18, 19) Also, studies of cognitive remediation for schizophrenia have found that improved cognition functioning is correlated with gains in psychosocial functioning. (20, 21)

Comparatively little is known about the impact of psychosocial skills training interventions on cognitive functioning. The purpose of the present study was to address the following questions about the relationships between cognitive functioning, psychosocial functioning, and psychosocial skills training in older people with SMI:

1. What is the cross-sectional association between cognitive impairment and psychosocial functioning among middle-aged and older adults with SMI?

2. Is psychosocial skills training associated with improved cognitive functioning at long-term follow-up?

To address these questions we conducted secondary exploratory analyses of cognitive functioning outcomes of a randomized trial comparing a psychosocial skills training intervention for older adults with SMI, the Helping Older People Experience Success program (HOPES) to treatment as usual (TAU) over three years of follow-up assessments. The HOPES program (22) included weekly group classes covering seven skill areas for 12 months, followed by monthly booster sessions to review the curriculum for 12 months, paired with monthly meetings for 24 months with a nurse who helped participants access health care and receive recommended preventive health care. Participants were assessed at baseline and yearly for three years. After 2 years of the program, the 90 people (mean age=60) who were assigned to HOPES demonstrated significant improvements in community functioning, community living skills, leisure and recreation skills, negative symptoms and self-efficacy compared with the 93 people (mean age=60) who were assigned to TAU. (2)

In the following analysis of 3-year cognitive outcome data we addressed two hypotheses: 1) cognitive functioning would be positively associated with psychosocial functioning at baseline; and 2) participation in HOPES would be associated with greater long-term improvement in cognitive functioning compared with TAU.

Methods

All study procedures were approved by the Dartmouth Committee for the Protection of Human Subjects, the NH Bureau of Behavioral Health Committee for the Protection of Human Subjects, and two institutional review boards in Boston.

Participants

Participants in the HOPES study were clients at one of three public mental health centers, including two in Boston and one in Nashua, NH. Inclusion criteria were: 1) age 50+; 2) SMI, defined as a DSM-IV diagnosis of major depression, bipolar disorder, schizoaffective disorder, or schizophrenia, based on the Structured Clinical Interview for DSM-IV and persistent functional impairment; 3) enrollment in services for 3 months or more; 4) willingness to provide informed consent; and 5) fluency in English. Exclusion criteria were: 1) residing in an institution; 2) primary diagnosis of dementia or significant cognitive impairment defined by Mini Mental Status Exam score less than 20; or 3) terminal illness expected to cause death within one year. Visual and auditory impairments were not formally assessed but were not exclusions. In fact, one deaf individual and several people with macular degeneration were enrolled and accommodated in the study.

The average age of the sample of 183 participants at baseline was 60.2 (SD=7.92). Most were Caucasian (85.8%), single and never married (64.5%), with at least a high school education (73.2%). Roughly equal numbers of participants were living independently (51.4%) versus in supervised or supported housing (48.6%) and the sample included a somewhat greater proportion of females (57.9%). A slight majority of individuals had schizophrenia or schizoaffective disorder (56.3%) compared to mood disorders (43.7%). All participants were taking psychotropic medications at baseline and throughout the study.

Measures

All measures were administered by blinded research assessors who completed several months of training and demonstrated high inter-rater reliability prior to evaluating participants.

Cognitive Functioning

Cognitive functions were assessed using selected subtests from the Delis-Kaplan Executive Functioning System (DKEFS). (23) The subtests from the D-KEFS used in the HOPES study included: the verbal fluency measures, which require respondents to name words that start with a particular letter or belong to a stated category; the Color-Word Interference test, which is based on the Stroop test; the California Verbal Learning Test-II, which evaluates the ability to recall a list of words from four categories; and the Trails tests, which assess abilities to sequence letters and numbers and to maintain a sequence that alternates between letters and numbers.

In order to reduce the number of variables included in the statistical analyses of cognitive data, in consultation with a licensed neuropsychologist, we selected seven representative indicators from among the cognitive functions that are commonly impaired in individuals with schizophrenia: verbal fluency (Letter Fluency, Category Fluency), psychomotor speed (Trails 2), memory (CVLT Trials 1–5, CVLT Long Delay Free Recall), and executive functioning (Color-Word Interference Inhibition, Trails 4). We also formed a composite cognitive score by standardizing and summing the seven specific test scores. Finally, we used the Wide-Range Achievement Test Reading subtest (24) as a measure of premorbid IQ.

Psychosocial Functioning

Psychosocial functioning measures are fully described and referenced in a prior report. (2) In brief, the following instruments were used. Community functioning was assessed with the Multnomah Community Ability Scale (25), completed by an informant, usually a mental health treatment provider. This scale assesses social appropriateness, behavioral problems, degree of interference of physical and psychiatric symptoms, and adaptation to mental illness. Higher scores reflect better functioning. Social functioning was evaluated with the Social Behavior Schedule (26), as adapted by Schooler et al. (27), which assesses behaviors such as social avoidance, appropriateness of interactions, and manners. SBS scores in this study represent a blending of self-report and informant ratings. Higher scores reflect worse functioning. Independent living skills were assessed with the Independent Living Skills Survey-Self Report (ILSS) (28), which includes 10 subscales that assess appearance and clothing, personal hygiene, care of personal possessions, food preparation, health management, leisure and recreation, money management, use of public transportation, job seeking, and job maintenance. Higher scores reflect better functioning. Negative symptoms were evaluated with the Scale for the Assessment of Negative Symptoms (SANS) (29), a clinician-rated measure of Affect, Alogia, Apathy, and Anhedonia/Asociality. Higher scores indicate more severe symptoms. Community living skills were measured with the UCSD Performance-Based Skills Assessment (UPSA) (20). Developed for use with middle-aged and older people with schizophrenia, the UPSA involves a series of role-plays designed to assess living skills in five areas (communication, trip planning, transportation, finances, and shopping). Higher scores denote better performance.

Procedures

After completing all baseline measures, participants were randomly assigned to HOPES (n=90) or TAU (n=93) and were assessed with the same measures at 12-months, 24-months, and 12 months post-treatment (36-months). Inclusion of the control (TAU) group ensured that reported findings regarding linking change between neurocognitive and psychosocial domains were not due to practice effects on the neurocognitive measures. Randomization to HOPES or TAU within each site was stratified by diagnosis (mood disorder or schizophrenia-spectrum) and gender. A research staff member used a computer program to complete the randomizations. Participants were paid for completing assessments but not for HOPES sessions. Participants in TAU continued to receive their usual services. Routine mental health services at all study sites included pharmacotherapy, case management, individual therapy, and limited rehabilitation services, such as psychoeducational groups. The HOPES program is described in detail elsewhere. (2, 22)

Statistical Analyses

Pearson’s correlations were computed to examine the relationships between cognitive functioning and psychosocial functioning at baseline. To evaluate whether these relationships differed as a function of diagnosis, we also performed the same analyses separately for the participants with schizophrenia-spectrum and mood disorders. Because there were no significant differences between HOPES and TAU at baseline and there were only three follow-up assessments, rather than fitting parametric curves with random effects we included the baseline as a covariate and fit baseline adjusted mean response profile models using the SAS PROC MIXED procedure. This approach, also referred to as covariance pattern models, is similar to a traditional analysis of covariance except that it can accommodate correlated data by selecting appropriate covariance structures as well as missing data with maximum likelihood estimation. Rather than fitting models for different outcomes with possibly different covariance structures, we obtained estimates of standard error in PROC MIXED by using the “empirical” estimate option. This method is based on “sandwich estimation,” and yields robust and asymptotically consistent estimates of variance and covariance regardless of the data’s actual covariance structure. For these analyses, the scores for the cognitive variables at each assessment (1, 2, and 3 year) were the dependent measures, treatment group, diagnosis, gender, and their interactions were the independent variables, and the baseline cognitive scores and the baseline WRAT-3 score were covariates. The main effect for HOPES was a test of whether the HOPES participants differed from TAU in cognitive functioning across the three follow-up assessments, controlling for the baseline assessment. Treatment effects were evaluated by conducting intent-to-treat analyses on the full sample, including all participants with baseline data and data for at least one follow-up point. A comparison of study participants who had complete 3-year follow-up data (n=139) and those who did not (n=44) found no significant differences in baseline measures of cognitive functioning. Statistical tests were conducted and differences were considered statistically significant based on a p value of .05 or less.

Exploratory analyses were conducted to evaluate whether improvements in cognitive functioning among the HOPES participants were associated with improvements in psychosocial functioning. We performed these analyses only on the cognitive measure for which there were significant differences between the HOPES and TAU groups. We computed simple partial correlations between the endpoint psychosocial variables and the endpoint cognitive measure (i.e., Color-Word Interference), which controls for baseline values of each.

Results

Associations between the measures of cognitive performance and psychosocial functioning were similar for the schizophrenia-spectrum and mood disorder groups. Thus, we present correlations for the combined, full sample in Table 1. In general, the performance-based measure of community living skills (i.e., UPSA) was more strongly related to cognitive functioning than broader measures of community functioning. Cognitive variables reflecting measures of flexibility, attention, and fluency were more strongly related to psychosocial functioning than were measures of verbal learning and memory.

Table 1.

Pearson’s correlations between psychosocial functioning and cognitive functioning at baseline

	Mult r df, p	SBS r df, p	SANS r df, p	ILSS r df, p	UPSA r df, p
D-KEFS Color-Word Interference Inhibition	.187 166, .015	−.224 177, .003	−.261 172, .001	.172 177, .022	.330 177, .001
D-KEFS Trails 2	.160 169, .037	−.179 179, .016	−.326 175, .001	.193 179, .009	.447 179, .001
D-KEFS Trails 4	.148 165, .056	−.219 176, .003	−.255 172, .001	.138 176, .067	.473 176, .001
D-KEFS Verbal Fluency (FAS)	.021 169, .787	−.143 180, .054	−.273 177, .001	.218 180, .003	.348 180, .001
D-KEFS Verbal Fluency (Category Total)	.064 170, .404	−.169 181, .023	−.303 176, .001	.175 181, .018	.460 181, .001
CVLT (Total 1–5)	.102 170, .184	−.187 181, .011	−.157 176, .037	.135 181, .089	.506 181, .001
CVLT (Long Delay Free Recall)	.015 170, .843	−.152 181, .040	−.065 176, .386	−.011 181, .883	.294 181, .001
Overall Cognitive Functioning	.146 170, .056	−.273 181, .001	−.336 176, .001	.223 181, .002	.674 181, .001

Table 2 presents descriptive statistics for the baseline, 1-, 2-, and 3-year cognitive assessments for the HOPES and TAU groups. No difference was found between HOPES and TAU in overall cognitive functioning. Individuals participating in HOPES compared to TAU demonstrated greater improvement in Color-Word Interference Inhibition, a measure of executive functioning (effect size: Cohen’s d= .19). The differences between HOPES and TAU on the other cognitive variables were not significant.

Table 2.

Descriptive Statistics and Treatment Group Effects on Cognitive Measures from Mixed Model Regression Analyses Group Effects

	Group	Baseline m (sd)	1 Year m (sd)	2 Year m (sd)	3 Year m (sd)	F (df)	p
D-KEFS Color-Word Interference Inhibition	HOPES	6.830 (3.76)	7.679 (3.54)	7.377 (3.97)	8.039 (4.24)	3.872 (1, 146)	.051
	TAU	6.341 (4.31)	6.570 (4.30)	6.768 (4.07)	6.765 (4.19)
D-KEFS Trails 2	HOPES	7.044 (4.09)	7.278 (3.94)	7.391 (4.40)	7.750 (4.44)	.237 (1, 145)	.627
	TAU	5.826 (4.01)	6.338 (4.32)	6.594 (3.90)	6.392 (3.84)
D-KEFS Trails 4	HOPES	5.382 (4.35)	5.597 (4.17)	5.676 (4.68)	5.423 (4.53)	.089 (1, 145)	.765
	TAU	4.371 (3.79)	4.727 (4.16)	4.794 (3.87)	4.940 (3.99)
D-KEFS Verbal Fluency (FAS)	HOPES	7.303 (3.43)	7.342 (3.20)	7.357 (3.28)	7.962 (2.87)	1.212 (1, 149)	.273
	TAU	6.419 (3.82)	6.950 (3.84)	6.841 (3.70)	6.824 (3.55)
D-KEFS Verbal Fluency (Category Total)	HOPES	6.311 (2.91)	6.684 (2.68)	6.643 (3.18)	6.962 (3.03)	.263 (1, 149)	.609
	TAU	6.118 (3.57)	6.350 (3.84)	6.362 (3.52)	6.314 (3.83)
CVLT (Total 1–5)	HOPES	37.211 (9.55)	40.974 (10.35)	40.652 (13.83)	41.039 (12.23)	.566 (1, 152)	.453
	TAU	33.441 (12.20)	37.22 (12.61)	38.52 (14.10)	40.326 (15.38)
CVLT (Long Delay Free Recall)	HOPES	−1.261 (1.045)	−1.006 (1.106)	−1.116 (1.513)	−.8529 (1.141)	.000 (1,148)	.989
	TAU	−1.613 (1.178)	−1.192 (1.182)	−1.294 (1.38)	−1.420 (1.480)
Overall Cognitive Functioning	HOPES	−.2462 (.492)	−.1661 (.518)	−.1689 (.680)	−.1470(.636)	.052 (1,145)	.819
	TAU	−.4224 (.634)	−.3152 (.705)	−.3469 (.735)	−.3621 (.777)

Table 3 shows the relationship between improvement on Color-Word Interference Inhibition and improvement in psychosocial functioning in the HOPES group. Improved performance on Color-Word Interference Inhibition was associated with improvement in three of the five measures of psychosocial functioning.

Table 3.

Simple partial correlations between endpoint psychosocial functions and endpoint cognitive functioning, controlling for baseline to evaluate relationship between change in executive functioning and change in psychosocial functioning.

DKEFS Color-Word Interference (Inhibition)
	r	df	p
Community Functioning (Multnomah)	.279	73	.015
Social Functioning (SBS)	−.261	77	.020
Negative Symptoms (SANS)	−.347	73	.002
Independent Living Skills (ILSS)	.149	75	.195
Community Living Skills (UPSA)	−.008	77	.942

Discussion

Results of this study confirm that cognitive functioning in older adults with SMI is significantly associated with psychosocial functioning in the community. Interestingly, measures of memory were not as strongly related to psychosocial functioning as expected based on strong associations found in studies of younger people with schizophrenia. (14–17) We did not find that HOPES was associated with improved overall cognitive functioning. However, a post-hoc exploratory analysis suggested a modest effect of psychosocial skills training with respect to greater improvement than TAU on a measure of executive functioning (Color-Word Interference Inhibition). This measure of executive functioning requires naming the color of the print in which different color words are written (e.g., red, blue). Some print-word combinations are congruent (e.g., the word “blue” is in blue print) and others are incongruent (e.g., the word “blue” is in red print). For incongruent print-word combinations, respondents must inhibit a natural tendency to respond by reading the word. Performance on the test is associated with activity in the prefrontal and parietal cortex, and is impaired in clinical populations such as schizophrenia, Huntington’s disease, and traumatic brain injury. In the general population, performance on this test has also been found to decline with age(31) and to be sensitive to impairments in activities of independent living. (32)

The finding that psychosocial skills training may have produced modest improvement in executive functioning at the 3-year follow-up is novel and suggests a possible important line of future investigation. However, this result should be interpreted with caution for several reasons. First, the study sample lacked significant ethnic heterogeneity, potentially reducing the generalizability of the findings. Second, the comparison group consisted of usual care not an attention-control group. Third, the D-KEFS Color-Word Interference and Trails 4 tests were conceptualized as a measure of executive functioning, but may differ from other measures of executive functioning such as the Wisconsin Card Sorting Test. Most importantly, post-hoc analyses of specific cognitive functions (including executive functioning as measured by the Color-Word interference test) should be interpreted as exploratory and only suggestive of a possible relationship between psychosocial skills training and improved executive functioning. Our study sample was not adequately powered to detect small effect sizes with modest statistical significance in the context of multiple comparisons of cognitive outcomes. Hence, the statistically significant difference between HOPES and TAU on executive functioning may have been spurious and needs to be replicated. In addition, the effect size for this difference (d=.19) is small in magnitude.

However, the finding of a potential relationship between psychosocial skills training and modest improvement in executive functioning is potentially important in the context of findings on the effect of other interventions on cognitive functioning in SMI, as well as the long-term nature of the follow-up period in this study, and the advanced age of the study participants. For example, a meta-analysis of 34 studies examining the overall effect size of cognitive outcomes related to treatment with conventional antipsychotic medications was .22 for overall cognition for nine cognitive measures in schizophrenia, and only .13 for executive functioning. (33) Similarly, results from the CATIE trial examining the effectiveness of atypical antipsychotic medications in schizophrenia also found only small improvement in cognitive composite scores. (34) Pharmacological interventions specifically aimed at improving cognition in schizophrenia (e.g., cholinesterase inhibitor therapy) have not been associated with consistent improvements in cognitive functioning. (35) In contrast, a meta-analysis of 40 studies evaluating the effectiveness of cognitive remediation found a medium effect size (d=.45) for improved global cognitive functioning. (36) However, this review highlighted surprising results of a recent well-designed study that showed no effect (d=.06) of cognitive remediation for a sample that was distinguished by having the second oldest group of participants (mean age=48) among the studies evaluated. (37) This study, along with others reporting small or negligible effectiveness of cognitive remediation in older adults with SMI (36, 38) provides additional support for exploring the potential benefits of psychosocial skills training on cognition in older adults suggested by our report. It is possible that sustaining cognitive functioning (let alone detecting a potential small improvement in executive functioning) may be an important finding in older individuals with SMI who are at increased risk for age-related cognitive decline.

We are aware of only one other randomized psychosocial intervention trial that evaluated changes in cognitive functioning among middle-aged and older people with SMI. In this study (39) cognitive behavioral social skills training (CBSST) was not associated with improved cognitive functioning, including executive functioning (using a measure, the Stroop test, on which the Color-Word Interference test is based). Possible explanations for this difference in findings include the intensity of the interventions (HOPES lasted 52 weeks versus 24 weeks of CBSST), duration of follow-up (3 years versus 1 year), and study sample size (n=183 versus n=65).

There are several reasons why psychosocial skills training in an intervention such as HOPES may be associated with improved executive functioning. HOPES involves teaching social skills through repeated role-plays and feedback designed to shape more effective interpersonal behavior, and practicing skills during homework and in vivo community trips. A wide range of strategies are also used to facilitate learning new skills including inhibition of spontaneous responses, frequent review of material to enhance memory, teaching acronyms to help people remember steps required to perform target skills, and problem-solving to overcome barriers to using skills and achieving goals. HOPES group leaders encourage participants to think flexibly about how to use skills most effectively, to generate new ideas for situations where skills can be used, and to problem-solve barriers, all executive functions. Two prior observational studies have reported that exposure to psychosocial treatment was associated with greater improvements in cognitive functioning. (21, 40) However, these studies were not RCTs, limiting the inferences that could be drawn regarding the causal role of rehabilitation in producing positive cognitive changes.

HOPES participants did not demonstrate differential improvements on specific measures of attention, verbal fluency, or verbal learning and memory, domains that are often considered components or precursors of executive functioning. This finding suggests that the potential cognitive benefits of HOPES (if confirmed) come from improvements in higher level processing associated with tasks such as problem-solving or idea generation.

As expected, executive functioning was significantly related to psychosocial functioning at baseline and was related to changes in psychosocial functioning among the HOPES participants at 3-year follow-up. These associations tended to be strongest for the broader measures of psychosocial functioning. It was surprising that change in executive functioning was not related to change in scores on the UPSA, a performance-based measure of community living skills, given the strong cross-sectional relationship between cognitive functioning and UPSA scores, which improved differentially over time in the HOPES versus the TAU group. (2)

In summary, this report presents findings of an exploratory analysis of cognitive functioning outcomes from a randomized trial suggesting that a psychosocial skills training intervention for older adults with SMI may be associated with improvement in executive functioning comparable to that achieved with routinely used pharmacological treatments. Standardized programs of psychosocial rehabilitation are rarely used in routine treatment settings in spite of the evidence supporting their ability to improve psychosocial functioning. Given the age-related declines in cognitive functioning that older people with SMI may experience over time, and the relationship between cognitive and psychosocial functioning, it is important to identify psychosocial and pharmacological interventions that not only improve psychosocial functioning but also have the potential to enhance cognitive functioning. These results add to an emerging literature on pharmacological and cognitive remediation strategies and suggest directions for research on complementary approaches to improving cognitive and psychosocial functioning in older adults with SMI.