A Follow-up Study Examining the Durability of a Hybrid, Remote and Computer-Based Cognitive Remediation Intervention for Adolescents with 22q11.2 Deletion Syndrome

Abstract

Aim

Schizophrenia and 22q11.2 deletion syndrome (22q11DS) share similar patterns of cognitive deficits. Up to 30% of those with 22q11DS develop schizophrenia during early adulthood. As cognitive decline has recently been found to predict onset of psychosis in adolescents with 22q11DS, early interventions such as cognitive remediation (CR) during adolescence are warranted. This paper investigates the durability of a remote, computerized, CR program for youth with 22q11DS. Our aim was to determine if the positive effects of CR persisted 6 months beyond intervention completion.

Methods

A longitudinal design with 21 participants serving as their own controls was used. Youth were seen for neurocognitive assessments at pre-treatment, after the targeted 8-month intervention, at post-treatment, and 6 months after for follow-up. During the intervention, cognitive coaches met remotely with participants for CR via video conferencing 3 times a week, and offered task specific strategies. To determine if intervention improvements held across the 6 month follow-up period, neurocognitive measures were statistically examined with repeated measures ANOVAs from pre-treatment through follow-up.

Results

Our CR intervention proved durable. Post-treatment improvements comprising cognitive flexibility, executive function, reaction time, and working memory were maintained over the follow-up period.

Conclusions

Results confirm previous research regarding the durability of CR treatment and extend these findings to youth with 22q11DS. The present study may serve to inform early intervention efforts focused on cognitive and functionally relevant rehabilitation goals for youth with 22q11DS and suggests that 22q11DS can potentially serve as a suitable model for examining the trajectory preceding psychosis.

Introduction

Caused by a microdeletion of Chromosome 22 at band q11.2., 22q11.2 deletion syndrome (22q11DS) is a multiple-anomaly syndrome characterized by cognitive and behavioral deficits,^1–3 anxiety and mood disorders⁴ and an elevated risk for schizophrenia.^5–7 Schizophrenia and 22q11DS share similar patterns of cognitive deficits,⁸ with 22q11DS closely resembling the core schizophrenia phenotype in the general population.⁵

Onset of schizophrenia generally occurs during late adolescence/early adulthood.^9,10 Notably, up to 30% of those with 22q11DS develop schizophrenia during this transition period.^5–7,11 Neurocognitive deficits are an important element of both schizophrenia and 22q11DS, and longitudinal cognitive decline has been found to predict onset of psychosis in a large, multi-site study of adolescents with 22q11DS.¹² Accordingly, early interventions that focus on cognitive improvement particularly during the crucial period of adolescence when brain development is marked by the remodeling of neuronal connections are warranted.^9,13,14 More broadly, 22q11DS serves as a relevant model for examining the trajectory preceding onset of psychosis,^12,15 and for investigating cognitive development⁶ and early treatment interventions promoting the development of adaptive behaviors^15,16 across the spectrum of psychosis.

By applying learning principles and behavior training, cognitive remediation (CR) has proved effective in targeting and improving cognitive deficits related to schizophrenia.¹⁷ Cognitive improvement following CR has been reported in the areas of processing speed, working memory, verbal memory, problem solving, executive functioning, cognitive flexibility and social functioning for young adults and adolescent populations at risk for, or with, early onset schizophrenia.^10,18–21 Recent studies found computerized, CR interventions to be feasible and effective with similar populations^{18, 19, 22} as well as youth with 22q11DS.^23,24

Although CR appears to enhance cognition, in order to yield the highest impact, these effects must be durable and extend functioning.¹⁷ Earlier durability studies with adult schizophrenia patients concluded that training effects endured 6 months^25–27 and up to 1-year²⁸ after the cessation of CR treatment. The positive effects of CR for adolescents at risk for, or with, early onset psychosis have also been found to persist at 3 months^20,21, 6 months²², and 12 months post-intervention.²⁹ Although effects in schizophrenia are promising, to our knowledge, no durability studies investigating CR and adolescents with 22q11DS currently exist: thus the durability of CR effects in this population remains unknown.

The present paper extends our previous work²⁴ by investigating the durability of the effects of a remote, computerized hybrid CR program for youth with 22q11DS. Specifically, our goal was to determine if the salutary effects of CR on several key domains of cognition in youth with 22q11DS reported in our previous paper would persist at 6 months after the cessation of CR treatment. We hypothesized that, relative to cognitive performance immediately following the intervention, there would be little evidence of a decrease of treatment effects at a 6-month follow-up.

Methods

Participants

The background and behavior characteristics of the participants as well as a description of the CR intervention, implementation and preliminary treatment findings have been presented in detail by Mariano and colleagues²⁴. Briefly, 21 adolescents confirmed by fluorescence in situ hybridization (FISH) to have a 22q11DS diagnosis, were recruited through parent support groups and clinicians. Participants included 9 males and 12 females, with IQ's ranging from 63 to 94 (M = 76.85). Refer to Tables 1 and 2 for additional demographic and behavioral information. To avoid the confound associated with the presence of prodromal symptoms, participants were assessed via parent report during the recruitment phase for positive symptoms of psychosis (hallucinations or delusions) and excluded if they exhibited these symptoms. Participants were required to meet with a cognitive coach via video-conferencing for approximately three hours per week after school to employ the CR intervention. No one was lost to attrition. All 21 consecutive participants completed the intervention (M = 7.96 months; M = 92.95 sessions) and requisite cognitive assessments.

Table 1.

Demographics^†

Gender
Males	9
Females	12
Age
		T1‡	T2	T3	T4
Mean		14.61	15.21	15.95	16.47
Standard Deviation		1.28	1.56	1.52	1.52
Range		12.2–16.9	13.01–17.34	13.53–18.15	14.04–18.67
Parent SES§
Mean		45.74
Standard Deviation		12.66
Range		23–66
Full Scale IQ
Mean		76.85
Standard Deviation		8.98
Range		63–94

^†Data on Age, Parent Socio-Economic Status and Full Scale IQ at Time 1were previously reported²⁴

^‡T = Time

^§SES = Socio-Economic Status, Hollingshead (1975)³⁰

Table 2.

BASC-2 PRS^† Behavior Function Scores^‡

	Baseline (T1)§	Pre-treatment (T2)	Post-treatment (T3)	Follow-up (T4)
Externalizing Problems
Mean	54.00	55.00	52.11	52.62
Standard Deviation	12.34	12.23	12.31	11.91
Range	38–93	40–86	38–83	38–92
Internalizing Problems
Mean	59.30	61.95	59.37	60.05
Standard Deviation	14.15	13.72	14.07	12.81
Range	37–86	44–86	41–98	39–89
Behavior Symptoms Index
Mean	60.70	62.95	60.16	60.38
Standard Deviation	13.88	12.88	12.96	12.55
Range	37–94	41–87	38–87	41–98

^†Behavior Assessment System for Children-Parent Rating Scales (BASC-PRS)³¹

^‡Data for Time 1 through Time 3 Behavior Function scores were previously reported²⁴

^§T = Time

Measures

CNS Vital Signs (CNS-VS)

Cognitive skills were assessed with the CNS-VS computerized neurocognitive test battery. The battery consists of several tests adapted from often used neuropsychological assessments.^32,33 Test scores are combined into composite scores, reflecting cognitive areas such as reaction time, cognitive flexibility, complex attention, executive function, processing speed, and working memory. Stimuli are randomized making each presentation of the CNS-VS distinct and useful for repeated assessment.^32,33 Reliability and validity coefficients for the CNS-VS are comparable to those of other computerized neuropsychological batteries.³² Spanning all scores, test-retest reliability is moderate to good, (0.45 to 0.87), and correlation with conventional neuropsychological tests yielded moderate concurrent validity scores (r = .26 to .79, p < .05).³⁴

Cognitive Remediation Program

The CR intervention used with participants was Challenging Our Minds (COM). The COM system is an online, child/adolescent version of the CogRehab system³⁵ previously used with adults with schizophrenia^36,37 and adolescents with learning disabilities/attention deficit.³⁸ Throughout the intervention, participants are required to “pass” three levels of progressively difficult cognitive tasks as they proceed across six cognitive domains (attention, executive function, memory, visual-spatial abilities, problem solving, and communication). In our study, master’s level, trained cognitive coaches facilitated participant’s progress through the intervention using a hybrid coaching technique that included a mix of drill-and-practice exercises along with the provision of standardized strategies.

Design

Written, informed consent was provided by all participants/parents upon their initial visit and assessment at our lab. The study was IRB-approved by the research ethics board of our university. A comprehensive description of the research design is available in a previously published manuscript²⁴. In summary, a longitudinal design was utilized with participants serving as their own controls in order to examine the effectiveness of a remote, computerized, hybrid CR program. Youth were seen at our lab for cognitive assessment at 4 time points: baseline, pre-treatment, post-treatment, and as presented in the current study, follow up. Target intervention length was 8 months. Trained cognitive coaches met with participants (who were located in their homes through out the United States) three times a week, remotely, via web conferencing, and suggested task-specific strategies to facilitate their progress through the intervention. Participants were paid $10.00 per CR session. To ascertain whether significant gains after treatment on the cognitive measure persisted, participants were re-assessed at follow-up with the CNS-VS six months (M = 6.2) after the assessment at the cessation of treatment was administered.

Analysis

To determine if intervention improvements held across the follow up period, outcome measures from pre-treatment through follow-up were statistically examined using IBM SPSS. Standardized scores were used for all variables with outlier scores truncated to four standard deviations to reduce skewness.³⁹

For the current set of analyses, descriptive statistics for cognitive outcome measures were calculated for three timepoints: pre-treatment, post-treatment, and follow-up. In an effort to establish whether or not outcome measures changed significantly at follow–up, we conducted a repeated measures analysis of variance across each level (pre-treatment, post-treatment, follow-up) on five CNS-VS composite scores (i.e. cognitive flexibility, executive function, reaction time, working memory, complex attention) that had improved significantly between baseline and post-intervention²⁴. The repeated measures ANOVA examining composite scores was then re-run with the inclusion of IQ and gender as co-variates. Additionally, we carried out a repeated measures analysis of variance on the subtest scores from which the five composite scores were derived. Planned, follow-up t-tests were performed on composites and subtests to determine if any significant changes were due to change in pre-treatment to follow up scores or, post-treatment to follow-up scores. The Bonferroni correction for multiple comparisons was applied to the repeated measures ANOVA’s examining composite and subtest scores, as well as to the follow-up t-tests on both the composite and subtest scores.

Results

Previously, we reported on the feasibility, effectiveness and fidelity of this remote and computer based CR intervention with 22q11DS youth at baseline, pre-treatment and (immediately) post-treatment.²⁴ The current study investigates the durability of our CR program and includes new follow-up data collected 6 months after cessation of the intervention. In this section, we will provide a brief summary of our prior findings and present a new analysis of outcome measures across the follow-up period.

Initial findings associated with our previous report²⁴ suggested that administering our CR program to 22q11DS adolescents was feasible, that coaches consistently offered similar strategies, and participants demonstrated increased accuracy and decreased response time when responding to progressively complex stimuli. More specifically, neurocognitive findings from baseline to post-intervention indicated that adolescents with 22q11DS exhibited significant change on five CNSVS composite scores including cognitive flexibility, executive function, reaction time, working memory, and complex attention (ES = .47 – .70, p ≤. 002). Four out of the nine subtests from which these five composite scores were derived (working memory, shifting attention, two Stroop tasks) were also significant (ES = .36 – .55, p-value ≤. 014). Planned, follow-up comparisons of these subtests indicated the significant models were driven by improvements in pre-treatment to post-treatment scores only (p-values ≤ .009).²⁴

In the present study, all participants returned six months after their post-treatment assessment to complete the follow-up neurocognitive assessment. Data from pre-treatment through follow-up are presented. Based on a repeated measures of analysis variance, the significant gains previously reported²⁴ between baseline and post-treatment remained stable across the follow-up period for composites (Table 3) and the subtests from which the composite scores were derived (Table 4).

Table 3.

Composite Scaled Scores^† on CNS-VS from Pre-Treatment to Follow-Up^‡

Composites§	Pre-treatment Scores Mean (S.D.)	Post-treatment Scores Mean (S.D.)	Follow-Up Scores Mean (S.D.)	Repeated Measures ANOVA			t-test: Pre:Follow-up		t-test: Pre:Post‡			t-test: Post:Follow-up
				Wilks' Lambda	p-value*	Effect Size (η2)	t	p-value*	t	p-value**	t	p-value*
Complex Attention	76.7 (22.1)	86.0 (22.6)	89.4 (27.0)	0.60	0.008	0.399	−3.36	.003	−2.55	.012	−.890	0.383
Cognitive Flexibility	72.7 (20.5)	80.7 (15.9)	80.2 (21.3)	0.70	0.036	0.303	−2.08	.051	−2.83	.010	.510	0.616
Executive Function	75.0 (20.8)	83.7 (14.5)	81.8 (21.0)	0.71	0.036	0.295	−1.94	.067	−2.72	.013	.510	0.616
Working Memory	79.3 (17.1)	87.2 (17.2)	83.4 (16.8)	0.80	0.113	0.205	−1.54	.139	−2.26	.035	1.62	0.121
Reaction Time	68.9 (20.6)	81.0 (17.1)	74.6 (17.6)	0.50	0.001	0.497	−1.49	.154	−4.44	.001	1.62	0.120

^†Standardized scores used for analysis

^‡Pre-treatment and Post-treatment data previously reported²⁴

^§Composite scores of which Wilks’ Lambda was significant from Baseline:Post-treatment²⁴

^*Bonferroni-corrected p-value ≤ .01

^**p-value ≤ .01

Table 4.

Scaled Scores^† on CNS-VS Subtests from Pre-Treatment to Follow-Up^‡

Subtests	Pre-treatment Scores: Mean (S.D.)	Post-treatment Scores: Mean (S.D.)	Follow-up Scores: Mean (S.D.)	Repeated Measures ANOVA			t-test: Pre:Follow-up		t-test: Pre:Post‡		t-test: Post:Follow-up
				Wilks' Lambda	p-value*	Effect Size (η2)	t	p-value*	t	p-value**	t	p-value*
Continuous Performance Task: Commission Errors§	95.4 (18.1)	89.5 (25.8)	98.1 (21.6)	0.90	0.376	0.098	−.984	0.337	1.16	0.259	−1.45	0.162
Continuous Performance Task: Omission Errors	84.4 (26.2)	88.4 (22.1)	93.0 (20.8)	0.83	0.170	0.170	−1.78	0.092	−.717	0.482	−1.25	0.225
Working Memory¶, 2-Back: Correct Responses	81.6 (14.5)	90.8 (15.1)	86.2 (16.5)	0.66	0.018	0.344	−1.71	0.103	−3.24	0.004	1.67	0.111
Working Memory, 2-Back: Incorrect Responses§	87.4 (23.6)	90.3 (20.4)	89.33 (20.5)	0.99	0.866	0.015	−.398	0.695	−.551	0.588	0.267	0.793
Shifting Attention Task: Errors	85.6 (22.9)	97.3 (15.5)	96.1 (21.5)	0.65	0.016	0.352	−2.75	0.011	−2.91	0.009	3.19	0.753
Shifting Attention Task: Correct	70.5 (18.6)	77.4 (15.2)	77.1 (15.9)	0.76	0.073	0.241	−2.30	0.032	−2.29	0.033	0.118	0.907
Stroop: Commission Errors	83.0 (19.9)	81.4 (20.3)	85.2 (21.4)	0.95	0.614	0.050	−.685	0.501	0.341	0.737	−.886	0.386
Stroop: Congruent Condition, Reaction Time§ to Correct Responses	70.4 (20.1)	79.7 (18.5)	72.9 (17.2)	0.67	0.021	0.335	−.721	0.479	−3.06	0.006	1.75	0.479
Stroop: Incongruent Condition, Reaction Time to Correct Responses	72.5 (21.1)	85.9 (16.7)	80.1 (17.6)	0.58	0.006	0.417	−1.88	0.074	−3.77	0.001	1.43	0.168

^†Standardized scores were used for analysis

^‡Pre-treatment and Post-treatment data previously reported²⁴

^*Bonferroni corrected p-values ≤ .006

^**Bonferroni corrected p-values ≤ .01

^§As raw scores for errors/incorrect responses/reaction time decreased, scaled scores increased

^¶For clarity, we are referring to the CNS-VS, Four–Part CPT, as Working Memory since only the subtest score which measured Working Memory specifically, comprised the Working Memory domain.

Composite scores

Table 3 presents mean and standard deviations for composite scores that persisted from pre-treatment through follow-up. We observed a significant Wilks' Lambda score for complex attention, F(1,20) = 11.27, p = .008 due to slightly higher scores at follow-up. When we re-ran the model for composite scores with inclusion of age and gender as covariates, the results did not change significantly. Additionally, planned, follow-up comparisons for the composite scores previously reported as significant from baseline to post-treatment (complex attention, cognitive flexibility, executive function, working memory, and reaction time)²⁴ indicated that improvements were maintained at 6 months beyond the post-intervention assessment.

Subtest scores

A repeated measures ANOVA and follow-up comparisons conducted on the subtests comprising all composites examined did not yield significant results (Table 4). This was driven by the finding that the 4 out of 9 subtest scores that previously improved²⁴ were maintained and several subtests scores were slightly higher at follow-up than post-treatment. Participants made fewer errors on continuous performance tasks (complex attention) and a Stroop task (cognitive flexibility). In contrast, a minor decline in shifting attention (executive function/attention) and working memory tasks was observed. Participants also took more time at follow up than they did at post-treatment in correctly responding to incongruent and congruent Stroop-related tasks (reaction time) (Table 4).

Discussion

The results of the present investigation indicate that, in response to a remote, computerized CR program, post treatment improvements in cognitive flexibility, executive function, complex attention, reaction time, and working memory persisted over a 6 month follow-up period. Although intervention methods across other studies varied, our findings are in accord with prior research examining adolescents at risk for, or with, early onset psychosis. Wykes et al.,²¹ found a significant effect for cognitive flexibility across both post-treatment and follow-up (3 months) for participants who received CR versus those who had received treatment as usual. In a recent, randomized, controlled CR study applied individually (40 sessions) to adolescent out-patients using paper and pencil tasks, Puig et al.,²⁰ (2014) reported that participants maintained cognitive gains in executive functions, verbal and working memory, and overall composite cognitive scores at 3 months post CR intervention. Compared to baseline measures only, cognitive improvements in reasoning and executive function were found 6 months following an 8–week/16 session computer-assisted CR intervention among another group of in-patient adolescents with, or at risk for psychosis.²²

In the present study, only the composite score for complex attention demonstrated improvement from pre-treatment to follow–up. However, the maintenance of outcome measures 6 months beyond program completion was anticipated; improvement in composite and subtest scores from post-treatment to follow up was not expected in the absence of targeted treatment during the follow-up interval. Consequently, our results substantiate prior research regarding the maintenance of CR-related improved cognitive abilities for adolescents at risk for psychosis and in addition, broaden the range of previous durability findings by examining pre, post and 6 month follow-up data from a longer-course, remote, computerized CR treatment program with a comparably-sized intervention sample.

It is possible that the intensity and duration of the hybrid-strategy treatment approach employed contributed to the persistence of training effects. Fiszdon et al.²⁶ suggested that durability of training effects in their study may have been due to real-world practice during the training and follow-up period. Since participants in our study were enrolled in middle and high school throughout the 8-month intervention, it may be that they remembered and used certain strategies that they were taught during treatment in their home and educational settings. In turn, continued strategy use and practice during follow-up could have contributed to sustained performance on the cognitive assessment measure 6 months after the intervention ceased.

Implications

Aside from durability, the ultimate goal of CR is generalization to functional skills¹⁷ and when combined with other rehabilitative modalities, CR does appear to impact functioning.^17,40,41 In fact, CR in early course schizophrenia has been shown to positively affect, maintain, and increasingly improve functioning up to one year after being implemented.^{40,42, 43}

Though the link between cognition and functional outcome is complex and mediated by both social and functional competency skills, it has been demonstrated that neurocognitive variables relate to specific functional domains of work/productivity and daily living skills in individuals with schizophrenia.^44–47 The associations between cognitive and functional outcomes have similarly been demonstrated for several of the cognitive skills that improved and persisted in our sample. For example, cognitive flexibility,⁴⁸ executive function,^21,44,47 processing speed,⁴⁷ and working memory have previously been shown to impact functioning in patients with schizophrenia.

Given the efficacy of using CR with adolescents who have 22q11DS,^23,24 along with research supporting that neurocognitive variables underlie functional outcomes for individuals with schizophrenia,^45,46 it appears plausible that using CR with adolescents diagnosed with 22q11DS may subsequently lead to improved functioning. Combined with other durability studies, our findings at follow-up regarding the maintenance of training effects are noteworthy in this regard. As educational and vocational demands for adolescents with 22q11DS increase and become more abstract, cognitive skills related to those environments could become more necessary. After CR treatment, the cognitive abilities required for education and work may maintain and continue to develop, particularly when given the opportunity to use and practice skills linked with those domains in real-world settings.^41,49,50 The results of the current investigation are important primarily because they further confirm and extend the findings regarding durability of CR treatment to the 22q11DS population. However, the present study may also inform treatment efforts focused on cognitive and vocationally relevant rehabilitation goals for adolescents with 22q11DS as they transition to adulthood, potentially mitigating the possible course of psychosis.

Limitations

The present investigation provides support for the durability of CR training in adolescents with 22q11DS, yet, it has its limitations. Due to our method of recruitment, our sample size was small: accordingly, these findings may not be representative of the entire 22q11DS population and should be replicated with a larger sample. Potential moderators including motivation, functional, behavioral and emotional capacities, access to parental, academic supports, strategy use/application, and educational experience may have also affected our results. Although it might be argued that practice effects could have contributed to the current study's results, the tasks comprising the cognitive assessment measure used at follow-up differed from the CR treatment tasks. Some scores decreased slightly over the follow-up period, and it's possible that they will continue to decrease, particularly without continued practice or, additional CR training. Therefore, we don't know the long term durability for all the composites measured. However, our findings are promising within the context of the previous research discussed. Future research of CR treatment with adolescents with 22q11DS involving a longitudinal, randomized, experimental design including strategy coaching/no coaching and functional skills components, along with cognitive, behavioral and real-world skills assessment beyond 6 months post-treatment, might serve to clarify other factors involved in the delivery, effectiveness, durability, and generalization of CR to real-world environments.

Despite these limitations, the results of the current study support our hypothesis that training effects of a remote, computerized hybrid CR program for youth with 22q11DS did endure 6 months post-treatment. Future studies that examine the long-term effects of CR and the extent to which cognitive skills potentiate improved functioning in 22q11DS are critical. Given the under-treatment of psychiatric disorders in 22q11DS,⁵¹ it is also necessary to promote strategies that pre-empt or mitigate potential mental health issues during adolescence.¹⁶ Coupled with a need for more evidence regarding the neuroplastic/neuroprotective benefit of CR in potentially shifting the path of schizophrenia,⁵² this and future studies with adolescents with 22q11DS are warranted to further evaluate CR as an early intervention and a means of investigating the trajectory of psychosis.