Role of learning potential in cognitive remediation: construct and predictive validity

Abstract

Background

The construct, convergent, discriminant, and predictive validity of Learning Potential (LP) was evaluated in a trial of cognitive remediation for adults with schizophrenia-spectrum disorders. LP utilizes a dynamic assessment approach to prospectively estimate an individual's learning capacity if provided the opportunity for specific related learning.

Methods

LP was assessed in 75 participants at study entry, of whom 41 completed an eight-week cognitive remediation (CR) intervention, and 22 received treatment-as-usual (TAU). LP was assessed in a “test-train-test” verbal learning paradigm. Incremental predictive validity was assessed as the degree to which LP predicted memory skill acquisition above and beyond prediction by static verbal learning ability.

Results

Examination of construct validity confirmed that LP scores reflected use of trained semantic clustering strategy. LP scores correlated with executive functioning and education history, but not other demographics or symptom severity. Following the eight-week active phase, TAU evidenced little substantial change in skill acquisition outcomes, which related to static baseline verbal learning ability but not LP. For the CR group, LP significantly predicted skill acquisition in domains of verbal and visuospatial memory, but not auditory working memory. Furthermore, LP predicted skill acquisition incrementally beyond relevant background characteristics, symptoms, and neurocognitive abilities.

Conclusions

Results suggest that LP assessment can significantly improve prediction of specific skill acquisition with cognitive training, particularly for the domain assessed, and thereby may prove useful in individualization of treatment.

1. Introduction

The term “learning potential” was initially applied to the assessment of training potential and educability of individuals with low IQs (Budoff & Friedman, 1964). Learning potential (LP) is quantified using a dynamic assessment approach intended to represent one's ability to quickly learn and apply a new skill under testing conditions, typically utilizing a “test-train-test” format. Dynamic assessment captures change in performance over time that occurs as a result of brief training, and in this way differs from static assessment, which is based on test performance at a single occasion without specific training. While static assessment captures current ability, it may not capture cognitive capacity. As such, dynamic LP assessment may inform prediction of future potential given adequate opportunity for learning or, in clinical context, intervention (Grigorenko, 2009).

Some forms of LP assessment share similarities with compensatory skills training used in cognitive remediation (CR), where capacity to perform a given task can be enhanced by training and practice in the use of strategies that reduce cognitive load. These and other approaches of CR have been found generally efficacious for individuals with schizophrenia-spectrum disorders (SSD) (McGurk et al., 2007; Wykes et al., 2011), but substantial variability in treatment outcome within and between studies raises questions about what individual differences influence response to treatment (Radhakrishnan et al., 2015). In particular, little is known about how background characteristics related to learning capacity influence treatment outcome (Green et al., 2000; Green et al., 2015; Kurtz, 2012). Given the time- and labor-intensive nature of CR, there is value in identifying variables that predict treatment efficacy and that could be used to inform and personally tailor treatment.

LP measures have been found to relate to readiness to learn in individuals with SSD (e.g., Fiszdon et al., 2006; Rempfer et al., 2011). LP has predicted treatment outcome incrementally above prediction by static factors in vocational rehabilitation (Sergi et al., 2005; Watzke et al., 2008; Watzke et al., 2009) and CR (Boosman et al., 2014; Wiedl & Wienobst, 1999). Previous research indicates that LP may not relate strongly to skills developed in the absence of related interventions (e.g., Green et al., 2015; Kurtz et al., 2010; Tenhula et al., 2007; Vaskinn et al., 2008). We suggest the utility of LP as a predictor of outcome will depend both on the capacity being assessed and the opportunity to develop that capacity through specific training.

Given that the “train” portion of the “test-train-test” in LP assessment often directly mirrors essential components of learning that contribute to efficacy of CR, effective measures of LP may have unique predictive power, beyond other background characteristics, as predictors of CR outcomes. The utility of LP assessment has been questioned in some cases due to limited relationship to functional status (Green et al., 2015), however, different methods of quantifying LP assessments have been shown to greatly influence their strength of association with other measures (Fiszdon & Johannesen, 2010). Thus, in determining the predictive utility of LP measures, one should also consider basic psychometric aspects of the derived LP score as well as construct validity with respect to outcome domain.

The current study examined the utility of LP assessment as a predictor of individual differences in skill acquisition during CR training among individuals with SSD, as well as LP's construct (convergent and discriminant) validity and incremental predictive validity. LP was assessed using a test-train-test administration of the California Verbal Learning Test-II (CVLT-II; e.g., Fiszdon et al., 2006) prior to an eight-week course of CR. The present report focuses specifically on LP psychometrics and CR skill acquisition.

Four principal hypotheses were tested: (H1) As evidence of convergent construct validity, LP score will reflect use of trained strategy (semantic clustering); (H2) As evidence of discriminant construct validity, LP score will not be highly correlated with baseline measures of neurocognition or background illness characteristics; (H3) As evidence of predictive validity, LP will predict skill acquisition achieved through CR training, and (H4) As evidence of incremental predictive validity, the relationship between LP and skill acquisition will remain after controlling for baseline neurocognitive and background characteristics, as identified in H2.

2. Methods

2.1 Participants

Data presented here are part of a study of cognitive remediation (CR) efficacy and predictors of outcome in SSD. Data from a subsample (N=43) was previously used in a comparison of LP computation methods (Fiszdon & Johannesen, 2010).

Volunteers with schizophrenia-spectrum diagnoses were recruited from outpatient clinics of VA medical center and local community clinics. Inclusion criteria were as follows: age 18-65, psychiatrically stable (no hospitalizations, changes in medications, or changes in housing in past 30 days), no substance abuse in past 30 days, and no evidence of serious traumatic brain injury or neurological disorder. Following baseline assessment, participants were randomly assigned (2:1 ratio) to CR (n=50) or to treatment as usual (TAU; n=25). The study was approved by local Institutional Review Boards and all participants provided written informed consent prior to initiating any study procedures.

2.2 Intervention

The CR intervention consisted of up to five hours of weekly training over 8 weeks. Individuals were randomized to complete 4 weeks of computerized training, focused on memory and attention (PSS CogReHab; Bell et al., 2001) prior to or following 4 weeks of individualized compensatory cognitive training (CRT; Delahunty et al., 2001). Individuals randomized to TAU continued to receive their usual psychiatric and psychological services.

2.3 Measures

A comprehensive assessment battery was administered at intake and at the end of 8 weeks. Variables collected included background characteristics that have been found to predict skill acquisition in CR, including age, gender, education, IQ, symptom severity, reasoning and problem solving, and attention (Fiszdon et al., 2005; Kurtz, 2012; Medalia & Richardson, 2005; Scheu et al., 2013; Twamley et al., 2011).

2.3.1 Independent variables

Learning Potential (LP) was assessed using methods described in Fiszdon et al. (2006). Briefly, participants completed three administrations of the California Verbal Learning Test-II (CVLT-II), using three different stimulus sets of 16 words each, referred to hereafter as List 1 (pre-LP-train), List 2 (LP-train), and List 3 (post-LP-train). List 1 and 3 were administered using standard CVLT-II procedures, with List 2 administered as a “train” condition, involving instruction on semantic clustering strategies. Specifically, participants were shown how semantic grouping can improve recall, asked to practice using semantic grouping during all five trials of List 2, and given corrective feedback as needed.

LP was indexed by the extent of change in test performance from List 1 (pre-LP-train) to List 3 (post-LP-train) based on the regression residual scoring approach (Fiszdon & Johannesen, 2010), computed by regressing List 3 score (Trial 1-5 total) on List 1 score (Trial 1-5 total) and expressed as a standard score (z-score). Using this approach, individual scores represent the difference in List 3 (post-LP-train) performance relative to expected performance based on List 1 (pre-LP-train), with positive values indicating greater than expected improvement. The residual score provides an index of LP that effectively controls for differences in List 1 (i.e., pre-LP-train) performance and has distinct psychometric advantages compared to simple difference scores (Fiszdon and Johannesen, 2010; Guthke, 1982; Weingartz et al., 2008).

2.3.2 Clinical measures

Verbal learning was assessed using the California Verbal Learning Test – II (CVLT-II). Trial 1-5 total and semantic clustering scores, respectively, were used in analysis of incremental and construct validity. Reasoning and problem-solving was assessed using the Wisconsin Card-Sorting Task (WCST; Heaton, 1981) - standardized scores for percent errors and percent perseverative errors. Attention and information-processing was assessed using the Continuous Performance Task (CPT X/A; Loong, 1991). Premorbid IQ was estimated using the Wide Range Achievement Test 3 - Reading Subscale (WRAT-R; Johnstone et al., 1996; Wilkinson, 1993), and current IQ was assessed using the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999), two-subtest version. Illness severity was assessed using four variables: (1) age of onset; (2) age first hospitalized; (3) lifetime number of hospitalizations; and (4) Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987; Bell et al., 1994) symptom rating.

2.3.3 Skill change outcome measures

Skill acquisition over the 8 week active phase was assessed based on pre-post performance scores on computerized cognitive tasks (CogReHab Software; Bracy, 1995) used in the CR intervention. Training administered in the active intervention involved a number of cognitive domains from which 3 memory tasks, differing in proximity to the domain of LP assessment (i.e., verbal learning), were selected. Digits Auditory is a progressive digits forward recall task assessing attention and working memory, which starts with 3 digits, and the number of digits presented increasing by one after each successful trial. Digits Auditory score is the total number of correctly recalled digits throughout the task. Shape Place is a spatial memory task in which multiple shapes are briefly displayed on a grid and once they disappear, participants are asked to select the shapes they saw and place them in their correct location on the grid. Shape Place score is the total number of shapes correctly recalled and placed throughout the task. Verbal Memory is a verbal learning task in which participants are asked to sort twenty words into prompted categories, study the words, and correctly select target words from a list that includes non-targets and semantic foils. Verbal Memory Correct is the percentage of correct words selected, and Categorizing score is the percentage of semantic grouping used in identifying correct words. These same performance measures were administered in the TAU condition at intake and following 8 weeks without training.

2.4 Data Analysis

Primary analyses were carried out across the full sample, collapsed by treatment condition for tests of skill acquisition. Given hypothesis that LP would predict change in related skills only for those involved in targeted treatment, condition (CR vs. TAU) was included as a factor in hypothesis testing. Between-group differences in demographic and background variables were investigated using t-tests for normally-distributed continuous variables and Kendall's tau-b for dichotomous and ordinal variables.

2.4.1 Hypotheses 1 & 2: internal and construct validity

As a test of convergent construct validity, a primary analysis examined the extent to which LP score depended on increased utilization of semantic clustering, in accord with the LP training strategy. This was tested by first regressing LP on List 1 (pre-LP-train) Semantic Clustering score, and then also on List 3 (post-LP-train) Semantic Clustering (controlling for List 1). The regression weight for List 3 Semantic Clustering represents the unique covariance in LP accounted for by change in semantic clustering strategy usage compared to pre-LP training.

Next, as a test of discriminant validity, relationships between LP and clinical variables previously shown to influence efficacy of CR in SSD (Fiszdon et al., 2005; Kurtz, 2012; Medalia & Richardson, 2005; Scheu et al., 2013; Twamley et al., 2011) were assessed by Pearson's r for normally-distributed continuous variables and Spearman's Rho for ordinal variables.

2.4.2 Hypotheses 3 & 4: incremental validity in predicting skill acquisition

Main analyses tested the predictive relationship of LP to skill acquisition over the 8-week active study phase. Individual differences in skill acquisition were represented as the regression residual of post-treatment on intake scores for each of the three computerized training tasks (Allison, 1990). Intake and 8-week scores were inspected for normality. Shape Place was winsorized (Tukey, 1962) prior to creating residuals due to two outliers whose performances at all occasions were much higher than all other participants (>95^th percentile). Change scores are notated using delta (“Δ”).

Separate forward-entry linear regression models were constructed for each dependent measure of skill acquisition. First, main effects aggregated across conditions were estimated by entering: CVLT-II List 1, then LP; where model change and LP regression weights represent unique incremental prediction of skill change by LP over standard CVLT-II test score. Next, the following variables were entered: CVLT-II List 1, Condition (CR vs. TAU), then LP, to test if the incremental relationship of LP to DVs is dependent on condition. In a final analysis, clinical variables were entered in the first block, with subsequent steps following as described above, and these predictors were retained if they contributed significantly to the model. Finally, models with significant ΔR² values for LP were estimated separately for TAU and CR samples.

3. Results

In the present sample, CR and TAU did not differ on key demographic and clinical characteristics (Table 1). Baseline CVLT-II List 1 total (pre-LP-train) and List 3 total (post-LP-train) had excellent internal reliability (List 1 alpha=0.93; List 3 alpha=0.90) and were highly correlated at r=0.75 (p<0.001). LP correlated significantly with List 3 (r=0.66, p<0.001), while LP and List 1 were, by construction, uncorrelated.

Table 1. Participant characteristics by treatment group.

	CR (n=50)	TAU (n=25)	Group difference
Age	47.3 (9.1)	48.9 (9.9)	t=0.67, p=0.50
Gender (%female)	21.6%	37.5%	tau-b=0.17, p=0.17
Education (years)	12.5 (1.8)	12.1 (2.3)	t=-0.70, p=0.49
IQ	92.5 (19.1)	88.1 (19.4)	t=-0.92, p=0.36
Premorbid IQ (T-score)	43.9 (9.7)	43.3 (11.1)	t=-0.20, p=0.83
PANSS:
Positive	12.1 (4.1)	11.5 (4.4)	t=-0.54, p=0.59
Negative	13.1 (4.4)	14.2 (4.9)	t=0.97, p=0.38
Cognitive	12.5 (3.0)	13.1 (3.8)	t=0.78, p=0.44
Emotional	8.8 (3.3)	8.4 (3.1)	t=-0.58, p=0.56
Hostility	5.5 (2.0)	5.0 (2.2)	t=-0.79, p=0.43
Age of Onset	20.8 (6.4)	22.0 (7.5)	t=0.65, p=0.51
# Hospitalizations	13.1 (17.4)	14.0 (24.3)	t=0.18, p=0.86
Age first hospitalized	25.5 (9.5)	23.4 (7.7)	t=-0.92, p=0.36
Medication Dosage (CPZ equivalent)	380.2 (377.2)	311.6 (263.7)	t=-0.74, p=0.46

3.1 LP internal and construct validity

Semantic Clustering at List 1 (pre-LP-train) was not related to LP (R²=0.03, p=0.16), but adding List 3 (post-LP-train) Semantic Clustering to the model predicted LP significantly (ΔR²=0.23, p<0.001). The full model demonstrated a suppressor effect, with Semantic Clustering List 1 Beta=-0.56 (p=0.003) and List 3 Beta=0.87 (p<0.001), suggesting that participants who did not utilize Semantic Clustering at List 1 but did use this strategy at List 3 improved more in number of words recalled from List 1 to List 3 than did participants whose Semantic Clustering did not change.

Further, LP residual correlated with WCST Percent Errors (r=0.27, p=0.02) and Percent Perseverative Errors (r=0.22, p=0.05) at a trend level. LP residual was not correlated with attention and information processing (CPT X/A; r=0.09, p=0.44). LP residuals did not relate significantly to age, gender, premorbid IQ, current IQ, age of onset, age first hospitalized, or number of hospitalizations (p's>0.05). Participants with higher education levels had higher LP (Rho=0.30, p=0.01). Symptoms were not related to LP (PANSS Total; r=-0.18, p=0.12).

3.2 LP incremental validity in predicting skill acquisition

Notably, the treatment groups were not equivalent on CVLT-II performance at intake. CR performed substantially better than TAU on List 1 total (t(60)=-2.26, p=0.03), with this group difference increasing after the 8 week active phase (t(60)=-4.22, p<0.001). Slopes were not significantly different, but numerically greater for CR (CR change=5.3; TAU change=1.6; time*group F=2.61, p=0.11). Although the present study examines individual differences in response controlling for level of performance within groups, group comparisons should be interpreted with this confound in mind. TAU participants did not change, on average, on computerized training tasks. Neither List 1 nor LP entered into models predicting skill acquisition in Verbal Memory Categorizing, Shape Place, or Digits Auditory for TAU. Nonetheless, in the TAU condition, change scores on these measures varied (see Table 2), and Verbal Memory Correct change was strongly predicted by CVLT-II performance (Table 4). This suggests that performance changed to a meaningful degree for some participants' in the TAU condition in the absence of targeted treatment.

Table 2. Skill change residual scores: means and standard deviations by condition.

Group	ΔVerbal Memory	ΔVerbal Categorizing	ΔShape Place	ΔDigits Auditory
CR	6.52 (15.68)	5.81 (18.45)	2.30 (7.77)	2.08 (8.47)
TAU	-12.14 (15.29)	-10.82 (10.83)	-4.28 (4.51)	-3.88 (6.33)

Table 4. LP residual and Verbal Memory Correct^a skill change models for CR and TAU.

Variable	B	SE(B)	Ɓ	ΔR2	Variable	B	SE(B)	Ɓ	ΔR2
CR					TAU
Step 1				0.12	Step 1				0.32*
WCSTb	0.26	0.15	0.28		WCST	0.28	0.21	0.25
List 1	0.42	0.65	0.11		List 1	1.85	0.73	0.48*
Step 2				0.12*	Step 2				0.04
WCST	0.17	0.15	0.19		WCST	0.24	0.22	0.21
List 1	0.57	0.61	0.15		List 1	2.06	0.75	0.54*
LP	6.23	2.56	0.36*		LP	2.87	2.78	0.21
Total				0.24*	Total				0.36*

^*p<0.05;

^**p<0.01;

^***p<0.001;

^aVerbal Memory Categorizing, Shape Place, or Digits Auditory did not have a significant R² for the TAU group. So, separate effects by group are only reported for Verbal Memory Correct;

^bWCST Percent Errors standard score.

In bivariate correlations aggregated across conditions, baseline List 1 and List 3 performances were related to skill acquisition for all outcomes (List 1 p's<0.001, List 3 p's<0.05) except Digits Auditory. LP score did not correlate significantly with skill acquisition without accounting for treatment condition (p's from 0.06 for Verbal Categorizing to 0.68 for Digits Auditory). These predictors were entered in multiple regression to test the hypothesis that LP score would predict skill acquisition after accounting for CVLT-II performance (List 1) and treatment condition.

Parameters of regression models for skill acquisition are presented in Table 3, and separate models for Verbal Memory Correct in CR and TAU are presented in Table 4. Other skill acquisition DVs are only described in text, as the total R² values were not significant for TAU. Aggregated across groups (without condition in the model), change in Verbal Memory Correct was positively related to both List 1 (Ɓ=0.44, p<.001) and LP (LP residual Ɓ=0.25, p=0.02). This relationship depended on condition (illustrated in Figure 1). When the two conditions were examined separately, in the TAU condition skill change was unrelated to LP, but was significantly related to List 1 performance. For CR, Verbal Memory Correct change (illustrated in Figure 2) related to LP score and was not dependent on List 1 performance.

Table 3. LP residual and skill change regression models for full sample.

Variable	B	SE(B)	Ɓ	ΔR2	Variable	B	SE(B)	Ɓ	ΔR2
ΔVerbal Memory Correct					ΔShape Place
Step 1				0.17***	Step 1				0.19***
List 1	1.76	0.50	0.41**		List 1	0.78	0.20	0.44***
Step 2				0.16***	Step 2				0.09**
List 1	1.22	0.48	0.28*		List 1	0.61	0.21	0.34**
Condition	15.41	4.14	0.42***		Condition	4.96	1.79	0.32**
Step 3				0.08**	Step 3				0.06*
List 1	1.34	0.46	0.31**		List 1	0.65	0.20	0.36**
Condition	15.80	3.93	0.43***		Condition	5.10	1.73	0.33**
LP	5.14	1.86	0.28**		LP	1.89	0.82	0.25*
Total				0.40***	Total				0.34***
ΔVerbal Categories					ΔDigits Auditory
Step 1				0.06ˆ	Step 1				0.05
List 1	1.06	0.34	0.25		List 1	0.44	0.25	0.22
Step 2				0.15**	Step 2				0.09*
List 1	0.52	0.52	0.12		List 1	0.25	0.25	0.13
Condition	15.24	4.51	0.41**		Condition	5.31	2.17	0.31*
Step 3				0.08*	Step 3				0.01
List 1	0.65	0.50	0.15		List 1	0.27	0.25	0.14
Condition	15.64	4.31	0.42***		Condition	5.36	2.18	0.31*
LP	5.32	2.04	0.29*		LP	0.77	1.03	0.09
Total				0.29***	Total				0.14*

^*p<0.05;

^**p<0.01;

^***p<0.001.

Figure 1. LP and verbal memory change in CR vs TAU groups.

Figure 2. Verbal memory change by LP (tertile) in CR group only.

Aggregated across groups, change in Verbal Memory Categorizing was positively related to both List 1 (Ɓ=0.28, p=0.03) and LP (Ɓ=0.27, p=0.03). When conditions were examined separately, for TAU, neither List 1 nor LP contributed significantly to the model. For CR, skill acquisition was moderately related to LP and was not dependent on List 1 performance.

Of the computerized CR outcomes, change in Digits Auditory was not related to LP or verbal learning (List 1). Only treatment condition entered significantly into these models.

Aggregated across groups, change in Shape Place related positively to List 1 (Ɓ=0.46, p<.001) and LP (Ɓ=0.23, p=0.04). When conditions were examined separately, for TAU, neither List 1 nor LP contributed significantly. For CR, skill acquisition related to LP and the unique contribution of List 1 performance remained significant.

Additional models were estimated to test if LP's incremental prediction persisted after accounting for relevant clinical variables, retained for model testing if found to contribute significantly to predicting skill acquisition in any domain. Of the clinical predictors, only reasoning and problem solving (WCST % errors) entered significantly. The relationships between LP and skill acquisition by treatment group remained significant after controlling for differences in reasoning and problem solving, and regression weights did not change substantially. Model parameters are presented in Table 5. Further, exploratory analyses showed relationships did not change after testing contributions of age, gender, premorbid IQ, current IQ, and attention by manual entry in predictive models

Table 5. LP residual and skill change models controlling for significant background clinical variables.

Variable	B	SE(B)	Ɓ	ΔR2	Variable	B	SE(B)	Ɓ	ΔR2
ΔVerbal Memory Correct					ΔShape Place
Step 1				0.21***	Step 1				0.09*
WCSTa	0.47	0.12	0.46***		WCST	0.13	0.05	0.30*
Step 2				0.07*	Step 2				0.12**
WCST	0.36	0.12	0.35**		WCST	0.07	0.05	0.16
List 1	1.19	0.51	0.28*		List 1	0.67	0.22	0.38**
Step 3				0.09**	Step 3				0.07*
WCST	0.24	0.12	0.23		WCST	0.02	0.05	0.05
List 1	0.94	0.49	0.22		List 1	0.58	0.22	0.32**
Condition	12.50	4.31	0.34**		Condition	4.68	1.92	0.30*
Step 4				0.06*	Step 4				0.06*
WCST	0.17	0.12	0.17		WCST	-0.01	0.05	-0.02
List 1	1.13	0.48	0.26*		List 1	0.66	0.21	0.37**
Condition	13.69	4.18	0.37**		Condition	5.19	1.87	0.33**
LP	4.49	1.90	0.25*		LP	1.91	0.85	0.25*
Total				0.42***	Total				0.34***
ΔVerbal Memory Categorizing					ΔDigits Auditory
Step 1				0.18***	Step 1				0.06
WCST	0.43	0.12	0.42***		WCST	0.12	0.06	0.24
Step 2				0.01	Step 2				0.02
WCST	0.39	0.13	0.38**		WCST	0.09	0.06	0.19
List 1	0.44	0.54	0.10		List 1	0.30	0.27	0.15
Step 3				0.08*	Step 3				0.06*
WCST	0.28	0.13	0.27*		WCST	0.04	0.07	0.09
List 1	0.20	0.53	0.05		List 1	0.20	0.26	0.10
Condition	11.87	4.68	0.32*		Condition	4.80	2.32	0.28*
Step 4				0.06*	Step 4				0.01
WCST	0.21	0.13	0.20		WCST	0.03	0.07	0.07
List 1	0.39	0.52	0.09		List 1	0.23	0.27	0.12
Condition	13.07	4.57	0.35**		Condition	4.98	2.35	0.29*
LP	4.52	2.08	0.24*		LP	0.65	1.07	0.08
Total				0.32***	Total				0.15

^*p<0.05;

^**p<0.01;

^***p<0.001;

^aWCST Percent Errors standard score.

4. Discussion

The present study aimed to examine validity of Learning Potential (LP) assessment and how LP might relate to skill acquisition in individuals with schizophrenia-spectrum disorders. LP demonstrated construct validity, reflecting utilization of semantic clustering. LP was related to education and reasoning and problem solving but not other background characteristics such as attention, demographics, symptoms, or IQ, taken as evidence in support of discriminant construct validity. LP was related to skill acquisition during eight weeks of CR on measures of verbal memory and associated use of semantic strategies, as well as on a measure of visual-spatial skill acquisition. These relationships remained significant after controlling for static verbal learning and background characteristics, supporting incremental predictive validity.

Previous studies of LP as predictor of treatment response in SSD have typically demonstrated incremental validity above static baseline predictors (Fiszdon et al., 2006; Rempfer et al., 2011; Sergi et al., 2005; Wiedl et al., 2001). The present study replicated these findings, particularly for predicting CR-associated change in verbal memory, and extended previous findings by demonstrating construct validity of LP assessment and its domain specificity for predicting skill acquisition. Overall, the data support important theoretical points in dynamic assessment and CR, specifically: 1) treatment response is heterogeneous, and treatment development and individualization require looking beyond the mean change score for treatment vs. control (Peer et al., 2008); 2) LP scores add to the understanding of this heterogeneity only in the context of a learning opportunity (Grigorenko, 2009); 3) consistent with the existence of different learning styles and types of learning, LP may not measure “general learning potential” but may instead be domain-specific (Kozhevnikov, 2007; Nadel and Hardt, 2011; Squire, 1992), and 4) the predictive utility of LP assessment is likely dependent not only on the relationship between LP domain and outcome, but also psychometric considerations of how the LP score is computed (Fiszdon and Johannesen, 2010).

The relationship between LP with List 3 but not List 1 Semantic Clustering score suggests that LP training response in fact indicates response to specific compensatory strategy training, supporting its methodological validity. This result also highlights the difference between static performance and ability to improve. Specifically, when both scores were considered, higher LP was predicted by lower initial semantic clustering strategy use and greater semantic clustering strategy use after strategy training, suggesting LP assessment effectively identifies participants with both room and capacity to improve in a brief trial of compensatory strategy training.

LP predicted greater skill acquisition only for those in treatment (CR) and not in TAU. The specificity of LP to CR skill acquisition suggests LP may be a useful tool to predict for whom CR may be most effective beyond what can be predicted by clinical background and static tests of neurocognitive ability and may represent a unique component of cognitive capacity related to responsiveness to active treatment (Fiszdon et al., 2006). In the present sample, this differential predictive validity was exemplified by change in Verbal Memory Correct, the skill most proximal to the domain of LP assessment. For TAU, only static verbal learning performance (List 1) predicted change in post-test scores, while for those receiving CR, only LP predicted improvement. Therefore, static verbal learning was the best predictor of skill change in the absence of specific skill training, and dynamic LP was the best predictor for skill acquisition that occurs in the context of skill training. This is consistent with the idea that dynamic assessment is most useful preceding a learning opportunity, and may be relatively insensitive to change occurring naturally outside of training. Further, LP was only strongly related to acquisition of verbal memory, measured with a task that shares several characteristics with the LP training. This suggests that the predictive validity of LP may be limited to acquisition of skills similar to those trained during the LP assessment.

Limitations of this study include a relatively small and unbalanced sample for predicting individual differences, baseline differences in test performance that were not annulled by randomization, and limitations of repeated CVLT-II assessment (e.g., fatigue). Furthermore, as we excluded individuals who reported substance abuse in the past 30 days, our findings may not generalize to some individuals with schizophrenia-spectrum disorders who are actively using drugs or alcohol (Whiteford et al., 2013). Effort and motivation may also play a significant role in LP performance, and examining these relationships would likely clarify the specific contribution of strategy learning to LP (e.g., Reddy et al., 2015).

Evidence-based practice compels therapists and clinical researchers to consider the prognostic value of individual differences to make informed, data-driven clinical decisions (Lilienfeld et al., 2014; Spring, 2007). During CR, computer programs and human providers make decisions that may be informed by the existing research and present data on learning potential. The present data focus on verbal learning and reflect the heterogeneity of skill acquisition during CR, a small portion of which can be predicted by static, historical, and dynamic factors (Boosman et al., 2014; Cella et al., 2015; Medalia and Choi, 2009; Radhakrishnan et al., 2015). In light of present results, one might speculate that a participant with high static verbal learning may not benefit from additional compensatory training in this domain (Paquin et al., 2014; Scheu et al., 2013). On the other hand, low static verbal learning along with flat or negative scores on the present learning potential assessment might indicate cognitive impairment involving both the primary substrates of verbal learning as well as frontally-mediated executive abilities that are recruited by the compensatory strategy (i.e., semantic clustering), suggesting a different profile of impairment from individuals with low initial verbal learning whose performance does improve following strategy training. It is reasonable to suggest that rehabilitation of these two profiles could involve different targets for CR augmentation or order of staging treatments (Cella et al., 2015).

Future directions include improving psychometric validation of LP measurement, developing briefer LP assessment techniques, identifying consensus techniques across research groups, and translating LP assessment to inform clinical decisions. Techniques for neurocognitive LP assessment are not limited to the CVLT-II and WCST (Boosman et al., 2014), and it may be possible to develop dynamic assessments in other domains to predict different types of skill acquisition. For example, existing brief intervention-like emotion regulation techniques could be adapted for use as dynamic assessments (Maldonado et al., 2015; Scott et al., 2015; Watkins et al., 2015). Further, proof-of-concept single-session social cognitive training techniques could provide dynamic assessment of theory of mind (Kayser et al., 2006; Sarfati et al., 2000), affect recognition (Combs et al., 2008; Penn & Combs, 2000; Russell et al., 2008; Silver et al., 2004), and social perception (Corrigan & Toomey, 1995). This diversity of dynamic assessment possibilities may also lend itself to a multitrait-multimethod or pattern-matching validation approach (Campbell and Fiske, 1959; Trochim, 1989). The research potential for brief single-session trainings and their use in dynamic assessment is large and mostly untapped.