Engagement with the auditory processing system during targeted auditory cognitive training mediates changes in cognitive outcomes in individuals with schizophrenia

Abstract

BACKGROUND

Individuals with schizophrenia who engage in targeted cognitive training (TCT) of the auditory system show generalized cognitive improvements. The high degree of variability in cognitive gains maybe due to individual differences in the level of engagement of the underlying neural system target.

METHODS

131 individuals with schizophrenia underwent 40 hours of TCT. We identified target engagement of auditory system processing efficiency by modeling subject-specific trajectories of auditory processing speed (APS) over time. Lowess analysis, mixed models repeated measures analysis, and latent growth curve modeling were used to examine whether APS trajectories were moderated by age and illness duration, and mediated improvements in cognitive outcome measures.

RESULTS

We observed signifcant improvements in APS from baseline to 20 hours of training (initial change), followed by a flat APS trajectory (plateau) at subsequent time-points. Participants showed inter-individual variability in the steepness of the initial APS change and in the APS plateau achieved and sustained between 20–40 hours. We found that participants who achieved the fastest APS plateau, showed the greatest transfer effects to untrained cognitive domains.

CONCLUSIONS

There is a significant association between an individual's ability to generate and sustain auditory processing efficiency and their degree of cognitive improvement after TCT, independent of baseline neurocognition. APS plateau may therefore represent a behavioral measure of target engagement mediating treatment response. Future studies should examine the optimal plateau of auditory processing efficiency required to induce significant cognitive improvements, in the context of inter-individual differences in neural plasticity and sensory system efficiency that characterize schizophrenia.

INTRODUCTION

Over the past year, the National Institute of Mental Health has implemented an “experimental medicine approach” for clinical trials, in which interventions serve not only as potential treatments, but as “probes to generate information about the mechanisms underlying a disorder” (Reardon, 2014). In this model, researchers must determine whether a given biomedical or psychological intervention exerts measurable and predicted effects on a well-defined target, one which reflects underlying mechanisms of action central to the psychiatric disease process under study (Insel, 2013). In addition, and perhaps most critically, studies must seek to relate measures of target engagement to clinical outcomes—thus validating (or refuting) the underlying scientific rationale for use of the intervention (Insel, 2014).

The definition of treatment target and mechanism of action may be straightforward when evaluating pharmacologic or even neuromodulatory treatments, but it becomes more challenging when considering psychological or behavioral interventions such as cognitive training, which usually have not been developed with neuroscientifically-informed “mechanisms of action” in mind and which, by definition, will engage multiple neurocognitive targets simultaneously. Indeed, Keshavan et al. have suggested that the next step for the development of cognitive training for psychiatric disorders is “to identify relevant distributed neural system targets in which training will induce the widest possible range of generalized clinical and behavioral improvement”—while also recognizing that no such intervention will ever address only one single neural system target (Keshavan, Vinogradov, Rumsey, Sherrill, & Wagner, 2014).

In this study, we explicitly apply the experimental medicine model to behavioral data obtained from three independent trials we have conducted using the same, well-defined, and highly specific form of cognitive training undertaken by 131 individuals with schizophrenia. We present the rationale, hypothesized mechanisms of action, and targeted neural systems underlying the training; summarize the data regarding efficacy of training; and define a behavioral measure of “neural system target engagement”. We then delineate target engagement in response to training and examine how target engagement mediates changes in cognitive outcomes. Our goal is to demonstrate that it is possible to develop measures of target engagement in cognitive training studies, based on the underlying scientific rationale and hypothesized mechanisms of action, and that it is possible to evaluate the relationship of target engagement to outcome with the aim of understanding moderators and mediators of treatment response. Such an approach is critical if we are to develop personalized behavioural interventions in psychiatry (“What lies beneath,” 2014).

Experimental medicine rationale for cognitive training exercises that target auditory processing and auditory working memory

Schizophrenia is characterized by disturbances in verbal learning systems that can be observed independently of positive and/or negative symptoms throughout the course of illness. Abnormalities in fronto-temporal networks have been documented during auditory and verbal encoding, working memory, and episodic and semantic memory (Ragland et al., 2009; Wolf et al., 2007). Disturbances of these language processing networks are present in antipsychotic-naïve individuals at the first psychotic episode (Hill, Beers, Kmiec, Keshavan, & Sweeney, 2004), precede the emergence of psychotic symptoms (Dutt et al., 2015), and predict the transition from prodromal to first episode psychosis (Sabb et al., 2010). These disturbances are paralleled by dysfunctional neural activity at the earliest stages of auditory processing (Cadenhead, Light, Shafer, & Braff, 2005; Solís-Vivanco et al., 2014). An established body of evidence suggests that early impaired feed-forward/ feedback operations in auditory working memory contribute to widespread cognitive and psychosocial impairments in schizophrenia (Adcock et al., 2009; Dale et al., 2010; Light, Swerdlow, & Braff, 2007; Javitt, 2009). Training that targets these interacting feedback and feed-forward neural operations during auditory and verbal processing is likely to induce a positive impact on cognition in schizophrenia (Vinogradov, Fisher, & de Villers-Sidani, 2012) (see also Figure 1).

Figure 1. Applying the experimental medicine model to targeted cognitive training of auditory/verbal processing in schizophrenia.

Schizophrenia is characterized by impaired feed-forward/ feedback operations in auditory working memory that are associated with cognitive dysfunction. Training is thus designed to improve the perception and temporally-detailed resolution of auditory inputs that feed forward into working memory operations and verbal encoding, while providing feedback engagement of attention and cognitive control operations. We hypothesize that engagement of this training target-- interacting feedback and feed-forward operations-- will be associated with enhanced cognitive performance, particularly in verbal learning and memory.

The form of targeted cognitive training (TCT) we have studied has been reported by us previously (Fisher, Holland, Merzenich, & Vinogradov, 2009; Fisher, Holland, Subramaniam, & Vinogradov, 2010; Fisher et al., 2014). It consists of an intensive schedule of computerized exercises that simultaneously target feedback and feed-forward operations in the auditory system. To improve the efficiency of feed-forward auditory perceptual processes and the temporally-detailed resolution of auditory inputs, training places implicit, increasing demands on auditory perception. “Feedback” attention and cognitive control operations are engaged by requiring the learner to signal ready at the start of each trial, by signaling correct/incorrect trials, and by embedding the psychophysical training within a suite of increasingly complex auditory and verbal working memory/verbal learning exercises. Exercises continuously adjust difficulty level to maintain an 80%–85% correct individual performance rate, ensuring a dense reward schedule while driving the individual to their learning threshold (Vinogradov et al., 2012).

The mechanism of action is thus posited to be the “re-tuning” of the bi-directional operations between perceptual representations in auditory cortex and prefrontal attention and auditory/verbal working memory functions. Indeed, our emerging magnetoencephalographic data indicate that TCT enhances both early auditory representations in primary auditory cortex as well as both early and later task-related activity in prefrontal sectors (Dale et al., 2010; Dale et al., 2015).

Heterogeneity of efficacy of this form of training

Studies reported to date indicate that there is a great deal of variability in the response to this form of intensive auditory system training (Fisher et al., 2009; Fisher et al., 2014; Keefe et al., 2012; Popov et al., 2011). For example, in the two RCTs from our group, while we obtained significant between-group differences in the MATRICS Global Cognition scores at the end of training, ~40% of participants did not show gains beyond expected practice effects (0.2 SDs) (Keefe et al., 2011). In the trial conducted by Murthy and colleagues, 43% of participants did not show cognitive gains and also did not show changes in auditory processing speed greater than 40 ms—suggesting that they had not engaged with the training. This led the researchers to identify this group as “non-learners,” as compared to those who did show learning effects both during baseline assessments and cognitive improvements after exposure to the TCT (Murthy et al., 2012).

This variability in treatment response in not surprising in the context of psychiatric treatment development, where it is increasingly clear that an effective intervention is not equally effective for all subjects in a given population (Chmura Kraemer, Kiernan, Essex, & Kupfer, 2008; Leucht, Hierl, Kissling, Dold, & Davis, 2012). There is increasing recognition of the need to study individualized treatment response and within-group heterogeneity. Understanding how the intervention works (mediators) and who responds and who does not (moderators) can prompt researchers to optimize and personalize the intervention -- making it more efficacious and cost-effective-- and to develop alternative strategies for non-responders. Thus, in order to determine critical mediators and moderators of response to cognitive training in the service of developing more personalized interventions, we sought to identify measures of neural system target engagement, and to evaluate the relationship of target engagement during training to cognitive outcome after training.

Defining a behavioral measure of target engagement for auditory system training

According to the experimental medicine model, the engagement of the neural system target – in this case, training-induced enhancement of auditory processing efficiency and auditory working memory operations- should translate to improved performance in untrained cognitive outcome measures. A wealth of data indicates that changes in performance on a trained task are associated with the degree of neuroplasticity that has occurred in the targeted neural system (Adcock et al., 2009; Bor et al., 2011; Haut, Lim, & MacDonald, 2010; Hooker et al., 2013; Ramsay & MacDonald, 2015; Subramaniam et al., 2014); thus such behavioral changes can reasonably serve as proxies for neural system target engagement. Based on the hypothesized mechanism of action of auditory TCT, as well as neurophysiological findings from our own group (Dale et al., 2016), we decided to measure auditory processing speed (APS) longitudinally over the course of training to identify metrics of target engagement that show a relationship to cognitive outcomes after training. APS is an index of auditory psychophysical efficiency, indicative of both auditory perceptual and attentional operations. APS is operationalized as performance threshold on the most basic of the training exercises in the program, a time-order judgment task of a sequence of 2 frequency-modulated (FM) sound sweeps (Supplementary Figure 1). In this exercise, participants are asked to identify the direction of tonal change in a sequence of two successive FM sound sweeps, as either “up” (from a lower to a higher pitch) or “down” (from a higher to a lower pitch). This exercise aims to improve successive signal interference (forward and backward auditory masking), and improvement in this ability during training demonstrates that an individual is generating improvement in auditory processing efficiency. We note that during the 40–50 hours of exercises that are part of the auditory TCT module we have studied, participants train heavily on this basic sound sweeps exercise during their first 20 hours of training, but less so later in the module, as they progress to other more complex auditory working memory exercises.

Prior studies indicated that APS improves robustly between baseline and 20 hours of TCT (Murthy et al., 2012, Keefe et al., 2012), and that these improvements are associated with gains in global cognition after 40 hours (Fisher et al., 2014), with no additional improvements in APS after 20 hours. Confirmatory analyses in larger samples are needed to clarify whether the engagement of the targeted neural system that drives generalized cognitive gains can be captured by the initial APS improvement and/or by the attainment and retention of a critical auditory processing efficiency, as indexed by an APS flat trajectory. In order to answer this question, we adopted a data-driven approach to identify measures of target engagement: first, we used mixed models repeated measure analysis to characterize the trajectory of APS in a sample of 131 individuals with schizophrenia undergoing auditory TCT. APS metrics obtained from these models were then tested as possible mediators of training-induced cognitive improvements using structural equation modeling. We hypothesized that APS metrics would capture dynamic changes of auditory system processing efficiency, which in turn would reflect training-induced plasticity in distributed prefrontal-temporal systems.

MATERIALS AND METHODS

Participants

We performed analyses on a combined data set of 131 participants who have been recruited and studied as part of three independent RCTs of TCT: ClinicalTrials.gov NCT00694889 (recent onset schizophrenia, N= 61, age 21.3±3.8, study completed and cognitive outcome data reported in Fisher et al. 2014), NCT01988714 and NCT02105779 (chronic schizophrenia, N=70, age 45.3±10.7, studies ongoing). Participants with recent onset schizophrenia (RO) were drawn from two research programs at the University of California, San Francisco and University of California at Davis. Participants with chronic schizophrenia (CSZ) were drawn from research programs at the San Francisco VA Medical Center and at a community mental health center. All participants carried a diagnosis of schizophrenia, schizophreniform, or schizoaffective disorder as determined by the Structured Clinical Interview for the DSM-IV TR Axis I disorder interview (First, MB., Spitzer, RL, Gibbon M, and Williams, 2002). Participants were clinically stable at the time of testing (no hospitalization within the past 3 months and stable dose of medication over the past month). Other inclusion criteria included: 1) good general physical health, 2) fluent and proficient in English, 3) IQ ≥ 70, 4) no DSM-IVA TR diagnosis of substance dependence 5) no active substance abuse during recruitment, assessment, or training, and 6) no neurological disorder.

Procedures

All participants gave written informed consent or assent for the study and were compensated for their participation in all assessments. Participants were asked to complete 40 hours (aiming for 1 hour per day for 5 days each week) of computerized auditory TCT provided by Posit Science Corporation, mean intervention time was 34.1 hours (SD = 14.6). The TCT program has been described in detail previously (Adcock et al., 2009). It consists of six computerized exercises designed to improve speed and accuracy of auditory information processing and auditory working memory. Participants rotated through 4 exercises each day of training, with training on more elemental auditory processing (including the time-order judgment task) more heavily weighted during the first 20 hours, and higher-level verbal and whole-language exercises emphasized during the second 20 hours. All participants received $5 for each day of study participation, a bonus $20 for 5 consecutive days of participation, and a bonus $50 at the completion of the training module. Payment was contingent on study participation and not performance. After an intake evaluation that determined study eligibility, participants underwent a battery of cognitive and clinical assessments, which were repeated at the end of training. CSZ participants completed training using desktop computers and staff exposure during the intervention was kept to a minimum: staff aided all participants to start each session but did not provide any coaching. In the RO study, participants were loaned laptop computers and participated in the intervention at home, except for 1 subject who preferred to participate in the laboratory. While in the trial, participants received treatment, as clinically indicated, by outside providers or clinic personnel not involved in the study (psychoeducation, psychotherapy, supported employment, adjustments in medications as clinically indicated). Demographic, clinical, and training characteristics are presented in Table 1.

Table 1.

Demographics, training adherence variables, and baseline clinical and functional scores.

	Mean/N	SD/%
Demographics
Age (years)	34.11	14.55
Education Level	12.6	2.36
IQ	101.18	12.54
Duration of illness (years)	9.68	14.89
Age at symptom onset	20.12	7.47
Sex
Male	94	71.80%
Female	37	28.20%
Race
Caucasian	64	48.90%
Black or African American	18	13.70%
Asian American	35	26.70%
Other	14	10.70%
Ethnicity
Hispanic or Latino	10	7.60%
Not Hispanic or Latino	117	89.30%
Unknown	4	3.10%
Training Adherence
Total Hours of Training	34.10	10.57
Hour of Training per Week	3.09	1.79
Clinical Symptoms
PANSS
Total score	2.12	0.49
Positive symptoms	2.26	0.99
Negative Symptoms	2.45	0.97
Disorganized Symptoms	2.05	0.69
Excitement Symptoms	1.55	0.50
Depression Symptoms	2.49	1.17
Others	2.00	0.58
GAF total score	46.28	9.915

APS and Cognitive, Clinical, and Functional Outcome Measures

APS Measure

APS, defined as described above was determined using a method based on the Zippy Estimation by Sequential Testing (ZEST) algorithm, an adaptive Bayesian procedure for determining sensitivity measures (i.e. estimating threshold). The ZEST algorithm adaptively modifies the interstimulus interval (ISI) between the two FM sound sweeps and the sweep duration, which is held equal to the ISI, as the performance changes trial by trial. The Bayesian procedure terminates after 100 trials. The resulting APS score is the number of milliseconds of ISI (and sweep duration) at which the subject correctly performs 66% of trials, allowing for a measure of psychophysical threshold under moderate perceptual challenge, as outlined in prior research on threshold assessments in auditory processing (Cacace & McFarland, 2013). APS was measured at baseline, and after 20, 30, and 40 hours of training.

Cognitive outcome measures

An abbreviated battery of Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS)-recommended measures was administered (Nuechterlein et al., 2008). The abbreviated battery included assessment of the following domains: speed of processing (Trail Making Test Part A, category fluency animal naming), working memory (letter-number span, WMS-III spatial span), verbal learning and verbal memory (HVLT-R immediate and delayed recall), visual learning and visual memory (BVMT-R immediate and delayed recall), problem solving (NAB Mazes). For the RO subgroup, the Tower Test from the Delis-Kaplan Executive Function System (D-KEFS) was used in place of the Neuropsychological Assessment Battery (NAB) Mazes (Delis, Kramer, Kaplan, & Holdnack, 2004). Alternate forms of the Hopkins Verbal Learning Test-Revised (HVLT-R) and BVMT-R were administered and counterbalanced at baseline and post-training. Cognitive assessment staff was trained and monitored at each site on manualized assessment procedures by the same senior researcher (M.F.) to ensure cross-site consistency. Staff completed extensive training on scoring criteria of individual items (e.g., scoring videotaped sessions, observation of sessions conducted by experienced staff, and participating in mock sessions). Raw scores were converted to z scores using age-appropriate normative data provided in testing manuals and age-appropriate, published normative data. All primary outcome measures were distinct and independent from tasks practiced during training.

Clinical and functional outcome measures

All participants were assessed at baseline with the 35-item version of the Positive and Negative Symptoms Scale (Kay, Fiszbein, & Opler, 1987). Five symptom dimensions were derived from the PANSS scores: Positive, Negative, Disorganized, Excitement, Depression and Anxiety, and Others (Van den Oord et al., 2006). Functional outcome was measured using the Modified Global Assessment of Functioning (mGAF) scale (Hall, 1995). Clinical assessment staff were trained and observed by expert clinical supervisors at each site.

Data analytic plan

We performed an intent-to-treat analysis on all subjects (N = 131), regardless of hours of intervention. 15 individuals (11.5%) completed less than 20 hours of TCT. Cognitive follow-up data were not collected from these individuals.

First, we used Spearman's rank correlation coefficients to assess the associations between participant age and duration of illness with baseline APS measures. Next, we conducted an initial locally-weighted regression (lowess) analysis to characterize the relationship of APS measures and time non-parametrically. The lowess analysis suggested that APS improved from baseline to 20 hours, followed by a flat APS trajectory from 20 hours through 40 hours (see Figure 2). We then conducted mixed models repeated measures (i.e., HLM) analyses to determine how APS trajectories changed significantly over time. We fitted an initial mixed model containing two spline terms, with the first term representing the APS change from baseline to 20 hours and the second term representing a linear spline for the APS measurements at 20, 30, and 40 hours. This model allows APS to increase or decrease from baseline to 20 hours, and from 20 to 40 hours. Random effects were included for the intercept, the APS first spline, the APS second spline, and were allowed to correlate. Participant age, duration of illness, and study group were included in the model as covariates.

Figure 2. Trajectory of auditory processing speed (APS) over time, as indexed by APS scores at baseline, and after 20,30 and 40 hours of auditory TCT (n=131).

Error bars represent standard errors of the means.

We then extended this mixed models analysis to explore whether age, duration of illness, and study group moderated the APS spline slopes. We added the interactions of each of these three covariates with APS spline slopes to the initial model to determine if they significantly interacted with the main effects. It is in fact theoretically possible that interactions of the slopes with one or more covariates might be significant, even if the main effects of the two APS splines are not. Results from this analysis showed no evidence for interaction of the covariates with the two APS splines. In light of the results of the lowess analysis, we then tried to fit a second reduced version of the first mixed model, with the first spline still representing the APS change from baseline to 20 hours (hereafter referred to as “initial APS change”), and the second linear spline removed from the analysis, which yields a zero slope for APS trajectories from 20 hours through 40 hours (hereafter referred to as “APS plateau”). A likelihood ratio (LR) test was performed to test whether this second, more parsimonious model fitted significantly worse than the first model.

The simpler model fitted the data no worse than the more complex initial model, and was used to conduct latent growth curve (LGC) modeling, which allows us to assess whether the subject-specific APS trajectories mediate changes in cognitive outcome measures after training. We first fitted an unconditional LGC model using data from all 4 time points. In this analysis, APS measures were rescaled to have metrics similar to those of the cognitive variables: each APS variable was divided by 10 to facilitate model convergence. In line with multilevel modeling, we set the residual variances of the distal variable across time to be equal, which also facilitated model stability and convergence. We then conditioned the LGC model for each MATRICS domain separately: we regressed initial APS change and plateau onto baseline cognitive performance, and then added in regressions of end-of-study MATRICS scores onto the initial APS change, the APS plateau, and the corresponding baseline cognitive performance. For each path, the unstandardized regression coefficient B and its significance level are reported along with the corresponding standardized regression coefficient β. As an example of the LGC model used in this study, Figure 3 illustrates the path diagram of the relationship between APS change over time and the MATRICS Global Cognition baseline and follow-up scores. The evaluation of the LGC model fit was based on current recommendations (Hu & Bentler, 1999). The goodness-of-fit of the LGC models was assessed using the χ2 test of exact model-data fit. A model was acceptable when χ2≥0.05. Aside from the χ2 statistic, we also report three commonly-used descriptive fit indices of global model fit. If any two of the following three conditions are met, the model also fits the data well on an approximate basis: Comparative Fit Index (CFI) values greater than or equal to .95, Standardized Root Mean Square Residual (SRMR) values less than or equal to .08, and Root Mean Square Error of Approximation (RMSEA) values less than or equal to .06 (Hu & Bentler, 1999). The mixed model and LGC models included all available data from all subjects automatically under maximum likelihood estimation, and cases with incomplete data were assumed to have missing values arising from either a missing completely at random (MCAR) or missing at random (MAR) missingness mechanism. We thus retained all available data for analysis.

Figure 3.

Latent Growth Curve (LGC) model for the target engagement measure of auditory processing speed (APS) over time, conditioned for MATRICS Global Cognition (GC) scores (N = 130).

Finally, Spearman's rank correlations were used to assess the association of initial APS change and APS plateau with years of education, total hours of training, training intensity and cognitive changes. The Mann-Whitney test was used to examine the association of gender with initial APS change and APS plateau. For correlational analyses, APS plateau was calculated by averaging across APS measures at 20, 30 and 40 hours.

Analyses were performed with an alpha level of 0.05, and bidirectional tests were used for all of the hypotheses. We used SPSS 21 for preliminary analyses, Stata's –lowess- command to perform the lowess analysis, Stata's -mixed- command for the mixed models analyses, and Mplus 7.3 for LGC.

RESULTS

Baseline APS showed a high degree of inter-individual variability, as did APS at each of the following 3 time-points (Figure 2). Baseline mean APS was positively correlated with age (r=.234, p=.008) and duration of illness (r=.241, p=.006). Age and duration of illness were positively correlated (r=.871, p=.000).

The lowess regression showed a substantial reduction in mean APS from baseline to 20 hours, but no change in APS performance after 20 hours. The first mixed models analysis found that APS decreased significantly from baseline to 20 hours (B = −35.86, p < .001), but there was no statistically significant change from 20 hours to 40 hours (B = 0.46, p = .82). There were no differences due to study group (B = 7.83, p = .72), age (B = 1.83, p = .06), and illness duration (B = −0.01, p = .83).

We then added interactions of the three covariates (group, age, and illness duration) to the initial main-effects model. No significant interaction effects were found for initial APS change-by-group (B = 28.32, p = .25), initial APS change-by-age (B = 0.03, p = .98), and initial APS change-by-illness duration (B = .03, p = .72), APS plateau-by-group (B =−4.46, p = .54), APS plateau-by-age (B = −.13, p = .70), and APS plateau-by-illness duration (B = −.03, p = .18). Taken collectively, these results suggest that study group, age, or illness duration do not moderate the effects of time on initial APS change and APS plateau (all p-values are .18 or larger). Additionally, initial APS change and APS plateau did not show significant associations with gender, years of education, total hours of training or training intensity.

Due to the non-significant effect of time from 20 hours through 40 hours, and the lack of moderators, a second simplified mixed model omitting the APS plateau linear spline term was fitted to the data. The results from this analysis were very similar to those from the first mixed model: APS scores decreased from baseline to 20 hours (B = −35.10, p < .001) and there were no statistically significant effects for study group (B = 9.08, p = .68), age (B = 1.91, p = .06), and illness duration (B = −0.005, p = .95). The likelihood ratio (LR) test comparing the fits of the two models was not statistically significant (χ2(4) = 5.14, p = .27), suggesting that the simpler model fitted the data no worse than the more complex initial model, and could be used to build LGC models to assess mediation.

All LGC models fitted the data well on an exact, as well as approximate basis. Please see Table 2 for β coefficients. In each LCG model, baseline cognitive scores predicted their follow-up counterpart. Positive β coefficients indicate that, for each MATRICS domain, better baseline performance was predictive of better follow-up performance. We also found that baseline global cognition, speed of processing, verbal working memory, visual working memory, verbal learning, verbal memory, visual memory, and problem solving performances predicted APS plateau. Negative beta coefficients suggested that lower baseline cognitive performance was predictive of a slower subject-specific APS plateau. APS plateau predicted in turn global cognition, speed of processing, verbal working memory, visual working memory, visual learning, visual memory, and problem solving outcomes, after accounting for the corresponding baseline performances. For these domains, a slower APS plateau reached after 20 hours of training predicted lower training-induced cognitive gains.

Table 2. Latent growth curve models coefficients.

For each cognitive domain, model standardized regression coefficients β are reported. APS is auditory processing speed. Distal 1 is the MATRICS baseline assessment on that measure (e.g. baseline global cognition), Distal 2 is the MATRICS post-training assessment (e.g. post-training global cognition). The model fit indexes reported here are the χ² test, the Comparative Fit Index (CFI), the Standardized Root Mean Square Residual (SRMR) and the Root Mean Square Error of Approximation (RMSEA).

Distal Variable	Global cognition	Visual Working Memory	Verbal Working Memory	Verbal Learning	Verbal Memory	Visual Learning	Visual Memory	Speed of processing	Problem solving
	β	β	β	β	β	β	β	β	β
Covariance Structure
APS plateau on Distal1	−0.41***	−0.25*	−0.29**	−0.38***	−0.36***	−0.15	−0.18*	−0.34***	−0.34**
Initial APS change on Distal 1	−0.04	0.11	−0.16*	−0.03	0.04	−0.05	−0.01	−0.02	−0.15
Distal2 on APS plateau	−0.15*	−0.22**	−0.29***	−0.03	0.02	−0.23***	−0.30***	−0.20***	−0.27*
Distal 2 on Initial APS change	0.01	0.13*	−0.02	−0.03	0.03	0.02	−0.02	−0.03	−0.19
APS plateau's variance	0.84***	0.94***	0.92***	0.86***	0.87***	0.98***	0.97***	0.88***	0.88***
Distal 2 on Distal 1	0.80***	0.54***	0.64***	0.76***	0.65***	0.69***	0.56***	0.66***	0.39***
Initial APS change's variance	1.00*	0.99*	0.97*	1.00*	1.00*	1.00*	1.00*	1.00*	0.98*
APS residual	0.11***	0.11***	0.11***	0.11***	0.11***	0.11***	0.11***	0.11***	0.11***
Distal 2 variance	0.24***	0.56***	0.40***	0.41***	0.59***	0.43***	0.53***	0.44***	0.64***
Mean Structure
APS plateau's intercept	0.73***	1.20***	1.00***	0.73***	0.81***	1.11***	1.11***	0.97***	1.09***
Initial APS change's intercept	0.53***	0.61***	0.43***	0.53**	0.62***	0.53***	0.58***	0.58***	0.52***
Global Model Fit
N	130.00	130.00	130.00	131.00	131.00	130.00	130.00	130.00	130.00
X2(df)	16.94 (13)	19.85 (13)	19.04 (13)	20.50 (13)	15.22 (13)	17.85 (13)	19.71 (13)	18.97 (13)	20.85 (13)
X2 p-value	0.20	0.10	0.12	0.08	0.29	0.16	0.10	0.12	0.08
CFI	0.98	0.97	0.97	0.97	0.99	0.98	0.97	0.98	0.96
RMSEA	0.05	0.06	0.06	0.07	0.04	0.05	0.06	0.06	0.07
SRMR	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08

Notes: Initial APS change's variance and APS plateau's variance are the residual variance estimates for the random initial APS change and APS plateau latent variables, respectively. The APS residual is the level 1 residual for APS and is set to a common estimate in line with multilevel modelling. The Distal 2 variance is the residual variance estimate for the Distal 2 variable.

^*p < .05;

^**p < .01;

^***p < .001.

In the LGC models, initial APS change was not predicted by baseline cognitive performances (except for verbal working memory) and did not predict cognitive outcomes, after accounting for the baseline performance. This finding was consistent with non significant Spearman's rank correlations between changes in cognitive scores and initial APS change. Because of the close-to-statistically-significant age effect in the mixed models (p=.06), we investigated whether including age as a covariate would alter the LGC results. Age was non-significant as a covariate in the LGCs and results for LGCs did not change with age included vs. not included. Therefore we decided to show in Table 2 the more parsimonious and more readily interpretable LGCs without age included.

Finally, to stimulate future research, we present descriptively the estimated mean APS plateau achieved by participants who demonstrated an improvement in MCCB Global Cognition that is greater than what would be predicted due to practice effects (0.2 SD, N = 56). In this subgroup of participants, we observed a mean APS plateau of 83.44 ms, with a SD of 69.69 ms. While the unconditional LGC model from which these estimates were obtained converged, it did not fit the data on an exact or an approximate basis (Hu & Bentler, 1999), due to the relatively small sample size (N = 56).

DISCUSSION

Modeling the APS trajectory over time permits us to operationalize target engagement of the auditory system in response to targeted cognitive training

In this study, we investigated whether modeling the trajectory of changes in auditory processing speed (APS) over time (along with their relation to cognitive outcome measures) could help identify behavioral metrics of auditory system target engagement, in individuals with schizophrenia who underwent 40 hours of targeted cognitive training (TCT) of auditory perceptual processing and working memory.

First, we found robust improvements, as well as significant inter-individual heterogeneity, in the APS measure after 20 hours of training, but no additional significant changes at 30 or 40 hours of training, consistent with the multi-site feasibility study of Keefe and colleagues (Keefe et al., 2012). As noted earlier, the sequence of exercises used in these studies trains heavily on the sound sweeps exercise during the first 20 hours, but only intermittently during the last 20 hours of training, which focuses on more complex auditory and verbal working memory exercises. Hence, we cannot ascertain whether the lack of further significant improvement in APS after 20 hours is due to the fact that participants have reached an asymptote in their psychophysical learning curve, or is due to the lack of exposure to a heavy dose of APS training during the last 20 hours. Nonetheless, the rapid, large improvement seen in APS in the first 20 hours appears to reflect the “steep” part of the learning curve during training exposure, and the APS plateau reached between 20 hours and 40 hours appears to capture important information about how well an individual has engaged in a sustained manner (and possibly consolidated) the neurocognitive target. These findings are consistent with the perceptual learning literature in healthy participants, which shows massive changes in performance after exposure to ~8 hours of auditory discrimination training, followed by threshold stabilization (Menning, Roberts, & Pantev, 2000). It is possible that the APS plateau may be discernable as early as after 8–10 hours of training in individuals with schizophrenia, and could indicate which individuals are successfully harnessing and sustaining plasticity mechanisms very early after exposure to TCT. If true, APS plateau could be used in future fast-fail approaches to quickly determine treatment uptake for a given individual.

We find no obvious moderators of target engagement

While Spearman's rank correlations indicated that age and duration of illness were positively correlated with mean APS scores at baseline, findings from the mixed models showed that initial APS change and APS plateau were not moderated by study group (chronic patients in VA or community mental health settings vs. recent-onset individuals in university clinics), age, or illness duration. In addition, age as a covariate did not alter significantly the parallel LGC mediation models for cognitive variables. Taken together, these findings suggest that, while age and duration of illness affect baseline APS, they do not appear to have a significant impact on the potential for target engagement and for response to training. These data are consistent with meta-analyses of cognitive remediation in schizophrenia (McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007; Wykes, Huddy, Cellard, McGurk, & Czobor, 2011) and with the cognitive training study of Anguera et al. in healthy older adults, which showed that elderly subjects were able to generate highly significant cognitive improvements, approaching the performance of untrained young adults, despite beginning at a worse baseline performance (Anguera et al., 2013). We also found that gender, subject education, total training time, and training intensity did not influence target engagement.

The fact that the “obvious suspects” do not appear to be moderators of treatment response, in conjunction with the high degree of heterogeneity we observe in APS change over time and in cognitive outcome measures, begs the question of which individual factors we are failing to identify at baseline that are key predictors of target engagement. We present our descriptive finding of a mean APS plateau of 83.44 ms in participants showing Global Cognition improvement >0.2 SD (unconfirmed by appropriate statistical LGC modeling methods), as an impetus to the field, and not as a clinically meaningful value that can predict who will or will not respond to treatment. Given the high degree of neurocognitive heterogeneity of schizophrenia, a reliable estimate of the critical APS plateau that is associated with significant cognitive improvement will only be achieved when we can study larger samples and identify neurocognitive sub-phenotypes, by extracting classes of subjects based on their individual response to training using growth mixture models.

APS plateau mediates the generalized response to targeted cognitive training

We examined whether initial APS change and APS plateau mediated TCT-induced cognitive improvements. The results of our LGC modeling revealed some highly informative answers to this question. First and foremost, participants showed a high degree of inter-individual variability in their baseline APS, initial APS change, and APS plateau. This indicates that participants showed heterogeneity both in terms of baseline auditory psychophysical efficiency and in terms of training-induced changes in that efficiency as early as after 20 hours of training. We posit that this represents variable engagement of prefrontal-temporal neural systems during training (see Dale et al. 2015), likely reflecting difference in sensory system “learning potential” or possibly in inherent plasticity mechanisms.

Notably, the training-induced APS plateau (and not the initial APS change) predicted cognitive outcomes after training. A lower baseline performance in global cognition, speed of processing, verbal working memory, visual working memory, visual memory, and problem solving, were all significantly associated with slower APS plateau reached after 20 hours of training, and this in turn predicted lower gains in these cognitive domains at the end of training. Conversely, higher baseline cognitive performance was associated with greater target engagement – as reflected in a better APS plateau - which was followed by larger gains in untrained higher-order cognitive outcome measures.

These findings suggest that the faster the APS plateau that is reached during training, the greater are the transfer effects to untrained cognitive domains— indicating that the capacity to harness and sustain improvements auditory sensory processing efficiency serves as a key mediator of treatment response. The fact that we found no significant relationships in Spearman's rank correlations between initial APS change and cognitive gains further indicates that the attainment of the APS plateau, and not the overall magnitude of APS improvements, represents the critical measure of target engagement, and reflects the mechanism of action by which auditory system training induces generalized cognitive gains. In other words, there is a significant association between an individual's ability to generate and sustain sensory processing efficiency in the auditory system and their degree of higher-order cognitive improvement after auditory system training, independent of their baseline neurocognition.

Conclusions and Future Directions

As per the experimental medicine model, the results of our analysis indicate that TCT of the auditory system in schizophrenia exerts measurable and predicted effects on APS, a behavioral measure that is posited to reflect efficiency in prefrontal-temporal neural systems. This finding is consistent with our recent MEG study demonstrating that TCT drives auditory cortical plasticity, which in turn correlates with prefrontal plasticity and generalized cognitive gains (Dale et al., 2015). In addition, our analysis suggests that, while it is necessary for the training target to be engaged (as reflected by a significant initial APS change), it is the APS plateau reached after 20 hours that shows a significant association with the magnitude of generalized cognitive gains. Our analysis also indicates that participants with better baseline cognitive performance, who in turn achieve a faster APS plateau, then show greater gains after TCT, while participants with worse baseline cognitive performance achieve slower APS plateau and show lower generalized cognitive gains at the end of training. Thus, several important additional research questions are raised.

First, which critical cognitive and neuroplastic mechanisms allow an individual to engage in psychophysical “learning”? What is the relationship between this capacity for sensory system learning/behavioral improvement and overall cognitive status? Why do some individuals with schizophrenia show evidence of robust target engagement during training, as evidenced by their ability to achieve a faster APS plateau, while others do not? At this point, we can only speculate, but previous research has indicated that individuals with schizophrenia manifest a high degree of heterogeneity in fundamental neuroplasticity mechanisms as measured, for example, via TMS of motor system plasticity, although the underlying neurobiological features that contribute to this heterogeneity are at present unknown (Daskalakis, Christensen, Fitzgerald, & Chen, 2008). In addition, individuals with schizophrenia show heterogeneity in their prefrontal attention mechanisms, and those with significant impairment might not be able to harness sustained attention in the service of processing salient stimuli required for training progression (e.g., Hasenkamp et al., 2011).

Our current sample size does not allow the identification of the critical APS plateau that must be reached to be associated with significant overall cognitive gains (e.g., of >0.2 SD). We also did not acquire the kinds of data (e.g., serum anticholinergic activity) that would allow us to examine potential medication effects on target engagement (Vinogradov et al., 2009). Finally, as noted earlier, we do not know whether further intensive training on the sound sweeps task later during the course of the training program would induce additional changes in APS and potentially drive greater cognitive improvements. An important next step will be to more closely examine psychophysical “learning”, its durability, and its relation to improvements in higher-order cognition in schizophrenia; we must also study larger samples and identify important neurocognitive sub-phenotypes of patients. Ultimately, once we are able to identify associations between target engagement measures, patterns of treatment response, and neurocognitive sub-phenotypes, we will be able to accomplish three important goals: 1) A deeper understanding of the range of key pathophysiological mechanisms that contribute to schizophrenia; 2) A more informed knowledge based on how these various mechanisms contribute to cognitive training response or non-response (which will in turn allow us to design more efficacious interventions); 3) The personalization of cognitive training treatments for individuals with schizophrenia.