Reinforcement learning performance and risk for psychosis in youth

Abstract

Early identification efforts for psychosis have thus far yielded many more individuals “at risk” than actually develop psychotic illness. Here we test whether measures of reinforcement learning (RL), known to be impaired in chronic schizophrenia, are related to the severity of clinical risk symptoms. Due to the reliance of RL on dopamine-rich fronto-striatal systems, and evidence of dopamine system dysfunction in the psychosis prodrome, RL measures are of specific interest in this clinical population. The current study examines relationships between psychosis risk symptoms and RL task performance in a sample of adolescents and young adults (n=70) receiving mental health services. We observed significant correlations between multiple measures of RL performance and measures of both positive and negative symptoms. These results suggest that RL measures may provide a psychosis risk signal in treatment-seeking youth. Further research is necessary to understand the potential predictive role of RL measures for conversion to psychosis.

1. Introduction

The onset of schizophrenia typically occurs during late adolescence or early adulthood and is usually preceded by one to two years of attenuated psychotic symptoms. There is a rapidly-growing literature on the early identification of people who appear to be at high risk for conversion to psychosis, with the hope that early intervention may alter the course of the illness (Marshall et al., 2005; Stafford et al., 2012; van der Gaag et al., 2013; Schiffman et al., in press). Unfortunately, the use of common clinical high risk (CHR) diagnostic interview procedures results in very substantial “false positive” rates, with only a minority of individuals actually converting to psychosis over the next two to three years (Fusar-Poli et al., 2012a). Not surprisingly, people with the most severe levels of subthreshold positive (Cannon et al., 2008) and negative (Piskulic et al., 2012) symptoms appear to be the most likely to convert to psychotic illness.

The limitations of interview-based assessments have spurred interest in the use of biological and neurocognitive markers to more accurately identify risk for conversion to psychosis among individuals who meet clinical high risk symptom criteria, such as neuropsychological tests (Fusar-Poli et al., 2012b), blood-based measures (Benros et al., 2012), structural and functional magnetic resonance imaging (fMRI; Pantelis et al., 2009), electroencephalography (EEG; Correll et al., 2010), and molecular imaging techniques (Brugger et al., 2011). To date, it is not entirely clear whether neuropsychological measures offer incremental predictive value for conversion, beyond that achieved with symptom severity measures (Fusar-Poli et al., 2012b; Lin et al., 2013; Seidman et al., 2010). While participants who convert often perform poorly on neuropsychological measures, relative to healthy controls, a substantial number of participants who do not convert also demonstrate cognitive deficits, reducing the positive predictive value of cognitive measures.

Of the brain imaging methods, molecular imaging of presynaptic dopamine synthesis, using positron emission tomography (PET), has shown considerable promise, with two studies by Howes and colleagues (2009; 2011) showing that elevated baseline levels of dopamine synthesis in the striatum were powerful predictors of subsequent conversion to psychosis. Interestingly, CHR participants who failed to convert appeared to have fully normal levels of presynaptic dopamine synthesis (2011), suggesting that the CHR population is composed of at least two types of individuals, those with dopamine abnormalities and those without — two groups who also cannot be discriminated on the basis of clinical features at time of baseline assessment, but who have very distinct clinical trajectories (Damjaha et al., 2014). This finding is in line with evidence linking psychosis to dopaminergic hyperactivity from patients with chronic schizophrenia (Laruelle & Abi-Dargham, 1999).

Molecular imaging methods such as PET, however, are expensive and are unlikely to ever be widely available. Thus, a biological/behavioral marker from an inexpensive, easily-administered behavioral test, would be highly useful if it could add in the prediction of conversion, especially if it were sensitive to perturbation of the dopamine system. Measures of reinforcement learning are a logical candidate to consider in this context, based on the large basic neuroscience literature pointing to a critical role for dopamine in reinforcement learning (RL) and motivation (see Montague et al., 2004; Schultz, 2013 for reviews). More specifically, in typically-functioning people, dopamine cell firing increases after the receipt of a better-than-expected outcome, while the same cells transiently stop firing after worse-than-expected outcome. By signaling these mismatches between expected and obtained outcomes, called reward prediction errors (RPEs), modulations of dopamine cell activity serve as the “teaching signals” that make reinforcement learning possible - either increasing or decreasing the likelihood that the same response will be made when the same choice is encountered in the same context in the future. These teaching signals are broadcast to the widely-distributed cortical and subcortical systems that mediate reinforcement learning. Thus, impaired RL cannot be taken as direct evidence of impaired dopamine signaling, as it is possible that dysfunction in the target region(s) makes it impossible to use an intact teaching signal. Nonetheless, impaired RL can be taken as evidence that the function of dopamine-dependent fronto-striatal circuits is compromised in ways that are likely relevant for psychosis risk.

Behavioral studies have, in fact, revealed abnormalities in RL in both first episode (Murray et al., 2008) and chronic schizophrenia (Waltz et al., 2007; Waltz et al., 2011). Furthermore, functional imaging studies have produced evidence of altered neural activity in dopamine rich subcortical structures, such as the basal ganglia (Koch et al., 2010; Gradin et al., 2011; Deserno et al., 2013), as well as dopamine targets in frontal cortex (anterior cingulate cortex and prefrontal cortex; Minzenberg et al., 2009; Goghari et al., 2011), known to figure critically in cognitive control. Interestingly, across studies in patient samples, RL impairments have shown correlations with the severity of both positive and negative symptoms — symptom dimensions related to risk for conversion in clinical high risk populations (Cannon et al., 2008; Piskulic et al., 2012).

Based on evidence that: 1) abnormalities in the dopamine system are implicated in conversion to psychosis; 2) the dopamine system plays a critical role in RL; and 3) evidence that RL is impaired in people with schizophrenia, the current study aimed to assess the potential link between psychosis risk symptoms (both positive and negative) and probabilistic RL task performance in a sample of mental health help-seeking youth. Our goal was to determine, as a first step, if deficits in RL are related to the severity of CHR symptoms in validating this conceptual approach to the prediction of conversion. Further, we thought it was particularly important to see if this signal could be detected in a heterogeneous clinical population where many participants would be expected to be experiencing a high level of distress, showing a wide range of symptomatology, as well as cognitive and functional impairment. Identifying a potential biomarker that does not require expensive imaging equipment, is less stressful, and can be conducted in a variety of settings is another goal of the study.

2. Methods

2.1 Participants

Participants included 70 adolescents and young adults (mean age = 15.78 years ± 3.05) who were recruited from the greater Baltimore, Maryland area, through advertisements and clinic referrals. Eligibility criteria for the study required that participants were between the ages of 12 and 22, were receiving mental health services at study entry, and (for participants under the age of 18) were accompanied by a guardian to provide consent for their participation.

2.2 General procedures

The study received Institutional Review Board approval at the University of Maryland School of Medicine; the Maryland Department of Health and Mental Hygiene; and the University of Maryland, Baltimore County (UMBC). Recruitment was through the UMBC YouthFIRST program. Participants and their guardians (for minors) completed the informed consent and a demographics form. Written assent was obtained for all youth <18 years old. Participants were then administered the Probabilistic Stimulus Selection (PSS) task. After the task, participants were administered a structured interview assessing psychosis-risk and a measure of general cognitive ability by study staff. Study staff, composed of two doctoral-level clinicians and several psychology doctoral students, then completed global functioning scales based on data collected during the interview.

2.3 Clinical, cognitive, and functional assessment

Structured Interview for Psychosis-Risk Syndromes (SIPS; Miller et al., 2002)

The SIPS is a clinician-administered interview designed to assess symptoms shown to be associated with psychosis. The SIPS evaluates nineteen symptoms in total (5 positive, 6 negative, 4 disorganized, and 4 general). The five positive symptoms (unusual thought content, suspiciousness, grandiosity, perceptual abnormalities, and disorganized speech) are used to evaluate level of risk. Symptoms are rated from 0 (absent) to 6 (extreme or psychotic) and the sum of the symptom scales (positive, negative, disorganized, general) can be used as continuous measures.

Study staff administering the SIPS were trained either by SIPS creators during a two day training (certification involved attaining 90% agreement with gold standard scores for case examples) or through an extensive training process within the research lab. This process required staff to read and rate vignettes from the SIPS authors, rate previously recorded interviews, observe two or more interviews conducted by trained staff, and lead interviews with an experienced co-rater sitting in. After two co-rated interviews matched, the new interviewer was able to administer the SIPS independently. The research team also met weekly to review interviews and ensure agreement. Reliability on symptom ratings among team members is high (intraclass correlation coefficient = .82) and diagnostic agreement is perfect (kappa = 1.0).

In addition to using the SIPS scores as a continuous measure, for specific analyses we also divided the sample of 70 participants into three subgroups: 1) participants who already met criterion for psychotic illness based on a score of 6 on one of five SIPS positive symptoms (N=9); 2) participants who met criterion for clinical high risk status (CHR+; N=20); and 3) participants who did not meet criteria for either psychotic illness or clinical high risk status (CHR−; N=41).

Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999)

In order to generate an estimated full-scale IQ, we administered the two-subtest (Vocabulary and Matrix Reasoning) version of the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999). This measure was developed to provide a short, reliable measure of intelligence linked to previous Wechsler intelligence scales. Fifty participants received this measure, as it was added to the protocol after data collection had begun on the RL task. Standardized scoring procedures and age-based norms were used to establish individual IQ scores.

Global functioning scales for role and social functioning (GF scales; Cornblatt et al., 2007)

The GF social scale assesses aspects of social functioning, including friendships, dating, and interpersonal or family conflict. The GF role scale evaluates the degree to which an individual is able to independently perform in age-appropriate roles, such as in school or work. The GF scale scores are rated for the highest and lowest level of functioning in the past year as well as for current functioning. Current ratings of functioning were used in this study. Scores for the GF scales range from 1 to 10, with 1–3 reflecting significant impairment, 4–6 reflecting moderate to serious impairment, and 7–10 reflects a range of mild impairment to superior functioning. Detailed descriptions of the level of functioning are given for each anchor point. Psychometrics of the two GF scales are good with high reported interrater reliability (intraclass correlation coefficients at or above .85) as well as moderate to high correlations (ranging from r = .49 to .70) with other measures of functioning (Cornblatt et al., 2007). Comparisons of participant subgroups (Psychosis, CHR+, and CHR−) on measures of symptom severity, intellectual functioning, and global functioning are shown in Table 1.

Table 1.

Sample characteristics of participants by diagnostic group

	CHR− (n=41)	CHR+ (n=20)	Psychosis (n=9)	Statistic
Mean Age	16.15 (3.19)	14.97 (2.76)	15.92 (2.97)	F(2,67) = 1.02, p = .367
Mean WASI IQ	105.64 (16.62)	97.80 (16.10)	105.29 (14.20)	F(2,47) = 0.92, p = .406
Mean Positive sx sum	4.07 (3.23)	11.65 (4.10)	17.56 (4.25)	F(2,67) = 65.53, p < .001
Mean Negative sx sum	7.98 (5.71)	10.75 (6.27)	17.44 (6.15)	F(2,67) = 9.65, p < .001
Mean Disorganized sx sum	3.39 (2.55)	5.50 (2.63)	8.89 (4.51)	F(2,67) = 14.59, p < .001
Mean General sx sum	7.34 (4.58)	8.35 (2.96)	9.56 (4.42)	F(2,67) = 1.20, p = .309
Mean GF role	7.03 (1.63)	6.55 (1.99)	5.00 (1.58)	F(2,66) = 5.05, p = .009
Mean GF social	7.05 (1.36)	6.05 (1.61)	5.33 (1.41)	F(2,66) = 6.83, p = .002
Gender
Female	21	15	2	F(2,67) = 3.93, p = .024
Male	20	5	7
Race
African American	19	11	6
Caucasian	14	5	3	F(2,67) = 0.53, p = .591
Multiple racial identities	7	4	0
American Indian	1	0	0
K-SADS Diagnosis*
Mood Disorder	17 (41.5%)	13 (65.0%)	2 (22.2%)
Anxiety Disorder	19 (46.3%)	13 (65.0%)	2 (22.2%)
ADHD	24 (58.5%)	12 (60.0%)	1 (11.1%)
Behavioral Disorder	9 (22.0%)	10 (50.0%)	5 (55.6%)
PTSD	9 (22.0%)	4 (20.0%)	2 (22.2%)
Psychotic Disorder	0	0	5 (55.6%)
Other Disorder	10 (24.4%)	2 (10.0%)	0
No Diagnosis	1 (2.4%)	0	0
Psychiatric medications*
Antipsychotic	5 (12.2%)	7 (35.0%)	4 (44.4%)
SSRIs	11 (26.8%)	9 (45.0%)	2 (22.2%)
Stimulants	18 (43.9%)	11 (55.0%)	0
Mood-stabilizers	3 (7.3%)	0	1 (11.1%)
Anxiolytic	3 (7.3%)	2 (10.0%)	1 (11.1%)
Any medication	27 (65.9%)	16 (80%)	5 (55.6%)
Mean Task Performance:
Early Acquisition Phase
Overall Learning	67.61 (18.19)	64.61 (17.22)	61.20 (15.03)	Χ2(2) = 0.98, p = .613
Go-learning (Win-stay)	75.51 (18.82)	69.33 (19.40)	60.66 (23.43)	Χ2(2) = 4.71, p = .095
NoGo-learning (Lose-shift)	58.79 (16.88)	60.50 (12.90)	61.17 (12.36)	Χ2(2) = 0.27, p = .875

Note: 8, 10, and 2 participants in the respective groups were missing WASI data; One person was missing both measures of real-world functioning. Abbreviations: sx=symptom; Mood Disorders included: Major Depressive Disorder, Bipolar Disorder, and Depressive Disorder Not Otherwise Specified; Anxiety Disorders included: Generalized Anxiety Disorder, Panic Disorder, Obsessive-Compulsive Disorder, Social Phobia, and Specific Phobia; Behavior Disorders included: Oppositional Defiant Disorder, Conduct Disorder, and Disruptive Behavior Disorder Not Otherwise Specified; Psychotic Disorders included: Schizophreniform and Schizoaffective Disorder; Other Disorders included: Substance Use Disorders, Adjustment Disorders, and Eating Disorders;

^*Many participants carried multiple diagnoses and were on multiple medications.

2.4 Experiment task

A revised version of an existing probabilistic learning task — the Probabilistic Stimulus Selections task — was used in the current study (Waltz et al., 2007; see Figure 1). The revised version uses clip art images of everyday objects (flashlight, clock, etc.), as stimuli, in order to enhance coding (instead of the Japanese hiragana characters used in the original version of the task; Frank et al., 2004). In this task, administered on a laptop computer using E-Prime software, participants learn to select the optimal stimulus, in three image pairs, using feedback based on their selections. There are two phases, an acquisition phase and a test phase. The acquisition phase consists of three pairs of images (AB, CD, EF) that are presented in random order, with stimulus choices reinforced at different rates (A: 80%; B: 20%; C: 70%; D: 30%; E: 60%; F: 40%). The acquisition phase ends when participants achieve a learning criterion (after a minimum of 2 blocks) or when participants complete the full 6 blocks (360 trials). The learning criterion is 65% correct for the AB pair, 60% for the CD pair, and 50% for the EF pair. Although participants can discontinue the acquisition phase of the learning task at different points, all participants complete a minimum of two acquisition blocks.

Figure 1.

Sample stimulus pairs from the Acquisition Phase. Participants performed 20 trials with each stimulus pair, per block, receiving probabilistic feedback. Training was discontinued when the subject reached learning criteria in all three conditions in same block (minimum of 2 blocks, maximum of 6 blocks).

The acquisition phase of the probabilistic RL task was followed by a test phase designed to study the transfer of learning from the acquisition phase. Unfortunately a substantial portion of the overall study population (24 of 70 total) failed to meet the acquisition learning criteria. Because it is not possible to interpret the transfer phase performance in the absence of adequate learning, these results will not be discussed further.

From the first two blocks of the Acquisition phase (completed by all participants; called “Early Acquisition”), we quantified the overall percentage of choices of the better stimulus from each pair (A from AB, C from CD, and E from EF), which we termed “Early Acquisition Average”. Then we computed rates of “win-stay” and “lose-shift” behavior, from the first two blocks for each of the three stimulus pairs. “Win-stays” were defined as valid rewards followed by the choice of the same stimulus, on the next presentation of that specific stimulus pair. “Lose-shifts” were defined as valid punishments followed by the choice of the alternate stimulus, on the next presentation of that specific stimulus pair.

2.5 Statistical analyses

Our first step was to assess relationships between participants’ SIPS sub-scores (positive, negative, disorganized, and general) and performance measures from the PSS task. Analyses were also conducted to examine relationships between measures of overall functioning (role-functioning and social-functioning, from the Global Functioning scales) and performance measures from the PSS task. Because few of our clinical and experimental variables were normally distributed (according to Shapiro-Wilk tests) we used Spearman correlation statistics to assess associations among these variables. To evaluate whether the nine individuals with psychosis accounted for correlational findings, all analyses were conducted with and without these participants.

As a second step, we compared subgroups of participants (Psychosis, CHR+, and CHR−) on measures of “Early Acquisition” performance, reward-driven learning (“Win-stay”), and punishment-driven learning (“Lose-shift”). Because our clinical and experimental variables were, by and large, not normally distributed, we used non-parametric Kruskal-Wallis tests. Finally, we examined potential effects of psychotropic medications by dividing the 61 participants without a diagnosis of psychotic illness into those taking antipsychotic drugs (APDs) and not, and those taking psychostimulants and not, as both have direct effects on the dopamine system. We used U-tests to test for differences in clinical and experimental measures between the 12 non-psychotic participants taking APDs and the 49 participants not taking APDs, and between the 29 non-psychotic participants taking psychostimulants and the 32 participants not taking psychostimulants.

2.6 Modeling analyses

In order to assess the impact of several hypothesized factors on PSS performance in help-seeking adolescents, additional computational analyses were done, using a mathematical model of RL and decision making based on the work of Frank and colleagues (2007). The fitting of data using this computational model allowed for the estimation of several parameters meant to capture important aspects of learning and decision making: α_G (the learning rate indicating the impact of gains/positive prediction errors on association strength); α_L (the learning rate indicating the impact of losses/negative prediction errors on association strength); and β (the temperature, or strictness with which the participant chooses the option of the greatest expected value). Details of the computational model used are in the Supplemental Materials.

3. Results

3.1 Correlational analyses: Entire sample

Correlational analyses with clinical symptom ratings suggested that measures of multiple aspects of psychopathology, as rated by the SIPS, related to measures of PSS task performance in our sample (Table 2). In particular, overall scores in the Early Acquisition phase of the PSS task correlated significantly (negatively) with ratings for negative symptoms in participants. That is, patients with higher levels of negative symptoms showed the most marked RL impairment. The ability to learn from rewards, as assessed by win-stay performance, showed significant negative correlations with the severity of positive, negative, and disorganization symptoms (Figure 2). Positive correlations were observed between win-stay rates and both role and social functioning, such that higher functioning individuals showed higher rates of staying with rewarded choices. None of the correlations with lose-shift performance approached significance, suggesting that it is the inability to learn from positive outcomes that is related to clinical features. The absence of any relationship with ratings of global symptoms suggests that the observed findings are unlikely to be accounted for by the overall severity of psychopathology. These observations suggest that the ability to use feedback to guide learning is related to everyday functioning and that RL task performance relates to the severity of symptoms specific to schizophrenia risk.

Table 2.

Analyses of Spearman correlations among measures of experimental task performance, symptoms, and real-world functioning

	All Subjects (N=70)			Non-psychotic Subjects (N=61)
	Overall Learning	Win-stay Rate	Lose-shift Rate	Overall Learning	Win-stay Rate	Lose-shift Rate
Positive Sx	−0.186	−0.346**	0.021	−0.155	−0.286*	−0.019
Negative Sx	−0.291*	−0.406***	−0.165	−0.238	−0.303*	−0.207
Disorganization Sx	−0.117	−0.286*	−0.016	−0.052	−0.192	−0.037
Global Sx	0.007	−0.052	0.109	0.033	0.005	0.094
Global Func: Role	0.201	0.264*	0.123	0.165	0.194	0.180
Global Func: Social	0.253*	0.417***	0.185	0.233	0.363**	0.211

Abbreviations: Sx, Symptoms; Func, Functioning.

^*= p < 0.05;

^**= p < 0.01;

^***= p < 0.001.

Figure 2.

Scatterplots illustrating relationships between win-stay rates and clinical variables. The nine participants meeting SIPS criteria for psychosis are depicted by gray circles. The remaining 61 participants in the study are depicted by black diamonds. (A) The correlation between win-stay rates and positive symptoms is significant both in the full sample, and in the reduced sample of 61 non-psychotic individuals, as is (B) the correlation between win-stay rates and negative symptoms. (C) The correlation between win-stay rates and disorganization symptoms is significant in the full sample, but not in the reduced sample. (D) The correlation between win-stay rates and social functioning scores is significant in both the full and reduced samples.

3.2 Correlational analyses, excluding individuals with psychosis

Not surprisingly, the exclusion of participants with the most severe positive symptoms resulted in a slight attenuation of relations between positive and disorganized symptoms and RL measures (see Table 2). Nonetheless, correlations between experimental measures and both positive symptom (ρ = −0.286; p = 0.025) and negative symptom (ρ = −0.303; p = 0.018) severity remained statistically significant, even after the exclusion of participants who met SIPS psychosis criteria.

3.3 Analyses controlling for the potential confounds of general cognitive ability and medication status

In the sample of (50) participants for whom WASI data were available, general cognitive ability (IQ) was strongly correlated with all three measures of task performance: Overall Acquisition (ρ = 0.485, p < 0.001), Win-stay Rate (ρ = 0.563, p < 0.001), and Lose-shift Rate (ρ = 0.407, p = 0.003). In order to examine the relation between task performance and psychosis-risk symptoms above and beyond the influence of cognitive ability, partial correlations controlling for WASI IQ were conducted (see Supplementary Table 1). When controlling for IQ, several correlations remained significant [e.g., win-stay rates and both positive symptoms (p = 0.015) and social functioning (p = 0.043)] or trended toward significance (p = 0.097 for the correlation between win-stay rates and negative symptoms). Thus, systematic relations between reward-driven learning and motivational deficits could not be attributed to deficits in general intellectual functioning.

When we compared the 12 individuals taking antipsychotic medications with the 49 individuals not taking antipsychotic medications (excluding the 9 individuals already meeting criteria for psychotic illness), we observed that the groups did not significantly differ on any clinical, cognitive, or experimental measures (see Supplementary Table 2). When we compared the 29 individuals taking stimulant medications with the 32 individuals not taking stimulants, we found that the groups differed on disorganization symptom severity, but not on any other clinical, cognitive, or experimental measure (see Supplementary Table 3).

3.4 Comparisons of participant subgroups

When we compared the three participant groups on overall Early Acquisition performance, a Kruskal-Wallis test revealed no significant effect of group [Χ²(2) = 0.98, p = 0.613]. However, when we compared the three participant groups on reward-driven learning (win-stay) performance, we observed a trend toward a significant effect of group [Χ²(2) = 4.71, p = 0.095]. When we compared the three participant groups on punishment-driven learning (lose-shift) performance, we observed no significant effect of group [Χ²(2) = 0.27, p = 0.875].

3.5 Results of computational modeling analyses

Computational modeling analyses (Supplementary Tables 4 and 5) confirmed the general impression that a number of participants performed at chance, failing to demonstrate acquisition of even the AB (80% versus 20%) pair. In short, 13 of the 70 participants made sequences of choices that could not be accounted for by a principled RL model (see Supplementary Materials for details). When we considered the 57 individuals whose data could be fit to our RL model, we observed a systematic relationship between negative symptom severity and the model parameter corresponding to the capacity for reward-driven, or Go-learning (β*α_G; Supplementary Table 7). This modeling result fits well with the behavioral results described above.

We observed that the 13 participants performing at chance had significantly lower IQs then the 57 participants showing above-chance performance [t(68)=3.169; p=0.003; Supplementary Table 6]. The 13 participants performing at chance had numerically, but not significantly, higher levels of positive, negative, and disorganized symptoms and numerically lower scores for role and social function.

4. Discussion

4.1 Summary of findings and implications

Previous research suggests that both individuals with chronic schizophrenia and first-episode psychosis have abnormalities in RL that are associated with both positive and negative symptoms. Here, we show, for the first time, that RL deficits are associated with the severity of positive and negative risk symptoms, as well as level of functioning, in a heterogeneous sample of treatment-seeking adolescents. A reduced ability to use positive feedback to guide learning was also associated with greater severity of disorganization symptoms. Importantly, the relationship between the ability to use positive feedback to guide subsequent choices (i.e., Win-stay rate) with negative and positive symptoms, as well as social functioning, remained significant even when the subjects meeting criteria for conversion to psychosis were excluded from the analysis.

The fact that RL impairments correlate with both positive and negative symptoms is noteworthy, as most accounts suggest that these symptoms arise from different processes. Specifically, psychosis has been linked to excessive release of striatal dopamine (Laruelle & Abi-Dargham, 1999, etc.), whereas many have suggested that negative symptoms are a reflection of prefrontal dysfunction (Fischer et al., 2012; Goghari et al, 2011; Potkin et al., 2002). A potential resolution of this apparent contradiction may be found in studies by Meyer-Lindenberg and colleagues (2002), who have produced evidence that the extent of prefrontal dysfunction is related to the extent of striatal dopamine release. If hypofrontality and excessive striatal dopamine release are two sides of the same coin — as the work of Meyer-Lindenberg and colleagues (2002) suggests — one would expect RL impairments to be related to both negative and positive symptom dimensions.

A similar argument can be advanced based on an elaboration of the theory proposed by Kapur and colleagues (2003). In brief, Kapur has proposed that contextually inappropriate dopamine release may result in stimuli being experienced as highly-salient and bearing personal significance. Such uncanny experiences are thought to be basis for the development of psychotic symptoms, as the person will strive to make sense of these aberrantly salient events. Thus, the experience of aberrant salience, and the drive to make sense of the experience, may be the basis for delusional ideas, and perhaps, hallucinations. However, as learning and motivation increasingly come to be driven by aberrant experiences of salience, one can imagine that normal motivational processes become compromised. Erratic phasic dopamine release may result in a reduced ability to use actual outcomes to drive goal-directed behavior (as the expected reward signal may not be delivered). Thus, the individual in such a state will be challenged by experiences of aberrant salience as well as the loss of experience of adaptive salience. In short, there may be a link between abnormalities in dopamine function and both the positive and negative symptoms seen in schizophrenia.

In a series of studies of RL in people with chronic schizophrenia (e.g., Waltz et al., 2007; Waltz et al., 2011), we have found RL deficits that appear to arise from failures to use positive prediction errors to guide behavior. These impairments have often correlated with negative symptom severity, as we observed in the current sample. Further, we have typically seen intact learning from negative prediction errors in samples of people with chronic schizophrenia — a finding that is echoed in the current sample by the fact that lose-shift performance showed no relationship to either positive or negative CHR symptoms. It is intriguing that the findings related to psychosis risk symptoms in this sample of treatment-seeking youth resemble observations from our studies in more chronically ill populations. These resemblances suggest that RL measures may be useful in identifying people who are most likely to develop a more severe psychotic illness over time.

Although we have not yet followed our sample over time, there may be aspects of this study that can provide clues as to the possible use of RL measures for the prediction of conversion to psychotic illness. Past studies have found that the severity of baseline levels of both positive and negative symptoms have been associated with severity at baseline has been associated with subsequent conversion to psychotic illness (Cannon et al., 2008; Piskulic et al., 2012; Riecher-Rossler et al., 2009; Ziermans et al., 2014). The RL correlations with measures of functioning likely reflect the substantial overlap between negative symptom rating scales and functional outcome rating scales. Thus, it appears that RL measures provide an important clinical signal, correlating with the severity of positive and negative symptoms, as well as impairment in social functioning. Whether they are predictive of conversion remains to be determined, but the present results suggest that RL deficits are related to functional deficits.

It is important to highlight that measures of overall learning and the ability to stick with a reinforced response correlated with real-world functioning, even when excluding the participants with full psychosis. Poorer Early Acquisition performance correlated with worse role functioning (school performance) and worse social functioning (peer relationships). Predicting functioning through a 20 minute RL task provides preliminary evidence for the utility of this approach for purposes beyond basic research, and suggests that our lab based measure may have applicability for functional outcomes (Green et al., 2000).

4.2 Limitations

The current study was a cross-sectional analysis of the association between RL task performance and the severity of CHR symptoms. A longitudinal analysis, across the at-risk and early-illness phases, would allow for the examination of whether performance on RL tasks can contribute to a prediction model for psychosis. We had not anticipated that such a large proportion of our sample would be unable to meet the acquisition learning criteria, which precluded examining the transfer phase data. The relatively poor learning performance characteristic of many of the adolescents in our study (even when compared with older samples of individuals with mental illness) may be attributed to incomplete maturation of the dopaminergic and fronto-striatal pathways upon which reinforcement learning is known to depend (Vink et al., 2014). In fact, studies of reinforcement learning across the lifespan have revealed that adolescents often perform more poorly than adults on tests of reinforcement learning (van der Schaaf et al., 2011) and uncertainty-based decision-making (Galvan et al., 2006), and also exhibit different activity patterns in fronto-striatal circuits, in conjunction with feedback processing (Peters et al., 2014; van den Bos et al., 2012). Additional task training and practice procedures would likely be needed to enhance learning in future studies.

Additionally, many of our participants were prescribed psychoactive medications with the intention of treating a variety of symptoms. However, among individuals who had not already developed psychosis, the administration of antipsychotic and stimulant medications had no discernible effects on task performance. Nonetheless, this type of post-hoc analysis of the impact of medication is very difficult to interpret given the lack of random assignment. Finally, our study lacked a control sample of typically-developing adolescents. For this reason, we focused the analyses on: 1) relationships between experimental behavioral markers and continuous measures of psychopathology, and 2) comparisons between groups of adolescents and young adults determined to be at clinical high risk for converting to psychotic illness, and help-seeking adolescents and young adults at risk for other psychiatric problems.

4.3 General conclusions

Our data suggest that performance on a probabilistic reinforcement learning task shows a systematic relation with the severity of all three dimensions of psychosis-risk symptoms (positive, negative, and disorganization) as well as real-world functioning among a sample of mental health help-seeking young people. Further research is needed to elucidate relations among reward processing, reinforcement learning, and clinical symptoms along multiple dimensions in individuals who are at risk for developing psychosis. If this non-invasive, inexpensive, and portable method enables the early detection of reinforcement learning deficits in individuals at risk for psychosis who eventually develop the disorder, this task may add incremental utility for identifying psychosis early on.