Probabilistic Learning by Positive and Negative Reinforcement in Generalized Anxiety Disorder

Abstract

This experiment examined learning tendencies in generalized anxiety disorder (GAD) using reinforcement feedback for probabilistic outcomes. One hundred sixty-six GAD and 105 non-GAD participants were randomized to a computerized probabilistic learning task that used either negative or positive reinforcement. Participants chose between stimuli with specific probabilities of reinforcement to learn which of each pair had the highest probability. Reinforced choices either removed an angry face (negative reinforcement) or made a happy face appear (positive reinforcement). Results showed that those with GAD learned the correct probabilistic choices at a slower rate over time and to a lesser degree than control participants regardless of reinforcement type. Estimations of the likelihood of receiving a good outcome posttask were also more inaccurate for those with GAD, especially when true likelihoods were high. Furthermore, compared with control participants, those with GAD reported lower perceived reinforcement sensitivity, higher behavioral inhibition sensitivity, and higher undesirable feelings toward probabilistic learning.

Generalized anxiety disorder (GAD) is both widespread and detrimental. Not only is lifetime GAD prevalence 14.2% (Moffitt et al., 2010), but it is also associated with significant disability, role impairment, and health problems (e.g., Denollet, Maas, Knottnerus, Keyzer, & Pop, 2009; Kessler et al., 2009). Unfortunately, GAD is resistant to treatment as well. Nearly half of GAD clients do not meaningfully improve in many therapy trials (Westen & Morrison, 2001). This resistance to treatment has spurred recent increases in research on the disorder and its resilient maintenance factors (Newman & Przeworski, 2018). Although previously unstudied, learning tendencies unique to GAD may be among the factors that maintain it. Theory suggests that GAD maintenance may be linked to deficits in learning probabilistic outcomes over time and learning in response to reinforcement.

There are reasons to expect that those with GAD may have difficulty with probabilistic learning, or learning the likelihood of specific outcomes for uncertain events. The primary symptom of GAD—excessive and uncontrollable worry—is itself repeatedly predicting high likelihoods for negative future outcomes. If these predicted likelihoods were to remain unchallenged by accurate probabilistic learning, worry would be maintained. Newman and Llera’s (2011) well-supported contrast avoidance model proposes that those with GAD use worry to create and sustain anxious feelings to prevent unwanted shifts in negative emotion. Stated differently, those with GAD prefer to remain continually distressed from worry so that if an undesirable event occurs, they will experience less of a spike in negative emotion (i.e., a weaker “negative contrast”). This process has been experimentally demonstrated (Llera & Newman, 2010). Moreover, unlike control participants, those with GAD have reported a preference for using worry to cope with shifts both outside and inside the lab (Llera & Newman, 2014, 2017). If the purpose of worry is to brace against drops in mood, accurate estimations would require sacrificing a precious coping mechanism. Thus, those with GAD may not learn from experiences that refute their negative predictions, maintaining the disorder.

Cognitive biases may skew GAD probability formation toward unrealistic pessimism. It is well established that those with GAD have an attentional bias toward threat (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007). Bias toward negative information may impede learning by causing those with GAD to selectively process negative outcomes, negatively biasing future predictions (poor probabilistic learning) and dismissing rewarding or relieving outcomes (poor reinforcement learning). Those with GAD also tend to remember personally relevant negative information better than positive information (Coles, Turk, & Heimberg, 2007). This negative information may be more easily accessible as evidence for forecasting the future, adversely distorting predictions. Furthermore, data on expectancy bias and the overprediction of fear have demonstrated that higher anxiety—as in GAD—predicts greater overestimation of future aversive or fearful feelings and events (see Rachman, 1994, for a review). This tendency of anxiety to increase expectancy for unsavory outcomes may interfere with accurate outcome probability formation in GAD—even after repeated opportunities to learn those probabilities.

Studies suggest those with GAD do have problems estimating outcome probabilities. Excessive worrying has been linked to higher predicted likelihoods for the occurrence of negative events (Berenbaum, Thompson, & Bredemeier, 2007; MacLeod, Williams, & Bekerian, 1991). These predicted probabilities have been found to be higher in both GAD analogues and those meeting full criteria compared with control participants (Berenbaum, Thompson, & Pomerantz, 2007; Butler & Mathews, 1983). Importantly, such estimations of negative event occurrence tend to be highly inaccurate in GAD. One study found that when GAD clients tracked the actual outcomes of their worrisome predictions, only 15% of their worries were correct (Borkovec, Hazlett-Stevens, & Diaz, 1999). Of these bad outcomes, participants coped better than expected in 79% of instances. In another study, 91% of GAD participants’ predictions about feared future events were wrong (LaFreniere & Newman, 2016b). Although these data suggest probabilistic learning deficits may be likely, no study has experimentally examined whether those with GAD exhibit worse probabilistic learning than non-GAD persons. Although the cognitive bias and prediction literatures address expectations for threatening futures, they have not studied the learning of outcome probabilities based on feedback over time. Probabilistic learning is most often captured by how well participants learn the probabilities that various choices will achieve a target goal through trial and error (e.g., Frank, D'Lauro, & Curran, 2007). Yet this paradigm has never been applied to GAD.

In addition to probabilistic learning deficits, those with GAD may have unique tendencies in reinforcement learning. In particular, they may be more likely to learn from negative reinforcement—increasing a behavior’s frequency by removing an aversive stimulus (i.e., relief)—over positive reinforcement—increasing a behavior’s frequency by applying an enjoyable stimulus (i.e., reward). Gray’s (1991) reinforcement sensitivity theory (RST) and its updated formulations (Corr, 2013; Gray & McNaughton, 2000) are consistent with this prediction. Gray’s theory posits trait differences in sensitivity to reinforcers (the behavioral approach system; BAS), punishers, and the anxiety-related detection of conflicting information (the behavioral inhibition system; BIS). Accordingly, anxiety symptoms have been negatively associated with the BAS and positively associated with the BIS (Campbell-Sills, Liverant, & Brown, 2004; Kimbrel, Nelson-Gray, & Mitchell, 2007; Muris, Meesters, de Kanter, & Timmerman, 2005). Furthermore, the contrast avoidance model suggests that those with GAD are less likely to allow themselves to feel good for very long because it increases the likelihood that they will experience a negative emotional contrast if events turn out poorly. GAD individuals do in fact report lessened positive affect during worry (Llera & Newman, 2010; McLaughlin, Borkovec, & Sibrava, 2007). Those with GAD are thought to return to worry after good events, lessening positive emotion so as not to be caught “off guard.” Thus, those with GAD may attend to, savor, and maintain positive emotions less, diminishing the reinforcement value of rewarding outcomes.

Yet individuals with GAD may be especially sensitive to relieving outcomes. Worry, the cardinal symptom of GAD, is thought to be perpetuated by negative reinforcement: (a) Those with GAD frequently imagine aversive outcomes for upcoming events (worry), however improbable, (b) causing and/or maintaining distress; (c) when these unlikely outcomes do not occur, their relief provides negative reinforcement, and their worry is continuously utilized as a harmful coping mechanism (Newman & Llera, 2011). Findings have supported each of these stages. For example, both trait and experimentally induced worry have been associated with markers of sympathetic and parasympathetic nervous system activation (Brosschot, Van Dijk, & Thayer, 2007; Hammel et al., 2011; Llera & Newman, 2010, 2014; Pieper, Brosschot, van der Leeden, & Thayer, 2010) as well as self-reported increases in anxiety (e.g., Llera & Newman, 2010; Stapinski, Abbott, & Rapee, 2010). Since GAD worry is excessive, creates aversive feelings, and predicts negative outcomes that rarely happen, worry is continually reinforced when distress abates post-outcome. Secondarily, those with GAD may also prefer negative reinforcement to positive reinforcement as well as find it more motivating. Such preferences may be because of negative reinforcement’s familiarity and lesser tendency to increase positive affect, which leaves them feeling vulnerable. If true, greater preference and higher motivation may further explain better response to negative over positive reinforcement.

Unfortunately, experimental studies have not addressed probabilistic and reinforcement learning processes in GAD directly. Cognitive studies of GAD have almost exclusively focused on information-processing biases that devote greater attention toward threat (Mathews & MacLeod, 2005), such as selective attention toward stimuli that suggest future harm (Bar-Haim et al., 2007), deficits in cognitive control functions for shifting away from threat (e.g., Bishop, 2009; Tempesta et al., 2013), and problems in cognitive inhibition (especially inhibition of worry; Hallion, Tolin, Assaf, Goethe, & Diefenbach, 2017). Interpretation biases—in which emotionally ambiguous stimuli are construed as threatening—have also been demonstrated (e.g., Hazlett-Stevens & Borkovec, 2004). Even though memory bias findings are quite mixed, several studies do find biases for remembering personally relevant threat words (Coles et al., 2007). Although some studies used anxious samples in reinforcement paradigms, none have addressed biases in probabilistic and reinforcement learning in GAD. Further research is necessary to determine whether, compared with non-GAD persons, those with GAD are less able to learn probability estimations for reinforced outcomes and learn better in response to negative over positive reinforcement. Given that reinforcement learning is central to theories of GAD maintenance, this void in the literature is surprising—a state of affairs the current study sought to remedy. Deficits in probabilistic and reinforcement learning may facilitate threat-related attentional and interpretive biases. By keeping those with GAD from learning that nonthreatening outcomes are actually likely, they permit such negative filtering to continue unchecked. These deficits may also interfere with the recognition and recall of positive or neutral events that suggest harm is rarer than expected. If we better understand how such learning processes promote dysfunctional worry, we may better design and apply treatments to target those mechanisms.

The aim of the current study was to determine what, if any, outcome probability and reinforcement learning tendencies existed in those with GAD. On the basis of the aforementioned rationales, our primary hypotheses were: (a) Those with GAD would learn more slowly and to a lesser degree than those without GAD on a probabilistic reinforcement task, exhibiting a lesser increasing linear trend in accuracy over time and making fewer accurate choices in the task’s final trials, and (b) after learning in the task, those with GAD would make less accurate probability estimations for the likelihood of desirable outcomes than the control group. Secondarily, we hypothesized that in the GAD group, those in the negative reinforcement condition would show better learning than those in the positive reinforcement condition. More specifically, negative reinforcement would lead to a greater increasing linear trend of accuracy and more accurate choices in the task’s final trials than positive reinforcement for those with GAD. In addition, we expected lower self-reported BAS sensitivity and higher BIS sensitivity in GAD relative to the non-GAD group. To better interpret study results, we also ran exploratory analyses on participants’ reaction times (expecting no differences between diagnostic groups), motivation (expecting negative reinforcement to be rated as more motivating for those with GAD), and reinforcement preferences (expecting those with GAD to prefer negative reinforcement).

Method

Probabilistic reinforcement learning was studied via a computer-based experimental within- and between-subjects design. An Institutional Review Board approved this study.

Participants

Participants were identified through a university subject pool. The GAD sample was composed of individuals who met the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013) criteria for generalized anxiety disorder on the Generalized Anxiety Disorder Questionnaire (GAD-Q-IV; Newman et al., 2002). Non-GAD control participants were students who did not meet GAD criteria and scored half a standard deviation below the mean of the screened sample. All participants were at least 18 years of age. Data were collected from 271 participants, 166 GAD and 105 non-GAD. Of those with GAD, 63.3% scored above the cutoff for mild depression on the Beck Depression Inventory II (BDI-II; Beck, Steer, & Brown, 1996). This figure approximately matches the national rate of comorbidity between GAD and major depressive disorder (Kessler, DuPont, Berglund, & Wittchen, 1999). RMASS power analysis for a two-level linear mixed model accounting for repeated measures suggested the need for 216 participants (54 per cell) to achieve adequate power (.80). There were 219 females and 52 males. Ethnic breakdown was 81.7% White, 7.3% Asian, 4.2% Black, 5.2% Hispanic, 0.5% Middle Eastern, and 1.0% Pacific Islander.

Materials

Generalized Anxiety Disorder Questionnaire.

The GAD-Q-IV for the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994; Newman et al., 2002) is a nine-item self-report measure intended to assess DSM-IV or DSM-5 diagnostic criteria for GAD. Newman et al. (2002) found the GAD-Q-IV had good kappa agreement (κ = .67) with a structured interview. The measure also had strong convergent and discriminant validity (Newman et al., 2002). Internal consistency (Cronbach’s alpha = .94) was also robust, and it evidenced sufficient retest reliability (κ = .64; 92% of the sample). Penn State Worry Questionnaire (PSWQ; Meyer, Miller, Metzger, & Borkovec, 1990) scores of undergraduates identified as having GAD with the GAD-Q-IV were also not significantly different from baseline PSWQ scores in a treatment-seeking community sample (see Newman et al., 2002). Although it can be scored using a dimensional system, the criterion-based scoring system was used in this study, requiring that participants met full GAD criteria. When using criterion-based scoring, Newman et al. found 96% specificity and 67% sensitivity in detecting GAD. Also, in a primary care therapy-seeking sample, criterion scoring was the optimal strategy for identifying GAD (with a sensitivity of .89 and specificity of .82; Moore, Anderson, Barnes, Haigh, & Fresco, 2014).

Beck Depression Inventory II.

The BDI-II (Beck et al., 1996) was used to measure comorbid depression. It is a widely used, 21-item self-report survey designed to measure the severity of depressive symptoms in adults. It has demonstrated high internal consistency (α = .91; Duzois, Dobson, & Ahnberg, 1998). Persons diagnosed with major depressive disorder scored significantly higher on the BDI-II than nondepressed populations (Arnau, Meagher, Norris, & Bramson, 2001), demonstrating criterion validity. Convergent and discriminant validity have been demonstrated for the instrument (Beck et al., 1996).

The Behavioral Inhibition System/Behavioral Activation System Scales.

The 24-item Behavioral Inhibition System/Behavioral Activation System Scales (Carver & White, 1994), or BIS/BAS scales, measure reinforcement sensitivity as described in Gray’s (1991) reinforcement sensitivity theory. The BAS subscale measures sensitivity to signals of reward, nonpunishment, escape from punishment (negative reinforcement), and motivation to approach rewarding/reinforcing outcomes. The BIS subscale reflects sensitivity to signals of punishment and nonreward as well as motivation to avoid punishing outcomes. Note that although the revised RST places punishment sensitivity in the fight, flight, and freeze system, the BIS scale is used to measure punishment sensitivity because it was created alongside the original theory. Items are rated from 1 (very true for me) to 4 (very false for me). Multiple studies have supported the two-factor structure of the scale (e.g., Carver & White, 1994; Jorm et al., 1998; Maack & Ebesutani, 2018). Cronbach’s alpha was .76 for the BIS and .83 for the BAS subscales in one psychometric study (Jorm et al., 1998). In the current study, Cronbach’s alpha was .85 for the BIS and .86 for the BAS scale. Adequate retest reliability, convergent, and discriminant validity have also been established for both subscales (Carver & White, 1994; Jorm et al., 1998). The most recent psychometric analyses of the BIS/BAS scales suggest the BIS and BAS are best captured by separate and unidimensional measures, as we use them here (Maack & Ebesutani, 2018).

Post-Questionnaire.

The Post-Questionnaire (PQ) measure was developed for this study to examine participant’s posttask probability estimations of reinforcement and personal experiences in response to task conditions (see Appendix A). Items were worded differently to match participants' condition. The items assessed participants' probability estimations and how appealing, desirable, motivating, and preferable they found their reinforcement condition. Individual item responses were used in our exploratory analyses to aid interpretation of our primary findings.

Manipulation check.

Participants rated the valence of their emotional response (highly unpleasant to highly pleasant) to the angry and happy faces on a 9-point scale using the Self-Assessment Manikin (SAM; Bradley & Lang, 1994).

Procedure

Participants first completed informed consent, the GAD-Q-IV, and the BIS/BAS scales. They then engaged in the probabilistic learning task (PLT) and one other computerized task for a different study (not reported here). Tasks were counterbalanced across participants. For the PLT, participants were randomized to complete either a negative or a positive reinforcement version (each described in the following). They received on-screen instructions for each task. E-Prime 2.0 was used to present tasks and collect data. After the PLT, participants completed the PQ and manipulation check.

Negative reinforcement probabilistic learning task.

Study tasks were modeled after the social reward learning task of Lin, Adolphs, and Rangel (2012). In their task, probabilistic icons were presented in pairs, and participants chose between them to receive a social reinforcer. All other aspects of the task beyond this basic structure were modified in the current study (as described in the following). In the negative reinforcement PLT, participant choices were reinforced with the removal of an angry face (see Fig. 1).

Fig. 1.

Graphic example of the trial-to-trial sequence of the (top) negative and (bottom) positive reinforcement versions of the probabilistic learning task (PLT). Initially, participants view the leftmost screen, then view the subsequent screens if a stimulus choice is reinforced.

Participants were seated in front of a computer screen with the middle finger of each hand placed on the “H” and “L” keyboard keys. They received the following instructions:

In the following task you will be presented with many pairs of slot machine images, as well as the angry face of someone who strongly disapproves of you. Each slot machine will have a specific color and design. There is no deep meaning to the specific color or specific design of the slot machines. These features are only meant to help you distinguish one slot machine from another. If one color/design appears more often than others, that does not necessarily mean it is special. You can choose the slot machine on the left by pressing the "H" key (left hand). You can choose the slot machine on the right by pressing the "L" key (right hand). Each particular slot machine color/design has a unique probability of removing the face from the screen. In other words, some slot machines have a higher chance of removing the face than others do. Each slot machine's probability is consistent throughout the task. Your goal is to remove the angry face from the screen by choosing the slot machine that is most likely to lead to the face disappearing. When the face goes away, you will see a blank screen. Otherwise, the face will remain. After each trial, another pair of slot machines will appear, and the next trial will begin. You must learn by trial and error which slot machine has the best likelihood of removing the face in each pair. Pay attention to whether you made the face disappear or not. Make the face disappear as soon as possible. Choose as quickly as you can. You have a very short time limit for each choice.

Before advancing, participants had the opportunity to ask questions. Afterward, a centered cross appeared for 500 ms, and then trials began. Trials displayed paired images of two animated slot machines differing in color (blue, orange, green) and frontal design pattern (circle of dots, checkered squares, X of dots) on the left and right of the screen as well as a large, persisting color photograph of an angry face in the center (see Fig. 1). Face stimuli were from the NimStim collection (Tottenham et al., 2009). Each slot machine icon was associated with one of three probabilistic outcome distributions: high probability (80% chance of face removal), chance probability (50% chance of removal), and low probability (20% chance of removal). Each trial included either the high probability stimulus paired with the chance probability stimulus or the chance probability stimulus paired with the low probability stimulus. Participants underwent 100 trials (50 of each pair), presented in random order. Location of each stimulus (left or right) was randomized. Participants had 2.5 s to choose one of the two slot machines. After selection, slot machines disappeared from the screen. If a choice was reinforced, the angry face also disappeared from the screen for 2 s. If not reinforced, the angry face remained on the screen (without slot machines) for 2 s. Pilot testing determined these time periods to be optimal for task learning, brevity, and efficiency. Participants were not informed of the differing stimulus response probabilities beforehand; they had to learn them across trials.

Positive reinforcement probabilistic learning task.

In the positive reinforcement PLT, participants attempted to make a happy face appear via their choices (see Fig. 1). This task was directly equivalent to the negative reinforcement PLT with a few exceptions. Instructions were modified to inform participants to expect the appearance of a happy face for reinforced choices instead of the disappearance of an angry face. The initial trial screen included a pair of slot machine icons with no face image present. On choosing a reinforced stimulus, a happy face appeared for 2 s. On choosing a stimulus that was not reinforced, the screen remained blank for 2 s.

Data management

The PLT automatically progressed to the next trial if participants did not make a response within 2.5 s (this was to omit the presence of punishment, which would occur if we corrected participants for not responding quickly). All nonresponses because of inaction were marked as missing data (which was minimal; see Results section). Learning accuracy was determined by number of times a participant chose the higher probability icon over the lower probability icon. Reaction time for each trial was provided by E-prime. For analyses of learning on the PLT, data were organized into 10 blocks of 10 trials each. Blocked data—a common practice in cognitive studies (Estes, 2014)—controls for outliers and random mistakes in reaction time and allows for accuracy to be analyzed as a continuous variable over time. Within each block, accurate responses were summed, and reaction time was averaged.

Each stimulus’s likelihood of providing the reinforcement response was computed by subtracting each stimuli’s correct probability from each participant’s estimated likelihood. For these estimates, greater accuracy is indicated by scores closer to 0. For total probability estimation accuracy, the absolute values of the three difference scores were summed. Lower scores indicated better probability estimations.

Planned analyses

Independent samples t tests determined if there were differences between diagnostic groups in number of missing values, SAM emotional valence scores, and BIS/BAS scores. To accommodate correlated error terms in the many repeated measurements within participant, multilevel modeling was used to analyze longitudinal PLT data by block. For longitudinal analyses of both accuracy and reaction time after probabilistic learning, we used linear mixed models that included GAD status, reinforcement condition, linear trend across blocks, quadratic trend across blocks, and all possible interactions between these variables as fixed effects, with the intercept as a random effect. Additionally, we ran separate factorial ANOVAs in which GAD status, condition, and their interaction predicted number of accurate responses in the last three blocks. Lastly, we examined prediction of whether a participant was a high learner (achieved at least 80% accuracy in the last three blocks) with a logistic regression in which high learner status was predicted by GAD status, condition, and their interaction.

Differences between estimated and actual reinforcement probabilities for each stimulus—high (80%), chance (50%), and low (20%)—were analyzed separately. For each analysis, we ran a factorial ANOVA in which GAD status, reinforcement type, and their interaction predicted the various measures of probability estimation accuracy. For perceptions of participant experience on the PLT, the same factorial ANOVA predicted task undesirability, how motivating participants found their reinforcement condition, and degree of preference for the other reinforcement type. For all t tests and ANOVAs, we ran Levene’s tests for equality of variances because of our unequal sample sizes.

Partial eta-squared effect sizes were calculated for factorial ANOVAs with the formula η² = SS_between/SS_between + SS_error (where SS means sum of squares; Rosenthal & Rosnow, 1991). Cohen’s d was calculated for t tests with the common formula d = (M₂ − M₁)/SD_pooled in which $S{D}_{\text{pooled}}=\sqrt{[S{D}_1^2+S{D}_2^2]∕2}$ (Cohen, 1988). For mixed models, Cohen’s d was calculated using a formula designed for any repeated measures analysis that provides a t statistic, d = t(2/n)^1/2 (Dunlap, Cortina, Vaslow, & Burke, 1996). We used odds ratios for logistic regression.

Results

Missingness

Only 0.59% of values were missing. There was no difference in number of missing values between those with GAD (M = 0.658, SD = 3.822) and those without (M = 0.496, SD = 1.043), t(267) = −0.437, p = .662, d = 0.058; Levene’s test, F(2, 267) = 1.089, p = .298. Therefore, nonresponses were low and approximately equivalent for both diagnostic groups.

Manipulation check

The manipulation’s effectiveness was supported by a significant difference in valence scores between the negative (M = 3.443, SD = 1.41) and positive (M = 6.05, SD = 1.70) reinforcement conditions, t(268) = −13.636, p < .001, d = 1.669; Levene’s test, F(2, 268) = 1.155, p = .283. Within only the GAD sample, the difference between negative (M = 3.474, SD = 1.34) and positive (M = 5.9551, SD = 1.718) reinforcement conditions still remained, t(163) = −10.209, p < .001, d = 1.609; Levene’s test, F(2, 268) = 1.547, p = .215. This was also true of the control group (negative M = 3.400, SD = 1.522; positive M = 6.220, SD = 1.682), t(103) = −9.018, p < .001, d = 1.769; Levene’s test, F(2, 103) = 0.56, p = .813.

The BIS/BAS Scales

There was a significant difference between diagnostic groups in both Behavioral Inhibition System scores, t(222) = −12.175, p < .001, d = 1.627; Levene’s test, F(2, 222) = 0.461, p = .498; and Behavioral Activation System scores, t(222) = 2.498, p = .013, d = 0.335; Levene’s test, F(2, 222) = 0.796, p = .373. This suggested that those with GAD had a stronger avoidance of and sensitivity to signals of punishment and nonreward (M = 24.724, SD = 3.530) than control participants (M = 19.157, SD = 3.296). At the same time, those with GAD reported a less sensitive BAS (M = 39.017, SD = 6.283), or tendency to approach rewards and be sensitive to signals of reward, than control participants (M = 41.037, SD = 5.784).

Probabilistic learning

Accuracy.

When predicting PLT accuracy over time, the overall linear and quadratic trends were significant (see Table 1). Thus, both diagnostic groups showed learning across trial blocks. Notably, there was also a significant interaction between GAD status and linear trend in which those with GAD learned less and at a slower rate than those without GAD. No other significant findings arose from the model (see Table 1). Because of the presence of a significant quadratic trend in the whole sample but a difference only in linear trend between diagnostic groups, we conducted two follow-up analyses of linear and quadratic trend within each diagnostic group separately (condition and its interactions with time trends were also included as predictors in each model to reflect our original model). Those with GAD had a significant linear trajectory of accuracy over time (i.e., learning), t(576) = 2.625, p = .009, d = 0.301, but no quadratic trend, t(698) = −1.547, p = .122, d = −0.177. For the control group, there was both a significant linear, t(453) = 4.546, p < .001, d = 0.602, and quadratic trend, t(493) = −2.915, p = .004, d = −0.386, of change in accuracy (i.e., learning). On average, those with GAD never began to stabilize and had a lower slope of change, whereas control participants generally began to level out in accuracy after a greater slope of change. There was also a main effect of GAD status on total accurate responses, F(1, 264) = 16.007, p < .001, η_p² = .058; Levene’s test, F(3, 262) = 0.262, p = .925, with control participants having more accurate responses (M = 21.115, SD = 4.816) than those with GAD (M = 18.617, SD = 4.99). Yet again, no differences were found between conditions, F(1, 264) = 0.395, p = .530, η_p² = .002, and no interaction effect was present, F(1, 264) = 0.615, p = .434, η_p² = .002. Similarly, logistic regression showed a significant difference between diagnostic groups in whether a participant had 80% or greater accuracy in the last three blocks as well, in which control participants outperformed GAD participants (β = −1.063, p = .012, odds ratio [OR] = 0.345). Again, there was no difference between reinforcement conditions (β = 0.789, p = .574, OR = 0.789) or any interaction effects (β = −0.509, p = .388, OR = 1.664). Thus, similar to multilevel results, those with GAD exhibited worse probabilistic learning than control participants near the end of the task. Overall, those with GAD increased in accuracy more slowly and were ultimately less accurate than those without GAD regardless of reinforcement type. Secondary results failed to support the hypothesis that those with GAD would show superior learning by negative reinforcement compared with control participants.

Table 1.

Linear Mixed Model Effects for Accuracy and Reaction Time Across 10 Blocks of the Probabilistic Learning Task (Operant Conditioning)

Variable	t	p	d
Accuracy
Intercept	22.941	< .001**	1.989
GAD status	0.141	.888	0.012
Condition	0.396	.692	0.034
Linear trend	4.479	< .001**	0.388
Quadratic trend	−2.826	.005*	−0.245
GAD × Linear	−2.031	.043*	−0.176
GAD × Quadratic	1.381	.168	0.120
Condition × Linear	−1.084	.278	−0.094
Condition × Quadratic	.887	.375	0.077
GAD × Condition	−0.438	.278	−0.038
GAD × Condition × Linear	1.180	.238	0.102
GAD × Condition × Quadratic	−1.121	.263	−0.097
Reaction time
Intercept	24.814	< .001**	2.152
GAD status	−1.547	.123	−0.134
Condition	−1.340	.181	−0.116
Linear trend	−4.886	< .001**	−0.424
Quadratic trend	3.404	.001**	0.295
GAD × Linear	0.405	.685	0.035
GAD × Quadratic	−0.532	.595	−0.046
Condition × Linear	0.989	.323	0.086
Condition × Quadratic	−1.096	.273	−0.095
GAD × Condition	0.210	.834	0.018
GAD × Condition × Linear	−0.103	.918	−0.009
GAD × Condition × Quadratic	0.558	.577	0.048

Note: GAD = generalized anxiety disorder.

^*p < .05.

^**p < .01.

Reaction time.

There was only a significant linear decrease and quadratic leveling out in reaction times across all participants (see Table 1). Both diagnostic groups got similarly faster in reaction time up to a point regardless of condition. There were no other significant effects (see Table 1).

Probability estimations after learning

There was a main effect of GAD status on probability estimation in which those with GAD were less accurate than those without GAD (see Table 2). However, there was more variability in the estimations of those with GAD (SD = 28.114) than control participants (SD = 20.178). For the high (80%) probability icon, there was a main effect of GAD status but no effect of condition or the interaction (see Table 2). Supporting our prediction, those with GAD underestimated the likelihood of high probability reinforcement to a greater degree than those without GAD. For the chance (50%) probability icon, results revealed no effects for likelihood estimation (see Table 2). For the low (20%) probability icon, there were also no likelihood estimation effects (see Table 2). Thus, those with GAD did not make worse probability estimations when the likelihood of reinforcement was low or at chance levels, but they did make worse estimations when that likelihood was high.

Table 2.

Factorial ANOVA Results for Probability Estimations of Stimulus Reinforcement Likelihoods After the Probabilistic Learning Task

Probability Estimation Outcome	GAD M(SD)	Non-GAD M(SD)	Predictor	F	p	ηp2x	Levene’s test
							F	p
Total probability estimation accuracy	54.827 (28.114)	47.600 (20.178)	GAD status	5.246	.023*	.019
			Condition	0.099	.753	.000	2.551	.056
			GAD × Condition	0.000	.983	.000
High (80%) probability stimulus accuracy	−25.382 (25.741)	−17.210 (23.000)	GAD status	6.835	.009**	.025
			Condition	0.053	.818	.000	1.092	.353
			GAD × Condition	0.138	.711	.001
Chance (50%) probability stimulus accuracy	−3.229 (18.804)	−4.771 (17.397)	GAD status	0.356	.551	.001
			Condition	1.737	.189	.007	0.432	.737
			GAD × Condition	0.031	.860	.000
Low (20%) probability stimulus accuracy	6.045 (15.676)	3.943 (13.783)	GAD status	1.444	.231	.005
			Condition	0.945	.332	.004	0.380	.768
			GAD × Condition	1.956	.163	.007

Note: GAD = generalized anxiety disorder.

^*p < .05.

^**p < .01.

Perceptions of task experience

Regarding task undesirability (PQ Item 2), there was a main effect of GAD status, F(1, 264) = 4.495, p = 0.035, η_p² = .017, but no condition, F(1, 264) = 1.776, p = .184, η_p² = .007, or interaction effects, F(1, 264) = 0.226, p = .635, η_p² = .001. Levene’s test for unequal variances was significant, F(3, 265) = 3.559, p = .015. However, a Welch’s unequal variances t test was significant for diagnostic group, F(1, 251.76) = 5.285, p = .022, but not for condition, F(1, 264.35) = 1.899, p = .169. Thus, regardless of reinforcement type, those with GAD found the PLT less desirable M = 4.642, SD = 2.609) than those without GAD (M = 3.971, SD = 2.128). For how motivating each participant found his or her respective reinforcement condition (PQ Item 6), there was only a significant main effect of condition, F(1, 267) = 6.074, p = .014, η_p² = .022; Levene’s test, F(3, 265) = 1.777, p = .152. Positive reinforcement was experienced as more motivating (M = 6.138, SD = 2.242) than negative reinforcement (M = 5.351, SD = 2.376) for both groups. No motivation effects were found for GAD status, F(1, 267) = 0.369, p = .544, η_p² = .001, or the interaction, F(1, 267) = 0.315, p = .315, η_p² = .004. When asked to what degree participants would prefer the condition they did not undergo (PQ Item 8), there was a main effect of condition, F(1, 268) = 57.203, p < .001, η_p² = .177, and a main effect of GAD status, F(1, 268) = 4.398, p = .037, η_p² = .016; Levene’s test, F(3, 266) = 2.470, p = .062, but no interaction, F(1, 268) = .062, p = .804, η_p² = .000. Regardless of GAD status, those in the negative reinforcement condition believed they would prefer positive reinforcement (M = 4.817, SD = 3.150) more than those in the positive condition believed they would prefer the negative (M = 2.022, SD = 2.845). In addition, those with GAD believed they would prefer the condition they did not receive (M = 3.612, SD = 3.446) more so than control participants (M = 3.010, SD = 3.043) regardless of which condition they actually experienced. Thus, both those with and without GAD believed they would prefer positive over negative reinforcement and found positive reinforcement more motivating.¹

Discussion

Theory suggests that deficits in probabilistic learning and high response sensitivity to negative reinforcement may maintain worry. Thus, we predicted that those with GAD would perform worse than non-GAD control participants at probabilistic learning and be more sensitive to negative over positive reinforcement within-group. Contrary to expectations, negative reinforcement did not lead to better learning than positive reinforcement within the GAD group. Yet supporting our hypotheses, probabilistic learning accuracy showed that those with GAD did learn at a slower rate and to a lesser degree than those without GAD. Furthermore, probability estimations reported after learning revealed that those with GAD were generally less accurate at estimating probabilities of desirable outcomes for both types of reinforcement. In follow-up analyses, both diagnostic groups were equally accurate at estimating good outcome likelihoods when those likelihoods were low or at chance levels, but those with GAD were less accurate than control participants when good outcome likelihoods were high. Moreover, none of our findings changed after controlling for depression symptoms. Thus, those with GAD appear to underestimate the probability of desirable outcomes even after having ample opportunity to learn these probabilities.

Our primary finding showed deficits in probabilistic learning for those with GAD. Compared with the control group, the learning of those with GAD increased in accuracy less over time, showed a slower trajectory of change, and ultimately reached a lower level of accuracy at the end of the task regardless of reinforcement type. This learning never began to stabilize in those with GAD, whereas control participants began to level off at higher degrees of accuracy. There were also significantly fewer high learners in the GAD group than the non-GAD group. Our analysis of reaction time analyses showed that this was neither because of those with GAD responding too quickly to be accurate nor anxiously hesitating to make a choice: No group differences in reaction times arose. Yet why might those with GAD show worse probabilistic learning? These results add to existing literature underscoring GAD-specific difficulties with learning outcome likelihoods over time. Prior studies showed both that those with GAD predicted high likelihoods that future negative outcomes would occur (Berenbaum, Thompson, & Pomerantz, 2007; Butler & Mathews, 1983) and that those predictions were highly inaccurate when actual outcomes were monitored in daily life (85%–91% of worries do not come true; Borkovec et al., 1999; LaFreniere & Newman, 2016b). Although previous studies had only examined this problem at a general level, the current study showed deficits in probabilistic learning at a finer grain—in-the-moment feedback learning on the scale of a cognitive task. But does this tendency align with theory?

Theory suggests that GAD persons should have probabilistic learning deficits as a function of their disorder. If those with GAD worry about future events to protect against unsavory affect change (Newman & Llera, 2011) and also view their worry as positive, useful, and self-defining (i.e., positive worry beliefs; Hebert, Dugas, Tulloch, & Holowka, 2014), then they are neither motivated to learn nor reinforced for learning the true probabilities of future events. They are intrinsically rewarded for making poor predictions about probabilistic outcomes because of worry’s utility for contrast avoidance. Thus, by learning the low probability of their fears coming true, they risk making themselves vulnerable to an unpleasant emotional contrast. Across their development long-term, they may not learn to estimate realistic likelihoods by attending to the actual event outcomes of their lives. To do so would sacrifice bracing for unexpected downturns—a trade too costly to make. Consistent with the current findings, these issues with likelihood may also reside at the level of a rapid cognitive task.

It is possible that such learning deficits may exist before clinical levels of worry arise because of some basic cognitive deficiency. For instance, biased memory for personally relevant threat may disproportionately weight negative outcomes more strongly during learning, distorting prediction formation and lessening reinforcement response later in life (Coles et al., 2007). Alternatively, these deficits may emerge through habit after continually engaging in unrealistic worry, then selectively attending to threatening outcomes while not registering the inaccuracy of most worrisome predictions. Insecure attachment styles in childhood and the parental factors associated with them—commonly linked to GAD (Newman, Shin, & Zuellig, 2016)—may also result in skewed probability learning. High uncertainty about parental reinforcement and unrealistic expectations for negative outcomes (parental rejection, mistreatment, etc.) may shape this process. In brief, probabilistic learning deficits facilitate contrast avoidance and other functions of worry by distorting the likelihoods of innocuous outcomes. These faulty predictions permit clients to stave off affective whiplash through continual worry.

It is also possible that those with GAD may underperform at general reinforcement learning beyond this task’s probabilistic nature. Again, no differences were found between learning response to positive versus negative reinforcement. Rather, those with GAD exhibited worse learning by both reinforcement types when compared with control participants. Whether the consequence is reward or relief, those with GAD may not attend to, respond to, or learn from good outcomes as well as healthy persons do. Other studies align with a general reinforcement deficiency in GAD. In this group, worry and rumination have predicted attenuated positive affective response to success and heightened negative affective response to failure (Ruscio, Seitchik, Gentes, Jones, & Hallion, 2011). Moreover, Zinbarg and Revelle (1989) found that those with high anxiety were poor at learning to make reward-achieving responses, though they rapidly learned to avoid punishment by inhibiting responses. In contrast, those low on anxiety did quickly learn to make rewarding responses but had difficulty making punishment-avoiding responses. In fact, our own analyses of BIS/BAS scores revealed that those with GAD reported significantly less sensitivity to and motivation toward signals of reward (positive reinforcers) and escape from ongoing punishment (negative reinforcers) than non-GAD persons (i.e., lower BAS sensitivity). Those with GAD may not be registering reinforcement very well, leading to worse learning from desirable consequences. When choices lead to outcomes that induce positive emotion, they may not dwell on that reward long enough to be adequately reinforced. In accordance with contrast avoidance, those with GAD are likely to quickly dismiss positive feelings because such feelings leave them vulnerable to greater shifts. Without acknowledging and learning from positive outcomes, it is then no surprise that people with GAD often have “apprehensive expectations” for the future.

Alternatively, negative stimuli and emotion may draw those with GAD away from reinforcers. Accordingly, compared with the non-GAD group, the GAD group had a significantly higher BIS scale score, revealing a higher sensitivity to signals of punishment and stronger motivation to avoid it than control participants. Elevated vigilance and reactivity toward threatening stimuli may cause those with GAD to attend more to negative, anxiety-causing information, taking up attentional resources that could be used to attend to positively reinforcing information. They exhibit a heightened alertness to perceived external threats and threatening faces and react with strong negativity to even neutral or ambiguous stimuli (Mogg, Millar, & Bradley, 2000; Waters, Mogg, Bradley, & Pine, 2008). In addition, poorer attentional control has been found in those with GAD compared with those without GAD (Olatunji, Ciesielski, Armstrong, Zhao, & Zald, 2011). It may be the case that those with GAD not only focus on punishing information but also have greater difficulty moving attention away from such information and toward reward.

The current study found that those with GAD not only dismissed positive experience in their present but also expected it less in their future. After participants finished the probabilistic learning task, we assessed their explicit estimations of the reinforcement probability. Compared with control participants, GAD participants’ probability estimations were less accurate when summed across all probabilistic stimuli. Although in previous studies those with GAD made higher probability estimations of negative outcomes, this study was the first to show they also made lower estimations of positive outcomes, particularly when the likelihood of such outcomes was high. In fact, on average, those with GAD perceived the 80% reinforcement stimulus to be nearly 50/50 chance (54.6%). Thus, on the basis of their prior learning, those with GAD appear to make correct predictions when pleasing outcomes truly are unlikely but are inaccurate when pleasing outcomes are actually quite likely. They may fail to learn when positive outcomes are probable so that high expectations will not set them up for negative contrasts. Such a process perpetuates worrying even when feared outcomes are highly improbable.

Turning to the phenomenological experience of these tasks, those with GAD found the PLT more undesirable than control participants regardless of reinforcement type. Thus, compared with control participants, those with GAD both performed more poorly on probabilistic reinforcement tasks and were also more likely to find them aversive. Such distaste for probabilistic choices may be related to intolerance of uncertainty—a construct highly associated with GAD (Dugas, Buhr, & Ladouceur, 2004). By nature, uncertain outcomes are probabilistic outcomes: One is unsure of an ensuing consequence and must make a probabilistic judgment about possible outcomes. If those with GAD are worse at doing so and the stakes are high, such choices may be bothersome. This discomfort can then further increase distress for avoiding contrast. Furthermore, the GAD group likely also experienced less reinforcing feedback because of making fewer accurate probabilistic choices, which may have made the task less enjoyable. Lastly, contrary to hypotheses, both those with and without GAD preferred to learn by positive over negative reinforcement. They also found it to be more motivating. Perhaps those with GAD do find positive reinforcement to be more enjoyable and motivating immediately after experiencing it but ultimately learn worse by not dwelling on and savoring that reward.

The current study was not without limitations. As with any computerized cognitive task, generalizability must be interpreted with some caution. Although it allows for strict experimental control, the real-world relevance of making rapid choices based on artificial reinforcers may be limited. Our study may have benefitted from including more trials. Our trial amount (100) was based on Lin et al.’s (2012) findings that most people learned correct PLT pair choices completely after only 40 trials. This was not the case in our sample, especially for those with GAD. Designing a longer task may provide more information on learning, inquiring into how long it takes until perfect choices are consistently made. Also, because of the use of the same icon for the 50% probability stimulus in both trial types (80% vs. 50%, 20% vs. 50%), one could argue that our task involves some degree of cognitive flexibility in addition to learning. This may be considered a limitation in our attempt to measure pure learning. In addition, although the manipulation check showed facial reinforcers did work to create aversion and pleasure, greater distress or pleasantness could have been induced, leading to stronger learning.

With regard to future studies, probabilistic learning deficits in GAD need to be studied to a much greater degree with varying designs and more complex tasks. Our study is only the first to demonstrate deficits in probabilistic learning over time in those with GAD. Replication is necessary. Also, testing different operant consequence comparisons (e.g., punishment vs. reward or positive vs. negative punishment) and different types of learning (e.g., classical conditioning or modeling) in a probabilistic framework is warranted as well. To test learning deficits more broadly, GAD and non-GAD persons’ performance will ultimately need to be compared with an identical but nonprobabilistic task of comparable difficulty. Another question for ensuing research is whether actively worrying affects probabilistic learning accuracy, perhaps tested by comparing a pretask worry induction to a relaxation period. We also suggest that new research address GAD reinforcement learning strategies across trials, perhaps by using trial-by-trial computational modeling to analyze patterns of outcome predictions and prediction errors (e.g., Radulescu, Daniel, & Niv, 2016). Finally, future studies would do well to see if GAD probabilistic learning can be improved by treatment.

Fortunately, GAD treatments for improving estimated outcome probabilities, such as worry outcome monitoring, already exist and are supported (LaFreniere & Newman, 2016a). Perhaps such therapies also alter probabilistic learning processes. The clinical implications of probabilistic learning deficits and worse conscious learning from reinforcement are significant. These deficits may be a maintenance factor for GAD, facilitating ongoing contrast avoidance by leaving faulty catastrophic predictions undisputed by a poorly trained mind. Therapy may benefit from guiding those with GAD to savor rewards and attend to favorable outcomes. Such treatment targets may not only help clients learn better and form healthier, more accurate predictions but also provide them greater exposure to contrasts by extending postoutcome positive feeling. This exposure may increase their tolerance for being open to unexpected emotional downturns, ultimately reducing the need to gird themselves with worry. Whatever outcome, auspicious or otherwise, these newfound deficits in probabilistic learning demand further exploration.

Appendix A

Poststudy Questionnaire

On a scale of 1 (not appealing at all) to 10 (highly appealing), how appealing did you find the slot machine task in general? On a scale of 1 (not undesirable at all) to 10 (highly undesirable), how undesirable did you find the slot machine task in general? In the slot machine task, what would you estimate the probability, out of 100, that choosing the blue slot machine resulted in the appearance of a happy face [disappearance of an angry face]? In the slot machine task, what would you estimate the probability, out of 100, that choosing the orange slot machine resulted in the appearance of a happy face [disappearance of an angry face]? In the slot machine task, what would you estimate the probability, out of 100, that choosing the green slot machine resulted in the appearance of a happy face [disappearance of an angry face]? On a scale of 1 (not motivating at all) to 10 (highly motivating), how motivating was the appearance of a happy face [disappearance of an angry face] for learning which slot machine to choose? For the slot machine task, do you think you would have preferred learning the correct choice of slot machine by working to remove an angry face [make a happy face appear] from the screen rather than getting a happy face [removing an angry face] on each correct choice? (Yes or No) If yes, on a scale of 1 (very slightly preferable) to 10 (very highly preferable), how preferable would you have found learning by removing an angry face [making a happy face appear] to learning by getting happy face responses [working to remove an angry face]?

Note: Language differed by condition in accordance with the reinforcement process of each condition. Alternative prompt text is indicated in square brackets.