Examining Social Reinforcement Learning in Social Anxiety

Introduction

Socially anxious individuals tend to have negatively biased thoughts about social situations and dread or avoid social situations, which can be highly impairing (Hirsch & Clark, 2004). Concerns about rejection and subsequent behavioral avoidance may be partially explained by basing one’s expectations of how others will respond to you in the future more on previous rejections than on previous approvals. This study uses computational modeling to investigate social anxiety-based differences in social reinforcement learning; that is, how high versus low socially anxious individuals learn differently from rewarding versus rejecting social outcomes.

Computational Modeling of Social Reinforcement Learning in Social Anxiety

Computational psychiatry aims to understand mental illnesses by algorithmically modeling neural and cognitive processes to pinpoint where dysfunctions occur (Montague et al., 2012). Computational studies, like the present study, are well suited to test how sensitivity to rewarding and punishing feedback and responses to uncertainty may contribute to anxiety disorders (Raymond et al., 2017).

Here, we examine biased social reinforcement learning, a potential treatment target that remains understudied in social anxiety disorder. Reinforcement learning (RL) describes the process by which people learn to predict outcomes and optimize behavior in an environment where taking actions leads to rewards (positive outcomes) and punishments (negative outcomes like rejection; Sutton & Barto, 1998). RL differences between healthy and clinical populations have been documented across many disorders (Whitton et al., 2016). Unlike other disorders in which RL has been studied, aberrant RL in social anxiety may be specific to the social domain. Specifically, several neuroimaging studies suggest that the relative motivational preference for social reward is disrupted in social anxiety, and processing of social punishments may be enhanced (Becker et al., 2017; Cremers et al., 2015; Freitas-Ferrari et al., 2010; Richey et al., 2014, 2017, 2019; Sripada et al., 2013). As such, computationally modeling social RL may help pinpoint where disruptions to learning occur in socially anxious individuals (i.e., are they quick to update their beliefs about others after negative interactions, but slow to do so after positive interactions?). This level of precision may eventually lead to the development of more targeted interventions.

Anxiety-Related Differences in Learning Rate

One useful RL parameter for understanding how people use information to update their beliefs is the learning rate, or the degree to which one weights recent information relative to more distal information. Imagine that a generally friendly coworker appears annoyed by something you say; if you have a higher learning rate for this negative social feedback, you might expect her to be more annoyed again tomorrow, whereas if you have a lower learning rate, you might instead draw more on your past interactions and expect her to be friendly the next day. In nonsocial environments, anxious individuals appear to update their expectancies too much from negative outcomes (Aylward et al., 2019; Huang et al., 2017). However, a recent study found that anxious participants were slower than healthy control participants at learning to stop investing in exploitative social partners, suggesting anxious individuals might update their expectancies less (rather than more) from negative social outcomes (Lamba et al., 2020). While these studies all provide evidence of suboptimal decision-making under uncertainty tied to learning rate differences in anxious individuals, discrepancies in their results suggest that more research is needed to understand how social RL may differ from RL more broadly.

Regarding social RL specifically, two studies assessed whether social anxiety affects learning rate for updating mental representations of other people, both in volatile learning environments (environments in which the probabilities of other people responding in rewarding and punishing ways change over time; Beltzer et al., 2019; Piray et al., 2019). Congruent with findings for anxious individuals with nonsocial stimuli (Browning et al., 2015), both studies found that socially anxious individuals appear to have difficulty making use of information about increasing environmental stability in threatening social environments (Beltzer et al., 2019; Piray et al., 2019). Specifically, using an online ball-throwing paradigm in which the avatars’ probabilities of throwing the ball back to the participant changed several times throughout the game, participants who were more (vs. less) socially anxious showed less of a decrease in learning rate—essentially, continuing to learn more from new interactions, rather than settling into a more stable mental representation—after the punisher avatar demonstrated a more stable punishment contingency by staying the same between two blocks (Beltzer et al., 2019). Consistent with this finding, Piray and colleagues (2019) found that on a reversal learning task using happy and angry faces in which the probability of a positive outcome for pressing a button (vs. not pressing) changed multiple times throughout the task, participants high (vs. low) in trait social anxiety adjusted their learning rate less on trials starting with angry faces, suggesting they adapted their learning process less in response to volatility of social punishment continencies.

These studies provide valuable information about learning rate in volatile social environments, but leave unanswered how learning rate in a socially anxious population occurs in more stable environments, where other people’s reward and punishment contingencies stay the same over time. Both are likely important to healthy social functioning; consider, for instance, friendships in high school versus adulthood. High schoolers might try on new identities and shift between friends more quickly than adults; someone who might be mostly friendly to you one month might join a new social group and become less welcoming to you the next month. Thus, in this more volatile social environment, focusing more on recent interactions with others (higher learning rate) might be more adaptive than in adulthood, where relationships with friends are likely more stable and one unexpected interaction more likely reflects noise than a change in their underlying reinforcement contingency. This study aims to fill a gap in the literature on whether socially anxious individuals exhibit aberrant learning rates in stable environments, in which the probabilities of other people responding in rewarding and punishing ways stay the same over time (as might occur frequently in daily life during periods without major changes in one’s social environment).

Anxiety-Related Differences in Choosing Rewarding and Avoiding Punishing Stimuli

Social anxiety may also be characterized by differences in how individuals apply learned expectations for social reward and punishment when faced with a choice. For instance, a socially anxious person who has learned that a particular colleague tends to reject their ideas might consistently avoid sharing their thoughts with that colleague, thereby showing good accuracy at avoiding social punishment. Here, being accurate means that, when you are faced with a decision, you choose the action more likely to lead to reward and less likely to lead to punishment. Thus, we can examine both how readily a person learns from recent information that the colleague is rejecting (i.e., learning rate) and the person’s accuracy in effectively choosing reward and avoiding punishment when they next have an opportunity to apply that learning with their colleague (by analyzing how frequently the person successfully chooses rewarding and avoids punishing stimuli, termed learning accuracy).

Findings on anxiety-related biases in reward and punishment learning accuracy have been mixed. One study found that within a depressed sample, higher anxiety symptoms predicted greater accuracy for avoiding punishing stimuli (Cavanagh et al., 2019), but not for choosing rewarding stimuli. This suggests that anxiety is specifically related to enhanced learning to avoid punishment. However, another study found that people with generalized anxiety disorder were worse than control participants at learning to choose rewarding and avoid punishing social outcomes (LaFreniere & Newman, 2019). Clearly, more research is needed to determine anxiety-linked learning accuracy around potential rewards and punishments.

Regarding social anxiety specifically, more socially anxious individuals appear to be more likely to avoid socially punishing stimuli compared to control participants. In one study (Abraham & Hermann, 2015), participants completed a version of a popular probabilistic selection task (Frank, Seeberger, & Reilly, 2004) that was adapted to include social stimuli and social feedback. In the first phase of the social probabilistic selection task, called the training phase, participants were presented with pairs of neutral faces with varying probabilities of providing social reward (a happy face) or punishment (an angry face) when chosen. On the pair with the most similar probabilities, more socially anxious participants were less likely than healthy control participants to choose the more rewarding face. This suggests there may have been disrupted learning for the socially anxious participants, but we cannot deduce whether it was aberrant reward or punishment learning without computationally modeling the training phase data (a gap in the literature that will be addressed by the current study). Following the training phase was a testing phase, during which the most rewarding and most punishing faces were presented in combination with each of the other faces, and no feedback was given. In the testing phase, more socially anxious participants were more accurate than control participants at avoiding the most punishing face.

Voegler and colleagues (2019) compared performance of socially anxious and healthy control participants on a non-social version of the same task under conditions of social observation versus no observation. Similar to Abraham and Hermann (2015), they found that socially anxious participants were more accurate at avoiding punishment than choosing reward (though only when they were not being observed). Together, these results suggest that social anxiety may be characterized by a tendency to be more accurate at learning to avoid social punishment than choose social reward.

Overview of the Present Study

The current study assessed the extent to which social anxiety is characterized by differences in learning how likely other people are to be socially rewarding or punishing. We analyzed how learning rates for social reward and punishment differ as a function of social anxiety symptoms by applying an RL model to the training phase of a social probabilistic selection task. We also analyzed how social anxiety affects accuracy at choosing social stimuli that predicted reward and avoiding social stimuli that predicted punishment during the testing phase. To do this, we extended Abraham and Hermann’s (2015) methodology by using a very similar task and adding computational modeling to provide more insight into potential social RL biases in social anxiety. This study contributes to the literature by: a) assessing whether social anxiety is characterized by differences in learning rates for social reward and social punishment when learning about other people in a stable environment; b) attempting to replicate findings of social anxiety-related avoidance of punishing stimuli; and c) extending this evaluation of reward and punishment learning accuracy to understand how social anxiety affects making fine-grained distinctions between social stimuli (described more fully below). Importantly, this test applies computational modeling to social stimuli with very high relevance; as Simion and Giorgio (2015) note, “Among other social cues in the environment, faces are probably the most important to us as humans” (p. 969).

Hypotheses

Given previous findings that higher anxiety is related to greater accuracy in learning to avoid punishing stimuli (Cavanagh et al., 2019), and that higher social anxiety is related to greater avoidance of punishing social stimuli (Abraham & Hermann, 2015), we hypothesized that more socially anxious participants would show higher accuracy in learning to avoid punishing social stimuli relative to accuracy in learning to choose rewarding social stimuli, as compared to participants lower in social anxiety symptoms.

To incorporate more of the data from the testing phase (i.e., trials that did not include the most rewarding and most punishing faces) and to add more nuance to our understanding of reward and punishment learning accuracy, we also conducted a secondary analysis on accuracy of learning to discriminate between stimulus pairs of varying difficulty and varying average reward probabilities. Understanding learning rate differences tied to both difficulty (difference between reward probabilities) and magnitude (average reward probability) allows us to capture more of the variable factors that arise in daily life when one is evaluating social contacts, especially given rewarding and punishing social behaviors are often subtle (e.g., a small slight at a party rather than outright shunning). Previous research has found that people tend to be slower and less accurate when making decisions between pairs with more similar reward probability, whereas they are faster (but not less accurate) when making decisions between highly rewarding pairs (Fontanesi et al., 2019). We hypothesized that more socially anxious participants would show higher accuracy (selecting the more rewarding face) on difficult (i.e., more similar) pairs that were relatively more punishing, whereas participants lower in social anxiety would show higher accuracy on difficult pairs that were relatively more rewarding. This effect would be consistent with better punishment learning accuracy (relative to reward learning accuracy) in participants with higher social anxiety, as compared to lower social anxiety.

Given findings of enhanced negative affect, positivity deficits, and disrupted social reward processing in social anxiety, we hypothesized that more socially anxious participants would show a more negative bias in learning rates, which might be expressed as higher punishment learning rates and/or lower reward learning rates, as compared to less socially anxious participants. That said, we expected that social anxiety-related differences in learning rates may have smaller effect sizes than the learning accuracy effects, and therefore might not be detected in the present study for two reasons. First, compared to other computational parameters (e.g., temperature), prior research has found less consistent evidence of altered learning rates in anhedonia (Robinson & Chase, 2017). Second, anxiety-related differences in learning rates may be related to poor adaptation to environmental volatility (Beltzer et al., 2019; Browning et al., 2015; Huang et al., 2017; Piray et al., 2019) and the present study investigated learning in a stable environment. Plans for analyses and hypotheses were preregistered at https://osf.io/4hu8e/?view_only=f81e893197574846a8aff6b62be39ca5.

Method

Participants

N=157 adults (18–45 years old) were recruited from the University of Virginia undergraduate participant pool and the surrounding community. Community participants were recruited through advertisements sent to university email lists for undergraduate and graduate students and flyers posted in public areas. Prospective participants were screened for social anxiety symptoms with the Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998). The sample included n=114 participants with moderate to severe social anxiety (scoring ≥29 out of 80, approximately ¼ of a standard deviation below the mean in a sample diagnosed with social phobia; Mattick & Clarke, 1998) and n=43 participants with low social anxiety (scoring ≤10, ¾ of a standard deviation below the mean of a community sample; Mattick & Clarke, 1998)¹. Because participants in the high social anxiety group were enrolled in a five-week ecological momentary assessment (EMA) for the parent data collection (not analyzed here), these participants were only eligible if they had a smartphone that was compatible with the mobile app used for the EMA. Participants in the low social anxiety group were not invited to the EMA portion of the study so this exclusion criterion did not apply.

From the social probabilistic selection task, data from the training phase were available for n=42 low social anxiety and all n=114 high social anxiety participants. Data from the testing block were available for n=41 low social anxiety and n=113 high social anxiety participants. In the low social anxiety group, data from the social probabilistic selection task did not save correctly for one participant, and there was a technical issue during the testing block for another. In the high social anxiety group, one participant’s testing block was discontinued early due to experimenter error. Demographics for the N=156 participants included in analyses are in Table 1.

Table 1.

Demographics by Social Anxiety Group

Variable	Low Social Anxiety	High Social Anxiety
n	42	114
Sex
Females (%)	29 (69.05%)	85 (74.56%)
Males (%)	13 (30.95%)	29 (25.44%)
Non-binary identity (%)	0 (0%)	0 (0%)
Mage in years (SDage)	19.31 (1.87)	20.37 (2.94)
Undergraduates (%)	39 (92.86%)	90 (78.95%)
Ethnicity
Latinx/Hispanic (%)	2 (4.76%)	3 (2.63%)
Not Latinx/Hispanic (%)	40 (95.24%)	111 (97.37%)
Prefer Not to Answer (%)	0 (0%)	0 (0%)
Race
White (%)	31 (73.81%)	84 (73.68%)
Asian (%)	9 (21.43%)	22 (19.30%)
African American/Black (%)	0 (0%)	10 (8.77%)
Middle Eastern (%)	2 (4.76%)	3 (2.63%)
American Indian/Alaska Native (%)	0 (0%)	0 (0%)
Native Hawaiian/Pacific Islander (%)	0 (0%)	3 (2.63%)
Social Anxiety Symptoms	M=4.98	M=38.45

Note. Social anxiety symptoms were measured by the sum of the straightforwardly worded items of the Social Anxiety Interaction Scale. By design, social anxiety symptoms were significantly lower in the low social anxiety group than in the high social anxiety group t(146.89) = 34.44, p < 0.001, d = 4.08.

Measures

Social Interaction Anxiety Scale

The Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998) is a 20-item self-report scale measuring social anxiety in dyads and groups (e.g., “I find myself worrying that I won’t know what to say in social situations.”). Participants rate their endorsement of each item on a 5-point Likert scale ranging from “not at all” to “extremely.”

Social Probabilistic Selection Task

The Social Probabilistic Selection Task (SPST; Abraham & Hermann, 2015) is a widely used probabilistic category-learning paradigm (Frank et al., 2004) that was adapted to include socially relevant information as stimuli (neutral faces) and socially evaluative reinforcement as feedback (reward: happy faces, punishment: angry faces). Prior studies using this task have found that, compared to healthy control participants, socially anxious participants have greater social punishment learning accuracy during the testing phase (Abraham & Hermann, 2015) and greater non-social punishment learning accuracy during the testing phase when not under social observation (using a non-social version of the task; Voegler et al., 2019), supporting the validity of the task to distinguish between known groups.

The SPST consisted of two phases: training and testing (see Figure 1). In the training phase, participants were shown two neutral faces at a time, and told that one face would become happy if chosen, while the other would become angry. They were instructed to select the face they thought was more likely to become happy. The training pairs had different complementary reward contingencies, but these were never explicitly told to participants; instead, they learned through trial and error. In one training pair, one face became happy 80% of the times it was chosen and angry 20% of the times it was chosen, whereas the other face became happy 20% of the time and angry 80% of the time. This contingency is referred to as 80/20, and we refer to the stimuli by their reward probabilities 80 and 20, respectively. The other pairs had 70/30 and 60/40 contingencies. Participants completed at least two training blocks of 60 trials, with each training block consisting of 20 trials of each pair (10 with the more rewarding face on the left, 10 with the more rewarding face on the right). To ensure that participants were performing similarly to each other before advancing to the testing phase, we used performance criteria that required higher accuracy on more difficult face pairs. Participants advanced to the testing phase once they reached accuracy criteria of 64% correct identifications (i.e., selecting the more rewarding face) for the 80/20 pair, 59% for the 70/30 pair, and 49% for the 60/40 pair, or once they completed 6 practice blocks (based on Frank et al., 2004). In the testing phase, participants were shown all faces from the training phase, but the faces were recombined in all possible pairs, including those shown (e.g., 80/20) and not shown (e.g., 80/30, 80/40, 80/60, and 80/70) during the training phase, and no feedback was given (i.e., no happy or angry faces follow the selection). Participants were instructed to choose the more rewarding face from each pair based on the probabilities learned during training. The testing phase consisted of a total of 120 trials, with each pair of faces shown eight times (four with the more rewarding face on the left, four with the more rewarding face on the right).

Figure 1. Social Probabilistic Selection Task.

Note. The Social Probabilistic Selection Task consisted of two phases: training and testing. In the training phase, neutral face stimuli were presented in pairs with varying probabilities of reward (becoming a happy face) or punishment (becoming an angry face) when chosen. In the example above, stimuli are referred to by their reward probabilities, and their punishment probabilities are 100-p(reward) (e.g., stimulus 80 would reward the participant 80% of the times chosen and punish the participant 20% of the times chosen). The instructions for the training phase explained, “Two faces will appear simultaneously on the computer screen. One person will become HAPPY when you choose them and the other will become ANGRY. At first you will not know which is which. You will learn through trial and error. Try to guess the HAPPY person as quickly and accurately as possible. In this task, no person is ALWAYS happy when you choose them, but some people have a higher chance of being happy than others. Try to pick the person you find to have the highest chance of becoming happy when you choose them.” During the testing phase, neutral faces were recombined into all the possible pairs, and participants were told to decide between them based on what they had learned in the training phase. No feedback was given during the testing phase. Instructions explained, “It is time to test what you have learned! During this set of trials you will NOT receive feedback (happy or angry) to your responses. If you see new combinations of faces in the test, please choose the person that seems more likely to become happy based on what you learned during the training sessions. If you are not sure which one to pick, just go with your gut instinct!” Although all possible face pairs were presented during the testing phase, only the pairs included in the calculation of reward and punishment learning accuracy are presented in this figure.

Based on research finding different effects of facial stimuli sex (Abraham & Hermann, 2015; Olsson et al., 2013) and race (Lindstrom et al., 2014) on social RL, White faces of men (for female participants) and women (for male participants) were used as stimuli for the present study to reduce variance due to this heterogeneity, though we recognize this was a tradeoff. Because the SPST was administered to high social anxiety participants twice as part of a larger data collection (at the session analyzed here and at a follow-up session not analyzed here), two sets of faces were used as stimuli (the NimStim and FACES sets; Ebner, Riediger, & Lindenberger, 2010; Tottenham et al., 2009), with their order counterbalanced for both low and high social anxiety participants. Both sets of faces included more models than were needed for this study, so we selected six women and six men from each set whose expressions of happiness, anger, and neutral emotion were well identified by previous samples (Ebner et al., 2010; Tottenham et al., 2009). From the NimStim set, which includes open- and closed-mouth emotional expressions, we selected open-mouthed happy and angry faces because previous samples identified their emotions more accurately (Tottenham et al., 2009). The faces were randomized to the six contingencies (80, 70, 60, 40, 30, and 20) for each participant. The SPST was presented in MATLAB.

Procedure

After obtaining informed consent, participants completed the SPST and questionnaires. This study was part of a larger data collection that included other behavioral tasks, EMA, and an intervention, which are analyzed elsewhere. The intervention and EMA portions of the study occurred after this study’s data were collected.

Plan for Analyses

We applied an RL model to the SPST to assess learning about static probabilities that others will be socially rewarding or punishing. The parameters analyzed include learning rates for rewarding and punishing social feedback, and accuracy of learning to choose rewarding faces and avoid punishing faces. We compared these parameters across social anxiety groups, as well as within the high social anxiety group.

Computational Model Selection and Performance

The SPST was computationally modeled using the hBayesDM package (Ahn et al., 2017) in R. This package offers hierarchical Bayesian modeling of the SPST using the Q-learning estimation procedure implemented in Frank et al. (2007). Hierarchical Bayesian modeling simultaneously estimates individual- and group-level parameters with a Markov chain Monte Carlo (MCMC) sampling scheme. Assuming that participants within each social anxiety group share some similarity while still estimating model parameters for each participant increases reliability over models fitted separately to each individual’s data (without consideration of group-level similarities) because data are often noisy for this task. Because the hBayesDM package implements Bayesian models, parameter estimates are provided as posterior distributions, rather than point estimates.

We compared two candidate Q-learning models. Q-learning models the process of participants updating their expectancies of what will happen when they choose each face by incorporating new feedback each time they choose that face and it becomes happy or angry. The weight they give to this new feedback is the learning rate. One candidate Q-learning model fitted trial-by-trial behavioral data from the training phase of the SPST with separate learning rate parameters for rewarding and punishing feedback (given findings that models with separate positive and negative learning rates tend to fit the data better; Gershman, 2015). This model could differentiate if participants learned faster from one type of feedback or the other. A Q value was computed for each stimulus i at trial t, and these Q values were updated by multiplying each prediction error by a learning rate α following the algorithm:

$$Q_i(t+1)=Q_i(t)+{\alpha }_R{[r(t)-Q_i(t)]}_{+}+{\alpha }_P{\left[r(t)-Q_i(t)\right]}_{-}$$

Where r(t) = 1 for rewarding (happy) and 0 for punishing (angry) feedback. The reward learning rate α_R was applied for positive prediction errors, when the outcome was better than expected, and the punishment learning rate α_P was applied for negative prediction errors, when the outcome was worse than expected. Another, simpler candidate model was also tested with a single learning rate α applied to both positive and negative prediction errors (modeling learning as a singular process, regardless of the type of feedback):

$$Q_i(t+1)=Q_i(t)+\alpha [r(t)-Q_i(t)]$$

Q values were entered into the following softmax equation with inverse temperature β to determine the probability of a participant selecting a given stimulus in each stimulus pair (e.g., A over B in the example below, but the same applies for CD and EF):

$$P_A(t)=\frac{e^{Q_A/\beta }}{e^{Q_A/\beta }+e^{Q_B/\beta }}$$

To ensure that MCMC chains were well mixed and converged to stationary distributions for stable parameter estimates, we checked whether $\widehat{R}$ values for parameters were approximately 1, produced and examined trace plots of the group-level parameters, and plotted and visually examined posterior distributions of the group- and individual-level parameters.

Primary Analyses

Learning Rates Between Social Anxiety Groups.

To assess whether learning rates for reward and punishment differed between social anxiety groups, a mixed effects model was performed predicting learning rate from fixed effects of social anxiety group and valence (reward vs. punishment), their interaction, and a random intercept for participant.

Reward and Punishment Learning Accuracy Between Social Anxiety Groups.

Participants who learn more (vs. less) from rewarding feedback should more reliably choose the most rewarding stimulus during the testing phase of the task, and participants who learn more from punishing feedback should more reliably avoid the most punishing stimulus during the testing phase. An index of reward learning accuracy (how well participants learn from happy faces) was calculated as the proportion of times the most rewarding stimulus (80) was chosen when paired with all stimuli other than the most punishing stimulus (70, 60, 40, and 30) during the testing phase of the SPST. Similarly, an index of punishment learning accuracy (how well participants learn from angry faces) was calculated as the proportion of times the most punishing stimulus (20) was avoided (not chosen) when paired with all stimuli other than the most rewarding stimulus (70, 60, 40, and 30) during the testing phase. The 80/20 pair was not used in this calculation because it would not be possible to disentangle whether participants were accurate because they were choosing the most rewarding face or avoiding the most punishing face. Because accuracy follows a binomial distribution, a generalized linear mixed model with a logit link function was performed, predicting accuracy from fixed effects of social anxiety group and choice type (pairs including 80 vs. pairs including 20), their interaction, and a random intercept for participant.

Secondary Analyses

Discriminability Between Social Anxiety Groups.

We performed a secondary analysis using data from the testing phase to determine whether social anxiety is characterized by differences in discriminating between more difficult (i.e., similarly rewarding/punishing) stimulus pairs. Specifically, this analysis tested whether decision difficulty and average reward/punishment magnitude affect decision accuracy differently for people high versus low in social anxiety symptoms. Here, we define difficulty as the absolute value of the difference in the probability of reward and punishment between the two stimuli in the pair (e.g., the 80/20 stimulus pair has difficulty 60 and is less difficult to discriminate than the 80/70 pair, which has difficulty 10; lower difficulty scores reflect pairs that are more difficult to discriminate). Magnitude refers to the average reward probability of the stimulus pair (e.g., the 80/70 pair has magnitude 75, which is more rewarding than the 80/20 pair, which has magnitude 50). This model was performed as a generalized linear mixed model with a logit link function predicting accuracy from fixed effects of social anxiety group, difficulty (centered), magnitude (centered), and all two- and three-way interactions, with a random intercept for participant. This secondary analysis had the benefit of using all the data from the testing phase (120 trials), whereas the reward and punishment learning accuracy analyses described above only used some of the data (64 trials).

Results

Computational Model Selection and Performance

Two candidate Q-learning models were fitted to the training phase data from the SPST, separately for groups low and high in social anxiety. We compared prediction accuracy of each model for each group’s data using LOOIC (leave-one-out information criterion) given our relatively small sample size (Vehtari et al., 2017). Because the more complex model had a lower LOOIC, it was selected for further analysis (see Table 2 for LOOIC values). From the more complex model, we visually inspected trace plots of the parameters for both groups, which indicated that MCMC samples were well mixed and converged to stationary values, consistent with $\widehat{R}$ values around one for all participants in both groups. The group-level parameters in both groups had unimodal posterior distributions. Visual inspection of the individual-level parameters in both groups showed relatively wide posterior distributions, indicating some uncertainty in the estimates at an individual level.

Table 2.

LOOIC of Candidate Models

	Low Social Anxiety	High Social Anxiety
Two Learning Rates	7,595.75	24,068.17
Single Learning Rate	7,601.66	24,130.49

Primary Analyses

Learning Rates Between Social Anxiety Groups

A mixed effects linear regression was performed predicting individual-level learning rate estimates from fixed effects of social anxiety group, prediction error valence, and their interaction, with a random intercept for participant. This model found no significant main effects or interactions (see Table 3).

Table 3.

Model Estimates Predicting Learning Rate by Social Anxiety Group

	Learning Rate
Predictors	Estimates	CI	t	p
(Intercept)	0.23	0.21 – 0.25	20.95	<0.001
SA Group	0.00	−0.02 – 0.03	0.45	0.652
PE Valence	−0.01	−0.03 – 0.00	−1.41	0.158
SA Group X PE Valence	0.00	−0.02 – 0.02	0.01	0.994
Random Effects
σ2	0.02
τ00 subjID	0.01
ICC	0.22
N subjID	156
Observations	312
Marginal R2 / Conditional R2	0.007 / 0.228

Note: SA Group = social anxiety group (high vs. low). PE Valence = prediction error valence (positive vs. negative).

Reward and Punishment Learning Accuracy Between Social Anxiety Groups

A generalized linear mixed model with a logit link function was performed predicting accuracy in the testing phase from fixed effects of social anxiety group, choice type, and their interaction, with a random intercept for participant. Significant main effects were found for both social anxiety group and choice type, but they are not interpreted here because they were subsumed within a significant interaction (see Figure 2; Table 4). Post-hoc pairwise comparisons of the estimated marginal means (with a Tukey adjustment) indicated that, contrary to hypotheses, the high social anxiety group’s punishment learning accuracy (M = 0.78, SE = 0.02) was significantly lower than their reward learning accuracy (M = 0.87, SE = 0.01, OR = 0.50, p < .001), the low social anxiety group’s punishment learning accuracy (M = 0.88, SE = 0.02, OR = 0.47, p = .003), and the low social anxiety group’s reward learning accuracy (M = 0.89, SE = 0.02, OR = 0.44, p = 0.001). No other pairwise comparisons were statistically significant. This result suggests that the low social anxiety group was similarly likely to choose the more rewarding face from pairs that included the most rewarding and most punishing faces. However, contrary to hypotheses, the high social anxiety group was less likely to avoid the most punishing face than to choose the most rewarding face. In other words, the high, but not low, social anxiety group showed less accuracy in learning to avoid punishing faces.

Figure 2. Reward and Punishment Learning Accuracy by Social Anxiety Group.

Note: Avoid Most Punishing Face = rate of choosing the more rewarding face on pairs that included the most punishing face. Choose Most Rewarding Face = rate of choosing the more rewarding face on pairs that included the most rewarding face.

Table 4.

Model Estimates Predicting Testing Phase Accuracy by Social Anxiety Group

	Accuracy
Predictors	Odds Ratios	CI	t	p
(Intercept)	6.07	4.93 – 7.46	17.04	<0.001
SA Group	0.80	0.65 – 0.98	−2.12	0.034
Choice Type	1.21	1.13 – 1.28	5.76	<0.001
SA Group * Choice Type	1.17	1.10 – 1.25	4.88	<0.001
Random Effects
σ2	3.29
τ00 id	1.14
ICC	0.26
N id	154
Observations	9780
Marginal R2 / Conditional R2	0.028 / 0.279

Note: SA Group = social anxiety group (high vs. low). Choice Type = trials including the most rewarding face (reward learning accuracy) vs. the most punishing face (punishment learning accuracy).

Secondary Analyses

Secondary analyses were performed to address questions of accuracy at varying degrees of difficulty and to incorporate more trials.

Discriminability Between Social Anxiety Groups

A generalized linear mixed model with a logit link function was performed predicting accuracy in the testing phase from fixed effects of social anxiety group, difficulty, magnitude, and all two- and three-way interactions, with a random intercept for participant. There were significant main effects of social anxiety group, difficulty, and magnitude, but these are not interpreted because they were subsumed within two significant interactions: social anxiety group X difficulty (Figure 3), and social anxiety group X magnitude (Figure 4). Post-hoc comparisons of the estimated linear trends for each social anxiety group indicated that the effect of difficulty on accuracy was significant and positive for each group, but this trend was stronger for the low social anxiety group. In other words, both groups were more likely to select the more rewarding face on less difficult pairs, and this effect was more pronounced for the low (b = 0.06, SE = 0.003) versus high social anxiety group (b = 0.04, SE = 0.002, p < .001). The effect of magnitude on accuracy was significant and positive within the high social anxiety group (b = 0.02, SE = 0.003), but non-significant in the low social anxiety group (b = − 0.004, SE = 0.006), and the effects in each of these groups were significantly different from each other (p < .001). In other words, the high social anxiety group was more likely to select the more rewarding face on more rewarding (vs. more punishing) pairs, but the relative magnitude of the pair did not affect accuracy for the low social anxiety group (see Table 5).

Figure 3. Accuracy as a Function of Difficulty by Social Anxiety Group.

Note: Difficulty = difference between reward probabilities of faces in a pair, such that higher scores reflect easier pairs (note that this variable is centered). Accuracy = rate of choosing the more rewarding face.

Figure 4. Accuracy as a Function of Magnitude by Social Anxiety Group.

Note: Magnitude = mean reward probability of faces in a pair, such that higher scores reflect more rewarding pairs (note that this variable is centered). Accuracy = rate of choosing the more rewarding face.

Table 5.

Discriminability Model Estimates Predicting Testing Phase Accuracy by Social Anxiety Group

	Accuracy
Predictors	Odds Ratios	CI	t	p
(Intercept)	5.26	4.54 – 6.10	21.97	<0.001
SA Group	0.79	0.68 – 0.92	−3.05	0.002
Difficulty	1.05	1.05 – 1.06	29.10	<0.001
Magnitude	1.01	1.00 – 1.01	2.28	0.023
SA Group X Difficulty	0.99	0.99 – 1.00	−4.74	<0.001
SA Group X Magnitude	1.01	1.00 – 1.02	3.37	0.001
Difficulty X Magnitude	1.00	1.00 – 1.00	−0.44	0.661
SA Group X Difficulty X Magnitude	1.00	1.00 – 1.00	1.14	0.255
Random Effects
σ2	3.29
τ00 id	0.59
ICC	0.15
N id	154
Observations	18316
Marginal R2 / Conditional R2	0.146 / 0.277

Note: SA Group = social anxiety group (high vs. low). Difficulty = difference between reward probabilities of the two faces in a pair. Magnitude = mean reward probability of the two faces in a pair.

Exploratory Analyses

To probe our surprising results more deeply, we performed several exploratory analyses assessing possible explanations for the null learning rate effect and lower punishment learning accuracy for those high in social anxiety. The possible explanations examined included that the high social anxiety group needed more training to learn the probabilities in the training phase, that they learned some pairs’ probabilities more accurately than others, or that participants might have grouped several faces together based on reward/punishment thresholds. However, based on our exploratory analyses, these factors likely do not explain the observed effects (see Supplement for more detail).

To account for possible effects of race on RL for in-group versus out-group faces, all primary and secondary analyses were also performed covarying for participants’ self-reported race (White vs. all other races, as our stimuli were all White faces). There were no statistically significant effects of race in any models. Model estimates remained similar, and the pattern of significance did not change for any models.

Discussion

This study used a social probabilistic selection task to assess how individuals high vs. low in social anxiety symptoms learn from social reward and punishment. We found no significant differences in reward and punishment learning rates based on social anxiety group. This null result was consistent with prior work suggesting that anxiety-related differences in learning rate might be more related to differences adapting to environmental volatility rather than learning differences in a stable environment. Contrary to hypotheses, participants high (vs. low) in social anxiety were less accurate at learning to avoid socially punishing stimuli than learning to choose socially rewarding stimuli. This result was evidenced in several ways. The high social anxiety group had lower accuracy at avoiding the most punishing face than at choosing the most rewarding face. Their accuracy at choosing the more rewarding face from a pair increased as the average reward value (i.e., magnitude) of the pair increased. And, the exploratory threshold model analyses (described in the Supplement) suggested they had greater discriminability among rewarding versus punishing faces. None of these effects were observed in the low social anxiety group.

Impaired Punishment Learning Accuracy in Socially Anxious Individuals

The finding that people with high social anxiety show impaired punishment learning accuracy runs counter to past studies finding greater accuracy in learning from punishment versus reward in participants with elevated social anxiety (Abraham & Hermann, 2015) and generalized anxiety (Cavanagh et al., 2019; LaFreniere & Newman, 2019). However, this result is theoretically consistent with a study by Lamba and colleagues (2020), which found that participants high (vs. low) in generalized anxiety symptoms tended to learn less from negative social outcomes, and consequently, overinvested in exploitative social partners. Given the mixed results for accuracy across studies using slightly different populations and behavioral tasks (e.g., learning from social vs. non-social stimuli, learning in static vs. volatile environments), the current finding bears replication, but the present results add to a literature that suggests that anxious individuals might actually show impaired learning from social punishment.

There are several intriguing potential explanations for this surprising effect. We examined whether the high (vs. low) social anxiety group might have needed more training to learn the probabilities or whether they learned some pairs’ probabilities more accurately than others, but found no evidence to support these explanations. Next, we considered whether high social anxiety participants might show a threshold effect specific to social punishment: that they respond to all faces that are more likely to become angry than happy (i.e., all faces above a certain punishment threshold) as “bad,” while still discriminating among the more rewarding faces by degree. Our exploratory findings were mixed with regards to whether they supported this explanation; however, given that accuracy increased linearly with the average reward value of a face pair in the high social anxiety group, this suggested that a continuous rather than threshold effect may be underlying this process (see Supplement).

Another potential explanation is that the high social anxiety group might have perceived the neutral face stimuli as more aversive than the low social anxiety group did, in line with facilitated avoidance learning for neutral faces among people high in social anxiety (Stevens et al., 2014). If so, the perceived magnitude of social punishment might be less than that of social reward for the high (vs. low) social anxiety participants, which might explain their impaired social punishment learning observed here. This potential explanation was not tested here, but could be examined empirically in a future study.

Intact Social Reward Learning in Socially Anxious Individuals

Interestingly, we found evidence of intact social reward learning in our high social anxiety group, which is surprising given prior work on social anhedonia in social anxiety (e.g., Blay et al., 2021; Richey et al., 2019). It is possible that we did not find evidence of a bias towards learning more from social punishment than reward in our high social anxiety group because we used an analogue, rather than clinically diagnosed, sample. However, it is not obvious why this would be the case (and there was no evidence of social anxiety severity effects based on the secondary analyses that considered social anxiety symptom severity dimensionally within the high social anxiety group; see Supplement).

It is also possible that social reward might be quite motivating for socially anxious individuals, especially earlier in development or in novel social contexts. Richey and colleagues’ sensitivity shift theory (2019) proposes that behaviorally inhibited children, who are more sensitive to both reward and punishment, go on to develop social anhedonia later in development if they experience adverse social contexts in late childhood and early adolescence (e.g., peer victimization, low parental warmth). Our sample consisted largely of undergraduate students who are older than this developmental stage, but it remains an interesting (though speculative) possibility that the immersive, novel, social context of college, with its many opportunities to meet new people and experience different social situations and outcomes, might be different enough from their prior contexts to allow for new learning outside of the negative social learning that might have occurred in high school and earlier.

Clinical Implications

The results of this study suggest that, although learning rates were similar across social anxiety groups, participants high (vs. low) in social anxiety were less accurate at learning to avoid social punishment. While we hypothesized the opposite, this surprising effect could be one factor fueling the social avoidance behavior seen in social anxiety. In particular, this study found that socially anxious individuals were worse at modulating their behavior to avoid social rejection (operationalized by lower accuracy at avoiding the most punishing face). In daily life, this may translate to more (vs. less) socially anxious individuals choosing to continue interacting with individuals who give them negative feedback when less socially anxious people might disengage. Given socially anxious individuals are particularly concerned about negative social evaluation, they might both take more extreme steps to avoid social situations and close interactions (e.g., not going to a party, self-disclosing very little with acquaintances, avoiding eye contact) and make poorer social choices when in social situations (e.g., choosing to engage with a more critical social partner). However, these actions that are designed to avoid rejection may, counterproductively, elicit negative reactions from others or prevent socially anxious individuals from building closer relationships, serving as a self-fulfilling prophecy for poorer quality and fewer social interactions, and reinforcing the maladaptive beliefs and expectancies that contributed to those negative interpersonal patterns. This is consistent with the idea of a self-perpetuating interpersonal stress generation cycle in social anxiety described by Alden and Taylor (2004) and subsequently expanded on by others (Farmer & Kashdan, 2015; Goodman et al., in prep.; Siegel et al., 2018). As such, clinicians may want to intervene on maladaptive social behaviors and/or cognitions to break this negative cycle. When selecting negative automatic thoughts to challenge, though, clinicians should consider that some negative expectancies of others may not be wholly unsupported, given socially anxious individuals’ potential difficulty learning to avoid negative social interactions.

Limitations

While the social probabilistic selection task has strengths (e.g., both the stimuli and outcomes are social, which allows us to cleanly model social RL, and faces have very high relevance to humans), its simplicity also limits its ecological validity and affective salience. First, the stimuli were static images, which lack features important to real social interactions (e.g., movement, sound). Future studies might consider adding more dynamic stimuli, like short videos of the neutral face becoming a happy or angry face, given prior findings that dynamic, versus static, social cues elicit higher arousal ratings (Sato & Yoshikawa, 2007) and more reactivity in brain regions involved in emotion processing (Pelphrey et al., 2007; Trautmann et al., 2009). Further, increasing the importance or personal relevance of the task to participants (e.g., by using faces or names of people in participants’ social networks or connecting the outcomes to real-world social reward and rejection) might increase the likelihood of finding meaningful effects (Radomsky & Rachman, 2004).

Additionally, our sample was largely comprised of non-Hispanic, White, undergraduate students, whose social context and developmental stage may meaningfully differentiate them from other populations. For instance, individuals holding marginalized racial and ethnic identities may learn differently from social reward and punishment coming from in-group versus out-group members given experiences of discrimination. Further, given prior research finding sex- and race-related effects on social RL, we opted to use White faces for our stimuli, with faces of men shown to female participants and faces of women shown to male participants, but this limits our understanding of social RL across the range of possible sex/gender, racial, and ethnic identities.

Finally, an a priori power analysis was not performed to determine sample size for these specific analyses, as sample size was determined for the primary aim of the parent data collection, which focused on an intervention that occurred after the collection of all data analyzed here. As such, we cannot quantify how well powered the current study was to find effects of interest. However, our sample size was comparable or larger than prior studies examining similar effects (e.g., Abraham & Hermann, 2015, had 62 participants, and Voegler et al., 2019, had 64 participants, both fewer than half the participants in the current study).

Conclusion and Future Directions

Individuals high in social anxiety symptoms showed less accurate, rather than enhanced, social punishment learning, and relatively intact social reward learning. Future studies that use more dynamic stimuli (e.g., video clips), socially anxious samples at different developmental stages (e.g., older adults), and diagnosed samples with more severe social avoidance, may provide additional insight into how social reinforcement learning relates to social anxiety and avoidance.

Highlights.

• Participants completed a social probabilistic selection task with emotional faces.

• Social reinforcement learning was modeled with Q-learning.

• Learning rates were not different for participants high vs. low in social anxiety.

• High (vs. low) social anxiety Ps were less accurate at avoiding angry faces.

• This learning bias may contribute to maintenance of negative social beliefs.