Abstract
Background:
Alterations in positive valence systems and social processes, including low reward responsiveness and high rejection sensitivity, have been observed in depression. Most reward research focuses on the monetary domain, but social reward responsiveness may be particularly relevant to understanding the etiology of depression, particularly in combination with other social processes. Pathways to depression are complex, and research testing interactions between multiple factors is needed. The present study examined the interactive effects of reward responsiveness and rejection sensitivity on depressive symptoms using both social and monetary reward electroencephalogram (EEG) tasks.
Methods:
Emerging adults (N = 120) completed peer interaction and monetary incentive delay tasks while EEG data were recorded, as well as self-report measures of rejection sensitivity and depressive symptoms.
Results:
The interaction between social reward responsiveness and self-reported rejection sensitivity was significantly associated with depressive symptoms, such that rejection sensitivity was associated with greater depressive symptoms for those with a relatively reduced response to social reward. The interaction between monetary reward responsiveness and rejection sensitivity was not significant.
Limitations:
The study was cross-sectional and used a non-clinical sample.
Conclusions:
Results suggest a possible pathway for depressive symptoms characterized by the combination of high rejection sensitivity and low social reward responsiveness. Findings highlight the need for consideration of multiple domains of reward responsiveness in clinical neuroscience research. With extension to longitudinal studies and clinical samples, the present findings may inform understanding of targets for intervention.
Keywords: reward, event-related potentials, neurophysiology, depression, rejection sensitivity
Introduction
The Research Domain Criteria initiative calls for the study of specific domains of behavior across levels of analysis, with psychopathology characterized as deviations from a typical range of functioning (Cuthbert, 2014; National Institute of Mental Health, 2020). Alterations in both the positive valence systems and social processes have been associated with depressive symptoms and risk (Keren et al., 2018; Kujawa & Burkhouse, 2017; Kupferberg et al., 2016). Positive valence systems refer to those involved in motivation and adjusting behavior to obtain rewards and include the constructs of reward responsiveness, reward learning, and reward valuation. Social processes refer to systems that drive responses during interpersonal interactions, including the constructs of affiliation and attachment, social communication, perception and understanding of self, and perception and understanding of others.
There has been growing interest in examining the intersection between reward responsiveness and social processes. One promising neurophysiological measure of individual differences in these processes is the reward positivity (RewP), also known as the feedback negativity. This event-related potential (ERP) derived from the electroencephalogram (EEG) is a relative positivity in the waveform that peaks about 300 ms following reward feedback over frontocentral sites (Bress et al., 2015; Kujawa et al., 2018; Proudfit, 2015). RewP has been associated with self-report measures of reward responsiveness and positive emotionality (Kujawa, Klein, et al., 2020) as well as activation of brain regions involved in reward processing, including the ventral striatum and medial prefrontal cortex (Becker et al., 2014; Carlson et al., 2011).
Much of the research on neurophysiological responses to reward in depression has been conducted using monetary reward tasks measuring RewP in response to feedback indicating a monetary gain. A reduced RewP in monetary reward tasks has been associated with depression and later increases in symptoms, suggesting it may reflect a vulnerability that makes some people more susceptible to depression (Bress et al., 2013; Kujawa & Burkhouse, 2017; Kujawa, Burkhouse, et al., 2019; Kujawa, Hajcak, et al., 2019; Nelson et al., 2016). Those with reduced neurophysiological response to reward may have lower motivation to engage in pleasant activities and/or experience less pleasure (Setterfield et al., 2016), which may lead to the later onset of depression, particularly in combination with stress (Goldstein et al., 2020). At the same time, associations between reward responsiveness and depression tend to be relatively weak (Kujawa & Burkhouse, 2017), which may be due in part to a lack of precision in monetary reward tasks in their ability to assess the core processes underlying depression. Additionally, some cross-sectional studies have not found a main effect of depressive symptoms on RewP in monetary reward tasks (Ait Oumeziane et al., 2019; Kujawa, Hajcak, et al., 2019; Novak et al., 2016). Compared to monetary reward, social reward responsiveness may be a stronger and more valid predictor of social behavior and depression (Davey et al., 2008; Forbes & Dahl, 2012; Silk et al., 2012). Consistent with this possibility, exposure to interpersonal stress is a particularly strong risk factor for depression (Hammen, 2005; Henry et al., 2019; Kessler et al., 2010). Further, low social reward responsiveness, assessed by RewP, has been shown to moderate effects of interpersonal stress, specifically, on depression (Pegg et al., 2019). Yet, little work has directly tested associations of both social and monetary reward responsiveness with depression.
Another known precursor to depression in the social processes domain is high sensitivity to rejection (Ayduk et al., 2001; Liu et al., 2014), which has also been conceptualized as a possible vulnerability associated with increases in depressive symptoms across time (De Rubeis et al., 2017). Individuals high in rejection sensitivity experience more negative emotions when faced with potential rejection (Downey & Feldman, 1996; Leng et al., 2018). Importantly, pathways to depression are complex and characterized by multiple interactive factors. Both low reward responsiveness and high rejection sensitivity appear to be key vulnerabilities for the emergence of depression, but they are typically examined in separate literatures. The combination of these factors may pose a greater risk for the development of depressive symptoms than each factor alone. Individuals who tend to experience high negative emotions in social contexts where rejection is possible and reduced responsiveness to positive reinforcement, particularly in the social domain, may be less likely to find enjoyment in and motivation to seek out social activities, potentially creating a pathway to the development of depressive symptoms. Moving beyond a focus on a single RDoC domain and measure, research is needed examining the combined effects of multiple processes and potential vulnerabilities on depressive symptoms.
Although previous work examining RewP and depression has been conducted primarily using monetary reward tasks (Kujawa & Burkhouse, 2017; Proudfit, 2015), there is evidence that similar neural responses can be elicited to social reward feedback in peer interaction tasks (Crowley et al., 2010; Ethridge et al., 2017; Kujawa et al., 2014; Sun & Yu, 2014). One study directly compared ERPs to social and monetary reward in emerging adults (Ethridge et al., 2017). A comparable RewP component emerged in response to reward feedback in both peer interaction and monetary reward tasks. At the same time, the magnitude of the difference between response to reward and nonreward conditions was larger for monetary RewP compared to social RewP, and responses to each type of reward were only modestly correlated, suggesting each indexes somewhat distinct reward-related processes (Ethridge et al., 2017). Taken together, extant literature suggests that social and monetary reward tasks may capture distinct individual differences in activation of positive valence systems and warrant further examination in depression research.
To extend research on reward responsiveness in depression, it is important to consider more complex pathways and interactions between multiple factors. These pathways may be particularly apparent when examining the social rather than the monetary reward domain, given its more direct relevance to the experience of emotion in interpersonal contexts, which have strong effects on depression risk (e.g., Hammen, 2005). Yet, to our knowledge, no prior studies have directly compared associations of social and monetary reward responsiveness with depression or their unique interactions with other established depression precursors, like rejection sensitivity.
In the present study, emerging adults (N = 120) completed two EEG tasks to assess neural responses to social (i.e., peer acceptance) and monetary reward feedback. Participants also completed self-report measures of rejection sensitivity and depressive symptoms. We first examined the associations of both social and monetary reward responsiveness, as measured by a reliable neural measure (i.e., RewP), with depressive symptoms. Next, we tested the interactive effects of reward responsiveness in each domain and rejection sensitivity on depressive symptoms. We hypothesized that low reward responsiveness in the peer interaction task would be most strongly associated with depressive symptoms. Based on evidence that social reward responsiveness moderates the effects of other interpersonal processes on depressive symptoms (Pegg et al., 2019), we predicted that the interaction between rejection sensitivity and social, but not monetary, reward responsiveness would also be associated with depressive symptoms, such that the combination of relatively low social reward responsiveness and high rejection sensitivity would be associated with greater depressive symptoms.
Methods
Participants
Undergraduate students (ages 18–22 years) were recruited via flyers and the psychology research participant pool at Vanderbilt University. Participants were compensated with research credit or $30, plus their earnings from the monetary reward delay task. Written informed consent was obtained from all participants prior to the start of study procedures. Following consent, participants completed questionnaires and a series of EEG tasks that were administered using a complete counterbalancing scheme (see Pegg & Kujawa, 2020 for our prior work on the monetary reward task). A total of 130 participants enrolled in the study, of which 5 were excluded for poor data quality on at least 11 electrodes across both tasks, 3 for poor EEG data quality for reference electrodes, and 1 for not completing either EEG task. One participant did not complete the rejection sensitivity measure. Thus, 120 participants were included in analyses. The mean age was 19.32 (SD = 1.15), 66.7% (n = 80) of the sample identified as female, and 10.8% (n = 13) identified as Hispanic or Latinx. In terms of race, participants identified as White (54.2%, n = 65), Asian (25.0%, n = 30), Black (10.8%, n = 13), or other or mixed race (10.0%, n = 12). On the peer interaction task, 1 additional participant was excluded for poor EEG data quality, 1 because they requested that their data not be used following debriefing, and 1 due to technical difficulties during data collection, resulting in 117 participants with viable data on this task. On the monetary reward task, 3 participants did not complete the task, 3 were excluded for failure to follow task instructions, and 1 was excluded for poor EEG data quality, resulting in 113 participants with viable data on this task. Study procedures were approved by the Vanderbilt University Institutional Review Board.
Measures
Island Getaway peer interaction task.
Participants completed a modified version of the Island Getaway peer interaction task while EEG data were collected as a measure of social reward responsiveness (Kujawa et al., 2014; Figure 1A). The premise of the task is that participants virtually “traveled” to the Hawaiian Islands with a group of 13 computerized coplayers they were led to believe were other college students playing the game at universities across the United States. Participants had their photograph taken as part of a profile they built for themselves by answering several questions, including their name, age, university, and general interests. They then read the profiles of the coplayers. Participants were instructed that they would vote on each player over several rounds and then receive feedback on how each player voted for them. The goal of the game was to make it to the final island without being voted off by their coplayers. During each round, participants were presented with the profile of each coplayer remaining in the game and decided to vote to accept (i.e., “Keep”) or reject (i.e., “Kick out”) them while that player simultaneously voted to accept or reject the participant. Participants had 5000 ms to vote. This was followed by a fixation cross for 2000 ms. The participant was then given feedback about how that coplayer voted for them for 2000 ms. If the coplayer voted to keep them, they received a green thumbs up. If the coplayer voted to kick them out, they received a red thumbs down. There was also a third form of feedback (i.e., a yellow rectangle) that indicated no vote was received for the participant possibly due to a network error. Feedback presentation was followed by a fixation cross for 1500 ms before the start of the next trial. Participants were told at the end of each round that a coplayer was kicked out of the game for having the most “kick” votes. The task was programmed so the participant always made it to the final island after a total of 6 rounds. Across the task, participants received equal (i.e., 21 trials each) acceptance, rejection, and no-vote feedback for a total of 63 trials. Additionally, the task was programmed such that there were an equal number of male and female players, including the participant. The number of male and female players that were “kicked out” of the game and that remained in the game until the end were also equal. Following completion of the study, participants were debriefed and given the option to opt out of including their data from the task in analyses.
Monetary Incentive Delay (MID) task.
Participants completed an ERP version of the MID task (Novak & Foti, 2015; Figure 1B). On each trial, participants first saw a cue for 500 ms that indicated whether the trial would be a monetary incentive trial (i.e., a blue dollar sign in a circle) or a non-incentive trial (i.e., a white outline of a circle). A fixation cross was then presented for 2000–2500 ms. A target (i.e., a white square) was then presented, which participants were instructed to respond to by clicking the left mouse button. This was followed by another fixation cross for a total of 1500 ms from target onset to feedback onset. On incentive trials, if the participant responded within the target window, they received a monetary reward of $0.40 and saw a green up arrow. If they did not respond within the target window, they received a monetary loss of $0.20 and saw a red down arrow. On non-incentive trials, participants did not win or lose money and were presented with a yellow line regardless of reaction time. The target was initially presented for 200 ms and presentation time decreased by 10 ms if the participant was successful on the previous trial and increased by 10 ms if the participant was unsuccessful. Task difficulty was adjusted such that participants won about 50% of the trials. Feedback was presented for 2000 ms. This was followed by a fixation cross for 1000 ms prior to the start of the next trial. Participants completed 70 total trials, including 50 incentive and 20 non-incentive trials. The difference in win versus loss amounts allowed participants to earn money, and participants were paid their total earnings. The inclusion of non-incentive trials in this task allows for the differentiation of ERP components between non-incentive versus potential win feedback during the anticipation of feedback stage of reward processing.
Rejection sensitivity.
Participants completed the Rejection Sensitivity Questionnaire – Adult version (ARSQ; Berenson et al., 2009). The ARSQ consists of 9 hypothetical interpersonal situations in which rejection is possible (e.g., “You call a friend when there is something on your mind that you feel you really need to talk about”). Participants respond to two questions that assess how concerned or anxious they would feel if they were rejected in this situation and how much they would expect to be accepted in this situation. For each hypothetical situation, a rejection sensitivity score is calculated by multiplying the level of rejection concern by the reverse score of acceptance expectancy. Then an average is calculated across all nine situations for an overall score, with higher scores indicating higher rejection sensitivity. ARSQ scores have been correlated with other indicators of interpersonal sensitivity and internalizing symptoms (Berenson et al., 2009). Scores on the ARSQ had good internal consistency in the present sample (Cronbach’s α = .80).
Depressive symptoms.
Participants completed the Inventory of Depression and Anxiety Symptoms (IDAS), a 64-item, validated measure of recent (i.e., past two weeks) depressive and anxiety symptoms (Watson et al., 2007). There are two broad scales of depressive symptoms, the general depression and dysphoria scales. The dysphoria scale focuses on the emotional and cognitive symptoms of depression, whereas the general depression scale consists of items assessing a larger range of depressive symptoms, such as fatigue and suicidality, and more closely corresponds to traditional measures of depression, such as the Beck Depression Inventory (Beck et al., 1996). To encompass a broader range of depressive symptoms, the general depression scale was used as a measure of depression in the present study. The general depression scale has shown strong convergent validity and good test-retest reliability (Watson et al., 2007). In the current sample, scores on the IDAS general depression scale ranged from 23 to 86 and had excellent internal consistency (Cronbach’s α = .90). Additionally, 15.0% (n = 18) of participants met the IDAS clinical cutoff for major depressive disorder (Stasik-O’Brien et al., 2019).
EEG Data Collection and Processing
Continuous EEG data were collected using a 64-electrode BrainProducts actiCHamp system (Munich, Germany). To measure electrooculogram, facial electrodes were attached 1 cm above and below the right eye and 1 cm on each outer corner of the eyes. Online data acquisition was referenced to Cz with a sampling rate of 1000 Hz and impedances below 30 kΩ. Data were processed using BrainVision Analyzer (BrainProducts, Munich, Germany). A band-pass filter with cutoffs of 0.1 and 30 Hz was used. Data were re-referenced offline to linked mastoids TP9 and TP10. Continuous EEG data were segmented −200 ms before to 1000 ms after feedback. Ocular correction was conducted using Gratton’s algorithm (Gratton et al., 1983). Semiautomatic artifact rejection was conducted with the following criteria: a voltage step greater than 50 μV/ms between sample points, maximum voltage difference of 175 μV within trials, a minimal allowed amplitude of −200 μV and maximal allowed amplitude of 200 μV, and minimum voltage difference of 0.5 μV within 100 ms intervals. Data were then visually inspected to remove remaining artifacts (<1% of data on average). Faulty recordings at single electrodes were interpolated using the signal from surrounding electrodes. For 5 participants with poor data at a mastoid electrode (TP9/TP10), data were interpolated at one or both mastoids in at least one of the tasks prior to mastoid re-reference.1 Data were averaged by type of feedback (win/acceptance or loss/rejection) and baseline corrected −200 to 0 before feedback onset.
The current version of Island Getaway included a novel no feedback condition, but because there is not a comparable unexpected neutral condition in the MID task and accept/win versus reject/loss conditions are commonly compared in the ERP literature (e.g., Bress & Hajcak, 2013; Kujawa et al., 2014), our analyses focused on accept/win and reject/loss conditions. For Island Getaway, participants had on average 20.92 (SD = 0.30, minimum = 19) trials for the accept condition and 20.87 (SD = 0.45, minimum = 18) trials for the rejection condition at Cz following artifact rejection. For MID, participants had on average 25.72 (SD = 1.59, minimum = 20) trials for the win condition and 24.06 (SD = 1.58, minimum = 19) trials for the loss condition at Cz following artifact rejection. RewP was scored as the mean amplitude 275–325 ms after feedback at Cz, which reflects the time window in which RewP was maximal for both tasks (Figure 2) and is generally consistent with prior work on the RewP (Babinski et al., 2019; Kujawa, Arfer, et al., 2020; Novak et al., 2016; Pegg et al., 2019; Rappaport et al., 2019). RewP was scored in the same time window and at Cz on both tasks based on visual inspection of the grand average data and to be consistent with prior monetary reward literature on RewP (e.g., Ethridge & Weinberg, 2018; Rappaport et al., 2019). This scoring approach was further supported by the results of principal component analyses described in Supplementary Material. Split-half reliability of ERPs were good for Island Getaway social reward and social rejection feedback, as well as for MID monetary reward and monetary loss feedback; Spearman-Brown coefficients: .88, .84, .91, and .88, respectively.
Data Analysis
We first examined whether there was a significant difference between RewP to reward and RewP to social rejection/monetary loss conditions in each task by conducting a 2 (valence: reward vs. rejection/loss) x 2 (task: Island Getaway vs. MID) repeated-measures ANOVA. Next, bivariate correlations were conducted between rejection sensitivity, depressive symptoms, individual RewP to reward and rejection/loss conditions, and RewP residuals. We calculated the social and monetary residual scores by saving the unstandardized residuals in linear regression models with response to social rejection/monetary loss as the predictor and response to reward as the outcome variable. Given that overall magnitude of ERPs in a single condition can be influenced by a range of physiological and cognitive processes, residual scores have been proposed as an alternative method to subtraction-based difference scores to isolate the variance in the ERP wave associated with the underlying process of interest (Meyer et al., 2017). Individual conditions (i.e., RewP to reward and rejection/loss) were included in addition to the residual scores in the correlation analyses to be able to test the associations with each condition as well as the reward condition adjusting for the rejection/loss condition as measured by the residual scores.
To examine the interaction between reward responsiveness and rejection sensitivity on depressive symptoms, a hierarchical regression analysis was conducted in which RewP to social/monetary reward, RewP to social rejection/monetary loss, and rejection sensitivity were entered into step 1 and the two interaction effects between RewP to social/monetary reward and rejection sensitivity were entered into step 2. RewP to both social and monetary reward were entered into the same model to assess whether there were unique associations for each domain. By including RewP to both conditions in the model, we are able to examine the unique effects of ERPs in each condition, partialing out the variance accounted for by the other condition, without first computing residual scores. The regression was conducted in two steps to examine the unique main effects of each predictor on depressive symptoms in step 1 prior to the addition of the interaction terms in step 2. To probe significant interactions, we examined simple slopes (1 SD above and below the mean and at the mean) as well as the Johnson-Neyman test to identify regions of significance that represent the range of values on the moderating variable where the association between the independent and dependent variables is significantly different from 0 at an alpha level of .05 (Johnson & Neyman, 1936) using PROCESS v3.4 macro for SPSS (Hayes, 2017). We estimated that we would have power to detect small to medium effect sizes with this sample size and regression model, f2 = .14 (Cohen, 1988; Faul et al., 2009).
Lastly, we conducted exploratory analyses to examine whether interactive effects were specific to positive feedback condition and to test whether results were consistent when controlling for gender, which are reported in the Supplementary Material.
Results
Results of a 2 (valence: RewP to reward vs. RewP to social rejection/monetary loss) x 2 (task: Island Getaway x MID) repeated-measures ANOVA revealed that there was a significant main effect of valence, F(1, 109) = 12.31, p < .001, ηp = .10, such that RewP to reward feedback (M = 9.98, SD = 7.62) was significantly more positive compared to RewP to social rejection feedback (M = 8.88, SD = 7.12) in the Island Getaway task, F(1, 116) = 6.00, p = .016, ηp = .05. Similarly, RewP to monetary win feedback (M = 15.20, SD = 8.15) was significantly more positive compared to RewP to monetary loss feedback (M = 13.64, SD = 7.92) in the MID task, F(1, 112) = 10.96, p = .001, ηp = .09.
Bivariate correlations are presented in Table 1. RewP magnitudes in each condition were positively correlated with one another, although correlations were relatively stronger within each task. As expected, depressive symptoms were moderately, positively correlated with rejection sensitivity. Interestingly, rejection sensitivity was negatively associated with both social and monetary RewP residuals. Depressive symptoms were not significantly correlated with RewP measures, including the social (p = .068) and monetary RewP residuals (p = .550).
Table 1.
Variables | M (SD) | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
---|---|---|---|---|---|---|---|---|
1. RewP to social reward | 9.98 (7.62) | -- | ||||||
2. RewP to social rejection | 8.88 (7.12) | .79*** | -- | |||||
3. Social RewP residual | 0.00 (4.71) | .62*** | .00 | -- | ||||
4. RewP to monetary reward | 15.20 (8.15) | .62*** | .61*** | .23* | -- | |||
5. RewP to monetary loss | 13.64 (7.92) | .54*** | .63*** | .09 | .81*** | -- | ||
6. Monetary RewP residual | 0.00 (4.83) | .31** | .18 | .27** | .59*** | .00 | -- | |
7. Rejection sensitivity | 10.13 (4.52) | −.06 | .07 | −.18* | .004 | .20* | −.27** | -- |
8. Depressive symptoms | 42.76 (12.27) | .01 | .15 | −.17 | .03 | .08 | −.06 | .42*** |
Note:
p<.05
p<.01
p<.001
RewP = reward positivity
Next, a hierarchical regression analysis was conducted to test whether interactions between RewP to social/monetary reward and rejection sensitivity were associated with depressive symptoms (see Table 2). In step 1, there were significant main effects of rejection sensitivity and RewP to social rejection on depressive symptoms. In step 2, as predicted, only the interaction between RewP to social reward and rejection sensitivity was significant, t(102) = −2.51, p = .014. The association between rejection sensitivity and depressive symptoms was significant at 1 SD below the mean (b = 1.60, SE = 0.33, t(112) = 4.84, p < .001), at the mean (b = 1.16, SE = 0.24, t(112) = 4.93, p < .001), and 1 SD above the mean (b = 0.72, SE = 0.31, t(112) = 2.32, p = .022) of RewP to social reward, with the magnitude of the association between rejection sensitivity and depressive symptoms increasing as RewP to social reward decreased. More specifically, the association between rejection sensitivity and depressive symptoms was significant at or below 18.70 on RewP to social reward based on the Johnson-Neyman test (RewP scores ranged from −3.28 to 30.33). For illustrative purposes and to depict the range of data, we grouped participants into high (top 1/3) and low (bottom 1/3) values of RewP to social reward and plotted the association between rejection sensitivity and depressive symptoms (see Figure 3). Further probing this interaction with rejection sensitivity as the moderator revealed that the association between a reduced RewP to social reward and depressive symptoms was trending significant for those at 1 SD above the mean (b = −0.48, SE = 0.25, t(112) = −1.93, p = .057), but not for those 1 SD below the mean (b = 0.04, SE = 0.26, t(112) = 0.14, p = .891) or at the mean (b = −0.22, SE = 0.22, t(112) = −1.00, p = .321), of rejection sensitivity. More specifically, the association between RewP to social reward and depressive symptoms was significant at or above 14.84 on rejection sensitivity based on the Johnson-Neyman test (rejection sensitivity scores ranged from 2.11 to 25.44).2 These results were consistent across scoring approaches and appeared specific to the RewP component, rather than the later but overlapping P3 (see Supplementary Material).
Table 2.
Predictor | b (SE) | β | p | rp |
---|---|---|---|---|
Step 1 | ||||
Rejection sensitivity | 0.93 (0.26) | 0.35 | <.001 | .33 |
RewP to social reward | −0.29 (0.22) | −0.20 | .195 | −.13 |
RewP to social rejection | 0.50 (0.25) | 0.32 | .046 | .19 |
RewP to monetary reward | 0.12 (0.23) | 0.09 | .607 | .05 |
RewP to monetary loss | −0.18 (0.24) | −0.12 | .457 | −.07 |
Step 2 | ||||
RewP to social reward X rejection sensitivity | −0.10 (0.04) | −0.83 | .014 | −.24 |
RewP to monetary reward X rejection sensitivity | 0.05 (0.04) | 0.46 | .252 | .11 |
Note: RewP = reward positivity; rp = partial correlations
Discussion
In the present study, we examined associations between social and monetary reward responsiveness, rejection sensitivity, and depressive symptoms in emerging adults. Although main effects of RewP on depressive symptoms were not significant for either the social or monetary reward domains, the interaction between social reward responsiveness and rejection sensitivity was significantly associated with depressive symptoms. That is, higher rejection sensitivity was more strongly associated with depressive symptoms for those who also exhibited relatively low neural response to social reward. Importantly, regression analyses indicated that these associations were specific to reward responsiveness in the social, but not monetary, reward domain.
This study is among the first to directly examine both social and monetary reward responsiveness in depressive symptoms and to do so using a neural measure that is robustly and reliably elicited in response to both types of feedback. Consistent with prior work, these results indicate that bivariate, cross-sectional associations between depressive symptoms and RewP tend to be modest in magnitude and are not always apparent with monetary reward tasks (Keren et al., 2018). While prior work has found associations between a reduced monetary RewP and depression and the later onset of depressive symptoms (Bress et al., 2013; Kujawa, Burkhouse, et al., 2019; Nelson et al., 2016), cross-sectional associations between monetary RewP and depressive symptoms have not always been found, as in the present study. For example, a significant longitudinal, but not cross-sectional, association was observed between monetary RewP and depressive symptoms in a sample of youth using a doors guessing game monetary reward task (Kujawa, Hajcak, et al., 2019). In work using the ERP version of the MID task, previous studies have also not found significant associations between neural response to reward and depressive symptoms cross-sectionally in both adult (Novak et al., 2016) and adolescent samples (Landes et al., 2018). Similarly, a study examining social and monetary RewP using the social incentive delay and MID tasks did not find a significant association between either social or monetary RewP and depressive symptoms (Ait Oumeziane et al., 2019). Additionally, a recent meta-analysis indicated that the association between RewP and depression was only significant in youth, but not adult, depression (Keren et al., 2018). Taken together, although previous research suggests that low reward responsiveness as measured by RewP does appear to be linked to the emergence of depression, other work and the present findings reveal that cross-sectional main effects are not always observed, particularly with the MID task, and highlight the need for a more nuanced examination of reward responsiveness in depression, including changes across development and moderating effects.
Young adults with relatively reduced social reward responsiveness may experience low motivation for and/or limited pleasure in social interactions (Setterfield et al., 2016). If these individuals are also highly sensitive to potential rejection and more likely to expect, interpret, and react to rejection (Downey & Feldman, 1996; Leng et al., 2018; Romero-Canyas et al., 2010), they may be further disinclined to participate and find enjoyment in social activities, possibly leading to the development of depression. An alternative explanation for the current findings is that high social reward responsiveness could buffer against the potential negative impacts of rejection sensitivity on depression. Individuals high in both rejection sensitivity and social reward responsiveness may be prone to experiencing negative emotions during potential rejection, but also find more enjoyment in their interactions, potentially protecting against the development of depressive symptoms. Additional work is needed to examine the causality and direction of the present findings.
The current results highlight the potential for further work on reward responsiveness in the social domain to advance understanding of the development and treatment of affective disorders. Previous work using monetary reward paradigms has examined low reward responsiveness in the development of depression across adolescence (e.g., Kujawa & Burkhouse, 2017; Kujawa, Hajcak, et al., 2019; Nelson et al., 2016). Reward systems in the brain are thought to undergo developmental transitions during adolescence (Braams et al., 2015; Galvan, 2010), a time where depression rates increase (Thapar et al., 2012) and peer relationships and social acceptance increase in salience (Allen et al., 2005). Results of this study and others (Ethridge et al., 2017; Pegg et al., 2019; Rappaport et al., 2019) provide evidence that tasks eliciting responses to other domains of reward, especially social, may assess processes relevant to depression and risk that are distinct from what can be gained from more commonly used monetary reward tasks. Examining neural responses across domains of reward, particularly earlier in development, may further inform understanding of developmental trajectories to depression.
Limitations
A cross-sectional design was employed and, thus, causality and direction of results cannot be determined. It will be important to examine these relations in a longitudinal sample to better understand timing of these relations and to target vulnerabilities for depression through intervention. Additionally, although 15.0% of the sample met the cutoff for major depressive disorder (Stasik-O’Brien et al., 2019) and a wide range of depressive symptoms were observed in this sample, extension to clinical samples is needed in future research. We also used a self-report measure of depressive symptoms, and additional research should use an interview-based clinical assessment to confirm diagnoses. Although the two tasks in the present study elicit similar neural responses to social and monetary reward, there are important differences between the tasks that warrant consideration in future work. For example, in the monetary reward task, positive feedback conveys information about both performance (i.e., success on a trial) and monetary reward, and neural responses to these two types of feedback cannot be disentangled with this task design. In the peer interaction task, positive feedback conveys information about both the potential outcome of the task (i.e., staying in the game) and social acceptance. Further, in the peer interaction task, there is likely important variability in neural responses to feedback obtained from specific peers, but the task does not include enough trials to reliably measure ERPs in response to feedback from subgroups of coplayers. Although a RewP component emerges in both tasks, they do show some notable distinctions in distributions across the scalp (as shown in Figure 2), with a more widespread positivity extending over parietal and occipital sites in the social reward task. Future research using source localization methods and combined EEG/neuroimaging studies is needed to evaluate the extent to which the RewP to each type of feedback is driven by activation in overlapping brain networks. Also, although we were able to elicit neural responses to social feedback in a real-time peer interaction task, a computerized EEG task does not fully mirror the complexity of real-world social interactions, and research is needed to further refine lab-based measures of social reward responsiveness. An additional important direction that was not explored in the present study is to consider the time-frequency decomposition of both tasks to examine whether social acceptance and social rejection are associated with similar modulations of EEG frequency bands as those of monetary reward and loss, and whether time-frequency analyses may be applied to better understand the relation between neural processing of social feedback and depression.
Conclusion
The present study is among the first to examine associations between neurophysiological responsiveness to both social and monetary reward and depressive symptoms in emerging adults. These findings emphasize the importance of considering different types of reward in understanding implications of reward processing on depressive symptoms. Results also suggest a pathway by which social reward responsiveness may interact with rejection sensitivity to potentially increase risk for depression. With extension to developmental and clinical samples, it may be possible to translate this work to inform understanding of targets for intervention and prevention.
Supplementary Material
Highlights.
Examined reward responsiveness (RR), rejection sensitivity, and depressive symptoms
Compared social vs. monetary RR
Low social RR, high rejection sensitivity related to greater depressive symptoms
Effects not significant for monetary RR
Results clarify possible pathway for depressive symptoms, may inform intervention
Acknowledgments
The authors would like to thank the study staff who assisted with data collection, particularly Haley Green, Emilia Cardenas, Lindsay Dickey, and Michael West.
Role of the Funding Source
This work was supported through institutional support from NCATS/NIH [UL1 TR000445]. SP was supported by NIH T32-MH18921 during completion of this work. These funders had no role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Footnotes
Conflicts of Interest
Declarations of interest: none.
No substantive changes in results were observed excluding participants with interpolated mastoid electrodes.
Given our prior work finding low social reward responsiveness moderated effects of interpersonal stress on the IDAS dysphoria scale (Pegg et al., 2019), the regression analysis was also conducted with dysphoria as the outcome. The RewP to social reward x rejection sensitivity interaction was trending towards significance, b = −0.05, SE = 0.02, β = −0.64, t(102) = −1.92, p = .058. The RewP to monetary reward x rejection sensitivity interaction was not significant, b = 0.03, SE = 0.02, β = 0.45, t(102) = 1.12, p = .266.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Ait Oumeziane B, Jones O, Foti D, 2019. Neural sensitivity to social and monetary reward in depression: clarifying general and domain-specific deficits. Front. Behav. Neurosci. 13, 199 10.3389/fnbeh.2019.00199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Allen JP, Porter MR, McFarland FC, Marsh P, McElhaney KB, 2005. The two faces of adolescents’ success with peers: adolescent popularity, social adaptation, and deviant behavior. Child Dev. 76(3), 747–760. 10.1111/j.1467-8624.2005.00875.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ayduk O, Downey G, Kim M, 2001. Rejection sensitivity and depressive symptoms in women. Pers. Soc. Psychol. Bull. 27(7), 868–877. 10.1177/0146167201277009 [DOI] [Google Scholar]
- Babinski DE, Kujawa A, Kessel EM, Arfer KB, Klein DN, 2019. Sensitivity to peer feedback in young adolescents with symptoms of ADHD: examination of neurophysiological and self-report measures. J. Abnorm. Child Psychol. 47, 605–617. 10.1007/s10802-018-0470-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beck AT, Steer RA, Brown GK, 1996. Manual for the Beck Depression Inventory-II. Psychological Corporation, New York. [Google Scholar]
- Becker MPI, Nitsch AM, Miltner WHR, Straube T, 2014. A single-trial estimation of the feedback-related negativity and its relation to BOLD responses in a time-estimation task. J. Neurosci. 34(8), 3005–3012. 10.1523/JNEUROSCI.3684-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berenson KR, Gyurak A, Ayduk O, Downey G, Garner MJ, Mogg K, Bradley BP, Pine DS, 2009. Rejection sensitivity and disruption of attention by social threat cues. J. Res. Pers. 43(6), 1064–1072. 10.1016/j.jrp.2009.07.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Braams BR, van Duijvenvoorde AC, Peper JS, Crone EA, 2015. Longitudinal changes in adolescent risk-taking: a comprehensive study of neural responses to rewards, pubertal development, and risk-taking behavior. J. Neurosci. 35(18), 7226–7238. 10.1523/JNEUROSCI.4764-14.2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bress JN, Foti D, Kotov R, Klein DN, Hajcak G, 2013. Blunted neural response to rewards prospectively predicts depression in adolescent girls. Psychophysiology. 50(1), 74–81. 10.1111/j.1469-8986.2012.01485.x [DOI] [PubMed] [Google Scholar]
- Bress JN, Hajcak G, 2013. Self-report and behavioral measures of reward sensitivity predict the feedback negativity. Psychophysiology. 50(7), 610–616. 10.1111/psyp.12053 [DOI] [PubMed] [Google Scholar]
- Bress JN, Meyer A, Proudfit GH, 2015. The stability of the feedback negativity and its relationship with depression during childhood and adolescence. Dev. Psychopathol. 27(4 Pt 1), 1285 10.1017/S0954579414001400 [DOI] [PubMed] [Google Scholar]
- Carlson JM, Foti D, Mujica-Parodi LR, Harmon-Jones E, Hajcak G, 2011. Ventral striatal and medial prefrontal BOLD activation is correlated with reward-related electrocortical activity: A combined ERP and fMRI study. NeuroImage. 57(4), 1608–1616. 10.1016/j.neuroimage.2011.05.037 [DOI] [PubMed] [Google Scholar]
- Cohen JE, 1988. Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey. [Google Scholar]
- Crowley MJ, Wu J, Molfese PJ, Mayes LC, 2010. Social exclusion in middle childhood: rejection events, slow-wave neural activity, and ostracism distress. Soc. Neurosci. 5(5–6), 483–495. 10.1080/17470919.2010.500169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cuthbert BN, 2014. Translating intermediate phenotypes to psychopathology: the NIMH Research Domain Criteria. Psychophysiology. 51(12), 1205–1206. 10.1111/psyp.12342 [DOI] [PubMed] [Google Scholar]
- Davey CG, Yücel M, Allen NB, 2008. The emergence of depression in adolescence: development of the prefrontal cortex and the representation of reward. Neurosci. Biobehav. Rev. 32(1), 1–19. 10.1016/j.neubiorev.2007.04.016 [DOI] [PubMed] [Google Scholar]
- De Rubeis J, Lugo RG, Witthöft M, Sütterlin S, Pawelzik MR, Vögele C, 2017. Rejection sensitivity as a vulnerability marker for depressive symptom deterioration in men. PLoS ONE. 12(10). 10.1371/journal.pone.0185802 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Downey G, Feldman SI, 1996. Implications of rejection sensitivity for intimate relationships. J. Pers. Soc. Psychol. 70(6), 1327–1343. [DOI] [PubMed] [Google Scholar]
- Ethridge P, Kujawa A, Dirks MA, Arfer KB, Kessel EM, Klein DN, Weinberg A, 2017. Neural responses to social and monetary reward in early adolescence and emerging adulthood. Psychophysiology. 54(12), 1786–1799. 10.1111/psyp.12957 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ethridge W, Weinberg A, 2018. Psychometric properties of neural responses to monetary and social rewards across development. Int. J. Psychophysiol. 132(Pt B), 311–322. 10.1016/j.ijpsycho.2018.01.011 [DOI] [PubMed] [Google Scholar]
- Faul F, Erdfelder E, Buchner A, Lang AG, 2009. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods. 41, 1149–1160. 10.3758/BRM.41.4.1149 [DOI] [PubMed] [Google Scholar]
- Forbes EE, Dahl RE, 2012. Research review: Altered reward function in adolescent depression: what, when and how? J. Child Psychol. Psychiatry. 53(1), 3–15. 10.1111/j.1469-7610.2011.02477.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Galvan A, 2010. Adolescent development of the reward system. Front. Hum. Neurosci. 4, 6 10.3389/neuro.09.006.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldstein BL, Kessel EM, Kujawa A, Finsaas MC, Davila J, Hajcak G, Klein DN, 2020. Stressful life events moderate the effect of neural reward responsiveness in childhood on depressive symptoms in adolescence. Psychol. Med. 50, 1548–1555. 10.1017/S0033291719001557 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gratton G, Coles MGH, Donchin E, 1983. A new method for off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55, 468–484. 10.1016/0013-4694(83)90135-9 [DOI] [PubMed] [Google Scholar]
- Hammen C, 2005. Stress and depression. Annu. Rev. Clin. Psychol. 1, 293–319. 10.1146/annurev.clinpsy.1.102803.143938 [DOI] [PubMed] [Google Scholar]
- Hayes A, 2017. Introduction to mediation, moderation, and conditional process analysis, second ed. Guilford Press, New York. [Google Scholar]
- Henry LM, Steele EH, Watson KH, Bettis AH, Gruhn M, Dunbar J, Reising M, Compas BE, 2019. Stress exposure and maternal depression as risk factors for symptoms of anxiety and depression in adolescents. Child Psychiatry Hum. Dev. Advance online publication. 10.1007/s10578-019-00940-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson PO, Neyman J, 1936. Tests of certain linear hypotheses and their applications to some educational problemss. Statistical Research Memoirs, 1, 57–93. [Google Scholar]
- Keren H, O’Callaghan G, Vidal-Ribas P, Buzzell GA, Brotman MA, Leibenluft E, Pan PM, Meffert L, Kaiser A, Wolke S, Pine DS, Stringaris A, 2018. Reward processing in depression: A conceptual and meta-analytic review across fMRI and EEG studies. Am. J. Psychiatry. 175, 1111–1120. 10.1176/appi.ajp.2018.17101124 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kessler RC, McLaughlin KA, Green JG, Gruber MJ, Sampson NA, Zaslavsky AM, Aguilar-Gaxiola S, Alhamzawi AO, Alonso J, Angermeyer M, Benjet C, Bromet E, Chatterji S, de Girolamo G, Demyttenaere K, Fayyad J, Florescu S, Gal G, Gureje O, … Williams DR, 2010. Childhood adversities and adult psychopathology in the WHO World Mental Health Surveys. Br. J. Psychiatry. 197(5), 378–385. 10.1192/bjp.bp.110.080499 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Arfer KB, Klein DN, Proudfit GH, 2014. Electrocortical reactivity to social feedback in youth: A pilot study of the Island Getaway task. Dev. Cogn. Neurosci. 10, 140–147. 10.1016/j.dcn.2014.08.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Burkhouse KL, 2017. Vulnerability to depression in youth: Advances from affective neuroscience. Biol. Psychiatry Cogn. Neurosci. 2(1), 28–37. 10.1016/j.bpsc.2016.09.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Burkhouse KL, Karich SR, Fitzgerald KD, Monk CS, Phan KL, 2019. Reduced reward responsiveness predicts change in depressive symptoms in anxious children and adolescents following treatment. J. Child Adolesc. Psychopharmacol. 29(5), 378–385. 10.1089/cap.2018.0172 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Carroll A, Mumper E, Mukherjee D, Kessel EM, Olino T, Hajcak G, Klein DN, 2018. A longitudinal examination of event-related potentials sensitive to monetary reward and loss feedback from late childhood to middle adolescence. Int. J. Psychophysiol. 132, 323–330. 10.1016/j.ijpsycho.2017.11.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Hajcak G, Klein DN, 2019. Reduced reward responsiveness moderates the effect of maternal depression on depressive symptoms in offspring: Evidence across levels of analysis. J. Child Psychol. Psychiatry. 60(1), 82–90. 10.1111/jcpp.12944 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Arfer KB, Finsaas MC, Kessel EM, Mumper E, Klein DN, 2020. Effects of maternal depression and mother-child relationship quality in early childhood on neural reactivity to rejection and peer stress in adolescence: A 9-year longitudinal study. Clin. Psychol. Sci. 8, 657–672. 10.1177/2167702620902463 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kujawa A, Klein DN, Pegg S, Weinberg A, 2020. Developmental trajectories to reduced activation of positive valence systems: A review of biological and environmental contributions. Dev. Cogn. Neurosci. 43, 1–20. 10.1016/j.dcn.2020.100791 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kupferberg A, Bicks L, Hasler G, 2016. Social functioning in major depressive disorder. Neurosci. Biobehav. Rev. 69, 313–332. 10.1016/j.neubiorev.2016.07.002 [DOI] [PubMed] [Google Scholar]
- Landes I, Bakos S, Kohls G, Bartling J, Schulte-Körne G, Greimel E, 2018. Altered neural processing of reward and punishment in adolescents with Major Depressive Disorder. J. Affect. Disord. 232, 23–33. 10.1016/j.jad.2018.01.017 [DOI] [PubMed] [Google Scholar]
- Leng Y, Qian X, Zhu Y, 2018. Modulation of brain response to peer rejection by rejection sensitivity: An exploratory study. Neuropsychologia, 117, 389–397. 10.1016/j.neuropsychologia.2018.07.003 [DOI] [PubMed] [Google Scholar]
- Liu RT, Kraines MA, Massing-Schaffer M, Alloy LB, 2014. Rejection sensitivity and depression: Mediation by stress generation. Psychiatry. 77(1), 86–97. 10.1521/psyc.2014.77.1.86 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meyer A, Lerner MD, de los Reyes A, Laird RD, Hajcak G, 2017. Considering ERP difference scores as individual difference measures: issues with subtraction and alternative approaches. Psychophysiology. 54, 114–122. 10.1111/psyp.12664. [DOI] [PubMed] [Google Scholar]
- National Institute of Mental Health, 2020. RDoC Matrix. https://www.nimh.nih.gov/research/research-funded-by-nimh/rdoc/constructs/rdoc-matrix.shtml (accessed 23 March 2020).
- Nelson BD, Perlman G, Klein DN, Kotov R, Hajcak G, 2016. Blunted neural response to rewards as a prospective predictor of the development of depression in adolescent girls. Am. J. Psychiatry. 173(12), 1223–1230. 10.1176/appi.ajp.2016.15121524 [DOI] [PubMed] [Google Scholar]
- Novak KD, Foti D, 2015. Teasing apart the anticipatory and consummatory processing of monetary incentives: an event-related potential study of reward dynamics. Psychophysiology. 52(11), 1470–1482. 10.1111/psyp.12504 [DOI] [PubMed] [Google Scholar]
- Novak BK, Novak KD, Lynam DR, Foti D, 2016. Individual differences in the time course of reward processing: stage-specific links with depression and impulsivity. Biol. Psychol. 119, 79–90. 10.1016/j.biopsycho.2016.07.008 [DOI] [PubMed] [Google Scholar]
- Pegg S, Ethridge P, Shields GS, Slavich GM, Weinberg A, Kujawa A, 2019. Blunted social reward responsiveness moderates the effect of lifetime social stress exposure on depressive symptoms. Front. Behav. Neurosci. 13, 178 10.3389/fnbeh.2019.00178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pegg S, Kujawa A, 2020. The effects of a brief motivation manipulation on reward responsiveness: A multi-method study with implications for depression. Int. J. Psychophysiol. 150, 100–107. 10.1016/j.ijpsycho.2020.02.004 [DOI] [PubMed] [Google Scholar]
- Proudfit GH, 2015. The reward positivity: From basic research on reward to a biomarker for depression. Psychophysiology. 52(4), 449–459. 10.1111/psyp.12370 [DOI] [PubMed] [Google Scholar]
- Rappaport BI, Hennefield L, Kujawa A, Arfer KB, Kelly D, Kappenman ES, Luby JL, Barch DM, 2019. Peer victimization and dysfunctional reward processing: ERP and behavioral responses to social and monetary rewards. Front. Behav. Neurosci. 13, 120 10.3389/fnbeh.2019.00120 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Romero-Canyas R, Downey G, Berenson K, Ayduk O, Kang NJ, 2010. Rejection sensitivity and the rejection–hostility link in romantic relationships. J. Pers. 78(1), 119–148. 10.1111/j.1467-6494.2009.00611.x [DOI] [PubMed] [Google Scholar]
- Setterfield M, Walsh M, Frey AL, McCabe C, 2016. Increased social anhedonia and reduced helping behaviour in young people with high depressive symptomatology. J. Affect. Disord. 205, 372–377. 10.1016/j.jad.2016.08.020 [DOI] [PubMed] [Google Scholar]
- Silk JS, Davis S, McMakin DL, Dahl RE, Forbes EE, 2012. Why do anxious children become depressed teenagers? The role of social evaluative threat and reward processing. Psychol. Med. 42(10), 2095–2107. 10.1017/S0033291712000207 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stasik-O’Brien SM, Brock RL, Chmielewski M, Naragon-Gainey K, Koffel E, Mcdade-Montez E, O’hara MW, Watson D, 2019. Clinical utility of the inventory of depression and anxiety symptoms (IDAS). Assessment. 26(5), 944–960. 10.1177/1073191118790036 [DOI] [PubMed] [Google Scholar]
- Sun S, Yu R, 2014. The feedback related negativity encodes both social rejection and explicit social expectancy violation. Front. Hum. Neurosci. 8, 556 10.3389/fnhum.2014.00556 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thapar A, Collishaw S, Pine DS, Thapar AK, 2012. Depression in adolescence. Lancet. 379(9820), 1056–1067. 10.1016/S0140-6736(11)60871-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watson D, O’Hara MW, Simms LJ, Kotov R, Chmielewski M, Mcdade-Montez EA, Gamez W, Stuart S, 2007. Development and validation of the inventory of depression and anxiety symptoms (IDAS). Psychol. Assess. 19(3), 253–268. 10.1037/1040-3590.19.3.253 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.