Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Sep 1.
Published in final edited form as: Biol Psychol. 2017 Jul 13;128:55–62. doi: 10.1016/j.biopsycho.2017.07.001

Social Processing in Early Adolescence: Associations between Neurophysiological, Self-Report, and Behavioral Measures

Autumn Kujawa a, Ellen M Kessel b, Ashley Carroll a, Kodi B Arfer c, Daniel N Klein b
PMCID: PMC5586492  NIHMSID: NIHMS896177  PMID: 28712730

Abstract

Peer relationships play a major role in adolescent development, but few methods exist for measuring social processing at the neurophysiological level. This study extends our pilot study of Island Getaway, a task for eliciting event-related potentials (ERPs) to peer feedback. We differentiated ERPs using principal components analysis (PCA) and examined associations with behavioral and self-report measures in young adolescents (N = 412). PCA revealed an early negativity in the ERP enhanced for rejection feedback, followed by a series of positivities (consistent with reward positivity [RewP], P300, and late positive potential) that were enhanced for acceptance feedback. Greater self-reported task engagement correlated with a larger RewP to acceptance and lower rates of rejecting peers. Youth higher in depressive symptoms exhibited a blunted RewP to social acceptance and reported lower engagement. Results highlight ERP components sensitive to peer feedback that may inform understanding of social processes relevant to typical and atypical development.

Keywords: social feedback, reward positivity, event-related potentials, depression, adolescents


Examining dimensions of social processing at multiple levels of analysis (e.g., self-report, behavior, physiology, circuits) has the potential to inform understanding of both typical and atypical development (e.g., Morris & Cuthbert, 2012). Yet, measurement of social processes, such as affiliation and attachment (National Institute of Mental Health, 2016), poses unique challenges for multi-method assessment. That is, special consideration is needed in order to develop methods that simulate real-world social interactions while also allowing for the collection of levels of data that require specialized equipment (e.g., electroencephalogram [EEG] or functional magnetic resonance imaging [fMRI]).

Adolescence marks a period of increasing importance of peer relationships and the development of more supportive, close friendships (Steinberg & Morris, 2001). In addition, risk for many internalizing disorders increases in adolescence (Costello, Egger, & Angold, 2005; Kessler, Avenevoli, & Ries Merikangas, 2001), and extensive data support a connection between social processes and the development of both depression and anxiety. For example, positive peer relationships appear protective against the development of symptoms, whereas interpersonal stressors, including maladaptive social interactions, peer victimization, and loss of relationships are consistently linked to risk for internalizing symptoms (Hammen, 2005; Hawker & Boulton, 2000; La Greca & Harrison, 2005; Vrshek-Schallhorn et al., 2015).

There has been growing interest in tasks to assess social processes in youth at the neural level. Some of the earliest work on brain circuits used the Cyberball task, a computerized ball-tossing game that elicits reactivity to social exclusion (Eisenberger, Lieberman, & Williams, 2003; Williams, Cheung, & Choi, 2000), with more recently developed fMRI tasks involving the exchange of personal information and pretense of more direct feedback from peers. For example, in the Chat Room task, participants rate how interested they are in participating in an online chat with peers, and then receive feedback regarding whether those peers were interested in chatting with them (Guyer et al., 2008). In the Social Network Aggression Task, participants are told that peers reviewed their personal profiles, and then receive positive, negative, and neutral feedback on their profiles, followed by the chance to retaliate against simulated peers (Achterberg et al., 2017). In the Virtual School Paradigm, participants create personal profiles and avatars that they then use to interact live with simulated peers who direct both positive and negative evaluative comments to the participant (Jarcho et al., 2016). Findings from these task have begun to reveal circuits involved in social feedback processing, including regions of the anterior cingulate cortex (ACC), medial prefrontal cortex (mPFC), insula, striatum and amygdala (Achterberg et al., 2017; Guyer, Choate, Pine, & Nelson, 2012; Jarcho et al., 2016).

Though still in the early stages, evidence from neuroimaging suggests that social feedback tasks may be useful for examining altered social processing in relation to depression and anxiety in youth. For example, youth with major depressive disorder (MDD) have been shown to exhibit altered patterns of neural activation to rejection in the Chat Room Interact task (Silk et al., 2014), and preliminary evidence indicates that youth at risk for depression show blunted responses to social acceptance in reward processing regions such as the ventral striatum (Olino, Silk, Osterritter, & Forbes, 2015). There is also some evidence to suggest that anxiety and anxiety risk are associated with altered patterns of brain activation during anticipation and receipt of negative peer feedback (Jarcho et al., 2013, 2016; Lau et al., 2012), and social feedback tasks may be particularly relevant to the development of social anxiety (Jarcho et al., 2013).

EEG and event-related potentials (ERPs) are neurophysiological measures that are economically assessed across development, can provide insight into the temporal dynamics of social processing, and may be particularly useful for clinical applications (Banaschewski & Brandeis, 2007; Kujawa & Burkhouse, 2017; Nelson & McCleery, 2008). Yet, little work has examined EEG measures of social feedback processing in youth. The reward positivity (RewP), also known as the feedback negativity, is an ERP component that appears as a relative positivity in response to positive or rewarding feedback compared to negative feedback approximately 300 ms after feedback over frontocentral sites (Foti, Weinberg, Dien, & Hajcak, 2011; Gehring & Willoughby, 2002). In monetary reward tasks, RewP is reliably assessed across development (Bress, Meyer, & Proudfit, 2015) and correlates with activation in rewardrelated brain regions, including ventral striatum and mPFC (Becker, Nitsch, Miltner, & Straube, 2014; Carlson, Foti, Mujica-Parodi, Harmon-Jones, & Hajcak, 2011), as well as self-report and behavioral measures of reward sensitivity (Bress & Hajcak, 2013). RewP to monetary rewards appears to be altered in those higher in internalizing symptoms, with evidence of a blunted reward response in youth with elevated symptoms of depression and some evidence of an enhanced RewP in children with elevated symptoms of social anxiety (Bress et al., 2015; Kessel, Kujawa, Proudfit, & Klein, 2015).

We developed a novel task called Island Getaway to measure RewP to social acceptance and rejection feedback in youth (Kujawa, Arfer, Klein, & Proudfit, 2014). During the task, participants vote to reject and accept simulated co-players, while also receiving a combination of rejection and acceptance feedback from peers. To simulate real life interactions, participants and co-players exchange personal information (e.g., photograph, likes/dislikes, and interests) throughout the task, and participants are led to believe that peer feedback may be based on this information. Following the task, participants complete a self-report measure of task engagement. Our initial pilot study (N = 19) indicated that an enhanced RewP was observable following social acceptance feedback, a pattern that has also been observed in adult studies of social feedback processing (Dekkers, van der Molen, Moor, van der Veen, & van der Molen, 2014; van der Molen, Dekkers, Westenberg, van der Veen, & van der Molen, 2016). Moreover, our preliminary correlations indicated that this task may be useful for examining effects of depression and social anxiety on social feedback processing. Though these effects were limited by the small sample size, greater social anxiety symptoms correlated with a larger RewP to acceptance feedback, and greater depressive symptoms were associated with a trend for a blunted RewP to acceptance (Kujawa, Arfer, et al., 2014).

The goals of the current study were to replicate and extend our previous findings in a large sample of young adolescents (N = 412) and refine methods for applying ERPs to understanding social processes across development. Given limited data on ERPs to social feedback in youth, our first goal was to use advanced ERP data analysis techniques (i.e., principal components analysis [PCA]) to evaluate distinct components sensitive to social feedback and to better characterize the timing and scalp distribution of the RewP in response to social feedback. Building on our previous work (Kujawa, Arfer, et al., 2014), our second goal was to evaluate correlations between neurophysiological measures of sensitivity to social feedback (i.e., RewP), self-report (i.e., task engagement), and behavioral measures (i.e., tendency to reject peers) from the Island Getaway task. Our final goal was to test the relevance of these measures to individual differences by examining correlations with symptoms of depression and social anxiety.

Materials and Methods

Participants

Participants were part of a larger community sample of children initially recruited when the children were 3 or 6 years old (Kujawa, Proudfit, & Klein, 2014; Olino, Klein, Dyson, Rose, & Durbin, 2010). Participants were invited back to the laboratory when they were approximately 12 years old and completed a series of EEG tasks in a counterbalanced order (descriptions and results of other tasks will be presented in future manuscripts). A total of 439 youth completed the social feedback task. Informed consent was obtained from all parents and assent was obtained from participants. Data were unavailable for 4 participants due to technical errors and 23 were excluded for excessive noise in the EEG data, leaving data for 412 participants. The mean age was 12.64 years (SD = 0.47 years), and the sample was 46.8% female, 13.1% Hispanic/Latino, 89.1% Caucasian, 7.8% African American, 2.4% Asian American, 0.2% Native American, and 0.5% other race.

Measures

Social feedback task

Participants completed the Island Getaway task (modified from Kujawa, Arfer, et al., 2014) while EEG data were recorded. The task code is available at: http://arfer.net/projects/survivor. Participants were told that they would be playing a game with eleven age-matched co-players in which they would be travelling in the Hawaiian Islands, and at each island, they would have to vote whether they wanted each co-player to continue on with them to the next island and would then receive feedback on how co-players voted for them. Trials were divided into 6 rounds. In the first round, participants created a profile including their photograph and demographic information and reviewed profiles of computerized co-players. In subsequent rounds, participants first responded to a poll question (e.g., “Who do you most admire?”) and reviewed co-player responses in order to facilitate an exchange of personal information for the remaining voting and feedback phases (see Kujawa, Arfer, et al., 2014 for a depiction of the poll phase).

Following review of profiles and poll responses in each round, participants completed a voting and feedback phase (Figure 1). Participants were prompted to vote to either accept (“Keep”) or reject (“Kick out”) each co-player, and after each vote, they then saw feedback indicating whether that co-player had voted to accept or reject them. Acceptance feedback was indicated by an image of a green “thumbs up” and rejection feedback was indicated by a red “thumbs down.” Each voting trial began with a co-player profile presented until participants voted. To simulate variation in co-player response speed, a co-player voting time was selected for each trial based on actual variability in participants’ voting speeds from previously collected data. If participants voted faster than the simulated voting time for that co-player, the message “Waiting for [co-player’s name] to vote…” was displayed. Lastly, a fixation “+” was presented for 1000 ms, followed by feedback displayed for 2000 ms. A blank screen was presented for 1500 ms before the start of the next trial.

Figure 1.

Figure 1

An example voting/feedback trial of the Island Getaway task.

Co-players were randomly assigned a voting pattern for each participant, such that 2 co-players rejected the participant on most (4 or 5 out of 6) rounds, 2 co-players accepted the participant on most rounds, and the remaining 7 co-players were equally likely to accept or reject the participant. To increase unpredictability of feedback, all co-players voted both to keep and kick out the participant at least once (with the exception of the co-player excluded after the first round). Variability in co-player voting patterns allows for examining participant voting behavior in response to distinct patterns of peer feedback, but the task did not include enough feedback trials to examine average ERP responses to feedback from specific co-players.

After each of the rounds, participants were told that one of the co-players had been sent home, and after completing the sixth, participants were informed that they made it to the “Big Island.” The task included a total of 51 feedback trials split evenly between acceptance and rejection, with the last trial type determined randomly, though the proportion of rejection and acceptance feedback in each round varied slightly across participants. The number of feedback trials in each round matched the number of co-players remaining in the game (e.g., 11 in Round 1, 10 in Round 2).

EEG data collection and processing

Continuous EEG was recorded using a 34-electrode cap (32 channels with the addition of FCz and Iz) and a BioSemi system (BioSemi, Amsterdam, Netherlands). The electrooculogram (EOG) generated from eye movements and blinks was recorded using facial electrodes placed approximately 1 cm above and below the eye and 1 cm from the outer corners of the eyes. Electrodes were also placed on the left and right mastoids. Recordings were digitized with a sampling rate of 1024 Hz.

Offline processing was conducted using BrainVision Analyzer software (Brain Products, Munich, Germany). Data were referenced to an average of the recordings from left and right mastoids, band-pass filtered with cutoffs of 0.1 and 30 Hz, and segmented for each trial 200 ms before feedback, continuing for 1000 ms after feedback onset. Eye-blink correction (Gratton, Coles, & Donchin, 1983) and semi-automatic artifact rejection procedures were conducted. Criteria of a voltage step of 50 µV between sample points, a maximum voltage difference of 300 µV within a 200 ms interval, and minimum activity of 0.5 µV within 100ms intervals were used to automatically detect artifacts, with additional artifacts removed by visual inspection. ERPs were averaged for acceptance and rejection feedback, and baseline corrected to activity 200 ms prior to feedback.

Behavioral and self-report measures of social processing

During the task, participants were instructed to vote off a minimum of 1 co-player per round, but were free to vote off as many co-players as they chose. Percentage of votes to reject peers was evaluated as a behavioral measure of social processing. Similar to previous ERP research using the Cyberball task (Kujawa, Arfer, et al., 2014; McPartland et al., 2011), following completion of the task, participants responded to 3 self-report items rated on a 5-point scale to assess engagement in the task (i.e., “I really wanted to stay in the game,” “I would’ve liked to play this game again,” and “After a while I lost interest in staying in the game” [reverse scored]). Scores on each item were averaged to derive a combined measure of task engagement ranging from 1 to 5, with higher scores indicating greater engagement. Behavioral and self-report data from Island Getaway were unavailable for 3 participants due to technical errors.

Internalizing symptoms

To measure current depressive symptoms, adolescents completed the 27-item self-report version of the Children’s Depression Inventory (CDI) (Kovacs, 1992). Three participants chose not to respond to more than 9 items on the CDI and were excluded from analyses of depressive symptoms. Cronbach’s alpha for the CDI in this sample was .82. To measure anxiety, adolescents completed the 41-item self-report version of the Screen for Child Anxiety Related Disorders (SCARED) (Birmaher et al., 1997). Consistent with our prior work (Kujawa, Arfer, et al., 2014) and theory that symptoms of social anxiety may be particularly relevant to social feedback processing, analyses focused on the 7-item social anxiety subscale. Cronbach’s alpha for the social anxiety subscale was .77.

Data Analysis

PCA

Temporospatial PCA on averaged ERP data was conducted using the ERP PCA Toolkit (Dien, 2010b). First, temporal PCA was conducted using all time points from each participant’s averaged data as variables, and participants, trial types, and recording sites as observations. A Promax rotation was used to rotate to simple structure in the temporal domain (Dien, 2010a; Dien, Khoe, & Mangun, 2007). A parallel test (Horn, 1965) was conducted on the resulting Scree plot (Cattell, 1966) in which the Scree of the actual dataset was compared to a Scree derived from a fully random dataset. The largest number of factors accounting for a greater proportion of variance than the random dataset were retained (Dien, 2010a). Based on this criterion, 10 temporal factors (TF) were retained. Following the temporal PCA, a spatial PCA was conducted on each temporal factor (Dien, 2010a; Dien et al., 2007). Variables consisted of all recording sites, and observations included participants, trial types, and temporal factor scores. Infomax was used to rotate the spatial factors to independence (Dien, 2010a). Based on the results of the parallel test, 4 spatial factors (SF) were extracted. The temporospatial PCA resulted in 40 factor combinations, with combined unique variance accounting for 32.5% of variance in the ERP data. To aid in interpretation, factor score were translated into voltages using the ERP PCA Toolkit (Dien, 2012), and robust analysis of variance (ANOVA; Keselman, Wilcox, & Lix, 2003) was conducted on factors accounting for greater than 1% of unique variance in order to identify components that significantly differentiated acceptance and rejection feedback.

Correlations between measures and with symptoms

Subsequent analyses on microvolt-scaled factor scores exported by ERP PCA Toolkit were conducted using SPSS 23. Bivariate correlations were computed to examine associations between ERP, self-report, behavioral measures, and symptoms of depression and social anxiety.

Results

PCA

Average ERPs to acceptance and rejection feedback at Cz prior to PCA are presented in Figure 2. PCA revealed 7 temporospatial factors accounting for a minimum of 1% unique variance in the ERP. Of these, 5 components significantly differentiated acceptance from rejection feedback in robust ANOVA (Table 1; Figure 3). These components included an early negativity, followed by a series of relative positivities for acceptance feedback. The early negativity component (TF4/SF1) appeared similar to an N1 component, possibly reflecting attentional processing (Coch & Gullick, 2012), and was enhanced (i.e., more negative) for rejection compared to acceptance feedback. TF2/SF1, maximal at Cz, appeared the most consistent with the RewP in terms of timing and scalp distribution, though the distribution was somewhat more posterior than observed in monetary reward tasks (Foti et al., 2011). The remaining 3 positive components were similar to P300 and late positive potential (LPP), and were consistent with observations in adults of a series of P300/LPP responses to emotional images and monetary rewards beginning around 300 ms after stimulus onset and emerging across stimulus duration (Foti, Hajcak, & Dien, 2009; Foti et al., 2011). All of the positivities in the current PCA were enhanced for acceptance relative to rejection feedback.

Figure 2.

Figure 2

Average ERP responses to acceptance and rejection feedback at Cz prior to PCA.

Table 1.

Temporospatial factor combinations sensitive to rejection vs. acceptance feedback

Temporospatial factor
combination
Unique variance
explained (%)
Temporal loading
peak (ms)
Peak
electrode
Description Accept vs. Reject
(t value)
TF4/SF1 1.01 158 Cz Central negativity for reject vs. accept 40.11***
TF2/SF1 5.71 356 Cz Central positivity for accept vs. reject 84.96***
TF2/SF2 1.29 356 O2 Ocipitoparietal positivity for accept vs. reject 63.71***
TF3/SF1 2.98 555 CP2 Centroparietal positivity for accept vs. reject 15.60***
TF1/SF1 7.55 911 CP2 Centroparietal positivity for accept vs. reject 21.58***
***

p < .001; t values derived from robust ANOVA; TF = temporal factor; SF = spatial factor

Figure 3.

Figure 3

ERPs and scalp distributions of PCA temporospatial factor combinations accounting for >1% unique variance in the ERP waves and significantly differentiating acceptance from rejection feedback. ERPs are presented at the electrode where each component was maximal (see Table 1). Scalp distributions reflect the relative response to acceptance vs. rejection (i.e., rejection minus acceptance for TF4/SF1 and acceptance minus rejection for remaining components).

Consistent with our previous work on social feedback processing (Kujawa, Arfer, et al., 2014) and to reduce the number of analyses performed, further analyses of correlations between measures focused on the microvolt-scaled factor score corresponding with the RewP component (TF2/SF1). Given recent recommendations (Meyer, Lerner, de los Reyes, Laird, & Hajcak, 2017), residual scores were computed for the RewP factor score to acceptance adjusting for responses to rejection, producing a difference measure that is uncorrelated with RewP to rejection feedback.

Correlations between ERP, self-report and behavioral measures from Island Getaway

Descriptive statistics and correlations between demographic variables, social processing measures, and symptoms are presented in Table 2. Measures derived from Island Getaway were not significantly correlated with age or sex. On average, participants reported relatively high levels of engagement and voted more often to accept than reject co-players. The proportion of participant rejection votes was greater for co-players who had rejected the participant in the previous round (M = 45.2%; SD = 20.7%) than for co-players who had accepted the participant in the previous round (41.1%; SD = 20.2%), t(408) = 5.79, p < .001. Youth who reported more engagement in the task exhibited a lower proportion of votes to reject peers and a larger RewP component. Proportion of rejection votes was not significantly correlated with RewP.

Table 2.

Descriptive statistics and correlations between self-report, behavioral, and neurophysiological measures from the Island Getaway task.

Age Sex (% female) Depressive
symptoms
Social anxiety
symptoms
Task
engagement
Reject votes
(%)
RewP residual
(µV)
Depressive symptoms −.03 .08 -
Social anxiety symptoms −.03 .16** .44*** -
Task engagement −.05 −.01 −.15** .02 -
Reject votes (%) .05 −.01 .01 .00 −.19*** -
RewP residual (µV) .06 −.03 −.10* −.07 .14** −.07 -

M(SD) 12.64(0.47) 4.69(4.98) 3.95(2.91) 3.84(.82) 42.00(17.30) 0.00(6.57)
% 46.8%
***

p < .001;

**

p < .01;

*

p < .05; RewP = reward positivity (microvolt-scaled PCA factor scores, residual score adjusting for response to rejection)

Correlations with symptoms of depression and social anxiety

Greater depressive symptoms were associated with less self-reported engagement in the task and a blunted RewP component to social acceptance (Figure 4). Depressive symptoms were not significantly correlated with residual scores for RewP to rejection adjusting for responses to acceptance, r(407) = .06, p = .21. Symptoms of social anxiety were not significantly correlated with RewP, behavioral, or self-report measures from the task (ps > .13)1.

Figure 4.

Figure 4

Scatter plot depicting the effect of CDI scores (i.e., depressive symptoms) on the component consistent with RewP to acceptance feedback (residuals adjusting for responses to rejection feedback).

Given the correlation between depressive symptoms and the component consistent with RewP, additional exploratory correlations examined associations between depressive symptoms and residual scores for the remaining PCA-derived factor scores. Greater depressive symptoms also predicted a blunted TF2/SF2 component (i.e., P300) to acceptance, r(407) = −.15, p < .01. No significant correlations were observed for the remaining components.

Discussion

The goal of the current study was to extend our pilot study of the Island Getaway task to a large sample of young adolescents and refine methods for examining ERPs to social feedback. Specific goals were to identify distinct ERP components that are sensitive to social feedback, examine associations of ERP measures of sensitivity to social feedback with self-reported task engagement and propensity to reject peers, and examine correlations between social feedback processing measures and symptoms of depression and social anxiety. Five ERP components emerged as sensitive to social feedback and these components appeared temporally and spatially similar to ERP components previously identified in response to monetary rewards and emotional images (e.g., N1, RewP, P300, LPP; Foti et al., 2009, 2011). Self-reported task engagement was modestly but significantly correlated with behavioral and ERP measures, such that youth who reported more engagement in the task were less likely to reject peers and showed a larger RewP to acceptance feedback. Finally, modest correlations were observed with depressive symptoms, such that youth with elevated symptoms of depression exhibited a blunted RewP to acceptance feedback and less engagement in the task. Taken together, these results support the utility of the Island Getaway task in eliciting neurophysiological, behavioral, and self-report measures of social processes.

Our primary correlational analyses focused on the component most closely corresponding with RewP, but PCA results revealed a number of ERP components that warrant further study. In particular, the series of positivities observed in the ERP wave likely reflect components of the P300 and LPP, which reflect attention towards and sustained processing of salient information and may be relevant to the development of both internalizing and externalizing symptoms in youth (Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Foti et al., 2009; Kujawa et al., 2016; Kujawa, Hajcak, Torpey, Kim, & Klein, 2012; Kujawa, MacNamara, Fitzgerald, Monk, & Phan, 2015). Although our exploratory analyses did not reveal significant associations between depressive symptoms and the component consistent with LPP, we did observe an association with P300 to acceptance. Importantly, the current community sample reported relatively low levels of symptoms on average, and it is possible that effects of the LPP on symptoms may be more apparent in clinical samples or as a vulnerability for the later emergence of symptoms (e.g.,Kujawa et al., 2016, 2015).

Consistent with previous findings in monetary reward tasks (Bress et al., 2015; Bress, Smith, Foti, Klein, & Hajcak, 2012), the component consistent with RewP tended to be more blunted in response to social feedback among youth with higher levels of depressive symptoms. This finding suggests that altered reward processing in depression may extend to social reward, and future research is needed to evaluate the unique effects of depression on both social and monetary reward processing. The correlation between RewP and depressive symptoms was small in magnitude, which is consistent with observations of small effects sizes when relating physiological to self-report measures in large samples (Patrick et al., 2013) and may also be attributed to relatively low levels of depressive symptoms in this young sample. Growing evidence supports the possibility that blunted monetary reward processing may be a vulnerability for the later emergence of depressive symptoms (Bress, Foti, Kotov, Klein, & Hajcak, 2013; Nelson, Perlman, Klein, Kotov, & Hajcak, 2016); thus, it is possible the effects of RewP to social feedback will become more apparent when examining increases in depressive symptoms across development. Although the effect of depressive symptoms on RewP was consistent with trends observed in our pilot study, we did not replicate the correlation between social anxiety and RewP to social feedback (Kujawa, Arfer, et al., 2014), possibly because studies with small samples may be more susceptible to error (Leon, Davis, & Kraemer, 2011). Nonetheless, it is also possible that distinct patterns of associations between RewP to social feedback and symptoms of depression and anxiety may emerge when evaluating these measures in clinical samples and across development, particularly later in adolescence and young adulthood.

Analyses of correspondence between the component consistent with RewP and self-report and behavioral measures obtained from the Island Getaway task revealed some modest associations but also indicated that they measure distinct aspects of social processing. For example, while RewP provides an early measure of reactivity to positive and negative feedback, self-reported task engagement likely reflects a broader experience of motivation and interest in the task. Although we previously observed an association between RewP and voting patterns (Kujawa, Arfer, et al., 2014), this correlation was not significant in the current study, nor were voting patterns significantly related to symptoms. In the previous version of the task, participants first voted on all co-players before passively viewing a block of feedback from peers. To increase attention to feedback in the current version, participants voted on a single co-player before receiving feedback from that co-player and then moved on to the next co-player. With this design, trial by trial feedback may have more of an effect on voting patterns, and as such, average rate of rejection votes may be a less sensitive measure of individual differences.

A few additional limitations and future directions should be noted. First, previous work suggests that expectations modulate responses to social feedback (Somerville, Heatherton, & Kelley, 2006; Sun & Yu, 2014); however, as the current task did not assess participant’s expectations for each trial, we were unable to evaluate the potential effects of expectancy on RewP. Relatedly, it is possible that feedback from co-players in the previous round may influence the magnitude of RewP, such that consistent and inconsistent feedback across trials may be characterized by somewhat distinct brain responses. Unfortunately, the current task does not include enough trials to test this possibility. In addition, because participants’ voting behavior varied considerably across individuals and recent evidence suggest that approximately 10 to 15 trials are needed for a stable measure of RewP (Levinson, Speed, Infantolino, & Hajcak, 2017), the majority of participants did not have sufficient trials to reliably evaluate ERP responses to positive and negative feedback as a function of participant votes (e.g., more desirable vs. less desirable peers). Future work is needed to apply alternative statistical approaches or develop modified versions of this task that may provide insight into the effects of expectancy and participant votes on ERP measures. For example, this may be accomplished by lengthening the task or fixing the proportion of acceptance and rejection votes the participant is required to make. In addition to ERPs in response to peer feedback, brain responses when providing feedback to peers or anticipating peer feedback may be relevant to the development of psychopathology. As such, additional research is need to identify ERPs that may be sensitive to participants voting to reject vs. accept peers, as well as those that may me modulated by anticipation of feedback from more or less desirable peers and/or the possibility of acceptance or rejection. Lastly, analytical approaches such as time frequency analyses (van der Molen et al., 2016; van Noordt, White, Wu, Mayes, & Crowley, 2015), should be considered in future work in order to further inform understanding of neurophysiological measures of social processing.

The current study is among the first to examine ERP components sensitive to peer acceptance and rejection in youth, providing insight into a range of components modulated by social feedback and to the temporal dynamics of social feedback processing. This study extends understanding of the RewP component as a measure of social feedback processing and suggests that, as observed in monetary reward tasks (Bress, Meyer, & Hajcak, 2013; Bress et al., 2015), RewP to social acceptance may be blunted among youth with elevated symptoms of depression, possibly reflecting broad deficits in reward-related brain function. With future research, ERP, self-report, and behavioral measures elicited by Island Getaway and similar tasks have the potential to inform understanding of social development across adolescence, as well as the role of social processes in the development of psychopathology.

Highlights.

  • Examined event-related potentials (ERP) sensitive to peer feedback in adolescence.

  • Series of ERPs sensitive to feedback emerged, including reward positivity (RewP).

  • Task engagement correlated with RewP and tendency to reject peers.

  • Greater depressive symptoms correlated with a blunted RewP to acceptance.

  • Measures from task may inform understanding of typical and atypical development.

Acknowledgments

We would like to thank Laura Klein, Dawna Shimabukuro, and Allison Stumper for their help with data collection.

This work was supported by National Institute of Mental Health Grant R01 MH069942 to DNK.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

We also tested whether symptoms of social anxiety or depression were associated with the tendency to vote to reject co-players who had rejected the participant in the previous round. No significant correlations were observed (ps > .80).

References

  1. Achterberg M, van Duijvenvoorde ACK, van der Meulen M, Euser S, Bakermans-Kranenburg MJ, Crone EA. The neural and behavioral correlates of social evaluation in childhood. Developmental Cognitive Neuroscience. 2017;24:107–117. doi: 10.1016/j.dcn.2017.02.007. https://doi.org/10.1016/j.dcn.2017.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Banaschewski T, Brandeis D. Annotation: What electrical brain activity tells us about brain function that other techniques cannot tell us – a child psychiatric perspective. Journal of Child Psychology and Psychiatry. 2007;48(5):415–435. doi: 10.1111/j.1469-7610.2006.01681.x. https://doi.org/10.1111/j.1469-7610.2006.01681.x. [DOI] [PubMed] [Google Scholar]
  3. Becker MPI, Nitsch AM, Miltner WHR, Straube T. A single-trial estimation of the feedback-related negativity and its relation to BOLD responses in a time-estimation task. The Journal of Neuroscience. 2014;34(8):3005–3012. doi: 10.1523/JNEUROSCI.3684-13.2014. https://doi.org/10.1523/JNEUROSCI.3684-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Birmaher B, Khetarpal S, Brent D, Cully M, Balach L, Kaufman J, Neer SM. The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36(4):545–553. doi: 10.1097/00004583-199704000-00018. https://doi.org/10.1097/00004583-199704000-00018. [DOI] [PubMed] [Google Scholar]
  5. Bress JN, Foti D, Kotov R, Klein DN, Hajcak G. Blunted neural response to rewards prospectively predicts depression in adolescent girls. Psychophysiology. 2013;50(1):74–81. doi: 10.1111/j.1469-8986.2012.01485.x. https://doi.org/10.1111/j.1469-8986.2012.01485.x. [DOI] [PubMed] [Google Scholar]
  6. Bress JN, Hajcak G. Self- report and behavioral measures of reward sensitivity predict the feedback negativity. Psychophysiology. 2013;50(7):610–616. doi: 10.1111/psyp.12053. https://doi.org/10.1111/psyp.12053. [DOI] [PubMed] [Google Scholar]
  7. Bress JN, Meyer A, Hajcak G. Differentiating anxiety and depression in children and adolescents: Evidence from event-related brain potentials. Journal of Clinical Child & Adolescent Psychology. 2013:1–12. doi: 10.1080/15374416.2013.814544. https://doi.org/10.1080/15374416.2013.814544. [DOI] [PubMed]
  8. Bress JN, Meyer A, Proudfit GH. The stability of the feedback negativity and its relationship with depression during childhood and adolescence. Development and Psychopathology. 2015;17:1285–1294. doi: 10.1017/S0954579414001400. [DOI] [PubMed] [Google Scholar]
  9. Bress JN, Smith E, Foti D, Klein DN, Hajcak G. Neural response to reward and depressive symptoms in late childhood to early adolescence. Biological Psychology. 2012;89:156–162. doi: 10.1016/j.biopsycho.2011.10.004. https://doi.org/10.1016/j.biopsycho.2011.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carlson JM, Foti D, Mujica-Parodi LR, Harmon-Jones E, Hajcak G. Ventral striatal and medial prefrontal BOLD activation is correlated with reward-related electrocortical activity: A combined ERP and fMRI study. Neuroimage. 2011;57(4):1608–1616. doi: 10.1016/j.neuroimage.2011.05.037. https://doi.org/10.1016/j.neuroimage.2011.05.037. [DOI] [PubMed] [Google Scholar]
  11. Cattell RB. The scree test for the number of factors. Multivariate Behavioral Research. 1966;1:245–276. doi: 10.1207/s15327906mbr0102_10. [DOI] [PubMed] [Google Scholar]
  12. Coch D, Gullick MM. Event-related potentials and development. In: Luck SJ, Kappenman ES, editors. The oxford handbook of event-related potential components. New York: Oxford University Press; 2012. pp. 475–511. [Google Scholar]
  13. Costello EJ, Egger HK, Angold A. The developmental epidemiology of anxiety disorders: Phenomenology, prevalence, and comorbidity. Child and Adolescent Psychiatric Clinics of North America. 2005;14:631–648. doi: 10.1016/j.chc.2005.06.003. [DOI] [PubMed] [Google Scholar]
  14. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. Brain potentials in affective picture processing: Covariation with autonomic arousal and affective report. Biological Psychology. 2000;52(2):95–111. doi: 10.1016/s0301-0511(99)00044-7. https://doi.org/10.1016/s0301-0511(99)00044-7. [DOI] [PubMed] [Google Scholar]
  15. Dekkers LMS, van der Molen MJW, Moor BG, van der Veen FM, van der Molen MW. Cardiac and electro-cortical concomitants of social feedback processing in women. Social Cognitive and Affective Neuroscience. 2014;10(11):1506–1514. doi: 10.1093/scan/nsv039. https://doi.org/10.1093/scan/nsv039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dien J. Evaluating two-step PCA of ERP data with Geomin, Infomax, Oblimin, Promax, and Varimax rotations. Psychophysiology. 2010a;47(1):170–183. doi: 10.1111/j.1469-8986.2009.00885.x. https://doi.org/10.1111/j.1469-8986.2009.00885.x. [DOI] [PubMed] [Google Scholar]
  17. Dien J. The ERP PCA Toolkit: An open source program for advanced statistical analysis of event-related potential data. Journal of Neuroscience Methods. 2010b;187(1):138–145. doi: 10.1016/j.jneumeth.2009.12.009. https://doi.org/10.1016/j.jneumeth.2009.12.009. [DOI] [PubMed] [Google Scholar]
  18. Dien J. Applying principal components analysis to event-related potentials: A tutorial. Developmental Neuropsychology. 2012;37(6):497–517. doi: 10.1080/87565641.2012.697503. https://doi.org/10.1080/87565641.2012.697503. [DOI] [PubMed] [Google Scholar]
  19. Dien J, Khoe W, Mangun GR. Evaluation of PCA and ICA of simulated ERPs: Promax vs. infomax rotations. Human Brain Mapping. 2007;28(8):742–763. doi: 10.1002/hbm.20304. https://doi.org/10.1002/hbm.20304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Eisenberger NI, Lieberman MD, Williams KD. Does rejection hurt? An fMRI study of social exclusion. Science. 2003;290:290–292. doi: 10.1126/science.1089134. https://doi.org/10.1126/science.1089134. [DOI] [PubMed] [Google Scholar]
  21. Foti D, Hajcak G, Dien J. Differentiating neural responses to emotional pictures: Evidence from temporal-spatial PCA. Psychophysiology. 2009;46(3):521–530. doi: 10.1111/j.1469-8986.2009.00796.x. https://doi.org/10.1111/j.1469-8986.2009.00796.x. [DOI] [PubMed] [Google Scholar]
  22. Foti D, Weinberg A, Dien J, Hajcak G. Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: Temporospatial principal components analysis and source localization of the feedback negativity. Human Brain Mapping. 2011;32(12):2207–2216. doi: 10.1002/hbm.21182. https://doi.org/10.1002/hbm.21182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gehring WJ, Willoughby AR. The medial frontal cortex and the rapid processing of monetary gains and losses. Science. 2002;295(5563):2279–2282. doi: 10.1126/science.1066893. https://doi.org/10.1126/science.1066893. [DOI] [PubMed] [Google Scholar]
  24. Guyer AE, Choate VR, Pine DS, Nelson EE. Neural circuitry underlying affective response to peer feedback in adolescence. Social Cognitive and Affective Neuroscience. 2012;7(1):81–92. doi: 10.1093/scan/nsr043. https://doi.org/10.1093/scan/nsr043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Guyer AE, Lau JYF, McClure-Tone EB, Parrish J, Shiffrin ND, Reynolds RC, Fox NA. Amygdala and ventrolateral prefrontal cortex function during anticipated peer evaluation in pediatric social anxiety. Archives of General Psychiatry. 2008;65(11):1303. doi: 10.1001/archpsyc.65.11.1303. https://doi.org/10.1001/archpsyc.65.11.1303. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hammen C. Stress and depression. Annu Rev Clin Psychol. 2005;1:293–319. doi: 10.1146/annurev.clinpsy.1.102803.143938. https://doi.org/10.1146/annurev.clinpsy.1.102803.143938. [DOI] [PubMed] [Google Scholar]
  27. Hawker DS, Boulton MJ. Twenty years’ research on peer victimization and psychosocial maladjustment: a meta-analytic review of cross-sectional studies. Journal of Child Psychology and Psychiatry, and Allied Disciplines. 2000;41(4):441–455. https://doi.org/10.1111/1469-7610.00629. [PubMed] [Google Scholar]
  28. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179–185. doi: 10.1007/BF02289447. https://doi.org/10.1007/bf02289447. [DOI] [PubMed] [Google Scholar]
  29. Jarcho JM, Davis MM, Shechner T, Degnan Ka, Henderson Ha, Stoddard J, Nelson EE. Early-childhood social reticence predicts brain function in preadolescent youths during distinct forms of peer evaluation. Psychological Science. 2016;27(6):821–835. doi: 10.1177/0956797616638319. https://doi.org/10.1177/0956797616638319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Jarcho JM, Leibenluft E, Walker OL, Fox NA, Pine DS, Nelson EE. Neuroimaging studies of pediatric social anxiety: paradigms, pitfalls and a new direction for investigating the neural mechanisms. Biology of Mood & Anxiety Disorders. 2013;3(1):14. doi: 10.1186/2045-5380-3-14. https://doi.org/10.1186/2045-5380-3-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Keselman HJ, Wilcox RR, Lix LM. A generally robust approach to hypothesis testing in independent and correlated groups designs. Psychophysiology. 2003;40(4):586–596. doi: 10.1111/1469-8986.00060. https://doi.org/10.1111/1469-8986.00060. [DOI] [PubMed] [Google Scholar]
  32. Kessel EM, Kujawa A, Proudfit GH, Klein DN. Neural sensitivity to reward diffentiates social from generalized anxiety in children. Journal of Child Psychology & Psychiatry. 2015;56(7):792–800. doi: 10.1111/jcpp.12355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kessler RC, Avenevoli S, Ries Merikangas K. Mood disorders in children and adolescents: an epidemiologic perspective. Biological Psychiatry. 2001;49(12):1002–1014. doi: 10.1016/s0006-3223(01)01129-5. https://doi.org/10.1016/S0006-3223(01)01129-5. [DOI] [PubMed] [Google Scholar]
  34. Kovacs M. Children’s Depression Inventory. Toronto, ON: Multi-Health Systems, Inc; 1992. [Google Scholar]
  35. Kujawa A, Arfer KB, Klein DN, Proudfit GH. Electrocortical reactivity to social feedback in youth: A pilot study of the Island Getaway task. Developmental Cognitive Neuroscience. 2014;10:140–147. doi: 10.1016/j.dcn.2014.08.008. https://doi.org/10.1016/j.dcn.2014.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kujawa A, Burkhouse KL. Vulnerability to depression in youth: Insights from affective neuroscience. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 2017;2(1):28–37. doi: 10.1016/j.bpsc.2016.09.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kujawa A, Hajcak G, Danzig AP, Black SR, Bromet EJ, Carlson GA, Klein DN. Neural reactivity to emotional stimuli prospectively predicts the impact of a natural disaster on psychiatric symptoms in children. Biological Psychiatry. 2016;80:381–389. doi: 10.1016/j.biopsych.2015.09.008. https://doi.org/10.1016/j.biopsych.2015.09.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Kujawa A, Hajcak G, Torpey D, Kim J, Klein DN. Electrocortical reactivity to emotional faces in young children and associations with maternal and paternal depression. Journal of Child Psychology and Psychiatry. 2012;53(2):207–215. doi: 10.1111/j.1469-7610.2011.02461.x. https://doi.org/10.1111/j.1469-7610.2011.02461.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kujawa A, MacNamara A, Fitzgerald KD, Monk CS, Phan KL. Enhanced neural reactivity to threatening faces in anxious youth: Evidence from event-related potentials. Journal of Abnormal Child Psychology. 2015 doi: 10.1007/s10802-015-0029-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Kujawa A, Proudfit GH, Klein DN. Neural reactivity to rewards and losses in offspring of mothers and fathers with histories of depressive and anxiety disorders. Journal of Abnormal Psychology. 2014;123(2):287–297. doi: 10.1037/a0036285. https://doi.org/10.1037/a0036285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. La Greca AM, Harrison HM. Adolescent peer relations, friendships, and romantic relationships: do they predict social anxiety and depression? Journal of Clinical Child and Adolescent Psychology. 2005;34(1):49–61. doi: 10.1207/s15374424jccp3401_5. https://doi.org/10.1207/s15374424jccp3401_5. [DOI] [PubMed] [Google Scholar]
  42. Lau JYF, Guyer AE, Tone EB, Jenness J, Parrish JM, Pine DS, Nelson EE. Neural responses to peer rejection in anxious adolescents: Contributions from the amygdala-hippocampal complex. International Journal of Behavioral Development. 2012;36(1):36–44. doi: 10.1177/0165025411406854. https://doi.org/10.1177/0165025411406854. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research. 2011;45(5):626–629. doi: 10.1016/j.jpsychires.2010.10.008. https://doi.org/10.1016/j.jpsychires.2010.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Levinson AR, Speed BC, Infantolino ZP, Hajcak G. Reliability of the electrocortical response to gains and losses in the doors task. Psychophysiology. 2017;54(4):601–607. doi: 10.1111/psyp.12813. https://doi.org/10.1111/psyp.12813. [DOI] [PubMed] [Google Scholar]
  45. McPartland JC, Crowley MJ, Perszyk DR, Naples AJ, Mukerji CE, Wu J, Mayes LC. Temporal dynamics reveal atypical brain response to social exclusion in autism. Developmental Cognitive Neuroscience. 2011;1(3):271–279. doi: 10.1016/j.dcn.2011.02.003. https://doi.org/10.1016/j.dcn.2011.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Meyer A, Lerner MD, de los Reyes A, Laird RD, Hajcak G. Considering ERP difference scores as individual difference measures: Issues with subtraction and alternative approaches. Psychophysiology. 2017;54:114–122. doi: 10.1111/psyp.12664. https://doi.org/10.1111/psyp.12664. [DOI] [PubMed] [Google Scholar]
  47. Morris SE, Cuthbert BN. Research domain criteria: Cognitive systems, neural circuits, and dimensions of behavior. Dialogues in Clinical Neuroscience. 2012;14(1):29–37. doi: 10.31887/DCNS.2012.14.1/smorris. https://doi.org/10.1097/ALN.0b013e318212ba87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. National Institute of Mental Health. RDoC Matrix. 2016 Retrieved March 28, 2017, from http://www.nimh.nih.gov/research-priorities/rdoc/constructs/rdoc-matrix.shtml.
  49. Nelson BD, Perlman G, Klein DN, Kotov R, Hajcak G. Blunted neural response to rewards prospectively predicts the development of depression in adolescent girls. The American Journal of Psychiatry. 2016;173(12):1223–1230. doi: 10.1176/appi.ajp.2016.15121524. [DOI] [PubMed] [Google Scholar]
  50. Nelson CA, McCleery JP. Use of event-related potentials in the study of typical and atypical development. Journal of the American Academy of Child & Adolescent Psychiatry. 2008;47(11):1252–1261. doi: 10.1097/CHI.0b013e318185a6d8. https://doi.org/10.1097/CHI.0b013e18185a6d8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Olino TM, Klein DN, Dyson MW, Rose SA, Durbin CE. Temperamental emotionality in preschool-aged children and depressive disorders in parents: Associations in a large community sample. Journal of Abnormal Psychology. 2010;119(3):468–478. doi: 10.1037/a0020112. https://doi.org/10.1037/a0020112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Olino TM, Silk JS, Osterritter C, Forbes EE. Social reward in youth at risk for depression: A preliminary investigation of subjective and neural differences. Journal of Child and Adolescent Psychopharmacology. 2015;25(9):711–721. doi: 10.1089/cap.2014.0165. https://doi.org/10.1089/cap.2014.0165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Patrick CJ, Venables NC, Yancey JR, Hicks BM, Nelson LD, Kramer MD. A construct-network approach to bridging diagnostic and physiological domains: Application to assessment of externalizing psychopathology. Journal of Abnormal Psychology. 2013;122(3):902. doi: 10.1037/a0032807. https://doi.org/10.1037/a0032807. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Silk JS, Siegle GJ, Lee KH, Nelson EE, Stroud LR, Dahl RE. Increased neural response to peer rejection associated with adolescent depression and pubertal development. Social Cognitive and Affective Neuroscience. 2014;9(11):1798–1807. doi: 10.1093/scan/nst175. https://doi.org/10.1093/scan/nst175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Somerville LH, Heatherton TF, Kelley WM. Anterior cingulate cortex responds differentially to expectancy violation and social rejection. Nature Neuroscience. 2006;9(8):1007–1008. doi: 10.1038/nn1728. https://doi.org/10.1038/nn1728. [DOI] [PubMed] [Google Scholar]
  56. Steinberg L, Morris AS. Adolescent development. Annual Review of Psychology. 2001;52:83–110. doi: 10.1146/annurev.psych.52.1.83. https://doi.org/10.1146/annurev.psych.52.1.83. [DOI] [PubMed] [Google Scholar]
  57. Sun S, Yu R. The feedback related negativity encodes both social rejection and explicit social expectancy violation. Frontiers in Human Neuroscience. 2014 Jul;8:556. doi: 10.3389/fnhum.2014.00556. https://doi.org/10.3389/fnhum.2014.00556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. van der Molen MJW, Dekkers LMS, Westenberg PM, van der Veen FM, van der Molen MW. Why don’t you like me? Midfrontal theta power in response to unexpected peer rejection feedback. NeuroImage. 2016;146:474–483. doi: 10.1016/j.neuroimage.2016.08.045. https://doi.org/10.1016/j.neuroimage.2016.08.045. [DOI] [PubMed] [Google Scholar]
  59. van Noordt SJR, White LO, Wu J, Mayes LC, Crowley MJ. Social exclusion modulates event-related frontal theta and tracks ostracism distress in children. NeuroImage. 2015;118:248–255. doi: 10.1016/j.neuroimage.2015.05.085. https://doi.org/10.1016/j.neuroimage.2015.05.085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Vrshek-Schallhorn S, Stroud C, College W, Mineka S, Hammen C, Zinbarg R, Craske M. Chronic and episodic interpersonal stress as statistically unique predictors of depression in two samples of emerging adults. Abnormal Psychology. 2015;124(4):918–932. doi: 10.1037/abn0000088. https://doi.org/10.1037/abn0000088. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Williams KD, Cheung CKT, Choi W. Cyberostracism: Effects of being ignored over the Internet. Journal of Personality and Social Psychology. 2000;79(5):748–762. doi: 10.1037//0022-3514.79.5.748. https://doi.org/10.1037/0022-3514.79.5.748. [DOI] [PubMed] [Google Scholar]

RESOURCES