One-Month Stability of Cyberball Post-Exclusion Ostracism Distress in Adolescents

Abstract

We examined one-month reliability, internal consistency, and validity of ostracism distress (Need Threat Scale) to simulated social exclusion during Cyberball. Thirty adolescents (13–18 yrs.) completed the Cyberball task, ostracism distress ratings, and measures of related clinical symptoms, repeated over one month. Need Threat Scale ratings of ostracism distress showed adequate test-retest reliability and internal consistency at both occasions. Construct validity was demonstrated via relationships with closely related constructs of anxiety, anxiety sensitivity, and emotion dysregulation, and weaker associations with more distal constructs of state paranoia and subclinical psychosis-like experiences. While ratings of ostracism distress and anxiety were significantly attenuated at retest, most participants continued to experience post-Cyberball ostracism distress at one-month follow-up, which indicates that the social exclusion induction of Cyberball persisted despite participants’ familiarity with the paradigm. Overall, results suggest that the primary construct of ostracism distress is preserved over repeated administration of Cyberball, with reliability sufficient for usage in longitudinal research. These findings have important implications for translating this laboratory simulation of social distress into developmental and clinical intervention studies.

Introduction

Nearly twenty years ago, Williams et al. (2000) began an important program of research examining the effects of being ignored over the Internet, or “cyberostracism.” Since then, the number of studies employing Cyberball, a task designed to induce the perception of social exclusion and ostracism, has rapidly increased. In Cyberball, the participant plays a computerized game of ball-toss with two or more virtual players whom the participant is led to believe are real people. The participant is initially included (thrown to), but is then excluded from the game. Being left out of the ball toss game Cyberball can induce ostracism distress (Williams et al. 2000). As testament to the burgeoning interest in Cyberball, an early paper introducing the Cyberball task in Behavior Research Methods, Williams and Jarvis (2006), has been cited 570 times to date (https://scholar.google.com, accessed 09/10/2018), the paper introducing the primary assessment of ostracism distress (Need Threat Scale) (van Beest and Williams 2006) has been cited 446 times (https://scholar.google.com, accessed 09/10/2017), and the first neuroimaging study of Cyberball has been cited 3536 times (https://scholar.google.com, accessed 09/10/2018). Many of the studies employing Cyberball examined experimental moderating factors, such as in-group and out-group processes (for a meta-analytic review, see Hartgerink et al. 2015), as well as the psychological and biological correlates of exclusion by others (for meta-analytic reviews, see Cacioppo et al. 2013; Rotge et al. 2015; Vijayakumar et al. 2017).

Recently, both functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) approaches have provided new insights into the development, maintenance, and remediation of maladaptive social cognitive and emotion regulation processes, as both predictors of illness and surrogate markers of targeted treatment mechanisms (Bor et al. 2011; Gee et al. 2013, 2012; Habel et al. 2010; Hooker et al. 2013; Mazza et al. 2010; Popov et al. 2011; Popova et al. 2014; Subramaniam et al. 2012; Taylor et al. 2012; Wölwer et al. 2012; Yee et al. 2010). Exposure and responses to ostracism have important neural correlates related to adolescent self-regulatory processes and psychopathology that are only partially amenable to self-report (Vijayakumar et al. 2017; White et al. 2012). Because of the wide range of psychiatric conditions that involve social distress, responding in Cyberball is inherently transdiagnostic and well-aligned with the research domain criteria (RDoC) initiative (Insel et al. 2010). Indeed, a range of studies on anxiety (Buckner et al. 2010; Zadro et al. 2006), depression (De Rubeis et al. 2016), autism (Bolling et al. 2011b; McPartland et al. 2011), psychosis (Perry et al. 2011), borderline personality disorder (Bungert et al. 2015), antisocial behavior (Brennan et al. 2018) and others, continue to yield new insights about the relevance of ostracism and social pain to mental health and coping. Given the transdiagnostic relevance of coping with social distress, as a psychopathology risk process and a potential treatment target, Cyberball is a promising paradigm for developmental psychopathology and treatment mechanism research (Ramsey and Jones 2015; Vijayakumar et al. 2017).

Researchers have recently re-emphasized the importance of psychometric reliability and validity in measures of clinically-relevant characteristics and constructs (Meyer et al. 2013; Pinkham et al. 2017). Importantly, an evaluation of the test-retest reliability of the ostracism distress assessment, as induced by Cyberball, is needed if we are to apply Cyberball in longitudinal and outcome research. To date, the repeatability and stability of the Cyberball ostracism distress assessment has not been examined. The lack of test-retest reliability data for Cyberball ostracism distress may partially reflect the social psychological research tradition in which Cyberball originated, which tends not to include longitudinal or treatment studies. Additionally, clinical mental health studies do not typically assess reliability in the context of experimental manipulations such as Cyberball. Other factors that may have discouraged previous reliability studies include the Cyberball task design itself, typically two sequential blocks (inclusion then exclusion), which would be more predictable on re-administration. As well, the fictitious nature of the other players is typically disclosed in a post-exclusion debriefing. However, Cyberball has been adapted to include an alternating block format for fMRI (Bolling et al. 2011c; Gradin et al. 2012; Sebastian et al. 2011) with varying numbers of trials per block, a manipulation that is less obvious and predictable, but producing comparable neurophysiological and behavioral effects. With an eye toward evaluating the repeatability of Cyberball, it is worth noting that no adverse events have been reported related to ostracism induction. Thus, it is ethically reasonable to withhold debriefing until after a repeated assessment has occurred.

Cyberball Task

The Cyberball social exclusion task is a well-established paradigm used to examine simulated social distress. The Cyberball alternating version is presented as a computer game of “catch” between the participant and two players, who throw the ball in a way that includes the participant fairly but then excludes the participant for extended periods of time (Bolling et al. 2011c; Sebastian et al. 2011). The other players in the task can be represented as cartoon figures (Williams and Jarvis 2006) or pictures and names matched to the age, sex, and ethnicity of the participant (Bolling et al. 2011a; Crowley et al. 2009, 2012). The primary outcome in the present study is the Need Threat Scale (van Beest and Williams 2006), a self-report measure of ostracism distress administered immediately after the Cyberball task.

Aims

The goals of this study were three-fold, to: 1) evaluate the psychometric adequacy of the Need Threat Scale as a repeated measure of ostracism distress; 2) examine the extent to which participating in the Cyberball task continues to induce ostracism distress after a repeated administration one month later; and 3) examine the construct validity of ostracism distress in relation to putatively related clinical measures.

We recruited 30 adolescent participants from the community. Twenty-eight participants completed two sessions approximately one month apart (mean days = 29.6, SD = 3.4). Using an alternating inclusion/exclusion blocks format, participants were administered the Cyberball task during both sessions, to assess the feasibility and effects of repeated administration.

Primary Hypotheses

1. Ostracism distress will show acceptable internal reliability and test-retest reliability across one month, as indicated by acceptable internal consistency reliability at baseline and one-month follow-up and by a strong correlation across testing occasions.

2. Recognizing that social exclusion in Cyberball could be less distressing following initial administration, owing to familiarity with the task and experience, it was critical to determine that task stimuli remain salient and evoke feelings of social distress at subsequent administrations. We hypothesized that, on a 1-month repeat assessment, participants will still report feeling bothered (significant ostracism distress) by social exclusion as indicated by scores greater than “not at all” (mean score of “1”) or “some distress” (mean score of “2”) on the Need Threat Scale.

3. Higher ostracism distress will be associated with anxiety symptoms, anxiety sensitivity, emotion dysregulation and higher paranoia.

Methods

Participants

Community participants included thirty 13-to-18-year-old male and female teenagers and their parent or guardian. This community sample was recruited via a mass mailing list provided by the credit mailing company Experian, targeting the towns surrounding New Haven, Connecticut, USA. Participating families had responded to a mass mailing from the Yale Child Study Center to express interest in their children participating in research. Of these community participants, twenty-eight returned for follow-up. Exclusion criteria included: having completed the Cyberball task previously, history of clinically significant head injury or central nervous system disorder, and history of developmental disorder. Participants were fluent in English and had no evidence of serious mental illness (psychosis, autism, bipolar disorder) assessed via a parental telephone screen. Participants were given monetary compensation of $40 for the first visit and $60 for the second visit.

Written informed consent was obtained for all 18-year-old participants, and written participant assent and parent/guardian consent were obtained for all participants under the age of 18 years. This study was approved by the Yale University School of Medicine Human Investigation Committee (HIC# 0104012378).

Procedures

Participants were recruited for two visits separated by one month (baseline, one-month follow-up). The two visits were designed to be as similar as possible. Baseline and follow-up visits occurred at the same time of day, day of the week, and with the same experimenter. Self-report demographic and psychological measures were obtained from the parent at the beginning of each research visit and by the participant at the end of each visit using an online questionnaire service (Qualtrics™). During the visit, participants completed two computerized tasks. They first completed a balloon risk task, not reported here. Next, they completed the Cyberball task immediately followed by a Cyberball ostracism distress questionnaire. EEG was recorded during both tasks, but is not included in this report. At the end of the follow-up visit, participants were interviewed about whether they noticed the deception used in Cyberball. Their responses were coded as indicating that they perceived no deception, possible deception, or definite deception. They were then debriefed about Cyberball and asked to maintain test security by not informing others about the deceptive nature of the task.

The Cyberball Social Exclusion Task

As previously described, the Cyberball task induces distress related to social exclusion by engaging the participant in a simple virtual ball toss game with “players” who abruptly exclude the participant, only throwing to one another. Both neural and behavioral findings indicate that this exclusion is experienced as distressing (Eisenberger et al. 2003; van Noordt et al. 2015; Williams 2007). Participants sat 60 cm away from a 19 in. LCD monitor in a dimly lit, sound-attenuated booth. At the outset of the game, the child saw a Google™ webpage, followed by a Cyberball web page, followed by a screen with a status bar. The child selected, from six different ball gloves, a glove to use in the game. During the game, each ball traveled randomly along different paths with sound effects for the toss and catch. The other “players” in the game were represented as silhouetted shoulder height images (black on a grey background) of individuals whose apparent gender was clear and had an ambiguous ethnic appearance. Participants chose the target of their throws using right or left click on a mouse on a lap-desk.

Our alternating version of Cyberball consisted of 124 trials across 8 blocks (4 inclusion and 4 exclusion). The first block was an inclusion block (50% inclusion), with subsequent blocks alternating between exclusion and inclusion. The exclusion blocks contained only one throw to the participant, in order to maintain attention to the game. This one throw during each exclusion block meant the participant experienced inclusion on only 7.69% of trials across 4 exclusion blocks (92.31% exclusion). During inclusion blocks, throws to others, “O” and to the participant “P”, were as follows (O/P): 10/10, 8/9, 9/8, 9/9. Exclusion blocks were as follows (O/P): 12/1, 12/1, 13/1, 11/1. Block length was jittered to reduce predictability.”

The Need Threat Scale

Immediately after Cyberball, participants completed a 20-item Need Threat Scale (van Beest and Williams 2006), from here on referred to as our “ostracism distress” measure. This measure assesses feelings of distress on four dimensions: belonging (“I felt rejected”), self-esteem (“I felt liked”), meaningful existence (“I felt invisible”), and control (“I felt powerful”) on a5-point scale from “Not at all” to “Extremely.” Higher scores indicated greater distress. The mean item rating across 20 items was the primary index of ostracism distress.

Questionnaire Measures

A parent or guardian of each participant completed a questionnaire about personal demographics, as well as child demographics, psychiatric history, and development. Participants completed several computerized questionnaires following the administration of the balloon task and Cyberball tasks, described in detail below.

Screen for Child Anxiety Related Emotional Disorders (SCARED-C) (Birmaher et al. 1999, 1997). The SCARED-C consists of 41 items that assess anxiety symptoms on a three-point scale, 0 (almost never), 1 (sometimes), 2 (often). Item scores are summed to yield a total anxiety score, as well as subscale scores for somatic symptoms/panic disorder, Generalized Anxiety Disorder (GAD), separation anxiety, social phobia, and school phobia.

Childhood Anxiety Sensitivity Index (CASI) (Silverman et al. 1991). The CASI is an 18-item instrument with excellent psychometric properties (Cronbach’s α =.87) designed to measure fear of anxiety symptoms in children and adolescents (Silverman et al. 2003). Each item is assessed on a 3-point Likert scale with 0=” none”, 1=” some”, and 2 =” a lot”. The total score is calculated by summing the ratings across all 18 items.

Difficulties in Emotion Regulation Scale (DERS) (Gratz and Roemer 2008). The DERS is a 36-item self-report questionnaire that assesses participants’ emotion regulation abilities. DERS items are rated on a five-point Likert scale ranging from 1 (“almost never”) to 5 (“almost always”). An example item is “When I’m upset, I become out of control”. Total DERS scores can range from 36 to 180, with higher scores indicating greater difficulty in regulating emotion.

Paranoia Checklist, 5-item state version (PCL-5; Schlier et al. 2016): The PCL-5 is a five-item self-report questionnaire designed to assess change in state paranoia levels related to intra-individual momentary fluctuations in paranoia, with prompts like “I need to be on my guard against others.” Reponses are rated on an 11-point scale from “−5” to “+5” regarding “How strongly do the following thoughts apply to you at the moment?” resulting in scores ranging from –25 to 25, with higher scores indicating greater paranoia. Internal consistency has been reported at 0.83 (Schlier et al. 2016).

Peters et al. Delusions Inventory (PDI; Peters et al. 1996, 2004): The PDI is a self-report questionnaire that assesses whether an individual has ever had particular delusion-like experiences, e.g., “Do you ever feel as if there is a conspiracy against you?” For each item endorsed, participants rate the degree of associated distress, preoccupation, and conviction on 5-point scales. The brief 21-item version is used in the present protocol (Peters et al. 2004). PDI scores have been found to correlate with clinician-rated interview measures of delusions and general psychosis-like experiences. Internal consistency was reported at alpha of 0.82. For reference, Peters et al. (2004)’s normative data included mean scores and standard deviations for healthy participants (6.7 ± 4.4) and participants with delusional disorder (11.9 ± 6.0) for number of items endorsed (Y/N Total).

Analytic Approach

Descriptive statistics were assessed to check assumptions of linear regression-based analyses. A robust estimation approach was used to assess the reliability of test-retest self-report measures, along with traditional linear correlation for reference. The robust estimation procedure included resampling, with replacement, paired values for each bivariate comparison, winsorizing the top and bottom 10% of values, and calculating the correlation. This procedure of resampling, winsorizing, and calculating the correlation was done 10,000 times to yield a distribution of coefficients and the 95% confidence intervals. These analyses were performed using STATSLAB (Campopiano et al. 2018). For Hypothesis 1, the test-retest reliability of ostracism distress from baseline to follow-up was tested with Pearson’s correlation and robust estimate correlation. A correlation coefficient > 0.7 was considered “adequate” (Cohen et al. 2003). An a priori power analysis indicated that statistical power to test this hypothesis is sufficient (0.80) to detect a correlation of r =0.52 (two-tailed, alpha = 0.05) using our target sample of N =30, with a Type II error probability of 0.07% at expected criterion of r = 0.70). Cronbach’s alpha of the ostracism distress scale at each occasion was used to test internal consistency. For Hypothesis 2, the presence of attenuation in ratings of distress and symptoms was tested using paired t-tests with bootstrapped 95% confidence intervals. A significant p value would suggest change in rating levels between baseline and follow-up. We conducted two one-sample t-tests for 1-month re-test data comparing post-exclusion ostracism distress scores against a mean score of “1”, which is equivalent to endorsing “not at all” for ostracism distress on each item, and then against a mean score of”‘ 2”,whichis equivalent to endorsing “some distress” on each item. For Hypothesis 3, robust correlations were used to assess whether ostracism distress was positively associated with psychosis-like symptoms, anxiety symptoms, anxiety sensitivity, and emotion dysregulation.

The data reported in this manuscript may be obtained by e-mailing the first author, CAD.

Results

Descriptive statistics, including means and standard deviations for major study variables are presented in Table 1.

Table 1.

Community control, CHR, and age-gender-matched control sample demographics and baseline scores

	Community sample (n = 28)	Effect size (Cohen’s d)
Gender (female / male)	11/17	-
Age	15.25 (1.48)	-
Ostracism Distress	2.89 (.69)	1.20
SCARED	25.50 (16.36)	1.10
CASI	9.79 (5.62)	1.86
DERS	83.32 (23.11)	1.14
PCL-5	–9.21 (10.55)	.38
PDI	6.04 (2.99)	.15

Mean values are reported with the standard deviation in parentheses. SCARED Screen for Child Anxiety Related Emotional Disorders, CAS1 Childhood Anxiety Sensitivity Index, DERS Difficulties in Emotion Regulation Scale, PCL-5 Paranoia Checklist, PDI Peters et al. Delusions Inventory

Hypothesis 1: test-retest and internal consistency reliability

Individual differences in ostracism distress ratings showed acceptable test-retest reliability over one month (r =.79, robust r =.72, 95% CI = 0.37–0.89, p <.01, see Fig. 1 for scatter plot). Observed power for this two-tailed Pearson’s correlation with a sample of 28 and alpha of 0.05 was greater than 0.99 with critical r = 0.37. Cronbach’s alpha in the current study for the ostracism distress (Need Threat Scale) total score was .93 at baseline and .92 at one-month follow-up. Test-retest reliabilities for other primary measures are reported in Table 2. All were acceptable (r ≥ .70) except for that of the PDI (r =.67, robust r =.58, 95% CI= . 19–.84,p <.01), which also had the lowest internal consistency reliability in this sample, (alpha = .68 at baseline and .67 at one-month follow-up).

Fig. 1.

Scatter plot of baseline and one-month follow-up ostracism distress. Dashed lines indicate sample means at baseline and one-month follow-up

Table 2.

Pearson’s correlation and Robust correlation coefficients for test-retest reliability across one month

	Pearson’s r	Robust r	95% CI
Ostracism Distress	.79**	.72	.37, .89
SCARED	.92**	.91	.82, .97
CASI	.85**	.83	.69, .93
DERS	.88**	.92	.83, .96
PCL-5	.83**	.82	.59, .95
PDI	.67**	.58	.19, .84

SCARED Screen for Child Anxiety Related Emotional Disorders, CASI Childhood Anxiety Sensitivity Index, DERS Difficulties in Emotion Regulation Scale, PCL-5 Paranoia Checklist, PDI Peters et al. Delusions Inventory;

^**p < .01

Hypothesis 2: stability or change in mean scores across occassions

Mean ostracism distress at one-month follow-up (2.57 ± .70) was significantly lower than at baseline (2.89 ± .69; mean difference =.32, t =3.77, p<.01). Similarly, the SCARED, CASI, PDI, and PCL, but not DERS, scores all significantly decreased from baseline to one-month follow-up (Table 3). Despite expecting a decrease in ostracism distress across assessments, we predicted that scores would indicate that ostracism distress was still experienced upon readministration of Cyberball one-month later. Using one-sample t-tests, we first compared mean ostracism at re-test against the score indicating participants were “not at all” bothered by being excluded in Cyberball (response of 1 on all items, t(27) = 12.06, p <.001). Next we conducted the same analysis compared to endorsement of some ostracism distress (a response of 2 on all items, t(27) = 4.39, p < .001). Participants continued to report substantial ostracism distress following social exclusion in Cyberball, even after experiencing the same task one month earlier. Finally, we considered these data descriptively in terms of percentage of participants reporting low levels of distress at baseline and follow-up. At baseline, one participant (3.6% of sample) had an ostracism distress score less than 2 (1.65). At one-month follow-up, four participants (14%) had ostracism distress scores less than 2 (mean 1.56). These analyses support the contention that a majority of participants continue to report being bothered (experiencing ostracism distress) by social exclusion during the Cyberball game, even after having experienced social exclusion (and having the opportunity to reflect on it) one month earlier. Regarding the maintenance of the deception, of twenty-seven participants assessed at retest, n = 7 (26%) indicated they perceived no deception, n =16 (59%) indicated they perceived possible deception and n = 4 (15%) indicated they perceived definite deception. One participant’s response was not recorded.

Table 3.

Paired t-test for test- retest change across one month

Scale	Baseline mean (SD)	One-month follow-up mean (SD)	Paired t-test	Bootstrapped 95% C.I.
Ostracism Distress	2.89 (.69)	2.57 (.69)	3.77**	0.16,0.48
SCARED	25.50 (16.36)	21.75 (15.58)	3.11**	1.43, 6.07
CASI	9.79 (5.62)	8.42 (6.48)	2.09*	0.14, 2.71
DERS	83.32 (23.11)	80.21 (21.90)	1.5	–0.78, 7.50
PCL-5	–9.21 (10.55)	–12.29 (9.93)	2.73*	0.82, 5.39
PDI	6.04 (2.99)	3.61 (2.74)	3.24**	0.54, 2.25

SCARED Screen for Child Anxiety Related Emotional Disorders, CASI Childhood Anxiety Sensitivity Index, DERS Difficulties in Emotion Regulation Scale, PCL-5 Paranoia Checklist, PDI Peters et al. Delusions Inventory; CIs refer to the confidence interval around the mean difference across test and retest.

^*p < .05,

^**p < .01

Hypothesis 3: associations of ostracism distress with symptom measures

Hypothesis 3, predicting a positive relationship between ostracism distress and symptom measures, was partially supported. Ostracism distress was positively correlated with anxiety sensitivity (CASI), emotion dysregulation (DERS), and anxiety (SCARED) at a trend level, but not with state paranoia (PCL-5) or delusional thinking (PDI), as shown in Table 4.

Table 4.

Baseline correlations with ostracism distress

Scale	Robust r	Bootstrapped 95% C.I.	P
Age	.11	−.32, .44	.70
SCARED	.41†	−.01, .73	.05
CASI	41**	.07, .74	.02
DERS	.42**	.11,.67	.01
PCL-5	.21	−.20, .60	.29
PDI	.15	−.21, .56	.32

SCARED Screen for Child Anxiety Related Emotional Disorders, CASI Childhood Anxiety Sensitivity Index, DERS Difficulties in Emotion Regulation Scale, PCL-5 Paranoia Checklist, PDI Peters et al. Delusions Inventory

^†p<0.1,

^**p<0.01

Discussion

This is the first study we know of to examine the stability of Cyberball-induced ostracism distress. Addressing Study Aim 1, test-retest reliability in measurement of ostracism distress responses to the Cyberball task was acceptable, and internal consistency remained high at both the baseline and one-month assessments. The test-retest reliability estimate was slightly lower than for the other measures included in our battery (Table 2), but still within the acceptable range (Cortina 1993). Notably, the ostracism distress scale measures momentary distress in response to a social exclusion experience, whereas most of the symptom scales used in this study (excepting the PCL-5) measure symptom expression over an extended time period. Thus, we could expect that the ostracism distress ratings would be more sensitive to change with experience over repeated measures, producing higher statistical variance compared to these symptom measures.

While participants maintained acceptable rank-order consistency across one month, not surprisingly, we observed a significant reduction in ostracism distress with repeated assessment. Importantly, absolute levels of ostracism distress (greater than a mean score of 2, which would indicate “some” ostracism distress) remained significant on repeat administration. This suggests that, the experience of social exclusion remains salient, and the associated construct of ostracism distress valid, over repeat administrations of Cyberball. Debriefing following the end of the one-month follow-up visit indicated that only 14% of participants were definitely aware of the deceptive nature of the task (i.e., that the other players were not real people), although 57% indicated that they thought a deception might be present. This suggests that the deception is repeatable, with only a small proportion of participants feeling certain that they had been deceived.

Notably, we also observed significant reductions in severity across testing occasions in all but one of our validation measures (anxiety, anxiety sensitivity, paranoia, and psychosis-like experience measures). Reductions in validation measures that were not directly related to Cyberball procedures could be interpreted as change due to increased comfort with general testing procedures at the one-month follow-up. Repeated measures changes (including practice effects) are common in psychopathology research (Maltby et al. 2005; Marsic et al. 2011). With regard to Cyberball and ostracism distress, below we discuss approaches that could further mitigate reduction in state ostracism across assessments.

The construct validity of the ostracism distress measure was supported by moderate correlations with baseline ratings of the convergent constructs anxiety sensitivity and emotion dysregulation. Although potentially supporting discriminant validity, we were surprised ostracism distress was not related to state paranoia or delusional thinking. This effect is likely partially due to the restricted range of scores associated with these symptoms in a healthy, non-psychiatric, community sample. Both measures were endorsed at a very low level by most participants, with little variation. It is likely that much larger samples or specifically recruiting paranoiaprone, prodromal, schizotypal, or similar samples would be necessary to achieve sufficient variance to test these relationships (Lincoln et al. 2017; Premkumar et al. 2015; Westermann et al. 2012).

The conclusions that can be drawn from this study are limited in part by sample size (n = 30 for the reliability assessment of the ostracism distress scale). Nonetheless, the study was sufficiently powered to address the primary hypothesis regarding test-retest reliability, and results have important implications for future research using the Need Threat Scale to measure ostracism distress in the Cyberball paradigm. Given the burgeoning research into all nature of psychopathology and development, the ability to use social exclusion as a repeated measure opens doors to new avenues of translation. Peer rejection and social withdrawal are risk factors for nearly every mental health problem, most of which onset or begin to manifest during adolescence. The social rejection induction approach, such as that in the simulated social exclusion of Cyberball, goes beyond traditional self-report or interview-based measures by providing insight into an individual’s in-the-moment reaction to an experienced social exclusion. Repeatability allows researchers to examine the developmental trajectory of adaptive and maladaptive responses to these inherent challenges of youth. Because a broad range of psychosocial interventions target social skills, and emotion regulation skills such as cognitive reappraisal that are meant to help individuals cope with social distress, repeatability also affords a more direct test of the impact of interventions on adaptive and maladaptive responses to social exclusion. Our results support the use of Cyberball social exclusion to provoke social distress for investigating mechanisms of treatment action in future clinical trials research.

For future research, we might suggest either removing or dampening the ruse to take advantage of the social distress induction inherent in the Cyberball task, regardless of deception. Participant perception of social exclusion appears to be automatic—Cyberball evokes social distress and related brain processes even when participants are told the exclusion is programmed (Bolling et al. 2011c; Crowley et al. 2009; Williams 2007) . Given the pervasiveness of technology, social media, and video gaming among teenagers and young adults (Adams and Kisler 2013; Anderson et al. 2017;Fam 2018;Hokby et al. 2016; Jenaro et al. 2007; Lee et al. 2014), characterizing the Cyberball “players” as “programmed to play the game like the participant in the picture” might engage the automatic tendency to perceive ostracism, without the reappraisal option of claiming deception. Recent news media has made it widely known that computers learn human social media behavior patterns (Shane 2018), and it may be that a social exclusion response is as readily induced by the feeling that another person would act this way rather than is acting this way. On the other hand, researchers have had some success elaborating the deception in Cyberball by employing steps to enhance the perception of “real” players in the game. For example, more “interactive” elements may be introduced with the other players, or participants may be allowed to choose which other players they wish to play with. Players may provide a brief video about themselves recorded on their smart phone and they could view similar putative videos provided by other players. Beyond the believability of Cyberball, the assessment of ostracism distress questions after social exclusion may clue participants into the deception. This may be addressed by asking fewer questions. We are currently using multisite ostracism dataset to develop an abbreviated item set. As well, embedding ostracism questions within a set of other unrelated items could distract from the primary motive of Cyberball. Finally, it is also possible that the repeated provocations could be varied in a developmental study, where Cyberball is used in an early assessment and a Chatroom type paradigm (Silk et al. 2012) is used in a second study visit.