Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Mar 1.
Published in final edited form as: Res Child Adolesc Psychopathol. 2021 Aug 19;50(3):375–385. doi: 10.1007/s10802-021-00862-5

Cortisol Reactivity and Socially Anxious Behavior in Previously Institutionalized Youth

Nicole B Perry 1, Carrie E DePasquale 2, Bonny Donzella 3, Megan R Gunnar 3
PMCID: PMC8857296  NIHMSID: NIHMS1747419  PMID: 34410535

Abstract

The current study investigated the association between cortisol stress reactivity to a social stressor and observed socially anxious behaviors both concurrently and over time among previously institutionalized (PI) (N=132; ages 7–17) youth and a comparison non-adopted (NA) sample (N = 176). Cortisol reactivity was captured during the Trier Social Stress Test for Children (TSST-C, Yim, I.S., Quas J.A., Rush E.B., Granger D.A., & Skoluda N., 2015) and youths’ social anxiety behaviors were coded during the speech portion of the TSST-C. Autoregressive cross-lagged panel models with structured residuals showed that for PI youth, greater cortisol reactivity predicted increases in socially anxious behavior during the TSST-C across three sessions. However, greater cortisol reactivity was negatively associated with concurrent social anxiety behavior. Thus, increases in cortisol reactivity across adolescence may aid in behavioral control in social situations in the short-term but may exacerbate PI youths’ socially anxious behavior over time. No significant associations emerged for NA youth.

Keywords: social anxiety, post-institutionalized, cortisol, HPA axis

Early institutional care has been shown to alter the hypothalamic-pituitary-adrenal (HPA) axis’ cortisol stress response (McLaughlin et al., 2015). The stress of not having a consistently responsive caregiver is thought to trigger chronic and/or prolonged HPA activation, which can have deleterious health effects. As a result, the body may down-regulate HPA axis activity, resulting in a lower than expected cortisol response. In support of this hypothesis, research has found children to have blunted cortisol reactivity to stress, or hypocortisolism, following early institutionalization (Hostinar, Johnson, & Gunnar, 2015; Koss, Mliner, Donzella, & Gunnar, 2016; however, see also Gunnar, Frenn, Wewerka, & Van Ryzin, 2009).

For decades, developmental scientists have used atypical physiological stress reactivity as an indicator of early risk and as a predictor of social and emotional behavioral disorders in children and adolescents (e.g., Alink et al., 2008; DePasquale, Lawler, Koss, & Gunnar, 2020; Susman, Dorn, Inoff-Germain, Nottelmann, & Chrousos, 1997). However, mechanisms linking HPA stress reactivity to socioemotional functioning are less clear. Developmental work has shown that asking children to up-regulate and down-regulate their arousal and behavior is associated with distinct patterns of brain activation and HPA reactivity (Pitskel, Bolling, Kaiser, Crowley, & Pelphrey, 2011). Further, adult work has demonstrated that greater control of arousal and behavior is associated with a larger cortisol response to a social evaluative stressor (de Veld et al., 2012). Thus, optimal social and emotional functioning, including the regulation of behaviors in socially challenging contexts, may require increased cognitive effort and engagement, resulting in greater cortical-limbic activation and HPA responding (i.e., greater cortisol in response to challenge). Alternatively, an inability to mount a stress response during emotionally charged and socially taxing situations may serve as a risk factor for poor socioemotional health.

Indeed, a blunted cortisol response has been linked to poorer social and emotional outcomes (e.g., Alink, Cicchetti, Kim, & Rogosch, 2011; Ayer et al., 2013; Ouellet-Morin et al., 2011). When compared to their non-adopted (NA) peers, previously institutionalized (PI) youth, with a greater likelihood of hypocortisolism, have been found to experience more social and emotional problem behaviors, have greater difficulty understanding social cues, are more likely to experience difficulties in social situations with peers, and have trouble establishing and forming healthy social relationships (Almas et al., 2015; Guyon-Harris, Humphreys, Fox, Nelson, & Zeanah, 2018; Hawk & Mcall, 2010; Hodges & Tizard, 1989; Tarullo, Bruce, & Gunnar, 2007).

Even after adoption into well-resourced homes, PI youth have been shown to demonstrate blunted cortisol and social deficits (e.g., Koss et al., 2016; Kumsta et al., 2017). However, the direct link between cortisol reactivity and social functioning has received little attention in PI populations. Recently, DePasquale and colleagues (2020) found evidence of a developmental cascade from institutional care, to blunted cortisol reactivity (hypocortisolism), to increased disinhibited social engagement, to lower social competence in kindergarten. In an additional study, researchers found that hypocortisolism mediated the association between institutional care and kindergartner’s peer difficulties (Pitula, DePasquale, Mliner, & Gunnar, 2019).

These studies provide initial evidence that there may be lasting negative impacts of early adverse environments on the developing HPA system and youths’ socioemotional functioning. Moreover, they suggest that bunted cortisol reactivity may be one potential mechanism through which early adversity may lead to maladaptive social behavior and decreased social competence. Longitudinal research investigating the change in PI youths’ social behavior over time, however, is scarce. To our knowledge, no study has assessed how changes in cortisol reactivity may be associated with changes in social behavior for PI youth, a necessity if we want to better understand how cortisol reactivity may contribute to the development of social adjustment.

Although previous work links hypocortisolism and PI youths’ maladaptive social behavior in early childhood (Depasquale et al., 2020; Pitula et al., 2019), investigating these associations into adolescence is particularly important given the well-documented social difficulties that emerge for PI youth prior to adolescence (Almas et al., 2015; Guyon-Harris, Humphreys, Fox, Nelson, & Zeanah, 2018; Julian & McCall, 2016; ) and the increasing complexity of the social landscape during the adolescent period. Adolescents are acutely aware of the behaviors displayed by social partners, particularly during emotionally charged contexts, and use these behaviors to develop character judgements and to decide whether to build a relationship, or even whether to continue a social interaction at all. Social behavior that is anxious, atypical, and awkward may hinder PI adolescents’ ability to interact with peers in socially desirable ways, leading to increased social rejection and further psychological maladjustment (e.g., Reijntjes, Kamphuis, Prinzie, & Telch, 2010). Indeed, previous developmental research has linked socially anxious behaviors with poorer peer relations and friendships in adolescence (Greca & Lopez, 1998). Social anxiety has also been associated with persistent deficits in social functioning and quality of life, such as educational attainment, career enhancement, financial dependency, and marital status (Katzelnick and Greist, 2001; Kessler, 2003; Stein et al., 2017). Researchers have identified that the onset of social anxiety usually occurs in childhood, with the most disabling symptoms emerging between the ages of 8 and 15 (Merikangas et al., 2010). Thus, understanding the way in which PI youth’s socially anxious behaviors change over time, and how they are associated with changes in HPA activity across the adolescent period is imperative.

It is possible that hypocortisolism and subsequent social difficulties persist from early childhood into adolescence. However, work from our laboratory using the same sample as the current study showed that puberty, which is characterized by physical and hormonal changes initiated and controlled by the neural and endocrine systems that interact with the HPA axis, may allow the HPA axis to recalibrate to the current more supportive environment. Specifically, results from that work showed that within-individual increases in pubertal stage for PI youth were associated with a cortisol response to stress that became more similar to that of nonadopted (NA) youth over time (Gunnar, DePasquale, Reid, Donzella, & Miller, 2019).

Because blunted cortisol has been associated with poorer social outcomes prior to adolescence (e.g., DePasquale et al., 2020; Pitula et al., 2019), we questioned whether increases to more normative levels of cortisol that we observed with puberty would be associated with changes in social behavior. To address this question, we used the same sample as the previously described study to examine the associations between PI youths’ cortisol stress reactivity and observed socially anxious behavior across 3 time points during the adolescent period.

In previous work using the first annual session of the current study, we showed that PI youth with greater cortisol reactivity during a social stress task exhibited less socially anxious behavior during the speech, suggesting that increased cortisol reactivity may be associated with greater concurrent behavioral control (Perry, Parenteau, Donzella, Desjardins, & Gunnar, 2019). However, given that this study was not longitudinal, it did not allow us to examine the association between cortisol reactivity and observed social behavior across adolescence, a time of change in both HPA reactivity and adolescent’s social skills. Thus, the current study used measures of cortisol reactivity and anxious social behavior across all three annual sessions to investigate these longitudinal associations. Based on our past findings using concurrent data, we hypothesized that as PI youth’s cortisol reactivity increased to levels similar to their NA peers, observed socially anxious behaviors would also decline.

Method

Participants

Participants were youth (age 7 to age 15 at the initial assessment) who were adopted internationally from institutional care (previously institutionalized, PI; N = 132) and a comparison non-adopted (NA) sample who were born and raised in their family of origin (N= 176). PI youth spent at least 50% of their pre-adoption life in institutionalized care, versus foster care or other arrangements (M= 95%, range 50 to 100%). Age at adoption ranged from 5.5 to 59 months (M = 19.30 months, SD = 12.47). More information on the PI sample including country of birth can be found in Gunnar et al. 2019. Family demographics and participant descriptives can be found in Table 1 for both NA and PI youth.

Table 1.

Demographic information for each group.

Post-institutionalized (n = 132) Non-adopted (n = 176)
Session 1 age in years, M (SD) 11.31 (2.40) (min =7.08 yrs; max=15.12 yrs) 11.17 (2.28)(min =7.27 yrs; max=15.0 yrs)
Session 2 age in years, M (SD) 12.25 (2.39) (min =8.08 yrs; max=16.26 yrs) 12.22 (2.27)(min =8.25 yrs; max=16.46 yrs)
Session 3 age in years, M (SD) 13.14 (2.30) (min =9.26 yrs; max=17.0 yrs) 13.18 (2.24)(min =9.26 yrs; max=17.56 yrs)
Sex, n (%) female 89 (67.4%) 93 (52.8%)
Child race, n (%)
 American Indian/Alaskan Native 13 (9.8%) --
 Asian 55 (41.7%) 6 1 (0.6%)
 Black/African American/African (4.5%) 4 (2 3%)
 White 51 (38.6%) 157 (89.2%)
 Multiracial 4 (3.0%) 13 (7.4%)
 Unknown/Other 3(2.3%) 1 (0.6%)
Child region of origin, n (%)
 Russia/Eastern Europe 78 (59.1%) –
 China/Southeast Asia 34 (25.8%) –
 Latin America 14 (10.6%) –
 Africa/Haiti 6 (4.5%) –
Annual family income, n (%)
 < $40,000 – 7 (4.5%)
 $40,001–70,000 19(15.9%) 18 (11.4%)
 $70,001–100,000 25 (21.0%) 40 (25.5%)
 $100,001–150,000 40 (33.6%) 42 (26.8%)
 $150,001–200,000 20 (16.8%) 28 (17.8%)
 > $200,000 15 (12.6%) 22 (14.0%)
Primary caregiver education, n (%) 2 (1.6%) 3 (1.9%)
 High school degree/GED or less 16 (13.1%) 25 (15.8%)
 Some college 42 (34.4%) 68 (43.0%)
 Undergraduate degree 62 (50.8%) 62 (39.2%)
 Graduate/Professional degree
Open in a new tab

Procedures

Participants were part of a study using a short-term accelerated longitudinal design to examine the association between puberty and stress reactivity in youth who experienced institutional care in early life. Measures were obtained at each session, and three sessions occurred over the span of 2 years, with approximately 1 year between visits (M=12.23 mo, SD = .90 months). Parents provided consent and youth provided assent prior to taking part in each of the assessments. The study was approved by the Institutional Review Board at the University of Minnesota (IRB protocol number 1210M21784).

Trier Social Stress Test for Children.

Youth participated in a modified version of the Trier Social Stress Test for Children (TSST-C; Yim. Quas, Rush, Granger, & Skoluda, 2015), a commonly used laboratory procedure to induce psychological stress, socially anxious behavior, and changes in cortisol concentration (Kirschbaum, Pirke, & Hellhammer, 1993). In this social evaluative task, participants give a 5-minute speech, during which they are instructed to introduce themselves to an imaginary classroom. Participants were given 5 minutes to prepare for their speech and write notes but could not use the notes during the speech period. The speech was given in a small room with a one-way mirror and visible camera. The experimenter stood behind the mirror, gave instructions through a speaker, and rated the speech for quality and effectiveness. Participants were told that the experimenter was behind the mirror with a teacher who would also be watching and judging their speech. Instructions before the speech were played from a recording of a male’s voice (the teacher) to ensure that all participants heard the same instructions and perceived someone else was behind the mirror to judge their speech. The recording also told youth that they were being videotaped so that other students could rate them, adding to the social-evaluative stress of the task. If participants stopped their speech before five minutes, they were told to “continue” by the experimenter over the speaker. The experimenters remained neutral which increased the uncertainty about one’s performance. After the speech section, participants performed a verbal arithmetic task aloud for an additional 5 minutes, a standard part of the TSST-C.

Cortisol Collection.

Salivary cortisol was collected four times during the TSST-C (0, +5, +20, and +40 minutes surrounding the TSST-C). These samples were selected to reflect cortisol reactivity to the task (see Gunnar et al., 2019 for a description of all samples), given the lag between when the adrenal secretes cortisol and when it appears in saliva. Sample 0 was collected 40 minutes after arrival to the lab, prior to the start of the TSST-C, and is considered a baseline or ground from which cortisol may rise in response to the TSST-C. During the 40 minutes prior to Sample 0, participants were asked to relax in a waiting room with their parent while their parent filled out questionnaires. Sample +5 was taken following the speech instructions and speech preparation period and may capture some anticipatory stress; +20 was taken following the speech delivery and math performance; +40 was taken following the completion of the TSST-C. Together, samples +5, +20, and +40 are intended to capture the cortisol reactivity to the TSST-C (see Figure 1 for a depiction of the collection samples).

Figure 1.

Figure 1.

Open in a new tab

TSST session timeline. Saliva samples are indicated by the water droplets beneath the timeline. Adapted with permission from (Gunnar et al., 2019). Only samples 0, +5, +20, +40 were used in the current manuscript.

The samples were kept frozen at −20°C until being shipped to the University of Trier, Germany to be assayed using a time-resolved fluorescence immunoassay (DELFIA), with intra- and inter- assay CV < 10%. All of the samples from each participant for a given assessment were included in the same assay batch. Samples were assayed in duplicate, duplicates were correlated > .99 and thus averaged. Because the HPA axis has a strong diurnal rhythm, all TSST-C sessions began between 3:00 and 4:30 pm. Participants were asked to refrain from eating and drinking (including water and especially caffeine) during the visit. One participant had biologically implausible values (>2 μg/dL at session 3) and that session was excluded for that participant. Across the three sessions, a total of 10 samples (0.04%) had values greater than 4SD from the mean. It was decided to include those values to allow natural variation to occur.

Cortisol Reactivity (AUCi).

Area under the curve with respect to increase (AUCi) is a formula used to capture a cortisol change. Specifically, AUCi emphasizes changes over time and is related to the sensitivity of the hormonal system, as the formula takes into account sensitivity and intensity. Thus, the AUCi formula is useful to derive a stress response over an event period (Fekedulegn et al., 2007) and can be thought of as an index of sensitivity of the system in response to a stressor, in this case, the TSST-C (Morris, Rao, Wang, & Garber, 2014). Formulas for the calculation of AUCi are derived from the trapezoid formula, using simple additions of areas of triangles and rectangles (Fekedulegn et al., 2007; Pruessner, Kirschbaum, Meinlschmid, & Hellhammer, 2003). AUCi is calculated from the area under the curve across samples minus the area under the curve below the baseline. In the present study, AUCi was calculated from participants’ salivary cortisol from the 4-time points relevant to the TSST, using the initial (prestress) sample as a baseline. To preserve power, any missing samples were imputed as follows: cortisol was taken as the mean of the nearest neighbor samples for a subject, and the modal time across the study for that sample was used. Imputations were only performed if a person was missing a single sample. This resulted in 4 imputations across all three sessions, or 0.01%. Thus, AUCi was included to examine participants’ cortisol response sensitivity to the speech and math task, with higher scores representing greater cortisol reactivity during the TSST-C.

It is important to note that approximately half of the sample could be classified as responding to the TSST-C by increasing in cortisol 15.5% above baseline levels (Miller, Plesswo, Kirschbaum, & Stalder, 2013. For each of the three sessions, we saw this response rate (overall/by group): S1 all: 46% (NA 51% PI 49%); S2 all: 40% (NA 41% PI 59%); S3 all: 32% (NA 32% PI 32%). However, we expected to find variation in the effectiveness of the task to produce a cortisol response, given that many children entered the study pre-pubertally and younger children tend to show lower responder rates to the task.

Socially Anxious Behavior.

Because the TSST-C was designed to elicit psychosocial stress and social anxiety, youths’ behavior during the 5-minute speech was coded on 7 scales to examine socially anxious behavior. We chose to code behavior during the speech portion rather than the math portion of the TSST-C because it is embedded in a social context. Thus, the speech portion allowed us to capture behaviors that youth employed while attempting to deliver a speech to judges and peers.

The socially anxious behaviors used in the current study were borrowed from a previously developed coding scheme (see Burkholder, Koss, Hostinar, Johnson, & Gunnar, 2016). Anxious behavior was coded in the forms of facial (e.g., widened eyes, eyes darting, winces, facial fear expressions), vocal (e.g., quaking of voice, overly anxious vocal content such as “Oh shoot” or “I can’t do this”, using back-to-back filler words such as “um, uh, like, and”, and stumbling over words), gross body (e.g., stereotypic swaying, gross self-touch such as rubbing arms, and large body movements), and fine body (e.g., fidgeting with hands or fingers, playing with clothing in non-obvious way such as with sleeves or hem, and twirling hair). Socially anxious behavior scores were not mutually exclusive and were rated 0 to 3, with 0 reflecting no evidence of the behavior and 3 reflecting high intensity/frequency; 22% of the videotapes were double coded and intraclass correlations (ICC) ranged from .70 to .83 for each scale.

A composite score was created by creating a mean of the facial, vocal, and gross body subscales to reflect socially anxiety behaviors (α=.81); fine motor was not included in this composite because it did not correlate with the other anxiety codes. Higher scores reflected greater social anxiety. Youth also self-reported on how stressed they were during the TSST-C. Interestingly, there were no significant correlations between self-reported stress and social anxiety behaviors for NA youth at any of the 3 assessments. However, youth-reported stress was correlated with observed socially anxious behavior at assessment 1 (r = .18, p<.05), assessment 2 (r = .23, p <.01), and assessment 3 (r =.34, p<.01) for PI youth.

Parent and Youth Report of Social Functioning.

Parents and youth filled out the MacArthur Health and Behavior Questionnaire (HBQ; Essex et al., 2002) during each laboratory visit. The parent-reported social anxiety subscale and the child-reported social inhibition subscale were collected at all 3 assessments and used in the current analyses. Internal reliability was high for parent-reported social anxiety (S1 α =.89, S2 α = .91, S3 α = .92). Internal reliability for child-reported social inhibition (S1 α =.65, S2 α = .75, S3 α = .77) was adequate across the 3 assessments. Lower alphas for youth report were expected, particularly at session 1 when youth ranged from age 7 to age 13. However, given their consistency with parent-reports we chose to include this valuable perspective.

Data Analytic Plan

Autoregressive cross-lagged panel models with structured residuals (Berry & Willoughby, 2017; Curran, Howard, Bainter, Lane, & McGinley, 2014) were used to examine concurrent and cross-lagged associations between cortisol reactivity and observed behaviors during the TSST-C across the adolescent period. Separate models were run for PI and NA youth. Data were analyzed using the ‘Lavaan’ package in R (R Core Team, 2016; Rosseel, 2012). Missing data were handled using full-information maximum likelihood.

Univariate models were built through the systematic addition of paths and then comparing model fit to test whether the added paths significantly improved model fit (favoring parsimony if there was no significant difference in fit). Good model fit was determined by Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) > .95 (Bentler, 1990; Tucker & Lewis, 1973), and root mean squared error of approximation (RMSEA) and standardized root mean squared residual (SRMR) < .08 (Hu & Bentler. 1999; MacCallum, Browne, & Sugawara, 1996). Model fit was compared using the chi-square test of difference using the ‘anova’ base R function. We examined univariate models with a latent slope on its own, which did not have good fit for cortisol reactivity (CFI: .08, TLI: .31, RMSEA: .14, SRMR: .11) or behavior (CFI: .000, TLI: −8.41, RMSEA: .84, SRMR: 2.51). A lack of evidence for a latent slope suggests that cortisol and behavior did not change linearly over time on average. Therefore, a latent intercept model with autoregressive paths (stability paths over time) was compared to a model with just a latent intercept. The model including autoregressive paths provided significantly better model fit for both cortisol reactivity (χ2 (1, N = 132) = 9.71, p = .002) and behavior (χ2 (1, N = 132) = 15.78, p < .001). These findings suggest that there was significant variation in where individuals start in their cortisol reactivity and behavior and how they change over time. Therefore, latent intercepts and autoregressive paths were included for both cortisol reactivity and observed behaviors in the bivariate model.

A bivariate model was created using the combination of the two above-determined univariate models for cortisol reactivity and socially anxious behaviors. Given that cortisol reactivity is known to become more prominent in adolescence compared to pre-pubertal aged children (Gunnar, Wewerka, Frenn, Long, & Griggs, 2009), and that social anxiety is more commom among females than males (Caballo et al, 2014), child age and sex were included as covariates. Latent intercepts for both variables were allowed to be correlated. Then, correlations among variable residuals within timepoint (i.e., concurrent associations) and cross-lagged paths from cortisol to behavior and behavior to cortisol were also added to the model. If the addition of all concurrent and cross-lagged paths significantly improved model fit, this model was accepted. Acceptance of the model including the cross-lagged paths indicates that there is evidence of cross-construct associations over time between cortisol reactivity and socially anxious behavior. If the more parsimonious bivariate model with only correlated intercepts was accepted, this indicated that there are no cross-construct associations over time.

Results

Descriptive Statistics

Means, standard deviations, and intercorrelations among focal variables are displayed in Table 2 for PI youth and Table 3 for NA youth. T-tests revealed no significant mean differences in the primary study variables across PI and NA youth. Means and standard deviations for cortisol samples at each assessment for PI and NA youth can be found in Table 4. To be sure that socially anxious behavior during the speech was not just an observed measure of general anxiety in our sample, we assessed the extent to which child-reported internalizing symptoms on the HBQ were associated with observed social anxiety behaviors. There was no significant correlation at any of the three time points between child-report of internalizing and the socially anxious behavior we observed in the laboratory for PI or NA youth, suggesting that observed behavior in socially stressful situations is not a direct reflection of general feelings of anxiety but is instead a separate construct that reflects an inability to manage social behavior during socially and emotionally challenging contexts. Importantly, the observed social anxiety behaviors we coded in the laboratory were correlated with parent report of youth’s social anxiety at S1 (r = .15, p <.05), youth report of social inhibition at S2 (r = .16, p <.05), and both parent report of anxiety (r = .16, p<.05) and youth report of social inhibition (r = .14, p<.05) at S3. These correlations highlight that the behaviors we observed in the lab to the TSST-C were associated with broader indices of social inhibition and social anxiety, providing evidence that youths’ social anxiety/inhibition can be observed in their daily social interactions by social partners through specific behaviors they display.

Table 2.

Descriptives and intercorrelations among all focal variables for PI youth (N = 132).

1 2 3 4 5 6 7
1. S1 Age
2. S1 Cortisol AUCi −0.01
3. S2 Cortisol AUCi 0.11 0.34***
4. S3 Cortisol AUCi 0.12 0.06 −0.08
5. S1 Social Behavior −0.31*** 0.14 0.07 −0.01
6. S2 Social Behavior −0.35*** 0.17 0.06 0.02 0.30**
7. S3 Social Behavior −0.37*** 0.11 0.06 −0.20* 0.29** 0.41***
131 122 105 102 124 114 98
Mean (SD) 11.3 (2.40) −0.00 (0.05) −0.00 (0.03) −0.00 (0.02) 1.50 (0.57) 1.27 (0.67) 1.08 (0.73)
Open in a new tab
*

p < .05

**

p < .01

***

p < .001

N

Table 3.

Descriptives and intercorrelations among all focal variables for NA youth (N = 176).

1 2 3 4 5 6 7
1. S1 Age
2. S1 Cortisol AUCi .00
3. S2 Cortisol AUCi 0.07 0.10
4. S3 Cortisol AUCi 0.08 −0.01 .20*
5. S1 Social Behavior −0.31*** −0.01 0.01 .00
6. S2 Social Behavior −0.55*** −0.05 −0.02 0.08 0.46**
7. S3 Social Behavior −0.14 −0.13 0.04 .05 0.36** 0.49***
175 160 138 129 172 149 124
Mean (SD) 11.2 (2.28) −0.00 (0.04) 0.00 (0.03) 0.00 (0.02) 1.48 (0.62) 1.38 (0.71) 1.26 (0.61)
Open in a new tab
*

p < .05

**

p < .01

***

p < .001

N

Table 4.

Means and standard deviations for cortisol samples at each assessment.

Previously-Institutionalized Non-adopted
Assessment 1 M(SD) M(SD)
Cortisol Sample 0 .09(.12) .11(11)
Cortisol Sample +5 .08(.08) .10(.09)
Cortisol Sample +20 .09(.08) .10(.09)
Cortisol Sample +40 .10(.10) .12(13)
Assessment 2 M(SD) MSD)
Cortisol Sample 0 .09(.07) .09(.07)
Cortisol Sample +5 .10(.09) .10(.09)
Cortisol Sample +20 .08(.06) .08(.06)
Cortisol Sample +40 .09(.08) .09(.08)
Assessment 3 M(SD) M(SD)
Cortisol Sample 0 .09(.05) .09(.05)
Cortisol Sample +5 .09(.05) .09(.05)
Cortisol Sample +20 .09(.05) .09(.05)
Cortisol Sample +40 .09(.05) .09(.05)
Open in a new tab

Cortisol Reactivity and Observed Socially Anxious Behavior – Cross-Lagged Model

For PI youth, the bivariate model of cortisol reactivity and socially anxious behavior that included concurrent and cross-lagged paths (CFI: .95, TLI: .94, RMSEA: .04, SRMR: .05) showed a significant improvement in model fit compared to the bivariate model without concurrent and cross-lagged paths (χ2 (4, N = 132) = 9.71, p < .05). Acceptance of this model suggests that modeling longitudinal associations between cortisol reactivity and socially anxious behavior better fit the data than excluding them; this indicates evidence of significant associations between the two constructs emerge over time. Final model results and fit statistics for PI youth can be found in Figure 2. Contrary to our hypothesis, greater cortisol reactivity significantly predicted increases in socially anxious behavior during the TSST-C over time, accounting for previous anxious behavior (p = .05). That is, cortisol reactivity at annual session 1 predicted increases in social anxiety behavior from annual session 1 to annual session 2 controlling for the stability in behavior over time. Similarly, cortisol reactivity at annual session 2 predicted increases in socially anxious behavior from annual session 2 to annual session 3 controlling for stability in behavior. These results make clear that as cortisol increases across the early adolescent and adolescent period, we see an increase in socially anxious behaviors displayed by PI youth. However, similar to our work using only assessment 1, greater cortisol reactivity also significantly negatively predicted concurrent socially anxious behavior at sessions 2 and 3 (p = .04). Thus, concurrent correlations suggest that increased cortisol may help control behavioral displays in the moment.

Figure 2.

Figure 2.

Open in a new tab

Final model results for cortisol reactivity-socially anxious behavior associations in PI youth (N = 132). CFI: .95, TLI: .94, RMSEA: .04, SRMR: .05. All models were fit using full information maximum likelihood to use all available data. Standardized estimates are shown. Covariates of child age and sex are not shown here for clarity, but are accounted for in the model. Significant paths are shown with solid lines, while non-significant paths are shown with dotted lines. *p < .05, ***p < .001

Model results for the comparison NA group can be found in Figure 3. Importantly, significant associations did not emerge. In fact, the bivariate model of cortisol reactivity and socially anxious behavior that included concurrent and cross-lagged paths did not show a significant improvement in model fit compared to the bivariate model without including them. These results suggest no longitudinal evidence for associations between the two constructs over time in NA youth.

Figure 3.

Figure 3.

Open in a new tab

Final model results for cortisol reactivity-socially anxious behavior associations in NA youth (N = 176). CFI: 1.00, TLI: 1.06, RMSEA: .00, SRMR: .05. All models were fit using full information maximum likelihood to use all available data. Standardized estimates are shown. Covariates of child age and sex are not shown here for clarity, but are accounted for in the model. Significant paths are shown with solid lines, while non-significant paths are shown with dotted lines. + p < .10

Discussion

Behaviors play a large role in how individuals are perceived by social partners and the extent to which social partners want to continue to engage in interpersonal interactions. Previous work has shown PI youth are more likely to display abnormal social behaviors, have a harder time in social exchanges, and have more difficulty sustaining social relationships than their NA counterparts (Almas et al., 2015; Hawk & McCall, 2011). Thus, PI youths’ continued engagement in behavior that is socially awkward or anxious may be particularly detrimental, especially during the adolescent period when being accepted by peers is critical for overall mental health and adaptive functioning throughout the lifespan. Little work investigates the change in PI youth’s social behaviors over time, and no empirical work, to our knowledge, has used repeated assessment of cortisol reactivity and social behavior to investigate the potential longitudinal and dynamic associations that may emerge across development. To address these gaps, we chose to examine these associations during the adolescent period, as this is a time of increasing social challenge and potential recalibration of the HPA axis for PI youth.

Significant concurrent associations showed a negative association between anxious behavior during the TSST-C and cortisol reactivity at annual sessions 2 and 3 such that as PI youth’s cortisol response increased, they were less likely to display concurrent socially anxious behavior. This finding directly aligns with our previously described work conducted with this sample at annual session 1 (Perry et al., 2019). Taken together, the concurrent findings from both studies suggest that in the moment, increased cortisol to a social stressor is associated with better control of social behavior for PI youth. There is considerable evidence that the regulation of emotion and behavior during social situations require effort at a physiological level (e.g., Pitskel, Bolling, Kaiser, Crowley, & Pelphrey, 2011). Thus, it is possible that greater cortisol reactivity during a socially stressful situation serves as a resource for PI youth to better handle the challenge and engage in socially acceptable behavioral displays in the moment. Given that there were no concurrent associations between cortisol reactivity and behavior found for NA youth in either study at any session, these findings suggest that adaptive behavioral functioning in the moment may require extra biological effort for PI adolescents specifically.

Contrary to our hypotheses, the opposite effect emerged when looking at these processes in PI youth longitudinally. We found that greater cortisol reactivity at annual session 1 predicted increases in socially anxious behavior from annual sessions 1 to 2. Similarly, cortisol reactivity at annual session 2 (controlling for cortisol reactivity at annual session 1) predicted increases in socially anxious behavior from annual session 2 to 3. Although internalizing symptoms and socially anxious behaviors are not correlated, these findings are comparable to previous work with this sample in which we found increases in cortisol reactivity to be associated with increases in PI youths’ report of internalizing symptoms across puberty (Perry, DePasquale, Donzella, & Gunnar, 2020). Together, these studies suggest that increases in cortisol across adolescence may not necessarily result in optimal social or emotional outcomes for PI youth. In addition, the lack of correlation between internalizing symptoms and social behavior show that they are not merely duplicate measures, but that they may each be driven by another process, such as the body’s response to challenge.

It is possible that increased cortisol reactivity during socially stressful situations sensitizes individuals to experience increased stress the next time they encounter a similar social challenge, resulting in increased difficulty managing social behavior over time. That is, increased cortisol reactivity in the moment may help modulate concurrent social behavior but it may also prime the individual to experience more stress and socially anxious behaviors at subsequent time points. Another possible explanation is that brain structures that developed in response to early life adversity, and in the context of hypocortisolism, are then met with a relatively sudden influx of cortisol during pubertal recalibration. This influx of cortisol could occupy glucocorticoid receptors (GRs) and subsequently induce dendritic remodeling in the amygdala (e.g., Vyas et al., 2002), that may result in increased, rather than decreased, social anxiety and behavioral dysregulation over time (Strüber et al., 2014).

Importantly, this study highlights the critical need for researchers to investigate the associations between cortisol reactivity and child outcomes both concurrently and longitudinally. Had we only examined the concurrent associations, we may have concluded that increased cortisol reactivity during the adolescent period is optimal for PI youth in that it is associated with greater behavioral control. However, it was not until the longitudinal associations were examined that evidence emerged indicating that continued increases in cortisol reactivity across adolescence may in fact not be beneficial in the long term.

These findings have significant implications for PI adolescents who experience normative, yet significant, social stressors continuously throughout adolescence. If the cyclical nature of this association continues, it suggests that PI youth may develop increased behavioral dysregulation and anxiety into early adulthood. Indeed, some empirical research has shown a late-onset pattern of self-rated emotional symptoms among PI young adults as compared to their NA counterparts (Sonuga-Barke et al., 2017). Thus, prevention and intervention efforts may need to focus on providing PI adolescents with additional tools to regulate their behavioral displays and feelings of anxiety when in socially stressful contexts. It should be noted, however, that it is also possible that recalibration of behavioral symptoms lags behind biological recalibration, resulting in less anxiety and more behavioral control by early adulthood. Longitudinal work following PI youth from adolescence into adulthood is needed to understand the lasting implications of recalibration for social and psychological functioning.

Although the current study has many strengths, it also has several limitations. First, although age was included as a covariate, we did not directly assess pubertal stage, which has been found to be associated with cortisol increases (Gunnar et al., 2020). However, previous work provides evidence of a strong correlation between pubertal stage and age (Gunnar et al., 2019). A second limitation is that the clinical relevance of these findings is not yet determined given that PI youth were not showing significantly higher mean levels of socially anxious behavior at any session compared to their NA peers. However, PI youth reported feeling more stress during the TSST-C than NA youth at all 3 assessments, which suggests that future research investigate the implications of greater feelings of stress in PI youth as they relate to behavioral and mental health. A final limitation is that findings from the current study cannot be generalized beyond the PI population. PI youth are not representative of children experiencing other types of early adversity such as maltreatment or extreme poverty. Thus, additional work is needed to better understand how these findings might translate to other populations of at-risk youth.

Even with these limitations, however, the current study adds considerably to the literature on PI youth’s social behaviors beyond early childhood, and elucidates how changes in cortisol reactivity may be associated with social functioning. Specifically, it is one of the first to examine the direct association between cortisol reactivity to stress and concurrent or future observed social behavior in PI youth. It is also the first to our knowledge to investigate how cortisol increases across adolescence are associated with changes in PI youths’ observed behavior, a particularly salient outcome given the social challenges PI youth face (Hawk & McCall, 2011).

Acknowledgments:

The authors thank the families, who devoted many hours to the longitudinal study from which the current data were taken. To all the staff and students who helped recruit participants and collect and process the data. This research was supported by a grant from the National Institute of Child Health and Human Development through the National Institutes of Health [5R01 HD075349] to the final author.

Footnotes

The authors have declared that they have no competing or potential conflicts of interest.

References

  1. Alink LR, van Ijzendoorn MH, Bakermans-Kranenburg MJ, Mesman J, Juffer F, & Koot HM (2008). Cortisol and externalizing behavior in children and adolescents: Mixed meta-analytic evidence for the inverse relation of basal cortisol and cortisol reactivity with externalizing behavior. Developmental Psychobiology, 50, 427–450. 10.1002/dev.20300 [DOI] [PubMed] [Google Scholar]
  2. Almas AN, Degnan KA, Walker OL, Radulescu A, Nelson CA, Zeanah CH, & Fox NA (2015).The effects of early institutionalization and foster careintervention on children’s social behaviors at the age of eight. Social Development, 24, 225–239. 10.1111/sode.12089 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ayer L, Greaves-Lord K, Althoff RR, Hudziak JJ, Dieleman GC, Verhulst FC, & van der Ende J (2013). Blunted HPA axis response to stress is related to a persistent dysregulation profile in youth. Biological Psychology, 93(3), 343–351. doi: 10.1016/j.biopsycho.2013.04.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Berry D & Willoughby MT (2016). On the practical interpretability of cross-lagged panel models: Rethinking the developmental workhorse. Child Development, 88, 1186–1206. 10.1111/cdev.12660 [DOI] [PubMed] [Google Scholar]
  5. Bentler PM (1990). Comparative fit indexes in structural models. Psychological bulletin, 107(2), 238. 10.1037/0033-2909.107.2.238 [DOI] [PubMed] [Google Scholar]
  6. Burkholder AR, Koss KJ, Hostinar CE, Johnson AE, & Gunnar MR (2016). Early life stress: Effects on the regulation of anxiety expression in children and adolescents. Social Development, 25(4), 777–793. 10.1111/sode.12170 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Caballo VE, Salazar IC, Irurtia MJ, Arias B, & Hofmann SG (2014). Differences in social anxiety between men and women across 18 countries. Personality and Individual Differences, 64, 35–40. 10.1016/j.paid.2014.02.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Curran PJ, Howard AL, Bainter SA, Lane ST, & McGinley JS (2014). The separation of between-person and within-person components of individual change over time: A latent curve model with structured residuals. Journal of Consulting and Clinical Psychology, 82(5), 879–894. 10.1037/a0035297 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. DePasquale CE, Lawler JM, Koss KJ, & Gunnar MR (2020). Cortisol and parenting predict pathways to disinhibited social engagement and social functioning in previously institutionalized children. Journal of Abnormal Child Psychology, 48, 797–808. 10.1007/s10802-020-00633-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. De Veld D, Riksen-Walraven JM, & de Weerth C (2012). The relation between emotion regulation strategies and physiological stress reactivity in middle childhood. Psychoneuroendocrinology, 37, 1309–19. [DOI] [PubMed] [Google Scholar]
  11. Fekedulegn DB, Andrew ME, Burchfiel CM, Violanti JM, Hartley TA, Charles LE, & Miller DB (2007). Area under the curve and other summary indicators of repeated waking cortisol measurements. Psychosomatic Medicine, 69(7), 651–659. doi: 10.1097/PSY.0b013e31814c405c [DOI] [PubMed] [Google Scholar]
  12. Gunnar MR, DePasquale CE, Reid BM, Donzella B, & Miller BS (2019). Pubertal stress recalibration reverses the effects of early life stress in post-institutionalized children. Proceedings of the National Academy of Sciences of the United States of America, 116(48), p. 23984–23988. 10.1073/pnas.1909699116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gunnar MR, Frenn K, Wewerka SS, & Van Ryzin MJ (2009). Moderate versus severe early life stress: associations with stress reactivity and regulation in 10–12-year-old children. Psychoneuroendocrinology, 34, 62–75. 10.1016/j.psyneuen.2008.08.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gunnar M, Wewerka S, Frenn K, Long J, & Griggs C (2009). Developmental changes in hypothalamus–pituitary–adrenal activity over the transition to adolescence: Normative changes and associations with puberty. Development and Psychopathology, 21(1), 69–85. doi: 10.1017/S0954579409000054 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Guyon-Harris KL, Humphreys KL, Fox NA, Nelson CA, & Zeanah CH (2018). Course of disinhibited social engagement disorder from early childhood to early adolescence. Journal of the American Academy of Child and Adolescent Psychiatry, 57, 329–335. 10.1016/j.jaac.2018.02.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hawk B, & McCall RB (2010). CBCL behavior problems of post-institutionalized international adoptees. Clinical Child and Family Psychology Review, 13(2), 199–211. doi: 10.1007/s10567-010-0068-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hodges J, & Tizard B (1989). Social and family relationships of ex-institutional adolescents. Journal of Child Psychology and Psychiatry, 30,77–97. 10.1111/j.1469-7610.1989.tb00770.x [DOI] [PubMed] [Google Scholar]
  18. Hostinar CE, Johnson AE, & Gunnar MR (2015). Early social deprivation and the social buffering of cortisol stress responses in late childhood: An experimental study. Developmental Psychology, 51(11), 1597–1608. 10.1037/dev0000029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hu LT, & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1–55. DOI: 10.1080/10705519909540118 [DOI] [Google Scholar]
  20. Julian MM, & McCall RB (2016). Social skills in children adopted from sociallyemotionally depriving situations. Adoption Quarterly, 19, 44–62. doi: 10.1080/10926755.2015.1088106 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Katzelnick DJ, & Greist JH (2001). Social anxiety disorder: an unrecognized problem in primary care. Journal of Clinical Psychiatry, 62, 11–15. [PubMed] [Google Scholar]
  22. Kessler RC (2003). The impairments caused by social phobia in the general population: implications for intervention. Acta Psychiatr Scand Suppl, 417, 19–27. doi: 10.1034/j.1600-0447.108.s417.2.x [DOI] [PubMed] [Google Scholar]
  23. Kirschbaum C, Pirke K, & Hellhammer DH (1993). The ‘Trier Social Stress Test’: A tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28, 76–81. doi: 10.1159/000119004 [DOI] [PubMed] [Google Scholar]
  24. Koss KJ, Mliner SB, Donzella B, & Gunnar MR (2016). Early adversity, hypocortisolism, and behavior problems at school entry: A study of internationally adopted children. Psychoneuroendocrinology, 66, 31–38. doi: 10.1016/j.psyneuen.2015.12.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kumsta R, Schlotz W, Golm D, Moser D, Kennedy M, Knights N, Kreppner J, Maughan B, Rutter M, & Edmund S-B (2017). {HPA} axis dysregulation in adult adoptees twenty years after severe institutional deprivation in childhood. Psychoneuroendocrinology, 86, 196–202. 10.1016/j.psyneuen.2017.09.021 [DOI] [PubMed] [Google Scholar]
  26. La Greca AM, Lopez N (1998). Social anxiety among adolescents: Linkages with peer relations and friendships. J Abnorm Child Psychol 26, 83–94. 10.1023/A:1022684520514 [DOI] [PubMed] [Google Scholar]
  27. MacCallum RC, Browne MW, & Sugawara HM (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological methods, 1, 130. 10.1037/1082-989X.1.2.130 [DOI] [Google Scholar]
  28. McLaughlin KA, Sheridan MA, Tibu F, Fox NA, Zeanah CH, & Nelson CA 3rd. (2015). Causal effects of the early caregiving environment on development of stress response systems in children. PNAS, 112(18), 5637–5642. doi: 10.1073/pnas.1423363112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, Benjet C, Georgiades K, Swendsen J (2010). Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry, 49, 980–990. doi: 10.1016/j.jaac.2010.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Miller R, Plesswo F, Kirschbaum C & Stalder T (2013). Classification criteria for distinguishing cortisol responders from non-responders to psychosocial stress: evaluation of salivary cortisol pulse detection in panel designs. Psychosom. Med, 75, pp. 832–840. doi: 10.1097/PSY.0000000000000002. [DOI] [PubMed] [Google Scholar]
  31. Morris MC, Rao U, Wang L, & Garber J (2014). Cortisol reactivity to experimentally manipulated psychosocial stress in young adults at varied risk for depression. Depression and Anxiety, 31, 44–52. doi: 10.1002/da.22125 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ouellet-Morin I, Odgers CL, Danese A, Bowes L, Shakoor S, Papadopoulos AS,...Arseneault L (2011a). Blunted cortisol responses to stress signal social and behavioral problems among maltreated/bullied 12-year-old children. Biological Psychiatry, 70, 1016–1023. 10.1016/j.biopsych.2011.06.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Perry NB, Depasquale C, Donzella B, & Gunnar M (2020) Associations between Stress Reactivity and Behavior Problems for Post-institutionalized Youth across Puberty. Special issue titled “The Developmental Psychobiology of Stress.” Development and Psychopathology, 35, 1854–1863. DOI: 10.1017/S0954579420001297 [DOI] [PubMed] [Google Scholar]
  34. Perry NB, Parenteau AM, Donzella B,C, Desjardins, & Gunnar MR (2019). Emotion regulation and cortisol reactivity during a social evaluative stressor: A study of postinstitutionalized youth. Developmental Psychobiology, 61(4), 557–572. 10.1002/dev.21828 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pitskel NB, Bolling DZ, Kaiser MD, Crowley MJ, & Pelphrey KA (2011). How grossed out are you? The neural bases of emotion regulation from childhood to adolescence. Developmental Cognitive Neuroscience, 1, 324–337. doi: 10.1016/j.dcn.2011.03.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Pitula CE, DePasquale CE, Mliner SB, & Gunnar MR (2014). Peer problems among post-institutionalized, internationally adopted children: Relations to hypocortisolism, parenting quality, and ADHD symptoms. Child Development, 90, 339–355. 10.1111/cdev.12986 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pruessner JC, Kirschbaum C, Meinlschmid G, & Hellhammer DH (2003). Two formulas for computation of the area under the curve represent measures of total hormone concentration versus time-dependent change. Psychoneuroendocrinology, 28(7), 916–931. doi: 10.1016/S0306-4530(02)00108-7 [DOI] [PubMed] [Google Scholar]
  38. R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. [Google Scholar]
  39. Reijntjes A, Kamphuis JH, Prinzie P & Telch MJ (2010). Peer victimization and internalizing problems in children: a meta-analysis of longitudinal studies. Child Abuse and Neglect, 34, 244–252. 10.1016/j.chiabu.2009.07.009 [DOI] [PubMed] [Google Scholar]
  40. Rosseel Y (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2),1–36. URL http://www.jstatsoft.org/v48/i02/. [Google Scholar]
  41. Sonuga-Barke EJ, Kennedy M, Kumsta R, Knights N, Golm D, Rutter M, … Kreppner J (2017). Child-to-adult neurodevelopmental and mental health trajectories after early life deprivation: The young adult follow-up of the longitudinal English and Romanian Adoptees study. The Lancet, 389, 1539–1548. 10.1016/S0140-6736(17)30045-4 [DOI] [PubMed] [Google Scholar]
  42. Stein DJ, Lim CC, Roest P, et al. … WHO World Mental Health Survey Collaborators. (2017). The cross-national epidemiology of social anxiety disorder: Data from the World Mental Health Survey Initiative. BMC, Medicine, 143, 10.1186/s12916-017-0889-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Strüber N, Strüber D, & Roth G (2014). Impact of early adversity on glucocorticoid regulation and later mental health disorders. Neuroscience & Biobehavioral Reviews, 38, 17–27. doi: 10.1016/j.neubiorev.2013.10.015 [DOI] [PubMed] [Google Scholar]
  44. Susman EJ, Dorn LD, Inoff-Germain G, Nottelmann ED, & Chrousos GP (2010). Cortisol reactivity, distress behavior, and behavioral and psychological problems in young adolescents: A longitudinal perspective. Journal of Research on Adolescence, 7, 81–105. doi: 10.1207/s15327795jra0701_5 [DOI] [Google Scholar]
  45. Tarullo AR, Bruce J, & Gunnar MR (2007). False belief and emotion understanding in postinstitutionalized children. Social Development, 16, 57–78. 10.1111/j.1467-9507.2007.00372.x [DOI] [Google Scholar]
  46. Tucker LR, & Lewis C (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10. doi: 10.1037/0033-2909.107.2.238 [DOI] [Google Scholar]
  47. Yim IS, Quas JA, Rush EB, Granger DA, & Skoluda N (2015). Experimental manipulation of the trier social stress test-modified (TSST-C) to vary arousal across development. Psychoneuroendocrinology, 57, 57–61. 10.1016/j.psyneuen.2015.03.021 [DOI] [PubMed] [Google Scholar]

RESOURCES