Skip to main content
PMC home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2022 May 11;47(9):991–1002. doi: 10.1093/jpepsy/jsac037

Assessing Aspects of Social Relationships in Youth Across Middle Childhood and Adolescence: The NIH Toolbox Pediatric Social Relationship Scales

Laura J Dietz 1,✉, Jill M Cyranowski 2, Kaitlyn M Fladeboe 3, Morgen A R Kelly 4, Paul A Pilkonis 5, Zeeshan Butt 6,7, John M Salsman 8, David Cella 9
PMCID: PMC9801708  PMID: 35543247

Abstract

Objective

Social relationships are a critical context for children’s socioemotional development and their quality is closely linked with concurrent and future physical and emotional wellbeing. However, brief self-report measures of social relationship quality that translate across middle childhood, adolescence, and adulthood are lacking, limiting the ability to assess the impact of social relationships on health outcomes over time. To address this gap, this article describes the development and testing of the National Institutes of Health (NIH) Toolbox Pediatric Social Relationship Scales, which were developed in parallel with the previously-reported Adult Social Relationship Scales.

Methods

Item sets were selected from the NIH Toolbox adult self-report item banks in the domains of social support, companionship, and social distress, and adapted for use in preadolescent (ages 8–11 years) and adolescent (ages 12–18 years) cohorts. Items were tested across a U.S. community sample of 1,038 youth ages 8–18 years. Classical test and item response theory approaches were used to identify items for inclusion in brief unidimensional scales. Concurrent validity was assessed by comparing resultant scales to established pediatric social relationship instruments.

Results

Internal reliability and concurrent validity were established for five unique scales, with 5–7 items each: Emotional Support, Friendship, Loneliness, Perceived Rejection, and Perceived Hostility.

Conclusions

These brief scales represent developmentally appropriate and valid instruments for assessing the quality of youth social relationships across childhood and adolescence. In conjunction with previously published adult scales, they provide an opportunity for prospective assessment of social relationships across the developmental spectrum.

Keywords: adolescents, measure validation, school-age children, social functioning and peers

Introduction

Social relationships with parents, siblings, and peers provide the foundation for children’s social and emotional development, shaping emergent communication, cognitive and behavioral skills that facilitate prosocial behavior, and the development of social support networks across the lifespan. Supportive relationships across childhood and adolescence are associated with enhanced emotional and behavioral regulation and serve as protective factors for those at risk for internalizing and externalizing disorders. Prospective longitudinal studies show that greater parental support and lower levels of family conflict and dysfunction predict fewer future depressive symptoms (Seeley et al., 2009; Sheeber et al., 1997). As youth age, supportive relationships with peers become increasingly important, and further promote resilience in the face of life stress. For example, supportive peer relationships are associated with positive school attendance and performance in at-risk youth (Moses & Villodas, 2017). Positive childhood experiences, including those with friends, are also associated with adult-reported social and emotional support in spite of adverse childhood experiences (Bethell et al., 2019). In contrast, peer exclusion, rejection, or victimization (e.g., bullying) predict depressive symptoms from middle childhood to early adolescence (Brendgen et al., 2005; Hazel et al., 2014) and suicidality in young adulthood (Copeland et al., 2013). Finally, loneliness and poor peer connectedness during adolescence similarly portend internalizing symptoms in early and middle adulthood (Landstedt et al., 2015).

Despite research linking childhood social relationships to patterns of social relationships and emotional outcomes later in life, few prospective studies have tracked key aspects of social relationship quality across the lifespan. This dearth of prospective social relationship research may relate, in part, to the lack of brief, developmentally-appropriate scales that can assess key social relationship constructs across child, adolescent, and adult populations (Hazel et al., 2014). Despite the extensive literature on the assessment of specific social relationships across childhood, including parental bonding, sociometric ratings by peers, and various parental and teacher-reported behavioral indices (Bevans et al., 2017), few self-report instruments have been developed to assess youth’s global perceptions of their social relationship quality that may, moreover, track across the lifespan. Given the health risks associated with loneliness (Cacioppo et al., 2000, 2002) and social distress (Hooley & Gotlib, 2000; Kiecolt-Glaser et al., 2010), and the salubrious, stress-resilience impacts of positive social supports (Holt-Lunstad et al., 2010; Uchino et al., 2018), the ability to assess such constructs is of particular importance for health psychologists. Evaluating the quality and patterns of social relationships in childhood and adolescence may be particularly important to understanding how these psychosocial factors contribute to health risks in adulthood, as pathways for disease progression, biological and behavioral, often begin during these early, developmental periods (Jopling et al., 2021; Matthews, 2005). We hypothesized that we could develop a set of brief, unidimensional and psychometrically robust scales utilizing a common set of items measuring perceived social relationship quality across samples of preadolescents and adolescents that paralleled constructs included in our National Institutes of Health (NIH) Toolbox Adult Social Relationship scales. The current report describes the development and testing of the NIH Toolbox Pediatric Social Relationship scales, developed as part of the NIH Toolbox Social Relationship Assessment Battery.

Social Relationship Assessment within the NIH Toolbox Project

Included as one of the four emotional health subdomains within the NIH Toolbox for the Assessment of Neurological and Behavioral Function (see Salsman et al., 2013), the NIH Toolbox Social Relationship Scales were developed as brief self-report measures designed to assess key aspects of social relationships across the developmental spectrum. Literature searches were initiated in February 2007 to identify conceptually-relevant pediatric and adult self-report scales relating to global perceptions of social relationship quality using Health and Psychosocial Instruments (HaPI), PsycINFO, and Medline databases. Search strategies crossed self-report assessment terms with content-relevant search terms related to both positive and negative aspects of social relationship quality. Abstracts (658 in all, including 234 in HaPI, 178 in PsycINFO, and 246 in Medline) were reviewed by two domain team coauthors (J.M.C. and M.A.R.K.), with additional articles identified from citation searches of key papers.

Based on the above literature review, as well as considerations relevant to NIH Toolbox project goals (i.e., assessment of constructs associated with emotional and physical health outcomes across diverse samples in the general population), three social relationship domains were identified for inclusion: social support, companionship, and social distress. Given the breadth and multidimensional nature of these domains, we sought to identify and include items representing key components related to each domain. This process was guided by the conceptual literature and item-level instrument review, with the goal of developing a set of unidimensional scales that were amenable to item response theory (IRT) analyses and provided reasonable conceptual coverage of social relationship constructs that could be measured across the lifespan.

Social Support

Social support was conceptualized as the extent to which an individual views social relationships as available to provide aid in times of need or when problems arise (Cohen, 2004). Three types of support were included: Instrumental Support (perception that people in one’s social network are available to provide material or functional aid); Informational Support (perception that people in one’s social network are available to provide information or advice needed to solve problems), and Emotional Support (perception that people in one’s social network are available to provide empathy and caring) (Cohen, 2004; House & Kahn, 1985). Notably, a recent review of 31 studies of social support in youth indicated that each utilized unique scales, less than half of which were validated (Gariépy et al., 2016). This review highlights the need for a standard set of validated pediatric assessment instruments in this domain.

Companionship

Much of the research measuring companionship in youth has focused on (a) the extent or quality of sibling or peer relationships (e.g., Samek et al., 2015) or (b) or feelings of loneliness (e.g., Asher et al., 1984). To parallel adult scales, we refined this definition of companionship to include global perceptions regarding Friendship (the availability of companions or friends with whom to affiliate or interact), Intimacy (the availability of people with whom one feels emotionally close), and Loneliness (the perception that one is alone, lonely or isolated from others).

Social Distress

Social distress was defined as the extent to which an individual perceives social interactions as negative or distressing, including the frequency with which people in one’s life behave in ways perceived as: Hostile/Critical (e.g., how often people argue with me, yell at me), Insensitive/Neglectful (e.g., how often people don’t listen when I ask for help); Rejecting/Ridiculing (e.g., how often people act like they don’t like me, or make fun of me), and Intrusive (e.g., how often people tell me what to do, or boss me around).

Methods

Item Pool Development for Pediatric Self-Report Scales

Given the goal of developing brief social relationship self-report scales with utility across developmental periods, an item pool relevant for both preadolescent (age 8–12 years) and adolescent (age 13–17 years) cohorts was developed in conjunction with adult self-report scales (see Cyranowski et al., 2013). Published self-report scales (including over 50 that could be used without payment of royalties) were reviewed for conceptual coverage and item content. Items were developed to reflect component definitions, maximize conceptual coverage, minimize item redundancy, and optimize utility across a wide age range of respondents. All items were reviewed by pediatric, cross-cultural, and Spanish-language panels to identify suitability for use across diverse age and cultural populations and to facilitate Spanish translation (see Victorson, 2013). This resulted in a pool of 97 Adult Social Relationship items administered for initial survey testing, from which a subset of 43 items were drawn to be tested in pediatric samples—in order to avoid taxing the comprehension, reading speed, or assessment burden of younger participants. Pediatric items were selected to maximize content relevance, developmental appropriateness, and comprehension in younger cohorts, while providing adequate coverage of conceptual subdomains. Of the 43 items selected for initial pediatric testing, 12 reflected social support, 15 companionship, and 16 social distress. For each item, respondents rate the frequency of their feelings or experiences over a past month time frame, on a 5-point Likert scale ranging from never (1), rarely (2), sometimes (3), usually (4), and always (5) (see Appendix).

Overview of Initial Psychometric Testing

Subject and Methods

A community sample of 1,038 youth (536 children aged 8–12 and 502 adolescents aged 13–18 years) was recruited by Toluna, an internet survey company. Based on U.S. census data, quota sampling was used to obtain a nationally representative sample with regard to race and gender across the two age strata (see http://www.toluna-group.com for detail). Participants completed NIH Toolbox surveys, validation scales, and a demographic questionnaire. Following survey completion, participants were eligible for prize or incentive-based compensation. This study was approved by the internal review board at the Evanston Northwestern Healthcare Research Institute (EH06-201, NIH Toolbox for Assessment of Neurological and Behavioral Function). The data underlying this article are available in https://dataverse.harvard.edu/dataverse/HealthMeasures, or are available upon request.

Psychometric Analyses and Item Selection Procedures

Classical test theory and IRT methods were used to evaluate item and scale properties. IRT is a class of psychometric techniques in which the probability of selecting each item-response category for each item is modeled as a function of a latent trait of interest. A practical application based on this property is the ability to create short scales that provide adequate precision across the range of the latent construct of interest.

Item testing and selection for both the Adult and Pediatric Social Relationship Scales occurred over a series of stages including initial item level evaluation, dimensionality, and IRT analyses, and final item selection. For the initial item level evaluation, we considered excluding items with response option categories endorsed by fewer than five people or inversions in mean total scores when comparing people across adjacent response options per item, i.e., lack of monotonicity. Both of these factors adversely affect subsequent IRT analyses. We also evaluated corrected item-total correlations to identify items that were less internally consistent within each domain. No pediatric items were excluded at this stage.

Next, we conducted dimensionality analyses to identify the factor structure of item responses and to determine the number and composition of unidimensional component scales within each Social Relationship domain. For each of the two (preadolescent and adolescent) samples, we randomly selected half of the sample for exploratory factor analyses (EFA) in Mplus version 5 (Muthen & Muthen, 2006) with unweighted least squares estimation and Quartimin rotation to identify underlying factors. We then used confirmatory factor analyses (CFA) with the second half of the sample to confirm the factor structure. Because of the ordinal nature of the data, we used polychoric correlations in factor analyses. Eigenvalues greater than 1.0 and scree plots were used to identify the number of factors. Items that displayed relatively weak factor loadings (i.e., loadings < .40) or cross-loadings across factors in the EFA analyses were considered for exclusion. For subsequent CFAs, we used weighted least squares estimation and fit statistics to assess dimensionality of the item pool. Unidimensionality was primarily confirmed using Comparative Fit Index or CFI (with CFI > .90 representing acceptable fit), although Root Mean Square Error of Approximation or RMSEA fit indices were also calculated (with RMSEA < .08 representing acceptable fit). Notably, our goal at this phase was to identify unidimensional factors utilizing analogous item sets that performed adequately (and similarly) in both the preadolescent and adolescent cohorts.

IRT analyses for identified component item sets were then run separately for each cohort, to further inform item selection (as outlined in Cyranowski et al., 2013). To calibrate data, we estimated parameters using Samejima’s graded response model (Samejima, 1969), plotting the information provided by individual items as reported in Multilog (Thissen et al., 2003), and assessing fit to a two-parameter IRT model using the S-X2 fit statistic (Orlando & Thissen, 2003) as implemented in IRTfit (Bjorner et al., 2006). Items with poor fit and poorly discriminating items (those with unacceptable IRT slopes, or IRT slopes < 2), were considered for exclusion. Differential item functioning (DIF) analyses were run on the basis of gender utilizing a logistic regression procedure to identify significant DIF (Choi et al., 2010). DIF analyses were run to ensure that items did not function differentially in males and females in either the preadolescent or adolescent samples. Finally, computerized adaptive testing (CAT) analyses were run to rank order items based on their informational value across CAT simulations. All together, 12 items were eliminated based on dimensionality and IRT analyses, including 5 social support, 3 companionship, and 4 social distress items.

Initial evaluation of concurrent validity. Three brief self-report scales were included in survey testing to evaluate concurrent validity of the NIH Toolbox Pediatric Social Relationship Scales.

Social Support: Monitoring the Future (MF) social support subscale (Johnston et al., 1980). This three-item subscale designed to assess the perceived availability of others to provide help and companionship and used in Monitoring the Future, the annual national survey of U.S. high-school students since 1977 (Johnston et al., 1980), was used to validate the pediatric social support scales. Cronbach’s alpha for this scale was .69 in preadolescent and .72 in adolescent samples.

Companionship: Asher Childhood Loneliness Scale (Asher et al., 1984). This 16-item scale was included to assess children’s feelings of loneliness or social dissatisfaction and has been shown to display good reliability. Cronbach’s alpha for this scale was .95 in both preadolescent and adolescent samples.

Social Distress: National Survey of Children (NSC) subscale (Peterson & Zill, 1986). This three-item scale on arguing with friends and being picked on by peers and older children appeared in the second wave of the National Survey of Children (NSC) (Peterson & Zill, 1986; Zill, 1990), and was used to validate the social distress scales. Cronbach’s alpha for this scale was .78 in the preadolescent and .66 in the adolescent samples.

Results

Demographic information for the pediatric sample is presented in Table I. Results of psychometric analyses and item selection procedures for each of three social relationship domains (social support, companionship, and social distress) are reported below for the separate preadolescent (N = 536) and adolescent (N = 502) samples.

Table I.

Sample Demographics

Preadolescent self-report Adolescent self-report
(N = 536) (N = 502)
Age cohort 8–12 years 13–17 years
Mean age (SD) 10.06 (1.38) 14.98 (1.43)
Sex (% male) 48.3 51.8
Race (%)
 White 82.5 85.9
 African American 11.4 9.6
 Asian 3.2 2.0
 Other 12.5 7.8
Hispanic (%) 12.5 7.8
Open in a new tab

Social Support Scales

Initial Item Evaluation

For the 12 items tested, Cronbach’s alpha was .93 in the preadolescent (8–12 years) cohort (with item-total correlations ranging from .63 to .78) and .91 in the adolescent (13–17 years) cohort (with item-total correlations ranging from .48 to .75). Data for the 12 social support items were negatively skewed, with few responses to items in the low range of scores (typically in the Never and Rarely categories) for both the preadolescent and adolescent cohorts, and required the collapsing of categories for subsequent IRT analyses.

Dimensionality and IRT Results

One factor was identified by the initial EFA of the 12 social support items in the preadolescent cohort, with all items loading > .60 (CFI = .95, RMSEA = .111). Two factors were obtained in the initial EFA for the adolescent cohort using scree plot and eigenvalue criteria, with Factor 1 consisting of eight items reflecting emotional/informational support, and Factor 2 comprised of four items reflecting instrumental support. The two-factor CFA model fit (CFI = .95, RMSEA = .1), while reasonable, was not better than the one-factor CFA model fit (CFI = .97, RMSEA = .08 after removing one item because of local dependence), and Factor 2 of the two-factor model displayed poor fit (CFI = .840, RMSEA = .325), relatively weak factor loadings, and one locally dependent item pair. IRT parameter estimates using all items indicated that these four instrumental support items were poorly discriminating in the adolescent sample, and two of these four items also showed poor discrimination in the preadolescent sample. One additional item displayed poor discrimination in IRT analyses run in the adolescent cohort. Thus, all five items displaying poor discrimination in IRT analyses (i.e., slopes < 2) run in either the preadolescent or adolescent samples were thereby eliminated. Notably, the remaining seven items represented the seven top-ranking items in CAT simulations run in each independent cohort. No misfitting items nor gender-based DIF were identified for either cohort. This left a single composite scale including originally-identified emotional and informational support items, paralleling results obtained in our adult cohort (see Cyranowski et al., 2013). This factor was therefore labeled Emotional Support. Of these seven items, four represented sister items of the adult Emotional Support scale. See Appendix for scale items and Table II for psychometric scale properties.

Table II.

Psychometric Properties for NIH Toolbox Pediatric Social Relationship Scales

# items Scale range (min–max) Mean (SD) Cronbach’s alpha CFI RMSEA
Preadolescent self-report
 Social Support Scale
  Emotional Support 7 7–35 31.11 (4.41) .913 .950 .111
 Companionship Scales
  Friendship 5 5–25 20.30 (3.96) .866 .995 .080
  Loneliness 7 7–35 12.43 (5.36) .926 .954 .168
 Social Distress Scales
  Perceived Rejection 7 7–35 12.67 (4.92) .902 .995 .059
  Perceived Hostility 5 5–25 11.66 (3.58) .878 .979 .119
Adolescent Self-Report
 Social Support Scale
  Emotional Support 7 7–35 30.09 (4.13) .905 .981 .092
 Companionship Scales
  Friendship 5 5–25 20.55 (3.75) .863 .992 .116
  Loneliness 7 7–35 11.49 (3.57) .924 .987 .087
 Social Distress Scales
  Perceived Rejection 7 7–35 12.78 (4.70) .902 .999 .020
  Perceived Hostility 5 5–25 11.49 (3.57) .883 .993 .073
Open in a new tab

Note. All scale scores calculated by summing items rated on a 5-point Likert scale (1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Usually, 5 = Always).

Concurrent Validity

Total scores on the seven-item Emotional Support scale was moderately correlated with the Monitoring the Future Support scale in preadolescent (r = .55, p < .01) and adolescent (r = .68, p < .01) cohorts. As expected, high scores on Emotional Support were also inversely associated with the Asher Childhood Loneliness (ACL) scale and the NSC Peer Distress Subscale in both cohorts (see Table III).

Table III.

Concurrent Validity Information for NIH Toolbox Pediatric Social Relationship Scales

Validation instruments
 NIH Toolbox Pediatric Social Relationship Scales
MF ACL NSC ES FRI LON PR PH
Preadolescent cohort (N = 536)
 MF Social Support
 Asher Childhood Loneliness –.588
 NSC Peer Distress –.222 .535
 Emotional Support (ES) .552 –.519 –.186
 Friendship (FRI) .584 –.779 –.371 .460
 Loneliness (LON) –.563 .810 .494 –.504 –.705
 Perceived Rejection (PR) –.395 .518 .472 –.436 –.384 .512
 Perceived Hostility (PH) –.567 .624 .541 –.486 –.477 .651 .784
Adolescent cohort (N = 502) MF ACL NSC ES FRI LON PR PH
 MF Social Support
 Asher Childhood Loneliness –.643
 NSC Peer Distress –.305 .462
 Emotional Support (ES) .683 –.608 –.305
 Friendship (FRI) .600 –.771 –.333 .512
 Loneliness (LON) –.609 .768 .440 –.549 –.636
 Perceived Rejection (PR) –.551 .628 .469 –.544 –.464 .629
 Perceived Hostility (PH) –.410 .471 .372 –.427 –.316 .470 .718
Open in a new tab

Note. MF = Monitoring the Future social support subscale; ACL = Asher Childhood Loneliness Scale; NSC = National Survey of Children peer distress.

All p values < .01.

Companionship Scales

Initial Item Evaluation

For the 15 items tested, Cronbach’s alpha was .94 and all the corrected item-total correlations exceeded .50 in both preadolescent and adolescent cohorts. Initial item evaluations indicated that 14 of the 15 companionship items displayed sparse data in one or more categories (typically at the Never or Rarely category) for either the preadolescent or adolescent cohort and required the collapsing of categories for IRT analyses.

Dimensionality and IRT Results

Two factors were identified via scree plot and eigenvalue criteria in both the preadolescent and adolescent cohorts. Consistent with results from the NIH Toolbox adult scales for companionship (Cyranowski et al., 2013), we eliminated three items that were associated with the construct of intimacy (e.g., “I get love and affection”), as these displayed relatively weaker primary factor loadings (ranging from .44 to .62 in the preadolescent and .36 to .47 in the adolescent cohort) and higher cross-factor loadings (ranging from .23 to .27 in the preadolescent and .29 to .36 in the adolescent cohort) in initial factor analyses.

Follow-up CFAs on the remaining 12 companionship items conducted on both cohorts indicated that a two-factor model provided a similar if not better fit than the one-factor model (preadolescent cohort: one-factor model CFI = .79, RMSEA = .23 as compared with two-factor model CFI = .88, RMSEA = .14; adolescent cohort: one-factor model CFI = .91, RMSEA = .14 as compared with two-factor model CFI = .913, RMSEA = .13). Across both cohorts, Factor 1 included five items related to friendship, whereas Factor 2 included seven items related to loneliness, which similarly aligned with the NIH Toolbox Adult Friendship and Loneliness scales (see Cyranowski et al., 2013). See Table II for CFI and RMSEA data for the final scales in both pediatric samples.

IRT analyses, run separately for the five friendship items and the seven loneliness items, indicated that slopes were acceptable for all items in the preadolescent cohort, and for all but one item in the adolescent cohort (loneliness item “I feel that there is no one I can go to when I need help,” slope = 1.96). All items were deemed to display adequate IRT model fit across the two age cohorts, however, and were retained in the final scales. None of the retained items displayed gender-based DIF. All five of the Friendship items, and four of the seven Loneliness items represented sister items from the adult companionship scales (see Appendix). See Table II for psychometric properties of the pediatric Friendship and Loneliness scales.

Concurrent Validity

Table III summarizes the Pearson correlations between the Friendship and Loneliness scales and the three validation instruments. Scores on the Friendship and Loneliness scales were negatively correlated but not redundant in preadolescent and adolescent samples (rs = –.71 and –.64, respectively, ps < .01). Friendship scale scores were negatively correlated with the ACL scale (rs = –.78 and –.77) and moderately associated with the Monitoring the Future Social Support subscale (rs = .58 and .60) for the preadolescent and adolescent samples. Loneliness scale scores were strongly correlated with the ACL scale in both samples (rs = .81 and .77).

Social Distress Scales

Initial Item Evaluation

For the 16 items tested, Cronbach’s alpha was .92 and .93 for preadolescent and adolescent cohorts. With the exception of two items, all corrected item-total correlations exceeded .60 in the preadolescent cohort, and with the exception of one item, all of the corrected item-total correlations exceeded .40 in the adolescent cohort. Initial item evaluations indicated that 9 of the 16 social distress items displayed sparse data in one or more categories for either age cohort and required the collapsing of categories for IRT analyses.

Dimensionality and IRT Results

Consistent with results from the NIH Toolbox adult scales for social distress (Cyranowski et al., 2013), we eliminated three items that were associated with the construct of intrusiveness (e.g., “people in my life try to give me more help than I need”), as these displayed relatively weaker primary loadings (ranging from .23 to .66 in the preadolescent and .32 to .68 in the adolescent cohort) and higher cross-loadings (ranging from .15 to .23 in the preadolescent and .01 to .25 in the adolescent cohort) in initial factor analyses. EFAs run on split samples with the remaining 13 items indicated a two-factor solution in both cohorts; for preadolescents, CFI = .989 and RMSEA = .066 with marginal local dependence between two hostility items; for adolescents, CFI = .973 and RMSEA = .095, with no local dependence. For both cohorts, Factor 1 consisted of items associated with perceived rejection, insensitivity or ridicule, and labeled Perceived Rejection and Factor 2 included items reflecting Perceived Hostility. One item displayed a differential pattern of factor loadings across the two age cohorts (e.g., “people in my life don’t listen when I ask for help,” which showed primary loadings on the Perceived Hostility factor for preadolescents and the Perceived Rejection factor for adolescents) and was thus eliminated from the final pediatric scale.

IRT analyses, run separately for the seven perceived rejection and five perceived hostility item sets, indicated that all items provided acceptable levels of information across the scale range and acceptable fit to the IRT model across both age cohorts. None of the retained items displayed gender-based DIF. Of the seven items included in the pediatric Perceived Rejection scale, three are the same as adult Perceived Rejection scale items, and one is sister to an adult Perceived Hostility scale item. Of the five items included in the pediatric Perceived Hostility scale, all five have sister items in the adult Perceived Hostility scale. See Appendix for final scale items and Table II for psychometric properties of each scale.

Concurrent Validity

The pediatric Perceived Rejection and Perceived Hostility scales were correlated but not redundant (r = .78 for the preadolescent cohort; r = .72 for the adolescent cohort). Table III outlines the correlations between the social distress scales and the NSC Peer Distress and ACL scales. Perceived Rejection scores were moderately correlated with the NSC in both samples (rs= .47, p < .01), and with the ACL in preadolescent and adolescent samples (rs = .52 and .63, ps < .01). The Perceived Hostility subscale correlated moderately with NSC and ACL scales in the preadolescent cohort (rs = .54 and .62) and more weakly in the adolescent cohort (rs = .37 and .47, ps < .01).

Discussion

Prospective research on the quality of children’s and adolescents’ social relationships has been limited by the lack of brief, validated self-report measures that can be used across developmental periods. The NIH Toolbox Project has developed social relationship measures that capture key concepts of social support, companionship, and social distress in children and adolescents. Significant effort was devoted to identifying and defining concepts that were relevant to social relationships across the lifespan, and to developing pools of self-report items for respondents aged 8 through adulthood. The development and psychometric properties of the NIH Toolbox Adult Social Relationship scales have been described previously (Cyranowski et al., 2013). The current report focuses on NIH Toolbox item sets tested in large youth cohorts at two developmental periods—preadolescence (ages 8–12 years) and adolescence (ages 13–18 years). IRT modeling allowed us to evaluate the performance of individual items in both cohorts, and to select the items that provided maximal information about the underlying constructs of interest. The resulting self-report scales (5–7 items) provide reliable and valid assessment of latent constructs related to emotional support, friendship, loneliness, perceived rejection, and perceived hostility in children and adolescents.

The Emotional Support scale includes items related to having someone who understands one’s concerns, someone to talk with about school difficulties or having “a bad day,” and someone to provide help or advice about problems. While soliciting advice from someone was conceptualized initially as “informational support,” factor analyses indicated that these advice-related items loaded with emotional support items, mirroring findings in the adult sample (Cyranowski et al., 2013). In contrast, items that clustered to create a separate Instrumental Support scale in adults (i.e., availability of others who assist with daily tasks such as meal preparation, transportation, and cleaning) did not form a coherent factor in the child and adolescent cohorts. The lack of an instrumental support factor in children and adolescents may reflect the dependent role youth have on caregivers for such instrumental activities (and thus a lack of sufficient variability).

Items reflecting Friendship and Loneliness emerged as significant dimensions of the construct of companionship in children and adolescents. As with the NIH Toolbox Adult Social Relationship Scales, friendship and loneliness may emerge as separate but related factors due to method variance, with Friendship items worded positively (“I get invited to do things with other people,” “I have someone to sit with at lunch,” and “I feel like I have lots of friends”) and Loneliness items worded negatively (“I feel alone and apart from others,” “I feel left out,” and “I feel like I don’t have any friends”). Given that these constructs consistently emerged as independent factors, it is also possible that Friendship and Loneliness represent somewhat different aspects of companionship that independently contribute to the overall assessment of youth social relationships. This paradox of connectedness is especially important to study in preadolescence and adolescents, who often negotiate social relationships through social media and on-line forums. Recent studies suggest the relationship between high use of social media and loneliness in preteens and teens, suggesting that high degrees of connectedness with friends and loneliness may co-exist, and that not all types of friendships alleviate loneliness in youth (Fardouly et al., 2018; Shensa et al., 2017).

Children’s and adolescents’ report of Perceived Rejection and Perceived Hostility emerged as significant indicators of social distress in relationships. Perceived Rejection items captured both the experience of invalidation (i.e., “people … act like my problems aren’t that important, act like they don’t care about me, act like they don’t have time for me”) as well as ridicule (i.e., “people … put me down, tease me in a mean way, make fun of me”). The items retained for the Perceived Hostility scale focused on youths’ experience of anger and criticism directed at them in social relationships (i.e., “people … yell at me, get mad at me, blame me when things go wrong”). Items reflecting perceived insensitivity/neglect and intrusiveness in youth social relationships did not yield factors that independently contributed to the construct of social distress. It is worth noting that questions about perceived insensitivity and neglect may have overlapped with those addressing perceived rejection (i.e., “people … don’t pay attention to me, don’t listen to me when I ask for help”). With regard to intrusiveness, youth may expect a high level of control and interruption in their daily interactions with parents and other adults and do not always experience it as distressing. However, in general, the constructs for social distress in the child and adolescent cohorts were comparable to those found in adults, suggesting a degree of continuity across indices of social distress across childhood, adolescence, and adulthood. This finding is consistent with prospective longitudinal research suggesting rejection and hostility in youth social relationship are significant predictors of problematic outcomes at later developmental periods. Peer exclusion and rejection are consistent predictors of high depressive symptoms from middle childhood to early adolescence and may have long-term, negative effects on young adults’ mental health (Brendgen et al., 2005; Copeland et al., 2013; Hazel et al., 2014). Depressive symptoms in adolescents have been associated with cardiovascular health risk markers (Dietz & Matthews, 2011), and adverse events in childhood, including lack of social support and mental health issues, are associated with negative physical health indices in adulthood (Jakubowski et al., 2018).

Several limitations in our instrument development and assessment of the NIH Toolbox Pediatric Social Relationship Scales are worth noting. First, despite using a multi-site design and including geographic areas with higher percentages of racial and ethnic minorities, these groups were relatively under-represented in the current sample. Further research will need to inform the possibility of cultural differences in assessing social relationships across under-represented groups of racial and ethnic minority youth, as well as the potential for DIF that may occur across minoritized samples. Second, the cross-sectional nature of the study and the use of a single-report for youth participating in data collection prevent an understanding of how social relationships may change over the preadolescent and adolescent periods and may obscure developmental differences across age-groups. Finally, these scales have not been administered to clinical cohorts of youth with impaired interpersonal functioning or social relationships due to depression, anxiety, or other mental health disorders. Future studies are required to determine whether these social relationship scales distinguish between community and clinical populations and capture known-group differences in interpersonal relationships and functioning.

The NIH Toolbox Pediatric Social Relationship Scales, developed with both IRT and classical test theory, provide a reliable and valid method of indexing aspects of social relationship quality in youth across two developmental periods, preadolescence and adolescence. These scales are brief, precise, and among the few in the public domain that afford assessment of social support, companionship, and social distress in youth aged 8–17 years. Further research is necessary to establish the construct validity and utility of these scales with diverse community and clinical populations, and the sensitivity of these scales for measuring change in social relationships in clinical intervention trials. However, the NIH Toolbox Pediatric Social Relationship Scales provide a promising means of indexing social relationships across key periods of social and emotional development in youth.

Funding

This work was supported by Federal funds from the Blueprint for Neuroscience Research and the Basic Behavioral and Social Science Opportunity Network (OppNet), National Institutes of Health (NIH) under Contract No. HHS-N-260-2006-00007-C, and by NIH grants MH085874 (J.M.C.), U01 AR052155 (P.A.P.), and KL2RR0254740 (Z.B.).

Conflicts of interest: None declared.

Appendix. Self-Report Scale Items for Pediatric (Age 8–18 Years) NIH Toolbox Pediatric Social Relationship Scales

Appendix. Self-report scale items for pediatric (age 8-18) NIH Toolbox Pediatric Social Relationship scalesInstruction Set for Social Support Items (items should be randomly ordered): In the past month, please rate how often…

Emotional Support
* 1. I have someone who understands my problems
* 2. I have someone who will listen to me when I need to talk
* 3. I have someone to talk with when I have a bad day
** 4. There is someone around to help me if I need it
* 5. I can get helpful advice from others when dealing with a problem
6. I get useful advice about important things in my life
7. I have someone to talk with about school problems
Instruction Set for Companionship Items (items should be randomly ordered): In the past month, please rate how often…
Friendship
* 1. I get invited to go out and do things with other people
* 2. I can find a friend when I need one
* 3. I feel like I have lots of friends
* 4. I have friends to sit with at lunch
* 5. I feel like I'm part of a group of friends
Loneliness
1. I feel that I have nobody to talk to
* 2. I feel alone and apart from others
* 3. I feel left out
4. I feel that I don't have any friends
5. I feel that there is no one I can go to when I need help
* 6. I feel alone
* 7. I feel lonely
Instruction Set for Social Distress Items (items should be randomly ordered): In the past month, please rate how often people in your life…
Perceived Rejection
* 1. Act like my problems aren't that important
* 2. Act like they don't have time for me
* 3. Act like they don't care about me
4. Put me down
5. Make me feel like I don't fit in
*** 6. Tease me in a mean way
7. Make fun of me
Perceived Hostility
* 1. Argue with me
* 2. Act in an angry way toward me
* 3. Yell at me
* 4. Get mad at me
* 5. Blame me when things go wrong
Open in a new tab

Note. All items should be rated using the following 5-point scale: 1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Usually, 5= Always.

*

Sister item in parallel adult short form scale.

**

Sister item included in adult version of Instrumental Support scale.

***

Sister item included in adult version of Perceived Hostility scale.

Contributor Information

Laura J Dietz, Department of Rehabilitation Science and Technology, School of Health & Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA 15213, USA.

Jill M Cyranowski, Department of Psychology, University of Pittsburgh, Pittsburgh, PA 15213, USA.

Kaitlyn M Fladeboe, Department of Pediatrics, University of Washington School of Medicine, Seattle, WA 98195, USA.

Morgen A R Kelly, Veterans Administration (VA) Pittsburgh Healthcare System, Pittsburgh, PA 15240, USA.

Paul A Pilkonis, Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA.

Zeeshan Butt, Phreesia, Inc, Raleigh,, NC 27601, USA; Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA.

John M Salsman, Department of Social Sciences & Health Policy, Wake Forrest School of Medicine, Winston-Salem, NC 27101, USA.

David Cella, Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA.

References

  1. Asher S. R., Hymel S., Renshaw P. D. (1984). Loneliness in children. Child Development, 55(4), 1456–1464. 10.2307/1130015 [DOI] [Google Scholar]
  2. Bethell C., Jones J., Gombojav N., Linkenbach J., Sege R. (2019). Positive childhood experiences and adult mental and relational health in a statewide sample: Associations across adverse childhood experiences levels. JAMA Pediatrics, 173(11), e193007. 10.1001/jamapediatrics.2019.3007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bevans K. B., Riley A. W., Landgraf J. M., Carle A. C., Teneralli R. E., Fiese B. H., Meltzer L. J., Ettinger A. K., Becker B. D., Forrest C. B. (2017). Children’s family experiences: Development of the PROMIS® pediatric family relationships measures. Quality of Life Research, 26(11), 3011–3023. 10.1007/s11136-017-1629-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bjorner J. B., Smith K. J., Orlando M., Stone C., Thissen D., Sun X. (2006). IRTFIT: A macro for item fit and local dependence tests under IRT models. Quality Metric, Inc. [Google Scholar]
  5. Brendgen M., Wanner B., Morin A. J., Vitaro F. (2005). Relations with parents and with peers, temperament, and trajectories of depressed mood during early adolescence. Journal of Abnormal Child Psychology, 33(5), 579–594. 10.1007/s10802-005-6739-2 [DOI] [PubMed] [Google Scholar]
  6. Cacioppo J. T., Ernst J. M., Burleson M. H., McClintock M. K., Malarkey W. B., Hawkley L. C., Kowalewski R. B., Paulsen A., Hobson J. A., Hugdahl K., Spiegel D., Berntson G. G. (2000). Lonely traits and concomitant physiological processes: The MacArthur social neuroscience studies. International Journal of Psychophysiology, 35(2–3), 143–154. 10.1016/s0167-8760(99)00049-5 [DOI] [PubMed] [Google Scholar]
  7. Cacioppo J. T., Hawkley L. C., Crawford L. E., Ernst J. M., Burleson M. H., Kowalewski R. B., Malarkey W. B., Van Cauter E., Berntson G. G. (2002). Loneliness and health: Potential mechanisms. Psychosomatic Medicine, 64(3), 407–417. 10.1097/00006842-200205000-00005 [DOI] [PubMed] [Google Scholar]
  8. Choi S. W., Reise S. P., Pilkonis P. A., Hays R. D., Cella D. (2010). Efficiency of static and computer adaptive short forms comparied to full length measures of depressive symptoms. Quality of Life Research, 19(1), 125–136. https://dx.doi.org/10.1007%2Fs11136-009-9560-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cohen S. (2004). Social relationships and health. The American Psychologist, 59(8), 676–684. 10.1037/0003-066x.59.8.676 [DOI] [PubMed] [Google Scholar]
  10. Copeland W. E., Wolke D., Angold A., Costello E. J. (2013). Adult psychiatric outcomes of bullying and being bullied by peers in childhood and adolescence. JAMA Psychiatry, 70(4), 419–426. 10.1001/jamapsychiatry.2013.504 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cyranowski J. M., Zill N., Bode R., Butt Z., Kelly M. A., Pilkonis P. A., Salsman J. M., Cella D. (2013). Assessing social support, companionship, and distress: National Institute of Health (NIH) Toolbox Adult Social Relationship Scales. Health Psychology, 32(3), 293–301. 10.1037/a0028586 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Dietz L. J., Matthews K. A. (2011). Depressive symptoms and subclinical markers of cardiovascular disease in adolescents. Journal of Adolescent Health, 48(6), 579–584. 10.1016/j.jadohealth.2010.09.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fardouly J., Magson N. R., Johnco C. J., Oar E. L., Rapee R. M. (2018). Parental control of the time preadolescents spend on social media: Links with preadolescents’ social media appearance comparisons and mental health. Journal of Youth and Adolescence, 47(7), 1456–1468. 10.1007/s10964-018-0870-1 [DOI] [PubMed] [Google Scholar]
  14. Gariépy G., Honkaniemi H., Quesnel-Vallée A. (2016). Social support and protection from depression: Systematic review of current findings in Western countries. The British Journal of Psychiatry, 209(4), 284–293. 10.1192/bjp.bp.115.169094 [DOI] [PubMed] [Google Scholar]
  15. Hazel N. A., Oppenheimer C. W., Technow J. R., Young J. F., Hankin B. L. (2014). Parent relationship quality buffers against the effect of peer stressors on depressive symptoms from middle childhood to adolescence. Developmental Psychology, 50(8), 2115–2123. 10.1037/a0037192 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Holt-Lunstad J., Smith T. B., Layton J. B. (2010). Social relationships and mortality risk: A meta-analytic review. PLoS Medicine, 7(7), e1000316. 10.1371/journal.pmed.1000316 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hooley J. M., Gotlib I. H. (2000). A diathesis-stress conceptualization of expressed emotion and clinical outcome. Applied and Preventive Psychology, 9(3), 135–151. https://psycnet.apa.org/doi/10.1016/S0962-1849(05)80001-0 [Google Scholar]
  18. House J. S., Kahn R. L. (1985). Measures and concepts of social support. In Cohen S., Syme S. L. (Eds.), Social support and health (pp. 83–108). Academic Press. [Google Scholar]
  19. Jakubowski K. P., Cundiff J. M., Matthews K. A. (2018). Cumulative childhood adversity and adult cardiometabolic disease: A meta-analysis. Health Psychology, 37(8), 701–715. 10.1037/hea0000637 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Johnston L. D., Bachman J. G., O’Malley P. M. (1980). Monitoring the future: questionnaire responses from the nation’s high school seniors 1977. Survey Research Center, Institute for Social Research, The University of Michigan. [Google Scholar]
  21. Jopling E., Rnic K., Tracy A., LeMoult J. (2021). Impact of loneliness on diurnal cortisol in youth. Psychoneuroendocrinology, 132, 105345. 10.1016/j.psyneuen.2021.105345 [DOI] [PubMed] [Google Scholar]
  22. Kiecolt-Glaser J. K., Gouin J. P., Hantsoo L. (2010). Close relationships, inflammation, and health. Neuroscience and Biobehavioral Reviews, 35(1), 33–38. 10.1016/j.neubiorev.2009.09.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Landstedt E., Hammarström A., Winefield H. (2015). How well do parental and peer relationships in adolescence predict health in adulthood? Scandinavian Journal of Public Health, 43(5), 460–468. 10.1177/1403494815576360 [DOI] [PubMed] [Google Scholar]
  24. Matthews K. A. (2005). Psychological perspectives on the development of coronary heart disease. The American Psychologist, 60(8), 783–796. 10.1037/0003-066X.60.8.783 [DOI] [PubMed] [Google Scholar]
  25. Moses J. O., Villodas M. T. (2017). The potential protective role of peer relationships on school engagement in at-risk adolescents. Journal of Youth and Adolescence, 46(11), 2255–2272. 10.1007/s10964-017-0644-1 [DOI] [PubMed] [Google Scholar]
  26. Muthen L. K., Muthen B. O. (2006). Mplus (Version 6). Computer software. Muthen & Muthen. [Google Scholar]
  27. Orlando M., Thissen D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298. 10.1177/0146621603027004004 [DOI] [Google Scholar]
  28. Peterson J. L., Zill N. (1986). Marital disruption, parent-child relationships, and behavior problems in children. Journal of Marriage and the Family, 48(2), 295–307. 10.2307/352397 [DOI] [Google Scholar]
  29. Salsman J. M., Butt Z., Pilkonis P. A., Cyranowski J. M., Zill N., Hendrie H. C., Kupst M. J., Kelly M. A., Bode R. K., Choi S. W., Lai J. S., Griffith J. W., Stoney C. M., Brouwers P., Knox S. S., Cella D. (2013). Emotion assessment using the NIH Toolbox. Neurology, 80(11 Suppl 3), S76–S86. 10.1212/WNL.0b013e3182872e11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Samejima F. (1969). Estimation of ability using a response pattern of graded scores. Psychometrika, 34, 1–97. [Google Scholar]
  31. Samek D. R., Rueter M. A., Keyes M. A., McGue M., Iacono W. G. (2015). Parent involvement, sibling companionship, and adolescent substance use: A longitudinal, genetically informed design. Journal of Family Psychology, 29(4), 614–623. 10.1037/fam0000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Seeley J. R., Stice E., Rohde P. (2009). Screening for depression prevention: Identifying adolescent girls at high risk for future depression. Journal of Abnormal Psychology, 118(1), 161–170. 10.1037/a0014741 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Sheeber L., Hops H., Alpert A., Davis B., Andrews J. (1997). Family support and conflict: Prospective relations to adolescent depression. Journal of Abnormal Child Psychology, 25(4), 333–344. http://www.ncbi.nlm.nih.gov/pubmed/9304449. Retrieved 25 April 2022. [DOI] [PubMed] [Google Scholar]
  34. Shensa A., Escobar-Viera C. G., Sidani J. E., Bowman N. D., Marshal M. P., Primack B. A. (2017). Problematic social media use and depressive symptoms among U.S. young adults: A nationally-representative study. Social Science & Medicine, 182, 150–157. 10.1016/j.socscimed.2017.03.061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Thissen D., Chen W. H., Bock R. D. (2003). MULTILOG (Version 7) [Computer software]. Scientific Software International, Inc. [Google Scholar]
  36. Uchino B. N., Trettevik R., Kent de Grey R. G., Cronan S., Hogan J., Baucom B. (2018). Social support, social integration, and inflammatory cytokines: A meta-analysis. Health Psychology, 37(5), 462–471. 10.1037/hea0000594 [DOI] [PubMed] [Google Scholar]
  37. Victorson D. (2013). Using the NIH Toolbox in special populations: Considerations for assessment of pediatric, geriatric, culturally diverse, non-English-speaking, and disabled individuals. Neurology, 80(11 Suppl 3), S13–S19. https://dx.doi.org/10.1212%2FWNL.0b013e3182872e26 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Zill N. (1990). National survey of children: Wave I, 1976, wave II, 1981, and wave III, 1987 [computer program]. Inter-University Consortium for Political and Social Research. [Google Scholar]

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES