Development and Psychometric Validation of the Pandemic-Related Traumatic Stress Scale for Children and Adults

Abstract

To assess the public health impact of the COVID-19 pandemic on mental health, investigators from the National Institutes of Health Environmental influences on Child Health Outcomes (ECHO) research program developed the Pandemic-Related Traumatic Stress Scale (PTSS). Based on the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) acute stress disorder symptom criteria, the PTSS is designed for adolescent (13–21 years) and adult self-report and caregiver-report on 3–12-year-olds. To evaluate psychometric properties, we used PTSS data collected between April 2020 and August 2021 from non-pregnant adult caregivers (n = 11,483), pregnant/postpartum individuals (n = 1,656), adolescents (n = 1,795), and caregivers reporting on 3–12-year-olds (n = 2,896). We used Mokken scale analysis to examine unidimensionality and reliability, Pearson correlations to evaluate relationships with other relevant variables, and analyses of variance to identify regional, age, and sex differences. Mokken analysis resulted in a moderately strong, unidimensional scale that retained nine of the original 10 items. We detected small to moderate positive associations with depression, anxiety, and general stress, and negative associations with life satisfaction. Adult caregivers had the highest PTSS scores, followed by adolescents, pregnant/postpartum individuals, and children. Caregivers of younger children, females, and older youth had higher PTSS scores compared to caregivers of older children, males, and younger youth, respectively.

At the onset of the COVID-19 pandemic, little was known about the effects of the pandemic itself or mitigation efforts, such as quarantines and school closures, on children’s and caregiver’s mental health (Esposito & Principi, 2020; Golberstein et al., 2020; Torales et al., 2020). Concerns about these effects underscored the need for population-level studies to ascertain the public health implications of living through a pandemic (Brooks et al., 2020; Fegert et al., 2020; Sprang & Silman, 2013). Twenty-four prior studies of the psychological impact of quarantine/isolation after exposure to an infectious disease suggest that quarantine is associated with an increased risk of posttraumatic stress disorder (PTSD; Brooks et al., 2020). In adults and children, this risk was reported to be three to four times higher in those who did versus did not experience a quarantine (Brooks et al., 2020). Since half of the people who experience a clinically significant level of acute stress symptoms go on to develop PTSD (American Psychiatric Association, 2022), investigators from the Environmental influences on Child Health Outcomes (ECHO) program designed the Pandemic-Related Traumatic Stress Scale (PTSS; Margolis et al., 2021) to feasibly evaluate youth and caregiver pandemic-related stress symptoms in the national ECHO cohort. The scale was administered as part of a broader COVID-19 survey evaluating family hardships (e.g., job loss), behavior changes (e.g., more/less sleep), and adaptive (e.g., mindfulness practices) or maladaptive (e.g., substances use) coping strategies.

Living during a pandemic may not be viewed as an inciting event in the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) definition of acute stress disorder or PTSD (American Psychiatric Association, 2022), but the COVID-19 pandemic did present to many as a life-threatening experience—both real and perceived. The highly infectious and deadly virus has been described as eliciting traumatic stress reactions above and beyond the general psychological distress experienced with pandemic-related disruptions to daily life, such as school and work closures (Bridgland et al., 2021; Horesh & Brown, 2020; Kira, 2021). Since March 2020, COVID-19 has infected nearly 80 million people in the United States, leading to an estimated 900,000 hospitalizations, of which 20% required intensive care unit intervention, and taking the lives of approximately 1 million Americans (Kaiser Family Foundation, 2022). Survivors have a one in three chance of experiencing long-term physical, mental, and cognitive consequences (Logue et al., 2021; Taquet et al., 2021). Thus, fear and experiences of contracting the virus, witnessing or putting a loved one at risk, and suffering severe illness or death reflect the life-threatening nature of the pandemic that can result in traumatic stress reactions (Kira, 2021; Kira et al., 2021).

Although little studied, many populations may be susceptible to pandemic-related traumatic stress. Pregnant and postpartum individuals are at increased risk for severe illness and poor birth outcomes from COVID-19 (Woodworth et al., 2020; Zambrano et al., 2020) and may have heightened fears for their own well-being and that of their infant (Basu et al., 2021; King et al., 2021; Preis et al., 2020). Youth represent another vulnerable population, as the pandemic greatly disrupted their daily life with school closures and lack of social interactions outside the immediate family (Creswell et al., 2021; Hertz et al., 2022; Verlenden et al., 2021). Caregivers also faced potentially traumatic experiences, taking on new roles and responsibilities that often conflicted with each other, such as abrupt shifts to working from home while also trying to manage their children’s care center closures or home-schooling, continuing to work outside the home while arranging and managing alternative childcare, or being unable to work because they needed to care for their children (Calear et al., 2022; Chen et al., 2022; Spinelli et al., 2020).

Initial studies suggest up to 50% of adults could be at risk for pandemic-related posttraumatic stress symptoms (Cooke et al., 2020; Czeisler et al., 2020; Xiong et al., 2020). While many studies measure depression, anxiety, and general stress during the pandemic (Loades et al., 2020; Prati & Mancini, 2021), few focus on traumatic reactions. Additionally, the few studies that have examined traumatic stress demonstrate large inconsistencies in prevalence—anywhere from 1% to 96% in adult samples (Hong et al., 2021) and 11% (Chen et al., 2021) to 86% (Hou et al., 2020) in the few available child studies. Moreover, most studies have been conducted outside the United States and only capture the first 6–10 months of the pandemic. Differentiating depressed mood, overall anxiety, and general stress from traumatic stress reactions fills an important gap in knowledge, given the likelihood of PTSD and related consequences stemming from traumatic stress.

Scales measuring pandemic-related traumatic stress across the lifespan were not readily available at the onset of the COVID-19 pandemic. Those that are now available are for adult samples (Cortez et al., 2020; Taylor et al., 2020). While some parent- and adolescent-report questionnaires include individual items querying COVID-19-related stress (Ladouceur, 2020), incorporate general stress and anxiety scales (Adolescent Brain Cognitive Development, 2020), or evaluate COVID-19-related worries and general mental states (Nikolaidis et al., 2021), reliable and psychometrically valid scales of pandemic-related traumatic stress do not exist for pediatric populations (Tambling et al., 2021). Further, extant adult measures combine stressors (e.g., job loss, financial strain, social distancing) with experienced stress and often confound nontraumatic and traumatic stress in the same measure. For example, the COVID-19 Stress Scales (Taylor et al., 2020) and COVID-19 Stressors Scale (Tambling et al., 2021) include resource-related concerns (e.g., sufficient household/personal supplies), financial concerns (e.g., job loss), social stressors (e.g., changes to work and school), and infection-related stressors (e.g., risk of infection) in addition to their psychological impact. Other scales evaluate COVID-19 fears, anxiety, suspicions, and somatic symptoms (Ahorsu et al., 2022; Feng et al., 2020; Qiu et al., 2020); and yet others rely on existing measures that were developed and validated to assess PTSD outside of this remarkable pandemic context (Bridgland et al., 2021; Pedrozo-Pupo et al., 2020).

Here, we investigated the psychometric properties of the PTSS (Margolis et al., 2021), which was designed to capture information assessed in a clinical interview using the DSM-5 acute stress disorder criteria classification (A and B only), but critically, without the requirement of the symptom occurring within 1 month of the exposure. The PTSS evaluates the five criteria classification domains of intrusion, dissociation, avoidance, arousal regulation problems, and negative mood, only the last of which is captured by general distress indicators (American Psychiatric Association, 2022). We investigated if: (a) responses to the PTSS were characterized by a single underlying latent dimension and (b) the scale had comparable structure across diverse populations, including populations particularly vulnerable to the effects of the COVID-19 pandemic and related mitigation strategies (i.e., pregnant/postpartum individuals, adult caregivers, adolescents, children). To establish validity evidence based on relationships with other variables (American Educational Research Association [AERA], the American Psychological Association, the National Council on Measurement in Education, 2014), we hypothesized that higher ratings of pandemic-related traumatic stress would be associated with higher levels of general perceived stress, anxiety, and depressive symptoms and lower levels of life satisfaction. We also explored group differences, asking: (a) does pandemic-related traumatic stress differ by US region? and hypothesizing: (b) postpartum individuals would have higher PTSS scores compared to pregnant individuals, based on studies finding greater traumatic stress levels in the postpartum period following a natural disaster or pandemic (Basu et al., 2021; Molgora & Accordini, 2020); (c) adult caregivers of younger (<5 years) children would have higher PTSS scores compared to those of older children (5–12 years) and adolescents (13–21 years), given prior work suggesting families with young children are more vulnerable to postdisaster trauma (Baker & Cormier, 2014; Brooks et al., 2020); (d) female adult caregivers would have higher PTSS scores compared to male adult caregivers, given research showing females are more likely to experience psychological distress and PTSD symptoms following traumatic events compared to males (Patel et al., 2022; Wang et al., 2020; Yuan et al., 2021); and (e) older and female children/adolescents would have higher PTSS scores compared to younger and male children/adolescents, respectively, based on emerging evidence that older and female youth experienced greater stress during the pandemic than younger and male youth, respectively (Loades et al., 2020; Magson et al., 2021; Marques de Miranda et al., 2020).

Method

Data were collected as part of the National Institutes of Health ECHO program, a large multicohort research consortium comprising 69 existing pediatric longitudinal observational cohorts with approximately 50,000 children and their caregivers from across the United States and Puerto Rico (Blaisdell et al., 2021; Gillman & Blaisdell, 2018). ECHO’s main objective is to understand the impact of early life environmental exposures (e.g., biological, chemical, built, social) on five primary child outcomes: pre-, peri-, and postnatal outcomes (e.g., small for gestational age, preterm birth); obesity; airways-related outcomes (e.g., asthma); neurodevelopment (e.g., cognition, psychopathology); and positive health (e.g., well-being). Here, we include 47 ECHO cohorts with data on pandemic-related stress. Local cohort and central ECHO institutional review boards (IRBs) reviewed all research methods and procedures, and the work of the ECHO Person Reported Outcome Core was overseen by the Northwestern University IRB.

Participants

Data were divided into four distinct participant samples. The child sample included n = 2,896 caregivers reporting on their 3–12-year-old children from 24 ECHO cohorts; the adolescent sample included n = 1,795 13–21-year-olds from 16 ECHO cohorts; the nonpregnant adult caregiver sample included n = 11,483 adult caregivers from 45 ECHO cohorts; and the pregnant/postpartum sample included n = 1,656 pregnant individuals or individuals who recently gave birth (i.e., gave birth after February 29, 2020, but before completing the PTSS) from 26 ECHO cohorts. The samples were mutually exclusive with respect to PTSS response data given the defining characteristics of such samples (i.e., being a 3–12-year-old child, a 13–21-year-old adolescent; a nonpregnant adult caregiver; or a pregnant/postpartum individual). We note there is some (<25%) family overlap across samples where a participant in one sample was related to another participant in a different sample. Specifically, of the individuals in the adult caregiver sample, 22.4% reported on their 3–12-year-old child, 10.8% had an adolescent in the 13–21-year-old adolescent self-report sample, and n = 11 adult caregivers had both a 3–12-year-old and an adolescent. Of the individuals in the pregnant/postpartum sample, <5% reported on their 3–12-year-old child, <.01% had an adolescent in the 13–21-year-old adolescent self-report sample; and no individual had both a 3–12-year-old and an adolescent.

Measures

Measures were collected between April 2020 and August 2021, except for existing sociodemographic data. Given the structure of ECHO and variability in whether and when cohorts collect specific assessments, not all cohorts administered the measures used to assess relationships with other related variables. Therefore, different subsamples provided data for each measure. We included multiple external measures of the same construct to maximize sample size and enhance validity evidence based on relationships with other variables, noting ≤3% within a subsample completed all external measures. Additionally, external measures were not always completed by participants on the exact same day as the PTSS, and we therefore limited these subsamples to participants who completed measures at the same time as or ±30 days from the PTSS items. Some validation measures used a 7-day recall period, but prior work finds little difference in participant response patterns for a 7- versus 30-day recall (Batterham et al., 2019; Lai et al., 2009), justifying the use of this 30-day window. See Supplemental Tables 2–5, for subsample descriptions by external validation measure.

Pandemic-Related Traumatic Stress was measured using the PTSS, which was developed by the senior author (A.E.M.) and a coauthor (M.A.A.), with contributions from other ECHO investigators (T.B., C.B.,N.R.B.,C.K.B.,J.B.H.,K.Z.L.). The scale was developed with the express purpose of measuring an individual’s response to traumatic stress induced by the COVID-19 pandemic and associated social isolation related to mitigation actions. Items were developed based on the DSM-5 acute stress criteria (American Psychiatric Association, 2022), and the scale was designed to have items map onto each of the five DSM-5 acute stress disorder symptom categories: (a) intrusion (e.g., distressing memories and dreams, flashbacks, catastrophized perceptions of expected events or conditions); (b) negative mood (e.g., anhedonia, anger disproportionate to the situation); (c) dissociation (e.g., feelings of time slowing); (d) avoidance (e.g., purposeful efforts to avoid thinking about the event or actions that are not congruent with required realities of persisting threats); and (e) difficulty regulating arousal (e.g., sleep disturbance, irritability, poor concentration). These concepts were written into survey item format based on examples given in the DSM-5 text, and items were discussed by the research team to reach a consensus on item clarity. One item pertaining to negative mood (“I no longer feel happy or satisfied”) was reframed into a positive statement for clarity (“I feel happy and satisfied with my life”). Parallel versions for the parent-report version for children 3–12 years old and self-report version for adolescents (13–21 years) and adult caregivers (18 years and older) were designed to capture response to traumatic stress across the lifespan from childhood to adulthood. Parent-report items were modified to be less inferential than the self-report version (e.g., instead of “I felt in a daze,” the item was reworded to “My child was in a daze”). Items were reviewed, revised, and finalized by the broader ECHO COVID-19 Working Group, which included experts in clinical and developmental psychology, neuropsychology, pediatrics, acute and posttraumatic stress disorders, epidemiology, and measurement science. Items were asked in the context of, “Since becoming aware of the COVID-19 pandemic,” and a 5-point Likert response scale was used for each item: (1) not at all, (2) rarely, (3) sometimes, (4) often, and (5) very often.

Sociodemographics included adult and child age (continuous), sex (1 = female), race (American Indian or Alaska Native, Asian, Black, Native Hawaiian or other Pacific Islander, White, multiracial, and self- or parent-reported “other race”) and ethnicity (1 = Hispanic); pregnant/postpartum individual or adult caregiver highest educational attainment (less than high school; high school degree or General Educational Development certificate; some college, no degree, or associate’s degree; bachelor’s degree; master’s, professional, or doctorate degree); annual family income (<$30,000, $30–49,999, $50–74,999, $75–99,999, $100–199,999, $200,000 or more); and current/last known state of residence (U.S. 50 states, Washington, DC, and Puerto Rico), which was also aggregated into region of residence, defined as the four U.S. Census regions (West, Midwest, South, and Northeast) and Puerto Rico. Because income is time variant, we restricted data to responses provided within 5 years of the pandemic-related stress items; if multiple responses were provided, the last known income was selected. For participants without residential address information, we used the state of their cohort recruitment site. Sociodemographic variables had acceptable missingness rates <10% (Bennett, 2001), except for caregiver educational attainment and annual family income, where missingness was >20% for some subgroup samples. We did not therefore investigate differences by these variables to avoid biased results.

Measures to Evaluate Relationships With Other Variables

Perceived Stress was measured using the Perceived Stress Scale 10- or four-item (PSS-10, PSS-4; Cohen & Janicki-Deverts, 2012; Kupst et al., 2015; Salsman et al., 2013) for pregnant/postpartum individuals and adult caregivers, which were harmonized by treating the six other items as missing for PSS-4; the PSS-10 (Kupst et al., 2015) or Patient Reported Measurement Information System (PROMIS) Pediatric Psychological Stress Experiences Short Form 4a (PPSE_Ped) for adolescents (Bevans et al.,2018); and the PROMIS Parent Proxy Psychological Stress Experiences Short Form 4a (PPSE_PP) for children (Bevans et al., 2018). All data were scored using the item response theory (IRT) based T-score metric (M = 50, SD = 10). PSS data were available for n = 999 (60.3%) pregnant/postpartum individuals, n = 4,365 (38%) adult caregivers, and n = 210 (11.7%) adolescents. PPSE_Ped data were available for n = 259 (14.4%) adolescents and PPSE_PP for n = 377 (13%) children.

Anxiety was measured using the PROMIS Anxiety Short Form 8a (pregnant/postpartum individuals and adult caregivers; Pilkonis et al., 2011), PROMIS Pediatric Anxiety Short Form 8a (adolescents; Irwin et al., 2010; Quinn et al., 2014), and the PROMIS Parent Proxy Anxiety Short Form 8a (children; Irwin et al., 2012). Instruments were scored using the PROMIS T-score metric (M = 50, SD = 10), and data were available for n = 56 (3.4%) pregnant/postpartum individuals, n = 530 (4.6%) adult caregivers, n = 74 adolescents (4.1%), and n = 297 (10.3%) children. We also used T -scoresfrom the parent-report Child Behavior Checklist–Preschool (CBCL-Pre; 1.5–5 years) or School Age (CBCL-Sch; 6–18 years) DSM-oriented Anxiety subscales (Achenbach & Rescorla, 2000, 2001). For the 3–12-year-old child sample, CBCL-Pre data were available for n = 149 (5.2%) children, and CBCL-Sch data were available for n = 308 (10.6%) children. For the adolescent sample, data on the CBCL-Sch were available for n = 493 (27.3%) adolescents.

Depressive Symptoms were measured with the PROMIS Depression Short Form 8a (pregnant/postpartum individuals and adult caregivers; Pilkonis et al., 2011), PROMIS Pediatric Depressive Symptoms Short Form 8a (adolescents; Irwin et al., 2010; Quinn et al., 2014), and the PROMIS Parent Proxy Depressive Symptoms Short Form 6a (children; Irwin et al., 2012). Instruments were scored using the PROMIS T score metric (M = 50, SD = 10), and data were available for n = 644 (38.9%) pregnant/postpartum individuals, n = 3,715 (32.4%) adult caregivers, n = 553 (30.8%) adolescents, and n = 49 (16.9%) children. We also used the CBCL DSM-oriented Depression subscale T scores (M = 50, SD = 10) for children and adolescents, which included the same sample sizes as described above.

Life Satisfaction was measured with a single-item indicator derived from the PROMIS Life Satisfaction instrument (Forrest et al., 2018). The item asked how often the child/adolescent or participant seemed/felt happy and satisfied with their life since becoming aware of the COVID-19 pandemic. A 5-point Likert scale response option was used, anchored by (1) not at all and (5) very often. The same item was used across all subgroup samples. See Supplemental Materials Methods, for additional details on measures to assess relationships with other variables.

Analytic Procedure

To psychometrically evaluate the PTSS items as a unidimensional scale, we used Mokken scaling, a nonparametric IRT-based method that describes the extent to which a set of items conform to an underlying unidimensional latent trait (Mokken, 2011). Mokken scaling does not make assumptions about item response functions like other factor analytic and parametric methods and can be used in an exploratory manner to identify the optimal set from an item pool that constitutes a monotonic, homogeneous IRT model (Molenaar & Sijtsma, 2000; Sijtsma & Molenaar, 2002). When assumptions hold, Mokken scaling results in an interpretable total score, wherein higher total scores correspond to higher values on the latent trait. We investigated whether there was evidence for the PTSS that followed the monotone homogeneity model (MHM) or the double monotonicity model (DMM) with invariant item ordering (IIO) using H_i and H^T to evaluate the extent to which the PTSS fit these models (Sijtsma et al., 2011). We used the automated item selection procedure (AISP) algorithm (Mokken, 2011; Molenaar & Sijtsma, 2000; Sijtsma & Molenaar, 2002). Items were only considered for removal based on theoretical justification and/or if they had significantly lower H_i values compared to other scale items (Crişan et al., 2020; Stochl et al., 2012). A minimum H_i = .3 was chosen as the floor for item scalability based on previous work suggesting the following thresholds: weak (.3–.4); moderate (.4–.5); and strong (.5–.1; Sijtsma & Molenaar, 2002). The same cutoff values were used for H^T (Sijtsma & van der Ark, 2017), which pertain to the degree of IIO sufficiency where positive values provide evidence for IIO (Ligtvoet et al., 2011). Cronbach’s α was estimated to investigate internal scale reliability (Cronbach, 1951).

We conducted Mokken scaling using complete cases for each participant sample to identify whether PTSS items performed similarly across these different samples, an indication of scale generalizability across ages, reporters, and subgroup populations (Sijtsma et al., 2011). Differential item functioning (DIF; Zumbo, 1999) was investigated for final scale items within each of the four participant samples by caregiver educational attainment using the ordinal logistic regression function in the lordif R package (Choi et al., 2011). We estimated the magnitude of uniform and nonuniform DIF using McFadden’s pseudo-R² standard criteria: negligible (< .13), moderate (.13–26), and large (> .26; McFadden, 1974).

To evaluate differences in PTSS scores by region, pregnant versus postpartum, caregiver sex, and child/adolescent age and sex, we conducted known-group differences analyses using one-way analysis of variance (ANOVA). We used η² to evaluate effect size using standard criteria: small (.01–.05), medium (.06–.14), and large (> .14; Cohen, 1988). We examined associations between PTSS sum scores and external, well-validated measures using Pearson r correlations to evaluate the relations with other variables hypothesized to be measuring constructs related to but not the same as pandemic-related traumatic stress (i.e., general stress, depression, anxiety, life satisfaction; AERA et al., 2014). We evaluated the strength of these correlations using standard criteria for evaluating correlation magnitude: small (.2), moderate (.5), and large (.8; Cohen, 1988).

Procedures to replicate these analyses are available upon request. Aside from the CBCL, all measures used in the analyses are freely available to download from the ECHOchildren.org website. This study—including its design, hypotheses, and analysis plan—was part of the ongoing ECHO program and not preregistered. The ECHO public use data set is available in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Data and Specimen Hub.

Results

Participants

Participants came from all 50 states, Washington, DC, and Puerto Rico (see Supplemental Figure 1, for regional distributions by sample). Participant details including demographics such as age, race, ethnicity, educational attainment, and income, by sample, are provided in Table 1. Briefly, the pregnant/postpartum sample self-identified as White (63.3%), Black (9.2%), American Indian or Alaska Native (4.1%), Asian (2.5%), multiracial (4.8%), and “other race” (2.5%); 18.6% were Hispanic; and 32.8% had some college or less. The adult caregiver sample (98.7% female) self-identified as White (65.2%), Black (14.4%), Asian (4.4%), American Indian or Alaska Native (1.9%), multiracial (3.8%), and “other race” (2.3%); 14.4% were Hispanic; and 40.1% had some college or less. The adolescent sample (53.8% female) was caregiver-identified as White (58%), Black (26.7%), Asian (1.7%), and multiracial (5.1%); 14.1% were Hispanic; and 41.6% had caregivers with some college or less. The child sample (48.6% female) was caregiver-identified as White (75%), Black (6.9%), American Indian or Alaska Native (4.3%), Asian (1.8%), and multiracial (8.2%); 11.8% were Hispanic; and 33.1% had caregivers with some college or less. Compared to the general U.S. population, there were higher proportions of White individuals and lower proportions of Hispanic individuals, higher proportions of individuals with bachelor’s degrees or higher, and slightly lower proportions of individuals with household incomes at or above $100,000 (see Supplemental Tables 6 and 7, for complete demographic comparisons with the general U.S. population).

Table 1.

Participant Sociodemographics by Subgroup Sample

Sociodemographic characteristics	Pregnant/postpartum
	All (n = 1,656)		Pregnant (n = 1,036)		Postpartum (n = 620)		Adult (n = 11,483)		Adolescent (n = 1,785)		Child (n = 2,896)
	n	%	n	%	n	%	n	%	n	%	n	%
	n cohorts	30		27		24		45		11		24
Child age, M (SD)							7.65 (4.95)		16.9 (1.89)		7.45 (2.5)
Caregiver age, M (SD)	29.8 (5.36)		28.12 (5.67)		30.52 (5.05)		30.31	(5.56)
Sex (female)	1,656	100	1,036	100	620	100	11,191	98.7	961	53.8	1,408	48.6
Race
American Indian or Alaska Native	67	4.1	48	4.6	19	3.1	217	1.9	6	.3	123	4.3
Asian	41	2.5	31	3.0	10	1.6	503	4.4	31	1.7	53	1.8
Black	152	9.2	86	8.3	66	10.7	1,657	14.4	476	26.7	201	6.9
Multiracial	80	4.8	49	4.7	31	5.0	438	3.8	91	5.1	238	8.2
Native Hawaiian or Other Pacific Islander	<5		<5		<5		26	.2	<5		<5
Other race, self-identified	41	2.5	29	2.8	12	1.9	259	2.3	12	.7	21	.7
White	1,165	70.4	714	68.9	451	72.7	7,487	65.2	1,033	58.0	2,172	75
Missing	107	6.5	78	7.5	29	4.7	896	7.8	135	7.6	84	2.9
Hispanic	313	18.9	207	20	106	17.1	1,658	14.4	252	14.1	343	11.8
Missing	<20		<10		<15		264	2.3	<5		<30
Caregiver educational attainment
<High School degree	83	5.0	50	4.8	33	5.3	613	5.3	136	7.6	106	3.7
High School degree, General Educational	198	12	123	12	75	12.1	1,248	10.9	253	14.2	249	8.6
Development certificate, or equivalent
Some college, no degree or associate’s degree or trade school	386	23.3	238	23	148	23.9	2,748	23.9	353	19.8	602	20.8
Bachelor’s degree	474	28.6	292	28.2	182	29.4	2,979	25.9	357	20.0	816	28.2
Master’s, professional, or doctorate degree	321	19.4	189	18.2	132	21.3	2,683	23.4	324	18.2	602	20.8
Missing	194	11.7	144	13.9	50	8.1	1,212	10.6	362	20.3	521	18.0
Income
<$30,000	298	18.0	194	18.7	104	16.8	1,808	15.8	378	21.2	408	14.1
$30,000-$49,999	140	8.5	86	8.3	54	8.7	1,076	9.4	174	9.8	236	8.2
$50,000–$74,999	197	11.9	129	12.5	68	11.0	1,049	9.1	153	8.6	286	9.9
$75,000-$99,999	231	14.0	156	15.1	75	12.1	956	8.3	153	8.6	267	9.2
$100,000-$199,999	300	18.1	201	19.4	99	16.0	2,069	18.0	372	20.8	549	19.0
$200,000 or more	93	5.6	59	5.7	34	5.5	1,095	9.5	173	9.7	233	8.1
Missing	397	24.0	211	20.4	186	30.0	3,430	29.9	382	21.4	917	31.7

Note. Race was self- or caregiver-reported.

Mokken Scaling Results

Mokken scaling resulted in moderately strong PTSSs for each of the four participant groups. Table 2 includes the AISP search algorithm results with all original items (Scale 1) and the final Mokken scales (Scale 2) with coefficients across the four groups. For each of the samples, the final scale consisted of nine of the 10 original items. The item, “felt (seemed) happy and satisfied with your (his/her) life” was removed based on psychometric justification (H_ij values were much lower than the rest of the items and never exceeded .31) and theoretical justification (the only positively valenced item capturing positive mood, which has been shown to be related but not the exact opposite of the DSM-5 acute stress disorder criteria of negative mood; Blackwell et al., 2022). Each participant subgroup analysis resulted in a nine-item reliable unidimensional scale with Cronbach’s α > .8 (range: .84–.87).

Table 2.

Mokken Scaling Analysis With (Scale 1) and Without (Scale 2) Item 1, by Subgroup Sample

	Pregnant/ postpartum		Adult		Adolescent		Child
Items	Scale 1	Scale 2	Scale 1	Scale 2	Scale 1	Scale 2	Scale 1	Scale 2
1. Happy and satisfied with lifea	.31	Did Not Scale	.3	Did Not Scale	.31	Did Not Scale	.31	Did Not Scale
2. Difficulty sleeping	.46	.44	.47	.43	.45	.44	.46	.48
3. Startled easily	.49	.47	.47	.43	DNS	.45	.45	.47
4. Angry outbursts	.47	.46	.45	.42	.44	.43	.45	.47
5. Sense of time slowing	.43	.43	DNS	.39	.46	.45	.44	.45
6. Spaced out or in a daze	.47	.46	.46	.47	.51	.50	.49	.50
7. Avoid thoughts and feelings about COVID-19	.4	.47	DNS	.46	DNS	.49	.53	.55
8. Avoid talking, reading, or watching information related to COVID-19	DNS	.4	DNS	.37	DNS	.42	.47	.49
9. Distressing dreams about COVID-19	.45	.46	.43	.43	.42	.46	.48	.50
10. Distressed when see reminder of COVID-19	.43	.44	.42	.45	.42	.48	.49	.51
H values	.46	.45	.43	.43	.44	.46	.46	.49
HT values of final scales		.41		.35		.36		.21
α	.87	.85	.81	.84	.81	.86	.87	.88

Note. DNS = Did Not Scale; bold values indicate values associated with the final COVID-19-related stress scale for each subgroup; pregnant/postpartum (n = 1,656) and adult (n = 11,483): ≥18 years; adolescent (n = 1,785): 13–21 years; child (n = 2,896): 3–12 years; items used a 5-point Likert response scale: (1) not at all, (2) rarely, (3) sometimes, (4) often, (5) very often; Scale 1 included all 10 original items; Scale 2 included a reduced set of nine negatively valenced items and did not include the one positively valenced item “happy and satisfied with life.” Scale 2 represents the final model.

^aItem was reverse coded.

There were slight differences in H^T values across subgroups. For pregnant/postpartum and adult caregivers, H^T = .41 and .35, respectively, suggesting moderate and weak evidence of IIO; the adolescent and child samples both had H^T = .28, suggesting insufficient evidence of IIO. These results support the MHM (vs. DMM) for all four subsamples. Thus, within subsamples, pregnant/postpartum individuals, adult caregivers, adolescents, and children can be ordered on the latent trait according to their total scores. Item-level invariance investigations across levels of caregiver education did not reveal nonnegligible DIF (pseudo-R² between models ≤.01), suggesting no meaningful uniform or nonuniform DIF. See Supplemental Table 8, for item-level descriptive statistics. Overall, the reliability of PTSS scores was consistent across ages, respondents, and other population characteristics.

Adult caregivers reported the highest level of pandemic-related traumatic stress (M = 18.57, SD = 6.46; range: 9–45), followed by adolescents (M = 18.33, SD = 7.16; range: 9–45), pregnant/postpartum individuals (M = 17.4, SD = 6.38; range: 9–45), and children as reported by their caregivers (M = 14.15, SD = 5.62; range: 9–41). All subgroup distributions were slightly positively skewed, and parent-reported child scores had the highest skewness (1.24) and kurtosis (4.24); this was further reflected by 27.6% (n = 800) of 3–12-year-olds having the lowest possible score. Across all samples, only n = 14 individuals had the highest possible score (Figure 1).

Figure 1.

Boxplots of ECHO COVID-19 Pandemic-Related Traumatic Stress Scale Score Distribution, by Subgroup Sample

Note. Pregnant/postpartum (n = 1,656) and adult (n = 11,483): ≥18 years; adolescent (n = 1,785): 13–21 years; child (n = 2,896): 3–12 years. ECHO = Environmental influences on Child Health Outcomes; PTSS = Pandemic-Related Traumatic Stress Scale. See the online article for the color version of this figure.

Individuals within the same family may share COVID-19-related stressors and experiences. For the subsample of adult caregiver–child dyads (n = 2,573) and adult caregiver–adolescent dyads (n = 1,245), correlations of PTSS scores were r = .47 and r = .16, respectively. For the subsample of pregnant/postpartum caregiver–child dyads (n = 82), the correlation was r = .69. No correlation was computed for pregnant/postpartum caregiver–adolescent dyads due to small sample size (n < 10).

Known-Group Differences

One-way ANOVAs revealed significant differences in PTSS scores by region, where individuals in the South and Midwest reported lower traumatic stress (M = 13.5–18.1) and individuals in the Northeast and West reported higher levels (M = 14.3–19.3). Findings were consistent across the four samples (Supplemental Table 9). As hypothesized, postpartum individuals had significantly higher stress compared to those who were pregnant at the time of survey collection, M = 17.87, SD = 6.6 vs. M = 17.1, SD = 6.23; F(1) = 5.34, p = .02, η² < .01, as did adults of <5-year-olds (M = 19.01, SD = 6.66) compared to adults of 5–12-year-olds (M = 18.91, SD = 6.39) and adolescents, M = 17.88, SD = 6.31; F(2) = 20.66, p < .001, η² < .01. As hypothesized, female adult caregivers and female adolescents had higher stress compared to adult male caregivers, M = 18.62, SD = 6.46 vs. M = 15.4, SD = 5.68; F(1) = 37.62, p < .001, η² < .01, and male adolescents, M = 20.23, SD = 7.12 vs. M = 16.1, SD = 6.45; F(1) = 161.47, p < .001, η² = .08, respectively. Similar trends were found in the child sample, M = 14.36, SD = 5.86 vs. M = 13.95, SD = 5.37; F(1) = 3.77, p = .05, η² < .01. Finally, confirming hypotheses, pandemic-related traumatic stress increased with age in both the child, F(9) = 10.95, p < .001, η² = .03, and adolescent, F(8) = 6.32, p < .001, η² = .03, samples (Supplemental Figure 2).

Relationships With Other Measures

Results showed moderate positive correlations (r =.46–.58) with all perceived stress measures and with PROMIS Anxiety (r = .50–.62) and Depressive Symptoms (r = .30–.48), and moderate negative correlations (r = −.35–−.37) with life satisfaction. Correlations with the CBCL DSM-oriented scales were moderate for the child parent-report sample(r = .28–.35) but slightly lower for the adolescent self-report sample (r = .20–.23), likely due to different respondents across the two measures as prior work shows notable discrepancies between parent and youth report (Upton et al., 2008). Despite being weaker, these correlations were still in the hypothesized positive direction and align with findings on the other external measures for the adolescent sample. See Table 3, for correlations.

Table 3.

Pearson r Correlations Between the PTSS and Other Measures

Measures	Pregnant/postpartum	Adult	Adolescent	Child
Perceived stress
PSS	.46	.46	.49	—
PROMIS psychological stress	—	—	.58	.47
Depression
PROMIS depression	.47	.35	.48	.3
CBCL DSM depression	—	—	.23	Pre: .35; Sch: .28
Anxiety
PROMIS anxiety	.5	.44	.62	.54
CBCL DSM anxiety	—	—	.2	Pre: .35; Sch: .25
Life satisfaction	-.35	-.33	-.35	-.37

Note. — = not applicable; PTSS = Pandemic-Related Traumatic Stress Scale; PSS = Perceived Stress Scale; PROMIS = Patient Reported Outcome Measurement System; CBCL = Child Behavior Checklist; DSM = Diagnostic and Statistical Manual of Mental Disorders; Pre = preschool version; Sch = school-age version; pregnant/postpartum (n = 1,656) and adult (n = 11,483): ≥18 years; adolescent (n = 1,785): 13–21 years; child (n = 2,896): 3–12 years.

Discussion

The PTSS scores showed sufficient reliability and validity as measured in a large and geographically, economically, racially, and ethnically diverse U.S. sample. Mokken scaling confirmed unidimensionality of a nine-item version. Small-to-moderate correlations with other variables suggest that PTSS scores capture a related but unique construct other than depressive or anxiety symptoms, general stress, and life satisfaction. Given that many investigations of the psychological impact of the COVID-19 pandemic and related mitigation efforts have primarily focused on anxiety and depression as constructs of pandemic-related distress (e.g., Loades et al., 2020; Prati & Mancini 2021), the PTSS focus on acute stress symptoms provides important additional insight into the psychological impacts of the pandemic. Prior studies show that isolation or quarantine after exposure to a deadly disease is associated with a fourfold increase in risk for posttraumatic stress disorder (Brooks et al., 2020); thus, capturing COVID-19-related traumatic stress symptoms provides an important risk marker of psychopathology. Future work examining the predictive power of the tool above and beyond measures of general distress is warranted.

The PTSS includes items that align with the five symptom categories included in the DSM-5 acute stress disorder criteria. One item querying the negative mood domain did not produce good model fit. The lack of convergence with the other nine items may be a measurement issue related to the positive wording of this item, as all others were negatively valenced (Lindwall et al., 2012). Importantly, the item was conceptually related to the other nine items, suggesting that it does add information above and beyond the four DSM-5 domains covered in the summary score. To capture all five DSM-5 domains, we encourage users to administer all 10 items but consider scoring the one item separately from the other nine. A total sum score for the nine-item PTSS, as done here, can be created if complete data are available. Several alternate scores can also be computed. If individual-level data are missing on some items, we suggest computing an average score, but only for individuals who complete more than 50% of items (i.e., at least five of the nine items; Fairclough & Cella, 1996; Graham, 2009). See Margolis et al.(2021), for additional scoring options, including total symptoms and symptom categories scores.

The primary aim of the PTSS is to identify moderate and high levels of pandemic-related traumatic stress that may signal increased risk for future mental health problems, such as PTSD. Results suggest that the instrument aptly captured variability at higher scores and thus can distinguish between individuals with average versus moderate or severe pandemic-related traumatic stress. While future work is needed to evaluate potential clinical cutoff criteria, general recommendations for patient-reported outcomes like the PTSS are to consider one standard deviation above the mean as moderate and two standard deviations above the mean as severe (Cella et al., 2010). For the adult caregiver sample, for example, “moderate” equates to a score of 25.03 and “severe” equates to a score of 31.49. Conversely, the PTSS was not as good at differentiating among lower scores. For example, a floor effect was observed for the 3–12-year-old sample, where the bottom quartile of children had the lowest possible PTSS score. However, this is not necessarily a meaningful limitation, as individuals with scores below average suggest they are at low risk for developing mental health problems; therefore, understanding individual differences at low scores may not provide additional clinical utility.

Results from ANOVAs exploring regional and known-group differences analyses suggest the instrument differentiates groups as follows: individuals living in the Northeast or West, postpartum individuals, caregivers of younger children, females, older children, and older adolescents had higher levels of pandemic-related traumatic stress compared to individuals living in the Midwest or South, pregnant individuals, caregivers of older children and adolescents, males, younger children, and younger adolescents, respectively. Such findings may reflect differences in pandemic experiences. For example, eight of the top 10 states with the fewest COVID-19 containment strategies were in the South and Midwest (Leatherby & Harris, 2020), both of which had the lowest pandemic-related traumatic stress scores compared to the West and Northeast regions. Additionally, older youth often experienced longer periods of school closures compared to younger youth (Shapiro & Taylor, 2020) and adolescence is a time of peak mental illness onset (Rapee et al., 2019), both of which may explain why older youth experienced greater pandemic-related traumatic stress. However, the magnitude of effect sizes for many of the analyses was negligible (η² < .01) or “small” (η² = .01–.05; Cohen, 1988), such that while statistically significant, some of these differences may not be clinically meaningful. One exception was the “medium” effect size of sex in the adolescent age group, highlighting a vulnerability among adolescent females as compared to males. Similar sex differences have been found for adolescent depression and anxiety during the pandemic (Loades et al., 2020; Magson et al., 2021; Marques de Miranda et al., 2020).

Importantly, adolescents in general had the second-highest levels of pandemic-related stress (adults had the highest) and therefore may be particularly vulnerable to the impacts of the pandemic, especially female adolescents. Prior work suggests adolescents had moderate to severe PTSD symptoms, depression, anxiety, and suicidal ideation resulting from the COVID-19 pandemic (Murata et al., 2021), and COVID-19 pandemic-specific distress was reported as a critical contributor to exacerbating existing mental health problems for youth (Magson et al., 2021; Raviv et al., 2021). Targeted interventions specifically addressing COVID-19 pandemic-related traumatic stress in adolescent populations are needed, in addition to further work identifying which adolescents may be at higher risk for developing clinically significant sequelae of experiencing such stress. While the average stress level for 13–21-year-olds was 18.32 out of 45, adolescents also had the most variability in scores, including 12.7% (n = 227) scoring the minimum value. This variability suggests some adolescents had fewer stress reaction symptoms despite the pandemic, whereas others experienced high levels of pandemic-related traumatic stress. Future research to understand what promotes thriving and what exacerbates stress can help guide intervention development.

The psychometric characteristics of the PTSS suggest the scale can be used across childhood, adolescence, and adulthood, an improvement over the few existing pandemic-related stress measures that are only validated in adult samples. While individuals within the same family may experience similar COVID-19 stressors, prior work using the PTSS items finds only small to moderate correlations between caregivers and children (Blackwell et al., 2022). We replicate such findings here, with small-to-moderate correlations found between adult caregiver PTSS score and their reports on their 3–12-year-old’s pandemic-related traumatic stress or their adolescent’s self-reported pandemic-related traumatic stress.

Finally, the utility of the PTSS spans beyond the immediate COVID-19 pandemic context. Unlike previous measures, which were developed to capture traumatic stress reactions to a single inciting event, the PTSS was developed as a tool to evaluate potential traumatic stress reactions to ongoing macrolevel threats without defined time bounds, the long-term impact of which is still unknown. The instrument could therefore be adapted to evaluate reactions to other acute onset stressors with lengthy durations in the future.

Limitations and Future Directions

The PTSS provides researchers and clinicians with a brief, robust questionnaire to assess pandemic-related stress symptoms across the lifespan from childhood through adulthood. However, several study limitations are noted. First, despite the large, nationwide sample, our sample was not representative compared to the general U.S. population and included a higher proportion of White individuals, was higher educated, and had higher income. Of note, the adult caregiver sample was nearly all female. Similarly, regional results should be interpreted with caution given the sociodemographic makeup of the sample by region may not reflect the general U.S. population for that region. Additionally, we had moderate missingness for income and, for certain subsamples, caregiver educational attainment. However, our sample has geographic, economic, racial, and ethnic diversity, which is lacking in prior work (Ahorsu et al., 2022; Bridgland et al., 2021; Feng et al., 2020; Pedrozo-Pupo et al., 2020; Qiu et al., 2020; Tambling et al., 2021; Taylor et al., 2020). The few existing validated instruments were only tested with adult samples, and most were not validated with U.S. participants (Ahorsu et al., 2022; Bridgland et al., 2021; Feng et al., 2020; Kira et al., 2021; Pedrozo-Pupo et al., 2020; Qiu et al., 2020; Tambling et al., 2021; Taylor et al., 2020), which is important given the vast differences in country-level COVID-19 policies and experiences.

Additionally, not all ECHO cohorts administered measures of stress, anxiety, and depression used as comparators for known-group analyses. In some cases, these subsample sizes were relatively small (e.g., n = 56 pregnant/postpartum individuals with PROMIS Anxiety) and not necessarily representative of the larger sample. However, consistent results were found across the pregnant/postpartum, adult caregiver, adolescent, and child samples regardless of sample size or individual measure.

Having all five DSM-5 acute stress disorder domains covered in a single summary score was not feasible with the current items, as the single positively valenced item measuring happiness and satisfaction with life did not fit with the other nine negatively valenced items. Future research can investigate if changing the valence of this item enables a conceptually and psychometrically cohesive scale. Additionally, a second item measuring negative symptoms might help capture this dimension. Relatedly, we did not have clinical diagnoses or clinical measures of acute stress disorder, PTSD, or other mood disorders apart from questionnaire measures of anxiety and depression. While the intent of the instrument was not to be a clinical assessment, future work would benefit from evaluating its clinical validity by coadministering the scale with existing diagnostic tools and in clinically enriched samples.

Finally, the instrument did not measure symptom duration and instead asked participants how often they experienced each item “since becoming aware of the COVID-19 pandemic.” This framing was pertinent during the study period but may become less applicable as time passes. Asking participants to report over a 2- or 3-year period or longer—when their stress experience likely changed at various times—could result in unreliable data. Using a shorter time frame, such as the past month, may prove beneficial to capturing individual’s “current” stress experiences. This revised framing can also be useful for repeated data collection over time. While timing of assessment should not impact score reliability and validity, timing may impact PTSS scores depending on how close/far from becoming aware of the pandemic an individual is, and future studies can track PTSS scores over time to investigate this topic.

Conclusion

As short-term psychosocial impacts of the COVID-19 pandemic emerge in the literature, long-term consequences of living through this unprecedented and protracted time in history have yet to emerge. Building on prior work and integrating current findings, there is potential to inform the design of efforts to identify and ameliorate traumatic stress and associated impacts. Here, we developed and psychometrically validated the PTSS and presented a strategy to quantify COVID-19 pandemic-related traumatic stress in a brief, respondent-sensitive, and scientifically rigorous approach. The age-specific forms measure parallel constructs to enable what has been demonstrated here to be a clinically and psychometrically sound lifespan-coherent measurement.

Public Significance Statement.

The PTSS reliably quantifies traumatic stress in the context of the COVID-19 pandemic, enabling researchers and clinicians to differentiate general stress, depression, and anxiety from pandemic-related traumatic stress symptoms. It may help to identify individuals with higher levels of traumatic stress symptoms who may benefit from targeted interventions.