Reliability and validity of the Perceived Stress Scale-10 in Hispanic Americans with English or Spanish language preference

Abstract

This study examined the psychometric properties of the Perceived Stress Scale-10 (PSS-10) among 436 community-dwelling Hispanic Americans with English or Spanish language preference. Multigroup confirmatory factor analysis examined the factorial invariance of the PSS-10 across language groups. Results supported a two-factor model (negative, positive) with equivalent response patterns and item intercepts but different factor covariances across languages. Internal consistency reliability of the PSS-10 total and subscale scores was good in both language groups. Convergent validity was supported by expected relationships of PSS-10 scores to measures of anxiety and depression. These results support the use of the PSS-10 among Hispanic Americans.

Introduction

The 14-item self-report Perceived Stress Scale (PSS; Cohen et al., 1983) is widely used to assess the degree to which situations in one’s life are appraised as stressful (Cohen et al., 1983). As a global stress measure, the PSS items are general in nature rather than event-specific, and evaluate the extent to which individuals perceive their lives to be “unpredictable, uncontrollable, and overloading” (Cohen et al., 1983, p. 387). Respondents rate the frequency of their feelings and thoughts about life events and situations over the previous month using a five-point scale ranging from (0) Never to (4) Very Often. The PSS yields a total score that describes overall perceived stress. A shorter 10-item version of the PSS (PSS-10; Cohen and Williamson, 1988) was derived by removing the four items with the lowest factor loadings (Items 4, 5, 12, and 13) from the original scale (PSS). The PSS-10 was recommended by the scale developers for use in future research, as it demonstrated psychometric properties comparable to the original, 14-item version.

The psychometric properties of the PSS-10 were originally evaluated in a large national sample of 2,387 American adults. (Cohen and Williamson 1988) reported that scores on the PSS-10 demonstrated adequate internal consistency reliability (α = .78); moderate concurrent criterion validity with the amount of stress experienced during an average week (r = .39, p < .001) and the frequency of stressful life events within the past year (r = .32, p < .001); and adequate convergent validity as evidenced by expected negative associations with perceived health status (r = −.22, p < .001) and positive associations with psychosomatic symptoms (rs = .28 to .34, p < .001) and health service utilization (r = .22, p < .001). Since then, other studies have similarly reported that the PSS-10 has good internal consistency reliability (e.g., Barbosa-Leiker et al., 2013; Golden-Kreutz et al., 2004; Reis et al., 2010), and adequate convergent validity based on associations with measures of physical and mental health (e.g., Mitchell et al., 2008; Roberti et al., 2006; Wu and Amtmann, 2013).

The dimensionality of the PSS-10 has received considerable empirical attention, although the underlying structure of the measure remains controversial. Studies have consistently identified a two-factor structure with six negatively worded items (Items 1–3, 6, 9, 10) comprising the first factor and four positively worded items (Items 4, 5, 7, 8) comprising the second factor (e.g., Barbosa-Leiker et al., 2013; Cohen and Williamson, 1988; Golden-Kreutz et al., 2004; Reis et al., 2010; Roberti et al., 2006). Cohen and Williamson (1988) originally argued that the distinction between the two factors was irrelevant, as factor structure corresponded to item directionality. They conceptualized perceived stress, as measured by the PSS-10, as a single construct that can be evaluated by both negatively and positively worded items. Other researchers, however, have posited that the two-factor model reflects distinct negative and positive components of the stress experience (Barbosa-Leiker et al., 2013; Golden-Kreutz et al., 2004; Roberti et al., 2006). Specifically, the two factors represent the negative feelings associated with stress (“Stress”) and positive feelings counter to stress (“Counter Stress”), consistent with Folkman’s (1997) modified stress theory. Factor 1 (negatively worded items) has also been termed “Perceived Helplessness” (Roberti et al., 2006) and “Negative Stress” (Reis et al., 2010). Factor 2 (positively worded items) has been termed “Perceived Self-Efficacy” (Roberti et al., 2006) and “Positive Stress” (Reis et al., 2010).

Recently, two studies proposed a bifactor model of the PSS-10 comprised of a single underlying general factor of perceived stress and two domain-specific factors composed of negatively (Factor 1) and positively (Factor 2) worded items (Jovanović and Gavrilov-Jerković, 2015; Wu and Amtmann, 2013). In a bifactor model, all items load onto a general factor representing the target construct (perceived stress) intended to be assessed by a measure (PSS-10), as well as one of the domain-specific factors. The general factor accounts for the common variance among all observed indicators (PSS-10 items), and the domain-specific factors account for the unique variance of subsets of indicators above and beyond the variance explained by the general factor (Brown, 2013; Rios and Wells, 2014). Both studies exploring this structure reported that the bifactor model of the PSS-10 demonstrated a better fit than the two-factor or one-factor models in an American sample of 446 adults with multiple sclerosis (Wu and Amtmann, 2013), and in Serbian clinical (N = 157 psychiatric outpatients) and non-clinical (N = 458 university students and adults) samples (Jovanović and Gavrilov-Jerković, 2015). Wu and Amtmann (2013) also concluded that the PSS-10 was sufficiently unidimensional, as the general factor of perceived stress accounted for greater variance than either of the domain-specific factors, and thus supported the use of the PSS-10 total score. Conversely, Jovanović and Gavrilov-Jerković (2015) suggested that both the total and subscale scores of the PSS-10 could be used, although variance explained by the general or domain-specific factors was not evaluated or reported. Further examination of the scale’s dimensionality using a bifactor approach is needed to determine whether the PSS-10 best represents a single dimension (total score) or different dimensions (subscale scores) of perceived stress, or whether both total and subscale scores should be used.

The PSS, in its 14- and 10-item versions, has been widely used across diverse samples and translated into 30 languages, including Spanish (Cohen, 2014). Currently, there are two validated Spanish language versions of the PSS, one that was developed in Spain and validated in a sample of 440 Spanish adults (European Spanish PSS-14; Remor, 2006), and another that culturally adapted the European Spanish PSS-14 for use in Mexico by revising the wording of items to be more culturally appropriate for Mexican samples, and was validated in a sample of 365 Mexican college students (Mexican Spanish PSS-14; Ramírez and Hernández, 2007). Both Spanish language versions of the PSS-14 demonstrated adequate internal consistency reliability (European Spanish: α = .81; Mexican Spanish: α = .83) and convergent validity evidenced by correlations with scores on measures of anxiety (European Spanish: r = .64, p < .001), depression (Mexican Spanish: r = .55, p = .001), and emotional exhaustion (Mexican Spanish: r = .52, p = .001). Additionally, exploratory and confirmatory factor analyses of the Mexican Spanish PSS-14 confirmed the two-factor structure found in the original English version of the PSS-14 (Cohen and Williamson, 1988). Although both European Spanish (Remor, 2006) and Mexican Spanish (Ramírez and Hernández, 2007) versions of the PSS-14 have been used in studies of Hispanic Americans in the United States (e.g., Chavez-Korell and Torres, 2013; Detjen et al., 2007; Segrin and Badger, 2013), the psychometric properties of these measures have not been evaluated outside of Spain or Mexico. Furthermore, there are no known studies that have validated the English or Spanish versions of the PSS-10 for use in Hispanic Americans, and the measurement equivalence of the different language versions has not yet been established. Without demonstrating such measurement equivalence, it is unknown if observed differences in PSS-10 scores across language groups reflect true differences in stress, or differences in how these groups define, experience, and communicate stress (Corral and Landrine, 2010; Geisinger, 1994). Given the large and growing Hispanic American population in the United States, a valid and reliable measure of perceived stress is needed for this population.

The primary purpose of the present study was to conduct a psychometric evaluation of the English (Cohen and Williamson, 1988) and Spanish (“Dr. Cohen’s Scales,” 2015) versions of the PSS-10 for Hispanic Americans with different language preferences. First, factorial validity of the one-factor, two-factor, and bifactor models of the PSS-10 was tested to determine the best fitting model in Hispanic Americans with English or Spanish language preference. Next, the measurement and structural invariance of the best fitting model of the PSS-10 was tested between English and Spanish language preference subgroups. Based on previous factor analytic studies, a two-factor solution corresponding to item directionality was hypothesized to best fit the data for both language preference subgroups. Once the factor structure was established, the internal consistency reliability of the PSS-10 total and subscale scores was examined for the full sample and for both language preference subgroups separately. Lastly, convergent validity of the English and Spanish versions of the PSS-10 was evaluated by examining correlations with measures of depression and anxiety. For both language preference subgroups, the PSS-10 full scale and the negatively-worded-item-subscale were hypothesized to be strongly and positively correlated with measures of depression and anxiety, while the positively-worded-item-subscale was hypothesized to be moderately and negatively correlated with measures of depression and anxiety.

Methods

Participants

Participants were a community sample of 436 self-identified Hispanic American adults with either an English (n = 210) or Spanish (n = 226) language preference. Individuals were eligible for inclusion if they self-identified as Hispanic American, were 21 years of age or older, were residents of the United States, and were sufficiently literate in either English or Spanish to complete informed consent procedures and study questionnaires.

Measures

Perceived Stress Scale-10 (PSS-10; Cohen and Williamson, 1988; “Dr. Cohen’s Scales,” 2015).

As described above, the PSS-10 is a 10-item self-report measure of global perceived stress. A total score ranging from 0 to 40 is computed by reverse scoring the four positively worded items and then summing all the scale items. Higher scores indicate greater levels of perceived stress. Subscale scores were computed by summing the six negatively worded items (Items 1, 2, 3, 6, 9, and 10) for Factor 1 (“Negative”) and the four positively worded items (Items 4, 5, 7, and 8) for Factor 2 (“Positive”), with higher scores indicating greater negative distress/stress feelings and greater positive stress feelings and coping abilities, respectively.

Generalized Anxiety Disorder-7 (GAD-7; García-Campayo et al., 2010; Spitzer et al., 2006).

The GAD-7 is a self-report measure that assesses the presence and severity of anxiety-related symptoms, as outlined by the Diagnostic and Statistical Manual of Mental Disorders – Fourth Edition – Text Revision (DSM-IV-TR; American Psychiatric Association [APA], 2000) Respondents rate the occurrence of each anxiety symptom over the past two weeks on a four-point scale ranging from (0) Not at All to (3) Nearly Every Day. Total scores range from 0 to 21, with higher scores indicating greater severity of anxiety. The GAD-7 was recently validated for use among Hispanic Americans with an English or Spanish language preference (Mills et al., 2014). Internal consistency reliability was good for the total sample (α = .93) and for both language preference groups (English: α = .91; Spanish: α = .94).

Patient Health Questionnaire-9 (PHQ-9; Huang et al., 2006; Spitzer et al., 1999).

The PHQ-9 is a self-report measure of depression. The PHQ-9 items correspond to the diagnostic criteria for depression as outlined by the DSM-IV-TR (APA, 2000). Respondents rate the occurrence of nine depressive symptoms over the past two weeks on a four-point scale ranging from 0 (not at all) to 3 (nearly every day). Total scores range from 0 to 27, with higher scores indicating a greater endorsement of depressive symptoms. Both English and Spanish versions of the PHQ-9 have been validated for use in Hispanic Americans (Merz et al., 2011). In the current study, internal consistency reliability was good for the total sample (α = .90) and for both language preference groups (αs = .90).

Procedure

The present study used data that were collected from a community-based cross-sectional study examining the validity of English and Spanish language measures among Hispanic Americans. Prior to human subject enrollment, study protocols and materials were reviewed and approved by the sponsoring universities’ Institutional Review Boards. Participants were recruited from Southern California using diverse recruitment strategies, including flyer distribution, word of mouth, health fairs, and engagement with community and religious leaders. Eligible participants provided written informed consent and completed survey questionnaires in their preferred language of English or Spanish. Participants were given $75 for their time and effort.

Data analysis

To test the measurement and structural invariance across English and Spanish language preference groups, multigroup confirmatory factor analysis (CFA) was used to test formal hypotheses of model parameter invariance across groups. A series of increasingly restrictive, nested models were fit to the data, with each model imposing additional equality constraints and being tested against the less restrictive model to determine the level of invariance across groups. First, to determine the best fitting model, the one-factor, two-factor, and bifactor models of the PSS-10 was estimated and tested separately for the English and Spanish language preference groups. For the one-factor model, all 10 items of the PSS-10 were specified to load onto a single factor (perceived stress). For the two-factor model, six negatively worded items (Items 1, 2, 3, 6, 9, and 10) were specified to load onto the first factor labeled “Negative,” and four positively worded items (Items 4, 5, 7, and 8) were specified to load onto the second factor labeled “Positive.” For the bifactor model, all items were specified to load onto a single general factor of perceived stress and onto one of the domain-specific factors (“Negative” or “Positive”) indicated in the two-factor model. After determining the best fitting factor solution for both language preference groups, the configural invariance model, which is the least restrictive, tested whether the overall factor structure was equivalent across language preference groups by examining the pattern of free and fixed model parameters without any equality constraints imposed. Next, the metric invariance model constrained each item’s factor loading to equivalence across groups and examined whether the relations between items and factors are equivalent in both groups. This level of invariance tested whether each item on the PSS-10 loaded equivalently onto the same factor in both English and Spanish language preference groups. Metric invariance, also known as weak measurement invariance, indicates equal measurement units of the scale and implies that respondents in both groups interpret the items in the same way. Then, the scalar invariance model additionally constrained the item intercepts and tested whether items have the same intercept (item means) across language preference groups. Scalar invariance, or strong measurement invariance, indicates that item scores from both groups have equivalent measurement metric and the same scalar, and allows for direct comparisons of factor means across groups. A lack of scalar invariance suggests the presence of a systematic bias in the response patterns between groups. Lastly, the factor variance/covariance invariance model (structural invariance), the most restrictive model, included additional constraints on factor variances and covariances to determine whether the relationship between latent factors was equivalent across language preference groups. This level of invariance tested whether English and Spanish language preference groups use the same range of the continuum of PSS-10 scores and whether the association between factors is equivalent across language groups.

Overall model fit was evaluated using multiple fit indices, as recommended by Bentler (2007). Because preliminary analyses revealed evidence of multivariate non-normality for the observed measures, the Satorra-Bentler scaled χ² (S-Bχ2; Satorra and Bentler, 1994) was used, with a non-significant test value (p > .05) indicating acceptable model fit. Additionally, three descriptive fit indices were examined: 1) the comparative fit index (CFI; Bentler, 1990), 2) the root mean square error of approximation (RMSEA; Steiger, 1990), and 3) the standardized root mean residual (SRMR; Hu and Bentler, 1999). CFI values greater than .90 were indicative of acceptable model fit and values greater than .95 were indicative of good model fit. For the RMSEA and SRMR indices, values less than .08 indicated acceptable model fit and values less than .05 indicated good model fit. A model was deemed to adequately fit the data if at least two descriptive fit indices met criteria for acceptable model fit. For tests of invariance, the S-Bχ² difference test (ΔS-Bχ2; Satorra and Bentler, 2001) was used to statistically test the significance of differences between nested models, with a non-significant value (p > .05) indicating that the nested model fit the data as well as the comparison model. However, because the S-Bχ² difference test also is sensitive to sample size, changes in descriptive fit indices (ΔCFI, ΔRMSEA, ΔSRMR) were also examined. Based on Chen’s (2007) recommended criteria, ΔCFI ≤ −.010, ΔRMSEA ≥ .015, and ΔSRMR ≥ .010 signified a decrement in model fit, indicating a lack of invariance. The factor structure and invariance analyses were conducted using Mplus version 7.2 (Muthén and Muthén, 1998–2012), and missing data was handled via the maximum likelihood estimation with robust standard errors method, which uses all available data points.

Once the factor structure was determined, Cronbach’s alpha was used to evaluate the internal consistency reliability of total and subscale scores for the English and Spanish language versions of the PSS-10. Independent samples t-tests were used to compare PSS-10 total and subscale scores across language preference groups. Convergent validity (i.e., relationship between a measure and conceptually related constructs; Groth-Marnat, 2009) was evaluated via bivariate correlations between scores on the PSS-10 and scores on the GAD-7 and the PHQ-9. Based on past research findings, levels of perceived stress were hypothesized to be significantly and moderately positively correlated with measures of anxiety (GAD-7) and depression (PHQ-9). Correlations between Factor 1 (“Negative”) subscale scores and GAD-7 and PHQ-9 total scores were hypothesized to be stronger than those between Factor 2 (“Positive”) and GAD-7 and PHQ-9 scores, and in the same direction as the PSS-10 total scores. In addition, due to the similarity of item content between the PSS-10 (e.g., felt nervous and “stressed”) and GAD-7 (e.g., felt nervous or anxious), PSS-10 total scores and Factor 1 (“Negative”) subscale scores were hypothesized to correlate more strongly with scores on the GAD-7 than with scores on the PHQ-9. Reliability and convergent validity of the PSS-10 were evaluated using SPSS version 22.0 (IBM Corp, 2013).

Results

Descriptive analyses

Sample characteristics for the English and Spanish language preference groups can be found in Table 1. The average age of the total sample was 42.5 years old (SD = 14.07). There were minimal missing data on the PSS-10, GAD-7, and PHQ-9 (no item had more than 2% missing data), so pairwise deletion of missing data was used (e.g., means and standard deviations were computed based on all available data for each variable; correlations were based on each pair of variables with non-missing data). For the total sample, the mean PSS-10 total score was 15.14 (SD = 6.52, range: 0–33), Factor 1 (“Negative) score was 10.03 (SD = 5.12, range: 0–24), and Factor 2 (“Positive”) score was 10.86 (SD = 3.25, range: 0–16). Mean PSS-10 total and factor scores for both language groups are reported in Table 1. Mean PSS-10 total scores were not statistically different between language preference groups, t(421) = .30, p = .764. Mean English “Negative” scores did not significantly differ from mean Spanish “Negative” scores, t(426) = 1.73, p = .084. However, mean “Positive” scores were higher in the English language preference group, t(428) = 2.07, p = .039.

Table 1.

Sample characteristics.

	English (n = 210)		Spanish (n = 226)
Age, M (SD)	38.50 (13.74)		46.24 (13.37)
Gender, n (%)
Female	107 (51.0%)		112 (49.6%)
Male	103 (49.0%)		114 (50.4%)
Education, n (%)
Less than High School	13 (6.2%)		108 (47.8%)
High school / Trade school	39 (18.6%)		48 (21.2%)
Some college / Associates degree	81 (38.6%)		41 (18.1%)
Bachelor’s degree	57 (27.1%)		17 (7.5%)
Postgraduate	18 (8.6%)		7 (3.1%)
Don’t Know / Missing	2 (1.0%)		5 (2.2%)
Employment Status, n (%)
Employed	141 (67.1%)		105 (46.5%)
Not Employed for Wages
Unemployed	30 (14.3%)		42 (18.6%)
Homemaker	6 (2.9%)		30 (13.3%)
Student / Retired / Disabled	19 (9.0%)		30 (13.3%)
Social Security / SSI	4 (1.9%)		9 (4.0%)
Don’t Know / Missing	10 (4.8%)		10 (4.4%)
Country of Birth, n (%)
United States	131 (62.4%)		32 (14.2%)
Mexico	52 (24.8%)		134 (59.3%)
Other	9 (4.3%)		4 (1.8%)
Missing	18 (8.6%)		56 (24.8%)
Annual Household Income, n (%)
Less than $15,000	37 (17.6%)		77 (34.1%)
$15,000 to $34,999	44 (20.9%)		75 (33.2%)
$35,000 to $50,000	39 (18.6%)		29 (12.8%)
$50,000 to $75,000	41 (19.5%)		11 (4.9%)
More than $75,000	34 (16.2%)		9 (4.0%)
Don’t Know / Missing	15 (7.1%)		25 (11.1%)
Marital Status, n (%)
Married	95 (45.2%)		116 (51.3%)
Not Married	115 (54.8%)		109 (48.2%)
Missing	0 (0.0%)		1 (0.4%)
Children, n (%)
Yes	121 (57.6%)		144 (63.7%)
No	89 (42.4%)		81 (35.8%)
Missing	0 (0.0%)		1 (0.4%)
	Scale Range	M (SD)	Scale Range	M (SD)
PSS-10
Total Score	0–33	15.24 (6.62)	0–31	15.05 (6.44)
“Negative”	0–24	10.46 (4.88)	0–22	9.61 (5.31)
“Positive”	3–16	11.19 (2.85)	0–16	10.55 (3.57)
GAD-7	0–21	4.17 (4.64)	0–21	5.68 (5.74)
PHQ-9	0–23	4.50 (5.09)	0–26	4.87 (5.39)

SSI: social security income; M: mean; SD: standard deviation; PSS-10: Perceived Stress Scale-10; GAD-7: Generalized Anxiety Disorder-7; PHQ-9: Patient Health Questionnaire-9.

Single-group CFA models

Confirmatory factor analysis was used to examine one-factor, two-factor, and bifactor models of the PSS-10 for the English and Spanish language preference groups (see Supplementary Table 1 for the goodness-of-fit indices). The one-factor model did not fit well statistically (English: S-Bχ² [35, N = 210] = 211.18, p < .001; Spanish: S-Bχ² [35, N = 226] = 212.46, p < .001) or descriptively (English: CFI = .76, RMSEA = .16, SRMR = .11; Spanish: CFI = .73, RMSEA = .15, SRMR = .15) for either language preference group. The two-factor model, comprised of “Negative” and “Positive” factors, provided acceptable fit in both English (S-Bχ² [34, N = 210] = 83.19, p < .001; CFI = .93, RMSEA = .08, SRMR = .05) and Spanish (S-Bχ² [34, N = 226] = 48.35, p = .05; CFI = .98, RMSEA = .04, SRMR = .06) language preference groups. Lastly, the bifactor model, which included a single general factor (perceived stress) and two specific domain factors (“Negative” and “Positive”), provided good fit for the English language preference group (S-Bχ² [25, N = 210] = 35.74, p = .08; CFI = .99, RMSEA = .05, SRMR = .03). However, the bifactor model was unable to be estimated for the Spanish language preference group due to negative residual variance, indicating model misspecification, from one of the factors. Because only the two-factor solution demonstrated acceptable model fit for both language preference groups, tests of measurement and structural invariance were only conducted on the two-factor model. For the two-factor model in the full sample, standardized factor loadings were statistically significant and ranged from 0.17 to 0.81 for the “Negative” factor, and 0.60 to 0.80 for the “Positive” factor. Interfactor correlation was small (r = .25, p < .001), and this finding that the two PSS-10 factors were not highly associated provides additional evidence against the one-factor model and the use of a total score.

Multigroup CFA models

Fit indices for the configural, metric, scalar, and factor variance covariance invariance models across language for the two-factor model of the PSS-10 are presented in Table 2.

Table 2.

Goodness-of-fit indices for configural, metric, scalar, and factor variance covariance invariance models of the Perceived Stress Scale-10.

Model	S-Bχ2	df	p	CFIa	RMSEAb	SRMRb	ΔS-Bχ2	Δdf	Δp	ΔCFIc	ΔRMSEAd	ΔSRMRe
1. Configural	138.332	68	< .001	.947	.069	.055	---	---	---	---	---	---
2. Metric	148.420	76	< .001	.945	.066	.062	9.673	8	.289	.000	-.004	.005
3. Scalar	164.514	84	< .001	.939	.066	.064	16.150	8	.040	-.006	.000	.002
3. Factor Variance Covariance	191.925	87	< .001	.920	.074	.097	22.307	3	< .001	-.019	.008	.033

S-Bχ²: Satorra-Bentler χ²statistic; CFI: comparative fit index; RMSEA: root mean square error of approximation; SRMR: standardized root mean residual.

^aAcceptable fit > .90, Good fit > .95.

^bAcceptable fit < .08, Good fit < .05.

^cImproved fit ≤ .010.

^dImproved fit ≤ .015.

^eImproved fit ≤ .010.

Configural invariance.

Configural invariance was examined by fitting the two-factor model to the data for the English and Spanish language preference groups, without any equality constraints imposed. For both language groups, the baseline two-factor model fit adequately according to the CFI, RMSEA, and SRMR. Factor loadings from English and Spanish baseline models are included in Supplementary Table 2. All unstandardized factor loadings were statistically significant and in the same direction for both languages (λs = .81 –1.31, ps < .001), providing further support for the configural invariance of the two-factor model.

Metric invariance.

Metric invariance was examined by constraining all factor loadings to equivalence across English and Spanish language preference groups. The metric invariance model fit the data well (CFI value > .90, RMSEA and SRMR values < .08), indicating that factor loadings were invariant across language preference groups. When the metric invariance model was compared to the less restrictive configural invariance model, results demonstrated no statistical (p > .05) or descriptive (all Δ values < .01) differences in model fit. Thus, the metric invariance model was deemed a superior fit to the data, and weak measurement invariance was satisfied.

Scalar invariance.

Scalar invariance was examined by constraining factor loadings and item intercepts to equivalence across English and Spanish language preference groups. The scalar invariance model fit the data well (CFI value > .90, RMSEA and SRMR values < .08), indicating that both factor loadings and item intercepts were invariant across language groups. When this constrained model was compared to the less restrictive metric invariance model, no descriptive differences (all Δ values < .01) were noted in model fit. Thus, strong measurement invariance was met.

Factor variance/covariance invariance.

Factor variance/covariance invariance was examined by constraining factor loadings, item intercepts, factor variances, and factor covariances to equivalence across language preference groups. The factor variance/covariance invariance model demonstrated acceptable model fit according to the CFI and RMSEA. When this model was compared to the less restrictive scalar invariance model, descriptive criteria (ΔCFI = −.019, ΔRMSEA = .008, ΔSRMR = .033) indicated a decrement in model fit, indicating a lack of factor variance/covariance invariance. Thus, the scalar invariance model was considered the superior fit to the data.

Internal consistency reliability

Internal consistency reliability for the PSS-10 total scores was adequate for the full sample (α = .82), English language preference group (α = .87), and Spanish language preference group (α = .78). Alphas for Factor 1 (“Negative”; 6 items) scores were adequate for the full sample and both language groups (all αs = .89). Alphas for Factor 2 (“Positive”; 4 items) scores were also adequate for the full sample (α = .78), English language preference group (α = .77) and Spanish language preference group (α = .78).

Convergent validity

Correlations between scores on the PSS-10 and scores on the convergent validity measures are presented in Supplementary Table 3. As anticipated, PSS-10 total and subscale scores were significantly correlated with scores on measures of anxiety (GAD-7) and depression (PHQ-9), in the hypothesized directions, for the total sample and for the English and Spanish language preference groups separately, supporting convergent validity. Specifically, PSS-10 total and Factor 1 (“Negative”) scores correlated strongly and positively with anxiety (rs = .64 – .71) and depression (rs = .56 – .64), while PSS-10 Factor 2 (“Positive”) scores were less strongly (small to moderate correlations) and negatively correlated with anxiety (rs = −.25 – −.46) and depression (rs = −.24 – −.46). Comparisons of correlations between language preference groups indicated that the Spanish language preference group had smaller correlations between the “Positive” subscale and anxiety (z = −2.52, p = .01) and depression (z = −2.46, p = .01) than the English language preference group.

Discussion

In the present study, the one-factor, two-factor, and bifactor models of the PSS-10 were evaluated separately for the English and Spanish language preference groups. The one-factor model, which reflected Cohen and Williamson’s (1988) original conceptualization of a unidimensional measure of perceived stress, showed poor fit for both language preference groups. The two-factor model, in which factor structure corresponded to item directionality (Factor 1: negatively worded items; Factor 2: positively worded items), provided acceptable fit in both language preference groups. The bifactor model, reflecting a two-factor structure of negatively and positively worded items with a single underlying general factor of perceived stress, provided adequate fit for the English language preference group, but did not converge to the data for the Spanish language preference group. Consequently, the two-factor model was the only structure that provided acceptable fit in both English and Spanish language preference groups.

Using this two-factor model structure, multigroup CFA was conducted to test the measurement and structural invariance of the English and Spanish versions of the PSS-10. Results demonstrated configural, metric, and scalar invariance of the PSS-10 across language preference groups. Factor variance covariance invariance of the PSS-10 was not supported, indicating that the correlational relationship between the two latent factors was not equivalent across language preference groups. The lack of structural invariance, however, does not signify that the PSS-10 is inappropriate for use among English and Spanish language preference groups, as the critical prerequisite for cross-group comparisons is measurement invariance (metric and scalar invariance) (Cheung and Rensvold, 2002).

In addition to factorial validity, PSS-10 scores demonstrated good convergent validity. Scores on the PSS-10 were significantly correlated in the expected directions with scores on the GAD-7 and PHQ-9, such that greater levels of perceived stress (PSS-10 total scores) and negative stress feelings (PSS-10 “Negative” scores) were associated with greater reports of anxiety and depressive symptoms, while greater levels of positive stress feelings (PSS-10 “Positive” scores) were associated with less reports of anxiety and depressive symptoms, in both English and Spanish language preference groups. This is consistent with the convergent validity analyses from previous studies (e.g., Ramírez and Hernández, 2007; Remor, 2006; Roberti et al., 2006; Wu and Amtmann, 2013). These results provide further support for two distinct PSS-10 subscales that are differentially related to clinical syndromes, as “Negative” scores were more strongly related to GAD-7 and PHQ-9 than “Positive” scores. Interestingly, while “Positive” scores were significantly and negatively correlated with GAD-7 and PHQ-9 scores for both groups, as predicted, the strength of associations differed between language preference groups. Smaller correlations were found in the Spanish language preference group, indicating that perceived coping abilities and positive stress feelings (“Positive” subscale) were less associated with anxiety and depressive symptoms in Spanish-speaking participants than English-speaking participants. Despite increasing support for two distinct factors, most studies have used the PSS-10 as a unidimensional measure of perceived stress, arguing that the two factors reflect the format of the items rather than conceptual differences (e.g., Mitchell et al., 2008; Ramírez and Hernández, 2007; Reis et al., 2010), and used the PSS-10 total score rather than the two subscale scores. Additional research is needed to examine the convergent validity of the two PSS-10 factors and their relationships with theoretically related constructs (e.g. anxiety, depression, perceived health status), and to determine whether to use the subscale scores, total score, or both among English- and Spanish-speaking Hispanic Americans.

The results of this study should be interpreted within the context of study limitations. First, the study participants were predominantly Mexican American and recruited from one geographic region in Southern California, limiting the generalizability of findings to other Hispanic American subgroups. Therefore, the factorial validity and measurement invariance of the PSS-10 should be further examined in Hispanic Americans with differing characteristics from the current study so that findings can be generalized to the greater Hispanic American community. In addition, although this study found support for a distinct two-factor model in both language preference groups, the fit of the bifactor model could not be examined in the Spanish language preference group, precluding model comparison between the two-factor and bifactor models. This lack of model convergence may reflect issues with the model specification, specifically identification with one of the group factors, and/or that an underlying single factor of distress may not apply in this Spanish language preference sample. Given the superior fit of the bifactor model for the English language preference group, it may be possible that a unidimensional structure best supports the data for only English-speaking participants and two distinct factors (“Negative” and “Positive”) of perceived stress best explain the data for Spanish-speaking participants. Future research should examine the fit and appropriateness of the bifactor model of the PSS-10, compared to the two-factor model, in larger, more diverse English- and Spanish-speaking samples. Furthermore, consistent with prior exploration of PSS-10 convergent validity, this study used two common and psychometrically sound measures of depression and anxiety. However, there are other variables that have been used for convergent validity in prior studies that future studies should also evaluate (e.g., emotional exhaustion).

Despite these limitations, the present study is the first known study to evaluate the psychometric properties of the English and Spanish language versions of the PSS-10 in a sample of Hispanic American adults. The findings support the use of the PSS-10 as a reliable and valid measure of perceived stress in Hispanic Americans with either an English or Spanish language preference. Additionally, the study provides additional support for the PSS-10 two-factor model and use of the subscale scores.