Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Nov 1.
Published in final edited form as: J Pers Assess. 2015 May 15;97(6):616–625. doi: 10.1080/00223891.2015.1038388

Examining the Factor Structure and Structural Invariance of the PANAS across Children, Adolescents, and Young Adults

Nicholas P Allan 1, Christopher J Lonigan 1,2, Beth M Phillips 3
PMCID: PMC4609236  NIHMSID: NIHMS685084  PMID: 25975209

Abstract

It is unclear what factor structure best represents the Positive and Negative Affect Schedule (PANAS) from childhood to adulthood. The PANAS structure was examined in a sample of 555 children (M age = 11.66, SD = 1.24), 608 adolescents (M age = 15.45, SD = 1.09), and 553 young adults (M age = 18.75, SD = 1.00). A partially invariant model consisting of Positive Affect, Fear, and Distress factors best represented the PANAS across all age groups, indicating that the underlying constructs are the same across age but that the factors become increasingly interrelated with increasing age.

Positive affect (PA) and negative affect (NA) have been implicated as core underlying individual differences in typical and atypical development (e.g. Rothbart & Bates, 2006; Watson, 2005). The Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988) is a potentially useful means of assessing PA and NA; however, recent conflicting results from factor analytic studies of the PANAS indicate that questions concerning the underlying structure of this measure and the constructs it assesses have not been resolved (e.g., Ebesutani et al., 2011; Leue & Beauducel, 2011). One question is whether NA is unidimensional or multidimensional. Another question is whether the PANAS is structurally invariant across the developmental spectrum from childhood to young adulthood. This study addressed both of these questions using data from large community samples of children, adolescents, and young adults.

Development of the PANAS

The PANAS was created by Watson et al. (1988) to reflect their theory of affect. In this model, affect is composed of two higher-order dimensions, PA and NA. The PA dimension reflects pleasurable engagement with the environment as well as a person’s proclivity to experience nonspecific pleasurable mood states. The NA dimension reflects unpleasurable engagement with the environment as well as a person’s proclivity to experience nonspecific negative mood states. At the time of development, Watson and colleagues surmised that PA and NA were orthogonal traits, and the PANAS was intended to reflect this hypothesized structure (Watson et al., 1988). Items were selected that loaded on either PA or NA, with little to no cross-loading on the other dimension (Watson & Clark, 1997).

Factor Structure of the PANAS

Studies that have been conducted to explore the structure of the PANAS have reported limited support for an orthogonal factor structure. Instead, early factor-analytic studies reported that a two-factor oblique model of PA and NA fit best. For example, in an adult sample from the general population, Crawford and Henry (2004) compared orthogonal and oblique models of PA and NA using confirmatory factor analysis (CFA) and reported that the best-fitting model was an oblique model in which the PA and NA factors were correlated at −.30. Other researchers have reported similar, modest factor correlations in samples across different geographical regions, cultures, and population characteristics (i.e., college students, clinically referred, elderly) as well as across state and trait ratings of PA and NA (i.e., right now, past week, in general; Crocker, 1997; Joiner, Catanzaro, & Laurent, 1996; Terracciano, McCrae, & Costa, 2003). In contrast to these results, however, some studies have found support for orthogonal PA and NA factors in children (e.g., Lonigan, Hooe, David, & Kistner, 1999).

Beyond findings of non-orthogonal NA and PA dimensions, results from a number of studies support a multidimensional structure of NA that consists of separate fear and distress dimensions. Gaudreau, Sanchez, and Blondin (2006) compared two- (i.e., PA and NA factors) and three-factor (i.e, PA, Fear, and Distress factors) models using CFAs in two different samples, the first comprising 305 French-Canadian athletes ranging in age from 14 to 47 (M age = 19.43, SD = 5.00) and the second comprising 217 French-Canadian athletes ranging in age from 14 to 60 (M age = 22.56, SD = 10.50). They reported that the three-factor solution consisting of oblique PA, Fear, and Distress factors fit the data better than did the two-factor model consisting of correlated PA and NA factors. Others have reported similar three-factor solutions using exploratory factor analytic techniques (e.g., Mehrabian, 1997; Killgore, 2000). Whereas Crawford and Henry (2004) reported that an oblique two-factor solution with correlated errors fit the data best in comparison to a three-factor model in a sample of 1,003 adults (M age = 42.9 years, SD = 13.7), they did not allow similar correlated errors in their final three-factor solution. Therefore, a direct comparison to their preferred two-factor model could not be made. However, fit indices provided by Crawford and Henry did indicate that the three-factor model without correlated errors fit better than the two-factor model without correlated errors.

The underlying structure of the PANAS may be more complex than previous correlated-traits models have tested. An assumption of correlated-traits (and hierarchical-factor) models is that there is a general factor that accounts for several related but dissociable factors, which in turn account for the underlying item clusters. In contrast, bifactor models operate under the assumption that there is a general factor accounting for common variance among all items as well as factors that can be explained by common variance in item clusters, independent of the general factor (e.g., Chen, Hayes, Carver, Laurenceau, & Zhang, 2012; Reise, 2012). Recently, two different bifactor models of the PANAS were proposed and tested (e.g., Ebesutani et al., 2011; Leue & Beauducel, 2011). Ebesutani et al., (2011) examined the factor structure of a 27-item measure modified from the PANAS to be more appropriate for youth, the Positive and Negative Affect Schedule for Children (PANAS-C; Laurent et al., 1999) in a mixed sample of 6- to 17-year-old children who were either clinic-referred (N = 662) or school-based (N = 911). CFA was used to compare several models, from which Ebesutani et al. determined that the bifactor model fit better than a unidimensional model and a model consisting of correlated PA and NA factors. In that model, NA was hierarchically structured to include lower-order Fear and Distress factors. Leue and Beauducel (2011) examined the factor structure of the German PANAS (Krohne, Egloff, Kohlmann, & Tausch, 1996) in two studies, the first comprising 354 German adults (M age = 32.30 years, SD = 11.66), the second comprising a community sample of 266 individuals as well as a sample of 98 male sex offenders (M age = 29.14 years, SD = 11.39). In the first study, competing models included (a) an orthogonal two-factor (PA and NA) model, (b) an oblique two-factor model, (c) a three-factor model (i.e., Correlated Fear and Distress factors and an uncorrelated PA factor), and (d) a bifactor model that consisted of a general factor labeled Affective Polarity, and uncorrelated PA and NA factors. The bifactor model offered significantly improved model fit as compared to the other models. In the second study, the bifactor model was the only model tested, and this model demonstrated adequate fit across both groups.

Although both Leue and Beauducel (2011) and Ebesutani et al. (2011) reported that a bifactor model best fit the PANAS or PANAS-C in their respective data, different theoretical assumptions guided the models tested within each study. Leue and Beauducel hypothesized that a general factor labeled Affective Polarity represented an individual’s general approach or withdrawal tendencies, independent of the level of affective salience in evocative stimuli. Therefore, they hypothesized that a general Affective Polarity factor would account for shared variance in PA and NA items assessing approach and withdrawal tendencies. In contrast, Ebesutani et al. posited that there were general PA and NA factors as well as completely dissociable Fear and Distress factors that accounted for variance independent of the general NA factor. Therefore, the models tested within these studies did not overlap, and no conclusion can be drawn as to the best-fitting bifactor PANAS model.

Developmental Structure of the PANAS

Studies establishing structural invariance of the PANAS across childhood are necessary to ensure that the same constructs are being assessed across the developmental spectrum, which allows for meaningful cross-age comparisons of changes in levels of affect as well as comparisons of relations with other developmentally important constructs such as psychopathology (Horn, McArdle, & Mason, 1983; Meredith, 1993). There are relatively few studies that have examined whether the PANAS exhibits a similar structure across development, and findings are mixed across studies (e.g., Bushman & Crowley, 2009; Ebesutani et al., 2011; Lonigan et al., 1999; Lonigan, Phillips, & Hooe, 2003). Lonigan et al. (1999) examined invariance of the PANAS in a sample of 152 9- to 11-year-old children (M age = 10.3 years, SD = .68) and 213 12- to 17-year-old children. Results indicated that a two-factor oblique model did not fit better than the orthogonal model for the older children but did fit better for the younger children. Lonigan et al. (1999) also examined structural invariance across age groups (using the two-factor orthogonal model to establish configural invariance) and reported that the factor structure did vary across age groups. In contrast, Lonigan et al. (2003) reported that the same factor structure fit equally well across a sample of 270 fourth-to eleventh-grade children (M age = 12.90 years, SD = 2.23) for which a median-split was used to divide children into younger and older groups. Bushman and Crowley (2009) examined structural invariance in the PANAS-C across a sample of 101 third-grade children (M age = 8.51 years, SD = .50) and a sample of 146 sixth-grade children (M age = 11.34 years, SD = .49). They reported that a model with factor loadings and factor intercorrelations invariant across groups resulted in significantly worse fit than a model with factor loadings and factor intercorrelations free to vary across groups. Although the reported results did not allow for determination of whether just holding factor intercorrelations to equality resulted in significantly worse fit, the correlation between the PA and NA factors was −.20 in the third-grade sample and −.13 in the sixth grade sample, suggesting that holding the factor intercorrelations to equality likely accounted for the misfit in the invariant model. From these results and findings showing age differences in other affect-related scales for which the correlation between anxiety and depression decreased with age (e.g., Ollendick, Seligman, Goza, Byrd, & Singh, 2003), it is possible that PA and NA differentiate as children mature. However, the equivocal findings and the relatively few studies that have examined structural invariance of the PANAS indicate that additional studies are needed to evaluate whether or not differentiation of NA and PA occurs with age.

The Current Study

Recent research has indicated that the factor structure of the PANAS may be more complex than previously proposed. Specifically, researchers have reported results that support a three-factor model, consisting of a PA factor and Fear and Distress factors as opposed to a single NA factor (e.g., Gaudreau et al., 2006). Others have reported results that support different bifactor models of the PANAS (e.g., Ebesutani et al., 2011; Leue & Beauducel, 2011). To date, however, there is no single study that has evaluated these different factor structures of the PANAS. The primary aim of the current study was to compare different one-factor, two-factor, three-factor, and bifactor models to determine the best-fitting model. Because results supporting the three-factor model comprising PA, Fear, and Distress factors (indicating that there are dissociable fear and distress components) as well as the bifactor model of PA, NA, and distinct Fear and Distress components have been reported, it was hypothesized that the best fitting model would be a bifactor model comprising general PA and NA factors and uncorrelated Fear and Distress factors.

The secondary aim of this study was to examine age-related changes in the structure of the PANAS. Results of extant studies are equivocal regarding structural invariance of the PANAS in children, adolescents, and young adults (e.g., Bushman & Crowley, 2009; Lonigan et al., 2003); however, across tested measurement models, several studies have provided initial evidence that the PANAS may be more dissociable across development (e.g., Bushman & Crowley, 2009; Ebesutani et al., 2011; Lonigan et al., 1999). Therefore, it was hypothesized that invariance for the PANAS would be partially supported, as only the factor correlations would vary such that factor correlations would be weaker in the older groups.

Method

Participants

Data for this study were collected from two samples, the first comprising 4th-through 12th-grade children and adolescents, and the second comprising college students. The first sample was split into a child sample (ages 9 to 13 years) and an adolescent sample (ages 14 to 17 years). For the children and adolescents, letters explaining the study and consent forms were sent home to the parents. This sample consisted of 555 children (M age = 11.66 years, SD = 1.24; 52.4% female) and 608 adolescents (M age = 15.45 years, SD = 1.09; 47.7% female). The child sample was 63.4% Caucasian, 21.4% African American, 7.2% Hispanic, 5.0% Asian, 1.1% Native American, and 1.8% bi-racial or other. The adolescent sample was 67.5% Caucasian, 20.9% African American, 6.8% Hispanic, 2.6% Asian, 0.7% Native American, and 1.5% biracial or other. For the young adult sample, data were collected from the psychology subject pool at a large public university in the southeastern United States. This sample consisted of 553 young adults (M age = 18.75 years, SD = 1.00; 60.0% female). The young adult sample was 73.7% Caucasian, 15.6% African American, 7.1% Hispanic, 2.4% Asian, and 1.3% bi-racial or other.

Measures

The Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). The PANAS was used to measure PA and NA across all age groups. The PANAS contains 20, one-word adjective items reflecting positive and negative affect. Participants rated the degree to which certain emotions were felt using a 5-point Likert-type scale, ranging from 1 (Very slightly or Not at all) to 5 (Extremely) using a general time frame to assess trait affectivity. Across PA and NA scales, the PANAS has demonstrated adequate reliability, in terms of internal consistency and test-retest, in child, adolescent, and adult samples (e.g., Anthony, Lonigan, Hooe, & Phillips, 2002; Lonigan et al., 2003; Watson & Clark, 1991). In the present study, adequate reliability was found for both the PA (α = .81) and the NA scale (α = .80). Reliability estimates were also calculated across the child, adolescent, and adult groups (PA αs = .78, .83, .83; NA αs = .74, .82, .83, respectively).

Procedure

For the child and adolescent groups, following parental consent and child assent for participation, children completed the PANAS and several other measures not relevant to the current study in large groups of 20 to 40 children. Assessments were conducted in a large multipurpose room or in a classroom at the child’s school. For the young adult group, following informed consent for participation, participants completed the PANAS and several other measures not relevant to the current study in large groups of 20 to 50 participants. Assessments were conducted in large classrooms at the participants’ university. For all groups, at least one graduate student and several trained undergraduate research assistants were present to check assessment packets for completion and answer general questions concerning assessment procedures. Individuals were not provided with specific instruction on items; if individuals asked about particular items, they were told to answer the item to the best of their abilities.

Data Analytic Plan

To determine the best fitting measurement model, a series of CFAs were conducted on the PANAS items using raw data within each sample (i.e., child, adolescent, young adult samples). Factors were scaled by fixing the factor variances to one as this method does not rely on selection of an item that does not vary too much or too little across groups.1 There is some disagreement about whether Likert-like scales such as the PANAS should be treated as continuous or categorical, with benefits and tradeoffs to either approach. A primary purpose of this study was to test the measurement invariance of a multidimensional measure. Schmitt and Kuljanin (2008) indicated that a big disadvantage of treating ordinal data as categorical in multidimensional models assessing measurement invariance is the lack of well-validated methods for testing these types of models. Stark, Chernyshenko, and Drasgow (2006) reported that treating polytomous categorical data as continuous is an effective approach for determining measurement invariance, with this approach performing similarly to categorical methods in all but the most extreme circumstances (i.e., dichotomous variables). In addition, on the basis of their findings in a recent simulation study, Rhemtulla, Brosseau-Liard, and Savalei (2012) recommended using continuous estimators when the measure contains five or more categories. Therefore, data in this study were analyzed as continuous using Full Information Maximum Likelihood (FIML) and the Satorra-Bentler scaled chi-square (S-B χ2) to correct for potential nonnormally distributed variables and to provide adjusted standard errors (Satorra & Bentler, 2001).

A series of nested, theoretically derived CFAs were fit within each age group to determine whether configural invariance was achieved. A two-factor orthogonal model consisting of PA and NA factors served as the baseline model. A two-factor oblique model consisting of PA and NA was then compared to the baseline model. Next, a three-factor model, consisting of PA, Fear, and Distress factors was compared to the best-fitting model between the orthogonal and oblique two-factor models. If similar fit was found between similar orthogonal and oblique models, the orthogonal model was considered best due to parsimony. In addition, the correlations between the PA factor and the Fear and Distress factors were either restricted to zero or allowed to be free depending on which of the initial models provided better fit. The same approach, in which the relations between factors were either restricted to zero or allowed to be free, was applied to all subsequent models. Bifactor models, based on the best-fitting correlated traits model were then examined. The correlated-factor models were nested within the bifactor models (see Chen, West, & Sousa, 2006). The first bifactor model contained a PA factor and a global NA factor, as well as Fear and Distress factors that were uncorrelated with the PA and NA factors. Again, the determination of a free or constrained (to zero) correlation between PA and NA was based on prior best-fitting models. The second bifactor model that was tested included a general Affective Polarity factor and uncorrelated PA, Fear, and Distress factors. The second bifactor model was not tested if PA and NA (or Fear and Distress) factors were best fit as orthogonal.

The best-fitting model was determined by examining model fit indices and factor loadings. Factor loadings were examined to ensure that meaningful factors emerged that accounted for nontrivial amounts of the variance in items. Fit indices were used to assess overall model fit, and the S-B χ2 was used to conduct likelihood ratio testing (LRT) to compare nested models (Satorra & Bentler, 2001). Using LRT, models were compared sequentially. If the LRT was significant, the model with more degrees of freedom was rejected as providing worse fit than the model with fewer degrees of freedom. Once the best-fitting model was selected using LRT, overall model fit was assessed using the S-B χ2, the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). A nonsignificant S-B χ2 is a generally accepted method of determining model fit. However, this method may be too stringent for practical research purposes because small model misfit or large sample sizes can lead to rejection of potentially useful models (see Hu & Bentler, 1999; Mulaik, 2007; Yuan, 2005, for discussion). In general, CFI values greater than .90 suggest adequate fit, and CFI values greater than .95 suggest good fit. RMSEA values of .08 suggest moderate fit; RMSEA values of .05 suggest good fit; and RMSEA values greater than .10 suggest poor fit. SRMR values close to .08 and below suggest adequate fit (Hu & Bentler, 1999; MacCallum, Browne, & Sugawara, 1996). Bifactor model loadings also were examined to determine whether there were additional meaningful factors (i.e., multiple items with significant factor loadings across multiple factors). For these analyses, factors were considered meaningful if three or more items loaded greater than .32 (i.e., accounting for more than 10% of the variance; Tabachnick & Fidell, 2007).

Following selection of the best-fitting model within group (i.e., child, adolescent, adult) by statistical and substantive criteria, measurement invariance and structural invariance were examined across the three age groups (i.e., child, adolescent, adult) according to procedures recommended by Meredith (1993). Because later steps require invariance in earlier steps, analyses of certain constraints were only conducted when prior constraints did not result in model misspecification. If configural invariance was achieved, an overall model was fit, with no restrictions across groups (i.e., factor loadings, intercepts, and factor covariances were free). For measurement invariance, equality constraints were imposed sequentially, from factor loadings to intercepts. Following this, structural invariance was examined. Factor variance equality was examined if factor loadings were invariant. Factor variance equality was only examined as a necessary step in examining whether factor covariance equality could be established. Factor covariance equality was examined to determine whether there were different relations between factors across groups. Finally equality of latent means was examined in the event that measurement invariance or partial measurement invariance was achieved (i.e., factor loadings and intercepts invariant). This was done to examine whether there were differences at the construct level across children, adolescents, and young adults. Finally, estimates of reliability and accompanying confidence intervals were calculated within a CFA framework (ρ; Raykov, 2001)

Results

Table 1 includes descriptive statistics for PANAS items across child, adolescent, and young adult samples. The top panel of Table 2 includes fit indices from CFAs for the different models for the sample of children. The best fitting model by sequential comparison, as well as by fit indices, was the bifactor model, consisting of orthogonal PA, NA, Fear, and Distress factors. Although this model provided adequate fit, the factors accounted for only limited variance in the items indicating that this model did not adequately capture the factor structure of the PANAS in children. The relatively large sample size allowed for small factor loadings to achieve significance. Using a more conservative estimate of loadings greater than .32 as substantive (i.e., accounting for more than 10% of the variance; Tabachnick & Fidell, 2007), only two items on the Fear factor and one item on the Distress factor had factor loadings greater than .32.

Table 1.

Descriptive Statistics for PANAS Items across Child, Adolescent, and Young Adult Samples

 PA Items Child (n = 555) Adolescent (n = 608) Young Adult (n = 553)
Mean SD Mean SD Mean SD
Active 4.12 1.08 3.96 .98 3.67 .97
Alert 3.18 1.27 3.45 1.03 3.44 .84
Attentive 3.13 1.19 3.34 .96 3.54 .81
Determined 3.33 1.34 3.71 1.10 3.82 .89
Enthusiastic 3.15 1.31 3.52 1.08 3.52 .92
Excited 3.48 1.22 3.39 1.05 3.35 .91
Inspired 3.00 1.23 3.28 1.09 3.21 1.00
Interested 3.34 1.20 3.39 1.05 3.58 .79
Proud 3.64 1.18 3.55 1.06 3.35 1.00
Strong 3.60 1.19 3.67 1.06 3.51 .95
 NA Items
Afraid 1.71 .97 1.76 .86 1.89 .89
Jittery 2.20 1.19 2.06 1.08 1.84 .97
Nervous 2.32 1.17 2.25 1.01 2.32 .98
Scared 1.70 .93 1.71 .85 1.76 .87
Distressed 2.00 1.11 2.09 1.04 2.19 .99
Guilty 1.79 1.03 1.84 .97 1.92 1.06
Hostile 1.92 1.03 2.01 1.07 1.80 .93
Irritable 2.22 1.09 2.51 1.10 2.45 .98
Upset 2.07 1.11 2.20 1.00 2.19 .90
Ashamed 1.61 .92 1.59 .88 1.53 .86
Open in a new tab

Note. PA = Positive affect. NA = Negative affect.

Table 2.

Fit Indices for Two-Factor, Three-Factor, and Bifactor PANAS Models in Child Sample

Child Sample S-B χ2 df Δ S-B χ2 CFI RMSEA SRMR
2-factor: Orthogonal PA and NA 511.47 170 -- .79 .06 .07
2-factor: Oblique PA and NA 512.21 169 .09 .79 .06 .07
3-factor: Orthogonal PA, Fear, and Distress 439.54 169 56.70*** .83 .05 .07
3-factor: Oblique PA, Fear, and Distress 439.05 167 .49 .83 .05 .07
Bifactor: PA, NA, Fear, Distress Orthogonal 395.47 160 40.10*** .86 .05 .06
3-factor: Orthogonal PA, Fear, and Distress (correlated residuals)a 416.09 168 26.87*** .85 .05 .06
Adolescent Sample
2-factor: Orthogonal PA and NA 391.44 170 -- .91 .05 .06
2-factor: Oblique PA and NA 384.11 169 6.74** .91 .05 .05
3-factor: Oblique PA, Fear, and Distress 349.98 167 26.76*** .92 .04 .05
Bifactor: PA, NA, Fear, Distress Orthogonal 292.06 159 41.31*** .94 .04 .05
Bifactor: Affective Polarity, Orthogonal PA, Fear, and Distress 258.23 151 40.36*** .95 .03 .04
3-factor: Oblique PA, Fear, and Distress (correlated residuals)b 277.24 165 58.47*** .95 .03 .05
Adult Sample
2-factor: Orthogonal PA and NA 391.44 170 -- .91 .05 .06
2-factor: Oblique PA and NA 384.11 169 6.74** .91 .05 .05
3-factor: Oblique PA, Fear, and Distress 349.98 167 26.76*** .92 .04 .05
Bifactor: PA, NA, Fear, Distress Orthogonal 292.06 159 41.31*** .94 .04 .05
Bifactor: Affective Polarity, Orthogonal PA, Fear, and Distress 258.23 151 40.36*** .95 .03 .04
3-factor: Oblique PA, Fear, and Distress (correlated residuals)c 277.24 165 58.47*** .95 .03 .05
Open in a new tab

Note. S-B χ2 = Satorra-Bentler scaled chi-square. CFI = Comparative fit index. RMSEA = Root mean square error of approximation. SRMR = Standardized root mean square residual. PA = Positive affect. NA = Negative affect. Nested models were compared sequentially and are listed in the order that they were compared.

a

This model was compared to the baseline three-factor orthogonal model. Correlated residuals were allowed for the scared and afraid adjective pairs.

b

This model was compared to the baseline three-factor oblique model. Correlated residuals were allowed for the scared and afraid adjective pairs and the excited and enthused adjective pairs.

c

This model was compared to the baseline three-factor oblique model. Correlated residuals were allowed for the ashamed and guilty adjective pairs, the scared and afraid adjective pairs, the excited and enthused adjective pairs, and the attentive and alert adjective pairs.

***

p < .001,

**

p < .01,

*

p < .05.

The next best-fitting model was the three-factor model consisting of PA, Fear, and Distress factors, with PA orthogonal to the Fear and Distress factors and oblique relations between the Fear and Distress factors. The fit of this model was further improved by allowing correlated residuals for the scared and afraid items (see Figure 1 for model parameters). The fit of this model was mixed. Whereas the CFI (.85) was below the acceptable threshold, the RMSEA (.05) and SRMR (.06) were within acceptable range. In this model, Fear and Distress were correlated at .75 (p < .05).

Figure 1.

Figure 1

Open in a new tab

Standardized parameter estimates for the three-factor model of PA, Fear, and Distress in children, adolescents, and adults, respectively. PA = Positive Affect. Error terms and residual variances have been omitted for clarity.

In the adolescent sample the best-fitting model was the bifactor model comprising orthogonal Affective Polarity, PA, Fear, and Distress factors (see middle panel of Table 2). Although this model provided adequate fit, the factors accounted for only limited variance in the items, indicating that this model did not adequately capture the factor structure of the PANAS in adolescents. Only 10 of the 20 items that were supposed to load on the Affective Polarity factor had loadings greater than .32. All 10 items that loaded were NA-related items. Further, only one item on the Fear factor and two items on the Distress factor had loadings greater than .32. The next best-fitting model was the bifactor model with correlated PA and NA factors and orthogonal Fear and Distress factors. However, only two items on the Fear factor and one item on the Distress factor had factor loadings greater than .32, indicating that this model did not adequately capture the factor structure of the PANAS in adolescents.

Therefore, the best-fitting model by both substantive and statistical criteria was the three-factor model with oblique PA, Fear, and Distress factors. The fit of this model was further improved by allowing correlated residuals between the scared and afraid items and the excited and enthusiastic items (see Figure 1 for model parameters). The fit of this model was acceptable (CFI = .95, RMSEA = .03, SRMR = .05). In this model, the correlation between PA and Fear was nonsignificant (r = −.09). Distress was significantly correlated with PA (r = −.21) and Fear (r = .93).

In the adult sample the best-fitting model was the bifactor model comprising orthogonal Affective Polarity, PA, Fear, and Distress factors (see bottom panel of Table 2). Although this model provided adequate fit, the factors accounted for only limited variance in the items, indicating that this model did not adequately capture the factor structure of the PANAS in adults. Only 11 of the 20 items that were supposed to load on the Affective Polarity factor had loadings greater than .32. All 10 NA-related items and one PA-related item loaded greater than .32. Further, only two items on the Fear factor and three items on the Distress factor had loadings greater than .32. The next best-fitting model was the bifactor model with correlated PA and NA factors and orthogonal Fear and Distress factors. However, only two items had factor loadings greater than .32 on each of the Fear and Distress factors, indicating that this model did not adequately capture the factor structure of the PANAS in adults.

Therefore, the best-fitting model by both substantive and statistical criteria was the three-factor model with oblique PA, Fear, and Distress factors. The fit of this model was further improved by allowing correlated residuals between the scared and afraid items, the excited and enthusiastic items, the guilty and ashamed items, and the attentive and alert items (see Figure 1 for model parameters). The fit of this model was acceptable (CFI = .92, RMSEA = .05, SRMR = .05). In this model the correlation between PA and Fear was significant (r = −.26). Distress was significantly correlated with PA (r = −.46) and Fear (r = .75).

Measurement Invariance

Examination of the PANAS across child, adolescent, and adult samples for configural invariance revealed that the three-factor oblique model of PA, Fear, and Distress was the best-fitting model for the adolescent and adult sample, but that the three-factor model with no relation between PA and Fear and Distress did not fit worse than the oblique model in the child sample. However, even though the oblique model did not fit the data better than the orthogonal model in the child sample, measurement invariance was still examined across all three age groups because allowing free factor covariances across groups would account for these nonsignificant factor associations in the child group while allowing for tests of indicator and factor loading invariance across groups. Table 3 includes fit statistics reflecting models with the inclusion of different invariance constraints across groups.2

Table 3.

Measurement and Structural Invariance Model Fit Parameters of the Three-Factor PANAS Model across Adolescent and Adult Samples

Model Constraints Imposed S-B χ2 df Δ S-B χ2 CFI RMSEA SRMR
Measurement Invariance
Baseline 1044.03 489 -- .92 .05 .05
Loading Invariance 1112.02 529 68.50** .91 .04 .07
Loading Invariancep 1090.96 523 47.81 .92 .04 .07
Intercept Invariance 1500.93 557 478.60*** .86 .05 .08
 Structural Invariance
Factor Variance Variant 1069.28 517 21.06*** .92 .04 .06
Factor Covariance Invariant 1088.75 523 19.47*** .92 .04 .06
Open in a new tab

Note. S-B χ2 = Satorra-Bentler scaled chi-square. CFI = Comparative fit index. RMSEA = Root mean square error of approximation. SRMR = Standardized root mean square residual. Baseline = Configural invariance model.

p

Model was partially invariant, allowing for “strong” to load freely on the PA factor across age groups, “irritable” to load freely on the Distress factor across age groups, and “jittery” to load freely on the Fear factor across age groups. Best fitting model at each level of invariance is in bold.

***

p < .001,

**

p < .01.

There was a significant difference between the model with factor loadings constrained to equality and the model with factor loadings allowed to freely vary across groups. Although most factor loadings were similar across groups in the model allowing factor loadings to freely vary, several items had significantly different factor loadings across age group. These included “strong,” with a factor loading of .30 in children, .47 in adolescents, and .58 in young adults on the PA factor, “jittery,” with a factor loading of .36 in children, and factor loadings of .51 and .57 in adolescents and young adults, respectively, on the Fear factor, and “irritable,” with a factor loading of .36 in children, .48 in adolescents, and .62 in young adults on the Distress factor. Therefore, the loadings of “strong,” “irritable,” and “jittery” were allowed to load freely on their respective factor within each age group. This model produced adequate fit, and model fit was not significantly different between this model and the model allowing all factor loadings to vary freely across groups. Constraining intercepts to equality across groups resulted in a significant reduction in model fit, but there was no clear pattern of difference in intercept values across age groups. This invariance is likely due to the large sample size, as the differences in magnitude were relatively modest across age groups. Because loadings were largely invariant, factor variances and covariances could be examined (although because intercepts were not invariance, factor means were not examined). As per convention, factor variances had been held to equality across groups but factor covariances had been allowed to vary. Allowing factor variances to be free across groups, by scaling models by the first item and freeing factor variances resulted in improved model fit. Constraining factor covariances to equality across groups resulted in a significant reduction in model fit. Therefore, the final model allowed factor variances and covariances to be estimated freely across groups.

Factor correlations were examined across groups (see top panel of Table 4), and LRT was used to examine differences in factor covariances between groups (see bottom panel of Table 4). The correlation between the PA factor and the Fear and Distress factors was significantly stronger in young adults than it was in children. There were no other significant differences.

Table 4.

Factor Intercorrelations and Comparisons for Three-Factor Partially Invariant Model of the PANAS across Children, Adolescent, and Young Adults

Factor Correlations
Child Adolescent Adult
Fear Distress Fear Distress Fear Distress
Positive Affect .04 -.07 -.08 -.21*** -.25*** -.46***
Fear -- .75*** -- .92*** -- .76***
Factor Correlation Comparisons
Child vs. Adolescent Δ χ2 Child vs. Adult Δ S-B χ2 Adolescent vs. Adult S-B Δ χ2
Positive Affect with Fear .25 5.78* 2.98
Positive Affect with Distress 2.62 12.76*** 3.80
Fear with Distress 3.10 .00 1.73
Open in a new tab

Note. All tests were conducted with one degree of freedom.

***

p < .001,

*

p < .05.

Reliability Estimates

Reliability was calculated for each factor within children, adolescents, and young adults (Raykov, 2001). For children, ρ was .67 (95% confidence interval [CI; .64, .71]) for PA, .58 (95% CI [.51, .65]) for Fear, and .66 (95% CI [.60, .71]) for Distress. For adolescents, ρ was .76 (95% confidence interval [CI; .73, .79]) for PA, .68 (95% CI [.62, .74]) for Fear, and .70 (95% CI [.66, .75]) for Distress. Finally, for young adults, ρ was .81 (95% confidence interval [CI; .78, .83]) for PA, .70 (95% CI [.65, .75]) for Fear, and .73 (95% CI [.68, .77]) for Distress.

Discussion

In this study, the best fitting model of the PANAS consisted of three oblique factors--PA, Fear, and Distress in the adolescent and young adult samples and of oblique Fear and Distress factors orthogonal to a PA factor in children. The three-factor model demonstrated partial measurement invariance across samples of children, adolescents, and young adults. The relation between the PA factor and the Fear and Distress factors was stronger in the young adult sample than it was in the sample of children. This finding suggests that PA and NA may start out as unrelated affective systems that become increasingly related over time, possibly through an interaction with the environment, other developmental systems, or both (discussed below). Although the two bifactor models provided better statistical fit to the data, coherent, interpretable factors did not emerge in either model. The emergence of Fear and Distress factors overlaps with the emergence of hierarchical fear and distress components that have been hypothesized to explain covariance patterns among internalizing disorders.

The PANAS Comprises Three Factors, PA, Fear, and Distress

The finding that the best fitting model of the PANAS comprised three factors, PA, Fear, and Distress clarifies previous equivocal findings concerning the structure of the PANAS (e.g., Crawford & Henry, 2004; Gaudreau et al., 2006; Killgore, 2000). Past research, although providing some support for the three-factor model, was primarily exploratory (e.g., Killgore, 2000; Mehrabian, 1997) or flawed in that similar factor structures were not tested across hypothesized two- and three-factor models. Additional support for the three-factor structure was provided by the factor intercorrelations, which were highly consistent with past studies, in both direction and magnitude (e.g., Gaudreau et al., 2006; Killgore, 2000; Mehrabian, 1997). This study also extended the finding of a three-factor model to children and adolescents, as past studies had been conducted predominantly in adult samples (e.g., Gaudreau et al., 2006; Killgore, 2000; Mehrabian, 1997).

Similarities and Differences in the Structure of the PANAS across Development

The results of this study provided evidence that PA, fear, and distress were similar constructs in children, adolescents, and young adults. Examination of measurement invariance indicated that, whereas intercepts varied across age groups, all but three factor loadings were invariant across age groups. Past studies have not found consistent results regarding measurement invariance. However, some studies used median splits to examine age differences (e.g., Lonigan et al., 1999, 2003), meaning that the age groups were somewhat arbitrarily created. Other studies conflated measurement and structural invariance because they did not examine invariance in a stepwise manner (e.g., Bushman & Crowley, 2009)--an approach that does not allow for exact determination of the cause of differences across groups (e.g., factor loadings, factor covariances). Given that the present study included well-defined age groups, moderately large samples within and across age groups, and systematic tests of measurement and structural invariance, it can be concluded that the underlying constructs of PA, fear, and distress are similar across children, adolescents, and adults.

There were differences in the relations between the PANAS factors between the children and young adult samples. The Fear and Distress factors were orthogonal to the PA factor in the child sample. In contrast, PA was significantly more (negatively) related with Fear and Distress in the young adult sample than in the child sample. The increasing associations with increasing age between PA and the NA factors (i.e., Fear and Distress) may occur because of the developmental process in which individuals transition from emotion-based systems of self-regulation to more cognitive-based systems of self-regulation (e.g., Eisenberg & Spinrad, 2004). The individual differences in this affect-based system are highly overlapping with the positive and negative reactive systems that are core features of temperament. Another core feature of temperament is a cognitively based self-regulatory system that has a more protracted development in comparison to the positive and negative reactive systems (Carlson, Zelazo, & Faja, 2013; Rothbart & Bates, 2006). This self-regulatory system, commonly referred to as effortful control in children and constraint in adults, regulates the more reactive emotional responses that are the result of an individual’s level of PA, fear, and distress (Digman, 1990; Lonigan, Vasey, Phillips, & Hazen, 2004; Rothbart & Bates, 2006). Therefore, the relations PA shares with NA and its constituent dimensions may be moderated by their relations with effortful control.

Limitations of Bifactor Models in Explaining the Factor Structure of the PANAS

Although model fit statistics favored the bifactor models of the PANAS over the three-factor model, neither of the bifactor models was accepted as the best-fitting model. There were several reasons for this decision. In general, bifactor models are useful when a construct is believed to consist of a global dimension as well as several dimensions that are unrelated to the global dimension and to each other (Chen et al., 2012; Reise, 2012). Testing the bifactor model proposed by Leue and Beauducel (2011), which consisted of a global Affective Polarity factor and would have suggested bipolar PA and NA dimensions (e.g., Russell & Carroll, 1999), did not reveal a pattern of factor loadings consistent with the hypothesized bifactor model. All substantive factor loadings corresponded only to items that also measured the Fear and Distress factors. Further, the inclusion of the global factor resulted in few significant factor loadings on the Fear and Distress factors, suggesting that the variance accounting for the Fear and Distress factors was divided between the Fear, Distress, and Affective Polarity factors. Leue and Beauducel also produced inconsistent results across their two samples. More than half of the items (i.e., 11 in one sample and 12 in the other) had minimal variance accounted for by the Affective Polarity factor (i.e., under 10%). Further, only six of these items were shared across samples. Testing the bifactor model proposed by Ebesutani et al. (2011), which consisted of a global NA factor, also did not reveal a pattern of factor loadings consistent with their hypothesized model. Whereas all items loaded on the general NA factor, few items loaded on the Fear and Distress factors, calling into question the need to distinguish between NA, Fear, and Distress factors. In their study, Ebesutani et al. reported somewhat different results. Items loaded on both the general NA factor as well as the Fear and Distress factors, with the exception of ”jittery,” which did not load significantly on the Fear factor. One difference between studies was that Ebesutani et al. used the PANAS-C, which consists of 15 NA items. The difference in measures used to assess NA may have accounted for the discrepant findings. Regardless, the mixed findings across studies do not provide support for a bifactor model of the PANAS.

Potential Utility of PA, Fear, and Distress in Relation to Internalizing Disorders

The finding that NA can be divided into lower-order fear and distress components can inform conceptual models of psychopathology, particularly regarding internalizing disorders. Watson and colleagues (Watson, 2005) proposed a hierarchical model of emotional disorders. In their model, all internalizing disorders (i.e., Panic Disorder [PD], Generalized Anxiety Disorder [GAD], Major Depressive Disorder [MDD]) are clustered together with bipolar disorders as emotional disorders. At the second-order level, internalizing disorders are divided into fear and distress disorders. Finally, the individual disorders are at the lowest level. According to Mineka, Watson, and Clark (1998), there are unique and common features for each individual-level disorder. The common components can be divided into features common at the global level and common at the second-order level. PA and NA have demonstrated value in research linking temperament and psychopathology because they have been identified as risk factors. PA is specific to lower-order MDD and GAD (GAD; Watson, Gamez, & Simms, 2005).

Previous conceptual models linking affective traits to internalizing disorders typically classify NA as a non-specific risk factor underlying internalizing disorders generally (e.g., Sellbom, Ben-Porath, & Bagby, 2008). Empirically, NA has been implicated as a general risk factor for internalizing disorders (e.g., Weinstock & Whisman, 2006). NA also has been implicated primarily as a risk factor for disorders classified as fear disorders in some studies (Sellbom et al., 2008) and primarily as a risk factor for distress disorders in other studies (Watson et al., 2005). The findings of the current study may help to clarify the relation between NA and internalizing disorders. The emergence of distinct fear and distress components of NA echoes the differentiation in internalizing disorders in the hierarchical model of Watson and colleagues. Therefore, the lower-order Fear and Distress factors of the PANAS may be specific to the fear and distress dimensions of the hierarchical model of emotional disorders, and the higher-order NA factor may serve as a general risk factor for internalizing disorders generally.

Implications for Use of PANAS in Research Contexts

The current findings have several implications for future research. These findings suggest that summing the fear and distress dimensions into a composite score will depend on the research purposes. If one is interested in accounting for general NA in studies, then a simple sum score of PANAS NA will be adequate in children, adolescents, and adults. However, the PANAS can also be used in a more refined manner by dividing the NA dimension further into lower-order fear and distress dimensions. For example, examining the discriminant validity for the fear and distress dimensions might prove especially fruitful given that these dimensions varied in their relations with PA. Across all uses of the PANAS, researchers should be aware of the modestly acceptable model fit statistics and estimates of reliability as well as the lack of invariance for several items across the age ranges included in the present studies. Future studies attempting to refine the PANAS might focus on replacing items that were not invariant across the age groups with items that met this standard. This might result in a more psychometrically sound measure of PA and NA from childhood to adulthood.

Limitations

There were several limitations to this study. There were no additional variables available across child, adolescent, and young adults that could have been used to examine convergent and discriminant validity. Although this would have been ideal, some concern can be alleviated in that past studies have found consistent patterns of relations for PA, NA, and Fear and Distress factors with external constructs, especially internalizing disorders (e.g., Ebesutani et al., 2011; Lonigan et al., 2003; Sellbom et al., 2008). Similar to other researchers who have attempted to fit PANAS data using a CFA approach (e.g., Crawford & Henry, 2004; Bushman & Crowley, 2010), all the models demonstrated some degree of model misfit. Further, several residual error covariances that seemed empirically supported were included to improve model fit. Finally, whereas the purpose of this paper was to validate the factor structure of the PANAS more specifically, it would be useful to examine whether other methods of assessing temperament (i.e., observational report; Dyson, Olino, Durbin, Goldsmith, & Klein, 2012) replicate the pattern of findings in the current study.

Conclusions & Future Directions

This study demonstrated that the PANAS measures similar PA, Fear, and Distress factors in child, adolescent, and young adult samples. Furthermore, there are stronger associations of Fear and Distress with PA as individuals transition from childhood to adulthood. Future research should determine whether these developmental differences in the structure of affect also are present in the relations between internalizing disorders (e.g., Watson, 2005). It is likely that an examination of the relations between the Fear, Distress, and PA dimensions of affect with the fear- and distress-related internalizing disorders across development can help refine our understanding of internalizing disorders and identify age-associated markers of risk for the development of different disorders.

Acknowledgments

Funding

Preparation of this work was supported, in part, by grants from the Institute of Education Sciences (R305B04074, R305B090021) and the National Institute of Child Health and Human Development (HD052120). Views expressed herein are solely those of the authors and have not been reviewed or approved by the grantors.

Footnotes

1

Models were also conducted in which factors were scaled with the first item fixed to one and the results were not substantively different.

2

For consistency across models, residuals for the ashamed and guilty adjective pairs, the scared and afraid adjective pairs, the excited and enthused adjective pairs, and the attentive and alert adjective pairs were allowed to covary within groups.

References

  1. Anthony JL, Lonigan CJ, Hooe ES, Phillips BM. An affect-based, hierarchical model of temperament and its relations with internalizing symptomatology. Journal of Clinical Child and Adolescent Psychology. 2002;31:480–490. doi: 10.1207/S15374424JCCP3104_7. [DOI] [PubMed] [Google Scholar]
  2. Bushman BB, Crowley SL. Is the structure of affect similar for younger and older children? Cross-sectional differences in negative and positive affectivity. Journal of Psychoeducational Assessment. 2010;28:31–39. doi: 10.1177/0734282909337584. [DOI] [Google Scholar]
  3. Chen FF, Hayes A, Carver CS, Laurenceau JP, Zhang Z. Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality. 2012;80:219–251. doi: 10.1111/j.1467-6494.2011.00739.x. [DOI] [PubMed] [Google Scholar]
  4. Chen FF, West SG, Sousa KH. A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research. 2006;41:189–225. doi: 10.1207/s15327906mbr4102_5. [DOI] [PubMed] [Google Scholar]
  5. Crawford JR, Henry JD. The Positive and Negative Affect Schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology. 2004;43:245–265. doi: 10.1348/0144665031752934. [DOI] [PubMed] [Google Scholar]
  6. Crocker PR. A confirmatory factor analysis of the Positive Affect Negative Affect Schedule (PANAS) with a youth sport sample. Journal of Sport and Exercise Psychology. 1997;19:91–97. [Google Scholar]
  7. Dyson MW, Olino TM, Durbin CE, Goldsmith HH, Klein DN. The structure of temperament in preschoolers: A two-stage factor analytic approach. Emotion. 2012;12:44–57. doi: 10.1037/a0025023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Ebesutani C, Smith A, Bernstein A, Chorpita BF, Higa-McMillan C, Nakamura B. Psychological Assessment. 2011;23:679–691. doi: 10.1037/a0023234. [DOI] [PubMed] [Google Scholar]
  9. Eisenberg N, Spinrad TL. Emotion-related regulation: Sharpening the definition. Child Development. 2004;75:334–339. doi: 10.1111/j.1467-8624.2004.00674.x. [DOI] [PubMed] [Google Scholar]
  10. Gaudreau P, Sanchez X, Blondin JP. Positive and negative affective states in a performance-related setting. European Journal of Psychological Assessment. 2006;22:240–249. doi: 10.1027/1015-5759.22.4.240. [DOI] [Google Scholar]
  11. Horn JL, McArdle JJ, Mason R. When is invariance not invarient: A practical scientist’s look at the ethereal concept of factor invariance. Southern Psychologist. 1983;1:179–188. [Google Scholar]
  12. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
  13. Joiner TE, Catanzaro SJ, Laurent J. Tripartite structure of positive and negative affect, depression, and anxiety in child and adolescent psychiatric inpatients. Journal of Abnormal Psychology. 1996;105:401–409. doi: 10.1037//0021-843x.105.3.401. [DOI] [PubMed] [Google Scholar]
  14. Killgore WDS. Evidence for a third factor on the Positive and Negative Affect Schedule in a college student sample. Perceptual and Motor Skills. 2000;90:147–152. doi: 10.2466/pms.2000.90.1.147. [DOI] [PubMed] [Google Scholar]
  15. Krohne HW, Egloff B, Kohlmann CW, Tausch A. Investigations with a German version of the Positive and Negative Affect Schedule (PANAS) Diagnostica. 1996;42:139–156. [Google Scholar]
  16. Krueger RF. The structure of common mental disorders. Archives of General Psychiatry. 1999;56:921–926. doi: 10.1001/archpsyc.56.10.921. [DOI] [PubMed] [Google Scholar]
  17. Laurent J, Catanzaro SJ, Rudolph KD, Lambert S, Osborne L, Gathright T, Potter KI. A measure of positive and negative affect for children: Scale development and preliminary validation. Psychological Assessment. 1999;11:326–338. [Google Scholar]
  18. Leue A, Beauducel A. The PANAS structure revisited: On the validity of a bifactor model in community and forensic samples. Psychological Assessment. 2011;23:215–225. doi: 10.1037/a0021400. [DOI] [PubMed] [Google Scholar]
  19. Lonigan CJ, Hooe ES, David CF, Kistner JA. Positive and negative affectivity in children: Confirmatory factor analysis of a two-factor model and its relation to symptoms of anxiety and depression. Journal of Consulting and Clinical Psychology. 1999;67:374–386. doi: 10.1037/0022-006X.71.3.465. [DOI] [PubMed] [Google Scholar]
  20. Lonigan CJ, Phillips BM, Hooe ES. Relations of positive and negative affectivity to anxiety and depression in children: Evidence from a latent variable longitudinal study. Journal of Consulting and Clinical Psychology. 2003;71:465–481. doi: 10.1037/0022-006X.71.3.465. [DOI] [PubMed] [Google Scholar]
  21. Lonigan CJ, Vasey MW, Phillips BM, Hazen RA. Temperament, anxiety, and the processing of threat-relevant stimuli. Journal of Clinical Child and Adolescent Psychology. 2004;33:8–20. doi: 10.1207/S15374424JCCP3301_2. [DOI] [PubMed] [Google Scholar]
  22. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychological Methods. 1996;1:130–149. [Google Scholar]
  23. Mehrabian A. Comparison of the PAD and PANAS as models for describing emotions and for differentiating anxiety from depression. Journal of Psychopathology and Behavioral Assessment. 1997;19:331–357. [Google Scholar]
  24. Meredith W. Measurement invariance, factor analysis and factorial invariance. Psychometrika. 1993;58:525–543. [Google Scholar]
  25. Mineka S, Watson D, Clark LA. Comorbidity of anxiety and unipolar mood disorders. Annual Review of Psychology. 1998;49:377–412. doi: 10.1146/annurev.psych.49.1.377. [DOI] [PubMed] [Google Scholar]
  26. Mulaik S. There is a place for approximate fit in structural equation modelling. Personality and Individual Differences. 2007;42:883–891. doi: 10.1016/j.paid.2006.10.024. [DOI] [Google Scholar]
  27. Ollendick TH, Seligman LD, Goza AB, Byrd DA, Singh K. Anxiety and depression in children and adolescents: A factor-analytic examination of the tripartite model. Journal of Child and Family Studies. 2003;12:157–170. doi: 10.1023/A:1022806731527. [DOI] [Google Scholar]
  28. Raykov T. Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. British Journal of Mathematical and Statistical Psychology. 2001;54:315–323. doi: 10.1348/000711001159582. [DOI] [PubMed] [Google Scholar]
  29. Reise SP. The rediscovery of bifactor measurement models. Multivariate Behavioral Research. 2012;47:667–696. doi: 10.1080/00273171.2012.715555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Rhemtulla M, Brosseau-Liard PÉ, Savalei V. When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods. 2012;17:354–373. doi: 10.1037/a0029315. [DOI] [PubMed] [Google Scholar]
  31. Rothbart MK, Bates JE. Temperament. In: Damon W, Lerner R, Eisenberg N, editors. Handbook of child psychology: Vol. 3. Social, emotional, and personality development. 6. New York: Wiley; 2006. pp. 99–166. [Google Scholar]
  32. Russell JA, Carroll JM. On the bipolarity of positive and negative affect. Psychological Bulletin. 1999;125:3–30. doi: 10.1037/0033-2909.125.1.3. [DOI] [PubMed] [Google Scholar]
  33. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66:507–514. doi: 10.1007/BF02296192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Schmitt N, Kuljanin G. Measurement invariance: Review of practice and implications. Human Resource Management Review. 2008;18:210–222. doi: 10.1016/j.hrmr.2008.03.003. [DOI] [Google Scholar]
  35. Sellbom M, Ben-Porath YS, Bagby RM. On the hierarchical structure of mood and anxiety disorders: Confirmatory evidence and elaboration of a model of temperament markers. Journal of Abnormal Psychology. 2008;117:576–590. doi: 10.1037/a0012536. [DOI] [PubMed] [Google Scholar]
  36. Stark S, Chernyshenko OS, Drasgow F. Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology. 2006;91:1292–1306. doi: 10.1037/0021-9010.91.6.1292. [DOI] [PubMed] [Google Scholar]
  37. Tabachnick BG, Fidell LS. Using Multivariate Statistics. Pearson Education Inc; Boston, MA: 2007. [Google Scholar]
  38. Terracciano A, McCrae RR, Costa PT., Jr Factorial and construct validity of the Italian Positive and Negative Affect Schedule (PANAS) European Journal of Psychological Assessment. 2003;19:131–141. doi: 10.1027//1015-5759.19.2.131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Watson D. Rethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM-V. Journal of Abnormal Psychology. 2005;114:522–536. doi: 10.1037/0021-843X.114.4.522. [DOI] [PubMed] [Google Scholar]
  40. Watson D, Clark LA. Self-versus peer ratings of specific emotional traits: Evidence of convergent and discriminant validity. Journal of Personality and Social Psychology. 1991;60:927–940. doi: 10.1037/0022-3514.60.6.927. [DOI] [Google Scholar]
  41. Watson D, Clark LA. Extraversion and its positive emotional core. In: Hogan R, Johnson J, Briggs S, editors. Handbook of personality psychology. San Diego, CA: Academic Press; 1997. pp. 767–793. [Google Scholar]
  42. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988;54:1063–1070. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
  43. Watson D, Gamez W, Simms LJ. Basic dimensions of temperament and their relation to anxiety and depression: A symptom-based perspective. Journal of Research in Personality. 2005;39:46–66. doi: 10.1016/j.jrp.2004.09.006. [DOI] [Google Scholar]
  44. Watson D, Tellegen A. Toward a consensual structure of mood. Psychological Bulletin. 1985;98:219–235. doi: 10.1037/0033-2909.98.2.219. [DOI] [PubMed] [Google Scholar]
  45. Weinstock LM, Whisman MA. Neuroticism as a common feature of the depressive and anxiety disorders: A test of the revised integrative hierarchical model in a national sample. Journal of Abnormal Psychology. 2006;115:68–74. doi: 10.1037/0021-843X.115.1.68. [DOI] [PubMed] [Google Scholar]

RESOURCES